Quick and Dirty PXE Boot Setup
for FreeBSD 4.x and later

Doug White
November 4, 2005

Requirements

You'll need the following pieces to set up a boot server:

• Machine(s) to be the DHCP, TFTP, and NFS server for the boot
  • Two network interfaces are recommended but not required; this assumes you want to build a separate imaging network separate from your internal LAN. I strongly suggest doing this to avoid accidentally reimaging machines.
  • Disk space to hold the release(s) you want to netboot and install
  • an NFS-shareable volume to serve the /boot directories for each release and system config to support
• FreeBSD release CD or ISO image for each release to install
• Network hardware to connect the netboot clients and the server

Boot server setup

• Configure the second network interface as a private address space separate from your LAN.
• Turn on tftpd in inetd.conf and restart inetd. Copy /boot/pxeboot to /tftproot/pxeboot.
• Turn on NFS serving. Create an NFS share to host /boot directories for each release and system configuration. Set the -alldirs flag on this share.
  • If security isn't a huge concern and you have sufficient space on /, create a /pxeroot directory and export / with flags -alldirs -maproot=nobody.
  • Modify /etc/hosts.allow and allow everyone on the second network interface to access rpcbind and mountd.
  • This share can be exported read-only and restricted to the second network interface IP range.
• Create another NFS share that serves the FreeBSD release files.
  • Again, if you don't care that much and have the space, create /releases and put the releases in there by version (i.e., 5.4-RELEASE); the / -alldirs export will cover this.
    • example command: cp -R /cdrom/5.4-RELEASE /releases/
  • This share can be exported read-only and restricted to the second network interface IP range.
• For each release and system configuration you want to support, duplicate the boot/ directory from the release CD into /pxeroot/configname/boot.
  • Example command: cp -Rp /cdrom/boot /pxeroot/configname/
  • Create one config called "standard" which is used for vanilla systems, then as many more as you need for the systems and OS versions you want to install.
• Modify /pxeroot/configname/boot/loader.conf to look like this:
  • mfsroot_load="YES"
    mfsroot_type="mfs_root"
    mfsroot_name="/boot/mfsroot"
    vfs.root.mountfrom="ufs:/dev/md0c"
  • Optionally turn off the menu with beastie_disable="YES"; this keeps the menu from overwriting the boot parameters from DHCP.
• Modify /pxeroot/configname/boot/device.hints if necessary
  • For Intel/IBM blades, disable atkbd, atkbdc, and psm
• Configure dhcpd to start up on boot and only bind to the second network interface. See /usr/local/etc/rc.d/isc-dhcpd.sh for the rcng config options.
• Configure dhcpd as follows:
  • Specify the server as authoritative.
  • Create a subnet declaration for the second network interface.
  • In this subnet create a range of IP addresses to hand out. Since the network isn't routable don't configure a router or DNS options.
  • Also in this subnet, add these two lines (192.168.X.1 is the IP address of the second interface on the boot server):
    • filename "pxeboot";
    • option root-path "192.168.X.1:/pxeroot/standard";
  • For each system you want to install with a different configuration, add a block like this. somehostname can be anything but must be unique, and AA:BB:CC:DD:EE:FF is the MAC address of the machine in question.
    • host somehostname {
      hardware ethernet AA:BB:CC:DD:EE:FF;
      option root-path "192.168.X.1:/pxeroot/configname";
      }
  • Restart dhcpd after making changes.

At this point you should be able to netboot the client system and end up in sysinstall. (Watch the output from loader to verify it's using the right configuration's boot directory.) From there, proceed as normal but select NFS as the install source and enter "192.168.X.1:/releases/5.4-RELEASE" in the path box (or whatever you want to install).

Common Problems

• NFS doesn't work:
  • Make sure hosts.allow is set up to access to rpcbind and mountd.
  • Make sure rpcbind and mountd are running.
• The machine's PXE software doesn't DHCP:
  • Verify network connections and cabling.
  • Verify the configured MAC address. Which MAC address belongs to which physical plug can be ambiguous; the bezel label for "NIC 1" may not be the same "NIC 1" in the BIOS or OS when getting the MAC address. The MAC address is displayed when the DHCP requests are sent. Modern PXE firmware will complain if the link is down; this might help identify which interface is which. tcpdump -e may also help.
  • Verify dhcpd is running. Check the dhcpd log and see if the server is seeing the DHCPDISCOVER message from the client (you may need to set up dhcpd or syslogd to log this separately; the default log facility is local7 which is discarded by default). If the server is complaining about having no free addresses check the "range" configuration option in the subnet declaration.
• The machine boots the wrong configuration:
  • Check the configured MAC address. Which MAC address belongs to which physical plug can be ambiguous; the bezel label for "NIC 1" may not be the same "NIC 1" in the BIOS or OS when getting the MAC address.
  • The dhcpd log will show the MAC address it receives the DHCPDISCOVER from.

Variations

• The machine hosting the releases can be different from the netboot server, since you tell sysinstall where to get the release files from. Sysinstall itself is totally unaware it was netbooted, since its actually run from the mfsroot image, as with other install media. You could even use a local CDROM drive.
• If you're desperate, you can NFS-export the CDROM from the netboot server and point sysinstall at that. This will be slow depending on the speed of CDROM drive as the NFS and CDROM blocksizes aren't optimal, but it will work.
• The install can be fully automated using sysinstall's install.cfg facility. You'll need to mount the mfsroot image in the config directory you want to modify (/pxeroot/configname/boot/mfsroot.gz) and copy in the file. Instructions on how to do this is in the Handbook and the PXE install article on freebsd.org.