PXE Booting m0n0wall with an NFS Root Filesystem

Nik Clayton



FreeBSD is a registered trademark of Wind River Systems, Inc. This is expected to change soon.

Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this document, and the FreeBSD Project was aware of the trademark claim, the designations have been followed by the ``™'' or the ``®'' symbol.

This document explains how to PXE boot a m0n0wall installation using NFS as the root filesystem. This can be useful in a number of different circumstances, especially when developing and testing additions to m0n0wall.

Some of the information here duplicates the very useful m0n0wall Hackers Guide, and should really be folded in to an extended version of that document.

1 Hosts

There are at least two, and possibly four different hosts involved in this process. They are:

  1. The m0n0wall client. This is the host that will be running the m0n0wall software, and is typically a small embeddable device, such as a Soekris 4801.

  2. The DHCP server. This host runs a DHCP server, configured to respond to requests from the m0n0wall client with specific information.

  3. The PXE boot server. This host runs a tftpboot server, and contains the kernel and loader used to initially boot the m0n0wall client.

  4. The NFS server. This host runs the NFS server daemon, and contains the files that make up the root filesystem for the m0n0wall client.

The DHCP, PXE boot, and NFS server can be the same machine, or you may need to distribute these functions amongst multiple machines. For example, on my test network, I already have a low-powered host acting as a DHCP server. So I configured that with the additional DHCP information, then configured a second host to act as PXE and NFS server. With the m0n0wall client, this meant I was using three hosts in total.

2 The underlying process

  1. The m0n0wall client boots. Configured to use DHCP, it sends a DHCP request out to the network.

  2. The DHCP server receives the request, and replies with four pieces of information:

    1. The IP address that the m0n0wall client should use.

    2. The IP address of the server from which the m0n0wall client should download the boot loader.

    3. The name of the boot loader file that should be downloaded.

    4. The IP address and path of the NFS server and filesystem that should be used as the m0n0wall client's root filesystem.

  3. The m0n0wall client configures itself with the provided IP address, and contacts the server and downloads the boot loader. This download, and subsequent downloads during this process, are carried out using TFTP (trivial FTP).

    After downloading the boot loader, the operating system kernel is downloaded, and started.

  4. When the kernel has completed probing and initialising the hardware, it mounts the NFS filesystem as the root filesystem, and proceeds to execute /sbin/init and the startup scripts in the normal fashion.

3 Configuring the DHCP server

Install ports/net/isc-dhcp3 if you have not already done so (or an equivalent DHCP server), and add the following information to dhcpd.conf.

host soekris {
    hardware ethernet 0:0:24:c1:36:c8;
    filename "pxeboot";
    option root-path "";

This specifies:

  1. The ethernet address of the m0n0wall client. You need to obtain this information.

  2. The IP address that should be allocated to the m0n0wall client. Choose an appropriate IP address here.

  3. The IP address of the PXE boot server. This may be the same as the DHCP server, or it may be a separate host.

  4. The name of the file that the the client will request from the PXE boot server. This is normally pxeboot. If you use a name other than pxeboot you will need to change the filename in Section 4.2.

  5. The IP address of the NFS server, and the path on the NFS server from which the m0n0wall client will mount its root filesystem. In this example, the PXE boot server and the NFS server are the same machine (

Ensure that the DHCP server has been restarted after making these changes.

4 Configuring the TFTP/PXE boot server

The TFTP/PXE server must be configured with the boot loader and the kernel, and must run a TFTP server to provide these files to clients on request.

4.1 Create a loader and kernel directory

You must create a directory for the loader and kernel that will be used by the m0n0wall client. By convention, this is called /tftpboot, but you may use another name if you wish.

If you choose another name, adjust the following examples as necessary.

# mkdir /tftpboot

4.2 Prepare the boot loader

On the host that will be the PXE boot server, you need a copy of /usr/src/sys/boot tree, and must compile the pxeboot command with support for TFTP.

To do this;

# cd /usr/src/sys/boot
# make clean depend

After creating the loader, copy it to /tftpboot.

# cp /usr/obj/i386/pxeldr/pxeboot /tftpboot

4.3 Prepare the kernel

  1. Download the kernel config file for the m0n0wall client from http://www.neon1.net/m0n0wall/index.html.

  2. Add the following options to the kernel to support NFS booting.

    options        NFS
    options        NFS_DEBUG
    options        NFS_ROOT
    options        BOOTP
    options        BOOTP_NFSROOT
    options        BOOTP_COMPAT

    Important: You should also disable any firewall (IPFILTER) and IPSec (IPSEC) options in the kernel for the time being. These can be added later when you have verified that the system works correctly. For the time being they serve to complicate things, and make debugging difficult.

  3. Configure and build the kernel in the usual fashion.

    # config KERNEL
    # cd /usr/src/sys/compile/KERNEL
    # make clean depend && make all

    Do not install the kernel. The kernel should now be stripped to remove extraneous information that's not necessary on the low-end devices that m0n0wall typically runs on, and compressed.

    # strip kernel
    # strip --remove-section=.note --remove-section=.comment kernel
    # gzip -9f kernel
  4. Copy the compressed kernel to /tftpboot.

    # cp kernel.gz /tftpboot

4.4 Enable the TFTP server

Make the following change to /etc/inetd.conf to enable the TFTP server.

tftp   dgram  udp wait root /usr/libexec/tftpd  tftpd -s /tftpboot

This instructs the TFTP server to serve files from the directories under /tftpboot. inetd(8) must be restarted for this change to take effect.

# kill -HUP `cat /var/run/inetd.pid`

5 Configuring the NFS server

The NFS server must be configured to export a directory tree to clients, and that tree must be populated with files from m0n0wall.

5.1 Create the directory used as the m0n0wall NFS root filesystem

Decide where you want the m0n0wall client's root filesystem to be on the NFS server. This document uses /usr/local/m0n0wall. If you choose a different directory, adjust the examples that follow appropriately.

Note: This directory is also specified in the root-path directive of the DHCP server configuration (Section 3). You must ensure that they are the same.

# mkdir /usr/local/m0n0wall

5.2 Add this directory to the NFS exports list

/etc/exports lists the directories that the NFS server makes available to remote clients. Ensure that the directory you created in Section 5.1 is listed, and that the root account on the client is mapped to uid 0 on the NFS server. You may also add additional options to restrict which hosts can mount the directory. See exports(5) for full details of this file's format.

In this example, all hosts in the 192.168.1/24 network will be able to mount the /usr/local/m0n0wall directory using NFS.

/usr/local/m0n0wall -maproot=0 -network 192.168.1 -mask

5.3 Enable NFS server functionality

If the host that will be hosting the m0n0wall client's root filesystem is not already an NFS server you will need to enable this functionality. To do this, add the line:


to /etc/rc.conf and reboot.

Alternatively, you can start the NFS daemons manually, without the need to reboot.

# mountd -r
# nfsd

For more information about NFS on FreeBSD, see the NFS section of the Advanced Networking chapter of the FreeBSD Handbook.

5.4 Populate the root filesystem

/usr/local/m0n0wall must now be filled with files from the m0n0wall distribution.

  1. Download the most recent m0n0wall rootfs tarball from http://www.m0n0.ch/wall/. You should now have a .tar.gz file.

  2. Extract the files to /usr/local/m0n0wall.

    # cd /usr/local/m0n0wall
    # tar zxvf /path/to/rootfs-filename.tar.gz

5.5 Create the configuration directory

The m0n0wall configuration file is cf/conf/config.xml. That directory and file doesn't exist yet, so create it, and copy the default configuration file there.

# cd /usr/local/m0n0wall
# mkdir -p cf/conf
# cp conf.default/config.xml cf/conf

5.6 Fix the default IP address

After booting, m0n0wall will ignore the address it received from the DHCP server ( if you used the example in Section 3), and uses by default.

To change this to some other address, edit cf/conf/config.xml, find the entry for, and replace it with the IP address you prefer (e.g.,

6 Fixup NFS oddities

6.1 Create etc/fstab.nfs and symlink to etc/fstab

During the boot process, m0n0wall unmounts and remounts all the filesystems. To make this work, create the file etc/fstab.nsf within the m0n0wall filesystem tree, with these settings.

ipaddr:/usr/local/m0n0wall / nfs rw 1 1

Replace ipaddr with the IP address of the host that is acting as the NFS server.

Then symlink this to etc/fstab.

# cd /usr/local/m0n0wall/etc
# ln -sf fstab.nfs fstab

6.2 Provide sbin/mount_nfs

m0n0wall does not normally use NFS, so the supplied filesystem image does not include an sbin/mount_nfs binary. Copy the binary from your existing system to the sbin directory under the m0n0wall client root filesystem.

# cp /sbin/mount_nfs /usr/local/m0n0wall/sbin

This, and other documents, can be downloaded from ftp://ftp.FreeBSD.org/pub/FreeBSD/doc/.

For questions about FreeBSD, read the documentation before contacting <questions@FreeBSD.org>.
For questions about this documentation, e-mail <doc@FreeBSD.org>.