Jason Helfman

doc@experts-exchange.com

2009 Jason Helfman $FreeBSD$ &tm-attrib.freebsd; &tm-attrib.general; &tm-attrib.intel; &tm-attrib.amd; This article describes building an internal &fbus.ap;. The freebsd-update-server software is written by &a.cperciva;, Security Officer of &os;. If you thought it was fun to update your system against an Official Update Server, just wait until you have an updated system from your very own &fbus.ap;. Experienced users or administrators responsible for several machines, or environments, know the difficult demands and challenges of maintaining such an infrastructure. Running a &fbus.ap; makes it easier to deploy security and software patches to selected test machines before rolling them out to production. It also means a number of systems can be updated from local network rather than a much slower Internet connection. This article outlines the steps involved in creating an internal &fbus.ap;. To build an internal &fbus.ap; some requirements should be met. A running &os; system. A user account with at least 4 GB of available space. This will allow for creation of updates for 7.1 and 7.2. Beyond this, new space requirements will need to be considered. An &man.ssh.1; account on a remote machine to upload distributed updates. An Apache web server, with over half of the space required for the build. For instance, our test builds total 4 GB, and the webserver space needed to distribute updates is 2.6 GB. Basic knowledge of shell scripting with Bourne shell, &man.sh.1;. Download freebsd-update-server software. A tarball may be downloaded, or use &man.csup.1; and the projects-all collection. Update scripts/build.conf appropriately. It is sourced during all build operations. Here is the default build.conf, which should be modified. # $FreeBSD: projects/freebsd-update-server/scripts/build.conf,v 1.1 2006/08/31 07:48:40 cperciva Exp $ # Main configuration file for FreeBSD Update builds. The # release-specific configuration data is lower down in # the scripts tree. # Location from which to fetch releases export FTP=ftp://ftp2.freebsd.org/pub/FreeBSD/releases # Host platform export HOSTPLATFORM=`uname -m` # Host name to use inside jails export BUILDHOSTNAME=${HOSTPLATFORM}-builder.daemonology.net # Location of SSH key export SSHKEY=/root/.ssh/id_dsa # SSH account into which files are uploaded MASTERACCT=builder@wadham.daemonology.net # Directory into which files are uploaded MASTERDIR=update-master.freebsd.org Parameters for consideration would be: FTP This is where the subroutine fetchiso() declared in scripts/build.subr will contact the configured source for downloading the &os; ISO. This can be configured to be an http address, as well. For our purposes, ISO's are on the same server as our internal http server that will be serving updates. The software has been configured to look in that location. For this setup, we have to alter the routine to fetch the ISO. By copying the source build.subr to scripts/RELEASE/ARCHITECTURE/build.subr this file will be sourced instead of the released source for build.subr. BUILDHOSTNAME Host where software will be built. Coincidentally, this information will be displayed on updated systems when issuing: &prompt.user; uname -v SSHKEY Key for uploading to update server where clients will fetch patches or upgrades. A key pair is created by executing ssh-keygen -t dsa. Altering this parameter is not necessary, as standard password authentication through ssh will suffice if configured properly. &man.ssh-keygen.1; has more detailed information in creating a key pair. MASTERACCT Account that files are uploaded to on remote system. MASTERDIR Directory where files are uploaded to on remote system. Now that build directives are set, the installation files are configured for a build. For this example, we will use RELEASE-7.2 under amd64 architecture. Configuration files for &i386; architecture are available with downloaded source. Create build environment directory under scripts/RELEASE-7.2/amd64. &prompt.user; mkdir -p /usr/local/freebsd-update-server/scripts/RELEASE-7.2/amd64 This is the build.conf file that should be placed in the directory just created. # SHA256 hash of RELEASE disc1.iso image. export RELH=1ea1f6f652d7c5f5eab7ef9f8edbed50cb664b08ed761850f95f48e86cc71ef5 # Components of the world, source, and kernels export WORLDPARTS="base catpages dict doc games info manpages proflibs lib32" export SOURCEPARTS="base bin contrib crypto etc games gnu include krb5 \ lib libexec release rescue sbin secure share sys tools \ ubin usbin cddl" export KERNELPARTS="generic" # EOL date export EOL=1275289200 To generate the "End of Life" number for build.conf, refer to the "Estimated EOL" posted on the &os; Security Website. Based on this date, you can issue date -j -f '%Y%m%d-%H%M%S' '20090401-000000' +%s, and substitute actual date parameters for those stated by &os;. The &man.sha256.1; hash key for the desired release, is published within the respective release announcement . The first step is to run scripts/make.sh. This will build some binaries, create directories, and generate an RSA signing key used for approving builds. In this step, a passphrase will have to be supplied for the final creation of the signing key. &prompt.root; sh scripts/make.sh cc -O2 -fno-strict-aliasing -pipe findstamps.c -o findstamps findstamps.c: In function 'usage': findstamps.c:45: warning: incompatible implicit declaration of built-in function 'exit' cc -O2 -fno-strict-aliasing -pipe unstamp.c -o unstamp install findstamps ../bin install unstamp ../bin rm -f findstamps unstamp Generating RSA private key, 4096 bit long modulus ................................................................................++ ...................++ e is 65537 (0x10001) Public key fingerprint: 27ef53e48dc869eea6c3136091cc6ab8589f967559824779e855d58a2294de9e Encrypting signing key for root enter aes-256-cbc encryption password: Verifying - enter aes-256-cbc encryption password: Note down the generated KeyPrint; this value is entered into /etc/freebsd-update.conf for binary updates. At this point, we are ready to stage a build. &prompt.root; cd /usr/local/freebsd-update-server &prompt.root; sh scripts/init.sh amd64 RELEASE-7.2 What follows is sample of an initial build run. &prompt.root; sh scripts/init.sh amd64 7.2-RELEASE Mon Aug 24 16:04:36 PDT 2009 Starting fetch for FreeBSD/amd64 7.2-RELEASE /usr/local/freebsd-update-server/work/7.2-RELE100% of 588 MB 359 kBps 00m00s Mon Aug 24 16:32:38 PDT 2009 Verifying disc1 hash for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 16:32:44 PDT 2009 Extracting components for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 16:34:05 PDT 2009 Constructing world+src image for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 16:35:57 PDT 2009 Extracting world+src for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 23:36:24 UTC 2009 Building world for FreeBSD/amd64 7.2-RELEASE Tue Aug 25 00:31:29 UTC 2009 Distributing world for FreeBSD/amd64 7.2-RELEASE Tue Aug 25 00:32:36 UTC 2009 Building and distributing kernels for FreeBSD/amd64 7.2-RELEASE Tue Aug 25 00:44:44 UTC 2009 Constructing world components for FreeBSD/amd64 7.2-RELEASE Tue Aug 25 00:44:56 UTC 2009 Distributing source for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 17:46:18 PDT 2009 Moving components into staging area for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 17:46:33 PDT 2009 Identifying extra documentation for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 17:47:13 PDT 2009 Extracting extra docs for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 17:47:18 PDT 2009 Indexing release for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 17:50:44 PDT 2009 Indexing world0 for FreeBSD/amd64 7.2-RELEASE Files built but not released: Files released but not built: Files which differ by more than contents: Files which differ between release and build: kernel|generic|/GENERIC/hptrr.ko kernel|generic|/GENERIC/kernel src|sys|/sys/conf/newvers.sh world|base|/boot/loader world|base|/boot/pxeboot world|base|/etc/mail/freebsd.cf world|base|/etc/mail/freebsd.submit.cf world|base|/etc/mail/sendmail.cf world|base|/etc/mail/submit.cf world|base|/lib/libcrypto.so.5 world|base|/usr/bin/ntpq world|base|/usr/lib/libalias.a world|base|/usr/lib/libalias_cuseeme.a world|base|/usr/lib/libalias_dummy.a world|base|/usr/lib/libalias_ftp.a ... ... Then the build of the world is performed again, with world patches. A more detailed explanation may be found in scripts/build.subr. Mon Aug 24 17:54:07 PDT 2009 Extracting world+src for FreeBSD/amd64 7.2-RELEASE Wed Sep 29 00:54:34 UTC 2010 Building world for FreeBSD/amd64 7.2-RELEASE Wed Sep 29 01:49:42 UTC 2010 Distributing world for FreeBSD/amd64 7.2-RELEASE Wed Sep 29 01:50:50 UTC 2010 Building and distributing kernels for FreeBSD/amd64 7.2-RELEASE Wed Sep 29 02:02:56 UTC 2010 Constructing world components for FreeBSD/amd64 7.2-RELEASE Wed Sep 29 02:03:08 UTC 2010 Distributing source for FreeBSD/amd64 7.2-RELEASE Tue Sep 28 19:04:31 PDT 2010 Moving components into staging area for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 19:04:46 PDT 2009 Extracting extra docs for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 19:04:51 PDT 2009 Indexing world1 for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 19:08:04 PDT 2009 Locating build stamps for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 19:10:19 PDT 2009 Cleaning staging area for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 19:10:19 PDT 2009 Preparing to copy files into staging area for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 19:10:20 PDT 2009 Copying data files into staging area for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 12:16:57 PDT 2009 Copying metadata files into staging area for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 12:16:59 PDT 2009 Constructing metadata index and tag for FreeBSD/amd64 7.2-RELEASE Files found which include build stamps: kernel|generic|/GENERIC/hptrr.ko kernel|generic|/GENERIC/kernel world|base|/boot/loader world|base|/boot/pxeboot world|base|/etc/mail/freebsd.cf world|base|/etc/mail/freebsd.submit.cf world|base|/etc/mail/sendmail.cf world|base|/etc/mail/submit.cf world|base|/lib/libcrypto.so.5 world|base|/usr/bin/ntpq world|base|/usr/include/osreldate.h world|base|/usr/lib/libalias.a world|base|/usr/lib/libalias_cuseeme.a world|base|/usr/lib/libalias_dummy.a world|base|/usr/lib/libalias_ftp.a ... ... And then the build completes... Values of build stamps, excluding library archive headers: v1.2 (Aug 25 2009 00:40:36) v1.2 (Aug 25 2009 00:38:22) @(#)FreeBSD 7.2-RELEASE #0: Tue Aug 25 00:38:29 UTC 2009 FreeBSD 7.2-RELEASE #0: Tue Aug 25 00:38:29 UTC 2009 root@server.myhost.com:/usr/obj/usr/src/sys/GENERIC 7.2-RELEASE Mon Aug 24 23:55:25 UTC 2009 Mon Aug 24 23:55:25 UTC 2009 ##### built by root@server.myhost.com on Tue Aug 25 00:16:15 UTC 2009 ##### built by root@server.myhost.com on Tue Aug 25 00:16:15 UTC 2009 ##### built by root@server.myhost.com on Tue Aug 25 00:16:15 UTC 2009 ##### built by root@server.myhost.com on Tue Aug 25 00:16:15 UTC 2009 Mon Aug 24 23:46:47 UTC 2009 ntpq 4.2.4p5-a Mon Aug 24 23:55:53 UTC 2009 (1) * Copyright (c) 1992-2009 The FreeBSD Project. Mon Aug 24 23:46:47 UTC 2009 Mon Aug 24 23:55:40 UTC 2009 Aug 25 2009 ntpd 4.2.4p5-a Mon Aug 24 23:55:52 UTC 2009 (1) ntpdate 4.2.4p5-a Mon Aug 24 23:55:53 UTC 2009 (1) ntpdc 4.2.4p5-a Mon Aug 24 23:55:53 UTC 2009 (1) Tue Aug 25 00:21:21 UTC 2009 Tue Aug 25 00:21:21 UTC 2009 Tue Aug 25 00:21:21 UTC 2009 Mon Aug 24 23:46:47 UTC 2009 FreeBSD/amd64 7.2-RELEASE initialization build complete. Please review the list of build stamps printed above to confirm that they look sensible, then run # sh -e approve.sh amd64 7.2-RELEASE to sign the release. Approve the build if everything looks ok. More information on determining if things are ok can be found in the distributed source file named USAGE. Execute scripts/approve.sh, as directed. This will sign the release, and move components into a staging area suitable for uploading. It is important to make sure that your key is mounted during this process. A simple df will show if it is mounted. If not mounted, mount the key with the passphrase supplied when creating it earlier. &prompt.root; cd /usr/local/freebsd-update-server &prompt.root; sh scripts/mountkey.sh &prompt.root; sh -e scripts/approve.sh amd64 7.2-RELEASE Wed Aug 26 12:50:06 PDT 2009 Signing build for FreeBSD/amd64 7.2-RELEASE Wed Aug 26 12:50:06 PDT 2009 Copying files to patch source directories for FreeBSD/amd64 7.2-RELEASE Wed Aug 26 12:50:06 PDT 2009 Copying files to upload staging area for FreeBSD/amd64 7.2-RELEASE Wed Aug 26 12:50:07 PDT 2009 Updating databases for FreeBSD/amd64 7.2-RELEASE Wed Aug 26 12:50:07 PDT 2009 Cleaning staging area for FreeBSD/amd64 7.2-RELEASE After completing the approval process, you may proceed with the upload. &prompt.root; cd /usr/local/freebsd-update-server &prompt.root; sh scripts/upload.sh amd64 RELEASE-7.2 The uploaded files will need to be in the DocumentRoot of the webserver in order for updates to be distributed. For further explanation, please refer to the Configuration section of the Apache documentation. Updates for the current release of the &os; system you are updating, and what you want to upgrade to need to be built in order for &os; Update Server to work properly. This is necessary for merging of files between releases. For example, if you are updating a system from &os; 7.1 to &os; 7.2, you will need to have update code built for &os; 7.1-RELEASE and &os; 7.2-RELEASE. Update client's KeyPrint and ServerName in /etc/freebsd-update.conf, and perform updates as instructed in the &os; Update instructions in the Handbook. For reference, here is the entire run of init.sh. In the event a security advisory is posted, a patch update can be built. For this example, we will be using 7.1-RELEASE. A couple of assumptions are made for a different release build: Setup the correct directory structure for the initial build. Perform an initial build for 7.1-RELEASE. Create patch directory under /usr/local/freebsd-update-server/patches/ for the respective release. &prompt.user; mkdir -p /usr/local/freebsd-update-server/patches/RELEASE-7.1/ As an example, take the patch for &man.named.8;. Read the advisory, and grab the necessary file from &os; Security Advisories. If you have trouble interpretting the advisory, please read this help page for more information. From the security brief, we can see it is called SA-09:12.bind. After downloading the file, it is required to rename the file to an appropriate patch level. It is suggested to keep this inline with official &os; patch levels, however, this is just a suggestion. For this build, let us follow the brief and call this p7. Rename the file: &prompt.user; cd /usr/local/freebsd-update-server/patches/RELEASE-7.1/; mv bind.patch 7-SA-09:12.bind or: &prompt.user; cd /usr/local/freebsd-update-server/patches/RELEASE-7.1/; fetch -o 7-SA-09:12.bind http://security.FreeBSD.org/patches/SA-09:12/bind.patch When running a patch level build, we are assuming that previous patches are in place. When a patch build is run, it will run all patches less than or equal to the number specified. Beyond this, you will have to take appropriate measures to verify authenticity of the patch. You can also add your own patches to any build. Use the number zero, or any other number. At this point, a diff is ready to be built. The software checks first to see if a scripts/init.sh has been run on the respective release prior to running the diff build. &prompt.root; cd /usr/local/freebsd-update-server &prompt.root; sh scripts/diff.sh amd64 RELEASE-7.1 7 What follows is sample of a differential build run. &prompt.root; sh -e scripts/diff.sh amd64 7.1-RELEASE 7 Wed Aug 26 10:09:59 PDT 2009 Extracting world+src for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 17:10:25 UTC 2009 Building world for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 18:05:11 UTC 2009 Distributing world for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 18:06:16 UTC 2009 Building and distributing kernels for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 18:17:50 UTC 2009 Constructing world components for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 18:18:02 UTC 2009 Distributing source for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 11:19:23 PDT 2009 Moving components into staging area for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 11:19:37 PDT 2009 Extracting extra docs for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 11:19:42 PDT 2009 Indexing world0 for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 11:23:02 PDT 2009 Extracting world+src for FreeBSD/amd64 7.1-RELEASE-p7 Thu Sep 30 18:23:29 UTC 2010 Building world for FreeBSD/amd64 7.1-RELEASE-p7 Thu Sep 30 19:18:15 UTC 2010 Distributing world for FreeBSD/amd64 7.1-RELEASE-p7 Thu Sep 30 19:19:18 UTC 2010 Building and distributing kernels for FreeBSD/amd64 7.1-RELEASE-p7 Thu Sep 30 19:30:52 UTC 2010 Constructing world components for FreeBSD/amd64 7.1-RELEASE-p7 Thu Sep 30 19:31:03 UTC 2010 Distributing source for FreeBSD/amd64 7.1-RELEASE-p7 Thu Sep 30 12:32:25 PDT 2010 Moving components into staging area for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 12:32:39 PDT 2009 Extracting extra docs for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 12:32:43 PDT 2009 Indexing world1 for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 12:35:54 PDT 2009 Locating build stamps for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 12:36:58 PDT 2009 Reverting changes due to build stamps for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 12:37:14 PDT 2009 Cleaning staging area for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 12:37:14 PDT 2009 Preparing to copy files into staging area for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 12:37:15 PDT 2009 Copying data files into staging area for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 12:43:23 PDT 2009 Copying metadata files into staging area for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 12:43:25 PDT 2009 Constructing metadata index and tag for FreeBSD/amd64 7.1-RELEASE-p7 ... ... Files found which include build stamps: kernel|generic|/GENERIC/hptrr.ko kernel|generic|/GENERIC/kernel world|base|/boot/loader world|base|/boot/pxeboot world|base|/etc/mail/freebsd.cf world|base|/etc/mail/freebsd.submit.cf world|base|/etc/mail/sendmail.cf world|base|/etc/mail/submit.cf world|base|/lib/libcrypto.so.5 world|base|/usr/bin/ntpq world|base|/usr/include/osreldate.h world|base|/usr/lib/libalias.a world|base|/usr/lib/libalias_cuseeme.a world|base|/usr/lib/libalias_dummy.a world|base|/usr/lib/libalias_ftp.a ... ... Values of build stamps, excluding library archive headers: v1.2 (Aug 26 2009 18:13:46) v1.2 (Aug 26 2009 18:11:44) @(#)FreeBSD 7.1-RELEASE-p7 #0: Wed Aug 26 18:11:50 UTC 2009 FreeBSD 7.1-RELEASE-p7 #0: Wed Aug 26 18:11:50 UTC 2009 root@server.myhost.com:/usr/obj/usr/src/sys/GENERIC 7.1-RELEASE-p7 Wed Aug 26 17:29:15 UTC 2009 Wed Aug 26 17:29:15 UTC 2009 ##### built by root@server.myhost.com on Wed Aug 26 17:49:58 UTC 2009 ##### built by root@server.myhost.com on Wed Aug 26 17:49:58 UTC 2009 ##### built by root@server.myhost.com on Wed Aug 26 17:49:58 UTC 2009 ##### built by root@server.myhost.com on Wed Aug 26 17:49:58 UTC 2009 Wed Aug 26 17:20:39 UTC 2009 ntpq 4.2.4p5-a Wed Aug 26 17:29:42 UTC 2009 (1) * Copyright (c) 1992-2009 The FreeBSD Project. Wed Aug 26 17:20:39 UTC 2009 Wed Aug 26 17:29:30 UTC 2009 Aug 26 2009 ntpd 4.2.4p5-a Wed Aug 26 17:29:41 UTC 2009 (1) ntpdate 4.2.4p5-a Wed Aug 26 17:29:42 UTC 2009 (1) ntpdc 4.2.4p5-a Wed Aug 26 17:29:42 UTC 2009 (1) Wed Aug 26 17:55:02 UTC 2009 Wed Aug 26 17:55:02 UTC 2009 Wed Aug 26 17:55:02 UTC 2009 Wed Aug 26 17:20:39 UTC 2009 ... ... Updates are printed, and approval is requested. New updates: kernel|generic|/GENERIC/kernel.symbols|f|0|0|0555|0|7c8dc176763f96ced0a57fc04e7c1b8d793f27e006dd13e0b499e1474ac47e10| kernel|generic|/GENERIC/kernel|f|0|0|0555|0|33197e8cf15bbbac263d17f39c153c9d489348c2c534f7ca1120a1183dec67b1| kernel|generic|/|d|0|0|0755|0|| src|base|/|d|0|0|0755|0|| src|bin|/|d|0|0|0755|0|| src|cddl|/|d|0|0|0755|0|| src|contrib|/contrib/bind9/bin/named/update.c|f|0|10000|0644|0|4d434abf0983df9bc47435670d307fa882ef4b348ed8ca90928d250f42ea0757| src|contrib|/contrib/bind9/lib/dns/openssldsa_link.c|f|0|10000|0644|0|c6805c39f3da2a06dd3f163f26c314a4692d4cd9a2d929c0acc88d736324f550| src|contrib|/contrib/bind9/lib/dns/opensslrsa_link.c|f|0|10000|0644|0|fa0f7417ee9da42cc8d0fd96ad24e7a34125e05b5ae075bd6e3238f1c022a712| ... ... FreeBSD/amd64 7.1-RELEASE update build complete. Please review the list of build stamps printed above and the list of updated files to confirm that they look sensible, then run # sh -e approve.sh amd64 7.1-RELEASE to sign the build. Follow the same process as noted before for appoving a build. &prompt.root; sh -e scripts/approve.sh amd64 7.1-RELEASE Wed Aug 26 12:50:06 PDT 2009 Signing build for FreeBSD/amd64 7.1-RELEASE Wed Aug 26 12:50:06 PDT 2009 Copying files to patch source directories for FreeBSD/amd64 7.1-RELEASE Wed Aug 26 12:50:06 PDT 2009 Copying files to upload staging area for FreeBSD/amd64 7.1-RELEASE Wed Aug 26 12:50:07 PDT 2009 Updating databases for FreeBSD/amd64 7.1-RELEASE Wed Aug 26 12:50:07 PDT 2009 Cleaning staging area for FreeBSD/amd64 7.1-RELEASE The FreeBSD/amd64 7.1-RELEASE update build has been signed and is ready to be uploaded. Remember to run # sh -e umountkey.sh to unmount the decrypted key once you have finished signing all the new builds. After approving the build, upload the software. &prompt.root; cd /usr/local/freebsd-update-server &prompt.root; sh scripts/upload.sh amd64 RELEASE-7.1 For reference, here is the entire run of diff.sh. If you build your own release using the native make release, freebsd-update-server code will work from your release. As an example, you may build a release without ports or documentation and add a custom kernel. After removing functionality pertaining to the documentation subroutine and altering the buildworld() subroutine in scripts/build.subr the freebsd-update-code will successfully build update code on this release. Add make -j NUMBER to scripts/build.subr to speed up processing. Adding flags to anything other than make buildworld and make obj may cause the build to become unreliable. Create a firewall rule to block outgoing RST packets. Due to a bug noted in this posting, you will have many time-outs and fail to update a system. Create an appropriate DNS SRV record for your update server, and put others behind it with variable weights. This effectively creates update mirrors. _http._tcp.update.myserver.com. IN SRV 0 2 80 host1.myserver.com. SRV 0 1 80 host2.myserver.com. SRV 0 0 80 host3.myserver.com. Please read the source documentation. It contains valuable information that will allow you to utilize all features of the software.