The codebaseMost of the package building magic occurs under the
- /var/portbuild directory. Unless
+ /var/portbuild/ directory. Unless
otherwise specified, all paths will be relative to
this location. ${arch} will
be used to specify one of the package architectures
@@ -167,7 +167,7 @@
interesting data (ports and src trees, bindist tarballs,
scripts, etc.) to disconnected nodes during the node-setup
phase. Then, the disconnected portbuild directory is
- nullfs-mounted for chroot builds.
+ nullfs-mounted for jail builds.
The
ports-${arch}
@@ -191,29 +191,31 @@
- Chroot Build Environment Setup
+ Jail Build Environment SetupPackage builds are performed in a
- chroot populated by the
+ jail populated by the
portbuild script using the
${arch}/${branch}/builds/${buildid}/bindist.tar
file.
- The following command builds a world from the
+ The makeworld command builds a world from the
${arch}/${branch}/builds/${buildid}/src/
tree and installs it into
- ${worlddir}. The tree will
+ ${arch}/${branch}/builds/${buildid}/bindist.tar.
+ The tree will
be updated first unless -nocvs is
- specified.
+ specified. It should be run as root:
- /var/portbuild&prompt.root; scripts/makeworld ${arch}${branch}${buildid} [-nocvs]
+ &prompt.root; cd /var/portbuild
+&prompt.root; scripts/makeworld ${arch}${branch}${buildid} [-nocvs]The bindist.tar tarball is created from the
previously installed world by the mkbindist
- script. It should be run as root with the following
- command:
+ script. It should be also be run as root:
- /var/portbuild&prompt.root; scripts/mkbindist ${arch}${branch}${buildid}
+ &prompt.root; cd /var/portbuild
+&prompt.root; scripts/mkbindist ${arch}${branch}${buildid}The per-machine tarballs are located in
${arch}/clients.
@@ -280,7 +282,7 @@
(For this case, the contents are identical for both server
and client.)
- RUBY_DEFAULT_VER= 1.9
+ RUBY_DEFAULT_VER= 1.9
@@ -291,8 +293,7 @@
(For this case, the contents are also identical for both
server and client.)
-
-.if !defined(CC) || ${CC} == "cc"
+ .if !defined(CC) || ${CC} == "cc"
CC=clang
.endif
.if !defined(CXX) || ${CXX} == "c++"
@@ -304,29 +305,29 @@
# Don't die on warnings
NO_WERROR=
WERROR=
-
+
Sample make.conf.server for
pkgng
- WITH_PKGNG=yes
-PKG_BIN=/usr/local/sbin/pkg
+ WITH_PKGNG=yes
+PKG_BIN=/usr/local/sbin/pkgSample make.conf.client for
pkgng
- WITH_PKGNG=yes
+ WITH_PKGNG=yesSample src.conf.server
to test new sort codebase
- WITH_BSD_SORT=yes
+ WITH_BSD_SORT=yes
@@ -416,7 +417,7 @@
dopackages.wrapper. These scripts
take a number of arguments. For example:
- dopackages.7 ${arch}${buildid}[-options]
+ &prompt.root; dopackages.7 ${arch}${buildid}[-options]
@@ -425,7 +426,7 @@
The symlinks go away, and you just use
dopackages.wrapper directly. For example:
- dopackages.wrapper ${arch}${branch}${buildid}[-options]
+ &prompt.root; dopackages.wrapper ${arch}${branch}${buildid}[-options]
@@ -628,7 +629,7 @@
-nocleanup, you need to clean up clients by running
- build cleanup ${arch}${branch}${buildid} -full
+ &prompt.root; build cleanup ${arch}${branch}${buildid} -fullerrors/,
logs/, packages/, and so
@@ -667,7 +668,7 @@
${arch}/${branch}/make.[0|1]:
- don't know how to make dns-all(continuing)
+ don't know how to make dns-all(continuing)To correct this problem, simply comment out or remove
the SUBDIR entries that point to empty
@@ -685,22 +686,22 @@
Update the i386-7 tree and do a complete build
- dopackages.7 i386 -nosrc -norestr -nofinish
- dopackages.wrapper i386 7 -nosrc -norestr -nofinish
+ &prompt.root; dopackages.7 i386 -nosrc -norestr -nofinish
+&prompt.root; dopackages.wrapper i386 7 -nosrc -norestr -nofinishRestart an interrupted amd64-8 build without updating
- dopackages.8 amd64 -nosrc -noports -norestr -continue -noindex -noduds -nofinish
- dopackages.wrapper amd64 8 -nosrc -noports -norestr -continue -noindex -noduds -nofinish
+ &prompt.root; dopackages.8 amd64 -nosrc -noports -norestr -continue -noindex -noduds -nofinish
+&prompt.root; dopackages.wrapper amd64 8 -nosrc -noports -norestr -continue -noindex -noduds -nofinishPost-process a completed sparc64-7 tree
- dopackages.7 sparc64 -finish
- dopackages.wrapper sparc64 7 -finish
+ &prompt.root; dopackages.7 sparc64 -finish
+&prompt.root; dopackages.wrapper sparc64 7 -finishHint: it is usually best to run the dopackages
@@ -767,7 +768,7 @@
package set. This can be accomplished with the following
invocation:
- path/qmanager/packagebuild amd647-exp20080904212103aclock-0.2.3_2.tbz
+ &prompt.root; path/qmanager/packagebuild amd647-exp20080904212103aclock-0.2.3_2.tbz
@@ -911,11 +912,12 @@
Cleaning up a BuildTo free up resources, you will need to clean up client machines by
- running build cleanup command. For example:
- &prompt.user; /var/portbuild/scripts/build cleanup i386 8-exp 20080714120411 -full
+ running build cleanup command. For example:
+ &prompt.root; /var/portbuild/scripts/build cleanup i386 8-exp 20080714120411 -full
+
If you forget to do this, then the old build
- chroots will not be cleaned up for 24 hours, and no
+ jails will not be cleaned up for 24 hours, and no
new jobs will be dispatched in their place since
pointyhat thinks the job slot is still occupied.
@@ -930,21 +932,22 @@
You may have problem with the umount
commands hanging. If so, you are going to have to use the
allgohans script to run an &man.ssh.1;
- command across all clients for that buildenv. For example:
-ssh -l root gohan24 df
+ command across all clients for that buildenv. For example:
- will get you a df, and
+ &prompt.user; ssh -l root gohan24 df
-allgohans "umount -f pointyhat.freebsd.org:/var/portbuild/i386/8-exp/ports"
+ will get you a df, and
+
+ &prompt.user; allgohans "umount -f pointyhat.freebsd.org:/var/portbuild/i386/8-exp/ports"
allgohans "umount -f pointyhat.freebsd.org:/var/portbuild/i386/8-exp/src"
- are supposed to get rid of the hanging mounts. You will have to
- keep doing them since there can be multiple mounts.
+ are supposed to get rid of the hanging mounts. You will have
+ to keep doing them since there can be multiple mounts.Ignore the following:
-umount: pointyhat.freebsd.org:/var/portbuild/i386/8-exp/ports: statfs: No such file or directory
+ umount: pointyhat.freebsd.org:/var/portbuild/i386/8-exp/ports: statfs: No such file or directory
umount: pointyhat.freebsd.org:/var/portbuild/i386/8-exp/ports: unknown file system
umount: Cleanup of /x/tmp/8-exp/chroot/53837/compat/linux/proc failed!
/x/tmp/8-exp/chroot/53837/compat/linux/proc: not a file system root directory
@@ -1006,8 +1009,8 @@
You can use qclient command to monitor the status
of build nodes, and to list the currently scheduled jobs:
- python path/qmanager/qclient jobs
- python path/qmanager/qclient status
+ &prompt.root; python path/qmanager/qclient jobs
+&prompt.root; python path/qmanager/qclient statusThe
scripts/stats ${branch}
@@ -1564,9 +1567,13 @@
Pick a directory to hold ports configuration and
- chroot subdirectories. It may be
+ chroot subdirectories. It may be
best to put it this on its own partition. (Example:
/usr2/.)
+
+ The filename chroot is a
+ a historical remnant.
+
@@ -1610,14 +1617,15 @@
GENERIC (or, if you are using more than
3.5G on &i386;, PAE).
-
- Required options:
-options NULLFS
-options TMPFS
-
-
- Suggested options:
-options GEOM_CONCAT
+
+ Required options:
+
+ options NULLFS
+options TMPFS
+
+ Suggested options:
+
+ options GEOM_CONCAT
options GEOM_STRIPE
options SHMMAXPGS=65536
options SEMMNI=40
@@ -1625,32 +1633,33 @@
options SEMUME=40
options SEMMNU=120
-options ALT_BREAK_TO_DEBUGGER
-
+options ALT_BREAK_TO_DEBUGGER
For PAE, it is not currently possible
- to load modules. Therefore, if you are running an architecture
- that supports Linux emulation, you will need to add:
-options COMPAT_LINUX
-options LINPROCFS
-
+ to load modules. Therefore, if you are running an
+ architecture that supports Linux emulation, you will need to
+ add:
+ options COMPAT_LINUX
+options LINPROCFS
+
Also for PAE, as of 20110912 you need
- the following. This needs to be investigated:
-nooption NFSD # New Network Filesystem Server
+ the following. This needs to be investigated:
+
+ nooption NFSD # New Network Filesystem Server
options NFSCLIENT # Network Filesystem Client
-options NFSSERVER # Network Filesystem Server
-
+options NFSSERVER # Network Filesystem Server
- As root, do the usual build steps, e.g.:
-make -j4 buildworld
-make buildkernel KERNCONF=${kernconf}
-make installkernel KERNCONF=${kernconf}
-make installworld
- The install steps use DESTDIR.
-
+ As root, do the usual build steps, e.g.:
+
+ &prompt.root; make -j4 buildworld
+&prompt.root; make buildkernel KERNCONF=${kernconf}
+&prompt.root; make installkernel KERNCONF=${kernconf}
+&prompt.root; make installworld
+
+ The install steps use DESTDIR.
@@ -1690,10 +1699,11 @@
- Also add the following users:
-squid:*:100:100::0:0:User &:/usr/local/squid:/bin/sh
-ganglia:*:102:102::0:0:User &:/usr/local/ganglia:/bin/sh
-
+ Also add the following users:
+
+ squid:*:100:100::0:0:User &:/usr/local/squid:/bin/sh
+ganglia:*:102:102::0:0:User &:/usr/local/ganglia:/bin/sh
+
Add them to etc/group as well.
@@ -1703,9 +1713,9 @@
- In etc/crontab: add
- * * * * * root /var/portbuild/scripts/client-metrics
-
+ In etc/crontab add:
+
+ * * * * * root /var/portbuild/scripts/client-metrics
@@ -1716,26 +1726,28 @@
- In etc/inetd.conf: add
- infoseek stream tcp nowait nobody /var/portbuild/scripts/reportload
-
+ In etc/inetd.conf add:
+
+ infoseek stream tcp nowait nobody /var/portbuild/scripts/reportload
- We run the cluster on UTC:
- cp /usr/share/zoneinfo/Etc/UTC etc/localtime
-
+ We run the cluster on UTC. If you have not set the
+ clock to UTC:
+
+ &prompt.root; cp /usr/share/zoneinfo/Etc/UTC etc/localtimeCreate the appropriate
- etc/rc.conf. (If you are using
+ /etc/rc.conf. (If you are using
pxeboot, and have multiple,
different, machines, you will need to put those in
the override directories.)
- Recommended entries for physical nodes:
-hostname="${hostname}"
+ Recommended entries for physical nodes:
+
+ hostname="${hostname}"
inetd_enable="YES"
linux_enable="YES"
nfs_client_enable="YES"
@@ -1750,17 +1762,17 @@
squid_enable="YES"
squid_chdir="/usr2/squid/logs"
squid_pidfile="/usr2/squid/logs/squid.pid"
-
-
+
- Required entries for VMWare-based nodes:
-vmware_guest_vmmemctl_enable="YES"
+ Required entries for VMWare-based nodes:
+
+ vmware_guest_vmmemctl_enable="YES"
vmware_guest_guestd_enable="YES"
-
-
+
- Recommended entries for VMWare-based nodes:
-hostname=""
+ Recommended entries for VMWare-based nodes:
+
+ hostname=""
ifconfig_em0="DHCP"
fsck_y_enable="YES"
@@ -1775,8 +1787,7 @@
squid_enable="YES"
squid_chdir="/usr2/squid/logs"
squid_pidfile="/usr2/squid/logs/squid.pid"
-
-
+
&man.ntpd.8; should not
be enabled for VMWare instances.
@@ -1797,8 +1808,9 @@
- Modify etc/sysctl.conf:
-9a10,30
+ Modify etc/sysctl.conf:
+
+ 9a10,30
> kern.corefile=/usr2/%N.core
> kern.sugid_coredump=1
> #debug.witness_ddb=0
@@ -1819,8 +1831,7 @@
> security.jail.chflags_allowed=1
> security.jail.enforce_statfs=1
>
-> vfs.lookup_shared=1
-
+> vfs.lookup_shared=1
@@ -1829,11 +1840,8 @@
@pointyhat.freebsd.org.
-
-
-
@@ -1841,13 +1849,13 @@
- Install the following ports:
-net/rsync
+ Install the following ports:
+
+ net/rsync
security/openssh-portable (with HPN on)
security/sudo
sysutils/ganglia-monitor-core (with GMETAD off)
-www/squid (with SQUID_AUFS on)
-
+www/squid (with SQUID_AUFS on)
There is a WIP to create a meta-port, but it is not yet
complete.
@@ -1855,7 +1863,7 @@
- Customize files in usr/local/etc/.
+ Customize files in usr/local/etc/.
Whether you do this on the client itself, or another
machine, will depend on whether you are using
pxeboot.
@@ -1864,7 +1872,7 @@
The trick of using conf
override subdirectories is less effective here, because
you would need to copy over all subdirectories of
- usr/. This is an implementation
+ /usr/. This is an implementation
detail of how the pxeboot works.
@@ -1873,8 +1881,9 @@
Modify
- usr/local/etc/gmond.conf:
-21,22c21,22
+ usr/local/etc/gmond.conf:
+
+ 21,22c21,22
< name = "unspecified"
< owner = "unspecified"
---
@@ -1883,8 +1892,7 @@
24c24
< url = "unspecified"
---
-> url = "http://pointyhat.freebsd.org"
-
+> url = "http://pointyhat.freebsd.org"
If there are machines from more than one cluster in the
@@ -1897,8 +1905,9 @@
Create
usr/local/etc/rc.d/portbuild.sh,
using the appropriate value for
- scratchdir:
-#!/bin/sh
+ scratchdir:
+
+ #!/bin/sh
#
# Configure a package build system post-boot
@@ -1915,14 +1924,14 @@
done
# Flag that we are ready to accept jobs
-touch /tmp/.boot_finished
-
+touch /tmp/.boot_finished
Modify
- usr/local/etc/squid/squid.conf:
-288,290c288,290
+ usr/local/etc/squid/squid.conf:
+
+ 288,290c288,290
< #auth_param basic children 5
< #auth_param basic realm Squid proxy-caching web server
< #auth_param basic credentialsttl 2 hours
@@ -1937,8 +1946,7 @@
2007a2011
> maximum_object_size 400 MB
2828a2838
-> negative_ttl 0 minutes
-
+> negative_ttl 0 minutesAlso, change usr/local
to usr2 in
@@ -1967,18 +1975,17 @@
Modify
- usr/local/etc/sudoers:
-38a39,42
+ /usr/local/etc/sudoers:
+
+ 38a39,42
>
> # local changes for package building
> %wheel ALL=(ALL) ALL
-> ports-${arch} ALL=(ALL) NOPASSWD: ALL
-
+> ports-${arch} ALL=(ALL) NOPASSWD: > ALL
-
@@ -1993,16 +2000,16 @@
- As root:
-mkdir portbuild
-chown ports-${arch}:ports-${arch} portbuild
-mkdir pkgbuild
-chown ports-${arch}:ports-${arch} pkgbuild
-mkdir squid
-mkdir squid/cache
-mkdir squid/logs
-chown -R squid:squid squid
-
+ As root:
+
+ &prompt.root; mkdir portbuild
+&prompt.root; chown ports-${arch}:ports-${arch} portbuild
+&prompt.root; mkdir pkgbuild
+&prompt.root; chown ports-${arch}:ports-${arch} pkgbuild
+&prompt.root; mkdir squid
+&prompt.root; mkdir squid/cache
+&prompt.root; mkdir squid/logs
+&prompt.root; chown -R squid:squid squid
@@ -2011,7 +2018,7 @@
between boots then they must either preserve their
/tmp, or revalidate their available
builds at boot time (see the script on the amd64
- machines). They must also clean up stale chroots from previous
+ machines). They must also clean up stale jails from previous
builds before creating /tmp/.boot_finished.
@@ -2022,11 +2029,11 @@
As root, initialize the squid
- directories:
- squid -z
+ directories:
+
+ &prompt.root; squid -z
-
@@ -2073,15 +2080,15 @@
of MASTER_SITE_BACKUP must be common
to all nodes, but the first entry in
MASTER_SITE_OVERRIDE should be the
- nearest local FTP mirror. Example:
-.if defined(FETCH_ORIGINAL)
+ nearest local FTP mirror. Example:
+
+ .if defined(FETCH_ORIGINAL)
MASTER_SITE_BACKUP=
.else
MASTER_SITE_OVERRIDE= \
ftp://friendly-local-ftp-mirror/pub/FreeBSD/ports/distfiles/${DIST_SUBDIR}/ \
ftp://${BACKUP_FTP_SITE}/pub/FreeBSD/distfiles/${DIST_SUBDIR}/
-.endif
-
+.endif
tar it up and move it to the right
@@ -2092,19 +2099,20 @@
Hint: you will need one of these for each machine;
however, if you have multiple machines at one site, you
should create a site-specific one (e.g. in
- /var/portbuild/conf/clients/)
+ /var/portbuild/conf/clients/)
and symlink to it.Create
-/var/portbuild/${arch}/portbuild-${hostname}
+/var/portbuild/${arch}/portbuild-${hostname}
using one of the existing ones as a guide. This
file contains overrides to
/var/portbuild/${arch}/portbuild.conf.
- Suggested values:
-disconnected=1
+ Suggested values:
+
+ disconnected=1
http_proxy="http://localhost:3128/"
squid_dir=/usr2/squid
scratchdir=/usr2/pkgbuild
@@ -2113,17 +2121,16 @@
rsync_gzip=-z
infoseek_host=localhost
-infoseek_port=${tunelled-tcp-port}
-
+infoseek_port=${tunelled-tcp-port}
- Possible other values:
-use_md_swap=1
+ Possible other values:
+
+ use_md_swap=1
md_size=9g
use_zfs=1
scp_cmd="/usr/local/bin/scp"
ssh_cmd="/usr/local/bin/ssh"
-
-
+
@@ -2141,12 +2148,11 @@
Add an appropriate data_source entry to
/usr/local/etc/gmetad.conf:
-
- data_source "arch/location Package Build Cluster" 30 hostname
-
- You will need to restart gmetad.
-
+ data_source "arch/location Package Build Cluster" 30 hostname
+
+ You will need to restart
+ gmetad.
@@ -2195,18 +2201,11 @@
Tell qmanager about the node. Example:
- python path/qmanager/qclient add
- name=uniquename
- arch=arch
- osversion=osversion
- numcpus=number
- haszfs=0
- online=1
- domain=domain
- primarypool=package
- pools="package all" maxjobs=1
- acl="ports-arch,deny_all"
-
+ &prompt.root; python path/qmanager/qclient \
+add name=uniquename arch=arch osversion=osversion \
+numcpus=number haszfs=0 online=1 \
+domain=domain primarypool=package pools="package all" \
+maxjobs=1 acl="ports-arch,deny_all"
@@ -2215,11 +2214,14 @@
How to configure a new &os; branch
+
+ Steps necessary before qmanager is started
+
+
+
When a new branch is created, some work needs to
be done to specify that the previous branch is no longer
- equivalent to HEAD. The following
- instructions apply to the previous
- branch number:
+ equivalent to HEAD.
@@ -2235,7 +2237,8 @@
For what was previously head, change
SRC_BRANCH_branch_TAG to
- RELENG_branch_0.
+ RELENG_branch_0
+ (literal zero).
@@ -2254,16 +2257,16 @@
(Only for old codebase)
Create a new zfs filesystem
- for sources:
- zfs create a/snap/src-branch
-
+ for sources:
+
+ &prompt.root; zfs create a/snap/src-branch(Only necessary for old codebase):
- Checkout a src tree in the new filesystem:
- cvs -Rq -d /r/ncvs co -d src-branch-r RELENG_branch
-
+ Checkout a src tree in the new filesystem:
+
+ &prompt.root; cvs -Rq -d /r/ncvs co -d src-branch-r RELENG_branch
@@ -2307,26 +2310,56 @@
(Only necessary for old codebase):
In the /var/portbuild/errorlogs/
- directory, create links for the webserver:
-ln -s ../arch/branch/builds/latest/bak/errors arch-branch-full
-ln -s ../arch/branch/builds/latest/bak/logs arch-branch-full-logs
-ln -s ../arch/branch/builds/latest/errors arch-branch-latest
-ln -s ../arch/branch/builds/latest/logs arch-branch-latest-logs
-ln -s ../arch/branch/builds/latest/bak/packages arch-branch-packages-full
-ln -s ../arch/branch/builds/latest/packages arch-branch-packages-latest
-
+ directory, create links for the webserver:
+
+ &prompt.root; ln -s ../arch/branch/builds/latest/bak/errors arch-branch-full
+&prompt.root; ln -s ../arch/branch/builds/latest/bak/logs arch-branch-full-logs
+&prompt.root; ln -s ../arch/branch/builds/latest/errors arch-branch-latest
+&prompt.root; ln -s ../arch/branch/builds/latest/logs arch-branch-latest-logs
+&prompt.root; ln -s ../arch/branch/builds/latest/bak/packages arch-branch-packages-full
+&prompt.root; ln -s ../arch/branch/builds/latest/packages arch-branch-packages-latest
+
+
+
+
+
+
+ Steps necessary after qmanager is started
+
+
+
+
+ Again, as
+ ports-arch:
+
+
+
- Kick-start the build for the branch with
- build create archbranch
-
+ For each branch that will be supported, do the following:
+
-
- Create bindist.tar
- .
+
+
+ Kick-start the build for the branch with the following
+ command:
+
+ &prompt.root; build create archbranch
+
+
+
+ Create
+ bindist.tar.
+
+
+
+
+
+
+
@@ -2354,10 +2387,11 @@
- (both):
-umount a/snap/src-old-branch/src;
-umount a/snap/src-old-branch;
-zfs destroy -r a/snap/src-old-branch
+ (both):
+
+ &prompt.root; umount a/snap/src-old-branch/src
+&prompt.root; umount a/snap/src-old-branch
+&prompt.root; zfs destroy -r a/snap/src-old-branch
@@ -2418,7 +2452,7 @@
with the following changes:
- Change
+ Change the value of
SRC_BRANCH_branch_TAG to
RELENG_branch_N
where N is the newest 'oldest' release
@@ -2456,6 +2490,11 @@
How to configure a new architecture
+
+ Steps necessary before qmanager is started
+
+
+
The initial steps need to be done using
sudo.
@@ -2469,20 +2508,19 @@
- mkdir /var/portbuild/arch
+ &prompt.root; mkdir /var/portbuild/arch
- Create a new zfs filesystem:
- zfs create -o mountpoint=/a/portbuild/arch a/portbuild/arch
-
+ Create a new zfs filesystem:
+
+ &prompt.root; zfs create -o mountpoint=/a/portbuild/arch a/portbuild/arch
-
-chown ports-arch:portmgr /var/portbuild/arch;
-chmod 755 /var/portbuild/arch;
-cd /var/portbuild/arch
+ &prompt.root; chown ports-arch:portmgr /var/portbuild/arch
+&prompt.root; chmod 755 /var/portbuild/arch
+&prompt.root; cd /var/portbuild/arch
@@ -2490,10 +2528,14 @@
- Create a directory for buildlogs and errorlogs:
- mkdir /dumpster/pointyhat/arch/archive
-
+ Create a directory for buildlogs and errorlogs:
+
+ (Specific to pointyhat; fix this! (XXX MCL)
+
+
+ &prompt.root; mkdir /dumpster/pointyhat/arch/archive
+
It is possible that /dumpster/pointyhat
will not have enough space. In that case, create the archive
@@ -2505,9 +2547,13 @@
- Create a link to the above for the webserver:
- ln -s /dumpster/pointyhat/arch/archive archive
-
+ Create a link to the above for the webserver:
+
+
+ Specific to pointyhat; fix this! (XXX MCL)
+
+
+ &prompt.root; ln -s /dumpster/pointyhat/arch/archive archive
@@ -2520,7 +2566,9 @@
In the
/var/portbuild/arch
- directory:mkdir clients
+ directory:
+
+ &prompt.root; mkdir clients
@@ -2528,17 +2576,18 @@
- mkdir loads
+ &prompt.root; mkdir loads
- mkdir lockfiles
+ &prompt.root; mkdir lockfilesCreate a local make.conf. In the
- most trivial case, you can
- ln ../make.conf ./make.conf
+ most trivial case, you can
+
+ &prompt.root; ln ../make.conf ./make.conf
@@ -2549,9 +2598,9 @@
(Only necessary for old codebase) Create
pnohang.arch.
(The easiest way may be to do the following on a client, and
- then copy it back):
- cc pnohang.c -o pnohang-arch
-
+ then copy it back):
+
+ &prompt.root; cc pnohang.c -o pnohang-arch
@@ -2566,7 +2615,7 @@
- cd .ssh && ssh-keygen
+ &prompt.root; cd .ssh && ssh-keygen
@@ -2575,9 +2624,10 @@
- Make the private configuration directory:
- mkdir /var/portbuild/conf/arch
-
+ If needed, make the private configuration
+ directory:
+
+ &prompt.root; mkdir /var/portbuild/conf/arch
@@ -2592,11 +2642,6 @@
- Tell qmanager about the arch:
- python path/qmanager/qclient add_acl name=ports-arch uidlist=ports-arch gidlist=portmgr sense=1
-
-
- (Only necessary for new codebase):
Add arch to SUPPORTED_ARCHS in
/var/portbuild/arch/server.conf.
@@ -2616,27 +2661,63 @@
(Only necessary for old codebase):
As with the procedure for creating a new branch:
- in the /var/portbuild/errorlogs/
- directory, create links for the webserver:
-ln -s ../arch/branch/builds/latest/bak/errors arch-branch-full
-ln -s ../arch/branch/builds/latest/bak/logs arch-branch-full-logs
-ln -s ../arch/branch/builds/latest/errors arch-branch-latest
-ln -s ../arch/branch/builds/latest/logs arch-branch-latest-logs
-ln -s ../arch/branch/builds/latest/bak/packages arch-branch-packages-full
-ln -s ../arch/branch/builds/latest/packages arch-branch-packages-latest
-
+ in the /var/portbuild/errorlogs/
+ directory, create links for the webserver:
+
+ &prompt.root; ln -s ../arch/branch/builds/latest/bak/errors arch-branch-full
+&prompt.root; ln -s ../arch/branch/builds/latest/bak/logs arch-branch-full-logs
+&prompt.root; ln -s ../arch/branch/builds/latest/errors arch-branch-latest
+&prompt.root; ln -s ../arch/branch/builds/latest/logs arch-branch-latest-logs
+&prompt.root; ln -s ../arch/branch/builds/latest/bak/packages arch-branch-packages-full
+&prompt.root; ln -s ../arch/branch/builds/latest/packages arch-branch-packages-latest
- In the /var/portbuild/errorlogs/
- directory, create two more links for the webserver:
-ln -s ../arch/archive/buildlogs arch-buildlogs
-ln -s ../arch/archive/errorlogs arch-errorlogs
+ In the /var/portbuild/errorlogs/
+ directory, create two more links for the webserver:
+
+ &prompt.root; ln -s ../arch/archive/buildlogs arch-buildlogs
+&prompt.root; ln -s ../arch/archive/errorlogs arch-errorlogs
+ One last time using sudo:
+
+
+
+
+ Add an appropriate arch entry for
+ /var/portbuild/scripts/dologs to the root
+ crontab. (This is a hack and should go away.)
+
+
+
+
+
+
+
+
+ Steps necessary after qmanager is started
+
+
+
+
+ Again using sudo:
+
+
+
+
+ Tell qmanager about the
+ arch:
+
+ &prompt.root; python path/qmanager/qclient add_acl name=ports-arch uidlist=ports-arch \
+gidlist=portmgr sense=1
+
+
+
+ Again, as
ports-arch:
@@ -2648,8 +2729,9 @@
- Kick-start the build for the branch with
- build create archbranch
+ Kick-start the build for the branch with
+
+ &prompt.root; build create archbranch
@@ -2673,19 +2755,20 @@
arch: add the arch to
/var/portbuild/scripts/dopackagestats.
+
-
- Add an appropriate arch entry for
- /var/portbuild/scripts/dologs to the root
- crontab. (This is a hack and should go away.)
-
-
-
+
+
+
+
How to configure a new head node (pointyhat instance)
+ This section only applies to the new codebase. New installations
+ using the old codebase are not supported.
+
This section is in progress.Please talk to Mark Linimon before making any changes.
@@ -2700,60 +2783,91 @@
- For each supported arch, add a
- ports-${arch}
- user and group. Add them to the wheel
- group. They should have the '*' password.
- Also, similarly, create the ports and
- portmgr users.
+ Create a user to own the portbuild
+ repository, such as portmgr, and add it
+ to the wheel group. It should have the
+ '*' password.
- For each supported arch, create
- /home/ports-${arch}/.ssh/
- and populate authorized_keys.
+ Add the following to
+ /boot/loader.conf:
+
+ console="vidconsole,comconsole"
-
- Create the appropriate files in
- /etc/.ssh/.
+ Create the appropriate
+ etc/rc.conf.
+
+ Required entries:
+
+ hostname="${hostname}"
+sshd_enable="YES"
+
+
+ Recommended entries:
+
+ background_fsck="NO"
+clear_tmp_enable="YES"
+dumpdev="AUTO"
+fsck_y_enable="YES"
+
+apache22_enable="YES"
+apache_flags=""
+apache_pidfile="/var/run/httpd.pid"
+gmetad_enable="YES"
+gmond_enable="YES"
+inetd_enable="YES"
+# XXX MCL these don't work!
+#inetd_flags="-R 10000 -lw"
+mountd_enable="YES"
+nfs_server_enable="YES"
+nfs_server_flags="-u -t -n 12"
+nfs_remote_port_only="YES"
+ntpd_enable="YES"
+ntpdate_enable="YES"
+ntpdate_flags="north-america.pool.ntp.org"
+rpcbind_enable="YES"
+rpc_lockd_enable="NO"
+rpc_statd_enable="YES"
+sendmail_enable="NONE"
+smartd_enable="YES"
+
- Add the following to /boot/loader.conf:
-console="vidconsole,comconsole"
-
+ Create etc/resolv.conf, if
+ necessary.
- Add the following to /etc/sysctl.conf:
-kern.maxfiles=40000
-kern.maxfilesperproc=38000
-
+ Create the appropriate files in
+ /etc/ssh/.
- Make sure the following change is made to
- /etc/ttys:
-ttyu0 "/usr/libexec/getty std.9600" vt100 on secure
-
+ Add the following to
+ /etc/sysctl.conf:
+
+ kern.maxfiles=40000
+kern.maxfilesperproc=38000
- TBA
+ Make sure the following change is made to
+ /etc/ttys:
+
+ ttyu0 "/usr/libexec/getty std.9600" vt100 on secure
-
@@ -2764,39 +2878,43 @@
Create a zfs volume named
a and mount it on
- /a:
-# zpool create a mirror da1 da2 mirror da3 da4 mirror da5 da6 mirror da7 da8
-
+ /a. An example:
+
+ &prompt.root; zpool create a mirror da1 da2 mirror da3 da4 mirror da5 da6 mirror da7 da8
- Set up the base portbuild directory:
-# mkdir -p /a/portbuild
-# cd /a/portbuild
-# chown portmgr:portmgr .
-# chmod 775 .
-
+ Create the portbuild directory:
+
+ &prompt.root; mkdir -p /a/portbuild
- TBA
+ Create and mount a new zfs
+ filesystem on it:
+
+ &prompt.root; zfs create -o mountpoint=/a/portbuild a/portbuild
+
+ Set up the portbuild directory:
+
+ &prompt.root; cd /a/portbuild
+&prompt.root; chown portmgr:portmgr .
+&prompt.root; chmod 775 .
+&prompt.root; ln -sf /a/portbuild /var/portbuild
+
-
Configuring src
-
TBA
-
-
@@ -2804,9 +2922,11 @@
- The following ports (or their latest successors) are required:
-databases/py-pysqlite23
-databases/py-sqlalchemy
+ The following ports (or their latest successors) are
+ required:
+
+ databases/py-sqlite3
+databases/py-sqlalchemy (only SQLITE is needed)
devel/git (WITH_SVN)
devel/py-configobj
devel/py-setuptools
@@ -2815,21 +2935,22 @@
net/rsync
sysutils/ganglia-monitor-core (with GMETAD off)
sysutils/ganglia-webfrontend (WITHOUT_X11)
-www/apache22 (with EXT_FILTER and THREADS)
-
+www/apache22 (with EXT_FILTER)
- Expect those to bring in:
-databases/sqlite3
-lang/perl-5.12
-lang/python27
-
+ Expect those to bring in, among others:
- The following ports (or their latest successors) are strongly suggested:
-benchmarks/bonnie++
+ databases/sqlite3
+lang/perl-5.14 (or successor)
+lang/python27 (or sucessor)
+
+ The following ports (or their latest successors) are
+ strongly suggested:
+
+ benchmarks/bonnie++
devel/ccache
mail/postfix
net/isc-dhcp41-server
-ports-mgmt/pkg_cutleaves
+ports-mgmt/pkg
ports-mgmt/pkg_tree
ports-mgmt/portaudit
ports-mgmt/portmaster
@@ -2837,31 +2958,240 @@
shells/bash
shells/zsh
sysutils/screen
-sysutils/smartmontools
-
+sysutils/smartmontools
- Configure mail by doing the following: TBA.
-
+ If not already created, add the following users:
+
+ squid:*:100:100::0:0:User &:/usr/local/squid:/bin/sh
+ganglia:*:102:102::0:0:User &:/usr/local/ganglia:/bin/sh
+
+ Add them to /etc/group as
+ well.
+
+
+ Configure mail by doing the following:
+
+ TBA.
+
+
+
+ Setting up and configuring the portbuild repository
+
+
+
+
+ Set up an account with subversion config files
+ installed. Again, we generally use
+ portmgr.
+
+
+
+ As that user, set up the repository:
+
+ &prompt.user; svn checkout svn://svn.freebsd.org/base/projects/portbuild /a/portbuild
+
+
+
+ Configure how build slaves will talk to your server
+ by making the following changes to
+ /var/portbuild/conf/client.conf:
+
+
+
+
+ Set CLIENT_NFS_MASTER to wherever
+ your build slaves will PXE boot from. (Possibly, the
+ hostname of your server.)
+
+
+
+ Set CLIENT_BACKUP_FTP_SITE
+ to a backup site for FTP fetches; again, possibly
+ the hostname of your server.
+
+
+
+ Set CLIENT_UPLOAD_HOST to
+ where completed packages will be uploaded.
+
+
+
+
+ Most of the other default values should be fine.
+
+
+
+ Most of the default values in
+ /var/portbuild/conf/common.conf
+ should be fine. This file holds definitions used by
+ both the server and all its clients.
+
+
+
+ /var/portbuild/conf/make.conf
+ is most likely obsolete.
+
+
+
+ Configure the server by making the following changes to
+ /var/portbuild/conf/server.conf:
+
+
+
+
+ Set SUPPORTED_ARCHS to the
+ list of architectures you wish to build packages for.
+
+
+
+ For each source branch you will be building for, set
+ SRC_BRANCHES and
+ SRC_BRANCH_branch_TAG
+ as detailed in .
+ You should not need to change
+ SRC_BRANCHES_PATTERN.
+
+
+
+ Set UPLOAD_DIRECTORY,
+ UPLOAD_TARGET, and
+ UPLOAD_USER as approprite
+ for your site.
+
+
+
+ Set MASTER_URL to the http
+ URL of your server. This will be stamped into the
+ package build logs and the indices thereof.
+
+
+
+
+ Most of the other default values should be fine.
+
+
+
+
-
- Other
+
+ pre-qmanager
- TBA
+ For each architecture, follow the steps in
+ .
+
+ qmanager
+
+
+
+
+ Copy the following files from
+ /var/portbuild/conf/etc/rc.d to
+ /usr/local/etc/rc.d:
+
+ buildproxy.sh
+pollmachine.sh
+qmanager.sh
+
+ As root, start each one of them. You may find it handy
+ to start each under screen for
+ debugging purposes.
+
+
+
+ Initialize the qmanager
+ database's acl list:
+
+ &prompt.root; python /var/portbuild/qmanager/qclient add_acl name=deny_all uidlist= gidlist= sense=0
+
+
+
+
+
+
+
+ Creating src and ports repositories
+
+
+
+
+ Run the following commands manually to create the
+ src and ports
+ repositories, respectively:
+
+ &prompt.user; sudo /var/portbuild/scripts/updatesnap.ports
+&prompt.user; sudo /var/portbuild/scripts/updatesnap
+
+
+ These will be periodically run from the root
+ crontab, which you will
+ install below.
+
+
+
+
+
+ Other services
+
+
+
+
+ Configure
+ /usr/local/etc/apache22/httpd.conf
+ as appropriate for your site.
+
+
+
+ Copy /var/portbuild/conf/apache.conf
+ to the appropriate Includes subdirectory, e.g.,
+ /usr/local/etc/apache22/Includes/apache.conf.
+ Configure it as appropriate for your site.
+
+
+
+ Set up /var/portbuild/crontab/root as
+ the root crontab via crontab -e. If you do
+ not support all the archs listed there, make sure to comment out
+ the appropriate dologs entries.
+
+
+
+ If your build slaves will be pxebooted, make sure to
+ enable the tftp entries in
+ /etc/inetd.conf.
+
+
+
+
+
+
+
+ Finishing up
+
+
+
+
+ For each architecture, follow the steps in .
+
+
+
+ You should now be ready to build packages.
+
@@ -2894,11 +3224,11 @@
Take the machine out of mlist and (for ata disks)
run smartctl on the drive:
- smartctl -t long /dev/ad0
+ &prompt.root; smartctl -t long /dev/ad0It will take about 1/2 hour:
- gohan51# smartctl -t long /dev/ad0
+ &prompt.root; smartctl -t long /dev/ad0
smartctl version 5.38 [i386-portbld-freebsd8.0] Copyright (C) 2002-8
Bruce Allen
Home page is http://smartmontools.sourceforge.net/