To set up an account on perforce, run /usr/local/bin/p4newuser and enter a password that is not the same as your freebsd login (if you have a password there) and not the same as any ssh passphrase etc. Attached are some hints. The repo has grown since this was written. We now have: //depot/projects/kse/sys/... //depot/projects/smpng/sys/... //depot/projects/nfs/sys/... as well as some user branches in //depot/user//... There are some FAQ's at the end. ------- Forwarded Messages Date: Wed, 18 Jul 2001 06:11:23 -0700 From: Peter Wemm To: jhb@freebsd.org Subject: p4 cookbook Besides the quickstart: http://www.perforce.com/perforce/doc.011/manuals/boilerplates/quickstart.html here are some pointers. Step 0: snarf: ftp://ftp.perforce.com/pub/perforce/r01.1/bin.freebsd4/p4 and put it in /usr/local/bin, chmod +x on your development box[es]: create ~/.p4config P4PORT=perforce.freebsd.org:1666 (or use a local ssh port forwarding tunnel) P4USER=jhb P4PASSWD=foo123 (this is the temporary one, it is used for a CHAP-style handshake and does not go in the clear. But dont use an important one here) P4CLIENT=jhb_ (this has to be unique per checkout) for tcsh: setenv P4CONFIG .p4config (whatever for bash, set it in .tcshrc) If you dont specify a full path in $P4CONFIG, p4 will search ., .., ../.., etc until it finds one. You can use this in the top of your development trees. If your firewall wont let you connect: ssh -C -L 1666:perforce.freebsd.org:1666 freefall.freebsd.org sleep 9999999 and use P4PORT=localhost:1666 ok. this should set up to talk to perforce. p4 keeps client state on the server so it can warn people who are going to edit the same file. To set up your client space, you want this: $ mkdir ~/p4 (or wherever) $ cd ~/p4 $ p4 client correctly> Root: /home/jhb/p4 (or wherever you're going to keep this) Options compress rmdir (leave the others off, or "no") View: //depot/projects/smpng/sys/... //jhb_/smpng/... //depot/user/jhb/preemption/... //jhb_/preemption/... This "view" (see the quickstart) maps part of the p4 tree into your client space. In this case you'll get the sys part of the smpng tree into /home/jhb/p4/smpng/.... (the /sys/ will get mapped out, you could use //jhb_/smpng/sys/... if you wanted the sys subdir to stay). The other dir will be /home/jhb/p4/preemption/. $ p4 sync stand back. This will sync up your client an what it has (ie: nothing) with what the server has. This will be big. Use ssh -C and/or Options: compress for the first time. This is roughly the equivalent of doing a cvs checkout -P from freefall over the network. $ p4 sync again, this syncs your client. Note that it should be nearly instant this time and shouldn't do anything unless somebody has committed something after you started syncing last time. This is the equivalent of cvs update now. $ p4 edit file in order to get fast syncing, the client space state is kept on the server(!). This means you need to tell it when you're doing stuff. At this point it will tell you if somebody else has the file open for edit. $ p4 revert file cancel editing the file and restore the original $ p4 diff -du file see your changes $ p4 submit check in *ALL* your changes. Note that the file list on the submit form can be edited. If you dont want a file to be included, just delete it from the list. $ p4 submit file check in just one file $ p4 submit ... check in all files opened for edit below the current directory The ... is an important wildcard, it means everything from here and below. In the client view above, remember the: //depot/projects/smpng/sys/... That means get everything from the sys dir and below. This works with all paths and is safe from shell expansion. If you are "detached" (eg: no net access) you can chmod the files to enable write access and hack away. Obviously when you get back online again, the server wont know about the files you've edited. You can compare your checkout like this: $ p4 diff -se ... (this gives you a list of files that you have edited that the server doesn't know about) $ p4 diff -se ... | xargs p4 edit (tell the server about it) Note: do NOT do a 'p4 sync' before comparing or the client might clobber files that you've edited. (I think it will complain that it is writable if the 'noclobber' client space option is set (it is by default), but dont depend on it. :-). This is the easy way to clean up after patches. start with a clean tree and then apply the patch. Then do p4 diff -se ... to find the changed files. you can use p4 add/delete for new files/deleted files as well. p4 diff -sd ... will list files you deleted, and find . -type f | xargs p4 add will add any new ones. (beware .orig!!) If you want to check out the tree onto multiple machines and dont want to pull the damn thing across the net each time (Y! dont care, WRS might) you can cheat. First, start with a relatively client space that has no pending files for submit. Take a .tar.gz of the checked out files. Create the client space on the new machine as above, but call it jhb_ or something else unique. But instead of the first 'p4 sync' do this: p4 flush @jhb_ where that is the name of the client space you copied *from*. This tells p4 that "trust me, my space now looks just like the one on jhb_". See the bottom of: http://www.perforce.com/perforce/doc.011/manuals/cmdref/flush.html It is really important to have each machine have its own client space. We do this a *lot* at Y! (and myself at home) and it is a lifesaver. Each machine has a legitimate client space and we use that for syncing tweaks etc. This single feature will save you lots and lots of time over an active project since you dont have to sync/merge patches anymore. Regarding Branches... There is a //depot/vendor/freebsd/... which contains the top of the *full* freebsd tree. This is being imported hourly. Paths look like: //depot/vendor/freebsd/sys/kern/subr_foo.c, //depot/vendor/freebsd/usr.sbin/config/main.c etc. There is a branch called "smpng" p4 branch -o smpng to see it: //depot/vendor/freebsd/sys/... //depot/projects/smpng/sys/... This branches the sys/... only part of freebsd into another part of the tree. Note that it requires a 'p4 integrate' command to actually pull the stuff across the branch. Hence we can have an automatic import without blowing up the build like an automatic import on cvs would do. There is a branch called "jhb_preemption" //depot/projects/smpng/sys/... //depot/user/jhb/preemption/... This is where I applied your preemption patches. This is what I'd like for you use, assuming this will work. To do an "import" from the vendor (freebsd) into the smpng branch, one would do this: $ p4 integ -b smpng (schedule revision merges) $ p4 resolve -a (merge the easy ones automatically) $ p4 resolve (fix any conflicts by hand) $ p4 submit (done. All those revisions are remembered) To do an import from the smpng stuff to your preemption workspace you do the same thing, except p4 integ -b jhb_preemption. To merge the preemption patches into the smpng tree, you do a "reverse" integrate: $ p4 integ -r -b jhb_preemption; p4 resolve -a etc. The basic idea is that people do non-trivial feature work on side branches and merge the changes incrementally into the base branches. Obviously this wont work for smpng -> cvs, but it does allow you to keep multiple feature sets on the go at once, and once they're usable merge them into the "common" smpng tree for generating a patch for public consumption or whatever. Note that integrations dont have to be submitted... you can revert the integration. This means that one could do this: $ p4 integ -r -b jhb_preemption $ p4 integ -r -b julian_kse $ p4 resolve -a $ p4 resolve and now in the uncommitted smpng common base area you have a merged tree that you can play with. This allows people to do "try and see" stuff. The real benefit of this is that we can stop having to generate patches for development work, as long as there is coordination to prevent toe stomping. We can see what everybody else is doing, and since checkin/sync is so damn fast, there is no reason to ever have unsubmitted stuff. There should be no more duplicated work because of inaccessible fixes etc. Once preemption reaches basic functionality (boots on alpha and x86 and doesn't have showstopper hang/crashes for trivial stuff like shutting down) then we reverse integrate the changes into the common smpng area. The smpng part of the tree should be something that testers can run / debug. Also note that perforce branching has a lot of smarts... One could do a "p4 integrate //depot/user/jhb/preemption/... //depot/user/jhb/proclock/..." and have p4 do the three way merges intelligently by using common root revisions etc. Essentially this means p4 can track your deltas up the branch through the common base and onto the other branch. And it does a pretty good job of it in general, as long as there is something reasonable for it to do. Have I totally lost you yet? :-) Sit down, keep a browser open at the online docs, use 'p4 help ' to look up the commands I listed above, and have a go. This will seem awkward to start with, but it will grow on you, and after sticking at it for a day or two you'll wonder how you ever got along without it. That I am 101% certain of. :-) I have essentailly been doing this since 1996 or 1997 for most non-trivial stuff. In fact, the only stuff I've *lost* has been in checked out cvs trees. Every non-trivial thing that has survived to make it into the real freebsd tree has gone via a perforce branch. I've finally got NFS in there and have the branch checked out on three machines and am editing the common change set on all three at once and syncing between them. You cant do that with cvsup. :-) The only thing I'm really worried about is WRS firewall issues (I hear pretty evil things) and what the link between freefall and WRS is like. Oh, one other thing.. "p4 user" allows you to change the Review list that you get change mail for. Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 ------- Message 2 Date: Wed, 18 Jul 2001 06:18:58 -0700 From: Peter Wemm To: jhb@freebsd.org Subject: p4 cookbook part 2: The suggested client space that I pasted in the email took a whopping 2 minutes 48 seconds to check out on my cable modem at home. A subsequent 'p4 sync' takes just under 2 seconds. peter@overcee[6:17am]~/fbp4-112> p4 client -o Client: peter_overcee Owner: peter Host: overcee.netplex.com.au Description: Peter Wemm's client at home. Root: /home/peter/fbp4 Options: noallwrite noclobber compress unlocked nomodtime rmdir LineEnd: local View: //depot/projects/smpng/sys/... //peter_overcee/smpng/... //depot/user/jhb/preemption/... //peter_overcee/preemption/... note 'compress' in the options line. Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 ------- Message 3 Date: Wed, 01 Aug 2001 17:28:54 -0700 From: Peter Wemm To: julian@freebsd.org Subject: p4 in a nutshell... This is assuming you've read the 10 minute pitch and the quick-start. Rule 0: www.perforce.com online docs are your friend. make a directory somewhere mkdir ~/p4 cd ~/p4 cat > .p4config << EOF P4PORT=perforce.freebsd.org:1666 P4CLIENT=julian_work P4USER=julian P4PASSWD=YOUR_PASSWORD_HERE EOF setenv P4CONFIG .p4config and/or export P4CONFIG=.p4config add these to your shell dot file[s] $ p4 passwd Old Password: YOUR_PASSWORD_HERE New Password: pick something else New Password again: repeat change .p4config so that the P4PASSWD line has the new password. [This is not sent in plaintext over the wire. It uses a ppp CHAP style cryptographic handshake. Do not use an "important" password however, root@freebsd.org could find it out.] Adjust P4CLIENT on each machine so that each machine has "julian_" and something unique. You can have a client space on your desktop and a compile machine. Commit on one and instantly have the changes available on the test box with a 'p4 sync'. I will use 'julian_work' for the rest of the examples. p4 client - you are now in a form - Change the options so that you have "rmdir" and "compression" (ie: remove the "no" part on those two). This is so you zlib compress the stream to perforce. In the 'view:' section, add: View: //depot/vendor/freebsd/sys/... //julian_work/sys/... //depot/projects/kse/sys/... //julian_work/kse/... //depot/user/julian/... //julian_work/julian/... This sets up the "view" that your client will see of the repository. The virgin //depot/freebsd/src/sys/kern/kern_switch.c will end up in ~/p4/sys/kern/kern_switch.c Your full path to the kse branch version of kern_switch.c is //depot/projects/kse/src/sys/kern/kern_switch.c and it will end up in ~/p4/kse/kern/kern_switch.c There is a left<->right mapping here, see the quickstart and/or user guide for more details. $ p4 sync Stand back. This will check out two kernel trees. This is why I suggested the 'compress' option. ok. Now, ~/p4/kse/... is a live work area.. At this point a cvs->p4 cheat sheet listing the differences is probably in order. - in cvs, files are read/write. In p4, they are readonly. - you 'p4 edit file' to tell the server that you are going to edit a file. at this point, the file becomes read/write and it is "open". If somebody else has the file open, you will get told "user xxx also has the file open". - cvs commit -> p4 submit. This lets you enter a commit message for *all* the open files by default. You can specify a path or file to limit this, or you can edit the form and remove all but the files you want to commit. Remember this. You can remove files from the commit form and they will be left "open". - p4 revert to unopen a file and revert the changes. - cvs update -> p4 sync and/or p4 resolve. Note that p4 splits the 'update' process into two phases.. checking out new files (sync) and merging changes (resolve). - cvs add -> p4 add - cvs add -d -> NO NEED. do not "add" directories. - cvs rm -> p4 delete - more commitlog -> p4 changes - to see a "change" - p4 describe -du - see online help: p4 help. p4 help command (eg: p4 help changes) Diffs in p4 suck a bit, but you can do- - p4 diff -du file#rev-rev - p4 diff2 -du file#rev file2#rev - p4 diff -du see all open files and any changes you have made but not submitted yet. Every hour, there is an import of freebsd into //depot/vendor/freebsd/... You "integrate" those changes into your branch like this: $ p4 opened (make sure you have no opened files already) $ p4 sync (make sure the tree is in sync) $ p4 integrate -b kse (schedule the merge) $ p4 resolve -a (do the merge, automatically if possible) $ p4 resolve (manually clean up any conflicts, see the docs!) $ p4 submit (commit the changes) do whatever testing to make sure it compiles. Do any fixups as a seperate step, do not mix corrections with the resolve process. $ cd ~/p4/kse; diff -ru2 ../sys . > exportable_patch.diff This is not perfect, but should do. I can help here. So can jhb. I would suggest you let me do the integrations until you're more comfortable. Golden rule: commits in p4 are really cheap. Commit often. Dont generate patches to get from one machine to another, commit on one box and then sync on the other. And vice versa. And *dont* leave files "open" unless you're actually working on them. Otherwise you'll have to do resolves if sometbody else edits it while you're idle. Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 ------- End of Forwarded Messages - "How do I get this stuff via cvsup?" Try something like this: *default host=cvsuo10.freebsd.org *default delete use-rel-suffix p4-cvs-ia64 release=cvs prefix=/home/p4cvs .. and you'll end up with /home/p4cvs/projects/ia64/... in rcs ,v format. I forgot about the CVSROOT (doh!) so you'll need to do something like this: setenv CVSROOT /home/p4cvs cvs init cvs -q checkout -P -d src projects/ia64 and after that, you can cd src; cvs update -d -P - "What collections are there?" As of today (may 18th, 2002), there are: p4-cvs-all Everything p4-cvs-ia64 //depot/projects/ia64/... p4-cvs-kse //depot/projects/kse/... p4-cvs-sparc64 //depot/projects/sparc64/... p4-cvs-releng-5_dp1 //depot/releng/5_dp1/... p4-cvs-releng-5_dp1-doc //depot/releng/5_dp1/doc/... p4-cvs-releng-5_dp1-src //depot/releng/5_dp1/src/... p4-cvs-trustedbsd //depot/projects/trustedbsd/... p4-cvs-trustedbsd-audit //depot/projects/trustedbsd/audit/... p4-cvs-trustedbsd-base //depot/projects/trustedbsd/base/... p4-cvs-trustedbsd-cap //depot/projects/trustedbsd/cap/... p4-cvs-trustedbsd-mac //depot/projects/trustedbsd/mac/... Trust me, you really dont want p4-cvs-all.. There would be several complete trees with about 10 copies of src/ in there.. I will be adding p4-cvs-cvsroot shortly, so check back soon. - "How often is the cvsup stuff replicated?" The replication scripts run constantly. It generally notices a p4 submit within 60 seconds and starts copying the change to cvs. freefall's cvsup server has immediate access to that, if you have access to it. (You can, if you have a good reason for it) cvsup-master is generally only up to 5 minutes behind freefall. cvsup10 can be up to an hour behind cvsup-master. - "Is there a cvsweb-like thing?" Yes, see http://people.freebsd.org/~peter/p4db/ - "Can I quickly see what's in a branch?" We keep some live diffs. http://people.freebsd.org/~jhb/smpng.patch http://people.freebsd.org/~peter/sparc64.diff http://people.freebsd.org/~peter/ia64.diff http://people.freebsd.org/~peter/kse.diff - "What is this for, anyway? Shouldn't this be in cvs?" The original point of doing this was to get two particular developers to keep their work-in-progress more easily accessible instead of being locked up at home or on laptop drives. These folks were working on very large long-running projects where there were non-trivial periods between stages that could be committed to cvs without creating absolute chaos. Other people wanted to see what they were working on, help, get easier access to it, etc. This "shadow tree" was the right tool for the job at the time. - "Why not do it on cvs branches?" Because CVS's branch tools couldn't suck any worse if a conscious effort was made to make them suck even more. A SCM is supposed to *help*, not get in your way. We spent too long fighting cvs instead of getting useful work done. It was the right tool for the job at the time. So there. :-) - "But perforce is too hard to learn how to use!" Well, don't use it. The idea was to provide a more convenient place to do stuff that was beyond CVS's easy capabilities. Yes, it works differently to cvs and requires thinking a bit differently, and if you cannot or do not want to deal with it, then so be it. It is more efficient for us, so that is what we're using. - "You should be using $MY_FAVORITE_SCM instead!" We like it, too bad. :-)