The new unionfs implementation for FreeBSD and status of merging

[Japanese]

1 Introduction

There were several known problems in unionfs implementation of FreeBSD up until 6.2-RELEASE. The specification is ambiguous and its locking implementation was buggy.

Because of these issues, mounting unionfs cd9660 file system as a lower layer had caused problems. This article describes these problems and provides patches to prevent and improve them. The most valuable codes of new unionfs has already been merged into -current/-stable. Our going improvements are getting merged into -current/-stable step by step.

2 What is unionfs

Unionfs makes it possible to mount one file system on top of another. For example, you can mount a memory file system on top of a CD-ROM. As a result, it looks as if you could write to files on the CD-ROM.

Changes are only made to the upper file system layer and no changes are made to the lower one. Therefore, you can use it to keep modifications without changing the lower layers.

For a more detailed explanation have a look at Section 6.7 on page 256 of "The Design and Implementation of the FreeBSD Operating System" by Marshall Kirk McKusick and George V. Neville-Neil.

3 Problems of original(old) unionfs implementation of FreeBSD

The problems of FreeBSD unionfs implementation up until 6.2-RELEASE are the following:

1. [Implementation Problem] The original(old) unionfs locks upper and/or lower layer locks as necessary for VOP_* calls. In this case, unionfs needs to lock up to 3 locks. Because it locks and unlocks multiple times independently of the kernel, dead-lock happens easily. In fact, the original(old) implementation is unstable and dead-locks happen quite often, especially on systems with high load.
2. [Implementation Problem] Parts of VOP_* call unionfs vnodes directly, others call upper/lower layer vnodes. If this happens in LOOKUP situations, strange problems occur. One of the known results is that pwd returns an absolute path of the upper/lower layer.
3. [Implementation Problem] Readdir is not fully implemented and only works with upper layer's list. As a result, files that only exist on lower layer cannot be seen.
4. [Implementation Problem] Because there is no white-out, if files or directories with the same name exist on both upper and lower layers, it looks like they were not removed.
5. [Implementation Problem] If files or directories exit on only lower layer, the upper layer does not have corresponding permissions. As a result, there are problems like you cannot write onto a writable file.
6. [Specification Problem] The specification of unionfs does not specify permission control when creating a shadow file or directory. In the original(old) implementation, "umask-ed 0777" was used. However, it produces undesirable situations. For example, executable permission is assigned to a file when it shouldn't.

And the original(old) unionfs implementation of FreeBSD had led some issues as follow:

• [Implementation Problem] Unstable. It was difficult to get stable servers using unionfs.
• [Specification Problem] In order to use OpenOffice.org, emacs, vim and something likes those on systems such as FreeSBIE, it had required extra steps like creating a files before operations.

4 Solution

We have made up new unionfs of FreeBSD. The most valuable codes of our new implementation has already been merged up until FreeBSD 8/FreeBSD 7.0/FreeBSD 6.3 fixing the following problems:

1. [Implementation Problem] The lock object within the unionfs vnode have been replaced by a pair of pointers to the lock objects in the upper/lower layers, and locking and unlocking of these vnodes are now controlled by unionfs. This strategy was chosen to introduce minimal changes to the other parts of the kernel. The locks on the upper/lower layers are managed to always keep the same state, so as not to interfere with the other parts of the kernel.
2. [Implementation Problem] Fixes http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/84107 (FIXED).
3. [Implementation Problem] Fixes http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/84498 (FIXED).
4. [Implementation Problem] Fixes http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/89755 (FIXED).
5. [Implementation Problem] Fixes http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/73094 (FIXED).
6. [Implementation Problem] Fixes any other known problems.

Solving [Ambiguous Specification Problems] involves discussions what the appropriate behaviour is.

As a result of the old implementation of FreeBSD unionfs up until 6.2-RELEASE, applications which check and determine their behavior accordingly do not run.

Because the specification of unionfs has ambiguity of its behavior, so it is difficult to implement appropriately. Therefore, I have proposed different options for different situations. This patch includes an option that allows unionfs to change its behavior on three ways.

Three solutions for solving [Ambiguous Specification Problems]:

• [traditional mode] - Backward compatibility mode. This mode imitates the behavior of the original(old) unionfs implementation of FreeBSD. It is default mode of merged unionfs implementation only on FreeBSD 6-STABLE bacause STABLE branch couldn't accept API or behavior changes.
• [transparent mode] - This mode attempts to maintain lower layer permissions in the upper layer as much as possible and solves most of the problems described above. It is default mode of our ongoing unionfs patchset of FreeBSD and we guess this mode is the best in most situations. This mode is defaulton FreeBSD 7/8.
• [masquerade mode] - This mode allows to specify permissions. By specifying permission explicitly, the user can maintain the security of file systems.

As a result, the following problem is also be fixed in [transparent mode].

• [Problem due to Ambiguous Specification] Fixed http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/86596 (FIXED).

5 How to Install

prompt5.1 shows how to apply the patches.

# cd /usr/src/
# patch -p3 < /anywhere/unionfs-p19-20070504.diff
# cp /usr/src/sys/fs/unionfs/union.h /usr/include/fs/unionfs/
# cd /usr/src/sbin/mount_unionfs/
# make obj depend && make && make install
# make clean
# cd /usr/src
# make buildkernel
# make installkernel
# shutdown -r now

6 How to Use

As much as anything else, see mount_unionfs(8). It is all in there.

7 Download

Patches for FreeBSD 8-current. They are numbered in chronological order, the newest patch has the highest number.

• not yet

Patches for FreeBSD 7-stable.

• not yet

Patches for FreeBSD 7-current.

• SHA256 (unionfs-p19-20070524.diff) = 174d1557f48d968633ec43e536c8574976dede43c5dab017a9dfdd615c6e78ab
• SHA256 (unionfs-p19-20070504.diff) = ece0c5a5684b50fdc1ef6649edcc25dae6490db7f1642d5068a03dc573227cf3
• SHA256 (unionfs-p18.diff) = a6ca145867657d9ca335b9c65f9ee26ed0373564d57516e6e656df0db9487193
• SHA256 (unionfs-p17.diff) = 364d2990e7e2a2382b10b5393a74a91d7d229f0a12c18c4838feb267b6931914
• SHA256 (unionfs-p16.diff) = 1328d7a44a38e0a247ddfae9136320d4b7894ceb3cb4c709fef241ef0cd2b7b7
• SHA256 (unionfs-p15.diff) = d45ba534a4d7c1d2010b9e2290a8713c6acfd8b0e4568e6f0cd9309bb8756919
• SHA256 (unionfs-p14.diff) = 79288951ca1fb5fddf75af83b29e807f40ae54040c280fabf0c8521720b1d8be
• SHA256 (unionfs-p13.diff) = 3dbd287608c39b65f8928d196963e05fa39afbd438840424cfbfb8cfbc05c8b8
• SHA256 (unionfs-p12.diff) = 2dd9ab54eb70a64305076c52e0d4372d9412c5432af139d8bb1a8025542538a3
• SHA256 (unionfs-p11.diff) = 11a5f45e4289eb18f58065020083adf583ebc32fdfc40a8937a69acf51602734
• SHA256 (unionfs-p10.diff) = ec1f21244ce48cf8817f18ce55da8c5e09a38c4945895233629ad27dd2754c5a
• SHA256 (unionfs-p9.diff) = 48210f2d5593b955fe782f3dc1bcc61cc5136ced31911f0efdfd684bce72f7d8
• SHA256 (unionfs-p8.diff) = 4544ad31ca6047f2816320367d74982e8665a7d60b2c8c73aa375723bed06cbd
• SHA256 (unionfs-p7.diff) = de808ff1f836612cc97f244793f17a6d3bcf4df8ddd0924a0331d0352d06c110
• SHA256 (unionfs-p6.diff) = 9f757b3e59beeb7c9d7ad837462d317e94fc7fc2610f97b212270780fc35f958
• SHA256 (unionfs-p5.diff) = df65da3197dd60d0c37251883a9290f9b03e6b4c1f8e2a5940886aa373ab296c
• SHA256 (unionfs-p4.diff) = 8a3fed6e835676a1dd27ca8872a042d10ef988c5e5912c9a7823835372255ce5
• SHA256 (unionfs-p3.diff) = ec7aeb1d55e6d6d86665d319a0404f61cb1066282363d70f8ce1ab41b66297b0
• SHA256 (unionfs-p2.diff) = beaf51723cfd7017a29a1e24e21c585f884328dcb808e586f4e96391c192d1f3
• SHA256 (unionfs-p1.diff) = c18b4e19a06875656819738e1de566d2b9faab418af91949e28d72ffacfd2572

Patches for FreeBSD 6-stable

• SHA256 (unionfs6-p19-20070504.diff) = 9eaf754432e67ba4dc4f5c331f19f5204ab1ae41d8cf06bde28b759a9c3eb036
• SHA256 (unionfs6-p18.diff) = 93c698d9815c8fbbef1e6ed5a0e98b47efd1d6f1563b7abc234f505653bf7912
• SHA256 (unionfs6-p17.diff) = dfded90daefeb5988ae9888b4beb7dfbbd4e937b299cdf49dcacf48936706930 (built by Craig Rodrigues)
• SHA256 (unionfs6-p16.diff) = 6e8fcae176bc119d3dbfebc531f045a561735edd320eacd1a4ccb603082bdb7d
• SHA256 (unionfs6-p15.diff) = 9eca9198a36298ed940d41190ac73681958cfd3c69a5ac6ffb3e393fb82af523
• SHA256 (unionfs6-p14.diff) = 12d57beada4d176413f156578d061bd00beac30ea8285f940714eba90e0380f8
• SHA256 (unionfs6-p13.diff) = d6a2d2b40d825daf293d5e12bec149ac376a606a8f544c1bd7a02a890e1be216
• SHA256 (unionfs6-p12.diff) = 5174e395f2adef79b6dad77532185b34ddc76bf035bf482d12c12e3c0a0cdc63
• SHA256 (unionfs6-p11.diff) = 3be3069ff5cca431efb24161f732ce9707ef9bc4691e00f06c42e7e64a96bad9
• SHA256 (unionfs6-p10.diff) = ecdaa60d72edaf77c3d67643a30e106cdaa0fada8c847c655ba123216925c112
• SHA256 (unionfs6-p9.diff) = c312ac0620a83d3a6b25a38cd061e1d0285cd9d97a37860a28e3ddb0cf2bf055
• SHA256 (unionfs6-p8-fix1.diff) = 618e44a7b127a9203b987c9f95cca8dccab17961d8c0bfd01165673ab9c950a9
• SHA256 (unionfs6-p8.diff) = e7c935a39a3483f14048c917b86081c175e8f84ba3688263a20878ec507bf85a
• SHA256 (unionfs6-p7.diff) = 6c85658eb01c25ac2413c9b8fed50b8b25fc51daf4a119e0f067f27856285937
• SHA256 (unionfs6-p6.diff) = f1e39bd48560810b3d811125318af12dab0d12bb9d4d942712b6b61798679399
• SHA256 (unionfs6-p5.diff) = 7e0c876c7428693367533b9a6bad44023851e43c0a4795a4939accc72ee9fdc7
• SHA256 (unionfs6-p4.diff) = aa5faf32dd06645cdce1a90219083707a79a25a7c0882aed8a11b19b62db0b0a
• SHA256 (unionfs6-p3.diff) = 49a80103631267d824dfa6775e572e16098aa789994bf58bb0259a8e10ead0d7
• SHA256 (unionfs6-p2.diff) = c6deb8be2cc0b3251a5a3ed04d9945f7756bb915b5a5bf77482546d34cec862c
• SHA256 (unionfs6-p1.diff) = afa06531d1525e4a96fb6f315f48a8119c79094d97994c155944bbeb0d4f64c5

7.1 Changes in unionfs-p19-20070504.diff / unionfs-p19-20070524.diff

• It has been became MPSAFE.
• Default copy mode has been changed from traditional-mode to transparent-mode. Some folks who have reported some issues have solved with transparent mode. We guess it is time to change the default copy mode. The transparent-mode is the best in most situations.
• Fixed kern/111262 issue.
• Added support of vfs_cache on unionfs. As a result, you can use applications that use procfs on unionfs.
• Removed unionfs internal cache mechanism because it has vfs_cache support instead. As a result, it just simplified code of unionfs.
• Added whiteout behavior option. ``-o whiteout=always'' is default mode(it is established practice) and ``-o whiteout=whenneeded'' is less disk-space using mode especially for resource restricted environments like embedded environments. (Contributed by Ed Schouten. Thanks)
• Fixed a mtx lock issue happened with nullfs.
• Fixed lock issues around unionfs.
• Added NULL check code pointed out by Coverity. (Pointed out by Stanislav Sedov. Thanks)

7.2 Changes in unionfs-p18.diff

• Improve access permission check treatments.
• Change it free unneeded memory ASAP.
• Added treatments to prevent readdir infinity loop useing with Linux binary compatibility feature.

7.3 Changes in unionfs-p17.diff

• Fs takes illegal access without lock of lower layer vnode if the both upper/lower layers have both vnode. To fix this problem, we change the lock mechanism to get locks for both upper/lower layer always.
• Kernel gets a dead-lock easily within above upper/lower-layer-always-lock-mechanism. To avoide above dead-lock, we changed vfs_lookup.c. By that change, it always locks vnodes parent first and children second. You could see the same lock-order-control implementation around cache_lookup.
• It takes the both open/close operations per kernel thread.
• It takes readdir-treat-status-management per kernel thread.
• It reopens vnode if needed when coping to upper layer on advlock.
• mount_unionfs(8) changes option style fitting for fstab(5) style. (by rodrigc)
• manual of mount_unionfs(8) was changed. (by rodrigc)

7.4 Changes in unionfs-p16.diff

• changed source code style and naming rules for fitting to FreeBSD kernel src style.
• committed the patch for sys/ufs/ufs/ufs_lookup.c to current tree. So the patch for sys/ufs/ufs/ufs_lookup.c is removed from unionfs-p16.diff.
• changed the patch for sys/fs/unionfs/union_subr.c fitting to changing in current tree.

7.5 Changes in unionfs-p15.diff

• deleted -r option in limbo and updated ount_unionfs.8.
• changed source code style and naming rules for fitting to FreeBSD kernel src style.

7.6 Changes in unionfs-p14.diff

• Added a patch of mount_unionfs.8. It means that this patchset is ready to be merged to FreeBSD base system. (hrs contributed it, thanks)
• Fixed a problem that sets EXTATTR(ACL, MAC) information to lower layer files and directories.
• Removed the third terms of the BSD License to get more easy to handle for FreeBSD.

7.7 Changes in unionfs-p13.diff

• Fixed a bug that leads a panic around compat2 just after VOP_RENAME call.

7.8 Changes in unionfs-p12.diff

• Fixed a bug that responses without lock when share lock is requested with VOP_LOOKUP.
• Fixed a bug that leads lock-around panic on FreeBSD 6.x.
• others, misc bug fixes

7.9 Changes in unionfs-p11.diff

• Changed a few implementation around the lock/unlock mechanism. Because of this, you can use both the unionfs and the nullfs together without LK_CANRECURSE.
• Fixed a bug that sometimes does not unlock if it cannot create shadow file.

7.10 Changes in unionfs-p10.diff

• Fixed a problem that does not unlock a vnode around some handling of VOP_RENAME.
• Added workaround implementation for panic by umount(8) -f.
• Changed around VOP_ADVLOCK handling to make shadow file into upper layer always to keep lock consistency.

7.11 Changes in unionfs-p9.diff

• Now you can use unionfs with nullfs. To make nullfs usable, I fixed the lock-handling of src/sys/fs/nullfs/null_vnops.c.

7.12 Changes in unionfs-p8.diff

• Fixed the issue that user who has access permission cannot change the directory because he cannot create its shadow directory. As a result of this fixed, now unionfs uses root permission creating shadow directory temporarily.

2006/02/13 - Added unionfs6-p8-fix1.diff fixes 6.x build fail

7.13 Changes in unionfs-p7.diff

• Fixed problem that a directory which was not empty could be removed. To fix this issue, I made some changes in src/sys/ufs/ufs/ufs_lookup.c regarding the handling of white-out.
• Fixed "Returning with 1 locks held." panic issue. The vnode lock was not freed if the creation of a shadow directory failed.

7.14 Changes in unionfs-p6.diff

• Fixed the issue that ln(1) fails to delete existing link when -f option is given. Some hard-link specific issues are also addressed.
• Added some tweak for VOP_GETWRITEMOUNT. Prior implementation had the possibility of causing write failure due to vn_start_write not functioning.

7.15 Changes in unionfs-p5.diff

• Fixed "can't fifo/vnode bypass -1" panic issue.
• Added comments that describes codework into source code.
• Make source code style consistent with style(9).

7.16 Changes in unionfs-p4.diff

• Fixed problems in 64bit arch.
• Merged source code for 6-stable and 7-current using _FreeBSD_version.

7.17 Changes in unionfs-p3.diff

• Fixed a bug that a shadow directory attribute were not set correctly.
• Fixed inappropriate locking issue (the fix in p2 was not sufficient).
• Initialize variables and componentnames correctly.

7.18 Changes in unionfs-p2.diff

• Fixed inappropriate unlock problem in the patch. It created fdvp of VOP_RENAME created a white-out without locking. As a result, if the filename already existed in the lower layer, it caused a panic.
• [-c <old|fullcopy|useful>] options are changed to [-c <traditional|transparent|masquerade>].

7.19 Changes in unionfs-p1.diff

• Rewritten from scratch seeing difficulties fixing based on existing code.
• Reimplemented and fixed known problems. However, readdir needs to be revisited.

8 Known Problems

• If you rename a directory in the lower layer, you cannot access the files in the directory anymore. This is the result of creating the white-out after creating a shadow directory in the upper layer and renaming it. To rename a directory while keeping the files in the directory accessible, make a full copy in the upper layer and rename it. It is a feature, not a bug.
• When a hard-link is created, VOP_GETWRITEMOUNT is called right after the lookup, thus if the file exists only in the lower layer, there is no vnode in the upper layer to call VOP_GETWRITEMOUNT with, resulting in a failure to create the hard-link. A true resolution would be to create a shadow file within VOP_GETWRITEMOUNT, but VOP_GETWRITEMOUNT is not allowed to make changes to the file system. For those file systems that use vop_stdgetwritemount, they simply acquire the mount point, and the vnode of the directory the file resides in can be used for this purpose. However, since this situation is not always guaranteed, it is not a safe solution. The current code works around the problem in this way, but it may cause problem in the future. Moreover, when a directory that contains more than one mount point is mounted by unionfs, this workaround will most probably cause problems, so it is advised to avoid mounting such directory with unionfs at this time.
• stat(2) cannot recognize a white-out. It cannot get the information of a white-out as the current ufs and other file system implementation do not return the vnode of a white-out (The manual tells that it can be acquired). However, the implementation of rm(1) and other commands take it for granted. Maybe, we should see this behavior not a problem but a feature. I do not know how FreeBSD came to this implementation, so I kept the existing behavior as it is.

9 Committed 1

The unionfs-17.diff (without patch for sys/kern/vfs_lookup.c) was committed to FreeBSD 7-current 2006-12-02 19:35:56 UTC.

rodrigc     2006-12-02 19:35:56 UTC

  FreeBSD src repository

  Modified files:
    sys/fs/unionfs       union.h union_subr.c union_vfsops.c 
                         union_vnops.c 
    sbin/mount_unionfs   Makefile mount_unionfs.8 mount_unionfs.c 
  Log:
  Many, many thanks to Masanori OZAWA <ozawa@ongs.co.jp>
  and Daichi GOTO <daichi@FreeBSD.org> for submitting this
  major rewrite of unionfs.  This rewrite was done to
  try to solve many of the longstanding crashing and locking
  issues in the existing unionfs implementation.  This
  implementation also adds a 'MASQUERADE mode', which allows
  the user to set different user, group, and file permission
  modes in the upper layer.
  
  Submitted by:   daichi, Masanori OZAWA
  Reviewed by:    rodrigc (modified for minor style issues)
  
  Revision  Changes      Path
  1.13      +1 -1        src/sbin/mount_unionfs/Makefile
  1.21      +184 -23     src/sbin/mount_unionfs/mount_unionfs.8
  1.25      +107 -74     src/sbin/mount_unionfs/mount_unionfs.c
  1.33      +87 -92      src/sys/fs/unionfs/union.h
  1.87      +948 -1080   src/sys/fs/unionfs/union_subr.c
  1.79      +394 -332    src/sys/fs/unionfs/union_vfsops.c
  1.135     +1864 -1613  src/sys/fs/unionfs/union_vnops.c

The most valuable codes of new unionfs have been committed to FreeBSD 6-stable 2007-02-13 05:56:43 UTC.

rodrigc     2007-02-13 05:56:43 UTC

  FreeBSD src repository

  Modified files:        (Branch: RELENG_6)
    sys/fs/unionfs       union.h union_subr.c union_vfsops.c 
                         union_vnops.c 
    sbin/mount_unionfs   Makefile mount_unionfs.8 mount_unionfs.c 
  Log:
  MFC: New unionfs implementation from Daichi GOTO and Masanori OZAWA,
       which fixes many locking and crashing problems in the previous
       implementation.  kib@ helped a lot by eliminating a source
       of deadlocks encountered with unionfs with these commits:
         rev. 1.50 of src/sys/fs/deadfs/dead_vnops.c
         rev. 1.97 of src/sys/kern/vfs_lookup.c
  Submitted by:   daichi, Masanori OZAWA <ozawa ongs co jp>
  
  Revision   Changes      Path
  1.12.8.1   +1 -1        src/sbin/mount_unionfs/Makefile
  1.20.2.1   +198 -25     src/sbin/mount_unionfs/mount_unionfs.8
  1.24.2.1   +108 -74     src/sbin/mount_unionfs/mount_unionfs.c
  1.31.2.1   +88 -94      src/sys/fs/unionfs/union.h
  1.86.2.1   +937 -1080   src/sys/fs/unionfs/union_subr.c
  1.76.2.2   +398 -334    src/sys/fs/unionfs/union_vfsops.c
  1.132.2.1  +1879 -1607  src/sys/fs/unionfs/union_vnops.c

10 Committed 2

The unionfs-p19-20070524.diff was committed to FreeBSD 8-current at 2007-10-14 JST, to RELENG_7 at 2007-10-22 JST and to RELENG_6 (without default mode change patch) at 2007-10-23 JST.

11 Appendix

11.1 FAQ

Q: Why did you rewrite from scratch?

A: The original(old) unionfs implementation of FreeBSD up until 6.2-RELEASE had many dead-lock scenes easily. Fixing of it was harder than rewriting it.

Q: What about backward compatibility?

A: If the traditional option is used, the new implementation behaves like the old one. In addition, it fixes http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/84107, http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/84498, http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/89755, http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/86596, http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/73094 and the other issues of unionfs related.

-c transparent option seems to be the most reasonable default behavior. It fixes most of the problems in the original implementation.

Q: Does the implementation of the patch meet the intended unionfs behavior?

A: I think it does. See pp.256-258, Section 6.7 `The Union Filesystem' of ``The Design and Implementation of the FreeBSD Operating System'' by Marshall Kirk McKusick and George V. Neville-Neil. for the unionfs specification.

Q: Why are shadow files created even for read access?

A: Because the system cannot modify the access time of the lower layer filesystem. Thus, it uses shadow file to update access time. You can use -o noatime option to prevent that behavior. See mount(8) manual page for detail.

Q: vi sometimes gets some messages as follow. Is it okey?

'filename: file modified more recently than this copy; use ! to override.'

A: It is no problem. vi reports above message when catched inode changes. It is correct.

11.2 Call for help

If you are interested in translating the Japanese text into English, please do so and send it to me :) My mail address is daichi AT-MARK freebsd.org or daichi AT-MAKR jp.freebsd.org.

11.3 Thanks!

Some guys have helped me with the unionfs work and I would like to thank all of them.

Yoshihiro OTA-san wrote most of the English text, Hiroo ONO-san and Hiroharu TAMARU-san translated the other parts. This page was born by those guys efforts. The manual page of new unionfs has been rewritten by Hiroki Sato-san (aka hrs). Good quality manual page was created by him.

Danny Braniss did some unionfs tests with his diskless unionfs system. Dario Freni, one of the key FreeSBIE developers, did some unionfs tests while developing FreeSBIE. Alexander Leidinger, one of the FreeBSD committers, is going to help to merge the unionfs patch to FreeBSD 7-current. Craig Rodrigues, daichi's src mentor and one of the FreeBSD committers, has very hard work around unionfs to get merge into FreeBSD 7-current. Especially by hard work of Craig Rodrigues, we could merge patches into FreeBSD 7-current.

Ed Schouten contributed whiteout-mode-switch ability. Stanislav Sedov has pointed out Coverity NULL check issue. Kris Kennaway always reports deadlock issues faster than anybody. Jeff Roberson and Ken Smith have reviewed our vast unionfs patches bunch.

And Fabian Keil gave me a patch that fixes my broken English of this page:)