GENERIC HEAD from 2012-02-10 21:26:25 UTC, r231443M, vmcore.277 KDB: debugger backends: ddb KDB: current backend: ddb 524288K of memory above 4GB ignored Copyright (c) 1992-2012 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #0 r231443M: Fri Feb 10 23:07:04 CET 2012 pho@x4.osted.lan:/usr/src/sys/i386/compile/PHO i386 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. CPU: AMD Phenom(tm) 9150e Quad-Core Processor (1800.01-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x100f23 Family = 10 Model = 2 Stepping = 3 Features=0x178bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x7ff TSC: P-state invariant real memory = 4294967296 (4096 MB) avail memory = 3531644928 (3368 MB) : Trying to mount root from ufs:/dev/ad4s1a [rw]... WARNING: / was not properly dismounted Setting hostuuid: 00000000-0000-0000-0000-00218515337d. Setting hostid: 0x6b64ac17. Starting ddb. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: /dev/ad4s1a: 8703 files, 870044 used, 142971 free (1195 frags, 17722 blocks, 0.1% fragmentation) /dev/ad4s1f: 20435 files, 869573 used, 143442 free (4818 frags, 17328 blocks, 0.5% fragmentation) /dev/label/tmp: 100708 files, 3682309 used, 114631672 free (2000 frags, 14328709 blocks, 0.0% fragmentation) /dev/ad4s1d: 781588 files, 9211186 used, 943013 free (366293 frags, 72090 blocks, 3.6% fragmentation) /dev/ad4s1e: 257498 files, 12554520 used, 12833997 free (53541 frags, 1597557 blocks, 0.2% fragmentation) Mounting local file systems:. Setting hostname: x4.osted.lan. re0: link state changed to DOWN Starting Network: lo0 re0 fwe0 fwip0. lo0: flags=8049 metric 0 mtu 16384 options=3 inet 127.0.0.1 netmask 0xff000000 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0xa nd6 options=21 re0: flags=8843 metric 0 mtu 1500 options=209b ether 00:21:85:15:33:7d inet 192.168.1.101 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::221:85ff:fe15:337d%re0 prefixlen 64 tentative scopeid 0x1 nd6 options=29 media: Ethernet autoselect (none) status: no carrier fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb nd6 options=29 ch 1 dma -1 fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 nd6 options=29 Starting devd. Starting Network: usbus0. Starting Network: usbus1. Starting Network: usbus2. Starting Network: usbus3. Starting Network: usbus4. Starting Network: usbus5. Starting Network: fwe0. fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb nd6 options=29 ch 1 dma -1 Starting Network: fwip0. fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 nd6 options=29 add net default: gateway 192.168.1.1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 re0: link state changed to UP ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/kde4/lib /usr/local/lib /usr/local/lib/compat /usr/local/lib/compat/pkg /usr/local/lib/nss /usr/local/lib/qt4 /usr/local/lib/virtualbox /usr/local/lib/wine a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout Creating and/or trimming log files. Starting syslogd. savecore: rebootlock order reversal: Feb 11 14:07:2 1st 0xe19193c0 bufwait (bufwait) @ kern/vfs_bio.c:2658 3 x4 savecore: r 2nd 0xc8192800 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:284 Keboot DB: stack backtrace: db_trace_self_wrapper(c0f90c33,2e687361,38323a63,a34,f27cc74c,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a9561b,c0f946c0,c1273390,11c,f27cc7dc,...) at kdb_backtrace+0x2a _witness_debugger(c0f946c0,c8192800,c0fc475a,c7d72678,c0fc43ec,...) at _witness_debugger+0x25 witness_checkorder(c8192800,9,c0fc43e3,11c,0,...) at witness_checkorder+0x86f _sx_xlock(c8192800,0,c0fc43e3,11c,cacac984,...) at _sx_xlock+0xa0 ufsdirhash_acquire(e1919360,cacac984,f27cc918,e232ca98,f27cc8ac,...) at ufsdirhash_acquire+0x48 ufsdirhash_add(cacac984,f27cc918,a98,f27cc898,f27cc89c,...) at ufsdirhash_add+0x13 ufs_direnter(cacadae0,cacab2b8,f27cc918,f27ccbc0,0,...) at ufs_direnter+0x7a9 ufs_makeinode(f27ccbc0,c10dcce0,f27ccb04,f27cca60,c0de3d75,...) at ufs_makeinode+0x65d ufs_create(f27ccb04,c0fe5b8b,0,0,f27ccb80,...) at ufs_create+0x30 VOP_CREATE_APV(c10dc760,f27ccb04,f27ccbc0,f27cca9c,0,...) at VOP_CREATE_APV+0xc5 vn_open_cred(f27ccb80,f27ccc48,180,0,c7db1e80,...) at vn_open_cred+0x205 vn_open(f27ccb80,f27ccc48,180,ca7c9d20,c0a4b1e1,...) at vn_open+0x3b kern_openat(cab02000,ffffff9c,28450000,0,602,...) at kern_openat+0x1ec kern_open(cab02000,28450000,0,601,180,...) at kern_open+0x35 sys_open(cab02000,f27cccec,c0fde84a,c0f9549b,282,...) at sys_open+0x30 syscall(f27ccd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (5, FreeBSD ELF32, sys_open), eip = 0x2819dda3, esp = 0xbfbfe2ac, ebp = 0xbfbfedb8 --- savecore: writing core to vmcore.276 Writing crash summary to /var/crash/core.txt.276. Additional ABI support: linux. Starting rpcbind. NFS access cache time=60 Clearing /tmp (X related). Starting mountd. Feb 11 14:07:46 x4 mountd[1897]: can't delete exports for /dev: Invalid argument mount option is unknown NFSv4 is disabled Starting nfsd. Recovering vi editor sessions:. Updating motd:. Starting ntpd. Starting default moused. Configuring syscons: keymap blanktime. Starting sshd. Starting cron. Local package initialization: backuplock order reversal: 1st 0xcaa665c8 ufs (ufs) @ kern/vfs_mount.c:1249 2nd 0xca819b38 syncer (syncer) @ kern/vfs_subr.c:2279 KDB: stack backtrace: db_trace_self_wrapper(c0f90c33,a3937,cccccccc,ccccccc,7fff,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a9561b,c0f946c0,c1272d50,8e7,f285d9dc,...) at kdb_backtrace+0x2a _witness_debugger(c0f946c0,ca819b38,c0f9d224,c7d726e0,c0f9c675,...) at _witness_debugger+0x25 witness_checkorder(ca819b38,9,c0f9c66c,8e7,0,...) at witness_checkorder+0x86f __lockmgr_args(ca819b38,80100,ca819ba4,0,0,...) at __lockmgr_args+0x834 vop_stdlock(f285dafc,c0aa57da,c1272d54,80100,ca819ae0,...) at vop_stdlock+0x65 VOP_LOCK1_APV(c10cb120,f285dafc,c0a4b63c,c10ed780,ca819ae0,...) at VOP_LOCK1_APV+0xb5 _vn_lock(ca819ae0,80100,c0f9c66c,8e7,c12a06a0,...) at _vn_lock+0x78 vputx(f285db80,c0afddbd,ca819ae0,0,c0f9c66c,...) at vputx+0x266 vrele(ca819ae0,0,c0f9c66c,db7,caa66570,...) at vrele+0x10 vfs_deallocate_syncvnode(ca72587c,0,c0f9bccf,51a,c7c30000,...) at vfs_deallocate_syncvnode+0x6d dounmount(ca72587c,8080000,cac00b80,49c,0,...) at dounmount+0x376 sys_unmount(cac00b80,f285dcec,c,c,206,...) at sys_unmount+0x421 syscall(f285dd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280e7e0b, esp = 0xbfbfe64c, ebp = 0xbfbfe718 --- lock order reversal: 1st 0xca819c94 ufs (ufs) @ kern/vfs_subr.c:2167 2nd 0xe1a542e0 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:262 3rd 0xcac9cdf0 ufs (ufs) @ kern/vfs_subr.c:2167 KDB: stack backtrace: db_trace_self_wrapper(c0f90c33,3a632e73,a323632,c0aa5a00,c0f94ab6,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a9561b,c0f946d9,c1272d60,877,f285d838,...) at kdb_backtrace+0x2a _witness_debugger(c0f946d9,cac9cdf0,c0f8322c,c7d72610,c0f9c675,...) at _witness_debugger+0x25 witness_checkorder(cac9cdf0,9,c0f9c66c,877,0,...) at witness_checkorder+0x86f __lockmgr_args(cac9cdf0,80100,cac9ce5c,0,0,...) at __lockmgr_args+0x834 ffs_lock(f285d95c,f285d95c,0,80100,cac9cd98,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c10dc760,f285d95c,c7d6b1e0,c10ed780,cac9cd98,...) at VOP_LOCK1_APV+0xb5 _vn_lock(cac9cd98,80100,c0f9c66c,877,4,...) at _vn_lock+0x78 vget(cac9cd98,80100,cac00b80,50,0,...) at vget+0xbb vfs_hash_get(ca72587c,15dac00,80000,cac00b80,f285daa0,...) at vfs_hash_get+0xed ffs_vgetf(ca72587c,15dac00,80000,f285daa0,1,...) at ffs_vgetf+0x49 softdep_sync_buf(ca819c3c,e1a54280,1,108,0,...) at softdep_sync_buf+0x4a3 ffs_syncvnode(ca819c3c,1,cac00b80,5df,0,...) at ffs_syncvnode+0x24c ffs_sync(ca72587c,1,f285dbc0,51a,c7c30000,...) at ffs_sync+0x29f dounmount(ca72587c,8080000,cac00b80,49c,0,...) at dounmount+0x3f8 sys_unmount(cac00b80,f285dcec,c,c,206,...) at sys_unmount+0x421 syscall(f285dd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280e7e0b, esp = 0xbfbfe64c, ebp = 0xbfbfe718 --- lock order reversal: 1st 0xcaa665c8 ufs (ufs) @ kern/vfs_mount.c:1249 2nd 0xca818c94 devfs (devfs) @ ufs/ffs/ffs_softdep.c:1835 KDB: stack backtrace: db_trace_self_wrapper(c0f90c33,38313a63,a3533,c0aa7c0c,8,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a9561b,c0f946c0,c1272d40,72b,f285d9a0,...) at kdb_backtrace+0x2a _witness_debugger(c0f946c0,ca818c94,c0f7b167,c7d72470,c0fc0859,...) at _witness_debugger+0x25 witness_checkorder(ca818c94,9,c0fc0850,72b,ca818d00,...) at witness_checkorder+0x86f __lockmgr_args(ca818c94,80400,ca818d00,0,0,...) at __lockmgr_args+0x834 vop_stdlock(f285dac0,4,c0f8b00b,80400,ca818c3c,...) at vop_stdlock+0x65 VOP_LOCK1_APV(c10b23e0,f285dac0,c12a6960,c10ed780,ca818c3c,...) at VOP_LOCK1_APV+0xb5 _vn_lock(ca818c3c,80400,c0fc0850,72b,ca72587c,...) at _vn_lock+0x78 softdep_flushworklist(ca72587c,f285db6c,cac00b80,5ed,0,...) at softdep_flushworklist+0x47 ffs_sync(ca72587c,1,f285dbc0,51a,c7c30000,...) at ffs_sync+0x32d dounmount(ca72587c,8080000,cac00b80,49c,0,...) at dounmount+0x3f8 sys_unmount(cac00b80,f285dcec,c,c,206,...) at sys_unmount+0x421 syscall(f285dd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280e7e0b, esp = 0xbfbfe64c, ebp = 0xbfbfe718 --- ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 100708 files, 3682309 used, 114631672 free (2000 frags, 14328709 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 100708 files, 3682309 used, 114631672 free (2000 frags, 14328709 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** fsck -y /tmpFeb 11 14:10:53 x4 mountd[1898]: can't delete exports for /dev: Invalid argument mount option is unknown watchdogd. Starting inetd. Sat Feb 11 14:10:53 CET 2012 FreeBSD/i386 (x4.osted.lan) (console) login: Feb 11 14:25:01 x4 su: pho to root on /dev/pts/0 Feb 11 14:25:56 x4 mountd[1898]: can't delete exports for /dev: Invalid argument mount option is unknown lock order reversal: 1st 0xe1b11300 bufwait (bufwait) @ kern/vfs_bio.c:2658 2nd 0xcb32321c snaplk (snaplk) @ ufs/ffs/ffs_snapshot.c:2293 KDB: stack backtrace: db_trace_self_wrapper(c0f90c33,632e746f,3932323a,1c000a33,5831df76,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a9561b,c0f946c0,c12729a8,8f5,f28903ec,...) at kdb_backtrace+0x2a _witness_debugger(c0f946c0,cb32321c,c0fc02f5,c7d72dc8,c0fc027f,...) at _witness_debugger+0x25 witness_checkorder(cb32321c,9,c0fc0276,8f5,0,...) at witness_checkorder+0x86f __lockmgr_args(cb32321c,80900,cb3e8220,0,0,...) at __lockmgr_args+0x834 ffs_copyonwrite(cb3e815c,e1b112a0,e1b112a0,0,0,...) at ffs_copyonwrite+0x2c2 ffs_geom_strategy(cb3e8254,e1b112a0,0,e1b112a0,60000,...) at ffs_geom_strategy+0xeb bufwrite(e1b112a0,0,c0fc3730,7e2,0) at bufwrite+0x159 ffs_bufwrite(e1b112a0,cb350d00,100,8000,0,...) at ffs_bufwrite+0x290 ffs_update(cb34c414,1,cb34c50c,0,0,...) at ffs_update+0x2dc softdep_sync_buf(cb2d2000,e1b08880,1,108,0,...) at softdep_sync_buf+0x86f ffs_syncvnode(cb2d2000,1,c0a4ae0d,c1146b18,4,...) at ffs_syncvnode+0x24c ffs_truncate(cb2d2000,400,0,880,ca7c5780,...) at ffs_truncate+0x827 ufs_direnter(cb2d2000,cb34c414,f28909f4,f2890b84,e1b115e0,...) at ufs_direnter+0x9a1 ufs_mkdir(f2890c14,c0fe59c5,0,0,f2890bac,...) at ufs_mkdir+0xa6f VOP_MKDIR_APV(c10dc760,f2890c14,f2890b84,f2890bac,0,...) at VOP_MKDIR_APV+0xc5 kern_mkdirat(cb29e5c0,ffffff9c,bfbfef49,0,1f8,...) at kern_mkdirat+0x291 kern_mkdir(cb29e5c0,bfbfef49,0,1f8,f2890d1c,...) at kern_mkdir+0x2e sys_mkdir(cb29e5c0,f2890cec,c0fde84a,c0f9539e,257,...) at sys_mkdir+0x29 syscall(f2890d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28194743, esp = 0xbfbfe78c, ebp = 0xbfbfe828 --- panic: check_inode_unwritten: busy inode cpuid = 0 KDB: enter: panic [ thread pid 2428 tid 100143 ] Stopped at kdb_enter+0x3a: movl $0,kdb_why db> run pho db:0:pho> bt Tracing pid 2428 tid 100143 td 0xcac00000 kdb_enter(c0f8cbd8,c0f8cbd8,c0fc0b28,f2869778,0,...) at kdb_enter+0x3a panic(c0fc0b28,4,c0fc0850,1c18,cd867700,...) at panic+0x1ac check_inode_unwritten(f28697ec,0,c0fc0850,1bdb,cb7b1828,...) at check_inode_unwritten+0x169 softdep_freefile(cb1ec2b8,c50,81b0,8000,cb7b1828,...) at softdep_freefile+0x8bc ffs_vfree(cb1ec2b8,c50,81b0,f2869914,1,...) at ffs_vfree+0x3f ffs_valloc(cb1ec2b8,81b0,ca7c5780,f2869914,0,...) at ffs_valloc+0x548 ufs_makeinode(f2869bc0,c10dcce0,f2869b04,f2869a60,c0de3d75,...) at ufs_makeinode+0xc3 ufs_create(f2869b04,c0fe5b8b,0,0,f2869b80,...) at ufs_create+0x30 VOP_CREATE_APV(c10dc760,f2869b04,f2869bc0,f2869a9c,0,...) at VOP_CREATE_APV+0xc5 vn_open_cred(f2869b80,f2869c48,1b0,0,ca7c5780,...) at vn_open_cred+0x205 vn_open(f2869b80,f2869c48,1b0,cac91af0,c1ba6010,...) at vn_open+0x3b kern_openat(cac00000,ffffff9c,bfbfe758,0,602,...) at kern_openat+0x1ec kern_open(cac00000,bfbfe758,0,601,1b0,...) at kern_open+0x35 sys_open(cac00000,f2869cec,c0fde84a,c0f7d6d9,206,...) at sys_open+0x30 syscall(f2869d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (5, FreeBSD ELF32, sys_open), eip = 0x2819cda3, esp = 0xbfbfe70c, ebp = 0xbfbfe728 --- db:0:bt> show allpcpu Current CPU: 0 cpuid = 0 dynamic pcpu = 0x737080 curthread = 0xcac00000: pid 2428 "creat" curpcb = 0xf2869d80 fpcurthread = none idlethread = 0xc7dd28a0: tid 100003 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: cpuid = 1 dynamic pcpu = 0x6a28080 curthread = 0xcab025c0: pid 2400 "md5" curpcb = 0xf2767d80 fpcurthread = none idlethread = 0xc7dd25c0: tid 100004 "idle: cpu1" APIC ID = 1 currentldt = 0x50 spin locks held: cpuid = 2 dynamic pcpu = 0x6a2b080 curthread = 0xcb29bb80: pid 2431 "swap" curpcb = 0xf28a8d80 fpcurthread = none idlethread = 0xc7dd22e0: tid 100005 "idle: cpu2" APIC ID = 2 currentldt = 0x50 spin locks held: cpuid = 3 dynamic pcpu = 0x6a2e080 curthread = 0xc818b5c0: pid 2430 "swap" curpcb = 0xf288ad80 fpcurthread = none idlethread = 0xc7dd2000: tid 100006 "idle: cpu3" APIC ID = 3 currentldt = 0x50 spin locks held: db:0:allpcpu> show alllocks Process 2443 (umount) thread 0xcd87c2e0 (100174) exclusive lockmgr bufwait (bufwait) r = 0 (0xe1b08260) locked @ kern/vfs_bio.c:2658 exclusive lockmgr snaplk (snaplk) r = 0 (0xcb32321c) locked @ kern/vfs_subr.c:2167 exclusive lockmgr ufs (ufs) r = 0 (0xcaf39724) locked @ kern/vfs_mount.c:1249 Process 2428 (creat) thread 0xcac00000 (100143) exclusive sleep mutex Softdep Lock (Softdep Lock) r = 0 (0xc12a6960) locked @ ufs/ffs/ffs_softdep.c:7131 exclusive lockmgr ufs (ufs) r = 0 (0xcb7b1880) locked @ ufs/ffs/ffs_vfsops.c:1643 exclusive lockmgr ufs (ufs) r = 0 (0xcb1ec310) locked @ kern/vfs_lookup.c:542 Process 2427 (creat) thread 0xcb29d8a0 (100160) exclusive lockmgr bufwait (bufwait) r = 0 (0xe1b12b60) locked @ ufs/ffs/ffs_vnops.c:262 exclusive lockmgr ufs (ufs) r = 0 (0xcb3e7c94) locked @ kern/vfs_lookup.c:542 Process 2400 (md5) thread 0xcab025c0 (100102) exclusive sleep mutex vm object (standard object) r = 0 (0xcb2b1480) locked @ dev/md/md.c:674 db:0:alllocks> show lockedvnods Locked vnodes 0xcaf396cc: tag ufs, type VDIR usecount 1, writecount 0, refcount 1 mountedhere 0xcaa69000 flags () v_object 0xcabd7630 ref 0 pages 0 lock type ufs: EXCL by thread 0xcd87c2e0 (pid 2443) #0 0xc0a432ce at __lockmgr_args+0xc1e #1 0xc0ca4cd1 at ffs_lock+0xa1 #2 0xc0de2da5 at VOP_LOCK1_APV+0xb5 #3 0xc0b09158 at _vn_lock+0x78 #4 0xc0af199c at dounmount+0x8c #5 0xc0af23f1 at sys_unmount+0x421 #6 0xc0dbfa47 at syscall+0x2c7 #7 0xc0da8d71 at Xint0x80_syscall+0x21 ino 235521, on dev ada0s1a 0xcb34cc3c: tag ufs, type VREG usecount 3, writecount 0, refcount 562 mountedhere 0 flags (VV_SYSTEM|VV_MD) v_object 0xcb2dc2d0 ref 0 pages 4288 lock type snaplk: EXCL by thread 0xcd87c2e0 (pid 2443) #0 0xc0a432ce at __lockmgr_args+0xc1e #1 0xc0ca4cd1 at ffs_lock+0xa1 #2 0xc0de2da5 at VOP_LOCK1_APV+0xb5 #3 0xc0b09158 at _vn_lock+0x78 #4 0xc0afbc0b at vget+0xbb #5 0xc0c9ef46 at ffs_sync+0x246 #6 0xc0af1d08 at dounmount+0x3f8 #7 0xc0af23f1 at sys_unmount+0x421 #8 0xc0dbfa47 at syscall+0x2c7 #9 0xc0da8d71 at Xint0x80_syscall+0x21 ino 5, on dev md5a 0xcb3e7c3c: tag ufs, type VDIR usecount 2, writecount 0, refcount 3 mountedhere 0 flags () v_object 0xcc13d990 ref 0 pages 0 lock type ufs: EXCL by thread 0xcb29d8a0 (pid 2427) #0 0xc0a42c74 at __lockmgr_args+0x5c4 #1 0xc0ca4cd1 at ffs_lock+0xa1 #2 0xc0de2da5 at VOP_LOCK1_APV+0xb5 #3 0xc0b09158 at _vn_lock+0x78 #4 0xc0aec496 at lookup+0x106 #5 0xc0aeda9f at namei+0x6bf #6 0xc0b086a0 at vn_open_cred+0x90 #7 0xc0b08c2b at vn_open+0x3b #8 0xc0b04e7c at kern_openat+0x1ec #9 0xc0b05325 at kern_open+0x35 #10 0xc0b05360 at sys_open+0x30 #11 0xc0dbfa47 at syscall+0x2c7 #12 0xc0da8d71 at Xint0x80_syscall+0x21 ino 196609, on dev md5a 0xcb1ec2b8: tag ufs, type VDIR usecount 2, writecount 0, refcount 3 mountedhere 0 flags () v_object 0xcb3fbd80 ref 0 pages 0 lock type ufs: EXCL by thread 0xcac00000 (pid 2428) #0 0xc0a42c74 at __lockmgr_args+0x5c4 #1 0xc0ca4cd1 at ffs_lock+0xa1 #2 0xc0de2da5 at VOP_LOCK1_APV+0xb5 #3 0xc0b09158 at _vn_lock+0x78 #4 0xc0aec496 at lookup+0x106 #5 0xc0aeda9f at namei+0x6bf #6 0xc0b086a0 at vn_open_cred+0x90 #7 0xc0b08c2b at vn_open+0x3b #8 0xc0b04e7c at kern_openat+0x1ec #9 0xc0b05325 at kern_open+0x35 #10 0xc0b05360 at sys_open+0x30 #11 0xc0dbfa47 at syscall+0x2c7 #12 0xc0da8d71 at Xint0x80_syscall+0x21 ino 196610, on dev md5a 0xcb7b1828: tag ufs, type VNON usecount 1, writecount 0, refcount 1 mountedhere 0 flags () lock type ufs: EXCL by thread 0xcac00000 (pid 2428) #0 0xc0a432ce at __lockmgr_args+0xc1e #1 0xc0c9e159 at ffs_vgetf+0x189 #2 0xc0c71d00 at ffs_valloc+0x530 #3 0xc0cb4cd3 at ufs_makeinode+0xc3 #4 0xc0cb5590 at ufs_create+0x30 #5 0xc0de3d75 at VOP_CREATE_APV+0xc5 #6 0xc0b08815 at vn_open_cred+0x205 #7 0xc0b08c2b at vn_open+0x3b #8 0xc0b04e7c at kern_openat+0x1ec #9 0xc0b05325 at kern_open+0x35 #10 0xc0b05360 at sys_open+0x30 #11 0xc0dbfa47 at syscall+0x2c7 #12 0xc0da8d71 at Xint0x80_syscall+0x21 ino 3152, on dev md5a db:0:lockedvnods> show mount 0xca725b50 /dev/ad4s1a on / (ufs) 0xca726000 devfs on /dev (devfs) 0xcaa695a8 /dev/ad4s1f on /home (ufs) 0xca7255a8 /dev/ad4s1d on /usr (ufs) 0xca7252d4 /dev/ad4s1e on /var (ufs) 0xca725000 procfs on /proc (procfs) 0xcaae1000 linprocfs on /usr/compat/linux/proc (linprocfs) 0xcaa692d4 /dev/label/tmp on /tmp (ufs) 0xcaa69000 /dev/md5a on /mnt (ufs) 0xca726b50 /dev/md6 on /mnt2 (ufs) More info: show mount db:0:mount> ps pid ppid pgrp uid state wmesg wchan cmd 2443 2433 2433 0 D+ biowr 0xe1b08200 umount 2433 2380 2433 0 S+ wait 0xcb42c84c sh 2431 2429 2411 1004 R+ CPU 2 swap 2430 2429 2411 1004 R+ CPU 3 swap 2429 2425 2411 1004 S+ wait 0xcb1e684c swap 2428 2426 2411 1004 R+ CPU 0 creat 2427 2426 2411 1004 R+ creat 2426 2423 2411 1004 S+ wait 0xcb1e5000 creat 2425 2422 2411 1004 S+ nanslp 0xc112a4e4 swap 2423 2422 2411 1004 S+ nanslp 0xc112a4e4 creat 2422 2421 2411 1004 S+ wait 0xcaead84c run 2421 2411 2411 1004 S+ wait 0xcabfe000 run 2417 0 0 0 DL mdwait 0xcb2a5800 [md6] 2411 2405 2411 1004 S+ nanslp 0xc112a4e4 run 2405 1 2387 0 S+ wait 0xca6be84c su 2400 0 0 0 RL CPU 1 [md5] 2380 2376 2380 0 S+ wait 0xcaae22c4 bash 2376 2373 2376 0 S+ pause 0xcb1e631c csh 2373 2219 2373 1001 S+ wait 0xcaae32c4 su 2308 1 2308 0 Ss+ ttyin 0xc7efda70 getty 2307 1 2307 0 Ss+ ttyin 0xc7efdc70 getty 2306 1 2306 0 Ss+ ttyin 0xc7efde70 getty 2305 1 2305 0 Ss+ ttyin 0xc8192070 getty 2304 1 2304 0 Ss+ ttyin 0xc8192270 getty 2303 1 2303 0 Ss+ ttyin 0xc8192470 getty 2302 1 2302 0 Ss+ ttyin 0xc7efc670 getty 2301 1 2301 0 Ss+ ttyin 0xc7efc470 getty 2300 1 2300 0 Ss+ ttyin 0xc7efc070 getty 2261 1 2261 0 Ss select 0xca7285a4 inetd 2229 1 2229 0 Ss nanslp 0xc112a4e4 watchdogd 2219 2218 2219 1001 Ss+ wait 0xcabfe2c4 bash 2218 2216 2216 1001 S select 0xca7c6064 sshd 2216 2151 2216 0 Ss select 0xca6b3ba4 sshd 2173 1 2173 0 Ss nanslp 0xc112a4e4 cron 2165 1 2165 25 Ss pause 0xcaae2b68 sendmail 2159 1 2159 0 Ss select 0xcaf3d724 sendmail 2151 1 2151 0 Ss select 0xcaf3d3a4 sshd 2091 1 2091 0 Ss select 0xca7c6da4 moused 2013 1 2013 0 Ss select 0xcaf46524 ntpd 1914 1908 1908 0 S (threaded) nfsd 100148 S rpcsvc 0xca60fc90 nfsd: service 100147 S rpcsvc 0xca6c8a90 nfsd: service 100146 S rpcsvc 0xca60fb90 nfsd: service 100129 S rpcsvc 0xca60fd10 nfsd: master 1908 1 1908 0 Ss select 0xcaf46564 nfsd 1898 1 1898 0 Ss select 0xca6b3be4 mountd 1799 1 1799 0 Ss select 0xca728224 rpcbind 1717 1 1717 0 Ss select 0xca7280a4 syslogd 1510 1 1510 0 Ss select 0xca6b2c24 devd 19 0 0 0 DL sdflush 0xc12a69a0 [softdepflush] 18 0 0 0 DL vlruwt 0xca6bc2c4 [vnlru] 17 0 0 0 DL syncer 0xc12a06b4 [syncer] 16 0 0 0 DL psleep 0xc12a0348 [bufdaemon] 9 0 0 0 DL pgzero 0xc12a885c [pagezero] 8 0 0 0 DL psleep 0xc12a8448 [vmdaemon] 7 0 0 0 DL psleep 0xc12a8410 [pagedaemon] 6 0 0 0 DL ccb_scan 0xc10eecd4 [xpt_thrd] 5 0 0 0 DL waiting_ 0xc12a21d8 [sctp_iterator] 4 0 0 0 DL ctl_work 0xc819a000 [ctl_thrd] 3 0 0 0 DL - 0xc817a23c [fdc0] 2 0 0 0 SL - 0xc8090000 [fw0_probe] 15 0 0 0 DL (threaded) [usb] 100058 D - 0xc8066d34 [usbus5] 100057 D - 0xc8066d04 [usbus5] 100056 D - 0xc8066cd4 [usbus5] 100055 D - 0xc8066ca4 [usbus5] 100053 D - 0xc8058b5c [usbus4] 100052 D - 0xc8058b2c [usbus4] 100051 D - 0xc8058afc [usbus4] 100050 D - 0xc8058acc [usbus4] 100049 D - 0xc8054b5c [usbus3] 100048 D - 0xc8054b2c [usbus3] 100047 D - 0xc8054afc [usbus3] 100046 D - 0xc8054acc [usbus3] 100045 D - 0xc804cb5c [usbus2] 100044 D - 0xc804cb2c [usbus2] 100043 D - 0xc804cafc [usbus2] 100042 D - 0xc804cacc [usbus2] 100040 D - 0xc8048b5c [usbus1] 100039 D - 0xc8048b2c [usbus1] 100038 D - 0xc8048afc [usbus1] 100037 D - 0xc8048acc [usbus1] 100035 D - 0xc803ab5c [usbus0] 100034 D - 0xc803ab2c [usbus0] 100033 D - 0xc803aafc [usbus0] 100032 D - 0xc803aacc [usbus0] 14 0 0 0 DL - 0xc11292e4 [yarrow] 13 0 0 0 DL (threaded) [geom] 100015 D - 0xc1126ee8 [g_down] 100014 D - 0xc1126ee4 [g_up] 100013 D - 0xc1126edc [g_event] 12 0 0 0 WL (threaded) [intr] 100066 I [irq12: psm0] 100065 I [irq1: atkbd0] 100063 I [swi0: uart] 100060 I [irq20: fwohci0] 100059 I [irq14: ata0] 100054 I [irq19: ehci0] 100041 I [irq18: ohci2 ohci4] 100036 I [irq17: ohci1 ohci3] 100031 I [irq16: hdac1 ohci0] 100030 I [irq22: ahci0] 100029 I [irq257: re0] 100028 I [irq256: hdac0] 100027 I [swi6: task queue] 100026 I [swi6: Giant taskq] 100024 I [swi5: +] 100018 I [swi2: cambio] 100012 I [swi3: vm] 100011 I [swi4: clock] 100010 I [swi4: clock] 100009 I [swi4: clock] 100008 I [swi4: clock] 100007 I [swi1: netisr 0] 11 0 0 0 RL (threaded) [idle] 100006 CanRun [idle: cpu3] 100005 CanRun [idle: cpu2] 100004 CanRun [idle: cpu1] 100003 CanRun [idle: cpu0] 1 0 1 0 SLs wait 0xc7dd0b10 [init] 10 0 0 0 DL audit_wo 0xc12a60e0 [audit] 0 0 0 0 DLs (threaded) [kernel] 100078 D - 0xc7ee4340 [mca taskq] 100068 D - 0xc11292e4 [deadlkres] 100061 D - 0xc7ee4640 [fw0_taskq] 100025 D - 0xc7ee4740 [thread taskq] 100023 D - 0xc7ee47c0 [acpi_task_2] 100022 D - 0xc7ee47c0 [acpi_task_1] 100021 D - 0xc7ee47c0 [acpi_task_0] 100020 D - 0xc7ee4800 [kqueue taskq] 100019 D - 0xc7ee4840 [ffs_trim taskq] 100016 D - 0xc7ee4880 [firmware taskq] 100000 D sched 0xc1126fc0 [swapper] 2424 2422 2411 1004 Z+ mkdir db:0:ps> allt Tracing command umount pid 2443 tid 100174 td 0xcd87c2e0 sched_switch(cd87c2e0,0,104,1a3,879ff1,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(cd87c2e0,0,c0f925e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(e1b08200,5c,c0f99f7a,0,0,...) at sleepq_wait+0x63 _sleep(e1b08200,c7d85484,5c,c0f99f7a,0,...) at _sleep+0x39c bwait(e1b08200,5c,c0f99f7a,e1b08200,f28c6a40,...) at bwait+0x6f bufwait(e1b08200,e1b08200,0,e1b08200,0,...) at bufwait+0x48 bufwrite(e1b08200,0,c0fc3730,7e2,0) at bufwrite+0x165 ffs_bufwrite(e1b08200,cb351400,100,8000,0,...) at ffs_bufwrite+0x290 ffs_update(cb34cc3c,1,c0fc3eb1,153,4,...) at ffs_update+0x2dc ffs_syncvnode(cb34cc3c,1,cd87c2e0,5df,0,...) at ffs_syncvnode+0x58f ffs_sync(caa69000,1,f28c6bc0,51a,c7c30000,...) at ffs_sync+0x29f dounmount(caa69000,8080000,cd87c2e0,49c,0,...) at dounmount+0x3f8 sys_unmount(cd87c2e0,f28c6cec,c,c,206,...) at sys_unmount+0x421 syscall(f28c6d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280e7e0b, esp = 0xbfbfe0dc, ebp = 0xbfbfe1a8 --- Tracing command sh pid 2433 tid 100166 td 0xcb29b5c0 sched_switch(cb29b5c0,0,104,1a3,ffd50a06,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(cb29b5c0,0,c0f925e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f925e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cb42c84c,6c,c0f954f7,100,0,...) at sleepq_wait_sig+0x17 _sleep(cb42c84c,cb42c8d4,16c,c0f954f7,0,...) at _sleep+0x385 kern_wait(cb29b5c0,ffffffff,f28aec60,2,0,...) at kern_wait+0x626 sys_wait4(cb29b5c0,f28aecec,c0fde84a,c0f954d6,216,...) at sys_wait4+0x3b syscall(f28aed28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2817645b, esp = 0xbfbfe80c, ebp = 0xbfbfe828 --- Tracing command swap pid 2431 tid 100164 td 0xcb29bb80 cpustop_handler(2,13,f28a8d1c,c0dbfe46,f5,...) at cpustop_handler+0x28 ipi_nmi_handler(f5,0,1024ab2,8049144,cb42d000,...) at ipi_nmi_handler+0x47 trap(f28a8d28) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0x804914a, esp = 0xbfbfe7d0, ebp = 0xbfbfe7e8 --- Tracing command swap pid 2430 tid 100154 td 0xc818b5c0 cpustop_handler(3,13,f288ad1c,c0dbfe46,f5,...) at cpustop_handler+0x28 ipi_nmi_handler(f5,0,1aa373e,8049144,cb1e6b10,...) at ipi_nmi_handler+0x47 trap(f288ad28) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0x8049151, esp = 0xbfbfe7d0, ebp = 0xbfbfe7e8 --- Tracing command swap pid 2429 tid 100155 td 0xc818b2e0 sched_switch(c818b2e0,0,104,1a3,3dd98caf,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(c818b2e0,0,c0f925e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f925e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cb1e684c,6c,c0f954f7,100,0,...) at sleepq_wait_sig+0x17 _sleep(cb1e684c,cb1e68d4,16c,c0f954f7,0,...) at _sleep+0x385 kern_wait(c818b2e0,97e,f288dc60,0,0,...) at kern_wait+0x626 sys_wait4(c818b2e0,f288dcec,c,c,216,...) at sys_wait4+0x3b syscall(f288dd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2811245b, esp = 0xbfbfe7cc, ebp = 0xbfbfe7e8 --- Tracing command creat pid 2428 tid 100143 td 0xcac00000 kdb_enter(c0f8cbd8,c0f8cbd8,c0fc0b28,f2869778,0,...) at kdb_enter+0x3a panic(c0fc0b28,4,c0fc0850,1c18,cd867700,...) at panic+0x1ac check_inode_unwritten(f28697ec,0,c0fc0850,1bdb,cb7b1828,...) at check_inode_unwritten+0x169 softdep_freefile(cb1ec2b8,c50,81b0,8000,cb7b1828,...) at softdep_freefile+0x8bc ffs_vfree(cb1ec2b8,c50,81b0,f2869914,1,...) at ffs_vfree+0x3f ffs_valloc(cb1ec2b8,81b0,ca7c5780,f2869914,0,...) at ffs_valloc+0x548 ufs_makeinode(f2869bc0,c10dcce0,f2869b04,f2869a60,c0de3d75,...) at ufs_makeinode+0xc3 ufs_create(f2869b04,c0fe5b8b,0,0,f2869b80,...) at ufs_create+0x30 VOP_CREATE_APV(c10dc760,f2869b04,f2869bc0,f2869a9c,0,...) at VOP_CREATE_APV+0xc5 vn_open_cred(f2869b80,f2869c48,1b0,0,ca7c5780,...) at vn_open_cred+0x205 vn_open(f2869b80,f2869c48,1b0,cac91af0,c1ba6010,...) at vn_open+0x3b kern_openat(cac00000,ffffff9c,bfbfe758,0,602,...) at kern_openat+0x1ec kern_open(cac00000,bfbfe758,0,601,1b0,...) at kern_open+0x35 sys_open(cac00000,f2869cec,c0fde84a,c0f7d6d9,206,...) at sys_open+0x30 syscall(f2869d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (5, FreeBSD ELF32, sys_open), eip = 0x2819cda3, esp = 0xbfbfe70c, ebp = 0xbfbfe728 --- Tracing command creat pid 2427 tid 100160 td 0xcb29d8a0 sched_switch(cb29d8a0,0,104,1a3,fe6792c0,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(cb29d8a0,0,c0f925e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(e1b12b00,5c,c0f99f7a,0,0,...) at sleepq_wait+0x63 _sleep(e1b12b00,c7d850c4,5c,c0f99f7a,0,...) at _sleep+0x39c bwait(e1b12b00,5c,c0f99f7a,e1b12b00,f289c5b0,...) at bwait+0x6f bufwait(e1b12b00,e1b12b00,cd840300,e1b12b60,e1b12b00,...) at bufwait+0x48 bufwrite(e1b12b00,e1b12b00,1,108,0,...) at bufwrite+0x165 ffs_syncvnode(cb3e7c3c,1,c1272710,cb3e7d34,c7d6e370,...) at ffs_syncvnode+0x385 ffs_truncate(cb3e7c3c,200,0,880,ca7c5780,...) at ffs_truncate+0x827 ufs_direnter(cb3e7c3c,cba516cc,f289c918,f289cbc0,0,...) at ufs_direnter+0x9a1 ufs_makeinode(f289cbc0,c10dcce0,f289cb04,f289ca60,c0de3d75,...) at ufs_makeinode+0x65d ufs_create(f289cb04,c0fe5b8b,0,0,f289cb80,...) at ufs_create+0x30 VOP_CREATE_APV(c10dc760,f289cb04,f289cbc0,f289ca9c,0,...) at VOP_CREATE_APV+0xc5 vn_open_cred(f289cb80,f289cc48,1b0,0,ca7c5780,...) at vn_open_cred+0x205 vn_open(f289cb80,f289cc48,1b0,cac74000,0,...) at vn_open+0x3b kern_openat(cb29d8a0,ffffff9c,bfbfe758,0,602,...) at kern_openat+0x1ec kern_open(cb29d8a0,bfbfe758,0,601,1b0,...) at kern_open+0x35 sys_open(cb29d8a0,f289ccec,c0fde84a,c0f96433,206,...) at sys_open+0x30 syscall(f289cd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (5, FreeBSD ELF32, sys_open), eip = 0x2819cda3, esp = 0xbfbfe70c, ebp = 0xbfbfe728 --- Tracing command creat pid 2426 tid 100163 td 0xcb29d000 sched_switch(cb29d000,0,104,1a3,3d0ad630,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(cb29d000,0,c0f925e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f925e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cb1e5000,6c,c0f954f7,100,0,...) at sleepq_wait_sig+0x17 _sleep(cb1e5000,cb1e5088,16c,c0f954f7,0,...) at _sleep+0x385 kern_wait(cb29d000,97b,f28a5c60,0,0,...) at kern_wait+0x626 sys_wait4(cb29d000,f28a5cec,c,c,216,...) at sys_wait4+0x3b syscall(f28a5d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2811245b, esp = 0xbfbfe7cc, ebp = 0xbfbfe7e8 --- Tracing command swap pid 2425 tid 100132 td 0xcac052e0 sched_switch(cac052e0,0,104,1a3,b1ec4a37,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(cac052e0,0,c0f925e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(3e9,c0a9be50,cac052e0,1,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c112a4e4,6c,c0f8e137,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c112a4e4,0,16c,c0f8e137,3e9,...) at _sleep+0x34f kern_nanosleep(cac052e0,f2848c50,f2848c58,1,0,...) at kern_nanosleep+0xc1 sys_nanosleep(cac052e0,f2848cec,c0fde84a,c0f959a4,202,...) at sys_nanosleep+0x6f syscall(f2848d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x28183e27, esp = 0xbfbfe7bc, ebp = 0xbfbfe7e8 --- Tracing command creat pid 2423 tid 100153 td 0xc818b8a0 sched_switch(c818b8a0,0,104,1a3,b1109d1a,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(c818b8a0,0,c0f925e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(3e9,c0a9be50,c818b8a0,1,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c112a4e4,6c,c0f8e137,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c112a4e4,0,16c,c0f8e137,3e9,...) at _sleep+0x34f kern_nanosleep(c818b8a0,f2887c50,f2887c58,1,0,...) at kern_nanosleep+0xc1 sys_nanosleep(c818b8a0,f2887cec,c0fde84a,c0f959a4,202,...) at sys_nanosleep+0x6f syscall(f2887d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x28183e27, esp = 0xbfbfe7bc, ebp = 0xbfbfe7e8 --- Tracing command run pid 2422 tid 100138 td 0xcac04000 sched_switch(cac04000,0,104,1a3,3b38339b,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(cac04000,0,c0f925e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f925e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(caead84c,6c,c0f954f7,100,0,...) at sleepq_wait_sig+0x17 _sleep(caead84c,caead8d4,16c,c0f954f7,0,...) at _sleep+0x385 kern_wait(cac04000,977,f285ac60,0,0,...) at kern_wait+0x626 sys_wait4(cac04000,f285acec,c,c,216,...) at sys_wait4+0x3b syscall(f285ad28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2811145b, esp = 0xbfbfe24c, ebp = 0xbfbfe268 --- Tracing command run pid 2421 tid 100114 td 0xcaa4bb80 sched_switch(caa4bb80,0,104,1a3,3985c88c,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(caa4bb80,0,c0f925e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f925e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cabfe000,6c,c0f954f7,100,0,...) at sleepq_wait_sig+0x17 _sleep(cabfe000,cabfe088,16c,c0f954f7,0,...) at _sleep+0x385 kern_wait(caa4bb80,976,f27e1c60,0,0,...) at kern_wait+0x626 sys_wait4(caa4bb80,f27e1cec,c,c,216,...) at sys_wait4+0x3b syscall(f27e1d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2811145b, esp = 0xbfbfe76c, ebp = 0xbfbfe788 --- Tracing command md6 pid 2417 tid 100127 td 0xcac7f2e0 sched_switch(cac7f2e0,0,104,1a3,639ead6,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(cac7f2e0,0,c0f925e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(cb2a5800,5c,c0f296e2,0,0,...) at sleepq_wait+0x63 _sleep(cb2a5800,cb2a5820,25c,c0f296e2,0,...) at _sleep+0x39c md_kthread(cb2a5800,f2820d28,c0f872f0,3db,cac78000,...) at md_kthread+0x11a fork_exit(c07696f0,cb2a5800,f2820d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xf2820d60, ebp = 0 --- Tracing command run pid 2411 tid 100156 td 0xcb29e5c0 sched_switch(cb29e5c0,0,104,1a3,ada1c169,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(cb29e5c0,0,c0f925e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(3e9,c0a9be50,cb29e5c0,1,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c112a4e4,6c,c0f8e137,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c112a4e4,0,16c,c0f8e137,3e9,...) at _sleep+0x34f kern_nanosleep(cb29e5c0,f2890c50,f2890c58,1,0,...) at kern_nanosleep+0xc1 sys_nanosleep(cb29e5c0,f2890cec,c0fde84a,c0f959a4,202,...) at sys_nanosleep+0x6f syscall(f2890d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x28182e27, esp = 0xbfbfe75c, ebp = 0xbfbfe788 --- Tracing command su pid 2405 tid 100085 td 0xca75c8a0 sched_switch(ca75c8a0,0,104,1a3,1d3bc6a,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(ca75c8a0,0,c0f925e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f925e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca6be84c,6c,c0f954f7,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca6be84c,ca6be8d4,16c,c0f954f7,0,...) at _sleep+0x385 kern_wait(ca75c8a0,96b,f2717c60,2,0,...) at kern_wait+0x626 sys_wait4(ca75c8a0,f2717cec,c,c,216,...) at sys_wait4+0x3b syscall(f2717d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2813245b, esp = 0xbfbfe34c, ebp = 0xbfbfe368 --- Tracing command md5 pid 2400 tid 100102 td 0xcab025c0 cpustop_handler(1,13,f2767c10,c0dbfe46,c0a818ed,...) at cpustop_handler+0x28 ipi_nmi_handler(c0a818ed,c1138078,cb29bb80,0,caae2000,...) at ipi_nmi_handler+0x47 trap(f2767c1c) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0dbd42e, esp = 0xf2767c5c, ebp = 0xf2767cb8 --- bcopy(cb2a6000,ca6f6688,c0f29635,2fe,0,...) at bcopy+0x1a md_kthread(cb2a6000,f2767d28,c0f872f0,3db,caae2000,...) at md_kthread+0x1c0 fork_exit(c07696f0,cb2a6000,f2767d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xf2767d60, ebp = 0 --- Tracing command bash pid 2380 tid 100101 td 0xcab028a0 sched_switch(cab028a0,0,104,1a3,fd46f0dd,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(cab028a0,0,c0f925e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f925e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(caae22c4,6c,c0f954f7,100,0,...) at sleepq_wait_sig+0x17 _sleep(caae22c4,caae234c,16c,c0f954f7,0,...) at _sleep+0x385 kern_wait(cab028a0,ffffffff,f2764c60,6,0,...) at kern_wait+0x626 sys_wait4(cab028a0,f2764cec,c0fde84a,c0f953e6,216,...) at sys_wait4+0x3b syscall(f2764d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x282cf45b, esp = 0xbfbfe34c, ebp = 0xbfbfe368 --- Tracing command csh pid 2376 tid 100157 td 0xcb29e2e0 sched_switch(cb29e2e0,0,104,1a3,4aef3416,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,74,...) at mi_switch+0x1ff sleepq_switch(cb29e2e0,0,c0f925e0,1a7,74,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f925e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cb1e631c,74,c0e13e2e,100,0,...) at sleepq_wait_sig+0x17 _sleep(cb1e631c,cb1e634c,174,c0e13e2e,0,...) at _sleep+0x385 kern_sigsuspend(cb29e2e0,0,0,0,0,...) at kern_sigsuspend+0xba sys_sigsuspend(cb29e2e0,f2893cec,c0fde84a,c0f953e6,202,...) at sys_sigsuspend+0x4d syscall(f2893d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28195ccb, esp = 0xbfbfe58c, ebp = 0xbfbfe5f8 --- Tracing command su pid 2373 tid 100096 td 0xca7592e0 sched_switch(ca7592e0,0,104,1a3,d061b2a5,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(ca7592e0,0,c0f925e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f925e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(caae32c4,6c,c0f954f7,100,0,...) at sleepq_wait_sig+0x17 _sleep(caae32c4,caae334c,16c,c0f954f7,0,...) at _sleep+0x385 kern_wait(ca7592e0,948,f2755c60,2,0,...) at kern_wait+0x626 sys_wait4(ca7592e0,f2755cec,c,c,216,...) at sys_wait4+0x3b syscall(f2755d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2813245b, esp = 0xbfbfe5bc, ebp = 0xbfbfe5d8 --- Tracing command getty pid 2308 tid 100130 td 0xcac7e8a0 sched_switch(cac7e8a0,0,104,1a3,d4728e29,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cac7e8a0,0,c0f925e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f925e0,160,0,cac7e8a0,cac7e8a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7efda70,0,c0f97418,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7efda70,c1128b90,c0f96df0,527,0,...) at _cv_wait_sig+0x260 tty_wait(c7efda00,c7efda70,f2829c44,1,0,...) at tty_wait+0x71 ttydisc_read(c7efda00,f2829c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c7fa6500,f2829c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(ca76a8f8,f2829c44,c7db1e80,0,cac7e8a0,...) at devfs_read_f+0x7e dofileread(f2829c44,ffffffff,ffffffff,0,f2829c20,...) at dofileread+0x9e kern_readv(cac7e8a0,0,f2829c44,f2829c64,1,...) at kern_readv+0x68 sys_read(cac7e8a0,f2829cec,c0fde84a,c0df22f6,246,...) at sys_read+0x4f syscall(f2829d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a6333, esp = 0xbfbfed7c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2307 tid 100112 td 0xcab012e0 sched_switch(cab012e0,0,104,1a3,d467e7b3,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cab012e0,0,c0f925e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f925e0,160,0,cab012e0,cab012e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7efdc70,0,c0f97418,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7efdc70,c1128b90,c0f96df0,527,0,...) at _cv_wait_sig+0x260 tty_wait(c7efdc00,c7efdc70,f27dbc44,1,0,...) at tty_wait+0x71 ttydisc_read(c7efdc00,f27dbc44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c7fa6600,f27dbc44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(cabe4578,f27dbc44,c7db1e80,0,cab012e0,...) at devfs_read_f+0x7e dofileread(f27dbc44,ffffffff,ffffffff,0,f27dbc20,...) at dofileread+0x9e kern_readv(cab012e0,0,f27dbc44,f27dbc64,1,...) at kern_readv+0x68 sys_read(cab012e0,f27dbcec,c0fde84a,c0df22f6,246,...) at sys_read+0x4f syscall(f27dbd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a6333, esp = 0xbfbfed7c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2306 tid 100149 td 0xca6bf5c0 sched_switch(ca6bf5c0,0,104,1a3,d474d795,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca6bf5c0,0,c0f925e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f925e0,160,0,ca6bf5c0,ca6bf5c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7efde70,0,c0f97418,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7efde70,c1128b90,c0f96df0,527,0,...) at _cv_wait_sig+0x260 tty_wait(c7efde00,c7efde70,f287bc44,1,0,...) at tty_wait+0x71 ttydisc_read(c7efde00,f287bc44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c7fa6700,f287bc44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(ca76b0a8,f287bc44,c7db1e80,0,ca6bf5c0,...) at devfs_read_f+0x7e dofileread(f287bc44,ffffffff,ffffffff,0,f287bc20,...) at dofileread+0x9e kern_readv(ca6bf5c0,0,f287bc44,f287bc64,1,...) at kern_readv+0x68 sys_read(ca6bf5c0,f287bcec,c0fde84a,c0df22f6,246,...) at sys_read+0x4f syscall(f287bd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a6333, esp = 0xbfbfed7c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2305 tid 100117 td 0xcaa4b2e0 sched_switch(caa4b2e0,0,104,1a3,d46431cc,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(caa4b2e0,0,c0f925e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f925e0,160,0,caa4b2e0,caa4b2e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8192070,0,c0f97418,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c8192070,c1128b90,c0f96df0,527,0,...) at _cv_wait_sig+0x260 tty_wait(c8192000,c8192070,f27eac44,1,0,...) at tty_wait+0x71 ttydisc_read(c8192000,f27eac44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c7fa6800,f27eac44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(ca76a0e0,f27eac44,c7db1e80,0,caa4b2e0,...) at devfs_read_f+0x7e dofileread(f27eac44,ffffffff,ffffffff,0,f27eac20,...) at dofileread+0x9e kern_readv(caa4b2e0,0,f27eac44,f27eac64,1,...) at kern_readv+0x68 sys_read(caa4b2e0,f27eacec,c0fde84a,c0df22f6,246,...) at sys_read+0x4f syscall(f27ead28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a6333, esp = 0xbfbfed7c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2304 tid 100150 td 0xca6bf2e0 sched_switch(ca6bf2e0,0,104,1a3,d46d5da0,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca6bf2e0,0,c0f925e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f925e0,160,0,ca6bf2e0,ca6bf2e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8192270,0,c0f97418,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c8192270,c1128b90,c0f96df0,527,0,...) at _cv_wait_sig+0x260 tty_wait(c8192200,c8192270,f287ec44,1,0,...) at tty_wait+0x71 ttydisc_read(c8192200,f287ec44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c7fa6900,f287ec44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(cac745e8,f287ec44,c7db1e80,0,ca6bf2e0,...) at devfs_read_f+0x7e dofileread(f287ec44,ffffffff,ffffffff,0,f287ec20,...) at dofileread+0x9e kern_readv(ca6bf2e0,0,f287ec44,f287ec64,1,...) at kern_readv+0x68 sys_read(ca6bf2e0,f287ecec,c0fde84a,c0df22f6,246,...) at sys_read+0x4f syscall(f287ed28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a6333, esp = 0xbfbfed7c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2303 tid 100139 td 0xcac00b80 sched_switch(cac00b80,0,104,1a3,d466beb6,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cac00b80,0,c0f925e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f925e0,160,0,cac00b80,cac00b80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8192470,0,c0f97418,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c8192470,c1128b90,c0f96df0,527,0,...) at _cv_wait_sig+0x260 tty_wait(c8192400,c8192470,f285dc44,1,0,...) at tty_wait+0x71 ttydisc_read(c8192400,f285dc44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c7fa6a00,f285dc44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(cac08578,f285dc44,c7db1e80,0,cac00b80,...) at devfs_read_f+0x7e dofileread(f285dc44,ffffffff,ffffffff,0,f285dc20,...) at dofileread+0x9e kern_readv(cac00b80,0,f285dc44,f285dc64,1,...) at kern_readv+0x68 sys_read(cac00b80,f285dcec,c0fde84a,c0df22f6,246,...) at sys_read+0x4f syscall(f285dd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a6333, esp = 0xbfbfed7c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2302 tid 100109 td 0xcaa4a000 sched_switch(caa4a000,0,104,1a3,d47ceb67,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(caa4a000,0,c0f925e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f925e0,160,0,caa4a000,caa4a000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7efc670,0,c0f97418,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7efc670,c1128b90,c0f96df0,527,0,...) at _cv_wait_sig+0x260 tty_wait(c7efc600,c7efc670,f27d2c44,1,0,...) at tty_wait+0x71 ttydisc_read(c7efc600,f27d2c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c7fa6b00,f27d2c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(ca7c9578,f27d2c44,c7db1e80,0,caa4a000,...) at devfs_read_f+0x7e dofileread(f27d2c44,ffffffff,ffffffff,0,f27d2c20,...) at dofileread+0x9e kern_readv(caa4a000,0,f27d2c44,f27d2c64,1,...) at kern_readv+0x68 sys_read(caa4a000,f27d2cec,c0fde84a,c0df22f6,246,...) at sys_read+0x4f syscall(f27d2d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a6333, esp = 0xbfbfed7c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2301 tid 100108 td 0xcab01b80 sched_switch(cab01b80,0,104,1a3,d47440e0,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cab01b80,0,c0f925e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f925e0,160,0,cab01b80,cab01b80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7efc470,0,c0f97418,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7efc470,c1128b90,c0f96df0,527,0,...) at _cv_wait_sig+0x260 tty_wait(c7efc400,c7efc470,f27cfc44,1,0,...) at tty_wait+0x71 ttydisc_read(c7efc400,f27cfc44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c7fa6c00,f27cfc44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(cac08a48,f27cfc44,c7db1e80,0,cab01b80,...) at devfs_read_f+0x7e dofileread(f27cfc44,ffffffff,ffffffff,0,f27cfc20,...) at dofileread+0x9e kern_readv(cab01b80,0,f27cfc44,f27cfc64,1,...) at kern_readv+0x68 sys_read(cab01b80,f27cfcec,c0fde84a,c0df22f6,246,...) at sys_read+0x4f syscall(f27cfd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a6333, esp = 0xbfbfed7c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2300 tid 100162 td 0xcb29d2e0 sched_switch(cb29d2e0,0,104,1a3,146503c,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cb29d2e0,0,c0f925e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b26a,c7efc004,0,c0f8b00b,cb29d2e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7efc070,0,f28a2af0,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7efc070,c7efc004,c0f96df0,527,0,...) at _cv_wait_sig+0x260 tty_wait(c7efc000,c7efc070,f28a2c44,1,0,...) at tty_wait+0x71 ttydisc_read(c7efc000,f28a2c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c7e17d00,f28a2c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(ca787a80,f28a2c44,c7db1e80,0,cb29d2e0,...) at devfs_read_f+0x7e dofileread(f28a2c44,ffffffff,ffffffff,0,f28a2c20,...) at dofileread+0x9e kern_readv(cb29d2e0,0,f28a2c44,f28a2c64,1,...) at kern_readv+0x68 sys_read(cb29d2e0,f28a2cec,c0fde84a,c0df22f6,246,...) at sys_read+0x4f syscall(f28a2d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a6333, esp = 0xbfbfed7c, ebp = 0xbfbfedb8 --- Tracing command inetd pid 2261 tid 100158 td 0xcb29e000 sched_switch(cb29e000,0,104,1a3,a24513eb,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cb29e000,0,c0f925e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b26a,ca728590,0,c0f8b00b,cb29e000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca7285a4,0,f2896a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca7285a4,ca728590,c0f94d30,6a4,cac91540,...) at _cv_wait_sig+0x260 seltdwait(cac91540,58,c7db1e80,cb29e000,c7d6e6b0,...) at seltdwait+0xa2 kern_select(cb29e000,6,bfbfdfa0,0,0,0,20,cb1e6000,4) at kern_select+0x655 sys_select(cb29e000,f2896cec,c0fde84a,c0f95543,206,...) at sys_select+0x66 syscall(f2896d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281b82b3, esp = 0xbfbfdf5c, ebp = 0xbfbfede8 --- Tracing command watchdogd pid 2229 tid 100159 td 0xcb29db80 sched_switch(cb29db80,0,104,1a3,3a1357c0,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(cb29db80,0,c0f925e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(ea61,c0a9be50,cb29db80,1,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c112a4e4,6c,c0f8e137,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c112a4e4,0,16c,c0f8e137,ea61,...) at _sleep+0x34f kern_nanosleep(cb29db80,f2899c50,f2899c58,3c,0,...) at kern_nanosleep+0xc1 sys_nanosleep(cb29db80,f2899cec,c0fde84a,c0df22f6,206,...) at sys_nanosleep+0x6f syscall(f2899d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x2819de27, esp = 0xbfbfecbc, ebp = 0xbfbfece8 --- Tracing command bash pid 2219 tid 100113 td 0xcab01000 sched_switch(cab01000,0,104,1a3,95a5e0b5,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(cab01000,0,c0f925e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f925e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cabfe2c4,6c,c0f954f7,100,0,...) at sleepq_wait_sig+0x17 _sleep(cabfe2c4,cabfe34c,16c,c0f954f7,0,...) at _sleep+0x385 kern_wait(cab01000,ffffffff,f27dec60,6,0,...) at kern_wait+0x626 sys_wait4(cab01000,f27decec,c0fde84a,c0f953e6,216,...) at sys_wait4+0x3b syscall(f27ded28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x282cf45b, esp = 0xbfbfe97c, ebp = 0xbfbfe998 --- Tracing command sshd pid 2218 tid 100084 td 0xca75cb80 sched_switch(ca75cb80,0,104,1a3,fd2bbd24,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca75cb80,0,c0f925e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b26a,ca7c6050,0,c0f8b00b,ca75cb80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca7c6064,0,f2713a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca7c6064,ca7c6050,c0f94d30,6a4,ca7c9af0,...) at _cv_wait_sig+0x260 seltdwait(ca7c9af0,58,ca7c5580,ca75cb80,f2713acc,...) at seltdwait+0xa2 kern_select(ca75cb80,c,288060b8,288060c0,0,0,20,ca6beb10,0) at kern_select+0x655 sys_select(ca75cb80,f2713cec,c0fde84a,c0f9564b,206,...) at sys_select+0x66 syscall(f2713d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284d72b3, esp = 0xbfbfde2c, ebp = 0xbfbfde78 --- Tracing command sshd pid 2216 tid 100095 td 0xca7595c0 sched_switch(ca7595c0,0,104,1a3,b24e4861,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca7595c0,0,c0f925e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b26a,ca6b3b90,0,c0f8b00b,ca7595c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca6b3ba4,0,f2752abc,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca6b3ba4,ca6b3b90,c0f94d30,6a4,f2752b44,...) at _cv_wait_sig+0x260 seltdwait(cab1622c,c0f94d30,5a1,ca7595c0,8,...) at seltdwait+0xa2 sys_poll(ca7595c0,f2752cec,c0fde84a,c0f7d6d9,202,...) at sys_poll+0x321 syscall(f2752d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x2847014b, esp = 0xbfbfde8c, ebp = 0xbfbfdec8 --- Tracing command cron pid 2173 tid 100094 td 0xca7598a0 sched_switch(ca7598a0,0,104,1a3,794d4e2a,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(ca7598a0,0,c0f925e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(ea61,c0a9be50,ca7598a0,0,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c112a4e4,6c,c0f8e137,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c112a4e4,0,16c,c0f8e137,ea61,...) at _sleep+0x34f kern_nanosleep(ca7598a0,f274fc50,f274fc58,3c,0,...) at kern_nanosleep+0xc1 sys_nanosleep(ca7598a0,f274fcec,c0fde84a,c0f959a4,206,...) at sys_nanosleep+0x6f syscall(f274fd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x2818fe27, esp = 0xbfbfec4c, ebp = 0xbfbfec78 --- Tracing command sendmail pid 2165 tid 100098 td 0xcab032e0 sched_switch(cab032e0,0,104,1a3,34185161,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,74,...) at mi_switch+0x1ff sleepq_switch(cab032e0,0,c0f925e0,1a7,74,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f925e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(caae2b68,74,c0e13e2e,100,0,...) at sleepq_wait_sig+0x17 _sleep(caae2b68,caae2b98,174,c0e13e2e,0,...) at _sleep+0x385 kern_sigsuspend(cab032e0,0,0,0,0,...) at kern_sigsuspend+0xba sys_sigsuspend(cab032e0,f275bcec,c0fde84a,c0f953e6,202,...) at sys_sigsuspend+0x4d syscall(f275bd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28348ccb, esp = 0xbfbfcf9c, ebp = 0xbfbfcfc8 --- Tracing command sendmail pid 2159 tid 100145 td 0xcab038a0 sched_switch(cab038a0,0,104,1a3,36c481ad,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cab038a0,0,c0f925e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f286fa20,c0a4b26a,caf3d710,0,cab038a0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(caf3d724,0,f286fa58,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(caf3d724,caf3d710,1389,6a4,ca76bd20,...) at _cv_timedwait_sig+0x270 seltdwait(f286fc10,f286fc18,c7fce300,cab038a0,c0a47aad,...) at seltdwait+0x8a kern_select(cab038a0,5,bfbfc510,0,0,f286fc5c,20,5,0) at kern_select+0x655 sys_select(cab038a0,f286fcec,c0fde84a,c0f95903,206,...) at sys_select+0x66 syscall(f286fd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x283f32b3, esp = 0xbfbfc47c, ebp = 0xbfbfcfa8 --- Tracing command sshd pid 2151 tid 100142 td 0xcac002e0 sched_switch(cac002e0,0,104,1a3,aa46dd03,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cac002e0,0,c0f925e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b26a,caf3d390,0,c0f8b00b,cac002e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(caf3d3a4,0,f2866a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(caf3d3a4,caf3d390,c0f94d30,6a4,cabe4dc8,...) at _cv_wait_sig+0x260 seltdwait(cabe4dc8,58,c7fceb80,cac002e0,cab0d910,...) at seltdwait+0xa2 kern_select(cac002e0,7,2880c0b8,0,0,0,20,cac79b10,a) at kern_select+0x655 sys_select(cac002e0,f2866cec,c,c,202,...) at sys_select+0x66 syscall(f2866d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284d72b3, esp = 0xbfbfdeec, ebp = 0xbfbfedf8 --- Tracing command moused pid 2091 tid 100141 td 0xcac005c0 sched_switch(cac005c0,0,104,1a3,d6e037d8,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cac005c0,0,c0f925e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b26a,ca7c6d90,0,c0f8b00b,cac005c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca7c6da4,0,f2863a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca7c6da4,ca7c6d90,c0f94d30,6a4,cabe4498,...) at _cv_wait_sig+0x260 seltdwait(cabe4498,58,c7db1e80,cac005c0,f6,...) at seltdwait+0xa2 kern_select(cac005c0,400,bfbfea04,0,0,0,20,caead000,1) at kern_select+0x655 sys_select(cac005c0,f2863cec,c0fde84a,c0df22f6,246,...) at sys_select+0x66 syscall(f2863d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281c52b3, esp = 0xbfbfe99c, ebp = 0xbfbfeb28 --- Tracing command ntpd pid 2013 tid 100106 td 0xcab022e0 sched_switch(cab022e0,0,104,1a3,c34f57ba,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cab022e0,0,c0f925e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b26a,caf46510,0,c0f8b00b,cab022e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(caf46524,0,f27c9a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(caf46524,caf46510,c0f94d30,6a4,cac910e0,...) at _cv_wait_sig+0x260 seltdwait(cac910e0,58,c7db1e80,cab022e0,0,...) at seltdwait+0xa2 kern_select(cab022e0,1c,bfbfece8,0,0,0,20,cabff588,0) at kern_select+0x655 sys_select(cab022e0,f27c9cec,c0fde84a,f27c9d80,202,...) at sys_select+0x66 syscall(f27c9d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x283722b3, esp = 0xbfbfecbc, ebp = 0xbfbfed88 --- Tracing command nfsd pid 1914 tid 100148 td 0xca6bf8a0 sched_switch(ca6bf8a0,0,104,1a3,93f1b5,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca6bf8a0,0,c0f925e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f2878be0,c0a4b26a,c8028780,0,ca6bf8a0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(ca60fc90,0,f2878c18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(ca60fc90,c8028780,1388,3b1,0,...) at _cv_timedwait_sig+0x270 svc_run_internal(f2878d14,c0a2dda8,c8028780,f2878d28,c0f872f0,...) at svc_run_internal+0x356 svc_thread_start(c8028780,f2878d28,c0f872f0,3db,cac7584c,...) at svc_thread_start+0x10 fork_exit(c0c4acf0,c8028780,f2878d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x2e, eip = 0xc, esp = 0x33, ebp = 0x1 --- Tracing command nfsd pid 1914 tid 100147 td 0xca6bfb80 sched_switch(ca6bfb80,0,104,1a3,93bbad,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca6bfb80,0,c0f925e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f2875be0,c0a4b26a,c8028780,0,ca6bfb80,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(ca6c8a90,0,f2875c18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(ca6c8a90,c8028780,1388,3b1,0,...) at _cv_timedwait_sig+0x270 svc_run_internal(f2875d14,c0a2dda8,c8028780,f2875d28,c0f872f0,...) at svc_run_internal+0x356 svc_thread_start(c8028780,f2875d28,c0f872f0,3db,cac7584c,...) at svc_thread_start+0x10 fork_exit(c0c4acf0,c8028780,f2875d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x2e, eip = 0xc, esp = 0x33, ebp = 0x1 --- Tracing command nfsd pid 1914 tid 100146 td 0xca6c0000 sched_switch(ca6c0000,0,104,1a3,93d881,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca6c0000,0,c0f925e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f2872be0,c0a4b26a,c8028780,0,ca6c0000,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(ca60fb90,0,f2872c18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(ca60fb90,c8028780,1388,3b1,28099160,...) at _cv_timedwait_sig+0x270 svc_run_internal(f2872d14,c0a2dda8,c8028780,f2872d28,c0f872f0,...) at svc_run_internal+0x356 svc_thread_start(c8028780,f2872d28,c0f872f0,3db,cac7584c,...) at svc_thread_start+0x10 fork_exit(c0c4acf0,c8028780,f2872d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x2e, eip = 0xc, esp = 0x33, ebp = 0x1 --- Tracing command nfsd pid 1914 tid 100129 td 0xcac7eb80 sched_switch(cac7eb80,0,104,1a3,93e80a,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cac7eb80,0,c0f925e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f2826498,c0a4b26a,c8028780,0,cac7eb80,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(ca60fd10,0,f28264d0,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(ca60fd10,c8028780,1388,3b1,f2826518,...) at _cv_timedwait_sig+0x270 svc_run_internal(cac7ed30,14,c0fbbd7a,c0f7ebbb,f2826810,...) at svc_run_internal+0x356 svc_run(c8028780,0,c0f7eb96,19f,0,...) at svc_run+0x7b nfsrvd_nfsd(cac7eb80,f2826810,c,c1273370,0,...) at nfsrvd_nfsd+0x1c3 nfssvc_nfsd(cac7eb80,f2826cec,bfbfe870,cac7eb80,f2826d1c,...) at nfssvc_nfsd+0x177 sys_nfssvc(cac7eb80,f2826cec,c0fde84a,c0f95903,206,...) at sys_nfssvc+0xeb syscall(f2826d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (155, FreeBSD ELF32, sys_nfssvc), eip = 0x280e890b, esp = 0xbfbfe82c, ebp = 0xbfbfea98 --- Tracing command nfsd pid 1908 tid 100091 td 0xcaa4ab80 sched_switch(caa4ab80,0,104,1a3,110a233,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(caa4ab80,0,c0f925e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b26a,caf46550,0,c0f8b00b,caa4ab80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(caf46564,0,f2746a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(caf46564,caf46550,c0f94d30,6a4,cac08620,...) at _cv_wait_sig+0x260 seltdwait(cac08620,58,c7db1e80,caa4ab80,cabe2cb0,...) at seltdwait+0xa2 kern_select(caa4ab80,7,bfbfecb4,0,0,0,20,caa4484c,1) at kern_select+0x655 sys_select(caa4ab80,f2746cec,c,c,202,...) at sys_select+0x66 syscall(f2746d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281922b3, esp = 0xbfbfea9c, ebp = 0xbfbfede8 --- Tracing command mountd pid 1898 tid 100097 td 0xca759000 sched_switch(ca759000,0,104,1a3,64e34b9,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca759000,0,c0f925e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b26a,ca6b3bd0,0,c0f8b00b,ca759000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca6b3be4,0,f2758a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca6b3be4,ca6b3bd0,c0f94d30,6a4,cabe40a8,...) at _cv_wait_sig+0x260 seltdwait(cabe40a8,58,c7db1e80,ca759000,478,...) at seltdwait+0xa2 kern_select(ca759000,a,bfbfed04,0,0,0,20,caae3000,bfbfed04) at kern_select+0x655 sys_select(ca759000,f2758cec,c0fde84a,c0f7c977,206,...) at sys_select+0x66 syscall(f2758d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281aa2b3, esp = 0xbfbfec9c, ebp = 0xbfbfede8 --- Tracing command rpcbind pid 1799 tid 100087 td 0xca6c02e0 sched_switch(ca6c02e0,0,104,1a3,a69b1fd8,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca6c02e0,0,c0f925e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f271fa84,c0a4b26a,ca728210,0,ca6c02e0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(ca728224,0,f271fabc,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(ca728224,ca728210,7531,6a4,f271fb74,...) at _cv_timedwait_sig+0x270 seltdwait(f271fc44,f271fc4c,5a1,ca6c02e0,c7d6eac0,...) at seltdwait+0x8a sys_poll(ca6c02e0,f271fcec,c0fde84a,c0f95634,212,...) at sys_poll+0x321 syscall(f271fd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x2814a14b, esp = 0xbfbfcbac, ebp = 0xbfbfed78 --- Tracing command syslogd pid 1717 tid 100093 td 0xca759b80 sched_switch(ca759b80,0,104,1a3,3cc41f86,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca759b80,0,c0f925e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b26a,ca728090,0,c0f8b00b,ca759b80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca7280a4,0,f274ca58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca7280a4,ca728090,c0f94d30,6a4,cac913f0,...) at _cv_wait_sig+0x260 seltdwait(cac913f0,58,c7db1e80,ca759b80,e1ad8b80,...) at seltdwait+0xa2 kern_select(ca759b80,9,2847f0ac,0,0,0,20,caae3b10,0) at kern_select+0x655 sys_select(ca759b80,f274ccec,c0fde84a,c0fc151f,246,...) at sys_select+0x66 syscall(f274cd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281a92b3, esp = 0xbfbfe29c, ebp = 0xbfbfede8 --- Tracing command devd pid 1510 tid 100111 td 0xcab015c0 sched_switch(cab015c0,0,104,1a3,5618917,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cab015c0,0,c0f925e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b26a,ca6b2c10,0,c0f8b00b,cab015c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca6b2c24,0,f27d8a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca6b2c24,ca6b2c10,c0f94d30,6a4,ca76bcb0,...) at _cv_wait_sig+0x260 seltdwait(ca76bcb0,58,c7db1e80,cab015c0,c1273430,...) at seltdwait+0xa2 kern_select(cab015c0,5,bfbfe960,0,0,0,20,cabfe84c,32) at kern_select+0x655 sys_select(cab015c0,f27d8cec,c0fde84a,c0ded5b7,206,...) at sys_select+0x66 syscall(f27d8d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x809152b, esp = 0xbfbfe92c, ebp = 0xbfbfee08 --- Tracing command softdepflush pid 19 tid 100077 td 0xca6c0b80 sched_switch(ca6c0b80,0,104,1a3,cd27f5b2,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,54,...) at mi_switch+0x1ff sleepq_switch(ca6c0b80,0,c0f925e0,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c12a69a0,54,c0fc2d90,0,0,...) at sleepq_timedwait+0x6b _sleep(c12a69a0,c12a6960,54,c0fc2d90,3e8,...) at _sleep+0x36a softdep_flush(0,ee320d28,c0f872f0,3db,ca6bc000,...) at softdep_flush+0x266 fork_exit(c0c9a1b0,0,ee320d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee320d60, ebp = 0 --- Tracing command vnlru pid 18 tid 100076 td 0xc80608a0 sched_switch(c80608a0,0,104,1a3,cb9a6b20,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,60,...) at mi_switch+0x1ff sleepq_switch(c80608a0,0,c0f925e0,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(ca6bc2c4,60,c0f9d5dd,0,0,...) at sleepq_timedwait+0x6b _sleep(ca6bc2c4,c12a0674,260,c0f9d5dd,3e8,...) at _sleep+0x36a vnlru_proc(0,ee31dd28,c0f872f0,3db,ca6bc2c4,...) at vnlru_proc+0xe7 fork_exit(c0afcbe0,0,ee31dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee31dd60, ebp = 0 --- Tracing command syncer pid 17 tid 100075 td 0xc8060b80 sched_switch(c8060b80,0,104,1a3,ce763eb0,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8060b80,0,c0f925e0,28b,c8060b80,...) at sleepq_switch+0x162 sleepq_timedwait(c12a06b4,0,ee31ac78,1,0,...) at sleepq_timedwait+0x6b _cv_timedwait(c12a06b4,c12a06a0,3e8,78b,4e20,...) at _cv_timedwait+0x270 sched_sync(0,ee31ad28,c0f872f0,3db,ca6bc588,...) at sched_sync+0x595 fork_exit(c0afbf50,0,ee31ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee31ad60, ebp = 0 --- Tracing command bufdaemon pid 16 tid 100074 td 0xc8189000 sched_switch(c8189000,0,104,1a3,c919114a,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,54,...) at mi_switch+0x1ff sleepq_switch(c8189000,0,c0f925e0,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c12a0348,54,c0f9a511,0,0,...) at sleepq_timedwait+0x6b _sleep(c12a0348,c12a0350,54,c0f9a511,3e8,...) at _sleep+0x36a buf_daemon(0,ee317d28,c0f872f0,3db,ca6bc84c,...) at buf_daemon+0x175 fork_exit(c0ae0720,0,ee317d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee317d60, ebp = 0 --- Tracing command pagezero pid 9 tid 100073 td 0xc81892e0 sched_switch(c81892e0,0,104,1a3,2314147a,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c81892e0,0,c0f925e0,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c12a885c,0,c0fc9dbe,0,0,...) at sleepq_timedwait+0x6b _sleep(c12a885c,c12a7300,0,c0fc9dbe,493e0,...) at _sleep+0x36a vm_pagezero(0,ee314d28,c0f872f0,3db,ca6bcb10,...) at vm_pagezero+0xdc fork_exit(c0ce14a0,0,ee314d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee314d60, ebp = 0 --- Tracing command vmdaemon pid 8 tid 100072 td 0xc81895c0 sched_switch(c81895c0,0,104,1a3,32c3980b,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,74,...) at mi_switch+0x1ff sleepq_switch(c81895c0,0,c0f925e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c12a8448,74,c0f9a511,0,0,...) at sleepq_wait+0x63 _sleep(c12a8448,c12a844c,74,c0f9a511,0,...) at _sleep+0x39c vm_daemon(0,ee311d28,c0f872f0,3db,ca6be000,...) at vm_daemon+0x59 fork_exit(c0cdadb0,0,ee311d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee311d60, ebp = 0 --- Tracing command pagedaemon pid 7 tid 100071 td 0xc81898a0 sched_switch(c81898a0,0,104,1a3,5e214793,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,54,...) at mi_switch+0x1ff sleepq_switch(c81898a0,0,c0f925e0,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c12a8410,54,c0f9a511,0,0,...) at sleepq_timedwait+0x6b _sleep(c12a8410,c12a7300,54,c0f9a511,1388,...) at _sleep+0x36a vm_pageout(0,ee30ed28,c0f872f0,3db,c7dd12c4,...) at vm_pageout+0x2c7 fork_exit(c0cdbe70,0,ee30ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee30ed60, ebp = 0 --- Tracing command xpt_thrd pid 6 tid 100070 td 0xc8189b80 sched_switch(c8189b80,0,104,1a3,a9b5884f,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c8189b80,0,c0f925e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c10eecd4,5c,c0de7dc7,0,0,...) at sleepq_wait+0x63 _sleep(c10eecd4,c10eed18,5c,c0de7dc7,0,...) at _sleep+0x39c xpt_scanner_thread(0,ee18bd28,c0f872f0,3db,c7dd1588,...) at xpt_scanner_thread+0x47 fork_exit(c049d370,0,ee18bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee18bd60, ebp = 0 --- Tracing command sctp_iterator pid 5 tid 100069 td 0xc818a000 sched_switch(c818a000,0,104,1a3,d12ff28,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c818a000,0,c0f925e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c12a21d8,0,c0fb0c5f,0,0,...) at sleepq_wait+0x63 _sleep(c12a21d8,c12a21a0,0,c0fb0c5f,0,...) at _sleep+0x39c sctp_iterator_thread(0,ee188d28,c0f872f0,3db,c7dd184c,...) at sctp_iterator_thread+0x5c fork_exit(c0b8e2f0,0,ee188d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee188d60, ebp = 0 --- Tracing command ctl_thrd pid 4 tid 100067 td 0xc818a5c0 sched_switch(c818a5c0,0,104,1a3,9751ebe6,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c818a5c0,0,c0f925e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c819a000,5c,c0df27d2,0,0,...) at sleepq_wait+0x63 _sleep(c819a000,c819a000,5c,c0df27d2,0,...) at _sleep+0x39c ctl_work_thread(c819a000,ee182d28,c0f872f0,3db,c7dd1b10,...) at ctl_work_thread+0x22f6 fork_exit(c04c6770,c819a000,ee182d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee182d60, ebp = 0 --- Tracing command fdc0 pid 3 tid 100064 td 0xc818b000 sched_switch(c818b000,0,104,1a3,f7725f7f,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c818b000,0,c0f925e0,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c817a23c,5c,c0f7f632,0,0,...) at sleepq_timedwait+0x6b _sleep(c817a23c,c817a2f0,5c,c0f7f632,3e8,...) at _sleep+0x36a fdc_thread(c817a200,ee179d28,c0f872f0,3db,c8045000,...) at fdc_thread+0x27d fork_exit(c0d606f0,c817a200,ee179d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee179d60, ebp = 0 --- Tracing command fw0_probe pid 2 tid 100062 td 0xc80525c0 sched_switch(c80525c0,0,104,1a3,a9b5a8ab,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(c80525c0,0,c0f925e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f925e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8090000,6c,c0f7f632,100,0,...) at sleepq_wait_sig+0x17 _sleep(c8090000,c8094488,16c,c0f7f632,0,...) at _sleep+0x385 fw_bus_probe_thread(c8090000,ee169d28,c0f872f0,3db,c80452c4,...) at fw_bus_probe_thread+0xa08 fork_exit(c06f09e0,c8090000,ee169d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee169d60, ebp = 0 --- Tracing command usb pid 15 tid 100058 td 0xc805d2e0 sched_switch(c805d2e0,0,104,1a3,974cc024,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c805d2e0,0,c0f925e0,268,c805d2e0,...) at sleepq_switch+0x162 sleepq_wait(c8066d34,0,ee152cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8066d34,c8066e4c,c0f65cf1,74,c8066d3c,...) at _cv_wait+0x260 usb_process(c8066d2c,ee152d28,c0f872f0,3db,c8045588,...) at usb_process+0x1a3 fork_exit(c08fd840,c8066d2c,ee152d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee152d60, ebp = 0 --- Tracing command usb pid 15 tid 100057 td 0xc805d5c0 sched_switch(c805d5c0,0,104,1a3,be97c635,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c805d5c0,0,c0f925e0,268,c805d5c0,...) at sleepq_switch+0x162 sleepq_wait(c8066d04,0,ee14fcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8066d04,c8066e4c,c0f65cf1,74,c8066d0c,...) at _cv_wait+0x260 usb_process(c8066cfc,ee14fd28,c0f872f0,3db,c8045588,...) at usb_process+0x1a3 fork_exit(c08fd840,c8066cfc,ee14fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee14fd60, ebp = 0 --- Tracing command usb pid 15 tid 100056 td 0xc805d8a0 sched_switch(c805d8a0,0,104,1a3,96e6639d,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c805d8a0,0,c0f925e0,268,c805d8a0,...) at sleepq_switch+0x162 sleepq_wait(c8066cd4,0,ee14ccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8066cd4,c8066e4c,c0f65cf1,74,c8066cdc,...) at _cv_wait+0x260 usb_process(c8066ccc,ee14cd28,c0f872f0,3db,c8045588,...) at usb_process+0x1a3 fork_exit(c08fd840,c8066ccc,ee14cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee14cd60, ebp = 0 --- Tracing command usb pid 15 tid 100055 td 0xc805db80 sched_switch(c805db80,0,104,1a3,96e65172,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c805db80,0,c0f925e0,268,c805db80,...) at sleepq_switch+0x162 sleepq_wait(c8066ca4,0,ee149cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8066ca4,c8066e4c,c0f65cf1,74,c8066cac,...) at _cv_wait+0x260 usb_process(c8066c9c,ee149d28,c0f872f0,3db,c8045588,...) at usb_process+0x1a3 fork_exit(c08fd840,c8066c9c,ee149d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee149d60, ebp = 0 --- Tracing command usb pid 15 tid 100053 td 0xc80602e0 sched_switch(c80602e0,0,104,1a3,96e63bf0,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c80602e0,0,c0f925e0,268,c80602e0,...) at sleepq_switch+0x162 sleepq_wait(c8058b5c,0,ee140cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8058b5c,c8058c74,c0f65cf1,74,c8058b64,...) at _cv_wait+0x260 usb_process(c8058b54,ee140d28,c0f872f0,3db,c8045588,...) at usb_process+0x1a3 fork_exit(c08fd840,c8058b54,ee140d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee140d60, ebp = 0 --- Tracing command usb pid 15 tid 100052 td 0xc80605c0 sched_switch(c80605c0,0,104,1a3,b61d68c0,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c80605c0,0,c0f925e0,268,c80605c0,...) at sleepq_switch+0x162 sleepq_wait(c8058b2c,0,ee13dcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8058b2c,c8058c74,c0f65cf1,74,c8058b34,...) at _cv_wait+0x260 usb_process(c8058b24,ee13dd28,c0f872f0,3db,c8045588,...) at usb_process+0x1a3 fork_exit(c08fd840,c8058b24,ee13dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee13dd60, ebp = 0 --- Tracing command usb pid 15 tid 100051 td 0xc8050000 sched_switch(c8050000,0,104,1a3,96820fc5,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8050000,0,c0f925e0,268,c8050000,...) at sleepq_switch+0x162 sleepq_wait(c8058afc,0,ee13acac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8058afc,c8058c74,c0f65cf1,74,c8058b04,...) at _cv_wait+0x260 usb_process(c8058af4,ee13ad28,c0f872f0,3db,c8045588,...) at usb_process+0x1a3 fork_exit(c08fd840,c8058af4,ee13ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee13ad60, ebp = 0 --- Tracing command usb pid 15 tid 100050 td 0xc80502e0 sched_switch(c80502e0,0,104,1a3,9681fd60,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c80502e0,0,c0f925e0,268,c80502e0,...) at sleepq_switch+0x162 sleepq_wait(c8058acc,0,ee137cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8058acc,c8058c74,c0f65cf1,74,c8058ad4,...) at _cv_wait+0x260 usb_process(c8058ac4,ee137d28,c0f872f0,3db,c8045588,...) at usb_process+0x1a3 fork_exit(c08fd840,c8058ac4,ee137d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee137d60, ebp = 0 --- Tracing command usb pid 15 tid 100049 td 0xc80505c0 sched_switch(c80505c0,0,104,1a3,9681e864,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c80505c0,0,c0f925e0,268,c80505c0,...) at sleepq_switch+0x162 sleepq_wait(c8054b5c,0,ee133cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8054b5c,c8054c74,c0f65cf1,74,c8054b64,...) at _cv_wait+0x260 usb_process(c8054b54,ee133d28,c0f872f0,3db,c8045588,...) at usb_process+0x1a3 fork_exit(c08fd840,c8054b54,ee133d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee133d60, ebp = 0 --- Tracing command usb pid 15 tid 100048 td 0xc80508a0 sched_switch(c80508a0,0,104,1a3,b300ebd9,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c80508a0,0,c0f925e0,268,c80508a0,...) at sleepq_switch+0x162 sleepq_wait(c8054b2c,0,ee130cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8054b2c,c8054c74,c0f65cf1,74,c8054b34,...) at _cv_wait+0x260 usb_process(c8054b24,ee130d28,c0f872f0,3db,c8045588,...) at usb_process+0x1a3 fork_exit(c08fd840,c8054b24,ee130d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee130d60, ebp = 0 --- Tracing command usb pid 15 tid 100047 td 0xc8050b80 sched_switch(c8050b80,0,104,1a3,90ae5fad,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8050b80,0,c0f925e0,268,c8050b80,...) at sleepq_switch+0x162 sleepq_wait(c8054afc,0,ee12dcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8054afc,c8054c74,c0f65cf1,74,c8054b04,...) at _cv_wait+0x260 usb_process(c8054af4,ee12dd28,c0f872f0,3db,c8045588,...) at usb_process+0x1a3 fork_exit(c08fd840,c8054af4,ee12dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee12dd60, ebp = 0 --- Tracing command usb pid 15 tid 100046 td 0xc8051000 sched_switch(c8051000,0,104,1a3,90ae4d5d,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8051000,0,c0f925e0,268,c8051000,...) at sleepq_switch+0x162 sleepq_wait(c8054acc,0,ee12acac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8054acc,c8054c74,c0f65cf1,74,c8054ad4,...) at _cv_wait+0x260 usb_process(c8054ac4,ee12ad28,c0f872f0,3db,c8045588,...) at usb_process+0x1a3 fork_exit(c08fd840,c8054ac4,ee12ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee12ad60, ebp = 0 --- Tracing command usb pid 15 tid 100045 td 0xc80512e0 sched_switch(c80512e0,0,104,1a3,90ae3c0a,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c80512e0,0,c0f925e0,268,c80512e0,...) at sleepq_switch+0x162 sleepq_wait(c804cb5c,0,ee126cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c804cb5c,c804cc74,c0f65cf1,74,c804cb64,...) at _cv_wait+0x260 usb_process(c804cb54,ee126d28,c0f872f0,3db,c8045588,...) at usb_process+0x1a3 fork_exit(c08fd840,c804cb54,ee126d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee126d60, ebp = 0 --- Tracing command usb pid 15 tid 100044 td 0xc80515c0 sched_switch(c80515c0,0,104,1a3,b1d2aebc,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c80515c0,0,c0f925e0,268,c80515c0,...) at sleepq_switch+0x162 sleepq_wait(c804cb2c,0,ee123cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c804cb2c,c804cc74,c0f65cf1,74,c804cb34,...) at _cv_wait+0x260 usb_process(c804cb24,ee123d28,c0f872f0,3db,c8045588,...) at usb_process+0x1a3 fork_exit(c08fd840,c804cb24,ee123d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee123d60, ebp = 0 --- Tracing command usb pid 15 tid 100043 td 0xc80518a0 sched_switch(c80518a0,0,104,1a3,904ab072,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c80518a0,0,c0f925e0,268,c80518a0,...) at sleepq_switch+0x162 sleepq_wait(c804cafc,0,ee120cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c804cafc,c804cc74,c0f65cf1,74,c804cb04,...) at _cv_wait+0x260 usb_process(c804caf4,ee120d28,c0f872f0,3db,c8045588,...) at usb_process+0x1a3 fork_exit(c08fd840,c804caf4,ee120d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee120d60, ebp = 0 --- Tracing command usb pid 15 tid 100042 td 0xc8051b80 sched_switch(c8051b80,0,104,1a3,904a9cb4,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8051b80,0,c0f925e0,268,c8051b80,...) at sleepq_switch+0x162 sleepq_wait(c804cacc,0,ee11dcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c804cacc,c804cc74,c0f65cf1,74,c804cad4,...) at _cv_wait+0x260 usb_process(c804cac4,ee11dd28,c0f872f0,3db,c8045588,...) at usb_process+0x1a3 fork_exit(c08fd840,c804cac4,ee11dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee11dd60, ebp = 0 --- Tracing command usb pid 15 tid 100040 td 0xc8041000 sched_switch(c8041000,0,104,1a3,904a87b0,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8041000,0,c0f925e0,268,c8041000,...) at sleepq_switch+0x162 sleepq_wait(c8048b5c,0,ee116cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8048b5c,c8048c74,c0f65cf1,74,c8048b64,...) at _cv_wait+0x260 usb_process(c8048b54,ee116d28,c0f872f0,3db,c8045588,...) at usb_process+0x1a3 fork_exit(c08fd840,c8048b54,ee116d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee116d60, ebp = 0 --- Tracing command usb pid 15 tid 100039 td 0xc80412e0 sched_switch(c80412e0,0,104,1a3,b08914d2,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c80412e0,0,c0f925e0,268,c80412e0,...) at sleepq_switch+0x162 sleepq_wait(c8048b2c,0,ee113cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8048b2c,c8048c74,c0f65cf1,74,c8048b34,...) at _cv_wait+0x260 usb_process(c8048b24,ee113d28,c0f872f0,3db,c8045588,...) at usb_process+0x1a3 fork_exit(c08fd840,c8048b24,ee113d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee113d60, ebp = 0 --- Tracing command usb pid 15 tid 100038 td 0xc80415c0 sched_switch(c80415c0,0,104,1a3,8fe6ff04,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c80415c0,0,c0f925e0,268,c80415c0,...) at sleepq_switch+0x162 sleepq_wait(c8048afc,0,ee110cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8048afc,c8048c74,c0f65cf1,74,c8048b04,...) at _cv_wait+0x260 usb_process(c8048af4,ee110d28,c0f872f0,3db,c8045588,...) at usb_process+0x1a3 fork_exit(c08fd840,c8048af4,ee110d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee110d60, ebp = 0 --- Tracing command usb pid 15 tid 100037 td 0xc80418a0 sched_switch(c80418a0,0,104,1a3,8fe6ec86,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c80418a0,0,c0f925e0,268,c80418a0,...) at sleepq_switch+0x162 sleepq_wait(c8048acc,0,ee10dcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8048acc,c8048c74,c0f65cf1,74,c8048ad4,...) at _cv_wait+0x260 usb_process(c8048ac4,ee10dd28,c0f872f0,3db,c8045588,...) at usb_process+0x1a3 fork_exit(c08fd840,c8048ac4,ee10dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee10dd60, ebp = 0 --- Tracing command usb pid 15 tid 100035 td 0xc8044000 sched_switch(c8044000,0,104,1a3,8fe6d834,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8044000,0,c0f925e0,268,c8044000,...) at sleepq_switch+0x162 sleepq_wait(c803ab5c,0,ee106cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c803ab5c,c803ac74,c0f65cf1,74,c803ab64,...) at _cv_wait+0x260 usb_process(c803ab54,ee106d28,c0f872f0,3db,c8045588,...) at usb_process+0x1a3 fork_exit(c08fd840,c803ab54,ee106d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee106d60, ebp = 0 --- Tracing command usb pid 15 tid 100034 td 0xc80442e0 sched_switch(c80442e0,0,104,1a3,aeb6794e,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c80442e0,0,c0f925e0,268,c80442e0,...) at sleepq_switch+0x162 sleepq_wait(c803ab2c,0,ee103cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c803ab2c,c803ac74,c0f65cf1,74,c803ab34,...) at _cv_wait+0x260 usb_process(c803ab24,ee103d28,c0f872f0,3db,c8045588,...) at usb_process+0x1a3 fork_exit(c08fd840,c803ab24,ee103d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee103d60, ebp = 0 --- Tracing command usb pid 15 tid 100033 td 0xc80445c0 sched_switch(c80445c0,0,104,1a3,8f82a051,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c80445c0,0,c0f925e0,268,c80445c0,...) at sleepq_switch+0x162 sleepq_wait(c803aafc,0,ee100cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c803aafc,c803ac74,c0f65cf1,74,c803ab04,...) at _cv_wait+0x260 usb_process(c803aaf4,ee100d28,c0f872f0,3db,c8045588,...) at usb_process+0x1a3 fork_exit(c08fd840,c803aaf4,ee100d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee100d60, ebp = 0 --- Tracing command usb pid 15 tid 100032 td 0xc80448a0 sched_switch(c80448a0,0,104,1a3,8f828e00,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c80448a0,0,c0f925e0,268,c80448a0,...) at sleepq_switch+0x162 sleepq_wait(c803aacc,0,ee0fdcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c803aacc,c803ac74,c0f65cf1,74,c803aad4,...) at _cv_wait+0x260 usb_process(c803aac4,ee0fdd28,c0f872f0,3db,c8045588,...) at usb_process+0x1a3 fork_exit(c08fd840,c803aac4,ee0fdd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee0fdd60, ebp = 0 --- Tracing command yarrow pid 14 tid 100017 td 0xc7ee05c0 sched_switch(c7ee05c0,0,104,1a3,fa20f37c,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7ee05c0,0,c0f925e0,28b,2,...) at sleepq_switch+0x162 sleepq_timedwait(c11292e4,0,c0f7f632,2,0,...) at sleepq_timedwait+0x6b _sleep(c11292e4,0,0,c0f7f632,64) at _sleep+0x36a pause(c0f7f632,64,c0f41ab0,111,0,...) at pause+0xa0 random_kthread(0,c7b51d28,c0f872f0,3db,c7dd0000,...) at random_kthread+0x1ef fork_exit(c0805ac0,0,c7b51d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b51d60, ebp = 0 --- Tracing command geom pid 13 tid 100015 td 0xc7ee0b80 sched_switch(c7ee0b80,0,104,1a3,8813a3,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c7ee0b80,0,c0f925e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c1126ee8,5c,c0f7f632,0,0,...) at sleepq_wait+0x63 _sleep(c1126ee8,c1126e48,25c,c0f7f632,0,...) at _sleep+0x39c g_io_schedule_down(c7ee0b80,0,c0f81181,6c,c7b4bd14,...) at g_io_schedule_down+0x5b g_down_procbody(0,c7b4bd28,c0f872f0,3db,c7dd02c4,...) at g_down_procbody+0x9d fork_exit(c09eee90,0,c7b4bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b4bd60, ebp = 0 --- Tracing command geom pid 13 tid 100014 td 0xc7ee1000 sched_switch(c7ee1000,0,104,1a3,ff936e2b,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c7ee1000,0,c0f925e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c1126ee4,5c,c0f7f632,0,0,...) at sleepq_wait+0x63 _sleep(c1126ee4,c1126e68,25c,c0f7f632,0,...) at _sleep+0x39c g_io_schedule_up(c7ee1000,0,c0f81181,5f,c7b48d14,...) at g_io_schedule_up+0x134 g_up_procbody(0,c7b48d28,c0f872f0,3db,c7dd02c4,...) at g_up_procbody+0x9d fork_exit(c09eef30,0,c7b48d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b48d60, ebp = 0 --- Tracing command geom pid 13 tid 100013 td 0xc7dd72e0 sched_switch(c7dd72e0,0,104,1a3,59ab81d,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c7dd72e0,0,c0f925e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c1126edc,5c,c0f7f632,0,0,...) at sleepq_wait+0x63 _sleep(c1126edc,c1126e0c,25c,c0f7f632,0,...) at _sleep+0x39c g_run_events(c1137140,0,c0f81181,79,c7b45d14,...) at g_run_events+0x54b g_event_procbody(0,c7b45d28,c0f872f0,3db,c7dd02c4,...) at g_event_procbody+0x90 fork_exit(c09eefd0,0,c7b45d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b45d60, ebp = 0 --- Tracing command intr pid 12 tid 100066 td 0xc818a8a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100065 td 0xc818ab80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100063 td 0xc80522e0 sched_switch(c80522e0,0,109,1a3,1464147,...) at sched_switch+0x3ec mi_switch(109,0,c0f87613,554,c80522e0,...) at mi_switch+0x1ff ithread_loop(c817bc60,ee16cd28,c0f872f0,3db,c7dd0588,...) at ithread_loop+0x1fb fork_exit(c0a318c0,c817bc60,ee16cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee16cd60, ebp = 0 --- Tracing command intr pid 12 tid 100060 td 0xc8052b80 sched_switch(c8052b80,0,109,1a3,7f57967e,...) at sched_switch+0x3ec mi_switch(109,0,c0f87613,554,c8052b80,...) at mi_switch+0x1ff ithread_loop(c8081a50,ee160d28,c0f872f0,3db,c7dd0588,...) at ithread_loop+0x1fb fork_exit(c0a318c0,c8081a50,ee160d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee160d60, ebp = 0 --- Tracing command intr pid 12 tid 100059 td 0xc805d000 sched_switch(c805d000,0,109,1a3,30d81f81,...) at sched_switch+0x3ec mi_switch(109,0,c0f87613,554,c805d000,...) at mi_switch+0x1ff ithread_loop(c80811e0,ee158d28,c0f872f0,3db,c7dd0588,...) at ithread_loop+0x1fb fork_exit(c0a318c0,c80811e0,ee158d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee158d60, ebp = 0 --- Tracing command intr pid 12 tid 100054 td 0xc8060000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100041 td 0xc8052000 sched_switch(c8052000,0,109,1a3,b4945eac,...) at sched_switch+0x3ec mi_switch(109,0,c0f87613,554,c8052000,...) at mi_switch+0x1ff ithread_loop(c802d870,ee11ad28,c0f872f0,3db,c7dd0588,...) at ithread_loop+0x1fb fork_exit(c0a318c0,c802d870,ee11ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee11ad60, ebp = 0 --- Tracing command intr pid 12 tid 100036 td 0xc8041b80 sched_switch(c8041b80,0,109,1a3,af008a9e,...) at sched_switch+0x3ec mi_switch(109,0,c0f87613,554,c8041b80,...) at mi_switch+0x1ff ithread_loop(c801c6b0,ee10ad28,c0f872f0,3db,c7dd0588,...) at ithread_loop+0x1fb fork_exit(c0a318c0,c801c6b0,ee10ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee10ad60, ebp = 0 --- Tracing command intr pid 12 tid 100031 td 0xc8044b80 sched_switch(c8044b80,0,109,1a3,9847a344,...) at sched_switch+0x3ec mi_switch(109,0,c0f87613,554,c8044b80,...) at mi_switch+0x1ff ithread_loop(c801c930,ee0fad28,c0f872f0,3db,c7dd0588,...) at ithread_loop+0x1fb fork_exit(c0a318c0,c801c930,ee0fad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee0fad60, ebp = 0 --- Tracing command intr pid 12 tid 100030 td 0xc7ee12e0 sched_switch(c7ee12e0,0,109,1a3,f7fd3a3f,...) at sched_switch+0x3ec mi_switch(109,0,c0f87613,554,c7ee12e0,...) at mi_switch+0x1ff ithread_loop(c8005b70,edfa4d28,c0f872f0,3db,c7dd0588,...) at ithread_loop+0x1fb fork_exit(c0a318c0,c8005b70,edfa4d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xedfa4d60, ebp = 0 --- Tracing command intr pid 12 tid 100029 td 0xc7ee15c0 sched_switch(c7ee15c0,0,109,1a3,fd2e1eda,...) at sched_switch+0x3ec mi_switch(109,0,c0f87613,554,c7ee15c0,...) at mi_switch+0x1ff ithread_loop(c801c060,edfa0d28,c0f872f0,3db,c7dd0588,...) at ithread_loop+0x1fb fork_exit(c0a318c0,c801c060,edfa0d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xedfa0d60, ebp = 0 --- Tracing command intr pid 12 tid 100028 td 0xc7ee18a0 sched_switch(c7ee18a0,0,109,1a3,8137ef04,...) at sched_switch+0x3ec mi_switch(109,0,c0f87613,554,c7ee18a0,...) at mi_switch+0x1ff ithread_loop(c8005550,c7ba0d28,c0f872f0,3db,c7dd0588,...) at ithread_loop+0x1fb fork_exit(c0a318c0,c8005550,c7ba0d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7ba0d60, ebp = 0 --- Tracing command intr pid 12 tid 100027 td 0xc7ee1b80 sched_switch(c7ee1b80,0,109,1a3,e3a92bda,...) at sched_switch+0x3ec mi_switch(109,0,c0f87613,554,c7ee1b80,...) at mi_switch+0x1ff ithread_loop(c7da4490,c7b6fd28,c0f872f0,3db,c7dd0588,...) at ithread_loop+0x1fb fork_exit(c0a318c0,c7da4490,c7b6fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b6fd60, ebp = 0 --- Tracing command intr pid 12 tid 100026 td 0xc7fcf000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100024 td 0xc7fcf5c0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100018 td 0xc7ee02e0 sched_switch(c7ee02e0,0,109,1a3,f7fdda98,...) at sched_switch+0x3ec mi_switch(109,0,c0f87613,554,c7ee02e0,...) at mi_switch+0x1ff ithread_loop(c7da44c0,c7b54d28,c0f872f0,3db,c7dd0588,...) at ithread_loop+0x1fb fork_exit(c0a318c0,c7da44c0,c7b54d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b54d60, ebp = 0 --- Tracing command intr pid 12 tid 100012 td 0xc7dd75c0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100011 td 0xc7dd78a0 sched_switch(c7dd78a0,0,109,1a3,6f826eda,...) at sched_switch+0x3ec mi_switch(109,0,c0f87613,554,c7dd78a0,...) at mi_switch+0x1ff ithread_loop(c7da4700,c7b3fd28,c0f872f0,3db,c7dd0588,...) at ithread_loop+0x1fb fork_exit(c0a318c0,c7da4700,c7b3fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b3fd60, ebp = 0 --- Tracing command intr pid 12 tid 100010 td 0xc7dd7b80 sched_switch(c7dd7b80,0,109,1a3,a44bdc46,...) at sched_switch+0x3ec mi_switch(109,0,c0f87613,554,c7dd7b80,...) at mi_switch+0x1ff ithread_loop(c7da4710,c7b3cd28,c0f872f0,3db,c7dd0588,...) at ithread_loop+0x1fb fork_exit(c0a318c0,c7da4710,c7b3cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b3cd60, ebp = 0 --- Tracing command intr pid 12 tid 100009 td 0xc7e1b000 sched_switch(c7e1b000,0,109,1a3,fa20cfb3,...) at sched_switch+0x3ec mi_switch(109,0,c0f87613,554,c7e1b000,...) at mi_switch+0x1ff ithread_loop(c7da4720,c7b39d28,c0f872f0,3db,c7dd0588,...) at ithread_loop+0x1fb fork_exit(c0a318c0,c7da4720,c7b39d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b39d60, ebp = 0 --- Tracing command intr pid 12 tid 100008 td 0xc7e1b2e0 sched_switch(c7e1b2e0,0,109,1a3,880e5d,...) at sched_switch+0x3ec mi_switch(109,0,c0f87613,554,c7e1b2e0,...) at mi_switch+0x1ff ithread_loop(c7da4730,c7b36d28,c0f872f0,3db,c7dd0588,...) at ithread_loop+0x1fb fork_exit(c0a318c0,c7da4730,c7b36d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b36d60, ebp = 0 --- Tracing command intr pid 12 tid 100007 td 0xc7e1b5c0 sched_switch(c7e1b5c0,0,109,1a3,e74bc882,...) at sched_switch+0x3ec mi_switch(109,0,c0f87613,554,c7e1b5c0,...) at mi_switch+0x1ff ithread_loop(c7da4740,c7b33d28,c0f872f0,3db,c7dd0588,...) at ithread_loop+0x1fb fork_exit(c0a318c0,c7da4740,c7b33d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b33d60, ebp = 0 --- Tracing command idle pid 11 tid 100006 td 0xc7dd2000 sched_switch(c7dd2000,0,108,19e,cb9a9369,...) at sched_switch+0x3ec mi_switch(108,0,c0f8eca2,3be,c7dd2000,...) at mi_switch+0x1ff sched_idletd(0,c7b2fd28,c0f872f0,3db,c7dd084c,...) at sched_idletd+0x19f fork_exit(c0a826e0,0,c7b2fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b2fd60, ebp = 0 --- Tracing command idle pid 11 tid 100005 td 0xc7dd22e0 sched_switch(c7dd22e0,0,108,19e,e5a6d754,...) at sched_switch+0x3ec mi_switch(108,0,c0f8eca2,3be,c7dd22e0,...) at mi_switch+0x1ff sched_idletd(0,c7b2cd28,c0f872f0,3db,c7dd084c,...) at sched_idletd+0x19f fork_exit(c0a826e0,0,c7b2cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b2cd60, ebp = 0 --- Tracing command idle pid 11 tid 100004 td 0xc7dd25c0 sched_switch(c7dd25c0,0,108,19e,ffd51f83,...) at sched_switch+0x3ec mi_switch(108,0,c0f8eca2,3be,c7dd25c0,...) at mi_switch+0x1ff sched_idletd(0,c7b29d28,c0f872f0,3db,c7dd084c,...) at sched_idletd+0x19f fork_exit(c0a826e0,0,c7b29d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b29d60, ebp = 0 --- Tracing command idle pid 11 tid 100003 td 0xc7dd28a0 sched_switch(c7dd28a0,0,108,19e,ff9053f9,...) at sched_switch+0x3ec mi_switch(108,0,c0f8eca2,3be,c7dd28a0,...) at mi_switch+0x1ff sched_idletd(0,c7b26d28,c0f872f0,3db,c7dd084c,...) at sched_idletd+0x19f fork_exit(c0a826e0,0,c7b26d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b26d60, ebp = 0 --- Tracing command init pid 1 tid 100002 td 0xc7dd2b80 sched_switch(c7dd2b80,0,104,1a3,37c35984,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(c7dd2b80,0,c0f925e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f925e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7dd0b10,6c,c0f954f7,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7dd0b10,c7dd0b98,16c,c0f954f7,0,...) at _sleep+0x385 kern_wait(c7dd2b80,ffffffff,c7b22c60,0,0,...) at kern_wait+0x626 sys_wait4(c7dd2b80,c7b22cec,c0fde84a,c0f9532e,286,...) at sys_wait4+0x3b syscall(c7b22d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x8060f97, esp = 0xbfbfe8bc, ebp = 0xbfbfe8d8 --- Tracing command audit pid 10 tid 100001 td 0xc7dd7000 sched_switch(c7dd7000,0,104,1a3,8f80a7a0,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7dd7000,0,c0f925e0,268,c7dd7000,...) at sleepq_switch+0x162 sleepq_wait(c12a60e0,0,c7b1fc8c,1,0,...) at sleepq_wait+0x63 _cv_wait(c12a60e0,c12a60c4,c0fbd7f7,194,0,...) at _cv_wait+0x260 audit_worker(0,c7b1fd28,c0f872f0,3db,c7dd1000,...) at audit_worker+0x84 fork_exit(c0c5b040,0,c7b1fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b1fd60, ebp = 0 --- Tracing command kernel pid 0 tid 100078 td 0xca6c08a0 sched_switch(ca6c08a0,0,104,1a3,32c3c42a,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca6c08a0,0,c0f925e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c7ee4340,0,c0f7f632,0,0,...) at sleepq_wait+0x63 _sleep(c7ee4340,c7ee4358,0,c0f7f632,0,...) at _sleep+0x39c taskqueue_thread_loop(c1307960,ee323d28,c0f872f0,3db,c1126fc0,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a9f550,c1307960,ee323d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee323d60, ebp = 0 --- Tracing command kernel pid 0 tid 100068 td 0xc818a2e0 sched_switch(c818a2e0,0,104,1a3,6f847a57,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c818a2e0,0,c0f925e0,28b,2,...) at sleepq_switch+0x162 sleepq_timedwait(c11292e4,0,c0f7f632,2,0,...) at sleepq_timedwait+0x6b _sleep(c11292e4,0,0,c0f7f632,bb8) at _sleep+0x36a pause(c0f7f632,bb8,124,122,c1127048,...) at pause+0xa0 deadlkres(0,ee185d28,c0f872f0,3db,c1126fc0,...) at deadlkres+0x324 fork_exit(c0a11090,0,ee185d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee185d60, ebp = 0 --- Tracing command kernel pid 0 tid 100061 td 0xc80528a0 sched_switch(c80528a0,0,104,1a3,fa586e2e,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c80528a0,0,c0f925e0,268,c80528a0,...) at sleepq_switch+0x162 sleepq_wait(c7ee4640,0,c0f8d941,c0f7f632,0,...) at sleepq_wait+0x63 msleep_spin(c7ee4640,c7ee4658,c0f7f632,0,c0f8b00b,...) at msleep_spin+0x259 taskqueue_thread_loop(c809449c,ee166d28,c0f872f0,3db,c1126fc0,...) at taskqueue_thread_loop+0x8f fork_exit(c0a9f550,c809449c,ee166d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee166d60, ebp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xc7fcf2e0 sched_switch(c7fcf2e0,0,104,1a3,3152ffba,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7fcf2e0,0,c0f925e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c7ee4740,0,c0f7f632,0,0,...) at sleepq_wait+0x63 _sleep(c7ee4740,c7ee4758,0,c0f7f632,0,...) at _sleep+0x39c taskqueue_thread_loop(c1145eac,c7b69d28,c0f872f0,3db,c1126fc0,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a9f550,c1145eac,c7b69d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b69d60, ebp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xc7fcf8a0 sched_switch(c7fcf8a0,0,104,1a3,a9b99743,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7fcf8a0,0,c0f925e0,268,c7fcf8a0,...) at sleepq_switch+0x162 sleepq_wait(c7ee47c0,0,c0f8d941,c0f7f632,0,...) at sleepq_wait+0x63 msleep_spin(c7ee47c0,c7ee47d8,c0f7f632,0,c0f8b00b,...) at msleep_spin+0x259 taskqueue_thread_loop(c10f1f38,c7b63d28,c0f872f0,3db,c1126fc0,...) at taskqueue_thread_loop+0x8f fork_exit(c0a9f550,c10f1f38,c7b63d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b63d60, ebp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xc7fcfb80 sched_switch(c7fcfb80,0,104,1a3,a9b985db,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7fcfb80,0,c0f925e0,268,c7fcfb80,...) at sleepq_switch+0x162 sleepq_wait(c7ee47c0,0,c0f8d941,c0f7f632,0,...) at sleepq_wait+0x63 msleep_spin(c7ee47c0,c7ee47d8,c0f7f632,0,c0f8b00b,...) at msleep_spin+0x259 taskqueue_thread_loop(c10f1f38,c7b60d28,c0f872f0,3db,c1126fc0,...) at taskqueue_thread_loop+0x8f fork_exit(c0a9f550,c10f1f38,c7b60d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b60d60, ebp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xc7e1b8a0 sched_switch(c7e1b8a0,0,104,1a3,a9b9758c,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7e1b8a0,0,c0f925e0,268,c7e1b8a0,...) at sleepq_switch+0x162 sleepq_wait(c7ee47c0,0,c0f8d941,c0f7f632,0,...) at sleepq_wait+0x63 msleep_spin(c7ee47c0,c7ee47d8,c0f7f632,0,c0f8b00b,...) at msleep_spin+0x259 taskqueue_thread_loop(c10f1f38,c7b5dd28,c0f872f0,3db,c1126fc0,...) at taskqueue_thread_loop+0x8f fork_exit(c0a9f550,c10f1f38,c7b5dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b5dd60, ebp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xc7e1bb80 sched_switch(c7e1bb80,0,104,1a3,a9b5f7cb,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7e1bb80,0,c0f925e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c7ee4800,0,c0f7f632,0,0,...) at sleepq_wait+0x63 _sleep(c7ee4800,c7ee4818,0,c0f7f632,0,...) at _sleep+0x39c taskqueue_thread_loop(c1127938,c7b5ad28,c0f872f0,3db,c1126fc0,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a9f550,c1127938,c7b5ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b5ad60, ebp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xc7ee0000 sched_switch(c7ee0000,0,104,1a3,a9b5e6ff,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7ee0000,0,c0f925e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c7ee4840,0,c0f7f632,0,0,...) at sleepq_wait+0x63 _sleep(c7ee4840,c7ee4858,0,c0f7f632,0,...) at _sleep+0x39c taskqueue_thread_loop(c12a66c4,c7b57d28,c0f872f0,3db,c1126fc0,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a9f550,c12a66c4,c7b57d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b57d60, ebp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xc7ee08a0 sched_switch(c7ee08a0,0,104,1a3,6dee72f7,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7ee08a0,0,c0f925e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c7ee4880,0,c0f7f632,0,0,...) at sleepq_wait+0x63 _sleep(c7ee4880,c7ee4898,0,c0f7f632,0,...) at _sleep+0x39c taskqueue_thread_loop(c11448c0,c7b4ed28,c0f872f0,3db,c1126fc0,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a9f550,c11448c0,c7b4ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b4ed60, ebp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xc1127290 sched_switch(c1127290,0,104,1a3,6122c125,...) at sched_switch+0x3ec mi_switch(104,0,c0f925e0,1f3,54,...) at mi_switch+0x1ff sleepq_switch(c1127290,0,c0f925e0,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c1126fc0,54,c0f8f5b4,0,0,...) at sleepq_timedwait+0x6b _sleep(c1126fc0,0,54,c0f8f5b4,2710,...) at _sleep+0x36a scheduler(0,181ec00,181ec00,181e000,1825000,...) at scheduler+0x240 mi_startup() at mi_startup+0xac begin() at begin+0x2c db:0:allt> call doadump Physical memory: 3435 MB Dumping 322 MB: 307 291 275 259 243 227 211 195 179 163 147 131 115 99 83 67 51 35 19 3 Dump complete = 0 db:0:doadump> reset (kgdb) bt #0 doadump (textdump=0xf2869374) at pcpu.h:244 #1 0xc0510a89 in db_fncall (dummy1=0xc0a90c7a, dummy2=0x0, dummy3=0xffffffff, dummy4=0xf2869408 "\034\224\206ò") at ../../../ddb/db_command.c:573 #2 0xc0510ebf in db_command (last_cmdp=0xc10f00fc, cmd_table=0x0, dopager=0x0) at ../../../ddb/db_command.c:449 #3 0xc0510f74 in db_command_script (command=0xc10f1008 "call doadump") at ../../../ddb/db_command.c:520 #4 0xc0515360 in db_script_exec (scriptname=0xc10f0960 "doadump", warnifnotfound=Variable "warnifnotfound" is not available. ) at ../../../ddb/db_script.c:302 #5 0xc05153f1 in db_run_cmd (addr=0x1, have_addr=0x0, count=0xc12d3260, modif=0xf2869540 "") at ../../../ddb/db_script.c:375 #6 0xc0510e81 in db_command (last_cmdp=0xc10f00fc, cmd_table=0x0, dopager=0x1) at ../../../ddb/db_command.c:449 #7 0xc0510fda in db_command_loop () at ../../../ddb/db_command.c:502 #8 0xc0512fcd in db_trap (type=0x3, code=0x0) at ../../../ddb/db_main.c:229 #9 0xc0a91136 in kdb_trap (type=0x3, code=0x0, tf=0xf28696f4) at ../../../kern/subr_kdb.c:629 #10 0xc0dc047b in trap (frame=0xf28696f4) at ../../../i386/i386/trap.c:722 #11 0xc0da8d0c in calltrap () at ../../../i386/i386/exception.s:168 #12 0xc0a90c7a in kdb_enter (why=0xc0f8cbd8 "panic", msg=0xc0f8cbd8 "panic") at cpufunc.h:71 #13 0xc0a5bb4c in panic (fmt=0xc0fc0b28 "check_inode_unwritten: busy inode") at ../../../kern/kern_shutdown.c:633 #14 0xc0c87659 in check_inode_unwritten (inodedep=0xcd867700) at ../../../ufs/ffs/ffs_softdep.c:7225 #15 0xc0c9478c in softdep_freefile (pvp=0xcb1ec2b8, ino=0xc50, mode=0x81b0) at ../../../ufs/ffs/ffs_softdep.c:7159 #16 0xc0c7053f in ffs_vfree (pvp=0xcb1ec2b8, ino=0xc50, mode=0x81b0) at ../../../ufs/ffs/ffs_alloc.c:2139 #17 0xc0c71d18 in ffs_valloc (pvp=0xcb1ec2b8, mode=0x81b0, cred=0xca7c5780, vpp=0xf2869914) at ../../../ufs/ffs/ffs_alloc.c:980 #18 0xc0cb4cd3 in ufs_makeinode (mode=0x81b0, dvp=0xcb1ec2b8, vpp=0xf2869bac, cnp=0xf2869bc0) at ../../../ufs/ufs/ufs_vnops.c:2585 #19 0xc0cb5590 in ufs_create (ap=0xf2869b04) at ../../../ufs/ufs/ufs_vnops.c:205 #20 0xc0de3d75 in VOP_CREATE_APV (vop=0xc10dcce0, a=0xf2869b04) at vnode_if.c:250 #21 0xc0b08815 in vn_open_cred (ndp=0xf2869b80, flagp=0xf2869c48, cmode=Variable "cmode" is not available. ) at vnode_if.h:109 #22 0xc0b08c2b in vn_open (ndp=0xf2869b80, flagp=0xf2869c48, cmode=0x1b0, fp=0xcac91af0) at ../../../kern/vfs_vnops.c:101 #23 0xc0b04e7c in kern_openat (td=0xcac00000, fd=0xffffff9c, path=0xbfbfe758
, pathseg=UIO_USERSPACE, flags=0x602, mode=0x1b0) at ../../../kern/vfs_syscalls.c:1133 #24 0xc0b05325 in kern_open (td=0xcac00000, path=0xbfbfe758
, pathseg=UIO_USERSPACE, flags=0x601, mode=0x1b0) at ../../../kern/vfs_syscalls.c:1084 #25 0xc0b05360 in sys_open (td=0xcac00000, uap=0xf2869cec) at ../../../kern/vfs_syscalls.c:1060 #26 0xc0dbfa47 in syscall (frame=0xf2869d28) at subr_syscall.c:131 #27 0xc0da8d71 in Xint0x80_syscall () at ../../../i386/i386/exception.s:266 #28 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) f 14 #14 0xc0c87659 in check_inode_unwritten (inodedep=0xcd867700) at ../../../ufs/ffs/ffs_softdep.c:7225 7225 panic("check_inode_unwritten: busy inode"); (kgdb) l 7220 if (inodedep->id_savedino1 != NULL) { 7221 free(inodedep->id_savedino1, M_SAVEDINO); 7222 inodedep->id_savedino1 = NULL; 7223 } 7224 if (free_inodedep(inodedep) == 0) 7225 panic("check_inode_unwritten: busy inode"); 7226 return (1); 7227 } 7228 7229 /* (kgdb) info loc No locals. (kgdb) p *inodedep $1 = {id_list = {wk_list = {le_next = 0x70707070, le_prev = 0x70707070}, wk_mp = 0xcaa69000, wk_type = 0x1, wk_state = 0xd}, id_hash = {le_next = 0xcb7f1300, le_prev = 0xca516920}, id_unlinked = {tqe_next = 0x70707070, tqe_prev = 0x70707070}, id_fs = 0xcb3f0000, id_ino = 0xc50, id_nlinkdelta = 0x0, id_savednlink = 0xffff, id_deps = {le_next = 0x70707070, le_prev = 0x70707070}, id_bmsafemap = 0x0, id_mkdiradd = 0x0, id_inoreflst = {tqh_first = 0xca81a0c0, tqh_last = 0xca81a0d0}, id_savedextsize = 0xffffffff, id_savedsize = 0xffffffffffffffff, id_dirremhd = {lh_first = 0x0}, id_pendinghd = {lh_first = 0x0}, id_bufwait = {lh_first = 0x0}, id_inowait = {lh_first = 0x0}, id_inoupdt = {tqh_first = 0x0, tqh_last = 0xcd867760}, id_newinoupdt = {tqh_first = 0x0, tqh_last = 0xcd867768}, id_extupdt = {tqh_first = 0x0, tqh_last = 0xcd867770}, id_newextupdt = {tqh_first = 0x0, tqh_last = 0xcd867778}, id_freeblklst = { tqh_first = 0x0, tqh_last = 0xcd867780}, id_un = {idu_savedino1 = 0x0, idu_savedino2 = 0x0}} (kgdb) f 15 #15 0xc0c9478c in softdep_freefile (pvp=0xcb1ec2b8, ino=0xc50, mode=0x81b0) at ../../../ufs/ffs/ffs_softdep.c:7159 7159 if (inodedep == NULL || check_inode_unwritten(inodedep)) { (kgdb) l 7154 clear_unlinked_inodedep(inodedep); 7155 /* Re-acquire inodedep as we've dropped lk. */ 7156 inodedep_lookup(pvp->v_mount, ino, 0, &inodedep); 7157 } 7158 } 7159 if (inodedep == NULL || check_inode_unwritten(inodedep)) { 7160 FREE_LOCK(&lk); 7161 handle_workitem_freefile(freefile); 7162 return; 7163 } (kgdb) p *pvp $2 = {v_type = VDIR, v_tag = 0xc0f8322c "ufs", v_op = 0xc10dc760, v_data = 0xcb1df89c, v_mount = 0xcaa69000, v_nmntvnodes = {tqe_next = 0xcb63c984, tqe_prev = 0xcb3e7c50}, v_un = {vu_mount = 0x0, vu_socket = 0x0, vu_cdev = 0x0, vu_fifoinfo = 0x0}, v_hashlist = {le_next = 0xcb2d2000, le_prev = 0xca452c00}, v_hash = 0x30002, v_cache_src = {lh_first = 0x0}, v_cache_dst = {tqh_first = 0x0, tqh_last = 0xcb1ec2e8}, v_cache_dd = 0x0, v_cstart = 0x0, v_lasta = 0x0, v_lastw = 0x0, v_clen = 0x0, v_lock = {lock_object = {lo_name = 0xc0f8322c "ufs", lo_flags = 0x57b0000, lo_data = 0x0, lo_witness = 0xc7d72610}, lk_lock = 0xcac00000, lk_exslpfail = 0x0, lk_timo = 0x33, lk_pri = 0x60, lk_stack = {depth = 0xd, pcs = {0xc0a42c74, 0xc0ca4cd1, 0xc0de2da5, 0xc0b09158, 0xc0aec496, 0xc0aeda9f, 0xc0b086a0, 0xc0b08c2b, 0xc0b04e7c, 0xc0b05325, 0xc0b05360, 0xc0dbfa47, 0xc0da8d71, 0x0, 0x0, 0x0, 0x0, 0x0}}}, v_interlock = { lock_object = {lo_name = 0xc0f94898 "vnode interlock", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc7d6b1e0}, mtx_lock = 0x4}, v_vnlock = 0xcb1ec310, v_holdcnt = 0x3, v_usecount = 0x2, v_iflag = 0x0, v_vflag = 0x0, v_writecount = 0x0, v_freelist = {tqe_next = 0x0, tqe_prev = 0x0}, v_bufobj = {bo_mtx = { lock_object = {lo_name = 0xc0f9d5b2 "bufobj interlock", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc7d70c10}, mtx_lock = 0x4}, bo_clean = {bv_hd = { tqh_first = 0x0, tqh_last = 0xcb1ec3c4}, bv_root = 0x0, bv_cnt = 0x0}, bo_dirty = {bv_hd = {tqh_first = 0xe1b12ca0, tqh_last = 0xe1b12cd8}, bv_root = 0xe1b12ca0, bv_cnt = 0x1}, bo_numoutput = 0x0, bo_flag = 0x1, bo_ops = 0xc10c99e0, bo_bsize = 0x8000, bo_object = 0xcb3fbd80, bo_synclist = { le_next = 0x0, le_prev = 0xca429464}, bo_private = 0xcb1ec2b8, __bo_vnode = 0xcb1ec2b8}, v_pollinfo = 0x0, v_label = 0x0, v_lockf = 0x0} (kgdb) p *pvp->v_mount $3 = {mnt_mtx = {lock_object = {lo_name = 0xc0f7edf1 "struct mount mtx", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc7d6b450}, mtx_lock = 0x4}, mnt_gen = 0x1, mnt_list = {tqe_next = 0xca726b50, tqe_prev = 0xcaa692ec}, mnt_op = 0xc10dc500, mnt_vfc = 0xc10dc360, mnt_vnodecovered = 0xcaf396cc, mnt_syncer = 0x0, mnt_ref = 0x1ac4, mnt_nvnodelist = {tqh_first = 0xcb2d2000, tqh_last = 0xcb7b183c}, mnt_nvnodelistsize = 0x1ac0, mnt_writeopcount = 0x3, mnt_kern_flag = 0x6100004d, mnt_flag = 0x100001100, mnt_noasync = 0x1, mnt_opt = 0xca6f43e0, mnt_optnew = 0x0, mnt_maxsymlinklen = 0x78, mnt_stat = { f_version = 0x20030518, f_type = 0x35, f_flags = 0x100001100, f_bsize = 0x1000, f_iosize = 0x8000, f_blocks = 0x7bfa5, f_bfree = 0x7aec4, f_bavail = 0x71013, f_files = 0x3fffe, f_ffree = 0x3fffa, f_syncwrites = 0x0, f_asyncwrites = 0x0, f_syncreads = 0x0, f_asyncreads = 0x0, f_spare = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, f_namemax = 0xff, f_owner = 0x0, f_fsid = {val = {0x4f366c64, 0x8f8aa4b5}}, f_charspare = '\0' , f_fstypename = "ufs", '\0' , f_mntfromname = "/dev/md5a", '\0' , f_mntonname = "/mnt", '\0' }, mnt_cred = 0xca7c5c00, mnt_data = 0xcafe2d00, mnt_time = 0x0, mnt_iosize_max = 0x20000, mnt_export = 0xca6bdd00, mnt_label = 0x0, mnt_hashseed = 0x3ebca2fe, mnt_lockref = 0x0, mnt_secondary_writes = 0x0, mnt_secondary_accwrites = 0x310ec, mnt_susp_owner = 0x0, mnt_gjprovider = 0x0, mnt_explock = {lock_object = { lo_name = 0xc0f7ee02 "explock", lo_flags = 0x5730000, lo_data = 0x0, lo_witness = 0xc7d710f0}, lk_lock = 0x1, lk_exslpfail = 0x0, lk_timo = 0x0, lk_pri = 0x60, lk_stack = {depth = 0x6, pcs = {0xc0a432ce, 0xc0ae89f7, 0xc0af128e, 0xc0af18ef, 0xc0dbfa47, 0xc0da8d71, 0x0 }}}} (kgdb) info loc ip = (struct inode *) 0xcb1df89c inodedep = (struct inodedep *) 0xcd867700 freefile = (struct freefile *) 0xcd6c5000 freeblks = Variable "freeblks" is not available. (kgdb) p *ip $4 = {i_nextsnap = {tqe_next = 0x0, tqe_prev = 0x0}, i_vnode = 0xcb1ec2b8, i_ump = 0xcafe2d00, i_flag = 0x6, i_dev = 0xcaa51800, i_number = 0x30002, i_effnlink = 0x2, i_fs = 0xcb3f0000, i_dquot = {0x0, 0x0}, i_count = 0x68, i_endoff = 0x1400, i_diroff = 0x0, i_offset = 0x1398, i_un = {dirhash = 0xcb1ed200, snapblklist = 0xcb1ed200}, i_ea_area = 0x0, i_ea_len = 0x0, i_ea_error = 0x0, i_ea_refs = 0x0, i_mode = 0x41f8, i_nlink = 0x2, i_size = 0x1400, i_flags = 0x0, i_gen = 0xffffffffdb3d2b6e, i_uid = 0x3ec, i_gid = 0x0, dinode_u = {din1 = 0xcb353000, din2 = 0xcb353000}} (kgdb) $ svn diff -x -p /usr/src/sys Index: /usr/src/sys/ufs/ufs/inode.h =================================================================== --- /usr/src/sys/ufs/ufs/inode.h (revision 231456) +++ /usr/src/sys/ufs/ufs/inode.h (working copy) @@ -158,6 +158,7 @@ struct inode { #define SHORTLINK(ip) \ (((ip)->i_ump->um_fstype == UFS1) ? \ (caddr_t)(ip)->i_din1->di_db : (caddr_t)(ip)->i_din2->di_db) +#define IS_SNAPSHOT(ip) ((ip)->i_flags & SF_SNAPSHOT) /* * Structure used to pass around logical block paths generated by Index: /usr/src/sys/ufs/ufs/ufs_quota.c =================================================================== --- /usr/src/sys/ufs/ufs/ufs_quota.c (revision 231456) +++ /usr/src/sys/ufs/ufs/ufs_quota.c (working copy) @@ -80,7 +80,7 @@ static int dqopen(struct vnode *, struct ufsmount static int dqget(struct vnode *, u_long, struct ufsmount *, int, struct dquot **); static int dqsync(struct vnode *, struct dquot *); -static void dqflush(struct vnode *); +static int dqflush(struct vnode *); static int quotaoff1(struct thread *td, struct mount *mp, int type); static int quotaoff_inchange(struct thread *td, struct mount *mp, int type); @@ -694,7 +694,9 @@ again: } MNT_IUNLOCK(mp); - dqflush(qvp); + if ((error = dqflush(qvp)) != 0) + return (error); + /* Clear um_quotas before closing the quota vnode to prevent * access to the closed vnode from dqget/dqsync */ @@ -1601,17 +1603,19 @@ out: /* * Flush all entries from the cache for a particular vnode. */ -static void +static int dqflush(struct vnode *vp) { struct dquot *dq, *nextdq; struct dqhash *dqh; + int error; /* * Move all dquot's that used to refer to this quota * file off their hash chains (they will eventually * fall off the head of the free list and be re-used). */ + error = 0; DQH_LOCK(); for (dqh = &dqhashtbl[dqhash]; dqh >= dqhashtbl; dqh--) { for (dq = LIST_FIRST(dqh); dq; dq = nextdq) { @@ -1619,12 +1623,13 @@ dqflush(struct vnode *vp) if (dq->dq_ump->um_quotas[dq->dq_type] != vp) continue; if (dq->dq_cnt) - panic("dqflush: stray dquot"); + error = EBUSY; LIST_REMOVE(dq, dq_hash); dq->dq_ump = (struct ufsmount *)0; } } DQH_UNLOCK(); + return (error); } /* Index: /usr/src/sys/ufs/ufs/ufs_vfsops.c =================================================================== --- /usr/src/sys/ufs/ufs/ufs_vfsops.c (revision 231456) +++ /usr/src/sys/ufs/ufs/ufs_vfsops.c (working copy) @@ -125,6 +125,14 @@ ufs_quotactl(mp, cmds, id, arg) case Q_QUOTAOFF: error = quotaoff(td, mp, type); + if (error == EBUSY) { + /* + * If busy, flush the filesystem and try again. + * If still busy, return error. + */ + (void) VFS_SYNC(mp, MNT_WAIT); + error = quotaoff(td, mp, type); + } break; case Q_SETQUOTA32: Index: /usr/src/sys/ufs/ffs/ffs_softdep.c =================================================================== --- /usr/src/sys/ufs/ffs/ffs_softdep.c (revision 231456) +++ /usr/src/sys/ufs/ffs/ffs_softdep.c (working copy) @@ -1895,11 +1895,21 @@ retry_flush: * Do another flush in case any vnodes were brought in * as part of the cleanup operations. */ - if ((error = ffs_flushfiles(oldmnt, flags, td)) != 0) + if ((error = ffs_flushfiles(oldmnt, flags, td)) != 0 && + error != EBUSY) + return (error); + if ((error = softdep_flushworklist(oldmnt, &depcount, td)) != 0) + return (error); + /* + * Normally ffs_flushfiles() will shut down quotas. If + * it finds quotas still referenced it will return EBUSY. + * When we are running with journaled soft updates we may + * have to flush the worklist (softdep_flushworklist()) to + * clear the quota references. Thus we cannot exit this loop + * until quotas are shut down and the worklist is clear. + */ + if ((oldmnt->mnt_flag & MNT_QUOTA) == 0 && depcount == 0) break; - if ((error = softdep_flushworklist(oldmnt, &depcount, td)) != 0 || - depcount == 0) - break; } /* * If we are unmounting then it is an error to fail. If we @@ -1909,27 +1919,24 @@ retry_flush: if (loopcnt == 0) { if (oldmnt->mnt_kern_flag & MNTK_UNMOUNT) panic("softdep_flushfiles: looping"); - error = EBUSY; + return (EBUSY); } - if (!error) - error = softdep_waitidle(oldmnt); - if (!error) { - if (oldmnt->mnt_kern_flag & MNTK_UNMOUNT) { - retry = 0; - MNT_ILOCK(oldmnt); - KASSERT((oldmnt->mnt_kern_flag & MNTK_NOINSMNTQ) != 0, - ("softdep_flushfiles: !MNTK_NOINSMNTQ")); - if (oldmnt->mnt_nvnodelistsize > 0) { - if (--retry_flush_count > 0) { - retry = 1; - loopcnt = 3; - } else - error = EBUSY; - } - MNT_IUNLOCK(oldmnt); - if (retry) - goto retry_flush; + if ((error = softdep_waitidle(oldmnt)) == 0 && + (oldmnt->mnt_kern_flag & MNTK_UNMOUNT) != 0) { + retry = 0; + MNT_ILOCK(oldmnt); + KASSERT((oldmnt->mnt_kern_flag & MNTK_NOINSMNTQ) != 0, + ("softdep_flushfiles: !MNTK_NOINSMNTQ")); + if (oldmnt->mnt_nvnodelistsize > 0) { + if (--retry_flush_count > 0) { + retry = 1; + loopcnt = 3; + } else + error = EBUSY; } + MNT_IUNLOCK(oldmnt); + if (retry) + goto retry_flush; } return (error); } @@ -2827,7 +2834,12 @@ softdep_prealloc(vp, waitok) { struct ufsmount *ump; - if (DOINGSUJ(vp) == 0) + /* + * Nothing to do if we are not running journaled soft updates. + * If we currently hold the snapshot lock, we must avoid handling + * other resources that could cause deadlock. + */ + if (DOINGSUJ(vp) == 0 || IS_SNAPSHOT(VTOI(vp))) return (0); ump = VFSTOUFS(vp->v_mount); ACQUIRE_LOCK(&lk); @@ -2873,7 +2885,12 @@ softdep_prelink(dvp, vp) ump = VFSTOUFS(dvp->v_mount); mtx_assert(&lk, MA_OWNED); - if (journal_space(ump, 0)) + /* + * Nothing to do if we have sufficient journal space. + * If we currently hold the snapshot lock, we must avoid + * handling other resources that could cause deadlock. + */ + if (journal_space(ump, 0) || (vp && IS_SNAPSHOT(VTOI(vp)))) return; stat_journal_low++; FREE_LOCK(&lk); @@ -4304,11 +4321,15 @@ inodedep_lookup_ip(ip) struct inode *ip; { struct inodedep *inodedep; + int dflags; KASSERT(ip->i_nlink >= ip->i_effnlink, ("inodedep_lookup_ip: bad delta")); - (void) inodedep_lookup(UFSTOVFS(ip->i_ump), ip->i_number, - DEPALLOC, &inodedep); + dflags = DEPALLOC; + if (IS_SNAPSHOT(ip)) + dflags |= NODELAY; + (void) inodedep_lookup(UFSTOVFS(ip->i_ump), ip->i_number, dflags, + &inodedep); inodedep->id_nlinkdelta = ip->i_nlink - ip->i_effnlink; return (inodedep); @@ -4696,7 +4717,7 @@ softdep_setup_inomapdep(bp, ip, newinum, mode) * the cylinder group map from which it was allocated. */ ACQUIRE_LOCK(&lk); - if ((inodedep_lookup(mp, newinum, DEPALLOC|NODELAY, &inodedep))) + if ((inodedep_lookup(mp, newinum, DEPALLOC | NODELAY, &inodedep))) panic("softdep_setup_inomapdep: dependency %p for new" "inode already exists", inodedep); bmsafemap = bmsafemap_lookup(mp, bp, ino_to_cg(fs, newinum)); @@ -5437,6 +5458,7 @@ softdep_setup_allocindir_page(ip, lbn, bp, ptrno, struct allocindir *aip; struct pagedep *pagedep; struct mount *mp; + int dflags; if (lbn != nbp->b_lblkno) panic("softdep_setup_allocindir_page: lbn %jd != lblkno %jd", @@ -5444,7 +5466,10 @@ softdep_setup_allocindir_page(ip, lbn, bp, ptrno, ASSERT_VOP_LOCKED(ITOV(ip), "softdep_setup_allocindir_page"); mp = UFSTOVFS(ip->i_ump); aip = newallocindir(ip, ptrno, newblkno, oldblkno, lbn); - (void) inodedep_lookup(mp, ip->i_number, DEPALLOC, &inodedep); + dflags = DEPALLOC; + if (IS_SNAPSHOT(ip)) + dflags |= NODELAY; + (void) inodedep_lookup(mp, ip->i_number, dflags, &inodedep); /* * If we are allocating a directory page, then we must * allocate an associated pagedep to track additions and @@ -5474,11 +5499,15 @@ softdep_setup_allocindir_meta(nbp, ip, bp, ptrno, struct inodedep *inodedep; struct allocindir *aip; ufs_lbn_t lbn; + int dflags; lbn = nbp->b_lblkno; ASSERT_VOP_LOCKED(ITOV(ip), "softdep_setup_allocindir_meta"); aip = newallocindir(ip, ptrno, newblkno, 0, lbn); - inodedep_lookup(UFSTOVFS(ip->i_ump), ip->i_number, DEPALLOC, &inodedep); + dflags = DEPALLOC; + if (IS_SNAPSHOT(ip)) + dflags |= NODELAY; + inodedep_lookup(UFSTOVFS(ip->i_ump), ip->i_number, dflags, &inodedep); WORKLIST_INSERT(&nbp->b_dep, &aip->ai_block.nb_list); if (setup_allocindir_phase2(bp, ip, inodedep, aip, lbn)) panic("softdep_setup_allocindir_meta: Block already existed"); @@ -6085,11 +6114,7 @@ softdep_journal_freeblocks(ip, cred, length, flags struct mount *mp; ufs2_daddr_t extblocks, datablocks; ufs_lbn_t tmpval, lbn, lastlbn; - int frags; - int lastoff, iboff; - int allocblock; - int error, i; - int needj; + int frags, lastoff, iboff, allocblock, needj, dflags, error, i; fs = ip->i_fs; mp = UFSTOVFS(ip->i_ump); @@ -6107,7 +6132,10 @@ softdep_journal_freeblocks(ip, cred, length, flags * we don't need to journal the block frees. The canceled journals * for the allocations will suffice. */ - inodedep_lookup(mp, ip->i_number, DEPALLOC, &inodedep); + dflags = DEPALLOC; + if (IS_SNAPSHOT(ip)) + dflags |= NODELAY; + inodedep_lookup(mp, ip->i_number, dflags, &inodedep); if ((inodedep->id_state & (UNLINKED | DEPCOMPLETE)) == UNLINKED && length == 0) needj = 0; @@ -6232,7 +6260,7 @@ softdep_journal_freeblocks(ip, cred, length, flags *((struct ufs2_dinode *)bp->b_data + ino_to_fsbo(fs, ip->i_number)) = *ip->i_din2; ACQUIRE_LOCK(&lk); - (void) inodedep_lookup(mp, ip->i_number, DEPALLOC, &inodedep); + (void) inodedep_lookup(mp, ip->i_number, dflags, &inodedep); if ((inodedep->id_state & IOSTARTED) != 0) panic("softdep_setup_freeblocks: inode busy"); /* @@ -6310,7 +6338,7 @@ softdep_journal_freeblocks(ip, cred, length, flags } ACQUIRE_LOCK(&lk); - inodedep_lookup(mp, ip->i_number, DEPALLOC, &inodedep); + inodedep_lookup(mp, ip->i_number, dflags, &inodedep); TAILQ_INSERT_TAIL(&inodedep->id_freeblklst, freeblks, fb_next); freeblks->fb_state |= DEPCOMPLETE | ONDEPLIST; /* @@ -6398,7 +6426,7 @@ softdep_setup_freeblocks(ip, length, flags) struct fs *fs; ufs2_daddr_t extblocks, datablocks; struct mount *mp; - int i, delay, error; + int i, delay, error, dflags; ufs_lbn_t tmpval; ufs_lbn_t lbn; @@ -6463,7 +6491,10 @@ softdep_setup_freeblocks(ip, length, flags) * Find and eliminate any inode dependencies. */ ACQUIRE_LOCK(&lk); - (void) inodedep_lookup(mp, ip->i_number, DEPALLOC, &inodedep); + dflags = DEPALLOC; + if (IS_SNAPSHOT(ip)) + dflags |= NODELAY; + (void) inodedep_lookup(mp, ip->i_number, dflags, &inodedep); if ((inodedep->id_state & IOSTARTED) != 0) panic("softdep_setup_freeblocks: inode busy"); /* @@ -8026,7 +8057,7 @@ softdep_setup_directory_add(bp, dp, diroffset, new dap->da_pagedep = pagedep; LIST_INSERT_HEAD(&pagedep->pd_diraddhd[DIRADDHASH(offset)], dap, da_pdlist); - inodedep_lookup(mp, newinum, DEPALLOC, &inodedep); + inodedep_lookup(mp, newinum, DEPALLOC | NODELAY, &inodedep); /* * If we're journaling, link the diradd into the jaddref so it * may be completed after the journal entry is written. Otherwise, @@ -8628,8 +8659,7 @@ newdirrem(bp, dp, ip, isrmdir, prevdirremp) * the number of freefile and freeblks structures. */ ACQUIRE_LOCK(&lk); - if (!(ip->i_flags & SF_SNAPSHOT) && - dep_current[D_DIRREM] > max_softdeps / 2) + if (!IS_SNAPSHOT(ip) && dep_current[D_DIRREM] > max_softdeps / 2) (void) request_cleanup(ITOV(dp)->v_mount, FLUSH_BLOCKS); FREE_LOCK(&lk); dirrem = malloc(sizeof(struct dirrem), @@ -8863,11 +8893,11 @@ softdep_setup_directory_change(bp, dp, ip, newinum /* * Lookup the jaddref for this journal entry. We must finish * initializing it and make the diradd write dependent on it. - * If we're not journaling Put it on the id_bufwait list if the inode - * is not yet written. If it is written, do the post-inode write - * processing to put it on the id_pendinghd list. + * If we're not journaling, put it on the id_bufwait list if the + * inode is not yet written. If it is written, do the post-inode + * write processing to put it on the id_pendinghd list. */ - inodedep_lookup(mp, newinum, DEPALLOC, &inodedep); + inodedep_lookup(mp, newinum, DEPALLOC | NODELAY, &inodedep); if (MOUNTEDSUJ(mp)) { jaddref = (struct jaddref *)TAILQ_LAST(&inodedep->id_inoreflst, inoreflst); @@ -8909,9 +8939,13 @@ softdep_change_linkcnt(ip) struct inode *ip; /* the inode with the increased link count */ { struct inodedep *inodedep; + int dflags; ACQUIRE_LOCK(&lk); - inodedep_lookup(UFSTOVFS(ip->i_ump), ip->i_number, DEPALLOC, &inodedep); + dflags = DEPALLOC; + if (IS_SNAPSHOT(ip)) + dflags |= NODELAY; + inodedep_lookup(UFSTOVFS(ip->i_ump), ip->i_number, dflags, &inodedep); if (ip->i_nlink < ip->i_effnlink) panic("softdep_change_linkcnt: bad delta"); inodedep->id_nlinkdelta = ip->i_nlink - ip->i_effnlink; @@ -12541,22 +12575,25 @@ softdep_request_cleanup(fs, vp, cred, resource) ufs2_daddr_t needed; int error; - mp = vp->v_mount; - ump = VFSTOUFS(mp); - mtx_assert(UFS_MTX(ump), MA_OWNED); - if (resource == FLUSH_BLOCKS_WAIT) - stat_cleanup_blkrequests += 1; - else - stat_cleanup_inorequests += 1; - /* * If we are being called because of a process doing a * copy-on-write, then it is not safe to process any * worklist items as we will recurse into the copyonwrite * routine. This will result in an incoherent snapshot. + * If the vnode that we hold is a snapshot, we must avoid + * handling other resources that could cause deadlock. */ - if (curthread->td_pflags & TDP_COWINPROGRESS) + if ((curthread->td_pflags & TDP_COWINPROGRESS) || IS_SNAPSHOT(VTOI(vp))) return (0); + + if (resource == FLUSH_BLOCKS_WAIT) + stat_cleanup_blkrequests += 1; + else + stat_cleanup_inorequests += 1; + + mp = vp->v_mount; + ump = VFSTOUFS(mp); + mtx_assert(UFS_MTX(ump), MA_OWNED); UFS_UNLOCK(ump); error = ffs_update(vp, 1); if (error != 0) { Index: /usr/src/sys/ufs/ffs/ffs_vnops.c =================================================================== --- /usr/src/sys/ufs/ffs/ffs_vnops.c (revision 231456) +++ /usr/src/sys/ufs/ffs/ffs_vnops.c (working copy) @@ -216,8 +216,10 @@ ffs_syncvnode(struct vnode *vp, int waitfor) struct buf *bp; struct buf *nbp; ufs_lbn_t lbn; - int error, wait, passes; + int error, wait, passes, noupdate; + noupdate = waitfor & NO_INO_UPDT; + waitfor &= ~NO_INO_UPDT; ip = VTOI(vp); ip->i_flag &= ~IN_NEEDSYNC; bo = &vp->v_bufobj; @@ -300,7 +302,10 @@ next: } if (waitfor != MNT_WAIT) { BO_UNLOCK(bo); - return (ffs_update(vp, waitfor)); + if (noupdate) + return (0); + else + return (ffs_update(vp, waitfor)); } /* Drain IO to see if we're done. */ bufobj_wwait(bo, 0, 0); @@ -317,7 +322,7 @@ next: */ if (bo->bo_dirty.bv_cnt > 0) { /* Write the inode after sync passes to flush deps. */ - if (wait && DOINGSOFTDEP(vp)) { + if (wait && DOINGSOFTDEP(vp) && noupdate == 0) { BO_UNLOCK(bo); ffs_update(vp, MNT_WAIT); BO_LOCK(bo); @@ -332,7 +337,9 @@ next: #endif } BO_UNLOCK(bo); - error = ffs_update(vp, MNT_WAIT); + error = 0; + if (noupdate == 0) + error = ffs_update(vp, MNT_WAIT); if (DOINGSUJ(vp)) softdep_journal_fsync(VTOI(vp)); return (error); Index: /usr/src/sys/ufs/ffs/ffs_extern.h =================================================================== --- /usr/src/sys/ufs/ffs/ffs_extern.h (revision 231456) +++ /usr/src/sys/ufs/ffs/ffs_extern.h (working copy) @@ -167,6 +167,13 @@ void softdep_freework(struct workhead *); #define FLUSH_INODES_WAIT 2 #define FLUSH_BLOCKS 3 #define FLUSH_BLOCKS_WAIT 4 +/* + * Flag to ffs_syncinode() to request flushing of data only, + * but skip the ffs_update() on the inode itself. Used to avoid + * deadlock when flushing snapshot inodes while holding snaplk. + * Avoid bit conflicts with MNT_WAIT values in sys/mount.h + */ +#define NO_INO_UPDT 0x10 int ffs_rdonly(struct inode *); Index: /usr/src/sys/ufs/ffs/ffs_vfsops.c =================================================================== --- /usr/src/sys/ufs/ffs/ffs_vfsops.c (revision 231456) +++ /usr/src/sys/ufs/ffs/ffs_vfsops.c (working copy) @@ -1341,13 +1341,17 @@ ffs_flushfiles(mp, flags, td) ump = VFSTOUFS(mp); #ifdef QUOTA if (mp->mnt_flag & MNT_QUOTA) { - int i; - error = vflush(mp, 0, SKIPSYSTEM|flags, td); - if (error) + int i, allerror; + + if ((error = vflush(mp, 0, SKIPSYSTEM|flags, td)) != 0) return (error); for (i = 0; i < MAXQUOTAS; i++) { - quotaoff(td, mp, i); + allerror = quotaoff(td, mp, i); + if (allerror != 0 && (error == 0 || error == EBUSY)) + error = allerror; } + if (error) + return (error); /* * Here we fall through to vflush again to ensure * that we have gotten rid of all the system vnodes. Index: /usr/src/sys/ufs/ffs/ffs_snapshot.c =================================================================== --- /usr/src/sys/ufs/ffs/ffs_snapshot.c (revision 231456) +++ /usr/src/sys/ufs/ffs/ffs_snapshot.c (working copy) @@ -225,18 +225,10 @@ ffs_snapshot(mp, snapfile) ump = VFSTOUFS(mp); fs = ump->um_fs; sn = NULL; - /* - * At the moment, journaled soft updates cannot support - * taking snapshots. - */ - if (MOUNTEDSUJ(mp)) { - vfs_mount_error(mp, "%s: Snapshots are not yet supported when " - "running with journaled soft updates", fs->fs_fsmnt); - return (EOPNOTSUPP); - } MNT_ILOCK(mp); flag = mp->mnt_flag; MNT_IUNLOCK(mp); + /* * Need to serialize access to snapshot code per filesystem. */ @@ -1902,7 +1894,7 @@ retry: bawrite(cbp); if ((vtype == VDIR || dopersistence) && ip->i_effnlink > 0) - (void) ffs_syncvnode(vp, MNT_WAIT); + (void) ffs_syncvnode(vp, MNT_WAIT|NO_INO_UPDT); continue; } /* @@ -1913,7 +1905,7 @@ retry: bawrite(cbp); if ((vtype == VDIR || dopersistence) && ip->i_effnlink > 0) - (void) ffs_syncvnode(vp, MNT_WAIT); + (void) ffs_syncvnode(vp, MNT_WAIT|NO_INO_UPDT); break; } savedcbp = cbp; @@ -1931,7 +1923,7 @@ retry: bawrite(savedcbp); if ((vtype == VDIR || dopersistence) && VTOI(vp)->i_effnlink > 0) - (void) ffs_syncvnode(vp, MNT_WAIT); + (void) ffs_syncvnode(vp, MNT_WAIT|NO_INO_UPDT); } /* * If we have been unable to allocate a block in which to do @@ -2395,7 +2387,7 @@ ffs_copyonwrite(devvp, bp) bawrite(cbp); if ((devvp == bp->b_vp || bp->b_vp->v_type == VDIR || dopersistence) && ip->i_effnlink > 0) - (void) ffs_syncvnode(vp, MNT_WAIT); + (void) ffs_syncvnode(vp, MNT_WAIT|NO_INO_UPDT); else launched_async_io = 1; continue; @@ -2408,7 +2400,7 @@ ffs_copyonwrite(devvp, bp) bawrite(cbp); if ((devvp == bp->b_vp || bp->b_vp->v_type == VDIR || dopersistence) && ip->i_effnlink > 0) - (void) ffs_syncvnode(vp, MNT_WAIT); + (void) ffs_syncvnode(vp, MNT_WAIT|NO_INO_UPDT); else launched_async_io = 1; break; @@ -2428,7 +2420,7 @@ ffs_copyonwrite(devvp, bp) bawrite(savedcbp); if ((devvp == bp->b_vp || bp->b_vp->v_type == VDIR || dopersistence) && VTOI(vp)->i_effnlink > 0) - (void) ffs_syncvnode(vp, MNT_WAIT); + (void) ffs_syncvnode(vp, MNT_WAIT|NO_INO_UPDT); else launched_async_io = 1; } @@ -2478,7 +2470,7 @@ ffs_sync_snap(mp, waitfor) } TAILQ_FOREACH(ip, &sn->sn_head, i_nextsnap) { vp = ITOV(ip); - ffs_syncvnode(vp, waitfor); + ffs_syncvnode(vp, waitfor|NO_INO_UPDT); } lockmgr(&sn->sn_lock, LK_RELEASE, NULL); } -- Test scenario: suj26.sh -U