GENERIC HEAD from 2012-02-13 01:44:12 UTC, r231573M, vmcore.287 KDB: debugger backends: ddb KDB: current backend: ddb 524288K of memory above 4GB ignored Copyright (c) 1992-2012 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #1 r231573M: Tue Feb 14 07:55:38 CET 2012 pho@x4.osted.lan:/usr/src/sys/i386/compile/PHO i386 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. CPU: AMD Phenom(tm) 9150e Quad-Core Processor (1800.02-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x100f23 Family = 10 Model = 2 Stepping = 3 Features=0x178bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x7ff TSC: P-state invariant real memory = 4294967296 (4096 MB) avail memory = 3531640832 (3368 MB) : Trying to mount root from ufs:/dev/ad4s1a [rw]... WARNING: / was not properly dismounted Setting hostuuid: 00000000-0000-0000-0000-00218515337d. Setting hostid: 0x6b64ac17. Starting ddb. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: /dev/ad4s1a: 9165 files, 895861 used, 117154 free (1378 frags, 14472 blocks, 0.1% fragmentation) /dev/ad4s1f: 20448 files, 869730 used, 143285 free (4789 frags, 17312 blocks, 0.5% fragmentation) /dev/label/tmp: 32977 files, 3687316 used, 114626665 free (1473 frags, 14328149 blocks, 0.0% fragmentation) /dev/ad4s1d: 781718 files, 9213318 used, 940881 free (366617 frags, 71783 blocks, 3.6% fragmentation) /dev/ad4s1e: 257522 files, 13599242 used, 11789275 free (53555 frags, 1466965 blocks, 0.2% fragmentation) Mounting local file systems:. Setting hostname: x4.osted.lan. re0: link state changed to DOWN Starting Network: lo0 re0 fwe0 fwip0. lo0: flags=8049 metric 0 mtu 16384 options=3 nd6 options=21 re0: flags=8843 metric 0 mtu 1500 options=209b ether 00:21:85:15:33:7d nd6 options=29 media: Ethernet autoselect (none) status: no carrier fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb nd6 options=29 ch 1 dma -1 fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 nd6 options=29 Starting devd. Starting Network: usbus0. Starting Network: usbus1. Starting Network: usbus2. Starting Network: usbus3. Starting Network: usbus4. Starting Network: usbus5. Starting Network: fwe0. fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb nd6 options=29 ch 1 dma -1 Starting Network: fwip0. fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 nd6 options=29 add net default: gateway 192.168.1.1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 re0: link state changed to UP ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/kde4/lib /usr/local/lib /usr/local/lib/compat /usr/local/lib/compat/pkg /usr/local/lib/nss /usr/local/lib/qt4 /usr/local/lib/virtualbox /usr/local/lib/wine a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout Creating and/or trimming log files. Starting syslogd. No core dumps found. Additional ABI support: linux. Starting rpcbind. NFS access cache time=60 lock order reversal: 1st 0xe191af60 bufwait (bufwait) @ kern/vfs_bio.c:2652 2nd 0xca799400 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:284 KDB: stack backtrace: db_trace_self_wrapper(c0f91ad3,2e687361,38323a63,c0000a34,c12742b0,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a95c8b,c0f95560,c12742a0,11c,f2755878,...) at kdb_backtrace+0x2a _witness_debugger(c0f95560,ca799400,c0fc561a,c7d79678,c0fc52ac,...) at _witness_debugger+0x25 witness_checkorder(ca799400,9,c0fc52a3,11c,0,...) at witness_checkorder+0x86f _sx_xlock(ca799400,0,c0fc52a3,11c,ca790570,...) at _sx_xlock+0xa0 ufsdirhash_acquire(e191af00,ca790570,f27559f4,e23723c0,f2755948,...) at ufsdirhash_acquire+0x48 ufsdirhash_add(ca790570,f27559f4,23c0,f2755934,f2755938,...) at ufsdirhash_add+0x13 ufs_direnter(cab0215c,cad31414,f27559f4,f2755b84,e191b580,...) at ufs_direnter+0x7a9 ufs_mkdir(f2755c14,c0fe6885,0,0,f2755bac,...) at ufs_mkdir+0xa6f VOP_MKDIR_APV(c10dd620,f2755c14,f2755b84,f2755bac,0,...) at VOP_MKDIR_APV+0xc5 kern_mkdirat(ca7a5000,ffffff9c,28404020,0,1c0,...) at kern_mkdirat+0x291 kern_mkdir(ca7a5000,28404020,0,1c0,f2755d1c,...) at kern_mkdir+0x2e sys_mkdir(ca7a5000,f2755cec,c0fdf70a,c0f9623e,202,...) at sys_mkdir+0x29 syscall(f2755d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x2817f743, esp = 0xbfbfe8cc, ebp = 0xbfbfed78 --- Clearing /tmp (X related). Starting mountd. Feb 15 09:24:20 x4 mountd[1836]: can't delete exports for /dev: Invalid argument mount option is unknown NFSv4 is disabled Starting nfsd. Recovering vi editor sessions:lock order reversal: 1st 0xcae3e1b4 ufs (ufs) @ kern/vfs_lookup.c:542 2nd 0xe1922c80 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:262 3rd 0xcae3ddf0 ufs (ufs) @ kern/vfs_subr.c:2167 KDB: stack backtrace: db_trace_self_wrapper(c0f91ad3,3a632e73,a323632,0,0,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a95c8b,c0f95579,c1274070,877,f276132c,...) at kdb_backtrace+0x2a _witness_debugger(c0f95579,cae3ddf0,c0f840cc,c7d79610,c0f9d515,...) at _witness_debugger+0x25 witness_checkorder(cae3ddf0,9,c0f9d50c,877,0,...) at witness_checkorder+0x86f __lockmgr_args(cae3ddf0,80100,cae3de5c,0,0,...) at __lockmgr_args+0x834 ffs_lock(f2761450,c0f9b593,cab068a0,80100,cae3dd98,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c10dd620,f2761450,c7d721e0,c10ee640,cae3dd98,...) at VOP_LOCK1_APV+0xb5 _vn_lock(cae3dd98,80100,c0f9d50c,877,4,...) at _vn_lock+0x78 vget(cae3dd98,80100,cab068a0,50,0,...) at vget+0xbb vfs_hash_get(cab11000,15edde,80000,cab068a0,f276159c,...) at vfs_hash_get+0xed ffs_vgetf(cab11000,15edde,80000,f276159c,1,...) at ffs_vgetf+0x49 softdep_sync_buf(cae3e15c,e1922c20,1,108,0,...) at softdep_sync_buf+0xac9 ffs_syncvnode(cae3e15c,1,f2761644,c0a4b851,ca816bd8,...) at ffs_syncvnode+0x24c ffs_truncate(cae3e15c,200,0,880,c7dc5c80,...) at ffs_truncate+0x827 ufs_direnter(cae3e15c,cae3dd98,f2761918,f2761bc0,0,...) at ufs_direnter+0x9a1 ufs_makeinode(f2761bc0,c10ddba0,f2761b04,f2761a60,c0de4bf5,...) at ufs_makeinode+0x65d ufs_create(f2761b04,c0fe6a4b,0,0,f2761b80,...) at ufs_create+0x30 VOP_CREATE_APV(c10dd620,f2761b04,f2761bc0,f2761a9c,0,...) at VOP_CREATE_APV+0xc5 vn_open_cred(f2761b80,f2761c48,1b0,0,c7dc5c80,...) at vn_open_cred+0x205 vn_open(f2761b80,f2761c48,1b0,ca7b4818,4f3b69d8,...) at vn_open+0x3b kern_openat(cab068a0,ffffff9c,28839f60,0,a03,...) at kern_openat+0x1ec kern_open(cab068a0,28839f60,0,a02,1b0,...) at kern_open+0x35 sys_open(cab068a0,f2761cec,c0fdf70a,c0f9623e,206,...) at sys_open+0x30 syscall(f2761d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (5, FreeBSD ELF32, sys_open), eip = 0x283eada3, esp = 0xbfbfc50c, ebp = 0xbfbfc598 --- . Updating motd:. Starting ntpd. Starting default moused. Configuring syscons: keymap blanktime. Starting sshd. Starting cron. Local package initialization: backuplock order reversal: 1st 0xca825b38 ufs (ufs) @ kern/vfs_mount.c:1249 2nd 0xcab02058 syncer (syncer) @ kern/vfs_subr.c:2279 KDB: stack backtrace: db_trace_self_wrapper(c0f91ad3,a3937,cccccccc,ccccccc,7fff,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a95c8b,c0f95560,c1273c50,8e7,f27d79dc,...) at kdb_backtrace+0x2a _witness_debugger(c0f95560,cab02058,c0f9e0c4,c7d796e0,c0f9d515,...) at _witness_debugger+0x25 witness_checkorder(cab02058,9,c0f9d50c,8e7,0,...) at witness_checkorder+0x86f __lockmgr_args(cab02058,80100,cab020c4,0,0,...) at __lockmgr_args+0x834 vop_stdlock(f27d7afc,c0aa5e4a,c1273c54,80100,cab02000,...) at vop_stdlock+0x65 VOP_LOCK1_APV(c10cbfe0,f27d7afc,c0a4bcac,c10ee640,cab02000,...) at VOP_LOCK1_APV+0xb5 _vn_lock(cab02000,80100,c0f9d50c,8e7,c12a15a0,...) at _vn_lock+0x78 vputx(f27d7b80,c0afe3fd,cab02000,0,c0f9d50c,...) at vputx+0x266 vrele(cab02000,0,c0f9d50c,db7,ca825ae0,...) at vrele+0x10 vfs_deallocate_syncvnode(cab115a8,0,c0f9cb6f,51a,c7c30000,...) at vfs_deallocate_syncvnode+0x6d dounmount(cab115a8,8080000,caae65c0,49c,0,...) at dounmount+0x376 sys_unmount(caae65c0,f27d7cec,c,c,206,...) at sys_unmount+0x421 syscall(f27d7d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280e7e0b, esp = 0xbfbfe64c, ebp = 0xbfbfe718 --- lock order reversal: 1st 0xca825b38 ufs (ufs) @ kern/vfs_mount.c:1249 2nd 0xca816df0 devfs (devfs) @ ufs/ffs/ffs_softdep.c:1835 KDB: stack backtrace: db_trace_self_wrapper(c0f91ad3,38313a63,a3533,c0aa827c,8,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a95c8b,c0f95560,c1273c40,72b,f27d79a0,...) at kdb_backtrace+0x2a _witness_debugger(c0f95560,ca816df0,c0f7c007,c7d79470,c0fc1724,...) at _witness_debugger+0x25 witness_checkorder(ca816df0,9,c0fc171b,72b,ca816e5c,...) at witness_checkorder+0x86f __lockmgr_args(ca816df0,80400,ca816e5c,0,0,...) at __lockmgr_args+0x834 vop_stdlock(f27d7ac0,4,c0f8beab,80400,ca816d98,...) at vop_stdlock+0x65 VOP_LOCK1_APV(c10b32a0,f27d7ac0,c12a7860,c10ee640,ca816d98,...) at VOP_LOCK1_APV+0xb5 _vn_lock(ca816d98,80400,c0fc171b,72b,cab115a8,...) at _vn_lock+0x78 softdep_flushworklist(cab115a8,f27d7b6c,caae65c0,5e9,0,...) at softdep_flushworklist+0x47 ffs_sync(cab115a8,1,f27d7bc0,51a,c7c30000,...) at ffs_sync+0x32d dounmount(cab115a8,8080000,caae65c0,49c,0,...) at dounmount+0x3f8 sys_unmount(caae65c0,f27d7cec,c,c,206,...) at sys_unmount+0x421 syscall(f27d7d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280e7e0b, esp = 0xbfbfe64c, ebp = 0xbfbfe718 --- ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 32977 files, 3687316 used, 114626665 free (1473 frags, 14328149 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 32977 files, 3687316 used, 114626665 free (1473 frags, 14328149 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** fsck -y /tmpFeb 15 09:27:28 x4 mountd[1837]: can't delete exports for /dev: Invalid argument mount option is unknown watchdogd. Starting inetd. Wed Feb 15 09:27:29 CET 2012 FreeBSD/i386 (x4.osted.lan) (console) login: Feb 15 09:29:22 x4 su: pho to root on /dev/pts/0 Feb 15 09:29:39 x4 mountd[1837]: can't delete exports for /dev: Invalid argument mount option is unknown lock order reversal: 1st 0xe1962420 bufwait (bufwait) @ ufs/ffs/ffs_softdep.c:12329 2nd 0xca785b1c snaplk (snaplk) @ ufs/ffs/ffs_snapshot.c:2299 KDB: stack backtrace: db_trace_self_wrapper(c0f91ad3,632e746f,3932323a,a000a39,f45ad500,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a95c8b,c0f95560,c1273538,8fb,f28873e4,...) at kdb_backtrace+0x2a _witness_debugger(c0f95560,ca785b1c,c0fc11cd,c7d79e30,c0fc1157,...) at _witness_debugger+0x25 witness_checkorder(ca785b1c,9,c0fc114e,8fb,0,...) at witness_checkorder+0x86f __lockmgr_args(ca785b1c,80900,cb240e5c,0,0,...) at __lockmgr_args+0x834 ffs_copyonwrite(cb240d98,e19623c0,0,e19623c0,cae1f000,...) at ffs_copyonwrite+0x2c2 ffs_geom_strategy(cb240e90,e19623c0,f2887568,f42,f2887570,...) at ffs_geom_strategy+0xeb ufs_strategy(f28875a4,c0fe6746,c0f8beab,e19623c0,0,...) at ufs_strategy+0xd3 VOP_STRATEGY_APV(c10dd620,f28875a4,cae1f0f8,c10ee6a0,cae1f000,...) at VOP_STRATEGY_APV+0xe5 bufstrategy(cae1f0f8,e19623c0,645,e19623c0,21068,...) at bufstrategy+0x61 bufwrite(e19623c0,80900,cae1f0f8,0,0,...) at bufwrite+0x159 softdep_sync_buf(cb25415c,e195efc0,1,108,0,...) at softdep_sync_buf+0x7cc ffs_syncvnode(cb25415c,1,c1273508,ca6ccb80,ca6ccb80,...) at ffs_syncvnode+0x24c ffs_truncate(cb25415c,400,0,880,c7dc5700,...) at ffs_truncate+0x827 ufs_direnter(cb25415c,cae1f000,f28879f4,f2887b84,e19623c0,...) at ufs_direnter+0x9a1 ufs_mkdir(f2887c14,c0fe6885,0,0,f2887bac,...) at ufs_mkdir+0xa6f VOP_MKDIR_APV(c10dd620,f2887c14,f2887b84,f2887bac,0,...) at VOP_MKDIR_APV+0xc5 kern_mkdirat(ca6ccb80,ffffff9c,bfbfef4a,0,1f8,...) at kern_mkdirat+0x291 kern_mkdir(ca6ccb80,bfbfef4a,0,1f8,f2887d1c,...) at kern_mkdir+0x2e sys_mkdir(ca6ccb80,f2887cec,c0fdf70a,c0f9623e,257,...) at sys_mkdir+0x29 syscall(f2887d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28194743, esp = 0xbfbfe7dc, ebp = 0xbfbfe878 --- fsync: giving up on dirty 0xcb240d98: tag devfs, type VCHR usecount 1, writecount 0, refcount 10 mountedhere 0xcafca100 flags (VV_COPYONWRITE) v_object 0xcb2c0120 ref 0 pages 46 lock type devfs: EXCL by thread 0xcb2492e0 (pid 2342) #0 0xc0a4393e at __lockmgr_args+0xc1e #1 0xc0ae76d5 at vop_stdlock+0x65 #2 0xc0de3c25 at VOP_LOCK1_APV+0xb5 #3 0xc0b09798 at _vn_lock+0x78 #4 0xc0c9adc7 at softdep_flushworklist+0x47 #5 0xc0c9fe2d at ffs_sync+0x32d #6 0xc0b09392 at vfs_write_suspend+0x122 #7 0xc0c83e0f at ffs_snapshot+0xcef #8 0xc0ca23d7 at ffs_mount+0x1d17 #9 0xc0af188b at vfs_donmount+0x12eb #10 0xc0af1f2f at sys_nmount+0x7f #11 0xc0dc08c7 at syscall+0x2c7 #12 0xc0da9bf1 at Xint0x80_syscall+0x21 dev md5a fsync: giving up on dirty 0xcb240d98: tag devfs, type VCHR usecount 1, writecount 0, refcount 10 mountedhere 0xcafca100 flags (VV_COPYONWRITE) v_object 0xcb2c0120 ref 0 pages 46 lock type devfs: EXCL by thread 0xcb2492e0 (pid 2342) #0 0xc0a4393e at __lockmgr_args+0xc1e #1 0xc0ae76d5 at vop_stdlock+0x65 #2 0xc0de3c25 at VOP_LOCK1_APV+0xb5 #3 0xc0b09798 at _vn_lock+0x78 #4 0xc0c9fef7 at ffs_sync+0x3f7 #5 0xc0b09392 at vfs_write_suspend+0x122 #6 0xc0c83e0f at ffs_snapshot+0xcef #7 0xc0ca23d7 at ffs_mount+0x1d17 #8 0xc0af188b at vfs_donmount+0x12eb #9 0xc0af1f2f at sys_nmount+0x7f #10 0xc0dc08c7 at syscall+0x2c7 #11 0xc0da9bf1 at Xint0x80_syscall+0x21 dev md5a Expensive timeout(9) function: 0xc08e4e40(0xc8052aa0) 0.199875363 s Feb 15 09:29:39 x4 mountd[1837]: can't delete exports for /dev: Invalid argument mount option is unknown Expensive timeout(9) function: 0xc0a9c4c0(0xc805e8a0) 0.199987943 s Expensive timeout(9) function: 0xc08e4e40(0xc8063c78) 0.200876293 s Expensive timeout(9) function: 0xc08e4e40(0xc804aaa0) 0.999938322 s interrupt total irq4: uart0 579 irq6: fdc0 1 irq9: acpi0 1 irq14: ata0 18 irq16: hdac1 ohci0 86 irq17: ohci1 ohci3 3 irq18: ohci2 ohci4 3 irq20: fwohci0 1 irq22: ahci0 421439 cpu0:timer 2727316 irq256: hdac0 4 irq257: re0 7555 cpu2:timer 3500070 cpu1:timer 3283585 cpu3:timer 3491820 Total 13432514 KDB: stack backtrace: db_trace_self_wrapper(c0f91ad3,414261,f29ca984,c0dc980e,62fa5200,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0f85ebe,ccf6c2,0,ccf6c2,0,...) at kdb_backtrace+0x2a watchdog_fire(1,0,c0f85d8b,217,cb52a5c0,...) at watchdog_fire+0xa9 hardclock_anycpu(1,0,f29caa54,c0a68c5d,0,...) at hardclock_anycpu+0x26f handleevents(f29caa74,1,c1273140,0,0,f29caa80,1240,87f30889,c169c569,6a2a180) at handleevents+0x108 timercb(c13087c0,0,2,c1129a90,0,...) at timercb+0x2ad lapic_handle_timer(f29caab8) at lapic_handle_timer+0x92 Xtimerint() at Xtimerint+0x20 --- interrupt, eip = 0xc0a4bac0, esp = 0xf29caaf8, ebp = 0xf29cab14 --- _mtx_lock_sleep(c1129a90,cb52a5c0,0,c0f8e8e8,5e4,...) at _mtx_lock_sleep+0xb0 _mtx_lock_flags(c1129a90,0,c0f8e8e8,5e4,f29cab90,...) at _mtx_lock_flags+0xff sysctl_root(f29cab90,0,c0f8e8e8,652,cb52a5c0,...) at sysctl_root+0x1f6 userland_sysctl(cb52a5c0,f29cabfc,4,bfbfdb94,bfbfdb90,...) at userland_sysctl+0x17a sys___sysctl(cb52a5c0,f29cacec,c0fdf70a,c0f967a3,293,...) at sys___sysctl+0x94 syscall(f29cad28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (202, FreeBSD ELF32, sys___sysctl), eip = 0x28192987, esp = 0xbfbfda8c, ebp = 0xbfbfdab8 --- KDB: enter: watchdog timeout [ thread pid 6973 tid 100243 ] Stopped at kdb_enter+0x3a: movl $0,kdb_why db> run pho db:0:pho> bt Tracing pid 6973 tid 100243 td 0xcb52a5c0 kdb_enter(c0e1eda7,c0f10c65,0,ccf6c2,0) at kdb_enter+0x3a watchdog_fire(1,0,c0f85d8b,217,cb52a5c0,...) at watchdog_fire+0xbd hardclock_anycpu(1,0,f29caa54,c0a68c5d,0,...) at hardclock_anycpu+0x26f handleevents(f29caa74,1,c1273140,0,0,f29caa80,1240,87f30889,c169c569,6a2a180) at handleevents+0x108 timercb(c13087c0,0,2,c1129a90,0,...) at timercb+0x2ad lapic_handle_timer(f29caab8) at lapic_handle_timer+0x92 Xtimerint() at Xtimerint+0x20 --- interrupt, eip = 0xc0a4bac0, esp = 0xf29caaf8, ebp = 0xf29cab14 --- _mtx_lock_sleep(c1129a90,cb52a5c0,0,c0f8e8e8,5e4,...) at _mtx_lock_sleep+0xb0 _mtx_lock_flags(c1129a90,0,c0f8e8e8,5e4,f29cab90,...) at _mtx_lock_flags+0xff sysctl_root(f29cab90,0,c0f8e8e8,652,cb52a5c0,...) at sysctl_root+0x1f6 userland_sysctl(cb52a5c0,f29cabfc,4,bfbfdb94,bfbfdb90,...) at userland_sysctl+0x17a sys___sysctl(cb52a5c0,f29cacec,c0fdf70a,c0f967a3,293,...) at sys___sysctl+0x94 syscall(f29cad28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (202, FreeBSD ELF32, sys___sysctl), eip = 0x28192987, esp = 0xbfbfda8c, ebp = 0xbfbfdab8 --- db:0:bt> show allpcpu Current CPU: 2 cpuid = 0 dynamic pcpu = 0x736180 curthread = 0xc7edb000: pid 13 "g_up" curpcb = 0xc7b48d80 fpcurthread = none idlethread = 0xc7dc98a0: tid 100003 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: cpuid = 1 dynamic pcpu = 0x6a27180 curthread = 0xc7e132e0: pid 12 "swi4: clock" curpcb = 0xc7b36d80 fpcurthread = none idlethread = 0xc7dc95c0: tid 100004 "idle: cpu1" APIC ID = 1 currentldt = 0x50 spin locks held: cpuid = 2 dynamic pcpu = 0x6a2a180 curthread = 0xcb52a5c0: pid 6973 "sysctl" curpcb = 0xf29cad80 fpcurthread = 0xcb52a5c0: pid 6973 "sysctl" idlethread = 0xc7dc92e0: tid 100005 "idle: cpu2" APIC ID = 2 currentldt = 0x50 spin locks held: cpuid = 3 dynamic pcpu = 0x6a2d180 curthread = 0xcb2492e0: pid 2371 "mksnap_ffs" curpcb = 0xf28d5d80 fpcurthread = none idlethread = 0xc7dc9000: tid 100006 "idle: cpu3" APIC ID = 3 currentldt = 0x50 spin locks held: db:0:allpcpu> show alllocks Process 2371 (mksnap_ffs) thread 0xcb2492e0 (100179) exclusive sleep mutex struct mount mtx (struct mount mtx) r = 0 (0xca76b5a8) locked @ kern/vfs_mount.c:1730 exclusive sleep mutex Giant (Giant) r = 0 (0xc1129a90) locked @ kern/vfs_mount.c:1102 Process 1710 (syslogd) thread 0xcaae55c0 (100103) exclusive lockmgr ufs (ufs) r = 0 (0xcad391b4) locked @ kern/vfs_syscalls.c:3583 db:0:alllocks> show lockedvnods Locked vnodes 0xcad3915c: tag ufs, type VREG usecount 2, writecount 1, refcount 6 mountedhere 0 flags () v_object 0xca80d630 ref 0 pages 11 lock type ufs: EXCL by thread 0xcaae55c0 (pid 1710) with exclusive waiters pending #0 0xc0a4393e at __lockmgr_args+0xc1e #1 0xc0ca5bf1 at ffs_lock+0xa1 #2 0xc0de3c25 at VOP_LOCK1_APV+0xb5 #3 0xc0b09798 at _vn_lock+0x78 #4 0xc0b07936 at sys_fsync+0x116 #5 0xc0dc08c7 at syscall+0x2c7 #6 0xc0da9bf1 at Xint0x80_syscall+0x21 ino 6052906, on dev ada0s1e db:0:lockedvnods> show mount 0xca76ab50 /dev/ad4s1a on / (ufs) 0xca76b000 devfs on /dev (devfs) 0xca76a87c /dev/ad4s1f on /home (ufs) 0xcab112d4 /dev/ad4s1d on /usr (ufs) 0xcab11000 /dev/ad4s1e on /var (ufs) 0xca76bb50 procfs on /proc (procfs) 0xca76b87c /dev/label/tmp on /tmp (ufs) 0xca76b5a8 /dev/md5a on /mnt (ufs) 0xca76b2d4 /dev/md6 on /mnt2 (ufs) More info: show mount db:0:mount> ps pid ppid pgrp uid state wmesg wchan cmd 6975 2273 2272 1001 S piperd 0xcad3c4b0 sed 6974 2273 2272 1001 S piperd 0xcb4ee4b0 tail 6973 2273 2272 1001 R CPU 2 sysctl 2420 2419 2157 0 D suspfs 0xca76b5f0 ls 2419 2157 2157 0 S wait 0xcb49a2c4 sh 2371 2369 2290 0 R+ CPU 3 mksnap_ffs 2370 2290 2290 0 S+ piperd 0xcab49320 grep 2369 2290 2290 0 S+ wait 0xcb49a84c sh 2367 2348 2314 1004 S+ suspfs 0xca76b5f0 mkdir 2366 2348 2314 1004 S+ suspfs 0xca76b5f0 mkdir 2365 2348 2314 1004 S+ suspfs 0xca76b5f0 mkdir 2364 2348 2314 1004 S+ suspfs 0xca76b5f0 mkdir 2363 2348 2314 1004 S+ suspfs 0xca76b5f0 mkdir 2362 2348 2314 1004 S+ suspfs 0xca76b5f0 mkdir 2361 2348 2314 1004 S+ suspfs 0xca76b5f0 mkdir 2360 2348 2314 1004 S+ suspfs 0xca76b5f0 mkdir 2359 2348 2314 1004 S+ suspfs 0xca76b5f0 mkdir 2358 2348 2314 1004 S+ suspfs 0xca76b5f0 mkdir 2357 2348 2314 1004 S+ suspfs 0xca76b5f0 mkdir 2356 2348 2314 1004 S+ suspfs 0xca76b5f0 mkdir 2355 2348 2314 1004 S+ suspfs 0xca76b5f0 mkdir 2354 2348 2314 1004 S+ suspfs 0xca76b5f0 mkdir 2353 2348 2314 1004 S+ suspfs 0xca76b5f0 mkdir 2352 2348 2314 1004 S+ suspfs 0xca76b5f0 mkdir 2351 2348 2314 1004 S+ suspfs 0xca76b5f0 mkdir 2350 2348 2314 1004 S+ suspfs 0xca76b5f0 mkdir 2349 2348 2314 1004 S+ suspfs 0xca76b5f0 mkdir 2348 2346 2314 1004 S+ wait 0xcb158000 mkdir 2346 2344 2314 1004 S+ wait 0xcb158588 mkdir 2344 2343 2314 1004 S+ wait 0xcaadd2c4 run 2343 2314 2314 1004 S+ wait 0xca6c1588 run 2320 0 0 0 DL mdwait 0xca6bd800 [md6] 2314 2308 2314 1004 S+ wait 0xcb15a84c run 2308 2290 2290 0 S+ wait 0xcb15ab10 su 2303 0 0 0 DL mdwait 0xca6be000 [md5] 2290 2255 2290 0 S+ wait 0xcb159b10 sh 2274 2272 2272 1001 S piperd 0xcab427d0 awk 2273 2272 2272 1001 S wait 0xcb1fd2c4 sh 2272 2271 2272 1001 Ss wait 0xcac5784c sh 2271 2261 2261 1001 S select 0xcb1018a4 sshd 2270 2268 2270 1001 Ds ufs 0xcad391b4 tail 2269 2267 2269 1001 Ss+ select 0xcb101364 top 2268 2259 2259 1001 S select 0xca6b46e4 sshd 2267 2260 2260 1001 S select 0xcab18224 sshd 2261 2090 2261 0 Ss select 0xcab184e4 sshd 2260 2090 2260 0 Ss select 0xca6b4de4 sshd 2259 2090 2259 0 Ss select 0xcb1028e4 sshd 2255 2254 2255 0 S+ pause 0xcaad38a4 csh 2254 2249 2254 1001 S+ wait 0xcac59588 su 2249 2248 2249 1001 Ss+ wait 0xcb1592c4 bash 2248 2246 2246 1001 S select 0xcb1017e4 sshd 2246 2090 2246 0 Ss select 0xcb102d24 sshd 2236 1 2236 0 Ss+ ttyin 0xc7d97470 getty 2235 1 2235 0 Ss+ ttyin 0xc7d97670 getty 2234 1 2234 0 Ss+ ttyin 0xc7d97870 getty 2233 1 2233 0 Ss+ ttyin 0xc7d97a70 getty 2232 1 2232 0 Ss+ ttyin 0xc7d97c70 getty 2231 1 2231 0 Ss+ ttyin 0xc7d97e70 getty 2230 1 2230 0 Ss+ ttyin 0xc8181070 getty 2229 1 2229 0 Ss+ ttyin 0xc8181270 getty 2228 1 2228 0 Ss+ ttyin 0xc8181470 getty 2189 1 2189 0 Ss select 0xca7692a4 inetd 2157 1 2157 0 Ss wait 0xcaeb0588 watchdogd 2110 1 2110 0 Ss nanslp 0xc112b3e4 cron 2102 1 2102 25 Ss pause 0xcac5a5e0 sendmail 2098 1 2098 0 Ss select 0xca6b42e4 sendmail 2090 1 2090 0 Ss select 0xca769124 sshd 2030 1 2030 0 Ss select 0xca6b44e4 moused 1952 1 1952 0 Ss select 0xca8216e4 ntpd 1853 1847 1847 0 S (threaded) nfsd 100135 S rpcsvc 0xcab64b50 nfsd: service 100134 S rpcsvc 0xcab64b90 nfsd: service 100133 S rpcsvc 0xcab64bd0 nfsd: service 100088 S rpcsvc 0xcae33210 nfsd: master 1847 1 1847 0 Ss select 0xca6b4524 nfsd 1837 1 1837 0 Ss select 0xca690e64 mountd 1738 1 1738 0 Ss select 0xca768564 rpcbind 1710 1 1710 0 Ds getbuf 0xe1915fbc syslogd 1503 1 1503 0 Ss select 0xca6b45e4 devd 19 0 0 0 DL sdflush 0xc12a78a0 [softdepflush] 18 0 0 0 DL vlruwt 0xca6c02c4 [vnlru] 17 0 0 0 DL syncer 0xc12a15b4 [syncer] 16 0 0 0 DL psleep 0xc12a1248 [bufdaemon] 9 0 0 0 DL pgzero 0xc12a975c [pagezero] 8 0 0 0 DL psleep 0xc12a9348 [vmdaemon] 7 0 0 0 DL psleep 0xc12a9310 [pagedaemon] 6 0 0 0 DL ccb_scan 0xc10efbd4 [xpt_thrd] 5 0 0 0 DL waiting_ 0xc12a30d8 [sctp_iterator] 4 0 0 0 DL ctl_work 0xc819c000 [ctl_thrd] 3 0 0 0 DL - 0xc7ffa03c [fdc0] 2 0 0 0 SL - 0xc808f000 [fw0_probe] 15 0 0 0 DL (threaded) [usb] 100058 D - 0xc8063d34 [usbus5] 100057 D - 0xc8063d04 [usbus5] 100056 D - 0xc8063cd4 [usbus5] 100055 D - 0xc8063ca4 [usbus5] 100053 D - 0xc8057b5c [usbus4] 100052 D - 0xc8057b2c [usbus4] 100051 D - 0xc8057afc [usbus4] 100050 D - 0xc8057acc [usbus4] 100049 D - 0xc8052b5c [usbus3] 100048 D - 0xc8052b2c [usbus3] 100047 D - 0xc8052afc [usbus3] 100046 D - 0xc8052acc [usbus3] 100045 D - 0xc804ab5c [usbus2] 100044 D - 0xc804ab2c [usbus2] 100043 D - 0xc804aafc [usbus2] 100042 D - 0xc804aacc [usbus2] 100040 D - 0xc8044b5c [usbus1] 100039 D - 0xc8044b2c [usbus1] 100038 D - 0xc8044afc [usbus1] 100037 D - 0xc8044acc [usbus1] 100035 D - 0xc8039b5c [usbus0] 100034 D - 0xc8039b2c [usbus0] 100033 D - 0xc8039afc [usbus0] 100032 D - 0xc8039acc [usbus0] 14 0 0 0 DL - 0xc112a1e4 [yarrow] 13 0 0 0 RL (threaded) [geom] 100015 D - 0xc1127de8 [g_down] 100014 Run CPU 0 [g_up] 100013 D - 0xc1127ddc [g_event] 12 0 0 0 RL (threaded) [intr] 100066 I [irq12: psm0] 100065 I [irq1: atkbd0] 100063 I [swi0: uart] 100060 I [irq20: fwohci0] 100059 I [irq14: ata0] 100054 I [irq19: ehci0] 100041 I [irq18: ohci2 ohci4] 100036 I [irq17: ohci1 ohci3] 100031 I [irq16: hdac1 ohci0] 100030 I [irq22: ahci0] 100029 I [irq257: re0] 100028 I [irq256: hdac0] 100027 I [swi6: task queue] 100026 I [swi6: Giant taskq] 100024 I [swi5: +] 100018 I [swi2: cambio] 100012 I [swi3: vm] 100011 I [swi4: clock] 100010 I [swi4: clock] 100009 RunQ [swi4: clock] 100008 Run CPU 1 [swi4: clock] 100007 I [swi1: netisr 0] 11 0 0 0 RL (threaded) [idle] 100006 CanRun [idle: cpu3] 100005 CanRun [idle: cpu2] 100004 CanRun [idle: cpu1] 100003 CanRun [idle: cpu0] 1 0 1 0 SLs wait 0xc7dc7b10 [init] 10 0 0 0 DL audit_wo 0xc12a6fe0 [audit] 0 0 0 0 DLs (threaded) [kernel] 100078 D - 0xc8036380 [mca taskq] 100068 D - 0xc112a1e4 [deadlkres] 100061 D - 0xc80366c0 [fw0_taskq] 100025 D - 0xc7f9ee80 [thread taskq] 100023 D - 0xc7f9f040 [acpi_task_2] 100022 D - 0xc7f9f040 [acpi_task_1] 100021 D - 0xc7f9f040 [acpi_task_0] 100020 D - 0xc7f9f080 [kqueue taskq] 100019 D - 0xc7f9f0c0 [ffs_trim taskq] 100016 D - 0xc7dade80 [firmware taskq] 100000 D sched 0xc1127ec0 [swapper] 2347 2344 2314 1004 Z+ swap db:0:ps> allt Tracing command sed pid 6975 tid 100219 td 0xcb4fab80 sched_switch(cb4fab80,0,104,1a3,77cfcbf5,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(cb4fab80,0,c0f93480,1a7,5c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cad3c4b0,5c,c0f95dcf,100,0,...) at sleepq_wait_sig+0x17 _sleep(cad3c4b0,cad3c628,15c,c0f95dcf,0,...) at _sleep+0x385 pipe_read(cac528f8,f2967c44,caeaee80,0,cb4fab80,...) at pipe_read+0x417 dofileread(f2967c44,ffffffff,ffffffff,0,f2967c20,...) at dofileread+0x9e kern_readv(cb4fab80,0,f2967c44,f2967c64,1,...) at kern_readv+0x68 sys_read(cb4fab80,f2967cec,c0fdf70a,c0f962cf,216,...) at sys_read+0x4f syscall(f2967d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x28195333, esp = 0xbfbfea1c, ebp = 0xbfbfea38 --- Tracing command tail pid 6974 tid 100239 td 0xcb52b2e0 sched_switch(cb52b2e0,0,104,1a3,77c713e5,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(cb52b2e0,0,c0f93480,1a7,5c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cb4ee4b0,5c,c0f95dcf,100,0,...) at sleepq_wait_sig+0x17 _sleep(cb4ee4b0,cb4ee628,15c,c0f95dcf,0,...) at _sleep+0x385 pipe_read(ca823d90,f29bec44,caeaee80,0,cb52b2e0,...) at pipe_read+0x417 dofileread(f29bec44,ffffffff,ffffffff,0,f29bec20,...) at dofileread+0x9e kern_readv(cb52b2e0,0,f29bec44,f29bec64,1,...) at kern_readv+0x68 sys_read(cb52b2e0,f29becec,c,c,216,...) at sys_read+0x4f syscall(f29bed28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x28192333, esp = 0xbfbfea8c, ebp = 0xbfbfeaa8 --- Tracing command sysctl pid 6973 tid 100243 td 0xcb52a5c0 kdb_enter(c0e1eda7,c0f10c65,0,ccf6c2,0) at kdb_enter+0x3a watchdog_fire(1,0,c0f85d8b,217,cb52a5c0,...) at watchdog_fire+0xbd hardclock_anycpu(1,0,f29caa54,c0a68c5d,0,...) at hardclock_anycpu+0x26f handleevents(f29caa74,1,c1273140,0,0,f29caa80,1240,87f30889,c169c569,6a2a180) at handleevents+0x108 timercb(c13087c0,0,2,c1129a90,0,...) at timercb+0x2ad lapic_handle_timer(f29caab8) at lapic_handle_timer+0x92 Xtimerint() at Xtimerint+0x20 --- interrupt, eip = 0xc0a4bac0, esp = 0xf29caaf8, ebp = 0xf29cab14 --- _mtx_lock_sleep(c1129a90,cb52a5c0,0,c0f8e8e8,5e4,...) at _mtx_lock_sleep+0xb0 _mtx_lock_flags(c1129a90,0,c0f8e8e8,5e4,f29cab90,...) at _mtx_lock_flags+0xff sysctl_root(f29cab90,0,c0f8e8e8,652,cb52a5c0,...) at sysctl_root+0x1f6 userland_sysctl(cb52a5c0,f29cabfc,4,bfbfdb94,bfbfdb90,...) at userland_sysctl+0x17a sys___sysctl(cb52a5c0,f29cacec,c0fdf70a,c0f967a3,293,...) at sys___sysctl+0x94 syscall(f29cad28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (202, FreeBSD ELF32, sys___sysctl), eip = 0x28192987, esp = 0xbfbfda8c, ebp = 0xbfbfdab8 --- Tracing command ls pid 2420 tid 100151 td 0xcaeb25c0 sched_switch(caeb25c0,0,104,1a3,483b98a,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,77,...) at mi_switch+0x1ff sleepq_switch(caeb25c0,0,c0f93480,268,0,...) at sleepq_switch+0x162 sleepq_wait(ca76b5f0,77,c0f9edb3,0,0,...) at sleepq_wait+0x63 _sleep(ca76b5f0,ca76b5a8,77,c0f9edb3,0,...) at _sleep+0x39c vn_start_write(cb25415c,f2881b44,1,c0de01b5,0,...) at vn_start_write+0xdd vn_close(cb25415c,5,c7dc4e80,caeb25c0,cac520a8,...) at vn_close+0x87 vn_closefile(cac520a8,caeb25c0,1,0,cac520a8,...) at vn_closefile+0xe4 _fdrop(cac520a8,caeb25c0,0,f2881c18,0,c0aa5e4a,c12731f0,caeb2670,c12731f0,c10c3d80,caeb25c0,ca77da2c,4ce,f2881c28,c0a64895,ca77da2c,8,c0f8726c,cac520a8) at _fdrop+0x43 closef(cac520a8,caeb25c0,4ce,f2881c4c,ca77da2c,...) at closef+0x2b0 kern_close(caeb25c0,7,f2881d1c,c0dc08c7,caeb25c0,...) at kern_close+0x139 sys_close(caeb25c0,f2881cec,c0fdf70a,c0f9620b,246,...) at sys_close+0x1a syscall(f2881d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (6, FreeBSD ELF32, sys_close), eip = 0x281e72f3, esp = 0xbfbfe5dc, ebp = 0xbfbfe5f8 --- Tracing command sh pid 2419 tid 100194 td 0xcb4a4000 sched_switch(cb4a4000,0,104,1a3,3b30d9d,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(cb4a4000,0,c0f93480,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cb49a2c4,6c,c0f96397,100,0,...) at sleepq_wait_sig+0x17 _sleep(cb49a2c4,cb49a34c,16c,c0f96397,0,...) at _sleep+0x385 kern_wait(cb4a4000,ffffffff,f2902c60,2,0,...) at kern_wait+0x626 sys_wait4(cb4a4000,f2902cec,c,c,216,...) at sys_wait4+0x3b syscall(f2902d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2817645b, esp = 0xbfbfeaec, ebp = 0xbfbfeb08 --- Tracing command mksnap_ffs pid 2371 tid 100179 td 0xcb2492e0 cpustop_handler(3,13,f28d5428,c0dc0cc6,30c47a5d,...) at cpustop_handler+0x28 ipi_nmi_handler(30c47a5d,1,c7d727f8,1,cb1fd588,...) at ipi_nmi_handler+0x47 trap(f28d5434) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0aa6146, esp = 0xf28d5474, ebp = 0xf28d5478 --- isitmychild(c7d72450,cb249390,6c2,f28d54ac,c0aa5e4a,...) at isitmychild+0x6 witness_checkorder(cb2540c4,9,c0fc45f0,5cf,0,...) at witness_checkorder+0x26c _mtx_lock_flags(cb2540c4,0,c0fc45f0,5cf,0,...) at _mtx_lock_flags+0xcc ffs_sync(ca76b5a8,1,c0f9ed30,4b2,c0f9ed30,...) at ffs_sync+0x1a6 vfs_write_suspend(ca76b5a8,f28d5874,0,8000,c7d6fe80,...) at vfs_write_suspend+0x122 ffs_snapshot(ca76b5a8,c809a8a0,c0fc45f0,1a7,0,...) at ffs_snapshot+0xcef ffs_mount(ca76b5a8,ca6cb800,ff,3b4,c0ae77ba,...) at ffs_mount+0x1d17 vfs_donmount(cb2492e0,11100,0,ca88d300,ca88d300,...) at vfs_donmount+0x12eb sys_nmount(cb2492e0,f28d5cec,c0fdf70a,c0f9633b,213,...) at sys_nmount+0x7f syscall(f28d5d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (378, FreeBSD ELF32, sys_nmount), eip = 0x280e5a2b, esp = 0xbfbfe14c, ebp = 0xbfbfe8c8 --- Tracing command grep pid 2370 tid 100193 td 0xcb4a42e0 sched_switch(cb4a42e0,0,104,1a3,a4fbfc09,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(cb4a42e0,0,c0f93480,1a7,5c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cab49320,5c,c0f95dcf,100,0,...) at sleepq_wait_sig+0x17 _sleep(cab49320,cab49498,15c,c0f95dcf,0,...) at _sleep+0x385 pipe_read(cac52818,f28ffc44,c7dc5d00,0,cb4a42e0,...) at pipe_read+0x417 dofileread(f28ffc44,ffffffff,ffffffff,0,f28ffc20,...) at dofileread+0x9e kern_readv(cb4a42e0,0,f28ffc44,f28ffc64,1,...) at kern_readv+0x68 sys_read(cb4a42e0,f28ffcec,c0fdf70a,c0f5c187,246,...) at sys_read+0x4f syscall(f28ffd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281d6333, esp = 0xbfbfe6ac, ebp = 0xbfbfe738 --- Tracing command sh pid 2369 tid 100192 td 0xcb4a45c0 sched_switch(cb4a45c0,0,104,1a3,a49b7028,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(cb4a45c0,0,c0f93480,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cb49a84c,6c,c0f96397,100,0,...) at sleepq_wait_sig+0x17 _sleep(cb49a84c,cb49a8d4,16c,c0f96397,0,...) at _sleep+0x385 kern_wait(cb4a45c0,ffffffff,f28fcc60,2,0,...) at kern_wait+0x626 sys_wait4(cb4a45c0,f28fccec,c0fdf70a,c0f96376,216,...) at sys_wait4+0x3b syscall(f28fcd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2817645b, esp = 0xbfbfe5ac, ebp = 0xbfbfe5c8 --- Tracing command mkdir pid 2367 tid 100079 td 0xca7aab80 sched_switch(ca7aab80,0,104,1a3,9cdfc0ba,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,77,...) at mi_switch+0x1ff sleepq_switch(ca7aab80,0,c0f93480,1a7,77,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca76b5f0,77,c0f9edb3,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca76b5f0,ca76b5a8,177,c0f9edb3,0,...) at _sleep+0x385 vn_start_write(0,f26ffc28,104,0,0,...) at vn_start_write+0xdd kern_mkdirat(ca7aab80,ffffff9c,bfbfe41f,0,1f8,...) at kern_mkdirat+0x1eb kern_mkdir(ca7aab80,bfbfe41f,0,1f8,f26ffd1c,...) at kern_mkdir+0x2e sys_mkdir(ca7aab80,f26ffcec,c0fdf70a,c0f96c2a,246,...) at sys_mkdir+0x29 syscall(f26ffd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28195743, esp = 0xbfbfdfcc, ebp = 0xbfbfe3f8 --- Tracing command mkdir pid 2366 tid 100102 td 0xcaae58a0 sched_switch(caae58a0,0,104,1a3,9ce06cad,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,77,...) at mi_switch+0x1ff sleepq_switch(caae58a0,0,c0f93480,1a7,77,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca76b5f0,77,c0f9edb3,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca76b5f0,ca76b5a8,177,c0f9edb3,0,...) at _sleep+0x385 vn_start_write(0,f2768c28,104,0,0,...) at vn_start_write+0xdd kern_mkdirat(caae58a0,ffffff9c,bfbfe41f,0,1f8,...) at kern_mkdirat+0x1eb kern_mkdir(caae58a0,bfbfe41f,0,1f8,f2768d1c,...) at kern_mkdir+0x2e sys_mkdir(caae58a0,f2768cec,c0fdf70a,c0f96c2a,246,...) at sys_mkdir+0x29 syscall(f2768d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28195743, esp = 0xbfbfdfcc, ebp = 0xbfbfe3f8 --- Tracing command mkdir pid 2365 tid 100113 td 0xcab06000 sched_switch(cab06000,0,104,1a3,9ce1e29b,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,77,...) at mi_switch+0x1ff sleepq_switch(cab06000,0,c0f93480,1a7,77,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca76b5f0,77,c0f9edb3,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca76b5f0,ca76b5a8,177,c0f9edb3,0,...) at _sleep+0x385 vn_start_write(0,f27bfc28,104,0,0,...) at vn_start_write+0xdd kern_mkdirat(cab06000,ffffff9c,bfbfe41f,0,1f8,...) at kern_mkdirat+0x1eb kern_mkdir(cab06000,bfbfe41f,0,1f8,f27bfd1c,...) at kern_mkdir+0x2e sys_mkdir(cab06000,f27bfcec,c0fdf70a,c0f96c2a,246,...) at sys_mkdir+0x29 syscall(f27bfd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28195743, esp = 0xbfbfdfcc, ebp = 0xbfbfe3f8 --- Tracing command mkdir pid 2364 tid 100131 td 0xcac602e0 sched_switch(cac602e0,0,104,1a3,9cdf329b,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,77,...) at mi_switch+0x1ff sleepq_switch(cac602e0,0,c0f93480,1a7,77,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca76b5f0,77,c0f9edb3,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca76b5f0,ca76b5a8,177,c0f9edb3,0,...) at _sleep+0x385 vn_start_write(0,f2845c28,104,0,0,...) at vn_start_write+0xdd kern_mkdirat(cac602e0,ffffff9c,bfbfe41f,0,1f8,...) at kern_mkdirat+0x1eb kern_mkdir(cac602e0,bfbfe41f,0,1f8,f2845d1c,...) at kern_mkdir+0x2e sys_mkdir(cac602e0,f2845cec,c0fdf70a,c0f96c2a,246,...) at sys_mkdir+0x29 syscall(f2845d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28195743, esp = 0xbfbfdfcc, ebp = 0xbfbfe3f8 --- Tracing command mkdir pid 2363 tid 100190 td 0xcb4a4b80 sched_switch(cb4a4b80,0,104,1a3,9ce3aaa2,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,77,...) at mi_switch+0x1ff sleepq_switch(cb4a4b80,0,c0f93480,1a7,77,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca76b5f0,77,c0f9edb3,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca76b5f0,ca76b5a8,177,c0f9edb3,0,...) at _sleep+0x385 vn_start_write(0,f28f6c28,104,0,0,...) at vn_start_write+0xdd kern_mkdirat(cb4a4b80,ffffff9c,bfbfe41f,0,1f8,...) at kern_mkdirat+0x1eb kern_mkdir(cb4a4b80,bfbfe41f,0,1f8,f28f6d1c,...) at kern_mkdir+0x2e sys_mkdir(cb4a4b80,f28f6cec,c0fdf70a,c0f96c2a,246,...) at sys_mkdir+0x29 syscall(f28f6d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28195743, esp = 0xbfbfdfcc, ebp = 0xbfbfe3f8 --- Tracing command mkdir pid 2362 tid 100189 td 0xcb4a7000 sched_switch(cb4a7000,0,104,1a3,9ce525b1,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,77,...) at mi_switch+0x1ff sleepq_switch(cb4a7000,0,c0f93480,1a7,77,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca76b5f0,77,c0f9edb3,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca76b5f0,ca76b5a8,177,c0f9edb3,0,...) at _sleep+0x385 vn_start_write(0,f28f3c28,104,0,0,...) at vn_start_write+0xdd kern_mkdirat(cb4a7000,ffffff9c,bfbfe41f,0,1f8,...) at kern_mkdirat+0x1eb kern_mkdir(cb4a7000,bfbfe41f,0,1f8,f28f3d1c,...) at kern_mkdir+0x2e sys_mkdir(cb4a7000,f28f3cec,c0fdf70a,c0f96c2a,246,...) at sys_mkdir+0x29 syscall(f28f3d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28195743, esp = 0xbfbfdfcc, ebp = 0xbfbfe3f8 --- Tracing command mkdir pid 2361 tid 100188 td 0xcb4a72e0 sched_switch(cb4a72e0,0,104,1a3,9ce4475c,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,77,...) at mi_switch+0x1ff sleepq_switch(cb4a72e0,0,c0f93480,1a7,77,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca76b5f0,77,c0f9edb3,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca76b5f0,ca76b5a8,177,c0f9edb3,0,...) at _sleep+0x385 vn_start_write(0,f28f0c28,104,0,0,...) at vn_start_write+0xdd kern_mkdirat(cb4a72e0,ffffff9c,bfbfe41f,0,1f8,...) at kern_mkdirat+0x1eb kern_mkdir(cb4a72e0,bfbfe41f,0,1f8,f28f0d1c,...) at kern_mkdir+0x2e sys_mkdir(cb4a72e0,f28f0cec,c0fdf70a,c0f96c2a,246,...) at sys_mkdir+0x29 syscall(f28f0d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28195743, esp = 0xbfbfdfcc, ebp = 0xbfbfe3f8 --- Tracing command mkdir pid 2360 tid 100187 td 0xcb4a75c0 sched_switch(cb4a75c0,0,104,1a3,9ce587b7,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,77,...) at mi_switch+0x1ff sleepq_switch(cb4a75c0,0,c0f93480,1a7,77,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca76b5f0,77,c0f9edb3,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca76b5f0,ca76b5a8,177,c0f9edb3,0,...) at _sleep+0x385 vn_start_write(0,f28edc28,104,0,0,...) at vn_start_write+0xdd kern_mkdirat(cb4a75c0,ffffff9c,bfbfe41f,0,1f8,...) at kern_mkdirat+0x1eb kern_mkdir(cb4a75c0,bfbfe41f,0,1f8,f28edd1c,...) at kern_mkdir+0x2e sys_mkdir(cb4a75c0,f28edcec,c0fdf70a,c0f96c2a,246,...) at sys_mkdir+0x29 syscall(f28edd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28195743, esp = 0xbfbfdfcc, ebp = 0xbfbfe3f8 --- Tracing command mkdir pid 2359 tid 100186 td 0xcb4a78a0 sched_switch(cb4a78a0,0,104,1a3,9ce16efe,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,77,...) at mi_switch+0x1ff sleepq_switch(cb4a78a0,0,c0f93480,1a7,77,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca76b5f0,77,c0f9edb3,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca76b5f0,ca76b5a8,177,c0f9edb3,0,...) at _sleep+0x385 vn_start_write(0,f28eac28,104,0,0,...) at vn_start_write+0xdd kern_mkdirat(cb4a78a0,ffffff9c,bfbfe41f,0,1f8,...) at kern_mkdirat+0x1eb kern_mkdir(cb4a78a0,bfbfe41f,0,1f8,f28ead1c,...) at kern_mkdir+0x2e sys_mkdir(cb4a78a0,f28eacec,c0fdf70a,c0f96c2a,246,...) at sys_mkdir+0x29 syscall(f28ead28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28195743, esp = 0xbfbfdfcc, ebp = 0xbfbfe3f8 --- Tracing command mkdir pid 2358 tid 100185 td 0xcb1745c0 sched_switch(cb1745c0,0,104,1a3,9cdd76b4,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,77,...) at mi_switch+0x1ff sleepq_switch(cb1745c0,0,c0f93480,1a7,77,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca76b5f0,77,c0f9edb3,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca76b5f0,ca76b5a8,177,c0f9edb3,0,...) at _sleep+0x385 vn_start_write(0,f28e7c28,104,0,0,...) at vn_start_write+0xdd kern_mkdirat(cb1745c0,ffffff9c,bfbfe41f,0,1f8,...) at kern_mkdirat+0x1eb kern_mkdir(cb1745c0,bfbfe41f,0,1f8,f28e7d1c,...) at kern_mkdir+0x2e sys_mkdir(cb1745c0,f28e7cec,c0fdf70a,c0f96c2a,246,...) at sys_mkdir+0x29 syscall(f28e7d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28195743, esp = 0xbfbfdfcc, ebp = 0xbfbfe3f8 --- Tracing command mkdir pid 2357 tid 100184 td 0xcb1748a0 sched_switch(cb1748a0,0,104,1a3,9ce2fe4b,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,77,...) at mi_switch+0x1ff sleepq_switch(cb1748a0,0,c0f93480,1a7,77,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca76b5f0,77,c0f9edb3,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca76b5f0,ca76b5a8,177,c0f9edb3,0,...) at _sleep+0x385 vn_start_write(0,f28e4c28,104,0,0,...) at vn_start_write+0xdd kern_mkdirat(cb1748a0,ffffff9c,bfbfe41f,0,1f8,...) at kern_mkdirat+0x1eb kern_mkdir(cb1748a0,bfbfe41f,0,1f8,f28e4d1c,...) at kern_mkdir+0x2e sys_mkdir(cb1748a0,f28e4cec,c0fdf70a,c0f96c2a,246,...) at sys_mkdir+0x29 syscall(f28e4d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28195743, esp = 0xbfbfdfcc, ebp = 0xbfbfe3f8 --- Tracing command mkdir pid 2356 tid 100156 td 0xca6cc2e0 sched_switch(ca6cc2e0,0,104,1a3,9ce29b22,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,77,...) at mi_switch+0x1ff sleepq_switch(ca6cc2e0,0,c0f93480,1a7,77,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca76b5f0,77,c0f9edb3,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca76b5f0,ca76b5a8,177,c0f9edb3,0,...) at _sleep+0x385 vn_start_write(0,f2890c28,104,0,0,...) at vn_start_write+0xdd kern_mkdirat(ca6cc2e0,ffffff9c,bfbfe41f,0,1f8,...) at kern_mkdirat+0x1eb kern_mkdir(ca6cc2e0,bfbfe41f,0,1f8,f2890d1c,...) at kern_mkdir+0x2e sys_mkdir(ca6cc2e0,f2890cec,c0fdf70a,c0f96c2a,246,...) at sys_mkdir+0x29 syscall(f2890d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28195743, esp = 0xbfbfdfcc, ebp = 0xbfbfe3f8 --- Tracing command mkdir pid 2355 tid 100148 td 0xcaeb4000 sched_switch(caeb4000,0,104,1a3,9ce4b375,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,77,...) at mi_switch+0x1ff sleepq_switch(caeb4000,0,c0f93480,1a7,77,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca76b5f0,77,c0f9edb3,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca76b5f0,ca76b5a8,177,c0f9edb3,0,...) at _sleep+0x385 vn_start_write(0,f2878c28,104,0,0,...) at vn_start_write+0xdd kern_mkdirat(caeb4000,ffffff9c,bfbfe41f,0,1f8,...) at kern_mkdirat+0x1eb kern_mkdir(caeb4000,bfbfe41f,0,1f8,f2878d1c,...) at kern_mkdir+0x2e sys_mkdir(caeb4000,f2878cec,c0fdf70a,c0f96c2a,246,...) at sys_mkdir+0x29 syscall(f2878d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28195743, esp = 0xbfbfdfcc, ebp = 0xbfbfe3f8 --- Tracing command mkdir pid 2354 tid 100136 td 0xcac5bb80 sched_switch(cac5bb80,0,104,1a3,9ce5ec1b,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,77,...) at mi_switch+0x1ff sleepq_switch(cac5bb80,0,c0f93480,1a7,77,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca76b5f0,77,c0f9edb3,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca76b5f0,ca76b5a8,177,c0f9edb3,0,...) at _sleep+0x385 vn_start_write(0,f2854c28,104,0,0,...) at vn_start_write+0xdd kern_mkdirat(cac5bb80,ffffff9c,bfbfe41f,0,1f8,...) at kern_mkdirat+0x1eb kern_mkdir(cac5bb80,bfbfe41f,0,1f8,f2854d1c,...) at kern_mkdir+0x2e sys_mkdir(cac5bb80,f2854cec,c0fdf70a,c0f96c2a,246,...) at sys_mkdir+0x29 syscall(f2854d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28195743, esp = 0xbfbfdfcc, ebp = 0xbfbfe3f8 --- Tracing command mkdir pid 2353 tid 100132 td 0xcac60000 sched_switch(cac60000,0,104,1a3,9cdeae78,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,77,...) at mi_switch+0x1ff sleepq_switch(cac60000,0,c0f93480,1a7,77,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca76b5f0,77,c0f9edb3,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca76b5f0,ca76b5a8,177,c0f9edb3,0,...) at _sleep+0x385 vn_start_write(0,f2848c28,104,0,0,...) at vn_start_write+0xdd kern_mkdirat(cac60000,ffffff9c,bfbfe41f,0,1f8,...) at kern_mkdirat+0x1eb kern_mkdir(cac60000,bfbfe41f,0,1f8,f2848d1c,...) at kern_mkdir+0x2e sys_mkdir(cac60000,f2848cec,c0fdf70a,c0f96c2a,246,...) at sys_mkdir+0x29 syscall(f2848d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28195743, esp = 0xbfbfdfcc, ebp = 0xbfbfe3f8 --- Tracing command mkdir pid 2352 tid 100085 td 0xca6cd5c0 sched_switch(ca6cd5c0,0,104,1a3,9ce362fa,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,77,...) at mi_switch+0x1ff sleepq_switch(ca6cd5c0,0,c0f93480,1a7,77,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca76b5f0,77,c0f9edb3,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca76b5f0,ca76b5a8,177,c0f9edb3,0,...) at _sleep+0x385 vn_start_write(0,f2717c28,104,0,0,...) at vn_start_write+0xdd kern_mkdirat(ca6cd5c0,ffffff9c,bfbfe41f,0,1f8,...) at kern_mkdirat+0x1eb kern_mkdir(ca6cd5c0,bfbfe41f,0,1f8,f2717d1c,...) at kern_mkdir+0x2e sys_mkdir(ca6cd5c0,f2717cec,c0fdf70a,c0f96c2a,246,...) at sys_mkdir+0x29 syscall(f2717d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28195743, esp = 0xbfbfdfcc, ebp = 0xbfbfe3f8 --- Tracing command mkdir pid 2351 tid 100090 td 0xca7a9000 sched_switch(ca7a9000,0,104,1a3,9ce0f002,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,77,...) at mi_switch+0x1ff sleepq_switch(ca7a9000,0,c0f93480,1a7,77,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca76b5f0,77,c0f9edb3,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca76b5f0,ca76b5a8,177,c0f9edb3,0,...) at _sleep+0x385 vn_start_write(0,f2743c28,104,0,0,...) at vn_start_write+0xdd kern_mkdirat(ca7a9000,ffffff9c,bfbfe41f,0,1f8,...) at kern_mkdirat+0x1eb kern_mkdir(ca7a9000,bfbfe41f,0,1f8,f2743d1c,...) at kern_mkdir+0x2e sys_mkdir(ca7a9000,f2743cec,c0fdf70a,c0f96c2a,246,...) at sys_mkdir+0x29 syscall(f2743d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28195743, esp = 0xbfbfdfcc, ebp = 0xbfbfe3f8 --- Tracing command mkdir pid 2350 tid 100082 td 0xca7aa2e0 sched_switch(ca7aa2e0,0,104,1a3,9cdc9e3c,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,77,...) at mi_switch+0x1ff sleepq_switch(ca7aa2e0,0,c0f93480,1a7,77,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca76b5f0,77,c0f9edb3,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca76b5f0,ca76b5a8,177,c0f9edb3,0,...) at _sleep+0x385 vn_start_write(0,f270bc28,104,0,0,...) at vn_start_write+0xdd kern_mkdirat(ca7aa2e0,ffffff9c,bfbfe41f,0,1f8,...) at kern_mkdirat+0x1eb kern_mkdir(ca7aa2e0,bfbfe41f,0,1f8,f270bd1c,...) at kern_mkdir+0x2e sys_mkdir(ca7aa2e0,f270bcec,c0fdf70a,c0f96c2a,246,...) at sys_mkdir+0x29 syscall(f270bd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28195743, esp = 0xbfbfdfcc, ebp = 0xbfbfe3f8 --- Tracing command mkdir pid 2349 tid 100120 td 0xcaae68a0 sched_switch(caae68a0,0,104,1a3,9cde3d73,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,77,...) at mi_switch+0x1ff sleepq_switch(caae68a0,0,c0f93480,1a7,77,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca76b5f0,77,c0f9edb3,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca76b5f0,ca76b5a8,177,c0f9edb3,0,...) at _sleep+0x385 vn_start_write(0,f27d4c28,104,0,0,...) at vn_start_write+0xdd kern_mkdirat(caae68a0,ffffff9c,bfbfe41f,0,1f8,...) at kern_mkdirat+0x1eb kern_mkdir(caae68a0,bfbfe41f,0,1f8,f27d4d1c,...) at kern_mkdir+0x2e sys_mkdir(caae68a0,f27d4cec,c0fdf70a,c0f96c2a,246,...) at sys_mkdir+0x29 syscall(f27d4d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28195743, esp = 0xbfbfdfcc, ebp = 0xbfbfe3f8 --- Tracing command mkdir pid 2348 tid 100183 td 0xcb1735c0 sched_switch(cb1735c0,0,104,1a3,9cdcfa51,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(cb1735c0,0,c0f93480,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cb158000,6c,c0f96397,100,0,...) at sleepq_wait_sig+0x17 _sleep(cb158000,cb158088,16c,c0f96397,0,...) at _sleep+0x385 kern_wait(cb1735c0,92d,f28e1c60,0,0,...) at kern_wait+0x626 sys_wait4(cb1735c0,f28e1cec,c0fdf70a,c0f96c2a,216,...) at sys_wait4+0x3b syscall(f28e1d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2811245b, esp = 0xbfbfe80c, ebp = 0xbfbfe828 --- Tracing command mkdir pid 2346 tid 100164 td 0xcb173b80 sched_switch(cb173b80,0,104,1a3,9cd9f803,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(cb173b80,0,c0f93480,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cb158588,6c,c0f96397,100,0,...) at sleepq_wait_sig+0x17 _sleep(cb158588,cb158610,16c,c0f96397,0,...) at _sleep+0x385 kern_wait(cb173b80,92c,f28a8c60,0,0,...) at kern_wait+0x626 sys_wait4(cb173b80,f28a8cec,c,c,216,...) at sys_wait4+0x3b syscall(f28a8d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2811245b, esp = 0xbfbfe80c, ebp = 0xbfbfe828 --- Tracing command run pid 2344 tid 100104 td 0xcaae52e0 sched_switch(caae52e0,0,104,1a3,63d021f3,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(caae52e0,0,c0f93480,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(caadd2c4,6c,c0f96397,100,0,...) at sleepq_wait_sig+0x17 _sleep(caadd2c4,caadd34c,16c,c0f96397,0,...) at _sleep+0x385 kern_wait(caae52e0,92a,f2770c60,0,0,...) at kern_wait+0x626 sys_wait4(caae52e0,f2770cec,c0fdf70a,c0f96c2a,216,...) at sys_wait4+0x3b syscall(f2770d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2811145b, esp = 0xbfbfe29c, ebp = 0xbfbfe2b8 --- Tracing command run pid 2343 tid 100087 td 0xca7a95c0 sched_switch(ca7a95c0,0,104,1a3,63cf9bf1,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(ca7a95c0,0,c0f93480,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca6c1588,6c,c0f96397,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca6c1588,ca6c1610,16c,c0f96397,0,...) at _sleep+0x385 kern_wait(ca7a95c0,928,f271fc60,0,0,...) at kern_wait+0x626 sys_wait4(ca7a95c0,f271fcec,c0fdf70a,c0f96c2a,216,...) at sys_wait4+0x3b syscall(f271fd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2811145b, esp = 0xbfbfe7bc, ebp = 0xbfbfe7d8 --- Tracing command md6 pid 2320 tid 100155 td 0xca6cc5c0 sched_switch(ca6cc5c0,0,104,1a3,2382f0f8,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(ca6cc5c0,0,c0f93480,268,0,...) at sleepq_switch+0x162 sleepq_wait(ca6bd800,5c,c0f2a562,0,0,...) at sleepq_wait+0x63 _sleep(ca6bd800,ca6bd820,25c,c0f2a562,0,...) at _sleep+0x39c md_kthread(ca6bd800,f288dd28,c0f88190,3db,cb15a2c4,...) at md_kthread+0x11a fork_exit(c0769770,ca6bd800,f288dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xf288dd60, ebp = 0 --- Tracing command run pid 2314 tid 100153 td 0xca6ccb80 sched_switch(ca6ccb80,0,104,1a3,6391a134,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(ca6ccb80,0,c0f93480,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cb15a84c,6c,c0f96397,100,0,...) at sleepq_wait_sig+0x17 _sleep(cb15a84c,cb15a8d4,16c,c0f96397,0,...) at _sleep+0x385 kern_wait(ca6ccb80,927,f2887c60,0,0,...) at kern_wait+0x626 sys_wait4(ca6ccb80,f2887cec,c,c,216,...) at sys_wait4+0x3b syscall(f2887d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2811145b, esp = 0xbfbfe7bc, ebp = 0xbfbfe7d8 --- Tracing command su pid 2308 tid 100152 td 0xca6cd000 sched_switch(ca6cd000,0,104,1a3,1e488d0d,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(ca6cd000,0,c0f93480,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cb15ab10,6c,c0f96397,100,0,...) at sleepq_wait_sig+0x17 _sleep(cb15ab10,cb15ab98,16c,c0f96397,0,...) at _sleep+0x385 kern_wait(ca6cd000,90a,f2884c60,2,0,...) at kern_wait+0x626 sys_wait4(ca6cd000,f2884cec,c,c,216,...) at sys_wait4+0x3b syscall(f2884d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2813245b, esp = 0xbfbfe37c, ebp = 0xbfbfe398 --- Tracing command md5 pid 2303 tid 100139 td 0xcaeb7b80 sched_switch(caeb7b80,0,104,1a3,4875e52,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(caeb7b80,0,c0f93480,268,0,...) at sleepq_switch+0x162 sleepq_wait(ca6be000,5c,c0f2a562,0,0,...) at sleepq_wait+0x63 _sleep(ca6be000,ca6be020,25c,c0f2a562,0,...) at _sleep+0x39c md_kthread(ca6be000,f285dd28,c0f88190,3db,caeb184c,...) at md_kthread+0x11a fork_exit(c0769770,ca6be000,f285dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xf285dd60, ebp = 0 --- Tracing command sh pid 2290 tid 100157 td 0xca6cc000 sched_switch(ca6cc000,0,104,1a3,a4944a4e,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(ca6cc000,0,c0f93480,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cb159b10,6c,c0f96397,100,0,...) at sleepq_wait_sig+0x17 _sleep(cb159b10,cb159b98,16c,c0f96397,0,...) at _sleep+0x385 kern_wait(ca6cc000,ffffffff,f2893c60,2,0,...) at kern_wait+0x626 sys_wait4(ca6cc000,f2893cec,c0fdf70a,c0f96376,216,...) at sys_wait4+0x3b syscall(f2893d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2817645b, esp = 0xbfbfe84c, ebp = 0xbfbfe868 --- Tracing command awk pid 2274 tid 100174 td 0xcb24a2e0 sched_switch(cb24a2e0,0,104,1a3,775b8c60,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(cb24a2e0,0,c0f93480,1a7,5c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cab427d0,5c,c0f95dcf,100,0,...) at sleepq_wait_sig+0x17 _sleep(cab427d0,cab42948,15c,c0f95dcf,0,...) at _sleep+0x385 pipe_read(cac1c9d8,f28c6c44,caeaee80,0,cb24a2e0,...) at pipe_read+0x417 dofileread(f28c6c44,ffffffff,ffffffff,0,f28c6c20,...) at dofileread+0x9e kern_readv(cb24a2e0,0,f28c6c44,f28c6c64,1,...) at kern_readv+0x68 sys_read(cb24a2e0,f28c6cec,f28c6cc4,f28c6d80,216,...) at sys_read+0x4f syscall(f28c6d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281c8333, esp = 0xbfbfe93c, ebp = 0xbfbfe958 --- Tracing command sh pid 2273 tid 100180 td 0xcb249000 sched_switch(cb249000,0,104,1a3,7791b586,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(cb249000,0,c0f93480,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cb1fd2c4,6c,c0f96397,100,0,...) at sleepq_wait_sig+0x17 _sleep(cb1fd2c4,cb1fd34c,16c,c0f96397,0,...) at _sleep+0x385 kern_wait(cb249000,ffffffff,f28d8c60,2,0,...) at kern_wait+0x626 sys_wait4(cb249000,f28d8cec,c0fdf70a,c0f96376,216,...) at sys_wait4+0x3b syscall(f28d8d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2817645b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9e8 --- Tracing command sh pid 2272 tid 100130 td 0xcac605c0 sched_switch(cac605c0,0,104,1a3,ff378a35,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(cac605c0,0,c0f93480,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cac5784c,6c,c0f96397,100,0,...) at sleepq_wait_sig+0x17 _sleep(cac5784c,cac578d4,16c,c0f96397,0,...) at _sleep+0x385 kern_wait(cac605c0,ffffffff,f2842c60,2,0,...) at kern_wait+0x626 sys_wait4(cac605c0,f2842cec,c,c,216,...) at sys_wait4+0x3b syscall(f2842d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2817645b, esp = 0xbfbfeb1c, ebp = 0xbfbfeb38 --- Tracing command sshd pid 2271 tid 100162 td 0xcb1742e0 sched_switch(cb1742e0,0,104,1a3,fa6ac94f,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cb1742e0,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b8da,cb101890,0,c0f8beab,cb1742e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cb1018a4,0,f28a2a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(cb1018a4,cb101890,c0f95bd0,6a4,cac1c8c0,...) at _cv_wait_sig+0x260 seltdwait(cac1c8c0,58,c7dc4a80,cb1742e0,c7dcb488,...) at seltdwait+0xa2 kern_select(cb1742e0,c,288060b8,288060c0,0,0,20,cb158b10,0) at kern_select+0x655 sys_select(cb1742e0,f28a2cec,c0fdf70a,c0f964eb,206,...) at sys_select+0x66 syscall(f28a2d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284d72b3, esp = 0xbfbfde2c, ebp = 0xbfbfde78 --- Tracing command tail pid 2270 tid 100080 td 0xca7aa8a0 sched_switch(ca7aa8a0,0,104,1a3,b763fb79,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,60,...) at mi_switch+0x1ff sleepq_switch(ca7aa8a0,0,c0f93480,268,60,...) at sleepq_switch+0x162 sleepq_wait(cad391b4,60,c0f840cc,4,0,...) at sleepq_wait+0x63 __lockmgr_args(cad391b4,80400,cad39220,0,0,...) at __lockmgr_args+0xb3f ffs_lock(f2703aa8,f2703ae0,246,80400,cad3915c,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c10dd620,f2703aa8,ca7aa950,c10ee640,cad3915c,...) at VOP_LOCK1_APV+0xb5 _vn_lock(cad3915c,80400,c0f9d50c,10ae,f2703bfc,...) at _vn_lock+0x78 vfs_knllock(cad3915c,0,c0f87ab1,59b,3e9,...) at vfs_knllock+0x29 kern_kevent(ca7aa8a0,4,0,1,f2703c44,...) at kern_kevent+0x653 sys_kevent(ca7aa8a0,f2703cec,c0fdf70a,c0dee437,286,...) at sys_kevent+0x19e syscall(f2703d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (363, FreeBSD ELF32, sys_kevent), eip = 0x28171d3b, esp = 0xbfbfeb1c, ebp = 0xbfbfebd8 --- Tracing command top pid 2269 tid 100094 td 0xca7a55c0 sched_switch(ca7a55c0,0,104,1a3,a203b55e,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca7a55c0,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f274fa20,c0a4b8da,cb101350,0,ca7a55c0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(cb101364,0,f274fa58,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(cb101364,cb101350,3e9,6a4,cad2e2d8,...) at _cv_timedwait_sig+0x270 seltdwait(f274fc10,f274fc18,caeaed00,ca7a55c0,3b31325b,...) at seltdwait+0x8a kern_select(ca7a55c0,2,bfbfebac,0,0,f274fc5c,20,1,0) at kern_select+0x655 sys_select(ca7a55c0,f274fcec,cb174000,f274fcc0,246,...) at sys_select+0x66 syscall(f274fd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281fd2b3, esp = 0xbfbfeb4c, ebp = 0xbfbfeca8 --- Tracing command sshd pid 2268 tid 100181 td 0xcb174b80 sched_switch(cb174b80,0,104,1a3,194e427a,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cb174b80,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b8da,ca6b46d0,0,c0f8beab,cb174b80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca6b46e4,0,f28dba58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca6b46e4,ca6b46d0,c0f95bd0,6a4,ca7dc888,...) at _cv_wait_sig+0x260 seltdwait(ca7dc888,58,c7dc5800,cb174b80,f28dbacc,...) at seltdwait+0xa2 kern_select(cb174b80,c,288060b8,288060c0,0,0,20,cb1fd000,0) at kern_select+0x655 sys_select(cb174b80,f28dbcec,c0fdf70a,c0f964eb,206,...) at sys_select+0x66 syscall(f28dbd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284d72b3, esp = 0xbfbfde2c, ebp = 0xbfbfde78 --- Tracing command sshd pid 2267 tid 100163 td 0xcb174000 sched_switch(cb174000,0,104,1a3,a2037e3c,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cb174000,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b8da,cab18210,0,c0f8beab,cb174000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cab18224,0,f28a5a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(cab18224,cab18210,c0f95bd0,6a4,ca7b55e8,...) at _cv_wait_sig+0x260 seltdwait(ca7b55e8,58,ca848000,cb174000,f28a5acc,...) at seltdwait+0xa2 kern_select(cb174000,a,288060b8,288060c0,0,0,20,cb15884c,0) at kern_select+0x655 sys_select(cb174000,f28a5cec,c0fdf70a,c0f964eb,206,...) at sys_select+0x66 syscall(f28a5d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284d72b3, esp = 0xbfbfde2c, ebp = 0xbfbfde78 --- Tracing command sshd pid 2261 tid 100178 td 0xcb2495c0 sched_switch(cb2495c0,0,104,1a3,fa4bcd5a,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cb2495c0,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b8da,cab184d0,0,c0f8beab,cb2495c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cab184e4,0,f28d2abc,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(cab184e4,cab184d0,c0f95bd0,6a4,f28d2b44,...) at _cv_wait_sig+0x260 seltdwait(caad012c,c0f95bd0,5a1,cb2495c0,0,...) at seltdwait+0xa2 sys_poll(cb2495c0,f28d2cec,c0fdf70a,c0dee437,202,...) at sys_poll+0x321 syscall(f28d2d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x2847014b, esp = 0xbfbfde8c, ebp = 0xbfbfdec8 --- Tracing command sshd pid 2260 tid 100177 td 0xcb2498a0 sched_switch(cb2498a0,0,104,1a3,f9158d24,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cb2498a0,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b8da,ca6b4dd0,0,c0f8beab,cb2498a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca6b4de4,0,f28cfabc,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca6b4de4,ca6b4dd0,c0f95bd0,6a4,f28cfb44,...) at _cv_wait_sig+0x260 seltdwait(cac4312c,c0f95bd0,5a1,cb2498a0,0,...) at seltdwait+0xa2 sys_poll(cb2498a0,f28cfcec,c,c,202,...) at sys_poll+0x321 syscall(f28cfd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x2847014b, esp = 0xbfbfde8c, ebp = 0xbfbfdec8 --- Tracing command sshd pid 2259 tid 100176 td 0xcb249b80 sched_switch(cb249b80,0,104,1a3,f9320cac,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cb249b80,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b8da,cb1028d0,0,c0f8beab,cb249b80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cb1028e4,0,f28ccabc,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(cb1028e4,cb1028d0,c0f95bd0,6a4,f28ccb44,...) at _cv_wait_sig+0x260 seltdwait(cac3012c,c0f95bd0,5a1,cb249b80,0,...) at seltdwait+0xa2 sys_poll(cb249b80,f28cccec,c0fdf70a,c0dee437,202,...) at sys_poll+0x321 syscall(f28ccd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x2847014b, esp = 0xbfbfde8c, ebp = 0xbfbfdec8 --- Tracing command csh pid 2255 tid 100092 td 0xca7a5b80 sched_switch(ca7a5b80,0,104,1a3,351bb16,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,74,...) at mi_switch+0x1ff sleepq_switch(ca7a5b80,0,c0f93480,1a7,74,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(caad38a4,74,c0e14cae,100,0,...) at sleepq_wait_sig+0x17 _sleep(caad38a4,caad38d4,174,c0e14cae,0,...) at _sleep+0x385 kern_sigsuspend(ca7a5b80,0,0,0,0,...) at kern_sigsuspend+0xba sys_sigsuspend(ca7a5b80,f2749cec,c0fdf70a,c0f96286,202,...) at sys_sigsuspend+0x4d syscall(f2749d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28195ccb, esp = 0xbfbfe58c, ebp = 0xbfbfe5f8 --- Tracing command su pid 2254 tid 100126 td 0xcac612e0 sched_switch(cac612e0,0,104,1a3,7651c2d,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(cac612e0,0,c0f93480,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cac59588,6c,c0f96397,100,0,...) at sleepq_wait_sig+0x17 _sleep(cac59588,cac59610,16c,c0f96397,0,...) at _sleep+0x385 kern_wait(cac612e0,8cf,f2836c60,2,0,...) at kern_wait+0x626 sys_wait4(cac612e0,f2836cec,c,c,216,...) at sys_wait4+0x3b syscall(f2836d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2813245b, esp = 0xbfbfe5bc, ebp = 0xbfbfe5d8 --- Tracing command bash pid 2249 tid 100160 td 0xc81855c0 sched_switch(c81855c0,0,104,1a3,e78327ac,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(c81855c0,0,c0f93480,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cb1592c4,6c,c0f96397,100,0,...) at sleepq_wait_sig+0x17 _sleep(cb1592c4,cb15934c,16c,c0f96397,0,...) at _sleep+0x385 kern_wait(c81855c0,ffffffff,f289cc60,6,0,...) at kern_wait+0x626 sys_wait4(c81855c0,f289ccec,c0fdf70a,c0f96286,216,...) at sys_wait4+0x3b syscall(f289cd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x282cf45b, esp = 0xbfbfe97c, ebp = 0xbfbfe998 --- Tracing command sshd pid 2248 tid 100172 td 0xcb24a8a0 sched_switch(cb24a8a0,0,104,1a3,b27d4723,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cb24a8a0,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b8da,cb1017d0,0,c0f8beab,cb24a8a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cb1017e4,0,f28c0a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(cb1017e4,cb1017d0,c0f95bd0,6a4,ca7b4c08,...) at _cv_wait_sig+0x260 seltdwait(ca7b4c08,58,c7dc5e80,cb24a8a0,f28c0acc,...) at seltdwait+0xa2 kern_select(cb24a8a0,c,288060b8,288060c0,0,0,20,cb200b10,0) at kern_select+0x655 sys_select(cb24a8a0,f28c0cec,c0fdf70a,c0f964eb,206,...) at sys_select+0x66 syscall(f28c0d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284d72b3, esp = 0xbfbfde2c, ebp = 0xbfbfde78 --- Tracing command sshd pid 2246 tid 100171 td 0xcb24ab80 sched_switch(cb24ab80,0,104,1a3,856210b8,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cb24ab80,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b8da,cb102d10,0,c0f8beab,cb24ab80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cb102d24,0,f28bdabc,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(cb102d24,cb102d10,c0f95bd0,6a4,f28bdb44,...) at _cv_wait_sig+0x260 seltdwait(caad0b2c,c0f95bd0,5a1,cb24ab80,8,...) at seltdwait+0xa2 sys_poll(cb24ab80,f28bdcec,c0fdf70a,c0f7e579,202,...) at sys_poll+0x321 syscall(f28bdd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x2847014b, esp = 0xbfbfde8c, ebp = 0xbfbfdec8 --- Tracing command getty pid 2236 tid 100167 td 0xcaeb2000 sched_switch(caeb2000,0,104,1a3,737cd4ce,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(caeb2000,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,caeb2000,caeb2000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7d97470,0,c0f982b8,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7d97470,c1129a90,c0f97c90,527,0,...) at _cv_wait_sig+0x260 tty_wait(c7d97400,c7d97470,f28b1c44,1,0,...) at tty_wait+0x71 ttydisc_read(c7d97400,f28b1c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c8193400,f28b1c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(cac56268,f28b1c44,c7dc4e80,0,caeb2000,...) at devfs_read_f+0x7e dofileread(f28b1c44,ffffffff,ffffffff,0,f28b1c20,...) at dofileread+0x9e kern_readv(caeb2000,0,f28b1c44,f28b1c64,1,...) at kern_readv+0x68 sys_read(caeb2000,f28b1cec,c0fdf70a,c0df3176,246,...) at sys_read+0x4f syscall(f28b1d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a6333, esp = 0xbfbfed7c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2235 tid 100166 td 0xcaeb22e0 sched_switch(caeb22e0,0,104,1a3,73a11296,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(caeb22e0,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,caeb22e0,caeb22e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7d97670,0,c0f982b8,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7d97670,c1129a90,c0f97c90,527,0,...) at _cv_wait_sig+0x260 tty_wait(c7d97600,c7d97670,f28aec44,1,0,...) at tty_wait+0x71 ttydisc_read(c7d97600,f28aec44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c8193500,f28aec44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(ca7dcc08,f28aec44,c7dc4e80,0,caeb22e0,...) at devfs_read_f+0x7e dofileread(f28aec44,ffffffff,ffffffff,0,f28aec20,...) at dofileread+0x9e kern_readv(caeb22e0,0,f28aec44,f28aec64,1,...) at kern_readv+0x68 sys_read(caeb22e0,f28aecec,c0fdf70a,c0df3176,246,...) at sys_read+0x4f syscall(f28aed28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a6333, esp = 0xbfbfed7c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2234 tid 100101 td 0xcaae5b80 sched_switch(caae5b80,0,104,1a3,7399c2a8,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(caae5b80,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,caae5b80,caae5b80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7d97870,0,c0f982b8,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7d97870,c1129a90,c0f97c90,527,0,...) at _cv_wait_sig+0x260 tty_wait(c7d97800,c7d97870,f2764c44,1,0,...) at tty_wait+0x71 ttydisc_read(c7d97800,f2764c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c8193600,f2764c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(ca7b5818,f2764c44,c7dc4e80,0,caae5b80,...) at devfs_read_f+0x7e dofileread(f2764c44,ffffffff,ffffffff,0,f2764c20,...) at dofileread+0x9e kern_readv(caae5b80,0,f2764c44,f2764c64,1,...) at kern_readv+0x68 sys_read(caae5b80,f2764cec,c0fdf70a,c0df3176,246,...) at sys_read+0x4f syscall(f2764d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a6333, esp = 0xbfbfed7c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2233 tid 100105 td 0xcaae5000 sched_switch(caae5000,0,104,1a3,73a072bd,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(caae5000,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,caae5000,caae5000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7d97a70,0,c0f982b8,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7d97a70,c1129a90,c0f97c90,527,0,...) at _cv_wait_sig+0x260 tty_wait(c7d97a00,c7d97a70,f2789c44,1,0,...) at tty_wait+0x71 ttydisc_read(c7d97a00,f2789c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c8075500,f2789c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(cac52690,f2789c44,c7dc4e80,0,caae5000,...) at devfs_read_f+0x7e dofileread(f2789c44,ffffffff,ffffffff,0,f2789c20,...) at dofileread+0x9e kern_readv(caae5000,0,f2789c44,f2789c64,1,...) at kern_readv+0x68 sys_read(caae5000,f2789cec,c0fdf70a,c0df3176,246,...) at sys_read+0x4f syscall(f2789d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a6333, esp = 0xbfbfed7c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2232 tid 100149 td 0xcaeb2b80 sched_switch(caeb2b80,0,104,1a3,739d90b8,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(caeb2b80,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,caeb2b80,caeb2b80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7d97c70,0,c0f982b8,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7d97c70,c1129a90,c0f97c90,527,0,...) at _cv_wait_sig+0x260 tty_wait(c7d97c00,c7d97c70,f287bc44,1,0,...) at tty_wait+0x71 ttydisc_read(c7d97c00,f287bc44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c8075600,f287bc44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(ca823c40,f287bc44,c7dc4e80,0,caeb2b80,...) at devfs_read_f+0x7e dofileread(f287bc44,ffffffff,ffffffff,0,f287bc20,...) at dofileread+0x9e kern_readv(caeb2b80,0,f287bc44,f287bc64,1,...) at kern_readv+0x68 sys_read(caeb2b80,f287bcec,c0fdf70a,c0df3176,246,...) at sys_read+0x4f syscall(f287bd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a6333, esp = 0xbfbfed7c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2231 tid 100128 td 0xcac60b80 sched_switch(cac60b80,0,104,1a3,73944a6e,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cac60b80,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,cac60b80,cac60b80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7d97e70,0,c0f982b8,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7d97e70,c1129a90,c0f97c90,527,0,...) at _cv_wait_sig+0x260 tty_wait(c7d97e00,c7d97e70,f283cc44,1,0,...) at tty_wait+0x71 ttydisc_read(c7d97e00,f283cc44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c8075700,f283cc44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(cac56310,f283cc44,c7dc4e80,0,cac60b80,...) at devfs_read_f+0x7e dofileread(f283cc44,ffffffff,ffffffff,0,f283cc20,...) at dofileread+0x9e kern_readv(cac60b80,0,f283cc44,f283cc64,1,...) at kern_readv+0x68 sys_read(cac60b80,f283ccec,c0fdf70a,c0df3176,246,...) at sys_read+0x4f syscall(f283cd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a6333, esp = 0xbfbfed7c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2230 tid 100127 td 0xcac61000 sched_switch(cac61000,0,104,1a3,7385ca4a,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cac61000,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,cac61000,cac61000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8181070,0,c0f982b8,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c8181070,c1129a90,c0f97c90,527,0,...) at _cv_wait_sig+0x260 tty_wait(c8181000,c8181070,f2839c44,1,0,...) at tty_wait+0x71 ttydisc_read(c8181000,f2839c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c8075800,f2839c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(ca7b5e00,f2839c44,c7dc4e80,0,cac61000,...) at devfs_read_f+0x7e dofileread(f2839c44,ffffffff,ffffffff,0,f2839c20,...) at dofileread+0x9e kern_readv(cac61000,0,f2839c44,f2839c64,1,...) at kern_readv+0x68 sys_read(cac61000,f2839cec,c0fdf70a,c0df3176,246,...) at sys_read+0x4f syscall(f2839d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a6333, esp = 0xbfbfed7c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2229 tid 100091 td 0xcaae6000 sched_switch(caae6000,0,104,1a3,73a45cdb,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(caae6000,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,caae6000,caae6000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8181270,0,c0f982b8,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c8181270,c1129a90,c0f97c90,527,0,...) at _cv_wait_sig+0x260 tty_wait(c8181200,c8181270,f2746c44,1,0,...) at tty_wait+0x71 ttydisc_read(c8181200,f2746c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c8075900,f2746c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(cac52c08,f2746c44,c7dc4e80,0,caae6000,...) at devfs_read_f+0x7e dofileread(f2746c44,ffffffff,ffffffff,0,f2746c20,...) at dofileread+0x9e kern_readv(caae6000,0,f2746c44,f2746c64,1,...) at kern_readv+0x68 sys_read(caae6000,f2746cec,c0fdf70a,c0df3176,246,...) at sys_read+0x4f syscall(f2746d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a6333, esp = 0xbfbfed7c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2228 tid 100095 td 0xca7a52e0 sched_switch(ca7a52e0,0,104,1a3,55a0e578,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca7a52e0,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b8da,c8181404,0,c0f8beab,ca7a52e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8181470,0,f2752af0,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c8181470,c8181404,c0f97c90,527,0,...) at _cv_wait_sig+0x260 tty_wait(c8181400,c8181470,f2752c44,1,0,...) at tty_wait+0x71 ttydisc_read(c8181400,f2752c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c7ed4e00,f2752c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(cac56540,f2752c44,c7dc4e80,0,ca7a52e0,...) at devfs_read_f+0x7e dofileread(f2752c44,ffffffff,ffffffff,0,f2752c20,...) at dofileread+0x9e kern_readv(ca7a52e0,0,f2752c44,f2752c64,1,...) at kern_readv+0x68 sys_read(ca7a52e0,f2752cec,c0fdf70a,c0df3176,246,...) at sys_read+0x4f syscall(f2752d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a6333, esp = 0xbfbfed7c, ebp = 0xbfbfedb8 --- Tracing command inetd pid 2189 tid 100086 td 0xca7a98a0 sched_switch(ca7a98a0,0,104,1a3,45db4fef,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca7a98a0,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b8da,ca769290,0,c0f8beab,ca7a98a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca7692a4,0,f271ba58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca7692a4,ca769290,c0f95bd0,6a4,cac52b60,...) at _cv_wait_sig+0x260 seltdwait(cac52b60,58,c7dc4e80,ca7a98a0,ca7a9950,...) at seltdwait+0xa2 kern_select(ca7a98a0,6,bfbfdfa0,0,0,0,20,ca6c184c,4) at kern_select+0x655 sys_select(ca7a98a0,f271bcec,c0fdf70a,c0f963e3,206,...) at sys_select+0x66 syscall(f271bd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281b82b3, esp = 0xbfbfdf5c, ebp = 0xbfbfede8 --- Tracing command watchdogd pid 2157 tid 100145 td 0xcaeb48a0 sched_switch(caeb48a0,0,104,1a3,37471fb,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(caeb48a0,0,c0f93480,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(caeb0588,6c,c0f96397,100,0,...) at sleepq_wait_sig+0x17 _sleep(caeb0588,caeb0610,16c,c0f96397,0,...) at _sleep+0x385 kern_wait(caeb48a0,973,f286fc60,0,0,...) at kern_wait+0x626 sys_wait4(caeb48a0,f286fcec,c,c,202,...) at sys_wait4+0x3b syscall(f286fd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2812c45b, esp = 0xbfbfec3c, ebp = 0xbfbfece8 --- Tracing command cron pid 2110 tid 100119 td 0xcaae6b80 sched_switch(caae6b80,0,104,1a3,9a4f7200,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(caae6b80,0,c0f93480,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(ea61,c0a9c4c0,caae6b80,0,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c112b3e4,6c,c0f8efd7,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c112b3e4,0,16c,c0f8efd7,ea61,...) at _sleep+0x34f kern_nanosleep(caae6b80,f27d1c50,f27d1c58,3c,0,...) at kern_nanosleep+0xc1 sys_nanosleep(caae6b80,f27d1cec,c0fdf70a,c0f96844,206,...) at sys_nanosleep+0x6f syscall(f27d1d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x2818fe27, esp = 0xbfbfec4c, ebp = 0xbfbfec78 --- Tracing command sendmail pid 2102 tid 100150 td 0xcaeb28a0 sched_switch(caeb28a0,0,104,1a3,e587a077,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,74,...) at mi_switch+0x1ff sleepq_switch(caeb28a0,0,c0f93480,1a7,74,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cac5a5e0,74,c0e14cae,100,0,...) at sleepq_wait_sig+0x17 _sleep(cac5a5e0,cac5a610,174,c0e14cae,0,...) at _sleep+0x385 kern_sigsuspend(caeb28a0,0,0,0,0,...) at kern_sigsuspend+0xba sys_sigsuspend(caeb28a0,f287ecec,c0fdf70a,c0f96286,202,...) at sys_sigsuspend+0x4d syscall(f287ed28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28348ccb, esp = 0xbfbfcf9c, ebp = 0xbfbfcfc8 --- Tracing command sendmail pid 2098 tid 100083 td 0xca7aa000 sched_switch(ca7aa000,0,104,1a3,9ef5cc42,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca7aa000,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f270fa20,c0a4b8da,ca6b42d0,0,ca7aa000,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(ca6b42e4,0,f270fa58,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(ca6b42e4,ca6b42d0,1389,6a4,ca7b5930,...) at _cv_timedwait_sig+0x270 seltdwait(f270fc10,f270fc18,caeaf280,ca7aa000,c0a4811d,...) at seltdwait+0x8a kern_select(ca7aa000,5,bfbfc510,0,0,f270fc5c,20,5,0) at kern_select+0x655 sys_select(ca7aa000,f270fcec,c0fdf70a,c0f967a3,206,...) at sys_select+0x66 syscall(f270fd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x283f32b3, esp = 0xbfbfc47c, ebp = 0xbfbfcfa8 --- Tracing command sshd pid 2090 tid 100118 td 0xcab05000 sched_switch(cab05000,0,104,1a3,f9e7db42,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cab05000,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b8da,ca769110,0,c0f8beab,cab05000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca769124,0,f27cea58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca769124,ca769110,c0f95bd0,6a4,cac56930,...) at _cv_wait_sig+0x260 seltdwait(cac56930,58,cae18900,cab05000,c7d79338,...) at seltdwait+0xa2 kern_select(cab05000,9,2880c0b8,0,0,0,20,cac3a000,a) at kern_select+0x655 sys_select(cab05000,f27cecec,c0fdf70a,c0f7e579,206,...) at sys_select+0x66 syscall(f27ced28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284d72b3, esp = 0xbfbfdeec, ebp = 0xbfbfedf8 --- Tracing command moused pid 2030 tid 100081 td 0xca7aa5c0 sched_switch(ca7aa5c0,0,104,1a3,c2f4eee5,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca7aa5c0,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b8da,ca6b44d0,0,c0f8beab,ca7aa5c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca6b44e4,0,f2707a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca6b44e4,ca6b44d0,c0f95bd0,6a4,ca7dcaf0,...) at _cv_wait_sig+0x260 seltdwait(ca7dcaf0,58,c7dc4e80,ca7aa5c0,f6,...) at seltdwait+0xa2 kern_select(ca7aa5c0,400,bfbfea04,0,0,0,20,ca7a3588,1) at kern_select+0x655 sys_select(ca7aa5c0,f2707cec,c0fdf70a,c0df3176,246,...) at sys_select+0x66 syscall(f2707d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281c52b3, esp = 0xbfbfe99c, ebp = 0xbfbfeb28 --- Tracing command ntpd pid 1952 tid 100143 td 0xcaeb7000 sched_switch(caeb7000,0,104,1a3,2128b7bf,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(caeb7000,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b8da,ca8216d0,0,c0f8beab,caeb7000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca8216e4,0,f2869a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca8216e4,ca8216d0,c0f95bd0,6a4,cac1caf0,...) at _cv_wait_sig+0x260 seltdwait(cac1caf0,58,c7dc4e80,caeb7000,0,...) at seltdwait+0xa2 kern_select(caeb7000,17,bfbfece8,0,0,0,20,caeb0b10,0) at kern_select+0x655 sys_select(caeb7000,f2869cec,c0fdf70a,f2869d80,206,...) at sys_select+0x66 syscall(f2869d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x283722b3, esp = 0xbfbfecbc, ebp = 0xbfbfed88 --- Tracing command nfsd pid 1853 tid 100135 td 0xcaadf5c0 sched_switch(caadf5c0,0,104,1a3,617cf7d2,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(caadf5c0,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f2851be0,c0a4b8da,c800f900,0,caadf5c0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(cab64b50,0,f2851c18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(cab64b50,c800f900,1388,3b1,0,...) at _cv_timedwait_sig+0x270 svc_run_internal(f2851d14,c0a2e418,c800f900,f2851d28,c0f88190,...) at svc_run_internal+0x356 svc_thread_start(c800f900,f2851d28,c0f88190,3db,c804284c,...) at svc_thread_start+0x10 fork_exit(c0c4b4e0,c800f900,f2851d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x2e, eip = 0xc, esp = 0x33, ebp = 0x1 --- Tracing command nfsd pid 1853 tid 100134 td 0xcaadf8a0 sched_switch(caadf8a0,0,104,1a3,cb2829c9,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(caadf8a0,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f284ebe0,c0a4b8da,c800f900,0,caadf8a0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(cab64b90,0,f284ec18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(cab64b90,c800f900,1388,3b1,0,...) at _cv_timedwait_sig+0x270 svc_run_internal(f284ed14,c0a2e418,c800f900,f284ed28,c0f88190,...) at svc_run_internal+0x356 svc_thread_start(c800f900,f284ed28,c0f88190,3db,c804284c,...) at svc_thread_start+0x10 fork_exit(c0c4b4e0,c800f900,f284ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x2e, eip = 0xc, esp = 0x33, ebp = 0x1 --- Tracing command nfsd pid 1853 tid 100133 td 0xcaadfb80 sched_switch(caadfb80,0,104,1a3,b433ba59,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(caadfb80,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f284bbe0,c0a4b8da,c800f900,0,caadfb80,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(cab64bd0,0,f284bc18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(cab64bd0,c800f900,1388,3b1,0,...) at _cv_timedwait_sig+0x270 svc_run_internal(f284bd14,c0a2e418,c800f900,f284bd28,c0f88190,...) at svc_run_internal+0x356 svc_thread_start(c800f900,f284bd28,c0f88190,3db,c804284c,...) at svc_thread_start+0x10 fork_exit(c0c4b4e0,c800f900,f284bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x2e, eip = 0xc, esp = 0x33, ebp = 0x1 --- Tracing command nfsd pid 1853 tid 100088 td 0xca6cd2e0 sched_switch(ca6cd2e0,0,104,1a3,b4330101,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca6cd2e0,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f2723498,c0a4b8da,c800f900,0,ca6cd2e0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(cae33210,0,f27234d0,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(cae33210,c800f900,1388,3b1,f2723518,...) at _cv_timedwait_sig+0x270 svc_run_internal(ca6cd490,14,c0fbcc3a,c0f7fa5b,f2723810,...) at svc_run_internal+0x356 svc_run(c800f900,0,c0f7fa36,19f,4,...) at svc_run+0x7b nfsrvd_nfsd(ca6cd2e0,f2723810,c,f2723730,c0aa73ff,...) at nfsrvd_nfsd+0x1c3 nfssvc_nfsd(ca6cd2e0,f2723cec,bfbfe870,ca6cd2e0,f2723d1c,...) at nfssvc_nfsd+0x177 sys_nfssvc(ca6cd2e0,f2723cec,c0fdf70a,c0f967a3,206,...) at sys_nfssvc+0xeb syscall(f2723d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (155, FreeBSD ELF32, sys_nfssvc), eip = 0x280e890b, esp = 0xbfbfe82c, ebp = 0xbfbfea98 --- Tracing command nfsd pid 1847 tid 100098 td 0xcab07000 sched_switch(cab07000,0,104,1a3,2b37c446,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cab07000,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b8da,ca6b4510,0,c0f8beab,cab07000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca6b4524,0,f275ba58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca6b4524,ca6b4510,c0f95bd0,6a4,cac56738,...) at _cv_wait_sig+0x260 seltdwait(cac56738,58,c7dc4e80,cab07000,cac73658,...) at seltdwait+0xa2 kern_select(cab07000,7,bfbfecb4,0,0,0,20,caad2588,1) at kern_select+0x655 sys_select(cab07000,f275bcec,c,c,202,...) at sys_select+0x66 syscall(f275bd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281922b3, esp = 0xbfbfea9c, ebp = 0xbfbfede8 --- Tracing command mountd pid 1837 tid 100115 td 0xcab058a0 sched_switch(cab058a0,0,104,1a3,2393d572,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cab058a0,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b8da,ca690e50,0,c0f8beab,cab058a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca690e64,0,f27c5a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca690e64,ca690e50,c0f95bd0,6a4,ca7b5578,...) at _cv_wait_sig+0x260 seltdwait(ca7b5578,58,c7dc4e80,cab058a0,478,...) at seltdwait+0xa2 kern_select(cab058a0,a,bfbfed04,0,0,0,20,cac3a84c,bfbfed04) at kern_select+0x655 sys_select(cab058a0,f27c5cec,c0fdf70a,c0f7d817,206,...) at sys_select+0x66 syscall(f27c5d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281aa2b3, esp = 0xbfbfec9c, ebp = 0xbfbfede8 --- Tracing command rpcbind pid 1738 tid 100117 td 0xcab052e0 sched_switch(cab052e0,0,104,1a3,fb297ea5,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cab052e0,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f27cba84,c0a4b8da,ca768550,0,cab052e0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(ca768564,0,f27cbabc,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(ca768564,ca768550,7531,6a4,f27cbb74,...) at _cv_timedwait_sig+0x270 seltdwait(f27cbc44,f27cbc4c,5a1,cab052e0,cab052e0,...) at seltdwait+0x8a sys_poll(cab052e0,f27cbcec,c0fdf70a,c0f964d4,212,...) at sys_poll+0x321 syscall(f27cbd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x2814a14b, esp = 0xbfbfcbac, ebp = 0xbfbfed78 --- Tracing command syslogd pid 1710 tid 100103 td 0xcaae55c0 sched_switch(caae55c0,0,104,1a3,b762ca16,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(caae55c0,0,c0f93480,268,0,...) at sleepq_switch+0x162 sleepq_wait(e1915fbc,5c,c0fc257d,0,0,...) at sleepq_wait+0x63 _sleep(e1915fbc,ca816bd8,25c,c0fc257d,0,...) at _sleep+0x39c getdirtybuf(c12a7860,4,c0fc171b,2fcd,f276cb6c,...) at getdirtybuf+0x258 flush_deplist(f276cb70,0,c0fc171b,2fa0,1,...) at flush_deplist+0x8c softdep_sync_metadata(cad3915c,c1274378,0,c7d75ac0,f276cba4,...) at softdep_sync_metadata+0x1bf ffs_syncvnode(cad3915c,1,c0afb38f,c10dd620,cad39254,...) at ffs_syncvnode+0x5c ffs_fsync(f276cc48,c0fe63e3,f276ccec,f276cc48,0,...) at ffs_fsync+0x27 VOP_FSYNC_APV(c10dd620,f276cc48,c0f9eb27,e04,0,...) at VOP_FSYNC_APV+0xc5 sys_fsync(caae55c0,f276ccec,c0fdf70a,c0f972e0,202,...) at sys_fsync+0x1df syscall(f276cd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (95, FreeBSD ELF32, sys_fsync), eip = 0x281a6207, esp = 0xbfbfe29c, ebp = 0xbfbfede8 --- Tracing command devd pid 1503 tid 100116 td 0xcab055c0 sched_switch(cab055c0,0,104,1a3,229cd1be,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cab055c0,0,c0f93480,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b8da,ca6b45d0,0,c0f8beab,cab055c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca6b45e4,0,f27c8a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca6b45e4,ca6b45d0,c0f95bd0,6a4,ca823578,...) at _cv_wait_sig+0x260 seltdwait(ca823578,58,c7dc4e80,cab055c0,c1274330,...) at seltdwait+0xa2 kern_select(cab055c0,5,bfbfe960,0,0,0,20,cac3a588,32) at kern_select+0x655 sys_select(cab055c0,f27c8cec,c0fdf70a,c0dee437,206,...) at sys_select+0x66 syscall(f27c8d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x809152b, esp = 0xbfbfe92c, ebp = 0xbfbfee08 --- Tracing command softdepflush pid 19 tid 100077 td 0xca6cdb80 sched_switch(ca6cdb80,0,104,1a3,8c63390c,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,54,...) at mi_switch+0x1ff sleepq_switch(ca6cdb80,0,c0f93480,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c12a78a0,54,c0fc3c5b,0,0,...) at sleepq_timedwait+0x6b _sleep(c12a78a0,c12a7860,54,c0fc3c5b,3e8,...) at _sleep+0x36a softdep_flush(0,ee320d28,c0f88190,3db,ca6c0000,...) at softdep_flush+0x266 fork_exit(c0c9af90,0,ee320d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee320d60, ebp = 0 --- Tracing command vnlru pid 18 tid 100076 td 0xc805e8a0 sched_switch(c805e8a0,0,104,1a3,4be860a3,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,60,...) at mi_switch+0x1ff sleepq_switch(c805e8a0,0,c0f93480,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(ca6c02c4,60,c0f9e47d,0,0,...) at sleepq_timedwait+0x6b _sleep(ca6c02c4,c12a1574,260,c0f9e47d,3e8,...) at _sleep+0x36a vnlru_proc(0,ee31dd28,c0f88190,3db,ca6c02c4,...) at vnlru_proc+0xe7 fork_exit(c0afd220,0,ee31dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee31dd60, ebp = 0 --- Tracing command syncer pid 17 tid 100075 td 0xc805eb80 sched_switch(c805eb80,0,104,1a3,a1bd0b53,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c805eb80,0,c0f93480,28b,c805eb80,...) at sleepq_switch+0x162 sleepq_timedwait(c12a15b4,0,ee31ac78,1,0,...) at sleepq_timedwait+0x6b _cv_timedwait(c12a15b4,c12a15a0,3e8,78b,4e20,...) at _cv_timedwait+0x270 sched_sync(0,ee31ad28,c0f88190,3db,ca6c0588,...) at sched_sync+0x595 fork_exit(c0afc590,0,ee31ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee31ad60, ebp = 0 --- Tracing command bufdaemon pid 16 tid 100074 td 0xc8183000 sched_switch(c8183000,0,104,1a3,a3061510,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,54,...) at mi_switch+0x1ff sleepq_switch(c8183000,0,c0f93480,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c12a1248,54,c0f9b3b1,0,0,...) at sleepq_timedwait+0x6b _sleep(c12a1248,c12a1250,54,c0f9b3b1,3e8,...) at _sleep+0x36a buf_daemon(0,ee317d28,c0f88190,3db,ca6c084c,...) at buf_daemon+0x175 fork_exit(c0ae0df0,0,ee317d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee317d60, ebp = 0 --- Tracing command pagezero pid 9 tid 100073 td 0xc81832e0 sched_switch(c81832e0,0,104,1a3,e627485c,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c81832e0,0,c0f93480,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c12a975c,0,c0fcac7e,0,0,...) at sleepq_timedwait+0x6b _sleep(c12a975c,c12a8200,0,c0fcac7e,493e0,...) at _sleep+0x36a vm_pagezero(0,ee314d28,c0f88190,3db,ca6c0b10,...) at vm_pagezero+0xdc fork_exit(c0ce2320,0,ee314d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee314d60, ebp = 0 --- Tracing command vmdaemon pid 8 tid 100072 td 0xc81835c0 sched_switch(c81835c0,0,104,1a3,4606bc42,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,74,...) at mi_switch+0x1ff sleepq_switch(c81835c0,0,c0f93480,268,0,...) at sleepq_switch+0x162 sleepq_wait(c12a9348,74,c0f9b3b1,0,0,...) at sleepq_wait+0x63 _sleep(c12a9348,c12a934c,74,c0f9b3b1,0,...) at _sleep+0x39c vm_daemon(0,ee311d28,c0f88190,3db,ca6c1000,...) at vm_daemon+0x59 fork_exit(c0cdbc30,0,ee311d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee311d60, ebp = 0 --- Tracing command pagedaemon pid 7 tid 100071 td 0xc81838a0 sched_switch(c81838a0,0,104,1a3,a1bce8e2,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,54,...) at mi_switch+0x1ff sleepq_switch(c81838a0,0,c0f93480,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c12a9310,54,c0f9b3b1,0,0,...) at sleepq_timedwait+0x6b _sleep(c12a9310,c12a8200,54,c0f9b3b1,1388,...) at _sleep+0x36a vm_pageout(0,ee30ed28,c0f88190,3db,c7dc82c4,...) at vm_pageout+0x2c7 fork_exit(c0cdccf0,0,ee30ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee30ed60, ebp = 0 --- Tracing command xpt_thrd pid 6 tid 100070 td 0xc8183b80 sched_switch(c8183b80,0,104,1a3,bca503fb,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c8183b80,0,c0f93480,268,0,...) at sleepq_switch+0x162 sleepq_wait(c10efbd4,5c,c0de8c47,0,0,...) at sleepq_wait+0x63 _sleep(c10efbd4,c10efc18,5c,c0de8c47,0,...) at _sleep+0x39c xpt_scanner_thread(0,ee18bd28,c0f88190,3db,c7dc8588,...) at xpt_scanner_thread+0x47 fork_exit(c049d360,0,ee18bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee18bd60, ebp = 0 --- Tracing command sctp_iterator pid 5 tid 100069 td 0xc8184000 sched_switch(c8184000,0,104,1a3,7d2d8d2c,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8184000,0,c0f93480,268,0,...) at sleepq_switch+0x162 sleepq_wait(c12a30d8,0,c0fb1b1f,0,0,...) at sleepq_wait+0x63 _sleep(c12a30d8,c12a30a0,0,c0fb1b1f,0,...) at _sleep+0x39c sctp_iterator_thread(0,ee188d28,c0f88190,3db,c7dc884c,...) at sctp_iterator_thread+0x5c fork_exit(c0b8ea60,0,ee188d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee188d60, ebp = 0 --- Tracing command ctl_thrd pid 4 tid 100067 td 0xc81845c0 sched_switch(c81845c0,0,104,1a3,aa56b0d9,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c81845c0,0,c0f93480,268,0,...) at sleepq_switch+0x162 sleepq_wait(c819c000,5c,c0df3652,0,0,...) at sleepq_wait+0x63 _sleep(c819c000,c819c000,5c,c0df3652,0,...) at _sleep+0x39c ctl_work_thread(c819c000,ee182d28,c0f88190,3db,c7dc8b10,...) at ctl_work_thread+0x22f6 fork_exit(c04c6760,c819c000,ee182d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee182d60, ebp = 0 --- Tracing command fdc0 pid 3 tid 100064 td 0xc8185000 sched_switch(c8185000,0,104,1a3,37c278c3,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c8185000,0,c0f93480,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c7ffa03c,5c,c0f804d2,0,0,...) at sleepq_timedwait+0x6b _sleep(c7ffa03c,c7ffa0f0,5c,c0f804d2,3e8,...) at _sleep+0x36a fdc_thread(c7ffa000,ee179d28,c0f88190,3db,c8042000,...) at fdc_thread+0x27d fork_exit(c0d61570,c7ffa000,ee179d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee179d60, ebp = 0 --- Tracing command fw0_probe pid 2 tid 100062 td 0xc80515c0 sched_switch(c80515c0,0,104,1a3,bca53b44,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(c80515c0,0,c0f93480,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c808f000,6c,c0f804d2,100,0,...) at sleepq_wait_sig+0x17 _sleep(c808f000,c8093488,16c,c0f804d2,0,...) at _sleep+0x385 fw_bus_probe_thread(c808f000,ee169d28,c0f88190,3db,c80422c4,...) at fw_bus_probe_thread+0xa08 fork_exit(c06f0a50,c808f000,ee169d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee169d60, ebp = 0 --- Tracing command usb pid 15 tid 100058 td 0xc805c2e0 sched_switch(c805c2e0,0,104,1a3,aa518474,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c805c2e0,0,c0f93480,268,c805c2e0,...) at sleepq_switch+0x162 sleepq_wait(c8063d34,0,ee152cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8063d34,c8063e4c,c0f66b91,74,c8063d3c,...) at _cv_wait+0x260 usb_process(c8063d2c,ee152d28,c0f88190,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fda50,c8063d2c,ee152d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee152d60, ebp = 0 --- Tracing command usb pid 15 tid 100057 td 0xc805c5c0 sched_switch(c805c5c0,0,104,1a3,617d5469,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c805c5c0,0,c0f93480,268,c805c5c0,...) at sleepq_switch+0x162 sleepq_wait(c8063d04,0,ee14fcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8063d04,c8063e4c,c0f66b91,74,c8063d0c,...) at _cv_wait+0x260 usb_process(c8063cfc,ee14fd28,c0f88190,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fda50,c8063cfc,ee14fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee14fd60, ebp = 0 --- Tracing command usb pid 15 tid 100056 td 0xc805c8a0 sched_switch(c805c8a0,0,104,1a3,a9eb1646,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c805c8a0,0,c0f93480,268,c805c8a0,...) at sleepq_switch+0x162 sleepq_wait(c8063cd4,0,ee14ccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8063cd4,c8063e4c,c0f66b91,74,c8063cdc,...) at _cv_wait+0x260 usb_process(c8063ccc,ee14cd28,c0f88190,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fda50,c8063ccc,ee14cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee14cd60, ebp = 0 --- Tracing command usb pid 15 tid 100055 td 0xc805cb80 sched_switch(c805cb80,0,104,1a3,a9eb020a,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c805cb80,0,c0f93480,268,c805cb80,...) at sleepq_switch+0x162 sleepq_wait(c8063ca4,0,ee149cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8063ca4,c8063e4c,c0f66b91,74,c8063cac,...) at _cv_wait+0x260 usb_process(c8063c9c,ee149d28,c0f88190,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fda50,c8063c9c,ee149d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee149d60, ebp = 0 --- Tracing command usb pid 15 tid 100053 td 0xc805e2e0 sched_switch(c805e2e0,0,104,1a3,a9eaec7a,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c805e2e0,0,c0f93480,268,c805e2e0,...) at sleepq_switch+0x162 sleepq_wait(c8057b5c,0,ee140cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8057b5c,c8057c74,c0f66b91,74,c8057b64,...) at _cv_wait+0x260 usb_process(c8057b54,ee140d28,c0f88190,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fda50,c8057b54,ee140d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee140d60, ebp = 0 --- Tracing command usb pid 15 tid 100052 td 0xc805e5c0 sched_switch(c805e5c0,0,104,1a3,4be7da86,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c805e5c0,0,c0f93480,268,c805e5c0,...) at sleepq_switch+0x162 sleepq_wait(c8057b2c,0,ee13dcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8057b2c,c8057c74,c0f66b91,74,c8057b34,...) at _cv_wait+0x260 usb_process(c8057b24,ee13dd28,c0f88190,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fda50,c8057b24,ee13dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee13dd60, ebp = 0 --- Tracing command usb pid 15 tid 100051 td 0xc804e000 sched_switch(c804e000,0,104,1a3,a9869ae0,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804e000,0,c0f93480,268,c804e000,...) at sleepq_switch+0x162 sleepq_wait(c8057afc,0,ee13acac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8057afc,c8057c74,c0f66b91,74,c8057b04,...) at _cv_wait+0x260 usb_process(c8057af4,ee13ad28,c0f88190,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fda50,c8057af4,ee13ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee13ad60, ebp = 0 --- Tracing command usb pid 15 tid 100050 td 0xc804e2e0 sched_switch(c804e2e0,0,104,1a3,a986896b,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804e2e0,0,c0f93480,268,c804e2e0,...) at sleepq_switch+0x162 sleepq_wait(c8057acc,0,ee137cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8057acc,c8057c74,c0f66b91,74,c8057ad4,...) at _cv_wait+0x260 usb_process(c8057ac4,ee137d28,c0f88190,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fda50,c8057ac4,ee137d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee137d60, ebp = 0 --- Tracing command usb pid 15 tid 100049 td 0xc804e5c0 sched_switch(c804e5c0,0,104,1a3,a98672e6,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804e5c0,0,c0f93480,268,c804e5c0,...) at sleepq_switch+0x162 sleepq_wait(c8052b5c,0,ee133cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8052b5c,c8052c74,c0f66b91,74,c8052b64,...) at _cv_wait+0x260 usb_process(c8052b54,ee133d28,c0f88190,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fda50,c8052b54,ee133d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee133d60, ebp = 0 --- Tracing command usb pid 15 tid 100048 td 0xc804e8a0 sched_switch(c804e8a0,0,104,1a3,4be7a691,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804e8a0,0,c0f93480,268,c804e8a0,...) at sleepq_switch+0x162 sleepq_wait(c8052b2c,0,ee130cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8052b2c,c8052c74,c0f66b91,74,c8052b34,...) at _cv_wait+0x260 usb_process(c8052b24,ee130d28,c0f88190,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fda50,c8052b24,ee130d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee130d60, ebp = 0 --- Tracing command usb pid 15 tid 100047 td 0xc804eb80 sched_switch(c804eb80,0,104,1a3,a3c188b9,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804eb80,0,c0f93480,268,c804eb80,...) at sleepq_switch+0x162 sleepq_wait(c8052afc,0,ee12dcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8052afc,c8052c74,c0f66b91,74,c8052b04,...) at _cv_wait+0x260 usb_process(c8052af4,ee12dd28,c0f88190,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fda50,c8052af4,ee12dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee12dd60, ebp = 0 --- Tracing command usb pid 15 tid 100046 td 0xc804f000 sched_switch(c804f000,0,104,1a3,a3c1770d,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804f000,0,c0f93480,268,c804f000,...) at sleepq_switch+0x162 sleepq_wait(c8052acc,0,ee12acac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8052acc,c8052c74,c0f66b91,74,c8052ad4,...) at _cv_wait+0x260 usb_process(c8052ac4,ee12ad28,c0f88190,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fda50,c8052ac4,ee12ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee12ad60, ebp = 0 --- Tracing command usb pid 15 tid 100045 td 0xc804f2e0 sched_switch(c804f2e0,0,104,1a3,a3c16330,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804f2e0,0,c0f93480,268,c804f2e0,...) at sleepq_switch+0x162 sleepq_wait(c804ab5c,0,ee126cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c804ab5c,c804ac74,c0f66b91,74,c804ab64,...) at _cv_wait+0x260 usb_process(c804ab54,ee126d28,c0f88190,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fda50,c804ab54,ee126d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee126d60, ebp = 0 --- Tracing command usb pid 15 tid 100044 td 0xc804f5c0 sched_switch(c804f5c0,0,104,1a3,4be7b4d1,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804f5c0,0,c0f93480,268,c804f5c0,...) at sleepq_switch+0x162 sleepq_wait(c804ab2c,0,ee123cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c804ab2c,c804ac74,c0f66b91,74,c804ab34,...) at _cv_wait+0x260 usb_process(c804ab24,ee123d28,c0f88190,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fda50,c804ab24,ee123d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee123d60, ebp = 0 --- Tracing command usb pid 15 tid 100043 td 0xc804f8a0 sched_switch(c804f8a0,0,104,1a3,a35ddfb2,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804f8a0,0,c0f93480,268,c804f8a0,...) at sleepq_switch+0x162 sleepq_wait(c804aafc,0,ee120cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c804aafc,c804ac74,c0f66b91,74,c804ab04,...) at _cv_wait+0x260 usb_process(c804aaf4,ee120d28,c0f88190,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fda50,c804aaf4,ee120d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee120d60, ebp = 0 --- Tracing command usb pid 15 tid 100042 td 0xc804fb80 sched_switch(c804fb80,0,104,1a3,a35dcbfa,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804fb80,0,c0f93480,268,c804fb80,...) at sleepq_switch+0x162 sleepq_wait(c804aacc,0,ee11dcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c804aacc,c804ac74,c0f66b91,74,c804aad4,...) at _cv_wait+0x260 usb_process(c804aac4,ee11dd28,c0f88190,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fda50,c804aac4,ee11dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee11dd60, ebp = 0 --- Tracing command usb pid 15 tid 100040 td 0xc803e000 sched_switch(c803e000,0,104,1a3,a35db68f,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c803e000,0,c0f93480,268,c803e000,...) at sleepq_switch+0x162 sleepq_wait(c8044b5c,0,ee116cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8044b5c,c8044c74,c0f66b91,74,c8044b64,...) at _cv_wait+0x260 usb_process(c8044b54,ee116d28,c0f88190,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fda50,c8044b54,ee116d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee116d60, ebp = 0 --- Tracing command usb pid 15 tid 100039 td 0xc803e2e0 sched_switch(c803e2e0,0,104,1a3,4be7d047,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c803e2e0,0,c0f93480,268,c803e2e0,...) at sleepq_switch+0x162 sleepq_wait(c8044b2c,0,ee113cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8044b2c,c8044c74,c0f66b91,74,c8044b34,...) at _cv_wait+0x260 usb_process(c8044b24,ee113d28,c0f88190,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fda50,c8044b24,ee113d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee113d60, ebp = 0 --- Tracing command usb pid 15 tid 100038 td 0xc803e5c0 sched_switch(c803e5c0,0,104,1a3,a2f9ed92,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c803e5c0,0,c0f93480,268,c803e5c0,...) at sleepq_switch+0x162 sleepq_wait(c8044afc,0,ee110cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8044afc,c8044c74,c0f66b91,74,c8044b04,...) at _cv_wait+0x260 usb_process(c8044af4,ee110d28,c0f88190,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fda50,c8044af4,ee110d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee110d60, ebp = 0 --- Tracing command usb pid 15 tid 100037 td 0xc803e8a0 sched_switch(c803e8a0,0,104,1a3,a2f9d918,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c803e8a0,0,c0f93480,268,c803e8a0,...) at sleepq_switch+0x162 sleepq_wait(c8044acc,0,ee10dcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8044acc,c8044c74,c0f66b91,74,c8044ad4,...) at _cv_wait+0x260 usb_process(c8044ac4,ee10dd28,c0f88190,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fda50,c8044ac4,ee10dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee10dd60, ebp = 0 --- Tracing command usb pid 15 tid 100035 td 0xc8040000 sched_switch(c8040000,0,104,1a3,a2f9c025,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8040000,0,c0f93480,268,c8040000,...) at sleepq_switch+0x162 sleepq_wait(c8039b5c,0,ee106cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8039b5c,c8039c74,c0f66b91,74,c8039b64,...) at _cv_wait+0x260 usb_process(c8039b54,ee106d28,c0f88190,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fda50,c8039b54,ee106d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee106d60, ebp = 0 --- Tracing command usb pid 15 tid 100034 td 0xc80402e0 sched_switch(c80402e0,0,104,1a3,617cc0f5,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c80402e0,0,c0f93480,268,c80402e0,...) at sleepq_switch+0x162 sleepq_wait(c8039b2c,0,ee103cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8039b2c,c8039c74,c0f66b91,74,c8039b34,...) at _cv_wait+0x260 usb_process(c8039b24,ee103d28,c0f88190,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fda50,c8039b24,ee103d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee103d60, ebp = 0 --- Tracing command usb pid 15 tid 100033 td 0xc80405c0 sched_switch(c80405c0,0,104,1a3,a2954ff3,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c80405c0,0,c0f93480,268,c80405c0,...) at sleepq_switch+0x162 sleepq_wait(c8039afc,0,ee100cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8039afc,c8039c74,c0f66b91,74,c8039b04,...) at _cv_wait+0x260 usb_process(c8039af4,ee100d28,c0f88190,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fda50,c8039af4,ee100d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee100d60, ebp = 0 --- Tracing command usb pid 15 tid 100032 td 0xc80408a0 sched_switch(c80408a0,0,104,1a3,a2953a08,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c80408a0,0,c0f93480,268,c80408a0,...) at sleepq_switch+0x162 sleepq_wait(c8039acc,0,ee0fdcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8039acc,c8039c74,c0f66b91,74,c8039ad4,...) at _cv_wait+0x260 usb_process(c8039ac4,ee0fdd28,c0f88190,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fda50,c8039ac4,ee0fdd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee0fdd60, ebp = 0 --- Tracing command yarrow pid 14 tid 100017 td 0xc7eda5c0 sched_switch(c7eda5c0,0,104,1a3,b74d6f34,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7eda5c0,0,c0f93480,28b,2,...) at sleepq_switch+0x162 sleepq_timedwait(c112a1e4,0,c0f804d2,2,0,...) at sleepq_timedwait+0x6b _sleep(c112a1e4,0,0,c0f804d2,64) at _sleep+0x36a pause(c0f804d2,64,c0f42950,111,0,...) at pause+0xa0 random_kthread(0,c7b51d28,c0f88190,3db,c7dc7000,...) at random_kthread+0x1ef fork_exit(c0805cd0,0,c7b51d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b51d60, ebp = 0 --- Tracing command geom pid 13 tid 100015 td 0xc7edab80 sched_switch(c7edab80,0,104,1a3,b763c9d1,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c7edab80,0,c0f93480,268,0,...) at sleepq_switch+0x162 sleepq_wait(c1127de8,5c,c0f804d2,0,0,...) at sleepq_wait+0x63 _sleep(c1127de8,c1127d48,25c,c0f804d2,0,...) at _sleep+0x39c g_io_schedule_down(c7edab80,0,c0f82021,6c,c7b4bd14,...) at g_io_schedule_down+0x5b g_down_procbody(0,c7b4bd28,c0f88190,3db,c7dc72c4,...) at g_down_procbody+0x9d fork_exit(c09ef3f0,0,c7b4bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b4bd60, ebp = 0 --- Tracing command geom pid 13 tid 100014 td 0xc7edb000 cpustop_handler(0,13,c7b48a30,c0dc0cc6,c7b489b0,...) at cpustop_handler+0x28 ipi_nmi_handler(c7b489b0,c0aa6157,c0f95956,c7b48a08,c7dc72c4,...) at ipi_nmi_handler+0x47 trap(c7b48a3c) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0db53e8, esp = 0xc7b48a7c, ebp = 0xc7b48a9c --- smp_tlb_shootdown(c7b48acc,c0db7ede,e3048000,e304c000,11ee,...) at smp_tlb_shootdown+0x98 smp_invlpg_range(e3048000,e304c000,11ee,c1baf474,c7d02000,...) at smp_invlpg_range+0x1c pmap_invalidate_range(c12d34c0,e3048000,e304c000) at pmap_invalidate_range+0x7e pmap_qremove(e3048000,4,c12a7860,4,c0f8beab,...) at pmap_qremove+0x54 allocbuf(e19679e0,0,0,c0f8fb42,4c6,...) at allocbuf+0x18d brelse(e19679e0,4af,246,0,c7edb0b0,...) at brelse+0x172 bufdone_finish(e19679e0,e19679e0,e19679e0,e1915f60,c7b48c40,...) at bufdone_finish+0x3a7 bufdone(e19679e0,0,c0fc45f0,78d,ca816bd8,...) at bufdone+0x6a ffs_backgroundwritedone(e19679e0,ca781d20,ca781d20,e19679e0,c7b48c80,...) at ffs_backgroundwritedone+0xcc bufdone(e19679e0,0,c0f82f97,74,ca6b9a40) at bufdone+0x53 g_vfs_done(ca6bc2f8,0,c0f9aa6a,c52,c7edb000,...) at g_vfs_done+0xf8 biodone(ca6bc2f8,c1127d68,25c,c0f804d2,0,...) at biodone+0xa5 g_io_schedule_up(c7edb000,0,c0f82021,5f,c7b48d14,...) at g_io_schedule_up+0xde g_up_procbody(0,c7b48d28,c0f88190,3db,c7dc72c4,...) at g_up_procbody+0x9d fork_exit(c09ef490,0,c7b48d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b48d60, ebp = 0 --- Tracing command geom pid 13 tid 100013 td 0xc7dca2e0 sched_switch(c7dca2e0,0,104,1a3,22dd950a,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c7dca2e0,0,c0f93480,268,0,...) at sleepq_switch+0x162 sleepq_wait(c1127ddc,5c,c0f804d2,0,0,...) at sleepq_wait+0x63 _sleep(c1127ddc,c1127d0c,25c,c0f804d2,0,...) at _sleep+0x39c g_run_events(c1138040,0,c0f82021,79,c7b45d14,...) at g_run_events+0x54b g_event_procbody(0,c7b45d28,c0f88190,3db,c7dc72c4,...) at g_event_procbody+0x90 fork_exit(c09ef530,0,c7b45d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b45d60, ebp = 0 --- Tracing command intr pid 12 tid 100066 td 0xc81848a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100065 td 0xc8184b80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100063 td 0xc80512e0 sched_switch(c80512e0,0,109,1a3,55a0d485,...) at sched_switch+0x3ec mi_switch(109,0,c0f884b3,554,c80512e0,...) at mi_switch+0x1ff ithread_loop(c7e12690,ee16cd28,c0f88190,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a31f30,c7e12690,ee16cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee16cd60, ebp = 0 --- Tracing command intr pid 12 tid 100060 td 0xc8051b80 sched_switch(c8051b80,0,109,1a3,92b13b87,...) at sched_switch+0x3ec mi_switch(109,0,c0f884b3,554,c8051b80,...) at mi_switch+0x1ff ithread_loop(c7e126a0,ee160d28,c0f88190,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a31f30,c7e126a0,ee160d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee160d60, ebp = 0 --- Tracing command intr pid 12 tid 100059 td 0xc805c000 sched_switch(c805c000,0,109,1a3,447657a6,...) at sched_switch+0x3ec mi_switch(109,0,c0f884b3,554,c805c000,...) at mi_switch+0x1ff ithread_loop(c7e126b0,ee158d28,c0f88190,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a31f30,c7e126b0,ee158d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee158d60, ebp = 0 --- Tracing command intr pid 12 tid 100054 td 0xc805e000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100041 td 0xc8051000 sched_switch(c8051000,0,109,1a3,c79a9a88,...) at sched_switch+0x3ec mi_switch(109,0,c0f884b3,554,c8051000,...) at mi_switch+0x1ff ithread_loop(c7e126d0,ee11ad28,c0f88190,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a31f30,c7e126d0,ee11ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee11ad60, ebp = 0 --- Tracing command intr pid 12 tid 100036 td 0xc803eb80 sched_switch(c803eb80,0,109,1a3,c22223ec,...) at sched_switch+0x3ec mi_switch(109,0,c0f884b3,554,c803eb80,...) at mi_switch+0x1ff ithread_loop(c7e126e0,ee10ad28,c0f88190,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a31f30,c7e126e0,ee10ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee10ad60, ebp = 0 --- Tracing command intr pid 12 tid 100031 td 0xc8040b80 sched_switch(c8040b80,0,109,1a3,384a4f4f,...) at sched_switch+0x3ec mi_switch(109,0,c0f884b3,554,c8040b80,...) at mi_switch+0x1ff ithread_loop(c7e126f0,ee0fad28,c0f88190,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a31f30,c7e126f0,ee0fad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee0fad60, ebp = 0 --- Tracing command intr pid 12 tid 100030 td 0xc7edb2e0 sched_switch(c7edb2e0,0,109,1a3,b76a8d7a,...) at sched_switch+0x3ec mi_switch(109,0,c0f884b3,554,c7edb2e0,...) at mi_switch+0x1ff ithread_loop(c7e12700,edfa4d28,c0f88190,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a31f30,c7e12700,edfa4d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xedfa4d60, ebp = 0 --- Tracing command intr pid 12 tid 100029 td 0xc7edb5c0 sched_switch(c7edb5c0,0,109,1a3,acc4712c,...) at sched_switch+0x3ec mi_switch(109,0,c0f884b3,554,c7edb5c0,...) at mi_switch+0x1ff ithread_loop(c7e12710,edfa0d28,c0f88190,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a31f30,c7e12710,edfa0d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xedfa0d60, ebp = 0 --- Tracing command intr pid 12 tid 100028 td 0xc7edb8a0 sched_switch(c7edb8a0,0,109,1a3,948c05e4,...) at sched_switch+0x3ec mi_switch(109,0,c0f884b3,554,c7edb8a0,...) at mi_switch+0x1ff ithread_loop(c7e12720,c7ba0d28,c0f88190,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a31f30,c7e12720,c7ba0d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7ba0d60, ebp = 0 --- Tracing command intr pid 12 tid 100027 td 0xc7edbb80 sched_switch(c7edbb80,0,109,1a3,53d4223a,...) at sched_switch+0x3ec mi_switch(109,0,c0f884b3,554,c7edbb80,...) at mi_switch+0x1ff ithread_loop(c7e12730,c7b6fd28,c0f88190,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a31f30,c7e12730,c7b6fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b6fd60, ebp = 0 --- Tracing command intr pid 12 tid 100026 td 0xc7fc7000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100024 td 0xc7fc75c0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100018 td 0xc7eda2e0 sched_switch(c7eda2e0,0,109,1a3,b76b2980,...) at sched_switch+0x3ec mi_switch(109,0,c0f884b3,554,c7eda2e0,...) at mi_switch+0x1ff ithread_loop(c7e12760,c7b54d28,c0f88190,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a31f30,c7e12760,c7b54d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b54d60, ebp = 0 --- Tracing command intr pid 12 tid 100012 td 0xc7dca5c0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100011 td 0xc7dca8a0 sched_switch(c7dca8a0,0,109,1a3,a1d7d952,...) at sched_switch+0x3ec mi_switch(109,0,c0f884b3,554,c7dca8a0,...) at mi_switch+0x1ff ithread_loop(c7e127b0,c7b3fd28,c0f88190,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a31f30,c7e127b0,c7b3fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b3fd60, ebp = 0 --- Tracing command intr pid 12 tid 100010 td 0xc7dcab80 sched_switch(c7dcab80,0,109,1a3,81a82c58,...) at sched_switch+0x3ec mi_switch(109,0,c0f884b3,554,c7dcab80,...) at mi_switch+0x1ff ithread_loop(c7e127c0,c7b3cd28,c0f88190,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a31f30,c7e127c0,c7b3cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b3cd60, ebp = 0 --- Tracing command intr pid 12 tid 100009 td 0xc7e13000 sched_switch(c7e13000,0,109,1a3,b74d45f7,...) at sched_switch+0x3ec mi_switch(109,0,c0f884b3,554,c7e13000,...) at mi_switch+0x1ff ithread_loop(c7e127d0,c7b39d28,c0f88190,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a31f30,c7e127d0,c7b39d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b39d60, ebp = 0 --- Tracing command intr pid 12 tid 100008 td 0xc7e132e0 cpustop_handler(1,13,c7b36b50,c0dc0cc6,f5,...) at cpustop_handler+0x28 ipi_nmi_handler(f5,0,1bc3e58,c0a4bac0,c7dc7588,...) at ipi_nmi_handler+0x47 trap(c7b36b5c) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0a4bab2, esp = 0xc7b36b9c, ebp = 0xc7b36bb8 --- _mtx_lock_sleep(c1129a90,c7e132e0,0,c0f8beab,94,...) at _mtx_lock_sleep+0xa2 _mtx_lock_flags(c1129a90,0,c0f8beab,94,c7b36c90,...) at _mtx_lock_flags+0xff lock_mtx(c1129a90,1,c0f8f17e,216,11,...) at lock_mtx+0x29 softclock(c112b420,c7b36cb8,c0a4b47d,c1138040,c7e0e338,...) at softclock+0x1ea intr_event_execute_handlers(c7dc7588,c7e0e300,c0f884b3,55e,c7e132e0,...) at intr_event_execute_handlers+0x125 ithread_loop(c7e127e0,c7b36d28,c0f88190,3db,c7dc7588,...) at ithread_loop+0xa9 fork_exit(c0a31f30,c7e127e0,c7b36d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b36d60, ebp = 0 --- Tracing command intr pid 12 tid 100007 td 0xc7e135c0 sched_switch(c7e135c0,0,109,1a3,b08357ad,...) at sched_switch+0x3ec mi_switch(109,0,c0f884b3,554,c7e135c0,...) at mi_switch+0x1ff ithread_loop(c7e127f0,c7b33d28,c0f88190,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a31f30,c7e127f0,c7b33d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b33d60, ebp = 0 --- Tracing command idle pid 11 tid 100006 td 0xc7dc9000 sched_switch(c7dc9000,0,108,19e,a1d7e837,...) at sched_switch+0x3ec mi_switch(108,0,c0f8fb42,3be,c7dc9000,...) at mi_switch+0x1ff sched_idletd(0,c7b2fd28,c0f88190,3db,c7dc784c,...) at sched_idletd+0x19f fork_exit(c0a82d50,0,c7b2fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b2fd60, ebp = 0 --- Tracing command idle pid 11 tid 100005 td 0xc7dc92e0 sched_switch(c7dc92e0,0,608,19e,777e31b6,...) at sched_switch+0x3ec mi_switch(608,0,c0f8e2bf,cd,0,...) at mi_switch+0x1ff critical_exit(0,ffffffff,c1138d40,2,c7b2cce8,...) at critical_exit+0xb5 cpu_idle(0,c7b2ccc4,c0f8fb42,3c0,c7dc92e0,...) at cpu_idle+0x97 sched_idletd(0,c7b2cd28,c0f88190,3db,c7dc784c,...) at sched_idletd+0x263 fork_exit(c0a82d50,0,c7b2cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b2cd60, ebp = 0 --- Tracing command idle pid 11 tid 100004 td 0xc7dc95c0 sched_switch(c7dc95c0,0,108,19e,b76a9e02,...) at sched_switch+0x3ec mi_switch(108,0,c0f8fb42,3be,c7dc95c0,...) at mi_switch+0x1ff sched_idletd(0,c7b29d28,c0f88190,3db,c7dc784c,...) at sched_idletd+0x19f fork_exit(c0a82d50,0,c7b29d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b29d60, ebp = 0 --- Tracing command idle pid 11 tid 100003 td 0xc7dc98a0 sched_switch(c7dc98a0,0,108,19e,4be873a9,...) at sched_switch+0x3ec mi_switch(108,0,c0f8fb42,3be,c7dc98a0,...) at mi_switch+0x1ff sched_idletd(0,c7b26d28,c0f88190,3db,c7dc784c,...) at sched_idletd+0x19f fork_exit(c0a82d50,0,c7b26d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b26d60, ebp = 0 --- Tracing command init pid 1 tid 100002 td 0xc7dc9b80 sched_switch(c7dc9b80,0,104,1a3,17f0e9d6,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(c7dc9b80,0,c0f93480,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93480,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7dc7b10,6c,c0f96397,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7dc7b10,c7dc7b98,16c,c0f96397,0,...) at _sleep+0x385 kern_wait(c7dc9b80,ffffffff,c7b22c60,0,0,...) at kern_wait+0x626 sys_wait4(c7dc9b80,c7b22cec,c0fdf70a,c0f961ce,286,...) at sys_wait4+0x3b syscall(c7b22d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x8060f97, esp = 0xbfbfe8bc, ebp = 0xbfbfe8d8 --- Tracing command audit pid 10 tid 100001 td 0xc7dca000 sched_switch(c7dca000,0,104,1a3,a29368b7,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7dca000,0,c0f93480,268,c7dca000,...) at sleepq_switch+0x162 sleepq_wait(c12a6fe0,0,c7b1fc8c,1,0,...) at sleepq_wait+0x63 _cv_wait(c12a6fe0,c12a6fc4,c0fbe6b7,194,0,...) at _cv_wait+0x260 audit_worker(0,c7b1fd28,c0f88190,3db,c7dc8000,...) at audit_worker+0x84 fork_exit(c0c5b830,0,c7b1fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b1fd60, ebp = 0 --- Tracing command kernel pid 0 tid 100078 td 0xca6cd8a0 sched_switch(ca6cd8a0,0,104,1a3,6616bf85,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca6cd8a0,0,c0f93480,268,0,...) at sleepq_switch+0x162 sleepq_wait(c8036380,0,c0f804d2,0,0,...) at sleepq_wait+0x63 _sleep(c8036380,c8036398,0,c0f804d2,0,...) at _sleep+0x39c taskqueue_thread_loop(c1308860,ee323d28,c0f88190,3db,c1127ec0,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a9fbc0,c1308860,ee323d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee323d60, ebp = 0 --- Tracing command kernel pid 0 tid 100068 td 0xc81842e0 sched_switch(c81842e0,0,104,1a3,617fbcaa,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c81842e0,0,c0f93480,28b,2,...) at sleepq_switch+0x162 sleepq_timedwait(c112a1e4,0,c0f804d2,2,0,...) at sleepq_timedwait+0x6b _sleep(c112a1e4,0,0,c0f804d2,bb8) at _sleep+0x36a pause(c0f804d2,bb8,124,122,c1127f48,...) at pause+0xa0 deadlkres(0,ee185d28,c0f88190,3db,c1127ec0,...) at deadlkres+0x324 fork_exit(c0a11700,0,ee185d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee185d60, ebp = 0 --- Tracing command kernel pid 0 tid 100061 td 0xc80518a0 sched_switch(c80518a0,0,104,1a3,b74e505a,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c80518a0,0,c0f93480,268,c80518a0,...) at sleepq_switch+0x162 sleepq_wait(c80366c0,0,c0f8e7e1,c0f804d2,0,...) at sleepq_wait+0x63 msleep_spin(c80366c0,c80366d8,c0f804d2,0,c0f8beab,...) at msleep_spin+0x259 taskqueue_thread_loop(c809349c,ee166d28,c0f88190,3db,c1127ec0,...) at taskqueue_thread_loop+0x8f fork_exit(c0a9fbc0,c809349c,ee166d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee166d60, ebp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xc7fc72e0 sched_switch(c7fc72e0,0,104,1a3,44927581,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7fc72e0,0,c0f93480,268,0,...) at sleepq_switch+0x162 sleepq_wait(c7f9ee80,0,c0f804d2,0,0,...) at sleepq_wait+0x63 _sleep(c7f9ee80,c7f9ee98,0,c0f804d2,0,...) at _sleep+0x39c taskqueue_thread_loop(c1146dac,c7b69d28,c0f88190,3db,c1127ec0,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a9fbc0,c1146dac,c7b69d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b69d60, ebp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xc7fc78a0 sched_switch(c7fc78a0,0,104,1a3,bca96356,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7fc78a0,0,c0f93480,268,c7fc78a0,...) at sleepq_switch+0x162 sleepq_wait(c7f9f040,0,c0f8e7e1,c0f804d2,0,...) at sleepq_wait+0x63 msleep_spin(c7f9f040,c7f9f058,c0f804d2,0,c0f8beab,...) at msleep_spin+0x259 taskqueue_thread_loop(c10f2e38,c7b63d28,c0f88190,3db,c1127ec0,...) at taskqueue_thread_loop+0x8f fork_exit(c0a9fbc0,c10f2e38,c7b63d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b63d60, ebp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xc7fc7b80 sched_switch(c7fc7b80,0,104,1a3,bca95512,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7fc7b80,0,c0f93480,268,c7fc7b80,...) at sleepq_switch+0x162 sleepq_wait(c7f9f040,0,c0f8e7e1,c0f804d2,0,...) at sleepq_wait+0x63 msleep_spin(c7f9f040,c7f9f058,c0f804d2,0,c0f8beab,...) at msleep_spin+0x259 taskqueue_thread_loop(c10f2e38,c7b60d28,c0f88190,3db,c1127ec0,...) at taskqueue_thread_loop+0x8f fork_exit(c0a9fbc0,c10f2e38,c7b60d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b60d60, ebp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xc7e138a0 sched_switch(c7e138a0,0,104,1a3,bca9448e,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7e138a0,0,c0f93480,268,c7e138a0,...) at sleepq_switch+0x162 sleepq_wait(c7f9f040,0,c0f8e7e1,c0f804d2,0,...) at sleepq_wait+0x63 msleep_spin(c7f9f040,c7f9f058,c0f804d2,0,c0f8beab,...) at msleep_spin+0x259 taskqueue_thread_loop(c10f2e38,c7b5dd28,c0f88190,3db,c1127ec0,...) at taskqueue_thread_loop+0x8f fork_exit(c0a9fbc0,c10f2e38,c7b5dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b5dd60, ebp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xc7e13b80 sched_switch(c7e13b80,0,104,1a3,bca571f9,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7e13b80,0,c0f93480,268,0,...) at sleepq_switch+0x162 sleepq_wait(c7f9f080,0,c0f804d2,0,0,...) at sleepq_wait+0x63 _sleep(c7f9f080,c7f9f098,0,c0f804d2,0,...) at _sleep+0x39c taskqueue_thread_loop(c1128838,c7b5ad28,c0f88190,3db,c1127ec0,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a9fbc0,c1128838,c7b5ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b5ad60, ebp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xc7eda000 sched_switch(c7eda000,0,104,1a3,bca56100,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7eda000,0,c0f93480,268,0,...) at sleepq_switch+0x162 sleepq_wait(c7f9f0c0,0,c0f804d2,0,0,...) at sleepq_wait+0x63 _sleep(c7f9f0c0,c7f9f0d8,0,c0f804d2,0,...) at _sleep+0x39c taskqueue_thread_loop(c12a75c4,c7b57d28,c0f88190,3db,c1127ec0,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a9fbc0,c12a75c4,c7b57d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b57d60, ebp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xc7eda8a0 sched_switch(c7eda8a0,0,104,1a3,809d252b,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7eda8a0,0,c0f93480,268,0,...) at sleepq_switch+0x162 sleepq_wait(c7dade80,0,c0f804d2,0,0,...) at sleepq_wait+0x63 _sleep(c7dade80,c7dade98,0,c0f804d2,0,...) at _sleep+0x39c taskqueue_thread_loop(c11457c0,c7b4ed28,c0f88190,3db,c1127ec0,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a9fbc0,c11457c0,c7b4ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b4ed60, ebp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xc1128190 sched_switch(c1128190,0,104,1a3,b1355e19,...) at sched_switch+0x3ec mi_switch(104,0,c0f93480,1f3,54,...) at mi_switch+0x1ff sleepq_switch(c1128190,0,c0f93480,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c1127ec0,54,c0f90454,0,0,...) at sleepq_timedwait+0x6b _sleep(c1127ec0,0,54,c0f90454,2710,...) at _sleep+0x36a scheduler(0,181ec00,181ec00,181e000,1825000,...) at scheduler+0x240 mi_startup() at mi_startup+0xac begin() at begin+0x2c db:0:allt> call doadump Physical memory: 3435 MB Dumping 189 MB: 174 158 142 126 110 94 78 62 46 30 14 Dump complete = 0 db:0:doadump> reset cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 2 (kgdb) proc 2371 [Switching to thread 153 (Thread 100179)]#0 cpustop_handler () at ../../../i386/i386/mp_machdep.c:1489 1489 CPU_SET_ATOMIC(cpu, &stopped_cpus); (kgdb) bt #0 cpustop_handler () at ../../../i386/i386/mp_machdep.c:1489 #1 0xc0db5f77 in ipi_nmi_handler () at ../../../i386/i386/mp_machdep.c:1471 #2 0xc0dc0cc6 in trap (frame=0xf28d5434) at ../../../i386/i386/trap.c:223 #3 0xc0da9b8c in calltrap () at ../../../i386/i386/exception.s:168 #4 0xc0aa6146 in isitmychild (parent=0xc7d72450, child=0xc7d721e0) at ../../../kern/subr_witness.c:1956 #5 0xc0aa827c in witness_checkorder (lock=0xcb2540c4, flags=0x9, file=0xc0fc45f0 "../../../ufs/ffs/ffs_vfsops.c", line=0x5cf, interlock=0x0) at ../../../kern/subr_witness.c:2779 #6 0xc0a4bc5c in _mtx_lock_flags (m=0xcb2540c4, opts=0x0, file=0xc0fc45f0 "../../../ufs/ffs/ffs_vfsops.c", line=0x5cf) at ../../../kern/kern_mutex.c:203 #7 0xc0c9fca6 in ffs_sync (mp=0xca76b5a8, waitfor=0x1) at ../../../ufs/ffs/ffs_vfsops.c:1487 #8 0xc0b09392 in vfs_write_suspend (mp=0xca76b5a8) at ../../../kern/vfs_vnops.c:1203 #9 0xc0c83e0f in ffs_snapshot (mp=0xca76b5a8, snapfile=0xc809a8a0 "/mnt/.snap/suj28") at ../../../ufs/ffs/ffs_snapshot.c:417 #10 0xc0ca23d7 in ffs_mount (mp=0xca76b5a8) at ../../../ufs/ffs/ffs_vfsops.c:481 #11 0xc0af188b in vfs_donmount (td=0xcb2492e0, fsflags=0x11100, fsoptions=0xca88d300) at ../../../kern/vfs_mount.c:957 #12 0xc0af1f2f in sys_nmount (td=0xcb2492e0, uap=0xf28d5cec) at ../../../kern/vfs_mount.c:420 #13 0xc0dc08c7 in syscall (frame=0xf28d5d28) at subr_syscall.c:135 #14 0xc0da9bf1 in Xint0x80_syscall () at ../../../i386/i386/exception.s:266 #15 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) f 7 #7 0xc0c9fca6 in ffs_sync (mp=0xca76b5a8, waitfor=0x1) at ../../../ufs/ffs/ffs_vfsops.c:1487 1487 VI_LOCK(vp); (kgdb) l 1482 * Depend on the mntvnode_slock to keep things stable enough 1483 * for a quick test. Since there might be hundreds of 1484 * thousands of vnodes, we cannot afford even a subroutine 1485 * call unless there's a good chance that we have work to do. 1486 */ 1487 VI_LOCK(vp); 1488 if (vp->v_iflag & VI_DOOMED) { 1489 VI_UNLOCK(vp); 1490 continue; 1491 } (kgdb) info loc mvp = (struct vnode *) 0xcac42200 vp = (struct vnode *) 0xcb254000 devvp = Variable "devvp" is not available. (kgdb) p *vp $1 = {v_type = VREG, v_tag = 0xc0f840cc "ufs", v_op = 0xc10dd620, v_data = 0xcb4843a0, v_mount = 0xca76b5a8, v_nmntvnodes = {tqe_next = 0xcac42200, tqe_prev = 0xcb254170}, v_un = {vu_mount = 0x0, vu_socket = 0x0, vu_cdev = 0x0, vu_fifoinfo = 0x0}, v_hashlist = {le_next = 0x0, le_prev = 0xca4637f8}, v_hash = 0x4, v_cache_src = {lh_first = 0x0}, v_cache_dst = {tqh_first = 0xcad451b0, tqh_last = 0xcad451c0}, v_cache_dd = 0x0, v_cstart = 0x0, v_lasta = 0x0, v_lastw = 0x0, v_clen = 0x0, v_lock = {lock_object = {lo_name = 0xc0f840cc "ufs", lo_flags = 0x57b0000, lo_data = 0x0, lo_witness = 0xc7d79610}, lk_lock = 0x1, lk_exslpfail = 0x0, lk_timo = 0x33, lk_pri = 0x60, lk_stack = {depth = 0x12, pcs = {0xc0a432e4, 0xc0ca5bf1, 0xc0de3c25, 0xc0b09798, 0xc0afc24b, 0xc0aebc5d, 0xc0c9ee19, 0xc0c9f3be, 0xc0cae064, 0xc0cae10a, 0xc0de3585, 0xc0ae50e6, 0xc0de4da5, 0xc0aed0bb, 0xc0aee0df, 0xc0affd3f, 0xc0affe9c, 0xc0affed6}}}, v_interlock = {lock_object = {lo_name = 0xc0f95738 "vnode interlock", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc7d721e0}, mtx_lock = 0x4}, v_vnlock = 0xcb254058, v_holdcnt = 0x1, v_usecount = 0x0, v_iflag = 0x0, v_vflag = 0x0, v_writecount = 0x0, v_freelist = {tqe_next = 0x0, tqe_prev = 0x0}, v_bufobj = {bo_mtx = {lock_object = {lo_name = 0xc0f9e452 "bufobj interlock", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc7d77c10}, mtx_lock = 0x4}, bo_clean = {bv_hd = {tqh_first = 0xe195f160, tqh_last = 0xe195f198}, bv_root = 0xe195f160, bv_cnt = 0x1}, bo_dirty = {bv_hd = {tqh_first = 0x0, tqh_last = 0xcb25411c}, bv_root = 0x0, bv_cnt = 0x0}, bo_numoutput = 0x0, bo_flag = 0x0, bo_ops = 0xc10ca8a0, bo_bsize = 0x8000, bo_object = 0x0, bo_synclist = {le_next = 0x0, le_prev = 0x0}, bo_private = 0xcb254000, __bo_vnode = 0xcb254000}, v_pollinfo = 0x0, v_label = 0x0, v_lockf = 0x0} (kgdb) p *mvp $2 = {v_type = VMARKER, v_tag = 0x0, v_op = 0x0, v_data = 0x0, v_mount = 0xca76b5a8, v_nmntvnodes = {tqe_next = 0xcb253d98, tqe_prev = 0xcb254014}, v_un = { vu_mount = 0x0, vu_socket = 0x0, vu_cdev = 0x0, vu_fifoinfo = 0x0}, v_hashlist = {le_next = 0x0, le_prev = 0x0}, v_hash = 0x0, v_cache_src = {lh_first = 0x0}, v_cache_dst = {tqh_first = 0x0, tqh_last = 0x0}, v_cache_dd = 0x0, v_cstart = 0x0, v_lasta = 0x0, v_lastw = 0x0, v_clen = 0x0, v_lock = {lock_object = { lo_name = 0x0, lo_flags = 0x0, lo_data = 0x0, lo_witness = 0x0}, lk_lock = 0x0, lk_exslpfail = 0x0, lk_timo = 0x0, lk_pri = 0x0, lk_stack = {depth = 0x0, pcs = {0x0 }}}, v_interlock = {lock_object = {lo_name = 0x0, lo_flags = 0x0, lo_data = 0x0, lo_witness = 0x0}, mtx_lock = 0x0}, v_vnlock = 0x0, v_holdcnt = 0x0, v_usecount = 0x0, v_iflag = 0x0, v_vflag = 0x0, v_writecount = 0x0, v_freelist = {tqe_next = 0x0, tqe_prev = 0x0}, v_bufobj = { bo_mtx = {lock_object = {lo_name = 0x0, lo_flags = 0x0, lo_data = 0x0, lo_witness = 0x0}, mtx_lock = 0x0}, bo_clean = {bv_hd = {tqh_first = 0x0, tqh_last = 0x0}, bv_root = 0x0, bv_cnt = 0x0}, bo_dirty = {bv_hd = {tqh_first = 0x0, tqh_last = 0x0}, bv_root = 0x0, bv_cnt = 0x0}, bo_numoutput = 0x0, bo_flag = 0x0, bo_ops = 0x0, bo_bsize = 0x0, bo_object = 0x0, bo_synclist = {le_next = 0x0, le_prev = 0x0}, bo_private = 0x0, __bo_vnode = 0x0}, v_pollinfo = 0x0, v_label = 0x0, v_lockf = 0x0} (kgdb) up #8 0xc0b09392 in vfs_write_suspend (mp=0xca76b5a8) at ../../../kern/vfs_vnops.c:1203 1203 if ((error = VFS_SYNC(mp, MNT_SUSPEND)) != 0) (kgdb) p *mp $3 = {mnt_mtx = {lock_object = {lo_name = 0xc0f7fc91 "struct mount mtx", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc7d72450}, mtx_lock = 0xcb2492e0}, mnt_gen = 0x1, mnt_list = {tqe_next = 0xca76b2d4, tqe_prev = 0xca76b894}, mnt_op = 0xc10dd3c0, mnt_vfc = 0xc10dd220, mnt_vnodecovered = 0xcadc3c3c, mnt_syncer = 0xcb253d98, mnt_ref = 0x1d, mnt_nvnodelist = {tqh_first = 0xcb25415c, tqh_last = 0xcb243170}, mnt_nvnodelistsize = 0x7, mnt_writeopcount = 0x0, mnt_kern_flag = 0x68000044, mnt_flag = 0x101011100, mnt_noasync = 0x1, mnt_opt = 0xc7ee1330, mnt_optnew = 0xca74a6d0, mnt_maxsymlinklen = 0x78, mnt_stat = { f_version = 0x20030518, f_type = 0x35, f_flags = 0x100001100, f_bsize = 0x1000, f_iosize = 0x8000, f_blocks = 0x7bfa5, f_bfree = 0x7adaa, f_bavail = 0x70ef9, f_files = 0x3fffe, f_ffree = 0x3fff8, f_syncwrites = 0x0, f_asyncwrites = 0x0, f_syncreads = 0x0, f_asyncreads = 0x0, f_spare = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, f_namemax = 0xff, f_owner = 0x0, f_fsid = {val = {0x4f3b6cf3, 0x672838c}}, f_charspare = '\0' , f_fstypename = "ufs", '\0' , f_mntfromname = "/dev/md5a", '\0' , f_mntonname = "/mnt", '\0' }, mnt_cred = 0xcae18c00, mnt_data = 0xca785c00, mnt_time = 0x0, mnt_iosize_max = 0x20000, mnt_export = 0xc7db2300, mnt_label = 0x0, mnt_hashseed = 0x98d2c1fa, mnt_lockref = 0x1, mnt_secondary_writes = 0x0, mnt_secondary_accwrites = 0x12, mnt_susp_owner = 0xcb2492e0, mnt_gjprovider = 0x0, mnt_explock = {lock_object = { lo_name = 0xc0f7fca2 "explock", lo_flags = 0x5730000, lo_data = 0x0, lo_witness = 0xc7d780f0}, lk_lock = 0x1, lk_exslpfail = 0x0, lk_timo = 0x0, lk_pri = 0x60, lk_stack = {depth = 0x6, pcs = {0xc0a4393e, 0xc0ae9037, 0xc0af18ce, 0xc0af1f2f, 0xc0dc08c7, 0xc0da9bf1, 0x0 }}}} (kgdb) $ svn diff -x -p /usr/src/sys Index: /usr/src/sys/sys/buf.h =================================================================== --- /usr/src/sys/sys/buf.h (revision 231573) +++ /usr/src/sys/sys/buf.h (working copy) @@ -479,10 +479,13 @@ void bwillwrite(void); int buf_dirty_count_severe(void); void bremfree(struct buf *); void bremfreef(struct buf *); /* XXX Force bremfree, only for nfs. */ -int bread(struct vnode *, daddr_t, int, struct ucred *, struct buf **); +#define bread(vp, blkno, size, cred, bpp) \ + breadn_flags(vp, blkno, size, 0, 0, 0, cred, 0, bpp) +#define breadn(vp, blkno, size, rablkno, rabsize, cnt, cred, bpp) \ + breadn_flags(vp, blkno, size, rablkno, rabsize, cnt, cred, 0, bpp) +int breadn_flags(struct vnode *, daddr_t, int, daddr_t *, int *, int, + struct ucred *, int, struct buf **); void breada(struct vnode *, daddr_t *, int *, int, struct ucred *); -int breadn(struct vnode *, daddr_t, int, daddr_t *, int *, int, - struct ucred *, struct buf **); void bdwrite(struct buf *); void bawrite(struct buf *); void bdirty(struct buf *); Index: /usr/src/sys/ufs/ufs/inode.h =================================================================== --- /usr/src/sys/ufs/ufs/inode.h (revision 231573) +++ /usr/src/sys/ufs/ufs/inode.h (working copy) @@ -158,6 +158,7 @@ struct inode { #define SHORTLINK(ip) \ (((ip)->i_ump->um_fstype == UFS1) ? \ (caddr_t)(ip)->i_din1->di_db : (caddr_t)(ip)->i_din2->di_db) +#define IS_SNAPSHOT(ip) ((ip)->i_flags & SF_SNAPSHOT) /* * Structure used to pass around logical block paths generated by Index: /usr/src/sys/ufs/ffs/ffs_softdep.c =================================================================== --- /usr/src/sys/ufs/ffs/ffs_softdep.c (revision 231573) +++ /usr/src/sys/ufs/ffs/ffs_softdep.c (working copy) @@ -2827,7 +2827,12 @@ softdep_prealloc(vp, waitok) { struct ufsmount *ump; - if (DOINGSUJ(vp) == 0) + /* + * Nothing to do if we are not running journaled soft updates. + * If we currently hold the snapshot lock, we must avoid handling + * other resources that could cause deadlock. + */ + if (DOINGSUJ(vp) == 0 || IS_SNAPSHOT(VTOI(vp))) return (0); ump = VFSTOUFS(vp->v_mount); ACQUIRE_LOCK(&lk); @@ -2873,7 +2878,12 @@ softdep_prelink(dvp, vp) ump = VFSTOUFS(dvp->v_mount); mtx_assert(&lk, MA_OWNED); - if (journal_space(ump, 0)) + /* + * Nothing to do if we have sufficient journal space. + * If we currently hold the snapshot lock, we must avoid + * handling other resources that could cause deadlock. + */ + if (journal_space(ump, 0) || (vp && IS_SNAPSHOT(VTOI(vp)))) return; stat_journal_low++; FREE_LOCK(&lk); @@ -4304,11 +4314,15 @@ inodedep_lookup_ip(ip) struct inode *ip; { struct inodedep *inodedep; + int dflags; KASSERT(ip->i_nlink >= ip->i_effnlink, ("inodedep_lookup_ip: bad delta")); - (void) inodedep_lookup(UFSTOVFS(ip->i_ump), ip->i_number, - DEPALLOC, &inodedep); + dflags = DEPALLOC; + if (IS_SNAPSHOT(ip)) + dflags |= NODELAY; + (void) inodedep_lookup(UFSTOVFS(ip->i_ump), ip->i_number, dflags, + &inodedep); inodedep->id_nlinkdelta = ip->i_nlink - ip->i_effnlink; return (inodedep); @@ -4696,7 +4710,7 @@ softdep_setup_inomapdep(bp, ip, newinum, mode) * the cylinder group map from which it was allocated. */ ACQUIRE_LOCK(&lk); - if ((inodedep_lookup(mp, newinum, DEPALLOC|NODELAY, &inodedep))) + if ((inodedep_lookup(mp, newinum, DEPALLOC | NODELAY, &inodedep))) panic("softdep_setup_inomapdep: dependency %p for new" "inode already exists", inodedep); bmsafemap = bmsafemap_lookup(mp, bp, ino_to_cg(fs, newinum)); @@ -5437,6 +5451,7 @@ softdep_setup_allocindir_page(ip, lbn, bp, ptrno, struct allocindir *aip; struct pagedep *pagedep; struct mount *mp; + int dflags; if (lbn != nbp->b_lblkno) panic("softdep_setup_allocindir_page: lbn %jd != lblkno %jd", @@ -5444,7 +5459,10 @@ softdep_setup_allocindir_page(ip, lbn, bp, ptrno, ASSERT_VOP_LOCKED(ITOV(ip), "softdep_setup_allocindir_page"); mp = UFSTOVFS(ip->i_ump); aip = newallocindir(ip, ptrno, newblkno, oldblkno, lbn); - (void) inodedep_lookup(mp, ip->i_number, DEPALLOC, &inodedep); + dflags = DEPALLOC; + if (IS_SNAPSHOT(ip)) + dflags |= NODELAY; + (void) inodedep_lookup(mp, ip->i_number, dflags, &inodedep); /* * If we are allocating a directory page, then we must * allocate an associated pagedep to track additions and @@ -5474,11 +5492,15 @@ softdep_setup_allocindir_meta(nbp, ip, bp, ptrno, struct inodedep *inodedep; struct allocindir *aip; ufs_lbn_t lbn; + int dflags; lbn = nbp->b_lblkno; ASSERT_VOP_LOCKED(ITOV(ip), "softdep_setup_allocindir_meta"); aip = newallocindir(ip, ptrno, newblkno, 0, lbn); - inodedep_lookup(UFSTOVFS(ip->i_ump), ip->i_number, DEPALLOC, &inodedep); + dflags = DEPALLOC; + if (IS_SNAPSHOT(ip)) + dflags |= NODELAY; + inodedep_lookup(UFSTOVFS(ip->i_ump), ip->i_number, dflags, &inodedep); WORKLIST_INSERT(&nbp->b_dep, &aip->ai_block.nb_list); if (setup_allocindir_phase2(bp, ip, inodedep, aip, lbn)) panic("softdep_setup_allocindir_meta: Block already existed"); @@ -6085,11 +6107,7 @@ softdep_journal_freeblocks(ip, cred, length, flags struct mount *mp; ufs2_daddr_t extblocks, datablocks; ufs_lbn_t tmpval, lbn, lastlbn; - int frags; - int lastoff, iboff; - int allocblock; - int error, i; - int needj; + int frags, lastoff, iboff, allocblock, needj, dflags, error, i; fs = ip->i_fs; mp = UFSTOVFS(ip->i_ump); @@ -6107,7 +6125,10 @@ softdep_journal_freeblocks(ip, cred, length, flags * we don't need to journal the block frees. The canceled journals * for the allocations will suffice. */ - inodedep_lookup(mp, ip->i_number, DEPALLOC, &inodedep); + dflags = DEPALLOC; + if (IS_SNAPSHOT(ip)) + dflags |= NODELAY; + inodedep_lookup(mp, ip->i_number, dflags, &inodedep); if ((inodedep->id_state & (UNLINKED | DEPCOMPLETE)) == UNLINKED && length == 0) needj = 0; @@ -6232,7 +6253,7 @@ softdep_journal_freeblocks(ip, cred, length, flags *((struct ufs2_dinode *)bp->b_data + ino_to_fsbo(fs, ip->i_number)) = *ip->i_din2; ACQUIRE_LOCK(&lk); - (void) inodedep_lookup(mp, ip->i_number, DEPALLOC, &inodedep); + (void) inodedep_lookup(mp, ip->i_number, dflags, &inodedep); if ((inodedep->id_state & IOSTARTED) != 0) panic("softdep_setup_freeblocks: inode busy"); /* @@ -6310,7 +6331,7 @@ softdep_journal_freeblocks(ip, cred, length, flags } ACQUIRE_LOCK(&lk); - inodedep_lookup(mp, ip->i_number, DEPALLOC, &inodedep); + inodedep_lookup(mp, ip->i_number, dflags, &inodedep); TAILQ_INSERT_TAIL(&inodedep->id_freeblklst, freeblks, fb_next); freeblks->fb_state |= DEPCOMPLETE | ONDEPLIST; /* @@ -6398,7 +6419,7 @@ softdep_setup_freeblocks(ip, length, flags) struct fs *fs; ufs2_daddr_t extblocks, datablocks; struct mount *mp; - int i, delay, error; + int i, delay, error, dflags; ufs_lbn_t tmpval; ufs_lbn_t lbn; @@ -6463,7 +6484,10 @@ softdep_setup_freeblocks(ip, length, flags) * Find and eliminate any inode dependencies. */ ACQUIRE_LOCK(&lk); - (void) inodedep_lookup(mp, ip->i_number, DEPALLOC, &inodedep); + dflags = DEPALLOC; + if (IS_SNAPSHOT(ip)) + dflags |= NODELAY; + (void) inodedep_lookup(mp, ip->i_number, dflags, &inodedep); if ((inodedep->id_state & IOSTARTED) != 0) panic("softdep_setup_freeblocks: inode busy"); /* @@ -8029,7 +8053,7 @@ softdep_setup_directory_add(bp, dp, diroffset, new dap->da_pagedep = pagedep; LIST_INSERT_HEAD(&pagedep->pd_diraddhd[DIRADDHASH(offset)], dap, da_pdlist); - inodedep_lookup(mp, newinum, DEPALLOC, &inodedep); + inodedep_lookup(mp, newinum, DEPALLOC | NODELAY, &inodedep); /* * If we're journaling, link the diradd into the jaddref so it * may be completed after the journal entry is written. Otherwise, @@ -8631,8 +8655,7 @@ newdirrem(bp, dp, ip, isrmdir, prevdirremp) * the number of freefile and freeblks structures. */ ACQUIRE_LOCK(&lk); - if (!(ip->i_flags & SF_SNAPSHOT) && - dep_current[D_DIRREM] > max_softdeps / 2) + if (!IS_SNAPSHOT(ip) && dep_current[D_DIRREM] > max_softdeps / 2) (void) request_cleanup(ITOV(dp)->v_mount, FLUSH_BLOCKS); FREE_LOCK(&lk); dirrem = malloc(sizeof(struct dirrem), @@ -8866,11 +8889,11 @@ softdep_setup_directory_change(bp, dp, ip, newinum /* * Lookup the jaddref for this journal entry. We must finish * initializing it and make the diradd write dependent on it. - * If we're not journaling Put it on the id_bufwait list if the inode - * is not yet written. If it is written, do the post-inode write - * processing to put it on the id_pendinghd list. + * If we're not journaling, put it on the id_bufwait list if the + * inode is not yet written. If it is written, do the post-inode + * write processing to put it on the id_pendinghd list. */ - inodedep_lookup(mp, newinum, DEPALLOC, &inodedep); + inodedep_lookup(mp, newinum, DEPALLOC | NODELAY, &inodedep); if (MOUNTEDSUJ(mp)) { jaddref = (struct jaddref *)TAILQ_LAST(&inodedep->id_inoreflst, inoreflst); @@ -8912,9 +8935,13 @@ softdep_change_linkcnt(ip) struct inode *ip; /* the inode with the increased link count */ { struct inodedep *inodedep; + int dflags; ACQUIRE_LOCK(&lk); - inodedep_lookup(UFSTOVFS(ip->i_ump), ip->i_number, DEPALLOC, &inodedep); + dflags = DEPALLOC; + if (IS_SNAPSHOT(ip)) + dflags |= NODELAY; + inodedep_lookup(UFSTOVFS(ip->i_ump), ip->i_number, dflags, &inodedep); if (ip->i_nlink < ip->i_effnlink) panic("softdep_change_linkcnt: bad delta"); inodedep->id_nlinkdelta = ip->i_nlink - ip->i_effnlink; @@ -12544,22 +12571,25 @@ softdep_request_cleanup(fs, vp, cred, resource) ufs2_daddr_t needed; int error; - mp = vp->v_mount; - ump = VFSTOUFS(mp); - mtx_assert(UFS_MTX(ump), MA_OWNED); - if (resource == FLUSH_BLOCKS_WAIT) - stat_cleanup_blkrequests += 1; - else - stat_cleanup_inorequests += 1; - /* * If we are being called because of a process doing a * copy-on-write, then it is not safe to process any * worklist items as we will recurse into the copyonwrite * routine. This will result in an incoherent snapshot. + * If the vnode that we hold is a snapshot, we must avoid + * handling other resources that could cause deadlock. */ - if (curthread->td_pflags & TDP_COWINPROGRESS) + if ((curthread->td_pflags & TDP_COWINPROGRESS) || IS_SNAPSHOT(VTOI(vp))) return (0); + + if (resource == FLUSH_BLOCKS_WAIT) + stat_cleanup_blkrequests += 1; + else + stat_cleanup_inorequests += 1; + + mp = vp->v_mount; + ump = VFSTOUFS(mp); + mtx_assert(UFS_MTX(ump), MA_OWNED); UFS_UNLOCK(ump); error = ffs_update(vp, 1); if (error != 0) { Index: /usr/src/sys/ufs/ffs/ffs_vnops.c =================================================================== --- /usr/src/sys/ufs/ffs/ffs_vnops.c (revision 231573) +++ /usr/src/sys/ufs/ffs/ffs_vnops.c (working copy) @@ -216,8 +216,10 @@ ffs_syncvnode(struct vnode *vp, int waitfor) struct buf *bp; struct buf *nbp; ufs_lbn_t lbn; - int error, wait, passes; + int error, wait, passes, noupdate; + noupdate = waitfor & NO_INO_UPDT; + waitfor &= ~NO_INO_UPDT; ip = VTOI(vp); ip->i_flag &= ~IN_NEEDSYNC; bo = &vp->v_bufobj; @@ -300,7 +302,10 @@ next: } if (waitfor != MNT_WAIT) { BO_UNLOCK(bo); - return (ffs_update(vp, waitfor)); + if (noupdate) + return (0); + else + return (ffs_update(vp, waitfor)); } /* Drain IO to see if we're done. */ bufobj_wwait(bo, 0, 0); @@ -317,7 +322,7 @@ next: */ if (bo->bo_dirty.bv_cnt > 0) { /* Write the inode after sync passes to flush deps. */ - if (wait && DOINGSOFTDEP(vp)) { + if (wait && DOINGSOFTDEP(vp) && noupdate == 0) { BO_UNLOCK(bo); ffs_update(vp, MNT_WAIT); BO_LOCK(bo); @@ -332,7 +337,9 @@ next: #endif } BO_UNLOCK(bo); - error = ffs_update(vp, MNT_WAIT); + error = 0; + if (noupdate == 0) + error = ffs_update(vp, MNT_WAIT); if (DOINGSUJ(vp)) softdep_journal_fsync(VTOI(vp)); return (error); Index: /usr/src/sys/ufs/ffs/ffs_extern.h =================================================================== --- /usr/src/sys/ufs/ffs/ffs_extern.h (revision 231573) +++ /usr/src/sys/ufs/ffs/ffs_extern.h (working copy) @@ -167,6 +167,13 @@ void softdep_freework(struct workhead *); #define FLUSH_INODES_WAIT 2 #define FLUSH_BLOCKS 3 #define FLUSH_BLOCKS_WAIT 4 +/* + * Flag to ffs_syncinode() to request flushing of data only, + * but skip the ffs_update() on the inode itself. Used to avoid + * deadlock when flushing snapshot inodes while holding snaplk. + * Avoid bit conflicts with MNT_WAIT values in sys/mount.h + */ +#define NO_INO_UPDT 0x10 int ffs_rdonly(struct inode *); Index: /usr/src/sys/ufs/ffs/ffs_inode.c =================================================================== --- /usr/src/sys/ufs/ffs/ffs_inode.c (revision 231573) +++ /usr/src/sys/ufs/ffs/ffs_inode.c (working copy) @@ -81,7 +81,7 @@ ffs_update(vp, waitfor) struct fs *fs; struct buf *bp; struct inode *ip; - int error; + int flags, error; ASSERT_VOP_ELOCKED(vp, "ffs_update"); ufs_itimes(vp); @@ -92,11 +92,32 @@ ffs_update(vp, waitfor) fs = ip->i_fs; if (fs->fs_ronly && ip->i_ump->um_fsckpid == 0) return (0); - error = bread(ip->i_devvp, fsbtodb(fs, ino_to_fsba(fs, ip->i_number)), - (int)fs->fs_bsize, NOCRED, &bp); - if (error) { - brelse(bp); - return (error); + /* + * If we are updating a snapshot and another process is currently + * writing the buffer containing the inode for this snapshot then + * a deadlock can occur when it tries to check the snapshot to see + * if that block needs to be copied. Thus when updating a snapshot + * we check to see if the buffer is already locked, and if it is + * we drop the snapshot lock until the buffer has been written + * and is available to us. + */ + flags = 0; + if (IS_SNAPSHOT(ip)) + flags = GB_LOCK_NOWAIT; + error = breadn_flags(ip->i_devvp, + fsbtodb(fs, ino_to_fsba(fs, ip->i_number)), + (int) fs->fs_bsize, 0, 0, 0, NOCRED, flags, &bp); + if (error != 0) { + if (error != EBUSY) { + brelse(bp); + return (error); + } + KASSERT((IS_SNAPSHOT(ip)), ("EBUSY from non-snapshot")); + VOP_UNLOCK(vp, 0); + (void) bread(ip->i_devvp, + fsbtodb(fs, ino_to_fsba(fs, ip->i_number)), + (int) fs->fs_bsize, NOCRED, &bp); + vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); } if (DOINGSOFTDEP(vp)) softdep_update_inodeblock(ip, bp, waitfor); Index: /usr/src/sys/ufs/ffs/ffs_snapshot.c =================================================================== --- /usr/src/sys/ufs/ffs/ffs_snapshot.c (revision 231573) +++ /usr/src/sys/ufs/ffs/ffs_snapshot.c (working copy) @@ -203,6 +203,7 @@ ffs_snapshot(mp, snapfile) ufs2_daddr_t numblks, blkno, *blkp, *snapblklist; int error, cg, snaploc; int i, size, len, loc; + ufs2_daddr_t blockno; uint64_t flag; struct timespec starttime = {0, 0}, endtime; char saved_nice = 0; @@ -225,18 +226,10 @@ ffs_snapshot(mp, snapfile) ump = VFSTOUFS(mp); fs = ump->um_fs; sn = NULL; - /* - * At the moment, journaled soft updates cannot support - * taking snapshots. - */ - if (MOUNTEDSUJ(mp)) { - vfs_mount_error(mp, "%s: Snapshots are not yet supported when " - "running with journaled soft updates", fs->fs_fsmnt); - return (EOPNOTSUPP); - } MNT_ILOCK(mp); flag = mp->mnt_flag; MNT_IUNLOCK(mp); + /* * Need to serialize access to snapshot code per filesystem. */ @@ -815,21 +808,26 @@ out1: if (space != NULL) free(space, M_UFSMNT); /* - * If another process is currently writing the buffer containing - * the inode for this snapshot then a deadlock can occur. Drop - * the snapshot lock until the buffer has been written. + * Preallocate all the direct blocks in the snapshot inode so + * that we never have to write the inode itself to commit an + * update to the contents of the snapshot. Note that once + * created, the size of the snapshot will never change, so + * there will never be a need to write the inode except to + * update the non-integrity-critical time fields and + * allocated-block count. */ - VREF(vp); /* Protect against ffs_snapgone() */ - VOP_UNLOCK(vp, 0); - (void) bread(ip->i_devvp, - fsbtodb(fs, ino_to_fsba(fs, ip->i_number)), - (int) fs->fs_bsize, NOCRED, &nbp); - brelse(nbp); - vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); - if (ip->i_effnlink == 0) - error = ENOENT; /* Snapshot file unlinked */ - else - vrele(vp); /* Drop extra reference */ + for (blockno = 0; blockno < NDADDR; blockno++) { + if (DIP(ip, i_db[blockno]) != 0) + continue; + error = UFS_BALLOC(vp, lblktosize(fs, blockno), + fs->fs_bsize, KERNCRED, BA_CLRBUF, &bp); + if (error) + break; + error = readblock(vp, bp, blockno); + bawrite(bp); + if (error != 0) + break; + } done: free(copy_fs->fs_csp, M_UFSMNT); free(copy_fs, M_UFSMNT); @@ -1902,7 +1900,7 @@ retry: bawrite(cbp); if ((vtype == VDIR || dopersistence) && ip->i_effnlink > 0) - (void) ffs_syncvnode(vp, MNT_WAIT); + (void) ffs_syncvnode(vp, MNT_WAIT|NO_INO_UPDT); continue; } /* @@ -1913,7 +1911,7 @@ retry: bawrite(cbp); if ((vtype == VDIR || dopersistence) && ip->i_effnlink > 0) - (void) ffs_syncvnode(vp, MNT_WAIT); + (void) ffs_syncvnode(vp, MNT_WAIT|NO_INO_UPDT); break; } savedcbp = cbp; @@ -1931,7 +1929,7 @@ retry: bawrite(savedcbp); if ((vtype == VDIR || dopersistence) && VTOI(vp)->i_effnlink > 0) - (void) ffs_syncvnode(vp, MNT_WAIT); + (void) ffs_syncvnode(vp, MNT_WAIT|NO_INO_UPDT); } /* * If we have been unable to allocate a block in which to do @@ -2395,7 +2393,7 @@ ffs_copyonwrite(devvp, bp) bawrite(cbp); if ((devvp == bp->b_vp || bp->b_vp->v_type == VDIR || dopersistence) && ip->i_effnlink > 0) - (void) ffs_syncvnode(vp, MNT_WAIT); + (void) ffs_syncvnode(vp, MNT_WAIT|NO_INO_UPDT); else launched_async_io = 1; continue; @@ -2408,7 +2406,7 @@ ffs_copyonwrite(devvp, bp) bawrite(cbp); if ((devvp == bp->b_vp || bp->b_vp->v_type == VDIR || dopersistence) && ip->i_effnlink > 0) - (void) ffs_syncvnode(vp, MNT_WAIT); + (void) ffs_syncvnode(vp, MNT_WAIT|NO_INO_UPDT); else launched_async_io = 1; break; @@ -2428,7 +2426,7 @@ ffs_copyonwrite(devvp, bp) bawrite(savedcbp); if ((devvp == bp->b_vp || bp->b_vp->v_type == VDIR || dopersistence) && VTOI(vp)->i_effnlink > 0) - (void) ffs_syncvnode(vp, MNT_WAIT); + (void) ffs_syncvnode(vp, MNT_WAIT|NO_INO_UPDT); else launched_async_io = 1; } @@ -2478,7 +2476,7 @@ ffs_sync_snap(mp, waitfor) } TAILQ_FOREACH(ip, &sn->sn_head, i_nextsnap) { vp = ITOV(ip); - ffs_syncvnode(vp, waitfor); + ffs_syncvnode(vp, waitfor|NO_INO_UPDT); } lockmgr(&sn->sn_lock, LK_RELEASE, NULL); } Index: /usr/src/sys/kern/vfs_bio.c =================================================================== --- /usr/src/sys/kern/vfs_bio.c (revision 231573) +++ /usr/src/sys/kern/vfs_bio.c (working copy) @@ -782,22 +782,7 @@ bremfreel(struct buf *bp) } } - /* - * Get a buffer with the specified data. Look in the cache first. We - * must clear BIO_ERROR and B_INVAL prior to initiating I/O. If B_CACHE - * is set, the buffer is valid and we do not have to do anything ( see - * getblk() ). This is really just a special case of breadn(). - */ -int -bread(struct vnode * vp, daddr_t blkno, int size, struct ucred * cred, - struct buf **bpp) -{ - - return (breadn(vp, blkno, size, 0, 0, 0, cred, bpp)); -} - -/* * Attempt to initiate asynchronous I/O on read-ahead blocks. We must * clear BIO_ERROR and B_INVAL prior to initiating I/O . If B_CACHE is set, * the buffer is valid and we do not have to do anything. @@ -834,19 +819,28 @@ breada(struct vnode * vp, daddr_t * rablkno, int * } /* - * Operates like bread, but also starts asynchronous I/O on - * read-ahead blocks. + * Entry point for bread() and breadn() via #defines in sys/buf.h. + * + * Get a buffer with the specified data. Look in the cache first. We + * must clear BIO_ERROR and B_INVAL prior to initiating I/O. If B_CACHE + * is set, the buffer is valid and we do not have to do anything, see + * getblk(). Also starts asynchronous I/O on read-ahead blocks. */ int -breadn(struct vnode * vp, daddr_t blkno, int size, - daddr_t * rablkno, int *rabsize, - int cnt, struct ucred * cred, struct buf **bpp) +breadn_flags(struct vnode * vp, daddr_t blkno, int size, + daddr_t * rablkno, int *rabsize, int cnt, + struct ucred * cred, int flags, struct buf **bpp) { struct buf *bp; int rv = 0, readwait = 0; CTR3(KTR_BUF, "breadn(%p, %jd, %d)", vp, blkno, size); - *bpp = bp = getblk(vp, blkno, size, 0, 0, 0); + /* + * Can only return NULL if GB_LOCK_NOWAIT flag is specified. + */ + *bpp = bp = getblk(vp, blkno, size, 0, 0, flags); + if (bp == NULL) + return (EBUSY); /* if not found in cache, do some I/O */ if ((bp->b_flags & B_CACHE) == 0) { -- Test scenario wip/suj28.sh + mdconfig -a -t swap -s 2g -u 5 + bsdlabel -w md5 auto + [ 0 -eq 1 ] + opt=-j + newfs -j md5a + mount /dev/md5a /mnt + chmod 777 /mnt + export runRUNTIME=5m + export RUNDIR=/mnt/stressX + export creatINCARNATIONS=2 + export 'TESTPROGS= testcases/creat/creat testcases/mkdir/mkdir testcases/swap/swap ' + rpid=2308 + mdstart2=6 + mntpoint2=/mnt2 + [ -d /mnt2 ] + su stress -c 'cd ..; ./testcases/run/run $TESTPROGS' + mount + grep -q /dev/md + grep /mnt2 + mdconfig -l + grep -q md6 + kill -0 2308 + rm -f /mnt/.snap/suj28 + grep -v 'Resource temporarily unavailable' + mksnap_ffs /mnt /mnt/.snap/suj28 + [ ! -s /mnt/.snap/suj28 ] + mdconfig -a -t vnode -f /mnt/.snap/suj28 -u 6 -o readonly + mount -r /dev/md6 /mnt2 + ls -l /mnt2 + mdconfig -d -u 6 mdconfig: ioctl(/dev/mdctl): Device busy + kill -0 2308 + rm -f /mnt/.snap/suj28 + grep -v 'Resource temporarily unavailable' + mksnap_ffs /mnt /mnt/.snap/suj28 + continue + [ ! -s /mnt/.snap/suj28 ] + continue + kill -0 2308 + rm -f /mnt/.snap/suj28 + grep -v 'Resource temporarily unavailable' + mksnap_ffs /mnt /mnt/.snap/suj28 load: 2.27 cmd: mksnap_ffs 2371 [running] 406.18r 0.00u 405.86s 100% 1084k load: 2.92 cmd: mksnap_ffs 2371 [running] 1044.56r 0.00u 1044.16s 100% 1084k load: 2.47 cmd: mksnap_ffs 2371 [running] 2455.36r 0.00u 2454.50s 100% 1084k