GENERIC HEAD from 2012-02-24 17:50:36 UTC, r232125M, vmcore.295 KDB: debugger backends: ddb KDB: current backend: ddb 524288K of memory above 4GB ignored Copyright (c) 1992-2012 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #3 r232125M: Sun Feb 26 06:55:04 CET 2012 pho@x4.osted.lan:/usr/src/sys/i386/compile/PHO i386 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. CPU: AMD Phenom(tm) 9150e Quad-Core Processor (1800.02-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x100f23 Family = 10 Model = 2 Stepping = 3 Features=0x178bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x7ff TSC: P-state invariant real memory = 4294967296 (4096 MB) avail memory = 3531640832 (3368 MB) : Trying to mount root from ufs:/dev/ad4s1a [rw]... Setting hostuuid: 00000000-0000-0000-0000-00218515337d. Setting hostid: 0x6b64ac17. Starting ddb. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: /dev/ad4s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1a: clean, 116665 free (1361 frags, 14413 blocks, 0.1% fragmentation) /dev/ad4s1f: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1f: clean, 146646 free (5038 frags, 17701 blocks, 0.5% fragmentation) /dev/label/tmp: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/label/tmp: clean, 116396761 free (2273 frags, 14549311 blocks, 0.0% fragmentation) /dev/ad4s1d: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1d: clean, 3803447 free (259991 frags, 442932 blocks, 2.6% fragmentation) /dev/ad4s1e: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1e: clean, 10900358 free (45838 frags, 1356815 blocks, 0.2% fragmentation) Mounting local file systems:. Setting hostname: x4.osted.lan. re0: link state changed to DOWN Starting Network: lo0 re0 fwe0 fwip0. lo0: flags=8049 metric 0 mtu 16384 options=3 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0xa inet 127.0.0.1 netmask 0xff000000 nd6 options=21 re0: f0:lags=8843 metric 0 mtu 1500 options=209b ether 00:21:85:15:33:7d inet 192.168.1.101 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::221:85ff:fe15:337d%re0 prefixlen 64 scopeid 0x1 nd6 options=29 media: Ethernet autoselect (100baseTX ) status: active fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb nd6 options=29 ch 1 dma -1 fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 nd6 options=29 Starting devd. Starting Network: fwe0. fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb nd6 options=29 ch 1 dma -1 Starting Network: fwip0. fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 nd6 options=29 add net default: gateway 192.168.1.1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/kde4/lib /usr/local/lib/compat /usr/local/lib/gcc46 /usr/local/lib/qt4 /usr/local/lib/virtualbox a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout Creating and/or trimming log files. Starting syslogd. No core dumps found. Additional ABI support: linux. Starting rpcbind. NFS access cache time=60 lock order reversal: 1st 0xe191de80 bufwait (bufwait) @ kern/vfs_bio.c:2658 2nd 0xca7a2600 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:284 KDB: stack backtrace: db_trace_self_wrapper(c0f92f73,2e687361,38323a63,a34,f277a7e8,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a9614b,c0f96a40,c1274ab8,11c,f277a878,...) at kdb_backtrace+0x2a _witness_debugger(c0f96a40,ca7a2600,c0fc6bda,c7d79610,c0fc686c,...) at _witness_debugger+0x25 witness_checkorder(ca7a2600,9,c0fc6863,11c,0,...) at witness_checkorder+0x86f _sx_xlock(ca7a2600,0,c0fc6863,11c,ca7fd740,...) at _sx_xlock+0xa0 ufsdirhash_acquire(e191de20,ca7fd740,f277a9f4,e23e6674,f277a948,...) at ufsdirhash_acquire+0x48 ufsdirhash_add(ca7fd740,f277a9f4,2674,f277a934,f277a938,...) at ufsdirhash_add+0x13 ufs_direnter(caaf515c,cacc1828,f277a9f4,f277ab84,e191e4a0,...) at ufs_direnter+0x7a9 ufs_mkdir(f277ac14,c0fe7f05,0,0,f277abac,...) at ufs_mkdir+0xa6f VOP_MKDIR_APV(c10ddda0,f277ac14,f277ab84,f277abac,0,...) at VOP_MKDIR_APV+0xc5 kern_mkdirat(ca7ab000,ffffff9c,28404020,0,1c0,...) at kern_mkdirat+0x291 kern_mkdir(ca7ab000,28404020,0,1c0,f277ad1c,...) at kern_mkdir+0x2e sys_mkdir(ca7ab000,f277acec,c0fe0d8a,c0f9774e,202,...) at sys_mkdir+0x29 syscall(f277ad28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x2817fba3, esp = 0xbfbfe8cc, ebp = 0xbfbfed78 --- Clearing /tmp (X related). Starting mountd. Feb 26 07:01:01 x4 mountd[1009]: can't delete exports for /dev: Invalid argument mount option is unknown Starting nfsuserd. Starting nfsd. Recovering vi editor sessions:lock order reversal: 1st 0xcad519dc ufs (ufs) @ kern/vfs_lookup.c:542 2nd 0xe1925040 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:260 3rd 0xcad5a5c8 ufs (ufs) @ kern/vfs_subr.c:2167 KDB: stack backtrace: db_trace_self_wrapper(c0f92f73,3a632e73,a303632,0,c62b8170,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a9614b,c0f96a59,c1274620,877,f281f33c,...) at kdb_backtrace+0x2a _witness_debugger(c0f96a59,cad5a5c8,c0f85397,c7d795a8,c0f9ea11,...) at _witness_debugger+0x25 witness_checkorder(cad5a5c8,9,c0f9ea08,877,0,...) at witness_checkorder+0x86f __lockmgr_args(cad5a5c8,80100,cad5a634,0,0,...) at __lockmgr_args+0x834 ffs_lock(f281f460,60,0,80100,cad5a570,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c10ddda0,f281f460,c7d721e0,c10eedc0,cad5a570,...) at VOP_LOCK1_APV+0xb5 _vn_lock(cad5a570,80100,c0f9ea08,877,4,...) at _vn_lock+0x78 vget(cad5a570,80100,cab262e0,50,0,...) at vget+0xbb vfs_hash_get(ca774000,15ee3d,80000,cab262e0,f281f5a4,...) at vfs_hash_get+0xed ffs_vgetf(ca774000,15ee3d,80000,f281f5a4,1,...) at ffs_vgetf+0x49 softdep_sync_buf(cad51984,e1924fe0,1,106,0,...) at softdep_sync_buf+0xac9 ffs_syncvnode(cad51984,1,c1274610,cad51a7c,c7d75308,...) at ffs_syncvnode+0x24c ffs_truncate(cad51984,200,0,880,c7dc5a80,...) at ffs_truncate+0x827 ufs_direnter(cad51984,cad5a570,f281f918,f281fbc0,0,...) at ufs_direnter+0x9a1 ufs_makeinode(f281fbc0,c10de320,f281fb04,f281fa60,c0de5bd5,...) at ufs_makeinode+0x65d ufs_create(f281fb04,c0fe80cb,0,0,f281fb80,...) at ufs_create+0x30 VOP_CREATE_APV(c10ddda0,f281fb04,f281fbc0,f281fa9c,0,...) at VOP_CREATE_APV+0xc5 vn_open_cred(f281fb80,f281fc48,1b0,0,c7dc5a80,...) at vn_open_cred+0x205 vn_open(f281fb80,f281fc48,1b0,ca7de620,4f49ca45,...) at vn_open+0x3b kern_openat(cab262e0,ffffff9c,28839f60,0,a03,...) at kern_openat+0x1ec kern_open(cab262e0,28839f60,0,a02,1b0,...) at kern_open+0x35 sys_open(cab262e0,f281fcec,c0fe0d8a,c0f9774e,206,...) at sys_open+0x30 syscall(f281fd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (5, FreeBSD ELF32, sys_open), eip = 0x283ec203, esp = 0xbfbfc50c, ebp = 0xbfbfc598 --- . Updating motd:. Starting ntpd. Starting default moused. Configuring syscons: keymap blanktime. Starting sshd. Starting cron. Local package initialization: backuplock order reversal: 1st 0xca8245c8 ufs (ufs) @ kern/vfs_mount.c:1249 2nd 0xcaaf5058 syncer (syncer) @ kern/vfs_subr.c:2279 KDB: stack backtrace: db_trace_self_wrapper(c0f92f73,a3937,cccccccc,ccccccc,7fff,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a9614b,c0f96a40,c1274200,8e7,f282e9dc,...) at kdb_backtrace+0x2a _witness_debugger(c0f96a40,caaf5058,c0f9f5c0,c7d79678,c0f9ea11,...) at _witness_debugger+0x25 witness_checkorder(caaf5058,9,c0f9ea08,8e7,0,...) at witness_checkorder+0x86f __lockmgr_args(caaf5058,80100,caaf50c4,0,0,...) at __lockmgr_args+0x834 vop_stdlock(f282eafc,c0aa630a,c1274204,80100,caaf5000,...) at vop_stdlock+0x65 VOP_LOCK1_APV(c10cc700,f282eafc,c0a4bd3c,c10eedc0,caaf5000,...) at VOP_LOCK1_APV+0xb5 _vn_lock(caaf5000,80100,c0f9ea08,8e7,c12a1d60,...) at _vn_lock+0x78 vputx(f282eb80,c0afeb1d,caaf5000,0,c0f9ea08,...) at vputx+0x266 vrele(caaf5000,0,c0f9ea08,db7,ca824570,...) at vrele+0x10 vfs_deallocate_syncvnode(ca7745a8,0,c0f9e06b,51a,c7c31000,...) at vfs_deallocate_syncvnode+0x6d dounmount(ca7745a8,8080000,ca8352e0,49c,0,...) at dounmount+0x376 sys_unmount(ca8352e0,f282ecec,c,c,206,...) at sys_unmount+0x421 syscall(f282ed28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280e800b, esp = 0xbfbfe64c, ebp = 0xbfbfe718 --- lock order reversal: 1st 0xca8245c8 ufs (ufs) @ kern/vfs_mount.c:1249 2nd 0xca823b38 devfs (devfs) @ ufs/ffs/ffs_softdep.c:1835 KDB: stack backtrace: db_trace_self_wrapper(c0f92f73,38313a63,a3533,c0aa873c,8,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a9614b,c0f96a40,c12741f0,72b,f282e9a0,...) at kdb_backtrace+0x2a _witness_debugger(c0f96a40,ca823b38,c0f8a354,c7d79408,c0fc2ce6,...) at _witness_debugger+0x25 witness_checkorder(ca823b38,9,c0fc2cdd,72b,ca823ba4,...) at witness_checkorder+0x86f __lockmgr_args(ca823b38,80400,ca823ba4,0,0,...) at __lockmgr_args+0x834 vop_stdlock(f282eac0,4,c0f8d34f,80400,ca823ae0,...) at vop_stdlock+0x65 VOP_LOCK1_APV(c10b38a0,f282eac0,c12a8040,c10eedc0,ca823ae0,...) at VOP_LOCK1_APV+0xb5 _vn_lock(ca823ae0,80400,c0fc2cdd,72b,ca7745a8,...) at _vn_lock+0x78 softdep_flushworklist(ca7745a8,f282eb6c,ca8352e0,5e9,0,...) at softdep_flushworklist+0x47 ffs_sync(ca7745a8,1,f282ebc0,51a,c7c31000,...) at ffs_sync+0x32d dounmount(ca7745a8,8080000,ca8352e0,49c,0,...) at dounmount+0x3f8 sys_unmount(ca8352e0,f282ecec,c,c,206,...) at sys_unmount+0x421 syscall(f282ed28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280e800b, esp = 0xbfbfe64c, ebp = 0xbfbfe718 --- ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes Feb 26 07:01:37 x4 su: pho to root on /dev/pts/1 ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 118102 files, 1917220 used, 116396761 free (2273 frags, 14549311 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 118102 files, 1917220 used, 116396761 free (2273 frags, 14549311 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** fsck -y /tmpFeb 26 07:04:35 x4 mountd[1010]: can't delete exports for /dev: Invalid argument mount option is unknown watchdogd. KDB: stack backtrace: db_trace_self_wrapper(c0f92f73,c0f9d21b,f29d98b0,c0ae7eca,cad54880,...) at db_trace_self_wrapper+0x26 kdb_backtrace(cad54880,f29d9914,c0de1195,f29d9924,cad54828,...) at kdb_backtrace+0x2a vfs_badlock(c10ddda0,f29d9924,c10eea00,cad54828) at vfs_badlock+0x23 assert_vop_elocked(cad54828,cb2012aa,0,f29d9980,f29d9964,...) at assert_vop_elocked+0x55 null_nodeget(cb20c5a8,cad54828,f29d9994,8000,cb2986cc,...) at null_nodeget+0x1f null_lookup(f29d9a20,c0f9f710,c10eea00,200000,f29d9ad4,...) at null_lookup+0xc3 VOP_LOOKUP_APV(cb2026e0,f29d9a20,f29d9b14,21e,c0f9de18,...) at VOP_LOOKUP_APV+0xe5 lookup(f29d9ad4,c0f9de18,10e,dc,ca824634,...) at lookup+0x6eb namei(f29d9ad4,cb2495c0,f29d9ad4,246,c11481d8,...) at namei+0x6bf kern_statat_vnhook(cb2495c0,200,ffffff9c,284201b8,0,...) at kern_statat_vnhook+0xStarting inetd.b f kern_statat(cb2495c0,200,ffffff9c,284201b8,0,...) at kern_statat+0x3c kern_lstat(cb2495c0,284201b8,0,f29d9c04,0,...) at kern_lstat+0x36 sys_lstat(cb2495c0,f29d9cec,c0fe0d8a,c0f97b41,246,...) at sys_lstat+0x2f syscall(f29d9d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (190, FreeBSD ELF32, sys_lstat), eip = 0x281d7b63, esp = 0xbfbfe56c, ebp = 0xbfbfe5f8 --- lowervp: 0xcad54828 is not exclusive locked but should be KDB: enter: lock violation [ thread pid 41009 tid 100248 ] Stopped at kdb_enter+0x3a: movl $0,kdb_why db> run pho db:0:pho> bt Tracing pid 41009 tid 100248 td 0xcb2495c0 kdb_enter(c0f9ebe2,c0f9fc4b,cad54828,c0f9f5c7,cad54828,...) at kdb_enter+0x3a vfs_badlock(c10ddda0,f29d9924,c10eea00,cad54828) at vfs_badlock+0x61 assert_vop_elocked(cad54828,cb2012aa,0,f29d9980,f29d9964,...) at assert_vop_elocked+0x55 null_nodeget(cb20c5a8,cad54828,f29d9994,8000,cb2986cc,...) at null_nodeget+0x1f null_lookup(f29d9a20,c0f9f710,c10eea00,200000,f29d9ad4,...) at null_lookup+0xc3 VOP_LOOKUP_APV(cb2026e0,f29d9a20,f29d9b14,21e,c0f9de18,...) at VOP_LOOKUP_APV+0xe5 lookup(f29d9ad4,c0f9de18,10e,dc,ca824634,...) at lookup+0x6eb namei(f29d9ad4,cb2495c0,f29d9ad4,246,c11481d8,...) at namei+0x6bf kern_statat_vnhook(cb2495c0,200,ffffff9c,284201b8,0,...) at kern_statat_vnhook+0xbf kern_statat(cb2495c0,200,ffffff9c,284201b8,0,...) at kern_statat+0x3c kern_lstat(cb2495c0,284201b8,0,f29d9c04,0,...) at kern_lstat+0x36 sys_lstat(cb2495c0,f29d9cec,c0fe0d8a,c0f97b41,246,...) at sys_lstat+0x2f syscall(f29d9d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (190, FreeBSD ELF32, sys_lstat), eip = 0x281d7b63, esp = 0xbfbfe56c, ebp = 0xbfbfe5f8 --- db:0:bt> show allpcpu Current CPU: 2 cpuid = 0 dynamic pcpu = 0x735a00 curthread = 0xc7dc98a0: pid 11 "idle: cpu0" curpcb = 0xc7b26d80 fpcurthread = none idlethread = 0xc7dc98a0: tid 100003 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: cpuid = 1 dynamic pcpu = 0x6a26a00 curthread = 0xcab275c0: pid 41042 "sh" curpcb = 0xf2789d80 fpcurthread = none idlethread = 0xc7dc95c0: tid 100004 "idle: cpu1" APIC ID = 1 currentldt = 0x50 spin locks held: cpuid = 2 dynamic pcpu = 0x6a29a00 curthread = 0xcb2495c0: pid 41009 "ls" curpcb = 0xf29d9d80 fpcurthread = none idlethread = 0xc7dc92e0: tid 100005 "idle: cpu2" APIC ID = 2 currentldt = 0x50 spin locks held: cpuid = 3 dynamic pcpu = 0x6a2ca00 curthread = 0xc7dc9000: pid 11 "idle: cpu3" curpcb = 0xc7b2fd80 fpcurthread = none idlethread = 0xc7dc9000: tid 100006 "idle: cpu3" APIC ID = 3 currentldt = 0x50 spin locks held: db:0:allpcpu> show alllocks Process 41009 (ls) thread 0xcb2495c0 (100248) shared lockmgr ufs (ufs) r = 0 (0xcad54880) locked @ kern/vfs_subr.c:2167 shared lockmgr ufs (ufs) r = 0 (0xca8245c8) locked @ /usr/src/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:595 db:0:alllocks> show lockedvnods Locked vnodes 0xca824570: tag ufs, type VDIR usecount 6, writecount 0, refcount 9 mountedhere 0xcab0587c flags () v_object 0xcaf7b40c ref 0 pages 1 lock type ufs: SHARED (count 1) #0 0xc0a43374 at __lockmgr_args+0x5c4 #1 0xc0ca6571 at ffs_lock+0xa1 #2 0xc0de4c05 at VOP_LOCK1_APV+0xb5 #3 0xcb200edf at null_lock+0xcf #4 0xc0de4c05 at VOP_LOCK1_APV+0xb5 #5 0xc0b09eb8 at _vn_lock+0x78 #6 0xc0aed1e6 at lookup+0x106 #7 0xc0aee7ef at namei+0x6bf #8 0xc0b0045f at kern_statat_vnhook+0xbf #9 0xc0b005bc at kern_statat+0x3c #10 0xc0b005f6 at kern_lstat+0x36 #11 0xc0b0069f at sys_lstat+0x2f #12 0xc0dc18a7 at syscall+0x2c7 #13 0xc0daabd1 at Xint0x80_syscall+0x21 ino 211968, on dev ada0s1a 0xcad54828: tag ufs, type VREG usecount 1, writecount 0, refcount 3 mountedhere 0 flags () v_object 0xcb0fa1bc ref 0 pages 1 lock type ufs: SHARED (count 1) #0 0xc0a43374 at __lockmgr_args+0x5c4 #1 0xc0ca6571 at ffs_lock+0xa1 #2 0xc0de4c05 at VOP_LOCK1_APV+0xb5 #3 0xc0b09eb8 at _vn_lock+0x78 #4 0xc0afc96b at vget+0xbb #5 0xc0aec36d at vfs_hash_get+0xed #6 0xc0c9f8d9 at ffs_vgetf+0x49 #7 0xc0c9fe5e at ffs_vget+0x2e #8 0xc0cae9c4 at ufs_lookup_ino+0xbc4 #9 0xc0caea6a at ufs_lookup+0x2a #10 0xc0de4565 at VOP_CACHEDLOOKUP_APV+0xc5 #11 0xc0ae5816 at vfs_cache_lookup+0xe6 #12 0xc0de5d85 at VOP_LOOKUP_APV+0xe5 #13 0xcb200d74 at null_lookup+0x94 #14 0xc0de5d85 at VOP_LOOKUP_APV+0xe5 #15 0xc0aed7cb at lookup+0x6eb #16 0xc0aee7ef at namei+0x6bf #17 0xc0b0045f at kern_statat_vnhook+0xbf ino 213364, on dev ada0s1a 0xcb1d76cc: tag null, type VDIR usecount 1, writecount 0, refcount 1 mountedhere 0 flags (VV_ROOT) lock type ufs: SHARED (count 1) #0 0xc0a43374 at __lockmgr_args+0x5c4 #1 0xc0ca6571 at ffs_lock+0xa1 #2 0xc0de4c05 at VOP_LOCK1_APV+0xb5 #3 0xcb200edf at null_lock+0xcf #4 0xc0de4c05 at VOP_LOCK1_APV+0xb5 #5 0xc0b09eb8 at _vn_lock+0x78 #6 0xc0aed1e6 at lookup+0x106 #7 0xc0aee7ef at namei+0x6bf #8 0xc0b0045f at kern_statat_vnhook+0xbf #9 0xc0b005bc at kern_statat+0x3c #10 0xc0b005f6 at kern_lstat+0x36 #11 0xc0b0069f at sys_lstat+0x2f #12 0xc0dc18a7 at syscall+0x2c7 #13 0xc0daabd1 at Xint0x80_syscall+0x21 vp=0xcb1d76cc, lowervp=0xca824570 0xcb0dfd98: tag null, type VDIR usecount 1, writecount 0, refcount 1 mountedhere 0 flags (VV_ROOT) lock type ufs: SHARED (count 1) #0 0xc0a43374 at __lockmgr_args+0x5c4 #1 0xc0ca6571 at ffs_lock+0xa1 #2 0xc0de4c05 at VOP_LOCK1_APV+0xb5 #3 0xcb200edf at null_lock+0xcf #4 0xc0de4c05 at VOP_LOCK1_APV+0xb5 #5 0xc0b09eb8 at _vn_lock+0x78 #6 0xc0aed1e6 at lookup+0x106 #7 0xc0aee7ef at namei+0x6bf #8 0xc0b0045f at kern_statat_vnhook+0xbf #9 0xc0b005bc at kern_statat+0x3c #10 0xc0b005f6 at kern_lstat+0x36 #11 0xc0b0069f at sys_lstat+0x2f #12 0xc0dc18a7 at syscall+0x2c7 #13 0xc0daabd1 at Xint0x80_syscall+0x21 vp=0xcb0dfd98, lowervp=0xca824570 0xcac1c2b8: tag null, type VDIR usecount 1, writecount 0, refcount 1 mountedhere 0 flags (VV_ROOT) lock type ufs: SHARED (count 1) #0 0xc0a43374 at __lockmgr_args+0x5c4 #1 0xc0ca6571 at ffs_lock+0xa1 #2 0xc0de4c05 at VOP_LOCK1_APV+0xb5 #3 0xcb200edf at null_lock+0xcf #4 0xc0de4c05 at VOP_LOCK1_APV+0xb5 #5 0xc0b09eb8 at _vn_lock+0x78 #6 0xc0aed1e6 at lookup+0x106 #7 0xc0aee7ef at namei+0x6bf #8 0xc0b0045f at kern_statat_vnhook+0xbf #9 0xc0b005bc at kern_statat+0x3c #10 0xc0b005f6 at kern_lstat+0x36 #11 0xc0b0069f at sys_lstat+0x2f #12 0xc0dc18a7 at syscall+0x2c7 #13 0xc0daabd1 at Xint0x80_syscall+0x21 vp=0xcac1c2b8, lowervp=0xca824570 0xcb2986cc: tag null, type VDIR usecount 4, writecount 0, refcount 4 mountedhere 0 flags (VV_ROOT) v_object 0xcaf7b40c ref 0 pages 1 lock type ufs: SHARED (count 1) #0 0xc0a43374 at __lockmgr_args+0x5c4 #1 0xc0ca6571 at ffs_lock+0xa1 #2 0xc0de4c05 at VOP_LOCK1_APV+0xb5 #3 0xcb200edf at null_lock+0xcf #4 0xc0de4c05 at VOP_LOCK1_APV+0xb5 #5 0xc0b09eb8 at _vn_lock+0x78 #6 0xc0aed1e6 at lookup+0x106 #7 0xc0aee7ef at namei+0x6bf #8 0xc0b0045f at kern_statat_vnhook+0xbf #9 0xc0b005bc at kern_statat+0x3c #10 0xc0b005f6 at kern_lstat+0x36 #11 0xc0b0069f at sys_lstat+0x2f #12 0xc0dc18a7 at syscall+0x2c7 #13 0xc0daabd1 at Xint0x80_syscall+0x21 vp=0xcb2986cc, lowervp=0xca824570 0xcb25415c: tag null, type VDIR usecount 1, writecount 0, refcount 1 mountedhere 0 flags (VV_ROOT) lock type ufs: SHARED (count 1) #0 0xc0a43374 at __lockmgr_args+0x5c4 #1 0xc0ca6571 at ffs_lock+0xa1 #2 0xc0de4c05 at VOP_LOCK1_APV+0xb5 #3 0xcb200edf at null_lock+0xcf #4 0xc0de4c05 at VOP_LOCK1_APV+0xb5 #5 0xc0b09eb8 at _vn_lock+0x78 #6 0xc0aed1e6 at lookup+0x106 #7 0xc0aee7ef at namei+0x6bf #8 0xc0b0045f at kern_statat_vnhook+0xbf #9 0xc0b005bc at kern_statat+0x3c #10 0xc0b005f6 at kern_lstat+0x36 #11 0xc0b0069f at sys_lstat+0x2f #12 0xc0dc18a7 at syscall+0x2c7 #13 0xc0daabd1 at Xint0x80_syscall+0x21 vp=0xcb25415c, lowervp=0xca824570 db:0:lockedvnods> show mount 0xca774b50 /dev/ad4s1a on / (ufs) 0xca775000 devfs on /dev (devfs) 0xca77487c /dev/ad4s1f on /home (ufs) 0xca7742d4 /dev/ad4s1d on /usr (ufs) 0xca774000 /dev/ad4s1e on /var (ufs) 0xcab055a8 procfs on /proc (procfs) 0xcb2932d4 /tmp on /mnt13 (nullfs) 0xcb20c2d4 /tmp on /mnt6 (nullfs) 0xcb3c25a8 /tmp on /mnt2 (nullfs) 0xcb20c5a8 /tmp on /mnt12 (nullfs) 0xcb2525a8 /tmp on /mnt8 (nullfs) 0xcab0587c /dev/label/tmp on /tmp (ufs) More info: show mount db:0:mount> ps pid ppid pgrp uid state wmesg wchan cmd 41031 1 41031 0 Ss select 0xca773264 inetd 41009 41007 41006 0 R CPU 2 ls 41007 41006 41006 0 S wait 0xcab20598 sh 41006 1 41006 0 Ss wait 0xca82ab30 watchdogd 40994 1203 1201 1001 S nanslp 0xc112bb84 sleep 40919 1248 40919 0 S+ nanslp 0xc112bb84 sleep 1248 1244 1248 0 S+ wait 0xca82a864 bash 1244 1243 1244 0 S+ pause 0xcad16b88 csh 1243 1223 1243 1001 S+ wait 0xcafa4598 su 1223 1222 1223 1001 Ss+ wait 0xcb0f5b30 bash 1222 1220 1220 1001 S select 0xca84b0e4 sshd 1220 1143 1220 0 Ss select 0xca773764 sshd 1204 1201 1201 1001 S piperd 0xcab21190 awk 1203 1201 1201 1001 S wait 0xcafa0864 sh 1202 1197 1202 1001 Ss+ select 0xca773c64 top 1201 1199 1201 1001 Ss wait 0xcab1fb30 sh 1200 1198 1200 1001 Ss kqread 0xcb0b3880 tail 1199 1193 1193 1001 S select 0xca6c0424 sshd 1198 1192 1192 1001 S select 0xca773824 sshd 1197 1191 1191 1001 S select 0xca84b424 sshd 1193 1143 1193 0 Ss select 0xca6bf264 sshd 1192 1143 1192 0 Ss select 0xca84bbe4 sshd 1191 1143 1191 0 Ss select 0xca6bf7e4 sshd 1154 1 1154 0 Ss nanslp 0xc112bb84 cron 1149 1 1149 25 Ss pause 0xca82cb88 sendmail 1146 1 1146 0 Ss select 0xca6c0264 sendmail 1143 1 1143 0 Ss select 0xca6bf324 sshd 1112 1 1112 0 Ss select 0xca6c0664 moused 1092 1 1092 0 Ss select 0xca6c0c24 ntpd 1038 1032 1032 0 S (threaded) nfsd 100152 S rpcsvc 0xcad40190 nfsd: service 100151 S rpcsvc 0xcad401d0 nfsd: service 100150 S rpcsvc 0xcad40210 nfsd: service 100121 S rpcsvc 0xca7665d0 nfsd: master 1032 1 1032 0 Ss select 0xca7739e4 nfsd 1030 1026 1026 0 S select 0xca773d64 nfsuserd 1029 1026 1026 0 S select 0xca6c03a4 nfsuserd 1028 1026 1026 0 S select 0xca773da4 nfsuserd 1027 1026 1026 0 S select 0xca6c0364 nfsuserd 1026 1 1026 0 Ss pause 0xcad178bc nfsuserd 1010 1 1010 0 Ss select 0xca6bf9e4 mountd 977 1 977 0 Ss select 0xca773de4 rpcbind 962 1 962 0 Ss select 0xca6bfda4 syslogd 839 1 839 0 Ss select 0xca6bf5a4 devd 20 1 20 0 Ss+ wait 0xca7a9b30 sh 19 0 0 0 DL sdflush 0xc12a8080 [softdepflush] 18 0 0 0 DL vlruwt 0xca6c12cc [vnlru] 17 0 0 0 DL syncer 0xc12a1d74 [syncer] 16 0 0 0 DL psleep 0xc12a1a08 [bufdaemon] 9 0 0 0 DL pgzero 0xc12a9f5c [pagezero] 8 0 0 0 DL psleep 0xc12a9b48 [vmdaemon] 7 0 0 0 DL psleep 0xc12a9b10 [pagedaemon] 6 0 0 0 DL ccb_scan 0xc10f0354 [xpt_thrd] 5 0 0 0 DL waiting_ 0xc12a3898 [sctp_iterator] 4 0 0 0 DL ctl_work 0xc819a000 [ctl_thrd] 3 0 0 0 DL - 0xc7ffa03c [fdc0] 2 0 0 0 SL - 0xc808f000 [fw0_probe] 15 0 0 0 DL (threaded) [usb] 100058 D - 0xc8063d34 [usbus5] 100057 D - 0xc8063d04 [usbus5] 100056 D - 0xc8063cd4 [usbus5] 100055 D - 0xc8063ca4 [usbus5] 100053 D - 0xc8056b5c [usbus4] 100052 D - 0xc8056b2c [usbus4] 100051 D - 0xc8056afc [usbus4] 100050 D - 0xc8056acc [usbus4] 100049 D - 0xc8052b5c [usbus3] 100048 D - 0xc8052b2c [usbus3] 100047 D - 0xc8052afc [usbus3] 100046 D - 0xc8052acc [usbus3] 100045 D - 0xc804ab5c [usbus2] 100044 D - 0xc804ab2c [usbus2] 100043 D - 0xc804aafc [usbus2] 100042 D - 0xc804aacc [usbus2] 100040 D - 0xc8044b5c [usbus1] 100039 D - 0xc8044b2c [usbus1] 100038 D - 0xc8044afc [usbus1] 100037 D - 0xc8044acc [usbus1] 100035 D - 0xc8038b5c [usbus0] 100034 D - 0xc8038b2c [usbus0] 100033 D - 0xc8038afc [usbus0] 100032 D - 0xc8038acc [usbus0] 14 0 0 0 DL - 0xc112a984 [yarrow] 13 0 0 0 DL (threaded) [geom] 100015 D - 0xc1128568 [g_down] 100014 D - 0xc1128564 [g_up] 100013 D - 0xc112855c [g_event] 12 0 0 0 WL (threaded) [intr] 100066 I [irq12: psm0] 100065 I [irq1: atkbd0] 100063 I [swi0: uart] 100060 I [irq20: fwohci0] 100059 I [irq14: ata0] 100054 I [irq19: ehci0] 100041 I [irq18: ohci2 ohci4] 100036 I [irq17: ohci1 ohci3] 100031 I [irq16: hdac1 ohci0] 100030 I [irq22: ahci0] 100029 I [irq257: re0] 100028 I [irq256: hdac0] 100027 I [swi6: task queue] 100026 I [swi6: Giant taskq] 100024 I [swi5: +] 100020 I [swi2: cambio] 100012 I [swi4: clock] 100011 I [swi4: clock] 100010 I [swi4: clock] 100009 I [swi4: clock] 100008 I [swi3: vm] 100007 I [swi1: netisr 0] 11 0 0 0 RL (threaded) [idle] 100006 Run CPU 3 [idle: cpu3] 100005 CanRun [idle: cpu2] 100004 CanRun [idle: cpu1] 100003 Run CPU 0 [idle: cpu0] 1 0 1 0 SLs wait 0xc7dc7b30 [init] 10 0 0 0 DL audit_wo 0xc12a77c0 [audit] 0 0 0 0 DLs (threaded) [kernel] 100078 D - 0xc8035380 [mca taskq] 100068 D - 0xc112a984 [deadlkres] 100061 D - 0xc80356c0 [fw0_taskq] 100025 D - 0xc7f9de80 [thread taskq] 100023 D - 0xc7f9e040 [acpi_task_2] 100022 D - 0xc7f9e040 [acpi_task_1] 100021 D - 0xc7f9e040 [acpi_task_0] 100019 D - 0xc7f9e080 [ffs_trim taskq] 100018 D - 0xc7f9e0c0 [kqueue taskq] 100016 D - 0xc7dade80 [firmware taskq] 100000 D sched 0xc1128640 [swapper] 41042 20 20 0 RE+ CPU 1 sh db:0:ps> allt Tracing command inetd pid 41031 tid 100086 td 0xca6ce000 sched_switch(ca6ce000,0,104,1a3,aff45662,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca6ce000,0,c0f94960,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b96a,ca773250,0,c0f8d34f,ca6ce000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca773264,0,f2721a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca773264,ca773250,c0f970b0,6b2,cb1800a8,...) at _cv_wait_sig+0x260 seltdwait(cb1800a8,58,c7dc4e80,ca6ce000,e3,...) at seltdwait+0xa2 kern_select(ca6ce000,8,bfbfdfa0,0,0,0,20,ca819b30,4) at kern_select+0x655 sys_select(ca6ce000,f2721cec,c0fe0d8a,c0f978f3,202,...) at sys_select+0x66 syscall(f2721d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281b86d3, esp = 0xbfbfdf5c, ebp = 0xbfbfede8 --- Tracing command ls pid 41009 tid 100248 td 0xcb2495c0 kdb_enter(c0f9ebe2,c0f9fc4b,cad54828,c0f9f5c7,cad54828,...) at kdb_enter+0x3a vfs_badlock(c10ddda0,f29d9924,c10eea00,cad54828) at vfs_badlock+0x61 assert_vop_elocked(cad54828,cb2012aa,0,f29d9980,f29d9964,...) at assert_vop_elocked+0x55 null_nodeget(cb20c5a8,cad54828,f29d9994,8000,cb2986cc,...) at null_nodeget+0x1f null_lookup(f29d9a20,c0f9f710,c10eea00,200000,f29d9ad4,...) at null_lookup+0xc3 VOP_LOOKUP_APV(cb2026e0,f29d9a20,f29d9b14,21e,c0f9de18,...) at VOP_LOOKUP_APV+0xe5 lookup(f29d9ad4,c0f9de18,10e,dc,ca824634,...) at lookup+0x6eb namei(f29d9ad4,cb2495c0,f29d9ad4,246,c11481d8,...) at namei+0x6bf kern_statat_vnhook(cb2495c0,200,ffffff9c,284201b8,0,...) at kern_statat_vnhook+0xbf kern_statat(cb2495c0,200,ffffff9c,284201b8,0,...) at kern_statat+0x3c kern_lstat(cb2495c0,284201b8,0,f29d9c04,0,...) at kern_lstat+0x36 sys_lstat(cb2495c0,f29d9cec,c0fe0d8a,c0f97b41,246,...) at sys_lstat+0x2f syscall(f29d9d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (190, FreeBSD ELF32, sys_lstat), eip = 0x281d7b63, esp = 0xbfbfe56c, ebp = 0xbfbfe5f8 --- Tracing command sh pid 41007 tid 100102 td 0xca7af000 sched_switch(ca7af000,0,104,1a3,906613d2,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(ca7af000,0,c0f94960,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f94960,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cab20598,6c,c0f978a7,100,0,...) at sleepq_wait_sig+0x17 _sleep(cab20598,cab20620,16c,c0f978a7,0,...) at _sleep+0x385 kern_wait(ca7af000,ffffffff,f276bc60,2,0,...) at kern_wait+0x597 sys_wait4(ca7af000,f276bcec,c,c,216,...) at sys_wait4+0x3b syscall(f276bd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2817666b, esp = 0xbfbfeadc, ebp = 0xbfbfeaf8 --- Tracing command watchdogd pid 41006 tid 100118 td 0xca833b80 sched_switch(ca833b80,0,104,1a3,90218ed7,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(ca833b80,0,c0f94960,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f94960,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca82ab30,6c,c0f978a7,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca82ab30,ca82abb8,16c,c0f978a7,0,...) at _sleep+0x385 kern_wait(ca833b80,a02f,f279fc60,0,0,...) at kern_wait+0x597 sys_wait4(ca833b80,f279fcec,c,c,202,...) at sys_wait4+0x3b syscall(f279fd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2812c66b, esp = 0xbfbfec3c, ebp = 0xbfbfece8 --- Tracing command sleep pid 40994 tid 100128 td 0xcab5bb80 sched_switch(cab5bb80,0,104,1a3,8104e7b5,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(cab5bb80,0,c0f94960,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(2711,c0a9c980,cab5bb80,2,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c112bb84,6c,c0f90477,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c112bb84,0,16c,c0f90477,2711,...) at _sleep+0x34f kern_nanosleep(cab5bb80,f27ccc50,f27ccc58,a,0,...) at kern_nanosleep+0xc1 sys_nanosleep(cab5bb80,f27cccec,c0fe0d8a,c0f98128,246,...) at sys_nanosleep+0x6f syscall(f27ccd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x2816e293, esp = 0xbfbfec5c, ebp = 0xbfbfeca8 --- Tracing command sleep pid 40919 tid 100219 td 0xcb1292e0 sched_switch(cb1292e0,0,104,1a3,8eba8367,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(cb1292e0,0,c0f94960,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(493e1,c0a9c980,cb1292e0,0,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c112bb84,6c,c0f90477,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c112bb84,0,16c,c0f90477,493e1,...) at _sleep+0x34f kern_nanosleep(cb1292e0,f2967c50,f2967c58,12c,0,...) at kern_nanosleep+0xc1 sys_nanosleep(cb1292e0,f2967cec,c0fe0d8a,c0f98128,246,...) at sys_nanosleep+0x6f syscall(f2967d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x2816e293, esp = 0xbfbfea2c, ebp = 0xbfbfea78 --- Tracing command bash pid 1248 tid 100119 td 0xca8338a0 sched_switch(ca8338a0,0,104,1a3,8e981e75,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(ca8338a0,0,c0f94960,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f94960,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca82a864,6c,c0f978a7,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca82a864,ca82a8ec,16c,c0f978a7,0,...) at _sleep+0x385 kern_wait(ca8338a0,ffffffff,f27a3c60,6,0,...) at kern_wait+0x597 sys_wait4(ca8338a0,f27a3cec,c0fe0d8a,c0f97796,216,...) at sys_wait4+0x3b syscall(f27a3d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x282d566b, esp = 0xbfbfe60c, ebp = 0xbfbfe628 --- Tracing command csh pid 1244 tid 100141 td 0xcab26000 sched_switch(cab26000,0,104,1a3,31c7a985,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,74,...) at mi_switch+0x1ff sleepq_switch(cab26000,0,c0f94960,1a7,74,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f94960,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cad16b88,74,c0e15dae,100,0,...) at sleepq_wait_sig+0x17 _sleep(cad16b88,cad16bb8,174,c0e15dae,0,...) at _sleep+0x385 kern_sigsuspend(cab26000,0,0,0,0,...) at kern_sigsuspend+0xba sys_sigsuspend(cab26000,f2822cec,c0fe0d8a,c0f97796,202,...) at sys_sigsuspend+0x4d syscall(f2822d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28195ecb, esp = 0xbfbfe58c, ebp = 0xbfbfe5f8 --- Tracing command su pid 1243 tid 100155 td 0xcad492e0 sched_switch(cad492e0,0,104,1a3,cb5bd179,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(cad492e0,0,c0f94960,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f94960,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cafa4598,6c,c0f978a7,100,0,...) at sleepq_wait_sig+0x17 _sleep(cafa4598,cafa4620,16c,c0f978a7,0,...) at _sleep+0x385 kern_wait(cad492e0,4dc,f286ec60,2,0,...) at kern_wait+0x597 sys_wait4(cad492e0,f286ecec,c,c,216,...) at sys_wait4+0x3b syscall(f286ed28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2813266b, esp = 0xbfbfe5bc, ebp = 0xbfbfe5d8 --- Tracing command bash pid 1223 tid 100174 td 0xcad32000 sched_switch(cad32000,0,104,1a3,ae78b257,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(cad32000,0,c0f94960,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f94960,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cb0f5b30,6c,c0f978a7,100,0,...) at sleepq_wait_sig+0x17 _sleep(cb0f5b30,cb0f5bb8,16c,c0f978a7,0,...) at _sleep+0x385 kern_wait(cad32000,ffffffff,f28afc60,6,0,...) at kern_wait+0x597 sys_wait4(cad32000,f28afcec,c0fe0d8a,c0f97796,216,...) at sys_wait4+0x3b syscall(f28afd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x282d566b, esp = 0xbfbfe97c, ebp = 0xbfbfe998 --- Tracing command sshd pid 1222 tid 100138 td 0xcab268a0 sched_switch(cab268a0,0,104,1a3,8e82cc10,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cab268a0,0,c0f94960,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b96a,ca84b0d0,0,c0f8d34f,cab268a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca84b0e4,0,f2819a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca84b0e4,ca84b0d0,c0f970b0,6b2,ca7bc690,...) at _cv_wait_sig+0x260 seltdwait(ca7bc690,58,cad3ce80,cab268a0,58,...) at seltdwait+0xa2 kern_select(cab268a0,c,288060b8,288060c0,0,0,20,cad17598,0) at kern_select+0x655 sys_select(cab268a0,f2819cec,c0fe0d8a,c0f979fb,206,...) at sys_select+0x66 syscall(f2819d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284d66d3, esp = 0xbfbfde2c, ebp = 0xbfbfde78 --- Tracing command sshd pid 1220 tid 100095 td 0xca7af2e0 sched_switch(ca7af2e0,0,104,1a3,d322f68b,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca7af2e0,0,c0f94960,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b96a,ca773750,0,c0f8d34f,ca7af2e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca773764,0,f2745abc,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca773764,ca773750,c0f970b0,6b2,f2745b44,...) at _cv_wait_sig+0x260 seltdwait(ca6cc52c,c0f970b0,5af,ca7af2e0,8,...) at seltdwait+0xa2 sys_poll(ca7af2e0,f2745cec,c0fe0d8a,c0f7f839,202,...) at sys_poll+0x321 syscall(f2745d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x2846f3ab, esp = 0xbfbfde8c, ebp = 0xbfbfdec8 --- Tracing command awk pid 1204 tid 100165 td 0xcad362e0 sched_switch(cad362e0,0,104,1a3,80e3e750,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(cad362e0,0,c0f94960,1a7,5c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f94960,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cab21190,5c,c0f972df,100,0,...) at sleepq_wait_sig+0x17 _sleep(cab21190,cab21308,15c,c0f972df,0,...) at _sleep+0x385 pipe_read(ca84e310,f2892c44,c7dc5a80,0,cad362e0,...) at pipe_read+0x417 dofileread(f2892c44,ffffffff,ffffffff,0,f2892c20,...) at dofileread+0x9e kern_readv(cad362e0,0,f2892c44,f2892c64,1,...) at kern_readv+0x68 sys_read(cad362e0,f2892cec,c0fe0d8a,f2892d80,216,...) at sys_read+0x4f syscall(f2892d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281c8753, esp = 0xbfbfe93c, ebp = 0xbfbfe958 --- Tracing command sh pid 1203 tid 100164 td 0xcad365c0 sched_switch(cad365c0,0,104,1a3,80f0121d,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(cad365c0,0,c0f94960,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f94960,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cafa0864,6c,c0f978a7,100,0,...) at sleepq_wait_sig+0x17 _sleep(cafa0864,cafa08ec,16c,c0f978a7,0,...) at _sleep+0x385 kern_wait(cad365c0,ffffffff,f288fc60,2,0,...) at kern_wait+0x597 sys_wait4(cad365c0,f288fcec,c0fe0d8a,c0f97886,216,...) at sys_wait4+0x3b syscall(f288fd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2817666b, esp = 0xbfbfe93c, ebp = 0xbfbfe958 --- Tracing command top pid 1202 tid 100117 td 0xca834000 sched_switch(ca834000,0,104,1a3,60a70751,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca834000,0,c0f94960,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f279ba20,c0a4b96a,ca773c50,0,ca834000,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(ca773c64,0,f279ba58,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(ca773c64,ca773c50,3e9,6b2,ca84eaf0,...) at _cv_timedwait_sig+0x270 seltdwait(f279bc10,f279bc18,cb0b3180,ca834000,325b1b0a,...) at seltdwait+0x8a kern_select(ca834000,2,bfbfebac,0,0,f279bc5c,20,1,0) at kern_select+0x655 sys_select(ca834000,f279bcec,c0fe0d8a,c0f5d447,246,...) at sys_select+0x66 syscall(f279bd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281fd6d3, esp = 0xbfbfeb4c, ebp = 0xbfbfeca8 --- Tracing command sh pid 1201 tid 100105 td 0xca7ab5c0 sched_switch(ca7ab5c0,0,104,1a3,dbf34c6d,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(ca7ab5c0,0,c0f94960,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f94960,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cab1fb30,6c,c0f978a7,100,0,...) at sleepq_wait_sig+0x17 _sleep(cab1fb30,cab1fbb8,16c,c0f978a7,0,...) at _sleep+0x385 kern_wait(ca7ab5c0,ffffffff,f2774c60,2,0,...) at kern_wait+0x597 sys_wait4(ca7ab5c0,f2774cec,c,c,216,...) at sys_wait4+0x3b syscall(f2774d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2817666b, esp = 0xbfbfeb1c, ebp = 0xbfbfeb38 --- Tracing command tail pid 1200 tid 100090 td 0xca6cd8a0 sched_switch(ca6cd8a0,0,104,1a3,a937089d,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,68,...) at mi_switch+0x1ff sleepq_switch(ca6cd8a0,0,c0f94960,1a7,68,...) at sleepq_switch+0x162 sleepq_catch_signals(3e9,c0a9c980,ca6cd8a0,0,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(cb0b3880,68,c0f8904e,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(cb0b3880,cb0b3880,168,c0f8904e,3e9,...) at _sleep+0x34f kern_kevent(ca6cd8a0,4,0,1,f2731c44,...) at kern_kevent+0x374 sys_kevent(ca6cd8a0,f2731cec,c0fe0d8a,c0f5d447,286,...) at sys_kevent+0x19e syscall(f2731d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (363, FreeBSD ELF32, sys_kevent), eip = 0x281721ab, esp = 0xbfbfeb1c, ebp = 0xbfbfebd8 --- Tracing command sshd pid 1199 tid 100099 td 0xc81858a0 sched_switch(c81858a0,0,104,1a3,ac8035e8,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c81858a0,0,c0f94960,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b96a,ca6c0410,0,c0f8d34f,c81858a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca6c0424,0,f275da58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca6c0424,ca6c0410,c0f970b0,6b2,ca7de268,...) at _cv_wait_sig+0x260 seltdwait(ca7de268,58,c7dc5b80,c81858a0,cac18910,...) at seltdwait+0xa2 kern_select(c81858a0,c,288060b8,288060c0,0,0,20,ca818598,0) at kern_select+0x655 sys_select(c81858a0,f275dcec,c0fe0d8a,c0f979fb,206,...) at sys_select+0x66 syscall(f275dd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284d66d3, esp = 0xbfbfde2c, ebp = 0xbfbfde78 --- Tracing command sshd pid 1198 tid 100163 td 0xcad368a0 sched_switch(cad368a0,0,104,1a3,a93884ec,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cad368a0,0,c0f94960,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b96a,ca773810,0,c0f8d34f,cad368a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca773824,0,f288ca58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca773824,ca773810,c0f970b0,6b2,ca84ed58,...) at _cv_wait_sig+0x260 seltdwait(ca84ed58,58,cb0b3000,cad368a0,f288cacc,...) at seltdwait+0xa2 kern_select(cad368a0,c,288060b8,288060c0,0,0,20,cafa0b30,0) at kern_select+0x655 sys_select(cad368a0,f288ccec,c0fe0d8a,c0f979fb,206,...) at sys_select+0x66 syscall(f288cd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284d66d3, esp = 0xbfbfde2c, ebp = 0xbfbfde78 --- Tracing command sshd pid 1197 tid 100100 td 0xc81855c0 sched_switch(c81855c0,0,104,1a3,60a915c3,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c81855c0,0,c0f94960,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b96a,ca84b410,0,c0f8d34f,c81855c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca84b424,0,f2763a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca84b424,ca84b410,c0f970b0,6b2,ca7bb0e0,...) at _cv_wait_sig+0x260 seltdwait(ca7bb0e0,58,ca776b00,c81855c0,f2763acc,...) at seltdwait+0xa2 kern_select(c81855c0,a,288060b8,288060c0,0,0,20,ca8182cc,0) at kern_select+0x655 sys_select(c81855c0,f2763cec,c0fe0d8a,c0f979fb,206,...) at sys_select+0x66 syscall(f2763d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284d66d3, esp = 0xbfbfde2c, ebp = 0xbfbfde78 --- Tracing command sshd pid 1193 tid 100107 td 0xca7ab000 sched_switch(ca7ab000,0,104,1a3,ac56a566,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca7ab000,0,c0f94960,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b96a,ca6bf250,0,c0f8d34f,ca7ab000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca6bf264,0,f277aabc,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca6bf264,ca6bf250,c0f970b0,6b2,f277ab44,...) at _cv_wait_sig+0x260 seltdwait(ca6ccc2c,c0f970b0,5af,ca7ab000,0,...) at seltdwait+0xa2 sys_poll(ca7ab000,f277acec,c0fe0d8a,c0def423,202,...) at sys_poll+0x321 syscall(f277ad28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x2846f3ab, esp = 0xbfbfde8c, ebp = 0xbfbfdec8 --- Tracing command sshd pid 1192 tid 100115 td 0xca8345c0 sched_switch(ca8345c0,0,104,1a3,ac4a5701,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca8345c0,0,c0f94960,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b96a,ca84bbd0,0,c0f8d34f,ca8345c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca84bbe4,0,f2793abc,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca84bbe4,ca84bbd0,c0f970b0,6b2,f2793b44,...) at _cv_wait_sig+0x260 seltdwait(ca78662c,c0f970b0,5af,ca8345c0,0,...) at seltdwait+0xa2 sys_poll(ca8345c0,f2793cec,c0fe0d8a,c0def423,202,...) at sys_poll+0x321 syscall(f2793d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x2846f3ab, esp = 0xbfbfde8c, ebp = 0xbfbfdec8 --- Tracing command sshd pid 1191 tid 100091 td 0xca7afb80 sched_switch(ca7afb80,0,104,1a3,b0ef4cc5,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca7afb80,0,c0f94960,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b96a,ca6bf7d0,0,c0f8d34f,ca7afb80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca6bf7e4,0,f2735abc,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca6bf7e4,ca6bf7d0,c0f970b0,6b2,f2735b44,...) at _cv_wait_sig+0x260 seltdwait(cab6b72c,c0f970b0,5af,ca7afb80,0,...) at seltdwait+0xa2 sys_poll(ca7afb80,f2735cec,c,c,202,...) at sys_poll+0x321 syscall(f2735d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x2846f3ab, esp = 0xbfbfde8c, ebp = 0xbfbfdec8 --- Tracing command cron pid 1154 tid 100101 td 0xc81852e0 sched_switch(c81852e0,0,104,1a3,a63e1d6f,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(c81852e0,0,c0f94960,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(ea61,c0a9c980,c81852e0,0,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c112bb84,6c,c0f90477,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c112bb84,0,16c,c0f90477,ea61,...) at _sleep+0x34f kern_nanosleep(c81852e0,f2767c50,f2767c58,3c,0,...) at kern_nanosleep+0xc1 sys_nanosleep(c81852e0,f2767cec,c0fe0d8a,c0f97d54,206,...) at sys_nanosleep+0x6f syscall(f2767d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x28190293, esp = 0xbfbfec4c, ebp = 0xbfbfec78 --- Tracing command sendmail pid 1149 tid 100088 td 0xca834b80 sched_switch(ca834b80,0,104,1a3,651dbbc1,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,74,...) at mi_switch+0x1ff sleepq_switch(ca834b80,0,c0f94960,1a7,74,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f94960,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca82cb88,74,c0e15dae,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca82cb88,ca82cbb8,174,c0e15dae,0,...) at _sleep+0x385 kern_sigsuspend(ca834b80,0,0,0,0,...) at kern_sigsuspend+0xba sys_sigsuspend(ca834b80,f2729cec,c0fe0d8a,c0f97796,202,...) at sys_sigsuspend+0x4d syscall(f2729d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28349ecb, esp = 0xbfbfcf9c, ebp = 0xbfbfcfc8 --- Tracing command sendmail pid 1146 tid 100143 td 0xca8358a0 sched_switch(ca8358a0,0,104,1a3,3ba270a9,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca8358a0,0,c0f94960,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f2828a20,c0a4b96a,ca6c0250,0,ca8358a0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(ca6c0264,0,f2828a58,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(ca6c0264,ca6c0250,1389,6b2,cab4e0e0,...) at _cv_timedwait_sig+0x270 seltdwait(f2828c10,f2828c18,cad3c600,ca8358a0,c0a481ad,...) at seltdwait+0x8a kern_select(ca8358a0,5,bfbfc510,0,0,f2828c5c,20,5,0) at kern_select+0x655 sys_select(ca8358a0,f2828cec,c0fe0d8a,c0f97cb3,206,...) at sys_select+0x66 syscall(f2828d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x283f46d3, esp = 0xbfbfc47c, ebp = 0xbfbfcfa8 --- Tracing command sshd pid 1143 tid 100081 td 0xca6ce2e0 sched_switch(ca6ce2e0,0,104,1a3,d0ba34d2,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca6ce2e0,0,c0f94960,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b96a,ca6bf310,0,c0f8d34f,ca6ce2e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca6bf324,0,f2707a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca6bf324,ca6bf310,c0f970b0,6b2,ca7bb620,...) at _cv_wait_sig+0x260 seltdwait(ca7bb620,58,cad3c500,ca6ce2e0,ca7a5bc8,...) at seltdwait+0xa2 kern_select(ca6ce2e0,9,2880c0b8,0,0,0,20,c8042864,a) at kern_select+0x655 sys_select(ca6ce2e0,f2707cec,c,c,206,...) at sys_select+0x66 syscall(f2707d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284d66d3, esp = 0xbfbfdeec, ebp = 0xbfbfedf8 --- Tracing command moused pid 1112 tid 100097 td 0xca6cd000 sched_switch(ca6cd000,0,104,1a3,e1769d7a,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca6cd000,0,c0f94960,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b96a,ca6c0650,0,c0f8d34f,ca6cd000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca6c0664,0,f2753a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca6c0664,ca6c0650,c0f970b0,6b2,ca7bc4d0,...) at _cv_wait_sig+0x260 seltdwait(ca7bc4d0,58,c7dc4e80,ca6cd000,f6,...) at seltdwait+0xa2 kern_select(ca6cd000,400,bfbfea04,0,0,0,20,ca818b30,1) at kern_select+0x655 sys_select(ca6cd000,f2753cec,c0fe0d8a,c0df4156,246,...) at sys_select+0x66 syscall(f2753d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281c56d3, esp = 0xbfbfe99c, ebp = 0xbfbfeb28 --- Tracing command ntpd pid 1092 tid 100140 td 0xcab262e0 sched_switch(cab262e0,0,104,1a3,9cf27abb,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cab262e0,0,c0f94960,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b96a,ca6c0c10,0,c0f8d34f,cab262e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca6c0c24,0,f281fa58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca6c0c24,ca6c0c10,c0f970b0,6b2,ca7dee00,...) at _cv_wait_sig+0x260 seltdwait(ca7dee00,58,c7dc4e80,cab262e0,0,...) at seltdwait+0xa2 kern_select(cab262e0,1c,bfbfece8,0,0,0,20,cad17000,0) at kern_select+0x655 sys_select(cab262e0,f281fcec,c0fe0d8a,f281fd80,202,...) at sys_select+0x66 syscall(f281fd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x283736d3, esp = 0xbfbfecbc, ebp = 0xbfbfed88 --- Tracing command nfsd pid 1038 tid 100152 td 0xcad49b80 sched_switch(cad49b80,0,104,1a3,fbcc515e,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cad49b80,0,c0f94960,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f2846be0,c0a4b96a,c800f900,0,cad49b80,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(cad40190,0,f2846c18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(cad40190,c800f900,1388,3b1,0,...) at _cv_timedwait_sig+0x270 svc_run_internal(f2846d14,c0a2e4a8,c800f900,f2846d28,c0f89490,...) at svc_run_internal+0x356 svc_thread_start(c800f900,f2846d28,c0f89490,3dc,ca82a2cc,...) at svc_thread_start+0x10 fork_exit(c0c4c630,c800f900,f2846d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x2e, eip = 0xc, esp = 0x33, ebp = 0x1 --- Tracing command nfsd pid 1038 tid 100151 td 0xcad4a000 sched_switch(cad4a000,0,104,1a3,fbcc36c5,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cad4a000,0,c0f94960,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f2843be0,c0a4b96a,c800f900,0,cad4a000,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(cad401d0,0,f2843c18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(cad401d0,c800f900,1388,3b1,0,...) at _cv_timedwait_sig+0x270 svc_run_internal(f2843d14,c0a2e4a8,c800f900,f2843d28,c0f89490,...) at svc_run_internal+0x356 svc_thread_start(c800f900,f2843d28,c0f89490,3dc,ca82a2cc,...) at svc_thread_start+0x10 fork_exit(c0c4c630,c800f900,f2843d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x2e, eip = 0xc, esp = 0x33, ebp = 0x1 --- Tracing command nfsd pid 1038 tid 100150 td 0xcad4a2e0 sched_switch(cad4a2e0,0,104,1a3,fc030592,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cad4a2e0,0,c0f94960,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f2840be0,c0a4b96a,c800f900,0,cad4a2e0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(cad40210,0,f2840c18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(cad40210,c800f900,1388,3b1,0,...) at _cv_timedwait_sig+0x270 svc_run_internal(f2840d14,c0a2e4a8,c800f900,f2840d28,c0f89490,...) at svc_run_internal+0x356 svc_thread_start(c800f900,f2840d28,c0f89490,3dc,ca82a2cc,...) at svc_thread_start+0x10 fork_exit(c0c4c630,c800f900,f2840d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x2e, eip = 0xc, esp = 0x33, ebp = 0x1 --- Tracing command nfsd pid 1038 tid 100121 td 0xca8332e0 sched_switch(ca8332e0,0,104,1a3,fbe78fbb,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca8332e0,0,c0f94960,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f27ab498,c0a4b96a,c800f900,0,ca8332e0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(ca7665d0,0,f27ab4d0,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(ca7665d0,c800f900,1388,3b1,f27ab518,...) at _cv_timedwait_sig+0x270 svc_run_internal(ca833490,14,c0fbe1ba,c0f80d1b,f27ab810,...) at svc_run_internal+0x356 svc_run(c800f900,0,c0f80cf6,19f,e18eb340,...) at svc_run+0x7b nfsrvd_nfsd(ca8332e0,f27ab810,c,c11481d8,c1274b00,...) at nfsrvd_nfsd+0x1c3 nfssvc_nfsd(ca8332e0,f27abcec,bfbfe870,ca8332e0,f27abd1c,...) at nfssvc_nfsd+0x177 sys_nfssvc(ca8332e0,f27abcec,c0fe0d8a,c0f97cb3,206,...) at sys_nfssvc+0xeb syscall(f27abd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (155, FreeBSD ELF32, sys_nfssvc), eip = 0x280e8b0b, esp = 0xbfbfe82c, ebp = 0xbfbfea98 --- Tracing command nfsd pid 1032 tid 100136 td 0xcab27000 sched_switch(cab27000,0,104,1a3,fcbcd34b,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cab27000,0,c0f94960,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b96a,ca7739d0,0,c0f8d34f,cab27000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca7739e4,0,f2813a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca7739e4,ca7739d0,c0f970b0,6b2,ca7de578,...) at _cv_wait_sig+0x260 seltdwait(ca7de578,58,c7dc4e80,cab27000,ca837910,...) at seltdwait+0xa2 kern_select(cab27000,7,bfbfecb4,0,0,0,20,cad17b30,1) at kern_select+0x655 sys_select(cab27000,f2813cec,c,c,202,...) at sys_select+0x66 syscall(f2813d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281926d3, esp = 0xbfbfea9c, ebp = 0xbfbfede8 --- Tracing command nfsuserd pid 1030 tid 100082 td 0xca7b08a0 sched_switch(ca7b08a0,0,104,1a3,f87a2d94,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca7b08a0,0,c0f94960,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f270fa20,c0a4b96a,ca773d50,0,ca7b08a0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(ca773d64,0,f270fa58,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(ca773d64,ca773d50,7531,6b2,ca84e8c0,...) at _cv_timedwait_sig+0x270 seltdwait(f270fc10,f270fc18,c7dc4e80,ca7b08a0,ca7a5d98,...) at seltdwait+0x8a kern_select(ca7b08a0,4,bfbfebc4,0,0,f270fc5c,20,1e,0) at kern_select+0x655 sys_select(ca7b08a0,f270fcec,c0fe0d8a,c0f979e4,202,...) at sys_select+0x66 syscall(f270fd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281916d3, esp = 0xbfbfeb1c, ebp = 0xbfbfec58 --- Tracing command nfsuserd pid 1029 tid 100113 td 0xcab272e0 sched_switch(cab272e0,0,104,1a3,f879dba2,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cab272e0,0,c0f94960,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f278ca20,c0a4b96a,ca6c0390,0,cab272e0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(ca6c03a4,0,f278ca58,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(ca6c03a4,ca6c0390,7531,6b2,ca84e8c0,...) at _cv_timedwait_sig+0x270 seltdwait(f278cc10,f278cc18,c7dc4e80,cab272e0,c7dcb3a0,...) at seltdwait+0x8a kern_select(cab272e0,4,bfbfebc4,0,0,f278cc5c,20,1e,0) at kern_select+0x655 sys_select(cab272e0,f278ccec,c0fe0d8a,c0f979e4,202,...) at sys_select+0x66 syscall(f278cd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281916d3, esp = 0xbfbfeb1c, ebp = 0xbfbfec58 --- Tracing command nfsuserd pid 1028 tid 100098 td 0xc8185b80 sched_switch(c8185b80,0,104,1a3,f87949f9,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8185b80,0,c0f94960,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f2759a20,c0a4b96a,ca773d90,0,c8185b80,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(ca773da4,0,f2759a58,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(ca773da4,ca773d90,7531,6b2,ca84e8c0,...) at _cv_timedwait_sig+0x270 seltdwait(f2759c10,f2759c18,c7dc4e80,c8185b80,c7dcb488,...) at seltdwait+0x8a kern_select(c8185b80,4,bfbfebc4,0,0,f2759c5c,20,1e,0) at kern_select+0x655 sys_select(c8185b80,f2759cec,c0fe0d8a,c0f979e4,202,...) at sys_select+0x66 syscall(f2759d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281916d3, esp = 0xbfbfeb1c, ebp = 0xbfbfec58 --- Tracing command nfsuserd pid 1027 tid 100084 td 0xca7b02e0 sched_switch(ca7b02e0,0,104,1a3,f8791b40,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca7b02e0,0,c0f94960,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f2718a20,c0a4b96a,ca6c0350,0,ca7b02e0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(ca6c0364,0,f2718a58,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(ca6c0364,ca6c0350,7531,6b2,ca84e8c0,...) at _cv_timedwait_sig+0x270 seltdwait(f2718c10,f2718c18,c7dc4e80,ca7b02e0,ca837bc8,...) at seltdwait+0x8a kern_select(ca7b02e0,4,bfbfebc4,0,0,f2718c5c,20,1e,0) at kern_select+0x655 sys_select(ca7b02e0,f2718cec,c0fe0d8a,c0f979e4,202,...) at sys_select+0x66 syscall(f2718d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281916d3, esp = 0xbfbfeb1c, ebp = 0xbfbfec58 --- Tracing command nfsuserd pid 1026 tid 100137 td 0xcab26b80 sched_switch(cab26b80,0,104,1a3,f4d9e206,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,74,...) at mi_switch+0x1ff sleepq_switch(cab26b80,0,c0f94960,1a7,74,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f94960,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cad178bc,74,c0e15dae,100,0,...) at sleepq_wait_sig+0x17 _sleep(cad178bc,cad178ec,174,c0e15dae,0,...) at _sleep+0x385 kern_sigsuspend(cab26b80,0,0,0,0,...) at kern_sigsuspend+0xba sys_sigsuspend(cab26b80,f2816cec,c,c,246,...) at sys_sigsuspend+0x4d syscall(f2816d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x280e6ecb, esp = 0xbfbfec5c, ebp = 0xbfbfedf8 --- Tracing command mountd pid 1010 tid 100142 td 0xca835b80 sched_switch(ca835b80,0,104,1a3,8ebb51c2,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca835b80,0,c0f94960,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b96a,ca6bf9d0,0,c0f8d34f,ca835b80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca6bf9e4,0,f2825a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca6bf9e4,ca6bf9d0,c0f970b0,6b2,ca7bca10,...) at _cv_wait_sig+0x260 seltdwait(ca7bca10,58,c7dc4e80,ca835b80,c1274980,...) at seltdwait+0xa2 kern_select(ca835b80,a,bfbfed04,0,0,0,20,cad16864,bfbfed04) at kern_select+0x655 sys_select(ca835b80,f2825cec,c0fe0d8a,c0f7ead7,206,...) at sys_select+0x66 syscall(f2825d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281aa6d3, esp = 0xbfbfec9c, ebp = 0xbfbfede8 --- Tracing command rpcbind pid 977 tid 100122 td 0xca833000 sched_switch(ca833000,0,104,1a3,698e1e,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca833000,0,c0f94960,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f27afa84,c0a4b96a,ca773dd0,0,ca833000,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(ca773de4,0,f27afabc,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(ca773de4,ca773dd0,7531,6b2,f27afb74,...) at _cv_timedwait_sig+0x270 seltdwait(f27afc44,f27afc4c,5af,ca833000,c7d75a58,...) at seltdwait+0x8a sys_poll(ca833000,f27afcec,c0fe0d8a,c0f979e4,212,...) at sys_poll+0x321 syscall(f27afd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x2814a3ab, esp = 0xbfbfcbac, ebp = 0xbfbfed78 --- Tracing command syslogd pid 962 tid 100093 td 0xca6cd5c0 sched_switch(ca6cd5c0,0,104,1a3,a9cf9da4,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca6cd5c0,0,c0f94960,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b96a,ca6bfd90,0,c0f8d34f,ca6cd5c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca6bfda4,0,f273da58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca6bfda4,ca6bfd90,c0f970b0,6b2,ca7bb310,...) at _cv_wait_sig+0x260 seltdwait(ca7bb310,58,c7dc4e80,ca6cd5c0,f273daf4,...) at seltdwait+0xa2 kern_select(ca6cd5c0,9,2847f0ac,0,0,0,20,ca8192cc,0) at kern_select+0x655 sys_select(ca6cd5c0,f273dcec,c0fe0d8a,c0fc39ac,246,...) at sys_select+0x66 syscall(f273dd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281a96d3, esp = 0xbfbfe29c, ebp = 0xbfbfede8 --- Tracing command devd pid 839 tid 100110 td 0xcab27b80 sched_switch(cab27b80,0,104,1a3,d2fb0283,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cab27b80,0,c0f94960,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4b96a,ca6bf590,0,c0f8d34f,cab27b80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca6bf5a4,0,f2783a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca6bf5a4,ca6bf590,c0f970b0,6b2,ca7dec40,...) at _cv_wait_sig+0x260 seltdwait(ca7dec40,58,c7dc4e80,cab27b80,c1274af0,...) at seltdwait+0xa2 kern_select(cab27b80,5,bfbfe960,0,0,0,20,ca82db30,34) at kern_select+0x655 sys_select(cab27b80,f2783cec,c0fe0d8a,c0def423,206,...) at sys_select+0x66 syscall(f2783d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x809169b, esp = 0xbfbfe92c, ebp = 0xbfbfee08 --- Tracing command sh pid 20 tid 100079 td 0xca7b0b80 sched_switch(ca7b0b80,0,104,1a3,b624525f,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(ca7b0b80,0,c0f94960,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f94960,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca7a9b30,6c,c0f978a7,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca7a9b30,ca7a9bb8,16c,c0f978a7,0,...) at _sleep+0x385 kern_wait(ca7b0b80,ffffffff,f26ffc60,2,0,...) at kern_wait+0x597 sys_wait4(ca7b0b80,f26ffcec,c0fe0d8a,c0f97886,216,...) at sys_wait4+0x3b syscall(f26ffd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2817666b, esp = 0xbfbfeabc, ebp = 0xbfbfead8 --- Tracing command softdepflush pid 19 tid 100077 td 0xca6ceb80 sched_switch(ca6ceb80,0,104,1a3,7a499aeb,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,54,...) at mi_switch+0x1ff sleepq_switch(ca6ceb80,0,c0f94960,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c12a8080,54,c0fc521d,0,0,...) at sleepq_timedwait+0x6b _sleep(c12a8080,c12a8040,54,c0fc521d,3e8,...) at _sleep+0x36a softdep_flush(0,ee320d28,c0f89490,3dc,ca6c1000,...) at softdep_flush+0x266 fork_exit(c0c9bad0,0,ee320d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee320d60, ebp = 0 --- Tracing command vnlru pid 18 tid 100076 td 0xc805e8a0 sched_switch(c805e8a0,0,104,1a3,4fe864e7,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,60,...) at mi_switch+0x1ff sleepq_switch(c805e8a0,0,c0f94960,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(ca6c12cc,60,c0f9f979,0,0,...) at sleepq_timedwait+0x6b _sleep(ca6c12cc,c12a1d34,260,c0f9f979,3e8,...) at _sleep+0x36a vnlru_proc(0,ee31dd28,c0f89490,3dc,ca6c12cc,...) at vnlru_proc+0xe7 fork_exit(c0afd940,0,ee31dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee31dd60, ebp = 0 --- Tracing command syncer pid 17 tid 100075 td 0xc805eb80 sched_switch(c805eb80,0,104,1a3,5260485b,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c805eb80,0,c0f94960,28b,c805eb80,...) at sleepq_switch+0x162 sleepq_timedwait(c12a1d74,0,ee31ac78,1,0,...) at sleepq_timedwait+0x6b _cv_timedwait(c12a1d74,c12a1d60,3e8,78b,4e20,...) at _cv_timedwait+0x270 sched_sync(0,ee31ad28,c0f89490,3dc,ca6c1598,...) at sched_sync+0x595 fork_exit(c0afccb0,0,ee31ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee31ad60, ebp = 0 --- Tracing command bufdaemon pid 16 tid 100074 td 0xc8183000 sched_switch(c8183000,0,104,1a3,4fe8b04a,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,54,...) at mi_switch+0x1ff sleepq_switch(c8183000,0,c0f94960,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c12a1a08,54,c0f9c8ad,0,0,...) at sleepq_timedwait+0x6b _sleep(c12a1a08,c12a1a10,54,c0f9c8ad,3e8,...) at _sleep+0x36a buf_daemon(0,ee317d28,c0f89490,3dc,ca6c1864,...) at buf_daemon+0x175 fork_exit(c0ae1470,0,ee317d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee317d60, ebp = 0 --- Tracing command pagezero pid 9 tid 100073 td 0xc81832e0 sched_switch(c81832e0,0,104,1a3,4594cb04,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c81832e0,0,c0f94960,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c12a9f5c,0,c0fcc2e6,0,0,...) at sleepq_timedwait+0x6b _sleep(c12a9f5c,c12a8a00,0,c0fcc2e6,493e0,...) at _sleep+0x36a vm_pagezero(0,ee314d28,c0f89490,3dc,ca6c1b30,...) at vm_pagezero+0xdc fork_exit(c0ce2f80,0,ee314d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee314d60, ebp = 0 --- Tracing command vmdaemon pid 8 tid 100072 td 0xc81835c0 sched_switch(c81835c0,0,104,1a3,458259e3,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,74,...) at mi_switch+0x1ff sleepq_switch(c81835c0,0,c0f94960,268,0,...) at sleepq_switch+0x162 sleepq_wait(c12a9b48,74,c0f9c8ad,0,0,...) at sleepq_wait+0x63 _sleep(c12a9b48,c12a9b4c,74,c0f9c8ad,0,...) at _sleep+0x39c vm_daemon(0,ee311d28,c0f89490,3dc,ca6c2000,...) at vm_daemon+0x59 fork_exit(c0cdc890,0,ee311d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee311d60, ebp = 0 --- Tracing command pagedaemon pid 7 tid 100071 td 0xc81838a0 sched_switch(c81838a0,0,104,1a3,79535d01,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,54,...) at mi_switch+0x1ff sleepq_switch(c81838a0,0,c0f94960,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c12a9b10,54,c0f9c8ad,0,0,...) at sleepq_timedwait+0x6b _sleep(c12a9b10,c12a8a00,54,c0f9c8ad,1388,...) at _sleep+0x36a vm_pageout(0,ee30ed28,c0f89490,3dc,c7dc82cc,...) at vm_pageout+0x2c7 fork_exit(c0cdd950,0,ee30ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee30ed60, ebp = 0 --- Tracing command xpt_thrd pid 6 tid 100070 td 0xc8183b80 sched_switch(c8183b80,0,104,1a3,bbf95f0e,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c8183b80,0,c0f94960,268,0,...) at sleepq_switch+0x162 sleepq_wait(c10f0354,5c,c0de9c27,0,0,...) at sleepq_wait+0x63 _sleep(c10f0354,c10f0398,5c,c0de9c27,0,...) at _sleep+0x39c xpt_scanner_thread(0,ee18bd28,c0f89490,3dc,c7dc8598,...) at xpt_scanner_thread+0x47 fork_exit(c049d6b0,0,ee18bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee18bd60, ebp = 0 --- Tracing command sctp_iterator pid 5 tid 100069 td 0xc8184000 sched_switch(c8184000,0,104,1a3,112efe61,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8184000,0,c0f94960,268,0,...) at sleepq_switch+0x162 sleepq_wait(c12a3898,0,c0fb301f,0,0,...) at sleepq_wait+0x63 _sleep(c12a3898,c12a3860,0,c0fb301f,0,...) at _sleep+0x39c sctp_iterator_thread(0,ee188d28,c0f89490,3dc,c7dc8864,...) at sctp_iterator_thread+0x5c fork_exit(c0b8f310,0,ee188d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee188d60, ebp = 0 --- Tracing command ctl_thrd pid 4 tid 100067 td 0xc81845c0 sched_switch(c81845c0,0,104,1a3,a9aa50b6,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c81845c0,0,c0f94960,268,0,...) at sleepq_switch+0x162 sleepq_wait(c819a000,5c,c0df4632,0,0,...) at sleepq_wait+0x63 _sleep(c819a000,c819a000,5c,c0df4632,0,...) at _sleep+0x39c ctl_work_thread(c819a000,ee182d28,c0f89490,3dc,c7dc8b30,...) at ctl_work_thread+0x22f6 fork_exit(c04c6ad0,c819a000,ee182d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee182d60, ebp = 0 --- Tracing command fdc0 pid 3 tid 100064 td 0xc8185000 sched_switch(c8185000,0,104,1a3,82decc92,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c8185000,0,c0f94960,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c7ffa03c,5c,c0f81792,0,0,...) at sleepq_timedwait+0x6b _sleep(c7ffa03c,c7ffa0f0,5c,c0f81792,3e8,...) at _sleep+0x36a fdc_thread(c7ffa000,ee179d28,c0f89490,3dc,c8042000,...) at fdc_thread+0x27d fork_exit(c0d62550,c7ffa000,ee179d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee179d60, ebp = 0 --- Tracing command fw0_probe pid 2 tid 100062 td 0xc80515c0 sched_switch(c80515c0,0,104,1a3,bbf99802,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(c80515c0,0,c0f94960,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f94960,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c808f000,6c,c0f81792,100,0,...) at sleepq_wait_sig+0x17 _sleep(c808f000,c8093488,16c,c0f81792,0,...) at _sleep+0x385 fw_bus_probe_thread(c808f000,ee169d28,c0f89490,3dc,c80422cc,...) at fw_bus_probe_thread+0xa08 fork_exit(c06f1180,c808f000,ee169d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee169d60, ebp = 0 --- Tracing command usb pid 15 tid 100058 td 0xc805c2e0 sched_switch(c805c2e0,0,104,1a3,a9a56e16,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c805c2e0,0,c0f94960,268,c805c2e0,...) at sleepq_switch+0x162 sleepq_wait(c8063d34,0,ee152cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8063d34,c8063e4c,c0f67e51,74,c8063d3c,...) at _cv_wait+0x260 usb_process(c8063d2c,ee152d28,c0f89490,3dc,c8042598,...) at usb_process+0x1a3 fork_exit(c08fe340,c8063d2c,ee152d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee152d60, ebp = 0 --- Tracing command usb pid 15 tid 100057 td 0xc805c5c0 sched_switch(c805c5c0,0,104,1a3,b05e3386,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c805c5c0,0,c0f94960,268,c805c5c0,...) at sleepq_switch+0x162 sleepq_wait(c8063d04,0,ee14fcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8063d04,c8063e4c,c0f67e51,74,c8063d0c,...) at _cv_wait+0x260 usb_process(c8063cfc,ee14fd28,c0f89490,3dc,c8042598,...) at usb_process+0x1a3 fork_exit(c08fe340,c8063cfc,ee14fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee14fd60, ebp = 0 --- Tracing command usb pid 15 tid 100056 td 0xc805c8a0 sched_switch(c805c8a0,0,104,1a3,a93efe8a,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c805c8a0,0,c0f94960,268,c805c8a0,...) at sleepq_switch+0x162 sleepq_wait(c8063cd4,0,ee14ccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8063cd4,c8063e4c,c0f67e51,74,c8063cdc,...) at _cv_wait+0x260 usb_process(c8063ccc,ee14cd28,c0f89490,3dc,c8042598,...) at usb_process+0x1a3 fork_exit(c08fe340,c8063ccc,ee14cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee14cd60, ebp = 0 --- Tracing command usb pid 15 tid 100055 td 0xc805cb80 sched_switch(c805cb80,0,104,1a3,a93eec2a,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c805cb80,0,c0f94960,268,c805cb80,...) at sleepq_switch+0x162 sleepq_wait(c8063ca4,0,ee149cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8063ca4,c8063e4c,c0f67e51,74,c8063cac,...) at _cv_wait+0x260 usb_process(c8063c9c,ee149d28,c0f89490,3dc,c8042598,...) at usb_process+0x1a3 fork_exit(c08fe340,c8063c9c,ee149d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee149d60, ebp = 0 --- Tracing command usb pid 15 tid 100053 td 0xc805e2e0 sched_switch(c805e2e0,0,104,1a3,a93ed800,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c805e2e0,0,c0f94960,268,c805e2e0,...) at sleepq_switch+0x162 sleepq_wait(c8056b5c,0,ee140cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8056b5c,c8056c74,c0f67e51,74,c8056b64,...) at _cv_wait+0x260 usb_process(c8056b54,ee140d28,c0f89490,3dc,c8042598,...) at usb_process+0x1a3 fork_exit(c08fe340,c8056b54,ee140d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee140d60, ebp = 0 --- Tracing command usb pid 15 tid 100052 td 0xc805e5c0 sched_switch(c805e5c0,0,104,1a3,ac67e498,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c805e5c0,0,c0f94960,268,c805e5c0,...) at sleepq_switch+0x162 sleepq_wait(c8056b2c,0,ee13dcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8056b2c,c8056c74,c0f67e51,74,c8056b34,...) at _cv_wait+0x260 usb_process(c8056b24,ee13dd28,c0f89490,3dc,c8042598,...) at usb_process+0x1a3 fork_exit(c08fe340,c8056b24,ee13dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee13dd60, ebp = 0 --- Tracing command usb pid 15 tid 100051 td 0xc804e000 sched_switch(c804e000,0,104,1a3,a8db1349,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804e000,0,c0f94960,268,c804e000,...) at sleepq_switch+0x162 sleepq_wait(c8056afc,0,ee13acac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8056afc,c8056c74,c0f67e51,74,c8056b04,...) at _cv_wait+0x260 usb_process(c8056af4,ee13ad28,c0f89490,3dc,c8042598,...) at usb_process+0x1a3 fork_exit(c08fe340,c8056af4,ee13ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee13ad60, ebp = 0 --- Tracing command usb pid 15 tid 100050 td 0xc804e2e0 sched_switch(c804e2e0,0,104,1a3,a8db020f,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804e2e0,0,c0f94960,268,c804e2e0,...) at sleepq_switch+0x162 sleepq_wait(c8056acc,0,ee137cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8056acc,c8056c74,c0f67e51,74,c8056ad4,...) at _cv_wait+0x260 usb_process(c8056ac4,ee137d28,c0f89490,3dc,c8042598,...) at usb_process+0x1a3 fork_exit(c08fe340,c8056ac4,ee137d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee137d60, ebp = 0 --- Tracing command usb pid 15 tid 100049 td 0xc804e5c0 sched_switch(c804e5c0,0,104,1a3,a8daed5c,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804e5c0,0,c0f94960,268,c804e5c0,...) at sleepq_switch+0x162 sleepq_wait(c8052b5c,0,ee133cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8052b5c,c8052c74,c0f67e51,74,c8052b64,...) at _cv_wait+0x260 usb_process(c8052b54,ee133d28,c0f89490,3dc,c8042598,...) at usb_process+0x1a3 fork_exit(c08fe340,c8052b54,ee133d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee133d60, ebp = 0 --- Tracing command usb pid 15 tid 100048 td 0xc804e8a0 sched_switch(c804e8a0,0,104,1a3,ac678938,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804e8a0,0,c0f94960,268,c804e8a0,...) at sleepq_switch+0x162 sleepq_wait(c8052b2c,0,ee130cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8052b2c,c8052c74,c0f67e51,74,c8052b34,...) at _cv_wait+0x260 usb_process(c8052b24,ee130d28,c0f89490,3dc,c8042598,...) at usb_process+0x1a3 fork_exit(c08fe340,c8052b24,ee130d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee130d60, ebp = 0 --- Tracing command usb pid 15 tid 100047 td 0xc804eb80 sched_switch(c804eb80,0,104,1a3,a316d801,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804eb80,0,c0f94960,268,c804eb80,...) at sleepq_switch+0x162 sleepq_wait(c8052afc,0,ee12dcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8052afc,c8052c74,c0f67e51,74,c8052b04,...) at _cv_wait+0x260 usb_process(c8052af4,ee12dd28,c0f89490,3dc,c8042598,...) at usb_process+0x1a3 fork_exit(c08fe340,c8052af4,ee12dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee12dd60, ebp = 0 --- Tracing command usb pid 15 tid 100046 td 0xc804f000 sched_switch(c804f000,0,104,1a3,a316c54f,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804f000,0,c0f94960,268,c804f000,...) at sleepq_switch+0x162 sleepq_wait(c8052acc,0,ee12acac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8052acc,c8052c74,c0f67e51,74,c8052ad4,...) at _cv_wait+0x260 usb_process(c8052ac4,ee12ad28,c0f89490,3dc,c8042598,...) at usb_process+0x1a3 fork_exit(c08fe340,c8052ac4,ee12ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee12ad60, ebp = 0 --- Tracing command usb pid 15 tid 100045 td 0xc804f2e0 sched_switch(c804f2e0,0,104,1a3,a316b1d4,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804f2e0,0,c0f94960,268,c804f2e0,...) at sleepq_switch+0x162 sleepq_wait(c804ab5c,0,ee126cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c804ab5c,c804ac74,c0f67e51,74,c804ab64,...) at _cv_wait+0x260 usb_process(c804ab54,ee126d28,c0f89490,3dc,c8042598,...) at usb_process+0x1a3 fork_exit(c08fe340,c804ab54,ee126d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee126d60, ebp = 0 --- Tracing command usb pid 15 tid 100044 td 0xc804f5c0 sched_switch(c804f5c0,0,104,1a3,ac671af5,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804f5c0,0,c0f94960,268,c804f5c0,...) at sleepq_switch+0x162 sleepq_wait(c804ab2c,0,ee123cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c804ab2c,c804ac74,c0f67e51,74,c804ab34,...) at _cv_wait+0x260 usb_process(c804ab24,ee123d28,c0f89490,3dc,c8042598,...) at usb_process+0x1a3 fork_exit(c08fe340,c804ab24,ee123d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee123d60, ebp = 0 --- Tracing command usb pid 15 tid 100043 td 0xc804f8a0 sched_switch(c804f8a0,0,104,1a3,a2b323c7,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804f8a0,0,c0f94960,268,c804f8a0,...) at sleepq_switch+0x162 sleepq_wait(c804aafc,0,ee120cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c804aafc,c804ac74,c0f67e51,74,c804ab04,...) at _cv_wait+0x260 usb_process(c804aaf4,ee120d28,c0f89490,3dc,c8042598,...) at usb_process+0x1a3 fork_exit(c08fe340,c804aaf4,ee120d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee120d60, ebp = 0 --- Tracing command usb pid 15 tid 100042 td 0xc804fb80 sched_switch(c804fb80,0,104,1a3,a2b310a0,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804fb80,0,c0f94960,268,c804fb80,...) at sleepq_switch+0x162 sleepq_wait(c804aacc,0,ee11dcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c804aacc,c804ac74,c0f67e51,74,c804aad4,...) at _cv_wait+0x260 usb_process(c804aac4,ee11dd28,c0f89490,3dc,c8042598,...) at usb_process+0x1a3 fork_exit(c08fe340,c804aac4,ee11dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee11dd60, ebp = 0 --- Tracing command usb pid 15 tid 100040 td 0xc803e000 sched_switch(c803e000,0,104,1a3,a2b2fb1e,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c803e000,0,c0f94960,268,c803e000,...) at sleepq_switch+0x162 sleepq_wait(c8044b5c,0,ee116cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8044b5c,c8044c74,c0f67e51,74,c8044b64,...) at _cv_wait+0x260 usb_process(c8044b54,ee116d28,c0f89490,3dc,c8042598,...) at usb_process+0x1a3 fork_exit(c08fe340,c8044b54,ee116d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee116d60, ebp = 0 --- Tracing command usb pid 15 tid 100039 td 0xc803e2e0 sched_switch(c803e2e0,0,104,1a3,ac6646e9,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c803e2e0,0,c0f94960,268,c803e2e0,...) at sleepq_switch+0x162 sleepq_wait(c8044b2c,0,ee113cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8044b2c,c8044c74,c0f67e51,74,c8044b34,...) at _cv_wait+0x260 usb_process(c8044b24,ee113d28,c0f89490,3dc,c8042598,...) at usb_process+0x1a3 fork_exit(c08fe340,c8044b24,ee113d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee113d60, ebp = 0 --- Tracing command usb pid 15 tid 100038 td 0xc803e5c0 sched_switch(c803e5c0,0,104,1a3,a24f5fa9,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c803e5c0,0,c0f94960,268,c803e5c0,...) at sleepq_switch+0x162 sleepq_wait(c8044afc,0,ee110cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8044afc,c8044c74,c0f67e51,74,c8044b04,...) at _cv_wait+0x260 usb_process(c8044af4,ee110d28,c0f89490,3dc,c8042598,...) at usb_process+0x1a3 fork_exit(c08fe340,c8044af4,ee110d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee110d60, ebp = 0 --- Tracing command usb pid 15 tid 100037 td 0xc803e8a0 sched_switch(c803e8a0,0,104,1a3,a24f4d64,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c803e8a0,0,c0f94960,268,c803e8a0,...) at sleepq_switch+0x162 sleepq_wait(c8044acc,0,ee10dcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8044acc,c8044c74,c0f67e51,74,c8044ad4,...) at _cv_wait+0x260 usb_process(c8044ac4,ee10dd28,c0f89490,3dc,c8042598,...) at usb_process+0x1a3 fork_exit(c08fe340,c8044ac4,ee10dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee10dd60, ebp = 0 --- Tracing command usb pid 15 tid 100035 td 0xc8040000 sched_switch(c8040000,0,104,1a3,a24f3762,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8040000,0,c0f94960,268,c8040000,...) at sleepq_switch+0x162 sleepq_wait(c8038b5c,0,ee106cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8038b5c,c8038c74,c0f67e51,74,c8038b64,...) at _cv_wait+0x260 usb_process(c8038b54,ee106d28,c0f89490,3dc,c8042598,...) at usb_process+0x1a3 fork_exit(c08fe340,c8038b54,ee106d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee106d60, ebp = 0 --- Tracing command usb pid 15 tid 100034 td 0xc80402e0 sched_switch(c80402e0,0,104,1a3,ac65df84,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c80402e0,0,c0f94960,268,c80402e0,...) at sleepq_switch+0x162 sleepq_wait(c8038b2c,0,ee103cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8038b2c,c8038c74,c0f67e51,74,c8038b34,...) at _cv_wait+0x260 usb_process(c8038b24,ee103d28,c0f89490,3dc,c8042598,...) at usb_process+0x1a3 fork_exit(c08fe340,c8038b24,ee103d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee103d60, ebp = 0 --- Tracing command usb pid 15 tid 100033 td 0xc80405c0 sched_switch(c80405c0,0,104,1a3,a1eac8c4,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c80405c0,0,c0f94960,268,c80405c0,...) at sleepq_switch+0x162 sleepq_wait(c8038afc,0,ee100cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8038afc,c8038c74,c0f67e51,74,c8038b04,...) at _cv_wait+0x260 usb_process(c8038af4,ee100d28,c0f89490,3dc,c8042598,...) at usb_process+0x1a3 fork_exit(c08fe340,c8038af4,ee100d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee100d60, ebp = 0 --- Tracing command usb pid 15 tid 100032 td 0xc80408a0 sched_switch(c80408a0,0,104,1a3,a1eab618,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c80408a0,0,c0f94960,268,c80408a0,...) at sleepq_switch+0x162 sleepq_wait(c8038acc,0,ee0fdcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8038acc,c8038c74,c0f67e51,74,c8038ad4,...) at _cv_wait+0x260 usb_process(c8038ac4,ee0fdd28,c0f89490,3dc,c8042598,...) at usb_process+0x1a3 fork_exit(c08fe340,c8038ac4,ee0fdd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee0fdd60, ebp = 0 --- Tracing command yarrow pid 14 tid 100017 td 0xc7eda5c0 sched_switch(c7eda5c0,0,104,1a3,b9120a8c,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7eda5c0,0,c0f94960,28b,2,...) at sleepq_switch+0x162 sleepq_timedwait(c112a984,0,c0f81792,2,0,...) at sleepq_timedwait+0x6b _sleep(c112a984,0,0,c0f81792,64) at _sleep+0x36a pause(c0f81792,64,c0f43c10,111,0,...) at pause+0xa0 random_kthread(0,c7b51d28,c0f89490,3dc,c7dc7000,...) at random_kthread+0x1ef fork_exit(c08065e0,0,c7b51d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b51d60, ebp = 0 --- Tracing command geom pid 13 tid 100015 td 0xc7edab80 sched_switch(c7edab80,0,104,1a3,b1c6a436,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c7edab80,0,c0f94960,268,0,...) at sleepq_switch+0x162 sleepq_wait(c1128568,5c,c0f81792,0,0,...) at sleepq_wait+0x63 _sleep(c1128568,c11284c8,25c,c0f81792,0,...) at _sleep+0x39c g_io_schedule_down(c7edab80,0,c0f832e1,6c,c7b4bd14,...) at g_io_schedule_down+0x5b g_down_procbody(0,c7b4bd28,c0f89490,3dc,c7dc72cc,...) at g_down_procbody+0x9d fork_exit(c09ef230,0,c7b4bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b4bd60, ebp = 0 --- Tracing command geom pid 13 tid 100014 td 0xc7edb000 sched_switch(c7edb000,0,104,1a3,b2698804,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c7edb000,0,c0f94960,268,0,...) at sleepq_switch+0x162 sleepq_wait(c1128564,5c,c0f81792,0,0,...) at sleepq_wait+0x63 _sleep(c1128564,c11284e8,25c,c0f81792,0,...) at _sleep+0x39c g_io_schedule_up(c7edb000,0,c0f832e1,5f,c7b48d14,...) at g_io_schedule_up+0x134 g_up_procbody(0,c7b48d28,c0f89490,3dc,c7dc72cc,...) at g_up_procbody+0x9d fork_exit(c09ef2d0,0,c7b48d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b48d60, ebp = 0 --- Tracing command geom pid 13 tid 100013 td 0xc7dca2e0 sched_switch(c7dca2e0,0,104,1a3,b37d6016,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c7dca2e0,0,c0f94960,268,0,...) at sleepq_switch+0x162 sleepq_wait(c112855c,5c,c0f81792,0,0,...) at sleepq_wait+0x63 _sleep(c112855c,c112848c,25c,c0f81792,0,...) at _sleep+0x39c g_run_events(c1138800,0,c0f832e1,79,c7b45d14,...) at g_run_events+0x54b g_event_procbody(0,c7b45d28,c0f89490,3dc,c7dc72cc,...) at g_event_procbody+0x90 fork_exit(c09ef370,0,c7b45d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b45d60, ebp = 0 --- Tracing command intr pid 12 tid 100066 td 0xc81848a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100065 td 0xc8184b80 sched_switch(c8184b80,0,109,1a3,a1e66405,...) at sched_switch+0x3ec mi_switch(109,0,c0f897b3,554,c8184b80,...) at mi_switch+0x1ff ithread_loop(c7e12680,ee17cd28,c0f89490,3dc,c7dc7598,...) at ithread_loop+0x1fb fork_exit(c0a31fd0,c7e12680,ee17cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee17cd60, ebp = 0 --- Tracing command intr pid 12 tid 100063 td 0xc80512e0 sched_switch(c80512e0,0,109,1a3,ab53638e,...) at sched_switch+0x3ec mi_switch(109,0,c0f897b3,554,c80512e0,...) at mi_switch+0x1ff ithread_loop(c7e12690,ee16cd28,c0f89490,3dc,c7dc7598,...) at ithread_loop+0x1fb fork_exit(c0a31fd0,c7e12690,ee16cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee16cd60, ebp = 0 --- Tracing command intr pid 12 tid 100060 td 0xc8051b80 sched_switch(c8051b80,0,109,1a3,921513a3,...) at sched_switch+0x3ec mi_switch(109,0,c0f897b3,554,c8051b80,...) at mi_switch+0x1ff ithread_loop(c7e126a0,ee160d28,c0f89490,3dc,c7dc7598,...) at ithread_loop+0x1fb fork_exit(c0a31fd0,c7e126a0,ee160d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee160d60, ebp = 0 --- Tracing command intr pid 12 tid 100059 td 0xc805c000 sched_switch(c805c000,0,109,1a3,43f2507c,...) at sched_switch+0x3ec mi_switch(109,0,c0f897b3,554,c805c000,...) at mi_switch+0x1ff ithread_loop(c7e126b0,ee158d28,c0f89490,3dc,c7dc7598,...) at ithread_loop+0x1fb fork_exit(c0a31fd0,c7e126b0,ee158d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee158d60, ebp = 0 --- Tracing command intr pid 12 tid 100054 td 0xc805e000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100041 td 0xc8051000 sched_switch(c8051000,0,109,1a3,c6e218b2,...) at sched_switch+0x3ec mi_switch(109,0,c0f897b3,554,c8051000,...) at mi_switch+0x1ff ithread_loop(c7e126d0,ee11ad28,c0f89490,3dc,c7dc7598,...) at ithread_loop+0x1fb fork_exit(c0a31fd0,c7e126d0,ee11ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee11ad60, ebp = 0 --- Tracing command intr pid 12 tid 100036 td 0xc803eb80 sched_switch(c803eb80,0,109,1a3,c1695b28,...) at sched_switch+0x3ec mi_switch(109,0,c0f897b3,554,c803eb80,...) at mi_switch+0x1ff ithread_loop(c7e126e0,ee10ad28,c0f89490,3dc,c7dc7598,...) at ithread_loop+0x1fb fork_exit(c0a31fd0,c7e126e0,ee10ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee10ad60, ebp = 0 --- Tracing command intr pid 12 tid 100031 td 0xc8040b80 sched_switch(c8040b80,0,109,1a3,aa09443f,...) at sched_switch+0x3ec mi_switch(109,0,c0f897b3,554,c8040b80,...) at mi_switch+0x1ff ithread_loop(c7e126f0,ee0fad28,c0f89490,3dc,c7dc7598,...) at ithread_loop+0x1fb fork_exit(c0a31fd0,c7e126f0,ee0fad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee0fad60, ebp = 0 --- Tracing command intr pid 12 tid 100030 td 0xc7edb2e0 sched_switch(c7edb2e0,0,109,1a3,b268355b,...) at sched_switch+0x3ec mi_switch(109,0,c0f897b3,554,c7edb2e0,...) at mi_switch+0x1ff ithread_loop(c7e12700,edfa4d28,c0f89490,3dc,c7dc7598,...) at ithread_loop+0x1fb fork_exit(c0a31fd0,c7e12700,edfa4d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xedfa4d60, ebp = 0 --- Tracing command intr pid 12 tid 100029 td 0xc7edb5c0 sched_switch(c7edb5c0,0,109,1a3,b3e93989,...) at sched_switch+0x3ec mi_switch(109,0,c0f897b3,554,c7edb5c0,...) at mi_switch+0x1ff ithread_loop(c7e12710,edfa0d28,c0f89490,3dc,c7dc7598,...) at ithread_loop+0x1fb fork_exit(c0a31fd0,c7e12710,edfa0d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xedfa0d60, ebp = 0 --- Tracing command intr pid 12 tid 100028 td 0xc7edb8a0 sched_switch(c7edb8a0,0,109,1a3,93f1dfe9,...) at sched_switch+0x3ec mi_switch(109,0,c0f897b3,554,c7edb8a0,...) at mi_switch+0x1ff ithread_loop(c7e12720,c7ba0d28,c0f89490,3dc,c7dc7598,...) at ithread_loop+0x1fb fork_exit(c0a31fd0,c7e12720,c7ba0d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7ba0d60, ebp = 0 --- Tracing command intr pid 12 tid 100027 td 0xc7edbb80 sched_switch(c7edbb80,0,109,1a3,4814c48,...) at sched_switch+0x3ec mi_switch(109,0,c0f897b3,554,c7edbb80,...) at mi_switch+0x1ff ithread_loop(c7e12730,c7b6fd28,c0f89490,3dc,c7dc7598,...) at ithread_loop+0x1fb fork_exit(c0a31fd0,c7e12730,c7b6fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b6fd60, ebp = 0 --- Tracing command intr pid 12 tid 100026 td 0xc7fc6000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100024 td 0xc7fc65c0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100020 td 0xc7e13b80 sched_switch(c7e13b80,0,109,1a3,b268e285,...) at sched_switch+0x3ec mi_switch(109,0,c0f897b3,554,c7e13b80,...) at mi_switch+0x1ff ithread_loop(c7e12760,c7b5ad28,c0f89490,3dc,c7dc7598,...) at ithread_loop+0x1fb fork_exit(c0a31fd0,c7e12760,c7b5ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b5ad60, ebp = 0 --- Tracing command intr pid 12 tid 100012 td 0xc7dca5c0 sched_switch(c7dca5c0,0,109,1a3,ae526bd9,...) at sched_switch+0x3ec mi_switch(109,0,c0f897b3,554,c7dca5c0,...) at mi_switch+0x1ff ithread_loop(c7e127a0,c7b42d28,c0f89490,3dc,c7dc7598,...) at ithread_loop+0x1fb fork_exit(c0a31fd0,c7e127a0,c7b42d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b42d60, ebp = 0 --- Tracing command intr pid 12 tid 100011 td 0xc7dca8a0 sched_switch(c7dca8a0,0,109,1a3,e49dd3b3,...) at sched_switch+0x3ec mi_switch(109,0,c0f897b3,554,c7dca8a0,...) at mi_switch+0x1ff ithread_loop(c7e127b0,c7b3fd28,c0f89490,3dc,c7dc7598,...) at ithread_loop+0x1fb fork_exit(c0a31fd0,c7e127b0,c7b3fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b3fd60, ebp = 0 --- Tracing command intr pid 12 tid 100010 td 0xc7dcab80 sched_switch(c7dcab80,0,109,1a3,b9109463,...) at sched_switch+0x3ec mi_switch(109,0,c0f897b3,554,c7dcab80,...) at mi_switch+0x1ff ithread_loop(c7e127c0,c7b3cd28,c0f89490,3dc,c7dc7598,...) at ithread_loop+0x1fb fork_exit(c0a31fd0,c7e127c0,c7b3cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b3cd60, ebp = 0 --- Tracing command intr pid 12 tid 100009 td 0xc7e13000 sched_switch(c7e13000,0,109,1a3,b9106fa5,...) at sched_switch+0x3ec mi_switch(109,0,c0f897b3,554,c7e13000,...) at mi_switch+0x1ff ithread_loop(c7e127d0,c7b39d28,c0f89490,3dc,c7dc7598,...) at ithread_loop+0x1fb fork_exit(c0a31fd0,c7e127d0,c7b39d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b39d60, ebp = 0 --- Tracing command intr pid 12 tid 100008 td 0xc7e132e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100007 td 0xc7e135c0 sched_switch(c7e135c0,0,109,1a3,22fe629d,...) at sched_switch+0x3ec mi_switch(109,0,c0f897b3,554,c7e135c0,...) at mi_switch+0x1ff ithread_loop(c7e127f0,c7b33d28,c0f89490,3dc,c7dc7598,...) at ithread_loop+0x1fb fork_exit(c0a31fd0,c7e127f0,c7b33d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b33d60, ebp = 0 --- Tracing command idle pid 11 tid 100006 td 0xc7dc9000 cpustop_handler(3,13,c7b2fc04,c0dc1ca6,f5,...) at cpustop_handler+0x28 ipi_nmi_handler(f5,0,0,c12d7064,c7dc7864,...) at ipi_nmi_handler+0x47 trap(c7b2fc10) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0da6535, esp = 0xc7b2fc50, ebp = 0xc7b2fc50 --- acpi_cpu_c1(c7b2fc6c,c10f0930,3,301,e4,...) at acpi_cpu_c1+0x5 acpi_cpu_idle(0,c7b2fca0,c0db47c9,0,ffffffff,...) at acpi_cpu_idle+0x122 cpu_idle_acpi(0,ffffffff,c1139b80,2,c7b2fce8,...) at cpu_idle_acpi+0x2f cpu_idle(0,c7b2fcc4,c0f90fe2,a0b,c7dc9000,...) at cpu_idle+0x89 sched_idletd(0,c7b2fd28,c0f89490,3dc,c7dc7864,...) at sched_idletd+0x263 fork_exit(c0a82df0,0,c7b2fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b2fd60, ebp = 0 --- Tracing command idle pid 11 tid 100005 td 0xc7dc92e0 sched_switch(c7dc92e0,0,108,19e,94ab4c7d,...) at sched_switch+0x3ec mi_switch(108,0,c0f90fe2,a09,c7dc92e0,...) at mi_switch+0x1ff sched_idletd(0,c7b2cd28,c0f89490,3dc,c7dc7864,...) at sched_idletd+0x2bf fork_exit(c0a82df0,0,c7b2cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b2cd60, ebp = 0 --- Tracing command idle pid 11 tid 100004 td 0xc7dc95c0 sched_switch(c7dc95c0,0,608,19e,b6230330,...) at sched_switch+0x3ec mi_switch(608,0,c0f8f75f,cd,0,...) at mi_switch+0x1ff critical_exit(0,ffffffff,c1138e80,2,c7b29ce8,...) at critical_exit+0xb5 cpu_idle(0,c7b29cc4,c0f90fe2,a0b,c7dc95c0,...) at cpu_idle+0x97 sched_idletd(0,c7b29d28,c0f89490,3dc,c7dc7864,...) at sched_idletd+0x263 fork_exit(c0a82df0,0,c7b29d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b29d60, ebp = 0 --- Tracing command idle pid 11 tid 100003 td 0xc7dc98a0 cpustop_handler(0,13,c7b26c04,c0dc1ca6,f5,...) at cpustop_handler+0x28 ipi_nmi_handler(f5,0,0,c12d66e0,c7dc7864,...) at ipi_nmi_handler+0x47 trap(c7b26c10) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0da6535, esp = 0xc7b26c50, ebp = 0xc7b26c50 --- acpi_cpu_c1(c7b26c6c,c10f0930,0,301,e4,...) at acpi_cpu_c1+0x5 acpi_cpu_idle(0,c7b26ca0,c0db47c9,0,ffffffff,...) at acpi_cpu_idle+0x122 cpu_idle_acpi(0,ffffffff,c1138800,2,c7b26ce8,...) at cpu_idle_acpi+0x2f cpu_idle(0,c7b26cc4,c0f90fe2,a0b,c7dc98a0,...) at cpu_idle+0x89 sched_idletd(0,c7b26d28,c0f89490,3dc,c7dc7864,...) at sched_idletd+0x263 fork_exit(c0a82df0,0,c7b26d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b26d60, ebp = 0 --- Tracing command init pid 1 tid 100002 td 0xc7dc9b80 sched_switch(c7dc9b80,0,104,1a3,ae73841c,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(c7dc9b80,0,c0f94960,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f94960,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7dc7b30,6c,c0f978a7,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7dc7b30,c7dc7bb8,16c,c0f978a7,0,...) at _sleep+0x385 kern_wait(c7dc9b80,ffffffff,c7b22c60,2,0,...) at kern_wait+0x597 sys_wait4(c7dc9b80,c7b22cec,c0fe0d8a,c0f976de,282,...) at sys_wait4+0x3b syscall(c7b22d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x806020f, esp = 0xbfbfe86c, ebp = 0xbfbfe888 --- Tracing command audit pid 10 tid 100001 td 0xc7dca000 sched_switch(c7dca000,0,104,1a3,a1e8eafb,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7dca000,0,c0f94960,268,c7dca000,...) at sleepq_switch+0x162 sleepq_wait(c12a77c0,0,c7b1fc8c,1,0,...) at sleepq_wait+0x63 _cv_wait(c12a77c0,c12a77a4,c0fbfc37,194,0,...) at _cv_wait+0x260 audit_worker(0,c7b1fd28,c0f89490,3dc,c7dc8000,...) at audit_worker+0x84 fork_exit(c0c5c980,0,c7b1fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b1fd60, ebp = 0 --- Tracing command kernel pid 0 tid 100078 td 0xca6ce8a0 sched_switch(ca6ce8a0,0,104,1a3,45827fc3,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca6ce8a0,0,c0f94960,268,0,...) at sleepq_switch+0x162 sleepq_wait(c8035380,0,c0f81792,0,0,...) at sleepq_wait+0x63 _sleep(c8035380,c8035398,0,c0f81792,0,...) at _sleep+0x39c taskqueue_thread_loop(c1309060,ee323d28,c0f89490,3dc,c1128640,...) at taskqueue_thread_loop+0xb5 fork_exit(c0aa0080,c1309060,ee323d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee323d60, ebp = 0 --- Tracing command kernel pid 0 tid 100068 td 0xc81842e0 sched_switch(c81842e0,0,104,1a3,17952c64,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c81842e0,0,c0f94960,28b,2,...) at sleepq_switch+0x162 sleepq_timedwait(c112a984,0,c0f81792,2,0,...) at sleepq_timedwait+0x6b _sleep(c112a984,0,0,c0f81792,bb8) at _sleep+0x36a pause(c0f81792,bb8,124,122,c11286c8,...) at pause+0xa0 deadlkres(0,ee185d28,c0f89490,3dc,c1128640,...) at deadlkres+0x324 fork_exit(c0a11540,0,ee185d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee185d60, ebp = 0 --- Tracing command kernel pid 0 tid 100061 td 0xc80518a0 sched_switch(c80518a0,0,104,1a3,b18b40b4,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c80518a0,0,c0f94960,268,c80518a0,...) at sleepq_switch+0x162 sleepq_wait(c80356c0,0,c0f8fc81,c0f81792,0,...) at sleepq_wait+0x63 msleep_spin(c80356c0,c80356d8,c0f81792,0,c0f8d34f,...) at msleep_spin+0x259 taskqueue_thread_loop(c809349c,ee166d28,c0f89490,3dc,c1128640,...) at taskqueue_thread_loop+0x8f fork_exit(c0aa0080,c809349c,ee166d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee166d60, ebp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xc7fc62e0 sched_switch(c7fc62e0,0,104,1a3,440eec3d,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7fc62e0,0,c0f94960,268,0,...) at sleepq_switch+0x162 sleepq_wait(c7f9de80,0,c0f81792,0,0,...) at sleepq_wait+0x63 _sleep(c7f9de80,c7f9de98,0,c0f81792,0,...) at _sleep+0x39c taskqueue_thread_loop(c114756c,c7b69d28,c0f89490,3dc,c1128640,...) at taskqueue_thread_loop+0xb5 fork_exit(c0aa0080,c114756c,c7b69d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b69d60, ebp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xc7fc68a0 sched_switch(c7fc68a0,0,104,1a3,bbfd644f,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7fc68a0,0,c0f94960,268,c7fc68a0,...) at sleepq_switch+0x162 sleepq_wait(c7f9e040,0,c0f8fc81,c0f81792,0,...) at sleepq_wait+0x63 msleep_spin(c7f9e040,c7f9e058,c0f81792,0,c0f8d34f,...) at msleep_spin+0x259 taskqueue_thread_loop(c10f35b8,c7b63d28,c0f89490,3dc,c1128640,...) at taskqueue_thread_loop+0x8f fork_exit(c0aa0080,c10f35b8,c7b63d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b63d60, ebp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xc7fc6b80 sched_switch(c7fc6b80,0,104,1a3,bbfd5472,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7fc6b80,0,c0f94960,268,c7fc6b80,...) at sleepq_switch+0x162 sleepq_wait(c7f9e040,0,c0f8fc81,c0f81792,0,...) at sleepq_wait+0x63 msleep_spin(c7f9e040,c7f9e058,c0f81792,0,c0f8d34f,...) at msleep_spin+0x259 taskqueue_thread_loop(c10f35b8,c7b60d28,c0f89490,3dc,c1128640,...) at taskqueue_thread_loop+0x8f fork_exit(c0aa0080,c10f35b8,c7b60d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b60d60, ebp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xc7e138a0 sched_switch(c7e138a0,0,104,1a3,bbfd41cb,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7e138a0,0,c0f94960,268,c7e138a0,...) at sleepq_switch+0x162 sleepq_wait(c7f9e040,0,c0f8fc81,c0f81792,0,...) at sleepq_wait+0x63 msleep_spin(c7f9e040,c7f9e058,c0f81792,0,c0f8d34f,...) at msleep_spin+0x259 taskqueue_thread_loop(c10f35b8,c7b5dd28,c0f89490,3dc,c1128640,...) at taskqueue_thread_loop+0x8f fork_exit(c0aa0080,c10f35b8,c7b5dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b5dd60, ebp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xc7eda000 sched_switch(c7eda000,0,104,1a3,bbf9cdc8,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7eda000,0,c0f94960,268,0,...) at sleepq_switch+0x162 sleepq_wait(c7f9e080,0,c0f81792,0,0,...) at sleepq_wait+0x63 _sleep(c7f9e080,c7f9e098,0,c0f81792,0,...) at _sleep+0x39c taskqueue_thread_loop(c12a7da4,c7b57d28,c0f89490,3dc,c1128640,...) at taskqueue_thread_loop+0xb5 fork_exit(c0aa0080,c12a7da4,c7b57d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b57d60, ebp = 0 --- Tracing command kernel pid 0 tid 100018 td 0xc7eda2e0 sched_switch(c7eda2e0,0,104,1a3,bbf9bdd7,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7eda2e0,0,c0f94960,268,0,...) at sleepq_switch+0x162 sleepq_wait(c7f9e0c0,0,c0f81792,0,0,...) at sleepq_wait+0x63 _sleep(c7f9e0c0,c7f9e0d8,0,c0f81792,0,...) at _sleep+0x39c taskqueue_thread_loop(c1128fb8,c7b54d28,c0f89490,3dc,c1128640,...) at taskqueue_thread_loop+0xb5 fork_exit(c0aa0080,c1128fb8,c7b54d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b54d60, ebp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xc7eda8a0 sched_switch(c7eda8a0,0,104,1a3,80b66e78,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7eda8a0,0,c0f94960,268,0,...) at sleepq_switch+0x162 sleepq_wait(c7dade80,0,c0f81792,0,0,...) at sleepq_wait+0x63 _sleep(c7dade80,c7dade98,0,c0f81792,0,...) at _sleep+0x39c taskqueue_thread_loop(c1145f80,c7b4ed28,c0f89490,3dc,c1128640,...) at taskqueue_thread_loop+0xb5 fork_exit(c0aa0080,c1145f80,c7b4ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b4ed60, ebp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xc1128910 sched_switch(c1128910,0,104,1a3,7b9513bd,...) at sched_switch+0x3ec mi_switch(104,0,c0f94960,1f3,54,...) at mi_switch+0x1ff sleepq_switch(c1128910,0,c0f94960,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c1128640,54,c0f918f4,0,0,...) at sleepq_timedwait+0x6b _sleep(c1128640,0,54,c0f918f4,2710,...) at _sleep+0x36a scheduler(0,181ec00,181ec00,181e000,1825000,...) at scheduler+0x240 mi_startup() at mi_startup+0xac begin() at begin+0x2c db:0:allt> call doadump Physical memory: 3435 MB Dumping 163 MB: 148 132 116 100 84 68 52 36 20 4 Dump complete = 0 db:0:doadump> reset cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 2 (kgdb) bt #0 doadump (textdump=0xf29d9528) at pcpu.h:244 #1 0xc0510f99 in db_fncall (dummy1=0xc0a9138a, dummy2=0x0, dummy3=0xffffffff, dummy4=0xf29d95bc "Ð\225\235ò") at ../../../ddb/db_command.c:573 #2 0xc05113cf in db_command (last_cmdp=0xc10f177c, cmd_table=0x0, dopager=0x0) at ../../../ddb/db_command.c:449 #3 0xc0511484 in db_command_script (command=0xc10f2688 "call doadump") at ../../../ddb/db_command.c:520 #4 0xc0515870 in db_script_exec (scriptname=0xc10f1fe0 "doadump", warnifnotfound=Variable "warnifnotfound" is not available. ) at ../../../ddb/db_script.c:302 #5 0xc0515901 in db_run_cmd (addr=0x1, have_addr=0x0, count=0xc12d4960, modif=0xf29d96f4 "") at ../../../ddb/db_script.c:375 #6 0xc0511391 in db_command (last_cmdp=0xc10f177c, cmd_table=0x0, dopager=0x1) at ../../../ddb/db_command.c:449 #7 0xc05114ea in db_command_loop () at ../../../ddb/db_command.c:502 #8 0xc05134dd in db_trap (type=0x3, code=0x0) at ../../../ddb/db_main.c:229 #9 0xc0a91846 in kdb_trap (type=0x3, code=0x0, tf=0xf29d98a8) at ../../../kern/subr_kdb.c:629 #10 0xc0dc22db in trap (frame=0xf29d98a8) at ../../../i386/i386/trap.c:722 #11 0xc0daab6c in calltrap () at ../../../i386/i386/exception.s:168 #12 0xc0a9138a in kdb_enter (why=0xc0f9ebe2 "vfslock", msg=0xc0f9fc4b "lock violation") at cpufunc.h:71 #13 0xc0af76d1 in vfs_badlock (msg=0xc0f9f5c7 "is not exclusive locked but should be", str=0xcb2012aa "lowervp", vp=0xcad54828) at ../../../kern/vfs_subr.c:3814 #14 0xc0afb6f5 in assert_vop_elocked (vp=0xcad54828, str=0xcb2012aa "lowervp") at ../../../kern/vfs_subr.c:3854 #15 0xcb1ff84f in null_nodeget (mp=0xcb20c5a8, lowervp=0xcad54828, vpp=0xf29d9994) at /usr/src/sys/modules/nullfs/../../fs/nullfs/null_subr.c:216 #16 0xcb200da3 in null_lookup (ap=0xf29d9a20) at /usr/src/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:367 #17 0xc0de5d85 in VOP_LOOKUP_APV (vop=0xcb2026e0, a=0xf29d9a20) at vnode_if.c:123 #18 0xc0aed7cb in lookup (ndp=0xf29d9ad4) at vnode_if.h:54 #19 0xc0aee7ef in namei (ndp=0xf29d9ad4) at ../../../kern/vfs_lookup.c:307 #20 0xc0b0045f in kern_statat_vnhook (td=0xcb2495c0, flag=0x200, fd=0xffffff9c, path=0x284201b8
, pathseg=UIO_USERSPACE, sbp=0xf29d9c04, hook=0) at ../../../kern/vfs_syscalls.c:2433 #21 0xc0b005bc in kern_statat (td=0xcb2495c0, flag=0x200, fd=0xffffff9c, path=0x284201b8
, pathseg=UIO_USERSPACE, sbp=0xf29d9c04) at ../../../kern/vfs_syscalls.c:2414 #22 0xc0b005f6 in kern_lstat (td=0xcb2495c0, path=0x284201b8
, pathseg=UIO_USERSPACE, sbp=0xf29d9c04) at ../../../kern/vfs_syscalls.c:2487 #23 0xc0b0069f in sys_lstat (td=0xcb2495c0, uap=0xf29d9cec) at ../../../kern/vfs_syscalls.c:2477 #24 0xc0dc18a7 in syscall (frame=0xf29d9d28) at subr_syscall.c:135 #25 0xc0daabd1 in Xint0x80_syscall () at ../../../i386/i386/exception.s:266 #26 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) f 15 #15 0xcb1ff84f in null_nodeget (mp=0xcb20c5a8, lowervp=0xcad54828, vpp=0xf29d9994) at /usr/src/sys/modules/nullfs/../../fs/nullfs/null_subr.c:216 216 ASSERT_VOP_ELOCKED(lowervp, "lowervp"); (kgdb) l 211 212 /* 213 * The insmntque1() call below requires the exclusive lock on 214 * the nullfs vnode. 215 */ 216 ASSERT_VOP_ELOCKED(lowervp, "lowervp"); 217 KASSERT(lowervp->v_usecount >= 1, ("Unreferenced vnode %p\n", lowervp)); 218 219 /* Lookup the hash firstly */ 220 *vpp = null_hashget(mp, lowervp); (kgdb) info loc xp = Variable "xp" is not available. (kgdb) p *lowervp $1 = {v_type = VREG, v_tag = 0xc0f85397 "ufs", v_op = 0xc10ddda0, v_data = 0xcaf6d244, v_mount = 0xca774b50, v_nmntvnodes = {tqe_next = 0xcadb4414, tqe_prev = 0xcaafa83c}, v_un = {vu_mount = 0x0, vu_socket = 0x0, vu_cdev = 0x0, vu_fifoinfo = 0x0}, v_hashlist = {le_next = 0x0, le_prev = 0xca441524}, v_hash = 0x34174, v_cache_src = {lh_first = 0x0}, v_cache_dst = {tqh_first = 0xcb3ae168, tqh_last = 0xcb3ae178}, v_cache_dd = 0x0, v_cstart = 0x0, v_lasta = 0x0, v_lastw = 0x0, v_clen = 0x0, v_lock = {lock_object = {lo_name = 0xc0f85397 "ufs", lo_flags = 0x57b0000, lo_data = 0x0, lo_witness = 0xc7d795a8}, lk_lock = 0x11, lk_exslpfail = 0x0, lk_timo = 0x33, lk_pri = 0x60, lk_stack = {depth = 0x12, pcs = {0xc0a43374, 0xc0ca6571, 0xc0de4c05, 0xc0b09eb8, 0xc0afc96b, 0xc0aec36d, 0xc0c9f8d9, 0xc0c9fe5e, 0xc0cae9c4, 0xc0caea6a, 0xc0de4565, 0xc0ae5816, 0xc0de5d85, 0xcb200d74, 0xc0de5d85, 0xc0aed7cb, 0xc0aee7ef, 0xc0b0045f}}}, v_interlock = {lock_object = {lo_name = 0xc0f96c18 "vnode interlock", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc7d721e0}, mtx_lock = 0x4}, v_vnlock = 0xcad54880, v_holdcnt = 0x3, v_usecount = 0x1, v_iflag = 0x0, v_vflag = 0x0, v_writecount = 0x0, v_freelist = {tqe_next = 0x0, tqe_prev = 0x0}, v_bufobj = {bo_mtx = {lock_object = {lo_name = 0xc0f9f94e "bufobj interlock", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc7d77ba8}, mtx_lock = 0x4}, bo_clean = {bv_hd = {tqh_first = 0xe1945980, tqh_last = 0xe19459b8}, bv_root = 0xe1945980, bv_cnt = 0x1}, bo_dirty = {bv_hd = {tqh_first = 0x0, tqh_last = 0xcad54944}, bv_root = 0x0, bv_cnt = 0x0}, bo_numoutput = 0x0, bo_flag = 0x0, bo_ops = 0xc10cafc0, bo_bsize = 0x4000, bo_object = 0xcb0fa1bc, bo_synclist = {le_next = 0x0, le_prev = 0xca43101c}, bo_private = 0xcad54828, __bo_vnode = 0xcad54828}, v_pollinfo = 0x0, v_label = 0x0, v_lockf = 0x0} (kgdb) p *lowervp->v_mount $2 = {mnt_mtx = {lock_object = {lo_name = 0xc0f80f51 "struct mount mtx", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc7d72450}, mtx_lock = 0x4}, mnt_gen = 0x1, mnt_list = {tqe_next = 0xca775000, tqe_prev = 0xc10cc4b0}, mnt_op = 0xc10ddb40, mnt_vfc = 0xc10dd9a0, mnt_vnodecovered = 0x0, mnt_syncer = 0xca824984, mnt_ref = 0x175, mnt_nvnodelist = {tqh_first = 0xca475c3c, tqh_last = 0xcb3e22cc}, mnt_nvnodelistsize = 0x175, mnt_writeopcount = 0x0, mnt_kern_flag = 0x60000040, mnt_flag = 0x5100, mnt_noasync = 0x0, mnt_opt = 0xca747520, mnt_optnew = 0x0, mnt_maxsymlinklen = 0x78, mnt_stat = { f_version = 0x20030518, f_type = 0x35, f_flags = 0x5100, f_bsize = 0x800, f_iosize = 0x4000, f_blocks = 0xf7517, f_bfree = 0x1c7b9, f_bavail = 0x8b28, f_files = 0x44ffe, f_ffree = 0x42bfe, f_syncwrites = 0x0, f_asyncwrites = 0x0, f_syncreads = 0x0, f_asyncreads = 0x0, f_spare = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, f_namemax = 0xff, f_owner = 0x0, f_fsid = {val = {0x4c972dc2, 0x33dea0d3}}, f_charspare = '\0' , f_fstypename = "ufs", '\0' , f_mntfromname = "/dev/ad4s1a", '\0' , f_mntonname = "/", '\0' }, mnt_cred = 0xca776500, mnt_data = 0xca78c000, mnt_time = 0x4f49ca57, mnt_iosize_max = 0x20000, mnt_export = 0xcb3c4a00, mnt_label = 0x0, mnt_hashseed = 0xf0d5f7d5, mnt_lockref = 0x0, mnt_secondary_writes = 0x0, mnt_secondary_accwrites = 0x9b5, mnt_susp_owner = 0x0, mnt_gjprovider = 0x0, mnt_explock = {lock_object = {lo_name = 0xc0f80f62 "explock", lo_flags = 0x5730000, lo_data = 0x0, lo_witness = 0xc7d77e80}, lk_lock = 0x1, lk_exslpfail = 0x0, lk_timo = 0x0, lk_pri = 0x60, lk_stack = {depth = 0x6, pcs = {0xc0a439ce, 0xc0ae9747, 0xc0af1fde, 0xc0af263f, 0xc0dc18a7, 0xc0daabd1, 0x0 }}}} (kgdb) $ svn diff -x -p /usr/src/sys Index: /usr/src/sys/fs/nullfs/null_vnops.c =================================================================== --- /usr/src/sys/fs/nullfs/null_vnops.c (revision 232125) +++ /usr/src/sys/fs/nullfs/null_vnops.c (working copy) @@ -697,12 +697,18 @@ null_inactive(struct vop_inactive_args *ap) static int null_reclaim(struct vop_reclaim_args *ap) { - struct vnode *vp = ap->a_vp; - struct null_node *xp = VTONULL(vp); - struct vnode *lowervp = xp->null_lowervp; + struct vnode *vp; + struct null_node *xp; + struct vnode *lowervp; - if (lowervp) - null_hashrem(xp); + vp = ap->a_vp; + xp = VTONULL(vp); + lowervp = xp->null_lowervp; + + KASSERT(lowervp != NULL && vp->v_vnlock != &vp->v_lock, + ("Reclaiming inclomplete null vnode %p", vp)); + + null_hashrem(xp); /* * Use the interlock to protect the clearing of v_data to * prevent faults in null_lock(). @@ -713,10 +719,7 @@ null_reclaim(struct vop_reclaim_args *ap) vp->v_object = NULL; vp->v_vnlock = &vp->v_lock; VI_UNLOCK(vp); - if (lowervp) - vput(lowervp); - else - panic("null_reclaim: reclaiming a node with no lowervp"); + vput(lowervp); free(xp, M_NULLFSNODE); return (0); Index: /usr/src/sys/fs/nullfs/null_subr.c =================================================================== --- /usr/src/sys/fs/nullfs/null_subr.c (revision 232125) +++ /usr/src/sys/fs/nullfs/null_subr.c (working copy) @@ -169,19 +169,28 @@ null_hashins(mp, xp) } static void -null_insmntque_dtr(struct vnode *vp, void *xp) +null_destroy_proto(struct vnode *vp, void *xp) { - vput(((struct null_node *)xp)->null_lowervp); + VI_LOCK(vp); vp->v_data = NULL; vp->v_vnlock = &vp->v_lock; - free(xp, M_NULLFSNODE); vp->v_op = &dead_vnodeops; + VI_UNLOCK(vp); (void) vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); vgone(vp); vput(vp); + free(xp, M_NULLFSNODE); } +static void +null_insmntque_dtr(struct vnode *vp, void *xp) +{ + + vput(((struct null_node *)xp)->null_lowervp); + null_destroy_proto(vp, xp); +} + /* * Make a new or get existing nullfs node. * Vp is the alias vnode, lowervp is the lower vnode. @@ -200,7 +209,11 @@ null_nodeget(mp, lowervp, vpp) struct vnode *vp; int error; - ASSERT_VOP_LOCKED(lowervp, "lowervp"); + /* + * The insmntque1() call below requires the exclusive lock on + * the nullfs vnode. + */ + ASSERT_VOP_ELOCKED(lowervp, "lowervp"); KASSERT(lowervp->v_usecount >= 1, ("Unreferenced vnode %p\n", lowervp)); /* Lookup the hash firstly */ @@ -213,12 +226,9 @@ null_nodeget(mp, lowervp, vpp) /* * We do not serialize vnode creation, instead we will check for * duplicates later, when adding new vnode to hash. - * * Note that duplicate can only appear in hash if the lowervp is * locked LK_SHARED. - */ - - /* + * * Do the MALLOC before the getnewvnode since doing so afterward * might cause a bogus v_data pointer to get dereferenced * elsewhere if MALLOC should block. @@ -250,9 +260,7 @@ null_nodeget(mp, lowervp, vpp) *vpp = null_hashins(mp, xp); if (*vpp != NULL) { vrele(lowervp); - vp->v_vnlock = &vp->v_lock; - xp->null_lowervp = NULL; - vrele(vp); + null_destroy_proto(vp, xp); return (0); } *vpp = vp; Index: /usr/src/sys/fs/nullfs/null_vfsops.c =================================================================== --- /usr/src/sys/fs/nullfs/null_vfsops.c (revision 232125) +++ /usr/src/sys/fs/nullfs/null_vfsops.c (working copy) @@ -185,7 +185,8 @@ nullfs_mount(struct mount *mp) MNT_IUNLOCK(mp); } MNT_ILOCK(mp); - mp->mnt_kern_flag |= lowerrootvp->v_mount->mnt_kern_flag & MNTK_MPSAFE; + mp->mnt_kern_flag |= lowerrootvp->v_mount->mnt_kern_flag & + (MNTK_MPSAFE | MNTK_SHARED_WRITES | MNTK_LOOKUP_SHARED); MNT_IUNLOCK(mp); mp->mnt_data = xmp; vfs_getnewfsid(mp); @@ -312,6 +313,12 @@ nullfs_vget(mp, ino, flags, vpp) struct vnode **vpp; { int error; + + KASSERT((flags & LK_TYPE_MASK) != 0, + ("nullfs_vget: no lock requested")); + flags &= ~LK_TYPE_MASK; + flags |= LK_EXCLUSIVE; + error = VFS_VGET(MOUNTTONULLMOUNT(mp)->nullm_vfs, ino, flags, vpp); if (error) return (error); -- Test scenario: nullfs.sh