GENERIC HEAD from 2011-09-21 00:15:29 UTC, r225750M, vmcore.198 KDB: debugger backends: ddb KDB: current backend: ddb 524288K of memory above 4GB ignored Copyright (c) 1992-2011 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-BETA2 #2 r225750M: Sun Sep 25 20:26:32 CEST 2011 pho@x4.osted.lan:/usr/src/sys/i386/compile/PHO i386 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. CPU: AMD Phenom(tm) 9150e Quad-Core Processor (1800.02-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x100f23 Family = 10 Model = 2 Stepping = 3 Features=0x178bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x7ff TSC: P-state invariant real memory = 4294967296 (4096 MB) avail memory = 3531739136 (3368 MB) : Trying to mount root from ufs:/dev/ad4s1a [rw]... Setting hostuuid: 00000000-0000-0000-0000-00218515337d. Setting hostid: 0x6b64ac17. Starting ddb. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: /dev/ad4s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1a: clean, 327537 free (1345 frags, 40774 blocks, 0.1% fragmentation) /dev/ad4s1f: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1f: clean, 190908 free (4388 frags, 23315 blocks, 0.4% fragmentation) /dev/label/tmp: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/label/tmp: clean, 94652557 free (2181 frags, 11831297 blocks, 0.0% fragmentation) /dev/ad4s1d: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1d: clean, 1518627 free (273499 frags, 155641 blocks, 2.7% fragmentation) /dev/ad4s1e: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1e: clean, 2660925 free (78085 frags, 322855 blocks, 0.3% fragmentation) Mounting local file systems:. Setting hostname: x4.osted.lan. re0: link state changed to DOWN Starting Network: lo0 re0 fwe0 fwip0. lo0: flags=8049 metric 0 mtu 16384 options=3 inet 127.0.0.1 netmask 0xff000000 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0xa nd6 options=21 re0: flags=8843 metric 0 mtu 1500 options=389b ether 00:21:85:15:33:7d inet 192.168.1.101 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::221:85ff:fe15:337d%re0 prefixlen 64 tentative scopeid 0x1 nd6 options=29 media: Ethernet autoselect (none) status: no carrier fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb nd6 options=29 ch 1 dma -1 fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 nd6 options=29 Starting devd. Starting Network: usbus0. Starting Network: usbus1. Starting Network: usbus2. Starting Network: usbus3. Starting Network: usbus4. Starting Network: usbus5. Starting Network: fwe0. fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb nd6 options=29 ch 1 dma -1 re0: link state changed to UP Starting Network: fwip0. fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 nd6 options=29 add net default: gateway 192.168.1.1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/kde4/lib /usr/local/lib /usr/local/lib/compat /usr/local/lib/compat/pkg /usr/local/lib/nss /usr/local/lib/qt4 /usr/local/lib/virtualbox /usr/local/lib/wine a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout Creating and/or trimming log files. Starting syslogd. No core dumps found. Additional ABI support: linuxkldload: can't load linux: No such file or directory /etc/rc: WARNING: Unable to load kernel module linux . Starting rpcbind. kldload: can't load nfsclient: No such file or directory /etc/rc: WARNING: Unable to load kernel module nfsclient lock order reversal: 1st 0xe0b1d660 bufwait (bufwait) @ kern/vfs_bio.c:2658 2nd 0xc7d4d800 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:284 KDB: stack backtrace: db_trace_self_wrapper(c0f078a8,2e687361,38323a63,c7000a34,a7a,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a4515b,c0f0b28b,c7165370,c71693a0,f18c0878,...) at kdb_backtrace+0x2a _witness_debugger(c0f0b28b,c7d4d800,c0f3ab06,c71693a0,c0f3a798,...) at _witness_debugger+0x25 witness_checkorder(c7d4d800,9,c0f3a78f,11c,0,...) at witness_checkorder+0x839 _sx_xlock(c7d4d800,0,c0f3a78f,11c,c786f2b8,...) at _sx_xlock+0x85 ufsdirhash_acquire(e0b1d600,c786f2b8,f18c09f4,e17d1f00,f18c0948,...) at ufsdirhash_acquire+0x48 ufsdirhash_add(c786f2b8,f18c09f4,1f00,f18c0934,f18c0938,...) at ufsdirhash_add+0x13 ufs_direnter(c7b77414,c7d4a2b8,f18c09f4,f18c0b84,e0b1dc80,...) at ufs_direnter+0x789 ufs_mkdir(f18c0c14,c0f58c8e,0,0,f18c0bac,...) at ufs_mkdir+0xa36 VOP_MKDIR_APV(c1052180,f18c0c14,f18c0b84,f18c0bac,0,...) at VOP_MKDIR_APV+0xc5 kern_mkdirat(c78a15c0,ffffff9c,28404020,0,1c0,...) at kern_mkdirat+0x291 kern_mkdir(c78a15c0,28404020,0,1c0,f18c0d1c,...) at kern_mkdir+0x2e sys_mkdir(c78a15c0,f18c0cec,c0f5252a,c0f0be06,202,...) at sys_mkdir+0x29 syscall(f18c0d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28174303, esp = 0xbfbfe8cc, ebp = 0xbfbfed78 --- Clearing /tmp (X related). kldload: can't load nfsserver: No such file or directory /etc/rc: WARNING: Unable to load kernel module nfsserver Starting mountd. NFSv4 is disabled Starting nfsd. Recovering vi editor sessions:lock order reversal: 1st 0xc78efc94 ufs (ufs) @ kern/vfs_lookup.c:530 2nd 0xe0b23e60 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:260 3rd 0xc78ef9dc ufs (ufs) @ kern/vfs_subr.c:2134 KDB: stack backtrace: db_trace_self_wrapper(c0f078a8,3a632e73,a303632,0,0,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a4515b,c0f0b2a4,c7165370,c7169338,f18ba344,...) at kdb_backtrace+0x2a _witness_debugger(c0f0b2a4,c78ef9dc,c0efa4e4,c7169338,c0f12fca,...) at _witness_debugger+0x25 witness_checkorder(c78ef9dc,9,c0f12fc1,856,0,...) at witness_checkorder+0x839 __lockmgr_args(c78ef9dc,80100,c78efa48,0,0,...) at __lockmgr_args+0x834 ffs_lock(f18ba468,c0a5657b,c0f12293,80100,c78ef984,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c1052180,f18ba468,c78a1c30,c1062980,c78ef984,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c78ef984,80100,c0f12fc1,856,4,...) at _vn_lock+0x78 vget(c78ef984,80100,c78a1b80,50,0,...) at vget+0xbb vfs_hash_get(c7865000,15f687,80000,c78a1b80,f18ba5ac,...) at vfs_hash_get+0xed ffs_vgetf(c7865000,15f687,80000,f18ba5ac,1,...) at ffs_vgetf+0x49 softdep_sync_buf(c78efc3c,e0b23e00,1,106,0,...) at softdep_sync_buf+0xac9 ffs_syncvnode(c78efc3c,1,c7165370,c,0,...) at ffs_syncvnode+0x24c ffs_truncate(c78efc3c,200,0,880,c7921280,...) at ffs_truncate+0x817 ufs_direnter(c78efc3c,c78ef984,f18ba918,f18babc0,0,...) at ufs_direnter+0x971 ufs_makeinode(f18babc0,c1052700,f18bab04,f18baa60,c0d67ba5,...) at ufs_makeinode+0x635 ufs_create(f18bab04,c0f58e54,0,0,f18bab80,...) at ufs_create+0x30 VOP_CREATE_APV(c1052180,f18bab04,f18babc0,f18baa9c,0,...) at VOP_CREATE_APV+0xc5 vn_open_cred(f18bab80,f18bac48,1b0,0,c7921280,...) at vn_open_cred+0x205 vn_open(f18bab80,f18bac48,1b0,c78b4af0,4e7f727d,...) at vn_open+0x3b kern_openat(c78a1b80,ffffff9c,28839f60,0,a03,...) at kern_openat+0x1ec kern_open(c78a1b80,28839f60,0,a02,1b0,...) at kern_open+0x35 sys_open(c78a1b80,f18bacec,c0f5252a,c0f0be06,212,...) at sys_open+0x30 syscall(f18bad28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (5, FreeBSD ELF32, sys_open), eip = 0x283d6343, esp = 0xbfbfc4fc, ebp = 0xbfbfc588 --- . Updating motd:. Starting ntpd. Starting default moused. Configuring syscons: keymap blanktime. Starting sshd. Starting cron. Local package initialization: backuplock order reversal: 1st 0xc7b77c94 ufs (ufs) @ kern/vfs_mount.c:1194 2nd 0xc78f0880 devfs (devfs) @ ufs/ffs/ffs_softdep.c:1833 KDB: stack backtrace: db_trace_self_wrapper(c0f078a8,38313a63,a3333,c0f3c165,c19b3888,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a4515b,c0f0b28b,c7169338,c7169200,f18cfa10,...) at kdb_backtrace+0x2a _witness_debugger(c0f0b28b,c78f0880,c0ef2347,c7169200,c0f36c99,...) at _witness_debugger+0x25 witness_checkorder(c78f0880,9,c0f36c90,729,c78f08ec,...) at witness_checkorder+0x839 __lockmgr_args(c78f0880,80400,c78f08ec,0,0,...) at __lockmgr_args+0x834 vop_stdlock(f18cfb30,4,c0f01f07,80400,c78f0828,...) at vop_stdlock+0x65 VOP_LOCK1_APV(c1027b00,f18cfb30,c121b990,c1062980,c78f0828,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c78f0828,80400,c0f36c90,729,c78655a8,...) at _vn_lock+0x78 softdep_flushworklist(c78655a8,f18cfbdc,c7d542e0,5dc,0,...) at softdep_flushworklist+0x47 ffs_sync(c78655a8,1,f18cfc30,4ef,c7025000,...) at ffs_sync+0x2fd dounmount(c78655a8,8080000,c7d542e0,474,5b85c5ea,...) at dounmount+0x480 sys_unmount(c7d542e0,f18cfcec,c,c,206,...) at sys_unmount+0x331 syscall(f18cfd28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280de9ab, esp = 0xbfbfe64c, ebp = 0xbfbfe718 --- ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes Sep 25 20:29:03 x4 su: pho to root on /dev/pts/0 ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 92437 files, 23661424 used, 94652557 free (2181 frags, 11831297 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 92437 files, 23661424 used, 94652557 free (2181 frags, 11831297 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** fsck -y /tmp watchdogd. Starting inetd. Sun Sep 25 20:31:44 CEST 2011 FreeBSD/i386 (x4.osted.lan) (console) login: lock order reversal: 1st 0xe0b463a0 bufwait (bufwait) @ kern/vfs_bio.c:2658 2nd 0xc789d91c snaplk (snaplk) @ ufs/ffs/ffs_snapshot.c:818 KDB: stack backtrace: db_trace_self_wrapper(c0f078a8,632e746f,3831383a,f199000a,c0a5657b,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a4515b,c0f0b28b,c7165370,c7169818,f1994424,...) at kdb_backtrace+0x2a _witness_debugger(c0f0b28b,c789d91c,c0f36735,c7169818,c0f366bf,...) at _witness_debugger+0x25 witness_checkorder(c789d91c,9,c0f366b6,332,c80e937c,...) at witness_checkorder+0x839 __lockmgr_args(c789d91c,80400,c80e937c,0,0,...) at __lockmgr_args+0x834 ffs_lock(f1994548,e0b42fa0,100000,80400,c80e92b8,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c1052180,f1994548,e0b43048,c1062980,c80e92b8,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c80e92b8,80400,c0f366b6,332,0,...) at _vn_lock+0x78 ffs_snapshot(c8181000,c78358c0,c0f39b70,1a2,0,...) at ffs_snapshot+0x29b1 ffs_mount(c8181000,c789d900,ff,394,c7162450,...) at ffs_mount+0x1c13 vfs_donmount(c81672e0,11100,c786c880,c786c880,bfbfe1c8,...) at vfs_donmount+0x1254 sys_nmount(c81672e0,f1994cec,c0f5252a,c0f0bf03,213,...) at sys_nmount+0x84 syscall(f1994d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (378, FreeBSD ELF32, sys_nmount), eip = 0x280dc62b, esp = 0xbfbfe13c, ebp = 0xbfbfe8b8 --- lock order reversal: 1st 0xc789d91c snaplk (snaplk) @ ufs/ufs/ufs_vnops.c:957 2nd 0xc80e9310 ufs (ufs) @ ufs/ffs/ffs_snapshot.c:1620 KDB: stack backtrace: db_trace_self_wrapper(c0f078a8,a303236,a00,c0f10588,5e8,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a4515b,c0f0b28b,c7169818,c7169338,f18d8704,...) at kdb_backtrace+0x2a _witness_debugger(c0f0b28b,c80e9310,c0efa4e4,c7169338,c0f366bf,...) at _witness_debugger+0x25 witness_checkorder(c80e9310,9,c0f366b6,654,0,...) at witness_checkorder+0x839 __lockmgr_args(c80e9310,80000,0,0,0,...) at __lockmgr_args+0x834 ffs_snapremove(c80e92b8,c0f365d7,3,c7c98000,f18d8894,...) at ffs_snapremove+0x11f ffs_truncate(c80e92b8,0,0,c00,0,...) at ffs_truncate+0x5f7 ufs_inactive(f18d8adc,c0f58364,c80e937c,c80e937c,c80e92b8,...) at ufs_inactive+0x1fb VOP_INACTIVE_APV(c1052180,f18d8adc,c0f12fc1,94e,c1062940,...) at VOP_INACTIVE_APV+0xc5 vinactive(c1052180,f18d8b10,c0f12fc1,8a5,0,...) at vinactive+0xb1 vputx(f18d8c3c,c0aad443,c80e92b8,ffffffdf,c80e92b8,...) at vputx+0x36c vput(c80e92b8,ffffffdf,c80e92b8,f18d8b94,0,...) at vput+0x10 kern_unlinkat(c7c98000,ffffff9c,bfbfeae6,0,0,...) at kern_unlinkat+0x2a3 kern_unlink(c7c98000,bfbfeae6,0,f18d8d1c,c0d43814,...) at kern_unlink+0x2f sys_unlink(c7c98000,f18d8cec,c0f5252a,c0f0be1a,246,...) at sys_unlink+0x22 syscall(f18d8d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (10, FreeBSD ELF32, sys_unlink), eip = 0x2817386f, esp = 0xbfbfe81c, ebp = 0xbfbfe898 --- fsync: giving up on dirty 0xc78ef6cc: tag devfs, type VCHR usecount 1, writecount 0, refcount 18 mountedhere 0xc817f400 flags () v_object 0xc8127cc0 ref 0 pages 106 lock type devfs: EXCL by thread 0xc78022e0 (pid 3248) #0 0xc09f451e at __lockmgr_args+0xc1e #1 0xc0a94ca5 at vop_stdlock+0x65 #2 0xc0d66bd5 at VOP_LOCK1_APV+0xb5 #3 0xc0ab5e48 at _vn_lock+0x78 #4 0xc0c473d7 at softdep_flushworklist+0x47 #5 0xc0c4ad7d at ffs_sync+0x2fd #6 0xc0ab5c22 at vfs_write_suspend+0x122 #7 0xc0c2f08a at ffs_snapshot+0xcfa #8 0xc0c4d253 at ffs_mount+0x1c13 #9 0xc0a9e994 at vfs_donmount+0x1254 #10 0xc0a9f064 at sys_nmount+0x84 #11 0xc0d43814 at syscall+0x2a4 #12 0xc0d2cc71 at Xint0x80_syscall+0x21 dev md5a Expensive timeout(9) function: 0xc09fd680(0) 0.200052258 s ~KDB: enter: Break to debugger [ thread pid 11 tid 100003 ] Stopped at kdb_enter+0x3a: movl $0,kdb_why db> set $lines 0 db> bt 3252 Tracing pid 3252 tid 100110 td 0xc78a18a0 cpustop_handler(2,13,f18bd3e4,c0d43c26,c6f2acb4,...) at cpustop_handler+0x28 ipi_nmi_handler(c6f2acb4,c6f2ac80,f18bd39c,c0d4bf3d,c7d4184c,...) at ipi_nmi_handler+0x47 trap(f18bd3f0) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0aba009, esp = 0xf18bd430, ebp = 0xf18bd438 --- strncmp(c0f39b70,c0ef9420,3,c78a18a0,f18bd4ac,...) at strncmp+0x39 fixup_filename(c82d9a48,f18bd474,c0a5657b,c0f39b79,c0ef9420,...) at fixup_filename+0x2b witness_checkorder(c82edd00,9,c0f39b70,5c2,0,...) at witness_checkorder+0x68 _mtx_lock_flags(c82edd00,0,c0f39b70,5c2,0,...) at _mtx_lock_flags+0xc4 ffs_sync(c8181000,1,277,c0f14822,0,...) at ffs_sync+0x178 vfs_write_suspend(c8181000,f18bd870,0,8000,c715fe80,...) at vfs_write_suspend+0x122 ffs_snapshot(c8181000,c7584a20,0,0,0,...) at ffs_snapshot+0xcfa ffs_mount(c8181000,c7bbcc00,ff,394,c7162450,...) at ffs_mount+0x1c13 vfs_donmount(c78a18a0,11100,c8249500,c8249500,bfbfe1c8,...) at vfs_donmount+0x1254 sys_nmount(c78a18a0,f18bdcec,c0f5252a,c0f0bf03,213,...) at sys_nmount+0x84 syscall(f18bdd28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (378, FreeBSD ELF32, sys_nmount), eip = 0x280dc62b, esp = 0xbfbfe13c, ebp = 0xbfbfe8b8 --- db> c ~KDB: enter: Break to debugger [ thread pid 11 tid 100003 ] Stopped at kdb_enter+0x3a: movl $0,kdb_why db> c^h ^hbt 3252 Tracing pid 3252 tid 100110 td 0xc78a18a0 cpustop_handler(1,13,f18bd46c,c0d43c26,f18bd404,...) at cpustop_handler+0x28 ipi_nmi_handler(f18bd404,c0a5661c,c11e7360,c78a1950,c7d4184c,...) at ipi_nmi_handler+0x47 trap(f18bd478) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0a9c543, esp = 0xf18bd4b8, ebp = 0xf18bd4d4 --- __mnt_vnode_next(f18bd530,c8181000,c0f39b70,5cb,0,...) at __mnt_vnode_next+0x103 ffs_sync(c8181000,1,277,c0f14822,0,...) at ffs_sync+0x2b2 vfs_write_suspend(c8181000,f18bd870,0,8000,c715fe80,...) at vfs_write_suspend+0x122 ffs_snapshot(c8181000,c7584a20,0,0,0,...) at ffs_snapshot+0xcfa ffs_mount(c8181000,c7bbcc00,ff,394,c7162450,...) at ffs_mount+0x1c13 vfs_donmount(c78a18a0,11100,c8249500,c8249500,bfbfe1c8,...) at vfs_donmount+0x1254 sys_nmount(c78a18a0,f18bdcec,c0f5252a,c0f0bf03,213,...) at sys_nmount+0x84 syscall(f18bdd28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (378, FreeBSD ELF32, sys_nmount), eip = 0x280dc62b, esp = 0xbfbfe13c, ebp = 0xbfbfe8b8 --- db> c ~KDB: enter: Break to debugger [ thread pid 11 tid 100003 ] Stopped at kdb_enter+0x3a: movl $0,kdb_why db> c^h ^hbt 3252 Tracing pid 3252 tid 100110 td 0xc78a18a0 cpustop_handler(3,13,f18bd408,c0d43c26,c71627f8,...) at cpustop_handler+0x28 ipi_nmi_handler(c71627f8,c71611a0,4,0,c7d4184c,...) at ipi_nmi_handler+0x47 trap(f18bd414) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc09fbba0, esp = 0xf18bd454, ebp = 0xf18bd474 --- _mtx_assert(c121b990,0,c0f36c90,611,c78a18a0,...) at _mtx_assert softdep_process_worklist(c8181000,1,f18bd4d4,c09fc353,c8181000,...) at softdep_process_worklist+0x191 softdep_flushworklist(c8181000,f18bd52c,c78a18a0,5dc,0,...) at softdep_flushworklist+0xac ffs_sync(c8181000,1,277,c0f14822,0,...) at ffs_sync+0x2fd vfs_write_suspend(c8181000,f18bd870,0,8000,c715fe80,...) at vfs_write_suspend+0x122 ffs_snapshot(c8181000,c7584a20,0,0,0,...) at ffs_snapshot+0xcfa ffs_mount(c8181000,c7bbcc00,ff,394,c7162450,...) at ffs_mount+0x1c13 vfs_donmount(c78a18a0,11100,c8249500,c8249500,bfbfe1c8,...) at vfs_donmount+0x1254 sys_nmount(c78a18a0,f18bdcec,c0f5252a,c0f0bf03,213,...) at sys_nmount+0x84 syscall(f18bdd28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (378, FreeBSD ELF32, sys_nmount), eip = 0x280dc62b, esp = 0xbfbfe13c, ebp = 0xbfbfe8b8 --- db> bt 3252^h^h^h^h^h^h^h ^h^h^h^h^h^h^hc ~KDB: enter: Break to debugger [ thread pid 11 tid 100003 ] Stopped at kdb_enter+0x3a: movl $0,kdb_why db> c^h ^hbt 3252 Tracing pid 3252 tid 100110 td 0xc78a18a0 cpustop_handler(3,13,f18bd3fc,c0d43c26,0,...) at cpustop_handler+0x28 ipi_nmi_handler(0,32dc8852,1,c71627f8,c7d4184c,...) at ipi_nmi_handler+0x47 trap(f18bd408) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0a556d0, esp = 0xf18bd448, ebp = 0xf18bd450 --- _isitmyx(c82d9a48,f18bd474,c0a5657b,c0f39b79,c0ef9420,...) at _isitmyx witness_checkorder(c82edd00,9,c0f39b70,5c2,0,...) at witness_checkorder+0x256 _mtx_lock_flags(c82edd00,0,c0f39b70,5c2,0,...) at _mtx_lock_flags+0xc4 ffs_sync(c8181000,1,277,c0f14822,0,...) at ffs_sync+0x178 vfs_write_suspend(c8181000,f18bd870,0,8000,c715fe80,...) at vfs_write_suspend+0x122 ffs_snapshot(c8181000,c7584a20,0,0,0,...) at ffs_snapshot+0xcfa ffs_mount(c8181000,c7bbcc00,ff,394,c7162450,...) at ffs_mount+0x1c13 vfs_donmount(c78a18a0,11100,c8249500,c8249500,bfbfe1c8,...) at vfs_donmount+0x1254 sys_nmount(c78a18a0,f18bdcec,c0f5252a,c0f0bf03,213,...) at sys_nmount+0x84 syscall(f18bdd28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (378, FreeBSD ELF32, sys_nmount), eip = 0x280dc62b, esp = 0xbfbfe13c, ebp = 0xbfbfe8b8 --- db> c ~KDB: enter: Break to debugger [ thread pid 11 tid 100003 ] Stopped at kdb_enter+0x3a: movl $0,kdb_why db> run pho db:0:pho> bt Tracing pid 11 tid 100003 td 0xc71bf8a0 kdb_enter(c0d81b3e,c0f07859,c6f37ba4,c087c8cb,c7574000,...) at kdb_enter+0x3a kdb_break(c7574000,0,c124ab20,c7574080,c7574058,...) at kdb_break+0x28 uart_intr(c7574000,0,c124ab20,c6f37be8,0,...) at uart_intr+0x11b intr_event_handle(c7208d80,c6f37c10,c6f37c34,0,c73ed200,...) at intr_event_handle+0x76 intr_execute_handlers(c72070d0,c6f37c10,0,c6f37c50,c0d2cfd5,...) at intr_execute_handlers+0x49 lapic_handle_intr(3c,c6f37c10) at lapic_handle_intr+0x36 Xapic_isr1() at Xapic_isr1+0x35 --- interrupt, eip = 0xc0d285d5, esp = 0xc6f37c50, ebp = 0xc6f37c50 --- acpi_cpu_c1(c6f37c6c,c1064050,0,301,935,...) at acpi_cpu_c1+0x5 acpi_cpu_idle(0,c6f37ca0,c0d36809,0,ffffffff,...) at acpi_cpu_idle+0x122 cpu_idle_acpi(0,ffffffff,c10abd40,2,c6f37ce8,...) at cpu_idle_acpi+0x2f cpu_idle(0,c6f37cc4,c0f0598b,a05,c71bf8a0,...) at cpu_idle+0x89 sched_idletd(0,c6f37d28,c0efe270,3e5,c71bd84c,...) at sched_idletd+0x263 fork_exit(c0a32300,0,c6f37d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f37d60, ebp = 0 --- db:0:bt> show allpcpu Current CPU: 0 cpuid = 0 dynamic pcpu = 0x7c1f80 curthread = 0xc71bf8a0: pid 11 "idle: cpu0" curpcb = 0xc6f37d80 fpcurthread = none idlethread = 0xc71bf8a0: tid 100003 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: cpuid = 1 dynamic pcpu = 0x5ec3f80 curthread = 0xc720b000: pid 12 "swi4: clock" curpcb = 0xc6f4ad80 fpcurthread = none idlethread = 0xc71bf5c0: tid 100004 "idle: cpu1" APIC ID = 1 currentldt = 0x50 spin locks held: cpuid = 2 dynamic pcpu = 0x5ec6f80 curthread = 0xc71bf2e0: pid 11 "idle: cpu2" curpcb = 0xc6f3dd80 fpcurthread = none idlethread = 0xc71bf2e0: tid 100005 "idle: cpu2" APIC ID = 2 currentldt = 0x50 spin locks held: cpuid = 3 dynamic pcpu = 0x5ec9f80 curthread = 0xc78a18a0: pid 3252 "mksnap_ffs" curpcb = 0xf18bdd80 fpcurthread = none idlethread = 0xc71bf000: tid 100006 "idle: cpu3" APIC ID = 3 currentldt = 0x50 spin locks held: db:0:allpcpu> show alllocks Process 3255 (sshd) thread 0xc81482e0 (100190) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xc7d8570c) locked @ kern/uipc_sockbuf.c:148 Process 3254 (sshd) thread 0xc78a72e0 (100084) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xc803b70c) locked @ kern/uipc_sockbuf.c:148 Process 3253 (sshd) thread 0xc8164b80 (100182) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xc7e7e56c) locked @ kern/uipc_sockbuf.c:148 Process 3252 (mksnap_ffs) thread 0xc78a18a0 (100110) exclusive sleep mutex vnode interlock (vnode interlock) r = 0 (0xc837d37c) locked @ ufs/ffs/ffs_vfsops.c:1474 exclusive sleep mutex struct mount mtx (struct mount mtx) r = 0 (0xc8181000) locked @ kern/vfs_mount.c:1687 exclusive sleep mutex Giant (Giant) r = 0 (0xc109d7d0) locked @ kern/vfs_mount.c:1069 Process 2152 (sshd) thread 0xc7802000 (100088) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xc7e04a4c) locked @ kern/uipc_sockbuf.c:148 db:0:alllocks> show lockedvnods Locked vnodes db:0:lockedvnods> show mount 0xc7865b50 /dev/ad4s1a on / (ufs) 0xc7866000 devfs on /dev (devfs) 0xc786587c /dev/ad4s1f on /home (ufs) 0xc78652d4 /dev/ad4s1d on /usr (ufs) 0xc7865000 /dev/ad4s1e on /var (ufs) 0xc7ba35a8 procfs on /proc (procfs) 0xc7ba32d4 /dev/label/tmp on /tmp (ufs) 0xc8181000 /dev/md5a on /mnt (ufs) More info: show mount db:0:mount> ps pid ppid pgrp uid state wmesg wchan cmd 5725 3265 3263 1001 S nanslp 0xc109f0e4 sleep 3266 3263 3263 1001 S piperd 0xc811cc40 awk 3265 3263 3263 1001 S wait 0xc81e62c4 sh 3264 3262 3264 1001 Ss+ select 0xc819bd24 top 3263 3261 3263 1001 Ss wait 0xc81e6000 sh 3262 3253 3253 1001 S select 0xc7b83224 sshd 3261 3254 3254 1001 S select 0xc81be1a4 sshd 3260 3259 3260 1001 Ss kqread 0xc8290d00 tail 3259 3255 3255 1001 S select 0xc7b82964 sshd 3255 2093 3255 0 Ss sbwait 0xc7d85738 sshd 3254 2093 3254 0 Ss sbwait 0xc803b738 sshd 3253 2093 3253 0 Ss sbwait 0xc7e7e598 sshd 3252 3250 2253 0 R+ CPU 3 mksnap_ffs 3251 2253 2253 0 S+ piperd 0xc7cb8ab8 grep 3250 2253 2253 0 S+ wait 0xc7d42588 sh 2288 2286 2275 1001 S+ suspfs 0xc8181048 creat 2287 2286 2275 1001 D+ suspfs 0xc8181048 creat 2286 2285 2275 1001 S+ wait 0xc812984c creat 2285 2284 2275 1001 S+ wait 0xc8129588 creat 2284 2283 2275 1001 S+ wait 0xc8140b10 run 2283 2278 2275 1001 S+ wait 0xc81292c4 run 2278 2275 2275 1001 S+ wait 0xc8128b10 run 2275 2271 2275 1001 S+ wait 0xc812884c bash 2271 2253 2253 0 S+ wait 0xc7e8c2c4 su 2266 0 0 0 DL mdwait 0xc817a800 [md5] 2253 2165 2253 0 S+ wait 0xc78e52c4 sh 2252 1 2252 0 Ss+ ttyin 0xc758b070 getty 2251 1 2251 0 Ss+ ttyin 0xc758b270 getty 2250 1 2250 0 Ss+ ttyin 0xc758b470 getty 2249 1 2249 0 Ss+ ttyin 0xc72eb070 getty 2248 1 2248 0 Ss+ ttyin 0xc72eb270 getty 2247 1 2247 0 Ss+ ttyin 0xc72eb470 getty 2246 1 2246 0 Ss+ ttyin 0xc72eb670 getty 2245 1 2245 0 Ss+ ttyin 0xc72eb870 getty 2244 1 2244 0 Ss+ ttyin 0xc72eba70 getty 2205 1 2205 0 Ss select 0xc7926424 inetd 2165 2161 2165 0 S+ wait 0xc7c92588 bash 2161 2160 2161 0 S+ pause 0xc7e8c8a4 csh 2160 2155 2160 1001 S+ wait 0xc78f62c4 su 2155 2154 2155 1001 Ss+ wait 0xc7e8c588 bash 2154 2152 2152 1001 S select 0xc77e0824 sshd 2152 2093 2152 0 Ss sbwait 0xc7e04a78 sshd 2113 1 2113 0 Ss nanslp 0xc109f0e4 cron 2105 1 2105 25 Ss pause 0xc7d4131c sendmail 2101 1 2101 0 Ss select 0xc77cd8e4 sendmail 2093 1 2093 0 Ss select 0xc77e0ce4 sshd 2033 1 2033 0 Ss select 0xc77e0d24 moused 1955 1 1955 0 Ss select 0xc7b833e4 ntpd 1854 1848 1848 0 S (threaded) nfsd 100130 S rpcsvc 0xc7415a90 nfsd: service 100129 S rpcsvc 0xc7415ad0 nfsd: service 100128 S rpcsvc 0xc7415b10 nfsd: service 100092 S rpcsvc 0xc7415b50 nfsd: master 1848 1 1848 0 Ss select 0xc77cdb24 nfsd 1838 1 1838 0 Ss select 0xc79261e4 mountd 1738 1 1738 0 Ss select 0xc7926224 rpcbind 1709 1 1709 0 Ss select 0xc7926264 syslogd 1509 1 1509 0 Ss select 0xc77cd724 devd 18 0 0 0 DL sdflush 0xc121b9e0 [softdepflush] 17 0 0 0 DL vlruwt 0xc77f7588 [vnlru] 16 0 0 0 DL syncer 0xc12152d4 [syncer] 9 0 0 0 DL psleep 0xc1214f68 [bufdaemon] 8 0 0 0 DL pgzero 0xc121d85c [pagezero] 7 0 0 0 DL psleep 0xc121d448 [vmdaemon] 6 0 0 0 DL psleep 0xc121d410 [pagedaemon] 5 0 0 0 DL ccb_scan 0xc1063dd4 [xpt_thrd] 4 0 0 0 DL waiting_ 0xc1216df8 [sctp_iterator] 3 0 0 0 DL - 0xc73ede3c [fdc0] 2 0 0 0 SL - 0xc7473000 [fw0_probe] 15 0 0 0 DL (threaded) [usb] 100058 D - 0xc7454d34 [usbus5] 100057 D - 0xc7454d04 [usbus5] 100056 D - 0xc7454cd4 [usbus5] 100055 D - 0xc7454ca4 [usbus5] 100053 D - 0xc7449b5c [usbus4] 100052 D - 0xc7449b2c [usbus4] 100051 D - 0xc7449afc [usbus4] 100050 D - 0xc7449acc [usbus4] 100049 D - 0xc7444b5c [usbus3] 100048 D - 0xc7444b2c [usbus3] 100047 D - 0xc7444afc [usbus3] 100046 D - 0xc7444acc [usbus3] 100045 D - 0xc7437b5c [usbus2] 100044 D - 0xc7437b2c [usbus2] 100043 D - 0xc7437afc [usbus2] 100042 D - 0xc7437acc [usbus2] 100040 D - 0xc7432b5c [usbus1] 100039 D - 0xc7432b2c [usbus1] 100038 D - 0xc7432afc [usbus1] 100037 D - 0xc7432acc [usbus1] 100035 D - 0xc7428b5c [usbus0] 100034 D - 0xc7428b2c [usbus0] 100033 D - 0xc7428afc [usbus0] 100032 D - 0xc7428acc [usbus0] 14 0 0 0 DL - 0xc109dee4 [yarrow] 13 0 0 0 DL (threaded) [geom] 100015 D - 0xc109bb08 [g_down] 100014 D - 0xc109bb04 [g_up] 100013 D - 0xc109bafc [g_event] 12 0 0 0 RL (threaded) [intr] 100066 I [irq12: psm0] 100065 I [irq1: atkbd0] 100063 I [swi0: uart] 100060 I [irq20: fwohci0] 100059 I [irq14: ata0] 100054 I [irq19: ehci0] 100041 I [irq18: ohci2 ohci4] 100036 I [irq17: ohci1 ohci3] 100031 I [irq16: hdac1 ohci0] 100030 I [irq22: ahci0] 100029 I [irq257: re0] 100028 I [irq256: hdac0] 100026 I [swi5: +] 100025 I [swi2: cambio] 100019 I [swi6: task queue] 100018 I [swi6: Giant taskq] 100012 I [swi4: clock] 100011 I [swi4: clock] 100010 I [swi4: clock] 100009 Run CPU 1 [swi4: clock] 100008 I [swi3: vm] 100007 I [swi1: netisr 0] 11 0 0 0 RL (threaded) [idle] 100006 CanRun [idle: cpu3] 100005 Run CPU 2 [idle: cpu2] 100004 CanRun [idle: cpu1] 100003 Run CPU 0 [idle: cpu0] 1 0 1 0 SLs wait 0xc71bdb10 [init] 10 0 0 0 DL audit_wo 0xc121b100 [audit] 0 0 0 0 DLs (threaded) [kernel] 100077 D - 0xc71c3200 [mca taskq] 100067 D - 0xc109dee4 [deadlkres] 100061 D - 0xc71c3b00 [fw0_taskq] 100027 D - 0xc71c2580 [thread taskq] 100024 D - 0xc71c2600 [kqueue taskq] 100023 D - 0xc71c2780 [acpi_task_2] 100022 D - 0xc71c2780 [acpi_task_1] 100021 D - 0xc71c2780 [acpi_task_0] 100020 D - 0xc71c27c0 [ffs_trim taskq] 100016 D - 0xc71c2880 [firmware taskq] 100000 D sched 0xc109bbe0 [swapper] db:0:ps> allt Tracing command sleep pid 5725 tid 100115 td 0xc7d545c0 sched_switch(c7d545c0,0,104,191,6084bc2c,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c7d545c0,0,c0f091e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(2711,c0a4b930,c7d545c0,1,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c109f0e4,6c,c0f04eb7,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c109f0e4,0,16c,c0f04eb7,2711,...) at _sleep+0x328 kern_nanosleep(c7d545c0,f18ccc50,f18ccc58,a,0,...) at kern_nanosleep+0xc1 sys_nanosleep(c7d545c0,f18cccec,c0f5252a,c0f0c7b4,246,...) at sys_nanosleep+0x6f syscall(f18ccd28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x28167817, esp = 0xbfbfec5c, ebp = 0xbfbfeca8 --- Tracing command awk pid 3266 tid 100188 td 0xc81488a0 sched_switch(c81488a0,0,104,191,6042c23c,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c81488a0,0,c0f091e0,1a7,5c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c811cc40,5c,c0f0b997,100,0,...) at sleepq_wait_sig+0x17 _sleep(c811cc40,c811cdb0,15c,c0f0b997,0,...) at _sleep+0x35c pipe_read(c7cb7e38,f19c8c44,c8258c80,0,c81488a0,...) at pipe_read+0x417 dofileread(f19c8c44,ffffffff,ffffffff,0,f19c8c20,...) at dofileread+0x9e kern_readv(c81488a0,0,f19c8c44,f19c8c64,1,...) at kern_readv+0x68 sys_read(c81488a0,f19c8cec,c0f5252a,f19c8d80,212,...) at sys_read+0x4f syscall(f19c8d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281ba2e3, esp = 0xbfbfe93c, ebp = 0xbfbfe958 --- Tracing command sh pid 3265 tid 100186 td 0xc8164000 sched_switch(c8164000,0,104,191,60588dfe,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c8164000,0,c0f091e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c81e62c4,6c,c0f0bf5f,100,0,...) at sleepq_wait_sig+0x17 _sleep(c81e62c4,c81e634c,16c,c0f0bf5f,0,...) at _sleep+0x35c kern_wait(c8164000,ffffffff,f19c2c60,2,0,...) at kern_wait+0x626 sys_wait4(c8164000,f19c2cec,c0f5252a,c0f0bf3e,212,...) at sys_wait4+0x3b syscall(f19c2d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2816c5cb, esp = 0xbfbfe97c, ebp = 0xbfbfe998 --- Tracing command top pid 3264 tid 100161 td 0xc81468a0 sched_switch(c81468a0,0,104,191,8b45b294,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c81468a0,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f1973a30,c09fc3da,c819bd10,0,c81468a0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c819bd24,0,f1973a68,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c819bd24,c819bd10,3e9,668,c83d3700,...) at _cv_timedwait_sig+0x252 seltdwait(f1973c14,f1973c1c,c8258d80,c81468a0,32333b30,...) at seltdwait+0x8a kern_select(c81468a0,2,bfbfebac,0,0,f1973c5c,20,1,0) at kern_select+0x504 sys_select(c81468a0,f1973cec,c0f5252a,c0ed3e3f,246,...) at sys_select+0x66 syscall(f1973d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281f0263, esp = 0xbfbfeb4c, ebp = 0xbfbfeca8 --- Tracing command sh pid 3263 tid 100187 td 0xc8148b80 sched_switch(c8148b80,0,104,191,b217a020,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c8148b80,0,c0f091e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c81e6000,6c,c0f0bf5f,100,0,...) at sleepq_wait_sig+0x17 _sleep(c81e6000,c81e6088,16c,c0f0bf5f,0,...) at _sleep+0x35c kern_wait(c8148b80,ffffffff,f19c5c60,2,0,...) at kern_wait+0x626 sys_wait4(c8148b80,f19c5cec,c,c,212,...) at sys_wait4+0x3b syscall(f19c5d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2816c5cb, esp = 0xbfbfeb2c, ebp = 0xbfbfeb48 --- Tracing command sshd pid 3262 tid 100096 td 0xc78012e0 sched_switch(c78012e0,0,104,191,8b4928b1,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c78012e0,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c09fc3da,c7b83210,0,c0f01f07,c78012e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7b83224,0,f183da68,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7b83224,c7b83210,c0f0b7da,668,c83d3738,...) at _cv_wait_sig+0x243 seltdwait(c83d3738,58,c8291380,c78012e0,c0f2c3bb,...) at seltdwait+0xa2 kern_select(c78012e0,a,288060b4,288060bc,0,0,20,c78e6000,0) at kern_select+0x504 sys_select(c78012e0,f183dcec,c0f5252a,c0f0c0b3,206,...) at sys_select+0x66 syscall(f183dd28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284c6263, esp = 0xbfbfde2c, ebp = 0xbfbfde78 --- Tracing command sshd pid 3261 tid 100163 td 0xc81462e0 sched_switch(c81462e0,0,104,191,af1dab6f,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c81462e0,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c09fc3da,c81be190,0,c0f01f07,c81462e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c81be1a4,0,f1979a68,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c81be1a4,c81be190,c0f0b7da,668,c8269070,...) at _cv_wait_sig+0x243 seltdwait(c8269070,58,c823a780,c81462e0,28823000,...) at seltdwait+0xa2 kern_select(c81462e0,c,288060b4,288060bc,0,0,20,c813eb10,0) at kern_select+0x504 sys_select(c81462e0,f1979cec,c0f5252a,c0f0c0b3,206,...) at sys_select+0x66 syscall(f1979d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284c6263, esp = 0xbfbfde2c, ebp = 0xbfbfde78 --- Tracing command tail pid 3260 tid 100080 td 0xc78a78a0 sched_switch(c78a78a0,0,104,191,8b1fc0bf,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,68,...) at mi_switch+0x1eb sleepq_switch(c78a78a0,0,c0f091e0,1a7,68,...) at sleepq_switch+0x162 sleepq_catch_signals(3e9,c0a4b930,c78a78a0,2,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c8290d00,68,c0efde34,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c8290d00,c8290d00,168,c0efde34,3e9,...) at _sleep+0x328 kern_kevent(c78a78a0,4,0,1,f17f5c44,...) at kern_kevent+0x374 sys_kevent(c78a78a0,f17f5cec,c0f5252a,c0d712c1,286,...) at sys_kevent+0x19e syscall(f17f5d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (363, FreeBSD ELF32, sys_kevent), eip = 0x2816b94b, esp = 0xbfbfeb1c, ebp = 0xbfbfebd8 --- Tracing command sshd pid 3259 tid 100162 td 0xc81465c0 sched_switch(c81465c0,0,104,191,bbeda852,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c81465c0,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c09fc3da,c7b82950,0,c0f01f07,c81465c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7b82964,0,f1976a68,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7b82964,c7b82950,c0f0b7da,668,c791e658,...) at _cv_wait_sig+0x243 seltdwait(c791e658,58,c8258e00,c81465c0,c0f2c3bb,...) at seltdwait+0xa2 kern_select(c81465c0,c,288060b4,288060bc,0,0,20,c8140000,0) at kern_select+0x504 sys_select(c81465c0,f1976cec,c0f5252a,c0f0c0b3,206,...) at sys_select+0x66 syscall(f1976d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284c6263, esp = 0xbfbfde2c, ebp = 0xbfbfde78 --- Tracing command sshd pid 3255 tid 100190 td 0xc81482e0 sched_switch(c81482e0,0,104,191,98fc64af,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,68,...) at mi_switch+0x1eb sleepq_switch(c81482e0,0,c0f091e0,1a7,68,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7d85738,68,c0f0f601,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7d85738,c7d856f8,168,c0f0f601,0) at _sleep+0x35c sbwait(c7d856d4,4,c0f0f6d9,5ee,c7d856f8,...) at sbwait+0x76 soreceive_generic(c7d85680,0,f19cec44,0,0,...) at soreceive_generic+0x320 soreceive(c7d85680,0,f19cec44,0,0,0) at soreceive+0x38 soo_read(c791e348,f19cec44,c8291280,0,c81482e0,...) at soo_read+0x4e dofileread(f19cec44,ffffffff,ffffffff,0,f19cec20,...) at dofileread+0x9e kern_readv(c81482e0,5,f19cec44,f19cec64,1,...) at kern_readv+0x68 sys_read(c81482e0,f19cecec,c0f5252a,c0d712c1,282,...) at sys_read+0x4f syscall(f19ced28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x284c62e3, esp = 0xbfbfddec, ebp = 0xbfbfde28 --- Tracing command sshd pid 3254 tid 100084 td 0xc78a72e0 sched_switch(c78a72e0,0,104,191,aedc220b,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,68,...) at mi_switch+0x1eb sleepq_switch(c78a72e0,0,c0f091e0,1a7,68,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c803b738,68,c0f0f601,100,0,...) at sleepq_wait_sig+0x17 _sleep(c803b738,c803b6f8,168,c0f0f601,0) at _sleep+0x35c sbwait(c803b6d4,4,c0f0f6d9,5ee,c803b6f8,...) at sbwait+0x76 soreceive_generic(c803b680,0,f1807c44,0,0,...) at soreceive_generic+0x320 soreceive(c803b680,0,f1807c44,0,0,0) at soreceive+0x38 soo_read(c83d38f8,f1807c44,c81b7080,0,c78a72e0,...) at soo_read+0x4e dofileread(f1807c44,ffffffff,ffffffff,0,f1807c20,...) at dofileread+0x9e kern_readv(c78a72e0,5,f1807c44,f1807c64,1,...) at kern_readv+0x68 sys_read(c78a72e0,f1807cec,c0f5252a,c0d712c1,282,...) at sys_read+0x4f syscall(f1807d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x284c62e3, esp = 0xbfbfddec, ebp = 0xbfbfde28 --- Tracing command sshd pid 3253 tid 100182 td 0xc8164b80 sched_switch(c8164b80,0,104,191,afa5fd6e,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,68,...) at mi_switch+0x1eb sleepq_switch(c8164b80,0,c0f091e0,1a7,68,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7e7e598,68,c0f0f601,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7e7e598,c7e7e558,168,c0f0f601,0) at _sleep+0x35c sbwait(c7e7e534,4,c0f0f6d9,5ee,c7e7e558,...) at sbwait+0x76 soreceive_generic(c7e7e4e0,0,f19b6c44,0,0,...) at soreceive_generic+0x320 soreceive(c7e7e4e0,0,f19b6c44,0,0,0) at soreceive+0x38 soo_read(c8269498,f19b6c44,c8291000,0,c8164b80,...) at soo_read+0x4e dofileread(f19b6c44,ffffffff,ffffffff,0,f19b6c20,...) at dofileread+0x9e kern_readv(c8164b80,5,f19b6c44,f19b6c64,1,...) at kern_readv+0x68 sys_read(c8164b80,f19b6cec,c,c,282,...) at sys_read+0x4f syscall(f19b6d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x284c62e3, esp = 0xbfbfddec, ebp = 0xbfbfde28 --- Tracing command mksnap_ffs pid 3252 tid 100110 td 0xc78a18a0 cpustop_handler(3,13,f18bd444,c0d43c26,c0a5657b,...) at cpustop_handler+0x28 ipi_nmi_handler(c0a5657b,c0f3c165,c0ef9420,3,c7d4184c,...) at ipi_nmi_handler+0x47 trap(f18bd450) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0a56750, esp = 0xf18bd490, ebp = 0xf18bd4ac --- witness_lock(c837d37c,8,c0f39b70,5c2,0,...) at witness_lock+0x1c0 _mtx_lock_flags(c837d37c,0,c0f39b70,5c2,0,...) at _mtx_lock_flags+0x113 ffs_sync(c8181000,1,277,c0f14822,0,...) at ffs_sync+0x178 vfs_write_suspend(c8181000,f18bd870,0,8000,c715fe80,...) at vfs_write_suspend+0x122 ffs_snapshot(c8181000,c7584a20,0,0,0,...) at ffs_snapshot+0xcfa ffs_mount(c8181000,c7bbcc00,ff,394,c7162450,...) at ffs_mount+0x1c13 vfs_donmount(c78a18a0,11100,c8249500,c8249500,bfbfe1c8,...) at vfs_donmount+0x1254 sys_nmount(c78a18a0,f18bdcec,c0f5252a,c0f0bf03,213,...) at sys_nmount+0x84 syscall(f18bdd28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (378, FreeBSD ELF32, sys_nmount), eip = 0x280dc62b, esp = 0xbfbfe13c, ebp = 0xbfbfe8b8 --- Tracing command grep pid 3251 tid 100158 td 0xc8146b80 sched_switch(c8146b80,0,104,191,6b858d53,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c8146b80,0,c0f091e0,1a7,5c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7cb8ab8,5c,c0f0b997,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7cb8ab8,c7cb8c28,15c,c0f0b997,0,...) at _sleep+0x35c pipe_read(c7d6b888,f196ac44,c7e08200,0,c8146b80,...) at pipe_read+0x417 dofileread(f196ac44,ffffffff,ffffffff,0,f196ac20,...) at dofileread+0x9e kern_readv(c8146b80,0,f196ac44,f196ac64,1,...) at kern_readv+0x68 sys_read(c8146b80,f196acec,c0f5252a,c0f0be97,246,...) at sys_read+0x4f syscall(f196ad28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281c92e3, esp = 0xbfbfe68c, ebp = 0xbfbfe718 --- Tracing command sh pid 3250 tid 100143 td 0xc7e8e5c0 sched_switch(c7e8e5c0,0,104,191,6af73705,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c7e8e5c0,0,c0f091e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7d42588,6c,c0f0bf5f,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7d42588,c7d42610,16c,c0f0bf5f,0,...) at _sleep+0x35c kern_wait(c7e8e5c0,ffffffff,f1923c60,2,0,...) at kern_wait+0x626 sys_wait4(c7e8e5c0,f1923cec,c0f5252a,c0f0bf3e,212,...) at sys_wait4+0x3b syscall(f1923d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2816c5cb, esp = 0xbfbfe5cc, ebp = 0xbfbfe5e8 --- Tracing command creat pid 2288 tid 100154 td 0xc81478a0 sched_switch(c81478a0,0,104,191,72736c48,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,77,...) at mi_switch+0x1eb sleepq_switch(c81478a0,0,c0f091e0,1a7,77,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8181048,77,c0f1481b,100,0,...) at sleepq_wait_sig+0x17 _sleep(c8181048,c8181000,177,c0f1481b,0,...) at _sleep+0x35c vn_start_write(0,f195ec30,104,f195ebf8,804cc80,...) at vn_start_write+0xdd kern_rmdirat(c81478a0,ffffff9c,804cc80,0,f195ec6c,...) at kern_rmdirat+0x1a1 kern_rmdir(c81478a0,804cc80,0,f195ed1c,c0d43814,...) at kern_rmdir+0x27 sys_rmdir(c81478a0,f195ecec,c0f5252a,c0f345d1,206,...) at sys_rmdir+0x22 syscall(f195ed28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (137, FreeBSD ELF32, sys_rmdir), eip = 0x280f056b, esp = 0xbfbfe62c, ebp = 0xbfbfe648 --- Tracing command creat pid 2287 tid 100155 td 0xc81475c0 sched_switch(c81475c0,0,104,191,6c2adcba,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,77,...) at mi_switch+0x1eb sleepq_switch(c81475c0,0,c0f091e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c8181048,77,c0f1481b,0,0,...) at sleepq_wait+0x63 _sleep(c8181048,c8181000,77,c0f1481b,0,...) at _sleep+0x372 vn_start_write(c82e0984,f1961b44,1,c09cf35d,2,...) at vn_start_write+0xdd vn_close(c82e0984,2,c7921780,c81475c0,520cc4c,...) at vn_close+0x87 vn_closefile(c7cb7658,c81475c0,c0ef9420,0,c7cb7658,...) at vn_closefile+0xe4 _fdrop(c7cb7658,c81475c0,0,f1961c18,0,c11e7198,c8147670,c11e7198,c1038760,c0efd2f1,c81475c0,c778fe2c,4ce,f1961c28,c0a143a7,c778fe2c,8,c0efd2e8,c7cb7658) at _fdrop+0x43 closef(c7cb7658,c81475c0,4ce,f1961c4c,c778fe2c,...) at closef+0x2b0 kern_close(c81475c0,3,f1961d1c,c0d43814,c81475c0,...) at kern_close+0x139 sys_close(c81475c0,f1961cec,c0f5252a,c0ef3ca5,213,...) at sys_close+0x1a syscall(f1961d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (6, FreeBSD ELF32, sys_close), eip = 0x281972a3, esp = 0xbfbfe73c, ebp = 0xbfbfe7f8 --- Tracing command creat pid 2286 tid 100169 td 0xc7d52000 sched_switch(c7d52000,0,104,191,7235a8eb,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c7d52000,0,c0f091e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c812984c,6c,c0f0bf5f,100,0,...) at sleepq_wait_sig+0x17 _sleep(c812984c,c81298d4,16c,c0f0bf5f,0,...) at _sleep+0x35c kern_wait(c7d52000,8ef,f198bc60,0,0,...) at kern_wait+0x626 sys_wait4(c7d52000,f198bcec,c0f5252a,c0f0c7c6,212,...) at sys_wait4+0x3b syscall(f198bd28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x281065cb, esp = 0xbfbfe7dc, ebp = 0xbfbfe7f8 --- Tracing command creat pid 2285 tid 100170 td 0xc7c99b80 sched_switch(c7c99b80,0,104,191,72361ccd,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c7c99b80,0,c0f091e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8129588,6c,c0f0bf5f,100,0,...) at sleepq_wait_sig+0x17 _sleep(c8129588,c8129610,16c,c0f0bf5f,0,...) at _sleep+0x35c kern_wait(c7c99b80,8ee,f198ec60,0,0,...) at kern_wait+0x626 sys_wait4(c7c99b80,f198ecec,c,c,212,...) at sys_wait4+0x3b syscall(f198ed28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x281065cb, esp = 0xbfbfe7dc, ebp = 0xbfbfe7f8 --- Tracing command run pid 2284 tid 100156 td 0xc81472e0 sched_switch(c81472e0,0,104,191,770c588f,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c81472e0,0,c0f091e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8140b10,6c,c0f0bf5f,100,0,...) at sleepq_wait_sig+0x17 _sleep(c8140b10,c8140b98,16c,c0f0bf5f,0,...) at _sleep+0x35c kern_wait(c81472e0,8ed,f1964c60,0,0,...) at kern_wait+0x626 sys_wait4(c81472e0,f1964cec,c0f5252a,c0f0c7c6,212,...) at sys_wait4+0x3b syscall(f1964d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x281055cb, esp = 0xbfbfe29c, ebp = 0xbfbfe2b8 --- Tracing command run pid 2283 tid 100171 td 0xc7c998a0 sched_switch(c7c998a0,0,104,191,770b8f51,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c7c998a0,0,c0f091e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c81292c4,6c,c0f0bf5f,100,0,...) at sleepq_wait_sig+0x17 _sleep(c81292c4,c812934c,16c,c0f0bf5f,0,...) at _sleep+0x35c kern_wait(c7c998a0,8ec,f1991c60,0,0,...) at kern_wait+0x626 sys_wait4(c7c998a0,f1991cec,c0f5252a,c0f0c7c6,212,...) at sys_wait4+0x3b syscall(f1991d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x281055cb, esp = 0xbfbfe7bc, ebp = 0xbfbfe7d8 --- Tracing command run pid 2278 tid 100173 td 0xc8167000 sched_switch(c8167000,0,104,191,62dfc6c7,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c8167000,0,c0f091e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8128b10,6c,c0f0bf5f,100,0,...) at sleepq_wait_sig+0x17 _sleep(c8128b10,c8128b98,16c,c0f0bf5f,0,...) at _sleep+0x35c kern_wait(c8167000,8eb,f1997c60,0,0,...) at kern_wait+0x626 sys_wait4(c8167000,f1997cec,c,c,212,...) at sys_wait4+0x3b syscall(f1997d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x281055cb, esp = 0xbfbfe7bc, ebp = 0xbfbfe7d8 --- Tracing command bash pid 2275 tid 100174 td 0xc8166b80 sched_switch(c8166b80,0,104,191,35b5315e,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c8166b80,0,c0f091e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c812884c,6c,c0f0bf5f,100,0,...) at sleepq_wait_sig+0x17 _sleep(c812884c,c81288d4,16c,c0f0bf5f,0,...) at _sleep+0x35c kern_wait(c8166b80,ffffffff,f199ac60,0,0,...) at kern_wait+0x626 sys_wait4(c8166b80,f199acec,c,c,212,...) at sys_wait4+0x3b syscall(f199ad28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x282c55cb, esp = 0xbfbfe42c, ebp = 0xbfbfe448 --- Tracing command su pid 2271 tid 100139 td 0xc7e902e0 sched_switch(c7e902e0,0,104,191,34bf358a,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c7e902e0,0,c0f091e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7e8c2c4,6c,c0f0bf5f,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7e8c2c4,c7e8c34c,16c,c0f0bf5f,0,...) at _sleep+0x35c kern_wait(c7e902e0,8e3,f1917c60,2,0,...) at kern_wait+0x626 sys_wait4(c7e902e0,f1917cec,c,c,212,...) at sys_wait4+0x3b syscall(f1917d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x281255cb, esp = 0xbfbfe36c, ebp = 0xbfbfe388 --- Tracing command md5 pid 2266 tid 100175 td 0xc81668a0 sched_switch(c81668a0,0,104,191,6c3930fa,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c81668a0,0,c0f091e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c817a800,5c,c0ea7231,0,0,...) at sleepq_wait+0x63 _sleep(c817a800,c817a820,25c,c0ea7231,0,...) at _sleep+0x372 md_kthread(c817a800,f199dd28,c0efe270,3e5,c8128588,...) at md_kthread+0x11a fork_exit(c07341b0,c817a800,f199dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xf199dd60, ebp = 0 --- Tracing command sh pid 2253 tid 100100 td 0xc757f5c0 sched_switch(c757f5c0,0,104,191,6ae545c9,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c757f5c0,0,c0f091e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c78e52c4,6c,c0f0bf5f,100,0,...) at sleepq_wait_sig+0x17 _sleep(c78e52c4,c78e534c,16c,c0f0bf5f,0,...) at _sleep+0x35c kern_wait(c757f5c0,ffffffff,f1853c60,2,0,...) at kern_wait+0x626 sys_wait4(c757f5c0,f1853cec,c0f5252a,c0f0bf3e,212,...) at sys_wait4+0x3b syscall(f1853d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2816c5cb, esp = 0xbfbfe83c, ebp = 0xbfbfe858 --- Tracing command getty pid 2252 tid 100117 td 0xc7d54000 sched_switch(c7d54000,0,104,191,3d3a36ba,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7d54000,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,c7d54000,c7d54000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c758b070,0,c0f0de40,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c758b070,c109d7d0,c0f0d818,527,0,...) at _cv_wait_sig+0x243 tty_wait(c758b000,c758b070,f18d2c44,1,0,...) at tty_wait+0x71 ttydisc_read(c758b000,f18d2c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c758c600,f18d2c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7d6b4d0,f18d2c44,c715fe00,0,c7d54000,...) at devfs_read_f+0x7e dofileread(f18d2c44,ffffffff,ffffffff,0,f18d2c20,...) at dofileread+0x9e kern_readv(c7d54000,0,f18d2c44,f18d2c64,1,...) at kern_readv+0x68 sys_read(c7d54000,f18d2cec,c0f5252a,c0f24b60,246,...) at sys_read+0x4f syscall(f18d2d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281982e3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2251 tid 100125 td 0xc78fe8a0 sched_switch(c78fe8a0,0,104,191,3d178e1e,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c78fe8a0,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,c78fe8a0,c78fe8a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c758b270,0,c0f0de40,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c758b270,c109d7d0,c0f0d818,527,0,...) at _cv_wait_sig+0x243 tty_wait(c758b200,c758b270,f18eac44,1,0,...) at tty_wait+0x71 ttydisc_read(c758b200,f18eac44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c7466500,f18eac44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7d6b0a8,f18eac44,c715fe00,0,c78fe8a0,...) at devfs_read_f+0x7e dofileread(f18eac44,ffffffff,ffffffff,0,f18eac20,...) at dofileread+0x9e kern_readv(c78fe8a0,0,f18eac44,f18eac64,1,...) at kern_readv+0x68 sys_read(c78fe8a0,f18eacec,c0f5252a,c0f24b60,246,...) at sys_read+0x4f syscall(f18ead28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281982e3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2250 tid 100133 td 0xc7e935c0 sched_switch(c7e935c0,0,104,191,3d901df3,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7e935c0,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,c7e935c0,c7e935c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c758b470,0,c0f0de40,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c758b470,c109d7d0,c0f0d818,527,0,...) at _cv_wait_sig+0x243 tty_wait(c758b400,c758b470,f1905c44,1,0,...) at tty_wait+0x71 ttydisc_read(c758b400,f1905c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c7466600,f1905c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7bd8460,f1905c44,c715fe00,0,c7e935c0,...) at devfs_read_f+0x7e dofileread(f1905c44,ffffffff,ffffffff,0,f1905c20,...) at dofileread+0x9e kern_readv(c7e935c0,0,f1905c44,f1905c64,1,...) at kern_readv+0x68 sys_read(c7e935c0,f1905cec,c0f5252a,c0f24b60,246,...) at sys_read+0x4f syscall(f1905d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281982e3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2249 tid 100103 td 0xc7c992e0 sched_switch(c7c992e0,0,104,191,3d2afc39,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7c992e0,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,c7c992e0,c7c992e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c72eb070,0,c0f0de40,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c72eb070,c109d7d0,c0f0d818,527,0,...) at _cv_wait_sig+0x243 tty_wait(c72eb000,c72eb070,f188dc44,1,0,...) at tty_wait+0x71 ttydisc_read(c72eb000,f188dc44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c7466700,f188dc44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7d1f620,f188dc44,c715fe00,0,c7c992e0,...) at devfs_read_f+0x7e dofileread(f188dc44,ffffffff,ffffffff,0,f188dc20,...) at dofileread+0x9e kern_readv(c7c992e0,0,f188dc44,f188dc64,1,...) at kern_readv+0x68 sys_read(c7c992e0,f188dcec,c0f5252a,c0f24b60,246,...) at sys_read+0x4f syscall(f188dd28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281982e3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2248 tid 100124 td 0xc7d535c0 sched_switch(c7d535c0,0,104,191,3d5df112,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7d535c0,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,c7d535c0,c7d535c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c72eb270,0,c0f0de40,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c72eb270,c109d7d0,c0f0d818,527,0,...) at _cv_wait_sig+0x243 tty_wait(c72eb200,c72eb270,f18e7c44,1,0,...) at tty_wait+0x71 ttydisc_read(c72eb200,f18e7c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c7466800,f18e7c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c787c188,f18e7c44,c715fe00,0,c7d535c0,...) at devfs_read_f+0x7e dofileread(f18e7c44,ffffffff,ffffffff,0,f18e7c20,...) at dofileread+0x9e kern_readv(c7d535c0,0,f18e7c44,f18e7c64,1,...) at kern_readv+0x68 sys_read(c7d535c0,f18e7cec,c0f5252a,c0f24b60,246,...) at sys_read+0x4f syscall(f18e7d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281982e3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2247 tid 100135 td 0xc7e93000 sched_switch(c7e93000,0,104,191,3d74e271,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7e93000,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,c7e93000,c7e93000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c72eb470,0,c0f0de40,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c72eb470,c109d7d0,c0f0d818,527,0,...) at _cv_wait_sig+0x243 tty_wait(c72eb400,c72eb470,f190bc44,1,0,...) at tty_wait+0x71 ttydisc_read(c72eb400,f190bc44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c7466900,f190bc44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c787cce8,f190bc44,c715fe00,0,c7e93000,...) at devfs_read_f+0x7e dofileread(f190bc44,ffffffff,ffffffff,0,f190bc20,...) at dofileread+0x9e kern_readv(c7e93000,0,f190bc44,f190bc64,1,...) at kern_readv+0x68 sys_read(c7e93000,f190bcec,c0f5252a,c0f24b60,246,...) at sys_read+0x4f syscall(f190bd28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281982e3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2246 tid 100109 td 0xc78a1b80 sched_switch(c78a1b80,0,104,191,3d58d3a0,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c78a1b80,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,c78a1b80,c78a1b80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c72eb670,0,c0f0de40,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c72eb670,c109d7d0,c0f0d818,527,0,...) at _cv_wait_sig+0x243 tty_wait(c72eb600,c72eb670,f18bac44,1,0,...) at tty_wait+0x71 ttydisc_read(c72eb600,f18bac44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c7466a00,f18bac44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7bd8578,f18bac44,c715fe00,0,c78a1b80,...) at devfs_read_f+0x7e dofileread(f18bac44,ffffffff,ffffffff,0,f18bac20,...) at dofileread+0x9e kern_readv(c78a1b80,0,f18bac44,f18bac64,1,...) at kern_readv+0x68 sys_read(c78a1b80,f18bacec,c0f5252a,c0f24b60,246,...) at sys_read+0x4f syscall(f18bad28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281982e3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2245 tid 100122 td 0xc7d53b80 sched_switch(c7d53b80,0,104,191,3d72c97f,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7d53b80,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,c7d53b80,c7d53b80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c72eb870,0,c0f0de40,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c72eb870,c109d7d0,c0f0d818,527,0,...) at _cv_wait_sig+0x243 tty_wait(c72eb800,c72eb870,f18e1c44,1,0,...) at tty_wait+0x71 ttydisc_read(c72eb800,f18e1c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c7466b00,f18e1c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7cb7150,f18e1c44,c715fe00,0,c7d53b80,...) at devfs_read_f+0x7e dofileread(f18e1c44,ffffffff,ffffffff,0,f18e1c20,...) at dofileread+0x9e kern_readv(c7d53b80,0,f18e1c44,f18e1c64,1,...) at kern_readv+0x68 sys_read(c7d53b80,f18e1cec,c0f5252a,c0f24b60,246,...) at sys_read+0x4f syscall(f18e1d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281982e3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2244 tid 100101 td 0xc757f2e0 sched_switch(c757f2e0,0,104,191,63b4bab6,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c757f2e0,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c09fc3da,c72eba04,0,c0f01f07,c757f2e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c72eba70,0,f1857af0,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c72eba70,c72eba04,c0f0d818,527,0,...) at _cv_wait_sig+0x243 tty_wait(c72eba00,c72eba70,f1857c44,1,0,...) at tty_wait+0x71 ttydisc_read(c72eba00,f1857c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c72ccd00,f1857c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c791e770,f1857c44,c715fe00,0,c757f2e0,...) at devfs_read_f+0x7e dofileread(f1857c44,ffffffff,ffffffff,0,f1857c20,...) at dofileread+0x9e kern_readv(c757f2e0,0,f1857c44,f1857c64,1,...) at kern_readv+0x68 sys_read(c757f2e0,f1857cec,c0f5252a,c0f24b60,246,...) at sys_read+0x4f syscall(f1857d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281982e3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command inetd pid 2205 tid 100149 td 0xc7d53000 sched_switch(c7d53000,0,104,191,28af40eb,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7d53000,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c09fc3da,c7926410,0,c0f01f07,c7d53000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7926424,0,f194fa68,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7926424,c7926410,c0f0b7da,668,c7bd8af0,...) at _cv_wait_sig+0x243 seltdwait(c7bd8af0,58,c715fe00,c7d53000,c0f01f07,...) at seltdwait+0xa2 kern_select(c7d53000,6,bfbfdfa0,0,0,0,20,c812a84c,4) at kern_select+0x504 sys_select(c7d53000,f194fcec,c0f5252a,c0f0bfab,206,...) at sys_select+0x66 syscall(f194fd28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281aa263, esp = 0xbfbfdf5c, ebp = 0xbfbfede8 --- Tracing command bash pid 2165 tid 100102 td 0xc7c995c0 sched_switch(c7c995c0,0,104,191,1883842c,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c7c995c0,0,c0f091e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7c92588,6c,c0f0bf5f,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7c92588,c7c92610,16c,c0f0bf5f,0,...) at _sleep+0x35c kern_wait(c7c995c0,ffffffff,f1889c60,6,0,...) at kern_wait+0x626 sys_wait4(c7c995c0,f1889cec,c0f5252a,c0f0be4e,212,...) at sys_wait4+0x3b syscall(f1889d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x282c55cb, esp = 0xbfbfe76c, ebp = 0xbfbfe788 --- Tracing command csh pid 2161 tid 100137 td 0xc7e908a0 sched_switch(c7e908a0,0,104,191,4235cefc,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,74,...) at mi_switch+0x1eb sleepq_switch(c7e908a0,0,c0f091e0,1a7,74,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7e8c8a4,74,c0d9316e,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7e8c8a4,c7e8c8d4,174,c0d9316e,0,...) at _sleep+0x35c kern_sigsuspend(c7e908a0,0,0,0,0,...) at kern_sigsuspend+0xba sys_sigsuspend(c7e908a0,f1911cec,c0f5252a,c0f0be4e,202,...) at sys_sigsuspend+0x4d syscall(f1911d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x2818c8cb, esp = 0xbfbfe58c, ebp = 0xbfbfe5f8 --- Tracing command su pid 2160 tid 100127 td 0xc78fe2e0 sched_switch(c78fe2e0,0,104,191,f12b25c3,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c78fe2e0,0,c0f091e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c78f62c4,6c,c0f0bf5f,100,0,...) at sleepq_wait_sig+0x17 _sleep(c78f62c4,c78f634c,16c,c0f0bf5f,0,...) at _sleep+0x35c kern_wait(c78fe2e0,871,f18f2c60,2,0,...) at kern_wait+0x626 sys_wait4(c78fe2e0,f18f2cec,c,c,212,...) at sys_wait4+0x3b syscall(f18f2d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x281255cb, esp = 0xbfbfe5bc, ebp = 0xbfbfe5d8 --- Tracing command bash pid 2155 tid 100138 td 0xc7e905c0 sched_switch(c7e905c0,0,104,191,fa3a9d95,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c7e905c0,0,c0f091e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7e8c588,6c,c0f0bf5f,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7e8c588,c7e8c610,16c,c0f0bf5f,0,...) at _sleep+0x35c kern_wait(c7e905c0,ffffffff,f1914c60,6,0,...) at kern_wait+0x626 sys_wait4(c7e905c0,f1914cec,c0f5252a,c0f0be4e,212,...) at sys_wait4+0x3b syscall(f1914d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x282c55cb, esp = 0xbfbfe97c, ebp = 0xbfbfe998 --- Tracing command sshd pid 2154 tid 100126 td 0xc78fe5c0 sched_switch(c78fe5c0,0,104,191,47b9ebc,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c78fe5c0,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c09fc3da,c77e0810,0,c0f01f07,c78fe5c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c77e0824,0,f18eea68,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c77e0824,c77e0810,c0f0b7da,668,c7d1fd20,...) at _cv_wait_sig+0x243 seltdwait(c7d1fd20,58,c7899980,c78fe5c0,c0f2c3bb,...) at seltdwait+0xa2 kern_select(c78fe5c0,c,288060b4,288060bc,0,0,20,c78f6588,0) at kern_select+0x504 sys_select(c78fe5c0,f18eecec,c0f5252a,c0f0c0b3,206,...) at sys_select+0x66 syscall(f18eed28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284c6263, esp = 0xbfbfde2c, ebp = 0xbfbfde78 --- Tracing command sshd pid 2152 tid 100088 td 0xc7802000 sched_switch(c7802000,0,104,191,bfbfbd6b,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,68,...) at mi_switch+0x1eb sleepq_switch(c7802000,0,c0f091e0,1a7,68,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7e04a78,68,c0f0f601,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7e04a78,c7e04a38,168,c0f0f601,0) at _sleep+0x35c sbwait(c7e04a14,4,c0f0f6d9,5ee,c7e04a38,...) at sbwait+0x76 soreceive_generic(c7e049c0,0,f1819c44,0,0,...) at soreceive_generic+0x320 soreceive(c7e049c0,0,f1819c44,0,0,0) at soreceive+0x38 soo_read(c7d1fce8,f1819c44,c7921a00,0,c7802000,...) at soo_read+0x4e dofileread(f1819c44,ffffffff,ffffffff,0,f1819c20,...) at dofileread+0x9e kern_readv(c7802000,5,f1819c44,f1819c64,1,...) at kern_readv+0x68 sys_read(c7802000,f1819cec,c0f5252a,c0ef4a79,282,...) at sys_read+0x4f syscall(f1819d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x284c62e3, esp = 0xbfbfddec, ebp = 0xbfbfde28 --- Tracing command cron pid 2113 tid 100132 td 0xc7e938a0 sched_switch(c7e938a0,0,104,191,178e0c0d,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c7e938a0,0,c0f091e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(ea61,c0a4b930,c7e938a0,2,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c109f0e4,6c,c0f04eb7,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c109f0e4,0,16c,c0f04eb7,ea61,...) at _sleep+0x328 kern_nanosleep(c7e938a0,f1902c50,f1902c58,3c,0,...) at kern_nanosleep+0xc1 sys_nanosleep(c7e938a0,f1902cec,c0f5252a,c0f0c40c,206,...) at sys_nanosleep+0x6f syscall(f1902d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x28186817, esp = 0xbfbfec4c, ebp = 0xbfbfec78 --- Tracing command sendmail pid 2105 tid 100112 td 0xc78a12e0 sched_switch(c78a12e0,0,104,191,70f83c9b,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,74,...) at mi_switch+0x1eb sleepq_switch(c78a12e0,0,c0f091e0,1a7,74,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7d4131c,74,c0d9316e,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7d4131c,c7d4134c,174,c0d9316e,0,...) at _sleep+0x35c kern_sigsuspend(c78a12e0,0,0,0,0,...) at kern_sigsuspend+0xba sys_sigsuspend(c78a12e0,f18c3cec,c0f5252a,c0f0be4e,202,...) at sys_sigsuspend+0x4d syscall(f18c3d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x2833d8cb, esp = 0xbfbfcf9c, ebp = 0xbfbfcfc8 --- Tracing command sendmail pid 2101 tid 100113 td 0xc78a1000 sched_switch(c78a1000,0,104,191,4992e69a,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c78a1000,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f18c6a30,c09fc3da,c77cd8d0,0,c78a1000,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c77cd8e4,0,f18c6a68,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c77cd8e4,c77cd8d0,1389,668,c787b038,...) at _cv_timedwait_sig+0x252 seltdwait(f18c6c14,f18c6c1c,c7922c00,c78a1000,f18c6b14,...) at seltdwait+0x8a kern_select(c78a1000,5,bfbfc510,0,0,f18c6c5c,20,5,0) at kern_select+0x504 sys_select(c78a1000,f18c6cec,c0f5252a,c0f0c36b,206,...) at sys_select+0x66 syscall(f18c6d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x283e5263, esp = 0xbfbfc47c, ebp = 0xbfbfcfa8 --- Tracing command sshd pid 2093 tid 100105 td 0xc7c98b80 sched_switch(c7c98b80,0,104,191,ae5343b2,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7c98b80,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c09fc3da,c77e0cd0,0,c0f01f07,c7c98b80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c77e0ce4,0,f1895a68,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c77e0ce4,c77e0cd0,c0f0b7da,668,c7bd8d58,...) at _cv_wait_sig+0x243 seltdwait(c7bd8d58,58,c7922b00,c7c98b80,c0f3c165,...) at seltdwait+0xa2 kern_select(c7c98b80,5,288cc0b4,0,0,0,20,c7c8fb10,a) at kern_select+0x504 sys_select(c7c98b80,f1895cec,c0f5252a,c0ef4a79,206,...) at sys_select+0x66 syscall(f1895d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284c6263, esp = 0xbfbfdeec, ebp = 0xbfbfedf8 --- Tracing command moused pid 2033 tid 100136 td 0xc7e90b80 sched_switch(c7e90b80,0,104,191,5995b667,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7e90b80,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c09fc3da,c77e0d10,0,c0f01f07,c7e90b80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c77e0d24,0,f190ea68,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c77e0d24,c77e0d10,c0f0b7da,668,c78b4770,...) at _cv_wait_sig+0x243 seltdwait(c78b4770,58,c715fe00,c7e90b80,14,...) at seltdwait+0xa2 kern_select(c7e90b80,6,bfbfea04,0,0,0,20,c7e8cb10,1) at kern_select+0x504 sys_select(c7e90b80,f190ecec,c0f5252a,c0f24b60,246,...) at sys_select+0x66 syscall(f190ed28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281b5263, esp = 0xbfbfe99c, ebp = 0xbfbfeb28 --- Tracing command ntpd pid 1955 tid 100098 td 0xc757fb80 sched_switch(c757fb80,0,104,191,cb80085a,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c757fb80,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c09fc3da,c7b833d0,0,c0f01f07,c757fb80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7b833e4,0,f1847a68,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7b833e4,c7b833d0,c0f0b7da,668,c7d6b188,...) at _cv_wait_sig+0x243 seltdwait(c7d6b188,58,c715fe00,c757fb80,0,...) at seltdwait+0xa2 kern_select(c757fb80,1c,bfbfece8,0,0,0,20,c78e584c,0) at kern_select+0x504 sys_select(c757fb80,f1847cec,c0f5252a,f1847d80,202,...) at sys_select+0x66 syscall(f1847d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28364263, esp = 0xbfbfecbc, ebp = 0xbfbfed88 --- Tracing command nfsd pid 1854 tid 100130 td 0xc78fd8a0 sched_switch(c78fd8a0,0,104,191,5eb5f70d,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c78fd8a0,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f18fcbe0,c09fc3da,c7562e00,0,c78fd8a0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c7415a90,0,f18fcc18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c7415a90,c7562e00,1388,3b1,1,...) at _cv_timedwait_sig+0x252 svc_run_internal(f18fcd14,c09df4c8,c7562e00,f18fcd28,c0efe270,...) at svc_run_internal+0x356 svc_thread_start(c7562e00,f18fcd28,c0efe270,3e5,c77f9588,...) at svc_thread_start+0x10 fork_exit(c0bf6c20,c7562e00,f18fcd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x804c12e, eip = 0xc, esp = 0x33, ebp = 0x1 --- Tracing command nfsd pid 1854 tid 100129 td 0xc78fdb80 sched_switch(c78fdb80,0,104,191,8b20237e,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c78fdb80,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f18f9be0,c09fc3da,c7562e00,0,c78fdb80,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c7415ad0,0,f18f9c18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c7415ad0,c7562e00,1388,3b1,281ae440,...) at _cv_timedwait_sig+0x252 svc_run_internal(f18f9d14,c09df4c8,c7562e00,f18f9d28,c0efe270,...) at svc_run_internal+0x356 svc_thread_start(c7562e00,f18f9d28,c0efe270,3e5,c77f9588,...) at svc_thread_start+0x10 fork_exit(c0bf6c20,c7562e00,f18f9d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x804c12e, eip = 0xc, esp = 0x33, ebp = 0x1 --- Tracing command nfsd pid 1854 tid 100128 td 0xc78fe000 sched_switch(c78fe000,0,104,191,f4e9c06a,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c78fe000,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f18f6be0,c09fc3da,c7562e00,0,c78fe000,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c7415b10,0,f18f6c18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c7415b10,c7562e00,1388,3b1,0,...) at _cv_timedwait_sig+0x252 svc_run_internal(f18f6d14,c09df4c8,c7562e00,f18f6d28,c0efe270,...) at svc_run_internal+0x356 svc_thread_start(c7562e00,f18f6d28,c0efe270,3e5,c77f9588,...) at svc_thread_start+0x10 fork_exit(c0bf6c20,c7562e00,f18f6d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x804c12e, eip = 0xc, esp = 0x33, ebp = 0x1 --- Tracing command nfsd pid 1854 tid 100092 td 0xc78a65c0 sched_switch(c78a65c0,0,104,191,df745bc6,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c78a65c0,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f1829498,c09fc3da,c7562e00,0,c78a65c0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c7415b50,0,f18294d0,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c7415b50,c7562e00,1388,3b1,f1829518,...) at _cv_timedwait_sig+0x252 svc_run_internal(c78a6770,14,c0f3219a,c0ef5e5b,f1829810,...) at svc_run_internal+0x356 svc_run(c7562e00,0,c0ef5e36,19f,c11e8270,...) at svc_run+0x7b nfsrvd_nfsd(c78a65c0,f1829810,c,c0ef9420,3,...) at nfsrvd_nfsd+0x1c3 nfssvc_nfsd(c78a65c0,f1829cec,bfbfe870,c78a65c0,f1829d1c,...) at nfssvc_nfsd+0x177 sys_nfssvc(c78a65c0,f1829cec,c0f5252a,c0f0c36b,206,...) at sys_nfssvc+0xeb syscall(f1829d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (155, FreeBSD ELF32, sys_nfssvc), eip = 0x280df4ab, esp = 0xbfbfe82c, ebp = 0xbfbfea98 --- Tracing command nfsd pid 1848 tid 100123 td 0xc7d538a0 sched_switch(c7d538a0,0,104,191,2ab8d54c,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7d538a0,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c09fc3da,c77cdb10,0,c0f01f07,c7d538a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c77cdb24,0,f18e4a68,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c77cdb24,c77cdb10,c0f0b7da,668,c787caf0,...) at _cv_wait_sig+0x243 seltdwait(c787caf0,58,c715fe00,c7d538a0,c7bc4bb0,...) at seltdwait+0xa2 kern_select(c7d538a0,7,bfbfecb4,0,0,0,20,c7c92b10,1) at kern_select+0x504 sys_select(c7d538a0,f18e4cec,c,c,202,...) at sys_select+0x66 syscall(f18e4d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28186263, esp = 0xbfbfea9c, ebp = 0xbfbfede8 --- Tracing command mountd pid 1838 tid 100097 td 0xc7801000 sched_switch(c7801000,0,104,191,44eea71a,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7801000,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c09fc3da,c79261d0,0,c0f01f07,c7801000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c79261e4,0,f1841a68,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c79261e4,c79261d0,c0f0b7da,668,c787b2a0,...) at _cv_wait_sig+0x243 seltdwait(c787b2a0,58,c715fe00,c7801000,c11e8218,...) at seltdwait+0xa2 kern_select(c7801000,9,bfbfecf4,0,0,0,20,c78e5b10,bfbfecf4) at kern_select+0x504 sys_select(c7801000,f1841cec,c0f5252a,c0ef3d17,206,...) at sys_select+0x66 syscall(f1841d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x2819c263, esp = 0xbfbfec8c, ebp = 0xbfbfedd8 --- Tracing command rpcbind pid 1738 tid 100094 td 0xc78a62e0 sched_switch(c78a62e0,0,104,191,82f90c10,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c78a62e0,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f1831a88,c09fc3da,c7926210,0,c78a62e0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c7926224,0,f1831ac0,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c7926224,c7926210,7531,668,f1831b78,...) at _cv_timedwait_sig+0x252 seltdwait(f1831c48,f1831c50,565,c78a62e0,f1831b48,...) at seltdwait+0x8a sys_poll(c78a62e0,f1831cec,c0f5252a,c0f0c09c,212,...) at sys_poll+0x300 syscall(f1831d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x2814143b, esp = 0xbfbfcbac, ebp = 0xbfbfed78 --- Tracing command syslogd pid 1709 tid 100083 td 0xc78a75c0 sched_switch(c78a75c0,0,104,191,63ebcd14,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c78a75c0,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c09fc3da,c7926250,0,c0f01f07,c78a75c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7926264,0,f1803a68,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7926264,c7926250,c0f0b7da,668,c7cb7818,...) at _cv_wait_sig+0x243 seltdwait(c7cb7818,58,c715fe00,c78a75c0,58,...) at seltdwait+0xa2 kern_select(c78a75c0,9,2847f0ac,0,0,0,20,c789f588,0) at kern_select+0x504 sys_select(c78a75c0,f1803cec,c0f5252a,c0d712c1,246,...) at sys_select+0x66 syscall(f1803d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x2819b263, esp = 0xbfbfe29c, ebp = 0xbfbfede8 --- Tracing command devd pid 1509 tid 100081 td 0xc78025c0 sched_switch(c78025c0,0,104,191,c3812cae,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c78025c0,0,c0f091e0,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c09fc3da,c77cd710,0,c0f01f07,c78025c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c77cd724,0,f17f9a68,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c77cd724,c77cd710,c0f0b7da,668,c787cab8,...) at _cv_wait_sig+0x243 seltdwait(c787cab8,58,c715fe00,c78025c0,c78025c0,...) at seltdwait+0xa2 kern_select(c78025c0,5,bfbfe960,0,0,0,20,c7431b10,34) at kern_select+0x504 sys_select(c78025c0,f17f9cec,c0f5252a,c0d712c1,206,...) at sys_select+0x66 syscall(f17f9d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x808b9cb, esp = 0xbfbfe92c, ebp = 0xbfbfee08 --- Tracing command softdepflush pid 18 tid 100076 td 0xc744f8a0 sched_switch(c744f8a0,0,104,191,cb7eb110,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,54,...) at mi_switch+0x1eb sleepq_switch(c744f8a0,0,c0f091e0,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c121b9e0,54,c0f392e5,0,0,...) at sleepq_timedwait+0x6b _sleep(c121b9e0,c121b990,54,c0f392e5,3e8,...) at _sleep+0x342 softdep_flush(0,ed40ed28,c0efe270,3e5,c77f72c4,...) at softdep_flush+0x266 fork_exit(c0c475a0,0,ed40ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed40ed60, ebp = 0 --- Tracing command vnlru pid 17 tid 100075 td 0xc744fb80 sched_switch(c744fb80,0,104,191,a094081c,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,60,...) at mi_switch+0x1eb sleepq_switch(c744fb80,0,c0f091e0,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c77f7588,60,c0f13f32,0,0,...) at sleepq_timedwait+0x6b _sleep(c77f7588,c1215294,260,c0f13f32,3e8,...) at _sleep+0x342 vnlru_proc(0,ed40bd28,c0efe270,3e5,c77f7588,...) at vnlru_proc+0xe7 fork_exit(c0aa9fd0,0,ed40bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed40bd60, ebp = 0 --- Tracing command syncer pid 16 tid 100074 td 0xc757c000 sched_switch(c757c000,0,104,191,cb8033c7,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c757c000,0,c0f091e0,28b,c757c000,...) at sleepq_switch+0x162 sleepq_timedwait(c12152d4,0,ed408c78,1,0,...) at sleepq_timedwait+0x6b _cv_timedwait(c12152d4,c12152c0,3e8,76a,4e20,...) at _cv_timedwait+0x252 sched_sync(0,ed408d28,c0efe270,3e5,c77f784c,...) at sched_sync+0x595 fork_exit(c0aa7f70,0,ed408d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed408d60, ebp = 0 --- Tracing command bufdaemon pid 9 tid 100073 td 0xc757c2e0 sched_switch(c757c2e0,0,104,191,8b1ff5c5,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,54,...) at mi_switch+0x1eb sleepq_switch(c757c2e0,0,c0f091e0,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c1214f68,54,c0f10ecf,0,0,...) at sleepq_timedwait+0x6b _sleep(c1214f68,c1214f70,54,c0f10ecf,3e8,...) at _sleep+0x342 buf_daemon(0,ed405d28,c0efe270,3e5,c77f7b10,...) at buf_daemon+0x175 fork_exit(c0a8ece0,0,ed405d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed405d60, ebp = 0 --- Tracing command pagezero pid 8 tid 100072 td 0xc757c5c0 sched_switch(c757c5c0,0,104,191,cf2e72fa,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c757c5c0,0,c0f091e0,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c121d85c,0,c0f3fc4c,0,0,...) at sleepq_timedwait+0x6b _sleep(c121d85c,c121c300,0,c0f3fc4c,493e0,...) at _sleep+0x342 vm_pagezero(0,ed402d28,c0efe270,3e5,c77f9000,...) at vm_pagezero+0xdc fork_exit(c0c8bd30,0,ed402d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed402d60, ebp = 0 --- Tracing command vmdaemon pid 7 tid 100071 td 0xc757c8a0 sched_switch(c757c8a0,0,104,191,6acb61b9,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,74,...) at mi_switch+0x1eb sleepq_switch(c757c8a0,0,c0f091e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c121d448,74,c0f10ecf,0,0,...) at sleepq_wait+0x63 _sleep(c121d448,c121d44c,74,c0f10ecf,0,...) at _sleep+0x372 vm_daemon(0,ed3ffd28,c0efe270,3e5,c71be2c4,...) at vm_daemon+0x59 fork_exit(c0c85b20,0,ed3ffd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed3ffd60, ebp = 0 --- Tracing command pagedaemon pid 6 tid 100070 td 0xc757cb80 sched_switch(c757cb80,0,104,191,493f945c,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,54,...) at mi_switch+0x1eb sleepq_switch(c757cb80,0,c0f091e0,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c121d410,54,c0f10ecf,0,0,...) at sleepq_timedwait+0x6b _sleep(c121d410,c121c300,54,c0f10ecf,1388,...) at _sleep+0x342 vm_pageout(0,ed3fcd28,c0efe270,3e5,c71be588,...) at vm_pageout+0x2c7 fork_exit(c0c86c00,0,ed3fcd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed3fcd60, ebp = 0 --- Tracing command xpt_thrd pid 5 tid 100069 td 0xc757d000 sched_switch(c757d000,0,104,191,70db5695,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c757d000,0,c0f091e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c1063dd4,5c,c0d6bbc5,0,0,...) at sleepq_wait+0x63 _sleep(c1063dd4,c1063e18,5c,c0d6bbc5,0,...) at _sleep+0x372 xpt_scanner_thread(0,ed399d28,c0efe270,3e5,c71be84c,...) at xpt_scanner_thread+0x47 fork_exit(c0492320,0,ed399d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed399d60, ebp = 0 --- Tracing command sctp_iterator pid 4 tid 100068 td 0xc757d2e0 sched_switch(c757d2e0,0,104,191,2fa3e972,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c757d2e0,0,c0f091e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c1216df8,0,c0f271df,0,0,...) at sleepq_wait+0x63 _sleep(c1216df8,c1216dc0,0,c0f271df,0,...) at _sleep+0x372 sctp_iterator_thread(0,ed396d28,c0efe270,3e5,c71beb10,...) at sctp_iterator_thread+0x5c fork_exit(c0b3a880,0,ed396d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed396d60, ebp = 0 --- Tracing command fdc0 pid 3 tid 100064 td 0xc757f000 sched_switch(c757f000,0,104,191,cb801557,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c757f000,0,c0f091e0,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c73ede3c,5c,c0ef68f2,0,0,...) at sleepq_timedwait+0x6b _sleep(c73ede3c,c73edef0,5c,c0ef68f2,3e8,...) at _sleep+0x342 fdc_thread(c73ede00,ed38ad28,c0efe270,3e5,c7431000,...) at fdc_thread+0x27d fork_exit(c0d0b000,c73ede00,ed38ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed38ad60, ebp = 0 --- Tracing command fw0_probe pid 2 tid 100062 td 0xc743f5c0 sched_switch(c743f5c0,0,104,191,70dbbbed,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c743f5c0,0,c0f091e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7473000,6c,c0ef68f2,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7473000,c7477488,16c,c0ef68f2,0,...) at _sleep+0x35c fw_bus_probe_thread(c7473000,ed37ad28,c0efe270,3e5,c74312c4,...) at fw_bus_probe_thread+0xa08 fork_exit(c06b6e50,c7473000,ed37ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed37ad60, ebp = 0 --- Tracing command usb pid 15 tid 100058 td 0xc744d2e0 sched_switch(c744d2e0,0,104,191,5f879c3a,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c744d2e0,0,c0f091e0,268,c744d2e0,...) at sleepq_switch+0x162 sleepq_wait(c7454d34,0,ed363cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7454d34,c7454dd4,c0edda91,6b,c7454d3c,...) at _cv_wait+0x243 usb_process(c7454d2c,ed363d28,c0efe270,3e5,c7431588,...) at usb_process+0x193 fork_exit(c08b10b0,c7454d2c,ed363d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed363d60, ebp = 0 --- Tracing command usb pid 15 tid 100057 td 0xc744d5c0 sched_switch(c744d5c0,0,104,191,8e3dab5,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c744d5c0,0,c0f091e0,268,c744d5c0,...) at sleepq_switch+0x162 sleepq_wait(c7454d04,0,ed360cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7454d04,c7454dd4,c0edda91,6b,c7454d0c,...) at _cv_wait+0x243 usb_process(c7454cfc,ed360d28,c0efe270,3e5,c7431588,...) at usb_process+0x193 fork_exit(c08b10b0,c7454cfc,ed360d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed360d60, ebp = 0 --- Tracing command usb pid 15 tid 100056 td 0xc744d8a0 sched_switch(c744d8a0,0,104,191,5f208607,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c744d8a0,0,c0f091e0,268,c744d8a0,...) at sleepq_switch+0x162 sleepq_wait(c7454cd4,0,ed35dcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7454cd4,c7454dd4,c0edda91,6b,c7454cdc,...) at _cv_wait+0x243 usb_process(c7454ccc,ed35dd28,c0efe270,3e5,c7431588,...) at usb_process+0x193 fork_exit(c08b10b0,c7454ccc,ed35dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed35dd60, ebp = 0 --- Tracing command usb pid 15 tid 100055 td 0xc744db80 sched_switch(c744db80,0,104,191,5f207554,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c744db80,0,c0f091e0,268,c744db80,...) at sleepq_switch+0x162 sleepq_wait(c7454ca4,0,ed35acac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7454ca4,c7454dd4,c0edda91,6b,c7454cac,...) at _cv_wait+0x243 usb_process(c7454c9c,ed35ad28,c0efe270,3e5,c7431588,...) at usb_process+0x193 fork_exit(c08b10b0,c7454c9c,ed35ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed35ad60, ebp = 0 --- Tracing command usb pid 15 tid 100053 td 0xc744f2e0 sched_switch(c744f2e0,0,104,191,5f205ff2,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c744f2e0,0,c0f091e0,268,c744f2e0,...) at sleepq_switch+0x162 sleepq_wait(c7449b5c,0,ed351cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7449b5c,c7449bfc,c0edda91,6b,c7449b64,...) at _cv_wait+0x243 usb_process(c7449b54,ed351d28,c0efe270,3e5,c7431588,...) at usb_process+0x193 fork_exit(c08b10b0,c7449b54,ed351d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed351d60, ebp = 0 --- Tracing command usb pid 15 tid 100052 td 0xc744f5c0 sched_switch(c744f5c0,0,104,191,8e30240,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c744f5c0,0,c0f091e0,268,c744f5c0,...) at sleepq_switch+0x162 sleepq_wait(c7449b2c,0,ed34ecac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7449b2c,c7449bfc,c0edda91,6b,c7449b34,...) at _cv_wait+0x243 usb_process(c7449b24,ed34ed28,c0efe270,3e5,c7431588,...) at usb_process+0x193 fork_exit(c08b10b0,c7449b24,ed34ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed34ed60, ebp = 0 --- Tracing command usb pid 15 tid 100051 td 0xc743b000 sched_switch(c743b000,0,104,191,580769dd,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c743b000,0,c0f091e0,268,c743b000,...) at sleepq_switch+0x162 sleepq_wait(c7449afc,0,ed34bcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7449afc,c7449bfc,c0edda91,6b,c7449b04,...) at _cv_wait+0x243 usb_process(c7449af4,ed34bd28,c0efe270,3e5,c7431588,...) at usb_process+0x193 fork_exit(c08b10b0,c7449af4,ed34bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed34bd60, ebp = 0 --- Tracing command usb pid 15 tid 100050 td 0xc743b2e0 sched_switch(c743b2e0,0,104,191,5807589b,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c743b2e0,0,c0f091e0,268,c743b2e0,...) at sleepq_switch+0x162 sleepq_wait(c7449acc,0,ed348cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7449acc,c7449bfc,c0edda91,6b,c7449ad4,...) at _cv_wait+0x243 usb_process(c7449ac4,ed348d28,c0efe270,3e5,c7431588,...) at usb_process+0x193 fork_exit(c08b10b0,c7449ac4,ed348d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed348d60, ebp = 0 --- Tracing command usb pid 15 tid 100049 td 0xc743b5c0 sched_switch(c743b5c0,0,104,191,580745c0,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c743b5c0,0,c0f091e0,268,c743b5c0,...) at sleepq_switch+0x162 sleepq_wait(c7444b5c,0,ed345cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7444b5c,c7444bfc,c0edda91,6b,c7444b64,...) at _cv_wait+0x243 usb_process(c7444b54,ed345d28,c0efe270,3e5,c7431588,...) at usb_process+0x193 fork_exit(c08b10b0,c7444b54,ed345d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed345d60, ebp = 0 --- Tracing command usb pid 15 tid 100048 td 0xc743b8a0 sched_switch(c743b8a0,0,104,191,8e270bb,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c743b8a0,0,c0f091e0,268,c743b8a0,...) at sleepq_switch+0x162 sleepq_wait(c7444b2c,0,ed342cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7444b2c,c7444bfc,c0edda91,6b,c7444b34,...) at _cv_wait+0x243 usb_process(c7444b24,ed342d28,c0efe270,3e5,c7431588,...) at usb_process+0x193 fork_exit(c08b10b0,c7444b24,ed342d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed342d60, ebp = 0 --- Tracing command usb pid 15 tid 100047 td 0xc743bb80 sched_switch(c743bb80,0,104,191,57a33888,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c743bb80,0,c0f091e0,268,c743bb80,...) at sleepq_switch+0x162 sleepq_wait(c7444afc,0,ed33fcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7444afc,c7444bfc,c0edda91,6b,c7444b04,...) at _cv_wait+0x243 usb_process(c7444af4,ed33fd28,c0efe270,3e5,c7431588,...) at usb_process+0x193 fork_exit(c08b10b0,c7444af4,ed33fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed33fd60, ebp = 0 --- Tracing command usb pid 15 tid 100046 td 0xc743d000 sched_switch(c743d000,0,104,191,57a327b2,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c743d000,0,c0f091e0,268,c743d000,...) at sleepq_switch+0x162 sleepq_wait(c7444acc,0,ed33ccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7444acc,c7444bfc,c0edda91,6b,c7444ad4,...) at _cv_wait+0x243 usb_process(c7444ac4,ed33cd28,c0efe270,3e5,c7431588,...) at usb_process+0x193 fork_exit(c08b10b0,c7444ac4,ed33cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed33cd60, ebp = 0 --- Tracing command usb pid 15 tid 100045 td 0xc743d2e0 sched_switch(c743d2e0,0,104,191,57a315a5,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c743d2e0,0,c0f091e0,268,c743d2e0,...) at sleepq_switch+0x162 sleepq_wait(c7437b5c,0,ed339cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7437b5c,c7437bfc,c0edda91,6b,c7437b64,...) at _cv_wait+0x243 usb_process(c7437b54,ed339d28,c0efe270,3e5,c7431588,...) at usb_process+0x193 fork_exit(c08b10b0,c7437b54,ed339d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed339d60, ebp = 0 --- Tracing command usb pid 15 tid 100044 td 0xc743d5c0 sched_switch(c743d5c0,0,104,191,8e1f877,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c743d5c0,0,c0f091e0,268,c743d5c0,...) at sleepq_switch+0x162 sleepq_wait(c7437b2c,0,ed336cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7437b2c,c7437bfc,c0edda91,6b,c7437b34,...) at _cv_wait+0x243 usb_process(c7437b24,ed336d28,c0efe270,3e5,c7431588,...) at usb_process+0x193 fork_exit(c08b10b0,c7437b24,ed336d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed336d60, ebp = 0 --- Tracing command usb pid 15 tid 100043 td 0xc743d8a0 sched_switch(c743d8a0,0,104,191,573f9aed,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c743d8a0,0,c0f091e0,268,c743d8a0,...) at sleepq_switch+0x162 sleepq_wait(c7437afc,0,ed333cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7437afc,c7437bfc,c0edda91,6b,c7437b04,...) at _cv_wait+0x243 usb_process(c7437af4,ed333d28,c0efe270,3e5,c7431588,...) at usb_process+0x193 fork_exit(c08b10b0,c7437af4,ed333d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed333d60, ebp = 0 --- Tracing command usb pid 15 tid 100042 td 0xc743db80 sched_switch(c743db80,0,104,191,573f8837,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c743db80,0,c0f091e0,268,c743db80,...) at sleepq_switch+0x162 sleepq_wait(c7437acc,0,e0ffdcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7437acc,c7437bfc,c0edda91,6b,c7437ad4,...) at _cv_wait+0x243 usb_process(c7437ac4,e0ffdd28,c0efe270,3e5,c7431588,...) at usb_process+0x193 fork_exit(c08b10b0,c7437ac4,e0ffdd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0ffdd60, ebp = 0 --- Tracing command usb pid 15 tid 100040 td 0xc742e000 sched_switch(c742e000,0,104,191,573f7598,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c742e000,0,c0f091e0,268,c742e000,...) at sleepq_switch+0x162 sleepq_wait(c7432b5c,0,e0ff6cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7432b5c,c7432bfc,c0edda91,6b,c7432b64,...) at _cv_wait+0x243 usb_process(c7432b54,e0ff6d28,c0efe270,3e5,c7431588,...) at usb_process+0x193 fork_exit(c08b10b0,c7432b54,e0ff6d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0ff6d60, ebp = 0 --- Tracing command usb pid 15 tid 100039 td 0xc742e2e0 sched_switch(c742e2e0,0,104,191,8e17817,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c742e2e0,0,c0f091e0,268,c742e2e0,...) at sleepq_switch+0x162 sleepq_wait(c7432b2c,0,e0ff3cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7432b2c,c7432bfc,c0edda91,6b,c7432b34,...) at _cv_wait+0x243 usb_process(c7432b24,e0ff3d28,c0efe270,3e5,c7431588,...) at usb_process+0x193 fork_exit(c08b10b0,c7432b24,e0ff3d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0ff3d60, ebp = 0 --- Tracing command usb pid 15 tid 100038 td 0xc742e5c0 sched_switch(c742e5c0,0,104,191,56dbd705,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c742e5c0,0,c0f091e0,268,c742e5c0,...) at sleepq_switch+0x162 sleepq_wait(c7432afc,0,e0ff0cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7432afc,c7432bfc,c0edda91,6b,c7432b04,...) at _cv_wait+0x243 usb_process(c7432af4,e0ff0d28,c0efe270,3e5,c7431588,...) at usb_process+0x193 fork_exit(c08b10b0,c7432af4,e0ff0d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0ff0d60, ebp = 0 --- Tracing command usb pid 15 tid 100037 td 0xc742e8a0 sched_switch(c742e8a0,0,104,191,56dbc513,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c742e8a0,0,c0f091e0,268,c742e8a0,...) at sleepq_switch+0x162 sleepq_wait(c7432acc,0,e0fedcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7432acc,c7432bfc,c0edda91,6b,c7432ad4,...) at _cv_wait+0x243 usb_process(c7432ac4,e0fedd28,c0efe270,3e5,c7431588,...) at usb_process+0x193 fork_exit(c08b10b0,c7432ac4,e0fedd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0fedd60, ebp = 0 --- Tracing command usb pid 15 tid 100035 td 0xc7430000 sched_switch(c7430000,0,104,191,56dbaf92,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7430000,0,c0f091e0,268,c7430000,...) at sleepq_switch+0x162 sleepq_wait(c7428b5c,0,e0fe6cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7428b5c,c7428bfc,c0edda91,6b,c7428b64,...) at _cv_wait+0x243 usb_process(c7428b54,e0fe6d28,c0efe270,3e5,c7431588,...) at usb_process+0x193 fork_exit(c08b10b0,c7428b54,e0fe6d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0fe6d60, ebp = 0 --- Tracing command usb pid 15 tid 100034 td 0xc74302e0 sched_switch(c74302e0,0,104,191,8e0f436,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c74302e0,0,c0f091e0,268,c74302e0,...) at sleepq_switch+0x162 sleepq_wait(c7428b2c,0,e0fe3cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7428b2c,c7428bfc,c0edda91,6b,c7428b34,...) at _cv_wait+0x243 usb_process(c7428b24,e0fe3d28,c0efe270,3e5,c7431588,...) at usb_process+0x193 fork_exit(c08b10b0,c7428b24,e0fe3d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0fe3d60, ebp = 0 --- Tracing command usb pid 15 tid 100033 td 0xc74305c0 sched_switch(c74305c0,0,104,191,56779d37,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c74305c0,0,c0f091e0,268,c74305c0,...) at sleepq_switch+0x162 sleepq_wait(c7428afc,0,e0fe0cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7428afc,c7428bfc,c0edda91,6b,c7428b04,...) at _cv_wait+0x243 usb_process(c7428af4,e0fe0d28,c0efe270,3e5,c7431588,...) at usb_process+0x193 fork_exit(c08b10b0,c7428af4,e0fe0d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0fe0d60, ebp = 0 --- Tracing command usb pid 15 tid 100032 td 0xc74308a0 sched_switch(c74308a0,0,104,191,56778ba5,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c74308a0,0,c0f091e0,268,c74308a0,...) at sleepq_switch+0x162 sleepq_wait(c7428acc,0,e0fddcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7428acc,c7428bfc,c0edda91,6b,c7428ad4,...) at _cv_wait+0x243 usb_process(c7428ac4,e0fddd28,c0efe270,3e5,c7431588,...) at usb_process+0x193 fork_exit(c08b10b0,c7428ac4,e0fddd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0fddd60, ebp = 0 --- Tracing command yarrow pid 14 tid 100017 td 0xc72d25c0 sched_switch(c72d25c0,0,104,191,cb7ffe79,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c72d25c0,0,c0f091e0,28b,2,...) at sleepq_switch+0x162 sleepq_timedwait(c109dee4,0,c0ef68f2,2,0,...) at sleepq_timedwait+0x6b _sleep(c109dee4,0,0,c0ef68f2,64,...) at _sleep+0x342 pause(c0ef68f2,64,c0ebd62e,111,0,...) at pause+0x47 random_kthread(0,c6f62d28,c0efe270,3e5,c71bd000,...) at random_kthread+0x1ef fork_exit(c07caaa0,0,c6f62d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f62d60, ebp = 0 --- Tracing command geom pid 13 tid 100015 td 0xc72d2b80 sched_switch(c72d2b80,0,104,191,a05e677a,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c72d2b80,0,c0f091e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c109bb08,5c,c0ef68f2,0,0,...) at sleepq_wait+0x63 _sleep(c109bb08,c109ba68,25c,c0ef68f2,0,...) at _sleep+0x372 g_io_schedule_down(c72d2b80,0,c0ef8439,6c,c6f5cd14,...) at g_io_schedule_down+0x5b g_down_procbody(0,c6f5cd28,c0efe270,3e5,c71bd2c4,...) at g_down_procbody+0x9d fork_exit(c09a1750,0,c6f5cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f5cd60, ebp = 0 --- Tracing command geom pid 13 tid 100014 td 0xc72d3000 sched_switch(c72d3000,0,104,191,a06c6c5e,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c72d3000,0,c0f091e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c109bb04,5c,c0ef68f2,0,0,...) at sleepq_wait+0x63 _sleep(c109bb04,c109ba88,25c,c0ef68f2,0,...) at _sleep+0x372 g_io_schedule_up(c72d3000,0,c0ef8439,5f,c6f59d14,...) at g_io_schedule_up+0x134 g_up_procbody(0,c6f59d28,c0efe270,3e5,c71bd2c4,...) at g_up_procbody+0x9d fork_exit(c09a17f0,0,c6f59d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f59d60, ebp = 0 --- Tracing command geom pid 13 tid 100013 td 0xc71c42e0 sched_switch(c71c42e0,0,104,191,474f9fc5,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c71c42e0,0,c0f091e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c109bafc,5c,c0ef68f2,0,0,...) at sleepq_wait+0x63 _sleep(c109bafc,c109ba2c,25c,c0ef68f2,0,...) at _sleep+0x372 g_run_events(c10abd40,0,c0ef8439,79,c6f56d14,...) at g_run_events+0x526 g_event_procbody(0,c6f56d28,c0efe270,3e5,c71bd2c4,...) at g_event_procbody+0x90 fork_exit(c09a1890,0,c6f56d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f56d60, ebp = 0 --- Tracing command intr pid 12 tid 100066 td 0xc757d8a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100065 td 0xc757db80 sched_switch(c757db80,0,109,191,5673e0d6,...) at sched_switch+0x3bc mi_switch(109,0,c0efe593,554,c757db80,...) at mi_switch+0x1eb ithread_loop(c720a6b0,ed38dd28,c0efe270,3e5,c71bd588,...) at ithread_loop+0x1fb fork_exit(c09e2fb0,c720a6b0,ed38dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed38dd60, ebp = 0 --- Tracing command intr pid 12 tid 100063 td 0xc743f2e0 sched_switch(c743f2e0,0,109,191,63b42c18,...) at sched_switch+0x3bc mi_switch(109,0,c0efe593,554,c743f2e0,...) at mi_switch+0x1eb ithread_loop(c720a6c0,ed37dd28,c0efe270,3e5,c71bd588,...) at ithread_loop+0x1fb fork_exit(c09e2fb0,c720a6c0,ed37dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed37dd60, ebp = 0 --- Tracing command intr pid 12 tid 100060 td 0xc743fb80 sched_switch(c743fb80,0,109,191,4c2e7e2d,...) at sched_switch+0x3bc mi_switch(109,0,c0efe593,554,c743fb80,...) at mi_switch+0x1eb ithread_loop(c720a6d0,ed371d28,c0efe270,3e5,c71bd588,...) at ithread_loop+0x1fb fork_exit(c09e2fb0,c720a6d0,ed371d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed371d60, ebp = 0 --- Tracing command intr pid 12 tid 100059 td 0xc744d000 sched_switch(c744d000,0,109,191,693d647b,...) at sched_switch+0x3bc mi_switch(109,0,c0efe593,554,c744d000,...) at mi_switch+0x1eb ithread_loop(c720a6e0,ed369d28,c0efe270,3e5,c71bd588,...) at ithread_loop+0x1fb fork_exit(c09e2fb0,c720a6e0,ed369d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed369d60, ebp = 0 --- Tracing command intr pid 12 tid 100054 td 0xc744f000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100041 td 0xc743f000 sched_switch(c743f000,0,109,191,77a74f59,...) at sched_switch+0x3bc mi_switch(109,0,c0efe593,554,c743f000,...) at mi_switch+0x1eb ithread_loop(c720a700,e0ffad28,c0efe270,3e5,c71bd588,...) at ithread_loop+0x1fb fork_exit(c09e2fb0,c720a700,e0ffad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0ffad60, ebp = 0 --- Tracing command intr pid 12 tid 100036 td 0xc742eb80 sched_switch(c742eb80,0,109,191,765db2d3,...) at sched_switch+0x3bc mi_switch(109,0,c0efe593,554,c742eb80,...) at mi_switch+0x1eb ithread_loop(c720a710,e0fead28,c0efe270,3e5,c71bd588,...) at ithread_loop+0x1fb fork_exit(c09e2fb0,c720a710,e0fead28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0fead60, ebp = 0 --- Tracing command intr pid 12 tid 100031 td 0xc7430b80 sched_switch(c7430b80,0,109,191,233318af,...) at sched_switch+0x3bc mi_switch(109,0,c0efe593,554,c7430b80,...) at mi_switch+0x1eb ithread_loop(c720a720,e0fdad28,c0efe270,3e5,c71bd588,...) at ithread_loop+0x1fb fork_exit(c09e2fb0,c720a720,e0fdad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0fdad60, ebp = 0 --- Tracing command intr pid 12 tid 100030 td 0xc72d32e0 sched_switch(c72d32e0,0,109,191,a06b70fc,...) at sched_switch+0x3bc mi_switch(109,0,c0efe593,554,c72d32e0,...) at mi_switch+0x1eb ithread_loop(c720a730,e0e84d28,c0efe270,3e5,c71bd588,...) at ithread_loop+0x1fb fork_exit(c09e2fb0,c720a730,e0e84d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0e84d60, ebp = 0 --- Tracing command intr pid 12 tid 100029 td 0xc72d35c0 sched_switch(c72d35c0,0,109,191,95f2ac83,...) at sched_switch+0x3bc mi_switch(109,0,c0efe593,554,c72d35c0,...) at mi_switch+0x1eb ithread_loop(c720a740,e0e80d28,c0efe270,3e5,c71bd588,...) at ithread_loop+0x1fb fork_exit(c09e2fb0,c720a740,e0e80d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0e80d60, ebp = 0 --- Tracing command intr pid 12 tid 100028 td 0xc72d38a0 sched_switch(c72d38a0,0,109,191,4ccf0887,...) at sched_switch+0x3bc mi_switch(109,0,c0efe593,554,c72d38a0,...) at mi_switch+0x1eb ithread_loop(c720a750,c6fb1d28,c0efe270,3e5,c71bd588,...) at ithread_loop+0x1fb fork_exit(c09e2fb0,c720a750,c6fb1d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6fb1d60, ebp = 0 --- Tracing command intr pid 12 tid 100026 td 0xc73b2000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100025 td 0xc73b22e0 sched_switch(c73b22e0,0,109,191,a06bce43,...) at sched_switch+0x3bc mi_switch(109,0,c0efe593,554,c73b22e0,...) at mi_switch+0x1eb ithread_loop(c720a770,c6f7ad28,c0efe270,3e5,c71bd588,...) at ithread_loop+0x1fb fork_exit(c09e2fb0,c720a770,c6f7ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f7ad60, ebp = 0 --- Tracing command intr pid 12 tid 100019 td 0xc72d2000 sched_switch(c72d2000,0,109,191,66f2e62,...) at sched_switch+0x3bc mi_switch(109,0,c0efe593,554,c72d2000,...) at mi_switch+0x1eb ithread_loop(c720a780,c6f68d28,c0efe270,3e5,c71bd588,...) at ithread_loop+0x1fb fork_exit(c09e2fb0,c720a780,c6f68d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f68d60, ebp = 0 --- Tracing command intr pid 12 tid 100018 td 0xc72d22e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100012 td 0xc71c45c0 sched_switch(c71c45c0,0,109,191,df742bd7,...) at sched_switch+0x3bc mi_switch(109,0,c0efe593,554,c71c45c0,...) at mi_switch+0x1eb ithread_loop(c720a7a0,c6f53d28,c0efe270,3e5,c71bd588,...) at ithread_loop+0x1fb fork_exit(c09e2fb0,c720a7a0,c6f53d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f53d60, ebp = 0 --- Tracing command intr pid 12 tid 100011 td 0xc71c48a0 sched_switch(c71c48a0,0,109,191,ba802a4,...) at sched_switch+0x3bc mi_switch(109,0,c0efe593,554,c71c48a0,...) at mi_switch+0x1eb ithread_loop(c720a7b0,c6f50d28,c0efe270,3e5,c71bd588,...) at ithread_loop+0x1fb fork_exit(c09e2fb0,c720a7b0,c6f50d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f50d60, ebp = 0 --- Tracing command intr pid 12 tid 100010 td 0xc71c4b80 sched_switch(c71c4b80,0,109,191,cb7e100a,...) at sched_switch+0x3bc mi_switch(109,0,c0efe593,554,c71c4b80,...) at mi_switch+0x1eb ithread_loop(c720a7c0,c6f4dd28,c0efe270,3e5,c71bd588,...) at ithread_loop+0x1fb fork_exit(c09e2fb0,c720a7c0,c6f4dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f4dd60, ebp = 0 --- Tracing command intr pid 12 tid 100009 td 0xc720b000 cpustop_handler(1,13,c6f4ab50,c0d43c26,f5,...) at cpustop_handler+0x28 ipi_nmi_handler(f5,0,1df41a0,c09fc5b6,c71bd588,...) at ipi_nmi_handler+0x47 trap(c6f4ab5c) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc09fc5a7, esp = 0xc6f4ab9c, ebp = 0xc6f4abb8 --- _mtx_lock_sleep(c109d7d0,c720b000,0,c0f01f07,93,...) at _mtx_lock_sleep+0xa7 _mtx_lock_flags(c109d7d0,0,c0f01f07,93,c6f4ac90,...) at _mtx_lock_flags+0xf7 lock_mtx(c109d7d0,1,c0f0505e,216,c10ac180,...) at lock_mtx+0x29 softclock(c109f120,c6f4acb8,c09fbfb4,c10aca40,c72082b8,...) at softclock+0x1ea intr_event_execute_handlers(c71bd588,c7208280,c0efe593,55e,c720b000,...) at intr_event_execute_handlers+0x125 ithread_loop(c720a7d0,c6f4ad28,c0efe270,3e5,c71bd588,...) at ithread_loop+0xa9 fork_exit(c09e2fb0,c720a7d0,c6f4ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f4ad60, ebp = 0 --- Tracing command intr pid 12 tid 100008 td 0xc720b2e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100007 td 0xc720b5c0 sched_switch(c720b5c0,0,109,191,2d21dc26,...) at sched_switch+0x3bc mi_switch(109,0,c0efe593,554,c720b5c0,...) at mi_switch+0x1eb ithread_loop(c720a7f0,c6f44d28,c0efe270,3e5,c71bd588,...) at ithread_loop+0x1fb fork_exit(c09e2fb0,c720a7f0,c6f44d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f44d60, ebp = 0 --- Tracing command idle pid 11 tid 100006 td 0xc71bf000 sched_switch(c71bf000,0,108,18c,47c0677b,...) at sched_switch+0x3bc mi_switch(108,0,c0f0598b,3b9,c71bf000,...) at mi_switch+0x1eb sched_idletd(0,c6f40d28,c0efe270,3e5,c71bd84c,...) at sched_idletd+0x19f fork_exit(c0a32300,0,c6f40d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f40d60, ebp = 0 --- Tracing command idle pid 11 tid 100005 td 0xc71bf2e0 cpustop_handler(2,13,c6f3dc04,c0d43c26,f5,...) at cpustop_handler+0x28 ipi_nmi_handler(f5,0,0,c124b178,c71bd84c,...) at ipi_nmi_handler+0x47 trap(c6f3dc10) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0d285d5, esp = 0xc6f3dc50, ebp = 0xc6f3dc50 --- acpi_cpu_c1(c6f3dc6c,c1064050,2,301,935,...) at acpi_cpu_c1+0x5 acpi_cpu_idle(0,c6f3dca0,c0d36809,0,ffffffff,...) at acpi_cpu_idle+0x122 cpu_idle_acpi(0,ffffffff,c10aca40,2,c6f3dce8,...) at cpu_idle_acpi+0x2f cpu_idle(0,c6f3dcc4,c0f0598b,3bb,c71bf2e0,...) at cpu_idle+0x89 sched_idletd(0,c6f3dd28,c0efe270,3e5,c71bd84c,...) at sched_idletd+0x263 fork_exit(c0a32300,0,c6f3dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f3dd60, ebp = 0 --- Tracing command idle pid 11 tid 100004 td 0xc71bf5c0 sched_switch(c71bf5c0,0,608,18c,75a8f0ce,...) at sched_switch+0x3bc mi_switch(608,0,c0f041bf,cd,0,...) at mi_switch+0x1eb critical_exit(0,ffffffff,c10ac3c0,2,c6f3ace8,...) at critical_exit+0xa8 cpu_idle(0,c6f3acc4,c0f0598b,3bb,c71bf5c0,...) at cpu_idle+0x97 sched_idletd(0,c6f3ad28,c0efe270,3e5,c71bd84c,...) at sched_idletd+0x263 fork_exit(c0a32300,0,c6f3ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f3ad60, ebp = 0 --- Tracing command idle pid 11 tid 100003 td 0xc71bf8a0 kdb_enter(c0d81b3e,c0f07859,c6f37ba4,c087c8cb,c7574000,...) at kdb_enter+0x3a kdb_break(c7574000,0,c124ab20,c7574080,c7574058,...) at kdb_break+0x28 uart_intr(c7574000,0,c124ab20,c6f37be8,0,...) at uart_intr+0x11b intr_event_handle(c7208d80,c6f37c10,c6f37c34,0,c73ed200,...) at intr_event_handle+0x76 intr_execute_handlers(c72070d0,c6f37c10,0,c6f37c50,c0d2cfd5,...) at intr_execute_handlers+0x49 lapic_handle_intr(3c,c6f37c10) at lapic_handle_intr+0x36 Xapic_isr1() at Xapic_isr1+0x35 --- interrupt, eip = 0xc0d285d5, esp = 0xc6f37c50, ebp = 0xc6f37c50 --- acpi_cpu_c1(c6f37c6c,c1064050,0,301,935,...) at acpi_cpu_c1+0x5 acpi_cpu_idle(0,c6f37ca0,c0d36809,0,ffffffff,...) at acpi_cpu_idle+0x122 cpu_idle_acpi(0,ffffffff,c10abd40,2,c6f37ce8,...) at cpu_idle_acpi+0x2f cpu_idle(0,c6f37cc4,c0f0598b,a05,c71bf8a0,...) at cpu_idle+0x89 sched_idletd(0,c6f37d28,c0efe270,3e5,c71bd84c,...) at sched_idletd+0x263 fork_exit(c0a32300,0,c6f37d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f37d60, ebp = 0 --- Tracing command init pid 1 tid 100002 td 0xc71bfb80 sched_switch(c71bfb80,0,104,191,9174d9ed,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c71bfb80,0,c0f091e0,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f091e0,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c71bdb10,6c,c0f0bf5f,100,0,...) at sleepq_wait_sig+0x17 _sleep(c71bdb10,c71bdb98,16c,c0f0bf5f,0,...) at _sleep+0x35c kern_wait(c71bfb80,ffffffff,c6f33c60,0,0,...) at kern_wait+0x626 sys_wait4(c71bfb80,c6f33cec,c0f5252a,c0f0bd96,286,...) at sys_wait4+0x3b syscall(c6f33d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x8060df7, esp = 0xbfbfe8bc, ebp = 0xbfbfe8d8 --- Tracing command audit pid 10 tid 100001 td 0xc71c4000 sched_switch(c71c4000,0,104,191,56754d13,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c71c4000,0,c0f091e0,268,c71c4000,...) at sleepq_switch+0x162 sleepq_wait(c121b100,0,c6f30c8c,1,0,...) at sleepq_wait+0x63 _cv_wait(c121b100,c121b0e4,c0f33c17,194,0,...) at _cv_wait+0x243 audit_worker(0,c6f30d28,c0efe270,3e5,c71be000,...) at audit_worker+0x84 fork_exit(c0c06f70,0,c6f30d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f30d60, ebp = 0 --- Tracing command kernel pid 0 tid 100077 td 0xc7802b80 sched_switch(c7802b80,0,104,191,6acd625d,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7802b80,0,c0f091e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c71c3200,0,c0ef68f2,0,0,...) at sleepq_wait+0x63 _sleep(c71c3200,c71c3218,0,c0ef68f2,0,...) at _sleep+0x372 taskqueue_thread_loop(c127d4a0,f17e9d28,c0efe270,3e5,c109bbe0,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a4f020,c127d4a0,f17e9d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xf17e9d60, ebp = 0 --- Tracing command kernel pid 0 tid 100067 td 0xc757d5c0 sched_switch(c757d5c0,0,104,191,a61722b,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c757d5c0,0,c0f091e0,28b,2,...) at sleepq_switch+0x162 sleepq_timedwait(c109dee4,0,c0ef68f2,2,0,...) at sleepq_timedwait+0x6b _sleep(c109dee4,0,0,c0ef68f2,bb8,...) at _sleep+0x342 pause(c0ef68f2,bb8,124,122,c109bc68,...) at pause+0x47 deadlkres(0,ed393d28,c0efe270,3e5,c109bbe0,...) at deadlkres+0x324 fork_exit(c09c32d0,0,ed393d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed393d60, ebp = 0 --- Tracing command kernel pid 0 tid 100061 td 0xc743f8a0 sched_switch(c743f8a0,0,104,191,a093b34b,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c743f8a0,0,c0f091e0,268,c78a18a0,...) at sleepq_switch+0x162 sleepq_wait(c71c3b00,0,c0f046e1,c0ef68f2,0,...) at sleepq_wait+0x63 msleep_spin(c71c3b00,c71c3b18,c0ef68f2,0,c0f01f07,...) at msleep_spin+0x21d taskqueue_thread_loop(c747749c,ed377d28,c0efe270,3e5,c109bbe0,...) at taskqueue_thread_loop+0x8f fork_exit(c0a4f020,c747749c,ed377d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed377d60, ebp = 0 --- Tracing command kernel pid 0 tid 100027 td 0xc72d3b80 sched_switch(c72d3b80,0,104,191,6959f2f2,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c72d3b80,0,c0f091e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c71c2580,0,c0ef68f2,0,0,...) at sleepq_wait+0x63 _sleep(c71c2580,c71c2598,0,c0ef68f2,0,...) at _sleep+0x372 taskqueue_thread_loop(c10baae8,c6f80d28,c0efe270,3e5,c109bbe0,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a4f020,c10baae8,c6f80d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f80d60, ebp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xc73b25c0 sched_switch(c73b25c0,0,104,191,70e30948,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c73b25c0,0,c0f091e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c71c2600,0,c0ef68f2,0,0,...) at sleepq_wait+0x63 _sleep(c71c2600,c71c2618,0,c0ef68f2,0,...) at _sleep+0x372 taskqueue_thread_loop(c109c578,c6f77d28,c0efe270,3e5,c109bbe0,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a4f020,c109c578,c6f77d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f77d60, ebp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xc73b28a0 sched_switch(c73b28a0,0,104,191,70e2f8c0,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c73b28a0,0,c0f091e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c71c2780,0,c0f046e1,c0ef68f2,0,...) at sleepq_wait+0x63 msleep_spin(c71c2780,c71c2798,c0ef68f2,0,c0f01f07,...) at msleep_spin+0x21d taskqueue_thread_loop(c1066cb8,c6f74d28,c0efe270,3e5,c109bbe0,...) at taskqueue_thread_loop+0x8f fork_exit(c0a4f020,c1066cb8,c6f74d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f74d60, ebp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xc73b2b80 sched_switch(c73b2b80,0,104,191,70e2ead8,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c73b2b80,0,c0f091e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c71c2780,0,c0f046e1,c0ef68f2,0,...) at sleepq_wait+0x63 msleep_spin(c71c2780,c71c2798,c0ef68f2,0,c0f01f07,...) at msleep_spin+0x21d taskqueue_thread_loop(c1066cb8,c6f71d28,c0efe270,3e5,c109bbe0,...) at taskqueue_thread_loop+0x8f fork_exit(c0a4f020,c1066cb8,c6f71d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f71d60, ebp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xc720b8a0 sched_switch(c720b8a0,0,104,191,70e2db3c,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c720b8a0,0,c0f091e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c71c2780,0,c0f046e1,c0ef68f2,0,...) at sleepq_wait+0x63 msleep_spin(c71c2780,c71c2798,c0ef68f2,0,c0f01f07,...) at msleep_spin+0x21d taskqueue_thread_loop(c1066cb8,c6f6ed28,c0efe270,3e5,c109bbe0,...) at taskqueue_thread_loop+0x8f fork_exit(c0a4f020,c1066cb8,c6f6ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f6ed60, ebp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xc720bb80 sched_switch(c720bb80,0,104,191,70dbe261,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c720bb80,0,c0f091e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c71c27c0,0,c0ef68f2,0,0,...) at sleepq_wait+0x63 _sleep(c71c27c0,c71c27d8,0,c0ef68f2,0,...) at _sleep+0x372 taskqueue_thread_loop(c121b6e4,c6f6bd28,c0efe270,3e5,c109bbe0,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a4f020,c121b6e4,c6f6bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f6bd60, ebp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xc72d28a0 sched_switch(c72d28a0,0,104,191,8cb9f32f,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c72d28a0,0,c0f091e0,268,0,...) at sleepq_switch+0x162 sleepq_wait(c71c2880,0,c0ef68f2,0,0,...) at sleepq_wait+0x63 _sleep(c71c2880,c71c2898,0,c0ef68f2,0,...) at _sleep+0x372 taskqueue_thread_loop(c10b94e0,c6f5fd28,c0efe270,3e5,c109bbe0,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a4f020,c10b94e0,c6f5fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f5fd60, ebp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xc109beb0 sched_switch(c109beb0,0,104,191,49599e19,...) at sched_switch+0x3bc mi_switch(104,0,c0f091e0,1f3,54,...) at mi_switch+0x1eb sleepq_switch(c109beb0,0,c0f091e0,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c109bbe0,54,c0f0629d,0,0,...) at sleepq_timedwait+0x6b _sleep(c109bbe0,0,54,c0f0629d,2710,...) at _sleep+0x342 scheduler(0,181ec00,181ec00,181e000,1825000,...) at scheduler+0x240 mi_startup() at mi_startup+0xac begin() at begin+0x2c db:0:allt> call doadump Physical memory: 3435 MB Physical memory: 3435 MB Dumping 145 MB:Dumping 145 MB: 130 130 114 114 98 98 82 82 66 66 50 50 34 34 18 18 2 2 Dump complete Dump complete = 0 db:0:doadump> reset (kgdb) proc 3252 [Switching to thread 123 (Thread 100110)]#0 cpustop_handler () at ../../../i386/i386/mp_machdep.c:1496 1496 CPU_SET_ATOMIC(cpu, &stopped_cpus); (kgdb) bt #0 cpustop_handler () at ../../../i386/i386/mp_machdep.c:1496 #1 0xc0d38ff7 in ipi_nmi_handler () at ../../../i386/i386/mp_machdep.c:1478 #2 0xc0d43c26 in trap (frame=0xf18bd450) at ../../../i386/i386/trap.c:219 #3 0xc0d2cc0c in calltrap () at ../../../i386/i386/exception.s:168 #4 0xc0a56750 in witness_lock (lock=0xc837d37c, flags=0x8, file=0xc0f39b79 "ufs/ffs/ffs_vfsops.c", line=0x5c2) at ../../../kern/subr_witness.c:1401 #5 0xc09fc783 in _mtx_lock_flags (m=0xc837d37c, opts=0x0, file=0xc0f39b70 "../../../ufs/ffs/ffs_vfsops.c", line=0x5c2) at ../../../kern/kern_mutex.c:206 #6 0xc0c4abf8 in ffs_sync (mp=0xc8181000, waitfor=0x1) at ../../../ufs/ffs/ffs_vfsops.c:1474 #7 0xc0ab5c22 in vfs_write_suspend (mp=0xc8181000) at ../../../kern/vfs_vnops.c:1155 #8 0xc0c2f08a in ffs_snapshot (mp=0xc8181000, snapfile=0xc7584a20 "/mnt/.snap/suj16") at ../../../ufs/ffs/ffs_snapshot.c:416 #9 0xc0c4d253 in ffs_mount (mp=0xc8181000) at ../../../ufs/ffs/ffs_vfsops.c:474 #10 0xc0a9e994 in vfs_donmount (td=0xc78a18a0, fsflags=0x11100, fsoptions=0xc8249500) at ../../../kern/vfs_mount.c:925 #11 0xc0a9f064 in sys_nmount (td=0xc78a18a0, uap=0xf18bdcec) at ../../../kern/vfs_mount.c:410 #12 0xc0d43814 in syscall (frame=0xf18bdd28) at subr_syscall.c:131 #13 0xc0d2cc71 in Xint0x80_syscall () at ../../../i386/i386/exception.s:266 #14 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) f 6 #6 0xc0c4abf8 in ffs_sync (mp=0xc8181000, waitfor=0x1) at ../../../ufs/ffs/ffs_vfsops.c:1474 1474 VI_LOCK(vp); (kgdb) l 1469 * Depend on the mntvnode_slock to keep things stable enough 1470 * for a quick test. Since there might be hundreds of 1471 * thousands of vnodes, we cannot afford even a subroutine 1472 * call unless there's a good chance that we have work to do. 1473 */ 1474 VI_LOCK(vp); 1475 if (vp->v_iflag & VI_DOOMED) { 1476 VI_UNLOCK(vp); 1477 continue; 1478 } (kgdb) p *vp $1 = {v_type = VREG, v_tag = 0xc0efa4e4 "ufs", v_op = 0xc1052180, v_data = 0xc838332c, v_mount = 0xc8181000, v_nmntvnodes = {tqe_next = 0xc7b79200, tqe_prev = 0xc839e584}, v_un = {vu_mount = 0x0, vu_socket = 0x0, vu_cdev = 0x0, vu_fifoinfo = 0x0}, v_hashlist = {le_next = 0x0, le_prev = 0xc7665140}, v_hash = 0x8051, v_cache_src = {lh_first = 0x0}, v_cache_dst = {tqh_first = 0x0, tqh_last = 0xc837d2e8}, v_cache_dd = 0x0, v_cstart = 0x0, v_lasta = 0x0, v_lastw = 0x0, v_clen = 0x0, v_lock = { lock_object = {lo_name = 0xc0efa4e4 "ufs", lo_flags = 0x57b0000, lo_data = 0x0, lo_witness = 0xc7169338}, lk_lock = 0x1, lk_exslpfail = 0x0, lk_timo = 0x33, lk_pri = 0x60, lk_stack = {depth = 0xc, pcs = {0xc09f451e, 0xc0c508d1, 0xc0d66bd5, 0xc0ab5e48, 0xc0ab75f8, 0xc0ab77e4, 0xc09ced33, 0xc09d12f0, 0xc09d16b9, 0xc09d185a, 0xc0d43814, 0xc0d2cc71, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}}, v_interlock = {lock_object = {lo_name = 0xc0f0b485 "vnode interlock", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc71621e0}, mtx_lock = 0xc78a18a0}, v_vnlock = 0xc837d310, v_holdcnt = 0x0, v_usecount = 0x0, v_iflag = 0x100, v_vflag = 0x0, v_writecount = 0x0, v_freelist = {tqe_next = 0xc837d15c, tqe_prev = 0xc839e660}, v_bufobj = {bo_mtx = {lock_object = {lo_name = 0xc0f13f07 "bufobj interlock", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc71679a0}, mtx_lock = 0x4}, bo_clean = {bv_hd = {tqh_first = 0x0, tqh_last = 0xc837d3c4}, bv_root = 0x0, bv_cnt = 0x0}, bo_dirty = {bv_hd = {tqh_first = 0x0, tqh_last = 0xc837d3d4}, bv_root = 0x0, bv_cnt = 0x0}, bo_numoutput = 0x0, bo_flag = 0x0, bo_ops = 0xc103f260, bo_bsize = 0x8000, bo_object = 0xc7dfc660, bo_synclist = {le_next = 0x0, le_prev = 0x0}, bo_private = 0xc837d2b8, __bo_vnode = 0xc837d2b8}, v_pollinfo = 0x0, v_label = 0x0, v_lockf = 0x0} (kgdb) p *mp $2 = {mnt_mtx = {lock_object = {lo_name = 0xc0ef6091 "struct mount mtx", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc7162450}, mtx_lock = 0xc78a18a0}, mnt_gen = 0x1, mnt_list = {tqe_next = 0x0, tqe_prev = 0xc7ba32ec}, mnt_op = 0xc1051f20, mnt_vfc = 0xc1051d80, mnt_vnodecovered = 0xc7d4b6cc, mnt_syncer = 0xc7bc9570, mnt_ref = 0x63, mnt_nvnodelist = {tqh_first = 0xc7b8aae0, tqh_last = 0xc82e0998}, mnt_nvnodelistsize = 0x5f, mnt_writeopcount = 0x0, mnt_kern_flag = 0x68000044, mnt_flag = 0x1011100, mnt_noasync = 0x1, mnt_opt = 0xc782c900, mnt_optnew = 0xc79256b0, mnt_maxsymlinklen = 0x78, mnt_stat = {f_version = 0x20030518, f_type = 0x35, f_flags = 0x1100, f_bsize = 0x1000, f_iosize = 0x8000, f_blocks = 0x7dfa5, f_bfree = 0x7cf98, f_bavail = 0x72e58, f_files = 0x1fffe, f_ffree = 0x1ffce, f_syncwrites = 0x0, f_asyncwrites = 0x0, f_syncreads = 0x0, f_asyncreads = 0x0, f_spare = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, f_namemax = 0xff, f_owner = 0x0, f_fsid = {val = {0x4e7f7392, 0xebad3ff2}}, f_charspare = '\0' , f_fstypename = "ufs", '\0' , f_mntfromname = "/dev/md5a", '\0' , f_mntonname = "/mnt", '\0' }, mnt_cred = 0xc785f600, mnt_data = 0xc7cad000, mnt_time = 0x0, mnt_iosize_max = 0x20000, mnt_export = 0xc7d1c800, mnt_label = 0x0, mnt_hashseed = 0x77716bff, mnt_lockref = 0x1, mnt_secondary_writes = 0x0, mnt_secondary_accwrites = 0x126f, mnt_susp_owner = 0xc78a18a0, mnt_gjprovider = 0x0, mnt_explock = {lock_object = {lo_name = 0xc0ef60a2 "explock", lo_flags = 0x5730000, lo_data = 0x0, lo_witness = 0xc7167ce0}, lk_lock = 0x1, lk_exslpfail = 0x0, lk_timo = 0x0, lk_pri = 0x60, lk_stack = {depth = 0x6, pcs = {0xc09f451e, 0xc0a963c7, 0xc0a9e9e3, 0xc0a9f064, 0xc0d43814, 0xc0d2cc71, 0x0 }}}} (kgdb) $ svn diff -x -p /usr/src/sys Index: /usr/src/sys/ufs/ffs/ffs_snapshot.c =================================================================== --- /usr/src/sys/ufs/ffs/ffs_snapshot.c (revision 225750) +++ /usr/src/sys/ufs/ffs/ffs_snapshot.c (working copy) @@ -203,7 +203,7 @@ ffs_snapshot(mp, snapfile) ufs2_daddr_t numblks, blkno, *blkp, *snapblklist; int error, cg, snaploc; int i, size, len, loc; - int flag; + uint64_t flag; struct timespec starttime = {0, 0}, endtime; char saved_nice = 0; long redo = 0, snaplistsize = 0; Index: /usr/src/sys/kern/subr_kdb.c =================================================================== --- /usr/src/sys/kern/subr_kdb.c (revision 225750) +++ /usr/src/sys/kern/subr_kdb.c (working copy) @@ -29,7 +29,6 @@ __FBSDID("$FreeBSD$"); #include "opt_kdb.h" #include "opt_stack.h" -#include "opt_watchdog.h" #include #include @@ -42,9 +41,6 @@ __FBSDID("$FreeBSD$"); #include #include #include -#ifdef SW_WATCHDOG -#include -#endif #include #include @@ -591,9 +587,6 @@ kdb_trap(int type, int code, struct trapframe *tf) cpuset_t other_cpus; #endif struct kdb_dbbe *be; -#ifdef SW_WATCHDOG - u_int wdoglvt; -#endif register_t intr; int handled; @@ -607,10 +600,6 @@ kdb_trap(int type, int code, struct trapframe *tf) intr = intr_disable(); -#ifdef SW_WATCHDOG - wdoglvt = wdog_kern_last_timeout(); - wdog_kern_pat(WD_TO_NEVER); -#endif #ifdef SMP other_cpus = all_cpus; CPU_CLR(PCPU_GET(cpuid), &other_cpus); @@ -642,9 +631,6 @@ kdb_trap(int type, int code, struct trapframe *tf) #ifdef SMP restart_cpus(stopped_cpus); #endif -#ifdef SW_WATCHDOG - wdog_kern_pat(wdoglvt); -#endif intr_restore(intr); -- Test scenario: suj16.sh $ ./suj16.sh load: 1.63 cmd: mksnap_ffs 3252 [running] 40.12r 0.00u 40.00s 98% 1020k load: 2.36 cmd: mksnap_ffs 3252 [running] 2022.42r 0.00u 2021.77s 100% 1020k