GENERIC from Sun Nov 7 19:54:03 2010 +0200, r214852+softdep_insmntque 974f335, vmcore.21 KDB: debugger backends: ddb KDB: current backend: ddb 524288K of memory above 4GB ignored Copyright (c) 1992-2010 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-CURRENT #0 r214852+974f335: Sun Nov 7 20:26:31 CET 2010 pho@x4.osted.lan:/var/tmp/deviant2/sys/i386/compile/PHO i386 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. CPU: AMD Phenom(tm) 9150e Quad-Core Processor (1800.02-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x100f23 Family = 10 Model = 2 Stepping = 3 Features=0x178bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x7ff TSC: P-state invariant real memory = 4294967296 (4096 MB) avail memory = 3536277504 (3372 MB) : Trying to mount root from ufs:/dev/ad4s1a [rw]... Setting hostuuid: 00000000-0000-0000-0000-00218515337d. Setting hostid: 0x6b64ac17. Starting ddb. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: /dev/ad4s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1a: clean, 817192 free (856 frags, 102042 blocks, 0.1% fragmentation) /dev/ad4s1f: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1f: clean, 794588 free (228 frags, 99295 blocks, 0.0% fragmentation) /dev/ad4s1g: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1g: clean, 117584947 free (443 frags, 14698063 blocks, 0.0% fragmentation) /dev/ad4s1d: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1d: clean, 4189888 free (104592 frags, 510662 blocks, 1.0% fragmentation) /dev/ad4s1e: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1e: clean, 20223995 free (13491 frags, 2526313 blocks, 0.1% fragmentation) Mounting local file systems:. Setting hostname: x4.osted.lan. re0: link state changed to DOWN Starting Network: lo0 re0 fwe0 fwip0. lo0: flags=8049 metric 0 mtu 16384 options=3 inet 127.0.0.1 netmask 0xff000000 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 nd6 options=21 re0: flags=8843 metric 0 mtu 1500 options=389b ether 00:21:85:15:33:7d inet 192.168.1.7 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::221:85ff:fe15:337d%re0 prefixlen 64 tentative scopeid 0x1 nd6 options=29 media: Ethernet autoselect (none) status: no carrier fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb ch 1 dma -1 fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 Starting devd. Starting Network: fwe0. fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb ch 1 dma -1 Starting Network: fwip0. fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 add net default: gateway 192.168.1.1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/lib /usr/local/lib/compat/pkg /usr/local/lib/wine a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout Creating and/or trimming log files. Starting syslogd. No core dumps found. Additional ABI support: linux. Starting rpcbind. NFS access cache time=60 Clearing /tmp (X related). Starting mountd. Starting nfsd. Updating motd:. Starting ntpd. Configuring syscons: keymap blanktime. Starting sshd. Starting cron. Local package initialization: backuplock order reversal: 1st 0xc7fbf5c8 ufs (ufs) @ kern/vfs_subr.c:2111 2nd 0xdad15600 bufwait (bufwait) @ ufs/ffs/ffs_softdep.c:11341 3rd 0xc8159df0 ufs (ufs) @ kern/vfs_subr.c:2111 KDB: stack backtrace: db_trace_self_wrapper(c0cec972,2e706564,31313a63,d313433,c0d1000a,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c08f626b,c0cefb90,c6d40168,c6d437d8,e982d890,...) at kdb_backtrace+0x2a _witness_debugger(c0cefb90,c8159df0,c0ce14b3,c6d437d8,c0cf6ecf,...) at _witness_debugger+0x25 witness_checkorder(c8159df0,9,c0cf6ec6,83f,0,...) at witness_checkorder+0x839 __lockmgr_args(c8159df0,80100,c8159e5c,0,0,...) at __lockmgr_args+0x814 ffs_lock(e982d9b4,c0906dfb,c0cf6245,80100,c8159d98,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c0dfdd40,e982d9b4,109,c0e18bc0,c8159d98,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c8159d98,80100,c0cf6ec6,83f,4,...) at _vn_lock+0x78 vget(c8159d98,80100,c80f35a0,50,0,...) at vget+0xbb vfs_hash_get(c7fbc2d4,12d8400,80000,c80f35a0,e982db08,...) at vfs_hash_get+0xed ffs_vgetf(c7fbc2d4,12d8400,80000,e982db08,1,...) at ffs_vgetf+0x49 softdep_sync_metadata(c7fbf570,0,c0d13d32,144,0,...) at softdep_sync_metadata+0x6a3 ffs_syncvnode(c7fbf570,1,c80f35a0,555,c0cf6ec6,...) at ffs_syncvnode+0x3e2 ffs_sync(c7fbc2d4,1,c0cf6586,4fd,80,...) at ffs_sync+0x26f dounmount(c7fbc2d4,8000000,c80f35a0,482,5b85c5ea,...) at dounmount+0x447 unmount(c80f35a0,e982dcec,281760f5,1,0,...) at unmount+0x310 syscallenter(c80f35a0,e982dce4,c0c067fd,c0e4d930,0,...) at syscallenter+0x263 syscall(e982dd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, unmount), eip = 0x280dbadf, esp = 0xbfbfe66c, ebp = 0xbfbfe738 --- lock order reversal: 1st 0xc7cce5c8 ufs (ufs) @ kern/vfs_mount.c:1208 2nd 0xc7c7a880 devfs (devfs) @ ufs/ffs/ffs_softdep.c:1606 KDB: stack backtrace: db_trace_self_wrapper(c0cec972,36313a63,a0d3630,c0906100,c0c69717,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c08f626b,c0cefb77,c6d437d8,c6d436a0,e982da04,...) at kdb_backtrace+0x2a _witness_debugger(c0cefb77,c7c7a880,c0cdd02a,c6d436a0,c0d10e67,...) at _witness_debugger+0x25 witness_checkorder(c7c7a880,9,c0d10e5e,646,c7c7a8ec,...) at witness_checkorder+0x839 __lockmgr_args(c7c7a880,80400,c7c7a8ec,0,0,...) at __lockmgr_args+0x814 vop_stdlock(e982db24,c0ce7b4a,df,80400,c7c7a828,...) at vop_stdlock+0x65 VOP_LOCK1_APV(c0dd61e0,e982db24,0,c0e18bc0,c7c7a828,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c7c7a828,80400,c0d10e5e,646,c7fbc2d4,...) at _vn_lock+0x78 softdep_flushworklist(c7fbc2d4,e982dbd0,c80f35a0,563,c0cf6ec6,...) at softdep_flushworklist+0x47 ffs_sync(c7fbc2d4,1,c0cf6586,4fd,80,...) at ffs_sync+0x2fd dounmount(c7fbc2d4,8000000,c80f35a0,482,5b85c5ea,...) at dounmount+0x447 unmount(c80f35a0,e982dcec,281760f5,1,0,...) at unmount+0x310 syscallenter(c80f35a0,e982dce4,c0c067fd,c0e4d930,0,...) at syscallenter+0x263 syscall(e982dd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, unmount), eip = 0x280dbadf, esp = 0xbfbfe66c, ebp = 0xbfbfe738 --- ** /dev/ad4s1g ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes Nov 7 20:40:04 x4 su: pho to root on /dev/pts/0 ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups lock order reversal: 1st 0xdad3ac00 bufwait (bufwait) @ kern/vfs_bio.c:2659 2nd 0xc840d200 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:284 KDB: stack backtrace: db_trace_self_wrapper(c0cec972,2e687361,38323a63,a0d34,a7b,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c08f626b,c0cefb77,c6d40168,c6d43840,e97577e8,...) at kdb_backtrace+0x2a _witness_debugger(c0cefb77,c840d200,c0d14512,c6d43840,c0d141a4,...) at _witness_debugger+0x25 witness_checkorder(c840d200,9,c0d1419b,11c,0,...) at witness_checkorder+0x839 _sx_xlock(c840d200,0,c0d1419b,11c,c83aa3a0,...) at _sx_xlock+0x85 ufsdirhash_acquire(dad3aba0,dbc54018,1e8,dbc54028,e97578b8,...) at ufsdirhash_acquire+0x48 ufsdirhash_add(c83aa3a0,e9757914,28,e97578a4,e97578a8,...) at ufsdirhash_add+0x13 ufs_direnter(c83ed414,c83f0ae0,e9757914,e9757ba4,0,...) at ufs_direnter+0x749 ufs_makeinode(e9757ba4,c0dfe280,e9757b00,e9757a5c,c0c25855,...) at ufs_makeinode+0x5c5 ufs_create(e9757b00,c0d2aae9,0,0,e9757b78,...) at ufs_create+0x30 VOP_CREATE_APV(c0dfdd40,e9757b00,e9757ba4,e9757a98,0,...) at VOP_CREATE_APV+0xc5 vn_open_cred(e9757b78,e9757c2c,1a0,0,c7c16b80,...) at vn_open_cred+0x205 vn_open(e9757b78,e9757c2c,1a0,c8237498,0,...) at vn_open+0x3b kern_openat(c7cdb870,ffffff9c,bfbfe838,0,602,...) at kern_openat+0x128 kern_open(c7cdb870,bfbfe838,0,601,1b0,...) at kern_open+0x35 open(c7cdb870,e9757cec,bfbfe838,1,0,...) at open+0x30 syscallenter(c7cdb870,e9757ce4,c0c067fd,c0e4d930,0,...) at syscallenter+0x263 syscall(e9757d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (5, FreeBSD ELF32, open), eip = 0x28172093, esp = 0xbfbfd1fc, ebp = 0xbfbfd218 --- 23467 files, 729034 used, 117584947 free (443 frags, 14698063 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** usage: kill [-s signal_name] pid ... kill -l [exit_status] kill -signal_name pid ... kill -signal_number pid ... fsck -y /tmp. Starting default moused. Starting inetd. Sun Nov 7 20:40:26 CET 2010 FreeBSD/i386 (x4.osted.lan) (console) login: panic: worklist_remove: 0xc8459300 jnewblk(0x10A) not on list cpuid = 1 KDB: enter: panic [ thread pid 20 tid 100074 ] Stopped at kdb_enter+0x3a: movl $0,kdb_why db> run pho db:0:pho> bt Tracing pid 20 tid 100074 td 0xc7161000 kdb_enter(c0ce93b9,c0ce93b9,c0d10f3c,e7538958,1,...) at kdb_enter+0x3a panic(c0d10f3c,c8459300,c0d12f0e,10a,c8459300,...) at panic+0x134 worklist_remove(c0fc6a98,0,c0d10e5e,2204,0,...) at worklist_remove+0x69 softdep_setup_blkfree(c7fbc87c,dae6a800,2eca8,0,8,...) at softdep_setup_blkfree+0x65 ffs_blkfree(c8b66900,c9b64000,c91ab414,2eca8,0,...) at ffs_blkfree+0x6fd indir_trunc(fffff7f4,ffffffff,2df30,0,8,...) at indir_trunc+0x56f indir_trunc(fffff7f3,ffffffff,c9b4e700,c994c700,c8b66900,...) at indir_trunc+0x796 handle_workitem_indirblk(4,c0ce7b4a,df,c994c700,c994c700,...) at handle_workitem_indirblk+0x76 handle_workitem_freeblocks(0,e7538c74,2,5e5,1e0,...) at handle_workitem_freeblocks+0x97 process_worklist_item(c0fc6a98,0,c0d10e5e,553,c7161000,...) at process_worklist_item+0x1f7 softdep_process_worklist(c7fbc87c,0,c0d10e5e,4e3,3e8,...) at softdep_process_worklist+0x8c softdep_flush(0,e7538d28,c0ce4597,33b,c7bf42a8,...) at softdep_flush+0x2a0 fork_exit(c0b07330,0,e7538d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7538d60, ebp = 0 --- db:0:bt> show allpcpu Current CPU: 1 cpuid = 0 dynamic pcpu = 0x60bd00 curthread = 0xc71612d0: pid 19 "syncer" curpcb = 0xe7535d80 fpcurthread = none idlethread = 0xc6d9b000: tid 100006 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: cpuid = 1 dynamic pcpu = 0x5cf0d00 curthread = 0xc7161000: pid 20 "softdepflush" curpcb = 0xe7538d80 fpcurthread = none idlethread = 0xc6d9b2d0: tid 100005 "idle: cpu1" APIC ID = 1 currentldt = 0x50 spin locks held: cpuid = 2 dynamic pcpu = 0x5cf3d00 curthread = 0xc6de95a0: pid 12 "swi4: clock" curpcb = 0xc6b27d80 fpcurthread = none idlethread = 0xc6d9b5a0: tid 100004 "idle: cpu2" APIC ID = 2 currentldt = 0x50 spin locks held: cpuid = 3 dynamic pcpu = 0x5cf6d00 curthread = 0xc7161870: pid 17 "bufdaemon" curpcb = 0xe752fd80 fpcurthread = none idlethread = 0xc6d9b870: tid 100003 "idle: cpu3" APIC ID = 3 currentldt = 0x50 spin locks held: db:0:allpcpu> show alllocks Process 22747 (suj6) thread 0xc841c000 (100177) exclusive lockmgr ufs (ufs) r = 0 (0xc98b4c94) locked @ kern/vfs_vnops.c:612 Process 22746 (suj6) thread 0xc9891000 (100238) exclusive lockmgr ufs (ufs) r = 0 (0xc9509b38) locked @ kern/vfs_vnops.c:612 Process 22745 (suj6) thread 0xc7cf5000 (100115) exclusive lockmgr bufwait (bufwait) r = 0 (0xdad256c0) locked @ kern/vfs_bio.c:2659 exclusive lockmgr ufs (ufs) r = 0 (0xc9186724) locked @ kern/vfs_vnops.c:612 Process 22744 (suj6) thread 0xc841e870 (100169) exclusive lockmgr ufs (ufs) r = 0 (0xc98b5724) locked @ kern/vfs_vnops.c:612 Process 22743 (suj6) thread 0xc7cda5a0 (100101) exclusive lockmgr ufs (ufs) r = 0 (0xc91855c8) locked @ kern/vfs_vnops.c:612 Process 22742 (suj6) thread 0xc7cf5b40 (100111) exclusive lockmgr bufwait (bufwait) r = 0 (0xdad3e680) locked @ kern/vfs_bio.c:2659 exclusive lockmgr bufwait (bufwait) r = 0 (0xdaf9c880) locked @ kern/vfs_bio.c:2659 exclusive lockmgr bufwait (bufwait) r = 0 (0xdad381c0) locked @ kern/vfs_bio.c:2659 exclusive lockmgr bufwait (bufwait) r = 0 (0xdae3df00) locked @ kern/vfs_bio.c:2659 exclusive lockmgr bufwait (bufwait) r = 0 (0xdad17820) locked @ kern/vfs_bio.c:2659 exclusive lockmgr bufwait (bufwait) r = 0 (0xdae790c0) locked @ kern/vfs_bio.c:2659 exclusive lockmgr bufwait (bufwait) r = 0 (0xdad42fa0) locked @ kern/vfs_bio.c:2659 exclusive lockmgr bufwait (bufwait) r = 0 (0xdaf58ca0) locked @ kern/vfs_bio.c:2659 exclusive lockmgr ufs (ufs) r = 0 (0xc997cc94) locked @ kern/vfs_vnops.c:612 Process 22741 (suj6) thread 0xc83a4000 (100160) exclusive lockmgr ufs (ufs) r = 0 (0xc98b4724) locked @ kern/vfs_vnops.c:612 Process 22740 (suj6) thread 0xc7cfa5a0 (100089) exclusive lockmgr bufwait (bufwait) r = 0 (0xdaeb5ae0) locked @ kern/vfs_bio.c:2659 exclusive lockmgr ufs (ufs) r = 0 (0xc9514b38) locked @ kern/vfs_vnops.c:612 Process 22739 (suj6) thread 0xc83a7870 (100147) exclusive lockmgr bufwait (bufwait) r = 0 (0xdafc1ce0) locked @ kern/vfs_bio.c:2659 exclusive lockmgr bufwait (bufwait) r = 0 (0xdad3a240) locked @ kern/vfs_bio.c:2659 exclusive lockmgr bufwait (bufwait) r = 0 (0xdaeef920) locked @ kern/vfs_bio.c:2659 exclusive lockmgr bufwait (bufwait) r = 0 (0xdad11020) locked @ kern/vfs_bio.c:2659 exclusive lockmgr bufwait (bufwait) r = 0 (0xdad54580) locked @ kern/vfs_bio.c:2659 exclusive lockmgr bufwait (bufwait) r = 0 (0xdae2f020) locked @ kern/vfs_bio.c:2659 exclusive lockmgr bufwait (bufwait) r = 0 (0xdaf4bb00) locked @ kern/vfs_bio.c:2659 exclusive lockmgr bufwait (bufwait) r = 0 (0xdaedb900) locked @ kern/vfs_bio.c:2659 exclusive lockmgr ufs (ufs) r = 0 (0xc98b59dc) locked @ kern/vfs_vnops.c:612 Process 22738 (suj6) thread 0xc8d065a0 (100187) exclusive lockmgr bufwait (bufwait) r = 0 (0xdae12c60) locked @ kern/vfs_bio.c:1912 exclusive lockmgr ufs (ufs) r = 0 (0xc91abdf0) locked @ kern/vfs_vnops.c:612 Process 1407 (sshd) thread 0xc704e870 (100077) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xc8245a34) locked @ kern/uipc_sockbuf.c:148 Process 20 (softdepflush) thread 0xc7161000 (100074) exclusive sleep mutex Softdep Lock (Softdep Lock) r = 0 (0xc0fc6a98) locked @ ufs/ffs/ffs_softdep.c:8708 exclusive lockmgr bufwait (bufwait) r = 0 (0xdae6a860) locked @ kern/vfs_bio.c:2659 Process 19 (syncer) thread 0xc71612d0 (100073) exclusive sleep mutex bufobj interlock (bufobj interlock) r = 0 (0xc950b920) locked @ kern/vfs_subr.c:3456 exclusive lockmgr syncer (syncer) r = 0 (0xc950b880) locked @ kern/vfs_subr.c:1721 Process 17 (bufdaemon) thread 0xc7161870 (100071) exclusive lockmgr devfs (devfs) r = 0 (0xc91ab46c) locked @ kern/vfs_bio.c:2373 db:0:alllocks> show lockedvnods Locked vnodes 0xc91ab414: tag devfs, type VCHR usecount 1, writecount 0, refcount 28 mountedhere 0xc7fbba00 flags () v_object 0xc80cadd0 ref 0 pages 94 lock type devfs: EXCL by thread 0xc7161870 (pid 17) #0 0xc08a739e at __lockmgr_args+0xbfe #1 0xc0944895 at vop_stdlock+0x65 #2 0xc0c24885 at VOP_LOCK1_APV+0xb5 #3 0xc0963bc8 at _vn_lock+0x78 #4 0xc093e23b at flushbufqueues+0x56b #5 0xc093e5b4 at buf_do_flush+0x94 #6 0xc093e70a at buf_daemon+0xea #7 0xc0893ee8 at fork_exit+0xb8 #8 0xc0bef634 at fork_trampoline+0x8 dev md5a 0xc950b828: tag syncer, type VNON usecount 1, writecount 0, refcount 2 mountedhere 0 flags () lock type syncer: EXCL by thread 0xc71612d0 (pid 19) #0 0xc08a739e at __lockmgr_args+0xbfe #1 0xc0944895 at vop_stdlock+0x65 #2 0xc0c24885 at VOP_LOCK1_APV+0xb5 #3 0xc0963bc8 at _vn_lock+0x78 #4 0xc0957072 at sync_vnode+0x142 #5 0xc09573f2 at sched_sync+0x282 #6 0xc0893ee8 at fork_exit+0xb8 #7 0xc0bef634 at fork_trampoline+0x8 0xc91abd98: tag ufs, type VREG usecount 1, writecount 1, refcount 452 mountedhere 0 flags () v_object 0xc8ed5880 ref 0 pages 1800 lock type ufs: EXCL by thread 0xc8d065a0 (pid 22738) #0 0xc08a739e at __lockmgr_args+0xbfe #1 0xc0b134c1 at ffs_lock+0xa1 #2 0xc0c24885 at VOP_LOCK1_APV+0xb5 #3 0xc0963bc8 at _vn_lock+0x78 #4 0xc0964f36 at vn_write+0x156 #5 0xc0909785 at dofilewrite+0x95 #6 0xc090af28 at kern_writev+0x58 #7 0xc090b03f at write+0x4f #8 0xc08ff663 at syscallenter+0x263 #9 0xc0c05f9f at syscall+0x4f #10 0xc0bef621 at Xint0x80_syscall+0x21 ino 6, on dev md5a 0xc98b5984: tag ufs, type VREG usecount 1, writecount 1, refcount 379 mountedhere 0 flags () v_object 0xc91c2bb0 ref 0 pages 1508 lock type ufs: EXCL by thread 0xc83a7870 (pid 22739) #0 0xc08a739e at __lockmgr_args+0xbfe #1 0xc0b134c1 at ffs_lock+0xa1 #2 0xc0c24885 at VOP_LOCK1_APV+0xb5 #3 0xc0963bc8 at _vn_lock+0x78 #4 0xc0964f36 at vn_write+0x156 #5 0xc0909785 at dofilewrite+0x95 #6 0xc090af28 at kern_writev+0x58 #7 0xc090b03f at write+0x4f #8 0xc08ff663 at syscallenter+0x263 #9 0xc0c05f9f at syscall+0x4f #10 0xc0bef621 at Xint0x80_syscall+0x21 ino 7, on dev md5a 0xc9514ae0: tag ufs, type VREG usecount 1, writecount 1, refcount 448 mountedhere 0 flags () v_object 0xc88b2ee0 ref 0 pages 1784 lock type ufs: EXCL by thread 0xc7cfa5a0 (pid 22740) #0 0xc08a739e at __lockmgr_args+0xbfe #1 0xc0b134c1 at ffs_lock+0xa1 #2 0xc0c24885 at VOP_LOCK1_APV+0xb5 #3 0xc0963bc8 at _vn_lock+0x78 #4 0xc0964f36 at vn_write+0x156 #5 0xc0909785 at dofilewrite+0x95 #6 0xc090af28 at kern_writev+0x58 #7 0xc090b03f at write+0x4f #8 0xc08ff663 at syscallenter+0x263 #9 0xc0c05f9f at syscall+0x4f #10 0xc0bef621 at Xint0x80_syscall+0x21 ino 8, on dev md5a 0xc9185570: tag ufs, type VREG usecount 1, writecount 1, refcount 405 mountedhere 0 flags () v_object 0xc8eccd48 ref 0 pages 1612 lock type ufs: EXCL by thread 0xc7cda5a0 (pid 22743) #0 0xc08a739e at __lockmgr_args+0xbfe #1 0xc0b134c1 at ffs_lock+0xa1 #2 0xc0c24885 at VOP_LOCK1_APV+0xb5 #3 0xc0963bc8 at _vn_lock+0x78 #4 0xc0964f36 at vn_write+0x156 #5 0xc0909785 at dofilewrite+0x95 #6 0xc090af28 at kern_writev+0x58 #7 0xc090b03f at write+0x4f #8 0xc08ff663 at syscallenter+0x263 #9 0xc0c05f9f at syscall+0x4f #10 0xc0bef621 at Xint0x80_syscall+0x21 ino 9, on dev md5a 0xc91866cc: tag ufs, type VREG usecount 1, writecount 1, refcount 410 mountedhere 0 flags () v_object 0xc9a49550 ref 0 pages 1632 lock type ufs: EXCL by thread 0xc7cf5000 (pid 22745) #0 0xc08a739e at __lockmgr_args+0xbfe #1 0xc0b134c1 at ffs_lock+0xa1 #2 0xc0c24885 at VOP_LOCK1_APV+0xb5 #3 0xc0963bc8 at _vn_lock+0x78 #4 0xc0964f36 at vn_write+0x156 #5 0xc0909785 at dofilewrite+0x95 #6 0xc090af28 at kern_writev+0x58 #7 0xc090b03f at write+0x4f #8 0xc08ff663 at syscallenter+0x263 #9 0xc0c05f9f at syscall+0x4f #10 0xc0bef621 at Xint0x80_syscall+0x21 ino 11, on dev md5a 0xc997cc3c: tag ufs, type VREG usecount 1, writecount 1, refcount 440 mountedhere 0 flags () v_object 0xc919bbb0 ref 0 pages 1752 lock type ufs: EXCL by thread 0xc7cf5b40 (pid 22742) #0 0xc08a739e at __lockmgr_args+0xbfe #1 0xc0b134c1 at ffs_lock+0xa1 #2 0xc0c24885 at VOP_LOCK1_APV+0xb5 #3 0xc0963bc8 at _vn_lock+0x78 #4 0xc0964f36 at vn_write+0x156 #5 0xc0909785 at dofilewrite+0x95 #6 0xc090af28 at kern_writev+0x58 #7 0xc090b03f at write+0x4f #8 0xc08ff663 at syscallenter+0x263 #9 0xc0c05f9f at syscall+0x4f #10 0xc0bef621 at Xint0x80_syscall+0x21 ino 14, on dev md5a 0xc98b56cc: tag ufs, type VREG usecount 1, writecount 1, refcount 414 mountedhere 0 flags () v_object 0xc83b7c38 ref 0 pages 1648 lock type ufs: EXCL by thread 0xc841e870 (pid 22744) #0 0xc08a739e at __lockmgr_args+0xbfe #1 0xc0b134c1 at ffs_lock+0xa1 #2 0xc0c24885 at VOP_LOCK1_APV+0xb5 #3 0xc0963bc8 at _vn_lock+0x78 #4 0xc0964f36 at vn_write+0x156 #5 0xc0909785 at dofilewrite+0x95 #6 0xc090af28 at kern_writev+0x58 #7 0xc090b03f at write+0x4f #8 0xc08ff663 at syscallenter+0x263 #9 0xc0c05f9f at syscall+0x4f #10 0xc0bef621 at Xint0x80_syscall+0x21 ino 16, on dev md5a 0xc98b46cc: tag ufs, type VREG usecount 1, writecount 1, refcount 449 mountedhere 0 flags () v_object 0xc919aa18 ref 0 pages 1788 lock type ufs: EXCL by thread 0xc83a4000 (pid 22741) #0 0xc08a739e at __lockmgr_args+0xbfe #1 0xc0b134c1 at ffs_lock+0xa1 #2 0xc0c24885 at VOP_LOCK1_APV+0xb5 #3 0xc0963bc8 at _vn_lock+0x78 #4 0xc0964f36 at vn_write+0x156 #5 0xc0909785 at dofilewrite+0x95 #6 0xc090af28 at kern_writev+0x58 #7 0xc090b03f at write+0x4f #8 0xc08ff663 at syscallenter+0x263 #9 0xc0c05f9f at syscall+0x4f #10 0xc0bef621 at Xint0x80_syscall+0x21 ino 19, on dev md5a 0xc98b4c3c: tag ufs, type VREG usecount 1, writecount 1, refcount 398 mountedhere 0 flags () v_object 0xc83d8770 ref 0 pages 1584 lock type ufs: EXCL by thread 0xc841c000 (pid 22747) #0 0xc08a739e at __lockmgr_args+0xbfe #1 0xc0b134c1 at ffs_lock+0xa1 #2 0xc0c24885 at VOP_LOCK1_APV+0xb5 #3 0xc0963bc8 at _vn_lock+0x78 #4 0xc0964f36 at vn_write+0x156 #5 0xc0909785 at dofilewrite+0x95 #6 0xc090af28 at kern_writev+0x58 #7 0xc090b03f at write+0x4f #8 0xc08ff663 at syscallenter+0x263 #9 0xc0c05f9f at syscall+0x4f #10 0xc0bef621 at Xint0x80_syscall+0x21 ino 22, on dev md5a 0xc9509ae0: tag ufs, type VREG usecount 1, writecount 1, refcount 409 mountedhere 0 flags () v_object 0xc83914c8 ref 0 pages 1628 lock type ufs: EXCL by thread 0xc9891000 (pid 22746) #0 0xc08a739e at __lockmgr_args+0xbfe #1 0xc0b134c1 at ffs_lock+0xa1 #2 0xc0c24885 at VOP_LOCK1_APV+0xb5 #3 0xc0963bc8 at _vn_lock+0x78 #4 0xc0964f36 at vn_write+0x156 #5 0xc0909785 at dofilewrite+0x95 #6 0xc090af28 at kern_writev+0x58 #7 0xc090b03f at write+0x4f #8 0xc08ff663 at syscallenter+0x263 #9 0xc0c05f9f at syscall+0x4f #10 0xc0bef621 at Xint0x80_syscall+0x21 ino 26, on dev md5a db:0:lockedvnods> show mount 0xc7c14b50 /dev/ad4s1a on / (ufs) 0xc7c15000 devfs on /dev (devfs) 0xc7fbc5a8 /dev/ad4s1f on /home (ufs) 0xc7fbc000 /dev/ad4s1d on /usr (ufs) 0xc7c15b50 /dev/ad4s1e on /var (ufs) 0xc8cf1b50 /dev/ad4s1g on /tmp (ufs) 0xc7fbc87c /dev/md5a on /mnt (ufs) More info: show mount db:0:mount> ps pid ppid pgrp uid state wmesg wchan cmd 22747 22057 22056 1001 D+ getblk 0xdaeb5ae0 suj6 22746 22057 22056 1001 R+ suj6 22745 22057 22056 1001 D+ getblk 0xdafc1ce0 suj6 22744 22057 22056 1001 D+ getblk 0xdaeb5ae0 suj6 22743 22057 22056 1001 R+ suj6 22742 22057 22056 1001 D+ getblk 0xdafc1ce0 suj6 22741 22057 22056 1001 D+ getblk 0xdaeb5ae0 suj6 22740 22057 22056 1001 D+ biowr 0xdaeb5a80 suj6 22739 22057 22056 1001 R+ suj6 22738 22057 22056 1001 DL+ biowr 0xdae12c00 suj6 22057 22056 22056 1001 S+ wait 0xc9888aa0 suj6 22056 22055 22056 1001 S+ wait 0xc838d000 bash 22055 22032 1425 0 S+ wait 0xc7c8a2a8 su 22049 0 0 0 DL mdwait 0xc83ac800 [md5] 22032 1425 1425 0 S+ wait 0xc838d2a8 sh 1680 1 1680 0 Ss+ ttyin 0xc6f8ea70 getty 1679 1 1679 0 Ss+ ttyin 0xc6f8ec70 getty 1678 1 1678 0 Ss+ ttyin 0xc6f8ee70 getty 1677 1 1677 0 Ss+ ttyin 0xc7170070 getty 1676 1 1676 0 Ss+ ttyin 0xc7170270 getty 1675 1 1675 0 Ss+ ttyin 0xc7170470 getty 1674 1 1674 0 Ss+ ttyin 0xc6f8d670 getty 1673 1 1673 0 Ss+ ttyin 0xc6f8d070 getty 1672 1 1672 0 Ss+ ttyin 0xc6f8d270 getty 1641 1 1641 0 Ss select 0xc7c17d64 inetd 1615 1 1615 0 Ss select 0xc7bc3b64 moused 1425 1421 1425 0 S+ wait 0xc838c000 sh 1421 1417 1421 0 S+ wait 0xc838c550 bash 1417 1416 1417 0 S+ pause 0xc838caf8 csh 1416 1410 1416 1001 S+ wait 0xc7c52550 su 1410 1409 1410 1001 Ss+ wait 0xc838d550 bash 1409 1407 1407 1001 S select 0xc72efba4 sshd 1407 1365 1407 0 Ss sbwait 0xc8245a60 sshd 1384 1 1384 0 Ss nanslp 0xc0e4e1c4 cron 1377 1 1377 25 Ss pause 0xc7c8a058 sendmail 1373 1 1373 0 Ss select 0xc72ef2e4 sendmail 1365 1 1365 0 Ss select 0xc7c17b24 sshd 1320 1 1320 0 Ss select 0xc7bc3c24 ntpd 1178 1177 1177 0 S (threaded) nfsd 100139 S rpcsvc 0xc7d1c950 nfsd: service 100138 S rpcsvc 0xc8246c50 nfsd: service 100137 S rpcsvc 0xc8246c90 nfsd: service 100099 S rpcsvc 0xc7d1c990 nfsd: master 1177 1 1177 0 Ss select 0xc72ef864 nfsd 1168 1 1168 0 Ss select 0xc72ef6e4 mountd 1071 1 1071 0 Ss select 0xc7c17e24 rpcbind 1038 1 1038 0 Ss select 0xc72ef424 syslogd 837 1 837 0 Ss select 0xc7c9a064 devd 21 0 0 0 DL flowclea 0xc0fbb328 [flowcleaner] 20 0 0 0 RL CPU 1 [softdepflush] 19 0 0 0 RL CPU 0 [syncer] 18 0 0 0 DL vlruwt 0xc7bf47f8 [vnlru] 17 0 0 0 RL CPU 3 [bufdaemon] 16 0 0 0 DL pgzero 0xc0fc895c [pagezero] 15 0 0 0 DL psleep 0xc0fc8550 [vmdaemon] 9 0 0 0 DL psleep 0xc0fc8518 [pagedaemon] 8 0 0 0 DL ccb_scan 0xc0e1a0d4 [xpt_thrd] 7 0 0 0 DL waiting_ 0xc0fbcb58 [sctp_iterator] 6 0 0 0 DL - 0xc705603c [fdc0] 5 0 0 0 SL - 0xc7059000 [fw0_probe] 14 0 0 0 DL (threaded) [usb] 100056 D - 0xc7034d34 [usbus5] 100055 D - 0xc7034d04 [usbus5] 100054 D - 0xc7034cd4 [usbus5] 100053 D - 0xc7034ca4 [usbus5] 100051 D - 0xc702db5c [usbus4] 100050 D - 0xc702db2c [usbus4] 100049 D - 0xc702dafc [usbus4] 100048 D - 0xc702dacc [usbus4] 100047 D - 0xc7026b5c [usbus3] 100046 D - 0xc7026b2c [usbus3] 100045 D - 0xc7026afc [usbus3] 100044 D - 0xc7026acc [usbus3] 100043 D - 0xc701db5c [usbus2] 100042 D - 0xc701db2c [usbus2] 100041 D - 0xc701dafc [usbus2] 100040 D - 0xc701dacc [usbus2] 100038 D - 0xc7017b5c [usbus1] 100037 D - 0xc7017b2c [usbus1] 100036 D - 0xc7017afc [usbus1] 100035 D - 0xc7017acc [usbus1] 100033 D - 0xc7007b5c [usbus0] 100032 D - 0xc7007b2c [usbus0] 100031 D - 0xc7007afc [usbus0] 100030 D - 0xc7007acc [usbus0] 13 0 0 0 DL - 0xc0e4e004 [yarrow] 4 0 0 0 DL - 0xc0e4bcc4 [g_down] 3 0 0 0 RL [g_up] 2 0 0 0 DL - 0xc0e4bcb8 [g_event] 12 0 0 0 RL (threaded) [intr] 100064 I [irq12: psm0] 100063 I [irq1: atkbd0] 100061 I [swi0: uart] 100058 I [irq20: fwohci0] 100057 I [irq14: ata0] 100052 I [irq19: ehci0] 100039 I [irq18: ohci2 ohci4] 100034 I [irq17: ohci1 ohci3] 100029 I [irq16: ohci0] 100028 I [irq22: atapci0] 100027 I [irq9: acpi0] 100026 I [swi2: cambio] 100022 I [swi6: task queue] 100021 I [swi6: Giant taskq] 100019 I [swi5: +] 100012 I [swi1: netisr 0] 100011 I [swi3: vm] 100010 I [swi4: clock] 100009 I [swi4: clock] 100008 I [swi4: clock] 100007 Run CPU 2 [swi4: clock] 11 0 0 0 RL (threaded) [idle] 100006 CanRun [idle: cpu0] 100005 CanRun [idle: cpu1] 100004 CanRun [idle: cpu2] 100003 CanRun [idle: cpu3] 1 0 1 0 SLs wait 0xc6d99d48 [init] 10 0 0 0 DL audit_wo 0xc0fc6300 [audit] 0 0 0 0 DLs (threaded) [kernel] 100065 D - 0xc0e4e004 [deadlkres] 100059 D - 0xc6eabb80 [fw0_taskq] 100025 D - 0xc6eabc80 [acpi_task_2] 100024 D - 0xc6eabc80 [acpi_task_1] 100023 D - 0xc6eabc80 [acpi_task_0] 100020 D - 0xc6eabd40 [thread taskq] 100018 D - 0xc6eabdc0 [kqueue taskq] 100013 D - 0xc6eabe00 [firmware taskq] 100000 D sched 0xc0e4bda0 [swapper] db:0:ps> allt Tracing command suj6 pid 22747 tid 100177 td 0xc841c000 sched_switch(c841c000,0,104,191,2de5ec68,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,50,...) at mi_switch+0x200 sleepq_switch(c841c000,0,c0cedbb9,268,50,...) at sleepq_switch+0x15f sleepq_wait(daeb5ae0,50,c0ce26c8,4,0,...) at sleepq_wait+0x63 __lockmgr_args(daeb5ae0,81900,c91ab50c,c0ce26c8,50,...) at __lockmgr_args+0xb1f getblk(c91ab414,e0,0,4000,0,...) at getblk+0x167 breadn(c91ab414,e0,0,4000,0,...) at breadn+0x44 bread(c91ab414,e0,0,4000,0,...) at bread+0x4c ffs_update(c98b4c3c,1,c0d13d32,163,0,...) at ffs_update+0x172 ffs_syncvnode(c98b4c3c,1,c0d10e5e,9ca,c98b4c3c,...) at ffs_syncvnode+0x48f softdep_prealloc(c98b4c3c,1,c0906117,c0c69717,e98de90c,...) at softdep_prealloc+0xdd ffs_balloc_ufs2(c98b4c3c,62c000,0,1000,c7c99780,...) at ffs_balloc_ufs2+0xaa ffs_write(e98deb94,c0d2aa1f,c98b4c3c,0,c98b4c3c,...) at ffs_write+0x2ad VOP_WRITE_APV(c0dfdd40,e98deb94,c98b4c3c,264,0,...) at VOP_WRITE_APV+0x136 vn_write(c82560e0,e98dec28,c7c99780,0,c841c000,...) at vn_write+0x1ca dofilewrite(e98dec28,ffffffff,ffffffff,0,c82560e0,...) at dofilewrite+0x95 kern_writev(c841c000,3,e98dec28,e98dec48,1,...) at kern_writev+0x58 write(c841c000,e98decec,e98ded28,c0cee3f2,0,...) at write+0x4f syscallenter(c841c000,e98dece4,e98dece4,0,c0e52fc0,...) at syscallenter+0x263 syscall(e98ded28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x28181253, esp = 0xbfbfd21c, ebp = 0xbfbfe8c8 --- Tracing command suj6 pid 22746 tid 100238 td 0xc9891000 sched_switch(c9891000,0,104,191,2db9256c,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,50,...) at mi_switch+0x200 sleepq_switch(c9891000,0,c0cedbb9,268,50,...) at sleepq_switch+0x15f sleepq_wait(daeb5ae0,50,c0ce26c8,4,0,...) at sleepq_wait+0x63 __lockmgr_args(daeb5ae0,81900,c91ab50c,c0ce26c8,50,...) at __lockmgr_args+0xb1f getblk(c91ab414,e0,0,4000,0,...) at getblk+0x167 breadn(c91ab414,e0,0,4000,0,...) at breadn+0x44 bread(c91ab414,e0,0,4000,0,...) at bread+0x4c ffs_update(c9509ae0,1,c0d13d32,163,0,...) at ffs_update+0x172 ffs_syncvnode(c9509ae0,1,c0d10e5e,9ca,c9509ae0,...) at ffs_syncvnode+0x48f softdep_prealloc(c9509ae0,1,c0906e9c,c0f8cd68,c98910ac,...) at softdep_prealloc+0xdd ffs_balloc_ufs2(c9509ae0,658000,0,1000,c7c99780,...) at ffs_balloc_ufs2+0xaa ffs_write(e99e6b94,c0d2aa1f,c9509ae0,0,c9509ae0,...) at ffs_write+0x2ad VOP_WRITE_APV(c0dfdd40,e99e6b94,c9509ae0,264,0,...) at VOP_WRITE_APV+0x136 vn_write(c7cad7e0,e99e6c28,c7c99780,0,c9891000,...) at vn_write+0x1ca dofilewrite(e99e6c28,ffffffff,ffffffff,0,c7cad7e0,...) at dofilewrite+0x95 kern_writev(c9891000,3,e99e6c28,e99e6c48,1,...) at kern_writev+0x58 write(c9891000,e99e6cec,e99e6d28,c0cee3f2,0,...) at write+0x4f syscallenter(c9891000,e99e6ce4,e99e6ce4,0,c0e52940,...) at syscallenter+0x263 syscall(e99e6d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x28181253, esp = 0xbfbfd21c, ebp = 0xbfbfe8c8 --- Tracing command suj6 pid 22745 tid 100115 td 0xc7cf5000 sched_switch(c7cf5000,0,104,191,2de70831,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,50,...) at mi_switch+0x200 sleepq_switch(c7cf5000,0,c0cedbb9,268,50,...) at sleepq_switch+0x15f sleepq_wait(dafc1ce0,50,c0ce26c8,4,0,...) at sleepq_wait+0x63 __lockmgr_args(dafc1ce0,81900,c91ab50c,c0ce26c8,50,...) at __lockmgr_args+0xb1f getblk(c91ab414,5bee0,0,4000,0,...) at getblk+0x167 breadn(c91ab414,5bee0,0,4000,0,...) at breadn+0x44 bread(c91ab414,5bee0,0,4000,0,...) at bread+0x4c ffs_alloccg(c964d1d0,1,28ce0,0,4000,...) at ffs_alloccg+0x123 ffs_hashalloc(28ce0,0,4000,4000,c0ae6760,...) at ffs_hashalloc+0x85 ffs_alloc(c964d1d0,197,0,28ce0,0,...) at ffs_alloc+0x2f2 ffs_balloc_ufs2(c91866cc,65c000,0,1000,c7c99780,...) at ffs_balloc_ufs2+0x1a98 ffs_write(e97d0b94,c0d2aa1f,c91866cc,0,c91866cc,...) at ffs_write+0x2ad VOP_WRITE_APV(c0dfdd40,e97d0b94,c91866cc,264,0,...) at VOP_WRITE_APV+0x136 vn_write(c7c5b4d0,e97d0c28,c7c99780,0,c7cf5000,...) at vn_write+0x1ca dofilewrite(e97d0c28,ffffffff,ffffffff,0,c7c5b4d0,...) at dofilewrite+0x95 kern_writev(c7cf5000,3,e97d0c28,e97d0c48,1,...) at kern_writev+0x58 write(c7cf5000,e97d0cec,e97d0d28,c0cee3f2,0,...) at write+0x4f syscallenter(c7cf5000,e97d0ce4,e97d0ce4,0,c0e522c0,...) at syscallenter+0x263 syscall(e97d0d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x28181253, esp = 0xbfbfd21c, ebp = 0xbfbfe8c8 --- Tracing command suj6 pid 22744 tid 100169 td 0xc841e870 sched_switch(c841e870,0,104,191,2de4744e,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,50,...) at mi_switch+0x200 sleepq_switch(c841e870,0,c0cedbb9,268,50,...) at sleepq_switch+0x15f sleepq_wait(daeb5ae0,50,c0ce26c8,4,0,...) at sleepq_wait+0x63 __lockmgr_args(daeb5ae0,81900,c91ab50c,c0ce26c8,50,...) at __lockmgr_args+0xb1f getblk(c91ab414,e0,0,4000,0,...) at getblk+0x167 breadn(c91ab414,e0,0,4000,0,...) at breadn+0x44 bread(c91ab414,e0,0,4000,0,...) at bread+0x4c ffs_update(c98b56cc,1,c0d13d32,163,0,...) at ffs_update+0x172 ffs_syncvnode(c98b56cc,1,c0d10e5e,9ca,c98b56cc,...) at ffs_syncvnode+0x48f softdep_prealloc(c98b56cc,1,c0906117,c0c69717,e98c490c,...) at softdep_prealloc+0xdd ffs_balloc_ufs2(c98b56cc,66c000,0,1000,c7c99780,...) at ffs_balloc_ufs2+0xaa ffs_write(e98c4b94,c0d2aa1f,c98b56cc,0,c98b56cc,...) at ffs_write+0x2ad VOP_WRITE_APV(c0dfdd40,e98c4b94,c98b56cc,264,0,...) at VOP_WRITE_APV+0x136 vn_write(c82562a0,e98c4c28,c7c99780,0,c841e870,...) at vn_write+0x1ca dofilewrite(e98c4c28,ffffffff,ffffffff,0,c82562a0,...) at dofilewrite+0x95 kern_writev(c841e870,3,e98c4c28,e98c4c48,1,...) at kern_writev+0x58 write(c841e870,e98c4cec,c08e3b7d,c0e524f8,0,...) at write+0x4f syscallenter(c841e870,e98c4ce4,e98c4cc0,c08e5518,c0e522c0,...) at syscallenter+0x263 syscall(e98c4d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x28181253, esp = 0xbfbfd21c, ebp = 0xbfbfe8c8 --- Tracing command suj6 pid 22743 tid 100101 td 0xc7cda5a0 sched_switch(c7cda5a0,0,602,18c,2de66483,...) at sched_switch+0x3bc mi_switch(602,0,c0ce9a6c,cd,4,...) at mi_switch+0x200 critical_exit(c7cda5a0,0,1,c6eae870,aa,...) at critical_exit+0xa8 ipi_bitmap_handler(8,28,28,c0e60064,aa,...) at ipi_bitmap_handler+0x82 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0x2f --- interrupt, eip = 0xc08aeebf, esp = 0xe978f678, ebp = 0xe978f688 --- _mtx_lock_spin(c0e60064,c7cda5a0,0,c0cedbb9,ed,...) at _mtx_lock_spin+0x3f _mtx_lock_spin_flags(c0e60064,0,c0cedbb9,ed,e978f6e8,...) at _mtx_lock_spin_flags+0x146 sleepq_lock(c0e4bcc4,c0856a79,c0e4bc28,0,0,...) at sleepq_lock+0x39 wakeup(c0e4bcc4,c836c8b8,c836c850,c7c2d680,c6eac450,...) at wakeup+0x17 g_io_request(c836c850,c7c2d680,0,dad155a0,e978f770,...) at g_io_request+0x34f g_vfs_strategy(c91ab50c,dad155a0) at g_vfs_strategy+0x71 ffs_geom_strategy(c91ab50c,dad155a0,e978f7a0,3,c7cda5a0,...) at ffs_geom_strategy+0x15f ufs_strategy(e978f7dc,c0d2a7e4,c7cda5a0,dad155a0,4,...) at ufs_strategy+0xd3 VOP_STRATEGY_APV(c0dfdd40,e978f7dc,c0cf4766,c0e18c20,c9185570,...) at VOP_STRATEGY_APV+0xe5 bufstrategy(c9185668,dad155a0,3a6,dad15600,dad155a0,...) at bufstrategy+0x61 bufwrite(dad155a0,dad15600,e978f878,c0b13125,dad155a0,...) at bufwrite+0x159 bawrite(dad155a0,0,c0d13d32,ff,0,...) at bawrite+0x5c ffs_syncvnode(c9185570,1,c0d10e5e,9ca,c9185570,...) at ffs_syncvnode+0x2a5 softdep_prealloc(c9185570,1,c0906117,c0c69717,e978f90c,...) at softdep_prealloc+0xdd ffs_balloc_ufs2(c9185570,648000,0,1000,c7c99780,...) at ffs_balloc_ufs2+0xaa ffs_write(e978fb94,c0d2aa1f,c9185570,0,c9185570,...) at ffs_write+0x2ad VOP_WRITE_APV(c0dfdd40,e978fb94,c9185570,264,0,...) at VOP_WRITE_APV+0x136 vn_write(c7c85d58,e978fc28,c7c99780,0,c7cda5a0,...) at vn_write+0x1ca dofilewrite(e978fc28,ffffffff,ffffffff,0,c7c85d58,...) at dofilewrite+0x95 kern_writev(c7cda5a0,3,e978fc28,e978fc48,1,...) at kern_writev+0x58 write(c7cda5a0,e978fcec,c08e3b7d,c0e51e78,0,...) at write+0x4f syscallenter(c7cda5a0,e978fce4,e978fcc0,c08e5518,c0e51c40,...) at syscallenter+0x263 syscall(e978fd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x28181253, esp = 0xbfbfd21c, ebp = 0xbfbfe8c8 --- Tracing command suj6 pid 22742 tid 100111 td 0xc7cf5b40 sched_switch(c7cf5b40,0,104,191,2de62d86,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,50,...) at mi_switch+0x200 sleepq_switch(c7cf5b40,0,c0cedbb9,268,50,...) at sleepq_switch+0x15f sleepq_wait(dafc1ce0,50,c0ce26c8,4,0,...) at sleepq_wait+0x63 __lockmgr_args(dafc1ce0,81900,c91ab50c,c0ce26c8,50,...) at __lockmgr_args+0xb1f getblk(c91ab414,5bee0,0,4000,0,...) at getblk+0x167 breadn(c91ab414,5bee0,0,4000,0,...) at breadn+0x44 bread(c91ab414,5bee0,0,4000,0,...) at bread+0x4c ffs_blkfree(c8b66900,c9b64000,c91ab414,28b20,0,...) at ffs_blkfree+0x2d2 handle_workitem_freefrag(c0fc6a98,0,c0d10e5e,139d,c0c03ac9,...) at handle_workitem_freefrag+0x74 setup_allocindir_phase2(c96b4a00,1b4,0,0,1b4,...) at setup_allocindir_phase2+0x452 softdep_setup_allocindir_page(c964d000,1b4,0,dad3e620,1a8,...) at softdep_setup_allocindir_page+0x140 ffs_reallocblks(e97c0a08,c0d2a663,0,c997cc3c,c950a2ac,...) at ffs_reallocblks+0x141b VOP_REALLOCBLKS_APV(c0dfdd40,e97c0a08,0,4000,0,...) at VOP_REALLOCBLKS_APV+0xd6 cluster_write(c997cc3c,daf58c40,6d4000,0,7f,...) at cluster_write+0x3c2 ffs_write(e97c0b94,c0d2aa1f,c997cc3c,0,c997cc3c,...) at ffs_write+0x51b VOP_WRITE_APV(c0dfdd40,e97c0b94,c997cc3c,264,0,...) at VOP_WRITE_APV+0x136 vn_write(c7c5a3b8,e97c0c28,c7c99780,0,c7cf5b40,...) at vn_write+0x1ca dofilewrite(e97c0c28,ffffffff,ffffffff,0,c7c5a3b8,...) at dofilewrite+0x95 kern_writev(c7cf5b40,3,e97c0c28,e97c0c48,1,...) at kern_writev+0x58 write(c7cf5b40,e97c0cec,e97c0d28,c0cee3f2,0,...) at write+0x4f syscallenter(c7cf5b40,e97c0ce4,e97c0ce4,0,c0e51c40,...) at syscallenter+0x263 syscall(e97c0d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x28181253, esp = 0xbfbfd21c, ebp = 0xbfbfe8c8 --- Tracing command suj6 pid 22741 tid 100160 td 0xc83a4000 sched_switch(c83a4000,0,104,191,2de601c5,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,50,...) at mi_switch+0x200 sleepq_switch(c83a4000,0,c0cedbb9,268,50,...) at sleepq_switch+0x15f sleepq_wait(daeb5ae0,50,c0ce26c8,4,0,...) at sleepq_wait+0x63 __lockmgr_args(daeb5ae0,81900,c91ab50c,c0ce26c8,50,...) at __lockmgr_args+0xb1f getblk(c91ab414,e0,0,4000,0,...) at getblk+0x167 breadn(c91ab414,e0,0,4000,0,...) at breadn+0x44 bread(c91ab414,e0,0,4000,0,...) at bread+0x4c ffs_update(c98b46cc,1,c0d13d32,163,0,...) at ffs_update+0x172 ffs_syncvnode(c98b46cc,1,c0d10e5e,9ca,c98b46cc,...) at ffs_syncvnode+0x48f softdep_prealloc(c98b46cc,1,c0906117,c0c69717,e98a990c,...) at softdep_prealloc+0xdd ffs_balloc_ufs2(c98b46cc,6f8000,0,1000,c7c99780,...) at ffs_balloc_ufs2+0xaa ffs_write(e98a9b94,c0d2aa1f,c98b46cc,0,c98b46cc,...) at ffs_write+0x2ad VOP_WRITE_APV(c0dfdd40,e98a9b94,c98b46cc,264,0,...) at VOP_WRITE_APV+0x136 vn_write(c7cad3b8,e98a9c28,c7c99780,0,c83a4000,...) at vn_write+0x1ca dofilewrite(e98a9c28,ffffffff,ffffffff,0,c7cad3b8,...) at dofilewrite+0x95 kern_writev(c83a4000,3,e98a9c28,e98a9c48,1,...) at kern_writev+0x58 write(c83a4000,e98a9cec,e98a9d28,c0cee3f2,0,...) at write+0x4f syscallenter(c83a4000,e98a9ce4,e98a9ce4,0,c0e51c40,...) at syscallenter+0x263 syscall(e98a9d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x28181253, esp = 0xbfbfd21c, ebp = 0xbfbfe8c8 --- Tracing command suj6 pid 22740 tid 100089 td 0xc7cfa5a0 sched_switch(c7cfa5a0,0,104,191,2de45b6a,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,4c,...) at mi_switch+0x200 sleepq_switch(c7cfa5a0,0,c0cedbb9,268,0,...) at sleepq_switch+0x15f sleepq_wait(daeb5a80,4c,c0cf4b06,0,0,...) at sleepq_wait+0x63 _sleep(daeb5a80,c6d568e4,4c,c0cf4b06,0,...) at _sleep+0x372 bwait(daeb5a80,4c,c0cf4b06,daeb5a80,e975f794,...) at bwait+0x6f bufwait(daeb5a80,daeb5a80,df,daeb5a80,c964d32c,...) at bufwait+0x48 bufwrite(daeb5a80,0,c0d1378c,756,0) at bufwrite+0x165 ffs_bufwrite(daeb5a80,c94c8a00,100,4000,0,...) at ffs_bufwrite+0x290 ffs_update(c9514ae0,1,c0d13d32,163,0,...) at ffs_update+0x28c ffs_syncvnode(c9514ae0,1,c0d10e5e,9ca,c9514ae0,...) at ffs_syncvnode+0x48f softdep_prealloc(c9514ae0,1,c0906117,c0c69717,e975f90c,...) at softdep_prealloc+0xdd ffs_balloc_ufs2(c9514ae0,6f4000,0,1000,c7c99780,...) at ffs_balloc_ufs2+0xaa ffs_write(e975fb94,c0d2aa1f,c9514ae0,0,c9514ae0,...) at ffs_write+0x2ad VOP_WRITE_APV(c0dfdd40,e975fb94,c9514ae0,264,0,...) at VOP_WRITE_APV+0x136 vn_write(c7c5a118,e975fc28,c7c99780,0,c7cfa5a0,...) at vn_write+0x1ca dofilewrite(e975fc28,ffffffff,ffffffff,0,c7c5a118,...) at dofilewrite+0x95 kern_writev(c7cfa5a0,3,e975fc28,e975fc48,1,...) at kern_writev+0x58 write(c7cfa5a0,e975fcec,e975fd28,c0cee3f2,0,...) at write+0x4f syscallenter(c7cfa5a0,e975fce4,e975fce4,0,c0e52fc0,...) at syscallenter+0x263 syscall(e975fd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x28181253, esp = 0xbfbfd21c, ebp = 0xbfbfe8c8 --- Tracing command suj6 pid 22739 tid 100147 td 0xc83a7870 sched_switch(c83a7870,0,103,18c,2de5a32b,...) at sched_switch+0x3bc mi_switch(103,0,c0cee4bc,2e2,c926b500,...) at mi_switch+0x200 turnstile_wait(c926b500,c6eaeb40,0,1ac,c0fbae14,...) at turnstile_wait+0x495 _mtx_lock_sleep(c0fbae14,c83a7870,0,c0cf4766,58b,...) at _mtx_lock_sleep+0x14d _mtx_lock_flags(c0fbae14,0,c0cf4766,58b,c0c10091,...) at _mtx_lock_flags+0xf7 brelse(dafc1c80,e1fe0c28,237a,0,0,...) at brelse+0x50f ffs_checkblk(4000,177,0,daec9aa0,16b,...) at ffs_checkblk+0x2d0 ffs_reallocblks(e9882a08,c0d2a663,0,c98b5984,c950a36c,...) at ffs_reallocblks+0x16a2 VOP_REALLOCBLKS_APV(c0dfdd40,e9882a08,0,4000,0,...) at VOP_REALLOCBLKS_APV+0xd6 cluster_write(c98b5984,daedb8a0,5e0000,0,7f,...) at cluster_write+0x3c2 ffs_write(e9882b94,c0d2aa1f,c98b5984,0,c98b5984,...) at ffs_write+0x51b VOP_WRITE_APV(c0dfdd40,e9882b94,c98b5984,264,0,...) at VOP_WRITE_APV+0x136 vn_write(c7c85a48,e9882c28,c7c99780,0,c83a7870,...) at vn_write+0x1ca dofilewrite(e9882c28,ffffffff,ffffffff,0,c7c85a48,...) at dofilewrite+0x95 kern_writev(c83a7870,3,e9882c28,e9882c48,1,...) at kern_writev+0x58 write(c83a7870,e9882cec,e9882d28,c0cee3f2,0,...) at write+0x4f syscallenter(c83a7870,e9882ce4,e9882ce4,0,c0e52940,...) at syscallenter+0x263 syscall(e9882d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x28181253, esp = 0xbfbfd21c, ebp = 0xbfbfe8c8 --- Tracing command suj6 pid 22738 tid 100187 td 0xc8d065a0 sched_switch(c8d065a0,0,104,191,2de39716,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,4c,...) at mi_switch+0x200 sleepq_switch(c8d065a0,0,c0cedbb9,268,0,...) at sleepq_switch+0x15f sleepq_wait(dae12c00,4c,c0cf4b06,0,0,...) at sleepq_wait+0x63 _sleep(dae12c00,c6d5609c,4c,c0cf4b06,0,...) at _sleep+0x372 bwait(dae12c00,4c,c0cf4b06,dae12c00,e98fc4c0,...) at bwait+0x6f bufwait(dae12c00,dae12c00,df,dae12c00,ddd90040,...) at bufwait+0x48 bufwrite(dae12c00,0,c0d1378c,756,dae12c00) at bufwrite+0x165 ffs_bufwrite(dae12c00,0,c0d10e5e,b42,c0d10e67,...) at ffs_bufwrite+0x290 softdep_process_journal(1f21,0,c8d065a0,dadd0a20,c91ab414,...) at softdep_process_journal+0x7a8 jwait(c0fc6a98,0,c0d10e5e,1f21,c7cda5a0,...) at jwait+0x24 softdep_disk_io_initiation(dadd0a20,c2bf53b0,0,dadd0a20,c91abd98,...) at softdep_disk_io_initiation+0x1ae5 ffs_geom_strategy(c91ab50c,dadd0a20,e98fc7a0,3,c8d065a0,...) at ffs_geom_strategy+0x14f ufs_strategy(e98fc7dc,c0d2a7e4,c8d065a0,dadd0a20,4,...) at ufs_strategy+0xd3 VOP_STRATEGY_APV(c0dfdd40,e98fc7dc,c0cf4766,c0e18c20,c91abd98,...) at VOP_STRATEGY_APV+0xe5 bufstrategy(c91abe90,dadd0a20,3a6,dadd0a80,dadd0a20,...) at bufstrategy+0x61 bufwrite(dadd0a20,dadd0a80,e98fc878,c0b13125,dadd0a20,...) at bufwrite+0x159 bawrite(dadd0a20,0,c0d13d32,ff,0,...) at bawrite+0x5c ffs_syncvnode(c91abd98,1,c0d10e5e,9ca,c91abd98,...) at ffs_syncvnode+0x2a5 softdep_prealloc(c91abd98,1,dadd0a80,e98fc8c4,c0906dfb,...) at softdep_prealloc+0xdd ffs_balloc_ufs2(c91abd98,701000,0,1000,c7c99780,...) at ffs_balloc_ufs2+0xaa ffs_write(e98fcb94,c0d2aa1f,c91abd98,0,c91abd98,...) at ffs_write+0x2ad VOP_WRITE_APV(c0dfdd40,e98fcb94,c91abd98,264,0,...) at VOP_WRITE_APV+0x136 vn_write(c999a7e0,e98fcc28,c7c99780,0,c8d065a0,...) at vn_write+0x1ca dofilewrite(e98fcc28,ffffffff,ffffffff,0,c999a7e0,...) at dofilewrite+0x95 kern_writev(c8d065a0,3,e98fcc28,e98fcc48,1,...) at kern_writev+0x58 write(c8d065a0,e98fccec,e98fcd28,c0cee3f2,0,...) at write+0x4f syscallenter(c8d065a0,e98fcce4,e98fcce4,0,c0e52fc0,...) at syscallenter+0x263 syscall(e98fcd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x28181253, esp = 0xbfbfd21c, ebp = 0xbfbfe8c8 --- Tracing command suj6 pid 22057 tid 100227 td 0xc9270870 sched_switch(c9270870,0,104,191,a7491a0c,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c9270870,0,c0cedbb9,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0cedbb9,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c9888aa0,5c,c0cf0511,100,0,...) at sleepq_wait_sig+0x17 _sleep(c9888aa0,c9888b28,15c,c0cf0511,0,...) at _sleep+0x35c kern_wait(c9270870,ffffffff,e99c5c44,0,0,...) at kern_wait+0xb86 wait4(c9270870,e99c5cec,bfbfd220,1,0,...) at wait4+0x3b syscallenter(c9270870,e99c5ce4,c0c067fd,c0e4d930,0,...) at syscallenter+0x263 syscall(e99c5d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x280efdeb, esp = 0xbfbfd1fc, ebp = 0xbfbfd218 --- Tracing command bash pid 22056 tid 100144 td 0xc7cfab40 sched_switch(c7cfab40,0,104,191,5ee9327f,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c7cfab40,0,c0cedbb9,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0cedbb9,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c838d000,5c,c0cf0511,100,0,...) at sleepq_wait_sig+0x17 _sleep(c838d000,c838d088,15c,c0cf0511,0,...) at _sleep+0x35c kern_wait(c7cfab40,ffffffff,e9878c44,0,0,...) at kern_wait+0xb86 wait4(c7cfab40,e9878cec,2836aa8c,1,0,...) at wait4+0x3b syscallenter(c7cfab40,e9878ce4,c0c067fd,c0e4d930,0,...) at syscallenter+0x263 syscall(e9878d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x282c2deb, esp = 0xbfbfe42c, ebp = 0xbfbfe448 --- Tracing command su pid 22055 tid 100095 td 0xc7c82000 sched_switch(c7c82000,0,104,191,5e2aae07,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c7c82000,0,c0cedbb9,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0cedbb9,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7c8a2a8,5c,c0cf0511,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7c8a2a8,c7c8a330,15c,c0cf0511,0,...) at _sleep+0x35c kern_wait(c7c82000,5628,e9777c44,2,0,...) at kern_wait+0xb86 wait4(c7c82000,e9777cec,281cba8c,1,0,...) at wait4+0x3b syscallenter(c7c82000,e9777ce4,c0c067fd,c0e4d930,0,...) at syscallenter+0x263 syscall(e9777d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x28123deb, esp = 0xbfbfe36c, ebp = 0xbfbfe388 --- Tracing command md5 pid 22049 tid 100196 td 0xc9270000 sched_switch(c9270000,0,104,191,2de9fbba,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,4c,...) at mi_switch+0x200 sleepq_switch(c9270000,0,c0cedbb9,268,0,...) at sleepq_switch+0x15f sleepq_wait(c83ac800,4c,c0cbad00,0,0,...) at sleepq_wait+0x63 _sleep(c83ac800,c83ac820,24c,c0cbad00,0,...) at _sleep+0x372 md_kthread(c83ac800,e994cd28,c0ce4597,33b,c92682a8,...) at md_kthread+0x115 fork_exit(c06d7d60,c83ac800,e994cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe994cd60, ebp = 0 --- Tracing command sh pid 22032 tid 100143 td 0xc80f3000 sched_switch(c80f3000,0,104,191,5d0894c6,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c80f3000,0,c0cedbb9,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0cedbb9,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c838d2a8,5c,c0cf0511,100,0,...) at sleepq_wait_sig+0x17 _sleep(c838d2a8,c838d330,15c,c0cf0511,0,...) at _sleep+0x35c kern_wait(c80f3000,ffffffff,e9875c44,2,0,...) at kern_wait+0xb86 wait4(c80f3000,e9875cec,e9875d28,c0cee3f2,0,...) at wait4+0x3b syscallenter(c80f3000,e9875ce4,e9875ce4,0,0,...) at syscallenter+0x263 syscall(e9875d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x28162deb, esp = 0xbfbfe87c, ebp = 0xbfbfe898 --- Tracing command getty pid 1680 tid 100123 td 0xc80f7870 sched_switch(c80f7870,0,104,191,ab780d63,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c80f7870,0,c0cedbb9,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0cedbb9,160,0,c80f7870,c80f7870,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6f8ea70,0,c0cf22ec,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6f8ea70,c0e4d930,c0cf1c8a,514,0,...) at _cv_wait_sig+0x243 tty_wait(c6f8ea00,c6f8ea70,e981bc28,1,0,...) at tty_wait+0x71 ttydisc_read(c6f8ea00,e981bc28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c6fc2800,e981bc28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7cadc08,e981bc28,c6d7be80,0,c80f7870,...) at devfs_read_f+0x7e dofileread(e981bc28,ffffffff,ffffffff,0,c7cadc08,...) at dofileread+0x9e kern_readv(c80f7870,0,e981bc28,e981bc48,1,...) at kern_readv+0x58 read(c80f7870,e981bcec,e981bd28,c0cee3f2,0,...) at read+0x4f syscallenter(c80f7870,e981bce4,e981bce4,0,0,...) at syscallenter+0x263 syscall(e981bd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196273, esp = 0xbfbfedac, ebp = 0xbfbfedd8 --- Tracing command getty pid 1679 tid 100184 td 0xc8d07000 sched_switch(c8d07000,0,104,191,abc2d75f,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c8d07000,0,c0cedbb9,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0cedbb9,160,0,c8d07000,c8d07000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6f8ec70,0,c0cf22ec,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6f8ec70,c0e4d930,c0cf1c8a,514,0,...) at _cv_wait_sig+0x243 tty_wait(c6f8ec00,c6f8ec70,e98f3c28,1,0,...) at tty_wait+0x71 ttydisc_read(c6f8ec00,e98f3c28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c6fc2900,e98f3c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c815d5b0,e98f3c28,c6d7be80,0,c8d07000,...) at devfs_read_f+0x7e dofileread(e98f3c28,ffffffff,ffffffff,0,c815d5b0,...) at dofileread+0x9e kern_readv(c8d07000,0,e98f3c28,e98f3c48,1,...) at kern_readv+0x58 read(c8d07000,e98f3cec,e98f3d28,c0cee3f2,0,...) at read+0x4f syscallenter(c8d07000,e98f3ce4,e98f3ce4,0,0,...) at syscallenter+0x263 syscall(e98f3d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196273, esp = 0xbfbfedac, ebp = 0xbfbfedd8 --- Tracing command getty pid 1678 tid 100157 td 0xc83a4870 sched_switch(c83a4870,0,104,191,ab958d76,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c83a4870,0,c0cedbb9,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0cedbb9,160,0,c83a4870,c83a4870,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6f8ee70,0,c0cf22ec,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6f8ee70,c0e4d930,c0cf1c8a,514,0,...) at _cv_wait_sig+0x243 tty_wait(c6f8ee00,c6f8ee70,e98a0c28,1,0,...) at tty_wait+0x71 ttydisc_read(c6f8ee00,e98a0c28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c6fc2a00,e98a0c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7c858f8,e98a0c28,c6d7be80,0,c83a4870,...) at devfs_read_f+0x7e dofileread(e98a0c28,ffffffff,ffffffff,0,c7c858f8,...) at dofileread+0x9e kern_readv(c83a4870,0,e98a0c28,e98a0c48,1,...) at kern_readv+0x58 read(c83a4870,e98a0cec,e98a0d28,c0cee3f2,0,...) at read+0x4f syscallenter(c83a4870,e98a0ce4,e98a0ce4,0,0,...) at syscallenter+0x263 syscall(e98a0d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196273, esp = 0xbfbfedac, ebp = 0xbfbfedd8 --- Tracing command getty pid 1677 tid 100159 td 0xc83a42d0 sched_switch(c83a42d0,0,104,191,abd7230b,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c83a42d0,0,c0cedbb9,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0cedbb9,160,0,c83a42d0,c83a42d0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7170070,0,c0cf22ec,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7170070,c0e4d930,c0cf1c8a,514,0,...) at _cv_wait_sig+0x243 tty_wait(c7170000,c7170070,e98a6c28,1,0,...) at tty_wait+0x71 ttydisc_read(c7170000,e98a6c28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c6fc2b00,e98a6c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7c5a578,e98a6c28,c6d7be80,0,c83a42d0,...) at devfs_read_f+0x7e dofileread(e98a6c28,ffffffff,ffffffff,0,c7c5a578,...) at dofileread+0x9e kern_readv(c83a42d0,0,e98a6c28,e98a6c48,1,...) at kern_readv+0x58 read(c83a42d0,e98a6cec,e98a6d28,c0cee3f2,0,...) at read+0x4f syscallenter(c83a42d0,e98a6ce4,e98a6ce4,0,0,...) at syscallenter+0x263 syscall(e98a6d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196273, esp = 0xbfbfedac, ebp = 0xbfbfedd8 --- Tracing command getty pid 1676 tid 100110 td 0xc7cf9000 sched_switch(c7cf9000,0,104,191,ab61a4cc,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7cf9000,0,c0cedbb9,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0cedbb9,160,0,c7cf9000,c7cf9000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7170270,0,c0cf22ec,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7170270,c0e4d930,c0cf1c8a,514,0,...) at _cv_wait_sig+0x243 tty_wait(c7170200,c7170270,e97bcc28,1,0,...) at tty_wait+0x71 ttydisc_read(c7170200,e97bcc28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c6fc2c00,e97bcc28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7c5a1f8,e97bcc28,c6d7be80,0,c7cf9000,...) at devfs_read_f+0x7e dofileread(e97bcc28,ffffffff,ffffffff,0,c7c5a1f8,...) at dofileread+0x9e kern_readv(c7cf9000,0,e97bcc28,e97bcc48,1,...) at kern_readv+0x58 read(c7cf9000,e97bccec,e97bcd28,c0cee3f2,0,...) at read+0x4f syscallenter(c7cf9000,e97bcce4,e97bcce4,0,0,...) at syscallenter+0x263 syscall(e97bcd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196273, esp = 0xbfbfedac, ebp = 0xbfbfedd8 --- Tracing command getty pid 1675 tid 100107 td 0xc7cf9870 sched_switch(c7cf9870,0,104,191,ab0d0239,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7cf9870,0,c0cedbb9,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0cedbb9,160,0,c7cf9870,c7cf9870,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7170470,0,c0cf22ec,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7170470,c0e4d930,c0cf1c8a,514,0,...) at _cv_wait_sig+0x243 tty_wait(c7170400,c7170470,e97b0c28,1,0,...) at tty_wait+0x71 ttydisc_read(c7170400,e97b0c28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c6fc2d00,e97b0c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7cad000,e97b0c28,c6d7be80,0,c7cf9870,...) at devfs_read_f+0x7e dofileread(e97b0c28,ffffffff,ffffffff,0,c7cad000,...) at dofileread+0x9e kern_readv(c7cf9870,0,e97b0c28,e97b0c48,1,...) at kern_readv+0x58 read(c7cf9870,e97b0cec,e97b0d28,c0cee3f2,0,...) at read+0x4f syscallenter(c7cf9870,e97b0ce4,e97b0ce4,0,0,...) at syscallenter+0x263 syscall(e97b0d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196273, esp = 0xbfbfedac, ebp = 0xbfbfedd8 --- Tracing command getty pid 1674 tid 100078 td 0xc7c83b40 sched_switch(c7c83b40,0,104,191,ab2cebd2,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7c83b40,0,c0cedbb9,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0cedbb9,160,0,c7c83b40,c7c83b40,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6f8d670,0,c0cf22ec,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6f8d670,c0e4d930,c0cf1c8a,514,0,...) at _cv_wait_sig+0x243 tty_wait(c6f8d600,c6f8d670,e9733c28,1,0,...) at tty_wait+0x71 ttydisc_read(c6f8d600,e9733c28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c6fc2e00,e9733c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c80cb8f8,e9733c28,c6d7be80,0,c7c83b40,...) at devfs_read_f+0x7e dofileread(e9733c28,ffffffff,ffffffff,0,c80cb8f8,...) at dofileread+0x9e kern_readv(c7c83b40,0,e9733c28,e9733c48,1,...) at kern_readv+0x58 read(c7c83b40,e9733cec,e9733d28,c0cee3f2,0,...) at read+0x4f syscallenter(c7c83b40,e9733ce4,e9733ce4,0,0,...) at syscallenter+0x263 syscall(e9733d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196273, esp = 0xbfbfedac, ebp = 0xbfbfedd8 --- Tracing command getty pid 1673 tid 100085 td 0xc7c82b40 sched_switch(c7c82b40,0,104,191,ab45e1bb,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7c82b40,0,c0cedbb9,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0cedbb9,160,0,c7c82b40,c7c82b40,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6f8d070,0,c0cf22ec,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6f8d070,c0e4d930,c0cf1c8a,514,0,...) at _cv_wait_sig+0x243 tty_wait(c6f8d000,c6f8d070,e974fc28,1,0,...) at tty_wait+0x71 ttydisc_read(c6f8d000,e974fc28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c715e000,e974fc28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7c855e8,e974fc28,c6d7be80,0,c7c82b40,...) at devfs_read_f+0x7e dofileread(e974fc28,ffffffff,ffffffff,0,c7c855e8,...) at dofileread+0x9e kern_readv(c7c82b40,0,e974fc28,e974fc48,1,...) at kern_readv+0x58 read(c7c82b40,e974fcec,e974fd28,c0cee3f2,0,...) at read+0x4f syscallenter(c7c82b40,e974fce4,e974fce4,0,0,...) at syscallenter+0x263 syscall(e974fd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196273, esp = 0xbfbfedac, ebp = 0xbfbfedd8 --- Tracing command getty pid 1672 tid 100112 td 0xc7cf5870 sched_switch(c7cf5870,0,104,191,aba0bcc2,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7cf5870,0,c0cedbb9,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08aedfa,c6f8d204,0,c0ce7b4a,c7cf5870,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6f8d270,0,e97c4ad8,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6f8d270,c6f8d204,c0cf1c8a,514,0,...) at _cv_wait_sig+0x243 tty_wait(c6f8d200,c6f8d270,e97c4c28,1,0,...) at tty_wait+0x71 ttydisc_read(c6f8d200,e97c4c28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c6de2c00,e97c4c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7d19e70,e97c4c28,c6d7be80,0,c7cf5870,...) at devfs_read_f+0x7e dofileread(e97c4c28,ffffffff,ffffffff,0,c7d19e70,...) at dofileread+0x9e kern_readv(c7cf5870,0,e97c4c28,e97c4c48,1,...) at kern_readv+0x58 read(c7cf5870,e97c4cec,e97c4d28,c0cee3f2,0,...) at read+0x4f syscallenter(c7cf5870,e97c4ce4,e97c4ce4,0,0,...) at syscallenter+0x263 syscall(e97c4d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196273, esp = 0xbfbfedac, ebp = 0xbfbfedd8 --- Tracing command inetd pid 1641 tid 100118 td 0xc80f8870 sched_switch(c80f8870,0,104,191,9daf9f38,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c80f8870,0,c0cedbb9,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08aedfa,c7c17d50,0,c0ce7b4a,c80f8870,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7c17d64,0,e980ca4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7c17d64,c7c17d50,c0ceff77,627,c815d118,...) at _cv_wait_sig+0x243 seltdwait(c815d118,58,c6d7be80,c80f8870,c0ce7b4a,...) at seltdwait+0xa2 kern_select(c80f8870,6,bfbfdfc0,0,0,0,20,c80f17f8,c80f17f8) at kern_select+0x504 select(c80f8870,e980ccec,e980cd28,c0cee3f2,0,...) at select+0x66 syscallenter(c80f8870,e980cce4,e980cce4,0,0,...) at syscallenter+0x263 syscall(e980cd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x281aa1f3, esp = 0xbfbfdf7c, ebp = 0xbfbfee08 --- Tracing command moused pid 1615 tid 100165 td 0xc84205a0 sched_switch(c84205a0,0,104,191,92e8945d,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c84205a0,0,c0cedbb9,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08aedfa,c7bc3b50,0,c0ce7b4a,c84205a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7bc3b64,0,e98b8a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7bc3b64,c7bc3b50,c0ceff77,627,c7c5ac08,...) at _cv_wait_sig+0x243 seltdwait(c7c5ac08,58,c6d7be80,c84205a0,0,...) at seltdwait+0xa2 kern_select(c84205a0,6,bfbfea24,0,0,0,20,c8411aa0,c8411aa0) at kern_select+0x504 select(c84205a0,e98b8cec,e98b8d28,c0cee3f2,0,...) at select+0x66 syscallenter(c84205a0,e98b8ce4,e98b8ce4,0,0,...) at syscallenter+0x263 syscall(e98b8d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x281b21f3, esp = 0xbfbfe9bc, ebp = 0xbfbfeb48 --- Tracing command sh pid 1425 tid 100150 td 0xc83a7000 sched_switch(c83a7000,0,104,191,400c7650,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c83a7000,0,c0cedbb9,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0cedbb9,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c838c000,5c,c0cf0511,100,0,...) at sleepq_wait_sig+0x17 _sleep(c838c000,c838c088,15c,c0cf0511,0,...) at _sleep+0x35c kern_wait(c83a7000,ffffffff,e988bc44,2,0,...) at kern_wait+0xb86 wait4(c83a7000,e988bcec,e988bd28,c0cee3f2,0,...) at wait4+0x3b syscallenter(c83a7000,e988bce4,e988bce4,0,0,...) at syscallenter+0x263 syscall(e988bd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x28162deb, esp = 0xbfbfe68c, ebp = 0xbfbfe6a8 --- Tracing command bash pid 1421 tid 100148 td 0xc83a75a0 sched_switch(c83a75a0,0,104,191,e49eb7e5,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c83a75a0,0,c0cedbb9,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0cedbb9,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c838c550,5c,c0cf0511,100,0,...) at sleepq_wait_sig+0x17 _sleep(c838c550,c838c5d8,15c,c0cf0511,0,...) at _sleep+0x35c kern_wait(c83a75a0,ffffffff,e9885c44,6,0,...) at kern_wait+0xb86 wait4(c83a75a0,e9885cec,e9885d28,c0cee3f2,0,...) at wait4+0x3b syscallenter(c83a75a0,e9885ce4,e9885ce4,0,0,...) at syscallenter+0x263 syscall(e9885d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x282c2deb, esp = 0xbfbfe7cc, ebp = 0xbfbfe7e8 --- Tracing command csh pid 1417 tid 100146 td 0xc83a7b40 sched_switch(c83a7b40,0,104,191,d05f7a74,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,68,...) at mi_switch+0x200 sleepq_switch(c83a7b40,0,c0cedbb9,1a7,68,...) at sleepq_switch+0x15f sleepq_catch_signals(c0cedbb9,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c838caf8,68,c0c9976f,100,0,...) at sleepq_wait_sig+0x17 _sleep(c838caf8,c838cb28,168,c0c9976f,0,...) at _sleep+0x35c kern_sigsuspend(c83a7b40,0,0,0,0,...) at kern_sigsuspend+0xba sigsuspend(c83a7b40,e987fcec,e987fd28,c0cee3f2,0,...) at sigsuspend+0x4d syscallenter(c83a7b40,e987fce4,e987fce4,0,0,...) at syscallenter+0x263 syscall(e987fd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x2817c87b, esp = 0xbfbfe5dc, ebp = 0xbfbfe648 --- Tracing command su pid 1416 tid 100136 td 0xc7c545a0 sched_switch(c7c545a0,0,104,191,76bfb4f7,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c7c545a0,0,c0cedbb9,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0cedbb9,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7c52550,5c,c0cf0511,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7c52550,c7c525d8,15c,c0cf0511,0,...) at _sleep+0x35c kern_wait(c7c545a0,589,e9848c44,2,0,...) at kern_wait+0xb86 wait4(c7c545a0,e9848cec,281cba8c,1,0,...) at wait4+0x3b syscallenter(c7c545a0,e9848ce4,c0c067fd,c0e4d930,0,...) at syscallenter+0x263 syscall(e9848d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x28123deb, esp = 0xbfbfe60c, ebp = 0xbfbfe628 --- Tracing command bash pid 1410 tid 100142 td 0xc80f32d0 sched_switch(c80f32d0,0,104,191,74b3d708,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c80f32d0,0,c0cedbb9,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0cedbb9,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c838d550,5c,c0cf0511,100,0,...) at sleepq_wait_sig+0x17 _sleep(c838d550,c838d5d8,15c,c0cf0511,0,...) at _sleep+0x35c kern_wait(c80f32d0,ffffffff,e9872c44,6,0,...) at kern_wait+0xb86 wait4(c80f32d0,e9872cec,e9872d28,c0cee3f2,0,...) at wait4+0x3b syscallenter(c80f32d0,e9872ce4,e9872ce4,0,0,...) at syscallenter+0x263 syscall(e9872d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x282c2deb, esp = 0xbfbfe9bc, ebp = 0xbfbfe9d8 --- Tracing command sshd pid 1409 tid 100082 td 0xc7cdc000 sched_switch(c7cdc000,0,104,191,5f232280,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7cdc000,0,c0cedbb9,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08aedfa,c72efb90,0,c0ce7b4a,c7cdc000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c72efba4,0,e9743a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c72efba4,c72efb90,c0ceff77,627,c8237380,...) at _cv_wait_sig+0x243 seltdwait(c8237380,58,c6d7b900,c7cdc000,246,...) at seltdwait+0xa2 kern_select(c7cdc000,a,288060b8,288060dc,0,0,20,c7cd3550,c7cd3550) at kern_select+0x504 select(c7cdc000,e9743cec,e9743d28,c0cee3f2,0,...) at select+0x66 syscallenter(c7cdc000,e9743ce4,e9743ce4,0,ec,...) at syscallenter+0x263 syscall(e9743d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x284b01f3, esp = 0xbfbfe1bc, ebp = 0xbfbfe208 --- Tracing command sshd pid 1407 tid 100077 td 0xc704e870 sched_switch(c704e870,0,104,191,8a2106d0,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,58,...) at mi_switch+0x200 sleepq_switch(c704e870,0,c0cedbb9,1a7,58,...) at sleepq_switch+0x15f sleepq_catch_signals(c0cedbb9,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8245a60,58,c0cf38aa,100,0,...) at sleepq_wait_sig+0x17 _sleep(c8245a60,c8245a20,158,c0cf38aa,0) at _sleep+0x35c sbwait(c82459fc,4,c0cf3979,5c9,c8245a20,...) at sbwait+0x76 soreceive_generic(c82459a8,0,e972fc28,0,0,...) at soreceive_generic+0x3f0 soreceive(c82459a8,0,e972fc28,0,0,0) at soreceive+0x38 soo_read(c7c5b508,e972fc28,c7c16c00,0,c704e870,...) at soo_read+0x4e dofileread(e972fc28,ffffffff,ffffffff,0,c7c5b508,...) at dofileread+0x9e kern_readv(c704e870,5,e972fc28,e972fc48,1,...) at kern_readv+0x58 read(c704e870,e972fcec,e972fd28,c0cee3f2,0,...) at read+0x4f syscallenter(c704e870,e972fce4,e972fce4,0,0,...) at syscallenter+0x263 syscall(e972fd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x284b0273, esp = 0xbfbfdd8c, ebp = 0xbfbfddc8 --- Tracing command cron pid 1384 tid 100093 td 0xc7cdb000 sched_switch(c7cdb000,0,104,191,257556cf,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c7cdb000,0,c0cedbb9,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(ea61,c08fc180,c7cdb000,0,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c0e4e1c4,5c,c0cea637,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c0e4e1c4,0,15c,c0cea637,ea61,...) at _sleep+0x328 kern_nanosleep(c7cdb000,e976fc34,e976fc3c,3c,0,...) at kern_nanosleep+0xc1 nanosleep(c7cdb000,e976fcec,e976fd28,c0cee3f2,0,...) at nanosleep+0x6f syscallenter(c7cdb000,e976fce4,e976fce4,0,0,...) at syscallenter+0x263 syscall(e976fd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x28184543, esp = 0xbfbfec6c, ebp = 0xbfbfec98 --- Tracing command sendmail pid 1377 tid 100097 td 0xc7c81b40 sched_switch(c7c81b40,0,104,191,7d8fca68,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,68,...) at mi_switch+0x200 sleepq_switch(c7c81b40,0,c0cedbb9,1a7,68,...) at sleepq_switch+0x15f sleepq_catch_signals(c0cedbb9,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7c8a058,68,c0c9976f,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7c8a058,c7c8a088,168,c0c9976f,0,...) at _sleep+0x35c kern_sigsuspend(c7c81b40,0,0,0,0,...) at kern_sigsuspend+0xba sigsuspend(c7c81b40,e977fcec,e977fd28,c0cee3f2,0,...) at sigsuspend+0x4d syscallenter(c7c81b40,e977fce4,e977fce4,0,0,...) at syscallenter+0x263 syscall(e977fd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x2833787b, esp = 0xbfbfcfbc, ebp = 0xbfbfcfe8 --- Tracing command sendmail pid 1373 tid 100134 td 0xc7c54b40 sched_switch(c7c54b40,0,104,191,3d5ba898,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7c54b40,0,c0cedbb9,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(e9840a14,c08aedfa,c72ef2d0,0,c7c54b40,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c72ef2e4,0,e9840a4c,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c72ef2e4,c72ef2d0,1389,627,c8237bd0,...) at _cv_timedwait_sig+0x252 seltdwait(e9840bf8,e9840c00,c6d7b880,c7c54b40,c15aea94,...) at seltdwait+0x8a kern_select(c7c54b40,5,bfbfc530,0,0,e9840c40,20,5,0) at kern_select+0x504 select(c7c54b40,e9840cec,e9840d28,c0cee3f2,0,...) at select+0x66 syscallenter(c7c54b40,e9840ce4,e9840ce4,0,ec,...) at syscallenter+0x263 syscall(e9840d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x283e01f3, esp = 0xbfbfc49c, ebp = 0xbfbfcfc8 --- Tracing command sshd pid 1365 tid 100098 td 0xc7c81870 sched_switch(c7c81870,0,104,191,3ed20fcb,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7c81870,0,c0cedbb9,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08aedfa,c7c17b10,0,c0ce7b4a,c7c81870,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7c17b24,0,e9783a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7c17b24,c7c17b10,c0ceff77,627,c80cb428,...) at _cv_wait_sig+0x243 seltdwait(c80cb428,58,c6d7b200,c7c81870,0,...) at seltdwait+0xa2 kern_select(c7c81870,5,2880c0b0,0,0,0,20,c7c53d48,c7c53d48) at kern_select+0x504 select(c7c81870,e9783cec,e9783d28,c0cee3f2,0,...) at select+0x66 syscallenter(c7c81870,e9783ce4,e9783ce4,fffffffe,0,...) at syscallenter+0x263 syscall(e9783d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x284b01f3, esp = 0xbfbfe27c, ebp = 0xbfbfee18 --- Tracing command ntpd pid 1320 tid 100120 td 0xc80f82d0 sched_switch(c80f82d0,0,104,191,ce3159d0,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c80f82d0,0,c0cedbb9,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08aedfa,c7bc3c10,0,c0ce7b4a,c80f82d0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7bc3c24,0,e9812a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7bc3c24,c7bc3c10,c0ceff77,627,c7cad930,...) at _cv_wait_sig+0x243 seltdwait(c7cad930,58,c6d7be80,c80f82d0,0,...) at seltdwait+0xa2 kern_select(c80f82d0,1c,bfbfed08,0,0,0,20,c80f12a8,c80f12a8) at kern_select+0x504 select(c80f82d0,e9812cec,e9812d80,e9812c88,0,...) at select+0x66 syscallenter(c80f82d0,e9812ce4,c0c064b6,fffffffe,246,...) at syscallenter+0x263 syscall(e9812d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x283581f3, esp = 0xbfbfecdc, ebp = 0xbfbfeda8 --- Tracing command nfsd pid 1178 tid 100139 td 0xc7cd42d0 sched_switch(c7cd42d0,0,104,191,37a92c2d,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7cd42d0,0,c0cedbb9,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(e9852be0,c08aedfa,c6fe5c80,0,c7cd42d0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c7d1c950,0,e9852c18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c7d1c950,c6fe5c80,1388,3b1,0,...) at _cv_timedwait_sig+0x252 svc_run_internal(e9852d14,c0893ee8,c6fe5c80,e9852d28,c0ce4597,...) at svc_run_internal+0x356 svc_thread_start(c6fe5c80,e9852d28,c0ce4597,33b,c7c53aa0,...) at svc_thread_start+0x10 fork_exit(c0abecb0,c6fe5c80,e9852d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x804c72e, eip = 0xc, esp = 0x33, ebp = 0 --- Tracing command nfsd pid 1178 tid 100138 td 0xc7cd45a0 sched_switch(c7cd45a0,0,104,191,37a8dc68,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7cd45a0,0,c0cedbb9,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(e984fbe0,c08aedfa,c6fe5c80,0,c7cd45a0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c8246c50,0,e984fc18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c8246c50,c6fe5c80,1388,3b1,0,...) at _cv_timedwait_sig+0x252 svc_run_internal(e984fd14,c0893ee8,c6fe5c80,e984fd28,c0ce4597,...) at svc_run_internal+0x356 svc_thread_start(c6fe5c80,e984fd28,c0ce4597,33b,c7c53aa0,...) at svc_thread_start+0x10 fork_exit(c0abecb0,c6fe5c80,e984fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x804c72e, eip = 0xc, esp = 0x33, ebp = 0 --- Tracing command nfsd pid 1178 tid 100137 td 0xc7cd4870 sched_switch(c7cd4870,0,104,191,37a93907,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7cd4870,0,c0cedbb9,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(e984cbe0,c08aedfa,c6fe5c80,0,c7cd4870,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c8246c90,0,e984cc18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c8246c90,c6fe5c80,1388,3b1,0,...) at _cv_timedwait_sig+0x252 svc_run_internal(e984cd14,c0893ee8,c6fe5c80,e984cd28,c0ce4597,...) at svc_run_internal+0x356 svc_thread_start(c6fe5c80,e984cd28,c0ce4597,33b,c7c53aa0,...) at svc_thread_start+0x10 fork_exit(c0abecb0,c6fe5c80,e984cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x804c72e, eip = 0xc, esp = 0x33, ebp = 0 --- Tracing command nfsd pid 1178 tid 100099 td 0xc7c815a0 sched_switch(c7c815a0,0,104,191,37a912de,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7c815a0,0,c0cedbb9,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(e9787ab0,c08aedfa,c6fe5c80,0,c7c815a0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c7d1c990,0,e9787ae8,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c7d1c990,c6fe5c80,1388,3b1,c7cd42d0,...) at _cv_timedwait_sig+0x252 svc_run_internal(c7c81748,14,c0d0e046,c0d0cb0c,e9787c0c,...) at svc_run_internal+0x356 svc_run(c6fe5c80,0,c0d0cde7,1c2,0,...) at svc_run+0x7b nfssvc_nfsd(bfbfe8a0,e9787c0c,c,c6d7be80,e9787c20,...) at nfssvc_nfsd+0xad nfssvc_nfsserver(c7c815a0,e9787cec,c7c53aa0,0,c7c815a0,...) at nfssvc_nfsserver+0x24f nfssvc(c7c815a0,e9787cec,e9787d28,c0cee3f2,0,...) at nfssvc+0x83 syscallenter(c7c815a0,e9787ce4,e9787ce4,0,0,...) at syscallenter+0x263 syscall(e9787d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (155, FreeBSD ELF32, nfssvc), eip = 0x280dc47b, esp = 0xbfbfe85c, ebp = 0xbfbfeac8 --- Tracing command nfsd pid 1177 tid 100092 td 0xc7c825a0 sched_switch(c7c825a0,0,104,191,211fe09f,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7c825a0,0,c0cedbb9,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08aedfa,c72ef850,0,c0ce7b4a,c7c825a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c72ef864,0,e976ba4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c72ef864,c72ef850,c0ceff77,627,c815d888,...) at _cv_wait_sig+0x243 seltdwait(c815d888,58,c6d7be80,c7c825a0,c0d16a71,...) at seltdwait+0xa2 kern_select(c7c825a0,5,bfbfecd0,0,0,0,20,c7c8a7f8,c7c8a7f8) at kern_select+0x504 select(c7c825a0,e976bcec,bfbffff4,1,0,...) at select+0x66 syscallenter(c7c825a0,e976bce4,c0c067fd,c0e4d930,0,...) at syscallenter+0x263 syscall(e976bd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x281841f3, esp = 0xbfbfeacc, ebp = 0xbfbfee08 --- Tracing command mountd pid 1168 tid 100091 td 0xc7cdb2d0 sched_switch(c7cdb2d0,0,104,191,5cc37b25,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7cdb2d0,0,c0cedbb9,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08aedfa,c72ef6d0,0,c0ce7b4a,c7cdb2d0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c72ef6e4,0,e9767a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c72ef6e4,c72ef6d0,c0ceff77,627,c815dd20,...) at _cv_wait_sig+0x243 seltdwait(c815dd20,58,c6d7be80,c7cdb2d0,c0f8de88,...) at seltdwait+0xa2 kern_select(c7cdb2d0,9,bfbfed5c,0,0,0,20,c7cd1aa0,c7cd1aa0) at kern_select+0x504 select(c7cdb2d0,e9767cec,e9767d28,c0cee3f2,0,...) at select+0x66 syscallenter(c7cdb2d0,e9767ce4,e9767ce4,0,246,...) at syscallenter+0x263 syscall(e9767d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x2819a1f3, esp = 0xbfbfed2c, ebp = 0xbfbfedf8 --- Tracing command rpcbind pid 1071 tid 100128 td 0xc80f3870 sched_switch(c80f3870,0,104,191,ce228fcd,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c80f3870,0,c0cedbb9,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(e982aa6c,c08aedfa,c7c17e10,0,c80f3870,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c7c17e24,0,e982aaa4,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c7c17e24,c7c17e10,7531,627,e982ab5c,...) at _cv_timedwait_sig+0x252 seltdwait(e982ac2c,e982ac34,535,c80f3870,e982ab2c,...) at seltdwait+0x8a poll(c80f3870,e982acec,e982ad28,c0cee3f2,0,...) at poll+0x300 syscallenter(c80f3870,e982ace4,e982ace4,0,0,...) at syscallenter+0x263 syscall(e982ad28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, poll), eip = 0x2814007b, esp = 0xbfbfcbcc, ebp = 0xbfbfed98 --- Tracing command syslogd pid 1038 tid 100080 td 0xc7c835a0 sched_switch(c7c835a0,0,104,191,6de53183,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7c835a0,0,c0cedbb9,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08aedfa,c72ef410,0,c0ce7b4a,c7c835a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c72ef424,0,e973ba4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c72ef424,c72ef410,c0ceff77,627,c80cbc78,...) at _cv_wait_sig+0x243 seltdwait(c80cbc78,58,c6d7be80,c7c835a0,0,...) at seltdwait+0xa2 kern_select(c7c835a0,9,2847f0ac,0,0,0,20,c7c8b550,c7c8b550) at kern_select+0x504 select(c7c835a0,e973bcec,e973bd28,c0cee3f2,0,...) at select+0x66 syscallenter(c7c835a0,e973bce4,e973bce4,0,246,...) at syscallenter+0x263 syscall(e973bd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x281991f3, esp = 0xbfbfe2bc, ebp = 0xbfbfee08 --- Tracing command devd pid 837 tid 100102 td 0xc7cda2d0 sched_switch(c7cda2d0,0,104,191,5c5069e3,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7cda2d0,0,c0cedbb9,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08aedfa,c7c9a050,0,c0ce7b4a,c7cda2d0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7c9a064,0,e9795a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7c9a064,c7c9a050,c0ceff77,627,c7c5ad58,...) at _cv_wait_sig+0x243 seltdwait(c7c5ad58,58,c6d7be80,c7cda2d0,3,...) at seltdwait+0xa2 kern_select(c7cda2d0,6,bfbfe980,0,0,0,20,c7c8bd48,c7c8bd48) at kern_select+0x504 select(c7cda2d0,e9795cec,e9795d28,c0cee3f2,0,...) at select+0x66 syscallenter(c7cda2d0,e9795ce4,e9795ce4,0,ec,...) at syscallenter+0x263 syscall(e9795d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x808b793, esp = 0xbfbfe94c, ebp = 0xbfbfee28 --- Tracing command flowcleaner pid 21 tid 100075 td 0xc704eb40 sched_switch(c704eb40,0,104,191,bdbd2869,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c704eb40,0,c0cedbb9,28b,c704eb40,...) at sleepq_switch+0x15f sleepq_timedwait(c0fbb328,0,e753bcac,1,0,...) at sleepq_timedwait+0x6b _cv_timedwait(c0fbb328,c0fbb330,4e20,620,c704eb40,...) at _cv_timedwait+0x252 flowtable_cleaner(0,e753bd28,c0ce4597,33b,c7bf4000,...) at flowtable_cleaner+0x1ae fork_exit(c096e2a0,0,e753bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe753bd60, ebp = 0 --- Tracing command softdepflush pid 20 tid 100074 td 0xc7161000 kdb_enter(c0ce93b9,c0ce93b9,c0d10f3c,e7538958,1,...) at kdb_enter+0x3a panic(c0d10f3c,c8459300,c0d12f0e,10a,c8459300,...) at panic+0x134 worklist_remove(c0fc6a98,0,c0d10e5e,2204,0,...) at worklist_remove+0x69 softdep_setup_blkfree(c7fbc87c,dae6a800,2eca8,0,8,...) at softdep_setup_blkfree+0x65 ffs_blkfree(c8b66900,c9b64000,c91ab414,2eca8,0,...) at ffs_blkfree+0x6fd indir_trunc(fffff7f4,ffffffff,2df30,0,8,...) at indir_trunc+0x56f indir_trunc(fffff7f3,ffffffff,c9b4e700,c994c700,c8b66900,...) at indir_trunc+0x796 handle_workitem_indirblk(4,c0ce7b4a,df,c994c700,c994c700,...) at handle_workitem_indirblk+0x76 handle_workitem_freeblocks(0,e7538c74,2,5e5,1e0,...) at handle_workitem_freeblocks+0x97 process_worklist_item(c0fc6a98,0,c0d10e5e,553,c7161000,...) at process_worklist_item+0x1f7 softdep_process_worklist(c7fbc87c,0,c0d10e5e,4e3,3e8,...) at softdep_process_worklist+0x8c softdep_flush(0,e7538d28,c0ce4597,33b,c7bf42a8,...) at softdep_flush+0x2a0 fork_exit(c0b07330,0,e7538d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7538d60, ebp = 0 --- Tracing command syncer pid 19 tid 100073 td 0xc71612d0 cpustop_handler(1,e7535af8,c0c060b6,c0d27040,ec,...) at cpustop_handler+0x34 ipi_nmi_handler(c0d27040,ec,0,c08aefc7,c7bf4550,...) at ipi_nmi_handler+0x2f trap(e7535b04) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc08aefbc, esp = 0xe7535b44, ebp = 0xe7535b60 --- _mtx_lock_sleep(c0fc6a98,c71612d0,0,c0d10e5e,2cd1,...) at _mtx_lock_sleep+0x9c _mtx_lock_flags(c0fc6a98,0,c0d10e5e,2cd1,c950b920,...) at _mtx_lock_flags+0xf7 softdep_get_depcounts(c7fbc87c,e7535bf8,e7535bf4,53e,c0ce7b4a,...) at softdep_get_depcounts+0x34 ffs_sync(c7fbc87c,3,c0cf6ec6,d94,c7fbc87c,...) at ffs_sync+0x10d sync_fsync(e7535c6c,c0d2a078,c950b8ec,e7535c6c,c950b828,...) at sync_fsync+0x18f VOP_FSYNC_APV(c0dec1c0,e7535c6c,c0cf6ec6,6b9,c71612d0,...) at VOP_FSYNC_APV+0xc5 sync_vnode(c0fbb138,c0fbb124,3e8,6e8,4e20,...) at sync_vnode+0x16b sched_sync(0,e7535d28,c0ce4597,33b,c7bf4550,...) at sched_sync+0x282 fork_exit(c0957170,0,e7535d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7535d60, ebp = 0 --- Tracing command vnlru pid 18 tid 100072 td 0xc71615a0 sched_switch(c71615a0,0,104,191,f2e16618,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,50,...) at mi_switch+0x200 sleepq_switch(c71615a0,0,c0cedbb9,28b,0,...) at sleepq_switch+0x15f sleepq_timedwait(c7bf47f8,50,c0cf7e3e,0,0,...) at sleepq_timedwait+0x6b _sleep(c7bf47f8,c0fbb0f8,250,c0cf7e3e,3e8,...) at _sleep+0x342 vnlru_proc(0,e7532d28,c0ce4597,33b,c7bf47f8,...) at vnlru_proc+0xe7 fork_exit(c09589b0,0,e7532d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7532d60, ebp = 0 --- Tracing command bufdaemon pid 17 tid 100071 td 0xc7161870 cpustop_handler(8,e752fb84,c0c060b6,c0d27040,ec,...) at cpustop_handler+0x34 ipi_nmi_handler(c0d27040,ec,0,c08aefc7,c7bf4aa0,...) at ipi_nmi_handler+0x2f trap(e752fb90) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc08aefd6, esp = 0xe752fbd0, ebp = 0xe752fbec --- _mtx_lock_sleep(c0fc6a98,c7161870,0,c0d10e5e,2bbd,...) at _mtx_lock_sleep+0xb6 _mtx_lock_flags(c0fc6a98,0,c0d10e5e,2bbd,dad37940,...) at _mtx_lock_flags+0xf7 softdep_count_dependencies(dad37940,0,c0cf4766,923,0,...) at softdep_count_dependencies+0x30 flushbufqueues(cf,c0ce7b4a,c7bf4b28,391,391,...) at flushbufqueues+0x450 buf_do_flush(c7bf4aa0,0,c0cf4766,8ad,3e8,...) at buf_do_flush+0x94 buf_daemon(0,e752fd28,c0ce4597,33b,c7bf4aa0,...) at buf_daemon+0xea fork_exit(c093e620,0,e752fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe752fd60, ebp = 0 --- Tracing command pagezero pid 16 tid 100070 td 0xc7161b40 sched_switch(c7161b40,0,104,191,a86b9c89,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7161b40,0,c0cedbb9,28b,0,...) at sleepq_switch+0x15f sleepq_timedwait(c0fc895c,0,c0d19525,0,0,...) at sleepq_timedwait+0x6b _sleep(c0fc895c,c0fc7400,0,c0d19525,493e0,...) at _sleep+0x342 vm_pagezero(0,e752cd28,c0ce4597,33b,c7bf4d48,...) at vm_pagezero+0xdc fork_exit(c0b4db20,0,e752cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe752cd60, ebp = 0 --- Tracing command vmdaemon pid 15 tid 100069 td 0xc7162000 sched_switch(c7162000,0,104,191,9de14904,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,68,...) at mi_switch+0x200 sleepq_switch(c7162000,0,c0cedbb9,268,0,...) at sleepq_switch+0x15f sleepq_wait(c0fc8550,68,c0cf5161,0,0,...) at sleepq_wait+0x63 _sleep(c0fc8550,c0fc8554,68,c0cf5161,0,...) at _sleep+0x372 vm_daemon(0,e7529d28,c0ce4597,33b,c6d9a2a8,...) at vm_daemon+0x59 fork_exit(c0b47960,0,e7529d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7529d60, ebp = 0 --- Tracing command pagedaemon pid 9 tid 100068 td 0xc71622d0 sched_switch(c71622d0,0,104,191,eb5e7e5e,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,44,...) at mi_switch+0x200 sleepq_switch(c71622d0,0,c0cedbb9,28b,0,...) at sleepq_switch+0x15f sleepq_timedwait(c0fc8518,44,c0cf5161,0,0,...) at sleepq_timedwait+0x6b _sleep(c0fc8518,c0fc7400,44,c0cf5161,1388,...) at _sleep+0x342 vm_pageout(0,e7526d28,c0ce4597,33b,c6d9a550,...) at vm_pageout+0x2c7 fork_exit(c0b489d0,0,e7526d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7526d60, ebp = 0 --- Tracing command xpt_thrd pid 8 tid 100067 td 0xc71625a0 sched_switch(c71625a0,0,104,191,98e77d4b,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,4c,...) at mi_switch+0x200 sleepq_switch(c71625a0,0,c0cedbb9,268,0,...) at sleepq_switch+0x15f sleepq_wait(c0e1a0d4,4c,c0c7fb6d,0,0,...) at sleepq_wait+0x63 _sleep(c0e1a0d4,c0e1a118,4c,c0c7fb6d,0,...) at _sleep+0x372 xpt_scanner_thread(0,e7523d28,c0ce4597,33b,c6d9a7f8,...) at xpt_scanner_thread+0x47 fork_exit(c0485cb0,0,e7523d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7523d60, ebp = 0 --- Tracing command sctp_iterator pid 7 tid 100066 td 0xc7162870 sched_switch(c7162870,0,104,191,fa79a6e2,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7162870,0,c0cedbb9,268,0,...) at sleepq_switch+0x15f sleepq_wait(c0fbcb58,0,c0d03e42,0,0,...) at sleepq_wait+0x63 _sleep(c0fbcb58,c0fbcb20,0,c0d03e42,0,...) at _sleep+0x372 sctp_iterator_thread(0,e7520d28,c0ce4597,33b,c6d9aaa0,...) at sctp_iterator_thread+0x5c fork_exit(c09e8ab0,0,e7520d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7520d60, ebp = 0 --- Tracing command fdc0 pid 6 tid 100062 td 0xc70245a0 sched_switch(c70245a0,0,104,191,15d5142,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,4c,...) at mi_switch+0x200 sleepq_switch(c70245a0,0,c0cedbb9,28b,0,...) at sleepq_switch+0x15f sleepq_timedwait(c705603c,4c,c0cde0eb,0,0,...) at sleepq_timedwait+0x6b _sleep(c705603c,c70560f0,4c,c0cde0eb,3e8,...) at _sleep+0x342 fdc_thread(c7056000,e7514d28,c0ce4597,33b,c6d9ad48,...) at fdc_thread+0x27d fork_exit(c0bcc6a0,c7056000,e7514d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7514d60, ebp = 0 --- Tracing command fw0_probe pid 5 tid 100060 td 0xc7024b40 sched_switch(c7024b40,0,104,191,a04c2f69,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c7024b40,0,c0cedbb9,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0cedbb9,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7059000,5c,c0cde0eb,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7059000,c705d488,15c,c0cde0eb,0,...) at _sleep+0x35c fw_bus_probe_thread(c7059000,e7504d28,c0ce4597,33b,c6f79000,...) at fw_bus_probe_thread+0xa08 fork_exit(c067cd70,c7059000,e7504d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7504d60, ebp = 0 --- Tracing command usb pid 14 tid 100056 td 0xc704c870 sched_switch(c704c870,0,104,191,8df3e7ed,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c704c870,0,c0cedbb9,268,c704c870,...) at sleepq_switch+0x15f sleepq_wait(c7034d34,0,e74f1cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7034d34,c7034dd4,c0cd682c,6c,c7034d3c,...) at _cv_wait+0x243 usb_process(c7034d2c,e74f1d28,c0ce4597,33b,c6f792a8,...) at usb_process+0x193 fork_exit(c07e87e0,c7034d2c,e74f1d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74f1d60, ebp = 0 --- Tracing command usb pid 14 tid 100055 td 0xc704cb40 sched_switch(c704cb40,0,104,191,186fd5ad,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c704cb40,0,c0cedbb9,268,c704cb40,...) at sleepq_switch+0x15f sleepq_wait(c7034d04,0,e74eecac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7034d04,c7034dd4,c0cd682c,6c,c7034d0c,...) at _cv_wait+0x243 usb_process(c7034cfc,e74eed28,c0ce4597,33b,c6f792a8,...) at usb_process+0x193 fork_exit(c07e87e0,c7034cfc,e74eed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74eed60, ebp = 0 --- Tracing command usb pid 14 tid 100054 td 0xc704e000 sched_switch(c704e000,0,104,191,8d8c37c6,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c704e000,0,c0cedbb9,268,c704e000,...) at sleepq_switch+0x15f sleepq_wait(c7034cd4,0,e74ebcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7034cd4,c7034dd4,c0cd682c,6c,c7034cdc,...) at _cv_wait+0x243 usb_process(c7034ccc,e74ebd28,c0ce4597,33b,c6f792a8,...) at usb_process+0x193 fork_exit(c07e87e0,c7034ccc,e74ebd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74ebd60, ebp = 0 --- Tracing command usb pid 14 tid 100053 td 0xc704e2d0 sched_switch(c704e2d0,0,104,191,8d8c278c,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c704e2d0,0,c0cedbb9,268,c704e2d0,...) at sleepq_switch+0x15f sleepq_wait(c7034ca4,0,e74e8cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7034ca4,c7034dd4,c0cd682c,6c,c7034cac,...) at _cv_wait+0x243 usb_process(c7034c9c,e74e8d28,c0ce4597,33b,c6f792a8,...) at usb_process+0x193 fork_exit(c07e87e0,c7034c9c,e74e8d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74e8d60, ebp = 0 --- Tracing command usb pid 14 tid 100051 td 0xc7022000 sched_switch(c7022000,0,104,191,8d8c1315,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7022000,0,c0cedbb9,268,c7022000,...) at sleepq_switch+0x15f sleepq_wait(c702db5c,0,e74dfcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c702db5c,c702dbfc,c0cd682c,6c,c702db64,...) at _cv_wait+0x243 usb_process(c702db54,e74dfd28,c0ce4597,33b,c6f792a8,...) at usb_process+0x193 fork_exit(c07e87e0,c702db54,e74dfd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74dfd60, ebp = 0 --- Tracing command usb pid 14 tid 100050 td 0xc70222d0 sched_switch(c70222d0,0,104,191,f50719d,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c70222d0,0,c0cedbb9,268,c70222d0,...) at sleepq_switch+0x15f sleepq_wait(c702db2c,0,e74dccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c702db2c,c702dbfc,c0cd682c,6c,c702db34,...) at _cv_wait+0x243 usb_process(c702db24,e74dcd28,c0ce4597,33b,c6f792a8,...) at usb_process+0x193 fork_exit(c07e87e0,c702db24,e74dcd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74dcd60, ebp = 0 --- Tracing command usb pid 14 tid 100049 td 0xc70225a0 sched_switch(c70225a0,0,104,191,8d26e681,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c70225a0,0,c0cedbb9,268,c70225a0,...) at sleepq_switch+0x15f sleepq_wait(c702dafc,0,e74d9cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c702dafc,c702dbfc,c0cd682c,6c,c702db04,...) at _cv_wait+0x243 usb_process(c702daf4,e74d9d28,c0ce4597,33b,c6f792a8,...) at usb_process+0x193 fork_exit(c07e87e0,c702daf4,e74d9d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74d9d60, ebp = 0 --- Tracing command usb pid 14 tid 100048 td 0xc7022870 sched_switch(c7022870,0,104,191,8d26d20a,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7022870,0,c0cedbb9,268,c7022870,...) at sleepq_switch+0x15f sleepq_wait(c702dacc,0,e74d6cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c702dacc,c702dbfc,c0cd682c,6c,c702dad4,...) at _cv_wait+0x243 usb_process(c702dac4,e74d6d28,c0ce4597,33b,c6f792a8,...) at usb_process+0x193 fork_exit(c07e87e0,c702dac4,e74d6d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74d6d60, ebp = 0 --- Tracing command usb pid 14 tid 100047 td 0xc7022b40 sched_switch(c7022b40,0,104,191,8d26bea0,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7022b40,0,c0cedbb9,268,c7022b40,...) at sleepq_switch+0x15f sleepq_wait(c7026b5c,0,e74d2cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7026b5c,c7026bfc,c0cd682c,6c,c7026b64,...) at _cv_wait+0x243 usb_process(c7026b54,e74d2d28,c0ce4597,33b,c6f792a8,...) at usb_process+0x193 fork_exit(c07e87e0,c7026b54,e74d2d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74d2d60, ebp = 0 --- Tracing command usb pid 14 tid 100046 td 0xc7023000 sched_switch(c7023000,0,104,191,c18afc3,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7023000,0,c0cedbb9,268,c7023000,...) at sleepq_switch+0x15f sleepq_wait(c7026b2c,0,e74cfcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7026b2c,c7026bfc,c0cd682c,6c,c7026b34,...) at _cv_wait+0x243 usb_process(c7026b24,e74cfd28,c0ce4597,33b,c6f792a8,...) at usb_process+0x193 fork_exit(c07e87e0,c7026b24,e74cfd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74cfd60, ebp = 0 --- Tracing command usb pid 14 tid 100045 td 0xc70232d0 sched_switch(c70232d0,0,104,191,8746a277,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c70232d0,0,c0cedbb9,268,c70232d0,...) at sleepq_switch+0x15f sleepq_wait(c7026afc,0,e74cccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7026afc,c7026bfc,c0cd682c,6c,c7026b04,...) at _cv_wait+0x243 usb_process(c7026af4,e74ccd28,c0ce4597,33b,c6f792a8,...) at usb_process+0x193 fork_exit(c07e87e0,c7026af4,e74ccd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74ccd60, ebp = 0 --- Tracing command usb pid 14 tid 100044 td 0xc70235a0 sched_switch(c70235a0,0,104,191,87469162,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c70235a0,0,c0cedbb9,268,c70235a0,...) at sleepq_switch+0x15f sleepq_wait(c7026acc,0,e74c9cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7026acc,c7026bfc,c0cd682c,6c,c7026ad4,...) at _cv_wait+0x243 usb_process(c7026ac4,e74c9d28,c0ce4597,33b,c6f792a8,...) at usb_process+0x193 fork_exit(c07e87e0,c7026ac4,e74c9d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74c9d60, ebp = 0 --- Tracing command usb pid 14 tid 100043 td 0xc7023870 sched_switch(c7023870,0,104,191,87467c6a,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7023870,0,c0cedbb9,268,c7023870,...) at sleepq_switch+0x15f sleepq_wait(c701db5c,0,e74c5cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c701db5c,c701dbfc,c0cd682c,6c,c701db64,...) at _cv_wait+0x243 usb_process(c701db54,e74c5d28,c0ce4597,33b,c6f792a8,...) at usb_process+0x193 fork_exit(c07e87e0,c701db54,e74c5d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74c5d60, ebp = 0 --- Tracing command usb pid 14 tid 100042 td 0xc7023b40 sched_switch(c7023b40,0,104,191,b05cdef,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7023b40,0,c0cedbb9,268,c7023b40,...) at sleepq_switch+0x15f sleepq_wait(c701db2c,0,e74c2cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c701db2c,c701dbfc,c0cd682c,6c,c701db34,...) at _cv_wait+0x243 usb_process(c701db24,e74c2d28,c0ce4597,33b,c6f792a8,...) at usb_process+0x193 fork_exit(c07e87e0,c701db24,e74c2d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74c2d60, ebp = 0 --- Tracing command usb pid 14 tid 100041 td 0xc7024000 sched_switch(c7024000,0,104,191,86e14324,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7024000,0,c0cedbb9,268,c7024000,...) at sleepq_switch+0x15f sleepq_wait(c701dafc,0,e74bfcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c701dafc,c701dbfc,c0cd682c,6c,c701db04,...) at _cv_wait+0x243 usb_process(c701daf4,e74bfd28,c0ce4597,33b,c6f792a8,...) at usb_process+0x193 fork_exit(c07e87e0,c701daf4,e74bfd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74bfd60, ebp = 0 --- Tracing command usb pid 14 tid 100040 td 0xc7013000 sched_switch(c7013000,0,104,191,86e12f47,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7013000,0,c0cedbb9,268,c7013000,...) at sleepq_switch+0x15f sleepq_wait(c701dacc,0,e74bccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c701dacc,c701dbfc,c0cd682c,6c,c701dad4,...) at _cv_wait+0x243 usb_process(c701dac4,e74bcd28,c0ce4597,33b,c6f792a8,...) at usb_process+0x193 fork_exit(c07e87e0,c701dac4,e74bcd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74bcd60, ebp = 0 --- Tracing command usb pid 14 tid 100038 td 0xc70135a0 sched_switch(c70135a0,0,104,191,86e11c0b,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c70135a0,0,c0cedbb9,268,c70135a0,...) at sleepq_switch+0x15f sleepq_wait(c7017b5c,0,e74b5cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7017b5c,c7017bfc,c0cd682c,6c,c7017b64,...) at _cv_wait+0x243 usb_process(c7017b54,e74b5d28,c0ce4597,33b,c6f792a8,...) at usb_process+0x193 fork_exit(c07e87e0,c7017b54,e74b5d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74b5d60, ebp = 0 --- Tracing command usb pid 14 tid 100037 td 0xc7013870 sched_switch(c7013870,0,104,191,9f35853,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7013870,0,c0cedbb9,268,c7013870,...) at sleepq_switch+0x15f sleepq_wait(c7017b2c,0,e74b2cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7017b2c,c7017bfc,c0cd682c,6c,c7017b34,...) at _cv_wait+0x243 usb_process(c7017b24,e74b2d28,c0ce4597,33b,c6f792a8,...) at usb_process+0x193 fork_exit(c07e87e0,c7017b24,e74b2d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74b2d60, ebp = 0 --- Tracing command usb pid 14 tid 100036 td 0xc7013b40 sched_switch(c7013b40,0,104,191,867bfaaa,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7013b40,0,c0cedbb9,268,c7013b40,...) at sleepq_switch+0x15f sleepq_wait(c7017afc,0,e74afcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7017afc,c7017bfc,c0cd682c,6c,c7017b04,...) at _cv_wait+0x243 usb_process(c7017af4,e74afd28,c0ce4597,33b,c6f792a8,...) at usb_process+0x193 fork_exit(c07e87e0,c7017af4,e74afd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74afd60, ebp = 0 --- Tracing command usb pid 14 tid 100035 td 0xc7015000 sched_switch(c7015000,0,104,191,867be6c6,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7015000,0,c0cedbb9,268,c7015000,...) at sleepq_switch+0x15f sleepq_wait(c7017acc,0,e74accac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7017acc,c7017bfc,c0cd682c,6c,c7017ad4,...) at _cv_wait+0x243 usb_process(c7017ac4,e74acd28,c0ce4597,33b,c6f792a8,...) at usb_process+0x193 fork_exit(c07e87e0,c7017ac4,e74acd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74acd60, ebp = 0 --- Tracing command usb pid 14 tid 100033 td 0xc70155a0 sched_switch(c70155a0,0,104,191,867bd0c3,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c70155a0,0,c0cedbb9,268,c70155a0,...) at sleepq_switch+0x15f sleepq_wait(c7007b5c,0,e74a5cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7007b5c,c7007bfc,c0cd682c,6c,c7007b64,...) at _cv_wait+0x243 usb_process(c7007b54,e74a5d28,c0ce4597,33b,c6f792a8,...) at usb_process+0x193 fork_exit(c07e87e0,c7007b54,e74a5d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74a5d60, ebp = 0 --- Tracing command usb pid 14 tid 100032 td 0xc7015870 sched_switch(c7015870,0,104,191,7cde10d,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7015870,0,c0cedbb9,268,c7015870,...) at sleepq_switch+0x15f sleepq_wait(c7007b2c,0,e74a2cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7007b2c,c7007bfc,c0cd682c,6c,c7007b34,...) at _cv_wait+0x243 usb_process(c7007b24,e74a2d28,c0ce4597,33b,c6f792a8,...) at usb_process+0x193 fork_exit(c07e87e0,c7007b24,e74a2d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74a2d60, ebp = 0 --- Tracing command usb pid 14 tid 100031 td 0xc7015b40 sched_switch(c7015b40,0,104,191,860f87d1,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7015b40,0,c0cedbb9,268,c7015b40,...) at sleepq_switch+0x15f sleepq_wait(c7007afc,0,e749fcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7007afc,c7007bfc,c0cd682c,6c,c7007b04,...) at _cv_wait+0x243 usb_process(c7007af4,e749fd28,c0ce4597,33b,c6f792a8,...) at usb_process+0x193 fork_exit(c07e87e0,c7007af4,e749fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe749fd60, ebp = 0 --- Tracing command usb pid 14 tid 100030 td 0xc6eaf2d0 sched_switch(c6eaf2d0,0,104,191,860f738f,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c6eaf2d0,0,c0cedbb9,268,c6eaf2d0,...) at sleepq_switch+0x15f sleepq_wait(c7007acc,0,e749ccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7007acc,c7007bfc,c0cd682c,6c,c7007ad4,...) at _cv_wait+0x243 usb_process(c7007ac4,e749cd28,c0ce4597,33b,c6f792a8,...) at usb_process+0x193 fork_exit(c07e87e0,c7007ac4,e749cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe749cd60, ebp = 0 --- Tracing command yarrow pid 13 tid 100017 td 0xc6eae5a0 sched_switch(c6eae5a0,0,104,191,2b350057,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c6eae5a0,0,c0cedbb9,28b,2,...) at sleepq_switch+0x15f sleepq_timedwait(c0e4e004,0,c0cde0eb,2,0,...) at sleepq_timedwait+0x6b _sleep(c0e4e004,0,0,c0cde0eb,64,...) at _sleep+0x342 pause(c0cde0eb,64,c0cc9f67,111,0,...) at pause+0x47 random_kthread(0,c6b45d28,c0ce4597,33b,c6f79550,...) at random_kthread+0x1ef fork_exit(c0759d80,0,c6b45d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b45d60, ebp = 0 --- Tracing command g_down pid 4 tid 100016 td 0xc6eae870 sched_switch(c6eae870,0,104,191,2de65bbc,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,4c,...) at mi_switch+0x200 sleepq_switch(c6eae870,0,c0cedbb9,268,0,...) at sleepq_switch+0x15f sleepq_wait(c0e4bcc4,4c,c0cde0eb,0,0,...) at sleepq_wait+0x63 _sleep(c0e4bcc4,c0e4bc28,24c,c0cde0eb,0,...) at _sleep+0x372 g_io_schedule_down(c6eae870,0,c0cdf879,74,0,...) at g_io_schedule_down+0x5b g_down_procbody(0,c6b42d28,c0ce4597,33b,c6d99000,...) at g_down_procbody+0x8d fork_exit(c0857f60,0,c6b42d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b42d60, ebp = 0 --- Tracing command g_up pid 3 tid 100015 td 0xc6eaeb40 sched_switch(c6eaeb40,0,104,191,2de9560a,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,4c,...) at mi_switch+0x200 sleepq_switch(c6eaeb40,0,c0cedbb9,268,0,...) at sleepq_switch+0x15f sleepq_wait(c0e4bcc0,4c,c0cde0eb,0,0,...) at sleepq_wait+0x63 _sleep(c0e4bcc0,c0e4bc48,24c,c0cde0eb,0,...) at _sleep+0x372 g_io_schedule_up(c6eaeb40,0,c0cdf879,5d,0,...) at g_io_schedule_up+0x134 g_up_procbody(0,c6b3fd28,c0ce4597,33b,c6d992a8,...) at g_up_procbody+0x8d fork_exit(c0857ff0,0,c6b3fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b3fd60, ebp = 0 --- Tracing command g_event pid 2 tid 100014 td 0xc6eaf000 sched_switch(c6eaf000,0,104,191,2998f95e,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,4c,...) at mi_switch+0x200 sleepq_switch(c6eaf000,0,c0cedbb9,28b,0,...) at sleepq_switch+0x15f sleepq_timedwait(c0e4bcb8,4c,c0cde0eb,0,0,...) at sleepq_timedwait+0x6b _sleep(c0e4bcb8,0,4c,c0cde0eb,64,...) at _sleep+0x342 g_event_procbody(0,c6b3cd28,c0ce4597,33b,c6d99550,...) at g_event_procbody+0xcb fork_exit(c0858080,0,c6b3cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b3cd60, ebp = 0 --- Tracing command intr pid 12 tid 100064 td 0xc7164000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100063 td 0xc70242d0 sched_switch(c70242d0,0,109,191,860b8186,...) at sched_switch+0x3bc mi_switch(109,0,c0ce47dd,52d,c6de0ef0,...) at mi_switch+0x200 ithread_loop(c6de83b0,e7517d28,c0ce4597,33b,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0896e30,c6de83b0,e7517d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7517d60, ebp = 0 --- Tracing command intr pid 12 tid 100061 td 0xc7024870 sched_switch(c7024870,0,109,191,ab95f831,...) at sched_switch+0x3bc mi_switch(109,0,c0ce47dd,52d,c6de12f0,...) at mi_switch+0x200 ithread_loop(c6de8020,e7507d28,c0ce4597,33b,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0896e30,c6de8020,e7507d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7507d60, ebp = 0 --- Tracing command intr pid 12 tid 100058 td 0xc704c2d0 sched_switch(c704c2d0,0,109,191,858a059e,...) at sched_switch+0x3bc mi_switch(109,0,c0ce47dd,52d,c6de0570,...) at mi_switch+0x200 ithread_loop(c6de8370,e74fbd28,c0ce4597,33b,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0896e30,c6de8370,e74fbd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74fbd60, ebp = 0 --- Tracing command intr pid 12 tid 100057 td 0xc704c5a0 sched_switch(c704c5a0,0,109,191,9335b1de,...) at sched_switch+0x3bc mi_switch(109,0,c0ce47dd,52d,c6de0870,...) at mi_switch+0x200 ithread_loop(c6de8690,e74f7d28,c0ce4597,33b,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0896e30,c6de8690,e74f7d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74f7d60, ebp = 0 --- Tracing command intr pid 12 tid 100052 td 0xc704e5a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100039 td 0xc70132d0 sched_switch(c70132d0,0,109,191,ab6f021d,...) at sched_switch+0x3bc mi_switch(109,0,c0ce47dd,52d,c6de0670,...) at mi_switch+0x200 ithread_loop(c6de8580,e74b9d28,c0ce4597,33b,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0896e30,c6de8580,e74b9d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74b9d60, ebp = 0 --- Tracing command intr pid 12 tid 100034 td 0xc70152d0 sched_switch(c70152d0,0,109,191,a5a4d72c,...) at sched_switch+0x3bc mi_switch(109,0,c0ce47dd,52d,c6de06f0,...) at mi_switch+0x200 ithread_loop(c6de8140,e74a9d28,c0ce4597,33b,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0896e30,c6de8140,e74a9d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74a9d60, ebp = 0 --- Tracing command intr pid 12 tid 100029 td 0xc6eaf5a0 sched_switch(c6eaf5a0,0,109,191,a3d15380,...) at sched_switch+0x3bc mi_switch(109,0,c0ce47dd,52d,c6de0770,...) at mi_switch+0x200 ithread_loop(c6de8700,e7499d28,c0ce4597,33b,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0896e30,c6de8700,e7499d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7499d60, ebp = 0 --- Tracing command intr pid 12 tid 100028 td 0xc6eaf870 sched_switch(c6eaf870,0,109,191,877084f5,...) at sched_switch+0x3bc mi_switch(109,0,c0ce47dd,52d,c6de0470,...) at mi_switch+0x200 ithread_loop(c6de8420,e738ad28,c0ce4597,33b,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0896e30,c6de8420,e738ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe738ad60, ebp = 0 --- Tracing command intr pid 12 tid 100027 td 0xc6eafb40 sched_switch(c6eafb40,0,109,191,860c300c,...) at sched_switch+0x3bc mi_switch(109,0,c0ce47dd,52d,c6de0af0,...) at mi_switch+0x200 ithread_loop(c6de8630,c6b8ad28,c0ce4597,33b,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0896e30,c6de8630,c6b8ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b8ad60, ebp = 0 --- Tracing command intr pid 12 tid 100026 td 0xc6fa3000 sched_switch(c6fa3000,0,109,191,9c6c5dae,...) at sched_switch+0x3bc mi_switch(109,0,c0ce47dd,52d,c6fa2070,...) at mi_switch+0x200 ithread_loop(c6de8760,c6b60d28,c0ce4597,33b,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0896e30,c6de8760,c6b60d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b60d60, ebp = 0 --- Tracing command intr pid 12 tid 100022 td 0xc6fa3b40 sched_switch(c6fa3b40,0,109,191,d1618c28,...) at sched_switch+0x3bc mi_switch(109,0,c0ce47dd,52d,c6fa2170,...) at mi_switch+0x200 ithread_loop(c6de8770,c6b54d28,c0ce4597,33b,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0896e30,c6de8770,c6b54d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b54d60, ebp = 0 --- Tracing command intr pid 12 tid 100021 td 0xc6de9870 sched_switch(c6de9870,0,109,191,3e7292e1,...) at sched_switch+0x3bc mi_switch(109,0,c0ce47dd,52d,c6de1070,...) at mi_switch+0x200 ithread_loop(c6de8780,c6b51d28,c0ce4597,33b,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0896e30,c6de8780,c6b51d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b51d60, ebp = 0 --- Tracing command intr pid 12 tid 100019 td 0xc6eae000 sched_switch(c6eae000,0,109,191,f9b95e59,...) at sched_switch+0x3bc mi_switch(109,0,c0ce47dd,52d,c6de0070,...) at mi_switch+0x200 ithread_loop(c6de8790,c6b4bd28,c0ce4597,33b,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0896e30,c6de8790,c6b4bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b4bd60, ebp = 0 --- Tracing command intr pid 12 tid 100012 td 0xc6da15a0 sched_switch(c6da15a0,0,109,191,7870a11c,...) at sched_switch+0x3bc mi_switch(109,0,c0ce47dd,52d,c6de00f0,...) at mi_switch+0x200 ithread_loop(c6de87a0,c6b36d28,c0ce4597,33b,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0896e30,c6de87a0,c6b36d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b36d60, ebp = 0 --- Tracing command intr pid 12 tid 100011 td 0xc6da1870 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100010 td 0xc6da1b40 sched_switch(c6da1b40,0,109,191,1423c985,...) at sched_switch+0x3bc mi_switch(109,0,c0ce47dd,52d,c6de01f0,...) at mi_switch+0x200 ithread_loop(c6de87c0,c6b30d28,c0ce4597,33b,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0896e30,c6de87c0,c6b30d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b30d60, ebp = 0 --- Tracing command intr pid 12 tid 100009 td 0xc6de9000 sched_switch(c6de9000,0,109,191,2b34fcbe,...) at sched_switch+0x3bc mi_switch(109,0,c0ce47dd,52d,c6de0270,...) at mi_switch+0x200 ithread_loop(c6de87d0,c6b2dd28,c0ce4597,33b,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0896e30,c6de87d0,c6b2dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b2dd60, ebp = 0 --- Tracing command intr pid 12 tid 100008 td 0xc6de92d0 sched_switch(c6de92d0,0,109,191,2998e50f,...) at sched_switch+0x3bc mi_switch(109,0,c0ce47dd,52d,c6de02f0,...) at mi_switch+0x200 ithread_loop(c6de87e0,c6b2ad28,c0ce4597,33b,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0896e30,c6de87e0,c6b2ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b2ad60, ebp = 0 --- Tracing command intr pid 12 tid 100007 td 0xc6de95a0 cpustop_handler(4,c6b27b74,c0c060b6,c6b27b14,c07bb28c,...) at cpustop_handler+0x34 ipi_nmi_handler(c6b27b14,c07bb28c,c6d82e00,dfd,c6d997f8,...) at ipi_nmi_handler+0x2f trap(c6b27b80) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc04fbd73, esp = 0xc6b27bc0, ebp = 0xc6b27bc0 --- acpi_timer_read(c0d3ec20,c0e4e200,c6b27c08,c08ca61d,c0d3ec20,...) at acpi_timer_read+0x13 acpi_timer_get_timecount_safe(c0d3ec20,c6de95a0,c6b27bec,246,c0e4e200,...) at acpi_timer_get_timecount_safe+0x18 binuptime(c6b27c78,0,c0cea750,1a1,c6de9844,...) at binuptime+0x3d softclock(c0e4e200,c6b27cb8,c08ae9d4,c0e52940,c6de0338,...) at softclock+0x221 intr_event_execute_handlers(c6d997f8,c6de0300,c0ce47dd,533,c6de0370,...) at intr_event_execute_handlers+0x125 ithread_loop(c6de87f0,c6b27d28,c0ce4597,33b,c6d997f8,...) at ithread_loop+0xac fork_exit(c0896e30,c6de87f0,c6b27d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b27d60, ebp = 0 --- Tracing command idle pid 11 tid 100006 td 0xc6d9b000 sched_switch(c6d9b000,0,108,18c,2de63e51,...) at sched_switch+0x3bc mi_switch(108,0,c0ceaf8a,3aa,c6d9b000,...) at mi_switch+0x200 sched_idletd(0,c6b23d28,c0ce4597,33b,c6d99aa0,...) at sched_idletd+0x19f fork_exit(c08e45c0,0,c6b23d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b23d60, ebp = 0 --- Tracing command idle pid 11 tid 100005 td 0xc6d9b2d0 sched_switch(c6d9b2d0,0,108,18c,2de5aeaf,...) at sched_switch+0x3bc mi_switch(108,0,c0ceaf8a,3aa,c6d9b2d0,...) at mi_switch+0x200 sched_idletd(0,c6b20d28,c0ce4597,33b,c6d99aa0,...) at sched_idletd+0x19f fork_exit(c08e45c0,0,c6b20d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b20d60, ebp = 0 --- Tracing command idle pid 11 tid 100004 td 0xc6d9b5a0 sched_switch(c6d9b5a0,0,108,18c,311b1d6c,...) at sched_switch+0x3bc mi_switch(108,0,c0ceaf8a,3aa,c6d9b5a0,...) at mi_switch+0x200 sched_idletd(0,c6b1dd28,c0ce4597,33b,c6d99aa0,...) at sched_idletd+0x19f fork_exit(c08e45c0,0,c6b1dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b1dd60, ebp = 0 --- Tracing command idle pid 11 tid 100003 td 0xc6d9b870 sched_switch(c6d9b870,0,108,18c,2de1baa4,...) at sched_switch+0x3bc mi_switch(108,0,c0ceaf8a,3aa,c6d9b870,...) at mi_switch+0x200 sched_idletd(0,c6b1ad28,c0ce4597,33b,c6d99aa0,...) at sched_idletd+0x19f fork_exit(c08e45c0,0,c6b1ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b1ad60, ebp = 0 --- Tracing command init pid 1 tid 100002 td 0xc6d9bb40 sched_switch(c6d9bb40,0,104,191,fc01c87c,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c6d9bb40,0,c0cedbb9,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0cedbb9,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6d99d48,5c,c0cf0511,100,0,...) at sleepq_wait_sig+0x17 _sleep(c6d99d48,c6d99dd0,15c,c0cf0511,0,...) at _sleep+0x35c kern_wait(c6d9bb40,ffffffff,c6b16c44,0,0,...) at kern_wait+0xb86 wait4(c6d9bb40,c6b16cec,c6b16d28,c0cee3f2,0,...) at wait4+0x3b syscallenter(c6d9bb40,c6b16ce4,c6b16ce4,0,ec,...) at syscallenter+0x263 syscall(c6b16d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x8054cd3, esp = 0xbfbfe8dc, ebp = 0xbfbfe8f8 --- Tracing command audit pid 10 tid 100001 td 0xc6da1000 sched_switch(c6da1000,0,104,191,860ce931,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c6da1000,0,c0cedbb9,268,c6da1000,...) at sleepq_switch+0x15f sleepq_wait(c0fc6300,0,c6b13c8c,1,0,...) at sleepq_wait+0x63 _cv_wait(c0fc6300,c0fc62e4,c0d0edad,194,0,...) at _cv_wait+0x243 audit_worker(0,c6b13d28,c0ce4597,33b,c6d9a000,...) at audit_worker+0x84 fork_exit(c0acee40,0,c6b13d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b13d60, ebp = 0 --- Tracing command kernel pid 0 tid 100065 td 0xc7162b40 sched_switch(c7162b40,0,104,191,87cfbe8f,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7162b40,0,c0cedbb9,28b,2,...) at sleepq_switch+0x15f sleepq_timedwait(c0e4e004,0,c0cde0eb,2,0,...) at sleepq_timedwait+0x6b _sleep(c0e4e004,0,0,c0cde0eb,bb8,...) at _sleep+0x342 pause(c0cde0eb,bb8,120,11e,c0e4be28,...) at pause+0x47 deadlkres(0,e751dd28,c0ce4597,33b,c0e4bda0,...) at deadlkres+0x302 fork_exit(c0878fa0,0,e751dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe751dd60, ebp = 0 --- Tracing command kernel pid 0 tid 100059 td 0xc704c000 sched_switch(c704c000,0,104,191,28879511,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c704c000,0,c0cedbb9,268,0,...) at sleepq_switch+0x15f sleepq_wait(c6eabb80,0,c0ce9f6f,c0cde0eb,0,...) at sleepq_wait+0x63 msleep_spin(c6eabb80,c6eabb9c,c0cde0eb,0,c0ce7b4a,...) at msleep_spin+0x21d taskqueue_thread_loop(c705d49c,e7501d28,c0ce4597,33b,c0e4bda0,...) at taskqueue_thread_loop+0x8f fork_exit(c08ff2a0,c705d49c,e7501d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7501d60, ebp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xc6fa32d0 sched_switch(c6fa32d0,0,104,191,8e042194,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c6fa32d0,0,c0cedbb9,268,0,...) at sleepq_switch+0x15f sleepq_wait(c6eabc80,0,c0ce9f6f,c0cde0eb,0,...) at sleepq_wait+0x63 msleep_spin(c6eabc80,c6eabc9c,c0cde0eb,0,c0ce7b4a,...) at msleep_spin+0x21d taskqueue_thread_loop(c0e1cfa0,c6b5dd28,c0ce4597,33b,c0e4bda0,...) at taskqueue_thread_loop+0x8f fork_exit(c08ff2a0,c0e1cfa0,c6b5dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b5dd60, ebp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xc6fa35a0 sched_switch(c6fa35a0,0,104,191,8e041222,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c6fa35a0,0,c0cedbb9,268,0,...) at sleepq_switch+0x15f sleepq_wait(c6eabc80,0,c0ce9f6f,c0cde0eb,0,...) at sleepq_wait+0x63 msleep_spin(c6eabc80,c6eabc9c,c0cde0eb,0,c0ce7b4a,...) at msleep_spin+0x21d taskqueue_thread_loop(c0e1cfa0,c6b5ad28,c0ce4597,33b,c0e4bda0,...) at taskqueue_thread_loop+0x8f fork_exit(c08ff2a0,c0e1cfa0,c6b5ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b5ad60, ebp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xc6fa3870 sched_switch(c6fa3870,0,104,191,8e040168,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c6fa3870,0,c0cedbb9,268,0,...) at sleepq_switch+0x15f sleepq_wait(c6eabc80,0,c0ce9f6f,c0cde0eb,0,...) at sleepq_wait+0x63 msleep_spin(c6eabc80,c6eabc9c,c0cde0eb,0,c0ce7b4a,...) at msleep_spin+0x21d taskqueue_thread_loop(c0e1cfa0,c6b57d28,c0ce4597,33b,c0e4bda0,...) at taskqueue_thread_loop+0x8f fork_exit(c08ff2a0,c0e1cfa0,c6b57d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b57d60, ebp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xc6de9b40 sched_switch(c6de9b40,0,104,191,9c6e8b77,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c6de9b40,0,c0cedbb9,268,0,...) at sleepq_switch+0x15f sleepq_wait(c6eabd40,0,c0cde0eb,0,0,...) at sleepq_wait+0x63 _sleep(c6eabd40,c6eabd5c,0,c0cde0eb,0,...) at _sleep+0x372 taskqueue_thread_loop(c0e60968,c6b4ed28,c0ce4597,33b,c0e4bda0,...) at taskqueue_thread_loop+0xb5 fork_exit(c08ff2a0,c0e60968,c6b4ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b4ed60, ebp = 0 --- Tracing command kernel pid 0 tid 100018 td 0xc6eae2d0 sched_switch(c6eae2d0,0,104,191,8df9a410,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c6eae2d0,0,c0cedbb9,268,0,...) at sleepq_switch+0x15f sleepq_wait(c6eabdc0,0,c0cde0eb,0,0,...) at sleepq_wait+0x63 _sleep(c6eabdc0,c6eabddc,0,c0cde0eb,0,...) at _sleep+0x372 taskqueue_thread_loop(c0e4c6f8,c6b48d28,c0ce4597,33b,c0e4bda0,...) at taskqueue_thread_loop+0xb5 fork_exit(c08ff2a0,c0e4c6f8,c6b48d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b48d60, ebp = 0 --- Tracing command kernel pid 0 tid 100013 td 0xc6da12d0 sched_switch(c6da12d0,0,104,191,a2ae69a3,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,0,...) at mi_switch+0x200 sleepq_switch(c6da12d0,0,c0cedbb9,268,0,...) at sleepq_switch+0x15f sleepq_wait(c6eabe00,0,c0cde0eb,0,0,...) at sleepq_wait+0x63 _sleep(c6eabe00,c6eabe1c,0,c0cde0eb,0,...) at _sleep+0x372 taskqueue_thread_loop(c0e5f3e0,c6b39d28,c0ce4597,33b,c0e4bda0,...) at taskqueue_thread_loop+0xb5 fork_exit(c08ff2a0,c0e5f3e0,c6b39d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b39d60, ebp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xc0e4c050 sched_switch(c0e4c050,0,104,191,e7d7e799,...) at sched_switch+0x3bc mi_switch(104,0,c0cedbb9,1f3,44,...) at mi_switch+0x200 sleepq_switch(c0e4c050,0,c0cedbb9,28b,0,...) at sleepq_switch+0x15f sleepq_timedwait(c0e4bda0,44,c0ceb89e,0,0,...) at sleepq_timedwait+0x6b _sleep(c0e4bda0,0,44,c0ceb89e,2710,...) at _sleep+0x342 scheduler(0,141ec00,141ec00,141e000,1425000,...) at scheduler+0x23e mi_startup() at mi_startup+0xac begin() at begin+0x2c db:0:allt> call doadump Physical memory: 3439 MB Dumping 231 MB: 216 200 184 168 152 136 120 104 88 72 56 40 24 8 Dump complete = 0xf db:0:doadump> reset cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 1 (kgdb) bt #0 doadump () at pcpu.h:231 #1 0xc04d5bc9 in db_fncall (dummy1=0xc08f208a, dummy2=0x0, dummy3=0xffffffff, dummy4=0xe75385fc "\020\206Sç") at ../../../ddb/db_command.c:548 #2 0xc04d5fff in db_command (last_cmdp=0xc0e1b17c, cmd_table=0x0, dopager=0x0) at ../../../ddb/db_command.c:445 #3 0xc04d60b4 in db_command_script (command=0xc0e1c088 "call doadump") at ../../../ddb/db_command.c:516 #4 0xc04da300 in db_script_exec (scriptname=0xc0e1b9e0 "doadump", warnifnotfound=Variable "warnifnotfound" is not available. ) at ../../../ddb/db_script.c:302 #5 0xc04da391 in db_run_cmd (addr=0x1, have_addr=0x0, count=0xc0fe9260, modif=0xe7538734 "") at ../../../ddb/db_script.c:375 #6 0xc04d5fc1 in db_command (last_cmdp=0xc0e1b17c, cmd_table=0x0, dopager=0x1) at ../../../ddb/db_command.c:445 #7 0xc04d611a in db_command_loop () at ../../../ddb/db_command.c:498 #8 0xc04d803d in db_trap (type=0x3, code=0x0) at ../../../ddb/db_main.c:229 #9 0xc08f1ebe in kdb_trap (type=0x3, code=0x0, tf=0xe75388d8) at ../../../kern/subr_kdb.c:546 #10 0xc0c066eb in trap (frame=0xe75388d8) at ../../../i386/i386/trap.c:732 #11 0xc0bef5bc in calltrap () at ../../../i386/i386/exception.s:168 #12 0xc08f208a in kdb_enter (why=0xc0ce93b9 "panic", msg=0xc0ce93b9 "panic") at cpufunc.h:71 #13 0xc08bec94 in panic (fmt=0xc0d10f3c "worklist_remove: %p %s(0x%X) not on list") at ../../../kern/kern_shutdown.c:574 #14 0xc0af8769 in worklist_remove (item=0xc8459300, locked=Variable "locked" is not available. ) at ../../../ufs/ffs/ffs_softdep.c:963 #15 0xc0aff3b5 in softdep_setup_blkfree (mp=0xc7fbc87c, bp=0xdae6a800, blkno=0x2eca8, frags=0x8, wkhd=0xe7538b00) at ../../../ufs/ffs/ffs_softdep.c:8722 #16 0xc0ae487d in ffs_blkfree (ump=0xc8b66900, fs=0xc9b64000, devvp=0xc91ab414, bno=0x2eca8, size=0x4000, inum=0x12, dephd=0xe7538b00) at ../../../ufs/ffs/ffs_alloc.c:1962 #17 0xc0affa1f in indir_trunc (freework=0xc98dbb00, dbn=0xb7cc0, lbn=0xfffffffffffff7f4) at ../../../ufs/ffs/ffs_softdep.c:6191 #18 0xc0affc46 in indir_trunc (freework=0xc9b4e700, dbn=0xb7ca0, lbn=0xfffffffffffff7f3) at ../../../ufs/ffs/ffs_softdep.c:6232 #19 0xc0b00176 in handle_workitem_indirblk (freework=0xc9b4e700) at ../../../ufs/ffs/ffs_softdep.c:5927 #20 0xc0b00227 in handle_workitem_freeblocks (freeblks=0xc994c700, flags=Variable "flags" is not available. ) at ../../../ufs/ffs/ffs_softdep.c:5992 #21 0xc0b004d7 in process_worklist_item (mp=0xc7fbc87c, flags=Variable "flags" is not available. ) at ../../../ufs/ffs/ffs_softdep.c:1524 #22 0xc0b0700c in softdep_process_worklist (mp=0xc7fbc87c, full=0x0) at ../../../ufs/ffs/ffs_softdep.c:1368 #23 0xc0b075d0 in softdep_flush () at ../../../ufs/ffs/ffs_softdep.c:1246 #24 0xc0893ee8 in fork_exit (callout=0xc0b07330 , arg=0x0, frame=0xe7538d28) at ../../../kern/kern_fork.c:835 #25 0xc0bef634 in fork_trampoline () at ../../../i386/i386/exception.s:275 (kgdb) f 14 #14 0xc0af8769 in worklist_remove (item=0xc8459300, locked=Variable "locked" is not available. ) at ../../../ufs/ffs/ffs_softdep.c:963 963 panic("worklist_remove: %p %s(0x%X) not on list", (kgdb) l 958 { 959 960 if (locked) 961 mtx_assert(&lk, MA_OWNED); 962 if ((item->wk_state & ONWORKLIST) == 0) 963 panic("worklist_remove: %p %s(0x%X) not on list", 964 item, TYPENAME(item->wk_type), item->wk_state); 965 if (item->wk_state & BLOCKEDWK) 966 panic("worklist_remove: blocked remove %p %s(0x%x)", 967 item, TYPENAME(item->wk_type), item->wk_state); (kgdb) p *item $1 = {wk_list = {le_next = 0x0, le_prev = 0xe7538b00}, wk_mp = 0xc7fbc87c, wk_type = 0x13, wk_state = 0x10a, wk_stack_ins = {depth = 0xa, pcs = {0xc0af8987, 0xc0aff9c7, 0xc0affc46, 0xc0b00176, 0xc0b00227, 0xc0b004d7, 0xc0b0700c, 0xc0b075d0, 0xc0893ee8, 0xc0bef634, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}, wk_stack_rem = {depth = 0xa, pcs = {0xc0af8816, 0xc0aff74c, 0xc0affc46, 0xc0b00176, 0xc0b00227, 0xc0b004d7, 0xc0b0700c, 0xc0b075d0, 0xc0893ee8, 0xc0bef634, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}, wk_onworklist = 0x1, wk_state_ins = 0x10a, wk_state_rem = 0x109} (kgdb) l *0xc0af8987 0xc0af8987 is in worklist_insert (../../../ufs/ffs/ffs_softdep.c:951). 946 item, TYPENAME(item->wk_type), item->wk_state); 947 item->wk_onworklist = 1; 948 item->wk_state |= ONWORKLIST; 949 LIST_INSERT_HEAD(head, item, wk_list); 950 stack_save(&item->wk_stack_ins); 951 item->wk_state_ins = item->wk_state; 952 } 953 954 static void 955 worklist_remove(item, locked) (kgdb) l *0xc0aff9c7 0xc0aff9c7 is in indir_trunc (../../../ufs/ffs/ffs_softdep.c:6190). 6185 } else { 6186 struct workhead freewk; 6187 6188 LIST_INIT(&freewk); 6189 WORKLIST_INSERT_UNLOCKED(&freewk, wk); 6190 wk->wk_state |= BLOCKEDWK; 6191 ffs_blkfree(ump, fs, freeblks->fb_devvp, 6192 jnewblk->jn_blkno, fs->fs_bsize, 6193 freeblks->fb_previousinum, &freewk); 6194 } (kgdb) l *0xc0affc46 0xc0affc46 is in indir_trunc (../../../ufs/ffs/ffs_softdep.c:6232). 6227 if (needj != 0) { 6228 nfreework = newfreework(freeblks, freework, 6229 nlbn, nb, fs->fs_frag, 0); 6230 freedeps++; 6231 } 6232 indir_trunc(nfreework, fsbtodb(fs, nb), nlbn); 6233 } else { 6234 struct freedep *freedep; 6235 6236 /* (kgdb) l *0xc0b00176 0xc0b00176 is in handle_workitem_indirblk (../../../ufs/ffs/ffs_softdep.c:5929). 5924 if (freework->fw_off == NINDIR(fs)) 5925 freework_freeblock(freework); 5926 else 5927 indir_trunc(freework, fsbtodb(fs, freework->fw_blkno), 5928 freework->fw_lbn); 5929 } 5930 5931 /* 5932 * Called when a freework structure attached to a cg buf is written. The 5933 * ref on either the parent or the freeblks structure is released and (kgdb) l *0xc0b00227 0xc0b00227 is in handle_workitem_freeblocks (../../../ufs/ffs/ffs_softdep.c:5992). 5987 ("handle_workitem_freeblocks: Unknown type %s", 5988 TYPENAME(wk->wk_type))); 5989 WORKLIST_REMOVE_UNLOCKED(wk); 5990 freework = WK_FREEWORK(wk); 5991 if (freework->fw_lbn <= -NDADDR) 5992 handle_workitem_indirblk(freework); 5993 else 5994 freework_freeblock(freework); 5995 } 5996 ACQUIRE_LOCK(&lk); (kgdb) l *0xc0b004d7 0xc0b004d7 is in process_worklist_item (../../../ufs/ffs/ffs_softdep.c:1524). 1519 vput(vp); 1520 break; 1521 1522 case D_FREEBLKS: 1523 /* releasing blocks and/or fragments from a file */ 1524 handle_workitem_freeblocks(WK_FREEBLKS(wk), flags & LK_NOWAIT); 1525 break; 1526 1527 case D_FREEFRAG: 1528 /* releasing a fragment when replaced as a file grows */ (kgdb) l *0xc0b0700c 0xc0b0700c is in softdep_process_worklist (../../../ufs/ffs/ffs_softdep.c:1368). 1363 ACQUIRE_LOCK(&lk); 1364 loopcount = 1; 1365 starttime = time_second; 1366 softdep_process_journal(mp, full?MNT_WAIT:0); 1367 while (ump->softdep_on_worklist > 0) { 1368 if ((cnt = process_worklist_item(mp, LK_NOWAIT)) == -1) 1369 break; 1370 else 1371 matchcnt += cnt; 1372 /* (kgdb) l *0xc0b075d0 0xc0b075d0 is in softdep_flush (../../../ufs/ffs/ffs_softdep.c:1246). 1241 if ((mp->mnt_flag & MNT_SOFTDEP) == 0) 1242 continue; 1243 if (vfs_busy(mp, MBF_NOWAIT | MBF_MNTLSTLOCK)) 1244 continue; 1245 vfslocked = VFS_LOCK_GIANT(mp); 1246 progress += softdep_process_worklist(mp, 0); 1247 ump = VFSTOUFS(mp); 1248 remaining += ump->softdep_on_worklist - 1249 ump->softdep_on_worklist_inprogress; 1250 VFS_UNLOCK_GIANT(vfslocked); (kgdb) l *0xc0893ee8 0xc0893ee8 is in fork_exit (../../../kern/kern_fork.c:841). 836 837 /* 838 * Check if a kernel thread misbehaved and returned from its main 839 * function. 840 */ 841 if (p->p_flag & P_KTHREAD) { 842 printf("Kernel thread \"%s\" (pid %d) exited prematurely.\n", 843 td->td_name, p->p_pid); 844 kproc_exit(0); 845 } (kgdb) l *0xc0af8816 0xc0af8816 is in worklist_remove (../../../ufs/ffs/ffs_softdep.c:972). 967 item, TYPENAME(item->wk_type), item->wk_state); 968 item->wk_onworklist = 0; 969 item->wk_state &= ~ONWORKLIST; 970 LIST_REMOVE(item, wk_list); 971 stack_save(&item->wk_stack_rem); 972 item->wk_state_rem = item->wk_state; 973 } 974 #endif /* DEBUG */ 975 976 /* (kgdb) l *0xc0aff74c 0xc0aff74c is in indir_trunc (../../../ufs/ffs/ffs_softdep.c:6157). 6152 cnt = 0; 6153 LIST_FOREACH_SAFE(wk, &wkhd, wk_list, wkn) { 6154 if (wk->wk_type != D_JNEWBLK) 6155 continue; 6156 WORKLIST_REMOVE_UNLOCKED(wk); 6157 jnewblk = WK_JNEWBLK(wk); 6158 if (jnewblk->jn_lbn > 0) 6159 i = (jnewblk->jn_lbn - -lbn) / lbnadd; 6160 else 6161 i = (jnewblk->jn_lbn - (lbn + level)) / lbnadd; (kgdb) l *0xc0affc46 0xc0affc46 is in indir_trunc (../../../ufs/ffs/ffs_softdep.c:6232). 6227 if (needj != 0) { 6228 nfreework = newfreework(freeblks, freework, 6229 nlbn, nb, fs->fs_frag, 0); 6230 freedeps++; 6231 } 6232 indir_trunc(nfreework, fsbtodb(fs, nb), nlbn); 6233 } else { 6234 struct freedep *freedep; 6235 6236 /* (kgdb) l *0xc0b00176 0xc0b00176 is in handle_workitem_indirblk (../../../ufs/ffs/ffs_softdep.c:5929). 5924 if (freework->fw_off == NINDIR(fs)) 5925 freework_freeblock(freework); 5926 else 5927 indir_trunc(freework, fsbtodb(fs, freework->fw_blkno), 5928 freework->fw_lbn); 5929 } 5930 5931 /* 5932 * Called when a freework structure attached to a cg buf is written. The 5933 * ref on either the parent or the freeblks structure is released and (kgdb) l *0xc0b00227 0xc0b00227 is in handle_workitem_freeblocks (../../../ufs/ffs/ffs_softdep.c:5992). 5987 ("handle_workitem_freeblocks: Unknown type %s", 5988 TYPENAME(wk->wk_type))); 5989 WORKLIST_REMOVE_UNLOCKED(wk); 5990 freework = WK_FREEWORK(wk); 5991 if (freework->fw_lbn <= -NDADDR) 5992 handle_workitem_indirblk(freework); 5993 else 5994 freework_freeblock(freework); 5995 } 5996 ACQUIRE_LOCK(&lk); (kgdb) l *0xc0b004d7 0xc0b004d7 is in process_worklist_item (../../../ufs/ffs/ffs_softdep.c:1524). 1519 vput(vp); 1520 break; 1521 1522 case D_FREEBLKS: 1523 /* releasing blocks and/or fragments from a file */ 1524 handle_workitem_freeblocks(WK_FREEBLKS(wk), flags & LK_NOWAIT); 1525 break; 1526 1527 case D_FREEFRAG: 1528 /* releasing a fragment when replaced as a file grows */ (kgdb) l *0xc0b0700c 0xc0b0700c is in softdep_process_worklist (../../../ufs/ffs/ffs_softdep.c:1368). 1363 ACQUIRE_LOCK(&lk); 1364 loopcount = 1; 1365 starttime = time_second; 1366 softdep_process_journal(mp, full?MNT_WAIT:0); 1367 while (ump->softdep_on_worklist > 0) { 1368 if ((cnt = process_worklist_item(mp, LK_NOWAIT)) == -1) 1369 break; 1370 else 1371 matchcnt += cnt; 1372 /* (kgdb) l *0xc0b075d0 0xc0b075d0 is in softdep_flush (../../../ufs/ffs/ffs_softdep.c:1246). 1241 if ((mp->mnt_flag & MNT_SOFTDEP) == 0) 1242 continue; 1243 if (vfs_busy(mp, MBF_NOWAIT | MBF_MNTLSTLOCK)) 1244 continue; 1245 vfslocked = VFS_LOCK_GIANT(mp); 1246 progress += softdep_process_worklist(mp, 0); 1247 ump = VFSTOUFS(mp); 1248 remaining += ump->softdep_on_worklist - 1249 ump->softdep_on_worklist_inprogress; 1250 VFS_UNLOCK_GIANT(vfslocked); (kgdb) l *0xc0893ee8 0xc0893ee8 is in fork_exit (../../../kern/kern_fork.c:841). 836 837 /* 838 * Check if a kernel thread misbehaved and returned from its main 839 * function. 840 */ 841 if (p->p_flag & P_KTHREAD) { 842 printf("Kernel thread \"%s\" (pid %d) exited prematurely.\n", 843 td->td_name, p->p_pid); 844 kproc_exit(0); 845 } (kgdb) git diff master diff --git a/sys/sys/queue.h b/sys/sys/queue.h index 257679b..4c7332c 100644 --- a/sys/sys/queue.h +++ b/sys/sys/queue.h @@ -429,6 +429,8 @@ struct { \ *(elm)->field.le_prev = LIST_NEXT((elm), field); \ TRASHIT(*oldnext); \ TRASHIT(*oldprev); \ + (elm)->field.le_next = (void *)0xd00fcafe; \ + (elm)->field.le_prev = (void *)0xd0ffcafe; \ } while (0) #define LIST_SWAP(head1, head2, type, field) do { \ diff --git a/sys/ufs/ffs/ffs_inode.c b/sys/ufs/ffs/ffs_inode.c index ffbe16e..ba2813d 100644 --- a/sys/ufs/ffs/ffs_inode.c +++ b/sys/ufs/ffs/ffs_inode.c @@ -582,9 +582,7 @@ ffs_indirtrunc(ip, lbn, dbn, lastbn, level, countp) * block to be kept. -1 indicates the entire * block so we need not calculate the index. */ - factor = 1; - for (i = SINGLE; i < level; i++) - factor *= NINDIR(fs); + factor = lbn_offset(fs, level); last = lastbn; if (lastbn > 0) last /= factor; diff --git a/sys/ufs/ffs/ffs_softdep.c b/sys/ufs/ffs/ffs_softdep.c index ca4ea0b..b7f5d1a 100644 --- a/sys/ufs/ffs/ffs_softdep.c +++ b/sys/ufs/ffs/ffs_softdep.c @@ -944,8 +944,11 @@ worklist_insert(head, item, locked) if (item->wk_state & ONWORKLIST) panic("worklist_insert: %p %s(0x%X) already on list", item, TYPENAME(item->wk_type), item->wk_state); + item->wk_onworklist = 1; item->wk_state |= ONWORKLIST; LIST_INSERT_HEAD(head, item, wk_list); + stack_save(&item->wk_stack_ins); + item->wk_state_ins = item->wk_state; } static void @@ -959,8 +962,14 @@ worklist_remove(item, locked) if ((item->wk_state & ONWORKLIST) == 0) panic("worklist_remove: %p %s(0x%X) not on list", item, TYPENAME(item->wk_type), item->wk_state); + if (item->wk_state & BLOCKEDWK) + panic("worklist_remove: blocked remove %p %s(0x%x)", + item, TYPENAME(item->wk_type), item->wk_state); + item->wk_onworklist = 0; item->wk_state &= ~ONWORKLIST; LIST_REMOVE(item, wk_list); + stack_save(&item->wk_stack_rem); + item->wk_state_rem = item->wk_state; } #endif /* DEBUG */ @@ -2270,7 +2279,6 @@ journal_mount(mp, fs, cred) int error; int i; - mp->mnt_kern_flag |= MNTK_SUJ; error = softdep_journal_lookup(mp, &vp); if (error != 0) { printf("Failed to find journal. Use tunefs to create one\n"); @@ -2295,20 +2303,26 @@ journal_mount(mp, fs, cred) } jblocks->jb_low = jblocks->jb_free / 3; /* Reserve 33%. */ jblocks->jb_min = jblocks->jb_free / 10; /* Suspend at 10%. */ - /* - * Only validate the journal contents if the filesystem is clean, - * otherwise we write the logs but they'll never be used. If the - * filesystem was still dirty when we mounted it the journal is - * invalid and a new journal can only be valid if it starts from a - * clean mount. - */ - if (fs->fs_clean) { - DIP_SET(ip, i_modrev, fs->fs_mtime); - ip->i_flags |= IN_MODIFIED; - ffs_update(vp, 1); - } VFSTOUFS(mp)->softdep_jblocks = jblocks; out: + if (error == 0) { + MNT_ILOCK(mp); + mp->mnt_kern_flag |= MNTK_SUJ; + MNT_IUNLOCK(mp); + /* + * Only validate the journal contents if the + * filesystem is clean, otherwise we write the logs + * but they'll never be used. If the filesystem was + * still dirty when we mounted it the journal is + * invalid and a new journal can only be valid if it + * starts from a clean mount. + */ + if (fs->fs_clean) { + DIP_SET(ip, i_modrev, fs->fs_mtime); + ip->i_flags |= IN_MODIFIED; + ffs_update(vp, 1); + } + } vput(vp); return (error); } @@ -5265,7 +5279,7 @@ softdep_setup_freeblocks(ip, length, flags) if (delay) WORKLIST_INSERT(&bp->b_dep, &freeblks->fb_list); else if (needj) - freeblks->fb_state |= DEPCOMPLETE | COMPLETE; + freeblks->fb_state |= COMPLETE; /* * Because the file length has been truncated to zero, any * pending block allocation dependency structures associated @@ -5327,8 +5341,9 @@ restart: if (inodedep_lookup(mp, ip->i_number, 0, &inodedep) != 0) (void) free_inodedep(inodedep); - if (delay) { + if (delay || needj) freeblks->fb_state |= DEPCOMPLETE; + if (delay) { /* * If the inode with zeroed block pointers is now on disk * we can start freeing blocks. Add freeblks to the worklist @@ -5339,6 +5354,8 @@ restart: if ((freeblks->fb_state & ALLCOMPLETE) == ALLCOMPLETE) add_to_worklist(&freeblks->fb_list, 1); } + if (needj && LIST_EMPTY(&freeblks->fb_jfreeblkhd)) + needj = 0; FREE_LOCK(&lk); /* @@ -5488,7 +5505,7 @@ cancel_allocdirect(adphead, adp, freeblks, delay) newblk = (struct newblk *)adp; /* * If the journal hasn't been written the jnewblk must be passed - * to the call to ffs_freeblk that reclaims the space. We accomplish + * to the call to ffs_blkfree that reclaims the space. We accomplish * this by linking the journal dependency into the freework to be * freed when freework_freeblock() is called. If the journal has * been written we can simply reclaim the journal space when the @@ -6016,11 +6033,11 @@ handle_complete_freeblocks(freeblks) vput(vp); } -#ifdef INVARIANTS - if (freeblks->fb_chkcnt != 0 && - ((fs->fs_flags & FS_UNCLEAN) == 0 || (flags & LK_NOWAIT) != 0)) - printf("handle_workitem_freeblocks: block count\n"); -#endif /* INVARIANTS */ + KASSERT(freeblks->fb_chkcnt == 0 || + ((fs->fs_flags & FS_UNCLEAN) != 0 && (flags & LK_NOWAIT) == 0), + ("handle_workitem_freeblocks: inode %ju block count %jd\n", + (uintmax_t)freeblks->fb_previousinum, + (intmax_t)freeblks->fb_chkcnt)); ACQUIRE_LOCK(&lk); /* @@ -6075,9 +6092,7 @@ indir_trunc(freework, dbn, lbn) fs_pendingblocks = 0; freedeps = 0; needj = UFSTOVFS(ump)->mnt_kern_flag & MNTK_SUJ; - lbnadd = 1; - for (i = level; i > 0; i--) - lbnadd *= NINDIR(fs); + lbnadd = lbn_offset(fs, level); /* * Get buffer of block pointers to be freed. This routine is not * called until the zero'ed inode has been written, so it is safe @@ -6136,20 +6151,17 @@ indir_trunc(freework, dbn, lbn) */ cnt = 0; LIST_FOREACH_SAFE(wk, &wkhd, wk_list, wkn) { - struct workhead freewk; if (wk->wk_type != D_JNEWBLK) continue; WORKLIST_REMOVE_UNLOCKED(wk); - LIST_INIT(&freewk); - WORKLIST_INSERT_UNLOCKED(&freewk, wk); jnewblk = WK_JNEWBLK(wk); if (jnewblk->jn_lbn > 0) i = (jnewblk->jn_lbn - -lbn) / lbnadd; else - i = (jnewblk->jn_lbn - (lbn + 1)) / lbnadd; + i = (jnewblk->jn_lbn - (lbn + level)) / lbnadd; KASSERT(i >= 0 && i < NINDIR(fs), - ("indir_trunc: Index out of range %d parent %jd lbn %jd", - i, lbn, jnewblk->jn_lbn)); + ("indir_trunc: Index out of range %d parent %jd lbn %jd level %d", + i, lbn, jnewblk->jn_lbn, level)); /* Clear the pointer so it isn't found below. */ if (ufs1fmt) { nb = bap1[i]; @@ -6161,8 +6173,25 @@ indir_trunc(freework, dbn, lbn) KASSERT(nb == jnewblk->jn_blkno, ("indir_trunc: Block mismatch %jd != %jd", nb, jnewblk->jn_blkno)); - ffs_blkfree(ump, fs, freeblks->fb_devvp, jnewblk->jn_blkno, - fs->fs_bsize, freeblks->fb_previousinum, &freewk); + if (level != 0) { + ufs_lbn_t nlbn; + + nlbn = (lbn + 1) - (i * lbnadd); + nfreework = newfreework(freeblks, freework, + nlbn, nb, fs->fs_frag, 0); + WORKLIST_INSERT_UNLOCKED(&nfreework->fw_jwork, wk); + freedeps++; + indir_trunc(nfreework, fsbtodb(fs, nb), nlbn); + } else { + struct workhead freewk; + + LIST_INIT(&freewk); + WORKLIST_INSERT_UNLOCKED(&freewk, wk); + wk->wk_state |= BLOCKEDWK; + ffs_blkfree(ump, fs, freeblks->fb_devvp, + jnewblk->jn_blkno, fs->fs_bsize, + freeblks->fb_previousinum, &freewk); + } cnt++; } ACQUIRE_LOCK(&lk); @@ -6772,7 +6801,8 @@ cancel_diradd(dap, dirrem, jremref, dotremref, dotdotremref) mkdir->md_jaddref = NULL; if (mkdir->md_state & MKDIR_PARENT) { if (cancel_jaddref(jaddref, NULL, - &dirrem->dm_jwork) == 0) { + &dirrem->dm_jwork) == 0 && + dotdotremref != NULL) { free_jremref(dotdotremref); dotdotremref = NULL; } @@ -8688,6 +8718,7 @@ softdep_setup_blkfree(mp, bp, blkno, frags, wkhd) jnewblk = WK_JNEWBLK(wk); KASSERT(jnewblk->jn_state & GOINGAWAY, ("softdep_setup_blkfree: jnewblk not canceled.")); + wk->wk_state &= ~BLOCKEDWK; WORKLIST_REMOVE(wk); #ifdef SUJ_DEBUG /* diff --git a/sys/ufs/ffs/fs.h b/sys/ufs/ffs/fs.h index 42c4cfe..ba98ed3 100644 --- a/sys/ufs/ffs/fs.h +++ b/sys/ufs/ffs/fs.h @@ -607,6 +607,11 @@ struct cg { : (fragroundup(fs, blkoff(fs, (size))))) /* + * Number of indirects in a filesystem block. + */ +#define NINDIR(fs) ((fs)->fs_nindir) + +/* * Indirect lbns are aligned on NDADDR addresses where single indirects * are the negated address of the lowest lbn reachable, double indirects * are this lbn - 1 and triple indirects are this lbn - 2. This yields @@ -631,6 +636,17 @@ lbn_level(ufs_lbn_t lbn) } return (-1); } + +static inline ufs_lbn_t +lbn_offset(struct fs *fs, int level) +{ + ufs_lbn_t res; + + for (res = 1; level > 0; level--) + res *= NINDIR(fs); + return (res); +} + /* * Number of inodes in a secondary storage block/fragment. */ @@ -638,11 +654,6 @@ lbn_level(ufs_lbn_t lbn) #define INOPF(fs) ((fs)->fs_inopb >> (fs)->fs_fragshift) /* - * Number of indirects in a filesystem block. - */ -#define NINDIR(fs) ((fs)->fs_nindir) - -/* * Softdep journal record format. */ diff --git a/sys/ufs/ffs/softdep.h b/sys/ufs/ffs/softdep.h index f584e91..74cf6cd 100644 --- a/sys/ufs/ffs/softdep.h +++ b/sys/ufs/ffs/softdep.h @@ -139,6 +139,7 @@ #define UNLINKNEXT 0x080000 /* inodedep has valid di_freelink */ #define UNLINKPREV 0x100000 /* inodedep is pointed at in the unlink list */ #define UNLINKONLIST 0x200000 /* inodedep is in the unlinked list on disk */ +#define BLOCKEDWK 0x400000 #define UNLINKLINKS (UNLINKNEXT | UNLINKPREV) #define ALLCOMPLETE (ATTACHED | COMPLETE | DEPCOMPLETE) @@ -164,11 +165,16 @@ * worklist is needed in the structure, then a wk_data field must be added * and the macros below changed to use it. */ +#include struct worklist { LIST_ENTRY(worklist) wk_list; /* list of work requests */ struct mount *wk_mp; /* Mount we live in */ - unsigned int wk_type:8, /* type of request */ - wk_state:24; /* state flags */ + unsigned int wk_type, /* type of request */ + wk_state; /* state flags */ + struct stack wk_stack_ins; + struct stack wk_stack_rem; + int wk_onworklist; + unsigned int wk_state_ins, wk_state_rem; }; #define WK_DATA(wk) ((void *)(wk)) #define WK_PAGEDEP(wk) ((struct pagedep *)(wk)) -- Test scenario: suj6.sh