GENERIC from Wed Oct 20 01:29:21 2010 +0300, softdep_insmntque a4ade72, vmcore.16 KDB: debugger backends: ddb KDB: current backend: ddb 524288K of memory above 4GB ignored Copyright (c) 1992-2010 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-CURRENT #0 r214053+a4ade72: Wed Oct 20 07:36:18 CEST 2010 pho@x4.osted.lan:/var/tmp/deviant2/sys/i386/compile/PHO i386 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. CPU: AMD Phenom(tm) 9150e Quad-Core Processor (1800.02-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x100f23 Family = 10 Model = 2 Stepping = 3 Features=0x178bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x7ff TSC: P-state invariant real memory = 4294967296 (4096 MB) avail memory = 3536277504 (3372 MB) : Trying to mount root from ufs:/dev/ad4s1a [rw]... WARNING: / was not properly dismounted Setting hostuuid: 00000000-0000-0000-0000-00218515337d. Setting hostid: 0x6b64ac17. Starting ddb. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: /dev/ad4s1a: 3331 files, 195498 used, 817517 free (845 frags, 102084 blocks, 0.1% fragmentation) /dev/ad4s1f: 3920 files, 77007 used, 936008 free (368 frags, 116955 blocks, 0.0% fragmentation) /dev/ad4s1g: 23460 files, 729206 used, 117584775 free (431 frags, 14698043 blocks, 0.0% fragmentation) /dev/ad4s1d: 671105 files, 3536880 used, 6617319 free (38351 frags, 822371 blocks, 0.4% fragmentation) /dev/ad4s1e: 264282 files, 4279734 used, 21108783 free (8895 frags, 2637486 blocks, 0.0% fragmentation) Mounting local file systems:. Setting hostname: x4.osted.lan. re0: link state changed to DOWN Starting Network: lo0 re0 fwe0 fwip0. lo0: flags=8049 metric 0 mtu 16384 options=3 inet 127.0.0.1 netmask 0xff000000 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 nd6 options=21 re0: flags=8843 metric 0 mtu 1500 options=389b ether 00:21:85:15:33:7d inet 192.168.1.7 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::221:85ff:fe15:337d%re0 prefixlen 64 tentative scopeid 0x1 nd6 options=29 media: Ethernet autoselect (none) status: no carrier fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb ch 1 dma -1 fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 Starting devd. Starting Network: fwe0. fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb ch 1 dma -1 Starting Network: fwip0. fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 add net default: gateway 192.168.1.1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/lib /usr/local/lib/compat/pkg /usr/local/lib/wine a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout Creating and/or trimming log files. Starting syslogd. savecore: reboot after panic: worklist_insert: 0xc888a100 jnewblk(0x8109) already on list Oct 20 10:52:48 x4 savecore: reboot after panic: worklist_insert: 0xc888a100 jnewblk(0x8109) already on list savecore: writing core to vmcore.15 Writing crash summary to /var/crash/core.txt.15. Additional ABI support: linux. Starting rpcbind. NFS access cache time=60 Clearing /tmp (X related). Starting mountd. Starting nfsd. Updating motd:. Starting ntpd. Configuring syscons: keymap blanktime. Starting sshd. Starting cron. Local package initialization: backuplock order reversal: 1st 0xc7fa5310 ufs (ufs) @ kern/vfs_subr.c:2111 2nd 0xdaf21080 bufwait (bufwait) @ ufs/ffs/ffs_softdep.c:11332 3rd 0xc850b310 ufs (ufs) @ kern/vfs_subr.c:2111 KDB: stack backtrace: db_trace_self_wrapper(c0ce808f,2e706564,31313a63,d323333,c0ce000a,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c08f3f9b,c0ceb2ad,c6d40168,c6d437d8,e977a890,...) at kdb_backtrace+0x2a _witness_debugger(c0ceb2ad,c850b310,c0cdcc1b,c6d437d8,c0cf25f3,...) at _witness_debugger+0x25 witness_checkorder(c850b310,9,c0cf25ea,83f,0,...) at witness_checkorder+0x839 __lockmgr_args(c850b310,80100,c850b37c,0,0,...) at __lockmgr_args+0x814 ffs_lock(e977a9b4,c0904acb,c0cf1969,80100,c850b2b8,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c0df9060,e977a9b4,109,c0e13ee0,c850b2b8,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c850b2b8,80100,c0cf25ea,83f,4,...) at _vn_lock+0x78 vget(c850b2b8,80100,c7ccf000,50,0,...) at vget+0xbb vfs_hash_get(c7fc12d4,7f9400,80000,c7ccf000,e977ab08,...) at vfs_hash_get+0xed ffs_vgetf(c7fc12d4,7f9400,80000,e977ab08,1,...) at ffs_vgetf+0x49 softdep_sync_metadata(c7fa52b8,0,c0d0f244,144,0,...) at softdep_sync_metadata+0x683 ffs_syncvnode(c7fa52b8,1,c7ccf000,555,c0cf25ea,...) at ffs_syncvnode+0x3e2 ffs_sync(c7fc12d4,1,c0cf1caa,4fd,80,...) at ffs_sync+0x26f dounmount(c7fc12d4,8000000,c7ccf000,482,5b85c5ea,...) at dounmount+0x447 unmount(c7ccf000,e977acec,281760f5,1,0,...) at unmount+0x310 syscallenter(c7ccf000,e977ace4,c0c04f1d,c0e48eb0,0,...) at syscallenter+0x263 syscall(e977ad28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, unmount), eip = 0x280dbadf, esp = 0xbfbfe66c, ebp = 0xbfbfe738 --- lock order reversal: 1st 0xc7fa5b38 ufs (ufs) @ kern/vfs_mount.c:1208 2nd 0xc7c75c94 devfs (devfs) @ ufs/ffs/ffs_softdep.c:1598 KDB: stack backtrace: db_trace_self_wrapper(c0ce808f,35313a63,a0d3839,c0c65300,e977a9a8,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c08f3f9b,c0ceb294,c6d437d8,c6d436a0,e977aa04,...) at kdb_backtrace+0x2a _witness_debugger(c0ceb294,c7c75c94,c0cd882c,c6d436a0,c0d0c3a5,...) at _witness_debugger+0x25 witness_checkorder(c7c75c94,9,c0d0c39c,63e,c7c75d00,...) at witness_checkorder+0x839 __lockmgr_args(c7c75c94,80400,c7c75d00,0,0,...) at __lockmgr_args+0x814 vop_stdlock(e977ab24,c0ce325f,df,80400,c7c75c3c,...) at vop_stdlock+0x65 VOP_LOCK1_APV(c0dd1600,e977ab24,0,c0e13ee0,c7c75c3c,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c7c75c3c,80400,c0d0c39c,63e,c7fc12d4,...) at _vn_lock+0x78 softdep_flushworklist(c7fc12d4,e977abd0,c7ccf000,563,c0cf25ea,...) at softdep_flushworklist+0x47 ffs_sync(c7fc12d4,1,c0cf1caa,4fd,80,...) at ffs_sync+0x2fd dounmount(c7fc12d4,8000000,c7ccf000,482,5b85c5ea,...) at dounmount+0x447 unmount(c7ccf000,e977acec,281760f5,1,0,...) at unmount+0x310 syscallenter(c7ccf000,e977ace4,c0c04f1d,c0e48eb0,0,...) at syscallenter+0x263 syscall(e977ad28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, unmount), eip = 0x280dbadf, esp = 0xbfbfe66c, ebp = 0xbfbfe738 --- ** /dev/ad4s1g ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 23460 files, 729206 used, 117584775 free (431 frags, 14698043 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** usage: kill [-s signal_name] pid ... kill -l [exit_status] kill -signal_name pid ... kill -signal_number pid ... fsck -y /tmp. Starting default moused. Starting inetd. Wed Oct 20 10:53:33 CEST 2010 FreeBSD/i386 (x4.osted.lan) (console) login: Oct 20 10:57:16 x4 su: pho to root on /dev/pts/0 lock order reversal: 1st 0xdaf16e00 bufwait (bufwait) @ kern/vfs_bio.c:2659 2nd 0xca2bb600 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:283 KDB: stack backtrace: db_trace_self_wrapper(c0ce808f,2e687361,38323a63,a0d33,a7b,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c08f3f9b,c0ceb294,c6d40168,c6d43840,e98ce7e8,...) at kdb_backtrace+0x2a _witness_debugger(c0ceb294,ca2bb600,c0d0fa24,c6d43840,c0d0f6b6,...) at _witness_debugger+0x25 witness_checkorder(ca2bb600,9,c0d0f6ad,11b,0,...) at witness_checkorder+0x839 _sx_xlock(ca2bb600,0,c0d0f6ad,11b,ca26f3a0,...) at _sx_xlock+0x85 ufsdirhash_acquire(daf16da0,e0594018,1e8,e0594028,e98ce8b8,...) at ufsdirhash_acquire+0x48 ufsdirhash_add(ca26f3a0,e98ce914,28,e98ce8a4,e98ce8a8,...) at ufsdirhash_add+0x13 ufs_direnter(ca29415c,ca2a415c,e98ce914,e98ceba4,0,...) at ufs_direnter+0x749 ufs_makeinode(e98ceba4,c0df95a0,e98ceb00,e98cea5c,c0c215a5,...) at ufs_makeinode+0x5c5 ufs_create(e98ceb00,c0d25fa7,0,0,e98ceb78,...) at ufs_create+0x30 VOP_CREATE_APV(c0df9060,e98ceb00,e98ceba4,e98cea98,0,...) at VOP_CREATE_APV+0xc5 vn_open_cred(e98ceb78,e98cec2c,1a0,0,ca268c00,...) at vn_open_cred+0x205 vn_open(e98ceb78,e98cec2c,1a0,c8548ab8,0,...) at vn_open+0x3b kern_openat(c88fa5a0,ffffff9c,bfbfe838,0,602,...) at kern_openat+0x128 kern_open(c88fa5a0,bfbfe838,0,601,1b0,...) at kern_open+0x35 open(c88fa5a0,e98cecec,bfbfe838,1,0,...) at open+0x30 syscallenter(c88fa5a0,e98cece4,c0c04f1d,c0e48eb0,0,...) at syscallenter+0x263 syscall(e98ced28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (5, FreeBSD ELF32, open), eip = 0x28172093, esp = 0xbfbfd1fc, ebp = 0xbfbfd218 --- panic: worklist_remove: 0xca308a00 jnewblk(0x109) not on list cpuid = 1 KDB: enter: panic [ thread pid 20 tid 100074 ] Stopped at kdb_enter+0x3a: movl $0,kdb_why db> run pho db:0:pho> bt Tracing pid 20 tid 100074 td 0xc7161000 kdb_enter(c0ce4ad6,c0ce4ad6,c0d0c4cd,e7537a74,1,...) at kdb_enter+0x3a panic(c0d0c4cd,ca308a00,c0d0e420,109,ca308a00,...) at panic+0x134 worklist_remove(c0fc2018,0,c0d0c39c,21fc,0,...) at worklist_remove+0x70 softdep_setup_blkfree(c7c155a8,dae84ce0,12640,0,8,...) at softdep_setup_blkfree+0x5e ffs_blkfree(ca65ea00,c7fcb000,ca6bc6cc,12640,0,...) at ffs_blkfree+0x6fd indir_trunc(fffffff4,ffffffff,c83cc400,cab2a800,ca65ea00,...) at indir_trunc+0x50f handle_workitem_indirblk(4,c0ce325f,df,cab2a800,cab2a800,...) at handle_workitem_indirblk+0x67 handle_workitem_freeblocks(0,e7537c74,2,5dd,1e0,...) at handle_workitem_freeblocks+0x95 process_worklist_item(c0fc2018,0,c0d0c39c,54b,c7161000,...) at process_worklist_item+0x21c softdep_process_worklist(c7c155a8,0,c0d0c39c,4db,3e8,...) at softdep_process_worklist+0x8c softdep_flush(0,e7537d28,c0cdfcac,349,c7bf42a8,...) at softdep_flush+0x2a0 fork_exit(c0b03680,0,e7537d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7537d60, ebp = 0 --- db:0:bt> show allpcpu Current CPU: 1 cpuid = 0 dynamic pcpu = 0x610a00 curthread = 0xc6de9000: pid 12 "swi4: clock" curpcb = 0xc6b2dd80 fpcurthread = none idlethread = 0xc6d9b000: tid 100006 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: cpuid = 1 dynamic pcpu = 0x5cf5a00 curthread = 0xc7161000: pid 20 "softdepflush" curpcb = 0xe7537d80 fpcurthread = none idlethread = 0xc6d9b2d0: tid 100005 "idle: cpu1" APIC ID = 1 currentldt = 0x50 spin locks held: cpuid = 2 dynamic pcpu = 0x5cf8a00 curthread = 0xc6d9b5a0: pid 11 "idle: cpu2" curpcb = 0xc6b1dd80 fpcurthread = none idlethread = 0xc6d9b5a0: tid 100004 "idle: cpu2" APIC ID = 2 currentldt = 0x50 spin locks held: cpuid = 3 dynamic pcpu = 0x5cfba00 curthread = 0xc71615a0: pid 18 "syncer" curpcb = 0xe7531d80 fpcurthread = none idlethread = 0xc6d9b870: tid 100003 "idle: cpu3" APIC ID = 3 currentldt = 0x50 spin locks held: db:0:allpcpu> show alllocks Process 3623 (suj5) thread 0xca7422d0 (100180) exclusive lockmgr ufs (ufs) r = 0 (0xca6bc880) locked @ kern/vfs_vnops.c:612 Process 3622 (suj5) thread 0xcab22870 (100212) exclusive lockmgr bufwait (bufwait) r = 0 (0xdad01780) locked @ kern/vfs_bio.c:2659 exclusive lockmgr ufs (ufs) r = 0 (0xca8f1724) locked @ kern/vfs_vnops.c:612 Process 3621 (suj5) thread 0xca7a0870 (100195) exclusive lockmgr ufs (ufs) r = 0 (0xca2a31b4) locked @ kern/vfs_vnops.c:612 Process 3619 (suj5) thread 0xc7c535a0 (100118) exclusive lockmgr ufs (ufs) r = 0 (0xca6bc9dc) locked @ kern/vfs_vnops.c:612 Process 3618 (suj5) thread 0xc88fab40 (100168) exclusive lockmgr ufs (ufs) r = 0 (0xca15646c) locked @ kern/vfs_vnops.c:612 Process 3617 (suj5) thread 0xc7c56870 (100089) exclusive lockmgr ufs (ufs) r = 0 (0xca6bec94) locked @ kern/vfs_vnops.c:612 Process 3615 (suj5) thread 0xc8006b40 (100113) exclusive lockmgr ufs (ufs) r = 0 (0xc9bab880) locked @ kern/vfs_vnops.c:612 Process 1809 (sshd) thread 0xc853a000 (100155) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xc89366fc) locked @ kern/uipc_sockbuf.c:148 Process 1563 (sshd) thread 0xc800d5a0 (100105) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xc851cd6c) locked @ kern/uipc_sockbuf.c:148 Process 20 (softdepflush) thread 0xc7161000 (100074) exclusive sleep mutex Softdep Lock (Softdep Lock) r = 0 (0xc0fc2018) locked @ ufs/ffs/ffs_softdep.c:8700 exclusive lockmgr bufwait (bufwait) r = 0 (0xdae84d40) locked @ kern/vfs_bio.c:2659 Process 18 (syncer) thread 0xc71615a0 (100072) exclusive sleep mutex struct mount mtx (struct mount mtx) r = 0 (0xc7c15b50) locked @ kern/vfs_mount.c:1664 exclusive lockmgr syncer (syncer) r = 0 (0xc7fa5724) locked @ kern/vfs_subr.c:1721 Process 3 (g_up) thread 0xc6eb8000 (100014) exclusive sleep mutex needsbuffer lock (needsbuffer lock) r = 0 (0xc0fb6380) locked @ kern/vfs_bio.c:330 db:0:alllocks> show lockedvnods Locked vnodes 0xc7fa56cc: tag syncer, type VNON usecount 1, writecount 0, refcount 1 mountedhere 0 flags () lock type syncer: EXCL by thread 0xc71615a0 (pid 18) #0 0xc08a548e at __lockmgr_args+0xbfe #1 0xc09424f5 at vop_stdlock+0x65 #2 0xc0c205d5 at VOP_LOCK1_APV+0xb5 #3 0xc0961778 at _vn_lock+0x78 #4 0xc0954c52 at sync_vnode+0x142 #5 0xc0954fd2 at sched_sync+0x282 #6 0xc0892338 at fork_exit+0xb8 #7 0xc0bebcd4 at fork_trampoline+0x8 0xc9bab828: tag ufs, type VREG usecount 1, writecount 1, refcount 309 mountedhere 0 flags () v_object 0xcab05440 ref 0 pages 1228 lock type ufs: EXCL by thread 0xc8006b40 (pid 3615) #0 0xc08a548e at __lockmgr_args+0xbfe #1 0xc0b0f5c1 at ffs_lock+0xa1 #2 0xc0c205d5 at VOP_LOCK1_APV+0xb5 #3 0xc0961778 at _vn_lock+0x78 #4 0xc0962ae6 at vn_write+0x156 #5 0xc0907455 at dofilewrite+0x95 #6 0xc0908bf8 at kern_writev+0x58 #7 0xc0908d0f at write+0x4f #8 0xc08fd373 at syscallenter+0x263 #9 0xc0c046bf at syscall+0x4f #10 0xc0bebcc1 at Xint0x80_syscall+0x21 ino 8, on dev md5a 0xca6bec3c: tag ufs, type VREG usecount 1, writecount 1, refcount 340 mountedhere 0 flags () v_object 0xca0c4880 ref 0 pages 1352 lock type ufs: EXCL by thread 0xc7c56870 (pid 3617) #0 0xc08a548e at __lockmgr_args+0xbfe #1 0xc0b0f5c1 at ffs_lock+0xa1 #2 0xc0c205d5 at VOP_LOCK1_APV+0xb5 #3 0xc0961778 at _vn_lock+0x78 #4 0xc0962ae6 at vn_write+0x156 #5 0xc0907455 at dofilewrite+0x95 #6 0xc0908bf8 at kern_writev+0x58 #7 0xc0908d0f at write+0x4f #8 0xc08fd373 at syscallenter+0x263 #9 0xc0c046bf at syscall+0x4f #10 0xc0bebcc1 at Xint0x80_syscall+0x21 ino 12, on dev md5a 0xca156414: tag ufs, type VREG usecount 1, writecount 1, refcount 347 mountedhere 0 flags () v_object 0xcab04770 ref 0 pages 1380 lock type ufs: EXCL by thread 0xc88fab40 (pid 3618) #0 0xc08a548e at __lockmgr_args+0xbfe #1 0xc0b0f5c1 at ffs_lock+0xa1 #2 0xc0c205d5 at VOP_LOCK1_APV+0xb5 #3 0xc0961778 at _vn_lock+0x78 #4 0xc0962ae6 at vn_write+0x156 #5 0xc0907455 at dofilewrite+0x95 #6 0xc0908bf8 at kern_writev+0x58 #7 0xc0908d0f at write+0x4f #8 0xc08fd373 at syscallenter+0x263 #9 0xc0c046bf at syscall+0x4f #10 0xc0bebcc1 at Xint0x80_syscall+0x21 ino 13, on dev md5a 0xca6bc984: tag ufs, type VREG usecount 1, writecount 1, refcount 334 mountedhere 0 flags () v_object 0xca79f3b8 ref 0 pages 1328 lock type ufs: EXCL by thread 0xc7c535a0 (pid 3619) #0 0xc08a548e at __lockmgr_args+0xbfe #1 0xc0b0f5c1 at ffs_lock+0xa1 #2 0xc0c205d5 at VOP_LOCK1_APV+0xb5 #3 0xc0961778 at _vn_lock+0x78 #4 0xc0962ae6 at vn_write+0x156 #5 0xc0907455 at dofilewrite+0x95 #6 0xc0908bf8 at kern_writev+0x58 #7 0xc0908d0f at write+0x4f #8 0xc08fd373 at syscallenter+0x263 #9 0xc0c046bf at syscall+0x4f #10 0xc0bebcc1 at Xint0x80_syscall+0x21 ino 15, on dev md5a 0xca8f16cc: tag ufs, type VREG usecount 1, writecount 1, refcount 271 mountedhere 0 flags () v_object 0xca2be330 ref 0 pages 1076 lock type ufs: EXCL by thread 0xcab22870 (pid 3622) #0 0xc08a548e at __lockmgr_args+0xbfe #1 0xc0b0f5c1 at ffs_lock+0xa1 #2 0xc0c205d5 at VOP_LOCK1_APV+0xb5 #3 0xc0961778 at _vn_lock+0x78 #4 0xc0962ae6 at vn_write+0x156 #5 0xc0907455 at dofilewrite+0x95 #6 0xc0908bf8 at kern_writev+0x58 #7 0xc0908d0f at write+0x4f #8 0xc08fd373 at syscallenter+0x263 #9 0xc0c046bf at syscall+0x4f #10 0xc0bebcc1 at Xint0x80_syscall+0x21 ino 16, on dev md5a 0xca6bc828: tag ufs, type VREG usecount 1, writecount 1, refcount 323 mountedhere 0 flags () v_object 0xca79e660 ref 0 pages 1284 lock type ufs: EXCL by thread 0xca7422d0 (pid 3623) #0 0xc08a548e at __lockmgr_args+0xbfe #1 0xc0b0f5c1 at ffs_lock+0xa1 #2 0xc0c205d5 at VOP_LOCK1_APV+0xb5 #3 0xc0961778 at _vn_lock+0x78 #4 0xc0962ae6 at vn_write+0x156 #5 0xc0907455 at dofilewrite+0x95 #6 0xc0908bf8 at kern_writev+0x58 #7 0xc0908d0f at write+0x4f #8 0xc08fd373 at syscallenter+0x263 #9 0xc0c046bf at syscall+0x4f #10 0xc0bebcc1 at Xint0x80_syscall+0x21 ino 19, on dev md5a 0xca2a315c: tag ufs, type VREG usecount 1, writecount 1, refcount 358 mountedhere 0 flags () v_object 0xca1eeaa0 ref 0 pages 1424 lock type ufs: EXCL by thread 0xca7a0870 (pid 3621) #0 0xc08a548e at __lockmgr_args+0xbfe #1 0xc0b0f5c1 at ffs_lock+0xa1 #2 0xc0c205d5 at VOP_LOCK1_APV+0xb5 #3 0xc0961778 at _vn_lock+0x78 #4 0xc0962ae6 at vn_write+0x156 #5 0xc0907455 at dofilewrite+0x95 #6 0xc0908bf8 at kern_writev+0x58 #7 0xc0908d0f at write+0x4f #8 0xc08fd373 at syscallenter+0x263 #9 0xc0c046bf at syscall+0x4f #10 0xc0bebcc1 at Xint0x80_syscall+0x21 ino 22, on dev md5a db:0:lockedvnods> show mount 0xc7c14b50 /dev/ad4s1a on / (ufs) 0xc7c15000 devfs on /dev (devfs) 0xc7fc15a8 /dev/ad4s1f on /home (ufs) 0xc7fc1000 /dev/ad4s1d on /usr (ufs) 0xc7c15b50 /dev/ad4s1e on /var (ufs) 0xc85ec000 /dev/ad4s1g on /tmp (ufs) 0xc7c155a8 /dev/md5a on /mnt (ufs) More info: show mount db:0:mount> ps pid ppid pgrp uid state wmesg wchan cmd 3623 3081 3080 1001 R+ suj5 3622 3081 3080 1001 R+ suj5 3621 3081 3080 1001 D+ getblk 0xdad01780 suj5 3619 3081 3080 1001 D+ getblk 0xdad01780 suj5 3618 3081 3080 1001 R+ suj5 3617 3081 3080 1001 D+ getblk 0xdad01780 suj5 3615 3081 3080 1001 R+ suj5 3081 3080 3080 1001 S+ wait 0xc8523000 suj5 3080 3079 3080 1001 S+ wait 0xc7c8b000 bash 3079 3056 1820 0 S+ wait 0xc8526aa0 su 3073 0 0 0 DL mdwait 0xc8793800 [md5] 3056 1820 1820 0 S+ wait 0xc85247f8 sh 1820 1584 1820 0 S+ wait 0xc8524d48 sh 1812 1811 1812 1001 Ss+ ttyin 0xc878e070 bash 1811 1809 1809 1001 S select 0xc7c9aa24 sshd 1809 1415 1809 0 Ss sbwait 0xc8936728 sshd 1584 1580 1584 0 S+ wait 0xc7cc7000 bash 1580 1579 1580 0 S+ pause 0xc8005300 csh 1579 1566 1579 1001 S+ wait 0xc7c527f8 su 1566 1565 1566 1001 Ss+ wait 0xc7cc5aa0 bash 1565 1563 1563 1001 S select 0xc7c17ce4 sshd 1563 1415 1563 0 Ss sbwait 0xc851cd98 sshd 1548 1 1548 0 Ss+ ttyin 0xc6ec3a70 getty 1547 1 1547 0 Ss+ ttyin 0xc6ec3c70 getty 1546 1 1546 0 Ss+ ttyin 0xc6ec3e70 getty 1545 1 1545 0 Ss+ ttyin 0xc7170070 getty 1544 1 1544 0 Ss+ ttyin 0xc7170270 getty 1543 1 1543 0 Ss+ ttyin 0xc7170470 getty 1542 1 1542 0 Ss+ ttyin 0xc6ec2670 getty 1541 1 1541 0 Ss+ ttyin 0xc6ec2070 getty 1540 1 1540 0 Ss+ ttyin 0xc6ec2270 getty 1509 1 1509 0 Ss select 0xc7cf5164 inetd 1483 1 1483 0 Ss select 0xc72ef664 moused 1436 1 1436 0 Ss nanslp 0xc0e49744 cron 1429 1 1429 25 Ss pause 0xc85265a8 sendmail 1423 1 1423 0 Ss select 0xc7c17324 sendmail 1415 1 1415 0 Ss select 0xc7c9ab24 sshd 1370 1 1370 0 Ss select 0xc72efde4 ntpd 1234 1233 1233 0 S (threaded) nfsd 100141 S rpcsvc 0xc8185250 nfsd: service 100140 S rpcsvc 0xc8185290 nfsd: service 100139 S rpcsvc 0xc81852d0 nfsd: service 100097 S rpcsvc 0xc6f60950 nfsd: master 1233 1 1233 0 Ss select 0xc7c9a764 nfsd 1224 1 1224 0 Ss select 0xc72ef9e4 mountd 1127 1 1127 0 Ss select 0xc72ef6e4 rpcbind 1040 1 1040 0 Ss select 0xc72ef5e4 syslogd 839 1 839 0 Ss select 0xc72ef464 devd 21 0 0 0 DL flowclea 0xc0fb68a8 [flowcleaner] 20 0 0 0 RL CPU 1 [softdepflush] 19 0 0 0 DL vlruwt 0xc7bf4550 [vnlru] 18 0 0 0 RL CPU 3 [syncer] 17 0 0 0 DL psleep 0xc0fb6348 [bufdaemon] 16 0 0 0 DL pgzero 0xc0fc3edc [pagezero] 15 0 0 0 DL psleep 0xc0fc3ad0 [vmdaemon] 9 0 0 0 DL psleep 0xc0fc3a98 [pagedaemon] 8 0 0 0 DL ccb_scan 0xc0e153d4 [xpt_thrd] 7 0 0 0 DL waiting_ 0xc0fb80d8 [sctp_iterator] 6 0 0 0 DL - 0xc705603c [fdc0] 5 0 0 0 SL - 0xc7059000 [fw0_probe] 14 0 0 0 DL (threaded) [usb] 100056 D - 0xc7034d34 [usbus5] 100055 D - 0xc7034d04 [usbus5] 100054 D - 0xc7034cd4 [usbus5] 100053 D - 0xc7034ca4 [usbus5] 100051 D - 0xc702db5c [usbus4] 100050 D - 0xc702db2c [usbus4] 100049 D - 0xc702dafc [usbus4] 100048 D - 0xc702dacc [usbus4] 100047 D - 0xc7026b5c [usbus3] 100046 D - 0xc7026b2c [usbus3] 100045 D - 0xc7026afc [usbus3] 100044 D - 0xc7026acc [usbus3] 100043 D - 0xc701db5c [usbus2] 100042 D - 0xc701db2c [usbus2] 100041 D - 0xc701dafc [usbus2] 100040 D - 0xc701dacc [usbus2] 100038 D - 0xc7017b5c [usbus1] 100037 D - 0xc7017b2c [usbus1] 100036 D - 0xc7017afc [usbus1] 100035 D - 0xc7017acc [usbus1] 100033 D - 0xc7007b5c [usbus0] 100032 D - 0xc7007b2c [usbus0] 100031 D - 0xc7007afc [usbus0] 100030 D - 0xc7007acc [usbus0] 13 0 0 0 DL - 0xc0e49584 [yarrow] 4 0 0 0 DL - 0xc0e47244 [g_down] 3 0 0 0 DL - 0xc0e47240 [g_up] 2 0 0 0 DL - 0xc0e47238 [g_event] 12 0 0 0 RL (threaded) [intr] 100064 I [irq12: psm0] 100063 I [irq1: atkbd0] 100061 I [swi0: uart] 100058 I [irq20: fwohci0] 100057 I [irq14: ata0] 100052 I [irq19: ehci0] 100039 I [irq18: ohci2 ohci4] 100034 I [irq17: ohci1 ohci3] 100029 I [irq16: ohci0] 100028 I [irq22: atapci0] 100027 I [irq9: acpi0] 100026 I [swi6: Giant taskq] 100024 I [swi5: +] 100019 I [swi2: cambio] 100018 I [swi6: task queue] 100012 I [swi4: clock] 100011 I [swi4: clock] 100010 I [swi4: clock] 100009 Run CPU 0 [swi4: clock] 100008 I [swi1: netisr 0] 100007 I [swi3: vm] 11 0 0 0 RL (threaded) [idle] 100006 CanRun [idle: cpu0] 100005 CanRun [idle: cpu1] 100004 Run CPU 255 [idle: cpu2] 100003 CanRun [idle: cpu3] 1 0 1 0 SLs wait 0xc6d99d48 [init] 10 0 0 0 DL audit_wo 0xc0fc1880 [audit] 0 0 0 0 DLs (threaded) [kernel] 100065 D deadlkre 0xc0e49584 [deadlkres] 100059 D - 0xc6eabb40 [fw0_taskq] 100025 D - 0xc6eabc80 [thread taskq] 100023 D - 0xc6eabd00 [kqueue taskq] 100022 D - 0xc6eabd40 [acpi_task_2] 100021 D - 0xc6eabd40 [acpi_task_1] 100020 D - 0xc6eabd40 [acpi_task_0] 100016 D - 0xc6eabdc0 [firmware taskq] 100000 D sched 0xc0e47320 [swapper] db:0:ps> allt Tracing command suj5 pid 3623 tid 100180 td 0xca7422d0 sched_switch(ca7422d0,0,104,191,fd45402d,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,4f,...) at mi_switch+0x200 sleepq_switch(ca7422d0,0,c0ce92d6,268,0,...) at sleepq_switch+0x15f sleepq_wait(ca6bc954,4f,c0cefed8,0,0,...) at sleepq_wait+0x63 _sleep(ca6bc954,ca6bc920,4f,c0cefed8,0,...) at _sleep+0x372 bufobj_wwait(ca6bc920,3,0,136,0,...) at bufobj_wwait+0x73 ffs_syncvnode(ca6bc828,1,c0d0c39c,9c2,ca6bc828,...) at ffs_syncvnode+0x3b4 softdep_prealloc(ca6bc828,1,c6d437d8,c,0,...) at softdep_prealloc+0xdd ffs_balloc_ufs2(ca6bc828,500000,0,1000,c8527380,...) at ffs_balloc_ufs2+0xaa ffs_write(e991db94,c0d25edd,ca6bc828,0,ca6bc828,...) at ffs_write+0x2ad VOP_WRITE_APV(c0df9060,e991db94,ca6bc828,264,0,...) at VOP_WRITE_APV+0x136 vn_write(c84065e8,e991dc28,c8527380,0,ca7422d0,...) at vn_write+0x1ca dofilewrite(e991dc28,ffffffff,ffffffff,0,c84065e8,...) at dofilewrite+0x95 kern_writev(ca7422d0,3,e991dc28,e991dc48,1,...) at kern_writev+0x58 write(ca7422d0,e991dcec,e991dd28,c0ce9b0f,0,...) at write+0x4f syscallenter(ca7422d0,e991dce4,e991dce4,0,c0e4d1c0,...) at syscallenter+0x263 syscall(e991dd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x28181253, esp = 0xbfbfd21c, ebp = 0xbfbfe8c8 --- Tracing command suj5 pid 3622 tid 100212 td 0xcab22870 sched_switch(cab22870,0,602,18c,fd423877,...) at sched_switch+0x3bc mi_switch(602,0,c0ce5189,cd,4,...) at mi_switch+0x200 critical_exit(cab22870,0,1,c0fc2018,0,...) at critical_exit+0xa8 ipi_bitmap_handler(8,28,28,cab22870,0,...) at ipi_bitmap_handler+0x82 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0x2f --- interrupt, eip = 0xc08acf89, esp = 0xe9997748, ebp = 0xe9997764 --- _mtx_lock_sleep(c0fc2018,cab22870,0,c0d0c39c,26c6,...) at _mtx_lock_sleep+0x99 _mtx_lock_flags(c0fc2018,0,c0d0c39c,26c6,c7c155a8,...) at _mtx_lock_flags+0xf7 softdep_update_inodeblock(c9bb4488,dad01720,1,4000,0,...) at softdep_update_inodeblock+0xd4 ffs_update(ca8f16cc,1,c0d0f244,163,c0f884f8,...) at ffs_update+0x1aa ffs_syncvnode(ca8f16cc,1,c0d0c39c,9c2,ca8f16cc,...) at ffs_syncvnode+0x48f softdep_prealloc(ca8f16cc,1,c6d437d8,c,0,...) at softdep_prealloc+0xdd ffs_balloc_ufs2(ca8f16cc,430000,0,1000,c8527380,...) at ffs_balloc_ufs2+0xaa ffs_write(e9997b94,c0d25edd,ca8f16cc,0,ca8f16cc,...) at ffs_write+0x2ad VOP_WRITE_APV(c0df9060,e9997b94,ca8f16cc,264,0,...) at VOP_WRITE_APV+0x136 vn_write(c7c59968,e9997c28,c8527380,0,cab22870,...) at vn_write+0x1ca dofilewrite(e9997c28,ffffffff,ffffffff,0,c7c59968,...) at dofilewrite+0x95 kern_writev(cab22870,3,e9997c28,e9997c48,1,...) at kern_writev+0x58 write(cab22870,e9997cec,e9997d28,c0ce9b0f,0,...) at write+0x4f syscallenter(cab22870,e9997ce4,e9997ce4,0,c0e4dec0,...) at syscallenter+0x263 syscall(e9997d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x28181253, esp = 0xbfbfd21c, ebp = 0xbfbfe8c8 --- Tracing command suj5 pid 3621 tid 100195 td 0xca7a0870 sched_switch(ca7a0870,0,104,191,fd431932,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,50,...) at mi_switch+0x200 sleepq_switch(ca7a0870,0,c0ce92d6,268,50,...) at sleepq_switch+0x15f sleepq_wait(dad01780,50,c0cdddf4,4,0,...) at sleepq_wait+0x63 __lockmgr_args(dad01780,81900,ca6bc7c4,c0cdddf4,50,...) at __lockmgr_args+0xb1f getblk(ca6bc6cc,e0,0,4000,0,...) at getblk+0x167 breadn(ca6bc6cc,e0,0,4000,0,...) at breadn+0x44 bread(ca6bc6cc,e0,0,4000,0,...) at bread+0x4c ffs_update(ca2a315c,1,c0d0f244,163,0,...) at ffs_update+0x172 ffs_syncvnode(ca2a315c,1,c0d0c39c,9c2,ca2a315c,...) at ffs_syncvnode+0x48f softdep_prealloc(ca2a315c,1,dae25de0,e994a8c4,c0904acb,...) at softdep_prealloc+0xdd ffs_balloc_ufs2(ca2a315c,58b000,0,1000,c8527380,...) at ffs_balloc_ufs2+0xaa ffs_write(e994ab94,c0d25edd,ca2a315c,0,ca2a315c,...) at ffs_write+0x2ad VOP_WRITE_APV(c0df9060,e994ab94,ca2a315c,264,0,...) at VOP_WRITE_APV+0x136 vn_write(c7d06968,e994ac28,c8527380,0,ca7a0870,...) at vn_write+0x1ca dofilewrite(e994ac28,ffffffff,ffffffff,0,c7d06968,...) at dofilewrite+0x95 kern_writev(ca7a0870,3,e994ac28,e994ac48,1,...) at kern_writev+0x58 write(ca7a0870,e994acec,e994ad28,c0ce9b0f,0,...) at write+0x4f syscallenter(ca7a0870,e994ace4,e994ace4,0,c0e4dec0,...) at syscallenter+0x263 syscall(e994ad28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x28181253, esp = 0xbfbfd21c, ebp = 0xbfbfe8c8 --- Tracing command suj5 pid 3619 tid 100118 td 0xc7c535a0 sched_switch(c7c535a0,0,104,191,fd42e390,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,50,...) at mi_switch+0x200 sleepq_switch(c7c535a0,0,c0ce92d6,268,50,...) at sleepq_switch+0x15f sleepq_wait(dad01780,50,c0cdddf4,4,0,...) at sleepq_wait+0x63 __lockmgr_args(dad01780,81900,ca6bc7c4,c0cdddf4,50,...) at __lockmgr_args+0xb1f getblk(ca6bc6cc,e0,0,4000,0,...) at getblk+0x167 breadn(ca6bc6cc,e0,0,4000,0,...) at breadn+0x44 bread(ca6bc6cc,e0,0,4000,0,...) at bread+0x4c ffs_update(ca6bc984,1,c0d0f244,163,0,...) at ffs_update+0x172 ffs_syncvnode(ca6bc984,1,c0d0c39c,9c2,ca6bc984,...) at ffs_syncvnode+0x48f softdep_prealloc(ca6bc984,1,dae78f20,e981c8c4,c0904acb,...) at softdep_prealloc+0xdd ffs_balloc_ufs2(ca6bc984,529000,0,1000,c8527380,...) at ffs_balloc_ufs2+0xaa ffs_write(e981cb94,c0d25edd,ca6bc984,0,ca6bc984,...) at ffs_write+0x2ad VOP_WRITE_APV(c0df9060,e981cb94,ca6bc984,264,0,...) at VOP_WRITE_APV+0x136 vn_write(c7c823b8,e981cc28,c8527380,0,c7c535a0,...) at vn_write+0x1ca dofilewrite(e981cc28,ffffffff,ffffffff,0,c7c823b8,...) at dofilewrite+0x95 kern_writev(c7c535a0,3,e981cc28,e981cc48,1,...) at kern_writev+0x58 write(c7c535a0,e981ccec,c08e18bc,c0e4d1f0,0,...) at write+0x4f syscallenter(c7c535a0,e981cce4,e981ccc0,c08e3248,c0e4d1c0,...) at syscallenter+0x263 syscall(e981cd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x28181253, esp = 0xbfbfd21c, ebp = 0xbfbfe8c8 --- Tracing command suj5 pid 3618 tid 100168 td 0xc88fab40 sched_switch(c88fab40,0,104,191,fd444ed0,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,4f,...) at mi_switch+0x200 sleepq_switch(c88fab40,0,c0ce92d6,268,0,...) at sleepq_switch+0x15f sleepq_wait(ca156540,4f,c0cefed8,0,0,...) at sleepq_wait+0x63 _sleep(ca156540,ca15650c,4f,c0cefed8,0,...) at _sleep+0x372 bufobj_wwait(ca15650c,3,0,eb,c0f882e8,...) at bufobj_wwait+0x73 ffs_syncvnode(ca156414,1,c0d0c39c,9c2,ca156414,...) at ffs_syncvnode+0x3b4 softdep_prealloc(ca156414,1,dae928a0,e98c88c4,c0904acb,...) at softdep_prealloc+0xdd ffs_balloc_ufs2(ca156414,55e000,0,1000,c8527380,...) at ffs_balloc_ufs2+0xaa ffs_write(e98c8b94,c0d25edd,ca156414,0,ca156414,...) at ffs_write+0x2ad VOP_WRITE_APV(c0df9060,e98c8b94,ca156414,264,0,...) at VOP_WRITE_APV+0x136 vn_write(c894d888,e98c8c28,c8527380,0,c88fab40,...) at vn_write+0x1ca dofilewrite(e98c8c28,ffffffff,ffffffff,0,c894d888,...) at dofilewrite+0x95 kern_writev(c88fab40,3,e98c8c28,e98c8c48,1,...) at kern_writev+0x58 write(c88fab40,e98c8cec,c08e18bc,c0e4def0,0,...) at write+0x4f syscallenter(c88fab40,e98c8ce4,e98c8cc0,c08e3248,c0e4dec0,...) at syscallenter+0x263 syscall(e98c8d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x28181253, esp = 0xbfbfd21c, ebp = 0xbfbfe8c8 --- Tracing command suj5 pid 3617 tid 100089 td 0xc7c56870 sched_switch(c7c56870,0,104,191,fd42fbac,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,50,...) at mi_switch+0x200 sleepq_switch(c7c56870,0,c0ce92d6,268,50,...) at sleepq_switch+0x15f sleepq_wait(dad01780,50,c0cdddf4,4,0,...) at sleepq_wait+0x63 __lockmgr_args(dad01780,81900,ca6bc7c4,c0cdddf4,50,...) at __lockmgr_args+0xb1f getblk(ca6bc6cc,e0,0,4000,0,...) at getblk+0x167 breadn(ca6bc6cc,e0,0,4000,0,...) at breadn+0x44 bread(ca6bc6cc,e0,0,4000,0,...) at bread+0x4c ffs_update(ca6bec3c,1,c0d0f244,163,0,...) at ffs_update+0x172 ffs_syncvnode(ca6bec3c,1,c0d0c39c,9c2,ca6bec3c,...) at ffs_syncvnode+0x48f softdep_prealloc(ca6bec3c,1,dad7e7e0,e975e8c4,c0904acb,...) at softdep_prealloc+0xdd ffs_balloc_ufs2(ca6bec3c,543000,0,1000,c8527380,...) at ffs_balloc_ufs2+0xaa ffs_write(e975eb94,c0d25edd,ca6bec3c,0,ca6bec3c,...) at ffs_write+0x2ad VOP_WRITE_APV(c0df9060,e975eb94,ca6bec3c,264,0,...) at VOP_WRITE_APV+0x136 vn_write(c7cdf540,e975ec28,c8527380,0,c7c56870,...) at vn_write+0x1ca dofilewrite(e975ec28,ffffffff,ffffffff,0,c7cdf540,...) at dofilewrite+0x95 kern_writev(c7c56870,3,e975ec28,e975ec48,1,...) at kern_writev+0x58 write(c7c56870,e975ecec,e975ed28,c0ce9b0f,0,...) at write+0x4f syscallenter(c7c56870,e975ece4,e975ece4,0,c0e4dec0,...) at syscallenter+0x263 syscall(e975ed28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x28181253, esp = 0xbfbfd21c, ebp = 0xbfbfe8c8 --- Tracing command suj5 pid 3615 tid 100113 td 0xc8006b40 sched_switch(c8006b40,0,104,191,fd3b11eb,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,50,...) at mi_switch+0x200 sleepq_switch(c8006b40,0,c0ce92d6,268,50,...) at sleepq_switch+0x15f sleepq_wait(dad01780,50,c0cdddf4,4,0,...) at sleepq_wait+0x63 __lockmgr_args(dad01780,81900,ca6bc7c4,c0cdddf4,50,...) at __lockmgr_args+0xb1f getblk(ca6bc6cc,e0,0,4000,0,...) at getblk+0x167 breadn(ca6bc6cc,e0,0,4000,0,...) at breadn+0x44 bread(ca6bc6cc,e0,0,4000,0,...) at bread+0x4c ffs_update(c9bab828,1,c0d0f244,163,c0f88550,...) at ffs_update+0x172 ffs_syncvnode(c9bab828,1,c0d0c39c,9c2,c9bab828,...) at ffs_syncvnode+0x48f softdep_prealloc(c9bab828,1,daee18e0,e98078c4,c0904acb,...) at softdep_prealloc+0xdd ffs_balloc_ufs2(c9bab828,4c7000,0,1000,c8527380,...) at ffs_balloc_ufs2+0xaa ffs_write(e9807b94,c0d25edd,c9bab828,0,c9bab828,...) at ffs_write+0x2ad VOP_WRITE_APV(c0df9060,e9807b94,c9bab828,264,0,...) at VOP_WRITE_APV+0x136 vn_write(c8171dc8,e9807c28,c8527380,0,c8006b40,...) at vn_write+0x1ca dofilewrite(e9807c28,ffffffff,ffffffff,0,c8171dc8,...) at dofilewrite+0x95 kern_writev(c8006b40,3,e9807c28,e9807c48,1,...) at kern_writev+0x58 write(c8006b40,e9807cec,c08e18bc,c0e4d1f0,0,...) at write+0x4f syscallenter(c8006b40,e9807ce4,e9807cc0,c08e3248,c0e4d1c0,...) at syscallenter+0x263 syscall(e9807d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x28181253, esp = 0xbfbfd21c, ebp = 0xbfbfe8c8 --- Tracing command suj5 pid 3081 tid 100154 td 0xc853a2d0 sched_switch(c853a2d0,0,104,191,f2d61161,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c853a2d0,0,c0ce92d6,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce92d6,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8523000,5c,c0cebc35,100,0,...) at sleepq_wait_sig+0x17 _sleep(c8523000,c8523088,15c,c0cebc35,0,...) at _sleep+0x35c kern_wait(c853a2d0,ffffffff,e9895c44,0,0,...) at kern_wait+0xb66 wait4(c853a2d0,e9895cec,c853a2d0,602,0,...) at wait4+0x3b syscallenter(c853a2d0,e9895ce4,c0ce325f,109,c853a2d0,...) at syscallenter+0x263 syscall(e9895d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x280efdeb, esp = 0xbfbfd1fc, ebp = 0xbfbfd218 --- Tracing command bash pid 3080 tid 100084 td 0xc7c80000 sched_switch(c7c80000,0,104,191,3c25db59,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c7c80000,0,c0ce92d6,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce92d6,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7c8b000,5c,c0cebc35,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7c8b000,c7c8b088,15c,c0cebc35,0,...) at _sleep+0x35c kern_wait(c7c80000,ffffffff,e974ac44,0,0,...) at kern_wait+0xb66 wait4(c7c80000,e974acec,2836aa8c,1,0,...) at wait4+0x3b syscallenter(c7c80000,e974ace4,c0c04f1d,c0e48eb0,0,...) at syscallenter+0x263 syscall(e974ad28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x282c2deb, esp = 0xbfbfe42c, ebp = 0xbfbfe448 --- Tracing command su pid 3079 tid 100169 td 0xc88fa870 sched_switch(c88fa870,0,104,191,3b694495,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c88fa870,0,c0ce92d6,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce92d6,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8526aa0,5c,c0cebc35,100,0,...) at sleepq_wait_sig+0x17 _sleep(c8526aa0,c8526b28,15c,c0cebc35,0,...) at _sleep+0x35c kern_wait(c88fa870,c08,e98cbc44,2,0,...) at kern_wait+0xb66 wait4(c88fa870,e98cbcec,281cba8c,1,0,...) at wait4+0x3b syscallenter(c88fa870,e98cbce4,c0c04f1d,c0e48eb0,0,...) at syscallenter+0x263 syscall(e98cbd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x28123deb, esp = 0xbfbfe36c, ebp = 0xbfbfe388 --- Tracing command md5 pid 3073 tid 100135 td 0xc8006870 sched_switch(c8006870,0,104,191,fd4968de,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,4c,...) at mi_switch+0x200 sleepq_switch(c8006870,0,c0ce92d6,268,0,...) at sleepq_switch+0x15f sleepq_wait(c8793800,4c,c0cb682d,0,0,...) at sleepq_wait+0x63 _sleep(c8793800,c8793820,24c,c0cb682d,0,...) at _sleep+0x372 md_kthread(c8793800,e9855d28,c0cdfcac,349,c8002000,...) at md_kthread+0x115 fork_exit(c06d6920,c8793800,e9855d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe9855d60, ebp = 0 --- Tracing command sh pid 3056 tid 100145 td 0xc853d000 sched_switch(c853d000,0,104,191,3a489411,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c853d000,0,c0ce92d6,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce92d6,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c85247f8,5c,c0cebc35,100,0,...) at sleepq_wait_sig+0x17 _sleep(c85247f8,c8524880,15c,c0cebc35,0,...) at _sleep+0x35c kern_wait(c853d000,ffffffff,e987ac44,2,0,...) at kern_wait+0xb66 wait4(c853d000,e987acec,e987ad28,c0ce9b0f,0,...) at wait4+0x3b syscallenter(c853d000,e987ace4,e987ace4,0,0,...) at syscallenter+0x263 syscall(e987ad28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x28162deb, esp = 0xbfbfe84c, ebp = 0xbfbfe868 --- Tracing command sh pid 1820 tid 100143 td 0xc853d5a0 sched_switch(c853d5a0,0,104,191,1d65b5c1,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c853d5a0,0,c0ce92d6,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce92d6,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8524d48,5c,c0cebc35,100,0,...) at sleepq_wait_sig+0x17 _sleep(c8524d48,c8524dd0,15c,c0cebc35,0,...) at _sleep+0x35c kern_wait(c853d5a0,ffffffff,e9874c44,2,0,...) at kern_wait+0xb66 wait4(c853d5a0,e9874cec,e9874d28,c0ce9b0f,0,...) at wait4+0x3b syscallenter(c853d5a0,e9874ce4,e9874ce4,0,0,...) at syscallenter+0x263 syscall(e9874d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x28162deb, esp = 0xbfbfe65c, ebp = 0xbfbfe678 --- Tracing command bash pid 1812 tid 100102 td 0xc7cce2d0 sched_switch(c7cce2d0,0,104,191,39b1f3fb,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7cce2d0,0,c0ce92d6,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08acdca,c878e004,0,c0ce325f,c7cce2d0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c878e070,0,e9794ad8,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c878e070,c878e004,c0ced3ae,514,0,...) at _cv_wait_sig+0x243 tty_wait(c878e000,c878e070,e9794c28,1,0,...) at tty_wait+0x71 ttydisc_read(c878e000,e9794c28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c7fbf600,e9794c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7d06818,e9794c28,c8782c00,0,c7cce2d0,...) at devfs_read_f+0x7e dofileread(e9794c28,ffffffff,ffffffff,0,c7d06818,...) at dofileread+0x9e kern_readv(c7cce2d0,0,e9794c28,e9794c48,1,...) at kern_readv+0x58 read(c7cce2d0,e9794cec,e9794d28,c0ce9b0f,0,...) at read+0x4f syscallenter(c7cce2d0,e9794ce4,e9794ce4,0,0,...) at syscallenter+0x263 syscall(e9794d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28354273, esp = 0xbfbfe00c, ebp = 0xbfbfe038 --- Tracing command sshd pid 1811 tid 100164 td 0xc88fd870 sched_switch(c88fd870,0,104,191,39b4739b,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c88fd870,0,c0ce92d6,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08acdca,c7c9aa10,0,c0ce325f,c88fd870,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7c9aa24,0,e98bca4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7c9aa24,c7c9aa10,c0ceb69b,627,c7c82d58,...) at _cv_wait_sig+0x243 seltdwait(c7c82d58,58,c8782a80,c88fd870,246,...) at seltdwait+0xa2 kern_select(c88fd870,a,288060b8,288060dc,0,0,20,c88f37f8,c88f37f8) at kern_select+0x504 select(c88fd870,e98bccec,e98bcd28,c0ce9b0f,0,...) at select+0x66 syscallenter(c88fd870,e98bcce4,e98bcce4,0,ec,...) at syscallenter+0x263 syscall(e98bcd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x284b01f3, esp = 0xbfbfe1bc, ebp = 0xbfbfe208 --- Tracing command sshd pid 1809 tid 100155 td 0xc853a000 sched_switch(c853a000,0,104,191,5624c612,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,58,...) at mi_switch+0x200 sleepq_switch(c853a000,0,c0ce92d6,1a7,58,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce92d6,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8936728,58,c0ceefce,100,0,...) at sleepq_wait_sig+0x17 _sleep(c8936728,c89366e8,158,c0ceefce,0) at _sleep+0x35c sbwait(c89366c4,4,c0cef09d,5c9,c89366e8,...) at sbwait+0x76 soreceive_generic(c8936670,0,e98a1c28,0,0,...) at soreceive_generic+0x3f0 soreceive(c8936670,0,e98a1c28,0,0,0) at soreceive+0x38 soo_read(c7c82070,e98a1c28,ca268180,0,c853a000,...) at soo_read+0x4e dofileread(e98a1c28,ffffffff,ffffffff,0,c7c82070,...) at dofileread+0x9e kern_readv(c853a000,5,e98a1c28,e98a1c48,1,...) at kern_readv+0x58 read(c853a000,e98a1cec,e98a1d28,c0ce9b0f,0,...) at read+0x4f syscallenter(c853a000,e98a1ce4,e98a1ce4,0,0,...) at syscallenter+0x263 syscall(e98a1d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x284b0273, esp = 0xbfbfdd8c, ebp = 0xbfbfddc8 --- Tracing command bash pid 1584 tid 100086 td 0xc7ccf870 sched_switch(c7ccf870,0,104,191,86cac64b,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c7ccf870,0,c0ce92d6,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce92d6,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7cc7000,5c,c0cebc35,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7cc7000,c7cc7088,15c,c0cebc35,0,...) at _sleep+0x35c kern_wait(c7ccf870,ffffffff,e9752c44,6,0,...) at kern_wait+0xb66 wait4(c7ccf870,e9752cec,e9752d28,c0ce9b0f,0,...) at wait4+0x3b syscallenter(c7ccf870,e9752ce4,e9752ce4,0,0,...) at syscallenter+0x263 syscall(e9752d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x282c2deb, esp = 0xbfbfe7bc, ebp = 0xbfbfe7d8 --- Tracing command csh pid 1580 tid 100107 td 0xc800d000 sched_switch(c800d000,0,104,191,58d8a43a,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,68,...) at mi_switch+0x200 sleepq_switch(c800d000,0,c0ce92d6,1a7,68,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce92d6,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8005300,68,c0c953f3,100,0,...) at sleepq_wait_sig+0x17 _sleep(c8005300,c8005330,168,c0c953f3,0,...) at _sleep+0x35c kern_sigsuspend(c800d000,0,0,0,0,...) at kern_sigsuspend+0xba sigsuspend(c800d000,e97c3cec,e97c3d28,c0ce9b0f,0,...) at sigsuspend+0x4d syscallenter(c800d000,e97c3ce4,e97c3ce4,0,0,...) at syscallenter+0x263 syscall(e97c3d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x2817c87b, esp = 0xbfbfe5cc, ebp = 0xbfbfe638 --- Tracing command su pid 1579 tid 100076 td 0xc7c56b40 sched_switch(c7c56b40,0,104,191,a0ac950,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c7c56b40,0,c0ce92d6,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce92d6,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7c527f8,5c,c0cebc35,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7c527f8,c7c52880,15c,c0cebc35,0,...) at _sleep+0x35c kern_wait(c7c56b40,62c,e972ac44,2,0,...) at kern_wait+0xb66 wait4(c7c56b40,e972acec,281cba8c,1,0,...) at wait4+0x3b syscallenter(c7c56b40,e972ace4,c0c04f1d,c0e48eb0,0,...) at syscallenter+0x263 syscall(e972ad28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x28123deb, esp = 0xbfbfe5fc, ebp = 0xbfbfe618 --- Tracing command bash pid 1566 tid 100093 td 0xc7ccf2d0 sched_switch(c7ccf2d0,0,104,191,ddcb7ea5,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c7ccf2d0,0,c0ce92d6,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce92d6,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7cc5aa0,5c,c0cebc35,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7cc5aa0,c7cc5b28,15c,c0cebc35,0,...) at _sleep+0x35c kern_wait(c7ccf2d0,ffffffff,e976ec44,6,0,...) at kern_wait+0xb66 wait4(c7ccf2d0,e976ecec,e976ed28,c0ce9b0f,0,...) at wait4+0x3b syscallenter(c7ccf2d0,e976ece4,e976ece4,0,0,...) at syscallenter+0x263 syscall(e976ed28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x282c2deb, esp = 0xbfbfe9bc, ebp = 0xbfbfe9d8 --- Tracing command sshd pid 1565 tid 100151 td 0xc853ab40 sched_switch(c853ab40,0,104,191,3c61026e,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c853ab40,0,c0ce92d6,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08acdca,c7c17cd0,0,c0ce325f,c853ab40,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7c17ce4,0,e988ca4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7c17ce4,c7c17cd0,c0ceb69b,627,c7c82230,...) at _cv_wait_sig+0x243 seltdwait(c7c82230,58,c7cf4680,c853ab40,246,...) at seltdwait+0xa2 kern_select(c853ab40,a,288060b8,288060dc,0,0,20,c85237f8,c85237f8) at kern_select+0x504 select(c853ab40,e988ccec,e988cd28,c0ce9b0f,0,...) at select+0x66 syscallenter(c853ab40,e988cce4,e988cce4,0,0,...) at syscallenter+0x263 syscall(e988cd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x284b01f3, esp = 0xbfbfe1bc, ebp = 0xbfbfe208 --- Tracing command sshd pid 1563 tid 100105 td 0xc800d5a0 sched_switch(c800d5a0,0,104,191,14bab114,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,58,...) at mi_switch+0x200 sleepq_switch(c800d5a0,0,c0ce92d6,1a7,58,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce92d6,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c851cd98,58,c0ceefce,100,0,...) at sleepq_wait_sig+0x17 _sleep(c851cd98,c851cd58,158,c0ceefce,0) at _sleep+0x35c sbwait(c851cd34,4,c0cef09d,5c9,c851cd58,...) at sbwait+0x76 soreceive_generic(c851cce0,0,e97bbc28,0,0,...) at soreceive_generic+0x3f0 soreceive(c851cce0,0,e97bbc28,0,0,0) at soreceive+0x38 soo_read(c7c592d8,e97bbc28,c8527a80,0,c800d5a0,...) at soo_read+0x4e dofileread(e97bbc28,ffffffff,ffffffff,0,c7c592d8,...) at dofileread+0x9e kern_readv(c800d5a0,5,e97bbc28,e97bbc48,1,...) at kern_readv+0x58 read(c800d5a0,e97bbcec,e97bbd28,c0ce9b0f,0,...) at read+0x4f syscallenter(c800d5a0,e97bbce4,e97bbce4,0,0,...) at syscallenter+0x263 syscall(e97bbd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x284b0273, esp = 0xbfbfdd8c, ebp = 0xbfbfddc8 --- Tracing command getty pid 1548 tid 100103 td 0xc7cce000 sched_switch(c7cce000,0,104,191,65b08f9b,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7cce000,0,c0ce92d6,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce92d6,160,0,c7cce000,c7cce000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6ec3a70,0,c0ceda10,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6ec3a70,c0e48eb0,c0ced3ae,514,0,...) at _cv_wait_sig+0x243 tty_wait(c6ec3a00,c6ec3a70,e9798c28,1,0,...) at tty_wait+0x71 ttydisc_read(c6ec3a00,e9798c28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c6fc2800,e9798c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7cdf5b0,e9798c28,c6d7be80,0,c7cce000,...) at devfs_read_f+0x7e dofileread(e9798c28,ffffffff,ffffffff,0,c7cdf5b0,...) at dofileread+0x9e kern_readv(c7cce000,0,e9798c28,e9798c48,1,...) at kern_readv+0x58 read(c7cce000,e9798cec,e9798d28,c0ce9b0f,0,...) at read+0x4f syscallenter(c7cce000,e9798ce4,e9798ce4,0,0,...) at syscallenter+0x263 syscall(e9798d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196273, esp = 0xbfbfedac, ebp = 0xbfbfedd8 --- Tracing command getty pid 1547 tid 100109 td 0xc800c870 sched_switch(c800c870,0,104,191,66219a9e,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c800c870,0,c0ce92d6,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce92d6,160,0,c800c870,c800c870,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6ec3c70,0,c0ceda10,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6ec3c70,c0e48eb0,c0ced3ae,514,0,...) at _cv_wait_sig+0x243 tty_wait(c6ec3c00,c6ec3c70,e97e1c28,1,0,...) at tty_wait+0x71 ttydisc_read(c6ec3c00,e97e1c28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c6fc2900,e97e1c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7c82dc8,e97e1c28,c6d7be80,0,c800c870,...) at devfs_read_f+0x7e dofileread(e97e1c28,ffffffff,ffffffff,0,c7c82dc8,...) at dofileread+0x9e kern_readv(c800c870,0,e97e1c28,e97e1c48,1,...) at kern_readv+0x58 read(c800c870,e97e1cec,e97e1d28,c0ce9b0f,0,...) at read+0x4f syscallenter(c800c870,e97e1ce4,e97e1ce4,0,0,...) at syscallenter+0x263 syscall(e97e1d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196273, esp = 0xbfbfedac, ebp = 0xbfbfedd8 --- Tracing command getty pid 1546 tid 100110 td 0xc800c5a0 sched_switch(c800c5a0,0,104,191,660e54b5,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c800c5a0,0,c0ce92d6,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce92d6,160,0,c800c5a0,c800c5a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6ec3e70,0,c0ceda10,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6ec3e70,c0e48eb0,c0ced3ae,514,0,...) at _cv_wait_sig+0x243 tty_wait(c6ec3e00,c6ec3e70,e97e5c28,1,0,...) at tty_wait+0x71 ttydisc_read(c6ec3e00,e97e5c28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c6fc2a00,e97e5c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7cdf850,e97e5c28,c6d7be80,0,c800c5a0,...) at devfs_read_f+0x7e dofileread(e97e5c28,ffffffff,ffffffff,0,c7cdf850,...) at dofileread+0x9e kern_readv(c800c5a0,0,e97e5c28,e97e5c48,1,...) at kern_readv+0x58 read(c800c5a0,e97e5cec,e97e5d28,c0ce9b0f,0,...) at read+0x4f syscallenter(c800c5a0,e97e5ce4,e97e5ce4,0,0,...) at syscallenter+0x263 syscall(e97e5d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196273, esp = 0xbfbfedac, ebp = 0xbfbfedd8 --- Tracing command getty pid 1545 tid 100111 td 0xc800c2d0 sched_switch(c800c2d0,0,104,191,65cf7bb5,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c800c2d0,0,c0ce92d6,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce92d6,160,0,c800c2d0,c800c2d0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7170070,0,c0ceda10,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7170070,c0e48eb0,c0ced3ae,514,0,...) at _cv_wait_sig+0x243 tty_wait(c7170000,c7170070,e97e9c28,1,0,...) at tty_wait+0x71 ttydisc_read(c7170000,e97e9c28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c6fc2b00,e97e9c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7cdfe38,e97e9c28,c6d7be80,0,c800c2d0,...) at devfs_read_f+0x7e dofileread(e97e9c28,ffffffff,ffffffff,0,c7cdfe38,...) at dofileread+0x9e kern_readv(c800c2d0,0,e97e9c28,e97e9c48,1,...) at kern_readv+0x58 read(c800c2d0,e97e9cec,e97e9d28,c0ce9b0f,0,...) at read+0x4f syscallenter(c800c2d0,e97e9ce4,e97e9ce4,0,0,...) at syscallenter+0x263 syscall(e97e9d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196273, esp = 0xbfbfedac, ebp = 0xbfbfedd8 --- Tracing command getty pid 1544 tid 100078 td 0xc7c80b40 sched_switch(c7c80b40,0,104,191,659e0315,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7c80b40,0,c0ce92d6,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce92d6,160,0,c7c80b40,c7c80b40,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7170270,0,c0ceda10,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7170270,c0e48eb0,c0ced3ae,514,0,...) at _cv_wait_sig+0x243 tty_wait(c7170200,c7170270,e9732c28,1,0,...) at tty_wait+0x71 ttydisc_read(c7170200,e9732c28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c6fc2c00,e9732c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c8406658,e9732c28,c6d7be80,0,c7c80b40,...) at devfs_read_f+0x7e dofileread(e9732c28,ffffffff,ffffffff,0,c8406658,...) at dofileread+0x9e kern_readv(c7c80b40,0,e9732c28,e9732c48,1,...) at kern_readv+0x58 read(c7c80b40,e9732cec,e9732d28,c0ce9b0f,0,...) at read+0x4f syscallenter(c7c80b40,e9732ce4,e9732ce4,0,0,...) at syscallenter+0x263 syscall(e9732d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196273, esp = 0xbfbfedac, ebp = 0xbfbfedd8 --- Tracing command getty pid 1543 tid 100112 td 0xc800c000 sched_switch(c800c000,0,104,191,662d5a87,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c800c000,0,c0ce92d6,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce92d6,160,0,c800c000,c800c000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7170470,0,c0ceda10,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7170470,c0e48eb0,c0ced3ae,514,0,...) at _cv_wait_sig+0x243 tty_wait(c7170400,c7170470,e9803c28,1,0,...) at tty_wait+0x71 ttydisc_read(c7170400,e9803c28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c6fc2d00,e9803c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7cdf658,e9803c28,c6d7be80,0,c800c000,...) at devfs_read_f+0x7e dofileread(e9803c28,ffffffff,ffffffff,0,c7cdf658,...) at dofileread+0x9e kern_readv(c800c000,0,e9803c28,e9803c48,1,...) at kern_readv+0x58 read(c800c000,e9803cec,e9803d28,c0ce9b0f,0,...) at read+0x4f syscallenter(c800c000,e9803ce4,e9803ce4,0,0,...) at syscallenter+0x263 syscall(e9803d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196273, esp = 0xbfbfedac, ebp = 0xbfbfedd8 --- Tracing command getty pid 1542 tid 100101 td 0xc7cce5a0 sched_switch(c7cce5a0,0,104,191,663ad5e8,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7cce5a0,0,c0ce92d6,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce92d6,160,0,c7cce5a0,c7cce5a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6ec2670,0,c0ceda10,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6ec2670,c0e48eb0,c0ced3ae,514,0,...) at _cv_wait_sig+0x243 tty_wait(c6ec2600,c6ec2670,e9790c28,1,0,...) at tty_wait+0x71 ttydisc_read(c6ec2600,e9790c28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c6fc2e00,e9790c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c8548310,e9790c28,c6d7be80,0,c7cce5a0,...) at devfs_read_f+0x7e dofileread(e9790c28,ffffffff,ffffffff,0,c8548310,...) at dofileread+0x9e kern_readv(c7cce5a0,0,e9790c28,e9790c48,1,...) at kern_readv+0x58 read(c7cce5a0,e9790cec,e9790d28,c0ce9b0f,0,...) at read+0x4f syscallenter(c7cce5a0,e9790ce4,e9790ce4,0,0,...) at syscallenter+0x263 syscall(e9790d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196273, esp = 0xbfbfedac, ebp = 0xbfbfedd8 --- Tracing command getty pid 1541 tid 100116 td 0xc7c53b40 sched_switch(c7c53b40,0,104,191,65ab0cea,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7c53b40,0,c0ce92d6,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce92d6,160,0,c7c53b40,c7c53b40,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6ec2070,0,c0ceda10,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6ec2070,c0e48eb0,c0ced3ae,514,0,...) at _cv_wait_sig+0x243 tty_wait(c6ec2000,c6ec2070,e9814c28,1,0,...) at tty_wait+0x71 ttydisc_read(c6ec2000,e9814c28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c715e000,e9814c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7c82d20,e9814c28,c6d7be80,0,c7c53b40,...) at devfs_read_f+0x7e dofileread(e9814c28,ffffffff,ffffffff,0,c7c82d20,...) at dofileread+0x9e kern_readv(c7c53b40,0,e9814c28,e9814c48,1,...) at kern_readv+0x58 read(c7c53b40,e9814cec,e9814d28,c0ce9b0f,0,...) at read+0x4f syscallenter(c7c53b40,e9814ce4,e9814ce4,0,0,...) at syscallenter+0x263 syscall(e9814d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196273, esp = 0xbfbfedac, ebp = 0xbfbfedd8 --- Tracing command getty pid 1540 tid 100133 td 0xc81105a0 sched_switch(c81105a0,0,104,191,aa700bc,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c81105a0,0,c0ce92d6,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08acdca,c6ec2204,0,c0ce325f,c81105a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6ec2270,0,e984bad8,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6ec2270,c6ec2204,c0ced3ae,514,0,...) at _cv_wait_sig+0x243 tty_wait(c6ec2200,c6ec2270,e984bc28,1,0,...) at tty_wait+0x71 ttydisc_read(c6ec2200,e984bc28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c6de2c00,e984bc28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c8406e38,e984bc28,c6d7be80,0,c81105a0,...) at devfs_read_f+0x7e dofileread(e984bc28,ffffffff,ffffffff,0,c8406e38,...) at dofileread+0x9e kern_readv(c81105a0,0,e984bc28,e984bc48,1,...) at kern_readv+0x58 read(c81105a0,e984bcec,e984bd28,c0ce9b0f,0,...) at read+0x4f syscallenter(c81105a0,e984bce4,e984bce4,0,0,...) at syscallenter+0x263 syscall(e984bd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196273, esp = 0xbfbfedac, ebp = 0xbfbfedd8 --- Tracing command inetd pid 1509 tid 100146 td 0xc853bb40 sched_switch(c853bb40,0,104,191,56a95e2c,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c853bb40,0,c0ce92d6,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08acdca,c7cf5150,0,c0ce325f,c853bb40,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7cf5164,0,e987da4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7cf5164,c7cf5150,c0ceb69b,627,c8406cb0,...) at _cv_wait_sig+0x243 seltdwait(c8406cb0,58,c6d7be80,c853bb40,c0ce325f,...) at seltdwait+0xa2 kern_select(c853bb40,6,bfbfdfc0,0,0,0,20,c8524550,c8524550) at kern_select+0x504 select(c853bb40,e987dcec,e987dd28,c0ce9b0f,0,...) at select+0x66 syscallenter(c853bb40,e987dce4,e987dce4,0,0,...) at syscallenter+0x263 syscall(e987dd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x281aa1f3, esp = 0xbfbfdf7c, ebp = 0xbfbfee08 --- Tracing command moused pid 1483 tid 100079 td 0xc7c80870 sched_switch(c7c80870,0,104,191,4bf34a7b,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7c80870,0,c0ce92d6,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08acdca,c72ef650,0,c0ce325f,c7c80870,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c72ef664,0,e9736a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c72ef664,c72ef650,c0ceb69b,627,c84067a8,...) at _cv_wait_sig+0x243 seltdwait(c84067a8,58,c6d7be80,c7c80870,0,...) at seltdwait+0xa2 kern_select(c7c80870,6,bfbfea24,0,0,0,20,c6f7b7f8,c6f7b7f8) at kern_select+0x504 select(c7c80870,e9736cec,e9736d28,c0ce9b0f,0,...) at select+0x66 syscallenter(c7c80870,e9736ce4,e9736ce4,0,0,...) at syscallenter+0x263 syscall(e9736d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x281b21f3, esp = 0xbfbfe9bc, ebp = 0xbfbfeb48 --- Tracing command cron pid 1436 tid 100149 td 0xc853b2d0 sched_switch(c853b2d0,0,104,191,87cecee4,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c853b2d0,0,c0ce92d6,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(ea61,c08f9e90,c853b2d0,2,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c0e49744,5c,c0ce5d54,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c0e49744,0,15c,c0ce5d54,ea61,...) at _sleep+0x328 kern_nanosleep(c853b2d0,e9886c34,e9886c3c,3c,0,...) at kern_nanosleep+0xc1 nanosleep(c853b2d0,e9886cec,e9886d28,c0ce9b0f,0,...) at nanosleep+0x6f syscallenter(c853b2d0,e9886ce4,e9886ce4,0,0,...) at syscallenter+0x263 syscall(e9886d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x28184543, esp = 0xbfbfec6c, ebp = 0xbfbfec98 --- Tracing command sendmail pid 1429 tid 100137 td 0xc81102d0 sched_switch(c81102d0,0,104,191,16b0c066,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,68,...) at mi_switch+0x200 sleepq_switch(c81102d0,0,c0ce92d6,1a7,68,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce92d6,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c85265a8,68,c0c953f3,100,0,...) at sleepq_wait_sig+0x17 _sleep(c85265a8,c85265d8,168,c0c953f3,0,...) at _sleep+0x35c kern_sigsuspend(c81102d0,0,0,0,0,...) at kern_sigsuspend+0xba sigsuspend(c81102d0,e9860cec,e9860d28,c0ce9b0f,0,...) at sigsuspend+0x4d syscallenter(c81102d0,e9860ce4,e9860ce4,0,0,...) at syscallenter+0x263 syscall(e9860d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x2833787b, esp = 0xbfbfcfbc, ebp = 0xbfbfcfe8 --- Tracing command sendmail pid 1423 tid 100091 td 0xc7ccf5a0 sched_switch(c7ccf5a0,0,104,191,2e1ae674,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7ccf5a0,0,c0ce92d6,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(e9766a14,c08acdca,c7c17310,0,c7ccf5a0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c7c17324,0,e9766a4c,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c7c17324,c7c17310,1389,627,c8406a10,...) at _cv_timedwait_sig+0x252 seltdwait(e9766bf8,e9766c00,c8527500,c7ccf5a0,c15aea94,...) at seltdwait+0x8a kern_select(c7ccf5a0,5,bfbfc530,0,0,e9766c40,20,5,0) at kern_select+0x504 select(c7ccf5a0,e9766cec,e9766d28,c0ce9b0f,0,...) at select+0x66 syscallenter(c7ccf5a0,e9766ce4,e9766ce4,0,0,...) at syscallenter+0x263 syscall(e9766d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x283e01f3, esp = 0xbfbfc49c, ebp = 0xbfbfcfc8 --- Tracing command sshd pid 1415 tid 100122 td 0xc8115870 sched_switch(c8115870,0,104,191,54f9b0ea,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c8115870,0,c0ce92d6,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08acdca,c7c9ab10,0,c0ce325f,c8115870,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7c9ab24,0,e982aa4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7c9ab24,c7c9ab10,c0ceb69b,627,c8406e00,...) at _cv_wait_sig+0x243 seltdwait(c8406e00,58,c7cf4a00,c8115870,c0d11f5e,...) at seltdwait+0xa2 kern_select(c8115870,5,2880c0b0,0,0,0,20,c810e7f8,c810e7f8) at kern_select+0x504 select(c8115870,e982acec,2880d070,1,0,...) at select+0x66 syscallenter(c8115870,e982ace4,c0c04f1d,c0e48eb0,0,...) at syscallenter+0x263 syscall(e982ad28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x284b01f3, esp = 0xbfbfe27c, ebp = 0xbfbfee18 --- Tracing command ntpd pid 1370 tid 100104 td 0xc7cc8b40 sched_switch(c7cc8b40,0,104,191,a8bc9148,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7cc8b40,0,c0ce92d6,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08acdca,c72efdd0,0,c0ce325f,c7cc8b40,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c72efde4,0,e979ca4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c72efde4,c72efdd0,c0ceb69b,627,c7c5a460,...) at _cv_wait_sig+0x243 seltdwait(c7c5a460,58,c6d7be80,c7cc8b40,0,...) at seltdwait+0xa2 kern_select(c7cc8b40,1c,bfbfed08,0,0,0,20,c7c8b7f8,c7c8b7f8) at kern_select+0x504 select(c7cc8b40,e979ccec,e979cd80,e979cc88,0,...) at select+0x66 syscallenter(c7cc8b40,e979cce4,c0c04bd6,fffffffe,246,...) at syscallenter+0x263 syscall(e979cd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x283581f3, esp = 0xbfbfecdc, ebp = 0xbfbfeda8 --- Tracing command nfsd pid 1234 tid 100141 td 0xc853db40 sched_switch(c853db40,0,104,191,981dfc95,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c853db40,0,c0ce92d6,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(e986ebe0,c08acdca,c6fe5c80,0,c853db40,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c8185250,0,e986ec18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c8185250,c6fe5c80,1388,3b1,0,...) at _cv_timedwait_sig+0x252 svc_run_internal(e986ed14,c0892338,c6fe5c80,e986ed28,c0cdfcac,...) at svc_run_internal+0x356 svc_thread_start(c6fe5c80,e986ed28,c0cdfcac,349,c7c52d48,...) at svc_thread_start+0x10 fork_exit(c0abbda0,c6fe5c80,e986ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x804c72e, eip = 0xc, esp = 0x33, ebp = 0 --- Tracing command nfsd pid 1234 tid 100140 td 0xc800d870 sched_switch(c800d870,0,104,191,981ab37e,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c800d870,0,c0ce92d6,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(e986bbe0,c08acdca,c6fe5c80,0,c800d870,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c8185290,0,e986bc18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c8185290,c6fe5c80,1388,3b1,0,...) at _cv_timedwait_sig+0x252 svc_run_internal(e986bd14,c0892338,c6fe5c80,e986bd28,c0cdfcac,...) at svc_run_internal+0x356 svc_thread_start(c6fe5c80,e986bd28,c0cdfcac,349,c7c52d48,...) at svc_thread_start+0x10 fork_exit(c0abbda0,c6fe5c80,e986bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x804c72e, eip = 0xc, esp = 0x33, ebp = 0 --- Tracing command nfsd pid 1234 tid 100139 td 0xc800db40 sched_switch(c800db40,0,104,191,981f58e7,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c800db40,0,c0ce92d6,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(e9868be0,c08acdca,c6fe5c80,0,c800db40,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c81852d0,0,e9868c18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c81852d0,c6fe5c80,1388,3b1,0,...) at _cv_timedwait_sig+0x252 svc_run_internal(e9868d14,c0892338,c6fe5c80,e9868d28,c0cdfcac,...) at svc_run_internal+0x356 svc_thread_start(c6fe5c80,e9868d28,c0cdfcac,349,c7c52d48,...) at svc_thread_start+0x10 fork_exit(c0abbda0,c6fe5c80,e9868d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x804c72e, eip = 0xc, esp = 0x33, ebp = 0 --- Tracing command nfsd pid 1234 tid 100097 td 0xc7c7e870 sched_switch(c7c7e870,0,104,191,981ea3f8,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7c7e870,0,c0ce92d6,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(e977eab0,c08acdca,c6fe5c80,0,c7c7e870,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c6f60950,0,e977eae8,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c6f60950,c6fe5c80,1388,3b1,e977eb30,...) at _cv_timedwait_sig+0x252 svc_run_internal(c7c7ea18,14,c0d09584,c0d0804a,e977ec0c,...) at svc_run_internal+0x356 svc_run(c6fe5c80,0,c0d08325,1c2,0,...) at svc_run+0x7b nfssvc_nfsd(bfbfe8a0,e977ec0c,c,c6d7be80,e977ec20,...) at nfssvc_nfsd+0xad nfssvc_nfsserver(c7c7e870,e977ecec,c7c52d48,0,c7c7e870,...) at nfssvc_nfsserver+0x24f nfssvc(c7c7e870,e977ecec,e977ed28,c0ce9b0f,0,...) at nfssvc+0x83 syscallenter(c7c7e870,e977ece4,e977ece4,0,0,...) at syscallenter+0x263 syscall(e977ed28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (155, FreeBSD ELF32, nfssvc), eip = 0x280dc47b, esp = 0xbfbfe85c, ebp = 0xbfbfeac8 --- Tracing command nfsd pid 1233 tid 100138 td 0xc8110000 sched_switch(c8110000,0,104,191,e58e8fe4,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c8110000,0,c0ce92d6,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08acdca,c7c9a750,0,c0ce325f,c8110000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7c9a764,0,e9865a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7c9a764,c7c9a750,c0ceb69b,627,c80f8850,...) at _cv_wait_sig+0x243 seltdwait(c80f8850,58,c6d7be80,c8110000,c0d11f5e,...) at seltdwait+0xa2 kern_select(c8110000,5,bfbfecd0,0,0,0,20,c85262a8,c85262a8) at kern_select+0x504 select(c8110000,e9865cec,bfbffff4,1,0,...) at select+0x66 syscallenter(c8110000,e9865ce4,c0c04f1d,c0e48eb0,0,...) at syscallenter+0x263 syscall(e9865d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x281841f3, esp = 0xbfbfeacc, ebp = 0xbfbfee08 --- Tracing command mountd pid 1224 tid 100125 td 0xc8115000 sched_switch(c8115000,0,104,191,3a0d4c1f,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c8115000,0,c0ce92d6,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08acdca,c72ef9d0,0,c0ce325f,c8115000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c72ef9e4,0,e9833a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c72ef9e4,c72ef9d0,c0ceb69b,627,c7cdf578,...) at _cv_wait_sig+0x243 seltdwait(c7cdf578,58,c6d7be80,c8115000,c0f89408,...) at seltdwait+0xa2 kern_select(c8115000,9,bfbfed5c,0,0,0,20,c810e000,c810e000) at kern_select+0x504 select(c8115000,e9833cec,e9833d28,c0ce9b0f,0,...) at select+0x66 syscallenter(c8115000,e9833ce4,e9833ce4,0,246,...) at syscallenter+0x263 syscall(e9833d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x2819a1f3, esp = 0xbfbfed2c, ebp = 0xbfbfedf8 --- Tracing command rpcbind pid 1127 tid 100131 td 0xc8110b40 sched_switch(c8110b40,0,104,191,b5ef6b04,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c8110b40,0,c0ce92d6,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(e9845a6c,c08acdca,c72ef6d0,0,c8110b40,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c72ef6e4,0,e9845aa4,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c72ef6e4,c72ef6d0,7531,627,e9845b5c,...) at _cv_timedwait_sig+0x252 seltdwait(e9845c2c,e9845c34,535,c8110b40,e9845b2c,...) at seltdwait+0x8a poll(c8110b40,e9845cec,e9845d28,c0ce9b0f,0,...) at poll+0x300 syscallenter(c8110b40,e9845ce4,e9845ce4,0,ec,...) at syscallenter+0x263 syscall(e9845d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, poll), eip = 0x2814007b, esp = 0xbfbfcbcc, ebp = 0xbfbfed98 --- Tracing command syslogd pid 1040 tid 100077 td 0xc704e870 sched_switch(c704e870,0,104,191,7c58d529,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c704e870,0,c0ce92d6,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08acdca,c72ef5d0,0,c0ce325f,c704e870,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c72ef5e4,0,e972ea4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c72ef5e4,c72ef5d0,c0ceb69b,627,c81711f8,...) at _cv_wait_sig+0x243 seltdwait(c81711f8,58,c6d7be80,c704e870,0,...) at seltdwait+0xa2 kern_select(c704e870,9,2847f0ac,0,0,0,20,c6f7bd48,c6f7bd48) at kern_select+0x504 select(c704e870,e972ecec,e972ed28,c0ce9b0f,0,...) at select+0x66 syscallenter(c704e870,e972ece4,e972ece4,0,246,...) at syscallenter+0x263 syscall(e972ed28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x281991f3, esp = 0xbfbfe2bc, ebp = 0xbfbfee08 --- Tracing command devd pid 839 tid 100095 td 0xc7c7eb40 sched_switch(c7c7eb40,0,104,191,399953ef,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7c7eb40,0,c0ce92d6,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08acdca,c72ef450,0,c0ce325f,c7c7eb40,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c72ef464,0,e9776a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c72ef464,c72ef450,c0ceb69b,627,c7c5a8c0,...) at _cv_wait_sig+0x243 seltdwait(c7c5a8c0,58,c6d7be80,c7c7eb40,3,...) at seltdwait+0xa2 kern_select(c7c7eb40,6,bfbfe980,0,0,0,20,c7c8a000,c7c8a000) at kern_select+0x504 select(c7c7eb40,e9776cec,e9776d28,c0ce9b0f,0,...) at select+0x66 syscallenter(c7c7eb40,e9776ce4,e9776ce4,0,ec,...) at syscallenter+0x263 syscall(e9776d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x808b793, esp = 0xbfbfe94c, ebp = 0xbfbfee28 --- Tracing command flowcleaner pid 21 tid 100075 td 0xc704eb40 sched_switch(c704eb40,0,104,191,db6b6a19,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c704eb40,0,c0ce92d6,28b,c704eb40,...) at sleepq_switch+0x15f sleepq_timedwait(c0fb68a8,0,e753acac,1,0,...) at sleepq_timedwait+0x6b _cv_timedwait(c0fb68a8,c0fb68b0,4e20,620,c704eb40,...) at _cv_timedwait+0x252 flowtable_cleaner(0,e753ad28,c0cdfcac,349,c7bf4000,...) at flowtable_cleaner+0x1ae fork_exit(c096be40,0,e753ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe753ad60, ebp = 0 --- Tracing command softdepflush pid 20 tid 100074 td 0xc7161000 kdb_enter(c0ce4ad6,c0ce4ad6,c0d0c4cd,e7537a74,1,...) at kdb_enter+0x3a panic(c0d0c4cd,ca308a00,c0d0e420,109,ca308a00,...) at panic+0x134 worklist_remove(c0fc2018,0,c0d0c39c,21fc,0,...) at worklist_remove+0x70 softdep_setup_blkfree(c7c155a8,dae84ce0,12640,0,8,...) at softdep_setup_blkfree+0x5e ffs_blkfree(ca65ea00,c7fcb000,ca6bc6cc,12640,0,...) at ffs_blkfree+0x6fd indir_trunc(fffffff4,ffffffff,c83cc400,cab2a800,ca65ea00,...) at indir_trunc+0x50f handle_workitem_indirblk(4,c0ce325f,df,cab2a800,cab2a800,...) at handle_workitem_indirblk+0x67 handle_workitem_freeblocks(0,e7537c74,2,5dd,1e0,...) at handle_workitem_freeblocks+0x95 process_worklist_item(c0fc2018,0,c0d0c39c,54b,c7161000,...) at process_worklist_item+0x21c softdep_process_worklist(c7c155a8,0,c0d0c39c,4db,3e8,...) at softdep_process_worklist+0x8c softdep_flush(0,e7537d28,c0cdfcac,349,c7bf42a8,...) at softdep_flush+0x2a0 fork_exit(c0b03680,0,e7537d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7537d60, ebp = 0 --- Tracing command vnlru pid 19 tid 100073 td 0xc71612d0 sched_switch(c71612d0,0,104,191,ea9beeb0,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,50,...) at mi_switch+0x200 sleepq_switch(c71612d0,0,c0ce92d6,28b,0,...) at sleepq_switch+0x15f sleepq_timedwait(c7bf4550,50,c0cf3562,0,0,...) at sleepq_timedwait+0x6b _sleep(c7bf4550,c0fb6678,250,c0cf3562,3e8,...) at _sleep+0x342 vnlru_proc(0,e7534d28,c0cdfcac,349,c7bf4550,...) at vnlru_proc+0xe7 fork_exit(c0956560,0,e7534d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7534d60, ebp = 0 --- Tracing command syncer pid 18 tid 100072 td 0xc71615a0 cpustop_handler(8,e7531af8,c0c047d6,c0d22c63,ec,...) at cpustop_handler+0x34 ipi_nmi_handler(c0d22c63,ec,0,c08acfa6,c7bf47f8,...) at ipi_nmi_handler+0x2f trap(e7531b04) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc08acfa6, esp = 0xe7531b44, ebp = 0xe7531b60 --- _mtx_lock_sleep(c0fc2018,c71615a0,0,c0d0c39c,2cc8,...) at _mtx_lock_sleep+0xb6 _mtx_lock_flags(c0fc2018,0,c0d0c39c,2cc8,c850bbd8,...) at _mtx_lock_flags+0xf7 softdep_get_depcounts(c85ec000,e7531bf8,e7531bf4,53e,c0ce325f,...) at softdep_get_depcounts+0x34 ffs_sync(c85ec000,3,c0cf25ea,d94,c85ec000,...) at ffs_sync+0x10d sync_fsync(e7531c6c,c0d25536,c850bba4,e7531c6c,c850bae0,...) at sync_fsync+0x18f VOP_FSYNC_APV(c0de75c0,e7531c6c,c0cf25ea,6b9,c71615a0,...) at VOP_FSYNC_APV+0xc5 sync_vnode(c0fb66b8,c0fb66a4,3e8,6e8,4e20,...) at sync_vnode+0x16b sched_sync(0,e7531d28,c0cdfcac,349,c7bf47f8,...) at sched_sync+0x282 fork_exit(c0954d50,0,e7531d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7531d60, ebp = 0 --- Tracing command bufdaemon pid 17 tid 100071 td 0xc7161870 sched_switch(c7161870,0,104,191,fd4414f7,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,44,...) at mi_switch+0x200 sleepq_switch(c7161870,0,c0ce92d6,28b,0,...) at sleepq_switch+0x15f sleepq_timedwait(c0fb6348,44,c0cf0885,0,0,...) at sleepq_timedwait+0x6b _sleep(c0fb6348,c0fb6350,44,c0cf0885,3e8,...) at _sleep+0x342 buf_daemon(0,e752ed28,c0cdfcac,349,c7bf4aa0,...) at buf_daemon+0x16e fork_exit(c093c290,0,e752ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe752ed60, ebp = 0 --- Tracing command pagezero pid 16 tid 100070 td 0xc7161b40 sched_switch(c7161b40,0,104,191,48d502c6,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7161b40,0,c0ce92d6,28b,0,...) at sleepq_switch+0x15f sleepq_timedwait(c0fc3edc,0,c0d14a02,0,0,...) at sleepq_timedwait+0x6b _sleep(c0fc3edc,c0fc2980,0,c0d14a02,493e0,...) at _sleep+0x342 vm_pagezero(0,e752bd28,c0cdfcac,349,c7bf4d48,...) at vm_pagezero+0xdc fork_exit(c0b49a10,0,e752bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe752bd60, ebp = 0 --- Tracing command vmdaemon pid 15 tid 100069 td 0xc7162000 sched_switch(c7162000,0,104,191,b84f6976,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,68,...) at mi_switch+0x200 sleepq_switch(c7162000,0,c0ce92d6,268,0,...) at sleepq_switch+0x15f sleepq_wait(c0fc3ad0,68,c0cf0885,0,0,...) at sleepq_wait+0x63 _sleep(c0fc3ad0,c0fc3ad4,68,c0cf0885,0,...) at _sleep+0x372 vm_daemon(0,e7528d28,c0cdfcac,349,c6d9a2a8,...) at vm_daemon+0x59 fork_exit(c0b43890,0,e7528d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7528d60, ebp = 0 --- Tracing command pagedaemon pid 9 tid 100068 td 0xc71622d0 sched_switch(c71622d0,0,104,191,eb24ba58,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,44,...) at mi_switch+0x200 sleepq_switch(c71622d0,0,c0ce92d6,28b,0,...) at sleepq_switch+0x15f sleepq_timedwait(c0fc3a98,44,c0cf0885,0,0,...) at sleepq_timedwait+0x6b _sleep(c0fc3a98,c0fc2980,44,c0cf0885,1388,...) at _sleep+0x342 vm_pageout(0,e7525d28,c0cdfcac,349,c6d9a550,...) at vm_pageout+0x2c7 fork_exit(c0b44900,0,e7525d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7525d60, ebp = 0 --- Tracing command xpt_thrd pid 8 tid 100067 td 0xc71625a0 sched_switch(c71625a0,0,104,191,b280f5cd,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,4c,...) at mi_switch+0x200 sleepq_switch(c71625a0,0,c0ce92d6,268,0,...) at sleepq_switch+0x15f sleepq_wait(c0e153d4,4c,c0c7b80d,0,0,...) at sleepq_wait+0x63 _sleep(c0e153d4,c0e15418,4c,c0c7b80d,0,...) at _sleep+0x372 xpt_scanner_thread(0,e7522d28,c0cdfcac,349,c6d9a7f8,...) at xpt_scanner_thread+0x47 fork_exit(c0485c70,0,e7522d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7522d60, ebp = 0 --- Tracing command sctp_iterator pid 7 tid 100066 td 0xc7162870 sched_switch(c7162870,0,104,191,de5dd6d4,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7162870,0,c0ce92d6,268,0,...) at sleepq_switch+0x15f sleepq_wait(c0fb80d8,0,c0cff380,0,0,...) at sleepq_wait+0x63 _sleep(c0fb80d8,c0fb80a0,0,c0cff380,0,...) at _sleep+0x372 sctp_iterator_thread(0,e751fd28,c0cdfcac,349,c6d9aaa0,...) at sctp_iterator_thread+0x5c fork_exit(c09e5eb0,0,e751fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe751fd60, ebp = 0 --- Tracing command fdc0 pid 6 tid 100062 td 0xc70245a0 sched_switch(c70245a0,0,104,191,f4433049,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,4c,...) at mi_switch+0x200 sleepq_switch(c70245a0,0,c0ce92d6,28b,0,...) at sleepq_switch+0x15f sleepq_timedwait(c705603c,4c,c0cd98ed,0,0,...) at sleepq_timedwait+0x6b _sleep(c705603c,c70560f0,4c,c0cd98ed,3e8,...) at _sleep+0x342 fdc_thread(c7056000,e7513d28,c0cdfcac,349,c6d9ad48,...) at fdc_thread+0x27d fork_exit(c0bc8590,c7056000,e7513d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7513d60, ebp = 0 --- Tracing command fw0_probe pid 5 tid 100060 td 0xc7024b40 sched_switch(c7024b40,0,104,191,baba734c,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c7024b40,0,c0ce92d6,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce92d6,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7059000,5c,c0cd98ed,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7059000,c705d488,15c,c0cd98ed,0,...) at _sleep+0x35c fw_bus_probe_thread(c7059000,e7503d28,c0cdfcac,349,c6f7b000,...) at fw_bus_probe_thread+0xa08 fork_exit(c067b970,c7059000,e7503d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7503d60, ebp = 0 --- Tracing command usb pid 14 tid 100056 td 0xc704c870 sched_switch(c704c870,0,104,191,a853e2eb,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c704c870,0,c0ce92d6,268,c704c870,...) at sleepq_switch+0x15f sleepq_wait(c7034d34,0,e74f0cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7034d34,c7034dd4,c0cd2114,6c,c7034d3c,...) at _cv_wait+0x243 usb_process(c7034d2c,e74f0d28,c0cdfcac,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e7390,c7034d2c,e74f0d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74f0d60, ebp = 0 --- Tracing command usb pid 14 tid 100055 td 0xc704cb40 sched_switch(c704cb40,0,104,191,d7694801,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c704cb40,0,c0ce92d6,268,c704cb40,...) at sleepq_switch+0x15f sleepq_wait(c7034d04,0,e74edcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7034d04,c7034dd4,c0cd2114,6c,c7034d0c,...) at _cv_wait+0x243 usb_process(c7034cfc,e74edd28,c0cdfcac,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e7390,c7034cfc,e74edd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74edd60, ebp = 0 --- Tracing command usb pid 14 tid 100054 td 0xc704e000 sched_switch(c704e000,0,104,191,a7ed11ed,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c704e000,0,c0ce92d6,268,c704e000,...) at sleepq_switch+0x15f sleepq_wait(c7034cd4,0,e74eacac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7034cd4,c7034dd4,c0cd2114,6c,c7034cdc,...) at _cv_wait+0x243 usb_process(c7034ccc,e74ead28,c0cdfcac,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e7390,c7034ccc,e74ead28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74ead60, ebp = 0 --- Tracing command usb pid 14 tid 100053 td 0xc704e2d0 sched_switch(c704e2d0,0,104,191,a7ed00d1,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c704e2d0,0,c0ce92d6,268,c704e2d0,...) at sleepq_switch+0x15f sleepq_wait(c7034ca4,0,e74e7cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7034ca4,c7034dd4,c0cd2114,6c,c7034cac,...) at _cv_wait+0x243 usb_process(c7034c9c,e74e7d28,c0cdfcac,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e7390,c7034c9c,e74e7d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74e7d60, ebp = 0 --- Tracing command usb pid 14 tid 100051 td 0xc7022000 sched_switch(c7022000,0,104,191,a7ecea57,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7022000,0,c0ce92d6,268,c7022000,...) at sleepq_switch+0x15f sleepq_wait(c702db5c,0,e74decac,1,0,...) at sleepq_wait+0x63 _cv_wait(c702db5c,c702dbfc,c0cd2114,6c,c702db64,...) at _cv_wait+0x243 usb_process(c702db54,e74ded28,c0cdfcac,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e7390,c702db54,e74ded28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74ded60, ebp = 0 --- Tracing command usb pid 14 tid 100050 td 0xc70222d0 sched_switch(c70222d0,0,104,191,ce4a1cf4,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c70222d0,0,c0ce92d6,268,c70222d0,...) at sleepq_switch+0x15f sleepq_wait(c702db2c,0,e74dbcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c702db2c,c702dbfc,c0cd2114,6c,c702db34,...) at _cv_wait+0x243 usb_process(c702db24,e74dbd28,c0cdfcac,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e7390,c702db24,e74dbd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74dbd60, ebp = 0 --- Tracing command usb pid 14 tid 100049 td 0xc70225a0 sched_switch(c70225a0,0,104,191,a788a31d,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c70225a0,0,c0ce92d6,268,c70225a0,...) at sleepq_switch+0x15f sleepq_wait(c702dafc,0,e74d8cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c702dafc,c702dbfc,c0cd2114,6c,c702db04,...) at _cv_wait+0x243 usb_process(c702daf4,e74d8d28,c0cdfcac,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e7390,c702daf4,e74d8d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74d8d60, ebp = 0 --- Tracing command usb pid 14 tid 100048 td 0xc7022870 sched_switch(c7022870,0,104,191,a7888f48,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7022870,0,c0ce92d6,268,c7022870,...) at sleepq_switch+0x15f sleepq_wait(c702dacc,0,e74d5cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c702dacc,c702dbfc,c0cd2114,6c,c702dad4,...) at _cv_wait+0x243 usb_process(c702dac4,e74d5d28,c0cdfcac,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e7390,c702dac4,e74d5d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74d5d60, ebp = 0 --- Tracing command usb pid 14 tid 100047 td 0xc7022b40 sched_switch(c7022b40,0,104,191,a78879d4,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7022b40,0,c0ce92d6,268,c7022b40,...) at sleepq_switch+0x15f sleepq_wait(c7026b5c,0,e74d1cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7026b5c,c7026bfc,c0cd2114,6c,c7026b64,...) at _cv_wait+0x243 usb_process(c7026b54,e74d1d28,c0cdfcac,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e7390,c7026b54,e74d1d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74d1d60, ebp = 0 --- Tracing command usb pid 14 tid 100046 td 0xc7023000 sched_switch(c7023000,0,104,191,cb122a2d,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7023000,0,c0ce92d6,268,c7023000,...) at sleepq_switch+0x15f sleepq_wait(c7026b2c,0,e74cecac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7026b2c,c7026bfc,c0cd2114,6c,c7026b34,...) at _cv_wait+0x243 usb_process(c7026b24,e74ced28,c0cdfcac,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e7390,c7026b24,e74ced28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74ced60, ebp = 0 --- Tracing command usb pid 14 tid 100045 td 0xc70232d0 sched_switch(c70232d0,0,104,191,a12ba837,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c70232d0,0,c0ce92d6,268,c70232d0,...) at sleepq_switch+0x15f sleepq_wait(c7026afc,0,e74cbcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7026afc,c7026bfc,c0cd2114,6c,c7026b04,...) at _cv_wait+0x243 usb_process(c7026af4,e74cbd28,c0cdfcac,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e7390,c7026af4,e74cbd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74cbd60, ebp = 0 --- Tracing command usb pid 14 tid 100044 td 0xc70235a0 sched_switch(c70235a0,0,104,191,a12b9770,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c70235a0,0,c0ce92d6,268,c70235a0,...) at sleepq_switch+0x15f sleepq_wait(c7026acc,0,e74c8cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7026acc,c7026bfc,c0cd2114,6c,c7026ad4,...) at _cv_wait+0x243 usb_process(c7026ac4,e74c8d28,c0cdfcac,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e7390,c7026ac4,e74c8d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74c8d60, ebp = 0 --- Tracing command usb pid 14 tid 100043 td 0xc7023870 sched_switch(c7023870,0,104,191,a12b8460,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7023870,0,c0ce92d6,268,c7023870,...) at sleepq_switch+0x15f sleepq_wait(c701db5c,0,e74c4cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c701db5c,c701dbfc,c0cd2114,6c,c701db64,...) at _cv_wait+0x243 usb_process(c701db54,e74c4d28,c0cdfcac,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e7390,c701db54,e74c4d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74c4d60, ebp = 0 --- Tracing command usb pid 14 tid 100042 td 0xc7023b40 sched_switch(c7023b40,0,104,191,ca0615d1,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7023b40,0,c0ce92d6,268,c7023b40,...) at sleepq_switch+0x15f sleepq_wait(c701db2c,0,e74c1cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c701db2c,c701dbfc,c0cd2114,6c,c701db34,...) at _cv_wait+0x243 usb_process(c701db24,e74c1d28,c0cdfcac,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e7390,c701db24,e74c1d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74c1d60, ebp = 0 --- Tracing command usb pid 14 tid 100041 td 0xc7024000 sched_switch(c7024000,0,104,191,a0c785c6,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7024000,0,c0ce92d6,268,c7024000,...) at sleepq_switch+0x15f sleepq_wait(c701dafc,0,e74becac,1,0,...) at sleepq_wait+0x63 _cv_wait(c701dafc,c701dbfc,c0cd2114,6c,c701db04,...) at _cv_wait+0x243 usb_process(c701daf4,e74bed28,c0cdfcac,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e7390,c701daf4,e74bed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74bed60, ebp = 0 --- Tracing command usb pid 14 tid 100040 td 0xc7013000 sched_switch(c7013000,0,104,191,a0c7721b,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7013000,0,c0ce92d6,268,c7013000,...) at sleepq_switch+0x15f sleepq_wait(c701dacc,0,e74bbcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c701dacc,c701dbfc,c0cd2114,6c,c701dad4,...) at _cv_wait+0x243 usb_process(c701dac4,e74bbd28,c0cdfcac,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e7390,c701dac4,e74bbd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74bbd60, ebp = 0 --- Tracing command usb pid 14 tid 100038 td 0xc70135a0 sched_switch(c70135a0,0,104,191,a0c75bb2,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c70135a0,0,c0ce92d6,268,c70135a0,...) at sleepq_switch+0x15f sleepq_wait(c7017b5c,0,e74b4cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7017b5c,c7017bfc,c0cd2114,6c,c7017b64,...) at _cv_wait+0x243 usb_process(c7017b54,e74b4d28,c0cdfcac,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e7390,c7017b54,e74b4d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74b4d60, ebp = 0 --- Tracing command usb pid 14 tid 100037 td 0xc7013870 sched_switch(c7013870,0,104,191,c8639126,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7013870,0,c0ce92d6,268,c7013870,...) at sleepq_switch+0x15f sleepq_wait(c7017b2c,0,e74b1cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7017b2c,c7017bfc,c0cd2114,6c,c7017b34,...) at _cv_wait+0x243 usb_process(c7017b24,e74b1d28,c0cdfcac,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e7390,c7017b24,e74b1d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74b1d60, ebp = 0 --- Tracing command usb pid 14 tid 100036 td 0xc7013b40 sched_switch(c7013b40,0,104,191,a062854c,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7013b40,0,c0ce92d6,268,c7013b40,...) at sleepq_switch+0x15f sleepq_wait(c7017afc,0,e74aecac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7017afc,c7017bfc,c0cd2114,6c,c7017b04,...) at _cv_wait+0x243 usb_process(c7017af4,e74aed28,c0cdfcac,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e7390,c7017af4,e74aed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74aed60, ebp = 0 --- Tracing command usb pid 14 tid 100035 td 0xc7015000 sched_switch(c7015000,0,104,191,a06272ad,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7015000,0,c0ce92d6,268,c7015000,...) at sleepq_switch+0x15f sleepq_wait(c7017acc,0,e74abcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7017acc,c7017bfc,c0cd2114,6c,c7017ad4,...) at _cv_wait+0x243 usb_process(c7017ac4,e74abd28,c0cdfcac,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e7390,c7017ac4,e74abd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74abd60, ebp = 0 --- Tracing command usb pid 14 tid 100033 td 0xc70155a0 sched_switch(c70155a0,0,104,191,a0625e25,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c70155a0,0,c0ce92d6,268,c70155a0,...) at sleepq_switch+0x15f sleepq_wait(c7007b5c,0,e74a4cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7007b5c,c7007bfc,c0cd2114,6c,c7007b64,...) at _cv_wait+0x243 usb_process(c7007b54,e74a4d28,c0cdfcac,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e7390,c7007b54,e74a4d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74a4d60, ebp = 0 --- Tracing command usb pid 14 tid 100032 td 0xc7015870 sched_switch(c7015870,0,104,191,c6c77d71,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7015870,0,c0ce92d6,268,c7015870,...) at sleepq_switch+0x15f sleepq_wait(c7007b2c,0,e74a1cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7007b2c,c7007bfc,c0cd2114,6c,c7007b34,...) at _cv_wait+0x243 usb_process(c7007b24,e74a1d28,c0cdfcac,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e7390,c7007b24,e74a1d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74a1d60, ebp = 0 --- Tracing command usb pid 14 tid 100031 td 0xc7015b40 sched_switch(c7015b40,0,104,191,9ffc4520,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7015b40,0,c0ce92d6,268,c7015b40,...) at sleepq_switch+0x15f sleepq_wait(c7007afc,0,e749ecac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7007afc,c7007bfc,c0cd2114,6c,c7007b04,...) at _cv_wait+0x243 usb_process(c7007af4,e749ed28,c0cdfcac,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e7390,c7007af4,e749ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe749ed60, ebp = 0 --- Tracing command usb pid 14 tid 100030 td 0xc6eb82d0 sched_switch(c6eb82d0,0,104,191,9ffc3094,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c6eb82d0,0,c0ce92d6,268,c6eb82d0,...) at sleepq_switch+0x15f sleepq_wait(c7007acc,0,e749bcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7007acc,c7007bfc,c0cd2114,6c,c7007ad4,...) at _cv_wait+0x243 usb_process(c7007ac4,e749bd28,c0cdfcac,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e7390,c7007ac4,e749bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe749bd60, ebp = 0 --- Tracing command yarrow pid 13 tid 100017 td 0xc6eb75a0 sched_switch(c6eb75a0,0,104,191,fdeeddc4,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c6eb75a0,0,c0ce92d6,28b,2,...) at sleepq_switch+0x15f sleepq_timedwait(c0e49584,0,c0cd98ed,2,0,...) at sleepq_timedwait+0x6b _sleep(c0e49584,0,0,c0cd98ed,64,...) at _sleep+0x342 pause(c0cd98ed,64,c0cc5a37,111,0,...) at pause+0x47 random_kthread(0,c6b45d28,c0cdfcac,349,c6f7b550,...) at random_kthread+0x1ef fork_exit(c0758ed0,0,c6b45d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b45d60, ebp = 0 --- Tracing command g_down pid 4 tid 100015 td 0xc6eb7b40 sched_switch(c6eb7b40,0,104,191,fd46374a,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,4c,...) at mi_switch+0x200 sleepq_switch(c6eb7b40,0,c0ce92d6,268,0,...) at sleepq_switch+0x15f sleepq_wait(c0e47244,4c,c0cd98ed,0,0,...) at sleepq_wait+0x63 _sleep(c0e47244,c0e471a8,24c,c0cd98ed,0,...) at _sleep+0x372 g_io_schedule_down(c6eb7b40,0,c0cdb04a,74,0,...) at g_io_schedule_down+0x5b g_down_procbody(0,c6b3fd28,c0cdfcac,349,c6d99000,...) at g_down_procbody+0x8d fork_exit(c0856730,0,c6b3fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b3fd60, ebp = 0 --- Tracing command g_up pid 3 tid 100014 td 0xc6eb8000 sched_switch(c6eb8000,0,104,191,fd49a9c4,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,4c,...) at mi_switch+0x200 sleepq_switch(c6eb8000,0,c0ce92d6,268,0,...) at sleepq_switch+0x15f sleepq_wait(c0e47240,4c,c0cd98ed,0,0,...) at sleepq_wait+0x63 _sleep(c0e47240,c0e471c8,24c,c0cd98ed,0,...) at _sleep+0x372 g_io_schedule_up(c6eb8000,0,c0cdb04a,5d,0,...) at g_io_schedule_up+0x134 g_up_procbody(0,c6b3cd28,c0cdfcac,349,c6d992a8,...) at g_up_procbody+0x8d fork_exit(c08567c0,0,c6b3cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b3cd60, ebp = 0 --- Tracing command g_event pid 2 tid 100013 td 0xc6da12d0 sched_switch(c6da12d0,0,104,191,f6f1d364,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,4c,...) at mi_switch+0x200 sleepq_switch(c6da12d0,0,c0ce92d6,28b,0,...) at sleepq_switch+0x15f sleepq_timedwait(c0e47238,4c,c0cd98ed,0,0,...) at sleepq_timedwait+0x6b _sleep(c0e47238,0,4c,c0cd98ed,64,...) at _sleep+0x342 g_event_procbody(0,c6b39d28,c0cdfcac,349,c6d99550,...) at g_event_procbody+0xcb fork_exit(c0856850,0,c6b39d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b39d60, ebp = 0 --- Tracing command intr pid 12 tid 100064 td 0xc7164000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100063 td 0xc70242d0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100061 td 0xc7024870 sched_switch(c7024870,0,109,191,aa6c287,...) at sched_switch+0x3bc mi_switch(109,0,c0cdff2b,52d,c6de1470,...) at mi_switch+0x200 ithread_loop(c6de82c0,e7506d28,c0cdfcac,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0895330,c6de82c0,e7506d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7506d60, ebp = 0 --- Tracing command intr pid 12 tid 100058 td 0xc704c2d0 sched_switch(c704c2d0,0,109,191,9ff7d74e,...) at sched_switch+0x3bc mi_switch(109,0,c0cdff2b,52d,c6de0570,...) at mi_switch+0x200 ithread_loop(c6de83a0,e74fad28,c0cdfcac,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0895330,c6de83a0,e74fad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74fad60, ebp = 0 --- Tracing command intr pid 12 tid 100057 td 0xc704c5a0 sched_switch(c704c5a0,0,109,191,ad90653a,...) at sched_switch+0x3bc mi_switch(109,0,c0cdff2b,52d,c6de0870,...) at mi_switch+0x200 ithread_loop(c6de83f0,e74f6d28,c0cdfcac,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0895330,c6de83f0,e74f6d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74f6d60, ebp = 0 --- Tracing command intr pid 12 tid 100052 td 0xc704e5a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100039 td 0xc70132d0 sched_switch(c70132d0,0,109,191,c5dc25ea,...) at sched_switch+0x3bc mi_switch(109,0,c0cdff2b,52d,c6de0670,...) at mi_switch+0x200 ithread_loop(c6de8650,e74b8d28,c0cdfcac,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0895330,c6de8650,e74b8d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74b8d60, ebp = 0 --- Tracing command intr pid 12 tid 100034 td 0xc70152d0 sched_switch(c70152d0,0,109,191,c011b8c4,...) at sched_switch+0x3bc mi_switch(109,0,c0cdff2b,52d,c6de06f0,...) at mi_switch+0x200 ithread_loop(c6de8080,e74a8d28,c0cdfcac,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0895330,c6de8080,e74a8d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74a8d60, ebp = 0 --- Tracing command intr pid 12 tid 100029 td 0xc6eb85a0 sched_switch(c6eb85a0,0,109,191,be22f7d6,...) at sched_switch+0x3bc mi_switch(109,0,c0cdff2b,52d,c6de0770,...) at mi_switch+0x200 ithread_loop(c6de86c0,e7498d28,c0cdfcac,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0895330,c6de86c0,e7498d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7498d60, ebp = 0 --- Tracing command intr pid 12 tid 100028 td 0xc6eb8870 sched_switch(c6eb8870,0,109,191,4897cc1c,...) at sched_switch+0x3bc mi_switch(109,0,c0cdff2b,52d,c6de0470,...) at mi_switch+0x200 ithread_loop(c6de80d0,e7389d28,c0cdfcac,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0895330,c6de80d0,e7389d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7389d60, ebp = 0 --- Tracing command intr pid 12 tid 100027 td 0xc6eb8b40 sched_switch(c6eb8b40,0,109,191,9ff8c078,...) at sched_switch+0x3bc mi_switch(109,0,c0cdff2b,52d,c6de0af0,...) at mi_switch+0x200 ithread_loop(c6de8630,c6b8ad28,c0cdfcac,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0895330,c6de8630,c6b8ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b8ad60, ebp = 0 --- Tracing command intr pid 12 tid 100026 td 0xc6fa3000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100024 td 0xc6fa35a0 sched_switch(c6fa35a0,0,109,191,2820f5ea,...) at sched_switch+0x3bc mi_switch(109,0,c0cdff2b,52d,c6fa20f0,...) at mi_switch+0x200 ithread_loop(c6de8770,c6b5ad28,c0cdfcac,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0895330,c6de8770,c6b5ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b5ad60, ebp = 0 --- Tracing command intr pid 12 tid 100019 td 0xc6eb7000 sched_switch(c6eb7000,0,109,191,b6dba048,...) at sched_switch+0x3bc mi_switch(109,0,c0cdff2b,52d,c6fa2170,...) at mi_switch+0x200 ithread_loop(c6de8780,c6b4bd28,c0cdfcac,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0895330,c6de8780,c6b4bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b4bd60, ebp = 0 --- Tracing command intr pid 12 tid 100018 td 0xc6eb72d0 sched_switch(c6eb72d0,0,109,191,b5455e85,...) at sched_switch+0x3bc mi_switch(109,0,c0cdff2b,52d,c6de0070,...) at mi_switch+0x200 ithread_loop(c6de8790,c6b48d28,c0cdfcac,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0895330,c6de8790,c6b48d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b48d60, ebp = 0 --- Tracing command intr pid 12 tid 100012 td 0xc6da15a0 sched_switch(c6da15a0,0,109,191,e875da2c,...) at sched_switch+0x3bc mi_switch(109,0,c0cdff2b,52d,c6de00f0,...) at mi_switch+0x200 ithread_loop(c6de87a0,c6b36d28,c0cdfcac,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0895330,c6de87a0,c6b36d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b36d60, ebp = 0 --- Tracing command intr pid 12 tid 100011 td 0xc6da1870 sched_switch(c6da1870,0,109,191,fdf13d23,...) at sched_switch+0x3bc mi_switch(109,0,c0cdff2b,52d,c6de0170,...) at mi_switch+0x200 ithread_loop(c6de87b0,c6b33d28,c0cdfcac,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0895330,c6de87b0,c6b33d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b33d60, ebp = 0 --- Tracing command intr pid 12 tid 100010 td 0xc6da1b40 sched_switch(c6da1b40,0,109,191,f6f1afe8,...) at sched_switch+0x3bc mi_switch(109,0,c0cdff2b,52d,c6de01f0,...) at mi_switch+0x200 ithread_loop(c6de87c0,c6b30d28,c0cdfcac,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0895330,c6de87c0,c6b30d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b30d60, ebp = 0 --- Tracing command intr pid 12 tid 100009 td 0xc6de9000 cpustop_handler(1,c6b2db74,c0c047d6,c0e4d840,c0ce66a7,...) at cpustop_handler+0x34 ipi_nmi_handler(c0e4d840,c0ce66a7,453,1ce4,c6d997f8,...) at ipi_nmi_handler+0x2f trap(c6b2db80) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc04fbc73, esp = 0xc6b2dbc0, ebp = 0xc6b2dbc0 --- acpi_timer_read(c0d3a100,c0e49780,c6b2dc08,c08c858d,c0d3a100,...) at acpi_timer_read+0x13 acpi_timer_get_timecount_safe(c0d3a100,c6de9000,c6b2dbec,246,c0e49780,...) at acpi_timer_get_timecount_safe+0x11 binuptime(c6b2dc78,0,c0ce5e6d,1a5,c6de92a4,...) at binuptime+0x3d softclock(c0e49780,c6b2dcb8,c08ac9a4,c0e4d1c0,c6de0238,...) at softclock+0x221 intr_event_execute_handlers(c6d997f8,c6de0200,c0cdff2b,533,c6de0270,...) at intr_event_execute_handlers+0x125 ithread_loop(c6de87d0,c6b2dd28,c0cdfcac,349,c6d997f8,...) at ithread_loop+0xac fork_exit(c0895330,c6de87d0,c6b2dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b2dd60, ebp = 0 --- Tracing command intr pid 12 tid 100008 td 0xc6de92d0 sched_switch(c6de92d0,0,109,191,6175d17e,...) at sched_switch+0x3bc mi_switch(109,0,c0cdff2b,52d,c6de02f0,...) at mi_switch+0x200 ithread_loop(c6de87e0,c6b2ad28,c0cdfcac,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c0895330,c6de87e0,c6b2ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b2ad60, ebp = 0 --- Tracing command intr pid 12 tid 100007 td 0xc6de95a0 fork_trampoline() at fork_trampoline Tracing command idle pid 11 tid 100006 td 0xc6d9b000 sched_switch(c6d9b000,0,108,18c,fd1c1,...) at sched_switch+0x3bc mi_switch(108,0,c0ce66a7,3aa,c6d9b000,...) at mi_switch+0x200 sched_idletd(0,c6b23d28,c0cdfcac,349,c6d99aa0,...) at sched_idletd+0x19f fork_exit(c08e22f0,0,c6b23d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b23d60, ebp = 0 --- Tracing command idle pid 11 tid 100005 td 0xc6d9b2d0 sched_switch(c6d9b2d0,0,108,18c,fd40699c,...) at sched_switch+0x3bc mi_switch(108,0,c0ce66a7,3aa,c6d9b2d0,...) at mi_switch+0x200 sched_idletd(0,c6b20d28,c0cdfcac,349,c6d99aa0,...) at sched_idletd+0x19f fork_exit(c08e22f0,0,c6b20d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b20d60, ebp = 0 --- Tracing command idle pid 11 tid 100004 td 0xc6d9b5a0 cpustop_handler(4,c6b1dbf4,c0c047d6,c6b0dcb4,c6d9b5a0,...) at cpustop_handler+0x34 ipi_nmi_handler(c6b0dcb4,c6d9b5a0,c6b1db7c,46,c6d99aa0,...) at ipi_nmi_handler+0x2f trap(c6b1dc00) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc08ac6d2, esp = 0xc6b1dc40, ebp = 0xc6b1dc70 --- _mtx_assert(c6d9b5a0,0,108,18c,11ae139,...) at _mtx_assert+0x142 mi_switch(108,0,c0ce66a7,3aa,c6d9b5a0,...) at mi_switch+0x200 sched_idletd(0,c6b1dd28,c0cdfcac,349,c6d99aa0,...) at sched_idletd+0x19f fork_exit(c08e22f0,0,c6b1dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b1dd60, ebp = 0 --- Tracing command idle pid 11 tid 100003 td 0xc6d9b870 sched_switch(c6d9b870,0,108,18c,fbfa81e5,...) at sched_switch+0x3bc mi_switch(108,0,c0ce66a7,3aa,c6d9b870,...) at mi_switch+0x200 sched_idletd(0,c6b1ad28,c0cdfcac,349,c6d99aa0,...) at sched_idletd+0x19f fork_exit(c08e22f0,0,c6b1ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b1ad60, ebp = 0 --- Tracing command init pid 1 tid 100002 td 0xc6d9bb40 sched_switch(c6d9bb40,0,104,191,c205ddc3,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c6d9bb40,0,c0ce92d6,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce92d6,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6d99d48,5c,c0cebc35,100,0,...) at sleepq_wait_sig+0x17 _sleep(c6d99d48,c6d99dd0,15c,c0cebc35,0,...) at _sleep+0x35c kern_wait(c6d9bb40,ffffffff,c6b16c44,0,0,...) at kern_wait+0xb66 wait4(c6d9bb40,c6b16cec,c6b16d28,c0ce9b0f,0,...) at wait4+0x3b syscallenter(c6d9bb40,c6b16ce4,c6b16ce4,0,0,...) at syscallenter+0x263 syscall(c6b16d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x8054cd3, esp = 0xbfbfe8dc, ebp = 0xbfbfe8f8 --- Tracing command audit pid 10 tid 100001 td 0xc6da1000 sched_switch(c6da1000,0,104,191,9ff92efa,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c6da1000,0,c0ce92d6,268,c6da1000,...) at sleepq_switch+0x15f sleepq_wait(c0fc1880,0,c6b13c8c,1,0,...) at sleepq_wait+0x63 _cv_wait(c0fc1880,c0fc1864,c0d0a2eb,194,0,...) at _cv_wait+0x243 audit_worker(0,c6b13d28,c0cdfcac,349,c6d9a000,...) at audit_worker+0x84 fork_exit(c0acbf30,0,c6b13d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b13d60, ebp = 0 --- Tracing command kernel pid 0 tid 100065 td 0xc7162b40 sched_switch(c7162b40,0,104,191,f28e97ce,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7162b40,0,c0ce92d6,28b,2,...) at sleepq_switch+0x15f sleepq_timedwait(c0e49584,0,c0cdde53,2,0,...) at sleepq_timedwait+0x6b _sleep(c0e49584,0,0,c0cdde53,bb8,...) at _sleep+0x342 pause(c0cdde53,bb8,120,11e,c0e473a8,...) at pause+0x47 deadlkres(0,e751cd28,c0cdfcac,349,c0e47320,...) at deadlkres+0x302 fork_exit(c0877130,0,e751cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe751cd60, ebp = 0 --- Tracing command kernel pid 0 tid 100059 td 0xc704c000 sched_switch(c704c000,0,104,191,f77c6abe,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c704c000,0,c0ce92d6,268,c6de9000,...) at sleepq_switch+0x15f sleepq_wait(c6eabb40,0,c0ce568c,c0cd98ed,0,...) at sleepq_wait+0x63 msleep_spin(c6eabb40,c6eabb5c,c0cd98ed,0,c0ce325f,...) at msleep_spin+0x21d taskqueue_thread_loop(c705d49c,e7500d28,c0cdfcac,349,c0e47320,...) at taskqueue_thread_loop+0x8f fork_exit(c08fcfb0,c705d49c,e7500d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7500d60, ebp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xc6fa32d0 sched_switch(c6fa32d0,0,104,191,48d5ecfd,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c6fa32d0,0,c0ce92d6,268,0,...) at sleepq_switch+0x15f sleepq_wait(c6eabc80,0,c0cd98ed,0,0,...) at sleepq_wait+0x63 _sleep(c6eabc80,c6eabc9c,0,c0cd98ed,0,...) at _sleep+0x372 taskqueue_thread_loop(c0e5bee8,c6b5dd28,c0cdfcac,349,c0e47320,...) at taskqueue_thread_loop+0xb5 fork_exit(c08fcfb0,c0e5bee8,c6b5dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b5dd60, ebp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xc6fa3870 sched_switch(c6fa3870,0,104,191,a85f5702,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c6fa3870,0,c0ce92d6,268,0,...) at sleepq_switch+0x15f sleepq_wait(c6eabd00,0,c0cd98ed,0,0,...) at sleepq_wait+0x63 _sleep(c6eabd00,c6eabd1c,0,c0cd98ed,0,...) at _sleep+0x372 taskqueue_thread_loop(c0e47c78,c6b57d28,c0cdfcac,349,c0e47320,...) at taskqueue_thread_loop+0xb5 fork_exit(c08fcfb0,c0e47c78,c6b57d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b57d60, ebp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xc6fa3b40 sched_switch(c6fa3b40,0,104,191,a85f44cd,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c6fa3b40,0,c0ce92d6,268,0,...) at sleepq_switch+0x15f sleepq_wait(c6eabd40,0,c0ce568c,c0cd98ed,0,...) at sleepq_wait+0x63 msleep_spin(c6eabd40,c6eabd5c,c0cd98ed,0,c0ce325f,...) at msleep_spin+0x21d taskqueue_thread_loop(c0e182a0,c6b54d28,c0cdfcac,349,c0e47320,...) at taskqueue_thread_loop+0x8f fork_exit(c08fcfb0,c0e182a0,c6b54d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b54d60, ebp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xc6de9870 sched_switch(c6de9870,0,104,191,a85f3462,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c6de9870,0,c0ce92d6,268,0,...) at sleepq_switch+0x15f sleepq_wait(c6eabd40,0,c0ce568c,c0cd98ed,0,...) at sleepq_wait+0x63 msleep_spin(c6eabd40,c6eabd5c,c0cd98ed,0,c0ce325f,...) at msleep_spin+0x21d taskqueue_thread_loop(c0e182a0,c6b51d28,c0cdfcac,349,c0e47320,...) at taskqueue_thread_loop+0x8f fork_exit(c08fcfb0,c0e182a0,c6b51d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b51d60, ebp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xc6de9b40 sched_switch(c6de9b40,0,104,191,a85f20d1,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c6de9b40,0,c0ce92d6,268,0,...) at sleepq_switch+0x15f sleepq_wait(c6eabd40,0,c0ce568c,c0cd98ed,0,...) at sleepq_wait+0x63 msleep_spin(c6eabd40,c6eabd5c,c0cd98ed,0,c0ce325f,...) at msleep_spin+0x21d taskqueue_thread_loop(c0e182a0,c6b4ed28,c0cdfcac,349,c0e47320,...) at taskqueue_thread_loop+0x8f fork_exit(c08fcfb0,c0e182a0,c6b4ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b4ed60, ebp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xc6eb7870 sched_switch(c6eb7870,0,104,191,bd2e98c6,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,0,...) at mi_switch+0x200 sleepq_switch(c6eb7870,0,c0ce92d6,268,0,...) at sleepq_switch+0x15f sleepq_wait(c6eabdc0,0,c0cd98ed,0,0,...) at sleepq_wait+0x63 _sleep(c6eabdc0,c6eabddc,0,c0cd98ed,0,...) at _sleep+0x372 taskqueue_thread_loop(c0e5a960,c6b42d28,c0cdfcac,349,c0e47320,...) at taskqueue_thread_loop+0xb5 fork_exit(c08fcfb0,c0e5a960,c6b42d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b42d60, ebp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xc0e475d0 sched_switch(c0e475d0,0,104,191,ebaef7b6,...) at sched_switch+0x3bc mi_switch(104,0,c0ce92d6,1f3,44,...) at mi_switch+0x200 sleepq_switch(c0e475d0,0,c0ce92d6,28b,0,...) at sleepq_switch+0x15f sleepq_timedwait(c0e47320,44,c0ce6fbb,0,0,...) at sleepq_timedwait+0x6b _sleep(c0e47320,0,44,c0ce6fbb,2710,...) at _sleep+0x342 scheduler(0,141ec00,141ec00,141e000,1425000,...) at scheduler+0x23e mi_startup() at mi_startup+0x96 begin() at begin+0x2c db:0:allt> call doadump Physical memory: 3439 MB Dumping 217 MB: 202 186 170 154 138 122 106 90 74 58 42 26 10 Dump complete = 0xf db:0:doadump> reset cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 1 (kgdb) bt #0 doadump () at pcpu.h:231 #1 0xc04d5b19 in db_fncall (dummy1=0xc08efdba, dummy2=0x0, dummy3=0xffffffff, dummy4=0xe7537718 ",wSç") at ../../../ddb/db_command.c:548 #2 0xc04d5f4f in db_command (last_cmdp=0xc0e1647c, cmd_table=0x0, dopager=0x0) at ../../../ddb/db_command.c:445 #3 0xc04d6004 in db_command_script (command=0xc0e17388 "call doadump") at ../../../ddb/db_command.c:516 #4 0xc04da250 in db_script_exec (scriptname=0xc0e16ce0 "doadump", warnifnotfound=Variable "warnifnotfound" is not available. ) at ../../../ddb/db_script.c:302 #5 0xc04da2e1 in db_run_cmd (addr=0x1, have_addr=0x0, count=0xc0fe51a0, modif=0xe7537850 "") at ../../../ddb/db_script.c:375 #6 0xc04d5f11 in db_command (last_cmdp=0xc0e1647c, cmd_table=0x0, dopager=0x1) at ../../../ddb/db_command.c:445 #7 0xc04d606a in db_command_loop () at ../../../ddb/db_command.c:498 #8 0xc04d7f8d in db_trap (type=0x3, code=0x0) at ../../../ddb/db_main.c:229 #9 0xc08efbee in kdb_trap (type=0x3, code=0x0, tf=0xe75379f4) at ../../../kern/subr_kdb.c:546 #10 0xc0c04e0b in trap (frame=0xe75379f4) at ../../../i386/i386/trap.c:732 #11 0xc0bebc5c in calltrap () at ../../../i386/i386/exception.s:168 #12 0xc08efdba in kdb_enter (why=0xc0ce4ad6 "panic", msg=0xc0ce4ad6 "panic") at cpufunc.h:71 #13 0xc08bcc14 in panic (fmt=0xc0d0c4cd "worklist_remove: %p %s(0x%X) not on list") at ../../../kern/kern_shutdown.c:574 #14 0xc0af5bd0 in worklist_remove (item=0xca308a00, locked=Variable "locked" is not available. ) at ../../../ufs/ffs/ffs_softdep.c:960 #15 0xc0afbdae in softdep_setup_blkfree (mp=0xc7c155a8, bp=0xdae84ce0, blkno=0x12640, frags=0x8, wkhd=0xe7537c00) at ../../../ufs/ffs/ffs_softdep.c:8713 #16 0xc0ae193d in ffs_blkfree (ump=0xca65ea00, fs=0xc7fcb000, devvp=0xca6bc6cc, bno=0x12640, size=0x4000, inum=0x5, dephd=0xe7537c00) at ../../../ufs/ffs/ffs_alloc.c:1962 #17 0xc0afc3ef in indir_trunc (freework=0xc83cc400, dbn=0x7200, lbn=0xfffffffffffffff4) at ../../../ufs/ffs/ffs_softdep.c:6183 #18 0xc0afcad7 in handle_workitem_indirblk (freework=0xc83cc400) at ../../../ufs/ffs/ffs_softdep.c:5919 #19 0xc0afcb85 in handle_workitem_freeblocks (freeblks=0xcab2a800, flags=Variable "flags" is not available. ) at ../../../ufs/ffs/ffs_softdep.c:5984 #20 0xc0afce3c in process_worklist_item (mp=0xc7c155a8, flags=Variable "flags" is not available. ) at ../../../ufs/ffs/ffs_softdep.c:1516 #21 0xc0b0335c in softdep_process_worklist (mp=0xc7c155a8, full=0x0) at ../../../ufs/ffs/ffs_softdep.c:1360 #22 0xc0b03920 in softdep_flush () at ../../../ufs/ffs/ffs_softdep.c:1238 #23 0xc0892338 in fork_exit (callout=0xc0b03680 , arg=0x0, frame=0xe7537d28) at ../../../kern/kern_fork.c:849 #24 0xc0bebcd4 in fork_trampoline () at ../../../i386/i386/exception.s:275 (kgdb) f 14 #14 0xc0af5bd0 in worklist_remove (item=0xca308a00, locked=Variable "locked" is not available. ) at ../../../ufs/ffs/ffs_softdep.c:960 960 panic("worklist_remove: %p %s(0x%X) not on list", (kgdb) l 955 { 956 957 if (locked) 958 mtx_assert(&lk, MA_OWNED); 959 if ((item->wk_state & ONWORKLIST) == 0) 960 panic("worklist_remove: %p %s(0x%X) not on list", 961 item, TYPENAME(item->wk_type), item->wk_state); 962 item->wk_state &= ~ONWORKLIST; 963 LIST_REMOVE(item, wk_list); 964 stack_save(&item->wk_stack_rem); (kgdb) p *item $1 = {wk_list = {le_next = 0x0, le_prev = 0xe7537c00}, wk_mp = 0xc7c155a8, wk_type = 0x13, wk_state = 0x109, wk_stack_rem = {depth = 0x9, pcs = {0xc0af5c39, 0xc0afc16c, 0xc0afcad7, 0xc0afcb85, 0xc0afce3c, 0xc0b0335c, 0xc0b03920, 0xc0892338, 0xc0bebcd4, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}} (kgdb) l *0xc0af5c39 0xc0af5c39 is in worklist_remove (../../../ufs/ffs/ffs_softdep.c:965). 960 panic("worklist_remove: %p %s(0x%X) not on list", 961 item, TYPENAME(item->wk_type), item->wk_state); 962 item->wk_state &= ~ONWORKLIST; 963 LIST_REMOVE(item, wk_list); 964 stack_save(&item->wk_stack_rem); 965 } 966 #endif /* DEBUG */ 967 968 /* 969 * Merge two jsegdeps keeping only the oldest one as newer references (kgdb) l *0xc0afc16c 0xc0afc16c is in indir_trunc (../../../ufs/ffs/ffs_softdep.c:6149). 6144 cnt = 0; 6145 LIST_FOREACH_SAFE(wk, &wkhd, wk_list, wkn) { 6146 if (wk->wk_type != D_JNEWBLK) 6147 continue; 6148 WORKLIST_REMOVE_UNLOCKED(wk); 6149 jnewblk = WK_JNEWBLK(wk); 6150 if (jnewblk->jn_lbn > 0) 6151 i = (jnewblk->jn_lbn - -lbn) / lbnadd; 6152 else 6153 i = (jnewblk->jn_lbn - (lbn + level)) / lbnadd; (kgdb) l *0xc0afcad7 0xc0afcad7 is in handle_workitem_indirblk (../../../ufs/ffs/ffs_softdep.c:5921). 5916 if (freework->fw_off == NINDIR(fs)) 5917 freework_freeblock(freework); 5918 else 5919 indir_trunc(freework, fsbtodb(fs, freework->fw_blkno), 5920 freework->fw_lbn); 5921 } 5922 5923 /* 5924 * Called when a freework structure attached to a cg buf is written. The 5925 * ref on either the parent or the freeblks structure is released and (kgdb) l *0xc0afcb85 0xc0afcb85 is in handle_workitem_freeblocks (../../../ufs/ffs/ffs_softdep.c:5984). 5979 ("handle_workitem_freeblocks: Unknown type %s", 5980 TYPENAME(wk->wk_type))); 5981 WORKLIST_REMOVE_UNLOCKED(wk); 5982 freework = WK_FREEWORK(wk); 5983 if (freework->fw_lbn <= -NDADDR) 5984 handle_workitem_indirblk(freework); 5985 else 5986 freework_freeblock(freework); 5987 } 5988 ACQUIRE_LOCK(&lk); (kgdb) l *0xc0afce3c 0xc0afce3c is in process_worklist_item (../../../ufs/ffs/ffs_softdep.c:1516). 1511 vput(vp); 1512 break; 1513 1514 case D_FREEBLKS: 1515 /* releasing blocks and/or fragments from a file */ 1516 handle_workitem_freeblocks(WK_FREEBLKS(wk), flags & LK_NOWAIT); 1517 break; 1518 1519 case D_FREEFRAG: 1520 /* releasing a fragment when replaced as a file grows */ (kgdb) l *0xc0b0335c 0xc0b0335c is in softdep_process_worklist (../../../ufs/ffs/ffs_softdep.c:1360). 1355 ACQUIRE_LOCK(&lk); 1356 loopcount = 1; 1357 starttime = time_second; 1358 softdep_process_journal(mp, full?MNT_WAIT:0); 1359 while (ump->softdep_on_worklist > 0) { 1360 if ((cnt = process_worklist_item(mp, LK_NOWAIT)) == -1) 1361 break; 1362 else 1363 matchcnt += cnt; 1364 /* (kgdb) l *0xc0b03920 0xc0b03920 is in softdep_flush (../../../ufs/ffs/ffs_softdep.c:1238). 1233 if ((mp->mnt_flag & MNT_SOFTDEP) == 0) 1234 continue; 1235 if (vfs_busy(mp, MBF_NOWAIT | MBF_MNTLSTLOCK)) 1236 continue; 1237 vfslocked = VFS_LOCK_GIANT(mp); 1238 progress += softdep_process_worklist(mp, 0); 1239 ump = VFSTOUFS(mp); 1240 remaining += ump->softdep_on_worklist - 1241 ump->softdep_on_worklist_inprogress; 1242 VFS_UNLOCK_GIANT(vfslocked); (kgdb) l *0xc0892338 0xc0892338 is in fork_exit (../../../kern/kern_fork.c:855). 850 851 /* 852 * Check if a kernel thread misbehaved and returned from its main 853 * function. 854 */ 855 if (p->p_flag & P_KTHREAD) { 856 printf("Kernel thread \"%s\" (pid %d) exited prematurely.\n", 857 td->td_name, p->p_pid); 858 kproc_exit(0); 859 } (kgdb) git diff master diff --git a/sys/ufs/ffs/ffs_inode.c b/sys/ufs/ffs/ffs_inode.c index ffbe16e..ba2813d 100644 --- a/sys/ufs/ffs/ffs_inode.c +++ b/sys/ufs/ffs/ffs_inode.c @@ -582,9 +582,7 @@ ffs_indirtrunc(ip, lbn, dbn, lastbn, level, countp) * block to be kept. -1 indicates the entire * block so we need not calculate the index. */ - factor = 1; - for (i = SINGLE; i < level; i++) - factor *= NINDIR(fs); + factor = lbn_offset(fs, level); last = lastbn; if (lastbn > 0) last /= factor; diff --git a/sys/ufs/ffs/ffs_softdep.c b/sys/ufs/ffs/ffs_softdep.c index ca4ea0b..427828c 100644 --- a/sys/ufs/ffs/ffs_softdep.c +++ b/sys/ufs/ffs/ffs_softdep.c @@ -961,6 +961,7 @@ worklist_remove(item, locked) item, TYPENAME(item->wk_type), item->wk_state); item->wk_state &= ~ONWORKLIST; LIST_REMOVE(item, wk_list); + stack_save(&item->wk_stack_rem); } #endif /* DEBUG */ @@ -2270,7 +2271,6 @@ journal_mount(mp, fs, cred) int error; int i; - mp->mnt_kern_flag |= MNTK_SUJ; error = softdep_journal_lookup(mp, &vp); if (error != 0) { printf("Failed to find journal. Use tunefs to create one\n"); @@ -2295,20 +2295,26 @@ journal_mount(mp, fs, cred) } jblocks->jb_low = jblocks->jb_free / 3; /* Reserve 33%. */ jblocks->jb_min = jblocks->jb_free / 10; /* Suspend at 10%. */ - /* - * Only validate the journal contents if the filesystem is clean, - * otherwise we write the logs but they'll never be used. If the - * filesystem was still dirty when we mounted it the journal is - * invalid and a new journal can only be valid if it starts from a - * clean mount. - */ - if (fs->fs_clean) { - DIP_SET(ip, i_modrev, fs->fs_mtime); - ip->i_flags |= IN_MODIFIED; - ffs_update(vp, 1); - } VFSTOUFS(mp)->softdep_jblocks = jblocks; out: + if (error == 0) { + MNT_ILOCK(mp); + mp->mnt_kern_flag |= MNTK_SUJ; + MNT_IUNLOCK(mp); + /* + * Only validate the journal contents if the + * filesystem is clean, otherwise we write the logs + * but they'll never be used. If the filesystem was + * still dirty when we mounted it the journal is + * invalid and a new journal can only be valid if it + * starts from a clean mount. + */ + if (fs->fs_clean) { + DIP_SET(ip, i_modrev, fs->fs_mtime); + ip->i_flags |= IN_MODIFIED; + ffs_update(vp, 1); + } + } vput(vp); return (error); } @@ -5265,7 +5271,7 @@ softdep_setup_freeblocks(ip, length, flags) if (delay) WORKLIST_INSERT(&bp->b_dep, &freeblks->fb_list); else if (needj) - freeblks->fb_state |= DEPCOMPLETE | COMPLETE; + freeblks->fb_state |= COMPLETE; /* * Because the file length has been truncated to zero, any * pending block allocation dependency structures associated @@ -5327,8 +5333,9 @@ restart: if (inodedep_lookup(mp, ip->i_number, 0, &inodedep) != 0) (void) free_inodedep(inodedep); - if (delay) { + if (delay || needj) freeblks->fb_state |= DEPCOMPLETE; + if (delay) { /* * If the inode with zeroed block pointers is now on disk * we can start freeing blocks. Add freeblks to the worklist @@ -5339,6 +5346,8 @@ restart: if ((freeblks->fb_state & ALLCOMPLETE) == ALLCOMPLETE) add_to_worklist(&freeblks->fb_list, 1); } + if (needj && LIST_EMPTY(&freeblks->fb_jfreeblkhd)) + needj = 0; FREE_LOCK(&lk); /* @@ -5488,7 +5497,7 @@ cancel_allocdirect(adphead, adp, freeblks, delay) newblk = (struct newblk *)adp; /* * If the journal hasn't been written the jnewblk must be passed - * to the call to ffs_freeblk that reclaims the space. We accomplish + * to the call to ffs_blkfree that reclaims the space. We accomplish * this by linking the journal dependency into the freework to be * freed when freework_freeblock() is called. If the journal has * been written we can simply reclaim the journal space when the @@ -6016,11 +6025,11 @@ handle_complete_freeblocks(freeblks) vput(vp); } -#ifdef INVARIANTS - if (freeblks->fb_chkcnt != 0 && - ((fs->fs_flags & FS_UNCLEAN) == 0 || (flags & LK_NOWAIT) != 0)) - printf("handle_workitem_freeblocks: block count\n"); -#endif /* INVARIANTS */ + KASSERT(freeblks->fb_chkcnt == 0 || + ((fs->fs_flags & FS_UNCLEAN) != 0 && (flags & LK_NOWAIT) == 0), + ("handle_workitem_freeblocks: inode %ju block count %jd\n", + (uintmax_t)freeblks->fb_previousinum, + (intmax_t)freeblks->fb_chkcnt)); ACQUIRE_LOCK(&lk); /* @@ -6075,9 +6084,7 @@ indir_trunc(freework, dbn, lbn) fs_pendingblocks = 0; freedeps = 0; needj = UFSTOVFS(ump)->mnt_kern_flag & MNTK_SUJ; - lbnadd = 1; - for (i = level; i > 0; i--) - lbnadd *= NINDIR(fs); + lbnadd = lbn_offset(fs, level); /* * Get buffer of block pointers to be freed. This routine is not * called until the zero'ed inode has been written, so it is safe @@ -6136,20 +6143,17 @@ indir_trunc(freework, dbn, lbn) */ cnt = 0; LIST_FOREACH_SAFE(wk, &wkhd, wk_list, wkn) { - struct workhead freewk; if (wk->wk_type != D_JNEWBLK) continue; WORKLIST_REMOVE_UNLOCKED(wk); - LIST_INIT(&freewk); - WORKLIST_INSERT_UNLOCKED(&freewk, wk); jnewblk = WK_JNEWBLK(wk); if (jnewblk->jn_lbn > 0) i = (jnewblk->jn_lbn - -lbn) / lbnadd; else - i = (jnewblk->jn_lbn - (lbn + 1)) / lbnadd; + i = (jnewblk->jn_lbn - (lbn + level)) / lbnadd; KASSERT(i >= 0 && i < NINDIR(fs), - ("indir_trunc: Index out of range %d parent %jd lbn %jd", - i, lbn, jnewblk->jn_lbn)); + ("indir_trunc: Index out of range %d parent %jd lbn %jd level %d", + i, lbn, jnewblk->jn_lbn, level)); /* Clear the pointer so it isn't found below. */ if (ufs1fmt) { nb = bap1[i]; @@ -6161,8 +6165,25 @@ indir_trunc(freework, dbn, lbn) KASSERT(nb == jnewblk->jn_blkno, ("indir_trunc: Block mismatch %jd != %jd", nb, jnewblk->jn_blkno)); - ffs_blkfree(ump, fs, freeblks->fb_devvp, jnewblk->jn_blkno, - fs->fs_bsize, freeblks->fb_previousinum, &freewk); + if (level != 0) { + ufs_lbn_t nlbn; + + nlbn = (lbn + 1) - (i * lbnadd); + nfreework = newfreework(freeblks, freework, + nlbn, nb, fs->fs_frag, 0); + WORKLIST_INSERT_UNLOCKED(&nfreework->fw_jwork, + &jnewblk->jn_list); + freedeps++; + indir_trunc(nfreework, fsbtodb(fs, nb), nlbn); + } else { + struct workhead freewk; + + LIST_INIT(&freewk); + WORKLIST_INSERT_UNLOCKED(&freewk, wk); + ffs_blkfree(ump, fs, freeblks->fb_devvp, + jnewblk->jn_blkno, fs->fs_bsize, + freeblks->fb_previousinum, &freewk); + } cnt++; } ACQUIRE_LOCK(&lk); @@ -6772,7 +6793,8 @@ cancel_diradd(dap, dirrem, jremref, dotremref, dotdotremref) mkdir->md_jaddref = NULL; if (mkdir->md_state & MKDIR_PARENT) { if (cancel_jaddref(jaddref, NULL, - &dirrem->dm_jwork) == 0) { + &dirrem->dm_jwork) == 0 && + dotdotremref != NULL) { free_jremref(dotdotremref); dotdotremref = NULL; } diff --git a/sys/ufs/ffs/fs.h b/sys/ufs/ffs/fs.h index 42c4cfe..ba98ed3 100644 --- a/sys/ufs/ffs/fs.h +++ b/sys/ufs/ffs/fs.h @@ -607,6 +607,11 @@ struct cg { : (fragroundup(fs, blkoff(fs, (size))))) /* + * Number of indirects in a filesystem block. + */ +#define NINDIR(fs) ((fs)->fs_nindir) + +/* * Indirect lbns are aligned on NDADDR addresses where single indirects * are the negated address of the lowest lbn reachable, double indirects * are this lbn - 1 and triple indirects are this lbn - 2. This yields @@ -631,6 +636,17 @@ lbn_level(ufs_lbn_t lbn) } return (-1); } + +static inline ufs_lbn_t +lbn_offset(struct fs *fs, int level) +{ + ufs_lbn_t res; + + for (res = 1; level > 0; level--) + res *= NINDIR(fs); + return (res); +} + /* * Number of inodes in a secondary storage block/fragment. */ @@ -638,11 +654,6 @@ lbn_level(ufs_lbn_t lbn) #define INOPF(fs) ((fs)->fs_inopb >> (fs)->fs_fragshift) /* - * Number of indirects in a filesystem block. - */ -#define NINDIR(fs) ((fs)->fs_nindir) - -/* * Softdep journal record format. */ diff --git a/sys/ufs/ffs/softdep.h b/sys/ufs/ffs/softdep.h index f584e91..b71cf84 100644 --- a/sys/ufs/ffs/softdep.h +++ b/sys/ufs/ffs/softdep.h @@ -164,11 +164,13 @@ * worklist is needed in the structure, then a wk_data field must be added * and the macros below changed to use it. */ +#include struct worklist { LIST_ENTRY(worklist) wk_list; /* list of work requests */ struct mount *wk_mp; /* Mount we live in */ unsigned int wk_type:8, /* type of request */ wk_state:24; /* state flags */ + struct stack wk_stack_rem; }; #define WK_DATA(wk) ((void *)(wk)) #define WK_PAGEDEP(wk) ((struct pagedep *)(wk)) -- Test scenario: s5.sh (suj5.sh - read)