GENERIC from Sat Jun 11 16:12:52 2011 +0200, r222958+jeff b4feab6c, vmcore.159 KDB: debugger backends: ddb KDB: current backend: ddb 524288K of memory above 4GB ignored Copyright (c) 1992-2011 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-CURRENT #0 r222958+b4feab6c: Sat Jun 11 16:15:46 CEST 2011 pho@x4.osted.lan:/var/tmp/deviant2/sys/i386/compile/PHO i386 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. CPU: AMD Phenom(tm) 9150e Quad-Core Processor (1800.02-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x100f23 Family = 10 Model = 2 Stepping = 3 Features=0x178bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x7ff TSC: P-state invariant real memory = 4294967296 (4096 MB) avail memory = 3535958016 (3372 MB) : Trying to mount root from ufs:/dev/ad4s1a [rw]... Setting hostuuid: 00000000-0000-0000-0000-00218515337d. Setting hostid: 0x6b64ac17. Starting ddb. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: /dev/ad4s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1a: clean, 334840 free (1520 frags, 41665 blocks, 0.2% fragmentation) /dev/ad4s1f: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1f: clean, 197478 free (4470 frags, 24126 blocks, 0.4% fragmentation) /dev/label/tmp: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/label/tmp: clean, 94306026 free (4434 frags, 11787699 blocks, 0.0% fragmentation) /dev/ad4s1d: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1d: clean, 2489091 free (231059 frags, 282254 blocks, 2.3% fragmentation) /dev/ad4s1e: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1e: clean, 8820920 free (49024 frags, 1096487 blocks, 0.2% fragmentation) Mounting local file systems:. Setting hostname: x4.osted.lan. re0: link state changed to DOWN Starting Network: lo0 re0 fwe0 fwip0. lo0: flags=8049 metric 0 mtu 16384 options=3 inet 127.0.0.1 netmask 0xff000000 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0xa nd6 options=21 re0: flags=8843 metric 0 mtu 1500 options=389b ether 00:21:85:15:33:7d inet 192.168.1.101 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::221:85ff:fe15:337d%re0 prefixlen 64 tentative scopeid 0x1 nd6 options=29 media: Ethernet autoselect (none) status: no carrier fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb ch 1 dma -1 fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 Starting devd. Starting Network: usbus0. Starting Network: usbus1. Starting Network: usbus2. Starting Network: usbus3. Starting Network: usbus4. Starting Network: usbus5. Starting Network: fwe0. fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb ch 1 dma -1 Starting Network: fwip0. fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 add net default: gateway 192.168.1.1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 re0: link state changed to UP ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/kde4/lib /usr/local/lib /usr/local/lib/compat /usr/local/lib/compat/pkg /usr/local/lib/nss /usr/local/lib/qt4 /usr/local/lib/virtualbox /usr/local/lib/wine a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout Creating and/or trimming log files. Starting syslogd. No core dumps found. Additional ABI support: linux. Starting rpcbind. NFS access cache time=60 lock order reversal: 1st 0xe071db40 bufwait (bufwait) @ kern/vfs_bio.c:2658 2nd 0xc78f7a00 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:284 KDB: stack backtrace: db_trace_self_wrapper(c0e95b56,2e687361,38323a63,c7000a34,a7a,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c09e61bb,c0e9955f,c6d5e2a0,c6d61ff8,ef2c7894,...) at kdb_backtrace+0x2a _witness_debugger(c0e9955f,c78f7a00,c0ec8822,c6d61ff8,c0ec84b4,...) at _witness_debugger+0x25 witness_checkorder(c78f7a00,9,c0ec84ab,11c,0,...) at witness_checkorder+0x839 _sx_xlock(c78f7a00,0,c0ec84ab,11c,c7417658,...) at _sx_xlock+0x85 ufsdirhash_acquire(e071dae0,ef2c79ec,30,e13dd148,ef2c7964,...) at ufsdirhash_acquire+0x48 ufsdirhash_add(c7417658,ef2c79ec,1148,ef2c7950,ef2c7954,...) at ufsdirhash_add+0x13 ufs_direnter(c773015c,c7989984,ef2c79ec,ef2c7bd0,e071e160,...) at ufs_direnter+0x749 ufs_mkdir(ef2c7bf8,c0ee67a7,0,0,ef2c7b3c,...) at ufs_mkdir+0x936 VOP_MKDIR_APV(c0fc7500,ef2c7bf8,ef2c7bd0,ef2c7b3c,0,...) at VOP_MKDIR_APV+0xc5 kern_mkdirat(c79bc000,ffffff9c,28404020,0,1c0,...) at kern_mkdirat+0x225 kern_mkdir(c79bc000,28404020,0,1c0,ef2c7c7c,...) at kern_mkdir+0x2e mkdir(c79bc000,ef2c7cec,ef2c7d28,c0e97daa,0,...) at mkdir+0x29 syscallenter(c79bc000,ef2c7ce4,ef2c7ce4,0,0,...) at syscallenter+0x263 syscall(ef2c7d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, mkdir), eip = 0x28172663, esp = 0xbfbfe8cc, ebp = 0xbfbfed78 --- Clearing /tmp (X related). Starting mountd. NFSv4 is disabled Starting nfsd. Recovering vi editor sessions:lock order reversal: 1st 0xc7a35c94 ufs (ufs) @ kern/vfs_lookup.c:501 2nd 0xe07256c0 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:261 3rd 0xc7a961b4 ufs (ufs) @ kern/vfs_subr.c:2134 KDB: stack backtrace: db_trace_self_wrapper(c0e95b56,3a632e73,a313632,28081700,280817c0,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c09e61bb,c0e99578,c6d5e2a0,c6d61f90,ef2bb340,...) at kdb_backtrace+0x2a _witness_debugger(c0e99578,c7a961b4,c0e88914,c6d61f90,c0ea117b,...) at _witness_debugger+0x25 witness_checkorder(c7a961b4,9,c0ea1172,856,0,...) at witness_checkorder+0x839 __lockmgr_args(c7a961b4,80100,c7a96220,0,0,...) at __lockmgr_args+0x814 ffs_lock(ef2bb464,c09f7aab,c0ea04d7,80100,c7a9615c,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c0fc7500,ef2bb464,c79bcc30,c0fe2500,c7a9615c,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c7a9615c,80100,c0ea1172,856,4,...) at _vn_lock+0x78 vget(c7a9615c,80100,c79bcb80,50,0,...) at vget+0xbb vfs_hash_get(c73f4b50,15f4ab,80000,c79bcb80,ef2bb5a8,...) at vfs_hash_get+0xed ffs_vgetf(c73f4b50,15f4ab,80000,ef2bb5a8,1,...) at ffs_vgetf+0x49 softdep_sync_buf(c7a35c3c,e0725660,1,107,0,...) at softdep_sync_buf+0xac9 ffs_syncvnode(c7a35c3c,1,c7a35d34,c6d5e2a0,c1167798,...) at ffs_syncvnode+0x238 ffs_truncate(c7a35c3c,200,0,880,c701ce00,...) at ffs_truncate+0x81b ufs_direnter(c7a35c3c,c7a9615c,ef2bb914,ef2bbba4,0,...) at ufs_direnter+0x924 ufs_makeinode(ef2bbba4,c0fc7a80,ef2bbb00,ef2bba5c,c0cfea25,...) at ufs_makeinode+0x5e5 ufs_create(ef2bbb00,c0ee696d,0,0,ef2bbb78,...) at ufs_create+0x30 VOP_CREATE_APV(c0fc7500,ef2bbb00,ef2bbba4,ef2bba98,0,...) at VOP_CREATE_APV+0xc5 vn_open_cred(ef2bbb78,ef2bbc2c,1b0,0,c701ce00,...) at vn_open_cred+0x205 vn_open(ef2bbb78,ef2bbc2c,1b0,c7879150,0,...) at vn_open+0x3b kern_openat(c79bcb80,ffffff9c,28839f60,0,a03,...) at kern_openat+0x12f kern_open(c79bcb80,28839f60,0,a02,1b0,...) at kern_open+0x35 open(c79bcb80,ef2bbcec,ef2bbd28,c0e97daa,0,...) at open+0x30 syscallenter(c79bcb80,ef2bbce4,ef2bbce4,0,0,...) at syscallenter+0x263 syscall(ef2bbd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (5, FreeBSD ELF32, open), eip = 0x283d16a3, esp = 0xbfbfc4fc, ebp = 0xbfbfc588 --- . Updating motd:. Starting ntpd. Starting default moused. Expensive timeout(9) function: 0xc07ff800(0xc11bd500) 0.014098439 s Configuring syscons: keymap blanktime. Starting sshd. Starting cron. Local package initialization: backuplock order reversal: 1st 0xc749b5c8 ufs (ufs) @ kern/vfs_mount.c:1193 2nd 0xc749adf0 devfs (devfs) @ ufs/ffs/ffs_softdep.c:1827 KDB: stack backtrace: db_trace_self_wrapper(c0e95b56,38313a63,a3732,c0ec9ea1,c15b3888,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c09e61bb,c0e9955f,c6d61f90,c6d61e58,ef1e5a04,...) at kdb_backtrace+0x2a _witness_debugger(c0e9955f,c749adf0,c0e80787,c6d61e58,c0ec4aa9,...) at _witness_debugger+0x25 witness_checkorder(c749adf0,9,c0ec4aa0,723,c749ae5c,...) at witness_checkorder+0x839 __lockmgr_args(c749adf0,80400,c749ae5c,0,0,...) at __lockmgr_args+0x814 vop_stdlock(ef1e5b24,4,c0e90227,80400,c749ad98,...) at vop_stdlock+0x65 VOP_LOCK1_APV(c0f9d000,ef1e5b24,c119b230,c0fe2500,c749ad98,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c749ad98,80400,c0ec4aa0,723,c77312d4,...) at _vn_lock+0x78 softdep_flushworklist(c77312d4,ef1e5bd0,c7722b80,574,0,...) at softdep_flushworklist+0x47 ffs_sync(c77312d4,1,ef1e5c14,4ee,0,...) at ffs_sync+0x2fd dounmount(c77312d4,8080000,c7722b80,473,5b85c5ea,...) at dounmount+0x447 unmount(c7722b80,ef1e5cec,28176705,1,0,...) at unmount+0x310 syscallenter(c7722b80,ef1e5ce4,c0cdb89d,c101d170,0,...) at syscallenter+0x263 syscall(ef1e5d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, unmount), eip = 0x280dcd5b, esp = 0xbfbfe64c, ebp = 0xbfbfe718 --- ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 102353 files, 24007955 used, 94306026 free (4434 frags, 11787699 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 102353 files, 24007955 used, 94306026 free (4434 frags, 11787699 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** fsck -y /tmp watchdogd. Starting inetd. Sat Jun 11 16:32:33 CEST 2011 FreeBSD/i386 (x4.osted.lan) (console) login: Jun 11 16:33:04 x4 su: pho to root on /dev/pts/0 panic: worklist_insert: 0xc867c700 freework(0x8009) already on list cpuid = 2 KDB: enter: panic [ thread pid 4810 tid 100094 ] Stopped at kdb_enter+0x3a: movl $0,kdb_why db> run pho db:0:pho> bt Tracing pid 4810 tid 100094 td 0xc74838a0 kdb_enter(c0e91c42,c0e91c42,c0ec4b7d,ef1f2640,2,...) at kdb_enter+0x3a panic(c0ec4b7d,c867c700,c0ec72bd,8009,e0860780,...) at panic+0x134 worklist_insert(c8e42000,c0fc4c28,c0ec4aa0,170d,0,...) at worklist_insert+0x73 freework_freeblock(c119b230,0,c0ec4aa0,1e00,8,...) at freework_freeblock+0x31a indir_trunc(fffffff4,ffffffff,c0ec4aa0,1c94,8,...) at indir_trunc+0x665 handle_workitem_freeblocks(c119b230,0,c0ec4aa0,1897,c8b3a000,...) at handle_workitem_freeblocks+0x24f softdep_journal_freeblocks(c7ebb0e8,c8b3a000,1d20a9f,0,800,...) at softdep_journal_freeblocks+0x10c4 ffs_truncate(c7e68570,1d20a9f,0,800,c8b3a000,...) at ffs_truncate+0x86d ufs_setattr(ef1f2bf0,c0ee68e6,c7e68570,ef1f2b88,c7e68570,...) at ufs_setattr+0x738 VOP_SETATTR_APV(c0fc7500,ef1f2bf0,c7e68570,28f,0,...) at VOP_SETATTR_APV+0xc5 vn_truncate(c7e472d8,1d20a9f,0,c8b3a000,c74838a0,...) at vn_truncate+0x145 kern_ftruncate(c74838a0,3,1d20a9f,0,ef1f2c7c,...) at kern_ftruncate+0xe7 ftruncate(c74838a0,ef1f2cec,ef1f2d28,c0e97daa,0,...) at ftruncate+0x28 syscallenter(c74838a0,ef1f2ce4,ef1f2ce4,0,c102c400,...) at syscallenter+0x263 syscall(ef1f2d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (480, FreeBSD ELF32, ftruncate), eip = 0x28169e6b, esp = 0xbfbfe75c, ebp = 0xbfbfe788 --- db:0:bt> show allpcpu Current CPU: 2 cpuid = 0 dynamic pcpu = 0x442400 curthread = 0xc7dc48a0: pid 4809 "suj17" curpcb = 0xef327d80 fpcurthread = none idlethread = 0xc6dbe8a0: tid 100003 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: cpuid = 1 dynamic pcpu = 0x5b47400 curthread = 0xc7045b80: pid 18 "softdepflush" curpcb = 0xecfa6d80 fpcurthread = none idlethread = 0xc6dbe5c0: tid 100004 "idle: cpu1" APIC ID = 1 currentldt = 0x50 spin locks held: cpuid = 2 dynamic pcpu = 0x5b4a400 curthread = 0xc74838a0: pid 4810 "suj17" curpcb = 0xef1f2d80 fpcurthread = none idlethread = 0xc6dbe2e0: tid 100005 "idle: cpu2" APIC ID = 2 currentldt = 0x50 spin locks held: cpuid = 3 dynamic pcpu = 0x5b4d400 curthread = 0xc77218a0: pid 4811 "suj17" curpcb = 0xef271d80 fpcurthread = none idlethread = 0xc6dbe000: tid 100006 "idle: cpu3" APIC ID = 3 currentldt = 0x50 spin locks held: db:0:allpcpu> show alllocks Process 4814 (suj17) thread 0xc7861000 (100168) exclusive lockmgr bufwait (bufwait) r = 0 (0xe08ed080) locked @ kern/vfs_bio.c:2658 exclusive lockmgr bufwait (bufwait) r = 0 (0xe089ccc0) locked @ kern/vfs_bio.c:2658 exclusive lockmgr bufwait (bufwait) r = 0 (0xe0761240) locked @ kern/vfs_bio.c:1911 exclusive lockmgr ufs (ufs) r = 0 (0xc7edc5c8) locked @ kern/vfs_vnops.c:612 Process 4813 (suj17) thread 0xc7dc0000 (100182) exclusive lockmgr bufwait (bufwait) r = 0 (0xe076cd20) locked @ kern/vfs_bio.c:2658 exclusive lockmgr bufwait (bufwait) r = 0 (0xe09bcd40) locked @ kern/vfs_bio.c:1911 exclusive lockmgr ufs (ufs) r = 0 (0xc7f5eb38) locked @ kern/vfs_vnops.c:612 Process 4812 (suj17) thread 0xc8b972e0 (100210) exclusive lockmgr bufwait (bufwait) r = 0 (0xe073a240) locked @ kern/vfs_bio.c:2658 exclusive lockmgr ufs (ufs) r = 0 (0xc7f85880) locked @ kern/vfs_vnops.c:612 Process 4811 (suj17) thread 0xc77218a0 (100120) exclusive lockmgr bufwait (bufwait) r = 0 (0xe0961a00) locked @ kern/vfs_bio.c:1911 exclusive lockmgr ufs (ufs) r = 0 (0xc7ede058) locked @ kern/vfs_vnops.c:655 Process 4810 (suj17) thread 0xc74838a0 (100094) exclusive lockmgr bufwait (bufwait) r = 0 (0xe0860780) locked @ ufs/ffs/ffs_softdep.c:5894 exclusive sleep mutex Softdep Lock (Softdep Lock) r = 0 (0xc119b230) locked @ ufs/ffs/ffs_softdep.c:7680 exclusive lockmgr ufs (ufs) r = 0 (0xc7e685c8) locked @ kern/vfs_vnops.c:655 Process 4809 (suj17) thread 0xc7dc48a0 (100153) exclusive lockmgr bufwait (bufwait) r = 0 (0xe086e480) locked @ kern/vfs_bio.c:2658 exclusive lockmgr bufwait (bufwait) r = 0 (0xe08fcac0) locked @ kern/vfs_bio.c:2658 exclusive lockmgr ufs (ufs) r = 0 (0xc7f5cb38) locked @ kern/vfs_vnops.c:612 Process 2938 (sshd) thread 0xc783eb80 (100169) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xc7b168ac) locked @ kern/uipc_sockbuf.c:148 Process 2937 (sshd) thread 0xc7e3e2e0 (100171) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xc83563cc) locked @ kern/uipc_sockbuf.c:148 Process 2936 (sshd) thread 0xc74832e0 (100096) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xc7fdd3cc) locked @ kern/uipc_sockbuf.c:148 Process 2226 (sshd) thread 0xc79bd000 (100131) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xc7a2da4c) locked @ kern/uipc_sockbuf.c:148 db:0:alllocks> show lockedvnods Locked vnodes 0xc7f5cae0: 0xc7f5cae0: tag ufs, type VREG tag ufs, type VREG usecount 1, writecount 1, refcount 249 mountedhere 0 usecount 1, writecount 1, refcount 249 mountedhere 0 flags () flags () v_object 0xc7e83908 ref 0 pages 988 v_object 0xc7e83908 ref 0 pages 988 lock type ufs: EXCL by thread 0xc7dc48a0 (pid 4809) lock type ufs: EXCL by thread 0xc7dc48a0 (pid 4809) #0 0xc099587e at __lockmgr_args+0xbfe #0 0xc099587e at __lockmgr_args+0xbfe #1 0xc0be7e41 at ffs_lock+0xa1 #1 0xc0be7e41 at ffs_lock+0xa1 #2 0xc0cfda55 at VOP_LOCK1_APV+0xb5 #2 0xc0cfda55 at VOP_LOCK1_APV+0xb5 #3 0xc0a557f8 at _vn_lock+0x78 #3 0xc0a557f8 at _vn_lock+0x78 #4 0xc0a56b66 at vn_write+0x156 #4 0xc0a56b66 at vn_write+0x156 #5 0xc09fa445 at dofilewrite+0x95 #5 0xc09fa445 at dofilewrite+0x95 #6 0xc09fbbe8 at kern_writev+0x58 #6 0xc09fbbe8 at kern_writev+0x58 #7 0xc09fbcff at write+0x4f #7 0xc09fbcff at write+0x4f #8 0xc09f0473 at syscallenter+0x263 #8 0xc09f0473 at syscallenter+0x263 #9 0xc0cdb03f at syscall+0x4f #9 0xc0cdb03f at syscall+0x4f #10 0xc0cc4701 at Xint0x80_syscall+0x21 #10 0xc0cc4701 at Xint0x80_syscall+0x21 ino 5, on dev md5a ino 5, on dev md5a 0xc7ede000: 0xc7ede000: tag ufs, type VREG tag ufs, type VREG usecount 1, writecount 1, refcount 584 mountedhere 0 usecount 1, writecount 1, refcount 584 mountedhere 0 flags () flags () v_object 0xc7e23880 ref 0 pages 2328 v_object 0xc7e23880 ref 0 pages 2328 lock type ufs: EXCL by thread 0xc77218a0 (pid 4811) lock type ufs: EXCL by thread 0xc77218a0 (pid 4811) #0 0xc099587e at __lockmgr_args+0xbfe #0 0xc099587e at __lockmgr_args+0xbfe #1 0xc0be7e41 at ffs_lock+0xa1 #1 0xc0be7e41 at ffs_lock+0xa1 #2 0xc0cfda55 at VOP_LOCK1_APV+0xb5 #2 0xc0cfda55 at VOP_LOCK1_APV+0xb5 #3 0xc0a557f8 at _vn_lock+0x78 #3 0xc0a557f8 at _vn_lock+0x78 #4 0xc0a56922 at vn_truncate+0xd2 #4 0xc0a56922 at vn_truncate+0xd2 #5 0xc09fb917 at kern_ftruncate+0xe7 #5 0xc09fb917 at kern_ftruncate+0xe7 #6 0xc09fb998 at ftruncate+0x28 #6 0xc09fb998 at ftruncate+0x28 #7 0xc09f0473 at syscallenter+0x263 #7 0xc09f0473 at syscallenter+0x263 #8 0xc0cdb03f at syscall+0x4f #8 0xc0cdb03f at syscall+0x4f #9 0xc0cc4701 at Xint0x80_syscall+0x21 #9 0xc0cc4701 at Xint0x80_syscall+0x21 ino 6, on dev md5a ino 6, on dev md5a 0xc7f85828: 0xc7f85828: tag ufs, type VREG tag ufs, type VREG usecount 1, writecount 1, refcount 335 mountedhere 0 usecount 1, writecount 1, refcount 335 mountedhere 0 flags () flags () v_object 0xc7ef47f8 ref 0 pages 1332 v_object 0xc7ef47f8 ref 0 pages 1332 lock type ufs: EXCL by thread 0xc8b972e0 (pid 4812) lock type ufs: EXCL by thread 0xc8b972e0 (pid 4812) #0 0xc099587e at __lockmgr_args+0xbfe #0 0xc099587e at __lockmgr_args+0xbfe #1 0xc0be7e41 at ffs_lock+0xa1 #1 0xc0be7e41 at ffs_lock+0xa1 #2 0xc0cfda55 at VOP_LOCK1_APV+0xb5 #2 0xc0cfda55 at VOP_LOCK1_APV+0xb5 #3 0xc0a557f8 at _vn_lock+0x78 #3 0xc0a557f8 at _vn_lock+0x78 #4 0xc0a56b66 at vn_write+0x156 #4 0xc0a56b66 at vn_write+0x156 #5 0xc09fa445 at dofilewrite+0x95 #5 0xc09fa445 at dofilewrite+0x95 #6 0xc09fbbe8 at kern_writev+0x58 #6 0xc09fbbe8 at kern_writev+0x58 #7 0xc09fbcff at write+0x4f #7 0xc09fbcff at write+0x4f #8 0xc09f0473 at syscallenter+0x263 #8 0xc09f0473 at syscallenter+0x263 #9 0xc0cdb03f at syscall+0x4f #9 0xc0cdb03f at syscall+0x4f #10 0xc0cc4701 at Xint0x80_syscall+0x21 #10 0xc0cc4701 at Xint0x80_syscall+0x21 ino 7, on dev md5a ino 7, on dev md5a 0xc7e68570: 0xc7e68570: tag ufs, type VREG tag ufs, type VREG usecount 1, writecount 1, refcount 29 mountedhere 0 usecount 1, writecount 1, refcount 29 mountedhere 0 flags () flags () v_object 0xc7ebc5d8 ref 0 pages 108 v_object 0xc7ebc5d8 ref 0 pages 108 lock type ufs: EXCL by thread 0xc74838a0 (pid 4810) lock type ufs: EXCL by thread 0xc74838a0 (pid 4810) #0 0xc099587e at __lockmgr_args+0xbfe #0 0xc099587e at __lockmgr_args+0xbfe #1 0xc0be7e41 at ffs_lock+0xa1 #1 0xc0be7e41 at ffs_lock+0xa1 #2 0xc0cfda55 at VOP_LOCK1_APV+0xb5 #2 0xc0cfda55 at VOP_LOCK1_APV+0xb5 #3 0xc0a557f8 at _vn_lock+0x78 #3 0xc0a557f8 at _vn_lock+0x78 #4 0xc0a56922 at vn_truncate+0xd2 #4 0xc0a56922 at vn_truncate+0xd2 #5 0xc09fb917 at kern_ftruncate+0xe7 #5 0xc09fb917 at kern_ftruncate+0xe7 #6 0xc09fb998 at ftruncate+0x28 #6 0xc09fb998 at ftruncate+0x28 #7 0xc09f0473 at syscallenter+0x263 #7 0xc09f0473 at syscallenter+0x263 #8 0xc0cdb03f at syscall+0x4f #8 0xc0cdb03f at syscall+0x4f #9 0xc0cc4701 at Xint0x80_syscall+0x21 #9 0xc0cc4701 at Xint0x80_syscall+0x21 ino 8, on dev md5a ino 8, on dev md5a 0xc7edc570: 0xc7edc570: tag ufs, type VREG tag ufs, type VREG usecount 1, writecount 1, refcount 192 mountedhere 0 usecount 1, writecount 1, refcount 192 mountedhere 0 flags () flags () v_object 0xc7e84770 ref 0 pages 760 v_object 0xc7e84770 ref 0 pages 760 lock type ufs: EXCL by thread 0xc7861000 (pid 4814) lock type ufs: EXCL by thread 0xc7861000 (pid 4814) #0 0xc099587e at __lockmgr_args+0xbfe #0 0xc099587e at __lockmgr_args+0xbfe #1 0xc0be7e41 at ffs_lock+0xa1 #1 0xc0be7e41 at ffs_lock+0xa1 #2 0xc0cfda55 at VOP_LOCK1_APV+0xb5 #2 0xc0cfda55 at VOP_LOCK1_APV+0xb5 #3 0xc0a557f8 at _vn_lock+0x78 #3 0xc0a557f8 at _vn_lock+0x78 #4 0xc0a56b66 at vn_write+0x156 #4 0xc0a56b66 at vn_write+0x156 #5 0xc09fa445 at dofilewrite+0x95 #5 0xc09fa445 at dofilewrite+0x95 #6 0xc09fbbe8 at kern_writev+0x58 #6 0xc09fbbe8 at kern_writev+0x58 #7 0xc09fbcff at write+0x4f #7 0xc09fbcff at write+0x4f #8 0xc09f0473 at syscallenter+0x263 #8 0xc09f0473 at syscallenter+0x263 #9 0xc0cdb03f at syscall+0x4f #9 0xc0cdb03f at syscall+0x4f #10 0xc0cc4701 at Xint0x80_syscall+0x21 #10 0xc0cc4701 at Xint0x80_syscall+0x21 ino 10, on dev md5a ino 10, on dev md5a 0xc7f5eae0: 0xc7f5eae0: tag ufs, type VREG tag ufs, type VREG usecount 1, writecount 1, refcount 275 mountedhere 0 usecount 1, writecount 1, refcount 275 mountedhere 0 flags () flags () v_object 0xc7f1c000 ref 0 pages 1092 v_object 0xc7f1c000 ref 0 pages 1092 lock type ufs: EXCL by thread 0xc7dc0000 (pid 4813) lock type ufs: EXCL by thread 0xc7dc0000 (pid 4813) #0 0xc099587e at __lockmgr_args+0xbfe #0 0xc099587e at __lockmgr_args+0xbfe #1 0xc0be7e41 at ffs_lock+0xa1 #1 0xc0be7e41 at ffs_lock+0xa1 #2 0xc0cfda55 at VOP_LOCK1_APV+0xb5 #2 0xc0cfda55 at VOP_LOCK1_APV+0xb5 #3 0xc0a557f8 at _vn_lock+0x78 #3 0xc0a557f8 at _vn_lock+0x78 #4 0xc0a56b66 at vn_write+0x156 #4 0xc0a56b66 at vn_write+0x156 #5 0xc09fa445 at dofilewrite+0x95 #5 0xc09fa445 at dofilewrite+0x95 #6 0xc09fbbe8 at kern_writev+0x58 #6 0xc09fbbe8 at kern_writev+0x58 #7 0xc09fbcff at write+0x4f #7 0xc09fbcff at write+0x4f #8 0xc09f0473 at syscallenter+0x263 #8 0xc09f0473 at syscallenter+0x263 #9 0xc0cdb03f at syscall+0x4f #9 0xc0cdb03f at syscall+0x4f #10 0xc0cc4701 at Xint0x80_syscall+0x21 #10 0xc0cc4701 at Xint0x80_syscall+0x21 ino 11, on dev md5a ino 11, on dev md5a db:0:lockedvnods> show mount 0xc73f3b50 /dev/ad4s1a on / (ufs) 0xc73f4000 devfs on /dev (devfs) 0xc77315a8 /dev/ad4s1f on /home (ufs) 0xc7731000 /dev/ad4s1d on /usr (ufs) 0xc73f4b50 /dev/ad4s1e on /var (ufs) 0xc73f487c procfs on /proc (procfs) 0xc73f45a8 /dev/label/tmp on /tmp (ufs) 0xc73f35a8 /dev/md5a on /mnt (ufs) More info: show mount db:0:mount> ps pid ppid pgrp uid state wmesg wchan cmd 5058 2948 2944 1001 S nanslp 0xc101ea84 sleep 4814 4808 4807 1001 R+ suj17 4813 4808 4807 1001 R+ suj17 4812 4808 4807 1001 R+ suj17 4811 4808 4807 1001 RL+ CPU 3 suj17 4810 4808 4807 1001 R+ CPU 2 suj17 4809 4808 4807 1001 R+ CPU 0 suj17 4808 4807 4807 1001 S+ wait 0xc7759834 suj17 4807 4806 4807 1001 S+ wait 0xc8018578 bash 4806 4782 2808 0 S+ wait 0xc7759af0 su 4801 0 0 0 DL mdwait 0xc7e30800 [md5] 4782 2808 2808 0 S+ wait 0xc8be5af0 sh 2949 2944 2944 1001 S piperd 0xc786c7a8 awk 2948 2944 2944 1001 S wait 0xc7dbd578 sh 2947 2946 2947 1001 Ss+ select 0xc783c0a4 top 2946 2938 2938 1001 S select 0xc783ca64 sshd 2945 2943 2945 1001 Ss kqread 0xc74b7780 tail 2944 2942 2944 1001 Ss wait 0xc785a578 sh 2943 2937 2937 1001 S select 0xc783c0e4 sshd 2942 2936 2936 1001 S select 0xc783c9e4 sshd 2938 2086 2938 0 Ss sbwait 0xc7b168d8 sshd 2937 2086 2937 0 Ss sbwait 0xc83563f8 sshd 2936 2086 2936 0 Ss sbwait 0xc7fdd3f8 sshd 2808 2804 2808 0 S+ wait 0xc7dbe578 sh 2804 2800 2804 0 S+ wait 0xc7dbe2bc bash 2800 2799 2800 0 S+ pause 0xc7835b48 csh 2799 2229 2799 1001 S+ wait 0xc78352bc su 2229 2228 2229 1001 Ss+ wait 0xc7dbc834 bash 2228 2226 2226 1001 S select 0xc7b30024 sshd 2226 2086 2226 0 Ss sbwait 0xc7a2da78 sshd 2225 1 2225 0 Ss+ ttyin 0xc7181070 getty 2224 1 2224 0 Ss+ ttyin 0xc7181270 getty 2223 1 2223 0 Ss+ ttyin 0xc7181470 getty 2222 1 2222 0 Ss+ ttyin 0xc6f59070 getty 2221 1 2221 0 Ss+ ttyin 0xc6f59270 getty 2220 1 2220 0 Ss+ ttyin 0xc6f59470 getty 2219 1 2219 0 Ss+ ttyin 0xc6f59670 getty 2218 1 2218 0 Ss+ ttyin 0xc6f59870 getty 2217 1 2217 0 Ss+ ttyin 0xc6f59a70 getty 2177 1 2177 0 Ss select 0xc771faa4 inetd 2153 1 2153 0 Ss nanslp 0xc101ea84 watchdogd 2106 1 2106 0 Ss nanslp 0xc101ea84 cron 2098 1 2098 25 Ss pause 0xc785c88c sendmail 2094 1 2094 0 Ss select 0xc742e664 sendmail 2086 1 2086 0 Ss select 0xc742e6a4 sshd 2026 1 2026 0 Ss select 0xc7b306a4 moused 1966 1 1966 0 Ss select 0xc783c1a4 ntpd 1848 1847 1847 0 S (threaded) nfsd 100143 S rpcsvc 0xc7372590 nfsd: service 100142 S rpcsvc 0xc73725d0 nfsd: service 100141 S rpcsvc 0xc7372610 nfsd: service 100138 S rpcsvc 0xc787f410 nfsd: master 1847 1 1847 0 Ss select 0xc742e6e4 nfsd 1837 1 1837 0 Ss select 0xc742e724 mountd 1738 1 1738 0 Ss select 0xc771fb64 rpcbind 1710 1 1710 0 Ss select 0xc6ecd924 syslogd 1502 1 1502 0 Ss select 0xc6ecd964 devd 18 0 0 0 RL CPU 1 [softdepflush] 17 0 0 0 DL vlruwt 0xc73a1578 [vnlru] 16 0 0 0 DL syncer 0xc1194c74 [syncer] 9 0 0 0 DL qsleep 0xc1194908 [bufdaemon] 8 0 0 0 DL pgzero 0xc119d0dc [pagezero] 7 0 0 0 DL psleep 0xc119ccc8 [vmdaemon] 6 0 0 0 DL psleep 0xc119cc90 [pagedaemon] 5 0 0 0 DL ccb_scan 0xc0fe3954 [xpt_thrd] 4 0 0 0 DL waiting_ 0xc1196798 [sctp_iterator] 3 0 0 0 DL - 0xc6fe9e3c [fdc0] 2 0 0 0 SL - 0xc706b000 [fw0_probe] 15 0 0 0 DL (threaded) [usb] 100057 D - 0xc704cd34 [usbus5] 100056 D - 0xc704cd04 [usbus5] 100055 D - 0xc704ccd4 [usbus5] 100054 D - 0xc704cca4 [usbus5] 100052 D - 0xc703eb5c [usbus4] 100051 D - 0xc703eb2c [usbus4] 100050 D - 0xc703eafc [usbus4] 100049 D - 0xc703eacc [usbus4] 100048 D - 0xc7038b5c [usbus3] 100047 D - 0xc7038b2c [usbus3] 100046 D - 0xc7038afc [usbus3] 100045 D - 0xc7038acc [usbus3] 100044 D - 0xc702fb5c [usbus2] 100043 D - 0xc702fb2c [usbus2] 100042 D - 0xc702fafc [usbus2] 100041 D - 0xc702facc [usbus2] 100039 D - 0xc7028b5c [usbus1] 100038 D - 0xc7028b2c [usbus1] 100037 D - 0xc7028afc [usbus1] 100036 D - 0xc7028acc [usbus1] 100034 D - 0xc701fb5c [usbus0] 100033 D - 0xc701fb2c [usbus0] 100032 D - 0xc701fafc [usbus0] 100031 D - 0xc701facc [usbus0] 14 0 0 0 DL - 0xc101d884 [yarrow] 13 0 0 0 DL (threaded) [geom] 100015 D - 0xc101b4a8 [g_down] 100014 D - 0xc101b4a4 [g_up] 100013 D - 0xc101b49c [g_event] 12 0 0 0 WL (threaded) [intr] 100065 I [irq12: psm0] 100064 I [irq1: atkbd0] 100062 I [swi0: uart] 100059 I [irq20: fwohci0] 100058 I [irq14: ata0] 100053 I [irq19: ehci0] 100040 I [irq18: ohci2 ohci4] 100035 I [irq17: ohci1 ohci3] 100030 I [irq16: ohci0] 100029 I [irq22: ahci0] 100028 I [irq256: re0] 100027 I [swi5: +] 100026 I [swi2: cambio] 100021 I [swi6: task queue] 100019 I [swi6: Giant taskq] 100012 I [swi3: vm] 100011 I [swi4: clock] 100010 I [swi4: clock] 100009 I [swi4: clock] 100008 I [swi4: clock] 100007 I [swi1: netisr 0] 11 0 0 0 RL (threaded) [idle] 100006 CanRun [idle: cpu3] 100005 CanRun [idle: cpu2] 100004 CanRun [idle: cpu1] 100003 CanRun [idle: cpu0] 1 0 1 0 SLs wait 0xc6dbcaf0 [init] 10 0 0 0 DL audit_wo 0xc119aa00 [audit] 0 0 0 0 DLs (threaded) [kernel] 100076 D - 0xc73729c0 [mca taskq] 100066 D - 0xc101d884 [deadlkres] 100060 D - 0xc7034480 [fw0_taskq] 100025 D - 0xc6f50540 [acpi_task_2] 100024 D - 0xc6f50540 [acpi_task_1] 100023 D - 0xc6f50540 [acpi_task_0] 100022 D - 0xc6f50580 [kqueue taskq] 100020 D - 0xc6f50600 [ffs_trim taskq] 100018 D - 0xc6f50680 [thread taskq] 100016 D - 0xc6d76b80 [firmware taskq] 100000 D sched 0xc101b580 [swapper] db:0:ps> allt Tracing command sleep pid 5058 tid 100193 td 0xc82ebb80 sched_switch(c82ebb80,0,104,191,807dc76b,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c82ebb80,0,c0e97420,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(2711,c09ec990,c82ebb80,2,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c101ea84,6c,c0e93177,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c101ea84,0,16c,c0e93177,2711,...) at _sleep+0x328 kern_nanosleep(c82ebb80,ef39fc34,ef39fc3c,a,0,...) at kern_nanosleep+0xc1 nanosleep(c82ebb80,ef39fcec,ef39fd28,c0e97daa,0,...) at nanosleep+0x6f syscallenter(c82ebb80,ef39fce4,ef39fce4,0,0,...) at syscallenter+0x263 syscall(ef39fd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x28165b77, esp = 0xbfbfec5c, ebp = 0xbfbfeca8 --- Tracing command suj17 pid 4814 tid 100168 td 0xc7861000 sched_switch(c7861000,0,104,191,7bad7071,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,60,...) at mi_switch+0x1eb sleepq_switch(c7861000,0,c0e97420,268,60,...) at sleepq_switch+0x162 sleepq_wait(e0810d80,60,c0e8a2b0,4,0,...) at sleepq_wait+0x63 __lockmgr_args(e0810d80,81900,c7fa9e90,c0e8a2b0,60,...) at __lockmgr_args+0xb1f getblk(c7fa9d98,b7d00,0,4000,0,...) at getblk+0x167 breadn(c7fa9d98,b7d00,0,4000,0,...) at breadn+0x44 bread(c7fa9d98,b7d00,0,4000,0,...) at bread+0x4c ffs_checkblk(4000,0,2,2,c0bb8810,...) at ffs_checkblk+0x1d6 ffs_reallocblks(ef354a08,c0ee5d90,0,c7edc570,c74a0b98,...) at ffs_reallocblks+0x12bc VOP_REALLOCBLKS_APV(c0fc7500,ef354a08,0,4000,0,...) at VOP_REALLOCBLKS_APV+0xd6 cluster_write(c7edc570,e07611e0,5f38000,0,0,...) at cluster_write+0x3c2 ffs_write(ef354b94,c0ee68a3,c7edc570,0,c7edc570,...) at ffs_write+0x51b VOP_WRITE_APV(c0fc7500,ef354b94,c7edc570,264,0,...) at VOP_WRITE_APV+0x136 vn_write(c7432cb0,ef354c28,c8b3a000,0,c7861000,...) at vn_write+0x1ca dofilewrite(ef354c28,ffffffff,ffffffff,0,c7432cb0,...) at dofilewrite+0x95 kern_writev(c7861000,3,ef354c28,ef354c48,1,...) at kern_writev+0x58 write(c7861000,ef354cec,ef354d28,c0e97daa,0,...) at write+0x4f syscallenter(c7861000,ef354ce4,ef354ce4,0,c102b700,...) at syscallenter+0x263 syscall(ef354d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x28181623, esp = 0xbfbfe78c, ebp = 0xbfbfe838 --- Tracing command suj17 pid 4813 tid 100182 td 0xc7dc0000 sched_switch(c7dc0000,0,104,191,7bad9d38,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,60,...) at mi_switch+0x1eb sleepq_switch(c7dc0000,0,c0e97420,268,60,...) at sleepq_switch+0x162 sleepq_wait(e0810d80,60,c0e8a2b0,4,0,...) at sleepq_wait+0x63 __lockmgr_args(e0810d80,81900,c7fa9e90,c0e8a2b0,60,...) at __lockmgr_args+0xb1f getblk(c7fa9d98,b7d00,0,4000,0,...) at getblk+0x167 breadn(c7fa9d98,b7d00,0,4000,0,...) at breadn+0x44 bread(c7fa9d98,b7d00,0,4000,0,...) at bread+0x4c ffs_checkblk(4000,1771,0,e09899e0,765,...) at ffs_checkblk+0x1d6 ffs_reallocblks(ef37ea08,c0ee5d90,0,c7f5eae0,c867f658,...) at ffs_reallocblks+0x1698 VOP_REALLOCBLKS_APV(c0fc7500,ef37ea08,0,4000,0,...) at VOP_REALLOCBLKS_APV+0xd6 cluster_write(c7f5eae0,e09bcce0,5dc8000,0,0,...) at cluster_write+0x3c2 ffs_write(ef37eb94,c0ee68a3,c7f5eae0,0,c7f5eae0,...) at ffs_write+0x51b VOP_WRITE_APV(c0fc7500,ef37eb94,c7f5eae0,264,0,...) at VOP_WRITE_APV+0x136 vn_write(c7fdb930,ef37ec28,c8b3a000,0,c7dc0000,...) at vn_write+0x1ca dofilewrite(ef37ec28,ffffffff,ffffffff,0,c7fdb930,...) at dofilewrite+0x95 kern_writev(c7dc0000,3,ef37ec28,ef37ec48,1,...) at kern_writev+0x58 write(c7dc0000,ef37ecec,ef37ed28,c0e97daa,0,...) at write+0x4f syscallenter(c7dc0000,ef37ece4,ef37ece4,0,c102b700,...) at syscallenter+0x263 syscall(ef37ed28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x28181623, esp = 0xbfbfe78c, ebp = 0xbfbfe838 --- Tracing command suj17 pid 4812 tid 100210 td 0xc8b972e0 sched_switch(c8b972e0,0,103,18c,7bac76d9,...) at sched_switch+0x3bc mi_switch(103,0,c0e97e7c,2e1,6d8,...) at mi_switch+0x1eb turnstile_wait(c7e3dd00,c7dc48a0,0,1ac,c1194954,...) at turnstile_wait+0x4a4 _mtx_lock_sleep(c1194954,c8b972e0,0,c0e9e7cb,72c,...) at _mtx_lock_sleep+0x14d _mtx_lock_flags(c1194954,0,c0e9e7cb,72c,c099d6b3,...) at _mtx_lock_flags+0xf7 getnewbuf(4000,4000,0,ad7,ffffffe0,...) at getnewbuf+0x5e getblk(c7f85828,1364,0,4000,0,...) at getblk+0x485 ffs_balloc_ufs2(c7f85828,4d90000,0,4000,c8b3a000,...) at ffs_balloc_ufs2+0x1c51 ffs_write(ef3ecb94,c0ee68a3,c7f85828,0,c7f85828,...) at ffs_write+0x2ad VOP_WRITE_APV(c0fc7500,ef3ecb94,c7f85828,264,0,...) at VOP_WRITE_APV+0x136 vn_write(c78593f0,ef3ecc28,c8b3a000,0,c8b972e0,...) at vn_write+0x1ca dofilewrite(ef3ecc28,ffffffff,ffffffff,0,c78593f0,...) at dofilewrite+0x95 kern_writev(c8b972e0,3,ef3ecc28,ef3ecc48,1,...) at kern_writev+0x58 write(c8b972e0,ef3eccec,ef3ecd28,c0e97daa,0,...) at write+0x4f syscallenter(c8b972e0,ef3ecce4,ef3ecce4,0,0,...) at syscallenter+0x263 syscall(ef3ecd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x28181623, esp = 0xbfbfe78c, ebp = 0xbfbfe838 --- Tracing command suj17 pid 4811 tid 100120 td 0xc77218a0 cpustop_handler(8,ef2713a4,c0cdb156,f4,0,...) at cpustop_handler+0x49 ipi_nmi_handler(f4,0,1ad43b8,c099d8f9,c7717578,...) at ipi_nmi_handler+0x44 trap(ef2713b0) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc099d8f9, esp = 0xef2713f0, ebp = 0xef27140c --- _mtx_lock_sleep(c119b230,c77218a0,0,c0ec4aa0,24b3,...) at _mtx_lock_sleep+0x99 _mtx_lock_flags(c119b230,0,c0ec4aa0,24b3,c0e92869,...) at _mtx_lock_flags+0xf7 softdep_disk_io_initiation(e09619a0,0,e09619a0,0,0,...) at softdep_disk_io_initiation+0x108 ffs_geom_strategy(c7fa9e90,e09619a0,df,e09619a0,e6d151c0,...) at ffs_geom_strategy+0x14f bufwrite(e09619a0,0,c0ec7950,768,0) at bufwrite+0x159 ffs_bufwrite(e09619a0,0,c0ec4aa0,ca5,ef27161c,...) at ffs_bufwrite+0x290 softdep_process_journal(ef2717bc,ef2717bc,ef2717d8,c0bdbaec,c119b230,...) at softdep_process_journal+0x7b8 jwait(c119b230,0,c0ec4aa0,24b3,722,...) at jwait+0x66 softdep_disk_io_initiation(e09baac0,c2bde6b0,0,e09baac0,c7ede000,...) at softdep_disk_io_initiation+0x193c ffs_geom_strategy(c7fa9e90,e09baac0,ef271824,3,c77218a0,...) at ffs_geom_strategy+0x14f ufs_strategy(ef271860,c0ee6668,c77218a0,e09baac0,4,...) at ufs_strategy+0xd3 VOP_STRATEGY_APV(c0fc7500,ef271860,c7ede0f8,c0fe2560,c7ede000,...) at VOP_STRATEGY_APV+0xe5 bufstrategy(c7ede0f8,e09baac0,3a6,c773b800,0,...) at bufstrategy+0x61 bufwrite(e09baac0,c7ede000,ef271ad0,c0bc1239,e09baac0,...) at bufwrite+0x159 bawrite(e09baac0,4ce2964,0,1,c8b3a000,...) at bawrite+0x5c ffs_truncate(c7ede000,4ce2965,0,800,c8b3a000,...) at ffs_truncate+0x7c9 ufs_setattr(ef271bf0,c0ee68e6,c7ede000,ef271b88,c7ede000,...) at ufs_setattr+0x738 VOP_SETATTR_APV(c0fc7500,ef271bf0,c7ede000,28f,0,...) at VOP_SETATTR_APV+0xc5 vn_truncate(c7fdb540,4ce2965,0,c8b3a000,c77218a0,...) at vn_truncate+0x145 kern_ftruncate(c77218a0,3,4ce2965,0,ef271c7c,...) at kern_ftruncate+0xe7 ftruncate(c77218a0,ef271cec,ef271d28,c0e97daa,0,...) at ftruncate+0x28 syscallenter(c77218a0,ef271ce4,ef271ce4,0,c102b700,...) at syscallenter+0x263 syscall(ef271d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (480, FreeBSD ELF32, ftruncate), eip = 0x28169e6b, esp = 0xbfbfe75c, ebp = 0xbfbfe788 --- Tracing command suj17 pid 4810 tid 100094 td 0xc74838a0 kdb_enter(c0e91c42,c0e91c42,c0ec4b7d,ef1f2640,2,...) at kdb_enter+0x3a panic(c0ec4b7d,c867c700,c0ec72bd,8009,e0860780,...) at panic+0x134 worklist_insert(c8e42000,c0fc4c28,c0ec4aa0,170d,0,...) at worklist_insert+0x73 freework_freeblock(c119b230,0,c0ec4aa0,1e00,8,...) at freework_freeblock+0x31a indir_trunc(fffffff4,ffffffff,c0ec4aa0,1c94,8,...) at indir_trunc+0x665 handle_workitem_freeblocks(c119b230,0,c0ec4aa0,1897,c8b3a000,...) at handle_workitem_freeblocks+0x24f softdep_journal_freeblocks(c7ebb0e8,c8b3a000,1d20a9f,0,800,...) at softdep_journal_freeblocks+0x10c4 ffs_truncate(c7e68570,1d20a9f,0,800,c8b3a000,...) at ffs_truncate+0x86d ufs_setattr(ef1f2bf0,c0ee68e6,c7e68570,ef1f2b88,c7e68570,...) at ufs_setattr+0x738 VOP_SETATTR_APV(c0fc7500,ef1f2bf0,c7e68570,28f,0,...) at VOP_SETATTR_APV+0xc5 vn_truncate(c7e472d8,1d20a9f,0,c8b3a000,c74838a0,...) at vn_truncate+0x145 kern_ftruncate(c74838a0,3,1d20a9f,0,ef1f2c7c,...) at kern_ftruncate+0xe7 ftruncate(c74838a0,ef1f2cec,ef1f2d28,c0e97daa,0,...) at ftruncate+0x28 syscallenter(c74838a0,ef1f2ce4,ef1f2ce4,0,c102c400,...) at syscallenter+0x263 syscall(ef1f2d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (480, FreeBSD ELF32, ftruncate), eip = 0x28169e6b, esp = 0xbfbfe75c, ebp = 0xbfbfe788 --- Tracing command suj17 pid 4809 tid 100153 td 0xc7dc48a0 cpustop_handler(1,ef32763c,c0cdb156,f4,0,...) at cpustop_handler+0x49 ipi_nmi_handler(f4,0,161a7e8,c099d907,c7dbdaf0,...) at ipi_nmi_handler+0x44 trap(ef327648) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc099d8f9, esp = 0xef327688, ebp = 0xef3276a4 --- _mtx_lock_sleep(c119b230,c7dc48a0,0,c0ec4aa0,4b2,...) at _mtx_lock_sleep+0x99 _mtx_lock_flags(c119b230,0,c0ec4aa0,4b2,e086e420,...) at _mtx_lock_flags+0xf7 workitem_alloc(3c,c0fc4b58,2,4,c773b800,...) at workitem_alloc+0x3d softdep_setup_blkmapdep(e086e420,c73f35a8,17d48,0,8,...) at softdep_setup_blkmapdep+0x4d ffs_alloccgblk(17d38,0,4000,5b6,0,...) at ffs_alloccgblk+0x2f3 ffs_alloccg(c7ef0570,1,17d38,0,4000,...) at ffs_alloccg+0x1bb ffs_hashalloc(17d38,0,4000,4000,c0bb8d80,...) at ffs_hashalloc+0x85 ffs_alloc(c7ef0570,8da,0,17d38,0,...) at ffs_alloc+0x2ef ffs_balloc_ufs2(c7f5cae0,2368000,0,4000,c8b3a000,...) at ffs_balloc_ufs2+0x1b5a ffs_write(ef327b94,c0ee68a3,c7f5cae0,0,c7f5cae0,...) at ffs_write+0x2ad VOP_WRITE_APV(c0fc7500,ef327b94,c7f5cae0,264,0,...) at VOP_WRITE_APV+0x136 vn_write(c7e425e8,ef327c28,c8b3a000,0,c7dc48a0,...) at vn_write+0x1ca dofilewrite(ef327c28,ffffffff,ffffffff,0,c7e425e8,...) at dofilewrite+0x95 kern_writev(c7dc48a0,3,ef327c28,ef327c48,1,...) at kern_writev+0x58 write(c7dc48a0,ef327cec,ef327d28,c0e97daa,0,...) at write+0x4f syscallenter(c7dc48a0,ef327ce4,ef327ce4,0,74b,...) at syscallenter+0x263 syscall(ef327d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x28181623, esp = 0xbfbfe78c, ebp = 0xbfbfe838 --- Tracing command suj17 pid 4808 tid 100190 td 0xc78388a0 sched_switch(c78388a0,0,104,191,47e65433,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c78388a0,0,c0e97420,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7759834,6c,c0e9a1fe,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7759834,c77598bc,16c,c0e9a1fe,0,...) at _sleep+0x35c kern_wait(c78388a0,ffffffff,ef396c44,0,0,...) at kern_wait+0xbe6 wait4(c78388a0,ef396cec,28195d28,1,0,...) at wait4+0x3b syscallenter(c78388a0,ef396ce4,c0cdb89d,c101d170,0,...) at syscallenter+0x263 syscall(ef396d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x280f093b, esp = 0xbfbfe81c, ebp = 0xbfbfe838 --- Tracing command bash pid 4807 tid 100188 td 0xc82ec5c0 sched_switch(c82ec5c0,0,104,191,47950a9c,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c82ec5c0,0,c0e97420,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8018578,6c,c0e9a1fe,100,0,...) at sleepq_wait_sig+0x17 _sleep(c8018578,c8018600,16c,c0e9a1fe,0,...) at _sleep+0x35c kern_wait(c82ec5c0,ffffffff,ef390c44,0,0,...) at kern_wait+0xbe6 wait4(c82ec5c0,ef390cec,28368d28,1,0,...) at wait4+0x3b syscallenter(c82ec5c0,ef390ce4,c0cdb89d,c101d170,0,...) at syscallenter+0x263 syscall(ef390d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x282c393b, esp = 0xbfbfe3dc, ebp = 0xbfbfe3f8 --- Tracing command su pid 4806 tid 100189 td 0xc7838b80 sched_switch(c7838b80,0,104,191,46cfc814,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c7838b80,0,c0e97420,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7759af0,6c,c0e9a1fe,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7759af0,c7759b78,16c,c0e9a1fe,0,...) at _sleep+0x35c kern_wait(c7838b80,12c7,ef393c44,2,0,...) at kern_wait+0xbe6 wait4(c7838b80,ef393cec,281c8d28,1,0,...) at wait4+0x3b syscallenter(c7838b80,ef393ce4,c0cdb89d,c101d170,0,...) at syscallenter+0x263 syscall(ef393d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x2812393b, esp = 0xbfbfe31c, ebp = 0xbfbfe338 --- Tracing command md5 pid 4801 tid 100139 td 0xc79bb5c0 sched_switch(c79bb5c0,0,104,191,7b7c5e1f,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c79bb5c0,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c7e30800,5c,c0e3da91,0,0,...) at sleepq_wait+0x63 _sleep(c7e30800,c7e30820,25c,c0e3da91,0,...) at _sleep+0x372 md_kthread(c7e30800,ef2d0d28,c0e8c598,390,c785d2bc,...) at md_kthread+0x11a fork_exit(c072b540,c7e30800,ef2d0d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xef2d0d60, ebp = 0 --- Tracing command sh pid 4782 tid 100219 td 0xc8bd9b80 sched_switch(c8bd9b80,0,104,191,4584c8c7,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c8bd9b80,0,c0e97420,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8be5af0,6c,c0e9a1fe,100,0,...) at sleepq_wait_sig+0x17 _sleep(c8be5af0,c8be5b78,16c,c0e9a1fe,0,...) at _sleep+0x35c kern_wait(c8bd9b80,ffffffff,ef407c44,2,0,...) at kern_wait+0xbe6 wait4(c8bd9b80,ef407cec,ef407d28,c0e97daa,0,...) at wait4+0x3b syscallenter(c8bd9b80,ef407ce4,ef407ce4,0,0,...) at syscallenter+0x263 syscall(ef407d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x2816a93b, esp = 0xbfbfe82c, ebp = 0xbfbfe848 --- Tracing command awk pid 2949 tid 100088 td 0xc74845c0 sched_switch(c74845c0,0,104,191,7eb1fb1d,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c74845c0,0,c0e97420,1a7,5c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c786c7a8,5c,c0e99c2f,100,0,...) at sleepq_wait_sig+0x17 _sleep(c786c7a8,c786c918,15c,c0e99c2f,0,...) at _sleep+0x35c pipe_read(c74a66c8,ef1c6c28,c8360800,0,c74845c0,...) at pipe_read+0x417 dofileread(ef1c6c28,ffffffff,ffffffff,0,c74a66c8,...) at dofileread+0x9e kern_readv(c74845c0,0,ef1c6c28,ef1c6c48,1,...) at kern_readv+0x58 read(c74845c0,ef1c6cec,ef1c6d80,ef1c6c88,0,...) at read+0x4f syscallenter(c74845c0,ef1c6ce4,c0cdb556,0,0,...) at syscallenter+0x263 syscall(ef1c6d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x281b8643, esp = 0xbfbfe93c, ebp = 0xbfbfe958 --- Tracing command sh pid 2948 tid 100155 td 0xc7dc42e0 sched_switch(c7dc42e0,0,104,191,7f55dee1,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c7dc42e0,0,c0e97420,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7dbd578,6c,c0e9a1fe,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7dbd578,c7dbd600,16c,c0e9a1fe,0,...) at _sleep+0x35c kern_wait(c7dc42e0,ffffffff,ef32dc44,2,0,...) at kern_wait+0xbe6 wait4(c7dc42e0,ef32dcec,ef32dd28,c0e97daa,0,...) at wait4+0x3b syscallenter(c7dc42e0,ef32dce4,ef32dce4,0,0,...) at syscallenter+0x263 syscall(ef32dd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x2816a93b, esp = 0xbfbfe98c, ebp = 0xbfbfe9a8 --- Tracing command top pid 2947 tid 100079 td 0xc742f5c0 sched_switch(c742f5c0,0,104,191,6d1691ac,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c742f5c0,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(ef1a1a14,c099d73a,c783c090,0,c742f5c0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c783c0a4,0,ef1a1a4c,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c783c0a4,c783c090,3e9,627,c7859b60,...) at _cv_timedwait_sig+0x252 seltdwait(ef1a1bf8,ef1a1c00,c7d1b000,c742f5c0,5b1b3348,...) at seltdwait+0x8a kern_select(c742f5c0,2,bfbfebac,0,0,ef1a1c40,20,1,0) at kern_select+0x504 select(c742f5c0,ef1a1cec,ef1a1d28,c0e97daa,0,...) at select+0x66 syscallenter(c742f5c0,ef1a1ce4,ef1a1ce4,0,0,...) at syscallenter+0x263 syscall(ef1a1d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x281ee5c3, esp = 0xbfbfeb4c, ebp = 0xbfbfeca8 --- Tracing command sshd pid 2946 tid 100183 td 0xc82ed5c0 sched_switch(c82ed5c0,0,104,191,6d1a6cad,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c82ed5c0,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c099d73a,c783ca50,0,c0e90227,c82ed5c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c783ca64,0,ef381a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c783ca64,c783ca50,c0e99a71,627,c7dbbcb0,...) at _cv_wait_sig+0x243 seltdwait(c7dbbcb0,58,c73f5100,c82ed5c0,246,...) at seltdwait+0xa2 kern_select(c82ed5c0,a,288060b4,288060bc,0,0,20,c8019578,c8019578) at kern_select+0x504 select(c82ed5c0,ef381cec,ef381d28,c0e97daa,0,...) at select+0x66 syscallenter(c82ed5c0,ef381ce4,ef381ce4,0,0,...) at syscallenter+0x263 syscall(ef381d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x284c45c3, esp = 0xbfbfe19c, ebp = 0xbfbfe1e8 --- Tracing command tail pid 2945 tid 100105 td 0xc783e5c0 sched_switch(c783e5c0,0,104,191,14fe2740,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,68,...) at mi_switch+0x1eb sleepq_switch(c783e5c0,0,c0e97420,1a7,68,...) at sleepq_switch+0x162 sleepq_catch_signals(3e9,c09ec990,c783e5c0,3,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c74b7780,68,c0e8c19c,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c74b7780,c74b7780,168,c0e8c19c,3e9,...) at _sleep+0x328 kern_kevent(c783e5c0,4,0,1,ef241c28,...) at kern_kevent+0x364 kevent(c783e5c0,ef241cec,ef241d28,c0e97daa,0,...) at kevent+0x19e syscallenter(c783e5c0,ef241ce4,ef241ce4,0,0,...) at syscallenter+0x263 syscall(ef241d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (363, FreeBSD ELF32, kevent), eip = 0x28169cab, esp = 0xbfbfeb1c, ebp = 0xbfbfebd8 --- Tracing command sh pid 2944 tid 100114 td 0xc7863000 sched_switch(c7863000,0,104,191,5d739e0e,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c7863000,0,c0e97420,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c785a578,6c,c0e9a1fe,100,0,...) at sleepq_wait_sig+0x17 _sleep(c785a578,c785a600,16c,c0e9a1fe,0,...) at _sleep+0x35c kern_wait(c7863000,ffffffff,ef25dc44,2,0,...) at kern_wait+0xbe6 wait4(c7863000,ef25dcec,2820fd28,1,0,...) at wait4+0x3b syscallenter(c7863000,ef25dce4,c0cdb89d,c101d170,0,...) at syscallenter+0x263 syscall(ef25dd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x2816a93b, esp = 0xbfbfeb3c, ebp = 0xbfbfeb58 --- Tracing command sshd pid 2943 tid 100084 td 0xc742a8a0 sched_switch(c742a8a0,0,104,191,1d67d514,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c742a8a0,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c099d73a,c783c0d0,0,c0e90227,c742a8a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c783c0e4,0,ef1b5a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c783c0e4,c783c0d0,c0e99a71,627,c74a6700,...) at _cv_wait_sig+0x243 seltdwait(c74a6700,58,c74b7380,c742a8a0,246,...) at seltdwait+0xa2 kern_select(c742a8a0,c,288060b4,288060bc,0,0,20,c73a3834,c73a3834) at kern_select+0x504 select(c742a8a0,ef1b5cec,ef1b5d28,c0e97daa,0,...) at select+0x66 syscallenter(c742a8a0,ef1b5ce4,ef1b5ce4,0,0,...) at syscallenter+0x263 syscall(ef1b5d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x284c45c3, esp = 0xbfbfe19c, ebp = 0xbfbfe1e8 --- Tracing command sshd pid 2942 tid 100160 td 0xc7dc32e0 sched_switch(c7dc32e0,0,104,191,591dd550,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7dc32e0,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c099d73a,c783c9d0,0,c0e90227,c7dc32e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c783c9e4,0,ef33ca4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c783c9e4,c783c9d0,c0e99a71,627,c74331c0,...) at _cv_wait_sig+0x243 seltdwait(c74331c0,58,c7d1b800,c7dc32e0,f33,...) at seltdwait+0xa2 kern_select(c7dc32e0,c,288060b4,288060bc,0,0,20,c7dbc578,c7dbc578) at kern_select+0x504 select(c7dc32e0,ef33ccec,ef33cd28,c0e97daa,0,...) at select+0x66 syscallenter(c7dc32e0,ef33cce4,ef33cce4,0,0,...) at syscallenter+0x263 syscall(ef33cd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x284c45c3, esp = 0xbfbfe19c, ebp = 0xbfbfe1e8 --- Tracing command sshd pid 2938 tid 100169 td 0xc783eb80 sched_switch(c783eb80,0,104,191,5b37d045,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,68,...) at mi_switch+0x1eb sleepq_switch(c783eb80,0,c0e97420,1a7,68,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7b168d8,68,c0e9d8c1,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7b168d8,c7b16898,168,c0e9d8c1,0) at _sleep+0x35c sbwait(c7b16874,4,c0e9d999,5ec,c7b16898,...) at sbwait+0x76 soreceive_generic(c7b16820,0,ef357c28,0,0,...) at soreceive_generic+0x3f0 soreceive(c7b16820,0,ef357c28,0,0,0) at soreceive+0x38 soo_read(c7e47d20,ef357c28,c7d1b480,0,c783eb80,...) at soo_read+0x4e dofileread(ef357c28,ffffffff,ffffffff,0,c7e47d20,...) at dofileread+0x9e kern_readv(c783eb80,5,ef357c28,ef357c48,1,...) at kern_readv+0x58 read(c783eb80,ef357cec,2888a010,1,0,...) at read+0x4f syscallenter(c783eb80,ef357ce4,c0cdb89d,c101d170,0,...) at syscallenter+0x263 syscall(ef357d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x284c4643, esp = 0xbfbfdd3c, ebp = 0xbfbfdd78 --- Tracing command sshd pid 2937 tid 100171 td 0xc7e3e2e0 sched_switch(c7e3e2e0,0,104,191,5977b452,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,68,...) at mi_switch+0x1eb sleepq_switch(c7e3e2e0,0,c0e97420,1a7,68,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c83563f8,68,c0e9d8c1,100,0,...) at sleepq_wait_sig+0x17 _sleep(c83563f8,c83563b8,168,c0e9d8c1,0) at _sleep+0x35c sbwait(c8356394,4,c0e9d999,5ec,c83563b8,...) at sbwait+0x76 soreceive_generic(c8356340,0,ef35dc28,0,0,...) at soreceive_generic+0x3f0 soreceive(c8356340,0,ef35dc28,0,0,0) at soreceive+0x38 soo_read(c74a6d20,ef35dc28,c74a1000,0,c7e3e2e0,...) at soo_read+0x4e dofileread(ef35dc28,ffffffff,ffffffff,0,c74a6d20,...) at dofileread+0x9e kern_readv(c7e3e2e0,5,ef35dc28,ef35dc48,1,...) at kern_readv+0x58 read(c7e3e2e0,ef35dcec,ef35dd28,c0e97daa,0,...) at read+0x4f syscallenter(c7e3e2e0,ef35dce4,ef35dce4,0,0,...) at syscallenter+0x263 syscall(ef35dd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x284c4643, esp = 0xbfbfdd3c, ebp = 0xbfbfdd78 --- Tracing command sshd pid 2936 tid 100096 td 0xc74832e0 sched_switch(c74832e0,0,104,191,59052786,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,68,...) at mi_switch+0x1eb sleepq_switch(c74832e0,0,c0e97420,1a7,68,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7fdd3f8,68,c0e9d8c1,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7fdd3f8,c7fdd3b8,168,c0e9d8c1,0) at _sleep+0x35c sbwait(c7fdd394,4,c0e9d999,5ec,c7fdd3b8,...) at sbwait+0x76 soreceive_generic(c7fdd340,0,ef1f8c28,0,0,...) at soreceive_generic+0x3f0 soreceive(c7fdd340,0,ef1f8c28,0,0,0) at soreceive+0x38 soo_read(c7433930,ef1f8c28,c7d1b080,0,c74832e0,...) at soo_read+0x4e dofileread(ef1f8c28,ffffffff,ffffffff,0,c7433930,...) at dofileread+0x9e kern_readv(c74832e0,5,ef1f8c28,ef1f8c48,1,...) at kern_readv+0x58 read(c74832e0,ef1f8cec,ef1f8d28,c0e97daa,0,...) at read+0x4f syscallenter(c74832e0,ef1f8ce4,ef1f8ce4,0,0,...) at syscallenter+0x263 syscall(ef1f8d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x284c4643, esp = 0xbfbfdd3c, ebp = 0xbfbfdd78 --- Tracing command sh pid 2808 tid 100150 td 0xc7863b80 sched_switch(c7863b80,0,104,191,21436ad7,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c7863b80,0,c0e97420,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7dbe578,6c,c0e9a1fe,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7dbe578,c7dbe600,16c,c0e9a1fe,0,...) at _sleep+0x35c kern_wait(c7863b80,ffffffff,ef31ec44,2,0,...) at kern_wait+0xbe6 wait4(c7863b80,ef31ecec,ef31ed28,c0e97daa,0,...) at wait4+0x3b syscallenter(c7863b80,ef31ece4,ef31ece4,0,0,...) at syscallenter+0x263 syscall(ef31ed28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x2816a93b, esp = 0xbfbfe6fc, ebp = 0xbfbfe718 --- Tracing command bash pid 2804 tid 100151 td 0xc7dc6000 sched_switch(c7dc6000,0,104,191,83adaadf,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c7dc6000,0,c0e97420,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7dbe2bc,6c,c0e9a1fe,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7dbe2bc,c7dbe344,16c,c0e9a1fe,0,...) at _sleep+0x35c kern_wait(c7dc6000,ffffffff,ef321c44,6,0,...) at kern_wait+0xbe6 wait4(c7dc6000,ef321cec,ef321d28,c0e97daa,0,...) at wait4+0x3b syscallenter(c7dc6000,ef321ce4,ef321ce4,0,0,...) at syscallenter+0x263 syscall(ef321d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x282c393b, esp = 0xbfbfe76c, ebp = 0xbfbfe788 --- Tracing command csh pid 2800 tid 100140 td 0xc783db80 sched_switch(c783db80,0,104,191,875973af,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,74,...) at mi_switch+0x1eb sleepq_switch(c783db80,0,c0e97420,1a7,74,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7835b48,74,c0d29e0e,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7835b48,c7835b78,174,c0d29e0e,0,...) at _sleep+0x35c kern_sigsuspend(c783db80,0,0,0,0,...) at kern_sigsuspend+0xba sigsuspend(c783db80,ef2d3cec,ef2d3d28,c0e97daa,0,...) at sigsuspend+0x4d syscallenter(c783db80,ef2d3ce4,ef2d3ce4,0,0,...) at syscallenter+0x263 syscall(ef2d3d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x2818ac7b, esp = 0xbfbfe58c, ebp = 0xbfbfe5f8 --- Tracing command su pid 2799 tid 100146 td 0xc783d2e0 sched_switch(c783d2e0,0,104,191,3b9ddbe7,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c783d2e0,0,c0e97420,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c78352bc,6c,c0e9a1fe,100,0,...) at sleepq_wait_sig+0x17 _sleep(c78352bc,c7835344,16c,c0e9a1fe,0,...) at _sleep+0x35c kern_wait(c783d2e0,af0,ef2e8c44,2,0,...) at kern_wait+0xbe6 wait4(c783d2e0,ef2e8cec,281c8d28,1,0,...) at wait4+0x3b syscallenter(c783d2e0,ef2e8ce4,c0cdb89d,c101d170,0,...) at syscallenter+0x263 syscall(ef2e8d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x2812393b, esp = 0xbfbfe5bc, ebp = 0xbfbfe5d8 --- Tracing command bash pid 2229 tid 100159 td 0xc7dc35c0 sched_switch(c7dc35c0,0,104,191,fe2d7cc3,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c7dc35c0,0,c0e97420,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7dbc834,6c,c0e9a1fe,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7dbc834,c7dbc8bc,16c,c0e9a1fe,0,...) at _sleep+0x35c kern_wait(c7dc35c0,ffffffff,ef339c44,6,0,...) at kern_wait+0xbe6 wait4(c7dc35c0,ef339cec,ef339d28,c0e97daa,0,...) at wait4+0x3b syscallenter(c7dc35c0,ef339ce4,ef339ce4,0,0,...) at syscallenter+0x263 syscall(ef339d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x282c393b, esp = 0xbfbfe97c, ebp = 0xbfbfe998 --- Tracing command sshd pid 2228 tid 100132 td 0xc79bcb80 sched_switch(c79bcb80,0,104,191,43e3bd4a,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c79bcb80,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c099d73a,c7b30010,0,c0e90227,c79bcb80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7b30024,0,ef2bba4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7b30024,c7b30010,c0e99a71,627,c7879d58,...) at _cv_wait_sig+0x243 seltdwait(c7879d58,58,c74b7900,c79bcb80,246,...) at seltdwait+0xa2 kern_select(c79bcb80,c,288060b4,288060bc,0,0,20,c79b9834,c79b9834) at kern_select+0x504 select(c79bcb80,ef2bbcec,ef2bbd28,c0e97daa,0,...) at select+0x66 syscallenter(c79bcb80,ef2bbce4,ef2bbce4,0,0,...) at syscallenter+0x263 syscall(ef2bbd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x284c45c3, esp = 0xbfbfe19c, ebp = 0xbfbfe1e8 --- Tracing command sshd pid 2226 tid 100131 td 0xc79bd000 sched_switch(c79bd000,0,104,191,e4396ed1,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,68,...) at mi_switch+0x1eb sleepq_switch(c79bd000,0,c0e97420,1a7,68,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7a2da78,68,c0e9d8c1,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7a2da78,c7a2da38,168,c0e9d8c1,0) at _sleep+0x35c sbwait(c7a2da14,4,c0e9d999,5ec,c7a2da38,...) at sbwait+0x76 soreceive_generic(c7a2d9c0,0,ef2b8c28,0,0,...) at soreceive_generic+0x3f0 soreceive(c7a2d9c0,0,ef2b8c28,0,0,0) at soreceive+0x38 soo_read(c7879818,ef2b8c28,c74b7080,0,c79bd000,...) at soo_read+0x4e dofileread(ef2b8c28,ffffffff,ffffffff,0,c7879818,...) at dofileread+0x9e kern_readv(c79bd000,5,ef2b8c28,ef2b8c48,1,...) at kern_readv+0x58 read(c79bd000,ef2b8cec,ef2b8d28,c0e97daa,0,...) at read+0x4f syscallenter(c79bd000,ef2b8ce4,ef2b8ce4,0,0,...) at syscallenter+0x263 syscall(ef2b8d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x284c4643, esp = 0xbfbfdd3c, ebp = 0xbfbfdd78 --- Tracing command getty pid 2225 tid 100113 td 0xc78632e0 sched_switch(c78632e0,0,104,191,80530745,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c78632e0,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,c78632e0,c78632e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7181070,0,c0e9c13e,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7181070,c101d170,c0e9badc,514,0,...) at _cv_wait_sig+0x243 tty_wait(c7181000,c7181070,ef25ac28,1,0,...) at tty_wait+0x71 ttydisc_read(c7181000,ef25ac28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c705e600,ef25ac28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7879a48,ef25ac28,c6d9fe00,0,c78632e0,...) at devfs_read_f+0x7e dofileread(ef25ac28,ffffffff,ffffffff,0,c7879a48,...) at dofileread+0x9e kern_readv(c78632e0,0,ef25ac28,ef25ac48,1,...) at kern_readv+0x58 read(c78632e0,ef25acec,ef25ad28,c0e97daa,0,...) at read+0x4f syscallenter(c78632e0,ef25ace4,ef25ace4,0,0,...) at syscallenter+0x263 syscall(ef25ad28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196643, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2224 tid 100118 td 0xc7722000 sched_switch(c7722000,0,104,191,806cb7ca,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7722000,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,c7722000,c7722000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7181270,0,c0e9c13e,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7181270,c101d170,c0e9badc,514,0,...) at _cv_wait_sig+0x243 tty_wait(c7181200,c7181270,ef269c28,1,0,...) at tty_wait+0x71 ttydisc_read(c7181200,ef269c28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c705e700,ef269c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c78341f8,ef269c28,c6d9fe00,0,c7722000,...) at devfs_read_f+0x7e dofileread(ef269c28,ffffffff,ffffffff,0,c78341f8,...) at dofileread+0x9e kern_readv(c7722000,0,ef269c28,ef269c48,1,...) at kern_readv+0x58 read(c7722000,ef269cec,ef269d28,c0e97daa,0,...) at read+0x4f syscallenter(c7722000,ef269ce4,ef269ce4,0,0,...) at syscallenter+0x263 syscall(ef269d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196643, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2223 tid 100128 td 0xc79bd8a0 sched_switch(c79bd8a0,0,104,191,807c2637,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c79bd8a0,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,c79bd8a0,c79bd8a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7181470,0,c0e9c13e,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7181470,c101d170,c0e9badc,514,0,...) at _cv_wait_sig+0x243 tty_wait(c7181400,c7181470,ef2afc28,1,0,...) at tty_wait+0x71 ttydisc_read(c7181400,ef2afc28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c705e800,ef2afc28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c78598c0,ef2afc28,c6d9fe00,0,c79bd8a0,...) at devfs_read_f+0x7e dofileread(ef2afc28,ffffffff,ffffffff,0,c78598c0,...) at dofileread+0x9e kern_readv(c79bd8a0,0,ef2afc28,ef2afc48,1,...) at kern_readv+0x58 read(c79bd8a0,ef2afcec,ef2afd28,c0e97daa,0,...) at read+0x4f syscallenter(c79bd8a0,ef2afce4,ef2afce4,0,0,...) at syscallenter+0x263 syscall(ef2afd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196643, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2222 tid 100107 td 0xc7177b80 sched_switch(c7177b80,0,104,191,807e14b6,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7177b80,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,c7177b80,c7177b80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6f59070,0,c0e9c13e,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6f59070,c101d170,c0e9badc,514,0,...) at _cv_wait_sig+0x243 tty_wait(c6f59000,c6f59070,ef248c28,1,0,...) at tty_wait+0x71 ttydisc_read(c6f59000,ef248c28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c705e900,ef248c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7dbb0a8,ef248c28,c6d9fe00,0,c7177b80,...) at devfs_read_f+0x7e dofileread(ef248c28,ffffffff,ffffffff,0,c7dbb0a8,...) at dofileread+0x9e kern_readv(c7177b80,0,ef248c28,ef248c48,1,...) at kern_readv+0x58 read(c7177b80,ef248cec,ef248d28,c0e97daa,0,...) at read+0x4f syscallenter(c7177b80,ef248ce4,ef248ce4,0,0,...) at syscallenter+0x263 syscall(ef248d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196643, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2221 tid 100134 td 0xc79bc5c0 sched_switch(c79bc5c0,0,104,191,80715671,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c79bc5c0,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,c79bc5c0,c79bc5c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6f59270,0,c0e9c13e,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6f59270,c101d170,c0e9badc,514,0,...) at _cv_wait_sig+0x243 tty_wait(c6f59200,c6f59270,ef2c1c28,1,0,...) at tty_wait+0x71 ttydisc_read(c6f59200,ef2c1c28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c705ea00,ef2c1c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c74a60a8,ef2c1c28,c6d9fe00,0,c79bc5c0,...) at devfs_read_f+0x7e dofileread(ef2c1c28,ffffffff,ffffffff,0,c74a60a8,...) at dofileread+0x9e kern_readv(c79bc5c0,0,ef2c1c28,ef2c1c48,1,...) at kern_readv+0x58 read(c79bc5c0,ef2c1cec,ef2c1d28,c0e97daa,0,...) at read+0x4f syscallenter(c79bc5c0,ef2c1ce4,ef2c1ce4,0,0,...) at syscallenter+0x263 syscall(ef2c1d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196643, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2220 tid 100109 td 0xc71775c0 sched_switch(c71775c0,0,104,191,8070144f,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c71775c0,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,c71775c0,c71775c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6f59470,0,c0e9c13e,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6f59470,c101d170,c0e9badc,514,0,...) at _cv_wait_sig+0x243 tty_wait(c6f59400,c6f59470,ef24ec28,1,0,...) at tty_wait+0x71 ttydisc_read(c6f59400,ef24ec28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c705eb00,ef24ec28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c74491c0,ef24ec28,c6d9fe00,0,c71775c0,...) at devfs_read_f+0x7e dofileread(ef24ec28,ffffffff,ffffffff,0,c74491c0,...) at dofileread+0x9e kern_readv(c71775c0,0,ef24ec28,ef24ec48,1,...) at kern_readv+0x58 read(c71775c0,ef24ecec,ef24ed28,c0e97daa,0,...) at read+0x4f syscallenter(c71775c0,ef24ece4,ef24ece4,0,0,...) at syscallenter+0x263 syscall(ef24ed28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196643, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2219 tid 100100 td 0xc74815c0 sched_switch(c74815c0,0,104,191,8078313d,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c74815c0,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,c74815c0,c74815c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6f59670,0,c0e9c13e,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6f59670,c101d170,c0e9badc,514,0,...) at _cv_wait_sig+0x243 tty_wait(c6f59600,c6f59670,ef204c28,1,0,...) at tty_wait+0x71 ttydisc_read(c6f59600,ef204c28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c705ec00,ef204c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7dbb038,ef204c28,c6d9fe00,0,c74815c0,...) at devfs_read_f+0x7e dofileread(ef204c28,ffffffff,ffffffff,0,c7dbb038,...) at dofileread+0x9e kern_readv(c74815c0,0,ef204c28,ef204c48,1,...) at kern_readv+0x58 read(c74815c0,ef204cec,ef204d28,c0e97daa,0,...) at read+0x4f syscallenter(c74815c0,ef204ce4,ef204ce4,0,0,...) at syscallenter+0x263 syscall(ef204d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196643, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2218 tid 100156 td 0xc7dc4000 sched_switch(c7dc4000,0,104,191,807d1528,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7dc4000,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,c7dc4000,c7dc4000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6f59870,0,c0e9c13e,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6f59870,c101d170,c0e9badc,514,0,...) at _cv_wait_sig+0x243 tty_wait(c6f59800,c6f59870,ef330c28,1,0,...) at tty_wait+0x71 ttydisc_read(c6f59800,ef330c28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c705ed00,ef330c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c78591c0,ef330c28,c6d9fe00,0,c7dc4000,...) at devfs_read_f+0x7e dofileread(ef330c28,ffffffff,ffffffff,0,c78591c0,...) at dofileread+0x9e kern_readv(c7dc4000,0,ef330c28,ef330c48,1,...) at kern_readv+0x58 read(c7dc4000,ef330cec,ef330d28,c0e97daa,0,...) at read+0x4f syscallenter(c7dc4000,ef330ce4,ef330ce4,0,0,...) at syscallenter+0x263 syscall(ef330d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196643, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2217 tid 100090 td 0xc7722b80 sched_switch(c7722b80,0,104,191,3c38a227,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7722b80,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c099d73a,c6f59a04,0,c0e90227,c7722b80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6f59a70,0,ef1e5ad8,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6f59a70,c6f59a04,c0e9badc,514,0,...) at _cv_wait_sig+0x243 tty_wait(c6f59a00,c6f59a70,ef1e5c28,1,0,...) at tty_wait+0x71 ttydisc_read(c6f59a00,ef1e5c28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c6ec6000,ef1e5c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7433d90,ef1e5c28,c6d9fe00,0,c7722b80,...) at devfs_read_f+0x7e dofileread(ef1e5c28,ffffffff,ffffffff,0,c7433d90,...) at dofileread+0x9e kern_readv(c7722b80,0,ef1e5c28,ef1e5c48,1,...) at kern_readv+0x58 read(c7722b80,ef1e5cec,ef1e5d28,c0e97daa,0,...) at read+0x4f syscallenter(c7722b80,ef1e5ce4,ef1e5ce4,0,0,...) at syscallenter+0x263 syscall(ef1e5d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196643, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command inetd pid 2177 tid 100127 td 0xc79bdb80 sched_switch(c79bdb80,0,104,191,6900291c,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c79bdb80,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c099d73a,c771fa90,0,c0e90227,c79bdb80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c771faa4,0,ef2aca4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c771faa4,c771fa90,c0e99a71,627,c7449150,...) at _cv_wait_sig+0x243 seltdwait(c7449150,58,c6d9fe00,c79bdb80,c0e90227,...) at seltdwait+0xa2 kern_select(c79bdb80,6,bfbfdfa0,0,0,0,20,c79ba834,c79ba834) at kern_select+0x504 select(c79bdb80,ef2accec,ef2acd28,c0e97daa,0,...) at select+0x66 syscallenter(c79bdb80,ef2acce4,ef2acce4,0,0,...) at syscallenter+0x263 syscall(ef2acd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x281a85c3, esp = 0xbfbfdf5c, ebp = 0xbfbfede8 --- Tracing command watchdogd pid 2153 tid 100147 td 0xc783d000 sched_switch(c783d000,0,104,191,88fe4220,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c783d000,0,c0e97420,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(ea61,c09ec990,c783d000,0,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c101ea84,6c,c0e93177,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c101ea84,0,16c,c0e93177,ea61,...) at _sleep+0x328 kern_nanosleep(c783d000,ef2ecc34,ef2ecc3c,3c,0,...) at kern_nanosleep+0xc1 nanosleep(c783d000,ef2eccec,ef2ecd28,c0e97daa,0,...) at nanosleep+0x6f syscallenter(c783d000,ef2ecce4,ef2ecce4,0,0,...) at syscallenter+0x263 syscall(ef2ecd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x28191b77, esp = 0xbfbfecbc, ebp = 0xbfbfece8 --- Tracing command cron pid 2106 tid 100123 td 0xc78622e0 sched_switch(c78622e0,0,104,191,bda3030d,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c78622e0,0,c0e97420,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(ea61,c09ec990,c78622e0,2,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c101ea84,6c,c0e93177,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c101ea84,0,16c,c0e93177,ea61,...) at _sleep+0x328 kern_nanosleep(c78622e0,ef27fc34,ef27fc3c,3c,0,...) at kern_nanosleep+0xc1 nanosleep(c78622e0,ef27fcec,ef27fd28,c0e97daa,0,...) at nanosleep+0x6f syscallenter(c78622e0,ef27fce4,ef27fce4,0,0,...) at syscallenter+0x263 syscall(ef27fd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x28184b77, esp = 0xbfbfec4c, ebp = 0xbfbfec78 --- Tracing command sendmail pid 2098 tid 100108 td 0xc71778a0 sched_switch(c71778a0,0,104,191,c0189321,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,74,...) at mi_switch+0x1eb sleepq_switch(c71778a0,0,c0e97420,1a7,74,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c785c88c,74,c0d29e0e,100,0,...) at sleepq_wait_sig+0x17 _sleep(c785c88c,c785c8bc,174,c0d29e0e,0,...) at _sleep+0x35c kern_sigsuspend(c71778a0,0,0,0,0,...) at kern_sigsuspend+0xba sigsuspend(c71778a0,ef24bcec,ef24bd28,c0e97daa,0,...) at sigsuspend+0x4d syscallenter(c71778a0,ef24bce4,ef24bce4,0,0,...) at syscallenter+0x263 syscall(ef24bd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x28338c7b, esp = 0xbfbfcf9c, ebp = 0xbfbfcfc8 --- Tracing command sendmail pid 2094 tid 100135 td 0xc79bc2e0 sched_switch(c79bc2e0,0,104,191,5b6150c7,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c79bc2e0,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(ef2c4a14,c099d73a,c742e650,0,c79bc2e0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c742e664,0,ef2c4a4c,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c742e664,c742e650,1389,627,c7449310,...) at _cv_timedwait_sig+0x252 seltdwait(ef2c4bf8,ef2c4c00,c701c500,c79bc2e0,ef2c4af8,...) at seltdwait+0x8a kern_select(c79bc2e0,5,bfbfc510,0,0,ef2c4c40,20,5,0) at kern_select+0x504 select(c79bc2e0,ef2c4cec,ef2c4d28,c0e97daa,0,...) at select+0x66 syscallenter(c79bc2e0,ef2c4ce4,ef2c4ce4,0,0,...) at syscallenter+0x263 syscall(ef2c4d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x283e05c3, esp = 0xbfbfc47c, ebp = 0xbfbfcfa8 --- Tracing command sshd pid 2086 tid 100081 td 0xc742f000 sched_switch(c742f000,0,104,191,5a5dea52,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c742f000,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c099d73a,c742e690,0,c0e90227,c742f000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c742e6a4,0,ef1a9a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c742e6a4,c742e690,c0e99a71,627,c78599d8,...) at _cv_wait_sig+0x243 seltdwait(c78599d8,58,c701c180,c742f000,ef1a9ab4,...) at seltdwait+0xa2 kern_select(c742f000,5,288cc0b4,0,0,0,20,c7429000,c7429000) at kern_select+0x504 select(c742f000,ef1a9cec,ef1a9d28,c0e97daa,0,...) at select+0x66 syscallenter(c742f000,ef1a9ce4,ef1a9ce4,0,0,...) at syscallenter+0x263 syscall(ef1a9d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x284c45c3, esp = 0xbfbfe25c, ebp = 0xbfbfedf8 --- Tracing command moused pid 2026 tid 100136 td 0xc79bc000 sched_switch(c79bc000,0,104,191,bc04f8d8,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c79bc000,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c099d73a,c7b30690,0,c0e90227,c79bc000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7b306a4,0,ef2c7a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7b306a4,c7b30690,c0e99a71,627,c7859b28,...) at _cv_wait_sig+0x243 seltdwait(c7859b28,58,c6d9fe00,c79bc000,14,...) at seltdwait+0xa2 kern_select(c79bc000,6,bfbfea04,0,0,0,20,c785daf0,c785daf0) at kern_select+0x504 select(c79bc000,ef2c7cec,ef2c7d28,c0e97daa,0,...) at select+0x66 syscallenter(c79bc000,ef2c7ce4,ef2c7ce4,0,0,...) at syscallenter+0x263 syscall(ef2c7d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x281b35c3, esp = 0xbfbfe99c, ebp = 0xbfbfeb28 --- Tracing command ntpd pid 1966 tid 100095 td 0xc74835c0 sched_switch(c74835c0,0,104,191,32ded4fd,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c74835c0,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c099d73a,c783c190,0,c0e90227,c74835c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c783c1a4,0,ef1f5a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c783c1a4,c783c190,c0e99a71,627,c74a68f8,...) at _cv_wait_sig+0x243 seltdwait(c74a68f8,58,c6d9fe00,c74835c0,0,...) at seltdwait+0xa2 kern_select(c74835c0,1c,bfbfece8,0,0,0,20,c77582bc,c77582bc) at kern_select+0x504 select(c74835c0,ef1f5cec,ef1f5d80,ef1f5c88,0,...) at select+0x66 syscallenter(c74835c0,ef1f5ce4,c0cdb556,fffffffe,246,...) at syscallenter+0x263 syscall(ef1f5d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x283625c3, esp = 0xbfbfecbc, ebp = 0xbfbfed88 --- Tracing command nfsd pid 1848 tid 100143 td 0xc771b8a0 sched_switch(c771b8a0,0,104,191,46c64734,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c771b8a0,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(ef2ddbe0,c099d73a,c703db80,0,c771b8a0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c7372590,0,ef2ddc18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c7372590,c703db80,1388,3b1,0,...) at _cv_timedwait_sig+0x252 svc_run_internal(ef2ddd14,c09809f8,c703db80,ef2ddd28,c0e8c598,...) at svc_run_internal+0x356 svc_thread_start(c703db80,ef2ddd28,c0e8c598,390,c785d578,...) at svc_thread_start+0x10 fork_exit(c0b910b0,c703db80,ef2ddd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x804c12e, eip = 0xc, esp = 0x33, ebp = 0x1 --- Tracing command nfsd pid 1848 tid 100142 td 0xc771bb80 sched_switch(c771bb80,0,104,191,3fb0bff5,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c771bb80,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(ef2dabe0,c099d73a,c703db80,0,c771bb80,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c73725d0,0,ef2dac18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c73725d0,c703db80,1388,3b1,0,...) at _cv_timedwait_sig+0x252 svc_run_internal(ef2dad14,c09809f8,c703db80,ef2dad28,c0e8c598,...) at svc_run_internal+0x356 svc_thread_start(c703db80,ef2dad28,c0e8c598,390,c785d578,...) at svc_thread_start+0x10 fork_exit(c0b910b0,c703db80,ef2dad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x804c12e, eip = 0xc, esp = 0x33, ebp = 0x1 --- Tracing command nfsd pid 1848 tid 100141 td 0xc7721000 sched_switch(c7721000,0,104,191,3e149e61,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7721000,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(ef2d7be0,c099d73a,c703db80,0,c7721000,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c7372610,0,ef2d7c18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c7372610,c703db80,1388,3b1,0,...) at _cv_timedwait_sig+0x252 svc_run_internal(ef2d7d14,c09809f8,c703db80,ef2d7d28,c0e8c598,...) at svc_run_internal+0x356 svc_thread_start(c703db80,ef2d7d28,c0e8c598,390,c785d578,...) at svc_thread_start+0x10 fork_exit(c0b910b0,c703db80,ef2d7d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x804c12e, eip = 0xc, esp = 0x33, ebp = 0x1 --- Tracing command nfsd pid 1848 tid 100138 td 0xc79bb8a0 sched_switch(c79bb8a0,0,104,191,471858dc,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c79bb8a0,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(ef2cd5b4,c099d73a,c703db80,0,c79bb8a0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c787f410,0,ef2cd5ec,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c787f410,c703db80,1388,3b1,ef2cd634,...) at _cv_timedwait_sig+0x252 svc_run_internal(c79bba50,14,c0ec029a,c0e8425b,ef2cd7f4,...) at svc_run_internal+0x356 svc_run(c703db80,0,c0e84236,19d,ef2cdc38,...) at svc_run+0x7b nfsrvd_nfsd(c79bb8a0,ef2cd7f4,c,c119bb80,ef2cd720,...) at nfsrvd_nfsd+0x8e nfssvc_nfsd(c79bb8a0,ef2cdcec,c785d578,0,ef2cdc7c,...) at nfssvc_nfsd+0x167 nfssvc(c79bb8a0,ef2cdcec,ef2cdd28,c0e97daa,0,...) at nfssvc+0xeb syscallenter(c79bb8a0,ef2cdce4,ef2cdce4,0,0,...) at syscallenter+0x263 syscall(ef2cdd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (155, FreeBSD ELF32, nfssvc), eip = 0x280dd85b, esp = 0xbfbfe82c, ebp = 0xbfbfea98 --- Tracing command nfsd pid 1847 tid 100137 td 0xc79bbb80 sched_switch(c79bbb80,0,104,191,ab508357,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c79bbb80,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c099d73a,c742e6d0,0,c0e90227,c79bbb80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c742e6e4,0,ef2caa4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c742e6e4,c742e6d0,c0e99a71,627,c7432428,...) at _cv_wait_sig+0x243 seltdwait(c7432428,58,c6d9fe00,c79bbb80,f33,...) at seltdwait+0xa2 kern_select(c79bbb80,7,bfbfecb4,0,0,0,20,c785d834,c785d834) at kern_select+0x504 select(c79bbb80,ef2cacec,bfbffff4,1,0,...) at select+0x66 syscallenter(c79bbb80,ef2cace4,c0cdb89d,c101d170,0,...) at syscallenter+0x263 syscall(ef2cad28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x281845c3, esp = 0xbfbfea9c, ebp = 0xbfbfede8 --- Tracing command mountd pid 1837 tid 100110 td 0xc71772e0 sched_switch(c71772e0,0,104,191,4549d136,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c71772e0,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c099d73a,c742e710,0,c0e90227,c71772e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c742e724,0,ef251a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c742e724,c742e710,c0e99a71,627,c7433268,...) at _cv_wait_sig+0x243 seltdwait(c7433268,58,c6d9fe00,c71772e0,c1167c10,...) at seltdwait+0xa2 kern_select(c71772e0,9,bfbfed3c,0,0,0,20,c785c2bc,c785c2bc) at kern_select+0x504 select(c71772e0,ef251cec,ef251d28,c0e97daa,0,...) at select+0x66 syscallenter(c71772e0,ef251ce4,ef251ce4,0,246,...) at syscallenter+0x263 syscall(ef251d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x2819a5c3, esp = 0xbfbfed0c, ebp = 0xbfbfedd8 --- Tracing command rpcbind pid 1738 tid 100082 td 0xc742ab80 sched_switch(c742ab80,0,104,191,c58a6fa7,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c742ab80,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(ef1ada6c,c099d73a,c771fb50,0,c742ab80,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c771fb64,0,ef1adaa4,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c771fb64,c771fb50,7531,627,ef1adb5c,...) at _cv_timedwait_sig+0x252 seltdwait(ef1adc2c,ef1adc34,535,c742ab80,ef1adb2c,...) at seltdwait+0x8a poll(c742ab80,ef1adcec,ef1add28,c0e97daa,0,...) at poll+0x300 syscallenter(c742ab80,ef1adce4,ef1adce4,0,0,...) at syscallenter+0x263 syscall(ef1add28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, poll), eip = 0x2813f7ab, esp = 0xbfbfcbac, ebp = 0xbfbfed78 --- Tracing command syslogd pid 1710 tid 100099 td 0xc74818a0 sched_switch(c74818a0,0,104,191,3b756a6f,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c74818a0,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c099d73a,c6ecd910,0,c0e90227,c74818a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6ecd924,0,ef201a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6ecd924,c6ecd910,c0e99a71,627,c7432380,...) at _cv_wait_sig+0x243 seltdwait(c7432380,58,c6d9fe00,c74818a0,0,...) at seltdwait+0xa2 kern_select(c74818a0,9,2847f0ac,0,0,0,20,c7757578,c7757578) at kern_select+0x504 select(c74818a0,ef201cec,ef201d28,c0e97daa,0,...) at select+0x66 syscallenter(c74818a0,ef201ce4,ef201ce4,0,246,...) at syscallenter+0x263 syscall(ef201d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x281995c3, esp = 0xbfbfe29c, ebp = 0xbfbfede8 --- Tracing command devd pid 1502 tid 100089 td 0xc7723000 sched_switch(c7723000,0,104,191,4483316e,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7723000,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c099d73a,c6ecd950,0,c0e90227,c7723000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6ecd964,0,ef1e1a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6ecd964,c6ecd950,c0e99a71,627,c7834c08,...) at _cv_wait_sig+0x243 seltdwait(c7834c08,58,c6d9fe00,c7723000,c0ec9ea1,...) at seltdwait+0xa2 kern_select(c7723000,5,bfbfe960,0,0,0,20,c771aaf0,c771aaf0) at kern_select+0x504 select(c7723000,ef1e1cec,ef1e1d28,c0e97daa,0,...) at select+0x66 syscallenter(c7723000,ef1e1ce4,ef1e1ce4,0,0,...) at syscallenter+0x263 syscall(ef1e1d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x808b98b, esp = 0xbfbfe92c, ebp = 0xbfbfee08 --- Tracing command softdepflush pid 18 tid 100075 td 0xc7045b80 cpustop_handler(2,ecfa6b54,c0cdb156,f4,0,...) at cpustop_handler+0x49 ipi_nmi_handler(f4,0,1c1cef0,c099d8f9,c73a12bc,...) at ipi_nmi_handler+0x44 trap(ecfa6b60) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc099d916, esp = 0xecfa6ba0, ebp = 0xecfa6bbc --- _mtx_lock_sleep(c119b230,c7045b80,0,c0ec4aa0,1ccf,...) at _mtx_lock_sleep+0xb6 _mtx_lock_flags(c119b230,0,c0ec4aa0,1ccf,df,...) at _mtx_lock_flags+0xf7 handle_workitem_freeblocks(0,ecfa6c6c,2,6b3,ecfa6c70,...) at handle_workitem_freeblocks+0x50 process_worklist_item(c119b230,0,c0ec4aa0,5e2,c7045b80,...) at process_worklist_item+0x2c2 softdep_process_worklist(c73f35a8,0,c0ec4aa0,55a,3e8,...) at softdep_process_worklist+0x88 softdep_flush(0,ecfa6d28,c0e8c598,390,c73a12bc,...) at softdep_flush+0x2a0 fork_exit(c0bdfbd0,0,ecfa6d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecfa6d60, ebp = 0 --- Tracing command vnlru pid 17 tid 100074 td 0xc7175000 sched_switch(c7175000,0,104,191,4c21a732,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,60,...) at mi_switch+0x1eb sleepq_switch(c7175000,0,c0e97420,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c73a1578,60,c0ea20e4,0,0,...) at sleepq_timedwait+0x6b _sleep(c73a1578,c1194c34,260,c0ea20e4,3e8,...) at _sleep+0x342 vnlru_proc(0,ecfa3d28,c0e8c598,390,c73a1578,...) at vnlru_proc+0xe7 fork_exit(c0a4a190,0,ecfa3d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecfa3d60, ebp = 0 --- Tracing command syncer pid 16 tid 100073 td 0xc71752e0 sched_switch(c71752e0,0,104,191,576596e3,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c71752e0,0,c0e97420,28b,c71752e0,...) at sleepq_switch+0x162 sleepq_timedwait(c1194c74,0,ecfa0c78,1,0,...) at sleepq_timedwait+0x6b _cv_timedwait(c1194c74,c1194c60,3e8,76a,4e20,...) at _cv_timedwait+0x252 sched_sync(0,ecfa0d28,c0e8c598,390,c73a1834,...) at sched_sync+0x595 fork_exit(c0a48130,0,ecfa0d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecfa0d60, ebp = 0 --- Tracing command bufdaemon pid 9 tid 100072 td 0xc71755c0 sched_switch(c71755c0,0,104,191,7b820769,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,54,...) at mi_switch+0x1eb sleepq_switch(c71755c0,0,c0e97420,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c1194908,54,c0e9f119,0,0,...) at sleepq_timedwait+0x6b _sleep(c1194908,c1194910,54,c0e9f119,64,...) at _sleep+0x342 buf_daemon(0,ecf9dd28,c0e8c598,390,c73a1af0,...) at buf_daemon+0x1bb fork_exit(c0a2f8a0,0,ecf9dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf9dd60, ebp = 0 --- Tracing command pagezero pid 8 tid 100071 td 0xc71758a0 sched_switch(c71758a0,0,104,191,7b8d28f2,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c71758a0,0,c0e97420,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c119d0dc,0,c0ecd968,0,0,...) at sleepq_timedwait+0x6b _sleep(c119d0dc,c119bb80,0,c0ecd968,493e0,...) at _sleep+0x342 vm_pagezero(0,ecf9ad28,c0e8c598,390,c73a3000,...) at vm_pagezero+0xdc fork_exit(c0c239a0,0,ecf9ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf9ad60, ebp = 0 --- Tracing command vmdaemon pid 7 tid 100070 td 0xc7175b80 sched_switch(c7175b80,0,104,191,d3644465,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,74,...) at mi_switch+0x1eb sleepq_switch(c7175b80,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c119ccc8,74,c0e9f112,0,0,...) at sleepq_wait+0x63 _sleep(c119ccc8,c119cccc,74,c0e9f112,0,...) at _sleep+0x372 vm_daemon(0,ecf97d28,c0e8c598,390,c6dbd2bc,...) at vm_daemon+0x59 fork_exit(c0c1d560,0,ecf97d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf97d60, ebp = 0 --- Tracing command pagedaemon pid 6 tid 100069 td 0xc7176000 sched_switch(c7176000,0,104,191,986c9f8a,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,54,...) at mi_switch+0x1eb sleepq_switch(c7176000,0,c0e97420,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c119cc90,54,c0e9f112,0,0,...) at sleepq_timedwait+0x6b _sleep(c119cc90,c119bb80,54,c0e9f112,1388,...) at _sleep+0x342 vm_pageout(0,ecf94d28,c0e8c598,390,c6dbd578,...) at vm_pageout+0x2c7 fork_exit(c0c1e580,0,ecf94d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf94d60, ebp = 0 --- Tracing command xpt_thrd pid 5 tid 100068 td 0xc71762e0 sched_switch(c71762e0,0,104,191,cf1fb841,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c71762e0,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c0fe3954,5c,c0d02a32,0,0,...) at sleepq_wait+0x63 _sleep(c0fe3954,c0fe3998,5c,c0d02a32,0,...) at _sleep+0x372 xpt_scanner_thread(0,ecf91d28,c0e8c598,390,c6dbd834,...) at xpt_scanner_thread+0x47 fork_exit(c048c530,0,ecf91d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf91d60, ebp = 0 --- Tracing command sctp_iterator pid 4 tid 100067 td 0xc71765c0 sched_switch(c71765c0,0,104,191,b5edc272,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c71765c0,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c1196798,0,c0eb533f,0,0,...) at sleepq_wait+0x63 _sleep(c1196798,c1196760,0,c0eb533f,0,...) at _sleep+0x372 sctp_iterator_thread(0,ecf8ed28,c0e8c598,390,c6dbdaf0,...) at sctp_iterator_thread+0x5c fork_exit(c0ad9ac0,0,ecf8ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf8ed60, ebp = 0 --- Tracing command fdc0 pid 3 tid 100063 td 0xc70372e0 sched_switch(c70372e0,0,104,191,50d9d39e,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c70372e0,0,c0e97420,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c6fe9e3c,5c,c0e84d12,0,0,...) at sleepq_timedwait+0x6b _sleep(c6fe9e3c,c6fe9ef0,5c,c0e84d12,3e8,...) at _sleep+0x342 fdc_thread(c6fe9e00,ecf82d28,c0e8c598,390,c7024000,...) at fdc_thread+0x27d fork_exit(c0ca2bc0,c6fe9e00,ecf82d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf82d60, ebp = 0 --- Tracing command fw0_probe pid 2 tid 100061 td 0xc70378a0 sched_switch(c70378a0,0,104,191,cf82fe67,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c70378a0,0,c0e97420,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c706b000,6c,c0e84d12,100,0,...) at sleepq_wait_sig+0x17 _sleep(c706b000,c706f488,16c,c0e84d12,0,...) at _sleep+0x35c fw_bus_probe_thread(c706b000,ecf72d28,c0e8c598,390,c70242bc,...) at fw_bus_probe_thread+0xa08 fork_exit(c06ae8d0,c706b000,ecf72d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf72d60, ebp = 0 --- Tracing command usb pid 15 tid 100057 td 0xc70435c0 sched_switch(c70435c0,0,104,191,bc863f9d,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70435c0,0,c0e97420,268,c70435c0,...) at sleepq_switch+0x162 sleepq_wait(c704cd34,0,ecf5fcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c704cd34,c704cdd4,c0e632de,6b,c704cd3c,...) at _cv_wait+0x243 usb_process(c704cd2c,ecf5fd28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c704cd2c,ecf5fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf5fd60, ebp = 0 --- Tracing command usb pid 15 tid 100056 td 0xc70438a0 sched_switch(c70438a0,0,104,191,ad3e5960,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70438a0,0,c0e97420,268,c70438a0,...) at sleepq_switch+0x162 sleepq_wait(c704cd04,0,ecf5ccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c704cd04,c704cdd4,c0e632de,6b,c704cd0c,...) at _cv_wait+0x243 usb_process(c704ccfc,ecf5cd28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c704ccfc,ecf5cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf5cd60, ebp = 0 --- Tracing command usb pid 15 tid 100055 td 0xc7043b80 sched_switch(c7043b80,0,104,191,bc1f01c3,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7043b80,0,c0e97420,268,c7043b80,...) at sleepq_switch+0x162 sleepq_wait(c704ccd4,0,ecf59cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c704ccd4,c704cdd4,c0e632de,6b,c704ccdc,...) at _cv_wait+0x243 usb_process(c704cccc,ecf59d28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c704cccc,ecf59d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf59d60, ebp = 0 --- Tracing command usb pid 15 tid 100054 td 0xc7045000 sched_switch(c7045000,0,104,191,bc1ef0a2,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7045000,0,c0e97420,268,c7045000,...) at sleepq_switch+0x162 sleepq_wait(c704cca4,0,ecf56cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c704cca4,c704cdd4,c0e632de,6b,c704ccac,...) at _cv_wait+0x243 usb_process(c704cc9c,ecf56d28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c704cc9c,ecf56d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf56d60, ebp = 0 --- Tracing command usb pid 15 tid 100052 td 0xc70455c0 sched_switch(c70455c0,0,104,191,bc1edb75,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70455c0,0,c0e97420,268,c70455c0,...) at sleepq_switch+0x162 sleepq_wait(c703eb5c,0,ecf4dcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c703eb5c,c703ebfc,c0e632de,6b,c703eb64,...) at _cv_wait+0x243 usb_process(c703eb54,ecf4dd28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c703eb54,ecf4dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf4dd60, ebp = 0 --- Tracing command usb pid 15 tid 100051 td 0xc7035000 sched_switch(c7035000,0,104,191,ad3cea93,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7035000,0,c0e97420,268,c7035000,...) at sleepq_switch+0x162 sleepq_wait(c703eb2c,0,ecf4acac,1,0,...) at sleepq_wait+0x63 _cv_wait(c703eb2c,c703ebfc,c0e632de,6b,c703eb34,...) at _cv_wait+0x243 usb_process(c703eb24,ecf4ad28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c703eb24,ecf4ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf4ad60, ebp = 0 --- Tracing command usb pid 15 tid 100050 td 0xc70352e0 sched_switch(c70352e0,0,104,191,bbbaac1c,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70352e0,0,c0e97420,268,c70352e0,...) at sleepq_switch+0x162 sleepq_wait(c703eafc,0,ecf47cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c703eafc,c703ebfc,c0e632de,6b,c703eb04,...) at _cv_wait+0x243 usb_process(c703eaf4,ecf47d28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c703eaf4,ecf47d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf47d60, ebp = 0 --- Tracing command usb pid 15 tid 100049 td 0xc70355c0 sched_switch(c70355c0,0,104,191,bbba9a57,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70355c0,0,c0e97420,268,c70355c0,...) at sleepq_switch+0x162 sleepq_wait(c703eacc,0,ecf44cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c703eacc,c703ebfc,c0e632de,6b,c703ead4,...) at _cv_wait+0x243 usb_process(c703eac4,ecf44d28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c703eac4,ecf44d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf44d60, ebp = 0 --- Tracing command usb pid 15 tid 100048 td 0xc70358a0 sched_switch(c70358a0,0,104,191,bbba8557,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70358a0,0,c0e97420,268,c70358a0,...) at sleepq_switch+0x162 sleepq_wait(c7038b5c,0,ecf40cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7038b5c,c7038bfc,c0e632de,6b,c7038b64,...) at _cv_wait+0x243 usb_process(c7038b54,ecf40d28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c7038b54,ecf40d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf40d60, ebp = 0 --- Tracing command usb pid 15 tid 100047 td 0xc7035b80 sched_switch(c7035b80,0,104,191,85c12a9f,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7035b80,0,c0e97420,268,c7035b80,...) at sleepq_switch+0x162 sleepq_wait(c7038b2c,0,ecf3dcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7038b2c,c7038bfc,c0e632de,6b,c7038b34,...) at _cv_wait+0x243 usb_process(c7038b24,ecf3dd28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c7038b24,ecf3dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf3dd60, ebp = 0 --- Tracing command usb pid 15 tid 100046 td 0xc7036000 sched_switch(c7036000,0,104,191,b5e22248,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7036000,0,c0e97420,268,c7036000,...) at sleepq_switch+0x162 sleepq_wait(c7038afc,0,ecf3acac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7038afc,c7038bfc,c0e632de,6b,c7038b04,...) at _cv_wait+0x243 usb_process(c7038af4,ecf3ad28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c7038af4,ecf3ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf3ad60, ebp = 0 --- Tracing command usb pid 15 tid 100045 td 0xc70362e0 sched_switch(c70362e0,0,104,191,b5e211f8,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70362e0,0,c0e97420,268,c70362e0,...) at sleepq_switch+0x162 sleepq_wait(c7038acc,0,ecf37cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7038acc,c7038bfc,c0e632de,6b,c7038ad4,...) at _cv_wait+0x243 usb_process(c7038ac4,ecf37d28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c7038ac4,ecf37d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf37d60, ebp = 0 --- Tracing command usb pid 15 tid 100044 td 0xc70365c0 sched_switch(c70365c0,0,104,191,b5e1ff03,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70365c0,0,c0e97420,268,c70365c0,...) at sleepq_switch+0x162 sleepq_wait(c702fb5c,0,ecf33cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c702fb5c,c702fbfc,c0e632de,6b,c702fb64,...) at _cv_wait+0x243 usb_process(c702fb54,ecf33d28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c702fb54,ecf33d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf33d60, ebp = 0 --- Tracing command usb pid 15 tid 100043 td 0xc70368a0 sched_switch(c70368a0,0,104,191,8492e690,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70368a0,0,c0e97420,268,c70368a0,...) at sleepq_switch+0x162 sleepq_wait(c702fb2c,0,e0bffcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c702fb2c,c702fbfc,c0e632de,6b,c702fb34,...) at _cv_wait+0x243 usb_process(c702fb24,e0bffd28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c702fb24,e0bffd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0bffd60, ebp = 0 --- Tracing command usb pid 15 tid 100042 td 0xc7036b80 sched_switch(c7036b80,0,104,191,b57e1975,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7036b80,0,c0e97420,268,c7036b80,...) at sleepq_switch+0x162 sleepq_wait(c702fafc,0,e0bfccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c702fafc,c702fbfc,c0e632de,6b,c702fb04,...) at _cv_wait+0x243 usb_process(c702faf4,e0bfcd28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c702faf4,e0bfcd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0bfcd60, ebp = 0 --- Tracing command usb pid 15 tid 100041 td 0xc7037000 sched_switch(c7037000,0,104,191,b57e0828,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7037000,0,c0e97420,268,c7037000,...) at sleepq_switch+0x162 sleepq_wait(c702facc,0,e0bf9cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c702facc,c702fbfc,c0e632de,6b,c702fad4,...) at _cv_wait+0x243 usb_process(c702fac4,e0bf9d28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c702fac4,e0bf9d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0bf9d60, ebp = 0 --- Tracing command usb pid 15 tid 100039 td 0xc70252e0 sched_switch(c70252e0,0,104,191,b57df278,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70252e0,0,c0e97420,268,c70252e0,...) at sleepq_switch+0x162 sleepq_wait(c7028b5c,0,e0bf2cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7028b5c,c7028bfc,c0e632de,6b,c7028b64,...) at _cv_wait+0x243 usb_process(c7028b54,e0bf2d28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c7028b54,e0bf2d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0bf2d60, ebp = 0 --- Tracing command usb pid 15 tid 100038 td 0xc70255c0 sched_switch(c70255c0,0,104,191,8364550d,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70255c0,0,c0e97420,268,c70255c0,...) at sleepq_switch+0x162 sleepq_wait(c7028b2c,0,e0befcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7028b2c,c7028bfc,c0e632de,6b,c7028b34,...) at _cv_wait+0x243 usb_process(c7028b24,e0befd28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c7028b24,e0befd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0befd60, ebp = 0 --- Tracing command usb pid 15 tid 100037 td 0xc70258a0 sched_switch(c70258a0,0,104,191,b519ff94,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70258a0,0,c0e97420,268,c70258a0,...) at sleepq_switch+0x162 sleepq_wait(c7028afc,0,e0beccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7028afc,c7028bfc,c0e632de,6b,c7028b04,...) at _cv_wait+0x243 usb_process(c7028af4,e0becd28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c7028af4,e0becd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0becd60, ebp = 0 --- Tracing command usb pid 15 tid 100036 td 0xc7025b80 sched_switch(c7025b80,0,104,191,b519e942,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7025b80,0,c0e97420,268,c7025b80,...) at sleepq_switch+0x162 sleepq_wait(c7028acc,0,e0be9cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7028acc,c7028bfc,c0e632de,6b,c7028ad4,...) at _cv_wait+0x243 usb_process(c7028ac4,e0be9d28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c7028ac4,e0be9d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0be9d60, ebp = 0 --- Tracing command usb pid 15 tid 100034 td 0xc70272e0 sched_switch(c70272e0,0,104,191,b519d0ea,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70272e0,0,c0e97420,268,c70272e0,...) at sleepq_switch+0x162 sleepq_wait(c701fb5c,0,e0be2cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c701fb5c,c701fbfc,c0e632de,6b,c701fb64,...) at _cv_wait+0x243 usb_process(c701fb54,e0be2d28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c701fb54,e0be2d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0be2d60, ebp = 0 --- Tracing command usb pid 15 tid 100033 td 0xc70275c0 sched_switch(c70275c0,0,104,191,81762bd2,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70275c0,0,c0e97420,268,c70275c0,...) at sleepq_switch+0x162 sleepq_wait(c701fb2c,0,e0bdfcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c701fb2c,c701fbfc,c0e632de,6b,c701fb34,...) at _cv_wait+0x243 usb_process(c701fb24,e0bdfd28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c701fb24,e0bdfd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0bdfd60, ebp = 0 --- Tracing command usb pid 15 tid 100032 td 0xc70278a0 sched_switch(c70278a0,0,104,191,b4b57d73,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70278a0,0,c0e97420,268,c70278a0,...) at sleepq_switch+0x162 sleepq_wait(c701fafc,0,e0bdccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c701fafc,c701fbfc,c0e632de,6b,c701fb04,...) at _cv_wait+0x243 usb_process(c701faf4,e0bdcd28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c701faf4,e0bdcd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0bdcd60, ebp = 0 --- Tracing command usb pid 15 tid 100031 td 0xc7027b80 sched_switch(c7027b80,0,104,191,b4b56b22,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7027b80,0,c0e97420,268,c7027b80,...) at sleepq_switch+0x162 sleepq_wait(c701facc,0,e0bd9cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c701facc,c701fbfc,c0e632de,6b,c701fad4,...) at _cv_wait+0x243 usb_process(c701fac4,e0bd9d28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c701fac4,e0bd9d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0bd9d60, ebp = 0 --- Tracing command yarrow pid 14 tid 100017 td 0xc6ecc5c0 sched_switch(c6ecc5c0,0,104,191,7b8db9d3,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c6ecc5c0,0,c0e97420,28b,2,...) at sleepq_switch+0x162 sleepq_timedwait(c101d884,0,c0e84d12,2,0,...) at sleepq_timedwait+0x6b _sleep(c101d884,0,0,c0e84d12,64,...) at _sleep+0x342 pause(c0e84d12,64,c0e51cae,111,0,...) at pause+0x47 random_kthread(0,c6b65d28,c0e8c598,390,c6dbc000,...) at random_kthread+0x1ef fork_exit(c07bf4a0,0,c6b65d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b65d60, ebp = 0 --- Tracing command geom pid 13 tid 100015 td 0xc6eccb80 sched_switch(c6eccb80,0,104,191,7b7b62e3,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c6eccb80,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c101b4a8,5c,c0e84d12,0,0,...) at sleepq_wait+0x63 _sleep(c101b4a8,c101b408,25c,c0e84d12,0,...) at _sleep+0x372 g_io_schedule_down(c6eccb80,0,c0e86865,6c,c6b5fd14,...) at g_io_schedule_down+0x5b g_down_procbody(0,c6b5fd28,c0e8c598,390,c6dbc2bc,...) at g_down_procbody+0x9d fork_exit(c0943060,0,c6b5fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b5fd60, ebp = 0 --- Tracing command geom pid 13 tid 100014 td 0xc6ece000 sched_switch(c6ece000,0,104,191,7b821729,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c6ece000,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c101b4a4,5c,c0e84d12,0,0,...) at sleepq_wait+0x63 _sleep(c101b4a4,c101b428,25c,c0e84d12,0,...) at _sleep+0x372 g_io_schedule_up(c6ece000,0,c0e86865,5f,c6b5cd14,...) at g_io_schedule_up+0x134 g_up_procbody(0,c6b5cd28,c0e8c598,390,c6dbc2bc,...) at g_up_procbody+0x9d fork_exit(c0943100,0,c6b5cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b5cd60, ebp = 0 --- Tracing command geom pid 13 tid 100013 td 0xc6dbf2e0 sched_switch(c6dbf2e0,0,104,191,448149bb,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c6dbf2e0,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c101b49c,5c,c0e84d12,0,0,...) at sleepq_wait+0x63 _sleep(c101b49c,c101b3cc,25c,c0e84d12,0,...) at _sleep+0x372 g_run_events(c102b700,0,c0e86865,79,c6b59d14,...) at g_run_events+0x526 g_event_procbody(0,c6b59d28,c0e8c598,390,c6dbc2bc,...) at g_event_procbody+0x90 fork_exit(c09431a0,0,c6b59d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b59d60, ebp = 0 --- Tracing command intr pid 12 tid 100065 td 0xc7176b80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100064 td 0xc7177000 sched_switch(c7177000,0,109,191,b4b1a7c5,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c7177000,...) at mi_switch+0x1eb ithread_loop(c6dbaa00,ecf85d28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbaa00,ecf85d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf85d60, ebp = 0 --- Tracing command intr pid 12 tid 100062 td 0xc70375c0 sched_switch(c70375c0,0,109,191,3c3863f6,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c70375c0,...) at mi_switch+0x1eb ithread_loop(c6dbaa10,ecf75d28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbaa10,ecf75d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf75d60, ebp = 0 --- Tracing command intr pid 12 tid 100059 td 0xc7043000 sched_switch(c7043000,0,109,191,b4b0e6be,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c7043000,...) at mi_switch+0x1eb ithread_loop(c6dbaa40,ecf69d28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbaa40,ecf69d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf69d60, ebp = 0 --- Tracing command intr pid 12 tid 100058 td 0xc70432e0 sched_switch(c70432e0,0,109,191,d0451945,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c70432e0,...) at mi_switch+0x1eb ithread_loop(c6dbaa70,ecf65d28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbaa70,ecf65d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf65d60, ebp = 0 --- Tracing command intr pid 12 tid 100053 td 0xc70452e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100040 td 0xc7025000 sched_switch(c7025000,0,109,191,d9c7cf8a,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c7025000,...) at mi_switch+0x1eb ithread_loop(c6dbaa90,e0bf6d28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbaa90,e0bf6d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0bf6d60, ebp = 0 --- Tracing command intr pid 12 tid 100035 td 0xc7027000 sched_switch(c7027000,0,109,191,d44f8430,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c7027000,...) at mi_switch+0x1eb ithread_loop(c6dbaaa0,e0be6d28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbaaa0,e0be6d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0be6d60, ebp = 0 --- Tracing command intr pid 12 tid 100030 td 0xc6ece2e0 sched_switch(c6ece2e0,0,109,191,d261963d,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c6ece2e0,...) at mi_switch+0x1eb ithread_loop(c6dbaab0,e0bd6d28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbaab0,e0bd6d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0bd6d60, ebp = 0 --- Tracing command intr pid 12 tid 100029 td 0xc6ece5c0 sched_switch(c6ece5c0,0,109,191,ec246d90,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c6ece5c0,...) at mi_switch+0x1eb ithread_loop(c6dbab40,e0a80d28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbab40,e0a80d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0a80d60, ebp = 0 --- Tracing command intr pid 12 tid 100028 td 0xc6ece8a0 sched_switch(c6ece8a0,0,109,191,77dcf870,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c6ece8a0,...) at mi_switch+0x1eb ithread_loop(c6dbab50,e0a7cd28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbab50,e0a7cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0a7cd60, ebp = 0 --- Tracing command intr pid 12 tid 100027 td 0xc6eceb80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100026 td 0xc6fb3000 sched_switch(c6fb3000,0,109,191,ec24f789,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c6fb3000,...) at mi_switch+0x1eb ithread_loop(c6dbab80,c6b80d28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbab80,c6b80d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b80d60, ebp = 0 --- Tracing command intr pid 12 tid 100021 td 0xc6e048a0 sched_switch(c6e048a0,0,109,191,f7cb0d0d,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c6e048a0,...) at mi_switch+0x1eb ithread_loop(c6dbabc0,c6b71d28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbabc0,c6b71d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b71d60, ebp = 0 --- Tracing command intr pid 12 tid 100019 td 0xc6ecc000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100012 td 0xc6dbf5c0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100011 td 0xc6dbf8a0 sched_switch(c6dbf8a0,0,109,191,70d2ec35,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c6dbf8a0,...) at mi_switch+0x1eb ithread_loop(c6dbabf0,c6b53d28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbabf0,c6b53d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b53d60, ebp = 0 --- Tracing command intr pid 12 tid 100010 td 0xc6dbfb80 sched_switch(c6dbfb80,0,109,191,7b8da216,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c6dbfb80,...) at mi_switch+0x1eb ithread_loop(c6dbac00,c6b50d28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbac00,c6b50d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b50d60, ebp = 0 --- Tracing command intr pid 12 tid 100009 td 0xc6e04000 sched_switch(c6e04000,0,109,191,7a0d29bc,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c6e04000,...) at mi_switch+0x1eb ithread_loop(c6dbac10,c6b4dd28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbac10,c6b4dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b4dd60, ebp = 0 --- Tracing command intr pid 12 tid 100008 td 0xc6e042e0 sched_switch(c6e042e0,0,109,191,7ba97df8,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c6e042e0,...) at mi_switch+0x1eb ithread_loop(c6dbac20,c6b4ad28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbac20,c6b4ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b4ad60, ebp = 0 --- Tracing command intr pid 12 tid 100007 td 0xc6e045c0 sched_switch(c6e045c0,0,109,191,7ef15cb7,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c6e045c0,...) at mi_switch+0x1eb ithread_loop(c6dbac30,c6b47d28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbac30,c6b47d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b47d60, ebp = 0 --- Tracing command idle pid 11 tid 100006 td 0xc6dbe000 sched_switch(c6dbe000,0,108,18c,7bad1a0b,...) at sched_switch+0x3bc mi_switch(108,0,c0e93c4b,a03,c6dbe000,...) at mi_switch+0x1eb sched_idletd(0,c6b43d28,c0e8c598,390,c6dbc834,...) at sched_idletd+0x2bf fork_exit(c09d3440,0,c6b43d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b43d60, ebp = 0 --- Tracing command idle pid 11 tid 100005 td 0xc6dbe2e0 sched_switch(c6dbe2e0,0,108,18c,7babc06a,...) at sched_switch+0x3bc mi_switch(108,0,c0e93c4b,3b9,c6dbe2e0,...) at mi_switch+0x1eb sched_idletd(0,c6b40d28,c0e8c598,390,c6dbc834,...) at sched_idletd+0x19f fork_exit(c09d3440,0,c6b40d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b40d60, ebp = 0 --- Tracing command idle pid 11 tid 100004 td 0xc6dbe5c0 sched_switch(c6dbe5c0,0,108,18c,7badd995,...) at sched_switch+0x3bc mi_switch(108,0,c0e93c4b,a03,c6dbe5c0,...) at mi_switch+0x1eb sched_idletd(0,c6b3dd28,c0e8c598,390,c6dbc834,...) at sched_idletd+0x2bf fork_exit(c09d3440,0,c6b3dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b3dd60, ebp = 0 --- Tracing command idle pid 11 tid 100003 td 0xc6dbe8a0 sched_switch(c6dbe8a0,0,108,18c,7bacd249,...) at sched_switch+0x3bc mi_switch(108,0,c0e93c4b,3b9,c6dbe8a0,...) at mi_switch+0x1eb sched_idletd(0,c6b3ad28,c0e8c598,390,c6dbc834,...) at sched_idletd+0x19f fork_exit(c09d3440,0,c6b3ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b3ad60, ebp = 0 --- Tracing command init pid 1 tid 100002 td 0xc6dbeb80 sched_switch(c6dbeb80,0,104,191,d2124e58,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c6dbeb80,0,c0e97420,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6dbcaf0,6c,c0e9a1fe,100,0,...) at sleepq_wait_sig+0x17 _sleep(c6dbcaf0,c6dbcb78,16c,c0e9a1fe,0,...) at _sleep+0x35c kern_wait(c6dbeb80,ffffffff,c6b36c44,0,0,...) at kern_wait+0xbe6 wait4(c6dbeb80,c6b36cec,c6b36d28,c0e97daa,0,...) at wait4+0x3b syscallenter(c6dbeb80,c6b36ce4,c6b36ce4,0,0,...) at syscallenter+0x263 syscall(c6b36d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x8060df7, esp = 0xbfbfe8bc, ebp = 0xbfbfe8d8 --- Tracing command audit pid 10 tid 100001 td 0xc6dbf000 sched_switch(c6dbf000,0,104,191,b4b33c21,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c6dbf000,0,c0e97420,268,c6dbf000,...) at sleepq_switch+0x162 sleepq_wait(c119aa00,0,c6b33c8c,1,0,...) at sleepq_wait+0x63 _cv_wait(c119aa00,c119a9e4,c0ec1d17,194,0,...) at _cv_wait+0x243 audit_worker(0,c6b33d28,c0e8c598,390,c6dbd000,...) at audit_worker+0x84 fork_exit(c0ba1200,0,c6b33d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b33d60, ebp = 0 --- Tracing command kernel pid 0 tid 100076 td 0xc70458a0 sched_switch(c70458a0,0,104,191,d364575d,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70458a0,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c73729c0,0,c0e84d12,0,0,...) at sleepq_wait+0x63 _sleep(c73729c0,c73729d8,0,c0e84d12,0,...) at _sleep+0x372 taskqueue_thread_loop(c11f2520,ecfa9d28,c0e8c598,390,c101b580,...) at taskqueue_thread_loop+0xb5 fork_exit(c09f00b0,c11f2520,ecfa9d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecfa9d60, ebp = 0 --- Tracing command kernel pid 0 tid 100066 td 0xc71768a0 sched_switch(c71768a0,0,104,191,6ffae334,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c71768a0,0,c0e97420,28b,2,...) at sleepq_switch+0x162 sleepq_timedwait(c101d884,0,c0e84d12,2,0,...) at sleepq_timedwait+0x6b _sleep(c101d884,0,0,c0e84d12,bb8,...) at _sleep+0x342 pause(c0e84d12,bb8,124,122,c101b608,...) at pause+0x47 deadlkres(0,ecf8bd28,c0e8c598,390,c101b580,...) at deadlkres+0x324 fork_exit(c0964c50,0,ecf8bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf8bd60, ebp = 0 --- Tracing command kernel pid 0 tid 100060 td 0xc7037b80 sched_switch(c7037b80,0,104,191,7727f84a,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7037b80,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c7034480,0,c0e929a1,c0e84d12,0,...) at sleepq_wait+0x63 msleep_spin(c7034480,c7034498,c0e84d12,0,c0e90227,...) at msleep_spin+0x21d taskqueue_thread_loop(c706f49c,ecf6fd28,c0e8c598,390,c101b580,...) at taskqueue_thread_loop+0x8f fork_exit(c09f00b0,c706f49c,ecf6fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf6fd60, ebp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xc6fb32e0 sched_switch(c6fb32e0,0,104,191,cf26b497,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c6fb32e0,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c6f50540,0,c0e929a1,c0e84d12,0,...) at sleepq_wait+0x63 msleep_spin(c6f50540,c6f50558,c0e84d12,0,c0e90227,...) at msleep_spin+0x21d taskqueue_thread_loop(c0fe6838,c6b7dd28,c0e8c598,390,c101b580,...) at taskqueue_thread_loop+0x8f fork_exit(c09f00b0,c0fe6838,c6b7dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b7dd60, ebp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xc6fb35c0 sched_switch(c6fb35c0,0,104,191,cf26a51a,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c6fb35c0,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c6f50540,0,c0e929a1,c0e84d12,0,...) at sleepq_wait+0x63 msleep_spin(c6f50540,c6f50558,c0e84d12,0,c0e90227,...) at msleep_spin+0x21d taskqueue_thread_loop(c0fe6838,c6b7ad28,c0e8c598,390,c101b580,...) at taskqueue_thread_loop+0x8f fork_exit(c09f00b0,c0fe6838,c6b7ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b7ad60, ebp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xc6fb38a0 sched_switch(c6fb38a0,0,104,191,cf26927e,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c6fb38a0,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c6f50540,0,c0e929a1,c0e84d12,0,...) at sleepq_wait+0x63 msleep_spin(c6f50540,c6f50558,c0e84d12,0,c0e90227,...) at msleep_spin+0x21d taskqueue_thread_loop(c0fe6838,c6b77d28,c0e8c598,390,c101b580,...) at taskqueue_thread_loop+0x8f fork_exit(c09f00b0,c0fe6838,c6b77d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b77d60, ebp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xc6fb3b80 sched_switch(c6fb3b80,0,104,191,cf203b10,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c6fb3b80,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c6f50580,0,c0e84d12,0,0,...) at sleepq_wait+0x63 _sleep(c6f50580,c6f50598,0,c0e84d12,0,...) at _sleep+0x372 taskqueue_thread_loop(c101bf18,c6b74d28,c0e8c598,390,c101b580,...) at taskqueue_thread_loop+0xb5 fork_exit(c09f00b0,c101bf18,c6b74d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b74d60, ebp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xc6e04b80 sched_switch(c6e04b80,0,104,191,cf202a46,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c6e04b80,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c6f50600,0,c0e84d12,0,0,...) at sleepq_wait+0x63 _sleep(c6f50600,c6f50618,0,c0e84d12,0,...) at _sleep+0x372 taskqueue_thread_loop(c119afe4,c6b6ed28,c0e8c598,390,c101b580,...) at taskqueue_thread_loop+0xb5 fork_exit(c09f00b0,c119afe4,c6b6ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b6ed60, ebp = 0 --- Tracing command kernel pid 0 tid 100018 td 0xc6ecc2e0 sched_switch(c6ecc2e0,0,104,191,d1efde89,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c6ecc2e0,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c6f50680,0,c0e84d12,0,0,...) at sleepq_wait+0x63 _sleep(c6f50680,c6f50698,0,c0e84d12,0,...) at _sleep+0x372 taskqueue_thread_loop(c103a488,c6b68d28,c0e8c598,390,c101b580,...) at taskqueue_thread_loop+0xb5 fork_exit(c09f00b0,c103a488,c6b68d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b68d60, ebp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xc6ecc8a0 sched_switch(c6ecc8a0,0,104,191,f5e1adb2,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c6ecc8a0,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c6d76b80,0,c0e84d12,0,0,...) at sleepq_wait+0x63 _sleep(c6d76b80,c6d76b98,0,c0e84d12,0,...) at _sleep+0x372 taskqueue_thread_loop(c1038ea0,c6b62d28,c0e8c598,390,c101b580,...) at taskqueue_thread_loop+0xb5 fork_exit(c09f00b0,c1038ea0,c6b62d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b62d60, ebp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xc101b840 sched_switch(c101b840,0,104,191,7fa77864,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,54,...) at mi_switch+0x1eb sleepq_switch(c101b840,0,c0e97420,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c101b580,54,c0e9455d,0,0,...) at sleepq_timedwait+0x6b _sleep(c101b580,0,54,c0e9455d,2710,...) at _sleep+0x342 scheduler(0,141ec00,141ec00,141e000,1425000,...) at scheduler+0x240 mi_startup() at mi_startup+0xac begin() at begin+0x2c db:0:allt> call doadump Physical memory: 3439 MB Physical memory: 3439 MB Dumping 178 MB:Dumping 178 MB: 163 163 147 147 131 131 115 115 99 99 83 83 67 67 51 51 35 35 19 19 3 3 Dump complete Dump complete = 0 db:0:doadump> reset cpu_reset: Restarting BSP cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 2 cpu_reset_proxy: Stopped CPU 2 (kgdb) bt #0 doadump (textdump=0xef1f2244) at pcpu.h:244 #1 0xc04de7e9 in db_fncall (dummy1=0xc09e19ea, dummy2=0x0, dummy3=0xffffffff, dummy4=0xef1f22d8 "ì\"\037ï") at ../../../ddb/db_command.c:572 #2 0xc04dec1f in db_command (last_cmdp=0xc0fe49fc, cmd_table=0x0, dopager=0x0) at ../../../ddb/db_command.c:448 #3 0xc04decd4 in db_command_script (command=0xc0fe5908 "call doadump") at ../../../ddb/db_command.c:519 #4 0xc04e3010 in db_script_exec (scriptname=0xc0fe5260 "doadump", warnifnotfound=Variable "warnifnotfound" is not available. ) at ../../../ddb/db_script.c:302 #5 0xc04e30a1 in db_run_cmd (addr=0x1, have_addr=0x0, count=0xc11bd500, modif=0xef1f2410 "") at ../../../ddb/db_script.c:375 #6 0xc04debe1 in db_command (last_cmdp=0xc0fe49fc, cmd_table=0x0, dopager=0x1) at ../../../ddb/db_command.c:448 #7 0xc04ded3a in db_command_loop () at ../../../ddb/db_command.c:501 #8 0xc04e0d2d in db_trap (type=0x3, code=0x0) at ../../../ddb/db_main.c:229 #9 0xc09e1b62 in kdb_trap (type=0x3, code=0x0, tf=0xef1f25c0) at ../../../kern/subr_kdb.c:534 #10 0xc0cdb78b in trap (frame=0xef1f25c0) at ../../../i386/i386/trap.c:719 #11 0xc0cc469c in calltrap () at ../../../i386/i386/exception.s:168 #12 0xc09e19ea in kdb_enter (why=0xc0e91c42 "panic", msg=0xc0e91c42 "panic") at cpufunc.h:71 #13 0xc09acf24 in panic (fmt=0xc0ec4b7d "worklist_insert: %p %s(0x%X) already on list") at ../../../kern/kern_shutdown.c:587 #14 0xc0bcb003 in worklist_insert (head=0xe08608ac, item=0xc867c700, locked=0x0) at ../../../ufs/ffs/ffs_softdep.c:1033 #15 0xc0bcdbaa in freework_freeblock (freework=0xc867c700) at ../../../ufs/ffs/ffs_softdep.c:5953 #16 0xc0bce5b5 in indir_trunc (freework=0xc867c700, dbn=0xe200, lbn=0xfffffffffffffff4) at ../../../ufs/ffs/ffs_softdep.c:7688 #17 0xc0bd039f in handle_workitem_freeblocks (freeblks=0xc8b9fb80, flags=0x0) at ../../../ufs/ffs/ffs_softdep.c:7317 #18 0xc0bddf74 in softdep_journal_freeblocks (ip=0xc7ebb0e8, cred=0xc8b3a000, length=0x1d20a9f, flags=0x800) at ../../../ufs/ffs/ffs_softdep.c:6297 #19 0xc0bc12dd in ffs_truncate (vp=0xc7e68570, length=0x1d20a9f, flags=Variable "flags" is not available. ) at ../../../ufs/ffs/ffs_inode.c:311 #20 0xc0bf8d48 in ufs_setattr (ap=0xef1f2bf0) at ../../../ufs/ufs/ufs_vnops.c:621 #21 0xc0cfe735 in VOP_SETATTR_APV (vop=0xc0fc7a80, a=0xef1f2bf0) at vnode_if.c:760 #22 0xc0a56995 in vn_truncate (fp=0xc7e472d8, length=0x1d20a9f, active_cred=0xc8b3a000, td=0xc74838a0) at vnode_if.h:335 #23 0xc09fb917 in kern_ftruncate (td=0xc74838a0, fd=0x3, length=0x1d20a9f) at file.h:247 #24 0xc09fb998 in ftruncate (td=0xc74838a0, uap=0xef1f2cec) at ../../../kern/sys_generic.c:595 #25 0xc09f0473 in syscallenter (td=0xc74838a0, sa=0xef1f2ce4) at ../../../kern/subr_trap.c:344 #26 0xc0cdb03f in syscall (frame=0xef1f2d28) at ../../../i386/i386/trap.c:1082 #27 0xc0cc4701 in Xint0x80_syscall () at ../../../i386/i386/exception.s:266 #28 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) f 14 #14 0xc0bcb003 in worklist_insert (head=0xe08608ac, item=0xc867c700, locked=0x0) at ../../../ufs/ffs/ffs_softdep.c:1033 1033 panic("worklist_insert: %p %s(0x%X) already on list", (kgdb) l 1028 { 1029 1030 if (locked) 1031 mtx_assert(&lk, MA_OWNED); 1032 if (item->wk_state & ONWORKLIST) 1033 panic("worklist_insert: %p %s(0x%X) already on list", 1034 item, TYPENAME(item->wk_type), item->wk_state); 1035 item->wk_state |= ONWORKLIST; 1036 LIST_INSERT_HEAD(head, item, wk_list); 1037 } (kgdb) p *item $1 = {wk_list = {le_next = 0x0, le_prev = 0xc8b9fb9c}, wk_mp = 0xc73f35a8, wk_type = 0xe, wk_state = 0x8009} (kgdb) p *item->wk_mp $2 = {mnt_mtx = {lock_object = {lo_name = 0xc0e84471 "struct mount mtx", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc6d5b450}, mtx_lock = 0x4}, mnt_gen = 0x1, mnt_list = {tqe_next = 0x0, tqe_prev = 0xc73f45c0}, mnt_op = 0xc0fc72a0, mnt_vfc = 0xc0fc7100, mnt_vnodecovered = 0xc7989c3c, mnt_syncer = 0xc7eddd98, mnt_ref = 0x13, mnt_nvnodelist = {tqh_first = 0xc7eddae0, tqh_last = 0xc7f4b6e0}, mnt_nvnodelistsize = 0xb, mnt_writeopcount = 0x6, mnt_kern_flag = 0x60000144, mnt_flag = 0x201300, mnt_xflag = 0x0, mnt_noasync = 0x1, mnt_opt = 0xc6d390b0, mnt_optnew = 0x0, mnt_maxsymlinklen = 0x78, mnt_stat = {f_version = 0x20030518, f_type = 0x1, f_flags = 0x201300, f_bsize = 0x800, f_iosize = 0x4000, f_blocks = 0xf7513, f_bfree = 0x18f397, f_bavail = 0x17b707, f_files = 0x44ffe, f_ffree = 0x44ff4, f_syncwrites = 0x0, f_asyncwrites = 0x0, f_syncreads = 0x0, f_asyncreads = 0x0, f_spare = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, f_namemax = 0xff, f_owner = 0x0, f_fsid = { val = {0x4df38103, 0x8d3d7ba}}, f_charspare = '\0' , f_fstypename = "ufs", '\0' , f_mntfromname = "/dev/md5a", '\0' , f_mntonname = "/mnt", '\0' }, mnt_cred = 0xc88d0d00, mnt_data = 0xc7e35600, mnt_time = 0x0, mnt_iosize_max = 0x20000, mnt_export = 0xc7c8cb00, mnt_label = 0x0, mnt_hashseed = 0x27b0de48, mnt_lockref = 0x1, mnt_secondary_writes = 0x1, mnt_secondary_accwrites = 0x23ad39, mnt_susp_owner = 0x0, mnt_gjprovider = 0x0, mnt_explock = {lock_object = {lo_name = 0xc0e84482 "explock", lo_flags = 0x5730000, lo_data = 0x0, lo_witness = 0xc6d60ad8}, lk_lock = 0x1, lk_exslpfail = 0x0, lk_timo = 0x0, lk_pri = 0x60, lk_stack = {depth = 0x7, pcs = {0xc099587e, 0xc0a36f27, 0xc0a3f046, 0xc0a3f664, 0xc09f0473, 0xc0cdb03f, 0xc0cc4701, 0x0 }}}} (kgdb) p *(struct ufsmount *)item->wk_mp->mnt_data $3 = {um_mountp = 0xc73f35a8, um_dev = 0xc7405800, um_cp = 0xc6da24c0, um_bo = 0xc7fa9e90, um_devvp = 0xc7fa9d98, um_fstype = 0x2, um_fs = 0xc773b800, um_extattr = { uepm_lock = {lock_object = {lo_name = 0xc0ec8a3b "ufs_extattr_sx", lo_flags = 0x2330000, lo_data = 0x0, lo_witness = 0xc6d61f28}, sx_lock = 0x1}, uepm_list = { lh_first = 0x0}, uepm_ucred = 0x0, uepm_flags = 0x1}, um_nindir = 0x800, um_bptrtodb = 0x2, um_seqinc = 0x8, um_lock = {lock_object = {lo_name = 0xc0ec7dd6 "FFS", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc6d61ec0}, mtx_lock = 0x4}, um_numindirdeps = 0x0, softdep_workitem_pending = {lh_first = 0xc8b35680}, softdep_worklist_tail = 0xc7f94100, softdep_journal_pending = {lh_first = 0x0}, softdep_journal_tail = 0xc7e35668, softdep_jblocks = 0xc7f00500, softdep_unlinked = { tqh_first = 0x0, tqh_last = 0xc7e35674}, softdep_dirtycg = {lh_first = 0xc7f94300}, softdep_on_journal = 0x0, softdep_on_worklist = 0x39, softdep_deps = 0x30c1, softdep_accdeps = 0xe89792, softdep_req = 0x0, um_quotas = {0x0, 0x0}, um_cred = {0x0, 0x0}, um_btime = {0x0, 0x0}, um_itime = {0x0, 0x0}, um_qflags = "\000", um_savedmaxfilesize = 0x0, um_candelete = 0x0, um_balloc = 0xc0bbc500 , um_blkatoff = 0xc0be16c0 , um_truncate = 0xc0bc0a70 , um_update = 0xc0bc0730 , um_valloc = 0xc0bb74a0 , um_vfree = 0xc0bb6190 , um_ifree = 0xc0be28b0 , um_rdonly = 0xc0bc0040 , um_snapgone = 0xc0bc25e0 } (kgdb) git diff master diff --git a/sys/ufs/ffs/ffs_alloc.c b/sys/ufs/ffs/ffs_alloc.c index 685b9e1..de34bb8 100644 --- a/sys/ufs/ffs/ffs_alloc.c +++ b/sys/ufs/ffs/ffs_alloc.c @@ -2035,7 +2035,7 @@ ffs_blkfree(ump, fs, devvp, bno, size, inum, dephd) */ if (devvp->v_type != VREG && (devvp->v_vflag & VV_COPYONWRITE) && - ffs_snapblkfree(fs, devvp, bno, size, inum)) { + ffs_snapblkfree(fs, devvp, bno, size, inum, dephd)) { return; } if (!ump->um_candelete) { diff --git a/sys/ufs/ffs/ffs_extern.h b/sys/ufs/ffs/ffs_extern.h index 58d6121..28432af 100644 --- a/sys/ufs/ffs/ffs_extern.h +++ b/sys/ufs/ffs/ffs_extern.h @@ -80,7 +80,8 @@ int ffs_realloccg(struct inode *, ufs2_daddr_t, ufs2_daddr_t, ufs2_daddr_t, int, int, int, struct ucred *, struct buf **); int ffs_sbupdate(struct ufsmount *, int, int); void ffs_setblock(struct fs *, u_char *, ufs1_daddr_t); -int ffs_snapblkfree(struct fs *, struct vnode *, ufs2_daddr_t, long, ino_t); +int ffs_snapblkfree(struct fs *, struct vnode *, ufs2_daddr_t, long, ino_t, + struct workhead *); void ffs_snapremove(struct vnode *vp); int ffs_snapshot(struct mount *mp, char *snapfile); void ffs_snapshot_mount(struct mount *mp); @@ -149,6 +150,9 @@ int softdep_prealloc(struct vnode *, int); int softdep_journal_lookup(struct mount *, struct vnode **); void softdep_journal_freeblocks(struct inode *, struct ucred *, off_t, int); void softdep_journal_fsync(struct inode *); +void softdep_buf_append(struct buf *, struct workhead *); +void softdep_inode_append(struct inode *, struct ucred *, struct workhead *); +void softdep_freework(struct workhead *); /* @@ -161,4 +165,14 @@ void softdep_journal_fsync(struct inode *); int ffs_rdonly(struct inode *); +TAILQ_HEAD(snaphead, inode); + +struct snapdata { + LIST_ENTRY(snapdata) sn_link; + struct snaphead sn_head; + daddr_t sn_listsize; + daddr_t *sn_blklist; + struct lock sn_lock; +}; + #endif /* !_UFS_FFS_EXTERN_H */ diff --git a/sys/ufs/ffs/ffs_snapshot.c b/sys/ufs/ffs/ffs_snapshot.c index 968be8a..5b58538 100644 --- a/sys/ufs/ffs/ffs_snapshot.c +++ b/sys/ufs/ffs/ffs_snapshot.c @@ -81,12 +81,13 @@ ffs_snapshot(mp, snapfile) } int -ffs_snapblkfree(fs, devvp, bno, size, inum) +ffs_snapblkfree(fs, devvp, bno, size, inum, wkhd) struct fs *fs; struct vnode *devvp; ufs2_daddr_t bno; long size; ino_t inum; + struct workhead *wkhd; { return (EINVAL); } @@ -126,16 +127,6 @@ ffs_copyonwrite(devvp, bp) #else FEATURE(ffs_snapshot, "FFS snapshot support"); -TAILQ_HEAD(snaphead, inode); - -struct snapdata { - LIST_ENTRY(snapdata) sn_link; - struct snaphead sn_head; - daddr_t sn_listsize; - daddr_t *sn_blklist; - struct lock sn_lock; -}; - LIST_HEAD(, snapdata) snapfree; static struct mtx snapfree_lock; MTX_SYSINIT(ffs_snapfree, &snapfree_lock, "snapdata free list", MTX_DEF); @@ -1635,7 +1626,7 @@ ffs_snapremove(vp) DIP_SET(ip, i_db[blkno], 0); else if ((dblk == blkstofrags(fs, blkno) && ffs_snapblkfree(fs, ip->i_devvp, dblk, fs->fs_bsize, - ip->i_number))) { + ip->i_number, NULL))) { DIP_SET(ip, i_blocks, DIP(ip, i_blocks) - btodb(fs->fs_bsize)); DIP_SET(ip, i_db[blkno], 0); @@ -1660,7 +1651,7 @@ ffs_snapremove(vp) ((ufs1_daddr_t *)(ibp->b_data))[loc]= 0; else if ((dblk == blkstofrags(fs, blkno) && ffs_snapblkfree(fs, ip->i_devvp, dblk, - fs->fs_bsize, ip->i_number))) { + fs->fs_bsize, ip->i_number, NULL))) { ip->i_din1->di_blocks -= btodb(fs->fs_bsize); ((ufs1_daddr_t *)(ibp->b_data))[loc]= 0; @@ -1674,7 +1665,7 @@ ffs_snapremove(vp) ((ufs2_daddr_t *)(ibp->b_data))[loc] = 0; else if ((dblk == blkstofrags(fs, blkno) && ffs_snapblkfree(fs, ip->i_devvp, dblk, - fs->fs_bsize, ip->i_number))) { + fs->fs_bsize, ip->i_number, NULL))) { ip->i_din2->di_blocks -= btodb(fs->fs_bsize); ((ufs2_daddr_t *)(ibp->b_data))[loc] = 0; } @@ -1722,12 +1713,13 @@ ffs_snapremove(vp) * must always have been allocated from a BLK_NOCOPY location. */ int -ffs_snapblkfree(fs, devvp, bno, size, inum) +ffs_snapblkfree(fs, devvp, bno, size, inum, wkhd) struct fs *fs; struct vnode *devvp; ufs2_daddr_t bno; long size; ino_t inum; + struct workhead *wkhd; { struct buf *ibp, *cbp, *savedcbp = 0; struct thread *td = curthread; @@ -1825,6 +1817,17 @@ retry: "Grabonremove: snapino", ip->i_number, (intmax_t)lbn, inum); #endif + /* + * If journaling is tracking this write we must add + * the work to the inode or indirect being written. + */ + if (wkhd != NULL) { + if (lbn < NDADDR) + softdep_inode_append(ip, + curthread->td_ucred, wkhd); + else + softdep_buf_append(ibp, wkhd); + } if (lbn < NDADDR) { DIP_SET(ip, i_db[lbn], bno); } else if (ip->i_ump->um_fstype == UFS1) { @@ -1902,6 +1905,8 @@ retry: * not be freed. Although space will be lost, the snapshot * will stay consistent. */ + if (error != 0 && wkhd != NULL) + softdep_freework(wkhd); lockmgr(vp->v_vnlock, LK_RELEASE, NULL); return (error); } diff --git a/sys/ufs/ffs/ffs_softdep.c b/sys/ufs/ffs/ffs_softdep.c index c125ee3..05d236f 100644 --- a/sys/ufs/ffs/ffs_softdep.c +++ b/sys/ufs/ffs/ffs_softdep.c @@ -584,6 +584,33 @@ softdep_get_depcounts(struct mount *mp, *softdepactiveaccp = 0; } +void +softdep_buf_append(bp, wkhd) + struct buf *bp; + struct workhead *wkhd; +{ + + panic("softdep_buf_appendwork called"); +} + +void +softdep_inode_append(ip, cred, wkhd) + struct inode *ip; + struct ucred *cred; + struct workhead *wkhd; +{ + + panic("softdep_inode_appendwork called"); +} + +void +softdep_freework(wkhd) + struct workhead *wkhd; +{ + + panic("softdep_freework called"); +} + #else FEATURE(softupdates, "FFS soft-updates support"); @@ -755,6 +782,7 @@ static int flush_newblk_dep(struct vnode *, struct mount *, ufs_lbn_t); static int flush_inodedep_deps(struct vnode *, struct mount *, ino_t); static int flush_deplist(struct allocdirectlst *, int, int *); static int sync_cgs(struct mount *, int); +static void sync_snap(struct mount *, int); static int handle_written_filepage(struct pagedep *, struct buf *); static int handle_written_sbdep(struct sbdep *, struct buf *); static void initiate_write_sbdep(struct sbdep *); @@ -867,7 +895,7 @@ static void freework_enqueue(struct freework *); static int handle_workitem_freeblocks(struct freeblks *, int); static int handle_complete_freeblocks(struct freeblks *, int); static void handle_workitem_indirblk(struct freework *); -static void handle_written_freework(struct freework *, int); +static void handle_written_freework(struct freework *); static void merge_inode_lists(struct allocdirectlst *,struct allocdirectlst *); static struct worklist *jnewblk_merge(struct worklist *, struct worklist *, struct workhead *); @@ -1632,6 +1660,7 @@ process_truncates(vp) if (cgwait) { FREE_LOCK(&lk); sync_cgs(mp, MNT_WAIT); + sync_snap(mp, MNT_WAIT); ACQUIRE_LOCK(&lk); continue; } @@ -5915,7 +5944,7 @@ complete_trunc_indir(freework) */ if (bp == NULL) { if (LIST_EMPTY(&indirdep->ir_freeblks->fb_jblkdephd)) - handle_written_freework(freework, 0); + handle_written_freework(freework); else WORKLIST_INSERT(&indirdep->ir_freeblks->fb_freeworkhd, &freework->fw_list); @@ -5967,7 +5996,7 @@ out: */ if (totblks > datablocks) return (0); - return (totblks - datablocks); + return (datablocks - totblks); } /* @@ -7221,6 +7250,7 @@ freework_freeblock(freework) cancel_jnewblk(jnewblk, &wkhd); needj = 0; } else if (needj) { + freework->fw_state |= DELAYEDFREE; freeblks->fb_cgwait++; WORKLIST_INSERT(&wkhd, &freework->fw_list); } @@ -7234,7 +7264,7 @@ freework_freeblock(freework) * made it to disk. We can immediately free the freeblk. */ if (needj == 0) - handle_written_freework(freework, 0); + handle_written_freework(freework); } /* @@ -7275,13 +7305,14 @@ handle_workitem_indirblk(freework) ump = VFSTOUFS(freeblks->fb_list.wk_mp); fs = ump->um_fs; if (freework->fw_state & DEPCOMPLETE) { - handle_written_freework(freework, 0); + handle_written_freework(freework); return; } if (freework->fw_off == NINDIR(fs)) { freework_freeblock(freework); return; } + freework->fw_state |= INPROGRESS; FREE_LOCK(&lk); indir_trunc(freework, fsbtodb(fs, freework->fw_blkno), freework->fw_lbn); @@ -7294,16 +7325,16 @@ handle_workitem_indirblk(freework) * the freeblks is added back to the worklist if there is more work to do. */ static void -handle_written_freework(freework, cgwrite) +handle_written_freework(freework) struct freework *freework; - int cgwrite; { struct freeblks *freeblks; struct freework *parent; freeblks = freework->fw_freeblks; parent = freework->fw_parent; - freeblks->fb_cgwait -= cgwrite; + if (freework->fw_state & DELAYEDFREE) + freeblks->fb_cgwait--; freework->fw_state |= COMPLETE; if ((freework->fw_state & ALLCOMPLETE) == ALLCOMPLETE) WORKITEM_FREE(freework, D_FREEWORK); @@ -7545,6 +7576,8 @@ indir_trunc(freework, dbn, lbn) return; } ACQUIRE_LOCK(&lk); + /* Protects against a race with complete_trunc_indir(). */ + freework->fw_state &= ~INPROGRESS; /* * If we have an indirdep we need to enforce the truncation order * and discard it when it is complete. @@ -7668,7 +7701,7 @@ indir_trunc(freework, dbn, lbn) if (freework->fw_blkno == dbn) { freework->fw_state |= ALLCOMPLETE; ACQUIRE_LOCK(&lk); - handle_written_freework(freework, 0); + handle_written_freework(freework); FREE_LOCK(&lk); } return; @@ -10361,8 +10394,7 @@ softdep_disk_write_complete(bp) continue; case D_FREEWORK: - /* Freework on an indirect block, not bmsafemap. */ - handle_written_freework(WK_FREEWORK(wk), 0); + handle_written_freework(WK_FREEWORK(wk)); break; case D_JSEGDEP: @@ -10533,7 +10565,7 @@ handle_jwork(wkhd) free_freedep(WK_FREEDEP(wk)); continue; case D_FREEWORK: - handle_written_freework(WK_FREEWORK(wk), 1); + handle_written_freework(WK_FREEWORK(wk)); continue; default: panic("handle_jwork: Unknown type %s\n", @@ -11766,6 +11798,42 @@ sync_cgs(mp, waitfor) } /* + * sync snapshots to force freework records waiting on snapshots to claim + * blocks to free. + */ +void +sync_snap(mp, waitfor) + struct mount *mp; + int waitfor; +{ + struct snapdata *sn; + struct vnode *devvp; + struct vnode *vp; + struct inode *ip; + + devvp = VFSTOUFS(mp)->um_devvp; + if ((devvp->v_vflag & VV_COPYONWRITE) == 0) + return; + for (;;) { + VI_LOCK(devvp); + sn = devvp->v_rdev->si_snapdata; + if (sn == NULL) { + VI_UNLOCK(devvp); + return; + } + if (lockmgr(&sn->sn_lock, + LK_INTERLOCK | LK_EXCLUSIVE | LK_SLEEPFAIL, + VI_MTX(devvp)) == 0) + break; + } + TAILQ_FOREACH(ip, &sn->sn_head, i_nextsnap) { + vp = ITOV(ip); + ffs_syncvnode(vp, waitfor); + } + lockmgr(&sn->sn_lock, LK_RELEASE, NULL); +} + +/* * This routine is called when we are trying to synchronously flush a * file. This routine must eliminate any filesystem metadata dependencies * so that the syncing routine can succeed. @@ -12731,6 +12799,53 @@ clear_inodedeps(td) } } +void +softdep_buf_append(bp, wkhd) + struct buf *bp; + struct workhead *wkhd; +{ + struct worklist *wk; + + ACQUIRE_LOCK(&lk); + while ((wk = LIST_FIRST(wkhd)) != NULL) { + WORKLIST_REMOVE(wk); + WORKLIST_INSERT(&bp->b_dep, wk); + } + FREE_LOCK(&lk); + +} + +void +softdep_inode_append(ip, cred, wkhd) + struct inode *ip; + struct ucred *cred; + struct workhead *wkhd; +{ + struct buf *bp; + struct fs *fs; + int error; + + fs = ip->i_fs; + error = bread(ip->i_devvp, fsbtodb(fs, ino_to_fsba(fs, ip->i_number)), + (int)fs->fs_bsize, cred, &bp); + if (error) { + softdep_freework(wkhd); + return; + } + softdep_buf_append(bp, wkhd); + bqrelse(bp); +} + +void +softdep_freework(wkhd) + struct workhead *wkhd; +{ + + ACQUIRE_LOCK(&lk); + handle_jwork(wkhd); + FREE_LOCK(&lk); +} + /* * Function to determine if the buffer has outstanding dependencies * that will cause a roll-back if the buffer is written. If wantcount diff --git a/sys/ufs/ufs/ufs_vnops.c b/sys/ufs/ufs/ufs_vnops.c index 34b1758..733413d 100644 --- a/sys/ufs/ufs/ufs_vnops.c +++ b/sys/ufs/ufs/ufs_vnops.c @@ -1838,6 +1838,8 @@ ufs_mkdir(ap) #ifdef QUOTA if ((error = getinoquota(ip)) || (error = chkiq(ip, 1, ucp, 0))) { + if (DOINGSOFTDEP(tvp)) + softdep_revert_link(dp, ip); UFS_VFREE(tvp, ip->i_number, dmode); vput(tvp); return (error); @@ -1850,6 +1852,8 @@ ufs_mkdir(ap) #ifdef QUOTA if ((error = getinoquota(ip)) || (error = chkiq(ip, 1, cnp->cn_cred, 0))) { + if (DOINGSOFTDEP(tvp)) + softdep_revert_link(dp, ip); UFS_VFREE(tvp, ip->i_number, dmode); vput(tvp); return (error); @@ -2608,6 +2612,8 @@ ufs_makeinode(mode, dvp, vpp, cnp) #ifdef QUOTA if ((error = getinoquota(ip)) || (error = chkiq(ip, 1, ucp, 0))) { + if (DOINGSOFTDEP(tvp)) + softdep_revert_link(pdir, ip); UFS_VFREE(tvp, ip->i_number, mode); vput(tvp); return (error); @@ -2620,6 +2626,8 @@ ufs_makeinode(mode, dvp, vpp, cnp) #ifdef QUOTA if ((error = getinoquota(ip)) || (error = chkiq(ip, 1, cnp->cn_cred, 0))) { + if (DOINGSOFTDEP(tvp)) + softdep_revert_link(pdir, ip); UFS_VFREE(tvp, ip->i_number, mode); vput(tvp); return (error); -- Test scenario: suj17.sh