GENERIC HEAD from 2010-10-14 23:31:58 UTC, r213890M, vmcore.7 KDB: debugger backends: ddb KDB: current backend: ddb 524288K of memory above 4GB ignored Copyright (c) 1992-2010 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-CURRENT #0 r213890M: Fri Oct 15 08:50:38 CEST 2010 pho@x4.osted.lan:/usr/src/sys/i386/compile/PHO i386 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. CPU: AMD Phenom(tm) 9150e Quad-Core Processor (1800.02-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x100f23 Family = 10 Model = 2 Stepping = 3 Features=0x178bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x7ff TSC: P-state invariant real memory = 4294967296 (4096 MB) avail memory = 3536277504 (3372 MB) : Trying to mount root from ufs:/dev/ad4s1a Setting hostuuid: 00000000-0000-0000-0000-00218515337d. Setting hostid: 0x6b64ac17. Starting ddb. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: /dev/ad4s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1a: clean, 896015 free (903 frags, 111889 blocks, 0.1% fragmentation) /dev/ad4s1f: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1f: clean, 936037 free (389 frags, 116956 blocks, 0.0% fragmentation) /dev/ad4s1g: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1g: clean, 117583293 free (437 frags, 14697857 blocks, 0.0% fragmentation) /dev/ad4s1d: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1d: clean, 7034036 free (40924 frags, 874139 blocks, 0.4% fragmentation) /dev/ad4s1e: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1e: clean, 23107159 free (10127 frags, 2887129 blocks, 0.0% fragmentation) Mounting local file systems:. Setting hostname: x4.osted.lan. re0: link state changed to DOWN Starting Network: lo0 re0 fwe0 fwip0. lo0: flags=8049 metric 0 mtu 16384 options=3 inet 127.0.0.1 netmask 0xff000000 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 nd6 options=21 re0: flags=8843 metric 0 mtu 1500 options=389b ether 00:21:85:15:33:7d inet 192.168.1.7 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::221:85ff:fe15:337d%re0 prefixlen 64 tentative scopeid 0x1 nd6 options=29 media: Ethernet autoselect (none) status: no carrier fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb ch 1 dma -1 fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 Starting devd. Starting Network: fwe0. fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb ch 1 dma -1 Starting Network: fwip0. fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 add net default: gateway 192.168.1.1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/lib /usr/local/lib/compat/pkg /usr/local/lib/wine a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout Creating and/or trimming log files. Starting syslogd. No core dumps found. Additional ABI support: linux. Starting rpcbind. NFS access cache time=60 Clearing /tmp (X related). Starting mountd. Starting nfsd. Updating motd:. Starting ntpd. Configuring syscons: keymap blanktime. Starting sshd. Starting cron. Local package initialization: backuplock order reversal: 1st 0xc7fc5058 ufs (ufs) @ kern/vfs_subr.c:2111 2nd 0xdad14c40 bufwait (bufwait) @ ufs/ffs/ffs_softdep.c:11310 3rd 0xc82b3310 ufs (ufs) @ kern/vfs_subr.c:2111 KDB: stack backtrace: db_trace_self_wrapper(c0ce496d,2e706564,31313a63,d303133,e992000a,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c08f266b,c0ce7b8b,c6d40168,c6d437d8,e9836890,...) at kdb_backtrace+0x2a _witness_debugger(c0ce7b8b,c82b3310,c0cd94a6,c6d437d8,c0ceed23,...) at _witness_debugger+0x25 witness_checkorder(c82b3310,9,c0ceed1a,83f,0,...) at witness_checkorder+0x839 __lockmgr_args(c82b3310,80100,c82b337c,0,0,...) at __lockmgr_args+0x814 ffs_lock(e98369b4,c090319b,c0cee162,80100,c82b32b8,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c0df5780,e98369b4,109,c0e10600,c82b32b8,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c82b32b8,80100,c0ceed1a,83f,4,...) at _vn_lock+0x78 vget(c82b32b8,80100,c81d25a0,50,0,...) at vget+0xbb vfs_hash_get(c7fca2d4,974c00,80000,c81d25a0,e9836b08,...) at vfs_hash_get+0xed ffs_vgetf(c7fca2d4,974c00,80000,e9836b08,1,...) at ffs_vgetf+0x49 softdep_sync_metadata(c7fc5000,0,c0d0b9a1,144,0,...) at softdep_sync_metadata+0x683 ffs_syncvnode(c7fc5000,1,c81d25a0,555,c0ceed1a,...) at ffs_syncvnode+0x3e2 ffs_sync(c7fca2d4,1,c0cee4a3,4fd,80,...) at ffs_sync+0x26f dounmount(c7fca2d4,8000000,c81d25a0,482,5b85c5ea,...) at dounmount+0x447 unmount(c81d25a0,e9836cec,28176456,1,0,...) at unmount+0x310 syscallenter(c81d25a0,e9836ce4,c0c016ad,c0e455b0,0,...) at syscallenter+0x263 syscall(e9836d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, unmount), eip = 0x280dba9f, esp = 0xbfbfe66c, ebp = 0xbfbfe738 --- lock order reversal: 1st 0xc7fc5880 ufs (ufs) @ kern/vfs_mount.c:1208 2nd 0xc7ca41b4 devfs (devfs) @ ufs/ffs/ffs_softdep.c:1597 KDB: stack backtrace: db_trace_self_wrapper(c0ce496d,35313a63,a0d3739,c0c61b00,e98369a8,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c08f266b,c0ce7b72,c6d437d8,c6d436a0,e9836a04,...) at kdb_backtrace+0x2a _witness_debugger(c0ce7b72,c7ca41b4,c0cd50b7,c6d436a0,c0d08b19,...) at _witness_debugger+0x25 witness_checkorder(c7ca41b4,9,c0d08b10,63d,c7ca4220,...) at witness_checkorder+0x839 __lockmgr_args(c7ca41b4,80400,c7ca4220,0,0,...) at __lockmgr_args+0x814 vop_stdlock(e9836b24,c0cdfaea,df,80400,c7ca415c,...) at vop_stdlock+0x65 VOP_LOCK1_APV(c0dcdda0,e9836b24,0,c0e10600,c7ca415c,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c7ca415c,80400,c0d08b10,63d,c7fca2d4,...) at _vn_lock+0x78 softdep_flushworklist(c7fca2d4,e9836bd0,c81d25a0,563,c0ceed1a,...) at softdep_flushworklist+0x47 ffs_sync(c7fca2d4,1,c0cee4a3,4fd,80,...) at ffs_sync+0x2fd dounmount(c7fca2d4,8000000,c81d25a0,482,5b85c5ea,...) at dounmount+0x447 unmount(c81d25a0,e9836cec,28176456,1,0,...) at unmount+0x310 syscallenter(c81d25a0,e9836ce4,c0c016ad,c0e455b0,0,...) at syscallenter+0x263 syscall(e9836d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, unmount), eip = 0x280dba9f, esp = 0xbfbfe66c, ebp = 0xbfbfe738 --- ** /dev/ad4s1g ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 23462 files, 730688 used, 117583293 free (437 frags, 14697857 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** usage: kill [-s signal_name] pid ... kill -l [exit_status] kill -signal_name pid ... kill -signal_number pid ... fsck -y /tmp. Starting default moused. Starting inetd. Fri Oct 15 09:14:29 CEST 2010 FreeBSD/i386 (x4.osted.lan) (console) login: panic: Bad link elm 0xc8900000 next->prev != elm cpuid = 2 KDB: enter: panic [ thread pid 1769 tid 102184 ] Stopped at kdb_enter+0x3a: movl $0,kdb_why db> run pho db:0:pho> bt Tracing pid 1769 tid 102184 td 0xc8900000 kdb_enter(c0ce1363,c0ce1363,c0c77a9f,eb065c18,2,...) at kdb_enter+0x3a panic(c0c77a9f,c8900000,c0ce23bf,3fe,c6f7b7f8,...) at panic+0x134 tidhash_remove(c8900000,284064c0,7fffffff,0,c6f7b7f8,...) at tidhash_remove+0x137 thr_exit(c8900000,eb065cec,c8900000,eb065cc0,0,...) at thr_exit+0x4e syscallenter(c8900000,eb065ce4,246,c6d3fd58,c89000ac,...) at syscallenter+0x263 syscall(eb065d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (431, FreeBSD ELF32, thr_exit), eip = 0x2810513b, esp = 0xbe6ebeac, ebp = 0xbe6ebed8 --- db:0:bt> show allpcpu Current CPU: 2 cpuid = 0 dynamic pcpu = 0x614300 curthread = 0xc6d9b000: pid 11 "idle: cpu0" curpcb = 0xc6b23d80 fpcurthread = none idlethread = 0xc6d9b000: tid 100006 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: cpuid = 1 dynamic pcpu = 0x5cf9300 curthread = 0xc82e95a0: pid 1773 "thr2" curpcb = 0xe9898d80 fpcurthread = none idlethread = 0xc6d9b2d0: tid 100005 "idle: cpu1" APIC ID = 1 currentldt = 0x50 spin locks held: cpuid = 2 dynamic pcpu = 0x5cfc300 curthread = 0xc8900000: pid 1769 "thr2" curpcb = 0xeb065d80 fpcurthread = none idlethread = 0xc6d9b5a0: tid 100004 "idle: cpu2" APIC ID = 2 currentldt = 0x50 spin locks held: cpuid = 3 dynamic pcpu = 0x5cff300 curthread = 0xc8694870: pid 1798 "netstat" curpcb = 0xe9a90d80 fpcurthread = none idlethread = 0xc6d9b870: tid 100003 "idle: cpu3" APIC ID = 3 currentldt = 0x50 spin locks held: db:0:allpcpu> show alllocks Process 1798 (netstat) thread 0xc8694870 (100307) exclusive sleep mutex 16 (UMA zone) r = 0 (0xc15ba788) locked @ vm/uma_core.c:3201 exclusive sleep mutex UMA lock (UMA lock) r = 0 (0xc0fbea8c) locked @ vm/uma_core.c:3183 exclusive sleep mutex Giant (Giant) r = 0 (0xc0e455b0) locked @ kern/kern_sysctl.c:1418 shared sx sysctl lock (sysctl lock) r = 0 (0xc0e45cc4) locked @ kern/kern_sysctl.c:1522 exclusive sx sysctl mem (sysctl mem) r = 0 (0xc0e45cd8) locked @ kern/kern_sysctl.c:1514 Process 1773 (thr2) thread 0xc82e95a0 (100154) exclusive sleep mutex vm object (standard object) r = 0 (0xc835a990) locked @ vm/vm_object.c:1019 shared sx user map (user map) r = 0 (0xc7cdd7a8) locked @ vm/vm_map.c:1980 Process 1769 (thr2) thread 0xc8900000 (102184) exclusive rw tidhash (tidhash) r = 0 (0xc0e45d68) locked @ kern/kern_thread.c:1006 Process 1499 (sshd) thread 0xc7c55870 (100111) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xc7ff4bd0) locked @ kern/uipc_sockbuf.c:148 Process 1496 (sshd) thread 0xc7cc6b40 (100103) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xc82b86fc) locked @ kern/uipc_sockbuf.c:148 Process 1495 (sshd) thread 0xc81d0b40 (100132) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xc80ce898) locked @ kern/uipc_sockbuf.c:148 Process 1395 (sshd) thread 0xc82eb870 (100148) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xc82aca34) locked @ kern/uipc_sockbuf.c:148 db:0:alllocks> show lockedvnods Locked vnodes db:0:lockedvnods> show mount 0xc7c12b50 /dev/ad4s1a on / (ufs) 0xc7c13000 devfs on /dev (devfs) 0xc7fca5a8 /dev/ad4s1f on /home (ufs) 0xc7fca000 /dev/ad4s1d on /usr (ufs) 0xc7c13b50 /dev/ad4s1e on /var (ufs) 0xc7c1387c /dev/ad4s1g on /tmp (ufs) More info: show mount db:0:mount> ps pid ppid pgrp uid state wmesg wchan cmd 1801 1507 1502 1001 S piperd 0xc884f930 awk 1800 1507 1502 1001 S piperd 0xc7cccab8 sed 1799 1507 1502 1001 R head 1798 1507 1502 1001 RE CPU 3 netstat 1775 1762 1761 1001 R+ (threaded) thr2 101178 RunQ thr2 101177 CanRun thr2 101173 RunQ thr2 101170 CanRun thr2 101161 RunQ thr2 101160 CanRun thr2 100376 RunQ thr2 101144 RunQ thr2 101139 CanRun thr2 101130 RunQ thr2 101127 CanRun thr2 101116 RunQ thr2 101110 CanRun thr2 101102 RunQ thr2 101096 CanRun thr2 101086 RunQ thr2 101081 CanRun thr2 101072 RunQ thr2 101064 CanRun thr2 101051 RunQ thr2 101041 CanRun thr2 101031 RunQ thr2 101026 CanRun thr2 101019 CanRun thr2 100080 S uwait 0xc7c89500 thr2 1774 1762 1761 1001 L+ (threaded) thr2 100121 L *TID lock 0xc86db080 thr2 1773 1762 1761 1001 R+ (threaded) thr2 100154 Run CPU 1 thr2 1772 1762 1761 1001 L+ (threaded) thr2 100118 L *TID lock 0xc86db080 thr2 1771 1762 1761 1001 R+ (threaded) thr2 102183 RunQ thr2 102179 RunQ thr2 100117 S uwait 0xc7cd0b80 thr2 1770 1762 1761 1001 L+ (threaded) thr2 100246 L *TID lock 0xc86db080 thr2 1769 1762 1761 1001 R+ (threaded) thr2 102418 RunQ thr2 102415 CanRun thr2 102414 RunQ thr2 102411 CanRun thr2 102410 RunQ thr2 102407 CanRun thr2 102406 RunQ thr2 102403 CanRun thr2 102402 RunQ thr2 102400 CanRun thr2 102398 CanRun thr2 102397 RunQ thr2 102394 CanRun thr2 102393 RunQ thr2 102390 CanRun thr2 102389 RunQ thr2 102386 CanRun thr2 102385 RunQ thr2 102382 CanRun thr2 102381 RunQ thr2 102378 CanRun thr2 102377 RunQ thr2 102374 CanRun thr2 102373 RunQ thr2 102370 CanRun thr2 102369 RunQ thr2 102366 CanRun thr2 102365 RunQ thr2 102362 CanRun thr2 102361 RunQ thr2 102358 CanRun thr2 102357 RunQ thr2 102354 CanRun thr2 102353 RunQ thr2 102350 CanRun thr2 102349 RunQ thr2 102346 CanRun thr2 102345 RunQ thr2 102342 CanRun thr2 102341 RunQ thr2 102338 CanRun thr2 102337 RunQ thr2 102334 CanRun thr2 102333 RunQ thr2 102330 CanRun thr2 102329 RunQ thr2 102326 CanRun thr2 102325 RunQ thr2 102322 CanRun thr2 102321 RunQ thr2 102318 CanRun thr2 102317 RunQ thr2 102314 CanRun thr2 102313 RunQ thr2 102310 CanRun thr2 102309 RunQ thr2 102306 CanRun thr2 102305 RunQ thr2 102302 CanRun thr2 102301 RunQ thr2 102298 CanRun thr2 102297 RunQ thr2 102294 CanRun thr2 102293 RunQ thr2 102290 CanRun thr2 102289 RunQ thr2 102286 CanRun thr2 102285 RunQ thr2 102282 CanRun thr2 102281 RunQ thr2 102278 CanRun thr2 102277 RunQ thr2 102274 CanRun thr2 102273 RunQ thr2 102270 CanRun thr2 102269 RunQ thr2 102266 CanRun thr2 102265 RunQ thr2 102261 CanRun thr2 102260 RunQ thr2 102257 CanRun thr2 102256 RunQ thr2 102253 RunQ thr2 102250 CanRun thr2 102249 RunQ thr2 102246 CanRun thr2 102245 RunQ thr2 102242 CanRun thr2 102241 RunQ thr2 102238 CanRun thr2 102237 RunQ thr2 102234 CanRun thr2 102233 RunQ thr2 102230 CanRun thr2 102229 RunQ thr2 102226 CanRun thr2 102225 RunQ thr2 102222 CanRun thr2 102221 RunQ thr2 102218 CanRun thr2 102217 RunQ thr2 102215 CanRun thr2 102214 RunQ thr2 102211 CanRun thr2 102210 RunQ thr2 102207 CanRun thr2 102206 RunQ thr2 102203 CanRun thr2 102202 RunQ thr2 102199 CanRun thr2 102198 RunQ thr2 102195 CanRun thr2 102194 RunQ thr2 102191 CanRun thr2 102190 RunQ thr2 102187 CanRun thr2 101157 RunQ thr2 102185 CanRun thr2 102184 Run CPU 2 thr2 100087 RunQ thr2 1768 1762 1761 1001 L+ (threaded) thr2 100166 L *TID lock 0xc86db080 thr2 1767 1762 1761 1001 R+ (threaded) thr2 101847 RunQ thr2 100199 S uwait 0xc8541900 thr2 1766 1762 1761 1001 L+ (threaded) thr2 100195 L *TID lock 0xc86db080 thr2 1765 1762 1761 1001 R+ (threaded) thr2 101186 RunQ thr2 101180 CanRun thr2 101167 RunQ thr2 101164 CanRun thr2 101155 RunQ thr2 101152 CanRun thr2 101145 CanRun thr2 101136 RunQ thr2 101133 CanRun thr2 101124 RunQ thr2 101117 CanRun thr2 101108 RunQ thr2 101103 CanRun thr2 101095 RunQ thr2 101090 CanRun thr2 101080 RunQ thr2 101075 CanRun thr2 101059 RunQ thr2 101052 CanRun thr2 101040 RunQ thr2 101032 CanRun thr2 100169 CanRun thr2 1764 1762 1761 1001 L+ (threaded) thr2 100158 L *TID lock 0xc86db080 thr2 1763 1762 1761 1001 L+ (threaded) thr2 100180 L *TID lock 0xc86db080 thr2 1762 1761 1761 1001 S+ wait 0xc81cc550 thr2 1761 1398 1761 1001 S+ nanslp 0xc0e45e44 initial thread 1508 1502 1502 1001 S piperd 0xc80e2498 awk 1507 1502 1502 1001 S wait 0xc7c53000 sh 1506 1505 1506 1001 Ss kqread 0xc7be3880 tail 1505 1499 1499 1001 S select 0xc7c159a4 sshd 1504 1503 1504 1001 Ss+ select 0xc7cf17a4 top 1503 1496 1496 1001 S select 0xc7cf15e4 sshd 1502 1501 1502 1001 Ss wait 0xc84e27f8 sh 1501 1495 1495 1001 S select 0xc7cf12a4 sshd 1499 1353 1499 0 Ss sbwait 0xc7ff4bfc sshd 1496 1353 1496 0 Ss sbwait 0xc82b8728 sshd 1495 1353 1495 0 Ss sbwait 0xc80ce8c4 sshd 1491 1 1491 0 Ss+ ttyin 0xc6ec3a70 getty 1490 1 1490 0 Ss+ ttyin 0xc6ec3c70 getty 1489 1 1489 0 Ss+ ttyin 0xc6ec3e70 getty 1488 1 1488 0 Ss+ ttyin 0xc7170070 getty 1487 1 1487 0 Ss+ ttyin 0xc7170270 getty 1486 1 1486 0 Ss+ ttyin 0xc7170470 getty 1485 1 1485 0 Ss+ ttyin 0xc6ec2670 getty 1484 1 1484 0 Ss+ ttyin 0xc6ec2070 getty 1483 1 1483 0 Ss+ ttyin 0xc6ec2270 getty 1452 1 1452 0 Ss select 0xc7bc39e4 inetd 1426 1 1426 0 Ss select 0xc7cf17e4 moused 1398 1397 1398 1001 Ss+ wait 0xc82e5000 bash 1397 1395 1395 1001 S select 0xc7cf14a4 sshd 1395 1353 1395 0 Ss sbwait 0xc82aca60 sshd 1372 1 1372 0 Ss nanslp 0xc0e45e44 cron 1365 1 1365 25 Ss pause 0xc82e5af8 sendmail 1361 1 1361 0 Ss select 0xc72ef5e4 sendmail 1353 1 1353 0 Ss select 0xc7c15aa4 sshd 1308 1 1308 0 Ss select 0xc72ef3e4 ntpd 1176 1175 1175 0 S (threaded) nfsd 100141 S rpcsvc 0xc7d114d0 nfsd: service 100140 S rpcsvc 0xc7d11510 nfsd: service 100139 S rpcsvc 0xc7d11550 nfsd: service 100105 S rpcsvc 0xc6f60950 nfsd: master 1175 1 1175 0 Ss select 0xc7cf1564 nfsd 1166 1 1166 0 Ss select 0xc7cf1ca4 mountd 1069 1 1069 0 Ss select 0xc72ef864 rpcbind 1036 1 1036 0 Ss select 0xc7cf18a4 syslogd 835 1 835 0 Ss select 0xc7cf1024 devd 21 0 0 0 DL flowclea 0xc0fb2f08 [flowcleaner] 20 0 0 0 DL sdflush 0xc0fbe6e0 [softdepflush] 19 0 0 0 DL vlruwt 0xc7bf4550 [vnlru] 18 0 0 0 DL syncer 0xc0fb2d18 [syncer] 17 0 0 0 DL psleep 0xc0fb2a48 [bufdaemon] 16 0 0 0 DL pgzero 0xc0fc055c [pagezero] 15 0 0 0 DL psleep 0xc0fc0150 [vmdaemon] 9 0 0 0 DL psleep 0xc0fc0118 [pagedaemon] 8 0 0 0 DL ccb_scan 0xc0e11ad4 [xpt_thrd] 7 0 0 0 DL waiting_ 0xc0fb4738 [sctp_iterator] 6 0 0 0 DL - 0xc705603c [fdc0] 5 0 0 0 SL - 0xc7059000 [fw0_probe] 14 0 0 0 DL (threaded) [usb] 100056 D - 0xc7034d34 [usbus5] 100055 D - 0xc7034d04 [usbus5] 100054 D - 0xc7034cd4 [usbus5] 100053 D - 0xc7034ca4 [usbus5] 100051 D - 0xc702db5c [usbus4] 100050 D - 0xc702db2c [usbus4] 100049 D - 0xc702dafc [usbus4] 100048 D - 0xc702dacc [usbus4] 100047 D - 0xc7026b5c [usbus3] 100046 D - 0xc7026b2c [usbus3] 100045 D - 0xc7026afc [usbus3] 100044 D - 0xc7026acc [usbus3] 100043 D - 0xc701db5c [usbus2] 100042 D - 0xc701db2c [usbus2] 100041 D - 0xc701dafc [usbus2] 100040 D - 0xc701dacc [usbus2] 100038 D - 0xc7017b5c [usbus1] 100037 D - 0xc7017b2c [usbus1] 100036 D - 0xc7017afc [usbus1] 100035 D - 0xc7017acc [usbus1] 100033 D - 0xc7007b5c [usbus0] 100032 D - 0xc7007b2c [usbus0] 100031 D - 0xc7007afc [usbus0] 100030 D - 0xc7007acc [usbus0] 13 0 0 0 DL - 0xc0e45c84 [yarrow] 4 0 0 0 DL - 0xc0e43944 [g_down] 3 0 0 0 DL - 0xc0e43940 [g_up] 2 0 0 0 DL - 0xc0e43938 [g_event] 12 0 0 0 WL (threaded) [intr] 100064 I [irq12: psm0] 100063 I [irq1: atkbd0] 100061 I [swi0: uart] 100058 I [irq20: fwohci0] 100057 I [irq14: ata0] 100052 I [irq19: ehci0] 100039 I [irq18: ohci2 ohci4] 100034 I [irq17: ohci1 ohci3] 100029 I [irq16: ohci0] 100028 I [irq22: atapci0] 100027 I [irq9: acpi0] 100026 I [swi6: Giant taskq] 100024 I [swi5: +] 100019 I [swi2: cambio] 100018 I [swi6: task queue] 100012 I [swi4: clock] 100011 I [swi4: clock] 100010 I [swi4: clock] 100009 I [swi4: clock] 100008 I [swi1: netisr 0] 100007 I [swi3: vm] 11 0 0 0 RL (threaded) [idle] 100006 Run CPU 0 [idle: cpu0] 100005 CanRun [idle: cpu1] 100004 CanRun [idle: cpu2] 100003 CanRun [idle: cpu3] 1 0 1 0 SLs wait 0xc6d99d48 [init] 10 0 0 0 DL audit_wo 0xc0fbdf00 [audit] 0 0 0 0 DLs (threaded) [kernel] 100065 D deadlkre 0xc0e45c84 [deadlkres] 100059 D - 0xc6eabb40 [fw0_taskq] 100025 D - 0xc6eabc80 [thread taskq] 100023 D - 0xc6eabd00 [kqueue taskq] 100022 D - 0xc6eabd40 [acpi_task_2] 100021 D - 0xc6eabd40 [acpi_task_1] 100020 D - 0xc6eabd40 [acpi_task_0] 100016 D - 0xc6eabdc0 [firmware taskq] 100000 D sched 0xc0e43a20 [swapper] db:0:ps> allt Tracing command awk pid 1801 tid 100136 td 0xc7cd6870 sched_switch(c7cd6870,0,104,191,bdf3b428,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,4c,...) at mi_switch+0x200 sleepq_switch(c7cd6870,0,c0ce5bb4,1a7,4c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce5bb4,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c884f930,4c,c0ce812a,100,0,...) at sleepq_wait_sig+0x17 _sleep(c884f930,c884faa0,14c,c0ce812a,0,...) at _sleep+0x35c pipe_read(c82d9c08,e985ac28,c6d7cc00,0,c7cd6870,...) at pipe_read+0x417 dofileread(e985ac28,ffffffff,ffffffff,0,c82d9c08,...) at dofileread+0x9e kern_readv(c7cd6870,0,e985ac28,e985ac48,1,...) at kern_readv+0x58 read(c7cd6870,e985acec,284bc000,1,0,...) at read+0x4f syscallenter(c7cd6870,e985ace4,c0c016ad,c0e455b0,0,...) at syscallenter+0x263 syscall(e985ad28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x281b7e53, esp = 0xbfbfea2c, ebp = 0xbfbfea48 --- Tracing command sed pid 1800 tid 100203 td 0xc86a3870 sched_switch(c86a3870,0,104,191,bdb8710e,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,4c,...) at mi_switch+0x200 sleepq_switch(c86a3870,0,c0ce5bb4,1a7,4c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce5bb4,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7cccab8,4c,c0ce812a,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7cccab8,c7cccc28,14c,c0ce812a,0,...) at _sleep+0x35c pipe_read(c7cfebd0,e9928c28,c6d7cc00,0,c86a3870,...) at pipe_read+0x417 dofileread(e9928c28,ffffffff,ffffffff,0,c7cfebd0,...) at dofileread+0x9e kern_readv(c86a3870,0,e9928c28,e9928c48,1,...) at kern_readv+0x58 read(c86a3870,e9928cec,e9928d28,c0ce63ed,0,...) at read+0x4f syscallenter(c86a3870,e9928ce4,e9928ce4,0,0,...) at syscallenter+0x263 syscall(e9928d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28186e53, esp = 0xbfbfea2c, ebp = 0xbfbfea48 --- Tracing command head pid 1799 tid 100205 td 0xc8829870 sched_switch(c8829870,0,104,191,bd96294e,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,4c,...) at mi_switch+0x200 sleepq_switch(c8829870,0,c0ce5bb4,1a7,4c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce5bb4,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7c9c930,4c,c0ce812a,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7c9c930,c7c9caa0,14c,c0ce812a,0,...) at _sleep+0x35c pipe_read(c7c5b380,e9934c28,c6d7cc00,0,c8829870,...) at pipe_read+0x417 dofileread(e9934c28,ffffffff,ffffffff,0,c7c5b380,...) at dofileread+0x9e kern_readv(c8829870,0,e9934c28,e9934c48,1,...) at kern_readv+0x58 read(c8829870,e9934cec,28406000,1,0,...) at read+0x4f syscallenter(c8829870,e9934ce4,c0c016ad,c0e455b0,0,...) at syscallenter+0x263 syscall(e9934d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28180e53, esp = 0xbfbfeb8c, ebp = 0xbfbfeba8 --- Tracing command netstat pid 1798 tid 100307 td 0xc8694870 cpustop_handler(8,e9a90a7c,c0c00f66,c0d1f3bc,ec,...) at cpustop_handler+0x34 ipi_nmi_handler(c0d1f3bc,ec,0,c08ab2f0,c88677f8,...) at ipi_nmi_handler+0x2f trap(e9a90a88) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc08ab2ff, esp = 0xe9a90ac8, ebp = 0xe9a90ad8 --- _mtx_lock_spin(c0e58394,c8694870,0,c0d1bc09,48e,...) at _mtx_lock_spin+0x3f _mtx_lock_spin_flags(c0e58394,0,c0d1bc09,48e,f6,...) at _mtx_lock_spin_flags+0x146 smp_tlb_shootdown(e9a90b50,c0bf7fcf,c5317000,c5318000,0,...) at smp_tlb_shootdown+0x66 smp_invlpg_range(c5317000,c5318000,0,1064,c5318000,...) at smp_invlpg_range+0x1c pmap_invalidate_range(c0fe0b60,c5317000,c5318000) at pmap_invalidate_range+0x4f pmap_qremove(c5317000,1,c0d1c52d,1064,c08838e7,...) at pmap_qremove+0x54 pmap_remove_pages(c7feead0,c0cdfaea,df,0,c88677f8,...) at pmap_remove_pages+0x563 vmspace_exit(c8694870,0,c0cdc32b,12d,c7feea24,...) at vmspace_exit+0x90 exit1(c8694870,0,e9a90c7c,c08fba43,c8694870,...) at exit1+0x5ee sys_exit(c8694870,e9a90cec,281f730c,1,0,...) at sys_exit+0x1d syscallenter(c8694870,e9a90ce4,c0c016ad,c0e455b0,0,...) at syscallenter+0x263 syscall(e9a90d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (1, FreeBSD ELF32, sys_exit), eip = 0x2814eaef, esp = 0xbfbfec7c, ebp = 0xbfbfec88 --- Tracing command thr2 pid 1775 tid 101178 td 0xc8941870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1775 tid 101177 td 0xc86d6b40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1775 tid 101173 td 0xc89fbb40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1775 tid 101170 td 0xc86e15a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1775 tid 101161 td 0xc89e4870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1775 tid 101160 td 0xc89942d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1775 tid 100376 td 0xc880f000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1775 tid 101144 td 0xc87f5000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1775 tid 101139 td 0xc896ab40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1775 tid 101130 td 0xc897e870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1775 tid 101127 td 0xc86f8000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1775 tid 101116 td 0xc86eeb40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1775 tid 101110 td 0xc854eb40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1775 tid 101102 td 0xc886f870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1775 tid 101096 td 0xc87db870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1775 tid 101086 td 0xc8972870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1775 tid 101081 td 0xc8951000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1775 tid 101072 td 0xc89e5000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1775 tid 101064 td 0xc8744b40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1775 tid 101051 td 0xc87102d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1775 tid 101041 td 0xc865cb40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1775 tid 101031 td 0xc8a27000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1775 tid 101026 td 0xc8a03b40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1775 tid 101019 td 0xc8a10000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1775 tid 100080 td 0xc7c8e2d0 sched_switch(c7c8e2d0,0,104,191,c020d9c2,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7c8e2d0,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce5bb4,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7c89500,0,c0ce2b6b,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7c89500,c0e48700,100,c0ce2b6b,0,...) at _sleep+0x35c do_wait(0,0,0,bfbfe530,e973acec) at do_wait+0x511 __umtx_op_wait(c7c8e2d0,e973acec,e973ac7c,c08fba43,c7c8e2d0,...) at __umtx_op_wait+0x5f _umtx_op(c7c8e2d0,e973acec,e973ad28,c0ce63ed,0,...) at _umtx_op+0x27 syscallenter(c7c8e2d0,e973ace4,e973ace4,0,ec,...) at syscallenter+0x263 syscall(e973ad28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (454, FreeBSD ELF32, _umtx_op), eip = 0x280ae883, esp = 0xbfbfe52c, ebp = 0xbfbfe548 --- Tracing command thr2 pid 1774 tid 100121 td 0xc7cd6b40 sched_switch(c7cd6b40,0,103,18c,c0c2fcee,...) at sched_switch+0x3bc mi_switch(103,0,c0ce64b7,2e2,c86db080,...) at mi_switch+0x200 turnstile_wait(c86db080,c867ab40,0,1ac,c0e45d84,...) at turnstile_wait+0x495 _mtx_lock_sleep(c0e45d84,c7cd6b40,0,c0ce6c8b,36f,...) at _mtx_lock_sleep+0x14d _mtx_lock_flags(c0e45d84,0,c0ce6c8b,36f,c7cd3000,...) at _mtx_lock_flags+0xf7 free_unr(c6d74080,187e5,e9abe000,e982cb20,c8a2d870) at free_unr+0x8c thread_dtor(c8a2d870,2cc,0,2,e9abc000,...) at thread_dtor+0x17f uma_zfree_arg(c1598a80,c8a2d870,0) at uma_zfree_arg+0x29 thread_free(c8a2d870,0,c0ce23bf,11b,e982cc14,...) at thread_free+0x52 thread_reap(c7cd6bec,c0f78a60,c0ddbf40,e982cc14,e982cbcc,...) at thread_reap+0xc0 thread_alloc(0,e982cbc0,c08c2d80,c7cdd048,280a7380,...) at thread_alloc+0xc create_thread(28404380,bf8ff000,100000,2808e490,28404380,28404380,2,0,246) at create_thread+0x9c kern_thr_new(c7cd6b40,e982cc14,34,280a7380,28404380,...) at kern_thr_new+0x77 thr_new(c7cd6b40,e982ccec,e982cd28,c0ce63ed,0,...) at thr_new+0x55 syscallenter(c7cd6b40,e982cce4,e982cce4,0,ec,...) at syscallenter+0x263 syscall(e982cd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (455, FreeBSD ELF32, thr_new), eip = 0x28104e5b, esp = 0xbfbfe4fc, ebp = 0xbfbfe5b8 --- Tracing command thr2 pid 1773 tid 100154 td 0xc82e95a0 cpustop_handler(2,e9898a34,c0c00f66,c090319b,c0d0e6c4,...) at cpustop_handler+0x34 ipi_nmi_handler(c090319b,c0d0e6c4,c0cd8871,3,c82e5550,...) at ipi_nmi_handler+0x2f trap(e9898a40) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0bf3ec8, esp = 0xe9898a80, ebp = 0xe9898aa0 --- smp_tlb_shootdown(e9898ac8,c0bf7fcf,e9b37000,e9b39000,c6d3d380,...) at smp_tlb_shootdown+0x98 smp_invlpg_range(e9b37000,e9b39000,c6d3d380,c6d3d178,e9b39000,...) at smp_invlpg_range+0x1c pmap_invalidate_range(c0fe0b60,e9b37000,e9b39000) at pmap_invalidate_range+0x4f pmap_qremove(e9b37000,2,c6d3d318,c0f76f38,2,...) at pmap_qremove+0x54 vm_thread_stack_dispose(c0e45d9c,c82e95a0,c89cd5a0,e9898b40,c89cd5a0,...) at vm_thread_stack_dispose+0x26 vm_thread_dispose(c89cd5a0,c0cdfaea,109,c89cd5a0,e9898b78,...) at vm_thread_dispose+0xaa thread_free(c89cd5a0,0,c0ce23bf,11b,e9898c14,...) at thread_free+0x39 thread_reap(df,c3901400,0,e9898c14,e9898bcc,...) at thread_reap+0xc0 thread_alloc(0,e9898bc0,c08c2d80,c7cdd7a8,280a7380,...) at thread_alloc+0xc create_thread(28404380,bf8ff000,100000,2808e490,28404380,28404380,2,0,246) at create_thread+0x9c kern_thr_new(c82e95a0,e9898c14,34,280a7380,28404380,...) at kern_thr_new+0x77 thr_new(c82e95a0,e9898cec,e9898d28,c0ce63ed,0,...) at thr_new+0x55 syscallenter(c82e95a0,e9898ce4,e9898ce4,0,ec,...) at syscallenter+0x263 syscall(e9898d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (455, FreeBSD ELF32, thr_new), eip = 0x28104e5b, esp = 0xbfbfe4fc, ebp = 0xbfbfe5b8 --- Tracing command thr2 pid 1772 tid 100118 td 0xc7cdb5a0 sched_switch(c7cdb5a0,0,103,18c,c0a8c6d3,...) at sched_switch+0x3bc mi_switch(103,0,c0ce64b7,2e2,c86db080,...) at mi_switch+0x200 turnstile_wait(c86db080,c867ab40,0,1ac,c0e45d84,...) at turnstile_wait+0x495 _mtx_lock_sleep(c0e45d84,c7cdb5a0,0,c0ce6c8b,36f,...) at _mtx_lock_sleep+0x14d _mtx_lock_flags(c0e45d84,0,c0ce6c8b,36f,c7cd37f8,...) at _mtx_lock_flags+0xf7 free_unr(c6d74080,188ea,e9da3000,e9820b20,c8a32870) at free_unr+0x8c thread_dtor(c8a32870,2cc,0,2,e9da1000,...) at thread_dtor+0x17f uma_zfree_arg(c1598a80,c8a32870,0) at uma_zfree_arg+0x29 thread_free(c8a32870,0,c0ce23bf,11b,e9820c14,...) at thread_free+0x52 thread_reap(c7cdb64c,c0f81388,c0ddbf40,e9820c14,e9820bcc,...) at thread_reap+0xc0 thread_alloc(0,e9820bc0,c08c2d80,c6da3980,280a7380,...) at thread_alloc+0xc create_thread(28404380,bf8ff000,100000,281fc190,28404380,28404380,2,0,246) at create_thread+0x9c kern_thr_new(c7cdb5a0,e9820c14,34,280a7380,28404380,...) at kern_thr_new+0x77 thr_new(c7cdb5a0,e9820cec,e9820d28,c0ce63ed,0,...) at thr_new+0x55 syscallenter(c7cdb5a0,e9820ce4,e9820ce4,0,ec,...) at syscallenter+0x263 syscall(e9820d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (455, FreeBSD ELF32, thr_new), eip = 0x28104e5b, esp = 0xbfbfe4fc, ebp = 0xbfbfe5b8 --- Tracing command thr2 pid 1771 tid 102183 td 0xc865c2d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1771 tid 102179 td 0xc89a02d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1771 tid 100117 td 0xc7cdb870 sched_switch(c7cdb870,0,104,191,c1292adf,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7cdb870,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce5bb4,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7cd0b80,0,c0ce2b6b,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7cd0b80,c0e49420,100,c0ce2b6b,0,...) at _sleep+0x35c do_wait(0,0,0,bfbfe530,e981ccec) at do_wait+0x511 __umtx_op_wait(c7cdb870,e981ccec,e981cc7c,c08fba43,c7cdb870,...) at __umtx_op_wait+0x5f _umtx_op(c7cdb870,e981ccec,e981cd28,c0ce63ed,0,...) at _umtx_op+0x27 syscallenter(c7cdb870,e981cce4,e981cce4,0,ec,...) at syscallenter+0x263 syscall(e981cd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (454, FreeBSD ELF32, _umtx_op), eip = 0x280ae883, esp = 0xbfbfe52c, ebp = 0xbfbfe548 --- Tracing command thr2 pid 1770 tid 100246 td 0xc880b870 sched_switch(c880b870,0,103,18c,c0647e82,...) at sched_switch+0x3bc mi_switch(103,0,c0ce64b7,2e2,c86db080,...) at mi_switch+0x200 turnstile_wait(c86db080,c867ab40,0,1ac,c0e45d84,...) at turnstile_wait+0x495 _mtx_lock_sleep(c0e45d84,c880b870,0,c0ce6c8b,36f,...) at _mtx_lock_sleep+0x14d _mtx_lock_flags(c0e45d84,0,c0ce6c8b,36f,c8867aa0,...) at _mtx_lock_flags+0xf7 free_unr(c6d74080,18785,e99d1000,e99c4b20,c887b5a0) at free_unr+0x8c thread_dtor(c887b5a0,2cc,0,2,e99cf000,...) at thread_dtor+0x17f uma_zfree_arg(c1598a80,c887b5a0,0) at uma_zfree_arg+0x29 thread_free(c887b5a0,0,c0ce23bf,11b,e99c4c14,...) at thread_free+0x52 thread_reap(c880b91c,c0f7f910,c0ddbf40,e99c4c14,e99c4bcc,...) at thread_reap+0xc0 thread_alloc(0,e99c4bc0,c08c2d80,c7fee7a8,280a7380,...) at thread_alloc+0xc create_thread(28404380,bf8ff000,100000,2808e490,28404380,28404380,2,0,246) at create_thread+0x9c kern_thr_new(c880b870,e99c4c14,34,280a7380,28404380,...) at kern_thr_new+0x77 thr_new(c880b870,e99c4cec,e99c4d28,c0ce63ed,0,...) at thr_new+0x55 syscallenter(c880b870,e99c4ce4,e99c4ce4,0,ec,...) at syscallenter+0x263 syscall(e99c4d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (455, FreeBSD ELF32, thr_new), eip = 0x28104e5b, esp = 0xbfbfe4fc, ebp = 0xbfbfe5b8 --- Tracing command thr2 pid 1769 tid 102418 td 0xc8947b40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102415 td 0xc89b9000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102414 td 0xc8830b40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102411 td 0xc8996000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102410 td 0xc88345a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102407 td 0xc87432d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102406 td 0xc8949000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102403 td 0xc87da870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102402 td 0xc867f870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102400 td 0xc860a870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102398 td 0xc85212d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102397 td 0xc8a12b40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102394 td 0xc89eeb40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102393 td 0xc886b870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102390 td 0xc851eb40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102389 td 0xc8a2fb40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102386 td 0xc873b2d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102385 td 0xc893c2d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102382 td 0xc88212d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102381 td 0xc85555a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102378 td 0xc8684000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102377 td 0xc87e22d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102374 td 0xc89ee2d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102373 td 0xc89435a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102370 td 0xc89465a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102369 td 0xc8827870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102366 td 0xc852ab40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102365 td 0xc87f7000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102362 td 0xc89145a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102361 td 0xc852c000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102358 td 0xc8a28870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102357 td 0xc861cb40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102354 td 0xc87a8000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102353 td 0xc8950870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102350 td 0xc8a385a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102349 td 0xc88822d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102346 td 0xc89cc870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102345 td 0xc890e2d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102342 td 0xc8965870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102341 td 0xc87a05a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102338 td 0xc890b2d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102337 td 0xc863e870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102334 td 0xc86cf2d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102333 td 0xc86bcb40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102330 td 0xc86c9b40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102329 td 0xc89c95a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102326 td 0xc8502870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102325 td 0xc8542000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102322 td 0xc86d6870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102321 td 0xc886b5a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102318 td 0xc863e5a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102317 td 0xc87ac870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102314 td 0xc8519b40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102313 td 0xc8a2db40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102310 td 0xc87e1b40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102309 td 0xc8932870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102306 td 0xc86715a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102305 td 0xc88815a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102302 td 0xc8537b40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102301 td 0xc87802d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102298 td 0xc8a2b5a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102297 td 0xc852e5a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102294 td 0xc89e12d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102293 td 0xc86ccb40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102290 td 0xc89c8870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102289 td 0xc87c8870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102286 td 0xc873d2d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102285 td 0xc8745870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102282 td 0xc88792d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102281 td 0xc8826b40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102278 td 0xc86475a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102277 td 0xc8975b40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102274 td 0xc87efb40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102273 td 0xc8513b40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102270 td 0xc8816000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102269 td 0xc868c000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102266 td 0xc8521000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102265 td 0xc8943b40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102261 td 0xc86c0870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102260 td 0xc89c3b40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102257 td 0xc87115a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102256 td 0xc86f15a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102253 td 0xc8a235a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102250 td 0xc8a062d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102249 td 0xc89945a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102246 td 0xc89a82d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102245 td 0xc87ee870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102242 td 0xc87ec5a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102241 td 0xc87d42d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102238 td 0xc888a870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102237 td 0xc8971b40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102234 td 0xc8723b40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102233 td 0xc87bf000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102230 td 0xc8a08000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102229 td 0xc879b5a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102226 td 0xc8a00870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102225 td 0xc8931000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102222 td 0xc890db40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102221 td 0xc8a065a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102218 td 0xc86655a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102217 td 0xc8677870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102215 td 0xc8923870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102214 td 0xc8984870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102211 td 0xc8849000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102210 td 0xc860e000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102207 td 0xc87fe2d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102206 td 0xc87f52d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102203 td 0xc89695a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102202 td 0xc863a000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102199 td 0xc87155a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102198 td 0xc852d000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102195 td 0xc87382d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102194 td 0xc8536b40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102191 td 0xc86512d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102190 td 0xc87a45a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102187 td 0xc86cb2d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 101157 td 0xc87bf5a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102185 td 0xc8a12000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1769 tid 102184 td 0xc8900000 kdb_enter(c0ce1363,c0ce1363,c0c77a9f,eb065c18,2,...) at kdb_enter+0x3a panic(c0c77a9f,c8900000,c0ce23bf,3fe,c6f7b7f8,...) at panic+0x134 tidhash_remove(c8900000,284064c0,7fffffff,0,c6f7b7f8,...) at tidhash_remove+0x137 thr_exit(c8900000,eb065cec,c8900000,eb065cc0,0,...) at thr_exit+0x4e syscallenter(c8900000,eb065ce4,246,c6d3fd58,c89000ac,...) at syscallenter+0x263 syscall(eb065d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (431, FreeBSD ELF32, thr_exit), eip = 0x2810513b, esp = 0xbe6ebeac, ebp = 0xbe6ebed8 --- Tracing command thr2 pid 1769 tid 100087 td 0xc7cc8b40 sched_switch(c7cc8b40,0,104,191,c128b9fb,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7cc8b40,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce5bb4,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7148980,0,c0ce2b6b,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7148980,c0e49330,100,c0ce2b6b,0,...) at _sleep+0x35c do_wait(0,0,0,bfbfe530,e9756cec) at do_wait+0x511 __umtx_op_wait(c7cc8b40,e9756cec,e9756c7c,c08fba43,c7cc8b40,...) at __umtx_op_wait+0x5f _umtx_op(c7cc8b40,e9756cec,e9756d28,c0ce63ed,0,...) at _umtx_op+0x27 syscallenter(c7cc8b40,e9756ce4,e9756ce4,0,74a,...) at syscallenter+0x263 syscall(e9756d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (454, FreeBSD ELF32, _umtx_op), eip = 0x280ae883, esp = 0xbfbfe52c, ebp = 0xbfbfe548 --- Tracing command thr2 pid 1768 tid 100166 td 0xc860b2d0 sched_switch(c860b2d0,0,103,18c,c0dd4c10,...) at sched_switch+0x3bc mi_switch(103,0,c0ce64b7,2e2,c86db080,...) at mi_switch+0x200 turnstile_wait(c86db080,c867ab40,0,1ac,c0e45d84,...) at turnstile_wait+0x495 _mtx_lock_sleep(c0e45d84,c860b2d0,0,c0ce6c8b,36f,...) at _mtx_lock_sleep+0x14d _mtx_lock_flags(c0e45d84,0,c0ce6c8b,36f,c84e1000,...) at _mtx_lock_flags+0xf7 free_unr(c6d74080,18871,e9c74000,e98c5b20,c8a112d0) at free_unr+0x8c thread_dtor(c8a112d0,2cc,0,2,e9c72000,...) at thread_dtor+0x17f uma_zfree_arg(c1598a80,c8a112d0,0) at uma_zfree_arg+0x29 thread_free(c8a112d0,0,c0ce23bf,11b,e98c5c14,...) at thread_free+0x52 thread_reap(c860b37c,c0f83698,c0ddbf40,e98c5c14,e98c5bcc,...) at thread_reap+0xc0 thread_alloc(0,e98c5bc0,c08c2d80,c7cdd220,280a7380,...) at thread_alloc+0xc create_thread(28404380,bf8ff000,100000,281fc190,28404380,28404380,2,0,246) at create_thread+0x9c kern_thr_new(c860b2d0,e98c5c14,34,280a7380,28404380,...) at kern_thr_new+0x77 thr_new(c860b2d0,e98c5cec,e98c5d28,c0ce63ed,0,...) at thr_new+0x55 syscallenter(c860b2d0,e98c5ce4,e98c5ce4,0,ec,...) at syscallenter+0x263 syscall(e98c5d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (455, FreeBSD ELF32, thr_new), eip = 0x28104e5b, esp = 0xbfbfe4fc, ebp = 0xbfbfe5b8 --- Tracing command thr2 pid 1767 tid 101847 td 0xc896b000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1767 tid 100199 td 0xc8542b40 sched_switch(c8542b40,0,104,191,c102d70a,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c8542b40,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce5bb4,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8541900,0,c0ce2b6b,100,0,...) at sleepq_wait_sig+0x17 _sleep(c8541900,c0e493c0,100,c0ce2b6b,0,...) at _sleep+0x35c do_wait(0,0,0,bfbfe530,e9925cec) at do_wait+0x511 __umtx_op_wait(c8542b40,e9925cec,e9925c7c,c08fba43,c8542b40,...) at __umtx_op_wait+0x5f _umtx_op(c8542b40,e9925cec,e9925d28,c0ce63ed,0,...) at _umtx_op+0x27 syscallenter(c8542b40,e9925ce4,e9925ce4,0,ec,...) at syscallenter+0x263 syscall(e9925d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (454, FreeBSD ELF32, _umtx_op), eip = 0x280ae883, esp = 0xbfbfe52c, ebp = 0xbfbfe548 --- Tracing command thr2 pid 1766 tid 100195 td 0xc87df2d0 sched_switch(c87df2d0,0,103,18c,c071bd3f,...) at sched_switch+0x3bc mi_switch(103,0,c0ce64b7,2e2,c86db080,...) at mi_switch+0x200 turnstile_wait(c86db080,c867ab40,0,1ac,c0e45d84,...) at turnstile_wait+0x495 _mtx_lock_sleep(c0e45d84,c87df2d0,0,c0ce6c8b,36f,...) at _mtx_lock_sleep+0x14d _mtx_lock_flags(c0e45d84,0,c0ce6c8b,36f,c82e8d48,...) at _mtx_lock_flags+0xf7 free_unr(c6d74080,18f78,eb156000,e990db20,c860a000) at free_unr+0x8c thread_dtor(c860a000,2cc,0,2,eb154000,...) at thread_dtor+0x17f uma_zfree_arg(c1598a80,c860a000,0) at uma_zfree_arg+0x29 thread_free(c860a000,0,c0ce23bf,11b,e990dc14,...) at thread_free+0x52 thread_reap(c87df37c,c0f84400,c0ddbf40,e990dc14,e990dbcc,...) at thread_reap+0xc0 thread_alloc(0,e990dbc0,c08c2d80,c7ff96bc,280a7380,...) at thread_alloc+0xc create_thread(28404380,bf8ff000,100000,2808e490,28404380,28404380,2,0,246) at create_thread+0x9c kern_thr_new(c87df2d0,e990dc14,34,280a7380,28404380,...) at kern_thr_new+0x77 thr_new(c87df2d0,e990dcec,e990dd28,c0ce63ed,0,...) at thr_new+0x55 syscallenter(c87df2d0,e990dce4,e990dce4,0,ec,...) at syscallenter+0x263 syscall(e990dd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (455, FreeBSD ELF32, thr_new), eip = 0x28104e5b, esp = 0xbfbfe4fc, ebp = 0xbfbfe5b8 --- Tracing command thr2 pid 1765 tid 101186 td 0xc881a5a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1765 tid 101180 td 0xc86d35a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1765 tid 101167 td 0xc8651000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1765 tid 101164 td 0xc892a870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1765 tid 101155 td 0xc86d3b40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1765 tid 101152 td 0xc8a1b000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1765 tid 101145 td 0xc86c5000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1765 tid 101136 td 0xc87a3b40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1765 tid 101133 td 0xc88f95a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1765 tid 101124 td 0xc88695a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1765 tid 101117 td 0xc8524b40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1765 tid 101108 td 0xc8929870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1765 tid 101103 td 0xc887b2d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1765 tid 101095 td 0xc89d12d0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1765 tid 101090 td 0xc87dfb40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1765 tid 101080 td 0xc8807000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1765 tid 101075 td 0xc8622b40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1765 tid 101059 td 0xc8a38000 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1765 tid 101052 td 0xc86c4870 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1765 tid 101040 td 0xc8a205a0 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1765 tid 101032 td 0xc87edb40 fork_trampoline() at fork_trampoline Tracing command thr2 pid 1765 tid 100169 td 0xc867ab40 sched_switch(c867ab40,0,602,18c,c0641817,...) at sched_switch+0x3bc mi_switch(602,0,c0ce1a16,cd,c867ab40,...) at mi_switch+0x200 critical_exit(c1015320,0,0,c0f81540,c0f81540,...) at critical_exit+0xa8 lapic_handle_timer(e98d79e4) at lapic_handle_timer+0x95 Xtimerint() at Xtimerint+0x20 --- interrupt, eip = 0xc0903a79, esp = 0xe98d7a24, ebp = 0xe98d7a58 --- witness_unlock(c0e45d84,8,c0ce6c8b,12e,c879a280,...) at witness_unlock+0x299 _mtx_unlock_flags(c0e45d84,0,c0ce6c8b,12e,c879a0d0,...) at _mtx_unlock_flags+0xc3 clean_unrhdrl(c6d74080,0,c0ce6c8b,36f,76,...) at clean_unrhdrl+0xbd free_unr(c6d74080,18a1e,ea12d000,e98d7b20,c88175a0) at free_unr+0x36d thread_dtor(c88175a0,2cc,0,2,ea12b000,...) at thread_dtor+0x17f uma_zfree_arg(c1598a80,c88175a0,0) at uma_zfree_arg+0x29 thread_free(c88175a0,0,c0ce23bf,11b,e98d7c14,...) at thread_free+0x52 thread_reap(4,c0cdfaea,109,e98d7c14,e98d7bcc,...) at thread_reap+0xc0 thread_alloc(0,0,c0ce2367,101,280a7380,...) at thread_alloc+0xc create_thread(284731c0,b1f26000,100000,281ea890,284731c0,284731c0,2,0,c0bfea07) at create_thread+0x9c kern_thr_new(c867ab40,e98d7c14,34,280a7380,284731c0,...) at kern_thr_new+0x77 thr_new(c867ab40,e98d7cec,e98d7d28,c0ce63ed,0,...) at thr_new+0x55 syscallenter(c867ab40,e98d7ce4,e98d7ce4,0,ec,...) at syscallenter+0x263 syscall(e98d7d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (455, FreeBSD ELF32, thr_new), eip = 0x28104e5b, esp = 0xbfbfe4fc, ebp = 0xbfbfe5b8 --- Tracing command thr2 pid 1764 tid 100158 td 0xc8503000 sched_switch(c8503000,0,103,18c,c0abf8cd,...) at sched_switch+0x3bc mi_switch(103,0,c0ce64b7,2e2,c86db080,...) at mi_switch+0x200 turnstile_wait(c86db080,c867ab40,0,1ac,c0e45d84,...) at turnstile_wait+0x495 _mtx_lock_sleep(c0e45d84,c8503000,0,c0ce6c8b,36f,...) at _mtx_lock_sleep+0x14d _mtx_lock_flags(c0e45d84,0,c0ce6c8b,36f,c84e2d48,...) at _mtx_lock_flags+0xf7 free_unr(c6d74080,18adc,ea38e000,e98a9b20,c88082d0) at free_unr+0x8c thread_dtor(c88082d0,2cc,0,2,ea38c000,...) at thread_dtor+0x17f uma_zfree_arg(c1598a80,c88082d0,0) at uma_zfree_arg+0x29 thread_free(c88082d0,0,c0ce23bf,11b,e98a9c14,...) at thread_free+0x52 thread_reap(c85030ac,c0f81e30,c0ddbf40,e98a9c14,e98a9bcc,...) at thread_reap+0xc0 thread_alloc(0,e98a9bc0,c08c2d80,c7ffa30c,280a7380,...) at thread_alloc+0xc create_thread(28404380,bf8ff000,100000,281fc190,28404380,28404380,2,0,246) at create_thread+0x9c kern_thr_new(c8503000,e98a9c14,34,280a7380,28404380,...) at kern_thr_new+0x77 thr_new(c8503000,e98a9cec,e98a9d28,c0ce63ed,0,...) at thr_new+0x55 syscallenter(c8503000,e98a9ce4,e98a9ce4,0,ec,...) at syscallenter+0x263 syscall(e98a9d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (455, FreeBSD ELF32, thr_new), eip = 0x28104e5b, esp = 0xbfbfe4fc, ebp = 0xbfbfe5b8 --- Tracing command thr2 pid 1763 tid 100180 td 0xc886a870 sched_switch(c886a870,0,103,18c,c0ff3ecb,...) at sched_switch+0x3bc mi_switch(103,0,c0ce64b7,2e2,c86db080,...) at mi_switch+0x200 turnstile_wait(c86db080,c867ab40,0,1ac,c0e45d84,...) at turnstile_wait+0x495 _mtx_lock_sleep(c0e45d84,c886a870,0,c0ce6c8b,36f,...) at _mtx_lock_sleep+0x14d _mtx_lock_flags(c0e45d84,0,c0ce6c8b,36f,c82e87f8,...) at _mtx_lock_flags+0xf7 free_unr(c6d74080,18db4,eac0d000,e98ecb20,c8521870) at free_unr+0x8c thread_dtor(c8521870,2cc,0,2,eac0b000,...) at thread_dtor+0x17f uma_zfree_arg(c1598a80,c8521870,0) at uma_zfree_arg+0x29 thread_free(c8521870,0,c0ce23bf,11b,e98ecc14,...) at thread_free+0x52 thread_reap(c886a91c,c0f7c528,c0ddbf40,e98ecc14,e98ecbcc,...) at thread_reap+0xc0 thread_alloc(0,e98ecbc0,c08c2d80,c7ff9a6c,280a7380,...) at thread_alloc+0xc create_thread(28404380,bf8ff000,100000,2808e490,28404380,28404380,2,0,246) at create_thread+0x9c kern_thr_new(c886a870,e98ecc14,34,280a7380,28404380,...) at kern_thr_new+0x77 thr_new(c886a870,e98eccec,e98ecd28,c0ce63ed,0,...) at thr_new+0x55 syscallenter(c886a870,e98ecce4,e98ecce4,0,ec,...) at syscallenter+0x263 syscall(e98ecd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (455, FreeBSD ELF32, thr_new), eip = 0x28104e5b, esp = 0xbfbfe4fc, ebp = 0xbfbfe5b8 --- Tracing command thr2 pid 1762 tid 100131 td 0xc81d1000 sched_switch(c81d1000,0,104,191,ce0e0013,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c81d1000,0,c0ce5bb4,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce5bb4,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c81cc550,5c,c0ce8513,100,0,...) at sleepq_wait_sig+0x17 _sleep(c81cc550,c81cc5d8,15c,c0ce8513,0,...) at _sleep+0x35c kern_wait(c81d1000,6e3,e984bc44,0,0,...) at kern_wait+0xb66 wait4(c81d1000,e984bcec,281c472c,1,0,...) at wait4+0x3b syscallenter(c81d1000,e984bce4,c0c016ad,c0e455b0,0,...) at syscallenter+0x263 syscall(e984bd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x2811cbeb, esp = 0xbfbfe9cc, ebp = 0xbfbfe9e8 --- Tracing command thr2 pid 1761 tid 100170 td 0xc867c5a0 sched_switch(c867c5a0,0,104,191,83c6a844,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c867c5a0,0,c0ce5bb4,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(3e9,c08f8560,c867c5a0,3,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c0e45e44,5c,c0ce2632,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c0e45e44,0,15c,c0ce2632,3e9,...) at _sleep+0x328 kern_nanosleep(c867c5a0,e98cec34,e98cec3c,1,0,...) at kern_nanosleep+0xc1 nanosleep(c867c5a0,e98cecec,e98ced28,c0ce63ed,0,...) at nanosleep+0x6f syscallenter(c867c5a0,e98cece4,e98cece4,0,0,...) at syscallenter+0x263 syscall(e98ced28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x281921a3, esp = 0xbfbfe9ec, ebp = 0xbfbfea18 --- Tracing command awk pid 1508 tid 100120 td 0xc7cdb000 sched_switch(c7cdb000,0,104,191,bd1cb7f7,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,4c,...) at mi_switch+0x200 sleepq_switch(c7cdb000,0,c0ce5bb4,1a7,4c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce5bb4,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c80e2498,4c,c0ce812a,100,0,...) at sleepq_wait_sig+0x17 _sleep(c80e2498,c80e2608,14c,c0ce812a,0,...) at _sleep+0x35c pipe_read(c7c5cd58,e9828c28,c6d7cc00,0,c7cdb000,...) at pipe_read+0x417 dofileread(e9828c28,ffffffff,ffffffff,0,c7c5cd58,...) at dofileread+0x9e kern_readv(c7cdb000,0,e9828c28,e9828c48,1,...) at kern_readv+0x58 read(c7cdb000,e9828cec,c6b10c80,e9828c80,0,...) at read+0x4f syscallenter(c7cdb000,e9828ce4,0,c0d1f3bc,ec,...) at syscallenter+0x263 syscall(e9828d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x281b7e53, esp = 0xbfbfe95c, ebp = 0xbfbfe978 --- Tracing command sh pid 1507 tid 100114 td 0xc7c55000 sched_switch(c7c55000,0,104,191,bd5f7f30,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c7c55000,0,c0ce5bb4,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce5bb4,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7c53000,5c,c0ce8513,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7c53000,c7c53088,15c,c0ce8513,0,...) at _sleep+0x35c kern_wait(c7c55000,ffffffff,e97f2c44,2,0,...) at kern_wait+0xb66 wait4(c7c55000,e97f2cec,e97f2d28,c0ce63ed,0,...) at wait4+0x3b syscallenter(c7c55000,e97f2ce4,e97f2ce4,0,0,...) at syscallenter+0x263 syscall(e97f2d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x28162beb, esp = 0xbfbfe97c, ebp = 0xbfbfe998 --- Tracing command tail pid 1506 tid 100113 td 0xc7c552d0 sched_switch(c7c552d0,0,104,191,83e3f545,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,58,...) at mi_switch+0x200 sleepq_switch(c7c552d0,0,c0ce5bb4,1a7,58,...) at sleepq_switch+0x15f sleepq_catch_signals(3e9,c08f8560,c7c552d0,1,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c7be3880,58,c0cdc181,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c7be3880,c7be3880,158,c0cdc181,3e9,...) at _sleep+0x328 kern_kevent(c7c552d0,4,0,1,e97eec28,...) at kern_kevent+0x364 kevent(c7c552d0,e97eecec,e97eed28,c0ce63ed,0,...) at kevent+0x19e syscallenter(c7c552d0,e97eece4,e97eece4,0,0,...) at syscallenter+0x263 syscall(e97eed28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (363, FreeBSD ELF32, kevent), eip = 0x2816930b, esp = 0xbfbfeb3c, ebp = 0xbfbfebf8 --- Tracing command sshd pid 1505 tid 100159 td 0xc8527b40 sched_switch(c8527b40,0,104,191,5b76ca29,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c8527b40,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08ab23a,c7c15990,0,c0cdfaea,c8527b40,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7c159a4,0,e98aca4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7c159a4,c7c15990,c0ce7f79,627,c7cfe0e0,...) at _cv_wait_sig+0x243 seltdwait(c7cfe0e0,58,c7cf0e80,c8527b40,246,...) at seltdwait+0xa2 kern_select(c8527b40,c,288060b8,288060dc,0,0,20,c84e22a8,c84e22a8) at kern_select+0x504 select(c8527b40,e98accec,e98acd28,c0ce63ed,0,...) at select+0x66 syscallenter(c8527b40,e98acce4,e98acce4,0,0,...) at syscallenter+0x263 syscall(e98acd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x284afdd3, esp = 0xbfbfe1bc, ebp = 0xbfbfe208 --- Tracing command top pid 1504 tid 100085 td 0xc7c8d870 sched_switch(c7c8d870,0,104,191,79545905,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7c8d870,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(e974ea14,c08ab23a,c7cf1790,0,c7c8d870,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c7cf17a4,0,e974ea4c,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c7cf17a4,c7cf1790,3e9,627,c7c5b498,...) at _cv_timedwait_sig+0x252 seltdwait(e974ebf8,e974ec00,c7c14780,c7c8d870,3b37315b,...) at seltdwait+0x8a kern_select(c7c8d870,2,bfbfebcc,0,0,e974ec40,20,1,0) at kern_select+0x504 select(c7c8d870,e974ecec,e974ed28,c0ce63ed,0,...) at select+0x66 syscallenter(c7c8d870,e974ece4,e974ece4,0,ec,...) at syscallenter+0x263 syscall(e974ed28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x281eddd3, esp = 0xbfbfeb6c, ebp = 0xbfbfecc8 --- Tracing command sshd pid 1503 tid 100160 td 0xc8544870 sched_switch(c8544870,0,104,191,7958bfac,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c8544870,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08ab23a,c7cf15d0,0,c0cdfaea,c8544870,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7cf15e4,0,e98afa4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7cf15e4,c7cf15d0,c0ce7f79,627,c82d92d8,...) at _cv_wait_sig+0x243 seltdwait(c82d92d8,58,c6d7b900,c8544870,246,...) at seltdwait+0xa2 kern_select(c8544870,a,288060b8,288060dc,0,0,20,c84e2550,c84e2550) at kern_select+0x504 select(c8544870,e98afcec,e98afd28,c0ce63ed,0,...) at select+0x66 syscallenter(c8544870,e98afce4,e98afce4,0,ec,...) at syscallenter+0x263 syscall(e98afd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x284afdd3, esp = 0xbfbfe1bc, ebp = 0xbfbfe208 --- Tracing command sh pid 1502 tid 100162 td 0xc855e5a0 sched_switch(c855e5a0,0,104,191,5cd9358d,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c855e5a0,0,c0ce5bb4,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce5bb4,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c84e27f8,5c,c0ce8513,100,0,...) at sleepq_wait_sig+0x17 _sleep(c84e27f8,c84e2880,15c,c0ce8513,0,...) at _sleep+0x35c kern_wait(c855e5a0,ffffffff,e98b6c44,2,0,...) at kern_wait+0xb66 wait4(c855e5a0,e98b6cec,2820a72c,1,0,...) at wait4+0x3b syscallenter(c855e5a0,e98b6ce4,c0c016ad,c0e455b0,0,...) at syscallenter+0x263 syscall(e98b6d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x28162beb, esp = 0xbfbfeb5c, ebp = 0xbfbfeb78 --- Tracing command sshd pid 1501 tid 100119 td 0xc7cdb2d0 sched_switch(c7cdb2d0,0,104,191,547553dc,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7cdb2d0,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08ab23a,c7cf1290,0,c0cdfaea,c7cdb2d0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7cf12a4,0,e9824a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7cf12a4,c7cf1290,c0ce7f79,627,c7c5b2d8,...) at _cv_wait_sig+0x243 seltdwait(c7c5b2d8,58,c7c14900,c7cdb2d0,c0d0e6bb,...) at seltdwait+0xa2 kern_select(c7cdb2d0,c,288060b8,288060dc,0,0,20,c7cd3550,c7cd3550) at kern_select+0x504 select(c7cdb2d0,e9824cec,e9824d28,c0ce63ed,0,...) at select+0x66 syscallenter(c7cdb2d0,e9824ce4,e9824ce4,0,0,...) at syscallenter+0x263 syscall(e9824d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x284afdd3, esp = 0xbfbfe1bc, ebp = 0xbfbfe208 --- Tracing command sshd pid 1499 tid 100111 td 0xc7c55870 sched_switch(c7c55870,0,104,191,5867126a,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,58,...) at mi_switch+0x200 sleepq_switch(c7c55870,0,c0ce5bb4,1a7,58,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce5bb4,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7ff4bfc,58,c0ceb8ac,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7ff4bfc,c7ff4bbc,158,c0ceb8ac,0) at _sleep+0x35c sbwait(c7ff4b98,4,c0ceb97b,5c9,c7ff4bbc,...) at sbwait+0x76 soreceive_generic(c7ff4b44,0,e97e6c28,0,0,...) at soreceive_generic+0x3f0 soreceive(c7ff4b44,0,e97e6c28,0,0,0) at soreceive+0x38 soo_read(c7d06d20,e97e6c28,c8575400,0,c7c55870,...) at soo_read+0x4e dofileread(e97e6c28,ffffffff,ffffffff,0,c7d06d20,...) at dofileread+0x9e kern_readv(c7c55870,5,e97e6c28,e97e6c48,1,...) at kern_readv+0x58 read(c7c55870,e97e6cec,e97e6d28,c0ce63ed,0,...) at read+0x4f syscallenter(c7c55870,e97e6ce4,e97e6ce4,0,0,...) at syscallenter+0x263 syscall(e97e6d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x284afe53, esp = 0xbfbfdd8c, ebp = 0xbfbfddc8 --- Tracing command sshd pid 1496 tid 100103 td 0xc7cc6b40 sched_switch(c7cc6b40,0,104,191,55759dea,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,58,...) at mi_switch+0x200 sleepq_switch(c7cc6b40,0,c0ce5bb4,1a7,58,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce5bb4,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c82b8728,58,c0ceb8ac,100,0,...) at sleepq_wait_sig+0x17 _sleep(c82b8728,c82b86e8,158,c0ceb8ac,0) at _sleep+0x35c sbwait(c82b86c4,4,c0ceb97b,5c9,c82b86e8,...) at sbwait+0x76 soreceive_generic(c82b8670,0,e97a2c28,0,0,...) at soreceive_generic+0x3f0 soreceive(c82b8670,0,e97a2c28,0,0,0) at soreceive+0x38 soo_read(c7c7c3b8,e97a2c28,c6d7cd80,0,c7cc6b40,...) at soo_read+0x4e dofileread(e97a2c28,ffffffff,ffffffff,0,c7c7c3b8,...) at dofileread+0x9e kern_readv(c7cc6b40,5,e97a2c28,e97a2c48,1,...) at kern_readv+0x58 read(c7cc6b40,e97a2cec,288d8010,1,0,...) at read+0x4f syscallenter(c7cc6b40,e97a2ce4,c0c016ad,c0e455b0,0,...) at syscallenter+0x263 syscall(e97a2d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x284afe53, esp = 0xbfbfdd8c, ebp = 0xbfbfddc8 --- Tracing command sshd pid 1495 tid 100132 td 0xc81d0b40 sched_switch(c81d0b40,0,104,191,543de456,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,58,...) at mi_switch+0x200 sleepq_switch(c81d0b40,0,c0ce5bb4,1a7,58,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce5bb4,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c80ce8c4,58,c0ceb8ac,100,0,...) at sleepq_wait_sig+0x17 _sleep(c80ce8c4,c80ce884,158,c0ceb8ac,0) at _sleep+0x35c sbwait(c80ce860,4,c0ceb97b,5c9,c80ce884,...) at sbwait+0x76 soreceive_generic(c80ce80c,0,e984ec28,0,0,...) at soreceive_generic+0x3f0 soreceive(c80ce80c,0,e984ec28,0,0,0) at soreceive+0x38 soo_read(c7cfe9a0,e984ec28,c6d7cb80,0,c81d0b40,...) at soo_read+0x4e dofileread(e984ec28,ffffffff,ffffffff,0,c7cfe9a0,...) at dofileread+0x9e kern_readv(c81d0b40,5,e984ec28,e984ec48,1,...) at kern_readv+0x58 read(c81d0b40,e984ecec,e984ed28,c0ce63ed,0,...) at read+0x4f syscallenter(c81d0b40,e984ece4,e984ece4,0,0,...) at syscallenter+0x263 syscall(e984ed28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x284afe53, esp = 0xbfbfdd8c, ebp = 0xbfbfddc8 --- Tracing command getty pid 1491 tid 100110 td 0xc7c55b40 sched_switch(c7c55b40,0,104,191,8354913d,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7c55b40,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce5bb4,160,0,c7c55b40,c7c55b40,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6ec3a70,0,c0cea2ee,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6ec3a70,c0e455b0,c0ce9c8c,514,0,...) at _cv_wait_sig+0x243 tty_wait(c6ec3a00,c6ec3a70,e97e2c28,1,0,...) at tty_wait+0x71 ttydisc_read(c6ec3a00,e97e2c28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c6fc2800,e97e2c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7cfe348,e97e2c28,c6d7be80,0,c7c55b40,...) at devfs_read_f+0x7e dofileread(e97e2c28,ffffffff,ffffffff,0,c7cfe348,...) at dofileread+0x9e kern_readv(c7c55b40,0,e97e2c28,e97e2c48,1,...) at kern_readv+0x58 read(c7c55b40,e97e2cec,e97e2d28,c0ce63ed,0,...) at read+0x4f syscallenter(c7c55b40,e97e2ce4,e97e2ce4,0,0,...) at syscallenter+0x263 syscall(e97e2d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28195e53, esp = 0xbfbfedac, ebp = 0xbfbfedd8 --- Tracing command getty pid 1490 tid 100094 td 0xc7cc8870 sched_switch(c7cc8870,0,104,191,835af250,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7cc8870,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce5bb4,160,0,c7cc8870,c7cc8870,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6ec3c70,0,c0cea2ee,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6ec3c70,c0e455b0,c0ce9c8c,514,0,...) at _cv_wait_sig+0x243 tty_wait(c6ec3c00,c6ec3c70,e9787c28,1,0,...) at tty_wait+0x71 ttydisc_read(c6ec3c00,e9787c28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c6fc2900,e9787c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7c7cdc8,e9787c28,c6d7be80,0,c7cc8870,...) at devfs_read_f+0x7e dofileread(e9787c28,ffffffff,ffffffff,0,c7c7cdc8,...) at dofileread+0x9e kern_readv(c7cc8870,0,e9787c28,e9787c48,1,...) at kern_readv+0x58 read(c7cc8870,e9787cec,e9787d28,c0ce63ed,0,...) at read+0x4f syscallenter(c7cc8870,e9787ce4,e9787ce4,0,0,...) at syscallenter+0x263 syscall(e9787d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28195e53, esp = 0xbfbfedac, ebp = 0xbfbfedd8 --- Tracing command getty pid 1489 tid 100126 td 0xc81d2000 sched_switch(c81d2000,0,104,191,834e8eee,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c81d2000,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce5bb4,160,0,c81d2000,c81d2000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6ec3e70,0,c0cea2ee,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6ec3e70,c0e455b0,c0ce9c8c,514,0,...) at _cv_wait_sig+0x243 tty_wait(c6ec3e00,c6ec3e70,e983cc28,1,0,...) at tty_wait+0x71 ttydisc_read(c6ec3e00,e983cc28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c6fc2a00,e983cc28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7c5cdc8,e983cc28,c6d7be80,0,c81d2000,...) at devfs_read_f+0x7e dofileread(e983cc28,ffffffff,ffffffff,0,c7c5cdc8,...) at dofileread+0x9e kern_readv(c81d2000,0,e983cc28,e983cc48,1,...) at kern_readv+0x58 read(c81d2000,e983ccec,e983cd28,c0ce63ed,0,...) at read+0x4f syscallenter(c81d2000,e983cce4,e983cce4,0,0,...) at syscallenter+0x263 syscall(e983cd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28195e53, esp = 0xbfbfedac, ebp = 0xbfbfedd8 --- Tracing command getty pid 1488 tid 100127 td 0xc81d1b40 sched_switch(c81d1b40,0,104,191,833835e6,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c81d1b40,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce5bb4,160,0,c81d1b40,c81d1b40,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7170070,0,c0cea2ee,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7170070,c0e455b0,c0ce9c8c,514,0,...) at _cv_wait_sig+0x243 tty_wait(c7170000,c7170070,e983fc28,1,0,...) at tty_wait+0x71 ttydisc_read(c7170000,e983fc28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c6fc2b00,e983fc28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7c7c770,e983fc28,c6d7be80,0,c81d1b40,...) at devfs_read_f+0x7e dofileread(e983fc28,ffffffff,ffffffff,0,c7c7c770,...) at dofileread+0x9e kern_readv(c81d1b40,0,e983fc28,e983fc48,1,...) at kern_readv+0x58 read(c81d1b40,e983fcec,e983fd28,c0ce63ed,0,...) at read+0x4f syscallenter(c81d1b40,e983fce4,e983fce4,0,0,...) at syscallenter+0x263 syscall(e983fd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28195e53, esp = 0xbfbfedac, ebp = 0xbfbfedd8 --- Tracing command getty pid 1487 tid 100098 td 0xc7cc7b40 sched_switch(c7cc7b40,0,104,191,82eacd52,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7cc7b40,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce5bb4,160,0,c7cc7b40,c7cc7b40,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7170270,0,c0cea2ee,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7170270,c0e455b0,c0ce9c8c,514,0,...) at _cv_wait_sig+0x243 tty_wait(c7170200,c7170270,e9793c28,1,0,...) at tty_wait+0x71 ttydisc_read(c7170200,e9793c28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c6fc2c00,e9793c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7c7c3f0,e9793c28,c6d7be80,0,c7cc7b40,...) at devfs_read_f+0x7e dofileread(e9793c28,ffffffff,ffffffff,0,c7c7c3f0,...) at dofileread+0x9e kern_readv(c7cc7b40,0,e9793c28,e9793c48,1,...) at kern_readv+0x58 read(c7cc7b40,e9793cec,e9793d28,c0ce63ed,0,...) at read+0x4f syscallenter(c7cc7b40,e9793ce4,e9793ce4,0,0,...) at syscallenter+0x263 syscall(e9793d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28195e53, esp = 0xbfbfedac, ebp = 0xbfbfedd8 --- Tracing command getty pid 1486 tid 100104 td 0xc7cc6870 sched_switch(c7cc6870,0,104,191,8378daa7,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7cc6870,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce5bb4,160,0,c7cc6870,c7cc6870,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7170470,0,c0cea2ee,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7170470,c0e455b0,c0ce9c8c,514,0,...) at _cv_wait_sig+0x243 tty_wait(c7170400,c7170470,e97a5c28,1,0,...) at tty_wait+0x71 ttydisc_read(c7170400,e97a5c28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c6fc2d00,e97a5c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7d06b98,e97a5c28,c6d7be80,0,c7cc6870,...) at devfs_read_f+0x7e dofileread(e97a5c28,ffffffff,ffffffff,0,c7d06b98,...) at dofileread+0x9e kern_readv(c7cc6870,0,e97a5c28,e97a5c48,1,...) at kern_readv+0x58 read(c7cc6870,e97a5cec,e97a5d28,c0ce63ed,0,...) at read+0x4f syscallenter(c7cc6870,e97a5ce4,e97a5ce4,0,0,...) at syscallenter+0x263 syscall(e97a5d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28195e53, esp = 0xbfbfedac, ebp = 0xbfbfedd8 --- Tracing command getty pid 1485 tid 100146 td 0xc82ed000 sched_switch(c82ed000,0,104,191,83445d28,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c82ed000,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce5bb4,160,0,c82ed000,c82ed000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6ec2670,0,c0cea2ee,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6ec2670,c0e455b0,c0ce9c8c,514,0,...) at _cv_wait_sig+0x243 tty_wait(c6ec2600,c6ec2670,e9880c28,1,0,...) at tty_wait+0x71 ttydisc_read(c6ec2600,e9880c28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c6fc2e00,e9880c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7c5c3f0,e9880c28,c6d7be80,0,c82ed000,...) at devfs_read_f+0x7e dofileread(e9880c28,ffffffff,ffffffff,0,c7c5c3f0,...) at dofileread+0x9e kern_readv(c82ed000,0,e9880c28,e9880c48,1,...) at kern_readv+0x58 read(c82ed000,e9880cec,e9880d28,c0ce63ed,0,...) at read+0x4f syscallenter(c82ed000,e9880ce4,e9880ce4,0,0,...) at syscallenter+0x263 syscall(e9880d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28195e53, esp = 0xbfbfedac, ebp = 0xbfbfedd8 --- Tracing command getty pid 1484 tid 100096 td 0xc7cc82d0 sched_switch(c7cc82d0,0,104,191,831b9fce,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7cc82d0,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce5bb4,160,0,c7cc82d0,c7cc82d0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6ec2070,0,c0cea2ee,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6ec2070,c0e455b0,c0ce9c8c,514,0,...) at _cv_wait_sig+0x243 tty_wait(c6ec2000,c6ec2070,e978dc28,1,0,...) at tty_wait+0x71 ttydisc_read(c6ec2000,e978dc28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c715e000,e978dc28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c82d91f8,e978dc28,c6d7be80,0,c7cc82d0,...) at devfs_read_f+0x7e dofileread(e978dc28,ffffffff,ffffffff,0,c82d91f8,...) at dofileread+0x9e kern_readv(c7cc82d0,0,e978dc28,e978dc48,1,...) at kern_readv+0x58 read(c7cc82d0,e978dcec,e978dd28,c0ce63ed,0,...) at read+0x4f syscallenter(c7cc82d0,e978dce4,e978dce4,0,0,...) at syscallenter+0x263 syscall(e978dd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28195e53, esp = 0xbfbfedac, ebp = 0xbfbfedd8 --- Tracing command getty pid 1483 tid 100143 td 0xc82ed870 sched_switch(c82ed870,0,104,191,8372112d,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c82ed870,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08ab23a,c6ec2204,0,c0cdfaea,c82ed870,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6ec2270,0,e9877ad8,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6ec2270,c6ec2204,c0ce9c8c,514,0,...) at _cv_wait_sig+0x243 tty_wait(c6ec2200,c6ec2270,e9877c28,1,0,...) at tty_wait+0x71 ttydisc_read(c6ec2200,e9877c28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c6de2c00,e9877c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7cfe188,e9877c28,c6d7be80,0,c82ed870,...) at devfs_read_f+0x7e dofileread(e9877c28,ffffffff,ffffffff,0,c7cfe188,...) at dofileread+0x9e kern_readv(c82ed870,0,e9877c28,e9877c48,1,...) at kern_readv+0x58 read(c82ed870,e9877cec,e9877d28,c0ce63ed,0,...) at read+0x4f syscallenter(c82ed870,e9877ce4,e9877ce4,0,0,...) at syscallenter+0x263 syscall(e9877d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28195e53, esp = 0xbfbfedac, ebp = 0xbfbfedd8 --- Tracing command inetd pid 1452 tid 100097 td 0xc7cc8000 sched_switch(c7cc8000,0,104,191,76bf9dad,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7cc8000,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08ab23a,c7bc39d0,0,c0cdfaea,c7cc8000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7bc39e4,0,e9790a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7bc39e4,c7bc39d0,c0ce7f79,627,c7cfe0a8,...) at _cv_wait_sig+0x243 seltdwait(c7cfe0a8,58,c6d7be80,c7cc8000,e9790ac0,...) at seltdwait+0xa2 kern_select(c7cc8000,6,bfbfdfc0,0,0,0,20,c7fa6000,c7fa6000) at kern_select+0x504 select(c7cc8000,e9790cec,e9790d28,c0ce63ed,0,...) at select+0x66 syscallenter(c7cc8000,e9790ce4,e9790ce4,0,0,...) at syscallenter+0x263 syscall(e9790d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x281a9dd3, esp = 0xbfbfdf7c, ebp = 0xbfbfee08 --- Tracing command moused pid 1426 tid 100122 td 0xc7cc6000 sched_switch(c7cc6000,0,104,191,69f37ec1,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7cc6000,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08ab23a,c7cf17d0,0,c0cdfaea,c7cc6000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7cf17e4,0,e9830a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7cf17e4,c7cf17d0,c0ce7f79,627,c82d9038,...) at _cv_wait_sig+0x243 seltdwait(c82d9038,58,c6d7be80,c7cc6000,0,...) at seltdwait+0xa2 kern_select(c7cc6000,6,bfbfea24,0,0,0,20,c81cdd48,c81cdd48) at kern_select+0x504 select(c7cc6000,e9830cec,e9830d28,c0ce63ed,0,...) at select+0x66 syscallenter(c7cc6000,e9830ce4,e9830ce4,0,0,...) at syscallenter+0x263 syscall(e9830d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x281b2dd3, esp = 0xbfbfe9bc, ebp = 0xbfbfeb48 --- Tracing command bash pid 1398 tid 100156 td 0xc82e9000 sched_switch(c82e9000,0,104,191,cd0ba967,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c82e9000,0,c0ce5bb4,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce5bb4,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c82e5000,5c,c0ce8513,100,0,...) at sleepq_wait_sig+0x17 _sleep(c82e5000,c82e5088,15c,c0ce8513,0,...) at _sleep+0x35c kern_wait(c82e9000,ffffffff,e989ec44,6,0,...) at kern_wait+0xb66 wait4(c82e9000,e989ecec,e989ed28,c0ce63ed,0,...) at wait4+0x3b syscallenter(c82e9000,e989ece4,e989ece4,0,0,...) at syscallenter+0x263 syscall(e989ed28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x282c2beb, esp = 0xbfbfe7ac, ebp = 0xbfbfe7c8 --- Tracing command sshd pid 1397 tid 100125 td 0xc81d22d0 sched_switch(c81d22d0,0,104,191,cd6add68,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c81d22d0,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08ab23a,c7cf1490,0,c0cdfaea,c81d22d0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7cf14a4,0,e9839a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7cf14a4,c7cf1490,c0ce7f79,627,c7c5c0e0,...) at _cv_wait_sig+0x243 seltdwait(c7c5c0e0,58,c7c14880,c81d22d0,246,...) at seltdwait+0xa2 kern_select(c81d22d0,a,288060b8,288060dc,0,0,20,c81cd550,c81cd550) at kern_select+0x504 select(c81d22d0,e9839cec,e9839d28,c0ce63ed,0,...) at select+0x66 syscallenter(c81d22d0,e9839ce4,e9839ce4,0,0,...) at syscallenter+0x263 syscall(e9839d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x284afdd3, esp = 0xbfbfe1bc, ebp = 0xbfbfe208 --- Tracing command sshd pid 1395 tid 100148 td 0xc82eb870 sched_switch(c82eb870,0,104,191,f634745,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,58,...) at mi_switch+0x200 sleepq_switch(c82eb870,0,c0ce5bb4,1a7,58,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce5bb4,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c82aca60,58,c0ceb8ac,100,0,...) at sleepq_wait_sig+0x17 _sleep(c82aca60,c82aca20,158,c0ceb8ac,0) at _sleep+0x35c sbwait(c82ac9fc,4,c0ceb97b,5c9,c82aca20,...) at sbwait+0x76 soreceive_generic(c82ac9a8,0,e9886c28,0,0,...) at soreceive_generic+0x3f0 soreceive(c82ac9a8,0,e9886c28,0,0,0) at soreceive+0x38 soo_read(c7d06ce8,e9886c28,c6d7c780,0,c82eb870,...) at soo_read+0x4e dofileread(e9886c28,ffffffff,ffffffff,0,c7d06ce8,...) at dofileread+0x9e kern_readv(c82eb870,5,e9886c28,e9886c48,1,...) at kern_readv+0x58 read(c82eb870,e9886cec,e9886d28,c0ce63ed,0,...) at read+0x4f syscallenter(c82eb870,e9886ce4,e9886ce4,0,0,...) at syscallenter+0x263 syscall(e9886d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x284afe53, esp = 0xbfbfdd8c, ebp = 0xbfbfddc8 --- Tracing command cron pid 1372 tid 100150 td 0xc82eb2d0 sched_switch(c82eb2d0,0,104,191,91b41ba0,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c82eb2d0,0,c0ce5bb4,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(ea61,c08f8560,c82eb2d0,2,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c0e45e44,5c,c0ce2632,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c0e45e44,0,15c,c0ce2632,ea61,...) at _sleep+0x328 kern_nanosleep(c82eb2d0,e988cc34,e988cc3c,3c,0,...) at kern_nanosleep+0xc1 nanosleep(c82eb2d0,e988ccec,e988cd28,c0ce63ed,0,...) at nanosleep+0x6f syscallenter(c82eb2d0,e988cce4,e988cce4,0,0,...) at syscallenter+0x263 syscall(e988cd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x281841a3, esp = 0xbfbfec6c, ebp = 0xbfbfec98 --- Tracing command sendmail pid 1365 tid 100152 td 0xc82e9b40 sched_switch(c82e9b40,0,104,191,55c95d36,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,68,...) at mi_switch+0x200 sleepq_switch(c82e9b40,0,c0ce5bb4,1a7,68,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce5bb4,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c82e5af8,68,c0c91bd8,100,0,...) at sleepq_wait_sig+0x17 _sleep(c82e5af8,c82e5b28,168,c0c91bd8,0,...) at _sleep+0x35c kern_sigsuspend(c82e9b40,0,0,0,0,...) at kern_sigsuspend+0xba sigsuspend(c82e9b40,e9892cec,e9892d28,c0ce63ed,0,...) at sigsuspend+0x4d syscallenter(c82e9b40,e9892ce4,e9892ce4,0,246,...) at syscallenter+0x263 syscall(e9892d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x2833783b, esp = 0xbfbfcfbc, ebp = 0xbfbfcfe8 --- Tracing command sendmail pid 1361 tid 100134 td 0xc81d05a0 sched_switch(c81d05a0,0,104,191,673dbd91,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c81d05a0,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(e9854a14,c08ab23a,c72ef5d0,0,c81d05a0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c72ef5e4,0,e9854a4c,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c72ef5e4,c72ef5d0,1389,627,e9854c00,...) at _cv_timedwait_sig+0x252 seltdwait(e9854bf8,e9854c00,0,e9854ad4,246,...) at seltdwait+0x8a kern_select(c81d05a0,0,bfbfc530,0,0,e9854c40,20,5,0) at kern_select+0x504 select(c81d05a0,e9854cec,e9854d28,c0ce63ed,0,...) at select+0x66 syscallenter(c81d05a0,e9854ce4,e9854ce4,0,c0e49f40,...) at syscallenter+0x263 syscall(e9854d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x283dfdd3, esp = 0xbfbfc49c, ebp = 0xbfbfcfc8 --- Tracing command sshd pid 1353 tid 100155 td 0xc82e92d0 sched_switch(c82e92d0,0,104,191,57e559c1,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c82e92d0,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08ab23a,c7c15a90,0,c0cdfaea,c82e92d0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7c15aa4,0,e989ba4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7c15aa4,c7c15a90,c0ce7f79,627,c7d06428,...) at _cv_wait_sig+0x243 seltdwait(c7d06428,58,c6d7b700,c82e92d0,e989bab4,...) at seltdwait+0xa2 kern_select(c82e92d0,5,2880c0b0,0,0,0,20,c82e52a8,c82e52a8) at kern_select+0x504 select(c82e92d0,e989bcec,e989bd28,c0ce63ed,0,...) at select+0x66 syscallenter(c82e92d0,e989bce4,e989bce4,0,0,...) at syscallenter+0x263 syscall(e989bd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x284afdd3, esp = 0xbfbfe27c, ebp = 0xbfbfee18 --- Tracing command ntpd pid 1308 tid 100129 td 0xc81d15a0 sched_switch(c81d15a0,0,104,191,7dc5f1c9,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c81d15a0,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08ab23a,c72ef3d0,0,c0cdfaea,c81d15a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c72ef3e4,0,e9845a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c72ef3e4,c72ef3d0,c0ce7f79,627,c7c5bb98,...) at _cv_wait_sig+0x243 seltdwait(c7c5bb98,58,c6d7be80,c81d15a0,0,...) at seltdwait+0xa2 kern_select(c81d15a0,1c,bfbfed08,0,0,0,20,c81ccaa0,c81ccaa0) at kern_select+0x504 select(c81d15a0,e9845cec,e9845d80,e9845c88,0,...) at select+0x66 syscallenter(c81d15a0,e9845ce4,c0c01366,fffffffe,246,...) at syscallenter+0x263 syscall(e9845d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x28358dd3, esp = 0xbfbfecdc, ebp = 0xbfbfeda8 --- Tracing command nfsd pid 1176 tid 100141 td 0xc7cdc870 sched_switch(c7cdc870,0,104,191,d802279e,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7cdc870,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(e986ebe0,c08ab23a,c6fe5c80,0,c7cdc870,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c7d114d0,0,e986ec18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c7d114d0,c6fe5c80,1388,3b1,0,...) at _cv_timedwait_sig+0x252 svc_run_internal(e986ed14,c08907a8,c6fe5c80,e986ed28,c0cdc537,...) at svc_run_internal+0x356 svc_thread_start(c6fe5c80,e986ed28,c0cdc537,349,c7cd5aa0,...) at svc_thread_start+0x10 fork_exit(c0ab91b0,c6fe5c80,e986ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x804c72e, eip = 0xc, esp = 0x33, ebp = 0 --- Tracing command nfsd pid 1176 tid 100140 td 0xc7cdcb40 sched_switch(c7cdcb40,0,104,191,d95ea40c,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7cdcb40,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(e986bbe0,c08ab23a,c6fe5c80,0,c7cdcb40,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c7d11510,0,e986bc18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c7d11510,c6fe5c80,1388,3b1,0,...) at _cv_timedwait_sig+0x252 svc_run_internal(e986bd14,c08907a8,c6fe5c80,e986bd28,c0cdc537,...) at svc_run_internal+0x356 svc_thread_start(c6fe5c80,e986bd28,c0cdc537,349,c7cd5aa0,...) at svc_thread_start+0x10 fork_exit(c0ab91b0,c6fe5c80,e986bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x804c72e, eip = 0xc, esp = 0x33, ebp = 0 --- Tracing command nfsd pid 1176 tid 100139 td 0xc81d0000 sched_switch(c81d0000,0,104,191,d8020bb7,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c81d0000,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(e9868be0,c08ab23a,c6fe5c80,0,c81d0000,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c7d11550,0,e9868c18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c7d11550,c6fe5c80,1388,3b1,0,...) at _cv_timedwait_sig+0x252 svc_run_internal(e9868d14,c08907a8,c6fe5c80,e9868d28,c0cdc537,...) at svc_run_internal+0x356 svc_thread_start(c6fe5c80,e9868d28,c0cdc537,349,c7cd5aa0,...) at svc_thread_start+0x10 fork_exit(c0ab91b0,c6fe5c80,e9868d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x804c72e, eip = 0xc, esp = 0x33, ebp = 0 --- Tracing command nfsd pid 1176 tid 100105 td 0xc7cc65a0 sched_switch(c7cc65a0,0,104,191,d8024d47,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7cc65a0,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(e97a8ab0,c08ab23a,c6fe5c80,0,c7cc65a0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c6f60950,0,e97a8ae8,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c6f60950,c6fe5c80,1388,3b1,e97a8b30,...) at _cv_timedwait_sig+0x252 svc_run_internal(c7cc6748,14,c0d05cf8,c0d047be,e97a8c0c,...) at svc_run_internal+0x356 svc_run(c6fe5c80,0,c0d04a99,1c2,0,...) at svc_run+0x7b nfssvc_nfsd(bfbfe8a0,e97a8c0c,c,c6d7be80,e97a8c20,...) at nfssvc_nfsd+0xad nfssvc_nfsserver(c7cc65a0,e97a8cec,c7cd5aa0,0,c7cc65a0,...) at nfssvc_nfsserver+0x24f nfssvc(c7cc65a0,e97a8cec,e97a8d28,c0ce63ed,0,...) at nfssvc+0x83 syscallenter(c7cc65a0,e97a8ce4,e97a8ce4,0,0,...) at syscallenter+0x263 syscall(e97a8d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (155, FreeBSD ELF32, nfssvc), eip = 0x280dc43b, esp = 0xbfbfe85c, ebp = 0xbfbfeac8 --- Tracing command nfsd pid 1175 tid 100100 td 0xc7cc75a0 sched_switch(c7cc75a0,0,104,191,baaa483b,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7cc75a0,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08ab23a,c7cf1550,0,c0cdfaea,c7cc75a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7cf1564,0,e9799a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7cf1564,c7cf1550,c0ce7f79,627,c7c5bbd0,...) at _cv_wait_sig+0x243 seltdwait(c7c5bbd0,58,c6d7be80,c7cc75a0,c0d0e6bb,...) at seltdwait+0xa2 kern_select(c7cc75a0,5,bfbfecd0,0,0,0,20,c7fa47f8,c7fa47f8) at kern_select+0x504 select(c7cc75a0,e9799cec,bfbffff4,1,0,...) at select+0x66 syscallenter(c7cc75a0,e9799ce4,c0c016ad,c0e455b0,0,...) at syscallenter+0x263 syscall(e9799d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x28183dd3, esp = 0xbfbfeacc, ebp = 0xbfbfee08 --- Tracing command mountd pid 1166 tid 100128 td 0xc81d1870 sched_switch(c81d1870,0,104,191,5378531e,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c81d1870,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08ab23a,c7cf1c90,0,c0cdfaea,c81d1870,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7cf1ca4,0,e9842a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7cf1ca4,c7cf1c90,c0ce7f79,627,c7c5b8f8,...) at _cv_wait_sig+0x243 seltdwait(c7c5b8f8,58,c6d7be80,c81d1870,c0f85c68,...) at seltdwait+0xa2 kern_select(c81d1870,9,bfbfed5c,0,0,0,20,c81ccd48,c81ccd48) at kern_select+0x504 select(c81d1870,e9842cec,e9842d28,c0ce63ed,0,...) at select+0x66 syscallenter(c81d1870,e9842ce4,e9842ce4,0,0,...) at syscallenter+0x263 syscall(e9842d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x28199dd3, esp = 0xbfbfed2c, ebp = 0xbfbfedf8 --- Tracing command rpcbind pid 1069 tid 100077 td 0xc7c8eb40 sched_switch(c7c8eb40,0,104,191,be60c1e5,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7c8eb40,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(e972ea6c,c08ab23a,c72ef850,0,c7c8eb40,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c72ef864,0,e972eaa4,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c72ef864,c72ef850,7531,627,e972eb5c,...) at _cv_timedwait_sig+0x252 seltdwait(e972ec2c,e972ec34,535,c7c8eb40,e972eb2c,...) at seltdwait+0x8a poll(c7c8eb40,e972ecec,e972ed28,c0ce63ed,0,...) at poll+0x300 syscallenter(c7c8eb40,e972ece4,e972ece4,0,0,...) at syscallenter+0x263 syscall(e972ed28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, poll), eip = 0x2813fcdb, esp = 0xbfbfcbcc, ebp = 0xbfbfed98 --- Tracing command syslogd pid 1036 tid 100135 td 0xc81d02d0 sched_switch(c81d02d0,0,104,191,4eb544fe,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c81d02d0,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08ab23a,c7cf1890,0,c0cdfaea,c81d02d0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7cf18a4,0,e9857a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7cf18a4,c7cf1890,c0ce7f79,627,c7c5c7e0,...) at _cv_wait_sig+0x243 seltdwait(c7c5c7e0,58,c6d7be80,c81d02d0,0,...) at seltdwait+0xa2 kern_select(c81d02d0,9,2847f0ac,0,0,0,20,c7fa6aa0,c7fa6aa0) at kern_select+0x504 select(c81d02d0,e9857cec,e9857d28,c0ce63ed,0,...) at select+0x66 syscallenter(c81d02d0,e9857ce4,e9857ce4,0,246,...) at syscallenter+0x263 syscall(e9857d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x28198dd3, esp = 0xbfbfe2bc, ebp = 0xbfbfee08 --- Tracing command devd pid 835 tid 100115 td 0xc7cdc000 sched_switch(c7cdc000,0,104,191,5535ed89,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7cdc000,0,c0ce5bb4,1a7,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c08ab23a,c7cf1010,0,c0cdfaea,c7cdc000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7cf1024,0,e9814a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7cf1024,c7cf1010,c0ce7f79,627,c7c5bd58,...) at _cv_wait_sig+0x243 seltdwait(c7c5bd58,58,c6d7be80,c7cdc000,3,...) at seltdwait+0xa2 kern_select(c7cdc000,6,bfbfe980,0,0,0,20,c7cd5000,c7cd5000) at kern_select+0x504 select(c7cdc000,e9814cec,e9814d28,c0ce63ed,0,...) at select+0x66 syscallenter(c7cdc000,e9814ce4,e9814ce4,0,0,...) at syscallenter+0x263 syscall(e9814d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x808b6d3, esp = 0xbfbfe94c, ebp = 0xbfbfee28 --- Tracing command flowcleaner pid 21 tid 100075 td 0xc704eb40 sched_switch(c704eb40,0,104,191,984f3661,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c704eb40,0,c0ce5bb4,28b,c704eb40,...) at sleepq_switch+0x15f sleepq_timedwait(c0fb2f08,0,e753acac,1,0,...) at sleepq_timedwait+0x6b _cv_timedwait(c0fb2f08,c0fb2f10,4e20,620,c704eb40,...) at _cv_timedwait+0x252 flowtable_cleaner(0,e753ad28,c0cdc537,349,c7bf4000,...) at flowtable_cleaner+0x1ae fork_exit(c0969260,0,e753ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe753ad60, ebp = 0 --- Tracing command softdepflush pid 20 tid 100074 td 0xc7161000 sched_switch(c7161000,0,104,191,74dda71e,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,44,...) at mi_switch+0x200 sleepq_switch(c7161000,0,c0ce5bb4,28b,0,...) at sleepq_switch+0x15f sleepq_timedwait(c0fbe6e0,44,c0d0a95b,0,0,...) at sleepq_timedwait+0x6b _sleep(c0fbe6e0,c0fbe698,44,c0d0a95b,3e8,...) at _sleep+0x342 softdep_flush(0,e7537d28,c0cdc537,349,c7bf42a8,...) at softdep_flush+0x25a fork_exit(c0b001b0,0,e7537d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7537d60, ebp = 0 --- Tracing command vnlru pid 19 tid 100073 td 0xc71612d0 sched_switch(c71612d0,0,104,191,69d3d51d,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,50,...) at mi_switch+0x200 sleepq_switch(c71612d0,0,c0ce5bb4,28b,0,...) at sleepq_switch+0x15f sleepq_timedwait(c7bf4550,50,c0cefc92,0,0,...) at sleepq_timedwait+0x6b _sleep(c7bf4550,c0fb2cd8,250,c0cefc92,3e8,...) at _sleep+0x342 vnlru_proc(0,e7534d28,c0cdc537,349,c7bf4550,...) at vnlru_proc+0xe7 fork_exit(c0953980,0,e7534d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7534d60, ebp = 0 --- Tracing command syncer pid 18 tid 100072 td 0xc71615a0 sched_switch(c71615a0,0,104,191,6cb544bc,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c71615a0,0,c0ce5bb4,28b,c71615a0,...) at sleepq_switch+0x15f sleepq_timedwait(c0fb2d18,0,e7531c78,1,0,...) at sleepq_timedwait+0x6b _cv_timedwait(c0fb2d18,c0fb2d04,3e8,6e8,4e20,...) at _cv_timedwait+0x252 sched_sync(0,e7531d28,c0cdc537,349,c7bf47f8,...) at sched_sync+0x512 fork_exit(c0952170,0,e7531d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7531d60, ebp = 0 --- Tracing command bufdaemon pid 17 tid 100071 td 0xc7161870 sched_switch(c7161870,0,104,191,696cd207,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,44,...) at mi_switch+0x200 sleepq_switch(c7161870,0,c0ce5bb4,28b,0,...) at sleepq_switch+0x15f sleepq_timedwait(c0fb2a48,44,c0ced163,0,0,...) at sleepq_timedwait+0x6b _sleep(c0fb2a48,c0fb2a50,44,c0ced163,3e8,...) at _sleep+0x342 buf_daemon(0,e752ed28,c0cdc537,349,c7bf4aa0,...) at buf_daemon+0x16e fork_exit(c093a970,0,e752ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe752ed60, ebp = 0 --- Tracing command pagezero pid 16 tid 100070 td 0xc7161b40 sched_switch(c7161b40,0,104,191,3609a90b,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7161b40,0,c0ce5bb4,28b,0,...) at sleepq_switch+0x15f sleepq_timedwait(c0fc055c,0,c0d1115f,0,0,...) at sleepq_timedwait+0x6b _sleep(c0fc055c,c0fbf000,0,c0d1115f,493e0,...) at _sleep+0x342 vm_pagezero(0,e752bd28,c0cdc537,349,c7bf4d48,...) at vm_pagezero+0xdc fork_exit(c0b461c0,0,e752bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe752bd60, ebp = 0 --- Tracing command vmdaemon pid 15 tid 100069 td 0xc7162000 sched_switch(c7162000,0,104,191,3601bf08,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,68,...) at mi_switch+0x200 sleepq_switch(c7162000,0,c0ce5bb4,268,0,...) at sleepq_switch+0x15f sleepq_wait(c0fc0150,68,c0ced163,0,0,...) at sleepq_wait+0x63 _sleep(c0fc0150,c0fc0154,68,c0ced163,0,...) at _sleep+0x372 vm_daemon(0,e7528d28,c0cdc537,349,c6d9a2a8,...) at vm_daemon+0x59 fork_exit(c0b40040,0,e7528d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7528d60, ebp = 0 --- Tracing command pagedaemon pid 9 tid 100068 td 0xc71622d0 sched_switch(c71622d0,0,104,191,96570d64,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,44,...) at mi_switch+0x200 sleepq_switch(c71622d0,0,c0ce5bb4,28b,0,...) at sleepq_switch+0x15f sleepq_timedwait(c0fc0118,44,c0ced163,0,0,...) at sleepq_timedwait+0x6b _sleep(c0fc0118,c0fbf000,44,c0ced163,1388,...) at _sleep+0x342 vm_pageout(0,e7525d28,c0cdc537,349,c6d9a550,...) at vm_pageout+0x2c7 fork_exit(c0b410b0,0,e7525d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7525d60, ebp = 0 --- Tracing command xpt_thrd pid 8 tid 100067 td 0xc71625a0 sched_switch(c71625a0,0,104,191,30d470ba,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,4c,...) at mi_switch+0x200 sleepq_switch(c71625a0,0,c0ce5bb4,268,0,...) at sleepq_switch+0x15f sleepq_wait(c0e11ad4,4c,c0c77fcd,0,0,...) at sleepq_wait+0x63 _sleep(c0e11ad4,c0e11b18,4c,c0c77fcd,0,...) at _sleep+0x372 xpt_scanner_thread(0,e7522d28,c0cdc537,349,c6d9a7f8,...) at xpt_scanner_thread+0x47 fork_exit(c0485c70,0,e7522d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7522d60, ebp = 0 --- Tracing command sctp_iterator pid 7 tid 100066 td 0xc7162870 sched_switch(c7162870,0,104,191,95ee4e64,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7162870,0,c0ce5bb4,268,0,...) at sleepq_switch+0x15f sleepq_wait(c0fb4738,0,c0cfbab0,0,0,...) at sleepq_wait+0x63 _sleep(c0fb4738,c0fb4700,0,c0cfbab0,0,...) at _sleep+0x372 sctp_iterator_thread(0,e751fd28,c0cdc537,349,c6d9aaa0,...) at sctp_iterator_thread+0x5c fork_exit(c09e32a0,0,e751fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe751fd60, ebp = 0 --- Tracing command fdc0 pid 6 tid 100062 td 0xc70245a0 sched_switch(c70245a0,0,104,191,6171c3d8,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,4c,...) at mi_switch+0x200 sleepq_switch(c70245a0,0,c0ce5bb4,28b,0,...) at sleepq_switch+0x15f sleepq_timedwait(c705603c,4c,c0cd6178,0,0,...) at sleepq_timedwait+0x6b _sleep(c705603c,c70560f0,4c,c0cd6178,3e8,...) at _sleep+0x342 fdc_thread(c7056000,e7513d28,c0cdc537,349,c6d9ad48,...) at fdc_thread+0x27d fork_exit(c0bc4d40,c7056000,e7513d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7513d60, ebp = 0 --- Tracing command fw0_probe pid 5 tid 100060 td 0xc7024b40 sched_switch(c7024b40,0,104,191,386d3680,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c7024b40,0,c0ce5bb4,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce5bb4,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7059000,5c,c0cd6178,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7059000,c705d488,15c,c0cd6178,0,...) at _sleep+0x35c fw_bus_probe_thread(c7059000,e7503d28,c0cdc537,349,c6f7b000,...) at fw_bus_probe_thread+0xa08 fork_exit(c067ab20,c7059000,e7503d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7503d60, ebp = 0 --- Tracing command usb pid 14 tid 100056 td 0xc704c870 sched_switch(c704c870,0,104,191,26099eb2,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c704c870,0,c0ce5bb4,268,c704c870,...) at sleepq_switch+0x15f sleepq_wait(c7034d34,0,e74f0cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7034d34,c7034dd4,c0cce98a,6c,c7034d3c,...) at _cv_wait+0x243 usb_process(c7034d2c,e74f0d28,c0cdc537,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e5830,c7034d2c,e74f0d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74f0d60, ebp = 0 --- Tracing command usb pid 14 tid 100055 td 0xc704cb40 sched_switch(c704cb40,0,104,191,87a433e7,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c704cb40,0,c0ce5bb4,268,c704cb40,...) at sleepq_switch+0x15f sleepq_wait(c7034d04,0,e74edcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7034d04,c7034dd4,c0cce98a,6c,c7034d0c,...) at _cv_wait+0x243 usb_process(c7034cfc,e74edd28,c0cdc537,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e5830,c7034cfc,e74edd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74edd60, ebp = 0 --- Tracing command usb pid 14 tid 100054 td 0xc704e000 sched_switch(c704e000,0,104,191,25a21afb,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c704e000,0,c0ce5bb4,268,c704e000,...) at sleepq_switch+0x15f sleepq_wait(c7034cd4,0,e74eacac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7034cd4,c7034dd4,c0cce98a,6c,c7034cdc,...) at _cv_wait+0x243 usb_process(c7034ccc,e74ead28,c0cdc537,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e5830,c7034ccc,e74ead28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74ead60, ebp = 0 --- Tracing command usb pid 14 tid 100053 td 0xc704e2d0 sched_switch(c704e2d0,0,104,191,25a20983,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c704e2d0,0,c0ce5bb4,268,c704e2d0,...) at sleepq_switch+0x15f sleepq_wait(c7034ca4,0,e74e7cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7034ca4,c7034dd4,c0cce98a,6c,c7034cac,...) at _cv_wait+0x243 usb_process(c7034c9c,e74e7d28,c0cdc537,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e5830,c7034c9c,e74e7d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74e7d60, ebp = 0 --- Tracing command usb pid 14 tid 100051 td 0xc7022000 sched_switch(c7022000,0,104,191,25a1f266,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7022000,0,c0ce5bb4,268,c7022000,...) at sleepq_switch+0x15f sleepq_wait(c702db5c,0,e74decac,1,0,...) at sleepq_wait+0x63 _cv_wait(c702db5c,c702dbfc,c0cce98a,6c,c702db64,...) at _cv_wait+0x243 usb_process(c702db54,e74ded28,c0cdc537,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e5830,c702db54,e74ded28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74ded60, ebp = 0 --- Tracing command usb pid 14 tid 100050 td 0xc70222d0 sched_switch(c70222d0,0,104,191,7ed792a7,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c70222d0,0,c0ce5bb4,268,c70222d0,...) at sleepq_switch+0x15f sleepq_wait(c702db2c,0,e74dbcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c702db2c,c702dbfc,c0cce98a,6c,c702db34,...) at _cv_wait+0x243 usb_process(c702db24,e74dbd28,c0cdc537,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e5830,c702db24,e74dbd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74dbd60, ebp = 0 --- Tracing command usb pid 14 tid 100049 td 0xc70225a0 sched_switch(c70225a0,0,104,191,253d66e1,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c70225a0,0,c0ce5bb4,268,c70225a0,...) at sleepq_switch+0x15f sleepq_wait(c702dafc,0,e74d8cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c702dafc,c702dbfc,c0cce98a,6c,c702db04,...) at _cv_wait+0x243 usb_process(c702daf4,e74d8d28,c0cdc537,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e5830,c702daf4,e74d8d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74d8d60, ebp = 0 --- Tracing command usb pid 14 tid 100048 td 0xc7022870 sched_switch(c7022870,0,104,191,253d52e4,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7022870,0,c0ce5bb4,268,c7022870,...) at sleepq_switch+0x15f sleepq_wait(c702dacc,0,e74d5cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c702dacc,c702dbfc,c0cce98a,6c,c702dad4,...) at _cv_wait+0x243 usb_process(c702dac4,e74d5d28,c0cdc537,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e5830,c702dac4,e74d5d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74d5d60, ebp = 0 --- Tracing command usb pid 14 tid 100047 td 0xc7022b40 sched_switch(c7022b40,0,104,191,253d3f8b,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7022b40,0,c0ce5bb4,268,c7022b40,...) at sleepq_switch+0x15f sleepq_wait(c7026b5c,0,e74d1cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7026b5c,c7026bfc,c0cce98a,6c,c7026b64,...) at _cv_wait+0x243 usb_process(c7026b54,e74d1d28,c0cdc537,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e5830,c7026b54,e74d1d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74d1d60, ebp = 0 --- Tracing command usb pid 14 tid 100046 td 0xc7023000 sched_switch(c7023000,0,104,191,7bbae55f,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7023000,0,c0ce5bb4,268,c7023000,...) at sleepq_switch+0x15f sleepq_wait(c7026b2c,0,e74cecac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7026b2c,c7026bfc,c0cce98a,6c,c7026b34,...) at _cv_wait+0x243 usb_process(c7026b24,e74ced28,c0cdc537,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e5830,c7026b24,e74ced28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74ced60, ebp = 0 --- Tracing command usb pid 14 tid 100045 td 0xc70232d0 sched_switch(c70232d0,0,104,191,1f6ed941,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c70232d0,0,c0ce5bb4,268,c70232d0,...) at sleepq_switch+0x15f sleepq_wait(c7026afc,0,e74cbcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7026afc,c7026bfc,c0cce98a,6c,c7026b04,...) at _cv_wait+0x243 usb_process(c7026af4,e74cbd28,c0cdc537,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e5830,c7026af4,e74cbd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74cbd60, ebp = 0 --- Tracing command usb pid 14 tid 100044 td 0xc70235a0 sched_switch(c70235a0,0,104,191,1f6ec6a8,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c70235a0,0,c0ce5bb4,268,c70235a0,...) at sleepq_switch+0x15f sleepq_wait(c7026acc,0,e74c8cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7026acc,c7026bfc,c0cce98a,6c,c7026ad4,...) at _cv_wait+0x243 usb_process(c7026ac4,e74c8d28,c0cdc537,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e5830,c7026ac4,e74c8d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74c8d60, ebp = 0 --- Tracing command usb pid 14 tid 100043 td 0xc7023870 sched_switch(c7023870,0,104,191,1f6eb1e6,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7023870,0,c0ce5bb4,268,c7023870,...) at sleepq_switch+0x15f sleepq_wait(c701db5c,0,e74c4cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c701db5c,c701dbfc,c0cce98a,6c,c701db64,...) at _cv_wait+0x243 usb_process(c701db54,e74c4d28,c0cdc537,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e5830,c701db54,e74c4d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74c4d60, ebp = 0 --- Tracing command usb pid 14 tid 100042 td 0xc7023b40 sched_switch(c7023b40,0,104,191,7a55e004,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7023b40,0,c0ce5bb4,268,c7023b40,...) at sleepq_switch+0x15f sleepq_wait(c701db2c,0,e74c1cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c701db2c,c701dbfc,c0cce98a,6c,c701db34,...) at _cv_wait+0x243 usb_process(c701db24,e74c1d28,c0cdc537,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e5830,c701db24,e74c1d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74c1d60, ebp = 0 --- Tracing command usb pid 14 tid 100041 td 0xc7024000 sched_switch(c7024000,0,104,191,1f0843c6,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7024000,0,c0ce5bb4,268,c7024000,...) at sleepq_switch+0x15f sleepq_wait(c701dafc,0,e74becac,1,0,...) at sleepq_wait+0x63 _cv_wait(c701dafc,c701dbfc,c0cce98a,6c,c701db04,...) at _cv_wait+0x243 usb_process(c701daf4,e74bed28,c0cdc537,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e5830,c701daf4,e74bed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74bed60, ebp = 0 --- Tracing command usb pid 14 tid 100040 td 0xc7013000 sched_switch(c7013000,0,104,191,1f083122,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7013000,0,c0ce5bb4,268,c7013000,...) at sleepq_switch+0x15f sleepq_wait(c701dacc,0,e74bbcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c701dacc,c701dbfc,c0cce98a,6c,c701dad4,...) at _cv_wait+0x243 usb_process(c701dac4,e74bbd28,c0cdc537,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e5830,c701dac4,e74bbd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74bbd60, ebp = 0 --- Tracing command usb pid 14 tid 100038 td 0xc70135a0 sched_switch(c70135a0,0,104,191,1f081b73,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c70135a0,0,c0ce5bb4,268,c70135a0,...) at sleepq_switch+0x15f sleepq_wait(c7017b5c,0,e74b4cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7017b5c,c7017bfc,c0cce98a,6c,c7017b64,...) at _cv_wait+0x243 usb_process(c7017b54,e74b4d28,c0cdc537,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e5830,c7017b54,e74b4d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74b4d60, ebp = 0 --- Tracing command usb pid 14 tid 100037 td 0xc7013870 sched_switch(c7013870,0,104,191,790c4dfb,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7013870,0,c0ce5bb4,268,c7013870,...) at sleepq_switch+0x15f sleepq_wait(c7017b2c,0,e74b1cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7017b2c,c7017bfc,c0cce98a,6c,c7017b34,...) at _cv_wait+0x243 usb_process(c7017b24,e74b1d28,c0cdc537,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e5830,c7017b24,e74b1d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74b1d60, ebp = 0 --- Tracing command usb pid 14 tid 100036 td 0xc7013b40 sched_switch(c7013b40,0,104,191,1ea2cfa5,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7013b40,0,c0ce5bb4,268,c7013b40,...) at sleepq_switch+0x15f sleepq_wait(c7017afc,0,e74aecac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7017afc,c7017bfc,c0cce98a,6c,c7017b04,...) at _cv_wait+0x243 usb_process(c7017af4,e74aed28,c0cdc537,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e5830,c7017af4,e74aed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74aed60, ebp = 0 --- Tracing command usb pid 14 tid 100035 td 0xc7015000 sched_switch(c7015000,0,104,191,1ea2bbd1,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7015000,0,c0ce5bb4,268,c7015000,...) at sleepq_switch+0x15f sleepq_wait(c7017acc,0,e74abcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7017acc,c7017bfc,c0cce98a,6c,c7017ad4,...) at _cv_wait+0x243 usb_process(c7017ac4,e74abd28,c0cdc537,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e5830,c7017ac4,e74abd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74abd60, ebp = 0 --- Tracing command usb pid 14 tid 100033 td 0xc70155a0 sched_switch(c70155a0,0,104,191,1ea2a7aa,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c70155a0,0,c0ce5bb4,268,c70155a0,...) at sleepq_switch+0x15f sleepq_wait(c7007b5c,0,e74a4cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7007b5c,c7007bfc,c0cce98a,6c,c7007b64,...) at _cv_wait+0x243 usb_process(c7007b54,e74a4d28,c0cdc537,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e5830,c7007b54,e74a4d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74a4d60, ebp = 0 --- Tracing command usb pid 14 tid 100032 td 0xc7015870 sched_switch(c7015870,0,104,191,777074dc,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7015870,0,c0ce5bb4,268,c7015870,...) at sleepq_switch+0x15f sleepq_wait(c7007b2c,0,e74a1cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7007b2c,c7007bfc,c0cce98a,6c,c7007b34,...) at _cv_wait+0x243 usb_process(c7007b24,e74a1d28,c0cdc537,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e5830,c7007b24,e74a1d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74a1d60, ebp = 0 --- Tracing command usb pid 14 tid 100031 td 0xc7015b40 sched_switch(c7015b40,0,104,191,1e35aab9,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7015b40,0,c0ce5bb4,268,c7015b40,...) at sleepq_switch+0x15f sleepq_wait(c7007afc,0,e749ecac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7007afc,c7007bfc,c0cce98a,6c,c7007b04,...) at _cv_wait+0x243 usb_process(c7007af4,e749ed28,c0cdc537,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e5830,c7007af4,e749ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe749ed60, ebp = 0 --- Tracing command usb pid 14 tid 100030 td 0xc6eb82d0 sched_switch(c6eb82d0,0,104,191,1e3597fe,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c6eb82d0,0,c0ce5bb4,268,c6eb82d0,...) at sleepq_switch+0x15f sleepq_wait(c7007acc,0,e749bcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7007acc,c7007bfc,c0cce98a,6c,c7007ad4,...) at _cv_wait+0x243 usb_process(c7007ac4,e749bd28,c0cdc537,349,c6f7b2a8,...) at usb_process+0x193 fork_exit(c07e5830,c7007ac4,e749bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe749bd60, ebp = 0 --- Tracing command yarrow pid 13 tid 100017 td 0xc6eb75a0 sched_switch(c6eb75a0,0,104,191,bb075042,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c6eb75a0,0,c0ce5bb4,28b,2,...) at sleepq_switch+0x15f sleepq_timedwait(c0e45c84,0,c0cd6178,2,0,...) at sleepq_timedwait+0x6b _sleep(c0e45c84,0,0,c0cd6178,64,...) at _sleep+0x342 pause(c0cd6178,64,c0cc2279,111,0,...) at pause+0x47 random_kthread(0,c6b45d28,c0cdc537,349,c6f7b550,...) at random_kthread+0x1ef fork_exit(c0757490,0,c6b45d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b45d60, ebp = 0 --- Tracing command g_down pid 4 tid 100015 td 0xc6eb7b40 sched_switch(c6eb7b40,0,104,191,16f1fac,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,4c,...) at mi_switch+0x200 sleepq_switch(c6eb7b40,0,c0ce5bb4,268,0,...) at sleepq_switch+0x15f sleepq_wait(c0e43944,4c,c0cd6178,0,0,...) at sleepq_wait+0x63 _sleep(c0e43944,c0e438a8,24c,c0cd6178,0,...) at _sleep+0x372 g_io_schedule_down(c6eb7b40,0,c0cd78d5,74,0,...) at g_io_schedule_down+0x5b g_down_procbody(0,c6b3fd28,c0cdc537,349,c6d99000,...) at g_down_procbody+0x8d fork_exit(c0854ba0,0,c6b3fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b3fd60, ebp = 0 --- Tracing command g_up pid 3 tid 100014 td 0xc6eb8000 sched_switch(c6eb8000,0,104,191,184236e,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,4c,...) at mi_switch+0x200 sleepq_switch(c6eb8000,0,c0ce5bb4,268,0,...) at sleepq_switch+0x15f sleepq_wait(c0e43940,4c,c0cd6178,0,0,...) at sleepq_wait+0x63 _sleep(c0e43940,c0e438c8,24c,c0cd6178,0,...) at _sleep+0x372 g_io_schedule_up(c6eb8000,0,c0cd78d5,5d,0,...) at g_io_schedule_up+0x134 g_up_procbody(0,c6b3cd28,c0cdc537,349,c6d992a8,...) at g_up_procbody+0x8d fork_exit(c0854c30,0,c6b3cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b3cd60, ebp = 0 --- Tracing command g_event pid 2 tid 100013 td 0xc6da12d0 sched_switch(c6da12d0,0,104,191,b6a157a9,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,4c,...) at mi_switch+0x200 sleepq_switch(c6da12d0,0,c0ce5bb4,28b,0,...) at sleepq_switch+0x15f sleepq_timedwait(c0e43938,4c,c0cd6178,0,0,...) at sleepq_timedwait+0x6b _sleep(c0e43938,0,4c,c0cd6178,64,...) at _sleep+0x342 g_event_procbody(0,c6b39d28,c0cdc537,349,c6d99550,...) at g_event_procbody+0xcb fork_exit(c0854cc0,0,c6b39d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b39d60, ebp = 0 --- Tracing command intr pid 12 tid 100064 td 0xc7164000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100063 td 0xc70242d0 sched_switch(c70242d0,0,109,191,1e31ac75,...) at sched_switch+0x3bc mi_switch(109,0,c0cdc7b6,52d,c6de0ef0,...) at mi_switch+0x200 ithread_loop(c6de85d0,e7516d28,c0cdc537,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c08937a0,c6de85d0,e7516d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7516d60, ebp = 0 --- Tracing command intr pid 12 tid 100061 td 0xc7024870 sched_switch(c7024870,0,109,191,8371cc51,...) at sched_switch+0x3bc mi_switch(109,0,c0cdc7b6,52d,c6de1470,...) at mi_switch+0x200 ithread_loop(c6de82c0,e7506d28,c0cdc537,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c08937a0,c6de82c0,e7506d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7506d60, ebp = 0 --- Tracing command intr pid 12 tid 100058 td 0xc704c2d0 sched_switch(c704c2d0,0,109,191,1daa833f,...) at sched_switch+0x3bc mi_switch(109,0,c0cdc7b6,52d,c6de0570,...) at mi_switch+0x200 ithread_loop(c6de83a0,e74fad28,c0cdc537,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c08937a0,c6de83a0,e74fad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74fad60, ebp = 0 --- Tracing command intr pid 12 tid 100057 td 0xc704c5a0 sched_switch(c704c5a0,0,109,191,2b31e845,...) at sched_switch+0x3bc mi_switch(109,0,c0cdc7b6,52d,c6de0870,...) at mi_switch+0x200 ithread_loop(c6de83f0,e74f6d28,c0cdc537,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c08937a0,c6de83f0,e74f6d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74f6d60, ebp = 0 --- Tracing command intr pid 12 tid 100052 td 0xc704e5a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100039 td 0xc70132d0 sched_switch(c70132d0,0,109,191,43900bf3,...) at sched_switch+0x3bc mi_switch(109,0,c0cdc7b6,52d,c6de0670,...) at mi_switch+0x200 ithread_loop(c6de8650,e74b8d28,c0cdc537,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c08937a0,c6de8650,e74b8d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74b8d60, ebp = 0 --- Tracing command intr pid 12 tid 100034 td 0xc70152d0 sched_switch(c70152d0,0,109,191,3dc4365b,...) at sched_switch+0x3bc mi_switch(109,0,c0cdc7b6,52d,c6de06f0,...) at mi_switch+0x200 ithread_loop(c6de8080,e74a8d28,c0cdc537,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c08937a0,c6de8080,e74a8d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe74a8d60, ebp = 0 --- Tracing command intr pid 12 tid 100029 td 0xc6eb85a0 sched_switch(c6eb85a0,0,109,191,3bf1e425,...) at sched_switch+0x3bc mi_switch(109,0,c0cdc7b6,52d,c6de0770,...) at mi_switch+0x200 ithread_loop(c6de86c0,e7498d28,c0cdc537,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c08937a0,c6de86c0,e7498d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7498d60, ebp = 0 --- Tracing command intr pid 12 tid 100028 td 0xc6eb8870 sched_switch(c6eb8870,0,109,191,1834d15,...) at sched_switch+0x3bc mi_switch(109,0,c0cdc7b6,52d,c6de0470,...) at mi_switch+0x200 ithread_loop(c6de80d0,e7389d28,c0cdc537,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c08937a0,c6de80d0,e7389d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7389d60, ebp = 0 --- Tracing command intr pid 12 tid 100027 td 0xc6eb8b40 sched_switch(c6eb8b40,0,109,191,1e325727,...) at sched_switch+0x3bc mi_switch(109,0,c0cdc7b6,52d,c6de0af0,...) at mi_switch+0x200 ithread_loop(c6de8630,c6b8ad28,c0cdc537,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c08937a0,c6de8630,c6b8ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b8ad60, ebp = 0 --- Tracing command intr pid 12 tid 100026 td 0xc6fa3000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100024 td 0xc6fa35a0 sched_switch(c6fa35a0,0,109,191,84364e95,...) at sched_switch+0x3bc mi_switch(109,0,c0cdc7b6,52d,c6fa20f0,...) at mi_switch+0x200 ithread_loop(c6de8770,c6b5ad28,c0cdc537,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c08937a0,c6de8770,c6b5ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b5ad60, ebp = 0 --- Tracing command intr pid 12 tid 100019 td 0xc6eb7000 sched_switch(c6eb7000,0,109,191,348ca8fa,...) at sched_switch+0x3bc mi_switch(109,0,c0cdc7b6,52d,c6fa2170,...) at mi_switch+0x200 ithread_loop(c6de8780,c6b4bd28,c0cdc537,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c08937a0,c6de8780,c6b4bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b4bd60, ebp = 0 --- Tracing command intr pid 12 tid 100018 td 0xc6eb72d0 sched_switch(c6eb72d0,0,109,191,6cd65d74,...) at sched_switch+0x3bc mi_switch(109,0,c0cdc7b6,52d,c6de0070,...) at mi_switch+0x200 ithread_loop(c6de8790,c6b48d28,c0cdc537,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c08937a0,c6de8790,c6b48d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b48d60, ebp = 0 --- Tracing command intr pid 12 tid 100012 td 0xc6da15a0 sched_switch(c6da15a0,0,109,191,abe66d11,...) at sched_switch+0x3bc mi_switch(109,0,c0cdc7b6,52d,c6de00f0,...) at mi_switch+0x200 ithread_loop(c6de87a0,c6b36d28,c0cdc537,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c08937a0,c6de87a0,c6b36d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b36d60, ebp = 0 --- Tracing command intr pid 12 tid 100011 td 0xc6da1870 sched_switch(c6da1870,0,109,191,b6a1201d,...) at sched_switch+0x3bc mi_switch(109,0,c0cdc7b6,52d,c6de0170,...) at mi_switch+0x200 ithread_loop(c6de87b0,c6b33d28,c0cdc537,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c08937a0,c6de87b0,c6b33d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b33d60, ebp = 0 --- Tracing command intr pid 12 tid 100010 td 0xc6da1b40 sched_switch(c6da1b40,0,109,191,bb074eed,...) at sched_switch+0x3bc mi_switch(109,0,c0cdc7b6,52d,c6de01f0,...) at mi_switch+0x200 ithread_loop(c6de87c0,c6b30d28,c0cdc537,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c08937a0,c6de87c0,c6b30d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b30d60, ebp = 0 --- Tracing command intr pid 12 tid 100009 td 0xc6de9000 sched_switch(c6de9000,0,109,191,c47808f1,...) at sched_switch+0x3bc mi_switch(109,0,c0cdc7b6,52d,c6de0270,...) at mi_switch+0x200 ithread_loop(c6de87d0,c6b2dd28,c0cdc537,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c08937a0,c6de87d0,c6b2dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b2dd60, ebp = 0 --- Tracing command intr pid 12 tid 100008 td 0xc6de92d0 sched_switch(c6de92d0,0,109,191,16e18fbf,...) at sched_switch+0x3bc mi_switch(109,0,c0cdc7b6,52d,c6de02f0,...) at mi_switch+0x200 ithread_loop(c6de87e0,c6b2ad28,c0cdc537,349,c6d997f8,...) at ithread_loop+0x206 fork_exit(c08937a0,c6de87e0,c6b2ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b2ad60, ebp = 0 --- Tracing command intr pid 12 tid 100007 td 0xc6de95a0 fork_trampoline() at fork_trampoline Tracing command idle pid 11 tid 100006 td 0xc6d9b000 cpustop_handler(1,c6b23c24,c0c00f66,c6b23bdc,c0c09feb,...) at cpustop_handler+0x34 ipi_nmi_handler(c6b23bdc,c0c09feb,c1425c80,0,c6d99aa0,...) at ipi_nmi_handler+0x2f trap(c6b23c30) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0c016e5, esp = 0xc6b23c70, ebp = 0xc6b23c70 --- rdtsc(c0e498c0,14,c0ce1de1,18c,c08aae14,...) at rdtsc+0x5 mi_switch(108,0,c0ce2f85,3aa,c6d9b000,...) at mi_switch+0x164 sched_idletd(0,c6b23d28,c0cdc537,349,c6d99aa0,...) at sched_idletd+0x19f fork_exit(c08e0980,0,c6b23d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b23d60, ebp = 0 --- Tracing command idle pid 11 tid 100005 td 0xc6d9b2d0 sched_switch(c6d9b2d0,0,108,18c,c123308c,...) at sched_switch+0x3bc mi_switch(108,0,c0ce2f85,3aa,c6d9b2d0,...) at mi_switch+0x200 sched_idletd(0,c6b20d28,c0cdc537,349,c6d99aa0,...) at sched_idletd+0x19f fork_exit(c08e0980,0,c6b20d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b20d60, ebp = 0 --- Tracing command idle pid 11 tid 100004 td 0xc6d9b5a0 sched_switch(c6d9b5a0,0,108,18c,c1293567,...) at sched_switch+0x3bc mi_switch(108,0,c0ce2f85,3aa,c6d9b5a0,...) at mi_switch+0x200 sched_idletd(0,c6b1dd28,c0cdc537,349,c6d99aa0,...) at sched_idletd+0x19f fork_exit(c08e0980,0,c6b1dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b1dd60, ebp = 0 --- Tracing command idle pid 11 tid 100003 td 0xc6d9b870 sched_switch(c6d9b870,0,108,18c,bb5813fb,...) at sched_switch+0x3bc mi_switch(108,0,c0ce2f85,3aa,c6d9b870,...) at mi_switch+0x200 sched_idletd(0,c6b1ad28,c0cdc537,349,c6d99aa0,...) at sched_idletd+0x19f fork_exit(c08e0980,0,c6b1ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b1ad60, ebp = 0 --- Tracing command init pid 1 tid 100002 td 0xc6d9bb40 sched_switch(c6d9bb40,0,104,191,c1bc63c3,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,5c,...) at mi_switch+0x200 sleepq_switch(c6d9bb40,0,c0ce5bb4,1a7,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0ce5bb4,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6d99d48,5c,c0ce8513,100,0,...) at sleepq_wait_sig+0x17 _sleep(c6d99d48,c6d99dd0,15c,c0ce8513,0,...) at _sleep+0x35c kern_wait(c6d9bb40,ffffffff,c6b16c44,0,0,...) at kern_wait+0xb66 wait4(c6d9bb40,c6b16cec,c6b16d28,c0ce63ed,0,...) at wait4+0x3b syscallenter(c6d9bb40,c6b16ce4,c6b16ce4,0,0,...) at syscallenter+0x263 syscall(c6b16d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x8054cd3, esp = 0xbfbfe8dc, ebp = 0xbfbfe8f8 --- Tracing command audit pid 10 tid 100001 td 0xc6da1000 sched_switch(c6da1000,0,104,191,1e330cb8,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c6da1000,0,c0ce5bb4,268,c6da1000,...) at sleepq_switch+0x15f sleepq_wait(c0fbdf00,0,c6b13c8c,1,0,...) at sleepq_wait+0x63 _cv_wait(c0fbdf00,c0fbdee4,c0d06a5f,194,0,...) at _cv_wait+0x243 audit_worker(0,c6b13d28,c0cdc537,349,c6d9a000,...) at audit_worker+0x84 fork_exit(c0ac9340,0,c6b13d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b13d60, ebp = 0 --- Tracing command kernel pid 0 tid 100065 td 0xc7162b40 sched_switch(c7162b40,0,104,191,f6115e60,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c7162b40,0,c0ce5bb4,28b,2,...) at sleepq_switch+0x15f sleepq_timedwait(c0e45c84,0,c0cda6de,2,0,...) at sleepq_timedwait+0x6b _sleep(c0e45c84,0,0,c0cda6de,bb8,...) at _sleep+0x342 pause(c0cda6de,bb8,120,11e,c0e43aa8,...) at pause+0x47 deadlkres(0,e751cd28,c0cdc537,349,c0e43a20,...) at deadlkres+0x302 fork_exit(c08755a0,0,e751cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe751cd60, ebp = 0 --- Tracing command kernel pid 0 tid 100059 td 0xc704c000 sched_switch(c704c000,0,104,191,c0657642,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c704c000,0,c0ce5bb4,268,c8694870,...) at sleepq_switch+0x15f sleepq_wait(c6eabb40,0,c0ce1f19,c0cd6178,0,...) at sleepq_wait+0x63 msleep_spin(c6eabb40,c6eabb5c,c0cd6178,0,c0cdfaea,...) at msleep_spin+0x21d taskqueue_thread_loop(c705d49c,e7500d28,c0cdc537,349,c0e43a20,...) at taskqueue_thread_loop+0x8f fork_exit(c08fb680,c705d49c,e7500d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe7500d60, ebp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xc6fa32d0 sched_switch(c6fa32d0,0,104,191,348ee5b0,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c6fa32d0,0,c0ce5bb4,268,0,...) at sleepq_switch+0x15f sleepq_wait(c6eabc80,0,c0cd6178,0,0,...) at sleepq_wait+0x63 _sleep(c6eabc80,c6eabc9c,0,c0cd6178,0,...) at _sleep+0x372 taskqueue_thread_loop(c0e585e8,c6b5dd28,c0cdc537,349,c0e43a20,...) at taskqueue_thread_loop+0xb5 fork_exit(c08fb680,c0e585e8,c6b5dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b5dd60, ebp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xc6fa3870 sched_switch(c6fa3870,0,104,191,26154650,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c6fa3870,0,c0ce5bb4,268,0,...) at sleepq_switch+0x15f sleepq_wait(c6eabd00,0,c0cd6178,0,0,...) at sleepq_wait+0x63 _sleep(c6eabd00,c6eabd1c,0,c0cd6178,0,...) at _sleep+0x372 taskqueue_thread_loop(c0e44378,c6b57d28,c0cdc537,349,c0e43a20,...) at taskqueue_thread_loop+0xb5 fork_exit(c08fb680,c0e44378,c6b57d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b57d60, ebp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xc6fa3b40 sched_switch(c6fa3b40,0,104,191,26153397,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c6fa3b40,0,c0ce5bb4,268,0,...) at sleepq_switch+0x15f sleepq_wait(c6eabd40,0,c0ce1f19,c0cd6178,0,...) at sleepq_wait+0x63 msleep_spin(c6eabd40,c6eabd5c,c0cd6178,0,c0cdfaea,...) at msleep_spin+0x21d taskqueue_thread_loop(c0e149a0,c6b54d28,c0cdc537,349,c0e43a20,...) at taskqueue_thread_loop+0x8f fork_exit(c08fb680,c0e149a0,c6b54d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b54d60, ebp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xc6de9870 sched_switch(c6de9870,0,104,191,26152356,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c6de9870,0,c0ce5bb4,268,0,...) at sleepq_switch+0x15f sleepq_wait(c6eabd40,0,c0ce1f19,c0cd6178,0,...) at sleepq_wait+0x63 msleep_spin(c6eabd40,c6eabd5c,c0cd6178,0,c0cdfaea,...) at msleep_spin+0x21d taskqueue_thread_loop(c0e149a0,c6b51d28,c0cdc537,349,c0e43a20,...) at taskqueue_thread_loop+0x8f fork_exit(c08fb680,c0e149a0,c6b51d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b51d60, ebp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xc6de9b40 sched_switch(c6de9b40,0,104,191,261510de,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c6de9b40,0,c0ce5bb4,268,0,...) at sleepq_switch+0x15f sleepq_wait(c6eabd40,0,c0ce1f19,c0cd6178,0,...) at sleepq_wait+0x63 msleep_spin(c6eabd40,c6eabd5c,c0cd6178,0,c0cdfaea,...) at msleep_spin+0x21d taskqueue_thread_loop(c0e149a0,c6b4ed28,c0cdc537,349,c0e43a20,...) at taskqueue_thread_loop+0x8f fork_exit(c08fb680,c0e149a0,c6b4ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b4ed60, ebp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xc6eb7870 sched_switch(c6eb7870,0,104,191,39c375f3,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,0,...) at mi_switch+0x200 sleepq_switch(c6eb7870,0,c0ce5bb4,268,0,...) at sleepq_switch+0x15f sleepq_wait(c6eabdc0,0,c0cd6178,0,0,...) at sleepq_wait+0x63 _sleep(c6eabdc0,c6eabddc,0,c0cd6178,0,...) at _sleep+0x372 taskqueue_thread_loop(c0e57060,c6b42d28,c0cdc537,349,c0e43a20,...) at taskqueue_thread_loop+0xb5 fork_exit(c08fb680,c0e57060,c6b42d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b42d60, ebp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xc0e43cd0 sched_switch(c0e43cd0,0,104,191,7de43705,...) at sched_switch+0x3bc mi_switch(104,0,c0ce5bb4,1f3,44,...) at mi_switch+0x200 sleepq_switch(c0e43cd0,0,c0ce5bb4,28b,0,...) at sleepq_switch+0x15f sleepq_timedwait(c0e43a20,44,c0ce3899,0,0,...) at sleepq_timedwait+0x6b _sleep(c0e43a20,0,44,c0ce3899,2710,...) at _sleep+0x342 scheduler(0,141ec00,141ec00,141e000,1425000,...) at scheduler+0x23e mi_startup() at mi_startup+0x96 begin() at begin+0x2c db:0:allt> call doadump Physical memory: 3439 MB Dumping 139 MB: 124 108 92 76 60 44 28 12 Dump complete = 0xf db:0:doadump> reset cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 2 (kgdb) bt #0 doadump () at pcpu.h:231 #1 0xc04d5b19 in db_fncall (dummy1=0xc08ee48a, dummy2=0x0, dummy3=0xffffffff, dummy4=0xeb0658bc "ÐX\006ë") at ../../../ddb/db_command.c:548 #2 0xc04d5f4f in db_command (last_cmdp=0xc0e12b7c, cmd_table=0x0, dopager=0x0) at ../../../ddb/db_command.c:445 #3 0xc04d6004 in db_command_script (command=0xc0e13a88 "call doadump") at ../../../ddb/db_command.c:516 #4 0xc04da250 in db_script_exec (scriptname=0xc0e133e0 "doadump", warnifnotfound=Variable "warnifnotfound" is not available. ) at ../../../ddb/db_script.c:302 #5 0xc04da2e1 in db_run_cmd (addr=0x1, have_addr=0x0, count=0xc0fe1820, modif=0xeb0659f4 "") at ../../../ddb/db_script.c:375 #6 0xc04d5f11 in db_command (last_cmdp=0xc0e12b7c, cmd_table=0x0, dopager=0x1) at ../../../ddb/db_command.c:445 #7 0xc04d606a in db_command_loop () at ../../../ddb/db_command.c:498 #8 0xc04d7f8d in db_trap (type=0x3, code=0x0) at ../../../ddb/db_main.c:229 #9 0xc08ee2be in kdb_trap (type=0x3, code=0x0, tf=0xeb065b98) at ../../../kern/subr_kdb.c:546 #10 0xc0c0159b in trap (frame=0xeb065b98) at ../../../i386/i386/trap.c:732 #11 0xc0be83ec in calltrap () at ../../../i386/i386/exception.s:168 #12 0xc08ee48a in kdb_enter (why=0xc0ce1363 "panic", msg=0xc0ce1363 "panic") at cpufunc.h:71 #13 0xc08bb074 in panic (fmt=0xc0c77a9f "Bad link elm %p next->prev != elm") at ../../../kern/kern_shutdown.c:575 #14 0xc08c8d77 in tidhash_remove (td=0xc8900000) at ../../../kern/kern_thread.c:1024 #15 0xc08c836e in thr_exit (td=0xc8900000, uap=0xeb065cec) at ../../../kern/kern_thr.c:288 #16 0xc08fba43 in syscallenter (td=0xc8900000, sa=0xeb065ce4) at ../../../kern/subr_trap.c:318 #17 0xc0c00e4f in syscall (frame=0xeb065d28) at ../../../i386/i386/trap.c:1095 #18 0xc0be8451 in Xint0x80_syscall () at ../../../i386/i386/exception.s:266 #19 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) f 14 #14 0xc08c8d77 in tidhash_remove (td=0xc8900000) at ../../../kern/kern_thread.c:1024 1024 LIST_REMOVE(td, td_hash); (kgdb) l 1019 } else 1020 thread_unlock(td); 1021 #endif 1022 thread_lock(td); 1023 if ((td->td_flags & TDF_TIDHASH) != 0) { 1024 LIST_REMOVE(td, td_hash); 1025 td->td_flags &= ~TDF_TIDHASH; 1026 } 1027 thread_unlock(td); 1028 rw_wunlock(&tidhash_lock); (kgdb) p *td $1 = {td_lock = 0xc0e4a5c0, td_proc = 0xc6f7b7f8, td_plist = {tqe_next = 0xc7cc8b40, tqe_prev = 0xc8a12008}, td_runq = {tqe_next = 0x0, tqe_prev = 0xc0e4a8c0}, td_slpq = {tqe_next = 0xc89dfb40, tqe_prev = 0xc8a12018}, td_lockq = { tqe_next = 0xc871eb40, tqe_prev = 0xe9f97c74}, td_hash = {le_next = 0xc7cd6870, le_prev = 0xc7cd6898}, td_cpuset = 0xc6d79e38, td_sel = 0x0, td_sleepqueue = 0xc882e700, td_turnstile = 0xc88ff700, td_umtxq = 0xc8837200, td_tid = 0x18f28, td_sigqueue = {sq_signals = {__bits = {0x0, 0x0, 0x0, 0x0}}, sq_kill = {__bits = {0x0, 0x0, 0x0, 0x0}}, sq_list = {tqh_first = 0x0, tqh_last = 0xc8900068}, sq_proc = 0xc6f7b7f8, sq_flags = 0x1}, td_flags = 0x90004, td_inhibitors = 0x0, td_pflags = 0x0, td_dupfd = 0x0, td_sqqueue = 0x0, td_wchan = 0x0, td_wmesg = 0x0, td_lastcpu = 0x0, td_oncpu = 0x2, td_owepreempt = 0x0, td_tsqueue = 0x0, td_locks = 0x1, td_rw_rlocks = 0x0, td_lk_slocks = 0x0, td_blocked = 0x0, td_lockname = 0x0, td_contested = {lh_first = 0x0}, td_sleeplocks = 0xc0f82040, td_intr_nesting_level = 0x0, td_pinned = 0x1, td_ucred = 0xc6d7c980, td_estcpu = 0x0, td_slptick = 0x0, td_blktick = 0x0, td_ru = { ru_utime = {tv_sec = 0x0, tv_usec = 0x0}, ru_stime = {tv_sec = 0x0, tv_usec = 0x0}, ru_maxrss = 0x0, ru_ixrss = 0x0, ru_idrss = 0x0, ru_isrss = 0x0, ru_minflt = 0x0, ru_majflt = 0x0, ru_nswap = 0x0, ru_inblock = 0x0, ru_oublock = 0x0, ru_msgsnd = 0x0, ru_msgrcv = 0x0, ru_nsignals = 0x0, ru_nvcsw = 0x0, ru_nivcsw = 0x0}, td_rux = {rux_runtime = 0x0, rux_uticks = 0x0, rux_sticks = 0x0, rux_iticks = 0x0, rux_uu = 0x0, rux_su = 0x0, rux_tu = 0x0}, td_incruntime = 0x0, td_runtime = 0x0, td_pticks = 0x0, td_sticks = 0x0, td_iticks = 0x0, td_uticks = 0x0, td_intrval = 0x0, td_oldsigmask = {__bits = {0x0, 0x0, 0x0, 0x0}}, td_sigmask = {__bits = {0x0, 0x0, 0x0, 0x0}}, td_generation = 0x0, td_sigstk = {ss_sp = 0x0, ss_size = 0x0, ss_flags = 0x0}, td_xsig = 0x0, td_profil_addr = 0x0, td_profil_ticks = 0x0, td_name = "thr2", '\0' , td_fpop = 0x0, td_dbgflags = 0x0, td_dbgksi = { ksi_link = {tqe_next = 0x0, tqe_prev = 0x0}, ksi_info = {si_signo = 0x0, si_errno = 0x0, si_code = 0x0, si_pid = 0x0, si_uid = 0x0, si_status = 0x0, si_addr = 0x0, si_value = {sival_int = 0x0, sival_ptr = 0x0, sigval_int = 0x0, sigval_ptr = 0x0}, _reason = {_fault = {_trapno = 0x0}, _timer = {_timerid = 0x0, _overrun = 0x0}, _mesgq = {_mqd = 0x0}, _poll = {_band = 0x0}, __spare__ = {__spare1__ = 0x0, __spare2__ = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}}}, ksi_flags = 0x0, ksi_sigq = 0x0}, td_ng_outbound = 0x0, td_osd = {osd_nslots = 0x0, osd_slots = 0x0, osd_next = {le_next = 0x0, le_prev = 0x0}}, td_map_def_user = 0x0, td_rqindex = 0x18, td_base_pri = 0xb8, td_priority = 0xb8, td_pri_class = 0x3, td_user_pri = 0xb8, td_base_user_pri = 0xb8, td_pcb = 0xeb065d80, td_state = TDS_RUNNING, td_retval = {0x0, 0x0}, td_slpcallout = {c_links = {sle = {sle_next = 0x0}, tqe = {tqe_next = 0x0, tqe_prev = 0x0}}, c_time = 0x0, c_arg = 0x0, c_func = 0, c_lock = 0x0, c_flags = 0x10, c_cpu = 0x0}, td_frame = 0xeb065d28, td_kstack_obj = 0xc88e3c38, td_kstack = 0xeb064000, td_kstack_pages = 0x2, td_critnest = 0x2, td_md = { md_spinlock_count = 0x1, md_saved_flags = 0x246}, td_sched = 0xc89002a4, td_ar = 0x0, td_lprof = {{lh_first = 0x0}, {lh_first = 0x0}}, td_dtrace = 0x0, td_errno = 0x0, td_vnet = 0x0, td_vnet_lpush = 0x0, td_intr_frame = 0x0} (kgdb) $ svn diff -x -p /usr/src/sys Index: /usr/src/sys/kern/kern_thread.c =================================================================== --- /usr/src/sys/kern/kern_thread.c (revision 213890) +++ /usr/src/sys/kern/kern_thread.c (working copy) @@ -361,6 +361,8 @@ thread_exit(void) td = curthread; p = td->td_proc; + KASSERT((td->td_flags & TDF_TIDHASH) == 0, + ("thread is still on hash table.")); PROC_SLOCK_ASSERT(p, MA_OWNED); mtx_assert(&Giant, MA_NOTOWNED); @@ -949,6 +951,9 @@ tdfind(lwpid_t tid, pid_t pid) rw_rlock(&tidhash_lock); LIST_FOREACH(td, TIDHASH(tid), td_hash) { + KASSERT(TIDHASH(td->td_tid) == TIDHASH(tid), + ("process %p (pid %d), thread %p changed thread id.", + td->td_proc, td->td_proc->p_pid, td)); if (td->td_tid == tid) { if (pid != -1 && td->td_proc->p_pid != pid) { td = NULL; @@ -993,9 +998,29 @@ tidhash_add(struct thread *td) void tidhash_remove(struct thread *td) { +#ifdef INVARIANTS + struct thread *td2; + lwpid_t tid; +#endif + rw_wlock(&tidhash_lock); +#ifdef INVARIANTS thread_lock(td); if ((td->td_flags & TDF_TIDHASH) != 0) { + tid = td->td_tid; + LIST_FOREACH(td2, TIDHASH(tid), td_hash) { + if (td2 == td) + break; + } + thread_unlock(td); + KASSERT(td2 != NULL, + ("process %p (pid %d), thread %p changed thread id.", + td->td_proc, td->td_proc->p_pid, td)); + } else + thread_unlock(td); +#endif + thread_lock(td); + if ((td->td_flags & TDF_TIDHASH) != 0) { LIST_REMOVE(td, td_hash); td->td_flags &= ~TDF_TIDHASH; } Index: /usr/src/sys/kern/init_main.c =================================================================== --- /usr/src/sys/kern/init_main.c (revision 213890) +++ /usr/src/sys/kern/init_main.c (working copy) @@ -443,7 +443,6 @@ proc0_init(void *dummy __unused) */ LIST_INSERT_HEAD(&allproc, p, p_list); LIST_INSERT_HEAD(PIDHASH(0), p, p_hash); - LIST_INSERT_HEAD(TIDHASH(0), td, td_hash); mtx_init(&pgrp0.pg_mtx, "process group", NULL, MTX_DEF | MTX_DUPOK); p->p_pgrp = &pgrp0; LIST_INSERT_HEAD(PGRPHASH(0), &pgrp0, pg_hash); @@ -462,6 +461,8 @@ proc0_init(void *dummy __unused) STAILQ_INIT(&p->p_ktr); p->p_nice = NZERO; td->td_tid = PID_MAX + 1; + LIST_INSERT_HEAD(TIDHASH(td->td_tid), td, td_hash); + td->td_flags |= TDF_TIDHASH; td->td_state = TDS_RUNNING; td->td_pri_class = PRI_TIMESHARE; td->td_user_pri = PUSER; -- Test scenario: thr2