GENERIC from Fri Jan 15 16:57:24 2021 +0000, main 47f7345baba, vmcore.1191 GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb ---<>--- Copyright (c) 1992-2021 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 13.0-ALPHA1 #0 main-c256108-g47f7345baba: Tue Jan 19 12:51:33 CET 2021 root@x4.osted.lan:/usr/src/sys/i386/compile/PHO i386 FreeBSD clang version 11.0.1 (git@github.com:llvm/llvm-project.git llvmorg-11.0.1-0-g43ff75f2c3fe) WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. VT(vga): resolution 640x480 CPU: AMD Phenom(tm) 9150e Quad-Core Processor (1800.02-MHz 686-class CPU) Origin="AuthenticAMD" Id=0x100f23 Family=0x10 Model=0x2 Stepping=3 Features=0x178bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x7ff SVM: NP,NAsids=64 TSC: P-state invariant real memory = 8589934592 (8192 MB) avail memory = 3522801664 (3359 MB) : Trying to mount root from ufs:/dev/ada0s1a [rw]... WARNING: WITNESS option enabled, expect reduced performance. Root mount waiting for:WARNING: DIAGNOSTIC option enabled, expect reduced performance. CAMugen4.1: at usbus4 ugen2.1: at usbus2 uhub4 usbus0 on usbus4 usbus1Expensive timeout(9) function: 0xa535e0(0x12be8000) 0.009020420 s uhub5(aprobe0:ahcich0:0:15:0): NOP FLUSHQUEUE. ACB: 00 00 00 00 00 00 00 00 00 00 00 00 on usbus2 uhub4: on usbus4 usbus2(aprobe0:ahcich0:0:15:0): CAM status: Command timeout uhub5: on usbus2 (aprobe0:ahcich0:0:15:0): Error 5, Retries exhausted usbus3 usbus4 usbus5 ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: ATA8-ACS SATA 2.x device ada0: Serial Number 9VMJ99G5 ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 476940MB (976773168 512 byte sectors) uhub0: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered uhub4: 2 ports with 2 removable, self powered uhub5: 2 ports with 2 removable, self powered Root mount waiting for: usbus5 Root mount waiting for: usbus5 Root mount waiting for: usbus5 Root mount waiting for: usbus5 uhub2: 10 ports with 10 removable, self powered mountroot: waiting for device /dev/ada0s1a... WARNING: / was not properly dismounted Dual Console: Serial Primary, Video Secondary Setting hostuuid: 00000000-0000-0000-0000-00218515337d. Setting hostid: 0x6b64ac17. warning: total configured swap (4194304 pages) exceeds maximum recommended amount (2009952 pages). warning: increase kern.maxswzone or reduce amount of swap. Starting file system checks: /dev/ada0s1a: 8161 files, 410760 used, 602255 free (2063 frags, 75024 blocks, 0.2% fragmentation) /dev/ada0s1f: CANNOT READ BLK: 352 /dev/ada0s1f: UNEXPECTED SOFT UPDATE INCONSISTENCY; RUN fsck MANUALLY. /dev/ada0s1d: CANNOT READ BLK: 3424 /dev/ada0s1d: UNEXPECTED SOFT UPDATE INCONSISTENCY; RUN fsck MANUALLY. ** SU+J Recovering /dev/ada0s1e ** Reading 33554432 byte journal from inode 6. ** Building recovery table. ** Resolving unreferenced inode list. ** Processing journal entries. ** 7 journal records in 3072 bytes for 7.29% utilization ** Freed 0 inodes (0 dirs) 0 blocks, and 0 frags. ***** FILE SYSTEM MARKED CLEAN ***** /dev/label/tmp: CANNOT READ BLK: 376704 /dev/label/tmp: UNEXPECTED SOFT UPDATE INCONSISTENCY; RUN fsck MANUALLY. THE FOLLOWING FILE SYSTEMS HAD AN UNEXPECTED INCONSISTENCY: ufs: /dev/ada0s1f (/home), ufs: /dev/ada0s1d (/usr), ufs: /dev/label/tmp (/tmp) File system preen failed, trying fsck -y -T ffs:-R -T ufs:-R ** /dev/ada0s1a ** Last Mounted on / ** Root file system ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 8161 files, 410760 used, 602255 free (2063 frags, 75024 blocks, 0.2% fragmentation) ***** FILE SYSTEM IS CLEAN ***** ** /dev/ada0s1f ** Last Mounted on /home ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 18518 files, 240925 used, 772090 free (11746 frags, 95043 blocks, 1.2% fragmentation) ***** FILE SYSTEM MARKED CLEAN ***** ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 743431 files, 89378909 used, 28935072 free (25720 frags, 3613669 blocks, 0.0% fragmentation) ***** FILE SYSTEM MARKED CLEAN ***** ** /dev/ada0s1d ** Last Mounted on /usr ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 425676 files, 8260248 used, 1893951 free (245911 frags, 206005 blocks, 2.4% fragmentation) ***** FILE SYSTEM MARKED CLEAN ***** ** /dev/ada0s1e ** SU+J Recovering /dev/ada0s1e Journal timestamp does not match fs mount time ** Skipping journal, falling through to full fsck ** Last Mounted on /var ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 269191 files, 9597352 used, 15791165 free (146813 frags, 1955544 blocks, 0.6% fragmentation) ***** FILE SYSTEM IS CLEAN ***** Mounting local filesystems:. ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/arduino/lib /usr/local/lib/compat /usr/local/lib/compat/pkg /usr/local/lib/e2fsprogs /usr/local/lib/gcc7 /usr/local/lib/gcc8 /usr/local/lib/mysql /usr/local/lib/nss /usr/local/lib/perl5/5.28/mach/CORE /usr/local/lib/pth /usr/local/lib/qt4 /usr/local/lib/qt5 /usr/local/llvm37/lib /usr/local/llvm39/lib a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout ldconfig: aout is not supported Setting hostname: x4.osted.lan. Setting up harvesting: [UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,[NET_ETHER],NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED Feeding entropy: . lo0: link state changed to UP re0: link state changed to DOWN Starting Network: lo0 re0. lo0: flags=8049 metric 0 mtu 16384 options=680003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=21 re0: flags=8843 metric 0 mtu 1500 options=8209b ether 00:21:85:15:33:7d inet 192.168.1.101 netmask 0xffffff00 broadcast 192.168.1.255 media: Ethernet autoselect (none) status: no carrier nd6 options=29 Starting devd. re0: link state changed to UP add host 127.0.0.1: gateway lo0 add net default: gateway 192.168.1.254 add host ::1: gateway lo0 fib 0: route already in table add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Starting nfsuserd. Creating and/or trimming log files. Starting nfscbd. Recovering vi editor sessions:. Updating motd:. Clearing /tmp (X related). Starting syslogd. Starting watchdogd. Jan 19 13:53:31 x4 savecore[619]: reboot Local package initialization: backup. Starting rpcbind. NFS access cache time=60 Starting mountd. Starting nfsd. Starting statd. Starting lockd. Mounting late filesystems:. Starting ntpd. Starting powerd. Starting default mousedmoused: unable to open /dev/psm0: No such file or directory . Starting smartd. Jan 19 13:54:07 x4 smartd[753]: Device: /dev/ada0, WARNING: A firmware update for this drive may be available, Jan 19 13:54:07 x4 smartd[753]: see the following Seagate web pages: Jan 19 13:54:07 x4 smartd[753]: http://knowledge.seagate.com/articles/en_US/FAQ/207931en Jan 19 13:54:07 x4 smartd[753]: http://knowledge.seagate.com/articles/en_US/FAQ/213891en Starting inetd. Performing sanity check on sshd configuration. Starting sshd. Starting cron. Configuring vt: keymap blanktime. Starting sendmail_submit. Starting sendmail_msp_queue. Tue Jan 19 13:54 FreeBSD/i386 (x4.osted.lan) (ttyu0) login: Jan 19 13:58:44 x4 su[1017]: pho to root on /dev/pts/0 20210119 13:59:17 all (1/1): perf.sh lock order reversal: 1st 0x2415490c syncer (syncer, lockmgr) @ kern/vfs_subr.c:2524 2nd 0x23ecf32c ufs (ufs, lockmgr) @ kern/vfs_subr.c:3133 lock order ufs -> syncer established at: #0 0x10555e8 at witness_checkorder+0x3c8 #1 0xfc7e52 at lockmgr_lock_flags+0x132 #2 0x10aaa04 at vop_stdlock+0x24 #3 0x1400048 at VOP_LOCK1_APV+0x38 #4 0x10d3929 at _vn_lock+0x49 #5 0x10bd7e7 at vput_final+0x107 #6 0x10b553b at dounmount+0x3db #7 0x10b50f2 at kern_unmount+0x2b2 #8 0x10b4e33 at sys_unmount+0x13 #9 0x13ebdd8 at syscall+0x1b8 #10 0xffc033f9 at _end+0xfddce021 lock order syncer -> ufs attempted at: #0 0x1055c67 at witness_checkorder+0xa47 #1 0xfc7e52 at lockmgr_lock_flags+0x132 #2 0x127b6d9 at ffs_lock+0x89 #3 0x1400048 at VOP_LOCK1_APV+0x38 #4 0x10d3929 at _vn_lock+0x49 #5 0x10bd7e7 at vput_final+0x107 #6 0x10bd1bb at vget_finish+0x8b #7 0x10bd120 at vget+0x50 #8 0x1279a8b at ffs_sync+0x4eb #9 0x10c7a11 at sync_fsync+0xe1 #10 0x13feebd at VOP_FSYNC_APV+0x4d #11 0x10c5129 at sched_sync+0x429 #12 0xfb54a8 at fork_exit+0x68 #13 0xffc0340e at _end+0xfddce036 interrupt total irq4: uart0 424 irq6: fdc0 1 irq9: acpi0 2 irq16: hdac1 ohci0 91 irq17: ohci1 ohci3 3 irq18: ohci2 ohci4 3 irq22: ahci0 505989 cpu0:timer 2868851 cpu1:timer 2622478 cpu2:timer 2503490 cpu3:timer 2555656 irq24: hdac0 4 irq25: re0 130415 Total 11187407 KDB: stack backtrace: db_trace_self_wrapper(8,1413ea3,e0,aab4cf,9bf80e0,...) at db_trace_self_wrapper+0x28/frame 0x243af654 hardclock(1,0) at hardclock+0x1a3/frame 0x243af67c handleevents(0) at handleevents+0xf5/frame 0x243af6b4 timercb(1e34f0c,0) at timercb+0x18c/frame 0x243af6e0 lapic_handle_timer(243af708,8,28,28,f7810f0,...) at lapic_handle_timer+0x7b/frame 0x243af6fc Xtimerint() at 0xffc04923/frame 0x243af6fc --- interrupt, eip = 0x1282c30, esp = 0x243af748, ebp = 0x243af854 --- ufs_lookup_ino(27bbbd38,243af8ec,243afb6c,0) at ufs_lookup_ino+0x6d0/frame 0x243af854 ufs_lookup(243af890) at ufs_lookup+0x16/frame 0x243af86c VOP_CACHEDLOOKUP_APV(1acb2bc,243af890) at VOP_CACHEDLOOKUP_APV+0x4d/frame 0x243af880 vfs_cache_lookup(243af8dc) at vfs_cache_lookup+0x93/frame 0x243af8ac VOP_LOOKUP_APV(1acb2bc,243af8dc) at VOP_LOOKUP_APV+0x4d/frame 0x243af8c0 cache_fplookup_final_modifying(1d1696,5010003,1,1acb2bc,243af930,...) at cache_fplookup_final_modifying+0x2af/frame 0x243af900 cache_fplookup_noentry(12b689c,27bbbd38,243afb18,243afb6c,20ce900c,...) at cache_fplookup_noentry+0x99/frame 0x243af938 cache_fplookup(243afb18,243afa08,243af9f8) at cache_fplookup+0x559/frame 0x243af9b4 namei(243afb18) at namei+0x165/frame 0x243afa3c kern_funlinkat(263e1000,ffffff9c,ffbfe694,ffffff38,0,0,0,0) at kern_funlinkat+0x115/frame 0x243afbd8 sys_unlink(263e1000,263e1298,0,263e1000,1ef2,...) at sys_unlink+0x1f/frame 0x243afc00 syscall(243afce8,3b,3b,3b,ffbfe7a0,...) at syscall+0x1b8/frame 0x243afcdc Xint0x80_syscall() at 0xffc033f9/frame 0x243afcdc --- syscall (10, FreeBSD ELF32, sys_unlink), eip = 0xffc01d18, esp = 0xffc0afe8, ebp = 0xffbfe73c --- KDB: enter: watchdog timeout [ thread pid 77066 tid 100220 ] Stopped at kdb_enter+0x35: movl $0,kdb_why db> run pho db:0:pho> set $lines 20000 db:0:pho> run pho1 db:1:pho1> bt Tracing pid 77066 tid 100220 td 0x263e1000 kdb_enter(1436862,143155a) at kdb_enter+0x35/frame 0x243af64c hardclock(1,0) at hardclock+0x1b2/frame 0x243af67c handleevents(0) at handleevents+0xf5/frame 0x243af6b4 timercb(1e34f0c,0) at timercb+0x18c/frame 0x243af6e0 lapic_handle_timer(243af708,8,28,28,f7810f0,...) at lapic_handle_timer+0x7b/frame 0x243af6fc Xtimerint() at 0xffc04923/frame 0x243af6fc --- interrupt, eip = 0x1282c30, esp = 0x243af748, ebp = 0x243af854 --- ufs_lookup_ino(27bbbd38,243af8ec,243afb6c,0) at ufs_lookup_ino+0x6d0/frame 0x243af854 ufs_lookup(243af890) at ufs_lookup+0x16/frame 0x243af86c VOP_CACHEDLOOKUP_APV(1acb2bc,243af890) at VOP_CACHEDLOOKUP_APV+0x4d/frame 0x243af880 vfs_cache_lookup(243af8dc) at vfs_cache_lookup+0x93/frame 0x243af8ac VOP_LOOKUP_APV(1acb2bc,243af8dc) at VOP_LOOKUP_APV+0x4d/frame 0x243af8c0 cache_fplookup_final_modifying(1d1696,5010003,1,1acb2bc,243af930,...) at cache_fplookup_final_modifying+0x2af/frame 0x243af900 cache_fplookup_noentry(12b689c,27bbbd38,243afb18,243afb6c,20ce900c,...) at cache_fplookup_noentry+0x99/frame 0x243af938 cache_fplookup(243afb18,243afa08,243af9f8) at cache_fplookup+0x559/frame 0x243af9b4 namei(243afb18) at namei+0x165/frame 0x243afa3c kern_funlinkat(263e1000,ffffff9c,ffbfe694,ffffff38,0,0,0,0) at kern_funlinkat+0x115/frame 0x243afbd8 sys_unlink(263e1000,263e1298,0,263e1000,1ef2,...) at sys_unlink+0x1f/frame 0x243afc00 syscall(243afce8,3b,3b,3b,ffbfe7a0,...) at syscall+0x1b8/frame 0x243afcdc Xint0x80_syscall() at 0xffc033f9/frame 0x243afcdc --- syscall (10, FreeBSD ELF32, sys_unlink), eip = 0xffc01d18, esp = 0xffc0afe8, ebp = 0xffbfe73c --- db:1:pho1> show allpcpu Current CPU: 1 cpuid = 0 dynamic pcpu = 0x816500 curthread = 0x9bd0500: pid 11 tid 100003 critnest 1 "idle: cpu0" curpcb = 0xb4b3d40 fpcurthread = none idlethread = 0x9bd0500: tid 100003 "idle: cpu0" APIC ID = 0 currentldt = 0x50 trampstk = 0xffc07ff0 kesp0 = 0xb4b3d30 common_tssp = 0xffc01260 tlb gen = 2857993 curvnet = 0 spin locks held: cpuid = 1 dynamic pcpu = 0x9d0f500 curthread = 0x263e1000: pid 77066 tid 100220 critnest 1 "perf" curpcb = 0x243afd40 fpcurthread = 0x263e1000: pid 77066 "perf" idlethread = 0x9bd0180: tid 100004 "idle: cpu1" APIC ID = 1 currentldt = 0x50 trampstk = 0xffc0aff0 kesp0 = 0x243afd30 common_tssp = 0xffc012c8 tlb gen = 2857993 curvnet = 0 spin locks held: cpuid = 2 dynamic pcpu = 0x9d15500 curthread = 0x9bcfe00: pid 11 tid 100005 critnest 1 "idle: cpu2" curpcb = 0xb4a9d40 fpcurthread = none idlethread = 0x9bcfe00: tid 100005 "idle: cpu2" APIC ID = 2 currentldt = 0x50 trampstk = 0xffc0bff0 kesp0 = 0xb4a9d30 common_tssp = 0xffc01330 tlb gen = 2857993 curvnet = 0 spin locks held: cpuid = 3 dynamic pcpu = 0x7f73500 curthread = 0x9bcfa80: pid 11 tid 100006 critnest 1 "idle: cpu3" curpcb = 0xb4a4d40 fpcurthread = none idlethread = 0x9bcfa80: tid 100006 "idle: cpu3" APIC ID = 3 currentldt = 0x50 trampstk = 0xffc0dff0 kesp0 = 0xb4a4d30 common_tssp = 0xffc01398 tlb gen = 2857991 curvnet = 0 spin locks held: db:1:pho1> show alllocks Process 77066 (perf) thread 0x263e1000 (100220) exclusive lockmgr bufwait (bufwait) r = 0 (0xb67d15c) locked @ kern/vfs_bio.c:3895 exclusive lockmgr ufs (ufs) r = 0 (0x27bbbd74) locked @ kern/vfs_cache.c:4183 Process 592 (syslogd) thread 0x18d07000 (100123) exclusive lockmgr ufs (ufs) r = 0 (0x235b11b4) locked @ kern/vfs_syscalls.c:3498 db:1:pho1> show lockedvnods Locked vnodes vnode 0x235b1178: type VREG usecount 1, writecount 1, refcount 4 seqc users 0 hold count flags () flags (VIRF_PGREAD|VMP_LAZYLIST) v_object 0x2063fb88 ref 0 pages 8 cleanbuf 2 dirtybuf 0 lock type ufs: EXCL by thread 0x18d07000 (pid 592, syslogd, tid 100123) #0 0xfc7eb2 at lockmgr_lock_flags+0x192 #1 0x127b6d9 at ffs_lock+0x89 #2 0x1400048 at VOP_LOCK1_APV+0x38 #3 0x10d3929 at _vn_lock+0x49 #4 0x10cf476 at kern_fsync+0x126 #5 0x10cf592 at sys_fsync+0x12 #6 0x13ebdd8 at syscall+0x1b8 #7 0xffc033f9 at _end+0xfddce021 nlink=1, effnlink=1, size=272726, extsize 0 generation=323d34d0, uid=0, gid=0, flags=0x0 ino 6052890, on dev ada0s1e vnode 0x27bbbd38: type VDIR usecount 5, writecount 0, refcount 183 seqc users 0 mountedhere 0 hold count flags () flags (VV_ROOT|VMP_LAZYLIST) v_object 0x2abe470c ref 0 pages 1576 cleanbuf 180 dirtybuf 0 lock type ufs: EXCL by thread 0x263e1000 (pid 77066, perf, tid 100220) with exclusive waiters pending with shared waiters pending #0 0xfc8818 at lockmgr_xlock_hard+0x3c8 #1 0xfc7ed3 at lockmgr_lock_flags+0x1b3 #2 0x127b6d9 at ffs_lock+0x89 #3 0x1400048 at VOP_LOCK1_APV+0x38 #4 0x10d3929 at _vn_lock+0x49 #5 0x10a7c0c at cache_fplookup_final_modifying+0x23c #6 0x10a6eb9 at cache_fplookup_noentry+0x99 #7 0x10a34d9 at cache_fplookup+0x559 #8 0x10af905 at namei+0x165 #9 0x10cbae5 at kern_funlinkat+0x115 #10 0x10cb9bf at sys_unlink+0x1f #11 0x13ebdd8 at syscall+0x1b8 #12 0xffc033f9 at _end+0xfddce021 nlink=3, effnlink=3, size=6390272, extsize 0 generation=47126e1c, uid=0, gid=0, flags=0x0 ino 2, on dev md10 db:1:pho1> show allchains chain 1: thread 100286 (pid 40916, sleep) is sleeping on 0x1d7e2cf "nanslp" chain 2: thread 100248 (pid 83021, ls) is blocked on lockmgr ufsEXCL thread 100220 (pid 77066, perf) is running on CPU 1 chain 3: thread 100251 (pid 83018, sh) is sleeping on 0x2b13b690 "wait" chain 4: thread 100186 (pid 77065, perf) is blocked on lockmgr ufsEXCL thread 100220 (pid 77066, perf) is running on CPU 1 chain 5: thread 100213 (pid 2437, md10) is sleeping on 0x2ac47000 "mdwait" chain 6: thread 100144 (pid 2291, tee) is sleeping on 0x2ac5e1a0 "piperd" chain 7: thread 100106 (pid 2290, sh) is sleeping on 0x18df0aa8 "pause" chain 8: thread 100215 (pid 2289, sh) is sleeping on 0x2abf03b0 "wait" chain 9: thread 100208 (pid 1200, sh) is sleeping on 0x20ce8340 "piperd" chain 10: thread 100214 (pid 1198, awk) is sleeping on 0x20ce8000 "piperd" chain 11: thread 100203 (pid 1196, sh) is sleeping on 0x249ded20 "wait" chain 12: thread 100204 (pid 1144, sh) is sleeping on 0x249de9d8 "wait" chain 13: thread 100211 (pid 1056, sh) is sleeping on 0x249de348 "wait" chain 14: thread 100198 (pid 1019, bash) is sleeping on 0x263e0a40 "wait" chain 15: thread 100200 (pid 1017, csh) is sleeping on 0x263e0418 "pause" chain 16: thread 100201 (pid 1016, su) is blocked on lockmgr a#$EXCL KDB: reentering KDB: stack backtrace: db_trace_self_wrapper(243af188,13eac3b,243af1a0,0,10,...) at db_trace_self_wrapper+0x28/frame 0x243af0c0 kdb_reenter(243af1a0,0,10,0,263df000,...) at kdb_reenter+0x29/frame 0x243af0c8 trap(243af194,8,28,28,14f32c0,...) at trap+0xeb/frame 0x243af188 calltrap() at 0xffc0319f/frame 0x243af188 --- trap 0xc, eip = 0x104e863, esp = 0x243af1d4, ebp = 0x243af1e8 --- print_lockchain(0,1a99a30,0,243af2b0,a17dba,...) at print_lockchain+0x33/frame 0x243af1e8 db_show_allchains(10368f5,0,ffffffff,243af228) at db_show_allchains+0x6f/frame 0x243af1fc db_command(1b06f6b) at db_command+0x26a/frame 0x243af2b0 db_script_exec(1432230,0,8,243af330) at db_script_exec+0x196/frame 0x243af304 db_command(1b06eca) at db_command+0x26a/frame 0x243af3b8 db_script_exec(bdbdaf,0,243af454,243af438) at db_script_exec+0x196/frame 0x243af40c db_command(243af50c,a1ac2f,263e1000,243af4c8,243af50c,...) at db_command+0x26a/frame 0x243af4c0 db_command_loop(263e1000,243af4c8,243af50c,0,243af4d0,...) at db_command_loop+0x3c/frame 0x243af4c8 db_trap(3,0) at db_trap+0xef/frame 0x243af50c kdb_trap(3,0,243af608) at kdb_trap+0x109/frame 0x243af534 trap(243af608,8,28,28,0,...) at trap+0x382/frame 0x243af5fc calltrap() at 0xffc0319f/frame 0x243af5fc --- trap 0x3, eip = 0x10368f5, esp = 0x243af648, ebp = 0x243af64c --- kdb_enter(1436862,143155a) at kdb_enter+0x35/frame 0x243af64c hardclock(1,0) at hardclock+0x1b2/frame 0x243af67c handleevents(0) at handleevents+0xf5/frame 0x243af6b4 timercb(1e34f0c,0) at timercb+0x18c/frame 0x243af6e0 lapic_handle_timer(243af708,8,28,28,f7810f0,...) at lapic_handle_timer+0x7b/frame 0x243af6fc Xtimerint() at 0xffc04923/frame 0x243af6fc --- interrupt, eip = 0x1282c30, esp = 0x243af748, ebp = 0x243af854 --- ufs_lookup_ino(27bbbd38,243af8ec,243afb6c,0) at ufs_lookup_ino+0x6d0/frame 0x243af854 ufs_lookup(243af890) at ufs_lookup+0x16/frame 0x243af86c VOP_CACHEDLOOKUP_APV(1acb2bc,243af890) at VOP_CACHEDLOOKUP_APV+0x4d/frame 0x243af880 vfs_cache_lookup(243af8dc) at vfs_cache_lookup+0x93/frame 0x243af8ac VOP_LOOKUP_APV(1acb2bc,243af8dc) at VOP_LOOKUP_APV+0x4d/frame 0x243af8c0 cache_fplookup_final_modifying(1d1696,5010003,1,1acb2bc,243af930,...) at cache_fplookup_final_modifying+0x2af/frame 0x243af900 cache_fplookup_noentry(12b689c,27bbbd38,243afb18,243afb6c,20ce900c,...) at cache_fplookup_noentry+0x99/frame 0x243af938 cache_fplookup(243afb18,243afa08,243af9f8) at cache_fplookup+0x559/frame 0x243af9b4 namei(243afb18) at namei+0x165/frame 0x243afa3c kern_funlinkat(263e1000,ffffff9c,ffbfe694,ffffff38,0,0,0,0) at kern_funlinkat+0x115/frame 0x243afbd8 sys_unlink(263e1000,263e1298,0,263e1000,1ef2,...) at sys_unlink+0x1f/frame 0x243afc00 syscall(243afce8,3b,3b,3b,ffbfe7a0,...) at syscall+0x1b8/frame 0x243afcdc Xint0x80_syscall() at 0xffc033f9/frame 0x243afcdc --- syscall (10, FreeBSD ELF32, sys_unlink), eip = 0xffc01d18, esp = 0xffc0afe8, ebp = 0xffbfe73c --- Script command 'show allchains' returned error db:1:pho1> show mount 0x18da9500 /dev/ada0s1a on / (ufs) 0x18c64500 devfs on /dev (devfs) 0x18c63a80 /dev/ada0s1f on /home (ufs) 0x18c63000 /dev/label/tmp on /tmp (ufs) 0x20a41500 /dev/ada0s1d on /usr (ufs) 0x20a40a80 /dev/ada0s1e on /var (ufs) 0x20a40000 procfs on /proc (procfs) 0x27c67500 /dev/md10 on /mnt (ufs) More info: show mount db:1:pho1> show bufqueues bqempty: 203 Buf domain 0 freebufs 764 lofreebufs 311 hifreebufs 466 bufspace 89391104 maxbufspace 94683136 hibufspace 94027776 lobufspace 89326372 bufspacethresh 91677074 numdirtybuffers 41 lodirtybuffers 732 hidirtybuffers 1464 dirtybufthresh 1317 cleanq count 4935 (87291904) dirtyq count 42 (1159168) wakeup 0 lim 28 CPU 14, 4, 4, 16, Locked buffers: 2 space 49152 Total buffers: 5779 space 89391104 db:0:pho> ps pid ppid pgrp uid state wmesg wchan cmd 40916 1196 1144 0 S nanslp 0x1d7e2cf sleep 83021 83018 601 0 D ufs 0x27bbbd74 ls 83018 601 601 0 S wait 0x2b13b690 sh 77066 2290 1056 0 R+ CPU 1 perf 77065 2290 1056 0 D+ ufs 0x27bbbd74 perf 2437 0 0 0 DL mdwait 0x2ac47000 [md10] 2291 1056 1056 0 S+ piperd 0x2ac5e1a0 tee 2290 2289 1056 0 S+ pause 0x18df0aa8 sh 2289 1056 1056 0 S+ wait 0x2abf03b0 sh 1200 1144 1144 0 S piperd 0x20ce8340 sh 1198 1144 1144 0 S piperd 0x20ce8000 awk 1196 1144 1144 0 S wait 0x249ded20 sh 1144 1 1144 0 Ss wait 0x249de9d8 sh 1056 1019 1056 0 S+ wait 0x249de348 sh 1019 1017 1019 0 S+ wait 0x263e0a40 bash 1017 1016 1017 0 S+ pause 0x263e0418 csh 1016 1007 1016 1001 S+ wait 0x249df3b0 su 1007 1006 1007 1001 Ss+ wait 0x23611d20 bash 1006 1004 1004 1001 S select 0x1889d424 sshd 1004 803 1004 0 Ss select 0x21048d64 sshd 846 1 846 0 Ss+ ttyin 0x182c4c74 getty 845 1 845 0 Ss+ ttyin 0x18be3c74 getty 844 1 844 0 Ss+ ttyin 0x18b3d074 getty 843 1 843 0 Ss+ ttyin 0x18b3d474 getty 842 1 842 0 Ss+ ttyin 0x18b3d874 getty 841 1 841 0 Ss+ ttyin 0x18b3dc74 getty 840 1 840 0 Ss+ ttyin 0x18879074 getty 839 1 839 0 Ss+ ttyin 0x18879474 getty 838 1 838 0 Ss+ ttyin 0x18879874 getty 825 1 825 25 Ss pause 0x188ac3b0 sendmail 819 1 819 0 Ss select 0x18db8da4 sendmail 806 1 806 0 Ss nanslp 0x1d7e2cf cron 803 1 803 0 Ss select 0x21048da4 sshd 779 1 779 0 Ss select 0x18db8f64 inetd 756 1 755 0 S nanslp 0x1d7e2ce smartd 713 1 713 0 Ss select 0x21048e64 powerd 710 1 710 0 Ss (threaded) ntpd 100133 S select 0x18db8e24 ntpd 689 1 689 0 Ss rpcsvc 0x1889a8d0 NLM: master 686 1 686 0 Ss select 0x1889d4a4 rpc.statd 685 682 682 0 S (threaded) nfsd 100111 S rpcsvc 0x1889a950 nfsd: master 100154 S rpcsvc 0x12b916d0 nfsd: service 100155 S rpcsvc 0x18d5ced0 nfsd: service 100156 S rpcsvc 0x18d5ce50 nfsd: service 100157 S rpcsvc 0x12b91650 nfsd: service 100158 S rpcsvc 0x18d5cdd0 nfsd: service 100159 S rpcsvc 0x12b915d0 nfsd: service 100160 S rpcsvc 0x18d5cd50 nfsd: service 100161 S rpcsvc 0x12b91550 nfsd: service 100162 S rpcsvc 0x18d5ccd0 nfsd: service 100163 S rpcsvc 0x18d5cc50 nfsd: service 100164 S rpcsvc 0x12b914d0 nfsd: service 100165 S rpcsvc 0x18d5cbd0 nfsd: service 100166 S rpcsvc 0x12b91450 nfsd: service 100167 S rpcsvc 0x18d5cb50 nfsd: service 100168 S rpcsvc 0x12b913d0 nfsd: service 100169 S rpcsvc 0x18d5cad0 nfsd: service 100170 S rpcsvc 0x12b91350 nfsd: service 100171 S rpcsvc 0x12b912d0 nfsd: service 100172 S rpcsvc 0x18d5ca50 nfsd: service 100173 S rpcsvc 0x12b91250 nfsd: service 100174 S rpcsvc 0x18d5c9d0 nfsd: service 100175 S rpcsvc 0x18d5c950 nfsd: service 100176 S rpcsvc 0x12b911d0 nfsd: service 100177 S rpcsvc 0x18d5c8d0 nfsd: service 100178 S rpcsvc 0x12b91150 nfsd: service 100179 S rpcsvc 0x18d5c850 nfsd: service 100180 S rpcsvc 0x12b910d0 nfsd: service 100181 S rpcsvc 0x12b91050 nfsd: service 100182 S rpcsvc 0x18d5c7d0 nfsd: service 100183 S rpcsvc 0x18d5c750 nfsd: service 100184 S rpcsvc 0x24404f50 nfsd: service 682 1 682 0 Ss select 0x21048e24 nfsd 675 1 675 0 Ss select 0x1889d524 mountd 651 1 651 0 Ss select 0x18db8ea4 rpcbind 601 1 601 0 Ss wait 0x20cad690 watchdogd 592 1 592 0 Ds bo_wwai 0x235b1290 syslogd 550 549 549 0 S (threaded) nfscbd 100126 S rpcsvc 0x1889aa50 nfscbd: master 100134 S rpcsvc 0x12b917d0 nfscbd: service 100135 S rpcsvc 0x1889a9d0 nfscbd: service 100136 S rpcsvc 0x12b91750 nfscbd: service 549 1 549 0 Ss accept 0x20d179b0 nfscbd 466 462 462 0 S select 0x18db8fa4 nfsuserd 465 462 462 0 S select 0x21048ea4 nfsuserd 464 462 462 0 S select 0x21048ee4 nfsuserd 463 462 462 0 S select 0x18db8f24 nfsuserd 462 1 462 0 Ss pause 0x18255418 nfsuserd 393 1 393 0 Ss select 0x18c7a364 devd 24 0 0 0 DL syncer 0x1def2b4 [syncer] 23 0 0 0 DL vlruwt 0x188acd20 [vnlru] 22 0 0 0 DL (threaded) [bufdaemon] 100099 D qsleep 0x1deea04 [bufdaemon] 100100 D - 0x1a9ffc0 [bufspacedaemon-0] 100115 D sdflush 0x2080de04 [/home worker] 100116 D sdflush 0x2080dc84 [/tmp worker] 100117 D sdflush 0x2080db04 [/usr worker] 100118 D sdflush 0x2080d984 [/var worker] 21 0 0 0 DL psleep 0x1e05d70 [vmdaemon] 20 0 0 0 DL (threaded) [pagedaemon] 100097 D psleep 0x1e04dbc [dom0] 100101 D launds 0x1e04dc8 [laundry: dom0] 100102 D umarcl 0x1299a20 [uma] 19 0 0 0 DL - 0x1bccbf8 [rand_harvestq] 18 0 0 0 DL - 0x1dee564 [soaiod4] 17 0 0 0 DL - 0x1dee564 [soaiod3] 16 0 0 0 DL - 0x1dee564 [soaiod2] 9 0 0 0 DL - 0x1dee564 [soaiod1] 8 0 0 0 DL - 0x12bb2dbc [fdc0] 15 0 0 0 DL (threaded) [usb] 100049 D - 0x12bdeaac [usbus0] 100050 D - 0x12bdeadc [usbus0] 100051 D - 0x12bdeb0c [usbus0] 100052 D - 0x12bdeb3c [usbus0] 100053 D - 0x12bdeb6c [usbus0] 100055 D - 0x1825caac [usbus1] 100056 D - 0x1825cadc [usbus1] 100057 D - 0x1825cb0c [usbus1] 100058 D - 0x1825cb3c [usbus1] 100059 D - 0x1825cb6c [usbus1] 100061 D - 0x18266aac [usbus2] 100062 D - 0x18266adc [usbus2] 100063 D - 0x18266b0c [usbus2] 100064 D - 0x18266b3c [usbus2] 100065 D - 0x18266b6c [usbus2] 100066 D - 0x18271aac [usbus3] 100067 D - 0x18271adc [usbus3] 100068 D - 0x18271b0c [usbus3] 100069 D - 0x18271b3c [usbus3] 100070 D - 0x18271b6c [usbus3] 100071 D - 0x18277aac [usbus4] 100072 D - 0x18277adc [usbus4] 100073 D - 0x18277b0c [usbus4] 100074 D - 0x18277b3c [usbus4] 100075 D - 0x18277b6c [usbus4] 100077 D - 0x12be3c84 [usbus5] 100078 D - 0x12be3cb4 [usbus5] 100079 D - 0x12be3ce4 [usbus5] 100080 D - 0x12be3d14 [usbus5] 100081 D - 0x12be3d44 [usbus5] 7 0 0 0 DL (threaded) [cam] 100044 D - 0x1b03f40 [doneq0] 100096 D - 0x1b03e6c [scanner] 6 0 0 0 DL crypto_ 0xb7fb9c4 [crypto returns 3] 5 0 0 0 DL crypto_ 0xb7fb98c [crypto returns 2] 4 0 0 0 DL crypto_ 0xb7fb954 [crypto returns 1] 3 0 0 0 DL crypto_ 0xb7fb91c [crypto returns 0] 2 0 0 0 DL crypto_ 0x1e03520 [crypto] 14 0 0 0 DL seqstat 0x9a4bc4c [sequencer 00] 13 0 0 0 DL (threaded) [geom] 100031 D - 0x1d6bb88 [g_event] 100032 D - 0x1d6bb8c [g_up] 100033 D - 0x1d6bb90 [g_down] 12 0 0 0 WL (threaded) [intr] 100018 I [swi6: task queue] 100020 I [swi6: Giant taskq] 100023 I [swi5: fast taskq] 100024 I [swi1: netisr 0] 100025 I [swi3: vm] 100026 I [swi4: clock (0)] 100027 I [swi4: clock (1)] 100028 I [swi4: clock (2)] 100029 I [swi4: clock (3)] 100045 I [irq24: hdac0] 100046 I [irq25: re0] 100047 I [irq22: ahci0] 100048 I [irq16: hdac1 ohci0] 100054 I [irq17: ohci1 ohci3] 100060 I [irq18: ohci2 ohci4] 100076 I [irq19: ehci0] 100082 I [irq14: ata0] 100083 I [swi0: uart] 100085 I [irq1: atkbd0] 11 0 0 0 RL (threaded) [idle] 100003 Run CPU 0 [idle: cpu0] 100004 CanRun [idle: cpu1] 100005 Run CPU 2 [idle: cpu2] 100006 Run CPU 3 [idle: cpu3] 1 0 1 0 SLs wait 0x9bce6f8 [init] 10 0 0 0 DL audit_w 0x1e0380c [audit] 0 0 0 0 DLs (threaded) [kernel] 100000 D swapin 0x1d6c05c [swapper] 100007 D - 0x9bfde00 [if_config_tqg_0] 100008 D - 0x9bfdd00 [softirq_0] 100009 D - 0x9bfdc00 [softirq_1] 100010 D - 0x9bfdb00 [softirq_2] 100011 D - 0x9bfda00 [softirq_3] 100012 D - 0x9bfd900 [if_io_tqg_0] 100013 D - 0x9bfd800 [if_io_tqg_1] 100014 D - 0x9bfd700 [if_io_tqg_2] 100015 D - 0x9bfd600 [if_io_tqg_3] 100016 D - 0x9bfd500 [kqueue_ctx taskq] 100017 D - 0x9bfd400 [inm_free taskq] 100019 D - 0x9bfd200 [in6m_free taskq] 100021 D - 0x9bfd000 [aiod_kick taskq] 100022 D - 0x12a58e00 [thread taskq] 100030 D - 0x12a58c00 [firmware taskq] 100035 D - 0x12a58b00 [crypto_0] 100036 D - 0x12a58b00 [crypto_1] 100037 D - 0x12a58b00 [crypto_2] 100038 D - 0x12a58b00 [crypto_3] 100086 D - 0x18224900 [mca taskq] 100091 D - 0x188a3e00 [acpi_task_0] 100092 D - 0x188a3e00 [acpi_task_1] 100093 D - 0x188a3e00 [acpi_task_2] 100095 D - 0x12a58a00 [CAM taskq] db:0:pho> show freepages DOMAIN: 0 FREE LIST 0: ORDER (SIZE) | NUMBER | POOL 0 -- -- -- -- 09 (002048K) | 000000 08 (001024K) | 000000 07 (000512K) | 000000 06 (000256K) | 000000 05 (000128K) | 000000 04 (000064K) | 000000 03 (000032K) | 000000 02 (000016K) | 000000 01 (000008K) | 000000 00 (000004K) | 000000 FREE LIST 1: ORDER (SIZE) | NUMBER | POOL 0 -- -- -- -- 09 (002048K) | 000000 08 (001024K) | 000015 07 (000512K) | 000154 06 (000256K) | 000966 05 (000128K) | 002919 04 (000064K) | 005640 03 (000032K) | 008957 02 (000016K) | 016182 01 (000008K) | 008813 00 (000004K) | 000276 FREE LIST 2: ORDER (SIZE) | NUMBER | POOL 0 -- -- -- -- 09 (002048K) | 000000 08 (001024K) | 000000 07 (000512K) | 000000 06 (000256K) | 000001 05 (000128K) | 000000 04 (000064K) | 000002 03 (000032K) | 000003 02 (000016K) | 000004 01 (000008K) | 000004 00 (000004K) | 000002 db:0:pho> acttrace Tracing command idle pid 11 tid 100003 td 0x9bd0500 (CPU 0) cpustop_handler(b4b3bd0,b4b3bc4,13eac0b,22fe020,0,...) at cpustop_handler+0x26/frame 0xb4b3afc ipi_nmi_handler(22fe020,0,b4b3b3c,fd3bd5,9bce3b0,...) at ipi_nmi_handler+0x30/frame 0xb4b3b08 trap(b4b3bd0,8,28,28,12b94e00,...) at trap+0xbb/frame 0xb4b3bc4 calltrap() at 0xffc0319f/frame 0xb4b3bc4 --- trap 0x13, eip = 0x139f535, esp = 0xb4b3c10, ebp = 0xb4b3c10 --- acpi_cpu_c1(1413ea3,312,1ef4,c118791c,d0f,...) at acpi_cpu_c1+0x5/frame 0xb4b3c10 acpi_cpu_idle(458dffb,0) at acpi_cpu_idle+0x22f/frame 0xb4b3c40 cpu_idle_acpi(458dffb,0) at cpu_idle_acpi+0x35/frame 0xb4b3c54 cpu_idle(0) at cpu_idle+0xa3/frame 0xb4b3c70 sched_idletd(0,b4b3ce8) at sched_idletd+0x32e/frame 0xb4b3cb8 fork_exit(101fb00,0,b4b3ce8,0,0,...) at fork_exit+0x68/frame 0xb4b3cd4 fork_trampoline() at 0xffc0340e/frame 0xb4b3cd4 --- kthread start Tracing command idle pid 11 tid 100005 td 0x9bcfe00 (CPU 2) cpustop_handler(b4a9bd0,b4a9bc4,13eac0b,1d7e590,b7fd0c0,...) at cpustop_handler+0x26/frame 0xb4a9afc ipi_nmi_handler(1d7e590,b7fd0c0,b4a9b40,fd3803,9bce3b0,...) at ipi_nmi_handler+0x30/frame 0xb4a9b08 trap(b4a9bd0,8,28,28,12b94a00,...) at trap+0xbb/frame 0xb4a9bc4 calltrap() at 0xffc0319f/frame 0xb4a9bc4 --- trap 0x13, eip = 0x139f535, esp = 0xb4a9c10, ebp = 0xb4a9c10 --- acpi_cpu_c1(1413ea3,312,1ef4,b78c3b95,d0f,...) at acpi_cpu_c1+0x5/frame 0xb4a9c10 acpi_cpu_idle(2695ab86,0) at acpi_cpu_idle+0x22f/frame 0xb4a9c40 cpu_idle_acpi(2695ab86,0) at cpu_idle_acpi+0x35/frame 0xb4a9c54 cpu_idle(0) at cpu_idle+0xa3/frame 0xb4a9c70 sched_idletd(0,b4a9ce8) at sched_idletd+0x32e/frame 0xb4a9cb8 fork_exit(101fb00,0,b4a9ce8,0,0,...) at fork_exit+0x68/frame 0xb4a9cd4 fork_trampoline() at 0xffc0340e/frame 0xb4a9cd4 --- kthread start Tracing command idle pid 11 tid 100006 td 0x9bcfa80 (CPU 3) cpustop_handler(b4a4bd0,b4a4bc4,13eac0b,1d7e690,9bcfa80,...) at cpustop_handler+0x26/frame 0xb4a4afc ipi_nmi_handler(1d7e690,9bcfa80,b4a4b40,fd3803,9bce3b0,...) at ipi_nmi_handler+0x30/frame 0xb4a4b08 trap(b4a4bd0,8,28,28,12b94800,...) at trap+0xbb/frame 0xb4a4bc4 calltrap() at 0xffc0319f/frame 0xb4a4bc4 --- trap 0x13, eip = 0x139f535, esp = 0xb4a4c10, ebp = 0xb4a4c10 --- acpi_cpu_c1(1413ea3,312,1ef4,ac78f334,d0f,...) at acpi_cpu_c1+0x5/frame 0xb4a4c10 acpi_cpu_idle(7f49422a,0) at acpi_cpu_idle+0x22f/frame 0xb4a4c40 cpu_idle_acpi(7f49422a,0) at cpu_idle_acpi+0x35/frame 0xb4a4c54 cpu_idle(0) at cpu_idle+0xa3/frame 0xb4a4c70 sched_idletd(0,b4a4ce8) at sched_idletd+0x32e/frame 0xb4a4cb8 fork_exit(101fb00,0,b4a4ce8,0,0,...) at fork_exit+0x68/frame 0xb4a4cd4 fork_trampoline() at 0xffc0340e/frame 0xb4a4cd4 --- kthread start Tracing command perf pid 77066 tid 100220 td 0x263e1000 (CPU 1) kdb_enter(1436862,143155a) at kdb_enter+0x35/frame 0x243af64c hardclock(1,0) at hardclock+0x1b2/frame 0x243af67c handleevents(0) at handleevents+0xf5/frame 0x243af6b4 timercb(1e34f0c,0) at timercb+0x18c/frame 0x243af6e0 lapic_handle_timer(243af708,8,28,28,f7810f0,...) at lapic_handle_timer+0x7b/frame 0x243af6fc Xtimerint() at 0xffc04923/frame 0x243af6fc --- interrupt, eip = 0x1282c30, esp = 0x243af748, ebp = 0x243af854 --- ufs_lookup_ino(27bbbd38,243af8ec,243afb6c,0) at ufs_lookup_ino+0x6d0/frame 0x243af854 ufs_lookup(243af890) at ufs_lookup+0x16/frame 0x243af86c VOP_CACHEDLOOKUP_APV(1acb2bc,243af890) at VOP_CACHEDLOOKUP_APV+0x4d/frame 0x243af880 vfs_cache_lookup(243af8dc) at vfs_cache_lookup+0x93/frame 0x243af8ac VOP_LOOKUP_APV(1acb2bc,243af8dc) at VOP_LOOKUP_APV+0x4d/frame 0x243af8c0 cache_fplookup_final_modifying(1d1696,5010003,1,1acb2bc,243af930,...) at cache_fplookup_final_modifying+0x2af/frame 0x243af900 cache_fplookup_noentry(12b689c,27bbbd38,243afb18,243afb6c,20ce900c,...) at cache_fplookup_noentry+0x99/frame 0x243af938 cache_fplookup(243afb18,243afa08,243af9f8) at cache_fplookup+0x559/frame 0x243af9b4 namei(243afb18) at namei+0x165/frame 0x243afa3c kern_funlinkat(263e1000,ffffff9c,ffbfe694,ffffff38,0,0,0,0) at kern_funlinkat+0x115/frame 0x243afbd8 sys_unlink(263e1000,263e1298,0,263e1000,1ef2,...) at sys_unlink+0x1f/frame 0x243afc00 syscall(243afce8,3b,3b,3b,ffbfe7a0,...) at syscall+0x1b8/frame 0x243afcdc Xint0x80_syscall() at 0xffc033f9/frame 0x243afcdc --- syscall (10, FreeBSD ELF32, sys_unlink), eip = 0xffc01d18, esp = 0xffc0afe8, ebp = 0xffbfe73c --- db:0:pho> alltrace Tracing command kernel pid 0 tid 100000 td 0x1d6c3c0 sched_switch(1d6c3c0,104) at sched_switch+0x4c0/frame 0x22f9c18 mi_switch(104) at mi_switch+0x1c4/frame 0x22f9c4c sleepq_switch(0,1d6c05c,22f9ccc,1d6c3c0,fffff479,...) at sleepq_switch+0xeb/frame 0x22f9c6c sleepq_timedwait(1d6c05c,54,246,0,40000,...) at sleepq_timedwait+0x3c/frame 0x22f9c8c _sleep(1d6c05c,0,54,151ce01,fffff470,9,0,0,100) at _sleep+0x234/frame 0x22f9ccc swapper(0,0,0,1ae6e08) at swapper+0x6d/frame 0x22f9d14 mi_startup() at mi_startup+0x2c6/frame 0x22f9d38 btext() at btext+0x5f Tracing command kernel pid 0 tid 100007 td 0x9bcf700 sched_switch(9bcf700,104) at sched_switch+0x4c0/frame 0xb49fbf8 mi_switch(104) at mi_switch+0x1c4/frame 0xb49fc2c sleepq_switch(9bfde00,0) at sleepq_switch+0xeb/frame 0xb49fc4c msleep_spin_sbt(9bfde00,9bfde40,14488cd,0,0,0,0,100) at msleep_spin_sbt+0x132/frame 0xb49fc84 gtaskqueue_thread_loop(9bfcc04,b49fce8) at gtaskqueue_thread_loop+0xfc/frame 0xb49fcb8 fork_exit(10357a0,9bfcc04,b49fce8,0,0,...) at fork_exit+0x68/frame 0xb49fcd4 fork_trampoline() at 0xffc0340e/frame 0xb49fcd4 --- kthread start Tracing command kernel pid 0 tid 100008 td 0x9bcf380 sched_switch(9bcf380,104) at sched_switch+0x4c0/frame 0xb49abf8 mi_switch(104) at mi_switch+0x1c4/frame 0xb49ac2c sleepq_switch(9bfdd00,0) at sleepq_switch+0xeb/frame 0xb49ac4c msleep_spin_sbt(9bfdd00,9bfdd40,14488cd,0,0,0,0,100) at msleep_spin_sbt+0x132/frame 0xb49ac84 gtaskqueue_thread_loop(9bfc804,b49ace8) at gtaskqueue_thread_loop+0xfc/frame 0xb49acb8 fork_exit(10357a0,9bfc804,b49ace8,0,0,...) at fork_exit+0x68/frame 0xb49acd4 fork_trampoline() at 0xffc0340e/frame 0xb49acd4 --- kthread start Tracing command kernel pid 0 tid 100009 td 0x9bcf000 sched_switch(9bcf000,104) at sched_switch+0x4c0/frame 0x128f4bf8 mi_switch(104) at mi_switch+0x1c4/frame 0x128f4c2c sleepq_switch(9bfdc00,0) at sleepq_switch+0xeb/frame 0x128f4c4c msleep_spin_sbt(9bfdc00,9bfdc40,14488cd,0,0,0,0,100) at msleep_spin_sbt+0x132/frame 0x128f4c84 gtaskqueue_thread_loop(9bfc814,128f4ce8) at gtaskqueue_thread_loop+0xfc/frame 0x128f4cb8 fork_exit(10357a0,9bfc814,128f4ce8,0,0,...) at fork_exit+0x68/frame 0x128f4cd4 fork_trampoline() at 0xffc0340e/frame 0x128f4cd4 --- kthread start Tracing command kernel pid 0 tid 100010 td 0x9bffc00 sched_switch(9bffc00,104) at sched_switch+0x4c0/frame 0x128efbf8 mi_switch(104) at mi_switch+0x1c4/frame 0x128efc2c sleepq_switch(9bfdb00,0) at sleepq_switch+0xeb/frame 0x128efc4c msleep_spin_sbt(9bfdb00,9bfdb40,14488cd,0,0,0,0,100) at msleep_spin_sbt+0x132/frame 0x128efc84 gtaskqueue_thread_loop(9bfc824,128efce8) at gtaskqueue_thread_loop+0xfc/frame 0x128efcb8 fork_exit(10357a0,9bfc824,128efce8,0,0,...) at fork_exit+0x68/frame 0x128efcd4 fork_trampoline() at 0xffc0340e/frame 0x128efcd4 --- kthread start Tracing command kernel pid 0 tid 100011 td 0x9bff880 sched_switch(9bff880,104) at sched_switch+0x4c0/frame 0x128eabf8 mi_switch(104) at mi_switch+0x1c4/frame 0x128eac2c sleepq_switch(9bfda00,0) at sleepq_switch+0xeb/frame 0x128eac4c msleep_spin_sbt(9bfda00,9bfda40,14488cd,0,0,0,0,100) at msleep_spin_sbt+0x132/frame 0x128eac84 gtaskqueue_thread_loop(9bfc834,128eace8) at gtaskqueue_thread_loop+0xfc/frame 0x128eacb8 fork_exit(10357a0,9bfc834,128eace8,0,0,...) at fork_exit+0x68/frame 0x128eacd4 fork_trampoline() at 0xffc0340e/frame 0x128eacd4 --- kthread start Tracing command kernel pid 0 tid 100012 td 0x9bff500 sched_switch(9bff500,104) at sched_switch+0x4c0/frame 0x128e5bf8 mi_switch(104) at mi_switch+0x1c4/frame 0x128e5c2c sleepq_switch(9bfd900,0) at sleepq_switch+0xeb/frame 0x128e5c4c msleep_spin_sbt(9bfd900,9bfd940,14488cd,0,0,0,0,100) at msleep_spin_sbt+0x132/frame 0x128e5c84 gtaskqueue_thread_loop(9bfc404,128e5ce8) at gtaskqueue_thread_loop+0xfc/frame 0x128e5cb8 fork_exit(10357a0,9bfc404,128e5ce8,0,0,...) at fork_exit+0x68/frame 0x128e5cd4 fork_trampoline() at 0xffc0340e/frame 0x128e5cd4 --- kthread start Tracing command kernel pid 0 tid 100013 td 0x9bff180 sched_switch(9bff180,104) at sched_switch+0x4c0/frame 0x128e0bf8 mi_switch(104) at mi_switch+0x1c4/frame 0x128e0c2c sleepq_switch(9bfd800,0) at sleepq_switch+0xeb/frame 0x128e0c4c msleep_spin_sbt(9bfd800,9bfd840,14488cd,0,0,0,0,100) at msleep_spin_sbt+0x132/frame 0x128e0c84 gtaskqueue_thread_loop(9bfc414,128e0ce8) at gtaskqueue_thread_loop+0xfc/frame 0x128e0cb8 fork_exit(10357a0,9bfc414,128e0ce8,0,0,...) at fork_exit+0x68/frame 0x128e0cd4 fork_trampoline() at 0xffc0340e/frame 0x128e0cd4 --- kthread start Tracing command kernel pid 0 tid 100014 td 0x9bfee00 sched_switch(9bfee00,104) at sched_switch+0x4c0/frame 0x128dbbf8 mi_switch(104) at mi_switch+0x1c4/frame 0x128dbc2c sleepq_switch(9bfd700,0) at sleepq_switch+0xeb/frame 0x128dbc4c msleep_spin_sbt(9bfd700,9bfd740,14488cd,0,0,0,0,100) at msleep_spin_sbt+0x132/frame 0x128dbc84 gtaskqueue_thread_loop(9bfc424,128dbce8) at gtaskqueue_thread_loop+0xfc/frame 0x128dbcb8 fork_exit(10357a0,9bfc424,128dbce8,0,0,...) at fork_exit+0x68/frame 0x128dbcd4 fork_trampoline() at 0xffc0340e/frame 0x128dbcd4 --- kthread start Tracing command kernel pid 0 tid 100015 td 0x9bfea80 sched_switch(9bfea80,104) at sched_switch+0x4c0/frame 0x128d6bf8 mi_switch(104) at mi_switch+0x1c4/frame 0x128d6c2c sleepq_switch(9bfd600,0) at sleepq_switch+0xeb/frame 0x128d6c4c msleep_spin_sbt(9bfd600,9bfd640,14488cd,0,0,0,0,100) at msleep_spin_sbt+0x132/frame 0x128d6c84 gtaskqueue_thread_loop(9bfc434,128d6ce8) at gtaskqueue_thread_loop+0xfc/frame 0x128d6cb8 fork_exit(10357a0,9bfc434,128d6ce8,0,0,...) at fork_exit+0x68/frame 0x128d6cd4 fork_trampoline() at 0xffc0340e/frame 0x128d6cd4 --- kthread start Tracing command kernel pid 0 tid 100016 td 0x9bfe700 sched_switch(9bfe700,104) at sched_switch+0x4c0/frame 0x128d1be8 mi_switch(104) at mi_switch+0x1c4/frame 0x128d1c20 sleepq_switch(9bfd500,0) at sleepq_switch+0xeb/frame 0x128d1c40 _sleep(9bfd500,9bfd540,0,14488cd,0,0,0,0,100) at _sleep+0x2bf/frame 0x128d1c80 taskqueue_thread_loop(1d6d1b8,128d1ce8) at taskqueue_thread_loop+0xd5/frame 0x128d1cb8 fork_exit(104aa20,1d6d1b8,128d1ce8,0,0,...) at fork_exit+0x68/frame 0x128d1cd4 fork_trampoline() at 0xffc0340e/frame 0x128d1cd4 --- kthread start Tracing command kernel pid 0 tid 100017 td 0x9bfe380 sched_switch(9bfe380,104) at sched_switch+0x4c0/frame 0x1291cbe8 mi_switch(104) at mi_switch+0x1c4/frame 0x1291cc20 sleepq_switch(9bfd400,0) at sleepq_switch+0xeb/frame 0x1291cc40 _sleep(9bfd400,9bfd440,0,14488cd,0,0,0,0,100) at _sleep+0x2bf/frame 0x1291cc80 taskqueue_thread_loop(1df03ec,1291cce8) at taskqueue_thread_loop+0xd5/frame 0x1291ccb8 fork_exit(104aa20,1df03ec,1291cce8,0,0,...) at fork_exit+0x68/frame 0x1291ccd4 fork_trampoline() at 0xffc0340e/frame 0x1291ccd4 --- kthread start Tracing command kernel pid 0 tid 100019 td 0x12a56c00 sched_switch(12a56c00,104) at sched_switch+0x4c0/frame 0x12912be8 mi_switch(104) at mi_switch+0x1c4/frame 0x12912c20 sleepq_switch(9bfd200,0) at sleepq_switch+0xeb/frame 0x12912c40 _sleep(9bfd200,9bfd240,0,14488cd,0,0,0,0,100) at _sleep+0x2bf/frame 0x12912c80 taskqueue_thread_loop(1e020d4,12912ce8) at taskqueue_thread_loop+0xd5/frame 0x12912cb8 fork_exit(104aa20,1e020d4,12912ce8,0,0,...) at fork_exit+0x68/frame 0x12912cd4 fork_trampoline() at 0xffc0340e/frame 0x12912cd4 --- kthread start Tracing command kernel pid 0 tid 100021 td 0x12a56500 sched_switch(12a56500,104) at sched_switch+0x4c0/frame 0x12908be8 mi_switch(104) at mi_switch+0x1c4/frame 0x12908c20 sleepq_switch(9bfd000,0) at sleepq_switch+0xeb/frame 0x12908c40 _sleep(9bfd000,9bfd040,0,14488cd,0,0,0,0,100) at _sleep+0x2bf/frame 0x12908c80 taskqueue_thread_loop(1dee998,12908ce8) at taskqueue_thread_loop+0xd5/frame 0x12908cb8 fork_exit(104aa20,1dee998,12908ce8,0,0,...) at fork_exit+0x68/frame 0x12908cd4 fork_trampoline() at 0xffc0340e/frame 0x12908cd4 --- kthread start Tracing command kernel pid 0 tid 100022 td 0x12a56180 sched_switch(12a56180,104) at sched_switch+0x4c0/frame 0x12903be8 mi_switch(104) at mi_switch+0x1c4/frame 0x12903c20 sleepq_switch(12a58e00,0) at sleepq_switch+0xeb/frame 0x12903c40 _sleep(12a58e00,12a58e40,0,14488cd,0,0,0,0,100) at _sleep+0x2bf/frame 0x12903c80 taskqueue_thread_loop(1d944d4,12903ce8) at taskqueue_thread_loop+0xd5/frame 0x12903cb8 fork_exit(104aa20,1d944d4,12903ce8,0,0,...) at fork_exit+0x68/frame 0x12903cd4 fork_trampoline() at 0xffc0340e/frame 0x12903cd4 --- kthread start Tracing command kernel pid 0 tid 100030 td 0x12a5d500 sched_switch(12a5d500,104) at sched_switch+0x4c0/frame 0x1292fbe8 mi_switch(104) at mi_switch+0x1c4/frame 0x1292fc20 sleepq_switch(12a58c00,0) at sleepq_switch+0xeb/frame 0x1292fc40 _sleep(12a58c00,12a58c40,0,14488cd,0,0,0,0,100) at _sleep+0x2bf/frame 0x1292fc80 taskqueue_thread_loop(1d8f6d4,1292fce8) at taskqueue_thread_loop+0xd5/frame 0x1292fcb8 fork_exit(104aa20,1d8f6d4,1292fce8,0,0,...) at fork_exit+0x68/frame 0x1292fcd4 fork_trampoline() at 0xffc0340e/frame 0x1292fcd4 --- kthread start Tracing command kernel pid 0 tid 100035 td 0x12a5c380 sched_switch(12a5c380,104) at sched_switch+0x4c0/frame 0x12972be8 mi_switch(104) at mi_switch+0x1c4/frame 0x12972c20 sleepq_switch(12a58b00,0) at sleepq_switch+0xeb/frame 0x12972c40 _sleep(12a58b00,12a58b40,0,14488cd,0,0,0,0,100) at _sleep+0x2bf/frame 0x12972c80 taskqueue_thread_loop(1e03550,12972ce8) at taskqueue_thread_loop+0xd5/frame 0x12972cb8 fork_exit(104aa20,1e03550,12972ce8,0,0,...) at fork_exit+0x68/frame 0x12972cd4 fork_trampoline() at 0xffc0340e/frame 0x12972cd4 --- kthread start Tracing command kernel pid 0 tid 100036 td 0x12a5c000 sched_switch(12a5c000,104) at sched_switch+0x4c0/frame 0x1296dbe8 mi_switch(104) at mi_switch+0x1c4/frame 0x1296dc20 sleepq_switch(12a58b00,0) at sleepq_switch+0xeb/frame 0x1296dc40 _sleep(12a58b00,12a58b40,0,14488cd,0,0,0,0,100) at _sleep+0x2bf/frame 0x1296dc80 taskqueue_thread_loop(1e03550,1296dce8) at taskqueue_thread_loop+0xd5/frame 0x1296dcb8 fork_exit(104aa20,1e03550,1296dce8,0,0,...) at fork_exit+0x68/frame 0x1296dcd4 fork_trampoline() at 0xffc0340e/frame 0x1296dcd4 --- kthread start Tracing command kernel pid 0 tid 100037 td 0x12ac2c00 sched_switch(12ac2c00,104) at sched_switch+0x4c0/frame 0x12968be8 mi_switch(104) at mi_switch+0x1c4/frame 0x12968c20 sleepq_switch(12a58b00,0) at sleepq_switch+0xeb/frame 0x12968c40 _sleep(12a58b00,12a58b40,0,14488cd,0,0,0,0,100) at _sleep+0x2bf/frame 0x12968c80 taskqueue_thread_loop(1e03550,12968ce8) at taskqueue_thread_loop+0xd5/frame 0x12968cb8 fork_exit(104aa20,1e03550,12968ce8,0,0,...) at fork_exit+0x68/frame 0x12968cd4 fork_trampoline() at 0xffc0340e/frame 0x12968cd4 --- kthread start Tracing command kernel pid 0 tid 100038 td 0x12ac2880 sched_switch(12ac2880,104) at sched_switch+0x4c0/frame 0x12963be8 mi_switch(104) at mi_switch+0x1c4/frame 0x12963c20 sleepq_switch(12a58b00,0) at sleepq_switch+0xeb/frame 0x12963c40 _sleep(12a58b00,12a58b40,0,14488cd,0,0,0,0,100) at _sleep+0x2bf/frame 0x12963c80 taskqueue_thread_loop(1e03550,12963ce8) at taskqueue_thread_loop+0xd5/frame 0x12963cb8 fork_exit(104aa20,1e03550,12963ce8,0,0,...) at fork_exit+0x68/frame 0x12963cd4 fork_trampoline() at 0xffc0340e/frame 0x12963cd4 --- kthread start Tracing command kernel pid 0 tid 100086 td 0x1829da80 sched_switch(1829da80,104) at sched_switch+0x4c0/frame 0xb590bf8 mi_switch(104) at mi_switch+0x1c4/frame 0xb590c2c sleepq_switch(18224900,0) at sleepq_switch+0xeb/frame 0xb590c4c msleep_spin_sbt(18224900,18224940,14488cd,0,0,0,0,100) at msleep_spin_sbt+0x132/frame 0xb590c84 taskqueue_thread_loop(1e0dfa4,b590ce8) at taskqueue_thread_loop+0xfc/frame 0xb590cb8 fork_exit(104aa20,1e0dfa4,b590ce8,0,0,...) at fork_exit+0x68/frame 0xb590cd4 fork_trampoline() at 0xffc0340e/frame 0xb590cd4 --- kthread start Tracing command kernel pid 0 tid 100091 td 0x188a6c00 sched_switch(188a6c00,104) at sched_switch+0x4c0/frame 0xb5c6bf8 mi_switch(104) at mi_switch+0x1c4/frame 0xb5c6c2c sleepq_switch(188a3e00,0) at sleepq_switch+0xeb/frame 0xb5c6c4c msleep_spin_sbt(188a3e00,188a3e40,14488cd,0,0,0,0,100) at msleep_spin_sbt+0x132/frame 0xb5c6c84 taskqueue_thread_loop(1b07e98,b5c6ce8) at taskqueue_thread_loop+0xfc/frame 0xb5c6cb8 fork_exit(104aa20,1b07e98,b5c6ce8,0,0,...) at fork_exit+0x68/frame 0xb5c6cd4 fork_trampoline() at 0xffc0340e/frame 0xb5c6cd4 --- kthread start Tracing command kernel pid 0 tid 100092 td 0x188a6880 sched_switch(188a6880,104) at sched_switch+0x4c0/frame 0xb5c1bf8 mi_switch(104) at mi_switch+0x1c4/frame 0xb5c1c2c sleepq_switch(188a3e00,0) at sleepq_switch+0xeb/frame 0xb5c1c4c msleep_spin_sbt(188a3e00,188a3e40,14488cd,0,0,0,0,100) at msleep_spin_sbt+0x132/frame 0xb5c1c84 taskqueue_thread_loop(1b07e98,b5c1ce8) at taskqueue_thread_loop+0xfc/frame 0xb5c1cb8 fork_exit(104aa20,1b07e98,b5c1ce8,0,0,...) at fork_exit+0x68/frame 0xb5c1cd4 fork_trampoline() at 0xffc0340e/frame 0xb5c1cd4 --- kthread start Tracing command kernel pid 0 tid 100093 td 0x188a6500 sched_switch(188a6500,104) at sched_switch+0x4c0/frame 0xb5bcbf8 mi_switch(104) at mi_switch+0x1c4/frame 0xb5bcc2c sleepq_switch(188a3e00,0) at sleepq_switch+0xeb/frame 0xb5bcc4c msleep_spin_sbt(188a3e00,188a3e40,14488cd,0,0,0,0,100) at msleep_spin_sbt+0x132/frame 0xb5bcc84 taskqueue_thread_loop(1b07e98,b5bcce8) at taskqueue_thread_loop+0xfc/frame 0xb5bccb8 fork_exit(104aa20,1b07e98,b5bcce8,0,0,...) at fork_exit+0x68/frame 0xb5bccd4 fork_trampoline() at 0xffc0340e/frame 0xb5bccd4 --- kthread start Tracing command kernel pid 0 tid 100095 td 0x188a5e00 sched_switch(188a5e00,104) at sched_switch+0x4c0/frame 0xb5b2be8 mi_switch(104) at mi_switch+0x1c4/frame 0xb5b2c20 sleepq_switch(12a58a00,0) at sleepq_switch+0xeb/frame 0xb5b2c40 _sleep(12a58a00,12a58a40,0,14488cd,0,0,0,0,100) at _sleep+0x2bf/frame 0xb5b2c80 taskqueue_thread_loop(1b03ef0,b5b2ce8) at taskqueue_thread_loop+0xd5/frame 0xb5b2cb8 fork_exit(104aa20,1b03ef0,b5b2ce8,0,0,...) at fork_exit+0x68/frame 0xb5b2cd4 fork_trampoline() at 0xffc0340e/frame 0xb5b2cd4 --- kthread start Tracing command init pid 1 tid 100002 td 0x9bd0880 sched_switch(9bd0880,104) at sched_switch+0x4c0/frame 0xb4b89c8 mi_switch(104) at mi_switch+0x1c4/frame 0xb4b89fc sleepq_switch(780,14fd263,6c,0,1d90454,...) at sleepq_switch+0xeb/frame 0xb4b8a1c sleepq_catch_signals(b4b8a8c,ffddc4,9bce6f8,6c,246,...) at sleepq_catch_signals+0x278/frame 0xb4b8a44 sleepq_wait_sig(9bce6f8,6c) at sleepq_wait_sig+0xe/frame 0xb4b8a4c _sleep(9bce6f8,9bce7a4,36c,1519f00,0,0,0,0,100) at _sleep+0x244/frame 0xb4b8a8c kern_wait6(9bd0880,7,0,0,b4b8be8,30,0,0) at kern_wait6+0x3b8/frame 0xb4b8ae8 sys_wait4(9bd0880,9bd0b18,9bd0880,9bd0880,b4b8cdc,...) at sys_wait4+0x73/frame 0xb4b8c00 syscall(b4b8ce8,3b,3b,3b,ffbfe8f0,...) at syscall+0x1b8/frame 0xb4b8cdc Xint0x80_syscall() at 0xffc033f9/frame 0xb4b8cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01e18, esp = 0xffc0dfe8, ebp = 0xffbfe808 --- Tracing command crypto pid 2 tid 100039 td 0x12ac2500 sched_switch(12ac2500,104) at sched_switch+0x4c0/frame 0x1295ebd8 mi_switch(104) at mi_switch+0x1c4/frame 0x1295ec0c sleepq_switch(1e03520,6c) at sleepq_switch+0xeb/frame 0x1295ec2c _sleep(1e03520,1e03508,6c,14e1c76,0,0,0,0,100) at _sleep+0x2bf/frame 0x1295ec6c crypto_proc(0,1295ece8) at crypto_proc+0x2bf/frame 0x1295ecb8 fork_exit(120cba0,0,1295ece8,0,0,...) at fork_exit+0x68/frame 0x1295ecd4 fork_trampoline() at 0xffc0340e/frame 0x1295ecd4 --- kthread start Tracing command crypto returns 0 pid 3 tid 100040 td 0x12ac2180 sched_switch(12ac2180,104) at sched_switch+0x4c0/frame 0x12959bd0 mi_switch(104) at mi_switch+0x1c4/frame 0x12959c04 sleepq_switch(b7fb91c,6c) at sleepq_switch+0xeb/frame 0x12959c24 _sleep(b7fb91c,b7fb900,6c,152fb97,0,0,0,0,100) at _sleep+0x2bf/frame 0x12959c64 crypto_ret_proc(b7fb900,12959ce8) at crypto_ret_proc+0x11e/frame 0x12959cb8 fork_exit(120d120,b7fb900,12959ce8,0,0,...) at fork_exit+0x68/frame 0x12959cd4 fork_trampoline() at 0xffc0340e/frame 0x12959cd4 --- kthread start Tracing command crypto returns 1 pid 4 tid 100041 td 0x12ac1e00 sched_switch(12ac1e00,104) at sched_switch+0x4c0/frame 0x129a4bd0 mi_switch(104) at mi_switch+0x1c4/frame 0x129a4c04 sleepq_switch(b7fb954,6c) at sleepq_switch+0xeb/frame 0x129a4c24 _sleep(b7fb954,b7fb938,6c,152fb97,0,0,0,0,100) at _sleep+0x2bf/frame 0x129a4c64 crypto_ret_proc(b7fb938,129a4ce8) at crypto_ret_proc+0x11e/frame 0x129a4cb8 fork_exit(120d120,b7fb938,129a4ce8,0,0,...) at fork_exit+0x68/frame 0x129a4cd4 fork_trampoline() at 0xffc0340e/frame 0x129a4cd4 --- kthread start Tracing command crypto returns 2 pid 5 tid 100042 td 0x12ac1a80 sched_switch(12ac1a80,104) at sched_switch+0x4c0/frame 0x1299fbd0 mi_switch(104) at mi_switch+0x1c4/frame 0x1299fc04 sleepq_switch(b7fb98c,6c) at sleepq_switch+0xeb/frame 0x1299fc24 _sleep(b7fb98c,b7fb970,6c,152fb97,0,0,0,0,100) at _sleep+0x2bf/frame 0x1299fc64 crypto_ret_proc(b7fb970,1299fce8) at crypto_ret_proc+0x11e/frame 0x1299fcb8 fork_exit(120d120,b7fb970,1299fce8,0,0,...) at fork_exit+0x68/frame 0x1299fcd4 fork_trampoline() at 0xffc0340e/frame 0x1299fcd4 --- kthread start Tracing command crypto returns 3 pid 6 tid 100043 td 0x12ac1700 sched_switch(12ac1700,104) at sched_switch+0x4c0/frame 0x1299abd0 mi_switch(104) at mi_switch+0x1c4/frame 0x1299ac04 sleepq_switch(b7fb9c4,6c) at sleepq_switch+0xeb/frame 0x1299ac24 _sleep(b7fb9c4,b7fb9a8,6c,152fb97,0,0,0,0,100) at _sleep+0x2bf/frame 0x1299ac64 crypto_ret_proc(b7fb9a8,1299ace8) at crypto_ret_proc+0x11e/frame 0x1299acb8 fork_exit(120d120,b7fb9a8,1299ace8,0,0,...) at fork_exit+0x68/frame 0x1299acd4 fork_trampoline() at 0xffc0340e/frame 0x1299acd4 --- kthread start Tracing command cam pid 7 tid 100044 td 0x12ac1380 sched_switch(12ac1380,104) at sched_switch+0x4c0/frame 0x12995be0 mi_switch(104) at mi_switch+0x1c4/frame 0x12995c14 sleepq_switch(1b03f40,5c) at sleepq_switch+0xeb/frame 0x12995c34 _sleep(1b03f40,1b03f00,5c,14488cd,0,0,0,0,100) at _sleep+0x2bf/frame 0x12995c74 xpt_done_td(1b03f00,12995ce8) at xpt_done_td+0x62/frame 0x12995cb8 fork_exit(90a2a0,1b03f00,12995ce8,0,0,...) at fork_exit+0x68/frame 0x12995cd4 fork_trampoline() at 0xffc0340e/frame 0x12995cd4 --- kthread start Tracing command cam pid 7 tid 100096 td 0x188a5a80 sched_switch(188a5a80,104) at sched_switch+0x4c0/frame 0xb5adbe8 mi_switch(104) at mi_switch+0x1c4/frame 0xb5adc20 sleepq_switch(1b03e6c,5c) at sleepq_switch+0xeb/frame 0xb5adc40 _sleep(1b03e6c,1b03edc,5c,14488cd,0,0,0,0,100) at _sleep+0x2bf/frame 0xb5adc80 xpt_scanner_thread(0,b5adce8) at xpt_scanner_thread+0x10d/frame 0xb5adcb8 fork_exit(90a530,0,b5adce8,0,0,...) at fork_exit+0x68/frame 0xb5adcd4 fork_trampoline() at 0xffc0340e/frame 0xb5adcd4 --- kthread start Tracing command fdc0 pid 8 tid 100084 td 0x1829e180 sched_switch(1829e180,104) at sched_switch+0x4c0/frame 0xb546ba8 mi_switch(104) at mi_switch+0x1c4/frame 0xb546be0 sleepq_switch(12bb2dbc,5c) at sleepq_switch+0xeb/frame 0xb546c00 _sleep(12bb2dbc,12bb2e78,5c,14488cd,0,0,0,0,100) at _sleep+0x2bf/frame 0xb546c40 fdc_thread(12bb2d80,b546ce8) at fdc_thread+0x361/frame 0xb546cb8 fork_exit(134e910,12bb2d80,b546ce8,0,0,...) at fork_exit+0x68/frame 0xb546cd4 fork_trampoline() at 0xffc0340e/frame 0xb546cd4 --- kthread start Tracing command soaiod1 pid 9 tid 100087 td 0x12ac1000 sched_switch(12ac1000,104) at sched_switch+0x4c0/frame 0x12990bc8 mi_switch(104) at mi_switch+0x1c4/frame 0x12990bfc sleepq_switch(0,1dee564,12990c7c,12ac1000,ffffdd5d,...) at sleepq_switch+0xeb/frame 0x12990c1c sleepq_timedwait(1dee564,0,0,144eb2b,14c29e5,...) at sleepq_timedwait+0x3c/frame 0x12990c3c _sleep(1dee564,1dee544,0,14488cd,ffffdd50,1d,0,0,100) at _sleep+0x234/frame 0x12990c7c soaio_kproc_loop(1,12990ce8) at soaio_kproc_loop+0x148/frame 0x12990cb8 fork_exit(1063010,1,12990ce8,0,0,...) at fork_exit+0x68/frame 0x12990cd4 fork_trampoline() at 0xffc0340e/frame 0x12990cd4 --- kthread start Tracing command audit pid 10 tid 100001 td 0x9bd0c00 sched_switch(9bd0c00,104) at sched_switch+0x4c0/frame 0xb4bdbf0 mi_switch(104) at mi_switch+0x1c4/frame 0xb4bdc24 sleepq_switch(1e0380c,0) at sleepq_switch+0xeb/frame 0xb4bdc44 _cv_wait(1e0380c,1e03768) at _cv_wait+0x135/frame 0xb4bdc7c audit_worker(0,b4bdce8) at audit_worker+0x78/frame 0xb4bdcb8 fork_exit(122eb20,0,b4bdce8,0,0,...) at fork_exit+0x68/frame 0xb4bdcd4 fork_trampoline() at 0xffc0340e/frame 0xb4bdcd4 --- kthread start Tracing command idle pid 11 tid 100003 td 0x9bd0500 (CPU 0) cpustop_handler(b4b3bd0,b4b3bc4,13eac0b,22fe020,0,...) at cpustop_handler+0x26/frame 0xb4b3afc ipi_nmi_handler(22fe020,0,b4b3b3c,fd3bd5,9bce3b0,...) at ipi_nmi_handler+0x30/frame 0xb4b3b08 trap(b4b3bd0,8,28,28,12b94e00,...) at trap+0xbb/frame 0xb4b3bc4 calltrap() at 0xffc0319f/frame 0xb4b3bc4 --- trap 0x13, eip = 0x139f535, esp = 0xb4b3c10, ebp = 0xb4b3c10 --- acpi_cpu_c1(1413ea3,312,1ef4,c118791c,d0f,...) at acpi_cpu_c1+0x5/frame 0xb4b3c10 acpi_cpu_idle(458dffb,0) at acpi_cpu_idle+0x22f/frame 0xb4b3c40 cpu_idle_acpi(458dffb,0) at cpu_idle_acpi+0x35/frame 0xb4b3c54 cpu_idle(0) at cpu_idle+0xa3/frame 0xb4b3c70 sched_idletd(0,b4b3ce8) at sched_idletd+0x32e/frame 0xb4b3cb8 fork_exit(101fb00,0,b4b3ce8,0,0,...) at fork_exit+0x68/frame 0xb4b3cd4 fork_trampoline() at 0xffc0340e/frame 0xb4b3cd4 --- kthread start Tracing command idle pid 11 tid 100004 td 0x9bd0180 sched_switch(9bd0180,608) at sched_switch+0x4c0/frame 0xb4aec28 mi_switch(608) at mi_switch+0x1c4/frame 0xb4aec60 critical_exit_preempt(0) at critical_exit_preempt+0x5c/frame 0xb4aec70 sched_idletd(0,b4aece8) at sched_idletd+0x32e/frame 0xb4aecb8 fork_exit(101fb00,0,b4aece8,0,0,...) at fork_exit+0x68/frame 0xb4aecd4 fork_trampoline() at 0xffc0340e/frame 0xb4aecd4 --- kthread start Tracing command idle pid 11 tid 100005 td 0x9bcfe00 (CPU 2) cpustop_handler(b4a9bd0,b4a9bc4,13eac0b,1d7e590,b7fd0c0,...) at cpustop_handler+0x26/frame 0xb4a9afc ipi_nmi_handler(1d7e590,b7fd0c0,b4a9b40,fd3803,9bce3b0,...) at ipi_nmi_handler+0x30/frame 0xb4a9b08 trap(b4a9bd0,8,28,28,12b94a00,...) at trap+0xbb/frame 0xb4a9bc4 calltrap() at 0xffc0319f/frame 0xb4a9bc4 --- trap 0x13, eip = 0x139f535, esp = 0xb4a9c10, ebp = 0xb4a9c10 --- acpi_cpu_c1(1413ea3,312,1ef4,b78c3b95,d0f,...) at acpi_cpu_c1+0x5/frame 0xb4a9c10 acpi_cpu_idle(2695ab86,0) at acpi_cpu_idle+0x22f/frame 0xb4a9c40 cpu_idle_acpi(2695ab86,0) at cpu_idle_acpi+0x35/frame 0xb4a9c54 cpu_idle(0) at cpu_idle+0xa3/frame 0xb4a9c70 sched_idletd(0,b4a9ce8) at sched_idletd+0x32e/frame 0xb4a9cb8 fork_exit(101fb00,0,b4a9ce8,0,0,...) at fork_exit+0x68/frame 0xb4a9cd4 fork_trampoline() at 0xffc0340e/frame 0xb4a9cd4 --- kthread start Tracing command idle pid 11 tid 100006 td 0x9bcfa80 (CPU 3) cpustop_handler(b4a4bd0,b4a4bc4,13eac0b,1d7e690,9bcfa80,...) at cpustop_handler+0x26/frame 0xb4a4afc ipi_nmi_handler(1d7e690,9bcfa80,b4a4b40,fd3803,9bce3b0,...) at ipi_nmi_handler+0x30/frame 0xb4a4b08 trap(b4a4bd0,8,28,28,12b94800,...) at trap+0xbb/frame 0xb4a4bc4 calltrap() at 0xffc0319f/frame 0xb4a4bc4 --- trap 0x13, eip = 0x139f535, esp = 0xb4a4c10, ebp = 0xb4a4c10 --- acpi_cpu_c1(1413ea3,312,1ef4,ac78f334,d0f,...) at acpi_cpu_c1+0x5/frame 0xb4a4c10 acpi_cpu_idle(7f49422a,0) at acpi_cpu_idle+0x22f/frame 0xb4a4c40 cpu_idle_acpi(7f49422a,0) at cpu_idle_acpi+0x35/frame 0xb4a4c54 cpu_idle(0) at cpu_idle+0xa3/frame 0xb4a4c70 sched_idletd(0,b4a4ce8) at sched_idletd+0x32e/frame 0xb4a4cb8 fork_exit(101fb00,0,b4a4ce8,0,0,...) at fork_exit+0x68/frame 0xb4a4cd4 fork_trampoline() at 0xffc0340e/frame 0xb4a4cd4 --- kthread start Tracing command intr pid 12 tid 100018 td 0x9bfe000 sched_switch(9bfe000,109) at sched_switch+0x4c0/frame 0x12917c38 mi_switch(109) at mi_switch+0x1c4/frame 0x12917c70 ithread_loop(9a5e500,12917ce8) at ithread_loop+0x85/frame 0x12917cb8 fork_exit(fb7f40,9a5e500,12917ce8,0,0,...) at fork_exit+0x68/frame 0x12917cd4 fork_trampoline() at 0xffc0340e/frame 0x12917cd4 --- kthread start Tracing command intr pid 12 tid 100020 td 0x12a56880 sched_switch(12a56880,109) at sched_switch+0x4c0/frame 0x1290dc38 mi_switch(109) at mi_switch+0x1c4/frame 0x1290dc70 ithread_loop(9a5e4f0,1290dce8) at ithread_loop+0x85/frame 0x1290dcb8 fork_exit(fb7f40,9a5e4f0,1290dce8,0,0,...) at fork_exit+0x68/frame 0x1290dcd4 fork_trampoline() at 0xffc0340e/frame 0x1290dcd4 --- kthread start Tracing command intr pid 12 tid 100023 td 0x12a55e00 __bss_end() at 0xffc03404 Tracing command intr pid 12 tid 100024 td 0x12a55a80 sched_switch(12a55a80,109) at sched_switch+0x4c0/frame 0x128f9c38 mi_switch(109) at mi_switch+0x1c4/frame 0x128f9c70 ithread_loop(9a5e4d0,128f9ce8) at ithread_loop+0x85/frame 0x128f9cb8 fork_exit(fb7f40,9a5e4d0,128f9ce8,0,0,...) at fork_exit+0x68/frame 0x128f9cd4 fork_trampoline() at 0xffc0340e/frame 0x128f9cd4 --- kthread start Tracing command intr pid 12 tid 100025 td 0x12a55700 __bss_end() at 0xffc03404 Tracing command intr pid 12 tid 100026 td 0x12a55380 sched_switch(12a55380,109) at sched_switch+0x4c0/frame 0x12943c38 mi_switch(109) at mi_switch+0x1c4/frame 0x12943c70 ithread_loop(9a5e4b0,12943ce8) at ithread_loop+0x85/frame 0x12943cb8 fork_exit(fb7f40,9a5e4b0,12943ce8,0,0,...) at fork_exit+0x68/frame 0x12943cd4 fork_trampoline() at 0xffc0340e/frame 0x12943cd4 --- kthread start Tracing command intr pid 12 tid 100027 td 0x12a55000 __bss_end() at 0xffc03404 Tracing command intr pid 12 tid 100028 td 0x12a5dc00 __bss_end() at 0xffc03404 Tracing command intr pid 12 tid 100029 td 0x12a5d880 __bss_end() at 0xffc03404 Tracing command intr pid 12 tid 100045 td 0x12bf4c00 sched_switch(12bf4c00,109) at sched_switch+0x4c0/frame 0x129fbc38 mi_switch(109) at mi_switch+0x1c4/frame 0x129fbc70 ithread_loop(12bf2fa0,129fbce8) at ithread_loop+0x85/frame 0x129fbcb8 fork_exit(fb7f40,12bf2fa0,129fbce8,0,0,...) at fork_exit+0x68/frame 0x129fbcd4 fork_trampoline() at 0xffc0340e/frame 0x129fbcd4 --- kthread start Tracing command intr pid 12 tid 100046 td 0x12bf4880 sched_switch(12bf4880,109) at sched_switch+0x4c0/frame 0x129f6c38 mi_switch(109) at mi_switch+0x1c4/frame 0x129f6c70 ithread_loop(12bf2f80,129f6ce8) at ithread_loop+0x85/frame 0x129f6cb8 fork_exit(fb7f40,12bf2f80,129f6ce8,0,0,...) at fork_exit+0x68/frame 0x129f6cd4 fork_trampoline() at 0xffc0340e/frame 0x129f6cd4 --- kthread start Tracing command intr pid 12 tid 100047 td 0x12bf4500 sched_switch(12bf4500,109) at sched_switch+0x4c0/frame 0x129f1c38 mi_switch(109) at mi_switch+0x1c4/frame 0x129f1c70 ithread_loop(12bf2f70,129f1ce8) at ithread_loop+0x85/frame 0x129f1cb8 fork_exit(fb7f40,12bf2f70,129f1ce8,0,0,...) at fork_exit+0x68/frame 0x129f1cd4 fork_trampoline() at 0xffc0340e/frame 0x129f1cd4 --- kthread start Tracing command intr pid 12 tid 100048 td 0x12bf4180 sched_switch(12bf4180,109) at sched_switch+0x4c0/frame 0x129ecc38 mi_switch(109) at mi_switch+0x1c4/frame 0x129ecc70 ithread_loop(12bf2f60,129ecce8) at ithread_loop+0x85/frame 0x129eccb8 fork_exit(fb7f40,12bf2f60,129ecce8,0,0,...) at fork_exit+0x68/frame 0x129eccd4 fork_trampoline() at 0xffc0340e/frame 0x129eccd4 --- kthread start Tracing command intr pid 12 tid 100054 td 0x18264c00 sched_switch(18264c00,109) at sched_switch+0x4c0/frame 0xb4e8c38 mi_switch(109) at mi_switch+0x1c4/frame 0xb4e8c70 ithread_loop(12bf2f50,b4e8ce8) at ithread_loop+0x85/frame 0xb4e8cb8 fork_exit(fb7f40,12bf2f50,b4e8ce8,0,0,...) at fork_exit+0x68/frame 0xb4e8cd4 fork_trampoline() at 0xffc0340e/frame 0xb4e8cd4 --- kthread start Tracing command intr pid 12 tid 100060 td 0x18263700 sched_switch(18263700,109) at sched_switch+0x4c0/frame 0xb4cac38 mi_switch(109) at mi_switch+0x1c4/frame 0xb4cac70 ithread_loop(12bf2f40,b4cace8) at ithread_loop+0x85/frame 0xb4cacb8 fork_exit(fb7f40,12bf2f40,b4cace8,0,0,...) at fork_exit+0x68/frame 0xb4cacd4 fork_trampoline() at 0xffc0340e/frame 0xb4cacd4 --- kthread start Tracing command intr pid 12 tid 100076 td 0x1827de00 __bss_end() at 0xffc03404 Tracing command intr pid 12 tid 100082 td 0x1829e880 __bss_end() at 0xffc03404 Tracing command intr pid 12 tid 100083 td 0x1829e500 sched_switch(1829e500,109) at sched_switch+0x4c0/frame 0xb54bc38 mi_switch(109) at mi_switch+0x1c4/frame 0xb54bc70 ithread_loop(12bf2f10,b54bce8) at ithread_loop+0x85/frame 0xb54bcb8 fork_exit(fb7f40,12bf2f10,b54bce8,0,0,...) at fork_exit+0x68/frame 0xb54bcd4 fork_trampoline() at 0xffc0340e/frame 0xb54bcd4 --- kthread start Tracing command intr pid 12 tid 100085 td 0x1829de00 __bss_end() at 0xffc03404 Tracing command geom pid 13 tid 100031 td 0x12a5d180 sched_switch(12a5d180,104) at sched_switch+0x4c0/frame 0x1292abe8 mi_switch(104) at mi_switch+0x1c4/frame 0x1292ac20 sleepq_switch(1d6bb88,5c) at sleepq_switch+0xeb/frame 0x1292ac40 _sleep(1d6bb88,1d6baf4,25c,14488cd,0,0,0,0,100) at _sleep+0x2bf/frame 0x1292ac80 g_run_events(0,1292ace8) at g_run_events+0x4d/frame 0x1292acb8 fork_exit(f46af0,0,1292ace8,0,0,...) at fork_exit+0x68/frame 0x1292acd4 fork_trampoline() at 0xffc0340e/frame 0x1292acd4 --- kthread start Tracing command geom pid 13 tid 100032 td 0x12a5ce00 sched_switch(12a5ce00,104) at sched_switch+0x4c0/frame 0x12925be0 mi_switch(104) at mi_switch+0x1c4/frame 0x12925c18 sleepq_switch(1d6bb8c,5c) at sleepq_switch+0xeb/frame 0x12925c38 _sleep(1d6bb8c,1d6bb38,25c,14488cd,0,0,0,0,100) at _sleep+0x2bf/frame 0x12925c78 g_io_schedule_up(12a5ce00) at g_io_schedule_up+0x36/frame 0x12925cac g_up_procbody(0,12925ce8) at g_up_procbody+0x5b/frame 0x12925cb8 fork_exit(f46b40,0,12925ce8,0,0,...) at fork_exit+0x68/frame 0x12925cd4 fork_trampoline() at 0xffc0340e/frame 0x12925cd4 --- kthread start Tracing command geom pid 13 tid 100033 td 0x12a5ca80 sched_switch(12a5ca80,104) at sched_switch+0x4c0/frame 0x1297cbe0 mi_switch(104) at mi_switch+0x1c4/frame 0x1297cc14 sleepq_switch(1d6bb90,5c) at sleepq_switch+0xeb/frame 0x1297cc34 _sleep(1d6bb90,1d6bb18,25c,14488cd,0,0,0,0,100) at _sleep+0x2bf/frame 0x1297cc74 g_io_schedule_down(12a5ca80) at g_io_schedule_down+0x36/frame 0x1297ccac g_down_procbody(0,1297cce8) at g_down_procbody+0x5b/frame 0x1297ccb8 fork_exit(f46ba0,0,1297cce8,0,0,...) at fork_exit+0x68/frame 0x1297ccd4 fork_trampoline() at 0xffc0340e/frame 0x1297ccd4 --- kthread start Tracing command sequencer 00 pid 14 tid 100034 td 0x12a5c700 sched_switch(12a5c700,104) at sched_switch+0x4c0/frame 0x12977be0 mi_switch(104) at mi_switch+0x1c4/frame 0x12977c18 sleepq_switch(9a4bc4c,0) at sleepq_switch+0xeb/frame 0x12977c38 _cv_wait(9a4bc4c,9a4bc04) at _cv_wait+0x135/frame 0x12977c70 seq_eventthread(9a4bc00,12977ce8) at seq_eventthread+0x7a/frame 0x12977cb8 fork_exit(dd7030,9a4bc00,12977ce8,0,0,...) at fork_exit+0x68/frame 0x12977cd4 fork_trampoline() at 0xffc0340e/frame 0x12977cd4 --- kthread start Tracing command usb pid 15 tid 100049 td 0x12bf3e00 sched_switch(12bf3e00,104) at sched_switch+0x4c0/frame 0x129e7c08 mi_switch(104) at mi_switch+0x1c4/frame 0x129e7c40 sleepq_switch(12bdeaac,0) at sleepq_switch+0xeb/frame 0x129e7c60 _cv_wait(12bdeaac,12bdecd8) at _cv_wait+0x135/frame 0x129e7c98 usb_process(12bdeaa4,129e7ce8) at usb_process+0x9d/frame 0x129e7cb8 fork_exit(e47200,12bdeaa4,129e7ce8,0,0,...) at fork_exit+0x68/frame 0x129e7cd4 fork_trampoline() at 0xffc0340e/frame 0x129e7cd4 --- kthread start Tracing command usb pid 15 tid 100050 td 0x12bf3a80 sched_switch(12bf3a80,104) at sched_switch+0x4c0/frame 0x129e2c08 mi_switch(104) at mi_switch+0x1c4/frame 0x129e2c40 sleepq_switch(12bdeadc,0) at sleepq_switch+0xeb/frame 0x129e2c60 _cv_wait(12bdeadc,12bdecd8) at _cv_wait+0x135/frame 0x129e2c98 usb_process(12bdead4,129e2ce8) at usb_process+0x9d/frame 0x129e2cb8 fork_exit(e47200,12bdead4,129e2ce8,0,0,...) at fork_exit+0x68/frame 0x129e2cd4 fork_trampoline() at 0xffc0340e/frame 0x129e2cd4 --- kthread start Tracing command usb pid 15 tid 100051 td 0x12bf3700 sched_switch(12bf3700,104) at sched_switch+0x4c0/frame 0x129ddc08 mi_switch(104) at mi_switch+0x1c4/frame 0x129ddc40 sleepq_switch(12bdeb0c,0) at sleepq_switch+0xeb/frame 0x129ddc60 _cv_wait(12bdeb0c,12bdecd8) at _cv_wait+0x135/frame 0x129ddc98 usb_process(12bdeb04,129ddce8) at usb_process+0x9d/frame 0x129ddcb8 fork_exit(e47200,12bdeb04,129ddce8,0,0,...) at fork_exit+0x68/frame 0x129ddcd4 fork_trampoline() at 0xffc0340e/frame 0x129ddcd4 --- kthread start Tracing command usb pid 15 tid 100052 td 0x12bf3380 sched_switch(12bf3380,104) at sched_switch+0x4c0/frame 0x129d8c08 mi_switch(104) at mi_switch+0x1c4/frame 0x129d8c40 sleepq_switch(12bdeb3c,0) at sleepq_switch+0xeb/frame 0x129d8c60 _cv_wait(12bdeb3c,12bdecd8) at _cv_wait+0x135/frame 0x129d8c98 usb_process(12bdeb34,129d8ce8) at usb_process+0x9d/frame 0x129d8cb8 fork_exit(e47200,12bdeb34,129d8ce8,0,0,...) at fork_exit+0x68/frame 0x129d8cd4 fork_trampoline() at 0xffc0340e/frame 0x129d8cd4 --- kthread start Tracing command usb pid 15 tid 100053 td 0x12bf3000 sched_switch(12bf3000,104) at sched_switch+0x4c0/frame 0xb4edc08 mi_switch(104) at mi_switch+0x1c4/frame 0xb4edc40 sleepq_switch(12bdeb6c,0) at sleepq_switch+0xeb/frame 0xb4edc60 _cv_wait(12bdeb6c,12bdecd8) at _cv_wait+0x135/frame 0xb4edc98 usb_process(12bdeb64,b4edce8) at usb_process+0x9d/frame 0xb4edcb8 fork_exit(e47200,12bdeb64,b4edce8,0,0,...) at fork_exit+0x68/frame 0xb4edcd4 fork_trampoline() at 0xffc0340e/frame 0xb4edcd4 --- kthread start Tracing command usb pid 15 tid 100055 td 0x18264880 sched_switch(18264880,104) at sched_switch+0x4c0/frame 0xb4e3c08 mi_switch(104) at mi_switch+0x1c4/frame 0xb4e3c40 sleepq_switch(1825caac,0) at sleepq_switch+0xeb/frame 0xb4e3c60 _cv_wait(1825caac,1825ccd8) at _cv_wait+0x135/frame 0xb4e3c98 usb_process(1825caa4,b4e3ce8) at usb_process+0x9d/frame 0xb4e3cb8 fork_exit(e47200,1825caa4,b4e3ce8,0,0,...) at fork_exit+0x68/frame 0xb4e3cd4 fork_trampoline() at 0xffc0340e/frame 0xb4e3cd4 --- kthread start Tracing command usb pid 15 tid 100056 td 0x18264500 sched_switch(18264500,104) at sched_switch+0x4c0/frame 0xb4dec08 mi_switch(104) at mi_switch+0x1c4/frame 0xb4dec40 sleepq_switch(1825cadc,0) at sleepq_switch+0xeb/frame 0xb4dec60 _cv_wait(1825cadc,1825ccd8) at _cv_wait+0x135/frame 0xb4dec98 usb_process(1825cad4,b4dece8) at usb_process+0x9d/frame 0xb4decb8 fork_exit(e47200,1825cad4,b4dece8,0,0,...) at fork_exit+0x68/frame 0xb4decd4 fork_trampoline() at 0xffc0340e/frame 0xb4decd4 --- kthread start Tracing command usb pid 15 tid 100057 td 0x18264180 sched_switch(18264180,104) at sched_switch+0x4c0/frame 0xb4d9c08 mi_switch(104) at mi_switch+0x1c4/frame 0xb4d9c40 sleepq_switch(1825cb0c,0) at sleepq_switch+0xeb/frame 0xb4d9c60 _cv_wait(1825cb0c,1825ccd8) at _cv_wait+0x135/frame 0xb4d9c98 usb_process(1825cb04,b4d9ce8) at usb_process+0x9d/frame 0xb4d9cb8 fork_exit(e47200,1825cb04,b4d9ce8,0,0,...) at fork_exit+0x68/frame 0xb4d9cd4 fork_trampoline() at 0xffc0340e/frame 0xb4d9cd4 --- kthread start Tracing command usb pid 15 tid 100058 td 0x18263e00 sched_switch(18263e00,104) at sched_switch+0x4c0/frame 0xb4d4c08 mi_switch(104) at mi_switch+0x1c4/frame 0xb4d4c40 sleepq_switch(1825cb3c,0) at sleepq_switch+0xeb/frame 0xb4d4c60 _cv_wait(1825cb3c,1825ccd8) at _cv_wait+0x135/frame 0xb4d4c98 usb_process(1825cb34,b4d4ce8) at usb_process+0x9d/frame 0xb4d4cb8 fork_exit(e47200,1825cb34,b4d4ce8,0,0,...) at fork_exit+0x68/frame 0xb4d4cd4 fork_trampoline() at 0xffc0340e/frame 0xb4d4cd4 --- kthread start Tracing command usb pid 15 tid 100059 td 0x18263a80 sched_switch(18263a80,104) at sched_switch+0x4c0/frame 0xb4cfc08 mi_switch(104) at mi_switch+0x1c4/frame 0xb4cfc40 sleepq_switch(1825cb6c,0) at sleepq_switch+0xeb/frame 0xb4cfc60 _cv_wait(1825cb6c,1825ccd8) at _cv_wait+0x135/frame 0xb4cfc98 usb_process(1825cb64,b4cfce8) at usb_process+0x9d/frame 0xb4cfcb8 fork_exit(e47200,1825cb64,b4cfce8,0,0,...) at fork_exit+0x68/frame 0xb4cfcd4 fork_trampoline() at 0xffc0340e/frame 0xb4cfcd4 --- kthread start Tracing command usb pid 15 tid 100061 td 0x18263380 sched_switch(18263380,104) at sched_switch+0x4c0/frame 0xb516c08 mi_switch(104) at mi_switch+0x1c4/frame 0xb516c40 sleepq_switch(18266aac,0) at sleepq_switch+0xeb/frame 0xb516c60 _cv_wait(18266aac,18266cd8) at _cv_wait+0x135/frame 0xb516c98 usb_process(18266aa4,b516ce8) at usb_process+0x9d/frame 0xb516cb8 fork_exit(e47200,18266aa4,b516ce8,0,0,...) at fork_exit+0x68/frame 0xb516cd4 fork_trampoline() at 0xffc0340e/frame 0xb516cd4 --- kthread start Tracing command usb pid 15 tid 100062 td 0x18263000 sched_switch(18263000,104) at sched_switch+0x4c0/frame 0xb511c08 mi_switch(104) at mi_switch+0x1c4/frame 0xb511c40 sleepq_switch(18266adc,0) at sleepq_switch+0xeb/frame 0xb511c60 _cv_wait(18266adc,18266cd8) at _cv_wait+0x135/frame 0xb511c98 usb_process(18266ad4,b511ce8) at usb_process+0x9d/frame 0xb511cb8 fork_exit(e47200,18266ad4,b511ce8,0,0,...) at fork_exit+0x68/frame 0xb511cd4 fork_trampoline() at 0xffc0340e/frame 0xb511cd4 --- kthread start Tracing command usb pid 15 tid 100063 td 0x1826fc00 sched_switch(1826fc00,104) at sched_switch+0x4c0/frame 0xb50cc08 mi_switch(104) at mi_switch+0x1c4/frame 0xb50cc40 sleepq_switch(18266b0c,0) at sleepq_switch+0xeb/frame 0xb50cc60 _cv_wait(18266b0c,18266cd8) at _cv_wait+0x135/frame 0xb50cc98 usb_process(18266b04,b50cce8) at usb_process+0x9d/frame 0xb50ccb8 fork_exit(e47200,18266b04,b50cce8,0,0,...) at fork_exit+0x68/frame 0xb50ccd4 fork_trampoline() at 0xffc0340e/frame 0xb50ccd4 --- kthread start Tracing command usb pid 15 tid 100064 td 0x1826f880 sched_switch(1826f880,104) at sched_switch+0x4c0/frame 0xb507c08 mi_switch(104) at mi_switch+0x1c4/frame 0xb507c40 sleepq_switch(18266b3c,0) at sleepq_switch+0xeb/frame 0xb507c60 _cv_wait(18266b3c,18266cd8) at _cv_wait+0x135/frame 0xb507c98 usb_process(18266b34,b507ce8) at usb_process+0x9d/frame 0xb507cb8 fork_exit(e47200,18266b34,b507ce8,0,0,...) at fork_exit+0x68/frame 0xb507cd4 fork_trampoline() at 0xffc0340e/frame 0xb507cd4 --- kthread start Tracing command usb pid 15 tid 100065 td 0x1826f500 sched_switch(1826f500,104) at sched_switch+0x4c0/frame 0xb502c08 mi_switch(104) at mi_switch+0x1c4/frame 0xb502c40 sleepq_switch(18266b6c,0) at sleepq_switch+0xeb/frame 0xb502c60 _cv_wait(18266b6c,18266cd8) at _cv_wait+0x135/frame 0xb502c98 usb_process(18266b64,b502ce8) at usb_process+0x9d/frame 0xb502cb8 fork_exit(e47200,18266b64,b502ce8,0,0,...) at fork_exit+0x68/frame 0xb502cd4 fork_trampoline() at 0xffc0340e/frame 0xb502cd4 --- kthread start Tracing command usb pid 15 tid 100066 td 0x1826f180 sched_switch(1826f180,104) at sched_switch+0x4c0/frame 0xb4fdc08 mi_switch(104) at mi_switch+0x1c4/frame 0xb4fdc40 sleepq_switch(18271aac,0) at sleepq_switch+0xeb/frame 0xb4fdc60 _cv_wait(18271aac,18271cd8) at _cv_wait+0x135/frame 0xb4fdc98 usb_process(18271aa4,b4fdce8) at usb_process+0x9d/frame 0xb4fdcb8 fork_exit(e47200,18271aa4,b4fdce8,0,0,...) at fork_exit+0x68/frame 0xb4fdcd4 fork_trampoline() at 0xffc0340e/frame 0xb4fdcd4 --- kthread start Tracing command usb pid 15 tid 100067 td 0x1826ee00 sched_switch(1826ee00,104) at sched_switch+0x4c0/frame 0xb4f8c08 mi_switch(104) at mi_switch+0x1c4/frame 0xb4f8c40 sleepq_switch(18271adc,0) at sleepq_switch+0xeb/frame 0xb4f8c60 _cv_wait(18271adc,18271cd8) at _cv_wait+0x135/frame 0xb4f8c98 usb_process(18271ad4,b4f8ce8) at usb_process+0x9d/frame 0xb4f8cb8 fork_exit(e47200,18271ad4,b4f8ce8,0,0,...) at fork_exit+0x68/frame 0xb4f8cd4 fork_trampoline() at 0xffc0340e/frame 0xb4f8cd4 --- kthread start Tracing command usb pid 15 tid 100068 td 0x1826ea80 sched_switch(1826ea80,104) at sched_switch+0x4c0/frame 0xb4f3c08 mi_switch(104) at mi_switch+0x1c4/frame 0xb4f3c40 sleepq_switch(18271b0c,0) at sleepq_switch+0xeb/frame 0xb4f3c60 _cv_wait(18271b0c,18271cd8) at _cv_wait+0x135/frame 0xb4f3c98 usb_process(18271b04,b4f3ce8) at usb_process+0x9d/frame 0xb4f3cb8 fork_exit(e47200,18271b04,b4f3ce8,0,0,...) at fork_exit+0x68/frame 0xb4f3cd4 fork_trampoline() at 0xffc0340e/frame 0xb4f3cd4 --- kthread start Tracing command usb pid 15 tid 100069 td 0x1826e700 sched_switch(1826e700,104) at sched_switch+0x4c0/frame 0xb53fc08 mi_switch(104) at mi_switch+0x1c4/frame 0xb53fc40 sleepq_switch(18271b3c,0) at sleepq_switch+0xeb/frame 0xb53fc60 _cv_wait(18271b3c,18271cd8) at _cv_wait+0x135/frame 0xb53fc98 usb_process(18271b34,b53fce8) at usb_process+0x9d/frame 0xb53fcb8 fork_exit(e47200,18271b34,b53fce8,0,0,...) at fork_exit+0x68/frame 0xb53fcd4 fork_trampoline() at 0xffc0340e/frame 0xb53fcd4 --- kthread start Tracing command usb pid 15 tid 100070 td 0x1826e380 sched_switch(1826e380,104) at sched_switch+0x4c0/frame 0xb53ac08 mi_switch(104) at mi_switch+0x1c4/frame 0xb53ac40 sleepq_switch(18271b6c,0) at sleepq_switch+0xeb/frame 0xb53ac60 _cv_wait(18271b6c,18271cd8) at _cv_wait+0x135/frame 0xb53ac98 usb_process(18271b64,b53ace8) at usb_process+0x9d/frame 0xb53acb8 fork_exit(e47200,18271b64,b53ace8,0,0,...) at fork_exit+0x68/frame 0xb53acd4 fork_trampoline() at 0xffc0340e/frame 0xb53acd4 --- kthread start Tracing command usb pid 15 tid 100071 td 0x1826e000 sched_switch(1826e000,104) at sched_switch+0x4c0/frame 0xb535c08 mi_switch(104) at mi_switch+0x1c4/frame 0xb535c40 sleepq_switch(18277aac,0) at sleepq_switch+0xeb/frame 0xb535c60 _cv_wait(18277aac,18277cd8) at _cv_wait+0x135/frame 0xb535c98 usb_process(18277aa4,b535ce8) at usb_process+0x9d/frame 0xb535cb8 fork_exit(e47200,18277aa4,b535ce8,0,0,...) at fork_exit+0x68/frame 0xb535cd4 fork_trampoline() at 0xffc0340e/frame 0xb535cd4 --- kthread start Tracing command usb pid 15 tid 100072 td 0x1827ec00 sched_switch(1827ec00,104) at sched_switch+0x4c0/frame 0xb530c08 mi_switch(104) at mi_switch+0x1c4/frame 0xb530c40 sleepq_switch(18277adc,0) at sleepq_switch+0xeb/frame 0xb530c60 _cv_wait(18277adc,18277cd8) at _cv_wait+0x135/frame 0xb530c98 usb_process(18277ad4,b530ce8) at usb_process+0x9d/frame 0xb530cb8 fork_exit(e47200,18277ad4,b530ce8,0,0,...) at fork_exit+0x68/frame 0xb530cd4 fork_trampoline() at 0xffc0340e/frame 0xb530cd4 --- kthread start Tracing command usb pid 15 tid 100073 td 0x1827e880 sched_switch(1827e880,104) at sched_switch+0x4c0/frame 0xb52bc08 mi_switch(104) at mi_switch+0x1c4/frame 0xb52bc40 sleepq_switch(18277b0c,0) at sleepq_switch+0xeb/frame 0xb52bc60 _cv_wait(18277b0c,18277cd8) at _cv_wait+0x135/frame 0xb52bc98 usb_process(18277b04,b52bce8) at usb_process+0x9d/frame 0xb52bcb8 fork_exit(e47200,18277b04,b52bce8,0,0,...) at fork_exit+0x68/frame 0xb52bcd4 fork_trampoline() at 0xffc0340e/frame 0xb52bcd4 --- kthread start Tracing command usb pid 15 tid 100074 td 0x1827e500 sched_switch(1827e500,104) at sched_switch+0x4c0/frame 0xb526c08 mi_switch(104) at mi_switch+0x1c4/frame 0xb526c40 sleepq_switch(18277b3c,0) at sleepq_switch+0xeb/frame 0xb526c60 _cv_wait(18277b3c,18277cd8) at _cv_wait+0x135/frame 0xb526c98 usb_process(18277b34,b526ce8) at usb_process+0x9d/frame 0xb526cb8 fork_exit(e47200,18277b34,b526ce8,0,0,...) at fork_exit+0x68/frame 0xb526cd4 fork_trampoline() at 0xffc0340e/frame 0xb526cd4 --- kthread start Tracing command usb pid 15 tid 100075 td 0x1827e180 sched_switch(1827e180,104) at sched_switch+0x4c0/frame 0xb521c08 mi_switch(104) at mi_switch+0x1c4/frame 0xb521c40 sleepq_switch(18277b6c,0) at sleepq_switch+0xeb/frame 0xb521c60 _cv_wait(18277b6c,18277cd8) at _cv_wait+0x135/frame 0xb521c98 usb_process(18277b64,b521ce8) at usb_process+0x9d/frame 0xb521cb8 fork_exit(e47200,18277b64,b521ce8,0,0,...) at fork_exit+0x68/frame 0xb521cd4 fork_trampoline() at 0xffc0340e/frame 0xb521cd4 --- kthread start Tracing command usb pid 15 tid 100077 td 0x1827da80 sched_switch(1827da80,104) at sched_switch+0x4c0/frame 0xb569c08 mi_switch(104) at mi_switch+0x1c4/frame 0xb569c40 sleepq_switch(12be3c84,0) at sleepq_switch+0xeb/frame 0xb569c60 _cv_wait(12be3c84,12be3eb0) at _cv_wait+0x135/frame 0xb569c98 usb_process(12be3c7c,b569ce8) at usb_process+0x9d/frame 0xb569cb8 fork_exit(e47200,12be3c7c,b569ce8,0,0,...) at fork_exit+0x68/frame 0xb569cd4 fork_trampoline() at 0xffc0340e/frame 0xb569cd4 --- kthread start Tracing command usb pid 15 tid 100078 td 0x1827d700 sched_switch(1827d700,104) at sched_switch+0x4c0/frame 0xb564c08 mi_switch(104) at mi_switch+0x1c4/frame 0xb564c40 sleepq_switch(12be3cb4,0) at sleepq_switch+0xeb/frame 0xb564c60 _cv_wait(12be3cb4,12be3eb0) at _cv_wait+0x135/frame 0xb564c98 usb_process(12be3cac,b564ce8) at usb_process+0x9d/frame 0xb564cb8 fork_exit(e47200,12be3cac,b564ce8,0,0,...) at fork_exit+0x68/frame 0xb564cd4 fork_trampoline() at 0xffc0340e/frame 0xb564cd4 --- kthread start Tracing command usb pid 15 tid 100079 td 0x1827d380 sched_switch(1827d380,104) at sched_switch+0x4c0/frame 0xb55fc08 mi_switch(104) at mi_switch+0x1c4/frame 0xb55fc40 sleepq_switch(12be3ce4,0) at sleepq_switch+0xeb/frame 0xb55fc60 _cv_wait(12be3ce4,12be3eb0) at _cv_wait+0x135/frame 0xb55fc98 usb_process(12be3cdc,b55fce8) at usb_process+0x9d/frame 0xb55fcb8 fork_exit(e47200,12be3cdc,b55fce8,0,0,...) at fork_exit+0x68/frame 0xb55fcd4 fork_trampoline() at 0xffc0340e/frame 0xb55fcd4 --- kthread start Tracing command usb pid 15 tid 100080 td 0x1827d000 sched_switch(1827d000,104) at sched_switch+0x4c0/frame 0xb55ac08 mi_switch(104) at mi_switch+0x1c4/frame 0xb55ac40 sleepq_switch(12be3d14,0) at sleepq_switch+0xeb/frame 0xb55ac60 _cv_wait(12be3d14,12be3eb0) at _cv_wait+0x135/frame 0xb55ac98 usb_process(12be3d0c,b55ace8) at usb_process+0x9d/frame 0xb55acb8 fork_exit(e47200,12be3d0c,b55ace8,0,0,...) at fork_exit+0x68/frame 0xb55acd4 fork_trampoline() at 0xffc0340e/frame 0xb55acd4 --- kthread start Tracing command usb pid 15 tid 100081 td 0x1829ec00 sched_switch(1829ec00,104) at sched_switch+0x4c0/frame 0xb555c08 mi_switch(104) at mi_switch+0x1c4/frame 0xb555c40 sleepq_switch(12be3d44,0) at sleepq_switch+0xeb/frame 0xb555c60 _cv_wait(12be3d44,12be3eb0) at _cv_wait+0x135/frame 0xb555c98 usb_process(12be3d3c,b555ce8) at usb_process+0x9d/frame 0xb555cb8 fork_exit(e47200,12be3d3c,b555ce8,0,0,...) at fork_exit+0x68/frame 0xb555cd4 fork_trampoline() at 0xffc0340e/frame 0xb555cd4 --- kthread start Tracing command soaiod2 pid 16 tid 100088 td 0x18798c00 sched_switch(18798c00,104) at sched_switch+0x4c0/frame 0x1298bbc8 mi_switch(104) at mi_switch+0x1c4/frame 0x1298bbfc sleepq_switch(0,1dee564,1298bc7c,18798c00,ffffdd5d,...) at sleepq_switch+0xeb/frame 0x1298bc1c sleepq_timedwait(1dee564,0,1d6c768,1298bcb8,14c29e5,...) at sleepq_timedwait+0x3c/frame 0x1298bc3c _sleep(1dee564,1dee544,0,14488cd,ffffdd50,1d,0,0,100) at _sleep+0x234/frame 0x1298bc7c soaio_kproc_loop(2,1298bce8) at soaio_kproc_loop+0x148/frame 0x1298bcb8 fork_exit(1063010,2,1298bce8,0,0,...) at fork_exit+0x68/frame 0x1298bcd4 fork_trampoline() at 0xffc0340e/frame 0x1298bcd4 --- kthread start Tracing command soaiod3 pid 17 tid 100089 td 0x18798880 sched_switch(18798880,104) at sched_switch+0x4c0/frame 0x12986bc8 mi_switch(104) at mi_switch+0x1c4/frame 0x12986bfc sleepq_switch(0,1dee564,12986c7c,18798880,ffffdd5d,...) at sleepq_switch+0xeb/frame 0x12986c1c sleepq_timedwait(1dee564,0,0,144eb2b,14c29e5,...) at sleepq_timedwait+0x3c/frame 0x12986c3c _sleep(1dee564,1dee544,0,14488cd,ffffdd50,1d,0,0,100) at _sleep+0x234/frame 0x12986c7c soaio_kproc_loop(3,12986ce8) at soaio_kproc_loop+0x148/frame 0x12986cb8 fork_exit(1063010,3,12986ce8,0,0,...) at fork_exit+0x68/frame 0x12986cd4 fork_trampoline() at 0xffc0340e/frame 0x12986cd4 --- kthread start Tracing command soaiod4 pid 18 tid 100090 td 0x18798500 sched_switch(18798500,104) at sched_switch+0x4c0/frame 0x12981bc8 mi_switch(104) at mi_switch+0x1c4/frame 0x12981bfc sleepq_switch(0,1dee564,12981c7c,18798500,ffffdd5d,...) at sleepq_switch+0xeb/frame 0x12981c1c sleepq_timedwait(1dee564,0,0,144eb2b,14c29e5,...) at sleepq_timedwait+0x3c/frame 0x12981c3c _sleep(1dee564,1dee544,0,14488cd,ffffdd50,1d,0,0,100) at _sleep+0x234/frame 0x12981c7c soaio_kproc_loop(4,12981ce8) at soaio_kproc_loop+0x148/frame 0x12981cb8 fork_exit(1063010,4,12981ce8,0,0,...) at fork_exit+0x68/frame 0x12981cd4 fork_trampoline() at 0xffc0340e/frame 0x12981cd4 --- kthread start Tracing command rand_harvestq pid 19 tid 100094 td 0x188a6180 sched_switch(188a6180,104) at sched_switch+0x4c0/frame 0xb5b7b90 mi_switch(104) at mi_switch+0x1c4/frame 0xb5b7bc8 sleepq_switch(0,1bccbf8,b5b7c48,188a6180,19999999,...) at sleepq_switch+0xeb/frame 0xb5b7be8 sleepq_timedwait(1bccbf8,0,1bd1cec,0,b5b7c44,...) at sleepq_timedwait+0x3c/frame 0xb5b7c08 _sleep(1bccbf8,0,0,14488cd,19999999,0,0,0,4) at _sleep+0x234/frame 0xb5b7c48 random_kthread(0,b5b7ce8) at random_kthread+0x50/frame 0xb5b7cb8 fork_exit(d4bb20,0,b5b7ce8,0,0,...) at fork_exit+0x68/frame 0xb5b7cd4 fork_trampoline() at 0xffc0340e/frame 0xb5b7cd4 --- kthread start Tracing command pagedaemon pid 20 tid 100097 td 0x188a5700 sched_switch(188a5700,104) at sched_switch+0x4c0/frame 0xb5a8a60 mi_switch(104) at mi_switch+0x1c4/frame 0xb5a8a94 sleepq_switch(0,1e04dbc,b5a8b14,188a5700,1999997c,...) at sleepq_switch+0xeb/frame 0xb5a8ab4 sleepq_timedwait(1e04dbc,54,10000,10000,14638a5,...) at sleepq_timedwait+0x3c/frame 0xb5a8ad4 _sleep(1e04dbc,1e04ac0,254,1444e66,1999997c,0,0,0,100) at _sleep+0x234/frame 0xb5a8b14 vm_pageout_worker(0,0,188a5700,188ad6f8,188a5700,...) at vm_pageout_worker+0x26a/frame 0xb5a8c94 vm_pageout(0,b5a8ce8) at vm_pageout+0x18a/frame 0xb5a8cb8 fork_exit(12bf800,0,b5a8ce8,0,0,...) at fork_exit+0x68/frame 0xb5a8cd4 fork_trampoline() at 0xffc0340e/frame 0xb5a8cd4 --- kthread start Tracing command pagedaemon pid 20 tid 100101 td 0x1829d700 sched_switch(1829d700,104) at sched_switch+0x4c0/frame 0xb58ba80 mi_switch(104) at mi_switch+0x1c4/frame 0xb58bab4 sleepq_switch(1e04dc8,54) at sleepq_switch+0xeb/frame 0xb58bad4 _sleep(1e04dc8,1e04a00,54,153a416,0,0,0,0,100) at _sleep+0x2bf/frame 0xb58bb14 vm_pageout_laundry_worker(0,b58bce8) at vm_pageout_laundry_worker+0xca5/frame 0xb58bcb8 fork_exit(12c0ab0,0,b58bce8,0,0,...) at fork_exit+0x68/frame 0xb58bcd4 fork_trampoline() at 0xffc0340e/frame 0xb58bcd4 --- kthread start Tracing command pagedaemon pid 20 tid 100102 td 0x1829d380 sched_switch(1829d380,104) at sched_switch+0x4c0/frame 0xb586bc8 mi_switch(104) at mi_switch+0x1c4/frame 0xb586c00 sleepq_switch(1e0460c,b586c44,1044fdb,1829d380,fffffed8,...) at sleepq_switch+0xeb/frame 0xb586c20 sleepq_timedwait(1299a20,54,0,0,14947e6,...) at sleepq_timedwait+0x3c/frame 0xb586c40 _sleep(1299a20,1e0460c,54,14a77c5,fffffed8,0,0,0,100) at _sleep+0x234/frame 0xb586c80 uma_reclaim_worker(0,b586ce8) at uma_reclaim_worker+0xe4/frame 0xb586cb8 fork_exit(1299f20,0,b586ce8,0,0,...) at fork_exit+0x68/frame 0xb586cd4 fork_trampoline() at 0xffc0340e/frame 0xb586cd4 --- kthread start Tracing command vmdaemon pid 21 tid 100098 td 0x188a5380 sched_switch(188a5380,104) at sched_switch+0x4c0/frame 0xb5a3ba8 mi_switch(104) at mi_switch+0x1c4/frame 0xb5a3bdc sleepq_switch(1e05d70,74) at sleepq_switch+0xeb/frame 0xb5a3bfc _sleep(1e05d70,1e05d5c,74,1444e66,0,0,0,0,100) at _sleep+0x2bf/frame 0xb5a3c3c vm_daemon(0,b5a3ce8) at vm_daemon+0x7f/frame 0xb5a3cb8 fork_exit(12caca0,0,b5a3ce8,0,0,...) at fork_exit+0x68/frame 0xb5a3cd4 fork_trampoline() at 0xffc0340e/frame 0xb5a3cd4 --- kthread start Tracing command bufdaemon pid 22 tid 100099 td 0x188a5000 sched_switch(188a5000,104) at sched_switch+0x4c0/frame 0xb5f2bc0 mi_switch(104) at mi_switch+0x1c4/frame 0xb5f2bf8 sleepq_switch(0,1deea04,b5f2c78,188a5000,1999997c,...) at sleepq_switch+0xeb/frame 0xb5f2c18 sleepq_timedwait(1deea04,54,10000,10000,1414f97,...) at sleepq_timedwait+0x3c/frame 0xb5f2c38 _sleep(1deea04,1a9ecc0,54,1444e6d,1999997c,0,0,0,100) at _sleep+0x234/frame 0xb5f2c78 buf_daemon(0,b5f2ce8) at buf_daemon+0xf6/frame 0xb5f2cb8 fork_exit(109b4a0,0,b5f2ce8,0,0,...) at fork_exit+0x68/frame 0xb5f2cd4 fork_trampoline() at 0xffc0340e/frame 0xb5f2cd4 --- kthread start Tracing command bufdaemon pid 22 tid 100100 td 0x18c6fc00 sched_switch(18c6fc00,104) at sched_switch+0x4c0/frame 0x19373bb8 mi_switch(104) at mi_switch+0x1c4/frame 0x19373bec sleepq_switch(0,1a9ffc0,19373c6c,18c6fc00,fffffed8,...) at sleepq_switch+0xeb/frame 0x19373c0c sleepq_timedwait(1a9ffc0,5c,b66aa04,b7c5444,1414f97,...) at sleepq_timedwait+0x3c/frame 0x19373c2c _sleep(1a9ffc0,1a9ff00,25c,14488cd,fffffed8,0,0,0,100) at _sleep+0x234/frame 0x19373c6c bufspace_daemon(1a9edc0,19373ce8) at bufspace_daemon+0x2cb/frame 0x19373cb8 fork_exit(109b780,1a9edc0,19373ce8,0,0,...) at fork_exit+0x68/frame 0x19373cd4 fork_trampoline() at 0xffc0340e/frame 0x19373cd4 --- kthread start Tracing command bufdaemon pid 22 tid 100115 td 0x18797380 sched_switch(18797380,104) at sched_switch+0x4c0/frame 0x204dbbc8 mi_switch(104) at mi_switch+0x1c4/frame 0x204dbbfc sleepq_switch(0,2080de04,204dbc7c,18797380,7fffff6c,...) at sleepq_switch+0xeb/frame 0x204dbc1c sleepq_timedwait(2080de04,54,204dbc70,fee630,1480981,...) at sleepq_timedwait+0x3c/frame 0x204dbc3c _sleep(2080de04,2080dd80,54,14770cb,7fffff6c,0,0,0,100) at _sleep+0x234/frame 0x204dbc7c softdep_flush(18c63a80,204dbce8) at softdep_flush+0x158/frame 0x204dbcb8 fork_exit(125da70,18c63a80,204dbce8,0,0,...) at fork_exit+0x68/frame 0x204dbcd4 fork_trampoline() at 0xffc0340e/frame 0x204dbcd4 --- kthread start Tracing command bufdaemon pid 22 tid 100116 td 0x18797000 sched_switch(18797000,104) at sched_switch+0x4c0/frame 0x204d6bc8 mi_switch(104) at mi_switch+0x1c4/frame 0x204d6bfc sleepq_switch(0,2080dc84,204d6c7c,18797000,7fffff6c,...) at sleepq_switch+0xeb/frame 0x204d6c1c sleepq_timedwait(2080dc84,54,204d6c70,fee630,1480981,...) at sleepq_timedwait+0x3c/frame 0x204d6c3c _sleep(2080dc84,2080dc00,54,14770cb,7fffff6c,0,0,0,100) at _sleep+0x234/frame 0x204d6c7c softdep_flush(18c63000,204d6ce8) at softdep_flush+0x158/frame 0x204d6cb8 fork_exit(125da70,18c63000,204d6ce8,0,0,...) at fork_exit+0x68/frame 0x204d6cd4 fork_trampoline() at 0xffc0340e/frame 0x204d6cd4 --- kthread start Tracing command bufdaemon pid 22 tid 100117 td 0x20b54c00 sched_switch(20b54c00,104) at sched_switch+0x4c0/frame 0x204d1bc8 mi_switch(104) at mi_switch+0x1c4/frame 0x204d1bfc sleepq_switch(0,2080db04,204d1c7c,20b54c00,7fffff6c,...) at sleepq_switch+0xeb/frame 0x204d1c1c sleepq_timedwait(2080db04,54,204d1c70,fee630,1480981,...) at sleepq_timedwait+0x3c/frame 0x204d1c3c _sleep(2080db04,2080da80,54,14770cb,7fffff6c,0,0,0,100) at _sleep+0x234/frame 0x204d1c7c softdep_flush(20a41500,204d1ce8) at softdep_flush+0x158/frame 0x204d1cb8 fork_exit(125da70,20a41500,204d1ce8,0,0,...) at fork_exit+0x68/frame 0x204d1cd4 fork_trampoline() at 0xffc0340e/frame 0x204d1cd4 --- kthread start Tracing command bufdaemon pid 22 tid 100118 td 0x20b54880 sched_switch(20b54880,104) at sched_switch+0x4c0/frame 0x204ccbc8 mi_switch(104) at mi_switch+0x1c4/frame 0x204ccbfc sleepq_switch(0,2080d984,204ccc7c,20b54880,7fffff6c,...) at sleepq_switch+0xeb/frame 0x204ccc1c sleepq_timedwait(2080d984,54,204ccc70,fee630,1480981,...) at sleepq_timedwait+0x3c/frame 0x204ccc3c _sleep(2080d984,2080d900,54,14770cb,7fffff6c,0,0,0,100) at _sleep+0x234/frame 0x204ccc7c softdep_flush(20a40a80,204ccce8) at softdep_flush+0x158/frame 0x204cccb8 fork_exit(125da70,20a40a80,204ccce8,0,0,...) at fork_exit+0x68/frame 0x204cccd4 fork_trampoline() at 0xffc0340e/frame 0x204cccd4 --- kthread start Tracing command vnlru pid 23 tid 100103 td 0x18d08c00 sched_switch(18d08c00,104) at sched_switch+0x4c0/frame 0xb5edb78 mi_switch(104) at mi_switch+0x1c4/frame 0xb5edbac sleepq_switch(0,188acd20,b5edc2c,18d08c00,fffffed8,...) at sleepq_switch+0xeb/frame 0xb5edbcc sleepq_timedwait(188acd20,60,b5edc20,fd3bd5,149c697,...) at sleepq_timedwait+0x3c/frame 0xb5edbec _sleep(188acd20,1aab240,260,1524db1,fffffed8,0,0,0,100) at _sleep+0x234/frame 0xb5edc2c vnlru_proc(0,b5edce8) at vnlru_proc+0xac1/frame 0xb5edcb8 fork_exit(10c3d90,0,b5edce8,0,0,...) at fork_exit+0x68/frame 0xb5edcd4 fork_trampoline() at 0xffc0340e/frame 0xb5edcd4 --- kthread start Tracing command syncer pid 24 tid 100104 td 0x18d08880 sched_switch(18d08880,104) at sched_switch+0x4c0/frame 0xb5e8ba0 mi_switch(104) at mi_switch+0x1c4/frame 0xb5e8bd4 sleepq_switch(b5e8c18,fd2eee,1def2a4,1def2b4,1a913cc,...) at sleepq_switch+0xeb/frame 0xb5e8bf4 sleepq_timedwait(1def2b4,0) at sleepq_timedwait+0x3c/frame 0xb5e8c14 _cv_timedwait_sbt(1def2b4,1def294,fffffed8,0,0,0,100) at _cv_timedwait_sbt+0x157/frame 0xb5e8c50 sched_sync(0,b5e8ce8) at sched_sync+0x652/frame 0xb5e8cb8 fork_exit(10c4d00,0,b5e8ce8,0,0,...) at fork_exit+0x68/frame 0xb5e8cd4 fork_trampoline() at 0xffc0340e/frame 0xb5e8cd4 --- kthread start Tracing command sh pid 1056 tid 100211 td 0x263e4e00 sched_switch(263e4e00,104) at sched_switch+0x4c0/frame 0x2437b9c8 mi_switch(104) at mi_switch+0x1c4/frame 0x2437b9fc sleepq_switch(2ac0,14fd263,6c,0,1d92794,...) at sleepq_switch+0xeb/frame 0x2437ba1c sleepq_catch_signals(2437ba8c,ffddc4,249de348,6c,200246,...) at sleepq_catch_signals+0x278/frame 0x2437ba44 sleepq_wait_sig(249de348,6c) at sleepq_wait_sig+0xe/frame 0x2437ba4c _sleep(249de348,249de3f4,36c,1519f00,0,0,0,0,100) at _sleep+0x244/frame 0x2437ba8c kern_wait6(263e4e00,7,0,0,2437bbe8,30,0,0) at kern_wait6+0x3b8/frame 0x2437bae8 sys_wait4(263e4e00,263e5098,263e4e00,263e4e00,2437bcdc,...) at sys_wait4+0x73/frame 0x2437bc00 syscall(2437bce8,3b,3b,3b,ffbfe7a4,...) at syscall+0x1b8/frame 0x2437bcdc Xint0x80_syscall() at 0xffc033f9/frame 0x2437bcdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01c98, esp = 0xffc07fe8, ebp = 0xffbfeccc --- Tracing command sh pid 83018 tid 100251 td 0x2b243880 sched_switch(2b243880,104) at sched_switch+0x4c0/frame 0x2afe29c8 mi_switch(104) at mi_switch+0x1c4/frame 0x2afe29fc sleepq_switch(980,14fd263,6c,0,1d90654,...) at sleepq_switch+0xeb/frame 0x2afe2a1c sleepq_catch_signals(2afe2a8c,ffddc4,2b13b690,6c,200246,...) at sleepq_catch_signals+0x278/frame 0x2afe2a44 sleepq_wait_sig(2b13b690,6c) at sleepq_wait_sig+0xe/frame 0x2afe2a4c _sleep(2b13b690,2b13b73c,36c,1519f00,0,0,0,0,100) at _sleep+0x244/frame 0x2afe2a8c kern_wait6(2b243880,7,0,0,2afe2be8,30,0,0) at kern_wait6+0x3b8/frame 0x2afe2ae8 sys_wait4(2b243880,2b243b18,2b243880,2b243880,2afe2cdc,...) at sys_wait4+0x73/frame 0x2afe2c00 syscall(2afe2ce8,3b,3b,3b,ffbfeb14,...) at syscall+0x1b8/frame 0x2afe2cdc Xint0x80_syscall() at 0xffc033f9/frame 0x2afe2cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01d98, esp = 0xffc0bfe8, ebp = 0xffbfe808 --- Tracing command ls pid 83021 tid 100248 td 0x27519380 sched_switch(27519380,104) at sched_switch+0x4c0/frame 0x2afd3770 mi_switch(104) at mi_switch+0x1c4/frame 0x2afd37a8 sleepq_switch(27bbbd74,60) at sleepq_switch+0xeb/frame 0x2afd37c8 sleeplk(27bbbde0,142734d,60,33,1) at sleeplk+0xf0/frame 0x2afd37fc lockmgr_slock_hard(27bbbde0,149c697,b64,0) at lockmgr_slock_hard+0x37b/frame 0x2afd3864 lockmgr_lock_flags(27bbbd74,200000,27bbbde0,149c697,b64) at lockmgr_lock_flags+0x1c7/frame 0x2afd3888 ffs_lock(2afd38d8) at ffs_lock+0x89/frame 0x2afd38b4 VOP_LOCK1_APV(1acb2bc,2afd38d8) at VOP_LOCK1_APV+0x38/frame 0x2afd38c8 _vn_lock(27bbbd38,200000,149c697,b64) at _vn_lock+0x49/frame 0x2afd38f8 vget_finish(27bbbd38,200000,2) at vget_finish+0x6a/frame 0x2afd391c cache_fplookup_final_child(12b689c,27bbbd38,2afd3a54,2afd3aa8,18de8404,...) at cache_fplookup_final_child+0x42/frame 0x2afd3940 cache_fplookup(2afd3a54,2afd3a10,2afd3a00) at cache_fplookup+0x859/frame 0x2afd39bc namei(2afd3a54) at namei+0x165/frame 0x2afd3a48 kern_statat(27519380,200,ffffff9c,206c14f4,0,2afd3b28,0) at kern_statat+0x10b/frame 0x2afd3b04 sys_fstatat(27519380,27519618,27519380,27519380,2afd3cdc,...) at sys_fstatat+0x29/frame 0x2afd3c00 syscall(2afd3ce8,3b,3b,3b,206c1424,...) at syscall+0x1b8/frame 0x2afd3cdc Xint0x80_syscall() at 0xffc033f9/frame 0x2afd3cdc --- syscall (10, FreeBSD ELF32, sys_unlink), eip = 0xffc01d18, esp = 0xffc0afe8, ebp = 0xffbfe73c --- Tracing command sh pid 1144 tid 100204 td 0x263e5500 sched_switch(263e5500,104) at sched_switch+0x4c0/frame 0x243859c8 mi_switch(104) at mi_switch+0x1c4/frame 0x243859fc sleepq_switch(c40,14fd263,6c,0,1d90914,...) at sleepq_switch+0xeb/frame 0x24385a1c sleepq_catch_signals(24385a8c,ffddc4,249de9d8,6c,200246,...) at sleepq_catch_signals+0x278/frame 0x24385a44 sleepq_wait_sig(249de9d8,6c) at sleepq_wait_sig+0xe/frame 0x24385a4c _sleep(249de9d8,249dea84,36c,1519f00,0,0,0,0,100) at _sleep+0x244/frame 0x24385a8c kern_wait6(263e5500,7,0,0,24385be8,30,0,0) at kern_wait6+0x3b8/frame 0x24385ae8 sys_wait4(263e5500,263e5798,263e5500,263e5500,24385cdc,...) at sys_wait4+0x73/frame 0x24385c00 syscall(24385ce8,3b,3b,3b,ffbfe614,...) at syscall+0x1b8/frame 0x24385cdc Xint0x80_syscall() at 0xffc033f9/frame 0x24385cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01d98, esp = 0xffc0bfe8, ebp = 0xffbfe808 --- Tracing command sh pid 1196 tid 100203 td 0x263e5880 sched_switch(263e5880,104) at sched_switch+0x4c0/frame 0x2438a9c8 mi_switch(104) at mi_switch+0x1c4/frame 0x2438a9fc sleepq_switch(3340,14fd263,6c,0,1d93014,...) at sleepq_switch+0xeb/frame 0x2438aa1c sleepq_catch_signals(2438aa8c,ffddc4,249ded20,6c,200246,...) at sleepq_catch_signals+0x278/frame 0x2438aa44 sleepq_wait_sig(249ded20,6c) at sleepq_wait_sig+0xe/frame 0x2438aa4c _sleep(249ded20,249dedcc,36c,1519f00,0,0,0,0,100) at _sleep+0x244/frame 0x2438aa8c kern_wait6(263e5880,7,0,0,2438abe8,30,0,0) at kern_wait6+0x3b8/frame 0x2438aae8 sys_wait4(263e5880,263e5b18,263e5880,263e5880,2438acdc,...) at sys_wait4+0x73/frame 0x2438ac00 syscall(2438ace8,3b,3b,3b,ffbfe460,...) at syscall+0x1b8/frame 0x2438acdc Xint0x80_syscall() at 0xffc033f9/frame 0x2438acdc --- syscall (10, FreeBSD ELF32, sys_unlink), eip = 0xffc01d18, esp = 0xffc0afe8, ebp = 0xffbfe73c --- Tracing command awk pid 1198 tid 100214 td 0x263e4380 sched_switch(263e4380,104) at sched_switch+0x4c0/frame 0x2436ca78 mi_switch(104) at mi_switch+0x1c4/frame 0x2436caac sleepq_switch(2000,14fd263,5c,0,1d91cd4,...) at sleepq_switch+0xeb/frame 0x2436cacc sleepq_catch_signals(2436cb3c,ffddc4,20ce8000,5c,20ce8000,...) at sleepq_catch_signals+0x278/frame 0x2436caf4 sleepq_wait_sig(20ce8000,5c) at sleepq_wait_sig+0xe/frame 0x2436cafc _sleep(20ce8000,20ce8188,15c,14fd413,0,0,0,0,100) at _sleep+0x244/frame 0x2436cb3c pipe_read(2365f7f8,2436cbc8,249fe200,0,263e4380) at pipe_read+0x454/frame 0x2436cb7c dofileread(2365f7f8,2436cbc8,ffffffff,ffffffff,0) at dofileread+0x6d/frame 0x2436cbac sys_read(263e4380,263e4618,0,263e4380,1eec,...) at sys_read+0x67/frame 0x2436cc00 syscall(2436cce8,3b,3b,3b,20665654,...) at syscall+0x1b8/frame 0x2436ccdc Xint0x80_syscall() at 0xffc033f9/frame 0x2436ccdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01c98, esp = 0xffc07fe8, ebp = 0xffbfeccc --- Tracing command sh pid 1200 tid 100208 td 0x263e1700 sched_switch(263e1700,104) at sched_switch+0x4c0/frame 0x24344a78 mi_switch(104) at mi_switch+0x1c4/frame 0x24344aac sleepq_switch(30c0,14fd263,5c,0,1d92d94,...) at sleepq_switch+0xeb/frame 0x24344acc sleepq_catch_signals(24344b3c,ffddc4,20ce8340,5c,20ce8340,...) at sleepq_catch_signals+0x278/frame 0x24344af4 sleepq_wait_sig(20ce8340,5c) at sleepq_wait_sig+0xe/frame 0x24344afc _sleep(20ce8340,20ce84c8,15c,14fd413,0,0,0,0,100) at _sleep+0x244/frame 0x24344b3c pipe_read(26391f78,24344bc8,249fe200,0,263e1700) at pipe_read+0x454/frame 0x24344b7c dofileread(26391f78,24344bc8,ffffffff,ffffffff,0) at dofileread+0x6d/frame 0x24344bac sys_read(263e1700,263e1998,0,263e1700,7cb,...) at sys_read+0x67/frame 0x24344c00 syscall(24344ce8,3b,3b,3b,2,...) at syscall+0x1b8/frame 0x24344cdc Xint0x80_syscall() at 0xffc033f9/frame 0x24344cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01e18, esp = 0xffc0dfe8, ebp = 0xffbfe808 --- Tracing command sh pid 2289 tid 100215 td 0x263e4000 sched_switch(263e4000,104) at sched_switch+0x4c0/frame 0x243679c8 mi_switch(104) at mi_switch+0x1c4/frame 0x243679fc sleepq_switch(2cc0,14fd263,6c,0,1d92994,...) at sleepq_switch+0xeb/frame 0x24367a1c sleepq_catch_signals(24367a8c,ffddc4,2abf03b0,6c,200246,...) at sleepq_catch_signals+0x278/frame 0x24367a44 sleepq_wait_sig(2abf03b0,6c) at sleepq_wait_sig+0xe/frame 0x24367a4c _sleep(2abf03b0,2abf045c,36c,1519f00,0,0,0,0,100) at _sleep+0x244/frame 0x24367a8c kern_wait6(263e4000,7,0,0,24367be8,30,0,0) at kern_wait6+0x3b8/frame 0x24367ae8 sys_wait4(263e4000,263e4298,263e4000,263e4000,24367cdc,...) at sys_wait4+0x73/frame 0x24367c00 syscall(24367ce8,3b,3b,3b,ffbfe580,...) at syscall+0x1b8/frame 0x24367cdc Xint0x80_syscall() at 0xffc033f9/frame 0x24367cdc --- syscall (10, FreeBSD ELF32, sys_unlink), eip = 0xffc01d18, esp = 0xffc0afe8, ebp = 0xffbfe73c --- Tracing command sh pid 2290 tid 100106 td 0x18c6f880 sched_switch(18c6f880,104) at sched_switch+0x4c0/frame 0x1930bac0 mi_switch(104) at mi_switch+0x1c4/frame 0x1930baf8 sleepq_switch(2880,14fd263,74,0,1d92554,...) at sleepq_switch+0xeb/frame 0x1930bb18 sleepq_catch_signals(1930bb88,ffddc4,18df0aa8,74,1930bb98,...) at sleepq_catch_signals+0x278/frame 0x1930bb40 sleepq_wait_sig(18df0aa8,74) at sleepq_wait_sig+0xe/frame 0x1930bb48 _sleep(18df0aa8,18df0aec,174,145ce5e,0,0,0,0,100) at _sleep+0x244/frame 0x1930bb88 kern_sigsuspend(18c6f880,0,0,0,0) at kern_sigsuspend+0xca/frame 0x1930bbd0 sys_sigsuspend(18c6f880,18c6fb18,18c6f880,18c6f880,1930bcdc,...) at sys_sigsuspend+0x36/frame 0x1930bc00 syscall(1930bce8,3b,3b,3b,ffbfe260,...) at syscall+0x1b8/frame 0x1930bcdc Xint0x80_syscall() at 0xffc033f9/frame 0x1930bcdc --- syscall (10, FreeBSD ELF32, sys_unlink), eip = 0xffc01d18, esp = 0xffc0afe8, ebp = 0xffbfe73c --- Tracing command tee pid 2291 tid 100144 td 0x20d34380 sched_switch(20d34380,104) at sched_switch+0x4c0/frame 0x20f42a78 mi_switch(104) at mi_switch+0x1c4/frame 0x20f42aac sleepq_switch(1040,14fd263,5c,0,1d90d14,...) at sleepq_switch+0xeb/frame 0x20f42acc sleepq_catch_signals(20f42b3c,ffddc4,2ac5e1a0,5c,2ac5e1a0,...) at sleepq_catch_signals+0x278/frame 0x20f42af4 sleepq_wait_sig(2ac5e1a0,5c) at sleepq_wait_sig+0xe/frame 0x20f42afc _sleep(2ac5e1a0,2ac5e328,15c,14fd413,0,0,0,0,100) at _sleep+0x244/frame 0x20f42b3c pipe_read(2ac77c6c,20f42bc8,18891500,0,20d34380) at pipe_read+0x454/frame 0x20f42b7c dofileread(2ac77c6c,20f42bc8,ffffffff,ffffffff,0) at dofileread+0x6d/frame 0x20f42bac sys_read(20d34380,20d34618,20d34380,20d34380,20f42cdc,...) at sys_read+0x67/frame 0x20f42c00 syscall(20f42ce8,3b,3b,3b,3,...) at syscall+0x1b8/frame 0x20f42cdc Xint0x80_syscall() at 0xffc033f9/frame 0x20f42cdc --- syscall (10, FreeBSD ELF32, sys_unlink), eip = 0xffc01d18, esp = 0xffc0afe8, ebp = 0xffbfe73c --- Tracing command perf pid 77065 tid 100186 td 0x24402e00 sched_switch(24402e00,104) at sched_switch+0x4c0/frame 0x24254730 mi_switch(104) at mi_switch+0x1c4/frame 0x24254764 sleepq_switch(27bbbd74,60) at sleepq_switch+0xeb/frame 0x24254784 sleeplk(27bbbde0,142734d,60,33,0) at sleeplk+0xf0/frame 0x242547b8 lockmgr_xlock_hard(27bbbde0,14cd14b,1057,0,27bbbd38,...) at lockmgr_xlock_hard+0x2fc/frame 0x24254824 lockmgr_lock_flags(27bbbd74,80000,27bbbde0,14cd14b,1057) at lockmgr_lock_flags+0x1b3/frame 0x24254848 ffs_lock(24254898) at ffs_lock+0x89/frame 0x24254874 VOP_LOCK1_APV(1acb2bc,24254898) at VOP_LOCK1_APV+0x38/frame 0x24254888 _vn_lock(27bbbd38,80000,14cd14b,1057) at _vn_lock+0x49/frame 0x242548b8 cache_fplookup_final_modifying(1d1698,5010003,1,1acb2bc,24254930,...) at cache_fplookup_final_modifying+0x23c/frame 0x24254900 cache_fplookup_noentry(12b689c,27bbbd38,24254b18,24254b6c,3073080c,...) at cache_fplookup_noentry+0x99/frame 0x24254938 cache_fplookup(24254b18,24254a08,242549f8) at cache_fplookup+0x559/frame 0x242549b4 namei(24254b18) at namei+0x165/frame 0x24254a3c kern_funlinkat(24402e00,ffffff9c,ffbfe694,ffffff38,0,0,0,0) at kern_funlinkat+0x115/frame 0x24254bd8 sys_unlink(24402e00,24403098,0,24402e00,1ef3,...) at sys_unlink+0x1f/frame 0x24254c00 syscall(24254ce8,3b,3b,3b,ffbfe7a0,...) at syscall+0x1b8/frame 0x24254cdc Xint0x80_syscall() at 0xffc033f9/frame 0x24254cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01d98, esp = 0xffc0bfe8, ebp = 0xffbfe808 --- Tracing command perf pid 77066 tid 100220 td 0x263e1000 (CPU 1) kdb_enter(1436862,143155a) at kdb_enter+0x35/frame 0x243af64c hardclock(1,0) at hardclock+0x1b2/frame 0x243af67c handleevents(0) at handleevents+0xf5/frame 0x243af6b4 timercb(1e34f0c,0) at timercb+0x18c/frame 0x243af6e0 lapic_handle_timer(243af708,8,28,28,f7810f0,...) at lapic_handle_timer+0x7b/frame 0x243af6fc Xtimerint() at 0xffc04923/frame 0x243af6fc --- interrupt, eip = 0x1282c30, esp = 0x243af748, ebp = 0x243af854 --- ufs_lookup_ino(27bbbd38,243af8ec,243afb6c,0) at ufs_lookup_ino+0x6d0/frame 0x243af854 ufs_lookup(243af890) at ufs_lookup+0x16/frame 0x243af86c VOP_CACHEDLOOKUP_APV(1acb2bc,243af890) at VOP_CACHEDLOOKUP_APV+0x4d/frame 0x243af880 vfs_cache_lookup(243af8dc) at vfs_cache_lookup+0x93/frame 0x243af8ac VOP_LOOKUP_APV(1acb2bc,243af8dc) at VOP_LOOKUP_APV+0x4d/frame 0x243af8c0 cache_fplookup_final_modifying(1d1696,5010003,1,1acb2bc,243af930,...) at cache_fplookup_final_modifying+0x2af/frame 0x243af900 cache_fplookup_noentry(12b689c,27bbbd38,243afb18,243afb6c,20ce900c,...) at cache_fplookup_noentry+0x99/frame 0x243af938 cache_fplookup(243afb18,243afa08,243af9f8) at cache_fplookup+0x559/frame 0x243af9b4 namei(243afb18) at namei+0x165/frame 0x243afa3c kern_funlinkat(263e1000,ffffff9c,ffbfe694,ffffff38,0,0,0,0) at kern_funlinkat+0x115/frame 0x243afbd8 sys_unlink(263e1000,263e1298,0,263e1000,1ef2,...) at sys_unlink+0x1f/frame 0x243afc00 syscall(243afce8,3b,3b,3b,ffbfe7a0,...) at syscall+0x1b8/frame 0x243afcdc Xint0x80_syscall() at 0xffc033f9/frame 0x243afcdc --- syscall (10, FreeBSD ELF32, sys_unlink), eip = 0xffc01d18, esp = 0xffc0afe8, ebp = 0xffbfe73c --- Tracing command md10 pid 2437 tid 100213 td 0x263e4700 sched_switch(263e4700,104) at sched_switch+0x4c0/frame 0x24371bd8 mi_switch(104) at mi_switch+0x1c4/frame 0x24371c10 sleepq_switch(2ac47000,5c) at sleepq_switch+0xeb/frame 0x24371c30 _sleep(2ac47000,2ac47028,25c,14be8e7,0,0,0,0,100) at _sleep+0x2bf/frame 0x24371c70 md_kthread(2ac47000,24371ce8) at md_kthread+0xc1/frame 0x24371cb8 fork_exit(be7e70,2ac47000,24371ce8,0,0,...) at fork_exit+0x68/frame 0x24371cd4 fork_trampoline() at 0xffc0340e/frame 0x24371cd4 --- kthread start Tracing command devd pid 393 tid 100127 td 0x20b54500 sched_switch(20b54500,104) at sched_switch+0x4c0/frame 0x20e13940 mi_switch(104) at mi_switch+0x1c4/frame 0x20e13978 sleepq_switch(31c0,0,0,0,1d92e94,...) at sleepq_switch+0xeb/frame 0x20e13998 sleepq_catch_signals(20b54500,20e139e8,fd2eee,18c7a360,18c7a364,...) at sleepq_catch_signals+0x278/frame 0x20e139c0 sleepq_timedwait_sig(18c7a364,0) at sleepq_timedwait_sig+0x14/frame 0x20e139e4 _cv_timedwait_sig_sbt(18c7a364,18c7a350,1a13b89,1f08,c0000000,3,200) at _cv_timedwait_sig_sbt+0x15a/frame 0x20e13a20 seltdwait(1f08,c0000000,3) at seltdwait+0x63/frame 0x20e13a50 kern_select(20b54500,6,ffbfcd3c,0,0,20e13bf0,20) at kern_select+0x749/frame 0x20e13bcc sys_select(20b54500,20b54798,0,20b54500,38c,...) at sys_select+0x3f/frame 0x20e13c00 syscall(20e13ce8,3b,3b,3b,ffbfcdbc,...) at syscall+0x1b8/frame 0x20e13cdc Xint0x80_syscall() at 0xffc033f9/frame 0x20e13cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01c98, esp = 0xffc07fe8, ebp = 0xffbfeccc --- Tracing command nfsuserd pid 462 tid 100112 td 0x1829d000 sched_switch(1829d000,104) at sched_switch+0x4c0/frame 0xb581ac0 mi_switch(104) at mi_switch+0x1c4/frame 0xb581af8 sleepq_switch(1300,14fd263,74,0,1d90fd4,...) at sleepq_switch+0xeb/frame 0xb581b18 sleepq_catch_signals(b581b88,ffddc4,18255418,74,b581b98,...) at sleepq_catch_signals+0x278/frame 0xb581b40 sleepq_wait_sig(18255418,74) at sleepq_wait_sig+0xe/frame 0xb581b48 _sleep(18255418,1825545c,174,145ce5e,0,0,0,0,100) at _sleep+0x244/frame 0xb581b88 kern_sigsuspend(1829d000,0,0,0,0) at kern_sigsuspend+0xca/frame 0xb581bd0 sys_sigsuspend(1829d000,1829d298,1829d000,1829d000,b581cdc,...) at sys_sigsuspend+0x36/frame 0xb581c00 syscall(b581ce8,3b,3b,3b,4,...) at syscall+0x1b8/frame 0xb581cdc Xint0x80_syscall() at 0xffc033f9/frame 0xb581cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01e18, esp = 0xffc0dfe8, ebp = 0xffbfe808 --- Tracing command nfsuserd pid 463 tid 100124 td 0x20cfd880 sched_switch(20cfd880,104) at sched_switch+0x4c0/frame 0xb577940 mi_switch(104) at mi_switch+0x1c4/frame 0xb577978 sleepq_switch(2ac0,0,0,0,1d92794,...) at sleepq_switch+0xeb/frame 0xb577998 sleepq_catch_signals(20cfd880,b5779e8,fd2eee,18db8f20,18db8f24,...) at sleepq_catch_signals+0x278/frame 0xb5779c0 sleepq_timedwait_sig(18db8f24,0) at sleepq_timedwait_sig+0x14/frame 0xb5779e4 _cv_timedwait_sig_sbt(18db8f24,18db8f10,abde82eb,1f10,e0000000,1,200) at _cv_timedwait_sig_sbt+0x15a/frame 0xb577a20 seltdwait(1f10,e0000000,1) at seltdwait+0x63/frame 0xb577a50 kern_select(20cfd880,4,ffbfdba0,0,0,b577bf0,20) at kern_select+0x749/frame 0xb577bcc sys_select(20cfd880,20cfdb18,0,20cfd880,18f7,...) at sys_select+0x3f/frame 0xb577c00 syscall(b577ce8,3b,3b,3b,ffbfdba0,...) at syscall+0x1b8/frame 0xb577cdc Xint0x80_syscall() at 0xffc033f9/frame 0xb577cdc --- syscall (10, FreeBSD ELF32, sys_unlink), eip = 0xffc01d18, esp = 0xffc0afe8, ebp = 0xffbfe73c --- Tracing command nfsuserd pid 464 tid 100107 td 0x18798180 sched_switch(18798180,104) at sched_switch+0x4c0/frame 0x204ef940 mi_switch(104) at mi_switch+0x1c4/frame 0x204ef978 sleepq_switch(1a80,0,0,0,1d91754,...) at sleepq_switch+0xeb/frame 0x204ef998 sleepq_catch_signals(18798180,204ef9e8,fd2eee,21048ee0,21048ee4,...) at sleepq_catch_signals+0x278/frame 0x204ef9c0 sleepq_timedwait_sig(21048ee4,0) at sleepq_timedwait_sig+0x14/frame 0x204ef9e4 _cv_timedwait_sig_sbt(21048ee4,21048ed0,fa34847d,1f0f,e0000000,1,200) at _cv_timedwait_sig_sbt+0x15a/frame 0x204efa20 seltdwait(1f0f,e0000000,1) at seltdwait+0x63/frame 0x204efa50 kern_select(18798180,4,ffbfdba0,0,0,204efbf0,20) at kern_select+0x749/frame 0x204efbcc sys_select(18798180,18798418,204efc50,18798180,22fe030,...) at sys_select+0x3f/frame 0x204efc00 syscall(204efce8,3b,3b,3b,ffbfdba0,...) at syscall+0x1b8/frame 0x204efcdc Xint0x80_syscall() at 0xffc033f9/frame 0x204efcdc --- syscall (10, FreeBSD ELF32, sys_unlink), eip = 0xffc01d18, esp = 0xffc0afe8, ebp = 0xffbfe73c --- Tracing command nfsuserd pid 465 tid 100131 td 0x20b54180 sched_switch(20b54180,104) at sched_switch+0x4c0/frame 0x20e0e940 mi_switch(104) at mi_switch+0x1c4/frame 0x20e0e978 sleepq_switch(a80,0,0,0,1d90754,...) at sleepq_switch+0xeb/frame 0x20e0e998 sleepq_catch_signals(20b54180,20e0e9e8,fd2eee,21048ea0,21048ea4,...) at sleepq_catch_signals+0x278/frame 0x20e0e9c0 sleepq_timedwait_sig(21048ea4,0) at sleepq_timedwait_sig+0x14/frame 0x20e0e9e4 _cv_timedwait_sig_sbt(21048ea4,21048e90,940fa775,1f10,e0000000,1,200) at _cv_timedwait_sig_sbt+0x15a/frame 0x20e0ea20 seltdwait(1f10,e0000000,1) at seltdwait+0x63/frame 0x20e0ea50 kern_select(20b54180,4,ffbfdba0,0,0,20e0ebf0,20) at kern_select+0x749/frame 0x20e0ebcc sys_select(20b54180,20b54418,0,20b54180,48f,...) at sys_select+0x3f/frame 0x20e0ec00 syscall(20e0ece8,3b,3b,3b,ffbfdba0,...) at syscall+0x1b8/frame 0x20e0ecdc Xint0x80_syscall() at 0xffc033f9/frame 0x20e0ecdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01c98, esp = 0xffc07fe8, ebp = 0xffbfeccc --- Tracing command nfsuserd pid 466 tid 100132 td 0x20b53e00 sched_switch(20b53e00,104) at sched_switch+0x4c0/frame 0x20e09940 mi_switch(104) at mi_switch+0x1c4/frame 0x20e09978 sleepq_switch(ac0,0,0,0,1d90794,...) at sleepq_switch+0xeb/frame 0x20e09998 sleepq_catch_signals(20b53e00,20e099e8,fd2eee,18db8fa0,18db8fa4,...) at sleepq_catch_signals+0x278/frame 0x20e099c0 sleepq_timedwait_sig(18db8fa4,0) at sleepq_timedwait_sig+0x14/frame 0x20e099e4 _cv_timedwait_sig_sbt(18db8fa4,18db8f90,a90d9bb3,1f10,e0000000,1,200) at _cv_timedwait_sig_sbt+0x15a/frame 0x20e09a20 seltdwait(1f10,e0000000,1) at seltdwait+0x63/frame 0x20e09a50 kern_select(20b53e00,4,ffbfdba0,0,0,20e09bf0,20) at kern_select+0x749/frame 0x20e09bcc sys_select(20b53e00,20b54098,0,20b53e00,fd1,...) at sys_select+0x3f/frame 0x20e09c00 syscall(20e09ce8,3b,3b,3b,ffbfdba0,...) at syscall+0x1b8/frame 0x20e09cdc Xint0x80_syscall() at 0xffc033f9/frame 0x20e09cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01e18, esp = 0xffc0dfe8, ebp = 0xffbfe808 --- Tracing command nfscbd pid 549 tid 100128 td 0x20d35c00 sched_switch(20d35c00,104) at sched_switch+0x4c0/frame 0x205b1a50 mi_switch(104) at mi_switch+0x1c4/frame 0x205b1a84 sleepq_switch(3240,14fd263,68,0,1d92f14,...) at sleepq_switch+0xeb/frame 0x205b1aa4 sleepq_catch_signals(205b1b14,ffddc4,20d179b0,68,1,...) at sleepq_catch_signals+0x278/frame 0x205b1acc sleepq_wait_sig(20d179b0,68) at sleepq_wait_sig+0xe/frame 0x205b1ad4 _sleep(20d179b0,20d178f0,168,146b249,0,0,0,0,100) at _sleep+0x244/frame 0x205b1b14 solisten_dequeue(20d178f0,205b1b98,1) at solisten_dequeue+0x77/frame 0x205b1b54 kern_accept4(20d35c00,3,205b1bd0,205b1bd8,1,205b1bd4) at kern_accept4+0x154/frame 0x205b1bb0 accept1(ffbfed54,ffbfedb4,1) at accept1+0x4c/frame 0x205b1bec sys_accept(20d35c00,20d35e98,20d35c00,20d35c00,205b1cdc,...) at sys_accept+0x18/frame 0x205b1c00 syscall(205b1ce8,3b,3b,3b,ffbfec54,...) at syscall+0x1b8/frame 0x205b1cdc Xint0x80_syscall() at 0xffc033f9/frame 0x205b1cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01d98, esp = 0xffc0bfe8, ebp = 0xffbfe808 --- Tracing command nfscbd pid 550 tid 100126 td 0x20cfd180 sched_switch(20cfd180,104) at sched_switch+0x4c0/frame 0x205eb8b0 mi_switch(104) at mi_switch+0x1c4/frame 0x205eb8e8 sleepq_switch(3e80,0,0,0,1d93b54,...) at sleepq_switch+0xeb/frame 0x205eb908 sleepq_catch_signals(20cfd180,205eb958,fd2eee,1824e0d0,1889aa50,...) at sleepq_catch_signals+0x278/frame 0x205eb930 sleepq_timedwait_sig(1889aa50,0) at sleepq_timedwait_sig+0x14/frame 0x205eb954 _cv_timedwait_sig_sbt(1889aa50,1824e0c0,fffffa38,4,0,0,100) at _cv_timedwait_sig_sbt+0x15a/frame 0x205eb990 svc_run_internal(20d34e00,1824e0c0,1824e0d0,1824e11c,0,...) at svc_run_internal+0x95a/frame 0x205eba60 svc_run(1824e000) at svc_run+0x192/frame 0x205eba90 nfscbd_nfsd(20cfd180,205ebb48) at nfscbd_nfsd+0xd0/frame 0x205ebb20 nfssvc_nfscl(20cfd180,20cfd418) at nfssvc_nfscl+0x233/frame 0x205ebbe8 sys_nfssvc(20cfd180,20cfd418,20cfd180,20cfd180,205ebcdc,...) at sys_nfssvc+0x92/frame 0x205ebc00 syscall(205ebce8,3b,3b,3b,ffbfee18,...) at syscall+0x1b8/frame 0x205ebcdc Xint0x80_syscall() at 0xffc033f9/frame 0x205ebcdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01e18, esp = 0xffc0dfe8, ebp = 0xffbfe808 --- Tracing command nfscbd pid 550 tid 100134 td 0x20d35500 sched_switch(20d35500,104) at sched_switch+0x4c0/frame 0x205a7b38 mi_switch(104) at mi_switch+0x1c4/frame 0x205a7b6c sleepq_switch(31c0,20d35500,0,0,1d92e94,...) at sleepq_switch+0xeb/frame 0x205a7b8c sleepq_catch_signals(205a7bf4,f92178,12b917d0,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x205a7bb4 sleepq_wait_sig(12b917d0,0) at sleepq_wait_sig+0xe/frame 0x205a7bbc _cv_wait_sig(12b917d0,1824e0c0) at _cv_wait_sig+0x138/frame 0x205a7bf4 svc_run_internal(205a7cd4,fb54a8,1824e0c0,205a7ce8,12219a0,...) at svc_run_internal+0x999/frame 0x205a7cb0 svc_thread_start(1824e0c0,205a7ce8) at svc_thread_start+0xd/frame 0x205a7cb8 fork_exit(12219a0,1824e0c0,205a7ce8,3b,3b,...) at fork_exit+0x68/frame 0x205a7cd4 fork_trampoline() at 0xffc0340e/frame 0x205a7cd4 --- kthread start Tracing command nfscbd pid 550 tid 100135 td 0x20d35180 sched_switch(20d35180,104) at sched_switch+0x4c0/frame 0x205a2b38 mi_switch(104) at mi_switch+0x1c4/frame 0x205a2b6c sleepq_switch(1e40,20d35180,0,0,1d91b14,...) at sleepq_switch+0xeb/frame 0x205a2b8c sleepq_catch_signals(205a2bf4,f92178,1889a9d0,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x205a2bb4 sleepq_wait_sig(1889a9d0,0) at sleepq_wait_sig+0xe/frame 0x205a2bbc _cv_wait_sig(1889a9d0,1824e0c0) at _cv_wait_sig+0x138/frame 0x205a2bf4 svc_run_internal(205a2cd4,fb54a8,1824e0c0,205a2ce8,12219a0,...) at svc_run_internal+0x999/frame 0x205a2cb0 svc_thread_start(1824e0c0,205a2ce8) at svc_thread_start+0xd/frame 0x205a2cb8 fork_exit(12219a0,1824e0c0,205a2ce8,3b,3b,...) at fork_exit+0x68/frame 0x205a2cd4 fork_trampoline() at 0xffc0340e/frame 0x205a2cd4 --- kthread start Tracing command nfscbd pid 550 tid 100136 td 0x20d34e00 sched_switch(20d34e00,104) at sched_switch+0x4c0/frame 0x2059db38 mi_switch(104) at mi_switch+0x1c4/frame 0x2059db6c sleepq_switch(11c0,20d34e00,0,0,1d90e94,...) at sleepq_switch+0xeb/frame 0x2059db8c sleepq_catch_signals(2059dbf4,f92178,12b91750,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x2059dbb4 sleepq_wait_sig(12b91750,0) at sleepq_wait_sig+0xe/frame 0x2059dbbc _cv_wait_sig(12b91750,1824e0c0) at _cv_wait_sig+0x138/frame 0x2059dbf4 svc_run_internal(2059dcd4,fb54a8,1824e0c0,2059dce8,12219a0,...) at svc_run_internal+0x999/frame 0x2059dcb0 svc_thread_start(1824e0c0,2059dce8) at svc_thread_start+0xd/frame 0x2059dcb8 fork_exit(12219a0,1824e0c0,2059dce8,3b,3b,...) at fork_exit+0x68/frame 0x2059dcd4 fork_trampoline() at 0xffc0340e/frame 0x2059dcd4 --- kthread start Tracing command syslogd pid 592 tid 100123 td 0x18d07000 sched_switch(18d07000,104) at sched_switch+0x4c0/frame 0x205b6a58 mi_switch(104) at mi_switch+0x1c4/frame 0x205b6a90 sleepq_switch(235b1290,5d) at sleepq_switch+0xeb/frame 0x205b6ab0 _sleep(235b1290,235b1248,5d,1510f93,0,0,0,0,100) at _sleep+0x2bf/frame 0x205b6af0 bufobj_wwait(235b1248,0,0) at bufobj_wwait+0x93/frame 0x205b6b30 ffs_syncvnode(235b1178,1,0) at ffs_syncvnode+0x3e6/frame 0x205b6b80 ffs_fsync(205b6bd0) at ffs_fsync+0x17/frame 0x205b6ba0 VOP_FSYNC_APV(1acb2bc,205b6bd0,18d07000,18dc25dc,20a40a80,...) at VOP_FSYNC_APV+0x4d/frame 0x205b6bb4 kern_fsync(18d07000,e,1) at kern_fsync+0x1ba/frame 0x205b6bec sys_fsync(18d07000,18d07298,0,18d07000,1ef4,...) at sys_fsync+0x12/frame 0x205b6c00 syscall(205b6ce8,3b,3b,3b,2068a000,...) at syscall+0x1b8/frame 0x205b6cdc Xint0x80_syscall() at 0xffc033f9/frame 0x205b6cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01d98, esp = 0xffc0bfe8, ebp = 0xffbfe808 --- Tracing command watchdogd pid 601 tid 100142 td 0x20d34a80 sched_switch(20d34a80,104) at sched_switch+0x4c0/frame 0x205989c8 mi_switch(104) at mi_switch+0x1c4/frame 0x205989fc sleepq_switch(1180,14fd263,6c,0,1d90e54,...) at sleepq_switch+0xeb/frame 0x20598a1c sleepq_catch_signals(20598a8c,ffddc4,20cad690,6c,200246,...) at sleepq_catch_signals+0x278/frame 0x20598a44 sleepq_wait_sig(20cad690,6c) at sleepq_wait_sig+0xe/frame 0x20598a4c _sleep(20cad690,20cad73c,36c,1519f00,0,0,0,0,100) at _sleep+0x244/frame 0x20598a8c kern_wait6(20d34a80,0,1444a,0,20598be8,30,0,0) at kern_wait6+0x3b8/frame 0x20598ae8 sys_wait4(20d34a80,20d34d18,20d34a80,20d34a80,20598cdc,...) at sys_wait4+0x73/frame 0x20598c00 syscall(20598ce8,3b,3b,3b,1444a,...) at syscall+0x1b8/frame 0x20598cdc Xint0x80_syscall() at 0xffc033f9/frame 0x20598cdc --- syscall (10, FreeBSD ELF32, sys_unlink), eip = 0xffc01d18, esp = 0xffc0afe8, ebp = 0xffbfe73c --- Tracing command rpcbind pid 651 tid 100143 td 0x20d34700 sched_switch(20d34700,104) at sched_switch+0x4c0/frame 0x20f47978 mi_switch(104) at mi_switch+0x1c4/frame 0x20f479b0 sleepq_switch(a80,0,0,0,1d90754,...) at sleepq_switch+0xeb/frame 0x20f479d0 sleepq_catch_signals(20d34700,20f47a20,fd2eee,18db8ea0,18db8ea4,...) at sleepq_catch_signals+0x278/frame 0x20f479f8 sleepq_timedwait_sig(18db8ea4,0) at sleepq_timedwait_sig+0x14/frame 0x20f47a1c _cv_timedwait_sig_sbt(18db8ea4,18db8e90,ff94e648,1ef4,e0000000,1,200) at _cv_timedwait_sig_sbt+0x15a/frame 0x20f47a58 seltdwait(1ef4,e0000000,1) at seltdwait+0x63/frame 0x20f47a88 kern_poll(20d34700,ffbfcbd4,8,20f47bf4,0) at kern_poll+0x450/frame 0x20f47bd8 sys_poll(20d34700,20d34998,0,20d34700,18ab,...) at sys_poll+0x4d/frame 0x20f47c00 syscall(20f47ce8,3b,3b,3b,40,...) at syscall+0x1b8/frame 0x20f47cdc Xint0x80_syscall() at 0xffc033f9/frame 0x20f47cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01d98, esp = 0xffc0bfe8, ebp = 0xffbfe808 --- Tracing command mountd pid 675 tid 100125 td 0x20cfd500 sched_switch(20cfd500,104) at sched_switch+0x4c0/frame 0xb572940 mi_switch(104) at mi_switch+0x1c4/frame 0xb572978 sleepq_switch(3c40,0,0,0,1d93914,...) at sleepq_switch+0xeb/frame 0xb572998 sleepq_catch_signals(20cfd500,b5729e8,fd2eee,1889d520,1889d524,...) at sleepq_catch_signals+0x278/frame 0xb5729c0 sleepq_timedwait_sig(1889d524,0) at sleepq_timedwait_sig+0x14/frame 0xb5729e4 _cv_timedwait_sig_sbt(1889d524,1889d510,4451c4a9,1ef4,4000000,0,200) at _cv_timedwait_sig_sbt+0x15a/frame 0xb572a20 seltdwait(1ef4,4000000,0) at seltdwait+0x63/frame 0xb572a50 kern_select(20cfd500,a,ffbfe824,0,0,b572bf0,20) at kern_select+0x749/frame 0xb572bcc sys_select(20cfd500,20cfd798,0,20cfd500,1ec1,...) at sys_select+0x3f/frame 0xb572c00 syscall(b572ce8,3b,3b,3b,ffbfe8a4,...) at syscall+0x1b8/frame 0xb572cdc Xint0x80_syscall() at 0xffc033f9/frame 0xb572cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01d98, esp = 0xffc0bfe8, ebp = 0xffbfe808 --- Tracing command nfsd pid 682 tid 100120 td 0x20cfdc00 sched_switch(20cfdc00,104) at sched_switch+0x4c0/frame 0xb57c978 mi_switch(104) at mi_switch+0x1c4/frame 0xb57c9ac sleepq_switch(2a80,20cfdc00,0,0,1d92754,...) at sleepq_switch+0xeb/frame 0xb57c9cc sleepq_catch_signals(b57ca34,f92178,21048e24,0,1414b23,...) at sleepq_catch_signals+0x278/frame 0xb57c9f4 sleepq_wait_sig(21048e24,0) at sleepq_wait_sig+0xe/frame 0xb57c9fc _cv_wait_sig(21048e24,21048e10) at _cv_wait_sig+0x138/frame 0xb57ca34 seltdwait(ffffffff,0,0) at seltdwait+0x7f/frame 0xb57ca50 kern_select(20cfdc00,7,ffbfeb44,0,0,0,20) at kern_select+0x749/frame 0xb57cbcc sys_select(20cfdc00,20cfde98,20cfdc00,20cfdc00,b57ccdc,...) at sys_select+0x3f/frame 0xb57cc00 syscall(b57cce8,3b,3b,3b,ffbfebc4,...) at syscall+0x1b8/frame 0xb57ccdc Xint0x80_syscall() at 0xffc033f9/frame 0xb57ccdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01e18, esp = 0xffc0dfe8, ebp = 0xffbfe808 --- Tracing command nfsd pid 685 tid 100111 td 0x18797a80 sched_switch(18797a80,104) at sched_switch+0x4c0/frame 0x204e5430 mi_switch(104) at mi_switch+0x1c4/frame 0x204e5468 sleepq_switch(3e40,0,0,0,1d93b14,...) at sleepq_switch+0xeb/frame 0x204e5488 sleepq_catch_signals(18797a80,204e54d8,fd2eee,187930d0,1889a950,...) at sleepq_catch_signals+0x278/frame 0x204e54b0 sleepq_timedwait_sig(1889a950,0) at sleepq_timedwait_sig+0x14/frame 0x204e54d4 _cv_timedwait_sig_sbt(1889a950,187930c0,fffffa38,4,0,0,100) at _cv_timedwait_sig_sbt+0x15a/frame 0x204e5510 svc_run_internal(24403180,187930c0,187930d0,1879311c,0,...) at svc_run_internal+0x95a/frame 0x204e55e0 svc_run(18793000) at svc_run+0x192/frame 0x204e5610 nfsrvd_nfsd(18797a80,204e57c8) at nfsrvd_nfsd+0x2e8/frame 0x204e5740 nfssvc_nfsd(18797a80,18797d18) at nfssvc_nfsd+0x4d8/frame 0x204e5be8 sys_nfssvc(18797a80,18797d18,18797a80,18797a80,204e5cdc,...) at sys_nfssvc+0x92/frame 0x204e5c00 syscall(204e5ce8,3b,3b,3b,ffbfec94,...) at syscall+0x1b8/frame 0x204e5cdc Xint0x80_syscall() at 0xffc033f9/frame 0x204e5cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01e18, esp = 0xffc0dfe8, ebp = 0xffbfe808 --- Tracing command nfsd pid 685 tid 100154 td 0x241f9c00 sched_switch(241f9c00,104) at sched_switch+0x4c0/frame 0x20f38b38 mi_switch(104) at mi_switch+0x1c4/frame 0x20f38b6c sleepq_switch(3180,241f9c00,0,0,1d92e54,...) at sleepq_switch+0xeb/frame 0x20f38b8c sleepq_catch_signals(20f38bf4,f92178,12b916d0,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x20f38bb4 sleepq_wait_sig(12b916d0,0) at sleepq_wait_sig+0xe/frame 0x20f38bbc _cv_wait_sig(12b916d0,187930c0) at _cv_wait_sig+0x138/frame 0x20f38bf4 svc_run_internal(20f38cd4,fb54a8,187930c0,20f38ce8,12219a0,...) at svc_run_internal+0x999/frame 0x20f38cb0 svc_thread_start(187930c0,20f38ce8) at svc_thread_start+0xd/frame 0x20f38cb8 fork_exit(12219a0,187930c0,20f38ce8,3b,3b,...) at fork_exit+0x68/frame 0x20f38cd4 fork_trampoline() at 0xffc0340e/frame 0x20f38cd4 --- kthread start Tracing command nfsd pid 685 tid 100155 td 0x241f9880 sched_switch(241f9880,104) at sched_switch+0x4c0/frame 0x20f33b38 mi_switch(104) at mi_switch+0x1c4/frame 0x20f33b6c sleepq_switch(780,241f9880,0,0,1d90454,...) at sleepq_switch+0xeb/frame 0x20f33b8c sleepq_catch_signals(20f33bf4,f92178,18d5ced0,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x20f33bb4 sleepq_wait_sig(18d5ced0,0) at sleepq_wait_sig+0xe/frame 0x20f33bbc _cv_wait_sig(18d5ced0,187930c0) at _cv_wait_sig+0x138/frame 0x20f33bf4 svc_run_internal(20f33cd4,fb54a8,187930c0,20f33ce8,12219a0,...) at svc_run_internal+0x999/frame 0x20f33cb0 svc_thread_start(187930c0,20f33ce8) at svc_thread_start+0xd/frame 0x20f33cb8 fork_exit(12219a0,187930c0,20f33ce8,3b,3b,...) at fork_exit+0x68/frame 0x20f33cd4 fork_trampoline() at 0xffc0340e/frame 0x20f33cd4 --- kthread start Tracing command nfsd pid 685 tid 100156 td 0x241f9500 sched_switch(241f9500,104) at sched_switch+0x4c0/frame 0x20f2eb38 mi_switch(104) at mi_switch+0x1c4/frame 0x20f2eb6c sleepq_switch(2780,241f9500,0,0,1d92454,...) at sleepq_switch+0xeb/frame 0x20f2eb8c sleepq_catch_signals(20f2ebf4,f92178,18d5ce50,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x20f2ebb4 sleepq_wait_sig(18d5ce50,0) at sleepq_wait_sig+0xe/frame 0x20f2ebbc _cv_wait_sig(18d5ce50,187930c0) at _cv_wait_sig+0x138/frame 0x20f2ebf4 svc_run_internal(20f2ecd4,fb54a8,187930c0,20f2ece8,12219a0,...) at svc_run_internal+0x999/frame 0x20f2ecb0 svc_thread_start(187930c0,20f2ece8) at svc_thread_start+0xd/frame 0x20f2ecb8 fork_exit(12219a0,187930c0,20f2ece8,3b,3b,...) at fork_exit+0x68/frame 0x20f2ecd4 fork_trampoline() at 0xffc0340e/frame 0x20f2ecd4 --- kthread start Tracing command nfsd pid 685 tid 100157 td 0x241f9180 sched_switch(241f9180,104) at sched_switch+0x4c0/frame 0x20f29b38 mi_switch(104) at mi_switch+0x1c4/frame 0x20f29b6c sleepq_switch(1180,241f9180,0,0,1d90e54,...) at sleepq_switch+0xeb/frame 0x20f29b8c sleepq_catch_signals(20f29bf4,f92178,12b91650,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x20f29bb4 sleepq_wait_sig(12b91650,0) at sleepq_wait_sig+0xe/frame 0x20f29bbc _cv_wait_sig(12b91650,187930c0) at _cv_wait_sig+0x138/frame 0x20f29bf4 svc_run_internal(20f29cd4,fb54a8,187930c0,20f29ce8,12219a0,...) at svc_run_internal+0x999/frame 0x20f29cb0 svc_thread_start(187930c0,20f29ce8) at svc_thread_start+0xd/frame 0x20f29cb8 fork_exit(12219a0,187930c0,20f29ce8,3b,3b,...) at fork_exit+0x68/frame 0x20f29cd4 fork_trampoline() at 0xffc0340e/frame 0x20f29cd4 --- kthread start Tracing command nfsd pid 685 tid 100158 td 0x241f8e00 sched_switch(241f8e00,104) at sched_switch+0x4c0/frame 0x20f24b38 mi_switch(104) at mi_switch+0x1c4/frame 0x20f24b6c sleepq_switch(740,241f8e00,0,0,1d90414,...) at sleepq_switch+0xeb/frame 0x20f24b8c sleepq_catch_signals(20f24bf4,f92178,18d5cdd0,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x20f24bb4 sleepq_wait_sig(18d5cdd0,0) at sleepq_wait_sig+0xe/frame 0x20f24bbc _cv_wait_sig(18d5cdd0,187930c0) at _cv_wait_sig+0x138/frame 0x20f24bf4 svc_run_internal(20f24cd4,fb54a8,187930c0,20f24ce8,12219a0,...) at svc_run_internal+0x999/frame 0x20f24cb0 svc_thread_start(187930c0,20f24ce8) at svc_thread_start+0xd/frame 0x20f24cb8 fork_exit(12219a0,187930c0,20f24ce8,3b,3b,...) at fork_exit+0x68/frame 0x20f24cd4 fork_trampoline() at 0xffc0340e/frame 0x20f24cd4 --- kthread start Tracing command nfsd pid 685 tid 100159 td 0x241f8a80 sched_switch(241f8a80,104) at sched_switch+0x4c0/frame 0x20fe3b38 mi_switch(104) at mi_switch+0x1c4/frame 0x20fe3b6c sleepq_switch(3140,241f8a80,0,0,1d92e14,...) at sleepq_switch+0xeb/frame 0x20fe3b8c sleepq_catch_signals(20fe3bf4,f92178,12b915d0,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x20fe3bb4 sleepq_wait_sig(12b915d0,0) at sleepq_wait_sig+0xe/frame 0x20fe3bbc _cv_wait_sig(12b915d0,187930c0) at _cv_wait_sig+0x138/frame 0x20fe3bf4 svc_run_internal(20fe3cd4,fb54a8,187930c0,20fe3ce8,12219a0,...) at svc_run_internal+0x999/frame 0x20fe3cb0 svc_thread_start(187930c0,20fe3ce8) at svc_thread_start+0xd/frame 0x20fe3cb8 fork_exit(12219a0,187930c0,20fe3ce8,3b,3b,...) at fork_exit+0x68/frame 0x20fe3cd4 fork_trampoline() at 0xffc0340e/frame 0x20fe3cd4 --- kthread start Tracing command nfsd pid 685 tid 100160 td 0x241f8700 sched_switch(241f8700,104) at sched_switch+0x4c0/frame 0x20fdeb38 mi_switch(104) at mi_switch+0x1c4/frame 0x20fdeb6c sleepq_switch(2740,241f8700,0,0,1d92414,...) at sleepq_switch+0xeb/frame 0x20fdeb8c sleepq_catch_signals(20fdebf4,f92178,18d5cd50,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x20fdebb4 sleepq_wait_sig(18d5cd50,0) at sleepq_wait_sig+0xe/frame 0x20fdebbc _cv_wait_sig(18d5cd50,187930c0) at _cv_wait_sig+0x138/frame 0x20fdebf4 svc_run_internal(20fdecd4,fb54a8,187930c0,20fdece8,12219a0,...) at svc_run_internal+0x999/frame 0x20fdecb0 svc_thread_start(187930c0,20fdece8) at svc_thread_start+0xd/frame 0x20fdecb8 fork_exit(12219a0,187930c0,20fdece8,3b,3b,...) at fork_exit+0x68/frame 0x20fdecd4 fork_trampoline() at 0xffc0340e/frame 0x20fdecd4 --- kthread start Tracing command nfsd pid 685 tid 100161 td 0x241f8380 sched_switch(241f8380,104) at sched_switch+0x4c0/frame 0x20fd9b38 mi_switch(104) at mi_switch+0x1c4/frame 0x20fd9b6c sleepq_switch(1140,241f8380,0,0,1d90e14,...) at sleepq_switch+0xeb/frame 0x20fd9b8c sleepq_catch_signals(20fd9bf4,f92178,12b91550,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x20fd9bb4 sleepq_wait_sig(12b91550,0) at sleepq_wait_sig+0xe/frame 0x20fd9bbc _cv_wait_sig(12b91550,187930c0) at _cv_wait_sig+0x138/frame 0x20fd9bf4 svc_run_internal(20fd9cd4,fb54a8,187930c0,20fd9ce8,12219a0,...) at svc_run_internal+0x999/frame 0x20fd9cb0 svc_thread_start(187930c0,20fd9ce8) at svc_thread_start+0xd/frame 0x20fd9cb8 fork_exit(12219a0,187930c0,20fd9ce8,3b,3b,...) at fork_exit+0x68/frame 0x20fd9cd4 fork_trampoline() at 0xffc0340e/frame 0x20fd9cd4 --- kthread start Tracing command nfsd pid 685 tid 100162 td 0x241f8000 sched_switch(241f8000,104) at sched_switch+0x4c0/frame 0x20fd4b38 mi_switch(104) at mi_switch+0x1c4/frame 0x20fd4b6c sleepq_switch(700,241f8000,0,0,1d903d4,...) at sleepq_switch+0xeb/frame 0x20fd4b8c sleepq_catch_signals(20fd4bf4,f92178,18d5ccd0,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x20fd4bb4 sleepq_wait_sig(18d5ccd0,0) at sleepq_wait_sig+0xe/frame 0x20fd4bbc _cv_wait_sig(18d5ccd0,187930c0) at _cv_wait_sig+0x138/frame 0x20fd4bf4 svc_run_internal(20fd4cd4,fb54a8,187930c0,20fd4ce8,12219a0,...) at svc_run_internal+0x999/frame 0x20fd4cb0 svc_thread_start(187930c0,20fd4ce8) at svc_thread_start+0xd/frame 0x20fd4cb8 fork_exit(12219a0,187930c0,20fd4ce8,3b,3b,...) at fork_exit+0x68/frame 0x20fd4cd4 fork_trampoline() at 0xffc0340e/frame 0x20fd4cd4 --- kthread start Tracing command nfsd pid 685 tid 100163 td 0x241fdc00 sched_switch(241fdc00,104) at sched_switch+0x4c0/frame 0x20fcfb38 mi_switch(104) at mi_switch+0x1c4/frame 0x20fcfb6c sleepq_switch(2700,241fdc00,0,0,1d923d4,...) at sleepq_switch+0xeb/frame 0x20fcfb8c sleepq_catch_signals(20fcfbf4,f92178,18d5cc50,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x20fcfbb4 sleepq_wait_sig(18d5cc50,0) at sleepq_wait_sig+0xe/frame 0x20fcfbbc _cv_wait_sig(18d5cc50,187930c0) at _cv_wait_sig+0x138/frame 0x20fcfbf4 svc_run_internal(20fcfcd4,fb54a8,187930c0,20fcfce8,12219a0,...) at svc_run_internal+0x999/frame 0x20fcfcb0 svc_thread_start(187930c0,20fcfce8) at svc_thread_start+0xd/frame 0x20fcfcb8 fork_exit(12219a0,187930c0,20fcfce8,3b,3b,...) at fork_exit+0x68/frame 0x20fcfcd4 fork_trampoline() at 0xffc0340e/frame 0x20fcfcd4 --- kthread start Tracing command nfsd pid 685 tid 100164 td 0x241fd880 sched_switch(241fd880,104) at sched_switch+0x4c0/frame 0x20fcab38 mi_switch(104) at mi_switch+0x1c4/frame 0x20fcab6c sleepq_switch(3100,241fd880,0,0,1d92dd4,...) at sleepq_switch+0xeb/frame 0x20fcab8c sleepq_catch_signals(20fcabf4,f92178,12b914d0,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x20fcabb4 sleepq_wait_sig(12b914d0,0) at sleepq_wait_sig+0xe/frame 0x20fcabbc _cv_wait_sig(12b914d0,187930c0) at _cv_wait_sig+0x138/frame 0x20fcabf4 svc_run_internal(20fcacd4,fb54a8,187930c0,20fcace8,12219a0,...) at svc_run_internal+0x999/frame 0x20fcacb0 svc_thread_start(187930c0,20fcace8) at svc_thread_start+0xd/frame 0x20fcacb8 fork_exit(12219a0,187930c0,20fcace8,3b,3b,...) at fork_exit+0x68/frame 0x20fcacd4 fork_trampoline() at 0xffc0340e/frame 0x20fcacd4 --- kthread start Tracing command nfsd pid 685 tid 100165 td 0x241fd500 sched_switch(241fd500,104) at sched_switch+0x4c0/frame 0x20fc5b38 mi_switch(104) at mi_switch+0x1c4/frame 0x20fc5b6c sleepq_switch(6c0,241fd500,0,0,1d90394,...) at sleepq_switch+0xeb/frame 0x20fc5b8c sleepq_catch_signals(20fc5bf4,f92178,18d5cbd0,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x20fc5bb4 sleepq_wait_sig(18d5cbd0,0) at sleepq_wait_sig+0xe/frame 0x20fc5bbc _cv_wait_sig(18d5cbd0,187930c0) at _cv_wait_sig+0x138/frame 0x20fc5bf4 svc_run_internal(20fc5cd4,fb54a8,187930c0,20fc5ce8,12219a0,...) at svc_run_internal+0x999/frame 0x20fc5cb0 svc_thread_start(187930c0,20fc5ce8) at svc_thread_start+0xd/frame 0x20fc5cb8 fork_exit(12219a0,187930c0,20fc5ce8,3b,3b,...) at fork_exit+0x68/frame 0x20fc5cd4 fork_trampoline() at 0xffc0340e/frame 0x20fc5cd4 --- kthread start Tracing command nfsd pid 685 tid 100166 td 0x241fd180 sched_switch(241fd180,104) at sched_switch+0x4c0/frame 0x20fc0b38 mi_switch(104) at mi_switch+0x1c4/frame 0x20fc0b6c sleepq_switch(1100,241fd180,0,0,1d90dd4,...) at sleepq_switch+0xeb/frame 0x20fc0b8c sleepq_catch_signals(20fc0bf4,f92178,12b91450,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x20fc0bb4 sleepq_wait_sig(12b91450,0) at sleepq_wait_sig+0xe/frame 0x20fc0bbc _cv_wait_sig(12b91450,187930c0) at _cv_wait_sig+0x138/frame 0x20fc0bf4 svc_run_internal(20fc0cd4,fb54a8,187930c0,20fc0ce8,12219a0,...) at svc_run_internal+0x999/frame 0x20fc0cb0 svc_thread_start(187930c0,20fc0ce8) at svc_thread_start+0xd/frame 0x20fc0cb8 fork_exit(12219a0,187930c0,20fc0ce8,3b,3b,...) at fork_exit+0x68/frame 0x20fc0cd4 fork_trampoline() at 0xffc0340e/frame 0x20fc0cd4 --- kthread start Tracing command nfsd pid 685 tid 100167 td 0x241fce00 sched_switch(241fce00,104) at sched_switch+0x4c0/frame 0x2420eb38 mi_switch(104) at mi_switch+0x1c4/frame 0x2420eb6c sleepq_switch(26c0,241fce00,0,0,1d92394,...) at sleepq_switch+0xeb/frame 0x2420eb8c sleepq_catch_signals(2420ebf4,f92178,18d5cb50,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x2420ebb4 sleepq_wait_sig(18d5cb50,0) at sleepq_wait_sig+0xe/frame 0x2420ebbc _cv_wait_sig(18d5cb50,187930c0) at _cv_wait_sig+0x138/frame 0x2420ebf4 svc_run_internal(2420ecd4,fb54a8,187930c0,2420ece8,12219a0,...) at svc_run_internal+0x999/frame 0x2420ecb0 svc_thread_start(187930c0,2420ece8) at svc_thread_start+0xd/frame 0x2420ecb8 fork_exit(12219a0,187930c0,2420ece8,3b,3b,...) at fork_exit+0x68/frame 0x2420ecd4 fork_trampoline() at 0xffc0340e/frame 0x2420ecd4 --- kthread start Tracing command nfsd pid 685 tid 100168 td 0x241fca80 sched_switch(241fca80,104) at sched_switch+0x4c0/frame 0x24209b38 mi_switch(104) at mi_switch+0x1c4/frame 0x24209b6c sleepq_switch(30c0,241fca80,0,0,1d92d94,...) at sleepq_switch+0xeb/frame 0x24209b8c sleepq_catch_signals(24209bf4,f92178,12b913d0,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x24209bb4 sleepq_wait_sig(12b913d0,0) at sleepq_wait_sig+0xe/frame 0x24209bbc _cv_wait_sig(12b913d0,187930c0) at _cv_wait_sig+0x138/frame 0x24209bf4 svc_run_internal(24209cd4,fb54a8,187930c0,24209ce8,12219a0,...) at svc_run_internal+0x999/frame 0x24209cb0 svc_thread_start(187930c0,24209ce8) at svc_thread_start+0xd/frame 0x24209cb8 fork_exit(12219a0,187930c0,24209ce8,3b,3b,...) at fork_exit+0x68/frame 0x24209cd4 fork_trampoline() at 0xffc0340e/frame 0x24209cd4 --- kthread start Tracing command nfsd pid 685 tid 100169 td 0x241fc700 sched_switch(241fc700,104) at sched_switch+0x4c0/frame 0x24204b38 mi_switch(104) at mi_switch+0x1c4/frame 0x24204b6c sleepq_switch(680,241fc700,0,0,1d90354,...) at sleepq_switch+0xeb/frame 0x24204b8c sleepq_catch_signals(24204bf4,f92178,18d5cad0,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x24204bb4 sleepq_wait_sig(18d5cad0,0) at sleepq_wait_sig+0xe/frame 0x24204bbc _cv_wait_sig(18d5cad0,187930c0) at _cv_wait_sig+0x138/frame 0x24204bf4 svc_run_internal(24204cd4,fb54a8,187930c0,24204ce8,12219a0,...) at svc_run_internal+0x999/frame 0x24204cb0 svc_thread_start(187930c0,24204ce8) at svc_thread_start+0xd/frame 0x24204cb8 fork_exit(12219a0,187930c0,24204ce8,3b,3b,...) at fork_exit+0x68/frame 0x24204cd4 fork_trampoline() at 0xffc0340e/frame 0x24204cd4 --- kthread start Tracing command nfsd pid 685 tid 100170 td 0x241fc380 sched_switch(241fc380,104) at sched_switch+0x4c0/frame 0x20ffcb38 mi_switch(104) at mi_switch+0x1c4/frame 0x20ffcb6c sleepq_switch(10c0,241fc380,0,0,1d90d94,...) at sleepq_switch+0xeb/frame 0x20ffcb8c sleepq_catch_signals(20ffcbf4,f92178,12b91350,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x20ffcbb4 sleepq_wait_sig(12b91350,0) at sleepq_wait_sig+0xe/frame 0x20ffcbbc _cv_wait_sig(12b91350,187930c0) at _cv_wait_sig+0x138/frame 0x20ffcbf4 svc_run_internal(20ffccd4,fb54a8,187930c0,20ffcce8,12219a0,...) at svc_run_internal+0x999/frame 0x20ffccb0 svc_thread_start(187930c0,20ffcce8) at svc_thread_start+0xd/frame 0x20ffccb8 fork_exit(12219a0,187930c0,20ffcce8,3b,3b,...) at fork_exit+0x68/frame 0x20ffccd4 fork_trampoline() at 0xffc0340e/frame 0x20ffccd4 --- kthread start Tracing command nfsd pid 685 tid 100171 td 0x241fc000 sched_switch(241fc000,104) at sched_switch+0x4c0/frame 0x20ff7b38 mi_switch(104) at mi_switch+0x1c4/frame 0x20ff7b6c sleepq_switch(3080,241fc000,0,0,1d92d54,...) at sleepq_switch+0xeb/frame 0x20ff7b8c sleepq_catch_signals(20ff7bf4,f92178,12b912d0,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x20ff7bb4 sleepq_wait_sig(12b912d0,0) at sleepq_wait_sig+0xe/frame 0x20ff7bbc _cv_wait_sig(12b912d0,187930c0) at _cv_wait_sig+0x138/frame 0x20ff7bf4 svc_run_internal(20ff7cd4,fb54a8,187930c0,20ff7ce8,12219a0,...) at svc_run_internal+0x999/frame 0x20ff7cb0 svc_thread_start(187930c0,20ff7ce8) at svc_thread_start+0xd/frame 0x20ff7cb8 fork_exit(12219a0,187930c0,20ff7ce8,3b,3b,...) at fork_exit+0x68/frame 0x20ff7cd4 fork_trampoline() at 0xffc0340e/frame 0x20ff7cd4 --- kthread start Tracing command nfsd pid 685 tid 100172 td 0x24401c00 sched_switch(24401c00,104) at sched_switch+0x4c0/frame 0x20ff2b38 mi_switch(104) at mi_switch+0x1c4/frame 0x20ff2b6c sleepq_switch(2680,24401c00,0,0,1d92354,...) at sleepq_switch+0xeb/frame 0x20ff2b8c sleepq_catch_signals(20ff2bf4,f92178,18d5ca50,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x20ff2bb4 sleepq_wait_sig(18d5ca50,0) at sleepq_wait_sig+0xe/frame 0x20ff2bbc _cv_wait_sig(18d5ca50,187930c0) at _cv_wait_sig+0x138/frame 0x20ff2bf4 svc_run_internal(20ff2cd4,fb54a8,187930c0,20ff2ce8,12219a0,...) at svc_run_internal+0x999/frame 0x20ff2cb0 svc_thread_start(187930c0,20ff2ce8) at svc_thread_start+0xd/frame 0x20ff2cb8 fork_exit(12219a0,187930c0,20ff2ce8,3b,3b,...) at fork_exit+0x68/frame 0x20ff2cd4 fork_trampoline() at 0xffc0340e/frame 0x20ff2cd4 --- kthread start Tracing command nfsd pid 685 tid 100173 td 0x24401880 sched_switch(24401880,104) at sched_switch+0x4c0/frame 0x20fedb38 mi_switch(104) at mi_switch+0x1c4/frame 0x20fedb6c sleepq_switch(1080,24401880,0,0,1d90d54,...) at sleepq_switch+0xeb/frame 0x20fedb8c sleepq_catch_signals(20fedbf4,f92178,12b91250,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x20fedbb4 sleepq_wait_sig(12b91250,0) at sleepq_wait_sig+0xe/frame 0x20fedbbc _cv_wait_sig(12b91250,187930c0) at _cv_wait_sig+0x138/frame 0x20fedbf4 svc_run_internal(20fedcd4,fb54a8,187930c0,20fedce8,12219a0,...) at svc_run_internal+0x999/frame 0x20fedcb0 svc_thread_start(187930c0,20fedce8) at svc_thread_start+0xd/frame 0x20fedcb8 fork_exit(12219a0,187930c0,20fedce8,3b,3b,...) at fork_exit+0x68/frame 0x20fedcd4 fork_trampoline() at 0xffc0340e/frame 0x20fedcd4 --- kthread start Tracing command nfsd pid 685 tid 100174 td 0x24401500 sched_switch(24401500,104) at sched_switch+0x4c0/frame 0x20fe8b38 mi_switch(104) at mi_switch+0x1c4/frame 0x20fe8b6c sleepq_switch(640,24401500,0,0,1d90314,...) at sleepq_switch+0xeb/frame 0x20fe8b8c sleepq_catch_signals(20fe8bf4,f92178,18d5c9d0,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x20fe8bb4 sleepq_wait_sig(18d5c9d0,0) at sleepq_wait_sig+0xe/frame 0x20fe8bbc _cv_wait_sig(18d5c9d0,187930c0) at _cv_wait_sig+0x138/frame 0x20fe8bf4 svc_run_internal(20fe8cd4,fb54a8,187930c0,20fe8ce8,12219a0,...) at svc_run_internal+0x999/frame 0x20fe8cb0 svc_thread_start(187930c0,20fe8ce8) at svc_thread_start+0xd/frame 0x20fe8cb8 fork_exit(12219a0,187930c0,20fe8ce8,3b,3b,...) at fork_exit+0x68/frame 0x20fe8cd4 fork_trampoline() at 0xffc0340e/frame 0x20fe8cd4 --- kthread start Tracing command nfsd pid 685 tid 100175 td 0x24401180 sched_switch(24401180,104) at sched_switch+0x4c0/frame 0x24236b38 mi_switch(104) at mi_switch+0x1c4/frame 0x24236b6c sleepq_switch(2640,24401180,0,0,1d92314,...) at sleepq_switch+0xeb/frame 0x24236b8c sleepq_catch_signals(24236bf4,f92178,18d5c950,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x24236bb4 sleepq_wait_sig(18d5c950,0) at sleepq_wait_sig+0xe/frame 0x24236bbc _cv_wait_sig(18d5c950,187930c0) at _cv_wait_sig+0x138/frame 0x24236bf4 svc_run_internal(24236cd4,fb54a8,187930c0,24236ce8,12219a0,...) at svc_run_internal+0x999/frame 0x24236cb0 svc_thread_start(187930c0,24236ce8) at svc_thread_start+0xd/frame 0x24236cb8 fork_exit(12219a0,187930c0,24236ce8,3b,3b,...) at fork_exit+0x68/frame 0x24236cd4 fork_trampoline() at 0xffc0340e/frame 0x24236cd4 --- kthread start Tracing command nfsd pid 685 tid 100176 td 0x24400e00 sched_switch(24400e00,104) at sched_switch+0x4c0/frame 0x24231b38 mi_switch(104) at mi_switch+0x1c4/frame 0x24231b6c sleepq_switch(3040,24400e00,0,0,1d92d14,...) at sleepq_switch+0xeb/frame 0x24231b8c sleepq_catch_signals(24231bf4,f92178,12b911d0,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x24231bb4 sleepq_wait_sig(12b911d0,0) at sleepq_wait_sig+0xe/frame 0x24231bbc _cv_wait_sig(12b911d0,187930c0) at _cv_wait_sig+0x138/frame 0x24231bf4 svc_run_internal(24231cd4,fb54a8,187930c0,24231ce8,12219a0,...) at svc_run_internal+0x999/frame 0x24231cb0 svc_thread_start(187930c0,24231ce8) at svc_thread_start+0xd/frame 0x24231cb8 fork_exit(12219a0,187930c0,24231ce8,3b,3b,...) at fork_exit+0x68/frame 0x24231cd4 fork_trampoline() at 0xffc0340e/frame 0x24231cd4 --- kthread start Tracing command nfsd pid 685 tid 100177 td 0x24400a80 sched_switch(24400a80,104) at sched_switch+0x4c0/frame 0x2422cb38 mi_switch(104) at mi_switch+0x1c4/frame 0x2422cb6c sleepq_switch(600,24400a80,0,0,1d902d4,...) at sleepq_switch+0xeb/frame 0x2422cb8c sleepq_catch_signals(2422cbf4,f92178,18d5c8d0,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x2422cbb4 sleepq_wait_sig(18d5c8d0,0) at sleepq_wait_sig+0xe/frame 0x2422cbbc _cv_wait_sig(18d5c8d0,187930c0) at _cv_wait_sig+0x138/frame 0x2422cbf4 svc_run_internal(2422ccd4,fb54a8,187930c0,2422cce8,12219a0,...) at svc_run_internal+0x999/frame 0x2422ccb0 svc_thread_start(187930c0,2422cce8) at svc_thread_start+0xd/frame 0x2422ccb8 fork_exit(12219a0,187930c0,2422cce8,3b,3b,...) at fork_exit+0x68/frame 0x2422ccd4 fork_trampoline() at 0xffc0340e/frame 0x2422ccd4 --- kthread start Tracing command nfsd pid 685 tid 100178 td 0x24400700 sched_switch(24400700,104) at sched_switch+0x4c0/frame 0x24227b38 mi_switch(104) at mi_switch+0x1c4/frame 0x24227b6c sleepq_switch(1040,24400700,0,0,1d90d14,...) at sleepq_switch+0xeb/frame 0x24227b8c sleepq_catch_signals(24227bf4,f92178,12b91150,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x24227bb4 sleepq_wait_sig(12b91150,0) at sleepq_wait_sig+0xe/frame 0x24227bbc _cv_wait_sig(12b91150,187930c0) at _cv_wait_sig+0x138/frame 0x24227bf4 svc_run_internal(24227cd4,fb54a8,187930c0,24227ce8,12219a0,...) at svc_run_internal+0x999/frame 0x24227cb0 svc_thread_start(187930c0,24227ce8) at svc_thread_start+0xd/frame 0x24227cb8 fork_exit(12219a0,187930c0,24227ce8,3b,3b,...) at fork_exit+0x68/frame 0x24227cd4 fork_trampoline() at 0xffc0340e/frame 0x24227cd4 --- kthread start Tracing command nfsd pid 685 tid 100179 td 0x24400380 sched_switch(24400380,104) at sched_switch+0x4c0/frame 0x24222b38 mi_switch(104) at mi_switch+0x1c4/frame 0x24222b6c sleepq_switch(2600,24400380,0,0,1d922d4,...) at sleepq_switch+0xeb/frame 0x24222b8c sleepq_catch_signals(24222bf4,f92178,18d5c850,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x24222bb4 sleepq_wait_sig(18d5c850,0) at sleepq_wait_sig+0xe/frame 0x24222bbc _cv_wait_sig(18d5c850,187930c0) at _cv_wait_sig+0x138/frame 0x24222bf4 svc_run_internal(24222cd4,fb54a8,187930c0,24222ce8,12219a0,...) at svc_run_internal+0x999/frame 0x24222cb0 svc_thread_start(187930c0,24222ce8) at svc_thread_start+0xd/frame 0x24222cb8 fork_exit(12219a0,187930c0,24222ce8,3b,3b,...) at fork_exit+0x68/frame 0x24222cd4 fork_trampoline() at 0xffc0340e/frame 0x24222cd4 --- kthread start Tracing command nfsd pid 685 tid 100180 td 0x24400000 sched_switch(24400000,104) at sched_switch+0x4c0/frame 0x2421db38 mi_switch(104) at mi_switch+0x1c4/frame 0x2421db6c sleepq_switch(3000,24400000,0,0,1d92cd4,...) at sleepq_switch+0xeb/frame 0x2421db8c sleepq_catch_signals(2421dbf4,f92178,12b910d0,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x2421dbb4 sleepq_wait_sig(12b910d0,0) at sleepq_wait_sig+0xe/frame 0x2421dbbc _cv_wait_sig(12b910d0,187930c0) at _cv_wait_sig+0x138/frame 0x2421dbf4 svc_run_internal(2421dcd4,fb54a8,187930c0,2421dce8,12219a0,...) at svc_run_internal+0x999/frame 0x2421dcb0 svc_thread_start(187930c0,2421dce8) at svc_thread_start+0xd/frame 0x2421dcb8 fork_exit(12219a0,187930c0,2421dce8,3b,3b,...) at fork_exit+0x68/frame 0x2421dcd4 fork_trampoline() at 0xffc0340e/frame 0x2421dcd4 --- kthread start Tracing command nfsd pid 685 tid 100181 td 0x24403c00 sched_switch(24403c00,104) at sched_switch+0x4c0/frame 0x24218b38 mi_switch(104) at mi_switch+0x1c4/frame 0x24218b6c sleepq_switch(1000,24403c00,0,0,1d90cd4,...) at sleepq_switch+0xeb/frame 0x24218b8c sleepq_catch_signals(24218bf4,f92178,12b91050,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x24218bb4 sleepq_wait_sig(12b91050,0) at sleepq_wait_sig+0xe/frame 0x24218bbc _cv_wait_sig(12b91050,187930c0) at _cv_wait_sig+0x138/frame 0x24218bf4 svc_run_internal(24218cd4,fb54a8,187930c0,24218ce8,12219a0,...) at svc_run_internal+0x999/frame 0x24218cb0 svc_thread_start(187930c0,24218ce8) at svc_thread_start+0xd/frame 0x24218cb8 fork_exit(12219a0,187930c0,24218ce8,3b,3b,...) at fork_exit+0x68/frame 0x24218cd4 fork_trampoline() at 0xffc0340e/frame 0x24218cd4 --- kthread start Tracing command nfsd pid 685 tid 100182 td 0x24403880 sched_switch(24403880,104) at sched_switch+0x4c0/frame 0x24213b38 mi_switch(104) at mi_switch+0x1c4/frame 0x24213b6c sleepq_switch(5c0,24403880,0,0,1d90294,...) at sleepq_switch+0xeb/frame 0x24213b8c sleepq_catch_signals(24213bf4,f92178,18d5c7d0,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x24213bb4 sleepq_wait_sig(18d5c7d0,0) at sleepq_wait_sig+0xe/frame 0x24213bbc _cv_wait_sig(18d5c7d0,187930c0) at _cv_wait_sig+0x138/frame 0x24213bf4 svc_run_internal(24213cd4,fb54a8,187930c0,24213ce8,12219a0,...) at svc_run_internal+0x999/frame 0x24213cb0 svc_thread_start(187930c0,24213ce8) at svc_thread_start+0xd/frame 0x24213cb8 fork_exit(12219a0,187930c0,24213ce8,3b,3b,...) at fork_exit+0x68/frame 0x24213cd4 fork_trampoline() at 0xffc0340e/frame 0x24213cd4 --- kthread start Tracing command nfsd pid 685 tid 100183 td 0x24403500 sched_switch(24403500,104) at sched_switch+0x4c0/frame 0x2425eb38 mi_switch(104) at mi_switch+0x1c4/frame 0x2425eb6c sleepq_switch(25c0,24403500,0,0,1d92294,...) at sleepq_switch+0xeb/frame 0x2425eb8c sleepq_catch_signals(2425ebf4,f92178,18d5c750,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x2425ebb4 sleepq_wait_sig(18d5c750,0) at sleepq_wait_sig+0xe/frame 0x2425ebbc _cv_wait_sig(18d5c750,187930c0) at _cv_wait_sig+0x138/frame 0x2425ebf4 svc_run_internal(2425ecd4,fb54a8,187930c0,2425ece8,12219a0,...) at svc_run_internal+0x999/frame 0x2425ecb0 svc_thread_start(187930c0,2425ece8) at svc_thread_start+0xd/frame 0x2425ecb8 fork_exit(12219a0,187930c0,2425ece8,3b,3b,...) at fork_exit+0x68/frame 0x2425ecd4 fork_trampoline() at 0xffc0340e/frame 0x2425ecd4 --- kthread start Tracing command nfsd pid 685 tid 100184 td 0x24403180 sched_switch(24403180,104) at sched_switch+0x4c0/frame 0x24259b38 mi_switch(104) at mi_switch+0x1c4/frame 0x24259b6c sleepq_switch(7c0,24403180,0,0,1d90494,...) at sleepq_switch+0xeb/frame 0x24259b8c sleepq_catch_signals(24259bf4,f92178,24404f50,0,1480795,...) at sleepq_catch_signals+0x278/frame 0x24259bb4 sleepq_wait_sig(24404f50,0) at sleepq_wait_sig+0xe/frame 0x24259bbc _cv_wait_sig(24404f50,187930c0) at _cv_wait_sig+0x138/frame 0x24259bf4 svc_run_internal(24259cd4,fb54a8,187930c0,24259ce8,12219a0,...) at svc_run_internal+0x999/frame 0x24259cb0 svc_thread_start(187930c0,24259ce8) at svc_thread_start+0xd/frame 0x24259cb8 fork_exit(12219a0,187930c0,24259ce8,3b,3b,...) at fork_exit+0x68/frame 0x24259cd4 fork_trampoline() at 0xffc0340e/frame 0x24259cd4 --- kthread start Tracing command rpc.statd pid 686 tid 100130 td 0x20cfce00 sched_switch(20cfce00,104) at sched_switch+0x4c0/frame 0x205e6940 mi_switch(104) at mi_switch+0x1c4/frame 0x205e6978 sleepq_switch(1c00,0,0,0,1d918d4,...) at sleepq_switch+0xeb/frame 0x205e6998 sleepq_catch_signals(20cfce00,205e69e8,fd2eee,1889d4a0,1889d4a4,...) at sleepq_catch_signals+0x278/frame 0x205e69c0 sleepq_timedwait_sig(1889d4a4,0) at sleepq_timedwait_sig+0x14/frame 0x205e69e4 _cv_timedwait_sig_sbt(1889d4a4,1889d490,6536ac81,1ef8,e0000000,1,200) at _cv_timedwait_sig_sbt+0x15a/frame 0x205e6a20 seltdwait(1ef8,e0000000,1) at seltdwait+0x63/frame 0x205e6a50 kern_select(20cfce00,8,ffbfeca8,0,0,205e6bf0,20) at kern_select+0x749/frame 0x205e6bcc sys_select(20cfce00,20cfd098,20cfce00,20cfce00,205e6cdc,...) at sys_select+0x3f/frame 0x205e6c00 syscall(205e6ce8,3b,3b,3b,ffbfeca8,...) at syscall+0x1b8/frame 0x205e6cdc Xint0x80_syscall() at 0xffc033f9/frame 0x205e6cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01d98, esp = 0xffc0bfe8, ebp = 0xffbfe808 --- Tracing command rpc.lockd pid 689 tid 100140 td 0x23615c00 sched_switch(23615c00,104) at sched_switch+0x4c0/frame 0x205f0910 mi_switch(104) at mi_switch+0x1c4/frame 0x205f0948 sleepq_switch(1e00,0,0,0,1d91ad4,...) at sleepq_switch+0xeb/frame 0x205f0968 sleepq_catch_signals(23615c00,205f09b8,fd2eee,2361f0d0,1889a8d0,...) at sleepq_catch_signals+0x278/frame 0x205f0990 sleepq_timedwait_sig(1889a8d0,0) at sleepq_timedwait_sig+0x14/frame 0x205f09b4 _cv_timedwait_sig_sbt(1889a8d0,2361f0c0,fffffa38,4,0,0,100) at _cv_timedwait_sig_sbt+0x15a/frame 0x205f09f0 svc_run_internal(4,2361f000,205f0c00,1204d91,18899920,...) at svc_run_internal+0x95a/frame 0x205f0ac0 svc_run(2361f000) at svc_run+0x192/frame 0x205f0af0 sys_nlm_syscall(23615c00,23615e98,23615c00,23615c00,205f0cdc,...) at sys_nlm_syscall+0x69d/frame 0x205f0c00 syscall(205f0ce8,3b,3b,3b,20750000,...) at syscall+0x1b8/frame 0x205f0cdc Xint0x80_syscall() at 0xffc033f9/frame 0x205f0cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01e18, esp = 0xffc0dfe8, ebp = 0xffbfe808 --- Tracing command ntpd pid 710 tid 100133 td 0x20b53a80 sched_switch(20b53a80,104) at sched_switch+0x4c0/frame 0x20e04978 mi_switch(104) at mi_switch+0x1c4/frame 0x20e049ac sleepq_switch(2a80,20b53a80,0,0,1d92754,...) at sleepq_switch+0xeb/frame 0x20e049cc sleepq_catch_signals(20e04a34,f92178,18db8e24,0,1414b23,...) at sleepq_catch_signals+0x278/frame 0x20e049f4 sleepq_wait_sig(18db8e24,0) at sleepq_wait_sig+0xe/frame 0x20e049fc _cv_wait_sig(18db8e24,18db8e10) at _cv_wait_sig+0x138/frame 0x20e04a34 seltdwait(ffffffff,0,0) at seltdwait+0x7f/frame 0x20e04a50 kern_select(20b53a80,1b,ffbfe25c,0,0,0,20) at kern_select+0x749/frame 0x20e04bcc sys_select(20b53a80,20b53d18,109b,20b53a80,204d40,...) at sys_select+0x3f/frame 0x20e04c00 syscall(20e04ce8,3b,3b,3b,209b4000,...) at syscall+0x1b8/frame 0x20e04cdc Xint0x80_syscall() at 0xffc033f9/frame 0x20e04cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01d98, esp = 0xffc0bfe8, ebp = 0xffbfe808 --- Tracing command powerd pid 713 tid 100146 td 0x20cfca80 sched_switch(20cfca80,104) at sched_switch+0x4c0/frame 0x205e1940 mi_switch(104) at mi_switch+0x1c4/frame 0x205e1978 sleepq_switch(3a80,0,0,0,1d93754,...) at sleepq_switch+0xeb/frame 0x205e1998 sleepq_catch_signals(20cfca80,205e19e8,fd2eee,21048e60,21048e64,...) at sleepq_catch_signals+0x278/frame 0x205e19c0 sleepq_timedwait_sig(21048e64,0) at sleepq_timedwait_sig+0x14/frame 0x205e19e4 _cv_timedwait_sig_sbt(21048e64,21048e50,5b097d83,1ef4,4000000,0,200) at _cv_timedwait_sig_sbt+0x15a/frame 0x205e1a20 seltdwait(1ef4,4000000,0) at seltdwait+0x63/frame 0x205e1a50 kern_select(20cfca80,0,ffbfece8,0,ffbfece8,205e1bf0,20) at kern_select+0x749/frame 0x205e1bcc sys_select(20cfca80,20cfcd18,205e1c50,20cfca80,b7f7030,...) at sys_select+0x3f/frame 0x205e1c00 syscall(205e1ce8,3b,3b,3b,ffbfed68,...) at syscall+0x1b8/frame 0x205e1cdc Xint0x80_syscall() at 0xffc033f9/frame 0x205e1cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01c98, esp = 0xffc07fe8, ebp = 0xffbfeccc --- Tracing command smartd pid 756 tid 100148 td 0x20cfc380 sched_switch(20cfc380,104) at sched_switch+0x4c0/frame 0x205d7a88 mi_switch(104) at mi_switch+0x1c4/frame 0x205d7ac0 sleepq_switch(b00,20cfc380,6c,0,1d907d4,...) at sleepq_switch+0xeb/frame 0x205d7ae0 sleepq_catch_signals(20cfc380,100,1d7e2ce,205d7b6c,20cfc380,...) at sleepq_catch_signals+0x278/frame 0x205d7b08 sleepq_timedwait_sig(1d7e2ce,6c,20cfc380,10000,50000,...) at sleepq_timedwait_sig+0x14/frame 0x205d7b2c _sleep(1d7e2ce,0,16c,1510816,a09c7a33,24b3,80000000,70,200) at _sleep+0x224/frame 0x205d7b6c kern_clock_nanosleep(20cfc380,0,0,205d7bec,205d7be4) at kern_clock_nanosleep+0x1a1/frame 0x205d7bc8 sys_nanosleep(20cfc380,20cfc618,20cfc380,20cfc380,205d7cdc,...) at sys_nanosleep+0x34/frame 0x205d7c00 syscall(205d7ce8,3b,3b,3b,2083c000,...) at syscall+0x1b8/frame 0x205d7cdc Xint0x80_syscall() at 0xffc033f9/frame 0x205d7cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01d98, esp = 0xffc0bfe8, ebp = 0xffbfe808 --- Tracing command inetd pid 779 tid 100188 td 0x241dfa80 sched_switch(241dfa80,104) at sched_switch+0x4c0/frame 0x20f84978 mi_switch(104) at mi_switch+0x1c4/frame 0x20f849ac sleepq_switch(3ac0,241dfa80,0,0,1d93794,...) at sleepq_switch+0xeb/frame 0x20f849cc sleepq_catch_signals(20f84a34,f92178,18db8f64,0,1414b23,...) at sleepq_catch_signals+0x278/frame 0x20f849f4 sleepq_wait_sig(18db8f64,0) at sleepq_wait_sig+0xe/frame 0x20f849fc _cv_wait_sig(18db8f64,18db8f50) at _cv_wait_sig+0x138/frame 0x20f84a34 seltdwait(ffffffff,0,0) at seltdwait+0x7f/frame 0x20f84a50 kern_select(241dfa80,9,ffbfe5dc,0,0,0,20) at kern_select+0x749/frame 0x20f84bcc sys_select(241dfa80,241dfd18,241dfa80,241dfa80,20f84cdc,...) at sys_select+0x3f/frame 0x20f84c00 syscall(20f84ce8,3b,3b,3b,ffbfe65c,...) at syscall+0x1b8/frame 0x20f84cdc Xint0x80_syscall() at 0xffc033f9/frame 0x20f84cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01e18, esp = 0xffc0dfe8, ebp = 0xffbfe808 --- Tracing command sshd pid 803 tid 100151 td 0x241e0880 sched_switch(241e0880,104) at sched_switch+0x4c0/frame 0x205c8978 mi_switch(104) at mi_switch+0x1c4/frame 0x205c89ac sleepq_switch(a40,241e0880,0,0,1d90714,...) at sleepq_switch+0xeb/frame 0x205c89cc sleepq_catch_signals(205c8a34,f92178,21048da4,0,1414b23,...) at sleepq_catch_signals+0x278/frame 0x205c89f4 sleepq_wait_sig(21048da4,0) at sleepq_wait_sig+0xe/frame 0x205c89fc _cv_wait_sig(21048da4,21048d90) at _cv_wait_sig+0x138/frame 0x205c8a34 seltdwait(ffffffff,0,0) at seltdwait+0x7f/frame 0x205c8a50 kern_select(241e0880,7,2047c180,0,0,0,20) at kern_select+0x749/frame 0x205c8bcc sys_select(241e0880,241e0b18,241e0880,241e0880,205c8cdc,...) at sys_select+0x3f/frame 0x205c8c00 syscall(205c8ce8,3b,3b,3b,20482000,...) at syscall+0x1b8/frame 0x205c8cdc Xint0x80_syscall() at 0xffc033f9/frame 0x205c8cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01d98, esp = 0xffc0bfe8, ebp = 0xffbfe808 --- Tracing command cron pid 806 tid 100152 td 0x241e0500 sched_switch(241e0500,104) at sched_switch+0x4c0/frame 0x20f93a88 mi_switch(104) at mi_switch+0x1c4/frame 0x20f93ac0 sleepq_switch(b40,241e0500,6c,0,1d90814,...) at sleepq_switch+0xeb/frame 0x20f93ae0 sleepq_catch_signals(241e0500,100,1d7e2cf,20f93b6c,241e0500,...) at sleepq_catch_signals+0x278/frame 0x20f93b08 sleepq_timedwait_sig(1d7e2cf,6c,241e0500,10000,50000,...) at sleepq_timedwait_sig+0x14/frame 0x20f93b2c _sleep(1d7e2cf,0,16c,1510816,19d3378d,1f0b,bf3d7e28,3,200) at _sleep+0x224/frame 0x20f93b6c kern_clock_nanosleep(241e0500,0,0,20f93bec,20f93be4) at kern_clock_nanosleep+0x1a1/frame 0x20f93bc8 sys_nanosleep(241e0500,241e0798,0,241e0500,1e57,...) at sys_nanosleep+0x34/frame 0x20f93c00 syscall(20f93ce8,3b,3b,3b,ffbfece4,...) at syscall+0x1b8/frame 0x20f93cdc Xint0x80_syscall() at 0xffc033f9/frame 0x20f93cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01e18, esp = 0xffc0dfe8, ebp = 0xffbfe808 --- Tracing command sendmail pid 819 tid 100187 td 0x241dfe00 sched_switch(241dfe00,104) at sched_switch+0x4c0/frame 0x20f89940 mi_switch(104) at mi_switch+0x1c4/frame 0x20f89978 sleepq_switch(a40,0,0,0,1d90714,...) at sleepq_switch+0xeb/frame 0x20f89998 sleepq_catch_signals(241dfe00,20f899e8,fd2eee,18db8da0,18db8da4,...) at sleepq_catch_signals+0x278/frame 0x20f899c0 sleepq_timedwait_sig(18db8da4,0) at sleepq_timedwait_sig+0x14/frame 0x20f899e4 _cv_timedwait_sig_sbt(18db8da4,18db8d90,561bf456,1ef4,50000000,0,200) at _cv_timedwait_sig_sbt+0x15a/frame 0x20f89a20 seltdwait(1ef4,50000000,0) at seltdwait+0x63/frame 0x20f89a50 kern_select(241dfe00,5,ffbfd0dc,0,0,20f89bf0,20) at kern_select+0x749/frame 0x20f89bcc sys_select(241dfe00,241e0098,241dfe00,241dfe00,20f89cdc,...) at sys_select+0x3f/frame 0x20f89c00 syscall(20f89ce8,3b,3b,3b,20a30000,...) at syscall+0x1b8/frame 0x20f89cdc Xint0x80_syscall() at 0xffc033f9/frame 0x20f89cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01c98, esp = 0xffc07fe8, ebp = 0xffbfeccc --- Tracing command sendmail pid 825 tid 100109 td 0x18d08180 sched_switch(18d08180,104) at sched_switch+0x4c0/frame 0xb5deac0 mi_switch(104) at mi_switch+0x1c4/frame 0xb5deaf8 sleepq_switch(1cc0,14fd263,74,0,1d91994,...) at sleepq_switch+0xeb/frame 0xb5deb18 sleepq_catch_signals(b5deb88,ffddc4,188ac3b0,74,b5deb98,...) at sleepq_catch_signals+0x278/frame 0xb5deb40 sleepq_wait_sig(188ac3b0,74) at sleepq_wait_sig+0xe/frame 0xb5deb48 _sleep(188ac3b0,188ac3f4,174,145ce5e,0,0,0,0,100) at _sleep+0x244/frame 0xb5deb88 kern_sigsuspend(18d08180,0,0,0,0) at kern_sigsuspend+0xca/frame 0xb5debd0 sys_sigsuspend(18d08180,18d08418,109b,18d08180,ffffffff,...) at sys_sigsuspend+0x36/frame 0xb5dec00 syscall(b5dece8,3b,3b,3b,20a30000,...) at syscall+0x1b8/frame 0xb5decdc Xint0x80_syscall() at 0xffc033f9/frame 0xb5decdc --- syscall (10, FreeBSD ELF32, sys_unlink), eip = 0xffc01d18, esp = 0xffc0afe8, ebp = 0xffbfe73c --- Tracing command getty pid 838 tid 100105 td 0x18d08500 sched_switch(18d08500,104) at sched_switch+0x4c0/frame 0xb5e3a28 mi_switch(104) at mi_switch+0x1c4/frame 0xb5e3a5c sleepq_switch(3b00,18d08500,0,0,1d937d4,...) at sleepq_switch+0xeb/frame 0xb5e3a7c sleepq_catch_signals(b5e3ae4,f92178,18879874,0,15247b5,...) at sleepq_catch_signals+0x278/frame 0xb5e3aa4 sleepq_wait_sig(18879874,0) at sleepq_wait_sig+0xe/frame 0xb5e3aac _cv_wait_sig(18879874,18879804) at _cv_wait_sig+0x138/frame 0xb5e3ae4 tty_wait(18879800,18879874,1887982c,18879874,15247b5,...) at tty_wait+0x35/frame 0xb5e3afc ttydisc_read(18879800,b5e3bc8,0) at ttydisc_read+0x1c8/frame 0xb5e3b30 ttydev_read(18ac0e00,b5e3bc8,0) at ttydev_read+0x3d/frame 0xb5e3b4c devfs_read_f(18dfc960,b5e3bc8,24455a00,0,18d08500) at devfs_read_f+0xaa/frame 0xb5e3b7c dofileread(18dfc960,b5e3bc8,ffffffff,ffffffff,0) at dofileread+0x6d/frame 0xb5e3bac sys_read(18d08500,18d08798,18d08500,18d08500,b5e3cdc,...) at sys_read+0x67/frame 0xb5e3c00 syscall(b5e3ce8,3b,3b,3b,0,...) at syscall+0x1b8/frame 0xb5e3cdc Xint0x80_syscall() at 0xffc033f9/frame 0xb5e3cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01c98, esp = 0xffc07fe8, ebp = 0xffbfeccc --- Tracing command getty pid 839 tid 100153 td 0x241e0180 sched_switch(241e0180,104) at sched_switch+0x4c0/frame 0x20f8ea28 mi_switch(104) at mi_switch+0x1c4/frame 0x20f8ea5c sleepq_switch(3800,241e0180,0,0,1d934d4,...) at sleepq_switch+0xeb/frame 0x20f8ea7c sleepq_catch_signals(20f8eae4,f92178,18879474,0,15247b5,...) at sleepq_catch_signals+0x278/frame 0x20f8eaa4 sleepq_wait_sig(18879474,0) at sleepq_wait_sig+0xe/frame 0x20f8eaac _cv_wait_sig(18879474,18879404) at _cv_wait_sig+0x138/frame 0x20f8eae4 tty_wait(18879400,18879474,1887942c,18879474,15247b5,...) at tty_wait+0x35/frame 0x20f8eafc ttydisc_read(18879400,20f8ebc8,0) at ttydisc_read+0x1c8/frame 0x20f8eb30 ttydev_read(18ac0c00,20f8ebc8,0) at ttydev_read+0x3d/frame 0x20f8eb4c devfs_read_f(18dfc438,20f8ebc8,249dda00,0,241e0180) at devfs_read_f+0xaa/frame 0x20f8eb7c dofileread(18dfc438,20f8ebc8,ffffffff,ffffffff,0) at dofileread+0x6d/frame 0x20f8ebac sys_read(241e0180,241e0418,241e0180,241e0180,20f8ecdc,...) at sys_read+0x67/frame 0x20f8ec00 syscall(20f8ece8,3b,3b,3b,0,...) at syscall+0x1b8/frame 0x20f8ecdc Xint0x80_syscall() at 0xffc033f9/frame 0x20f8ecdc --- syscall (10, FreeBSD ELF32, sys_unlink), eip = 0xffc01d18, esp = 0xffc0afe8, ebp = 0xffbfe73c --- Tracing command getty pid 840 tid 100137 td 0x20b53700 sched_switch(20b53700,104) at sched_switch+0x4c0/frame 0x205ffa28 mi_switch(104) at mi_switch+0x1c4/frame 0x205ffa5c sleepq_switch(3900,20b53700,0,0,1d935d4,...) at sleepq_switch+0xeb/frame 0x205ffa7c sleepq_catch_signals(205ffae4,f92178,18879074,0,15247b5,...) at sleepq_catch_signals+0x278/frame 0x205ffaa4 sleepq_wait_sig(18879074,0) at sleepq_wait_sig+0xe/frame 0x205ffaac _cv_wait_sig(18879074,18879004) at _cv_wait_sig+0x138/frame 0x205ffae4 tty_wait(18879000,18879074,1887902c,18879074,15247b5,...) at tty_wait+0x35/frame 0x205ffafc ttydisc_read(18879000,205ffbc8,0) at ttydisc_read+0x1c8/frame 0x205ffb30 ttydev_read(18ac0a00,205ffbc8,0) at ttydev_read+0x3d/frame 0x205ffb4c devfs_read_f(18de7f78,205ffbc8,24455900,0,20b53700) at devfs_read_f+0xaa/frame 0x205ffb7c dofileread(18de7f78,205ffbc8,ffffffff,ffffffff,0) at dofileread+0x6d/frame 0x205ffbac sys_read(20b53700,20b53998,20b53700,20b53700,205ffcdc,...) at sys_read+0x67/frame 0x205ffc00 syscall(205ffce8,3b,3b,3b,0,...) at syscall+0x1b8/frame 0x205ffcdc Xint0x80_syscall() at 0xffc033f9/frame 0x205ffcdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01d98, esp = 0xffc0bfe8, ebp = 0xffbfe808 --- Tracing command getty pid 841 tid 100113 td 0x18797700 sched_switch(18797700,104) at sched_switch+0x4c0/frame 0x204e0a28 mi_switch(104) at mi_switch+0x1c4/frame 0x204e0a5c sleepq_switch(2a00,18797700,0,0,1d926d4,...) at sleepq_switch+0xeb/frame 0x204e0a7c sleepq_catch_signals(204e0ae4,f92178,18b3dc74,0,15247b5,...) at sleepq_catch_signals+0x278/frame 0x204e0aa4 sleepq_wait_sig(18b3dc74,0) at sleepq_wait_sig+0xe/frame 0x204e0aac _cv_wait_sig(18b3dc74,18b3dc04) at _cv_wait_sig+0x138/frame 0x204e0ae4 tty_wait(18b3dc00,18b3dc74,18b3dc2c,18b3dc74,15247b5,...) at tty_wait+0x35/frame 0x204e0afc ttydisc_read(18b3dc00,204e0bc8,0) at ttydisc_read+0x1c8/frame 0x204e0b30 ttydev_read(18ac0800,204e0bc8,0) at ttydev_read+0x3d/frame 0x204e0b4c devfs_read_f(18dfc654,204e0bc8,24455b00,0,18797700) at devfs_read_f+0xaa/frame 0x204e0b7c dofileread(18dfc654,204e0bc8,ffffffff,ffffffff,0) at dofileread+0x6d/frame 0x204e0bac sys_read(18797700,18797998,18797700,18797700,204e0cdc,...) at sys_read+0x67/frame 0x204e0c00 syscall(204e0ce8,3b,3b,3b,0,...) at syscall+0x1b8/frame 0x204e0cdc Xint0x80_syscall() at 0xffc033f9/frame 0x204e0cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01e18, esp = 0xffc0dfe8, ebp = 0xffbfe808 --- Tracing command getty pid 842 tid 100138 td 0x20b53380 sched_switch(20b53380,104) at sched_switch+0x4c0/frame 0x205faa28 mi_switch(104) at mi_switch+0x1c4/frame 0x205faa5c sleepq_switch(2b00,20b53380,0,0,1d927d4,...) at sleepq_switch+0xeb/frame 0x205faa7c sleepq_catch_signals(205faae4,f92178,18b3d874,0,15247b5,...) at sleepq_catch_signals+0x278/frame 0x205faaa4 sleepq_wait_sig(18b3d874,0) at sleepq_wait_sig+0xe/frame 0x205faaac _cv_wait_sig(18b3d874,18b3d804) at _cv_wait_sig+0x138/frame 0x205faae4 tty_wait(18b3d800,18b3d874,18b3d82c,18b3d874,15247b5,...) at tty_wait+0x35/frame 0x205faafc ttydisc_read(18b3d800,205fabc8,0) at ttydisc_read+0x1c8/frame 0x205fab30 ttydev_read(18ac0600,205fabc8,0) at ttydev_read+0x3d/frame 0x205fab4c devfs_read_f(20dff7f8,205fabc8,18d0d200,0,20b53380) at devfs_read_f+0xaa/frame 0x205fab7c dofileread(20dff7f8,205fabc8,ffffffff,ffffffff,0) at dofileread+0x6d/frame 0x205fabac sys_read(20b53380,20b53618,20b53380,20b53380,205facdc,...) at sys_read+0x67/frame 0x205fac00 syscall(205face8,3b,3b,3b,0,...) at syscall+0x1b8/frame 0x205facdc Xint0x80_syscall() at 0xffc033f9/frame 0x205facdc --- syscall (10, FreeBSD ELF32, sys_unlink), eip = 0xffc01d18, esp = 0xffc0afe8, ebp = 0xffbfe73c --- Tracing command getty pid 843 tid 100129 td 0x20d35880 sched_switch(20d35880,104) at sched_switch+0x4c0/frame 0x205aca28 mi_switch(104) at mi_switch+0x1c4/frame 0x205aca5c sleepq_switch(2800,20d35880,0,0,1d924d4,...) at sleepq_switch+0xeb/frame 0x205aca7c sleepq_catch_signals(205acae4,f92178,18b3d474,0,15247b5,...) at sleepq_catch_signals+0x278/frame 0x205acaa4 sleepq_wait_sig(18b3d474,0) at sleepq_wait_sig+0xe/frame 0x205acaac _cv_wait_sig(18b3d474,18b3d404) at _cv_wait_sig+0x138/frame 0x205acae4 tty_wait(18b3d400,18b3d474,18b3d42c,18b3d474,15247b5,...) at tty_wait+0x35/frame 0x205acafc ttydisc_read(18b3d400,205acbc8,0) at ttydisc_read+0x1c8/frame 0x205acb30 ttydev_read(18ac0400,205acbc8,0) at ttydev_read+0x3d/frame 0x205acb4c devfs_read_f(18dfc924,205acbc8,24455d00,0,20d35880) at devfs_read_f+0xaa/frame 0x205acb7c dofileread(18dfc924,205acbc8,ffffffff,ffffffff,0) at dofileread+0x6d/frame 0x205acbac sys_read(20d35880,20d35b18,20d35880,20d35880,205accdc,...) at sys_read+0x67/frame 0x205acc00 syscall(205acce8,3b,3b,3b,0,...) at syscall+0x1b8/frame 0x205accdc Xint0x80_syscall() at 0xffc033f9/frame 0x205accdc --- syscall (10, FreeBSD ELF32, sys_unlink), eip = 0xffc01d18, esp = 0xffc0afe8, ebp = 0xffbfe73c --- Tracing command getty pid 844 tid 100141 td 0x23615880 sched_switch(23615880,104) at sched_switch+0x4c0/frame 0x20f07a28 mi_switch(104) at mi_switch+0x1c4/frame 0x20f07a5c sleepq_switch(2900,23615880,0,0,1d925d4,...) at sleepq_switch+0xeb/frame 0x20f07a7c sleepq_catch_signals(20f07ae4,f92178,18b3d074,0,15247b5,...) at sleepq_catch_signals+0x278/frame 0x20f07aa4 sleepq_wait_sig(18b3d074,0) at sleepq_wait_sig+0xe/frame 0x20f07aac _cv_wait_sig(18b3d074,18b3d004) at _cv_wait_sig+0x138/frame 0x20f07ae4 tty_wait(18b3d000,18b3d074,18b3d02c,18b3d074,15247b5,...) at tty_wait+0x35/frame 0x20f07afc ttydisc_read(18b3d000,20f07bc8,0) at ttydisc_read+0x1c8/frame 0x20f07b30 ttydev_read(18ac0200,20f07bc8,0) at ttydev_read+0x3d/frame 0x20f07b4c devfs_read_f(20dffce4,20f07bc8,249dd900,0,23615880) at devfs_read_f+0xaa/frame 0x20f07b7c dofileread(20dffce4,20f07bc8,ffffffff,ffffffff,0) at dofileread+0x6d/frame 0x20f07bac sys_read(23615880,23615b18,23615880,23615880,20f07cdc,...) at sys_read+0x67/frame 0x20f07c00 syscall(20f07ce8,3b,3b,3b,0,...) at syscall+0x1b8/frame 0x20f07cdc Xint0x80_syscall() at 0xffc033f9/frame 0x20f07cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01d98, esp = 0xffc0bfe8, ebp = 0xffbfe808 --- Tracing command getty pid 845 tid 100191 td 0x23615500 sched_switch(23615500,104) at sched_switch+0x4c0/frame 0x20f02a28 mi_switch(104) at mi_switch+0x1c4/frame 0x20f02a5c sleepq_switch(1200,23615500,0,0,1d90ed4,...) at sleepq_switch+0xeb/frame 0x20f02a7c sleepq_catch_signals(20f02ae4,f92178,18be3c74,0,15247b5,...) at sleepq_catch_signals+0x278/frame 0x20f02aa4 sleepq_wait_sig(18be3c74,0) at sleepq_wait_sig+0xe/frame 0x20f02aac _cv_wait_sig(18be3c74,18be3c04) at _cv_wait_sig+0x138/frame 0x20f02ae4 tty_wait(18be3c00,18be3c74,18be3c2c,18be3c74,15247b5,...) at tty_wait+0x35/frame 0x20f02afc ttydisc_read(18be3c00,20f02bc8,0) at ttydisc_read+0x1c8/frame 0x20f02b30 ttydev_read(18ac0000,20f02bc8,0) at ttydev_read+0x3d/frame 0x20f02b4c devfs_read_f(18dfc7bc,20f02bc8,249fec00,0,23615500) at devfs_read_f+0xaa/frame 0x20f02b7c dofileread(18dfc7bc,20f02bc8,ffffffff,ffffffff,0) at dofileread+0x6d/frame 0x20f02bac sys_read(23615500,23615798,20f02c50,23615500,9a5b030,...) at sys_read+0x67/frame 0x20f02c00 syscall(20f02ce8,3b,3b,3b,0,...) at syscall+0x1b8/frame 0x20f02cdc Xint0x80_syscall() at 0xffc033f9/frame 0x20f02cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01e18, esp = 0xffc0dfe8, ebp = 0xffbfe808 --- Tracing command getty pid 846 tid 100192 td 0x23615180 sched_switch(23615180,104) at sched_switch+0x4c0/frame 0x20efda28 mi_switch(104) at mi_switch+0x1c4/frame 0x20efda5c sleepq_switch(e00,23615180,0,0,1d90ad4,...) at sleepq_switch+0xeb/frame 0x20efda7c sleepq_catch_signals(20efdae4,f92178,182c4c74,0,15247b5,...) at sleepq_catch_signals+0x278/frame 0x20efdaa4 sleepq_wait_sig(182c4c74,0) at sleepq_wait_sig+0xe/frame 0x20efdaac _cv_wait_sig(182c4c74,182c4c04) at _cv_wait_sig+0x138/frame 0x20efdae4 tty_wait(182c4c00,182c4c74,182c4c2c,182c4c74,15247b5,...) at tty_wait+0x35/frame 0x20efdafc ttydisc_read(182c4c00,20efdbc8,0) at ttydisc_read+0x1c8/frame 0x20efdb30 ttydev_read(1824f200,20efdbc8,0) at ttydev_read+0x3d/frame 0x20efdb4c devfs_read_f(18dfc474,20efdbc8,24455800,0,23615180) at devfs_read_f+0xaa/frame 0x20efdb7c dofileread(18dfc474,20efdbc8,ffffffff,ffffffff,0) at dofileread+0x6d/frame 0x20efdbac sys_read(23615180,23615418,23615180,23615180,20efdcdc,...) at sys_read+0x67/frame 0x20efdc00 syscall(20efdce8,3b,3b,3b,0,...) at syscall+0x1b8/frame 0x20efdcdc Xint0x80_syscall() at 0xffc033f9/frame 0x20efdcdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01c98, esp = 0xffc07fe8, ebp = 0xffbfeccc --- Tracing command sleep pid 40916 tid 100286 td 0x308f9a80 sched_switch(308f9a80,104) at sched_switch+0x4c0/frame 0x2bab9a88 mi_switch(104) at mi_switch+0x1c4/frame 0x2bab9ac0 sleepq_switch(b40,308f9a80,6c,0,1d90814,...) at sleepq_switch+0xeb/frame 0x2bab9ae0 sleepq_catch_signals(308f9a80,100,1d7e2cf,2bab9b6c,308f9a80,...) at sleepq_catch_signals+0x278/frame 0x2bab9b08 sleepq_timedwait_sig(1d7e2cf,6c,2bab9b60,fd34c7,308f60ac,...) at sleepq_timedwait_sig+0x14/frame 0x2bab9b2c _sleep(1d7e2cf,0,16c,1510816,ad67bde2,1ef6,a0000000,0,200) at _sleep+0x224/frame 0x2bab9b6c kern_clock_nanosleep(308f9a80,0,0,2bab9bec,2bab9be4) at kern_clock_nanosleep+0x1a1/frame 0x2bab9bc8 sys_nanosleep(308f9a80,308f9d18,308f9a80,308f9a80,2bab9cdc,...) at sys_nanosleep+0x34/frame 0x2bab9c00 syscall(2bab9ce8,3b,3b,3b,ffbfe720,...) at syscall+0x1b8/frame 0x2bab9cdc Xint0x80_syscall() at 0xffc033f9/frame 0x2bab9cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01e18, esp = 0xffc0dfe8, ebp = 0xffbfe808 --- Tracing command sshd pid 1004 tid 100149 td 0x20cfc000 sched_switch(20cfc000,104) at sched_switch+0x4c0/frame 0x205d29b0 mi_switch(104) at mi_switch+0x1c4/frame 0x205d29e4 sleepq_switch(3a40,20cfc000,0,0,1d93714,...) at sleepq_switch+0xeb/frame 0x205d2a04 sleepq_catch_signals(205d2a6c,f92178,21048d64,0,1414b23,...) at sleepq_catch_signals+0x278/frame 0x205d2a2c sleepq_wait_sig(21048d64,0) at sleepq_wait_sig+0xe/frame 0x205d2a34 _cv_wait_sig(21048d64,21048d50) at _cv_wait_sig+0x138/frame 0x205d2a6c seltdwait(ffffffff,0,0) at seltdwait+0x7f/frame 0x205d2a88 kern_poll(20cfc000,ffbfe7a4,1,0,0) at kern_poll+0x450/frame 0x205d2bd8 sys_poll(20cfc000,20cfc298,20cfc000,20cfc000,205d2cdc,...) at sys_poll+0x4d/frame 0x205d2c00 syscall(205d2ce8,3b,3b,3b,20482000,...) at syscall+0x1b8/frame 0x205d2cdc Xint0x80_syscall() at 0xffc033f9/frame 0x205d2cdc --- syscall (10, FreeBSD ELF32, sys_unlink), eip = 0xffc01d18, esp = 0xffc0afe8, ebp = 0xffbfe73c --- Tracing command sshd pid 1006 tid 100194 td 0x23614a80 sched_switch(23614a80,104) at sched_switch+0x4c0/frame 0x20ef3978 mi_switch(104) at mi_switch+0x1c4/frame 0x20ef39ac sleepq_switch(3c00,23614a80,0,0,1d938d4,...) at sleepq_switch+0xeb/frame 0x20ef39cc sleepq_catch_signals(20ef3a34,f92178,1889d424,0,1414b23,...) at sleepq_catch_signals+0x278/frame 0x20ef39f4 sleepq_wait_sig(1889d424,0) at sleepq_wait_sig+0xe/frame 0x20ef39fc _cv_wait_sig(1889d424,1889d410) at _cv_wait_sig+0x138/frame 0x20ef3a34 seltdwait(ffffffff,0,0) at seltdwait+0x7f/frame 0x20ef3a50 kern_select(23614a80,e,2047c270,2047c278,0,0,20) at kern_select+0x749/frame 0x20ef3bcc sys_select(23614a80,23614d18,0,23614a80,1daa,...) at sys_select+0x3f/frame 0x20ef3c00 syscall(20ef3ce8,3b,3b,3b,20482000,...) at syscall+0x1b8/frame 0x20ef3cdc Xint0x80_syscall() at 0xffc033f9/frame 0x20ef3cdc --- syscall (10, FreeBSD ELF32, sys_unlink), eip = 0xffc01d18, esp = 0xffc0afe8, ebp = 0xffbfe73c --- Tracing command bash pid 1007 tid 100193 td 0x23614e00 sched_switch(23614e00,104) at sched_switch+0x4c0/frame 0x20ef89c8 mi_switch(104) at mi_switch+0x1c4/frame 0x20ef89fc sleepq_switch(f40,14fd263,6c,0,1d90c14,...) at sleepq_switch+0xeb/frame 0x20ef8a1c sleepq_catch_signals(20ef8a8c,ffddc4,23611d20,6c,200246,...) at sleepq_catch_signals+0x278/frame 0x20ef8a44 sleepq_wait_sig(23611d20,6c) at sleepq_wait_sig+0xe/frame 0x20ef8a4c _sleep(23611d20,23611dcc,36c,1519f00,0,0,0,0,100) at _sleep+0x244/frame 0x20ef8a8c kern_wait6(23614e00,7,0,0,20ef8be8,36,0,0) at kern_wait6+0x3b8/frame 0x20ef8ae8 sys_wait4(23614e00,23615098,23614e00,23614e00,20ef8cdc,...) at sys_wait4+0x73/frame 0x20ef8c00 syscall(20ef8ce8,3b,3b,3b,0,...) at syscall+0x1b8/frame 0x20ef8cdc Xint0x80_syscall() at 0xffc033f9/frame 0x20ef8cdc --- syscall (10, FreeBSD ELF32, sys_unlink), eip = 0xffc01d18, esp = 0xffc0afe8, ebp = 0xffbfe73c --- Tracing command su pid 1016 tid 100201 td 0x241df000 sched_switch(241df000,104) at sched_switch+0x4c0/frame 0x20f759c8 mi_switch(104) at mi_switch+0x1c4/frame 0x20f759fc sleepq_switch(10c0,14fd263,6c,0,1d90d94,...) at sleepq_switch+0xeb/frame 0x20f75a1c sleepq_catch_signals(20f75a8c,ffddc4,249df3b0,6c,200246,...) at sleepq_catch_signals+0x278/frame 0x20f75a44 sleepq_wait_sig(249df3b0,6c) at sleepq_wait_sig+0xe/frame 0x20f75a4c _sleep(249df3b0,249df45c,36c,1519f00,0,0,0,0,100) at _sleep+0x244/frame 0x20f75a8c kern_wait6(241df000,0,3f9,0,20f75be8,32,0,0) at kern_wait6+0x3b8/frame 0x20f75ae8 sys_wait4(241df000,241df298,241df000,241df000,20f75cdc,...) at sys_wait4+0x73/frame 0x20f75c00 syscall(20f75ce8,3b,3b,3b,ffbfe634,...) at syscall+0x1b8/frame 0x20f75cdc Xint0x80_syscall() at 0xffc033f9/frame 0x20f75cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01c98, esp = 0xffc07fe8, ebp = 0xffbfeccc --- Tracing command csh pid 1017 tid 100200 td 0x263e2500 sched_switch(263e2500,104) at sched_switch+0x4c0/frame 0x24358ac0 mi_switch(104) at mi_switch+0x1c4/frame 0x24358af8 sleepq_switch(700,14fd263,74,0,1d903d4,...) at sleepq_switch+0xeb/frame 0x24358b18 sleepq_catch_signals(24358b88,ffddc4,263e0418,74,24358b98,...) at sleepq_catch_signals+0x278/frame 0x24358b40 sleepq_wait_sig(263e0418,74) at sleepq_wait_sig+0xe/frame 0x24358b48 _sleep(263e0418,263e045c,174,145ce5e,0,0,0,0,100) at _sleep+0x244/frame 0x24358b88 kern_sigsuspend(263e2500,2,0,0,0) at kern_sigsuspend+0xca/frame 0x24358bd0 sys_sigsuspend(263e2500,263e2798,263e2500,263e2500,24358cdc,...) at sys_sigsuspend+0x36/frame 0x24358c00 syscall(24358ce8,3b,3b,3b,ffbfe094,...) at syscall+0x1b8/frame 0x24358cdc Xint0x80_syscall() at 0xffc033f9/frame 0x24358cdc --- syscall (10, FreeBSD ELF32, sys_unlink), eip = 0xffc01d18, esp = 0xffc0afe8, ebp = 0xffbfe73c --- Tracing command bash pid 1019 tid 100198 td 0x263e2c00 sched_switch(263e2c00,104) at sched_switch+0x4c0/frame 0x243629c8 mi_switch(104) at mi_switch+0x1c4/frame 0x243629fc sleepq_switch(1280,14fd263,6c,0,1d90f54,...) at sleepq_switch+0xeb/frame 0x24362a1c sleepq_catch_signals(24362a8c,ffddc4,263e0a40,6c,200246,...) at sleepq_catch_signals+0x278/frame 0x24362a44 sleepq_wait_sig(263e0a40,6c) at sleepq_wait_sig+0xe/frame 0x24362a4c _sleep(263e0a40,263e0aec,36c,1519f00,0,0,0,0,100) at _sleep+0x244/frame 0x24362a8c kern_wait6(263e2c00,7,0,0,24362be8,36,0,0) at kern_wait6+0x3b8/frame 0x24362ae8 sys_wait4(263e2c00,263e2e98,263e2c00,263e2c00,24362cdc,...) at sys_wait4+0x73/frame 0x24362c00 syscall(24362ce8,3b,3b,3b,0,...) at syscall+0x1b8/frame 0x24362cdc Xint0x80_syscall() at 0xffc033f9/frame 0x24362cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0xffc01d98, esp = 0xffc0bfe8, ebp = 0xffbfe808 --- db:0:pho> dump Physical memory: 3428 MB Dumping 431 MB: 416 400 384 368 352 336 320 304 288 272 256 240 224 208 192 176 160 144 128 112 96 80 64 48 32 16 Dump complete db> reset cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 1 (kgdb) thread 112 [Switching to thread 112 (Thread 100248)] #0 0x0101d9a0 in sched_switch (td=0x27519380, flags=0x104) at ../../../kern/sched_ule.c:2146 2146 cpu_switch(td, newtd, mtx); (kgdb) bt #0 0x0101d9a0 in sched_switch (td=0x27519380, flags=0x104) at ../../../kern/sched_ule.c:2146 #1 0x00ffe854 in mi_switch (flags=0x104) at ../../../kern/kern_synch.c:542 #2 0x0104585b in sleepq_switch (wchan=, wchan@entry=0x200000, pri=0x60, pri@entry=0x0) at ../../../kern/subr_sleepqueue.c:608 #3 0x01045748 in sleepq_wait (wchan=0x27bbbde0, pri=0x142734d) at ../../../kern/subr_sleepqueue.c:659 #4 0x00fca000 in sleeplk (lk=lk@entry=0x27bbbd74, flags=flags@entry=0x200000, ilk=0x27bbbde0, wmesg=0x142734d "ufs", pri=, timo=0x33, queue=0x1) at ../../../kern/kern_lock.c:301 #5 0x00fc82db in lockmgr_slock_hard (lk=lk@entry=0x27bbbd74, flags=, flags@entry=0x200000, ilk=0x27bbbde0, file=0x149c697 "../../../kern/vfs_subr.c", line=0xb64, lwa=0x0) at ../../../kern/kern_lock.c:696 #6 0x00fc7ee7 in lockmgr_lock_flags (lk=0x27bbbd74, flags=0x200000, ilk=0x27bbbde0, file=0x149c697 "../../../kern/vfs_subr.c", line=0xb64) at ../../../kern/kern_lock.c:1054 #7 0x0127b6d9 in ffs_lock (ap=0x2afd38d8) at ../../../ufs/ffs/ffs_vnops.c:494 #8 0x01400048 in VOP_LOCK1_APV (vop=0x1acb2bc , a=0x2afd38d8) at vnode_if.c:2186 #9 0x010d3929 in VOP_LOCK1 (vp=0x27bbbd38, flags=0x200000, file=0x149c697 "../../../kern/vfs_subr.c", line=) at ./vnode_if.h:1100 #10 _vn_lock (vp=0x27bbbd38, flags=0x200000, file=0x149c697 "../../../kern/vfs_subr.c", line=0xb64) at ../../../kern/vfs_vnops.c:1741 #11 0x010bd19a in vget_finish (vp=0x27bbbd38, flags=0x200000, vs=VGET_USECOUNT) at ../../../kern/vfs_subr.c:2916 #12 0x010a6692 in cache_fplookup_final_child (fpl=fpl@entry=0x2afd3950, tvs=, tvs@entry=VGET_USECOUNT) at ../../../kern/vfs_cache.c:4082 #13 0x010a37d9 in cache_fplookup_final (fpl=) at ../../../kern/vfs_cache.c:4388 #14 cache_fplookup_impl (dvp=, fpl=) at ../../../kern/vfs_cache.c:5271 #15 cache_fplookup (ndp=0x2afd3a54, status=0x2afd3a10, pwdp=0x2afd3a00) at ../../../kern/vfs_cache.c:5427 #16 0x010af905 in namei (ndp=0x2afd3a54) at ../../../kern/vfs_lookup.c:601 #17 0x010cc63b in kern_statat (td=0x27519380, flag=0x200, fd=0xffffff9c, path=0x206c14f4 "ÿÿÿÿ@", pathseg=UIO_USERSPACE, sbp=0x2afd3b28, hook=0x0) at ../../../kern/vfs_syscalls.c:2405 #18 0x010cceb9 in sys_fstatat (td=0x27519380, uap=0x27519618) at ../../../kern/vfs_syscalls.c:2382 #19 0x013ebdd8 in syscallenter (td=) at ../../../i386/i386/../../kern/subr_syscall.c:189 #20 syscall (frame=0x2afd3ce8) at ../../../i386/i386/trap.c:1127 #21 0xffc033f9 in ?? () #22 0x2afd3ce8 in ?? () Backtrace stopped: Cannot access memory at address 0xffc0afac (kgdb) f 11 #11 0x010bd19a in vget_finish (vp=0x27bbbd38, flags=0x200000, vs=VGET_USECOUNT) at ../../../kern/vfs_subr.c:2916 2916 error = vn_lock(vp, flags); (kgdb) p *vp $1 = {v_type = VDIR, v_irflag = 0x0, v_seqc = 0x1d1698, v_nchash = 0x36a8c7c4, v_op = 0x1acb2bc , v_data = 0x2ac43960, v_mount = 0x27c67500, v_nmntvnodes = {tqe_next = 0x241548d0, tqe_prev = 0x27c67568}, { v_mountedhere = 0x0, v_unpcb = 0x0, v_rdev = 0x0, v_fifoinfo = 0x0}, v_hashlist = {le_next = 0x0, le_prev = 0x186f3d04}, v_cache_src = {lh_first = 0x2942a3a8}, v_cache_dst = {tqh_first = 0x0, tqh_last = 0x27bbbd68}, v_cache_dd = 0x0, v_lock = {lock_object = {lo_name = 0x142734d "ufs", lo_flags = 0x6fb0000, lo_data = 0x0, lo_witness = 0x2b3b808}, lk_lock = 0x263e1006, lk_exslpfail = 0x0, lk_timo = 0x33, lk_pri = 0x60, lk_stack = { depth = 0xd, pcs = {0xfc8818, 0xfc7ed3, 0x127b6d9, 0x1400048, 0x10d3929, 0x10a7c0c, 0x10a6eb9, 0x10a34d9, 0x10af905, 0x10cbae5, 0x10cb9bf, 0x13ebdd8, 0xffc033f9, 0x0, 0x0, 0x0, 0x0, 0x0}}}, v_interlock = { lock_object = {lo_name = 0x1538441 "vnode interlock", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0x2b31178}, mtx_lock = 0x0}, v_vnlock = 0x27bbbd74, v_vnodelist = {tqe_next = 0x2416d8d0, tqe_prev = 0x207d0528}, v_lazylist = {tqe_next = 0x0, tqe_prev = 0x27c67ef0}, v_bufobj = {bo_lock = {lock_object = {lo_name = 0x14fd6c5 "bufobj interlock", lo_flags = 0x5230000, lo_data = 0x0, lo_witness = 0x2b393e0}, rw_lock = 0x1}, bo_ops = 0x1a9e270 , bo_object = 0x2abe470c, bo_synclist = {le_next = 0x0, le_prev = 0x0}, bo_private = 0x27bbbd38, bo_clean = {bv_hd = {tqh_first = 0xb69b7a8, tqh_last = 0xb711974}, bv_root = { pt_root = 0x1856a9a0}, bv_cnt = 0xb4}, bo_dirty = {bv_hd = {tqh_first = 0x0, tqh_last = 0x27bbbe40}, bv_root = {pt_root = 0x0}, bv_cnt = 0x0}, bo_numoutput = 0x0, bo_flag = 0x0, bo_domain = 0x0, bo_bsize = 0x8000}, v_pollinfo = 0x0, v_label = 0x0, v_lockf = 0x0, v_rl = {rl_waiters = {tqh_first = 0x0, tqh_last = 0x27bbbe6c}, rl_currdep = 0x0}, v_cstart = 0x0, v_lasta = 0x0, v_lastw = 0x0, v_clen = 0x0, v_holdcnt = 0xb7, v_usecount = 0x5, v_iflag = 0x0, v_vflag = 0x1, v_mflag = 0x1, v_dbatchcpu = 0xffff, v_writecount = 0x0, v_seqc_users = 0x0, v_hash = 0x2} (kgdb) p *vp->v_vnlock $2 = {lock_object = {lo_name = 0x142734d "ufs", lo_flags = 0x6fb0000, lo_data = 0x0, lo_witness = 0x2b3b808}, lk_lock = 0x263e1006, lk_exslpfail = 0x0, lk_timo = 0x33, lk_pri = 0x60, lk_stack = {depth = 0xd, pcs = {0xfc8818, 0xfc7ed3, 0x127b6d9, 0x1400048, 0x10d3929, 0x10a7c0c, 0x10a6eb9, 0x10a34d9, 0x10af905, 0x10cbae5, 0x10cb9bf, 0x13ebdd8, 0xffc033f9, 0x0, 0x0, 0x0, 0x0, 0x0}}} (kgdb) git diff -- Test scenario: misc/perf.sh