GENERIC HEAD from 2017-05-30 12:00:56 UTC, r319211M, vmcore.1021 GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2017 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 12.0-CURRENT #6 r319211M: Wed May 31 08:54:09 CEST 2017 pho@x4.osted.lan:/usr/src/sys/i386/compile/PHO i386 FreeBSD clang version 4.0.0 (tags/RELEASE_400/final 297347) (based on LLVM 4.0.0) WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. VT(vga): resolution 640x480 CPU: AMD Phenom(tm) 9150e Quad-Core Processor (1800.01-MHz 686-class CPU) Origin="AuthenticAMD" Id=0x100f23 Family=0x10 Model=0x2 Stepping=3 Features=0x178bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x7ff SVM: NP,NAsids=64 TSC: P-state invariant real memory = 8589934592 (8192 MB) avail memory = 3525517312 (3362 MB) : Trying to mount root from ufs:/dev/ada0s1a [rw]... uhub1: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered uhub0: 2 ports with 2 removable, self powered uhub4: 2 ports with 2 removable, self powered uhub5: 2 ports with 2 removable, self powered Setting hostuuid: 00000000-0000-0000-0000-00218515337d. Setting hostid: 0x6b64ac17. warning: total configured swap (4194304 pages) exceeds maximum recommended amount (2097312 pages). warning: increase kern.maxswzone or reduce amount of swap. Starting file system checks: /dev/ada0s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ada0s1a: clean, 617734 free (2742 frags, 76874 blocks, 0.3% fragmentation) /dev/ada0s1f: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ada0s1f: clean, 859250 free (10682 frags, 106071 blocks, 1.1% fragmentation) /dev/label/tmp: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/label/tmp: clean, 54983198 free (37854 frags, 6868168 blocks, 0.0% fragmentation) /dev/ada0s1d: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ada0s1d: clean, 1704080 free (366560 frags, 167190 blocks, 3.6% fragmentation) /dev/ada0s1e: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ada0s1e: clean, 1477759 free (260991 frags, 152096 blocks, 1.0% fragmentation) Mounting local file systems:. ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/arduino/lib /usr/local/lib/compat /usr/local/lib/compat/pkg /usr/local/lib/gcc49 /usr/local/lib/mysql /usr/local/lib/nss /usr/local/lib/perl5/5.24/mach/CORE /usr/local/lib/pth /usr/local/lib/qt4 /usr/local/llvm37/lib /usr/local/llvm39/lib a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout Setting hostname: x4.osted.lan. Setting up harvesting:[UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,NET_ETHER,NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED Feeding entropy:. uhub2: 10 ports with 10 removable, self powered re0: link state changed to DOWN ugen2.2: at usbus2 Starting Network: lo0 re0. lo0: flags=8049 metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 nd6 options=21 groups: lo re0: flags=8843 metric 0 mtu 1500 options=8209b ether 00:21:85:15:33:7d hwaddr 00:21:85:15:33:7d inet 192.168.1.101 netmask 0xffffff00 broadcast 192.168.1.255 nd6 options=29 media: Ethernet autoselect (none) status: no carrier Starting devd. uslcom0 on uhub0 uslcom0: on usbus2 add net default: gateway 192.168.1.1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Creating and/or trimming log files. Starting syslogd. Starting rpcbind. NFS access cache time=60 Starting watchdogd. Starting ddb. No core dumps found. Additional ABI support: linux. lock order reversal: 1st 0xe4919d10 bufwait (bufwait) @ kern/vfs_bio.c:3509 2nd 0xc93d8e00 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:281 stack backtrace: #0 0xc0cbd801 at witness_debugger+0x81 #1 0xc0cbd721 at witness_checkorder+0xd11 #2 0xc0c61a5a at _sx_xlock+0x5a #3 0xc0f7a7d3 at ufsdirhash_add+0x53 #4 0xc0f7f24a at ufs_direnter+0x47a #5 0xc0f88544 at ufs_mkdir+0x8b4 #6 0xc125b569 at VOP_MKDIR_APV+0xf9 #7 0xc0d39cbe at kern_mkdirat+0x21e #8 0xc0d39a91 at sys_mkdir+0x31 #9 0xc122631c at syscall+0x75c #10 0xc1211cfe at Xint0x80_syscall+0x2e Clearing /tmp (X related). Starting nfsuserd. Starting mountd. Starting nfsd. Starting statd. Starting lockd. Recovering vi editor sessions:Insufficient disk space; try again later May 31 08:58:40 x4 sendmail[555]: v4V6weFR000555: low on space (SMTP-DAEMON needs 219 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 Insufficient disk space; try again later returntosender: cannot select queue for root Insufficient disk space; try again later returntosender: cannot select queue for postmaster May 31 08:58:40 x4 sendmail[555]: v4V6weFS000555: low on space (SMTP-DAEMON needs 0 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:40 x4 sendmail[555]: v4V6weFR000555: SYSERR(root): returntosender: cannot select queue for root May 31 08:58:40 x4 sendmail[555]: v4V6weFT000555: low on space (SMTP-DAEMON needs 0 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:40 x4 sendmail[555]: v4V6weFR000555: SYSERR(root): returntosender: cannot select queue for postmaster May 31 08:58:40 x4 sendmail[555]: v4V6weFR000555: low on space (SMTP-DAEMON needs 516 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:40 x4 sendmail[555]: v4V6weFR000555: low on space (SMTP-DAEMON needs 516 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 Insufficient disk space; try again later May 31 08:58:41 x4 sendmail[557]: v4V6wfqG000557: low on space (SMTP-DAEMON needs 219 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 Insufficient disk space; try again later returntosender: cannot select queue for root May 31 08:58:41 x4 sendmail[557]: v4V6wfqH000557: low on space (SMTP-DAEMON needs 0 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:41 x4 sendmail[557]: v4V6wfqG000557: SYSERR(root): returntosender: cannot select queue for root Insufficient disk space; try again later returntosender: cannot select queue for postmaster May 31 08:58:41 x4 sendmail[557]: v4V6wfqI000557: low on space (SMTP-DAEMON needs 0 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:41 x4 sendmail[557]: v4V6wfqG000557: SYSERR(root): returntosender: cannot select queue for postmaster May 31 08:58:41 x4 sendmail[557]: v4V6wfqG000557: low on space (SMTP-DAEMON needs 516 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 Insufficient disk space; try again later May 31 08:58:41 x4 sendmail[557]: v4V6wfqG000557: low on space (SMTP-DAEMON needs 516 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:41 x4 sendmail[559]: v4V6wfqC000559: low on space (SMTP-DAEMON needs 219 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 Insufficient disk space; try again later returntosender: cannot select queue for root May 31 08:58:41 x4 sendmail[559]: v4V6wfqD000559: low on space (SMTP-DAEMON needs 0 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:41 x4 sendmail[559]: v4V6wfqC000559: SYSERR(root): returntosender: cannot select queue for root Insufficient disk space; try again later returntosender: cannot select queue for postmaster May 31 08:58:41 x4 sendmail[559]: v4V6wfqE000559: low on space (SMTP-DAEMON needs 0 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:41 x4 sendmail[559]: v4V6wfqC000559: SYSERR(root): returntosender: cannot select queue for postmaster May 31 08:58:41 x4 sendmail[559]: v4V6wfqC000559: low on space (SMTP-DAEMON needs 516 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:41 x4 sendmail[559]: v4V6wfqC000559: low on space (SMTP-DAEMON needs 516 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 Insufficient disk space; try again later May 31 08:58:41 x4 sendmail[561]: v4V6wfCs000561: low on space (SMTP-DAEMON needs 219 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 Insufficient disk space; try again later returntosender: cannot select queue for root May 31 08:58:41 x4 sendmail[561]: v4V6wfCt000561: low on space (SMTP-DAEMON needs 0 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:41 x4 sendmail[561]: v4V6wfCs000561: SYSERR(root): returntosender: cannot select queue for root Insufficient disk space; try again later returntosender: cannot select queue for postmaster May 31 08:58:41 x4 sendmail[561]: v4V6wfCu000561: low on space (SMTP-DAEMON needs 0 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:41 x4 sendmail[561]: v4V6wfCs000561: SYSERR(root): returntosender: cannot select queue for postmaster May 31 08:58:41 x4 sendmail[561]: v4V6wfCs000561: low on space (SMTP-DAEMON needs 516 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 Insufficient disk space; try again later May 31 08:58:41 x4 sendmail[561]: v4V6wfCs000561: low on space (SMTP-DAEMON needs 516 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:41 x4 sendmail[563]: v4V6wfUi000563: low on space (SMTP-DAEMON needs 219 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 Insufficient disk space; try again later returntosender: cannot select queue for root May 31 08:58:41 x4 sendmail[563]: v4V6wfUj000563: low on space (SMTP-DAEMON needs 0 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:41 x4 sendmail[563]: v4V6wfUi000563: SYSERR(root): returntosender: cannot select queue for root Insufficient disk space; try again later returntosender: cannot select queue for postmaster May 31 08:58:41 x4 sendmail[563]: v4V6wfUk000563: low on space (SMTP-DAEMON needs 0 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:41 x4 sendmail[563]: v4V6wfUi000563: SYSERR(root): returntosender: cannot select queue for postmaster May 31 08:58:41 x4 sendmail[563]: v4V6wfUi000563: low on space (SMTP-DAEMON needs 516 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 Insufficient disk space; try again later May 31 08:58:41 x4 sendmail[563]: v4V6wfUi000563: low on space (SMTP-DAEMON needs 516 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:41 x4 sendmail[565]: v4V6wf7S000565: low on space (SMTP-DAEMON needs 219 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 Insufficient disk space; try again later returntosender: cannot select queue for root May 31 08:58:41 x4 sendmail[565]: v4V6wf7T000565: low on space (SMTP-DAEMON needs 0 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:41 x4 sendmail[565]: v4V6wf7S000565: SYSERR(root): returntosender: cannot select queue for root Insufficient disk space; try again later returntosender: cannot select queue for postmaster May 31 08:58:41 x4 sendmail[565]: v4V6wf7U000565: low on space (SMTP-DAEMON needs 0 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:41 x4 sendmail[565]: v4V6wf7S000565: SYSERR(root): returntosender: cannot select queue for postmaster May 31 08:58:41 x4 sendmail[565]: v4V6wf7S000565: low on space (SMTP-DAEMON needs 516 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:41 x4 sendmail[565]: v4V6wf7S000565: low on space (SMTP-DAEMON needs 516 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 Insufficient disk space; try again later May 31 08:58:41 x4 sendmail[567]: v4V6wfIi000567: low on space (SMTP-DAEMON needs 219 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 Insufficient disk space; try again later returntosender: cannot select queue for root May 31 08:58:41 x4 sendmail[567]: v4V6wfIj000567: low on space (SMTP-DAEMON needs 0 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:41 x4 sendmail[567]: v4V6wfIi000567: SYSERR(root): returntosender: cannot select queue for root Insufficient disk space; try again later returntosender: cannot select queue for postmaster May 31 08:58:41 x4 sendmail[567]: v4V6wfIk000567: low on space (SMTP-DAEMON needs 0 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:41 x4 sendmail[567]: v4V6wfIi000567: SYSERR(root): returntosender: cannot select queue for postmaster May 31 08:58:41 x4 sendmail[567]: v4V6wfIi000567: low on space (SMTP-DAEMON needs 516 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:41 x4 sendmail[567]: v4V6wfIi000567: low on space (SMTP-DAEMON needs 516 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 Insufficient disk space; try again later May 31 08:58:41 x4 sendmail[569]: v4V6wfT0000569: low on space (SMTP-DAEMON needs 219 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 Insufficient disk space; try again later returntosender: cannot select queue for root May 31 08:58:41 x4 sendmail[569]: v4V6wfT1000569: low on space (SMTP-DAEMON needs 0 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:41 x4 sendmail[569]: v4V6wfT0000569: SYSERR(root): returntosender: cannot select queue for root Insufficient disk space; try again later returntosender: cannot select queue for postmaster May 31 08:58:41 x4 sendmail[569]: v4V6wfT2000569: low on space (SMTP-DAEMON needs 0 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:41 x4 sendmail[569]: v4V6wfT0000569: SYSERR(root): returntosender: cannot select queue for postmaster May 31 08:58:41 x4 sendmail[569]: v4V6wfT0000569: low on space (SMTP-DAEMON needs 516 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:41 x4 sendmail[569]: v4V6wfT0000569: low on space (SMTP-DAEMON needs 516 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 Insufficient disk space; try again later May 31 08:58:41 x4 sendmail[571]: v4V6wfTT000571: low on space (SMTP-DAEMON needs 219 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 Insufficient disk space; try again later returntosender: cannot select queue for root May 31 08:58:41 x4 sendmail[571]: v4V6wfTU000571: low on space (SMTP-DAEMON needs 0 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:41 x4 sendmail[571]: v4V6wfTT000571: SYSERR(root): returntosender: cannot select queue for root Insufficient disk space; try again later returntosender: cannot select queue for postmaster May 31 08:58:41 x4 sendmail[571]: v4V6wfTV000571: low on space (SMTP-DAEMON needs 0 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:41 x4 sendmail[571]: v4V6wfTT000571: SYSERR(root): returntosender: cannot select queue for postmaster May 31 08:58:41 x4 sendmail[571]: v4V6wfTT000571: low on space (SMTP-DAEMON needs 516 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:41 x4 sendmail[571]: v4V6wfTT000571: low on space (SMTP-DAEMON needs 516 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 Insufficient disk space; try again later May 31 08:58:41 x4 sendmail[573]: v4V6wflT000573: low on space (SMTP-DAEMON needs 219 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 Insufficient disk space; try again later returntosender: cannot select queue for root May 31 08:58:41 x4 sendmail[573]: v4V6wflU000573: low on space (SMTP-DAEMON needs 0 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:41 x4 sendmail[573]: v4V6wflT000573: SYSERR(root): returntosender: cannot select queue for root Insufficient disk space; try again later returntosender: cannot select queue for postmaster May 31 08:58:41 x4 sendmail[573]: v4V6wflV000573: low on space (SMTP-DAEMON needs 0 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:41 x4 sendmail[573]: v4V6wflT000573: SYSERR(root): returntosender: cannot select queue for postmaster May 31 08:58:41 x4 sendmail[573]: v4V6wflT000573: low on space (SMTP-DAEMON needs 516 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 . Updating motd:. Mounting late file systems:. Starting ntpd. Starting powerd. Starting smartd. May 31 08:58:41 x4 sendmail[573]: v4V6wflT000573: low on space (SMTP-DAEMON needs 516 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 May 31 08:58:43 x4 smartd[637]: Device: /dev/ada0, WARNING: A firmware update for this drive may be available, May 31 08:58:43 x4 smartd[637]: see the following Seagate web pages: May 31 08:58:43 x4 smartd[637]: http://knowledge.seagate.com/articles/en_US/FAQ/207931en May 31 08:58:43 x4 smartd[637]: http://knowledge.seagate.com/articles/en_US/FAQ/213891en Configuring vt: keymap blanktime. Performing sanity check on sshd configuration. Starting sshd. Starting sendmail_submit. May 31 08:58:45 x4 sm-mta[679]: v4V6wjX9000678: low on space (SMTP-DAEMON needs 0 bytes + 100 blocks in /var/spool/mqueue), max avail: 0 Starting sendmail_msp_queue. lock order reversal: 1st 0xca08b304 ufs (ufs) @ kern/vfs_subr.c:2602 2nd 0xe4927e10 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:280 Starting cron. 3rd 0xca20273c ufs (ufs) @ kern/vfs_subr.c:2602 stack backtrace: #0 0xc0cbd801 at witness_debugger+0x81 #1 0xc0cbd721 at witness_checkorder+0xd11 #2 0xc0c2cae7 at __lockmgr_args+0x7a7 #3 0xc0f74b67 at ffs_lock+0xa7 #4 0xc125c300 at VOP_LOCK1_APV+0xe0 #5 0xc0d3d243 at _vn_lock+0x63 #6 0xc0d297a2 at vget+0x92 #7 0xc0d1786d at vfs_hash_get+0xcd #8 0xc0f6fd24 at ffs_vgetf+0x44 #9 0xc0f5a132 at flush_pagedep_deps+0x742 #10 0xc0f598f7 at softdep_sync_buf+0x437 #11 0xc0f7598f at ffs_syncvnode+0x2ef #12 0xc0f34d64 at ffs_truncate+0x974 #13 0xc0f7f3f2 at ufs_direnter+0x622 #14 0xc0f8937f at ufs_makeinode+0x5af #15 0xc0f84d18 at ufs_create+0x38 #16 0xc1258ed9 at VOP_CREATE_APV+0xf9 #17 0xc0d3ca39 at vn_open_cred+0x289 Local package initialization: backuplock order reversal: 1st 0xc9aca46c ufs (ufs) @ kern/vfs_mount.c:1355 2nd 0xc928119c devfs (devfs) @ ufs/ffs/ffs_softdep.c:1908 stack backtrace: May 31 08:58:45 #0 0xc0cbd801 at witness_debuggx4 sm-mta[679]: er+0x81 #1 0xc0cbd721 at witnesv4V6wjX9000678: s_checkorder+0xd11 #2 0xc0c2c26f at lockmgr_lock_fast_path+0x1low on space (SMdf #3 0xc0d138db at vop_stdlockTP-DAEMON needs +0x3b #4 0xc125c300 at VOP_LOCK0 bytes + 100 bl1_APV+0xe0 #5 0xc0d3d243 at _vnocks in /var/spo_lock+0x63 #6 0xc0f3eaa5 at sofol/mqueue), max tdep_flushworklist+0xa5 #7 0xc0avail: 0 May 3f72e62 at ffs_sync+0xd2 #8 0xc0d3eae5 at vfs_write_suspend+0x11 08:58:45 x4 kec5 #9 0xc0d3ed45 at vfs_write_srnel: pid 682 (suspend_umnt+0x45 #10 0xc0f727a9endmail), uid 25 at ffs_unmount+0xa9 #11 0xc0d1 inumber 1436723e784 at dounmount+0xc54 #12 0xc on /var: filesy0d1db07 at sys_unmount+0x667 #1stem full 3 0xc122631c at syscall+0x75c #14 0xc1211cfe at Xint0x80_syscall+0x2e ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes May 31 09:00:00 x4 kernel: pid 738 (dd), uid 2 inumber 1742859 on /var: filesystem full ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 921258 files, 63330783 used, 54983198 free (37854 frags, 6868168 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** fsck -y /tmp. Starting nfscbd. Starting default moused. Starting inetd. Wed May 31 09:01 FreeBSD/i386 (x4.osted.lan) (ttyu0) login: May 31 09:02:30 x4 su: pho to root on /dev/pts/1 20170531 09:02:48 all (1/1): crossmp11.sh May 31 09:04:18 x4 kernel: pid 4486 (rw), uid 1004 inumber 73743 on /mnt8: filesystem full May 31 09:04:18 x4 kernel: pid 4516 (creat), uid 1004 inumber 7 on /mnt8: filesystem full May 31 09:04:19 x4 kernel: pid 4498 (creat), uid 1004 inumber 73730 on /mnt8: filesystem full May 31 09:04:20 x4 kernel: pid 4494 (rw), uid 1004 inumber 73751 on /mnt8: filesystem full May 31 09:04:20 x4 kernel: pid 4489 (mkdir), uid 1004 inumber 50112 on /mnt8: filesystem full May 31 09:04:21 x4 kernel: pid 4518 (rw), uid 1004 inumber 73764 on /mnt8: filesystem full May 31 09:04:27 x4 kernel: pid 4543 (rw), uid 1004 inumber 73859 on /mnt6: filesystem full May 31 09:04:27 x4 kernel: pid 4554 (mkdir), uid 1004 inumber 177 on /mnt6: filesystem full May 31 09:04:29 x4 kernel: pid 4504 (rw), uid 1004 inumber 49160 on /mnt6: filesystem full May 31 09:04:34 x4 kernel: pid 4502 (rw), uid 1004 inumber 73732 on /mnt8: filesystem full May 31 09:04:55 x4 kernel: pid 4492 (rw), uid 1004 inumber 73740 on /mnt8: filesystem full May 31 09:05:21 x4 kernel: pid 7042 (mkdir), uid 1004 inumber 1511 on /mnt7: filesystem full May 31 09:05:21 x4 kernel: pid 7079 (creat), uid 1004 inumber 74114 on /mnt7: filesystem full May 31 09:05:21 x4 kernel: pid 7020 (rw), uid 1004 inumber 74307 on /mnt7: filesystem full May 31 09:05:31 x4 kernel: pid 7004 (rw), uid 1004 inumber 16 on /mnt7: filesystem full May 31 09:05:31 x4 kernel: pid 7073 (creat), uid 1004 inumber 74242 on /mnt7: filesystem full May 31 09:05:33 x4 kernel: pid 7095 (mkdir), uid 1004 inumber 25044 on /mnt5: filesystem full May 31 09:05:33 x4 kernel: pid 7023 (rw), uid 1004 inumber 49162 on /mnt5: filesystem full May 31 09:05:36 x4 kernel: pid 7080 (rw), uid 1004 inumber 73747 on /mnt6: filesystem full May 31 09:05:36 x4 kernel: pid 7055 (mkdir), uid 1004 inumber 13 on /mnt6: filesystem full May 31 09:07:34 x4 kernel: pid 10269 (mkdir), uid 1004 inumber 4 on /mnt7: filesystem full May 31 09:07:34 x4 kernel: pid 10232 (rw), uid 1004 inumber 24594 on /mnt7: filesystem full May 31 09:07:34 x4 kernel: pid 10212 (creat), uid 1004 inumber 24582 on /mnt5: filesystem full May 31 09:07:45 x4 kernel: pid 10250 (rw), uid 1004 inumber 49167 on /mnt6: filesystem full May 31 09:07:45 x4 kernel: pid 10293 (mkdir), uid 1004 inumber 106 on /mnt6: filesystem full May 31 09:07:54 x4 kernel: pid 10315 (mkdir), uid 1004 inumber 37 on /mnt5: filesystem full May 31 09:07:54 x4 kernel: pid 10252 (rw), uid 1004 inumber 14 on /mnt5: filesystem full May 31 09:07:57 x4 kernel: pid 10256 (rw), uid 1004 inumber 16 on /mnt5: filesystem full May 31 09:08:00 x4 kernel: pid 10287 (mkdir), uid 1004 inumber 24578 on /mnt6: filesystem full May 31 09:08:00 x4 kernel: pid 10260 (rw), uid 1004 inumber 24588 on /mnt6: filesystem full May 31 09:08:16 x4 kernel: pid 10286 (mkdir), uid 1004 inumber 24961 on /mnt8: filesystem full May 31 09:08:16 x4 kernel: pid 10214 (creat), uid 1004 inumber 49157 on /mnt8: filesystem full May 31 09:08:16 x4 kernel: pid 10247 (rw), uid 1004 inumber 49155 on /mnt8: filesystem full May 31 09:08:30 x4 kernel: pid 10277 (mkdir), uid 1004 inumber 24596 on /mnt7: filesystem full May 31 09:08:30 x4 kernel: pid 10242 (rw), uid 1004 inumber 24707 on /mnt7: filesystem full umtx leak: 432/952. 20170531 09:09:17 all (1/1): crossmp11.sh May 31 09:09:53 x4 kernel: pid 15501 (rw), uid 1004 inumber 24585 on /mnt8: filesystem full May 31 09:09:53 x4 kernel: pid 15507 (mkdir), uid 1004 inumber 73776 on /mnt8: filesystem full May 31 09:09:54 x4 kernel: pid 15464 (creat), uid 1004 inumber 49152 on /mnt8: filesystem full May 31 09:09:54 x4 kernel: pid 15496 (rw), uid 1004 inumber 49168 on /mnt8: filesystem full May 31 09:09:55 x4 kernel: pid 15464 (creat), uid 1004 inumber 49152 on /mnt8: filesystem full May 31 09:09:55 x4 kernel: pid 15526 (mkdir), uid 1004 inumber 24767 on /mnt8: filesystem full May 31 09:11:00 x4 kernel: pid 16888 (dd), uid 2 inumber 1742863 on /var: filesystem full May 31 09:12:19 x4 kernel: pid 17878 (mkdir), uid 1004 inumber 24824 on /mnt8: filesystem full May 31 09:12:19 x4 kernel: pid 17848 (creat), uid 1004 inumber 10 on /mnt8: filesystem full May 31 09:12:20 x4 kernel: pid 17852 (rw), uid 1004 inumber 223 on /mnt8: filesystem full May 31 09:12:21 x4 kernel: pid 17869 (creat), uid 1004 inumber 13 on /mnt8: filesystem full May 31 09:12:22 x4 kernel: pid 17848 (creat), uid 1004 inumber 10 on /mnt8: filesystem full May 31 09:12:23 x4 kernel: pid 17866 (creat), uid 1004 inumber 18 on /mnt8: filesystem full May 31 09:12:29 x4 kernel: pid 17854 (rw), uid 1004 inumber 20 on /mnt8: filesystem full May 31 09:12:32 x4 kernel: pid 17856 (rw), uid 1004 inumber 122 on /mnt8: filesystem full May 31 09:12:32 x4 kernel: pid 17864 (creat), uid 1004 inumber 14 on /mnt8: filesystem full May 31 09:12:35 x4 kernel: pid 18169 (creat), uid 1004 inumber 26785 on /mnt7: filesystem full May 31 09:12:35 x4 kernel: pid 18143 (mkdir), uid 1004 inumber 49220 on /mnt7: filesystem full May 31 09:12:35 x4 kernel: pid 18150 (rw), uid 1004 inumber 24599 on /mnt7: filesystem full May 31 09:12:37 x4 kernel: pid 18152 (rw), uid 1004 inumber 24593 on /mnt7: filesystem full May 31 09:12:37 x4 kernel: pid 18142 (mkdir), uid 1004 inumber 49218 on /mnt7: filesystem full May 31 09:12:37 x4 kernel: pid 18171 (creat), uid 1004 inumber 75951 on /mnt7: filesystem full May 31 09:12:41 x4 kernel: pid 17855 (rw), uid 1004 inumber 22 on /mnt8: filesystem full May 31 09:12:43 x4 kernel: pid 18157 (rw), uid 1004 inumber 24591 on /mnt7: filesystem full May 31 09:12:44 x4 kernel: pid 17381 (mkdir), uid 1004 inumber 24599 on /mnt6: filesystem full May 31 09:12:44 x4 kernel: pid 17387 (rw), uid 1004 inumber 10 on /mnt6: filesystem full May 31 09:12:44 x4 kernel: pid 17400 (creat), uid 1004 inumber 73743 on /mnt6: filesystem full May 31 09:12:58 x4 kernel: pid 17849 (rw), uid 1004 inumber 24 on /mnt8: filesystem full May 31 09:13:04 x4 kernel: pid 17792 (rw), uid 1004 inumber 73733 on /mnt5: filesystem full May 31 09:13:06 x4 kernel: pid 17791 (rw), uid 1004 inumber 8 on /mnt5: filesystem full May 31 09:13:14 x4 kernel: pid 17401 (creat), uid 1004 inumber 73733 on /mnt6: filesystem full May 31 09:13:14 x4 kernel: pid 17857 (rw), uid 1004 inumber 24578 on /mnt8: filesystem full May 31 09:13:15 x4 kernel: pid 17398 (creat), uid 1004 inumber 11 on /mnt6: filesystem full May 31 09:13:15 x4 kernel: pid 17380 (mkdir), uid 1004 inumber 1161 on /mnt6: filesystem full May 31 09:14:09 x4 kernel: pid 19485 (mkdir), uid 1004 inumber 1142 on /mnt7: filesystem full May 31 09:14:09 x4 kernel: pid 19440 (creat), uid 1004 inumber 15 on /mnt7: filesystem full May 31 09:14:09 x4 kernel: pid 19436 (rw), uid 1004 inumber 16 on /mnt7: filesystem full May 31 09:14:14 x4 kernel: pid 19458 (mkdir), uid 1004 inumber 76349 on /mnt7: filesystem full May 31 09:14:17 x4 kernel: pid 19426 (rw), uid 1004 inumber 14 on /mnt7: filesystem full May 31 09:14:17 x4 kernel: pid 19437 (creat), uid 1004 inumber 73737 on /mnt7: filesystem full May 31 09:14:17 x4 kernel: pid 19477 (mkdir), uid 1004 inumber 74608 on /mnt7: filesystem full May 31 09:14:17 x4 kernel: pid 19420 (rw), uid 1004 inumber 31 on /mnt7: filesystem full May 31 09:14:17 x4 kernel: pid 19451 (creat), uid 1004 inumber 22 on /mnt7: filesystem full May 31 09:14:17 x4 kernel: pid 19443 (rw), uid 1004 inumber 26 on /mnt7: filesystem full May 31 09:14:17 x4 kernel: pid 19449 (creat), uid 1004 inumber 25 on /mnt7: filesystem full May 31 09:14:18 x4 kernel: pid 19452 (creat), uid 1004 inumber 24 on /mnt7: filesystem full May 31 09:14:19 x4 kernel: pid 19446 (mkdir), uid 1004 inumber 73809 on /mnt6: filesystem full May 31 09:14:19 x4 kernel: pid 19471 (rw), uid 1004 inumber 73746 on /mnt6: filesystem full May 31 09:14:19 x4 kernel: pid 19453 (creat), uid 1004 inumber 9 on /mnt7: filesystem full May 31 09:14:20 x4 kernel: pid 19464 (rw), uid 1004 inumber 73730 on /mnt6: filesystem full May 31 09:14:26 x4 kernel: pid 19676 (rw), uid 1004 inumber 73753 on /mnt5: filesystem full May 31 09:14:26 x4 kernel: pid 19687 (creat), uid 1004 inumber 73748 on /mnt5: filesystem full May 31 09:14:27 x4 kernel: pid 19678 (creat), uid 1004 inumber 73732 on /mnt5: filesystem full May 31 09:14:27 x4 kernel: pid 19667 (rw), uid 1004 inumber 73730 on /mnt5: filesystem full May 31 09:14:38 x4 kernel: pid 19872 (rw), uid 1004 inumber 73741 on /mnt8: filesystem full May 31 09:14:38 x4 kernel: pid 19863 (creat), uid 1004 inumber 73738 on /mnt8: filesystem full May 31 09:14:57 x4 kernel: pid 19670 (rw), uid 1004 inumber 73756 on /mnt5: filesystem full May 31 09:14:57 x4 kernel: pid 19678 (creat), uid 1004 inumber 73732 on /mnt5: filesystem full May 31 09:15:55 x4 kernel: pid 19873 (rw), uid 1004 inumber 73733 on /mnt8: filesystem full umtx leak: 8/960. 20170531 09:16:01 all (1/1): crossmp11.sh May 31 09:16:47 x4 kernel: pid 23059 (rw), uid 1004 inumber 73735 on /mnt6: filesystem full May 31 09:16:47 x4 kernel: pid 22966 (mkdir), uid 1004 inumber 49182 on /mnt6: filesystem full May 31 09:16:48 x4 kernel: pid 23050 (creat), uid 1004 inumber 49169 on /mnt6: filesystem full May 31 09:16:49 x4 kernel: pid 23064 (rw), uid 1004 inumber 49174 on /mnt6: filesystem full May 31 09:16:51 x4 kernel: pid 22967 (mkdir), uid 1004 inumber 73816 on /mnt6: filesystem full May 31 09:16:51 x4 kernel: pid 23048 (creat), uid 1004 inumber 49153 on /mnt6: filesystem full May 31 09:16:52 x4 kernel: pid 23048 (creat), uid 1004 inumber 49153 on /mnt6: filesystem full May 31 09:16:58 x4 kernel: pid 23005 (rw), uid 1004 inumber 24586 on /mnt7: filesystem full May 31 09:16:59 x4 kernel: pid 23070 (creat), uid 1004 inumber 24579 on /mnt7: filesystem full May 31 09:16:59 x4 kernel: pid 23009 (rw), uid 1004 inumber 24585 on /mnt7: filesystem full May 31 09:17:31 x4 kernel: pid 23014 (creat), uid 1004 inumber 24587 on /mnt8: filesystem full May 31 09:17:31 x4 kernel: pid 23008 (rw), uid 1004 inumber 24595 on /mnt8: filesystem full May 31 09:17:33 x4 kernel: pid 22969 (creat), uid 1004 inumber 24591 on /mnt8: filesystem full May 31 09:17:33 x4 kernel: pid 23012 (mkdir), uid 1004 inumber 73775 on /mnt8: filesystem full May 31 09:17:33 x4 kernel: pid 23006 (rw), uid 1004 inumber 24613 on /mnt8: filesystem full May 31 09:17:54 x4 kernel: pid 23016 (creat), uid 1004 inumber 24582 on /mnt8: filesystem full May 31 09:17:56 x4 kernel: pid 23022 (creat), uid 1004 inumber 24586 on /mnt8: filesystem full May 31 09:17:58 x4 kernel: pid 22994 (rw), uid 1004 inumber 24619 on /mnt8: filesystem full May 31 09:17:58 x4 kernel: pid 23004 (creat), uid 1004 inumber 24596 on /mnt8: filesystem full May 31 09:18:09 x4 kernel: pid 23027 (rw), uid 1004 inumber 73740 on /mnt7: filesystem full May 31 09:18:38 x4 kernel: pid 24290 (rw), uid 1004 inumber 12 on /mnt5: filesystem full May 31 09:18:38 x4 kernel: pid 24291 (creat), uid 1004 inumber 16 on /mnt5: filesystem full May 31 09:18:39 x4 kernel: pid 24297 (rw), uid 1004 inumber 73734 on /mnt5: filesystem full May 31 09:18:39 x4 kernel: pid 24298 (creat), uid 1004 inumber 19 on /mnt5: filesystem full May 31 09:19:17 x4 kernel: pid 24299 (rw), uid 1004 inumber 73736 on /mnt5: filesystem full May 31 09:19:31 x4 kernel: pid 25206 (rw), uid 1004 inumber 24582 on /mnt7: filesystem full May 31 09:19:33 x4 kernel: pid 25220 (rw), uid 1004 inumber 49166 on /mnt8: filesystem full May 31 09:19:59 x4 kernel: pid 24280 (rw), uid 1004 inumber 73730 on /mnt5: filesystem full May 31 09:20:01 x4 kernel: pid 24288 (creat), uid 1004 inumber 15 on /mnt5: filesystem full May 31 09:22:07 x4 kernel: pid 26180 (dd), uid 2 inumber 1742914 on /var: filesystem full May 31 09:22:14 x4 kernel: pid 26162 (rw), uid 1004 inumber 73759 on /mnt8: filesystem full May 31 09:24:00 x4 kernel: pid 26773 (rw), uid 1004 inumber 24584 on /mnt7: filesystem full May 31 09:33:00 x4 kernel: pid 57213 (dd), uid 2 inumber 1742924 on /var: filesystem full Stopping inetd. Waiting for PIDS: 809. Stopping moused. Waiting for PIDS: 779. Stopping nfscbd. Waiting for PIDS: 774 775. Shutting down local packages:. Stopping cron. Waiting for PIDS: 686. Stopping sshd. Waiting for PIDS: 676. Stopping smartd. Waiting for PIDS: 639. Stopping powerd. Waiting for PIDS: 609. Stopping ntpd. Waiting for PIDS: 606, 606. Stopping lockd. Waiting for PIDS: 549. Stopping statd. Waiting for PIDS: 546. Stopping nfsd. Waiting for PIDS: 542 545. Stopping mountd. Waiting for PIDS: 536. Stopping nfsuserd. Waiting for PIDS: 525 526 527 528 529. Stopping watchdogd. Waiting for PIDS: 483. Stopping rpcbind. Waiting for PIDS: 395. Stopping devd. Waiting for PIDS: 314. Writing entropy file:. Writing early boot entropy file:. . Terminated May 31 09:44:17 x4 syslogd: exiting on signal 15 May 31 09:44:37 init: some processes would not die; ps axl advised Enter full pathname of shell or RETURN for /bin/sh: # ps -l UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND 0 95809 1 0 20 0 6628 2876 wait Ss u0 0:00.01 -sh (sh) 0 95810 95809 0 20 0 6564 2316 - R+ u0 0:00.00 ps -l 0 26898 1 0 20 0 6012 1812 mntref D 1- 0:03.36 umount /mnt6 0 28279 1 0 20 0 6012 1816 mntref D 1- 0:09.58 umount -f /mnt8 # KDB: enter: Break to debugger [ thread pid 11 tid 100006 ] Stopped at kdb_alt_break_internal+0x101: movl $0,kdb_why db> run pho db:0:pho> bt Tracing pid 11 tid 100006 td 0xc8e66a20 kdb_alt_break_internal(0,e4b42ac4,c0a5b2a7,2,c91a7888,...) at kdb_alt_break_internal+0x101/frame 0xe4b42a98 kdb_alt_break(2,c91a7888,c15e472b,0,40000,...) at kdb_alt_break+0x16/frame 0xe4b42aa4 uart_intr_rxready(c91a7800,6e4fbab4,e4b20c80,c8e66a20,e4b42ae8,...) at uart_intr_rxready+0x87/frame 0xe4b42ac4 uart_intr(c91a7800,b70,1,6e4fbab4,0,...) at uart_intr+0x120/frame 0xe4b42b04 intr_event_handle(c8eb3e00,e4b42b98,c1b21880,40000000,1,...) at intr_event_handle+0x75/frame 0xe4b42b30 intr_execute_handlers(c8eae908,e4b42b98,3) at intr_execute_handlers+0xb0/frame 0xe4b42b68 lapic_handle_intr(32,e4b42b98) at lapic_handle_intr+0x5b/frame 0xe4b42b88 Xapic_isr1() at Xapic_isr1+0x55/frame 0xe4b42b88 --- interrupt, eip = 0xc123fbe5, esp = 0xe4b42bd8, ebp = 0xe4b42bd8 --- acpi_cpu_c1(e4b42c2c,c122eef9,e4b20c90,0,c1833803,...) at acpi_cpu_c1+0x5/frame 0xe4b42bd8 acpi_cpu_idle(ffd0882,0,ffd0882,e4b42c48,c123ffb6,...) at acpi_cpu_idle+0x325/frame 0xe4b42c18 cpu_idle_acpi(ffd0882,0,c1b21880,2,c8e66a20,...) at cpu_idle_acpi+0x3f/frame 0xe4b42c2c cpu_idle(0,e4b42c80,c1628197,407,0,...) at cpu_idle+0xa6/frame 0xe4b42c48 sched_idletd(0,e4b42ce8,c161c740,406,0,...) at sched_idletd+0x16a/frame 0xe4b42cac fork_exit(c0c88380,0,e4b42ce8) at fork_exit+0x7e/frame 0xe4b42cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4b42cd4 --- trap 0, eip = 0, esp = 0xe4b42d20, ebp = 0 --- db:0:bt> show allpcpu Current CPU: 3 cpuid = 0 dynamic pcpu = 0x6f0780 curthread = 0xc8e676c0: pid 11 tid 100003 "idle: cpu0" curpcb = 0xe4b36d40 fpcurthread = none idlethread = 0xc8e676c0: tid 100003 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: cpuid = 1 dynamic pcpu = 0x231e1780 curthread = 0xc8e67360: pid 11 tid 100004 "idle: cpu1" curpcb = 0xe4b3ad40 fpcurthread = none idlethread = 0xc8e67360: tid 100004 "idle: cpu1" APIC ID = 1 currentldt = 0x50 spin locks held: cpuid = 2 dynamic pcpu = 0x231e5780 curthread = 0xc8e67000: pid 11 tid 100005 "idle: cpu2" curpcb = 0xe4b3ed40 fpcurthread = none idlethread = 0xc8e67000: tid 100005 "idle: cpu2" APIC ID = 2 currentldt = 0x50 spin locks held: cpuid = 3 dynamic pcpu = 0x231e9780 curthread = 0xc8e66a20: pid 11 tid 100006 "idle: cpu3" curpcb = 0xe4b42d40 fpcurthread = none idlethread = 0xc8e66a20: tid 100006 "idle: cpu3" APIC ID = 3 currentldt = 0x50 spin locks held: db:0:allpcpu> show alllocks db:0:alllocks> show lockedvnods Locked vnodes db:0:lockedvnods> show mount 0xc9785000 /dev/ada0s1a on / (ufs) 0xc9717000 devfs on /dev (devfs) 0xc9ad3000 /dev/ada0s1f on /home (ufs) 0xc9b71000 /dev/ada0s1d on /usr (ufs) 0xc9bbc000 /dev/ada0s1e on /var (ufs) 0xc9c0d000 procfs on /proc (procfs) 0xc9c14000 fdescfs on /dev/fd (fdescfs) 0xc9cc3000 linprocfs on /compat/linux/proc (linprocfs) 0xc9b1d000 /dev/label/tmp on /tmp (ufs) More info: show mount db:0:mount> show page vm_cnt.v_free_count: 48957 vm_cnt.v_inactive_count: 208 vm_cnt.v_active_count: 745540 vm_cnt.v_laundry_count: 40149 vm_cnt.v_wire_count: 27930 vm_cnt.v_free_reserved: 1173 vm_cnt.v_free_min: 5485 vm_cnt.v_free_target: 18421 vm_cnt.v_inactive_target: 27631 db:0:page> ps pid ppid pgrp uid state wmesg wchan cmd 95809 1 95809 0 Ss+ ttyin 0xc91a7674 sh 28279 1 2271 0 D+ mntref 0xcc7a9000 umount 26898 1 2271 0 D+ mntref 0xce983000 umount 21570 0 0 0 DL mdwait 0xc9d0b800 [md8] 21559 0 0 0 DL mdwait 0xc9d0b000 [md7] 21548 0 0 0 DL mdwait 0xccb7e800 [md6] 21537 0 0 0 DL mdwait 0xce473800 [md5] 22 0 0 0 DL syncer 0xc1b8da5c [syncer] 21 0 0 0 DL vlruwt 0xc8e57000 [vnlru] 20 0 0 0 DL (threaded) [bufdaemon] 100094 D psleep 0xc1b8c784 [bufdaemon] 100106 D sdflush 0xc9ab7c84 [/home worker] 100108 D sdflush 0xc906d284 [/usr worker] 100109 D sdflush 0xc906d084 [/var worker] 100177 D sdflush 0xc90c9284 [/tmp worker] 19 0 0 0 DL - 0xc1b8d3cc [bufspacedaemon] 18 0 0 0 DL psleep 0xc1b9392c [vmdaemon] 17 0 0 0 DL (threaded) [pagedaemon] 100089 D psleep 0xc1bf7185 [pagedaemon] 100092 D launds 0xc1b938f4 [laundry: dom0] 100093 D umarcl 0xc1b93520 [uma] 16 0 0 0 DL - 0xc19eb3e8 [rand_harvestq] 15 0 0 0 DL waiting 0xc1bf2a74 [sctp_iterator] 9 0 0 0 DL - 0xc1b8c15c [soaiod4] 8 0 0 0 DL - 0xc1b8c15c [soaiod3] 7 0 0 0 DL - 0xc1b8c15c [soaiod2] 6 0 0 0 DL - 0xc1b8c15c [soaiod1] 5 0 0 0 DL - 0xc9174a3c [fdc0] 14 0 0 0 DL (threaded) [usb] 100042 D - 0xc90e5acc [usbus0] 100043 D - 0xc90e5afc [usbus0] 100044 D - 0xc90e5b2c [usbus0] 100045 D - 0xc90e5b5c [usbus0] 100046 D - 0xc90e5b8c [usbus0] 100048 D - 0xc9157acc [usbus1] 100049 D - 0xc9157afc [usbus1] 100050 D - 0xc9157b2c [usbus1] 100051 D - 0xc9157b5c [usbus1] 100052 D - 0xc9157b8c [usbus1] 100054 D - 0xc915dacc [usbus2] 100055 D - 0xc915dafc [usbus2] 100056 D - 0xc915db2c [usbus2] 100057 D - 0xc915db5c [usbus2] 100058 D - 0xc915db8c [usbus2] 100059 D - 0xc9164acc [usbus3] 100060 D - 0xc9164afc [usbus3] 100061 D - 0xc9164b2c [usbus3] 100062 D - 0xc9164b5c [usbus3] 100063 D - 0xc9164b8c [usbus3] 100064 D - 0xc9169acc [usbus4] 100065 D - 0xc9169afc [usbus4] 100066 D - 0xc9169b2c [usbus4] 100067 D - 0xc9169b5c [usbus4] 100068 D - 0xc9169b8c [usbus4] 100070 D - 0xc90d0ca4 [usbus5] 100071 D - 0xc90d0cd4 [usbus5] 100072 D - 0xc90d0d04 [usbus5] 100073 D - 0xc90d0d34 [usbus5] 100074 D - 0xc90d0d64 [usbus5] 100115 D - 0xc972ec08 [ucom] 4 0 0 0 DL (threaded) [cam] 100037 D - 0xc1937880 [doneq0] 100088 D - 0xc193772c [scanner] 3 0 0 0 DL crypto_ 0xc1b924bc [crypto returns] 2 0 0 0 DL crypto_ 0xc1b92404 [crypto] 13 0 0 0 DL (threaded) [geom] 100028 D - 0xc1bedb44 [g_event] 100029 D - 0xc1bedb48 [g_up] 100030 D - 0xc1bedb4c [g_down] 12 0 0 0 WL (threaded) [intr] 100007 I [swi3: vm] 100008 I [swi4: clock (0)] 100009 I [swi4: clock (1)] 100010 I [swi4: clock (2)] 100011 I [swi4: clock (3)] 100012 I [swi1: netisr 0] 100023 I [swi6: task queue] 100024 I [swi6: Giant taskq] 100027 I [swi5: fast taskq] 100038 I [irq256: hdac0] 100039 I [irq257: re0] 100040 I [irq22: ahci0] 100041 I [irq16: hdac1 ohci0] 100047 I [irq17: ohci1 ohci3] 100053 I [irq18: ohci2 ohci4] 100069 I [irq19: ehci0] 100075 I [irq14: ata0] 100076 I [swi0: uart] 100078 I [irq1: atkbd0] 100079 I [irq12: psm0] 11 0 0 0 RL (threaded) [idle] 100003 Run CPU 0 [idle: cpu0] 100004 Run CPU 1 [idle: cpu1] 100005 Run CPU 2 [idle: cpu2] 100006 Run CPU 3 [idle: cpu3] 1 0 1 0 SLs wait 0xc8e59a2c [init] 10 0 0 0 DL audit_w 0xc1bf5990 [audit] 0 0 0 0 DLs (threaded) [kernel] 100000 D swapin 0xc1bedb64 [swapper] 100013 D - 0xc8e34f00 [if_config_tqg_0] 100014 D - 0xc8e34e80 [softirq_0] 100015 D - 0xc8e34e00 [softirq_1] 100016 D - 0xc8e34d80 [softirq_2] 100017 D - 0xc8e34d00 [softirq_3] 100018 D - 0xc8e34c80 [if_io_tqg_0] 100019 D - 0xc8e34c00 [if_io_tqg_1] 100020 D - 0xc8e34b80 [if_io_tqg_2] 100021 D - 0xc8e34b00 [if_io_tqg_3] 100022 D - 0xc8e34a80 [kqueue_ctx taskq] 100025 D - 0xc8e34900 [thread taskq] 100026 D - 0xc8e34880 [aiod_kick taskq] 100031 D - 0xc8e34680 [firmware taskq] 100034 D - 0xc8e33d80 [acpi_task_0] 100035 D - 0xc8e33d80 [acpi_task_1] 100036 D - 0xc8e33d80 [acpi_task_2] 100080 D - 0xc9187980 [mca taskq] 100087 D - 0xc8e33d00 [CAM taskq] db:0:ps> allt Tracing command sh pid 95809 tid 100363 td 0xcc9c36c0 sched_switch(cc9c36c0,0,104,18a,cc9c36c0,...) at sched_switch+0x512/frame 0xc85f9900 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xc85f9948 sleepq_switch(cc9c36c0,0,c162c48d,1c5,10,...) at sleepq_switch+0x11c/frame 0xc85f9980 sleepq_catch_signals(c85f99e4,c0c36350,c91a7614,0,0,...) at sleepq_catch_signals+0x36e/frame 0xc85f99c0 sleepq_wait_sig(c91a7674,0,c85f9a04,101,0,...) at sleepq_wait_sig+0x14/frame 0xc85f99e4 _cv_wait_sig(c91a7674,c91a7604,c1632385,5a8,c91a7600,...) at _cv_wait_sig+0x21f/frame 0xc85f9a28 tty_wait(c91a7600,c91a7674,c85f9b90,1,0,...) at tty_wait+0x4b/frame 0xc85f9a4c ttydisc_read(c91a7600,c85f9b90,0,d0,c18f91c4,...) at ttydisc_read+0x235/frame 0xc85f9a94 ttydev_read(c8f19e00,c85f9b90,0,ca1c9028,c85f9af4,...) at ttydev_read+0x55/frame 0xc85f9ab4 devfs_read_f(c97d40e0,c85f9b90,c8e2fd00,0,cc9c36c0,...) at devfs_read_f+0xc5/frame 0xc85f9aec dofileread(c97d40e0,c85f9b90,ffffffff,ffffffff,0,...) at dofileread+0xa7/frame 0xc85f9b20 kern_readv(cc9c36c0,0,c85f9b90,c85f9bc0,bfbfec6a,...) at kern_readv+0x93/frame 0xc85f9b70 sys_read(cc9c36c0,c85f9c00,c161c17f,4,0,...) at sys_read+0x54/frame 0xc85f9bc0 syscall(c85f9ce8) at syscall+0x75c/frame 0xc85f9cdc Xint0x80_syscall() at Xint0x80_syscall+0x2e/frame 0xc85f9cdc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2820a7a7, esp = 0xbfbfec24, ebp = 0xbfbfec38 --- Tracing command umount pid 28279 tid 100296 td 0xcc90da20 sched_switch(cc90da20,0,104,18a,c84df744,...) at sched_switch+0x512/frame 0xc84df720 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xc84df764 sleepq_switch(cc90da20,0,c162c48d,2ae,0,...) at sleepq_switch+0x11c/frame 0xc84df79c sleepq_wait(cc7a9000,60,c16386f4,0,0,...) at sleepq_wait+0x48/frame 0xc84df7c0 _sleep(cc7a9000,cc7a9000,60,c16386f4,0,...) at _sleep+0x38f/frame 0xc84df814 vfs_mount_destroy(cc7a9000,10,0,5e5,0,...) at vfs_mount_destroy+0xdf/frame 0xc84df878 dounmount(cc7a9000,8080000,cc90da20,4e4,43e,...) at dounmount+0x14a5/frame 0xc84dfa70 sys_unmount(cc90da20,c84dfc00,c161c17f,4,0,...) at sys_unmount+0x667/frame 0xc84dfbc0 syscall(c84dfce8) at syscall+0x75c/frame 0xc84dfcdc Xint0x80_syscall() at Xint0x80_syscall+0x2e/frame 0xc84dfcdc --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280a9bef, esp = 0xbfbfd424, ebp = 0xbfbfd900 --- Tracing command umount pid 26898 tid 100263 td 0xcc8c1000 sched_switch(cc8c1000,0,104,18a,f37ed744,...) at sched_switch+0x512/frame 0xf37ed720 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xf37ed764 sleepq_switch(cc8c1000,0,c162c48d,2ae,0,...) at sleepq_switch+0x11c/frame 0xf37ed79c sleepq_wait(ce983000,60,c16386f4,0,0,...) at sleepq_wait+0x48/frame 0xf37ed7c0 _sleep(ce983000,ce983000,60,c16386f4,0,...) at _sleep+0x38f/frame 0xf37ed814 vfs_mount_destroy(ce983000,10,0,5e5,0,...) at vfs_mount_destroy+0xdf/frame 0xf37ed878 dounmount(ce983000,8000000,cc8c1000,4e4,43d,...) at dounmount+0x14a5/frame 0xf37eda70 sys_unmount(cc8c1000,f37edc00,c161c17f,4,c1b20b1c,...) at sys_unmount+0x667/frame 0xf37edbc0 syscall(f37edce8) at syscall+0x75c/frame 0xf37edcdc Xint0x80_syscall() at Xint0x80_syscall+0x2e/frame 0xf37edcdc --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280a9bef, esp = 0xbfbfd424, ebp = 0xbfbfd900 --- Tracing command md8 pid 21570 tid 100283 td 0xcc8d26c0 sched_switch(cc8d26c0,0,104,18a,c84a0b9c,...) at sched_switch+0x512/frame 0xc84a0b78 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xc84a0bbc sleepq_switch(cc8d26c0,0,c162c48d,2ae,0,...) at sleepq_switch+0x11c/frame 0xc84a0bf4 sleepq_wait(c9d0b800,5c,c134649a,0,0,...) at sleepq_wait+0x48/frame 0xc84a0c18 _sleep(c9d0b800,c9d0b820,25c,c134649a,0,...) at _sleep+0x38f/frame 0xc84a0c6c md_kthread(c9d0b800,c84a0ce8,c161c740,406,0,...) at md_kthread+0x1da/frame 0xc84a0cac fork_exit(c07e2da0,c9d0b800,c84a0ce8) at fork_exit+0x7e/frame 0xc84a0cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xc84a0cd4 --- trap 0, eip = 0, esp = 0xc84a0d20, ebp = 0 --- Tracing command md7 pid 21559 tid 100334 td 0xcc9836c0 sched_switch(cc9836c0,0,104,18a,c8585b9c,...) at sched_switch+0x512/frame 0xc8585b78 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xc8585bbc sleepq_switch(cc9836c0,0,c162c48d,2ae,0,...) at sleepq_switch+0x11c/frame 0xc8585bf4 sleepq_wait(c9d0b000,5c,c134649a,0,0,...) at sleepq_wait+0x48/frame 0xc8585c18 _sleep(c9d0b000,c9d0b020,25c,c134649a,0,...) at _sleep+0x38f/frame 0xc8585c6c md_kthread(c9d0b000,c8585ce8,c161c740,406,0,...) at md_kthread+0x1da/frame 0xc8585cac fork_exit(c07e2da0,c9d0b000,c8585ce8) at fork_exit+0x7e/frame 0xc8585cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xc8585cd4 --- trap 0, eip = 0, esp = 0xc8585d20, ebp = 0 --- Tracing command md6 pid 21548 tid 100101 td 0xc92b76c0 sched_switch(c92b76c0,0,104,18a,edfb6b9c,...) at sched_switch+0x512/frame 0xedfb6b78 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedfb6bbc sleepq_switch(c92b76c0,0,c162c48d,2ae,0,...) at sleepq_switch+0x11c/frame 0xedfb6bf4 sleepq_wait(ccb7e800,5c,c134649a,0,0,...) at sleepq_wait+0x48/frame 0xedfb6c18 _sleep(ccb7e800,ccb7e820,25c,c134649a,0,...) at _sleep+0x38f/frame 0xedfb6c6c md_kthread(ccb7e800,edfb6ce8,c161c740,406,0,...) at md_kthread+0x1da/frame 0xedfb6cac fork_exit(c07e2da0,ccb7e800,edfb6ce8) at fork_exit+0x7e/frame 0xedfb6cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedfb6cd4 --- trap 0, eip = 0, esp = 0xedfb6d20, ebp = 0 --- Tracing command md5 pid 21537 tid 100268 td 0xcc4ce360 sched_switch(cc4ce360,0,104,18a,c8460b9c,...) at sched_switch+0x512/frame 0xc8460b78 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xc8460bbc sleepq_switch(cc4ce360,0,c162c48d,2ae,0,...) at sleepq_switch+0x11c/frame 0xc8460bf4 sleepq_wait(ce473800,5c,c134649a,0,0,...) at sleepq_wait+0x48/frame 0xc8460c18 _sleep(ce473800,ce473820,25c,c134649a,0,...) at _sleep+0x38f/frame 0xc8460c6c md_kthread(ce473800,c8460ce8,c161c740,406,0,...) at md_kthread+0x1da/frame 0xc8460cac fork_exit(c07e2da0,ce473800,c8460ce8) at fork_exit+0x7e/frame 0xc8460cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xc8460cd4 --- trap 0, eip = 0, esp = 0xc8460d20, ebp = 0 --- Tracing command syncer pid 22 tid 100096 td 0xc96ad360 sched_switch(c96ad360,0,104,18a,c96ad360,...) at sched_switch+0x512/frame 0xedf9cb40 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedf9cb88 sleepq_switch(c96ad360,0,c162c48d,2d1,c96ad360,...) at sleepq_switch+0x11c/frame 0xedf9cbc0 sleepq_timedwait(c1b8da5c,0,edf9cc24,0,0,...) at sleepq_timedwait+0x53/frame 0xedf9cc00 _cv_timedwait_sbt(c1b8da5c,c1b8da3c,fffffed8,0,0,...) at _cv_timedwait_sbt+0x24b/frame 0xedf9cc48 sched_sync(0,edf9cce8,c161c740,406,0,...) at sched_sync+0x1fd/frame 0xedf9ccac fork_exit(c0d2ed90,0,edf9cce8) at fork_exit+0x7e/frame 0xedf9ccd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedf9ccd4 --- trap 0, eip = 0, esp = 0xedf9cd20, ebp = 0 --- Tracing command vnlru pid 21 tid 100095 td 0xc96ad6c0 sched_switch(c96ad6c0,0,104,18a,c96ad6c0,...) at sched_switch+0x512/frame 0xedf98b20 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedf98b68 sleepq_switch(c96ad6c0,0,c162c48d,2d1,b71,...) at sleepq_switch+0x11c/frame 0xedf98ba0 sleepq_timedwait(c8e57000,60,0,0,0,...) at sleepq_timedwait+0x53/frame 0xedf98bdc _sleep(c8e57000,c1b8da08,260,c163a093,fffffed8,...) at _sleep+0x363/frame 0xedf98c30 vnlru_proc(0,edf98ce8,c161c740,406,0,...) at vnlru_proc+0xbf/frame 0xedf98cac fork_exit(c0d2e400,0,edf98ce8) at fork_exit+0x7e/frame 0xedf98cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedf98cd4 --- trap 0, eip = 0, esp = 0xedf98d20, ebp = 0 --- Tracing command bufdaemon pid 20 tid 100094 td 0xc96ada20 sched_switch(c96ada20,0,104,18a,c96ada20,...) at sched_switch+0x512/frame 0xedf94b68 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedf94bb0 sleepq_switch(c96ada20,0,c162c48d,2d1,b71,...) at sleepq_switch+0x11c/frame 0xedf94be8 sleepq_timedwait(c1b8c784,54,0,0,0,...) at sleepq_timedwait+0x53/frame 0xedf94c24 _sleep(c1b8c784,c1b8c700,54,c1636baa,fffffed8,...) at _sleep+0x363/frame 0xedf94c78 buf_daemon(0,edf94ce8,c161c740,406,0,...) at buf_daemon+0x8c/frame 0xedf94cac fork_exit(c0d09020,0,edf94ce8) at fork_exit+0x7e/frame 0xedf94cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedf94cd4 --- trap 0, eip = 0, esp = 0xedf94d20, ebp = 0 --- Tracing command bufdaemon pid 20 tid 100106 td 0xc97bf360 sched_switch(c97bf360,0,104,18a,c97bf360,...) at sched_switch+0x512/frame 0xedfd6b00 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedfd6b48 sleepq_switch(c97bf360,0,c162c48d,2d1,b70,...) at sleepq_switch+0x11c/frame 0xedfd6b80 sleepq_timedwait(c9ab7c84,54,0,0,0,...) at sleepq_timedwait+0x53/frame 0xedfd6bbc _sleep(c9ab7c84,c9ab7c00,54,c16713a5,7fffff6c,...) at _sleep+0x363/frame 0xedfd6c10 softdep_flush(c9ad3000,edfd6ce8,c161c740,406,0,...) at softdep_flush+0x2a9/frame 0xedfd6cac fork_exit(c0f42520,c9ad3000,edfd6ce8) at fork_exit+0x7e/frame 0xedfd6cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedfd6cd4 --- trap 0, eip = 0, esp = 0xedfd6d20, ebp = 0 --- Tracing command bufdaemon pid 20 tid 100108 td 0xc96ac360 sched_switch(c96ac360,0,104,18a,c96ac360,...) at sched_switch+0x512/frame 0xedfdeb00 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedfdeb48 sleepq_switch(c96ac360,0,c162c48d,2d1,b70,...) at sleepq_switch+0x11c/frame 0xedfdeb80 sleepq_timedwait(c906d284,54,0,0,0,...) at sleepq_timedwait+0x53/frame 0xedfdebbc _sleep(c906d284,c906d200,54,c16713a5,7fffff6c,...) at _sleep+0x363/frame 0xedfdec10 softdep_flush(c9b71000,edfdece8,c161c740,406,0,...) at softdep_flush+0x2a9/frame 0xedfdecac fork_exit(c0f42520,c9b71000,edfdece8) at fork_exit+0x7e/frame 0xedfdecd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedfdecd4 --- trap 0, eip = 0, esp = 0xedfded20, ebp = 0 --- Tracing command bufdaemon pid 20 tid 100109 td 0xc96ac000 sched_switch(c96ac000,0,104,18a,c96ac000,...) at sched_switch+0x512/frame 0xedfe2b00 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedfe2b48 sleepq_switch(c96ac000,0,c162c48d,2d1,b70,...) at sleepq_switch+0x11c/frame 0xedfe2b80 sleepq_timedwait(c906d084,54,0,0,0,...) at sleepq_timedwait+0x53/frame 0xedfe2bbc _sleep(c906d084,c906d000,54,c16713a5,7fffff6c,...) at _sleep+0x363/frame 0xedfe2c10 softdep_flush(c9bbc000,edfe2ce8,c161c740,406,0,...) at softdep_flush+0x2a9/frame 0xedfe2cac fork_exit(c0f42520,c9bbc000,edfe2ce8) at fork_exit+0x7e/frame 0xedfe2cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedfe2cd4 --- trap 0, eip = 0, esp = 0xedfe2d20, ebp = 0 --- Tracing command bufdaemon pid 20 tid 100177 td 0xc91b86c0 sched_switch(c91b86c0,0,104,18a,c91b86c0,...) at sched_switch+0x512/frame 0xf3631b00 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xf3631b48 sleepq_switch(c91b86c0,0,c162c48d,2d1,b70,...) at sleepq_switch+0x11c/frame 0xf3631b80 sleepq_timedwait(c90c9284,54,0,0,0,...) at sleepq_timedwait+0x53/frame 0xf3631bbc _sleep(c90c9284,c90c9200,54,c16713a5,7fffff6c,...) at _sleep+0x363/frame 0xf3631c10 softdep_flush(c9b1d000,f3631ce8,c161c740,406,0,...) at softdep_flush+0x2a9/frame 0xf3631cac fork_exit(c0f42520,c9b1d000,f3631ce8) at fork_exit+0x7e/frame 0xf3631cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3631cd4 --- trap 0, eip = 0, esp = 0xf3631d20, ebp = 0 --- Tracing command bufspacedaemon pid 19 tid 100091 td 0xc96ae000 sched_switch(c96ae000,0,104,18a,c96ae000,...) at sched_switch+0x512/frame 0xedf88b68 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedf88bb0 sleepq_switch(c96ae000,0,c162c48d,2d1,b70,...) at sleepq_switch+0x11c/frame 0xedf88be8 sleepq_timedwait(c1b8d3cc,5c,0,0,0,...) at sleepq_timedwait+0x53/frame 0xedf88c24 _sleep(c1b8d3cc,c1b8d180,25c,c160c2db,fffffed8,...) at _sleep+0x363/frame 0xedf88c78 bufspace_daemon(0,edf88ce8,c161c740,406,0,...) at bufspace_daemon+0x1ff/frame 0xedf88cac fork_exit(c0d08930,0,edf88ce8) at fork_exit+0x7e/frame 0xedf88cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedf88cd4 --- trap 0, eip = 0, esp = 0xedf88d20, ebp = 0 --- Tracing command vmdaemon pid 18 tid 100090 td 0xc8f246c0 sched_switch(c8f246c0,0,104,18a,c8f246c0,...) at sched_switch+0x512/frame 0xedf84b38 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedf84b80 sleepq_switch(c8f246c0,0,c162c48d,2ae,0,...) at sleepq_switch+0x11c/frame 0xedf84bb8 sleepq_wait(c1b9392c,74,c1636baa,0,0,...) at sleepq_wait+0x48/frame 0xedf84bdc _sleep(c1b9392c,c1b93914,74,c1636baa,0,...) at _sleep+0x38f/frame 0xedf84c30 vm_daemon(0,edf84ce8,c161c740,406,0,...) at vm_daemon+0xa9/frame 0xedf84cac fork_exit(c0fb70a0,0,edf84ce8) at fork_exit+0x7e/frame 0xedf84cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedf84cd4 --- trap 0, eip = 0, esp = 0xedf84d20, ebp = 0 --- Tracing command pagedaemon pid 17 tid 100089 td 0xc8f24a20 sched_switch(c8f24a20,0,104,18a,c8f24a20,...) at sched_switch+0x512/frame 0xedf80b28 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedf80b70 sleepq_switch(c8f24a20,0,c162c48d,2d1,b70,...) at sleepq_switch+0x11c/frame 0xedf80ba8 sleepq_timedwait(c1bf7185,54,0,0,0,...) at sleepq_timedwait+0x53/frame 0xedf80be4 _sleep(c1bf7185,c1bf7100,254,c1636baa,fffffed8,...) at _sleep+0x363/frame 0xedf80c38 vm_pageout(0,edf80ce8,c161c740,406,0,...) at vm_pageout+0x467/frame 0xedf80cac fork_exit(c0fb45a0,0,edf80ce8) at fork_exit+0x7e/frame 0xedf80cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedf80cd4 --- trap 0, eip = 0, esp = 0xedf80d20, ebp = 0 --- Tracing command pagedaemon pid 17 tid 100092 td 0xc91b96c0 sched_switch(c91b96c0,0,104,18a,c91b96c0,...) at sched_switch+0x512/frame 0xedf8cb08 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedf8cb50 sleepq_switch(c91b96c0,0,c162c48d,2ae,0,...) at sleepq_switch+0x11c/frame 0xedf8cb88 sleepq_wait(c1b938f4,54,c167992a,0,0,...) at sleepq_wait+0x48/frame 0xedf8cbac _sleep(c1b938f4,c1bf6e80,54,c167992a,0,...) at _sleep+0x38f/frame 0xedf8cc00 vm_pageout_laundry_worker(0,edf8cce8,c161c740,406,0,...) at vm_pageout_laundry_worker+0x37e/frame 0xedf8ccac fork_exit(c0fb58d0,0,edf8cce8) at fork_exit+0x7e/frame 0xedf8ccd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedf8ccd4 --- trap 0, eip = 0, esp = 0xedf8cd20, ebp = 0 --- Tracing command pagedaemon pid 17 tid 100093 td 0xc91b9360 sched_switch(c91b9360,0,104,18a,edf90ba4,...) at sched_switch+0x512/frame 0xedf90b80 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedf90bc4 sleepq_switch(c91b9360,0,c162c48d,2ae,c1b9350c,...) at sleepq_switch+0x11c/frame 0xedf90bfc sleepq_wait(c1b93520,54,edf90c50,0,0,...) at sleepq_wait+0x48/frame 0xedf90c20 _sleep(c1b93520,c1b9350c,54,c1676081,0,...) at _sleep+0x38f/frame 0xedf90c74 uma_reclaim_worker(0,edf90ce8,c161c740,406,0,...) at uma_reclaim_worker+0xaa/frame 0xedf90cac fork_exit(c0f94ab0,0,edf90ce8) at fork_exit+0x7e/frame 0xedf90cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedf90cd4 --- trap 0, eip = 0, esp = 0xedf90d20, ebp = 0 --- Tracing command rand_harvestq pid 16 tid 100086 td 0xc92cf6c0 sched_switch(c92cf6c0,0,104,18a,c92cf6c0,...) at sched_switch+0x512/frame 0xeddf4b38 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xeddf4b80 sleepq_switch(c92cf6c0,0,c162c48d,2d1,b70,...) at sleepq_switch+0x11c/frame 0xeddf4bb8 sleepq_timedwait(c19eb3e8,0,0,0,0,...) at sleepq_timedwait+0x53/frame 0xeddf4bf4 _sleep(c19eb3e8,0,0,c160c2db,19999999,...) at _sleep+0x363/frame 0xeddf4c48 random_kthread(0,eddf4ce8,c161c740,406,0,...) at random_kthread+0x7c/frame 0xeddf4cac fork_exit(c096b270,0,eddf4ce8) at fork_exit+0x7e/frame 0xeddf4cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeddf4cd4 --- trap 0, eip = 0, esp = 0xeddf4d20, ebp = 0 --- Tracing command sctp_iterator pid 15 tid 100085 td 0xc92cfa20 sched_switch(c92cfa20,0,104,18a,c92cfa20,...) at sched_switch+0x512/frame 0xeddf0b88 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xeddf0bd0 sleepq_switch(c92cfa20,0,c162c48d,2ae,0,...) at sleepq_switch+0x11c/frame 0xeddf0c08 sleepq_wait(c1bf2a74,0,c1654687,0,0,...) at sleepq_wait+0x48/frame 0xeddf0c2c _sleep(c1bf2a74,c1bf2a3c,0,c1654687,0,...) at _sleep+0x38f/frame 0xeddf0c80 sctp_iterator_thread(0,eddf0ce8,c161c740,406,0,...) at sctp_iterator_thread+0x7c/frame 0xeddf0cac fork_exit(c0df14d0,0,eddf0ce8) at fork_exit+0x7e/frame 0xeddf0cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeddf0cd4 --- trap 0, eip = 0, esp = 0xeddf0d20, ebp = 0 --- Tracing command soaiod4 pid 9 tid 100084 td 0xc92d0000 sched_switch(c92d0000,0,104,18a,c92d0000,...) at sched_switch+0x512/frame 0xeddebb60 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xeddebba8 sleepq_switch(c92d0000,0,c162c48d,2d1,b80,...) at sleepq_switch+0x11c/frame 0xeddebbe0 sleepq_timedwait(c1b8c15c,0,1d,0,0,...) at sleepq_timedwait+0x53/frame 0xeddebc1c _sleep(c1b8c15c,c1b8c13c,0,c160c2db,ffffdd50,...) at _sleep+0x363/frame 0xeddebc70 soaio_kproc_loop(4,eddebce8,c161c740,406,0,...) at soaio_kproc_loop+0x1fd/frame 0xeddebcac fork_exit(c0cccad0,4,eddebce8) at fork_exit+0x7e/frame 0xeddebcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeddebcd4 --- trap 0, eip = 0, esp = 0xeddebd20, ebp = 0 --- Tracing command soaiod3 pid 8 tid 100083 td 0xc92d0360 sched_switch(c92d0360,0,104,18a,c92d0360,...) at sched_switch+0x512/frame 0xedde7b60 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedde7ba8 sleepq_switch(c92d0360,0,c162c48d,2d1,b81,...) at sleepq_switch+0x11c/frame 0xedde7be0 sleepq_timedwait(c1b8c15c,0,1d,0,0,...) at sleepq_timedwait+0x53/frame 0xedde7c1c _sleep(c1b8c15c,c1b8c13c,0,c160c2db,ffffdd50,...) at _sleep+0x363/frame 0xedde7c70 soaio_kproc_loop(3,edde7ce8,c161c740,406,0,...) at soaio_kproc_loop+0x1fd/frame 0xedde7cac fork_exit(c0cccad0,3,edde7ce8) at fork_exit+0x7e/frame 0xedde7cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedde7cd4 --- trap 0, eip = 0, esp = 0xedde7d20, ebp = 0 --- Tracing command soaiod2 pid 7 tid 100082 td 0xc92d06c0 sched_switch(c92d06c0,0,104,18a,c92d06c0,...) at sched_switch+0x512/frame 0xedde3b60 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedde3ba8 sleepq_switch(c92d06c0,0,c162c48d,2d1,b80,...) at sleepq_switch+0x11c/frame 0xedde3be0 sleepq_timedwait(c1b8c15c,0,1d,0,0,...) at sleepq_timedwait+0x53/frame 0xedde3c1c _sleep(c1b8c15c,c1b8c13c,0,c160c2db,ffffdd50,...) at _sleep+0x363/frame 0xedde3c70 soaio_kproc_loop(2,edde3ce8,c161c740,406,0,...) at soaio_kproc_loop+0x1fd/frame 0xedde3cac fork_exit(c0cccad0,2,edde3ce8) at fork_exit+0x7e/frame 0xedde3cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedde3cd4 --- trap 0, eip = 0, esp = 0xedde3d20, ebp = 0 --- Tracing command soaiod1 pid 6 tid 100081 td 0xc92d0a20 sched_switch(c92d0a20,0,104,18a,c92d0a20,...) at sched_switch+0x512/frame 0xedddfb60 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedddfba8 sleepq_switch(c92d0a20,0,c162c48d,2d1,b80,...) at sleepq_switch+0x11c/frame 0xedddfbe0 sleepq_timedwait(c1b8c15c,0,1d,0,0,...) at sleepq_timedwait+0x53/frame 0xedddfc1c _sleep(c1b8c15c,c1b8c13c,0,c160c2db,ffffdd50,...) at _sleep+0x363/frame 0xedddfc70 soaio_kproc_loop(1,edddfce8,c161c740,406,0,...) at soaio_kproc_loop+0x1fd/frame 0xedddfcac fork_exit(c0cccad0,1,edddfce8) at fork_exit+0x7e/frame 0xedddfcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedddfcd4 --- trap 0, eip = 0, esp = 0xedddfd20, ebp = 0 --- Tracing command fdc0 pid 5 tid 100077 td 0xc91ba6c0 sched_switch(c91ba6c0,0,104,18a,c91ba6c0,...) at sched_switch+0x512/frame 0xeddcfb30 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xeddcfb78 sleepq_switch(c91ba6c0,0,c162c48d,2ae,0,...) at sleepq_switch+0x11c/frame 0xeddcfbb0 sleepq_wait(c9174a3c,5c,c160c2db,0,0,...) at sleepq_wait+0x48/frame 0xeddcfbd4 _sleep(c9174a3c,c9174af0,5c,c160c2db,0,...) at _sleep+0x38f/frame 0xeddcfc28 fdc_thread(c9174a00,eddcfce8,c161c740,406,0,...) at fdc_thread+0x14bc/frame 0xeddcfcac fork_exit(c118f6a0,c9174a00,eddcfce8) at fork_exit+0x7e/frame 0xeddcfcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeddcfcd4 --- trap 0, eip = 0, esp = 0xeddcfd20, ebp = 0 --- Tracing command usb pid 14 tid 100042 td 0xc9109a20 sched_switch(c9109a20,0,104,18a,c9109a20,...) at sched_switch+0x512/frame 0xedd23b98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedd23be0 sleepq_switch(c9109a20,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedd23c18 sleepq_wait(c90e5acc,0,edd23c5c,1,0,...) at sleepq_wait+0x48/frame 0xedd23c3c _cv_wait(c90e5acc,c90e5cf8,c15ee676,76,c90e5ad4,...) at _cv_wait+0x21f/frame 0xedd23c80 usb_process(c90e5ac4,edd23ce8,c161c740,406,0,...) at usb_process+0x190/frame 0xedd23cac fork_exit(c0a98e00,c90e5ac4,edd23ce8) at fork_exit+0x7e/frame 0xedd23cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd23cd4 --- trap 0, eip = 0, esp = 0xedd23d20, ebp = 0 --- Tracing command usb pid 14 tid 100043 td 0xc91096c0 sched_switch(c91096c0,0,104,18a,c91096c0,...) at sched_switch+0x512/frame 0xedd27b98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedd27be0 sleepq_switch(c91096c0,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedd27c18 sleepq_wait(c90e5afc,0,edd27c5c,1,0,...) at sleepq_wait+0x48/frame 0xedd27c3c _cv_wait(c90e5afc,c90e5cf8,c15ee676,76,c90e5b04,...) at _cv_wait+0x21f/frame 0xedd27c80 usb_process(c90e5af4,edd27ce8,c161c740,406,0,...) at usb_process+0x190/frame 0xedd27cac fork_exit(c0a98e00,c90e5af4,edd27ce8) at fork_exit+0x7e/frame 0xedd27cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd27cd4 --- trap 0, eip = 0, esp = 0xedd27d20, ebp = 0 --- Tracing command usb pid 14 tid 100044 td 0xc9109360 sched_switch(c9109360,0,104,18a,c9109360,...) at sched_switch+0x512/frame 0xedd2bb98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedd2bbe0 sleepq_switch(c9109360,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedd2bc18 sleepq_wait(c90e5b2c,0,edd2bc5c,1,0,...) at sleepq_wait+0x48/frame 0xedd2bc3c _cv_wait(c90e5b2c,c90e5cf8,c15ee676,76,c90e5b34,...) at _cv_wait+0x21f/frame 0xedd2bc80 usb_process(c90e5b24,edd2bce8,c161c740,406,0,...) at usb_process+0x190/frame 0xedd2bcac fork_exit(c0a98e00,c90e5b24,edd2bce8) at fork_exit+0x7e/frame 0xedd2bcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd2bcd4 --- trap 0, eip = 0, esp = 0xedd2bd20, ebp = 0 --- Tracing command usb pid 14 tid 100045 td 0xc9109000 sched_switch(c9109000,0,104,18a,c9109000,...) at sched_switch+0x512/frame 0xedd2fb98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedd2fbe0 sleepq_switch(c9109000,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedd2fc18 sleepq_wait(c90e5b5c,0,edd2fc5c,1,0,...) at sleepq_wait+0x48/frame 0xedd2fc3c _cv_wait(c90e5b5c,c90e5cf8,c15ee676,76,c90e5b64,...) at _cv_wait+0x21f/frame 0xedd2fc80 usb_process(c90e5b54,edd2fce8,c161c740,406,0,...) at usb_process+0x190/frame 0xedd2fcac fork_exit(c0a98e00,c90e5b54,edd2fce8) at fork_exit+0x7e/frame 0xedd2fcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd2fcd4 --- trap 0, eip = 0, esp = 0xedd2fd20, ebp = 0 --- Tracing command usb pid 14 tid 100046 td 0xc90f8a20 sched_switch(c90f8a20,0,104,18a,c90f8a20,...) at sched_switch+0x512/frame 0xedd33b98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedd33be0 sleepq_switch(c90f8a20,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedd33c18 sleepq_wait(c90e5b8c,0,edd33c5c,1,0,...) at sleepq_wait+0x48/frame 0xedd33c3c _cv_wait(c90e5b8c,c90e5cf8,c15ee676,76,c90e5b94,...) at _cv_wait+0x21f/frame 0xedd33c80 usb_process(c90e5b84,edd33ce8,c161c740,406,0,...) at usb_process+0x190/frame 0xedd33cac fork_exit(c0a98e00,c90e5b84,edd33ce8) at fork_exit+0x7e/frame 0xedd33cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd33cd4 --- trap 0, eip = 0, esp = 0xedd33d20, ebp = 0 --- Tracing command usb pid 14 tid 100048 td 0xc90f8360 sched_switch(c90f8360,0,104,18a,c90f8360,...) at sched_switch+0x512/frame 0xedd3cb98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedd3cbe0 sleepq_switch(c90f8360,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedd3cc18 sleepq_wait(c9157acc,0,edd3cc5c,1,0,...) at sleepq_wait+0x48/frame 0xedd3cc3c _cv_wait(c9157acc,c9157cf8,c15ee676,76,c9157ad4,...) at _cv_wait+0x21f/frame 0xedd3cc80 usb_process(c9157ac4,edd3cce8,c161c740,406,0,...) at usb_process+0x190/frame 0xedd3ccac fork_exit(c0a98e00,c9157ac4,edd3cce8) at fork_exit+0x7e/frame 0xedd3ccd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd3ccd4 --- trap 0, eip = 0, esp = 0xedd3cd20, ebp = 0 --- Tracing command usb pid 14 tid 100049 td 0xc90f8000 sched_switch(c90f8000,0,104,18a,c90f8000,...) at sched_switch+0x512/frame 0xedd40b98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedd40be0 sleepq_switch(c90f8000,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedd40c18 sleepq_wait(c9157afc,0,edd40c5c,1,0,...) at sleepq_wait+0x48/frame 0xedd40c3c _cv_wait(c9157afc,c9157cf8,c15ee676,76,c9157b04,...) at _cv_wait+0x21f/frame 0xedd40c80 usb_process(c9157af4,edd40ce8,c161c740,406,0,...) at usb_process+0x190/frame 0xedd40cac fork_exit(c0a98e00,c9157af4,edd40ce8) at fork_exit+0x7e/frame 0xedd40cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd40cd4 --- trap 0, eip = 0, esp = 0xedd40d20, ebp = 0 --- Tracing command usb pid 14 tid 100050 td 0xc8f47a20 sched_switch(c8f47a20,0,104,18a,c8f47a20,...) at sched_switch+0x512/frame 0xedd44b98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedd44be0 sleepq_switch(c8f47a20,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedd44c18 sleepq_wait(c9157b2c,0,edd44c5c,1,0,...) at sleepq_wait+0x48/frame 0xedd44c3c _cv_wait(c9157b2c,c9157cf8,c15ee676,76,c9157b34,...) at _cv_wait+0x21f/frame 0xedd44c80 usb_process(c9157b24,edd44ce8,c161c740,406,0,...) at usb_process+0x190/frame 0xedd44cac fork_exit(c0a98e00,c9157b24,edd44ce8) at fork_exit+0x7e/frame 0xedd44cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd44cd4 --- trap 0, eip = 0, esp = 0xedd44d20, ebp = 0 --- Tracing command usb pid 14 tid 100051 td 0xc915c000 sched_switch(c915c000,0,104,18a,c915c000,...) at sched_switch+0x512/frame 0xedd48b98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedd48be0 sleepq_switch(c915c000,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedd48c18 sleepq_wait(c9157b5c,0,edd48c5c,1,0,...) at sleepq_wait+0x48/frame 0xedd48c3c _cv_wait(c9157b5c,c9157cf8,c15ee676,76,c9157b64,...) at _cv_wait+0x21f/frame 0xedd48c80 usb_process(c9157b54,edd48ce8,c161c740,406,0,...) at usb_process+0x190/frame 0xedd48cac fork_exit(c0a98e00,c9157b54,edd48ce8) at fork_exit+0x7e/frame 0xedd48cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd48cd4 --- trap 0, eip = 0, esp = 0xedd48d20, ebp = 0 --- Tracing command usb pid 14 tid 100052 td 0xc915ba20 sched_switch(c915ba20,0,104,18a,c915ba20,...) at sched_switch+0x512/frame 0xedd4cb98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedd4cbe0 sleepq_switch(c915ba20,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedd4cc18 sleepq_wait(c9157b8c,0,edd4cc5c,1,0,...) at sleepq_wait+0x48/frame 0xedd4cc3c _cv_wait(c9157b8c,c9157cf8,c15ee676,76,c9157b94,...) at _cv_wait+0x21f/frame 0xedd4cc80 usb_process(c9157b84,edd4cce8,c161c740,406,0,...) at usb_process+0x190/frame 0xedd4ccac fork_exit(c0a98e00,c9157b84,edd4cce8) at fork_exit+0x7e/frame 0xedd4ccd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd4ccd4 --- trap 0, eip = 0, esp = 0xedd4cd20, ebp = 0 --- Tracing command usb pid 14 tid 100054 td 0xc915b360 sched_switch(c915b360,0,104,18a,c915b360,...) at sched_switch+0x512/frame 0xedd55b98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedd55be0 sleepq_switch(c915b360,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedd55c18 sleepq_wait(c915dacc,0,edd55c5c,1,0,...) at sleepq_wait+0x48/frame 0xedd55c3c _cv_wait(c915dacc,c915dcf8,c15ee676,76,c915dad4,...) at _cv_wait+0x21f/frame 0xedd55c80 usb_process(c915dac4,edd55ce8,c161c740,406,0,...) at usb_process+0x190/frame 0xedd55cac fork_exit(c0a98e00,c915dac4,edd55ce8) at fork_exit+0x7e/frame 0xedd55cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd55cd4 --- trap 0, eip = 0, esp = 0xedd55d20, ebp = 0 --- Tracing command usb pid 14 tid 100055 td 0xc915b000 sched_switch(c915b000,0,104,18a,c915b000,...) at sched_switch+0x512/frame 0xedd59b98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedd59be0 sleepq_switch(c915b000,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedd59c18 sleepq_wait(c915dafc,0,edd59c5c,1,0,...) at sleepq_wait+0x48/frame 0xedd59c3c _cv_wait(c915dafc,c915dcf8,c15ee676,76,c915db04,...) at _cv_wait+0x21f/frame 0xedd59c80 usb_process(c915daf4,edd59ce8,c161c740,406,0,...) at usb_process+0x190/frame 0xedd59cac fork_exit(c0a98e00,c915daf4,edd59ce8) at fork_exit+0x7e/frame 0xedd59cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd59cd4 --- trap 0, eip = 0, esp = 0xedd59d20, ebp = 0 --- Tracing command usb pid 14 tid 100056 td 0xc915aa20 sched_switch(c915aa20,0,104,18a,c915aa20,...) at sched_switch+0x512/frame 0xedd5db98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedd5dbe0 sleepq_switch(c915aa20,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedd5dc18 sleepq_wait(c915db2c,0,edd5dc5c,1,0,...) at sleepq_wait+0x48/frame 0xedd5dc3c _cv_wait(c915db2c,c915dcf8,c15ee676,76,c915db34,...) at _cv_wait+0x21f/frame 0xedd5dc80 usb_process(c915db24,edd5dce8,c161c740,406,0,...) at usb_process+0x190/frame 0xedd5dcac fork_exit(c0a98e00,c915db24,edd5dce8) at fork_exit+0x7e/frame 0xedd5dcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd5dcd4 --- trap 0, eip = 0, esp = 0xedd5dd20, ebp = 0 --- Tracing command usb pid 14 tid 100057 td 0xc915a6c0 sched_switch(c915a6c0,0,104,18a,c915a6c0,...) at sched_switch+0x512/frame 0xedd61b98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedd61be0 sleepq_switch(c915a6c0,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedd61c18 sleepq_wait(c915db5c,0,edd61c5c,1,0,...) at sleepq_wait+0x48/frame 0xedd61c3c _cv_wait(c915db5c,c915dcf8,c15ee676,76,c915db64,...) at _cv_wait+0x21f/frame 0xedd61c80 usb_process(c915db54,edd61ce8,c161c740,406,0,...) at usb_process+0x190/frame 0xedd61cac fork_exit(c0a98e00,c915db54,edd61ce8) at fork_exit+0x7e/frame 0xedd61cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd61cd4 --- trap 0, eip = 0, esp = 0xedd61d20, ebp = 0 --- Tracing command usb pid 14 tid 100058 td 0xc915a360 sched_switch(c915a360,0,104,18a,c915a360,...) at sched_switch+0x512/frame 0xedd65b98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedd65be0 sleepq_switch(c915a360,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedd65c18 sleepq_wait(c915db8c,0,edd65c5c,1,0,...) at sleepq_wait+0x48/frame 0xedd65c3c _cv_wait(c915db8c,c915dcf8,c15ee676,76,c915db94,...) at _cv_wait+0x21f/frame 0xedd65c80 usb_process(c915db84,edd65ce8,c161c740,406,0,...) at usb_process+0x190/frame 0xedd65cac fork_exit(c0a98e00,c915db84,edd65ce8) at fork_exit+0x7e/frame 0xedd65cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd65cd4 --- trap 0, eip = 0, esp = 0xedd65d20, ebp = 0 --- Tracing command usb pid 14 tid 100059 td 0xc915a000 sched_switch(c915a000,0,104,18a,c915a000,...) at sched_switch+0x512/frame 0xedd6ab98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedd6abe0 sleepq_switch(c915a000,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedd6ac18 sleepq_wait(c9164acc,0,edd6ac5c,1,0,...) at sleepq_wait+0x48/frame 0xedd6ac3c _cv_wait(c9164acc,c9164cf8,c15ee676,76,c9164ad4,...) at _cv_wait+0x21f/frame 0xedd6ac80 usb_process(c9164ac4,edd6ace8,c161c740,406,0,...) at usb_process+0x190/frame 0xedd6acac fork_exit(c0a98e00,c9164ac4,edd6ace8) at fork_exit+0x7e/frame 0xedd6acd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd6acd4 --- trap 0, eip = 0, esp = 0xedd6ad20, ebp = 0 --- Tracing command usb pid 14 tid 100060 td 0xc9159a20 sched_switch(c9159a20,0,104,18a,c9159a20,...) at sched_switch+0x512/frame 0xedd6eb98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedd6ebe0 sleepq_switch(c9159a20,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedd6ec18 sleepq_wait(c9164afc,0,edd6ec5c,1,0,...) at sleepq_wait+0x48/frame 0xedd6ec3c _cv_wait(c9164afc,c9164cf8,c15ee676,76,c9164b04,...) at _cv_wait+0x21f/frame 0xedd6ec80 usb_process(c9164af4,edd6ece8,c161c740,406,0,...) at usb_process+0x190/frame 0xedd6ecac fork_exit(c0a98e00,c9164af4,edd6ece8) at fork_exit+0x7e/frame 0xedd6ecd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd6ecd4 --- trap 0, eip = 0, esp = 0xedd6ed20, ebp = 0 --- Tracing command usb pid 14 tid 100061 td 0xc91596c0 sched_switch(c91596c0,0,104,18a,c91596c0,...) at sched_switch+0x512/frame 0xedd72b98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedd72be0 sleepq_switch(c91596c0,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedd72c18 sleepq_wait(c9164b2c,0,edd72c5c,1,0,...) at sleepq_wait+0x48/frame 0xedd72c3c _cv_wait(c9164b2c,c9164cf8,c15ee676,76,c9164b34,...) at _cv_wait+0x21f/frame 0xedd72c80 usb_process(c9164b24,edd72ce8,c161c740,406,0,...) at usb_process+0x190/frame 0xedd72cac fork_exit(c0a98e00,c9164b24,edd72ce8) at fork_exit+0x7e/frame 0xedd72cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd72cd4 --- trap 0, eip = 0, esp = 0xedd72d20, ebp = 0 --- Tracing command usb pid 14 tid 100062 td 0xc9159360 sched_switch(c9159360,0,104,18a,c9159360,...) at sched_switch+0x512/frame 0xedd76b98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedd76be0 sleepq_switch(c9159360,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedd76c18 sleepq_wait(c9164b5c,0,edd76c5c,1,0,...) at sleepq_wait+0x48/frame 0xedd76c3c _cv_wait(c9164b5c,c9164cf8,c15ee676,76,c9164b64,...) at _cv_wait+0x21f/frame 0xedd76c80 usb_process(c9164b54,edd76ce8,c161c740,406,0,...) at usb_process+0x190/frame 0xedd76cac fork_exit(c0a98e00,c9164b54,edd76ce8) at fork_exit+0x7e/frame 0xedd76cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd76cd4 --- trap 0, eip = 0, esp = 0xedd76d20, ebp = 0 --- Tracing command usb pid 14 tid 100063 td 0xc9159000 sched_switch(c9159000,0,104,18a,c9159000,...) at sched_switch+0x512/frame 0xedd7ab98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedd7abe0 sleepq_switch(c9159000,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedd7ac18 sleepq_wait(c9164b8c,0,edd7ac5c,1,0,...) at sleepq_wait+0x48/frame 0xedd7ac3c _cv_wait(c9164b8c,c9164cf8,c15ee676,76,c9164b94,...) at _cv_wait+0x21f/frame 0xedd7ac80 usb_process(c9164b84,edd7ace8,c161c740,406,0,...) at usb_process+0x190/frame 0xedd7acac fork_exit(c0a98e00,c9164b84,edd7ace8) at fork_exit+0x7e/frame 0xedd7acd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd7acd4 --- trap 0, eip = 0, esp = 0xedd7ad20, ebp = 0 --- Tracing command usb pid 14 tid 100064 td 0xc9171360 sched_switch(c9171360,0,104,18a,c9171360,...) at sched_switch+0x512/frame 0xedd7fb98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedd7fbe0 sleepq_switch(c9171360,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedd7fc18 sleepq_wait(c9169acc,0,edd7fc5c,1,0,...) at sleepq_wait+0x48/frame 0xedd7fc3c _cv_wait(c9169acc,c9169cf8,c15ee676,76,c9169ad4,...) at _cv_wait+0x21f/frame 0xedd7fc80 usb_process(c9169ac4,edd7fce8,c161c740,406,0,...) at usb_process+0x190/frame 0xedd7fcac fork_exit(c0a98e00,c9169ac4,edd7fce8) at fork_exit+0x7e/frame 0xedd7fcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd7fcd4 --- trap 0, eip = 0, esp = 0xedd7fd20, ebp = 0 --- Tracing command usb pid 14 tid 100065 td 0xc9171000 sched_switch(c9171000,0,104,18a,c9171000,...) at sched_switch+0x512/frame 0xedd83b98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedd83be0 sleepq_switch(c9171000,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedd83c18 sleepq_wait(c9169afc,0,edd83c5c,1,0,...) at sleepq_wait+0x48/frame 0xedd83c3c _cv_wait(c9169afc,c9169cf8,c15ee676,76,c9169b04,...) at _cv_wait+0x21f/frame 0xedd83c80 usb_process(c9169af4,edd83ce8,c161c740,406,0,...) at usb_process+0x190/frame 0xedd83cac fork_exit(c0a98e00,c9169af4,edd83ce8) at fork_exit+0x7e/frame 0xedd83cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd83cd4 --- trap 0, eip = 0, esp = 0xedd83d20, ebp = 0 --- Tracing command usb pid 14 tid 100066 td 0xc9170a20 sched_switch(c9170a20,0,104,18a,c9170a20,...) at sched_switch+0x512/frame 0xedd87b98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedd87be0 sleepq_switch(c9170a20,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedd87c18 sleepq_wait(c9169b2c,0,edd87c5c,1,0,...) at sleepq_wait+0x48/frame 0xedd87c3c _cv_wait(c9169b2c,c9169cf8,c15ee676,76,c9169b34,...) at _cv_wait+0x21f/frame 0xedd87c80 usb_process(c9169b24,edd87ce8,c161c740,406,0,...) at usb_process+0x190/frame 0xedd87cac fork_exit(c0a98e00,c9169b24,edd87ce8) at fork_exit+0x7e/frame 0xedd87cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd87cd4 --- trap 0, eip = 0, esp = 0xedd87d20, ebp = 0 --- Tracing command usb pid 14 tid 100067 td 0xc91706c0 sched_switch(c91706c0,0,104,18a,c91706c0,...) at sched_switch+0x512/frame 0xedd8bb98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedd8bbe0 sleepq_switch(c91706c0,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedd8bc18 sleepq_wait(c9169b5c,0,edd8bc5c,1,0,...) at sleepq_wait+0x48/frame 0xedd8bc3c _cv_wait(c9169b5c,c9169cf8,c15ee676,76,c9169b64,...) at _cv_wait+0x21f/frame 0xedd8bc80 usb_process(c9169b54,edd8bce8,c161c740,406,0,...) at usb_process+0x190/frame 0xedd8bcac fork_exit(c0a98e00,c9169b54,edd8bce8) at fork_exit+0x7e/frame 0xedd8bcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd8bcd4 --- trap 0, eip = 0, esp = 0xedd8bd20, ebp = 0 --- Tracing command usb pid 14 tid 100068 td 0xc9170360 sched_switch(c9170360,0,104,18a,c9170360,...) at sched_switch+0x512/frame 0xedd8fb98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedd8fbe0 sleepq_switch(c9170360,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedd8fc18 sleepq_wait(c9169b8c,0,edd8fc5c,1,0,...) at sleepq_wait+0x48/frame 0xedd8fc3c _cv_wait(c9169b8c,c9169cf8,c15ee676,76,c9169b94,...) at _cv_wait+0x21f/frame 0xedd8fc80 usb_process(c9169b84,edd8fce8,c161c740,406,0,...) at usb_process+0x190/frame 0xedd8fcac fork_exit(c0a98e00,c9169b84,edd8fce8) at fork_exit+0x7e/frame 0xedd8fcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd8fcd4 --- trap 0, eip = 0, esp = 0xedd8fd20, ebp = 0 --- Tracing command usb pid 14 tid 100070 td 0xc916fa20 sched_switch(c916fa20,0,104,18a,c916fa20,...) at sched_switch+0x512/frame 0xedd99b98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedd99be0 sleepq_switch(c916fa20,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedd99c18 sleepq_wait(c90d0ca4,0,edd99c5c,1,0,...) at sleepq_wait+0x48/frame 0xedd99c3c _cv_wait(c90d0ca4,c90d0ed0,c15ee676,76,c90d0cac,...) at _cv_wait+0x21f/frame 0xedd99c80 usb_process(c90d0c9c,edd99ce8,c161c740,406,0,...) at usb_process+0x190/frame 0xedd99cac fork_exit(c0a98e00,c90d0c9c,edd99ce8) at fork_exit+0x7e/frame 0xedd99cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd99cd4 --- trap 0, eip = 0, esp = 0xedd99d20, ebp = 0 --- Tracing command usb pid 14 tid 100071 td 0xc916f6c0 sched_switch(c916f6c0,0,104,18a,c916f6c0,...) at sched_switch+0x512/frame 0xedd9db98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedd9dbe0 sleepq_switch(c916f6c0,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedd9dc18 sleepq_wait(c90d0cd4,0,edd9dc5c,1,0,...) at sleepq_wait+0x48/frame 0xedd9dc3c _cv_wait(c90d0cd4,c90d0ed0,c15ee676,76,c90d0cdc,...) at _cv_wait+0x21f/frame 0xedd9dc80 usb_process(c90d0ccc,edd9dce8,c161c740,406,0,...) at usb_process+0x190/frame 0xedd9dcac fork_exit(c0a98e00,c90d0ccc,edd9dce8) at fork_exit+0x7e/frame 0xedd9dcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd9dcd4 --- trap 0, eip = 0, esp = 0xedd9dd20, ebp = 0 --- Tracing command usb pid 14 tid 100072 td 0xc916f360 sched_switch(c916f360,0,104,18a,c916f360,...) at sched_switch+0x512/frame 0xedda1b98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedda1be0 sleepq_switch(c916f360,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedda1c18 sleepq_wait(c90d0d04,0,edda1c5c,1,0,...) at sleepq_wait+0x48/frame 0xedda1c3c _cv_wait(c90d0d04,c90d0ed0,c15ee676,76,c90d0d0c,...) at _cv_wait+0x21f/frame 0xedda1c80 usb_process(c90d0cfc,edda1ce8,c161c740,406,0,...) at usb_process+0x190/frame 0xedda1cac fork_exit(c0a98e00,c90d0cfc,edda1ce8) at fork_exit+0x7e/frame 0xedda1cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedda1cd4 --- trap 0, eip = 0, esp = 0xedda1d20, ebp = 0 --- Tracing command usb pid 14 tid 100073 td 0xc916f000 sched_switch(c916f000,0,104,18a,c916f000,...) at sched_switch+0x512/frame 0xedda5b98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedda5be0 sleepq_switch(c916f000,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedda5c18 sleepq_wait(c90d0d34,0,edda5c5c,1,0,...) at sleepq_wait+0x48/frame 0xedda5c3c _cv_wait(c90d0d34,c90d0ed0,c15ee676,76,c90d0d3c,...) at _cv_wait+0x21f/frame 0xedda5c80 usb_process(c90d0d2c,edda5ce8,c161c740,406,0,...) at usb_process+0x190/frame 0xedda5cac fork_exit(c0a98e00,c90d0d2c,edda5ce8) at fork_exit+0x7e/frame 0xedda5cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedda5cd4 --- trap 0, eip = 0, esp = 0xedda5d20, ebp = 0 --- Tracing command usb pid 14 tid 100074 td 0xc915ca20 sched_switch(c915ca20,0,104,18a,c915ca20,...) at sched_switch+0x512/frame 0xedda9b98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedda9be0 sleepq_switch(c915ca20,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xedda9c18 sleepq_wait(c90d0d64,0,edda9c5c,1,0,...) at sleepq_wait+0x48/frame 0xedda9c3c _cv_wait(c90d0d64,c90d0ed0,c15ee676,76,c90d0d6c,...) at _cv_wait+0x21f/frame 0xedda9c80 usb_process(c90d0d5c,edda9ce8,c161c740,406,0,...) at usb_process+0x190/frame 0xedda9cac fork_exit(c0a98e00,c90d0d5c,edda9ce8) at fork_exit+0x7e/frame 0xedda9cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedda9cd4 --- trap 0, eip = 0, esp = 0xedda9d20, ebp = 0 --- Tracing command usb pid 14 tid 100115 td 0xc969b000 sched_switch(c969b000,0,104,18a,c969b000,...) at sched_switch+0x512/frame 0xf34dfb98 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xf34dfbe0 sleepq_switch(c969b000,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xf34dfc18 sleepq_wait(c972ec08,0,f34dfc5c,1,0,...) at sleepq_wait+0x48/frame 0xf34dfc3c _cv_wait(c972ec08,c972ef34,c15ee676,76,c972ec10,...) at _cv_wait+0x21f/frame 0xf34dfc80 usb_process(c972ec00,f34dfce8,c161c740,406,0,...) at usb_process+0x190/frame 0xf34dfcac fork_exit(c0a98e00,c972ec00,f34dfce8) at fork_exit+0x7e/frame 0xf34dfcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf34dfcd4 --- trap 0, eip = 0, esp = 0xf34dfd20, ebp = 0 --- Tracing command cam pid 4 tid 100037 td 0xc8f45000 sched_switch(c8f45000,0,104,18a,c8f45000,...) at sched_switch+0x512/frame 0xe4bbfb68 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xe4bbfbb0 sleepq_switch(c8f45000,0,c162c48d,2ae,0,...) at sleepq_switch+0x11c/frame 0xe4bbfbe8 sleepq_wait(c1937880,5c,c160c2db,0,0,...) at sleepq_wait+0x48/frame 0xe4bbfc0c _sleep(c1937880,c1937800,5c,c160c2db,0,...) at _sleep+0x38f/frame 0xe4bbfc60 xpt_done_td(c1937800,e4bbfce8,c161c740,406,0,...) at xpt_done_td+0xee/frame 0xe4bbfcac fork_exit(c04e2870,c1937800,e4bbfce8) at fork_exit+0x7e/frame 0xe4bbfcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4bbfcd4 --- trap 0, eip = 0, esp = 0xe4bbfd20, ebp = 0 --- Tracing command cam pid 4 tid 100088 td 0xc92cf000 sched_switch(c92cf000,0,104,18a,c92cf000,...) at sched_switch+0x512/frame 0xeddfcb60 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xeddfcba8 sleepq_switch(c92cf000,0,c162c48d,2ae,0,...) at sleepq_switch+0x11c/frame 0xeddfcbe0 sleepq_wait(c193772c,5c,c160c2db,0,0,...) at sleepq_wait+0x48/frame 0xeddfcc04 _sleep(c193772c,c1937780,5c,c160c2db,0,...) at _sleep+0x38f/frame 0xeddfcc58 xpt_scanner_thread(0,eddfcce8,c161c740,406,0,...) at xpt_scanner_thread+0x1b2/frame 0xeddfccac fork_exit(c04e2c30,0,eddfcce8) at fork_exit+0x7e/frame 0xeddfccd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeddfccd4 --- trap 0, eip = 0, esp = 0xeddfcd20, ebp = 0 --- Tracing command crypto returns pid 3 tid 100033 td 0xc8f46000 sched_switch(c8f46000,0,104,18a,c8f46000,...) at sched_switch+0x512/frame 0xe4bafb68 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xe4bafbb0 sleepq_switch(c8f46000,0,c162c48d,2ae,0,...) at sleepq_switch+0x11c/frame 0xe4bafbe8 sleepq_wait(c1b924bc,6c,c16688a0,0,0,...) at sleepq_wait+0x48/frame 0xe4bafc0c _sleep(c1b924bc,c1b924a8,6c,c16688a0,0,...) at _sleep+0x38f/frame 0xe4bafc60 crypto_ret_proc(0,e4bafce8,c161c740,406,0,...) at crypto_ret_proc+0x1f5/frame 0xe4bafcac fork_exit(c0ee4e10,0,e4bafce8) at fork_exit+0x7e/frame 0xe4bafcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4bafcd4 --- trap 0, eip = 0, esp = 0xe4bafd20, ebp = 0 --- Tracing command crypto pid 2 tid 100032 td 0xc8f46360 sched_switch(c8f46360,0,104,18a,e4babb9c,...) at sched_switch+0x512/frame 0xe4babb78 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xe4babbbc sleepq_switch(c8f46360,0,c162c48d,2ae,0,...) at sleepq_switch+0x11c/frame 0xe4babbf4 sleepq_wait(c1b92404,6c,c1668894,0,0,...) at sleepq_wait+0x48/frame 0xe4babc18 _sleep(c1b92404,c1b923ec,6c,c1668894,0,...) at _sleep+0x38f/frame 0xe4babc6c crypto_proc(0,e4babce8,c161c740,406,0,...) at crypto_proc+0x9d/frame 0xe4babcac fork_exit(c0ee4a10,0,e4babce8) at fork_exit+0x7e/frame 0xe4babcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4babcd4 --- trap 0, eip = 0, esp = 0xe4babd20, ebp = 0 --- Tracing command geom pid 13 tid 100028 td 0xc8f47360 sched_switch(c8f47360,0,104,18a,c8f47360,...) at sched_switch+0x512/frame 0xe4b9bb78 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xe4b9bbc0 sleepq_switch(c8f47360,0,c162c48d,2ae,0,...) at sleepq_switch+0x11c/frame 0xe4b9bbf8 sleepq_wait(c1bedb44,5c,c160c2db,0,0,...) at sleepq_wait+0x48/frame 0xe4b9bc1c _sleep(c1bedb44,c1b0fe78,25c,c160c2db,0,...) at _sleep+0x38f/frame 0xe4b9bc70 g_run_events(0,e4b9bce8,c161c740,406,0,...) at g_run_events+0x141/frame 0xe4b9bcac fork_exit(c0b9da00,0,e4b9bce8) at fork_exit+0x7e/frame 0xe4b9bcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4b9bcd4 --- trap 0, eip = 0, esp = 0xe4b9bd20, ebp = 0 --- Tracing command geom pid 13 tid 100029 td 0xc8f47000 sched_switch(c8f47000,0,104,18a,c8f47000,...) at sched_switch+0x512/frame 0xe4b9fb68 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xe4b9fbb0 sleepq_switch(c8f47000,0,c162c48d,2ae,0,...) at sleepq_switch+0x11c/frame 0xe4b9fbe8 sleepq_wait(c1bedb48,5c,c160c2db,0,0,...) at sleepq_wait+0x48/frame 0xe4b9fc0c _sleep(c1bedb48,c1b0fec0,25c,c160c2db,0,...) at _sleep+0x38f/frame 0xe4b9fc60 g_io_schedule_up(c8f47000,0,c160e312,5f,e4b9fcd4,...) at g_io_schedule_up+0xcf/frame 0xe4b9fc94 g_up_procbody(0,e4b9fce8,c161c740,406,0,...) at g_up_procbody+0x7d/frame 0xe4b9fcac fork_exit(c0b9da80,0,e4b9fce8) at fork_exit+0x7e/frame 0xe4b9fcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4b9fcd4 --- trap 0, eip = 0, esp = 0xe4b9fd20, ebp = 0 --- Tracing command geom pid 13 tid 100030 td 0xc8f46a20 sched_switch(c8f46a20,0,104,18a,e4ba3b8c,...) at sched_switch+0x512/frame 0xe4ba3b68 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xe4ba3bac sleepq_switch(c8f46a20,0,c162c48d,2ae,0,...) at sleepq_switch+0x11c/frame 0xe4ba3be4 sleepq_wait(c1bedb4c,5c,c160c2db,0,0,...) at sleepq_wait+0x48/frame 0xe4ba3c08 _sleep(c1bedb4c,c1b0fea0,25c,c160c2db,0,...) at _sleep+0x38f/frame 0xe4ba3c5c g_io_schedule_down(c8f46a20,0,c160e312,6b,e4ba3cd4,...) at g_io_schedule_down+0xec/frame 0xe4ba3c94 g_down_procbody(0,e4ba3ce8,c161c740,406,0,...) at g_down_procbody+0x7d/frame 0xe4ba3cac fork_exit(c0b9db00,0,e4ba3ce8) at fork_exit+0x7e/frame 0xe4ba3cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4ba3cd4 --- trap 0, eip = 0, esp = 0xe4ba3d20, ebp = 0 --- Tracing command intr pid 12 tid 100007 td 0xc8e666c0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100008 td 0xc8e66360 sched_switch(c8e66360,0,109,18a,0,...) at sched_switch+0x512/frame 0xe4b4bc28 mi_switch(109,0,c161cabd,59f,c161c740,...) at mi_switch+0x226/frame 0xe4b4bc70 ithread_loop(c8d01d90,e4b4bce8,c161c740,406,0,...) at ithread_loop+0x200/frame 0xe4b4bcac fork_exit(c0c1a9b0,c8d01d90,e4b4bce8) at fork_exit+0x7e/frame 0xe4b4bcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4b4bcd4 --- trap 0, eip = 0, esp = 0xe4b4bd20, ebp = 0 --- Tracing command intr pid 12 tid 100009 td 0xc8e66000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100010 td 0xc8e5ba20 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100011 td 0xc8e5b6c0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100012 td 0xc8e5b360 sched_switch(c8e5b360,0,109,18a,0,...) at sched_switch+0x512/frame 0xe4b5bc28 mi_switch(109,0,c161cabd,59f,c161c740,...) at mi_switch+0x226/frame 0xe4b5bc70 ithread_loop(c8d01d50,e4b5bce8,c161c740,406,0,...) at ithread_loop+0x200/frame 0xe4b5bcac fork_exit(c0c1a9b0,c8d01d50,e4b5bce8) at fork_exit+0x7e/frame 0xe4b5bcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4b5bcd4 --- trap 0, eip = 0, esp = 0xe4b5bd20, ebp = 0 --- Tracing command intr pid 12 tid 100023 td 0xc8f22000 sched_switch(c8f22000,0,109,18a,0,...) at sched_switch+0x512/frame 0xe4b87c28 mi_switch(109,0,c161cabd,59f,c161c740,...) at mi_switch+0x226/frame 0xe4b87c70 ithread_loop(c8d01d40,e4b87ce8,c161c740,406,0,...) at ithread_loop+0x200/frame 0xe4b87cac fork_exit(c0c1a9b0,c8d01d40,e4b87ce8) at fork_exit+0x7e/frame 0xe4b87cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4b87cd4 --- trap 0, eip = 0, esp = 0xe4b87d20, ebp = 0 --- Tracing command intr pid 12 tid 100024 td 0xc8e68a20 sched_switch(c8e68a20,0,109,18a,0,...) at sched_switch+0x512/frame 0xe4b8bc28 mi_switch(109,0,c161cabd,59f,c161c740,...) at mi_switch+0x226/frame 0xe4b8bc70 ithread_loop(c8d01d30,e4b8bce8,c161c740,406,0,...) at ithread_loop+0x200/frame 0xe4b8bcac fork_exit(c0c1a9b0,c8d01d30,e4b8bce8) at fork_exit+0x7e/frame 0xe4b8bcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4b8bcd4 --- trap 0, eip = 0, esp = 0xe4b8bd20, ebp = 0 --- Tracing command intr pid 12 tid 100027 td 0xc8f476c0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100038 td 0xc910aa20 sched_switch(c910aa20,0,109,18a,0,...) at sched_switch+0x512/frame 0xe4bf2c28 mi_switch(109,0,c161cabd,59f,c161c740,...) at mi_switch+0x226/frame 0xe4bf2c70 ithread_loop(c9083d40,e4bf2ce8,c161c740,406,0,...) at ithread_loop+0x200/frame 0xe4bf2cac fork_exit(c0c1a9b0,c9083d40,e4bf2ce8) at fork_exit+0x7e/frame 0xe4bf2cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4bf2cd4 --- trap 0, eip = 0, esp = 0xe4bf2d20, ebp = 0 --- Tracing command intr pid 12 tid 100039 td 0xc910a6c0 sched_switch(c910a6c0,0,109,18a,0,...) at sched_switch+0x512/frame 0xf0efbc28 mi_switch(109,0,c161cabd,59f,c161c740,...) at mi_switch+0x226/frame 0xf0efbc70 ithread_loop(c9083780,f0efbce8,c161c740,406,0,...) at ithread_loop+0x200/frame 0xf0efbcac fork_exit(c0c1a9b0,c9083780,f0efbce8) at fork_exit+0x7e/frame 0xf0efbcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf0efbcd4 --- trap 0, eip = 0, esp = 0xf0efbd20, ebp = 0 --- Tracing command intr pid 12 tid 100040 td 0xc910a360 sched_switch(c910a360,0,109,18a,0,...) at sched_switch+0x512/frame 0xf0f00c28 mi_switch(109,0,c161cabd,59f,c161c740,...) at mi_switch+0x226/frame 0xf0f00c70 ithread_loop(c90833e0,f0f00ce8,c161c740,406,0,...) at ithread_loop+0x200/frame 0xf0f00cac fork_exit(c0c1a9b0,c90833e0,f0f00ce8) at fork_exit+0x7e/frame 0xf0f00cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf0f00cd4 --- trap 0, eip = 0, esp = 0xf0f00d20, ebp = 0 --- Tracing command intr pid 12 tid 100041 td 0xc910a000 sched_switch(c910a000,0,109,18a,0,...) at sched_switch+0x512/frame 0xedd1fc28 mi_switch(109,0,c161cabd,59f,c161c740,...) at mi_switch+0x226/frame 0xedd1fc70 ithread_loop(c90f0e60,edd1fce8,c161c740,406,0,...) at ithread_loop+0x200/frame 0xedd1fcac fork_exit(c0c1a9b0,c90f0e60,edd1fce8) at fork_exit+0x7e/frame 0xedd1fcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd1fcd4 --- trap 0, eip = 0, esp = 0xedd1fd20, ebp = 0 --- Tracing command intr pid 12 tid 100047 td 0xc90f86c0 sched_switch(c90f86c0,0,109,18a,0,...) at sched_switch+0x512/frame 0xedd38c28 mi_switch(109,0,c161cabd,59f,c161c740,...) at mi_switch+0x226/frame 0xedd38c70 ithread_loop(c90f0c10,edd38ce8,c161c740,406,0,...) at ithread_loop+0x200/frame 0xedd38cac fork_exit(c0c1a9b0,c90f0c10,edd38ce8) at fork_exit+0x7e/frame 0xedd38cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd38cd4 --- trap 0, eip = 0, esp = 0xedd38d20, ebp = 0 --- Tracing command intr pid 12 tid 100053 td 0xc915b6c0 sched_switch(c915b6c0,0,109,18a,0,...) at sched_switch+0x512/frame 0xedd51c28 mi_switch(109,0,c161cabd,59f,c161c740,...) at mi_switch+0x226/frame 0xedd51c70 ithread_loop(c90f0a20,edd51ce8,c161c740,406,0,...) at ithread_loop+0x200/frame 0xedd51cac fork_exit(c0c1a9b0,c90f0a20,edd51ce8) at fork_exit+0x7e/frame 0xedd51cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd51cd4 --- trap 0, eip = 0, esp = 0xedd51d20, ebp = 0 --- Tracing command intr pid 12 tid 100069 td 0xc9170000 sched_switch(c9170000,0,109,18a,0,...) at sched_switch+0x512/frame 0xedd95c28 mi_switch(109,0,c161cabd,59f,c161c740,...) at mi_switch+0x226/frame 0xedd95c70 ithread_loop(c90f0410,edd95ce8,c161c740,406,0,...) at ithread_loop+0x200/frame 0xedd95cac fork_exit(c0c1a9b0,c90f0410,edd95ce8) at fork_exit+0x7e/frame 0xedd95cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedd95cd4 --- trap 0, eip = 0, esp = 0xedd95d20, ebp = 0 --- Tracing command intr pid 12 tid 100075 td 0xc915c6c0 sched_switch(c915c6c0,0,109,18a,0,...) at sched_switch+0x512/frame 0xeddafc28 mi_switch(109,0,c161cabd,59f,c161c740,...) at mi_switch+0x226/frame 0xeddafc70 ithread_loop(c91970d0,eddafce8,c161c740,406,0,...) at ithread_loop+0x200/frame 0xeddafcac fork_exit(c0c1a9b0,c91970d0,eddafce8) at fork_exit+0x7e/frame 0xeddafcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeddafcd4 --- trap 0, eip = 0, esp = 0xeddafd20, ebp = 0 --- Tracing command intr pid 12 tid 100076 td 0xc915c360 sched_switch(c915c360,0,109,18a,0,...) at sched_switch+0x512/frame 0xeddc1c28 mi_switch(109,0,c161cabd,59f,c161c740,...) at mi_switch+0x226/frame 0xeddc1c70 ithread_loop(c9197ce0,eddc1ce8,c161c740,406,0,...) at ithread_loop+0x200/frame 0xeddc1cac fork_exit(c0c1a9b0,c9197ce0,eddc1ce8) at fork_exit+0x7e/frame 0xeddc1cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeddc1cd4 --- trap 0, eip = 0, esp = 0xeddc1d20, ebp = 0 --- Tracing command intr pid 12 tid 100078 td 0xc91ba360 sched_switch(c91ba360,0,109,18a,0,...) at sched_switch+0x512/frame 0xeddd3c28 mi_switch(109,0,c161cabd,59f,c161c740,...) at mi_switch+0x226/frame 0xeddd3c70 ithread_loop(c91978c0,eddd3ce8,c161c740,406,0,...) at ithread_loop+0x200/frame 0xeddd3cac fork_exit(c0c1a9b0,c91978c0,eddd3ce8) at fork_exit+0x7e/frame 0xeddd3cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeddd3cd4 --- trap 0, eip = 0, esp = 0xeddd3d20, ebp = 0 --- Tracing command intr pid 12 tid 100079 td 0xc91ba000 fork_trampoline() at fork_trampoline Tracing command idle pid 11 tid 100003 td 0xc8e676c0 (CPU 0) cpustop_handler(0,13,e4b36b8c,c1224aae,0,...) at cpustop_handler+0x25/frame 0xe4b36a40 ipi_nmi_handler(0,e4b36a90,c0c3747b,c8e676c0,c1bad58c,...) at ipi_nmi_handler+0x36/frame 0xe4b36a50 trap(e4b36b98) at trap+0xae/frame 0xe4b36b8c calltrap() at calltrap+0x6/frame 0xe4b36b8c --- trap 0x13, eip = 0xc123fbe5, esp = 0xe4b36bd8, ebp = 0xe4b36bd8 --- acpi_cpu_c1(e4b36c2c,c122eef9,c2027c90,0,c1833803,...) at acpi_cpu_c1+0x5/frame 0xe4b36bd8 acpi_cpu_idle(a4b98b2,0,a4b98b2,e4b36c48,c123ffb6,...) at acpi_cpu_idle+0x325/frame 0xe4b36c18 cpu_idle_acpi(a4b98b2,0,c1b20380,3,c8e676c0,...) at cpu_idle_acpi+0x3f/frame 0xe4b36c2c cpu_idle(0,e4b36c80,c1628197,a6a,0,...) at cpu_idle+0xa6/frame 0xe4b36c48 sched_idletd(0,e4b36ce8,c161c740,406,0,...) at sched_idletd+0x16a/frame 0xe4b36cac fork_exit(c0c88380,0,e4b36ce8) at fork_exit+0x7e/frame 0xe4b36cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4b36cd4 --- trap 0, eip = 0, esp = 0xe4b36d20, ebp = 0 --- Tracing command idle pid 11 tid 100004 td 0xc8e67360 (CPU 1) cpustop_handler(1,13,e4b3ab8c,c1224aae,8,...) at cpustop_handler+0x25/frame 0xe4b3aa40 ipi_nmi_handler(8,c162c48d,3f3,c162c48d,312,...) at ipi_nmi_handler+0x36/frame 0xe4b3aa50 trap(e4b3ab98) at trap+0xae/frame 0xe4b3ab8c calltrap() at calltrap+0x6/frame 0xe4b3ab8c --- trap 0x13, eip = 0xc123fbe5, esp = 0xe4b3abd8, ebp = 0xe4b3abd8 --- acpi_cpu_c1(e4b3ac2c,c122eef9,e4b18c90,0,c1833803,...) at acpi_cpu_c1+0x5/frame 0xe4b3abd8 acpi_cpu_idle(7f44acc2,0,7f44acc2,e4b3ac48,c123ffb6,...) at acpi_cpu_idle+0x325/frame 0xe4b3ac18 cpu_idle_acpi(7f44acc2,0,c1b20a80,3,c8e67360,...) at cpu_idle_acpi+0x3f/frame 0xe4b3ac2c cpu_idle(0,e4b3ac80,c1628197,a6a,0,...) at cpu_idle+0xa6/frame 0xe4b3ac48 sched_idletd(0,e4b3ace8,c161c740,406,0,...) at sched_idletd+0x16a/frame 0xe4b3acac fork_exit(c0c88380,0,e4b3ace8) at fork_exit+0x7e/frame 0xe4b3acd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4b3acd4 --- trap 0, eip = 0, esp = 0xe4b3ad20, ebp = 0 --- Tracing command idle pid 11 tid 100005 td 0xc8e67000 (CPU 2) cpustop_handler(2,13,e4b3eb8c,c1224aae,a,...) at cpustop_handler+0x25/frame 0xe4b3ea40 ipi_nmi_handler(a,c1626f2d,c1badc1c,2,0,...) at ipi_nmi_handler+0x36/frame 0xe4b3ea50 trap(e4b3eb98) at trap+0xae/frame 0xe4b3eb8c calltrap() at calltrap+0x6/frame 0xe4b3eb8c --- trap 0x13, eip = 0xc123fbe5, esp = 0xe4b3ebd8, ebp = 0xe4b3ebd8 --- acpi_cpu_c1(e4b3ec2c,c122eef9,e4b1cc90,0,c1833803,...) at acpi_cpu_c1+0x5/frame 0xe4b3ebd8 acpi_cpu_idle(61343ad6,0,61343ad6,e4b3ec48,c123ffb6,...) at acpi_cpu_idle+0x325/frame 0xe4b3ec18 cpu_idle_acpi(61343ad6,0,c1b21180,1,c8e67000,...) at cpu_idle_acpi+0x3f/frame 0xe4b3ec2c cpu_idle(0,e4b3ec80,c1628197,a6a,0,...) at cpu_idle+0xa6/frame 0xe4b3ec48 sched_idletd(0,e4b3ece8,c161c740,406,0,...) at sched_idletd+0x16a/frame 0xe4b3ecac fork_exit(c0c88380,0,e4b3ece8) at fork_exit+0x7e/frame 0xe4b3ecd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4b3ecd4 --- trap 0, eip = 0, esp = 0xe4b3ed20, ebp = 0 --- Tracing command idle pid 11 tid 100006 td 0xc8e66a20 (CPU 3) kdb_alt_break_internal(0,e4b42ac4,c0a5b2a7,2,c91a7888,...) at kdb_alt_break_internal+0x101/frame 0xe4b42a98 kdb_alt_break(2,c91a7888,c15e472b,0,40000,...) at kdb_alt_break+0x16/frame 0xe4b42aa4 uart_intr_rxready(c91a7800,6e4fbab4,e4b20c80,c8e66a20,e4b42ae8,...) at uart_intr_rxready+0x87/frame 0xe4b42ac4 uart_intr(c91a7800,b70,1,6e4fbab4,0,...) at uart_intr+0x120/frame 0xe4b42b04 intr_event_handle(c8eb3e00,e4b42b98,c1b21880,40000000,1,...) at intr_event_handle+0x75/frame 0xe4b42b30 intr_execute_handlers(c8eae908,e4b42b98,3) at intr_execute_handlers+0xb0/frame 0xe4b42b68 lapic_handle_intr(32,e4b42b98) at lapic_handle_intr+0x5b/frame 0xe4b42b88 Xapic_isr1() at Xapic_isr1+0x55/frame 0xe4b42b88 --- interrupt, eip = 0xc123fbe5, esp = 0xe4b42bd8, ebp = 0xe4b42bd8 --- acpi_cpu_c1(e4b42c2c,c122eef9,e4b20c90,0,c1833803,...) at acpi_cpu_c1+0x5/frame 0xe4b42bd8 acpi_cpu_idle(ffd0882,0,ffd0882,e4b42c48,c123ffb6,...) at acpi_cpu_idle+0x325/frame 0xe4b42c18 cpu_idle_acpi(ffd0882,0,c1b21880,2,c8e66a20,...) at cpu_idle_acpi+0x3f/frame 0xe4b42c2c cpu_idle(0,e4b42c80,c1628197,407,0,...) at cpu_idle+0xa6/frame 0xe4b42c48 sched_idletd(0,e4b42ce8,c161c740,406,0,...) at sched_idletd+0x16a/frame 0xe4b42cac fork_exit(c0c88380,0,e4b42ce8) at fork_exit+0x7e/frame 0xe4b42cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4b42cd4 --- trap 0, eip = 0, esp = 0xe4b42d20, ebp = 0 --- Tracing command init pid 1 tid 100002 td 0xc8e67a20 sched_switch(c8e67a20,0,104,18a,e4b3192c,...) at sched_switch+0x512/frame 0xe4b31908 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xe4b3194c sleepq_switch(c8e67a20,0,c162c48d,1c5,5b0,...) at sleepq_switch+0x11c/frame 0xe4b31984 sleepq_catch_signals(c162c48d,173,c8e59ae4,222,100,...) at sleepq_catch_signals+0x36e/frame 0xe4b319c4 sleepq_wait_sig(c8e59a2c,6c,c163071f,100,0,...) at sleepq_wait_sig+0x14/frame 0xe4b319e8 _sleep(c8e59a2c,c8e59ad4,16c,c163071f,0,...) at _sleep+0x37c/frame 0xe4b31a3c kern_wait6(c8e67a20,7,0,0,e4b31acc,...) at kern_wait6+0x23d/frame 0xe4b31aa0 sys_wait4(c8e67a20,e4b31c00,c161c17f,4,ffffffff,...) at sys_wait4+0x87/frame 0xe4b31bc0 syscall(e4b31ce8) at syscall+0x75c/frame 0xe4b31cdc Xint0x80_syscall() at Xint0x80_syscall+0x2e/frame 0xe4b31cdc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x8065ad7, esp = 0xbfbfe834, ebp = 0xbfbfe848 --- Tracing command audit pid 10 tid 100001 td 0xc8e68000 sched_switch(c8e68000,0,104,18a,c8e68000,...) at sched_switch+0x512/frame 0xe4b2db68 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xe4b2dbb0 sleepq_switch(c8e68000,0,c162c48d,2ae,c18efa58,...) at sleepq_switch+0x11c/frame 0xe4b2dbe8 sleepq_wait(c1bf5990,0,e4b2dc2c,1,0,...) at sleepq_wait+0x48/frame 0xe4b2dc0c _cv_wait(c1bf5990,c1bf5914,c166aeee,1bc,0,...) at _cv_wait+0x21f/frame 0xe4b2dc50 audit_worker(0,e4b2dce8,c161c740,406,0,...) at audit_worker+0x3c4/frame 0xe4b2dcac fork_exit(c0f081d0,0,e4b2dce8) at fork_exit+0x7e/frame 0xe4b2dcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4b2dcd4 --- trap 0, eip = 0, esp = 0xe4b2dd20, ebp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xc1bedee0 sched_switch(c1bedee0,0,104,18a,c1bedee0,...) at sched_switch+0x512/frame 0xc2022be0 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xc2022c28 sleepq_switch(c1bedee0,0,c162c48d,2d1,b74,...) at sleepq_switch+0x11c/frame 0xc2022c60 sleepq_timedwait(c1bedb64,54,9,0,0,...) at sleepq_timedwait+0x53/frame 0xc2022c9c _sleep(c1bedb64,0,54,c1676c36,fffff470,...) at _sleep+0x363/frame 0xc2022cf0 swapper() at swapper+0xfa/frame 0xc2022d38 begin() at begin+0x22 Tracing command kernel pid 0 tid 100013 td 0xc8e5b000 sched_switch(c8e5b000,0,104,18a,e4b5fbb4,...) at sched_switch+0x512/frame 0xe4b5fb90 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xe4b5fbd4 sleepq_switch(c8e5b000,0,c162c48d,2ae,c8e34f00,...) at sleepq_switch+0x11c/frame 0xe4b5fc0c sleepq_wait(c8e34f00,0,c1626049,c160c2db,0,...) at sleepq_wait+0x48/frame 0xe4b5fc30 msleep_spin_sbt(c8e34f00,c8e34f1c,c160c2db,0,0,...) at msleep_spin_sbt+0x226/frame 0xe4b5fc74 gtaskqueue_thread_loop(c8e24c04,e4b5fce8,c161c740,406,0,...) at gtaskqueue_thread_loop+0xf3/frame 0xe4b5fcac fork_exit(c0c9c370,c8e24c04,e4b5fce8) at fork_exit+0x7e/frame 0xe4b5fcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4b5fcd4 --- trap 0, eip = 0, esp = 0xe4b5fd20, ebp = 0 --- Tracing command kernel pid 0 tid 100014 td 0xc8f24360 sched_switch(c8f24360,0,104,18a,e4b63bb4,...) at sched_switch+0x512/frame 0xe4b63b90 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xe4b63bd4 sleepq_switch(c8f24360,0,c162c48d,2ae,c8e34e80,...) at sleepq_switch+0x11c/frame 0xe4b63c0c sleepq_wait(c8e34e80,0,c1626049,c160c2db,0,...) at sleepq_wait+0x48/frame 0xe4b63c30 msleep_spin_sbt(c8e34e80,c8e34e9c,c160c2db,0,0,...) at msleep_spin_sbt+0x226/frame 0xe4b63c74 gtaskqueue_thread_loop(c8e24804,e4b63ce8,c161c740,406,0,...) at gtaskqueue_thread_loop+0xf3/frame 0xe4b63cac fork_exit(c0c9c370,c8e24804,e4b63ce8) at fork_exit+0x7e/frame 0xe4b63cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4b63cd4 --- trap 0, eip = 0, esp = 0xe4b63d20, ebp = 0 --- Tracing command kernel pid 0 tid 100015 td 0xc8f24000 sched_switch(c8f24000,0,104,18a,e4b67bb4,...) at sched_switch+0x512/frame 0xe4b67b90 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xe4b67bd4 sleepq_switch(c8f24000,0,c162c48d,2ae,c8e34e00,...) at sleepq_switch+0x11c/frame 0xe4b67c0c sleepq_wait(c8e34e00,0,c1626049,c160c2db,0,...) at sleepq_wait+0x48/frame 0xe4b67c30 msleep_spin_sbt(c8e34e00,c8e34e1c,c160c2db,0,0,...) at msleep_spin_sbt+0x226/frame 0xe4b67c74 gtaskqueue_thread_loop(c8e24814,e4b67ce8,c161c740,406,0,...) at gtaskqueue_thread_loop+0xf3/frame 0xe4b67cac fork_exit(c0c9c370,c8e24814,e4b67ce8) at fork_exit+0x7e/frame 0xe4b67cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4b67cd4 --- trap 0, eip = 0, esp = 0xe4b67d20, ebp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xc8f23a20 sched_switch(c8f23a20,0,104,18a,e4b6bbb4,...) at sched_switch+0x512/frame 0xe4b6bb90 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xe4b6bbd4 sleepq_switch(c8f23a20,0,c162c48d,2ae,c8e34d80,...) at sleepq_switch+0x11c/frame 0xe4b6bc0c sleepq_wait(c8e34d80,0,c1626049,c160c2db,0,...) at sleepq_wait+0x48/frame 0xe4b6bc30 msleep_spin_sbt(c8e34d80,c8e34d9c,c160c2db,0,0,...) at msleep_spin_sbt+0x226/frame 0xe4b6bc74 gtaskqueue_thread_loop(c8e24824,e4b6bce8,c161c740,406,0,...) at gtaskqueue_thread_loop+0xf3/frame 0xe4b6bcac fork_exit(c0c9c370,c8e24824,e4b6bce8) at fork_exit+0x7e/frame 0xe4b6bcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4b6bcd4 --- trap 0, eip = 0, esp = 0xe4b6bd20, ebp = 0 --- Tracing command kernel pid 0 tid 100017 td 0xc8f236c0 sched_switch(c8f236c0,0,104,18a,e4b6fbb4,...) at sched_switch+0x512/frame 0xe4b6fb90 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xe4b6fbd4 sleepq_switch(c8f236c0,0,c162c48d,2ae,c8e34d00,...) at sleepq_switch+0x11c/frame 0xe4b6fc0c sleepq_wait(c8e34d00,0,c1626049,c160c2db,0,...) at sleepq_wait+0x48/frame 0xe4b6fc30 msleep_spin_sbt(c8e34d00,c8e34d1c,c160c2db,0,0,...) at msleep_spin_sbt+0x226/frame 0xe4b6fc74 gtaskqueue_thread_loop(c8e24834,e4b6fce8,c161c740,406,0,...) at gtaskqueue_thread_loop+0xf3/frame 0xe4b6fcac fork_exit(c0c9c370,c8e24834,e4b6fce8) at fork_exit+0x7e/frame 0xe4b6fcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4b6fcd4 --- trap 0, eip = 0, esp = 0xe4b6fd20, ebp = 0 --- Tracing command kernel pid 0 tid 100018 td 0xc8f23360 sched_switch(c8f23360,0,104,18a,e4b73bb4,...) at sched_switch+0x512/frame 0xe4b73b90 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xe4b73bd4 sleepq_switch(c8f23360,0,c162c48d,2ae,c8e34c80,...) at sleepq_switch+0x11c/frame 0xe4b73c0c sleepq_wait(c8e34c80,0,c1626049,c160c2db,0,...) at sleepq_wait+0x48/frame 0xe4b73c30 msleep_spin_sbt(c8e34c80,c8e34c9c,c160c2db,0,0,...) at msleep_spin_sbt+0x226/frame 0xe4b73c74 gtaskqueue_thread_loop(c8e24404,e4b73ce8,c161c740,406,0,...) at gtaskqueue_thread_loop+0xf3/frame 0xe4b73cac fork_exit(c0c9c370,c8e24404,e4b73ce8) at fork_exit+0x7e/frame 0xe4b73cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4b73cd4 --- trap 0, eip = 0, esp = 0xe4b73d20, ebp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xc8f23000 sched_switch(c8f23000,0,104,18a,e4b77bb4,...) at sched_switch+0x512/frame 0xe4b77b90 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xe4b77bd4 sleepq_switch(c8f23000,0,c162c48d,2ae,c8e34c00,...) at sleepq_switch+0x11c/frame 0xe4b77c0c sleepq_wait(c8e34c00,0,c1626049,c160c2db,0,...) at sleepq_wait+0x48/frame 0xe4b77c30 msleep_spin_sbt(c8e34c00,c8e34c1c,c160c2db,0,0,...) at msleep_spin_sbt+0x226/frame 0xe4b77c74 gtaskqueue_thread_loop(c8e24414,e4b77ce8,c161c740,406,0,...) at gtaskqueue_thread_loop+0xf3/frame 0xe4b77cac fork_exit(c0c9c370,c8e24414,e4b77ce8) at fork_exit+0x7e/frame 0xe4b77cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4b77cd4 --- trap 0, eip = 0, esp = 0xe4b77d20, ebp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xc8f22a20 sched_switch(c8f22a20,0,104,18a,e4b7bbb4,...) at sched_switch+0x512/frame 0xe4b7bb90 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xe4b7bbd4 sleepq_switch(c8f22a20,0,c162c48d,2ae,c8e34b80,...) at sleepq_switch+0x11c/frame 0xe4b7bc0c sleepq_wait(c8e34b80,0,c1626049,c160c2db,0,...) at sleepq_wait+0x48/frame 0xe4b7bc30 msleep_spin_sbt(c8e34b80,c8e34b9c,c160c2db,0,0,...) at msleep_spin_sbt+0x226/frame 0xe4b7bc74 gtaskqueue_thread_loop(c8e24424,e4b7bce8,c161c740,406,0,...) at gtaskqueue_thread_loop+0xf3/frame 0xe4b7bcac fork_exit(c0c9c370,c8e24424,e4b7bce8) at fork_exit+0x7e/frame 0xe4b7bcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4b7bcd4 --- trap 0, eip = 0, esp = 0xe4b7bd20, ebp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xc8f226c0 sched_switch(c8f226c0,0,104,18a,e4b7fbb4,...) at sched_switch+0x512/frame 0xe4b7fb90 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xe4b7fbd4 sleepq_switch(c8f226c0,0,c162c48d,2ae,c8e34b00,...) at sleepq_switch+0x11c/frame 0xe4b7fc0c sleepq_wait(c8e34b00,0,c1626049,c160c2db,0,...) at sleepq_wait+0x48/frame 0xe4b7fc30 msleep_spin_sbt(c8e34b00,c8e34b1c,c160c2db,0,0,...) at msleep_spin_sbt+0x226/frame 0xe4b7fc74 gtaskqueue_thread_loop(c8e24434,e4b7fce8,c161c740,406,0,...) at gtaskqueue_thread_loop+0xf3/frame 0xe4b7fcac fork_exit(c0c9c370,c8e24434,e4b7fce8) at fork_exit+0x7e/frame 0xe4b7fcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4b7fcd4 --- trap 0, eip = 0, esp = 0xe4b7fd20, ebp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xc8f22360 sched_switch(c8f22360,0,104,18a,e4b83ba4,...) at sched_switch+0x512/frame 0xe4b83b80 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xe4b83bc4 sleepq_switch(c8f22360,0,c162c48d,2ae,0,...) at sleepq_switch+0x11c/frame 0xe4b83bfc sleepq_wait(c8e34a80,0,c160c2db,0,0,...) at sleepq_wait+0x48/frame 0xe4b83c20 _sleep(c8e34a80,c8e34a9c,0,c160c2db,0,...) at _sleep+0x38f/frame 0xe4b83c74 taskqueue_thread_loop(c1bee468,e4b83ce8,c161c740,406,0,...) at taskqueue_thread_loop+0x144/frame 0xe4b83cac fork_exit(c0cb0ef0,c1bee468,e4b83ce8) at fork_exit+0x7e/frame 0xe4b83cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4b83cd4 --- trap 0, eip = 0, esp = 0xe4b83d20, ebp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xc8e686c0 sched_switch(c8e686c0,0,104,18a,e4b8fba4,...) at sched_switch+0x512/frame 0xe4b8fb80 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xe4b8fbc4 sleepq_switch(c8e686c0,0,c162c48d,2ae,0,...) at sleepq_switch+0x11c/frame 0xe4b8fbfc sleepq_wait(c8e34900,0,c160c2db,0,0,...) at sleepq_wait+0x48/frame 0xe4b8fc20 _sleep(c8e34900,c8e3491c,0,c160c2db,0,...) at _sleep+0x38f/frame 0xe4b8fc74 taskqueue_thread_loop(c1bf22a4,e4b8fce8,c161c740,406,0,...) at taskqueue_thread_loop+0x144/frame 0xe4b8fcac fork_exit(c0cb0ef0,c1bf22a4,e4b8fce8) at fork_exit+0x7e/frame 0xe4b8fcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4b8fcd4 --- trap 0, eip = 0, esp = 0xe4b8fd20, ebp = 0 --- Tracing command kernel pid 0 tid 100026 td 0xc8e68360 sched_switch(c8e68360,0,104,18a,e4b93ba4,...) at sched_switch+0x512/frame 0xe4b93b80 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xe4b93bc4 sleepq_switch(c8e68360,0,c162c48d,2ae,0,...) at sleepq_switch+0x11c/frame 0xe4b93bfc sleepq_wait(c8e34880,0,c160c2db,0,0,...) at sleepq_wait+0x48/frame 0xe4b93c20 _sleep(c8e34880,c8e3489c,0,c160c2db,0,...) at _sleep+0x38f/frame 0xe4b93c74 taskqueue_thread_loop(c1bf2320,e4b93ce8,c161c740,406,0,...) at taskqueue_thread_loop+0x144/frame 0xe4b93cac fork_exit(c0cb0ef0,c1bf2320,e4b93ce8) at fork_exit+0x7e/frame 0xe4b93cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4b93cd4 --- trap 0, eip = 0, esp = 0xe4b93d20, ebp = 0 --- Tracing command kernel pid 0 tid 100031 td 0xc8f466c0 sched_switch(c8f466c0,0,104,18a,e4ba7ba4,...) at sched_switch+0x512/frame 0xe4ba7b80 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xe4ba7bc4 sleepq_switch(c8f466c0,0,c162c48d,2ae,0,...) at sleepq_switch+0x11c/frame 0xe4ba7bfc sleepq_wait(c8e34680,0,c160c2db,0,0,...) at sleepq_wait+0x48/frame 0xe4ba7c20 _sleep(c8e34680,c8e3469c,0,c160c2db,0,...) at _sleep+0x38f/frame 0xe4ba7c74 taskqueue_thread_loop(c1b2e4a0,e4ba7ce8,c161c740,406,0,...) at taskqueue_thread_loop+0x144/frame 0xe4ba7cac fork_exit(c0cb0ef0,c1b2e4a0,e4ba7ce8) at fork_exit+0x7e/frame 0xe4ba7cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4ba7cd4 --- trap 0, eip = 0, esp = 0xe4ba7d20, ebp = 0 --- Tracing command kernel pid 0 tid 100034 td 0xc8f45a20 sched_switch(c8f45a20,0,104,18a,e4bb3bb4,...) at sched_switch+0x512/frame 0xe4bb3b90 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xe4bb3bd4 sleepq_switch(c8f45a20,0,c162c48d,2ae,c8e33d80,...) at sleepq_switch+0x11c/frame 0xe4bb3c0c sleepq_wait(c8e33d80,0,c1626049,c160c2db,0,...) at sleepq_wait+0x48/frame 0xe4bb3c30 msleep_spin_sbt(c8e33d80,c8e33d9c,c160c2db,0,0,...) at msleep_spin_sbt+0x226/frame 0xe4bb3c74 taskqueue_thread_loop(c1be369c,e4bb3ce8,c161c740,406,0,...) at taskqueue_thread_loop+0xf3/frame 0xe4bb3cac fork_exit(c0cb0ef0,c1be369c,e4bb3ce8) at fork_exit+0x7e/frame 0xe4bb3cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4bb3cd4 --- trap 0, eip = 0, esp = 0xe4bb3d20, ebp = 0 --- Tracing command kernel pid 0 tid 100035 td 0xc8f456c0 sched_switch(c8f456c0,0,104,18a,e4bb7bb4,...) at sched_switch+0x512/frame 0xe4bb7b90 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xe4bb7bd4 sleepq_switch(c8f456c0,0,c162c48d,2ae,c8e33d80,...) at sleepq_switch+0x11c/frame 0xe4bb7c0c sleepq_wait(c8e33d80,0,c1626049,c160c2db,0,...) at sleepq_wait+0x48/frame 0xe4bb7c30 msleep_spin_sbt(c8e33d80,c8e33d9c,c160c2db,0,0,...) at msleep_spin_sbt+0x226/frame 0xe4bb7c74 taskqueue_thread_loop(c1be369c,e4bb7ce8,c161c740,406,0,...) at taskqueue_thread_loop+0xf3/frame 0xe4bb7cac fork_exit(c0cb0ef0,c1be369c,e4bb7ce8) at fork_exit+0x7e/frame 0xe4bb7cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4bb7cd4 --- trap 0, eip = 0, esp = 0xe4bb7d20, ebp = 0 --- Tracing command kernel pid 0 tid 100036 td 0xc8f45360 sched_switch(c8f45360,0,104,18a,e4bbbbb4,...) at sched_switch+0x512/frame 0xe4bbbb90 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xe4bbbbd4 sleepq_switch(c8f45360,0,c162c48d,2ae,c8e33d80,...) at sleepq_switch+0x11c/frame 0xe4bbbc0c sleepq_wait(c8e33d80,0,c1626049,c160c2db,0,...) at sleepq_wait+0x48/frame 0xe4bbbc30 msleep_spin_sbt(c8e33d80,c8e33d9c,c160c2db,0,0,...) at msleep_spin_sbt+0x226/frame 0xe4bbbc74 taskqueue_thread_loop(c1be369c,e4bbbce8,c161c740,406,0,...) at taskqueue_thread_loop+0xf3/frame 0xe4bbbcac fork_exit(c0cb0ef0,c1be369c,e4bbbce8) at fork_exit+0x7e/frame 0xe4bbbcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xe4bbbcd4 --- trap 0, eip = 0, esp = 0xe4bbbd20, ebp = 0 --- Tracing command kernel pid 0 tid 100080 td 0xc91b9a20 sched_switch(c91b9a20,0,104,18a,edddbbb4,...) at sched_switch+0x512/frame 0xedddbb90 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xedddbbd4 sleepq_switch(c91b9a20,0,c162c48d,2ae,c9187980,...) at sleepq_switch+0x11c/frame 0xedddbc0c sleepq_wait(c9187980,0,c1626049,c160c2db,0,...) at sleepq_wait+0x48/frame 0xedddbc30 msleep_spin_sbt(c9187980,c918799c,c160c2db,0,0,...) at msleep_spin_sbt+0x226/frame 0xedddbc74 taskqueue_thread_loop(c1be1a94,edddbce8,c161c740,406,0,...) at taskqueue_thread_loop+0xf3/frame 0xedddbcac fork_exit(c0cb0ef0,c1be1a94,edddbce8) at fork_exit+0x7e/frame 0xedddbcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xedddbcd4 --- trap 0, eip = 0, esp = 0xedddbd20, ebp = 0 --- Tracing command kernel pid 0 tid 100087 td 0xc92cf360 sched_switch(c92cf360,0,104,18a,eddf8ba4,...) at sched_switch+0x512/frame 0xeddf8b80 mi_switch(104,0,c162c48d,226,0,...) at mi_switch+0x226/frame 0xeddf8bc4 sleepq_switch(c92cf360,0,c162c48d,2ae,0,...) at sleepq_switch+0x11c/frame 0xeddf8bfc sleepq_wait(c8e33d00,0,c160c2db,0,0,...) at sleepq_wait+0x48/frame 0xeddf8c20 _sleep(c8e33d00,c8e33d1c,0,c160c2db,0,...) at _sleep+0x38f/frame 0xeddf8c74 taskqueue_thread_loop(c19377a8,eddf8ce8,c161c740,406,0,...) at taskqueue_thread_loop+0x144/frame 0xeddf8cac fork_exit(c0cb0ef0,c19377a8,eddf8ce8) at fork_exit+0x7e/frame 0xeddf8cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeddf8cd4 --- trap 0, eip = 0, esp = 0xeddf8d20, ebp = 0 --- db:0:allt> dump Physical memory: 3427 MB Dumping 217 MB: 202 186 170 154 138 122 106 90 74 58 42 26 10 Dump complete db:0:dump> reset cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 3 (kgdb) proc 26898 [Switching to thread 107 (Thread 100263)]#0 sched_switch (td=, newtd=, flags=) at ../../../kern/sched_ule.c:1988 1988 cpuid = PCPU_GET(cpuid); Current language: auto; currently minimal (kgdb) bt #0 sched_switch (td=, newtd=, flags=) at ../../../kern/sched_ule.c:1988 #1 0xc0c63ff6 in mi_switch (flags=) at ../../../kern/kern_synch.c:438 #2 0xc0cab56c in sleepq_switch (wchan=, pri=0x60) at ../../../kern/subr_sleepqueue.c:608 #3 0xc0cab3f8 in sleepq_wait (wchan=0xce983000, pri=0x60) at ../../../kern/subr_sleepqueue.c:687 #4 0xc0c6375f in _sleep (lock=, priority=, wmesg=, sbt=0x0, pr=0x0, flags=) at ../../../kern/kern_synch.c:216 #5 0xc0d1c66f in vfs_mount_destroy (mp=0xce983000) at ../../../kern/vfs_mount.c:590 #6 0xc0d1efd5 in dounmount (mp=0xce983000, flags=0x8000000, td=0xcc8c1000) at ../../../kern/vfs_mount.c:1518 #7 0xc0d1db07 in sys_unmount (td=0xcc8c1000, uap=0xf37edc00) at ../../../kern/vfs_mount.c:1293 #8 0xc122631c in syscall (frame=) at subr_syscall.c:136 #9 0xc1211cfe in Xint0x80_syscall () at ../../../i386/i386/exception.s:277 #10 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) f 5 #5 0xc0d1c66f in vfs_mount_destroy (mp=0xce983000) at ../../../kern/vfs_mount.c:590 590 msleep(mp, MNT_MTX(mp), PVFS, "mntref", 0); (kgdb) l 585 if (mp->mnt_kern_flag & MNTK_MWAIT) { 586 mp->mnt_kern_flag &= ~MNTK_MWAIT; 587 wakeup(mp); 588 } 589 while (mp->mnt_ref) 590 msleep(mp, MNT_MTX(mp), PVFS, "mntref", 0); 591 KASSERT(mp->mnt_ref == 0, 592 ("%s: invalid refcount in the drain path @ %s:%d", __func__, 593 __FILE__, __LINE__)); 594 if (mp->mnt_writeopcount != 0) (kgdb) p *mp $1 = {mnt_mtx = {lock_object = {lo_name = 0xc162f065 "struct mount mtx", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc89ac588}, mtx_lock = 0x4}, mnt_gen = 0x3, mnt_list = {tqe_next = 0xcc7a9000, tqe_prev = 0xc9b1d018}, mnt_op = 0xc191c34c, mnt_vfc = 0xc191c3e4, mnt_vnodecovered = 0xc9b50870, mnt_syncer = 0x0, mnt_ref = 0x1, mnt_nvnodelist = {tqh_first = 0x0, tqh_last = 0xce983034}, mnt_nvnodelistsize = 0x0, mnt_writeopcount = 0x0, mnt_kern_flag = 0x4180616c, mnt_flag = 0x0, mnt_opt = 0xc9c1a850, mnt_optnew = 0x0, mnt_maxsymlinklen = 0x78, mnt_stat = {f_version = 0x20140518, f_type = 0x35, f_flags = 0x201000, f_bsize = 0x1000, f_iosize = 0x8000, f_blocks = 0x2e7a5, f_bfree = 0x2bd75, f_bavail = 0x281f8, f_files = 0x17ffe, f_ffree = 0xc5e2, f_syncwrites = 0x32e4f, f_asyncwrites = 0x1973e, f_syncreads = 0x9cd, f_asyncreads = 0x67f, f_spare = 0xce9830c4, f_namemax = 0xff, f_owner = 0x0, f_fsid = {val = 0xce98311c}, f_charspare = 0xce983124 "", f_fstypename = 0xce983174 "ufs", f_mntfromname = 0xce983184 "/dev/md6a", f_mntonname = 0xce983584 "/mnt6"}, mnt_cred = 0xcae0f900, mnt_data = 0x0, mnt_time = 0x0, mnt_iosize_max = 0x20000, mnt_export = 0x0, mnt_label = 0x0, mnt_hashseed = 0xc56490e3, mnt_lockref = 0x0, mnt_secondary_writes = 0x0, mnt_secondary_accwrites = 0xe2ac9, mnt_susp_owner = 0x0, mnt_gjprovider = 0x0, mnt_listmtx = {lock_object = {lo_name = 0xc1638ad7 "struct mount vlist mtx", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc89b5690}, mtx_lock = 0x4}, mnt_activevnodelist = {tqh_first = 0x0, tqh_last = 0xce9839c8}, mnt_activevnodelistsize = 0x0, mnt_tmpfreevnodelist = {tqh_first = 0x0, tqh_last = 0xce9839d4}, mnt_tmpfreevnodelistsize = 0x0, mnt_explock = {lock_object = {lo_name = 0xc160bd37 "explock", lo_flags = 0x6730000, lo_data = 0x0, lo_witness = 0xc89b3d60}, lk_lock = 0x1, lk_exslpfail = 0x0, lk_timo = 0x0, lk_pri = 0x60, lk_stack = {depth = 0x0, pcs = 0xce983a04}}, mnt_upper_link = {tqe_next = 0x0, tqe_prev = 0x0}, mnt_uppers = {tqh_first = 0x0, tqh_last = 0xce983a54}, mnt_refs = 0xce983a5c, mnt_unrefs = 0xce983d5c} (kgdb) p mp-> mnt_refs[0] $3 = {refs = 0x2c244, file = 0xc1639238 "../../../kern/vfs_subr.c", line = 0x220} (kgdb) p mp-> mnt_refs[1] $4 = {refs = 0x5f838, file = 0xc1639238 "../../../kern/vfs_subr.c", line = 0x63f} (kgdb) p mp-> mnt_refs[2] $5 = {refs = 0x604, file = 0xc163ad00 "../../../kern/vfs_syscalls.c", line = 0x169} (kgdb) p mp-> mnt_refs[3] $6 = {refs = 0x2cc987, file = 0xc1637feb "../../../kern/vfs_default.c", line = 0x263} (kgdb) p mp-> mnt_refs[4] $7 = {refs = 0x2b777, file = 0xc163af76 "../../../kern/vfs_vnops.c", line = 0x690} (kgdb) p mp-> mnt_refs[5] $8 = {refs = 0x29d2, file = 0xc1639238 "../../../kern/vfs_subr.c", line = 0x1582} (kgdb) p mp-> mnt_refs[6] $9 = {refs = 0x5ec71, file = 0xc163af76 "../../../kern/vfs_vnops.c", line = 0x6c4} (kgdb) p mp-> mnt_refs[7] $10 = {refs = 0x3e3, file = 0xc166f9f7 "../../../ufs/ffs/ffs_softdep.c", line = 0x3495} (kgdb) p mp-> mnt_refs[8] $11 = {refs = 0x6, file = 0xc1639238 "../../../kern/vfs_subr.c", line = 0x14b9} (kgdb) p mp-> mnt_refs[9] $12 = {refs = 0x1, file = 0xc163869f "../../../kern/vfs_mount.c", line = 0x4e0} (kgdb) p mp-> mnt_refs[10] $13 = {refs = 0x0, file = 0x0, line = 0x0} (kgdb) p mp-> mnt_unrefs[0] $14 = {refs = 0x2904d, file = 0xc1639238 "../../../kern/vfs_subr.c", line = 0x24d} (kgdb) p mp-> mnt_unrefs[1] $15 = {refs = 0x604, file = 0xc163ad00 "../../../kern/vfs_syscalls.c", line = 0xf7} (kgdb) p mp-> mnt_unrefs[2] $16 = {refs = 0x2742a7, file = 0xc163af76 "../../../kern/vfs_vnops.c", line = 0x6e7} (kgdb) p mp-> mnt_unrefs[3] $17 = {refs = 0xe2ac9, file = 0xc163af76 "../../../kern/vfs_vnops.c", line = 0x6fd} (kgdb) p mp-> mnt_unrefs[4] $18 = {refs = 0x29d2, file = 0xc1639238 "../../../kern/vfs_subr.c", line = 0x14ef} (kgdb) p mp-> mnt_unrefs[5] $19 = {refs = 0x5f838, file = 0xc1639238 "../../../kern/vfs_subr.c", line = 0x60e} (kgdb) p mp-> mnt_unrefs[6] $20 = {refs = 0x3b2, file = 0xc166f9f7 "../../../ufs/ffs/ffs_softdep.c", line = 0x34a7} (kgdb) p mp-> mnt_unrefs[7] $21 = {refs = 0x30, file = 0xc166f9f7 "../../../ufs/ffs/ffs_softdep.c", line = 0x3494} (kgdb) p mp-> mnt_unrefs[8] $22 = {refs = 0x6, file = 0xc1639238 "../../../kern/vfs_subr.c", line = 0x14de} (kgdb) p mp-> mnt_unrefs[9] $23 = {refs = 0x31f7, file = 0xc1639238 "../../../kern/vfs_subr.c", line = 0x22f} (kgdb) p mp-> mnt_unrefs[10] $24 = {refs = 0x0, file = 0x0, line = 0x0} (kgdb) (kgdb) proc 28279 [Switching to thread 108 (Thread 100296)]#0 sched_switch (td=, newtd=, flags=) at ../../../kern/sched_ule.c:1988 1988 cpuid = PCPU_GET(cpuid); (kgdb) bt #0 sched_switch (td=, newtd=, flags=) at ../../../kern/sched_ule.c:1988 #1 0xc0c63ff6 in mi_switch (flags=) at ../../../kern/kern_synch.c:438 #2 0xc0cab56c in sleepq_switch (wchan=, pri=0x60) at ../../../kern/subr_sleepqueue.c:608 #3 0xc0cab3f8 in sleepq_wait (wchan=0xcc7a9000, pri=0x60) at ../../../kern/subr_sleepqueue.c:687 #4 0xc0c6375f in _sleep (lock=, priority=, wmesg=, sbt=0x0, pr=0x0, flags=) at ../../../kern/kern_synch.c:216 #5 0xc0d1c66f in vfs_mount_destroy (mp=0xcc7a9000) at ../../../kern/vfs_mount.c:590 #6 0xc0d1efd5 in dounmount (mp=0xcc7a9000, flags=0x8080000, td=0xcc90da20) at ../../../kern/vfs_mount.c:1518 #7 0xc0d1db07 in sys_unmount (td=0xcc90da20, uap=0xc84dfc00) at ../../../kern/vfs_mount.c:1293 #8 0xc122631c in syscall (frame=) at subr_syscall.c:136 #9 0xc1211cfe in Xint0x80_syscall () at ../../../i386/i386/exception.s:277 #10 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) f 5 #5 0xc0d1c66f in vfs_mount_destroy (mp=0xcc7a9000) at ../../../kern/vfs_mount.c:590 590 msleep(mp, MNT_MTX(mp), PVFS, "mntref", 0); (kgdb) p *mp $26 = {mnt_mtx = {lock_object = {lo_name = 0xc162f065 "struct mount mtx", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc89ac588}, mtx_lock = 0x4}, mnt_gen = 0x7, mnt_list = {tqe_next = 0xcc7eb000, tqe_prev = 0xc9b1d018}, mnt_op = 0xc191c34c, mnt_vfc = 0xc191c3e4, mnt_vnodecovered = 0xcd660870, mnt_syncer = 0x0, mnt_ref = 0x1, mnt_nvnodelist = {tqh_first = 0x0, tqh_last = 0xcc7a9034}, mnt_nvnodelistsize = 0x0, mnt_writeopcount = 0x0, mnt_kern_flag = 0x4180616d, mnt_flag = 0x0, mnt_opt = 0xc9342510, mnt_optnew = 0x0, mnt_maxsymlinklen = 0x78, mnt_stat = {f_version = 0x20140518, f_type = 0x35, f_flags = 0x201000, f_bsize = 0x1000, f_iosize = 0x8000, f_blocks = 0x2e7a5, f_bfree = 0x2e782, f_bavail = 0x2ac05, f_files = 0x17ffe, f_ffree = 0x73ce, f_syncwrites = 0x617c7, f_asyncwrites = 0x47bd2, f_syncreads = 0x165c, f_asyncreads = 0x949, f_spare = 0xcc7a90c4, f_namemax = 0xff, f_owner = 0x0, f_fsid = {val = 0xcc7a911c}, f_charspare = 0xcc7a9124 "", f_fstypename = 0xcc7a9174 "ufs", f_mntfromname = 0xcc7a9184 "/dev/md8a", f_mntonname = 0xcc7a9584 "/mnt8"}, mnt_cred = 0xcad12c00, mnt_data = 0x0, mnt_time = 0x0, mnt_iosize_max = 0x20000, mnt_export = 0x0, mnt_label = 0x0, mnt_hashseed = 0x621b5ee5, mnt_lockref = 0x0, mnt_secondary_writes = 0x0, mnt_secondary_accwrites = 0x1849bb, mnt_susp_owner = 0x0, mnt_gjprovider = 0x0, mnt_listmtx = {lock_object = {lo_name = 0xc1638ad7 "struct mount vlist mtx", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc89b5690}, mtx_lock = 0x4}, mnt_activevnodelist = {tqh_first = 0x0, tqh_last = 0xcc7a99c8}, mnt_activevnodelistsize = 0x0, mnt_tmpfreevnodelist = {tqh_first = 0x0, tqh_last = 0xcc7a99d4}, mnt_tmpfreevnodelistsize = 0x0, mnt_explock = {lock_object = {lo_name = 0xc160bd37 "explock", lo_flags = 0x6730000, lo_data = 0x0, lo_witness = 0xc89b3d60}, lk_lock = 0x1, lk_exslpfail = 0x0, lk_timo = 0x0, lk_pri = 0x60, lk_stack = {depth = 0x0, pcs = 0xcc7a9a04}}, mnt_upper_link = {tqe_next = 0x0, tqe_prev = 0x0}, mnt_uppers = {tqh_first = 0x0, tqh_last = 0xcc7a9a54}, mnt_refs = 0xcc7a9a5c, mnt_unrefs = 0xcc7a9d5c} (kgdb) p mp->mnt_refs[0] $27 = {refs = 0x90881, file = 0xc1639238 "../../../kern/vfs_subr.c", line = 0x220} (kgdb) p mp->mnt_refs[1] $28 = {refs = 0x5bc47, file = 0xc1639238 "../../../kern/vfs_subr.c", line = 0x63f} (kgdb) p mp->mnt_refs[2] $29 = {refs = 0x1c6f, file = 0xc163ad00 "../../../kern/vfs_syscalls.c", line = 0x169} (kgdb) p mp->mnt_refs[3] $30 = {refs = 0x2b9042, file = 0xc1637feb "../../../kern/vfs_default.c", line = 0x263} (kgdb) p mp->mnt_refs[4] $31 = {refs = 0x224087, file = 0xc163af76 "../../../kern/vfs_vnops.c", line = 0x690} (kgdb) p mp->mnt_refs[5] $32 = {refs = 0x70a25, file = 0xc163af76 "../../../kern/vfs_vnops.c", line = 0x6c4} (kgdb) p mp->mnt_refs[6] $33 = {refs = 0x5ca9, file = 0xc1639238 "../../../kern/vfs_subr.c", line = 0x1582} (kgdb) p mp->mnt_refs[7] $34 = {refs = 0x457, file = 0xc166f9f7 "../../../ufs/ffs/ffs_softdep.c", line = 0x3495} (kgdb) p mp->mnt_refs[8] $35 = {refs = 0x1f, file = 0xc1639238 "../../../kern/vfs_subr.c", line = 0x14b9} (kgdb) p mp->mnt_refs[9] $36 = {refs = 0x1, file = 0xc163869f "../../../kern/vfs_mount.c", line = 0x4e0} (kgdb) p mp->mnt_refs[10] $37 = {refs = 0x1, file = 0xc163af76 "../../../kern/vfs_vnops.c", line = 0x748} (kgdb) p mp->mnt_refs[11] $38 = {refs = 0x0, file = 0x0, line = 0x0} (kgdb) p mp->mnt_unrefs[0] $39 = {refs = 0x90807, file = 0xc1639238 "../../../kern/vfs_subr.c", line = 0x24d} (kgdb) p mp->mnt_unrefs[1] $40 = {refs = 0x1c6f, file = 0xc163ad00 "../../../kern/vfs_syscalls.c", line = 0xf7} (kgdb) p mp->mnt_unrefs[2] $41 = {refs = 0x1b3e87, file = 0xc163af76 "../../../kern/vfs_vnops.c", line = 0x6e7} (kgdb) p mp->mnt_unrefs[3] $42 = {refs = 0x1849bb, file = 0xc163af76 "../../../kern/vfs_vnops.c", line = 0x6fd} (kgdb) p mp->mnt_unrefs[4] $43 = {refs = 0x5bc47, file = 0xc1639238 "../../../kern/vfs_subr.c", line = 0x60e} (kgdb) p mp->mnt_unrefs[5] $44 = {refs = 0x5ca9, file = 0xc1639238 "../../../kern/vfs_subr.c", line = 0x14ef} (kgdb) p mp->mnt_unrefs[6] $45 = {refs = 0x44d, file = 0xc166f9f7 "../../../ufs/ffs/ffs_softdep.c", line = 0x34a7} (kgdb) p mp->mnt_unrefs[7] $46 = {refs = 0x1e, file = 0xc1639238 "../../../kern/vfs_subr.c", line = 0x14de} (kgdb) p mp->mnt_unrefs[8] $47 = {refs = 0x9, file = 0xc166f9f7 "../../../ufs/ffs/ffs_softdep.c", line = 0x3494} (kgdb) p mp->mnt_unrefs[9] $48 = {refs = 0x7a, file = 0xc1639238 "../../../kern/vfs_subr.c", line = 0x22f} (kgdb) p mp->mnt_unrefs[10] $49 = {refs = 0x2152ae, file = 0xc163af76 "../../../kern/vfs_vnops.c", line = 0x663} (kgdb) p mp->mnt_unrefs[11] $50 = {refs = 0x1, file = 0xc1639238 "../../../kern/vfs_subr.c", line = 0x14c3} (kgdb) p mp->mnt_unrefs[12] $51 = {refs = 0x0, file = 0x0, line = 0x0} (kgdb) $ svn diff -x -p /usr/src/sys Index: /usr/src/sys/kern/kern_proc.c =================================================================== --- /usr/src/sys/kern/kern_proc.c (revision 319211) +++ /usr/src/sys/kern/kern_proc.c (working copy) @@ -226,6 +226,8 @@ proc_dtor(void *mem, int size, void *arg) #endif /* Free all OSD associated to this thread. */ osd_thread_exit(td); + td_softdep_cleanup(td); + MPASS(td->td_su == NULL); /* Make sure all thread destructors are executed */ EVENTHANDLER_INVOKE(thread_dtor, td); Index: /usr/src/sys/kern/vfs_mount.c =================================================================== --- /usr/src/sys/kern/vfs_mount.c (revision 319211) +++ /usr/src/sys/kern/vfs_mount.c (working copy) @@ -430,25 +430,107 @@ sys_nmount(struct thread *td, struct nmount_args * */ void -vfs_ref(struct mount *mp) +vfs_ref1(struct mount *mp, const char *file, int line) { CTR2(KTR_VFS, "%s: mp %p", __func__, mp); MNT_ILOCK(mp); - MNT_REF(mp); + mp->mnt_ref++; + mnt_record_ref(mp, file, line); MNT_IUNLOCK(mp); } void -vfs_rel(struct mount *mp) +vfs_rel1(struct mount *mp, const char *file, int line) { CTR2(KTR_VFS, "%s: mp %p", __func__, mp); MNT_ILOCK(mp); - MNT_REL(mp); + KASSERT(mp->mnt_ref > 0, ("negative mnt_ref")); + mp->mnt_ref--; + mnt_record_unref(mp, file, line); + if (mp->mnt_ref == 0) + wakeup(mp); MNT_IUNLOCK(mp); } +static void +mnt_check_refs(struct mount *mp) +{ + struct mount_refs *mr; + int refs, i; + + refs = 0; + for (i = 0; i < nitems(mp->mnt_refs); i++) { + mr = &mp->mnt_refs[i]; + refs += mr->refs; + } + for (i = 0; i < nitems(mp->mnt_unrefs); i++) { + mr = &mp->mnt_unrefs[i]; + refs -= mr->refs; + } + if (refs != mp->mnt_ref) + panic("mp %p refs %d mnt_ref %d\n", mp, refs, mp->mnt_ref); +} + +void +mnt_record_ref(struct mount *mp, const char *file, int line) +{ + struct mount_refs *mr; + int i; + + mtx_assert(MNT_MTX(mp), MA_OWNED); + for (i = 0; i < nitems(mp->mnt_refs); i++) { + mr = &mp->mnt_refs[i]; + if (mr->file != NULL && strcmp(mr->file, file) == 0 && + mr->line == line) { + mr->refs++; + mnt_check_refs(mp); + return; + } + } + for (i = 0; i < nitems(mp->mnt_refs); i++) { + mr = &mp->mnt_refs[i]; + if (mr->file == NULL) { + mr->file = file; + mr->line = line; + mr->refs++; + mnt_check_refs(mp); + return; + } + } + panic("mount_refs overflow"); +} + +void +mnt_record_unref(struct mount *mp, const char *file, int line) +{ + struct mount_refs *mr; + int i; + + mtx_assert(MNT_MTX(mp), MA_OWNED); + for (i = 0; i < nitems(mp->mnt_unrefs); i++) { + mr = &mp->mnt_unrefs[i]; + if (mr->file != NULL && strcmp(mr->file, file) == 0 && + mr->line == line) { + mr->refs++; + mnt_check_refs(mp); + return; + } + } + for (i = 0; i < nitems(mp->mnt_unrefs); i++) { + mr = &mp->mnt_unrefs[i]; + if (mr->file == NULL) { + mr->file = file; + mr->line = line; + mr->refs++; + mnt_check_refs(mp); + return; + } + } + panic("mount_unrefs overflow"); +} + /* * Allocate and initialize the mount point struct. */ @@ -461,6 +543,8 @@ vfs_mount_alloc(struct vnode *vp, struct vfsconf * mp = uma_zalloc(mount_zone, M_WAITOK); bzero(&mp->mnt_startzero, __rangeof(struct mount, mnt_startzero, mnt_endzero)); +bzero(&mp->mnt_refs, sizeof(mp->mnt_refs)); +bzero(&mp->mnt_unrefs, sizeof(mp->mnt_unrefs)); TAILQ_INIT(&mp->mnt_nvnodelist); mp->mnt_nvnodelistsize = 0; TAILQ_INIT(&mp->mnt_activevnodelist); Index: /usr/src/sys/sys/mount.h =================================================================== --- /usr/src/sys/sys/mount.h (revision 319211) +++ /usr/src/sys/sys/mount.h (working copy) @@ -183,6 +183,11 @@ struct vfsopt { * Unmarked fields are considered stable as long as a ref is held. * */ +struct mount_refs { + int refs; + const char *file; + int line; +}; struct mount { struct mtx mnt_mtx; /* mount structure interlock */ int mnt_gen; /* struct mount generation */ @@ -223,6 +228,8 @@ struct mount { struct lock mnt_explock; /* vfs_export walkers lock */ TAILQ_ENTRY(mount) mnt_upper_link; /* (m) we in the all uppers */ TAILQ_HEAD(, mount) mnt_uppers; /* (m) upper mounts over us*/ +struct mount_refs mnt_refs[64]; +struct mount_refs mnt_unrefs[64]; }; /* @@ -258,14 +265,17 @@ void __mnt_vnode_markerfree_active(struct #define MNT_VNODE_FOREACH_ACTIVE_ABORT(mp, mvp) \ __mnt_vnode_markerfree_active(&(mvp), (mp)) +void mnt_record_ref(struct mount *mp, const char *file, int line); +void mnt_record_unref(struct mount *mp, const char *file, int line); #define MNT_ILOCK(mp) mtx_lock(&(mp)->mnt_mtx) #define MNT_ITRYLOCK(mp) mtx_trylock(&(mp)->mnt_mtx) #define MNT_IUNLOCK(mp) mtx_unlock(&(mp)->mnt_mtx) #define MNT_MTX(mp) (&(mp)->mnt_mtx) -#define MNT_REF(mp) (mp)->mnt_ref++ +#define MNT_REF(mp) do { (mp)->mnt_ref++; mnt_record_ref((mp), __FILE__, __LINE__);} while (0) #define MNT_REL(mp) do { \ KASSERT((mp)->mnt_ref > 0, ("negative mnt_ref")); \ (mp)->mnt_ref--; \ + mnt_record_unref((mp), __FILE__, __LINE__); \ if ((mp)->mnt_ref == 0) \ wakeup((mp)); \ } while (0) @@ -920,8 +930,10 @@ void vfs_mountedfrom(struct mount *, const char *f void vfs_notify_upper(struct vnode *, int); void vfs_oexport_conv(const struct oexport_args *oexp, struct export_args *exp); -void vfs_ref(struct mount *); -void vfs_rel(struct mount *); +void vfs_ref1(struct mount *, const char *, int); +#define vfs_ref(mp) vfs_ref1((mp), __FILE__, __LINE__) +void vfs_rel1(struct mount *, const char *, int); +#define vfs_rel(mp) vfs_rel1((mp), __FILE__, __LINE__) struct mount *vfs_mount_alloc(struct vnode *, struct vfsconf *, const char *, struct ucred *); int vfs_suser(struct mount *, struct thread *); Index: /usr/src/sys/ufs/ffs/ffs_softdep.c =================================================================== --- /usr/src/sys/ufs/ffs/ffs_softdep.c (revision 319211) +++ /usr/src/sys/ufs/ffs/ffs_softdep.c (working copy) @@ -13253,6 +13253,35 @@ softdep_slowdown(vp) return (1); } +static void +softdep_request_cleanup_flush(mp, ump) + struct mount *mp; + struct ufsmount *ump; +{ + struct vnode *lvp, *mvp; + + MNT_VNODE_FOREACH_ALL(lvp, mp, mvp) { + if (TAILQ_FIRST(&lvp->v_bufobj.bo_dirty.bv_hd) == 0) { + VI_UNLOCK(lvp); + continue; + } + if (vget(lvp, LK_EXCLUSIVE | LK_INTERLOCK | LK_NOWAIT, + curthread)) + continue; + if (lvp->v_vflag & VV_NOSYNC) { /* unlinked */ + vput(lvp); + continue; + } + (void) ffs_syncvnode(lvp, MNT_NOWAIT, 0); + vput(lvp); + } + lvp = ump->um_devvp; + if (vn_lock(lvp, LK_EXCLUSIVE | LK_NOWAIT) == 0) { + VOP_FSYNC(lvp, MNT_NOWAIT, curthread); + VOP_UNLOCK(lvp, 0); + } +} + /* * Called by the allocation routines when they are about to fail * in the hope that we can free up the requested resource (inodes @@ -13274,7 +13303,6 @@ softdep_request_cleanup(fs, vp, cred, resource) { struct ufsmount *ump; struct mount *mp; - struct vnode *lvp, *mvp; long starttime; ufs2_daddr_t needed; int error; @@ -13369,40 +13397,33 @@ retry: * resources. We walk the vnodes associated with the mount point * until we get the needed worklist requests that we can reap. */ + error = 1; if ((resource == FLUSH_BLOCKS_WAIT && fs->fs_cstotal.cs_nbfree <= needed) || (resource == FLUSH_INODES_WAIT && fs->fs_pendinginodes > 0 && fs->fs_cstotal.cs_nifree <= needed)) { - MNT_VNODE_FOREACH_ALL(lvp, mp, mvp) { - if (TAILQ_FIRST(&lvp->v_bufobj.bo_dirty.bv_hd) == 0) { - VI_UNLOCK(lvp); - continue; + ACQUIRE_LOCK(ump); + if ((ump->um_softdep->sd_flags & FLUSH_RC_ACTIVE) == 0) { + ump->um_softdep->sd_flags |= FLUSH_RC_ACTIVE; + FREE_LOCK(ump); + softdep_request_cleanup_flush(mp, ump); + ACQUIRE_LOCK(ump); + ump->um_softdep->sd_flags &= ~FLUSH_RC_ACTIVE; + FREE_LOCK(ump); + if (ump->softdep_on_worklist > 0) { + stat_cleanup_retries += 1; + goto retry; } - if (vget(lvp, LK_EXCLUSIVE | LK_INTERLOCK | LK_NOWAIT, - curthread)) - continue; - if (lvp->v_vflag & VV_NOSYNC) { /* unlinked */ - vput(lvp); - continue; - } - (void) ffs_syncvnode(lvp, MNT_NOWAIT, 0); - vput(lvp); + } else { + FREE_LOCK(ump); + error = 0; } - lvp = ump->um_devvp; - if (vn_lock(lvp, LK_EXCLUSIVE | LK_NOWAIT) == 0) { - VOP_FSYNC(lvp, MNT_NOWAIT, curthread); - VOP_UNLOCK(lvp, 0); - } - if (ump->softdep_on_worklist > 0) { - stat_cleanup_retries += 1; - goto retry; - } stat_cleanup_failures += 1; } if (time_second - starttime > stat_cleanup_high_delay) stat_cleanup_high_delay = time_second - starttime; UFS_LOCK(ump); - return (1); + return (error); } static bool Index: /usr/src/sys/ufs/ffs/softdep.h =================================================================== --- /usr/src/sys/ufs/ffs/softdep.h (revision 319211) +++ /usr/src/sys/ufs/ffs/softdep.h (working copy) @@ -1065,6 +1065,7 @@ struct mount_softdeps { #define FLUSH_EXIT 0x0001 /* time to exit */ #define FLUSH_CLEANUP 0x0002 /* need to clear out softdep structures */ #define FLUSH_STARTING 0x0004 /* flush thread not yet started */ +#define FLUSH_RC_ACTIVE 0x0008 /* * Keep the old names from when these were in the ufsmount structure. -- Test scenario: misc/crossmp11.sh