GENERIC HEAD from 2016-11-28 17:55:32 UTC, r309257M, vmcore.982 GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2016 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 12.0-CURRENT #0 r309257M: Tue Nov 29 17:23:37 CET 2016 pho@x4.osted.lan:/usr/src/sys/i386/compile/PHO i386 FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on LLVM 3.8.0) WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. VT(vga): resolution 640x480 CPU: AMD Phenom(tm) 9150e Quad-Core Processor (1800.02-MHz 686-class CPU) Origin="AuthenticAMD" Id=0x100f23 Family=0x10 Model=0x2 Stepping=3 Features=0x178bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x7ff SVM: NP,NAsids=64 TSC: P-state invariant real memory = 8589934592 (8192 MB) avail memory = 3524435968 (3361 MB) : Trying to mount root from ufs:/dev/ada0s1a [rw]... Setting hostuuid: 00000000-0000-0000-0000-00218515337d. Setting hostid: 0x6b64ac17. warning: total configured swap (4194304 pages) exceeds maximum recommended amount (2097312 pages). warning: increase kern.maxswzone or reduce amount of swap. Starting file system checks: /dev/ada0s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ada0s1a: clean, 633335 free (2687 frags, 78831 blocks, 0.3% fragmentation) /dev/ada0s1f: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ada0s1f: clean, 846290 free (10658 frags, 104454 blocks, 1.1% fragmentation) /dev/label/tmp: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/label/tmp: clean, 54188974 free (22182 frags, 6770849 blocks, 0.0% fragmentation) /dev/ada0s1d: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ada0s1d: clean, 2298561 free (296089 frags, 250309 blocks, 2.9% fragmentation) /dev/ada0s1e: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ada0s1e: clean, 3834098 free (228866 frags, 450654 blocks, 0.9% fragmentation) Mounting local file systems:. ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/arduino/lib /usr/local/lib/compat /usr/local/lib/compat/pkg /usr/local/lib/gcc48 /usr/local/lib/mysql /usr/local/lib/perl5/5.20/mach/CORE /usr/local/lib/pth /usr/local/lib/qt4 /usr/local/llvm37/lib a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout Setting hostname: x4.osted.lan. Setting up harvesting:[UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,NET_ETHER,NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED Feeding entropy:uhub5: 10 ports with 10 removable, self powered . re0: link state changed to DOWN Starting Network: lo0 re0. lo0: flags=8049 metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 nd6 options=21 groups: lo re0: flags=8843 metric 0 mtu 1500 options=8209b ether 00:21:85:15:33:7d inet 192.168.1.101 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::221:85ff:fe15:337d%re0 prefixlen 64 scopeid 0x1 nd6 options=23 media: Ethernet autoselect (none) status: no carrier Starting devd. re0: link state changed to UP add net default: gateway 192.168.1.1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Starting rtsold. Creating and/or trimming log files. Starting syslogd. Starting rpcbind. NFS access cache time=60 Starting watchdogd. Starting ddb. No core dumps found. Additional ABI support: linux. lock order reversal: 1st 0xead077c0 bufwait (bufwait) @ kern/vfs_bio.c:3512 2nd 0xc9711e00 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:281 stack backtrace: #0 0xc0cb6f71 at witness_debugger+0x81 #1 0xc0cb6e96 at witness_checkorder+0xd46 #2 0xc0c5ccc0 at _sx_xlock+0x70 #3 0xc0f4d083 at ufsdirhash_add+0x53 #4 0xc0f51b27 at ufs_direnter+0x4b7 #5 0xc0f5aad9 at ufs_mkdir+0x809 #6 0xc1221789 at VOP_MKDIR_APV+0xf9 #7 0xc0d2d44c at kern_mkdirat+0x22c #8 0xc0d2d211 at sys_mkdir+0x31 #9 0xc11ef7e3 at syscall+0x3d3 #10 0xc11da4be at Xint0x80_syscall+0x2e Clearing /tmp (X related). Starting nfsuserd. Starting mountd. Starting nfsd. Starting statd. Starting lockd. Recovering vi editor sessions:lock order reversal: 1st 0xca16f73c ufs (ufs) @ kern/vfs_lookup.c:616 2nd 0xead2c660 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:277 3rd 0xca1d8a0c ufs (ufs) @ kern/vfs_subr.c:2592 stack backtrace: #0 0xc0cb6f71 at witness_debugger+0x81 #1 0xc0cb6e96 at witness_checkorder+0xd46 #2 0xc0c29832 at __lockmgr_args+0x482 #3 0xc0f47328 at ffs_lock+0x98 #4 0xc1222520 at VOP_LOCK1_APV+0xe0 #5 0xc0d306ba at _vn_lock+0xba #6 0xc0d1e3ee at vget+0x8e #7 0xc0d0fe41 at vfs_hash_get+0xd1 #8 0xc0f424f7 at ffs_vgetf+0x47 #9 0xc0f38891 at softdep_sync_buf+0xb11 #10 0xc0f4815f at ffs_syncvnode+0x2ef #11 0xc0f1c895 at ffs_truncate+0x8e5 #12 0xc0f51ce5 at ufs_direnter+0x675 #13 0xc0f5b91a at ufs_makeinode+0x55a #14 0xc0f575e8 at ufs_create+0x38 #15 0xc121f0e9 at VOP_CREATE_APV+0xf9 #16 0xc0d2feb5 at vn_open_cred+0x2a5 #17 0xc0d2fbfd at vn_open+0x3d . Updating motd:. Mounting late file systems:. Starting ntpd. Starting powerd. Starting smartd. Nov 29 20:12:27 x4 smartd[664]: Device: /dev/ada0, WARNING: A firmware update for this drive may be available, Nov 29 20:12:27 x4 smartd[664]: see the following Seagate web pages: Nov 29 20:12:27 x4 smartd[664]: http://knowledge.seagate.com/articles/en_US/FAQ/207931en Nov 29 20:12:27 x4 smartd[664]: http://knowledge.seagate.com/articles/en_US/FAQ/213891en Configuring vt: keymap blanktime. Performing sanity check on sshd configuration. Starting sshd. Starting sendmail_submit. Starting sendmail_msp_queue. Starting cron. Local package initialization: backuplock order reversal: 1st 0xc96668a4 ufs (ufs) @ kern/vfs_mount.c:1276 2nd 0xc9666cdc devfs (devfs) @ ufs/ffs/ffs_softdep.c:1908 stack backtrace: #0 0xc0cb6f71 at witness_debugger+0x81 #1 0xc0cb6e96 at witness_checkorder+0xd46 #2 0xc0c29832 at __lockmgr_args+0x482 #3 0xc0d0bcf3 at vop_stdlock+0x53 #4 0xc1222520 at VOP_LOCK1_APV+0xe0 #5 0xc0d306ba at _vn_lock+0xba #6 0xc0f262a8 at softdep_flushworklist+0x68 #7 0xc0f4584a at ffs_sync+0x2aa #8 0xc0d31e05 at vfs_write_suspend+0x1c5 #9 0xc0d32035 at vfs_write_suspend_umnt+0x45 #10 0xc0f44fbf at ffs_unmount+0xaf #11 0xc0d1637f at dounmount+0x73f #12 0xc0d15b35 at sys_unmount+0x315 #13 0xc11ef7e3 at syscall+0x3d3 #14 0xc11da4be at Xint0x80_syscall+0x2e ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes Nov 29 20:13:22 x4 rpc.statd: Failed to contact host t1.osted.lan: RPC: Port mapper failure - RPC: Timed out Nov 29 20:14:02 x4 su: pho to root on /dev/pts/1 Nov 29 20:14:28 x4 rpc.statd: Failed to contact host t1.osted.lan: RPC: Port mapper failure - RPC: Timed out ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 548570 files, 64125007 used, 54188974 free (22182 frags, 6770849 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** fsck -y /tmp. Starting default moused. Starting inetd. Tue Nov 29 20:15 FreeBSD/i386 (x4.osted.lan) (ttyu0) login: Nov 29 20:15:33 x4 rpc.statd: Failed to contact host t1.osted.lan: RPC: Port mapper failure - RPC: Timed out 20161129 20:15:45 all (1/150): 256m.sh 20161129 20:15:47 all (2/150): advlock.sh 20161129 20:16:53 all (3/150): alternativeFlushPath.sh 20161129 20:37:15 all (4/150): audit.sh 20161129 20:37:18 all (5/150): bio.sh lock order reversal: 1st 0xcbb2319c syncer (syncer) @ kern/vfs_subr.c:2142 2nd 0xcc425a0c ufs (ufs) @ kern/vfs_subr.c:2592 stack backtrace: #0 0xc0cb6f71 at witness_debugger+0x81 #1 0xc0cb6e96 at witness_checkorder+0xd46 #2 0xc0c29832 at __lockmgr_args+0x482 #3 0xc0f47328 at ffs_lock+0x98 #4 0xc1222520 at VOP_LOCK1_APV+0xe0 #5 0xc0d306ba at _vn_lock+0xba #6 0xc0d1e3ee at vget+0x8e #7 0xc0d2084a at vfs_msync+0xba #8 0xc0d25922 at sync_fsync+0xe2 #9 0xc1220fb9 at VOP_FSYNC_APV+0xf9 #10 0xc0d23825 at sched_sync+0x2a5 #11 0xc0c150ae at fork_exit+0x7e #12 0xc11da4d0 at fork_trampoline+0x8 20161129 20:48:52 all (6/150): buildworld.sh 20161129 21:09:27 all (7/150): buildworld2.sh 20161129 21:09:30 all (8/150): burnin.sh 20161129 21:30:16 all (9/150): callout_reset_on.sh Expensive timeout(9) function: 0xc0c6a020(0xc97de9d8) 0.010532551 s Expensive timeout(9) function: 0xc0eadb20(0) 0.014177306 s 20161129 21:37:50 all (10/150): callout_reset_on2.sh 20161129 21:47:55 all (11/150): churn.sh 20161129 21:57:59 all (12/150): cluster.sh 20161129 22:10:49 all (13/150): cmp.sh 20161129 22:13:51 all (14/150): context.sh 20161129 22:18:58 all (15/150): context2.sh 20161129 22:24:04 all (16/150): contigmalloc.sh 20161129 22:26:24 all (17/150): contigmalloc2.sh 20161129 22:26:27 all (18/150): core.sh 20161129 22:30:03 all (19/150): core2.sh 20161129 22:32:16 all (20/150): core3.sh Nov 29 22:32:18 x4 kernel: Failed to write core file for process core3 (error 5) Nov 29 22:32:49 x4 last message repeated 6747 times Nov 29 22:34:25 x4 last message repeated 20971 times Nov 29 22:34:25 x4 mountd[564]: bad exports list line /mnt Nov 29 22:34:25 x4 kernel: Failed to write core file for process core3 (error 5) Nov 29 22:34:51 x4 last message repeated 5496 times Nov 29 22:34:51 x4 mountd[564]: bad exports list line /mnt Nov 29 22:34:51 x4 kernel: Failed to write core file for process core3 (error 5) Nov 29 22:34:59 x4 last message repeated 1839 times Nov 29 22:34:59 x4 mountd[564]: bad exports list line /mnt Nov 29 22:34:59 x4 kernel: Failed to write core file for process core3 (error 5) Nov 29 22:35:30 x4 last message repeated 5777 times Nov 29 22:37:31 x4 last message repeated 30821 times Nov 29 22:42:17 x4 last message repeated 70963 times 20161129 22:52:19 all (21/150): core4.sh Nov 29 22:52:21 x4 kernel: Failed to write core file for process core4 (error 5) 20161129 22:52:41 all (22/150): core5.sh Nov 29 22:52:39 x4 last message repeated 59 times lock order reversal: 1st 0xc9bcc3a0 filedesc structure (filedesc structure) @ kern/kern_descrip.c:840 2nd 0xc9bed19c ufs (ufs) @ kern/vfs_subr.c:4991 stack backtrace: #0 0xc0cb6f71 at witness_debugger+0x81 #1 0xc0cb6e96 at witness_checkorder+0xd46 #2 0xc0c29832 at __lockmgr_args+0x482 #3 0xc0f47328 at ffs_lock+0x98 #4 0xc1222520 at VOP_LOCK1_APV+0xe0 #5 0xc0d306ba at _vn_lock+0xba #6 0xc0d20cd9 at vfs_knllock+0x29 #7 0xc0c0a516 at knlist_remove_kq+0x86 #8 0xc0c0a47f at knlist_remove+0x1f #9 0xc0d25c9f at filt_vfsdetach+0x2f #10 0xc0c0b02e at knote_fdclose+0xde #11 0xc0bfd978 at closefp+0x68 #12 0xc0bfbe10 at kern_dup+0x5d0 #13 0xc0bfb831 at sys_dup2+0x31 #14 0xc11ef7e3 at syscall+0x3d3 #15 0xc11da4be at Xint0x80_syscall+0x2e 20161129 23:02:45 all (23/150): credleak.sh 20161129 23:03:43 all (24/150): crossmp.sh 20161129 23:28:32 all (25/150): crossmp10.sh 20161129 23:28:34 all (26/150): crossmp2.sh 20161129 23:30:55 all (27/150): crossmp3.sh 20161129 23:37:26 all (28/150): crossmp5.sh 20161129 23:38:45 all (29/150): crossmp6.sh 20161129 23:41:41 all (30/150): crossmp7.sh 20161129 23:41:43 all (31/150): crossmp8.sh 20161129 23:42:59 all (32/150): crossmp9.sh 20161129 23:48:04 all (33/150): dangling.sh 20161129 23:58:56 all (34/150): datamove.sh 20161129 23:59:02 all (35/150): datamove2.sh 20161130 00:09:22 all (36/150): datamove3.sh 20161130 00:09:25 all (37/150): datamove4.sh 20161130 00:09:35 all (38/150): datamove5.sh 20161130 00:09:52 all (39/150): db.sh 20161130 00:12:55 all (40/150): dd.sh 20161130 00:15:49 all (41/150): dev.sh /tmp grew 39 20161130 00:20:56 all (42/150): dev2.sh 20161130 00:26:00 all (43/150): devfd.sh 20161130 00:26:14 all (44/150): devfs.sh 20161130 00:31:18 all (45/150): devfs2.sh 20161130 00:31:22 all (46/150): devfs3.sh 20161130 00:36:26 all (47/150): dfull.sh Nov 30 00:36:31 x4 kernel: pid 91349 (rw), uid 0 inumber 49155 on /mnt: filesystem full Nov 30 00:36:31 x4 kernel: pid 91352 (creat), uid 0 inumber 16387 on /mnt: filesystem full Nov 30 00:38:32 x4 kernel: pid 96230 (rw), uid 0 inumber 32771 on /mnt: filesystem full Nov 30 00:38:34 x4 kernel: pid 96236 (rw), uid 0 inumber 49156 on /mnt: filesystem full Nov 30 00:39:34 x4 kernel: pid 99291 (rw), uid 0 inumber 33271 on /mnt: filesystem full Nov 30 00:40:49 x4 kernel: pid 1279 (rw), uid 0 inumber 5 on /mnt: filesystem full Nov 30 00:41:52 x4 kernel: pid 3755 (rw), uid 0 inumber 309 on /mnt: filesystem full Nov 30 00:43:03 x4 kernel: pid 5519 (rw), uid 0 inumber 39 on /mnt: filesystem full Nov 30 00:44:24 x4 kernel: pid 8614 (rw), uid 0 inumber 49197 on /mnt: filesystem full Nov 30 00:45:08 x4 kernel: pid 8634 (rw), uid 0 inumber 344 on /mnt: filesystem full Nov 30 00:46:12 x4 kernel: pid 12428 (rw), uid 0 inumber 16413 on /mnt: filesystem full Nov 30 00:46:12 x4 kernel: pid 12427 (mkdir), uid 0 inumber 16432 on /mnt: filesystem full Nov 30 00:46:12 x4 kernel: pid 12432 (creat), uid 0 inumber 32768 on /mnt: filesystem full Nov 30 00:47:13 x4 kernel: pid 14942 (rw), uid 0 inumber 32819 on /mnt: filesystem full Nov 30 00:49:26 x4 kernel: pid 19379 (rw), uid 0 inumber 81 on /mnt: filesystem full Nov 30 00:49:26 x4 kernel: pid 19383 (mkdir), uid 0 inumber 16544 on /mnt: filesystem full Nov 30 00:50:29 x4 kernel: pid 21666 (rw), uid 0 inumber 90 on /mnt: filesystem full Nov 30 00:50:29 x4 kernel: pid 21680 (mkdir), uid 0 inumber 32802 on /mnt: filesystem full Nov 30 00:51:29 x4 kernel: pid 24123 (rw), uid 0 inumber 49295 on /mnt: filesystem full Nov 30 00:51:30 x4 kernel: pid 24127 (rw), uid 0 inumber 32829 on /mnt: filesystem full Nov 30 00:51:33 x4 kernel: pid 24157 (creat), uid 0 inumber 50030 on /mnt: filesystem full Nov 30 00:51:33 x4 kernel: pid 24121 (rw), uid 0 inumber 32833 on /mnt: filesystem full Nov 30 00:51:33 x4 kernel: pid 24135 (mkdir), uid 0 inumber 33348 on /mnt: filesystem full Nov 30 00:51:59 x4 kernel: pid 24143 (rw), uid 0 inumber 16806 on /mnt: filesystem full Nov 30 00:52:01 x4 kernel: pid 24145 (rw), uid 0 inumber 50078 on /mnt: filesystem full Nov 30 00:52:08 x4 kernel: pid 24147 (rw), uid 0 inumber 16808 on /mnt: filesystem full Nov 30 00:52:08 x4 kernel: pid 24130 (creat), uid 0 inumber 16695 on /mnt: filesystem full Nov 30 00:52:30 x4 kernel: pid 26129 (rw), uid 0 inumber 49227 on /mnt: filesystem full Nov 30 00:53:31 x4 kernel: pid 28447 (creat), uid 0 inumber 33454 on /mnt: filesystem full Nov 30 00:53:31 x4 kernel: pid 28437 (rw), uid 0 inumber 33362 on /mnt: filesystem full Nov 30 00:53:50 x4 kernel: pid 28448 (creat), uid 0 inumber 19173 on /mnt: filesystem full Nov 30 00:53:50 x4 kernel: pid 28441 (rw), uid 0 inumber 33085 on /mnt: filesystem full Nov 30 00:54:32 x4 kernel: pid 30888 (rw), uid 0 inumber 49298 on /mnt: filesystem full Nov 30 00:55:40 x4 kernel: pid 32196 (creat), uid 0 inumber 16536 on /mnt: filesystem full Nov 30 00:55:40 x4 kernel: pid 32199 (rw), uid 0 inumber 16521 on /mnt: filesystem full Nov 30 00:55:40 x4 kernel: pid 32203 (mkdir), uid 0 inumber 17205 on /mnt: filesystem full Nov 30 00:56:39 x4 kernel: pid 32178 (rw), uid 0 inumber 34285 on /mnt: filesystem full Nov 30 00:56:39 x4 kernel: pid 32181 (creat), uid 0 inumber 33039 on /mnt: filesystem full Nov 30 00:56:41 x4 kernel: pid 34036 (rw), uid 0 inumber 16398 on /mnt: filesystem full Nov 30 00:56:42 x4 kernel: pid 34041 (rw), uid 0 inumber 16545 on /mnt: filesystem full Nov 30 00:57:41 x4 kernel: pid 37104 (rw), uid 0 inumber 32806 on /mnt: filesystem full Nov 30 00:58:42 x4 kernel: pid 39550 (rw), uid 0 inumber 32819 on /mnt: filesystem full Nov 30 00:58:44 x4 kernel: pid 39552 (rw), uid 0 inumber 32808 on /mnt: filesystem full Nov 30 00:59:48 x4 kernel: pid 41996 (rw), uid 0 inumber 16540 on /mnt: filesystem full Nov 30 01:00:49 x4 kernel: pid 43851 (rw), uid 0 inumber 49225 on /mnt: filesystem full Nov 30 01:01:59 x4 kernel: pid 45135 (rw), uid 0 inumber 16555 on /mnt: filesystem full Nov 30 01:02:02 x4 kernel: pid 45134 (rw), uid 0 inumber 16536 on /mnt: filesystem full Nov 30 01:02:02 x4 kernel: pid 45156 (creat), uid 0 inumber 49284 on /mnt: filesystem full Nov 30 01:02:18 x4 kernel: pid 45128 (rw), uid 0 inumber 16561 on /mnt: filesystem full Nov 30 01:02:21 x4 kernel: pid 45138 (rw), uid 0 inumber 16559 on /mnt: filesystem full Nov 30 01:02:57 x4 kernel: pid 47592 (rw), uid 0 inumber 49225 on /mnt: filesystem full Nov 30 01:02:58 x4 kernel: pid 47596 (rw), uid 0 inumber 49227 on /mnt: filesystem full Nov 30 01:04:02 x4 kernel: pid 49633 (rw), uid 0 inumber 32811 on /mnt: filesystem full Nov 30 01:05:06 x4 kernel: pid 51317 (rw), uid 0 inumber 33195 on /mnt: filesystem full 20161130 01:07:06 all (48/150): dtrace.sh 20161130 01:13:42 all (49/150): dup2.sh 20161130 01:13:48 all (50/150): execi386.sh 20161130 01:13:50 all (51/150): execve.sh 20161130 01:14:25 all (52/150): exlock.sh 20161130 01:18:06 all (53/150): ext2fs.sh 20161130 01:18:08 all (54/150): ext2fs2.sh 20161130 01:18:11 all (55/150): extattr_set_fd.sh 20161130 01:18:14 all (56/150): extattr.sh 20161130 01:18:22 all (57/150): extattrctl.sh lock order reversal: 1st 0xd04ba21c ufs_extattr_sx (ufs_extattr_sx) @ ufs/ufs/ufs_extattr.c:116 2nd 0xd088fe44 ufs (ufs) @ ufs/ffs/ffs_vfsops.c:1710 stack backtrace: #0 0xc0cb6f71 at witness_debugger+0x81 #1 0xc0cb6e96 at witness_checkorder+0xd46 #2 0xc0c29832 at __lockmgr_args+0x482 #3 0xc0f42602 at ffs_vgetf+0x152 #4 0xc0f4249f at ffs_vget+0x2f #5 0xc0f56bac at ufs_root+0x4c #6 0xc0f4e071 at ufs_extattr_autostart+0x81 #7 0xc0f44e45 at ffs_mount+0x2075 #8 0xc0d146d0 at vfs_donmount+0xf80 #9 0xc0d1372a at sys_nmount+0x7a #10 0xc11ef7e3 at syscall+0x3d3 #11 0xc11da4be at Xint0x80_syscall+0x2e 20161130 01:18:25 all (58/150): f_offset.sh 20161130 01:21:09 all (59/150): fcntl.sh 20161130 01:21:16 all (60/150): fdatasync.sh 20161130 01:24:21 all (61/150): fdescfs.sh 20161130 01:29:25 all (62/150): fdgrowtable.sh 20161130 01:33:41 all (63/150): ffs_blkfree.sh lock order reversal: 1st 0xeaedb100 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:277 2nd 0xc961401c snaplk (snaplk) @ ufs/ffs/ffs_snapshot.c:2298 stack backtrace: #0 0xc0cb6f71 at witness_debugger+0x81 #1 0xc0cb6e96 at witness_checkorder+0xd46 #2 0xc0c29832 at __lockmgr_args+0x482 #3 0xc0f24fdf at ffs_copyonwrite+0x1cf #4 0xc0f46175 at ffs_geom_strategy+0x85 #5 0xc0f5b124 at ufs_strategy+0xb4 #6 0xc1222a6a at VOP_STRATEGY_APV+0x10a #7 0xc0cfa22d at bufstrategy+0x4d #8 0xc0cf9fdd at bufwrite+0x1cd #9 0xc0f48215 at ffs_syncvnode+0x3a5 #10 0xc0f471a6 at ffs_fsync+0x26 #11 0xc1220fb9 at VOP_FSYNC_APV+0xf9 #12 0xc0f58baf at ufs_remove+0xdf #13 0xc12211b2 at VOP_REMOVE_APV+0x122 #14 0xc0d299c6 at kern_unlinkat+0x266 #15 0xc0d29752 at sys_unlink+0x32 #16 0xc11ef7e3 at syscall+0x3d3 #17 0xc11da4be at Xint0x80_syscall+0x2e lock order reversal: 1st 0xc961401c snaplk (snaplk) @ kern/vfs_vnops.c:442 2nd 0xcc87173c ufs (ufs) @ ufs/ffs/ffs_snapshot.c:1629 stack backtrace: #0 0xc0cb6f71 at witness_debugger+0x81 #1 0xc0cb6e96 at witness_checkorder+0xd46 #2 0xc0c29832 at __lockmgr_args+0x482 #3 0xc0f230d6 at ffs_snapremove+0x116 #4 0xc0f1c269 at ffs_truncate+0x2b9 #5 0xc0f5001d at ufs_inactive+0x1ed #6 0xc1222189 at VOP_INACTIVE_APV+0xf9 #7 0xc0d1e865 at vinactive+0x115 #8 0xc0d1ee54 at vputx+0x364 #9 0xc0d3082c at vn_close+0xfc #10 0xc0d2f3ef at vn_closefile+0x4f #11 0xc0bfd2d1 at _fdrop+0x21 #12 0xc0c0061e at closef+0x21e #13 0xc0c000de at fdescfree_fds+0x7e #14 0xc0bffc8b at fdescfree+0x50b #15 0xc0c0fc41 at exit1+0x5e1 #16 0xc0c0f652 at sys_sys_exit+0x22 #17 0xc11ef7e3 at syscall+0x3d3 20161130 01:34:01 all (64/150): ffs_sync.sh 20161130 01:37:34 all (65/150): ffs_syncvnode.sh Nov 30 01:44:09 x4 kernel: pid 3442 (rw), uid 1004 inumber 196610 on /mnt: filesystem full Nov 30 01:45:02 x4 kernel: pid 3444 (rw), uid 1004 inumber 142000 on /mnt: filesystem full Nov 30 01:45:50 x4 kernel: pid 9290 (rw), uid 1004 inumber 65547 on /mnt: filesystem full 20161130 02:08:45 all (66/150): ffs_syncvnode2.sh Nov 30 02:11:26 x4 kernel: pid 47427 (rw), uid 1004 inumber 131091 on /mnt: filesystem full Nov 30 02:11:29 x4 kernel: pid 47436 (rw), uid 1004 inumber 131124 on /mnt: filesystem full Nov 30 02:13:36 x4 kernel: pid 52390 (rw), uid 1004 inumber 131157 on /mnt: filesystem full Nov 30 02:13:39 x4 kernel: pid 52395 (rw), uid 1004 inumber 132358 on /mnt: filesystem full Nov 30 02:13:39 x4 kernel: pid 52407 (mkdir), uid 1004 inumber 78 on /mnt: filesystem full Nov 30 02:13:49 x4 kernel: pid 52398 (rw), uid 1004 inumber 132352 on /mnt: filesystem full Nov 30 02:13:49 x4 kernel: pid 52405 (mkdir), uid 1004 inumber 4 on /mnt: filesystem full Nov 30 02:15:13 x4 kernel: pid 57367 (rw), uid 1004 inumber 131126 on /mnt: filesystem full Nov 30 02:15:17 x4 kernel: pid 57360 (rw), uid 1004 inumber 131167 on /mnt: filesystem full Nov 30 02:15:17 x4 kernel: pid 57381 (creat), uid 1004 inumber 131172 on /mnt: filesystem full Nov 30 02:25:18 x4 kernel: pid 83411 (rw), uid 1004 inumber 196642 on /mnt: filesystem full Nov 30 02:25:30 x4 kernel: pid 83427 (mkdir), uid 1004 inumber 131142 on /mnt: filesystem full Nov 30 02:25:30 x4 kernel: pid 83405 (rw), uid 1004 inumber 196668 on /mnt: filesystem full Nov 30 02:27:40 x4 kernel: pid 88403 (rw), uid 1004 inumber 196668 on /mnt: filesystem full Nov 30 02:27:42 x4 kernel: pid 88396 (rw), uid 1004 inumber 196617 on /mnt: filesystem full Nov 30 02:29:36 x4 kernel: pid 94610 (rw), uid 1004 inumber 196621 on /mnt: filesystem full Nov 30 02:30:07 x4 kernel: pid 94614 (rw), uid 1004 inumber 196630 on /mnt: filesystem full Nov 30 02:30:07 x4 kernel: pid 94599 (mkdir), uid 1004 inumber 131655 on /mnt: filesystem full Nov 30 02:30:08 x4 kernel: pid 94592 (rw), uid 1004 inumber 196627 on /mnt: filesystem full Nov 30 02:30:08 x4 kernel: pid 94597 (creat), uid 1004 inumber 196700 on /mnt: filesystem full Nov 30 02:33:42 x4 kernel: pid 287 (rw), uid 1004 inumber 136931 on /mnt: filesystem full Nov 30 02:33:44 x4 kernel: pid 3451 (rw), uid 1004 inumber 196615 on /mnt: filesystem full Nov 30 02:34:07 x4 kernel: pid 3450 (rw), uid 1004 inumber 196621 on /mnt: filesystem full 20161130 02:39:53 all (67/150): fifo.sh 20161130 02:40:00 all (68/150): fifo2.sh 20161130 02:42:14 all (69/150): fifo3.sh 20161130 02:45:00 all (70/150): flock_open_close.sh 20161130 02:51:04 all (71/150): flock.sh 20161130 02:52:06 all (72/150): fork.sh 20161130 02:52:15 all (73/150): fork2.sh 20161130 02:55:33 all (74/150): forkbomb.sh witness_lock_list_get: witness exhausted maxproc limit exceeded by uid 1004 (pid 74818); see tuning(7) and login.conf(5) maxproc limit exceeded by uid 1004 (pid 76335); see tuning(7) and login.conf(5) 20161130 02:55:43 all (75/150): fpclone.sh Expensive timeout(9) function: 0xc0a7f5d0(0xc90e8aa0) 0.020935763 s 20161130 02:58:06 all (76/150): fpclone2.sh 20161130 02:58:11 all (77/150): fpu.sh 20161130 03:00:53 all (78/150): fragments.sh Nov 30 03:01:02 x4 kernel: pid 97035 (fragments), uid 1004 inumber 6334 on /mnt: filesystem full Nov 30 03:01:02 x4 kernel: pid 97035 (fragments), uid 1004 inumber 10 on /mnt: filesystem full Nov 30 03:01:03 x4 kernel: pid 97782 (fragments), uid 1004 inumber 6680 on /mnt: filesystem full Nov 30 03:01:04 x4 kernel: pid 97776 (fragments), uid 1004 inumber 6686 on /mnt: filesystem full Nov 30 03:01:05 x4 kernel: pid 97777 (fragments), uid 1004 inumber 6734 on /mnt: filesystem full Nov 30 03:01:06 x4 kernel: pid 97783 (fragments), uid 1004 inumber 6763 on /mnt: filesystem full Nov 30 03:01:07 x4 kernel: pid 97777 (fragments), uid 1004 inumber 6786 on /mnt: filesystem full Nov 30 03:01:07 x4 kernel: pid 97780 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:08 x4 kernel: pid 97779 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:09 x4 kernel: pid 97783 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:10 x4 kernel: pid 97783 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:11 x4 kernel: pid 97780 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:12 x4 kernel: pid 97781 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:13 x4 kernel: pid 97780 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:14 x4 kernel: pid 97781 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:15 x4 kernel: pid 97779 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:16 x4 kernel: pid 97780 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:17 x4 kernel: pid 97776 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:18 x4 kernel: pid 97782 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:19 x4 kernel: pid 97779 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:20 x4 kernel: pid 97780 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:21 x4 kernel: pid 97781 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:22 x4 kernel: pid 97781 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:23 x4 kernel: pid 97779 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:24 x4 kernel: pid 97783 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:25 x4 kernel: pid 97779 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:26 x4 kernel: pid 97783 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:27 x4 kernel: pid 97783 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:28 x4 kernel: pid 97782 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:29 x4 kernel: pid 97779 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:30 x4 kernel: pid 97782 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:31 x4 kernel: pid 97781 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:31 x4 kernel: pid 97776 (fragments), uid 1004 inumber 6852 on /mnt: filesystem full Nov 30 03:01:33 x4 kernel: pid 97778 (fragments), uid 1004 inumber 6913 on /mnt: filesystem full Nov 30 03:01:34 x4 kernel: pid 97779 (fragments), uid 1004 inumber 6910 on /mnt: filesystem full Nov 30 03:01:35 x4 kernel: pid 97778 (fragments), uid 1004 inumber 6608 on /mnt: filesystem full Nov 30 03:01:36 x4 kernel: pid 97778 (fragments), uid 1004 inumber 6699 on /mnt: filesystem full Nov 30 03:01:37 x4 kernel: pid 97778 (fragments), uid 1004 inumber 6775 on /mnt: filesystem full Nov 30 03:01:38 x4 kernel: pid 99635 (fragments), uid 1004 inumber 6624 on /mnt: filesystem full Nov 30 03:01:39 x4 kernel: pid 99629 (fragments), uid 1004 inumber 6630 on /mnt: filesystem full Nov 30 03:01:40 x4 kernel: pid 99631 (fragments), uid 1004 inumber 6734 on /mnt: filesystem full Nov 30 03:01:40 x4 kernel: pid 99634 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:41 x4 kernel: pid 99634 (fragments), uid 1004 inumber 6917 on /mnt: filesystem full Nov 30 03:01:42 x4 kernel: pid 99637 (fragments), uid 1004 inumber 6586 on /mnt: filesystem full Nov 30 03:01:43 x4 kernel: pid 99637 (fragments), uid 1004 inumber 6638 on /mnt: filesystem full Nov 30 03:01:44 x4 kernel: pid 99643 (fragments), uid 1004 inumber 6723 on /mnt: filesystem full Nov 30 03:01:45 x4 kernel: pid 99639 (fragments), uid 1004 inumber 6833 on /mnt: filesystem full Nov 30 03:01:46 x4 kernel: pid 99636 (fragments), uid 1004 inumber 6889 on /mnt: filesystem full Nov 30 03:01:47 x4 kernel: pid 99636 (fragments), uid 1004 inumber 6939 on /mnt: filesystem full Nov 30 03:01:48 x4 kernel: pid 99643 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:49 x4 kernel: pid 99641 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:50 x4 kernel: pid 99636 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:51 x4 kernel: pid 99636 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:52 x4 kernel: pid 99642 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:53 x4 kernel: pid 99636 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:54 x4 kernel: pid 99643 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:55 x4 kernel: pid 99641 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:01:56 x4 kernel: pid 99643 (fragments), uid 1004 inumber 7026 on /mnt: filesystem full Nov 30 03:01:57 x4 kernel: pid 1012 (fragments), uid 1004 inumber 6679 on /mnt: filesystem full Nov 30 03:01:58 x4 kernel: pid 1009 (fragments), uid 1004 inumber 6682 on /mnt: filesystem full Nov 30 03:01:59 x4 kernel: pid 1014 (fragments), uid 1004 inumber 6687 on /mnt: filesystem full Nov 30 03:02:00 x4 kernel: pid 1010 (fragments), uid 1004 inumber 6770 on /mnt: filesystem full Nov 30 03:02:01 x4 kernel: pid 1008 (fragments), uid 1004 inumber 6805 on /mnt: filesystem full Nov 30 03:02:02 x4 kernel: pid 1012 (fragments), uid 1004 inumber 6933 on /mnt: filesystem full Nov 30 03:02:03 x4 kernel: pid 1015 (fragments), uid 1004 inumber 6586 on /mnt: filesystem full Nov 30 03:02:04 x4 kernel: pid 1013 (fragments), uid 1004 inumber 6646 on /mnt: filesystem full Nov 30 03:02:05 x4 kernel: pid 1013 (fragments), uid 1004 inumber 6839 on /mnt: filesystem full Nov 30 03:02:06 x4 kernel: pid 1018 (fragments), uid 1004 inumber 6682 on /mnt: filesystem full Nov 30 03:02:07 x4 kernel: pid 1017 (fragments), uid 1004 inumber 6765 on /mnt: filesystem full Nov 30 03:02:07 x4 kernel: pid 1017 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:08 x4 kernel: pid 1018 (fragments), uid 1004 inumber 6858 on /mnt: filesystem full Nov 30 03:02:09 x4 kernel: pid 1017 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:09 x4 kernel: pid 1023 (fragments), uid 1004 inumber 7066 on /mnt: filesystem full Nov 30 03:02:10 x4 kernel: pid 1645 (fragments), uid 1004 inumber 6646 on /mnt: filesystem full Nov 30 03:02:11 x4 kernel: pid 1649 (fragments), uid 1004 inumber 6647 on /mnt: filesystem full Nov 30 03:02:12 x4 kernel: pid 1648 (fragments), uid 1004 inumber 6697 on /mnt: filesystem full Nov 30 03:02:13 x4 kernel: pid 1645 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:14 x4 kernel: pid 1644 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:15 x4 kernel: pid 1650 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:16 x4 kernel: pid 1648 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:17 x4 kernel: pid 1645 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:18 x4 kernel: pid 1647 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:19 x4 kernel: pid 1650 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:20 x4 kernel: pid 1648 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:21 x4 kernel: pid 1650 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:22 x4 kernel: pid 1650 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:23 x4 kernel: pid 1645 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:24 x4 kernel: pid 1645 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:25 x4 kernel: pid 1650 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:26 x4 kernel: pid 1649 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:27 x4 kernel: pid 1645 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:28 x4 kernel: pid 1647 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:29 x4 kernel: pid 1645 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:30 x4 kernel: pid 1650 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:31 x4 kernel: pid 1647 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:32 x4 kernel: pid 1649 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:33 x4 kernel: pid 1649 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:34 x4 kernel: pid 1650 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:35 x4 kernel: pid 1649 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:36 x4 kernel: pid 1645 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:37 x4 kernel: pid 1650 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:38 x4 kernel: pid 1646 (fragments), uid 1004 inumber 6828 on /mnt: filesystem full Nov 30 03:02:39 x4 kernel: pid 1646 (fragments), uid 1004 inumber 6633 on /mnt: filesystem full Nov 30 03:02:40 x4 kernel: pid 3503 (fragments), uid 1004 inumber 6671 on /mnt: filesystem full Nov 30 03:02:41 x4 kernel: pid 3503 (fragments), uid 1004 inumber 6710 on /mnt: filesystem full Nov 30 03:02:42 x4 kernel: pid 3497 (fragments), uid 1004 inumber 6793 on /mnt: filesystem full Nov 30 03:02:42 x4 kernel: pid 3500 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:43 x4 kernel: pid 3497 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:44 x4 kernel: pid 3502 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:45 x4 kernel: pid 3500 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:46 x4 kernel: pid 3502 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:47 x4 kernel: pid 3499 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:48 x4 kernel: pid 3496 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:49 x4 kernel: pid 3503 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:50 x4 kernel: pid 3501 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:52 x4 kernel: pid 3496 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:53 x4 kernel: pid 3499 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:54 x4 kernel: pid 3496 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:55 x4 kernel: pid 3497 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:56 x4 kernel: pid 3497 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:57 x4 kernel: pid 3502 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:58 x4 kernel: pid 3501 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:02:59 x4 kernel: pid 3501 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:00 x4 kernel: pid 3500 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:01 x4 kernel: pid 3499 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:02 x4 kernel: pid 3496 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:03 x4 kernel: pid 3501 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:04 x4 kernel: pid 3497 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:04 x4 kernel: pid 3503 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:05 x4 kernel: pid 3498 (fragments), uid 1004 inumber 6853 on /mnt: filesystem full Nov 30 03:03:06 x4 kernel: pid 3498 (fragments), uid 1004 inumber 6696 on /mnt: filesystem full Nov 30 03:03:07 x4 kernel: pid 4734 (fragments), uid 1004 inumber 6635 on /mnt: filesystem full Nov 30 03:03:08 x4 kernel: pid 4738 (fragments), uid 1004 inumber 6672 on /mnt: filesystem full Nov 30 03:03:09 x4 kernel: pid 4737 (fragments), uid 1004 inumber 6775 on /mnt: filesystem full Nov 30 03:03:09 x4 kernel: pid 4738 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:10 x4 kernel: pid 4736 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:11 x4 kernel: pid 4738 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:12 x4 kernel: pid 4735 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:13 x4 kernel: pid 4732 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:14 x4 kernel: pid 4732 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:15 x4 kernel: pid 4736 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:16 x4 kernel: pid 4734 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:17 x4 kernel: pid 4735 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:18 x4 kernel: pid 4738 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:19 x4 kernel: pid 4736 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:20 x4 kernel: pid 4737 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:21 x4 kernel: pid 4736 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:22 x4 kernel: pid 4736 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:23 x4 kernel: pid 4734 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:24 x4 kernel: pid 4732 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:25 x4 kernel: pid 4733 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:26 x4 kernel: pid 4736 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:27 x4 kernel: pid 4732 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:28 x4 kernel: pid 4739 (fragments), uid 1004 inumber 6841 on /mnt: filesystem full Nov 30 03:03:29 x4 kernel: pid 4739 (fragments), uid 1004 inumber 6734 on /mnt: filesystem full Nov 30 03:03:30 x4 kernel: pid 5984 (fragments), uid 1004 inumber 6685 on /mnt: filesystem full Nov 30 03:03:31 x4 kernel: pid 5986 (fragments), uid 1004 inumber 6756 on /mnt: filesystem full Nov 30 03:03:32 x4 kernel: pid 5986 (fragments), uid 1004 inumber 6800 on /mnt: filesystem full Nov 30 03:03:33 x4 kernel: pid 5985 (fragments), uid 1004 inumber 6736 on /mnt: filesystem full Nov 30 03:03:34 x4 kernel: pid 5980 (fragments), uid 1004 inumber 6626 on /mnt: filesystem full Nov 30 03:03:35 x4 kernel: pid 5980 (fragments), uid 1004 inumber 6733 on /mnt: filesystem full Nov 30 03:03:36 x4 kernel: pid 6605 (fragments), uid 1004 inumber 6592 on /mnt: filesystem full Nov 30 03:03:37 x4 kernel: pid 6602 (fragments), uid 1004 inumber 6664 on /mnt: filesystem full Nov 30 03:03:37 x4 kernel: pid 6607 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:38 x4 kernel: pid 6608 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:39 x4 kernel: pid 6604 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:40 x4 kernel: pid 6608 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:41 x4 kernel: pid 6606 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:42 x4 kernel: pid 6608 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:43 x4 kernel: pid 6602 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:44 x4 kernel: pid 6609 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:45 x4 kernel: pid 6606 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:47 x4 kernel: pid 6607 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:48 x4 kernel: pid 6602 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:49 x4 kernel: pid 6609 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:50 x4 kernel: pid 6606 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:51 x4 kernel: pid 6605 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:52 x4 kernel: pid 6606 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:53 x4 kernel: pid 6602 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:53 x4 kernel: pid 6603 (fragments), uid 1004 inumber 6848 on /mnt: filesystem full Nov 30 03:03:54 x4 kernel: pid 6603 (fragments), uid 1004 inumber 6737 on /mnt: filesystem full Nov 30 03:03:55 x4 kernel: pid 7231 (fragments), uid 1004 inumber 6632 on /mnt: filesystem full Nov 30 03:03:56 x4 kernel: pid 7230 (fragments), uid 1004 inumber 6583 on /mnt: filesystem full Nov 30 03:03:57 x4 kernel: pid 7231 (fragments), uid 1004 inumber 6887 on /mnt: filesystem full Nov 30 03:03:58 x4 kernel: pid 7231 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:03:58 x4 kernel: pid 7227 (fragments), uid 1004 inumber 6748 on /mnt: filesystem full Nov 30 03:03:59 x4 kernel: pid 7854 (fragments), uid 1004 inumber 6677 on /mnt: filesystem full Nov 30 03:04:00 x4 kernel: pid 7850 (fragments), uid 1004 inumber 6857 on /mnt: filesystem full Nov 30 03:04:01 x4 kernel: pid 7855 (fragments), uid 1004 inumber 6983 on /mnt: filesystem full Nov 30 03:04:02 x4 kernel: pid 7852 (fragments), uid 1004 inumber 7125 on /mnt: filesystem full Nov 30 03:04:03 x4 kernel: pid 7863 (fragments), uid 1004 inumber 6675 on /mnt: filesystem full Nov 30 03:04:04 x4 kernel: pid 7858 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:04 x4 kernel: pid 7859 (fragments), uid 1004 inumber 6921 on /mnt: filesystem full Nov 30 03:04:05 x4 kernel: pid 7860 (fragments), uid 1004 inumber 6903 on /mnt: filesystem full Nov 30 03:04:06 x4 kernel: pid 7862 (fragments), uid 1004 inumber 6667 on /mnt: filesystem full Nov 30 03:04:07 x4 kernel: pid 8456 (fragments), uid 1004 inumber 6577 on /mnt: filesystem full Nov 30 03:04:09 x4 kernel: pid 8452 (fragments), uid 1004 inumber 6643 on /mnt: filesystem full Nov 30 03:04:10 x4 kernel: pid 8457 (fragments), uid 1004 inumber 6768 on /mnt: filesystem full Nov 30 03:04:11 x4 kernel: pid 8456 (fragments), uid 1004 inumber 6646 on /mnt: filesystem full Nov 30 03:04:12 x4 kernel: pid 8452 (fragments), uid 1004 inumber 6808 on /mnt: filesystem full Nov 30 03:04:12 x4 kernel: pid 8452 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:13 x4 kernel: pid 8456 (fragments), uid 1004 inumber 6782 on /mnt: filesystem full Nov 30 03:04:13 x4 kernel: pid 8451 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:14 x4 kernel: pid 8451 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:14 x4 kernel: pid 8456 (fragments), uid 1004 inumber 6613 on /mnt: filesystem full Nov 30 03:04:15 x4 kernel: pid 8455 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:16 x4 kernel: pid 8458 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:17 x4 kernel: pid 8455 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:17 x4 kernel: pid 8492 (fragments), uid 1004 inumber 6628 on /mnt: filesystem full Nov 30 03:04:18 x4 kernel: pid 8494 (fragments), uid 1004 inumber 6723 on /mnt: filesystem full Nov 30 03:04:19 x4 kernel: pid 8491 (fragments), uid 1004 inumber 6710 on /mnt: filesystem full Nov 30 03:04:20 x4 kernel: pid 8494 (fragments), uid 1004 inumber 6759 on /mnt: filesystem full Nov 30 03:04:21 x4 kernel: pid 8492 (fragments), uid 1004 inumber 6680 on /mnt: filesystem full Nov 30 03:04:22 x4 kernel: pid 9111 (fragments), uid 1004 inumber 6675 on /mnt: filesystem full Nov 30 03:04:23 x4 kernel: pid 9110 (fragments), uid 1004 inumber 6636 on /mnt: filesystem full Nov 30 03:04:24 x4 kernel: pid 9112 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:24 x4 kernel: pid 9112 (fragments), uid 1004 inumber 6685 on /mnt: filesystem full Nov 30 03:04:25 x4 kernel: pid 9116 (fragments), uid 1004 inumber 6859 on /mnt: filesystem full Nov 30 03:04:26 x4 kernel: pid 9120 (fragments), uid 1004 inumber 6676 on /mnt: filesystem full Nov 30 03:04:27 x4 kernel: pid 9121 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:28 x4 kernel: pid 9119 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:29 x4 kernel: pid 9122 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:30 x4 kernel: pid 9118 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:31 x4 kernel: pid 9117 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:32 x4 kernel: pid 9122 (fragments), uid 1004 inumber 6810 on /mnt: filesystem full Nov 30 03:04:33 x4 kernel: pid 9123 (fragments), uid 1004 inumber 6588 on /mnt: filesystem full Nov 30 03:04:34 x4 kernel: pid 9739 (fragments), uid 1004 inumber 6648 on /mnt: filesystem full Nov 30 03:04:34 x4 kernel: pid 9739 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:35 x4 kernel: pid 9740 (fragments), uid 1004 inumber 6903 on /mnt: filesystem full Nov 30 03:04:36 x4 kernel: pid 9744 (fragments), uid 1004 inumber 7027 on /mnt: filesystem full Nov 30 03:04:36 x4 kernel: pid 9742 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:37 x4 kernel: pid 9743 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:38 x4 kernel: pid 9739 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:39 x4 kernel: pid 9743 (fragments), uid 1004 inumber 7068 on /mnt: filesystem full Nov 30 03:04:40 x4 kernel: pid 9753 (fragments), uid 1004 inumber 6726 on /mnt: filesystem full Nov 30 03:04:41 x4 kernel: pid 9754 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:42 x4 kernel: pid 9753 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:43 x4 kernel: pid 9747 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:44 x4 kernel: pid 9751 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:45 x4 kernel: pid 9753 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:47 x4 last message repeated 2 times Nov 30 03:04:48 x4 kernel: pid 9749 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:49 x4 kernel: pid 9751 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:50 x4 kernel: pid 9748 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:51 x4 kernel: pid 9753 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:51 x4 kernel: pid 9752 (fragments), uid 1004 inumber 6834 on /mnt: filesystem full Nov 30 03:04:52 x4 kernel: pid 10873 (fragments), uid 1004 inumber 6681 on /mnt: filesystem full Nov 30 03:04:53 x4 kernel: pid 10868 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:54 x4 kernel: pid 10873 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:55 x4 kernel: pid 10868 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:56 x4 kernel: pid 10875 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:57 x4 kernel: pid 10867 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:58 x4 kernel: pid 10875 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:04:59 x4 kernel: pid 10867 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:00 x4 kernel: pid 10872 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:01 x4 kernel: pid 10872 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:02 x4 kernel: pid 10867 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:02 x4 kernel: pid 10871 (fragments), uid 1004 inumber 6847 on /mnt: filesystem full Nov 30 03:05:03 x4 kernel: pid 11609 (fragments), uid 1004 inumber 6626 on /mnt: filesystem full Nov 30 03:05:04 x4 kernel: pid 11616 (fragments), uid 1004 inumber 6601 on /mnt: filesystem full Nov 30 03:05:05 x4 kernel: pid 11616 (fragments), uid 1004 inumber 6709 on /mnt: filesystem full Nov 30 03:05:06 x4 kernel: pid 11616 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:07 x4 kernel: pid 11609 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:08 x4 kernel: pid 11613 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:08 x4 kernel: pid 11617 (fragments), uid 1004 inumber 6605 on /mnt: filesystem full Nov 30 03:05:09 x4 kernel: pid 11623 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:10 x4 kernel: pid 11619 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:11 x4 kernel: pid 11624 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:11 x4 kernel: pid 11619 (fragments), uid 1004 inumber 6808 on /mnt: filesystem full Nov 30 03:05:12 x4 kernel: pid 11621 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:12 x4 kernel: pid 11618 (fragments), uid 1004 inumber 7103 on /mnt: filesystem full Nov 30 03:05:13 x4 kernel: pid 11630 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:13 x4 kernel: pid 11630 (fragments), uid 1004 inumber 6757 on /mnt: filesystem full Nov 30 03:05:14 x4 kernel: pid 11632 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:15 x4 kernel: pid 11625 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:16 x4 kernel: pid 11631 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:16 x4 kernel: pid 11630 (fragments), uid 1004 inumber 7016 on /mnt: filesystem full Nov 30 03:05:17 x4 kernel: pid 12252 (fragments), uid 1004 inumber 6860 on /mnt: filesystem full Nov 30 03:05:18 x4 kernel: pid 12262 (fragments), uid 1004 inumber 6670 on /mnt: filesystem full Nov 30 03:05:19 x4 kernel: pid 12259 (fragments), uid 1004 inumber 6958 on /mnt: filesystem full Nov 30 03:05:20 x4 kernel: pid 12255 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:21 x4 kernel: pid 12260 (fragments), uid 1004 inumber 7065 on /mnt: filesystem full Nov 30 03:05:22 x4 kernel: pid 12267 (fragments), uid 1004 inumber 6641 on /mnt: filesystem full Nov 30 03:05:22 x4 kernel: pid 12263 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:23 x4 kernel: pid 12264 (fragments), uid 1004 inumber 6759 on /mnt: filesystem full Nov 30 03:05:24 x4 kernel: pid 12274 (fragments), uid 1004 inumber 6683 on /mnt: filesystem full Nov 30 03:05:24 x4 kernel: pid 12274 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:25 x4 kernel: pid 12273 (fragments), uid 1004 inumber 6838 on /mnt: filesystem full Nov 30 03:05:26 x4 kernel: pid 12273 (fragments), uid 1004 inumber 7054 on /mnt: filesystem full Nov 30 03:05:27 x4 kernel: pid 12272 (fragments), uid 1004 inumber 6715 on /mnt: filesystem full Nov 30 03:05:28 x4 kernel: pid 12897 (fragments), uid 1004 inumber 6850 on /mnt: filesystem full Nov 30 03:05:29 x4 kernel: pid 12898 (fragments), uid 1004 inumber 7045 on /mnt: filesystem full Nov 30 03:05:29 x4 kernel: pid 12896 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:30 x4 kernel: pid 12906 (fragments), uid 1004 inumber 6643 on /mnt: filesystem full Nov 30 03:05:31 x4 kernel: pid 12903 (fragments), uid 1004 inumber 6685 on /mnt: filesystem full Nov 30 03:05:32 x4 kernel: pid 12904 (fragments), uid 1004 inumber 6717 on /mnt: filesystem full Nov 30 03:05:33 x4 kernel: pid 12914 (fragments), uid 1004 inumber 6782 on /mnt: filesystem full Nov 30 03:05:34 x4 kernel: pid 12909 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:34 x4 kernel: pid 12910 (fragments), uid 1004 inumber 7067 on /mnt: filesystem full Nov 30 03:05:35 x4 kernel: pid 12919 (fragments), uid 1004 inumber 6776 on /mnt: filesystem full Nov 30 03:05:36 x4 kernel: pid 12920 (fragments), uid 1004 inumber 6615 on /mnt: filesystem full Nov 30 03:05:37 x4 kernel: pid 13546 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:37 x4 kernel: pid 13539 (fragments), uid 1004 inumber 6810 on /mnt: filesystem full Nov 30 03:05:38 x4 kernel: pid 13540 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:39 x4 kernel: pid 13539 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:39 x4 kernel: pid 13540 (fragments), uid 1004 inumber 7068 on /mnt: filesystem full Nov 30 03:05:40 x4 kernel: pid 13547 (fragments), uid 1004 inumber 6670 on /mnt: filesystem full Nov 30 03:05:41 x4 kernel: pid 13549 (fragments), uid 1004 inumber 6718 on /mnt: filesystem full Nov 30 03:05:42 x4 kernel: pid 13562 (fragments), uid 1004 inumber 6802 on /mnt: filesystem full Nov 30 03:05:42 x4 kernel: pid 13555 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:43 x4 kernel: pid 13561 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:44 x4 kernel: pid 13557 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:45 x4 kernel: pid 13559 (fragments), uid 1004 inumber 7095 on /mnt: filesystem full Nov 30 03:05:46 x4 kernel: pid 13569 (fragments), uid 1004 inumber 6808 on /mnt: filesystem full Nov 30 03:05:47 x4 kernel: pid 13580 (fragments), uid 1004 inumber 6740 on /mnt: filesystem full Nov 30 03:05:48 x4 kernel: pid 13575 (fragments), uid 1004 inumber 6951 on /mnt: filesystem full Nov 30 03:05:48 x4 kernel: pid 13581 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:49 x4 kernel: pid 13576 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:50 x4 kernel: pid 13581 (fragments), uid 1004 inumber 7059 on /mnt: filesystem full Nov 30 03:05:51 x4 kernel: pid 14196 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:51 x4 kernel: pid 14196 (fragments), uid 1004 inumber 6628 on /mnt: filesystem full Nov 30 03:05:52 x4 kernel: pid 14199 (fragments), uid 1004 inumber 6630 on /mnt: filesystem full Nov 30 03:05:53 x4 kernel: pid 14207 (fragments), uid 1004 inumber 6707 on /mnt: filesystem full Nov 30 03:05:54 x4 kernel: pid 14210 (fragments), uid 1004 inumber 7097 on /mnt: filesystem full Nov 30 03:05:54 x4 kernel: pid 14216 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:55 x4 kernel: pid 14214 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:05:56 x4 kernel: pid 14213 (fragments), uid 1004 inumber 6805 on /mnt: filesystem full Nov 30 03:05:57 x4 kernel: pid 14218 (fragments), uid 1004 inumber 6704 on /mnt: filesystem full Nov 30 03:05:58 x4 kernel: pid 14219 (fragments), uid 1004 inumber 6927 on /mnt: filesystem full Nov 30 03:05:59 x4 kernel: pid 14844 (fragments), uid 1004 inumber 6729 on /mnt: filesystem full Nov 30 03:06:00 x4 kernel: pid 14843 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:06:00 x4 kernel: pid 14842 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:06:01 x4 kernel: pid 14845 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:06:03 x4 kernel: pid 14847 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:06:03 x4 kernel: pid 14845 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:06:04 x4 kernel: pid 14848 (fragments), uid 1004 inumber 6848 on /mnt: filesystem full Nov 30 03:06:05 x4 kernel: pid 14852 (fragments), uid 1004 inumber 6792 on /mnt: filesystem full Nov 30 03:06:05 x4 kernel: pid 14855 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:06:06 x4 kernel: pid 14849 (fragments), uid 1004 inumber 6665 on /mnt: filesystem full Nov 30 03:06:07 x4 kernel: pid 14863 (fragments), uid 1004 inumber 6588 on /mnt: filesystem full Nov 30 03:06:08 x4 kernel: pid 14858 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:06:09 x4 kernel: pid 14857 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:06:09 x4 kernel: pid 14865 (fragments), uid 1004 inumber 6605 on /mnt: filesystem full Nov 30 03:06:10 x4 kernel: pid 14869 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:06:10 x4 kernel: pid 14870 (fragments), uid 1004 inumber 6797 on /mnt: filesystem full Nov 30 03:06:11 x4 kernel: pid 14870 (fragments), uid 1004 inumber 6678 on /mnt: filesystem full Nov 30 03:06:11 x4 kernel: pid 15492 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:06:12 x4 kernel: pid 15491 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:06:13 x4 kernel: pid 15493 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:06:14 x4 kernel: pid 15494 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:06:15 x4 kernel: pid 15490 (fragments), uid 1004 inumber 6848 on /mnt: filesystem full Nov 30 03:06:16 x4 kernel: pid 15498 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:06:16 x4 kernel: pid 15502 (fragments), uid 1004 inumber 6694 on /mnt: filesystem full Nov 30 03:06:17 x4 kernel: pid 15500 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:06:17 x4 kernel: pid 15500 (fragments), uid 1004 inumber 6939 on /mnt: filesystem full Nov 30 03:06:18 x4 kernel: pid 15505 (fragments), uid 1004 inumber 6851 on /mnt: filesystem full Nov 30 03:06:19 x4 kernel: pid 15504 (fragments), uid 1004 inumber 6870 on /mnt: filesystem full Nov 30 03:06:20 x4 kernel: pid 15511 (fragments), uid 1004 inumber 2 on /mnt: filesystem full Nov 30 03:06:20 x4 kernel: pid 15515 (fragments), uid 1004 inumber 6807 on /mnt: filesystem full Nov 30 03:06:21 x4 kernel: pid 15693 (fragments), uid 1004 inumber 6702 on /mnt: filesystem full Nov 30 03:06:22 x4 kernel: pid 16145 (fragments), uid 1004 inumber 6780 on /mnt: filesystem full Nov 30 03:06:23 x4 kernel: pid 16142 (fragments), uid 1004 inumber 6658 on /mnt: filesystem full 20161130 03:06:26 all (79/150): fs.sh Nov 30 03:12:41 x4 kernel: pid 31439 (rw), uid 1004 inumber 2115 on /mnt: filesystem full Nov 30 03:20:41 x4 kernel: pid 52024 (rw), uid 1004 inumber 644 on /mnt: filesystem full Nov 30 03:20:54 x4 kernel: pid 52035 (creat), uid 1004 inumber 645 on /mnt: out of inodes Nov 30 03:20:56 x4 kernel: pid 52021 (creat), uid 1004 inumber 640 on /mnt: out of inodes Nov 30 03:20:57 x4 kernel: pid 52023 (creat), uid 1004 inumber 4 on /mnt: out of inodes Nov 30 03:20:59 x4 kernel: pid 52026 (creat), uid 1004 inumber 5 on /mnt: out of inodes Nov 30 03:21:02 x4 kernel: pid 52032 (creat), uid 1004 inumber 641 on /mnt: out of inodes Nov 30 03:21:03 x4 kernel: pid 52035 (creat), uid 1004 inumber 645 on /mnt: out of inodes Nov 30 03:21:06 x4 kernel: pid 52026 (creat), uid 1004 inumber 5 on /mnt: out of inodes Nov 30 03:21:08 x4 kernel: pid 52030 (creat), uid 1004 inumber 1280 on /mnt: out of inodes Nov 30 03:21:10 x4 kernel: pid 52035 (creat), uid 1004 inumber 645 on /mnt: out of inodes Nov 30 03:21:11 x4 kernel: pid 52023 (creat), uid 1004 inumber 4 on /mnt: out of inodes Nov 30 03:21:13 x4 kernel: pid 52037 (creat), uid 1004 inumber 643 on /mnt: out of inodes Nov 30 03:21:15 x4 kernel: pid 52028 (creat), uid 1004 inumber 1922 on /mnt: out of inodes Nov 30 03:21:17 x4 kernel: pid 52023 (creat), uid 1004 inumber 4 on /mnt: out of inodes Nov 30 03:21:19 x4 kernel: pid 52035 (creat), uid 1004 inumber 645 on /mnt: out of inodes Nov 30 03:21:21 x4 kernel: pid 52026 (creat), uid 1004 inumber 5 on /mnt: out of inodes Nov 30 03:21:23 x4 kernel: pid 52037 (creat), uid 1004 inumber 643 on /mnt: out of inodes Nov 30 03:21:28 x4 last message repeated 2 times Nov 30 03:21:29 x4 kernel: pid 52035 (creat), uid 1004 inumber 645 on /mnt: out of inodes Nov 30 03:21:33 x4 kernel: pid 52030 (creat), uid 1004 inumber 1280 on /mnt: out of inodes Nov 30 03:21:34 x4 kernel: pid 52030 (creat), uid 1004 inumber 1280 on /mnt: out of inodes Nov 30 03:21:36 x4 kernel: pid 52021 (creat), uid 1004 inumber 640 on /mnt: out of inodes Nov 30 03:21:37 x4 kernel: pid 52023 (creat), uid 1004 inumber 4 on /mnt: out of inodes Nov 30 03:21:38 x4 kernel: pid 52037 (creat), uid 1004 inumber 643 on /mnt: out of inodes Nov 30 03:21:40 x4 kernel: pid 52035 (creat), uid 1004 inumber 645 on /mnt: out of inodes Nov 30 03:21:41 x4 kernel: pid 52023 (creat), uid 1004 inumber 4 on /mnt: out of inodes Nov 30 03:21:42 x4 kernel: pid 52026 (creat), uid 1004 inumber 5 on /mnt: out of inodes Nov 30 03:21:43 x4 kernel: pid 52035 (creat), uid 1004 inumber 645 on /mnt: out of inodes Nov 30 03:21:45 x4 kernel: pid 52037 (creat), uid 1004 inumber 643 on /mnt: out of inodes Nov 30 03:21:46 x4 kernel: pid 52035 (creat), uid 1004 inumber 645 on /mnt: out of inodes Nov 30 03:21:51 x4 kernel: pid 52032 (creat), uid 1004 inumber 641 on /mnt: out of inodes Nov 30 03:21:52 x4 kernel: pid 52032 (creat), uid 1004 inumber 641 on /mnt: out of inodes Nov 30 03:21:56 x4 kernel: pid 52021 (creat), uid 1004 inumber 640 on /mnt: out of inodes Nov 30 03:22:00 x4 kernel: pid 52032 (creat), uid 1004 inumber 641 on /mnt: out of inodes Nov 30 03:22:01 x4 kernel: pid 52030 (creat), uid 1004 inumber 1280 on /mnt: out of inodes Nov 30 03:22:05 x4 kernel: pid 52021 (creat), uid 1004 inumber 640 on /mnt: out of inodes Nov 30 03:22:07 x4 kernel: pid 52030 (creat), uid 1004 inumber 1280 on /mnt: out of inodes Nov 30 03:22:09 x4 kernel: pid 52023 (creat), uid 1004 inumber 4 on /mnt: out of inodes Nov 30 03:22:13 x4 kernel: pid 52037 (creat), uid 1004 inumber 643 on /mnt: out of inodes Nov 30 03:22:14 x4 kernel: pid 52036 (mkdir), uid 1004 inumber 1920 on /mnt: out of inodes Nov 30 03:22:18 x4 kernel: pid 52035 (creat), uid 1004 inumber 645 on /mnt: out of inodes Nov 30 03:22:19 x4 kernel: pid 52021 (creat), uid 1004 inumber 640 on /mnt: out of inodes Nov 30 03:22:20 x4 kernel: pid 52037 (creat), uid 1004 inumber 643 on /mnt: out of inodes Nov 30 03:22:22 x4 kernel: pid 52030 (creat), uid 1004 inumber 1280 on /mnt: out of inodes Nov 30 03:22:27 x4 kernel: pid 52032 (creat), uid 1004 inumber 641 on /mnt: out of inodes Nov 30 03:22:28 x4 kernel: pid 52021 (creat), uid 1004 inumber 640 on /mnt: out of inodes Nov 30 03:22:29 x4 kernel: pid 52028 (creat), uid 1004 inumber 1922 on /mnt: out of inodes Nov 30 03:22:31 x4 kernel: pid 52028 (creat), uid 1004 inumber 1922 on /mnt: out of inodes Nov 30 03:22:33 x4 kernel: pid 52021 (creat), uid 1004 inumber 640 on /mnt: out of inodes Nov 30 03:22:34 x4 kernel: pid 52021 (creat), uid 1004 inumber 640 on /mnt: out of inodes Nov 30 03:22:36 x4 kernel: pid 52023 (creat), uid 1004 inumber 4 on /mnt: out of inodes Nov 30 03:22:51 x4 kernel: pid 57668 (creat), uid 1004 inumber 1945 on /mnt: out of inodes Nov 30 03:22:55 x4 kernel: pid 57641 (rw), uid 1004 inumber 1922 on /mnt: out of inodes Nov 30 03:22:57 x4 kernel: pid 57660 (creat), uid 1004 inumber 14 on /mnt: out of inodes Nov 30 03:23:06 x4 kernel: pid 57656 (creat), uid 1004 inumber 1280 on /mnt: out of inodes Nov 30 03:23:55 x4 kernel: pid 57662 (creat), uid 1004 inumber 15 on /mnt: out of inodes Nov 30 03:24:23 x4 kernel: pid 57668 (creat), uid 1004 inumber 1945 on /mnt: out of inodes Nov 30 03:25:43 x4 kernel: pid 62637 (rw), uid 1004 inumber 9 on /mnt: filesystem full Nov 30 03:26:55 x4 kernel: pid 67640 (creat), uid 1004 inumber 4 on /mnt: out of inodes Nov 30 03:26:57 x4 kernel: pid 67646 (creat), uid 1004 inumber 5 on /mnt: out of inodes Nov 30 03:27:00 x4 kernel: pid 67665 (mkdir), uid 1004 inumber 1920 on /mnt: out of inodes Nov 30 03:27:01 x4 kernel: pid 67657 (creat), uid 1004 inumber 1281 on /mnt: out of inodes Nov 30 03:27:10 x4 kernel: pid 67652 (creat), uid 1004 inumber 1283 on /mnt: out of inodes Nov 30 03:27:11 x4 kernel: pid 67659 (creat), uid 1004 inumber 7 on /mnt: out of inodes Nov 30 03:27:19 x4 kernel: pid 67656 (creat), uid 1004 inumber 643 on /mnt: out of inodes Nov 30 03:27:21 x4 kernel: pid 67646 (creat), uid 1004 inumber 5 on /mnt: out of inodes Nov 30 03:27:24 x4 kernel: pid 67652 (creat), uid 1004 inumber 1283 on /mnt: out of inodes Nov 30 03:27:27 x4 kernel: pid 67669 (mkdir), uid 1004 inumber 1920 on /mnt: out of inodes Nov 30 03:27:32 x4 kernel: pid 67654 (creat), uid 1004 inumber 640 on /mnt: out of inodes Nov 30 03:27:36 x4 kernel: pid 67658 (creat), uid 1004 inumber 641 on /mnt: out of inodes Nov 30 03:27:40 x4 kernel: pid 67658 (creat), uid 1004 inumber 641 on /mnt: out of inodes Nov 30 03:27:49 x4 kernel: pid 67652 (creat), uid 1004 inumber 1283 on /mnt: out of inodes Nov 30 03:27:54 x4 kernel: pid 67660 (creat), uid 1004 inumber 8 on /mnt: out of inodes Nov 30 03:27:56 x4 kernel: pid 67657 (creat), uid 1004 inumber 1281 on /mnt: out of inodes Nov 30 03:28:01 x4 kernel: pid 67649 (creat), uid 1004 inumber 1932 on /mnt: out of inodes Nov 30 03:28:09 x4 kernel: pid 67651 (creat), uid 1004 inumber 1933 on /mnt: out of inodes Nov 30 03:28:15 x4 kernel: pid 67656 (creat), uid 1004 inumber 643 on /mnt: out of inodes Nov 30 03:28:16 x4 kernel: pid 67651 (creat), uid 1004 inumber 1933 on /mnt: out of inodes Nov 30 03:28:18 x4 kernel: pid 67655 (creat), uid 1004 inumber 1280 on /mnt: out of inodes Nov 30 03:28:21 x4 kernel: pid 67658 (creat), uid 1004 inumber 641 on /mnt: out of inodes Nov 30 03:28:24 x4 kernel: pid 67650 (creat), uid 1004 inumber 1282 on /mnt: out of inodes Nov 30 03:28:32 x4 kernel: pid 67652 (creat), uid 1004 inumber 1283 on /mnt: out of inodes Nov 30 03:28:36 x4 kernel: pid 67644 (creat), uid 1004 inumber 1934 on /mnt: out of inodes Nov 30 03:28:41 x4 kernel: pid 67649 (creat), uid 1004 inumber 1932 on /mnt: out of inodes Nov 30 03:28:48 x4 kernel: pid 73430 (mkdir), uid 1004 inumber 239 on /mnt: out of inodes Nov 30 03:28:50 x4 kernel: pid 73434 (mkdir), uid 1004 inumber 138 on /mnt: out of inodes Nov 30 03:28:52 x4 kernel: pid 73456 (mkdir), uid 1004 inumber 1893 on /mnt: out of inodes Nov 30 03:28:53 x4 kernel: pid 73439 (mkdir), uid 1004 inumber 244 on /mnt: out of inodes Nov 30 03:28:55 x4 kernel: pid 73458 (creat), uid 1004 inumber 191 on /mnt: out of inodes Nov 30 03:28:58 x4 kernel: pid 73452 (mkdir), uid 1004 inumber 2408 on /mnt: out of inodes Nov 30 03:29:04 x4 kernel: pid 73451 (creat), uid 1004 inumber 197 on /mnt: out of inodes Nov 30 03:29:06 x4 kernel: pid 73449 (creat), uid 1004 inumber 198 on /mnt: out of inodes Nov 30 03:29:09 x4 kernel: pid 73451 (creat), uid 1004 inumber 197 on /mnt: out of inodes Nov 30 03:29:14 x4 kernel: pid 73441 (creat), uid 1004 inumber 166 on /mnt: out of inodes Nov 30 03:29:15 x4 kernel: pid 73445 (creat), uid 1004 inumber 199 on /mnt: out of inodes Nov 30 03:29:17 x4 kernel: pid 73446 (mkdir), uid 1004 inumber 181 on /mnt: out of inodes Nov 30 03:29:18 x4 kernel: pid 73443 (creat), uid 1004 inumber 201 on /mnt: out of inodes Nov 30 03:29:20 x4 kernel: pid 73451 (creat), uid 1004 inumber 197 on /mnt: out of inodes Nov 30 03:29:23 x4 kernel: pid 73431 (mkdir), uid 1004 inumber 1229 on /mnt: out of inodes Nov 30 03:29:31 x4 kernel: pid 73451 (creat), uid 1004 inumber 197 on /mnt: out of inodes Nov 30 03:29:35 x4 kernel: pid 73445 (creat), uid 1004 inumber 199 on /mnt: out of inodes Nov 30 03:29:38 x4 kernel: pid 73454 (mkdir), uid 1004 inumber 208 on /mnt: out of inodes Nov 30 03:29:44 x4 kernel: pid 73450 (mkdir), uid 1004 inumber 192 on /mnt: out of inodes Nov 30 03:29:47 x4 kernel: pid 73459 (creat), uid 1004 inumber 215 on /mnt: out of inodes Nov 30 03:29:49 x4 kernel: pid 73447 (creat), uid 1004 inumber 216 on /mnt: out of inodes Nov 30 03:30:00 x4 kernel: pid 73427 (rw), uid 1004 inumber 1761 on /mnt: filesystem full Nov 30 03:30:04 x4 kernel: pid 73449 (creat), uid 1004 inumber 198 on /mnt: out of inodes Nov 30 03:30:10 x4 kernel: pid 73455 (creat), uid 1004 inumber 203 on /mnt: out of inodes Nov 30 03:30:12 x4 kernel: pid 73441 (creat), uid 1004 inumber 166 on /mnt: out of inodes Nov 30 03:30:15 x4 kernel: pid 73459 (creat), uid 1004 inumber 215 on /mnt: out of inodes Nov 30 03:30:26 x4 kernel: pid 73441 (creat), uid 1004 inumber 166 on /mnt: out of inodes Nov 30 03:30:28 x4 kernel: pid 73445 (creat), uid 1004 inumber 199 on /mnt: out of inodes Nov 30 03:30:39 x4 kernel: pid 73459 (creat), uid 1004 inumber 215 on /mnt: out of inodes Nov 30 03:30:43 x4 kernel: pid 73453 (creat), uid 1004 inumber 200 on /mnt: out of inodes Nov 30 03:30:52 x4 kernel: pid 79706 (mkdir), uid 1004 inumber 686 on /mnt: out of inodes Nov 30 03:30:57 x4 kernel: pid 79699 (mkdir), uid 1004 inumber 1919 on /mnt: out of inodes Nov 30 03:31:00 x4 kernel: pid 79700 (mkdir), uid 1004 inumber 1172 on /mnt: out of inodes Nov 30 03:31:04 x4 kernel: pid 79696 (rw), uid 1004 inumber 641 on /mnt: out of inodes Nov 30 03:32:10 x4 kernel: pid 79705 (mkdir), uid 1004 inumber 479 on /mnt: out of inodes Nov 30 03:32:47 x4 kernel: pid 86529 (rw), uid 1004 inumber 1943 on /mnt: filesystem full Nov 30 03:36:57 x4 kernel: pid 99056 (creat), uid 1004 inumber 689 on /mnt: out of inodes Nov 30 03:36:59 x4 kernel: pid 99044 (creat), uid 1004 inumber 1282 on /mnt: out of inodes Nov 30 03:37:02 x4 kernel: pid 99028 (rw), uid 1004 inumber 1324 on /mnt: out of inodes 20161130 03:38:51 all (80/150): ftruncate.sh 20161130 03:41:58 all (81/150): ftruncate2.sh 20161130 03:45:00 all (82/150): fts.sh 20161130 04:15:43 all (83/150): fts2.sh 20161130 04:16:52 all (84/150): full.sh vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed 20161130 04:28:02 all (85/150): fullpath.sh 20161130 04:29:09 all (86/150): fullpath2.sh 20161130 04:34:43 all (87/150): gnop.sh GEOM_NOP: Device md5.nop created. GEOM_NOP: Device md5.nop removed. GEOM_NOP: Device md5.nop created. swap_pager: I/O error - pageout failed; blkno 248064,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 248080,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 248096,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 248112,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 248128,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 248144,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 248160,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 248176,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 248192,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 248208,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 248224,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 248240,size 65536, error 12 GEOM_NOP: Device md5.nop removed. GEOM_NOP: Device md5.nop created. GEOM_NOP: Device md5.nop removed. GEOM_NOP: Device md5.nop created. GEOM_NOP: Device md5.nop removed. 20161130 04:58:37 all (88/150): gnop2.sh GEOM_NOP: Device md5.nop created. GEOM_NOP: Device md5.nop removed. GEOM_NOP: Device md5.nop created. GEOM_NOP: Device md5.nop removed. GEOM_NOP: Device md5.nop created. GEOM_NOP: Device md5.nop removed. GEOM_NOP: Device md5.nop created. GEOM_NOP: Device md5.nop removed. 20161130 05:03:03 all (89/150): gnop3.sh GEOM_NOP: Device md5.nop created. GEOM_NOP: Device md5.nop removed. 20161130 05:25:54 all (90/150): gnop4.sh GEOM_NOP: Device md5.nop created. GEOM_NOP: Device md5.nop removed. *** Excessive run time: gnop4.sh, 45 min 20161130 06:11:26 all (91/150): graid0.sh GEOM_STRIPE: Device data created (id=1271137348). GEOM_STRIPE: Disk md5 attached to data. GEOM_STRIPE: Disk md6 attached to data. GEOM_STRIPE: Disk md7 attached to data. GEOM_STRIPE: Device stripe/data activated. swap_pager: I/O error - pageout failed; blkno 30560,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 30624,size 65536, error 12 GEOM_STRIPE: Disk md7 removed from data. GEOM_STRIPE: Device stripe/data deactivated. GEOM_STRIPE: Disk md6 removed from data. GEOM_STRIPE: Disk md5 removed from data. GEOM_STRIPE: Device data destroyed. 20161130 06:24:00 all (92/150): graid1.sh GEOM_MIRROR: Device mirror/data launched (3/3). swap_pager: I/O error - pageout failed; blkno 72304,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72320,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72336,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72352,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72368,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72384,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72400,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72416,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72432,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72448,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72464,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72480,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72496,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72512,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72528,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72544,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72560,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72576,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72592,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72608,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72624,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72640,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72656,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72672,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72688,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72704,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72720,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72736,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72752,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72768,size 65536, error 12 swap_pager: I/O error - pageout failed; blkno 72784,size 65536, error 12 GEOM_MIRROR: Device data: provider destroyed. GEOM_MIRROR: Device data destroyed. GEOM_MIRROR: Device data destroyed. 20161130 06:35:03 all (93/150): graid3.sh GEOM_RAID3: Device raid3/data launched (3/3). GEOM_RAID3: Device data: provider raid3/data destroyed. GEOM_RAID3: Device data destroyed. 20161130 06:55:40 all (94/150): holdcnt0.sh 20161130 06:55:41 all (95/150): holdcnt02.sh 20161130 06:55:43 all (96/150): holdcnt03.sh 20161130 06:55:45 all (97/150): inversion.sh 20161130 07:25:51 all (98/150): isofs.sh 20161130 07:26:14 all (99/150): isofs2.sh 20161130 07:26:17 all (100/150): isofs3.sh Nov 30 07:26:45 x4 mountd[564]: can't delete exports for /mnt: Invalid argument 20161130 07:34:29 all (101/150): jail.sh 20161130 07:34:35 all (102/150): jail2.sh 20161130 07:34:38 all (103/150): jail3.sh 20161130 07:34:41 all (104/150): jail4.sh 20161130 07:34:43 all (105/150): jumbo.sh [zone: mbuf_jumbo_page] kern.ipc.nmbjumbop limit reached 20161130 07:36:47 all (106/150): kern_umtx_inf_loop.sh 20161130 07:36:50 all (107/150): kevent.sh 20161130 07:38:53 all (108/150): kevent2.sh 20161130 07:39:02 all (109/150): kevent3.sh 20161130 07:39:05 all (110/150): kevent4.sh 20161130 07:41:27 all (111/150): kevent5.sh 20161130 07:45:48 all (112/150): kevent6.sh 20161130 07:46:13 all (113/150): kevent8.sh 20161130 07:46:24 all (114/150): kevent9.sh 20161130 07:51:30 all (115/150): killpg.sh 20161130 07:51:32 all (116/150): kinfo.sh 20161130 08:21:37 all (117/150): kinfo2.sh 20161130 08:51:41 all (118/150): kinfo3.sh 20161130 09:06:46 all (119/150): ldt.sh 20161130 09:06:50 all (120/150): ldt2.sh 20161130 09:06:51 all (121/150): libMicro.sh 20161130 09:07:54 all (122/150): linger.sh 20161130 09:35:09 all (123/150): linger2.sh Nov 30 09:35:12 x4 kernel: pid 27341 (linger2), uid 1004 inumber 889 on /mnt: filesystem full Nov 30 09:35:12 x4 kernel: pid 27337 (linger2), uid 1004 inumber 890 on /mnt: filesystem full 20161130 09:35:17 all (124/150): linger3.sh 20161130 09:39:12 all (125/150): linger4.sh 20161130 09:59:47 all (126/150): link.sh Nov 30 10:01:02 x4 mountd[564]: bad exports list line /mnt 20161130 10:03:24 all (127/150): lockd.sh 20161130 10:04:31 all (128/150): lockf.sh 20161130 10:09:39 all (129/150): lockf2.sh 20161130 10:14:57 all (130/150): lockf3.sh Nov 30 10:28:23 x4 kernel: pid 88630 (swap), uid 0, was killed: out of swap space 20161130 10:39:25 all (131/150): lockf4.sh 20161130 10:54:07 all (132/150): lookup_shared.sh 20161130 11:16:09 all (133/150): lstat.sh 20161130 11:26:13 all (134/150): mac_chkexec.sh 20161130 11:26:15 all (135/150): mac.sh 20161130 11:26:17 all (136/150): machipc.sh 20161130 11:26:18 all (137/150): machipc2.sh 20161130 11:26:20 all (138/150): marcus.sh swap_pager: I/O error - pageout failed; blkno 7712,size 65536, error 12 20161130 11:57:58 all (139/150): marcus2.sh vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed vm_thread_new: kstack allocation failed FreeBSD/i386 (x4.osted.lan) (ttyu0) login: root Password: Nov 30 12:10:21 x4 login: ROOT LOGIN (root) ON ttyu0 Last login: Tue Nov 22 19:47:00 on ttyu0 FreeBSD 12.0-CURRENT (PHO) #0 r309257M: Tue Nov 29 17:23:37 CET 2016 You have new mail. root@x4:~ # bash vm_thread_new: kstack allocation failed bash: fork: Cannot allocate memory [root@x4 ~]# vm_thread_new: kstack allocation failed [root@x4 ~]# sysctl vm vm_thread_new: kstack allocation failed bash: fork: Cannot allocate memory [root@x4 ~]# invm_thread_new: kstack allocation failed it 1 [root@x4 ~]# vm_thread_new: kstack allocation failed /etc/rc.shutdownvm_th: Cannot fork: Cread_neannot allocate mw: kstemory ack allocation failed vm_thread_new: kstack allocation failed /etc/rc.shutdownvm_t: Cannot fork: Chread_nannot allocate mew: kemory stack allocation failed Nov 30 12:11:02 x4 sm-mta[91393]: uAUBB1dv091393: SYSERR(root): doworklist: cannot fork: Cannot allocate memory /etc/rc.shutdownNov 30 12:11:03 x4 init: /etc/rcNov 30 12:11:03 x4 syslogd: exiting on signal 15 Enter full pathname of shell or RETURN for /bin/sh: # sysctl vm vm.vmtotal: System wide totals computed every five seconds: (values in kilobytes) =============================================== Processes: (RUNQ: 1 Disk Wait: 0 Page Wait: 0 Sleep: 1) Virtual Memory: (Total: 2168156K Active: 2168156K) Real Memory: (Total: 226444K Active: 226444K) Shared Virtual Memory: (Total: 1488K Active: 1488K) Shared Real Memory: (Total: 1456K Active: 1456K) Free Memory: 2964980K vm.loadavg: { 0.05 16.83 38.12 } vm.v_free_min: 5484 vm.v_free_target: 18417 vm.v_free_reserved: 1173 vm.v_inactive_target: 27625 vm.v_pageout_free_min: 34 vm.swap_enabled: 1 vm.kvm_free: 4190208 vm.kvm_size: 1073737728 vm.pmap.pv_entry_count: 1291 vm.pmap.pde.promotions: 414 vm.pmap.pde.p_failures: 203 vm.pmap.pde.mappings: 43200 vm.pmap.pde.demotions: 53 vm.pmap.shpgperproc: 200 vm.pmap.pv_entry_max: 2095632 vm.pmap.pg_ps_enabled: 1 vm.pmap.pat_works: 1 vm.reserv.reclaimed: 265 vm.reserv.partpopq: LEVEL SIZE NUMBER -1: 0K, 0 vm.reserv.fullpop: 0 vm.reserv.freed: 237735 vm.reserv.broken: 279 vm.default_policy: rr vm.ndomains: 1 vm.phys_segs: SEGMENT 0: start: 0x1000 end: 0x9f000 domain: 0 free list: 0xc1b1f8d4 SEGMENT 1: start: 0x100000 end: 0x400000 domain: 0 free list: 0xc1b1f8d4 SEGMENT 2: start: 0x2000000 end: 0x201e000 domain: 0 free list: 0xc1b1f850 SEGMENT 3: start: 0x2028000 end: 0xd45da000 domain: 0 free list: 0xc1b1f850 vm.phys_free: DOMAIN 0: FREE LIST 0: ORDER (SIZE) | NUMBER | POOL 0 -- -- -- -- 10 ( 4096K) | 0 9 ( 2048K) | 0 8 ( 1024K) | 2 7 ( 512K) | 264 6 ( 256K) | 2346 5 ( 128K) | 5504 4 ( 64K) | 8923 3 ( 32K) | 12876 2 ( 16K) | 16638 1 ( 8K) | 19974 0 ( 4K) | 27939 FREE LIST 1: ORDER (SIZE) | NUMBER | POOL 0 -- -- -- -- 10 ( 4096K) | 0 9 ( 2048K) | 0 8 ( 1024K) | 0 7 ( 512K) | 0 6 ( 256K) | 0 5 ( 128K) | 0 4 ( 64K) | 2 3 ( 32K) | 10 2 ( 16K) | 35 1 ( 8K) | 53 0 ( 4K) | 79 vm.max_wired: 285687 vm.background_launder_max: 20480 vm.background_launder_rate: 4096 vm.background_launder_target: 1293 vm.act_scan_laundry_weight: 3 vm.pageout_oom_seq: 12 vm.pageout_lock_miss: 0 vm.disable_swapspace_pageouts: 0 vm.swap_idle_enabled: 0 vm.lowmem_period: 10 vm.pageout_update_period: 600 vm.pageout_wakeup_thresh: 6028 vm.panic_on_oom: 0 vm.page_blacklist: vm.tryrelock_restart: 912 vm.boot_pages: 64 vm.old_msync: 0 vm.old_mlock: 0 vm.stats.object.bypasses: 3991199 vm.stats.object.collapses: 14096429 vm.stats.vm.v_kthreadpages: 0 vm.stats.vm.v_rforkpages: 57100 vm.stats.vm.v_vforkpages: 15986261 vm.stats.vm.v_forkpages: 293500689 vm.stats.vm.v_kthreads: 151 vm.stats.vm.v_rforks: 1142 vm.stats.vm.v_vforks: 454049 vm.stats.vm.v_forks: 8022234 vm.stats.vm.v_interrupt_free_min: 2 vm.stats.vm.v_pageout_free_min: 34 vm.stats.vm.v_laundry_count: 25777 vm.stats.vm.v_inactive_count: 413 vm.stats.vm.v_inactive_target: 27625 vm.stats.vm.v_active_count: 17341 vm.stats.vm.v_wire_count: 77783 vm.stats.vm.v_free_count: 741228 vm.stats.vm.v_free_min: 5484 vm.stats.vm.v_free_target: 18417 vm.stats.vm.v_free_reserved: 1173 vm.stats.vm.v_page_count: 862544 vm.stats.vm.v_page_size: 4096 vm.stats.vm.v_tfree: 1067920710 vm.stats.vm.v_pfree: 252573950 vm.stats.vm.v_dfree: 55570643 vm.stats.vm.v_pdshortfalls: 3339 vm.stats.vm.v_pdpages: 3052333674 vm.stats.vm.v_pdwakeups: 2602 vm.stats.vm.v_reactivated: 8280838 vm.stats.vm.v_intrans: 2925212 vm.stats.vm.v_vnodepgsout: 10599080 vm.stats.vm.v_vnodepgsin: 1324593 vm.stats.vm.v_vnodeout: 1127536 vm.stats.vm.v_vnodein: 480221 vm.stats.vm.v_swappgsout: 21565595 vm.stats.vm.v_swappgsin: 9270389 vm.stats.vm.v_swapout: 1479419 vm.stats.vm.v_swapin: 2638252 vm.stats.vm.v_ozfod: 19873473 vm.stats.vm.v_zfod: 432099401 vm.stats.vm.v_cow_optim: 93465 vm.stats.vm.v_cow_faults: 115278434 vm.stats.vm.v_io_faults: 994151 vm.stats.vm.v_vm_faults: 632206903 vm.stats.sys.v_soft: 9284837 vm.stats.sys.v_intr: 18752370 vm.stats.sys.v_syscall: 657745130 vm.stats.sys.v_trap: 679806581 vm.stats.sys.v_swtch: 2272240391 vm.v_free_severe: 3328 vm.max_kernel_address: 4294963200 vm.min_kernel_address: 3220172800 vm.exec_map_entries: 16 vm.swap_idle_threshold2: 10 vm.swap_idle_threshold1: 2 vm.kstacks: 6819 vm.kstack_cache_size: 128 vm.zone_warnings: 1 vm.zone_count: 205 vm.nswapdev: 1 vm.dmmax: 32 vm.swap_async_max: 4 vm.swap_maxpages: 4194304 vm.swzone: 36175872 vm.overcommit: 0 vm.swap_reserved: 2165530624 vm.swap_total: 17179869184 vm.kmem_map_free: 343322624 vm.kmem_map_size: 193548288 vm.kmem_size_scale: 3 vm.kmem_size_max: 536870912 vm.kmem_size_min: 12582912 vm.kmem_zmax: 65536 vm.kmem_size: 536870912 vm.md_malloc_wait: 0 # # KDB: enter: Break to debugger [ thread pid 11 tid 100005 ] Stopped at kdb_alt_break_internal+0x162: movl $0,kdb_why db> show freepages DOMAIN: 0 FREE LIST 0: ORDER (SIZE) | NUMBER | POOL 0 -- -- -- -- 10 (004096K) | 000000 09 (002048K) | 000000 08 (001024K) | 000002 07 (000512K) | 000264 06 (000256K) | 002346 05 (000128K) | 005504 04 (000064K) | 008923 03 (000032K) | 012876 02 (000016K) | 016638 01 (000008K) | 019974 00 (000004K) | 028016 FREE LIST 1: ORDER (SIZE) | NUMBER | POOL 0 -- -- -- -- 10 (004096K) | 000000 09 (002048K) | 000000 08 (001024K) | 000000 07 (000512K) | 000000 06 (000256K) | 000000 05 (000128K) | 000000 04 (000064K) | 000002 03 (000032K) | 000010 02 (000016K) | 000035 01 (000008K) | 000053 00 (000004K) | 000079 db> show uma Zone Size Used Free Requests Sleeps Bucket FFS2 dinode 256 4938 44817 98661657 0 61 FFS1 dinode 128 0 0 3629419 0 125 FFS inode 112 4938 45217 102369008 0 125 SWAPMETA 276 1246 21014 789519 0 29 selfd 32 1027 1348 14662987 0 253 rtentry 112 13 92 13 0 125 ripcb 304 0 13 1145 0 29 udplite_inpcb 304 0 0 0 0 29 sctp_asconf_ack 24 0 0 0 0 253 sctp_asconf 24 0 0 0 0 253 sctp_stream_msg_out 72 0 0 0 0 125 sctp_readq 116 0 0 0 0 125 sctp_chunk 104 0 0 0 0 125 sctp_raddr 540 0 0 0 0 13 sctp_laddr 24 0 0 4 0 253 sctp_asoc 1656 0 0 0 0 5 sctp_ep 1052 0 0 0 0 9 tcpreass 20 0 1600 71463 0 253 sackhole 20 0 0 7115 0 253 hostcache 76 2 154 12 0 125 syncache 128 0 155 16033 0 125 tcptw 60 1 263 8049 0 253 tcpcb 776 0 60 44476 0 13 tcp_inpcb 304 1 116 44476 0 29 udpcb 20 2 798 141664 0 253 udp_inpcb 304 2 115 141664 0 29 ipq 32 0 0 1483867 0 253 unpcb 172 1 275 7233 0 61 socket 524 3 375 194525 0 13 KNOTE 72 0 385 14485070 0 125 itimer 232 0 0 0 0 61 ksiginfo 80 6734 1166 12658055 0 125 procdesc 80 0 0 1142 0 125 pipe 416 0 99 9672502 0 29 AIOLIO 172 0 0 0 0 61 AIOL 64 0 0 0 0 253 AIOCB 428 0 0 0 0 29 AIOP 16 0 0 0 0 253 AIO 128 0 0 0 0 125 Mountpoints 780 8 92 78888 0 13 DIRHASH 1024 572 220 1997415 0 13 NCLNODE 360 0 0 729225 0 29 rentr 12 0 0 0 0 253 NAMEI 1024 0 972 749352249 0 14 LTS VFS Cache 312 0 0 3183 0 29 L VFS Cache 292 1 12 935662 0 29 STS VFS Cache 92 0 0 1731950 0 125 S VFS Cache 72 3699 21326 60067468 0 126 BUF TRIE 44 507 5317 74027529 0 253 VNODEPOLL 64 1 61 256 0 253 VNODE 360 5073 44757 111446105 0 29 FPU_save_area 512 0 0 0 0 29 cryptodesc 56 0 0 0 0 253 cryptop 60 0 0 0 0 253 vtnet_tx_hdr 16 0 0 0 0 253 nvme_request 100 0 0 0 0 125 ttyoutq 256 5508 237 7914120 0 61 ttyinq 152 10335 377 14839035 0 61 g_bio 276 0 2436 358406033 0 47 mbuf_jumbo_16k 16384 0 0 0 0 1 mbuf_jumbo_9k 9216 0 0 0 0 1 mbuf_jumbo_page 4096 0 1 16783988 3179052 1 mbuf_cluster 2048 1268 8 10746823 0 5 mbuf 256 48 1279 244925948 0 253 mbuf_packet 256 256 1012 60235724 0 253 audit_record 1112 0 0 0 0 9 cpuset 40 99 501 42878 0 253 THREAD 836 6819 9513 6368752 0 13 PROC 840 24 18928 8478097 0 13 MAC labels 20 0 0 0 0 253 umtx_shm 52 0 0 0 0 253 umtx pi 52 0 0 1 0 253 TURNSTILE 72 16333 1988 16333 0 125 rl_entry 28 1843 1017 1843 0 253 filedesc0 888 25 1235 8477577 0 13 Files 56 344 65260 171164502 0 253 ptr pcpu 4 0 0 0 0 253 64 pcpu 8 2375 697 2655 0 253 SLEEPQUEUE 48 16333 1027 16333 0 253 65536 65536 1 0 1 0 1 65536 65536 8 0 9 0 1 65536 65536 1 2 564 0 1 65536 65536 0 0 0 0 1 65536 65536 0 0 974 0 1 65536 65536 3 0 351 0 1 65536 65536 2 0 1026 0 1 65536 65536 0 0 0 0 1 32768 32768 0 0 0 0 1 32768 32768 0 3 238148 0 1 32768 32768 0 3 169343 0 1 32768 32768 1 0 1 0 1 32768 32768 2 0 288 0 1 32768 32768 0 0 1224 0 1 32768 32768 0 0 934 0 1 32768 32768 5 0 42616 0 1 16384 16384 1 0 1 0 1 16384 16384 1 0 15542 0 1 16384 16384 4 0 31916 0 1 16384 16384 1 0 1 0 1 16384 16384 2 0 487 0 1 16384 16384 1 0 286024 0 1 16384 16384 0 8 2060 0 1 16384 16384 1 0 1 0 1 8192 8192 1 0 1 0 1 8192 8192 1 0 2 0 1 8192 8192 7 1 561576 0 1 8192 8192 1 0 1 0 1 8192 8192 10 1 43254 0 1 8192 8192 1 0 59526564 0 1 8192 8192 0 0 2177 0 1 8192 8192 0 0 0 0 1 4096 4096 2 0 123 0 1 4096 4096 27 0 28 0 1 4096 4096 6 0 2208733 0 1 4096 4096 16 4 42059411 0 1 4096 4096 39 1209 8563389 0 1 4096 4096 8 0 23602 0 1 4096 4096 0 1 2496 0 1 4096 4096 4 0 4 0 1 2048 2048 1913 3 777310 0 5 2048 2048 7 1 7 0 5 2048 2048 1 5 113824466 0 5 2048 2048 3 1 3 0 5 2048 2048 14 56 12973402 0 5 2048 2048 21 3 54059031 0 5 2048 2048 3 1 19892 0 5 2048 2048 2 2 2 0 5 1024 1024 1 3 94 0 13 1024 1024 16 8 18 0 13 1024 1024 7 53 922132315 0 13 1024 1024 2 6 2 0 13 1024 1024 13 27 61809 0 13 1024 1024 4 8 1509 0 13 1024 1024 4 24 9031 0 13 1024 1024 3 5 923 0 13 512 512 1 63 4057 0 29 512 512 36 28 11296968 0 29 512 512 0 120 44270577 0 29 512 512 3 117 41673160 0 29 512 512 15 41 1956780 0 29 512 512 5 11 684 0 29 512 512 719 249 1773860 0 29 512 512 0 0 0 0 29 256 256 3 27 623902 0 61 256 256 322 57383 91975803 0 61 256 256 79 251 10143079 0 61 256 256 19 161 5862 0 61 256 256 25810 1775 6489234 0 61 256 256 13728 267 2902429 0 61 256 256 802 623 33166976 0 61 256 256 1 14 1 0 61 128 128 40 394 363118 0 125 128 128 573 512 4169087 0 125 128 128 29763 524 26065848 0 125 128 128 28 158 473022 0 125 128 128 1693 6212 135053312 0 125 128 128 805 528 17751515 0 125 128 128 80 509 665859 0 125 128 128 19 1376 37226967 0 125 64 64 39 1077 516653 0 253 64 64 506 858 756606537 0 253 64 64 16982 44212 78787993 0 253 64 64 830 15166 94089669 0 253 64 64 1626 45432 577256934 0 253 64 64 28 2142 38635278 0 253 64 64 1656 762 160376 0 253 64 64 39 147 39 0 253 32 32 29 1596 11645921 0 253 32 32 671 204 5990277 0 253 32 32 1918 1832 42182734 0 253 32 32 2 873 1464983 0 253 32 32 1041 1084 250338502 0 253 32 32 173 64577 98300358 0 253 32 32 963 662 82729 0 253 32 32 71 554 2305174 0 253 16 16 41 1209 21615117 0 253 16 16 101 1149 384485 0 253 16 16 156 1094 90057796 0 253 16 16 17 1233 5142660 0 253 16 16 2524 1226 2453770 0 253 16 16 49 1201 725584 0 253 16 16 266 1234 2260008 0 253 16 16 5 1245 287308 0 253 mt_zone 2060 404 0 425 0 3 fakepg 68 0 522 1116 0 125 VMSPACE 232 2 38860 8478010 0 61 MAP ENTRY 72 22 17138 205158032 0 125 KMAP ENTRY 72 5 105 5 0 125 MAP 140 3 81 3 0 61 RADIX NODE 44 23623 125253 218236248 0 253 VM OBJECT 148 7279 54470 157537483 0 61 vmem btag 28 61364 161430 2885052 0 253 256 Bucket 1024 1006 562 20133173 0 253 128 Bucket 512 602 102 1677386 0 29 64 Bucket 256 2190 255 2300038 0 61 32 Bucket 128 1923 247 1428703 0 125 16 Bucket 64 255 799 1101901 0 253 12 Bucket 48 0 0 53129 0 253 8 Bucket 32 20 1105 964030 0 253 6 Bucket 24 0 334 206238 0 253 4 Bucket 16 1238 762 6542335 0 253 UMA Hash 128 4 27 15 0 0 UMA Slabs 88 4253 112 394535 0 0 UMA Zones 480 204 4 3809 0 0 UMA Kegs 256 203 7 3808 0 0 db> run pho db:0:pho> bt Tracing pid 11 tid 100005 td 0xc8e4e000 kdb_alt_break_internal(0,eaf2eac8,c0a5f7e7,2,c9053284,...) at kdb_alt_break_internal+0x162/frame 0xeaf2ea9c kdb_alt_break(2,c9053284,c159a25b,0,40000,...) at kdb_alt_break+0x16/frame 0xeaf2eaa8 uart_intr_rxready(c9053200,8,c15d85e3,88d,0,...) at uart_intr_rxready+0x87/frame 0xeaf2eac8 uart_intr(c9053200,2fc,0,c11f75bd,0,...) at uart_intr+0x110/frame 0xeaf2eb08 intr_event_handle(c8e0ab00,eaf2eb84,1,2,0,...) at intr_event_handle+0x7a/frame 0xeaf2eb34 intr_execute_handlers(c8ea4108,eaf2eb84,2) at intr_execute_handlers+0x42/frame 0xeaf2eb54 lapic_handle_intr(30,eaf2eb84) at lapic_handle_intr+0x5d/frame 0xeaf2eb74 Xapic_isr1() at Xapic_isr1+0x55/frame 0xeaf2eb74 --- interrupt, eip = 0xc1208045, esp = 0xeaf2ebc4, ebp = 0xeaf2ebc4 --- acpi_cpu_c1(eaf2ebd8,246,28aa6296,eaf2ec1c,c11f7f78,...) at acpi_cpu_c1+0x5/frame 0xeaf2ebc4 acpi_cpu_idle(1f79c164,0,1f79c164,eaf2ec38,c1208423,...) at acpi_cpu_idle+0x32a/frame 0xeaf2ec08 cpu_idle_acpi(1f79c164,0,c1aadc88,3,c8e4e000,...) at cpu_idle_acpi+0x3f/frame 0xeaf2ec1c cpu_idle(0,eaf2ec80,c15d85e3,a57,c1aace00,...) at cpu_idle+0xa3/frame 0xeaf2ec38 sched_idletd(0,eaf2ece8,c15cd71c,406,0,...) at sched_idletd+0x403/frame 0xeaf2ecac fork_exit(c0c82e10,0,eaf2ece8) at fork_exit+0x7e/frame 0xeaf2ecd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeaf2ecd4 --- trap 0, eip = 0, esp = 0xeaf2ed20, ebp = 0 --- db:0:bt> show allpcpu Current CPU: 2 cpuid = 0 dynamic pcpu = 0x763e80 curthread = 0xc8e4e6a0: pid 11 tid 100003 "idle: cpu0" curpcb = 0xeaf28d40 fpcurthread = none idlethread = 0xc8e4e6a0: tid 100003 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: cpuid = 1 dynamic pcpu = 0x29650e80 curthread = 0xc8e4e350: pid 11 tid 100004 "idle: cpu1" curpcb = 0xeaf2bd40 fpcurthread = none idlethread = 0xc8e4e350: tid 100004 "idle: cpu1" APIC ID = 1 currentldt = 0x50 spin locks held: cpuid = 2 dynamic pcpu = 0x29653e80 curthread = 0xc8e4e000: pid 11 tid 100005 "idle: cpu2" curpcb = 0xeaf2ed40 fpcurthread = none idlethread = 0xc8e4e000: tid 100005 "idle: cpu2" APIC ID = 2 currentldt = 0x50 spin locks held: cpuid = 3 dynamic pcpu = 0x29656e80 curthread = 0xc8e4d9f0: pid 11 tid 100006 "idle: cpu3" curpcb = 0xeaf31d40 fpcurthread = none idlethread = 0xc8e4d9f0: tid 100006 "idle: cpu3" APIC ID = 3 currentldt = 0x50 spin locks held: db:0:allpcpu> show alllocks db:0:alllocks> show lockedvnods Locked vnodes db:0:lockedvnods> show mount 0xc96fa000 /dev/ada0s1a on / (ufs) 0xc964a618 devfs on /dev (devfs) 0xc9a80924 /dev/ada0s1f on /home (ufs) 0xc9a8030c /dev/ada0s1d on /usr (ufs) 0xc9a80000 /dev/ada0s1e on /var (ufs) 0xc9a7fc30 procfs on /proc (procfs) 0xc96f9c30 /dev/label/tmp on /tmp (ufs) 0xc96fac30 /dev/md5a on /mnt (ufs) More info: show mount db:0:mount> show page vm_cnt.v_free_count: 741305 vm_cnt.v_inactive_count: 456 vm_cnt.v_active_count: 17228 vm_cnt.v_laundry_count: 25777 vm_cnt.v_wire_count: 77776 vm_cnt.v_free_reserved: 1173 vm_cnt.v_free_min: 5484 vm_cnt.v_free_target: 18417 vm_cnt.v_inactive_target: 27625 db:0:page> ps pid ppid pgrp uid state wmesg wchan cmd 91402 1 91402 0 Ss+ ttyin 0xc9053070 sh 90072 0 0 0 DL mdwait 0xcb3d7000 [md5] 22 0 0 0 DL syncer 0xc1b1a4a8 [syncer] 21 0 0 0 DL vlruwt 0xc93ba690 [vnlru] 20 0 0 0 DL (threaded) [bufdaemon] 100083 D psleep 0xc1b19184 [bufdaemon] 100104 D sdflush 0xc9060c84 [/home worker] 100106 D sdflush 0xc9061084 [/usr worker] 100107 D sdflush 0xc905fe84 [/var worker] 100241 D sdflush 0xca66e684 [/tmp worker] 101839 D sdflush 0xcf962284 [/mnt worker] 19 0 0 0 DL - 0xc1b19dcc [bufspacedaemon] 18 0 0 0 DL psleep 0xc1b1f828 [vmdaemon] 17 0 0 0 DL (threaded) [pagedaemon] 100080 D psleep 0xc1b7da85 [pagedaemon] 100087 D launds 0xc1b1f7f4 [laundry: dom0] 100088 D umarcl 0xc1b1f420 [uma] 16 0 0 0 DL - 0xc1b18bec [soaiod4] 15 0 0 0 DL - 0xc1b18bec [soaiod3] 9 0 0 0 DL - 0xc1b18bec [soaiod2] 8 0 0 0 DL - 0xc1b18bec [soaiod1] 7 0 0 0 DL - 0xc19780e4 [rand_harvestq] 6 0 0 0 DL waiting 0xc1b7935c [sctp_iterator] 5 0 0 0 DL - 0xc906063c [fdc0] 14 0 0 0 DL (threaded) [usb] 100033 D - 0xc9062acc [usbus0] 100034 D - 0xc9062afc [usbus0] 100035 D - 0xc9062b2c [usbus0] 100036 D - 0xc9062b5c [usbus0] 100037 D - 0xc9062b8c [usbus0] 100039 D - 0xc90c5acc [usbus1] 100040 D - 0xc90c5afc [usbus1] 100041 D - 0xc90c5b2c [usbus1] 100042 D - 0xc90c5b5c [usbus1] 100043 D - 0xc90c5b8c [usbus1] 100045 D - 0xc90d4acc [usbus2] 100046 D - 0xc90d4afc [usbus2] 100047 D - 0xc90d4b2c [usbus2] 100048 D - 0xc90d4b5c [usbus2] 100049 D - 0xc90d4b8c [usbus2] 100050 D - 0xc90daacc [usbus3] 100051 D - 0xc90daafc [usbus3] 100052 D - 0xc90dab2c [usbus3] 100053 D - 0xc90dab5c [usbus3] 100054 D - 0xc90dab8c [usbus3] 100055 D - 0xc90e8acc [usbus4] 100056 D - 0xc90e8afc [usbus4] 100057 D - 0xc90e8b2c [usbus4] 100058 D - 0xc90e8b5c [usbus4] 100059 D - 0xc90e8b8c [usbus4] 100061 D - 0xc908eca4 [usbus5] 100062 D - 0xc908ecd4 [usbus5] 100063 D - 0xc908ed04 [usbus5] 100064 D - 0xc908ed34 [usbus5] 100065 D - 0xc908ed64 [usbus5] 4 0 0 0 DL (threaded) [cam] 100025 D - 0xc18c4180 [doneq0] 100075 D - 0xc18c402c [scanner] 3 0 0 0 DL crypto_ 0xc1b1e7a0 [crypto returns] 2 0 0 0 DL crypto_ 0xc1b1e6e8 [crypto] 13 0 0 0 DL (threaded) [geom] 100020 D - 0xc1b74464 [g_event] 100021 D - 0xc1b74468 [g_up] 100022 D - 0xc1b7446c [g_down] 12 0 0 0 WL (threaded) [intr] 100007 I [swi4: clock (0)] 100008 I [swi4: clock (1)] 100009 I [swi4: clock (2)] 100010 I [swi4: clock (3)] 100011 I [swi1: netisr 0] 100012 I [swi3: vm] 100014 I [swi6: task queue] 100015 I [swi6: Giant taskq] 100018 I [swi5: fast taskq] 100029 I [irq256: hdac0] 100030 I [irq257: re0] 100031 I [irq22: ahci0] 100032 I [irq16: hdac1 ohci0] 100038 I [irq17: ohci1 ohci3] 100044 I [irq18: ohci2 ohci4] 100060 I [irq19: ehci0] 100066 I [irq14: ata0] 100067 I [swi0: uart] 100069 I [irq1: atkbd0] 100070 I [irq12: psm0] 11 0 0 0 RL (threaded) [idle] 100003 Run CPU 0 [idle: cpu0] 100004 Run CPU 1 [idle: cpu1] 100005 Run CPU 2 [idle: cpu2] 100006 Run CPU 3 [idle: cpu3] 1 0 1 0 SLs wait 0xc8e4a9d8 [init] 10 0 0 0 DL audit_w 0xc1b7c26c [audit] 0 0 0 0 DLs (threaded) [kernel] 100000 D swapin 0xc1b74484 [swapper] 100013 D - 0xc8f1c000 [kqueue_ctx taskq] 100016 D - 0xc8f1be00 [thread taskq] 100017 D - 0xc8f1bd80 [aiod_kick taskq] 100019 D - 0xc8f1bc80 [firmware taskq] 100026 D - 0xc8f1b880 [acpi_task_0] 100027 D - 0xc8f1b880 [acpi_task_1] 100028 D - 0xc8f1b880 [acpi_task_2] 100071 D - 0xc913e980 [mca taskq] 100074 D - 0xc8f1ba80 [CAM taskq] 100086 D - 0xc913d380 [if_config_tqg_0] 100089 D - 0xc913d300 [if_io_tqg_0] 100090 D - 0xc913d280 [if_io_tqg_1] 100091 D - 0xc913d200 [if_io_tqg_2] 100092 D - 0xc913d180 [if_io_tqg_3] db:0:ps> allt Tracing command sh pid 91402 tid 107636 td 0xd21049f0 sched_switch(d21049f0,0,104,190,ffb8194c,...) at sched_switch+0x4e1/frame 0xffb81914 mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xffb8194c sleepq_switch(d21049f0,0,c15dc5e5,1c2,c9053014,...) at sleepq_switch+0xf0/frame 0xffb81980 sleepq_catch_signals(ffb819e8,c0c330b0,c9053014,0,c187fcb0,...) at sleepq_catch_signals+0x443/frame 0xffb819c4 sleepq_wait_sig(c9053070,0,ffb81a20,101,0,...) at sleepq_wait_sig+0x14/frame 0xffb819e8 _cv_wait_sig(c9053070,c9053004,c15e20a5,580,1,...) at _cv_wait_sig+0x211/frame 0xffb81a34 tty_wait(c9053000,c9053070,ffb81b90,1,0,...) at tty_wait+0x4b/frame 0xffb81a58 ttydisc_read(c9053000,ffb81b90,0,ab,c1888e7c,...) at ttydisc_read+0x3a7/frame 0xffb81a9c ttydev_read(c8e9c700,ffb81b90,0,0,d21049f0,...) at ttydev_read+0x55/frame 0xffb81abc devfs_read_f(c9781268,ffb81b90,c8d01c00,0,d21049f0,...) at devfs_read_f+0xc7/frame 0xffb81af4 dofileread(c9781268,ffb81b90,ffffffff,ffffffff,0,...) at dofileread+0xb6/frame 0xffb81b28 kern_readv(d21049f0,0,ffb81b90,2,bfbfed06,...) at kern_readv+0x93/frame 0xffb81b70 sys_read(d21049f0,ffb81c50,c15cd1c7,4,0,...) at sys_read+0x59/frame 0xffb81bc0 syscall(ffb81ce8) at syscall+0x3d3/frame 0xffb81cdc Xint0x80_syscall() at Xint0x80_syscall+0x2e/frame 0xffb81cdc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x28203e83, esp = 0xbfbfecc4, ebp = 0xbfbfecd8 --- Tracing command md5 pid 90072 tid 103225 td 0xcc239000 sched_switch(cc239000,0,104,190,fb20bbb4,...) at sched_switch+0x4e1/frame 0xfb20bb7c mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xfb20bbb4 sleepq_switch(cc239000,0,c15dc5e5,28b,cc239000,...) at sleepq_switch+0xf0/frame 0xfb20bbe8 sleepq_wait(cb3d7000,5c,c13056aa,0,0,...) at sleepq_wait+0x48/frame 0xfb20bc0c _sleep(cb3d7000,cb3d7020,25c,c13056aa,0,...) at _sleep+0x342/frame 0xfb20bc68 md_kthread(cb3d7000,fb20bce8,c15cd71c,406,0,...) at md_kthread+0x1c7/frame 0xfb20bcac fork_exit(c07ee590,cb3d7000,fb20bce8) at fork_exit+0x7e/frame 0xfb20bcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xfb20bcd4 --- trap 0, eip = 0, esp = 0xfb20bd20, ebp = 0 --- Tracing command syncer pid 22 tid 100085 td 0xc93c4000 sched_switch(c93c4000,0,104,190,f4310b90,...) at sched_switch+0x4e1/frame 0xf4310b58 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf4310b90 sleepq_switch(c93c4000,0,c15dc5e5,2ae,0,...) at sleepq_switch+0xf0/frame 0xf4310bc4 sleepq_timedwait(c1b1a4a8,0,f4310c24,0,0,...) at sleepq_timedwait+0x48/frame 0xf4310be8 _cv_timedwait_sbt(c1b1a4a8,c1b1a488,fffffed8,0,0,...) at _cv_timedwait_sbt+0x23c/frame 0xf4310c38 sched_sync(0,f4310ce8,c15cd71c,406,0,...) at sched_sync+0x6c0/frame 0xf4310cac fork_exit(c0d23580,0,f4310ce8) at fork_exit+0x7e/frame 0xf4310cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4310cd4 --- trap 0, eip = 0, esp = 0xf4310d20, ebp = 0 --- Tracing command vnlru pid 21 tid 100084 td 0xc93c4350 sched_switch(c93c4350,0,104,190,f430db84,...) at sched_switch+0x4e1/frame 0xf430db4c mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xf430db84 sleepq_switch(c93c4350,0,c15dc5e5,2ae,c93c4350,...) at sleepq_switch+0xf0/frame 0xf430dbb8 sleepq_timedwait(c93ba690,60,0,0,0,...) at sleepq_timedwait+0x48/frame 0xf430dbdc _sleep(c93ba690,c1b1a454,260,c15e999b,fffffed8,...) at _sleep+0x316/frame 0xf430dc38 vnlru_proc(0,f430dce8,c15cd71c,406,0,...) at vnlru_proc+0xbf/frame 0xf430dcac fork_exit(c0d22c70,0,f430dce8) at fork_exit+0x7e/frame 0xf430dcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf430dcd4 --- trap 0, eip = 0, esp = 0xf430dd20, ebp = 0 --- Tracing command bufdaemon pid 20 tid 100083 td 0xc93c46a0 sched_switch(c93c46a0,0,104,190,f430abc4,...) at sched_switch+0x4e1/frame 0xf430ab8c mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xf430abc4 sleepq_switch(c93c46a0,0,c15dc5e5,2ae,c93c46a0,...) at sleepq_switch+0xf0/frame 0xf430abf8 sleepq_timedwait(c1b19184,54,0,0,0,...) at sleepq_timedwait+0x48/frame 0xf430ac1c _sleep(c1b19184,c1b19100,54,c15e66be,fffffed8,...) at _sleep+0x316/frame 0xf430ac78 buf_daemon(0,f430ace8,c15cd71c,406,0,...) at buf_daemon+0x8c/frame 0xf430acac fork_exit(c0d018f0,0,f430ace8) at fork_exit+0x7e/frame 0xf430acd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf430acd4 --- trap 0, eip = 0, esp = 0xf430ad20, ebp = 0 --- Tracing command bufdaemon pid 20 tid 100104 td 0xc982f350 sched_switch(c982f350,0,104,190,f435bbb4,...) at sched_switch+0x4e1/frame 0xf435bb7c mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xf435bbb4 sleepq_switch(c982f350,0,c15dc5e5,2ae,c982f350,...) at sleepq_switch+0xf0/frame 0xf435bbe8 sleepq_timedwait(c9060c84,54,0,0,0,...) at sleepq_timedwait+0x48/frame 0xf435bc0c _sleep(c9060c84,c9060c00,54,c161b7d9,7fffff6c,...) at _sleep+0x316/frame 0xf435bc68 softdep_flush(c9a80924,f435bce8,c15cd71c,406,0,...) at softdep_flush+0x192/frame 0xf435bcac fork_exit(c0f297b0,c9a80924,f435bce8) at fork_exit+0x7e/frame 0xf435bcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf435bcd4 --- trap 0, eip = 0, esp = 0xf435bd20, ebp = 0 --- Tracing command bufdaemon pid 20 tid 100106 td 0xc982e9f0 sched_switch(c982e9f0,0,104,190,f4361bb4,...) at sched_switch+0x4e1/frame 0xf4361b7c mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xf4361bb4 sleepq_switch(c982e9f0,0,c15dc5e5,2ae,c982e9f0,...) at sleepq_switch+0xf0/frame 0xf4361be8 sleepq_timedwait(c9061084,54,0,0,0,...) at sleepq_timedwait+0x48/frame 0xf4361c0c _sleep(c9061084,c9061000,54,c161b7d9,7fffff6c,...) at _sleep+0x316/frame 0xf4361c68 softdep_flush(c9a8030c,f4361ce8,c15cd71c,406,0,...) at softdep_flush+0x192/frame 0xf4361cac fork_exit(c0f297b0,c9a8030c,f4361ce8) at fork_exit+0x7e/frame 0xf4361cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4361cd4 --- trap 0, eip = 0, esp = 0xf4361d20, ebp = 0 --- Tracing command bufdaemon pid 20 tid 100107 td 0xc982e6a0 sched_switch(c982e6a0,0,104,190,f4364bb4,...) at sched_switch+0x4e1/frame 0xf4364b7c mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xf4364bb4 sleepq_switch(c982e6a0,0,c15dc5e5,2ae,c982e6a0,...) at sleepq_switch+0xf0/frame 0xf4364be8 sleepq_timedwait(c905fe84,54,0,0,0,...) at sleepq_timedwait+0x48/frame 0xf4364c0c _sleep(c905fe84,c905fe00,54,c161b7d9,7fffff6c,...) at _sleep+0x316/frame 0xf4364c68 softdep_flush(c9a80000,f4364ce8,c15cd71c,406,0,...) at softdep_flush+0x192/frame 0xf4364cac fork_exit(c0f297b0,c9a80000,f4364ce8) at fork_exit+0x7e/frame 0xf4364cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4364cd4 --- trap 0, eip = 0, esp = 0xf4364d20, ebp = 0 --- Tracing command bufdaemon pid 20 tid 100241 td 0xca5fe000 sched_switch(ca5fe000,0,104,190,f9e7bbb4,...) at sched_switch+0x4e1/frame 0xf9e7bb7c mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xf9e7bbb4 sleepq_switch(ca5fe000,0,c15dc5e5,2ae,ca5fe000,...) at sleepq_switch+0xf0/frame 0xf9e7bbe8 sleepq_timedwait(ca66e684,54,0,0,0,...) at sleepq_timedwait+0x48/frame 0xf9e7bc0c _sleep(ca66e684,ca66e600,54,c161b7d9,7fffff6c,...) at _sleep+0x316/frame 0xf9e7bc68 softdep_flush(c96f9c30,f9e7bce8,c15cd71c,406,0,...) at softdep_flush+0x192/frame 0xf9e7bcac fork_exit(c0f297b0,c96f9c30,f9e7bce8) at fork_exit+0x7e/frame 0xf9e7bcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf9e7bcd4 --- trap 0, eip = 0, esp = 0xf9e7bd20, ebp = 0 --- Tracing command bufdaemon pid 20 tid 101839 td 0xd6cda6a0 sched_switch(d6cda6a0,0,104,190,ff6e2bb4,...) at sched_switch+0x4e1/frame 0xff6e2b7c mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xff6e2bb4 sleepq_switch(d6cda6a0,0,c15dc5e5,2ae,d6cda6a0,...) at sleepq_switch+0xf0/frame 0xff6e2be8 sleepq_timedwait(cf962284,54,0,0,0,...) at sleepq_timedwait+0x48/frame 0xff6e2c0c _sleep(cf962284,cf962200,54,c161b7d9,7fffff6c,...) at _sleep+0x316/frame 0xff6e2c68 softdep_flush(c96fac30,ff6e2ce8,c15cd71c,406,0,...) at softdep_flush+0x192/frame 0xff6e2cac fork_exit(c0f297b0,c96fac30,ff6e2ce8) at fork_exit+0x7e/frame 0xff6e2cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xff6e2cd4 --- trap 0, eip = 0, esp = 0xff6e2d20, ebp = 0 --- Tracing command bufspacedaemon pid 19 tid 100082 td 0xc93c49f0 sched_switch(c93c49f0,0,104,190,f4307bc4,...) at sched_switch+0x4e1/frame 0xf4307b8c mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xf4307bc4 sleepq_switch(c93c49f0,0,c15dc5e5,2ae,c93c49f0,...) at sleepq_switch+0xf0/frame 0xf4307bf8 sleepq_timedwait(c1b19dcc,5c,0,0,0,...) at sleepq_timedwait+0x48/frame 0xf4307c1c _sleep(c1b19dcc,c1b19b80,25c,c15bee13,fffffed8,...) at _sleep+0x316/frame 0xf4307c78 bufspace_daemon(0,f4307ce8,c15cd71c,406,0,...) at bufspace_daemon+0x1ff/frame 0xf4307cac fork_exit(c0d012a0,0,f4307ce8) at fork_exit+0x7e/frame 0xf4307cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4307cd4 --- trap 0, eip = 0, esp = 0xf4307d20, ebp = 0 --- Tracing command vmdaemon pid 18 tid 100081 td 0xc93c5000 sched_switch(c93c5000,0,104,190,f4304b84,...) at sched_switch+0x4e1/frame 0xf4304b4c mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xf4304b84 sleepq_switch(c93c5000,0,c15dc5e5,28b,c93c5000,...) at sleepq_switch+0xf0/frame 0xf4304bb8 sleepq_wait(c1b1f828,74,c15e66be,0,0,...) at sleepq_wait+0x48/frame 0xf4304bdc _sleep(c1b1f828,c1b1f810,74,c15e66be,0,...) at _sleep+0x342/frame 0xf4304c38 vm_daemon(0,f4304ce8,c15cd71c,406,0,...) at vm_daemon+0xb6/frame 0xf4304cac fork_exit(c0f86450,0,f4304ce8) at fork_exit+0x7e/frame 0xf4304cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4304cd4 --- trap 0, eip = 0, esp = 0xf4304d20, ebp = 0 --- Tracing command pagedaemon pid 17 tid 100080 td 0xc93c5350 sched_switch(c93c5350,0,104,190,f4301b9c,...) at sched_switch+0x4e1/frame 0xf4301b64 mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xf4301b9c sleepq_switch(c93c5350,0,c15dc5e5,2ae,c93c5350,...) at sleepq_switch+0xf0/frame 0xf4301bd0 sleepq_timedwait(c1b7da85,54,0,0,0,...) at sleepq_timedwait+0x48/frame 0xf4301bf4 _sleep(c1b7da85,c1b7da00,254,c15e66be,fffffed8,...) at _sleep+0x316/frame 0xf4301c50 vm_pageout(0,f4301ce8,c15cd71c,406,0,...) at vm_pageout+0x307/frame 0xf4301cac fork_exit(c0f84060,0,f4301ce8) at fork_exit+0x7e/frame 0xf4301cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4301cd4 --- trap 0, eip = 0, esp = 0xf4301d20, ebp = 0 --- Tracing command pagedaemon pid 17 tid 100087 td 0xc960d9f0 sched_switch(c960d9f0,0,104,190,f4316b5c,...) at sched_switch+0x4e1/frame 0xf4316b24 mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xf4316b5c sleepq_switch(c960d9f0,0,c15dc5e5,28b,c960d9f0,...) at sleepq_switch+0xf0/frame 0xf4316b90 sleepq_wait(c1b1f7f4,54,c16238b6,0,0,...) at sleepq_wait+0x48/frame 0xf4316bb4 _sleep(c1b1f7f4,c1b7d800,54,c16238b6,0,...) at _sleep+0x342/frame 0xf4316c10 vm_pageout_laundry_worker(0,f4316ce8,c15cd71c,406,0,...) at vm_pageout_laundry_worker+0xb4c/frame 0xf4316cac fork_exit(c0f85020,0,f4316ce8) at fork_exit+0x7e/frame 0xf4316cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4316cd4 --- trap 0, eip = 0, esp = 0xf4316d20, ebp = 0 --- Tracing command pagedaemon pid 17 tid 100088 td 0xc960d6a0 sched_switch(c960d6a0,0,104,190,f4319bcc,...) at sched_switch+0x4e1/frame 0xf4319b94 mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xf4319bcc sleepq_switch(c960d6a0,0,c15dc5e5,28b,c960d6a0,...) at sleepq_switch+0xf0/frame 0xf4319c00 sleepq_wait(c1b1f420,54,f4319c6c,0,0,...) at sleepq_wait+0x48/frame 0xf4319c24 _sleep(c1b1f420,c1b1f40c,54,c16202d4,0,...) at _sleep+0x342/frame 0xf4319c80 uma_reclaim_worker(0,f4319ce8,c15cd71c,406,0,...) at uma_reclaim_worker+0x90/frame 0xf4319cac fork_exit(c0f66250,0,f4319ce8) at fork_exit+0x7e/frame 0xf4319cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4319cd4 --- trap 0, eip = 0, esp = 0xf4319d20, ebp = 0 --- Tracing command soaiod4 pid 16 tid 100079 td 0xc93c56a0 sched_switch(c93c56a0,0,104,190,f42febbc,...) at sched_switch+0x4e1/frame 0xf42feb84 mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xf42febbc sleepq_switch(c93c56a0,0,c15dc5e5,2ae,c93c56a0,...) at sleepq_switch+0xf0/frame 0xf42febf0 sleepq_timedwait(c1b18bec,0,1d,0,0,...) at sleepq_timedwait+0x48/frame 0xf42fec14 _sleep(c1b18bec,c1b18bcc,0,c15bee13,ffffdd50,...) at _sleep+0x316/frame 0xf42fec70 soaio_kproc_loop(4,f42fece8,c15cd71c,406,0,...) at soaio_kproc_loop+0x13a/frame 0xf42fecac fork_exit(c0cc5f80,4,f42fece8) at fork_exit+0x7e/frame 0xf42fecd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf42fecd4 --- trap 0, eip = 0, esp = 0xf42fed20, ebp = 0 --- Tracing command soaiod3 pid 15 tid 100078 td 0xc90e7350 sched_switch(c90e7350,0,104,190,f42fbbbc,...) at sched_switch+0x4e1/frame 0xf42fbb84 mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xf42fbbbc sleepq_switch(c90e7350,0,c15dc5e5,2ae,c90e7350,...) at sleepq_switch+0xf0/frame 0xf42fbbf0 sleepq_timedwait(c1b18bec,0,1d,0,0,...) at sleepq_timedwait+0x48/frame 0xf42fbc14 _sleep(c1b18bec,c1b18bcc,0,c15bee13,ffffdd50,...) at _sleep+0x316/frame 0xf42fbc70 soaio_kproc_loop(3,f42fbce8,c15cd71c,406,0,...) at soaio_kproc_loop+0x13a/frame 0xf42fbcac fork_exit(c0cc5f80,3,f42fbce8) at fork_exit+0x7e/frame 0xf42fbcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf42fbcd4 --- trap 0, eip = 0, esp = 0xf42fbd20, ebp = 0 --- Tracing command soaiod2 pid 9 tid 100077 td 0xc90e76a0 sched_switch(c90e76a0,0,104,190,f42f8bbc,...) at sched_switch+0x4e1/frame 0xf42f8b84 mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xf42f8bbc sleepq_switch(c90e76a0,0,c15dc5e5,2ae,c90e76a0,...) at sleepq_switch+0xf0/frame 0xf42f8bf0 sleepq_timedwait(c1b18bec,0,1d,0,0,...) at sleepq_timedwait+0x48/frame 0xf42f8c14 _sleep(c1b18bec,c1b18bcc,0,c15bee13,ffffdd50,...) at _sleep+0x316/frame 0xf42f8c70 soaio_kproc_loop(2,f42f8ce8,c15cd71c,406,0,...) at soaio_kproc_loop+0x13a/frame 0xf42f8cac fork_exit(c0cc5f80,2,f42f8ce8) at fork_exit+0x7e/frame 0xf42f8cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf42f8cd4 --- trap 0, eip = 0, esp = 0xf42f8d20, ebp = 0 --- Tracing command soaiod1 pid 8 tid 100076 td 0xc90e79f0 sched_switch(c90e79f0,0,104,190,f42f5bbc,...) at sched_switch+0x4e1/frame 0xf42f5b84 mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xf42f5bbc sleepq_switch(c90e79f0,0,c15dc5e5,2ae,c90e79f0,...) at sleepq_switch+0xf0/frame 0xf42f5bf0 sleepq_timedwait(c1b18bec,0,1d,0,0,...) at sleepq_timedwait+0x48/frame 0xf42f5c14 _sleep(c1b18bec,c1b18bcc,0,c15bee13,ffffdd50,...) at _sleep+0x316/frame 0xf42f5c70 soaio_kproc_loop(1,f42f5ce8,c15cd71c,406,0,...) at soaio_kproc_loop+0x13a/frame 0xf42f5cac fork_exit(c0cc5f80,1,f42f5ce8) at fork_exit+0x7e/frame 0xf42f5cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf42f5cd4 --- trap 0, eip = 0, esp = 0xf42f5d20, ebp = 0 --- Tracing command rand_harvestq pid 7 tid 100073 td 0xc91116a0 sched_switch(c91116a0,0,104,190,f416cb9c,...) at sched_switch+0x4e1/frame 0xf416cb64 mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xf416cb9c sleepq_switch(c91116a0,0,c15dc5e5,2ae,c91116a0,...) at sleepq_switch+0xf0/frame 0xf416cbd0 sleepq_timedwait(c19780e4,0,0,0,0,...) at sleepq_timedwait+0x48/frame 0xf416cbf4 _sleep(c19780e4,0,0,c15bee13,19999999,...) at _sleep+0x316/frame 0xf416cc50 random_kthread(0,f416cce8,c15cd71c,406,0,...) at random_kthread+0x24b/frame 0xf416ccac fork_exit(c09753a0,0,f416cce8) at fork_exit+0x7e/frame 0xf416ccd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf416ccd4 --- trap 0, eip = 0, esp = 0xf416cd20, ebp = 0 --- Tracing command sctp_iterator pid 6 tid 100072 td 0xc91119f0 sched_switch(c91119f0,0,104,190,f4169bcc,...) at sched_switch+0x4e1/frame 0xf4169b94 mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xf4169bcc sleepq_switch(c91119f0,0,c15dc5e5,28b,c91119f0,...) at sleepq_switch+0xf0/frame 0xf4169c00 sleepq_wait(c1b7935c,0,c16006f1,0,0,...) at sleepq_wait+0x48/frame 0xf4169c24 _sleep(c1b7935c,c1b79324,0,c16006f1,0,...) at _sleep+0x342/frame 0xf4169c80 sctp_iterator_thread(0,f4169ce8,c15cd71c,406,0,...) at sctp_iterator_thread+0x7c/frame 0xf4169cac fork_exit(c0ddeec0,0,f4169ce8) at fork_exit+0x7e/frame 0xf4169cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4169cd4 --- trap 0, eip = 0, esp = 0xf4169d20, ebp = 0 --- Tracing command fdc0 pid 5 tid 100068 td 0xc91129f0 sched_switch(c91129f0,0,104,190,f415cb78,...) at sched_switch+0x4e1/frame 0xf415cb40 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf415cb78 sleepq_switch(c91129f0,0,c15dc5e5,28b,c91129f0,...) at sleepq_switch+0xf0/frame 0xf415cbac sleepq_wait(c906063c,5c,c15bee13,0,0,...) at sleepq_wait+0x48/frame 0xf415cbd0 _sleep(c906063c,c90606f0,5c,c15bee13,0,...) at _sleep+0x342/frame 0xf415cc2c fdc_thread(c9060600,f415cce8,c15cd71c,406,0,...) at fdc_thread+0x12fc/frame 0xf415ccac fork_exit(c11596e0,c9060600,f415cce8) at fork_exit+0x7e/frame 0xf415ccd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf415ccd4 --- trap 0, eip = 0, esp = 0xf415cd20, ebp = 0 --- Tracing command usb pid 14 tid 100033 td 0xc902c000 sched_switch(c902c000,0,104,190,f40d3be8,...) at sched_switch+0x4e1/frame 0xf40d3bb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf40d3be8 sleepq_switch(c902c000,0,c15dc5e5,28b,c9062acc,...) at sleepq_switch+0xf0/frame 0xf40d3c1c sleepq_wait(c9062acc,0,f40d3c70,1,0,...) at sleepq_wait+0x48/frame 0xf40d3c40 _cv_wait(c9062acc,c9062cf8,c15a364a,76,c9062ad4,...) at _cv_wait+0x20b/frame 0xf40d3c84 usb_process(c9062ac4,f40d3ce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf40d3cac fork_exit(c0a9a200,c9062ac4,f40d3ce8) at fork_exit+0x7e/frame 0xf40d3cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf40d3cd4 --- trap 0, eip = 0, esp = 0xf40d3d20, ebp = 0 --- Tracing command usb pid 14 tid 100034 td 0xc902b9f0 sched_switch(c902b9f0,0,104,190,f40d6be8,...) at sched_switch+0x4e1/frame 0xf40d6bb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf40d6be8 sleepq_switch(c902b9f0,0,c15dc5e5,28b,c9062afc,...) at sleepq_switch+0xf0/frame 0xf40d6c1c sleepq_wait(c9062afc,0,f40d6c70,1,0,...) at sleepq_wait+0x48/frame 0xf40d6c40 _cv_wait(c9062afc,c9062cf8,c15a364a,76,c9062b04,...) at _cv_wait+0x20b/frame 0xf40d6c84 usb_process(c9062af4,f40d6ce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf40d6cac fork_exit(c0a9a200,c9062af4,f40d6ce8) at fork_exit+0x7e/frame 0xf40d6cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf40d6cd4 --- trap 0, eip = 0, esp = 0xf40d6d20, ebp = 0 --- Tracing command usb pid 14 tid 100035 td 0xc902b6a0 sched_switch(c902b6a0,0,104,190,f40d9be8,...) at sched_switch+0x4e1/frame 0xf40d9bb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf40d9be8 sleepq_switch(c902b6a0,0,c15dc5e5,28b,c9062b2c,...) at sleepq_switch+0xf0/frame 0xf40d9c1c sleepq_wait(c9062b2c,0,f40d9c70,1,0,...) at sleepq_wait+0x48/frame 0xf40d9c40 _cv_wait(c9062b2c,c9062cf8,c15a364a,76,c9062b34,...) at _cv_wait+0x20b/frame 0xf40d9c84 usb_process(c9062b24,f40d9ce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf40d9cac fork_exit(c0a9a200,c9062b24,f40d9ce8) at fork_exit+0x7e/frame 0xf40d9cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf40d9cd4 --- trap 0, eip = 0, esp = 0xf40d9d20, ebp = 0 --- Tracing command usb pid 14 tid 100036 td 0xc902b350 sched_switch(c902b350,0,104,190,f40dcbe8,...) at sched_switch+0x4e1/frame 0xf40dcbb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf40dcbe8 sleepq_switch(c902b350,0,c15dc5e5,28b,c9062b5c,...) at sleepq_switch+0xf0/frame 0xf40dcc1c sleepq_wait(c9062b5c,0,f40dcc70,1,0,...) at sleepq_wait+0x48/frame 0xf40dcc40 _cv_wait(c9062b5c,c9062cf8,c15a364a,76,c9062b64,...) at _cv_wait+0x20b/frame 0xf40dcc84 usb_process(c9062b54,f40dcce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf40dccac fork_exit(c0a9a200,c9062b54,f40dcce8) at fork_exit+0x7e/frame 0xf40dccd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf40dccd4 --- trap 0, eip = 0, esp = 0xf40dcd20, ebp = 0 --- Tracing command usb pid 14 tid 100037 td 0xc902b000 sched_switch(c902b000,0,104,190,f40dfbe8,...) at sched_switch+0x4e1/frame 0xf40dfbb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf40dfbe8 sleepq_switch(c902b000,0,c15dc5e5,28b,c9062b8c,...) at sleepq_switch+0xf0/frame 0xf40dfc1c sleepq_wait(c9062b8c,0,f40dfc70,1,0,...) at sleepq_wait+0x48/frame 0xf40dfc40 _cv_wait(c9062b8c,c9062cf8,c15a364a,76,c9062b94,...) at _cv_wait+0x20b/frame 0xf40dfc84 usb_process(c9062b84,f40dfce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf40dfcac fork_exit(c0a9a200,c9062b84,f40dfce8) at fork_exit+0x7e/frame 0xf40dfcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf40dfcd4 --- trap 0, eip = 0, esp = 0xf40dfd20, ebp = 0 --- Tracing command usb pid 14 tid 100039 td 0xc8f246a0 sched_switch(c8f246a0,0,104,190,f40e6be8,...) at sched_switch+0x4e1/frame 0xf40e6bb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf40e6be8 sleepq_switch(c8f246a0,0,c15dc5e5,28b,c90c5acc,...) at sleepq_switch+0xf0/frame 0xf40e6c1c sleepq_wait(c90c5acc,0,f40e6c70,1,0,...) at sleepq_wait+0x48/frame 0xf40e6c40 _cv_wait(c90c5acc,c90c5cf8,c15a364a,76,c90c5ad4,...) at _cv_wait+0x20b/frame 0xf40e6c84 usb_process(c90c5ac4,f40e6ce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf40e6cac fork_exit(c0a9a200,c90c5ac4,f40e6ce8) at fork_exit+0x7e/frame 0xf40e6cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf40e6cd4 --- trap 0, eip = 0, esp = 0xf40e6d20, ebp = 0 --- Tracing command usb pid 14 tid 100040 td 0xc90cd9f0 sched_switch(c90cd9f0,0,104,190,f40e9be8,...) at sched_switch+0x4e1/frame 0xf40e9bb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf40e9be8 sleepq_switch(c90cd9f0,0,c15dc5e5,28b,c90c5afc,...) at sleepq_switch+0xf0/frame 0xf40e9c1c sleepq_wait(c90c5afc,0,f40e9c70,1,0,...) at sleepq_wait+0x48/frame 0xf40e9c40 _cv_wait(c90c5afc,c90c5cf8,c15a364a,76,c90c5b04,...) at _cv_wait+0x20b/frame 0xf40e9c84 usb_process(c90c5af4,f40e9ce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf40e9cac fork_exit(c0a9a200,c90c5af4,f40e9ce8) at fork_exit+0x7e/frame 0xf40e9cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf40e9cd4 --- trap 0, eip = 0, esp = 0xf40e9d20, ebp = 0 --- Tracing command usb pid 14 tid 100041 td 0xc90cd6a0 sched_switch(c90cd6a0,0,104,190,f40ecbe8,...) at sched_switch+0x4e1/frame 0xf40ecbb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf40ecbe8 sleepq_switch(c90cd6a0,0,c15dc5e5,28b,c90c5b2c,...) at sleepq_switch+0xf0/frame 0xf40ecc1c sleepq_wait(c90c5b2c,0,f40ecc70,1,0,...) at sleepq_wait+0x48/frame 0xf40ecc40 _cv_wait(c90c5b2c,c90c5cf8,c15a364a,76,c90c5b34,...) at _cv_wait+0x20b/frame 0xf40ecc84 usb_process(c90c5b24,f40ecce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf40eccac fork_exit(c0a9a200,c90c5b24,f40ecce8) at fork_exit+0x7e/frame 0xf40eccd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf40eccd4 --- trap 0, eip = 0, esp = 0xf40ecd20, ebp = 0 --- Tracing command usb pid 14 tid 100042 td 0xc90cd350 sched_switch(c90cd350,0,104,190,f40efbe8,...) at sched_switch+0x4e1/frame 0xf40efbb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf40efbe8 sleepq_switch(c90cd350,0,c15dc5e5,28b,c90c5b5c,...) at sleepq_switch+0xf0/frame 0xf40efc1c sleepq_wait(c90c5b5c,0,f40efc70,1,0,...) at sleepq_wait+0x48/frame 0xf40efc40 _cv_wait(c90c5b5c,c90c5cf8,c15a364a,76,c90c5b64,...) at _cv_wait+0x20b/frame 0xf40efc84 usb_process(c90c5b54,f40efce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf40efcac fork_exit(c0a9a200,c90c5b54,f40efce8) at fork_exit+0x7e/frame 0xf40efcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf40efcd4 --- trap 0, eip = 0, esp = 0xf40efd20, ebp = 0 --- Tracing command usb pid 14 tid 100043 td 0xc90cd000 sched_switch(c90cd000,0,104,190,f40f2be8,...) at sched_switch+0x4e1/frame 0xf40f2bb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf40f2be8 sleepq_switch(c90cd000,0,c15dc5e5,28b,c90c5b8c,...) at sleepq_switch+0xf0/frame 0xf40f2c1c sleepq_wait(c90c5b8c,0,f40f2c70,1,0,...) at sleepq_wait+0x48/frame 0xf40f2c40 _cv_wait(c90c5b8c,c90c5cf8,c15a364a,76,c90c5b94,...) at _cv_wait+0x20b/frame 0xf40f2c84 usb_process(c90c5b84,f40f2ce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf40f2cac fork_exit(c0a9a200,c90c5b84,f40f2ce8) at fork_exit+0x7e/frame 0xf40f2cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf40f2cd4 --- trap 0, eip = 0, esp = 0xf40f2d20, ebp = 0 --- Tracing command usb pid 14 tid 100045 td 0xc90cc6a0 sched_switch(c90cc6a0,0,104,190,f40f9be8,...) at sched_switch+0x4e1/frame 0xf40f9bb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf40f9be8 sleepq_switch(c90cc6a0,0,c15dc5e5,28b,c90d4acc,...) at sleepq_switch+0xf0/frame 0xf40f9c1c sleepq_wait(c90d4acc,0,f40f9c70,1,0,...) at sleepq_wait+0x48/frame 0xf40f9c40 _cv_wait(c90d4acc,c90d4cf8,c15a364a,76,c90d4ad4,...) at _cv_wait+0x20b/frame 0xf40f9c84 usb_process(c90d4ac4,f40f9ce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf40f9cac fork_exit(c0a9a200,c90d4ac4,f40f9ce8) at fork_exit+0x7e/frame 0xf40f9cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf40f9cd4 --- trap 0, eip = 0, esp = 0xf40f9d20, ebp = 0 --- Tracing command usb pid 14 tid 100046 td 0xc90cc350 sched_switch(c90cc350,0,104,190,f40fcbe8,...) at sched_switch+0x4e1/frame 0xf40fcbb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf40fcbe8 sleepq_switch(c90cc350,0,c15dc5e5,28b,c90d4afc,...) at sleepq_switch+0xf0/frame 0xf40fcc1c sleepq_wait(c90d4afc,0,f40fcc70,1,0,...) at sleepq_wait+0x48/frame 0xf40fcc40 _cv_wait(c90d4afc,c90d4cf8,c15a364a,76,c90d4b04,...) at _cv_wait+0x20b/frame 0xf40fcc84 usb_process(c90d4af4,f40fcce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf40fccac fork_exit(c0a9a200,c90d4af4,f40fcce8) at fork_exit+0x7e/frame 0xf40fccd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf40fccd4 --- trap 0, eip = 0, esp = 0xf40fcd20, ebp = 0 --- Tracing command usb pid 14 tid 100047 td 0xc90cc000 sched_switch(c90cc000,0,104,190,f40ffbe8,...) at sched_switch+0x4e1/frame 0xf40ffbb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf40ffbe8 sleepq_switch(c90cc000,0,c15dc5e5,28b,c90d4b2c,...) at sleepq_switch+0xf0/frame 0xf40ffc1c sleepq_wait(c90d4b2c,0,f40ffc70,1,0,...) at sleepq_wait+0x48/frame 0xf40ffc40 _cv_wait(c90d4b2c,c90d4cf8,c15a364a,76,c90d4b34,...) at _cv_wait+0x20b/frame 0xf40ffc84 usb_process(c90d4b24,f40ffce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf40ffcac fork_exit(c0a9a200,c90d4b24,f40ffce8) at fork_exit+0x7e/frame 0xf40ffcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf40ffcd4 --- trap 0, eip = 0, esp = 0xf40ffd20, ebp = 0 --- Tracing command usb pid 14 tid 100048 td 0xc90cb9f0 sched_switch(c90cb9f0,0,104,190,f4102be8,...) at sched_switch+0x4e1/frame 0xf4102bb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf4102be8 sleepq_switch(c90cb9f0,0,c15dc5e5,28b,c90d4b5c,...) at sleepq_switch+0xf0/frame 0xf4102c1c sleepq_wait(c90d4b5c,0,f4102c70,1,0,...) at sleepq_wait+0x48/frame 0xf4102c40 _cv_wait(c90d4b5c,c90d4cf8,c15a364a,76,c90d4b64,...) at _cv_wait+0x20b/frame 0xf4102c84 usb_process(c90d4b54,f4102ce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf4102cac fork_exit(c0a9a200,c90d4b54,f4102ce8) at fork_exit+0x7e/frame 0xf4102cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4102cd4 --- trap 0, eip = 0, esp = 0xf4102d20, ebp = 0 --- Tracing command usb pid 14 tid 100049 td 0xc90cb6a0 sched_switch(c90cb6a0,0,104,190,f4105be8,...) at sched_switch+0x4e1/frame 0xf4105bb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf4105be8 sleepq_switch(c90cb6a0,0,c15dc5e5,28b,c90d4b8c,...) at sleepq_switch+0xf0/frame 0xf4105c1c sleepq_wait(c90d4b8c,0,f4105c70,1,0,...) at sleepq_wait+0x48/frame 0xf4105c40 _cv_wait(c90d4b8c,c90d4cf8,c15a364a,76,c90d4b94,...) at _cv_wait+0x20b/frame 0xf4105c84 usb_process(c90d4b84,f4105ce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf4105cac fork_exit(c0a9a200,c90d4b84,f4105ce8) at fork_exit+0x7e/frame 0xf4105cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4105cd4 --- trap 0, eip = 0, esp = 0xf4105d20, ebp = 0 --- Tracing command usb pid 14 tid 100050 td 0xc90cb350 sched_switch(c90cb350,0,104,190,f4109be8,...) at sched_switch+0x4e1/frame 0xf4109bb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf4109be8 sleepq_switch(c90cb350,0,c15dc5e5,28b,c90daacc,...) at sleepq_switch+0xf0/frame 0xf4109c1c sleepq_wait(c90daacc,0,f4109c70,1,0,...) at sleepq_wait+0x48/frame 0xf4109c40 _cv_wait(c90daacc,c90dacf8,c15a364a,76,c90daad4,...) at _cv_wait+0x20b/frame 0xf4109c84 usb_process(c90daac4,f4109ce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf4109cac fork_exit(c0a9a200,c90daac4,f4109ce8) at fork_exit+0x7e/frame 0xf4109cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4109cd4 --- trap 0, eip = 0, esp = 0xf4109d20, ebp = 0 --- Tracing command usb pid 14 tid 100051 td 0xc90cb000 sched_switch(c90cb000,0,104,190,f410cbe8,...) at sched_switch+0x4e1/frame 0xf410cbb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf410cbe8 sleepq_switch(c90cb000,0,c15dc5e5,28b,c90daafc,...) at sleepq_switch+0xf0/frame 0xf410cc1c sleepq_wait(c90daafc,0,f410cc70,1,0,...) at sleepq_wait+0x48/frame 0xf410cc40 _cv_wait(c90daafc,c90dacf8,c15a364a,76,c90dab04,...) at _cv_wait+0x20b/frame 0xf410cc84 usb_process(c90daaf4,f410cce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf410ccac fork_exit(c0a9a200,c90daaf4,f410cce8) at fork_exit+0x7e/frame 0xf410ccd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf410ccd4 --- trap 0, eip = 0, esp = 0xf410cd20, ebp = 0 --- Tracing command usb pid 14 tid 100052 td 0xc902d9f0 sched_switch(c902d9f0,0,104,190,f410fbe8,...) at sched_switch+0x4e1/frame 0xf410fbb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf410fbe8 sleepq_switch(c902d9f0,0,c15dc5e5,28b,c90dab2c,...) at sleepq_switch+0xf0/frame 0xf410fc1c sleepq_wait(c90dab2c,0,f410fc70,1,0,...) at sleepq_wait+0x48/frame 0xf410fc40 _cv_wait(c90dab2c,c90dacf8,c15a364a,76,c90dab34,...) at _cv_wait+0x20b/frame 0xf410fc84 usb_process(c90dab24,f410fce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf410fcac fork_exit(c0a9a200,c90dab24,f410fce8) at fork_exit+0x7e/frame 0xf410fcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf410fcd4 --- trap 0, eip = 0, esp = 0xf410fd20, ebp = 0 --- Tracing command usb pid 14 tid 100053 td 0xc90e7000 sched_switch(c90e7000,0,104,190,f4112be8,...) at sched_switch+0x4e1/frame 0xf4112bb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf4112be8 sleepq_switch(c90e7000,0,c15dc5e5,28b,c90dab5c,...) at sleepq_switch+0xf0/frame 0xf4112c1c sleepq_wait(c90dab5c,0,f4112c70,1,0,...) at sleepq_wait+0x48/frame 0xf4112c40 _cv_wait(c90dab5c,c90dacf8,c15a364a,76,c90dab64,...) at _cv_wait+0x20b/frame 0xf4112c84 usb_process(c90dab54,f4112ce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf4112cac fork_exit(c0a9a200,c90dab54,f4112ce8) at fork_exit+0x7e/frame 0xf4112cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4112cd4 --- trap 0, eip = 0, esp = 0xf4112d20, ebp = 0 --- Tracing command usb pid 14 tid 100054 td 0xc90e69f0 sched_switch(c90e69f0,0,104,190,f4115be8,...) at sched_switch+0x4e1/frame 0xf4115bb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf4115be8 sleepq_switch(c90e69f0,0,c15dc5e5,28b,c90dab8c,...) at sleepq_switch+0xf0/frame 0xf4115c1c sleepq_wait(c90dab8c,0,f4115c70,1,0,...) at sleepq_wait+0x48/frame 0xf4115c40 _cv_wait(c90dab8c,c90dacf8,c15a364a,76,c90dab94,...) at _cv_wait+0x20b/frame 0xf4115c84 usb_process(c90dab84,f4115ce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf4115cac fork_exit(c0a9a200,c90dab84,f4115ce8) at fork_exit+0x7e/frame 0xf4115cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4115cd4 --- trap 0, eip = 0, esp = 0xf4115d20, ebp = 0 --- Tracing command usb pid 14 tid 100055 td 0xc90e66a0 sched_switch(c90e66a0,0,104,190,f4119be8,...) at sched_switch+0x4e1/frame 0xf4119bb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf4119be8 sleepq_switch(c90e66a0,0,c15dc5e5,28b,c90e8acc,...) at sleepq_switch+0xf0/frame 0xf4119c1c sleepq_wait(c90e8acc,0,f4119c70,1,0,...) at sleepq_wait+0x48/frame 0xf4119c40 _cv_wait(c90e8acc,c90e8cf8,c15a364a,76,c90e8ad4,...) at _cv_wait+0x20b/frame 0xf4119c84 usb_process(c90e8ac4,f4119ce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf4119cac fork_exit(c0a9a200,c90e8ac4,f4119ce8) at fork_exit+0x7e/frame 0xf4119cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4119cd4 --- trap 0, eip = 0, esp = 0xf4119d20, ebp = 0 --- Tracing command usb pid 14 tid 100056 td 0xc90e6350 sched_switch(c90e6350,0,104,190,f411cbe8,...) at sched_switch+0x4e1/frame 0xf411cbb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf411cbe8 sleepq_switch(c90e6350,0,c15dc5e5,28b,c90e8afc,...) at sleepq_switch+0xf0/frame 0xf411cc1c sleepq_wait(c90e8afc,0,f411cc70,1,0,...) at sleepq_wait+0x48/frame 0xf411cc40 _cv_wait(c90e8afc,c90e8cf8,c15a364a,76,c90e8b04,...) at _cv_wait+0x20b/frame 0xf411cc84 usb_process(c90e8af4,f411cce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf411ccac fork_exit(c0a9a200,c90e8af4,f411cce8) at fork_exit+0x7e/frame 0xf411ccd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf411ccd4 --- trap 0, eip = 0, esp = 0xf411cd20, ebp = 0 --- Tracing command usb pid 14 tid 100057 td 0xc90e6000 sched_switch(c90e6000,0,104,190,f411fbe8,...) at sched_switch+0x4e1/frame 0xf411fbb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf411fbe8 sleepq_switch(c90e6000,0,c15dc5e5,28b,c90e8b2c,...) at sleepq_switch+0xf0/frame 0xf411fc1c sleepq_wait(c90e8b2c,0,f411fc70,1,0,...) at sleepq_wait+0x48/frame 0xf411fc40 _cv_wait(c90e8b2c,c90e8cf8,c15a364a,76,c90e8b34,...) at _cv_wait+0x20b/frame 0xf411fc84 usb_process(c90e8b24,f411fce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf411fcac fork_exit(c0a9a200,c90e8b24,f411fce8) at fork_exit+0x7e/frame 0xf411fcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf411fcd4 --- trap 0, eip = 0, esp = 0xf411fd20, ebp = 0 --- Tracing command usb pid 14 tid 100058 td 0xc90e59f0 sched_switch(c90e59f0,0,104,190,f4122be8,...) at sched_switch+0x4e1/frame 0xf4122bb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf4122be8 sleepq_switch(c90e59f0,0,c15dc5e5,28b,c90e8b5c,...) at sleepq_switch+0xf0/frame 0xf4122c1c sleepq_wait(c90e8b5c,0,f4122c70,1,0,...) at sleepq_wait+0x48/frame 0xf4122c40 _cv_wait(c90e8b5c,c90e8cf8,c15a364a,76,c90e8b64,...) at _cv_wait+0x20b/frame 0xf4122c84 usb_process(c90e8b54,f4122ce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf4122cac fork_exit(c0a9a200,c90e8b54,f4122ce8) at fork_exit+0x7e/frame 0xf4122cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4122cd4 --- trap 0, eip = 0, esp = 0xf4122d20, ebp = 0 --- Tracing command usb pid 14 tid 100059 td 0xc90e56a0 sched_switch(c90e56a0,0,104,190,f4125be8,...) at sched_switch+0x4e1/frame 0xf4125bb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf4125be8 sleepq_switch(c90e56a0,0,c15dc5e5,28b,c90e8b8c,...) at sleepq_switch+0xf0/frame 0xf4125c1c sleepq_wait(c90e8b8c,0,f4125c70,1,0,...) at sleepq_wait+0x48/frame 0xf4125c40 _cv_wait(c90e8b8c,c90e8cf8,c15a364a,76,c90e8b94,...) at _cv_wait+0x20b/frame 0xf4125c84 usb_process(c90e8b84,f4125ce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf4125cac fork_exit(c0a9a200,c90e8b84,f4125ce8) at fork_exit+0x7e/frame 0xf4125cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4125cd4 --- trap 0, eip = 0, esp = 0xf4125d20, ebp = 0 --- Tracing command usb pid 14 tid 100061 td 0xc90e5000 sched_switch(c90e5000,0,104,190,f412dbe8,...) at sched_switch+0x4e1/frame 0xf412dbb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf412dbe8 sleepq_switch(c90e5000,0,c15dc5e5,28b,c908eca4,...) at sleepq_switch+0xf0/frame 0xf412dc1c sleepq_wait(c908eca4,0,f412dc70,1,0,...) at sleepq_wait+0x48/frame 0xf412dc40 _cv_wait(c908eca4,c908eed0,c15a364a,76,c908ecac,...) at _cv_wait+0x20b/frame 0xf412dc84 usb_process(c908ec9c,f412dce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf412dcac fork_exit(c0a9a200,c908ec9c,f412dce8) at fork_exit+0x7e/frame 0xf412dcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf412dcd4 --- trap 0, eip = 0, esp = 0xf412dd20, ebp = 0 --- Tracing command usb pid 14 tid 100062 td 0xc90e09f0 sched_switch(c90e09f0,0,104,190,f4130be8,...) at sched_switch+0x4e1/frame 0xf4130bb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf4130be8 sleepq_switch(c90e09f0,0,c15dc5e5,28b,c908ecd4,...) at sleepq_switch+0xf0/frame 0xf4130c1c sleepq_wait(c908ecd4,0,f4130c70,1,0,...) at sleepq_wait+0x48/frame 0xf4130c40 _cv_wait(c908ecd4,c908eed0,c15a364a,76,c908ecdc,...) at _cv_wait+0x20b/frame 0xf4130c84 usb_process(c908eccc,f4130ce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf4130cac fork_exit(c0a9a200,c908eccc,f4130ce8) at fork_exit+0x7e/frame 0xf4130cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4130cd4 --- trap 0, eip = 0, esp = 0xf4130d20, ebp = 0 --- Tracing command usb pid 14 tid 100063 td 0xc90e06a0 sched_switch(c90e06a0,0,104,190,f4133be8,...) at sched_switch+0x4e1/frame 0xf4133bb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf4133be8 sleepq_switch(c90e06a0,0,c15dc5e5,28b,c908ed04,...) at sleepq_switch+0xf0/frame 0xf4133c1c sleepq_wait(c908ed04,0,f4133c70,1,0,...) at sleepq_wait+0x48/frame 0xf4133c40 _cv_wait(c908ed04,c908eed0,c15a364a,76,c908ed0c,...) at _cv_wait+0x20b/frame 0xf4133c84 usb_process(c908ecfc,f4133ce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf4133cac fork_exit(c0a9a200,c908ecfc,f4133ce8) at fork_exit+0x7e/frame 0xf4133cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4133cd4 --- trap 0, eip = 0, esp = 0xf4133d20, ebp = 0 --- Tracing command usb pid 14 tid 100064 td 0xc90e0350 sched_switch(c90e0350,0,104,190,f4136be8,...) at sched_switch+0x4e1/frame 0xf4136bb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf4136be8 sleepq_switch(c90e0350,0,c15dc5e5,28b,c908ed34,...) at sleepq_switch+0xf0/frame 0xf4136c1c sleepq_wait(c908ed34,0,f4136c70,1,0,...) at sleepq_wait+0x48/frame 0xf4136c40 _cv_wait(c908ed34,c908eed0,c15a364a,76,c908ed3c,...) at _cv_wait+0x20b/frame 0xf4136c84 usb_process(c908ed2c,f4136ce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf4136cac fork_exit(c0a9a200,c908ed2c,f4136ce8) at fork_exit+0x7e/frame 0xf4136cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4136cd4 --- trap 0, eip = 0, esp = 0xf4136d20, ebp = 0 --- Tracing command usb pid 14 tid 100065 td 0xc90e0000 sched_switch(c90e0000,0,104,190,f4139be8,...) at sched_switch+0x4e1/frame 0xf4139bb0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf4139be8 sleepq_switch(c90e0000,0,c15dc5e5,28b,c908ed64,...) at sleepq_switch+0xf0/frame 0xf4139c1c sleepq_wait(c908ed64,0,f4139c70,1,0,...) at sleepq_wait+0x48/frame 0xf4139c40 _cv_wait(c908ed64,c908eed0,c15a364a,76,c908ed6c,...) at _cv_wait+0x20b/frame 0xf4139c84 usb_process(c908ed5c,f4139ce8,c15cd71c,406,0,...) at usb_process+0x150/frame 0xf4139cac fork_exit(c0a9a200,c908ed5c,f4139ce8) at fork_exit+0x7e/frame 0xf4139cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4139cd4 --- trap 0, eip = 0, esp = 0xf4139d20, ebp = 0 --- Tracing command cam pid 4 tid 100025 td 0xc8e4f6a0 sched_switch(c8e4f6a0,0,104,190,eaf6bba4,...) at sched_switch+0x4e1/frame 0xeaf6bb6c mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xeaf6bba4 sleepq_switch(c8e4f6a0,0,c15dc5e5,28b,c8e4f6a0,...) at sleepq_switch+0xf0/frame 0xeaf6bbd8 sleepq_wait(c18c4180,5c,c15bee13,0,0,...) at sleepq_wait+0x48/frame 0xeaf6bbfc _sleep(c18c4180,c18c4100,5c,c15bee13,0,...) at _sleep+0x342/frame 0xeaf6bc58 xpt_done_td(c18c4100,eaf6bce8,c15cd71c,406,0,...) at xpt_done_td+0x17e/frame 0xeaf6bcac fork_exit(c04dd790,c18c4100,eaf6bce8) at fork_exit+0x7e/frame 0xeaf6bcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeaf6bcd4 --- trap 0, eip = 0, esp = 0xeaf6bd20, ebp = 0 --- Tracing command cam pid 4 tid 100075 td 0xc9111000 sched_switch(c9111000,0,104,190,f4172ba4,...) at sched_switch+0x4e1/frame 0xf4172b6c mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xf4172ba4 sleepq_switch(c9111000,0,c15dc5e5,28b,c9111000,...) at sleepq_switch+0xf0/frame 0xf4172bd8 sleepq_wait(c18c402c,5c,c15bee13,0,0,...) at sleepq_wait+0x48/frame 0xf4172bfc _sleep(c18c402c,c18c407c,5c,c15bee13,0,...) at _sleep+0x342/frame 0xf4172c58 xpt_scanner_thread(0,f4172ce8,c15cd71c,406,0,...) at xpt_scanner_thread+0xa2/frame 0xf4172cac fork_exit(c04ddb40,0,f4172ce8) at fork_exit+0x7e/frame 0xf4172cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4172cd4 --- trap 0, eip = 0, esp = 0xf4172d20, ebp = 0 --- Tracing command crypto returns pid 3 tid 100024 td 0xc8e4f9f0 sched_switch(c8e4f9f0,0,104,190,eaf68bac,...) at sched_switch+0x4e1/frame 0xeaf68b74 mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xeaf68bac sleepq_switch(c8e4f9f0,0,c15dc5e5,28b,c8e4f9f0,...) at sleepq_switch+0xf0/frame 0xeaf68be0 sleepq_wait(c1b1e7a0,6c,c16135e0,0,0,...) at sleepq_wait+0x48/frame 0xeaf68c04 _sleep(c1b1e7a0,c1b1e78c,6c,c16135e0,0,...) at _sleep+0x342/frame 0xeaf68c60 crypto_ret_proc(0,eaf68ce8,c15cd71c,406,0,...) at crypto_ret_proc+0x1f5/frame 0xeaf68cac fork_exit(c0ed1310,0,eaf68ce8) at fork_exit+0x7e/frame 0xeaf68cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeaf68cd4 --- trap 0, eip = 0, esp = 0xeaf68d20, ebp = 0 --- Tracing command crypto pid 2 tid 100023 td 0xc8f22000 sched_switch(c8f22000,0,104,190,eaf65bbc,...) at sched_switch+0x4e1/frame 0xeaf65b84 mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xeaf65bbc sleepq_switch(c8f22000,0,c15dc5e5,28b,c8f22000,...) at sleepq_switch+0xf0/frame 0xeaf65bf0 sleepq_wait(c1b1e6e8,6c,c16135d4,0,0,...) at sleepq_wait+0x48/frame 0xeaf65c14 _sleep(c1b1e6e8,c1b1e6d0,6c,c16135d4,0,...) at _sleep+0x342/frame 0xeaf65c70 crypto_proc(0,eaf65ce8,c15cd71c,406,0,...) at crypto_proc+0x2c1/frame 0xeaf65cac fork_exit(c0ed0ee0,0,eaf65ce8) at fork_exit+0x7e/frame 0xeaf65cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeaf65cd4 --- trap 0, eip = 0, esp = 0xeaf65d20, ebp = 0 --- Tracing command geom pid 13 tid 100020 td 0xc8f229f0 sched_switch(c8f229f0,0,104,190,eaf5cbbc,...) at sched_switch+0x4e1/frame 0xeaf5cb84 mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xeaf5cbbc sleepq_switch(c8f229f0,0,c15dc5e5,28b,c8f229f0,...) at sleepq_switch+0xf0/frame 0xeaf5cbf0 sleepq_wait(c1b74464,5c,c15bee13,0,0,...) at sleepq_wait+0x48/frame 0xeaf5cc14 _sleep(c1b74464,c1a9cabc,25c,c15bee13,0,...) at _sleep+0x342/frame 0xeaf5cc70 g_run_events(0,eaf5cce8,c15cd71c,406,0,...) at g_run_events+0x67/frame 0xeaf5ccac fork_exit(c0b9d640,0,eaf5cce8) at fork_exit+0x7e/frame 0xeaf5ccd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeaf5ccd4 --- trap 0, eip = 0, esp = 0xeaf5cd20, ebp = 0 --- Tracing command geom pid 13 tid 100021 td 0xc8f226a0 sched_switch(c8f226a0,0,104,190,eaf5fbac,...) at sched_switch+0x4e1/frame 0xeaf5fb74 mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xeaf5fbac sleepq_switch(c8f226a0,0,c15dc5e5,28b,c8f226a0,...) at sleepq_switch+0xf0/frame 0xeaf5fbe0 sleepq_wait(c1b74468,5c,c15bee13,0,0,...) at sleepq_wait+0x48/frame 0xeaf5fc04 _sleep(c1b74468,c1a9cb04,25c,c15bee13,0,...) at _sleep+0x342/frame 0xeaf5fc60 g_io_schedule_up(c8f226a0,0,c15c0c72,5f,eaf5fcd4,...) at g_io_schedule_up+0xcf/frame 0xeaf5fc94 g_up_procbody(0,eaf5fce8,c15cd71c,406,0,...) at g_up_procbody+0x7d/frame 0xeaf5fcac fork_exit(c0b9d6c0,0,eaf5fce8) at fork_exit+0x7e/frame 0xeaf5fcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeaf5fcd4 --- trap 0, eip = 0, esp = 0xeaf5fd20, ebp = 0 --- Tracing command geom pid 13 tid 100022 td 0xc8f22350 sched_switch(c8f22350,0,104,190,eaf62ba8,...) at sched_switch+0x4e1/frame 0xeaf62b70 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xeaf62ba8 sleepq_switch(c8f22350,0,c15dc5e5,28b,c8f22350,...) at sleepq_switch+0xf0/frame 0xeaf62bdc sleepq_wait(c1b7446c,5c,c15bee13,0,0,...) at sleepq_wait+0x48/frame 0xeaf62c00 _sleep(c1b7446c,c1a9cae4,25c,c15bee13,0,...) at _sleep+0x342/frame 0xeaf62c5c g_io_schedule_down(c8f22350,0,c15c0c72,6b,eaf62cd4,...) at g_io_schedule_down+0xec/frame 0xeaf62c94 g_down_procbody(0,eaf62ce8,c15cd71c,406,0,...) at g_down_procbody+0x7d/frame 0xeaf62cac fork_exit(c0b9d740,0,eaf62ce8) at fork_exit+0x7e/frame 0xeaf62cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeaf62cd4 --- trap 0, eip = 0, esp = 0xeaf62d20, ebp = 0 --- Tracing command intr pid 12 tid 100007 td 0xc8e4d6a0 sched_switch(c8e4d6a0,0,109,190,c187ff48,...) at sched_switch+0x4e1/frame 0xeaf35c38 mi_switch(109,0,c15cd9a1,555,c15cd71c,...) at mi_switch+0x18c/frame 0xeaf35c70 ithread_loop(c8e413f0,eaf35ce8,c15cd71c,406,0,...) at ithread_loop+0x1f0/frame 0xeaf35cac fork_exit(c0c17ff0,c8e413f0,eaf35ce8) at fork_exit+0x7e/frame 0xeaf35cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeaf35cd4 --- trap 0, eip = 0, esp = 0xeaf35d20, ebp = 0 --- Tracing command intr pid 12 tid 100008 td 0xc8e4d350 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100009 td 0xc8e4d000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100010 td 0xc8e4c9f0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100011 td 0xc8e4c6a0 sched_switch(c8e4c6a0,0,109,190,c187ff48,...) at sched_switch+0x4e1/frame 0xeaf41c38 mi_switch(109,0,c15cd9a1,555,c15cd71c,...) at mi_switch+0x18c/frame 0xeaf41c70 ithread_loop(c8e413b0,eaf41ce8,c15cd71c,406,0,...) at ithread_loop+0x1f0/frame 0xeaf41cac fork_exit(c0c17ff0,c8e413b0,eaf41ce8) at fork_exit+0x7e/frame 0xeaf41cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeaf41cd4 --- trap 0, eip = 0, esp = 0xeaf41d20, ebp = 0 --- Tracing command intr pid 12 tid 100012 td 0xc8e4c350 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100014 td 0xc8f24350 sched_switch(c8f24350,0,109,190,c187ff48,...) at sched_switch+0x4e1/frame 0xeaf4ac38 mi_switch(109,0,c15cd9a1,555,c15cd71c,...) at mi_switch+0x18c/frame 0xeaf4ac70 ithread_loop(c8e41390,eaf4ace8,c15cd71c,406,0,...) at ithread_loop+0x1f0/frame 0xeaf4acac fork_exit(c0c17ff0,c8e41390,eaf4ace8) at fork_exit+0x7e/frame 0xeaf4acd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeaf4acd4 --- trap 0, eip = 0, esp = 0xeaf4ad20, ebp = 0 --- Tracing command intr pid 12 tid 100015 td 0xc8f24000 sched_switch(c8f24000,0,109,190,c187ff48,...) at sched_switch+0x4e1/frame 0xeaf4dc38 mi_switch(109,0,c15cd9a1,555,c15cd71c,...) at mi_switch+0x18c/frame 0xeaf4dc70 ithread_loop(c8e41380,eaf4dce8,c15cd71c,406,0,...) at ithread_loop+0x1f0/frame 0xeaf4dcac fork_exit(c0c17ff0,c8e41380,eaf4dce8) at fork_exit+0x7e/frame 0xeaf4dcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeaf4dcd4 --- trap 0, eip = 0, esp = 0xeaf4dd20, ebp = 0 --- Tracing command intr pid 12 tid 100018 td 0xc8f23350 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100029 td 0xc902d000 sched_switch(c902d000,0,109,190,c187ff48,...) at sched_switch+0x4e1/frame 0xeafa6c38 mi_switch(109,0,c15cd9a1,555,c15cd71c,...) at mi_switch+0x18c/frame 0xeafa6c70 ithread_loop(c8e411d0,eafa6ce8,c15cd71c,406,0,...) at ithread_loop+0x1f0/frame 0xeafa6cac fork_exit(c0c17ff0,c8e411d0,eafa6ce8) at fork_exit+0x7e/frame 0xeafa6cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeafa6cd4 --- trap 0, eip = 0, esp = 0xeafa6d20, ebp = 0 --- Tracing command intr pid 12 tid 100030 td 0xc902c9f0 sched_switch(c902c9f0,0,109,190,c187ff48,...) at sched_switch+0x4e1/frame 0xf76aec38 mi_switch(109,0,c15cd9a1,555,c15cd71c,...) at mi_switch+0x18c/frame 0xf76aec70 ithread_loop(c8e411e0,f76aece8,c15cd71c,406,0,...) at ithread_loop+0x1f0/frame 0xf76aecac fork_exit(c0c17ff0,c8e411e0,f76aece8) at fork_exit+0x7e/frame 0xf76aecd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf76aecd4 --- trap 0, eip = 0, esp = 0xf76aed20, ebp = 0 --- Tracing command intr pid 12 tid 100031 td 0xc902c6a0 sched_switch(c902c6a0,0,109,190,c187ff48,...) at sched_switch+0x4e1/frame 0xf76b2c38 mi_switch(109,0,c15cd9a1,555,c15cd71c,...) at mi_switch+0x18c/frame 0xf76b2c70 ithread_loop(c8e411c0,f76b2ce8,c15cd71c,406,0,...) at ithread_loop+0x1f0/frame 0xf76b2cac fork_exit(c0c17ff0,c8e411c0,f76b2ce8) at fork_exit+0x7e/frame 0xf76b2cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf76b2cd4 --- trap 0, eip = 0, esp = 0xf76b2d20, ebp = 0 --- Tracing command intr pid 12 tid 100032 td 0xc902c350 sched_switch(c902c350,0,109,190,c187ff48,...) at sched_switch+0x4e1/frame 0xf40d0c38 mi_switch(109,0,c15cd9a1,555,c15cd71c,...) at mi_switch+0x18c/frame 0xf40d0c70 ithread_loop(c8e410f0,f40d0ce8,c15cd71c,406,0,...) at ithread_loop+0x1f0/frame 0xf40d0cac fork_exit(c0c17ff0,c8e410f0,f40d0ce8) at fork_exit+0x7e/frame 0xf40d0cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf40d0cd4 --- trap 0, eip = 0, esp = 0xf40d0d20, ebp = 0 --- Tracing command intr pid 12 tid 100038 td 0xc8f249f0 sched_switch(c8f249f0,0,109,190,c187ff48,...) at sched_switch+0x4e1/frame 0xf40e3c38 mi_switch(109,0,c15cd9a1,555,c15cd71c,...) at mi_switch+0x18c/frame 0xf40e3c70 ithread_loop(c8e410e0,f40e3ce8,c15cd71c,406,0,...) at ithread_loop+0x1f0/frame 0xf40e3cac fork_exit(c0c17ff0,c8e410e0,f40e3ce8) at fork_exit+0x7e/frame 0xf40e3cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf40e3cd4 --- trap 0, eip = 0, esp = 0xf40e3d20, ebp = 0 --- Tracing command intr pid 12 tid 100044 td 0xc90cc9f0 sched_switch(c90cc9f0,0,109,190,c187ff48,...) at sched_switch+0x4e1/frame 0xf40f6c38 mi_switch(109,0,c15cd9a1,555,c15cd71c,...) at mi_switch+0x18c/frame 0xf40f6c70 ithread_loop(c8e410d0,f40f6ce8,c15cd71c,406,0,...) at ithread_loop+0x1f0/frame 0xf40f6cac fork_exit(c0c17ff0,c8e410d0,f40f6ce8) at fork_exit+0x7e/frame 0xf40f6cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf40f6cd4 --- trap 0, eip = 0, esp = 0xf40f6d20, ebp = 0 --- Tracing command intr pid 12 tid 100060 td 0xc90e5350 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100066 td 0xc9113350 sched_switch(c9113350,0,109,190,c187ff48,...) at sched_switch+0x4e1/frame 0xf413ec38 mi_switch(109,0,c15cd9a1,555,c15cd71c,...) at mi_switch+0x18c/frame 0xf413ec70 ithread_loop(c8e410b0,f413ece8,c15cd71c,406,0,...) at ithread_loop+0x1f0/frame 0xf413ecac fork_exit(c0c17ff0,c8e410b0,f413ece8) at fork_exit+0x7e/frame 0xf413ecd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf413ecd4 --- trap 0, eip = 0, esp = 0xf413ed20, ebp = 0 --- Tracing command intr pid 12 tid 100067 td 0xc9113000 sched_switch(c9113000,0,109,190,c187ff48,...) at sched_switch+0x4e1/frame 0xf414fc38 mi_switch(109,0,c15cd9a1,555,c15cd71c,...) at mi_switch+0x18c/frame 0xf414fc70 ithread_loop(c8e41060,f414fce8,c15cd71c,406,0,...) at ithread_loop+0x1f0/frame 0xf414fcac fork_exit(c0c17ff0,c8e41060,f414fce8) at fork_exit+0x7e/frame 0xf414fcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf414fcd4 --- trap 0, eip = 0, esp = 0xf414fd20, ebp = 0 --- Tracing command intr pid 12 tid 100069 td 0xc91126a0 sched_switch(c91126a0,0,109,190,c187ff48,...) at sched_switch+0x4e1/frame 0xf415fc38 mi_switch(109,0,c15cd9a1,555,c15cd71c,...) at mi_switch+0x18c/frame 0xf415fc70 ithread_loop(c8e41050,f415fce8,c15cd71c,406,0,...) at ithread_loop+0x1f0/frame 0xf415fcac fork_exit(c0c17ff0,c8e41050,f415fce8) at fork_exit+0x7e/frame 0xf415fcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf415fcd4 --- trap 0, eip = 0, esp = 0xf415fd20, ebp = 0 --- Tracing command intr pid 12 tid 100070 td 0xc9112350 fork_trampoline() at fork_trampoline Tracing command idle pid 11 tid 100003 td 0xc8e4e6a0 (CPU 0) cpustop_handler(0,13,eaf28b78,c11ee337,c8e210c0,...) at cpustop_handler+0x25/frame 0xeaf28a28 ipi_nmi_handler(c8e210c0,20ebebd7,eaf28a70,c0c178ae,eaf28a60,...) at ipi_nmi_handler+0x36/frame 0xeaf28a38 trap(eaf28b84) at trap+0x37/frame 0xeaf28b78 calltrap() at calltrap+0x6/frame 0xeaf28b78 --- trap 0x13, eip = 0xc1208045, esp = 0xeaf28bc4, ebp = 0xeaf28bc4 --- acpi_cpu_c1(eaf28bd8,246,241f1f06,eaf28c1c,c11f7f78,...) at acpi_cpu_c1+0x5/frame 0xeaf28bc4 acpi_cpu_idle(659c318,0,659c318,eaf28c38,c1208423,...) at acpi_cpu_idle+0x32a/frame 0xeaf28c08 cpu_idle_acpi(659c318,0,c1aace88,2,c8e4e6a0,...) at cpu_idle_acpi+0x3f/frame 0xeaf28c1c cpu_idle(0,eaf28c80,c15d85e3,a57,c1aae300,...) at cpu_idle+0xa3/frame 0xeaf28c38 sched_idletd(0,eaf28ce8,c15cd71c,406,0,...) at sched_idletd+0x403/frame 0xeaf28cac fork_exit(c0c82e10,0,eaf28ce8) at fork_exit+0x7e/frame 0xeaf28cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeaf28cd4 --- trap 0, eip = 0, esp = 0xeaf28d20, ebp = 0 --- Tracing command idle pid 11 tid 100004 td 0xc8e4e350 (CPU 1) cpustop_handler(1,13,eaf2bb78,c11ee337,229,...) at cpustop_handler+0x25/frame 0xeaf2ba28 ipi_nmi_handler(229,4b7620e7,a0cd2ae1,1,20099e,...) at ipi_nmi_handler+0x36/frame 0xeaf2ba38 trap(eaf2bb84) at trap+0x37/frame 0xeaf2bb78 calltrap() at calltrap+0x6/frame 0xeaf2bb78 --- trap 0x13, eip = 0xc1208045, esp = 0xeaf2bbc4, ebp = 0xeaf2bbc4 --- acpi_cpu_c1(eaf2bbd8,246,40068b71,eaf2bc1c,c11f7f78,...) at acpi_cpu_c1+0x5/frame 0xeaf2bbc4 acpi_cpu_idle(36d6515b,0,36d6515b,eaf2bc38,c1208423,...) at acpi_cpu_idle+0x32a/frame 0xeaf2bc08 cpu_idle_acpi(36d6515b,0,c1aad588,4,c8e4e350,...) at cpu_idle_acpi+0x3f/frame 0xeaf2bc1c cpu_idle(0,eaf2bc80,c15d85e3,407,c1aace00,...) at cpu_idle+0xa3/frame 0xeaf2bc38 sched_idletd(0,eaf2bce8,c15cd71c,406,0,...) at sched_idletd+0x403/frame 0xeaf2bcac fork_exit(c0c82e10,0,eaf2bce8) at fork_exit+0x7e/frame 0xeaf2bcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeaf2bcd4 --- trap 0, eip = 0, esp = 0xeaf2bd20, ebp = 0 --- Tracing command idle pid 11 tid 100005 td 0xc8e4e000 (CPU 2) kdb_alt_break_internal(0,eaf2eac8,c0a5f7e7,2,c9053284,...) at kdb_alt_break_internal+0x162/frame 0xeaf2ea9c kdb_alt_break(2,c9053284,c159a25b,0,40000,...) at kdb_alt_break+0x16/frame 0xeaf2eaa8 uart_intr_rxready(c9053200,8,c15d85e3,88d,0,...) at uart_intr_rxready+0x87/frame 0xeaf2eac8 uart_intr(c9053200,2fc,0,c11f75bd,0,...) at uart_intr+0x110/frame 0xeaf2eb08 intr_event_handle(c8e0ab00,eaf2eb84,1,2,0,...) at intr_event_handle+0x7a/frame 0xeaf2eb34 intr_execute_handlers(c8ea4108,eaf2eb84,2) at intr_execute_handlers+0x42/frame 0xeaf2eb54 lapic_handle_intr(30,eaf2eb84) at lapic_handle_intr+0x5d/frame 0xeaf2eb74 Xapic_isr1() at Xapic_isr1+0x55/frame 0xeaf2eb74 --- interrupt, eip = 0xc1208045, esp = 0xeaf2ebc4, ebp = 0xeaf2ebc4 --- acpi_cpu_c1(eaf2ebd8,246,28aa6296,eaf2ec1c,c11f7f78,...) at acpi_cpu_c1+0x5/frame 0xeaf2ebc4 acpi_cpu_idle(1f79c164,0,1f79c164,eaf2ec38,c1208423,...) at acpi_cpu_idle+0x32a/frame 0xeaf2ec08 cpu_idle_acpi(1f79c164,0,c1aadc88,3,c8e4e000,...) at cpu_idle_acpi+0x3f/frame 0xeaf2ec1c cpu_idle(0,eaf2ec80,c15d85e3,a57,c1aace00,...) at cpu_idle+0xa3/frame 0xeaf2ec38 sched_idletd(0,eaf2ece8,c15cd71c,406,0,...) at sched_idletd+0x403/frame 0xeaf2ecac fork_exit(c0c82e10,0,eaf2ece8) at fork_exit+0x7e/frame 0xeaf2ecd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeaf2ecd4 --- trap 0, eip = 0, esp = 0xeaf2ed20, ebp = 0 --- Tracing command idle pid 11 tid 100006 td 0xc8e4d9f0 (CPU 3) cpustop_handler(3,13,eaf31b78,c11ee337,229,...) at cpustop_handler+0x25/frame 0xeaf31a28 ipi_nmi_handler(229,39401cc3,dfb9ac7,3,3869af,...) at ipi_nmi_handler+0x36/frame 0xeaf31a38 trap(eaf31b84) at trap+0x37/frame 0xeaf31b78 calltrap() at calltrap+0x6/frame 0xeaf31b78 --- trap 0x13, eip = 0xc1208045, esp = 0xeaf31bc4, ebp = 0xeaf31bc4 --- acpi_cpu_c1(eaf31bd8,246,34a2318d,eaf31c1c,c11f7f78,...) at acpi_cpu_c1+0x5/frame 0xeaf31bc4 acpi_cpu_idle(16dba9aa,0,16dba9aa,eaf31c38,c1208423,...) at acpi_cpu_idle+0x32a/frame 0xeaf31c08 cpu_idle_acpi(16dba9aa,0,c1aae388,3,c8e4d9f0,...) at cpu_idle_acpi+0x3f/frame 0xeaf31c1c cpu_idle(0,eaf31c80,c15d85e3,a57,c1aace00,...) at cpu_idle+0xa3/frame 0xeaf31c38 sched_idletd(0,eaf31ce8,c15cd71c,406,0,...) at sched_idletd+0x403/frame 0xeaf31cac fork_exit(c0c82e10,0,eaf31ce8) at fork_exit+0x7e/frame 0xeaf31cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeaf31cd4 --- trap 0, eip = 0, esp = 0xeaf31d20, ebp = 0 --- Tracing command init pid 1 tid 100002 td 0xc8e4e9f0 sched_switch(c8e4e9f0,0,104,190,eaf2493c,...) at sched_switch+0x4e1/frame 0xeaf24904 mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xeaf2493c sleepq_switch(c8e4e9f0,0,c15dc5e5,1c2,c15dc5e5,...) at sleepq_switch+0xf0/frame 0xeaf24970 sleepq_catch_signals(c15dc5e5,170,c8e4aa90,153,c8e4e9f0,...) at sleepq_catch_signals+0x443/frame 0xeaf249b4 sleepq_wait_sig(c8e4a9d8,6c,c15e0759,100,0,...) at sleepq_wait_sig+0x14/frame 0xeaf249d8 _sleep(c8e4a9d8,c8e4aa80,16c,c15e0759,0,...) at _sleep+0x32f/frame 0xeaf24a34 kern_wait6(c8e4e9f0,7,0,0,eaf24acc,...) at kern_wait6+0x4b3/frame 0xeaf24aa0 sys_wait4(c8e4e9f0,eaf24c50,c15cd1c7,4,e,...) at sys_wait4+0x8b/frame 0xeaf24bc0 syscall(eaf24ce8) at syscall+0x3d3/frame 0xeaf24cdc Xint0x80_syscall() at Xint0x80_syscall+0x2e/frame 0xeaf24cdc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x8065597, esp = 0xbfbfe88c, ebp = 0xbfbfe8a0 --- Tracing command audit pid 10 tid 100001 td 0xc8e4f000 sched_switch(c8e4f000,0,104,190,eaf21bbc,...) at sched_switch+0x4e1/frame 0xeaf21b84 mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xeaf21bbc sleepq_switch(c8e4f000,0,c15dc5e5,28b,c1b7c26c,...) at sleepq_switch+0xf0/frame 0xeaf21bf0 sleepq_wait(c1b7c26c,0,eaf21c44,1,0,...) at sleepq_wait+0x48/frame 0xeaf21c14 _cv_wait(c1b7c26c,c1b7c1f8,c1615a86,1ab,c1aace00,...) at _cv_wait+0x20b/frame 0xeaf21c58 audit_worker(0,eaf21ce8,c15cd71c,406,0,...) at audit_worker+0x94/frame 0xeaf21cac fork_exit(c0ef3c70,0,eaf21ce8) at fork_exit+0x7e/frame 0xeaf21cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeaf21cd4 --- trap 0, eip = 0, esp = 0xeaf21d20, ebp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xc1b747d0 sched_switch(c1b747d0,0,104,190,c2022c3c,...) at sched_switch+0x4e1/frame 0xc2022c04 mi_switch(104,0,c15dc5e5,215,0,...) at mi_switch+0x18c/frame 0xc2022c3c sleepq_switch(c1b747d0,0,c15dc5e5,2ae,c1b747d0,...) at sleepq_switch+0xf0/frame 0xc2022c70 sleepq_timedwait(c1b74484,54,9,0,0,...) at sleepq_timedwait+0x48/frame 0xc2022c94 _sleep(c1b74484,0,54,c1620d2f,fffff470,...) at _sleep+0x316/frame 0xc2022cf0 swapper() at swapper+0x2b7/frame 0xc2022d38 begin() at begin+0x22 Tracing command kernel pid 0 tid 100013 td 0xc8e4c000 sched_switch(c8e4c000,0,104,190,eaf47bc0,...) at sched_switch+0x4e1/frame 0xeaf47b88 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xeaf47bc0 sleepq_switch(c8e4c000,0,c15dc5e5,28b,c8e4c000,...) at sleepq_switch+0xf0/frame 0xeaf47bf4 sleepq_wait(c8f1c000,0,c15bee13,0,0,...) at sleepq_wait+0x48/frame 0xeaf47c18 _sleep(c8f1c000,c8f1c01c,0,c15bee13,0,...) at _sleep+0x342/frame 0xeaf47c74 taskqueue_thread_loop(c1b74d58,eaf47ce8,c15cd71c,406,0,...) at taskqueue_thread_loop+0x144/frame 0xeaf47cac fork_exit(c0caa9e0,c1b74d58,eaf47ce8) at fork_exit+0x7e/frame 0xeaf47cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeaf47cd4 --- trap 0, eip = 0, esp = 0xeaf47d20, ebp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xc8f239f0 sched_switch(c8f239f0,0,104,190,eaf50bc0,...) at sched_switch+0x4e1/frame 0xeaf50b88 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xeaf50bc0 sleepq_switch(c8f239f0,0,c15dc5e5,28b,c8f239f0,...) at sleepq_switch+0xf0/frame 0xeaf50bf4 sleepq_wait(c8f1be00,0,c15bee13,0,0,...) at sleepq_wait+0x48/frame 0xeaf50c18 _sleep(c8f1be00,c8f1be1c,0,c15bee13,0,...) at _sleep+0x342/frame 0xeaf50c74 taskqueue_thread_loop(c1b78b9c,eaf50ce8,c15cd71c,406,0,...) at taskqueue_thread_loop+0x144/frame 0xeaf50cac fork_exit(c0caa9e0,c1b78b9c,eaf50ce8) at fork_exit+0x7e/frame 0xeaf50cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeaf50cd4 --- trap 0, eip = 0, esp = 0xeaf50d20, ebp = 0 --- Tracing command kernel pid 0 tid 100017 td 0xc8f236a0 sched_switch(c8f236a0,0,104,190,eaf53bc0,...) at sched_switch+0x4e1/frame 0xeaf53b88 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xeaf53bc0 sleepq_switch(c8f236a0,0,c15dc5e5,28b,c8f236a0,...) at sleepq_switch+0xf0/frame 0xeaf53bf4 sleepq_wait(c8f1bd80,0,c15bee13,0,0,...) at sleepq_wait+0x48/frame 0xeaf53c18 _sleep(c8f1bd80,c8f1bd9c,0,c15bee13,0,...) at _sleep+0x342/frame 0xeaf53c74 taskqueue_thread_loop(c1b78c18,eaf53ce8,c15cd71c,406,0,...) at taskqueue_thread_loop+0x144/frame 0xeaf53cac fork_exit(c0caa9e0,c1b78c18,eaf53ce8) at fork_exit+0x7e/frame 0xeaf53cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeaf53cd4 --- trap 0, eip = 0, esp = 0xeaf53d20, ebp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xc8f23000 sched_switch(c8f23000,0,104,190,eaf59bc0,...) at sched_switch+0x4e1/frame 0xeaf59b88 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xeaf59bc0 sleepq_switch(c8f23000,0,c15dc5e5,28b,c8f23000,...) at sleepq_switch+0xf0/frame 0xeaf59bf4 sleepq_wait(c8f1bc80,0,c15bee13,0,0,...) at sleepq_wait+0x48/frame 0xeaf59c18 _sleep(c8f1bc80,c8f1bc9c,0,c15bee13,0,...) at _sleep+0x342/frame 0xeaf59c74 taskqueue_thread_loop(c1abaf20,eaf59ce8,c15cd71c,406,0,...) at taskqueue_thread_loop+0x144/frame 0xeaf59cac fork_exit(c0caa9e0,c1abaf20,eaf59ce8) at fork_exit+0x7e/frame 0xeaf59cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeaf59cd4 --- trap 0, eip = 0, esp = 0xeaf59d20, ebp = 0 --- Tracing command kernel pid 0 tid 100026 td 0xc8e4f350 sched_switch(c8e4f350,0,104,190,eaf6ebd8,...) at sched_switch+0x4e1/frame 0xeaf6eba0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xeaf6ebd8 sleepq_switch(c8e4f350,0,c15dc5e5,28b,0,...) at sleepq_switch+0xf0/frame 0xeaf6ec0c sleepq_wait(c8f1b880,0,c15d658a,c15bee13,0,...) at sleepq_wait+0x48/frame 0xeaf6ec30 msleep_spin_sbt(c8f1b880,c8f1b89c,c15bee13,0,0,...) at msleep_spin_sbt+0x228/frame 0xeaf6ec74 taskqueue_thread_loop(c1b6ad04,eaf6ece8,c15cd71c,406,0,...) at taskqueue_thread_loop+0xf3/frame 0xeaf6ecac fork_exit(c0caa9e0,c1b6ad04,eaf6ece8) at fork_exit+0x7e/frame 0xeaf6ecd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeaf6ecd4 --- trap 0, eip = 0, esp = 0xeaf6ed20, ebp = 0 --- Tracing command kernel pid 0 tid 100027 td 0xc902d6a0 sched_switch(c902d6a0,0,104,190,eaf71bd8,...) at sched_switch+0x4e1/frame 0xeaf71ba0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xeaf71bd8 sleepq_switch(c902d6a0,0,c15dc5e5,28b,0,...) at sleepq_switch+0xf0/frame 0xeaf71c0c sleepq_wait(c8f1b880,0,c15d658a,c15bee13,0,...) at sleepq_wait+0x48/frame 0xeaf71c30 msleep_spin_sbt(c8f1b880,c8f1b89c,c15bee13,0,0,...) at msleep_spin_sbt+0x228/frame 0xeaf71c74 taskqueue_thread_loop(c1b6ad04,eaf71ce8,c15cd71c,406,0,...) at taskqueue_thread_loop+0xf3/frame 0xeaf71cac fork_exit(c0caa9e0,c1b6ad04,eaf71ce8) at fork_exit+0x7e/frame 0xeaf71cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeaf71cd4 --- trap 0, eip = 0, esp = 0xeaf71d20, ebp = 0 --- Tracing command kernel pid 0 tid 100028 td 0xc902d350 sched_switch(c902d350,0,104,190,eaf74bd8,...) at sched_switch+0x4e1/frame 0xeaf74ba0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xeaf74bd8 sleepq_switch(c902d350,0,c15dc5e5,28b,0,...) at sleepq_switch+0xf0/frame 0xeaf74c0c sleepq_wait(c8f1b880,0,c15d658a,c15bee13,0,...) at sleepq_wait+0x48/frame 0xeaf74c30 msleep_spin_sbt(c8f1b880,c8f1b89c,c15bee13,0,0,...) at msleep_spin_sbt+0x228/frame 0xeaf74c74 taskqueue_thread_loop(c1b6ad04,eaf74ce8,c15cd71c,406,0,...) at taskqueue_thread_loop+0xf3/frame 0xeaf74cac fork_exit(c0caa9e0,c1b6ad04,eaf74ce8) at fork_exit+0x7e/frame 0xeaf74cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xeaf74cd4 --- trap 0, eip = 0, esp = 0xeaf74d20, ebp = 0 --- Tracing command kernel pid 0 tid 100071 td 0xc9112000 sched_switch(c9112000,0,104,190,f4165bd8,...) at sched_switch+0x4e1/frame 0xf4165ba0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf4165bd8 sleepq_switch(c9112000,0,c15dc5e5,28b,0,...) at sleepq_switch+0xf0/frame 0xf4165c0c sleepq_wait(c913e980,0,c15d658a,c15bee13,0,...) at sleepq_wait+0x48/frame 0xf4165c30 msleep_spin_sbt(c913e980,c913e99c,c15bee13,0,0,...) at msleep_spin_sbt+0x228/frame 0xf4165c74 taskqueue_thread_loop(c1b69138,f4165ce8,c15cd71c,406,0,...) at taskqueue_thread_loop+0xf3/frame 0xf4165cac fork_exit(c0caa9e0,c1b69138,f4165ce8) at fork_exit+0x7e/frame 0xf4165cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4165cd4 --- trap 0, eip = 0, esp = 0xf4165d20, ebp = 0 --- Tracing command kernel pid 0 tid 100074 td 0xc9111350 sched_switch(c9111350,0,104,190,f416fbc0,...) at sched_switch+0x4e1/frame 0xf416fb88 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf416fbc0 sleepq_switch(c9111350,0,c15dc5e5,28b,c9111350,...) at sleepq_switch+0xf0/frame 0xf416fbf4 sleepq_wait(c8f1ba80,0,c15bee13,0,0,...) at sleepq_wait+0x48/frame 0xf416fc18 _sleep(c8f1ba80,c8f1ba9c,0,c15bee13,0,...) at _sleep+0x342/frame 0xf416fc74 taskqueue_thread_loop(c18c40a4,f416fce8,c15cd71c,406,0,...) at taskqueue_thread_loop+0x144/frame 0xf416fcac fork_exit(c0caa9e0,c18c40a4,f416fce8) at fork_exit+0x7e/frame 0xf416fcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf416fcd4 --- trap 0, eip = 0, esp = 0xf416fd20, ebp = 0 --- Tracing command kernel pid 0 tid 100086 td 0xc93c39f0 sched_switch(c93c39f0,0,104,190,f4313bd8,...) at sched_switch+0x4e1/frame 0xf4313ba0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf4313bd8 sleepq_switch(c93c39f0,0,c15dc5e5,28b,0,...) at sleepq_switch+0xf0/frame 0xf4313c0c sleepq_wait(c913d380,0,c15d658a,c15bee13,0,...) at sleepq_wait+0x48/frame 0xf4313c30 msleep_spin_sbt(c913d380,c913d39c,c15bee13,0,0,...) at msleep_spin_sbt+0x228/frame 0xf4313c74 gtaskqueue_thread_loop(c8f09004,f4313ce8,c15cd71c,406,0,...) at gtaskqueue_thread_loop+0xf3/frame 0xf4313cac fork_exit(c0c966c0,c8f09004,f4313ce8) at fork_exit+0x7e/frame 0xf4313cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4313cd4 --- trap 0, eip = 0, esp = 0xf4313d20, ebp = 0 --- Tracing command kernel pid 0 tid 100089 td 0xc93c36a0 sched_switch(c93c36a0,0,104,190,f431cbd8,...) at sched_switch+0x4e1/frame 0xf431cba0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf431cbd8 sleepq_switch(c93c36a0,0,c15dc5e5,28b,0,...) at sleepq_switch+0xf0/frame 0xf431cc0c sleepq_wait(c913d300,0,c15d658a,c15bee13,0,...) at sleepq_wait+0x48/frame 0xf431cc30 msleep_spin_sbt(c913d300,c913d31c,c15bee13,0,0,...) at msleep_spin_sbt+0x228/frame 0xf431cc74 gtaskqueue_thread_loop(c8f08c04,f431cce8,c15cd71c,406,0,...) at gtaskqueue_thread_loop+0xf3/frame 0xf431ccac fork_exit(c0c966c0,c8f08c04,f431cce8) at fork_exit+0x7e/frame 0xf431ccd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf431ccd4 --- trap 0, eip = 0, esp = 0xf431cd20, ebp = 0 --- Tracing command kernel pid 0 tid 100090 td 0xc93c3350 sched_switch(c93c3350,0,104,190,f431fbd8,...) at sched_switch+0x4e1/frame 0xf431fba0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf431fbd8 sleepq_switch(c93c3350,0,c15dc5e5,28b,0,...) at sleepq_switch+0xf0/frame 0xf431fc0c sleepq_wait(c913d280,0,c15d658a,c15bee13,0,...) at sleepq_wait+0x48/frame 0xf431fc30 msleep_spin_sbt(c913d280,c913d29c,c15bee13,0,0,...) at msleep_spin_sbt+0x228/frame 0xf431fc74 gtaskqueue_thread_loop(c8f08c14,f431fce8,c15cd71c,406,0,...) at gtaskqueue_thread_loop+0xf3/frame 0xf431fcac fork_exit(c0c966c0,c8f08c14,f431fce8) at fork_exit+0x7e/frame 0xf431fcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf431fcd4 --- trap 0, eip = 0, esp = 0xf431fd20, ebp = 0 --- Tracing command kernel pid 0 tid 100091 td 0xc93c3000 sched_switch(c93c3000,0,104,190,f4322bd8,...) at sched_switch+0x4e1/frame 0xf4322ba0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf4322bd8 sleepq_switch(c93c3000,0,c15dc5e5,28b,0,...) at sleepq_switch+0xf0/frame 0xf4322c0c sleepq_wait(c913d200,0,c15d658a,c15bee13,0,...) at sleepq_wait+0x48/frame 0xf4322c30 msleep_spin_sbt(c913d200,c913d21c,c15bee13,0,0,...) at msleep_spin_sbt+0x228/frame 0xf4322c74 gtaskqueue_thread_loop(c8f08c24,f4322ce8,c15cd71c,406,0,...) at gtaskqueue_thread_loop+0xf3/frame 0xf4322cac fork_exit(c0c966c0,c8f08c24,f4322ce8) at fork_exit+0x7e/frame 0xf4322cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4322cd4 --- trap 0, eip = 0, esp = 0xf4322d20, ebp = 0 --- Tracing command kernel pid 0 tid 100092 td 0xc91139f0 sched_switch(c91139f0,0,104,190,f4325bd8,...) at sched_switch+0x4e1/frame 0xf4325ba0 mi_switch(104,0,c15dc5e5,215,c187ff48,...) at mi_switch+0x18c/frame 0xf4325bd8 sleepq_switch(c91139f0,0,c15dc5e5,28b,0,...) at sleepq_switch+0xf0/frame 0xf4325c0c sleepq_wait(c913d180,0,c15d658a,c15bee13,0,...) at sleepq_wait+0x48/frame 0xf4325c30 msleep_spin_sbt(c913d180,c913d19c,c15bee13,0,0,...) at msleep_spin_sbt+0x228/frame 0xf4325c74 gtaskqueue_thread_loop(c8f08c34,f4325ce8,c15cd71c,406,0,...) at gtaskqueue_thread_loop+0xf3/frame 0xf4325cac fork_exit(c0c966c0,c8f08c34,f4325ce8) at fork_exit+0x7e/frame 0xf4325cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4325cd4 --- trap 0, eip = 0, esp = 0xf4325d20, ebp = 0 --- db:0:allt> dump Physical memory: 3427 MB Dumping 412 MB: 397 381 365 349 333 317 301 285 269 253 237 221 205 189 173 157 141 125 109 93 77 61 45 29 13 Dump complete db:0:dump> reset cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 2 (kgdb) p/d nprocs $1 = 25 (kgdb) $ svn diff -x -p /usr/src/sys Index: /usr/src/sys/vm/_vm_radix.h =================================================================== --- /usr/src/sys/vm/_vm_radix.h (revision 309257) +++ /usr/src/sys/vm/_vm_radix.h (working copy) @@ -36,12 +36,8 @@ */ struct vm_radix { uintptr_t rt_root; - uint8_t rt_flags; }; -#define RT_INSERT_INPROG 0x01 -#define RT_TRIE_MODIFIED 0x02 - #ifdef _KERNEL static __inline boolean_t Index: /usr/src/sys/vm/vm_object.c =================================================================== --- /usr/src/sys/vm/vm_object.c (revision 309257) +++ /usr/src/sys/vm/vm_object.c (working copy) @@ -205,7 +205,6 @@ vm_object_zinit(void *mem, int size, int flags) object->type = OBJT_DEAD; object->ref_count = 0; object->rtree.rt_root = 0; - object->rtree.rt_flags = 0; object->paging_in_progress = 0; object->resident_page_count = 0; object->shadow_count = 0; Index: /usr/src/sys/vm/vm_radix.c =================================================================== --- /usr/src/sys/vm/vm_radix.c (revision 309257) +++ /usr/src/sys/vm/vm_radix.c (working copy) @@ -341,8 +341,6 @@ vm_radix_insert(struct vm_radix *rtree, vm_page_t index = page->pindex; -restart: - /* * The owner of record for root is not really important because it * will never be used. @@ -360,32 +358,10 @@ vm_radix_insert(struct vm_radix *rtree, vm_page_t panic("%s: key %jx is already present", __func__, (uintmax_t)index); clev = vm_radix_keydiff(m->pindex, index); - - /* - * During node allocation the trie that is being - * walked can be modified because of recursing radix - * trie operations. - * If this is the case, the recursing functions signal - * such situation and the insert operation must - * start from scratch again. - * The freed radix node will then be in the UMA - * caches very likely to avoid the same situation - * to happen. - */ - rtree->rt_flags |= RT_INSERT_INPROG; tmp = vm_radix_node_get(vm_radix_trimkey(index, clev + 1), 2, clev); - rtree->rt_flags &= ~RT_INSERT_INPROG; - if (tmp == NULL) { - rtree->rt_flags &= ~RT_TRIE_MODIFIED; + if (tmp == NULL) return (ENOMEM); - } - if ((rtree->rt_flags & RT_TRIE_MODIFIED) != 0) { - rtree->rt_flags &= ~RT_TRIE_MODIFIED; - tmp->rn_count = 0; - vm_radix_node_put(tmp); - goto restart; - } *parentp = tmp; vm_radix_addpage(tmp, index, clev, page); vm_radix_addpage(tmp, m->pindex, clev, m); @@ -409,21 +385,9 @@ vm_radix_insert(struct vm_radix *rtree, vm_page_t */ newind = rnode->rn_owner; clev = vm_radix_keydiff(newind, index); - - /* See the comments above. */ - rtree->rt_flags |= RT_INSERT_INPROG; tmp = vm_radix_node_get(vm_radix_trimkey(index, clev + 1), 2, clev); - rtree->rt_flags &= ~RT_INSERT_INPROG; - if (tmp == NULL) { - rtree->rt_flags &= ~RT_TRIE_MODIFIED; + if (tmp == NULL) return (ENOMEM); - } - if ((rtree->rt_flags & RT_TRIE_MODIFIED) != 0) { - rtree->rt_flags &= ~RT_TRIE_MODIFIED; - tmp->rn_count = 0; - vm_radix_node_put(tmp); - goto restart; - } *parentp = tmp; vm_radix_addpage(tmp, index, clev, page); slot = vm_radix_slot(newind, clev); @@ -708,20 +672,6 @@ vm_radix_remove(struct vm_radix *rtree, vm_pindex_ vm_page_t m; int i, slot; - /* - * Detect if a page is going to be removed from a trie which is - * already undergoing another trie operation. - * Right now this is only possible for vm_radix_remove() recursing - * into vm_radix_insert(). - * If this is the case, the caller must be notified about this - * situation. It will also takecare to update the RT_TRIE_MODIFIED - * accordingly. - * The RT_TRIE_MODIFIED bit is set here because the remove operation - * will always succeed. - */ - if ((rtree->rt_flags & RT_INSERT_INPROG) != 0) - rtree->rt_flags |= RT_TRIE_MODIFIED; - rnode = vm_radix_getroot(rtree); if (vm_radix_isleaf(rnode)) { m = vm_radix_topage(rnode); @@ -776,9 +726,6 @@ vm_radix_reclaim_allnodes(struct vm_radix *rtree) { struct vm_radix_node *root; - KASSERT((rtree->rt_flags & RT_INSERT_INPROG) == 0, - ("vm_radix_reclaim_allnodes: unexpected trie recursion")); - root = vm_radix_getroot(rtree); if (root == NULL) return; -- what /usr/src/sys/i386/compile/PHO/kernel.debug /usr/src/sys/i386/compile/PHO/kernel.debug: ls -l /usr/src/sys/i386/compile/PHO/kernel.debug /var/crash/vmcore.982 -rwxr-xr-x 1 root wheel 88349068 29 nov. 17:25 /usr/src/sys/i386/compile/PHO/kernel.debug -rw------- 1 root wheel 432504832 30 nov. 12:30 /var/crash/vmcore.982 vmstat -N /usr/src/sys/i386/compile/PHO/kernel.debug -M /var/crash/vmcore.982 -m Type InUse MemUse HighUse Requests Size(s) CAM queue 14 3K - 73 16,256 entropy 0 0K - 2189397 4096 scsi_cd 0 0K - 13811 16 CAM dev queue 6 1K - 6 64 hdaa 9 56K - 9 256,512,1024,2048,16384,32768 hdac 2 1K - 2 64,512 hdacc 2 1K - 2 16 feeder 21 1K - 27 16,64 mixer 4 16K - 4 4096 UART 3 2K - 3 16,512,1024 USB 36 15K - 36 16,64,256,2048 USBdev 60 10K - 297348 32,128,256 ddb_capture 1 64K - 1 65536 vtbuf 24 1944K - 46 2048 vt 11 3K - 11 256 sysmouse 0 0K - 16516 2048 acpiintr 1 1K - 1 32 acpica 2578 135K - 154282 16,32,64,128,256,512,1024 acpitask 1 4K - 1 4096 DEVFS2 793 100K - 16255 16,32,64,128 DEVFS3 821 103K - 13417571 128,256 DEVFS1 795 199K - 971530 256 DEVFS_RULE 57 14K - 57 32,256 DEVFS 16 1K - 213304 16,64 DEVFSP 0 0K - 492915 32 NFSD srvcache 27 4K - 2178661 128 NFSD V4client 1 1K - 1 128 NFSD lckfile 1 1K - 1 128 NFSD string 1 1K - 1 16 NFSD usrgroup 101 23K - 102 64,4096 NFS fh 0 0K - 1812180 32 NFSCL diroffdiroff 0 0K - 1599 512 NFSD session 1 1K - 1 512 newnfsclient_req 0 0K - 526 64 newnfsmnt 0 0K - 2639 512 pfs_nodes 21 3K - 21 128 pfs_vncache 0 0K - 5674922 32 GEOM 938 75K - 3630240 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768 raid_data 0 0K - 774168 16,128 isadev 7 1K - 7 64 isofs_mount 0 0K - 3 128 isofs_node 0 0K - 5242 128 acpisem 30 4K - 30 64,128 acpidev 31 1K - 31 32 CAM SIM 6 1K - 6 128 CAM XPT 29 3K - 1315 16,32,128,256,512,1024,2048,65536 CAM DEV 8 16K - 27 2048 cdev 4 1K - 4 128 filedesc 0 0K - 23762 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536 sigio 0 0K - 16577 32 filecaps 0 0K - 105486 16,64 kdtrace 6845 1707K - 14846851 64,256 kenv 130 9K - 134 16,32,64,128,4096 kqueue 420 14K - 20390601 32,256,1024,32768 proc-args 2 1K - 3296420 16,32,64,128,256 Fail Points 0 0K - 920 1024 hhook 13 2K - 13 128 ithread 83 7K - 83 16,64,128 prison 0 0K - 1 16 KTRACE 100 13K - 100 128 CAM CCB 0 0K - 10782771 2048 linker 190 549K - 944 16,32,64,256,512,1024,2048,4096,8192,16384,32768,65536 CAM path 10 1K - 86 16 lockf 2 2K - 817016971 32,64,128,256,512,1024 loginclass 3 1K - 3 64 cache 1 1K - 1 16 devbuf 2522 2881K - 316089 16,32,64,128,256,512,1024,2048,4096,8192,32768,65536 temp 24 234K - 1110283083 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536 CAM periph 6 1K - 32990 16,32,64,128,256 module 531 34K - 559 64,128 mtx_pool 2 8K - 2 4096 osd 3 1K - 10 16,32,64,128 pmchooks 1 1K - 1 64 pgrp 2 1K - 36376 64 session 2 1K - 956 128 proc 2 8K - 2 4096 subproc 18978 4839K - 8496530 256,4096 cred 13707 3427K - 2897078 256 kbdmux 6 22K - 6 16,256,1024,2048,16384 plimit 1 1K - 5819 256 uidinfo 5 2K - 333 128,1024 dumper 1 1K - 1 512 md_disk 1 2K - 233 16,2048 sysctl 0 0K - 5552488 16,32,64 sysctloid 3845 122K - 4028 16,32,64,128 sysctltmp 0 0K - 65442 16,32,64,128,256,1024 md_sectors 0 0K - 121 4096 tidhash 1 8K - 1 8192 callout 5 1308K - 5 umtx 32664 3063K - 32664 64,128 p1003.1b 1 1K - 1 16 SWAP 2 2189K - 2 64 bus 1356 67K - 4491278 16,32,64,128,256,1024 bus-sc 79 130K - 3907 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536 devstat 28 57K - 28 16,4096 eventhandler 100 5K - 107 32,64,128 taskqueue 17 3K - 17 16,32,128,1024 kobj 376 752K - 775465 2048 Per-cpu 1 1K - 1 16 rman 239 28K - 587 16,32,128 sbuf 1 4K - 118948321 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536 toponodes 18 2K - 18 64 stack 0 0K - 9202 128 taskqueue 27 2K - 33 16,32,128 terminal 11 3K - 11 256 Unitno 52 2K - 21576869 16,64 vmem 4 337K - 618 64,128,256,512,1024,2048,4096,8192,16384,32768,65536 Witness 1539 3238K - 1539 2048,8192 ioctlops 0 0K - 1888 256,512,1024,2048 select 826 52K - 826 64 iov 0 0K - 767002260 16,64,128,256,512 msg 4 25K - 4 1024,4096,16384 sem 4 101K - 4 1024,4096 shm 1 16K - 11 1024,16384 tty 690 345K - 928265 512,65536 pts 677 85K - 928158 128 mbuf_tag 0 0K - 7 32 shmfd 1 4K - 1 4096 soname 1 1K - 80609050 16,32,128 pcb 17 332K - 165048 16,32,64,512,1024,4096,65536 acl 0 0K - 83732551 512,4096 vfscache 4 593K - 8 512,16384,32768,65536 cl_savebuf 0 0K - 4303762 32,64 export_host 14 2K - 473008 128 vfs_hash 1 128K - 5 4096,8192 vnodes 1 1K - 1373 32,128 mount 13118 1631K - 24417779 16,32,64,128,256 vnodemarker 0 0K - 14148429 512 fadvise 0 0K - 1 32 arc4random 1 2K - 1 2048 BPF 2 1K - 33034 64,512 ifnet 3 3K - 3 64,1024 ifaddr 43 8K - 43 16,32,64,128,256,1024,2048 ether_multi 40 2K - 47 16,32,64 clone 6 1K - 6 128 lltable 14 3K - 32 128,256 routetbl 33 4K - 236533 16,32,64,128,256 igmp 2 1K - 2 128 in_mfilter 0 0K - 1 512 in_multi 2 1K - 2 128 ip_moptions 0 0K - 2 32,128 encap_export_host 2 2K - 2 1024 sctp_a_it 0 0K - 3 16 sctp_vrf 1 1K - 1 64 sctp_ifa 5 1K - 5 128 sctp_ifn 2 1K - 2 128 sctp_iter 0 0K - 3 256 hostcache 1 16K - 1 16384 tcpfunc 1 1K - 1 16 syncache 1 64K - 1 65536 in6_mfilter 0 0K - 3 512 in6_multi 25 2K - 27 16,128 ip6_moptions 0 0K - 6 32,128 ip6opt 0 0K - 7 32,256 mld 2 1K - 2 64 ip6ndp 4 1K - 5 64,128 inpcbpolicy 3 1K - 187285 16 ipsecpolicy 6 1K - 406636 128 NFS FHA 0 0K - 2181224 64 NLM 651 21K - 1227 16,32,2048 crypto 1 1K - 1 512 rpc 10 19K - 4138001 16,32,64,256,512,1024,2048,8192 audit_evclass 189 3K - 233 16 audit_pipe 0 0K - 16516 256 pagedep 6 161K - 36861132 128,32768 inodedep 7 641K - 92015018 256 bmsafemap 5 20K - 66349497 128,4096 newblk 5 640K - 54140369 128 indirdep 0 0K - 763393 64,16384,32768 freefrag 0 0K - 11616732 64 freeblks 0 0K - 7197344 128 freefile 0 0K - 64477931 32 diradd 1 1K - 75302321 64 mkdir 0 0K - 18727664 64 dirrem 1 1K - 74731825 64 newdirblk 0 0K - 9581364 32 freework 5 1K - 8856753 64,128 freedep 0 0K - 9569 32 jaddref 0 0K - 7833419 64 jremref 0 0K - 7833408 64 jnewblk 0 0K - 9340839 64 jfreefrag 0 0K - 1859237 64 jseg 0 0K - 68011 128 jsegdep 0 0K - 26866903 32 sbdep 0 0K - 1055085 32 savedino 0 0K - 32138224 256 jblocks 2 5K - 11 128,256,512,1024,2048,4096 softdep 5 3K - 42616 512 ufs_dirhash 62 21K - 1289648 16,32,64,128,256,512,1024,2048,4096,8192 ufs_extattr 0 0K - 2 128 ufs_quota 1 128K - 1 ufs_mount 19 76K - 129067 256,2048,4096,8192,16384,32768,65536 vm_pgdata 2 65K - 2 64,65536 UMAHash 6 165K - 52 256,512,1024,2048,4096,8192,16384,32768,65536 atkbddev 2 1K - 2 32 pci_link 16 2K - 16 64,128 CAM I/O Scheduler 1 1K - 1 64 memdesc 1 4K - 1 4096 apmdev 1 1K - 33455 64 madt_table 0 0K - 1 2048 io_apic 1 2K - 1 2048 MCA 6 1K - 6 128 msi 2 1K - 2 64 nexusdev 5 1K - 5 16 linux 14 1K - 14 32 fdesc_mount 0 0K - 15181 16 tmpfs mount 0 0K - 101 128 tmpfs name 0 0K - 1052943 16 vmstat -N /usr/src/sys/i386/compile/PHO/kernel.debug -M /var/crash/vmcore.982 -z ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP UMA Kegs: 256, 0, 203, 7, 3808, 0, 0 UMA Zones: 480, 0, 204, 4, 3809, 0, 0 UMA Slabs: 88, 0, 4253, 112, 394535, 0, 0 UMA Hash: 128, 0, 4, 27, 15, 0, 0 4 Bucket: 16, 0, 1238, 762, 6542335, 0, 0 6 Bucket: 24, 0, 0, 334, 206238, 0, 0 8 Bucket: 32, 0, 20, 1105, 964030, 0, 0 12 Bucket: 48, 0, 0, 0, 53129, 0, 0 16 Bucket: 64, 0, 255, 799, 1101901, 21, 0 32 Bucket: 128, 0, 1923, 247, 1428703,92997, 0 64 Bucket: 256, 0, 2190, 255, 2300038,17208, 0 128 Bucket: 512, 0, 602, 102, 1677386,23499, 0 256 Bucket: 1024, 0, 1006, 562,20133173,8863158, 0 vmem btag: 28, 0, 61364, 161430, 2885052,4358, 0 VM OBJECT: 148, 0, 7279, 54470,157537483, 0, 0 RADIX NODE: 44, 853398, 23623, 125253,218236248, 0, 0 MAP: 140, 0, 3, 81, 3, 0, 0 KMAP ENTRY: 72, 0, 5, 105, 5, 0, 0 MAP ENTRY: 72, 0, 22, 17138,205158032, 0, 0 VMSPACE: 232, 0, 2, 38860, 8478010, 0, 0 fakepg: 68, 0, 0, 522, 1116, 0, 0 mt_zone: 2060, 0, 404, 0, 425, 0, 0 16: 16, 0, 5, 1245, 287308, 0, 0 16: 16, 0, 266, 1234, 2260008, 0, 0 16: 16, 0, 49, 1201, 725584, 0, 0 16: 16, 0, 2524, 1226, 2453770, 0, 0 16: 16, 0, 17, 1233, 5142660, 0, 0 16: 16, 0, 156, 1094,90057796, 0, 0 16: 16, 0, 101, 1149, 384485, 0, 0 16: 16, 0, 41, 1209,21615117, 0, 0 32: 32, 0, 71, 554, 2305174, 0, 0 32: 32, 0, 963, 662, 82729, 0, 0 32: 32, 0, 173, 64577,98300358, 0, 0 32: 32, 0, 1041, 1084,250338502, 0, 0 32: 32, 0, 2, 873, 1464983, 0, 0 32: 32, 0, 1918, 1832,42182734, 0, 0 32: 32, 0, 671, 204, 5990277, 0, 0 32: 32, 0, 29, 1596,11645921, 0, 0 64: 64, 0, 39, 147, 39, 0, 0 64: 64, 0, 1656, 762, 160376, 834, 0 64: 64, 0, 28, 2142,38635278, 0, 0 64: 64, 0, 1626, 45432,577256934, 0, 0 64: 64, 0, 830, 15166,94089669, 0, 0 64: 64, 0, 16982, 44212,78787993, 0, 0 64: 64, 0, 506, 858,756606537, 0, 0 64: 64, 0, 39, 1077, 516653, 0, 0 128: 128, 0, 19, 1376,37226967, 0, 0 128: 128, 0, 80, 509, 665859, 0, 0 128: 128, 0, 805, 528,17751515, 0, 0 128: 128, 0, 1693, 6212,135053312, 0, 0 128: 128, 0, 28, 158, 473022, 0, 0 128: 128, 0, 29763, 524,26065848, 0, 0 128: 128, 0, 573, 512, 4169087, 0, 0 128: 128, 0, 40, 394, 363118, 0, 0 256: 256, 0, 1, 14, 1, 0, 0 256: 256, 0, 802, 623,33166976, 1, 0 256: 256, 0, 13728, 267, 2902429, 0, 0 256: 256, 0, 25810, 1775, 6489234, 0, 0 256: 256, 0, 19, 161, 5862, 0, 0 256: 256, 0, 79, 251,10143079, 0, 0 256: 256, 0, 322, 57383,91975803, 0, 0 256: 256, 0, 3, 27, 623902, 0, 0 512: 512, 0, 0, 0, 0, 0, 0 512: 512, 0, 719, 249, 1773860, 858, 0 512: 512, 0, 5, 11, 684, 0, 0 512: 512, 0, 15, 41, 1956780, 0, 0 512: 512, 0, 3, 117,41673160, 0, 0 512: 512, 0, 0, 120,44270577, 0, 0 512: 512, 0, 36, 28,11296968, 0, 0 512: 512, 0, 1, 63, 4057, 0, 0 1024: 1024, 0, 3, 5, 923, 0, 0 1024: 1024, 0, 4, 24, 9031, 0, 0 1024: 1024, 0, 4, 8, 1509, 0, 0 1024: 1024, 0, 13, 27, 61809, 0, 0 1024: 1024, 0, 2, 6, 2, 0, 0 1024: 1024, 0, 7, 53,922132315, 0, 0 1024: 1024, 0, 16, 8, 18, 0, 0 1024: 1024, 0, 1, 3, 94, 0, 0 2048: 2048, 0, 2, 2, 2, 0, 0 2048: 2048, 0, 3, 1, 19892, 0, 0 2048: 2048, 0, 21, 3,54059031, 0, 0 2048: 2048, 0, 14, 56,12973402, 63, 0 2048: 2048, 0, 3, 1, 3, 0, 0 2048: 2048, 0, 1, 5,113824466, 0, 0 2048: 2048, 0, 7, 1, 7, 0, 0 2048: 2048, 0, 1913, 3, 777310, 0, 0 4096: 4096, 0, 4, 0, 4, 0, 0 4096: 4096, 0, 0, 1, 2496, 0, 0 4096: 4096, 0, 8, 0, 23602, 0, 0 4096: 4096, 0, 39, 1209, 8563389, 0, 0 4096: 4096, 0, 16, 4,42059411, 0, 0 4096: 4096, 0, 6, 0, 2208733, 0, 0 4096: 4096, 0, 27, 0, 28, 0, 0 4096: 4096, 0, 2, 0, 123, 0, 0 8192: 8192, 0, 0, 0, 0, 0, 0 8192: 8192, 0, 0, 0, 2177, 0, 0 8192: 8192, 0, 1, 0,59526564, 0, 0 8192: 8192, 0, 10, 1, 43254, 0, 0 8192: 8192, 0, 1, 0, 1, 0, 0 8192: 8192, 0, 7, 1, 561576, 0, 0 8192: 8192, 0, 1, 0, 2, 0, 0 8192: 8192, 0, 1, 0, 1, 0, 0 16384: 16384, 0, 1, 0, 1, 0, 0 16384: 16384, 0, 0, 8, 2060, 0, 0 16384: 16384, 0, 1, 0, 286024, 0, 0 16384: 16384, 0, 2, 0, 487, 0, 0 16384: 16384, 0, 1, 0, 1, 0, 0 16384: 16384, 0, 4, 0, 31916, 0, 0 16384: 16384, 0, 1, 0, 15542, 0, 0 16384: 16384, 0, 1, 0, 1, 0, 0 32768: 32768, 0, 5, 0, 42616, 0, 0 32768: 32768, 0, 0, 0, 934, 0, 0 32768: 32768, 0, 0, 0, 1224, 0, 0 32768: 32768, 0, 2, 0, 288, 0, 0 32768: 32768, 0, 1, 0, 1, 0, 0 32768: 32768, 0, 0, 3, 169343, 0, 0 32768: 32768, 0, 0, 3, 238148, 0, 0 32768: 32768, 0, 0, 0, 0, 0, 0 65536: 65536, 0, 0, 0, 0, 0, 0 65536: 65536, 0, 2, 0, 1026, 0, 0 65536: 65536, 0, 3, 0, 351, 0, 0 65536: 65536, 0, 0, 0, 974, 0, 0 65536: 65536, 0, 0, 0, 0, 0, 0 65536: 65536, 0, 1, 2, 564, 0, 0 65536: 65536, 0, 8, 0, 9, 0, 0 65536: 65536, 0, 1, 0, 1, 0, 0 SLEEPQUEUE: 48, 0, 16333, 1027, 16333, 0, 0 64 pcpu: 8, 0, 2375, 697, 2655, 0, 0 ptr pcpu: 4, 0, 0, 0, 0, 0, 0 Files: 56, 0, 344, 65260,171164502, 0, 0 filedesc0: 888, 0, 25, 1235, 8477577, 0, 0 rl_entry: 28, 0, 1843, 1017, 1843, 0, 0 TURNSTILE: 72, 0, 16333, 1988, 16333, 0, 0 umtx pi: 52, 0, 0, 0, 1, 0, 0 umtx_shm: 52, 0, 0, 0, 0, 0, 0 MAC labels: 20, 0, 0, 0, 0, 0, 0 PROC: 840, 0, 24, 18928, 8478097, 0, 0 THREAD: 836, 0, 6819, 9513, 6368752, 0, 0 cpuset: 40, 0, 99, 501, 42878, 0, 0 audit_record: 1112, 0, 0, 0, 0, 0, 0 mbuf_packet: 256, 209715, 256, 1026,60235724,119891, 0 mbuf: 256, 209715, 48, 1279,244925948, 0, 0 mbuf_cluster: 2048, 32768, 1268, 8,10746823,79536, 0 mbuf_jumbo_page: 4096, 16384, 0, 1,16783988, 0,3179052 mbuf_jumbo_9k: 9216, 14562, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 10920, 0, 0, 0, 0, 0 g_bio: 276, 0, 0, 2436,358406033, 953, 0 ttyinq: 152, 0, 10335, 377,14839035, 0, 0 ttyoutq: 256, 0, 5508, 237, 7914120, 0, 0 nvme_request: 100, 0, 0, 0, 0, 0, 0 vtnet_tx_hdr: 16, 0, 0, 0, 0, 0, 0 cryptop: 60, 0, 0, 0, 0, 0, 0 cryptodesc: 56, 0, 0, 0, 0, 0, 0 FPU_save_area: 512, 0, 0, 0, 0, 0, 0 VNODE: 360, 0, 5073, 44757,111446105, 0, 0 VNODEPOLL: 64, 0, 1, 61, 256, 0, 0 BUF TRIE: 44, 0, 507, 5317,74027529, 0, 0 S VFS Cache: 72, 0, 3699, 21326,60067468, 0, 0 STS VFS Cache: 92, 0, 0, 0, 1731950, 0, 0 L VFS Cache: 292, 0, 1, 12, 935662, 0, 0 LTS VFS Cache: 312, 0, 0, 0, 3183, 0, 0 NAMEI: 1024, 0, 0, 972,749352249, 0, 0 rentr: 12, 0, 0, 0, 0, 0, 0 NCLNODE: 360, 0, 0, 0, 729225, 0, 0 DIRHASH: 1024, 0, 572, 220, 1997415, 0, 0 Mountpoints: 780, 0, 8, 92, 78888, 0, 0 AIO: 128, 0, 0, 0, 0, 0, 0 AIOP: 16, 0, 0, 0, 0, 0, 0 AIOCB: 428, 0, 0, 0, 0, 0, 0 AIOL: 64, 0, 0, 0, 0, 0, 0 AIOLIO: 172, 0, 0, 0, 0, 0, 0 pipe: 416, 0, 0, 99, 9672502, 0, 0 procdesc: 80, 0, 0, 0, 1142, 0, 0 ksiginfo: 80, 0, 6734, 1166,12658055, 0, 0 itimer: 232, 0, 0, 0, 0, 0, 0 KNOTE: 72, 0, 0, 385,14485070, 0, 0 socket: 524, 109676, 3, 375, 194525, 0, 0 unpcb: 172, 109687, 1, 275, 7233, 0, 0 ipq: 32, 1125, 0, 0, 1483867,58552, 0 udp_inpcb: 304, 109681, 2, 115, 141664, 0, 0 udpcb: 20, 109800, 2, 798, 141664, 0, 0 tcp_inpcb: 304, 109681, 1, 116, 44476, 0, 0 tcpcb: 776, 109680, 0, 60, 44476, 0, 0 tcptw: 60, 21978, 1, 263, 8049, 0, 0 syncache: 128, 15376, 0, 155, 16033, 0, 0 hostcache: 76, 15392, 2, 154, 12, 0, 0 sackhole: 20, 0, 0, 0, 7115, 0, 0 tcpreass: 20, 2200, 0, 1600, 71463, 0, 0 sctp_ep: 1052, 109677, 0, 0, 0, 0, 0 sctp_asoc: 1656, 40000, 0, 0, 0, 0, 0 sctp_laddr: 24, 80160, 0, 0, 4, 0, 0 sctp_raddr: 540, 80003, 0, 0, 0, 0, 0 sctp_chunk: 104, 400026, 0, 0, 0, 0, 0 sctp_readq: 116, 400010, 0, 0, 0, 0, 0 sctp_stream_msg_out: 72, 400015, 0, 0, 0, 0, 0 sctp_asconf: 24, 400132, 0, 0, 0, 0, 0 sctp_asconf_ack: 24, 400132, 0, 0, 0, 0, 0 udplite_inpcb: 304, 109681, 0, 0, 0, 0, 0 ripcb: 304, 109681, 0, 13, 1145, 0, 0 rtentry: 112, 0, 13, 92, 13, 0, 0 selfd: 32, 0, 1027, 1348,14662987, 0, 0 SWAPMETA: 276, 131082, 1246, 21014, 789519, 0, 0 FFS inode: 112, 0, 4938, 45217,102369008, 0, 0 FFS1 dinode: 128, 0, 0, 0, 3629419, 0, 0 FFS2 dinode: 256, 0, 4938, 44817,98661657, 0, 0