GENERIC HEAD from 2018-09-29 01:26:07 UTC, r339009M, vmcore.645 GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb ---<>--- Copyright (c) 1992-2018 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 12.0-ALPHA8 #0 r339009M: Sat Sep 29 07:18:37 CEST 2018 pho@t2.osted.lan:/usr/src/sys/amd64/compile/PHO amd64 FreeBSD clang version 6.0.1 (tags/RELEASE_601/final 335540) (based on LLVM 6.0.1) WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. Entering uma_startup with 15 boot pages configured startup_alloc from "UMA Kegs", 13 boot pages left startup_alloc from "UMA Zones", 12 boot pages left startup_alloc from "UMA Zones", 11 boot pages left startup_alloc from "UMA Zones", 10 boot pages left startup_alloc from "UMA Zones", 9 boot pages left startup_alloc from "UMA Zones", 8 boot pages left startup_alloc from "UMA Zones", 7 boot pages left startup_alloc from "UMA Zones", 6 boot pages left startup_alloc from "UMA Zones", 5 boot pages left startup_alloc from "UMA Zones", 4 boot pages left startup_alloc from "UMA Zones", 3 boot pages left startup_alloc from "UMA Hash", 2 boot pages left startup_alloc from "UMA Zones", 1 boot pages left Entering uma_startup1 with 0 boot pages left Entering uma_startup2 with 0 boot pages left VT(vga): resolution 640x480 CPU: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz (1995.24-MHz K8-class CPU) Origin="GenuineIntel" Id=0x206d7 Family=0x6 Model=0x2d Stepping=7 Features=0xbfebfbff Features2=0x1fbee3ff AMD Features=0x2c100800 AMD Features2=0x1 XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID TSC: P-state invariant, performance statistics real memory = 68719476736 (65536 MB) avail memory = 35336466432 (33699 MB) : Trying to mount root from ufs:/dev/da0p2 [rw]... WARNING: DIAGNOSTIC option enabled, expect reduced performance. Expensive timeout(9) function: 0xffffffff80a00040(0xffffffff81af3bd0) 0.006703300 s uhub2: 4 ports with 4 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub0: 2 ports with 2 removable, self powered Setting hostuuid: 2bde2bde-f4e2-e111-aab2-001e6756b69b. Setting hostid: 0x0035ff86. Starting ddb. Starting file system checks: /dev/da0p2: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/da0p2: clean, 52959254 free (1144702 frags, 6476819 blocks, 1.2% fragmentation) /dev/ada0p2: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ada0p2: clean, 45109111 free (294903 frags, 5601776 blocks, 0.5% fragmentation) /dev/da0p3: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/da0p3: clean, 25332999 free (759 frags, 3166530 blocks, 0.0% fragmentation) Mounting local file systems:. ugen2.2: at usbus2 uhub3 numa-domain 0 on uhub0 uhub3: on usbus2 ugen0.2: at usbus0 uhub4 numa-domain 0 on uhub1 uhub4: on usbus0 ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat /usr/local/lib/gcc48 /usr/local/lib/nss /usr/local/lib/perl5/5.24/mach/CORE /usr/local/lib/qt4 /usr/local/lib/qt5 /usr/local/llvm37/lib 32-bit compatibility ldconfig path: /usr/lib32 /usr/local/lib32/compat Setting hostname: t2.osted.lan. Setting up harvesting:[UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,NET_ETHER,NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED Feeding entropy:uhub4: 6 ports with 6 removable, self powered . uhub3: 8 ports with 8 removable, self powered lo0: link state changed to UP ugen2.3: at usbus2 uhub5 numa-domain 0 on uhub3 uhub5: on usbus2 uhub5: MTT enabled uhub5: 4 ports with 4 removable, self powered ugen2.4: at usbus2 ukbd0 numa-domain 0 on uhub3 ukbd0: on usbus2 kbd2 at ukbd0 Starting Network: lo0 igb0 igb1. lo0: flags=8049 metric 0 mtu 16384 options=680003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=21 igb0: flags=8843 metric 0 mtu 1500 options=e527bb ether 00:1e:67:56:b6:9b inet 192.168.1.109 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::21e:67ff:fe56:b69b%igb0 prefixlen 64 scopeid 0x1 media: Ethernet autoselect (1000baseT ) status: active nd6 options=23 igb1: flags=8802 metric 0 mtu 1500 options=e527bb ether 00:1e:67:56:b6:9c media: Ethernet autoselect status: no carrier nd6 options=29 Starting devd. Starting Network: igb1. igb1: flags=8802 metric 0 mtu 1500 options=e527bb ether 00:1e:67:56:b6:9c media: Ethernet autoselect status: no carrier nd6 options=29 Configuring vt: keymap. ums0 numa-domain 0 on uhub3 ums0: on usbus2 ums0: 3 buttons and [Z] coordinates ID=0 add net default: gateway 192.168.1.1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Starting rtsold. Creating and/or trimming log files. Starting syslogd. Starting rpcbind. NFS access cache time=60 Starting watchdogd. No core dumps found. Starting casperd. lock order reversal: 1st 0xfffffe00008c4100 bufwait (bufwait) @ kern/vfs_bio.c:3916 2nd 0xfffff80007e33200 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:289 stack backtrace: #0 0xffffffff80bf8ad3 at witness_debugger+0x73 #1 0xffffffff80bf8823 at witness_checkorder+0xab3 #2 0xffffffff80b99348 at _sx_xlock+0x68 #3 0xffffffff80eaffed at ufsdirhash_add+0x4d #4 0xffffffff80eb4b22 at ufs_direnter+0x472 #5 0xffffffff80ebd040 at ufs_mkdir+0x8f0 #6 0xffffffff811e7b92 at VOP_MKDIR_APV+0xc2 #7 0xffffffff80c7150e at kern_mkdirat+0x1be #8 0xffffffff81068847 at amd64_syscall+0x2a7 #9 0xffffffff81044dcd at fast_syscall_common+0x101 Clearing /tmp (X related). Starting nfsuserd. Starting mountd. Starting nfsd. Starting statd. Starting lockd. Recovering vi editor sessions:. Updating motd:. Mounting late file systems:. Starting ntpd. Starting powerd. Configuring vt: keymap blanktime. Performing sanity check on sshd configuration. Starting sshd. Starting sendmail_submit. Starting sendmail_msp_queue. Starting cron. Local package initialization: Sep 29 07:32:49 t2 su[863]: pho to root on /dev/pts/0 backup. sysctl: unknown oid 'debug.mnowait_failure.zalloc_whitelist' at line 13 sysctl: unknown oid 'debug.fail_point.uma_zalloc_arg' at line 14 Starting nfscbd. Starting default mousedmoused: unable to open /dev/psm0: No such file or directory . Starting inetd. Sat Sep 29 07:33 FreeBSD/amd64 (t2.osted.lan) (ttyu1) login: 20180929 07:41:16 all (1/294): 1st.sh 20180929 07:41:21 all (2/294): 256m.sh 20180929 07:41:23 all (3/294): advlock.sh lock order reversal: 1st 0xfffff8000b1b1558 ufs (ufs) @ kern/vfs_mount.c:1343 2nd 0xfffff80860d6da48 devfs (devfs) @ ufs/ffs/ffs_vfsops.c:1376 stack backtrace: #0 0xffffffff80bf8ad3 at witness_debugger+0x73 #1 0xffffffff80bf8823 at witness_checkorder+0xab3 #2 0xffffffff80b648d9 at lockmgr_lock_fast_path+0x169 #3 0xffffffff811e88f6 at VOP_LOCK1_APV+0xa6 #4 0xffffffff80c74330 at _vn_lock+0x60 #5 0xffffffff80ea6654 at ffs_flushfiles+0xa4 #6 0xffffffff80e89d32 at softdep_flushfiles+0x82 #7 0xffffffff80ea8cf9 at ffs_unmount+0xd9 #8 0xffffffff80c5ad42 at dounmount+0x512 #9 0xffffffff80c5a7b0 at sys_unmount+0x310 #10 0xffffffff81068847 at amd64_syscall+0x2a7 #11 0xffffffff81044dcd at fast_syscall_common+0x101 lock order reversal: 1st 0xfffff8000b002cc0 ufs (ufs) @ kern/vfs_subr.c:2590 2nd 0xfffffe00008c4100 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:282 3rd 0xfffff80860a80068 ufs (ufs) @ kern/vfs_subr.c:2590 stack backtrace: #0 0xffffffff80bf8ad3 at witness_debugger+0x73 #1 0xffffffff80bf8823 at witness_checkorder+0xab3 #2 0xffffffff80b64d9b at lockmgr_xlock_hard+0x6b #3 0xffffffff80b6562e at __lockmgr_args+0x3ce #4 0xffffffff80eaae5c at ffs_lock+0x7c #5 0xffffffff811e88f6 at VOP_LOCK1_APV+0xa6 #6 0xffffffff80c74330 at _vn_lock+0x60 #7 0xffffffff80c62806 at vget+0x96 #8 0xffffffff80c54ac1 at vfs_hash_get+0xd1 #9 0xffffffff80ea679f at ffs_vgetf+0x3f #10 0xffffffff80e9c921 at softdep_sync_buf+0xa01 #11 0xffffffff80eabbf4 at ffs_syncvnode+0x224 #12 0xffffffff80e81ad5 at ffs_truncate+0x6e5 #13 0xffffffff80eb4ceb at ufs_direnter+0x63b #14 0xffffffff80ebde0c at ufs_makeinode+0x61c #15 0xffffffff80eb9c34 at ufs_create+0x34 #16 0xffffffff811e55fc at VOP_CREATE_APV+0xbc #17 0xffffffff80c73c0c at vn_open_cred+0x29c mount leak: 1/65 20180929 07:42:31 all (4/294): aesni.sh aesni0: on motherboard GEOM_ELI: Device md10.eli created. GEOM_ELI: Encryption: AES-XTS 128 GEOM_ELI: Crypto: hardware GEOM_ELI: md10 has been killed. GEOM_ELI: Device md10.eli destroyed. aesni0: detached mount leak: 1/66 20180929 07:50:26 all (5/294): alternativeFlushPath.sh lock order reversal: 1st 0xfffff80842b0a040 filedesc structure (filedesc structure) @ kern/kern_descrip.c:842 2nd 0xfffff808475a92e0 ufs (ufs) @ kern/vfs_subr.c:5045 stack backtrace: #0 0xffffffff80bf8ad3 at witness_debugger+0x73 #1 0xffffffff80bf8823 at witness_checkorder+0xab3 #2 0xffffffff80b64d9b at lockmgr_xlock_hard+0x6b #3 0xffffffff80b6562e at __lockmgr_args+0x3ce #4 0xffffffff80eaae5c at ffs_lock+0x7c #5 0xffffffff811e88f6 at VOP_LOCK1_APV+0xa6 #6 0xffffffff80c74330 at _vn_lock+0x60 #7 0xffffffff80b4576a at knlist_remove_kq+0x8a #8 0xffffffff80c69e17 at filt_vfsdetach+0x27 #9 0xffffffff80b461b1 at knote_fdclose+0x151 #10 0xffffffff80b39416 at closefp+0x66 #11 0xffffffff80b37e18 at kern_dup+0x5f8 #12 0xffffffff81068847 at amd64_syscall+0x2a7 #13 0xffffffff81044dcd at fast_syscall_common+0x101 freework leak: 7/18. 20180929 08:10:43 all (6/294): audit.sh 20180929 08:10:51 all (7/294): audit2.sh Sep 29 08:11:09 t2 kernel: pid 63224 (lockf), uid 1004 inumber 111732 on /mnt: filesystem full Sep 29 08:11:09 t2 kernel: pid 63283 (rw), uid 1004 inumber 92758 on /mnt: filesystem full Sep 29 08:11:09 t2 kernel: pid 63240 (creat), uid 1004 inumber 65542 on /mnt: filesystem full mount leak: 1/67 20180929 08:17:36 all (8/294): bench.sh mount leak: 5/72 20180929 08:19:14 all (9/294): bio.sh lock order reversal: 1st 0xfffff805c81c1558 syncer (syncer) @ kern/vfs_subr.c:2170 2nd 0xfffff807d999ccc0 ufs (ufs) @ kern/vfs_subr.c:2590 stack backtrace: #0 0xffffffff80bf8ad3 at witness_debugger+0x73 #1 0xffffffff80bf8823 at witness_checkorder+0xab3 #2 0xffffffff80b64d9b at lockmgr_xlock_hard+0x6b #3 0xffffffff80b6562e at __lockmgr_args+0x3ce #4 0xffffffff80eaae5c at ffs_lock+0x7c #5 0xffffffff811e88f6 at VOP_LOCK1_APV+0xa6 #6 0xffffffff80c74330 at _vn_lock+0x60 #7 0xffffffff80c62806 at vget+0x96 #8 0xffffffff80c64a76 at vfs_msync+0xa6 #9 0xffffffff80c69a42 at sync_fsync+0xc2 #10 0xffffffff811e73f2 at VOP_FSYNC_APV+0xc2 #11 0xffffffff80c678b4 at sched_sync+0x284 #12 0xffffffff80b501f4 at fork_exit+0x84 #13 0xffffffff810454ce at fork_trampoline+0xe mount leak: 1/73 20180929 08:24:55 all (10/294): bug131876.sh 20180929 08:25:15 all (11/294): buildworld.sh freework leak: 1/19. mount leak: 1/74 20180929 08:46:14 all (12/294): buildworld2.sh 20180929 08:46:20 all (13/294): buildworld4.sh mount leak: 1/75 20180929 08:56:29 all (14/294): burnin.sh pts leak: 1/3 mount leak: 1/76 20180929 09:17:41 all (15/294): callout_reset_on.sh Expensive timeout(9) function: 0xffffffff80ba7420(0xfffff8085502ca50) 0.006750698 s 20180929 09:19:23 all (16/294): callout_reset_on2.sh Expensive timeout(9) function: 0xffffffff80dbcb10(0xfffff80849e44000) 11.062722716 s 20180929 09:29:28 all (17/294): churn.sh 20180929 09:39:33 all (18/294): cluster.sh newblk leak: 13715/13728. 20180929 09:51:57 all (19/294): cmp.sh mount leak: 328/404 20180929 09:57:17 all (20/294): context.sh Sep 29 09:59:47 t2 kernel: pid 36295 (swap), uid 0, was killed: out of swap space Sep 29 09:59:48 t2 kernel: pid 36330 (swap), uid 0, was killed: out of swap space Sep 29 09:59:53 t2 kernel: pid 36325 (swap), uid 0, was killed: out of swap space Sep 29 10:00:02 t2 kernel: pid 36306 (swap), uid 0, was killed: out of swap space Sep 29 10:00:08 t2 kernel: pid 36320 (swap), uid 0, was killed: out of swap space Sep 29 10:00:19 t2 kernel: pid 36315 (swap), uid 0, was killed: out of swap space Sep 29 10:01:54 t2 kernel: pid 36310 (swap), uid 0, was killed: out of swap space 20180929 10:02:25 all (21/294): context2.sh Sep 29 10:02:30 t2 kernel: pid 58435 (swap), uid 0, was killed: out of swap space Sep 29 10:02:39 t2 kernel: pid 58438 (swap), uid 0, was killed: out of swap space Sep 29 10:03:04 t2 kernel: pid 58433 (swap), uid 0, was killed: out of swap space 20180929 10:07:30 all (22/294): contigmalloc.sh 20180929 10:09:46 all (23/294): contigmalloc2.sh 20180929 10:09:52 all (24/294): core.sh 20180929 10:12:06 all (25/294): core2.sh mount leak: 1/405 20180929 10:14:39 all (26/294): core3.sh Sep 29 10:14:42 t2 kernel: Failed to write core file for process core3 (error 5) Sep 29 10:15:13 t2 syslogd: last message repeated 4390 times Sep 29 10:15:51 t2 syslogd: last message repeated 5283 times Sep 29 10:15:51 t2 mountd[702]: bad exports list line '/mnt': symbolic link in export path or statfs failed Sep 29 10:15:51 t2 kernel: Failed to write core file for process core3 (error 5) Sep 29 10:16:22 t2 syslogd: last message repeated 4650 times Sep 29 10:16:41 t2 syslogd: last message repeated 2818 times Sep 29 10:16:41 t2 mountd[702]: bad exports list line '/mnt': symbolic link in export path or statfs failed Sep 29 10:16:41 t2 kernel: Failed to write core file for process core3 (error 5) Sep 29 10:17:12 t2 syslogd: last message repeated 4692 times Sep 29 10:19:13 t2 syslogd: last message repeated 17273 times Sep 29 10:19:56 t2 syslogd: last message repeated 5851 times Sep 29 10:19:57 t2 mountd[702]: bad exports list line '/mnt': symbolic link in export path or statfs failed Sep 29 10:19:57 t2 kernel: Failed to write core file for process core3 (error 5) Sep 29 10:20:28 t2 syslogd: last message repeated 4396 times Sep 29 10:22:28 t2 syslogd: last message repeated 17972 times Sep 29 10:24:23 t2 syslogd: last message repeated 17003 times Sep 29 10:24:23 t2 mountd[702]: bad exports list line '/mnt': symbolic link in export path or statfs failed Sep 29 10:24:23 t2 kernel: Failed to write core file for process core3 (error 5) Sep 29 10:24:41 t2 syslogd: last message repeated 2844 times Sep 29 10:25:30 t2 mountd[702]: bad exports list line '/mnt': symbolic link in export path or statfs failed Sep 29 10:25:48 t2 syslogd: last message repeated 1 times Sep 29 10:28:04 t2 syslogd: last message repeated 5 times mount leak: 32299/32704 GEOM leak: 15/149 20180929 10:34:42 all (27/294): core4.sh Sep 29 10:34:28 t2 syslogd: last message repeated 12 times Sep 29 10:34:46 t2 kernel: Failed to write core file for process core4 (error 5) mount leak: 20/32724 20180929 10:35:09 all (28/294): core5.sh Sep 29 10:35:07 t2 syslogd: last message repeated 59 times mount leak: 1/32725 20180929 10:45:15 all (29/294): cpuset.sh Sep 29 10:47:36 t2 kernel: pid 750 (ntpd), uid 0, was killed: out of swap space Sep 29 10:47:44 t2 kernel: pid 702 (mountd), uid 0, was killed: out of swap space Sep 29 10:47:45 t2 kernel: pid 712 (rpc.statd), uid 0, was killed: out of swap space Sep 29 10:47:46 t2 kernel: pid 715 (rpc.lockd), uid 0, was killed: out of swap space Sep 29 10:47:47 t2 kernel: pid 473 (devd), uid 0, was killed: out of swap space Sep 29 10:47:50 t2 kernel: pid 708 (nfsd), uid 0, was killed: out of swap space Sep 29 10:47:55 t2 kernel: pid 2372 (top), uid 1001, was killed: out of swap space Sep 29 10:48:00 t2 kernel: pid 852 (sshd), uid 1001, was killed: out of swap space Sep 29 10:48:04 t2 kernel: pid 817 (sendmail), uid 25, was killed: out of swap space Sep 29 10:48:08 t2 kernel: pid 814 (sendmail), uid 0, was killed: out of swap space Sep 29 10:48:10 t2 kernel: pid 2371 (sshd), uid 1001, was killed: out of swap space Sep 29 10:48:11 t2 kernel: pid 95296 (bash), uid 1001, was killed: out of swap space Sep 29 10:48:12 t2 kernel: pid 2370 (sshd), uid 1001, was killed: out of swap space Sep 29 10:48:14 t2 kernel: pid 347 (vmstat), uid 1001, was killed: out of swap space pid 573 (rpcbind), uid 0, was killed: out of swap space pid 711 (nfsd), uid 0, was killed: out of swap space pid 753 (powerd), uid 0, was killed: out of swap space pid 971 (getty), uid 0, was killed: out of swap space FreeBSD/amd64 (t2.osted.lan) (ttyu1) login: pid 352 (getty), uid 0, was killed: out of swap space FreeBSD/amd64 (t2.osted.lan) (ttyu1) login: pid 353 (getty), uid 0, was killed: out of swap space FreeBSD/amd64 (t2.osted.lan) (ttyu1) login: pid 354 (getty), uid 0, was killed: out of swap space FreeBSD/amd64 (t2.osted.lan) (ttyu1) login: pid 355 (getty), uid 0, was killed: out of swap space 2018-09-29T10:48:25.575422+02:00 t2.osted.lan init 356 - - getty repeating too quickly on port /dev/ttyu1, sleeping 30 secs pid 970 (getty), uid 0, was killed: out of swap space pid 357 (getty), uid 0, was killed: out of swap space pid 358 (getty), uid 0, was killed: out of swap space pid 359 (getty), uid 0, was killed: out of swap space pid 360 (getty), uid 0, was killed: out of swap space 2018-09-29T10:48:31.973465+02:00 t2.osted.lan init 361 - - getty repeating too quickly on port /dev/ttyv7, sleeping 30 secs pid 969 (getty), uid 0, was killed: out of swap space pid 362 (getty), uid 0, was killed: out of swap space pid 363 (getty), uid 0, was killed: out of swap space pid 364 (getty), uid 0, was killed: out of swap space pid 367 (ls), uid 0, was killed: out of swap space pid 365 (getty), uid 0, was killed: out of swap space 2018-09-29T10:48:39.639452+02:00 t2.osted.lan init 368 - - getty repeating too quickly on port /dev/ttyv6, sleeping 30 secs pid 366 (sh), uid 0, was killed: out of swap space pid 968 (getty), uid 0, was killed: out of swap space pid 369 (getty), uid 0, was killed: out of swap space pid 370 (getty), uid 0, was killed: out of swap space pid 371 (getty), uid 0, was killed: out of swap space pid 372 (getty), uid 0, was killed: out of swap space 2018-09-29T10:48:47.301254+02:00 t2.osted.lan init 373 - - getty repeating too quickly on port /dev/ttyv5, sleeping 30 secs pid 967 (getty), uid 0, was killed: out of swap space pid 374 (getty), uid 0, was killed: out of swap space pid 375 (getty), uid 0, was killed: out of swap space pid 376 (getty), uid 0, was killed: out of swap space pid 377 (getty), uid 0, was killed: out of swap space 2018-09-29T10:48:54.116332+02:00 t2.osted.lan init 378 - - getty repeating too quickly on port /dev/ttyv4, sleeping 30 secs pid 966 (getty), uid 0, was killed: out of swap space FreeBSD/amd64 (t2.osted.lan) (ttyu1) login: pid 379 (getty), uid 0, was killed: out of swap space pid 380 (getty), uid 0, was killed: out of swap space FreeBSD/amd64 (t2.osted.lan) (ttyu1) login: KDB: enter: Break to debugger [ thread pid 11 tid 100007 ] Stopped at kdb_alt_break_internal+0x106: movq $0,kdb_why db> run pho db:0:pho> set $lines 20000 db:0:pho> run pho1 db:1:pho1> bt Tracing pid 11 tid 100007 td 0xfffff80003696000 kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007876e730 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007876e740 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007876e770 uart_intr() at uart_intr+0x138/frame 0xfffffe007876e7b0 intr_event_handle() at intr_event_handle+0xcb/frame 0xfffffe007876e800 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007876e830 lapic_handle_intr() at lapic_handle_intr+0x5f/frame 0xfffffe007876e850 Xapic_isr1() at Xapic_isr1+0xd9/frame 0xfffffe007876e850 --- interrupt, rip = 0xffffffff8046209c, rsp = 0xfffffe007876e920, rbp = 0xfffffe007876e960 --- acpi_cpu_idle() at acpi_cpu_idle+0x34c/frame 0xfffffe007876e960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007876e980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe007876e9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007876ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe007876eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007876eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db:1:pho1> show allpcpu Current CPU: 4 cpuid = 0 dynamic pcpu = 0xb754c0 curthread = 0xfffff80003697000: pid 11 tid 100003 "idle: cpu0" curpcb = 0xfffffe007875ab80 fpcurthread = none idlethread = 0xfffff80003697000: tid 100003 "idle: cpu0" curpmap = 0xffffffff8208df28 tssp = 0xffffffff821b92a0 commontssp = 0xffffffff821b92a0 rsp0 = 0xfffffe007875ab80 gs32p = 0xffffffff821bfed8 ldt = 0xffffffff821bff18 tss = 0xffffffff821bff08 curvnet = 0 spin locks held: cpuid = 1 dynamic pcpu = 0xfffffe00850174c0 curthread = 0xfffff808400f5580: pid 11 tid 100004 "idle: cpu1" curpcb = 0xfffffe007875fb80 fpcurthread = none idlethread = 0xfffff808400f5580: tid 100004 "idle: cpu1" curpmap = 0xffffffff8208df28 tssp = 0xffffffff821b9308 commontssp = 0xffffffff821b9308 rsp0 = 0xfffffe007875fb80 gs32p = 0xffffffff821bff40 ldt = 0xffffffff821bff80 tss = 0xffffffff821bff70 curvnet = 0 spin locks held: cpuid = 2 dynamic pcpu = 0xfffffe00850234c0 curthread = 0xfffff808400f5000: pid 11 tid 100005 "idle: cpu2" curpcb = 0xfffffe0078764b80 fpcurthread = none idlethread = 0xfffff808400f5000: tid 100005 "idle: cpu2" curpmap = 0xffffffff8208df28 tssp = 0xffffffff821b9370 commontssp = 0xffffffff821b9370 rsp0 = 0xfffffe0078764b80 gs32p = 0xffffffff821bffa8 ldt = 0xffffffff821bffe8 tss = 0xffffffff821bffd8 curvnet = 0 spin locks held: cpuid = 3 dynamic pcpu = 0xfffffe008502f4c0 curthread = 0xfffff80003696580: pid 11 tid 100006 "idle: cpu3" curpcb = 0xfffffe0078769b80 fpcurthread = none idlethread = 0xfffff80003696580: tid 100006 "idle: cpu3" curpmap = 0xffffffff8208df28 tssp = 0xffffffff821b93d8 commontssp = 0xffffffff821b93d8 rsp0 = 0xfffffe0078769b80 gs32p = 0xffffffff821c0010 ldt = 0xffffffff821c0050 tss = 0xffffffff821c0040 curvnet = 0 spin locks held: cpuid = 4 dynamic pcpu = 0xfffffe008503b4c0 curthread = 0xfffff80003696000: pid 11 tid 100007 "idle: cpu4" curpcb = 0xfffffe007876eb80 fpcurthread = none idlethread = 0xfffff80003696000: tid 100007 "idle: cpu4" curpmap = 0xffffffff8208df28 tssp = 0xffffffff821b9440 commontssp = 0xffffffff821b9440 rsp0 = 0xfffffe007876eb80 gs32p = 0xffffffff821c0078 ldt = 0xffffffff821c00b8 tss = 0xffffffff821c00a8 curvnet = 0 spin locks held: cpuid = 5 dynamic pcpu = 0xfffffe00850474c0 curthread = 0xfffff808400f4580: pid 11 tid 100008 "idle: cpu5" curpcb = 0xfffffe0078773b80 fpcurthread = none idlethread = 0xfffff808400f4580: tid 100008 "idle: cpu5" curpmap = 0xffffffff8208df28 tssp = 0xffffffff821b94a8 commontssp = 0xffffffff821b94a8 rsp0 = 0xfffffe0078773b80 gs32p = 0xffffffff821c00e0 ldt = 0xffffffff821c0120 tss = 0xffffffff821c0110 curvnet = 0 spin locks held: cpuid = 6 dynamic pcpu = 0xfffffe00850534c0 curthread = 0xfffff808400f4000: pid 11 tid 100009 "idle: cpu6" curpcb = 0xfffffe0078778b80 fpcurthread = none idlethread = 0xfffff808400f4000: tid 100009 "idle: cpu6" curpmap = 0xffffffff8208df28 tssp = 0xffffffff821b9510 commontssp = 0xffffffff821b9510 rsp0 = 0xfffffe0078778b80 gs32p = 0xffffffff821c0148 ldt = 0xffffffff821c0188 tss = 0xffffffff821c0178 curvnet = 0 spin locks held: cpuid = 7 dynamic pcpu = 0xfffffe008505f4c0 curthread = 0xfffff800036a1580: pid 11 tid 100010 "idle: cpu7" curpcb = 0xfffffe007877db80 fpcurthread = none idlethread = 0xfffff800036a1580: tid 100010 "idle: cpu7" curpmap = 0xffffffff8208df28 tssp = 0xffffffff821b9578 commontssp = 0xffffffff821b9578 rsp0 = 0xfffffe007877db80 gs32p = 0xffffffff821c01b0 ldt = 0xffffffff821c01f0 tss = 0xffffffff821c01e0 curvnet = 0 spin locks held: cpuid = 8 dynamic pcpu = 0xfffffe007e6d44c0 curthread = 0xfffff800036a1000: pid 11 tid 100011 "idle: cpu8" curpcb = 0xfffffe0078782b80 fpcurthread = none idlethread = 0xfffff800036a1000: tid 100011 "idle: cpu8" curpmap = 0xffffffff8208df28 tssp = 0xffffffff821b95e0 commontssp = 0xffffffff821b95e0 rsp0 = 0xfffffe0078782b80 gs32p = 0xffffffff821c0218 ldt = 0xffffffff821c0258 tss = 0xffffffff821c0248 curvnet = 0 spin locks held: cpuid = 9 dynamic pcpu = 0xfffffe007e6e04c0 curthread = 0xfffff808400ff580: pid 11 tid 100012 "idle: cpu9" curpcb = 0xfffffe0078787b80 fpcurthread = none idlethread = 0xfffff808400ff580: tid 100012 "idle: cpu9" curpmap = 0xffffffff8208df28 tssp = 0xffffffff821b9648 commontssp = 0xffffffff821b9648 rsp0 = 0xfffffe0078787b80 gs32p = 0xffffffff821c0280 ldt = 0xffffffff821c02c0 tss = 0xffffffff821c02b0 curvnet = 0 spin locks held: cpuid = 10 dynamic pcpu = 0xfffffe007e6ec4c0 curthread = 0xfffff808400ff000: pid 11 tid 100013 "idle: cpu10" curpcb = 0xfffffe007878cb80 fpcurthread = none idlethread = 0xfffff808400ff000: tid 100013 "idle: cpu10" curpmap = 0xffffffff8208df28 tssp = 0xffffffff821b96b0 commontssp = 0xffffffff821b96b0 rsp0 = 0xfffffe007878cb80 gs32p = 0xffffffff821c02e8 ldt = 0xffffffff821c0328 tss = 0xffffffff821c0318 curvnet = 0 spin locks held: cpuid = 11 dynamic pcpu = 0xfffffe007e6f84c0 curthread = 0xfffff800036a0580: pid 11 tid 100014 "idle: cpu11" curpcb = 0xfffffe0078791b80 fpcurthread = none idlethread = 0xfffff800036a0580: tid 100014 "idle: cpu11" curpmap = 0xffffffff8208df28 tssp = 0xffffffff821b9718 commontssp = 0xffffffff821b9718 rsp0 = 0xfffffe0078791b80 gs32p = 0xffffffff821c0350 ldt = 0xffffffff821c0390 tss = 0xffffffff821c0380 curvnet = 0 spin locks held: cpuid = 12 dynamic pcpu = 0xfffffe007e7044c0 curthread = 0xfffff800036a0000: pid 11 tid 100015 "idle: cpu12" curpcb = 0xfffffe0078796b80 fpcurthread = none idlethread = 0xfffff800036a0000: tid 100015 "idle: cpu12" curpmap = 0xffffffff8208df28 tssp = 0xffffffff821b9780 commontssp = 0xffffffff821b9780 rsp0 = 0xfffffe0078796b80 gs32p = 0xffffffff821c03b8 ldt = 0xffffffff821c03f8 tss = 0xffffffff821c03e8 curvnet = 0 spin locks held: cpuid = 13 dynamic pcpu = 0xfffffe007e7104c0 curthread = 0xfffff808400fe580: pid 11 tid 100016 "idle: cpu13" curpcb = 0xfffffe007879bb80 fpcurthread = none idlethread = 0xfffff808400fe580: tid 100016 "idle: cpu13" curpmap = 0xffffffff8208df28 tssp = 0xffffffff821b97e8 commontssp = 0xffffffff821b97e8 rsp0 = 0xfffffe007879bb80 gs32p = 0xffffffff821c0420 ldt = 0xffffffff821c0460 tss = 0xffffffff821c0450 curvnet = 0 spin locks held: cpuid = 14 dynamic pcpu = 0xfffffe007e71c4c0 curthread = 0xfffff808400fe000: pid 11 tid 100017 "idle: cpu14" curpcb = 0xfffffe00787a0b80 fpcurthread = none idlethread = 0xfffff808400fe000: tid 100017 "idle: cpu14" curpmap = 0xffffffff8208df28 tssp = 0xffffffff821b9850 commontssp = 0xffffffff821b9850 rsp0 = 0xfffffe00787a0b80 gs32p = 0xffffffff821c0488 ldt = 0xffffffff821c04c8 tss = 0xffffffff821c04b8 curvnet = 0 spin locks held: cpuid = 15 dynamic pcpu = 0xfffffe007e7284c0 curthread = 0xfffff808400f6580: pid 11 tid 100018 "idle: cpu15" curpcb = 0xfffffe00787a5b80 fpcurthread = none idlethread = 0xfffff808400f6580: tid 100018 "idle: cpu15" curpmap = 0xffffffff8208df28 tssp = 0xffffffff821b98b8 commontssp = 0xffffffff821b98b8 rsp0 = 0xfffffe00787a5b80 gs32p = 0xffffffff821c04f0 ldt = 0xffffffff821c0530 tss = 0xffffffff821c0520 curvnet = 0 spin locks held: cpuid = 16 dynamic pcpu = 0xfffffe007e7344c0 curthread = 0xfffff80840102000: pid 11 tid 100019 "idle: cpu16" curpcb = 0xfffffe00787aab80 fpcurthread = none idlethread = 0xfffff80840102000: tid 100019 "idle: cpu16" curpmap = 0xffffffff8208df28 tssp = 0xffffffff821b9920 commontssp = 0xffffffff821b9920 rsp0 = 0xfffffe00787aab80 gs32p = 0xffffffff821c0558 ldt = 0xffffffff821c0598 tss = 0xffffffff821c0588 curvnet = 0 spin locks held: cpuid = 17 dynamic pcpu = 0xfffffe007e7404c0 curthread = 0xfffff800036a3580: pid 11 tid 100020 "idle: cpu17" curpcb = 0xfffffe00787afb80 fpcurthread = none idlethread = 0xfffff800036a3580: tid 100020 "idle: cpu17" curpmap = 0xffffffff8208df28 tssp = 0xffffffff821b9988 commontssp = 0xffffffff821b9988 rsp0 = 0xfffffe00787afb80 gs32p = 0xffffffff821c05c0 ldt = 0xffffffff821c0600 tss = 0xffffffff821c05f0 curvnet = 0 spin locks held: cpuid = 18 dynamic pcpu = 0xfffffe007e74c4c0 curthread = 0xfffff800036a3000: pid 11 tid 100021 "idle: cpu18" curpcb = 0xfffffe00787b4b80 fpcurthread = none idlethread = 0xfffff800036a3000: tid 100021 "idle: cpu18" curpmap = 0xffffffff8208df28 tssp = 0xffffffff821b99f0 commontssp = 0xffffffff821b99f0 rsp0 = 0xfffffe00787b4b80 gs32p = 0xffffffff821c0628 ldt = 0xffffffff821c0668 tss = 0xffffffff821c0658 curvnet = 0 spin locks held: cpuid = 19 dynamic pcpu = 0xfffffe007e7584c0 curthread = 0xfffff80840101580: pid 11 tid 100022 "idle: cpu19" curpcb = 0xfffffe00787b9b80 fpcurthread = none idlethread = 0xfffff80840101580: tid 100022 "idle: cpu19" curpmap = 0xffffffff8208df28 tssp = 0xffffffff821b9a58 commontssp = 0xffffffff821b9a58 rsp0 = 0xfffffe00787b9b80 gs32p = 0xffffffff821c0690 ldt = 0xffffffff821c06d0 tss = 0xffffffff821c06c0 curvnet = 0 spin locks held: cpuid = 20 dynamic pcpu = 0xfffffe007e7644c0 curthread = 0xfffff80840101000: pid 11 tid 100023 "idle: cpu20" curpcb = 0xfffffe00787beb80 fpcurthread = none idlethread = 0xfffff80840101000: tid 100023 "idle: cpu20" curpmap = 0xffffffff8208df28 tssp = 0xffffffff821b9ac0 commontssp = 0xffffffff821b9ac0 rsp0 = 0xfffffe00787beb80 gs32p = 0xffffffff821c06f8 ldt = 0xffffffff821c0738 tss = 0xffffffff821c0728 curvnet = 0 spin locks held: cpuid = 21 dynamic pcpu = 0xfffffe007e7704c0 curthread = 0xfffff800036a2580: pid 11 tid 100024 "idle: cpu21" curpcb = 0xfffffe00787c3b80 fpcurthread = none idlethread = 0xfffff800036a2580: tid 100024 "idle: cpu21" curpmap = 0xffffffff8208df28 tssp = 0xffffffff821b9b28 commontssp = 0xffffffff821b9b28 rsp0 = 0xfffffe00787c3b80 gs32p = 0xffffffff821c0760 ldt = 0xffffffff821c07a0 tss = 0xffffffff821c0790 curvnet = 0 spin locks held: cpuid = 22 dynamic pcpu = 0xfffffe007e77c4c0 curthread = 0xfffff800036a2000: pid 11 tid 100025 "idle: cpu22" curpcb = 0xfffffe00787c8b80 fpcurthread = none idlethread = 0xfffff800036a2000: tid 100025 "idle: cpu22" curpmap = 0xffffffff8208df28 tssp = 0xffffffff821b9b90 commontssp = 0xffffffff821b9b90 rsp0 = 0xfffffe00787c8b80 gs32p = 0xffffffff821c07c8 ldt = 0xffffffff821c0808 tss = 0xffffffff821c07f8 curvnet = 0 spin locks held: cpuid = 23 dynamic pcpu = 0xfffffe007e7884c0 curthread = 0xfffff80840100580: pid 11 tid 100026 "idle: cpu23" curpcb = 0xfffffe00787cdb80 fpcurthread = none idlethread = 0xfffff80840100580: tid 100026 "idle: cpu23" curpmap = 0xffffffff8208df28 tssp = 0xffffffff821b9bf8 commontssp = 0xffffffff821b9bf8 rsp0 = 0xfffffe00787cdb80 gs32p = 0xffffffff821c0830 ldt = 0xffffffff821c0870 tss = 0xffffffff821c0860 curvnet = 0 spin locks held: db:1:pho1> show alllocks Process 367 (ls) thread 0xfffff8000be40000 (100854) shared lockmgr ufs (ufs) r = 0 (0xfffff808837247d0) locked @ kern/vfs_lookup.c:662 Process 95680 (ufsbench) thread 0xfffff8086e133580 (102448) exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe00011acd90) locked @ kern/vfs_bio.c:1670 exclusive lockmgr ufs (ufs) r = 0 (0xfffff803369d0068) locked @ kern/vfs_vnops.c:864 Process 95679 (ufsbench) thread 0xfffff802ee2ac580 (102016) exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe000352cac8) locked @ kern/vfs_bio.c:1670 exclusive lockmgr ufs (ufs) r = 0 (0xfffff803369d02e0) locked @ kern/vfs_vnops.c:864 Process 95678 (ufsbench) thread 0xfffff803148a5000 (102373) exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe0001dacf70) locked @ kern/vfs_bio.c:1670 exclusive lockmgr ufs (ufs) r = 0 (0xfffff803c9dbc558) locked @ kern/vfs_vnops.c:864 Process 95676 (ufsbench) thread 0xfffff8084c2c0580 (102035) exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe00048490c0) locked @ kern/vfs_bio.c:1670 exclusive lockmgr ufs (ufs) r = 0 (0xfffff8067deffcc0) locked @ kern/vfs_vnops.c:864 Process 95673 (ufsbench) thread 0xfffff80842fae000 (100278) exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe0002354db8) locked @ kern/vfs_bio.c:1670 exclusive lockmgr ufs (ufs) r = 0 (0xfffff8047c0b72e0) locked @ kern/vfs_vnops.c:864 Process 95671 (ufsbench) thread 0xfffff8004cde0000 (100755) shared lockmgr ufs (ufs) r = 0 (0xfffff808837247d0) locked @ kern/vfs_subr.c:2590 Process 95670 (ufsbench) thread 0xfffff80860c3b000 (102398) exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe0002a5c740) locked @ kern/vfs_bio.c:1670 exclusive lockmgr ufs (ufs) r = 0 (0xfffff803c9dbc2e0) locked @ kern/vfs_vnops.c:864 Process 95669 (ufsbench) thread 0xfffff80010fd8000 (101119) exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe0002c0ca28) locked @ kern/vfs_bio.c:1670 exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe00035cbf30) locked @ kern/vfs_bio.c:3916 exclusive lockmgr ufs (ufs) r = 0 (0xfffff80881215cc0) locked @ kern/vfs_subr.c:2590 Process 95665 (ufsbench) thread 0xfffff80847bfb000 (100836) exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe0005cf37d0) locked @ kern/vfs_bio.c:1670 exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe0003e5f360) locked @ kern/vfs_bio.c:3916 exclusive lockmgr ufs (ufs) r = 0 (0xfffff8085820d2e0) locked @ kern/vfs_lookup.c:662 db:1:pho1> show lockedvnods Locked vnodes vnode 0xfffff80883724768: tag ufs, type VDIR usecount 4, writecount 0, refcount 7 mountedhere 0 flags (VV_ROOT|VI_ACTIVE) v_object 0xfffff80848f7c800 ref 0 pages 1 cleanbuf 1 dirtybuf 0 lock type ufs: SHARED (count 2) #0 0xffffffff80b64ce2 at lockmgr_slock_hard+0x332 #1 0xffffffff80b65659 at __lockmgr_args+0x3f9 #2 0xffffffff80eaae5c at ffs_lock+0x7c #3 0xffffffff811e88f6 at VOP_LOCK1_APV+0xa6 #4 0xffffffff80c74330 at _vn_lock+0x60 #5 0xffffffff80c5643a at lookup+0xda #6 0xffffffff80c55f2d at namei+0x4bd #7 0xffffffff80c6e3a7 at kern_statat+0x77 #8 0xffffffff80c6eb3f at sys_fstatat+0x2f #9 0xffffffff81068847 at amd64_syscall+0x2a7 #10 0xffffffff81044dcd at fast_syscall_common+0x101 ino 2, on dev md10a vnode 0xfffff8085820d278: tag ufs, type VDIR usecount 6, writecount 0, refcount 136 mountedhere 0 flags (VI_ACTIVE) v_object 0xfffff805b10c0200 ref 0 pages 127 cleanbuf 12 dirtybuf 116 lock type ufs: EXCL by thread 0xfffff80847bfb000 (pid 95665, ufsbench, tid 100836) with shared waiters pending #0 0xffffffff80b64ce2 at lockmgr_slock_hard+0x332 #1 0xffffffff80b65659 at __lockmgr_args+0x3f9 #2 0xffffffff80eaae5c at ffs_lock+0x7c #3 0xffffffff811e88f6 at VOP_LOCK1_APV+0xa6 #4 0xffffffff80c74330 at _vn_lock+0x60 #5 0xffffffff80c5643a at lookup+0xda #6 0xffffffff80c55f2d at namei+0x4bd #7 0xffffffff80c73a44 at vn_open_cred+0xd4 #8 0xffffffff80c6c47c at kern_openat+0x20c #9 0xffffffff81068847 at amd64_syscall+0x2a7 #10 0xffffffff81044dcd at fast_syscall_common+0x101 ino 521840, on dev md10a vnode 0xfffff80881215c58: tag ufs, type VDIR usecount 5, writecount 0, refcount 134 mountedhere 0 flags (VI_ACTIVE) v_object 0xfffff80857a29a00 ref 0 pages 126 cleanbuf 26 dirtybuf 101 lock type ufs: EXCL by thread 0xfffff80010fd8000 (pid 95669, ufsbench, tid 101119) with exclusive waiters pending with shared waiters pending #0 0xffffffff80b64ce2 at lockmgr_slock_hard+0x332 #1 0xffffffff80b65659 at __lockmgr_args+0x3f9 #2 0xffffffff80eaae5c at ffs_lock+0x7c #3 0xffffffff811e88f6 at VOP_LOCK1_APV+0xa6 #4 0xffffffff80c74330 at _vn_lock+0x60 #5 0xffffffff80c62806 at vget+0x96 #6 0xffffffff80c4861b at cache_lookup+0x6eb #7 0xffffffff80c4d28c at vfs_cache_lookup+0xac #8 0xffffffff811e51fc at VOP_LOOKUP_APV+0xbc #9 0xffffffff80c569d1 at lookup+0x671 #10 0xffffffff80c55f2d at namei+0x4bd #11 0xffffffff80c73a44 at vn_open_cred+0xd4 #12 0xffffffff80c6c47c at kern_openat+0x20c #13 0xffffffff81068847 at amd64_syscall+0x2a7 #14 0xffffffff81044dcd at fast_syscall_common+0x101 ino 66416, on dev md10a vnode 0xfffff8067deffc58: tag ufs, type VREG usecount 1, writecount 1, refcount 2 flags (VI_ACTIVE) v_object 0xfffff8035ddf8400 ref 0 pages 0 cleanbuf 1 dirtybuf 0 lock type ufs: EXCL by thread 0xfffff8084c2c0580 (pid 95676, ufsbench, tid 102035) #0 0xffffffff80b650ea at lockmgr_xlock_hard+0x3ba #1 0xffffffff80b6562e at __lockmgr_args+0x3ce #2 0xffffffff80eaae5c at ffs_lock+0x7c #3 0xffffffff811e88f6 at VOP_LOCK1_APV+0xa6 #4 0xffffffff80c74330 at _vn_lock+0x60 #5 0xffffffff80c76eb8 at vn_write+0x198 #6 0xffffffff80c76953 at vn_io_fault_doio+0x43 #7 0xffffffff80c749f1 at vn_io_fault1+0x171 #8 0xffffffff80c72dd5 at vn_io_fault+0x195 #9 0xffffffff80bfd2f7 at dofilewrite+0x97 #10 0xffffffff80bfcf1a at kern_writev+0x4a #11 0xffffffff80bfcec6 at sys_write+0x86 #12 0xffffffff81068847 at amd64_syscall+0x2a7 #13 0xffffffff81044dcd at fast_syscall_common+0x101 ino 966390, on dev md10a vnode 0xfffff8047c0b7278: tag ufs, type VREG usecount 1, writecount 1, refcount 2 flags (VI_ACTIVE) v_object 0xfffff8080bcb2c00 ref 0 pages 0 cleanbuf 1 dirtybuf 0 lock type ufs: EXCL by thread 0xfffff80842fae000 (pid 95673, ufsbench, tid 100278) #0 0xffffffff80b650ea at lockmgr_xlock_hard+0x3ba #1 0xffffffff80b6562e at __lockmgr_args+0x3ce #2 0xffffffff80eaae5c at ffs_lock+0x7c #3 0xffffffff811e88f6 at VOP_LOCK1_APV+0xa6 #4 0xffffffff80c74330 at _vn_lock+0x60 #5 0xffffffff80c76eb8 at vn_write+0x198 #6 0xffffffff80c76953 at vn_io_fault_doio+0x43 #7 0xffffffff80c749f1 at vn_io_fault1+0x171 #8 0xffffffff80c72dd5 at vn_io_fault+0x195 #9 0xffffffff80bfd2f7 at dofilewrite+0x97 #10 0xffffffff80bfcf1a at kern_writev+0x4a #11 0xffffffff80bfcec6 at sys_write+0x86 #12 0xffffffff81068847 at amd64_syscall+0x2a7 #13 0xffffffff81044dcd at fast_syscall_common+0x101 ino 616778, on dev md10a vnode 0xfffff803369d0000: tag ufs, type VREG usecount 1, writecount 1, refcount 2 flags (VI_ACTIVE) v_object 0xfffff8035ddf8800 ref 0 pages 0 cleanbuf 1 dirtybuf 0 lock type ufs: EXCL by thread 0xfffff8086e133580 (pid 95680, ufsbench, tid 102448) #0 0xffffffff80b650ea at lockmgr_xlock_hard+0x3ba #1 0xffffffff80b6562e at __lockmgr_args+0x3ce #2 0xffffffff80eaae5c at ffs_lock+0x7c #3 0xffffffff811e88f6 at VOP_LOCK1_APV+0xa6 #4 0xffffffff80c74330 at _vn_lock+0x60 #5 0xffffffff80c76eb8 at vn_write+0x198 #6 0xffffffff80c76953 at vn_io_fault_doio+0x43 #7 0xffffffff80c749f1 at vn_io_fault1+0x171 #8 0xffffffff80c72dd5 at vn_io_fault+0x195 #9 0xffffffff80bfd2f7 at dofilewrite+0x97 #10 0xffffffff80bfcf1a at kern_writev+0x4a #11 0xffffffff80bfcec6 at sys_write+0x86 #12 0xffffffff81068847 at amd64_syscall+0x2a7 #13 0xffffffff81044dcd at fast_syscall_common+0x101 ino 966391, on dev md10a vnode 0xfffff803369d0278: tag ufs, type VREG usecount 1, writecount 1, refcount 2 flags (VI_ACTIVE) v_object 0xfffff8035ddf8700 ref 0 pages 0 cleanbuf 1 dirtybuf 0 lock type ufs: EXCL by thread 0xfffff802ee2ac580 (pid 95679, ufsbench, tid 102016) #0 0xffffffff80b650ea at lockmgr_xlock_hard+0x3ba #1 0xffffffff80b6562e at __lockmgr_args+0x3ce #2 0xffffffff80eaae5c at ffs_lock+0x7c #3 0xffffffff811e88f6 at VOP_LOCK1_APV+0xa6 #4 0xffffffff80c74330 at _vn_lock+0x60 #5 0xffffffff80c76eb8 at vn_write+0x198 #6 0xffffffff80c76953 at vn_io_fault_doio+0x43 #7 0xffffffff80c749f1 at vn_io_fault1+0x171 #8 0xffffffff80c72dd5 at vn_io_fault+0x195 #9 0xffffffff80bfd2f7 at dofilewrite+0x97 #10 0xffffffff80bfcf1a at kern_writev+0x4a #11 0xffffffff80bfcec6 at sys_write+0x86 #12 0xffffffff81068847 at amd64_syscall+0x2a7 #13 0xffffffff81044dcd at fast_syscall_common+0x101 ino 616779, on dev md10a vnode 0xfffff803c9dbc278: tag ufs, type VREG usecount 1, writecount 1, refcount 2 flags (VI_ACTIVE) v_object 0xfffff8035ddf8d00 ref 0 pages 0 cleanbuf 1 dirtybuf 0 lock type ufs: EXCL by thread 0xfffff80860c3b000 (pid 95670, ufsbench, tid 102398) #0 0xffffffff80b650ea at lockmgr_xlock_hard+0x3ba #1 0xffffffff80b6562e at __lockmgr_args+0x3ce #2 0xffffffff80eaae5c at ffs_lock+0x7c #3 0xffffffff811e88f6 at VOP_LOCK1_APV+0xa6 #4 0xffffffff80c74330 at _vn_lock+0x60 #5 0xffffffff80c76eb8 at vn_write+0x198 #6 0xffffffff80c76953 at vn_io_fault_doio+0x43 #7 0xffffffff80c749f1 at vn_io_fault1+0x171 #8 0xffffffff80c72dd5 at vn_io_fault+0x195 #9 0xffffffff80bfd2f7 at dofilewrite+0x97 #10 0xffffffff80bfcf1a at kern_writev+0x4a #11 0xffffffff80bfcec6 at sys_write+0x86 #12 0xffffffff81068847 at amd64_syscall+0x2a7 #13 0xffffffff81044dcd at fast_syscall_common+0x101 ino 616780, on dev md10a vnode 0xfffff803c9dbc4f0: tag ufs, type VREG usecount 1, writecount 1, refcount 2 flags (VI_ACTIVE) v_object 0xfffff8035ddf8e00 ref 0 pages 0 cleanbuf 1 dirtybuf 0 lock type ufs: EXCL by thread 0xfffff803148a5000 (pid 95678, ufsbench, tid 102373) #0 0xffffffff80b650ea at lockmgr_xlock_hard+0x3ba #1 0xffffffff80b6562e at __lockmgr_args+0x3ce #2 0xffffffff80eaae5c at ffs_lock+0x7c #3 0xffffffff811e88f6 at VOP_LOCK1_APV+0xa6 #4 0xffffffff80c74330 at _vn_lock+0x60 #5 0xffffffff80c76eb8 at vn_write+0x198 #6 0xffffffff80c76953 at vn_io_fault_doio+0x43 #7 0xffffffff80c749f1 at vn_io_fault1+0x171 #8 0xffffffff80c72dd5 at vn_io_fault+0x195 #9 0xffffffff80bfd2f7 at dofilewrite+0x97 #10 0xffffffff80bfcf1a at kern_writev+0x4a #11 0xffffffff80bfcec6 at sys_write+0x86 #12 0xffffffff81068847 at amd64_syscall+0x2a7 #13 0xffffffff81044dcd at fast_syscall_common+0x101 ino 966392, on dev md10a db:1:pho1> show allchains chain 1: thread 100544 (pid 381, getty) sleeping on 0xfffff80007c794b0 "ttyin" chain 2: thread 100545 (pid 378, init) sleeping on 0xffffffff81e96242 "nanslp" chain 3: thread 100546 (pid 373, init) sleeping on 0xffffffff81e96230 "nanslp" chain 4: thread 100287 (pid 368, init) sleeping on 0xffffffff81e9623a "nanslp" chain 5: thread 100854 (pid 367, ls) blocked on lockmgr ufsEXCL thread 100836 (pid 95665, ufsbench) sleeping on 0xffffffff81a47ac0 "vmwait" chain 6: thread 100279 (pid 361, init) sleeping on 0xffffffff81e96246 "nanslp" chain 7: thread 100292 (pid 356, getty) sleeping on 0xfffff80840e420b0 "ttyin" chain 8: thread 102448 (pid 95680, ufsbench) sleeping on 0xffffffff81a47ac0 "vmwait" chain 9: thread 102016 (pid 95679, ufsbench) sleeping on 0xffffffff81a47ac0 "vmwait" chain 10: thread 102373 (pid 95678, ufsbench) sleeping on 0xffffffff81a47ac0 "vmwait" chain 11: thread 100539 (pid 95677, ufsbench) sleeping on 0xffffffff81a47ac0 "pfault" chain 12: thread 102035 (pid 95676, ufsbench) sleeping on 0xffffffff81a47ac0 "vmwait" chain 13: thread 100781 (pid 95675, ufsbench) sleeping on 0xffffffff81a47ac0 "pfault" chain 14: thread 102326 (pid 95674, ufsbench) blocked on lockmgr ufsEXCL thread 101119 (pid 95669, ufsbench) sleeping on 0xffffffff81a47ac0 "vmwait" chain 15: thread 100278 (pid 95673, ufsbench) sleeping on 0xffffffff81a47ac0 "vmwait" chain 16: thread 102405 (pid 95672, ufsbench) sleeping on 0xffffffff81a47ac0 "pfault" chain 17: thread 100755 (pid 95671, ufsbench) blocked on lockmgr ufsEXCL thread 101119 (pid 95669, ufsbench) sleeping on 0xffffffff81a47ac0 "vmwait" chain 18: thread 102398 (pid 95670, ufsbench) sleeping on 0xffffffff81a47ac0 "vmwait" chain 19: thread 101119 (pid 95669, ufsbench) sleeping on 0xffffffff81a47ac0 "vmwait" chain 20: thread 100619 (pid 95668, ufsbench) sleeping on 0xffffffff81a47ac0 "pfault" chain 21: thread 100888 (pid 95667, ufsbench) blocked on lockmgr ufsEXCL thread 100836 (pid 95665, ufsbench) sleeping on 0xffffffff81a47ac0 "vmwait" chain 22: thread 102086 (pid 95666, ufsbench) sleeping on 0xffffffff81a47ac0 "pfault" chain 23: thread 100836 (pid 95665, ufsbench) sleeping on 0xffffffff81a47ac0 "vmwait" chain 24: thread 100686 (pid 95664, ufsbench) sleeping on 0xffffffff81a47ac0 "pfault" chain 25: thread 100673 (pid 95663, ufsbench) blocked on lockmgr ufsEXCL thread 100836 (pid 95665, ufsbench) sleeping on 0xffffffff81a47ac0 "vmwait" chain 26: thread 100728 (pid 95651, md10) sleeping on 0xfffff8086b21c000 "mdwait" chain 27: thread 100543 (pid 965, getty) sleeping on 0xfffff80007c798b0 "ttyin" chain 28: thread 100542 (pid 964, getty) sleeping on 0xfffff80007c79cb0 "ttyin" chain 29: thread 100221 (pid 963, getty) sleeping on 0xfffff80007c7a0b0 "ttyin" chain 30: thread 100537 (pid 944, inetd) sleeping on 0xfffff80007b91c40 "select" chain 31: thread 100532 (pid 911, nfscbd: master) sleeping on 0xfffff80847757160 "rpcsvc" chain 32: thread 100533 (pid 911, nfscbd: service) sleeping on 0xfffff8084294ad60 "rpcsvc" chain 33: thread 100534 (pid 911, nfscbd: service) sleeping on 0xfffff8000b23c960 "rpcsvc" chain 34: thread 100535 (pid 911, nfscbd: service) sleeping on 0xfffff80840f30360 "rpcsvc" chain 35: thread 100509 (pid 910, nfscbd) sleeping on 0xfffff8000b53b838 "accept" chain 36: thread 100504 (pid 821, cron) sleeping on 0xffffffff81e96245 "nanslp" chain 37: thread 100507 (pid 811, sshd) sleeping on 0xfffff8000b9679c0 "select" chain 38: thread 100297 (pid 696, nfsuserd) sleeping on 0xfffff808427ce140 "select" chain 39: thread 100296 (pid 695, nfsuserd) sleeping on 0xfffff8000b15c6c0 "select" chain 40: thread 100283 (pid 693, nfsuserd) sleeping on 0xfffff8000b967a40 "select" chain 41: thread 100282 (pid 692, nfsuserd) sleeping on 0xfffff80007b9a3c0 "select" chain 42: thread 100249 (pid 691, nfsuserd) sleeping on 0xfffff808402c20a8 "pause" chain 43: thread 100295 (pid 672, casperd) sleeping on 0xfffff80840ef4f40 "select" chain 44: thread 100294 (pid 671, casperd) sleeping on 0xfffff8000b967ac0 "select" chain 45: thread 100285 (pid 651, watchdogd) sleeping on 0xffffffff81e96241 "nanslp" chain 46: thread 100261 (pid 530, rtsold) sleeping on 0xfffff80842838340 "select" chain 47: thread 100288 (pid 487, moused) sleeping on 0xfffff80842972440 "select" chain 48: thread 100209 (pid 45, vnlru) sleeping on 0xfffff808429bb528 "vlruwt" chain 49: thread 100208 (pid 44, syncer) sleeping on 0xffffffff82038680 "syncer" chain 50: thread 100207 (pid 43, bufdaemon) sleeping on 0xffffffff82037b20 "qsleep" chain 51: thread 100210 (pid 43, bufspacedaemon-0) sleeping on 0xffffffff81a0a580 "-" chain 52: thread 100212 (pid 43, bufspacedaemon-1) sleeping on 0xffffffff81a12840 "-" chain 53: thread 100214 (pid 43, bufspacedaemon-2) sleeping on 0xffffffff81a1ab00 "-" chain 54: thread 100216 (pid 43, bufspacedaemon-3) sleeping on 0xffffffff81a22dc0 "-" chain 55: thread 100218 (pid 43, bufspacedaemon-4) sleeping on 0xffffffff81a2b080 "-" chain 56: thread 100219 (pid 43, bufspacedaemon-5) sleeping on 0xffffffff81a33340 "-" chain 57: thread 100220 (pid 43, bufspacedaemon-6) sleeping on 0xffffffff81a3b600 "-" chain 58: thread 100233 (pid 43, / worker) sleeping on 0xfffff80007d5e8e8 "sdflush" chain 59: thread 100241 (pid 43, /tmp worker) sleeping on 0xfffff80842de80e8 "sdflush" chain 60: thread 100243 (pid 43, /media worker) sleeping on 0xfffff80007d5ece8 "sdflush" chain 61: thread 100206 (pid 42, vmdaemon) sleeping on 0xffffffff820549c8 "psleep" chain 62: thread 100205 (pid 41, dom0) sleeping on 0xffffffff820b23d8 "psleep" chain 63: thread 100211 (pid 41, laundry: dom0) sleeping on 0xffffffff820b23e0 "launds" chain 64: thread 100213 (pid 41, dom1) sleeping on 0xffffffff81e94fd7 "pwait" chain 65: thread 100215 (pid 41, laundry: dom1) sleeping on 0xffffffff81e94fcb "laundp" chain 66: thread 100217 (pid 41, uma) sleeping on 0xffffffff80ec8de0 "umarcl" chain 67: thread 100204 (pid 40, enc_daemon0) sleeping on 0xfffff808426fe528 "idle" chain 68: thread 100201 (pid 39, rand_harvestq) sleeping on 0xffffffff81cf5a08 "-" chain 69: thread 100197 (pid 38, sctp_iterator) sleeping on 0xffffffff820b0ac0 "waiting_for_work" chain 70: thread 100196 (pid 37, soaiod4) sleeping on 0xffffffff8203751c "-" chain 71: thread 100195 (pid 36, soaiod3) sleeping on 0xffffffff8203751c "-" chain 72: thread 100194 (pid 35, soaiod2) sleeping on 0xffffffff8203751c "-" chain 73: thread 100193 (pid 34, soaiod1) sleeping on 0xffffffff8203751c "-" chain 74: thread 100171 (pid 33, usbus0) sleeping on 0xfffffe0000678d10 "-" chain 75: thread 100172 (pid 33, usbus0) sleeping on 0xfffffe0000678d68 "-" chain 76: thread 100173 (pid 33, usbus0) sleeping on 0xfffffe0000678dc0 "-" chain 77: thread 100174 (pid 33, usbus0) sleeping on 0xfffffe0000678e18 "-" chain 78: thread 100175 (pid 33, usbus0) sleeping on 0xfffffe0000678e70 "-" chain 79: thread 100178 (pid 33, usbus1) sleeping on 0xfffffe009e487460 "-" chain 80: thread 100179 (pid 33, usbus1) sleeping on 0xfffffe009e4874b8 "-" chain 81: thread 100180 (pid 33, usbus1) sleeping on 0xfffffe009e487510 "-" chain 82: thread 100181 (pid 33, usbus1) sleeping on 0xfffffe009e487568 "-" chain 83: thread 100182 (pid 33, usbus1) sleeping on 0xfffffe009e4875c0 "-" chain 84: thread 100184 (pid 33, usbus2) sleeping on 0xfffffe00006e2d10 "-" chain 85: thread 100185 (pid 33, usbus2) sleeping on 0xfffffe00006e2d68 "-" chain 86: thread 100186 (pid 33, usbus2) sleeping on 0xfffffe00006e2dc0 "-" chain 87: thread 100187 (pid 33, usbus2) sleeping on 0xfffffe00006e2e18 "-" chain 88: thread 100188 (pid 33, usbus2) sleeping on 0xfffffe00006e2e70 "-" chain 89: thread 100163 (pid 32, doneq0) sleeping on 0xffffffff81bd0cc0 "-" chain 90: thread 100164 (pid 32, doneq1) sleeping on 0xffffffff81bd0d40 "-" chain 91: thread 100165 (pid 32, doneq2) sleeping on 0xffffffff81bd0dc0 "-" chain 92: thread 100166 (pid 32, doneq3) sleeping on 0xffffffff81bd0e40 "-" chain 93: thread 100167 (pid 32, doneq4) sleeping on 0xffffffff81bd0ec0 "-" chain 94: thread 100203 (pid 32, scanner) sleeping on 0xffffffff81bd0b88 "-" chain 95: thread 100162 (pid 31, sequencer 00) sleeping on 0xfffff8084023fc88 "seqstate" chain 96: thread 100161 (pid 30, crypto returns 23) sleeping on 0xfffff800038348d0 "crypto_ret_wait" chain 97: thread 100160 (pid 29, crypto returns 22) sleeping on 0xfffff80003834870 "crypto_ret_wait" chain 98: thread 100159 (pid 28, crypto returns 21) sleeping on 0xfffff80003834810 "crypto_ret_wait" chain 99: thread 100158 (pid 27, crypto returns 20) sleeping on 0xfffff800038347b0 "crypto_ret_wait" chain 100: thread 100157 (pid 26, crypto returns 19) sleeping on 0xfffff80003834750 "crypto_ret_wait" chain 101: thread 100156 (pid 25, crypto returns 18) sleeping on 0xfffff800038346f0 "crypto_ret_wait" chain 102: thread 100155 (pid 24, crypto returns 17) sleeping on 0xfffff80003834690 "crypto_ret_wait" chain 103: thread 100154 (pid 23, crypto returns 16) sleeping on 0xfffff80003834630 "crypto_ret_wait" chain 104: thread 100153 (pid 22, crypto returns 15) sleeping on 0xfffff800038345d0 "crypto_ret_wait" chain 105: thread 100152 (pid 21, crypto returns 14) sleeping on 0xfffff80003834570 "crypto_ret_wait" chain 106: thread 100151 (pid 20, crypto returns 13) sleeping on 0xfffff80003834510 "crypto_ret_wait" chain 107: thread 100150 (pid 19, crypto returns 12) sleeping on 0xfffff800038344b0 "crypto_ret_wait" chain 108: thread 100149 (pid 18, crypto returns 11) sleeping on 0xfffff80003834450 "crypto_ret_wait" chain 109: thread 100148 (pid 17, crypto returns 10) sleeping on 0xfffff800038343f0 "crypto_ret_wait" chain 110: thread 100147 (pid 16, crypto returns 9) sleeping on 0xfffff80003834390 "crypto_ret_wait" chain 111: thread 100146 (pid 15, crypto returns 8) sleeping on 0xfffff80003834330 "crypto_ret_wait" chain 112: thread 100145 (pid 14, crypto returns 7) sleeping on 0xfffff800038342d0 "crypto_ret_wait" chain 113: thread 100144 (pid 9, crypto returns 6) sleeping on 0xfffff80003834270 "crypto_ret_wait" chain 114: thread 100143 (pid 8, crypto returns 5) sleeping on 0xfffff80003834210 "crypto_ret_wait" chain 115: thread 100142 (pid 7, crypto returns 4) sleeping on 0xfffff800038341b0 "crypto_ret_wait" chain 116: thread 100141 (pid 6, crypto returns 3) sleeping on 0xfffff80003834150 "crypto_ret_wait" chain 117: thread 100140 (pid 5, crypto returns 2) sleeping on 0xfffff800038340f0 "crypto_ret_wait" chain 118: thread 100139 (pid 4, crypto returns 1) sleeping on 0xfffff80003834090 "crypto_ret_wait" chain 119: thread 100138 (pid 3, crypto returns 0) sleeping on 0xfffff80003834030 "crypto_ret_wait" chain 120: thread 100137 (pid 2, crypto) sleeping on 0xffffffff8204efa0 "crypto_wait" chain 121: thread 100109 (pid 13, g_event) sleeping on 0xffffffff8208d2d0 "-" chain 122: thread 100110 (pid 13, g_up) sleeping on 0xffffffff8208d2e0 "-" chain 123: thread 100111 (pid 13, g_down) sleeping on 0xffffffff8208d2c8 "-" chain 124: thread 100002 (pid 1, init) sleeping on 0xfffff808400f3528 "wait" chain 125: thread 100001 (pid 10, audit) sleeping on 0xffffffff820b17e0 "audit_worker_cv" chain 126: thread 100000 (pid 0, swapper) sleeping on 0xffffffff8208d310 "swapin" chain 127: thread 100029 (pid 0, thread taskq) sleeping on 0xfffff8084011b100 "-" chain 128: thread 100031 (pid 0, config_0) sleeping on 0xfffff80003156b00 "-" chain 129: thread 100032 (pid 0, kqueue_ctx taskq) sleeping on 0xfffff800036b3d00 "-" chain 130: thread 100033 (pid 0, aiod_kick taskq) sleeping on 0xfffff800036b3c00 "-" chain 131: thread 100034 (pid 0, if_config_tqg_0) sleeping on 0xfffff80003156a00 "-" chain 132: thread 100035 (pid 0, softirq_0) sleeping on 0xfffff80003156900 "-" chain 133: thread 100036 (pid 0, softirq_1) sleeping on 0xfffff80003156800 "-" chain 134: thread 100037 (pid 0, softirq_2) sleeping on 0xfffff80003156700 "-" chain 135: thread 100038 (pid 0, softirq_3) sleeping on 0xfffff80003156600 "-" chain 136: thread 100039 (pid 0, softirq_4) sleeping on 0xfffff80003156500 "-" chain 137: thread 100040 (pid 0, softirq_5) sleeping on 0xfffff80003156400 "-" chain 138: thread 100041 (pid 0, softirq_6) sleeping on 0xfffff80003156300 "-" chain 139: thread 100042 (pid 0, softirq_7) sleeping on 0xfffff80003156200 "-" chain 140: thread 100043 (pid 0, softirq_8) sleeping on 0xfffff80003156100 "-" chain 141: thread 100044 (pid 0, softirq_9) sleeping on 0xfffff80003156000 "-" chain 142: thread 100045 (pid 0, softirq_10) sleeping on 0xfffff8084008ee00 "-" chain 143: thread 100046 (pid 0, softirq_11) sleeping on 0xfffff8084008ed00 "-" chain 144: thread 100047 (pid 0, softirq_12) sleeping on 0xfffff8084008ec00 "-" chain 145: thread 100048 (pid 0, softirq_13) sleeping on 0xfffff8084008eb00 "-" chain 146: thread 100049 (pid 0, softirq_14) sleeping on 0xfffff8084008ea00 "-" chain 147: thread 100050 (pid 0, softirq_15) sleeping on 0xfffff8084008e900 "-" chain 148: thread 100051 (pid 0, softirq_16) sleeping on 0xfffff8084008e800 "-" chain 149: thread 100052 (pid 0, softirq_17) sleeping on 0xfffff8084008e700 "-" chain 150: thread 100053 (pid 0, softirq_18) sleeping on 0xfffff8084008e600 "-" chain 151: thread 100054 (pid 0, softirq_19) sleeping on 0xfffff8084008e500 "-" chain 152: thread 100055 (pid 0, softirq_20) sleeping on 0xfffff8084008e400 "-" chain 153: thread 100056 (pid 0, softirq_21) sleeping on 0xfffff8084008e300 "-" chain 154: thread 100057 (pid 0, softirq_22) sleeping on 0xfffff8084008e200 "-" chain 155: thread 100058 (pid 0, softirq_23) sleeping on 0xfffff8084008e100 "-" chain 156: thread 100059 (pid 0, if_io_tqg_0) sleeping on 0xfffff8084008e000 "-" chain 157: thread 100060 (pid 0, if_io_tqg_1) sleeping on 0xfffff80003155e00 "-" chain 158: thread 100061 (pid 0, if_io_tqg_2) sleeping on 0xfffff80003155d00 "-" chain 159: thread 100062 (pid 0, if_io_tqg_3) sleeping on 0xfffff80003155c00 "-" chain 160: thread 100063 (pid 0, if_io_tqg_4) sleeping on 0xfffff80003155b00 "-" chain 161: thread 100064 (pid 0, if_io_tqg_5) sleeping on 0xfffff80003155a00 "-" chain 162: thread 100065 (pid 0, if_io_tqg_6) sleeping on 0xfffff80003155900 "-" chain 163: thread 100066 (pid 0, if_io_tqg_7) sleeping on 0xfffff80003155800 "-" chain 164: thread 100067 (pid 0, if_io_tqg_8) sleeping on 0xfffff80003155700 "-" chain 165: thread 100068 (pid 0, if_io_tqg_9) sleeping on 0xfffff80003155600 "-" chain 166: thread 100069 (pid 0, if_io_tqg_10) sleeping on 0xfffff80003155500 "-" chain 167: thread 100070 (pid 0, if_io_tqg_11) sleeping on 0xfffff80003155400 "-" chain 168: thread 100071 (pid 0, if_io_tqg_12) sleeping on 0xfffff80003155300 "-" chain 169: thread 100072 (pid 0, if_io_tqg_13) sleeping on 0xfffff80003155200 "-" chain 170: thread 100073 (pid 0, if_io_tqg_14) sleeping on 0xfffff80003155100 "-" chain 171: thread 100074 (pid 0, if_io_tqg_15) sleeping on 0xfffff80003155000 "-" chain 172: thread 100075 (pid 0, if_io_tqg_16) sleeping on 0xfffff8084008de00 "-" chain 173: thread 100076 (pid 0, if_io_tqg_17) sleeping on 0xfffff8084008dd00 "-" chain 174: thread 100077 (pid 0, if_io_tqg_18) sleeping on 0xfffff8084008dc00 "-" chain 175: thread 100078 (pid 0, if_io_tqg_19) sleeping on 0xfffff8084008db00 "-" chain 176: thread 100079 (pid 0, if_io_tqg_20) sleeping on 0xfffff8084008da00 "-" chain 177: thread 100080 (pid 0, if_io_tqg_21) sleeping on 0xfffff8084008d900 "-" chain 178: thread 100081 (pid 0, if_io_tqg_22) sleeping on 0xfffff8084008d800 "-" chain 179: thread 100082 (pid 0, if_io_tqg_23) sleeping on 0xfffff8084008d700 "-" chain 180: thread 100112 (pid 0, firmware taskq) sleeping on 0xfffff8084011a000 "-" chain 181: thread 100113 (pid 0, crypto_0) sleeping on 0xfffff808400d2e00 "-" chain 182: thread 100114 (pid 0, crypto_1) sleeping on 0xfffff808400d2e00 "-" chain 183: thread 100115 (pid 0, crypto_2) sleeping on 0xfffff808400d2e00 "-" chain 184: thread 100116 (pid 0, crypto_3) sleeping on 0xfffff808400d2e00 "-" chain 185: thread 100117 (pid 0, crypto_4) sleeping on 0xfffff808400d2e00 "-" chain 186: thread 100118 (pid 0, crypto_5) sleeping on 0xfffff808400d2e00 "-" chain 187: thread 100119 (pid 0, crypto_6) sleeping on 0xfffff808400d2e00 "-" chain 188: thread 100120 (pid 0, crypto_7) sleeping on 0xfffff808400d2e00 "-" chain 189: thread 100121 (pid 0, crypto_8) sleeping on 0xfffff808400d2e00 "-" chain 190: thread 100122 (pid 0, crypto_9) sleeping on 0xfffff808400d2e00 "-" chain 191: thread 100123 (pid 0, crypto_10) sleeping on 0xfffff808400d2e00 "-" chain 192: thread 100124 (pid 0, crypto_11) sleeping on 0xfffff808400d2e00 "-" chain 193: thread 100125 (pid 0, crypto_12) sleeping on 0xfffff808400d2e00 "-" chain 194: thread 100126 (pid 0, crypto_13) sleeping on 0xfffff808400d2e00 "-" chain 195: thread 100127 (pid 0, crypto_14) sleeping on 0xfffff808400d2e00 "-" chain 196: thread 100128 (pid 0, crypto_15) sleeping on 0xfffff808400d2e00 "-" chain 197: thread 100129 (pid 0, crypto_16) sleeping on 0xfffff808400d2e00 "-" chain 198: thread 100130 (pid 0, crypto_17) sleeping on 0xfffff808400d2e00 "-" chain 199: thread 100131 (pid 0, crypto_18) sleeping on 0xfffff808400d2e00 "-" chain 200: thread 100132 (pid 0, crypto_19) sleeping on 0xfffff808400d2e00 "-" chain 201: thread 100133 (pid 0, crypto_20) sleeping on 0xfffff808400d2e00 "-" chain 202: thread 100134 (pid 0, crypto_21) sleeping on 0xfffff808400d2e00 "-" chain 203: thread 100135 (pid 0, crypto_22) sleeping on 0xfffff808400d2e00 "-" chain 204: thread 100136 (pid 0, crypto_23) sleeping on 0xfffff808400d2e00 "-" chain 205: thread 100192 (pid 0, mca taskq) sleeping on 0xfffff80840e59d00 "-" chain 206: thread 100198 (pid 0, acpi_task_0) sleeping on 0xfffff808427c4400 "-" chain 207: thread 100199 (pid 0, acpi_task_1) sleeping on 0xfffff808427c4400 "-" chain 208: thread 100200 (pid 0, acpi_task_2) sleeping on 0xfffff808427c4400 "-" chain 209: thread 100202 (pid 0, CAM taskq) sleeping on 0xfffff808400d2c00 "-" chain 210: thread 100242 (pid 0, /media trim) sleeping on 0xfffff8000b17a900 "-" db:1:pho1> show mount 0xfffff80007d80000 /dev/da0p2 on / (ufs) 0xfffff808429b7000 devfs on /dev (devfs) 0xfffff80842dbb000 /dev/da0p3 on /tmp (ufs) 0xfffff8000b158000 procfs on /proc (procfs) 0xfffff80842dba000 /dev/ada0p2 on /media (ufs) 0xfffff8003c22b000 /dev/md10a on /mnt (ufs) More info: show mount db:1:pho1> show bufqueues bqempty: 25818 Buf domain 0 freebufs 9731 lofreebufs 438 hifreebufs 658 bufspace 14103552 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 5263 (13630976) dirtyq count 0 (0) wakeup 0 lim 12 CPU 9, 4, 8, 1, 5, 10, 5, 10, 11, 5, 4, 10, 4, 2, 10, 10, 0, 2, 2, 2, 4, 5, 7, 8, Locked buffers: 1 space 0 Total buffers: 13211 space 14099456 Buf domain 1 freebufs 4837 lofreebufs 438 hifreebufs 658 bufspace 174832128 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 10182 (174544896) dirtyq count 0 (0) wakeup 0 lim 12 CPU 3, 2, 2, 8, 3, 1, 3, 7, 8, 0, 5, 0, 3, 3, 3, 8, 7, 8, 2, 10, 7, 9, 5, 7, Locked buffers: 0 space 0 Total buffers: 18041 space 174832128 Buf domain 2 freebufs 9554 lofreebufs 438 hifreebufs 658 bufspace 15800832 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 5463 (15026688) dirtyq count 0 (0) wakeup 0 lim 12 CPU 6, 2, 9, 3, 4, 0, 1, 5, 4, 2, 4, 7, 10, 8, 9, 11, 1, 1, 2, 4, 8, 6, 5, 4, Locked buffers: 0 space 0 Total buffers: 13318 space 15800832 Buf domain 3 freebufs 7331 lofreebufs 438 hifreebufs 658 bufspace 94197248 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 4 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 7666 (93611008) dirtyq count 4 (131072) wakeup 0 lim 12 CPU 11, 10, 0, 5, 2, 10, 3, 1, 3, 4, 8, 2, 9, 7, 1, 2, 3, 3, 10, 9, 4, 7, 11, 5, Locked buffers: 2 space 0 Total buffers: 15618 space 94189056 Buf domain 4 freebufs 9305 lofreebufs 438 hifreebufs 658 bufspace 27773952 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 5713 (27023360) dirtyq count 0 (0) wakeup 0 lim 12 CPU 5, 6, 1, 11, 9, 11, 9, 0, 7, 1, 0, 3, 9, 5, 3, 0, 0, 2, 8, 10, 2, 3, 0, 9, Locked buffers: 1 space 0 Total buffers: 14101 space 27769856 Buf domain 5 freebufs 9317 lofreebufs 438 hifreebufs 658 bufspace 24768000 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 118 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 5559 (23690240) dirtyq count 118 (540672) wakeup 0 lim 12 CPU 2, 2, 8, 7, 9, 11, 8, 4, 8, 11, 2, 0, 6, 5, 6, 1, 11, 9, 0, 6, 6, 7, 8, 1, Locked buffers: 1 space 0 Total buffers: 13398 space 24763904 Buf domain 6 freebufs 4661 lofreebufs 438 hifreebufs 658 bufspace 34255360 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 103 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 10270 (33279488) dirtyq count 103 (421888) wakeup 0 lim 12 CPU 11, 2, 9, 4, 1, 1, 1, 10, 1, 5, 8, 1, 2, 3, 6, 0, 8, 2, 2, 7, 3, 5, 3, 1, Locked buffers: 5 space 8192 Total buffers: 18244 space 34243072 db:1:pho1> show page vm_cnt.v_free_count: 8068013 vm_cnt.v_inactive_count: 741 vm_cnt.v_active_count: 24053 vm_cnt.v_laundry_count: 274 vm_cnt.v_wire_count: 558977 vm_cnt.v_free_reserved: 11400 vm_cnt.v_free_min: 54670 vm_cnt.v_free_target: 184480 vm_cnt.v_inactive_target: 276720 db:1:pho1> show pageq pq_free 8068013 dom 0 page_cnt 8359062 free 8066694 pq_act 24053 pq_inact 740 pq_laund 274 pq_unsw 0 dom 1 page_cnt 295454 free 1319 pq_act 0 pq_inact 1 pq_laund 0 pq_unsw 0 db:0:pho> show freepages DOMAIN: 0 FREE LIST 0: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000892 | 000010 11 (008192K) | 000673 | 000025 10 (004096K) | 000868 | 000046 09 (002048K) | 001168 | 000052 08 (001024K) | 000942 | 000011 07 (000512K) | 002205 | 000015 06 (000256K) | 003670 | 000033 05 (000128K) | 005134 | 000119 04 (000064K) | 008235 | 000329 03 (000032K) | 014379 | 000371 02 (000016K) | 020198 | 000424 01 (000008K) | 032316 | 001787 00 (000004K) | 001714 | 007955 FREE LIST 1: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000000 | 000000 10 (004096K) | 000000 | 000000 09 (002048K) | 000000 | 000000 08 (001024K) | 000000 | 000000 07 (000512K) | 000000 | 000000 06 (000256K) | 000001 | 000000 05 (000128K) | 000001 | 000000 04 (000064K) | 000001 | 000000 03 (000032K) | 000001 | 000000 02 (000016K) | 000000 | 000000 01 (000008K) | 000000 | 000000 00 (000004K) | 000001 | 000000 DOMAIN: 1 FREE LIST 0: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000000 | 000000 10 (004096K) | 000000 | 000000 09 (002048K) | 000000 | 000000 08 (001024K) | 000000 | 000000 07 (000512K) | 000000 | 000000 06 (000256K) | 000002 | 000000 05 (000128K) | 000001 | 000000 04 (000064K) | 000006 | 000000 03 (000032K) | 000005 | 000004 02 (000016K) | 000013 | 000010 01 (000008K) | 000026 | 000046 00 (000004K) | 000128 | 000627 FREE LIST 1: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000000 | 000000 10 (004096K) | 000000 | 000000 09 (002048K) | 000000 | 000000 08 (001024K) | 000000 | 000000 07 (000512K) | 000000 | 000000 06 (000256K) | 000000 | 000000 05 (000128K) | 000000 | 000000 04 (000064K) | 000000 | 000000 03 (000032K) | 000000 | 000000 02 (000016K) | 000000 | 000000 01 (000008K) | 000000 | 000000 00 (000004K) | 000000 | 000000 db:0:pho> acttrace Tracing command idle pid 11 tid 100003 td 0xfffff80003697000 (CPU 0) cpustop_handler() at cpustop_handler+0x28/frame 0xffffffff82058d30 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xffffffff82058d50 trap() at trap+0x49/frame 0xffffffff82058e60 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff82058e60 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe007875a900, rbp = 0xfffffe007875a910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe007875a910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe007875a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007875a980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe007875a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007875aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007875aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007875aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xfffff808400f5580 (CPU 1) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006bacdf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006bace10 trap() at trap+0x49/frame 0xfffffe0006bacf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bacf20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe007875f900, rbp = 0xfffffe007875f910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe007875f910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe007875f960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007875f980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe007875f9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007875fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007875fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007875fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffff808400f5000 (CPU 2) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006bb8df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006bb8e10 trap() at trap+0x49/frame 0xfffffe0006bb8f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bb8f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe0078764900, rbp = 0xfffffe0078764910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe0078764910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe0078764960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078764980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787649a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078764a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078764ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078764ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xfffff80003696580 (CPU 3) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006bc4df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006bc4e10 trap() at trap+0x49/frame 0xfffffe0006bc4f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bc4f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe0078769900, rbp = 0xfffffe0078769910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe0078769910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe0078769960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078769980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787699a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078769a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078769ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078769ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100007 td 0xfffff80003696000 (CPU 4) kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007876e730 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007876e740 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007876e770 uart_intr() at uart_intr+0x138/frame 0xfffffe007876e7b0 intr_event_handle() at intr_event_handle+0xcb/frame 0xfffffe007876e800 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007876e830 lapic_handle_intr() at lapic_handle_intr+0x5f/frame 0xfffffe007876e850 Xapic_isr1() at Xapic_isr1+0xd9/frame 0xfffffe007876e850 --- interrupt, rip = 0xffffffff8046209c, rsp = 0xfffffe007876e920, rbp = 0xfffffe007876e960 --- acpi_cpu_idle() at acpi_cpu_idle+0x34c/frame 0xfffffe007876e960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007876e980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe007876e9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007876ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe007876eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007876eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xfffff808400f4580 (CPU 5) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006bdcdf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006bdce10 trap() at trap+0x49/frame 0xfffffe0006bdcf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bdcf20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe0078773900, rbp = 0xfffffe0078773910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe0078773910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe0078773960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078773980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787739a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078773a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078773ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078773ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xfffff808400f4000 (CPU 6) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006be8df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006be8e10 trap() at trap+0x49/frame 0xfffffe0006be8f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006be8f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe0078778900, rbp = 0xfffffe0078778910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe0078778910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe0078778960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078778980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787789a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078778a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078778ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078778ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xfffff800036a1580 (CPU 7) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006bf4df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006bf4e10 trap() at trap+0x49/frame 0xfffffe0006bf4f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bf4f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe007877d900, rbp = 0xfffffe007877d910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe007877d910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe007877d960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007877d980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe007877d9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007877da70 fork_exit() at fork_exit+0x84/frame 0xfffffe007877dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007877dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffff800036a1000 (CPU 8) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0000269df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0000269e10 trap() at trap+0x49/frame 0xfffffe0000269f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0000269f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe0078782900, rbp = 0xfffffe0078782910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe0078782910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe0078782960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078782980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787829a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078782a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078782ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078782ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100012 td 0xfffff808400ff580 (CPU 9) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0000275df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0000275e10 trap() at trap+0x49/frame 0xfffffe0000275f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0000275f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe0078787900, rbp = 0xfffffe0078787910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe0078787910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe0078787960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078787980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787879a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078787a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078787ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078787ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xfffff808400ff000 (CPU 10) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0000281df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0000281e10 trap() at trap+0x49/frame 0xfffffe0000281f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0000281f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe007878c900, rbp = 0xfffffe007878c910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe007878c910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe007878c960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007878c980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe007878c9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007878ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe007878cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007878cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100014 td 0xfffff800036a0580 (CPU 11) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe000028ddf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe000028de10 trap() at trap+0x49/frame 0xfffffe000028df20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe000028df20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe0078791900, rbp = 0xfffffe0078791910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe0078791910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe0078791960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078791980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787919a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078791a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078791ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078791ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100015 td 0xfffff800036a0000 (CPU 12) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0000299df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0000299e10 trap() at trap+0x49/frame 0xfffffe0000299f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0000299f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe0078796900, rbp = 0xfffffe0078796910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe0078796910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe0078796960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078796980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787969a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078796a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078796ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078796ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100016 td 0xfffff808400fe580 (CPU 13) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00002a5df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00002a5e10 trap() at trap+0x49/frame 0xfffffe00002a5f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00002a5f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe007879b900, rbp = 0xfffffe007879b910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe007879b910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe007879b960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007879b980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe007879b9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007879ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe007879bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007879bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100017 td 0xfffff808400fe000 (CPU 14) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00002b1df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00002b1e10 trap() at trap+0x49/frame 0xfffffe00002b1f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00002b1f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe00787a0900, rbp = 0xfffffe00787a0910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe00787a0910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe00787a0960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe00787a0980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787a09a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe00787a0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787a0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787a0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100018 td 0xfffff808400f6580 (CPU 15) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00002bddf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00002bde10 trap() at trap+0x49/frame 0xfffffe00002bdf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00002bdf20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe00787a5900, rbp = 0xfffffe00787a5910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe00787a5910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe00787a5960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe00787a5980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787a59a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe00787a5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787a5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787a5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100019 td 0xfffff80840102000 (CPU 16) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00002c9df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00002c9e10 trap() at trap+0x49/frame 0xfffffe00002c9f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00002c9f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe00787aa900, rbp = 0xfffffe00787aa910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe00787aa910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe00787aa960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe00787aa980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787aa9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe00787aaa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787aaab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787aaab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100020 td 0xfffff800036a3580 (CPU 17) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00002d5df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00002d5e10 trap() at trap+0x49/frame 0xfffffe00002d5f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00002d5f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe00787af900, rbp = 0xfffffe00787af910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe00787af910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe00787af960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe00787af980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787af9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe00787afa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787afab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787afab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100021 td 0xfffff800036a3000 (CPU 18) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00002e1df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00002e1e10 trap() at trap+0x49/frame 0xfffffe00002e1f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00002e1f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe00787b4900, rbp = 0xfffffe00787b4910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe00787b4910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe00787b4960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe00787b4980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787b49a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe00787b4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787b4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787b4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100022 td 0xfffff80840101580 (CPU 19) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00002eddf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00002ede10 trap() at trap+0x49/frame 0xfffffe00002edf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00002edf20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe00787b9900, rbp = 0xfffffe00787b9910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe00787b9910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe00787b9960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe00787b9980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787b99a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe00787b9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787b9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787b9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100023 td 0xfffff80840101000 (CPU 20) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00002f9df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00002f9e10 trap() at trap+0x49/frame 0xfffffe00002f9f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00002f9f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe00787be900, rbp = 0xfffffe00787be910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe00787be910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe00787be960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe00787be980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787be9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe00787bea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787beab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787beab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100024 td 0xfffff800036a2580 (CPU 21) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0000305df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0000305e10 trap() at trap+0x49/frame 0xfffffe0000305f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0000305f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe00787c3900, rbp = 0xfffffe00787c3910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe00787c3910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe00787c3960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe00787c3980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787c39a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe00787c3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787c3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787c3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100025 td 0xfffff800036a2000 (CPU 22) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0000311df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0000311e10 trap() at trap+0x49/frame 0xfffffe0000311f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0000311f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe00787c8900, rbp = 0xfffffe00787c8910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe00787c8910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe00787c8960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe00787c8980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787c89a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe00787c8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787c8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787c8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100026 td 0xfffff80840100580 (CPU 23) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe000031ddf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe000031de10 trap() at trap+0x49/frame 0xfffffe000031df20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe000031df20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe00787cd900, rbp = 0xfffffe00787cd910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe00787cd910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe00787cd960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe00787cd980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787cd9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe00787cda70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787cdab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787cdab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db:0:pho> ps pid ppid pgrp uid state wmesg wchan cmd 381 1 381 0 Ss+ ttyin 0xfffff80007c794b0 getty 378 1 1 0 SW nanslp 0xffffffff81e96242 init 373 1 1 0 SW nanslp 0xffffffff81e96230 init 368 1 1 0 SW nanslp 0xffffffff81e9623a init 367 1 651 0 D ufs 0xfffff8085820d2e0 ls 361 1 1 0 SW nanslp 0xffffffff81e96246 init 356 1 356 0 Ss+ ttyin 0xfffff80840e420b0 getty 95680 1 1002 0 D+ vmwait 0xffffffff81a47ac0 ufsbench 95679 1 1002 0 D+ vmwait 0xffffffff81a47ac0 ufsbench 95678 1 1002 0 D+ vmwait 0xffffffff81a47ac0 ufsbench 95677 1 1002 0 DW+ pfault 0xffffffff81a47ac0 ufsbench 95676 1 1002 0 D+ vmwait 0xffffffff81a47ac0 ufsbench 95675 1 1002 0 DW+ pfault 0xffffffff81a47ac0 ufsbench 95674 1 1002 0 D+ ufs 0xfffff80881215cc0 ufsbench 95673 1 1002 0 D+ vmwait 0xffffffff81a47ac0 ufsbench 95672 1 1002 0 DW+ pfault 0xffffffff81a47ac0 ufsbench 95671 1 1002 0 D+ ufs 0xfffff80881215cc0 ufsbench 95670 1 1002 0 D+ vmwait 0xffffffff81a47ac0 ufsbench 95669 1 1002 0 D+ vmwait 0xffffffff81a47ac0 ufsbench 95668 1 1002 0 DW+ pfault 0xffffffff81a47ac0 ufsbench 95667 1 1002 0 D+ ufs 0xfffff8085820d2e0 ufsbench 95666 1 1002 0 DW+ pfault 0xffffffff81a47ac0 ufsbench 95665 1 1002 0 D+ vmwait 0xffffffff81a47ac0 ufsbench 95664 1 1002 0 DW+ pfault 0xffffffff81a47ac0 ufsbench 95663 1 1002 0 D+ ufs 0xfffff8085820d2e0 ufsbench 95651 0 0 0 DL mdwait 0xfffff8086b21c000 [md10] 965 1 965 0 Ss+ ttyin 0xfffff80007c798b0 getty 964 1 964 0 Ss+ ttyin 0xfffff80007c79cb0 getty 963 1 963 0 Ss+ ttyin 0xfffff80007c7a0b0 getty 944 1 944 0 Ss select 0xfffff80007b91c40 inetd 911 910 910 0 S (threaded) nfscbd 100532 S rpcsvc 0xfffff80847757160 nfscbd: master 100533 S rpcsvc 0xfffff8084294ad60 nfscbd: service 100534 S rpcsvc 0xfffff8000b23c960 nfscbd: service 100535 S rpcsvc 0xfffff80840f30360 nfscbd: service 910 1 910 0 SWs accept 0xfffff8000b53b838 nfscbd 821 1 821 0 SWs nanslp 0xffffffff81e96245 cron 811 1 811 0 Ss select 0xfffff8000b9679c0 sshd 696 691 691 0 S select 0xfffff808427ce140 nfsuserd 695 691 691 0 S select 0xfffff8000b15c6c0 nfsuserd 693 691 691 0 S select 0xfffff8000b967a40 nfsuserd 692 691 691 0 S select 0xfffff80007b9a3c0 nfsuserd 691 1 691 0 SWs pause 0xfffff808402c20a8 nfsuserd 672 1 672 0 Ss select 0xfffff80840ef4f40 casperd 671 1 671 0 Ss select 0xfffff8000b967ac0 casperd 651 1 651 0 Ss nanslp 0xffffffff81e96241 watchdogd 530 1 530 0 Ss select 0xfffff80842838340 rtsold 487 1 487 0 Ss select 0xfffff80842972440 moused 45 0 0 0 DL vlruwt 0xfffff808429bb528 [vnlru] 44 0 0 0 DL syncer 0xffffffff82038680 [syncer] 43 0 0 0 DL (threaded) [bufdaemon] 100207 D qsleep 0xffffffff82037b20 [bufdaemon] 100210 D - 0xffffffff81a0a580 [bufspacedaemon-0] 100212 D - 0xffffffff81a12840 [bufspacedaemon-1] 100214 D - 0xffffffff81a1ab00 [bufspacedaemon-2] 100216 D - 0xffffffff81a22dc0 [bufspacedaemon-3] 100218 D - 0xffffffff81a2b080 [bufspacedaemon-4] 100219 D - 0xffffffff81a33340 [bufspacedaemon-5] 100220 D - 0xffffffff81a3b600 [bufspacedaemon-6] 100233 D sdflush 0xfffff80007d5e8e8 [/ worker] 100241 D sdflush 0xfffff80842de80e8 [/tmp worker] 100243 D sdflush 0xfffff80007d5ece8 [/media worker] 42 0 0 0 DL psleep 0xffffffff820549c8 [vmdaemon] 41 0 0 0 DL (threaded) [pagedaemon] 100205 D psleep 0xffffffff820b23d8 [dom0] 100211 D launds 0xffffffff820b23e0 [laundry: dom0] 100213 D pwait 0xffffffff81e94fd7 [dom1] 100215 D laundp 0xffffffff81e94fcb [laundry: dom1] 100217 D umarcl 0xffffffff80ec8de0 [uma] 40 0 0 0 DL idle 0xfffff808426fe528 [enc_daemon0] 39 0 0 0 DL - 0xffffffff81cf5a08 [rand_harvestq] 38 0 0 0 DL waiting 0xffffffff820b0ac0 [sctp_iterator] 37 0 0 0 DL - 0xffffffff8203751c [soaiod4] 36 0 0 0 DL - 0xffffffff8203751c [soaiod3] 35 0 0 0 DL - 0xffffffff8203751c [soaiod2] 34 0 0 0 DL - 0xffffffff8203751c [soaiod1] 33 0 0 0 DL (threaded) [usb] 100171 D - 0xfffffe0000678d10 [usbus0] 100172 D - 0xfffffe0000678d68 [usbus0] 100173 D - 0xfffffe0000678dc0 [usbus0] 100174 D - 0xfffffe0000678e18 [usbus0] 100175 D - 0xfffffe0000678e70 [usbus0] 100178 D - 0xfffffe009e487460 [usbus1] 100179 D - 0xfffffe009e4874b8 [usbus1] 100180 D - 0xfffffe009e487510 [usbus1] 100181 D - 0xfffffe009e487568 [usbus1] 100182 D - 0xfffffe009e4875c0 [usbus1] 100184 D - 0xfffffe00006e2d10 [usbus2] 100185 D - 0xfffffe00006e2d68 [usbus2] 100186 D - 0xfffffe00006e2dc0 [usbus2] 100187 D - 0xfffffe00006e2e18 [usbus2] 100188 D - 0xfffffe00006e2e70 [usbus2] 32 0 0 0 DL (threaded) [cam] 100163 D - 0xffffffff81bd0cc0 [doneq0] 100164 D - 0xffffffff81bd0d40 [doneq1] 100165 D - 0xffffffff81bd0dc0 [doneq2] 100166 D - 0xffffffff81bd0e40 [doneq3] 100167 D - 0xffffffff81bd0ec0 [doneq4] 100203 D - 0xffffffff81bd0b88 [scanner] 31 0 0 0 DL seqstat 0xfffff8084023fc88 [sequencer 00] 30 0 0 0 DL crypto_ 0xfffff800038348d0 [crypto returns 23] 29 0 0 0 DL crypto_ 0xfffff80003834870 [crypto returns 22] 28 0 0 0 DL crypto_ 0xfffff80003834810 [crypto returns 21] 27 0 0 0 DL crypto_ 0xfffff800038347b0 [crypto returns 20] 26 0 0 0 DL crypto_ 0xfffff80003834750 [crypto returns 19] 25 0 0 0 DL crypto_ 0xfffff800038346f0 [crypto returns 18] 24 0 0 0 DL crypto_ 0xfffff80003834690 [crypto returns 17] 23 0 0 0 DL crypto_ 0xfffff80003834630 [crypto returns 16] 22 0 0 0 DL crypto_ 0xfffff800038345d0 [crypto returns 15] 21 0 0 0 DL crypto_ 0xfffff80003834570 [crypto returns 14] 20 0 0 0 DL crypto_ 0xfffff80003834510 [crypto returns 13] 19 0 0 0 DL crypto_ 0xfffff800038344b0 [crypto returns 12] 18 0 0 0 DL crypto_ 0xfffff80003834450 [crypto returns 11] 17 0 0 0 DL crypto_ 0xfffff800038343f0 [crypto returns 10] 16 0 0 0 DL crypto_ 0xfffff80003834390 [crypto returns 9] 15 0 0 0 DL crypto_ 0xfffff80003834330 [crypto returns 8] 14 0 0 0 DL crypto_ 0xfffff800038342d0 [crypto returns 7] 9 0 0 0 DL crypto_ 0xfffff80003834270 [crypto returns 6] 8 0 0 0 DL crypto_ 0xfffff80003834210 [crypto returns 5] 7 0 0 0 DL crypto_ 0xfffff800038341b0 [crypto returns 4] 6 0 0 0 DL crypto_ 0xfffff80003834150 [crypto returns 3] 5 0 0 0 DL crypto_ 0xfffff800038340f0 [crypto returns 2] 4 0 0 0 DL crypto_ 0xfffff80003834090 [crypto returns 1] 3 0 0 0 DL crypto_ 0xfffff80003834030 [crypto returns 0] 2 0 0 0 DL crypto_ 0xffffffff8204efa0 [crypto] 13 0 0 0 DL (threaded) [geom] 100109 D - 0xffffffff8208d2d0 [g_event] 100110 D - 0xffffffff8208d2e0 [g_up] 100111 D - 0xffffffff8208d2c8 [g_down] 12 0 0 0 WL (threaded) [intr] 100027 I [swi6: task queue] 100028 I [swi6: Giant taskq] 100030 I [swi5: fast taskq] 100083 I [swi3: vm] 100084 I [swi1: netisr 0] 100085 I [swi4: clock (0)] 100086 I [swi4: clock (1)] 100087 I [swi4: clock (2)] 100088 I [swi4: clock (3)] 100089 I [swi4: clock (4)] 100090 I [swi4: clock (5)] 100091 I [swi4: clock (6)] 100092 I [swi4: clock (7)] 100093 I [swi4: clock (8)] 100094 I [swi4: clock (9)] 100095 I [swi4: clock (10)] 100096 I [swi4: clock (11)] 100097 I [swi4: clock (12)] 100098 I [swi4: clock (13)] 100099 I [swi4: clock (14)] 100100 I [swi4: clock (15)] 100101 I [swi4: clock (16)] 100102 I [swi4: clock (17)] 100103 I [swi4: clock (18)] 100104 I [swi4: clock (19)] 100105 I [swi4: clock (20)] 100106 I [swi4: clock (21)] 100107 I [swi4: clock (22)] 100108 I [swi4: clock (23)] 100168 I [irq264: isci0] 100169 I [irq265: isci0] 100170 I [irq22: ehci0] 100176 I [irq266: hdac0] 100177 I [irq281: xhci0] 100183 I [irq20: ehci1] 100189 I [irq282: ahci0] 100190 I [irq283: hdac1] 100191 I [swi0: uart] 11 0 0 0 RL (threaded) [idle] 100003 Run CPU 0 [idle: cpu0] 100004 Run CPU 1 [idle: cpu1] 100005 Run CPU 2 [idle: cpu2] 100006 Run CPU 3 [idle: cpu3] 100007 Run CPU 4 [idle: cpu4] 100008 Run CPU 5 [idle: cpu5] 100009 Run CPU 6 [idle: cpu6] 100010 Run CPU 7 [idle: cpu7] 100011 Run CPU 8 [idle: cpu8] 100012 Run CPU 9 [idle: cpu9] 100013 Run CPU 10 [idle: cpu10] 100014 Run CPU 11 [idle: cpu11] 100015 Run CPU 12 [idle: cpu12] 100016 Run CPU 13 [idle: cpu13] 100017 Run CPU 14 [idle: cpu14] 100018 Run CPU 15 [idle: cpu15] 100019 Run CPU 16 [idle: cpu16] 100020 Run CPU 17 [idle: cpu17] 100021 Run CPU 18 [idle: cpu18] 100022 Run CPU 19 [idle: cpu19] 100023 Run CPU 20 [idle: cpu20] 100024 Run CPU 21 [idle: cpu21] 100025 Run CPU 22 [idle: cpu22] 100026 Run CPU 23 [idle: cpu23] 1 0 1 0 SLs wait 0xfffff808400f3528 [init] 10 0 0 0 DL audit_w 0xffffffff820b17e0 [audit] 0 0 0 0 DLs (threaded) [kernel] 100000 D swapin 0xffffffff8208d310 [swapper] 100029 D - 0xfffff8084011b100 [thread taskq] 100031 D - 0xfffff80003156b00 [config_0] 100032 D - 0xfffff800036b3d00 [kqueue_ctx taskq] 100033 D - 0xfffff800036b3c00 [aiod_kick taskq] 100034 D - 0xfffff80003156a00 [if_config_tqg_0] 100035 D - 0xfffff80003156900 [softirq_0] 100036 D - 0xfffff80003156800 [softirq_1] 100037 D - 0xfffff80003156700 [softirq_2] 100038 D - 0xfffff80003156600 [softirq_3] 100039 D - 0xfffff80003156500 [softirq_4] 100040 D - 0xfffff80003156400 [softirq_5] 100041 D - 0xfffff80003156300 [softirq_6] 100042 D - 0xfffff80003156200 [softirq_7] 100043 D - 0xfffff80003156100 [softirq_8] 100044 D - 0xfffff80003156000 [softirq_9] 100045 D - 0xfffff8084008ee00 [softirq_10] 100046 D - 0xfffff8084008ed00 [softirq_11] 100047 D - 0xfffff8084008ec00 [softirq_12] 100048 D - 0xfffff8084008eb00 [softirq_13] 100049 D - 0xfffff8084008ea00 [softirq_14] 100050 D - 0xfffff8084008e900 [softirq_15] 100051 D - 0xfffff8084008e800 [softirq_16] 100052 D - 0xfffff8084008e700 [softirq_17] 100053 D - 0xfffff8084008e600 [softirq_18] 100054 D - 0xfffff8084008e500 [softirq_19] 100055 D - 0xfffff8084008e400 [softirq_20] 100056 D - 0xfffff8084008e300 [softirq_21] 100057 D - 0xfffff8084008e200 [softirq_22] 100058 D - 0xfffff8084008e100 [softirq_23] 100059 D - 0xfffff8084008e000 [if_io_tqg_0] 100060 D - 0xfffff80003155e00 [if_io_tqg_1] 100061 D - 0xfffff80003155d00 [if_io_tqg_2] 100062 D - 0xfffff80003155c00 [if_io_tqg_3] 100063 D - 0xfffff80003155b00 [if_io_tqg_4] 100064 D - 0xfffff80003155a00 [if_io_tqg_5] 100065 D - 0xfffff80003155900 [if_io_tqg_6] 100066 D - 0xfffff80003155800 [if_io_tqg_7] 100067 D - 0xfffff80003155700 [if_io_tqg_8] 100068 D - 0xfffff80003155600 [if_io_tqg_9] 100069 D - 0xfffff80003155500 [if_io_tqg_10] 100070 D - 0xfffff80003155400 [if_io_tqg_11] 100071 D - 0xfffff80003155300 [if_io_tqg_12] 100072 D - 0xfffff80003155200 [if_io_tqg_13] 100073 D - 0xfffff80003155100 [if_io_tqg_14] 100074 D - 0xfffff80003155000 [if_io_tqg_15] 100075 D - 0xfffff8084008de00 [if_io_tqg_16] 100076 D - 0xfffff8084008dd00 [if_io_tqg_17] 100077 D - 0xfffff8084008dc00 [if_io_tqg_18] 100078 D - 0xfffff8084008db00 [if_io_tqg_19] 100079 D - 0xfffff8084008da00 [if_io_tqg_20] 100080 D - 0xfffff8084008d900 [if_io_tqg_21] 100081 D - 0xfffff8084008d800 [if_io_tqg_22] 100082 D - 0xfffff8084008d700 [if_io_tqg_23] 100112 D - 0xfffff8084011a000 [firmware taskq] 100113 D - 0xfffff808400d2e00 [crypto_0] 100114 D - 0xfffff808400d2e00 [crypto_1] 100115 D - 0xfffff808400d2e00 [crypto_2] 100116 D - 0xfffff808400d2e00 [crypto_3] 100117 D - 0xfffff808400d2e00 [crypto_4] 100118 D - 0xfffff808400d2e00 [crypto_5] 100119 D - 0xfffff808400d2e00 [crypto_6] 100120 D - 0xfffff808400d2e00 [crypto_7] 100121 D - 0xfffff808400d2e00 [crypto_8] 100122 D - 0xfffff808400d2e00 [crypto_9] 100123 D - 0xfffff808400d2e00 [crypto_10] 100124 D - 0xfffff808400d2e00 [crypto_11] 100125 D - 0xfffff808400d2e00 [crypto_12] 100126 D - 0xfffff808400d2e00 [crypto_13] 100127 D - 0xfffff808400d2e00 [crypto_14] 100128 D - 0xfffff808400d2e00 [crypto_15] 100129 D - 0xfffff808400d2e00 [crypto_16] 100130 D - 0xfffff808400d2e00 [crypto_17] 100131 D - 0xfffff808400d2e00 [crypto_18] 100132 D - 0xfffff808400d2e00 [crypto_19] 100133 D - 0xfffff808400d2e00 [crypto_20] 100134 D - 0xfffff808400d2e00 [crypto_21] 100135 D - 0xfffff808400d2e00 [crypto_22] 100136 D - 0xfffff808400d2e00 [crypto_23] 100192 D - 0xfffff80840e59d00 [mca taskq] 100198 D - 0xfffff808427c4400 [acpi_task_0] 100199 D - 0xfffff808427c4400 [acpi_task_1] 100200 D - 0xfffff808427c4400 [acpi_task_2] 100202 D - 0xfffff808400d2c00 [CAM taskq] 100242 D - 0xfffff8000b17a900 [/media trim] db:0:pho> allt Tracing command getty pid 381 tid 100544 td 0xfffff8000bf19580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a64f9630 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a64f9660 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a64f96a0 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00a64f9700 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00a64f9730 _cv_wait_sig() at _cv_wait_sig+0x1ce/frame 0xfffffe00a64f9790 tty_wait() at tty_wait+0x42/frame 0xfffffe00a64f97c0 ttydisc_read() at ttydisc_read+0x212/frame 0xfffffe00a64f9820 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00a64f9850 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00a64f98b0 dofileread() at dofileread+0x95/frame 0xfffffe00a64f98f0 kern_readv() at kern_readv+0x4a/frame 0xfffffe00a64f9930 sys_read() at sys_read+0x86/frame 0xfffffe00a64f9980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe00a64f9ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a64f9ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80042f64a, rsp = 0x7fffffffebe8, rbp = 0x7fffffffec30 --- Tracing command init pid 378 tid 100545 td 0xfffff8000bf19000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe007876df40 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe007876df50 trap() at trap+0x5c/frame 0xfffffe007876e060 calltrap() at calltrap+0x8/frame 0xfffffe007876e060 --- trap 0xc, rip = 0xffffffff810426c6, rsp = 0xfffffe007876e130, rbp = 0xfffffe007876e140 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007876e140 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007876e1e0 db_command() at db_command+0x279/frame 0xfffffe007876e2b0 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007876e350 db_command() at db_command+0x279/frame 0xfffffe007876e420 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007876e430 db_trap() at db_trap+0xef/frame 0xfffffe007876e4c0 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007876e530 trap() at trap+0x806/frame 0xfffffe007876e640 calltrap() at calltrap+0x8/frame 0xfffffe007876e640 --- trap 0x3, rip = 0xffffffff80bd8196, rsp = 0xfffffe007876e710, rbp = 0xfffffe007876e730 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007876e730 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007876e740 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007876e770 uart_intr() at uart_intr+0x138/frame 0xfffffe007876e7b0 intr_event_handle() at intr_event_handle+0xcb/frame 0xfffffe007876e800 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007876e830 lapic_handle_intr() at lapic_handle_intr+0x5f/frame 0xfffffe007876e850 Xapic_isr1() at Xapic_isr1+0xd9/frame 0xfffffe007876e850 --- interrupt, rip = 0xffffffff8046209c, rsp = 0xfffffe007876e920, rbp = 0xfffffe007876e960 --- acpi_cpu_idle() at acpi_cpu_idle+0x34c/frame 0xfffffe007876e960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007876e980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe007876e9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007876ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe007876eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007876eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command init pid 373 tid 100546 td 0xfffff80860fc0580 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe007876df40 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe007876df50 trap() at trap+0x5c/frame 0xfffffe007876e060 calltrap() at calltrap+0x8/frame 0xfffffe007876e060 --- trap 0xc, rip = 0xffffffff810426c6, rsp = 0xfffffe007876e130, rbp = 0xfffffe007876e140 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007876e140 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007876e1e0 db_command() at db_command+0x279/frame 0xfffffe007876e2b0 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007876e350 db_command() at db_command+0x279/frame 0xfffffe007876e420 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007876e430 db_trap() at db_trap+0xef/frame 0xfffffe007876e4c0 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007876e530 trap() at trap+0x806/frame 0xfffffe007876e640 calltrap() at calltrap+0x8/frame 0xfffffe007876e640 --- trap 0x3, rip = 0xffffffff80bd8196, rsp = 0xfffffe007876e710, rbp = 0xfffffe007876e730 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007876e730 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007876e740 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007876e770 uart_intr() at uart_intr+0x138/frame 0xfffffe007876e7b0 intr_event_handle() at intr_event_handle+0xcb/frame 0xfffffe007876e800 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007876e830 lapic_handle_intr() at lapic_handle_intr+0x5f/frame 0xfffffe007876e850 Xapic_isr1() at Xapic_isr1+0xd9/frame 0xfffffe007876e850 --- interrupt, rip = 0xffffffff8046209c, rsp = 0xfffffe007876e920, rbp = 0xfffffe007876e960 --- acpi_cpu_idle() at acpi_cpu_idle+0x34c/frame 0xfffffe007876e960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007876e980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe007876e9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007876ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe007876eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007876eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command init pid 368 tid 100287 td 0xfffff8000b3a6580 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe007876df40 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe007876df50 trap() at trap+0x5c/frame 0xfffffe007876e060 calltrap() at calltrap+0x8/frame 0xfffffe007876e060 --- trap 0xc, rip = 0xffffffff810426c6, rsp = 0xfffffe007876e130, rbp = 0xfffffe007876e140 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007876e140 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007876e1e0 db_command() at db_command+0x279/frame 0xfffffe007876e2b0 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007876e350 db_command() at db_command+0x279/frame 0xfffffe007876e420 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007876e430 db_trap() at db_trap+0xef/frame 0xfffffe007876e4c0 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007876e530 trap() at trap+0x806/frame 0xfffffe007876e640 calltrap() at calltrap+0x8/frame 0xfffffe007876e640 --- trap 0x3, rip = 0xffffffff80bd8196, rsp = 0xfffffe007876e710, rbp = 0xfffffe007876e730 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007876e730 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007876e740 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007876e770 uart_intr() at uart_intr+0x138/frame 0xfffffe007876e7b0 intr_event_handle() at intr_event_handle+0xcb/frame 0xfffffe007876e800 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007876e830 lapic_handle_intr() at lapic_handle_intr+0x5f/frame 0xfffffe007876e850 Xapic_isr1() at Xapic_isr1+0xd9/frame 0xfffffe007876e850 --- interrupt, rip = 0xffffffff8046209c, rsp = 0xfffffe007876e920, rbp = 0xfffffe007876e960 --- acpi_cpu_idle() at acpi_cpu_idle+0x34c/frame 0xfffffe007876e960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007876e980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe007876e9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007876ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe007876eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007876eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command ls pid 367 tid 100854 td 0xfffff8000be40000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a846d070 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a846d0a0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a846d0e0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a846d110 sleeplk() at sleeplk+0x149/frame 0xfffffe00a846d170 lockmgr_slock_hard() at lockmgr_slock_hard+0x2db/frame 0xfffffe00a846d200 __lockmgr_args() at __lockmgr_args+0x3f9/frame 0xfffffe00a846d2a0 ffs_lock() at ffs_lock+0x7c/frame 0xfffffe00a846d2f0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xa6/frame 0xfffffe00a846d320 _vn_lock() at _vn_lock+0x60/frame 0xfffffe00a846d390 vget() at vget+0x96/frame 0xfffffe00a846d3d0 cache_lookup() at cache_lookup+0x6eb/frame 0xfffffe00a846d4a0 vfs_cache_lookup() at vfs_cache_lookup+0xac/frame 0xfffffe00a846d500 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xbc/frame 0xfffffe00a846d530 lookup() at lookup+0x671/frame 0xfffffe00a846d5d0 namei() at namei+0x4bd/frame 0xfffffe00a846d690 kern_statat() at kern_statat+0x77/frame 0xfffffe00a846d880 sys_fstatat() at sys_fstatat+0x2f/frame 0xfffffe00a846d980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe00a846dab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a846dab0 --- syscall (552, FreeBSD ELF64, sys_fstatat), rip = 0x80048eb8a, rsp = 0x7fffffff8d48, rbp = 0x7fffffff8e00 --- Tracing command init pid 361 tid 100279 td 0xfffff8000b2f3000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe007876df40 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe007876df50 trap() at trap+0x5c/frame 0xfffffe007876e060 calltrap() at calltrap+0x8/frame 0xfffffe007876e060 --- trap 0xc, rip = 0xffffffff810426c6, rsp = 0xfffffe007876e130, rbp = 0xfffffe007876e140 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007876e140 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007876e1e0 db_command() at db_command+0x279/frame 0xfffffe007876e2b0 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007876e350 db_command() at db_command+0x279/frame 0xfffffe007876e420 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007876e430 db_trap() at db_trap+0xef/frame 0xfffffe007876e4c0 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007876e530 trap() at trap+0x806/frame 0xfffffe007876e640 calltrap() at calltrap+0x8/frame 0xfffffe007876e640 --- trap 0x3, rip = 0xffffffff80bd8196, rsp = 0xfffffe007876e710, rbp = 0xfffffe007876e730 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007876e730 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007876e740 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007876e770 uart_intr() at uart_intr+0x138/frame 0xfffffe007876e7b0 intr_event_handle() at intr_event_handle+0xcb/frame 0xfffffe007876e800 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007876e830 lapic_handle_intr() at lapic_handle_intr+0x5f/frame 0xfffffe007876e850 Xapic_isr1() at Xapic_isr1+0xd9/frame 0xfffffe007876e850 --- interrupt, rip = 0xffffffff8046209c, rsp = 0xfffffe007876e920, rbp = 0xfffffe007876e960 --- acpi_cpu_idle() at acpi_cpu_idle+0x34c/frame 0xfffffe007876e960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007876e980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe007876e9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007876ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe007876eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007876eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command getty pid 356 tid 100292 td 0xfffff80842f5a580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a6009630 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a6009660 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a60096a0 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00a6009700 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00a6009730 _cv_wait_sig() at _cv_wait_sig+0x1ce/frame 0xfffffe00a6009790 tty_wait() at tty_wait+0x42/frame 0xfffffe00a60097c0 ttydisc_read() at ttydisc_read+0x212/frame 0xfffffe00a6009820 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00a6009850 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00a60098b0 dofileread() at dofileread+0x95/frame 0xfffffe00a60098f0 kern_readv() at kern_readv+0x4a/frame 0xfffffe00a6009930 sys_read() at sys_read+0x86/frame 0xfffffe00a6009980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe00a6009ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a6009ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80042f64a, rsp = 0x7fffffffebe8, rbp = 0x7fffffffec30 --- Tracing command ufsbench pid 95680 tid 102448 td 0xfffff8086e133580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe028dc4ce90 mi_switch() at mi_switch+0x16f/frame 0xfffffe028dc4cec0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe028dc4cf00 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe028dc4cf30 _sleep() at _sleep+0x358/frame 0xfffffe028dc4cfd0 vm_wait_doms() at vm_wait_doms+0xd2/frame 0xfffffe028dc4cff0 vm_wait_domain() at vm_wait_domain+0x68/frame 0xfffffe028dc4d020 vm_domain_alloc_fail() at vm_domain_alloc_fail+0x73/frame 0xfffffe028dc4d050 vm_page_alloc_domain_after() at vm_page_alloc_domain_after+0x274/frame 0xfffffe028dc4d0c0 vm_page_grab_pages() at vm_page_grab_pages+0x274/frame 0xfffffe028dc4d160 allocbuf() at allocbuf+0x340/frame 0xfffffe028dc4d1d0 getblkx() at getblkx+0x715/frame 0xfffffe028dc4d290 getblk() at getblk+0x15/frame 0xfffffe028dc4d2b0 ffs_balloc_ufs2() at ffs_balloc_ufs2+0xdbd/frame 0xfffffe028dc4d450 ffs_write() at ffs_write+0x21e/frame 0xfffffe028dc4d4f0 VOP_WRITE_APV() at VOP_WRITE_APV+0x13f/frame 0xfffffe028dc4d600 vn_write() at vn_write+0x273/frame 0xfffffe028dc4d690 vn_io_fault_doio() at vn_io_fault_doio+0x43/frame 0xfffffe028dc4d6f0 vn_io_fault1() at vn_io_fault1+0x171/frame 0xfffffe028dc4d830 vn_io_fault() at vn_io_fault+0x195/frame 0xfffffe028dc4d8a0 dofilewrite() at dofilewrite+0x97/frame 0xfffffe028dc4d8f0 kern_writev() at kern_writev+0x4a/frame 0xfffffe028dc4d930 sys_write() at sys_write+0x86/frame 0xfffffe028dc4d980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe028dc4dab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe028dc4dab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x80041560a, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe490 --- Tracing command ufsbench pid 95679 tid 102016 td 0xfffff802ee2ac580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00b1975e90 mi_switch() at mi_switch+0x16f/frame 0xfffffe00b1975ec0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b1975f00 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b1975f30 _sleep() at _sleep+0x358/frame 0xfffffe00b1975fd0 vm_wait_doms() at vm_wait_doms+0xd2/frame 0xfffffe00b1975ff0 vm_wait_domain() at vm_wait_domain+0x68/frame 0xfffffe00b1976020 vm_domain_alloc_fail() at vm_domain_alloc_fail+0x73/frame 0xfffffe00b1976050 vm_page_alloc_domain_after() at vm_page_alloc_domain_after+0x274/frame 0xfffffe00b19760c0 vm_page_grab_pages() at vm_page_grab_pages+0x274/frame 0xfffffe00b1976160 allocbuf() at allocbuf+0x340/frame 0xfffffe00b19761d0 getblkx() at getblkx+0x715/frame 0xfffffe00b1976290 getblk() at getblk+0x15/frame 0xfffffe00b19762b0 ffs_balloc_ufs2() at ffs_balloc_ufs2+0xdbd/frame 0xfffffe00b1976450 ffs_write() at ffs_write+0x21e/frame 0xfffffe00b19764f0 VOP_WRITE_APV() at VOP_WRITE_APV+0x13f/frame 0xfffffe00b1976600 vn_write() at vn_write+0x273/frame 0xfffffe00b1976690 vn_io_fault_doio() at vn_io_fault_doio+0x43/frame 0xfffffe00b19766f0 vn_io_fault1() at vn_io_fault1+0x171/frame 0xfffffe00b1976830 vn_io_fault() at vn_io_fault+0x195/frame 0xfffffe00b19768a0 dofilewrite() at dofilewrite+0x97/frame 0xfffffe00b19768f0 kern_writev() at kern_writev+0x4a/frame 0xfffffe00b1976930 sys_write() at sys_write+0x86/frame 0xfffffe00b1976980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe00b1976ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b1976ab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x80041560a, rsp = 0x7fffffffe3a8, rbp = 0x7fffffffe490 --- Tracing command ufsbench pid 95678 tid 102373 td 0xfffff803148a5000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe028dad5e90 mi_switch() at mi_switch+0x16f/frame 0xfffffe028dad5ec0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe028dad5f00 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe028dad5f30 _sleep() at _sleep+0x358/frame 0xfffffe028dad5fd0 vm_wait_doms() at vm_wait_doms+0xd2/frame 0xfffffe028dad5ff0 vm_wait_domain() at vm_wait_domain+0x68/frame 0xfffffe028dad6020 vm_domain_alloc_fail() at vm_domain_alloc_fail+0x73/frame 0xfffffe028dad6050 vm_page_alloc_domain_after() at vm_page_alloc_domain_after+0x274/frame 0xfffffe028dad60c0 vm_page_grab_pages() at vm_page_grab_pages+0x274/frame 0xfffffe028dad6160 allocbuf() at allocbuf+0x340/frame 0xfffffe028dad61d0 getblkx() at getblkx+0x715/frame 0xfffffe028dad6290 getblk() at getblk+0x15/frame 0xfffffe028dad62b0 ffs_balloc_ufs2() at ffs_balloc_ufs2+0xdbd/frame 0xfffffe028dad6450 ffs_write() at ffs_write+0x21e/frame 0xfffffe028dad64f0 VOP_WRITE_APV() at VOP_WRITE_APV+0x13f/frame 0xfffffe028dad6600 vn_write() at vn_write+0x273/frame 0xfffffe028dad6690 vn_io_fault_doio() at vn_io_fault_doio+0x43/frame 0xfffffe028dad66f0 vn_io_fault1() at vn_io_fault1+0x171/frame 0xfffffe028dad6830 vn_io_fault() at vn_io_fault+0x195/frame 0xfffffe028dad68a0 dofilewrite() at dofilewrite+0x97/frame 0xfffffe028dad68f0 kern_writev() at kern_writev+0x4a/frame 0xfffffe028dad6930 sys_write() at sys_write+0x86/frame 0xfffffe028dad6980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe028dad6ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe028dad6ab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x80041560a, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe490 --- Tracing command ufsbench pid 95677 tid 100539 td 0xfffff80049f89580 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe007876df40 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe007876df50 trap() at trap+0x5c/frame 0xfffffe007876e060 calltrap() at calltrap+0x8/frame 0xfffffe007876e060 --- trap 0xc, rip = 0xffffffff810426c6, rsp = 0xfffffe007876e130, rbp = 0xfffffe007876e140 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007876e140 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007876e1e0 db_command() at db_command+0x279/frame 0xfffffe007876e2b0 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007876e350 db_command() at db_command+0x279/frame 0xfffffe007876e420 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007876e430 db_trap() at db_trap+0xef/frame 0xfffffe007876e4c0 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007876e530 trap() at trap+0x806/frame 0xfffffe007876e640 calltrap() at calltrap+0x8/frame 0xfffffe007876e640 --- trap 0x3, rip = 0xffffffff80bd8196, rsp = 0xfffffe007876e710, rbp = 0xfffffe007876e730 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007876e730 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007876e740 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007876e770 uart_intr() at uart_intr+0x138/frame 0xfffffe007876e7b0 intr_event_handle() at intr_event_handle+0xcb/frame 0xfffffe007876e800 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007876e830 lapic_handle_intr() at lapic_handle_intr+0x5f/frame 0xfffffe007876e850 Xapic_isr1() at Xapic_isr1+0xd9/frame 0xfffffe007876e850 --- interrupt, rip = 0xffffffff8046209c, rsp = 0xfffffe007876e920, rbp = 0xfffffe007876e960 --- acpi_cpu_idle() at acpi_cpu_idle+0x34c/frame 0xfffffe007876e960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007876e980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe007876e9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007876ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe007876eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007876eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command ufsbench pid 95676 tid 102035 td 0xfffff8084c2c0580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00b1a34e90 mi_switch() at mi_switch+0x16f/frame 0xfffffe00b1a34ec0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b1a34f00 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b1a34f30 _sleep() at _sleep+0x358/frame 0xfffffe00b1a34fd0 vm_wait_doms() at vm_wait_doms+0xd2/frame 0xfffffe00b1a34ff0 vm_wait_domain() at vm_wait_domain+0x68/frame 0xfffffe00b1a35020 vm_domain_alloc_fail() at vm_domain_alloc_fail+0x73/frame 0xfffffe00b1a35050 vm_page_alloc_domain_after() at vm_page_alloc_domain_after+0x274/frame 0xfffffe00b1a350c0 vm_page_grab_pages() at vm_page_grab_pages+0x274/frame 0xfffffe00b1a35160 allocbuf() at allocbuf+0x340/frame 0xfffffe00b1a351d0 getblkx() at getblkx+0x715/frame 0xfffffe00b1a35290 getblk() at getblk+0x15/frame 0xfffffe00b1a352b0 ffs_balloc_ufs2() at ffs_balloc_ufs2+0xdbd/frame 0xfffffe00b1a35450 ffs_write() at ffs_write+0x21e/frame 0xfffffe00b1a354f0 VOP_WRITE_APV() at VOP_WRITE_APV+0x13f/frame 0xfffffe00b1a35600 vn_write() at vn_write+0x273/frame 0xfffffe00b1a35690 vn_io_fault_doio() at vn_io_fault_doio+0x43/frame 0xfffffe00b1a356f0 vn_io_fault1() at vn_io_fault1+0x171/frame 0xfffffe00b1a35830 vn_io_fault() at vn_io_fault+0x195/frame 0xfffffe00b1a358a0 dofilewrite() at dofilewrite+0x97/frame 0xfffffe00b1a358f0 kern_writev() at kern_writev+0x4a/frame 0xfffffe00b1a35930 sys_write() at sys_write+0x86/frame 0xfffffe00b1a35980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe00b1a35ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b1a35ab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x80041560a, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe490 --- Tracing command ufsbench pid 95675 tid 100781 td 0xfffff80861dfc000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe007876df40 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe007876df50 trap() at trap+0x5c/frame 0xfffffe007876e060 calltrap() at calltrap+0x8/frame 0xfffffe007876e060 --- trap 0xc, rip = 0xffffffff810426c6, rsp = 0xfffffe007876e130, rbp = 0xfffffe007876e140 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007876e140 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007876e1e0 db_command() at db_command+0x279/frame 0xfffffe007876e2b0 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007876e350 db_command() at db_command+0x279/frame 0xfffffe007876e420 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007876e430 db_trap() at db_trap+0xef/frame 0xfffffe007876e4c0 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007876e530 trap() at trap+0x806/frame 0xfffffe007876e640 calltrap() at calltrap+0x8/frame 0xfffffe007876e640 --- trap 0x3, rip = 0xffffffff80bd8196, rsp = 0xfffffe007876e710, rbp = 0xfffffe007876e730 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007876e730 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007876e740 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007876e770 uart_intr() at uart_intr+0x138/frame 0xfffffe007876e7b0 intr_event_handle() at intr_event_handle+0xcb/frame 0xfffffe007876e800 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007876e830 lapic_handle_intr() at lapic_handle_intr+0x5f/frame 0xfffffe007876e850 Xapic_isr1() at Xapic_isr1+0xd9/frame 0xfffffe007876e850 --- interrupt, rip = 0xffffffff8046209c, rsp = 0xfffffe007876e920, rbp = 0xfffffe007876e960 --- acpi_cpu_idle() at acpi_cpu_idle+0x34c/frame 0xfffffe007876e960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007876e980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe007876e9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007876ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe007876eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007876eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command ufsbench pid 95674 tid 102326 td 0xfffff80646632000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe045b9e7220 mi_switch() at mi_switch+0x16f/frame 0xfffffe045b9e7250 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe045b9e7290 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe045b9e72c0 sleeplk() at sleeplk+0x149/frame 0xfffffe045b9e7320 lockmgr_xlock_hard() at lockmgr_xlock_hard+0x307/frame 0xfffffe045b9e73a0 lockmgr_upgrade() at lockmgr_upgrade+0x11c/frame 0xfffffe045b9e73f0 __lockmgr_args() at __lockmgr_args+0x2e6/frame 0xfffffe045b9e7490 ffs_lock() at ffs_lock+0x7c/frame 0xfffffe045b9e74e0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xa6/frame 0xfffffe045b9e7510 _vn_lock() at _vn_lock+0x60/frame 0xfffffe045b9e7580 lookup() at lookup+0x57e/frame 0xfffffe045b9e7620 namei() at namei+0x4bd/frame 0xfffffe045b9e76e0 vn_open_cred() at vn_open_cred+0xd4/frame 0xfffffe045b9e7820 kern_openat() at kern_openat+0x20c/frame 0xfffffe045b9e7980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe045b9e7ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe045b9e7ab0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x8003a419a, rsp = 0x7fffffffdb38, rbp = 0x7fffffffdc20 --- Tracing command ufsbench pid 95673 tid 100278 td 0xfffff80842fae000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a2d5ee90 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a2d5eec0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a2d5ef00 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a2d5ef30 _sleep() at _sleep+0x358/frame 0xfffffe00a2d5efd0 vm_wait_doms() at vm_wait_doms+0xd2/frame 0xfffffe00a2d5eff0 vm_wait_domain() at vm_wait_domain+0x68/frame 0xfffffe00a2d5f020 vm_domain_alloc_fail() at vm_domain_alloc_fail+0x73/frame 0xfffffe00a2d5f050 vm_page_alloc_domain_after() at vm_page_alloc_domain_after+0x274/frame 0xfffffe00a2d5f0c0 vm_page_grab_pages() at vm_page_grab_pages+0x274/frame 0xfffffe00a2d5f160 allocbuf() at allocbuf+0x340/frame 0xfffffe00a2d5f1d0 getblkx() at getblkx+0x715/frame 0xfffffe00a2d5f290 getblk() at getblk+0x15/frame 0xfffffe00a2d5f2b0 ffs_balloc_ufs2() at ffs_balloc_ufs2+0xdbd/frame 0xfffffe00a2d5f450 ffs_write() at ffs_write+0x21e/frame 0xfffffe00a2d5f4f0 VOP_WRITE_APV() at VOP_WRITE_APV+0x13f/frame 0xfffffe00a2d5f600 vn_write() at vn_write+0x273/frame 0xfffffe00a2d5f690 vn_io_fault_doio() at vn_io_fault_doio+0x43/frame 0xfffffe00a2d5f6f0 vn_io_fault1() at vn_io_fault1+0x171/frame 0xfffffe00a2d5f830 vn_io_fault() at vn_io_fault+0x195/frame 0xfffffe00a2d5f8a0 dofilewrite() at dofilewrite+0x97/frame 0xfffffe00a2d5f8f0 kern_writev() at kern_writev+0x4a/frame 0xfffffe00a2d5f930 sys_write() at sys_write+0x86/frame 0xfffffe00a2d5f980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe00a2d5fab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a2d5fab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x80041560a, rsp = 0x7fffffffe3a8, rbp = 0x7fffffffe490 --- Tracing command ufsbench pid 95672 tid 102405 td 0xfffff80847d89000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe007876df40 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe007876df50 trap() at trap+0x5c/frame 0xfffffe007876e060 calltrap() at calltrap+0x8/frame 0xfffffe007876e060 --- trap 0xc, rip = 0xffffffff810426c6, rsp = 0xfffffe007876e130, rbp = 0xfffffe007876e140 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007876e140 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007876e1e0 db_command() at db_command+0x279/frame 0xfffffe007876e2b0 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007876e350 db_command() at db_command+0x279/frame 0xfffffe007876e420 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007876e430 db_trap() at db_trap+0xef/frame 0xfffffe007876e4c0 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007876e530 trap() at trap+0x806/frame 0xfffffe007876e640 calltrap() at calltrap+0x8/frame 0xfffffe007876e640 --- trap 0x3, rip = 0xffffffff80bd8196, rsp = 0xfffffe007876e710, rbp = 0xfffffe007876e730 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007876e730 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007876e740 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007876e770 uart_intr() at uart_intr+0x138/frame 0xfffffe007876e7b0 intr_event_handle() at intr_event_handle+0xcb/frame 0xfffffe007876e800 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007876e830 lapic_handle_intr() at lapic_handle_intr+0x5f/frame 0xfffffe007876e850 Xapic_isr1() at Xapic_isr1+0xd9/frame 0xfffffe007876e850 --- interrupt, rip = 0xffffffff8046209c, rsp = 0xfffffe007876e920, rbp = 0xfffffe007876e960 --- acpi_cpu_idle() at acpi_cpu_idle+0x34c/frame 0xfffffe007876e960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007876e980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe007876e9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007876ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe007876eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007876eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command ufsbench pid 95671 tid 100755 td 0xfffff8004cde0000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a795b0c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a795b0f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a795b130 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a795b160 sleeplk() at sleeplk+0x149/frame 0xfffffe00a795b1c0 lockmgr_slock_hard() at lockmgr_slock_hard+0x2db/frame 0xfffffe00a795b250 __lockmgr_args() at __lockmgr_args+0x3f9/frame 0xfffffe00a795b2f0 ffs_lock() at ffs_lock+0x7c/frame 0xfffffe00a795b340 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xa6/frame 0xfffffe00a795b370 _vn_lock() at _vn_lock+0x60/frame 0xfffffe00a795b3e0 vget() at vget+0x96/frame 0xfffffe00a795b420 cache_lookup() at cache_lookup+0x6eb/frame 0xfffffe00a795b4f0 vfs_cache_lookup() at vfs_cache_lookup+0xac/frame 0xfffffe00a795b550 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xbc/frame 0xfffffe00a795b580 lookup() at lookup+0x671/frame 0xfffffe00a795b620 namei() at namei+0x4bd/frame 0xfffffe00a795b6e0 vn_open_cred() at vn_open_cred+0xd4/frame 0xfffffe00a795b820 kern_openat() at kern_openat+0x20c/frame 0xfffffe00a795b980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe00a795bab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a795bab0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x8003a419a, rsp = 0x7fffffffdb38, rbp = 0x7fffffffdc20 --- Tracing command ufsbench pid 95670 tid 102398 td 0xfffff80860c3b000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe028db52e90 mi_switch() at mi_switch+0x16f/frame 0xfffffe028db52ec0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe028db52f00 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe028db52f30 _sleep() at _sleep+0x358/frame 0xfffffe028db52fd0 vm_wait_doms() at vm_wait_doms+0xd2/frame 0xfffffe028db52ff0 vm_wait_domain() at vm_wait_domain+0x68/frame 0xfffffe028db53020 vm_domain_alloc_fail() at vm_domain_alloc_fail+0x73/frame 0xfffffe028db53050 vm_page_alloc_domain_after() at vm_page_alloc_domain_after+0x274/frame 0xfffffe028db530c0 vm_page_grab_pages() at vm_page_grab_pages+0x274/frame 0xfffffe028db53160 allocbuf() at allocbuf+0x340/frame 0xfffffe028db531d0 getblkx() at getblkx+0x715/frame 0xfffffe028db53290 getblk() at getblk+0x15/frame 0xfffffe028db532b0 ffs_balloc_ufs2() at ffs_balloc_ufs2+0xdbd/frame 0xfffffe028db53450 ffs_write() at ffs_write+0x21e/frame 0xfffffe028db534f0 VOP_WRITE_APV() at VOP_WRITE_APV+0x13f/frame 0xfffffe028db53600 vn_write() at vn_write+0x273/frame 0xfffffe028db53690 vn_io_fault_doio() at vn_io_fault_doio+0x43/frame 0xfffffe028db536f0 vn_io_fault1() at vn_io_fault1+0x171/frame 0xfffffe028db53830 vn_io_fault() at vn_io_fault+0x195/frame 0xfffffe028db538a0 dofilewrite() at dofilewrite+0x97/frame 0xfffffe028db538f0 kern_writev() at kern_writev+0x4a/frame 0xfffffe028db53930 sys_write() at sys_write+0x86/frame 0xfffffe028db53980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe028db53ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe028db53ab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x80041560a, rsp = 0x7fffffffe3a8, rbp = 0x7fffffffe490 --- Tracing command ufsbench pid 95669 tid 101119 td 0xfffff80010fd8000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a67c5f60 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a67c5f90 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a67c5fd0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a67c6000 _sleep() at _sleep+0x358/frame 0xfffffe00a67c60a0 vm_wait_doms() at vm_wait_doms+0xd2/frame 0xfffffe00a67c60c0 vm_wait_domain() at vm_wait_domain+0x68/frame 0xfffffe00a67c60f0 vm_domain_alloc_fail() at vm_domain_alloc_fail+0x73/frame 0xfffffe00a67c6120 vm_page_alloc_domain_after() at vm_page_alloc_domain_after+0x274/frame 0xfffffe00a67c6190 vm_page_grab_pages() at vm_page_grab_pages+0x274/frame 0xfffffe00a67c6230 allocbuf() at allocbuf+0x340/frame 0xfffffe00a67c62a0 getblkx() at getblkx+0x715/frame 0xfffffe00a67c6360 getblk() at getblk+0x15/frame 0xfffffe00a67c6380 ffs_nodealloccg() at ffs_nodealloccg+0x213/frame 0xfffffe00a67c6420 ffs_hashalloc() at ffs_hashalloc+0xc4/frame 0xfffffe00a67c6470 ffs_valloc() at ffs_valloc+0x454/frame 0xfffffe00a67c6500 ufs_makeinode() at ufs_makeinode+0xb2/frame 0xfffffe00a67c6690 ufs_create() at ufs_create+0x34/frame 0xfffffe00a67c66b0 VOP_CREATE_APV() at VOP_CREATE_APV+0xbc/frame 0xfffffe00a67c66e0 vn_open_cred() at vn_open_cred+0x29c/frame 0xfffffe00a67c6820 kern_openat() at kern_openat+0x20c/frame 0xfffffe00a67c6980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe00a67c6ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a67c6ab0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x8003a419a, rsp = 0x7fffffffdb38, rbp = 0x7fffffffdc20 --- Tracing command ufsbench pid 95668 tid 100619 td 0xfffff80842f33000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe007876df40 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe007876df50 trap() at trap+0x5c/frame 0xfffffe007876e060 calltrap() at calltrap+0x8/frame 0xfffffe007876e060 --- trap 0xc, rip = 0xffffffff810426c6, rsp = 0xfffffe007876e130, rbp = 0xfffffe007876e140 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007876e140 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007876e1e0 db_command() at db_command+0x279/frame 0xfffffe007876e2b0 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007876e350 db_command() at db_command+0x279/frame 0xfffffe007876e420 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007876e430 db_trap() at db_trap+0xef/frame 0xfffffe007876e4c0 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007876e530 trap() at trap+0x806/frame 0xfffffe007876e640 calltrap() at calltrap+0x8/frame 0xfffffe007876e640 --- trap 0x3, rip = 0xffffffff80bd8196, rsp = 0xfffffe007876e710, rbp = 0xfffffe007876e730 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007876e730 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007876e740 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007876e770 uart_intr() at uart_intr+0x138/frame 0xfffffe007876e7b0 intr_event_handle() at intr_event_handle+0xcb/frame 0xfffffe007876e800 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007876e830 lapic_handle_intr() at lapic_handle_intr+0x5f/frame 0xfffffe007876e850 Xapic_isr1() at Xapic_isr1+0xd9/frame 0xfffffe007876e850 --- interrupt, rip = 0xffffffff8046209c, rsp = 0xfffffe007876e920, rbp = 0xfffffe007876e960 --- acpi_cpu_idle() at acpi_cpu_idle+0x34c/frame 0xfffffe007876e960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007876e980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe007876e9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007876ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe007876eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007876eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command ufsbench pid 95667 tid 100888 td 0xfffff80884cd6580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a8517260 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a8517290 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a85172d0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a8517300 sleeplk() at sleeplk+0x149/frame 0xfffffe00a8517360 lockmgr_slock_hard() at lockmgr_slock_hard+0x2db/frame 0xfffffe00a85173f0 __lockmgr_args() at __lockmgr_args+0x3f9/frame 0xfffffe00a8517490 ffs_lock() at ffs_lock+0x7c/frame 0xfffffe00a85174e0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xa6/frame 0xfffffe00a8517510 _vn_lock() at _vn_lock+0x60/frame 0xfffffe00a8517580 lookup() at lookup+0xda/frame 0xfffffe00a8517620 namei() at namei+0x4bd/frame 0xfffffe00a85176e0 vn_open_cred() at vn_open_cred+0xd4/frame 0xfffffe00a8517820 kern_openat() at kern_openat+0x20c/frame 0xfffffe00a8517980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe00a8517ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a8517ab0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x8003a419a, rsp = 0x7fffffffe2d8, rbp = 0x7fffffffe3c0 --- Tracing command ufsbench pid 95666 tid 102086 td 0xfffff802d2d05000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe007876df40 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe007876df50 trap() at trap+0x5c/frame 0xfffffe007876e060 calltrap() at calltrap+0x8/frame 0xfffffe007876e060 --- trap 0xc, rip = 0xffffffff810426c6, rsp = 0xfffffe007876e130, rbp = 0xfffffe007876e140 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007876e140 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007876e1e0 db_command() at db_command+0x279/frame 0xfffffe007876e2b0 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007876e350 db_command() at db_command+0x279/frame 0xfffffe007876e420 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007876e430 db_trap() at db_trap+0xef/frame 0xfffffe007876e4c0 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007876e530 trap() at trap+0x806/frame 0xfffffe007876e640 calltrap() at calltrap+0x8/frame 0xfffffe007876e640 --- trap 0x3, rip = 0xffffffff80bd8196, rsp = 0xfffffe007876e710, rbp = 0xfffffe007876e730 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007876e730 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007876e740 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007876e770 uart_intr() at uart_intr+0x138/frame 0xfffffe007876e7b0 intr_event_handle() at intr_event_handle+0xcb/frame 0xfffffe007876e800 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007876e830 lapic_handle_intr() at lapic_handle_intr+0x5f/frame 0xfffffe007876e850 Xapic_isr1() at Xapic_isr1+0xd9/frame 0xfffffe007876e850 --- interrupt, rip = 0xffffffff8046209c, rsp = 0xfffffe007876e920, rbp = 0xfffffe007876e960 --- acpi_cpu_idle() at acpi_cpu_idle+0x34c/frame 0xfffffe007876e960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007876e980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe007876e9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007876ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe007876eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007876eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command ufsbench pid 95665 tid 100836 td 0xfffff80847bfb000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a8412f60 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a8412f90 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a8412fd0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a8413000 _sleep() at _sleep+0x358/frame 0xfffffe00a84130a0 vm_wait_doms() at vm_wait_doms+0xd2/frame 0xfffffe00a84130c0 vm_wait_domain() at vm_wait_domain+0x68/frame 0xfffffe00a84130f0 vm_domain_alloc_fail() at vm_domain_alloc_fail+0x73/frame 0xfffffe00a8413120 vm_page_alloc_domain_after() at vm_page_alloc_domain_after+0x274/frame 0xfffffe00a8413190 vm_page_grab_pages() at vm_page_grab_pages+0x274/frame 0xfffffe00a8413230 allocbuf() at allocbuf+0x340/frame 0xfffffe00a84132a0 getblkx() at getblkx+0x715/frame 0xfffffe00a8413360 getblk() at getblk+0x15/frame 0xfffffe00a8413380 ffs_nodealloccg() at ffs_nodealloccg+0x213/frame 0xfffffe00a8413420 ffs_hashalloc() at ffs_hashalloc+0xc4/frame 0xfffffe00a8413470 ffs_valloc() at ffs_valloc+0x454/frame 0xfffffe00a8413500 ufs_makeinode() at ufs_makeinode+0xb2/frame 0xfffffe00a8413690 ufs_create() at ufs_create+0x34/frame 0xfffffe00a84136b0 VOP_CREATE_APV() at VOP_CREATE_APV+0xbc/frame 0xfffffe00a84136e0 vn_open_cred() at vn_open_cred+0x29c/frame 0xfffffe00a8413820 kern_openat() at kern_openat+0x20c/frame 0xfffffe00a8413980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe00a8413ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a8413ab0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x8003a419a, rsp = 0x7fffffffe2d8, rbp = 0x7fffffffe3c0 --- Tracing command ufsbench pid 95664 tid 100686 td 0xfffff8004c790580 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe007876df40 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe007876df50 trap() at trap+0x5c/frame 0xfffffe007876e060 calltrap() at calltrap+0x8/frame 0xfffffe007876e060 --- trap 0xc, rip = 0xffffffff810426c6, rsp = 0xfffffe007876e130, rbp = 0xfffffe007876e140 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007876e140 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007876e1e0 db_command() at db_command+0x279/frame 0xfffffe007876e2b0 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007876e350 db_command() at db_command+0x279/frame 0xfffffe007876e420 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007876e430 db_trap() at db_trap+0xef/frame 0xfffffe007876e4c0 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007876e530 trap() at trap+0x806/frame 0xfffffe007876e640 calltrap() at calltrap+0x8/frame 0xfffffe007876e640 --- trap 0x3, rip = 0xffffffff80bd8196, rsp = 0xfffffe007876e710, rbp = 0xfffffe007876e730 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007876e730 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007876e740 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007876e770 uart_intr() at uart_intr+0x138/frame 0xfffffe007876e7b0 intr_event_handle() at intr_event_handle+0xcb/frame 0xfffffe007876e800 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007876e830 lapic_handle_intr() at lapic_handle_intr+0x5f/frame 0xfffffe007876e850 Xapic_isr1() at Xapic_isr1+0xd9/frame 0xfffffe007876e850 --- interrupt, rip = 0xffffffff8046209c, rsp = 0xfffffe007876e920, rbp = 0xfffffe007876e960 --- acpi_cpu_idle() at acpi_cpu_idle+0x34c/frame 0xfffffe007876e960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007876e980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe007876e9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007876ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe007876eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007876eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command ufsbench pid 95663 tid 100673 td 0xfffff808615a5580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a6703260 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a6703290 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a67032d0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a6703300 sleeplk() at sleeplk+0x149/frame 0xfffffe00a6703360 lockmgr_slock_hard() at lockmgr_slock_hard+0x2db/frame 0xfffffe00a67033f0 __lockmgr_args() at __lockmgr_args+0x3f9/frame 0xfffffe00a6703490 ffs_lock() at ffs_lock+0x7c/frame 0xfffffe00a67034e0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xa6/frame 0xfffffe00a6703510 _vn_lock() at _vn_lock+0x60/frame 0xfffffe00a6703580 lookup() at lookup+0xda/frame 0xfffffe00a6703620 namei() at namei+0x4bd/frame 0xfffffe00a67036e0 vn_open_cred() at vn_open_cred+0xd4/frame 0xfffffe00a6703820 kern_openat() at kern_openat+0x20c/frame 0xfffffe00a6703980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe00a6703ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a6703ab0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x8003a419a, rsp = 0x7fffffffe2d8, rbp = 0x7fffffffe3c0 --- Tracing command md10 pid 95651 tid 100728 td 0xfffff8004ca8c580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a78d48d0 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a78d4900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a78d4940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a78d4970 _sleep() at _sleep+0x358/frame 0xfffffe00a78d4a10 md_kthread() at md_kthread+0x1e5/frame 0xfffffe00a78d4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a78d4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a78d4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command getty pid 965 tid 100543 td 0xfffff808475f9000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a64f4630 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a64f4660 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a64f46a0 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00a64f4700 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00a64f4730 _cv_wait_sig() at _cv_wait_sig+0x1ce/frame 0xfffffe00a64f4790 tty_wait() at tty_wait+0x42/frame 0xfffffe00a64f47c0 ttydisc_read() at ttydisc_read+0x212/frame 0xfffffe00a64f4820 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00a64f4850 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00a64f48b0 dofileread() at dofileread+0x95/frame 0xfffffe00a64f48f0 kern_readv() at kern_readv+0x4a/frame 0xfffffe00a64f4930 sys_read() at sys_read+0x86/frame 0xfffffe00a64f4980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe00a64f4ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a64f4ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80042f64a, rsp = 0x7fffffffebe8, rbp = 0x7fffffffec30 --- Tracing command getty pid 964 tid 100542 td 0xfffff808475f9580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a64ef630 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a64ef660 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a64ef6a0 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00a64ef700 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00a64ef730 _cv_wait_sig() at _cv_wait_sig+0x1ce/frame 0xfffffe00a64ef790 tty_wait() at tty_wait+0x42/frame 0xfffffe00a64ef7c0 ttydisc_read() at ttydisc_read+0x212/frame 0xfffffe00a64ef820 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00a64ef850 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00a64ef8b0 dofileread() at dofileread+0x95/frame 0xfffffe00a64ef8f0 kern_readv() at kern_readv+0x4a/frame 0xfffffe00a64ef930 sys_read() at sys_read+0x86/frame 0xfffffe00a64ef980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe00a64efab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a64efab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80042f64a, rsp = 0x7fffffffebe8, rbp = 0x7fffffffec30 --- Tracing command getty pid 963 tid 100221 td 0xfffff80007dd0580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009dff9630 mi_switch() at mi_switch+0x16f/frame 0xfffffe009dff9660 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009dff96a0 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe009dff9700 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe009dff9730 _cv_wait_sig() at _cv_wait_sig+0x1ce/frame 0xfffffe009dff9790 tty_wait() at tty_wait+0x42/frame 0xfffffe009dff97c0 ttydisc_read() at ttydisc_read+0x212/frame 0xfffffe009dff9820 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe009dff9850 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe009dff98b0 dofileread() at dofileread+0x95/frame 0xfffffe009dff98f0 kern_readv() at kern_readv+0x4a/frame 0xfffffe009dff9930 sys_read() at sys_read+0x86/frame 0xfffffe009dff9980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe009dff9ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009dff9ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80042f64a, rsp = 0x7fffffffebe8, rbp = 0x7fffffffec30 --- Tracing command inetd pid 944 tid 100537 td 0xfffff80847119000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a64d65c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a64d65f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a64d6630 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00a64d6690 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00a64d66c0 _cv_wait_sig() at _cv_wait_sig+0x1ce/frame 0xfffffe00a64d6720 seltdwait() at seltdwait+0x8d/frame 0xfffffe00a64d6760 kern_select() at kern_select+0x8bf/frame 0xfffffe00a64d6940 sys_select() at sys_select+0x56/frame 0xfffffe00a64d6980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe00a64d6ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a64d6ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80044b62a, rsp = 0x7fffffffde38, rbp = 0x7fffffffecd0 --- Tracing command nfscbd pid 911 tid 100532 td 0xfffff80842dfd000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a64bd470 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a64bd4a0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a64bd4e0 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00a64bd540 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00a64bd580 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x1f1/frame 0xfffffe00a64bd5f0 svc_run_internal() at svc_run_internal+0xa92/frame 0xfffffe00a64bd730 svc_run() at svc_run+0x1fe/frame 0xfffffe00a64bd790 nfscbd_nfsd() at nfscbd_nfsd+0xe6/frame 0xfffffe00a64bd830 nfssvc_nfscl() at nfssvc_nfscl+0x213/frame 0xfffffe00a64bd960 sys_nfssvc() at sys_nfssvc+0x9c/frame 0xfffffe00a64bd980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe00a64bdab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a64bdab0 --- syscall (155, FreeBSD ELF64, sys_nfssvc), rip = 0x8002d45ba, rsp = 0x7fffffffe9a8, rbp = 0x7fffffffece0 --- Tracing command nfscbd pid 911 tid 100533 td 0xfffff808429d4000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a64c27c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a64c27f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a64c2830 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00a64c2890 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00a64c28c0 _cv_wait_sig() at _cv_wait_sig+0x1ce/frame 0xfffffe00a64c2920 svc_run_internal() at svc_run_internal+0xbb5/frame 0xfffffe00a64c2a60 svc_thread_start() at svc_thread_start+0xb/frame 0xfffffe00a64c2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a64c2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a64c2ab0 --- trap 0xc, rip = 0x8002d45ba, rsp = 0x7fffffffe9a8, rbp = 0x7fffffffece0 --- Tracing command nfscbd pid 911 tid 100534 td 0xfffff808429c8580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a64c77c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a64c77f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a64c7830 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00a64c7890 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00a64c78c0 _cv_wait_sig() at _cv_wait_sig+0x1ce/frame 0xfffffe00a64c7920 svc_run_internal() at svc_run_internal+0xbb5/frame 0xfffffe00a64c7a60 svc_thread_start() at svc_thread_start+0xb/frame 0xfffffe00a64c7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a64c7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a64c7ab0 --- trap 0xc, rip = 0x8002d45ba, rsp = 0x7fffffffe9a8, rbp = 0x7fffffffece0 --- Tracing command nfscbd pid 911 tid 100535 td 0xfffff808475f8580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a64cc7c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a64cc7f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a64cc830 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00a64cc890 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00a64cc8c0 _cv_wait_sig() at _cv_wait_sig+0x1ce/frame 0xfffffe00a64cc920 svc_run_internal() at svc_run_internal+0xbb5/frame 0xfffffe00a64cca60 svc_thread_start() at svc_thread_start+0xb/frame 0xfffffe00a64cca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a64ccab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a64ccab0 --- trap 0xc, rip = 0x8002d45ba, rsp = 0x7fffffffe9a8, rbp = 0x7fffffffece0 --- Tracing command nfscbd pid 910 tid 100509 td 0xfffff80842711580 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe007876df40 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe007876df50 trap() at trap+0x5c/frame 0xfffffe007876e060 calltrap() at calltrap+0x8/frame 0xfffffe007876e060 --- trap 0xc, rip = 0xffffffff810426c6, rsp = 0xfffffe007876e130, rbp = 0xfffffe007876e140 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007876e140 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007876e1e0 db_command() at db_command+0x279/frame 0xfffffe007876e2b0 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007876e350 db_command() at db_command+0x279/frame 0xfffffe007876e420 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007876e430 db_trap() at db_trap+0xef/frame 0xfffffe007876e4c0 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007876e530 trap() at trap+0x806/frame 0xfffffe007876e640 calltrap() at calltrap+0x8/frame 0xfffffe007876e640 --- trap 0x3, rip = 0xffffffff80bd8196, rsp = 0xfffffe007876e710, rbp = 0xfffffe007876e730 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007876e730 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007876e740 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007876e770 uart_intr() at uart_intr+0x138/frame 0xfffffe007876e7b0 intr_event_handle() at intr_event_handle+0xcb/frame 0xfffffe007876e800 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007876e830 lapic_handle_intr() at lapic_handle_intr+0x5f/frame 0xfffffe007876e850 Xapic_isr1() at Xapic_isr1+0xd9/frame 0xfffffe007876e850 --- interrupt, rip = 0xffffffff8046209c, rsp = 0xfffffe007876e920, rbp = 0xfffffe007876e960 --- acpi_cpu_idle() at acpi_cpu_idle+0x34c/frame 0xfffffe007876e960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007876e980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe007876e9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007876ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe007876eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007876eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cron pid 821 tid 100504 td 0xfffff8000b4d7000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe007876df40 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe007876df50 trap() at trap+0x5c/frame 0xfffffe007876e060 calltrap() at calltrap+0x8/frame 0xfffffe007876e060 --- trap 0xc, rip = 0xffffffff810426c6, rsp = 0xfffffe007876e130, rbp = 0xfffffe007876e140 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007876e140 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007876e1e0 db_command() at db_command+0x279/frame 0xfffffe007876e2b0 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007876e350 db_command() at db_command+0x279/frame 0xfffffe007876e420 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007876e430 db_trap() at db_trap+0xef/frame 0xfffffe007876e4c0 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007876e530 trap() at trap+0x806/frame 0xfffffe007876e640 calltrap() at calltrap+0x8/frame 0xfffffe007876e640 --- trap 0x3, rip = 0xffffffff80bd8196, rsp = 0xfffffe007876e710, rbp = 0xfffffe007876e730 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007876e730 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007876e740 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007876e770 uart_intr() at uart_intr+0x138/frame 0xfffffe007876e7b0 intr_event_handle() at intr_event_handle+0xcb/frame 0xfffffe007876e800 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007876e830 lapic_handle_intr() at lapic_handle_intr+0x5f/frame 0xfffffe007876e850 Xapic_isr1() at Xapic_isr1+0xd9/frame 0xfffffe007876e850 --- interrupt, rip = 0xffffffff8046209c, rsp = 0xfffffe007876e920, rbp = 0xfffffe007876e960 --- acpi_cpu_idle() at acpi_cpu_idle+0x34c/frame 0xfffffe007876e960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007876e980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe007876e9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007876ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe007876eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007876eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sshd pid 811 tid 100507 td 0xfffff8084035a580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a64405c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a64405f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a6440630 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00a6440690 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00a64406c0 _cv_wait_sig() at _cv_wait_sig+0x1ce/frame 0xfffffe00a6440720 seltdwait() at seltdwait+0x8d/frame 0xfffffe00a6440760 kern_select() at kern_select+0x8bf/frame 0xfffffe00a6440940 sys_select() at sys_select+0x56/frame 0xfffffe00a6440980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe00a6440ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a6440ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80087862a, rsp = 0x7fffffffe698, rbp = 0x7fffffffe6d0 --- Tracing command nfsuserd pid 696 tid 100297 td 0xfffff80842e4f000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a60225a0 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a60225d0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a6022610 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00a6022670 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00a60226b0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x1f1/frame 0xfffffe00a6022720 seltdwait() at seltdwait+0x6b/frame 0xfffffe00a6022760 kern_select() at kern_select+0x8bf/frame 0xfffffe00a6022940 sys_select() at sys_select+0x56/frame 0xfffffe00a6022980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe00a6022ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a6022ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80041562a, rsp = 0x7fffffffd9a8, rbp = 0x7fffffffdaf0 --- Tracing command nfsuserd pid 695 tid 100296 td 0xfffff80842e4f580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a601d5a0 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a601d5d0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a601d610 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00a601d670 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00a601d6b0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x1f1/frame 0xfffffe00a601d720 seltdwait() at seltdwait+0x6b/frame 0xfffffe00a601d760 kern_select() at kern_select+0x8bf/frame 0xfffffe00a601d940 sys_select() at sys_select+0x56/frame 0xfffffe00a601d980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe00a601dab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a601dab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80041562a, rsp = 0x7fffffffd9a8, rbp = 0x7fffffffdaf0 --- Tracing command nfsuserd pid 693 tid 100283 td 0xfffff8000b229000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a2d785a0 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a2d785d0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a2d78610 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00a2d78670 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00a2d786b0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x1f1/frame 0xfffffe00a2d78720 seltdwait() at seltdwait+0x6b/frame 0xfffffe00a2d78760 kern_select() at kern_select+0x8bf/frame 0xfffffe00a2d78940 sys_select() at sys_select+0x56/frame 0xfffffe00a2d78980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe00a2d78ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a2d78ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80041562a, rsp = 0x7fffffffd9a8, rbp = 0x7fffffffdaf0 --- Tracing command nfsuserd pid 692 tid 100282 td 0xfffff8000b229580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a2d735a0 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a2d735d0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a2d73610 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00a2d73670 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00a2d736b0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x1f1/frame 0xfffffe00a2d73720 seltdwait() at seltdwait+0x6b/frame 0xfffffe00a2d73760 kern_select() at kern_select+0x8bf/frame 0xfffffe00a2d73940 sys_select() at sys_select+0x56/frame 0xfffffe00a2d73980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe00a2d73ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a2d73ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80041562a, rsp = 0x7fffffffd9a8, rbp = 0x7fffffffdaf0 --- Tracing command nfsuserd pid 691 tid 100249 td 0xfffff800038e3580 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe007876df40 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe007876df50 trap() at trap+0x5c/frame 0xfffffe007876e060 calltrap() at calltrap+0x8/frame 0xfffffe007876e060 --- trap 0xc, rip = 0xffffffff810426c6, rsp = 0xfffffe007876e130, rbp = 0xfffffe007876e140 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007876e140 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007876e1e0 db_command() at db_command+0x279/frame 0xfffffe007876e2b0 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007876e350 db_command() at db_command+0x279/frame 0xfffffe007876e420 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007876e430 db_trap() at db_trap+0xef/frame 0xfffffe007876e4c0 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007876e530 trap() at trap+0x806/frame 0xfffffe007876e640 calltrap() at calltrap+0x8/frame 0xfffffe007876e640 --- trap 0x3, rip = 0xffffffff80bd8196, rsp = 0xfffffe007876e710, rbp = 0xfffffe007876e730 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007876e730 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007876e740 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007876e770 uart_intr() at uart_intr+0x138/frame 0xfffffe007876e7b0 intr_event_handle() at intr_event_handle+0xcb/frame 0xfffffe007876e800 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007876e830 lapic_handle_intr() at lapic_handle_intr+0x5f/frame 0xfffffe007876e850 Xapic_isr1() at Xapic_isr1+0xd9/frame 0xfffffe007876e850 --- interrupt, rip = 0xffffffff8046209c, rsp = 0xfffffe007876e920, rbp = 0xfffffe007876e960 --- acpi_cpu_idle() at acpi_cpu_idle+0x34c/frame 0xfffffe007876e960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007876e980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe007876e9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007876ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe007876eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007876eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command casperd pid 672 tid 100295 td 0xfffff80007e3d000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a60185c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a60185f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a6018630 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00a6018690 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00a60186c0 _cv_wait_sig() at _cv_wait_sig+0x1ce/frame 0xfffffe00a6018720 seltdwait() at seltdwait+0x8d/frame 0xfffffe00a6018760 kern_select() at kern_select+0x8bf/frame 0xfffffe00a6018940 sys_select() at sys_select+0x56/frame 0xfffffe00a6018980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe00a6018ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a6018ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80085562a, rsp = 0x7fffffffe7f8, rbp = 0x7fffffffece0 --- Tracing command casperd pid 671 tid 100294 td 0xfffff80007e3d580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a60135c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a60135f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a6013630 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00a6013690 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00a60136c0 _cv_wait_sig() at _cv_wait_sig+0x1ce/frame 0xfffffe00a6013720 seltdwait() at seltdwait+0x8d/frame 0xfffffe00a6013760 kern_select() at kern_select+0x8bf/frame 0xfffffe00a6013940 sys_select() at sys_select+0x56/frame 0xfffffe00a6013980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe00a6013ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a6013ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80085562a, rsp = 0x7fffffffe658, rbp = 0x7fffffffe720 --- Tracing command watchdogd pid 651 tid 100285 td 0xfffff80847149000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a2d82700 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a2d82730 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a2d82770 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00a2d827d0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00a2d82810 _sleep() at _sleep+0x2e9/frame 0xfffffe00a2d828b0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1b7/frame 0xfffffe00a2d82940 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe00a2d82980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe00a2d82ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a2d82ab0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003bb1ba, rsp = 0x7fffffffeab8, rbp = 0x7fffffffeb00 --- Tracing command rtsold pid 530 tid 100261 td 0xfffff8000b2f3580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a2cc2640 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a2cc2670 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a2cc26b0 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00a2cc2710 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00a2cc2740 _cv_wait_sig() at _cv_wait_sig+0x1ce/frame 0xfffffe00a2cc27a0 seltdwait() at seltdwait+0x8d/frame 0xfffffe00a2cc27e0 kern_poll() at kern_poll+0x43a/frame 0xfffffe00a2cc2960 sys_poll() at sys_poll+0x50/frame 0xfffffe00a2cc2980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe00a2cc2ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a2cc2ab0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8003bbb5a, rsp = 0x7fffffffec58, rbp = 0x7fffffffece0 --- Tracing command moused pid 487 tid 100288 td 0xfffff8000b175000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a2d915c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a2d915f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a2d91630 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00a2d91690 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00a2d916c0 _cv_wait_sig() at _cv_wait_sig+0x1ce/frame 0xfffffe00a2d91720 seltdwait() at seltdwait+0x8d/frame 0xfffffe00a2d91760 kern_select() at kern_select+0x8bf/frame 0xfffffe00a2d91940 sys_select() at sys_select+0x56/frame 0xfffffe00a2d91980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe00a2d91ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a2d91ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80046562a, rsp = 0x7fffffffe968, rbp = 0x7fffffffec80 --- Tracing command vnlru pid 45 tid 100209 td 0xfffff808429c6000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009dfbd880 mi_switch() at mi_switch+0x16f/frame 0xfffffe009dfbd8b0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009dfbd8f0 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe009dfbd930 _sleep() at _sleep+0x2fc/frame 0xfffffe009dfbd9d0 vnlru_proc() at vnlru_proc+0xa4/frame 0xfffffe009dfbda70 fork_exit() at fork_exit+0x84/frame 0xfffffe009dfbdab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009dfbdab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command syncer pid 44 tid 100208 td 0xfffff808429c6580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009dfb88c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009dfb88f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009dfb8930 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe009dfb8970 _cv_timedwait_sbt() at _cv_timedwait_sbt+0x1f1/frame 0xfffffe009dfb89e0 sched_sync() at sched_sync+0x609/frame 0xfffffe009dfb8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009dfb8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009dfb8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 43 tid 100207 td 0xfffff80007d0b000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009dfb38d0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009dfb3900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009dfb3940 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe009dfb3980 _sleep() at _sleep+0x2fc/frame 0xfffffe009dfb3a20 buf_daemon() at buf_daemon+0xd8/frame 0xfffffe009dfb3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009dfb3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009dfb3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 43 tid 100210 td 0xfffff80007d26580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009dfc28c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009dfc28f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009dfc2930 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe009dfc2970 _sleep() at _sleep+0x2fc/frame 0xfffffe009dfc2a10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe009dfc2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009dfc2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009dfc2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 43 tid 100212 td 0xfffff80007d26000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009dfcc8c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009dfcc8f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009dfcc930 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe009dfcc970 _sleep() at _sleep+0x2fc/frame 0xfffffe009dfcca10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe009dfcca70 fork_exit() at fork_exit+0x84/frame 0xfffffe009dfccab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009dfccab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 43 tid 100214 td 0xfffff808429e3580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009dfd68c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009dfd68f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009dfd6930 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe009dfd6970 _sleep() at _sleep+0x2fc/frame 0xfffffe009dfd6a10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe009dfd6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009dfd6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009dfd6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 43 tid 100216 td 0xfffff808429e3000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009dfe08c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009dfe08f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009dfe0930 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe009dfe0970 _sleep() at _sleep+0x2fc/frame 0xfffffe009dfe0a10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe009dfe0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009dfe0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009dfe0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 43 tid 100218 td 0xfffff80007d25580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009dfea8c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009dfea8f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009dfea930 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe009dfea970 _sleep() at _sleep+0x2fc/frame 0xfffffe009dfeaa10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe009dfeaa70 fork_exit() at fork_exit+0x84/frame 0xfffffe009dfeaab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009dfeaab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 43 tid 100219 td 0xfffff80007d25000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009dfef8c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009dfef8f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009dfef930 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe009dfef970 _sleep() at _sleep+0x2fc/frame 0xfffffe009dfefa10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe009dfefa70 fork_exit() at fork_exit+0x84/frame 0xfffffe009dfefab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009dfefab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 43 tid 100220 td 0xfffff808429e0580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009dff48c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009dff48f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009dff4930 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe009dff4970 _sleep() at _sleep+0x2fc/frame 0xfffffe009dff4a10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe009dff4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009dff4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009dff4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 43 tid 100233 td 0xfffff80007e7c000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a2c368f0 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a2c36920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a2c36960 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe00a2c369a0 _sleep() at _sleep+0x2fc/frame 0xfffffe00a2c36a40 softdep_flush() at softdep_flush+0x17f/frame 0xfffffe00a2c36a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a2c36ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a2c36ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 43 tid 100241 td 0xfffff80842dfe580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a2c5e8f0 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a2c5e920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a2c5e960 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe00a2c5e9a0 _sleep() at _sleep+0x2fc/frame 0xfffffe00a2c5ea40 softdep_flush() at softdep_flush+0x17f/frame 0xfffffe00a2c5ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a2c5eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a2c5eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 43 tid 100243 td 0xfffff8000b175580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a2c688f0 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a2c68920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a2c68960 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe00a2c689a0 _sleep() at _sleep+0x2fc/frame 0xfffffe00a2c68a40 softdep_flush() at softdep_flush+0x17f/frame 0xfffffe00a2c68a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a2c68ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a2c68ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command vmdaemon pid 42 tid 100206 td 0xfffff80007d0b580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009dfae8b0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009dfae8e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009dfae920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009dfae950 _sleep() at _sleep+0x358/frame 0xfffffe009dfae9f0 vm_daemon() at vm_daemon+0xaa6/frame 0xfffffe009dfaea70 fork_exit() at fork_exit+0x84/frame 0xfffffe009dfaeab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009dfaeab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 41 tid 100205 td 0xfffff808429c8000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009dfa9560 mi_switch() at mi_switch+0x16f/frame 0xfffffe009dfa9590 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009dfa95d0 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe009dfa9610 _sleep() at _sleep+0x2fc/frame 0xfffffe009dfa96b0 vm_pageout_worker() at vm_pageout_worker+0x24e/frame 0xfffffe009dfa9a50 vm_pageout() at vm_pageout+0x133/frame 0xfffffe009dfa9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009dfa9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009dfa9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 41 tid 100211 td 0xfffff808429e9580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009dfc76b0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009dfc76e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009dfc7720 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009dfc7750 _sleep() at _sleep+0x358/frame 0xfffffe009dfc77f0 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0xc70/frame 0xfffffe009dfc7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009dfc7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009dfc7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 41 tid 100213 td 0xfffff808429e9000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009dfd1560 mi_switch() at mi_switch+0x16f/frame 0xfffffe009dfd1590 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009dfd15d0 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe009dfd1610 _sleep() at _sleep+0x2fc/frame 0xfffffe009dfd16b0 pause_sbt() at pause_sbt+0x11f/frame 0xfffffe009dfd16f0 vm_pageout_worker() at vm_pageout_worker+0x1fb/frame 0xfffffe009dfd1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009dfd1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009dfd1ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 41 tid 100215 td 0xfffff80007d2c580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009dfdb660 mi_switch() at mi_switch+0x16f/frame 0xfffffe009dfdb690 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009dfdb6d0 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe009dfdb710 _sleep() at _sleep+0x2fc/frame 0xfffffe009dfdb7b0 pause_sbt() at pause_sbt+0x11f/frame 0xfffffe009dfdb7f0 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0xb22/frame 0xfffffe009dfdba70 fork_exit() at fork_exit+0x84/frame 0xfffffe009dfdbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009dfdbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 41 tid 100217 td 0xfffff80007d2c000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009dfe58f0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009dfe5920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009dfe5960 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe009dfe59a0 _sleep() at _sleep+0x2fc/frame 0xfffffe009dfe5a40 uma_reclaim_worker() at uma_reclaim_worker+0xda/frame 0xfffffe009dfe5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009dfe5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009dfe5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon0 pid 40 tid 100204 td 0xfffff8084270f000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009dfa48b0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009dfa48e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009dfa4920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009dfa4950 _sleep() at _sleep+0x358/frame 0xfffffe009dfa49f0 enc_daemon() at enc_daemon+0x1df/frame 0xfffffe009dfa4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009dfa4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009dfa4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command rand_harvestq pid 39 tid 100201 td 0xfffff80007afb580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009df958c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009df958f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009df95930 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe009df95970 _sleep() at _sleep+0x2fc/frame 0xfffffe009df95a10 random_kthread() at random_kthread+0x296/frame 0xfffffe009df95a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009df95ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009df95ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sctp_iterator pid 38 tid 100197 td 0xfffff80007afc580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009df69910 mi_switch() at mi_switch+0x16f/frame 0xfffffe009df69940 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009df69980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009df699b0 _sleep() at _sleep+0x358/frame 0xfffffe009df69a50 sctp_iterator_thread() at sctp_iterator_thread+0x59/frame 0xfffffe009df69a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009df69ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009df69ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod4 pid 37 tid 100196 td 0xfffff8084270f580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009df4c8e0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009df4c910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009df4c950 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe009df4c990 _sleep() at _sleep+0x2fc/frame 0xfffffe009df4ca30 soaio_kproc_loop() at soaio_kproc_loop+0x170/frame 0xfffffe009df4ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe009df4cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009df4cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod3 pid 36 tid 100195 td 0xfffff80007a73000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009df478e0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009df47910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009df47950 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe009df47990 _sleep() at _sleep+0x2fc/frame 0xfffffe009df47a30 soaio_kproc_loop() at soaio_kproc_loop+0x170/frame 0xfffffe009df47a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009df47ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009df47ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod2 pid 35 tid 100194 td 0xfffff80007a73580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009df428e0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009df42910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009df42950 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe009df42990 _sleep() at _sleep+0x2fc/frame 0xfffffe009df42a30 soaio_kproc_loop() at soaio_kproc_loop+0x170/frame 0xfffffe009df42a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009df42ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009df42ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod1 pid 34 tid 100193 td 0xfffff80842711000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009df3d8e0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009df3d910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009df3d950 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe009df3d990 _sleep() at _sleep+0x2fc/frame 0xfffffe009df3da30 soaio_kproc_loop() at soaio_kproc_loop+0x170/frame 0xfffffe009df3da70 fork_exit() at fork_exit+0x84/frame 0xfffffe009df3dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009df3dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 33 tid 100171 td 0xfffff80840dd7000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de9f930 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de9f960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de9f9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de9f9d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe009de9fa30 usb_process() at usb_process+0x12b/frame 0xfffffe009de9fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de9fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de9fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 33 tid 100172 td 0xfffff8000436e580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009dea4930 mi_switch() at mi_switch+0x16f/frame 0xfffffe009dea4960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009dea49a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009dea49d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe009dea4a30 usb_process() at usb_process+0x12b/frame 0xfffffe009dea4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009dea4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009dea4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 33 tid 100173 td 0xfffff8000436e000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009dea9930 mi_switch() at mi_switch+0x16f/frame 0xfffffe009dea9960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009dea99a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009dea99d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe009dea9a30 usb_process() at usb_process+0x12b/frame 0xfffffe009dea9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009dea9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009dea9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 33 tid 100174 td 0xfffff80840dd3580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009deae930 mi_switch() at mi_switch+0x16f/frame 0xfffffe009deae960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009deae9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009deae9d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe009deaea30 usb_process() at usb_process+0x12b/frame 0xfffffe009deaea70 fork_exit() at fork_exit+0x84/frame 0xfffffe009deaeab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009deaeab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 33 tid 100175 td 0xfffff80840dd3000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009deb3930 mi_switch() at mi_switch+0x16f/frame 0xfffffe009deb3960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009deb39a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009deb39d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe009deb3a30 usb_process() at usb_process+0x12b/frame 0xfffffe009deb3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009deb3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009deb3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 33 tid 100178 td 0xfffff80004929580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009def2930 mi_switch() at mi_switch+0x16f/frame 0xfffffe009def2960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009def29a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009def29d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe009def2a30 usb_process() at usb_process+0x12b/frame 0xfffffe009def2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009def2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009def2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 33 tid 100179 td 0xfffff80004929000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009def7930 mi_switch() at mi_switch+0x16f/frame 0xfffffe009def7960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009def79a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009def79d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe009def7a30 usb_process() at usb_process+0x12b/frame 0xfffffe009def7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009def7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009def7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 33 tid 100180 td 0xfffff80840e7f580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009defc930 mi_switch() at mi_switch+0x16f/frame 0xfffffe009defc960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009defc9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009defc9d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe009defca30 usb_process() at usb_process+0x12b/frame 0xfffffe009defca70 fork_exit() at fork_exit+0x84/frame 0xfffffe009defcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009defcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 33 tid 100181 td 0xfffff80840e7f000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009df01930 mi_switch() at mi_switch+0x16f/frame 0xfffffe009df01960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009df019a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009df019d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe009df01a30 usb_process() at usb_process+0x12b/frame 0xfffffe009df01a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009df01ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009df01ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 33 tid 100182 td 0xfffff80004928580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009df06930 mi_switch() at mi_switch+0x16f/frame 0xfffffe009df06960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009df069a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009df069d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe009df06a30 usb_process() at usb_process+0x12b/frame 0xfffffe009df06a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009df06ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009df06ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 33 tid 100184 td 0xfffff80840e7b580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009df10930 mi_switch() at mi_switch+0x16f/frame 0xfffffe009df10960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009df109a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009df109d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe009df10a30 usb_process() at usb_process+0x12b/frame 0xfffffe009df10a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009df10ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009df10ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 33 tid 100185 td 0xfffff80840e7b000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009df15930 mi_switch() at mi_switch+0x16f/frame 0xfffffe009df15960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009df159a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009df159d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe009df15a30 usb_process() at usb_process+0x12b/frame 0xfffffe009df15a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009df15ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009df15ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 33 tid 100186 td 0xfffff8000493d580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009df1a930 mi_switch() at mi_switch+0x16f/frame 0xfffffe009df1a960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009df1a9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009df1a9d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe009df1aa30 usb_process() at usb_process+0x12b/frame 0xfffffe009df1aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe009df1aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009df1aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 33 tid 100187 td 0xfffff8000493d000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009df1f930 mi_switch() at mi_switch+0x16f/frame 0xfffffe009df1f960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009df1f9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009df1f9d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe009df1fa30 usb_process() at usb_process+0x12b/frame 0xfffffe009df1fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe009df1fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009df1fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 33 tid 100188 td 0xfffff80840e90580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009df24930 mi_switch() at mi_switch+0x16f/frame 0xfffffe009df24960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009df249a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009df249d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe009df24a30 usb_process() at usb_process+0x12b/frame 0xfffffe009df24a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009df24ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009df24ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 32 tid 100163 td 0xfffff8084035c000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de778e0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de77910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de77950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de77980 _sleep() at _sleep+0x358/frame 0xfffffe009de77a20 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe009de77a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de77ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de77ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 32 tid 100164 td 0xfffff800038e4580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de7c8e0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de7c910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de7c950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de7c980 _sleep() at _sleep+0x358/frame 0xfffffe009de7ca20 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe009de7ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de7cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de7cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 32 tid 100165 td 0xfffff800038e4000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de818e0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de81910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de81950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de81980 _sleep() at _sleep+0x358/frame 0xfffffe009de81a20 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe009de81a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de81ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de81ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 32 tid 100166 td 0xfffff8084035b580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de868e0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de86910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de86950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de86980 _sleep() at _sleep+0x358/frame 0xfffffe009de86a20 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe009de86a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de86ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de86ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 32 tid 100167 td 0xfffff8084035b000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de8b8e0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de8b910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de8b950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de8b980 _sleep() at _sleep+0x358/frame 0xfffffe009de8ba20 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe009de8ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de8bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de8bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 32 tid 100203 td 0xfffff8084279b580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009df9f8d0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009df9f900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009df9f940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009df9f970 _sleep() at _sleep+0x358/frame 0xfffffe009df9fa10 xpt_scanner_thread() at xpt_scanner_thread+0x93/frame 0xfffffe009df9fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe009df9fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009df9fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sequencer 00 pid 31 tid 100162 td 0xfffff808402b1580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de728f0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de72920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de72960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de72990 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe009de729f0 seq_eventthread() at seq_eventthread+0xcc/frame 0xfffffe009de72a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de72ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de72ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 23 pid 30 tid 100161 td 0xfffff808402bb000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de6d8c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de6d8f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de6d930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de6d960 _sleep() at _sleep+0x358/frame 0xfffffe009de6da00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe009de6da70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de6dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de6dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 22 pid 29 tid 100160 td 0xfffff808402bb580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de688c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de688f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de68930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de68960 _sleep() at _sleep+0x358/frame 0xfffffe009de68a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe009de68a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de68ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de68ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 21 pid 28 tid 100159 td 0xfffff80003849000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de638c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de638f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de63930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de63960 _sleep() at _sleep+0x358/frame 0xfffffe009de63a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe009de63a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de63ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de63ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 20 pid 27 tid 100158 td 0xfffff80003849580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de5e8c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de5e8f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de5e930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de5e960 _sleep() at _sleep+0x358/frame 0xfffffe009de5ea00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe009de5ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de5eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de5eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 19 pid 26 tid 100157 td 0xfffff808402bc000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de598c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de598f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de59930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de59960 _sleep() at _sleep+0x358/frame 0xfffffe009de59a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe009de59a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de59ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de59ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 18 pid 25 tid 100156 td 0xfffff808402bc580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de548c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de548f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de54930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de54960 _sleep() at _sleep+0x358/frame 0xfffffe009de54a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe009de54a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de54ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de54ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 17 pid 24 tid 100155 td 0xfffff8000384a000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de4f8c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de4f8f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de4f930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de4f960 _sleep() at _sleep+0x358/frame 0xfffffe009de4fa00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe009de4fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de4fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de4fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 16 pid 23 tid 100154 td 0xfffff8000384a580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de4a8c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de4a8f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de4a930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de4a960 _sleep() at _sleep+0x358/frame 0xfffffe009de4aa00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe009de4aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de4aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de4aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 15 pid 22 tid 100153 td 0xfffff808402ad000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de458c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de458f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de45930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de45960 _sleep() at _sleep+0x358/frame 0xfffffe009de45a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe009de45a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de45ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de45ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 14 pid 21 tid 100152 td 0xfffff808402ad580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de408c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de408f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de40930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de40960 _sleep() at _sleep+0x358/frame 0xfffffe009de40a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe009de40a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de40ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de40ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 13 pid 20 tid 100151 td 0xfffff80003841000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de3b8c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de3b8f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de3b930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de3b960 _sleep() at _sleep+0x358/frame 0xfffffe009de3ba00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe009de3ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de3bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de3bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 12 pid 19 tid 100150 td 0xfffff80003841580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de368c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de368f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de36930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de36960 _sleep() at _sleep+0x358/frame 0xfffffe009de36a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe009de36a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de36ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de36ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 11 pid 18 tid 100149 td 0xfffff808402b0000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de318c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de318f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de31930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de31960 _sleep() at _sleep+0x358/frame 0xfffffe009de31a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe009de31a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de31ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de31ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 10 pid 17 tid 100148 td 0xfffff808402b0580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de2c8c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de2c8f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de2c930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de2c960 _sleep() at _sleep+0x358/frame 0xfffffe009de2ca00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe009de2ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de2cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de2cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 9 pid 16 tid 100147 td 0xfffff80003842000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de278c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de278f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de27930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de27960 _sleep() at _sleep+0x358/frame 0xfffffe009de27a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe009de27a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de27ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de27ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 8 pid 15 tid 100146 td 0xfffff80003842580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de228c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de228f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de22930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de22960 _sleep() at _sleep+0x358/frame 0xfffffe009de22a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe009de22a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de22ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de22ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 7 pid 14 tid 100145 td 0xfffff808402b1000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de1d8c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de1d8f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de1d930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de1d960 _sleep() at _sleep+0x358/frame 0xfffffe009de1da00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe009de1da70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de1dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de1dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 6 pid 9 tid 100144 td 0xfffff808402a3580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de188c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de188f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de18930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de18960 _sleep() at _sleep+0x358/frame 0xfffffe009de18a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe009de18a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de18ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de18ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 5 pid 8 tid 100143 td 0xfffff808402a4000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de138c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de138f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de13930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de13960 _sleep() at _sleep+0x358/frame 0xfffffe009de13a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe009de13a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de13ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de13ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 4 pid 7 tid 100142 td 0xfffff808402a4580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de0e8c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de0e8f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de0e930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de0e960 _sleep() at _sleep+0x358/frame 0xfffffe009de0ea00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe009de0ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de0eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de0eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 3 pid 6 tid 100141 td 0xfffff80003832000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de098c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de098f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de09930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de09960 _sleep() at _sleep+0x358/frame 0xfffffe009de09a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe009de09a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de09ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de09ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 2 pid 5 tid 100140 td 0xfffff80003832580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de048c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de048f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009de04930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009de04960 _sleep() at _sleep+0x358/frame 0xfffffe009de04a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe009de04a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de04ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de04ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 1 pid 4 tid 100139 td 0xfffff808402a5000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006fff8c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006fff8f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006fff930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006fff960 _sleep() at _sleep+0x358/frame 0xfffffe0006fffa00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006fffa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006fffab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006fffab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 0 pid 3 tid 100138 td 0xfffff808402a5580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006ffa8c0 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006ffa8f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006ffa930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006ffa960 _sleep() at _sleep+0x358/frame 0xfffffe0006ffaa00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006ffaa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ffaab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ffaab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 2 tid 100137 td 0xfffff80003833000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006ff58f0 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006ff5920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006ff5960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006ff5990 _sleep() at _sleep+0x358/frame 0xfffffe0006ff5a30 crypto_proc() at crypto_proc+0x2c5/frame 0xfffffe0006ff5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ff5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ff5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100109 td 0xfffff80840244000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006f698f0 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006f69920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006f69960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f69990 _sleep() at _sleep+0x358/frame 0xfffffe0006f69a30 g_run_events() at g_run_events+0x4f/frame 0xfffffe0006f69a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f69ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f69ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100110 td 0xfffff800037d9580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006f6e8f0 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006f6e920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006f6e960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f6e990 _sleep() at _sleep+0x358/frame 0xfffffe0006f6ea30 g_io_schedule_up() at g_io_schedule_up+0xa5/frame 0xfffffe0006f6ea60 g_up_procbody() at g_up_procbody+0x6d/frame 0xfffffe0006f6ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f6eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f6eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100111 td 0xfffff800037d9000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006f738e0 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006f73910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006f73950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f73980 _sleep() at _sleep+0x358/frame 0xfffffe0006f73a20 g_io_schedule_down() at g_io_schedule_down+0x14c/frame 0xfffffe0006f73a60 g_down_procbody() at g_down_procbody+0x6d/frame 0xfffffe0006f73a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f73ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f73ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100027 td 0xfffff80840100000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00787d29e0 mi_switch() at mi_switch+0x16f/frame 0xfffffe00787d2a10 ithread_loop() at ithread_loop+0x313/frame 0xfffffe00787d2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787d2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787d2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100028 td 0xfffff800036b7580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00787d79e0 mi_switch() at mi_switch+0x16f/frame 0xfffffe00787d7a10 ithread_loop() at ithread_loop+0x313/frame 0xfffffe00787d7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787d7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787d7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100030 td 0xfffff80840120580 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100083 td 0xfffff80003766000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100084 td 0xfffff808401d0580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006e2c9e0 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006e2ca10 ithread_loop() at ithread_loop+0x313/frame 0xfffffe0006e2ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e2cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e2cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100085 td 0xfffff808401d0000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006e319e0 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006e31a10 ithread_loop() at ithread_loop+0x313/frame 0xfffffe0006e31a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e31ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e31ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100086 td 0xfffff80003765580 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100087 td 0xfffff80003765000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100088 td 0xfffff808401cf580 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100089 td 0xfffff808401cf000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100090 td 0xfffff808401c3580 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100091 td 0xfffff808401d5000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100092 td 0xfffff8000376d580 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100093 td 0xfffff8000376d000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100094 td 0xfffff808401d2580 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100095 td 0xfffff808401d2000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100096 td 0xfffff8000376a580 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100097 td 0xfffff8000376a000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100098 td 0xfffff808401d1580 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100099 td 0xfffff808401d1000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100100 td 0xfffff80003770580 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100101 td 0xfffff80003770000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100102 td 0xfffff808401d8580 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100103 td 0xfffff808401d8000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100104 td 0xfffff8000376f580 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100105 td 0xfffff8000376f000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100106 td 0xfffff808401d7580 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100107 td 0xfffff808401d7000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100108 td 0xfffff808401d5580 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100168 td 0xfffff8000436f580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de909e0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de90a10 ithread_loop() at ithread_loop+0x313/frame 0xfffffe009de90a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de90ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de90ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100169 td 0xfffff8000436f000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100170 td 0xfffff80840dd7580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009de9a9e0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009de9aa10 ithread_loop() at ithread_loop+0x313/frame 0xfffffe009de9aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe009de9aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009de9aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100176 td 0xfffff8084035c580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009deb89e0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009deb8a10 ithread_loop() at ithread_loop+0x313/frame 0xfffffe009deb8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009deb8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009deb8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100177 td 0xfffff80840e80000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100183 td 0xfffff80004928000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009df0b9e0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009df0ba10 ithread_loop() at ithread_loop+0x313/frame 0xfffffe009df0ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe009df0bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009df0bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100189 td 0xfffff80840e90000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009df299e0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009df29a10 ithread_loop() at ithread_loop+0x313/frame 0xfffffe009df29a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009df29ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009df29ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100190 td 0xfffff8000493c580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009df2e9e0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009df2ea10 ithread_loop() at ithread_loop+0x313/frame 0xfffffe009df2ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe009df2eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009df2eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100191 td 0xfffff8000493c000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009df339e0 mi_switch() at mi_switch+0x16f/frame 0xfffffe009df33a10 ithread_loop() at ithread_loop+0x313/frame 0xfffffe009df33a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009df33ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009df33ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100003 td 0xfffff80003697000 (CPU 0) cpustop_handler() at cpustop_handler+0x28/frame 0xffffffff82058d30 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xffffffff82058d50 trap() at trap+0x49/frame 0xffffffff82058e60 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff82058e60 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe007875a900, rbp = 0xfffffe007875a910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe007875a910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe007875a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007875a980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe007875a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007875aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007875aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007875aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xfffff808400f5580 (CPU 1) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006bacdf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006bace10 trap() at trap+0x49/frame 0xfffffe0006bacf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bacf20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe007875f900, rbp = 0xfffffe007875f910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe007875f910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe007875f960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007875f980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe007875f9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007875fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007875fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007875fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffff808400f5000 (CPU 2) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006bb8df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006bb8e10 trap() at trap+0x49/frame 0xfffffe0006bb8f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bb8f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe0078764900, rbp = 0xfffffe0078764910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe0078764910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe0078764960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078764980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787649a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078764a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078764ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078764ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xfffff80003696580 (CPU 3) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006bc4df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006bc4e10 trap() at trap+0x49/frame 0xfffffe0006bc4f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bc4f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe0078769900, rbp = 0xfffffe0078769910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe0078769910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe0078769960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078769980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787699a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078769a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078769ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078769ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100007 td 0xfffff80003696000 (CPU 4) kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007876e730 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007876e740 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007876e770 uart_intr() at uart_intr+0x138/frame 0xfffffe007876e7b0 intr_event_handle() at intr_event_handle+0xcb/frame 0xfffffe007876e800 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007876e830 lapic_handle_intr() at lapic_handle_intr+0x5f/frame 0xfffffe007876e850 Xapic_isr1() at Xapic_isr1+0xd9/frame 0xfffffe007876e850 --- interrupt, rip = 0xffffffff8046209c, rsp = 0xfffffe007876e920, rbp = 0xfffffe007876e960 --- acpi_cpu_idle() at acpi_cpu_idle+0x34c/frame 0xfffffe007876e960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007876e980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe007876e9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007876ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe007876eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007876eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xfffff808400f4580 (CPU 5) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006bdcdf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006bdce10 trap() at trap+0x49/frame 0xfffffe0006bdcf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bdcf20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe0078773900, rbp = 0xfffffe0078773910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe0078773910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe0078773960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078773980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787739a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078773a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078773ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078773ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xfffff808400f4000 (CPU 6) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006be8df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006be8e10 trap() at trap+0x49/frame 0xfffffe0006be8f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006be8f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe0078778900, rbp = 0xfffffe0078778910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe0078778910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe0078778960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078778980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787789a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078778a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078778ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078778ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xfffff800036a1580 (CPU 7) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006bf4df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006bf4e10 trap() at trap+0x49/frame 0xfffffe0006bf4f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bf4f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe007877d900, rbp = 0xfffffe007877d910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe007877d910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe007877d960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007877d980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe007877d9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007877da70 fork_exit() at fork_exit+0x84/frame 0xfffffe007877dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007877dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffff800036a1000 (CPU 8) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0000269df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0000269e10 trap() at trap+0x49/frame 0xfffffe0000269f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0000269f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe0078782900, rbp = 0xfffffe0078782910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe0078782910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe0078782960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078782980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787829a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078782a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078782ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078782ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100012 td 0xfffff808400ff580 (CPU 9) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0000275df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0000275e10 trap() at trap+0x49/frame 0xfffffe0000275f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0000275f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe0078787900, rbp = 0xfffffe0078787910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe0078787910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe0078787960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078787980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787879a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078787a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078787ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078787ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xfffff808400ff000 (CPU 10) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0000281df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0000281e10 trap() at trap+0x49/frame 0xfffffe0000281f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0000281f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe007878c900, rbp = 0xfffffe007878c910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe007878c910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe007878c960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007878c980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe007878c9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007878ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe007878cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007878cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100014 td 0xfffff800036a0580 (CPU 11) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe000028ddf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe000028de10 trap() at trap+0x49/frame 0xfffffe000028df20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe000028df20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe0078791900, rbp = 0xfffffe0078791910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe0078791910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe0078791960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078791980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787919a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078791a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078791ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078791ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100015 td 0xfffff800036a0000 (CPU 12) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0000299df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0000299e10 trap() at trap+0x49/frame 0xfffffe0000299f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0000299f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe0078796900, rbp = 0xfffffe0078796910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe0078796910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe0078796960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078796980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787969a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078796a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078796ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078796ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100016 td 0xfffff808400fe580 (CPU 13) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00002a5df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00002a5e10 trap() at trap+0x49/frame 0xfffffe00002a5f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00002a5f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe007879b900, rbp = 0xfffffe007879b910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe007879b910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe007879b960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007879b980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe007879b9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007879ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe007879bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007879bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100017 td 0xfffff808400fe000 (CPU 14) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00002b1df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00002b1e10 trap() at trap+0x49/frame 0xfffffe00002b1f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00002b1f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe00787a0900, rbp = 0xfffffe00787a0910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe00787a0910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe00787a0960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe00787a0980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787a09a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe00787a0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787a0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787a0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100018 td 0xfffff808400f6580 (CPU 15) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00002bddf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00002bde10 trap() at trap+0x49/frame 0xfffffe00002bdf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00002bdf20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe00787a5900, rbp = 0xfffffe00787a5910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe00787a5910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe00787a5960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe00787a5980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787a59a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe00787a5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787a5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787a5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100019 td 0xfffff80840102000 (CPU 16) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00002c9df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00002c9e10 trap() at trap+0x49/frame 0xfffffe00002c9f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00002c9f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe00787aa900, rbp = 0xfffffe00787aa910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe00787aa910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe00787aa960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe00787aa980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787aa9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe00787aaa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787aaab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787aaab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100020 td 0xfffff800036a3580 (CPU 17) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00002d5df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00002d5e10 trap() at trap+0x49/frame 0xfffffe00002d5f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00002d5f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe00787af900, rbp = 0xfffffe00787af910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe00787af910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe00787af960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe00787af980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787af9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe00787afa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787afab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787afab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100021 td 0xfffff800036a3000 (CPU 18) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00002e1df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00002e1e10 trap() at trap+0x49/frame 0xfffffe00002e1f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00002e1f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe00787b4900, rbp = 0xfffffe00787b4910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe00787b4910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe00787b4960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe00787b4980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787b49a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe00787b4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787b4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787b4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100022 td 0xfffff80840101580 (CPU 19) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00002eddf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00002ede10 trap() at trap+0x49/frame 0xfffffe00002edf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00002edf20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe00787b9900, rbp = 0xfffffe00787b9910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe00787b9910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe00787b9960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe00787b9980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787b99a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe00787b9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787b9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787b9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100023 td 0xfffff80840101000 (CPU 20) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00002f9df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00002f9e10 trap() at trap+0x49/frame 0xfffffe00002f9f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00002f9f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe00787be900, rbp = 0xfffffe00787be910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe00787be910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe00787be960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe00787be980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787be9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe00787bea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787beab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787beab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100024 td 0xfffff800036a2580 (CPU 21) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0000305df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0000305e10 trap() at trap+0x49/frame 0xfffffe0000305f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0000305f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe00787c3900, rbp = 0xfffffe00787c3910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe00787c3910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe00787c3960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe00787c3980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787c39a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe00787c3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787c3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787c3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100025 td 0xfffff800036a2000 (CPU 22) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0000311df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0000311e10 trap() at trap+0x49/frame 0xfffffe0000311f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0000311f20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe00787c8900, rbp = 0xfffffe00787c8910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe00787c8910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe00787c8960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe00787c8980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787c89a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe00787c8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787c8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787c8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100026 td 0xfffff80840100580 (CPU 23) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe000031ddf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe000031de10 trap() at trap+0x49/frame 0xfffffe000031df20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe000031df20 --- trap 0x13, rip = 0xffffffff811cb5d8, rsp = 0xfffffe00787cd900, rbp = 0xfffffe00787cd910 --- acpi_cpu_idle_mwait() at acpi_cpu_idle_mwait+0x78/frame 0xfffffe00787cd910 acpi_cpu_idle() at acpi_cpu_idle+0x203/frame 0xfffffe00787cd960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe00787cd980 cpu_idle() at cpu_idle+0xa7/frame 0xfffffe00787cd9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe00787cda70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787cdab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787cdab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command init pid 1 tid 100002 td 0xfffff80003697580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0078755560 mi_switch() at mi_switch+0x16f/frame 0xfffffe0078755590 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00787555d0 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe0078755630 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0078755660 _sleep() at _sleep+0x34e/frame 0xfffffe0078755700 kern_wait6() at kern_wait6+0x4af/frame 0xfffffe0078755790 sys_wait4() at sys_wait4+0x78/frame 0xfffffe0078755980 amd64_syscall() at amd64_syscall+0x2a7/frame 0xfffffe0078755ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0078755ab0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x2ae76a, rsp = 0x7fffffffe788, rbp = 0x7fffffffe7f0 --- Tracing command audit pid 10 tid 100001 td 0xfffff808400f6000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0078750910 mi_switch() at mi_switch+0x16f/frame 0xfffffe0078750940 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0078750980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00787509b0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe0078750a10 audit_worker() at audit_worker+0x93/frame 0xfffffe0078750a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078750ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078750ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xffffffff8208d860 sched_switch() at sched_switch+0x7ca/frame 0xffffffff8270b9d0 mi_switch() at mi_switch+0x16f/frame 0xffffffff8270ba00 sleepq_switch() at sleepq_switch+0x10d/frame 0xffffffff8270ba40 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xffffffff8270ba80 _sleep() at _sleep+0x2fc/frame 0xffffffff8270bb20 swapper() at swapper+0x77/frame 0xffffffff8270bb70 btext() at btext+0x2c Tracing command kernel pid 0 tid 100029 td 0xfffff800036b7000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00787dc900 mi_switch() at mi_switch+0x16f/frame 0xfffffe00787dc930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00787dc970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00787dc9a0 _sleep() at _sleep+0x358/frame 0xfffffe00787dca40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00787dca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787dcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787dcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100031 td 0xfffff80840120000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00787e6930 mi_switch() at mi_switch+0x16f/frame 0xfffffe00787e6960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00787e69a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00787e69d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe00787e6a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00787e6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787e6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787e6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100032 td 0xfffff800036b6580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00787eb900 mi_switch() at mi_switch+0x16f/frame 0xfffffe00787eb930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00787eb970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00787eb9a0 _sleep() at _sleep+0x358/frame 0xfffffe00787eba40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00787eba70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787ebab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787ebab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100033 td 0xfffff800036b6000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00787f0900 mi_switch() at mi_switch+0x16f/frame 0xfffffe00787f0930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00787f0970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00787f09a0 _sleep() at _sleep+0x358/frame 0xfffffe00787f0a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00787f0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787f0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787f0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100034 td 0xfffff8084011f580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00787f5930 mi_switch() at mi_switch+0x16f/frame 0xfffffe00787f5960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00787f59a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00787f59d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe00787f5a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00787f5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787f5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787f5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100035 td 0xfffff8084011f000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00787fa930 mi_switch() at mi_switch+0x16f/frame 0xfffffe00787fa960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00787fa9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00787fa9d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe00787faa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00787faa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787faab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787faab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100036 td 0xfffff80840102580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00787ff930 mi_switch() at mi_switch+0x16f/frame 0xfffffe00787ff960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00787ff9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00787ff9d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe00787ffa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00787ffa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00787ffab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00787ffab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100037 td 0xfffff8084015b000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe000053d930 mi_switch() at mi_switch+0x16f/frame 0xfffffe000053d960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000053d9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000053d9d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe000053da40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000053da70 fork_exit() at fork_exit+0x84/frame 0xfffffe000053dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000053dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100038 td 0xfffff800036ed580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0000542930 mi_switch() at mi_switch+0x16f/frame 0xfffffe0000542960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005429a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005429d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe0000542a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000542a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000542ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000542ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100039 td 0xfffff800036ed000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0000547930 mi_switch() at mi_switch+0x16f/frame 0xfffffe0000547960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005479a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005479d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe0000547a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000547a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000547ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000547ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100040 td 0xfffff8084015a580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe000054c930 mi_switch() at mi_switch+0x16f/frame 0xfffffe000054c960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000054c9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000054c9d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe000054ca40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000054ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe000054cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000054cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100041 td 0xfffff8084015a000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0000551930 mi_switch() at mi_switch+0x16f/frame 0xfffffe0000551960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005519a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005519d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe0000551a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000551a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000551ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000551ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100042 td 0xfffff800036ec580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0000556930 mi_switch() at mi_switch+0x16f/frame 0xfffffe0000556960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005569a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005569d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe0000556a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000556a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000556ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000556ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100043 td 0xfffff800036ec000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe000055b930 mi_switch() at mi_switch+0x16f/frame 0xfffffe000055b960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000055b9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000055b9d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe000055ba40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000055ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe000055bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000055bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100044 td 0xfffff80840157580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0000560930 mi_switch() at mi_switch+0x16f/frame 0xfffffe0000560960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005609a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005609d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe0000560a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000560a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000560ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000560ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100045 td 0xfffff80840157000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0000565930 mi_switch() at mi_switch+0x16f/frame 0xfffffe0000565960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005659a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005659d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe0000565a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000565a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000565ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000565ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100046 td 0xfffff800036ef580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe000056a930 mi_switch() at mi_switch+0x16f/frame 0xfffffe000056a960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000056a9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000056a9d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe000056aa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000056aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe000056aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000056aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100047 td 0xfffff800036ef000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe000056f930 mi_switch() at mi_switch+0x16f/frame 0xfffffe000056f960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000056f9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000056f9d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe000056fa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000056fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe000056fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000056fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100048 td 0xfffff8084015d580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0000574930 mi_switch() at mi_switch+0x16f/frame 0xfffffe0000574960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005749a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005749d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe0000574a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000574a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000574ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000574ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100049 td 0xfffff8084015d000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0000579930 mi_switch() at mi_switch+0x16f/frame 0xfffffe0000579960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005799a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005799d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe0000579a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000579a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000579ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000579ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100050 td 0xfffff800036ee580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe000057e930 mi_switch() at mi_switch+0x16f/frame 0xfffffe000057e960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000057e9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000057e9d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe000057ea40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000057ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe000057eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000057eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100051 td 0xfffff800036ee000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0000583930 mi_switch() at mi_switch+0x16f/frame 0xfffffe0000583960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005839a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005839d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe0000583a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000583a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000583ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000583ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100052 td 0xfffff8084015c580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0000588930 mi_switch() at mi_switch+0x16f/frame 0xfffffe0000588960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005889a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005889d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe0000588a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000588a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000588ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000588ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100053 td 0xfffff8084015c000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe000058d930 mi_switch() at mi_switch+0x16f/frame 0xfffffe000058d960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000058d9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000058d9d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe000058da40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000058da70 fork_exit() at fork_exit+0x84/frame 0xfffffe000058dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000058dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100054 td 0xfffff8084015b580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0000592930 mi_switch() at mi_switch+0x16f/frame 0xfffffe0000592960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005929a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005929d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe0000592a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000592a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000592ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000592ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100055 td 0xfffff80840161000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0000597930 mi_switch() at mi_switch+0x16f/frame 0xfffffe0000597960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005979a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005979d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe0000597a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000597a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000597ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000597ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100056 td 0xfffff800036f1580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe000059c930 mi_switch() at mi_switch+0x16f/frame 0xfffffe000059c960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000059c9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000059c9d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe000059ca40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000059ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe000059cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000059cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100057 td 0xfffff800036f1000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00005a1930 mi_switch() at mi_switch+0x16f/frame 0xfffffe00005a1960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005a19a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005a19d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe00005a1a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00005a1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005a1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005a1ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100058 td 0xfffff8084015f580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00005a6930 mi_switch() at mi_switch+0x16f/frame 0xfffffe00005a6960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005a69a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005a69d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe00005a6a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00005a6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005a6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005a6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100059 td 0xfffff8084015f000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00005ab930 mi_switch() at mi_switch+0x16f/frame 0xfffffe00005ab960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005ab9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005ab9d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe00005aba40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00005aba70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005abab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005abab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100060 td 0xfffff800036f0580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00005b0930 mi_switch() at mi_switch+0x16f/frame 0xfffffe00005b0960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005b09a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005b09d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe00005b0a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00005b0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005b0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005b0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100061 td 0xfffff800036f0000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00005b5930 mi_switch() at mi_switch+0x16f/frame 0xfffffe00005b5960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005b59a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005b59d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe00005b5a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00005b5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005b5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005b5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100062 td 0xfffff8084015e580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00005ba930 mi_switch() at mi_switch+0x16f/frame 0xfffffe00005ba960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005ba9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005ba9d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe00005baa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00005baa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005baab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005baab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100063 td 0xfffff8084015e000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00005bf930 mi_switch() at mi_switch+0x16f/frame 0xfffffe00005bf960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005bf9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005bf9d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe00005bfa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00005bfa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005bfab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005bfab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100064 td 0xfffff8000373f580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00005c4930 mi_switch() at mi_switch+0x16f/frame 0xfffffe00005c4960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005c49a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005c49d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe00005c4a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00005c4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005c4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005c4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100065 td 0xfffff8000373f000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00005c9930 mi_switch() at mi_switch+0x16f/frame 0xfffffe00005c9960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005c99a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005c99d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe00005c9a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00005c9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005c9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005c9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100066 td 0xfffff808401ae580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00005ce930 mi_switch() at mi_switch+0x16f/frame 0xfffffe00005ce960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005ce9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005ce9d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe00005cea40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00005cea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005ceab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005ceab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100067 td 0xfffff808401ae000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00005d3930 mi_switch() at mi_switch+0x16f/frame 0xfffffe00005d3960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005d39a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005d39d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe00005d3a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00005d3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005d3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005d3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100068 td 0xfffff8000373c580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00005d8930 mi_switch() at mi_switch+0x16f/frame 0xfffffe00005d8960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005d89a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005d89d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe00005d8a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00005d8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005d8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005d8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100069 td 0xfffff8000373c000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00005dd930 mi_switch() at mi_switch+0x16f/frame 0xfffffe00005dd960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005dd9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005dd9d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe00005dda40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00005dda70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005ddab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005ddab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100070 td 0xfffff808401ab580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00005e2930 mi_switch() at mi_switch+0x16f/frame 0xfffffe00005e2960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005e29a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005e29d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe00005e2a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00005e2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005e2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005e2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100071 td 0xfffff808401ab000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00005e7930 mi_switch() at mi_switch+0x16f/frame 0xfffffe00005e7960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005e79a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005e79d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe00005e7a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00005e7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005e7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005e7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100072 td 0xfffff80840161580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00005ec930 mi_switch() at mi_switch+0x16f/frame 0xfffffe00005ec960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005ec9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005ec9d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe00005eca40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00005eca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005ecab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005ecab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100073 td 0xfffff808401c3000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00005f1930 mi_switch() at mi_switch+0x16f/frame 0xfffffe00005f1960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005f19a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005f19d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe00005f1a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00005f1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005f1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005f1ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100074 td 0xfffff80003756580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00005f6930 mi_switch() at mi_switch+0x16f/frame 0xfffffe00005f6960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005f69a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005f69d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe00005f6a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00005f6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005f6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005f6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100075 td 0xfffff80003756000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00005fb930 mi_switch() at mi_switch+0x16f/frame 0xfffffe00005fb960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005fb9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005fb9d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe00005fba40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00005fba70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005fbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005fbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100076 td 0xfffff808401c0580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006e04930 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006e04960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e049a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e049d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe0006e04a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0006e04a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e04ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e04ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100077 td 0xfffff808401c0000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006e09930 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006e09960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e099a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e099d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe0006e09a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0006e09a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e09ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e09ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100078 td 0xfffff80003753580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006e0e930 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006e0e960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e0e9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e0e9d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe0006e0ea40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0006e0ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e0eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e0eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100079 td 0xfffff80003753000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006e13930 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006e13960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e139a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e139d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe0006e13a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0006e13a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e13ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e13ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100080 td 0xfffff808401be580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006e18930 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006e18960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e189a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e189d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe0006e18a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0006e18a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e18ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e18ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100081 td 0xfffff808401be000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006e1d930 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006e1d960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e1d9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e1d9d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe0006e1da40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0006e1da70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e1dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e1dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100082 td 0xfffff80003766580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006e22930 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006e22960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e229a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e229d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe0006e22a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0006e22a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e22ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e22ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100112 td 0xfffff80840243580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006f78900 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006f78930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006f78970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f789a0 _sleep() at _sleep+0x358/frame 0xfffffe0006f78a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006f78a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f78ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f78ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100113 td 0xfffff80840243000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006f7d900 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006f7d930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006f7d970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f7d9a0 _sleep() at _sleep+0x358/frame 0xfffffe0006f7da40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006f7da70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f7dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f7dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100114 td 0xfffff800037d8580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006f82900 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006f82930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006f82970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f829a0 _sleep() at _sleep+0x358/frame 0xfffffe0006f82a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006f82a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f82ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f82ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100115 td 0xfffff800037d8000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006f87900 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006f87930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006f87970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f879a0 _sleep() at _sleep+0x358/frame 0xfffffe0006f87a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006f87a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f87ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f87ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100116 td 0xfffff80840242580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006f8c900 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006f8c930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006f8c970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f8c9a0 _sleep() at _sleep+0x358/frame 0xfffffe0006f8ca40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006f8ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f8cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f8cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100117 td 0xfffff80840242000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006f91900 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006f91930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006f91970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f919a0 _sleep() at _sleep+0x358/frame 0xfffffe0006f91a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006f91a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f91ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f91ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100118 td 0xfffff80003829580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006f96900 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006f96930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006f96970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f969a0 _sleep() at _sleep+0x358/frame 0xfffffe0006f96a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006f96a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f96ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f96ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100119 td 0xfffff80003829000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006f9b900 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006f9b930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006f9b970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f9b9a0 _sleep() at _sleep+0x358/frame 0xfffffe0006f9ba40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006f9ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f9bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f9bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100120 td 0xfffff8084029a580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006fa0900 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006fa0930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006fa0970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006fa09a0 _sleep() at _sleep+0x358/frame 0xfffffe0006fa0a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006fa0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006fa0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006fa0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100121 td 0xfffff8084029a000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006fa5900 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006fa5930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006fa5970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006fa59a0 _sleep() at _sleep+0x358/frame 0xfffffe0006fa5a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006fa5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006fa5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006fa5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100122 td 0xfffff80003826580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006faa900 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006faa930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006faa970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006faa9a0 _sleep() at _sleep+0x358/frame 0xfffffe0006faaa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006faaa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006faaab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006faaab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100123 td 0xfffff80003826000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006faf900 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006faf930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006faf970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006faf9a0 _sleep() at _sleep+0x358/frame 0xfffffe0006fafa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006fafa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006fafab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006fafab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100124 td 0xfffff80840298580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006fb4900 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006fb4930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006fb4970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006fb49a0 _sleep() at _sleep+0x358/frame 0xfffffe0006fb4a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006fb4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006fb4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006fb4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100125 td 0xfffff80840298000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006fb9900 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006fb9930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006fb9970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006fb99a0 _sleep() at _sleep+0x358/frame 0xfffffe0006fb9a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006fb9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006fb9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006fb9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100126 td 0xfffff80840244580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006fbe900 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006fbe930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006fbe970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006fbe9a0 _sleep() at _sleep+0x358/frame 0xfffffe0006fbea40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006fbea70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006fbeab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006fbeab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100127 td 0xfffff808402a3000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006fc3900 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006fc3930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006fc3970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006fc39a0 _sleep() at _sleep+0x358/frame 0xfffffe0006fc3a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006fc3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006fc3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006fc3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100128 td 0xfffff80003831580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006fc8900 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006fc8930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006fc8970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006fc89a0 _sleep() at _sleep+0x358/frame 0xfffffe0006fc8a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006fc8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006fc8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006fc8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100129 td 0xfffff80003831000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006fcd900 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006fcd930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006fcd970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006fcd9a0 _sleep() at _sleep+0x358/frame 0xfffffe0006fcda40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006fcda70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006fcdab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006fcdab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100130 td 0xfffff808402a2580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006fd2900 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006fd2930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006fd2970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006fd29a0 _sleep() at _sleep+0x358/frame 0xfffffe0006fd2a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006fd2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006fd2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006fd2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100131 td 0xfffff808402a2000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006fd7900 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006fd7930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006fd7970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006fd79a0 _sleep() at _sleep+0x358/frame 0xfffffe0006fd7a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006fd7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006fd7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006fd7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100132 td 0xfffff80003830580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006fdc900 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006fdc930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006fdc970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006fdc9a0 _sleep() at _sleep+0x358/frame 0xfffffe0006fdca40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006fdca70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006fdcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006fdcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100133 td 0xfffff80003830000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006fe1900 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006fe1930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006fe1970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006fe19a0 _sleep() at _sleep+0x358/frame 0xfffffe0006fe1a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006fe1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006fe1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006fe1ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100134 td 0xfffff808402a1580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006fe6900 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006fe6930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006fe6970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006fe69a0 _sleep() at _sleep+0x358/frame 0xfffffe0006fe6a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006fe6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006fe6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006fe6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100135 td 0xfffff808402a1000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006feb900 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006feb930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006feb970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006feb9a0 _sleep() at _sleep+0x358/frame 0xfffffe0006feba40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006feba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006febab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006febab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100136 td 0xfffff80003833580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe0006ff0900 mi_switch() at mi_switch+0x16f/frame 0xfffffe0006ff0930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006ff0970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006ff09a0 _sleep() at _sleep+0x358/frame 0xfffffe0006ff0a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006ff0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ff0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ff0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100192 td 0xfffff80840e8f580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009df38930 mi_switch() at mi_switch+0x16f/frame 0xfffffe009df38960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009df389a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009df389d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe009df38a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe009df38a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009df38ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009df38ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100198 td 0xfffff80007afc000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009df86930 mi_switch() at mi_switch+0x16f/frame 0xfffffe009df86960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009df869a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009df869d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe009df86a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe009df86a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009df86ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009df86ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100199 td 0xfffff8084279f580 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009df8b930 mi_switch() at mi_switch+0x16f/frame 0xfffffe009df8b960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009df8b9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009df8b9d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe009df8ba40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe009df8ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe009df8bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009df8bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100200 td 0xfffff8084279f000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009df90930 mi_switch() at mi_switch+0x16f/frame 0xfffffe009df90960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009df909a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009df909d0 msleep_spin_sbt() at msleep_spin_sbt+0x1b3/frame 0xfffffe009df90a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe009df90a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009df90ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009df90ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100202 td 0xfffff80007afb000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe009df9a900 mi_switch() at mi_switch+0x16f/frame 0xfffffe009df9a930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009df9a970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009df9a9a0 _sleep() at _sleep+0x358/frame 0xfffffe009df9aa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe009df9aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe009df9aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009df9aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100242 td 0xfffff80842dfe000 sched_switch() at sched_switch+0x7ca/frame 0xfffffe00a2c63900 mi_switch() at mi_switch+0x16f/frame 0xfffffe00a2c63930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a2c63970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a2c639a0 _sleep() at _sleep+0x358/frame 0xfffffe00a2c63a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00a2c63a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a2c63ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a2c63ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db:0:pho> dump Dumping 2796 out of 34701 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Dump complete db> reset cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 4 (kgdb) proc 95680 [Switching to thread 265 (Thread 102448)]#0 sched_switch (td=0xfffff8086e133580, newtd=0xfffff80840100580, flags=) at ../../../kern/sched_ule.c:2112 2112 cpuid = PCPU_GET(cpuid); Current language: auto; currently minimal (kgdb) bt #0 sched_switch (td=0xfffff8086e133580, newtd=0xfffff80840100580, flags=) at ../../../kern/sched_ule.c:2112 #1 0xffffffff80b9bb2f in mi_switch (flags=0x104, newtd=0x0) at ../../../kern/kern_synch.c:439 #2 0xffffffff80be6aed in sleepq_switch (wchan=0xffffffff81a47ac0, pri=0x54) at ../../../kern/subr_sleepqueue.c:613 #3 0xffffffff80be6993 in sleepq_wait (wchan=0xffffffff81a47ac0, pri=0x54) at ../../../kern/subr_sleepqueue.c:692 #4 0xffffffff80b9b3f8 in _sleep (ident=0xffffffff81a47ac0, lock=, priority=0x254, wmesg=0xffffffff8127293b "vmwait", sbt=, pr=0x0, flags=0x100) at ../../../kern/kern_synch.c:217 #5 0xffffffff80ee7192 in vm_wait_doms (wdoms=0xfffffe028dc4d010) at vmmeter.h:232 #6 0xffffffff80ee7228 in vm_wait_domain (domain=) at ../../../vm/vm_page.c:2997 #7 0xffffffff80ee4ff3 in vm_domain_alloc_fail (vmd=0xffffffff820b2440, object=, req=0x10232) at ../../../vm/vm_page.c:3049 #8 0xffffffff80ee4b14 in vm_page_alloc_domain_after (object=, pindex=0x0, domain=, req=0x10232, mpred=0x0) at ../../../vm/vm_page.c:1868 #9 0xffffffff80ee8a64 in vm_page_grab_pages (object=0xfffff8035ddf8800, pindex=0x0, allocflags=, ma=0xfffffe00011aceb8, count=) at ../../../vm/vm_page.c:1752 #10 0xffffffff80c43030 in allocbuf (bp=0xfffffe00011acd10, size=) at ../../../kern/vfs_bio.c:3077 #11 0xffffffff80c41185 in getblkx (vp=, blkno=, size=, slpflag=0x0, slptimeo=0x0, flags=0x8, bpp=0xfffffe028dc4d2a8) at ../../../kern/vfs_bio.c:4135 #12 0xffffffff80c43765 in getblk (vp=, blkno=, size=, slpflag=, slptimeo=, flags=) at ../../../kern/vfs_bio.c:3837 #13 0xffffffff80e8032d in ffs_balloc_ufs2 (vp=0xfffff803369d0000, startoffset=, size=, cred=0xfffff8003ca77600, flags=, bpp=0xfffffe028dc4d4b8) at ../../../ufs/ffs/ffs_balloc.c:806 #14 0xffffffff80eaa93e in ffs_write (ap=0xfffffe028dc4d620) at ../../../ufs/ffs/ffs_vnops.c:772 #15 0xffffffff811e6a7f in VOP_WRITE_APV (vop=0xffffffff81b63e80, a=0xfffffe028dc4d620) at vnode_if.c:1000 #16 0xffffffff80c76f93 in vn_write (fp=0xfffff80007f500f0, uio=, active_cred=0xfffff8003ca77600, flags=, td=) at vnode_if.h:413 #17 0xffffffff80c76953 in vn_io_fault_doio (args=0xfffffe028dc4d840, uio=0xfffffe028dc4d940, td=0xfffff8086e133580) at ../../../kern/vfs_vnops.c:946 #18 0xffffffff80c749f1 in vn_io_fault1 () at ../../../kern/vfs_vnops.c:1064 #19 0xffffffff80c72dd5 in vn_io_fault (fp=, uio=0xfffff803369d0220, active_cred=0xfffff8000b669078, flags=, td=) at ../../../kern/vfs_vnops.c:1168 #20 0xffffffff80bfd2f7 in dofilewrite (td=0xfffff8086e133580, fd=0x3, fp=0xfffff80007f500f0, auio=0xfffffe028dc4d940, offset=, flags=0x0) at file.h:314 #21 0xffffffff80bfcf1a in kern_writev (td=0xfffff8086e133580, fd=0x3, auio=0xfffffe028dc4d940) at ../../../kern/sys_generic.c:491 #22 0xffffffff80bfcec6 in sys_write (td=, uap=) at ../../../kern/sys_generic.c:406 #23 0xffffffff81068847 in amd64_syscall (td=0xfffff8086e133580, traced=0x0) at subr_syscall.c:135 #24 0xffffffff81044dcd in fast_syscall_common () at ../../../amd64/amd64/exception.S:504 #25 0x000000080041560a in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) f 5 #5 0xffffffff80ee7192 in vm_wait_doms (wdoms=0xfffffe028dc4d010) at vmmeter.h:232 232 return (DOMAINSET_SUBSET(&vm_min_domains, mask)); (kgdb) p *wdoms $1 = {__bits = 0xfffffe028dc4d010} (kgdb) f 6 #6 0xffffffff80ee7228 in vm_wait_domain (domain=) at ../../../vm/vm_page.c:2997 2997 vm_wait_doms(&wdom); (kgdb) l 2992 } else { 2993 if (pageproc == NULL) 2994 panic("vm_wait in early boot"); 2995 DOMAINSET_ZERO(&wdom); 2996 DOMAINSET_SET(vmd->vmd_domain, &wdom); 2997 vm_wait_doms(&wdom); 2998 } 2999 } 3000 3001 /* (kgdb) info loc wdom = {__bits = 0xfffffe028dc4d010} (kgdb) $ svnlite diff -x -p /usr/src/sys Index: /usr/src/sys/dev/hwpmc/hwpmc_logging.c =================================================================== --- /usr/src/sys/dev/hwpmc/hwpmc_logging.c (revision 339009) +++ /usr/src/sys/dev/hwpmc/hwpmc_logging.c (working copy) @@ -1257,13 +1257,14 @@ pmclog_initialize() int ncpus = pmc_dom_hdrs[domain]->pdbh_ncpus; int total = ncpus*pmc_nlogbuffers_pcpu; - plb = malloc_domain(sizeof(struct pmclog_buffer)*total, M_PMC, domain, M_WAITOK|M_ZERO); + plb = pmc_malloc_domain(sizeof(struct pmclog_buffer) * total, + domain, M_WAITOK | M_ZERO); pmc_dom_hdrs[domain]->pdbh_plbs = plb; for (int i = 0; i < total; i++, plb++) { void *buf; - buf = malloc_domain(1024 * pmclog_buffer_size, M_PMC, domain, - M_WAITOK|M_ZERO); + buf = pmc_malloc_domain(1024 * pmclog_buffer_size, + domain, M_WAITOK | M_ZERO); PMCLOG_INIT_BUFFER_DESCRIPTOR(plb, buf, domain); pmc_plb_rele_unlocked(plb); } Index: /usr/src/sys/dev/hwpmc/hwpmc_mod.c =================================================================== --- /usr/src/sys/dev/hwpmc/hwpmc_mod.c (revision 339009) +++ /usr/src/sys/dev/hwpmc/hwpmc_mod.c (working copy) @@ -36,6 +36,7 @@ __FBSDID("$FreeBSD$"); #include +#include #include #include #include @@ -78,14 +79,6 @@ __FBSDID("$FreeBSD$"); #include "hwpmc_soft.h" -#ifdef NUMA -#define NDOMAINS vm_ndomains -#else -#define NDOMAINS 1 -#define malloc_domain(size, type, domain, flags) malloc((size), (type), (flags)) -#define free_domain(addr, type) free(addr, type) -#endif - #define PMC_EPOCH_ENTER() struct epoch_tracker pmc_et; epoch_enter_preempt(global_epoch_preempt, &pmc_et) #define PMC_EPOCH_EXIT() epoch_exit_preempt(global_epoch_preempt, &pmc_et) @@ -826,6 +819,17 @@ pmc_rdtsc(void) } /* + * XXX + */ + +void * +pmc_malloc_domain(size_t size, int domain, int flags) +{ + + return (malloc_domainset(size, M_PMC, domainset_prefer[domain], flags)); +} + +/* * Get the file name for an executable. This is a simple wrapper * around vn_fullpath(9). */ @@ -5655,9 +5659,9 @@ pmc_initialize(void) continue; pc = pcpu_find(cpu); domain = pc->pc_domain; - sb = malloc_domain(sizeof(struct pmc_samplebuffer) + - pmc_nsamples * sizeof(struct pmc_sample), M_PMC, domain, - M_WAITOK|M_ZERO); + sb = pmc_malloc_domain(sizeof(struct pmc_samplebuffer) + + pmc_nsamples * sizeof(struct pmc_sample), domain, + M_WAITOK | M_ZERO); sb->ps_read = sb->ps_write = sb->ps_samples; sb->ps_fence = sb->ps_samples + pmc_nsamples; @@ -5664,8 +5668,9 @@ pmc_initialize(void) KASSERT(pmc_pcpu[cpu] != NULL, ("[pmc,%d] cpu=%d Null per-cpu data", __LINE__, cpu)); - sb->ps_callchains = malloc_domain(pmc_callchaindepth * pmc_nsamples * - sizeof(uintptr_t), M_PMC, domain, M_WAITOK|M_ZERO); + sb->ps_callchains = pmc_malloc_domain(pmc_callchaindepth * + pmc_nsamples * sizeof(uintptr_t), domain, + M_WAITOK | M_ZERO); for (n = 0, ps = sb->ps_samples; n < pmc_nsamples; n++, ps++) ps->ps_pc = sb->ps_callchains + @@ -5673,9 +5678,9 @@ pmc_initialize(void) pmc_pcpu[cpu]->pc_sb[PMC_HR] = sb; - sb = malloc_domain(sizeof(struct pmc_samplebuffer) + - pmc_nsamples * sizeof(struct pmc_sample), M_PMC, domain, - M_WAITOK|M_ZERO); + sb = pmc_malloc_domain(sizeof(struct pmc_samplebuffer) + + pmc_nsamples * sizeof(struct pmc_sample), domain, + M_WAITOK | M_ZERO); sb->ps_read = sb->ps_write = sb->ps_samples; sb->ps_fence = sb->ps_samples + pmc_nsamples; @@ -5682,8 +5687,9 @@ pmc_initialize(void) KASSERT(pmc_pcpu[cpu] != NULL, ("[pmc,%d] cpu=%d Null per-cpu data", __LINE__, cpu)); - sb->ps_callchains = malloc_domain(pmc_callchaindepth * pmc_nsamples * - sizeof(uintptr_t), M_PMC, domain, M_WAITOK|M_ZERO); + sb->ps_callchains = pmc_malloc_domain(pmc_callchaindepth * + pmc_nsamples * sizeof(uintptr_t), domain, + M_WAITOK | M_ZERO); for (n = 0, ps = sb->ps_samples; n < pmc_nsamples; n++, ps++) ps->ps_pc = sb->ps_callchains + @@ -5691,9 +5697,9 @@ pmc_initialize(void) pmc_pcpu[cpu]->pc_sb[PMC_SR] = sb; - sb = malloc_domain(sizeof(struct pmc_samplebuffer) + - pmc_nsamples * sizeof(struct pmc_sample), M_PMC, domain, - M_WAITOK|M_ZERO); + sb = pmc_malloc_domain(sizeof(struct pmc_samplebuffer) + + pmc_nsamples * sizeof(struct pmc_sample), domain, + M_WAITOK | M_ZERO); sb->ps_read = sb->ps_write = sb->ps_samples; sb->ps_fence = sb->ps_samples + pmc_nsamples; @@ -5700,8 +5706,9 @@ pmc_initialize(void) KASSERT(pmc_pcpu[cpu] != NULL, ("[pmc,%d] cpu=%d Null per-cpu data", __LINE__, cpu)); - sb->ps_callchains = malloc_domain(pmc_callchaindepth * pmc_nsamples * - sizeof(uintptr_t), M_PMC, domain, M_WAITOK|M_ZERO); + sb->ps_callchains = pmc_malloc_domain(pmc_callchaindepth * + pmc_nsamples * sizeof(uintptr_t), domain, + M_WAITOK | M_ZERO); for (n = 0, ps = sb->ps_samples; n < pmc_nsamples; n++, ps++) ps->ps_pc = sb->ps_callchains + @@ -5921,13 +5928,13 @@ pmc_cleanup(void) KASSERT(pmc_pcpu[cpu]->pc_sb[PMC_UR] != NULL, ("[pmc,%d] Null userret cpu sample buffer cpu=%d", __LINE__, cpu)); - free_domain(pmc_pcpu[cpu]->pc_sb[PMC_HR]->ps_callchains, M_PMC); - free_domain(pmc_pcpu[cpu]->pc_sb[PMC_HR], M_PMC); - free_domain(pmc_pcpu[cpu]->pc_sb[PMC_SR]->ps_callchains, M_PMC); - free_domain(pmc_pcpu[cpu]->pc_sb[PMC_SR], M_PMC); - free_domain(pmc_pcpu[cpu]->pc_sb[PMC_UR]->ps_callchains, M_PMC); - free_domain(pmc_pcpu[cpu]->pc_sb[PMC_UR], M_PMC); - free_domain(pmc_pcpu[cpu], M_PMC); + free(pmc_pcpu[cpu]->pc_sb[PMC_HR]->ps_callchains, M_PMC); + free(pmc_pcpu[cpu]->pc_sb[PMC_HR], M_PMC); + free(pmc_pcpu[cpu]->pc_sb[PMC_SR]->ps_callchains, M_PMC); + free(pmc_pcpu[cpu]->pc_sb[PMC_SR], M_PMC); + free(pmc_pcpu[cpu]->pc_sb[PMC_UR]->ps_callchains, M_PMC); + free(pmc_pcpu[cpu]->pc_sb[PMC_UR], M_PMC); + free(pmc_pcpu[cpu], M_PMC); } free(pmc_pcpu, M_PMC); Index: /usr/src/sys/kern/kern_cpuset.c =================================================================== --- /usr/src/sys/kern/kern_cpuset.c (revision 339009) +++ /usr/src/sys/kern/kern_cpuset.c (working copy) @@ -114,6 +114,7 @@ __FBSDID("$FreeBSD$"); */ LIST_HEAD(domainlist, domainset); +struct domainset *domainset_prefer[MAXMEMDOM]; static uma_zone_t cpuset_zone; static uma_zone_t domainset_zone; @@ -1343,6 +1344,25 @@ cpuset_setithread(lwpid_t id, int cpu) return _cpuset_setthread(id, &mask, NULL); } +/* XXX */ +void +domainset_init(void) +{ + struct domainset _dset, *dset; + int i; + + domainset_zone = uma_zcreate("domainset", sizeof(struct domainset), + NULL, NULL, NULL, NULL, UMA_ALIGN_CACHE, 0); + + dset = &_dset; + DOMAINSET_COPY(&all_domains, &dset->ds_mask); + dset->ds_policy = DOMAINSET_POLICY_PREFER; + for (i = 0; i < vm_ndomains; i++) { + dset->ds_prefer = i; + domainset_prefer[i] = domainset_create(dset); + } +} + /* * Create the domainset for cpuset 0, 1 and cpuset 2. */ @@ -1350,14 +1370,11 @@ void domainset_zero(void) { struct domainset *dset; - int i; mtx_init(&cpuset_lock, "cpuset", NULL, MTX_SPIN | MTX_RECURSE); dset = &domainset0; - DOMAINSET_ZERO(&dset->ds_mask); - for (i = 0; i < vm_ndomains; i++) - DOMAINSET_SET(i, &dset->ds_mask); + DOMAINSET_COPY(&all_domains, &dset->ds_mask); dset->ds_policy = DOMAINSET_POLICY_FIRSTTOUCH; dset->ds_prefer = -1; curthread->td_domain.dr_policy = _domainset_create(dset, NULL); @@ -1390,8 +1407,6 @@ cpuset_thread0(void) cpuset_zone = uma_zcreate("cpuset", sizeof(struct cpuset), NULL, NULL, NULL, NULL, UMA_ALIGN_CACHE, 0); - domainset_zone = uma_zcreate("domainset", sizeof(struct domainset), - NULL, NULL, NULL, NULL, UMA_ALIGN_CACHE, 0); /* * Create the root system set (0) for the whole machine. Doesn't use Index: /usr/src/sys/kern/kern_malloc.c =================================================================== --- /usr/src/sys/kern/kern_malloc.c (revision 339009) +++ /usr/src/sys/kern/kern_malloc.c (working copy) @@ -68,6 +68,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include #include #include @@ -641,6 +642,27 @@ malloc_domain(size_t size, struct malloc_type *mtp } void * +malloc_domainset(size_t size, struct malloc_type *mtp, struct domainset *ds, + int flags) +{ + struct domainset_ref dr; + struct vm_domainset_iter di; + void *p; + int domain; + + dr.dr_policy = ds; + dr.dr_iterator = curthread->td_domain.dr_iterator; + vm_domainset_iter_malloc_init(&di, &dr, &domain, &flags); + do { + p = malloc_domain(size, mtp, domain, flags); + if (p != NULL) + break; + } while (vm_domainset_iter_malloc(&di, &domain, &flags)); + + return (p); +} + +void * mallocarray(size_t nmemb, size_t size, struct malloc_type *type, int flags) { Index: /usr/src/sys/kern/kern_pmc.c =================================================================== --- /usr/src/sys/kern/kern_pmc.c (revision 339009) +++ /usr/src/sys/kern/kern_pmc.c (working copy) @@ -35,9 +35,9 @@ __FBSDID("$FreeBSD$"); #include "opt_hwpmc_hooks.h" -#include +#include #include -#include +#include #include #include #include @@ -364,8 +364,9 @@ init_hwpmc(void *dummy __unused) KASSERT(pmc_softs != NULL, ("cannot allocate soft events table")); for (domain = 0; domain < NDOMAINS; domain++) { - pmc_dom_hdrs[domain] = malloc_domain(sizeof(struct pmc_domain_buffer_header), M_PMC, domain, - M_WAITOK|M_ZERO); + pmc_dom_hdrs[domain] = malloc_domainset( + sizeof(struct pmc_domain_buffer_header), M_PMC, + domainset_prefer[domain], M_WAITOK | M_ZERO); mtx_init(&pmc_dom_hdrs[domain]->pdbh_mtx, "pmc_bufferlist_mtx", "pmc-leaf", MTX_SPIN); TAILQ_INIT(&pmc_dom_hdrs[domain]->pdbh_head); } Index: /usr/src/sys/sys/domainset.h =================================================================== --- /usr/src/sys/sys/domainset.h (revision 339009) +++ /usr/src/sys/sys/domainset.h (working copy) @@ -32,8 +32,8 @@ #define _SYS_DOMAINSET_H_ #include - #include +#include #define _NDOMAINSETBITS _BITSET_BITS #define _NDOMAINSETWORDS __bitset_words(DOMAINSET_SETSIZE) @@ -96,6 +96,9 @@ struct domainset { domainid_t ds_order[MAXMEMDOM]; /* nth domain table. */ }; +extern struct domainset *domainset_prefer[MAXMEMDOM]; + +void domainset_init(void); void domainset_zero(void); /* Index: /usr/src/sys/sys/malloc.h =================================================================== --- /usr/src/sys/sys/malloc.h (revision 339009) +++ /usr/src/sys/sys/malloc.h (working copy) @@ -230,8 +230,13 @@ void *malloc(size_t size, struct malloc_type *type _malloc_item; \ }) +struct domainset; + void *malloc_domain(size_t size, struct malloc_type *type, int domain, int flags) __malloc_like __result_use_check __alloc_size(1); +void *malloc_domainset(size_t size, struct malloc_type *type, + struct domainset *ds, int flags) __malloc_like __result_use_check + __alloc_size(1); void *mallocarray(size_t nmemb, size_t size, struct malloc_type *type, int flags) __malloc_like __result_use_check __alloc_size2(1, 2); Index: /usr/src/sys/sys/pmc.h =================================================================== --- /usr/src/sys/sys/pmc.h (revision 339009) +++ /usr/src/sys/sys/pmc.h (working copy) @@ -1218,5 +1218,7 @@ struct pmc_mdep *pmc_mdep_alloc(int nclasses); void pmc_mdep_free(struct pmc_mdep *md); void pmc_flush_samples(int cpu); uint64_t pmc_rdtsc(void); +void *pmc_malloc_domain(size_t size, int domain, int flags); + #endif /* _KERNEL */ #endif /* _SYS_PMC_H_ */ Index: /usr/src/sys/vm/uma_core.c =================================================================== --- /usr/src/sys/vm/uma_core.c (revision 339009) +++ /usr/src/sys/vm/uma_core.c (working copy) @@ -79,6 +79,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include #include #include @@ -156,6 +157,9 @@ static enum { BOOT_COLD = 0, BOOT_STRAPPED, BOOT_P static struct callout uma_callout; #define UMA_TIMEOUT 20 /* Seconds for callout interval. */ +/* A round-robin domain selection policy used for slab allocations. */ +static struct domainset uma_domainset_rr; + /* * This structure is passed as the zone ctor arg so that I don't have to create * a special allocation function just for zones. @@ -990,6 +994,8 @@ zone_drain(uma_zone_t zone) /* * Allocate a new slab for a keg. This does not insert the slab onto a list. + * If the allocation was successful, the keg lock will be held upon return, + * otherwise the keg will be left unlocked. * * Arguments: * wait Shall we wait? @@ -1011,13 +1017,12 @@ keg_alloc_slab(uma_keg_t keg, uma_zone_t zone, int KASSERT(domain >= 0 && domain < vm_ndomains, ("keg_alloc_slab: domain %d out of range", domain)); mtx_assert(&keg->uk_lock, MA_OWNED); - slab = NULL; - mem = NULL; allocf = keg->uk_allocf; KEG_UNLOCK(keg); - size = keg->uk_ppera * PAGE_SIZE; + slab = NULL; + mem = NULL; if (keg->uk_flags & UMA_ZONE_OFFPAGE) { slab = zone_alloc_item(keg->uk_slabzone, NULL, domain, wait); if (slab == NULL) @@ -1040,6 +1045,7 @@ keg_alloc_slab(uma_keg_t keg, uma_zone_t zone, int wait |= M_NODUMP; /* zone is passed for legacy reasons. */ + size = keg->uk_ppera * PAGE_SIZE; mem = allocf(zone, size, domain, &flags, wait); if (mem == NULL) { if (keg->uk_flags & UMA_ZONE_OFFPAGE) @@ -1078,20 +1084,18 @@ keg_alloc_slab(uma_keg_t keg, uma_zone_t zone, int goto out; } } -out: KEG_LOCK(keg); CTR3(KTR_UMA, "keg_alloc_slab: allocated slab %p for %s(%p)", slab, keg->uk_name, keg); - if (slab != NULL) { - if (keg->uk_flags & UMA_ZONE_HASH) - UMA_HASH_INSERT(&keg->uk_hash, slab, mem); + if (keg->uk_flags & UMA_ZONE_HASH) + UMA_HASH_INSERT(&keg->uk_hash, slab, mem); - keg->uk_pages += keg->uk_ppera; - keg->uk_free += keg->uk_ipers; - } + keg->uk_pages += keg->uk_ppera; + keg->uk_free += keg->uk_ipers; +out: return (slab); } @@ -1558,7 +1562,8 @@ keg_ctor(void *mem, int size, void *udata, int fla keg->uk_init = arg->uminit; keg->uk_fini = arg->fini; keg->uk_align = arg->align; - keg->uk_cursor = 0; + keg->uk_cursor.dr_policy = &uma_domainset_rr; + keg->uk_cursor.dr_iterator = 0; keg->uk_free = 0; keg->uk_reserve = 0; keg->uk_pages = 0; @@ -1980,6 +1985,7 @@ uma_startup(void *mem, int npages) struct uma_zctor_args args; uma_keg_t masterkeg; uintptr_t m; + int i; #ifdef DIAGNOSTIC printf("Entering %s with %d boot pages configured\n", __func__, npages); @@ -1987,6 +1993,14 @@ uma_startup(void *mem, int npages) rw_init(&uma_rwlock, "UMA lock"); + /* We need to hand-initialize the domain selection policy. */ + DOMAINSET_COPY(&all_domains, &uma_domainset_rr.ds_mask); + uma_domainset_rr.ds_policy = DOMAINSET_POLICY_ROUNDROBIN; + uma_domainset_rr.ds_prefer = -1; + uma_domainset_rr.ds_cnt = DOMAINSET_COUNT(&all_domains); + for (i = 0; i < vm_ndomains; i++) + uma_domainset_rr.ds_order[i] = i; + /* Use bootpages memory for the zone of zones and zone of kegs. */ m = (uintptr_t)mem; zones = (uma_zone_t)m; @@ -2597,7 +2611,7 @@ uma_zalloc_domain(uma_zone_t zone, void *udata, in * only 'domain'. */ static uma_slab_t -keg_first_slab(uma_keg_t keg, int domain, int rr) +keg_first_slab(uma_keg_t keg, int domain, bool rr) { uma_domain_t dom; uma_slab_t slab; @@ -2625,39 +2639,44 @@ static uma_slab_t return (NULL); } +static bool +keg_use_reserve_slab(uma_keg_t keg, int flags) +{ + + return (keg->uk_free > ((flags & M_USE_RESERVE) != 0 ? 0 : + keg->uk_reserve)); +} + static uma_slab_t keg_fetch_slab(uma_keg_t keg, uma_zone_t zone, int rdomain, int flags) { + struct vm_domainset_iter di; uma_domain_t dom; uma_slab_t slab; - int allocflags, domain, reserve, rr, start; + int allocflags, domain; + bool rr; mtx_assert(&keg->uk_lock, MA_OWNED); - slab = NULL; - reserve = 0; - allocflags = flags; - if ((flags & M_USE_RESERVE) == 0) - reserve = keg->uk_reserve; /* * Round-robin for non first-touch zones when there is more than one * domain. */ + allocflags = flags; if (vm_ndomains == 1) rdomain = 0; rr = rdomain == UMA_ANYDOMAIN; +again: if (rr) { - keg->uk_cursor = (keg->uk_cursor + 1) % vm_ndomains; - domain = start = keg->uk_cursor; - /* Only block on the second pass. */ if ((flags & (M_WAITOK | M_NOVM)) == M_WAITOK) - allocflags = (allocflags & ~M_WAITOK) | M_NOWAIT; + allocflags = (flags & ~M_WAITOK) | M_NOWAIT; + vm_domainset_iter_malloc_init(&di, &keg->uk_cursor, &domain, + &allocflags); } else - domain = start = rdomain; + domain = rdomain; -again: - do { - if (keg->uk_free > reserve && + for (;;) { + if (keg_use_reserve_slab(keg, flags) && (slab = keg_first_slab(keg, domain, rr)) != NULL) { MPASS(slab->us_keg == keg); return (slab); @@ -2673,7 +2692,7 @@ keg_fetch_slab(uma_keg_t keg, uma_zone_t zone, int keg->uk_flags |= UMA_ZFLAG_FULL; /* * If this is not a multi-zone, set the FULL bit. - * Otherwise slab_multi() takes care of it. + * Otherwise zone_fetch_slab_multi() takes care of it. */ if ((zone->uz_flags & UMA_ZFLAG_MULTI) == 0) { zone->uz_flags |= UMA_ZFLAG_FULL; @@ -2692,20 +2711,24 @@ keg_fetch_slab(uma_keg_t keg, uma_zone_t zone, int * and return. We assume that the caller is going to remove * at least one item. */ - if (slab) { + if (slab != NULL) { MPASS(slab->us_keg == keg); dom = &keg->uk_domain[slab->us_domain]; LIST_INSERT_HEAD(&dom->ud_part_slab, slab, us_link); return (slab); } - if (rr) - domain = (domain + 1) % vm_ndomains; - } while (domain != start); - /* Retry domain scan with blocking. */ - if (allocflags != flags) { - allocflags = flags; - goto again; + KEG_LOCK(keg); + if (!rr) + break; + if (vm_domainset_iter_malloc(&di, &domain, &allocflags) != 0) { + if ((flags & M_WAITOK) == 0) + break; + KEG_UNLOCK(keg); + vm_wait_doms(&uma_domainset_rr.ds_mask); + KEG_LOCK(keg); + goto again; + } } /* @@ -2713,7 +2736,7 @@ keg_fetch_slab(uma_keg_t keg, uma_zone_t zone, int * could have while we were unlocked. Check again before we * fail. */ - if (keg->uk_free > reserve && + if (keg_use_reserve_slab(keg, flags) && (slab = keg_first_slab(keg, domain, rr)) != NULL) { MPASS(slab->us_keg == keg); return (slab); @@ -2743,7 +2766,7 @@ zone_fetch_slab(uma_zone_t zone, uma_keg_t keg, in } /* - * uma_zone_fetch_slab_multi: Fetches a slab from one available keg. Returns + * zone_fetch_slab_multi: Fetches a slab from one available keg. Returns * with the keg locked. On NULL no lock is held. * * The last pointer is used to seed the search. It is not required. @@ -3580,14 +3603,13 @@ uma_prealloc(uma_zone_t zone, int items) domain = 0; if (slabs * keg->uk_ipers < items) slabs++; - while (slabs > 0) { + while (slabs-- > 0) { slab = keg_alloc_slab(keg, zone, domain, M_WAITOK); if (slab == NULL) - break; + return; MPASS(slab->us_keg == keg); dom = &keg->uk_domain[slab->us_domain]; LIST_INSERT_HEAD(&dom->ud_free_slab, slab, us_link); - slabs--; domain = (domain + 1) % vm_ndomains; } KEG_UNLOCK(keg); Index: /usr/src/sys/vm/uma_int.h =================================================================== --- /usr/src/sys/vm/uma_int.h (revision 339009) +++ /usr/src/sys/vm/uma_int.h (working copy) @@ -31,6 +31,7 @@ */ #include +#include #include /* @@ -226,7 +227,7 @@ struct uma_keg { struct uma_hash uk_hash; LIST_HEAD(,uma_zone) uk_zones; /* Keg's zones */ - uint32_t uk_cursor; /* Domain alloc cursor. */ + struct domainset_ref uk_cursor; /* Domain alloc cursor */ uint32_t uk_align; /* Alignment mask */ uint32_t uk_pages; /* Total page count */ uint32_t uk_free; /* Count of items free in slabs */ Index: /usr/src/sys/vm/vm_domainset.c =================================================================== --- /usr/src/sys/vm/vm_domainset.c (revision 339009) +++ /usr/src/sys/vm/vm_domainset.c (working copy) @@ -62,25 +62,23 @@ static int vm_domainset_default_stride = 64; * Determine which policy is to be used for this allocation. */ static void -vm_domainset_iter_init(struct vm_domainset_iter *di, struct vm_object *obj, - vm_pindex_t pindex) +vm_domainset_iter_init(struct vm_domainset_iter *di, struct domainset_ref *dr, + struct vm_object *obj, vm_pindex_t pindex) { - struct domainset *domain; - struct thread *td; /* - * object policy takes precedence over thread policy. The policies + * Object policy takes precedence over thread policy. The policies * are immutable and unsynchronized. Updates can race but pointer * loads are assumed to be atomic. */ - if (obj != NULL && (domain = obj->domain.dr_policy) != NULL) { - di->di_domain = domain; - di->di_iter = &obj->domain.dr_iterator; - } else { - td = curthread; - di->di_domain = td->td_domain.dr_policy; - di->di_iter = &td->td_domain.dr_iterator; + if (dr == NULL) { + if (obj != NULL && obj->domain.dr_policy != NULL) + dr = &obj->domain; + else + dr = &curthread->td_domain; } + di->di_domain = dr->dr_policy; + di->di_iter = &dr->dr_iterator; di->di_policy = di->di_domain->ds_policy; if (di->di_policy == DOMAINSET_POLICY_INTERLEAVE) { #if VM_NRESERVLEVEL > 0 @@ -212,7 +210,7 @@ vm_domainset_iter_page_init(struct vm_domainset_it vm_pindex_t pindex, int *domain, int *req) { - vm_domainset_iter_init(di, obj, pindex); + vm_domainset_iter_init(di, NULL, obj, pindex); di->di_flags = *req; *req = (di->di_flags & ~(VM_ALLOC_WAITOK | VM_ALLOC_WAITFAIL)) | VM_ALLOC_NOWAIT; @@ -258,13 +256,12 @@ vm_domainset_iter_page(struct vm_domainset_iter *d return (0); } - void vm_domainset_iter_malloc_init(struct vm_domainset_iter *di, - struct vm_object *obj, int *domain, int *flags) + struct domainset_ref *dr, int *domain, int *flags) { - vm_domainset_iter_init(di, obj, 0); + vm_domainset_iter_init(di, dr, kernel_object, 0); if (di->di_policy == DOMAINSET_POLICY_INTERLEAVE) di->di_policy = DOMAINSET_POLICY_ROUNDROBIN; di->di_flags = *flags; @@ -307,6 +304,7 @@ vm_domainset_iter_malloc(struct vm_domainset_iter } #else /* !NUMA */ + int vm_domainset_iter_page(struct vm_domainset_iter *di, int *domain, int *flags) { @@ -315,8 +313,8 @@ vm_domainset_iter_page(struct vm_domainset_iter *d } void -vm_domainset_iter_page_init(struct vm_domainset_iter *di, - struct vm_object *obj, vm_pindex_t pindex, int *domain, int *flags) +vm_domainset_iter_page_init(struct vm_domainset_iter *di, struct vm_object *obj, + vm_pindex_t pindex, int *domain, int *flags) { *domain = 0; @@ -331,10 +329,10 @@ vm_domainset_iter_malloc(struct vm_domainset_iter void vm_domainset_iter_malloc_init(struct vm_domainset_iter *di, - struct vm_object *obj, int *domain, int *flags) + struct domainset_ref *dr, int *domain, int *flags) { *domain = 0; } -#endif +#endif /* NUMA */ Index: /usr/src/sys/vm/vm_domainset.h =================================================================== --- /usr/src/sys/vm/vm_domainset.h (revision 339009) +++ /usr/src/sys/vm/vm_domainset.h (working copy) @@ -45,7 +45,7 @@ void vm_domainset_iter_page_init(struct vm_domains struct vm_object *, vm_pindex_t, int *, int *); int vm_domainset_iter_malloc(struct vm_domainset_iter *, int *, int *); void vm_domainset_iter_malloc_init(struct vm_domainset_iter *, - struct vm_object *, int *, int *); + struct domainset_ref *, int *, int *); void vm_wait_doms(const domainset_t *); Index: /usr/src/sys/vm/vm_glue.c =================================================================== --- /usr/src/sys/vm/vm_glue.c (revision 339009) +++ /usr/src/sys/vm/vm_glue.c (working copy) @@ -68,6 +68,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include #include #include @@ -300,7 +301,7 @@ struct kstack_cache_entry *kstack_cache; static int kstack_cache_size = 128; static int kstacks; static struct mtx kstack_cache_mtx; -MTX_SYSINIT(kstack_cache, &kstack_cache_mtx, "kstkch", MTX_DEF); +static struct domainset *kstack_domainset; SYSCTL_INT(_vm, OID_AUTO, kstack_cache_size, CTLFLAG_RW, &kstack_cache_size, 0, ""); @@ -307,6 +308,23 @@ SYSCTL_INT(_vm, OID_AUTO, kstack_cache_size, CTLFL SYSCTL_INT(_vm, OID_AUTO, kstacks, CTLFLAG_RD, &kstacks, 0, ""); +static void +kstack_init(void *dummy __unused) +{ + struct domainset dset; + int i; + + mtx_init(&kstack_cache_mtx, "kstkch", NULL, MTX_DEF); + + DOMAINSET_ZERO(&dset.ds_mask); + for (i = 0; i < vm_ndomains; i++) + DOMAINSET_SET(i, &dset.ds_mask); + dset.ds_policy = DOMAINSET_POLICY_ROUNDROBIN; + dset.ds_prefer = -1; + kstack_domainset = domainset_create(&dset); +} +SYSINIT(kstack_init, SI_SUB_VM_CONF, SI_ORDER_ANY, kstack_init, NULL); + /* * Create the kernel stack (including pcb for i386) for a new thread. * This routine directly affects the fork perf for a process and @@ -369,6 +387,14 @@ vm_thread_new(struct thread *td, int pages) return (0); } + /* + * Ensure that kstack objects can draw pages from any memory + * domain. Otherwise a local memory shortage can block a process + * swap-in. + */ + ksobj->domain.dr_policy = kstack_domainset; + ksobj->domain.dr_iterator = 0; + atomic_add_int(&kstacks, 1); if (KSTACK_GUARD_PAGES != 0) { pmap_qremove(ks, KSTACK_GUARD_PAGES); Index: /usr/src/sys/vm/vm_init.c =================================================================== --- /usr/src/sys/vm/vm_init.c (revision 339009) +++ /usr/src/sys/vm/vm_init.c (working copy) @@ -144,6 +144,8 @@ vm_mem_init(void *dummy) kmem_init_zero_region(); pmap_init(); vm_pager_init(); + + domainset_init(); } void Index: /usr/src/sys/vm/vm_kern.c =================================================================== --- /usr/src/sys/vm/vm_kern.c (revision 339009) +++ /usr/src/sys/vm/vm_kern.c (working copy) @@ -235,7 +235,7 @@ kmem_alloc_attr(vm_size_t size, int flags, vm_padd vm_offset_t addr; int domain; - vm_domainset_iter_malloc_init(&di, kernel_object, &domain, &flags); + vm_domainset_iter_malloc_init(&di, NULL, &domain, &flags); do { addr = kmem_alloc_attr_domain(domain, size, flags, low, high, memattr); @@ -319,7 +319,7 @@ kmem_alloc_contig(vm_size_t size, int flags, vm_pa vm_offset_t addr; int domain; - vm_domainset_iter_malloc_init(&di, kernel_object, &domain, &flags); + vm_domainset_iter_malloc_init(&di, NULL, &domain, &flags); do { addr = kmem_alloc_contig_domain(domain, size, flags, low, high, alignment, boundary, memattr); @@ -406,7 +406,7 @@ kmem_malloc(vm_size_t size, int flags) vm_offset_t addr; int domain; - vm_domainset_iter_malloc_init(&di, kernel_object, &domain, &flags); + vm_domainset_iter_malloc_init(&di, NULL, &domain, &flags); do { addr = kmem_malloc_domain(domain, size, flags); if (addr != 0) Index: /usr/src/sys/vm/vm_object.c =================================================================== --- /usr/src/sys/vm/vm_object.c (revision 339009) +++ /usr/src/sys/vm/vm_object.c (working copy) @@ -274,6 +274,7 @@ _vm_object_allocate(objtype_t type, vm_pindex_t si panic("_vm_object_allocate: type %d is undefined", type); } object->size = size; + object->domain.dr_policy = NULL; object->generation = 1; object->ref_count = 1; object->memattr = VM_MEMATTR_DEFAULT; Index: /usr/src/sys/vm/vm_page.c =================================================================== --- /usr/src/sys/vm/vm_page.c (revision 339009) +++ /usr/src/sys/vm/vm_page.c (working copy) @@ -2841,16 +2841,14 @@ vm_page_reclaim_contig(int req, u_long npages, vm_ void vm_domain_set(struct vm_domain *vmd) { + int domain; + domain = vmd->vmd_domain; mtx_lock(&vm_domainset_lock); - if (!vmd->vmd_minset && vm_paging_min(vmd)) { - vmd->vmd_minset = 1; - DOMAINSET_SET(vmd->vmd_domain, &vm_min_domains); - } - if (!vmd->vmd_severeset && vm_paging_severe(vmd)) { - vmd->vmd_severeset = 1; - DOMAINSET_SET(vmd->vmd_domain, &vm_severe_domains); - } + if (!vm_page_count_min_domain(domain) && vm_paging_min(vmd)) + DOMAINSET_SET(domain, &vm_min_domains); + if (!vm_page_count_severe_domain(domain) && vm_paging_severe(vmd)) + DOMAINSET_SET(domain, &vm_severe_domains); mtx_unlock(&vm_domainset_lock); } @@ -2860,19 +2858,19 @@ vm_domain_set(struct vm_domain *vmd) void vm_domain_clear(struct vm_domain *vmd) { + int domain; + domain = vmd->vmd_domain; mtx_lock(&vm_domainset_lock); - if (vmd->vmd_minset && !vm_paging_min(vmd)) { - vmd->vmd_minset = 0; - DOMAINSET_CLR(vmd->vmd_domain, &vm_min_domains); + if (vm_page_count_min_domain(domain) && !vm_paging_min(vmd)) { + DOMAINSET_CLR(domain, &vm_min_domains); if (vm_min_waiters != 0) { vm_min_waiters = 0; wakeup(&vm_min_domains); } } - if (vmd->vmd_severeset && !vm_paging_severe(vmd)) { - vmd->vmd_severeset = 0; - DOMAINSET_CLR(vmd->vmd_domain, &vm_severe_domains); + if (vm_page_count_severe_domain(domain) && !vm_paging_severe(vmd)) { + DOMAINSET_CLR(domain, &vm_severe_domains); if (vm_severe_waiters != 0) { vm_severe_waiters = 0; wakeup(&vm_severe_domains); Index: /usr/src/sys/vm/vm_pagequeue.h =================================================================== --- /usr/src/sys/vm/vm_pagequeue.h (revision 339009) +++ /usr/src/sys/vm/vm_pagequeue.h (working copy) @@ -124,8 +124,6 @@ struct vm_domain { int vmd_pageout_wanted; /* (a, p) pageout daemon wait channel */ int vmd_pageout_pages_needed; /* (d) page daemon waiting for pages? */ - bool vmd_minset; /* (d) Are we in vm_min_domains? */ - bool vmd_severeset; /* (d) Are we in vm_severe_domains? */ enum { VM_LAUNDRY_IDLE = 0, VM_LAUNDRY_BACKGROUND, Index: /usr/src/sys/vm/vm_swapout.c =================================================================== --- /usr/src/sys/vm/vm_swapout.c (revision 339009) +++ /usr/src/sys/vm/vm_swapout.c (working copy) @@ -742,7 +742,7 @@ swapper_selector(bool wkilled_only) /* * Limit swapper to swap in one non-WKILLED process in MAXSLP/2 * interval, assuming that there is: - * - no memory shortage; + * - no global memory shortage; * - no parallel swap-ins; * - no other swap-ins in the current SWAPIN_INTERVAL. */ @@ -750,7 +750,7 @@ static bool swapper_wkilled_only(void) { - return (vm_page_count_min() || swap_inprogress > 0 || + return (vm_page_count_min_set(&all_domains) || swap_inprogress > 0 || (u_int)(ticks - last_swapin) < SWAPIN_INTERVAL); } Index: /usr/src/sys/x86/iommu/busdma_dmar.c =================================================================== --- /usr/src/sys/x86/iommu/busdma_dmar.c (revision 339009) +++ /usr/src/sys/x86/iommu/busdma_dmar.c (working copy) @@ -67,6 +67,8 @@ __FBSDID("$FreeBSD$"); #include #include +extern struct domainset *domainset_prefer[MAXMEMDOM]; + /* * busdma_dmar.c, the implementation of the busdma(9) interface using * DMAR units from Intel VT-d. @@ -447,8 +449,8 @@ dmar_bus_dmamem_alloc(bus_dma_tag_t dmat, void** v if (tag->common.maxsize < PAGE_SIZE && tag->common.alignment <= tag->common.maxsize && attr == VM_MEMATTR_DEFAULT) { - *vaddr = malloc_domain(tag->common.maxsize, M_DEVBUF, - tag->common.domain, mflags); + *vaddr = malloc_domainset(tag->common.maxsize, M_DEVBUF, + domainset_prefer[tag->common.domain], mflags); map->flags |= BUS_DMAMAP_DMAR_MALLOC; } else { *vaddr = (void *)kmem_alloc_attr_domain(tag->common.domain, Index: /usr/src/sys/x86/x86/busdma_bounce.c =================================================================== --- /usr/src/sys/x86/x86/busdma_bounce.c (revision 339009) +++ /usr/src/sys/x86/x86/busdma_bounce.c (working copy) @@ -31,6 +31,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include #include #include @@ -411,9 +412,9 @@ bounce_bus_dmamem_alloc(bus_dma_tag_t dmat, void** *mapp = NULL; if (dmat->segments == NULL) { - dmat->segments = (bus_dma_segment_t *)malloc_domain( + dmat->segments = (bus_dma_segment_t *)malloc_domainset( sizeof(bus_dma_segment_t) * dmat->common.nsegments, - M_DEVBUF, dmat->common.domain, mflags); + M_DEVBUF, domainset_prefer[dmat->common.domain], mflags); if (dmat->segments == NULL) { CTR4(KTR_BUSDMA, "%s: tag %p tag flags 0x%x error %d", __func__, dmat, dmat->common.flags, ENOMEM); @@ -452,8 +453,8 @@ bounce_bus_dmamem_alloc(bus_dma_tag_t dmat, void** (dmat->common.alignment <= dmat->common.maxsize) && dmat->common.lowaddr >= ptoa((vm_paddr_t)Maxmem) && attr == VM_MEMATTR_DEFAULT) { - *vaddr = malloc_domain(dmat->common.maxsize, M_DEVBUF, - dmat->common.domain, mflags); + *vaddr = malloc_domainset(dmat->common.maxsize, M_DEVBUF, + domainset_prefer[dmat->common.domain], mflags); } else if (dmat->common.nsegments >= howmany(dmat->common.maxsize, MIN(dmat->common.maxsegsz, PAGE_SIZE)) && dmat->common.alignment <= PAGE_SIZE && -- Test scenario: misc/cpuset.sh