GENERIC from Sun Aug 25 22:30:49 2013 +0300, r254734+nbufkv b13df36, vmcore.32 GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2013 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #0 r254734+b13df36: Sun Aug 25 21:46:25 CEST 2013 pho@t1.osted.lan:/var/tmp/deviant2/sys/amd64/compile/PHO amd64 FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. CPU: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz (1995.23-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x206d7 Family = 0x6 Model = 0x2d Stepping = 7 Features=0xbfebfbff Features2=0x1fbee3ff AMD Features=0x2c100800 AMD Features2=0x1 TSC: P-state invariant, performance statistics real memory = 68719476736 (65536 MB) avail memory = 32154943488 (30665 MB) : Trying to mount root from ufs:/dev/da0p2 [rw]... Setting hostuuid: 3141f787-07e3-e111-b968-001e6756c168. Setting hostid: 0x564ac5b6. Starting ddb. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: /dev/da0p2: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/da0p2: clean, 72368290 free (111050 frags, 9032155 blocks, 0.1% fragmentation) /dev/da0p3: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/da0p3: clean, 22943841 free (969 frags, 2867859 blocks, 0.0% fragmentation) Mounting local file systems:. Writing entropy file:. Setting hostname: t1.osted.lan. Starting Network: lo0 igb0 igb1. lo0: flags=8049 metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 nd6 options=21 igb0: flags=8843 metric 0 mtu 1500 options=401bb ether 00:1e:67:56:c1:68 inet 192.168.1.108 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::21e:67ff:fe56:c168%igb0 prefixlen 64 scopeid 0x1 nd6 options=29 media: Ethernet autoselect (100baseTX ) status: active igb1: flags=8c02 metric 0 mtu 1500 options=401bb ether 00:1e:67:56:c1:69 nd6 options=29 media: Ethernet autoselect status: no carrier Starting devd. Starting Network: igb1. igb1: flags=8c02 metric 0 mtu 1500 options=401bb ether 00:1e:67:56:c1:69 nd6 options=29 media: Ethernet autoselect status: no carrier Configuring keyboard: keymap. ums0: on usbus2 ums0: 3 buttons and [Z] coordinates ID=0 Starting ums0 moused. Expensive timeout(9) function: 0xffffffff807136f0(0xffffffff81588bd0) 0.007800407 s add net default: gateway 192.168.1.1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib 32-bit compatibility ldconfig path: /usr/lib32 Creating and/or trimming log files. Starting syslogd. No core dumps found. Starting rpcbind. NFS access cache time=60 lock order reversal: 1st 0xfffffe07860d6788 bufwait (bufwait) @ kern/vfs_bio.c:3053 2nd 0xfffff80016f47000 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:284 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe081e176300 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe081e1763b0 witness_checkorder() at witness_checkorder+0xd4f/frame 0xfffffe081e176440 _sx_xlock() at _sx_xlock+0x75/frame 0xfffffe081e176480 ufsdirhash_add() at ufsdirhash_add+0x4c/frame 0xfffffe081e1764c0 ufs_direnter() at ufs_direnter+0x688/frame 0xfffffe081e176580 ufs_mkdir() at ufs_mkdir+0x863/frame 0xfffffe081e176780 VOP_MKDIR_APV() at VOP_MKDIR_APV+0x10e/frame 0xfffffe081e1767b0 kern_mkdirat() at kern_mkdirat+0x20e/frame 0xfffffe081e1769a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e176ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e176ab0 --- syscall (136, FreeBSD ELF64, sys_mkdir), rip = 0x80093978a, rsp = 0x7fffffffd798, rbp = 0x7fffffffdc70 --- Clearing /tmp (X related). Starting nfsuserd. Starting mountd. Starting nfsd. Recovering vi editor sessions:. Updating motd:. Mounting late file systems:. Starting ntpd. Starting powerd. Configuring syscons: keymap blanktime. Performing sanity check on sshd configuration. Starting sshd. Starting cron. Local package initialization: backup. Starting default mousedmoused: unable to open /dev/psm0: No such file or directory . Sun Aug 25 21:56:47 CEST 2013 FreeBSD/amd64 (t1.osted.lan) (console) login: Aug 25 21:59:37 t1 su: pho to root on /dev/pts/1 20130825 21:59:43 all (1/8): sendfile.sh lock order reversal: 1st 0xfffff800685c2490 so_snd_sx (so_snd_sx) @ kern/uipc_sockbuf.c:145 2nd 0xfffff80068abda28 ufs (ufs) @ kern/uipc_syscalls.c:2353 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe081e1e92d0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe081e1e9380 witness_checkorder() at witness_checkorder+0xd4f/frame 0xfffffe081e1e9410 __lockmgr_args() at __lockmgr_args+0x456/frame 0xfffffe081e1e9540 ffs_lock() at ffs_lock+0x92/frame 0xfffffe081e1e9590 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xf5/frame 0xfffffe081e1e95c0 _vn_lock() at _vn_lock+0xc6/frame 0xfffffe081e1e9630 vn_sendfile() at vn_sendfile+0x748/frame 0xfffffe081e1e9910 do_sendfile() at do_sendfile+0x116/frame 0xfffffe081e1e99a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e1e9ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e1e9ab0 --- syscall (393, FreeBSD ELF64, sys_sendfile), rip = 0x800aace4a, rsp = 0x7fffffffd5f8, rbp = 0x7fffffffd6e0 --- 20130825 22:06:28 all (2/8): sendfile2.sh 20130825 22:06:29 all (3/8): sendfile3.sh 20130825 22:07:37 all (4/8): sendfile4.sh 20130825 22:07:37 all (5/8): sendfile5.sh lock order reversal: 1st 0xfffff80110013c98 ufs (ufs) @ kern/vfs_subr.c:2099 2nd 0xfffffe078616c778 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:262 3rd 0xfffff801104e62d8 ufs (ufs) @ kern/vfs_subr.c:2099 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe081e349190 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe081e349240 witness_checkorder() at witness_checkorder+0xd4f/frame 0xfffffe081e3492d0 __lockmgr_args() at __lockmgr_args+0x6f2/frame 0xfffffe081e349400 ffs_lock() at ffs_lock+0x92/frame 0xfffffe081e349450 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xf5/frame 0xfffffe081e349480 _vn_lock() at _vn_lock+0xc6/frame 0xfffffe081e3494f0 vget() at vget+0x70/frame 0xfffffe081e349540 vfs_hash_get() at vfs_hash_get+0xf5/frame 0xfffffe081e349590 ffs_vgetf() at ffs_vgetf+0x41/frame 0xfffffe081e349620 softdep_sync_buf() at softdep_sync_buf+0x8fa/frame 0xfffffe081e3496d0 ffs_syncvnode() at ffs_syncvnode+0x258/frame 0xfffffe081e349750 ffs_sync() at ffs_sync+0x212/frame 0xfffffe081e349820 dounmount() at dounmount+0x360/frame 0xfffffe081e3498a0 sys_unmount() at sys_unmount+0x376/frame 0xfffffe081e3499a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e349ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e349ab0 --- syscall (22, FreeBSD ELF64, sys_unmount), rip = 0x80088b38a, rsp = 0x7fffffffcf08, rbp = 0x7fffffffd020 --- lock order reversal: 1st 0xfffff80068a1a068 ufs (ufs) @ kern/vfs_mount.c:1237 2nd 0xfffff801104fda28 devfs (devfs) @ ufs/ffs/ffs_vfsops.c:1411 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe081e3493b0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe081e349460 witness_checkorder() at witness_checkorder+0xd4f/frame 0xfffffe081e3494f0 __lockmgr_args() at __lockmgr_args+0x6f2/frame 0xfffffe081e349620 vop_stdlock() at vop_stdlock+0x3c/frame 0xfffffe081e349640 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xf5/frame 0xfffffe081e349670 _vn_lock() at _vn_lock+0xc6/frame 0xfffffe081e3496e0 ffs_flushfiles() at ffs_flushfiles+0x123/frame 0xfffffe081e349750 softdep_flushfiles() at softdep_flushfiles+0x17f/frame 0xfffffe081e3497b0 ffs_unmount() at ffs_unmount+0x1a2/frame 0xfffffe081e349820 dounmount() at dounmount+0x39e/frame 0xfffffe081e3498a0 sys_unmount() at sys_unmount+0x376/frame 0xfffffe081e3499a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e349ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e349ab0 --- syscall (22, FreeBSD ELF64, sys_unmount), rip = 0x80088b38a, rsp = 0x7fffffffcf08, rbp = 0x7fffffffd020 --- 20130825 22:08:24 all (6/8): sendfile6.sh 20130825 22:08:25 all (7/8): sendfile7.sh lock order reversal: 1st 0xfffff80068abc068 tmpfs (tmpfs) @ kern/uipc_syscalls.c:2353 2nd 0xfffffe0786027818 bufwait (bufwait) @ vm/vm_pager.c:308 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe081e349170 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe081e349220 witness_checkorder() at witness_checkorder+0xd4f/frame 0xfffffe081e3492b0 __lockmgr_args() at __lockmgr_args+0x6f2/frame 0xfffffe081e3493e0 initpbuf() at initpbuf+0xd1/frame 0xfffffe081e349410 getpbuf() at getpbuf+0x10d/frame 0xfffffe081e349440 swap_pager_getpages() at swap_pager_getpages+0x27f/frame 0xfffffe081e3494d0 uiomove_object() at uiomove_object+0x146/frame 0xfffffe081e349540 VOP_READ_APV() at VOP_READ_APV+0x108/frame 0xfffffe081e349570 vn_rdwr() at vn_rdwr+0x27c/frame 0xfffffe081e349630 vn_sendfile() at vn_sendfile+0x981/frame 0xfffffe081e349910 do_sendfile() at do_sendfile+0x116/frame 0xfffffe081e3499a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e349ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e349ab0 --- syscall (393, FreeBSD ELF64, sys_sendfile), rip = 0x800887e4a, rsp = 0x7fffffffd668, rbp = 0x7fffffffd730 --- 20130825 22:14:56 all (8/8): tmpfs7.sh 20130825 22:14:58 all (1/8): sendfile.sh 20130825 22:21:41 all (2/8): sendfile2.sh 20130825 22:21:42 all (3/8): sendfile3.sh 20130825 22:22:52 all (4/8): sendfile4.sh 20130825 22:22:52 all (5/8): sendfile5.sh 20130825 22:23:39 all (6/8): sendfile6.sh 20130825 22:23:40 all (7/8): sendfile7.sh 20130825 22:29:00 all (8/8): tmpfs7.sh 20130825 22:29:02 all (1/8): sendfile.sh 20130825 22:35:45 all (2/8): sendfile2.sh 20130825 22:35:46 all (3/8): sendfile3.sh 20130825 22:36:57 all (4/8): sendfile4.sh 20130825 22:36:57 all (5/8): sendfile5.sh Aug 25 22:37:34 t1 su: BAD SU pho to root on /dev/pts/2 Aug 25 22:37:38 t1 su: pho to root on /dev/pts/2 20130825 22:37:45 all (6/8): sendfile6.sh 20130825 22:37:46 all (7/8): sendfile7.sh 20130825 22:48:10 all (8/8): tmpfs7.sh 20130825 22:48:12 all (1/8): sendfile.sh 20130825 22:54:55 all (2/8): sendfile2.sh 20130825 22:54:56 all (3/8): sendfile3.sh 20130825 22:56:05 all (4/8): sendfile4.sh 20130825 22:56:06 all (5/8): sendfile5.sh 20130825 22:56:53 all (6/8): sendfile6.sh 20130825 22:56:54 all (7/8): sendfile7.sh 20130825 23:06:03 all (8/8): tmpfs7.sh 20130825 23:06:05 all (1/8): sendfile.sh Shutting down local packages:. Stopping cron. Waiting for PIDS: 966. Stopping sshd. Waiting for PIDS: 956. 90 second watchdog timeout expired. Shutdown terminated. Sun Aug 25 23:14:41 CEST 2013 Aug 25 23:14:41 t1 init: /bin/sh on /etc/rc.shutdown terminated abnormally, going to single user mode Aug 25 23:14:41 t1 syslogd: exiting on signal 15 Aug 25 23:15:01 init: some processes would not die; ps axl advised Enter full pathname of shell or RETURN for /bin/sh: # ps auxwwl USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND UID PPID CPU PRI NI MWCHAN root 11 2396.8 0.0 0 384 - RL 9:56PM 1556:07.89 [idle] 0 0 0 155 0 - root 0 0.0 0.0 0 416 - DLs 9:56PM 0:00.84 [kernel] 0 0 0 -52 0 - root 1 0.0 0.0 9344 940 - ILs 9:56PM 0:00.47 /sbin/init -- 0 0 0 20 0 wait root 2 0.0 0.0 0 16 - DL 9:56PM 0:00.00 [sctp_iterator] 0 0 0 -16 0 waiting_ root 3 0.0 0.0 0 16 - DL 9:56PM 0:00.00 [xpt_thrd] 0 0 0 -16 0 ccb_scan root 4 0.0 0.0 0 16 - DL 9:56PM 0:00.04 [enc_daemon0] 0 0 0 -16 0 idle root 5 0.0 0.0 0 16 - DL 9:56PM 17:24.54 [pagedaemon] 0 0 0 -16 0 psleep root 6 0.0 0.0 0 16 - DL 9:56PM 0:01.41 [vmdaemon] 0 0 0 -16 0 psleep root 7 0.0 0.0 0 16 - DL 9:56PM 0:00.00 [pagezero] 0 0 0 155 0 pgzero root 8 0.0 0.0 0 16 - DL 9:56PM 0:00.07 [bufdaemon] 0 0 0 -16 0 psleep root 9 0.0 0.0 0 16 - DL 9:56PM 0:00.54 [syncer] 0 0 0 16 0 syncer root 10 0.0 0.0 0 16 - DL 9:56PM 0:00.00 [audit] 0 0 0 -16 0 audit_wo root 12 0.0 0.0 0 912 - WL 9:56PM 1:01.71 [intr] 0 0 0 -76 0 - root 13 0.0 0.0 0 48 - DL 9:56PM 1:11.41 [geom] 0 0 0 -8 0 - root 14 0.0 0.0 0 16 - DL 9:56PM 0:00.36 [yarrow] 0 0 0 -16 0 - root 15 0.0 0.0 0 192 - DL 9:56PM 0:02.73 [usb] 0 0 0 -68 0 - root 16 0.0 0.0 0 16 - DL 9:56PM 0:00.06 [vnlru] 0 0 0 -16 0 vlruwt root 17 0.0 0.0 0 16 - DL 9:56PM 0:02.95 [softdepflush] 0 0 0 20 0 sdflush root 6843 0.0 0.0 14316 1664 u1- D 11:13PM 0:00.26 /sbin/swapoff -a 0 1 0 20 0 pgrbwt root 6846 0.0 0.0 16904 2988 u1 Ss 11:15PM 0:00.01 -sh (sh) 0 1 0 20 0 wait root 6847 0.0 0.0 16504 1956 u1 R+ 11:16PM 0:00.00 ps auxwwl 0 6846 0 20 0 - pho 4883 0.0 0.0 8084 16 2- D 10:40PM 0:00.29 ./sendfile_shm1 1001 1 0 20 0 pgrbwt # i # KDB: enter: Break to debugger [ thread pid 11 tid 100004 ] Stopped at kdb_alt_break_internal+0x17f: movq $0,kdb_why db> run pho db:0:pho> bt Tracing pid 11 tid 100004 td 0xfffff8000b210490 kdb_alt_break_internal() at kdb_alt_break_internal+0x17f/frame 0xfffffe078ac2c730 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe078ac2c740 uart_intr_rxready() at uart_intr_rxready+0x96/frame 0xfffffe078ac2c770 uart_intr() at uart_intr+0x111/frame 0xfffffe078ac2c7b0 intr_event_handle() at intr_event_handle+0x8c/frame 0xfffffe078ac2c800 intr_execute_handlers() at intr_execute_handlers+0x48/frame 0xfffffe078ac2c830 lapic_handle_intr() at lapic_handle_intr+0x73/frame 0xfffffe078ac2c860 Xapic_isr1() at Xapic_isr1+0xa4/frame 0xfffffe078ac2c860 --- interrupt, rip = 0xffffffff80c90bc6, rsp = 0xfffffe078ac2c920, rbp = 0xfffffe078ac2c930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe078ac2c930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe078ac2c980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe078ac2c9a0 cpu_idle() at cpu_idle+0x93/frame 0xfffffe078ac2c9c0 sched_idletd() at sched_idletd+0x1e8/frame 0xfffffe078ac2ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ac2cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ac2cab0 --- trap 0, rip = 0, rsp = 0xfffffe078ac2cb70, rbp = 0 --- db:0:bt> show allpcpu Current CPU: 1 cpuid = 0 dynamic pcpu = 0x558f80 curthread = 0xfffff8000b210920: pid 11 "idle: cpu0" curpcb = 0xfffffe07fc5fcb80 fpcurthread = none idlethread = 0xfffff8000b210920: tid 100003 "idle: cpu0" curpmap = 0xffffffff815cbe08 tssp = 0xffffffff815e3310 commontssp = 0xffffffff815e3310 rsp0 = 0xfffffe07fc5fcb80 gs32p = 0xffffffff815e4d68 ldt = 0xffffffff815e4da8 tss = 0xffffffff815e4d98 spin locks held: cpuid = 1 dynamic pcpu = 0xfffffe087b211f80 curthread = 0xfffff8000b210490: pid 11 "idle: cpu1" curpcb = 0xfffffe078ac2cb80 fpcurthread = none idlethread = 0xfffff8000b210490: tid 100004 "idle: cpu1" curpmap = 0xffffffff815cbe08 tssp = 0xffffffff815e3378 commontssp = 0xffffffff815e3378 rsp0 = 0xfffffe078ac2cb80 gs32p = 0xffffffff815e4dd0 ldt = 0xffffffff815e4e10 tss = 0xffffffff815e4e00 spin locks held: cpuid = 2 dynamic pcpu = 0xfffffe087b218f80 curthread = 0xfffff8000b210000: pid 11 "idle: cpu2" curpcb = 0xfffffe078ac31b80 fpcurthread = none idlethread = 0xfffff8000b210000: tid 100005 "idle: cpu2" curpmap = 0xffffffff815cbe08 tssp = 0xffffffff815e33e0 commontssp = 0xffffffff815e33e0 rsp0 = 0xfffffe078ac31b80 gs32p = 0xffffffff815e4e38 ldt = 0xffffffff815e4e78 tss = 0xffffffff815e4e68 spin locks held: cpuid = 3 dynamic pcpu = 0xfffffe087b21ff80 curthread = 0xfffff8000b220000: pid 11 "idle: cpu3" curpcb = 0xfffffe078ac36b80 fpcurthread = none idlethread = 0xfffff8000b220000: tid 100006 "idle: cpu3" curpmap = 0xffffffff815cbe08 tssp = 0xffffffff815e3448 commontssp = 0xffffffff815e3448 rsp0 = 0xfffffe078ac36b80 gs32p = 0xffffffff815e4ea0 ldt = 0xffffffff815e4ee0 tss = 0xffffffff815e4ed0 spin locks held: cpuid = 4 dynamic pcpu = 0xfffffe087b226f80 curthread = 0xfffff8000b21f920: pid 11 "idle: cpu4" curpcb = 0xfffffe078ac3bb80 fpcurthread = none idlethread = 0xfffff8000b21f920: tid 100007 "idle: cpu4" curpmap = 0xffffffff815cbe08 tssp = 0xffffffff815e34b0 commontssp = 0xffffffff815e34b0 rsp0 = 0xfffffe078ac3bb80 gs32p = 0xffffffff815e4f08 ldt = 0xffffffff815e4f48 tss = 0xffffffff815e4f38 spin locks held: cpuid = 5 dynamic pcpu = 0xfffffe087b22df80 curthread = 0xfffff8000b21f490: pid 11 "idle: cpu5" curpcb = 0xfffffe078ac40b80 fpcurthread = none idlethread = 0xfffff8000b21f490: tid 100008 "idle: cpu5" curpmap = 0xffffffff815cbe08 tssp = 0xffffffff815e3518 commontssp = 0xffffffff815e3518 rsp0 = 0xfffffe078ac40b80 gs32p = 0xffffffff815e4f70 ldt = 0xffffffff815e4fb0 tss = 0xffffffff815e4fa0 spin locks held: cpuid = 6 dynamic pcpu = 0xfffffe087b234f80 curthread = 0xfffff8000b21f000: pid 11 "idle: cpu6" curpcb = 0xfffffe078ac45b80 fpcurthread = none idlethread = 0xfffff8000b21f000: tid 100009 "idle: cpu6" curpmap = 0xffffffff815cbe08 tssp = 0xffffffff815e3580 commontssp = 0xffffffff815e3580 rsp0 = 0xfffffe078ac45b80 gs32p = 0xffffffff815e4fd8 ldt = 0xffffffff815e5018 tss = 0xffffffff815e5008 spin locks held: cpuid = 7 dynamic pcpu = 0xfffffe087b23bf80 curthread = 0xfffff8000b211920: pid 11 "idle: cpu7" curpcb = 0xfffffe078ac4ab80 fpcurthread = none idlethread = 0xfffff8000b211920: tid 100010 "idle: cpu7" curpmap = 0xffffffff815cbe08 tssp = 0xffffffff815e35e8 commontssp = 0xffffffff815e35e8 rsp0 = 0xfffffe078ac4ab80 gs32p = 0xffffffff815e5040 ldt = 0xffffffff815e5080 tss = 0xffffffff815e5070 spin locks held: cpuid = 8 dynamic pcpu = 0xfffffe087b242f80 curthread = 0xfffff8000b221920: pid 11 "idle: cpu8" curpcb = 0xfffffe078ac4fb80 fpcurthread = none idlethread = 0xfffff8000b221920: tid 100011 "idle: cpu8" curpmap = 0xffffffff815cbe08 tssp = 0xffffffff815e3650 commontssp = 0xffffffff815e3650 rsp0 = 0xfffffe078ac4fb80 gs32p = 0xffffffff815e50a8 ldt = 0xffffffff815e50e8 tss = 0xffffffff815e50d8 spin locks held: cpuid = 9 dynamic pcpu = 0xfffffe087b249f80 curthread = 0xfffff8000b221490: pid 11 "idle: cpu9" curpcb = 0xfffffe078ac54b80 fpcurthread = none idlethread = 0xfffff8000b221490: tid 100012 "idle: cpu9" curpmap = 0xffffffff815cbe08 tssp = 0xffffffff815e36b8 commontssp = 0xffffffff815e36b8 rsp0 = 0xfffffe078ac54b80 gs32p = 0xffffffff815e5110 ldt = 0xffffffff815e5150 tss = 0xffffffff815e5140 spin locks held: cpuid = 10 dynamic pcpu = 0xfffffe087b250f80 curthread = 0xfffff8000b221000: pid 11 "idle: cpu10" curpcb = 0xfffffe078ac59b80 fpcurthread = none idlethread = 0xfffff8000b221000: tid 100013 "idle: cpu10" curpmap = 0xffffffff815cbe08 tssp = 0xffffffff815e3720 commontssp = 0xffffffff815e3720 rsp0 = 0xfffffe078ac59b80 gs32p = 0xffffffff815e5178 ldt = 0xffffffff815e51b8 tss = 0xffffffff815e51a8 spin locks held: cpuid = 11 dynamic pcpu = 0xfffffe087b257f80 curthread = 0xfffff8000b220920: pid 11 "idle: cpu11" curpcb = 0xfffffe078ac5eb80 fpcurthread = none idlethread = 0xfffff8000b220920: tid 100014 "idle: cpu11" curpmap = 0xffffffff815cbe08 tssp = 0xffffffff815e3788 commontssp = 0xffffffff815e3788 rsp0 = 0xfffffe078ac5eb80 gs32p = 0xffffffff815e51e0 ldt = 0xffffffff815e5220 tss = 0xffffffff815e5210 spin locks held: cpuid = 12 dynamic pcpu = 0xfffffe087b25ef80 curthread = 0xfffff8000b220490: pid 11 "idle: cpu12" curpcb = 0xfffffe078ac63b80 fpcurthread = none idlethread = 0xfffff8000b220490: tid 100015 "idle: cpu12" curpmap = 0xffffffff815cbe08 tssp = 0xffffffff815e37f0 commontssp = 0xffffffff815e37f0 rsp0 = 0xfffffe078ac63b80 gs32p = 0xffffffff815e5248 ldt = 0xffffffff815e5288 tss = 0xffffffff815e5278 spin locks held: cpuid = 13 dynamic pcpu = 0xfffffe087b265f80 curthread = 0xfffff8000b223490: pid 11 "idle: cpu13" curpcb = 0xfffffe078ac68b80 fpcurthread = none idlethread = 0xfffff8000b223490: tid 100016 "idle: cpu13" curpmap = 0xffffffff815cbe08 tssp = 0xffffffff815e3858 commontssp = 0xffffffff815e3858 rsp0 = 0xfffffe078ac68b80 gs32p = 0xffffffff815e52b0 ldt = 0xffffffff815e52f0 tss = 0xffffffff815e52e0 spin locks held: cpuid = 14 dynamic pcpu = 0xfffffe087b26cf80 curthread = 0xfffff8000b223000: pid 11 "idle: cpu14" curpcb = 0xfffffe078ac6db80 fpcurthread = none idlethread = 0xfffff8000b223000: tid 100017 "idle: cpu14" curpmap = 0xffffffff815cbe08 tssp = 0xffffffff815e38c0 commontssp = 0xffffffff815e38c0 rsp0 = 0xfffffe078ac6db80 gs32p = 0xffffffff815e5318 ldt = 0xffffffff815e5358 tss = 0xffffffff815e5348 spin locks held: cpuid = 15 dynamic pcpu = 0xfffffe087b273f80 curthread = 0xfffff8000b222920: pid 11 "idle: cpu15" curpcb = 0xfffffe078ac72b80 fpcurthread = none idlethread = 0xfffff8000b222920: tid 100018 "idle: cpu15" curpmap = 0xffffffff815cbe08 tssp = 0xffffffff815e3928 commontssp = 0xffffffff815e3928 rsp0 = 0xfffffe078ac72b80 gs32p = 0xffffffff815e5380 ldt = 0xffffffff815e53c0 tss = 0xffffffff815e53b0 spin locks held: cpuid = 16 dynamic pcpu = 0xfffffe087b27af80 curthread = 0xfffff8000b222490: pid 11 "idle: cpu16" curpcb = 0xfffffe078ac77b80 fpcurthread = none idlethread = 0xfffff8000b222490: tid 100019 "idle: cpu16" curpmap = 0xffffffff815cbe08 tssp = 0xffffffff815e3990 commontssp = 0xffffffff815e3990 rsp0 = 0xfffffe078ac77b80 gs32p = 0xffffffff815e53e8 ldt = 0xffffffff815e5428 tss = 0xffffffff815e5418 spin locks held: cpuid = 17 dynamic pcpu = 0xfffffe087b281f80 curthread = 0xfffff8000b222000: pid 11 "idle: cpu17" curpcb = 0xfffffe078ac7cb80 fpcurthread = none idlethread = 0xfffff8000b222000: tid 100020 "idle: cpu17" curpmap = 0xffffffff815cbe08 tssp = 0xffffffff815e39f8 commontssp = 0xffffffff815e39f8 rsp0 = 0xfffffe078ac7cb80 gs32p = 0xffffffff815e5450 ldt = 0xffffffff815e5490 tss = 0xffffffff815e5480 spin locks held: cpuid = 18 dynamic pcpu = 0xfffffe087b288f80 curthread = 0xfffff8000b225000: pid 11 "idle: cpu18" curpcb = 0xfffffe078ac81b80 fpcurthread = none idlethread = 0xfffff8000b225000: tid 100021 "idle: cpu18" curpmap = 0xffffffff815cbe08 tssp = 0xffffffff815e3a60 commontssp = 0xffffffff815e3a60 rsp0 = 0xfffffe078ac81b80 gs32p = 0xffffffff815e54b8 ldt = 0xffffffff815e54f8 tss = 0xffffffff815e54e8 spin locks held: cpuid = 19 dynamic pcpu = 0xfffffe087b28ff80 curthread = 0xfffff8000b224920: pid 11 "idle: cpu19" curpcb = 0xfffffe078ac86b80 fpcurthread = none idlethread = 0xfffff8000b224920: tid 100022 "idle: cpu19" curpmap = 0xffffffff815cbe08 tssp = 0xffffffff815e3ac8 commontssp = 0xffffffff815e3ac8 rsp0 = 0xfffffe078ac86b80 gs32p = 0xffffffff815e5520 ldt = 0xffffffff815e5560 tss = 0xffffffff815e5550 spin locks held: cpuid = 20 dynamic pcpu = 0xfffffe087b296f80 curthread = 0xfffff8000b224490: pid 11 "idle: cpu20" curpcb = 0xfffffe078ac8bb80 fpcurthread = none idlethread = 0xfffff8000b224490: tid 100023 "idle: cpu20" curpmap = 0xffffffff815cbe08 tssp = 0xffffffff815e3b30 commontssp = 0xffffffff815e3b30 rsp0 = 0xfffffe078ac8bb80 gs32p = 0xffffffff815e5588 ldt = 0xffffffff815e55c8 tss = 0xffffffff815e55b8 spin locks held: cpuid = 21 dynamic pcpu = 0xfffffe087b29df80 curthread = 0xfffff8000b224000: pid 11 "idle: cpu21" curpcb = 0xfffffe078ac90b80 fpcurthread = none idlethread = 0xfffff8000b224000: tid 100024 "idle: cpu21" curpmap = 0xffffffff815cbe08 tssp = 0xffffffff815e3b98 commontssp = 0xffffffff815e3b98 rsp0 = 0xfffffe078ac90b80 gs32p = 0xffffffff815e55f0 ldt = 0xffffffff815e5630 tss = 0xffffffff815e5620 spin locks held: cpuid = 22 dynamic pcpu = 0xfffffe087b2a4f80 curthread = 0xfffff8000b223920: pid 11 "idle: cpu22" curpcb = 0xfffffe078ac95b80 fpcurthread = none idlethread = 0xfffff8000b223920: tid 100025 "idle: cpu22" curpmap = 0xffffffff815cbe08 tssp = 0xffffffff815e3c00 commontssp = 0xffffffff815e3c00 rsp0 = 0xfffffe078ac95b80 gs32p = 0xffffffff815e5658 ldt = 0xffffffff815e5698 tss = 0xffffffff815e5688 spin locks held: cpuid = 23 dynamic pcpu = 0xfffffe087b2abf80 curthread = 0xfffff8000b226920: pid 11 "idle: cpu23" curpcb = 0xfffffe078ac9ab80 fpcurthread = none idlethread = 0xfffff8000b226920: tid 100026 "idle: cpu23" curpmap = 0xffffffff815cbe08 tssp = 0xffffffff815e3c68 commontssp = 0xffffffff815e3c68 rsp0 = 0xfffffe078ac9ab80 gs32p = 0xffffffff815e56c0 ldt = 0xffffffff815e5700 tss = 0xffffffff815e56f0 spin locks held: db:0:allpcpu> show alllocks Process 4883 (sendfile_shm1) thread 0xfffff80016d37490 (100154) exclusive sx so_snd_sx (so_snd_sx) r = 0 (0xfffff800685c2f30) locked @ kern/uipc_sockbuf.c:145 db:0:alllocks> show lockedvnods Locked vnodes db:0:lockedvnods> show mount 0xfffff800165e1b58 /dev/da0p2 on / (ufs) 0xfffff800165e2000 devfs on /dev (devfs) 0xfffff80016ffd3c8 /dev/da0p3 on /tmp (ufs) 0xfffff8002c011790 procfs on /proc (procfs) More info: show mount db:0:mount> ps pid ppid pgrp uid state wmesg wchan cmd 6846 1 6846 0 Ss+ ttyin 0xfffff8000b332ca8 sh 6843 1 6785 0 D+ pgrbwt 0xfffff807f5388178 swapoff 4883 1 4882 1001 D+ pgrbwt 0xfffff807f5388178 sendfile_shm1 17 0 0 0 DL sdflush 0xffffffff8156ed0c [softdepflush] 16 0 0 0 DL vlruwt 0xfffff800161374b8 [vnlru] 9 0 0 0 DL syncer 0xffffffff81568030 [syncer] 8 0 0 0 DL psleep 0xffffffff81567a04 [bufdaemon] 7 0 0 0 DL pgzero 0xffffffff8157030c [pagezero] 6 0 0 0 DL psleep 0xffffffff8156fa40 [vmdaemon] 5 0 0 0 DL psleep 0xffffffff815e2b84 [pagedaemon] 4 0 0 0 DL idle 0xfffff8000b20f970 [enc_daemon0] 3 0 0 0 DL ccb_scan 0xffffffff81341eb0 [xpt_thrd] 2 0 0 0 DL waiting_ 0xffffffff815d6988 [sctp_iterator] 15 0 0 0 DL (threaded) [usb] 100119 D - 0xfffffe0001163e18 [usbus2] 100118 D - 0xfffffe0001163dc0 [usbus2] 100117 D - 0xfffffe0001163d68 [usbus2] 100116 D - 0xfffffe0001163d10 [usbus2] 100114 D - 0xfffffe000115c1b0 [usbus1] 100113 D - 0xfffffe000115c158 [usbus1] 100112 D - 0xfffffe000115c100 [usbus1] 100111 D - 0xfffffe000115c0a8 [usbus1] 100074 D - 0xfffffe0000ea9e18 [usbus0] 100073 D - 0xfffffe0000ea9dc0 [usbus0] 100072 D - 0xfffffe0000ea9d68 [usbus0] 100071 D - 0xfffffe0000ea9d10 [usbus0] 14 0 0 0 DL - 0xffffffff8135b108 [yarrow] 13 0 0 0 DL (threaded) [geom] 100055 D - 0xffffffff815cb298 [g_down] 100054 D - 0xffffffff815cb290 [g_up] 100053 D - 0xffffffff815cb288 [g_event] 12 0 0 0 WL (threaded) [intr] 100122 I [swi0: uart] 100121 I [irq287: hdac1] 100120 I [irq286: ahci0] 100115 I [irq20: ehci1] 100110 I [irq285: xhci0] 100109 I [irq284: igb1:link] 100107 I [irq283: igb1:que 7] 100105 I [irq282: igb1:que 6] 100103 I [irq281: igb1:que 5] 100101 I [irq280: igb1:que 4] 100099 I [irq279: igb1:que 3] 100097 I [irq278: igb1:que 2] 100095 I [irq277: igb1:que 1] 100093 I [irq276: igb1:que 0] 100092 I [irq275: igb0:link] 100090 I [irq274: igb0:que 7] 100088 I [irq273: igb0:que 6] 100086 I [irq272: igb0:que 5] 100084 I [irq271: igb0:que 4] 100082 I [irq270: igb0:que 3] 100080 I [irq269: igb0:que 2] 100078 I [irq268: igb0:que 1] 100076 I [irq267: igb0:que 0] 100075 I [irq266: hdac0] 100070 I [irq22: ehci0] 100069 I [irq265: isci0] 100068 I [irq264: isci0] 100067 I [swi6: task queue] 100062 I [swi2: cambio] 100061 I [swi5: fast taskq] 100058 I [swi6: Giant taskq] 100052 I [swi4: clock] 100051 I [swi4: clock] 100050 I [swi4: clock] 100049 I [swi4: clock] 100048 I [swi4: clock] 100047 I [swi4: clock] 100046 I [swi4: clock] 100045 I [swi4: clock] 100044 I [swi4: clock] 100043 I [swi4: clock] 100042 I [swi4: clock] 100041 I [swi4: clock] 100040 I [swi4: clock] 100039 I [swi4: clock] 100038 I [swi4: clock] 100037 I [swi4: clock] 100036 I [swi4: clock] 100035 I [swi4: clock] 100034 I [swi4: clock] 100033 I [swi4: clock] 100032 I [swi4: clock] 100031 I [swi4: clock] 100030 I [swi4: clock] 100029 I [swi4: clock] 100028 I [swi3: vm] 100027 I [swi1: netisr 0] 11 0 0 0 RL (threaded) [idle] 100026 Run CPU 23 [idle: cpu23] 100025 Run CPU 22 [idle: cpu22] 100024 Run CPU 21 [idle: cpu21] 100023 Run CPU 20 [idle: cpu20] 100022 Run CPU 19 [idle: cpu19] 100021 Run CPU 18 [idle: cpu18] 100020 Run CPU 17 [idle: cpu17] 100019 Run CPU 16 [idle: cpu16] 100018 Run CPU 15 [idle: cpu15] 100017 Run CPU 14 [idle: cpu14] 100016 Run CPU 13 [idle: cpu13] 100015 Run CPU 12 [idle: cpu12] 100014 Run CPU 11 [idle: cpu11] 100013 Run CPU 10 [idle: cpu10] 100012 Run CPU 9 [idle: cpu9] 100011 Run CPU 8 [idle: cpu8] 100010 Run CPU 7 [idle: cpu7] 100009 Run CPU 6 [idle: cpu6] 100008 Run CPU 5 [idle: cpu5] 100007 Run CPU 4 [idle: cpu4] 100006 Run CPU 3 [idle: cpu3] 100005 Run CPU 2 [idle: cpu2] 100004 Run CPU 1 [idle: cpu1] 100003 Run CPU 0 [idle: cpu0] 1 0 1 0 SLs wait 0xfffff8000b20f000 [init] 10 0 0 0 DL audit_wo 0xffffffff815da0f8 [audit] 0 0 0 0 DLs (threaded) [kernel] 100124 D - 0xffffffff81377aba [deadlkres] 100123 D - 0xfffff8000c833a00 [mca taskq] 100108 D - 0xfffff8000c834000 [igb1 que] 100106 D - 0xfffff8000c834100 [igb1 que] 100104 D - 0xfffff8000c834200 [igb1 que] 100102 D - 0xfffff8000c834300 [igb1 que] 100100 D - 0xfffff8000c834400 [igb1 que] 100098 D - 0xfffff8000c834500 [igb1 que] 100096 D - 0xfffff8000c834600 [igb1 que] 100094 D - 0xfffff8000c834700 [igb1 que] 100091 D - 0xfffff8000c835200 [igb0 que] 100089 D - 0xfffff8000c835300 [igb0 que] 100087 D - 0xfffff8000c835400 [igb0 que] 100085 D - 0xfffff8000c835500 [igb0 que] 100083 D - 0xfffff8000c835600 [igb0 que] 100081 D - 0xfffff8000c835700 [igb0 que] 100079 D - 0xfffff8000c835800 [igb0 que] 100077 D - 0xfffff8000c835900 [igb0 que] 100066 D - 0xfffff8000b239000 [kqueue taskq] 100065 D - 0xfffff8000b239100 [acpi_task_2] 100064 D - 0xfffff8000b239100 [acpi_task_1] 100063 D - 0xfffff8000b239100 [acpi_task_0] 100060 D - 0xfffff8000b239500 [ffs_trim taskq] 100059 D - 0xfffff8000b239600 [thread taskq] 100056 D - 0xfffff8000b239900 [firmware taskq] 100000 D swapin 0xffffffff815cb398 [swapper] db:0:ps> allt Tracing command sh pid 6846 tid 100255 td 0xfffff80016d57920 sched_switch() at sched_switch+0x482/frame 0xfffffe081e2a9620 mi_switch() at mi_switch+0x179/frame 0xfffffe081e2a9660 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e2a96a0 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081e2a9710 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe081e2a9740 _cv_wait_sig() at _cv_wait_sig+0x1e2/frame 0xfffffe081e2a97b0 tty_wait() at tty_wait+0x42/frame 0xfffffe081e2a97e0 ttydisc_read() at ttydisc_read+0x353/frame 0xfffffe081e2a9840 ttydev_read() at ttydev_read+0x6d/frame 0xfffffe081e2a9870 devfs_read_f() at devfs_read_f+0x7f/frame 0xfffffe081e2a98d0 dofileread() at dofileread+0x7b/frame 0xfffffe081e2a9910 kern_readv() at kern_readv+0x48/frame 0xfffffe081e2a9950 sys_read() at sys_read+0x63/frame 0xfffffe081e2a99a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e2a9ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e2a9ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800de0caa, rsp = 0x7fffffffdba8, rbp = 0x7fffffffdbd0 --- Tracing command swapoff pid 6843 tid 100270 td 0xfffff80110aa9920 sched_switch() at sched_switch+0x482/frame 0xfffffe081e2f4680 mi_switch() at mi_switch+0x179/frame 0xfffffe081e2f46c0 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e2f4700 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081e2f4730 _sleep() at _sleep+0x33d/frame 0xfffffe081e2f47c0 vm_page_busy_sleep() at vm_page_busy_sleep+0xbb/frame 0xfffffe081e2f47f0 vm_page_grab() at vm_page_grab+0x18f/frame 0xfffffe081e2f4840 swapoff_one() at swapoff_one+0x2b7/frame 0xfffffe081e2f48b0 sys_swapoff() at sys_swapoff+0x178/frame 0xfffffe081e2f49a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e2f4ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e2f4ab0 --- syscall (424, FreeBSD ELF64, sys_swapoff), rip = 0x800a9cb6a, rsp = 0x7fffffffd5e8, rbp = 0x7fffffffd610 --- Tracing command sendfile_shm1 pid 4883 tid 100154 td 0xfffff80016d37490 sched_switch() at sched_switch+0x482/frame 0xfffffe081e0a1470 mi_switch() at mi_switch+0x179/frame 0xfffffe081e0a14b0 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e0a14f0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081e0a1520 _sleep() at _sleep+0x33d/frame 0xfffffe081e0a15b0 vm_page_busy_sleep() at vm_page_busy_sleep+0xbb/frame 0xfffffe081e0a15e0 vm_page_grab() at vm_page_grab+0x18f/frame 0xfffffe081e0a1630 vn_sendfile() at vn_sendfile+0x8a5/frame 0xfffffe081e0a1910 do_sendfile() at do_sendfile+0x116/frame 0xfffffe081e0a19a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e0a1ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e0a1ab0 --- syscall (393, FreeBSD ELF64, sys_sendfile), rip = 0x800887e4a, rsp = 0x7fffffffd878, rbp = 0x7fffffffd8c0 --- Tracing command softdepflush pid 17 tid 100134 td 0xfffff8001606e920 sched_switch() at sched_switch+0x482/frame 0xfffffe081dea28f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081dea2930 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081dea2970 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe081dea29a0 _sleep() at _sleep+0x312/frame 0xfffffe081dea2a30 softdep_flush() at softdep_flush+0x20c/frame 0xfffffe081dea2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081dea2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081dea2ab0 --- trap 0, rip = 0, rsp = 0xfffffe081dea2b70, rbp = 0 --- Tracing command vnlru pid 16 tid 100133 td 0xfffff80016133000 sched_switch() at sched_switch+0x482/frame 0xfffffe081de9d8b0 mi_switch() at mi_switch+0x179/frame 0xfffffe081de9d8f0 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081de9d930 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe081de9d960 _sleep() at _sleep+0x312/frame 0xfffffe081de9d9f0 vnlru_proc() at vnlru_proc+0x48/frame 0xfffffe081de9da70 fork_exit() at fork_exit+0x84/frame 0xfffffe081de9dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081de9dab0 --- trap 0, rip = 0, rsp = 0xfffffe081de9db70, rbp = 0 --- Tracing command syncer pid 9 tid 100132 td 0xfffff80016133490 sched_switch() at sched_switch+0x482/frame 0xfffffe081de988a0 mi_switch() at mi_switch+0x179/frame 0xfffffe081de988e0 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081de98920 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe081de98950 _cv_timedwait_sbt() at _cv_timedwait_sbt+0x1fe/frame 0xfffffe081de989d0 sched_sync() at sched_sync+0x5cb/frame 0xfffffe081de98a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081de98ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081de98ab0 --- trap 0, rip = 0, rsp = 0xfffffe081de98b70, rbp = 0 --- Tracing command bufdaemon pid 8 tid 100131 td 0xfffff80016133920 sched_switch() at sched_switch+0x482/frame 0xfffffe081de93900 mi_switch() at mi_switch+0x179/frame 0xfffffe081de93940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081de93980 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe081de939b0 _sleep() at _sleep+0x312/frame 0xfffffe081de93a40 buf_daemon() at buf_daemon+0x68/frame 0xfffffe081de93a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081de93ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081de93ab0 --- trap 0, rip = 0, rsp = 0xfffffe081de93b70, rbp = 0 --- Tracing command pagezero pid 7 tid 100130 td 0xfffff8000c9ea920 sched_switch() at sched_switch+0x482/frame 0xfffffe081de8e900 mi_switch() at mi_switch+0x179/frame 0xfffffe081de8e940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081de8e980 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe081de8e9b0 _sleep() at _sleep+0x312/frame 0xfffffe081de8ea40 vm_pagezero() at vm_pagezero+0x88/frame 0xfffffe081de8ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe081de8eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081de8eab0 --- trap 0, rip = 0, rsp = 0xfffffe081de8eb70, rbp = 0 --- Tracing command vmdaemon pid 6 tid 100129 td 0xfffff8001606d000 sched_switch() at sched_switch+0x482/frame 0xfffffe081de898c0 mi_switch() at mi_switch+0x179/frame 0xfffffe081de89900 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081de89940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081de89970 _sleep() at _sleep+0x33d/frame 0xfffffe081de89a00 vm_daemon() at vm_daemon+0x7e/frame 0xfffffe081de89a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081de89ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081de89ab0 --- trap 0, rip = 0, rsp = 0xfffffe081de89b70, rbp = 0 --- Tracing command pagedaemon pid 5 tid 100128 td 0xfffff8001606d490 sched_switch() at sched_switch+0x482/frame 0xfffffe081de848b0 mi_switch() at mi_switch+0x179/frame 0xfffffe081de848f0 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081de84930 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe081de84960 _sleep() at _sleep+0x312/frame 0xfffffe081de849f0 vm_pageout() at vm_pageout+0x237/frame 0xfffffe081de84a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081de84ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081de84ab0 --- trap 0, rip = 0, rsp = 0xfffffe081de84b70, rbp = 0 --- Tracing command enc_daemon0 pid 4 tid 100127 td 0xfffff8001606d920 sched_switch() at sched_switch+0x482/frame 0xfffffe081ddfe8c0 mi_switch() at mi_switch+0x179/frame 0xfffffe081ddfe900 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081ddfe940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081ddfe970 _sleep() at _sleep+0x33d/frame 0xfffffe081ddfea00 enc_daemon() at enc_daemon+0x109/frame 0xfffffe081ddfea70 fork_exit() at fork_exit+0x84/frame 0xfffffe081ddfeab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081ddfeab0 --- trap 0, rip = 0, rsp = 0xfffffe081ddfeb70, rbp = 0 --- Tracing command xpt_thrd pid 3 tid 100126 td 0xfffff8001606e000 sched_switch() at sched_switch+0x482/frame 0xfffffe081dcb9900 mi_switch() at mi_switch+0x179/frame 0xfffffe081dcb9940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081dcb9980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081dcb99b0 _sleep() at _sleep+0x33d/frame 0xfffffe081dcb9a40 xpt_scanner_thread() at xpt_scanner_thread+0xdc/frame 0xfffffe081dcb9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081dcb9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081dcb9ab0 --- trap 0, rip = 0, rsp = 0xfffffe081dcb9b70, rbp = 0 --- Tracing command sctp_iterator pid 2 tid 100125 td 0xfffff8000c9e6000 sched_switch() at sched_switch+0x482/frame 0xfffffe081dcb4910 mi_switch() at mi_switch+0x179/frame 0xfffffe081dcb4950 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081dcb4990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081dcb49c0 _sleep() at _sleep+0x33d/frame 0xfffffe081dcb4a50 sctp_iterator_thread() at sctp_iterator_thread+0x59/frame 0xfffffe081dcb4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081dcb4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081dcb4ab0 --- trap 0, rip = 0, rsp = 0xfffffe081dcb4b70, rbp = 0 --- Tracing command usb pid 15 tid 100119 td 0xfffff8000c88c920 sched_switch() at sched_switch+0x482/frame 0xfffffe081bc13920 mi_switch() at mi_switch+0x179/frame 0xfffffe081bc13960 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081bc139a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081bc139d0 _cv_wait() at _cv_wait+0x1df/frame 0xfffffe081bc13a40 usb_process() at usb_process+0xd4/frame 0xfffffe081bc13a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081bc13ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081bc13ab0 --- trap 0, rip = 0, rsp = 0xfffffe081bc13b70, rbp = 0 --- Tracing command usb pid 15 tid 100118 td 0xfffff8000c89e000 sched_switch() at sched_switch+0x482/frame 0xfffffe081bc0e920 mi_switch() at mi_switch+0x179/frame 0xfffffe081bc0e960 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081bc0e9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081bc0e9d0 _cv_wait() at _cv_wait+0x1df/frame 0xfffffe081bc0ea40 usb_process() at usb_process+0xd4/frame 0xfffffe081bc0ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe081bc0eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081bc0eab0 --- trap 0, rip = 0, rsp = 0xfffffe081bc0eb70, rbp = 0 --- Tracing command usb pid 15 tid 100117 td 0xfffff8000c89e490 sched_switch() at sched_switch+0x482/frame 0xfffffe081bc09920 mi_switch() at mi_switch+0x179/frame 0xfffffe081bc09960 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081bc099a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081bc099d0 _cv_wait() at _cv_wait+0x1df/frame 0xfffffe081bc09a40 usb_process() at usb_process+0xd4/frame 0xfffffe081bc09a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081bc09ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081bc09ab0 --- trap 0, rip = 0, rsp = 0xfffffe081bc09b70, rbp = 0 --- Tracing command usb pid 15 tid 100116 td 0xfffff8000c89e920 sched_switch() at sched_switch+0x482/frame 0xfffffe081bc04920 mi_switch() at mi_switch+0x179/frame 0xfffffe081bc04960 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081bc049a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081bc049d0 _cv_wait() at _cv_wait+0x1df/frame 0xfffffe081bc04a40 usb_process() at usb_process+0xd4/frame 0xfffffe081bc04a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081bc04ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081bc04ab0 --- trap 0, rip = 0, rsp = 0xfffffe081bc04b70, rbp = 0 --- Tracing command usb pid 15 tid 100114 td 0xfffff8000c88b000 sched_switch() at sched_switch+0x482/frame 0xfffffe081ba75920 mi_switch() at mi_switch+0x179/frame 0xfffffe081ba75960 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081ba759a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081ba759d0 _cv_wait() at _cv_wait+0x1df/frame 0xfffffe081ba75a40 usb_process() at usb_process+0xd4/frame 0xfffffe081ba75a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081ba75ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081ba75ab0 --- trap 0, rip = 0, rsp = 0xfffffe081ba75b70, rbp = 0 --- Tracing command usb pid 15 tid 100113 td 0xfffff8000c88b490 sched_switch() at sched_switch+0x482/frame 0xfffffe081ba70920 mi_switch() at mi_switch+0x179/frame 0xfffffe081ba70960 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081ba709a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081ba709d0 _cv_wait() at _cv_wait+0x1df/frame 0xfffffe081ba70a40 usb_process() at usb_process+0xd4/frame 0xfffffe081ba70a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081ba70ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081ba70ab0 --- trap 0, rip = 0, rsp = 0xfffffe081ba70b70, rbp = 0 --- Tracing command usb pid 15 tid 100112 td 0xfffff8000c88b920 sched_switch() at sched_switch+0x482/frame 0xfffffe081ba6b920 mi_switch() at mi_switch+0x179/frame 0xfffffe081ba6b960 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081ba6b9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081ba6b9d0 _cv_wait() at _cv_wait+0x1df/frame 0xfffffe081ba6ba40 usb_process() at usb_process+0xd4/frame 0xfffffe081ba6ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe081ba6bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081ba6bab0 --- trap 0, rip = 0, rsp = 0xfffffe081ba6bb70, rbp = 0 --- Tracing command usb pid 15 tid 100111 td 0xfffff8000c88c000 sched_switch() at sched_switch+0x482/frame 0xfffffe081ba66920 mi_switch() at mi_switch+0x179/frame 0xfffffe081ba66960 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081ba669a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081ba669d0 _cv_wait() at _cv_wait+0x1df/frame 0xfffffe081ba66a40 usb_process() at usb_process+0xd4/frame 0xfffffe081ba66a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081ba66ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081ba66ab0 --- trap 0, rip = 0, rsp = 0xfffffe081ba66b70, rbp = 0 --- Tracing command usb pid 15 tid 100074 td 0xfffff8000b3fa920 sched_switch() at sched_switch+0x482/frame 0xfffffe0000301920 mi_switch() at mi_switch+0x179/frame 0xfffffe0000301960 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe00003019a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00003019d0 _cv_wait() at _cv_wait+0x1df/frame 0xfffffe0000301a40 usb_process() at usb_process+0xd4/frame 0xfffffe0000301a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000301ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000301ab0 --- trap 0, rip = 0, rsp = 0xfffffe0000301b70, rbp = 0 --- Tracing command usb pid 15 tid 100073 td 0xfffff8000c81f000 sched_switch() at sched_switch+0x482/frame 0xfffffe00002fc920 mi_switch() at mi_switch+0x179/frame 0xfffffe00002fc960 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe00002fc9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00002fc9d0 _cv_wait() at _cv_wait+0x1df/frame 0xfffffe00002fca40 usb_process() at usb_process+0xd4/frame 0xfffffe00002fca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00002fcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00002fcab0 --- trap 0, rip = 0, rsp = 0xfffffe00002fcb70, rbp = 0 --- Tracing command usb pid 15 tid 100072 td 0xfffff8000c81f490 sched_switch() at sched_switch+0x482/frame 0xfffffe00002f7920 mi_switch() at mi_switch+0x179/frame 0xfffffe00002f7960 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe00002f79a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00002f79d0 _cv_wait() at _cv_wait+0x1df/frame 0xfffffe00002f7a40 usb_process() at usb_process+0xd4/frame 0xfffffe00002f7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00002f7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00002f7ab0 --- trap 0, rip = 0, rsp = 0xfffffe00002f7b70, rbp = 0 --- Tracing command usb pid 15 tid 100071 td 0xfffff8000c81f920 sched_switch() at sched_switch+0x482/frame 0xfffffe00002f2920 mi_switch() at mi_switch+0x179/frame 0xfffffe00002f2960 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe00002f29a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00002f29d0 _cv_wait() at _cv_wait+0x1df/frame 0xfffffe00002f2a40 usb_process() at usb_process+0xd4/frame 0xfffffe00002f2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00002f2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00002f2ab0 --- trap 0, rip = 0, rsp = 0xfffffe00002f2b70, rbp = 0 --- Tracing command yarrow pid 14 tid 100057 td 0xfffff8000b266490 sched_switch() at sched_switch+0x482/frame 0xfffffe078ad358f0 mi_switch() at mi_switch+0x179/frame 0xfffffe078ad35930 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe078ad35970 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe078ad359a0 msleep_spin_sbt() at msleep_spin_sbt+0x1f0/frame 0xfffffe078ad35a30 random_kthread() at random_kthread+0x68/frame 0xfffffe078ad35a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ad35ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ad35ab0 --- trap 0, rip = 0, rsp = 0xfffffe078ad35b70, rbp = 0 --- Tracing command geom pid 13 tid 100055 td 0xfffff8000b252920 sched_switch() at sched_switch+0x482/frame 0xfffffe078ad2b8d0 mi_switch() at mi_switch+0x179/frame 0xfffffe078ad2b910 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe078ad2b950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe078ad2b980 _sleep() at _sleep+0x33d/frame 0xfffffe078ad2ba10 g_io_schedule_down() at g_io_schedule_down+0x4c/frame 0xfffffe078ad2ba60 g_down_procbody() at g_down_procbody+0x7d/frame 0xfffffe078ad2ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ad2bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ad2bab0 --- trap 0, rip = 0, rsp = 0xfffffe078ad2bb70, rbp = 0 --- Tracing command geom pid 13 tid 100054 td 0xfffff8000b253000 sched_switch() at sched_switch+0x482/frame 0xfffffe078ad268f0 mi_switch() at mi_switch+0x179/frame 0xfffffe078ad26930 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe078ad26970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe078ad269a0 _sleep() at _sleep+0x33d/frame 0xfffffe078ad26a30 g_io_schedule_up() at g_io_schedule_up+0xbc/frame 0xfffffe078ad26a60 g_up_procbody() at g_up_procbody+0x7d/frame 0xfffffe078ad26a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ad26ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ad26ab0 --- trap 0, rip = 0, rsp = 0xfffffe078ad26b70, rbp = 0 --- Tracing command geom pid 13 tid 100053 td 0xfffff8000b253490 sched_switch() at sched_switch+0x482/frame 0xfffffe078ad218f0 mi_switch() at mi_switch+0x179/frame 0xfffffe078ad21930 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe078ad21970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe078ad219a0 _sleep() at _sleep+0x33d/frame 0xfffffe078ad21a30 g_run_events() at g_run_events+0x4f4/frame 0xfffffe078ad21a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ad21ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ad21ab0 --- trap 0, rip = 0, rsp = 0xfffffe078ad21b70, rbp = 0 --- Tracing command intr pid 12 tid 100122 td 0xfffff8000c9ea000 sched_switch() at sched_switch+0x482/frame 0xfffffe081dca09f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081dca0a30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe081dca0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081dca0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081dca0ab0 --- trap 0, rip = 0, rsp = 0xfffffe081dca0b70, rbp = 0 --- Tracing command intr pid 12 tid 100121 td 0xfffff8000c9ea490 sched_switch() at sched_switch+0x482/frame 0xfffffe081dc989f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081dc98a30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe081dc98a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081dc98ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081dc98ab0 --- trap 0, rip = 0, rsp = 0xfffffe081dc98b70, rbp = 0 --- Tracing command intr pid 12 tid 100120 td 0xfffff8000c88c490 sched_switch() at sched_switch+0x482/frame 0xfffffe081bc189f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081bc18a30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe081bc18a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081bc18ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081bc18ab0 --- trap 0, rip = 0, rsp = 0xfffffe081bc18b70, rbp = 0 --- Tracing command intr pid 12 tid 100115 td 0xfffff8000c87b920 sched_switch() at sched_switch+0x482/frame 0xfffffe081bbfd9f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081bbfda30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe081bbfda70 fork_exit() at fork_exit+0x84/frame 0xfffffe081bbfdab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081bbfdab0 --- trap 0, rip = 0, rsp = 0xfffffe081bbfdb70, rbp = 0 --- Tracing command intr pid 12 tid 100110 td 0xfffff8000c87a000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100109 td 0xfffff8000c87a490 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100107 td 0xfffff8000c87b000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100105 td 0xfffff8000c873490 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100103 td 0xfffff8000c879000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100101 td 0xfffff8000c879920 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100099 td 0xfffff8000c872000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100097 td 0xfffff8000c872920 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100095 td 0xfffff8000c851000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100093 td 0xfffff8000c851920 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100092 td 0xfffff8000c856000 sched_switch() at sched_switch+0x482/frame 0xfffffe00003a59f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00003a5a30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe00003a5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00003a5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00003a5ab0 --- trap 0, rip = 0, rsp = 0xfffffe00003a5b70, rbp = 0 --- Tracing command intr pid 12 tid 100090 td 0xfffff8000c84f490 sched_switch() at sched_switch+0x482/frame 0xfffffe000039b9f0 mi_switch() at mi_switch+0x179/frame 0xfffffe000039ba30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe000039ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe000039bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000039bab0 --- trap 0, rip = 0, rsp = 0xfffffe000039bb70, rbp = 0 --- Tracing command intr pid 12 tid 100088 td 0xfffff8000c850000 sched_switch() at sched_switch+0x482/frame 0xfffffe00003919f0 mi_switch() at mi_switch+0x179/frame 0xfffffe0000391a30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe0000391a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000391ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000391ab0 --- trap 0, rip = 0, rsp = 0xfffffe0000391b70, rbp = 0 --- Tracing command intr pid 12 tid 100086 td 0xfffff8000c850920 sched_switch() at sched_switch+0x482/frame 0xfffffe00003879f0 mi_switch() at mi_switch+0x179/frame 0xfffffe0000387a30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe0000387a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000387ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000387ab0 --- trap 0, rip = 0, rsp = 0xfffffe0000387b70, rbp = 0 --- Tracing command intr pid 12 tid 100084 td 0xfffff8000c84e000 sched_switch() at sched_switch+0x482/frame 0xfffffe000037d9f0 mi_switch() at mi_switch+0x179/frame 0xfffffe000037da30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe000037da70 fork_exit() at fork_exit+0x84/frame 0xfffffe000037dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000037dab0 --- trap 0, rip = 0, rsp = 0xfffffe000037db70, rbp = 0 --- Tracing command intr pid 12 tid 100082 td 0xfffff8000c84e920 sched_switch() at sched_switch+0x482/frame 0xfffffe00003739f0 mi_switch() at mi_switch+0x179/frame 0xfffffe0000373a30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe0000373a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000373ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000373ab0 --- trap 0, rip = 0, rsp = 0xfffffe0000373b70, rbp = 0 --- Tracing command intr pid 12 tid 100080 td 0xfffff8000c84c000 sched_switch() at sched_switch+0x482/frame 0xfffffe00003699f0 mi_switch() at mi_switch+0x179/frame 0xfffffe0000369a30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe0000369a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000369ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000369ab0 --- trap 0, rip = 0, rsp = 0xfffffe0000369b70, rbp = 0 --- Tracing command intr pid 12 tid 100078 td 0xfffff8000c84c920 sched_switch() at sched_switch+0x482/frame 0xfffffe000035f9f0 mi_switch() at mi_switch+0x179/frame 0xfffffe000035fa30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe000035fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe000035fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000035fab0 --- trap 0, rip = 0, rsp = 0xfffffe000035fb70, rbp = 0 --- Tracing command intr pid 12 tid 100076 td 0xfffff8000c84d490 sched_switch() at sched_switch+0x482/frame 0xfffffe00003559f0 mi_switch() at mi_switch+0x179/frame 0xfffffe0000355a30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe0000355a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000355ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000355ab0 --- trap 0, rip = 0, rsp = 0xfffffe0000355b70, rbp = 0 --- Tracing command intr pid 12 tid 100075 td 0xfffff8000b3fa490 sched_switch() at sched_switch+0x482/frame 0xfffffe00003069f0 mi_switch() at mi_switch+0x179/frame 0xfffffe0000306a30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe0000306a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000306ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000306ab0 --- trap 0, rip = 0, rsp = 0xfffffe0000306b70, rbp = 0 --- Tracing command intr pid 12 tid 100070 td 0xfffff8000b3e3920 sched_switch() at sched_switch+0x482/frame 0xfffffe00002ed9f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00002eda30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe00002eda70 fork_exit() at fork_exit+0x84/frame 0xfffffe00002edab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00002edab0 --- trap 0, rip = 0, rsp = 0xfffffe00002edb70, rbp = 0 --- Tracing command intr pid 12 tid 100069 td 0xfffff8000b3f9000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100068 td 0xfffff8000b3f9490 sched_switch() at sched_switch+0x482/frame 0xfffffe078adad9f0 mi_switch() at mi_switch+0x179/frame 0xfffffe078adada30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe078adada70 fork_exit() at fork_exit+0x84/frame 0xfffffe078adadab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078adadab0 --- trap 0, rip = 0, rsp = 0xfffffe078adadb70, rbp = 0 --- Tracing command intr pid 12 tid 100067 td 0xfffff8000b3f9920 sched_switch() at sched_switch+0x482/frame 0xfffffe078ad679f0 mi_switch() at mi_switch+0x179/frame 0xfffffe078ad67a30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe078ad67a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ad67ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ad67ab0 --- trap 0, rip = 0, rsp = 0xfffffe078ad67b70, rbp = 0 --- Tracing command intr pid 12 tid 100062 td 0xfffff8000b3e3000 sched_switch() at sched_switch+0x482/frame 0xfffffe078ad4e9f0 mi_switch() at mi_switch+0x179/frame 0xfffffe078ad4ea30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe078ad4ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ad4eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ad4eab0 --- trap 0, rip = 0, rsp = 0xfffffe078ad4eb70, rbp = 0 --- Tracing command intr pid 12 tid 100061 td 0xfffff8000b3e3490 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100058 td 0xfffff8000b266000 sched_switch() at sched_switch+0x482/frame 0xfffffe078ad3a9f0 mi_switch() at mi_switch+0x179/frame 0xfffffe078ad3aa30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe078ad3aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ad3aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ad3aab0 --- trap 0, rip = 0, rsp = 0xfffffe078ad3ab70, rbp = 0 --- Tracing command intr pid 12 tid 100052 td 0xfffff8000b253920 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100051 td 0xfffff8000b254000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100050 td 0xfffff8000b251000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100049 td 0xfffff8000b251490 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100048 td 0xfffff8000b251920 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100047 td 0xfffff8000b252000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100046 td 0xfffff8000b252490 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100045 td 0xfffff8000b24d490 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100044 td 0xfffff8000b24d920 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100043 td 0xfffff8000b24e000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100042 td 0xfffff8000b24e490 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100041 td 0xfffff8000b24e920 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100040 td 0xfffff8000b244920 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100039 td 0xfffff8000b24c000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100038 td 0xfffff8000b24c490 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100037 td 0xfffff8000b24c920 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100036 td 0xfffff8000b24d000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100035 td 0xfffff8000b243000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100034 td 0xfffff8000b243490 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100033 td 0xfffff8000b243920 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100032 td 0xfffff8000b244000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100031 td 0xfffff8000b244490 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100030 td 0xfffff8000b225490 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100029 td 0xfffff8000b225920 sched_switch() at sched_switch+0x482/frame 0xfffffe078aca99f0 mi_switch() at mi_switch+0x179/frame 0xfffffe078aca9a30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe078aca9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078aca9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078aca9ab0 --- trap 0, rip = 0, rsp = 0xfffffe078aca9b70, rbp = 0 --- Tracing command intr pid 12 tid 100028 td 0xfffff8000b226000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100027 td 0xfffff8000b226490 sched_switch() at sched_switch+0x482/frame 0xfffffe078ac9f9f0 mi_switch() at mi_switch+0x179/frame 0xfffffe078ac9fa30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe078ac9fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ac9fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ac9fab0 --- trap 0, rip = 0, rsp = 0xfffffe078ac9fb70, rbp = 0 --- Tracing command idle pid 11 tid 100026 td 0xfffff8000b226920 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc5ebce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc5ebd00 trap() at trap+0x42/frame 0xfffffe07fc5ebf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc5ebf20 --- trap 0x13, rip = 0xffffffff80c90bc6, rsp = 0xfffffe07fc5ebfe0, rbp = 0xfffffe078ac9a930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe078ac9a930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe078ac9a980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe078ac9a9a0 cpu_idle() at cpu_idle+0x93/frame 0xfffffe078ac9a9c0 sched_idletd() at sched_idletd+0x1e8/frame 0xfffffe078ac9aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ac9aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ac9aab0 --- trap 0, rip = 0, rsp = 0xfffffe078ac9ab70, rbp = 0 --- Tracing command idle pid 11 tid 100025 td 0xfffff8000b223920 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc5e4ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc5e4d00 trap() at trap+0x42/frame 0xfffffe07fc5e4f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc5e4f20 --- trap 0x13, rip = 0xffffffff80c90bc6, rsp = 0xfffffe07fc5e4fe0, rbp = 0xfffffe078ac95930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe078ac95930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe078ac95980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe078ac959a0 cpu_idle() at cpu_idle+0x93/frame 0xfffffe078ac959c0 sched_idletd() at sched_idletd+0x1e8/frame 0xfffffe078ac95a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ac95ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ac95ab0 --- trap 0, rip = 0, rsp = 0xfffffe078ac95b70, rbp = 0 --- Tracing command idle pid 11 tid 100024 td 0xfffff8000b224000 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc5ddce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc5ddd00 trap() at trap+0x42/frame 0xfffffe07fc5ddf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc5ddf20 --- trap 0x13, rip = 0xffffffff80c90bc6, rsp = 0xfffffe07fc5ddfe0, rbp = 0xfffffe078ac90930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe078ac90930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe078ac90980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe078ac909a0 cpu_idle() at cpu_idle+0x93/frame 0xfffffe078ac909c0 sched_idletd() at sched_idletd+0x1e8/frame 0xfffffe078ac90a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ac90ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ac90ab0 --- trap 0, rip = 0, rsp = 0xfffffe078ac90b70, rbp = 0 --- Tracing command idle pid 11 tid 100023 td 0xfffff8000b224490 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc5d6ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc5d6d00 trap() at trap+0x42/frame 0xfffffe07fc5d6f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc5d6f20 --- trap 0x13, rip = 0xffffffff80c90bc6, rsp = 0xfffffe07fc5d6fe0, rbp = 0xfffffe078ac8b930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe078ac8b930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe078ac8b980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe078ac8b9a0 cpu_idle() at cpu_idle+0x93/frame 0xfffffe078ac8b9c0 sched_idletd() at sched_idletd+0x1e8/frame 0xfffffe078ac8ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ac8bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ac8bab0 --- trap 0, rip = 0, rsp = 0xfffffe078ac8bb70, rbp = 0 --- Tracing command idle pid 11 tid 100022 td 0xfffff8000b224920 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc5cfce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc5cfd00 trap() at trap+0x42/frame 0xfffffe07fc5cff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc5cff20 --- trap 0x13, rip = 0xffffffff80c90bc6, rsp = 0xfffffe07fc5cffe0, rbp = 0xfffffe078ac86930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe078ac86930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe078ac86980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe078ac869a0 cpu_idle() at cpu_idle+0x93/frame 0xfffffe078ac869c0 sched_idletd() at sched_idletd+0x1e8/frame 0xfffffe078ac86a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ac86ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ac86ab0 --- trap 0, rip = 0, rsp = 0xfffffe078ac86b70, rbp = 0 --- Tracing command idle pid 11 tid 100021 td 0xfffff8000b225000 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc5c8ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc5c8d00 trap() at trap+0x42/frame 0xfffffe07fc5c8f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc5c8f20 --- trap 0x13, rip = 0xffffffff80c90bc6, rsp = 0xfffffe07fc5c8fe0, rbp = 0xfffffe078ac81930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe078ac81930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe078ac81980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe078ac819a0 cpu_idle() at cpu_idle+0x93/frame 0xfffffe078ac819c0 sched_idletd() at sched_idletd+0x1e8/frame 0xfffffe078ac81a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ac81ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ac81ab0 --- trap 0, rip = 0, rsp = 0xfffffe078ac81b70, rbp = 0 --- Tracing command idle pid 11 tid 100020 td 0xfffff8000b222000 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc5c1ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc5c1d00 trap() at trap+0x42/frame 0xfffffe07fc5c1f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc5c1f20 --- trap 0x13, rip = 0xffffffff80c90bc6, rsp = 0xfffffe07fc5c1fe0, rbp = 0xfffffe078ac7c930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe078ac7c930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe078ac7c980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe078ac7c9a0 cpu_idle() at cpu_idle+0x93/frame 0xfffffe078ac7c9c0 sched_idletd() at sched_idletd+0x1e8/frame 0xfffffe078ac7ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ac7cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ac7cab0 --- trap 0, rip = 0, rsp = 0xfffffe078ac7cb70, rbp = 0 --- Tracing command idle pid 11 tid 100019 td 0xfffff8000b222490 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc5bace0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc5bad00 trap() at trap+0x42/frame 0xfffffe07fc5baf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc5baf20 --- trap 0x13, rip = 0xffffffff80c90bc6, rsp = 0xfffffe07fc5bafe0, rbp = 0xfffffe078ac77930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe078ac77930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe078ac77980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe078ac779a0 cpu_idle() at cpu_idle+0x93/frame 0xfffffe078ac779c0 sched_idletd() at sched_idletd+0x1e8/frame 0xfffffe078ac77a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ac77ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ac77ab0 --- trap 0, rip = 0, rsp = 0xfffffe078ac77b70, rbp = 0 --- Tracing command idle pid 11 tid 100018 td 0xfffff8000b222920 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc5b3ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc5b3d00 trap() at trap+0x42/frame 0xfffffe07fc5b3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc5b3f20 --- trap 0x13, rip = 0xffffffff80c90bc6, rsp = 0xfffffe07fc5b3fe0, rbp = 0xfffffe078ac72930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe078ac72930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe078ac72980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe078ac729a0 cpu_idle() at cpu_idle+0x93/frame 0xfffffe078ac729c0 sched_idletd() at sched_idletd+0x1e8/frame 0xfffffe078ac72a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ac72ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ac72ab0 --- trap 0, rip = 0, rsp = 0xfffffe078ac72b70, rbp = 0 --- Tracing command idle pid 11 tid 100017 td 0xfffff8000b223000 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc5acce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc5acd00 trap() at trap+0x42/frame 0xfffffe07fc5acf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc5acf20 --- trap 0x13, rip = 0xffffffff80c90bc6, rsp = 0xfffffe07fc5acfe0, rbp = 0xfffffe078ac6d930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe078ac6d930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe078ac6d980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe078ac6d9a0 cpu_idle() at cpu_idle+0x93/frame 0xfffffe078ac6d9c0 sched_idletd() at sched_idletd+0x1e8/frame 0xfffffe078ac6da70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ac6dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ac6dab0 --- trap 0, rip = 0, rsp = 0xfffffe078ac6db70, rbp = 0 --- Tracing command idle pid 11 tid 100016 td 0xfffff8000b223490 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc5a5ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc5a5d00 trap() at trap+0x42/frame 0xfffffe07fc5a5f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc5a5f20 --- trap 0x13, rip = 0xffffffff80c90bc6, rsp = 0xfffffe07fc5a5fe0, rbp = 0xfffffe078ac68930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe078ac68930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe078ac68980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe078ac689a0 cpu_idle() at cpu_idle+0x93/frame 0xfffffe078ac689c0 sched_idletd() at sched_idletd+0x1e8/frame 0xfffffe078ac68a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ac68ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ac68ab0 --- trap 0, rip = 0, rsp = 0xfffffe078ac68b70, rbp = 0 --- Tracing command idle pid 11 tid 100015 td 0xfffff8000b220490 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc59ece0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc59ed00 trap() at trap+0x42/frame 0xfffffe07fc59ef20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc59ef20 --- trap 0x13, rip = 0xffffffff80c90bc6, rsp = 0xfffffe07fc59efe0, rbp = 0xfffffe078ac63930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe078ac63930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe078ac63980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe078ac639a0 cpu_idle() at cpu_idle+0x93/frame 0xfffffe078ac639c0 sched_idletd() at sched_idletd+0x1e8/frame 0xfffffe078ac63a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ac63ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ac63ab0 --- trap 0, rip = 0, rsp = 0xfffffe078ac63b70, rbp = 0 --- Tracing command idle pid 11 tid 100014 td 0xfffff8000b220920 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc597ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc597d00 trap() at trap+0x42/frame 0xfffffe07fc597f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc597f20 --- trap 0x13, rip = 0xffffffff80c90bc6, rsp = 0xfffffe07fc597fe0, rbp = 0xfffffe078ac5e930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe078ac5e930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe078ac5e980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe078ac5e9a0 cpu_idle() at cpu_idle+0x93/frame 0xfffffe078ac5e9c0 sched_idletd() at sched_idletd+0x1e8/frame 0xfffffe078ac5ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ac5eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ac5eab0 --- trap 0, rip = 0, rsp = 0xfffffe078ac5eb70, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xfffff8000b221000 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc590ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc590d00 trap() at trap+0x42/frame 0xfffffe07fc590f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc590f20 --- trap 0x13, rip = 0xffffffff80c90bc6, rsp = 0xfffffe07fc590fe0, rbp = 0xfffffe078ac59930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe078ac59930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe078ac59980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe078ac599a0 cpu_idle() at cpu_idle+0x93/frame 0xfffffe078ac599c0 sched_idletd() at sched_idletd+0x1e8/frame 0xfffffe078ac59a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ac59ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ac59ab0 --- trap 0, rip = 0, rsp = 0xfffffe078ac59b70, rbp = 0 --- Tracing command idle pid 11 tid 100012 td 0xfffff8000b221490 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc589ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc589d00 trap() at trap+0x42/frame 0xfffffe07fc589f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc589f20 --- trap 0x13, rip = 0xffffffff80c90bc6, rsp = 0xfffffe07fc589fe0, rbp = 0xfffffe078ac54930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe078ac54930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe078ac54980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe078ac549a0 cpu_idle() at cpu_idle+0x93/frame 0xfffffe078ac549c0 sched_idletd() at sched_idletd+0x1e8/frame 0xfffffe078ac54a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ac54ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ac54ab0 --- trap 0, rip = 0, rsp = 0xfffffe078ac54b70, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffff8000b221920 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc582ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc582d00 trap() at trap+0x42/frame 0xfffffe07fc582f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc582f20 --- trap 0x13, rip = 0xffffffff80c90bc6, rsp = 0xfffffe07fc582fe0, rbp = 0xfffffe078ac4f930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe078ac4f930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe078ac4f980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe078ac4f9a0 cpu_idle() at cpu_idle+0x93/frame 0xfffffe078ac4f9c0 sched_idletd() at sched_idletd+0x1e8/frame 0xfffffe078ac4fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ac4fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ac4fab0 --- trap 0, rip = 0, rsp = 0xfffffe078ac4fb70, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xfffff8000b211920 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc57bce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc57bd00 trap() at trap+0x42/frame 0xfffffe07fc57bf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc57bf20 --- trap 0x13, rip = 0xffffffff80c90bc6, rsp = 0xfffffe07fc57bfe0, rbp = 0xfffffe078ac4a930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe078ac4a930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe078ac4a980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe078ac4a9a0 cpu_idle() at cpu_idle+0x93/frame 0xfffffe078ac4a9c0 sched_idletd() at sched_idletd+0x1e8/frame 0xfffffe078ac4aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ac4aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ac4aab0 --- trap 0, rip = 0, rsp = 0xfffffe078ac4ab70, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xfffff8000b21f000 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc574ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc574d00 trap() at trap+0x42/frame 0xfffffe07fc574f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc574f20 --- trap 0x13, rip = 0xffffffff80c90bc6, rsp = 0xfffffe07fc574fe0, rbp = 0xfffffe078ac45930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe078ac45930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe078ac45980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe078ac459a0 cpu_idle() at cpu_idle+0x93/frame 0xfffffe078ac459c0 sched_idletd() at sched_idletd+0x1e8/frame 0xfffffe078ac45a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ac45ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ac45ab0 --- trap 0, rip = 0, rsp = 0xfffffe078ac45b70, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xfffff8000b21f490 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc56dce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc56dd00 trap() at trap+0x42/frame 0xfffffe07fc56df20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc56df20 --- trap 0x13, rip = 0xffffffff80c90bc6, rsp = 0xfffffe07fc56dfe0, rbp = 0xfffffe078ac40930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe078ac40930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe078ac40980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe078ac409a0 cpu_idle() at cpu_idle+0x93/frame 0xfffffe078ac409c0 sched_idletd() at sched_idletd+0x1e8/frame 0xfffffe078ac40a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ac40ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ac40ab0 --- trap 0, rip = 0, rsp = 0xfffffe078ac40b70, rbp = 0 --- Tracing command idle pid 11 tid 100007 td 0xfffff8000b21f920 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc566ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc566d00 trap() at trap+0x42/frame 0xfffffe07fc566f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc566f20 --- trap 0x13, rip = 0xffffffff80c90bc6, rsp = 0xfffffe07fc566fe0, rbp = 0xfffffe078ac3b930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe078ac3b930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe078ac3b980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe078ac3b9a0 cpu_idle() at cpu_idle+0x93/frame 0xfffffe078ac3b9c0 sched_idletd() at sched_idletd+0x1e8/frame 0xfffffe078ac3ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ac3bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ac3bab0 --- trap 0, rip = 0, rsp = 0xfffffe078ac3bb70, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xfffff8000b220000 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc55fce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc55fd00 trap() at trap+0x42/frame 0xfffffe07fc55ff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc55ff20 --- trap 0x13, rip = 0xffffffff80c90bc6, rsp = 0xfffffe07fc55ffe0, rbp = 0xfffffe078ac36930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe078ac36930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe078ac36980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe078ac369a0 cpu_idle() at cpu_idle+0x93/frame 0xfffffe078ac369c0 sched_idletd() at sched_idletd+0x1e8/frame 0xfffffe078ac36a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ac36ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ac36ab0 --- trap 0, rip = 0, rsp = 0xfffffe078ac36b70, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffff8000b210000 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc558ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc558d00 trap() at trap+0x42/frame 0xfffffe07fc558f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc558f20 --- trap 0x13, rip = 0xffffffff80c90bc6, rsp = 0xfffffe07fc558fe0, rbp = 0xfffffe078ac31930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe078ac31930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe078ac31980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe078ac319a0 cpu_idle() at cpu_idle+0x93/frame 0xfffffe078ac319c0 sched_idletd() at sched_idletd+0x1e8/frame 0xfffffe078ac31a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ac31ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ac31ab0 --- trap 0, rip = 0, rsp = 0xfffffe078ac31b70, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xfffff8000b210490 kdb_alt_break_internal() at kdb_alt_break_internal+0x17f/frame 0xfffffe078ac2c730 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe078ac2c740 uart_intr_rxready() at uart_intr_rxready+0x96/frame 0xfffffe078ac2c770 uart_intr() at uart_intr+0x111/frame 0xfffffe078ac2c7b0 intr_event_handle() at intr_event_handle+0x8c/frame 0xfffffe078ac2c800 intr_execute_handlers() at intr_execute_handlers+0x48/frame 0xfffffe078ac2c830 lapic_handle_intr() at lapic_handle_intr+0x73/frame 0xfffffe078ac2c860 Xapic_isr1() at Xapic_isr1+0xa4/frame 0xfffffe078ac2c860 --- interrupt, rip = 0xffffffff80c90bc6, rsp = 0xfffffe078ac2c920, rbp = 0xfffffe078ac2c930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe078ac2c930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe078ac2c980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe078ac2c9a0 cpu_idle() at cpu_idle+0x93/frame 0xfffffe078ac2c9c0 sched_idletd() at sched_idletd+0x1e8/frame 0xfffffe078ac2ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ac2cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ac2cab0 --- trap 0, rip = 0, rsp = 0xfffffe078ac2cb70, rbp = 0 --- Tracing command idle pid 11 tid 100003 td 0xfffff8000b210920 cpustop_handler() at cpustop_handler+0x28/frame 0xffffffff815745e0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xffffffff81574600 trap() at trap+0x42/frame 0xffffffff81574820 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff81574820 --- trap 0x13, rip = 0xffffffff80c90bc6, rsp = 0xffffffff815748e0, rbp = 0xfffffe07fc5fc930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe07fc5fc930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe07fc5fc980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe07fc5fc9a0 cpu_idle() at cpu_idle+0x93/frame 0xfffffe07fc5fc9c0 sched_idletd() at sched_idletd+0x1e8/frame 0xfffffe07fc5fca70 fork_exit() at fork_exit+0x84/frame 0xfffffe07fc5fcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe07fc5fcab0 --- trap 0, rip = 0, rsp = 0xfffffe07fc5fcb70, rbp = 0 --- Tracing command init pid 1 tid 100002 td 0xfffff8000b211000 sched_switch() at sched_switch+0x482/frame 0xfffffe07fc5f7560 mi_switch() at mi_switch+0x179/frame 0xfffffe07fc5f75a0 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe07fc5f75e0 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe07fc5f7650 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe07fc5f7680 _sleep() at _sleep+0x320/frame 0xfffffe07fc5f7710 kern_wait6() at kern_wait6+0x3e7/frame 0xfffffe07fc5f77b0 sys_wait4() at sys_wait4+0x76/frame 0xfffffe07fc5f79a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe07fc5f7ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe07fc5f7ab0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x41323a, rsp = 0x7fffffffd6d8, rbp = 0x7fffffffd7c0 --- Tracing command audit pid 10 tid 100001 td 0xfffff8000b211490 sched_switch() at sched_switch+0x482/frame 0xfffffe07fc5f28f0 mi_switch() at mi_switch+0x179/frame 0xfffffe07fc5f2930 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe07fc5f2970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe07fc5f29a0 _cv_wait() at _cv_wait+0x1df/frame 0xfffffe07fc5f2a10 audit_worker() at audit_worker+0xa3/frame 0xfffffe07fc5f2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe07fc5f2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe07fc5f2ab0 --- trap 0, rip = 0, rsp = 0xfffffe07fc5f2b70, rbp = 0 --- Tracing command kernel pid 0 tid 100124 td 0xfffff8000c9e6490 sched_switch() at sched_switch+0x482/frame 0xfffffe081dcae890 mi_switch() at mi_switch+0x179/frame 0xfffffe081dcae8d0 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081dcae910 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe081dcae940 _sleep() at _sleep+0x312/frame 0xfffffe081dcae9d0 pause_sbt() at pause_sbt+0x70/frame 0xfffffe081dcaea20 deadlkres() at deadlkres+0x66/frame 0xfffffe081dcaea70 fork_exit() at fork_exit+0x84/frame 0xfffffe081dcaeab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081dcaeab0 --- trap 0, rip = 0, rsp = 0xfffffe081dcaeb70, rbp = 0 --- Tracing command kernel pid 0 tid 100123 td 0xfffff8000c9e6920 sched_switch() at sched_switch+0x482/frame 0xfffffe081dca9900 mi_switch() at mi_switch+0x179/frame 0xfffffe081dca9940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081dca9980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081dca99b0 msleep_spin_sbt() at msleep_spin_sbt+0x200/frame 0xfffffe081dca9a40 taskqueue_thread_loop() at taskqueue_thread_loop+0x8d/frame 0xfffffe081dca9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081dca9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081dca9ab0 --- trap 0, rip = 0, rsp = 0xfffffe081dca9b70, rbp = 0 --- Tracing command kernel pid 0 tid 100108 td 0xfffff8000c87a920 sched_switch() at sched_switch+0x482/frame 0xfffffe081ba36900 mi_switch() at mi_switch+0x179/frame 0xfffffe081ba36940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081ba36980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081ba369b0 _sleep() at _sleep+0x33d/frame 0xfffffe081ba36a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe081ba36a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081ba36ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081ba36ab0 --- trap 0, rip = 0, rsp = 0xfffffe081ba36b70, rbp = 0 --- Tracing command kernel pid 0 tid 100106 td 0xfffff8000c87b490 sched_switch() at sched_switch+0x482/frame 0xfffffe081ba2c900 mi_switch() at mi_switch+0x179/frame 0xfffffe081ba2c940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081ba2c980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081ba2c9b0 _sleep() at _sleep+0x33d/frame 0xfffffe081ba2ca40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe081ba2ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe081ba2cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081ba2cab0 --- trap 0, rip = 0, rsp = 0xfffffe081ba2cb70, rbp = 0 --- Tracing command kernel pid 0 tid 100104 td 0xfffff8000c873920 sched_switch() at sched_switch+0x482/frame 0xfffffe081ba22900 mi_switch() at mi_switch+0x179/frame 0xfffffe081ba22940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081ba22980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081ba229b0 _sleep() at _sleep+0x33d/frame 0xfffffe081ba22a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe081ba22a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081ba22ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081ba22ab0 --- trap 0, rip = 0, rsp = 0xfffffe081ba22b70, rbp = 0 --- Tracing command kernel pid 0 tid 100102 td 0xfffff8000c879490 sched_switch() at sched_switch+0x482/frame 0xfffffe081ba18900 mi_switch() at mi_switch+0x179/frame 0xfffffe081ba18940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081ba18980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081ba189b0 _sleep() at _sleep+0x33d/frame 0xfffffe081ba18a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe081ba18a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081ba18ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081ba18ab0 --- trap 0, rip = 0, rsp = 0xfffffe081ba18b70, rbp = 0 --- Tracing command kernel pid 0 tid 100100 td 0xfffff8000c856920 sched_switch() at sched_switch+0x482/frame 0xfffffe081ba0e900 mi_switch() at mi_switch+0x179/frame 0xfffffe081ba0e940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081ba0e980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081ba0e9b0 _sleep() at _sleep+0x33d/frame 0xfffffe081ba0ea40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe081ba0ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe081ba0eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081ba0eab0 --- trap 0, rip = 0, rsp = 0xfffffe081ba0eb70, rbp = 0 --- Tracing command kernel pid 0 tid 100098 td 0xfffff8000c872490 sched_switch() at sched_switch+0x482/frame 0xfffffe081ba04900 mi_switch() at mi_switch+0x179/frame 0xfffffe081ba04940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081ba04980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081ba049b0 _sleep() at _sleep+0x33d/frame 0xfffffe081ba04a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe081ba04a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081ba04ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081ba04ab0 --- trap 0, rip = 0, rsp = 0xfffffe081ba04b70, rbp = 0 --- Tracing command kernel pid 0 tid 100096 td 0xfffff8000c873000 sched_switch() at sched_switch+0x482/frame 0xfffffe00003f9900 mi_switch() at mi_switch+0x179/frame 0xfffffe00003f9940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe00003f9980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00003f99b0 _sleep() at _sleep+0x33d/frame 0xfffffe00003f9a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe00003f9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00003f9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00003f9ab0 --- trap 0, rip = 0, rsp = 0xfffffe00003f9b70, rbp = 0 --- Tracing command kernel pid 0 tid 100094 td 0xfffff8000c851490 sched_switch() at sched_switch+0x482/frame 0xfffffe00003ef900 mi_switch() at mi_switch+0x179/frame 0xfffffe00003ef940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe00003ef980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00003ef9b0 _sleep() at _sleep+0x33d/frame 0xfffffe00003efa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe00003efa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00003efab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00003efab0 --- trap 0, rip = 0, rsp = 0xfffffe00003efb70, rbp = 0 --- Tracing command kernel pid 0 tid 100091 td 0xfffff8000c856490 sched_switch() at sched_switch+0x482/frame 0xfffffe00003a0900 mi_switch() at mi_switch+0x179/frame 0xfffffe00003a0940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe00003a0980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00003a09b0 _sleep() at _sleep+0x33d/frame 0xfffffe00003a0a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe00003a0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00003a0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00003a0ab0 --- trap 0, rip = 0, rsp = 0xfffffe00003a0b70, rbp = 0 --- Tracing command kernel pid 0 tid 100089 td 0xfffff8000c84f920 sched_switch() at sched_switch+0x482/frame 0xfffffe0000396900 mi_switch() at mi_switch+0x179/frame 0xfffffe0000396940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe0000396980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00003969b0 _sleep() at _sleep+0x33d/frame 0xfffffe0000396a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe0000396a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000396ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000396ab0 --- trap 0, rip = 0, rsp = 0xfffffe0000396b70, rbp = 0 --- Tracing command kernel pid 0 tid 100087 td 0xfffff8000c850490 sched_switch() at sched_switch+0x482/frame 0xfffffe000038c900 mi_switch() at mi_switch+0x179/frame 0xfffffe000038c940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe000038c980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000038c9b0 _sleep() at _sleep+0x33d/frame 0xfffffe000038ca40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe000038ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe000038cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000038cab0 --- trap 0, rip = 0, rsp = 0xfffffe000038cb70, rbp = 0 --- Tracing command kernel pid 0 tid 100085 td 0xfffff8000c84d920 sched_switch() at sched_switch+0x482/frame 0xfffffe0000382900 mi_switch() at mi_switch+0x179/frame 0xfffffe0000382940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe0000382980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00003829b0 _sleep() at _sleep+0x33d/frame 0xfffffe0000382a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe0000382a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000382ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000382ab0 --- trap 0, rip = 0, rsp = 0xfffffe0000382b70, rbp = 0 --- Tracing command kernel pid 0 tid 100083 td 0xfffff8000c84e490 sched_switch() at sched_switch+0x482/frame 0xfffffe0000378900 mi_switch() at mi_switch+0x179/frame 0xfffffe0000378940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe0000378980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00003789b0 _sleep() at _sleep+0x33d/frame 0xfffffe0000378a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe0000378a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000378ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000378ab0 --- trap 0, rip = 0, rsp = 0xfffffe0000378b70, rbp = 0 --- Tracing command kernel pid 0 tid 100081 td 0xfffff8000c84f000 sched_switch() at sched_switch+0x482/frame 0xfffffe000036e900 mi_switch() at mi_switch+0x179/frame 0xfffffe000036e940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe000036e980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000036e9b0 _sleep() at _sleep+0x33d/frame 0xfffffe000036ea40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe000036ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe000036eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000036eab0 --- trap 0, rip = 0, rsp = 0xfffffe000036eb70, rbp = 0 --- Tracing command kernel pid 0 tid 100079 td 0xfffff8000c84c490 sched_switch() at sched_switch+0x482/frame 0xfffffe0000364900 mi_switch() at mi_switch+0x179/frame 0xfffffe0000364940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe0000364980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00003649b0 _sleep() at _sleep+0x33d/frame 0xfffffe0000364a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe0000364a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000364ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000364ab0 --- trap 0, rip = 0, rsp = 0xfffffe0000364b70, rbp = 0 --- Tracing command kernel pid 0 tid 100077 td 0xfffff8000c84d000 sched_switch() at sched_switch+0x482/frame 0xfffffe000035a900 mi_switch() at mi_switch+0x179/frame 0xfffffe000035a940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe000035a980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000035a9b0 _sleep() at _sleep+0x33d/frame 0xfffffe000035aa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe000035aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe000035aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000035aab0 --- trap 0, rip = 0, rsp = 0xfffffe000035ab70, rbp = 0 --- Tracing command kernel pid 0 tid 100066 td 0xfffff8000b3fa000 sched_switch() at sched_switch+0x482/frame 0xfffffe078ad62900 mi_switch() at mi_switch+0x179/frame 0xfffffe078ad62940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe078ad62980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe078ad629b0 _sleep() at _sleep+0x33d/frame 0xfffffe078ad62a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe078ad62a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ad62ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ad62ab0 --- trap 0, rip = 0, rsp = 0xfffffe078ad62b70, rbp = 0 --- Tracing command kernel pid 0 tid 100065 td 0xfffff8000b3df000 sched_switch() at sched_switch+0x482/frame 0xfffffe078ad5d900 mi_switch() at mi_switch+0x179/frame 0xfffffe078ad5d940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe078ad5d980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe078ad5d9b0 msleep_spin_sbt() at msleep_spin_sbt+0x200/frame 0xfffffe078ad5da40 taskqueue_thread_loop() at taskqueue_thread_loop+0x8d/frame 0xfffffe078ad5da70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ad5dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ad5dab0 --- trap 0, rip = 0, rsp = 0xfffffe078ad5db70, rbp = 0 --- Tracing command kernel pid 0 tid 100064 td 0xfffff8000b3df490 sched_switch() at sched_switch+0x482/frame 0xfffffe078ad58900 mi_switch() at mi_switch+0x179/frame 0xfffffe078ad58940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe078ad58980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe078ad589b0 msleep_spin_sbt() at msleep_spin_sbt+0x200/frame 0xfffffe078ad58a40 taskqueue_thread_loop() at taskqueue_thread_loop+0x8d/frame 0xfffffe078ad58a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ad58ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ad58ab0 --- trap 0, rip = 0, rsp = 0xfffffe078ad58b70, rbp = 0 --- Tracing command kernel pid 0 tid 100063 td 0xfffff8000b3df920 sched_switch() at sched_switch+0x482/frame 0xfffffe078ad53900 mi_switch() at mi_switch+0x179/frame 0xfffffe078ad53940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe078ad53980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe078ad539b0 msleep_spin_sbt() at msleep_spin_sbt+0x200/frame 0xfffffe078ad53a40 taskqueue_thread_loop() at taskqueue_thread_loop+0x8d/frame 0xfffffe078ad53a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ad53ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ad53ab0 --- trap 0, rip = 0, rsp = 0xfffffe078ad53b70, rbp = 0 --- Tracing command kernel pid 0 tid 100060 td 0xfffff8000b254490 sched_switch() at sched_switch+0x482/frame 0xfffffe078ad44900 mi_switch() at mi_switch+0x179/frame 0xfffffe078ad44940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe078ad44980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe078ad449b0 _sleep() at _sleep+0x33d/frame 0xfffffe078ad44a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe078ad44a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ad44ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ad44ab0 --- trap 0, rip = 0, rsp = 0xfffffe078ad44b70, rbp = 0 --- Tracing command kernel pid 0 tid 100059 td 0xfffff8000b254920 sched_switch() at sched_switch+0x482/frame 0xfffffe078ad3f900 mi_switch() at mi_switch+0x179/frame 0xfffffe078ad3f940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe078ad3f980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe078ad3f9b0 _sleep() at _sleep+0x33d/frame 0xfffffe078ad3fa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe078ad3fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ad3fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ad3fab0 --- trap 0, rip = 0, rsp = 0xfffffe078ad3fb70, rbp = 0 --- Tracing command kernel pid 0 tid 100056 td 0xfffff8000b266920 sched_switch() at sched_switch+0x482/frame 0xfffffe078ad30900 mi_switch() at mi_switch+0x179/frame 0xfffffe078ad30940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe078ad30980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe078ad309b0 _sleep() at _sleep+0x33d/frame 0xfffffe078ad30a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe078ad30a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ad30ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ad30ab0 --- trap 0, rip = 0, rsp = 0xfffffe078ad30b70, rbp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xffffffff815cb860 sched_switch() at sched_switch+0x482/frame 0xffffffff818989e0 mi_switch() at mi_switch+0x179/frame 0xffffffff81898a20 sleepq_switch() at sleepq_switch+0x185/frame 0xffffffff81898a60 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xffffffff81898a90 _sleep() at _sleep+0x312/frame 0xffffffff81898b20 swapper() at swapper+0x211/frame 0xffffffff81898b70 btext() at btext+0x2c db:0:allt> call doadump Dumping 1330 out of 31644 MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% Dump complete = 0 db:0:doadump> reset cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 1 (kgdb) proc 4883 [Switching to thread 136 (Thread 100154)]#0 sched_switch (td=0xfffff80016d37490, newtd=, flags=) at ../../../kern/sched_ule.c:1954 1954 cpuid = PCPU_GET(cpuid); Current language: auto; currently minimal (kgdb) bt #0 sched_switch (td=0xfffff80016d37490, newtd=, flags=) at ../../../kern/sched_ule.c:1954 #1 0xffffffff808c45a9 in mi_switch (flags=0x104, newtd=0x0) at ../../../kern/kern_synch.c:493 #2 0xffffffff808fe765 in sleepq_switch (wchan=, pri=) at ../../../kern/subr_sleepqueue.c:541 #3 0xffffffff808fe593 in sleepq_wait (wchan=0xfffff807f5388178, pri=0x54) at ../../../kern/subr_sleepqueue.c:620 #4 0xffffffff808c3efd in _sleep (ident=, lock=, priority=0x254, wmesg=0xffffffff80fb7072 "pgrbwt", sbt=0x0, pr=0x0) at ../../../kern/kern_synch.c:255 #5 0xffffffff80b3d88b in vm_page_busy_sleep (m=0xfffff807f5388178, wmesg=0xffffffff80fb7072 "pgrbwt") at ../../../vm/vm_page.c:590 #6 0xffffffff80b4120f in vm_page_grab (object=0xfffff80068e95e00, pindex=0x448, allocflags=0x20) at ../../../vm/vm_page.c:2725 #7 0xffffffff8093e1e5 in vn_sendfile (fp=, sockfd=0x6, hdr_uio=, trl_uio=0x0, offset=0x0, nbytes=0x127dbd3, sent=, flags=, td=0xfffffe081e0a1730) at ../../../kern/uipc_syscalls.c:1987 #8 0xffffffff8093d876 in do_sendfile (td=0xfffff80016d37490, uap=0xfffffe081e0a1a40, compat=0x0) at file.h:373 #9 0xffffffff80cadcd2 in amd64_syscall (td=0xfffff80016d37490, traced=0x0) at subr_syscall.c:134 #10 0xffffffff80c95d7b in Xfast_syscall () at ../../../amd64/amd64/exception.S:391 #11 0x0000000800887e4a in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) f 5 #5 0xffffffff80b3d88b in vm_page_busy_sleep (m=0xfffff807f5388178, wmesg=0xffffffff80fb7072 "pgrbwt") at ../../../vm/vm_page.c:590 590 msleep(m, vm_page_lockptr(m), PVM | PDROP, wmesg, 0); (kgdb) l 585 if ((x & VPB_BIT_WAITERS) == 0 && 586 !atomic_cmpset_int(&m->busy_lock, x, x | VPB_BIT_WAITERS)) { 587 vm_page_unlock(m); 588 return; 589 } 590 msleep(m, vm_page_lockptr(m), PVM | PDROP, wmesg, 0); 591 } 592 593 /* 594 * vm_page_trysbusy: (kgdb) p *m $1 = {plinks = {q = {tqe_next = 0xfffff807d40bdb80, tqe_prev = 0xfffff807d298f450}, s = {ss = {sle_next = 0xfffff807d40bdb80}, pv = 0xfffff807d298f450}, memguard = {p = 0xfffff807d40bdb80, v = 0xfffff807d298f450}}, listq = {tqe_next = 0xfffff807f754b6a8, tqe_prev = 0xfffff807d5873400}, object = 0xfffff80068e95e00, pindex = 0x448, phys_addr = 0x65ac3b000, md = {pv_list = {tqh_first = 0x0, tqh_last = 0xfffff807f53881b0}, pv_gen = 0x37, pat_mode = 0x6}, queue = 0x0, segind = 0x6, hold_count = 0x0, order = 0xd, pool = 0x0, cow = 0x0, wire_count = 0x0, aflags = 0x2, oflags = 0x0, flags = 0x0, act_count = 0x0, __pad0 = 0x0, valid = 0x0, dirty = 0x0, busy_lock = 0x6} (kgdb) git diff -- Test scenario: sendfile_shm1 + VM pressure