panic: lock (sleep mutex) swapdev not locked @ vm/swap_pager.c:867, vmcore.67 KDB: debugger backends: ddb KDB: current backend: ddb ---<>--- Copyright (c) 1992-2019 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 13.0-CURRENT #2 r350199M: Sun Jul 21 18:26:09 CEST 2019 pho@mercat1.netperf.freebsd.org:/usr/src/sys/amd64/compile/PHO amd64 FreeBSD clang version 8.0.1 (branches/release_80 364487) (based on LLVM 8.0.1) WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. Entering uma_startup with 10 boot pages configured startup_alloc from "UMA Kegs", 8 boot pages left startup_alloc from "UMA Zones", 7 boot pages left startup_alloc from "UMA Zones", 6 boot pages left startup_alloc from "UMA Zones", 5 boot pages left startup_alloc from "UMA Zones", 4 boot pages left startup_alloc from "UMA Zones", 3 boot pages left startup_alloc from "UMA Hash", 2 boot pages left startup_alloc from "UMA Zones", 1 boot pages left Entering uma_startup1 with 0 boot pages left Entering uma_startup2 with 0 boot pages left VT(vga): resolution 640x480 CPU: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz (3500.07-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306f2 Family=0x6 Model=0x3f Stepping=2 Features=0xbfebfbff Features2=0x7ffefbff AMD Features=0x2c100800 AMD Features2=0x21 Structured Extended Features=0x37ab XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics real memory = 34359738368 (32768 MB) avail memory = 33182638080 (31645 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs FreeBSD/SMP: 1 package(s) x 6 core(s) x 2 hardware threads random: unblocking device. ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard Launching APs: 1 4 7 6 10 11 3 2 9 5 8 Timecounter "TSC-low" frequency 1750036740 Hz quality 1000 random: entropy device external interface kbd1 at kbdmux0 module_register_init: MOD_LOAD (vesa, 0xffffffff8112b410, 0) error 19 random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" 000.000055 [4254] netmap_init netmap: loaded module [ath_hal] loaded nexus0 vtvga0: on motherboard cryptosoft0: on motherboard acpi0: on motherboard acpi0: Power Button (fixed) cpu0: on acpi0 atrtc0: port 0x70-0x71,0x74-0x77 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Event timer "HPET4" frequency 14318180 Hz quality 340 Event timer "HPET5" frequency 14318180 Hz quality 340 Event timer "HPET6" frequency 14318180 Hz quality 340 Event timer "HPET7" frequency 14318180 Hz quality 340 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: on acpi0 pci0: on pcib0 pci0: at device 11.1 (no driver attached) pci0: at device 11.2 (no driver attached) pci0: at device 16.1 (no driver attached) pci0: at device 16.6 (no driver attached) pci0: at device 18.1 (no driver attached) acpi_syscontainer0: on acpi0 acpi_syscontainer1: on acpi0 acpi_syscontainer2: on acpi0 acpi_syscontainer3: on acpi0 pcib1: port 0xcf8-0xcff on acpi0 pci1: on pcib1 pcib2: irq 26 at device 1.0 on pci1 pci2: on pcib2 pcib3: irq 32 at device 2.0 on pci1 pci3: on pcib3 pci3: at device 0.0 (no driver attached) pci3: at device 0.1 (no driver attached) pcib4: irq 40 at device 3.0 on pci1 pci4: on pcib4 pci1: at device 17.0 (no driver attached) ahci0: port 0xf110-0xf117,0xf100-0xf103,0xf0f0-0xf0f7,0xf0e0-0xf0e3,0xf020-0xf03f mem 0xfb438000-0xfb4387ff irq 16 at device 17.4 on pci1 ahci0: AHCI v1.30 with 4 6Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahciem0: on ahci0 xhci0: mem 0xfb400000-0xfb40ffff irq 19 at device 20.0 on pci1 xhci0: 32 bytes context size, 64-bit DMA usbus0 on xhci0 usbus0: 5.0Gbps Super Speed USB v3.0 pci1: at device 22.0 (no driver attached) pci1: at device 22.1 (no driver attached) ehci0: mem 0xfb434000-0xfb4343ff irq 18 at device 26.0 on pci1 usbus1: EHCI version 1.0 usbus1 on ehci0 usbus1: 480Mbps High Speed USB v2.0 pcib5: irq 16 at device 28.0 on pci1 pci5: on pcib5 pcib6: irq 18 at device 28.2 on pci1 pci6: on pcib6 pcib7: at device 0.0 on pci6 pci7: on pcib7 vgapci0: port 0xe000-0xe07f mem 0xfa000000-0xfaffffff,0xfb000000-0xfb01ffff irq 18 at device 0.0 on pci7 vgapci0: Boot video device pcib8: irq 16 at device 28.4 on pci1 pci8: on pcib8 igb0: port 0xd020-0xd03f mem 0xfb320000-0xfb33ffff,0xfb344000-0xfb347fff irq 16 at device 0.0 on pci8 igb0: Using 1024 TX descriptors and 1024 RX descriptors igb0: queue equality override not set, capping rx_queues at 6 and tx_queues at 6 igb0: Using 6 RX queues 6 TX queues igb0: Using MSI-X i/0, rev 1.10/1.00, addr 2> on usbus0 kbd2 at ukbd0 . lo0: link state changed to UP Autoloading module: ioat.ko ioat0: mem 0xfb42c000-0xfb42ffff irq 31 at device 4.0 on pci1 ioat0: Capabilities: 2f7 ioat1: mem 0xfb428000-0xfb42bfff irq 39 at device 4.1 on pci1 ioat1: Capabilities: 2f7 ioat2: mem 0xfb424000-0xfb427fff irq 31 at device 4.2 on pci1 ioat2: Capabilities: f7 ioat3: mem 0xfb420000-0xfb423fff irq 39 at device 4.3 on pci1 ioat3: Capabilities: f7 ioat4: mem 0xfb41c000-0xfb41ffff irq 31 at device 4.4 on pci1 ioat4: Capabilities: f7 ioat5: mem 0xfb418000-0xfb41bfff irq 39 at device 4.5 on pci1 ioat5: Capabilities: f7 ioat6: mem 0xfb414000-0xfb417fff irq 31 at device 4.6 on pci1 ioat6: Capabilities: f7 ioat7: mem 0xfb410000-0xfb413fff irq 39 at device 4.7 on pci1 ioat7: Capabilities: f7 Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Starting Network: igb1. igb1: flags=8802 metric 0 mtu 1500 options=e507bb ether 0c:c4:7a:a8:cd:db media: Ethernet autoselect status: no carrier nd6 options=29 Autoloading module: uhid.ko Autoloading module: ums.ko ums0 on uhub3 ums0: on usbus0 ums0: 3 buttons and [Z] coordinates ID=0 Starting dhclient. DHCPREQUEST on igb0 to 255.255.255.255 port 67 DHCPACK from 192if_delmulti_locked: detaching ifnet instance 0xfffff800049d1000 .168.5.1 bound to 192.168.5.171 -- renewal in 300 seconds. add host 127.0.0.1: gateway lo0 fib 0: route already in table add host ::1: gateway lo0 fib 0: route already in table add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Creating and/or trimming log files. Starting syslogd. Setting date via ntp. 21 Jul 18:35:40 ntpdate[2055]: step time server 192.168.5.1 offset -0.868745 sec Starting rpcbind. NFS access cache time=60 Starting watchdogd. No core dumps found. lock order reversal: 1st 0xfffffe00008a13a8 bufwait (bufwait) @ kern/vfs_bio.c:3904 2nd 0xfffff801e42ff200 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:289 stack backtrace: #0 0xffffffff80c3b0a3 at witness_debugger+0x73 #1 0xffffffff80c3aded at witness_checkorder+0xa7d #2 0xffffffff80bd7d28 at _sx_xlock+0x68 #3 0xffffffff80ee5d4d at ufsdirhash_add+0x4d #4 0xffffffff80eea6e6 at ufs_direnter+0x446 #5 0xffffffff80ef28f9 at ufs_mkdir+0x8c9 #6 0xffffffff8122c55c at VOP_MKDIR_APV+0x8c #7 0xffffffff80cb4533 at kern_mkdirat+0x1b3 #8 0xffffffff810a5bc6 at amd64_syscall+0x2d6 #9 0xffffffff8107dc4d at fast_syscall_common+0x101 Clearing /tmp (X related). Starting statd. Starting lockd. fscklock order reversal: 1st 0xfffff8002ec84068 ufs (ufs) @ kern/vfs_subr.c:2687 2nd 0xfffffe00008a13a8 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:283 3rd 0xfffff8003c7bf068 ufs (ufs) @ kern/vfs_subr.c:2687 stack backtrace: #0 0xffffffff80c3b0a3 at witness_debugger+0x73 #1 0xffffffff80c3aded at witness_checkorder+0xa7d #2 0xffffffff80ba1bcb at lockmgr_xlock_hard+0x6b #3 0xffffffff80ba263e at __lockmgr_args+0x5fe #4 0xffffffff80ee06bc at ffs_lock+0x7c #5 0xffffffff8122d0b0 at VOP_LOCK1_APV+0x70 #6 0xffffffff80cb7965 at _vn_lock+0x65 #7 0xffffffff80ca58f6 at vget+0xa6 #8 0xffffffff80c976ac at vfs_hash_get+0xcc #9 0xffffffff80edbeef at ffs_vgetf+0x3f #10 0xffffffff80ed0fef at softdep_sync_buf+0x56f #11 0xffffffff80ee14d4 at ffs_syncvnode+0x234 #12 0xffffffff80edec7d at ffs_sync+0x20d #13 0xffffffff80c9dfff at dounmount+0x4bf #14 0xffffffff80c9daa0 at sys_unmount+0x300 #15 0xffffffff810a5bc6 at amd64_syscall+0x2d6 #16 0xffffffff8107dc4d at fast_syscall_common+0x101 lock order reversal: 1st 0xfffff8002e3017d0 ufs (ufs) @ kern/vfs_mount.c:1354 2nd 0xfffff8002ec842e0 devfs (devfs) @ ufs/ffs/ffs_softdep.c:1926 stack backtrace: #0 0xffffffff80c3b0a3 at witness_debugger+0x73 #1 0xffffffff80c3aded at witness_checkorder+0xa7d #2 0xffffffff80ba16cb at lockmgr_lock_fast_path+0x16b #3 0xffffffff8122d0b0 at VOP_LOCK1_APV+0x70 #4 0xffffffff80cb7965 at _vn_lock+0x65 #5 0xffffffff80ebddbd at softdep_flushworklist+0x5d #6 0xffffffff80edee76 at ffs_sync+0x406 #7 0xffffffff80c9dfff at dounmount+0x4bf #8 0xffffffff80c9daa0 at sys_unmount+0x300 #9 0xffffffff810a5bc6 at amd64_syscall+0x2d6 #10 0xffffffff8107dc4d at fast_syscall_common+0x101 ** /dev/gpt/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 252 files, 28409 used, 1236998 free (86 frags, 154614 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** Updating motd:. Mounting late filesystems:. Security policy loaded: MAC/ntpd (mac_ntpd) Starting ntpd. Configuring vt: blanktime. Performing sanity check on sshd configuration. Starting sshd. Starting cron. Sun Jul 21 18:35 FreeBSD/amd64 (mercat1.netperf.freebsd.org) (ttyu1) login: root Jul 21 18:38:22 mercat1 login[2336]: ROOT LOGIN (root) ON ttyu1 Last login: Sun Jul 21 13:58:17 on ttyu1 FreeBSD 13.0-CURRENT (PHO) #2 r350199M: Sun Jul 21 18:26:09 CEST 2019 root@mercat1:~ # Jul 21 18:38:33 mercat1 su[2354]: pho to root on /dev/pts/0 20190721 18:38:58 all (1/11): oom.sh 20190721 18:39:00 all (2/11): sort.sh panic: lock (sleep mutex) swapdev not locked @ vm/swap_pager.c:867 cpuid = 1 time = 1563727491 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0006f2b270 vpanic() at vpanic+0x19d/frame 0xfffffe0006f2b2c0 panic() at panic+0x43/frame 0xfffffe0006f2b320 witness_unlock() at witness_unlock+0xea/frame 0xfffffe0006f2b360 __mtx_unlock_flags() at __mtx_unlock_flags+0x4d/frame 0xfffffe0006f2b390 swp_pager_trimswapspace() at swp_pager_trimswapspace+0xeb/frame 0xfffffe0006f2b3e0 swap_pager_putpages() at swap_pager_putpages+0x56c/frame 0xfffffe0006f2b480 vm_pageout_flush() at vm_pageout_flush+0xff/frame 0xfffffe0006f2b580 vm_pageout_cluster() at vm_pageout_cluster+0x3e6/frame 0xfffffe0006f2b7f0 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0x2f6/frame 0xfffffe0006f2ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f2bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f2bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- KDB: enter: panic [ thread pid 30 tid 100130 ] Stopped at kdb_enter+0x3b: movq $0,kdb_why db> run pho db:0:pho> set $lines 20000 db:0:pho> run pho1 db:1:pho1> bt Tracing pid 30 tid 100130 td 0xfffff800049ea000 kdb_enter() at kdb_enter+0x3b/frame 0xfffffe0006f2b270 vpanic() at vpanic+0x1ba/frame 0xfffffe0006f2b2c0 panic() at panic+0x43/frame 0xfffffe0006f2b320 witness_unlock() at witness_unlock+0xea/frame 0xfffffe0006f2b360 __mtx_unlock_flags() at __mtx_unlock_flags+0x4d/frame 0xfffffe0006f2b390 swp_pager_trimswapspace() at swp_pager_trimswapspace+0xeb/frame 0xfffffe0006f2b3e0 swap_pager_putpages() at swap_pager_putpages+0x56c/frame 0xfffffe0006f2b480 vm_pageout_flush() at vm_pageout_flush+0xff/frame 0xfffffe0006f2b580 vm_pageout_cluster() at vm_pageout_cluster+0x3e6/frame 0xfffffe0006f2b7f0 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0x2f6/frame 0xfffffe0006f2ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f2bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f2bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db:1:pho1> show allpcpu Current CPU: 1 cpuid = 0 dynamic pcpu = 0xd60800 curthread = 0xfffff800037ab000: pid 11 tid 100003 "idle: cpu0" curpcb = 0xfffffe0076766b80 fpcurthread = none idlethread = 0xfffff800037ab000: tid 100003 "idle: cpu0" curpmap = 0xffffffff82086d30 tssp = 0xffffffff821be420 commontssp = 0xffffffff821be420 rsp0 = 0xfffffe0076766b80 gs32p = 0xffffffff821c5058 ldt = 0xffffffff821c5098 tss = 0xffffffff821c5088 tlb gen = 2496356 curvnet = 0 spin locks held: cpuid = 1 dynamic pcpu = 0xfffffe0084fea800 curthread = 0xfffff800049ea000: pid 30 tid 100130 "laundry: dom0" curpcb = 0xfffffe0006f2bb80 fpcurthread = none idlethread = 0xfffff800037ac5a0: tid 100004 "idle: cpu1" curpmap = 0xffffffff82086d30 tssp = 0xffffffff821be488 commontssp = 0xffffffff821be488 rsp0 = 0xfffffe0006f2bb80 gs32p = 0xffffffff821c50c0 ldt = 0xffffffff821c5100 tss = 0xffffffff821c50f0 tlb gen = 2497678 curvnet = 0 spin locks held: cpuid = 2 dynamic pcpu = 0xfffffe0084ff6800 curthread = 0xfffff800037ac000: pid 11 tid 100005 "idle: cpu2" curpcb = 0xfffffe0076770b80 fpcurthread = none idlethread = 0xfffff800037ac000: tid 100005 "idle: cpu2" curpmap = 0xffffffff82086d30 tssp = 0xffffffff821be4f0 commontssp = 0xffffffff821be4f0 rsp0 = 0xfffffe0076770b80 gs32p = 0xffffffff821c5128 ldt = 0xffffffff821c5168 tss = 0xffffffff821c5158 tlb gen = 2497582 curvnet = 0 spin locks held: cpuid = 3 dynamic pcpu = 0xfffffe0085002800 curthread = 0xfffff800037ad5a0: pid 11 tid 100006 "idle: cpu3" curpcb = 0xfffffe0076775b80 fpcurthread = none idlethread = 0xfffff800037ad5a0: tid 100006 "idle: cpu3" curpmap = 0xffffffff82086d30 tssp = 0xffffffff821be558 commontssp = 0xffffffff821be558 rsp0 = 0xfffffe0076775b80 gs32p = 0xffffffff821c5190 ldt = 0xffffffff821c51d0 tss = 0xffffffff821c51c0 tlb gen = 2478041 curvnet = 0 spin locks held: cpuid = 4 dynamic pcpu = 0xfffffe008500e800 curthread = 0xfffff800037ad000: pid 11 tid 100007 "idle: cpu4" curpcb = 0xfffffe007677ab80 fpcurthread = none idlethread = 0xfffff800037ad000: tid 100007 "idle: cpu4" curpmap = 0xffffffff82086d30 tssp = 0xffffffff821be5c0 commontssp = 0xffffffff821be5c0 rsp0 = 0xfffffe007677ab80 gs32p = 0xffffffff821c51f8 ldt = 0xffffffff821c5238 tss = 0xffffffff821c5228 tlb gen = 2497646 curvnet = 0 spin locks held: cpuid = 5 dynamic pcpu = 0xfffffe008501a800 curthread = 0xfffff800037ae5a0: pid 11 tid 100008 "idle: cpu5" curpcb = 0xfffffe007677fb80 fpcurthread = none idlethread = 0xfffff800037ae5a0: tid 100008 "idle: cpu5" curpmap = 0xffffffff82086d30 tssp = 0xffffffff821be628 commontssp = 0xffffffff821be628 rsp0 = 0xfffffe007677fb80 gs32p = 0xffffffff821c5260 ldt = 0xffffffff821c52a0 tss = 0xffffffff821c5290 tlb gen = 2497614 curvnet = 0 spin locks held: cpuid = 6 dynamic pcpu = 0xfffffe0085026800 curthread = 0xfffff800037ae000: pid 11 tid 100009 "idle: cpu6" curpcb = 0xfffffe0076784b80 fpcurthread = none idlethread = 0xfffff800037ae000: tid 100009 "idle: cpu6" curpmap = 0xffffffff82086d30 tssp = 0xffffffff821be690 commontssp = 0xffffffff821be690 rsp0 = 0xfffffe0076784b80 gs32p = 0xffffffff821c52c8 ldt = 0xffffffff821c5308 tss = 0xffffffff821c52f8 tlb gen = 2496420 curvnet = 0 spin locks held: cpuid = 7 dynamic pcpu = 0xfffffe0085032800 curthread = 0xfffff8000379a5a0: pid 11 tid 100010 "idle: cpu7" curpcb = 0xfffffe0076789b80 fpcurthread = none idlethread = 0xfffff8000379a5a0: tid 100010 "idle: cpu7" curpmap = 0xffffffff82086d30 tssp = 0xffffffff821be6f8 commontssp = 0xffffffff821be6f8 rsp0 = 0xfffffe0076789b80 gs32p = 0xffffffff821c5330 ldt = 0xffffffff821c5370 tss = 0xffffffff821c5360 tlb gen = 2490051 curvnet = 0 spin locks held: cpuid = 8 dynamic pcpu = 0xfffffe008503e800 curthread = 0xfffff8000379a000: pid 11 tid 100011 "idle: cpu8" curpcb = 0xfffffe007678eb80 fpcurthread = none idlethread = 0xfffff8000379a000: tid 100011 "idle: cpu8" curpmap = 0xffffffff82086d30 tssp = 0xffffffff821be760 commontssp = 0xffffffff821be760 rsp0 = 0xfffffe007678eb80 gs32p = 0xffffffff821c5398 ldt = 0xffffffff821c53d8 tss = 0xffffffff821c53c8 tlb gen = 2497454 curvnet = 0 spin locks held: cpuid = 9 dynamic pcpu = 0xfffffe008504a800 curthread = 0xfffff8000379b5a0: pid 11 tid 100012 "idle: cpu9" curpcb = 0xfffffe0076793b80 fpcurthread = none idlethread = 0xfffff8000379b5a0: tid 100012 "idle: cpu9" curpmap = 0xffffffff82086d30 tssp = 0xffffffff821be7c8 commontssp = 0xffffffff821be7c8 rsp0 = 0xfffffe0076793b80 gs32p = 0xffffffff821c5400 ldt = 0xffffffff821c5440 tss = 0xffffffff821c5430 tlb gen = 2461403 curvnet = 0 spin locks held: cpuid = 10 dynamic pcpu = 0xfffffe0085056800 curthread = 0xfffff8000379b000: pid 11 tid 100013 "idle: cpu10" curpcb = 0xfffffe0076798b80 fpcurthread = none idlethread = 0xfffff8000379b000: tid 100013 "idle: cpu10" curpmap = 0xffffffff82086d30 tssp = 0xffffffff821be830 commontssp = 0xffffffff821be830 rsp0 = 0xfffffe0076798b80 gs32p = 0xffffffff821c5468 ldt = 0xffffffff821c54a8 tss = 0xffffffff821c5498 tlb gen = 2490364 curvnet = 0 spin locks held: cpuid = 11 dynamic pcpu = 0xfffffe0085062800 curthread = 0xfffff8000379c5a0: pid 11 tid 100014 "idle: cpu11" curpcb = 0xfffffe007679db80 fpcurthread = none idlethread = 0xfffff8000379c5a0: tid 100014 "idle: cpu11" curpmap = 0xffffffff82086d30 tssp = 0xffffffff821be898 commontssp = 0xffffffff821be898 rsp0 = 0xfffffe007679db80 gs32p = 0xffffffff821c54d0 ldt = 0xffffffff821c5510 tss = 0xffffffff821c5500 tlb gen = 2497550 curvnet = 0 spin locks held: db:1:pho1> show alllocks Process 30 (pagedaemon) thread 0xfffff800049ea000 (100130) exclusive sleep mutex async swbuf mutex (async swbuf mutex) r = 0 (0xffffffff82049680) locked @ vm/swap_pager.c:903 db:1:pho1> show lockedvnods Locked vnodes db:1:pho1> show allchains chain 1: thread 100268 (pid 3723, sort) sleeping on 0xffffffff81a47e40 "pfault" chain 2: thread 100267 (pid 3722, sort) sleeping on 0xffffffff81a47e40 "pfault" chain 3: thread 100266 (pid 3721, sort) sleeping on 0xffffffff81a47e40 "pfault" chain 4: thread 100265 (pid 3720, sort) sleeping on 0xffffffff81a47e40 "pfault" chain 5: thread 100150 (pid 3719, sort) sleeping on 0xffffffff81a47e40 "pfault" chain 6: thread 100256 (pid 3718, sort) sleeping on 0xffffffff81a47e40 "pfault" chain 7: thread 100261 (pid 3715, sh) sleeping on 0xfffff8002ecb6000 "wait" chain 8: thread 100199 (pid 3714, tee) sleeping on 0xfffff80004980be0 "piperd" chain 9: thread 100187 (pid 3713, sh) sleeping on 0xfffff80003aff000 "wait" chain 10: thread 100237 (pid 2525, sh) sleeping on 0xfffff8002e4835f0 "piperd" chain 11: thread 100236 (pid 2523, awk) sleeping on 0xfffff8003c8a25f0 "piperd" chain 12: thread 100216 (pid 2522, sh) sleeping on 0xfffff801e4758530 "wait" chain 13: thread 100224 (pid 2470, sh) sleeping on 0xfffff801e463e000 "wait" chain 14: thread 100219 (pid 2360, sh) sleeping on 0xfffff801e4ad2530 "wait" chain 15: thread 100156 (pid 2356, bash) sleeping on 0xfffff80006002530 "wait" chain 16: thread 100189 (pid 2354, csh) sleeping on 0xfffff8002eaa75d8 "pause" chain 17: thread 100217 (pid 2353, su) sleeping on 0xfffff801e4c21530 "wait" chain 18: thread 100200 (pid 2349, bash) sleeping on 0xfffff801e463e530 "wait" chain 19: thread 100196 (pid 2348, sshd) sleeping on 0xfffff8002e5bbc40 "select" chain 20: thread 100201 (pid 2345, sshd) sleeping on 0xfffff80003d089c0 "select" chain 21: thread 100170 (pid 2341, csh) sleeping on 0xfffff800064f70b0 "ttyin" chain 22: thread 100214 (pid 2336, login) blocked on lockmgr EXCL thread 0 (pid -512, ) is inactive chain 23: thread 100171 (pid 2335, getty) sleeping on 0xfffff800064f74b0 "ttyin" chain 24: thread 100213 (pid 2334, getty) sleeping on 0xfffff800060fdcb0 "ttyin" chain 25: thread 100212 (pid 2333, getty) sleeping on 0xfffff800060fc0b0 "ttyin" chain 26: thread 100211 (pid 2332, getty) sleeping on 0xfffff800060fc4b0 "ttyin" chain 27: thread 100210 (pid 2331, getty) sleeping on 0xfffff800060fc8b0 "ttyin" chain 28: thread 100209 (pid 2330, getty) sleeping on 0xfffff800060fccb0 "ttyin" chain 29: thread 100208 (pid 2329, getty) sleeping on 0xfffff800060fb0b0 "ttyin" chain 30: thread 100168 (pid 2328, getty) sleeping on 0xfffff800060fb4b0 "ttyin" chain 31: thread 100142 (pid 2327, getty) sleeping on 0xfffff800060fb8b0 "ttyin" chain 32: thread 100202 (pid 2277, sshd) sleeping on 0xfffff80003f0cd40 "select" chain 33: thread 100198 (pid 2248, ntpd) sleeping on 0xfffff80003e2d9c0 "select" chain 34: thread 100275 (pid 2248, ntpd) sleeping on 0xfffff80003ae6100 "usem" chain 35: thread 100195 (pid 2187, NLM: master) sleeping on 0xfffff801e4862560 "rpcsvc" chain 36: thread 100190 (pid 2184, rpc.statd) sleeping on 0xfffff80006401cc0 "select" chain 37: thread 100162 (pid 2138, watchdogd) sleeping on 0xffffffff81e8f0b1 "nanslp" chain 38: thread 100159 (pid 2058, rpcbind) sleeping on 0xfffff80003788d40 "select" chain 39: thread 100145 (pid 2048, syslogd) sleeping on 0xfffff800048b5f40 "select" chain 40: thread 100174 (pid 1977, devd) sleeping on 0xfffff80006401dc0 "select" chain 41: thread 100149 (pid 1972, dhclient) sleeping on 0xfffff80003f0ce40 "select" chain 42: thread 100161 (pid 1928, dhclient) sleeping on 0xfffff80003d08b40 "select" chain 43: thread 100164 (pid 1925, dhclient) sleeping on 0xfffff80003e2dcc0 "select" chain 44: thread 100141 (pid 34, syncer) sleeping on 0xffffffff820315c0 "syncer" chain 45: thread 100140 (pid 33, vnlru) sleeping on 0xfffff8002e529000 "vlruwt" chain 46: thread 100132 (pid 32, bufdaemon) sleeping on 0xffffffff82030a58 "qsleep" chain 47: thread 100133 (pid 32, bufspacedaemon-0) sleeping on 0xffffffff81a0a8c0 "-" chain 48: thread 100134 (pid 32, bufspacedaemon-1) sleeping on 0xffffffff81a12b80 "-" chain 49: thread 100135 (pid 32, bufspacedaemon-2) sleeping on 0xffffffff81a1ae40 "-" chain 50: thread 100136 (pid 32, bufspacedaemon-3) sleeping on 0xffffffff81a23100 "-" chain 51: thread 100137 (pid 32, bufspacedaemon-4) sleeping on 0xffffffff81a2b3c0 "-" chain 52: thread 100138 (pid 32, bufspacedaemon-5) sleeping on 0xffffffff81a33680 "-" chain 53: thread 100139 (pid 32, bufspacedaemon-6) sleeping on 0xffffffff81a3b940 "-" chain 54: thread 100154 (pid 32, / worker) blocked on lockmgr 0R.EXCL thread 0 (pid -1, ) is inactive chain 55: thread 100197 (pid 32, /tmp worker) sleeping on 0xfffff8000616d4e8 "sdflush" chain 56: thread 100129 (pid 31, vmdaemon) sleeping on 0xffffffff8204d788 "psleep" chain 57: thread 100128 (pid 30, dom0) sleeping on 0xffffffff81e8de47 "pwait" chain 58: thread 100131 (pid 30, uma) sleeping on 0xffffffff80f00fb0 "umarcl" chain 59: thread 100127 (pid 29, enc_daemon1) sleeping on 0xfffff80003af6000 "idle" chain 60: thread 100126 (pid 28, enc_daemon0) sleeping on 0xfffff80003af6530 "idle" chain 61: thread 100123 (pid 27, rand_harvestq) sleeping on 0xffffffff81cee7a0 "-" chain 62: thread 100119 (pid 26, sctp_iterator) sleeping on 0xffffffff820b5c30 "waiting_for_work" chain 63: thread 100118 (pid 25, soaiod4) sleeping on 0xffffffff8203045c "-" chain 64: thread 100117 (pid 24, soaiod3) sleeping on 0xffffffff8203045c "-" chain 65: thread 100116 (pid 23, soaiod2) sleeping on 0xffffffff8203045c "-" chain 66: thread 100115 (pid 22, soaiod1) sleeping on 0xffffffff8203045c "-" chain 67: thread 100096 (pid 21, usbus0) sleeping on 0xfffffe009a1aa460 "-" chain 68: thread 100097 (pid 21, usbus0) sleeping on 0xfffffe009a1aa4b8 "-" chain 69: thread 100098 (pid 21, usbus0) sleeping on 0xfffffe009a1aa510 "-" chain 70: thread 100099 (pid 21, usbus0) sleeping on 0xfffffe009a1aa568 "-" chain 71: thread 100100 (pid 21, usbus0) sleeping on 0xfffffe009a1aa5c0 "-" chain 72: thread 100102 (pid 21, usbus1) sleeping on 0xfffffe0006dd6d10 "-" chain 73: thread 100103 (pid 21, usbus1) sleeping on 0xfffffe0006dd6d68 "-" chain 74: thread 100104 (pid 21, usbus1) sleeping on 0xfffffe0006dd6dc0 "-" chain 75: thread 100105 (pid 21, usbus1) sleeping on 0xfffffe0006dd6e18 "-" chain 76: thread 100106 (pid 21, usbus1) sleeping on 0xfffffe0006dd6e70 "-" chain 77: thread 100107 (pid 21, usbus2) sleeping on 0xfffffe009a63dd10 "-" chain 78: thread 100108 (pid 21, usbus2) sleeping on 0xfffffe009a63dd68 "-" chain 79: thread 100109 (pid 21, usbus2) sleeping on 0xfffffe009a63ddc0 "-" chain 80: thread 100110 (pid 21, usbus2) sleeping on 0xfffffe009a63de18 "-" chain 81: thread 100111 (pid 21, usbus2) sleeping on 0xfffffe009a63de70 "-" chain 82: thread 100091 (pid 20, doneq0) sleeping on 0xffffffff81bc9b40 "-" chain 83: thread 100092 (pid 20, doneq1) sleeping on 0xffffffff81bc9bc0 "-" chain 84: thread 100093 (pid 20, doneq2) sleeping on 0xffffffff81bc9c40 "-" chain 85: thread 100125 (pid 20, scanner) sleeping on 0xffffffff81bc9a08 "-" chain 86: thread 100090 (pid 19, crypto returns 11) sleeping on 0xfffff80003fe1450 "crypto_ret_wait" chain 87: thread 100089 (pid 18, crypto returns 10) sleeping on 0xfffff80003fe13f0 "crypto_ret_wait" chain 88: thread 100088 (pid 17, crypto returns 9) sleeping on 0xfffff80003fe1390 "crypto_ret_wait" chain 89: thread 100087 (pid 16, crypto returns 8) sleeping on 0xfffff80003fe1330 "crypto_ret_wait" chain 90: thread 100086 (pid 15, crypto returns 7) sleeping on 0xfffff80003fe12d0 "crypto_ret_wait" chain 91: thread 100085 (pid 9, crypto returns 6) sleeping on 0xfffff80003fe1270 "crypto_ret_wait" chain 92: thread 100084 (pid 8, crypto returns 5) sleeping on 0xfffff80003fe1210 "crypto_ret_wait" chain 93: thread 100083 (pid 7, crypto returns 4) sleeping on 0xfffff80003fe11b0 "crypto_ret_wait" chain 94: thread 100082 (pid 6, crypto returns 3) sleeping on 0xfffff80003fe1150 "crypto_ret_wait" chain 95: thread 100081 (pid 5, crypto returns 2) sleeping on 0xfffff80003fe10f0 "crypto_ret_wait" chain 96: thread 100080 (pid 4, crypto returns 1) sleeping on 0xfffff80003fe1090 "crypto_ret_wait" chain 97: thread 100079 (pid 3, crypto returns 0) sleeping on 0xfffff80003fe1030 "crypto_ret_wait" chain 98: thread 100078 (pid 2, crypto) sleeping on 0xffffffff82047df8 "crypto_wait" chain 99: thread 100065 (pid 14, sequencer 00) sleeping on 0xfffff8000392ec88 "seqstate" chain 100: thread 100062 (pid 13, g_event) sleeping on 0xffffffff820860c0 "-" chain 101: thread 100063 (pid 13, g_up) sleeping on 0xffffffff820860d0 "-" chain 102: thread 100064 (pid 13, g_down) sleeping on 0xffffffff820860b8 "-" chain 103: thread 100002 (pid 1, init) sleeping on 0xfffff800037af530 "wait" chain 104: thread 100001 (pid 10, audit) sleeping on 0xffffffff820b6950 "audit_worker_cv" chain 105: thread 100000 (pid 0, swapper) sleeping on 0xffffffff82086100 "swapin" chain 106: thread 100015 (pid 0, kqueue_ctx taskq) sleeping on 0xfffff8000369de00 "-" chain 107: thread 100016 (pid 0, aiod_kick taskq) sleeping on 0xfffff8000369dd00 "-" chain 108: thread 100019 (pid 0, thread taskq) sleeping on 0xfffff8000369da00 "-" chain 109: thread 100021 (pid 0, config_0) sleeping on 0xfffff80003130c00 "-" chain 110: thread 100022 (pid 0, if_io_tqg_0) sleeping on 0xfffff80003130b00 "-" chain 111: thread 100023 (pid 0, if_io_tqg_1) sleeping on 0xfffff80003130a00 "-" chain 112: thread 100024 (pid 0, if_io_tqg_2) sleeping on 0xfffff80003130900 "-" chain 113: thread 100025 (pid 0, if_io_tqg_3) sleeping on 0xfffff80003130800 "-" chain 114: thread 100026 (pid 0, if_io_tqg_4) sleeping on 0xfffff80003130700 "-" chain 115: thread 100027 (pid 0, if_io_tqg_5) sleeping on 0xfffff80003130600 "-" chain 116: thread 100028 (pid 0, if_io_tqg_6) sleeping on 0xfffff80003130500 "-" chain 117: thread 100029 (pid 0, if_io_tqg_7) sleeping on 0xfffff80003130400 "-" chain 118: thread 100030 (pid 0, if_io_tqg_8) sleeping on 0xfffff80003130300 "-" chain 119: thread 100031 (pid 0, if_io_tqg_9) sleeping on 0xfffff80003130200 "-" chain 120: thread 100032 (pid 0, if_io_tqg_10) sleeping on 0xfffff80003130100 "-" chain 121: thread 100033 (pid 0, if_io_tqg_11) sleeping on 0xfffff80003130000 "-" chain 122: thread 100034 (pid 0, softirq_0) sleeping on 0xfffff8000312fe00 "-" chain 123: thread 100035 (pid 0, softirq_1) sleeping on 0xfffff8000312fd00 "-" chain 124: thread 100036 (pid 0, softirq_2) sleeping on 0xfffff8000312fc00 "-" chain 125: thread 100037 (pid 0, softirq_3) sleeping on 0xfffff8000312fb00 "-" chain 126: thread 100038 (pid 0, softirq_4) sleeping on 0xfffff8000312fa00 "-" chain 127: thread 100039 (pid 0, softirq_5) sleeping on 0xfffff8000312f900 "-" chain 128: thread 100040 (pid 0, softirq_6) sleeping on 0xfffff8000312f800 "-" chain 129: thread 100041 (pid 0, softirq_7) sleeping on 0xfffff8000312f700 "-" chain 130: thread 100042 (pid 0, softirq_8) sleeping on 0xfffff8000312f600 "-" chain 131: thread 100043 (pid 0, softirq_9) sleeping on 0xfffff8000312f500 "-" chain 132: thread 100044 (pid 0, softirq_10) sleeping on 0xfffff8000312f400 "-" chain 133: thread 100045 (pid 0, softirq_11) sleeping on 0xfffff8000312f300 "-" chain 134: thread 100046 (pid 0, if_config_tqg_0) sleeping on 0xfffff800039d3100 "-" chain 135: thread 100061 (pid 0, firmware taskq) sleeping on 0xfffff8000393b100 "-" chain 136: thread 100066 (pid 0, crypto_0) sleeping on 0xfffff8000393b000 "-" chain 137: thread 100067 (pid 0, crypto_1) sleeping on 0xfffff8000393b000 "-" chain 138: thread 100068 (pid 0, crypto_2) sleeping on 0xfffff8000393b000 "-" chain 139: thread 100069 (pid 0, crypto_3) sleeping on 0xfffff8000393b000 "-" chain 140: thread 100070 (pid 0, crypto_4) sleeping on 0xfffff8000393b000 "-" chain 141: thread 100071 (pid 0, crypto_5) sleeping on 0xfffff8000393b000 "-" chain 142: thread 100072 (pid 0, crypto_6) sleeping on 0xfffff8000393b000 "-" chain 143: thread 100073 (pid 0, crypto_7) sleeping on 0xfffff8000393b000 "-" chain 144: thread 100074 (pid 0, crypto_8) sleeping on 0xfffff8000393b000 "-" chain 145: thread 100075 (pid 0, crypto_9) sleeping on 0xfffff8000393b000 "-" chain 146: thread 100076 (pid 0, crypto_10) sleeping on 0xfffff8000393b000 "-" chain 147: thread 100077 (pid 0, crypto_11) sleeping on 0xfffff8000393b000 "-" chain 148: thread 100114 (pid 0, mca taskq) blocked on lockmgr (null)EXCL thread 0 (pid 1970434661, ) is inactive chain 149: thread 100120 (pid 0, acpi_task_0) sleeping on 0xfffff8002e4eb600 "-" chain 150: thread 100121 (pid 0, acpi_task_1) sleeping on 0xfffff8002e4eb600 "-" chain 151: thread 100122 (pid 0, acpi_task_2) sleeping on 0xfffff8002e4eb600 "-" chain 152: thread 100124 (pid 0, CAM taskq) sleeping on 0xfffff8000393ce00 "-" db:1:pho1> show mount 0xfffff80003720000 /dev/gpt/work on / (ufs) 0xfffff8000371f000 devfs on /dev (devfs) 0xfffff801e4746000 /dev/gpt/tmp on /tmp (ufs) More info: show mount db:1:pho1> show bufqueues bqempty: 104450 Buf domain 0 freebufs 15014 lofreebufs 219 hifreebufs 329 bufspace 1961984 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 49 (880640) dirtyq count 0 (0) wakeup 0 lim 25 CPU 9, 6, 3, 2, 7, 6, 8, 6, 11, 3, 6, 3, Locked buffers: 0 space 0 Total buffers: 133 space 1961984 Buf domain 1 freebufs 14878 lofreebufs 219 hifreebufs 329 bufspace 6217728 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 8 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 134 (3182592) dirtyq count 8 (262144) wakeup 0 lim 25 CPU 2, 0, 14, 10, 20, 13, 3, 14, 15, 11, 5, 6, Locked buffers: 0 space 0 Total buffers: 262 space 6217728 Buf domain 2 freebufs 14835 lofreebufs 219 hifreebufs 329 bufspace 7811072 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 199 (5402624) dirtyq count 0 (0) wakeup 0 lim 25 CPU 15, 16, 11, 5, 8, 6, 2, 2, 12, 11, 7, 4, Locked buffers: 0 space 0 Total buffers: 298 space 7811072 Buf domain 3 freebufs 14937 lofreebufs 219 hifreebufs 329 bufspace 4284416 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 124 (2744320) dirtyq count 0 (0) wakeup 0 lim 25 CPU 3, 3, 3, 7, 7, 4, 1, 6, 7, 21, 7, 3, Locked buffers: 0 space 0 Total buffers: 197 space 4284416 Buf domain 4 freebufs 14994 lofreebufs 219 hifreebufs 329 bufspace 2514944 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 50 (770048) dirtyq count 0 (0) wakeup 0 lim 25 CPU 15, 4, 6, 4, 0, 3, 3, 18, 18, 10, 6, 2, Locked buffers: 0 space 0 Total buffers: 139 space 2514944 Buf domain 5 freebufs 14902 lofreebufs 219 hifreebufs 329 bufspace 5505024 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 124 (3289088) dirtyq count 0 (0) wakeup 0 lim 25 CPU 24, 3, 4, 3, 15, 5, 3, 14, 12, 9, 9, 6, Locked buffers: 0 space 0 Total buffers: 231 space 5505024 Buf domain 6 freebufs 14995 lofreebufs 219 hifreebufs 329 bufspace 2519040 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 49 (856064) dirtyq count 0 (0) wakeup 0 lim 25 CPU 4, 3, 2, 2, 7, 7, 1, 6, 23, 11, 20, 3, Locked buffers: 0 space 0 Total buffers: 138 space 2519040 db:1:pho1> show page vm_cnt.v_free_count: 32959 vm_cnt.v_inactive_count: 1 vm_cnt.v_active_count: 5051391 vm_cnt.v_laundry_count: 2820659 vm_cnt.v_wire_count: 223186 vm_cnt.v_free_reserved: 10649 vm_cnt.v_free_min: 51289 vm_cnt.v_free_target: 173209 vm_cnt.v_inactive_target: 259813 db:1:pho1> show pageq pq_free 32959 dom 0 page_cnt 8128423 free 32959 pq_act 5051391 pq_inact 1 pq_laund 2820659 pq_unsw 0 db:0:pho> show freepages DOMAIN: 0 FREE LIST 0: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000000 | 000000 10 (004096K) | 000000 | 000000 09 (002048K) | 000000 | 000000 08 (001024K) | 000005 | 000000 07 (000512K) | 000014 | 000001 06 (000256K) | 000001 | 000001 05 (000128K) | 000002 | 000000 04 (000064K) | 000002 | 000000 03 (000032K) | 000004 | 000001 02 (000016K) | 000009 | 000001 01 (000008K) | 000037 | 000004 00 (000004K) | 000090 | 000007 FREE LIST 1: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000000 | 000000 10 (004096K) | 000000 | 000000 09 (002048K) | 000000 | 000000 08 (001024K) | 000000 | 000000 07 (000512K) | 000000 | 000000 06 (000256K) | 000000 | 000000 05 (000128K) | 000000 | 000000 04 (000064K) | 000000 | 000000 03 (000032K) | 000000 | 000000 02 (000016K) | 000000 | 000000 01 (000008K) | 000000 | 000000 00 (000004K) | 000000 | 000000 db:0:pho> show uma Zone Size Used Free Requests Sleeps Bucket FFS2 dinode 256 1054 191 1335 0 61 FFS1 dinode 128 0 0 0 0 125 FFS inode 160 1054 290 1336 0 61 swblk 136 919478 25 919489 0 61 swpctrie 144 61588 26 61591 0 61 selfd 64 79 727 8981 0 253 rtentry 208 12 140 15 0 61 IPsec SA lft_c 16 0 0 0 0 253 ripcb 488 1 7 3 0 29 udplite_inpcb 488 0 0 0 0 29 sctp_asconf_ack 48 0 0 0 0 253 sctp_asconf 40 0 0 0 0 253 sctp_stream_msg_out 112 0 0 0 0 125 sctp_readq 152 0 0 0 0 61 sctp_chunk 152 0 0 0 0 61 sctp_raddr 736 0 0 0 0 13 sctp_laddr 48 0 0 4 0 253 sctp_asoc 2408 0 0 0 0 3 sctp_ep 1280 0 0 0 0 9 tcp_log_node 120 0 0 0 0 125 tcp_log_bucket 144 0 0 0 0 61 tcp_log 408 0 0 0 0 29 tcpreass 48 0 0 0 0 253 tfo_ccache_entries 80 0 0 0 0 125 tfo 4 0 0 0 0 253 sackhole 32 0 0 0 0 253 hostcache 96 0 0 0 0 125 syncache 168 0 0 1 0 61 tcptw 88 0 0 0 0 125 tcpcb 976 9 7 14 0 13 tcp_inpcb 488 9 23 14 0 29 udpcb 32 20 600 189 0 253 udp_inpcb 488 20 52 189 0 29 ipq 56 0 0 0 0 253 unpcb 256 19 131 4943 0 61 socket 872 51 2985 5159 0 13 KNOTE 160 0 0 4 0 61 itimer 352 0 0 0 0 29 ksiginfo 112 129 571 749 0 125 pipe 760 7 23 2247 0 13 AIOLIO 280 0 0 0 0 29 AIOCB 752 0 0 0 0 13 AIOP 32 0 0 0 0 253 AIO 208 0 0 0 0 61 Mountpoints 2896 3 6 4 0 3 procdesc 136 1 28 10 0 61 NCLNODE 744 0 0 0 0 13 DIRHASH 1024 0 0 122 0 13 LTS VFS Cache 368 0 0 0 0 29 L VFS Cache 328 0 0 0 0 29 STS VFS Cache 148 0 0 0 0 61 S VFS Cache 108 1124 311 2537 0 125 rentr 24 0 0 2 0 253 NAMEI 1024 0 0 74075 0 13 BUF TRIE 144 214 105734 723 0 61 VNODEPOLL 120 0 0 0 0 125 VNODE 632 1163 61 1448 0 13 vtnet_tx_hdr 24 0 0 0 0 253 crypto_session 24 0 0 0 0 253 cryptodesc 120 0 0 0 0 125 cryptop 128 0 0 0 0 125 nvme_request 128 0 0 0 0 125 g_bio 408 4 14 1994656 0 29 FPU_save_area 832 0 0 0 0 13 DMAR_MAP_ENTRY 120 0 0 0 0 125 ttyoutq 256 173 142 282 0 61 ttyinq 160 330 222 540 0 61 epoch_record pcpu 256 4 12 4 0 61 mbuf_extpgs 256 0 0 0 0 61 mbuf_jumbo_16k 16384 0 0 0 0 1 mbuf_jumbo_9k 9216 0 0 0 0 1 mbuf_jumbo_page 4096 0 0 2 0 1 mbuf_cluster 2048 3886 0 6520 0 5 mbuf 256 3894 2 6267 0 253 mbuf_packet 256 4 0 175 0 253 audit_record 1280 0 0 0 0 9 domainset 40 0 0 0 0 253 cpuset 104 27 407 27 0 125 THREAD 1440 272 44 275 0 9 PROC 1328 79 101 4807 0 9 MAC labels 40 0 0 0 0 253 pbuf 984 0 1010 0 0 1 clpbuf 984 0 0 30 0 5 vnpbuf 984 0 0 0 0 13 mdpbuf 984 0 0 0 0 1 nfspbuf 984 0 0 0 0 5 swwbuf 984 1 0 492895 0 1 swrbuf 984 0 0 40 0 5 umtx_shm 88 0 0 0 0 125 umtx pi 96 0 0 0 0 125 rl_entry 40 120 1068 120 0 253 TURNSTILE 136 317 543 317 0 61 rangeset pctrie nodes 144 0 0 0 0 61 filedesc0 1104 80 52 4808 0 9 Files 80 131 2025 39971 0 125 SLEEPQUEUE 88 317 427 317 0 125 65536 65536 0 0 0 0 1 65536 65536 0 0 0 0 1 65536 65536 1 0 1 0 1 65536 65536 2 0 60 0 1 65536 65536 3 0 2157 0 1 65536 65536 0 0 0 0 1 65536 65536 3 0 3 0 1 65536 65536 3 0 29 0 1 32768 32768 0 0 8 0 1 32768 32768 12 0 12 0 1 32768 32768 3 0 4 0 1 32768 32768 0 0 0 0 1 32768 32768 1 0 2491 0 1 32768 32768 12 0 12 0 1 32768 32768 1 0 1 0 1 32768 32768 10 0 68 0 1 16384 16384 0 0 12 0 1 16384 16384 5 0 5 0 1 16384 16384 37 0 37 0 1 16384 16384 5 0 227 0 1 16384 16384 14 0 2847 0 1 16384 16384 0 0 0 0 1 16384 16384 2 0 2 0 1 16384 16384 1 0 1 0 1 8192 8192 0 0 36 0 1 8192 8192 88 0 89 0 1 8192 8192 14 0 16 0 1 8192 8192 2 0 2 0 1 8192 8192 14 0 1674 0 1 8192 8192 12 0 12 0 1 8192 8192 0 0 0 0 1 8192 8192 2 0 3 0 1 4096 4096 0 0 4 0 1 4096 4096 2 0 339 0 1 4096 4096 46 0 90058 0 1 4096 4096 16 0 47 0 1 4096 4096 369 0 2615 0 1 4096 4096 0 0 11 0 1 4096 4096 0 0 1 0 1 4096 4096 88 0 4816 0 1 2048 2048 0 0 20 0 5 2048 2048 3 3 4 0 5 2048 2048 99 7 102 0 5 2048 2048 2 2 45 0 5 2048 2048 10 2 508 0 5 2048 2048 0 0 0 0 5 2048 2048 2 2 2 0 5 2048 2048 20 14 498133 0 5 1024 1024 2 6 56 0 13 1024 1024 18 18 118 0 13 1024 1024 58 34 1216 0 13 1024 1024 16 4 11559 0 13 1024 1024 185 47 8069 0 13 1024 1024 0 0 0 0 13 1024 1024 1 3 1 0 13 1024 1024 4 12 26 0 13 512 512 12 9 210 0 29 512 512 49 56 50 0 29 512 512 147 98 265 0 29 512 512 11 31 3536 0 29 512 512 22 55 6179 0 29 512 512 52 74 52 0 29 512 512 19 51 38 0 29 512 512 192 60 279 0 29 256 256 28 137 929 0 61 256 256 0 0 75 0 61 256 256 663 222 4090 0 61 256 256 10 80 3457 0 61 256 256 262 233 11227 0 61 256 256 302 178 5090 0 61 256 256 59 181 412 0 61 256 256 35 100 44 0 61 128 128 40 363 200 0 125 128 128 45 172 100 0 125 128 128 1424 529 31286 0 125 128 128 2759 372 6435 0 125 128 128 14522 420 884497 0 125 128 128 336 253 700 0 125 128 128 41 207 43 0 125 128 128 74 453 98 0 125 64 64 36 584 70860 0 253 64 64 53 381 70 0 253 64 64 9331 1022 15282 0 253 64 64 636 728 685 0 253 64 64 9326 532 17263 0 253 64 64 84 784 4854 0 253 64 64 184 64 195 0 253 64 64 51 321 192 0 253 32 32 9 487 110 0 253 32 32 33 959 280 0 253 32 32 396 1960 11523 0 253 32 32 3237 1103 3256 0 253 32 32 686 1174 24553 0 253 32 32 68 800 75 0 253 32 32 242 502 307 0 253 32 32 356 1876 476 0 253 16 16 36 2205 194 0 253 16 16 0 0 19 0 253 16 16 520 2717 19839 0 253 16 16 2860 2120 3094 0 253 16 16 526 2213 851145 0 253 16 16 2 496 3 0 253 16 16 19 1475 51 0 253 16 16 27 1218 56 0 253 mt_zone 24 428 402 428 0 253 mt_stats_zone 64 428 84 428 0 253 64 pcpu 8 3027 1581 3312 0 253 fakepg 104 0 266 8 0 125 VMSPACE 2552 46 56 4775 0 3 MAP ENTRY 112 1767 2993 260045 0 125 KMAP ENTRY 112 7 131 7 0 125 MAP 232 2 65 2 0 61 RADIX NODE 144 531980 205 1773110 0 61 VM OBJECT 256 1731 1089 83023 0 61 vmem btag 56 11968 1148 11990 0 253 vmem 1856 0 1 0 0 5 256 Bucket 2048 -1 0 10163 0 5 128 Bucket 1024 0 0 5077 0 13 64 Bucket 512 0 0 23850 0 29 32 Bucket 256 -1 0 8359 0 61 16 Bucket 128 14 326 616 0 125 12 Bucket 96 0 0 832 0 125 8 Bucket 64 -1 0 3031 0 253 6 Bucket 48 0 0 672 0 253 4 Bucket 32 25 1463 15851 0 253 UMA Hash 256 1 10 7 0 0 UMA Slabs 112 3355 1044 9336 0 0 UMA Zones 1960 82 0 82 0 0 UMA Kegs 216 75 4 75 0 0 db:0:pho> show umacache Zone Size Used Free Requests Bucket vm pgcache 104 34292 0 267235 253 vm pgcache 104 10755 0 822281 253 buf free cache 984 1377 105 2150 13 buffer arena-65 65536 0 0 659 1 buffer arena-61 61440 0 0 659 1 buffer arena-57 57344 0 0 659 1 buffer arena-53 53248 0 0 659 1 buffer arena-49 49152 0 0 659 1 buffer arena-45 45056 0 0 659 1 buffer arena-40 40960 0 0 659 1 buffer arena-36 36864 0 0 659 1 buffer arena-32 32768 276 7 1014 1 buffer arena-28 28672 0 0 659 1 buffer arena-24 24576 0 0 659 1 buffer arena-20 20480 0 0 659 1 buffer arena-16 16384 3 2 669 1 buffer arena-12 12288 0 0 659 1 buffer arena-81 8192 0 0 659 1 buffer arena-40 4096 0 0 659 1 db:0:pho> acttrace Tracing command pagedaemon pid 30 tid 100130 td 0xfffff800049ea000 (CPU 1) kdb_enter() at kdb_enter+0x3b/frame 0xfffffe0006f2b270 vpanic() at vpanic+0x1ba/frame 0xfffffe0006f2b2c0 panic() at panic+0x43/frame 0xfffffe0006f2b320 witness_unlock() at witness_unlock+0xea/frame 0xfffffe0006f2b360 __mtx_unlock_flags() at __mtx_unlock_flags+0x4d/frame 0xfffffe0006f2b390 swp_pager_trimswapspace() at swp_pager_trimswapspace+0xeb/frame 0xfffffe0006f2b3e0 swap_pager_putpages() at swap_pager_putpages+0x56c/frame 0xfffffe0006f2b480 vm_pageout_flush() at vm_pageout_flush+0xff/frame 0xfffffe0006f2b580 vm_pageout_cluster() at vm_pageout_cluster+0x3e6/frame 0xfffffe0006f2b7f0 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0x2f6/frame 0xfffffe0006f2ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f2bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f2bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100003 td 0xfffff800037ab000 (CPU 0) cpustop_handler() at cpustop_handler+0x2d/frame 0xffffffff82051af0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xffffffff82051b10 trap() at trap+0x48/frame 0xffffffff82051c20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff82051c20 --- trap 0x13, rip = 0xffffffff8120f3c6, rsp = 0xfffffe0076766900, rbp = 0xfffffe0076766900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076766900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076766950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076766970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe0076766990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe0076766a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076766ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076766ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffff800037ac000 (CPU 2) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006b7fdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006b7fe10 trap() at trap+0x48/frame 0xfffffe0006b7ff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b7ff20 --- trap 0x13, rip = 0xffffffff8120f3c6, rsp = 0xfffffe0076770900, rbp = 0xfffffe0076770900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076770900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076770950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076770970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe0076770990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe0076770a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076770ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076770ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xfffff800037ad5a0 (CPU 3) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006b8bdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006b8be10 trap() at trap+0x48/frame 0xfffffe0006b8bf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b8bf20 --- trap 0x13, rip = 0xffffffff8120f3c6, rsp = 0xfffffe0076775900, rbp = 0xfffffe0076775900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076775900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076775950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076775970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe0076775990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe0076775a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076775ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076775ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100007 td 0xfffff800037ad000 (CPU 4) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006b97df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006b97e10 trap() at trap+0x48/frame 0xfffffe0006b97f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b97f20 --- trap 0x13, rip = 0xffffffff8120f3c6, rsp = 0xfffffe007677a900, rbp = 0xfffffe007677a900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007677a900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe007677a950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007677a970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe007677a990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe007677aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007677aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007677aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xfffff800037ae5a0 (CPU 5) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006ba3df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006ba3e10 trap() at trap+0x48/frame 0xfffffe0006ba3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006ba3f20 --- trap 0x13, rip = 0xffffffff8120f3c6, rsp = 0xfffffe007677f900, rbp = 0xfffffe007677f900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007677f900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe007677f950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007677f970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe007677f990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe007677fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007677fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007677fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xfffff800037ae000 (CPU 6) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bafdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bafe10 trap() at trap+0x48/frame 0xfffffe0006baff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006baff20 --- trap 0x13, rip = 0xffffffff8120f3c6, rsp = 0xfffffe0076784900, rbp = 0xfffffe0076784900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076784900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076784950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076784970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe0076784990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe0076784a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076784ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076784ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xfffff8000379a5a0 (CPU 7) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bbbdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bbbe10 trap() at trap+0x48/frame 0xfffffe0006bbbf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bbbf20 --- trap 0x13, rip = 0xffffffff8120f3c6, rsp = 0xfffffe0076789900, rbp = 0xfffffe0076789900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076789900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076789950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076789970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe0076789990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe0076789a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076789ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076789ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffff8000379a000 (CPU 8) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bc7df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bc7e10 trap() at trap+0x48/frame 0xfffffe0006bc7f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bc7f20 --- trap 0x13, rip = 0xffffffff8120f3c6, rsp = 0xfffffe007678e900, rbp = 0xfffffe007678e900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007678e900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe007678e950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007678e970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe007678e990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe007678ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe007678eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007678eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100012 td 0xfffff8000379b5a0 (CPU 9) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bd3df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bd3e10 trap() at trap+0x48/frame 0xfffffe0006bd3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bd3f20 --- trap 0x13, rip = 0xffffffff8120f3c6, rsp = 0xfffffe0076793900, rbp = 0xfffffe0076793900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076793900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076793950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076793970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe0076793990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe0076793a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076793ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076793ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xfffff8000379b000 (CPU 10) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bdfdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bdfe10 trap() at trap+0x48/frame 0xfffffe0006bdff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bdff20 --- trap 0x13, rip = 0xffffffff8120f3c6, rsp = 0xfffffe0076798900, rbp = 0xfffffe0076798900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076798900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076798950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076798970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe0076798990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe0076798a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076798ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076798ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100014 td 0xfffff8000379c5a0 (CPU 11) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bebdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bebe10 trap() at trap+0x48/frame 0xfffffe0006bebf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bebf20 --- trap 0x13, rip = 0xffffffff8120f3c6, rsp = 0xfffffe007679d900, rbp = 0xfffffe007679d900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007679d900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe007679d950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007679d970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe007679d990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe007679da70 fork_exit() at fork_exit+0x84/frame 0xfffffe007679dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007679dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db:0:pho> ps pid ppid pgrp uid state wmesg wchan cmd 4784 2522 2470 0 ?W sleep 4752 3715 2360 0 ?W+ sleep 3723 3715 2360 0 D+ pfault 0xffffffff81a47e40 sort 3722 3715 2360 0 D+ pfault 0xffffffff81a47e40 sort 3721 3715 2360 0 D+ pfault 0xffffffff81a47e40 sort 3720 3715 2360 0 D+ pfault 0xffffffff81a47e40 sort 3719 3715 2360 0 D+ pfault 0xffffffff81a47e40 sort 3718 3715 2360 0 D+ pfault 0xffffffff81a47e40 sort 3715 3713 2360 0 SW+ wait 0xfffff8002ecb6000 sh 3714 2360 2360 0 S+ piperd 0xfffff80004980be0 tee 3713 2360 2360 0 SW+ wait 0xfffff80003aff000 sh 2525 2470 2470 0 S piperd 0xfffff8002e4835f0 sh 2523 2470 2470 0 S piperd 0xfffff8003c8a25f0 awk 2522 2470 2470 0 SW wait 0xfffff801e4758530 sh 2470 1 2470 0 SWs wait 0xfffff801e463e000 sh 2360 2356 2360 0 SW+ wait 0xfffff801e4ad2530 sh 2356 2354 2356 0 SW+ wait 0xfffff80006002530 bash 2354 2353 2354 0 SW+ pause 0xfffff8002eaa75d8 csh 2353 2349 2353 2006 SW+ wait 0xfffff801e4c21530 su 2349 2348 2349 2006 SWs+ wait 0xfffff801e463e530 bash 2348 2345 2345 2006 S select 0xfffff8002e5bbc40 sshd 2345 2277 2345 0 Ss select 0xfffff80003d089c0 sshd 2341 2336 2341 0 S+ ttyin 0xfffff800064f70b0 csh 2336 1 2336 0 SWs+ wait 0xfffff801e4ad2a60 login 2335 1 2335 0 Ss+ ttyin 0xfffff800064f74b0 getty 2334 1 2334 0 Ss+ ttyin 0xfffff800060fdcb0 getty 2333 1 2333 0 Ss+ ttyin 0xfffff800060fc0b0 getty 2332 1 2332 0 Ss+ ttyin 0xfffff800060fc4b0 getty 2331 1 2331 0 Ss+ ttyin 0xfffff800060fc8b0 getty 2330 1 2330 0 Ss+ ttyin 0xfffff800060fccb0 getty 2329 1 2329 0 Ss+ ttyin 0xfffff800060fb0b0 getty 2328 1 2328 0 Ss+ ttyin 0xfffff800060fb4b0 getty 2327 1 2327 0 Ss+ ttyin 0xfffff800060fb8b0 getty 2281 1 2281 0 ?Ws cron 2277 1 2277 0 Ss select 0xfffff80003f0cd40 sshd 2248 1 2248 123 Ss (threaded) ntpd 100198 S select 0xfffff80003e2d9c0 ntpd 100275 S usem 0xfffff80003ae6100 ntpd 2187 1 2187 0 Ss rpcsvc 0xfffff801e4862560 NLM: master 2184 1 2184 0 Ss select 0xfffff80006401cc0 rpc.statd 2138 1 2138 0 Ss nanslp 0xffffffff81e8f0b1 watchdogd 2058 1 2058 0 Ss select 0xfffff80003788d40 rpcbind 2048 1 2048 0 Ss select 0xfffff800048b5f40 syslogd 1977 1 1977 0 Ss select 0xfffff80006401dc0 devd 1972 1 1972 65 Ss select 0xfffff80003f0ce40 dhclient 1928 1 1928 0 Ss select 0xfffff80003d08b40 dhclient 1925 1 1925 0 Ss select 0xfffff80003e2dcc0 dhclient 34 0 0 0 DL syncer 0xffffffff820315c0 [syncer] 33 0 0 0 DL vlruwt 0xfffff8002e529000 [vnlru] 32 0 0 0 DL (threaded) [bufdaemon] 100132 D qsleep 0xffffffff82030a58 [bufdaemon] 100133 D - 0xffffffff81a0a8c0 [bufspacedaemon-0] 100134 D - 0xffffffff81a12b80 [bufspacedaemon-1] 100135 D - 0xffffffff81a1ae40 [bufspacedaemon-2] 100136 D - 0xffffffff81a23100 [bufspacedaemon-3] 100137 D - 0xffffffff81a2b3c0 [bufspacedaemon-4] 100138 D - 0xffffffff81a33680 [bufspacedaemon-5] 100139 D - 0xffffffff81a3b940 [bufspacedaemon-6] 100154 D sdflush 0xfffff80003ed9ae8 [/ worker] 100197 D sdflush 0xfffff8000616d4e8 [/tmp worker] 31 0 0 0 DL psleep 0xffffffff8204d788 [vmdaemon] 30 0 0 0 RL (threaded) [pagedaemon] 100128 D pwait 0xffffffff81e8de47 [dom0] 100130 Run CPU 1 [laundry: dom0] 100131 D umarcl 0xffffffff80f00fb0 [uma] 29 0 0 0 DL idle 0xfffff80003af6000 [enc_daemon1] 28 0 0 0 DL idle 0xfffff80003af6530 [enc_daemon0] 27 0 0 0 DL - 0xffffffff81cee7a0 [rand_harvestq] 26 0 0 0 DL waiting 0xffffffff820b5c30 [sctp_iterator] 25 0 0 0 DL - 0xffffffff8203045c [soaiod4] 24 0 0 0 DL - 0xffffffff8203045c [soaiod3] 23 0 0 0 DL - 0xffffffff8203045c [soaiod2] 22 0 0 0 DL - 0xffffffff8203045c [soaiod1] 21 0 0 0 DL (threaded) [usb] 100096 D - 0xfffffe009a1aa460 [usbus0] 100097 D - 0xfffffe009a1aa4b8 [usbus0] 100098 D - 0xfffffe009a1aa510 [usbus0] 100099 D - 0xfffffe009a1aa568 [usbus0] 100100 D - 0xfffffe009a1aa5c0 [usbus0] 100102 D - 0xfffffe0006dd6d10 [usbus1] 100103 D - 0xfffffe0006dd6d68 [usbus1] 100104 D - 0xfffffe0006dd6dc0 [usbus1] 100105 D - 0xfffffe0006dd6e18 [usbus1] 100106 D - 0xfffffe0006dd6e70 [usbus1] 100107 D - 0xfffffe009a63dd10 [usbus2] 100108 D - 0xfffffe009a63dd68 [usbus2] 100109 D - 0xfffffe009a63ddc0 [usbus2] 100110 D - 0xfffffe009a63de18 [usbus2] 100111 D - 0xfffffe009a63de70 [usbus2] 20 0 0 0 DL (threaded) [cam] 100091 D - 0xffffffff81bc9b40 [doneq0] 100092 D - 0xffffffff81bc9bc0 [doneq1] 100093 D - 0xffffffff81bc9c40 [doneq2] 100125 D - 0xffffffff81bc9a08 [scanner] 19 0 0 0 DL crypto_ 0xfffff80003fe1450 [crypto returns 11] 18 0 0 0 DL crypto_ 0xfffff80003fe13f0 [crypto returns 10] 17 0 0 0 DL crypto_ 0xfffff80003fe1390 [crypto returns 9] 16 0 0 0 DL crypto_ 0xfffff80003fe1330 [crypto returns 8] 15 0 0 0 DL crypto_ 0xfffff80003fe12d0 [crypto returns 7] 9 0 0 0 DL crypto_ 0xfffff80003fe1270 [crypto returns 6] 8 0 0 0 DL crypto_ 0xfffff80003fe1210 [crypto returns 5] 7 0 0 0 DL crypto_ 0xfffff80003fe11b0 [crypto returns 4] 6 0 0 0 DL crypto_ 0xfffff80003fe1150 [crypto returns 3] 5 0 0 0 DL crypto_ 0xfffff80003fe10f0 [crypto returns 2] 4 0 0 0 DL crypto_ 0xfffff80003fe1090 [crypto returns 1] 3 0 0 0 DL crypto_ 0xfffff80003fe1030 [crypto returns 0] 2 0 0 0 DL crypto_ 0xffffffff82047df8 [crypto] 14 0 0 0 DL seqstat 0xfffff8000392ec88 [sequencer 00] 13 0 0 0 DL (threaded) [geom] 100062 D - 0xffffffff820860c0 [g_event] 100063 D - 0xffffffff820860d0 [g_up] 100064 D - 0xffffffff820860b8 [g_down] 12 0 0 0 WL (threaded) [intr] 100017 I [swi6: task queue] 100018 I [swi6: Giant taskq] 100020 I [swi5: fast taskq] 100047 I [swi1: netisr 0] 100048 I [swi3: vm] 100049 I [swi4: clock (0)] 100050 I [swi4: clock (1)] 100051 I [swi4: clock (2)] 100052 I [swi4: clock (3)] 100053 I [swi4: clock (4)] 100054 I [swi4: clock (5)] 100055 I [swi4: clock (6)] 100056 I [swi4: clock (7)] 100057 I [swi4: clock (8)] 100058 I [swi4: clock (9)] 100059 I [swi4: clock (10)] 100060 I [swi4: clock (11)] 100094 I [irq56: ahci0] 100095 I [irq19: xhci0] 100101 I [irq18: ehci0 ehci1] 100112 I [irq71: ahci1] 100113 I [swi0: uart uart] 100177 I [irq72: ioat0] 100178 I [irq73: ioat1] 100179 I [irq74: ioat2] 100180 I [irq75: ioat3] 100181 I [irq76: ioat4] 100182 I [irq77: ioat5] 100183 I [irq78: ioat6] 100184 I [irq79: ioat7] 11 0 0 0 RL (threaded) [idle] 100003 Run CPU 0 [idle: cpu0] 100004 CanRun [idle: cpu1] 100005 Run CPU 2 [idle: cpu2] 100006 Run CPU 3 [idle: cpu3] 100007 Run CPU 4 [idle: cpu4] 100008 Run CPU 5 [idle: cpu5] 100009 Run CPU 6 [idle: cpu6] 100010 Run CPU 7 [idle: cpu7] 100011 Run CPU 8 [idle: cpu8] 100012 Run CPU 9 [idle: cpu9] 100013 Run CPU 10 [idle: cpu10] 100014 Run CPU 11 [idle: cpu11] 1 0 1 0 SLs wait 0xfffff800037af530 [init] 10 0 0 0 DL audit_w 0xffffffff820b6950 [audit] 0 0 0 0 DLs (threaded) [kernel] 100000 D swapin 0xffffffff82086100 [swapper] 100015 D - 0xfffff8000369de00 [kqueue_ctx taskq] 100016 D - 0xfffff8000369dd00 [aiod_kick taskq] 100019 D - 0xfffff8000369da00 [thread taskq] 100021 D - 0xfffff80003130c00 [config_0] 100022 D - 0xfffff80003130b00 [if_io_tqg_0] 100023 D - 0xfffff80003130a00 [if_io_tqg_1] 100024 D - 0xfffff80003130900 [if_io_tqg_2] 100025 D - 0xfffff80003130800 [if_io_tqg_3] 100026 D - 0xfffff80003130700 [if_io_tqg_4] 100027 D - 0xfffff80003130600 [if_io_tqg_5] 100028 D - 0xfffff80003130500 [if_io_tqg_6] 100029 D - 0xfffff80003130400 [if_io_tqg_7] 100030 D - 0xfffff80003130300 [if_io_tqg_8] 100031 D - 0xfffff80003130200 [if_io_tqg_9] 100032 D - 0xfffff80003130100 [if_io_tqg_10] 100033 D - 0xfffff80003130000 [if_io_tqg_11] 100034 D - 0xfffff8000312fe00 [softirq_0] 100035 D - 0xfffff8000312fd00 [softirq_1] 100036 D - 0xfffff8000312fc00 [softirq_2] 100037 D - 0xfffff8000312fb00 [softirq_3] 100038 D - 0xfffff8000312fa00 [softirq_4] 100039 D - 0xfffff8000312f900 [softirq_5] 100040 D - 0xfffff8000312f800 [softirq_6] 100041 D - 0xfffff8000312f700 [softirq_7] 100042 D - 0xfffff8000312f600 [softirq_8] 100043 D - 0xfffff8000312f500 [softirq_9] 100044 D - 0xfffff8000312f400 [softirq_10] 100045 D - 0xfffff8000312f300 [softirq_11] 100046 D - 0xfffff800039d3100 [if_config_tqg_0] 100061 D - 0xfffff8000393b100 [firmware taskq] 100066 D - 0xfffff8000393b000 [crypto_0] 100067 D - 0xfffff8000393b000 [crypto_1] 100068 D - 0xfffff8000393b000 [crypto_2] 100069 D - 0xfffff8000393b000 [crypto_3] 100070 D - 0xfffff8000393b000 [crypto_4] 100071 D - 0xfffff8000393b000 [crypto_5] 100072 D - 0xfffff8000393b000 [crypto_6] 100073 D - 0xfffff8000393b000 [crypto_7] 100074 D - 0xfffff8000393b000 [crypto_8] 100075 D - 0xfffff8000393b000 [crypto_9] 100076 D - 0xfffff8000393b000 [crypto_10] 100077 D - 0xfffff8000393b000 [crypto_11] 100114 D - 0xfffff800064b5200 [mca taskq] 100120 D - 0xfffff8002e4eb600 [acpi_task_0] 100121 D - 0xfffff8002e4eb600 [acpi_task_1] 100122 D - 0xfffff8002e4eb600 [acpi_task_2] 100124 D - 0xfffff8000393ce00 [CAM taskq] db:0:pho> allt Tracing command sleep pid 4784 tid 100239 td 0xfffff800036c7000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0006f2aa80 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0006f2aa90 trap() at trap+0x5b/frame 0xfffffe0006f2aba0 calltrap() at calltrap+0x8/frame 0xfffffe0006f2aba0 --- trap 0xc, rip = 0xffffffff8107b516, rsp = 0xfffffe0006f2ac70, rbp = 0xfffffe0006f2ac80 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe0006f2ac80 _db_stack_trace_all() at _db_stack_trace_all+0xc4/frame 0xfffffe0006f2ad20 db_command() at db_command+0x279/frame 0xfffffe0006f2adf0 db_script_exec() at db_script_exec+0x188/frame 0xfffffe0006f2ae90 db_command() at db_command+0x279/frame 0xfffffe0006f2af60 db_command_loop() at db_command_loop+0x64/frame 0xfffffe0006f2af70 db_trap() at db_trap+0xef/frame 0xfffffe0006f2b000 kdb_trap() at kdb_trap+0x16c/frame 0xfffffe0006f2b080 trap() at trap+0x811/frame 0xfffffe0006f2b190 calltrap() at calltrap+0x8/frame 0xfffffe0006f2b190 --- trap 0x3, rip = 0xffffffff80c1925b, rsp = 0xfffffe0006f2b260, rbp = 0xfffffe0006f2b270 --- kdb_enter() at kdb_enter+0x3b/frame 0xfffffe0006f2b270 vpanic() at vpanic+0x1ba/frame 0xfffffe0006f2b2c0 panic() at panic+0x43/frame 0xfffffe0006f2b320 witness_unlock() at witness_unlock+0xea/frame 0xfffffe0006f2b360 __mtx_unlock_flags() at __mtx_unlock_flags+0x4d/frame 0xfffffe0006f2b390 swp_pager_trimswapspace() at swp_pager_trimswapspace+0xeb/frame 0xfffffe0006f2b3e0 swap_pager_putpages() at swap_pager_putpages+0x56c/frame 0xfffffe0006f2b480 vm_pageout_flush() at vm_pageout_flush+0xff/frame 0xfffffe0006f2b580 vm_pageout_cluster() at vm_pageout_cluster+0x3e6/frame 0xfffffe0006f2b7f0 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0x2f6/frame 0xfffffe0006f2ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f2bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f2bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sleep pid 4752 tid 100240 td 0xfffff801e4370000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0006f2aa80 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0006f2aa90 trap() at trap+0x5b/frame 0xfffffe0006f2aba0 calltrap() at calltrap+0x8/frame 0xfffffe0006f2aba0 --- trap 0xc, rip = 0xffffffff8107b516, rsp = 0xfffffe0006f2ac70, rbp = 0xfffffe0006f2ac80 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe0006f2ac80 _db_stack_trace_all() at _db_stack_trace_all+0xc4/frame 0xfffffe0006f2ad20 db_command() at db_command+0x279/frame 0xfffffe0006f2adf0 db_script_exec() at db_script_exec+0x188/frame 0xfffffe0006f2ae90 db_command() at db_command+0x279/frame 0xfffffe0006f2af60 db_command_loop() at db_command_loop+0x64/frame 0xfffffe0006f2af70 db_trap() at db_trap+0xef/frame 0xfffffe0006f2b000 kdb_trap() at kdb_trap+0x16c/frame 0xfffffe0006f2b080 trap() at trap+0x811/frame 0xfffffe0006f2b190 calltrap() at calltrap+0x8/frame 0xfffffe0006f2b190 --- trap 0x3, rip = 0xffffffff80c1925b, rsp = 0xfffffe0006f2b260, rbp = 0xfffffe0006f2b270 --- kdb_enter() at kdb_enter+0x3b/frame 0xfffffe0006f2b270 vpanic() at vpanic+0x1ba/frame 0xfffffe0006f2b2c0 panic() at panic+0x43/frame 0xfffffe0006f2b320 witness_unlock() at witness_unlock+0xea/frame 0xfffffe0006f2b360 __mtx_unlock_flags() at __mtx_unlock_flags+0x4d/frame 0xfffffe0006f2b390 swp_pager_trimswapspace() at swp_pager_trimswapspace+0xeb/frame 0xfffffe0006f2b3e0 swap_pager_putpages() at swap_pager_putpages+0x56c/frame 0xfffffe0006f2b480 vm_pageout_flush() at vm_pageout_flush+0xff/frame 0xfffffe0006f2b580 vm_pageout_cluster() at vm_pageout_cluster+0x3e6/frame 0xfffffe0006f2b7f0 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0x2f6/frame 0xfffffe0006f2ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f2bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f2bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sort pid 3723 tid 100268 td 0xfffff8022a3dc5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab681660 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab681690 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab6816d0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00ab681700 _sleep() at _sleep+0x2da/frame 0xfffffe00ab6817a0 vm_waitpfault() at vm_waitpfault+0x6b/frame 0xfffffe00ab6817c0 vm_fault_hold() at vm_fault_hold+0xabb/frame 0xfffffe00ab681910 vm_fault() at vm_fault+0x60/frame 0xfffffe00ab681950 trap_pfault() at trap_pfault+0x188/frame 0xfffffe00ab6819a0 trap() at trap+0x46b/frame 0xfffffe00ab681ab0 calltrap() at calltrap+0x8/frame 0xfffffe00ab681ab0 --- trap 0xc, rip = 0x800440885, rsp = 0x7fffffffdce8, rbp = 0x7fffffffded0 --- Tracing command sort pid 3722 tid 100267 td 0xfffff8022a3ce000 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab67c660 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab67c690 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab67c6d0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00ab67c700 _sleep() at _sleep+0x2da/frame 0xfffffe00ab67c7a0 vm_waitpfault() at vm_waitpfault+0x6b/frame 0xfffffe00ab67c7c0 vm_fault_hold() at vm_fault_hold+0xabb/frame 0xfffffe00ab67c910 vm_fault() at vm_fault+0x60/frame 0xfffffe00ab67c950 trap_pfault() at trap_pfault+0x188/frame 0xfffffe00ab67c9a0 trap() at trap+0x46b/frame 0xfffffe00ab67cab0 calltrap() at calltrap+0x8/frame 0xfffffe00ab67cab0 --- trap 0xc, rip = 0x800440885, rsp = 0x7fffffffdce8, rbp = 0x7fffffffded0 --- Tracing command sort pid 3721 tid 100266 td 0xfffff8022a3ce5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab677660 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab677690 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab6776d0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00ab677700 _sleep() at _sleep+0x2da/frame 0xfffffe00ab6777a0 vm_waitpfault() at vm_waitpfault+0x6b/frame 0xfffffe00ab6777c0 vm_fault_hold() at vm_fault_hold+0xabb/frame 0xfffffe00ab677910 vm_fault() at vm_fault+0x60/frame 0xfffffe00ab677950 trap_pfault() at trap_pfault+0x188/frame 0xfffffe00ab6779a0 trap() at trap+0x46b/frame 0xfffffe00ab677ab0 calltrap() at calltrap+0x8/frame 0xfffffe00ab677ab0 --- trap 0xc, rip = 0x800440885, rsp = 0x7fffffffdce8, rbp = 0x7fffffffded0 --- Tracing command sort pid 3720 tid 100265 td 0xfffff8022a3cd000 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab672660 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab672690 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab6726d0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00ab672700 _sleep() at _sleep+0x2da/frame 0xfffffe00ab6727a0 vm_waitpfault() at vm_waitpfault+0x6b/frame 0xfffffe00ab6727c0 vm_fault_hold() at vm_fault_hold+0xabb/frame 0xfffffe00ab672910 vm_fault() at vm_fault+0x60/frame 0xfffffe00ab672950 trap_pfault() at trap_pfault+0x188/frame 0xfffffe00ab6729a0 trap() at trap+0x46b/frame 0xfffffe00ab672ab0 calltrap() at calltrap+0x8/frame 0xfffffe00ab672ab0 --- trap 0xc, rip = 0x800440885, rsp = 0x7fffffffdce8, rbp = 0x7fffffffded0 --- Tracing command sort pid 3719 tid 100150 td 0xfffff8002eca2000 sched_switch() at sched_switch+0x713/frame 0xfffffe00aa7f1660 mi_switch() at mi_switch+0x174/frame 0xfffffe00aa7f1690 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00aa7f16d0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00aa7f1700 _sleep() at _sleep+0x2da/frame 0xfffffe00aa7f17a0 vm_waitpfault() at vm_waitpfault+0x6b/frame 0xfffffe00aa7f17c0 vm_fault_hold() at vm_fault_hold+0xabb/frame 0xfffffe00aa7f1910 vm_fault() at vm_fault+0x60/frame 0xfffffe00aa7f1950 trap_pfault() at trap_pfault+0x188/frame 0xfffffe00aa7f19a0 trap() at trap+0x46b/frame 0xfffffe00aa7f1ab0 calltrap() at calltrap+0x8/frame 0xfffffe00aa7f1ab0 --- trap 0xc, rip = 0x800440885, rsp = 0x7fffffffdce8, rbp = 0x7fffffffded0 --- Tracing command sort pid 3718 tid 100256 td 0xfffff8022a3cd5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab645660 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab645690 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab6456d0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00ab645700 _sleep() at _sleep+0x2da/frame 0xfffffe00ab6457a0 vm_waitpfault() at vm_waitpfault+0x6b/frame 0xfffffe00ab6457c0 vm_fault_hold() at vm_fault_hold+0xabb/frame 0xfffffe00ab645910 vm_fault() at vm_fault+0x60/frame 0xfffffe00ab645950 trap_pfault() at trap_pfault+0x188/frame 0xfffffe00ab6459a0 trap() at trap+0x46b/frame 0xfffffe00ab645ab0 calltrap() at calltrap+0x8/frame 0xfffffe00ab645ab0 --- trap 0xc, rip = 0x800440885, rsp = 0x7fffffffdce8, rbp = 0x7fffffffded0 --- Tracing command sh pid 3715 tid 100261 td 0xfffff8000605e5a0 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0006f2aa80 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0006f2aa90 trap() at trap+0x5b/frame 0xfffffe0006f2aba0 calltrap() at calltrap+0x8/frame 0xfffffe0006f2aba0 --- trap 0xc, rip = 0xffffffff8107b516, rsp = 0xfffffe0006f2ac70, rbp = 0xfffffe0006f2ac80 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe0006f2ac80 _db_stack_trace_all() at _db_stack_trace_all+0xc4/frame 0xfffffe0006f2ad20 db_command() at db_command+0x279/frame 0xfffffe0006f2adf0 db_script_exec() at db_script_exec+0x188/frame 0xfffffe0006f2ae90 db_command() at db_command+0x279/frame 0xfffffe0006f2af60 db_command_loop() at db_command_loop+0x64/frame 0xfffffe0006f2af70 db_trap() at db_trap+0xef/frame 0xfffffe0006f2b000 kdb_trap() at kdb_trap+0x16c/frame 0xfffffe0006f2b080 trap() at trap+0x811/frame 0xfffffe0006f2b190 calltrap() at calltrap+0x8/frame 0xfffffe0006f2b190 --- trap 0x3, rip = 0xffffffff80c1925b, rsp = 0xfffffe0006f2b260, rbp = 0xfffffe0006f2b270 --- kdb_enter() at kdb_enter+0x3b/frame 0xfffffe0006f2b270 vpanic() at vpanic+0x1ba/frame 0xfffffe0006f2b2c0 panic() at panic+0x43/frame 0xfffffe0006f2b320 witness_unlock() at witness_unlock+0xea/frame 0xfffffe0006f2b360 __mtx_unlock_flags() at __mtx_unlock_flags+0x4d/frame 0xfffffe0006f2b390 swp_pager_trimswapspace() at swp_pager_trimswapspace+0xeb/frame 0xfffffe0006f2b3e0 swap_pager_putpages() at swap_pager_putpages+0x56c/frame 0xfffffe0006f2b480 vm_pageout_flush() at vm_pageout_flush+0xff/frame 0xfffffe0006f2b580 vm_pageout_cluster() at vm_pageout_cluster+0x3e6/frame 0xfffffe0006f2b7f0 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0x2f6/frame 0xfffffe0006f2ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f2bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f2bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command tee pid 3714 tid 100199 td 0xfffff8002eaa1000 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab52d6a0 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab52d6d0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab52d710 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ab52d770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ab52d7a0 _sleep() at _sleep+0x2d0/frame 0xfffffe00ab52d840 pipe_read() at pipe_read+0x356/frame 0xfffffe00ab52d8a0 dofileread() at dofileread+0x81/frame 0xfffffe00ab52d8f0 kern_readv() at kern_readv+0x4a/frame 0xfffffe00ab52d930 sys_read() at sys_read+0x84/frame 0xfffffe00ab52d980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00ab52dab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ab52dab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80041522a, rsp = 0x7fffffffe6b8, rbp = 0x7fffffffe700 --- Tracing command sh pid 3713 tid 100187 td 0xfffff80003af0000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0006f2aa80 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0006f2aa90 trap() at trap+0x5b/frame 0xfffffe0006f2aba0 calltrap() at calltrap+0x8/frame 0xfffffe0006f2aba0 --- trap 0xc, rip = 0xffffffff8107b516, rsp = 0xfffffe0006f2ac70, rbp = 0xfffffe0006f2ac80 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe0006f2ac80 _db_stack_trace_all() at _db_stack_trace_all+0xc4/frame 0xfffffe0006f2ad20 db_command() at db_command+0x279/frame 0xfffffe0006f2adf0 db_script_exec() at db_script_exec+0x188/frame 0xfffffe0006f2ae90 db_command() at db_command+0x279/frame 0xfffffe0006f2af60 db_command_loop() at db_command_loop+0x64/frame 0xfffffe0006f2af70 db_trap() at db_trap+0xef/frame 0xfffffe0006f2b000 kdb_trap() at kdb_trap+0x16c/frame 0xfffffe0006f2b080 trap() at trap+0x811/frame 0xfffffe0006f2b190 calltrap() at calltrap+0x8/frame 0xfffffe0006f2b190 --- trap 0x3, rip = 0xffffffff80c1925b, rsp = 0xfffffe0006f2b260, rbp = 0xfffffe0006f2b270 --- kdb_enter() at kdb_enter+0x3b/frame 0xfffffe0006f2b270 vpanic() at vpanic+0x1ba/frame 0xfffffe0006f2b2c0 panic() at panic+0x43/frame 0xfffffe0006f2b320 witness_unlock() at witness_unlock+0xea/frame 0xfffffe0006f2b360 __mtx_unlock_flags() at __mtx_unlock_flags+0x4d/frame 0xfffffe0006f2b390 swp_pager_trimswapspace() at swp_pager_trimswapspace+0xeb/frame 0xfffffe0006f2b3e0 swap_pager_putpages() at swap_pager_putpages+0x56c/frame 0xfffffe0006f2b480 vm_pageout_flush() at vm_pageout_flush+0xff/frame 0xfffffe0006f2b580 vm_pageout_cluster() at vm_pageout_cluster+0x3e6/frame 0xfffffe0006f2b7f0 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0x2f6/frame 0xfffffe0006f2ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f2bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f2bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sh pid 2525 tid 100237 td 0xfffff800036c75a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab5e66a0 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab5e66d0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab5e6710 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ab5e6770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ab5e67a0 _sleep() at _sleep+0x2d0/frame 0xfffffe00ab5e6840 pipe_read() at pipe_read+0x356/frame 0xfffffe00ab5e68a0 dofileread() at dofileread+0x81/frame 0xfffffe00ab5e68f0 kern_readv() at kern_readv+0x4a/frame 0xfffffe00ab5e6930 sys_read() at sys_read+0x84/frame 0xfffffe00ab5e6980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00ab5e6ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ab5e6ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80047622a, rsp = 0x7fffffffe228, rbp = 0x7fffffffe330 --- Tracing command awk pid 2523 tid 100236 td 0xfffff800036c6000 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab5e16a0 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab5e16d0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab5e1710 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ab5e1770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ab5e17a0 _sleep() at _sleep+0x2d0/frame 0xfffffe00ab5e1840 pipe_read() at pipe_read+0x356/frame 0xfffffe00ab5e18a0 dofileread() at dofileread+0x81/frame 0xfffffe00ab5e18f0 kern_readv() at kern_readv+0x4a/frame 0xfffffe00ab5e1930 sys_read() at sys_read+0x84/frame 0xfffffe00ab5e1980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00ab5e1ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ab5e1ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80047922a, rsp = 0x7fffffffe328, rbp = 0x7fffffffe340 --- Tracing command sh pid 2522 tid 100216 td 0xfffff8002eca35a0 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0006f2aa80 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0006f2aa90 trap() at trap+0x5b/frame 0xfffffe0006f2aba0 calltrap() at calltrap+0x8/frame 0xfffffe0006f2aba0 --- trap 0xc, rip = 0xffffffff8107b516, rsp = 0xfffffe0006f2ac70, rbp = 0xfffffe0006f2ac80 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe0006f2ac80 _db_stack_trace_all() at _db_stack_trace_all+0xc4/frame 0xfffffe0006f2ad20 db_command() at db_command+0x279/frame 0xfffffe0006f2adf0 db_script_exec() at db_script_exec+0x188/frame 0xfffffe0006f2ae90 db_command() at db_command+0x279/frame 0xfffffe0006f2af60 db_command_loop() at db_command_loop+0x64/frame 0xfffffe0006f2af70 db_trap() at db_trap+0xef/frame 0xfffffe0006f2b000 kdb_trap() at kdb_trap+0x16c/frame 0xfffffe0006f2b080 trap() at trap+0x811/frame 0xfffffe0006f2b190 calltrap() at calltrap+0x8/frame 0xfffffe0006f2b190 --- trap 0x3, rip = 0xffffffff80c1925b, rsp = 0xfffffe0006f2b260, rbp = 0xfffffe0006f2b270 --- kdb_enter() at kdb_enter+0x3b/frame 0xfffffe0006f2b270 vpanic() at vpanic+0x1ba/frame 0xfffffe0006f2b2c0 panic() at panic+0x43/frame 0xfffffe0006f2b320 witness_unlock() at witness_unlock+0xea/frame 0xfffffe0006f2b360 __mtx_unlock_flags() at __mtx_unlock_flags+0x4d/frame 0xfffffe0006f2b390 swp_pager_trimswapspace() at swp_pager_trimswapspace+0xeb/frame 0xfffffe0006f2b3e0 swap_pager_putpages() at swap_pager_putpages+0x56c/frame 0xfffffe0006f2b480 vm_pageout_flush() at vm_pageout_flush+0xff/frame 0xfffffe0006f2b580 vm_pageout_cluster() at vm_pageout_cluster+0x3e6/frame 0xfffffe0006f2b7f0 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0x2f6/frame 0xfffffe0006f2ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f2bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f2bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sh pid 2470 tid 100224 td 0xfffff801e463c000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0006f2aa80 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0006f2aa90 trap() at trap+0x5b/frame 0xfffffe0006f2aba0 calltrap() at calltrap+0x8/frame 0xfffffe0006f2aba0 --- trap 0xc, rip = 0xffffffff8107b516, rsp = 0xfffffe0006f2ac70, rbp = 0xfffffe0006f2ac80 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe0006f2ac80 _db_stack_trace_all() at _db_stack_trace_all+0xc4/frame 0xfffffe0006f2ad20 db_command() at db_command+0x279/frame 0xfffffe0006f2adf0 db_script_exec() at db_script_exec+0x188/frame 0xfffffe0006f2ae90 db_command() at db_command+0x279/frame 0xfffffe0006f2af60 db_command_loop() at db_command_loop+0x64/frame 0xfffffe0006f2af70 db_trap() at db_trap+0xef/frame 0xfffffe0006f2b000 kdb_trap() at kdb_trap+0x16c/frame 0xfffffe0006f2b080 trap() at trap+0x811/frame 0xfffffe0006f2b190 calltrap() at calltrap+0x8/frame 0xfffffe0006f2b190 --- trap 0x3, rip = 0xffffffff80c1925b, rsp = 0xfffffe0006f2b260, rbp = 0xfffffe0006f2b270 --- kdb_enter() at kdb_enter+0x3b/frame 0xfffffe0006f2b270 vpanic() at vpanic+0x1ba/frame 0xfffffe0006f2b2c0 panic() at panic+0x43/frame 0xfffffe0006f2b320 witness_unlock() at witness_unlock+0xea/frame 0xfffffe0006f2b360 __mtx_unlock_flags() at __mtx_unlock_flags+0x4d/frame 0xfffffe0006f2b390 swp_pager_trimswapspace() at swp_pager_trimswapspace+0xeb/frame 0xfffffe0006f2b3e0 swap_pager_putpages() at swap_pager_putpages+0x56c/frame 0xfffffe0006f2b480 vm_pageout_flush() at vm_pageout_flush+0xff/frame 0xfffffe0006f2b580 vm_pageout_cluster() at vm_pageout_cluster+0x3e6/frame 0xfffffe0006f2b7f0 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0x2f6/frame 0xfffffe0006f2ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f2bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f2bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sh pid 2360 tid 100219 td 0xfffff8003cea8000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0006f2aa80 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0006f2aa90 trap() at trap+0x5b/frame 0xfffffe0006f2aba0 calltrap() at calltrap+0x8/frame 0xfffffe0006f2aba0 --- trap 0xc, rip = 0xffffffff8107b516, rsp = 0xfffffe0006f2ac70, rbp = 0xfffffe0006f2ac80 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe0006f2ac80 _db_stack_trace_all() at _db_stack_trace_all+0xc4/frame 0xfffffe0006f2ad20 db_command() at db_command+0x279/frame 0xfffffe0006f2adf0 db_script_exec() at db_script_exec+0x188/frame 0xfffffe0006f2ae90 db_command() at db_command+0x279/frame 0xfffffe0006f2af60 db_command_loop() at db_command_loop+0x64/frame 0xfffffe0006f2af70 db_trap() at db_trap+0xef/frame 0xfffffe0006f2b000 kdb_trap() at kdb_trap+0x16c/frame 0xfffffe0006f2b080 trap() at trap+0x811/frame 0xfffffe0006f2b190 calltrap() at calltrap+0x8/frame 0xfffffe0006f2b190 --- trap 0x3, rip = 0xffffffff80c1925b, rsp = 0xfffffe0006f2b260, rbp = 0xfffffe0006f2b270 --- kdb_enter() at kdb_enter+0x3b/frame 0xfffffe0006f2b270 vpanic() at vpanic+0x1ba/frame 0xfffffe0006f2b2c0 panic() at panic+0x43/frame 0xfffffe0006f2b320 witness_unlock() at witness_unlock+0xea/frame 0xfffffe0006f2b360 __mtx_unlock_flags() at __mtx_unlock_flags+0x4d/frame 0xfffffe0006f2b390 swp_pager_trimswapspace() at swp_pager_trimswapspace+0xeb/frame 0xfffffe0006f2b3e0 swap_pager_putpages() at swap_pager_putpages+0x56c/frame 0xfffffe0006f2b480 vm_pageout_flush() at vm_pageout_flush+0xff/frame 0xfffffe0006f2b580 vm_pageout_cluster() at vm_pageout_cluster+0x3e6/frame 0xfffffe0006f2b7f0 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0x2f6/frame 0xfffffe0006f2ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f2bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f2bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bash pid 2356 tid 100156 td 0xfffff800049eb000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0006f2aa80 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0006f2aa90 trap() at trap+0x5b/frame 0xfffffe0006f2aba0 calltrap() at calltrap+0x8/frame 0xfffffe0006f2aba0 --- trap 0xc, rip = 0xffffffff8107b516, rsp = 0xfffffe0006f2ac70, rbp = 0xfffffe0006f2ac80 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe0006f2ac80 _db_stack_trace_all() at _db_stack_trace_all+0xc4/frame 0xfffffe0006f2ad20 db_command() at db_command+0x279/frame 0xfffffe0006f2adf0 db_script_exec() at db_script_exec+0x188/frame 0xfffffe0006f2ae90 db_command() at db_command+0x279/frame 0xfffffe0006f2af60 db_command_loop() at db_command_loop+0x64/frame 0xfffffe0006f2af70 db_trap() at db_trap+0xef/frame 0xfffffe0006f2b000 kdb_trap() at kdb_trap+0x16c/frame 0xfffffe0006f2b080 trap() at trap+0x811/frame 0xfffffe0006f2b190 calltrap() at calltrap+0x8/frame 0xfffffe0006f2b190 --- trap 0x3, rip = 0xffffffff80c1925b, rsp = 0xfffffe0006f2b260, rbp = 0xfffffe0006f2b270 --- kdb_enter() at kdb_enter+0x3b/frame 0xfffffe0006f2b270 vpanic() at vpanic+0x1ba/frame 0xfffffe0006f2b2c0 panic() at panic+0x43/frame 0xfffffe0006f2b320 witness_unlock() at witness_unlock+0xea/frame 0xfffffe0006f2b360 __mtx_unlock_flags() at __mtx_unlock_flags+0x4d/frame 0xfffffe0006f2b390 swp_pager_trimswapspace() at swp_pager_trimswapspace+0xeb/frame 0xfffffe0006f2b3e0 swap_pager_putpages() at swap_pager_putpages+0x56c/frame 0xfffffe0006f2b480 vm_pageout_flush() at vm_pageout_flush+0xff/frame 0xfffffe0006f2b580 vm_pageout_cluster() at vm_pageout_cluster+0x3e6/frame 0xfffffe0006f2b7f0 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0x2f6/frame 0xfffffe0006f2ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f2bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f2bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command csh pid 2354 tid 100189 td 0xfffff8002eaa15a0 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0006f2aa80 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0006f2aa90 trap() at trap+0x5b/frame 0xfffffe0006f2aba0 calltrap() at calltrap+0x8/frame 0xfffffe0006f2aba0 --- trap 0xc, rip = 0xffffffff8107b516, rsp = 0xfffffe0006f2ac70, rbp = 0xfffffe0006f2ac80 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe0006f2ac80 _db_stack_trace_all() at _db_stack_trace_all+0xc4/frame 0xfffffe0006f2ad20 db_command() at db_command+0x279/frame 0xfffffe0006f2adf0 db_script_exec() at db_script_exec+0x188/frame 0xfffffe0006f2ae90 db_command() at db_command+0x279/frame 0xfffffe0006f2af60 db_command_loop() at db_command_loop+0x64/frame 0xfffffe0006f2af70 db_trap() at db_trap+0xef/frame 0xfffffe0006f2b000 kdb_trap() at kdb_trap+0x16c/frame 0xfffffe0006f2b080 trap() at trap+0x811/frame 0xfffffe0006f2b190 calltrap() at calltrap+0x8/frame 0xfffffe0006f2b190 --- trap 0x3, rip = 0xffffffff80c1925b, rsp = 0xfffffe0006f2b260, rbp = 0xfffffe0006f2b270 --- kdb_enter() at kdb_enter+0x3b/frame 0xfffffe0006f2b270 vpanic() at vpanic+0x1ba/frame 0xfffffe0006f2b2c0 panic() at panic+0x43/frame 0xfffffe0006f2b320 witness_unlock() at witness_unlock+0xea/frame 0xfffffe0006f2b360 __mtx_unlock_flags() at __mtx_unlock_flags+0x4d/frame 0xfffffe0006f2b390 swp_pager_trimswapspace() at swp_pager_trimswapspace+0xeb/frame 0xfffffe0006f2b3e0 swap_pager_putpages() at swap_pager_putpages+0x56c/frame 0xfffffe0006f2b480 vm_pageout_flush() at vm_pageout_flush+0xff/frame 0xfffffe0006f2b580 vm_pageout_cluster() at vm_pageout_cluster+0x3e6/frame 0xfffffe0006f2b7f0 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0x2f6/frame 0xfffffe0006f2ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f2bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f2bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command su pid 2353 tid 100217 td 0xfffff80003a555a0 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0006f2aa80 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0006f2aa90 trap() at trap+0x5b/frame 0xfffffe0006f2aba0 calltrap() at calltrap+0x8/frame 0xfffffe0006f2aba0 --- trap 0xc, rip = 0xffffffff8107b516, rsp = 0xfffffe0006f2ac70, rbp = 0xfffffe0006f2ac80 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe0006f2ac80 _db_stack_trace_all() at _db_stack_trace_all+0xc4/frame 0xfffffe0006f2ad20 db_command() at db_command+0x279/frame 0xfffffe0006f2adf0 db_script_exec() at db_script_exec+0x188/frame 0xfffffe0006f2ae90 db_command() at db_command+0x279/frame 0xfffffe0006f2af60 db_command_loop() at db_command_loop+0x64/frame 0xfffffe0006f2af70 db_trap() at db_trap+0xef/frame 0xfffffe0006f2b000 kdb_trap() at kdb_trap+0x16c/frame 0xfffffe0006f2b080 trap() at trap+0x811/frame 0xfffffe0006f2b190 calltrap() at calltrap+0x8/frame 0xfffffe0006f2b190 --- trap 0x3, rip = 0xffffffff80c1925b, rsp = 0xfffffe0006f2b260, rbp = 0xfffffe0006f2b270 --- kdb_enter() at kdb_enter+0x3b/frame 0xfffffe0006f2b270 vpanic() at vpanic+0x1ba/frame 0xfffffe0006f2b2c0 panic() at panic+0x43/frame 0xfffffe0006f2b320 witness_unlock() at witness_unlock+0xea/frame 0xfffffe0006f2b360 __mtx_unlock_flags() at __mtx_unlock_flags+0x4d/frame 0xfffffe0006f2b390 swp_pager_trimswapspace() at swp_pager_trimswapspace+0xeb/frame 0xfffffe0006f2b3e0 swap_pager_putpages() at swap_pager_putpages+0x56c/frame 0xfffffe0006f2b480 vm_pageout_flush() at vm_pageout_flush+0xff/frame 0xfffffe0006f2b580 vm_pageout_cluster() at vm_pageout_cluster+0x3e6/frame 0xfffffe0006f2b7f0 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0x2f6/frame 0xfffffe0006f2ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f2bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f2bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bash pid 2349 tid 100200 td 0xfffff801e463c5a0 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0006f2aa80 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0006f2aa90 trap() at trap+0x5b/frame 0xfffffe0006f2aba0 calltrap() at calltrap+0x8/frame 0xfffffe0006f2aba0 --- trap 0xc, rip = 0xffffffff8107b516, rsp = 0xfffffe0006f2ac70, rbp = 0xfffffe0006f2ac80 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe0006f2ac80 _db_stack_trace_all() at _db_stack_trace_all+0xc4/frame 0xfffffe0006f2ad20 db_command() at db_command+0x279/frame 0xfffffe0006f2adf0 db_script_exec() at db_script_exec+0x188/frame 0xfffffe0006f2ae90 db_command() at db_command+0x279/frame 0xfffffe0006f2af60 db_command_loop() at db_command_loop+0x64/frame 0xfffffe0006f2af70 db_trap() at db_trap+0xef/frame 0xfffffe0006f2b000 kdb_trap() at kdb_trap+0x16c/frame 0xfffffe0006f2b080 trap() at trap+0x811/frame 0xfffffe0006f2b190 calltrap() at calltrap+0x8/frame 0xfffffe0006f2b190 --- trap 0x3, rip = 0xffffffff80c1925b, rsp = 0xfffffe0006f2b260, rbp = 0xfffffe0006f2b270 --- kdb_enter() at kdb_enter+0x3b/frame 0xfffffe0006f2b270 vpanic() at vpanic+0x1ba/frame 0xfffffe0006f2b2c0 panic() at panic+0x43/frame 0xfffffe0006f2b320 witness_unlock() at witness_unlock+0xea/frame 0xfffffe0006f2b360 __mtx_unlock_flags() at __mtx_unlock_flags+0x4d/frame 0xfffffe0006f2b390 swp_pager_trimswapspace() at swp_pager_trimswapspace+0xeb/frame 0xfffffe0006f2b3e0 swap_pager_putpages() at swap_pager_putpages+0x56c/frame 0xfffffe0006f2b480 vm_pageout_flush() at vm_pageout_flush+0xff/frame 0xfffffe0006f2b580 vm_pageout_cluster() at vm_pageout_cluster+0x3e6/frame 0xfffffe0006f2b7f0 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0x2f6/frame 0xfffffe0006f2ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f2bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f2bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sshd pid 2348 tid 100196 td 0xfffff800036c65a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab5235c0 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab5235f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab523630 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ab523690 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ab5236c0 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ab523720 seltdwait() at seltdwait+0x8d/frame 0xfffffe00ab523760 kern_select() at kern_select+0x953/frame 0xfffffe00ab523940 sys_select() at sys_select+0x56/frame 0xfffffe00ab523980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00ab523ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ab523ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80090220a, rsp = 0x7fffffffa578, rbp = 0x7fffffffa5b0 --- Tracing command sshd pid 2345 tid 100201 td 0xfffff8002e463000 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab537640 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab537670 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab5376b0 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ab537710 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ab537740 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ab5377a0 seltdwait() at seltdwait+0x8d/frame 0xfffffe00ab5377e0 kern_poll() at kern_poll+0x448/frame 0xfffffe00ab537960 sys_poll() at sys_poll+0x50/frame 0xfffffe00ab537980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00ab537ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ab537ab0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8008a435a, rsp = 0x7fffffffe5d8, rbp = 0x7fffffffe610 --- Tracing command csh pid 2341 tid 100170 td 0xfffff80006181000 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab459620 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab459650 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab459690 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ab4596f0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ab459720 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ab459780 tty_wait() at tty_wait+0x42/frame 0xfffffe00ab4597b0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00ab459810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00ab459840 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00ab4598a0 dofileread() at dofileread+0x81/frame 0xfffffe00ab4598f0 kern_readv() at kern_readv+0x4a/frame 0xfffffe00ab459930 sys_read() at sys_read+0x84/frame 0xfffffe00ab459980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00ab459ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ab459ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004fa22a, rsp = 0x7fffffffe1f8, rbp = 0x7fffffffe220 --- Tracing command login pid 2336 tid 100214 td 0xfffff801e4af65a0 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0006f2aa80 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0006f2aa90 trap() at trap+0x5b/frame 0xfffffe0006f2aba0 calltrap() at calltrap+0x8/frame 0xfffffe0006f2aba0 --- trap 0xc, rip = 0xffffffff8107b516, rsp = 0xfffffe0006f2ac70, rbp = 0xfffffe0006f2ac80 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe0006f2ac80 _db_stack_trace_all() at _db_stack_trace_all+0xc4/frame 0xfffffe0006f2ad20 db_command() at db_command+0x279/frame 0xfffffe0006f2adf0 db_script_exec() at db_script_exec+0x188/frame 0xfffffe0006f2ae90 db_command() at db_command+0x279/frame 0xfffffe0006f2af60 db_command_loop() at db_command_loop+0x64/frame 0xfffffe0006f2af70 db_trap() at db_trap+0xef/frame 0xfffffe0006f2b000 kdb_trap() at kdb_trap+0x16c/frame 0xfffffe0006f2b080 trap() at trap+0x811/frame 0xfffffe0006f2b190 calltrap() at calltrap+0x8/frame 0xfffffe0006f2b190 --- trap 0x3, rip = 0xffffffff80c1925b, rsp = 0xfffffe0006f2b260, rbp = 0xfffffe0006f2b270 --- kdb_enter() at kdb_enter+0x3b/frame 0xfffffe0006f2b270 vpanic() at vpanic+0x1ba/frame 0xfffffe0006f2b2c0 panic() at panic+0x43/frame 0xfffffe0006f2b320 witness_unlock() at witness_unlock+0xea/frame 0xfffffe0006f2b360 __mtx_unlock_flags() at __mtx_unlock_flags+0x4d/frame 0xfffffe0006f2b390 swp_pager_trimswapspace() at swp_pager_trimswapspace+0xeb/frame 0xfffffe0006f2b3e0 swap_pager_putpages() at swap_pager_putpages+0x56c/frame 0xfffffe0006f2b480 vm_pageout_flush() at vm_pageout_flush+0xff/frame 0xfffffe0006f2b580 vm_pageout_cluster() at vm_pageout_cluster+0x3e6/frame 0xfffffe0006f2b7f0 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0x2f6/frame 0xfffffe0006f2ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f2bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f2bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command getty pid 2335 tid 100171 td 0xfffff800061825a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab45e620 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab45e650 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab45e690 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ab45e6f0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ab45e720 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ab45e780 tty_wait() at tty_wait+0x42/frame 0xfffffe00ab45e7b0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00ab45e810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00ab45e840 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00ab45e8a0 dofileread() at dofileread+0x81/frame 0xfffffe00ab45e8f0 kern_readv() at kern_readv+0x4a/frame 0xfffffe00ab45e930 sys_read() at sys_read+0x84/frame 0xfffffe00ab45e980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00ab45eab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ab45eab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80043122a, rsp = 0x7fffffffebc8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2334 tid 100213 td 0xfffff801e4af5000 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab573620 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab573650 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab573690 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ab5736f0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ab573720 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ab573780 tty_wait() at tty_wait+0x42/frame 0xfffffe00ab5737b0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00ab573810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00ab573840 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00ab5738a0 dofileread() at dofileread+0x81/frame 0xfffffe00ab5738f0 kern_readv() at kern_readv+0x4a/frame 0xfffffe00ab573930 sys_read() at sys_read+0x84/frame 0xfffffe00ab573980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00ab573ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ab573ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80043122a, rsp = 0x7fffffffebc8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2333 tid 100212 td 0xfffff8002eca25a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab56e620 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab56e650 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab56e690 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ab56e6f0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ab56e720 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ab56e780 tty_wait() at tty_wait+0x42/frame 0xfffffe00ab56e7b0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00ab56e810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00ab56e840 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00ab56e8a0 dofileread() at dofileread+0x81/frame 0xfffffe00ab56e8f0 kern_readv() at kern_readv+0x4a/frame 0xfffffe00ab56e930 sys_read() at sys_read+0x84/frame 0xfffffe00ab56e980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00ab56eab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ab56eab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80043122a, rsp = 0x7fffffffebc8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2332 tid 100211 td 0xfffff8002eb4d000 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab569620 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab569650 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab569690 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ab5696f0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ab569720 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ab569780 tty_wait() at tty_wait+0x42/frame 0xfffffe00ab5697b0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00ab569810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00ab569840 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00ab5698a0 dofileread() at dofileread+0x81/frame 0xfffffe00ab5698f0 kern_readv() at kern_readv+0x4a/frame 0xfffffe00ab569930 sys_read() at sys_read+0x84/frame 0xfffffe00ab569980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00ab569ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ab569ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80043122a, rsp = 0x7fffffffebc8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2331 tid 100210 td 0xfffff8002eb4d5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab564620 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab564650 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab564690 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ab5646f0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ab564720 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ab564780 tty_wait() at tty_wait+0x42/frame 0xfffffe00ab5647b0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00ab564810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00ab564840 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00ab5648a0 dofileread() at dofileread+0x81/frame 0xfffffe00ab5648f0 kern_readv() at kern_readv+0x4a/frame 0xfffffe00ab564930 sys_read() at sys_read+0x84/frame 0xfffffe00ab564980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00ab564ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ab564ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80043122a, rsp = 0x7fffffffebc8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2330 tid 100209 td 0xfffff80003ec0000 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab55f620 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab55f650 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab55f690 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ab55f6f0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ab55f720 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ab55f780 tty_wait() at tty_wait+0x42/frame 0xfffffe00ab55f7b0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00ab55f810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00ab55f840 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00ab55f8a0 dofileread() at dofileread+0x81/frame 0xfffffe00ab55f8f0 kern_readv() at kern_readv+0x4a/frame 0xfffffe00ab55f930 sys_read() at sys_read+0x84/frame 0xfffffe00ab55f980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00ab55fab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ab55fab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80043122a, rsp = 0x7fffffffebc8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2329 tid 100208 td 0xfffff80003ec05a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab55a620 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab55a650 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab55a690 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ab55a6f0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ab55a720 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ab55a780 tty_wait() at tty_wait+0x42/frame 0xfffffe00ab55a7b0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00ab55a810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00ab55a840 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00ab55a8a0 dofileread() at dofileread+0x81/frame 0xfffffe00ab55a8f0 kern_readv() at kern_readv+0x4a/frame 0xfffffe00ab55a930 sys_read() at sys_read+0x84/frame 0xfffffe00ab55a980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00ab55aab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ab55aab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80043122a, rsp = 0x7fffffffebc8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2328 tid 100168 td 0xfffff80003ebf5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab44f620 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab44f650 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab44f690 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ab44f6f0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ab44f720 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ab44f780 tty_wait() at tty_wait+0x42/frame 0xfffffe00ab44f7b0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00ab44f810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00ab44f840 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00ab44f8a0 dofileread() at dofileread+0x81/frame 0xfffffe00ab44f8f0 kern_readv() at kern_readv+0x4a/frame 0xfffffe00ab44f930 sys_read() at sys_read+0x84/frame 0xfffffe00ab44f980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00ab44fab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ab44fab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80043122a, rsp = 0x7fffffffebc8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2327 tid 100142 td 0xfffff800036c3000 sched_switch() at sched_switch+0x713/frame 0xfffffe00aa7c9620 mi_switch() at mi_switch+0x174/frame 0xfffffe00aa7c9650 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00aa7c9690 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00aa7c96f0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00aa7c9720 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00aa7c9780 tty_wait() at tty_wait+0x42/frame 0xfffffe00aa7c97b0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00aa7c9810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00aa7c9840 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00aa7c98a0 dofileread() at dofileread+0x81/frame 0xfffffe00aa7c98f0 kern_readv() at kern_readv+0x4a/frame 0xfffffe00aa7c9930 sys_read() at sys_read+0x84/frame 0xfffffe00aa7c9980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00aa7c9ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00aa7c9ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80043122a, rsp = 0x7fffffffebc8, rbp = 0x7fffffffec10 --- Tracing command cron pid 2281 tid 100165 td 0xfffff8002eca3000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0006f2aa80 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0006f2aa90 trap() at trap+0x5b/frame 0xfffffe0006f2aba0 calltrap() at calltrap+0x8/frame 0xfffffe0006f2aba0 --- trap 0xc, rip = 0xffffffff8107b516, rsp = 0xfffffe0006f2ac70, rbp = 0xfffffe0006f2ac80 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe0006f2ac80 _db_stack_trace_all() at _db_stack_trace_all+0xc4/frame 0xfffffe0006f2ad20 db_command() at db_command+0x279/frame 0xfffffe0006f2adf0 db_script_exec() at db_script_exec+0x188/frame 0xfffffe0006f2ae90 db_command() at db_command+0x279/frame 0xfffffe0006f2af60 db_command_loop() at db_command_loop+0x64/frame 0xfffffe0006f2af70 db_trap() at db_trap+0xef/frame 0xfffffe0006f2b000 kdb_trap() at kdb_trap+0x16c/frame 0xfffffe0006f2b080 trap() at trap+0x811/frame 0xfffffe0006f2b190 calltrap() at calltrap+0x8/frame 0xfffffe0006f2b190 --- trap 0x3, rip = 0xffffffff80c1925b, rsp = 0xfffffe0006f2b260, rbp = 0xfffffe0006f2b270 --- kdb_enter() at kdb_enter+0x3b/frame 0xfffffe0006f2b270 vpanic() at vpanic+0x1ba/frame 0xfffffe0006f2b2c0 panic() at panic+0x43/frame 0xfffffe0006f2b320 witness_unlock() at witness_unlock+0xea/frame 0xfffffe0006f2b360 __mtx_unlock_flags() at __mtx_unlock_flags+0x4d/frame 0xfffffe0006f2b390 swp_pager_trimswapspace() at swp_pager_trimswapspace+0xeb/frame 0xfffffe0006f2b3e0 swap_pager_putpages() at swap_pager_putpages+0x56c/frame 0xfffffe0006f2b480 vm_pageout_flush() at vm_pageout_flush+0xff/frame 0xfffffe0006f2b580 vm_pageout_cluster() at vm_pageout_cluster+0x3e6/frame 0xfffffe0006f2b7f0 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0x2f6/frame 0xfffffe0006f2ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f2bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f2bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sshd pid 2277 tid 100202 td 0xfffff800049ec000 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab53c5c0 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab53c5f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab53c630 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ab53c690 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ab53c6c0 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ab53c720 seltdwait() at seltdwait+0x8d/frame 0xfffffe00ab53c760 kern_select() at kern_select+0x953/frame 0xfffffe00ab53c940 sys_select() at sys_select+0x56/frame 0xfffffe00ab53c980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00ab53cab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ab53cab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80090220a, rsp = 0x7fffffffe668, rbp = 0x7fffffffe6a0 --- Tracing command ntpd pid 2248 tid 100198 td 0xfffff8002e4635a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab5285c0 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab5285f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab528630 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ab528690 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ab5286c0 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ab528720 seltdwait() at seltdwait+0x8d/frame 0xfffffe00ab528760 kern_select() at kern_select+0x953/frame 0xfffffe00ab528940 sys_select() at sys_select+0x56/frame 0xfffffe00ab528980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00ab528ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ab528ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80082e20a, rsp = 0x7fffffffdf18, rbp = 0x7fffffffdf50 --- Tracing command ntpd pid 2248 tid 100275 td 0xfffff801e463a000 sched_switch() at sched_switch+0x713/frame 0xfffffe00004416b0 mi_switch() at mi_switch+0x174/frame 0xfffffe00004416e0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0000441720 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe0000441780 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00004417b0 _sleep() at _sleep+0x2d0/frame 0xfffffe0000441850 umtxq_sleep() at umtxq_sleep+0x153/frame 0xfffffe00004418b0 do_sem2_wait() at do_sem2_wait+0x41c/frame 0xfffffe0000441930 __umtx_op_sem2_wait() at __umtx_op_sem2_wait+0x76/frame 0xfffffe0000441980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe0000441ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0000441ab0 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x80072258a, rsp = 0x7fffdff7bed8, rbp = 0x7fffdff7bf50 --- Tracing command rpc.lockd pid 2187 tid 100195 td 0xfffff800036c5000 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab51e4e0 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab51e510 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab51e550 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ab51e5b0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00ab51e5f0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x185/frame 0xfffffe00ab51e660 svc_run_internal() at svc_run_internal+0xa77/frame 0xfffffe00ab51e7a0 svc_run() at svc_run+0x24e/frame 0xfffffe00ab51e800 sys_nlm_syscall() at sys_nlm_syscall+0x77b/frame 0xfffffe00ab51e980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00ab51eab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ab51eab0 --- syscall (154, FreeBSD ELF64, sys_nlm_syscall), rip = 0x8002f062a, rsp = 0x7fffffffec08, rbp = 0x7fffffffecc0 --- Tracing command rpc.statd pid 2184 tid 100190 td 0xfffff800036c55a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab4f95a0 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab4f95d0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab4f9610 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ab4f9670 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00ab4f96b0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x185/frame 0xfffffe00ab4f9720 seltdwait() at seltdwait+0x6b/frame 0xfffffe00ab4f9760 kern_select() at kern_select+0x953/frame 0xfffffe00ab4f9940 sys_select() at sys_select+0x56/frame 0xfffffe00ab4f9980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00ab4f9ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ab4f9ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80042520a, rsp = 0x7fffffffea98, rbp = 0x7fffffffebe0 --- Tracing command watchdogd pid 2138 tid 100162 td 0xfffff8000605f5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab431710 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab431740 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab431780 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ab4317e0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00ab431820 _sleep() at _sleep+0x265/frame 0xfffffe00ab4318c0 kern_clock_nanosleep() at kern_clock_nanosleep+0x206/frame 0xfffffe00ab431940 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe00ab431980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00ab431ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ab431ab0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003bdb6a, rsp = 0x7fffffffea98, rbp = 0x7fffffffeae0 --- Tracing command rpcbind pid 2058 tid 100159 td 0xfffff8002e4625a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab422620 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab422650 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab422690 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ab4226f0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00ab422730 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x185/frame 0xfffffe00ab4227a0 seltdwait() at seltdwait+0x6b/frame 0xfffffe00ab4227e0 kern_poll() at kern_poll+0x448/frame 0xfffffe00ab422960 sys_poll() at sys_poll+0x50/frame 0xfffffe00ab422980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00ab422ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ab422ab0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8003cc35a, rsp = 0x7fffffffca38, rbp = 0x7fffffffec50 --- Tracing command syslogd pid 2048 tid 100145 td 0xfffff8002e461000 sched_switch() at sched_switch+0x713/frame 0xfffffe00aa7d85c0 mi_switch() at mi_switch+0x174/frame 0xfffffe00aa7d85f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00aa7d8630 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00aa7d8690 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00aa7d86c0 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00aa7d8720 seltdwait() at seltdwait+0x8d/frame 0xfffffe00aa7d8760 kern_select() at kern_select+0x953/frame 0xfffffe00aa7d8940 sys_select() at sys_select+0x56/frame 0xfffffe00aa7d8980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00aa7d8ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00aa7d8ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80043620a, rsp = 0x7fffffffe748, rbp = 0x7fffffffecc0 --- Tracing command devd pid 1977 tid 100174 td 0xfffff8000406b000 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab46d5a0 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab46d5d0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab46d610 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ab46d670 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00ab46d6b0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x185/frame 0xfffffe00ab46d720 seltdwait() at seltdwait+0x6b/frame 0xfffffe00ab46d760 kern_select() at kern_select+0x953/frame 0xfffffe00ab46d940 sys_select() at sys_select+0x56/frame 0xfffffe00ab46d980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00ab46dab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ab46dab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x2f530a, rsp = 0x7fffffffcb48, rbp = 0x7fffffffece0 --- Tracing command dhclient pid 1972 tid 100149 td 0xfffff8002e612000 sched_switch() at sched_switch+0x713/frame 0xfffffe00aa7ec620 mi_switch() at mi_switch+0x174/frame 0xfffffe00aa7ec650 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00aa7ec690 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00aa7ec6f0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00aa7ec730 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x185/frame 0xfffffe00aa7ec7a0 seltdwait() at seltdwait+0x6b/frame 0xfffffe00aa7ec7e0 kern_poll() at kern_poll+0x448/frame 0xfffffe00aa7ec960 sys_poll() at sys_poll+0x50/frame 0xfffffe00aa7ec980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00aa7ecab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00aa7ecab0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8003ef35a, rsp = 0x7fffffffec28, rbp = 0x7fffffffec60 --- Tracing command dhclient pid 1928 tid 100161 td 0xfffff8002ea995a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab42c640 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab42c670 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab42c6b0 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ab42c710 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ab42c740 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ab42c7a0 seltdwait() at seltdwait+0x8d/frame 0xfffffe00ab42c7e0 kern_poll() at kern_poll+0x448/frame 0xfffffe00ab42c960 sys_poll() at sys_poll+0x50/frame 0xfffffe00ab42c980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00ab42cab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ab42cab0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8003ef35a, rsp = 0x7fffffffec38, rbp = 0x7fffffffec60 --- Tracing command dhclient pid 1925 tid 100164 td 0xfffff8000406a000 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab43b5c0 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab43b5f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab43b630 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ab43b690 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ab43b6c0 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ab43b720 seltdwait() at seltdwait+0x8d/frame 0xfffffe00ab43b760 kern_select() at kern_select+0x953/frame 0xfffffe00ab43b940 sys_select() at sys_select+0x56/frame 0xfffffe00ab43b980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00ab43bab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ab43bab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80044d20a, rsp = 0x7fffffffe9e8, rbp = 0x7fffffffeaa0 --- Tracing command syncer pid 34 tid 100141 td 0xfffff8002e611000 sched_switch() at sched_switch+0x713/frame 0xfffffe00aa7c48c0 mi_switch() at mi_switch+0x174/frame 0xfffffe00aa7c48f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00aa7c4930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00aa7c4970 _cv_timedwait_sbt() at _cv_timedwait_sbt+0x185/frame 0xfffffe00aa7c49e0 sched_sync() at sched_sync+0x609/frame 0xfffffe00aa7c4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00aa7c4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00aa7c4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command vnlru pid 33 tid 100140 td 0xfffff8002e6115a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00a37fd890 mi_switch() at mi_switch+0x174/frame 0xfffffe00a37fd8c0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00a37fd900 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00a37fd940 _sleep() at _sleep+0x279/frame 0xfffffe00a37fd9e0 vnlru_proc() at vnlru_proc+0xa4/frame 0xfffffe00a37fda70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a37fdab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a37fdab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100132 td 0xfffff8002e610000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006f358d0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006f35900 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006f35940 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006f35980 _sleep() at _sleep+0x279/frame 0xfffffe0006f35a20 buf_daemon() at buf_daemon+0xc8/frame 0xfffffe0006f35a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f35ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f35ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100133 td 0xfffff800048c45a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006ffa8c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ffa8f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ffa930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006ffa970 _sleep() at _sleep+0x279/frame 0xfffffe0006ffaa10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe0006ffaa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ffaab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ffaab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100134 td 0xfffff800048c4000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006fff8c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006fff8f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006fff930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006fff970 _sleep() at _sleep+0x279/frame 0xfffffe0006fffa10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe0006fffa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006fffab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006fffab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100135 td 0xfffff800048c55a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00a37c48c0 mi_switch() at mi_switch+0x174/frame 0xfffffe00a37c48f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00a37c4930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00a37c4970 _sleep() at _sleep+0x279/frame 0xfffffe00a37c4a10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe00a37c4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a37c4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a37c4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100136 td 0xfffff800048c5000 sched_switch() at sched_switch+0x713/frame 0xfffffe00a37e98c0 mi_switch() at mi_switch+0x174/frame 0xfffffe00a37e98f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00a37e9930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00a37e9970 _sleep() at _sleep+0x279/frame 0xfffffe00a37e9a10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe00a37e9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a37e9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a37e9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100137 td 0xfffff800048c65a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00a37ee8c0 mi_switch() at mi_switch+0x174/frame 0xfffffe00a37ee8f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00a37ee930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00a37ee970 _sleep() at _sleep+0x279/frame 0xfffffe00a37eea10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe00a37eea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a37eeab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a37eeab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100138 td 0xfffff800048c6000 sched_switch() at sched_switch+0x713/frame 0xfffffe00a37f38c0 mi_switch() at mi_switch+0x174/frame 0xfffffe00a37f38f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00a37f3930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00a37f3970 _sleep() at _sleep+0x279/frame 0xfffffe00a37f3a10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe00a37f3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a37f3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a37f3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100139 td 0xfffff800048ce5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00a37f88c0 mi_switch() at mi_switch+0x174/frame 0xfffffe00a37f88f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00a37f8930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00a37f8970 _sleep() at _sleep+0x279/frame 0xfffffe00a37f8a10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe00a37f8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a37f8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a37f8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100154 td 0xfffff80003ebe000 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab4098f0 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab409920 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab409960 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00ab4099a0 _sleep() at _sleep+0x279/frame 0xfffffe00ab409a40 softdep_flush() at softdep_flush+0x17f/frame 0xfffffe00ab409a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00ab409ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ab409ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100197 td 0xfffff8000406c000 sched_switch() at sched_switch+0x713/frame 0xfffffe00ab41d8f0 mi_switch() at mi_switch+0x174/frame 0xfffffe00ab41d920 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ab41d960 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00ab41d9a0 _sleep() at _sleep+0x279/frame 0xfffffe00ab41da40 softdep_flush() at softdep_flush+0x17f/frame 0xfffffe00ab41da70 fork_exit() at fork_exit+0x84/frame 0xfffffe00ab41dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ab41dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command vmdaemon pid 31 tid 100129 td 0xfffff80003adf5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006f268c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006f268f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006f26930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f26960 _sleep() at _sleep+0x2da/frame 0xfffffe0006f26a00 vm_daemon() at vm_daemon+0xabf/frame 0xfffffe0006f26a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f26ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f26ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 30 tid 100128 td 0xfffff8002e523000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006f214e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006f21510 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006f21550 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006f21590 _sleep() at _sleep+0x279/frame 0xfffffe0006f21630 pause_sbt() at pause_sbt+0x10f/frame 0xfffffe0006f21670 vm_pageout_worker() at vm_pageout_worker+0x21d/frame 0xfffffe0006f21a30 vm_pageout() at vm_pageout+0x156/frame 0xfffffe0006f21a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f21ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f21ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 30 tid 100130 td 0xfffff800049ea000 (CPU 1) kdb_enter() at kdb_enter+0x3b/frame 0xfffffe0006f2b270 vpanic() at vpanic+0x1ba/frame 0xfffffe0006f2b2c0 panic() at panic+0x43/frame 0xfffffe0006f2b320 witness_unlock() at witness_unlock+0xea/frame 0xfffffe0006f2b360 __mtx_unlock_flags() at __mtx_unlock_flags+0x4d/frame 0xfffffe0006f2b390 swp_pager_trimswapspace() at swp_pager_trimswapspace+0xeb/frame 0xfffffe0006f2b3e0 swap_pager_putpages() at swap_pager_putpages+0x56c/frame 0xfffffe0006f2b480 vm_pageout_flush() at vm_pageout_flush+0xff/frame 0xfffffe0006f2b580 vm_pageout_cluster() at vm_pageout_cluster+0x3e6/frame 0xfffffe0006f2b7f0 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0x2f6/frame 0xfffffe0006f2ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f2bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f2bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 30 tid 100131 td 0xfffff800049eb5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006f308f0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006f30920 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006f30960 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006f309a0 _sleep() at _sleep+0x279/frame 0xfffffe0006f30a40 uma_reclaim_worker() at uma_reclaim_worker+0xda/frame 0xfffffe0006f30a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f30ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f30ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon1 pid 29 tid 100127 td 0xfffff80003ae25a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006f108b0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006f108e0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006f10920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f10950 _sleep() at _sleep+0x2da/frame 0xfffffe0006f109f0 enc_daemon() at enc_daemon+0x1cf/frame 0xfffffe0006f10a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f10ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f10ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon0 pid 28 tid 100126 td 0xfffff80003ae1000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006f0b8b0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006f0b8e0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006f0b920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f0b950 _sleep() at _sleep+0x2da/frame 0xfffffe0006f0b9f0 enc_daemon() at enc_daemon+0x1cf/frame 0xfffffe0006f0ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f0bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f0bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command rand_harvestq pid 27 tid 100123 td 0xfffff8002e5155a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006efc8c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006efc8f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006efc930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006efc970 _sleep() at _sleep+0x279/frame 0xfffffe0006efca10 random_kthread() at random_kthread+0x26a/frame 0xfffffe0006efca70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006efcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006efcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sctp_iterator pid 26 tid 100119 td 0xfffff8002e5135a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006edc910 mi_switch() at mi_switch+0x174/frame 0xfffffe0006edc940 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006edc980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006edc9b0 _sleep() at _sleep+0x2da/frame 0xfffffe0006edca50 sctp_iterator_thread() at sctp_iterator_thread+0x59/frame 0xfffffe0006edca70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006edcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006edcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod4 pid 25 tid 100118 td 0xfffff80003ae15a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006ecb8e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ecb910 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ecb950 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006ecb990 _sleep() at _sleep+0x279/frame 0xfffffe0006ecba30 soaio_kproc_loop() at soaio_kproc_loop+0x170/frame 0xfffffe0006ecba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ecbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ecbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod3 pid 24 tid 100117 td 0xfffff80003ae0000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006ec68e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ec6910 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ec6950 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006ec6990 _sleep() at _sleep+0x279/frame 0xfffffe0006ec6a30 soaio_kproc_loop() at soaio_kproc_loop+0x170/frame 0xfffffe0006ec6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ec6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ec6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod2 pid 23 tid 100116 td 0xfffff80003ae05a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006ec18e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ec1910 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ec1950 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006ec1990 _sleep() at _sleep+0x279/frame 0xfffffe0006ec1a30 soaio_kproc_loop() at soaio_kproc_loop+0x170/frame 0xfffffe0006ec1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ec1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ec1ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod1 pid 22 tid 100115 td 0xfffff80003adf000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006ebc8e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ebc910 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ebc950 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006ebc990 _sleep() at _sleep+0x279/frame 0xfffffe0006ebca30 soaio_kproc_loop() at soaio_kproc_loop+0x170/frame 0xfffffe0006ebca70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ebcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ebcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100096 td 0xfffff80003b7c5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e45930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e45960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e459a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e459d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e45a30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e45a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e45ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e45ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100097 td 0xfffff80003b7c000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e4a930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e4a960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e4a9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e4a9d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e4aa30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e4aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e4aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e4aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100098 td 0xfffff80003b7d5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e4f930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e4f960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e4f9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e4f9d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e4fa30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e4fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e4fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e4fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100099 td 0xfffff80003b7d000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e54930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e54960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e549a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e549d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e54a30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e54a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e54ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e54ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100100 td 0xfffff80003b875a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e59930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e59960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e599a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e599d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e59a30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e59a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e59ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e59ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100102 td 0xfffff800040685a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e63930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e63960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e639a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e639d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e63a30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e63a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e63ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e63ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100103 td 0xfffff800049e8000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e68930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e68960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e689a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e689d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e68a30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e68a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e68ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e68ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100104 td 0xfffff800049e95a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e6d930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e6d960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e6d9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e6d9d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e6da30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e6da70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e6dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e6dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100105 td 0xfffff800049e9000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e72930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e72960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e729a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e729d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e72a30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e72a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e72ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e72ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100106 td 0xfffff800049ea5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e77930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e77960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e779a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e779d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e77a30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e77a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e77ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e77ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100107 td 0xfffff80003c5c5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e94930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e94960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e949a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e949d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e94a30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e94a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e94ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e94ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100108 td 0xfffff80003c5c000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e99930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e99960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e999a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e999d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e99a30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e99a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e99ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e99ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100109 td 0xfffff80003c5d5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e9e930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e9e960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e9e9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e9e9d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e9ea30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e9ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e9eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e9eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100110 td 0xfffff80003c5d000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006ea3930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ea3960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ea39a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006ea39d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006ea3a30 usb_process() at usb_process+0x13b/frame 0xfffffe0006ea3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ea3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ea3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100111 td 0xfffff80003c5e5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006ea8930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ea8960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ea89a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006ea89d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006ea8a30 usb_process() at usb_process+0x13b/frame 0xfffffe0006ea8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ea8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ea8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100091 td 0xfffff80004068000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e2c8e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e2c910 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e2c950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e2c980 _sleep() at _sleep+0x2da/frame 0xfffffe0006e2ca20 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe0006e2ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e2cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e2cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100092 td 0xfffff800040695a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e318e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e31910 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e31950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e31980 _sleep() at _sleep+0x2da/frame 0xfffffe0006e31a20 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe0006e31a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e31ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e31ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100093 td 0xfffff80004069000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e368e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e36910 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e36950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e36980 _sleep() at _sleep+0x2da/frame 0xfffffe0006e36a20 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe0006e36a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e36ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e36ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100125 td 0xfffff8002e5235a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006f068d0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006f06900 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006f06940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f06970 _sleep() at _sleep+0x2da/frame 0xfffffe0006f06a10 xpt_scanner_thread() at xpt_scanner_thread+0x93/frame 0xfffffe0006f06a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f06ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f06ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 11 pid 19 tid 100090 td 0xfffff80003fe95a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e278c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e278f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e27930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e27960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e27a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e27a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e27ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e27ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 10 pid 18 tid 100089 td 0xfffff80003fdd000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e228c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e228f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e22930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e22960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e22a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e22a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e22ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e22ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 9 pid 17 tid 100088 td 0xfffff80003fdd5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e1d8c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e1d8f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e1d930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e1d960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e1da00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e1da70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e1dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e1dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 8 pid 16 tid 100087 td 0xfffff80003fdc000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e188c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e188f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e18930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e18960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e18a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e18a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e18ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e18ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 7 pid 15 tid 100086 td 0xfffff80003fdc5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e138c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e138f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e13930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e13960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e13a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e13a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e13ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e13ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 6 pid 9 tid 100085 td 0xfffff80003fdb000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e0e8c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e0e8f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e0e930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e0e960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e0ea00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e0ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e0eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e0eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 5 pid 8 tid 100084 td 0xfffff80003fdb5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e098c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e098f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e09930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e09960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e09a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e09a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e09ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e09ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 4 pid 7 tid 100083 td 0xfffff80003fda000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e048c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e048f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e04930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e04960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e04a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e04a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e04ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e04ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 3 pid 6 tid 100082 td 0xfffff80003fda5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00005fd8c0 mi_switch() at mi_switch+0x174/frame 0xfffffe00005fd8f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005fd930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005fd960 _sleep() at _sleep+0x2da/frame 0xfffffe00005fda00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00005fda70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005fdab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005fdab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 2 pid 5 tid 100081 td 0xfffff80003ff1000 sched_switch() at sched_switch+0x713/frame 0xfffffe00005f88c0 mi_switch() at mi_switch+0x174/frame 0xfffffe00005f88f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005f8930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005f8960 _sleep() at _sleep+0x2da/frame 0xfffffe00005f8a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00005f8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005f8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005f8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 1 pid 4 tid 100080 td 0xfffff80003ff15a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00005f38c0 mi_switch() at mi_switch+0x174/frame 0xfffffe00005f38f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005f3930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005f3960 _sleep() at _sleep+0x2da/frame 0xfffffe00005f3a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00005f3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005f3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005f3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 0 pid 3 tid 100079 td 0xfffff80003ff0000 sched_switch() at sched_switch+0x713/frame 0xfffffe00005ee8c0 mi_switch() at mi_switch+0x174/frame 0xfffffe00005ee8f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005ee930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005ee960 _sleep() at _sleep+0x2da/frame 0xfffffe00005eea00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00005eea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005eeab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005eeab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 2 tid 100078 td 0xfffff80003ff05a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00005e98f0 mi_switch() at mi_switch+0x174/frame 0xfffffe00005e9920 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005e9960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005e9990 _sleep() at _sleep+0x2da/frame 0xfffffe00005e9a30 crypto_proc() at crypto_proc+0x2c4/frame 0xfffffe00005e9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005e9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005e9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sequencer 00 pid 14 tid 100065 td 0xfffff80003914000 sched_switch() at sched_switch+0x713/frame 0xfffffe00005a88f0 mi_switch() at mi_switch+0x174/frame 0xfffffe00005a8920 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005a8960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005a8990 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe00005a89f0 seq_eventthread() at seq_eventthread+0xdc/frame 0xfffffe00005a8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005a8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005a8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100062 td 0xfffff800039af5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe000058d8f0 mi_switch() at mi_switch+0x174/frame 0xfffffe000058d920 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe000058d960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000058d990 _sleep() at _sleep+0x2da/frame 0xfffffe000058da30 g_run_events() at g_run_events+0x4f/frame 0xfffffe000058da70 fork_exit() at fork_exit+0x84/frame 0xfffffe000058dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000058dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100063 td 0xfffff800039af000 sched_switch() at sched_switch+0x713/frame 0xfffffe00005928f0 mi_switch() at mi_switch+0x174/frame 0xfffffe0000592920 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0000592960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0000592990 _sleep() at _sleep+0x2da/frame 0xfffffe0000592a30 g_io_schedule_up() at g_io_schedule_up+0xa5/frame 0xfffffe0000592a60 g_up_procbody() at g_up_procbody+0x6d/frame 0xfffffe0000592a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000592ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000592ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100064 td 0xfffff800039145a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00005978f0 mi_switch() at mi_switch+0x174/frame 0xfffffe0000597920 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0000597960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0000597990 _sleep() at _sleep+0x2da/frame 0xfffffe0000597a30 g_io_schedule_down() at g_io_schedule_down+0x14c/frame 0xfffffe0000597a60 g_down_procbody() at g_down_procbody+0x6d/frame 0xfffffe0000597a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000597ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000597ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100017 td 0xfffff8000379d000 sched_switch() at sched_switch+0x713/frame 0xfffffe00767b89e0 mi_switch() at mi_switch+0x174/frame 0xfffffe00767b8a10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe00767b8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00767b8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00767b8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100018 td 0xfffff800037aa5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00767bd9e0 mi_switch() at mi_switch+0x174/frame 0xfffffe00767bda10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe00767bda70 fork_exit() at fork_exit+0x84/frame 0xfffffe00767bdab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00767bdab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100020 td 0xfffff800037645a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100047 td 0xfffff800039be000 sched_switch() at sched_switch+0x713/frame 0xfffffe00004ee9e0 mi_switch() at mi_switch+0x174/frame 0xfffffe00004eea10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe00004eea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004eeab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004eeab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100048 td 0xfffff800039bf5a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100049 td 0xfffff800039bf000 sched_switch() at sched_switch+0x713/frame 0xfffffe00004f89e0 mi_switch() at mi_switch+0x174/frame 0xfffffe00004f8a10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe00004f8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004f8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004f8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100050 td 0xfffff800039c05a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100051 td 0xfffff800039c0000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100052 td 0xfffff800039ce5a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100053 td 0xfffff800039ce000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100054 td 0xfffff800037235a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100055 td 0xfffff800039ab000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100056 td 0xfffff800039ac5a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100057 td 0xfffff800039ac000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100058 td 0xfffff800039ad5a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100059 td 0xfffff800039ad000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100060 td 0xfffff800039ae5a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100094 td 0xfffff80003b7b5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e3b9e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e3ba10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe0006e3ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e3bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e3bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100095 td 0xfffff80003b7b000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e409e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e40a10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe0006e40a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e40ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e40ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100101 td 0xfffff80003b87000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e5e9e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e5ea10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe0006e5ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e5eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e5eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100112 td 0xfffff80003c5e000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100113 td 0xfffff80003c6e5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006eb29e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006eb2a10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe0006eb2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006eb2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006eb2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100177 td 0xfffff8002eca4000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100178 td 0xfffff8002eca55a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100179 td 0xfffff8002eca5000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100180 td 0xfffff8002ecaf5a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100181 td 0xfffff8002ecaf000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100182 td 0xfffff8003c7b95a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100183 td 0xfffff8003c7b9000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100184 td 0xfffff801370305a0 fork_trampoline() at fork_trampoline Tracing command idle pid 11 tid 100003 td 0xfffff800037ab000 (CPU 0) cpustop_handler() at cpustop_handler+0x2d/frame 0xffffffff82051af0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xffffffff82051b10 trap() at trap+0x48/frame 0xffffffff82051c20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff82051c20 --- trap 0x13, rip = 0xffffffff8120f3c6, rsp = 0xfffffe0076766900, rbp = 0xfffffe0076766900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076766900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076766950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076766970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe0076766990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe0076766a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076766ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076766ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xfffff800037ac5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe007676b940 mi_switch() at mi_switch+0x174/frame 0xfffffe007676b970 critical_exit_preempt() at critical_exit_preempt+0x65/frame 0xfffffe007676b990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe007676ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe007676bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007676bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffff800037ac000 (CPU 2) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006b7fdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006b7fe10 trap() at trap+0x48/frame 0xfffffe0006b7ff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b7ff20 --- trap 0x13, rip = 0xffffffff8120f3c6, rsp = 0xfffffe0076770900, rbp = 0xfffffe0076770900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076770900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076770950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076770970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe0076770990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe0076770a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076770ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076770ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xfffff800037ad5a0 (CPU 3) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006b8bdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006b8be10 trap() at trap+0x48/frame 0xfffffe0006b8bf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b8bf20 --- trap 0x13, rip = 0xffffffff8120f3c6, rsp = 0xfffffe0076775900, rbp = 0xfffffe0076775900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076775900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076775950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076775970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe0076775990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe0076775a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076775ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076775ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100007 td 0xfffff800037ad000 (CPU 4) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006b97df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006b97e10 trap() at trap+0x48/frame 0xfffffe0006b97f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b97f20 --- trap 0x13, rip = 0xffffffff8120f3c6, rsp = 0xfffffe007677a900, rbp = 0xfffffe007677a900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007677a900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe007677a950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007677a970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe007677a990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe007677aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007677aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007677aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xfffff800037ae5a0 (CPU 5) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006ba3df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006ba3e10 trap() at trap+0x48/frame 0xfffffe0006ba3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006ba3f20 --- trap 0x13, rip = 0xffffffff8120f3c6, rsp = 0xfffffe007677f900, rbp = 0xfffffe007677f900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007677f900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe007677f950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007677f970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe007677f990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe007677fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007677fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007677fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xfffff800037ae000 (CPU 6) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bafdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bafe10 trap() at trap+0x48/frame 0xfffffe0006baff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006baff20 --- trap 0x13, rip = 0xffffffff8120f3c6, rsp = 0xfffffe0076784900, rbp = 0xfffffe0076784900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076784900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076784950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076784970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe0076784990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe0076784a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076784ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076784ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xfffff8000379a5a0 (CPU 7) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bbbdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bbbe10 trap() at trap+0x48/frame 0xfffffe0006bbbf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bbbf20 --- trap 0x13, rip = 0xffffffff8120f3c6, rsp = 0xfffffe0076789900, rbp = 0xfffffe0076789900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076789900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076789950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076789970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe0076789990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe0076789a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076789ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076789ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffff8000379a000 (CPU 8) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bc7df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bc7e10 trap() at trap+0x48/frame 0xfffffe0006bc7f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bc7f20 --- trap 0x13, rip = 0xffffffff8120f3c6, rsp = 0xfffffe007678e900, rbp = 0xfffffe007678e900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007678e900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe007678e950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007678e970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe007678e990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe007678ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe007678eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007678eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100012 td 0xfffff8000379b5a0 (CPU 9) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bd3df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bd3e10 trap() at trap+0x48/frame 0xfffffe0006bd3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bd3f20 --- trap 0x13, rip = 0xffffffff8120f3c6, rsp = 0xfffffe0076793900, rbp = 0xfffffe0076793900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076793900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076793950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076793970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe0076793990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe0076793a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076793ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076793ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xfffff8000379b000 (CPU 10) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bdfdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bdfe10 trap() at trap+0x48/frame 0xfffffe0006bdff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bdff20 --- trap 0x13, rip = 0xffffffff8120f3c6, rsp = 0xfffffe0076798900, rbp = 0xfffffe0076798900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076798900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076798950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076798970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe0076798990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe0076798a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076798ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076798ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100014 td 0xfffff8000379c5a0 (CPU 11) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bebdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bebe10 trap() at trap+0x48/frame 0xfffffe0006bebf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bebf20 --- trap 0x13, rip = 0xffffffff8120f3c6, rsp = 0xfffffe007679d900, rbp = 0xfffffe007679d900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007679d900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe007679d950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007679d970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe007679d990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe007679da70 fork_exit() at fork_exit+0x84/frame 0xfffffe007679dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007679dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command init pid 1 tid 100002 td 0xfffff800037ab5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0076761560 mi_switch() at mi_switch+0x174/frame 0xfffffe0076761590 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00767615d0 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe0076761630 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0076761660 _sleep() at _sleep+0x2d0/frame 0xfffffe0076761700 kern_wait6() at kern_wait6+0x48f/frame 0xfffffe0076761790 sys_wait4() at sys_wait4+0x78/frame 0xfffffe0076761980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe0076761ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0076761ab0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x2b10da, rsp = 0x7fffffffe778, rbp = 0x7fffffffe7e0 --- Tracing command audit pid 10 tid 100001 td 0xfffff800037aa000 sched_switch() at sched_switch+0x713/frame 0xfffffe007675c910 mi_switch() at mi_switch+0x174/frame 0xfffffe007675c940 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe007675c980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe007675c9b0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe007675ca10 audit_worker() at audit_worker+0x93/frame 0xfffffe007675ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe007675cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007675cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xffffffff82086640 sched_switch() at sched_switch+0x713/frame 0xffffffff828ea980 mi_switch() at mi_switch+0x174/frame 0xffffffff828ea9b0 sleepq_switch() at sleepq_switch+0x110/frame 0xffffffff828ea9f0 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xffffffff828eaa30 _sleep() at _sleep+0x279/frame 0xffffffff828eaad0 swapper() at swapper+0x77/frame 0xffffffff828eab20 mi_startup() at mi_startup+0x2ee/frame 0xffffffff828eab70 btext() at btext+0x2c Tracing command kernel pid 0 tid 100015 td 0xfffff8000379c000 sched_switch() at sched_switch+0x713/frame 0xfffffe00767ae900 mi_switch() at mi_switch+0x174/frame 0xfffffe00767ae930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00767ae970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00767ae9a0 _sleep() at _sleep+0x2da/frame 0xfffffe00767aea40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00767aea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00767aeab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00767aeab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xfffff8000379d5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00767b3900 mi_switch() at mi_switch+0x174/frame 0xfffffe00767b3930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00767b3970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00767b39a0 _sleep() at _sleep+0x2da/frame 0xfffffe00767b3a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00767b3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00767b3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00767b3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xfffff80003763000 sched_switch() at sched_switch+0x713/frame 0xfffffe00767c2900 mi_switch() at mi_switch+0x174/frame 0xfffffe00767c2930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00767c2970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00767c29a0 _sleep() at _sleep+0x2da/frame 0xfffffe00767c2a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00767c2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00767c2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00767c2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xfffff80003764000 sched_switch() at sched_switch+0x713/frame 0xfffffe00767cc930 mi_switch() at mi_switch+0x174/frame 0xfffffe00767cc960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00767cc9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00767cc9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00767cca40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00767cca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00767ccab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00767ccab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xfffff800037655a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0000471930 mi_switch() at mi_switch+0x174/frame 0xfffffe0000471960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004719a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004719d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0000471a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000471a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000471ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000471ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xfffff80003765000 sched_switch() at sched_switch+0x713/frame 0xfffffe0000476930 mi_switch() at mi_switch+0x174/frame 0xfffffe0000476960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004769a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004769d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0000476a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000476a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000476ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000476ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xfffff800037665a0 sched_switch() at sched_switch+0x713/frame 0xfffffe000047b930 mi_switch() at mi_switch+0x174/frame 0xfffffe000047b960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe000047b9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000047b9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe000047ba40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000047ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe000047bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000047bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xfffff80003766000 sched_switch() at sched_switch+0x713/frame 0xfffffe0000480930 mi_switch() at mi_switch+0x174/frame 0xfffffe0000480960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004809a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004809d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0000480a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000480a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000480ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000480ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100026 td 0xfffff800037675a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0000485930 mi_switch() at mi_switch+0x174/frame 0xfffffe0000485960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004859a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004859d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0000485a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000485a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000485ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000485ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100027 td 0xfffff80003767000 sched_switch() at sched_switch+0x713/frame 0xfffffe000048a930 mi_switch() at mi_switch+0x174/frame 0xfffffe000048a960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe000048a9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000048a9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe000048aa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000048aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe000048aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000048aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100028 td 0xfffff800037285a0 sched_switch() at sched_switch+0x713/frame 0xfffffe000048f930 mi_switch() at mi_switch+0x174/frame 0xfffffe000048f960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe000048f9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000048f9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe000048fa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000048fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe000048fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000048fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100029 td 0xfffff80003728000 sched_switch() at sched_switch+0x713/frame 0xfffffe0000494930 mi_switch() at mi_switch+0x174/frame 0xfffffe0000494960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004949a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004949d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0000494a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000494a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000494ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000494ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100030 td 0xfffff800037295a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0000499930 mi_switch() at mi_switch+0x174/frame 0xfffffe0000499960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004999a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004999d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0000499a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000499a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000499ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000499ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100031 td 0xfffff80003729000 sched_switch() at sched_switch+0x713/frame 0xfffffe000049e930 mi_switch() at mi_switch+0x174/frame 0xfffffe000049e960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe000049e9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000049e9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe000049ea40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000049ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe000049eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000049eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100032 td 0xfffff8000372a5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00004a3930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004a3960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004a39a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004a39d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004a3a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004a3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004a3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004a3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100033 td 0xfffff8000372a000 sched_switch() at sched_switch+0x713/frame 0xfffffe00004a8930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004a8960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004a89a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004a89d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004a8a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004a8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004a8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004a8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100034 td 0xfffff8000372b5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00004ad930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004ad960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004ad9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004ad9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004ada40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004ada70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004adab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004adab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100035 td 0xfffff8000372b000 sched_switch() at sched_switch+0x713/frame 0xfffffe00004b2930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004b2960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004b29a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004b29d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004b2a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004b2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004b2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004b2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100036 td 0xfffff800037635a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00004b7930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004b7960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004b79a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004b79d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004b7a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004b7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004b7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004b7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100037 td 0xfffff80003723000 sched_switch() at sched_switch+0x713/frame 0xfffffe00004bc930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004bc960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004bc9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004bc9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004bca40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004bca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004bcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004bcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100038 td 0xfffff800037245a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00004c1930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004c1960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004c19a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004c19d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004c1a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004c1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004c1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004c1ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100039 td 0xfffff80003724000 sched_switch() at sched_switch+0x713/frame 0xfffffe00004c6930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004c6960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004c69a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004c69d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004c6a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004c6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004c6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004c6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100040 td 0xfffff800037255a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00004cb930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004cb960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004cb9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004cb9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004cba40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004cba70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004cbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004cbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100041 td 0xfffff80003725000 sched_switch() at sched_switch+0x713/frame 0xfffffe00004d0930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004d0960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004d09a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004d09d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004d0a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004d0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004d0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004d0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100042 td 0xfffff800037265a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00004d5930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004d5960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004d59a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004d59d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004d5a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004d5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004d5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004d5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100043 td 0xfffff80003726000 sched_switch() at sched_switch+0x713/frame 0xfffffe00004da930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004da960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004da9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004da9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004daa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004daa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004daab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004daab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100044 td 0xfffff800037275a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00004df930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004df960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004df9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004df9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004dfa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004dfa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004dfab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004dfab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100045 td 0xfffff80003727000 sched_switch() at sched_switch+0x713/frame 0xfffffe00004e4930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004e4960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004e49a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004e49d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004e4a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004e4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004e4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004e4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100046 td 0xfffff800039be5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00004e9930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004e9960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004e99a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004e99d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004e9a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004e9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004e9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004e9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100061 td 0xfffff800039ae000 sched_switch() at sched_switch+0x713/frame 0xfffffe0000588900 mi_switch() at mi_switch+0x174/frame 0xfffffe0000588930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0000588970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005889a0 _sleep() at _sleep+0x2da/frame 0xfffffe0000588a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0000588a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000588ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000588ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100066 td 0xfffff800039155a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00005ad900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005ad930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005ad970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005ad9a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005ada40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005ada70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005adab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005adab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100067 td 0xfffff80003915000 sched_switch() at sched_switch+0x713/frame 0xfffffe00005b2900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005b2930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005b2970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005b29a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005b2a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005b2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005b2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005b2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100068 td 0xfffff800039165a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00005b7900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005b7930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005b7970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005b79a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005b7a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005b7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005b7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005b7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100069 td 0xfffff80003916000 sched_switch() at sched_switch+0x713/frame 0xfffffe00005bc900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005bc930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005bc970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005bc9a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005bca40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005bca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005bcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005bcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100070 td 0xfffff800039175a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00005c1900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005c1930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005c1970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005c19a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005c1a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005c1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005c1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005c1ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100071 td 0xfffff80003917000 sched_switch() at sched_switch+0x713/frame 0xfffffe00005c6900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005c6930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005c6970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005c69a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005c6a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005c6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005c6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005c6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100072 td 0xfffff800039ab5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00005cb900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005cb930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005cb970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005cb9a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005cba40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005cba70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005cbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005cbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100073 td 0xfffff80003fe9000 sched_switch() at sched_switch+0x713/frame 0xfffffe00005d0900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005d0930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005d0970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005d09a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005d0a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005d0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005d0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005d0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100074 td 0xfffff80003fee5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00005d5900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005d5930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005d5970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005d59a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005d5a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005d5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005d5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005d5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100075 td 0xfffff80003fee000 sched_switch() at sched_switch+0x713/frame 0xfffffe00005da900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005da930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005da970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005da9a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005daa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005daa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005daab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005daab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100076 td 0xfffff80003fef5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00005df900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005df930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005df970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005df9a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005dfa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005dfa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005dfab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005dfab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100077 td 0xfffff80003fef000 sched_switch() at sched_switch+0x713/frame 0xfffffe00005e4900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005e4930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005e4970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005e49a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005e4a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005e4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005e4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005e4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100114 td 0xfffff80003c6e000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006eb7930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006eb7960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006eb79a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006eb79d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0006eb7a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0006eb7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006eb7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006eb7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100120 td 0xfffff8002e513000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006eed930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006eed960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006eed9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006eed9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0006eeda40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0006eeda70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006eedab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006eedab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100121 td 0xfffff8002e5145a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006ef2930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ef2960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ef29a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006ef29d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0006ef2a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0006ef2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ef2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ef2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100122 td 0xfffff8002e514000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006ef7930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ef7960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ef79a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006ef79d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0006ef7a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0006ef7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ef7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ef7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100124 td 0xfffff8002e515000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006f01900 mi_switch() at mi_switch+0x174/frame 0xfffffe0006f01930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006f01970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f019a0 _sleep() at _sleep+0x2da/frame 0xfffffe0006f01a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006f01a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f01ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f01ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db:0:pho> dump Dumping 1683 out of 32621 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Dump complete db> (kgdb) bt #0 doadump (textdump=0x0) at include/pcpu.h:246 #1 0xffffffff8049d29b in db_dump (dummy=, dummy3=, dummy4=) at ../../../ddb/db_command.c:575 #2 0xffffffff8049d069 in db_command (cmd_table=, dopager=0x0) at ../../../ddb/db_command.c:482 #3 0xffffffff804a1fe8 in db_script_exec (scriptname=, warnifnotfound=) at ../../../ddb/db_script.c:304 #4 0xffffffff8049d069 in db_command (cmd_table=, dopager=0x1) at ../../../ddb/db_command.c:482 #5 0xffffffff8049cde4 in db_command_loop () at ../../../ddb/db_command.c:535 #6 0xffffffff8049ff8f in db_trap (type=, code=) at ../../../ddb/db_main.c:252 #7 0xffffffff80c19b4c in kdb_trap (type=0x3, code=0x0, tf=) at ../../../kern/subr_kdb.c:692 #8 0xffffffff810a4c21 in trap (frame=0xfffffe0006f2b1a0) at ../../../amd64/amd64/trap.c:621 #9 0xffffffff8107d365 in calltrap () at ../../../amd64/amd64/exception.S:232 #10 0xffffffff80c1925b in kdb_enter (why=0xffffffff8133a14e "panic", msg=) at include/cpufunc.h:65 #11 0xffffffff80bce94a in vpanic (fmt=, ap=) at ../../../kern/kern_shutdown.c:894 #12 0xffffffff80bce6c3 in panic (fmt=) at ../../../kern/kern_shutdown.c:832 #13 0xffffffff80c3b79a in witness_unlock (lock=0xffffffff81e88898, flags=0x80, file=0xfffffe0006f2b160 "", line=0x363) at ../../../kern/subr_witness.c:1635 #14 0xffffffff80bad77d in __mtx_unlock_flags (c=0xffffffff820497b0, opts=0x0, file=, line=) at ../../../kern/kern_mutex.c:278 #15 0xffffffff80ef7ccb in swp_pager_trimswapspace () at ../../../vm/swap_pager.c:867 #16 0xffffffff80ef696c in swap_pager_putpages (object=0xfffff8003c00d300, ma=0xfffffe0006f2b5e8, count=0x20, flags=0x4, rtvals=0xfffffe0006f2b490) at ../../../vm/swap_pager.c:1593 #17 0xffffffff80f230ef in vm_pageout_flush (mc=0xfffffe0006f2b5e8, count=0x20, flags=0x4, mreq=0x0, prunlen=0x0, eio=0x0) at vm_pager.h:131 #18 0xffffffff80f26816 in vm_pageout_cluster (m=) at ../../../vm/vm_pageout.c:421 #19 0xffffffff80f25566 in vm_pageout_laundry_worker (arg=) at ../../../vm/vm_pageout.c:667 #20 0xffffffff80b8cb34 in fork_exit (callout=0xffffffff80f25270 , arg=0x0, frame=0xfffffe0006f2bac0) at ../../../kern/kern_fork.c:1057 #21 0xffffffff8107e35e in fork_trampoline () at ../../../amd64/amd64/exception.S:996 #22 0x0000000000000000 in ?? () Current language: auto; currently minimal (kgdb) f 15 #15 0xffffffff80ef7ccb in swp_pager_trimswapspace () at ../../../vm/swap_pager.c:867 867 mtx_unlock(&sw_dev_mtx); (kgdb) l 862 &npages, max_trim_alloc); 863 864 /* Quit if there's nothing free. */ 865 if (blk == SWAPBLK_NONE) 866 break; 867 mtx_unlock(&sw_dev_mtx); 868 CTR5(KTR_SPARE5, "%s: cursor %p blk %p " 869 "size %5d trimmer %p", __func__, 870 (void*)cursor, (void*)blk, npages, 871 (void*)sp->sw_trimmer); (kgdb) info loc sp = (struct swdevt *) 0xfffff80003a6c300 bp = (struct buf *) 0xfffff8002e86d400 blk = 0x16 cursor = 0xe02f96 nblks = 0x1000000 npages = 0x4 (kgdb) -- Test scenario: misc/sort.sh