KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2017 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 12.0-CURRENT #1 r324234M: Tue Oct 3 10:34:02 UTC 2017 pho@zoo.freebsd.org:/tank/users/pho/mercat1/usr/src/sys/amd64/compile/PHO amd64 FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. VT(vga): resolution 640x480 CPU: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz (3500.07-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306f2 Family=0x6 Model=0x3f Stepping=2 Features=0xbfebfbff Features2=0x7ffefbff AMD Features=0x2c100800 AMD Features2=0x21 Structured Extended Features=0x37ab XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics real memory = 34359738368 (32768 MB) avail memory = 33116585984 (31582 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs FreeBSD/SMP: 1 package(s) x 6 core(s) x 2 hardware threads arc4random: no preloaded entropy cache ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard SMP: AP CPU #1 Launched! SMP: AP CPU #9 Launched! SMP: AP CPU #11 Launched! SMP: AP CPU #6 Launched! SMP: AP CPU #8 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #4 Launched! SMP: AP CPU #10 Launched! SMP: AP CPU #5 Launched! SMP: AP CPU #7 Launched! Timecounter "TSC-low" frequency 1750034218 Hz quality 1000 random: entropy device external interface [ath_hal] loaded module_register_init: MOD_LOAD (vesa, 0xffffffff80f5c440, 0) error 19 random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" kbd1 at kbdmux0 netmap: loaded module nexus0 vtvga0: on motherboard cryptosoft0: on motherboard acpi0: on motherboard acpi0: Power Button (fixed) cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 cpu4: on acpi0 cpu5: on acpi0 cpu6: on acpi0 cpu7: on acpi0 cpu8: on acpi0 cpu9: on acpi0 cpu10: on acpi0 cpu11: on acpi0 atrtc0: port 0x70-0x71,0x74-0x77 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Event timer "HPET4" frequency 14318180 Hz quality 340 Event timer "HPET5" frequency 14318180 Hz quality 340 Event timer "HPET6" frequency 14318180 Hz quality 340 Event timer "HPET7" frequency 14318180 Hz quality 340 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: on acpi0 pci0: on pcib0 pci0: at device 11.1 (no driver attached) pci0: at device 11.2 (no driver attached) pci0: at device 16.1 (no driver attached) pci0: at device 16.6 (no driver attached) pci0: at device 18.1 (no driver attached) acpi_syscontainer0: on acpi0 acpi_syscontainer1: on acpi0 acpi_syscontainer2: on acpi0 acpi_syscontainer3: on acpi0 pcib1: port 0xcf8-0xcff on acpi0 pci1: on pcib1 pcib2: irq 26 at device 1.0 on pci1 pci2: on pcib2 pcib3: irq 32 at device 2.0 on pci1 pci3: on pcib3 pci3: at device 0.0 (no driver attached) pci3: at device 0.1 (no driver attached) pcib4: irq 40 at device 3.0 on pci1 pci4: on pcib4 pci1: at device 17.0 (no driver attached) ahci0: port 0xf110-0xf117,0xf100-0xf103,0xf0f0-0xf0f7,0xf0e0-0xf0e3,0xf020-0xf03f mem 0xfb438000-0xfb4387ff irq 16 at device 17.4 on pci1 ahci0: AHCI v1.30 with 4 6Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahciem0: on ahci0 xhci0: mem 0xfb400000-0xfb40ffff irq 19 at device 20.0 on pci1 xhci0: 32 bytes context size, 64-bit DMA usbus0 on xhci0 usbus0: 5.0Gbps Super Speed USB v3.0 pci1: at device 22.0 (no driver attached) pci1: at device 22.1 (no driver attached) ehci0: mem 0xfb434000-0xfb4343ff irq 18 at device 26.0 on pci1 usbus1: EHCI version 1.0 usbus1 on ehci0 usbus1: 480Mbps High Speed USB v2.0 pcib5: irq 16 at device 28.0 on pci1 pci5: on pcib5 pcib6: irq 18 at device 28.2 on pci1 pci6: on pcib6 pcib7: at device 0.0 on pci6 pci7: on pcib7 vgapci0: port 0xe000-0xe07f mem 0xfa000000-0xfaffffff,0xfb000000-0xfb01ffff irq 18 at device 0.0 on pci7 vgapci0: Boot video device pcib8: irq 16 at device 28.4 on pci1 pci8: on pcib8 igb0: port 0xd020-0xd03f mem 0xfb320000-0xfb33ffff,0xfb344000-0xfb347fff irq 16 at device 0.0 on pci8 igb0: attach_pre capping queues at 8 igb0: using 1024 tx descriptors and 1024 rx descriptors igb0: msix_init qsets capped at 8 igb0: pxm cpus: 6 queue msgs: 9 admincnt: 1 igb0: queue equality override not set, capping rx_queues at 6 and tx_queues at 6 igb0: using 6 rx queues 6 tx queues igb0: Using MSIX interrupts with 7 vectors igb0: allocated for 6 tx_queues igb0: allocated for 6 rx_queues taskqgroup_attach_cpu: setaffinity failed: 3 taskqgroup_attach_cpu: setaffinity failed: 3 taskqgroup_attach_cpu: setaffinity failed: 3 taskqgroup_attach_cpu: setaffinity failed: 3 taskqgroup_attach_cpu: setaffinity failed: 3 taskqgroup_attach_cpu: setaffinity failed: 3 taskqgroup_attach: setaffinity failed: 3 taskqgroup_attach: setaffinity failed: 3 taskqgroup_attach: setaffinity failed: 3 taskqgroup_attach: setaffinity failed: 3 taskqgroup_attach: setaffinity failed: 3 taskquhub0: 21 ports with 21 removable, self powered Root mount waiting for: usbus2 usbus1 usbus0 ugen1.2: at usbus1 uhub3 on uhub1 uhub3: on usbus1 ugen2.2: at usbus2 uhub4 on uhub2 uhub4: on usbus2 ugen0.2: at usbus0 uhub5 on uhub0 uhub5: on usbus0 uhub3: 6 ports with 6 removable, self powered uhub4: 8 ports wugen0.3: at usbus0 ukbd0 on uhub5 ukbd0: on usbus0 kbd2 at ukbd0 Expensive timeout(9) function: 0xffffffff80ad78e0(0) 0.027527100 s NFS ROOT: 192.168.5.1:/zoo/mercat1 random: unblocking device. arc4random: no preloaded entropy cache arc4random: no preloaded entropy cache arc4random: no preloaded entropy cache arc4random: no preloaded entropy cache arc4random: no preloaded entropy cache arc4random: no preloaded entropy cache arc4random: no preloaded entropy cache arc4random: no preloaded entropy cache arc4random: no preloaded entropy cache arc4random: no preloaded entropy cache Link state changed to up igb0: link state changed to UP Interface igb0 IP-Address 192.168.5.171 Broadcast 192.168.5.255 Setting hostuuid: 00000000-0000-0000-0000-002590c99976. Setting hostid: 0x90b83241. Starting ddb. Entropy harvesting:sysctl: unknown oid 'kern.random.sys.harvest.interrupt' interruptssysctl: unknown oid 'kern.random.sys.harvest.ethernet' ethernetsysctl: unknown oid 'kern.random.sys.harvest.point_to_point' point_to_pointsysctl: unknown oid 'kern.random.sys.harvest.swi' swi. Starting file system checks: mount_nfs: can't update /var/db/mounttab for 192.168.5.1:/zoo/mercat1 Mounting local file systems:WARNING: /work was not properly dismounted WARNING: /tmp was not properly dismounted . Writing entropy file:. Setting hostname: mercat1.netperf.freebsd.org. Starting Network: lo0 igb0 igb1. lo0: flags=8049 metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 nd6 options=21 groups: lo igb0: flags=8843 metric 0 mtu 1500 options=e505bb ether 0c:c4:7a:a8:cd:da inet 192.168.5.171 netmask 0xffffff00 broadcast 192.168.5.255 inet6 fe80::ec4:7aff:fea8:cdda%igb0 prefixlen 64 scopeid 0x1 nd6 options=21 media: Ethernet autoselect (1000baseT ) status: active igb1: flags=8802 metric 0 mtu 1500 options=e505bb ether 0c:c4:7a:a8:cd:db nd6 options=29 media: Ethernet autoselect status: no carrier Starting devd. Starting Network: igb1. igb1: flags=8802 metric 0 mtu 1500 options=e505bb ether 0c:c4:7a:a8:cd:db nd6 options=29 media: Ethernet autoselect status: no carrier ums0 on uhub5 ums0: on usbus0 ums0: 3 buttons and [Z] coordinates ID=0 Starting dhclient. Cannot open or create pidfile: Operation not supported can't open and lock /var/db/dhclient.leases.igb0: Operation not supported exiting. /etc/rc.d/dhclient: WARNING: failed to start dhclient devd: cannot open pid file: Operation not supported Starting pflogd: add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Mounting NFS file systems:. ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/gcc48 /usr/local/lib/perl5/5.20/mach/CORE 32-bit compatibility ldconfig path: /usr/lib32 Creating and/or trimming log files. Starting syslogd. syslogd: cannot open pid file: Operation not supported Setting date via ntp. 3 Oct 12:38:45 ntpdate[553]: step time server 192.168.5.1 offset -0.820386 sec Starting rpcbind. NFS access cache time=60 No core dumps found. Starting casperd. Clearing /tmp (X related). Starting statd. Starting lockd. Updating motd:. Mounting late file systems:. Configuring syscons: blanktime. Performing sanity check on sshd configuration. Starting sshd. Starting cron. Local package initialization: (skipping /usr/local/etc/rc.d/fsck.sh, not executable). Tue Oct 3 12:38 FreeBSD/amd64 (mercat1.netperf.freebsd.org) (ttyu1) login: root Oct 3 12:39:04 mercat1 login: ROOT LOGIN (root) ON ttyu1 Last login: Tue Oct 3 07:57:53 on ttyu1 FreeBSD 12.0-CURRENT (PHO) #1 r324234M: Tue Oct 3 10:34:02 UTC 2017 Welcome to FreeBSD! Before seeking technical support, please use the following resources: o Security advisories and updated errata information for all releases are at http://www.FreeBSD.org/releases/ - always consult the ERRATA section for your release first as it's updated frequently. o The Handbook and FAQ documents are at http://www.FreeBSD.org/ and, along with the mailing lists, can be searched by going to http://www.FreeBSD.org/search/. If the doc package has been installed (or fetched via pkg install lang-freebsd-doc, where lang is the 2-letter language code, e.g. en), they are also available formatted in /usr/local/share/doc/freebsd. If you still have a question or problem, please take the output of `uname -a', along with any relevant error messages, and email it as a question to the questions@FreeBSD.org mailing list. If you are unfamiliar with FreeBSD's directory layout, please refer to the hier(7) manual page. If you are not familiar with manual pages, type `man man'. Edit /etc/motd to change this login announcement. root@mercat1:~ # uname -a FreeBSD mercat1.netperf.freebsd.org 12.0-CURRENT FreeBSD 12.0-CURRENT #1 r324234M: Tue Oct 3 10:34:02 UTC 2017 pho@zoo.freebsd.org:/tank/users/pho/mercat1/usr/src/sys/amd64/compile/PHO amd64 root@mercat1:~ # cd /work/stress2/misc root@mercat1:/work/stress2/misc # ./all.sh holdcnt0 holdcnt0.sh* holdcnt02.sh* holdcnt03.sh* root@mercat1:/work/stress2/misc # ./all.sh holdcnt0.sh Note: Can not ping $BLASTHOST: 192.168.1.1 20171003 12:39:34 all: holdcnt0.sh 20171003 12:39:34 all (1/1): holdcnt0.sh lock order reversal: 1st 0xfffff8000df7d7b8 ufs (ufs) @ kern/vfs_lookup.c:669 2nd 0xfffffe07c16c2240 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:280 3rd 0xfffff80012f05c98 ufs (ufs) @ kern/vfs_subr.c:2606 stack backtrace: #0 0xffffffff80ab1c96 at witness_debugger+0x76 #1 0xffffffff80ab1b99 at witness_checkorder+0xea9 #2 0xffffffff80a27d61 at __lockmgr_args+0xa71 #3 0xffffffff80d2a7f2 at ffs_lock+0x92 #4 0xffffffff8105db2a at VOP_LOCK1_APV+0xfa #5 0xffffffff80b2aa8a at _vn_lock+0xba #6 0xffffffff80b18e9a at vget+0xaa #7 0xffffffff80b0aadc at vfs_hash_get+0xcc #8 0xffffffff80d262a1 at ffs_vgetf+0x41 #9 0xffffffff80d1c5e7 at softdep_sync_buf+0xa47 #10 0xffffffff80d2b456 at ffs_syncvnode+0x2b6 #11 0xffffffff80d020c0 at ffs_truncate+0x760 #12 0xffffffff80d3414e at ufs_direnter+0x67e #13 0xffffffff80d3d2d3 at ufs_makeinode+0x5a3 #14 0xffffffff80d39164 at ufs_create+0x34 #15 0xffffffff8105abce at VOP_CREATE_APV+0xfe #16 0xffffffff80b2a291 at vn_open_cred+0x2d1 #17 0xffffffff80b2292f at kern_openat+0x26f lock order reversal: 1st 0xfffffe07c16ea800 bufwait (bufwait) @ kern/vfs_bio.c:3553 2nd 0xfffff800744e9600 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:281 stack backtrace: #0 0xffffffff80ab1c96 at witness_debugger+0x76 #1 0xffffffff80ab1b99 at witness_checkorder+0xea9 #2 0xffffffff80a5868b at _sx_xlock+0x6b #3 0xffffffff80d2f6da at ufsdirhash_add+0x4a #4 0xffffffff80d33fae at ufs_direnter+0x4de #5 0xffffffff80d3d2d3 at ufs_makeinode+0x5a3 #6 0xffffffff80d39164 at ufs_create+0x34 #7 0xffffffff8105abce at VOP_CREATE_APV+0xfe #8 0xffffffff80b2a291 at vn_open_cred+0x2d1 #9 0xffffffff80b2292f at kern_openat+0x26f #10 0xffffffff80edf59b at amd64_syscall+0x2eb #11 0xffffffff80ebb4eb at Xfast_syscall+0xfb panic: missed invl_started cpuid = 8 time = 1507027308 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe085cfe91e0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe085cfe9290 vpanic() at vpanic+0x19f/frame 0xfffffe085cfe9310 kassert_panic() at kassert_panic+0x139/frame 0xfffffe085cfe9380 pmap_delayed_invl_finished() at pmap_delayed_invl_finished+0x55/frame 0xfffffe085cfe93b0 reclaim_pv_chunk_leave_pmap() at reclaim_pv_chunk_leave_pmap+0x70/frame 0xfffffe085cfe93e0 reclaim_pv_chunk() at reclaim_pv_chunk+0x1281/frame 0xfffffe085cfe9520 get_pv_entry() at get_pv_entry+0x326/frame 0xfffffe085cfe9580 pmap_enter() at pmap_enter+0x8f6/frame 0xfffffe085cfe9680 vm_fault_hold() at vm_fault_hold+0x21d/frame 0xfffffe085cfe97e0 vm_fault() at vm_fault+0x77/frame 0xfffffe085cfe9820 trap_pfault() at trap_pfault+0x11a/frame 0xfffffe085cfe98a0 trap() at trap+0x58f/frame 0xfffffe085cfe9ab0 calltrap() at calltrap+0x8/frame 0xfffffe085cfe9ab0 --- trap 0xc, rip = 0x4012c4, rsp = 0x7fffffffe6b0, rbp = 0x7fffffffe820 --- KDB: enter: panic [ thread pid 1060 tid 100187 ] Stopped at kdb_enter+0x3e: movq $0,kdb_why db> run pho db:0:pho> bt Tracing pid 1060 tid 100187 td 0xfffff8001222b000 kdb_enter() at kdb_enter+0x3e/frame 0xfffffe085cfe9290 vpanic() at vpanic+0x1bf/frame 0xfffffe085cfe9310 kassert_panic() at kassert_panic+0x139/frame 0xfffffe085cfe9380 pmap_delayed_invl_finished() at pmap_delayed_invl_finished+0x55/frame 0xfffffe085cfe93b0 reclaim_pv_chunk_leave_pmap() at reclaim_pv_chunk_leave_pmap+0x70/frame 0xfffffe085cfe93e0 reclaim_pv_chunk() at reclaim_pv_chunk+0x1281/frame 0xfffffe085cfe9520 get_pv_entry() at get_pv_entry+0x326/frame 0xfffffe085cfe9580 pmap_enter() at pmap_enter+0x8f6/frame 0xfffffe085cfe9680 vm_fault_hold() at vm_fault_hold+0x21d/frame 0xfffffe085cfe97e0 vm_fault() at vm_fault+0x77/frame 0xfffffe085cfe9820 trap_pfault() at trap_pfault+0x11a/frame 0xfffffe085cfe98a0 trap() at trap+0x58f/frame 0xfffffe085cfe9ab0 calltrap() at calltrap+0x8/frame 0xfffffe085cfe9ab0 --- trap 0xc, rip = 0x4012c4, rsp = 0x7fffffffe6b0, rbp = 0x7fffffffe820 --- db:0:bt> show allpcpu Current CPU: 8 cpuid = 0 dynamic pcpu = 0x9e4f00 curthread = 0xfffff8000772d000: pid 11 tid 100003 "idle: cpu0" curpcb = 0xfffffe07c15b6b80 fpcurthread = none idlethread = 0xfffff8000772d000: tid 100003 "idle: cpu0" curpmap = 0xffffffff81e14690 tssp = 0xffffffff81e3fb90 commontssp = 0xffffffff81e3fb90 rsp0 = 0xfffffe07c15b6b80 gs32p = 0xffffffff81e463e8 ldt = 0xffffffff81e46428 tss = 0xffffffff81e46418 spin locks held: cpuid = 1 dynamic pcpu = 0xfffffe0845ff8f00 curthread = 0xfffff800745ac560: pid 1066 tid 100193 "holdcnt0" curpcb = 0xfffffe085d009b80 fpcurthread = 0xfffff800745ac560: pid 1066 "holdcnt0" idlethread = 0xfffff8000772c560: tid 100004 "idle: cpu1" curpmap = 0xfffff80012607130 tssp = 0xffffffff81e3fbf8 commontssp = 0xffffffff81e3fbf8 rsp0 = 0xfffffe085d009b80 gs32p = 0xffffffff81e46450 ldt = 0xffffffff81e46490 tss = 0xffffffff81e46480 spin locks held: cpuid = 2 dynamic pcpu = 0xfffffe0846000f00 curthread = 0xfffff8001222b560: pid 1059 tid 100186 "holdcnt0" curpcb = 0xfffffe085cfe4b80 fpcurthread = 0xfffff8001222b560: pid 1059 "holdcnt0" idlethread = 0xfffff8000772c000: tid 100005 "idle: cpu2" curpmap = 0xfffff80074106130 tssp = 0xffffffff81e3fc60 commontssp = 0xffffffff81e3fc60 rsp0 = 0xfffffe085cfe4b80 gs32p = 0xffffffff81e464b8 ldt = 0xffffffff81e464f8 tss = 0xffffffff81e464e8 spin locks held: cpuid = 3 dynamic pcpu = 0xfffffe0846008f00 curthread = 0xfffff800745ac000: pid 1067 tid 100194 "holdcnt0" curpcb = 0xfffffe085d00eb80 fpcurthread = 0xfffff800745ac000: pid 1067 "holdcnt0" idlethread = 0xfffff8000772b560: tid 100006 "idle: cpu3" curpmap = 0xfffff8001224e130 tssp = 0xffffffff81e3fcc8 commontssp = 0xffffffff81e3fcc8 rsp0 = 0xfffffe085d00eb80 gs32p = 0xffffffff81e46520 ldt = 0xffffffff81e46560 tss = 0xffffffff81e46550 spin locks held: cpuid = 4 dynamic pcpu = 0xfffffe0846010f00 curthread = 0xfffff800745af000: pid 1061 tid 100188 "holdcnt0" curpcb = 0xfffffe085cfeeb80 fpcurthread = 0xfffff800745af000: pid 1061 "holdcnt0" idlethread = 0xfffff8000772b000: tid 100007 "idle: cpu4" curpmap = 0xfffff80012520130 tssp = 0xffffffff81e3fd30 commontssp = 0xffffffff81e3fd30 rsp0 = 0xfffffe085cfeeb80 gs32p = 0xffffffff81e46588 ldt = 0xffffffff81e465c8 tss = 0xffffffff81e465b8 spin locks held: cpuid = 5 dynamic pcpu = 0xfffffe0846018f00 curthread = 0xfffff800745ae000: pid 1063 tid 100190 "holdcnt0" curpcb = 0xfffffe085cff8b80 fpcurthread = 0xfffff800745ae000: pid 1063 "holdcnt0" idlethread = 0xfffff8000772a560: tid 100008 "idle: cpu5" curpmap = 0xfffff800122dd130 tssp = 0xffffffff81e3fd98 commontssp = 0xffffffff81e3fd98 rsp0 = 0xfffffe085cff8b80 gs32p = 0xffffffff81e465f0 ldt = 0xffffffff81e46630 tss = 0xffffffff81e46620 spin locks held: cpuid = 6 dynamic pcpu = 0xfffffe0846020f00 curthread = 0xfffff800745ad560: pid 1064 tid 100191 "holdcnt0" curpcb = 0xfffffe085cffdb80 fpcurthread = 0xfffff800745ad560: pid 1064 "holdcnt0" idlethread = 0xfffff8000772a000: tid 100009 "idle: cpu6" curpmap = 0xfffff8007430b130 tssp = 0xffffffff81e3fe00 commontssp = 0xffffffff81e3fe00 rsp0 = 0xfffffe085cffdb80 gs32p = 0xffffffff81e46658 ldt = 0xffffffff81e46698 tss = 0xffffffff81e46688 spin locks held: cpuid = 7 dynamic pcpu = 0xfffffe0846028f00 curthread = 0xfffff80012233000: pid 1058 tid 100168 "holdcnt0" curpcb = 0xfffffe085cf8ab80 fpcurthread = 0xfffff80012233000: pid 1058 "holdcnt0" idlethread = 0xfffff80007741560: tid 100010 "idle: cpu7" curpmap = 0xfffff8001269c130 tssp = 0xffffffff81e3fe68 commontssp = 0xffffffff81e3fe68 rsp0 = 0xfffffe085cf8ab80 gs32p = 0xffffffff81e466c0 ldt = 0xffffffff81e46700 tss = 0xffffffff81e466f0 spin locks held: cpuid = 8 dynamic pcpu = 0xfffffe083fbb9f00 curthread = 0xfffff8001222b000: pid 1060 tid 100187 "holdcnt0" curpcb = 0xfffffe085cfe9b80 fpcurthread = 0xfffff8001222b000: pid 1060 "holdcnt0" idlethread = 0xfffff80007741000: tid 100011 "idle: cpu8" curpmap = 0xfffff8000773a130 tssp = 0xffffffff81e3fed0 commontssp = 0xffffffff81e3fed0 rsp0 = 0xfffffe085cfe9b80 gs32p = 0xffffffff81e46728 ldt = 0xffffffff81e46768 tss = 0xffffffff81e46758 spin locks held: cpuid = 9 dynamic pcpu = 0xfffffe083fbc1f00 curthread = 0xfffff800745ae560: pid 1062 tid 100189 "holdcnt0" curpcb = 0xfffffe085cff3b80 fpcurthread = 0xfffff800745ae560: pid 1062 "holdcnt0" idlethread = 0xfffff80007740560: tid 100012 "idle: cpu9" curpmap = 0xfffff80074105130 tssp = 0xffffffff81e3ff38 commontssp = 0xffffffff81e3ff38 rsp0 = 0xfffffe085cff3b80 gs32p = 0xffffffff81e46790 ldt = 0xffffffff81e467d0 tss = 0xffffffff81e467c0 spin locks held: cpuid = 10 dynamic pcpu = 0xfffffe083fbc9f00 curthread = 0xfffff80007740000: pid 11 tid 100013 "idle: cpu10" curpcb = 0xfffffe07c15e8b80 fpcurthread = none idlethread = 0xfffff80007740000: tid 100013 "idle: cpu10" curpmap = 0xffffffff81e14690 tssp = 0xffffffff81e3ffa0 commontssp = 0xffffffff81e3ffa0 rsp0 = 0xfffffe07c15e8b80 gs32p = 0xffffffff81e467f8 ldt = 0xffffffff81e46838 tss = 0xffffffff81e46828 spin locks held: cpuid = 11 dynamic pcpu = 0xfffffe083fbd1f00 curthread = 0xfffff8000773f560: pid 11 tid 100014 "idle: cpu11" curpcb = 0xfffffe07c15edb80 fpcurthread = none idlethread = 0xfffff8000773f560: tid 100014 "idle: cpu11" curpmap = 0xffffffff81e14690 tssp = 0xffffffff81e40008 commontssp = 0xffffffff81e40008 rsp0 = 0xfffffe07c15edb80 gs32p = 0xffffffff81e46860 ldt = 0xffffffff81e468a0 tss = 0xffffffff81e46890 spin locks held: db:0:allpcpu> show alllocks Process 1067 (holdcnt0) thread 0xfffff800745ac000 (100194) exclusive sleep mutex pmap (pmap) r = 0 (0xfffff80012520130) locked @ amd64/amd64/pmap.c:2947 exclusive sleep mutex pmap (pmap) r = 0 (0xfffff8001224e130) locked @ amd64/amd64/pmap.c:4465 shared rw vm object (vm object) r = 0 (0xfffff80012833b40) locked @ vm/vm_fault.c:603 shared sx vm map (user) (vm map (user)) r = 0 (0xfffff8001224e078) locked @ vm/vm_map.c:4015 Process 1066 (holdcnt0) thread 0xfffff800745ac560 (100193) exclusive sleep mutex pmap (pmap) r = 0 (0xfffff80012607130) locked @ amd64/amd64/pmap.c:4844 shared rw vm object (vm object) r = 0 (0xfffff80074527b40) locked @ vm/vm_fault.c:1433 shared sx vm map (user) (vm map (user)) r = 0 (0xfffff80012607078) locked @ vm/vm_map.c:4015 Process 1065 (holdcnt0) thread 0xfffff800745ad000 (100192) shared sx vm map (user) (vm map (user)) r = 0 (0xfffff8001209c078) locked @ vm/vm_map.c:4015 Process 1064 (holdcnt0) thread 0xfffff800745ad560 (100191) exclusive sleep mutex pmap (pmap) r = 0 (0xfffff8007430b130) locked @ amd64/amd64/pmap.c:5857 shared sx vm map (user) (vm map (user)) r = 0 (0xfffff8007430b078) locked @ vm/vm_map.c:4015 Process 1063 (holdcnt0) thread 0xfffff800745ae000 (100190) exclusive sleep mutex pmap (pmap) r = 0 (0xfffff800122dd130) locked @ amd64/amd64/pmap.c:4844 shared rw vm object (vm object) r = 0 (0xfffff80012604870) locked @ vm/vm_fault.c:1433 shared sx vm map (user) (vm map (user)) r = 0 (0xfffff800122dd078) locked @ vm/vm_map.c:4015 Process 1062 (holdcnt0) thread 0xfffff800745ae560 (100189) shared rw vm object (vm object) r = 0 (0xfffff80012e21870) locked @ vm/vm_fault.c:1433 shared sx vm map (user) (vm map (user)) r = 0 (0xfffff80074105078) locked @ vm/vm_map.c:4015 Process 1061 (holdcnt0) thread 0xfffff800745af000 (100188) shared sx vm map (user) (vm map (user)) r = 0 (0xfffff80012520078) locked @ vm/vm_map.c:4015 Process 1060 (holdcnt0) thread 0xfffff8001222b000 (100187) exclusive rw pmap pv list (pmap pv list) r = 0 (0xffffffff81dcef40) locked @ amd64/amd64/pmap.c:3007 exclusive sleep mutex pmap (pmap) r = 0 (0xfffff8000773a130) locked @ amd64/amd64/pmap.c:4465 shared rw vm object (vm object) r = 0 (0xfffff8000f7f3960) locked @ vm/vm_fault.c:603 shared sx vm map (user) (vm map (user)) r = 0 (0xfffff8000773a078) locked @ vm/vm_map.c:4015 Process 1059 (holdcnt0) thread 0xfffff8001222b560 (100186) exclusive lockmgr ufs (ufs) r = 0 (0xfffff80074523068) locked @ kern/vfs_vnops.c:881 Process 1058 (holdcnt0) thread 0xfffff80012233000 (100168) exclusive rw vm object (vm object) r = 0 (0xfffff800745273c0) locked @ kern/vfs_bio.c:3343 exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe07c6516060) locked @ kern/vfs_bio.c:1475 exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe07c2145ba0) locked @ kern/vfs_bio.c:3553 exclusive lockmgr ufs (ufs) r = 0 (0xfffff8007453aa28) locked @ kern/vfs_vnops.c:881 db:0:alllocks> show lockedvnods Locked vnodes vnode 0xfffff80074523000: tag ufs, type VREG usecount 1908, writecount 2, refcount 2056 mountedhere 0 flags (VI_ACTIVE) v_object 0xfffff80074527780 ref 1907 pages 1376 cleanbuf 145 dirtybuf 2 lock type ufs: EXCL by thread 0xfffff8001222b560 (pid 1059, holdcnt0, tid 100186) #0 0xffffffff80a28666 at __lockmgr_args+0x1376 #1 0xffffffff80d2a7f2 at ffs_lock+0x92 #2 0xffffffff8105db2a at VOP_LOCK1_APV+0xfa #3 0xffffffff80b2aa8a at _vn_lock+0xba #4 0xffffffff80b2d4f2 at vn_write+0x112 #5 0xffffffff80b2d772 at vn_io_fault_doio+0x22 #6 0xffffffff80b2b16b at vn_io_fault1+0x1bb #7 0xffffffff80b29388 at vn_io_fault+0x188 #8 0xffffffff80ab709b at dofilewrite+0xab #9 0xffffffff80ab6c88 at kern_writev+0x68 #10 0xffffffff80ab6c0b at sys_write+0x8b #11 0xffffffff80edf59b at amd64_syscall+0x2eb #12 0xffffffff80ebb4eb at Xfast_syscall+0xfb ino 178, on dev md10a vnode 0xfffff8007453a9c0: tag ufs, type VREG usecount 1908, writecount 2, refcount 2054 mountedhere 0 flags (VI_ACTIVE) v_object 0xfffff800745273c0 ref 1907 pages 1376 cleanbuf 143 dirtybuf 2 lock type ufs: EXCL by thread 0xfffff80012233000 (pid 1058, holdcnt0, tid 100168) #0 0xffffffff80a28666 at __lockmgr_args+0x1376 #1 0xffffffff80d2a7f2 at ffs_lock+0x92 #2 0xffffffff8105db2a at VOP_LOCK1_APV+0xfa #3 0xffffffff80b2aa8a at _vn_lock+0xba #4 0xffffffff80b2d4f2 at vn_write+0x112 #5 0xffffffff80b2d772 at vn_io_fault_doio+0x22 #6 0xffffffff80b2b16b at vn_io_fault1+0x1bb #7 0xffffffff80b29388 at vn_io_fault+0x188 #8 0xffffffff80ab709b at dofilewrite+0xab #9 0xffffffff80ab6c88 at kern_writev+0x68 #10 0xffffffff80ab6c0b at sys_write+0x8b #11 0xffffffff80edf59b at amd64_syscall+0x2eb #12 0xffffffff80ebb4eb at Xfast_syscall+0xfb ino 182, on dev md10a db:0:lockedvnods> show allchains chain 1: thread 100142 (pid 1078, swap) sleeping on 0xffffffff818885ec "pfault" chain 2: thread 100195 (pid 1077, swap) sleeping on 0xffffffff81c26f8a "nanslp" chain 3: thread 100196 (pid 1076, swap) sleeping on 0xfffff80012261000 "wait" chain 4: thread 100192 (pid 1065, holdcnt0) sleeping on 0xffffffff818885ec "vmwait" chain 5: thread 100178 (pid 1056, holdcnt0) sleeping on 0xfffff80074103000 "wait" chain 6: thread 100185 (pid 1052, md10) sleeping on 0xfffff8007446d000 "mdwait" chain 7: thread 100184 (pid 1030, tee) sleeping on 0xfffff8001225f2f8 "piperd" chain 8: thread 100152 (pid 1029, sh) sleeping on 0xfffff80012125a80 "wait" chain 9: thread 100129 (pid 1028, sh) sleeping on 0xfffff80012262540 "wait" chain 10: thread 100132 (pid 745, sh) sleeping on 0xfffff80012536540 "wait" chain 11: thread 100166 (pid 741, csh) sleeping on 0xfffff80074104b28 "pause" chain 12: thread 100125 (pid 740, login) sleeping on 0xfffff800124a6540 "wait" chain 13: thread 100141 (pid 739, getty) sleeping on 0xfffff800081594b0 "ttyin" chain 14: thread 100123 (pid 738, getty) sleeping on 0xfffff8000f2a30b0 "ttyin" chain 15: thread 100165 (pid 737, getty) sleeping on 0xfffff8000f2a34b0 "ttyin" chain 16: thread 100164 (pid 736, getty) sleeping on 0xfffff8000f2a38b0 "ttyin" chain 17: thread 100163 (pid 735, getty) sleeping on 0xfffff8000f2a3cb0 "ttyin" chain 18: thread 100162 (pid 734, getty) sleeping on 0xfffff8000f2a40b0 "ttyin" chain 19: thread 100151 (pid 733, getty) sleeping on 0xfffff8000f2a44b0 "ttyin" chain 20: thread 100156 (pid 732, getty) sleeping on 0xfffff8000f2a48b0 "ttyin" chain 21: thread 100111 (pid 731, getty) sleeping on 0xfffff8000f2a4cb0 "ttyin" chain 22: thread 100146 (pid 695, cron) sleeping on 0xffffffff81c26f80 "nanslp" chain 23: thread 100134 (pid 690, sshd) sleeping on 0xfffff8001236a440 "select" chain 24: thread 100147 (pid 617, NLM: master) sleeping on 0xfffff8000f1ca360 "rpcsvc" chain 25: thread 100140 (pid 613, rpc.statd) sleeping on 0xfffff800123423c0 "select" chain 26: thread 100128 (pid 588, casperd) sleeping on 0xfffff8001264e340 "select" chain 27: thread 100118 (pid 587, casperd) sleeping on 0xfffff8000f8432c0 "select" chain 28: thread 100136 (pid 556, rpcbind) sleeping on 0xfffff8000f8bc340 "select" chain 29: thread 100139 (pid 551, syslogd) sleeping on 0xfffff8001236a4c0 "select" chain 30: thread 100119 (pid 464, devd) sleeping on 0xfffff8000f2fe1c0 "select" chain 31: thread 100114 (pid 33, newnfs 0) sleeping on 0xffffffff81e09990 "-" chain 32: thread 100110 (pid 23, vnlru) sleeping on 0xfffff8000f082540 "vlruwt" chain 33: thread 100108 (pid 22, syncer) sleeping on 0xffffffff81dbb400 "syncer" chain 34: thread 100107 (pid 21, bufspacedaemon) sleeping on 0xffffffff81dba938 "-" chain 35: thread 100105 (pid 20, bufdaemon) sleeping on 0xffffffff81dba748 "psleep" chain 36: thread 100126 (pid 20, /work worker) sleeping on 0xfffff8001249e4e8 "sdflush" chain 37: thread 100127 (pid 20, /tmp worker) sleeping on 0xfffff8000f3236e8 "sdflush" chain 38: thread 100104 (pid 19, vmdaemon) sleeping on 0xffffffff81dc4a78 "psleep" chain 39: thread 100103 (pid 18, pagedaemon) sleeping on 0xffffffff81c25d12 "psleep" chain 40: thread 100106 (pid 18, laundry: dom0) sleeping on 0xffffffff81dc431c "wswbuf0" chain 41: thread 100109 (pid 18, uma) sleeping on 0xffffffff81dc45a8 "umarcl" chain 42: thread 100102 (pid 17, enc_daemon1) sleeping on 0xfffff8000df9d000 "idle" chain 43: thread 100101 (pid 16, enc_daemon0) sleeping on 0xfffff8000df9d540 "idle" chain 44: thread 100098 (pid 15, rand_harvestq) sleeping on 0xffffffff81af9fd8 "-" chain 45: thread 100094 (pid 9, sctp_iterator) sleeping on 0xffffffff81e37750 "waiting_for_work" chain 46: thread 100093 (pid 8, soaiod4) sleeping on 0xffffffff81dba134 "-" chain 47: thread 100092 (pid 7, soaiod3) sleeping on 0xffffffff81dba134 "-" chain 48: thread 100091 (pid 6, soaiod2) sleeping on 0xffffffff81dba134 "-" chain 49: thread 100090 (pid 5, soaiod1) sleeping on 0xffffffff81dba134 "-" chain 50: thread 100071 (pid 14, usbus0) sleeping on 0xfffffe00012f1460 "-" chain 51: thread 100072 (pid 14, usbus0) sleeping on 0xfffffe00012f14b8 "-" chain 52: thread 100073 (pid 14, usbus0) sleeping on 0xfffffe00012f1510 "-" chain 53: thread 100074 (pid 14, usbus0) sleeping on 0xfffffe00012f1568 "-" chain 54: thread 100075 (pid 14, usbus0) sleeping on 0xfffffe00012f15c0 "-" chain 55: thread 100077 (pid 14, usbus1) sleeping on 0xfffffe000114dd10 "-" chain 56: thread 100078 (pid 14, usbus1) sleeping on 0xfffffe000114dd68 "-" chain 57: thread 100079 (pid 14, usbus1) sleeping on 0xfffffe000114ddc0 "-" chain 58: thread 100080 (pid 14, usbus1) sleeping on 0xfffffe000114de18 "-" chain 59: thread 100081 (pid 14, usbus1) sleeping on 0xfffffe000114de70 "-" chain 60: thread 100082 (pid 14, usbus2) sleeping on 0xfffffe00012f9d10 "-" chain 61: thread 100083 (pid 14, usbus2) sleeping on 0xfffffe00012f9d68 "-" chain 62: thread 100084 (pid 14, usbus2) sleeping on 0xfffffe00012f9dc0 "-" chain 63: thread 100085 (pid 14, usbus2) sleeping on 0xfffffe00012f9e18 "-" chain 64: thread 100086 (pid 14, usbus2) sleeping on 0xfffffe00012f9e70 "-" chain 65: thread 100066 (pid 4, doneq0) sleeping on 0xffffffff819d5340 "-" chain 66: thread 100067 (pid 4, doneq1) sleeping on 0xffffffff819d53c0 "-" chain 67: thread 100068 (pid 4, doneq2) sleeping on 0xffffffff819d5440 "-" chain 68: thread 100100 (pid 4, scanner) sleeping on 0xffffffff819d5208 "-" chain 69: thread 100065 (pid 3, crypto returns) sleeping on 0xffffffff81dc2b00 "crypto_ret_wait" chain 70: thread 100064 (pid 2, crypto) sleeping on 0xffffffff81dc29b8 "crypto_wait" chain 71: thread 100060 (pid 13, g_event) sleeping on 0xffffffff81e13a40 "-" chain 72: thread 100061 (pid 13, g_up) sleeping on 0xffffffff81e13a48 "-" chain 73: thread 100062 (pid 13, g_down) sleeping on 0xffffffff81e13a50 "-" chain 74: thread 100002 (pid 1, init) sleeping on 0xfffff80007729540 "wait" chain 75: thread 100001 (pid 10, audit) sleeping on 0xffffffff81e3ce28 "audit_worker_cv" chain 76: thread 100000 (pid 0, swapper) sleeping on 0xffffffff81e13a88 "swapin" chain 77: thread 100015 (pid 0, thread taskq) sleeping on 0xfffff8000776f000 "-" chain 78: thread 100017 (pid 0, aiod_kick taskq) sleeping on 0xfffff8000776ed00 "-" chain 79: thread 100018 (pid 0, kqueue_ctx taskq) sleeping on 0xfffff8000776ec00 "-" chain 80: thread 100021 (pid 0, if_config_tqg_0) sleeping on 0xfffff8000776e900 "-" chain 81: thread 100022 (pid 0, softirq_0) sleeping on 0xfffff8000776e800 "-" chain 82: thread 100023 (pid 0, softirq_1) sleeping on 0xfffff8000776e700 "-" chain 83: thread 100024 (pid 0, softirq_2) sleeping on 0xfffff8000776e600 "-" chain 84: thread 100025 (pid 0, softirq_3) sleeping on 0xfffff8000776e500 "-" chain 85: thread 100026 (pid 0, softirq_4) sleeping on 0xfffff8000776e400 "-" chain 86: thread 100027 (pid 0, softirq_5) sleeping on 0xfffff8000776e300 "-" chain 87: thread 100028 (pid 0, softirq_6) sleeping on 0xfffff8000776e200 "-" chain 88: thread 100029 (pid 0, softirq_7) sleeping on 0xfffff8000776e100 "-" chain 89: thread 100030 (pid 0, softirq_8) sleeping on 0xfffff8000776e000 "-" chain 90: thread 100031 (pid 0, softirq_9) sleeping on 0xfffff8000776de00 "-" chain 91: thread 100032 (pid 0, softirq_10) sleeping on 0xfffff8000776dd00 "-" chain 92: thread 100033 (pid 0, softirq_11) sleeping on 0xfffff8000776dc00 "-" chain 93: thread 100034 (pid 0, if_io_tqg_0) sleeping on 0xfffff8000776db00 "-" chain 94: thread 100035 (pid 0, if_io_tqg_1) sleeping on 0xfffff8000776da00 "-" chain 95: thread 100036 (pid 0, if_io_tqg_2) sleeping on 0xfffff800077fc100 "-" chain 96: thread 100037 (pid 0, if_io_tqg_3) sleeping on 0xfffff800077fc000 "-" chain 97: thread 100038 (pid 0, if_io_tqg_4) sleeping on 0xfffff800077fae00 "-" chain 98: thread 100039 (pid 0, if_io_tqg_5) sleeping on 0xfffff800077fad00 "-" chain 99: thread 100040 (pid 0, if_io_tqg_6) sleeping on 0xfffff800077fac00 "-" chain 100: thread 100041 (pid 0, if_io_tqg_7) sleeping on 0xfffff800077fab00 "-" chain 101: thread 100042 (pid 0, if_io_tqg_8) sleeping on 0xfffff800077faa00 "-" chain 102: thread 100043 (pid 0, if_io_tqg_9) sleeping on 0xfffff800077fa900 "-" chain 103: thread 100044 (pid 0, if_io_tqg_10) sleeping on 0xfffff800077fa800 "-" chain 104: thread 100045 (pid 0, if_io_tqg_11) sleeping on 0xfffff800077fa700 "-" chain 105: thread 100063 (pid 0, firmware taskq) sleeping on 0xfffff8000776d900 "-" chain 106: thread 100089 (pid 0, mca taskq) sleeping on 0xfffff8000d021200 "-" chain 107: thread 100095 (pid 0, acpi_task_0) sleeping on 0xfffff8000f05c100 "-" chain 108: thread 100096 (pid 0, acpi_task_1) sleeping on 0xfffff8000f05c100 "-" chain 109: thread 100097 (pid 0, acpi_task_2) sleeping on 0xfffff8000f05c100 "-" chain 110: thread 100099 (pid 0, CAM taskq) sleeping on 0xfffff8000776d800 "-" db:0:allchains> show mount 0xfffff8000f7fe000 192.168.5.1:/zoo/mercat1 on / (nfs) 0xfffff8000f7ff000 devfs on /dev (devfs) 0xfffff8001238c000 /dev/gpt/work on /work (ufs) 0xfffff8001237f000 /dev/gpt/tmp on /tmp (ufs) 0xfffff80012b8b000 192.168.5.1:/zoo/pho on /zoo/pho (nfs) 0xfffff8001237e000 192.168.5.1:/tank/users/pho on /tank/users/pho (nfs) 0xfffff8000f7fd000 /dev/md10a on /mnt (ufs) More info: show mount db:0:mount> show page vm_cnt.v_free_count: 10629 vm_cnt.v_inactive_count: 36272 vm_cnt.v_active_count: 6386838 vm_cnt.v_laundry_count: 1116501 vm_cnt.v_wire_count: 580711 vm_cnt.v_free_reserved: 10653 vm_cnt.v_free_min: 51306 vm_cnt.v_free_target: 173265 vm_cnt.v_inactive_target: 259897 db:0:page> ps pid ppid pgrp uid state wmesg wchan cmd 1078 1077 745 0 D+ pfault 0xffffffff818885ec swap 1077 1076 745 0 S+ nanslp 0xffffffff81c26f8a swap 1076 1029 745 0 S+ wait 0xfffff80012261000 swap 1067 1056 745 0 R+ CPU 3 holdcnt0 1066 1056 745 0 R+ CPU 1 holdcnt0 1065 1056 745 0 D+ vmwait 0xffffffff818885ec holdcnt0 1064 1056 745 0 R+ CPU 6 holdcnt0 1063 1056 745 0 R+ CPU 5 holdcnt0 1062 1056 745 0 R+ CPU 9 holdcnt0 1061 1056 745 0 R+ CPU 4 holdcnt0 1060 1056 745 0 R+ CPU 8 holdcnt0 1059 1056 745 0 R+ CPU 2 holdcnt0 1058 1056 745 0 R+ CPU 7 holdcnt0 1056 1029 745 0 SW+ wait 0xfffff80074103000 holdcnt0 1052 0 0 0 DL mdwait 0xfffff8007446d000 [md10] 1030 745 745 0 S+ piperd 0xfffff8001225f2f8 tee 1029 1028 745 0 SW+ wait 0xfffff80012125a80 sh 1028 745 745 0 SW+ wait 0xfffff80012262540 sh 745 741 745 0 SW+ wait 0xfffff80012536540 sh 741 740 741 0 SW+ pause 0xfffff80074104b28 csh 740 1 740 0 SWs+ wait 0xfffff800124a6540 login 739 1 739 0 Ss+ ttyin 0xfffff800081594b0 getty 738 1 738 0 Ss+ ttyin 0xfffff8000f2a30b0 getty 737 1 737 0 Ss+ ttyin 0xfffff8000f2a34b0 getty 736 1 736 0 Ss+ ttyin 0xfffff8000f2a38b0 getty 735 1 735 0 Ss+ ttyin 0xfffff8000f2a3cb0 getty 734 1 734 0 Ss+ ttyin 0xfffff8000f2a40b0 getty 733 1 733 0 Ss+ ttyin 0xfffff8000f2a44b0 getty 732 1 732 0 Ss+ ttyin 0xfffff8000f2a48b0 getty 731 1 731 0 Ss+ ttyin 0xfffff8000f2a4cb0 getty 695 1 695 0 SWs nanslp 0xffffffff81c26f80 cron 690 1 690 0 Ss select 0xfffff8001236a440 sshd 617 1 617 0 Ss rpcsvc 0xfffff8000f1ca360 NLM: master 613 1 613 0 Ss select 0xfffff800123423c0 rpc.statd 588 1 588 0 Ss select 0xfffff8001264e340 casperd 587 1 587 0 Ss select 0xfffff8000f8432c0 casperd 556 1 556 0 Ss select 0xfffff8000f8bc340 rpcbind 551 1 551 0 Ss select 0xfffff8001236a4c0 syslogd 464 1 464 0 Ss select 0xfffff8000f2fe1c0 devd 33 0 0 0 SL - 0xffffffff81e09990 [newnfs 0] 23 0 0 0 DL vlruwt 0xfffff8000f082540 [vnlru] 22 0 0 0 DL syncer 0xffffffff81dbb400 [syncer] 21 0 0 0 DL - 0xffffffff81dba938 [bufspacedaemon] 20 0 0 0 DL (threaded) [bufdaemon] 100105 D psleep 0xffffffff81dba748 [bufdaemon] 100126 D sdflush 0xfffff8001249e4e8 [/work worker] 100127 D sdflush 0xfffff8000f3236e8 [/tmp worker] 19 0 0 0 DL psleep 0xffffffff81dc4a78 [vmdaemon] 18 0 0 0 DL (threaded) [pagedaemon] 100103 D psleep 0xffffffff81c25d12 [pagedaemon] 100106 D wswbuf0 0xffffffff81dc431c [laundry: dom0] 100109 D umarcl 0xffffffff81dc45a8 [uma] 17 0 0 0 DL idle 0xfffff8000df9d000 [enc_daemon1] 16 0 0 0 DL idle 0xfffff8000df9d540 [enc_daemon0] 15 0 0 0 DL - 0xffffffff81af9fd8 [rand_harvestq] 9 0 0 0 DL waiting 0xffffffff81e37750 [sctp_iterator] 8 0 0 0 DL - 0xffffffff81dba134 [soaiod4] 7 0 0 0 DL - 0xffffffff81dba134 [soaiod3] 6 0 0 0 DL - 0xffffffff81dba134 [soaiod2] 5 0 0 0 DL - 0xffffffff81dba134 [soaiod1] 14 0 0 0 DL (threaded) [usb] 100071 D - 0xfffffe00012f1460 [usbus0] 100072 D - 0xfffffe00012f14b8 [usbus0] 100073 D - 0xfffffe00012f1510 [usbus0] 100074 D - 0xfffffe00012f1568 [usbus0] 100075 D - 0xfffffe00012f15c0 [usbus0] 100077 D - 0xfffffe000114dd10 [usbus1] 100078 D - 0xfffffe000114dd68 [usbus1] 100079 D - 0xfffffe000114ddc0 [usbus1] 100080 D - 0xfffffe000114de18 [usbus1] 100081 D - 0xfffffe000114de70 [usbus1] 100082 D - 0xfffffe00012f9d10 [usbus2] 100083 D - 0xfffffe00012f9d68 [usbus2] 100084 D - 0xfffffe00012f9dc0 [usbus2] 100085 D - 0xfffffe00012f9e18 [usbus2] 100086 D - 0xfffffe00012f9e70 [usbus2] 4 0 0 0 DL (threaded) [cam] 100066 D - 0xffffffff819d5340 [doneq0] 100067 D - 0xffffffff819d53c0 [doneq1] 100068 D - 0xffffffff819d5440 [doneq2] 100100 D - 0xffffffff819d5208 [scanner] 3 0 0 0 DL crypto_ 0xffffffff81dc2b00 [crypto returns] 2 0 0 0 DL crypto_ 0xffffffff81dc29b8 [crypto] 13 0 0 0 DL (threaded) [geom] 100060 D - 0xffffffff81e13a40 [g_event] 100061 D - 0xffffffff81e13a48 [g_up] 100062 D - 0xffffffff81e13a50 [g_down] 12 0 0 0 WL (threaded) [intr] 100016 I [swi5: fast taskq] 100019 I [swi6: task queue] 100020 I [swi6: Giant taskq] 100046 I [swi4: clock (0)] 100047 I [swi4: clock (1)] 100048 I [swi4: clock (2)] 100049 I [swi4: clock (3)] 100050 I [swi4: clock (4)] 100051 I [swi4: clock (5)] 100052 I [swi4: clock (6)] 100053 I [swi4: clock (7)] 100054 I [swi4: clock (8)] 100055 I [swi4: clock (9)] 100056 I [swi4: clock (10)] 100057 I [swi4: clock (11)] 100058 I [swi1: netisr 0] 100059 I [swi3: vm] 100069 I [irq264: ahci0] 100070 I [irq19: xhci0] 100076 I [irq18: ehci0 ehci1] 100087 I [irq279: ahci1] 100088 I [swi0: uart uart] 11 0 0 0 RL (threaded) [idle] 100003 Run CPU 0 [idle: cpu0] 100004 CanRun [idle: cpu1] 100005 CanRun [idle: cpu2] 100006 CanRun [idle: cpu3] 100007 CanRun [idle: cpu4] 100008 CanRun [idle: cpu5] 100009 CanRun [idle: cpu6] 100010 CanRun [idle: cpu7] 100011 CanRun [idle: cpu8] 100012 CanRun [idle: cpu9] 100013 Run CPU 10 [idle: cpu10] 100014 Run CPU 11 [idle: cpu11] 1 0 1 0 SLs wait 0xfffff80007729540 [init] 10 0 0 0 DL audit_w 0xffffffff81e3ce28 [audit] 0 0 0 0 DLs (threaded) [kernel] 100000 D swapin 0xffffffff81e13a88 [swapper] 100015 D - 0xfffff8000776f000 [thread taskq] 100017 D - 0xfffff8000776ed00 [aiod_kick taskq] 100018 D - 0xfffff8000776ec00 [kqueue_ctx taskq] 100021 D - 0xfffff8000776e900 [if_config_tqg_0] 100022 D - 0xfffff8000776e800 [softirq_0] 100023 D - 0xfffff8000776e700 [softirq_1] 100024 D - 0xfffff8000776e600 [softirq_2] 100025 D - 0xfffff8000776e500 [softirq_3] 100026 D - 0xfffff8000776e400 [softirq_4] 100027 D - 0xfffff8000776e300 [softirq_5] 100028 D - 0xfffff8000776e200 [softirq_6] 100029 D - 0xfffff8000776e100 [softirq_7] 100030 D - 0xfffff8000776e000 [softirq_8] 100031 D - 0xfffff8000776de00 [softirq_9] 100032 D - 0xfffff8000776dd00 [softirq_10] 100033 D - 0xfffff8000776dc00 [softirq_11] 100034 D - 0xfffff8000776db00 [if_io_tqg_0] 100035 D - 0xfffff8000776da00 [if_io_tqg_1] 100036 D - 0xfffff800077fc100 [if_io_tqg_2] 100037 D - 0xfffff800077fc000 [if_io_tqg_3] 100038 D - 0xfffff800077fae00 [if_io_tqg_4] 100039 D - 0xfffff800077fad00 [if_io_tqg_5] 100040 D - 0xfffff800077fac00 [if_io_tqg_6] 100041 D - 0xfffff800077fab00 [if_io_tqg_7] 100042 D - 0xfffff800077faa00 [if_io_tqg_8] 100043 D - 0xfffff800077fa900 [if_io_tqg_9] 100044 D - 0xfffff800077fa800 [if_io_tqg_10] 100045 D - 0xfffff800077fa700 [if_io_tqg_11] 100063 D - 0xfffff8000776d900 [firmware taskq] 100089 D - 0xfffff8000d021200 [mca taskq] 100095 D - 0xfffff8000f05c100 [acpi_task_0] 100096 D - 0xfffff8000f05c100 [acpi_task_1] 100097 D - 0xfffff8000f05c100 [acpi_task_2] 100099 D - 0xfffff8000776d800 [CAM taskq] db:0:ps> allt Tracing command swap pid 1078 tid 100142 td 0xfffff8000f7ec000 sched_switch() at sched_switch+0x486/frame 0xfffffe085cf08510 mi_switch() at mi_switch+0x195/frame 0xfffffe085cf08550 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cf08590 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085cf085c0 _sleep() at _sleep+0x30b/frame 0xfffffe085cf08660 vm_waitpfault() at vm_waitpfault+0x74/frame 0xfffffe085cf08680 vm_fault_hold() at vm_fault_hold+0x1213/frame 0xfffffe085cf087e0 vm_fault() at vm_fault+0x77/frame 0xfffffe085cf08820 trap_pfault() at trap_pfault+0x11a/frame 0xfffffe085cf088a0 trap() at trap+0x58f/frame 0xfffffe085cf08ab0 calltrap() at calltrap+0x8/frame 0xfffffe085cf08ab0 --- trap 0xc, rip = 0x800bc5a14, rsp = 0x7fffffffe358, rbp = 0x7fffffffe3a0 --- Tracing command swap pid 1077 tid 100195 td 0xfffff8000f7ea000 sched_switch() at sched_switch+0x486/frame 0xfffffe085d0136d0 mi_switch() at mi_switch+0x195/frame 0xfffffe085d013710 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085d013750 sleepq_catch_signals() at sleepq_catch_signals+0x466/frame 0xfffffe085d0137c0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe085d0137f0 _sleep() at _sleep+0x2e0/frame 0xfffffe085d013890 kern_clock_nanosleep() at kern_clock_nanosleep+0x1af/frame 0xfffffe085d013920 user_clock_nanosleep() at user_clock_nanosleep+0x72/frame 0xfffffe085d013980 amd64_syscall() at amd64_syscall+0x2eb/frame 0xfffffe085d013ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe085d013ab0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x800b53aca, rsp = 0x7fffffffe0b8, rbp = 0x7fffffffe100 --- Tracing command swap pid 1076 tid 100196 td 0xfffff8000f7de560 sched_switch() at sched_switch+0x486/frame 0xfffffe085d018520 mi_switch() at mi_switch+0x195/frame 0xfffffe085d018560 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085d0185a0 sleepq_catch_signals() at sleepq_catch_signals+0x466/frame 0xfffffe085d018610 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe085d018640 _sleep() at _sleep+0x301/frame 0xfffffe085d0186e0 kern_wait6() at kern_wait6+0x467/frame 0xfffffe085d018790 sys_wait4() at sys_wait4+0x7a/frame 0xfffffe085d018980 amd64_syscall() at amd64_syscall+0x2eb/frame 0xfffffe085d018ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe085d018ab0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x800b5394a, rsp = 0x7fffffffe6d8, rbp = 0x7fffffffe7f0 --- Tracing command holdcnt0 pid 1067 tid 100194 td 0xfffff800745ac000 (CPU 3) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07c79dbcf0 ipi_nmi_handler() at ipi_nmi_handler+0x4a/frame 0xfffffe07c79dbd10 trap() at trap+0x49/frame 0xfffffe07c79dbf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07c79dbf20 --- trap 0x13, rip = 0xffffffff80a93d32, rsp = 0xfffffe085d00e2d0, rbp = 0xfffffe085d00e2e0 --- lock_delay() at lock_delay+0x42/frame 0xfffffe085d00e2e0 __rw_wlock_hard() at __rw_wlock_hard+0x128/frame 0xfffffe085d00e3a0 _rw_wlock_cookie() at _rw_wlock_cookie+0xbe/frame 0xfffffe085d00e3e0 reclaim_pv_chunk() at reclaim_pv_chunk+0x94d/frame 0xfffffe085d00e520 get_pv_entry() at get_pv_entry+0x326/frame 0xfffffe085d00e580 pmap_enter() at pmap_enter+0x8f6/frame 0xfffffe085d00e680 vm_fault_hold() at vm_fault_hold+0x21d/frame 0xfffffe085d00e7e0 vm_fault() at vm_fault+0x77/frame 0xfffffe085d00e820 trap_pfault() at trap_pfault+0x11a/frame 0xfffffe085d00e8a0 trap() at trap+0x58f/frame 0xfffffe085d00eab0 calltrap() at calltrap+0x8/frame 0xfffffe085d00eab0 --- trap 0xc, rip = 0x4012c4, rsp = 0x7fffffffe6b0, rbp = 0x7fffffffe820 --- Tracing command holdcnt0 pid 1066 tid 100193 td 0xfffff800745ac560 (CPU 1) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07c79cbcf0 ipi_nmi_handler() at ipi_nmi_handler+0x4a/frame 0xfffffe07c79cbd10 trap() at trap+0x49/frame 0xfffffe07c79cbf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07c79cbf20 --- trap 0x13, rip = 0xffffffff80ab0e80, rsp = 0xfffffe085d009420, rbp = 0xfffffe085d0094a0 --- witness_checkorder() at witness_checkorder+0x190/frame 0xfffffe085d0094a0 _rw_wlock_cookie() at _rw_wlock_cookie+0x6c/frame 0xfffffe085d0094e0 pmap_try_insert_pv_entry() at pmap_try_insert_pv_entry+0xfa/frame 0xfffffe085d009530 pmap_enter_quick_locked() at pmap_enter_quick_locked+0x32b/frame 0xfffffe085d0095c0 pmap_enter_quick() at pmap_enter_quick+0x6e/frame 0xfffffe085d009610 vm_fault_prefault() at vm_fault_prefault+0x204/frame 0xfffffe085d009680 vm_fault_hold() at vm_fault_hold+0x1c0e/frame 0xfffffe085d0097e0 vm_fault() at vm_fault+0x77/frame 0xfffffe085d009820 trap_pfault() at trap_pfault+0x11a/frame 0xfffffe085d0098a0 trap() at trap+0x58f/frame 0xfffffe085d009ab0 calltrap() at calltrap+0x8/frame 0xfffffe085d009ab0 --- trap 0xc, rip = 0x4012c4, rsp = 0x7fffffffe6b0, rbp = 0x7fffffffe820 --- Tracing command holdcnt0 pid 1065 tid 100192 td 0xfffff800745ad000 sched_switch() at sched_switch+0x486/frame 0xfffffe085d004310 mi_switch() at mi_switch+0x195/frame 0xfffffe085d004350 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085d004390 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085d0043c0 _sleep() at _sleep+0x30b/frame 0xfffffe085d004460 vm_wait() at vm_wait+0xc5/frame 0xfffffe085d004480 _pmap_allocpte() at _pmap_allocpte+0x156/frame 0xfffffe085d004580 pmap_enter() at pmap_enter+0x69d/frame 0xfffffe085d004680 vm_fault_hold() at vm_fault_hold+0x1d11/frame 0xfffffe085d0047e0 vm_fault() at vm_fault+0x77/frame 0xfffffe085d004820 trap_pfault() at trap_pfault+0x11a/frame 0xfffffe085d0048a0 trap() at trap+0x58f/frame 0xfffffe085d004ab0 calltrap() at calltrap+0x8/frame 0xfffffe085d004ab0 --- trap 0xc, rip = 0x4012c4, rsp = 0x7fffffffe6b0, rbp = 0x7fffffffe820 --- Tracing command holdcnt0 pid 1064 tid 100191 td 0xfffff800745ad560 (CPU 6) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07c79f3cf0 ipi_nmi_handler() at ipi_nmi_handler+0x4a/frame 0xfffffe07c79f3d10 trap() at trap+0x49/frame 0xfffffe07c79f3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07c79f3f20 --- trap 0x13, rip = 0xffffffff80ec8c44, rsp = 0xfffffe085cffd5d0, rbp = 0xfffffe085cffd5d0 --- pmap_pde() at pmap_pde+0x4/frame 0xfffffe085cffd5d0 pmap_is_prefaultable() at pmap_is_prefaultable+0x51/frame 0xfffffe085cffd610 vm_fault_prefault() at vm_fault_prefault+0xf2/frame 0xfffffe085cffd680 vm_fault_hold() at vm_fault_hold+0x1c0e/frame 0xfffffe085cffd7e0 vm_fault() at vm_fault+0x77/frame 0xfffffe085cffd820 trap_pfault() at trap_pfault+0x11a/frame 0xfffffe085cffd8a0 trap() at trap+0x58f/frame 0xfffffe085cffdab0 calltrap() at calltrap+0x8/frame 0xfffffe085cffdab0 --- trap 0xc, rip = 0x4012c4, rsp = 0x7fffffffe6b0, rbp = 0x7fffffffe820 --- Tracing command holdcnt0 pid 1063 tid 100190 td 0xfffff800745ae000 (CPU 5) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07c79ebcf0 ipi_nmi_handler() at ipi_nmi_handler+0x4a/frame 0xfffffe07c79ebd10 trap() at trap+0x49/frame 0xfffffe07c79ebf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07c79ebf20 --- trap 0x13, rip = 0xffffffff80a4bdea, rsp = 0xfffffe085cff85a0, rbp = 0xfffffe085cff85c0 --- _rw_wunlock_cookie() at _rw_wunlock_cookie+0x8a/frame 0xfffffe085cff85c0 pmap_enter_quick() at pmap_enter_quick+0x9e/frame 0xfffffe085cff8610 vm_fault_prefault() at vm_fault_prefault+0x204/frame 0xfffffe085cff8680 vm_fault_hold() at vm_fault_hold+0x1c0e/frame 0xfffffe085cff87e0 vm_fault() at vm_fault+0x77/frame 0xfffffe085cff8820 trap_pfault() at trap_pfault+0x11a/frame 0xfffffe085cff88a0 trap() at trap+0x58f/frame 0xfffffe085cff8ab0 calltrap() at calltrap+0x8/frame 0xfffffe085cff8ab0 --- trap 0xc, rip = 0x4012c4, rsp = 0x7fffffffe6b0, rbp = 0x7fffffffe820 --- Tracing command holdcnt0 pid 1062 tid 100189 td 0xfffff800745ae560 (CPU 9) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07c1594cf0 ipi_nmi_handler() at ipi_nmi_handler+0x4a/frame 0xfffffe07c1594d10 trap() at trap+0x49/frame 0xfffffe07c1594f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07c1594f20 --- trap 0x13, rip = 0xffffffff80d6c5e6, rsp = 0xfffffe085cff3610, rbp = 0xfffffe085cff3610 --- vm_radix_lookup() at vm_radix_lookup+0x66/frame 0xfffffe085cff3610 vm_fault_prefault() at vm_fault_prefault+0x141/frame 0xfffffe085cff3680 vm_fault_hold() at vm_fault_hold+0x1c0e/frame 0xfffffe085cff37e0 vm_fault() at vm_fault+0x77/frame 0xfffffe085cff3820 trap_pfault() at trap_pfault+0x11a/frame 0xfffffe085cff38a0 trap() at trap+0x58f/frame 0xfffffe085cff3ab0 calltrap() at calltrap+0x8/frame 0xfffffe085cff3ab0 --- trap 0xc, rip = 0x4012c4, rsp = 0x7fffffffe6b0, rbp = 0x7fffffffe820 --- Tracing command holdcnt0 pid 1061 tid 100188 td 0xfffff800745af000 (CPU 4) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07c79e3cf0 ipi_nmi_handler() at ipi_nmi_handler+0x4a/frame 0xfffffe07c79e3d10 trap() at trap+0x49/frame 0xfffffe07c79e3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07c79e3f20 --- trap 0x13, rip = 0xffffffff80a93d32, rsp = 0xfffffe085cfee4d0, rbp = 0xfffffe085cfee4e0 --- lock_delay() at lock_delay+0x42/frame 0xfffffe085cfee4e0 __mtx_lock_sleep() at __mtx_lock_sleep+0x108/frame 0xfffffe085cfee580 __mtx_lock_flags() at __mtx_lock_flags+0x103/frame 0xfffffe085cfee5d0 pmap_is_prefaultable() at pmap_is_prefaultable+0x44/frame 0xfffffe085cfee610 vm_fault_prefault() at vm_fault_prefault+0xf2/frame 0xfffffe085cfee680 vm_fault_hold() at vm_fault_hold+0x1c0e/frame 0xfffffe085cfee7e0 vm_fault() at vm_fault+0x77/frame 0xfffffe085cfee820 trap_pfault() at trap_pfault+0x11a/frame 0xfffffe085cfee8a0 trap() at trap+0x58f/frame 0xfffffe085cfeeab0 calltrap() at calltrap+0x8/frame 0xfffffe085cfeeab0 --- trap 0xc, rip = 0x4012c4, rsp = 0x7fffffffe6b0, rbp = 0x7fffffffe820 --- Tracing command holdcnt0 pid 1060 tid 100187 td 0xfffff8001222b000 (CPU 8) kdb_enter() at kdb_enter+0x3e/frame 0xfffffe085cfe9290 vpanic() at vpanic+0x1bf/frame 0xfffffe085cfe9310 kassert_panic() at kassert_panic+0x139/frame 0xfffffe085cfe9380 pmap_delayed_invl_finished() at pmap_delayed_invl_finished+0x55/frame 0xfffffe085cfe93b0 reclaim_pv_chunk_leave_pmap() at reclaim_pv_chunk_leave_pmap+0x70/frame 0xfffffe085cfe93e0 reclaim_pv_chunk() at reclaim_pv_chunk+0x1281/frame 0xfffffe085cfe9520 get_pv_entry() at get_pv_entry+0x326/frame 0xfffffe085cfe9580 pmap_enter() at pmap_enter+0x8f6/frame 0xfffffe085cfe9680 vm_fault_hold() at vm_fault_hold+0x21d/frame 0xfffffe085cfe97e0 vm_fault() at vm_fault+0x77/frame 0xfffffe085cfe9820 trap_pfault() at trap_pfault+0x11a/frame 0xfffffe085cfe98a0 trap() at trap+0x58f/frame 0xfffffe085cfe9ab0 calltrap() at calltrap+0x8/frame 0xfffffe085cfe9ab0 --- trap 0xc, rip = 0x4012c4, rsp = 0x7fffffffe6b0, rbp = 0x7fffffffe820 --- Tracing command holdcnt0 pid 1059 tid 100186 td 0xfffff8001222b560 (CPU 2) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07c79d3cf0 ipi_nmi_handler() at ipi_nmi_handler+0x4a/frame 0xfffffe07c79d3d10 trap() at trap+0x49/frame 0xfffffe07c79d3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07c79d3f20 --- trap 0x13, rip = 0xffffffff80d456b7, rsp = 0xfffffe085cfe42c0, rbp = 0xfffffe085cfe4320 --- uma_zalloc_arg() at uma_zalloc_arg+0x177/frame 0xfffffe085cfe4320 g_vfs_strategy() at g_vfs_strategy+0x8f/frame 0xfffffe085cfe4350 ufs_strategy() at ufs_strategy+0x83/frame 0xfffffe085cfe4380 VOP_STRATEGY_APV() at VOP_STRATEGY_APV+0x10c/frame 0xfffffe085cfe43b0 bufstrategy() at bufstrategy+0x46/frame 0xfffffe085cfe43f0 bufwrite() at bufwrite+0x208/frame 0xfffffe085cfe4440 ffs_write() at ffs_write+0x3f7/frame 0xfffffe085cfe44e0 VOP_WRITE_APV() at VOP_WRITE_APV+0x17f/frame 0xfffffe085cfe4600 vn_write() at vn_write+0x22d/frame 0xfffffe085cfe4680 vn_io_fault_doio() at vn_io_fault_doio+0x22/frame 0xfffffe085cfe46c0 vn_io_fault1() at vn_io_fault1+0x1bb/frame 0xfffffe085cfe4810 vn_io_fault() at vn_io_fault+0x188/frame 0xfffffe085cfe4890 dofilewrite() at dofilewrite+0xab/frame 0xfffffe085cfe48e0 kern_writev() at kern_writev+0x68/frame 0xfffffe085cfe4930 sys_write() at sys_write+0x8b/frame 0xfffffe085cfe4980 amd64_syscall() at amd64_syscall+0x2eb/frame 0xfffffe085cfe4ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe085cfe4ab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x8009ae48a, rsp = 0x7fffffffe6c8, rbp = 0x7fffffffe820 --- Tracing command holdcnt0 pid 1058 tid 100168 td 0xfffff80012233000 (CPU 7) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07c79fbcf0 ipi_nmi_handler() at ipi_nmi_handler+0x4a/frame 0xfffffe07c79fbd10 trap() at trap+0x49/frame 0xfffffe07c79fbf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07c79fbf20 --- trap 0x13, rip = 0xffffffff80ab25f2, rsp = 0xfffffe085cf8a150, rbp = 0xfffffe085cf8a1c0 --- witness_unlock() at witness_unlock+0xb2/frame 0xfffffe085cf8a1c0 _rw_wunlock_cookie() at _rw_wunlock_cookie+0x66/frame 0xfffffe085cf8a1f0 bdwrite() at bdwrite+0x1db/frame 0xfffffe085cf8a260 ffs_balloc_ufs2() at ffs_balloc_ufs2+0x1f33/frame 0xfffffe085cf8a440 ffs_write() at ffs_write+0x218/frame 0xfffffe085cf8a4e0 VOP_WRITE_APV() at VOP_WRITE_APV+0x17f/frame 0xfffffe085cf8a600 vn_write() at vn_write+0x22d/frame 0xfffffe085cf8a680 vn_io_fault_doio() at vn_io_fault_doio+0x22/frame 0xfffffe085cf8a6c0 vn_io_fault1() at vn_io_fault1+0x1bb/frame 0xfffffe085cf8a810 vn_io_fault() at vn_io_fault+0x188/frame 0xfffffe085cf8a890 dofilewrite() at dofilewrite+0xab/frame 0xfffffe085cf8a8e0 kern_writev() at kern_writev+0x68/frame 0xfffffe085cf8a930 sys_write() at sys_write+0x8b/frame 0xfffffe085cf8a980 amd64_syscall() at amd64_syscall+0x2eb/frame 0xfffffe085cf8aab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe085cf8aab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x8009ae48a, rsp = 0x7fffffffe6c8, rbp = 0x7fffffffe820 --- Tracing command holdcnt0 pid 1056 tid 100178 td 0xfffff80012f90000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe085cfe87c0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe085cfe8870 kdb_reenter() at kdb_reenter+0x33/frame 0xfffffe085cfe8880 trap() at trap+0x63/frame 0xfffffe085cfe8a90 calltrap() at calltrap+0x8/frame 0xfffffe085cfe8a90 --- trap 0xc, rip = 0xffffffff80eba408, rsp = 0xfffffe085cfe8b60, rbp = 0xfffffe085cfe8b70 --- db_trace_thread() at db_trace_thread+0x28/frame 0xfffffe085cfe8b70 _db_stack_trace_all() at _db_stack_trace_all+0xe4/frame 0xfffffe085cfe8c20 db_command() at db_command+0x252/frame 0xfffffe085cfe8cf0 db_script_exec() at db_script_exec+0x18f/frame 0xfffffe085cfe8d90 db_command() at db_command+0x252/frame 0xfffffe085cfe8e60 db_command_loop() at db_command_loop+0x64/frame 0xfffffe085cfe8e70 db_trap() at db_trap+0xef/frame 0xfffffe085cfe8f00 kdb_trap() at kdb_trap+0x191/frame 0xfffffe085cfe8fa0 trap() at trap+0x4df/frame 0xfffffe085cfe91b0 calltrap() at calltrap+0x8/frame 0xfffffe085cfe91b0 --- trap 0x3, rip = 0xffffffff80a9292e, rsp = 0xfffffe085cfe9280, rbp = 0xfffffe085cfe9290 --- kdb_enter() at kdb_enter+0x3e/frame 0xfffffe085cfe9290 vpanic() at vpanic+0x1bf/frame 0xfffffe085cfe9310 kassert_panic() at kassert_panic+0x139/frame 0xfffffe085cfe9380 pmap_delayed_invl_finished() at pmap_delayed_invl_finished+0x55/frame 0xfffffe085cfe93b0 reclaim_pv_chunk_leave_pmap() at reclaim_pv_chunk_leave_pmap+0x70/frame 0xfffffe085cfe93e0 reclaim_pv_chunk() at reclaim_pv_chunk+0x1281/frame 0xfffffe085cfe9520 get_pv_entry() at get_pv_entry+0x326/frame 0xfffffe085cfe9580 pmap_enter() at pmap_enter+0x8f6/frame 0xfffffe085cfe9680 vm_fault_hold() at vm_fault_hold+0x21d/frame 0xfffffe085cfe97e0 vm_fault() at vm_fault+0x77/frame 0xfffffe085cfe9820 trap_pfault() at trap_pfault+0x11a/frame 0xfffffe085cfe98a0 trap() at trap+0x58f/frame 0xfffffe085cfe9ab0 calltrap() at calltrap+0x8/frame 0xfffffe085cfe9ab0 --- trap 0xc, rip = 0x4012c4, rsp = 0x7fffffffe6b0, rbp = 0x7fffffffe820 --- Tracing command md10 pid 1052 tid 100185 td 0xfffff800743cd560 sched_switch() at sched_switch+0x486/frame 0xfffffe085cfdf8d0 mi_switch() at mi_switch+0x195/frame 0xfffffe085cfdf910 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cfdf950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085cfdf980 _sleep() at _sleep+0x30b/frame 0xfffffe085cfdfa20 md_kthread() at md_kthread+0xe5/frame 0xfffffe085cfdfa70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cfdfab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cfdfab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command tee pid 1030 tid 100184 td 0xfffff8000f7ea560 sched_switch() at sched_switch+0x486/frame 0xfffffe085cfda670 mi_switch() at mi_switch+0x195/frame 0xfffffe085cfda6b0 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cfda6f0 sleepq_catch_signals() at sleepq_catch_signals+0x466/frame 0xfffffe085cfda760 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe085cfda790 _sleep() at _sleep+0x301/frame 0xfffffe085cfda830 pipe_read() at pipe_read+0x296/frame 0xfffffe085cfda890 dofileread() at dofileread+0xb2/frame 0xfffffe085cfda8e0 kern_readv() at kern_readv+0x68/frame 0xfffffe085cfda930 sys_read() at sys_read+0x8b/frame 0xfffffe085cfda980 amd64_syscall() at amd64_syscall+0x2eb/frame 0xfffffe085cfdaab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe085cfdaab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8009ae4ca, rsp = 0x7fffffffe7f8, rbp = 0x7fffffffe840 --- Tracing command sh pid 1029 tid 100152 td 0xfffff80012135000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe085cfe87c0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe085cfe8870 kdb_reenter() at kdb_reenter+0x33/frame 0xfffffe085cfe8880 trap() at trap+0x63/frame 0xfffffe085cfe8a90 calltrap() at calltrap+0x8/frame 0xfffffe085cfe8a90 --- trap 0xc, rip = 0xffffffff80eba408, rsp = 0xfffffe085cfe8b60, rbp = 0xfffffe085cfe8b70 --- db_trace_thread() at db_trace_thread+0x28/frame 0xfffffe085cfe8b70 _db_stack_trace_all() at _db_stack_trace_all+0xe4/frame 0xfffffe085cfe8c20 db_command() at db_command+0x252/frame 0xfffffe085cfe8cf0 db_script_exec() at db_script_exec+0x18f/frame 0xfffffe085cfe8d90 db_command() at db_command+0x252/frame 0xfffffe085cfe8e60 db_command_loop() at db_command_loop+0x64/frame 0xfffffe085cfe8e70 db_trap() at db_trap+0xef/frame 0xfffffe085cfe8f00 kdb_trap() at kdb_trap+0x191/frame 0xfffffe085cfe8fa0 trap() at trap+0x4df/frame 0xfffffe085cfe91b0 calltrap() at calltrap+0x8/frame 0xfffffe085cfe91b0 --- trap 0x3, rip = 0xffffffff80a9292e, rsp = 0xfffffe085cfe9280, rbp = 0xfffffe085cfe9290 --- kdb_enter() at kdb_enter+0x3e/frame 0xfffffe085cfe9290 vpanic() at vpanic+0x1bf/frame 0xfffffe085cfe9310 kassert_panic() at kassert_panic+0x139/frame 0xfffffe085cfe9380 pmap_delayed_invl_finished() at pmap_delayed_invl_finished+0x55/frame 0xfffffe085cfe93b0 reclaim_pv_chunk_leave_pmap() at reclaim_pv_chunk_leave_pmap+0x70/frame 0xfffffe085cfe93e0 reclaim_pv_chunk() at reclaim_pv_chunk+0x1281/frame 0xfffffe085cfe9520 get_pv_entry() at get_pv_entry+0x326/frame 0xfffffe085cfe9580 pmap_enter() at pmap_enter+0x8f6/frame 0xfffffe085cfe9680 vm_fault_hold() at vm_fault_hold+0x21d/frame 0xfffffe085cfe97e0 vm_fault() at vm_fault+0x77/frame 0xfffffe085cfe9820 trap_pfault() at trap_pfault+0x11a/frame 0xfffffe085cfe98a0 trap() at trap+0x58f/frame 0xfffffe085cfe9ab0 calltrap() at calltrap+0x8/frame 0xfffffe085cfe9ab0 --- trap 0xc, rip = 0x4012c4, rsp = 0x7fffffffe6b0, rbp = 0x7fffffffe820 --- Tracing command sh pid 1028 tid 100129 td 0xfffff8000f7ec560 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe085cfe87c0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe085cfe8870 kdb_reenter() at kdb_reenter+0x33/frame 0xfffffe085cfe8880 trap() at trap+0x63/frame 0xfffffe085cfe8a90 calltrap() at calltrap+0x8/frame 0xfffffe085cfe8a90 --- trap 0xc, rip = 0xffffffff80eba408, rsp = 0xfffffe085cfe8b60, rbp = 0xfffffe085cfe8b70 --- db_trace_thread() at db_trace_thread+0x28/frame 0xfffffe085cfe8b70 _db_stack_trace_all() at _db_stack_trace_all+0xe4/frame 0xfffffe085cfe8c20 db_command() at db_command+0x252/frame 0xfffffe085cfe8cf0 db_script_exec() at db_script_exec+0x18f/frame 0xfffffe085cfe8d90 db_command() at db_command+0x252/frame 0xfffffe085cfe8e60 db_command_loop() at db_command_loop+0x64/frame 0xfffffe085cfe8e70 db_trap() at db_trap+0xef/frame 0xfffffe085cfe8f00 kdb_trap() at kdb_trap+0x191/frame 0xfffffe085cfe8fa0 trap() at trap+0x4df/frame 0xfffffe085cfe91b0 calltrap() at calltrap+0x8/frame 0xfffffe085cfe91b0 --- trap 0x3, rip = 0xffffffff80a9292e, rsp = 0xfffffe085cfe9280, rbp = 0xfffffe085cfe9290 --- kdb_enter() at kdb_enter+0x3e/frame 0xfffffe085cfe9290 vpanic() at vpanic+0x1bf/frame 0xfffffe085cfe9310 kassert_panic() at kassert_panic+0x139/frame 0xfffffe085cfe9380 pmap_delayed_invl_finished() at pmap_delayed_invl_finished+0x55/frame 0xfffffe085cfe93b0 reclaim_pv_chunk_leave_pmap() at reclaim_pv_chunk_leave_pmap+0x70/frame 0xfffffe085cfe93e0 reclaim_pv_chunk() at reclaim_pv_chunk+0x1281/frame 0xfffffe085cfe9520 get_pv_entry() at get_pv_entry+0x326/frame 0xfffffe085cfe9580 pmap_enter() at pmap_enter+0x8f6/frame 0xfffffe085cfe9680 vm_fault_hold() at vm_fault_hold+0x21d/frame 0xfffffe085cfe97e0 vm_fault() at vm_fault+0x77/frame 0xfffffe085cfe9820 trap_pfault() at trap_pfault+0x11a/frame 0xfffffe085cfe98a0 trap() at trap+0x58f/frame 0xfffffe085cfe9ab0 calltrap() at calltrap+0x8/frame 0xfffffe085cfe9ab0 --- trap 0xc, rip = 0x4012c4, rsp = 0x7fffffffe6b0, rbp = 0x7fffffffe820 --- Tracing command sh pid 745 tid 100132 td 0xfffff80012545560 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe085cfe87c0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe085cfe8870 kdb_reenter() at kdb_reenter+0x33/frame 0xfffffe085cfe8880 trap() at trap+0x63/frame 0xfffffe085cfe8a90 calltrap() at calltrap+0x8/frame 0xfffffe085cfe8a90 --- trap 0xc, rip = 0xffffffff80eba408, rsp = 0xfffffe085cfe8b60, rbp = 0xfffffe085cfe8b70 --- db_trace_thread() at db_trace_thread+0x28/frame 0xfffffe085cfe8b70 _db_stack_trace_all() at _db_stack_trace_all+0xe4/frame 0xfffffe085cfe8c20 db_command() at db_command+0x252/frame 0xfffffe085cfe8cf0 db_script_exec() at db_script_exec+0x18f/frame 0xfffffe085cfe8d90 db_command() at db_command+0x252/frame 0xfffffe085cfe8e60 db_command_loop() at db_command_loop+0x64/frame 0xfffffe085cfe8e70 db_trap() at db_trap+0xef/frame 0xfffffe085cfe8f00 kdb_trap() at kdb_trap+0x191/frame 0xfffffe085cfe8fa0 trap() at trap+0x4df/frame 0xfffffe085cfe91b0 calltrap() at calltrap+0x8/frame 0xfffffe085cfe91b0 --- trap 0x3, rip = 0xffffffff80a9292e, rsp = 0xfffffe085cfe9280, rbp = 0xfffffe085cfe9290 --- kdb_enter() at kdb_enter+0x3e/frame 0xfffffe085cfe9290 vpanic() at vpanic+0x1bf/frame 0xfffffe085cfe9310 kassert_panic() at kassert_panic+0x139/frame 0xfffffe085cfe9380 pmap_delayed_invl_finished() at pmap_delayed_invl_finished+0x55/frame 0xfffffe085cfe93b0 reclaim_pv_chunk_leave_pmap() at reclaim_pv_chunk_leave_pmap+0x70/frame 0xfffffe085cfe93e0 reclaim_pv_chunk() at reclaim_pv_chunk+0x1281/frame 0xfffffe085cfe9520 get_pv_entry() at get_pv_entry+0x326/frame 0xfffffe085cfe9580 pmap_enter() at pmap_enter+0x8f6/frame 0xfffffe085cfe9680 vm_fault_hold() at vm_fault_hold+0x21d/frame 0xfffffe085cfe97e0 vm_fault() at vm_fault+0x77/frame 0xfffffe085cfe9820 trap_pfault() at trap_pfault+0x11a/frame 0xfffffe085cfe98a0 trap() at trap+0x58f/frame 0xfffffe085cfe9ab0 calltrap() at calltrap+0x8/frame 0xfffffe085cfe9ab0 --- trap 0xc, rip = 0x4012c4, rsp = 0x7fffffffe6b0, rbp = 0x7fffffffe820 --- Tracing command csh pid 741 tid 100166 td 0xfffff80012f92560 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe085cfe87c0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe085cfe8870 kdb_reenter() at kdb_reenter+0x33/frame 0xfffffe085cfe8880 trap() at trap+0x63/frame 0xfffffe085cfe8a90 calltrap() at calltrap+0x8/frame 0xfffffe085cfe8a90 --- trap 0xc, rip = 0xffffffff80eba408, rsp = 0xfffffe085cfe8b60, rbp = 0xfffffe085cfe8b70 --- db_trace_thread() at db_trace_thread+0x28/frame 0xfffffe085cfe8b70 _db_stack_trace_all() at _db_stack_trace_all+0xe4/frame 0xfffffe085cfe8c20 db_command() at db_command+0x252/frame 0xfffffe085cfe8cf0 db_script_exec() at db_script_exec+0x18f/frame 0xfffffe085cfe8d90 db_command() at db_command+0x252/frame 0xfffffe085cfe8e60 db_command_loop() at db_command_loop+0x64/frame 0xfffffe085cfe8e70 db_trap() at db_trap+0xef/frame 0xfffffe085cfe8f00 kdb_trap() at kdb_trap+0x191/frame 0xfffffe085cfe8fa0 trap() at trap+0x4df/frame 0xfffffe085cfe91b0 calltrap() at calltrap+0x8/frame 0xfffffe085cfe91b0 --- trap 0x3, rip = 0xffffffff80a9292e, rsp = 0xfffffe085cfe9280, rbp = 0xfffffe085cfe9290 --- kdb_enter() at kdb_enter+0x3e/frame 0xfffffe085cfe9290 vpanic() at vpanic+0x1bf/frame 0xfffffe085cfe9310 kassert_panic() at kassert_panic+0x139/frame 0xfffffe085cfe9380 pmap_delayed_invl_finished() at pmap_delayed_invl_finished+0x55/frame 0xfffffe085cfe93b0 reclaim_pv_chunk_leave_pmap() at reclaim_pv_chunk_leave_pmap+0x70/frame 0xfffffe085cfe93e0 reclaim_pv_chunk() at reclaim_pv_chunk+0x1281/frame 0xfffffe085cfe9520 get_pv_entry() at get_pv_entry+0x326/frame 0xfffffe085cfe9580 pmap_enter() at pmap_enter+0x8f6/frame 0xfffffe085cfe9680 vm_fault_hold() at vm_fault_hold+0x21d/frame 0xfffffe085cfe97e0 vm_fault() at vm_fault+0x77/frame 0xfffffe085cfe9820 trap_pfault() at trap_pfault+0x11a/frame 0xfffffe085cfe98a0 trap() at trap+0x58f/frame 0xfffffe085cfe9ab0 calltrap() at calltrap+0x8/frame 0xfffffe085cfe9ab0 --- trap 0xc, rip = 0x4012c4, rsp = 0x7fffffffe6b0, rbp = 0x7fffffffe820 --- Tracing command login pid 740 tid 100125 td 0xfffff800078a3000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe085cfe87c0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe085cfe8870 kdb_reenter() at kdb_reenter+0x33/frame 0xfffffe085cfe8880 trap() at trap+0x63/frame 0xfffffe085cfe8a90 calltrap() at calltrap+0x8/frame 0xfffffe085cfe8a90 --- trap 0xc, rip = 0xffffffff80eba408, rsp = 0xfffffe085cfe8b60, rbp = 0xfffffe085cfe8b70 --- db_trace_thread() at db_trace_thread+0x28/frame 0xfffffe085cfe8b70 _db_stack_trace_all() at _db_stack_trace_all+0xe4/frame 0xfffffe085cfe8c20 db_command() at db_command+0x252/frame 0xfffffe085cfe8cf0 db_script_exec() at db_script_exec+0x18f/frame 0xfffffe085cfe8d90 db_command() at db_command+0x252/frame 0xfffffe085cfe8e60 db_command_loop() at db_command_loop+0x64/frame 0xfffffe085cfe8e70 db_trap() at db_trap+0xef/frame 0xfffffe085cfe8f00 kdb_trap() at kdb_trap+0x191/frame 0xfffffe085cfe8fa0 trap() at trap+0x4df/frame 0xfffffe085cfe91b0 calltrap() at calltrap+0x8/frame 0xfffffe085cfe91b0 --- trap 0x3, rip = 0xffffffff80a9292e, rsp = 0xfffffe085cfe9280, rbp = 0xfffffe085cfe9290 --- kdb_enter() at kdb_enter+0x3e/frame 0xfffffe085cfe9290 vpanic() at vpanic+0x1bf/frame 0xfffffe085cfe9310 kassert_panic() at kassert_panic+0x139/frame 0xfffffe085cfe9380 pmap_delayed_invl_finished() at pmap_delayed_invl_finished+0x55/frame 0xfffffe085cfe93b0 reclaim_pv_chunk_leave_pmap() at reclaim_pv_chunk_leave_pmap+0x70/frame 0xfffffe085cfe93e0 reclaim_pv_chunk() at reclaim_pv_chunk+0x1281/frame 0xfffffe085cfe9520 get_pv_entry() at get_pv_entry+0x326/frame 0xfffffe085cfe9580 pmap_enter() at pmap_enter+0x8f6/frame 0xfffffe085cfe9680 vm_fault_hold() at vm_fault_hold+0x21d/frame 0xfffffe085cfe97e0 vm_fault() at vm_fault+0x77/frame 0xfffffe085cfe9820 trap_pfault() at trap_pfault+0x11a/frame 0xfffffe085cfe98a0 trap() at trap+0x58f/frame 0xfffffe085cfe9ab0 calltrap() at calltrap+0x8/frame 0xfffffe085cfe9ab0 --- trap 0xc, rip = 0x4012c4, rsp = 0x7fffffffe6b0, rbp = 0x7fffffffe820 --- Tracing command getty pid 739 tid 100141 td 0xfffff8000f7e5000 sched_switch() at sched_switch+0x486/frame 0xfffffe085cf035e0 mi_switch() at mi_switch+0x195/frame 0xfffffe085cf03620 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cf03660 sleepq_catch_signals() at sleepq_catch_signals+0x466/frame 0xfffffe085cf036d0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe085cf03700 _cv_wait_sig() at _cv_wait_sig+0x1da/frame 0xfffffe085cf03770 tty_wait() at tty_wait+0x42/frame 0xfffffe085cf037a0 ttydisc_read() at ttydisc_read+0x34f/frame 0xfffffe085cf03800 ttydev_read() at ttydev_read+0x6d/frame 0xfffffe085cf03830 devfs_read_f() at devfs_read_f+0xb3/frame 0xfffffe085cf03890 dofileread() at dofileread+0xb2/frame 0xfffffe085cf038e0 kern_readv() at kern_readv+0x68/frame 0xfffffe085cf03930 sys_read() at sys_read+0x8b/frame 0xfffffe085cf03980 amd64_syscall() at amd64_syscall+0x2eb/frame 0xfffffe085cf03ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe085cf03ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800bc74ca, rsp = 0x7fffffffec68, rbp = 0x7fffffffecb0 --- Tracing command getty pid 738 tid 100123 td 0xfffff800081b0560 sched_switch() at sched_switch+0x486/frame 0xfffffe085cea95e0 mi_switch() at mi_switch+0x195/frame 0xfffffe085cea9620 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cea9660 sleepq_catch_signals() at sleepq_catch_signals+0x466/frame 0xfffffe085cea96d0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe085cea9700 _cv_wait_sig() at _cv_wait_sig+0x1da/frame 0xfffffe085cea9770 tty_wait() at tty_wait+0x42/frame 0xfffffe085cea97a0 ttydisc_read() at ttydisc_read+0x34f/frame 0xfffffe085cea9800 ttydev_read() at ttydev_read+0x6d/frame 0xfffffe085cea9830 devfs_read_f() at devfs_read_f+0xb3/frame 0xfffffe085cea9890 dofileread() at dofileread+0xb2/frame 0xfffffe085cea98e0 kern_readv() at kern_readv+0x68/frame 0xfffffe085cea9930 sys_read() at sys_read+0x8b/frame 0xfffffe085cea9980 amd64_syscall() at amd64_syscall+0x2eb/frame 0xfffffe085cea9ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe085cea9ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800bc74ca, rsp = 0x7fffffffec68, rbp = 0x7fffffffecb0 --- Tracing command getty pid 737 tid 100165 td 0xfffff80012f93000 sched_switch() at sched_switch+0x486/frame 0xfffffe085cf7b5e0 mi_switch() at mi_switch+0x195/frame 0xfffffe085cf7b620 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cf7b660 sleepq_catch_signals() at sleepq_catch_signals+0x466/frame 0xfffffe085cf7b6d0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe085cf7b700 _cv_wait_sig() at _cv_wait_sig+0x1da/frame 0xfffffe085cf7b770 tty_wait() at tty_wait+0x42/frame 0xfffffe085cf7b7a0 ttydisc_read() at ttydisc_read+0x34f/frame 0xfffffe085cf7b800 ttydev_read() at ttydev_read+0x6d/frame 0xfffffe085cf7b830 devfs_read_f() at devfs_read_f+0xb3/frame 0xfffffe085cf7b890 dofileread() at dofileread+0xb2/frame 0xfffffe085cf7b8e0 kern_readv() at kern_readv+0x68/frame 0xfffffe085cf7b930 sys_read() at sys_read+0x8b/frame 0xfffffe085cf7b980 amd64_syscall() at amd64_syscall+0x2eb/frame 0xfffffe085cf7bab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe085cf7bab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800bc74ca, rsp = 0x7fffffffec68, rbp = 0x7fffffffecb0 --- Tracing command getty pid 736 tid 100164 td 0xfffff80012f93560 sched_switch() at sched_switch+0x486/frame 0xfffffe085cf765e0 mi_switch() at mi_switch+0x195/frame 0xfffffe085cf76620 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cf76660 sleepq_catch_signals() at sleepq_catch_signals+0x466/frame 0xfffffe085cf766d0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe085cf76700 _cv_wait_sig() at _cv_wait_sig+0x1da/frame 0xfffffe085cf76770 tty_wait() at tty_wait+0x42/frame 0xfffffe085cf767a0 ttydisc_read() at ttydisc_read+0x34f/frame 0xfffffe085cf76800 ttydev_read() at ttydev_read+0x6d/frame 0xfffffe085cf76830 devfs_read_f() at devfs_read_f+0xb3/frame 0xfffffe085cf76890 dofileread() at dofileread+0xb2/frame 0xfffffe085cf768e0 kern_readv() at kern_readv+0x68/frame 0xfffffe085cf76930 sys_read() at sys_read+0x8b/frame 0xfffffe085cf76980 amd64_syscall() at amd64_syscall+0x2eb/frame 0xfffffe085cf76ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe085cf76ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800bc74ca, rsp = 0x7fffffffec68, rbp = 0x7fffffffecb0 --- Tracing command getty pid 735 tid 100163 td 0xfffff80012f00000 sched_switch() at sched_switch+0x486/frame 0xfffffe085cf715e0 mi_switch() at mi_switch+0x195/frame 0xfffffe085cf71620 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cf71660 sleepq_catch_signals() at sleepq_catch_signals+0x466/frame 0xfffffe085cf716d0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe085cf71700 _cv_wait_sig() at _cv_wait_sig+0x1da/frame 0xfffffe085cf71770 tty_wait() at tty_wait+0x42/frame 0xfffffe085cf717a0 ttydisc_read() at ttydisc_read+0x34f/frame 0xfffffe085cf71800 ttydev_read() at ttydev_read+0x6d/frame 0xfffffe085cf71830 devfs_read_f() at devfs_read_f+0xb3/frame 0xfffffe085cf71890 dofileread() at dofileread+0xb2/frame 0xfffffe085cf718e0 kern_readv() at kern_readv+0x68/frame 0xfffffe085cf71930 sys_read() at sys_read+0x8b/frame 0xfffffe085cf71980 amd64_syscall() at amd64_syscall+0x2eb/frame 0xfffffe085cf71ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe085cf71ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800bc74ca, rsp = 0x7fffffffec68, rbp = 0x7fffffffecb0 --- Tracing command getty pid 734 tid 100162 td 0xfffff80007816560 sched_switch() at sched_switch+0x486/frame 0xfffffe085cf6c5e0 mi_switch() at mi_switch+0x195/frame 0xfffffe085cf6c620 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cf6c660 sleepq_catch_signals() at sleepq_catch_signals+0x466/frame 0xfffffe085cf6c6d0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe085cf6c700 _cv_wait_sig() at _cv_wait_sig+0x1da/frame 0xfffffe085cf6c770 tty_wait() at tty_wait+0x42/frame 0xfffffe085cf6c7a0 ttydisc_read() at ttydisc_read+0x34f/frame 0xfffffe085cf6c800 ttydev_read() at ttydev_read+0x6d/frame 0xfffffe085cf6c830 devfs_read_f() at devfs_read_f+0xb3/frame 0xfffffe085cf6c890 dofileread() at dofileread+0xb2/frame 0xfffffe085cf6c8e0 kern_readv() at kern_readv+0x68/frame 0xfffffe085cf6c930 sys_read() at sys_read+0x8b/frame 0xfffffe085cf6c980 amd64_syscall() at amd64_syscall+0x2eb/frame 0xfffffe085cf6cab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe085cf6cab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800bc74ca, rsp = 0x7fffffffec68, rbp = 0x7fffffffecb0 --- Tracing command getty pid 733 tid 100151 td 0xfffff80007821560 sched_switch() at sched_switch+0x486/frame 0xfffffe085cf355e0 mi_switch() at mi_switch+0x195/frame 0xfffffe085cf35620 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cf35660 sleepq_catch_signals() at sleepq_catch_signals+0x466/frame 0xfffffe085cf356d0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe085cf35700 _cv_wait_sig() at _cv_wait_sig+0x1da/frame 0xfffffe085cf35770 tty_wait() at tty_wait+0x42/frame 0xfffffe085cf357a0 ttydisc_read() at ttydisc_read+0x34f/frame 0xfffffe085cf35800 ttydev_read() at ttydev_read+0x6d/frame 0xfffffe085cf35830 devfs_read_f() at devfs_read_f+0xb3/frame 0xfffffe085cf35890 dofileread() at dofileread+0xb2/frame 0xfffffe085cf358e0 kern_readv() at kern_readv+0x68/frame 0xfffffe085cf35930 sys_read() at sys_read+0x8b/frame 0xfffffe085cf35980 amd64_syscall() at amd64_syscall+0x2eb/frame 0xfffffe085cf35ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe085cf35ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800bc74ca, rsp = 0x7fffffffec68, rbp = 0x7fffffffecb0 --- Tracing command getty pid 732 tid 100156 td 0xfffff80007821000 sched_switch() at sched_switch+0x486/frame 0xfffffe085cf4e5e0 mi_switch() at mi_switch+0x195/frame 0xfffffe085cf4e620 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cf4e660 sleepq_catch_signals() at sleepq_catch_signals+0x466/frame 0xfffffe085cf4e6d0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe085cf4e700 _cv_wait_sig() at _cv_wait_sig+0x1da/frame 0xfffffe085cf4e770 tty_wait() at tty_wait+0x42/frame 0xfffffe085cf4e7a0 ttydisc_read() at ttydisc_read+0x34f/frame 0xfffffe085cf4e800 ttydev_read() at ttydev_read+0x6d/frame 0xfffffe085cf4e830 devfs_read_f() at devfs_read_f+0xb3/frame 0xfffffe085cf4e890 dofileread() at dofileread+0xb2/frame 0xfffffe085cf4e8e0 kern_readv() at kern_readv+0x68/frame 0xfffffe085cf4e930 sys_read() at sys_read+0x8b/frame 0xfffffe085cf4e980 amd64_syscall() at amd64_syscall+0x2eb/frame 0xfffffe085cf4eab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe085cf4eab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800bc74ca, rsp = 0x7fffffffec68, rbp = 0x7fffffffecb0 --- Tracing command getty pid 731 tid 100111 td 0xfffff800078a4000 sched_switch() at sched_switch+0x486/frame 0xfffffe085ce6d5e0 mi_switch() at mi_switch+0x195/frame 0xfffffe085ce6d620 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085ce6d660 sleepq_catch_signals() at sleepq_catch_signals+0x466/frame 0xfffffe085ce6d6d0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe085ce6d700 _cv_wait_sig() at _cv_wait_sig+0x1da/frame 0xfffffe085ce6d770 tty_wait() at tty_wait+0x42/frame 0xfffffe085ce6d7a0 ttydisc_read() at ttydisc_read+0x34f/frame 0xfffffe085ce6d800 ttydev_read() at ttydev_read+0x6d/frame 0xfffffe085ce6d830 devfs_read_f() at devfs_read_f+0xb3/frame 0xfffffe085ce6d890 dofileread() at dofileread+0xb2/frame 0xfffffe085ce6d8e0 kern_readv() at kern_readv+0x68/frame 0xfffffe085ce6d930 sys_read() at sys_read+0x8b/frame 0xfffffe085ce6d980 amd64_syscall() at amd64_syscall+0x2eb/frame 0xfffffe085ce6dab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe085ce6dab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800bc74ca, rsp = 0x7fffffffec68, rbp = 0x7fffffffecb0 --- Tracing command cron pid 695 tid 100146 td 0xfffff8000f7e3000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe085cfe87c0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe085cfe8870 kdb_reenter() at kdb_reenter+0x33/frame 0xfffffe085cfe8880 trap() at trap+0x63/frame 0xfffffe085cfe8a90 calltrap() at calltrap+0x8/frame 0xfffffe085cfe8a90 --- trap 0xc, rip = 0xffffffff80eba408, rsp = 0xfffffe085cfe8b60, rbp = 0xfffffe085cfe8b70 --- db_trace_thread() at db_trace_thread+0x28/frame 0xfffffe085cfe8b70 _db_stack_trace_all() at _db_stack_trace_all+0xe4/frame 0xfffffe085cfe8c20 db_command() at db_command+0x252/frame 0xfffffe085cfe8cf0 db_script_exec() at db_script_exec+0x18f/frame 0xfffffe085cfe8d90 db_command() at db_command+0x252/frame 0xfffffe085cfe8e60 db_command_loop() at db_command_loop+0x64/frame 0xfffffe085cfe8e70 db_trap() at db_trap+0xef/frame 0xfffffe085cfe8f00 kdb_trap() at kdb_trap+0x191/frame 0xfffffe085cfe8fa0 trap() at trap+0x4df/frame 0xfffffe085cfe91b0 calltrap() at calltrap+0x8/frame 0xfffffe085cfe91b0 --- trap 0x3, rip = 0xffffffff80a9292e, rsp = 0xfffffe085cfe9280, rbp = 0xfffffe085cfe9290 --- kdb_enter() at kdb_enter+0x3e/frame 0xfffffe085cfe9290 vpanic() at vpanic+0x1bf/frame 0xfffffe085cfe9310 kassert_panic() at kassert_panic+0x139/frame 0xfffffe085cfe9380 pmap_delayed_invl_finished() at pmap_delayed_invl_finished+0x55/frame 0xfffffe085cfe93b0 reclaim_pv_chunk_leave_pmap() at reclaim_pv_chunk_leave_pmap+0x70/frame 0xfffffe085cfe93e0 reclaim_pv_chunk() at reclaim_pv_chunk+0x1281/frame 0xfffffe085cfe9520 get_pv_entry() at get_pv_entry+0x326/frame 0xfffffe085cfe9580 pmap_enter() at pmap_enter+0x8f6/frame 0xfffffe085cfe9680 vm_fault_hold() at vm_fault_hold+0x21d/frame 0xfffffe085cfe97e0 vm_fault() at vm_fault+0x77/frame 0xfffffe085cfe9820 trap_pfault() at trap_pfault+0x11a/frame 0xfffffe085cfe98a0 trap() at trap+0x58f/frame 0xfffffe085cfe9ab0 calltrap() at calltrap+0x8/frame 0xfffffe085cfe9ab0 --- trap 0xc, rip = 0x4012c4, rsp = 0x7fffffffe6b0, rbp = 0x7fffffffe820 --- Tracing command sshd pid 690 tid 100134 td 0xfffff8000dfa7560 sched_switch() at sched_switch+0x486/frame 0xfffffe085cee0560 mi_switch() at mi_switch+0x195/frame 0xfffffe085cee05a0 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cee05e0 sleepq_catch_signals() at sleepq_catch_signals+0x466/frame 0xfffffe085cee0650 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe085cee0680 _cv_wait_sig() at _cv_wait_sig+0x1da/frame 0xfffffe085cee06f0 seltdwait() at seltdwait+0x8b/frame 0xfffffe085cee0730 kern_select() at kern_select+0x94f/frame 0xfffffe085cee0940 sys_select() at sys_select+0x54/frame 0xfffffe085cee0980 amd64_syscall() at amd64_syscall+0x2eb/frame 0xfffffe085cee0ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe085cee0ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8021f84aa, rsp = 0x7fffffffe698, rbp = 0x7fffffffe6d0 --- Tracing command rpc.lockd pid 617 tid 100147 td 0xfffff80012234560 sched_switch() at sched_switch+0x486/frame 0xfffffe085cf21420 mi_switch() at mi_switch+0x195/frame 0xfffffe085cf21460 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cf214a0 sleepq_catch_signals() at sleepq_catch_signals+0x466/frame 0xfffffe085cf21510 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe085cf21540 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x1f5/frame 0xfffffe085cf215c0 svc_run_internal() at svc_run_internal+0x400/frame 0xfffffe085cf21750 svc_run() at svc_run+0x210/frame 0xfffffe085cf217d0 sys_nlm_syscall() at sys_nlm_syscall+0x764/frame 0xfffffe085cf21980 amd64_syscall() at amd64_syscall+0x2eb/frame 0xfffffe085cf21ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe085cf21ab0 --- syscall (154, FreeBSD ELF64, sys_nlm_syscall), rip = 0x800a889aa, rsp = 0x7fffffffec28, rbp = 0x7fffffffece0 --- Tracing command rpc.statd pid 613 tid 100140 td 0xfffff80012135560 sched_switch() at sched_switch+0x486/frame 0xfffffe085cefe550 mi_switch() at mi_switch+0x195/frame 0xfffffe085cefe590 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cefe5d0 sleepq_catch_signals() at sleepq_catch_signals+0x466/frame 0xfffffe085cefe640 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe085cefe670 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x1f5/frame 0xfffffe085cefe6f0 seltdwait() at seltdwait+0x81/frame 0xfffffe085cefe730 kern_select() at kern_select+0x94f/frame 0xfffffe085cefe940 sys_select() at sys_select+0x54/frame 0xfffffe085cefe980 amd64_syscall() at amd64_syscall+0x2eb/frame 0xfffffe085cefeab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe085cefeab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800bbb4aa, rsp = 0x7fffffffeab8, rbp = 0x7fffffffec00 --- Tracing command casperd pid 588 tid 100128 td 0xfffff80012467000 sched_switch() at sched_switch+0x486/frame 0xfffffe085cec2560 mi_switch() at mi_switch+0x195/frame 0xfffffe085cec25a0 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cec25e0 sleepq_catch_signals() at sleepq_catch_signals+0x466/frame 0xfffffe085cec2650 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe085cec2680 _cv_wait_sig() at _cv_wait_sig+0x1da/frame 0xfffffe085cec26f0 seltdwait() at seltdwait+0x8b/frame 0xfffffe085cec2730 kern_select() at kern_select+0x94f/frame 0xfffffe085cec2940 sys_select() at sys_select+0x54/frame 0xfffffe085cec2980 amd64_syscall() at amd64_syscall+0x2eb/frame 0xfffffe085cec2ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe085cec2ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8011e44aa, rsp = 0x7fffffffe808, rbp = 0x7fffffffecf0 --- Tracing command casperd pid 587 tid 100118 td 0xfffff80012061000 sched_switch() at sched_switch+0x486/frame 0xfffffe085ce90560 mi_switch() at mi_switch+0x195/frame 0xfffffe085ce905a0 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085ce905e0 sleepq_catch_signals() at sleepq_catch_signals+0x466/frame 0xfffffe085ce90650 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe085ce90680 _cv_wait_sig() at _cv_wait_sig+0x1da/frame 0xfffffe085ce906f0 seltdwait() at seltdwait+0x8b/frame 0xfffffe085ce90730 kern_select() at kern_select+0x94f/frame 0xfffffe085ce90940 sys_select() at sys_select+0x54/frame 0xfffffe085ce90980 amd64_syscall() at amd64_syscall+0x2eb/frame 0xfffffe085ce90ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe085ce90ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8011e44aa, rsp = 0x7fffffffe668, rbp = 0x7fffffffe730 --- Tracing command rpcbind pid 556 tid 100136 td 0xfffff80012235000 sched_switch() at sched_switch+0x486/frame 0xfffffe085ceea5e0 mi_switch() at mi_switch+0x195/frame 0xfffffe085ceea620 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085ceea660 sleepq_catch_signals() at sleepq_catch_signals+0x466/frame 0xfffffe085ceea6d0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe085ceea700 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x1f5/frame 0xfffffe085ceea780 seltdwait() at seltdwait+0x81/frame 0xfffffe085ceea7c0 kern_poll() at kern_poll+0x440/frame 0xfffffe085ceea960 sys_poll() at sys_poll+0x61/frame 0xfffffe085ceea980 amd64_syscall() at amd64_syscall+0x2eb/frame 0xfffffe085ceeaab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe085ceeaab0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x800b6031a, rsp = 0x7fffffffca68, rbp = 0x7fffffffec70 --- Tracing command syslogd pid 551 tid 100139 td 0xfffff80012545000 sched_switch() at sched_switch+0x486/frame 0xfffffe085cef9560 mi_switch() at mi_switch+0x195/frame 0xfffffe085cef95a0 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cef95e0 sleepq_catch_signals() at sleepq_catch_signals+0x466/frame 0xfffffe085cef9650 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe085cef9680 _cv_wait_sig() at _cv_wait_sig+0x1da/frame 0xfffffe085cef96f0 seltdwait() at seltdwait+0x8b/frame 0xfffffe085cef9730 kern_select() at kern_select+0x94f/frame 0xfffffe085cef9940 sys_select() at sys_select+0x54/frame 0xfffffe085cef9980 amd64_syscall() at amd64_syscall+0x2eb/frame 0xfffffe085cef9ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe085cef9ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800bcb4aa, rsp = 0x7fffffffe6c8, rbp = 0x7fffffffece0 --- Tracing command devd pid 464 tid 100119 td 0xfffff8000f7e8000 sched_switch() at sched_switch+0x486/frame 0xfffffe085ce95550 mi_switch() at mi_switch+0x195/frame 0xfffffe085ce95590 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085ce955d0 sleepq_catch_signals() at sleepq_catch_signals+0x466/frame 0xfffffe085ce95640 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe085ce95670 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x1f5/frame 0xfffffe085ce956f0 seltdwait() at seltdwait+0x81/frame 0xfffffe085ce95730 kern_select() at kern_select+0x94f/frame 0xfffffe085ce95940 sys_select() at sys_select+0x54/frame 0xfffffe085ce95980 amd64_syscall() at amd64_syscall+0x2eb/frame 0xfffffe085ce95ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe085ce95ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x468a8a, rsp = 0x7fffffffcb68, rbp = 0x7fffffffed00 --- Tracing command newnfs 0 pid 33 tid 100114 td 0xfffff8000dfba000 sched_switch() at sched_switch+0x486/frame 0xfffffe085ce7c870 mi_switch() at mi_switch+0x195/frame 0xfffffe085ce7c8b0 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085ce7c8f0 sleepq_catch_signals() at sleepq_catch_signals+0x466/frame 0xfffffe085ce7c960 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe085ce7c990 _sleep() at _sleep+0x2e0/frame 0xfffffe085ce7ca30 nfssvc_iod() at nfssvc_iod+0x101/frame 0xfffffe085ce7ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe085ce7cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085ce7cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command vnlru pid 23 tid 100110 td 0xfffff8000f7e8560 sched_switch() at sched_switch+0x486/frame 0xfffffe085cd92890 mi_switch() at mi_switch+0x195/frame 0xfffffe085cd928d0 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cd92910 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe085cd92940 _sleep() at _sleep+0x2f2/frame 0xfffffe085cd929e0 vnlru_proc() at vnlru_proc+0x184/frame 0xfffffe085cd92a70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cd92ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cd92ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command syncer pid 22 tid 100108 td 0xfffff8000f7e9000 sched_switch() at sched_switch+0x486/frame 0xfffffe085cd888b0 mi_switch() at mi_switch+0x195/frame 0xfffffe085cd888f0 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cd88930 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe085cd88960 _cv_timedwait_sbt() at _cv_timedwait_sbt+0x1f5/frame 0xfffffe085cd889e0 sched_sync() at sched_sync+0x5ee/frame 0xfffffe085cd88a70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cd88ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cd88ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufspacedaemon pid 21 tid 100107 td 0xfffff8000f7e9560 sched_switch() at sched_switch+0x486/frame 0xfffffe085cd7e900 mi_switch() at mi_switch+0x195/frame 0xfffffe085cd7e940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cd7e980 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe085cd7e9b0 _sleep() at _sleep+0x2f2/frame 0xfffffe085cd7ea50 bufspace_daemon() at bufspace_daemon+0x1b7/frame 0xfffffe085cd7ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cd7eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cd7eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 20 tid 100105 td 0xfffff8000f7eb000 sched_switch() at sched_switch+0x486/frame 0xfffffe085cd798f0 mi_switch() at mi_switch+0x195/frame 0xfffffe085cd79930 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cd79970 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe085cd799a0 _sleep() at _sleep+0x2f2/frame 0xfffffe085cd79a40 buf_daemon() at buf_daemon+0x68/frame 0xfffffe085cd79a70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cd79ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cd79ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 20 tid 100126 td 0xfffff80007824000 sched_switch() at sched_switch+0x486/frame 0xfffffe085ceb88f0 mi_switch() at mi_switch+0x195/frame 0xfffffe085ceb8930 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085ceb8970 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe085ceb89a0 _sleep() at _sleep+0x2f2/frame 0xfffffe085ceb8a40 softdep_flush() at softdep_flush+0x182/frame 0xfffffe085ceb8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe085ceb8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085ceb8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 20 tid 100127 td 0xfffff80007823560 sched_switch() at sched_switch+0x486/frame 0xfffffe085cebd8f0 mi_switch() at mi_switch+0x195/frame 0xfffffe085cebd930 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cebd970 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe085cebd9a0 _sleep() at _sleep+0x2f2/frame 0xfffffe085cebda40 softdep_flush() at softdep_flush+0x182/frame 0xfffffe085cebda70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cebdab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cebdab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command vmdaemon pid 19 tid 100104 td 0xfffff8000dfbd560 sched_switch() at sched_switch+0x486/frame 0xfffffe085cd748b0 mi_switch() at mi_switch+0x195/frame 0xfffffe085cd748f0 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cd74930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085cd74960 _sleep() at _sleep+0x30b/frame 0xfffffe085cd74a00 vm_daemon() at vm_daemon+0x96/frame 0xfffffe085cd74a70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cd74ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cd74ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 18 tid 100103 td 0xfffff8000f089000 sched_switch() at sched_switch+0x486/frame 0xfffffe085cd6f850 mi_switch() at mi_switch+0x195/frame 0xfffffe085cd6f890 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cd6f8d0 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe085cd6f900 _sleep() at _sleep+0x2f2/frame 0xfffffe085cd6f9a0 pause_sbt() at pause_sbt+0xac/frame 0xfffffe085cd6f9f0 vm_pageout() at vm_pageout+0x23e/frame 0xfffffe085cd6fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cd6fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cd6fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 18 tid 100106 td 0xfffff8000f7ed560 sched_switch() at sched_switch+0x486/frame 0xfffffe085cd83420 mi_switch() at mi_switch+0x195/frame 0xfffffe085cd83460 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cd834a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085cd834d0 _sleep() at _sleep+0x30b/frame 0xfffffe085cd83570 getpbuf() at getpbuf+0x98/frame 0xfffffe085cd835a0 swap_pager_putpages() at swap_pager_putpages+0x18c/frame 0xfffffe085cd83610 vm_pageout_flush() at vm_pageout_flush+0x12f/frame 0xfffffe085cd83700 vm_pageout_cluster() at vm_pageout_cluster+0x3ca/frame 0xfffffe085cd83980 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0x9e7/frame 0xfffffe085cd83a70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cd83ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cd83ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 18 tid 100109 td 0xfffff8000f7ed000 sched_switch() at sched_switch+0x486/frame 0xfffffe085cd8d8f0 mi_switch() at mi_switch+0x195/frame 0xfffffe085cd8d930 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cd8d970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085cd8d9a0 _sleep() at _sleep+0x30b/frame 0xfffffe085cd8da40 uma_reclaim_worker() at uma_reclaim_worker+0x80/frame 0xfffffe085cd8da70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cd8dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cd8dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon1 pid 17 tid 100102 td 0xfffff8000dfba560 sched_switch() at sched_switch+0x486/frame 0xfffffe085cd6a8b0 mi_switch() at mi_switch+0x195/frame 0xfffffe085cd6a8f0 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cd6a930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085cd6a960 _sleep() at _sleep+0x30b/frame 0xfffffe085cd6aa00 enc_daemon() at enc_daemon+0x125/frame 0xfffffe085cd6aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cd6aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cd6aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon0 pid 16 tid 100101 td 0xfffff8000dfbb000 sched_switch() at sched_switch+0x486/frame 0xfffffe085cd658b0 mi_switch() at mi_switch+0x195/frame 0xfffffe085cd658f0 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cd65930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085cd65960 _sleep() at _sleep+0x30b/frame 0xfffffe085cd65a00 enc_daemon() at enc_daemon+0x125/frame 0xfffffe085cd65a70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cd65ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cd65ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command rand_harvestq pid 15 tid 100098 td 0xfffff8000f091560 sched_switch() at sched_switch+0x486/frame 0xfffffe085cd568d0 mi_switch() at mi_switch+0x195/frame 0xfffffe085cd56910 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cd56950 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe085cd56980 _sleep() at _sleep+0x2f2/frame 0xfffffe085cd56a20 random_kthread() at random_kthread+0x236/frame 0xfffffe085cd56a70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cd56ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cd56ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sctp_iterator pid 9 tid 100094 td 0xfffff8000f093560 sched_switch() at sched_switch+0x486/frame 0xfffffe085cd42900 mi_switch() at mi_switch+0x195/frame 0xfffffe085cd42940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cd42980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085cd429b0 _sleep() at _sleep+0x30b/frame 0xfffffe085cd42a50 sctp_iterator_thread() at sctp_iterator_thread+0x59/frame 0xfffffe085cd42a70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cd42ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cd42ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod4 pid 8 tid 100093 td 0xfffff8000dfbb560 sched_switch() at sched_switch+0x486/frame 0xfffffe085cd3c8e0 mi_switch() at mi_switch+0x195/frame 0xfffffe085cd3c920 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cd3c960 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe085cd3c990 _sleep() at _sleep+0x2f2/frame 0xfffffe085cd3ca30 soaio_kproc_loop() at soaio_kproc_loop+0x141/frame 0xfffffe085cd3ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cd3cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cd3cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod3 pid 7 tid 100092 td 0xfffff8000dfbc000 sched_switch() at sched_switch+0x486/frame 0xfffffe085cd378e0 mi_switch() at mi_switch+0x195/frame 0xfffffe085cd37920 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cd37960 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe085cd37990 _sleep() at _sleep+0x2f2/frame 0xfffffe085cd37a30 soaio_kproc_loop() at soaio_kproc_loop+0x141/frame 0xfffffe085cd37a70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cd37ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cd37ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod2 pid 6 tid 100091 td 0xfffff8000dfbc560 sched_switch() at sched_switch+0x486/frame 0xfffffe085cd328e0 mi_switch() at mi_switch+0x195/frame 0xfffffe085cd32920 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cd32960 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe085cd32990 _sleep() at _sleep+0x2f2/frame 0xfffffe085cd32a30 soaio_kproc_loop() at soaio_kproc_loop+0x141/frame 0xfffffe085cd32a70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cd32ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cd32ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod1 pid 5 tid 100090 td 0xfffff8000dfbd000 sched_switch() at sched_switch+0x486/frame 0xfffffe085cd2d8e0 mi_switch() at mi_switch+0x195/frame 0xfffffe085cd2d920 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cd2d960 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe085cd2d990 _sleep() at _sleep+0x2f2/frame 0xfffffe085cd2da30 soaio_kproc_loop() at soaio_kproc_loop+0x141/frame 0xfffffe085cd2da70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cd2dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cd2dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 14 tid 100071 td 0xfffff80007da9560 sched_switch() at sched_switch+0x486/frame 0xfffffe085c8ec920 mi_switch() at mi_switch+0x195/frame 0xfffffe085c8ec960 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085c8ec9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085c8ec9d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe085c8eca40 usb_process() at usb_process+0xd4/frame 0xfffffe085c8eca70 fork_exit() at fork_exit+0x84/frame 0xfffffe085c8ecab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085c8ecab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 14 tid 100072 td 0xfffff80007da9000 sched_switch() at sched_switch+0x486/frame 0xfffffe085c8f1920 mi_switch() at mi_switch+0x195/frame 0xfffffe085c8f1960 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085c8f19a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085c8f19d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe085c8f1a40 usb_process() at usb_process+0xd4/frame 0xfffffe085c8f1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe085c8f1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085c8f1ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 14 tid 100073 td 0xfffff80007da8560 sched_switch() at sched_switch+0x486/frame 0xfffffe085c8f6920 mi_switch() at mi_switch+0x195/frame 0xfffffe085c8f6960 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085c8f69a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085c8f69d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe085c8f6a40 usb_process() at usb_process+0xd4/frame 0xfffffe085c8f6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe085c8f6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085c8f6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 14 tid 100074 td 0xfffff80007da8000 sched_switch() at sched_switch+0x486/frame 0xfffffe085c8fb920 mi_switch() at mi_switch+0x195/frame 0xfffffe085c8fb960 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085c8fb9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085c8fb9d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe085c8fba40 usb_process() at usb_process+0xd4/frame 0xfffffe085c8fba70 fork_exit() at fork_exit+0x84/frame 0xfffffe085c8fbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085c8fbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 14 tid 100075 td 0xfffff80007da0560 sched_switch() at sched_switch+0x486/frame 0xfffffe085c900920 mi_switch() at mi_switch+0x195/frame 0xfffffe085c900960 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085c9009a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085c9009d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe085c900a40 usb_process() at usb_process+0xd4/frame 0xfffffe085c900a70 fork_exit() at fork_exit+0x84/frame 0xfffffe085c900ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085c900ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 14 tid 100077 td 0xfffff800078a9560 sched_switch() at sched_switch+0x486/frame 0xfffffe085ca8d920 mi_switch() at mi_switch+0x195/frame 0xfffffe085ca8d960 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085ca8d9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085ca8d9d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe085ca8da40 usb_process() at usb_process+0xd4/frame 0xfffffe085ca8da70 fork_exit() at fork_exit+0x84/frame 0xfffffe085ca8dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085ca8dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 14 tid 100078 td 0xfffff80008133000 sched_switch() at sched_switch+0x486/frame 0xfffffe085ca92920 mi_switch() at mi_switch+0x195/frame 0xfffffe085ca92960 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085ca929a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085ca929d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe085ca92a40 usb_process() at usb_process+0xd4/frame 0xfffffe085ca92a70 fork_exit() at fork_exit+0x84/frame 0xfffffe085ca92ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085ca92ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 14 tid 100079 td 0xfffff80008132560 sched_switch() at sched_switch+0x486/frame 0xfffffe085ca97920 mi_switch() at mi_switch+0x195/frame 0xfffffe085ca97960 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085ca979a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085ca979d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe085ca97a40 usb_process() at usb_process+0xd4/frame 0xfffffe085ca97a70 fork_exit() at fork_exit+0x84/frame 0xfffffe085ca97ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085ca97ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 14 tid 100080 td 0xfffff80008132000 sched_switch() at sched_switch+0x486/frame 0xfffffe085ca9c920 mi_switch() at mi_switch+0x195/frame 0xfffffe085ca9c960 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085ca9c9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085ca9c9d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe085ca9ca40 usb_process() at usb_process+0xd4/frame 0xfffffe085ca9ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe085ca9cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085ca9cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 14 tid 100081 td 0xfffff80008131560 sched_switch() at sched_switch+0x486/frame 0xfffffe085caa1920 mi_switch() at mi_switch+0x195/frame 0xfffffe085caa1960 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085caa19a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085caa19d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe085caa1a40 usb_process() at usb_process+0xd4/frame 0xfffffe085caa1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe085caa1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085caa1ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 14 tid 100082 td 0xfffff80008131000 sched_switch() at sched_switch+0x486/frame 0xfffffe085cc89920 mi_switch() at mi_switch+0x195/frame 0xfffffe085cc89960 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cc899a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085cc899d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe085cc89a40 usb_process() at usb_process+0xd4/frame 0xfffffe085cc89a70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cc89ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cc89ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 14 tid 100083 td 0xfffff80008130560 sched_switch() at sched_switch+0x486/frame 0xfffffe085cc8e920 mi_switch() at mi_switch+0x195/frame 0xfffffe085cc8e960 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cc8e9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085cc8e9d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe085cc8ea40 usb_process() at usb_process+0xd4/frame 0xfffffe085cc8ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cc8eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cc8eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 14 tid 100084 td 0xfffff80008130000 sched_switch() at sched_switch+0x486/frame 0xfffffe085cc93920 mi_switch() at mi_switch+0x195/frame 0xfffffe085cc93960 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cc939a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085cc939d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe085cc93a40 usb_process() at usb_process+0xd4/frame 0xfffffe085cc93a70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cc93ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cc93ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 14 tid 100085 td 0xfffff8000812f560 sched_switch() at sched_switch+0x486/frame 0xfffffe085cc98920 mi_switch() at mi_switch+0x195/frame 0xfffffe085cc98960 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cc989a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085cc989d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe085cc98a40 usb_process() at usb_process+0xd4/frame 0xfffffe085cc98a70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cc98ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cc98ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 14 tid 100086 td 0xfffff8000812f000 sched_switch() at sched_switch+0x486/frame 0xfffffe085cc9d920 mi_switch() at mi_switch+0x195/frame 0xfffffe085cc9d960 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cc9d9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085cc9d9d0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe085cc9da40 usb_process() at usb_process+0xd4/frame 0xfffffe085cc9da70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cc9dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cc9dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 4 tid 100066 td 0xfffff800078a5560 sched_switch() at sched_switch+0x486/frame 0xfffffe00002878d0 mi_switch() at mi_switch+0x195/frame 0xfffffe0000287910 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe0000287950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0000287980 _sleep() at _sleep+0x30b/frame 0xfffffe0000287a20 xpt_done_td() at xpt_done_td+0xa2/frame 0xfffffe0000287a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000287ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000287ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 4 tid 100067 td 0xfffff800078a5000 sched_switch() at sched_switch+0x486/frame 0xfffffe000028c8d0 mi_switch() at mi_switch+0x195/frame 0xfffffe000028c910 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe000028c950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000028c980 _sleep() at _sleep+0x30b/frame 0xfffffe000028ca20 xpt_done_td() at xpt_done_td+0xa2/frame 0xfffffe000028ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe000028cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000028cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 4 tid 100068 td 0xfffff800078a4560 sched_switch() at sched_switch+0x486/frame 0xfffffe00002918d0 mi_switch() at mi_switch+0x195/frame 0xfffffe0000291910 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe0000291950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0000291980 _sleep() at _sleep+0x30b/frame 0xfffffe0000291a20 xpt_done_td() at xpt_done_td+0xa2/frame 0xfffffe0000291a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000291ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000291ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 4 tid 100100 td 0xfffff8000f089560 sched_switch() at sched_switch+0x486/frame 0xfffffe085cd608c0 mi_switch() at mi_switch+0x195/frame 0xfffffe085cd60900 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cd60940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085cd60970 _sleep() at _sleep+0x30b/frame 0xfffffe085cd60a10 xpt_scanner_thread() at xpt_scanner_thread+0x5c/frame 0xfffffe085cd60a70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cd60ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cd60ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns pid 3 tid 100065 td 0xfffff800078a9000 sched_switch() at sched_switch+0x486/frame 0xfffffe00002828e0 mi_switch() at mi_switch+0x195/frame 0xfffffe0000282920 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe0000282960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0000282990 _sleep() at _sleep+0x30b/frame 0xfffffe0000282a30 crypto_ret_proc() at crypto_ret_proc+0x1b8/frame 0xfffffe0000282a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000282ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000282ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 2 tid 100064 td 0xfffff80007835560 sched_switch() at sched_switch+0x486/frame 0xfffffe000027d8e0 mi_switch() at mi_switch+0x195/frame 0xfffffe000027d920 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe000027d960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000027d990 _sleep() at _sleep+0x30b/frame 0xfffffe000027da30 crypto_proc() at crypto_proc+0x38d/frame 0xfffffe000027da70 fork_exit() at fork_exit+0x84/frame 0xfffffe000027dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000027dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100060 td 0xfffff80007839560 sched_switch() at sched_switch+0x486/frame 0xfffffe00002698e0 mi_switch() at mi_switch+0x195/frame 0xfffffe0000269920 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe0000269960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0000269990 _sleep() at _sleep+0x30b/frame 0xfffffe0000269a30 g_run_events() at g_run_events+0x4e7/frame 0xfffffe0000269a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000269ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000269ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100061 td 0xfffff80007839000 sched_switch() at sched_switch+0x486/frame 0xfffffe000026e8e0 mi_switch() at mi_switch+0x195/frame 0xfffffe000026e920 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe000026e960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000026e990 _sleep() at _sleep+0x30b/frame 0xfffffe000026ea30 g_io_schedule_up() at g_io_schedule_up+0xa3/frame 0xfffffe000026ea60 g_up_procbody() at g_up_procbody+0x6d/frame 0xfffffe000026ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe000026eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000026eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100062 td 0xfffff80007838560 sched_switch() at sched_switch+0x486/frame 0xfffffe00002738d0 mi_switch() at mi_switch+0x195/frame 0xfffffe0000273910 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe0000273950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0000273980 _sleep() at _sleep+0x30b/frame 0xfffffe0000273a20 g_io_schedule_down() at g_io_schedule_down+0x4c/frame 0xfffffe0000273a60 g_down_procbody() at g_down_procbody+0x6d/frame 0xfffffe0000273a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000273ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000273ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100016 td 0xfffff8000773e560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100019 td 0xfffff8000777f000 sched_switch() at sched_switch+0x486/frame 0xfffffe08391359e0 mi_switch() at mi_switch+0x195/frame 0xfffffe0839135a20 ithread_loop() at ithread_loop+0x1ce/frame 0xfffffe0839135a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0839135ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0839135ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100020 td 0xfffff8000777e560 sched_switch() at sched_switch+0x486/frame 0xfffffe083913a9e0 mi_switch() at mi_switch+0x195/frame 0xfffffe083913aa20 ithread_loop() at ithread_loop+0x1ce/frame 0xfffffe083913aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe083913aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe083913aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100046 td 0xfffff8000777f560 sched_switch() at sched_switch+0x486/frame 0xfffffe08391bc9e0 mi_switch() at mi_switch+0x195/frame 0xfffffe08391bca20 ithread_loop() at ithread_loop+0x1ce/frame 0xfffffe08391bca70 fork_exit() at fork_exit+0x84/frame 0xfffffe08391bcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe08391bcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100047 td 0xfffff80007835000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100048 td 0xfffff80007834560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100049 td 0xfffff80007834000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100050 td 0xfffff80007833560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100051 td 0xfffff80007833000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100052 td 0xfffff8000782e560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100053 td 0xfffff8000782e000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100054 td 0xfffff8000782d560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100055 td 0xfffff8000782d000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100056 td 0xfffff8000783b560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100057 td 0xfffff8000783b000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100058 td 0xfffff8000783a560 sched_switch() at sched_switch+0x486/frame 0xfffffe08391f89e0 mi_switch() at mi_switch+0x195/frame 0xfffffe08391f8a20 ithread_loop() at ithread_loop+0x1ce/frame 0xfffffe08391f8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe08391f8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe08391f8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100059 td 0xfffff8000783a000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100069 td 0xfffff80007daa560 sched_switch() at sched_switch+0x486/frame 0xfffffe00002969e0 mi_switch() at mi_switch+0x195/frame 0xfffffe0000296a20 ithread_loop() at ithread_loop+0x1ce/frame 0xfffffe0000296a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000296ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000296ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100070 td 0xfffff80007daa000 sched_switch() at sched_switch+0x486/frame 0xfffffe085c8e79e0 mi_switch() at mi_switch+0x195/frame 0xfffffe085c8e7a20 ithread_loop() at ithread_loop+0x1ce/frame 0xfffffe085c8e7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe085c8e7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085c8e7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100076 td 0xfffff80007da0000 sched_switch() at sched_switch+0x486/frame 0xfffffe085ca889e0 mi_switch() at mi_switch+0x195/frame 0xfffffe085ca88a20 ithread_loop() at ithread_loop+0x1ce/frame 0xfffffe085ca88a70 fork_exit() at fork_exit+0x84/frame 0xfffffe085ca88ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085ca88ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100087 td 0xfffff800081b2560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100088 td 0xfffff800081b2000 sched_switch() at sched_switch+0x486/frame 0xfffffe085cd1f9e0 mi_switch() at mi_switch+0x195/frame 0xfffffe085cd1fa20 ithread_loop() at ithread_loop+0x1ce/frame 0xfffffe085cd1fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cd1fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cd1fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100003 td 0xfffff8000772d000 (CPU 0) cpustop_handler() at cpustop_handler+0x28/frame 0xffffffff81dcc7f0 ipi_nmi_handler() at ipi_nmi_handler+0x4a/frame 0xffffffff81dcc810 trap() at trap+0x49/frame 0xffffffff81dcca20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff81dcca20 --- trap 0x13, rip = 0xffffffff8104339b, rsp = 0xfffffe07c15b6960, rbp = 0xfffffe07c15b6970 --- cpu_idle() at cpu_idle+0xbb/frame 0xfffffe07c15b6970 sched_idletd() at sched_idletd+0x4f0/frame 0xfffffe07c15b6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe07c15b6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe07c15b6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xfffff8000772c560 sched_switch() at sched_switch+0x486/frame 0xfffffe07c15bb910 mi_switch() at mi_switch+0x195/frame 0xfffffe07c15bb950 critical_exit() at critical_exit+0x8b/frame 0xfffffe07c15bb970 sched_idletd() at sched_idletd+0x4f0/frame 0xfffffe07c15bba70 fork_exit() at fork_exit+0x84/frame 0xfffffe07c15bbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe07c15bbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffff8000772c000 sched_switch() at sched_switch+0x486/frame 0xfffffe07c15c0930 mi_switch() at mi_switch+0x195/frame 0xfffffe07c15c0970 sched_idletd() at sched_idletd+0x10c/frame 0xfffffe07c15c0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe07c15c0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe07c15c0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xfffff8000772b560 sched_switch() at sched_switch+0x486/frame 0xfffffe07c15c5910 mi_switch() at mi_switch+0x195/frame 0xfffffe07c15c5950 critical_exit() at critical_exit+0x8b/frame 0xfffffe07c15c5970 sched_idletd() at sched_idletd+0x4f0/frame 0xfffffe07c15c5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe07c15c5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe07c15c5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100007 td 0xfffff8000772b000 sched_switch() at sched_switch+0x486/frame 0xfffffe07c15ca930 mi_switch() at mi_switch+0x195/frame 0xfffffe07c15ca970 sched_idletd() at sched_idletd+0x10c/frame 0xfffffe07c15caa70 fork_exit() at fork_exit+0x84/frame 0xfffffe07c15caab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe07c15caab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xfffff8000772a560 sched_switch() at sched_switch+0x486/frame 0xfffffe07c15cf910 mi_switch() at mi_switch+0x195/frame 0xfffffe07c15cf950 critical_exit() at critical_exit+0x8b/frame 0xfffffe07c15cf970 sched_idletd() at sched_idletd+0x4f0/frame 0xfffffe07c15cfa70 fork_exit() at fork_exit+0x84/frame 0xfffffe07c15cfab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe07c15cfab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xfffff8000772a000 sched_switch() at sched_switch+0x486/frame 0xfffffe07c15d4910 mi_switch() at mi_switch+0x195/frame 0xfffffe07c15d4950 critical_exit() at critical_exit+0x8b/frame 0xfffffe07c15d4970 sched_idletd() at sched_idletd+0x4f0/frame 0xfffffe07c15d4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe07c15d4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe07c15d4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xfffff80007741560 sched_switch() at sched_switch+0x486/frame 0xfffffe07c15d9930 mi_switch() at mi_switch+0x195/frame 0xfffffe07c15d9970 sched_idletd() at sched_idletd+0x10c/frame 0xfffffe07c15d9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe07c15d9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe07c15d9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffff80007741000 sched_switch() at sched_switch+0x486/frame 0xfffffe07c15de910 mi_switch() at mi_switch+0x195/frame 0xfffffe07c15de950 critical_exit() at critical_exit+0x8b/frame 0xfffffe07c15de970 sched_idletd() at sched_idletd+0x4f0/frame 0xfffffe07c15dea70 fork_exit() at fork_exit+0x84/frame 0xfffffe07c15deab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe07c15deab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100012 td 0xfffff80007740560 sched_switch() at sched_switch+0x486/frame 0xfffffe07c15e3930 mi_switch() at mi_switch+0x195/frame 0xfffffe07c15e3970 sched_idletd() at sched_idletd+0x3d6/frame 0xfffffe07c15e3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe07c15e3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe07c15e3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xfffff80007740000 (CPU 10) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07c159ccf0 ipi_nmi_handler() at ipi_nmi_handler+0x4a/frame 0xfffffe07c159cd10 trap() at trap+0x49/frame 0xfffffe07c159cf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07c159cf20 --- trap 0x13, rip = 0xffffffff8104339b, rsp = 0xfffffe07c15e8960, rbp = 0xfffffe07c15e8970 --- cpu_idle() at cpu_idle+0xbb/frame 0xfffffe07c15e8970 sched_idletd() at sched_idletd+0x4f0/frame 0xfffffe07c15e8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe07c15e8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe07c15e8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100014 td 0xfffff8000773f560 (CPU 11) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07c15a4cf0 ipi_nmi_handler() at ipi_nmi_handler+0x4a/frame 0xfffffe07c15a4d10 trap() at trap+0x49/frame 0xfffffe07c15a4f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07c15a4f20 --- trap 0x13, rip = 0xffffffff81042fd6, rsp = 0xfffffe07c15ed8e0, rbp = 0xfffffe07c15ed8e0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe07c15ed8e0 acpi_cpu_idle() at acpi_cpu_idle+0x2f3/frame 0xfffffe07c15ed930 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe07c15ed950 cpu_idle() at cpu_idle+0x90/frame 0xfffffe07c15ed970 sched_idletd() at sched_idletd+0x4f0/frame 0xfffffe07c15eda70 fork_exit() at fork_exit+0x84/frame 0xfffffe07c15edab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe07c15edab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command init pid 1 tid 100002 td 0xfffff8000772d560 sched_switch() at sched_switch+0x486/frame 0xfffffe07c15b1520 mi_switch() at mi_switch+0x195/frame 0xfffffe07c15b1560 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe07c15b15a0 sleepq_catch_signals() at sleepq_catch_signals+0x466/frame 0xfffffe07c15b1610 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe07c15b1640 _sleep() at _sleep+0x301/frame 0xfffffe07c15b16e0 kern_wait6() at kern_wait6+0x467/frame 0xfffffe07c15b1790 sys_wait4() at sys_wait4+0x7a/frame 0xfffffe07c15b1980 amd64_syscall() at amd64_syscall+0x2eb/frame 0xfffffe07c15b1ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe07c15b1ab0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x4236ca, rsp = 0x7fffffffe788, rbp = 0x7fffffffe7f0 --- Tracing command audit pid 10 tid 100001 td 0xfffff8000772e000 sched_switch() at sched_switch+0x486/frame 0xfffffe07c15ac8f0 mi_switch() at mi_switch+0x195/frame 0xfffffe07c15ac930 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe07c15ac970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe07c15ac9a0 _cv_wait() at _cv_wait+0x1d3/frame 0xfffffe07c15aca10 audit_worker() at audit_worker+0xa3/frame 0xfffffe07c15aca70 fork_exit() at fork_exit+0x84/frame 0xfffffe07c15acab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe07c15acab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xffffffff81e13fe0 sched_switch() at sched_switch+0x486/frame 0xffffffff823b79d0 mi_switch() at mi_switch+0x195/frame 0xffffffff823b7a10 sleepq_switch() at sleepq_switch+0x183/frame 0xffffffff823b7a50 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xffffffff823b7a80 _sleep() at _sleep+0x2f2/frame 0xffffffff823b7b20 swapper() at swapper+0x208/frame 0xffffffff823b7b70 btext() at btext+0x2c Tracing command kernel pid 0 tid 100015 td 0xfffff8000773f000 sched_switch() at sched_switch+0x486/frame 0xfffffe07c15f28f0 mi_switch() at mi_switch+0x195/frame 0xfffffe07c15f2930 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe07c15f2970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe07c15f29a0 _sleep() at _sleep+0x30b/frame 0xfffffe07c15f2a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xd8/frame 0xfffffe07c15f2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe07c15f2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe07c15f2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100017 td 0xfffff8000773e000 sched_switch() at sched_switch+0x486/frame 0xfffffe07c15fc8f0 mi_switch() at mi_switch+0x195/frame 0xfffffe07c15fc930 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe07c15fc970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe07c15fc9a0 _sleep() at _sleep+0x30b/frame 0xfffffe07c15fca40 taskqueue_thread_loop() at taskqueue_thread_loop+0xd8/frame 0xfffffe07c15fca70 fork_exit() at fork_exit+0x84/frame 0xfffffe07c15fcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe07c15fcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100018 td 0xfffff8000772e560 sched_switch() at sched_switch+0x486/frame 0xfffffe08391308f0 mi_switch() at mi_switch+0x195/frame 0xfffffe0839130930 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe0839130970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe08391309a0 _sleep() at _sleep+0x30b/frame 0xfffffe0839130a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xd8/frame 0xfffffe0839130a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0839130ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0839130ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xfffff8000777e000 sched_switch() at sched_switch+0x486/frame 0xfffffe083913f900 mi_switch() at mi_switch+0x195/frame 0xfffffe083913f940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe083913f980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe083913f9b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe083913fa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x9d/frame 0xfffffe083913fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe083913fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe083913fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xfffff8000777d560 sched_switch() at sched_switch+0x486/frame 0xfffffe0839144900 mi_switch() at mi_switch+0x195/frame 0xfffffe0839144940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe0839144980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe08391449b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe0839144a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x9d/frame 0xfffffe0839144a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0839144ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0839144ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xfffff8000777d000 sched_switch() at sched_switch+0x486/frame 0xfffffe0839149900 mi_switch() at mi_switch+0x195/frame 0xfffffe0839149940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe0839149980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe08391499b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe0839149a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x9d/frame 0xfffffe0839149a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0839149ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0839149ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xfffff8000777c560 sched_switch() at sched_switch+0x486/frame 0xfffffe083914e900 mi_switch() at mi_switch+0x195/frame 0xfffffe083914e940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe083914e980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe083914e9b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe083914ea40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x9d/frame 0xfffffe083914ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe083914eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe083914eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xfffff8000777c000 sched_switch() at sched_switch+0x486/frame 0xfffffe0839153900 mi_switch() at mi_switch+0x195/frame 0xfffffe0839153940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe0839153980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe08391539b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe0839153a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x9d/frame 0xfffffe0839153a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0839153ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0839153ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100026 td 0xfffff8000777b560 sched_switch() at sched_switch+0x486/frame 0xfffffe0839158900 mi_switch() at mi_switch+0x195/frame 0xfffffe0839158940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe0839158980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe08391589b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe0839158a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x9d/frame 0xfffffe0839158a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0839158ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0839158ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100027 td 0xfffff8000777b000 sched_switch() at sched_switch+0x486/frame 0xfffffe083915d900 mi_switch() at mi_switch+0x195/frame 0xfffffe083915d940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe083915d980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe083915d9b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe083915da40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x9d/frame 0xfffffe083915da70 fork_exit() at fork_exit+0x84/frame 0xfffffe083915dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe083915dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100028 td 0xfffff80007785560 sched_switch() at sched_switch+0x486/frame 0xfffffe0839162900 mi_switch() at mi_switch+0x195/frame 0xfffffe0839162940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe0839162980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe08391629b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe0839162a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x9d/frame 0xfffffe0839162a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0839162ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0839162ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100029 td 0xfffff80007785000 sched_switch() at sched_switch+0x486/frame 0xfffffe0839167900 mi_switch() at mi_switch+0x195/frame 0xfffffe0839167940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe0839167980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe08391679b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe0839167a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x9d/frame 0xfffffe0839167a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0839167ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0839167ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100030 td 0xfffff80007784560 sched_switch() at sched_switch+0x486/frame 0xfffffe083916c900 mi_switch() at mi_switch+0x195/frame 0xfffffe083916c940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe083916c980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe083916c9b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe083916ca40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x9d/frame 0xfffffe083916ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe083916cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe083916cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100031 td 0xfffff80007784000 sched_switch() at sched_switch+0x486/frame 0xfffffe0839171900 mi_switch() at mi_switch+0x195/frame 0xfffffe0839171940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe0839171980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe08391719b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe0839171a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x9d/frame 0xfffffe0839171a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0839171ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0839171ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100032 td 0xfffff80007783560 sched_switch() at sched_switch+0x486/frame 0xfffffe0839176900 mi_switch() at mi_switch+0x195/frame 0xfffffe0839176940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe0839176980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe08391769b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe0839176a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x9d/frame 0xfffffe0839176a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0839176ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0839176ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100033 td 0xfffff80007783000 sched_switch() at sched_switch+0x486/frame 0xfffffe083917b900 mi_switch() at mi_switch+0x195/frame 0xfffffe083917b940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe083917b980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe083917b9b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe083917ba40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x9d/frame 0xfffffe083917ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe083917bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe083917bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100034 td 0xfffff80007782560 sched_switch() at sched_switch+0x486/frame 0xfffffe0839180900 mi_switch() at mi_switch+0x195/frame 0xfffffe0839180940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe0839180980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe08391809b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe0839180a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x9d/frame 0xfffffe0839180a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0839180ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0839180ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100035 td 0xfffff80007782000 sched_switch() at sched_switch+0x486/frame 0xfffffe0839185900 mi_switch() at mi_switch+0x195/frame 0xfffffe0839185940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe0839185980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe08391859b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe0839185a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x9d/frame 0xfffffe0839185a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0839185ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0839185ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100036 td 0xfffff80007816000 sched_switch() at sched_switch+0x486/frame 0xfffffe083918a900 mi_switch() at mi_switch+0x195/frame 0xfffffe083918a940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe083918a980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe083918a9b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe083918aa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x9d/frame 0xfffffe083918aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe083918aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe083918aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100037 td 0xfffff80007815560 sched_switch() at sched_switch+0x486/frame 0xfffffe083918f900 mi_switch() at mi_switch+0x195/frame 0xfffffe083918f940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe083918f980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe083918f9b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe083918fa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x9d/frame 0xfffffe083918fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe083918fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe083918fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100038 td 0xfffff80007815000 sched_switch() at sched_switch+0x486/frame 0xfffffe0839194900 mi_switch() at mi_switch+0x195/frame 0xfffffe0839194940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe0839194980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe08391949b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe0839194a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x9d/frame 0xfffffe0839194a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0839194ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0839194ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100039 td 0xfffff80007814560 sched_switch() at sched_switch+0x486/frame 0xfffffe0839199900 mi_switch() at mi_switch+0x195/frame 0xfffffe0839199940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe0839199980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe08391999b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe0839199a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x9d/frame 0xfffffe0839199a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0839199ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0839199ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100040 td 0xfffff80007814000 sched_switch() at sched_switch+0x486/frame 0xfffffe083919e900 mi_switch() at mi_switch+0x195/frame 0xfffffe083919e940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe083919e980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe083919e9b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe083919ea40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x9d/frame 0xfffffe083919ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe083919eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe083919eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100041 td 0xfffff80007813560 sched_switch() at sched_switch+0x486/frame 0xfffffe08391a3900 mi_switch() at mi_switch+0x195/frame 0xfffffe08391a3940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe08391a3980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe08391a39b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe08391a3a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x9d/frame 0xfffffe08391a3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe08391a3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe08391a3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100042 td 0xfffff80007813000 sched_switch() at sched_switch+0x486/frame 0xfffffe08391a8900 mi_switch() at mi_switch+0x195/frame 0xfffffe08391a8940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe08391a8980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe08391a89b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe08391a8a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x9d/frame 0xfffffe08391a8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe08391a8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe08391a8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100043 td 0xfffff80007807560 sched_switch() at sched_switch+0x486/frame 0xfffffe08391ad900 mi_switch() at mi_switch+0x195/frame 0xfffffe08391ad940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe08391ad980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe08391ad9b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe08391ada40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x9d/frame 0xfffffe08391ada70 fork_exit() at fork_exit+0x84/frame 0xfffffe08391adab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe08391adab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100044 td 0xfffff80007807000 sched_switch() at sched_switch+0x486/frame 0xfffffe08391b2900 mi_switch() at mi_switch+0x195/frame 0xfffffe08391b2940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe08391b2980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe08391b29b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe08391b2a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x9d/frame 0xfffffe08391b2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe08391b2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe08391b2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100045 td 0xfffff80007824560 sched_switch() at sched_switch+0x486/frame 0xfffffe08391b7900 mi_switch() at mi_switch+0x195/frame 0xfffffe08391b7940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe08391b7980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe08391b79b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe08391b7a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x9d/frame 0xfffffe08391b7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe08391b7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe08391b7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100063 td 0xfffff80007838000 sched_switch() at sched_switch+0x486/frame 0xfffffe00002788f0 mi_switch() at mi_switch+0x195/frame 0xfffffe0000278930 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe0000278970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00002789a0 _sleep() at _sleep+0x30b/frame 0xfffffe0000278a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xd8/frame 0xfffffe0000278a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000278ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000278ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100089 td 0xfffff800081b1560 sched_switch() at sched_switch+0x486/frame 0xfffffe085cd28900 mi_switch() at mi_switch+0x195/frame 0xfffffe085cd28940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cd28980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085cd289b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe085cd28a40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9d/frame 0xfffffe085cd28a70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cd28ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cd28ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100095 td 0xfffff8000f093000 sched_switch() at sched_switch+0x486/frame 0xfffffe085cd47900 mi_switch() at mi_switch+0x195/frame 0xfffffe085cd47940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cd47980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085cd479b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe085cd47a40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9d/frame 0xfffffe085cd47a70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cd47ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cd47ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100096 td 0xfffff8000f092560 sched_switch() at sched_switch+0x486/frame 0xfffffe085cd4c900 mi_switch() at mi_switch+0x195/frame 0xfffffe085cd4c940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cd4c980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085cd4c9b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe085cd4ca40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9d/frame 0xfffffe085cd4ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cd4cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cd4cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100097 td 0xfffff8000f092000 sched_switch() at sched_switch+0x486/frame 0xfffffe085cd51900 mi_switch() at mi_switch+0x195/frame 0xfffffe085cd51940 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cd51980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085cd519b0 msleep_spin_sbt() at msleep_spin_sbt+0x1e7/frame 0xfffffe085cd51a40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9d/frame 0xfffffe085cd51a70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cd51ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cd51ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100099 td 0xfffff8000f091000 sched_switch() at sched_switch+0x486/frame 0xfffffe085cd5b8f0 mi_switch() at mi_switch+0x195/frame 0xfffffe085cd5b930 sleepq_switch() at sleepq_switch+0x183/frame 0xfffffe085cd5b970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe085cd5b9a0 _sleep() at _sleep+0x30b/frame 0xfffffe085cd5ba40 taskqueue_thread_loop() at taskqueue_thread_loop+0xd8/frame 0xfffffe085cd5ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe085cd5bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085cd5bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db:0:allt> dump Dumping 1704 out of 32627 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Dump complete db:0:dump> reset cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 8 (kgdb) bt #0 doadump (textdump=0x0) at pcpu.h:232 #1 0xffffffff803a677e in db_dump (dummy=, dummy2=0x0, dummy3=0x0, dummy4=0x0) at ../../../ddb/db_command.c:572 #2 0xffffffff803a6112 in db_command (cmd_table=0x0) at ../../../ddb/db_command.c:479 #3 0xffffffff803aaddf in db_script_exec (scriptname=0xffffffff81e042f0 "dump", warnifnotfound=) at ../../../ddb/db_script.c:302 #4 0xffffffff803a6112 in db_command (cmd_table=0x0) at ../../../ddb/db_command.c:479 #5 0xffffffff803a5ea4 in db_command_loop () at ../../../ddb/db_command.c:532 #6 0xffffffff803a908f in db_trap (type=, code=) at ../../../ddb/db_main.c:248 #7 0xffffffff80a93231 in kdb_trap (type=0x3, code=0xffff0ff0, tf=) at ../../../kern/subr_kdb.c:654 #8 0xffffffff80ede5ff in trap (frame=0xfffffe085cfe91c0) at ../../../amd64/amd64/trap.c:536 #9 0xffffffff80ebb194 in calltrap () at ../../../amd64/amd64/exception.S:237 #10 0xffffffff80a9292e in kdb_enter (why=0xffffffff8142fe42 "panic", msg=) at cpufunc.h:63 #11 0xffffffff80a5076f in vpanic (fmt=, ap=) at ../../../kern/kern_shutdown.c:772 #12 0xffffffff80a505a9 in kassert_panic (fmt=) at ../../../kern/kern_shutdown.c:669 #13 0xffffffff80ecb235 in pmap_delayed_invl_finished () at ../../../amd64/amd64/pmap.c:488 #14 0xffffffff80edae70 in reclaim_pv_chunk_leave_pmap (pmap=0xfffff8000773a130, locked_pmap=0xfffff8000773a130, start_di=0x1) at ../../../amd64/amd64/pmap.c:2876 #15 0xffffffff80edad71 in reclaim_pv_chunk (locked_pmap=0xfffff8000773a130, lockp=0xfffffe085cfe9650) at ../../../amd64/amd64/pmap.c:3057 #16 0xffffffff80eceb16 in get_pv_entry (pmap=0xfffff8000773a130, lockp=0xfffffe085cfe9650) at ../../../amd64/amd64/pmap.c:3167 #17 0xffffffff80ecd3e6 in pmap_enter (pmap=0xfffff8000773a130, va=0x18bc8ee000, m=0xfffff80876547e10, prot=0x1, flags=0x101, psind=0x0) at ../../../amd64/amd64/pmap.c:4559 #18 0xffffffff80d49ccd in vm_fault_hold (map=0xfffff8000773a000, vaddr=0x18bc8ee000, fault_type=, fault_flags=0x0, m_hold=0x0) at ../../../vm/vm_fault.c:315 #19 0xffffffff80d49a67 in vm_fault (map=0xfffff8000773a000, vaddr=, fault_type=0x1, fault_flags=0x80) at ../../../vm/vm_fault.c:512 #20 0xffffffff80eded5a in trap_pfault (frame=0xfffffe085cfe9ac0, usermode=0x1) at ../../../amd64/amd64/trap.c:706 #21 0xffffffff80ede6af in trap (frame=0xfffffe085cfe9ac0) at ../../../amd64/amd64/trap.c:329 #22 0xffffffff80ebb194 in calltrap () at ../../../amd64/amd64/exception.S:237 #23 0x00000000004012c4 in ?? () Previous frame inner to this frame (corrupt stack?) Current language: auto; currently minimal (kgdb) f 13 #13 0xffffffff80ecb235 in pmap_delayed_invl_finished () at ../../../amd64/amd64/pmap.c:488 488 KASSERT(invl_gen->gen != 0, ("missed invl_started")); (kgdb) l 483 { 484 struct pmap_invl_gen *invl_gen, *next; 485 struct turnstile *ts; 486 487 invl_gen = &curthread->td_md.md_invl_gen; 488 KASSERT(invl_gen->gen != 0, ("missed invl_started")); 489 mtx_lock(&invl_gen_mtx); 490 next = LIST_NEXT(invl_gen, link); 491 if (next == NULL) { 492 turnstile_chain_lock(&invl_gen_ts); (kgdb) info loc invl_gen = (struct pmap_invl_gen *) 0xfffff8001222b498 next = (struct pmap_invl_gen *) 0xfffff8000773a130 ts = (struct turnstile *) 0xfffff8000773a170 (kgdb) p *invl_gen $1 = {gen = 0x0, link = {le_next = 0x0, le_prev = 0xffffffff81dd0860}} (kgdb) (kgdb) f 14 #14 0xffffffff80edae70 in reclaim_pv_chunk_leave_pmap (pmap=0xfffff8000773a130, locked_pmap=0xfffff8000773a130, start_di=0x1) at ../../../amd64/amd64/pmap.c:2876 2876 pmap_delayed_invl_finished(); (kgdb) l 2871 return; 2872 pmap_invalidate_all(pmap); 2873 if (pmap != locked_pmap) 2874 PMAP_UNLOCK(pmap); 2875 if (start_di) 2876 pmap_delayed_invl_finished(); 2877 } 2878 2879 /* 2880 * We are in a serious low memory condition. Resort to (kgdb) info loc No locals. (kgdb) p *pmap $2 = {pm_mtx = {lock_object = {lo_name = 0xffffffff814956e9 "pmap", lo_flags = 0x1430000, lo_data = 0x0, lo_witness = 0xfffffe0000bf1880}, mtx_lock = 0xfffff8001222b000}, pm_pml4 = 0xfffff80074591000, pm_cr3 = 0x74591000, pm_pvchunk = {tqh_first = 0xfffff8059bd6a000, tqh_last = 0xfffff8076d35f008}, pm_active = {__bits = {0x100, 0x0, 0x0, 0x0}}, pm_type = PT_X86, pm_stats = {resident_count = 0x1aff22, wired_count = 0x0}, pm_root = { rt_root = 0x0}, pm_eptgen = 0x0, pm_flags = 0x100, pm_pcids = {{pm_pcid = 0x506, pm_gen = 0x0}, {pm_pcid = 0x49f, pm_gen = 0x0}, {pm_pcid = 0x43c, pm_gen = 0x0}, {pm_pcid = 0x512, pm_gen = 0x0}, {pm_pcid = 0x286, pm_gen = 0x0}, {pm_pcid = 0x27b, pm_gen = 0x0}, {pm_pcid = 0x4cd, pm_gen = 0x0}, {pm_pcid = 0x500, pm_gen = 0x0}, {pm_pcid = 0x426, pm_gen = 0x1}, {pm_pcid = 0x422, pm_gen = 0x0}, {pm_pcid = 0x420, pm_gen = 0x0}, { pm_pcid = 0x445, pm_gen = 0x0}, {pm_pcid = 0x0, pm_gen = 0x0} }} (kgdb) (kgdb) f 15 #15 0xffffffff80edad71 in reclaim_pv_chunk (locked_pmap=0xfffff8000773a130, lockp=0xfffffe085cfe9650) at ../../../amd64/amd64/pmap.c:3057 3057 reclaim_pv_chunk_leave_pmap(pmap, locked_pmap, start_di); (kgdb) l 3052 TAILQ_REMOVE(&pv_chunks, pc_marker, pc_lru); 3053 TAILQ_INSERT_AFTER(&pv_chunks, pc, pc_marker, pc_lru); 3054 } 3055 TAILQ_REMOVE(&pv_chunks, pc_marker, pc_lru); 3056 mtx_unlock(&pv_chunks_mutex); 3057 reclaim_pv_chunk_leave_pmap(pmap, locked_pmap, start_di); 3058 if (m_pc == NULL && !SLIST_EMPTY(&free)) { 3059 m_pc = SLIST_FIRST(&free); 3060 SLIST_REMOVE_HEAD(&free, plinks.s.ss); 3061 /* Recycle a freed page table page. */ (kgdb) info loc pc = (struct pv_chunk *) 0xfffff8007459e000 pc_marker = (struct pv_chunk *) 0xfffffe085cfe94c0 pc_marker_b = {pc_pmap = 0x0, pc_list = {tqe_next = 0x0, tqe_prev = 0x0}, pc_map = {0x0, 0x0, 0x0}, pc_lru = {tqe_next = 0xfffffe085d00e4c0, tqe_prev = 0xfffff8000fdd5030}} pvh = (struct md_page *) 0x0 pde = (pd_entry_t *) 0xfffff80074598020 pmap = 0xfffff8000773a130 pmap1 = 0xfffff8000773a130 pte = (pt_entry_t *) 0xfffff8007459d6d0 tpte = 0x10a1d405 PG_G = 0x100 PG_A = 0x20 PG_M = 0x40 PG_RW = 0x2 pv = 0xfffff8007459efe8 va = 0x8008da000 m = 0xfffff8084a746548 m_pc = 0xfffff8084cfc91b0 free = {slh_first = 0x0} inuse = 0x0 bit = 0x27 field = 0x3 freed = 0xa8 start_di = 0x1 (kgdb) -- Test scenario: misc/holdcnt0.txt