KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2018 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 12.0-CURRENT #1 r330873M: Tue Mar 13 21:19:44 CET 2018 pho@mercat1.netperf.freebsd.org:/usr/src/sys/amd64/compile/PHO amd64 FreeBSD clang version 6.0.0 (tags/RELEASE_600/final 326565) (based on LLVM 6.0.0) WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. Entering uma_startup with 7 boot pages configured startup_alloc from "UMA Kegs", 6 boot pages left startup_alloc from "UMA Zones", 5 boot pages left startup_alloc from "UMA Zones", 4 boot pages left startup_alloc from "UMA Zones", 3 boot pages left startup_alloc from "UMA Zones", 2 boot pages left startup_alloc from "UMA Hash", 1 boot pages left Entering uma_startup1 with 0 boot pages left Entering uma_startup2 with 0 boot pages left VT(vga): resolution 640x480 CPU: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz (3500.07-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306f2 Family=0x6 Model=0x3f Stepping=2 Features=0xbfebfbff Features2=0x7ffefbff AMD Features=0x2c100800 AMD Features2=0x21 Structured Extended Features=0x37ab XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics real memory = 34359738368 (32768 MB) avail memory = 33179262976 (31642 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs FreeBSD/SMP: 1 package(s) x 6 core(s) x 2 hardware threads arc4random: no preloaded entropy cache ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard SMP: AP CPU #1 Launched! SMP: AP CPU #11 Launched! SMP: AP CPU #10 Launched! SMP: AP CPU #5 Launched! SMP: AP CPU #9 Launched! SMP: AP CPU #8 Launched! SMP: AP CPU #4 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #6 Launched! SMP: AP CPU #7 Launched! Timecounter "TSC-low" frequency 1750036570 Hz quality 1000 random: entropy device external interface module_register_init: MOD_LOAD (vesa, 0xffffffff80fec5d0, 0) error 19 random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" kbd1 at kbdmux0 netmap: loaded module [ath_hal] loaded nexus0 vtvga0: on motherboard cryptosoft0: on motherboard acpi0: on motherboard acpi0: Power Button (fixed) cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 cpu4: on acpi0 cpu5: on acpi0 cpu6: on acpi0 cpu7: on acpi0 cpu8: on acpi0 cpu9: on acpi0 cpu10: on acpi0 cpu11: on acpi0 atrtc0: port 0x70-0x71,0x74-0x77 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Event timer "HPET4" frequency 14318180 Hz quality 340 Event timer "HPET5" frequency 14318180 Hz quality 340 Event timer "HPET6" frequency 14318180 Hz quality 340 Event timer "HPET7" frequency 14318180 Hz quality 340 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: on acpi0 pci0: on pcib0 pci0: at device 11.1 (no driver attached) pci0: at device 11.2 (no driver attached) pci0: at device 16.1 (no driver attached) pci0: at device 16.6 (no driver attached) pci0: at device 18.1 (no driver attached) acpi_syscontainer0: on acpi0 acpi_syscontainer1: on acpi0 acpi_syscontainer2: on acpi0 acpi_syscontainer3: on acpi0 pcib1: port 0xcf8-0xcff on acpi0 pci1: on pcib1 pcib2: irq 26 at device 1.0 on pci1 pci2: on pcib2 pcib3: irq 32 at device 2.0 on pci1 pci3: on pcib3 pci3: at device 0.0 (no driver attached) pci3: at device 0.1 (no driver attached) pcib4: irq 40 at device 3.0 on pci1 pci4: on pcib4 pci1: at device 17.0 (no driver attached) ahci0: port 0xf110-0xf117,0xf100-0xf103,0xf0f0-0xf0f7,0xf0e0-0xf0e3,0xf020-0xf03f mem 0xfb438000-0xfb4387ff irq 16 at device 17.4 on pci1 ahci0: AHCI v1.30 with 4 6Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahciem0: on ahci0 xhci0: mem 0xfb400000-0xfb40ffff irq 19 at device 20.0 on pci1 xhci0: 32 bytes context size, 64-bit DMA usbus0 on xhci0 usbus0: 5.0Gbps Super Speed USB v3.0 pci1: at device 22.0 (no driver attached) pci1: at device 22.1 (no driver attached) ehci0: mem 0xfb434000-0xfb4343ff irq 18 at device 26.0 on pci1 usbus1: EHCI version 1.0 usbus1 on ehci0 usbus1: 480Mbps High Speed USB v2.0 pcib5: irq 16 at device 28.0 on pci1 pci5: on pcib5 pcib6: irq 18 at device 28.2 on pci1 pci6: on pcib6 pcib7: at device 0.0 on pci6 pci7: on pcib7 vgapci0: port 0xe000-0xe07f mem 0xfa000000-0xfaffffff,0xfb000000-0xfb01ffff irq 18 at device 0.0 on pci7 vgapci0: Boot video device pcib8: irq 16 at device 28.4 on pci1 pci8: on pcib8 igb0: port 0xd020-0xd03f mem 0xfb320000-0xfb33ffff,0xfb344000-0xfb347fff irq 16 at device 0.0 on pci8 igb0: attach_pre capping queues at 8 igb0: using 1024 tx descriptors and 1024 rx descriptors igb0: msix_init qsets capped at 8 igb0: pxm cpus: 6 queue msgs: 9 admincnt: 1 igb0: queue equality override not set, capping rx_queues at 6 and tx_queues at 6 igb0: using 6 rx queues 6 tx queues igb0: Using MSIX interrupts with 7 vectors igb0: allocated for 6 tx_queues igb0: allocated for 6 rx_queues igb0: Ethernet address: 0c:c4:7a:a8:cd:da igb0: netmap queues/slots: TX 6/1024, RX 6/1024 igb1: port 0xd000-0xd01f mem 0xfb300000-0xfb31ffff,0xfb340000-0xfb343fff irq 17 at device 0.1 on pci8 igb1: attach_pre capping queues at 8 igb1: using 1024 tx descriptors and 1024 rx descriptors igb1: msix_init qsets capped at 8 igb1: pxm cpus: 6 queue msgs: 9 admincnt: 1 igb1: queue equality override not set, capping rx_queues at 6 and tx_queues at 6 igb1: using 6 rx queues 6 tx queues igb1: Using MSIX interrupts with 7 vectors igb1: allocated for 6 tx_queues igb1: allocated for 6 rx_queues igb1: Ethernet address: 0c:c4:7a:a8:cd:db igb1: netmap queues/slots: TX 6/1024, RX 6/1024 ehci1: mem 0xfb433000-0xfb4333ff irq 18 at device 29.0 on pci1 usbus2: EHCI version 1.0 usbus2 on ehci1 usbus2: 480Mbps High Speed USB v2.0 isab0: at device 31.0 on pci1 isa0: on isab0 ahci1: port 0xf070-0xf077,0xf060-0xf063,0xf050-0xf057,0xf040-0xf043,0xf000-0xf01f mem 0xfb432000-0xfb4327ff irq 16 at device 31.2 on pci1 ahci1: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported ahcich4: at channel 0 on ahci1 ahcich5: at channel 1 on ahci1 ahcich6: at channel 2 on ahci1 ahcich7: at channel 3 on ahci1 ahcich8: at channel 4 on ahci1 ahcich9: at channel 5 on ahci1 ahciem1: on ahci1 acpi_button0: on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 uart1: console (115200,n,8,1) orm0: at iomem 0xc0000-0xc7fff,0xcb000-0xcbfff pnpid ORM0000 on isa0 est0: on cpu0 p4tcc0: on cpu0 est1: on cpu1 p4tcc1: on cpu1 est2: on cpu2 p4tcc2: on cpu2 est3: on cpu3 p4tcc3: on cpu3 est4: on cpu4 p4tcc4: on cpu4 est5: on cpu5 p4tcc5: on cpu5 est6: on cpu6 p4tcc6: on cpu6 est7: on cpu7 p4tcc7: on cpu7 est8: on cpu8 p4tcc8: on cpu8 est9: on cpu9 p4tcc9: on cpu9 est10: on cpu10 p4tcc10: on cpu10 est11: on cpu11 p4tcc11: on cpu11 NULL mp in getnewvnode(9), tag crossmp Timecounters tick every 1.000 msec arc4random: no preloaded entropy cache ugen0.1: <0x8086 XHCI root HUB> at usbus0 ugen2.1: at usbus2 uhub0: <0x8086 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0 ugen1.1: at usbus1 uhub1: on usbus2 uhub2: SEMB S-E-S 2.00 device ses1: SEMB SES Device 2.00/0.05, addr 2> on usbus2 ugen1.2: at usbus1 uhub4 on uhub2 uhub4: on usbus1 ugen0.2: at usbus0 uhub5 on uhub0 uhub5: on usbus0 uhub4: 6 ports with 6 removable, self powered uhub3: 8 ports with 8 removable, self powered uhub5: 4 ports with 3 removable, self powered Writing entropy . Setting hostname: mercat1.netperf.freebsd.org. ugen0.3: at usbus0 ukbd0 on uhub5 ukbd0: on usbus0 kbd2 at ukbd0 Starting Network: lo0 igb0 igb1. lo0: flags=8049 metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 nd6 options=21 groups: lo igb0: flags=8843 metric 0 mtu 1500 options=e505bb ether 0c:c4:7a:a8:cd:da inet6 fe80::ec4:7aff:fea8:cdda%igb0 prefixlen 64 scopeid 0x1 nd6 options=21 media: Ethernet autoselect status: no carrier igb1: flags=8802 metric 0 mtu 1500 options=e505bb ether 0c:c4:7a:a8:cd:db nd6 options=29 media: Ethernet autoselect status: no carrier Starting devd. Starting Network: igb1. igb1: flags=8802 metric 0 mtu 1500 options=e505bb ether 0c:c4:7a:a8:cd:db nd6 options=29 on usbus0 Starting pflogd:ums0: 3 buttons and [Z] coordinates ID=0 add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Creating and/or trimming log files. Starting syslogd. Setting date via ntp. 15 Mar 06:57:36 ntpdate[533]: no servers can be used, exiting Starting rpcbind. No core dumps found. ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib 32-bit compatibility ldconfig path: /usr/lib32 NFS access cache time=60 Starting casperd. lock order reversal: 1st 0xfffffe0000ecb078 bufwait (bufwait) @ kern/vfs_bio.c:3892 2nd 0xfffff8000400e400 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:289 stack backtrace: #0 0xffffffff80b15573 at witness_debugger+0x73 #1 0xffffffff80b153f4 at witness_checkorder+0xe34 #2 0xffffffff80ab7b58 at _sx_xlock+0x68 #3 0xffffffff80db648d at ufsdirhash_add+0x4d #4 0xffffffff80dbafaf at ufs_direnter+0x49f #5 0xffffffff80dc35ac at ufs_mkdir+0x8dc #6 0xffffffff810e28fd at VOP_MKDIR_APV+0xfd #7 0xffffffff80b8f746 at kern_mkdirat+0x1d6 #8 0xffffffff80f6ce3b at amd64_syscall+0x7ab #9 0xffffffff80f4801d at fast_syscall_common+0x101 Clearing /tmp (X related). Starting statd. Link state changed to up Starting lockd. Updating motd:. Mounting late file systems:. Configuring syscons: blanktime. Performing sanity check on sshd configuration. Starting sshd. Starting cron. Thu Mar 15 06:57 FreeBSD/amd64 (mercat1.netperf.freebsd.org) (ttyu1) login: root Mar 15 06:58:01 mercat1 login: ROOT LOGIN (root) ON ttyu1 Last login: Wed Mar 14 07:23:38 on ttyu1 FreeBSD 12.0-CURRENT (PHO) #1 r330873M: Tue Mar 13 21:19:44 CET 2018 root@mercat1:~ # Mar 15 06:58:08 mercat1 su: pho to root on /dev/pts/0 lock order reversal: 1st 0xfffff8001bbe9cc0 ufs (ufs) @ kern/vfs_mount.c:1280 2nd 0xfffff8001b2712e0 devfs (devfs) @ ufs/ffs/ffs_vfsops.c:1583 stack backtrace: #0 0xffffffff80b15573 at witness_debugger+0x73 #1 0xffffffff80b153f4 at witness_checkorder+0xe34 #2 0xffffffff80a83ffe at lockmgr_lock_fast_path+0x17e #3 0xffffffff810e3611 at VOP_LOCK1_APV+0xe1 #4 0xffffffff80b92636 at _vn_lock+0x66 #5 0xffffffff80daf8ac at ffs_sync+0x2cc #6 0xffffffff80b78885 at dounmount+0x555 #7 0xffffffff80b782a0 at sys_unmount+0x310 #8 0xffffffff80f6ce3b at amd64_syscall+0x7ab #9 0xffffffff80f4801d at fast_syscall_common+0x101 20180315 06:58:46 all (1/5): datamove.sh freework leak: 29/38. newblk leak: 5/13. 20180315 06:58:49 all (2/5): datamove2.sh lock order reversal: 1st 0xfffff8001bf3bcc0 syncer (syncer) @ kern/vfs_subr.c:2152 2nd 0xfffff8014b4e02e0 ufs (ufs) @ kern/vfs_subr.c:2602 stack backtrace: #0 0xffffffff80b15573 at witness_debugger+0x73 #1 0xffffffff80b153f4 at witness_checkorder+0xe34 #2 0xffffffff80a844cb at lockmgr_xlock_hard+0x6b #3 0xffffffff80a84eaa at __lockmgr_args+0x51a #4 0xffffffff80db124f at ffs_lock+0xaf #5 0xffffffff810e3611 at VOP_LOCK1_APV+0xe1 #6 0xffffffff80b92636 at _vn_lock+0x66 #7 0xffffffff80b809e8 at vget+0x98 #8 0xffffffff80b82d36 at vfs_msync+0xa6 #9 0xffffffff80b87d56 at sync_fsync+0xc6 #10 0xffffffff810e218d at VOP_FSYNC_APV+0xfd #11 0xffffffff80b85bc4 at sched_sync+0x284 #12 0xffffffff80a6fa24 at fork_exit+0x84 #13 0xffffffff80f484ee at fork_trampoline+0xe KDB: enter: Break to debugger [ thread pid 11 tid 100003 ] Stopped at kdb_alt_break_internal+0x106: movq $0,kdb_why db> ps pid ppid pgrp uid state wmesg wchan cmd 2934 1 1 0 D ufs 0xfffff8001bb3da48 getty 2933 1 1 0 D ufs 0xfffff8001bb3da48 getty 2932 1 1 0 D ufs 0xfffff8001bb3da48 getty 2931 1 1 0 D ufs 0xfffff8001bb3da48 getty 2930 1 1 0 D ufs 0xfffff8001bb3da48 getty 2929 1 1 0 D ufs 0xfffff8001bb3da48 getty 2928 1 1 0 D ufs 0xfffff8001bb3da48 getty 2927 1 1 0 D ufs 0xfffff8001bb3da48 getty 2926 1 1 0 D ufs 0xfffff8001bb3da48 getty 2658 1419 812 0 D+ vmwait 0xffffffff819a0e40 datamove2 2657 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2656 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2655 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2654 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2653 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2652 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2651 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2650 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2649 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap --More-- db> show page vm_cnt.v_free_count: 44234 vm_cnt.v_inactive_count: 1 vm_cnt.v_active_count: 7073534 vm_cnt.v_laundry_count: 900399 vm_cnt.v_wire_count: 112168 vm_cnt.v_free_reserved: 10652 vm_cnt.v_free_min: 51303 vm_cnt.v_free_target: 173256 vm_cnt.v_inactive_target: 259884 db> run pho db:0:pho> run pho1 db:1:pho1> bt Tracing pid 11 tid 100003 td 0xfffff80003938000 kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db:1:pho1> show allpcpu Current CPU: 0 cpuid = 0 dynamic pcpu = 0xa101c0 curthread = 0xfffff80003938000: pid 11 tid 100003 "idle: cpu0" curpcb = 0xfffffe007895ab80 fpcurthread = none idlethread = 0xfffff80003938000: tid 100003 "idle: cpu0" curpmap = 0xffffffff81f40758 tssp = 0xffffffff81f6ac10 commontssp = 0xffffffff81f6ac10 rsp0 = 0xfffffe007895ab80 gs32p = 0xffffffff81f71458 ldt = 0xffffffff81f71498 tss = 0xffffffff81f71488 curvnet = 0 spin locks held: cpuid = 1 dynamic pcpu = 0xfffffe00856c51c0 curthread = 0xfffff80003937560: pid 11 tid 100004 "idle: cpu1" curpcb = 0xfffffe007895fb80 fpcurthread = none idlethread = 0xfffff80003937560: tid 100004 "idle: cpu1" curpmap = 0xffffffff81f40758 tssp = 0xffffffff81f6ac78 commontssp = 0xffffffff81f6ac78 rsp0 = 0xfffffe007895fb80 gs32p = 0xffffffff81f714c0 ldt = 0xffffffff81f71500 tss = 0xffffffff81f714f0 curvnet = 0 spin locks held: cpuid = 2 dynamic pcpu = 0xfffffe00856ce1c0 curthread = 0xfffff80003937000: pid 11 tid 100005 "idle: cpu2" curpcb = 0xfffffe0078964b80 fpcurthread = none idlethread = 0xfffff80003937000: tid 100005 "idle: cpu2" curpmap = 0xffffffff81f40758 tssp = 0xffffffff81f6ace0 commontssp = 0xffffffff81f6ace0 rsp0 = 0xfffffe0078964b80 gs32p = 0xffffffff81f71528 ldt = 0xffffffff81f71568 tss = 0xffffffff81f71558 curvnet = 0 spin locks held: cpuid = 3 dynamic pcpu = 0xfffffe00856d71c0 curthread = 0xfffff80003936560: pid 11 tid 100006 "idle: cpu3" curpcb = 0xfffffe0078969b80 fpcurthread = none idlethread = 0xfffff80003936560: tid 100006 "idle: cpu3" curpmap = 0xffffffff81f40758 tssp = 0xffffffff81f6ad48 commontssp = 0xffffffff81f6ad48 rsp0 = 0xfffffe0078969b80 gs32p = 0xffffffff81f71590 ldt = 0xffffffff81f715d0 tss = 0xffffffff81f715c0 curvnet = 0 spin locks held: cpuid = 4 dynamic pcpu = 0xfffffe00856e01c0 curthread = 0xfffff80003936000: pid 11 tid 100007 "idle: cpu4" curpcb = 0xfffffe007896eb80 fpcurthread = none idlethread = 0xfffff80003936000: tid 100007 "idle: cpu4" curpmap = 0xffffffff81f40758 tssp = 0xffffffff81f6adb0 commontssp = 0xffffffff81f6adb0 rsp0 = 0xfffffe007896eb80 gs32p = 0xffffffff81f715f8 ldt = 0xffffffff81f71638 tss = 0xffffffff81f71628 curvnet = 0 spin locks held: cpuid = 5 dynamic pcpu = 0xfffffe00856e91c0 curthread = 0xfffff80003935560: pid 11 tid 100008 "idle: cpu5" curpcb = 0xfffffe0078973b80 fpcurthread = none idlethread = 0xfffff80003935560: tid 100008 "idle: cpu5" curpmap = 0xffffffff81f40758 tssp = 0xffffffff81f6ae18 commontssp = 0xffffffff81f6ae18 rsp0 = 0xfffffe0078973b80 gs32p = 0xffffffff81f71660 ldt = 0xffffffff81f716a0 tss = 0xffffffff81f71690 curvnet = 0 spin locks held: cpuid = 6 dynamic pcpu = 0xfffffe00856f21c0 curthread = 0xfffff80003935000: pid 11 tid 100009 "idle: cpu6" curpcb = 0xfffffe0078978b80 fpcurthread = none idlethread = 0xfffff80003935000: tid 100009 "idle: cpu6" curpmap = 0xffffffff81f40758 tssp = 0xffffffff81f6ae80 commontssp = 0xffffffff81f6ae80 rsp0 = 0xfffffe0078978b80 gs32p = 0xffffffff81f716c8 ldt = 0xffffffff81f71708 tss = 0xffffffff81f716f8 curvnet = 0 spin locks held: cpuid = 7 dynamic pcpu = 0xfffffe00856fb1c0 curthread = 0xfffff80003949560: pid 11 tid 100010 "idle: cpu7" curpcb = 0xfffffe007897db80 fpcurthread = none idlethread = 0xfffff80003949560: tid 100010 "idle: cpu7" curpmap = 0xffffffff81f40758 tssp = 0xffffffff81f6aee8 commontssp = 0xffffffff81f6aee8 rsp0 = 0xfffffe007897db80 gs32p = 0xffffffff81f71730 ldt = 0xffffffff81f71770 tss = 0xffffffff81f71760 curvnet = 0 spin locks held: cpuid = 8 dynamic pcpu = 0xfffffe00857041c0 curthread = 0xfffff80003949000: pid 11 tid 100011 "idle: cpu8" curpcb = 0xfffffe0078982b80 fpcurthread = none idlethread = 0xfffff80003949000: tid 100011 "idle: cpu8" curpmap = 0xffffffff81f40758 tssp = 0xffffffff81f6af50 commontssp = 0xffffffff81f6af50 rsp0 = 0xfffffe0078982b80 gs32p = 0xffffffff81f71798 ldt = 0xffffffff81f717d8 tss = 0xffffffff81f717c8 curvnet = 0 spin locks held: cpuid = 9 dynamic pcpu = 0xfffffe008570d1c0 curthread = 0xfffff80003948560: pid 11 tid 100012 "idle: cpu9" curpcb = 0xfffffe0078987b80 fpcurthread = none idlethread = 0xfffff80003948560: tid 100012 "idle: cpu9" curpmap = 0xffffffff81f40758 tssp = 0xffffffff81f6afb8 commontssp = 0xffffffff81f6afb8 rsp0 = 0xfffffe0078987b80 gs32p = 0xffffffff81f71800 ldt = 0xffffffff81f71840 tss = 0xffffffff81f71830 curvnet = 0 spin locks held: cpuid = 10 dynamic pcpu = 0xfffffe00857161c0 curthread = 0xfffff80003948000: pid 11 tid 100013 "idle: cpu10" curpcb = 0xfffffe007898cb80 fpcurthread = none idlethread = 0xfffff80003948000: tid 100013 "idle: cpu10" curpmap = 0xffffffff81f40758 tssp = 0xffffffff81f6b020 commontssp = 0xffffffff81f6b020 rsp0 = 0xfffffe007898cb80 gs32p = 0xffffffff81f71868 ldt = 0xffffffff81f718a8 tss = 0xffffffff81f71898 curvnet = 0 spin locks held: cpuid = 11 dynamic pcpu = 0xfffffe007f1e71c0 curthread = 0xfffff80003947560: pid 11 tid 100014 "idle: cpu11" curpcb = 0xfffffe0078991b80 fpcurthread = none idlethread = 0xfffff80003947560: tid 100014 "idle: cpu11" curpmap = 0xffffffff81f40758 tssp = 0xffffffff81f6b088 commontssp = 0xffffffff81f6b088 rsp0 = 0xfffffe0078991b80 gs32p = 0xffffffff81f718d0 ldt = 0xffffffff81f71910 tss = 0xffffffff81f71900 curvnet = 0 spin locks held: db:1:pho1> show alllocks Process 2934 (getty) thread 0xfffff80050abe000 (100189) shared lockmgr ufs (ufs) r = 0 (0xfffff8001bb3e068) locked @ kern/vfs_subr.c:2602 Process 2933 (getty) thread 0xfffff8001badb560 (100225) shared lockmgr ufs (ufs) r = 0 (0xfffff8001bb3e068) locked @ kern/vfs_subr.c:2602 Process 2932 (getty) thread 0xfffff8001badd560 (100201) shared lockmgr ufs (ufs) r = 0 (0xfffff8001bb3e068) locked @ kern/vfs_subr.c:2602 Process 2931 (getty) thread 0xfffff8005005c560 (100210) shared lockmgr ufs (ufs) r = 0 (0xfffff8001bb3e068) locked @ kern/vfs_subr.c:2602 Process 2930 (getty) thread 0xfffff8001bd64560 (100146) shared lockmgr ufs (ufs) r = 0 (0xfffff8001bb3e068) locked @ kern/vfs_subr.c:2602 Process 2929 (getty) thread 0xfffff80050acc000 (100185) shared lockmgr ufs (ufs) r = 0 (0xfffff8001bb3e068) locked @ kern/vfs_subr.c:2602 Process 2928 (getty) thread 0xfffff80050abf560 (100186) shared lockmgr ufs (ufs) r = 0 (0xfffff8001bb3e068) locked @ kern/vfs_subr.c:2602 Process 2927 (getty) thread 0xfffff80050abf000 (100187) shared lockmgr ufs (ufs) r = 0 (0xfffff8001bb3e068) locked @ kern/vfs_subr.c:2602 Process 2926 (getty) thread 0xfffff80050abe560 (100188) shared lockmgr ufs (ufs) r = 0 (0xfffff8001bb3e068) locked @ kern/vfs_subr.c:2602 Process 2618 (swap) thread 0xfffff8014b7e1000 (100265) shared sx vm map (user) (vm map (user)) r = 0 (0xfffff8014b379078) locked @ vm/vm_map.c:4065 Process 2562 (swap) thread 0xfffff8001bec7560 (100328) shared sx vm map (user) (vm map (user)) r = 0 (0xfffff8014b76d078) locked @ vm/vm_map.c:4065 Process 29 (pagedaemon) thread 0xfffff800041a5000 (100128) exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe00011453f8) locked @ kern/vfs_bio.c:1674 exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe0000e25fd0) locked @ vm/vm_pager.c:374 exclusive lockmgr ufs (ufs) r = 0 (0xfffff8001bb3da48) locked @ kern/vfs_subr.c:2602 db:1:pho1> show lockedvnods Locked vnodes vnode 0xfffff8001bb3e000: tag ufs, type VDIR usecount 9, writecount 0, refcount 10 mountedhere 0 flags (VI_ACTIVE) v_object 0xfffff8001ba83a00 ref 0 pages 0 cleanbuf 0 dirtybuf 0 lock type ufs: SHARED (count 9) #0 0xffffffff80a8440b at lockmgr_slock_hard+0x32b #1 0xffffffff80a84ed7 at __lockmgr_args+0x547 #2 0xffffffff80db124f at ffs_lock+0xaf #3 0xffffffff810e3611 at VOP_LOCK1_APV+0xe1 #4 0xffffffff80b92636 at _vn_lock+0x66 #5 0xffffffff80b809e8 at vget+0x98 #6 0xffffffff80b65dda at cache_lookup+0x6fa #7 0xffffffff80b6acfc at vfs_cache_lookup+0xac #8 0xffffffff810e0007 at VOP_LOOKUP_APV+0xf7 #9 0xffffffff80b744a2 at lookup+0x692 #10 0xffffffff80b739eb at namei+0x53b #11 0xffffffff80b91e67 at vn_open_cred+0x207 #12 0xffffffff80b8a7cc at kern_openat+0x20c #13 0xffffffff80f6ce3b at amd64_syscall+0x7ab #14 0xffffffff80f4801d at fast_syscall_common+0x101 ino 56339712, on dev gpt/work vnode 0xfffff8001bb3d9e0: tag ufs, type VREG usecount 272, writecount 0, refcount 439 flags (VI_ACTIVE) v_object 0xfffff8001ba83700 ref 271 pages 235 cleanbuf 3 dirtybuf 0 lock type ufs: EXCL by thread 0xfffff800041a5000 (pid 29, pagedaemon, tid 100128) with shared waiters pending #0 0xffffffff80a8480c at lockmgr_xlock_hard+0x3ac #1 0xffffffff80a84eaa at __lockmgr_args+0x51a #2 0xffffffff80db124f at ffs_lock+0xaf #3 0xffffffff810e3611 at VOP_LOCK1_APV+0xe1 #4 0xffffffff80b92636 at _vn_lock+0x66 #5 0xffffffff80b809e8 at vget+0x98 #6 0xffffffff80df02fd at vm_pageout_laundry_worker+0x8cd #7 0xffffffff80a6fa24 at fork_exit+0x84 #8 0xffffffff80f484ee at fork_trampoline+0xe ino 56340986, on dev gpt/work db:1:pho1> show allchains chain 1: thread 100189 (pid 2934, getty) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 2: thread 100225 (pid 2933, getty) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 3: thread 100201 (pid 2932, getty) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 4: thread 100210 (pid 2931, getty) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 5: thread 100146 (pid 2930, getty) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 6: thread 100185 (pid 2929, getty) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 7: thread 100186 (pid 2928, getty) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 8: thread 100187 (pid 2927, getty) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 9: thread 100188 (pid 2926, getty) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 10: thread 100200 (pid 2658, datamove2) sleeping on 0xffffffff819a0e40 "vmwait" chain 11: thread 100222 (pid 2657, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 12: thread 100303 (pid 2656, swap) sleeping on 0xffffffff819a0e40 "pfault" chain 13: thread 100302 (pid 2655, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 14: thread 100301 (pid 2654, swap) sleeping on 0xffffffff819a0e40 "pfault" chain 15: thread 100315 (pid 2653, swap) sleeping on 0xffffffff819a0e40 "pfault" chain 16: thread 100314 (pid 2652, swap) sleeping on 0xffffffff819a0e40 "pfault" chain 17: thread 100313 (pid 2651, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 18: thread 100312 (pid 2650, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 19: thread 100311 (pid 2649, swap) sleeping on 0xffffffff819a0e40 "pfault" chain 20: thread 100310 (pid 2648, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 21: thread 100294 (pid 2647, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 22: thread 100293 (pid 2646, swap) sleeping on 0xffffffff819a0e40 "pfault" chain 23: thread 100292 (pid 2645, swap) sleeping on 0xffffffff819a0e40 "pfault" chain 24: thread 100309 (pid 2644, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 25: thread 100308 (pid 2643, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 26: thread 100307 (pid 2642, swap) sleeping on 0xffffffff819a0e40 "pfault" chain 27: thread 100306 (pid 2641, swap) sleeping on 0xffffffff819a0e40 "pfault" chain 28: thread 100305 (pid 2640, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 29: thread 100304 (pid 2639, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 30: thread 100285 (pid 2638, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 31: thread 100284 (pid 2637, swap) sleeping on 0xffffffff819a0e40 "pfault" chain 32: thread 100283 (pid 2636, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 33: thread 100300 (pid 2635, swap) sleeping on 0xffffffff819a0e40 "pfault" chain 34: thread 100299 (pid 2634, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 35: thread 100298 (pid 2633, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 36: thread 100297 (pid 2632, swap) sleeping on 0xffffffff819a0e40 "pfault" chain 37: thread 100296 (pid 2631, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 38: thread 100295 (pid 2630, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 39: thread 100276 (pid 2629, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 40: thread 100275 (pid 2628, swap) sleeping on 0xffffffff819a0e40 "pfault" chain 41: thread 100274 (pid 2627, swap) sleeping on 0xffffffff819a0e40 "pfault" chain 42: thread 100291 (pid 2626, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 43: thread 100290 (pid 2625, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 44: thread 100289 (pid 2624, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 45: thread 100288 (pid 2623, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 46: thread 100287 (pid 2622, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 47: thread 100286 (pid 2621, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 48: thread 100267 (pid 2620, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 49: thread 100266 (pid 2619, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 50: thread 100265 (pid 2618, swap) sleeping on 0xffffffff819a0e40 "vmwait" chain 51: thread 100282 (pid 2617, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 52: thread 100281 (pid 2616, swap) sleeping on 0xffffffff819a0e40 "pfault" chain 53: thread 100280 (pid 2615, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 54: thread 100279 (pid 2614, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 55: thread 100278 (pid 2613, swap) sleeping on 0xffffffff819a0e40 "pfault" chain 56: thread 100277 (pid 2612, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 57: thread 100258 (pid 2611, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 58: thread 100257 (pid 2610, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 59: thread 100256 (pid 2609, swap) sleeping on 0xffffffff819a0e40 "pfault" chain 60: thread 100273 (pid 2608, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 61: thread 100272 (pid 2607, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 62: thread 100271 (pid 2606, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 63: thread 100270 (pid 2605, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 64: thread 100269 (pid 2604, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 65: thread 100268 (pid 2603, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 66: thread 100249 (pid 2602, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 67: thread 100248 (pid 2601, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 68: thread 100247 (pid 2600, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 69: thread 100264 (pid 2599, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 70: thread 100263 (pid 2598, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 71: thread 100262 (pid 2597, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 72: thread 100261 (pid 2596, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 73: thread 100260 (pid 2595, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 74: thread 100259 (pid 2594, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 75: thread 100240 (pid 2593, swap) sleeping on 0xffffffff819a0e40 "pfault" chain 76: thread 100239 (pid 2592, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 77: thread 100238 (pid 2591, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 78: thread 100255 (pid 2590, swap) sleeping on 0xffffffff819a0e40 "pfault" chain 79: thread 100254 (pid 2589, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 80: thread 100253 (pid 2588, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 81: thread 100252 (pid 2587, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 82: thread 100251 (pid 2586, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 83: thread 100250 (pid 2585, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 84: thread 100231 (pid 2584, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 85: thread 100230 (pid 2583, swap) sleeping on 0xffffffff819a0e40 "pfault" chain 86: thread 100229 (pid 2582, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 87: thread 100246 (pid 2581, swap) sleeping on 0xffffffff819a0e40 "pfault" chain 88: thread 100245 (pid 2580, swap) sleeping on 0xfffff8084ad10010 "vmpfw" chain 89: thread 100244 (pid 2579, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 90: thread 100243 (pid 2578, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 91: thread 100242 (pid 2577, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 92: thread 100241 (pid 2576, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 93: thread 100228 (pid 2575, swap) sleeping on 0xffffffff819a0e40 "pfault" chain 94: thread 100154 (pid 2574, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 95: thread 100316 (pid 2573, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 96: thread 100237 (pid 2572, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 97: thread 100236 (pid 2571, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 98: thread 100235 (pid 2570, swap) sleeping on 0xffffffff819a0e40 "pfault" chain 99: thread 100234 (pid 2569, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 100: thread 100233 (pid 2568, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 101: thread 100232 (pid 2567, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 102: thread 100336 (pid 2566, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 103: thread 100335 (pid 2565, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 104: thread 100194 (pid 2564, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 105: thread 100329 (pid 2563, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 106: thread 100328 (pid 2562, swap) sleeping on 0xffffffff819a0e40 "vmwait" chain 107: thread 100327 (pid 2561, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 108: thread 100334 (pid 2560, swap) sleeping on 0xffffffff819a0e40 "pfault" chain 109: thread 100333 (pid 2559, swap) sleeping on 0xffffffff819a0e40 "pfault" chain 110: thread 100332 (pid 2558, swap) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 111: thread 100172 (pid 795, sshd) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 112: thread 100177 (pid 791, dhclient) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 113: thread 100170 (pid 699, sshd) sleeping on 0xfffff800039303c0 "select" chain 114: thread 100173 (pid 576, casperd) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 115: thread 100149 (pid 536, rpcbind) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 116: thread 100153 (pid 531, syslogd) blocked on lockmgr ufsEXCL thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 117: thread 100132 (pid 33, vnlru) sleeping on 0xfffff8001b379a70 "vlruwt" chain 118: thread 100131 (pid 32, syncer) sleeping on 0xffffffff81ee9c00 "syncer" chain 119: thread 100130 (pid 31, bufdaemon) sleeping on 0xffffffff81ee90a8 "qsleep" chain 120: thread 100133 (pid 31, bufspacedaemon-0) sleeping on 0xffffffff81963780 "-" chain 121: thread 100134 (pid 31, bufspacedaemon-1) sleeping on 0xffffffff8196ba40 "-" chain 122: thread 100135 (pid 31, bufspacedaemon-2) sleeping on 0xffffffff81973d00 "-" chain 123: thread 100136 (pid 31, bufspacedaemon-3) sleeping on 0xffffffff8197bfc0 "-" chain 124: thread 100137 (pid 31, bufspacedaemon-4) sleeping on 0xffffffff81984280 "-" chain 125: thread 100138 (pid 31, bufspacedaemon-5) sleeping on 0xffffffff8198c4f8 "bufspace" chain 126: thread 100139 (pid 31, bufspacedaemon-6) sleeping on 0xffffffff81994800 "-" chain 127: thread 100151 (pid 31, / worker) sleeping on 0xfffff8001be516e8 "sdflush" chain 128: thread 100198 (pid 31, /tmp worker) sleeping on 0xfffff8014b1d0ce8 "sdflush" chain 129: thread 100127 (pid 30, vmdaemon) sleeping on 0xffffffff81ef5618 "psleep" chain 130: thread 100126 (pid 29, dom0) sleeping on 0xffffffff81d40570 "pwait" chain 131: thread 100128 (pid 29, laundry: dom0) sleeping on 0xfffff8084ad10010 "grbmaw" chain 132: thread 100129 (pid 29, uma) sleeping on 0xffffffff80dcf2f0 "umarcl" chain 133: thread 100125 (pid 28, enc_daemon1) sleeping on 0xfffff8001b28d000 "idle" chain 134: thread 100124 (pid 27, enc_daemon0) sleeping on 0xfffff8001b28d538 "idle" chain 135: thread 100121 (pid 26, rand_harvestq) sleeping on 0xffffffff81c147c8 "-" chain 136: thread 100117 (pid 25, sctp_iterator) sleeping on 0xffffffff81f630c0 "waiting_for_work" chain 137: thread 100116 (pid 24, soaiod4) sleeping on 0xffffffff81ee8aa4 "-" chain 138: thread 100115 (pid 23, soaiod3) sleeping on 0xffffffff81ee8aa4 "-" chain 139: thread 100114 (pid 22, soaiod2) sleeping on 0xffffffff81ee8aa4 "-" chain 140: thread 100113 (pid 21, soaiod1) sleeping on 0xffffffff81ee8aa4 "-" chain 141: thread 100094 (pid 20, usbus0) sleeping on 0xfffffe009c7e1460 "-" chain 142: thread 100095 (pid 20, usbus0) sleeping on 0xfffffe009c7e14b8 "-" chain 143: thread 100096 (pid 20, usbus0) sleeping on 0xfffffe009c7e1510 "-" chain 144: thread 100097 (pid 20, usbus0) sleeping on 0xfffffe009c7e1568 "-" chain 145: thread 100098 (pid 20, usbus0) sleeping on 0xfffffe009c7e15c0 "-" chain 146: thread 100100 (pid 20, usbus1) sleeping on 0xfffffe009a5eed10 "-" chain 147: thread 100101 (pid 20, usbus1) sleeping on 0xfffffe009a5eed68 "-" chain 148: thread 100102 (pid 20, usbus1) sleeping on 0xfffffe009a5eedc0 "-" chain 149: thread 100103 (pid 20, usbus1) sleeping on 0xfffffe009a5eee18 "-" chain 150: thread 100104 (pid 20, usbus1) sleeping on 0xfffffe009a5eee70 "-" chain 151: thread 100105 (pid 20, usbus2) sleeping on 0xfffffe009ca62d10 "-" chain 152: thread 100106 (pid 20, usbus2) sleeping on 0xfffffe009ca62d68 "-" chain 153: thread 100107 (pid 20, usbus2) sleeping on 0xfffffe009ca62dc0 "-" chain 154: thread 100108 (pid 20, usbus2) sleeping on 0xfffffe009ca62e18 "-" chain 155: thread 100109 (pid 20, usbus2) sleeping on 0xfffffe009ca62e70 "-" chain 156: thread 100089 (pid 19, doneq0) sleeping on 0xffffffff81aefac0 "-" chain 157: thread 100090 (pid 19, doneq1) sleeping on 0xffffffff81aefb40 "-" chain 158: thread 100091 (pid 19, doneq2) sleeping on 0xffffffff81aefbc0 "-" chain 159: thread 100123 (pid 19, scanner) sleeping on 0xffffffff81aef988 "-" chain 160: thread 100088 (pid 18, crypto returns 11) sleeping on 0xfffff80003ad2450 "crypto_ret_wait" chain 161: thread 100087 (pid 17, crypto returns 10) sleeping on 0xfffff80003ad23f0 "crypto_ret_wait" chain 162: thread 100086 (pid 16, crypto returns 9) sleeping on 0xfffff80003ad2390 "crypto_ret_wait" chain 163: thread 100085 (pid 15, crypto returns 8) sleeping on 0xfffff80003ad2330 "crypto_ret_wait" chain 164: thread 100084 (pid 14, crypto returns 7) sleeping on 0xfffff80003ad22d0 "crypto_ret_wait" chain 165: thread 100083 (pid 9, crypto returns 6) sleeping on 0xfffff80003ad2270 "crypto_ret_wait" chain 166: thread 100082 (pid 8, crypto returns 5) sleeping on 0xfffff80003ad2210 "crypto_ret_wait" chain 167: thread 100081 (pid 7, crypto returns 4) sleeping on 0xfffff80003ad21b0 "crypto_ret_wait" chain 168: thread 100080 (pid 6, crypto returns 3) sleeping on 0xfffff80003ad2150 "crypto_ret_wait" chain 169: thread 100079 (pid 5, crypto returns 2) sleeping on 0xfffff80003ad20f0 "crypto_ret_wait" chain 170: thread 100078 (pid 4, crypto returns 1) sleeping on 0xfffff80003ad2090 "crypto_ret_wait" chain 171: thread 100077 (pid 3, crypto returns 0) sleeping on 0xfffff80003ad2030 "crypto_ret_wait" chain 172: thread 100076 (pid 2, crypto) sleeping on 0xffffffff81eefe58 "crypto_wait" chain 173: thread 100060 (pid 13, g_event) sleeping on 0xffffffff81f3fb20 "-" chain 174: thread 100061 (pid 13, g_up) sleeping on 0xffffffff81f3fb28 "-" chain 175: thread 100062 (pid 13, g_down) sleeping on 0xffffffff81f3fb30 "-" chain 176: thread 100002 (pid 1, init) sleeping on 0xfffff80003934538 "wait" chain 177: thread 100001 (pid 10, audit) sleeping on 0xffffffff81f63da8 "audit_worker_cv" chain 178: thread 100000 (pid 0, swapper) sleeping on 0xffffffff819a0e40 "vmwait" chain 179: thread 100017 (pid 0, thread taskq) sleeping on 0xfffff80003970600 "-" chain 180: thread 100019 (pid 0, kqueue_ctx taskq) sleeping on 0xfffff80003970100 "-" chain 181: thread 100020 (pid 0, aiod_kick taskq) sleeping on 0xfffff8000396fe00 "-" chain 182: thread 100021 (pid 0, if_config_tqg_0) sleeping on 0xfffff8000397e000 "-" chain 183: thread 100022 (pid 0, softirq_0) sleeping on 0xfffff8000397de00 "-" chain 184: thread 100023 (pid 0, softirq_1) sleeping on 0xfffff8000397dd00 "-" chain 185: thread 100024 (pid 0, softirq_2) sleeping on 0xfffff8000397dc00 "-" chain 186: thread 100025 (pid 0, softirq_3) sleeping on 0xfffff8000397db00 "-" chain 187: thread 100026 (pid 0, softirq_4) sleeping on 0xfffff8000397da00 "-" chain 188: thread 100027 (pid 0, softirq_5) sleeping on 0xfffff8000397d900 "-" chain 189: thread 100028 (pid 0, softirq_6) sleeping on 0xfffff8000397d800 "-" chain 190: thread 100029 (pid 0, softirq_7) sleeping on 0xfffff8000397d700 "-" chain 191: thread 100030 (pid 0, softirq_8) sleeping on 0xfffff8000397d600 "-" chain 192: thread 100031 (pid 0, softirq_9) sleeping on 0xfffff8000397d500 "-" chain 193: thread 100032 (pid 0, softirq_10) sleeping on 0xfffff8000397d400 "-" chain 194: thread 100033 (pid 0, softirq_11) sleeping on 0xfffff8000397d300 "-" chain 195: thread 100034 (pid 0, if_io_tqg_0) sleeping on 0xfffff8000397d200 "-" chain 196: thread 100035 (pid 0, if_io_tqg_1) sleeping on 0xfffff8000397d100 "-" chain 197: thread 100036 (pid 0, if_io_tqg_2) sleeping on 0xfffff8000397d000 "-" chain 198: thread 100037 (pid 0, if_io_tqg_3) sleeping on 0xfffff8000397ce00 "-" chain 199: thread 100038 (pid 0, if_io_tqg_4) sleeping on 0xfffff8000397cd00 "-" chain 200: thread 100039 (pid 0, if_io_tqg_5) sleeping on 0xfffff8000397cc00 "-" chain 201: thread 100040 (pid 0, if_io_tqg_6) sleeping on 0xfffff8000397cb00 "-" chain 202: thread 100041 (pid 0, if_io_tqg_7) sleeping on 0xfffff8000397ca00 "-" chain 203: thread 100042 (pid 0, if_io_tqg_8) sleeping on 0xfffff8000397c900 "-" chain 204: thread 100043 (pid 0, if_io_tqg_9) sleeping on 0xfffff8000397c800 "-" chain 205: thread 100044 (pid 0, if_io_tqg_10) sleeping on 0xfffff8000397c700 "-" chain 206: thread 100045 (pid 0, if_io_tqg_11) sleeping on 0xfffff8000397c600 "-" chain 207: thread 100063 (pid 0, firmware taskq) sleeping on 0xfffff80003a0c100 "-" chain 208: thread 100064 (pid 0, crypto_0) sleeping on 0xfffff80003a0be00 "-" chain 209: thread 100065 (pid 0, crypto_1) sleeping on 0xfffff80003a0be00 "-" chain 210: thread 100066 (pid 0, crypto_2) sleeping on 0xfffff80003a0be00 "-" chain 211: thread 100067 (pid 0, crypto_3) sleeping on 0xfffff80003a0be00 "-" chain 212: thread 100068 (pid 0, crypto_4) sleeping on 0xfffff80003a0be00 "-" chain 213: thread 100069 (pid 0, crypto_5) sleeping on 0xfffff80003a0be00 "-" chain 214: thread 100070 (pid 0, crypto_6) sleeping on 0xfffff80003a0be00 "-" chain 215: thread 100071 (pid 0, crypto_7) sleeping on 0xfffff80003a0be00 "-" chain 216: thread 100072 (pid 0, crypto_8) sleeping on 0xfffff80003a0be00 "-" chain 217: thread 100073 (pid 0, crypto_9) sleeping on 0xfffff80003a0be00 "-" chain 218: thread 100074 (pid 0, crypto_10) sleeping on 0xfffff80003a0be00 "-" chain 219: thread 100075 (pid 0, crypto_11) sleeping on 0xfffff80003a0be00 "-" chain 220: thread 100112 (pid 0, mca taskq) sleeping on 0xfffff8000c0bec00 "-" chain 221: thread 100118 (pid 0, acpi_task_0) sleeping on 0xfffff8001b38d700 "-" chain 222: thread 100119 (pid 0, acpi_task_1) sleeping on 0xfffff8001b38d700 "-" chain 223: thread 100120 (pid 0, acpi_task_2) sleeping on 0xfffff8001b38d700 "-" chain 224: thread 100122 (pid 0, CAM taskq) sleeping on 0xfffff80003971300 "-" db:1:pho1> show mount 0xfffff8001ba86000 /dev/gpt/work on / (ufs) 0xfffff8001ba87000 devfs on /dev (devfs) 0xfffff8014b1cd000 /dev/gpt/tmp on /tmp (ufs) More info: show mount db:1:pho1> show page vm_cnt.v_free_count: 44234 vm_cnt.v_inactive_count: 1 vm_cnt.v_active_count: 7073534 vm_cnt.v_laundry_count: 900399 vm_cnt.v_wire_count: 112168 vm_cnt.v_free_reserved: 10652 vm_cnt.v_free_min: 51303 vm_cnt.v_free_target: 173256 vm_cnt.v_inactive_target: 259884 db:1:pho1> show freepages DOMAIN: 0 FREE LIST 0: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000000 | 000000 10 (004096K) | 000000 | 000000 09 (002048K) | 000000 | 000000 08 (001024K) | 000057 | 000000 07 (000512K) | 000047 | 000000 06 (000256K) | 000024 | 000002 05 (000128K) | 000094 | 000004 04 (000064K) | 000161 | 000006 03 (000032K) | 000347 | 000035 02 (000016K) | 000209 | 000091 01 (000008K) | 000379 | 000216 00 (000004K) | 000140 | 000000 FREE LIST 1: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000000 | 000000 10 (004096K) | 000000 | 000000 09 (002048K) | 000000 | 000000 08 (001024K) | 000000 | 000000 07 (000512K) | 000000 | 000000 06 (000256K) | 000000 | 000000 05 (000128K) | 000000 | 000000 04 (000064K) | 000000 | 000000 03 (000032K) | 000000 | 000000 02 (000016K) | 000000 | 000000 01 (000008K) | 000000 | 000000 00 (000004K) | 000000 | 000000 db:1:pho1> show pageq pq_free 44234 dom 0 page_cnt 8130451 free 44234 pq_act 7073534 pq_inact 1 pq_laund 900399 pq_unsw 0 db:1:pho1> show bufqueues bqempty: 105708 Buf domain 0 freebufs 15133 lofreebufs 219 hifreebufs 329 bufspace 0 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 0 (0) dirtyq count 0 (0) wakeup 1 lim 25 CPU 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, Locked buffers: 0 space 0 Total buffers: 778 space 0 Buf domain 1 freebufs 15133 lofreebufs 219 hifreebufs 329 bufspace 0 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 0 (0) dirtyq count 0 (0) wakeup 1 lim 25 CPU 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, Locked buffers: 0 space 0 Total buffers: 366 space 0 Buf domain 2 freebufs 15130 lofreebufs 219 hifreebufs 329 bufspace 98304 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 0 (0) dirtyq count 0 (0) wakeup 1 lim 25 CPU 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, Locked buffers: 3 space 65536 Total buffers: 412 space 65536 Buf domain 3 freebufs 15133 lofreebufs 219 hifreebufs 329 bufspace 0 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 0 (0) dirtyq count 0 (0) wakeup 1 lim 25 CPU 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, Locked buffers: 0 space 0 Total buffers: 1042 space 0 Buf domain 4 freebufs 15133 lofreebufs 219 hifreebufs 329 bufspace 0 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 0 (0) dirtyq count 0 (0) wakeup 1 lim 25 CPU 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, Locked buffers: 0 space 0 Total buffers: 148 space 0 Buf domain 5 freebufs 15133 lofreebufs 219 hifreebufs 329 bufspace 0 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 0 (0) dirtyq count 0 (0) wakeup 1 lim 25 CPU 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, Locked buffers: 0 space 0 Total buffers: 333 space 0 Buf domain 6 freebufs 15133 lofreebufs 219 hifreebufs 329 bufspace 0 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 0 (0) dirtyq count 0 (0) wakeup 1 lim 25 CPU 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, Locked buffers: 0 space 0 Total buffers: 331 space 0 db:1:pho1> ps pid ppid pgrp uid state wmesg wchan cmd 2934 1 1 0 D ufs 0xfffff8001bb3da48 getty 2933 1 1 0 D ufs 0xfffff8001bb3da48 getty 2932 1 1 0 D ufs 0xfffff8001bb3da48 getty 2931 1 1 0 D ufs 0xfffff8001bb3da48 getty 2930 1 1 0 D ufs 0xfffff8001bb3da48 getty 2929 1 1 0 D ufs 0xfffff8001bb3da48 getty 2928 1 1 0 D ufs 0xfffff8001bb3da48 getty 2927 1 1 0 D ufs 0xfffff8001bb3da48 getty 2926 1 1 0 D ufs 0xfffff8001bb3da48 getty 2658 1419 812 0 D+ vmwait 0xffffffff819a0e40 datamove2 2657 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2656 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2655 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2654 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2653 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2652 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2651 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2650 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2649 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2648 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2647 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2646 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2645 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2644 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2643 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2642 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2641 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2640 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2639 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2638 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2637 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2636 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2635 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2634 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2633 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2632 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2631 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2630 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2629 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2628 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2627 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2626 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2625 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2624 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2623 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2622 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2621 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2620 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2619 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2618 2557 812 0 D+ vmwait 0xffffffff819a0e40 swap 2617 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2616 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2615 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2614 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2613 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2612 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2611 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2610 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2609 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2608 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2607 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2606 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2605 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2604 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2603 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2602 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2601 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2600 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2599 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2598 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2597 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2596 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2595 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2594 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2593 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2592 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2591 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2590 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2589 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2588 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2587 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2586 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2585 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2584 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2583 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2582 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2581 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2580 2557 812 0 D+ vmpfw 0xfffff8084ad10010 swap 2579 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2578 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2577 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2576 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2575 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2574 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2573 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2572 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2571 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2570 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2569 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2568 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2567 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2566 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2565 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2564 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2563 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2562 2557 812 0 D+ vmwait 0xffffffff819a0e40 swap 2561 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2560 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2559 2557 812 0 DW+ pfault 0xffffffff819a0e40 swap 2558 2557 812 0 D+ ufs 0xfffff8001bb3da48 swap 2557 2555 812 0 ?W+ swap 2555 1419 812 0 ?W+ swap 1419 1417 812 0 ?W+ sh 1417 812 812 0 ?W+ sh 812 805 812 0 ?W+ sh 805 801 805 0 ?W+ bash 801 800 801 0 ?W+ csh 800 799 800 2006 ?W+ su 799 1 799 2006 ?Ws+ bash 795 699 795 0 Ds ufs 0xfffff8001bb3da48 sshd 791 1 791 65 Ds ufs 0xfffff8001bb3da48 dhclient 746 1 746 0 ?Ws+ login 703 1 703 0 ?Ws cron 699 1 699 0 Ss select 0xfffff800039303c0 sshd 576 1 576 0 Ds ufs 0xfffff8001bb3da48 casperd 536 1 536 0 Ds ufs 0xfffff8001bb3da48 rpcbind 531 1 531 0 Ds ufs 0xfffff8001bb3da48 syslogd 33 0 0 0 DL vlruwt 0xfffff8001b379a70 [vnlru] 32 0 0 0 DL syncer 0xffffffff81ee9c00 [syncer] 31 0 0 0 DL (threaded) [bufdaemon] 100130 D qsleep 0xffffffff81ee90a8 [bufdaemon] 100133 D - 0xffffffff81963780 [bufspacedaemon-0] 100134 D - 0xffffffff8196ba40 [bufspacedaemon-1] 100135 D - 0xffffffff81973d00 [bufspacedaemon-2] 100136 D - 0xffffffff8197bfc0 [bufspacedaemon-3] 100137 D - 0xffffffff81984280 [bufspacedaemon-4] 100138 D bufspac 0xffffffff8198c4f8 [bufspacedaemon-5] 100139 D - 0xffffffff81994800 [bufspacedaemon-6] 100151 D sdflush 0xfffff8001be516e8 [/ worker] 100198 D sdflush 0xfffff8014b1d0ce8 [/tmp worker] 30 0 0 0 DL psleep 0xffffffff81ef5618 [vmdaemon] 29 0 0 0 DL (threaded) [pagedaemon] 100126 D pwait 0xffffffff81d40570 [dom0] 100128 D grbmaw 0xfffff8084ad10010 [laundry: dom0] 100129 D umarcl 0xffffffff80dcf2f0 [uma] 28 0 0 0 DL idle 0xfffff8001b28d000 [enc_daemon1] 27 0 0 0 DL idle 0xfffff8001b28d538 [enc_daemon0] 26 0 0 0 DL - 0xffffffff81c147c8 [rand_harvestq] 25 0 0 0 DL waiting 0xffffffff81f630c0 [sctp_iterator] 24 0 0 0 DL - 0xffffffff81ee8aa4 [soaiod4] 23 0 0 0 DL - 0xffffffff81ee8aa4 [soaiod3] 22 0 0 0 DL - 0xffffffff81ee8aa4 [soaiod2] 21 0 0 0 DL - 0xffffffff81ee8aa4 [soaiod1] 20 0 0 0 DL (threaded) [usb] 100094 D - 0xfffffe009c7e1460 [usbus0] 100095 D - 0xfffffe009c7e14b8 [usbus0] 100096 D - 0xfffffe009c7e1510 [usbus0] 100097 D - 0xfffffe009c7e1568 [usbus0] 100098 D - 0xfffffe009c7e15c0 [usbus0] 100100 D - 0xfffffe009a5eed10 [usbus1] 100101 D - 0xfffffe009a5eed68 [usbus1] 100102 D - 0xfffffe009a5eedc0 [usbus1] 100103 D - 0xfffffe009a5eee18 [usbus1] 100104 D - 0xfffffe009a5eee70 [usbus1] 100105 D - 0xfffffe009ca62d10 [usbus2] 100106 D - 0xfffffe009ca62d68 [usbus2] 100107 D - 0xfffffe009ca62dc0 [usbus2] 100108 D - 0xfffffe009ca62e18 [usbus2] 100109 D - 0xfffffe009ca62e70 [usbus2] 19 0 0 0 DL (threaded) [cam] 100089 D - 0xffffffff81aefac0 [doneq0] 100090 D - 0xffffffff81aefb40 [doneq1] 100091 D - 0xffffffff81aefbc0 [doneq2] 100123 D - 0xffffffff81aef988 [scanner] 18 0 0 0 DL crypto_ 0xfffff80003ad2450 [crypto returns 11] 17 0 0 0 DL crypto_ 0xfffff80003ad23f0 [crypto returns 10] 16 0 0 0 DL crypto_ 0xfffff80003ad2390 [crypto returns 9] 15 0 0 0 DL crypto_ 0xfffff80003ad2330 [crypto returns 8] 14 0 0 0 DL crypto_ 0xfffff80003ad22d0 [crypto returns 7] 9 0 0 0 DL crypto_ 0xfffff80003ad2270 [crypto returns 6] 8 0 0 0 DL crypto_ 0xfffff80003ad2210 [crypto returns 5] 7 0 0 0 DL crypto_ 0xfffff80003ad21b0 [crypto returns 4] 6 0 0 0 DL crypto_ 0xfffff80003ad2150 [crypto returns 3] 5 0 0 0 DL crypto_ 0xfffff80003ad20f0 [crypto returns 2] 4 0 0 0 DL crypto_ 0xfffff80003ad2090 [crypto returns 1] 3 0 0 0 DL crypto_ 0xfffff80003ad2030 [crypto returns 0] 2 0 0 0 DL crypto_ 0xffffffff81eefe58 [crypto] 13 0 0 0 DL (threaded) [geom] 100060 D - 0xffffffff81f3fb20 [g_event] 100061 D - 0xffffffff81f3fb28 [g_up] 100062 D - 0xffffffff81f3fb30 [g_down] 12 0 0 0 WL (threaded) [intr] 100015 I [swi6: task queue] 100016 I [swi6: Giant taskq] 100018 I [swi5: fast taskq] 100046 I [swi4: clock (0)] 100047 I [swi4: clock (1)] 100048 I [swi4: clock (2)] 100049 I [swi4: clock (3)] 100050 I [swi4: clock (4)] 100051 I [swi4: clock (5)] 100052 I [swi4: clock (6)] 100053 I [swi4: clock (7)] 100054 I [swi4: clock (8)] 100055 I [swi4: clock (9)] 100056 I [swi4: clock (10)] 100057 I [swi4: clock (11)] 100058 I [swi1: netisr 0] 100059 I [swi3: vm] 100092 I [irq264: ahci0] 100093 I [irq19: xhci0] 100099 I [irq18: ehci0 ehci1] 100110 I [irq279: ahci1] 100111 I [swi0: uart uart] 11 0 0 0 RL (threaded) [idle] 100003 Run CPU 0 [idle: cpu0] 100004 Run CPU 1 [idle: cpu1] 100005 Run CPU 2 [idle: cpu2] 100006 Run CPU 3 [idle: cpu3] 100007 Run CPU 4 [idle: cpu4] 100008 Run CPU 5 [idle: cpu5] 100009 Run CPU 6 [idle: cpu6] 100010 Run CPU 7 [idle: cpu7] 100011 Run CPU 8 [idle: cpu8] 100012 Run CPU 9 [idle: cpu9] 100013 Run CPU 10 [idle: cpu10] 100014 Run CPU 11 [idle: cpu11] 1 0 1 0 SLs wait 0xfffff80003934538 [init] 10 0 0 0 DL audit_w 0xffffffff81f63da8 [audit] 0 0 0 0 DLs (threaded) [kernel] 100000 D vmwait 0xffffffff819a0e40 [swapper] 100017 D - 0xfffff80003970600 [thread taskq] 100019 D - 0xfffff80003970100 [kqueue_ctx taskq] 100020 D - 0xfffff8000396fe00 [aiod_kick taskq] 100021 D - 0xfffff8000397e000 [if_config_tqg_0] 100022 D - 0xfffff8000397de00 [softirq_0] 100023 D - 0xfffff8000397dd00 [softirq_1] 100024 D - 0xfffff8000397dc00 [softirq_2] 100025 D - 0xfffff8000397db00 [softirq_3] 100026 D - 0xfffff8000397da00 [softirq_4] 100027 D - 0xfffff8000397d900 [softirq_5] 100028 D - 0xfffff8000397d800 [softirq_6] 100029 D - 0xfffff8000397d700 [softirq_7] 100030 D - 0xfffff8000397d600 [softirq_8] 100031 D - 0xfffff8000397d500 [softirq_9] 100032 D - 0xfffff8000397d400 [softirq_10] 100033 D - 0xfffff8000397d300 [softirq_11] 100034 D - 0xfffff8000397d200 [if_io_tqg_0] 100035 D - 0xfffff8000397d100 [if_io_tqg_1] 100036 D - 0xfffff8000397d000 [if_io_tqg_2] 100037 D - 0xfffff8000397ce00 [if_io_tqg_3] 100038 D - 0xfffff8000397cd00 [if_io_tqg_4] 100039 D - 0xfffff8000397cc00 [if_io_tqg_5] 100040 D - 0xfffff8000397cb00 [if_io_tqg_6] 100041 D - 0xfffff8000397ca00 [if_io_tqg_7] 100042 D - 0xfffff8000397c900 [if_io_tqg_8] 100043 D - 0xfffff8000397c800 [if_io_tqg_9] 100044 D - 0xfffff8000397c700 [if_io_tqg_10] 100045 D - 0xfffff8000397c600 [if_io_tqg_11] 100063 D - 0xfffff80003a0c100 [firmware taskq] 100064 D - 0xfffff80003a0be00 [crypto_0] 100065 D - 0xfffff80003a0be00 [crypto_1] 100066 D - 0xfffff80003a0be00 [crypto_2] 100067 D - 0xfffff80003a0be00 [crypto_3] 100068 D - 0xfffff80003a0be00 [crypto_4] 100069 D - 0xfffff80003a0be00 [crypto_5] 100070 D - 0xfffff80003a0be00 [crypto_6] 100071 D - 0xfffff80003a0be00 [crypto_7] 100072 D - 0xfffff80003a0be00 [crypto_8] 100073 D - 0xfffff80003a0be00 [crypto_9] 100074 D - 0xfffff80003a0be00 [crypto_10] 100075 D - 0xfffff80003a0be00 [crypto_11] 100112 D - 0xfffff8000c0bec00 [mca taskq] 100118 D - 0xfffff8001b38d700 [acpi_task_0] 100119 D - 0xfffff8001b38d700 [acpi_task_1] 100120 D - 0xfffff8001b38d700 [acpi_task_2] 100122 D - 0xfffff80003971300 [CAM taskq] 1418 812 812 0 Z+ tee 798 795 795 2006 Z sshd 792 746 792 0 Z+ csh 1418 812 812 0 Z+ tee db:0:pho> allt Tracing command getty pid 2934 tid 100189 td 0xfffff80050abe000 sched_switch() at sched_switch+0x942/frame 0xfffffe009eefe0b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe009eefe0e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009eefe120 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009eefe150 sleeplk() at sleeplk+0x149/frame 0xfffffe009eefe1b0 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe009eefe230 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe009eefe2d0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe009eefe320 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe009eefe350 _vn_lock() at _vn_lock+0x66/frame 0xfffffe009eefe3c0 vget() at vget+0x98/frame 0xfffffe009eefe400 cache_lookup() at cache_lookup+0x6fa/frame 0xfffffe009eefe4e0 vfs_cache_lookup() at vfs_cache_lookup+0xac/frame 0xfffffe009eefe540 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xf7/frame 0xfffffe009eefe570 lookup() at lookup+0x692/frame 0xfffffe009eefe610 namei() at namei+0x53b/frame 0xfffffe009eefe6d0 vn_open_cred() at vn_open_cred+0x207/frame 0xfffffe009eefe810 kern_openat() at kern_openat+0x20c/frame 0xfffffe009eefe980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009eefeab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009eefeab0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x800221eea, rsp = 0x7fffffffd228, rbp = 0x7fffffffd310 --- Tracing command getty pid 2933 tid 100225 td 0xfffff8001badb560 sched_switch() at sched_switch+0x942/frame 0xfffffe009efad0b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe009efad0e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009efad120 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009efad150 sleeplk() at sleeplk+0x149/frame 0xfffffe009efad1b0 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe009efad230 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe009efad2d0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe009efad320 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe009efad350 _vn_lock() at _vn_lock+0x66/frame 0xfffffe009efad3c0 vget() at vget+0x98/frame 0xfffffe009efad400 cache_lookup() at cache_lookup+0x6fa/frame 0xfffffe009efad4e0 vfs_cache_lookup() at vfs_cache_lookup+0xac/frame 0xfffffe009efad540 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xf7/frame 0xfffffe009efad570 lookup() at lookup+0x692/frame 0xfffffe009efad610 namei() at namei+0x53b/frame 0xfffffe009efad6d0 vn_open_cred() at vn_open_cred+0x207/frame 0xfffffe009efad810 kern_openat() at kern_openat+0x20c/frame 0xfffffe009efad980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009efadab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009efadab0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x800221eea, rsp = 0x7fffffffd228, rbp = 0x7fffffffd310 --- Tracing command getty pid 2932 tid 100201 td 0xfffff8001badd560 sched_switch() at sched_switch+0x942/frame 0xfffffe009ef350b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ef350e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ef35120 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009ef35150 sleeplk() at sleeplk+0x149/frame 0xfffffe009ef351b0 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe009ef35230 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe009ef352d0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe009ef35320 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe009ef35350 _vn_lock() at _vn_lock+0x66/frame 0xfffffe009ef353c0 vget() at vget+0x98/frame 0xfffffe009ef35400 cache_lookup() at cache_lookup+0x6fa/frame 0xfffffe009ef354e0 vfs_cache_lookup() at vfs_cache_lookup+0xac/frame 0xfffffe009ef35540 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xf7/frame 0xfffffe009ef35570 lookup() at lookup+0x692/frame 0xfffffe009ef35610 namei() at namei+0x53b/frame 0xfffffe009ef356d0 vn_open_cred() at vn_open_cred+0x207/frame 0xfffffe009ef35810 kern_openat() at kern_openat+0x20c/frame 0xfffffe009ef35980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ef35ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ef35ab0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x800221eea, rsp = 0x7fffffffd228, rbp = 0x7fffffffd310 --- Tracing command getty pid 2931 tid 100210 td 0xfffff8005005c560 sched_switch() at sched_switch+0x942/frame 0xfffffe009ef620b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ef620e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ef62120 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009ef62150 sleeplk() at sleeplk+0x149/frame 0xfffffe009ef621b0 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe009ef62230 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe009ef622d0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe009ef62320 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe009ef62350 _vn_lock() at _vn_lock+0x66/frame 0xfffffe009ef623c0 vget() at vget+0x98/frame 0xfffffe009ef62400 cache_lookup() at cache_lookup+0x6fa/frame 0xfffffe009ef624e0 vfs_cache_lookup() at vfs_cache_lookup+0xac/frame 0xfffffe009ef62540 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xf7/frame 0xfffffe009ef62570 lookup() at lookup+0x692/frame 0xfffffe009ef62610 namei() at namei+0x53b/frame 0xfffffe009ef626d0 vn_open_cred() at vn_open_cred+0x207/frame 0xfffffe009ef62810 kern_openat() at kern_openat+0x20c/frame 0xfffffe009ef62980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ef62ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ef62ab0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x800221eea, rsp = 0x7fffffffd228, rbp = 0x7fffffffd310 --- Tracing command getty pid 2930 tid 100146 td 0xfffff8001bd64560 sched_switch() at sched_switch+0x942/frame 0xfffffe009ee270b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ee270e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ee27120 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009ee27150 sleeplk() at sleeplk+0x149/frame 0xfffffe009ee271b0 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe009ee27230 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe009ee272d0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe009ee27320 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe009ee27350 _vn_lock() at _vn_lock+0x66/frame 0xfffffe009ee273c0 vget() at vget+0x98/frame 0xfffffe009ee27400 cache_lookup() at cache_lookup+0x6fa/frame 0xfffffe009ee274e0 vfs_cache_lookup() at vfs_cache_lookup+0xac/frame 0xfffffe009ee27540 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xf7/frame 0xfffffe009ee27570 lookup() at lookup+0x692/frame 0xfffffe009ee27610 namei() at namei+0x53b/frame 0xfffffe009ee276d0 vn_open_cred() at vn_open_cred+0x207/frame 0xfffffe009ee27810 kern_openat() at kern_openat+0x20c/frame 0xfffffe009ee27980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ee27ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ee27ab0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x800221eea, rsp = 0x7fffffffd228, rbp = 0x7fffffffd310 --- Tracing command getty pid 2929 tid 100185 td 0xfffff80050acc000 sched_switch() at sched_switch+0x942/frame 0xfffffe009eeea0b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe009eeea0e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009eeea120 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009eeea150 sleeplk() at sleeplk+0x149/frame 0xfffffe009eeea1b0 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe009eeea230 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe009eeea2d0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe009eeea320 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe009eeea350 _vn_lock() at _vn_lock+0x66/frame 0xfffffe009eeea3c0 vget() at vget+0x98/frame 0xfffffe009eeea400 cache_lookup() at cache_lookup+0x6fa/frame 0xfffffe009eeea4e0 vfs_cache_lookup() at vfs_cache_lookup+0xac/frame 0xfffffe009eeea540 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xf7/frame 0xfffffe009eeea570 lookup() at lookup+0x692/frame 0xfffffe009eeea610 namei() at namei+0x53b/frame 0xfffffe009eeea6d0 vn_open_cred() at vn_open_cred+0x207/frame 0xfffffe009eeea810 kern_openat() at kern_openat+0x20c/frame 0xfffffe009eeea980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009eeeaab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009eeeaab0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x800221eea, rsp = 0x7fffffffd228, rbp = 0x7fffffffd310 --- Tracing command getty pid 2928 tid 100186 td 0xfffff80050abf560 sched_switch() at sched_switch+0x942/frame 0xfffffe009eeef0b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe009eeef0e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009eeef120 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009eeef150 sleeplk() at sleeplk+0x149/frame 0xfffffe009eeef1b0 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe009eeef230 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe009eeef2d0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe009eeef320 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe009eeef350 _vn_lock() at _vn_lock+0x66/frame 0xfffffe009eeef3c0 vget() at vget+0x98/frame 0xfffffe009eeef400 cache_lookup() at cache_lookup+0x6fa/frame 0xfffffe009eeef4e0 vfs_cache_lookup() at vfs_cache_lookup+0xac/frame 0xfffffe009eeef540 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xf7/frame 0xfffffe009eeef570 lookup() at lookup+0x692/frame 0xfffffe009eeef610 namei() at namei+0x53b/frame 0xfffffe009eeef6d0 vn_open_cred() at vn_open_cred+0x207/frame 0xfffffe009eeef810 kern_openat() at kern_openat+0x20c/frame 0xfffffe009eeef980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009eeefab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009eeefab0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x800221eea, rsp = 0x7fffffffd228, rbp = 0x7fffffffd310 --- Tracing command getty pid 2927 tid 100187 td 0xfffff80050abf000 sched_switch() at sched_switch+0x942/frame 0xfffffe009eef40b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe009eef40e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009eef4120 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009eef4150 sleeplk() at sleeplk+0x149/frame 0xfffffe009eef41b0 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe009eef4230 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe009eef42d0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe009eef4320 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe009eef4350 _vn_lock() at _vn_lock+0x66/frame 0xfffffe009eef43c0 vget() at vget+0x98/frame 0xfffffe009eef4400 cache_lookup() at cache_lookup+0x6fa/frame 0xfffffe009eef44e0 vfs_cache_lookup() at vfs_cache_lookup+0xac/frame 0xfffffe009eef4540 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xf7/frame 0xfffffe009eef4570 lookup() at lookup+0x692/frame 0xfffffe009eef4610 namei() at namei+0x53b/frame 0xfffffe009eef46d0 vn_open_cred() at vn_open_cred+0x207/frame 0xfffffe009eef4810 kern_openat() at kern_openat+0x20c/frame 0xfffffe009eef4980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009eef4ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009eef4ab0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x800221eea, rsp = 0x7fffffffd228, rbp = 0x7fffffffd310 --- Tracing command getty pid 2926 tid 100188 td 0xfffff80050abe560 sched_switch() at sched_switch+0x942/frame 0xfffffe009eef90b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe009eef90e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009eef9120 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009eef9150 sleeplk() at sleeplk+0x149/frame 0xfffffe009eef91b0 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe009eef9230 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe009eef92d0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe009eef9320 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe009eef9350 _vn_lock() at _vn_lock+0x66/frame 0xfffffe009eef93c0 vget() at vget+0x98/frame 0xfffffe009eef9400 cache_lookup() at cache_lookup+0x6fa/frame 0xfffffe009eef94e0 vfs_cache_lookup() at vfs_cache_lookup+0xac/frame 0xfffffe009eef9540 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xf7/frame 0xfffffe009eef9570 lookup() at lookup+0x692/frame 0xfffffe009eef9610 namei() at namei+0x53b/frame 0xfffffe009eef96d0 vn_open_cred() at vn_open_cred+0x207/frame 0xfffffe009eef9810 kern_openat() at kern_openat+0x20c/frame 0xfffffe009eef9980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009eef9ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009eef9ab0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x800221eea, rsp = 0x7fffffffd228, rbp = 0x7fffffffd310 --- Tracing command datamove2 pid 2658 tid 100200 td 0xfffff80050abc000 sched_switch() at sched_switch+0x942/frame 0xfffffe009ef30520 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ef30550 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ef30590 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009ef305c0 _sleep() at _sleep+0x367/frame 0xfffffe009ef30670 vm_wait_doms() at vm_wait_doms+0x85/frame 0xfffffe009ef30690 vm_wait_domain() at vm_wait_domain+0x68/frame 0xfffffe009ef306c0 vm_domain_alloc_fail() at vm_domain_alloc_fail+0x94/frame 0xfffffe009ef306f0 vm_page_alloc_domain_after() at vm_page_alloc_domain_after+0x3ee/frame 0xfffffe009ef30770 vm_page_alloc() at vm_page_alloc+0x64/frame 0xfffffe009ef307d0 pmap_pinit_type() at pmap_pinit_type+0x28/frame 0xfffffe009ef30810 pmap_pinit() at pmap_pinit+0x10/frame 0xfffffe009ef30820 vmspace_fork() at vmspace_fork+0x55/frame 0xfffffe009ef30880 fork1() at fork1+0x4ce/frame 0xfffffe009ef30930 sys_fork() at sys_fork+0x4c/frame 0xfffffe009ef30980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ef30ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ef30ab0 --- syscall (2, FreeBSD ELF64, sys_fork), rip = 0x80079ca0a, rsp = 0x7fffffffe6e8, rbp = 0x7fffffffe760 --- Tracing command swap pid 2657 tid 100222 td 0xfffff8014b45a000 sched_switch() at sched_switch+0x942/frame 0xfffffe009ef9e490 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ef9e4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ef9e500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009ef9e530 sleeplk() at sleeplk+0x149/frame 0xfffffe009ef9e590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe009ef9e610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe009ef9e6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe009ef9e700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe009ef9e730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe009ef9e7a0 vget() at vget+0x98/frame 0xfffffe009ef9e7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe009ef9e910 vm_fault() at vm_fault+0x75/frame 0xfffffe009ef9e950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe009ef9e9a0 trap() at trap+0x3be/frame 0xfffffe009ef9eab0 calltrap() at calltrap+0x8/frame 0xfffffe009ef9eab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2656 tid 100303 td 0xfffff8014bb23000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command swap pid 2655 tid 100302 td 0xfffff8014bb23560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0f30490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0f304c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0f30500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0f30530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0f30590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0f30610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0f306b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0f30700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0f30730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0f307a0 vget() at vget+0x98/frame 0xfffffe00a0f307e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0f30910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0f30950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0f309a0 trap() at trap+0x3be/frame 0xfffffe00a0f30ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0f30ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2654 tid 100301 td 0xfffff8014bb24000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command swap pid 2653 tid 100315 td 0xfffff8014bcc4000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command swap pid 2652 tid 100314 td 0xfffff8014bcc4560 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command swap pid 2651 tid 100313 td 0xfffff8014bcc5000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0f67490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0f674c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0f67500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0f67530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0f67590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0f67610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0f676b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0f67700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0f67730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0f677a0 vget() at vget+0x98/frame 0xfffffe00a0f677e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0f67910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0f67950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0f679a0 trap() at trap+0x3be/frame 0xfffffe00a0f67ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0f67ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2650 tid 100312 td 0xfffff8014bb27560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0f62490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0f624c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0f62500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0f62530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0f62590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0f62610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0f626b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0f62700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0f62730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0f627a0 vget() at vget+0x98/frame 0xfffffe00a0f627e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0f62910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0f62950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0f629a0 trap() at trap+0x3be/frame 0xfffffe00a0f62ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0f62ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2649 tid 100311 td 0xfffff8014bbf2000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command swap pid 2648 tid 100310 td 0xfffff8014bbf2560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0f58490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0f584c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0f58500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0f58530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0f58590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0f58610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0f586b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0f58700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0f58730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0f587a0 vget() at vget+0x98/frame 0xfffffe00a0f587e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0f58910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0f58950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0f589a0 trap() at trap+0x3be/frame 0xfffffe00a0f58ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0f58ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2647 tid 100294 td 0xfffff8014b987560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0f08490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0f084c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0f08500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0f08530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0f08590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0f08610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0f086b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0f08700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0f08730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0f087a0 vget() at vget+0x98/frame 0xfffffe00a0f087e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0f08910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0f08950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0f089a0 trap() at trap+0x3be/frame 0xfffffe00a0f08ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0f08ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2646 tid 100293 td 0xfffff8014ba53000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command swap pid 2645 tid 100292 td 0xfffff8014ba53560 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command swap pid 2644 tid 100309 td 0xfffff8014bbf3000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0f53490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0f534c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0f53500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0f53530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0f53590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0f53610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0f536b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0f53700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0f53730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0f537a0 vget() at vget+0x98/frame 0xfffffe00a0f537e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0f53910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0f53950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0f539a0 trap() at trap+0x3be/frame 0xfffffe00a0f53ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0f53ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2643 tid 100308 td 0xfffff8014bbf3560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0f4e490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0f4e4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0f4e500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0f4e530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0f4e590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0f4e610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0f4e6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0f4e700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0f4e730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0f4e7a0 vget() at vget+0x98/frame 0xfffffe00a0f4e7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0f4e910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0f4e950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0f4e9a0 trap() at trap+0x3be/frame 0xfffffe00a0f4eab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0f4eab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2642 tid 100307 td 0xfffff8014bbf4000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command swap pid 2641 tid 100306 td 0xfffff8014bbf4560 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command swap pid 2640 tid 100305 td 0xfffff8014bbf5000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0f3f490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0f3f4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0f3f500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0f3f530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0f3f590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0f3f610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0f3f6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0f3f700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0f3f730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0f3f7a0 vget() at vget+0x98/frame 0xfffffe00a0f3f7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0f3f910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0f3f950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0f3f9a0 trap() at trap+0x3be/frame 0xfffffe00a0f3fab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0f3fab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2639 tid 100304 td 0xfffff8014bbf5560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0f3a490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0f3a4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0f3a500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0f3a530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0f3a590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0f3a610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0f3a6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0f3a700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0f3a730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0f3a7a0 vget() at vget+0x98/frame 0xfffffe00a0f3a7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0f3a910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0f3a950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0f3a9a0 trap() at trap+0x3be/frame 0xfffffe00a0f3aab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0f3aab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2638 tid 100285 td 0xfffff8014b97c000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0edb490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0edb4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0edb500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0edb530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0edb590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0edb610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0edb6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0edb700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0edb730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0edb7a0 vget() at vget+0x98/frame 0xfffffe00a0edb7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0edb910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0edb950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0edb9a0 trap() at trap+0x3be/frame 0xfffffe00a0edbab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0edbab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2637 tid 100284 td 0xfffff8014b97c560 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command swap pid 2636 tid 100283 td 0xfffff8014b97d000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0ed1490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0ed14c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0ed1500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0ed1530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0ed1590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0ed1610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0ed16b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0ed1700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0ed1730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0ed17a0 vget() at vget+0x98/frame 0xfffffe00a0ed17e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0ed1910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0ed1950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0ed19a0 trap() at trap+0x3be/frame 0xfffffe00a0ed1ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0ed1ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2635 tid 100300 td 0xfffff8014bb24560 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command swap pid 2634 tid 100299 td 0xfffff8014bb25000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0f21490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0f214c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0f21500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0f21530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0f21590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0f21610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0f216b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0f21700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0f21730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0f217a0 vget() at vget+0x98/frame 0xfffffe00a0f217e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0f21910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0f21950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0f219a0 trap() at trap+0x3be/frame 0xfffffe00a0f21ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0f21ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2633 tid 100298 td 0xfffff8014bb25560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0f1c490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0f1c4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0f1c500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0f1c530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0f1c590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0f1c610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0f1c6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0f1c700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0f1c730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0f1c7a0 vget() at vget+0x98/frame 0xfffffe00a0f1c7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0f1c910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0f1c950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0f1c9a0 trap() at trap+0x3be/frame 0xfffffe00a0f1cab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0f1cab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2632 tid 100297 td 0xfffff8014bb26000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command swap pid 2631 tid 100296 td 0xfffff8014bb26560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0f12490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0f124c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0f12500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0f12530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0f12590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0f12610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0f126b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0f12700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0f12730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0f127a0 vget() at vget+0x98/frame 0xfffffe00a0f127e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0f12910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0f12950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0f129a0 trap() at trap+0x3be/frame 0xfffffe00a0f12ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0f12ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2630 tid 100295 td 0xfffff8014bb27000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0f0d490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0f0d4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0f0d500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0f0d530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0f0d590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0f0d610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0f0d6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0f0d700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0f0d730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0f0d7a0 vget() at vget+0x98/frame 0xfffffe00a0f0d7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0f0d910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0f0d950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0f0d9a0 trap() at trap+0x3be/frame 0xfffffe00a0f0dab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0f0dab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2629 tid 100276 td 0xfffff8014b7e4560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0eae490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0eae4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0eae500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0eae530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0eae590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0eae610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0eae6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0eae700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0eae730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0eae7a0 vget() at vget+0x98/frame 0xfffffe00a0eae7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0eae910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0eae950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0eae9a0 trap() at trap+0x3be/frame 0xfffffe00a0eaeab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0eaeab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2628 tid 100275 td 0xfffff8014b8ad000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command swap pid 2627 tid 100274 td 0xfffff8014b8ad560 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command swap pid 2626 tid 100291 td 0xfffff8014ba54000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0ef9490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0ef94c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0ef9500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0ef9530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0ef9590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0ef9610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0ef96b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0ef9700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0ef9730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0ef97a0 vget() at vget+0x98/frame 0xfffffe00a0ef97e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0ef9910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0ef9950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0ef99a0 trap() at trap+0x3be/frame 0xfffffe00a0ef9ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0ef9ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2625 tid 100290 td 0xfffff8014ba54560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0ef4490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0ef44c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0ef4500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0ef4530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0ef4590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0ef4610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0ef46b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0ef4700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0ef4730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0ef47a0 vget() at vget+0x98/frame 0xfffffe00a0ef47e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0ef4910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0ef4950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0ef49a0 trap() at trap+0x3be/frame 0xfffffe00a0ef4ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0ef4ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2624 tid 100289 td 0xfffff8014ba55000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0eef490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0eef4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0eef500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0eef530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0eef590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0eef610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0eef6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0eef700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0eef730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0eef7a0 vget() at vget+0x98/frame 0xfffffe00a0eef7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0eef910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0eef950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0eef9a0 trap() at trap+0x3be/frame 0xfffffe00a0eefab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0eefab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2623 tid 100288 td 0xfffff8014ba55560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0eea490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0eea4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0eea500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0eea530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0eea590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0eea610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0eea6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0eea700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0eea730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0eea7a0 vget() at vget+0x98/frame 0xfffffe00a0eea7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0eea910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0eea950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0eea9a0 trap() at trap+0x3be/frame 0xfffffe00a0eeaab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0eeaab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2622 tid 100287 td 0xfffff8014ba56000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0ee5490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0ee54c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0ee5500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0ee5530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0ee5590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0ee5610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0ee56b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0ee5700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0ee5730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0ee57a0 vget() at vget+0x98/frame 0xfffffe00a0ee57e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0ee5910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0ee5950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0ee59a0 trap() at trap+0x3be/frame 0xfffffe00a0ee5ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0ee5ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2621 tid 100286 td 0xfffff8014ba56560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0ee0490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0ee04c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0ee0500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0ee0530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0ee0590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0ee0610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0ee06b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0ee0700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0ee0730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0ee07a0 vget() at vget+0x98/frame 0xfffffe00a0ee07e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0ee0910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0ee0950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0ee09a0 trap() at trap+0x3be/frame 0xfffffe00a0ee0ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0ee0ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2620 tid 100267 td 0xfffff8014b7e0000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e81490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e814c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e81500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e81530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0e81590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0e81610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0e816b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0e81700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0e81730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0e817a0 vget() at vget+0x98/frame 0xfffffe00a0e817e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0e81910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e81950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e819a0 trap() at trap+0x3be/frame 0xfffffe00a0e81ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e81ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2619 tid 100266 td 0xfffff8014b7e0560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e7c490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e7c4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e7c500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e7c530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0e7c590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0e7c610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0e7c6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0e7c700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0e7c730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0e7c7a0 vget() at vget+0x98/frame 0xfffffe00a0e7c7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0e7c910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e7c950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e7c9a0 trap() at trap+0x3be/frame 0xfffffe00a0e7cab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e7cab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2618 tid 100265 td 0xfffff8014b7e1000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e77540 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e77570 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e775b0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e775e0 _sleep() at _sleep+0x367/frame 0xfffffe00a0e77690 vm_wait_doms() at vm_wait_doms+0x85/frame 0xfffffe00a0e776b0 _pmap_allocpte() at _pmap_allocpte+0x1e5/frame 0xfffffe00a0e77720 pmap_enter() at pmap_enter+0x532/frame 0xfffffe00a0e777e0 vm_fault_hold() at vm_fault_hold+0x1cd4/frame 0xfffffe00a0e77910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e77950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e779a0 trap() at trap+0x3be/frame 0xfffffe00a0e77ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e77ab0 --- trap 0xc, rip = 0x8008292c4, rsp = 0x7fffffffe208, rbp = 0x7fffffffe250 --- Tracing command swap pid 2617 tid 100282 td 0xfffff8014b97d560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0ecc490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0ecc4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0ecc500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0ecc530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0ecc590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0ecc610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0ecc6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0ecc700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0ecc730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0ecc7a0 vget() at vget+0x98/frame 0xfffffe00a0ecc7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0ecc910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0ecc950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0ecc9a0 trap() at trap+0x3be/frame 0xfffffe00a0eccab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0eccab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2616 tid 100281 td 0xfffff8014b97e000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command swap pid 2615 tid 100280 td 0xfffff8014b97e560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0ec2490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0ec24c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0ec2500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0ec2530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0ec2590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0ec2610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0ec26b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0ec2700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0ec2730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0ec27a0 vget() at vget+0x98/frame 0xfffffe00a0ec27e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0ec2910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0ec2950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0ec29a0 trap() at trap+0x3be/frame 0xfffffe00a0ec2ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0ec2ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2614 tid 100279 td 0xfffff8014b983000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0ebd490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0ebd4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0ebd500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0ebd530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0ebd590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0ebd610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0ebd6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0ebd700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0ebd730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0ebd7a0 vget() at vget+0x98/frame 0xfffffe00a0ebd7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0ebd910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0ebd950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0ebd9a0 trap() at trap+0x3be/frame 0xfffffe00a0ebdab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0ebdab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2613 tid 100278 td 0xfffff8014b983560 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command swap pid 2612 tid 100277 td 0xfffff8014b987000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0eb3490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0eb34c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0eb3500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0eb3530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0eb3590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0eb3610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0eb36b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0eb3700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0eb3730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0eb37a0 vget() at vget+0x98/frame 0xfffffe00a0eb37e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0eb3910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0eb3950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0eb39a0 trap() at trap+0x3be/frame 0xfffffe00a0eb3ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0eb3ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2611 tid 100258 td 0xfffff8014b64b560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e54490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e544c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e54500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e54530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0e54590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0e54610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0e546b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0e54700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0e54730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0e547a0 vget() at vget+0x98/frame 0xfffffe00a0e547e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0e54910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e54950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e549a0 trap() at trap+0x3be/frame 0xfffffe00a0e54ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e54ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2610 tid 100257 td 0xfffff8014b711000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e4f490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e4f4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e4f500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e4f530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0e4f590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0e4f610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0e4f6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0e4f700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0e4f730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0e4f7a0 vget() at vget+0x98/frame 0xfffffe00a0e4f7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0e4f910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e4f950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e4f9a0 trap() at trap+0x3be/frame 0xfffffe00a0e4fab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e4fab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2609 tid 100256 td 0xfffff8014b711560 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command swap pid 2608 tid 100273 td 0xfffff8014b8ae000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e9f490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e9f4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e9f500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e9f530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0e9f590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0e9f610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0e9f6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0e9f700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0e9f730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0e9f7a0 vget() at vget+0x98/frame 0xfffffe00a0e9f7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0e9f910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e9f950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e9f9a0 trap() at trap+0x3be/frame 0xfffffe00a0e9fab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e9fab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2607 tid 100272 td 0xfffff8014b8ae560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e9a490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e9a4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e9a500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e9a530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0e9a590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0e9a610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0e9a6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0e9a700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0e9a730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0e9a7a0 vget() at vget+0x98/frame 0xfffffe00a0e9a7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0e9a910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e9a950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e9a9a0 trap() at trap+0x3be/frame 0xfffffe00a0e9aab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e9aab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2606 tid 100271 td 0xfffff8014b8af000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e95490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e954c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e95500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e95530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0e95590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0e95610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0e956b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0e95700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0e95730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0e957a0 vget() at vget+0x98/frame 0xfffffe00a0e957e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0e95910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e95950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e959a0 trap() at trap+0x3be/frame 0xfffffe00a0e95ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e95ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2605 tid 100270 td 0xfffff8014b8af560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e90490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e904c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e90500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e90530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0e90590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0e90610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0e906b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0e90700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0e90730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0e907a0 vget() at vget+0x98/frame 0xfffffe00a0e907e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0e90910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e90950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e909a0 trap() at trap+0x3be/frame 0xfffffe00a0e90ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e90ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2604 tid 100269 td 0xfffff8014b8b0000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e8b490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e8b4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e8b500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e8b530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0e8b590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0e8b610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0e8b6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0e8b700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0e8b730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0e8b7a0 vget() at vget+0x98/frame 0xfffffe00a0e8b7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0e8b910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e8b950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e8b9a0 trap() at trap+0x3be/frame 0xfffffe00a0e8bab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e8bab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2603 tid 100268 td 0xfffff8014b8b0560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e86490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e864c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e86500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e86530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0e86590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0e86610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0e866b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0e86700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0e86730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0e867a0 vget() at vget+0x98/frame 0xfffffe00a0e867e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0e86910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e86950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e869a0 trap() at trap+0x3be/frame 0xfffffe00a0e86ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e86ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2602 tid 100249 td 0xfffff8014b647000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e27490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e274c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e27500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e27530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0e27590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0e27610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0e276b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0e27700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0e27730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0e277a0 vget() at vget+0x98/frame 0xfffffe00a0e277e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0e27910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e27950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e279a0 trap() at trap+0x3be/frame 0xfffffe00a0e27ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e27ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2601 tid 100248 td 0xfffff8014b647560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e22490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e224c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e22500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e22530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0e22590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0e22610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0e226b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0e22700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0e22730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0e227a0 vget() at vget+0x98/frame 0xfffffe00a0e227e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0e22910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e22950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e229a0 trap() at trap+0x3be/frame 0xfffffe00a0e22ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e22ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2600 tid 100247 td 0xfffff8014b648000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e1d490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e1d4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e1d500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e1d530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0e1d590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0e1d610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0e1d6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0e1d700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0e1d730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0e1d7a0 vget() at vget+0x98/frame 0xfffffe00a0e1d7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0e1d910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e1d950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e1d9a0 trap() at trap+0x3be/frame 0xfffffe00a0e1dab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e1dab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2599 tid 100264 td 0xfffff8014b7e1560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e72490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e724c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e72500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e72530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0e72590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0e72610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0e726b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0e72700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0e72730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0e727a0 vget() at vget+0x98/frame 0xfffffe00a0e727e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0e72910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e72950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e729a0 trap() at trap+0x3be/frame 0xfffffe00a0e72ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e72ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2598 tid 100263 td 0xfffff8014b7e2000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e6d490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e6d4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e6d500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e6d530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0e6d590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0e6d610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0e6d6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0e6d700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0e6d730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0e6d7a0 vget() at vget+0x98/frame 0xfffffe00a0e6d7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0e6d910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e6d950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e6d9a0 trap() at trap+0x3be/frame 0xfffffe00a0e6dab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e6dab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2597 tid 100262 td 0xfffff8014b7e2560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e68490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e684c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e68500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e68530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0e68590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0e68610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0e686b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0e68700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0e68730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0e687a0 vget() at vget+0x98/frame 0xfffffe00a0e687e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0e68910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e68950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e689a0 trap() at trap+0x3be/frame 0xfffffe00a0e68ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e68ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2596 tid 100261 td 0xfffff8014b7e3000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e63490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e634c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e63500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e63530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0e63590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0e63610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0e636b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0e63700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0e63730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0e637a0 vget() at vget+0x98/frame 0xfffffe00a0e637e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0e63910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e63950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e639a0 trap() at trap+0x3be/frame 0xfffffe00a0e63ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e63ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2595 tid 100260 td 0xfffff8014b7e3560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e5e490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e5e4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e5e500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e5e530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0e5e590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0e5e610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0e5e6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0e5e700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0e5e730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0e5e7a0 vget() at vget+0x98/frame 0xfffffe00a0e5e7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0e5e910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e5e950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e5e9a0 trap() at trap+0x3be/frame 0xfffffe00a0e5eab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e5eab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2594 tid 100259 td 0xfffff8014b7e4000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e59490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e594c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e59500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e59530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0e59590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0e59610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0e596b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0e59700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0e59730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0e597a0 vget() at vget+0x98/frame 0xfffffe00a0e597e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0e59910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e59950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e599a0 trap() at trap+0x3be/frame 0xfffffe00a0e59ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e59ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2593 tid 100240 td 0xfffff8014b3ed560 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command swap pid 2592 tid 100239 td 0xfffff8014b581000 sched_switch() at sched_switch+0x942/frame 0xfffffe009eff3490 mi_switch() at mi_switch+0x18c/frame 0xfffffe009eff34c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009eff3500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009eff3530 sleeplk() at sleeplk+0x149/frame 0xfffffe009eff3590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe009eff3610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe009eff36b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe009eff3700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe009eff3730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe009eff37a0 vget() at vget+0x98/frame 0xfffffe009eff37e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe009eff3910 vm_fault() at vm_fault+0x75/frame 0xfffffe009eff3950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe009eff39a0 trap() at trap+0x3be/frame 0xfffffe009eff3ab0 calltrap() at calltrap+0x8/frame 0xfffffe009eff3ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2591 tid 100238 td 0xfffff8014b581560 sched_switch() at sched_switch+0x942/frame 0xfffffe009efee490 mi_switch() at mi_switch+0x18c/frame 0xfffffe009efee4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009efee500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009efee530 sleeplk() at sleeplk+0x149/frame 0xfffffe009efee590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe009efee610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe009efee6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe009efee700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe009efee730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe009efee7a0 vget() at vget+0x98/frame 0xfffffe009efee7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe009efee910 vm_fault() at vm_fault+0x75/frame 0xfffffe009efee950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe009efee9a0 trap() at trap+0x3be/frame 0xfffffe009efeeab0 calltrap() at calltrap+0x8/frame 0xfffffe009efeeab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2590 tid 100255 td 0xfffff8014b712000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command swap pid 2589 tid 100254 td 0xfffff8014b712560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e40490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e404c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e40500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e40530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0e40590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0e40610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0e406b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0e40700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0e40730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0e407a0 vget() at vget+0x98/frame 0xfffffe00a0e407e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0e40910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e40950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e409a0 trap() at trap+0x3be/frame 0xfffffe00a0e40ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e40ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2588 tid 100253 td 0xfffff8014b713000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e3b490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e3b4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e3b500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e3b530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0e3b590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0e3b610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0e3b6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0e3b700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0e3b730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0e3b7a0 vget() at vget+0x98/frame 0xfffffe00a0e3b7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0e3b910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e3b950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e3b9a0 trap() at trap+0x3be/frame 0xfffffe00a0e3bab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e3bab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2587 tid 100252 td 0xfffff8014b713560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e36490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e364c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e36500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e36530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0e36590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0e36610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0e366b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0e36700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0e36730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0e367a0 vget() at vget+0x98/frame 0xfffffe00a0e367e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0e36910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e36950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e369a0 trap() at trap+0x3be/frame 0xfffffe00a0e36ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e36ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2586 tid 100251 td 0xfffff8014b714000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e31490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e314c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e31500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e31530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0e31590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0e31610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0e316b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0e31700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0e31730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0e317a0 vget() at vget+0x98/frame 0xfffffe00a0e317e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0e31910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e31950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e319a0 trap() at trap+0x3be/frame 0xfffffe00a0e31ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e31ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2585 tid 100250 td 0xfffff8014b714560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e2c490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e2c4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e2c500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e2c530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0e2c590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0e2c610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0e2c6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0e2c700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0e2c730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0e2c7a0 vget() at vget+0x98/frame 0xfffffe00a0e2c7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0e2c910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e2c950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e2c9a0 trap() at trap+0x3be/frame 0xfffffe00a0e2cab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e2cab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2584 tid 100231 td 0xfffff8001be8c000 sched_switch() at sched_switch+0x942/frame 0xfffffe009efcb490 mi_switch() at mi_switch+0x18c/frame 0xfffffe009efcb4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009efcb500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009efcb530 sleeplk() at sleeplk+0x149/frame 0xfffffe009efcb590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe009efcb610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe009efcb6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe009efcb700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe009efcb730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe009efcb7a0 vget() at vget+0x98/frame 0xfffffe009efcb7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe009efcb910 vm_fault() at vm_fault+0x75/frame 0xfffffe009efcb950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe009efcb9a0 trap() at trap+0x3be/frame 0xfffffe009efcbab0 calltrap() at calltrap+0x8/frame 0xfffffe009efcbab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2583 tid 100230 td 0xfffff8001be8c560 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command swap pid 2582 tid 100229 td 0xfffff8001be90000 sched_switch() at sched_switch+0x942/frame 0xfffffe009efc1490 mi_switch() at mi_switch+0x18c/frame 0xfffffe009efc14c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009efc1500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009efc1530 sleeplk() at sleeplk+0x149/frame 0xfffffe009efc1590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe009efc1610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe009efc16b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe009efc1700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe009efc1730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe009efc17a0 vget() at vget+0x98/frame 0xfffffe009efc17e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe009efc1910 vm_fault() at vm_fault+0x75/frame 0xfffffe009efc1950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe009efc19a0 trap() at trap+0x3be/frame 0xfffffe009efc1ab0 calltrap() at calltrap+0x8/frame 0xfffffe009efc1ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2581 tid 100246 td 0xfffff8014b648560 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command swap pid 2580 tid 100245 td 0xfffff8014b649000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e13630 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e13660 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e136a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e136d0 _sleep() at _sleep+0x367/frame 0xfffffe00a0e13780 vm_page_busy_sleep() at vm_page_busy_sleep+0xc5/frame 0xfffffe00a0e137b0 vm_page_sleep_if_busy() at vm_page_sleep_if_busy+0xb0/frame 0xfffffe00a0e137e0 vm_fault_hold() at vm_fault_hold+0x2df/frame 0xfffffe00a0e13910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e13950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e139a0 trap() at trap+0x3be/frame 0xfffffe00a0e13ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e13ab0 --- trap 0xc, rip = 0x8008292c4, rsp = 0x7fffffffe208, rbp = 0x7fffffffe250 --- Tracing command swap pid 2579 tid 100244 td 0xfffff8014b649560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e0e490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e0e4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e0e500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e0e530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0e0e590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0e0e610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0e0e6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0e0e700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0e0e730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0e0e7a0 vget() at vget+0x98/frame 0xfffffe00a0e0e7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0e0e910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e0e950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e0e9a0 trap() at trap+0x3be/frame 0xfffffe00a0e0eab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e0eab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2578 tid 100243 td 0xfffff8014b64a000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e09490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e094c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e09500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e09530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0e09590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0e09610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0e096b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0e09700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0e09730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0e097a0 vget() at vget+0x98/frame 0xfffffe00a0e097e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0e09910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e09950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e099a0 trap() at trap+0x3be/frame 0xfffffe00a0e09ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e09ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2577 tid 100242 td 0xfffff8014b64a560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0e04490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0e044c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0e04500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0e04530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0e04590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0e04610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0e046b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0e04700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0e04730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0e047a0 vget() at vget+0x98/frame 0xfffffe00a0e047e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0e04910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0e04950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0e049a0 trap() at trap+0x3be/frame 0xfffffe00a0e04ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0e04ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2576 tid 100241 td 0xfffff8014b64b000 sched_switch() at sched_switch+0x942/frame 0xfffffe009effd490 mi_switch() at mi_switch+0x18c/frame 0xfffffe009effd4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009effd500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009effd530 sleeplk() at sleeplk+0x149/frame 0xfffffe009effd590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe009effd610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe009effd6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe009effd700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe009effd730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe009effd7a0 vget() at vget+0x98/frame 0xfffffe009effd7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe009effd910 vm_fault() at vm_fault+0x75/frame 0xfffffe009effd950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe009effd9a0 trap() at trap+0x3be/frame 0xfffffe009effdab0 calltrap() at calltrap+0x8/frame 0xfffffe009effdab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2575 tid 100228 td 0xfffff8001be90560 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command swap pid 2574 tid 100154 td 0xfffff8001be92560 sched_switch() at sched_switch+0x942/frame 0xfffffe009ee4f490 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ee4f4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ee4f500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009ee4f530 sleeplk() at sleeplk+0x149/frame 0xfffffe009ee4f590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe009ee4f610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe009ee4f6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe009ee4f700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe009ee4f730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe009ee4f7a0 vget() at vget+0x98/frame 0xfffffe009ee4f7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe009ee4f910 vm_fault() at vm_fault+0x75/frame 0xfffffe009ee4f950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe009ee4f9a0 trap() at trap+0x3be/frame 0xfffffe009ee4fab0 calltrap() at calltrap+0x8/frame 0xfffffe009ee4fab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2573 tid 100316 td 0xfffff8014bcc3560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0f76490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0f764c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0f76500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0f76530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0f76590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0f76610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0f766b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0f76700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0f76730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0f767a0 vget() at vget+0x98/frame 0xfffffe00a0f767e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0f76910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0f76950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0f769a0 trap() at trap+0x3be/frame 0xfffffe00a0f76ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0f76ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2572 tid 100237 td 0xfffff8014b582000 sched_switch() at sched_switch+0x942/frame 0xfffffe009efe9490 mi_switch() at mi_switch+0x18c/frame 0xfffffe009efe94c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009efe9500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009efe9530 sleeplk() at sleeplk+0x149/frame 0xfffffe009efe9590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe009efe9610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe009efe96b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe009efe9700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe009efe9730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe009efe97a0 vget() at vget+0x98/frame 0xfffffe009efe97e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe009efe9910 vm_fault() at vm_fault+0x75/frame 0xfffffe009efe9950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe009efe99a0 trap() at trap+0x3be/frame 0xfffffe009efe9ab0 calltrap() at calltrap+0x8/frame 0xfffffe009efe9ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2571 tid 100236 td 0xfffff8014b582560 sched_switch() at sched_switch+0x942/frame 0xfffffe009efe4490 mi_switch() at mi_switch+0x18c/frame 0xfffffe009efe44c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009efe4500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009efe4530 sleeplk() at sleeplk+0x149/frame 0xfffffe009efe4590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe009efe4610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe009efe46b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe009efe4700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe009efe4730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe009efe47a0 vget() at vget+0x98/frame 0xfffffe009efe47e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe009efe4910 vm_fault() at vm_fault+0x75/frame 0xfffffe009efe4950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe009efe49a0 trap() at trap+0x3be/frame 0xfffffe009efe4ab0 calltrap() at calltrap+0x8/frame 0xfffffe009efe4ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2570 tid 100235 td 0xfffff8014b583000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command swap pid 2569 tid 100234 td 0xfffff8014b583560 sched_switch() at sched_switch+0x942/frame 0xfffffe009efda490 mi_switch() at mi_switch+0x18c/frame 0xfffffe009efda4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009efda500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009efda530 sleeplk() at sleeplk+0x149/frame 0xfffffe009efda590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe009efda610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe009efda6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe009efda700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe009efda730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe009efda7a0 vget() at vget+0x98/frame 0xfffffe009efda7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe009efda910 vm_fault() at vm_fault+0x75/frame 0xfffffe009efda950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe009efda9a0 trap() at trap+0x3be/frame 0xfffffe009efdaab0 calltrap() at calltrap+0x8/frame 0xfffffe009efdaab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2568 tid 100233 td 0xfffff8014b584000 sched_switch() at sched_switch+0x942/frame 0xfffffe009efd5490 mi_switch() at mi_switch+0x18c/frame 0xfffffe009efd54c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009efd5500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009efd5530 sleeplk() at sleeplk+0x149/frame 0xfffffe009efd5590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe009efd5610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe009efd56b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe009efd5700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe009efd5730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe009efd57a0 vget() at vget+0x98/frame 0xfffffe009efd57e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe009efd5910 vm_fault() at vm_fault+0x75/frame 0xfffffe009efd5950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe009efd59a0 trap() at trap+0x3be/frame 0xfffffe009efd5ab0 calltrap() at calltrap+0x8/frame 0xfffffe009efd5ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2567 tid 100232 td 0xfffff8014b584560 sched_switch() at sched_switch+0x942/frame 0xfffffe009efd0490 mi_switch() at mi_switch+0x18c/frame 0xfffffe009efd04c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009efd0500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009efd0530 sleeplk() at sleeplk+0x149/frame 0xfffffe009efd0590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe009efd0610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe009efd06b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe009efd0700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe009efd0730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe009efd07a0 vget() at vget+0x98/frame 0xfffffe009efd07e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe009efd0910 vm_fault() at vm_fault+0x75/frame 0xfffffe009efd0950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe009efd09a0 trap() at trap+0x3be/frame 0xfffffe009efd0ab0 calltrap() at calltrap+0x8/frame 0xfffffe009efd0ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2566 tid 100336 td 0xfffff8001beff000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0fda490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0fda4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0fda500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0fda530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0fda590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0fda610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0fda6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0fda700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0fda730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0fda7a0 vget() at vget+0x98/frame 0xfffffe00a0fda7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0fda910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0fda950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0fda9a0 trap() at trap+0x3be/frame 0xfffffe00a0fdaab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0fdaab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2565 tid 100335 td 0xfffff8001beff560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0fd5490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0fd54c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0fd5500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0fd5530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0fd5590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0fd5610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0fd56b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0fd5700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0fd5730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0fd57a0 vget() at vget+0x98/frame 0xfffffe00a0fd57e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0fd5910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0fd5950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0fd59a0 trap() at trap+0x3be/frame 0xfffffe00a0fd5ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0fd5ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2564 tid 100194 td 0xfffff8001bdda000 sched_switch() at sched_switch+0x942/frame 0xfffffe009ef17490 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ef174c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ef17500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009ef17530 sleeplk() at sleeplk+0x149/frame 0xfffffe009ef17590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe009ef17610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe009ef176b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe009ef17700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe009ef17730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe009ef177a0 vget() at vget+0x98/frame 0xfffffe009ef177e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe009ef17910 vm_fault() at vm_fault+0x75/frame 0xfffffe009ef17950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe009ef179a0 trap() at trap+0x3be/frame 0xfffffe009ef17ab0 calltrap() at calltrap+0x8/frame 0xfffffe009ef17ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2563 tid 100329 td 0xfffff801b8110000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0fb7490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0fb74c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0fb7500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0fb7530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0fb7590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0fb7610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0fb76b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0fb7700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0fb7730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0fb77a0 vget() at vget+0x98/frame 0xfffffe00a0fb77e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0fb7910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0fb7950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0fb79a0 trap() at trap+0x3be/frame 0xfffffe00a0fb7ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0fb7ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2562 tid 100328 td 0xfffff8001bec7560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0fb2540 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0fb2570 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0fb25b0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0fb25e0 _sleep() at _sleep+0x367/frame 0xfffffe00a0fb2690 vm_wait_doms() at vm_wait_doms+0x85/frame 0xfffffe00a0fb26b0 _pmap_allocpte() at _pmap_allocpte+0x1e5/frame 0xfffffe00a0fb2720 pmap_enter() at pmap_enter+0x532/frame 0xfffffe00a0fb27e0 vm_fault_hold() at vm_fault_hold+0x1cd4/frame 0xfffffe00a0fb2910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0fb2950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0fb29a0 trap() at trap+0x3be/frame 0xfffffe00a0fb2ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0fb2ab0 --- trap 0xc, rip = 0x401d69, rsp = 0x7fffffffe560, rbp = 0x7fffffffe580 --- Tracing command swap pid 2561 tid 100327 td 0xfffff8005005b000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0fad490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0fad4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0fad500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0fad530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0fad590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0fad610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0fad6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0fad700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0fad730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0fad7a0 vget() at vget+0x98/frame 0xfffffe00a0fad7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0fad910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0fad950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0fad9a0 trap() at trap+0x3be/frame 0xfffffe00a0fadab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0fadab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2560 tid 100334 td 0xfffff8001bd63000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command swap pid 2559 tid 100333 td 0xfffff8001bd63560 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command swap pid 2558 tid 100332 td 0xfffff8001bd62000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a0fc6490 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a0fc64c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a0fc6500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a0fc6530 sleeplk() at sleeplk+0x149/frame 0xfffffe00a0fc6590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe00a0fc6610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe00a0fc66b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe00a0fc6700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe00a0fc6730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe00a0fc67a0 vget() at vget+0x98/frame 0xfffffe00a0fc67e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe00a0fc6910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a0fc6950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a0fc69a0 trap() at trap+0x3be/frame 0xfffffe00a0fc6ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a0fc6ab0 --- trap 0xc, rip = 0x800795165, rsp = 0x7fffffffe4e0, rbp = 0x7fffffffe560 --- Tracing command swap pid 2557 tid 100167 td 0xfffff800041a4000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command swap pid 2555 tid 100211 td 0xfffff8001ba5d560 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sh pid 1419 tid 100195 td 0xfffff80050abd000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sh pid 1417 tid 100206 td 0xfffff8001bd0a000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sh pid 812 tid 100197 td 0xfffff8005005d000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bash pid 805 tid 100196 td 0xfffff8001bd0a560 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command csh pid 801 tid 100180 td 0xfffff8005005e000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command su pid 800 tid 100181 td 0xfffff8005005d560 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bash pid 799 tid 100143 td 0xfffff8001bd0c560 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sshd pid 795 tid 100172 td 0xfffff8001be92000 sched_switch() at sched_switch+0x942/frame 0xfffffe009eea9490 mi_switch() at mi_switch+0x18c/frame 0xfffffe009eea94c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009eea9500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009eea9530 sleeplk() at sleeplk+0x149/frame 0xfffffe009eea9590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe009eea9610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe009eea96b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe009eea9700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe009eea9730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe009eea97a0 vget() at vget+0x98/frame 0xfffffe009eea97e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe009eea9910 vm_fault() at vm_fault+0x75/frame 0xfffffe009eea9950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe009eea99a0 trap() at trap+0x3be/frame 0xfffffe009eea9ab0 calltrap() at calltrap+0x8/frame 0xfffffe009eea9ab0 --- trap 0xc, rip = 0x800761e30, rsp = 0x7fffffffe3c8, rbp = 0x7fffffffe500 --- Tracing command dhclient pid 791 tid 100177 td 0xfffff8001bf18000 sched_switch() at sched_switch+0x942/frame 0xfffffe009eec2490 mi_switch() at mi_switch+0x18c/frame 0xfffffe009eec24c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009eec2500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009eec2530 sleeplk() at sleeplk+0x149/frame 0xfffffe009eec2590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe009eec2610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe009eec26b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe009eec2700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe009eec2730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe009eec27a0 vget() at vget+0x98/frame 0xfffffe009eec27e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe009eec2910 vm_fault() at vm_fault+0x75/frame 0xfffffe009eec2950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe009eec29a0 trap() at trap+0x3be/frame 0xfffffe009eec2ab0 calltrap() at calltrap+0x8/frame 0xfffffe009eec2ab0 --- trap 0xc, rip = 0x8004142b0, rsp = 0x7fffffffe558, rbp = 0x7fffffffe8c0 --- Tracing command login pid 746 tid 100190 td 0xfffff80050abd560 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cron pid 703 tid 100175 td 0xfffff8001badc560 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80f46396, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80af4366, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sshd pid 699 tid 100170 td 0xfffff8001bef7560 sched_switch() at sched_switch+0x942/frame 0xfffffe009ee9f5b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ee9f5e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ee9f620 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe009ee9f680 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe009ee9f6b0 _cv_wait_sig() at _cv_wait_sig+0x1fe/frame 0xfffffe009ee9f710 seltdwait() at seltdwait+0x8d/frame 0xfffffe009ee9f750 kern_select() at kern_select+0x886/frame 0xfffffe009ee9f940 sys_select() at sys_select+0x56/frame 0xfffffe009ee9f980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ee9fab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ee9fab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800874cda, rsp = 0x7fffffffe698, rbp = 0x7fffffffe6d0 --- Tracing command casperd pid 576 tid 100173 td 0xfffff8001bd0b000 sched_switch() at sched_switch+0x942/frame 0xfffffe009eeae490 mi_switch() at mi_switch+0x18c/frame 0xfffffe009eeae4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009eeae500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009eeae530 sleeplk() at sleeplk+0x149/frame 0xfffffe009eeae590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe009eeae610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe009eeae6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe009eeae700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe009eeae730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe009eeae7a0 vget() at vget+0x98/frame 0xfffffe009eeae7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe009eeae910 vm_fault() at vm_fault+0x75/frame 0xfffffe009eeae950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe009eeae9a0 trap() at trap+0x3be/frame 0xfffffe009eeaeab0 calltrap() at calltrap+0x8/frame 0xfffffe009eeaeab0 --- trap 0xc, rip = 0x80071dde0, rsp = 0x7fffffffe658, rbp = 0x7fffffffe720 --- Tracing command rpcbind pid 536 tid 100149 td 0xfffff8001bd0c000 sched_switch() at sched_switch+0x942/frame 0xfffffe009ee36490 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ee364c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ee36500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009ee36530 sleeplk() at sleeplk+0x149/frame 0xfffffe009ee36590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe009ee36610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe009ee366b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe009ee36700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe009ee36730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe009ee367a0 vget() at vget+0x98/frame 0xfffffe009ee367e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe009ee36910 vm_fault() at vm_fault+0x75/frame 0xfffffe009ee36950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe009ee369a0 trap() at trap+0x3be/frame 0xfffffe009ee36ab0 calltrap() at calltrap+0x8/frame 0xfffffe009ee36ab0 --- trap 0xc, rip = 0x8003a4780, rsp = 0x7fffffffca58, rbp = 0x7fffffffec70 --- Tracing command syslogd pid 531 tid 100153 td 0xfffff8001be93000 sched_switch() at sched_switch+0x942/frame 0xfffffe009ee4a490 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ee4a4c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ee4a500 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009ee4a530 sleeplk() at sleeplk+0x149/frame 0xfffffe009ee4a590 lockmgr_slock_hard() at lockmgr_slock_hard+0x2d4/frame 0xfffffe009ee4a610 __lockmgr_args() at __lockmgr_args+0x547/frame 0xfffffe009ee4a6b0 ffs_lock() at ffs_lock+0xaf/frame 0xfffffe009ee4a700 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe1/frame 0xfffffe009ee4a730 _vn_lock() at _vn_lock+0x66/frame 0xfffffe009ee4a7a0 vget() at vget+0x98/frame 0xfffffe009ee4a7e0 vm_fault_hold() at vm_fault_hold+0xaf8/frame 0xfffffe009ee4a910 vm_fault() at vm_fault+0x75/frame 0xfffffe009ee4a950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe009ee4a9a0 trap() at trap+0x3be/frame 0xfffffe009ee4aab0 calltrap() at calltrap+0x8/frame 0xfffffe009ee4aab0 --- trap 0xc, rip = 0x8003fc2b0, rsp = 0x7fffffffd4c8, rbp = 0x7fffffffd830 --- Tracing command vnlru pid 33 tid 100132 td 0xfffff8001ba60000 sched_switch() at sched_switch+0x942/frame 0xfffffe00005e1870 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005e18a0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005e18e0 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe00005e1920 _sleep() at _sleep+0x307/frame 0xfffffe00005e19d0 vnlru_proc() at vnlru_proc+0xa4/frame 0xfffffe00005e1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005e1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005e1ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command syncer pid 32 tid 100131 td 0xfffff8001ba60560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005dc8c0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005dc8f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005dc930 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe00005dc970 _cv_timedwait_sbt() at _cv_timedwait_sbt+0x221/frame 0xfffffe00005dc9e0 sched_sync() at sched_sync+0x609/frame 0xfffffe00005dca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005dcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005dcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100130 td 0xfffff8001ba61000 sched_switch() at sched_switch+0x942/frame 0xfffffe00005d78c0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005d78f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005d7930 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe00005d7970 _sleep() at _sleep+0x307/frame 0xfffffe00005d7a20 buf_daemon() at buf_daemon+0xe8/frame 0xfffffe00005d7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005d7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005d7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100133 td 0xfffff8001ba7c560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005e68b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005e68e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005e6920 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe00005e6960 _sleep() at _sleep+0x307/frame 0xfffffe00005e6a10 bufspace_daemon() at bufspace_daemon+0x2d6/frame 0xfffffe00005e6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005e6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005e6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100134 td 0xfffff8001ba7c000 sched_switch() at sched_switch+0x942/frame 0xfffffe00005eb8b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005eb8e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005eb920 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe00005eb960 _sleep() at _sleep+0x307/frame 0xfffffe00005eba10 bufspace_daemon() at bufspace_daemon+0x2d6/frame 0xfffffe00005eba70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005ebab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005ebab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100135 td 0xfffff8001ba7b560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005f08b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005f08e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005f0920 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe00005f0960 _sleep() at _sleep+0x307/frame 0xfffffe00005f0a10 bufspace_daemon() at bufspace_daemon+0x2d6/frame 0xfffffe00005f0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005f0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005f0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100136 td 0xfffff8001ba7b000 sched_switch() at sched_switch+0x942/frame 0xfffffe00005f58b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005f58e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005f5920 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe00005f5960 _sleep() at _sleep+0x307/frame 0xfffffe00005f5a10 bufspace_daemon() at bufspace_daemon+0x2d6/frame 0xfffffe00005f5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005f5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005f5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100137 td 0xfffff8001ba7a560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005fa8b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005fa8e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005fa920 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe00005fa960 _sleep() at _sleep+0x307/frame 0xfffffe00005faa10 bufspace_daemon() at bufspace_daemon+0x2d6/frame 0xfffffe00005faa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005faab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005faab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100138 td 0xfffff8001ba7a000 sched_switch() at sched_switch+0x942/frame 0xfffffe00005ff8b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005ff8e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005ff920 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe00005ff960 _sleep() at _sleep+0x307/frame 0xfffffe00005ffa10 bufspace_daemon() at bufspace_daemon+0x20e/frame 0xfffffe00005ffa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005ffab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005ffab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100139 td 0xfffff8001ba68560 sched_switch() at sched_switch+0x942/frame 0xfffffe009ee048b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ee048e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ee04920 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe009ee04960 _sleep() at _sleep+0x307/frame 0xfffffe009ee04a10 bufspace_daemon() at bufspace_daemon+0x2d6/frame 0xfffffe009ee04a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009ee04ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009ee04ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100151 td 0xfffff8001bdf0000 sched_switch() at sched_switch+0x942/frame 0xfffffe009ee408e0 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ee40910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ee40950 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe009ee40990 _sleep() at _sleep+0x307/frame 0xfffffe009ee40a40 softdep_flush() at softdep_flush+0x17f/frame 0xfffffe009ee40a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009ee40ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009ee40ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100198 td 0xfffff80003c62560 sched_switch() at sched_switch+0x942/frame 0xfffffe009ee598e0 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ee59910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ee59950 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe009ee59990 _sleep() at _sleep+0x307/frame 0xfffffe009ee59a40 softdep_flush() at softdep_flush+0x17f/frame 0xfffffe009ee59a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009ee59ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009ee59ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command vmdaemon pid 30 tid 100127 td 0xfffff8001b2a4560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005c88a0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005c88d0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005c8910 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005c8940 _sleep() at _sleep+0x367/frame 0xfffffe00005c89f0 vm_daemon() at vm_daemon+0x85/frame 0xfffffe00005c8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005c8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005c8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 29 tid 100126 td 0xfffff8001b380000 sched_switch() at sched_switch+0x942/frame 0xfffffe00005c37b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005c37e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005c3820 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe00005c3860 _sleep() at _sleep+0x307/frame 0xfffffe00005c3910 pause_sbt() at pause_sbt+0x144/frame 0xfffffe00005c3960 vm_pageout_worker() at vm_pageout_worker+0x242/frame 0xfffffe00005c3a50 vm_pageout() at vm_pageout+0x133/frame 0xfffffe00005c3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005c3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005c3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 29 tid 100128 td 0xfffff800041a5000 sched_switch() at sched_switch+0x942/frame 0xfffffe00005cccd0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005ccd00 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005ccd40 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005ccd70 _sleep() at _sleep+0x367/frame 0xfffffe00005cce20 vm_page_busy_sleep() at vm_page_busy_sleep+0xc5/frame 0xfffffe00005cce50 vm_page_grab_pages() at vm_page_grab_pages+0x3d4/frame 0xfffffe00005ccef0 allocbuf() at allocbuf+0x33a/frame 0xfffffe00005ccf60 getblk() at getblk+0x819/frame 0xfffffe00005ccff0 cluster_rbuild() at cluster_rbuild+0x23b/frame 0xfffffe00005cd0a0 cluster_read() at cluster_read+0x63d/frame 0xfffffe00005cd190 ffs_balloc_ufs2() at ffs_balloc_ufs2+0x1ade/frame 0xfffffe00005cd340 ffs_write() at ffs_write+0x20f/frame 0xfffffe00005cd3e0 VOP_WRITE_APV() at VOP_WRITE_APV+0x17a/frame 0xfffffe00005cd4f0 vnode_pager_generic_putpages() at vnode_pager_generic_putpages+0x581/frame 0xfffffe00005cd5e0 VOP_PUTPAGES_APV() at VOP_PUTPAGES_APV+0xfd/frame 0xfffffe00005cd610 vnode_pager_putpages() at vnode_pager_putpages+0x7d/frame 0xfffffe00005cd680 vm_pageout_flush() at vm_pageout_flush+0x144/frame 0xfffffe00005cd710 vm_pageout_cluster() at vm_pageout_cluster+0x3fa/frame 0xfffffe00005cd990 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0x797/frame 0xfffffe00005cda70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005cdab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005cdab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 29 tid 100129 td 0xfffff800041a4560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005d28e0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005d2910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005d2950 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe00005d2990 _sleep() at _sleep+0x307/frame 0xfffffe00005d2a40 uma_reclaim_worker() at uma_reclaim_worker+0xda/frame 0xfffffe00005d2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005d2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005d2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon1 pid 28 tid 100125 td 0xfffff8001b2a1560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005be8a0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005be8d0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005be910 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005be940 _sleep() at _sleep+0x367/frame 0xfffffe00005be9f0 enc_daemon() at enc_daemon+0x1ca/frame 0xfffffe00005bea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005beab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005beab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon0 pid 27 tid 100124 td 0xfffff8001b2a2000 sched_switch() at sched_switch+0x942/frame 0xfffffe00005b98a0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005b98d0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005b9910 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005b9940 _sleep() at _sleep+0x367/frame 0xfffffe00005b99f0 enc_daemon() at enc_daemon+0x1ca/frame 0xfffffe00005b9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005b9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005b9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command rand_harvestq pid 26 tid 100121 td 0xfffff8001b388560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005aa8b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005aa8e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005aa920 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe00005aa960 _sleep() at _sleep+0x307/frame 0xfffffe00005aaa10 random_kthread() at random_kthread+0x57/frame 0xfffffe00005aaa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005aaab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005aaab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sctp_iterator pid 25 tid 100117 td 0xfffff8001b38a560 sched_switch() at sched_switch+0x942/frame 0xfffffe0000596900 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000596930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000596970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005969a0 _sleep() at _sleep+0x367/frame 0xfffffe0000596a50 sctp_iterator_thread() at sctp_iterator_thread+0x59/frame 0xfffffe0000596a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000596ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000596ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod4 pid 24 tid 100116 td 0xfffff8001b2a2560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005908d0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000590900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000590940 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe0000590980 _sleep() at _sleep+0x307/frame 0xfffffe0000590a30 soaio_kproc_loop() at soaio_kproc_loop+0x196/frame 0xfffffe0000590a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000590ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000590ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod3 pid 23 tid 100115 td 0xfffff8001b2a3000 sched_switch() at sched_switch+0x942/frame 0xfffffe000058b8d0 mi_switch() at mi_switch+0x18c/frame 0xfffffe000058b900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000058b940 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe000058b980 _sleep() at _sleep+0x307/frame 0xfffffe000058ba30 soaio_kproc_loop() at soaio_kproc_loop+0x196/frame 0xfffffe000058ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe000058bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000058bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod2 pid 22 tid 100114 td 0xfffff8001b2a3560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005868d0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000586900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000586940 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe0000586980 _sleep() at _sleep+0x307/frame 0xfffffe0000586a30 soaio_kproc_loop() at soaio_kproc_loop+0x196/frame 0xfffffe0000586a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000586ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000586ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod1 pid 21 tid 100113 td 0xfffff8001b2a4000 sched_switch() at sched_switch+0x942/frame 0xfffffe00005818d0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000581900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000581940 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe0000581980 _sleep() at _sleep+0x307/frame 0xfffffe0000581a30 soaio_kproc_loop() at soaio_kproc_loop+0x196/frame 0xfffffe0000581a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000581ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000581ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100094 td 0xfffff80004021560 sched_switch() at sched_switch+0x942/frame 0xfffffe0000522930 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000522960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005229a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005229d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe0000522a30 usb_process() at usb_process+0x12b/frame 0xfffffe0000522a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000522ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000522ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100095 td 0xfffff80004021000 sched_switch() at sched_switch+0x942/frame 0xfffffe0000527930 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000527960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005279a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005279d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe0000527a30 usb_process() at usb_process+0x12b/frame 0xfffffe0000527a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000527ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000527ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100096 td 0xfffff80004020560 sched_switch() at sched_switch+0x942/frame 0xfffffe000052c930 mi_switch() at mi_switch+0x18c/frame 0xfffffe000052c960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000052c9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000052c9d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe000052ca30 usb_process() at usb_process+0x12b/frame 0xfffffe000052ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe000052cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000052cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100097 td 0xfffff80004020000 sched_switch() at sched_switch+0x942/frame 0xfffffe0000531930 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000531960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005319a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005319d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe0000531a30 usb_process() at usb_process+0x12b/frame 0xfffffe0000531a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000531ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000531ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100098 td 0xfffff80004018560 sched_switch() at sched_switch+0x942/frame 0xfffffe0000536930 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000536960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005369a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005369d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe0000536a30 usb_process() at usb_process+0x12b/frame 0xfffffe0000536a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000536ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000536ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100100 td 0xfffff80003c64560 sched_switch() at sched_switch+0x942/frame 0xfffffe0000540930 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000540960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005409a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005409d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe0000540a30 usb_process() at usb_process+0x12b/frame 0xfffffe0000540a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000540ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000540ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100101 td 0xfffff800041a7000 sched_switch() at sched_switch+0x942/frame 0xfffffe0000545930 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000545960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005459a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005459d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe0000545a30 usb_process() at usb_process+0x12b/frame 0xfffffe0000545a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000545ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000545ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100102 td 0xfffff800041a6560 sched_switch() at sched_switch+0x942/frame 0xfffffe000054a930 mi_switch() at mi_switch+0x18c/frame 0xfffffe000054a960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000054a9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000054a9d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe000054aa30 usb_process() at usb_process+0x12b/frame 0xfffffe000054aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe000054aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000054aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100103 td 0xfffff800041a6000 sched_switch() at sched_switch+0x942/frame 0xfffffe000054f930 mi_switch() at mi_switch+0x18c/frame 0xfffffe000054f960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000054f9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000054f9d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe000054fa30 usb_process() at usb_process+0x12b/frame 0xfffffe000054fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe000054fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000054fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100104 td 0xfffff800041a5560 sched_switch() at sched_switch+0x942/frame 0xfffffe0000554930 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000554960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005549a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005549d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe0000554a30 usb_process() at usb_process+0x12b/frame 0xfffffe0000554a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000554ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000554ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100105 td 0xfffff8000c095560 sched_switch() at sched_switch+0x942/frame 0xfffffe0000559930 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000559960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005599a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005599d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe0000559a30 usb_process() at usb_process+0x12b/frame 0xfffffe0000559a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000559ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000559ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100106 td 0xfffff8000c095000 sched_switch() at sched_switch+0x942/frame 0xfffffe000055e930 mi_switch() at mi_switch+0x18c/frame 0xfffffe000055e960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000055e9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000055e9d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe000055ea30 usb_process() at usb_process+0x12b/frame 0xfffffe000055ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe000055eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000055eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100107 td 0xfffff8000c094560 sched_switch() at sched_switch+0x942/frame 0xfffffe0000563930 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000563960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005639a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005639d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe0000563a30 usb_process() at usb_process+0x12b/frame 0xfffffe0000563a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000563ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000563ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100108 td 0xfffff8000c094000 sched_switch() at sched_switch+0x942/frame 0xfffffe0000568930 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000568960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005689a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005689d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe0000568a30 usb_process() at usb_process+0x12b/frame 0xfffffe0000568a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000568ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000568ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100109 td 0xfffff8000c093560 sched_switch() at sched_switch+0x942/frame 0xfffffe000056d930 mi_switch() at mi_switch+0x18c/frame 0xfffffe000056d960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000056d9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000056d9d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe000056da30 usb_process() at usb_process+0x12b/frame 0xfffffe000056da70 fork_exit() at fork_exit+0x84/frame 0xfffffe000056dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000056dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 19 tid 100089 td 0xfffff80003ad8000 sched_switch() at sched_switch+0x942/frame 0xfffffe00005098d0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000509900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000509940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0000509970 _sleep() at _sleep+0x367/frame 0xfffffe0000509a20 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe0000509a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000509ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000509ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 19 tid 100090 td 0xfffff80003ace560 sched_switch() at sched_switch+0x942/frame 0xfffffe000050e8d0 mi_switch() at mi_switch+0x18c/frame 0xfffffe000050e900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000050e940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000050e970 _sleep() at _sleep+0x367/frame 0xfffffe000050ea20 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe000050ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe000050eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000050eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 19 tid 100091 td 0xfffff80003c64000 sched_switch() at sched_switch+0x942/frame 0xfffffe00005138d0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000513900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000513940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0000513970 _sleep() at _sleep+0x367/frame 0xfffffe0000513a20 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe0000513a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000513ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000513ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 19 tid 100123 td 0xfffff8001b380560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005b48c0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005b48f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005b4930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005b4960 _sleep() at _sleep+0x367/frame 0xfffffe00005b4a10 xpt_scanner_thread() at xpt_scanner_thread+0x93/frame 0xfffffe00005b4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005b4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005b4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 11 pid 18 tid 100088 td 0xfffff80003ad8560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005048b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005048e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000504920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0000504950 _sleep() at _sleep+0x367/frame 0xfffffe0000504a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0000504a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000504ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000504ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 10 pid 17 tid 100087 td 0xfffff80003ad9000 sched_switch() at sched_switch+0x942/frame 0xfffffe00004ff8b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004ff8e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004ff920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004ff950 _sleep() at _sleep+0x367/frame 0xfffffe00004ffa00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00004ffa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004ffab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004ffab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 9 pid 16 tid 100086 td 0xfffff80003ad9560 sched_switch() at sched_switch+0x942/frame 0xfffffe00004fa8b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004fa8e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004fa920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004fa950 _sleep() at _sleep+0x367/frame 0xfffffe00004faa00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00004faa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004faab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004faab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 8 pid 15 tid 100085 td 0xfffff80003ada000 sched_switch() at sched_switch+0x942/frame 0xfffffe00004f58b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004f58e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004f5920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004f5950 _sleep() at _sleep+0x367/frame 0xfffffe00004f5a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00004f5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004f5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004f5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 7 pid 14 tid 100084 td 0xfffff80003ada560 sched_switch() at sched_switch+0x942/frame 0xfffffe00004f08b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004f08e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004f0920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004f0950 _sleep() at _sleep+0x367/frame 0xfffffe00004f0a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00004f0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004f0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004f0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 6 pid 9 tid 100083 td 0xfffff80003adb000 sched_switch() at sched_switch+0x942/frame 0xfffffe00004eb8b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004eb8e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004eb920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004eb950 _sleep() at _sleep+0x367/frame 0xfffffe00004eba00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00004eba70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004ebab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004ebab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 5 pid 8 tid 100082 td 0xfffff80003adb560 sched_switch() at sched_switch+0x942/frame 0xfffffe00004e68b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004e68e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004e6920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004e6950 _sleep() at _sleep+0x367/frame 0xfffffe00004e6a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00004e6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004e6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004e6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 4 pid 7 tid 100081 td 0xfffff80003aca000 sched_switch() at sched_switch+0x942/frame 0xfffffe00004e18b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004e18e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004e1920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004e1950 _sleep() at _sleep+0x367/frame 0xfffffe00004e1a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00004e1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004e1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004e1ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 3 pid 6 tid 100080 td 0xfffff80003aca560 sched_switch() at sched_switch+0x942/frame 0xfffffe00004dc8b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004dc8e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004dc920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004dc950 _sleep() at _sleep+0x367/frame 0xfffffe00004dca00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00004dca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004dcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004dcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 2 pid 5 tid 100079 td 0xfffff80003acb000 sched_switch() at sched_switch+0x942/frame 0xfffffe00004d78b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004d78e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004d7920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004d7950 _sleep() at _sleep+0x367/frame 0xfffffe00004d7a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00004d7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004d7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004d7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 1 pid 4 tid 100078 td 0xfffff80003acb560 sched_switch() at sched_switch+0x942/frame 0xfffffe00004d28b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004d28e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004d2920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004d2950 _sleep() at _sleep+0x367/frame 0xfffffe00004d2a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00004d2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004d2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004d2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 0 pid 3 tid 100077 td 0xfffff80003acc000 sched_switch() at sched_switch+0x942/frame 0xfffffe00004cd8b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004cd8e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004cd920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004cd950 _sleep() at _sleep+0x367/frame 0xfffffe00004cda00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00004cda70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004cdab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004cdab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 2 tid 100076 td 0xfffff80003acc560 sched_switch() at sched_switch+0x942/frame 0xfffffe00004c88e0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004c8910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004c8950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004c8980 _sleep() at _sleep+0x367/frame 0xfffffe00004c8a30 crypto_proc() at crypto_proc+0x2ba/frame 0xfffffe00004c8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004c8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004c8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100060 td 0xfffff80003a2f560 sched_switch() at sched_switch+0x942/frame 0xfffffe00004788e0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000478910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000478950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0000478980 _sleep() at _sleep+0x367/frame 0xfffffe0000478a30 g_run_events() at g_run_events+0x4f/frame 0xfffffe0000478a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000478ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000478ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100061 td 0xfffff80003a2f000 sched_switch() at sched_switch+0x942/frame 0xfffffe000047d8e0 mi_switch() at mi_switch+0x18c/frame 0xfffffe000047d910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000047d950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000047d980 _sleep() at _sleep+0x367/frame 0xfffffe000047da30 g_io_schedule_up() at g_io_schedule_up+0xa5/frame 0xfffffe000047da60 g_up_procbody() at g_up_procbody+0x6d/frame 0xfffffe000047da70 fork_exit() at fork_exit+0x84/frame 0xfffffe000047dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000047dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100062 td 0xfffff80003a2e560 sched_switch() at sched_switch+0x942/frame 0xfffffe00004828d0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000482900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000482940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0000482970 _sleep() at _sleep+0x367/frame 0xfffffe0000482a20 g_io_schedule_down() at g_io_schedule_down+0x14c/frame 0xfffffe0000482a60 g_down_procbody() at g_down_procbody+0x6d/frame 0xfffffe0000482a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000482ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000482ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100015 td 0xfffff80003947000 sched_switch() at sched_switch+0x942/frame 0xfffffe00789969f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0078996a20 ithread_loop() at ithread_loop+0x1f1/frame 0xfffffe0078996a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078996ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078996ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100016 td 0xfffff80003946560 sched_switch() at sched_switch+0x942/frame 0xfffffe007899b9f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe007899ba20 ithread_loop() at ithread_loop+0x1f1/frame 0xfffffe007899ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe007899bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007899bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100018 td 0xfffff80003939560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100046 td 0xfffff80003a26560 sched_switch() at sched_switch+0x942/frame 0xfffffe00004329f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000432a20 ithread_loop() at ithread_loop+0x1f1/frame 0xfffffe0000432a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000432ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000432ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100047 td 0xfffff80003a26000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100048 td 0xfffff80003a25560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100049 td 0xfffff80003a25000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100050 td 0xfffff80003a24560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100051 td 0xfffff80003a24000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100052 td 0xfffff80003a18560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100053 td 0xfffff80003a18000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100054 td 0xfffff80003a02560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100055 td 0xfffff80003a32000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100056 td 0xfffff80003a31560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100057 td 0xfffff80003a31000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100058 td 0xfffff80003a30560 sched_switch() at sched_switch+0x942/frame 0xfffffe000046e9f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe000046ea20 ithread_loop() at ithread_loop+0x1f1/frame 0xfffffe000046ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe000046eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000046eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100059 td 0xfffff80003a30000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100092 td 0xfffff80004022560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005189f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000518a20 ithread_loop() at ithread_loop+0x1f1/frame 0xfffffe0000518a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000518ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000518ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100093 td 0xfffff80004022000 sched_switch() at sched_switch+0x942/frame 0xfffffe000051d9f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe000051da20 ithread_loop() at ithread_loop+0x1f1/frame 0xfffffe000051da70 fork_exit() at fork_exit+0x84/frame 0xfffffe000051dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000051dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100099 td 0xfffff80004018000 sched_switch() at sched_switch+0x942/frame 0xfffffe000053b9f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe000053ba20 ithread_loop() at ithread_loop+0x1f1/frame 0xfffffe000053ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe000053bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000053bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100110 td 0xfffff8000c093000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100111 td 0xfffff8000c08b560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005779f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000577a20 ithread_loop() at ithread_loop+0x1f1/frame 0xfffffe0000577a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000577ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000577ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100003 td 0xfffff80003938000 (CPU 0) kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xfffff80003937560 (CPU 1) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00007ffdf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00007ffe10 trap() at trap+0x49/frame 0xfffffe00007fff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00007fff20 --- trap 0x13, rip = 0xffffffff810c7d76, rsp = 0xfffffe007895f910, rbp = 0xfffffe007895f910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007895f910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895f960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895f980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895f9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffff80003937000 (CPU 2) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00071b2df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00071b2e10 trap() at trap+0x49/frame 0xfffffe00071b2f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00071b2f20 --- trap 0x13, rip = 0xffffffff810c7d76, rsp = 0xfffffe0078964910, rbp = 0xfffffe0078964910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0078964910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe0078964960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078964980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe00789649a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078964a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078964ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078964ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xfffff80003936560 (CPU 3) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00071bbdf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00071bbe10 trap() at trap+0x49/frame 0xfffffe00071bbf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00071bbf20 --- trap 0x13, rip = 0xffffffff810c7d76, rsp = 0xfffffe0078969910, rbp = 0xfffffe0078969910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0078969910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe0078969960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078969980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe00789699a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078969a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078969ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078969ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100007 td 0xfffff80003936000 (CPU 4) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00071c4df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00071c4e10 trap() at trap+0x49/frame 0xfffffe00071c4f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00071c4f20 --- trap 0x13, rip = 0xffffffff810c7d76, rsp = 0xfffffe007896e910, rbp = 0xfffffe007896e910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007896e910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007896e960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007896e980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007896e9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007896ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe007896eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007896eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xfffff80003935560 (CPU 5) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00071cddf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00071cde10 trap() at trap+0x49/frame 0xfffffe00071cdf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00071cdf20 --- trap 0x13, rip = 0xffffffff810c7d76, rsp = 0xfffffe0078973910, rbp = 0xfffffe0078973910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0078973910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe0078973960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078973980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe00789739a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078973a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078973ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078973ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xfffff80003935000 (CPU 6) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00071d6df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00071d6e10 trap() at trap+0x49/frame 0xfffffe00071d6f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00071d6f20 --- trap 0x13, rip = 0xffffffff810c7d76, rsp = 0xfffffe0078978910, rbp = 0xfffffe0078978910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0078978910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe0078978960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078978980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe00789789a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078978a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078978ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078978ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xfffff80003949560 (CPU 7) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00071dfdf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00071dfe10 trap() at trap+0x49/frame 0xfffffe00071dff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00071dff20 --- trap 0x13, rip = 0xffffffff810c7d76, rsp = 0xfffffe007897d910, rbp = 0xfffffe007897d910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007897d910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007897d960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007897d980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007897d9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007897da70 fork_exit() at fork_exit+0x84/frame 0xfffffe007897dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007897dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffff80003949000 (CPU 8) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00071e8df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00071e8e10 trap() at trap+0x49/frame 0xfffffe00071e8f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00071e8f20 --- trap 0x13, rip = 0xffffffff810c7d76, rsp = 0xfffffe0078982910, rbp = 0xfffffe0078982910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0078982910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe0078982960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078982980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe00789829a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078982a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078982ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078982ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100012 td 0xfffff80003948560 (CPU 9) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00071f1df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00071f1e10 trap() at trap+0x49/frame 0xfffffe00071f1f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00071f1f20 --- trap 0x13, rip = 0xffffffff810c7d76, rsp = 0xfffffe0078987910, rbp = 0xfffffe0078987910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0078987910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe0078987960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078987980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe00789879a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078987a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078987ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078987ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xfffff80003948000 (CPU 10) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00071fadf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00071fae10 trap() at trap+0x49/frame 0xfffffe00071faf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00071faf20 --- trap 0x13, rip = 0xffffffff810c7d76, rsp = 0xfffffe007898c910, rbp = 0xfffffe007898c910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007898c910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007898c960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007898c980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007898c9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007898ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe007898cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007898cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100014 td 0xfffff80003947560 (CPU 11) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00071ffdf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00071ffe10 trap() at trap+0x49/frame 0xfffffe00071fff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00071fff20 --- trap 0x13, rip = 0xffffffff810c7d76, rsp = 0xfffffe0078991910, rbp = 0xfffffe0078991910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0078991910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe0078991960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078991980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe00789919a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078991a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078991ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078991ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command init pid 1 tid 100002 td 0xfffff80003938560 sched_switch() at sched_switch+0x942/frame 0xfffffe0078955550 mi_switch() at mi_switch+0x18c/frame 0xfffffe0078955580 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789555c0 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe0078955620 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0078955650 _sleep() at _sleep+0x359/frame 0xfffffe0078955700 kern_wait6() at kern_wait6+0x4c4/frame 0xfffffe0078955790 sys_wait4() at sys_wait4+0x78/frame 0xfffffe0078955980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe0078955ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0078955ab0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x2a71ca, rsp = 0x7fffffffe788, rbp = 0x7fffffffe7f0 --- Tracing command audit pid 10 tid 100001 td 0xfffff80003939000 sched_switch() at sched_switch+0x942/frame 0xfffffe0078950910 mi_switch() at mi_switch+0x18c/frame 0xfffffe0078950940 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0078950980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789509b0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe0078950a10 audit_worker() at audit_worker+0x93/frame 0xfffffe0078950a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078950ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078950ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xffffffff81f400a0 sched_switch() at sched_switch+0x942/frame 0xffffffff824f49b0 mi_switch() at mi_switch+0x18c/frame 0xffffffff824f49e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xffffffff824f4a20 sleepq_wait() at sleepq_wait+0x43/frame 0xffffffff824f4a50 _sleep() at _sleep+0x367/frame 0xffffffff824f4b00 vm_wait_min() at vm_wait_min+0x64/frame 0xffffffff824f4b20 swapper() at swapper+0x25/frame 0xffffffff824f4b70 btext() at btext+0x2c Tracing command kernel pid 0 tid 100017 td 0xfffff80003946000 sched_switch() at sched_switch+0x942/frame 0xfffffe00789a08f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789a0920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789a0960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789a0990 _sleep() at _sleep+0x367/frame 0xfffffe00789a0a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00789a0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789a0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789a0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xfffff80003976000 sched_switch() at sched_switch+0x942/frame 0xfffffe00789aa8f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789aa920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789aa960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789aa990 _sleep() at _sleep+0x367/frame 0xfffffe00789aaa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00789aaa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789aaab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789aaab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xfffff80003975560 sched_switch() at sched_switch+0x942/frame 0xfffffe00789af8f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789af920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789af960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789af990 _sleep() at _sleep+0x367/frame 0xfffffe00789afa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00789afa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789afab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789afab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xfffff80003975000 sched_switch() at sched_switch+0x942/frame 0xfffffe00789b4920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789b4950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789b4990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789b49c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789b4a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789b4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789b4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789b4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xfffff80003974560 sched_switch() at sched_switch+0x942/frame 0xfffffe00789b9920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789b9950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789b9990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789b99c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789b9a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789b9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789b9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789b9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xfffff80003974000 sched_switch() at sched_switch+0x942/frame 0xfffffe00789be920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789be950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789be990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789be9c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789bea40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789bea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789beab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789beab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xfffff80003973560 sched_switch() at sched_switch+0x942/frame 0xfffffe00789c3920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789c3950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789c3990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789c39c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789c3a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789c3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789c3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789c3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xfffff80003973000 sched_switch() at sched_switch+0x942/frame 0xfffffe00789c8920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789c8950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789c8990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789c89c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789c8a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789c8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789c8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789c8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100026 td 0xfffff80003972560 sched_switch() at sched_switch+0x942/frame 0xfffffe00789cd920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789cd950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789cd990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789cd9c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789cda40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789cda70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789cdab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789cdab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100027 td 0xfffff80003972000 sched_switch() at sched_switch+0x942/frame 0xfffffe00789d2920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789d2950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789d2990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789d29c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789d2a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789d2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789d2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789d2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100028 td 0xfffff8000398a560 sched_switch() at sched_switch+0x942/frame 0xfffffe00789d7920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789d7950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789d7990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789d79c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789d7a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789d7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789d7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789d7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100029 td 0xfffff8000398a000 sched_switch() at sched_switch+0x942/frame 0xfffffe00789dc920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789dc950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789dc990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789dc9c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789dca40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789dca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789dcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789dcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100030 td 0xfffff80003985560 sched_switch() at sched_switch+0x942/frame 0xfffffe00789e1920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789e1950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789e1990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789e19c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789e1a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789e1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789e1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789e1ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100031 td 0xfffff80003985000 sched_switch() at sched_switch+0x942/frame 0xfffffe00789e6920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789e6950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789e6990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789e69c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789e6a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789e6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789e6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789e6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100032 td 0xfffff80003984560 sched_switch() at sched_switch+0x942/frame 0xfffffe00789eb920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789eb950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789eb990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789eb9c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789eba40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789eba70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789ebab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789ebab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100033 td 0xfffff80003984000 sched_switch() at sched_switch+0x942/frame 0xfffffe00789f0920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789f0950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789f0990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789f09c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789f0a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789f0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789f0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789f0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100034 td 0xfffff80003983560 sched_switch() at sched_switch+0x942/frame 0xfffffe00789f5920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789f5950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789f5990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789f59c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789f5a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789f5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789f5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789f5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100035 td 0xfffff80003983000 sched_switch() at sched_switch+0x942/frame 0xfffffe00789fa920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789fa950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789fa990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789fa9c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789faa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789faa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789faab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789faab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100036 td 0xfffff80003976560 sched_switch() at sched_switch+0x942/frame 0xfffffe00789ff920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789ff950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789ff990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789ff9c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789ffa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789ffa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789ffab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789ffab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100037 td 0xfffff80003a02000 sched_switch() at sched_switch+0x942/frame 0xfffffe0000405920 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000405950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000405990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004059c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe0000405a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000405a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000405ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000405ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100038 td 0xfffff80003a01560 sched_switch() at sched_switch+0x942/frame 0xfffffe000040a920 mi_switch() at mi_switch+0x18c/frame 0xfffffe000040a950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000040a990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000040a9c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe000040aa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000040aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe000040aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000040aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100039 td 0xfffff80003a01000 sched_switch() at sched_switch+0x942/frame 0xfffffe000040f920 mi_switch() at mi_switch+0x18c/frame 0xfffffe000040f950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000040f990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000040f9c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe000040fa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000040fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe000040fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000040fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100040 td 0xfffff80003a00560 sched_switch() at sched_switch+0x942/frame 0xfffffe0000414920 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000414950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000414990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004149c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe0000414a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000414a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000414ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000414ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100041 td 0xfffff80003a00000 sched_switch() at sched_switch+0x942/frame 0xfffffe0000419920 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000419950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000419990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004199c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe0000419a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000419a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000419ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000419ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100042 td 0xfffff800039ff560 sched_switch() at sched_switch+0x942/frame 0xfffffe000041e920 mi_switch() at mi_switch+0x18c/frame 0xfffffe000041e950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000041e990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000041e9c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe000041ea40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000041ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe000041eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000041eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100043 td 0xfffff800039ff000 sched_switch() at sched_switch+0x942/frame 0xfffffe0000423920 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000423950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000423990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004239c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe0000423a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000423a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000423ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000423ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100044 td 0xfffff800039fe560 sched_switch() at sched_switch+0x942/frame 0xfffffe0000428920 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000428950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000428990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004289c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe0000428a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000428a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000428ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000428ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100045 td 0xfffff800039fe000 sched_switch() at sched_switch+0x942/frame 0xfffffe000042d920 mi_switch() at mi_switch+0x18c/frame 0xfffffe000042d950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000042d990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000042d9c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe000042da40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000042da70 fork_exit() at fork_exit+0x84/frame 0xfffffe000042dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000042dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100063 td 0xfffff80003a2e000 sched_switch() at sched_switch+0x942/frame 0xfffffe00004878f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000487920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000487960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0000487990 _sleep() at _sleep+0x367/frame 0xfffffe0000487a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0000487a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000487ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000487ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100064 td 0xfffff80003ac9560 sched_switch() at sched_switch+0x942/frame 0xfffffe000048c8f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe000048c920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000048c960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000048c990 _sleep() at _sleep+0x367/frame 0xfffffe000048ca40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe000048ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe000048cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000048cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100065 td 0xfffff80003ac9000 sched_switch() at sched_switch+0x942/frame 0xfffffe00004918f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000491920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000491960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0000491990 _sleep() at _sleep+0x367/frame 0xfffffe0000491a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0000491a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000491ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000491ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100066 td 0xfffff80003ac8560 sched_switch() at sched_switch+0x942/frame 0xfffffe00004968f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000496920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000496960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0000496990 _sleep() at _sleep+0x367/frame 0xfffffe0000496a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0000496a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000496ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000496ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100067 td 0xfffff80003ac8000 sched_switch() at sched_switch+0x942/frame 0xfffffe000049b8f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe000049b920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000049b960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000049b990 _sleep() at _sleep+0x367/frame 0xfffffe000049ba40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe000049ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe000049bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000049bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100068 td 0xfffff80003ac7560 sched_switch() at sched_switch+0x942/frame 0xfffffe00004a08f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004a0920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004a0960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004a0990 _sleep() at _sleep+0x367/frame 0xfffffe00004a0a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00004a0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004a0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004a0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100069 td 0xfffff80003ac7000 sched_switch() at sched_switch+0x942/frame 0xfffffe00004a58f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004a5920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004a5960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004a5990 _sleep() at _sleep+0x367/frame 0xfffffe00004a5a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00004a5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004a5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004a5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100070 td 0xfffff80003ac6560 sched_switch() at sched_switch+0x942/frame 0xfffffe00004aa8f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004aa920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004aa960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004aa990 _sleep() at _sleep+0x367/frame 0xfffffe00004aaa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00004aaa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004aaab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004aaab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100071 td 0xfffff80003ac6000 sched_switch() at sched_switch+0x942/frame 0xfffffe00004af8f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004af920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004af960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004af990 _sleep() at _sleep+0x367/frame 0xfffffe00004afa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00004afa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004afab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004afab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100072 td 0xfffff80003a32560 sched_switch() at sched_switch+0x942/frame 0xfffffe00004b48f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004b4920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004b4960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004b4990 _sleep() at _sleep+0x367/frame 0xfffffe00004b4a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00004b4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004b4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004b4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100073 td 0xfffff80003ace000 sched_switch() at sched_switch+0x942/frame 0xfffffe00004b98f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004b9920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004b9960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004b9990 _sleep() at _sleep+0x367/frame 0xfffffe00004b9a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00004b9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004b9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004b9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100074 td 0xfffff80003acd560 sched_switch() at sched_switch+0x942/frame 0xfffffe00004be8f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004be920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004be960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004be990 _sleep() at _sleep+0x367/frame 0xfffffe00004bea40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00004bea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004beab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004beab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100075 td 0xfffff80003acd000 sched_switch() at sched_switch+0x942/frame 0xfffffe00004c38f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004c3920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004c3960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004c3990 _sleep() at _sleep+0x367/frame 0xfffffe00004c3a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00004c3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004c3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004c3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100112 td 0xfffff8000c08b000 sched_switch() at sched_switch+0x942/frame 0xfffffe000057c920 mi_switch() at mi_switch+0x18c/frame 0xfffffe000057c950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000057c990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000057c9c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe000057ca40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe000057ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe000057cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000057cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100118 td 0xfffff8001b38a000 sched_switch() at sched_switch+0x942/frame 0xfffffe000059b920 mi_switch() at mi_switch+0x18c/frame 0xfffffe000059b950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000059b990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000059b9c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe000059ba40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe000059ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe000059bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000059bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100119 td 0xfffff8001b389560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005a0920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005a0950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005a0990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005a09c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00005a0a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00005a0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005a0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005a0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100120 td 0xfffff8001b389000 sched_switch() at sched_switch+0x942/frame 0xfffffe00005a5920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005a5950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005a5990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005a59c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00005a5a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00005a5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005a5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005a5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100122 td 0xfffff8001b388000 sched_switch() at sched_switch+0x942/frame 0xfffffe00005af8f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005af920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005af960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005af990 _sleep() at _sleep+0x367/frame 0xfffffe00005afa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005afa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005afab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005afab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db:0:pho> dump Dumping 1296 out of 32625 MB:..2%..12%..21%..31%..41%..51%..61%..71%..81%..91% Dump complete db> reset Index: /usr/src/sys/kern/vfs_bio.c =================================================================== --- /usr/src/sys/kern/vfs_bio.c (revision 330873) +++ /usr/src/sys/kern/vfs_bio.c (working copy) @@ -50,6 +50,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -100,6 +101,52 @@ struct buf_ops buf_ops_bio = { .bop_bdflush = bufbdflush, }; +struct bufqueue { + struct mtx_padalign bq_lock; + TAILQ_HEAD(, buf) bq_queue; + uint8_t bq_index; + uint16_t bq_subqueue; + int bq_len; +} __aligned(CACHE_LINE_SIZE); + +#define BQ_LOCKPTR(bq) (&(bq)->bq_lock) +#define BQ_LOCK(bq) mtx_lock(BQ_LOCKPTR((bq))) +#define BQ_UNLOCK(bq) mtx_unlock(BQ_LOCKPTR((bq))) +#define BQ_ASSERT_LOCKED(bq) mtx_assert(BQ_LOCKPTR((bq)), MA_OWNED) + +struct bufdomain { + struct bufqueue bd_subq[MAXCPU + 1]; /* Per-cpu sub queues + global */ + struct bufqueue bd_dirtyq; + struct bufqueue *bd_cleanq; + struct mtx_padalign bd_run_lock; + /* Constants */ + long bd_maxbufspace; + long bd_hibufspace; + long bd_lobufspace; + long bd_bufspacethresh; + int bd_hifreebuffers; + int bd_lofreebuffers; + int bd_hidirtybuffers; + int bd_lodirtybuffers; + int bd_dirtybufthresh; + int bd_lim; + /* atomics */ + int bd_wanted; + int __aligned(CACHE_LINE_SIZE) bd_numdirtybuffers; + int __aligned(CACHE_LINE_SIZE) bd_running; + long __aligned(CACHE_LINE_SIZE) bd_bufspace; + int __aligned(CACHE_LINE_SIZE) bd_freebuffers; +} __aligned(CACHE_LINE_SIZE); + +#define BD_LOCKPTR(bd) (&(bd)->bd_cleanq->bq_lock) +#define BD_LOCK(bd) mtx_lock(BD_LOCKPTR((bd))) +#define BD_UNLOCK(bd) mtx_unlock(BD_LOCKPTR((bd))) +#define BD_ASSERT_LOCKED(bd) mtx_assert(BD_LOCKPTR((bd)), MA_OWNED) +#define BD_RUN_LOCKPTR(bd) (&(bd)->bd_run_lock) +#define BD_RUN_LOCK(bd) mtx_lock(BD_RUN_LOCKPTR((bd))) +#define BD_RUN_UNLOCK(bd) mtx_unlock(BD_RUN_LOCKPTR((bd))) +#define BD_DOMAIN(bd) (bd - bdomain) + static struct buf *buf; /* buffer header pool */ extern struct buf *swbuf; /* Swap buffer header pool. */ caddr_t unmapped_buf; @@ -123,8 +170,8 @@ static int vfs_bio_clcheck(struct vnode *vp, int s daddr_t lblkno, daddr_t blkno); static void breada(struct vnode *, daddr_t *, int *, int, struct ucred *, int, void (*)(struct buf *)); -static int buf_flush(struct vnode *vp, int); -static int flushbufqueues(struct vnode *, int, int); +static int buf_flush(struct vnode *vp, struct bufdomain *, int); +static int flushbufqueues(struct vnode *, struct bufdomain *, int, int); static void buf_daemon(void); static __inline void bd_wakeup(void); static int sysctl_runningspace(SYSCTL_HANDLER_ARGS); @@ -133,6 +180,16 @@ static void bufkva_free(struct buf *); static int buf_import(void *, void **, int, int, int); static void buf_release(void *, void **, int); static void maxbcachebuf_adjust(void); +static inline struct bufdomain *bufdomain(struct buf *); +static void bq_remove(struct bufqueue *bq, struct buf *bp); +static void bq_insert(struct bufqueue *bq, struct buf *bp, bool unlock); +static int buf_recycle(struct bufdomain *, bool kva); +static void bq_init(struct bufqueue *bq, int qindex, int cpu, + const char *lockname); +static void bd_init(struct bufdomain *bd); +static int bd_flushall(struct bufdomain *bd); +static int sysctl_bufdomain_long(SYSCTL_HANDLER_ARGS); +static int sysctl_bufdomain_int(SYSCTL_HANDLER_ARGS); static int sysctl_bufspace(SYSCTL_HANDLER_ARGS); int vmiodirenable = TRUE; @@ -147,7 +204,9 @@ static counter_u64_t bufkvaspace; SYSCTL_COUNTER_U64(_vfs, OID_AUTO, bufkvaspace, CTLFLAG_RD, &bufkvaspace, "Kernel virtual memory used for buffers"); static long maxbufspace; -SYSCTL_LONG(_vfs, OID_AUTO, maxbufspace, CTLFLAG_RW, &maxbufspace, 0, +SYSCTL_PROC(_vfs, OID_AUTO, maxbufspace, + CTLTYPE_LONG|CTLFLAG_MPSAFE|CTLFLAG_RW, &maxbufspace, + __offsetof(struct bufdomain, bd_maxbufspace), sysctl_bufdomain_long, "L", "Maximum allowed value of bufspace (including metadata)"); static long bufmallocspace; SYSCTL_LONG(_vfs, OID_AUTO, bufmallocspace, CTLFLAG_RD, &bufmallocspace, 0, @@ -156,14 +215,20 @@ static long maxbufmallocspace; SYSCTL_LONG(_vfs, OID_AUTO, maxmallocbufspace, CTLFLAG_RW, &maxbufmallocspace, 0, "Maximum amount of malloced memory for buffers"); static long lobufspace; -SYSCTL_LONG(_vfs, OID_AUTO, lobufspace, CTLFLAG_RW, &lobufspace, 0, +SYSCTL_PROC(_vfs, OID_AUTO, lobufspace, + CTLTYPE_LONG|CTLFLAG_MPSAFE|CTLFLAG_RW, &lobufspace, + __offsetof(struct bufdomain, bd_lobufspace), sysctl_bufdomain_long, "L", "Minimum amount of buffers we want to have"); long hibufspace; -SYSCTL_LONG(_vfs, OID_AUTO, hibufspace, CTLFLAG_RW, &hibufspace, 0, +SYSCTL_PROC(_vfs, OID_AUTO, hibufspace, + CTLTYPE_LONG|CTLFLAG_MPSAFE|CTLFLAG_RW, &hibufspace, + __offsetof(struct bufdomain, bd_hibufspace), sysctl_bufdomain_long, "L", "Maximum allowed value of bufspace (excluding metadata)"); long bufspacethresh; -SYSCTL_LONG(_vfs, OID_AUTO, bufspacethresh, CTLFLAG_RW, &bufspacethresh, - 0, "Bufspace consumed before waking the daemon to free some"); +SYSCTL_PROC(_vfs, OID_AUTO, bufspacethresh, + CTLTYPE_LONG|CTLFLAG_MPSAFE|CTLFLAG_RW, &bufspacethresh, + __offsetof(struct bufdomain, bd_bufspacethresh), sysctl_bufdomain_long, "L", + "Bufspace consumed before waking the daemon to free some"); static counter_u64_t buffreekvacnt; SYSCTL_COUNTER_U64(_vfs, OID_AUTO, buffreekvacnt, CTLFLAG_RW, &buffreekvacnt, "Number of times we have freed the KVA space from some buffer"); @@ -190,26 +255,37 @@ SYSCTL_INT(_vfs, OID_AUTO, altbufferflushes, CTLFL static int recursiveflushes; SYSCTL_INT(_vfs, OID_AUTO, recursiveflushes, CTLFLAG_RW, &recursiveflushes, 0, "Number of flushes skipped due to being recursive"); -static int numdirtybuffers; -SYSCTL_INT(_vfs, OID_AUTO, numdirtybuffers, CTLFLAG_RD, &numdirtybuffers, 0, +static int sysctl_numdirtybuffers(SYSCTL_HANDLER_ARGS); +SYSCTL_PROC(_vfs, OID_AUTO, numdirtybuffers, + CTLTYPE_INT|CTLFLAG_MPSAFE|CTLFLAG_RD, NULL, 0, sysctl_numdirtybuffers, "I", "Number of buffers that are dirty (has unwritten changes) at the moment"); static int lodirtybuffers; -SYSCTL_INT(_vfs, OID_AUTO, lodirtybuffers, CTLFLAG_RW, &lodirtybuffers, 0, +SYSCTL_PROC(_vfs, OID_AUTO, lodirtybuffers, + CTLTYPE_LONG|CTLFLAG_MPSAFE|CTLFLAG_RW, &lodirtybuffers, + __offsetof(struct bufdomain, bd_lodirtybuffers), sysctl_bufdomain_int, "L", "How many buffers we want to have free before bufdaemon can sleep"); static int hidirtybuffers; -SYSCTL_INT(_vfs, OID_AUTO, hidirtybuffers, CTLFLAG_RW, &hidirtybuffers, 0, +SYSCTL_PROC(_vfs, OID_AUTO, hidirtybuffers, + CTLTYPE_LONG|CTLFLAG_MPSAFE|CTLFLAG_RW, &hidirtybuffers, + __offsetof(struct bufdomain, bd_hidirtybuffers), sysctl_bufdomain_int, "L", "When the number of dirty buffers is considered severe"); int dirtybufthresh; -SYSCTL_INT(_vfs, OID_AUTO, dirtybufthresh, CTLFLAG_RW, &dirtybufthresh, - 0, "Number of bdwrite to bawrite conversions to clear dirty buffers"); +SYSCTL_PROC(_vfs, OID_AUTO, dirtybufthresh, + CTLTYPE_LONG|CTLFLAG_MPSAFE|CTLFLAG_RW, &dirtybufthresh, + __offsetof(struct bufdomain, bd_dirtybufthresh), sysctl_bufdomain_int, "L", + "Number of bdwrite to bawrite conversions to clear dirty buffers"); static int numfreebuffers; SYSCTL_INT(_vfs, OID_AUTO, numfreebuffers, CTLFLAG_RD, &numfreebuffers, 0, "Number of free buffers"); static int lofreebuffers; -SYSCTL_INT(_vfs, OID_AUTO, lofreebuffers, CTLFLAG_RW, &lofreebuffers, 0, +SYSCTL_PROC(_vfs, OID_AUTO, lofreebuffers, + CTLTYPE_LONG|CTLFLAG_MPSAFE|CTLFLAG_RW, &lofreebuffers, + __offsetof(struct bufdomain, bd_lofreebuffers), sysctl_bufdomain_int, "L", "Target number of free buffers"); static int hifreebuffers; -SYSCTL_INT(_vfs, OID_AUTO, hifreebuffers, CTLFLAG_RW, &hifreebuffers, 0, +SYSCTL_PROC(_vfs, OID_AUTO, hifreebuffers, + CTLTYPE_LONG|CTLFLAG_MPSAFE|CTLFLAG_RW, &hifreebuffers, + __offsetof(struct bufdomain, bd_hifreebuffers), sysctl_bufdomain_int, "L", "Threshold for clean buffer recycling"); static counter_u64_t getnewbufcalls; SYSCTL_COUNTER_U64(_vfs, OID_AUTO, getnewbufcalls, CTLFLAG_RD, @@ -294,66 +370,19 @@ static int bdirtywait; #define QUEUE_CLEAN 3 /* non-B_DELWRI buffers */ #define QUEUE_SENTINEL 4 /* not an queue index, but mark for sentinel */ -struct bufqueue { - struct mtx_padalign bq_lock; - TAILQ_HEAD(, buf) bq_queue; - uint8_t bq_index; - uint16_t bq_subqueue; - int bq_len; -} __aligned(CACHE_LINE_SIZE); +/* Maximum number of buffer domains. */ +#define BUF_DOMAINS 8 -#define BQ_LOCKPTR(bq) (&(bq)->bq_lock) -#define BQ_LOCK(bq) mtx_lock(BQ_LOCKPTR((bq))) -#define BQ_UNLOCK(bq) mtx_unlock(BQ_LOCKPTR((bq))) -#define BQ_ASSERT_LOCKED(bq) mtx_assert(BQ_LOCKPTR((bq)), MA_OWNED) +struct bufdomainset bdlodirty; /* Domains > lodirty */ +struct bufdomainset bdhidirty; /* Domains > hidirty */ -struct bufqueue __exclusive_cache_line bqempty; -struct bufqueue __exclusive_cache_line bqdirty; - -struct bufdomain { - struct bufqueue bd_subq[MAXCPU + 1]; /* Per-cpu sub queues + global */ - struct bufqueue *bd_cleanq; - struct mtx_padalign bd_run_lock; - /* Constants */ - long bd_maxbufspace; - long bd_hibufspace; - long bd_lobufspace; - long bd_bufspacethresh; - int bd_hifreebuffers; - int bd_lofreebuffers; - int bd_lim; - /* atomics */ - int bd_wanted; - int __aligned(CACHE_LINE_SIZE) bd_running; - long __aligned(CACHE_LINE_SIZE) bd_bufspace; - int __aligned(CACHE_LINE_SIZE) bd_freebuffers; -} __aligned(CACHE_LINE_SIZE); - -#define BD_LOCKPTR(bd) (&(bd)->bd_cleanq->bq_lock) -#define BD_LOCK(bd) mtx_lock(BD_LOCKPTR((bd))) -#define BD_UNLOCK(bd) mtx_unlock(BD_LOCKPTR((bd))) -#define BD_ASSERT_LOCKED(bd) mtx_assert(BD_LOCKPTR((bd)), MA_OWNED) -#define BD_RUN_LOCKPTR(bd) (&(bd)->bd_run_lock) -#define BD_RUN_LOCK(bd) mtx_lock(BD_RUN_LOCKPTR((bd))) -#define BD_RUN_UNLOCK(bd) mtx_unlock(BD_RUN_LOCKPTR((bd))) -#define BD_DOMAIN(bd) (bd - bdclean) - -/* Maximum number of clean buffer domains. */ -#define CLEAN_DOMAINS 8 - /* Configured number of clean queues. */ -static int __read_mostly clean_domains; +static int __read_mostly buf_domains; -struct bufdomain __exclusive_cache_line bdclean[CLEAN_DOMAINS]; +BITSET_DEFINE(bufdomainset, BUF_DOMAINS); +struct bufdomain __exclusive_cache_line bdomain[BUF_DOMAINS]; +struct bufqueue __exclusive_cache_line bqempty; -static void bq_remove(struct bufqueue *bq, struct buf *bp); -static void bq_insert(struct bufqueue *bq, struct buf *bp, bool unlock); -static int buf_recycle(struct bufdomain *, bool kva); -static void bq_init(struct bufqueue *bq, int qindex, int cpu, - const char *lockname); -static void bd_init(struct bufdomain *bd); -static int bd_flushall(struct bufdomain *bd); - /* * per-cpu empty buffer cache. */ @@ -393,6 +422,44 @@ sysctl_runningspace(SYSCTL_HANDLER_ARGS) return (error); } +static int +sysctl_bufdomain_int(SYSCTL_HANDLER_ARGS) +{ + int error; + int value; + int i; + + value = *(int *)arg1; + error = sysctl_handle_int(oidp, &value, 0, req); + if (error != 0 || req->newptr == NULL) + return (error); + *(int *)arg1 = value; + for (i = 0; i < buf_domains; i++) + *(int *)(((uintptr_t)&bdomain[i]) + arg2) = + value / buf_domains; + + return (error); +} + +static int +sysctl_bufdomain_long(SYSCTL_HANDLER_ARGS) +{ + long value; + int error; + int i; + + value = *(long *)arg1; + error = sysctl_handle_long(oidp, &value, 0, req); + if (error != 0 || req->newptr == NULL) + return (error); + *(long *)arg1 = value; + for (i = 0; i < buf_domains; i++) + *(long *)(((uintptr_t)&bdomain[i]) + arg2) = + value / buf_domains; + + return (error); +} + #if defined(COMPAT_FREEBSD4) || defined(COMPAT_FREEBSD5) || \ defined(COMPAT_FREEBSD6) || defined(COMPAT_FREEBSD7) static int @@ -403,8 +470,8 @@ sysctl_bufspace(SYSCTL_HANDLER_ARGS) int i; lvalue = 0; - for (i = 0; i < clean_domains; i++) - lvalue += bdclean[i].bd_bufspace; + for (i = 0; i < buf_domains; i++) + lvalue += bdomain[i].bd_bufspace; if (sizeof(int) == sizeof(long) || req->oldlen >= sizeof(long)) return (sysctl_handle_long(oidp, &lvalue, 0, req)); if (lvalue > INT_MAX) @@ -421,12 +488,24 @@ sysctl_bufspace(SYSCTL_HANDLER_ARGS) int i; lvalue = 0; - for (i = 0; i < clean_domains; i++) - lvalue += bdclean[i].bd_bufspace; + for (i = 0; i < buf_domains; i++) + lvalue += bdomain[i].bd_bufspace; return (sysctl_handle_long(oidp, &lvalue, 0, req)); } #endif +static int +sysctl_numdirtybuffers(SYSCTL_HANDLER_ARGS) +{ + int value; + int i; + + value = 0; + for (i = 0; i < buf_domains; i++) + value += bdomain[i].bd_numdirtybuffers; + return (sysctl_handle_int(oidp, &value, 0, req)); +} + /* * bdirtywakeup: * @@ -444,6 +523,42 @@ bdirtywakeup(void) } /* + * bd_clear: + * + * Clear a domain from the appropriate bitsets when dirtybuffers + * is decremented. + */ +static void +bd_clear(struct bufdomain *bd) +{ + + mtx_lock(&bdirtylock); + if (bd->bd_numdirtybuffers <= bd->bd_lodirtybuffers) + BIT_CLR(BUF_DOMAINS, BD_DOMAIN(bd), &bdlodirty); + if (bd->bd_numdirtybuffers <= bd->bd_hidirtybuffers) + BIT_CLR(BUF_DOMAINS, BD_DOMAIN(bd), &bdhidirty); + mtx_unlock(&bdirtylock); +} + +/* + * bd_set: + * + * Set a domain in the appropriate bitsets when dirtybuffers + * is incremented. + */ +static void +bd_set(struct bufdomain *bd) +{ + + mtx_lock(&bdirtylock); + if (bd->bd_numdirtybuffers > bd->bd_lodirtybuffers) + BIT_SET(BUF_DOMAINS, BD_DOMAIN(bd), &bdlodirty); + if (bd->bd_numdirtybuffers > bd->bd_hidirtybuffers) + BIT_SET(BUF_DOMAINS, BD_DOMAIN(bd), &bdhidirty); + mtx_unlock(&bdirtylock); +} + +/* * bdirtysub: * * Decrement the numdirtybuffers count by one and wakeup any @@ -450,12 +565,17 @@ bdirtywakeup(void) * threads blocked in bwillwrite(). */ static void -bdirtysub(void) +bdirtysub(struct buf *bp) { + struct bufdomain *bd; + int num; - if (atomic_fetchadd_int(&numdirtybuffers, -1) == - (lodirtybuffers + hidirtybuffers) / 2) + bd = bufdomain(bp); + num = atomic_fetchadd_int(&bd->bd_numdirtybuffers, -1); + if (num == (bd->bd_lodirtybuffers + bd->bd_hidirtybuffers) / 2) bdirtywakeup(); + if (num == bd->bd_lodirtybuffers || num == bd->bd_hidirtybuffers) + bd_clear(bd); } /* @@ -465,16 +585,21 @@ static void * daemon if needed. */ static void -bdirtyadd(void) +bdirtyadd(struct buf *bp) { + struct bufdomain *bd; + int num; /* * Only do the wakeup once as we cross the boundary. The * buf daemon will keep running until the condition clears. */ - if (atomic_fetchadd_int(&numdirtybuffers, 1) == - (lodirtybuffers + hidirtybuffers) / 2) + bd = bufdomain(bp); + num = atomic_fetchadd_int(&bd->bd_numdirtybuffers, 1); + if (num == (bd->bd_lodirtybuffers + bd->bd_hidirtybuffers) / 2) bd_wakeup(); + if (num == bd->bd_lodirtybuffers || num == bd->bd_hidirtybuffers) + bd_set(bd); } /* @@ -539,11 +664,11 @@ bufspace_adjust(struct buf *bp, int bufsize) KASSERT((bp->b_flags & B_MALLOC) == 0, ("bufspace_adjust: malloc buf %p", bp)); - bd = &bdclean[bp->b_domain]; + bd = bufdomain(bp); diff = bufsize - bp->b_bufsize; if (diff < 0) { atomic_subtract_long(&bd->bd_bufspace, -diff); - } else { + } else if (diff > 0) { space = atomic_fetchadd_long(&bd->bd_bufspace, diff); /* Wake up the daemon on the transition. */ if (space < bd->bd_bufspacethresh && @@ -638,7 +763,7 @@ bufspace_wait(struct bufdomain *bd, struct vnode * * recursion. */ td->td_pflags |= TDP_BUFNEED | TDP_NORUNNINGBUF; - fl = buf_flush(vp, flushbufqtarget); + fl = buf_flush(vp, bd, flushbufqtarget); td->td_pflags &= norunbuf; BD_LOCK(bd); if (fl != 0) @@ -700,6 +825,15 @@ bufspace_daemon(void *arg) if (buf_recycle(bd, false) != 0) { if (bd_flushall(bd)) continue; + /* + * Speedup dirty if we've run out of clean + * buffers. This is possible in particular + * because softdep may held many bufs locked + * pending writes to other bufs which are + * marked for delayed write, exhausting + * clean space until they are written. + */ + bd_speedup(); BD_LOCK(bd); if (bd->bd_wanted) { msleep(&bd->bd_wanted, BD_LOCKPTR(bd), @@ -1025,7 +1159,6 @@ bufinit(void) ("maxbcachebuf (%d) must be >= MAXBSIZE (%d)\n", maxbcachebuf, MAXBSIZE)); bq_init(&bqempty, QUEUE_EMPTY, -1, "bufq empty lock"); - bq_init(&bqdirty, QUEUE_DIRTY, -1, "bufq dirty lock"); mtx_init(&rbreqlock, "runningbufspace lock", NULL, MTX_DEF); mtx_init(&bdlock, "buffer daemon lock", NULL, MTX_DEF); mtx_init(&bdirtylock, "dirty buf lock", NULL, MTX_DEF); @@ -1093,7 +1226,6 @@ bufinit(void) */ hidirtybuffers = nbuf / 4 + 20; dirtybufthresh = hidirtybuffers * 9 / 10; - numdirtybuffers = 0; /* * To support extreme low-memory systems, make sure hidirtybuffers * cannot eat up all available buffer space. This occurs when our @@ -1128,22 +1260,26 @@ bufinit(void) * One queue per-256mb up to the max. More queues gives better * concurrency but less accurate LRU. */ - clean_domains = MIN(howmany(maxbufspace, 256*1024*1024), CLEAN_DOMAINS); - for (i = 0 ; i < clean_domains; i++) { + buf_domains = MIN(howmany(maxbufspace, 256*1024*1024), BUF_DOMAINS); + for (i = 0 ; i < buf_domains; i++) { struct bufdomain *bd; - bd = &bdclean[i]; + bd = &bdomain[i]; bd_init(bd); - bd->bd_freebuffers = nbuf / clean_domains; - bd->bd_hifreebuffers = hifreebuffers / clean_domains; - bd->bd_lofreebuffers = lofreebuffers / clean_domains; + bd->bd_freebuffers = nbuf / buf_domains; + bd->bd_hifreebuffers = hifreebuffers / buf_domains; + bd->bd_lofreebuffers = lofreebuffers / buf_domains; bd->bd_bufspace = 0; - bd->bd_maxbufspace = maxbufspace / clean_domains; - bd->bd_hibufspace = hibufspace / clean_domains; - bd->bd_lobufspace = lobufspace / clean_domains; - bd->bd_bufspacethresh = bufspacethresh / clean_domains; + bd->bd_maxbufspace = maxbufspace / buf_domains; + bd->bd_hibufspace = hibufspace / buf_domains; + bd->bd_lobufspace = lobufspace / buf_domains; + bd->bd_bufspacethresh = bufspacethresh / buf_domains; + bd->bd_numdirtybuffers = 0; + bd->bd_hidirtybuffers = hidirtybuffers / buf_domains; + bd->bd_lodirtybuffers = lodirtybuffers / buf_domains; + bd->bd_dirtybufthresh = dirtybufthresh / buf_domains; /* Don't allow more than 2% of bufs in the per-cpu caches. */ - bd->bd_lim = nbuf / clean_domains / 50 / mp_ncpus; + bd->bd_lim = nbuf / buf_domains / 50 / mp_ncpus; } getnewbufcalls = counter_u64_alloc(M_WAITOK); getnewbufrestarts = counter_u64_alloc(M_WAITOK); @@ -1327,6 +1463,13 @@ bpmap_qenter(struct buf *bp) (vm_offset_t)(bp->b_offset & PAGE_MASK)); } +static inline struct bufdomain * +bufdomain(struct buf *bp) +{ + + return (&bdomain[bp->b_domain]); +} + static struct bufqueue * bufqueue(struct buf *bp) { @@ -1339,9 +1482,9 @@ bufqueue(struct buf *bp) case QUEUE_EMPTY: return (&bqempty); case QUEUE_DIRTY: - return (&bqdirty); + return (&bufdomain(bp)->bd_dirtyq); case QUEUE_CLEAN: - return (&bdclean[bp->b_domain].bd_subq[bp->b_subqueue]); + return (&bufdomain(bp)->bd_subq[bp->b_subqueue]); default: break; } @@ -1404,14 +1547,14 @@ binsfree(struct buf *bp, int qindex) bq_remove(bq, bp); BQ_UNLOCK(bq); } + bd = bufdomain(bp); if (qindex == QUEUE_CLEAN) { - bd = &bdclean[bp->b_domain]; if (bd->bd_lim != 0) bq = &bd->bd_subq[PCPU_GET(cpuid)]; else bq = bd->bd_cleanq; } else - bq = &bqdirty; + bq = &bd->bd_dirtyq; bq_insert(bq, bp, true); } @@ -1439,7 +1582,7 @@ buf_free(struct buf *bp) if (!LIST_EMPTY(&bp->b_dep)) buf_deallocate(bp); bufkva_free(bp); - atomic_add_int(&bdclean[bp->b_domain].bd_freebuffers, 1); + atomic_add_int(&bufdomain(bp)->bd_freebuffers, 1); BUF_UNLOCK(bp); uma_zfree(buf_zone, bp); } @@ -1715,9 +1858,10 @@ bd_init(struct bufdomain *bd) int domain; int i; - domain = bd - bdclean; + domain = bd - bdomain; bd->bd_cleanq = &bd->bd_subq[mp_maxid + 1]; bq_init(bd->bd_cleanq, QUEUE_CLEAN, mp_maxid + 1, "bufq clean lock"); + bq_init(&bd->bd_dirtyq, QUEUE_DIRTY, -1, "bufq dirty lock"); for (i = 0; i <= mp_maxid; i++) bq_init(&bd->bd_subq[i], QUEUE_CLEAN, i, "bufq clean subqueue lock"); @@ -1809,7 +1953,7 @@ bq_insert(struct bufqueue *bq, struct buf *bp, boo if (bp->b_qindex != QUEUE_NONE) panic("bq_insert: free buffer %p onto another queue?", bp); - bd = &bdclean[bp->b_domain]; + bd = bufdomain(bp); if (bp->b_flags & B_AGE) { /* Place this buf directly on the real queue. */ if (bq->bq_index == QUEUE_CLEAN) @@ -1926,8 +2070,8 @@ bufkva_reclaim(vmem_t *vmem, int flags) done = false; for (i = 0; i < 5; i++) { - for (q = 0; q < clean_domains; q++) - if (buf_recycle(&bdclean[q], true) != 0) + for (q = 0; q < buf_domains; q++) + if (buf_recycle(&bdomain[q], true) != 0) done = true; if (done) break; @@ -2319,7 +2463,7 @@ bdirty(struct buf *bp) if ((bp->b_flags & B_DELWRI) == 0) { bp->b_flags |= /* XXX B_DONE | */ B_DELWRI; reassignbuf(bp); - bdirtyadd(); + bdirtyadd(bp); } } @@ -2347,7 +2491,7 @@ bundirty(struct buf *bp) if (bp->b_flags & B_DELWRI) { bp->b_flags &= ~B_DELWRI; reassignbuf(bp); - bdirtysub(); + bdirtysub(bp); } /* * Since it is now being written, we can clear its deferred write flag. @@ -2419,9 +2563,9 @@ void bwillwrite(void) { - if (numdirtybuffers >= hidirtybuffers) { + if (buf_dirty_count_severe()) { mtx_lock(&bdirtylock); - while (numdirtybuffers >= hidirtybuffers) { + while (buf_dirty_count_severe()) { bdirtywait = 1; msleep(&bdirtywait, &bdirtylock, (PRIBIO + 4), "flswai", 0); @@ -2437,7 +2581,7 @@ int buf_dirty_count_severe(void) { - return(numdirtybuffers >= hidirtybuffers); + return (!BIT_EMPTY(BUF_DOMAINS, &bdhidirty)); } /* @@ -2522,7 +2666,7 @@ brelse(struct buf *bp) if (!LIST_EMPTY(&bp->b_dep)) buf_deallocate(bp); if (bp->b_flags & B_DELWRI) - bdirtysub(); + bdirtysub(bp); bp->b_flags &= ~(B_DELWRI | B_CACHE); if ((bp->b_flags & B_VMIO) == 0) { allocbuf(bp, 0); @@ -3135,9 +3279,9 @@ getnewbuf(struct vnode *vp, int slpflag, int slpti else metadata = false; if (vp == NULL) - bd = &bdclean[0]; + bd = &bdomain[0]; else - bd = &bdclean[vp->v_bufobj.bo_domain]; + bd = &bdomain[vp->v_bufobj.bo_domain]; counter_u64_add(getnewbufcalls, 1); reserved = false; @@ -3183,11 +3327,11 @@ static struct kproc_desc buf_kp = { SYSINIT(bufdaemon, SI_SUB_KTHREAD_BUF, SI_ORDER_FIRST, kproc_start, &buf_kp); static int -buf_flush(struct vnode *vp, int target) +buf_flush(struct vnode *vp, struct bufdomain *bd, int target) { int flushed; - flushed = flushbufqueues(vp, target, 0); + flushed = flushbufqueues(vp, bd, target, 0); if (flushed == 0) { /* * Could not find any buffers without rollback @@ -3196,7 +3340,7 @@ static int */ if (vp != NULL && target > 2) target /= 2; - flushbufqueues(vp, target, 1); + flushbufqueues(vp, bd, target, 1); } return (flushed); } @@ -3204,6 +3348,8 @@ static int static void buf_daemon() { + struct bufdomain *bd; + int speedupreq; int lodirty; int i; @@ -3216,11 +3362,11 @@ buf_daemon() /* * Start the buf clean daemons as children threads. */ - for (i = 0 ; i < clean_domains; i++) { + for (i = 0 ; i < buf_domains; i++) { int error; error = kthread_add((void (*)(void *))bufspace_daemon, - &bdclean[i], curproc, NULL, 0, 0, "bufspacedaemon-%d", i); + &bdomain[i], curproc, NULL, 0, 0, "bufspacedaemon-%d", i); if (error) panic("error %d spawning bufspace daemon", error); } @@ -3235,20 +3381,30 @@ buf_daemon() mtx_unlock(&bdlock); kproc_suspend_check(bufdaemonproc); - lodirty = lodirtybuffers; - if (bd_speedupreq) { - lodirty = numdirtybuffers / 2; - bd_speedupreq = 0; - } + /* - * Do the flush. Limit the amount of in-transit I/O we - * allow to build up, otherwise we would completely saturate - * the I/O system. + * Save speedupreq for this pass and reset to capture new + * requests. */ - while (numdirtybuffers > lodirty) { - if (buf_flush(NULL, numdirtybuffers - lodirty) == 0) - break; - kern_yield(PRI_USER); + speedupreq = bd_speedupreq; + bd_speedupreq = 0; + + /* + * Flush each domain sequentially according to its level and + * the speedup request. + */ + for (i = 0; i < buf_domains; i++) { + bd = &bdomain[i]; + if (speedupreq) + lodirty = bd->bd_numdirtybuffers / 2; + else + lodirty = bd->bd_lodirtybuffers; + while (bd->bd_numdirtybuffers > lodirty) { + if (buf_flush(NULL, bd, + bd->bd_numdirtybuffers - lodirty) == 0) + break; + kern_yield(PRI_USER); + } } /* @@ -3262,7 +3418,7 @@ buf_daemon() * to avoid endless loops on unlockable buffers. */ mtx_lock(&bdlock); - if (numdirtybuffers <= lodirtybuffers) { + if (!BIT_EMPTY(BUF_DOMAINS, &bdlodirty)) { /* * We reached our low water mark, reset the * request and sleep until we are needed again. @@ -3301,7 +3457,8 @@ SYSCTL_INT(_vfs, OID_AUTO, flushwithdeps, CTLFLAG_ 0, "Number of buffers flushed with dependecies that require rollbacks"); static int -flushbufqueues(struct vnode *lvp, int target, int flushdeps) +flushbufqueues(struct vnode *lvp, struct bufdomain *bd, int target, + int flushdeps) { struct bufqueue *bq; struct buf *sentinel; @@ -3314,7 +3471,7 @@ static int bool unlock; flushed = 0; - bq = &bqdirty; + bq = &bd->bd_dirtyq; bp = NULL; sentinel = malloc(sizeof(struct buf), M_TEMP, M_WAITOK | M_ZERO); sentinel->b_qindex = QUEUE_SENTINEL; @@ -3650,7 +3807,7 @@ bp_unmapped_get_kva(struct buf *bp, daddr_t blkno, panic("GB_NOWAIT_BD and GB_UNMAPPED %p", bp); } counter_u64_add(mappingrestarts, 1); - bufspace_wait(&bdclean[bp->b_domain], bp->b_vp, gbflags, 0, 0); + bufspace_wait(bufdomain(bp), bp->b_vp, gbflags, 0, 0); } has_addr: if (need_mapping) { @@ -3848,7 +4005,7 @@ loop: */ if (flags & GB_NOCREAT) return NULL; - if (bdclean[bo->bo_domain].bd_freebuffers == 0 && + if (bdomain[bo->bo_domain].bd_freebuffers == 0 && TD_IS_IDLETHREAD(curthread)) return NULL; @@ -3905,7 +4062,7 @@ loop: if (gbincore(bo, blkno)) { BO_UNLOCK(bo); bp->b_flags |= B_INVAL; - bufspace_release(&bdclean[bp->b_domain], maxsize); + bufspace_release(bufdomain(bp), maxsize); brelse(bp); goto loop; } @@ -3940,7 +4097,7 @@ loop: } allocbuf(bp, size); - bufspace_release(&bdclean[bp->b_domain], maxsize); + bufspace_release(bufdomain(bp), maxsize); bp->b_flags &= ~B_DONE; } CTR4(KTR_BUF, "getblk(%p, %ld, %d) = %p", vp, (long)blkno, size, bp); @@ -3969,7 +4126,7 @@ geteblk(int size, int flags) return (NULL); } allocbuf(bp, size); - bufspace_release(&bdclean[bp->b_domain], maxsize); + bufspace_release(bufdomain(bp), maxsize); bp->b_flags |= B_INVAL; /* b_dep cleared by getnewbuf() */ BUF_ASSERT_HELD(bp); return (bp); @@ -4838,7 +4995,7 @@ bufobj_init(struct bufobj *bo, void *private) static volatile int bufobj_cleanq; bo->bo_domain = - atomic_fetchadd_int(&bufobj_cleanq, 1) % clean_domains; + atomic_fetchadd_int(&bufobj_cleanq, 1) % buf_domains; rw_init(BO_LOCKPTR(bo), "bufobj interlock"); bo->bo_private = private; TAILQ_INIT(&bo->bo_clean.bv_hd); @@ -5160,6 +5317,7 @@ DB_SHOW_COMMAND(buffer, db_show_buffer) } db_printf("\n"); } + BUF_LOCKPRINTINFO(bp); #if defined(FULL_BUF_TRACKING) db_printf("b_io_tracking: b_io_tcnt = %u\n", bp->b_io_tcnt); @@ -5174,19 +5332,19 @@ DB_SHOW_COMMAND(buffer, db_show_buffer) db_printf("b_io_tracking: %s\n", bp->b_io_tracking); #endif db_printf(" "); - BUF_LOCKPRINTINFO(bp); } DB_SHOW_COMMAND(bufqueues, bufqueues) { struct bufdomain *bd; - int i, j; + struct buf *bp; + long total; + int i, j, cnt; db_printf("bqempty: %d\n", bqempty.bq_len); - db_printf("bqdirty: %d\n", bqdirty.bq_len); - for (i = 0; i < clean_domains; i++) { - bd = &bdclean[i]; + for (i = 0; i < buf_domains; i++) { + bd = &bdomain[i]; db_printf("Buf domain %d\n", i); db_printf("\tfreebufs\t%d\n", bd->bd_freebuffers); db_printf("\tlofreebufs\t%d\n", bd->bd_lofreebuffers); @@ -5198,7 +5356,21 @@ DB_SHOW_COMMAND(bufqueues, bufqueues) db_printf("\tlobufspace\t%ld\n", bd->bd_lobufspace); db_printf("\tbufspacethresh\t%ld\n", bd->bd_bufspacethresh); db_printf("\n"); - db_printf("\tcleanq count\t%d\n", bd->bd_cleanq->bq_len); + db_printf("\tnumdirtybuffers\t%d\n", bd->bd_numdirtybuffers); + db_printf("\tlodirtybuffers\t%d\n", bd->bd_lodirtybuffers); + db_printf("\thidirtybuffers\t%d\n", bd->bd_hidirtybuffers); + db_printf("\tdirtybufthresh\t%d\n", bd->bd_dirtybufthresh); + db_printf("\n"); + total = 0; + TAILQ_FOREACH(bp, &bd->bd_cleanq->bq_queue, b_freelist) + total += bp->b_bufsize; + db_printf("\tcleanq count\t%d (%ld)\n", + bd->bd_cleanq->bq_len, total); + total = 0; + TAILQ_FOREACH(bp, &bd->bd_dirtyq.bq_queue, b_freelist) + total += bp->b_bufsize; + db_printf("\tdirtyq count\t%d (%ld)\n", + bd->bd_dirtyq.bq_len, total); db_printf("\twakeup\t\t%d\n", bd->bd_wanted); db_printf("\tlim\t\t%d\n", bd->bd_lim); db_printf("\tCPU "); @@ -5205,6 +5377,22 @@ DB_SHOW_COMMAND(bufqueues, bufqueues) for (j = 0; j <= mp_maxid; j++) db_printf("%d, ", bd->bd_subq[j].bq_len); db_printf("\n"); + cnt = 0; + total = 0; + for (j = 0; j < nbuf; j++) + if (buf[j].b_domain == i && BUF_ISLOCKED(&buf[j])) { + cnt++; + total += buf[j].b_bufsize; + } + db_printf("\tLocked buffers: %d space %jd\n", cnt, total); + cnt = 0; + total = 0; + for (j = 0; j < nbuf; j++) + if (buf[j].b_domain == i) { + cnt++; + total += buf[j].b_bufsize; + } + db_printf("\tTotal buffers: %d space %jd\n", cnt, total); } } Index: /usr/src/sys/vm/vm_page.c =================================================================== --- /usr/src/sys/vm/vm_page.c (revision 330873) +++ /usr/src/sys/vm/vm_page.c (working copy) @@ -139,7 +139,9 @@ extern int vmem_startup_count(void); struct vm_domain vm_dom[MAXMEMDOM]; struct mtx_padalign __exclusive_cache_line pa_lock[PA_LOCK_COUNT]; + struct mtx_padalign __exclusive_cache_line vm_domainset_lock; +/* The following fields are protected by the domainset lock. */ domainset_t __exclusive_cache_line vm_min_domains; domainset_t __exclusive_cache_line vm_severe_domains; static int vm_min_waiters; @@ -146,7 +148,6 @@ static int vm_min_waiters; static int vm_severe_waiters; static int vm_pageproc_waiters; - /* * bogus page -- for I/O to/from partially complete buffers, * or for paging into sparsely invalid regions. @@ -184,7 +185,6 @@ static void vm_page_insert_radixdone(vm_page_t m, vm_page_t mpred); static int vm_page_reclaim_run(int req_class, int domain, u_long npages, vm_page_t m_run, vm_paddr_t high); -static void vm_domain_free_wakeup(struct vm_domain *); static int vm_domain_alloc_fail(struct vm_domain *vmd, vm_object_t object, int req); @@ -430,6 +430,7 @@ vm_page_domain_init(int domain) MTX_DEF | MTX_DUPOK); } mtx_init(&vmd->vmd_free_mtx, "vm page free queue", NULL, MTX_DEF); + mtx_init(&vmd->vmd_pageout_mtx, "vm pageout lock", NULL, MTX_DEF); snprintf(vmd->vmd_name, sizeof(vmd->vmd_name), "%d", domain); } @@ -731,8 +732,8 @@ vm_page_startup(vm_offset_t vaddr) vmd = VM_DOMAIN(seg->domain); vm_domain_free_lock(vmd); vm_phys_free_contig(m, pagecount); - vm_domain_freecnt_adj(vmd, (int)pagecount); vm_domain_free_unlock(vmd); + vm_domain_freecnt_inc(vmd, pagecount); vm_cnt.v_page_count += (u_int)pagecount; vmd = VM_DOMAIN(seg->domain); @@ -1694,7 +1695,6 @@ vm_page_alloc_domain_after(vm_object_t object, vm_ struct vm_domain *vmd; vm_page_t m; int flags; - u_int free_count; KASSERT((object != NULL) == ((req & VM_ALLOC_NOOBJ) == 0) && (object != NULL || (req & VM_ALLOC_SBUSY) == 0) && @@ -1747,6 +1747,9 @@ again: #endif } } + if (m != NULL) + vm_domain_freecnt_dec(vmd, 1); + vm_domain_free_unlock(vmd); if (m == NULL) { /* * Not allocatable, give up. @@ -1760,15 +1763,7 @@ again: * At this point we had better have found a good page. */ KASSERT(m != NULL, ("missing page")); - free_count = vm_domain_freecnt_adj(vmd, -1); - vm_domain_free_unlock(vmd); - /* - * Don't wakeup too often - wakeup the pageout daemon when - * we would be nearly out of memory. - */ - if (vm_paging_needed(vmd, free_count)) - pagedaemon_wakeup(vmd->vmd_domain); #if VM_NRESERVLEVEL > 0 found: #endif @@ -1804,7 +1799,6 @@ found: if (object != NULL) { if (vm_page_insert_after(m, object, pindex, mpred)) { - pagedaemon_wakeup(domain); if (req & VM_ALLOC_WIRED) { vm_wire_sub(1); m->wire_count = 0; @@ -1961,13 +1955,14 @@ retry: goto retry; #endif } + if (m_ret != NULL) + vm_domain_freecnt_dec(vmd, npages); + vm_domain_free_unlock(vmd); if (m_ret == NULL) { if (vm_domain_alloc_fail(vmd, object, req)) goto again; return (NULL); } - vm_domain_freecnt_adj(vmd, -npages); - vm_domain_free_unlock(vmd); #if VM_NRESERVLEVEL > 0 found: #endif @@ -2006,7 +2001,6 @@ found: m->oflags = oflags; if (object != NULL) { if (vm_page_insert_after(m, object, pindex, mpred)) { - pagedaemon_wakeup(domain); if ((req & VM_ALLOC_WIRED) != 0) vm_wire_sub(npages); KASSERT(m->object == NULL, @@ -2035,9 +2029,6 @@ found: pmap_page_set_memattr(m, memattr); pindex++; } - vmd = VM_DOMAIN(domain); - if (vm_paging_needed(vmd, vmd->vmd_free_count)) - pagedaemon_wakeup(domain); return (m_ret); } @@ -2100,7 +2091,7 @@ vm_page_alloc_freelist_domain(int domain, int free { struct vm_domain *vmd; vm_page_t m; - u_int flags, free_count; + u_int flags; /* * Do not allocate reserved pages unless the req has asked for it. @@ -2111,13 +2102,14 @@ again: if (vm_domain_available(vmd, req, 1)) m = vm_phys_alloc_freelist_pages(domain, freelist, VM_FREEPOOL_DIRECT, 0); + if (m != NULL) + vm_domain_freecnt_dec(vmd, 1); + vm_domain_free_unlock(vmd); if (m == NULL) { if (vm_domain_alloc_fail(vmd, NULL, req)) goto again; return (NULL); } - free_count = vm_domain_freecnt_adj(vmd, -1); - vm_domain_free_unlock(vmd); vm_page_alloc_check(m); /* @@ -2138,8 +2130,6 @@ again: } /* Unmanaged pages don't use "act_count". */ m->oflags = VPO_UNMANAGED; - if (vm_paging_needed(vmd, free_count)) - pagedaemon_wakeup(domain); return (m); } @@ -2539,15 +2529,19 @@ unlock: if (m_mtx != NULL) mtx_unlock(m_mtx); if ((m = SLIST_FIRST(&free)) != NULL) { + int cnt; + vmd = VM_DOMAIN(domain); + cnt = 0; vm_domain_free_lock(vmd); do { MPASS(vm_phys_domain(m) == domain); SLIST_REMOVE_HEAD(&free, plinks.s.ss); vm_page_free_phys(vmd, m); + cnt++; } while ((m = SLIST_FIRST(&free)) != NULL); - vm_domain_free_wakeup(vmd); vm_domain_free_unlock(vmd); + vm_domain_freecnt_inc(vmd, cnt); } return (error); } @@ -2710,7 +2704,7 @@ vm_domain_set(struct vm_domain *vmd) /* * Clear the domain from the appropriate page level domainset. */ -static void +void vm_domain_clear(struct vm_domain *vmd) { @@ -2731,6 +2725,22 @@ vm_domain_clear(struct vm_domain *vmd) wakeup(&vm_severe_domains); } } + + /* + * If pageout daemon needs pages, then tell it that there are + * some free. + */ + if (vmd->vmd_pageout_pages_needed && + vmd->vmd_free_count >= vmd->vmd_pageout_free_min) { + wakeup(&vmd->vmd_pageout_pages_needed); + vmd->vmd_pageout_pages_needed = 0; + } + + /* See comments in vm_wait_doms(). */ + if (vm_pageproc_waiters) { + vm_pageproc_waiters = 0; + wakeup(&vm_pageproc_waiters); + } mtx_unlock(&vm_domainset_lock); } @@ -2769,7 +2779,7 @@ u_int vm_wait_count(void) { - return (vm_severe_waiters + vm_min_waiters); + return (vm_severe_waiters + vm_min_waiters + vm_pageproc_waiters); } static void @@ -2787,9 +2797,8 @@ vm_wait_doms(const domainset_t *wdoms) if (curproc == pageproc) { mtx_lock(&vm_domainset_lock); vm_pageproc_waiters++; - msleep(&vm_pageproc_waiters, &vm_domainset_lock, PVM, + msleep(&vm_pageproc_waiters, &vm_domainset_lock, PVM | PDROP, "pageprocwait", 1); - mtx_unlock(&vm_domainset_lock); } else { /* * XXX Ideally we would wait only until the allocation could @@ -2819,14 +2828,17 @@ vm_wait_domain(int domain) domainset_t wdom; vmd = VM_DOMAIN(domain); - vm_domain_free_assert_locked(vmd); + vm_domain_free_assert_unlocked(vmd); if (curproc == pageproc) { - vmd->vmd_pageout_pages_needed = 1; - msleep(&vmd->vmd_pageout_pages_needed, - vm_domain_free_lockptr(vmd), PDROP | PSWP, "VMWait", 0); + mtx_lock(&vm_domainset_lock); + if (vmd->vmd_free_count < vmd->vmd_pageout_free_min) { + vmd->vmd_pageout_pages_needed = 1; + msleep(&vmd->vmd_pageout_pages_needed, + &vm_domainset_lock, PDROP | PSWP, "VMWait", 0); + } else + mtx_unlock(&vm_domainset_lock); } else { - vm_domain_free_unlock(vmd); if (pageproc == NULL) panic("vm_wait in early boot"); DOMAINSET_ZERO(&wdom); @@ -2876,7 +2888,7 @@ static int vm_domain_alloc_fail(struct vm_domain *vmd, vm_object_t object, int req) { - vm_domain_free_assert_locked(vmd); + vm_domain_free_assert_unlocked(vmd); atomic_add_int(&vmd->vmd_pageout_deficit, max((u_int)req >> VM_ALLOC_COUNT_SHIFT, 1)); @@ -2888,10 +2900,8 @@ vm_domain_alloc_fail(struct vm_domain *vmd, vm_obj VM_OBJECT_WLOCK(object); if (req & VM_ALLOC_WAITOK) return (EAGAIN); - } else { - vm_domain_free_unlock(vmd); - pagedaemon_wakeup(vmd->vmd_domain); } + return (0); } @@ -3063,46 +3073,6 @@ vm_page_activate(vm_page_t m) } /* - * vm_domain_free_wakeup: - * - * Helper routine for vm_page_free_toq(). This routine is called - * when a page is added to the free queues. - * - * The page queues must be locked. - */ -static void -vm_domain_free_wakeup(struct vm_domain *vmd) -{ - - vm_domain_free_assert_locked(vmd); - - /* - * if pageout daemon needs pages, then tell it that there are - * some free. - */ - if (vmd->vmd_pageout_pages_needed && - vmd->vmd_free_count >= vmd->vmd_pageout_free_min) { - wakeup(&vmd->vmd_pageout_pages_needed); - vmd->vmd_pageout_pages_needed = 0; - } - /* - * wakeup processes that are waiting on memory if we hit a - * high water mark. And wakeup scheduler process if we have - * lots of memory. this process will swapin processes. - */ - if ((vmd->vmd_minset && !vm_paging_min(vmd)) || - (vmd->vmd_severeset && !vm_paging_severe(vmd))) - vm_domain_clear(vmd); - - /* See comments in vm_wait(); */ - if (vm_pageproc_waiters) { - vm_pageproc_waiters = 0; - wakeup(&vm_pageproc_waiters); - } - -} - -/* * vm_page_free_prep: * * Prepares the given page to be put on the free list, @@ -3183,7 +3153,8 @@ vm_page_free_prep(vm_page_t m, bool pagequeue_lock /* * Insert the page into the physical memory allocator's free page - * queues. This is the last step to free a page. + * queues. This is the last step to free a page. The caller is + * responsible for adjusting the free page count. */ static void vm_page_free_phys(struct vm_domain *vmd, vm_page_t m) @@ -3191,7 +3162,6 @@ vm_page_free_phys(struct vm_domain *vmd, vm_page_t vm_domain_free_assert_locked(vmd); - vm_domain_freecnt_adj(vmd, 1); #if VM_NRESERVLEVEL > 0 if (!vm_reserv_free_page(m)) #endif @@ -3203,24 +3173,28 @@ vm_page_free_phys_pglist(struct pglist *tq) { struct vm_domain *vmd; vm_page_t m; + int cnt; if (TAILQ_EMPTY(tq)) return; vmd = NULL; + cnt = 0; TAILQ_FOREACH(m, tq, listq) { if (vmd != vm_pagequeue_domain(m)) { if (vmd != NULL) { - vm_domain_free_wakeup(vmd); vm_domain_free_unlock(vmd); + vm_domain_freecnt_inc(vmd, cnt); + cnt = 0; } vmd = vm_pagequeue_domain(m); vm_domain_free_lock(vmd); } vm_page_free_phys(vmd, m); + cnt++; } if (vmd != NULL) { - vm_domain_free_wakeup(vmd); vm_domain_free_unlock(vmd); + vm_domain_freecnt_inc(vmd, cnt); } } @@ -3243,8 +3217,8 @@ vm_page_free_toq(vm_page_t m) vmd = vm_pagequeue_domain(m); vm_domain_free_lock(vmd); vm_page_free_phys(vmd, m); - vm_domain_free_wakeup(vmd); vm_domain_free_unlock(vmd); + vm_domain_freecnt_inc(vmd, 1); } /* Index: /usr/src/sys/vm/vm_pageout.c =================================================================== --- /usr/src/sys/vm/vm_pageout.c (revision 330873) +++ /usr/src/sys/vm/vm_pageout.c (working copy) @@ -1790,7 +1790,13 @@ vm_pageout_worker(void *arg) * The pageout daemon worker is never done, so loop forever. */ while (TRUE) { - vm_domain_free_lock(vmd); + vm_domain_pageout_lock(vmd); + /* + * We need to clear wanted before we check the limits. This + * prevents races with wakers who will check wanted after they + * reach the limit. + */ + atomic_store_int(&vmd->vmd_pageout_wanted, 0); /* * Might the page daemon need to run again? @@ -1801,7 +1807,7 @@ vm_pageout_worker(void *arg) * we have performed a level >= 1 (page reclamation) * scan, then sleep a bit and try again. */ - vm_domain_free_unlock(vmd); + vm_domain_pageout_unlock(vmd); if (pass > 1) pause("pwait", hz / VM_INACT_SCAN_RATE); } else { @@ -1809,12 +1815,18 @@ vm_pageout_worker(void *arg) * No, sleep until the next wakeup or until pages * need to have their reference stats updated. */ - vmd->vmd_pageout_wanted = false; if (mtx_sleep(&vmd->vmd_pageout_wanted, - vm_domain_free_lockptr(vmd), PDROP | PVM, + vm_domain_pageout_lockptr(vmd), PDROP | PVM, "psleep", hz / VM_INACT_SCAN_RATE) == 0) VM_CNT_INC(v_pdwakeups); } + /* Prevent spurious wakeups by ensuring that wanted is set. */ + atomic_store_int(&vmd->vmd_pageout_wanted, 1); + + /* + * Use the controller to calculate how many pages to free in + * this interval. + */ shortage = pidctrl_daemon(&vmd->vmd_pid, vmd->vmd_free_count); if (shortage && pass == 0) pass = 1; @@ -1970,10 +1982,14 @@ pagedaemon_wakeup(int domain) struct vm_domain *vmd; vmd = VM_DOMAIN(domain); - vm_domain_free_assert_unlocked(vmd); + vm_domain_pageout_assert_unlocked(vmd); + if (curproc == pageproc) + return; - if (!vmd->vmd_pageout_wanted && curthread->td_proc != pageproc) { - vmd->vmd_pageout_wanted = true; + if (atomic_fetchadd_int(&vmd->vmd_pageout_wanted, 1) == 0) { + vm_domain_pageout_lock(vmd); + atomic_store_int(&vmd->vmd_pageout_wanted, 1); wakeup(&vmd->vmd_pageout_wanted); + vm_domain_pageout_unlock(vmd); } } Index: /usr/src/sys/vm/vm_pagequeue.h =================================================================== --- /usr/src/sys/vm/vm_pagequeue.h (revision 330873) +++ /usr/src/sys/vm/vm_pagequeue.h (working copy) @@ -76,17 +76,31 @@ struct vm_pagequeue { #include struct sysctl_oid; +/* + * One vm_domain per-numa domain. Contains pagequeues, free page structures, + * and accounting. + * + * Lock Key: + * f vmd_free_mtx + * p vmd_pageout_mtx + * d vm_domainset_lock + * a atomic + * c const after boot +*/ struct vm_domain { struct vm_pagequeue vmd_pagequeues[PQ_COUNT]; struct mtx_padalign vmd_free_mtx; - struct vmem *vmd_kernel_arena; - u_int vmd_domain; /* Domain number. */ - u_int vmd_page_count; - long vmd_segs; /* bitmask of the segments */ + struct mtx_padalign vmd_pageout_mtx; + struct vmem *vmd_kernel_arena; /* (c) per-domain kva arena. */ + u_int vmd_domain; /* (c) Domain number. */ + u_int vmd_page_count; /* (c) Total page count. */ + long vmd_segs; /* (c) bitmask of the segments */ + u_int __aligned(CACHE_LINE_SIZE) vmd_free_count; /* (a,f) free page count */ + u_int vmd_pageout_deficit; /* (a) Estimated number of pages deficit */ + uint8_t vmd_pad[CACHE_LINE_SIZE - (sizeof(u_int) * 2)]; - /* Paging control variables, locked by domain_free_mtx. */ + /* Paging control variables, used within single threaded page daemon. */ struct pidctrl vmd_pid; /* Pageout controller. */ - u_int vmd_free_count; boolean_t vmd_oom; int vmd_oom_seq; int vmd_last_active_scan; @@ -94,11 +108,10 @@ struct vm_domain { struct vm_page vmd_marker; /* marker for pagedaemon private use */ struct vm_page vmd_inacthead; /* marker for LRU-defeating insertions */ - int vmd_pageout_pages_needed; /* page daemon waiting for pages? */ - int vmd_pageout_deficit; /* Estimated number of pages deficit */ - bool vmd_pageout_wanted; /* pageout daemon wait channel */ - bool vmd_minset; /* Are we in vm_min_domains? */ - bool vmd_severeset; /* Are we in vm_severe_domains? */ + int vmd_pageout_wanted; /* (a, p) pageout daemon wait channel */ + int vmd_pageout_pages_needed; /* (d) page daemon waiting for pages? */ + bool vmd_minset; /* (d) Are we in vm_min_domains? */ + bool vmd_severeset; /* (d) Are we in vm_severe_domains? */ int vmd_inactq_scans; enum { VM_LAUNDRY_IDLE = 0, @@ -142,6 +155,17 @@ extern struct vm_domain vm_dom[MAXMEMDOM]; #define vm_domain_free_unlock(d) \ mtx_unlock(vm_domain_free_lockptr((d))) +#define vm_domain_pageout_lockptr(d) \ + (&(d)->vmd_pageout_mtx) +#define vm_domain_pageout_assert_locked(n) \ + mtx_assert(vm_domain_pageout_lockptr((n)), MA_OWNED) +#define vm_domain_pageout_assert_unlocked(n) \ + mtx_assert(vm_domain_pageout_lockptr((n)), MA_NOTOWNED) +#define vm_domain_pageout_lock(d) \ + mtx_lock(vm_domain_pageout_lockptr((d))) +#define vm_domain_pageout_unlock(d) \ + mtx_unlock(vm_domain_pageout_lockptr((d))) + static __inline void vm_pagequeue_cnt_add(struct vm_pagequeue *pq, int addend) { @@ -155,6 +179,7 @@ vm_pagequeue_cnt_add(struct vm_pagequeue *pq, int #define vm_pagequeue_cnt_dec(pq) vm_pagequeue_cnt_add((pq), -1) void vm_domain_set(struct vm_domain *vmd); +void vm_domain_clear(struct vm_domain *vmd); int vm_domain_available(struct vm_domain *vmd, int req, int npages); /* @@ -221,18 +246,40 @@ vm_laundry_target(struct vm_domain *vmd) return (vm_paging_target(vmd)); } -static inline u_int -vm_domain_freecnt_adj(struct vm_domain *vmd, int adj) +void pagedaemon_wakeup(int domain); + +static inline void +vm_domain_freecnt_inc(struct vm_domain *vmd, int adj) { - u_int ret; + u_int old, new; - vm_domain_free_assert_locked(vmd); - ret = vmd->vmd_free_count += adj; - if ((!vmd->vmd_minset && vm_paging_min(vmd)) || - (!vmd->vmd_severeset && vm_paging_severe(vmd))) + old = atomic_fetchadd_int(&vmd->vmd_free_count, adj); + new = old + adj; + /* + * Only update bitsets on transitions. Notice we short-circuit the + * rest of the checks if we're above min already. + */ + if (old < vmd->vmd_free_min && (new >= vmd->vmd_free_min || + (old < vmd->vmd_free_severe && new >= vmd->vmd_free_severe) || + (old < vmd->vmd_pageout_free_min && + new >= vmd->vmd_pageout_free_min))) + vm_domain_clear(vmd); +} + +static inline void +vm_domain_freecnt_dec(struct vm_domain *vmd, int adj) +{ + u_int old, new; + + old = atomic_fetchadd_int(&vmd->vmd_free_count, -adj); + new = old - adj; + KASSERT(new >= 0, ("vm_domain_freecnt_dec: free count underflow")); + if (vm_paging_needed(vmd, new) && !vm_paging_needed(vmd, old)) + pagedaemon_wakeup(vmd->vmd_domain); + /* Only update bitsets on transitions. */ + if ((old >= vmd->vmd_free_min && new < vmd->vmd_free_min) || + (old >= vmd->vmd_free_severe && new < vmd->vmd_free_severe)) vm_domain_set(vmd); - - return (ret); } Index: /usr/src/sys/vm/vm_reserv.c =================================================================== --- /usr/src/sys/vm/vm_reserv.c (revision 330873) +++ /usr/src/sys/vm/vm_reserv.c (working copy) @@ -593,7 +593,7 @@ vm_reserv_extend_contig(int req, vm_object_t objec } for (i = 0; i < npages; i++) vm_reserv_populate(rv, index + i); - vm_domain_freecnt_adj(vmd, -npages); + vm_domain_freecnt_dec(vmd, npages); out: vm_domain_free_unlock(vmd); return (m); @@ -789,7 +789,7 @@ vm_reserv_extend(int req, vm_object_t object, vm_p struct vm_domain *vmd; vm_page_t m, msucc; vm_reserv_t rv; - int index, free_count; + int index; VM_OBJECT_ASSERT_WLOCKED(object); @@ -822,14 +822,10 @@ vm_reserv_extend(int req, vm_object_t object, vm_p m = NULL; if (m != NULL) { vm_reserv_populate(rv, index); - free_count = vm_domain_freecnt_adj(vmd, -1); - } else - free_count = vmd->vmd_free_count; + vm_domain_freecnt_dec(vmd, 1); + } vm_domain_free_unlock(vmd); - if (vm_paging_needed(vmd, free_count)) - pagedaemon_wakeup(domain); - return (m); } -- Test scenario: misc/datamove2.sh