panic: handle_disk_io_initiation: Unexpected type jnewblk, vmcore.616 KDB: debugger backends: ddb KDB: current backend: ddb ---<>--- Copyright (c) 1992-2022 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 14.0-CURRENT #0 main-n259014-dabb3db7a817f-dirty: Thu Nov 3 13:44:53 CET 2022 pho@mercat1.netperf.freebsd.org:/usr/src/sys/amd64/compile/PHO amd64 FreeBSD clang version 14.0.5 (https://github.com/llvm/llvm-project.git llvmorg-14.0.5-0-gc12386ae247c) WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. VT(vga): resolution 640x480 CPU: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz (3500.13-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306f2 Family=0x6 Model=0x3f Stepping=2 Features=0xbfebfbff Features2=0x7ffefbff AMD Features=0x2c100800 AMD Features2=0x21 Structured Extended Features=0x37ab XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics real memory = 34359738368 (32768 MB) avail memory = 33174519808 (31637 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs FreeBSD/SMP: 1 package(s) x 6 core(s) x 2 hardware threads random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" random: unblocking device. ioapic0 irqs 0-23 ioapic1 irqs 24-47 Launching APs: 1 9 11 7 2 10 6 5 8 4 3 random: entropy device external interface kbd1 at kbdmux0 vtvga0: smbios0: at iomem 0xf0560-0xf057e smbios0: Version: 2.8, BCD Revision: 2.7 aesni0: acpi0: acpi0: Power Button (fixed) cpu0: on acpi0 atrtc0: port 0x70-0x71,0x74-0x77 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Event timer "HPET4" frequency 14318180 Hz quality 340 Event timer "HPET5" frequency 14318180 Hz quality 340 Event timer "HPET6" frequency 14318180 Hz quality 340 Event timer "HPET7" frequency 14318180 Hz quality 340 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: on acpi0 pci0: on pcib0 pci0: at device 11.1 (no driver attached) pci0: at device 11.2 (no driver attached) pci0: at device 16.1 (no driver attached) pci0: at device 16.6 (no driver attached) pci0: at device 18.1 (no driver attached) acpi_syscontainer0: on acpi0 acpi_syscontainer1: on acpi0 acpi_syscontainer2: on acpi0 acpi_syscontainer3: on acpi0 apei0: on acpi0 pcib1: port 0xcf8-0xcff on acpi0 pci1: on pcib1 pcib2: irq 26 at device 1.0 on pci1 pci2: on pcib2 pcib3: irq 32 at device 2.0 on pci1 pci3: on pcib3 pci3: at device 0.0 (no driver attached) pci3: at device 0.1 (no driver attached) pcib4: irq 40 at device 3.0 on pci1 pci4: on pcib4 pci1: at device 17.0 (no driver attached) ahci0: port 0xf110-0xf117,0xf100-0xf103,0xf0f0-0xf0f7,0xf0e0-0xf0e3,0xf020-0xf03f mem 0xfb438000-0xfb4387ff irq 16 at device 17.4 on pci1 ahci0: AHCI v1.30 with 4 6Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahciem0: on ahci0 xhci0: mem 0xfb400000-0xfb40ffff irq 19 at device 20.0 on pci1 xhci0: 32 bytes context size, 64-bit DMA usbus0 on xhci0 usbus0: 5.0Gbps Super Speed USB v3.0 pci1: at device 22.0 (no driver attached) pci1: at device 22.1 (no driver attached) ehci0: mem 0xfb434000-0xfb4343ff irq 18 at device 26.0 on pci1 usbus1: EHCI version 1.0 usbus1 on ehci0 usbus1: 480Mbps High Speed USB v2.0 pcib5: irq 16 at device 28.0 on pci1 pci5: on pcib5 pcib6: irq 18 at device 28.2 on pci1 pci6: on pcib6 pcib7: at device 0.0 on pci6 pci7: on pcib7 vgapci0: port 0xe000-0xe07f mem 0xfa000000-0xfaffffff,0xfb000000-0xfb01ffff irq 18 at device 0.0 on pci7 vgapci0: Boot video device pcib8: irq 16 at device 28.4 on pci1 pci8: on pcib8 igb0: port 0xd020-0xd03f mem 0xfb320000-0xfb33ffff,0xfb344000-0xfb347fff irq 16 at device 0.0 on pci8 igb0: EEPROM V1.63-0 eTrack 0x800009fa igb0: Using 1024 TX descriptors and 1024 RX descriptors igb0: queue equality override not set, capping rx_queues at 6 and tx_queues at 6 igb0: Using 6 fsck2+ fsck -fy /tmp ** /dev/gpt/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ugen0.3: at usbus0 ukbd0 on uhub3 ukbd0: on usbus0 kbd2 at ukbd0 ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts UNREF FILE I=160518 OWNER=root MODE=100640 SIZE=0 MTIME=Nov 3 16:54 2022 RECONNECT? yes NO lost+found DIRECTORY CREATE? yes UNREF FILE I=160519 OWNER=root MODE=100644 SIZE=0 MTIME=Nov 3 16:54 2022 RECONNECT? yes ** Phase 5 - Check Cyl groups FREE BLK COUNT(S) WRONG IN SUPERBLK SALVAGE? yes SUMMARY INFORMATION BAD SALVAGE? yes 382 files, 13813 used, 1251594 free (138 frags, 156432 blocks, 0.0% fragmentation) ***** FILE SYSTEM MARKED CLEAN ***** ***** FILE SYSTEM WAS MODIFIED ***** + set +x Updating CPU Microcode... CPU: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz (3500.00-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306f2 Family=0x6 Model=0x3f Stepping=2 Features=0xbfebfbff Features2=0x7ffefbff AMD Features=0x2c100800 AMD Features2=0x21 Structured Extended Features=0x37ab Structured Extended Features3=0x9c000400 XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics Done. ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/lib/compat /usr/local/lib/compat/pkg /usr/local/lib/e2fsprogs /usr/local/lib/gcc10 /usr/local/lib/gcc8 /usr/local/lib/gcc9 /usr/local/lib/perl5/5.32/mach/CORE 32-bit compatibility ldconfig path: /usr/lib32 /usr/local/lib32/compat Setting hostname: mercat1.netperf.freebsd.org. Setting up harvesting: PURE_RDRAND,[CALLOUT],[UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,[NET_ETHER],NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED Feeding entropy: . Autoloading module: acpi_wmi.ko Autoloading module: ichsmb.ko Autoloading module: ioat.ko ioat0: mem 0xfb42c000-0xfb42ffff irq 31 at device 4.0 on pci1 ioat0: Capabilities: 2f7 ioat1: mem 0xfb428000-0xfb42bfff irq 39 at device 4.1 on pci1 ioat1: Capabilities: 2f7 ioat2: mem 0xfb424000-0xfb427fff irq 31 at device 4.2 on pci1 ioat2: Capabilities: f7 ioat3: mem 0xfb420000-0xfb423fff irq 39 at device 4.3 on pci1 ioat3: Capabilities: f7 ioat4: mem 0xfb41c000-0xfb41ffff irq 31 at device 4.4 on pci1 ioat4: Capabilities: f7 ioat5: mem 0xfb418000-0xfb41bfff irq 39 at device 4.5 on pci1 ioat5: Capabilities: f7 ioat6: mem 0xfb414000-0xfb417fff irq 31 at device 4.6 on pci1 ioat6: Capabilities: f7 ioat7: mem 0xfb410000-0xfb413fff irq 39 at device 4.7 on pci1 ioat7: Capabilities: f7 ichsmb0: port 0x580-0x59f mem 0xfb431000-0xfb4310ff irq 18 at device 31.3 on pci1 smbus0: on ichsmb0 acpi_wmi0: on acpi0 acpi_wmi0: cannot find EC device lo0: link state changed to UP Starting Network: lo0 igb0 igb1 mce0 mce1. lo0: flags=8049 metric 0 mtu 16384 options=680003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=21 igb0: flags=8863 metric 0 mtu 1500 options=4e507bb ether 0c:c4:7a:a8:cd:da inet6 fe80::ec4:7aff:fea8:cdda%igb0 prefixlen 64 scopeid 0x1 media: Ethernet autoselect status: no carrier nd6 options=21 igb1: flags=8822 metric 0 mtu 1500 options=4e507bb ether 0c:c4:7a:a8:cd:db media: Ethernet autoselect status: no carrier nd6 options=29 mce0: flags=8822 metric 0 mtu 1500 options RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,HWRXTSTMP,MEXTPG,TXTLS4,TXTLS6,VXLAN_HWCSUM,VXLAN_HWTSO,RXTLS4,RXTLS6 ether 7c:fe:90:30:ee:38 media: Ethernet autoselect status: no carrier nd6 options=29 mce1: flags=8822 metric 0 mtu 1500 options RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,HWRXTSTMP,MEXTPG,TXTLS4,TXTLS6,VXLAN_HWCSUM,VXLAN_HWTSO,RXTLS4,RXTLS6 ether 7c:fe:90:30:ee:39 media: Ethernet autoselect status: no carrier nd6 options=29 Starting devd. igb0: link state changed to UP Autoloading module: acpi_wmi.ko Autoloading module: mlx5.ko interface mlx5.1 already present in the KLD 'kernel'! linker_load_file: /boot/kernel/mlx5.ko - unsupported file type kldload: an error occurred while loading module mlx5.ko. Please check dmesg(8) for more details. Autoloading moduinterface mlx5.1 already present in the KLD 'kernel'! linker_load_file: /boot/kernel/mlx5.ko - unsupported file type le: mlx5.ko kldload: an error occurred while loading module mlx5.ko. Please check dmesg(8) for more details. Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Starting Network: igb1. igb1: flags=8822 metric 0 mtu 1500 options=4e507bb ether 0c:c4:7a:a8:cd:db media: Ethernet autoselect status: no carrier nd6 options=29 Autoloading module: ichsmb.ko Starting Network: mce0. mce0: flags=8822 metric 0 mtu 1500 options RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,HWRXTSTMP,MEXTPG,TXTLS4,TXTLS6,VXLAN_HWCSUM,VXLAN_HWTSO,RXTLS4,RXTLS6 ether 7c:fe:90:30:ee:38 media: Ethernet autoselect status: no carrier nd6 options=29 Starting Network: mce1. mce1: flags=8822 metric 0 mtu 1500 options RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,HWRXTSTMP,MEXTPG,TXTLS4,TXTLS6,VXLAN_HWCSUM,VXLAN_HWTSO,RXTLS4,RXTLS6 ether 7c:fe:90:30:ee:39 media: Ethernet autoselect status: no carrier nd6 options=29 Autoloading module: uhid.ko Autoloading module: ums.ko Autoloading module: usbhid.ko ums0 on uhub3 ums0: on usbus0 ums0: 3 buttons and [Z] coordinates ID=0 Starting dhclient. if_delmulti_locked: detaching ifnet instance 0xfffff800041d5000 DHCPDISCOVER on igb0 to 255.255.255.255 port 67 interval 6 DHCPOFFER from 192.168.5.1 DHCPREQUEST on igb0 to 255.255.255.255 port 67 DHCPACK from 192if_delmulti_locked: detaching ifnet instance 0xfffff800041d5000 .168.5.1 bound to 192.168.5.171 -- renewal in 300 seconds. add host 127.0.0.1: gateway lo0 fib 0: route already in table add host ::1: gateway lo0 fib 0: route already in table add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Creating and/or trimming log files. Recovering vi editor sessions:. Updating motd:. Clearing /tmp (X related). Starting syslogd. Starting watchdogd. Nov 3 17:30:57 mercat1 savecore[2619]: reboot after panic: flush_pagedep_deps: failed to flush inodedep 0xfffff8043e11dd80 ino 332257 dap 0xfffff8045dba0a80 Setting date via ntp. 3 Nov 17:31:43 ntpdate[2622]: step time server 192.168.5.1 offset -1.747064 sec Starting powerd. Security policy loaded: MAC/ntpd (mac_ntpd) Starting ntpd. Mounting late filesystems:. Starting inetd. Configuring vt: blanktime. Performing sanity check on sshd configuration. Starting sshd. Starting cron. sysctl: unknown oid 'debug.fail_point.pmap_allocpte_page_alloc' at line 12 Thu Nov 3 17:31 FreeBSD/amd64 (mercat1.netperf.freebsd.org) (ttyu1) login: Nov 3 17:34:51 mercat1 su[2779]: pho to root on /dev/pts/0 20221103 17:41:14 all (1/10): suj16.sh lock order reversal: 1st 0xfffffe0037f5ab60 bufwait (bufwait, lockmgr) @ ufs/ffs/ffs_vnops.c:323 2nd 0xfffff8000420ae30 snaplk (snaplk, lockmgr) @ ufs/ffs/ffs_snapshot.c:2314 lock order snaplk -> bufwait established at: #0 0xffffffff80c627cd at witness_checkorder+0x32d #1 0xffffffff80bbdf1c at lockmgr_xlock_hard+0x6c #2 0xffffffff80bbe829 at __lockmgr_args+0x1f9 #3 0xffffffff80ce3980 at bnoreuselist+0xc0 #4 0xffffffff80ccffbe at vop_stdadvise+0x14e #5 0xffffffff811af8e9 at VOP_ADVISE_APV+0x59 #6 0xffffffff806efde2 at mdstart_vnode+0x5c2 #7 0xffffffff806ed72c at md_kthread+0x20c #8 0xffffffff80ba6bb0 at fork_exit+0x80 #9 0xffffffff810ad8be at fork_trampoline+0xe lock order bufwait -> snaplk attempted at: #0 0xffffffff80c6309d at witness_checkorder+0xbfd #1 0xffffffff80bbdf1c at lockmgr_xlock_hard+0x6c #2 0xffffffff80bbe829 at __lockmgr_args+0x1f9 #3 0xffffffff80f1e8e3 at ffs_copyonwrite+0x1a3 #4 0xffffffff80f45adf at ffs_geom_strategy+0xcf #5 0xffffffff80f5af73 at ufs_strategy+0x83 #6 0xffffffff811ad371 at VOP_STRATEGY_APV+0x61 #7 0xffffffff80cb675e at bufstrategy+0x3e #8 0xffffffff80cb65fc at bufwrite+0x24c #9 0xffffffff80f48182 at ffs_syncvnode+0x3a2 #10 0xffffffff80f46b5e at ffs_fsync+0x1e #11 0xffffffff811abaec at VOP_FSYNC_APV+0x3c #12 0xffffffff80f58b84 at ufs_remove+0x114 #13 0xffffffff811abc95 at VOP_REMOVE_APV+0x85 #14 0xffffffff80cf5a6b at kern_funlinkat+0x39b #15 0xffffffff80cf56c8 at sys_unlink+0x28 #16 0xffffffff810da455 at amd64_syscall+0x145 #17 0xffffffff810ad16b at fast_syscall_common+0xf8 20221103 18:11:17 all (2/10): suj19.sh Nov 3 18:11:20 mercat1 kernel: pid 83260 (rw), uid 2007 inumber 65804 on /mnt: filesystem full Nov 3 18:15:56 mercat1 kernel: pid 7586 (mkdir), uid 2007 inumber 66368 on /mnt: filesystem full Nov 3 18:16:19 mercat1 kernel: pid 7573 (rw), uid 2007 inumber 65872 on /mnt: filesystem full Nov 3 18:19:16 mercat1 kernel: pid 19928 (mkdir), uid 2007 inumber 106250 on /mnt: filesystem full Nov 3 18:19:48 mercat1 kernel: pid 31690 (mkdir), uid 2007 inumber 65974 on /mnt: filesystem full Nov 3 18:19:50 mercat1 kernel: pid 31677 (rw), uid 2007 inumber 2919 on /mnt: filesystem full Nov 3 18:22:35 mercat1 kernel: pid 44340 (mkdir), uid 2007 inumber 1931 on /mnt: filesystem full Nov 3 18:24:19 mercat1 kernel: pid 55825 (rw), uid 2007 inumber 74540 on /mnt: filesystem full Nov 3 18:24:27 mercat1 kernel: pid 55821 (rw), uid 2007 inumber 75355 on /mnt: filesystem full Nov 3 18:25:03 mercat1 kernel: pid 55811 (rw), uid 2007 inumber 70139 on /mnt: filesystem full Nov 3 18:25:10 mercat1 kernel: pid 55827 (rw), uid 2007 inumber 74560 on /mnt: filesystem full Nov 3 18:25:24 mercat1 kernel: pid 55829 (rw), uid 2007 inumber 80843 on /mnt: filesystem full Nov 3 18:26:01 mercat1 kernel: pid 68415 (mkdir), uid 2007 inumber 32935 on /mnt: filesystem full Nov 3 18:27:26 mercat1 kernel: pid 81042 (rw), uid 2007 inumber 65855 on /mnt: filesystem full Nov 3 18:29:42 mercat1 kernel: pid 93670 (rw), uid 2007 inumber 65829 on /mnt: filesystem full Nov 3 18:29:58 mercat1 kernel: pid 93673 (rw), uid 2007 inumber 69090 on /mnt: filesystem full Nov 3 18:30:02 mercat1 kernel: pid 93660 (rw), uid 2007 inumber 69746 on /mnt: filesystem full Nov 3 18:30:08 mercat1 kernel: pid 93675 (rw), uid 2007 inumber 70326 on /mnt: filesystem full Nov 3 18:30:16 mercat1 kernel: pid 93666 (rw), uid 2007 inumber 70942 on /mnt: filesystem full lock order reversal: 1st 0xfffff8000472aea0 FFS (FFS Lock, sleep mutex) @ ufs/ufs/ufs_quota.c:755 2nd 0xfffffe0142aeb040 struct mount mtx (struct mount mtx, sleep mutex) @ ufs/ufs/ufs_quota.c:761 lock order struct mount mtx -> FFS Lock established at: #0 0xffffffff80c627cd at witness_checkorder+0x32d #1 0xffffffff80bca9f4 at __mtx_lock_flags+0x94 #2 0xffffffff80f1fd24 at softdep_flushfiles+0x394 #3 0xffffffff80f439fb at ffs_unmount+0xab #4 0xffffffff80cdad1a at dounmount+0x7ea #5 0xffffffff80cda4c7 at kern_unmount+0x2e7 #6 0xffffffff810da455 at amd64_syscall+0x145 #7 0xffffffff810ad16b at fast_syscall_common+0xf8 lock order FFS Lock -> struct mount mtx attempted at: #0 0xffffffff80c6309d at witness_checkorder+0xbfd #1 0xffffffff80bca9f4 at __mtx_lock_flags+0x94 #2 0xffffffff80f559ed at quotaoff_inchange+0xbd #3 0xffffffff80f4201f at ffs_flushfiles+0x15f #4 0xffffffff80f1fa3d at softdep_flushfiles+0xad #5 0xffffffff80f439fb at ffs_unmount+0xab #6 0xffffffff80cdad1a at dounmount+0x7ea #7 0xffffffff80cda4c7 at kern_unmount+0x2e7 #8 0xffffffff810da455 at amd64_syscall+0x145 #9 0xffffffff810ad16b at fast_syscall_common+0xf8 20221103 18:31:26 all (3/10): suj20.sh 20221103 18:31:50 all (4/10): suj21.sh 20221103 18:32:11 all (5/10): suj22.sh 20221103 18:35:42 all (6/10): suj24.sh 20221103 18:35:44 all (7/10): suj25.sh 20221103 18:35:50 all (8/10): suj26.sh Nov 3 18:45:00 mercat1 kernel: pid 70826 (swap), jid 0, uid 2007, was killed: a thread waited too long to allocate a page Nov 3 18:55:33 mercat1 kernel: pid 7521 (swap), jid 0, uid 2007, was killed: failed to reclaim memory Nov 3 18:57:33 mercat1 kernel: pid 14233 (swap), jid 0, uid 2007, was killed: a thread waited too long to allocate a page 20221103 19:07:22 all (9/10): suj27.sh Nov 3 19:10:35 mercat1 kernel: pid 84881 (rw), uid 2007 inumber 11778 on /mnt: filesystem full panic: handle_disk_io_initiation: Unexpected type jnewblk cpuid = 3 time = 1667499258 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe01419d24a0 vpanic() at vpanic+0x151/frame 0xfffffe01419d24f0 panic() at panic+0x43/frame 0xfffffe01419d2550 softdep_disk_io_initiation() at softdep_disk_io_initiation+0x14be/frame 0xfffffe01419d2610 ffs_geom_strategy() at ffs_geom_strategy+0x18c/frame 0xfffffe01419d2640 ufs_strategy() at ufs_strategy+0x83/frame 0xfffffe01419d2670 VOP_STRATEGY_APV() at VOP_STRATEGY_APV+0x61/frame 0xfffffe01419d2690 bufstrategy() at bufstrategy+0x3e/frame 0xfffffe01419d26c0 bufwrite() at bufwrite+0x24c/frame 0xfffffe01419d2710 vfs_bio_awrite() at vfs_bio_awrite+0x21e/frame 0xfffffe01419d2780 ffs_syncvnode() at ffs_syncvnode+0x36a/frame 0xfffffe01419d2810 ffs_copyonwrite() at ffs_copyonwrite+0x5ad/frame 0xfffffe01419d28b0 ffs_geom_strategy() at ffs_geom_strategy+0xcf/frame 0xfffffe01419d28e0 bufwrite() at bufwrite+0x24c/frame 0xfffffe01419d2930 ffs_update() at ffs_update+0x336/frame 0xfffffe01419d2990 ufs_makeinode() at ufs_makeinode+0x3e6/frame 0xfffffe01419d2b00 ufs_create() at ufs_create+0x34/frame 0xfffffe01419d2b20 VOP_CREATE_APV() at VOP_CREATE_APV+0x62/frame 0xfffffe01419d2b40 vn_open_cred() at vn_open_cred+0x2f6/frame 0xfffffe01419d2cb0 kern_openat() at kern_openat+0x29c/frame 0xfffffe01419d2e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe01419d2f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01419d2f30 --- syscall (499, FreeBSD ELF64, openat), rip = 0x598515ef5ba, rsp = 0x5984eec7f68, rbp = 0x5984eec8020 --- KDB: enter: panic [ thread pid 22394 tid 100556 ] Stopped at kdb_enter+0x32: movq $0,0x129a133(%rip) db> run pho db:0:pho> set $lines 20000 db:0:pho> run pho1 db:1:pho1> dump Dumping 1949 out of 32624 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Dump complete db:1:pho1> bt Tracing pid 22394 tid 100556 td 0xfffffe01441231e0 kdb_enter() at kdb_enter+0x32/frame 0xfffffe01419d24a0 vpanic() at vpanic+0x182/frame 0xfffffe01419d24f0 panic() at panic+0x43/frame 0xfffffe01419d2550 softdep_disk_io_initiation() at softdep_disk_io_initiation+0x14be/frame 0xfffffe01419d2610 ffs_geom_strategy() at ffs_geom_strategy+0x18c/frame 0xfffffe01419d2640 ufs_strategy() at ufs_strategy+0x83/frame 0xfffffe01419d2670 VOP_STRATEGY_APV() at VOP_STRATEGY_APV+0x61/frame 0xfffffe01419d2690 bufstrategy() at bufstrategy+0x3e/frame 0xfffffe01419d26c0 bufwrite() at bufwrite+0x24c/frame 0xfffffe01419d2710 vfs_bio_awrite() at vfs_bio_awrite+0x21e/frame 0xfffffe01419d2780 ffs_syncvnode() at ffs_syncvnode+0x36a/frame 0xfffffe01419d2810 ffs_copyonwrite() at ffs_copyonwrite+0x5ad/frame 0xfffffe01419d28b0 ffs_geom_strategy() at ffs_geom_strategy+0xcf/frame 0xfffffe01419d28e0 bufwrite() at bufwrite+0x24c/frame 0xfffffe01419d2930 ffs_update() at ffs_update+0x336/frame 0xfffffe01419d2990 ufs_makeinode() at ufs_makeinode+0x3e6/frame 0xfffffe01419d2b00 ufs_create() at ufs_create+0x34/frame 0xfffffe01419d2b20 VOP_CREATE_APV() at VOP_CREATE_APV+0x62/frame 0xfffffe01419d2b40 vn_open_cred() at vn_open_cred+0x2f6/frame 0xfffffe01419d2cb0 kern_openat() at kern_openat+0x29c/frame 0xfffffe01419d2e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe01419d2f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01419d2f30 --- syscall (499, FreeBSD ELF64, openat), rip = 0x598515ef5ba, rsp = 0x5984eec7f68, rbp = 0x5984eec8020 --- db:1:pho1> show allpcpu Current CPU: 3 cpuid = 0 dynamic pcpu = 0x9c0840 curthread = 0xfffffe014f1b0900: pid 22401 tid 100591 critnest 0 "rw" curpcb = 0xfffffe014f1b0e20 fpcurthread = 0xfffffe014f1b0900: pid 22401 "rw" idlethread = 0xfffffe00e4c553a0: tid 100003 "idle: cpu0" self = 0xffffffff82610000 curpmap = 0xfffffe0155f1eb30 tssp = 0xffffffff82610384 rsp0 = 0xfffffe01419ec000 kcr3 = 0x8000000203732738 ucr3 = 0x8000000078c53f38 scr3 = 0x652407ec7 gs32p = 0xffffffff82610404 ldt = 0xffffffff82610444 tss = 0xffffffff82610434 curvnet = 0 spin locks held: cpuid = 1 dynamic pcpu = 0xfffffe00bb278840 curthread = 0xfffffe00e4c54c80: pid 11 tid 100004 critnest 1 "idle: cpu1" curpcb = 0xfffffe00e4c551a0 fpcurthread = none idlethread = 0xfffffe00e4c54c80: tid 100004 "idle: cpu1" self = 0xffffffff82611000 curpmap = 0xffffffff81e8d230 tssp = 0xffffffff82611384 rsp0 = 0xfffffe00377ec000 kcr3 = 0x8000000002571002 ucr3 = 0xffffffffffffffff scr3 = 0x6cc144930 gs32p = 0xffffffff82611404 ldt = 0xffffffff82611444 tss = 0xffffffff82611434 curvnet = 0 spin locks held: cpuid = 2 dynamic pcpu = 0xfffffe00bb285840 curthread = 0xfffffe00e4c54560: pid 11 tid 100005 critnest 1 "idle: cpu2" curpcb = 0xfffffe00e4c54a80 fpcurthread = none idlethread = 0xfffffe00e4c54560: tid 100005 "idle: cpu2" self = 0xffffffff82612000 curpmap = 0xffffffff81e8d230 tssp = 0xffffffff82612384 rsp0 = 0xfffffe00377fb000 kcr3 = 0x8000000002571002 ucr3 = 0xffffffffffffffff scr3 = 0x6cc149b6b gs32p = 0xffffffff82612404 ldt = 0xffffffff82612444 tss = 0xffffffff82612434 curvnet = 0 spin locks held: cpuid = 3 dynamic pcpu = 0xfffffe00bb292840 curthread = 0xfffffe01441231e0: pid 22394 tid 100556 critnest 1 "creat" curpcb = 0xfffffe0144123700 fpcurthread = 0xfffffe01441231e0: pid 22394 "creat" idlethread = 0xfffffe00e4c53e40: tid 100006 "idle: cpu3" self = 0xffffffff82613000 curpmap = 0xfffffe015790c530 tssp = 0xffffffff82613384 rsp0 = 0xfffffe01419d3000 kcr3 = 0x800000020c84f1e0 ucr3 = 0x8000000547f919e0 scr3 = 0x12cffe96c gs32p = 0xffffffff82613404 ldt = 0xffffffff82613444 tss = 0xffffffff82613434 curvnet = 0 spin locks held: cpuid = 4 dynamic pcpu = 0xfffffe00bb29f840 curthread = 0xfffffe0142ae2e40: pid 22397 tid 100272 critnest 0 "rw" curpcb = 0xfffffe0142ae3360 fpcurthread = 0xfffffe0142ae2e40: pid 22397 "rw" idlethread = 0xfffffe00e4c53720: tid 100007 "idle: cpu4" self = 0xffffffff82614000 curpmap = 0xfffffe014405ab30 tssp = 0xffffffff82614384 rsp0 = 0xfffffe0141884000 kcr3 = 0x80000002917dd28f ucr3 = 0x800000046f9e7a8f scr3 = 0x5e52b79c7 gs32p = 0xffffffff82614404 ldt = 0xffffffff82614444 tss = 0xffffffff82614434 curvnet = 0 spin locks held: cpuid = 5 dynamic pcpu = 0xfffffe00bb2ac840 curthread = 0xfffffe014f1ae560: pid 22391 tid 100587 critnest 0 "rw" curpcb = 0xfffffe014f1aea80 fpcurthread = 0xfffffe014f1ae560: pid 22391 "rw" idlethread = 0xfffffe00e4c53000: tid 100008 "idle: cpu5" self = 0xffffffff82615000 curpmap = 0xfffffe0144623530 tssp = 0xffffffff82615384 rsp0 = 0xfffffe0156e41000 kcr3 = 0x80000002bbcd92d2 ucr3 = 0x8000000497aaaad2 scr3 = 0x19a636a79 gs32p = 0xffffffff82615404 ldt = 0xffffffff82615444 tss = 0xffffffff82615434 curvnet = 0 spin locks held: cpuid = 6 dynamic pcpu = 0xfffffe00bb2b9840 curthread = 0xfffffe014f1a8e40: pid 22396 tid 100584 critnest 0 "swap" curpcb = 0xfffffe014f1a9360 fpcurthread = 0xfffffe014f1a8e40: pid 22396 "swap" idlethread = 0xfffffe00e4c5bc80: tid 100009 "idle: cpu6" self = 0xffffffff82616000 curpmap = 0xfffffe003cffeb30 tssp = 0xffffffff82616384 rsp0 = 0xfffffe0141870000 kcr3 = 0x80000001bc43e485 ucr3 = 0x8000000139809c85 scr3 = 0x6cc159aee gs32p = 0xffffffff82616404 ldt = 0xffffffff82616444 tss = 0xffffffff82616434 curvnet = 0 spin locks held: cpuid = 7 dynamic pcpu = 0xfffffe00bb2c6840 curthread = 0xfffffe014f1b1740: pid 69482 tid 100652 critnest 0 "md10" curpcb = 0xfffffe014f1b1c60 fpcurthread = none idlethread = 0xfffffe00e4c5b560: tid 100010 "idle: cpu7" self = 0xffffffff82617000 curpmap = 0xffffffff81e8d230 tssp = 0xffffffff82617384 rsp0 = 0xfffffe0156f87000 kcr3 = 0x8000000002571005 ucr3 = 0xffffffffffffffff scr3 = 0x23916ff8 gs32p = 0xffffffff82617404 ldt = 0xffffffff82617444 tss = 0xffffffff82617434 curvnet = 0 spin locks held: cpuid = 8 dynamic pcpu = 0xfffffe00bb2d3840 curthread = 0xfffffe00e4c5ae40: pid 11 tid 100011 critnest 1 "idle: cpu8" curpcb = 0xfffffe00e4c5b360 fpcurthread = none idlethread = 0xfffffe00e4c5ae40: tid 100011 "idle: cpu8" self = 0xffffffff82618000 curpmap = 0xffffffff81e8d230 tssp = 0xffffffff82618384 rsp0 = 0xfffffe00e4845000 kcr3 = 0x8000000002571002 ucr3 = 0xffffffffffffffff scr3 = 0x12cffebf9 gs32p = 0xffffffff82618404 ldt = 0xffffffff82618444 tss = 0xffffffff82618434 curvnet = 0 spin locks held: cpuid = 9 dynamic pcpu = 0xfffffe00bb2e0840 curthread = 0xfffffe0158d1d1e0: pid 22402 tid 100669 critnest 0 "swap" curpcb = 0xfffffe0158d1d700 fpcurthread = 0xfffffe0158d1d1e0: pid 22402 "swap" idlethread = 0xfffffe00e4c5a720: tid 100012 "idle: cpu9" self = 0xffffffff82619000 curpmap = 0xfffffe0157929130 tssp = 0xffffffff82619384 rsp0 = 0xfffffe0156fe1000 kcr3 = 0x800000024ed456d1 ucr3 = 0x800000030a83ced1 scr3 = 0x5e52b7a82 gs32p = 0xffffffff82619404 ldt = 0xffffffff82619444 tss = 0xffffffff82619434 curvnet = 0 spin locks held: cpuid = 10 dynamic pcpu = 0xfffffe00bb2ed840 curthread = 0xfffffe014462b900: pid 22399 tid 100589 critnest 0 "swap" curpcb = 0xfffffe014462be20 fpcurthread = 0xfffffe014462b900: pid 22399 "swap" idlethread = 0xfffffe00e4c5a000: tid 100013 "idle: cpu10" self = 0xffffffff8261a000 curpmap = 0xfffffe013bb9e530 tssp = 0xffffffff8261a384 rsp0 = 0xfffffe0141929000 kcr3 = 0x8000000600198297 ucr3 = 0x80000001efcfba97 scr3 = 0x5e52b7902 gs32p = 0xffffffff8261a404 ldt = 0xffffffff8261a444 tss = 0xffffffff8261a434 curvnet = 0 spin locks held: cpuid = 11 dynamic pcpu = 0xfffffe00bb2fa840 curthread = 0xfffffe010e3a6ac0: pid 9 tid 100195 critnest 0 "dom0" curpcb = 0xfffffe010e3a6fe0 fpcurthread = none idlethread = 0xfffffe00e4c57740: tid 100014 "idle: cpu11" self = 0xffffffff8261b000 curpmap = 0xffffffff81e8d230 tssp = 0xffffffff8261b384 rsp0 = 0xfffffe010db44000 kcr3 = 0x8000000002571002 ucr3 = 0xffffffffffffffff scr3 = 0x3d742ca99 gs32p = 0xffffffff8261b404 ldt = 0xffffffff8261b444 tss = 0xffffffff8261b434 curvnet = 0 spin locks held: db:1:pho1> show alllocks Process 22403 (rw) thread 0xfffffe015b1f1720 (100691) exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe0039da6840) locked @ kern/vfs_bio.c:3988 exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe003864ad50) locked @ kern/vfs_bio.c:3988 exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe0039c716b0) locked @ kern/vfs_bio.c:3988 exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe003c78eab0) locked @ kern/vfs_bio.c:3988 exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe003c6c7810) locked @ kern/vfs_bio.c:3988 exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe003c18ca20) locked @ kern/vfs_bio.c:3988 exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe003be2ef80) locked @ kern/vfs_bio.c:3988 exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe0037ba99d0) locked @ kern/vfs_bio.c:3988 exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe003c160f30) locked @ kern/vfs_bio.c:3988 exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe003c4d70b0) locked @ kern/vfs_bio.c:3988 exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe003c1d3700) locked @ kern/vfs_bio.c:3988 exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe003c03b610) locked @ kern/vfs_bio.c:3988 exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe0038d023a0) locked @ kern/vfs_bio.c:3988 exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe0037c256b0) locked @ kern/vfs_bio.c:3988 exclusive lockmgr ufs (ufs) r = 0 (0xfffffe0155fe09d0) locked @ kern/vfs_vnops.c:1166 Process 22400 (rw) thread 0xfffffe0144631740 (100592) exclusive lockmgr ufs (ufs) r = 0 (0xfffffe01585eea38) locked @ kern/vfs_vnops.c:1166 Process 22398 (rw) thread 0xfffffe0158d181e0 (100653) exclusive lockmgr ufs (ufs) r = 0 (0xfffffe015a2aee80) locked @ kern/vfs_vnops.c:1166 Process 22397 (rw) thread 0xfffffe0142ae2e40 (100272) exclusive sleep mutex vnode interlock (vnode interlock) r = 0 (0xfffffe015ae64838) locked @ kern/kern_rangelock.c:239 Process 22394 (creat) thread 0xfffffe01441231e0 (100556) exclusive rw SUrw (SUrw) r = 0 (0xfffff80171a07400) locked @ ufs/ffs/ffs_softdep.c:10234 exclusive lockmgr snaplk (snaplk) r = 0 (0xfffff8000420ae30) locked @ ufs/ffs/ffs_snapshot.c:2314 exclusive lockmgr ufs (ufs) r = 0 (0xfffffe0158cd27e0) locked @ ufs/ffs/ffs_vfsops.c:1888 exclusive lockmgr ufs (ufs) r = 0 (0xfffffe015ad23a38) locked @ kern/vfs_lookup.c:974 Process 22393 (rw) thread 0xfffffe014f1ade40 (100399) exclusive lockmgr ufs (ufs) r = 0 (0xfffffe015acf7588) locked @ kern/vfs_vnops.c:1166 Process 22390 (creat) thread 0xfffffe014462fac0 (100334) exclusive lockmgr ufs (ufs) r = 0 (0xfffffe015aedf330) locked @ ufs/ffs/ffs_vfsops.c:1888 exclusive lockmgr ufs (ufs) r = 0 (0xfffffe0156759520) locked @ kern/vfs_lookup.c:974 Process 22388 (rw) thread 0xfffffe0158d1ae40 (100659) exclusive lockmgr ufs (ufs) r = 0 (0xfffffe01582af520) locked @ kern/vfs_vnops.c:1166 Process 22386 (rw) thread 0xfffffe014f1ab900 (100607) exclusive lockmgr ufs (ufs) r = 0 (0xfffffe0158a2de80) locked @ kern/vfs_vnops.c:1166 Process 22385 (mkdir) thread 0xfffffe00e4d7ec80 (100217) exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe0037ad5330) locked @ kern/vfs_bio.c:1725 exclusive lockmgr ufs (ufs) r = 0 (0xfffffe01598de9d0) locked @ ufs/ffs/ffs_vfsops.c:1888 exclusive lockmgr ufs (ufs) r = 0 (0xfffffe0158cdc330) locked @ kern/vfs_lookup.c:974 Process 22383 (mkdir) thread 0xfffffe0143837e40 (100240) exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe00388c27c0) locked @ kern/vfs_bio.c:3988 exclusive lockmgr ufs (ufs) r = 0 (0xfffffe01594fac28) locked @ kern/vfs_subr.c:3032 exclusive lockmgr ufs (ufs) r = 1 (0xfffffe015650e0d8) locked @ kern/vfs_subr.c:3032 Process 9 (pagedaemon) thread 0xfffffe010e3a6ac0 (100195) exclusive sleep mutex vm active pagequeue (vm pagequeue) r = 0 (0xffffffff81fb6440) locked @ vm/vm_pageout.c:283 db:1:pho1> show lockedvnods Locked vnodes vnode 0xfffffe01567594b0: type VDIR usecount 2, writecount 0, refcount 2 seqc users 1 mountedhere 0 hold count flags () flags (VMP_LAZYLIST) v_object 0xfffff803ce10be70 ref 0 pages 0 cleanbuf 0 dirtybuf 1 lock type ufs: EXCL by thread 0xfffffe014462fac0 (pid 22390, creat, tid 100334) #0 0xffffffff80bbd85f at lockmgr_lock_flags+0x1cf #1 0xffffffff80f46cbd at ffs_lock+0x9d #2 0xffffffff811acefc at VOP_LOCK1_APV+0x3c #3 0xffffffff80cfe1e4 at _vn_lock+0x54 #4 0xffffffff80cd55a0 at vfs_lookup+0x140 #5 0xffffffff80cd4bcd at namei+0x33d #6 0xffffffff80cfd4fa at vn_open_cred+0x13a #7 0xffffffff80cf3c1c at kern_openat+0x29c #8 0xffffffff810da455 at amd64_syscall+0x145 #9 0xffffffff810ad16b at fast_syscall_common+0xf8 nlink=2, effnlink=2, size=14336, extsize 0 generation=78adedcf, uid=2007, gid=0, flags=0x0 ino 196993, on dev md10 vnode 0xfffffe015ad239c8: type VDIR usecount 2, writecount 0, refcount 2 seqc users 1 mountedhere 0 hold count flags () flags (VMP_LAZYLIST) v_object 0xfffff80828d88b58 ref 0 pages 0 cleanbuf 0 dirtybuf 1 lock type ufs: EXCL by thread 0xfffffe01441231e0 (pid 22394, creat, tid 100556) #0 0xffffffff80bbd85f at lockmgr_lock_flags+0x1cf #1 0xffffffff80f46cbd at ffs_lock+0x9d #2 0xffffffff811acefc at VOP_LOCK1_APV+0x3c #3 0xffffffff80cfe1e4 at _vn_lock+0x54 #4 0xffffffff80cd55a0 at vfs_lookup+0x140 #5 0xffffffff80cd4bcd at namei+0x33d #6 0xffffffff80cfd4fa at vn_open_cred+0x13a #7 0xffffffff80cf3c1c at kern_openat+0x29c #8 0xffffffff810da455 at amd64_syscall+0x145 #9 0xffffffff810ad16b at fast_syscall_common+0xf8 nlink=2, effnlink=2, size=16896, extsize 0 generation=14e3b449, uid=2007, gid=0, flags=0x0 ino 196994, on dev md10 vnode 0xfffffe01585ee9c8: type VREG usecount 1, writecount 1, refcount 3100 seqc users 0 hold count flags () flags (VIRF_PGREAD|VMP_LAZYLIST) v_object 0xfffff803df144420 ref 0 pages 26440 cleanbuf 3070 dirtybuf 28 lock type ufs: EXCL by thread 0xfffffe0144631740 (pid 22400, rw, tid 100592) #0 0xffffffff80bbd85f at lockmgr_lock_flags+0x1cf #1 0xffffffff80f46cbd at ffs_lock+0x9d #2 0xffffffff811acefc at VOP_LOCK1_APV+0x3c #3 0xffffffff80cfe1e4 at _vn_lock+0x54 #4 0xffffffff80d0333d at vn_write+0x1ed #5 0xffffffff80d02d53 at vn_io_fault_doio+0x43 #6 0xffffffff80cfeba3 at vn_io_fault1+0x163 #7 0xffffffff80cfc40c at vn_io_fault+0x1cc #8 0xffffffff80c687b1 at dofilewrite+0x81 #9 0xffffffff80c682e0 at sys_write+0xc0 #10 0xffffffff810da455 at amd64_syscall+0x145 #11 0xffffffff810ad16b at fast_syscall_common+0xf8 nlink=1, effnlink=1, size=108249088, extsize 0 generation=11470cbb, uid=2007, gid=0, flags=0x0 ino 200270, on dev md10 vnode 0xfffffe015a2aee10: type VREG usecount 1, writecount 1, refcount 1724 seqc users 0 hold count flags () flags (VIRF_PGREAD|VMP_LAZYLIST) v_object 0xfffff80477b0a108 ref 0 pages 16448 cleanbuf 1711 dirtybuf 11 lock type ufs: EXCL by thread 0xfffffe0158d181e0 (pid 22398, rw, tid 100653) #0 0xffffffff80bbd85f at lockmgr_lock_flags+0x1cf #1 0xffffffff80f46cbd at ffs_lock+0x9d #2 0xffffffff811acefc at VOP_LOCK1_APV+0x3c #3 0xffffffff80cfe1e4 at _vn_lock+0x54 #4 0xffffffff80d0333d at vn_write+0x1ed #5 0xffffffff80d02d53 at vn_io_fault_doio+0x43 #6 0xffffffff80cfeba3 at vn_io_fault1+0x163 #7 0xffffffff80cfc40c at vn_io_fault+0x1cc #8 0xffffffff80c687b1 at dofilewrite+0x81 #9 0xffffffff80c682e0 at sys_write+0xc0 #10 0xffffffff810da455 at amd64_syscall+0x145 #11 0xffffffff810ad16b at fast_syscall_common+0xf8 nlink=1, effnlink=1, size=67313664, extsize 0 generation=36150da3, uid=2007, gid=0, flags=0x0 ino 197370, on dev md10 vnode 0xfffffe01582af4b0: type VREG usecount 1, writecount 1, refcount 2024 seqc users 0 hold count flags () flags (VIRF_PGREAD|VMP_LAZYLIST) v_object 0xfffff803ce110738 ref 0 pages 16176 cleanbuf 2015 dirtybuf 7 lock type ufs: EXCL by thread 0xfffffe0158d1ae40 (pid 22388, rw, tid 100659) #0 0xffffffff80bbd85f at lockmgr_lock_flags+0x1cf #1 0xffffffff80f46cbd at ffs_lock+0x9d #2 0xffffffff811acefc at VOP_LOCK1_APV+0x3c #3 0xffffffff80cfe1e4 at _vn_lock+0x54 #4 0xffffffff80d0333d at vn_write+0x1ed #5 0xffffffff80d02d53 at vn_io_fault_doio+0x43 #6 0xffffffff80cfeba3 at vn_io_fault1+0x163 #7 0xffffffff80cfc40c at vn_io_fault+0x1cc #8 0xffffffff80c687b1 at dofilewrite+0x81 #9 0xffffffff80c682e0 at sys_write+0xc0 #10 0xffffffff810da455 at amd64_syscall+0x145 #11 0xffffffff810ad16b at fast_syscall_common+0xf8 nlink=1, effnlink=1, size=66199552, extsize 0 generation=bb1f41ff, uid=2007, gid=0, flags=0x0 ino 65668, on dev md10 vnode 0xfffffe015acf7518: type VREG usecount 1, writecount 1, refcount 1955 seqc users 0 hold count flags () flags (VIRF_PGREAD|VMP_LAZYLIST) v_object 0xfffff804775eeb58 ref 0 pages 16800 cleanbuf 1952 dirtybuf 1 lock type ufs: EXCL by thread 0xfffffe014f1ade40 (pid 22393, rw, tid 100399) #0 0xffffffff80bbd85f at lockmgr_lock_flags+0x1cf #1 0xffffffff80f46cbd at ffs_lock+0x9d #2 0xffffffff811acefc at VOP_LOCK1_APV+0x3c #3 0xffffffff80cfe1e4 at _vn_lock+0x54 #4 0xffffffff80d0333d at vn_write+0x1ed #5 0xffffffff80d02d53 at vn_io_fault_doio+0x43 #6 0xffffffff80cfeba3 at vn_io_fault1+0x163 #7 0xffffffff80cfc40c at vn_io_fault+0x1cc #8 0xffffffff80c687b1 at dofilewrite+0x81 #9 0xffffffff80c682e0 at sys_write+0xc0 #10 0xffffffff810da455 at amd64_syscall+0x145 #11 0xffffffff810ad16b at fast_syscall_common+0xf8 nlink=1, effnlink=1, size=68751360, extsize 0 generation=33d632ee, uid=2007, gid=0, flags=0x0 ino 65673, on dev md10 vnode 0xfffffe015a3ba000: type VREG usecount 2, writecount 0, refcount 68 seqc users 0 hold count flags () flags (VIRF_PGREAD|VV_SYSTEM|VMP_LAZYLIST) v_object 0xfffff803df2afc60 ref 0 pages 528 cleanbuf 65 dirtybuf 1 lock type snaplk: EXCL by thread 0xfffffe01441231e0 (pid 22394, creat, tid 100556) with exclusive waiters pending #0 0xffffffff80bbe35c at lockmgr_xlock_hard+0x4ac #1 0xffffffff80bbe829 at __lockmgr_args+0x1f9 #2 0xffffffff80f1e8e3 at ffs_copyonwrite+0x1a3 #3 0xffffffff80f45adf at ffs_geom_strategy+0xcf #4 0xffffffff80cb65fc at bufwrite+0x24c #5 0xffffffff80f17096 at ffs_update+0x336 #6 0xffffffff80f5b616 at ufs_makeinode+0x3e6 #7 0xffffffff80f57834 at ufs_create+0x34 #8 0xffffffff811a9ab2 at VOP_CREATE_APV+0x62 #9 0xffffffff80cfd6b6 at vn_open_cred+0x2f6 #10 0xffffffff80cf3c1c at kern_openat+0x29c #11 0xffffffff810da455 at amd64_syscall+0x145 #12 0xffffffff810ad16b at fast_syscall_common+0xf8 nlink=1, effnlink=1, size=2147516416, extsize 0 generation=76cfe6f, uid=0, gid=5, flags=0x200000 ino 101, on dev md10 vnode 0xfffffe0158a2de10: type VREG usecount 1, writecount 1, refcount 484 seqc users 0 hold count flags () flags (VIRF_PGREAD|VMP_LAZYLIST) v_object 0xfffff803df91ee70 ref 0 pages 3856 cleanbuf 478 dirtybuf 4 lock type ufs: EXCL by thread 0xfffffe014f1ab900 (pid 22386, rw, tid 100607) #0 0xffffffff80bbd85f at lockmgr_lock_flags+0x1cf #1 0xffffffff80f46cbd at ffs_lock+0x9d #2 0xffffffff811acefc at VOP_LOCK1_APV+0x3c #3 0xffffffff80cfe1e4 at _vn_lock+0x54 #4 0xffffffff80d0333d at vn_write+0x1ed #5 0xffffffff80d02d53 at vn_io_fault_doio+0x43 #6 0xffffffff80cfeba3 at vn_io_fault1+0x163 #7 0xffffffff80cfc40c at vn_io_fault+0x1cc #8 0xffffffff80c687b1 at dofilewrite+0x81 #9 0xffffffff80c682e0 at sys_write+0xc0 #10 0xffffffff810da455 at amd64_syscall+0x145 #11 0xffffffff810ad16b at fast_syscall_common+0xf8 nlink=1, effnlink=1, size=15736832, extsize 0 generation=78b70fcd, uid=2007, gid=0, flags=0x0 ino 199048, on dev md10 vnode 0xfffffe0155fe0960: type VREG usecount 1, writecount 1, refcount 376 seqc users 1 hold count flags () flags (VIRF_PGREAD|VMP_LAZYLIST) v_object 0xfffff803df598210 ref 0 pages 2992 cleanbuf 360 dirtybuf 14 lock type ufs: EXCL by thread 0xfffffe015b1f1720 (pid 22403, rw, tid 100691) #0 0xffffffff80bbd85f at lockmgr_lock_flags+0x1cf #1 0xffffffff80f46cbd at ffs_lock+0x9d #2 0xffffffff811acefc at VOP_LOCK1_APV+0x3c #3 0xffffffff80cfe1e4 at _vn_lock+0x54 #4 0xffffffff80d0333d at vn_write+0x1ed #5 0xffffffff80d02d53 at vn_io_fault_doio+0x43 #6 0xffffffff80cfeba3 at vn_io_fault1+0x163 #7 0xffffffff80cfc40c at vn_io_fault+0x1cc #8 0xffffffff80c687b1 at dofilewrite+0x81 #9 0xffffffff80c682e0 at sys_write+0xc0 #10 0xffffffff810da455 at amd64_syscall+0x145 #11 0xffffffff810ad16b at fast_syscall_common+0xf8 nlink=1, effnlink=1, size=12222464, extsize 0 generation=a7e7546d, uid=2007, gid=0, flags=0x0 ino 199074, on dev md10 vnode 0xfffffe015650e068: type VDIR usecount 3, writecount 0, refcount 2 seqc users 1 mountedhere 0 hold count flags () flags (VMP_LAZYLIST) v_object 0xfffff80477f3b840 ref 0 pages 0 cleanbuf 0 dirtybuf 1 lock type ufs: EXCL by thread 0xfffffe0143837e40 (pid 22383, mkdir, tid 100240) #0 0xffffffff80bbd85f at lockmgr_lock_flags+0x1cf #1 0xffffffff80f46cbd at ffs_lock+0x9d #2 0xffffffff811acefc at VOP_LOCK1_APV+0x3c #3 0xffffffff80cfe1e4 at _vn_lock+0x54 #4 0xffffffff80cd55a0 at vfs_lookup+0x140 #5 0xffffffff80cd4bcd at namei+0x33d #6 0xffffffff80cfa321 at kern_frmdirat+0x161 #7 0xffffffff810da455 at amd64_syscall+0x145 #8 0xffffffff810ad16b at fast_syscall_common+0xf8 nlink=2, effnlink=2, size=512, extsize 0 generation=bb13a84d, uid=2007, gid=0, flags=0x0 ino 12, on dev md10 vnode 0xfffffe01594fabb8: type VDIR usecount 1, writecount 0, refcount 2 seqc users 1 mountedhere 0 hold count flags () flags (VMP_LAZYLIST) lock type ufs: EXCL by thread 0xfffffe0143837e40 (pid 22383, mkdir, tid 100240) #0 0xffffffff80bbd85f at lockmgr_lock_flags+0x1cf #1 0xffffffff80f46cbd at ffs_lock+0x9d #2 0xffffffff811acefc at VOP_LOCK1_APV+0x3c #3 0xffffffff80cfe1e4 at _vn_lock+0x54 #4 0xffffffff80ce507a at vget_finish+0x7a #5 0xffffffff80cd3261 at vfs_hash_get+0xd1 #6 0xffffffff80f420f8 at ffs_vgetf+0x58 #7 0xffffffff80f5132e at ufs_lookup_ino+0xdde #8 0xffffffff811a98ea at VOP_CACHEDLOOKUP_APV+0x5a #9 0xffffffff80cc4d86 at vfs_cache_lookup+0xa6 #10 0xffffffff811a972a at VOP_LOOKUP_APV+0x5a #11 0xffffffff80cd5919 at vfs_lookup+0x4b9 #12 0xffffffff80cd4bcd at namei+0x33d #13 0xffffffff80cfa321 at kern_frmdirat+0x161 #14 0xffffffff810da455 at amd64_syscall+0x145 #15 0xffffffff810ad16b at fast_syscall_common+0xf8 nlink=0, effnlink=0, size=512, extsize 0 generation=957a2a3a, uid=2007, gid=0, flags=0x0 ino 11, on dev md10 vnode 0xfffffe0158cdc2c0: type VDIR usecount 2, writecount 0, refcount 2 seqc users 1 mountedhere 0 hold count flags () flags (VMP_LAZYLIST) v_object 0xfffff80665aba210 ref 0 pages 0 cleanbuf 0 dirtybuf 1 lock type ufs: EXCL by thread 0xfffffe00e4d7ec80 (pid 22385, mkdir, tid 100217) #0 0xffffffff80bbd85f at lockmgr_lock_flags+0x1cf #1 0xffffffff80f46cbd at ffs_lock+0x9d #2 0xffffffff811acefc at VOP_LOCK1_APV+0x3c #3 0xffffffff80cfe1e4 at _vn_lock+0x54 #4 0xffffffff80cd55a0 at vfs_lookup+0x140 #5 0xffffffff80cd4bcd at namei+0x33d #6 0xffffffff80cf9f95 at kern_mkdirat+0x105 #7 0xffffffff810da455 at amd64_syscall+0x145 #8 0xffffffff810ad16b at fast_syscall_common+0xf8 nlink=3, effnlink=3, size=512, extsize 0 generation=10af12eb, uid=2007, gid=0, flags=0x0 ino 8, on dev md10 vnode 0xfffffe0158cd2770: type VREG usecount 1, writecount 0, refcount 1 seqc users 1 hold count flags () flags (VMP_LAZYLIST) lock type ufs: EXCL by thread 0xfffffe01441231e0 (pid 22394, creat, tid 100556) #0 0xffffffff80bbe35c at lockmgr_xlock_hard+0x4ac #1 0xffffffff80bbe829 at __lockmgr_args+0x1f9 #2 0xffffffff80f4220d at ffs_vgetf+0x16d #3 0xffffffff80f100ef at ffs_valloc+0x58f #4 0xffffffff80f5b31b at ufs_makeinode+0xeb #5 0xffffffff80f57834 at ufs_create+0x34 #6 0xffffffff811a9ab2 at VOP_CREATE_APV+0x62 #7 0xffffffff80cfd6b6 at vn_open_cred+0x2f6 #8 0xffffffff80cf3c1c at kern_openat+0x29c #9 0xffffffff810da455 at amd64_syscall+0x145 #10 0xffffffff810ad16b at fast_syscall_common+0xf8 nlink=1, effnlink=1, size=0, extsize 0 generation=b7310607, uid=2007, gid=0, flags=0x0 ino 199750, on dev md10 vnode 0xfffffe01598de960: type VDIR usecount 1, writecount 0, refcount 2 seqc users 1 mountedhere 0 hold count flags () flags (VMP_LAZYLIST) lock type ufs: EXCL by thread 0xfffffe00e4d7ec80 (pid 22385, mkdir, tid 100217) #0 0xffffffff80bbe35c at lockmgr_xlock_hard+0x4ac #1 0xffffffff80bbe829 at __lockmgr_args+0x1f9 #2 0xffffffff80f4220d at ffs_vgetf+0x16d #3 0xffffffff80f100ef at ffs_valloc+0x58f #4 0xffffffff80f5a43f at ufs_mkdir+0xff #5 0xffffffff811ac1b5 at VOP_MKDIR_APV+0x65 #6 0xffffffff80cfa09d at kern_mkdirat+0x20d #7 0xffffffff810da455 at amd64_syscall+0x145 #8 0xffffffff810ad16b at fast_syscall_common+0xf8 nlink=2, effnlink=2, size=512, extsize 0 generation=f9652179, uid=2007, gid=0, flags=0x0 ino 13, on dev md10 vnode 0xfffffe015aedf2c0: type VREG usecount 1, writecount 0, refcount 1 seqc users 1 hold count flags () flags (VMP_LAZYLIST) lock type ufs: EXCL by thread 0xfffffe014462fac0 (pid 22390, creat, tid 100334) #0 0xffffffff80bbe35c at lockmgr_xlock_hard+0x4ac #1 0xffffffff80bbe829 at __lockmgr_args+0x1f9 #2 0xffffffff80f4220d at ffs_vgetf+0x16d #3 0xffffffff80f100ef at ffs_valloc+0x58f #4 0xffffffff80f5b31b at ufs_makeinode+0xeb #5 0xffffffff80f57834 at ufs_create+0x34 #6 0xffffffff811a9ab2 at VOP_CREATE_APV+0x62 #7 0xffffffff80cfd6b6 at vn_open_cred+0x2f6 #8 0xffffffff80cf3c1c at kern_openat+0x29c #9 0xffffffff810da455 at amd64_syscall+0x145 #10 0xffffffff810ad16b at fast_syscall_common+0xf8 nlink=1, effnlink=1, size=0, extsize 0 generation=48da73a0, uid=2007, gid=0, flags=0x0 ino 199751, on dev md10 db:1:pho1> show mount 0xfffffe010e3aa700 /dev/gpt/work on / (ufs) 0xfffffe010e3ab280 devfs on /dev (devfs) 0xfffffe01437b4280 /dev/gpt/tmp on /tmp (ufs) 0xfffffe0157090280 /dev/md10 on /mnt (ufs) More info: show mount db:1:pho1> show bufqueues bqempty: 83396 Buf domain 0 freebufs 7791 lofreebufs 219 hifreebufs 329 bufspace 240574464 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 59 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 7160 (234610688) dirtyq count 59 (1933312) wakeup 0 lim 25 CPU 15, 12, 9, 2, 21, 21, 13, 4, 11, 4, 10, 1, Locked buffers: 1 space 32768 Total buffers: 20576 space 240574464 Buf domain 1 freebufs 15128 lofreebufs 219 hifreebufs 329 bufspace 36864 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 1 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 3 (28672) dirtyq count 1 (4096) wakeup 0 lim 25 CPU 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, Locked buffers: 0 space 0 Total buffers: 12768 space 36864 Buf domain 2 freebufs 9721 lofreebufs 219 hifreebufs 329 bufspace 177127424 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 35 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 5255 (172109824) dirtyq count 35 (1019904) wakeup 0 lim 25 CPU 1, 7, 23, 14, 21, 14, 12, 0, 1, 1, 21, 7, Locked buffers: 0 space 0 Total buffers: 12800 space 177127424 Buf domain 3 freebufs 15109 lofreebufs 219 hifreebufs 329 bufspace 458752 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 3 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 18 (434176) dirtyq count 3 (12288) wakeup 0 lim 25 CPU 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, Locked buffers: 0 space 0 Total buffers: 7348 space 458752 Buf domain 4 freebufs 15043 lofreebufs 219 hifreebufs 329 bufspace 2682880 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 29 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 41 (1167360) dirtyq count 31 (925696) wakeup 0 lim 25 CPU 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, Locked buffers: 20 space 655360 Total buffers: 19136 space 2682880 Buf domain 5 freebufs 12660 lofreebufs 219 hifreebufs 329 bufspace 80846848 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 24 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 2322 (75984896) dirtyq count 25 (761856) wakeup 0 lim 25 CPU 2, 4, 23, 5, 4, 23, 8, 5, 14, 20, 17, 0, Locked buffers: 17 space 528384 Total buffers: 4278 space 80846848 Buf domain 6 freebufs 7944 lofreebufs 219 hifreebufs 329 bufspace 235425792 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 51 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 7013 (229773312) dirtyq count 51 (1556480) wakeup 0 lim 25 CPU 5, 6, 21, 14, 7, 2, 6, 8, 24, 19, 9, 3, Locked buffers: 4 space 102400 Total buffers: 29025 space 235425792 db:1:pho1> show page vm_cnt.v_free_count: 30653 vm_cnt.v_inactive_count: 54016 vm_cnt.v_active_count: 6547542 vm_cnt.v_laundry_count: 1039177 vm_cnt.v_wire_count: 452140 vm_cnt.v_free_reserved: 10643 vm_cnt.v_free_min: 51262 vm_cnt.v_free_target: 173119 vm_cnt.v_inactive_target: 259678 db:1:pho1> show pageq pq_free 30653 dom 0 page_cnt 8123862 free 30653 pq_act 6547542 pq_inact 54016 pq_laund 1039177 pq_unsw 0 db:0:pho> show freepages DOMAIN: 0 FREE LIST 0: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000000 | 000000 10 (004096K) | 000000 | 000000 09 (002048K) | 000000 | 000000 08 (001024K) | 000000 | 000000 07 (000512K) | 000000 | 000000 06 (000256K) | 000000 | 000001 05 (000128K) | 000022 | 000001 04 (000064K) | 000255 | 000004 03 (000032K) | 000763 | 000010 02 (000016K) | 003690 | 000040 01 (000008K) | 000000 | 000588 00 (000004K) | 000001 | 003342 FREE LIST 1: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000000 | 000000 10 (004096K) | 000000 | 000000 09 (002048K) | 000000 | 000000 08 (001024K) | 000000 | 000000 07 (000512K) | 000000 | 000000 06 (000256K) | 000000 | 000000 05 (000128K) | 000000 | 000000 04 (000064K) | 000001 | 000000 03 (000032K) | 000000 | 000000 02 (000016K) | 000003 | 000000 01 (000008K) | 000013 | 000000 00 (000004K) | 000025 | 000000 db:0:pho> show uma Zone Size Used Free Requests Sleeps Bucket Total Mem XFree FFS inode 1160 170087 12725 68936521 0 8 212061920 0 malloc-4096 4096 34122 0 130609 0 2 139763712 0 VNODE 600 170214 20652 69290491 0 16 114519600 0 RADIX NODE 144 533171 2970 50558369 0 62 77204304 0 FFS2 dinode 256 170087 21268 68936519 0 62 48986880 0 VM OBJECT 264 66096 49854 91605688 0 30 30610800 0 mbuf_cluster 2048 11401 15 98701 0 8 23379968 0 S VFS Cache 104 170350 4643 5933496 0 126 18199272 0 BUF TRIE 144 1633 104319 16479527 0 62 15257088 0 swblk 136 18783 90895 1012848 0 62 14916208 0 mbuf 256 15530 5025 5211565 0 254 5262080 0 UMA Slabs 0 112 40977 3261 93479 0 126 4954656 0 vmem btag 56 54840 20964 207593 0 254 4245024 0 malloc-256 256 15345 0 106523524 0 62 3928320 0 malloc-128 128 29441 9 207733864 0 126 3769600 0 malloc-128 128 25998 352 335254 0 126 3372800 0 pbuf 2776 0 808 0 0 0 2243008 0 malloc-128 128 15764 170 79943340 0 126 2039552 0 malloc-4096 4096 335 0 2123629 0 2 1372160 0 swpctrie 144 1758 6334 110313 0 62 1165248 0 malloc-384 384 2836 14 112370780 0 30 1094400 0 THREAD 1824 453 86 742 0 8 983136 0 malloc-8192 8192 86 0 86 0 1 704512 0 malloc-64 64 8619 2154 1825064 0 254 689472 0 malloc-64 64 9497 393 177675309 0 254 632960 0 VMSPACE 2560 71 172 1509850 0 4 622080 0 L VFS Cache 320 1816 92 1816 0 30 610560 0 malloc-128 128 2070 2394 4065565 0 126 571392 0 MAP ENTRY 96 2529 2763 79549572 0 126 508032 0 UMA Zones 2048 230 5 230 0 8 481280 0 FPU_save_area 832 455 85 7965 0 16 449280 0 malloc-32768 32768 13 0 13 0 1 425984 0 malloc-32768 32768 12 0 12 0 1 393216 0 lkpimm 168 3 2325 3 0 62 391104 0 lkpicurr 168 13 2315 13 0 62 391104 0 malloc-4096 4096 91 0 1627792 0 2 372736 0 malloc-32768 32768 11 0 637 0 1 360448 0 PROC 1368 90 163 1627791 0 8 346104 0 malloc-512 512 512 120 315757 0 30 323584 0 malloc-64 64 4143 645 4171 0 254 306432 0 malloc-32 32 8210 1366 8210 0 254 306432 0 malloc-128 128 2320 36 87144730 0 126 301568 0 malloc-256 256 125 1015 156407950 0 62 291840 0 malloc-16 16 15899 1851 16061 0 254 284000 0 malloc-16384 16384 16 0 60467 0 1 262144 0 filedesc0 1072 91 126 1627792 0 8 232624 0 Mountpoints 2944 4 68 117928 0 4 211968 0 malloc-65536 65536 3 0 3 0 1 196608 0 malloc-65536 65536 3 0 122018 0 1 196608 0 malloc-65536 65536 3 0 114 0 1 196608 0 malloc-65536 65536 3 0 3 0 1 196608 0 malloc-1024 1024 160 28 589881 0 16 192512 0 Files 80 134 2166 73492921 0 126 184000 0 malloc-8192 8192 22 0 474880 0 1 180224 0 malloc-2048 2048 85 1 86 0 8 176128 0 malloc-256 256 259 401 83756452 0 62 168960 0 malloc-16384 16384 10 0 18 0 1 163840 0 malloc-64 64 792 1728 2196441 0 254 161280 0 ksiginfo 112 242 1054 7515 0 126 145152 0 malloc-1024 1024 123 13 323 0 16 139264 0 PWD 32 32 4252 10770743 0 254 137088 0 malloc-8192 8192 16 0 58 0 1 131072 0 malloc-4096 4096 31 0 356362 0 2 126976 0 malloc-384 384 254 76 254 0 30 126720 0 PGRP 88 30 1396 133 0 126 125488 0 malloc-64 64 135 1818 1863762 0 254 124992 0 fakepg 104 0 1170 84 0 126 121680 0 TURNSTILE 136 540 279 540 0 62 111384 0 malloc-8192 8192 13 0 13 0 1 106496 0 malloc-256 256 248 157 1934009 0 62 103680 0 udp_inpcb 424 11 232 374 0 30 103032 0 DEVCTL 1024 0 100 1297929 0 0 102400 0 32 Bucket 256 18 372 4683873 0 62 99840 0 malloc-32768 32768 3 0 121597 0 1 98304 0 malloc-4096 4096 24 0 28 0 2 98304 0 128 Bucket 1024 28 63 5777636 0 16 93184 0 malloc-8192 8192 11 0 13 0 1 90112 0 malloc-1024 1024 68 20 5464507 0 16 90112 0 ttyinq 160 330 220 540 0 62 88000 0 ttyoutq 256 173 142 282 0 62 80640 0 malloc-256 256 49 266 75464087 0 62 80640 0 malloc-32 32 376 2144 1194412 0 254 80640 0 UMA Kegs 384 200 3 200 0 30 77952 0 malloc-256 256 255 45 1615154 0 62 76800 0 SLEEPQUEUE 88 540 324 540 0 126 76032 0 socket 960 32 44 6026 0 16 72960 0 malloc-2048 2048 17 17 773 0 8 69632 0 256 Bucket 2048 26 8 1369060 0 8 69632 0 rl_entry 40 235 1482 235 0 254 68680 0 malloc-64 64 273 798 158492 0 254 68544 0 malloc-16 16 498 3752 4396375 0 254 68000 0 malloc-65536 65536 1 0 1 0 1 65536 0 malloc-128 128 125 340 62585 0 126 59520 0 malloc-1024 1024 48 8 53 0 16 57344 0 64 Bucket 512 50 62 7974219 0 30 57344 0 malloc-32 32 198 1566 827459 0 254 56448 0 malloc-32 32 279 1485 1369432 0 254 56448 0 malloc-32 32 591 1173 263625 0 254 56448 0 pipe 728 10 67 384899 0 16 56056 0 malloc-384 384 109 31 110 0 30 53760 0 malloc-64 64 70 749 27972242 0 254 52416 0 malloc-32 32 329 1309 2595 0 254 52416 0 pcpu-16 16 72 3000 72 0 254 49152 0 malloc-16 16 431 2569 1294789 0 254 48000 0 cpuset 104 27 407 27 0 126 45136 0 malloc-2048 2048 18 4 20 0 8 45056 0 pcpu-8 8 3527 2105 3539 0 254 45056 0 malloc-2048 2048 14 6 237767 0 8 40960 0 pcpu-64 64 460 180 460 0 254 40960 0 malloc-384 384 44 56 49 0 30 38400 0 unpcb 256 13 122 5607 0 62 34560 0 malloc-384 384 33 57 9914 0 30 34560 0 16 Bucket 144 63 161 2096685 0 62 32256 0 malloc-16 16 42 1958 2831030 0 254 32000 0 malloc-16 16 243 1757 287 0 254 32000 0 malloc-128 128 50 198 65 0 126 31744 0 tcpcb 1104 5 23 10 0 8 30912 0 malloc-256 256 24 96 144 0 62 30720 0 malloc-1024 1024 18 10 292 0 16 28672 0 malloc-1024 1024 15 13 26 0 16 28672 0 malloc-512 512 26 22 27 0 30 24576 0 rtentry 168 12 132 16 0 62 24192 0 malloc-16 16 24 1476 190633 0 254 24000 0 malloc-16 16 478 1022 6291295 0 254 24000 0 malloc-128 128 9 177 10 0 126 23808 0 DIRHASH 1024 12 8 1573859 0 16 20480 0 g_bio 408 17 33 47003842 0 30 20400 0 udp_inpcb ports 32 3 627 10 0 254 20160 0 malloc-64 64 12 303 16 0 254 20160 0 malloc-32 32 20 610 64 0 254 20160 0 routing nhops 256 9 66 16 0 62 19200 0 malloc-384 384 22 28 8704 0 30 19200 0 tcp_inpcb 424 5 40 10 0 30 19080 0 malloc-16384 16384 1 0 1 0 1 16384 0 malloc-8192 8192 2 0 2 0 1 16384 0 malloc-4096 4096 4 0 354131 0 2 16384 0 malloc-512 512 4 28 3185933 0 30 16384 0 malloc-512 512 12 20 14 0 30 16384 0 malloc-2048 2048 4 2 5686 0 8 12288 0 malloc-1024 1024 7 5 19 0 16 12288 0 udpcb 32 11 367 374 0 254 12096 0 ertt 72 5 163 10 0 126 12096 0 malloc-16 16 5 745 6 0 254 12000 0 8 Bucket 80 105 45 2293314 0 126 12000 0 malloc-384 384 6 24 20 0 30 11520 0 malloc-256 256 9 36 21 0 62 11520 0 swwbuf 2776 4 0 476382 0 1 11104 0 NAMEI 1024 4 4 164106775 0 16 8192 0 malloc-8192 8192 1 0 1 0 1 8192 0 malloc-4096 4096 2 0 115782 0 2 8192 0 malloc-2048 2048 2 2 10 0 8 8192 0 malloc-512 512 2 14 54 0 30 8192 0 tcp_inpcb ports 32 2 250 2 0 254 8064 0 malloc-32 32 9 243 235950 0 254 8064 0 kenv 258 18 12 21837 0 30 7740 0 malloc-384 384 2 18 175 0 30 7680 0 ripcb 424 1 17 3 0 30 7632 0 epoch_record pcpu 256 4 12 4 0 62 4096 0 malloc-2048 2048 1 1 1 0 8 4096 0 malloc-1024 1024 2 2 24 0 16 4096 0 malloc-512 512 1 7 13 0 30 4096 0 SMR CPU 32 7 120 7 0 254 4064 0 vtnet_tx_hdr 24 0 167 0 0 254 4008 0 UMA Slabs 1 176 10 12 10 0 62 3872 0 ada_ccb 272 0 14 1119774 0 30 3808 0 KMAP ENTRY 96 33 6 35 0 0 3744 0 vmem 1856 1 1 1 0 8 3712 0 SMR SHARED 24 7 120 7 0 254 3048 0 mbuf_packet 256 10 0 5189718 0 254 2560 0 FFS1 dinode 128 0 0 0 0 126 0 0 da_ccb 544 0 0 0 0 16 0 0 tfo_ccache_entries 80 0 0 0 0 126 0 0 tfo 4 0 0 0 0 254 0 0 sackhole 32 0 0 0 0 254 0 0 hostcache 64 0 0 0 0 254 0 0 syncache 168 0 0 1 0 62 0 0 ipq 56 0 0 0 0 254 0 0 tcp_log_id_node 120 0 0 0 0 126 0 0 tcp_log_id_bucket 176 0 0 0 0 62 0 0 tcp_log 416 0 0 0 0 30 0 0 tcpreass 48 0 0 0 0 254 0 0 ripcb ports 32 0 0 0 0 254 0 0 udplite_inpcb ports 32 0 0 0 0 254 0 0 udplite_inpcb 424 0 0 0 0 30 0 0 IPsec SA lft_c 16 0 0 0 0 254 0 0 KNOTE 160 0 0 0 0 62 0 0 itimer 352 0 0 0 0 30 0 0 AIOLIO 272 0 0 0 0 30 0 0 AIOCB 552 0 0 0 0 16 0 0 AIO 208 0 0 0 0 62 0 0 TMPFS node 224 0 0 0 0 62 0 0 NCLNODE 760 0 0 0 0 16 0 0 LTS VFS Cache 360 0 0 0 0 30 0 0 STS VFS Cache 144 0 0 0 0 62 0 0 cryptop 280 0 0 0 0 30 0 0 linux_dma_object 32 0 0 0 0 254 0 0 linux_dma_pctrie 144 0 0 0 0 62 0 0 IOMMU_MAP_ENTRY 104 0 0 0 0 126 0 0 mbuf_jumbo_16k 16384 0 0 0 0 1 0 0 mbuf_jumbo_9k 9216 0 0 0 0 1 0 0 mbuf_jumbo_page 4096 0 0 3 0 2 0 0 audit_record 1280 0 0 0 0 8 0 0 domainset 40 0 0 0 0 254 0 0 MAC labels 40 0 0 0 0 254 0 0 clpbuf 2776 0 0 221333 0 2 0 0 vnpbuf 2776 0 0 0 0 4 0 0 mdpbuf 2776 0 0 75277 0 1 0 0 nfspbuf 2776 0 0 0 0 2 0 0 swrbuf 2776 0 0 108134 0 2 0 0 ertt_txseginfo 40 0 0 524 0 254 0 0 umtx_shm 88 0 0 0 0 126 0 0 umtx pi 96 0 0 0 0 126 0 0 rangeset pctrie nodes 144 0 0 0 0 62 0 0 malloc-65536 65536 0 0 0 0 1 0 0 malloc-65536 65536 0 0 0 0 1 0 0 malloc-65536 65536 0 0 88 0 1 0 0 malloc-32768 32768 0 0 0 0 1 0 0 malloc-32768 32768 0 0 241078 0 1 0 0 malloc-32768 32768 0 0 2125353 0 1 0 0 malloc-32768 32768 0 0 0 0 1 0 0 malloc-16384 16384 0 0 0 0 1 0 0 malloc-16384 16384 0 0 0 0 1 0 0 malloc-16384 16384 0 0 4 0 1 0 0 malloc-16384 16384 0 0 22 0 1 0 0 malloc-16384 16384 0 0 0 0 1 0 0 malloc-8192 8192 0 0 0 0 1 0 0 malloc-4096 4096 0 0 0 0 2 0 0 malloc-2048 2048 0 0 0 0 8 0 0 malloc-512 512 0 0 235848 0 30 0 0 malloc-512 512 0 0 0 0 30 0 0 pcpu-32 32 0 0 0 0 254 0 0 pcpu-4 4 0 0 0 0 254 0 0 4 Bucket 48 0 0 244191 0 254 0 0 2 Bucket 32 0 0 607464 0 254 0 0 UMA Hash 256 0 0 0 0 62 0 0 db:0:pho> show umacache Zone Size Used Free Requests Bucket debugnet_mbuf_packet 2048 0 0 0 0 debugnet_mbuf_cluster 2048 -6144 0 0 0 debugnet_mbuf 256 -10240 0 0 0 vm pgcache 4096 116597 0 96733368 3 vm pgcache 4096 2494790 0 237715689 8 buf free cache 848 22535 0 27824644 16 buffer arena-65 65536 0 0 0 1 buffer arena-61 61440 0 0 0 1 buffer arena-57 57344 0 0 0 1 buffer arena-53 53248 0 0 0 1 buffer arena-49 49152 0 0 0 1 buffer arena-45 45056 0 0 0 1 buffer arena-40 40960 0 0 0 1 buffer arena-36 36864 0 0 0 1 buffer arena-32 32768 212 0 18453168 1 buffer arena-28 28672 0 0 0 1 buffer arena-24 24576 0 0 0 1 buffer arena-20 20480 0 0 0 1 buffer arena-16 16384 4 0 436075 1 buffer arena-12 12288 0 0 0 1 buffer arena-81 8192 0 0 0 1 buffer arena-40 4096 0 0 0 2 kstack_cache 16384 453 0 742 1 db:0:pho> acttrace Tracing command pagedaemon pid 9 tid 100195 td 0xfffffe010e3a6ac0 (CPU 11) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003cea2df0 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003cea2e00 trap() at trap+0x57/frame 0xfffffe003cea2f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003cea2f20 --- trap 0x13, rip = 0xffffffff80f9448b, rsp = 0xfffffe010db43c00, rbp = 0xfffffe010db43eb0 --- vm_pageout_worker() at vm_pageout_worker+0x9ab/frame 0xfffffe010db43eb0 vm_pageout() at vm_pageout+0x1d7/frame 0xfffffe010db43ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe010db43f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010db43f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xfffffe00e4c54c80 (CPU 1) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce20df0 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce20e00 trap() at trap+0x57/frame 0xfffffe003ce20f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce20f20 --- trap 0x13, rip = 0xffffffff810965a6, rsp = 0xfffffe00377ebdd0, rbp = 0xfffffe00377ebdd0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00377ebdd0 acpi_cpu_idle() at acpi_cpu_idle+0x278/frame 0xfffffe00377ebe10 cpu_idle_acpi() at cpu_idle_acpi+0x4d/frame 0xfffffe00377ebe30 cpu_idle() at cpu_idle+0xac/frame 0xfffffe00377ebe50 sched_idletd() at sched_idletd+0x4b1/frame 0xfffffe00377ebef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377ebf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377ebf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffffe00e4c54560 (CPU 2) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce2ddf0 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce2de00 trap() at trap+0x57/frame 0xfffffe003ce2df20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce2df20 --- trap 0x13, rip = 0xffffffff810965a6, rsp = 0xfffffe00377fadd0, rbp = 0xfffffe00377fadd0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00377fadd0 acpi_cpu_idle() at acpi_cpu_idle+0x278/frame 0xfffffe00377fae10 cpu_idle_acpi() at cpu_idle_acpi+0x4d/frame 0xfffffe00377fae30 cpu_idle() at cpu_idle+0xac/frame 0xfffffe00377fae50 sched_idletd() at sched_idletd+0x4b1/frame 0xfffffe00377faef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377faf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377faf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command md10 pid 69482 tid 100652 td 0xfffffe014f1b1740 (CPU 7) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce6edf0 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce6ee00 trap() at trap+0x57/frame 0xfffffe003ce6ef20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce6ef20 --- trap 0x13, rip = 0xffffffff810d56e6, rsp = 0xfffffe0156f86d20, rbp = 0xfffffe0156f86d20 --- memmove_erms() at memmove_erms+0x116/frame 0xfffffe0156f86d20 uiomove_fromphys() at uiomove_fromphys+0x87/frame 0xfffffe0156f86d90 physcopyin() at physcopyin+0xbe/frame 0xfffffe0156f86e20 mdstart_swap() at mdstart_swap+0x40b/frame 0xfffffe0156f86ea0 md_kthread() at md_kthread+0x20c/frame 0xfffffe0156f86ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe0156f86f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0156f86f30 --- trap 0xc, rip = 0x3e103b8e929a, rsp = 0x3e1038c7cdb8, rbp = 0x3e1038c7ce70 --- Tracing command rw pid 22391 tid 100587 td 0xfffffe014f1ae560 (CPU 5) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce54df0 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce54e00 trap() at trap+0x57/frame 0xfffffe003ce54f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce54f20 --- trap 0x13, rip = 0xffffffff80f6e9f3, rsp = 0xfffffe0156e40a70, rbp = 0xfffffe0156e40a80 --- mtrash_ctor() at mtrash_ctor+0x33/frame 0xfffffe0156e40a80 item_ctor() at item_ctor+0x11f/frame 0xfffffe0156e40ad0 zone_alloc_item() at zone_alloc_item+0x138/frame 0xfffffe0156e40b20 malloc() at malloc+0x86/frame 0xfffffe0156e40b60 cloneuio() at cloneuio+0x2b/frame 0xfffffe0156e40b90 vn_io_fault1() at vn_io_fault1+0x133/frame 0xfffffe0156e40cd0 vn_io_fault() at vn_io_fault+0x1cc/frame 0xfffffe0156e40d50 dofileread() at dofileread+0x83/frame 0xfffffe0156e40d90 sys_read() at sys_read+0xc0/frame 0xfffffe0156e40e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe0156e40f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0156e40f30 --- syscall (3, FreeBSD ELF64, read), rip = 0x6d7af00867a, rsp = 0x6d7ada56e98, rbp = 0x6d7ada57f50 --- Tracing command creat pid 22394 tid 100556 td 0xfffffe01441231e0 (CPU 3) kdb_enter() at kdb_enter+0x32/frame 0xfffffe01419d24a0 vpanic() at vpanic+0x182/frame 0xfffffe01419d24f0 panic() at panic+0x43/frame 0xfffffe01419d2550 softdep_disk_io_initiation() at softdep_disk_io_initiation+0x14be/frame 0xfffffe01419d2610 ffs_geom_strategy() at ffs_geom_strategy+0x18c/frame 0xfffffe01419d2640 ufs_strategy() at ufs_strategy+0x83/frame 0xfffffe01419d2670 VOP_STRATEGY_APV() at VOP_STRATEGY_APV+0x61/frame 0xfffffe01419d2690 bufstrategy() at bufstrategy+0x3e/frame 0xfffffe01419d26c0 bufwrite() at bufwrite+0x24c/frame 0xfffffe01419d2710 vfs_bio_awrite() at vfs_bio_awrite+0x21e/frame 0xfffffe01419d2780 ffs_syncvnode() at ffs_syncvnode+0x36a/frame 0xfffffe01419d2810 ffs_copyonwrite() at ffs_copyonwrite+0x5ad/frame 0xfffffe01419d28b0 ffs_geom_strategy() at ffs_geom_strategy+0xcf/frame 0xfffffe01419d28e0 bufwrite() at bufwrite+0x24c/frame 0xfffffe01419d2930 ffs_update() at ffs_update+0x336/frame 0xfffffe01419d2990 ufs_makeinode() at ufs_makeinode+0x3e6/frame 0xfffffe01419d2b00 ufs_create() at ufs_create+0x34/frame 0xfffffe01419d2b20 VOP_CREATE_APV() at VOP_CREATE_APV+0x62/frame 0xfffffe01419d2b40 vn_open_cred() at vn_open_cred+0x2f6/frame 0xfffffe01419d2cb0 kern_openat() at kern_openat+0x29c/frame 0xfffffe01419d2e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe01419d2f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01419d2f30 --- syscall (499, FreeBSD ELF64, openat), rip = 0x598515ef5ba, rsp = 0x5984eec7f68, rbp = 0x5984eec8020 --- Tracing command swap pid 22396 tid 100584 td 0xfffffe014f1a8e40 (CPU 6) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce61df0 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce61e00 trap() at trap+0x57/frame 0xfffffe003ce61f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce61f20 --- trap 0x13, rip = 0x24455ce45755, rsp = 0x244d7db467b0, rbp = 0x244d7db467e0 --- Tracing command rw pid 22397 tid 100272 td 0xfffffe0142ae2e40 (CPU 4) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce47df0 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce47e00 trap() at trap+0x57/frame 0xfffffe003ce47f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce47f20 --- trap 0x13, rip = 0xffffffff80c6266b, rsp = 0xfffffe0141883a40, rbp = 0xfffffe0141883bf0 --- witness_checkorder() at witness_checkorder+0x1cb/frame 0xfffffe0141883bf0 __mtx_lock_spin_flags() at __mtx_lock_spin_flags+0x8e/frame 0xfffffe0141883c30 wakeup() at wakeup+0xf/frame 0xfffffe0141883c50 rangelock_calc_block() at rangelock_calc_block+0x45/frame 0xfffffe0141883c80 rangelock_enqueue() at rangelock_enqueue+0xdb/frame 0xfffffe0141883cd0 vn_io_fault() at vn_io_fault+0x19c/frame 0xfffffe0141883d50 dofileread() at dofileread+0x83/frame 0xfffffe0141883d90 sys_read() at sys_read+0xc0/frame 0xfffffe0141883e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe0141883f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0141883f30 --- syscall (3, FreeBSD ELF64, read), rip = 0x6d7af00867a, rsp = 0x6d7ada56e98, rbp = 0x6d7ada57f50 --- Tracing command swap pid 22399 tid 100589 td 0xfffffe014462b900 (CPU 10) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce95df0 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce95e00 trap() at trap+0x57/frame 0xfffffe003ce95f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce95f20 --- trap 0x13, rip = 0x24455ce45757, rsp = 0x244d7db467b0, rbp = 0x244d7db467e0 --- Tracing command rw pid 22401 tid 100591 td 0xfffffe014f1b0900 (CPU 0) cpustop_handler() at cpustop_handler+0x28/frame 0xffffffff81feb300 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xffffffff81feb310 trap() at trap+0x57/frame 0xffffffff81feb430 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff81feb430 --- trap 0x13, rip = 0xffffffff80c624cc, rsp = 0xfffffe01419ebaa0, rbp = 0xfffffe01419ebc50 --- witness_checkorder() at witness_checkorder+0x2c/frame 0xfffffe01419ebc50 __mtx_lock_flags() at __mtx_lock_flags+0x94/frame 0xfffffe01419ebca0 rangelock_unlock() at rangelock_unlock+0x39/frame 0xfffffe01419ebcd0 vn_io_fault() at vn_io_fault+0x1e8/frame 0xfffffe01419ebd50 dofileread() at dofileread+0x83/frame 0xfffffe01419ebd90 sys_read() at sys_read+0xc0/frame 0xfffffe01419ebe00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe01419ebf30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01419ebf30 --- syscall (3, FreeBSD ELF64, read), rip = 0x6d7af00867a, rsp = 0x6d7ada56e98, rbp = 0x6d7ada57f50 --- Tracing command swap pid 22402 tid 100669 td 0xfffffe0158d1d1e0 (CPU 9) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce88df0 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce88e00 trap() at trap+0x57/frame 0xfffffe003ce88f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce88f20 --- trap 0x13, rip = 0x24455ce4575c, rsp = 0x244d7db467b0, rbp = 0x244d7db467e0 --- db:0:pho> ps pid ppid pgrp uid state wmesg wchan cmd 28752 28750 3364 0 ?W sed 28750 28555 3364 0 SW wait 0xfffffe00e4c52570 sh 28731 28729 3285 0 ?W+ mksnap_ffs 28730 69471 3285 0 S+ piperd 0xfffffe0141a242d8 grep 28729 69471 3285 0 SW+ wait 0xfffffe010dde1000 sh 28555 3396 3364 0 S piperd 0xfffffe0141a24000 sh 22403 22384 69518 2007 D+ snaplk 0xfffff8000420ae30 rw 22402 22389 69518 2007 R+ CPU 9 swap 22401 22384 69518 2007 R+ CPU 0 rw 22400 22384 69518 2007 D+ snaplk 0xfffff8000420ae30 rw 22399 22389 69518 2007 R+ CPU 10 swap 22398 22384 69518 2007 D+ snaplk 0xfffff8000420ae30 rw 22397 22384 69518 2007 R+ CPU 4 rw 22396 22389 69518 2007 R+ CPU 6 swap 22394 22387 69518 2007 RL+ CPU 3 creat 22393 22384 69518 2007 D+ snaplk 0xfffff8000420ae30 rw 22392 22389 69518 2007 ?W+ swap 22391 22384 69518 2007 R+ CPU 5 rw 22390 22387 69518 2007 D+ getblk 0xfffffe003a6be950 creat 22389 22380 69518 2007 SW+ wait 0xfffffe0144060010 swap 22388 22384 69518 2007 D+ snaplk 0xfffff8000420ae30 rw 22387 22377 69518 2007 SW+ wait 0xfffffe0158d13ac0 creat 22386 22384 69518 2007 D+ snaplk 0xfffff8000420ae30 rw 22385 22382 69518 2007 D+ getblk 0xfffffe003878d2e0 mkdir 22384 22379 69518 2007 SW+ wait 0xfffffe0155efc570 rw 22383 22382 69518 2007 D+ getblk 0xfffffe003878d2e0 mkdir 22382 22378 69518 2007 SW+ wait 0xfffffe0143770568 mkdir 22380 69523 69518 2007 S+ nanslp 0xffffffff81eac8a8 swap 22379 69523 69518 2007 S+ nanslp 0xffffffff81eac8a0 rw 22378 69523 69518 2007 S+ nanslp 0xffffffff81eac8a7 mkdir 22377 69523 69518 2007 S+ nanslp 0xffffffff81eac8a7 creat 69523 69522 69518 2007 SW+ wait 0xfffffe015b1eeab8 run 69522 69518 69518 2007 SW+ wait 0xfffffe014376e558 run 69518 69486 69518 2007 S+ nanslp 0xffffffff81eac8a5 run 69486 69471 3285 0 SW+ wait 0xfffffe0158d14018 su 69482 0 0 0 RL CPU 7 [md10] 69471 69469 3285 0 SW+ wait 0xfffffe010e3a3018 sh 69470 3285 3285 0 S+ piperd 0xfffffe0156cf12d8 tee 69469 3285 3285 0 SW+ wait 0xfffffe0155f2d560 sh 3399 3364 3364 0 S piperd 0xfffffe014409f6c0 sh 3397 3364 3364 0 S piperd 0xfffffe01436fcb60 awk 3396 3364 3364 0 SW wait 0xfffffe0155ef9ab0 sh 3364 1 3364 0 SWs wait 0xfffffe0142ad9018 sh 3285 2781 3285 0 SW+ wait 0xfffffe010dde4018 sh 2781 2779 2781 0 SW+ wait 0xfffffe003ce1bab8 bash 2779 2778 2779 0 SW+ pause 0xfffffe013ba4bb60 csh 2778 2773 2778 2006 SW+ wait 0xfffffe0141e40ab8 su 2773 2772 2773 2006 SWs+ wait 0xfffffe00e4c52018 bash 2772 2769 2769 2006 S select 0xfffff80003eff540 sshd 2769 2728 2769 0 Ss select 0xfffff8000494f1c0 sshd 2755 1 2755 0 Ss+ ttyin 0xfffff8000616e8b0 getty 2754 1 2754 0 Ss+ ttyin 0xfffff8000616ecb0 getty 2753 1 2753 0 Ss+ ttyin 0xfffff800282880b0 getty 2752 1 2752 0 Ss+ ttyin 0xfffff800282884b0 getty 2751 1 2751 0 Ss+ ttyin 0xfffff800282888b0 getty 2750 1 2750 0 Ss+ ttyin 0xfffff80028288cb0 getty 2749 1 2749 0 Ss+ ttyin 0xfffff800282870b0 getty 2748 1 2748 0 Ss+ ttyin 0xfffff800282874b0 getty 2747 1 2747 0 Ss+ ttyin 0xfffff800282878b0 getty 2746 1 2746 0 Ss+ ttyin 0xfffff80028287cb0 getty 2732 1 2732 0 SWs nanslp 0xffffffff81eac8ab cron 2728 1 2728 0 Ss select 0xfffff8000494ed40 sshd 2709 1 2709 0 Ss select 0xfffff800320a6a40 inetd 2683 1 2683 123 Ss select 0xfffff800320a6dc0 ntpd 2657 1 2657 0 Ss select 0xfffff8000494e9c0 powerd 2602 1 2602 0 Ss nanslp 0xffffffff81eac8a8 watchdogd 2594 1 2594 0 Ss select 0xfffff80003eff5c0 syslogd 2407 1 2407 0 Ss select 0xfffff80003daecc0 devd 2402 1 2402 65 Ss select 0xfffff800320a6e40 dhclient 2326 1 2326 0 Ss select 0xfffff80003daed40 dhclient 2323 1 2323 0 Ss select 0xfffff800320a6ac0 dhclient 19 0 0 0 DL vlruwt 0xfffffe010dde4570 [vnlru] 18 0 0 0 DL syncer 0xffffffff81f9d568 [syncer] 17 0 0 0 DL (threaded) [bufdaemon] 100197 D psleep 0xffffffff81f9c570 [bufdaemon] 100198 D - 0xffffffff81a0b0c0 [bufspacedaemon-0] 100199 D - 0xffffffff81a13380 [bufspacedaemon-1] 100200 D - 0xffffffff81a1b640 [bufspacedaemon-2] 100201 D - 0xffffffff81a23900 [bufspacedaemon-3] 100202 D - 0xffffffff81a2bbc0 [bufspacedaemon-4] 100203 D - 0xffffffff81a33e80 [bufspacedaemon-5] 100204 D - 0xffffffff81a3c140 [bufspacedaemon-6] 100219 D sdflush 0xfffff80032488ce8 [/ worker] 100225 D sdflush 0xfffff80032724ce8 [/tmp worker] 100742 D sdflush 0xfffff80171a074e8 [/mnt worker] 16 0 0 0 RL [vmdaemon] 9 0 0 0 RL (threaded) [pagedaemon] 100195 Run CPU 11 [dom0] 100205 D swbufa 0xffffffff81fb5ca0 [laundry: dom0] 100206 D umarcls 0xffffffff8168181b [uma] 8 0 0 0 DL idle 0xfffffe010dde2ab8 [enc_daemon1] 7 0 0 0 DL idle 0xfffffe010dde3010 [enc_daemon0] 6 0 0 0 DL - 0xffffffff81ce0348 [rand_harvestq] 15 0 0 0 DL (threaded) [usb] 100128 D - 0xfffffe010a1a2440 [usbus0] 100129 D - 0xfffffe010a1a2498 [usbus0] 100130 D - 0xfffffe010a1a24f0 [usbus0] 100131 D - 0xfffffe010a1a2548 [usbus0] 100132 D - 0xfffffe010a1a25a0 [usbus0] 100134 D - 0xfffffe00e4d6bcf0 [usbus1] 100135 D - 0xfffffe00e4d6bd48 [usbus1] 100136 D - 0xfffffe00e4d6bda0 [usbus1] 100137 D - 0xfffffe00e4d6bdf8 [usbus1] 100138 D - 0xfffffe00e4d6be50 [usbus1] 100139 D - 0xfffffe010a562cf0 [usbus2] 100140 D - 0xfffffe010a562d48 [usbus2] 100141 D - 0xfffffe010a562da0 [usbus2] 100142 D - 0xfffffe010a562df8 [usbus2] 100143 D - 0xfffffe010a562e50 [usbus2] 5 0 0 0 DL - 0xffffffff81fcaf78 [busdma] 4 0 0 0 DL (threaded) [cam] 100120 D - 0xffffffff81bc9d40 [doneq0] 100121 D - 0xffffffff81bc9dc0 [doneq1] 100122 D - 0xffffffff81bc9e40 [doneq2] 100123 D - 0xffffffff81bc9cc0 [async] 100152 D - 0xffffffff81bc9b90 [scanner] 14 0 0 0 DL seqstat 0xfffff80003ba6488 [sequencer 00] 3 0 0 0 DL (threaded) [crypto] 100106 D crypto_ 0xffffffff81fb3e30 [crypto] 100107 D crypto_ 0xfffff8000444ac30 [crypto returns 0] 100108 D crypto_ 0xfffff8000444ac80 [crypto returns 1] 100109 D crypto_ 0xfffff8000444acd0 [crypto returns 2] 100110 D crypto_ 0xfffff8000444ad20 [crypto returns 3] 100111 D crypto_ 0xfffff8000444ad70 [crypto returns 4] 100112 D crypto_ 0xfffff8000444adc0 [crypto returns 5] 100113 D crypto_ 0xfffff8000444ae10 [crypto returns 6] 100114 D crypto_ 0xfffff8000444ae60 [crypto returns 7] 100115 D crypto_ 0xfffff8000444aeb0 [crypto returns 8] 100116 D crypto_ 0xfffff8000444af00 [crypto returns 9] 100117 D crypto_ 0xfffff8000444af50 [crypto returns 10] 100118 D crypto_ 0xfffff8000444afa0 [crypto returns 11] 13 0 0 0 DL (threaded) [geom] 100091 D - 0xffffffff81e8bed0 [g_event] 100092 D - 0xffffffff81e8bed8 [g_up] 100093 D - 0xffffffff81e8bee0 [g_down] 2 0 0 0 WL (threaded) [clock] 100077 I [clock (0)] 100078 I [clock (1)] 100079 I [clock (2)] 100080 I [clock (3)] 100081 I [clock (4)] 100082 I [clock (5)] 100083 I [clock (6)] 100084 I [clock (7)] 100085 I [clock (8)] 100086 I [clock (9)] 100087 I [clock (10)] 100088 I [clock (11)] 12 0 0 0 RL (threaded) [intr] 100041 I [swi6: task queue] 100045 I [swi6: Giant taskq] 100047 I [swi5: fast taskq] 100089 I [swi1: netisr 0] 100124 I [swi4: apei] 100125 RunQ [irq56: ahci0] 100127 I [irq19: xhci0] 100133 I [irq18: ehci0 ehci1+] 100144 I [irq71: ahci1] 100145 I [swi0: uart uart] 100159 I [irq73: mlx5_core0] 100160 I [irq74: mlx5_core0] 100161 I [irq72: mlx5_core0] 100162 I [irq75: mlx5_core0] 100163 I [irq76: mlx5_core0] 100164 I [irq77: mlx5_core0] 100165 I [irq78: mlx5_core0] 100166 I [irq79: mlx5_core0] 100167 I [irq80: mlx5_core0] 100168 I [irq81: mlx5_core0] 100169 I [irq82: mlx5_core0] 100170 I [irq83: mlx5_core0] 100171 I [irq84: mlx5_core0] 100172 I [irq85: mlx5_core0] 100173 I [irq86: mlx5_core0] 100179 I [irq88: mlx5_core1] 100180 I [irq89: mlx5_core1] 100181 I [irq87: mlx5_core1] 100182 I [irq90: mlx5_core1] 100183 I [irq91: mlx5_core1] 100184 I [irq92: mlx5_core1] 100185 I [irq93: mlx5_core1] 100186 I [irq94: mlx5_core1] 100187 I [irq95: mlx5_core1] 100188 I [irq96: mlx5_core1] 100189 I [irq97: mlx5_core1] 100190 I [irq98: mlx5_core1] 100191 I [irq99: mlx5_core1] 100192 I [irq100: mlx5_core1] 100193 I [irq101: mlx5_core1] 100249 I [irq102: ioat0] 100250 I [irq103: ioat1] 100251 I [irq104: ioat2] 100252 I [irq105: ioat3] 100253 I [irq106: ioat4] 100254 I [irq107: ioat5] 100255 I [irq108: ioat6] 100256 I [irq109: ioat7] 11 0 0 0 RL (threaded) [idle] 100003 CanRun [idle: cpu0] 100004 Run CPU 1 [idle: cpu1] 100005 Run CPU 2 [idle: cpu2] 100006 CanRun [idle: cpu3] 100007 CanRun [idle: cpu4] 100008 CanRun [idle: cpu5] 100009 CanRun [idle: cpu6] 100010 CanRun [idle: cpu7] 100011 CanRun [idle: cpu8] 100012 CanRun [idle: cpu9] 100013 CanRun [idle: cpu10] 100014 CanRun [idle: cpu11] 1 0 1 0 SLs wait 0xfffffe00e4c51010 [init] 10 0 0 0 DL audit_w 0xffffffff81fb43d8 [audit] 0 0 0 0 DLs (threaded) [kernel] 100000 D swapin 0xffffffff81e8c450 [swapper] 100015 D - 0xfffff80003bf2100 [if_config_tqg_0] 100016 D - 0xfffff80003bf2000 [softirq_0] 100017 D - 0xfffff80003bf3e00 [softirq_1] 100018 D - 0xfffff80003bf3d00 [softirq_2] 100019 D - 0xfffff80003bf3c00 [softirq_3] 100020 D - 0xfffff80003bf3b00 [softirq_4] 100021 D - 0xfffff80003bf3a00 [softirq_5] 100022 D - 0xfffff80003bf3900 [softirq_6] 100023 D - 0xfffff80003bf3800 [softirq_7] 100024 D - 0xfffff80003bf3700 [softirq_8] 100025 D - 0xfffff80003bf3600 [softirq_9] 100026 D - 0xfffff80003bf3500 [softirq_10] 100027 D - 0xfffff80003bf3400 [softirq_11] 100028 D - 0xfffff80003bf3300 [if_io_tqg_0] 100029 D - 0xfffff80003bf3200 [if_io_tqg_1] 100030 D - 0xfffff80003bf3100 [if_io_tqg_2] 100031 D - 0xfffff80003bf3000 [if_io_tqg_3] 100032 D - 0xfffff80003bf4e00 [if_io_tqg_4] 100033 D - 0xfffff80003bf4d00 [if_io_tqg_5] 100034 D - 0xfffff80003bf4c00 [if_io_tqg_6] 100035 D - 0xfffff80003bf4b00 [if_io_tqg_7] 100036 D - 0xfffff80003bf4a00 [if_io_tqg_8] 100037 D - 0xfffff80003bf4900 [if_io_tqg_9] 100038 D - 0xfffff80003bf4800 [if_io_tqg_10] 100039 D - 0xfffff80003bf4700 [if_io_tqg_11] 100040 D - 0xfffff80003a12e00 [kqueue_ctx taskq] 100042 D - 0xfffff80003a12c00 [inm_free taskq] 100043 D - 0xfffff80003a12b00 [aiod_kick taskq] 100044 D - 0xfffff80003a12a00 [deferred_unmount ta] 100046 D - 0xfffff80003a12800 [thread taskq] 100048 D - 0xfffff80003a12600 [pci_hp taskq] 100049 D - 0xfffff80003a12500 [in6m_free taskq] 100050 D - 0xfffff80003a12400 [linuxkpi_irq_wq] 100051 D - 0xfffff80003a12300 [linuxkpi_short_wq_0] 100052 D - 0xfffff80003a12300 [linuxkpi_short_wq_1] 100053 D - 0xfffff80003a12300 [linuxkpi_short_wq_2] 100054 D - 0xfffff80003a12300 [linuxkpi_short_wq_3] 100055 D - 0xfffff80003a12300 [linuxkpi_short_wq_4] 100056 D - 0xfffff80003a12300 [linuxkpi_short_wq_5] 100057 D - 0xfffff80003a12300 [linuxkpi_short_wq_6] 100058 D - 0xfffff80003a12300 [linuxkpi_short_wq_7] 100059 D - 0xfffff80003a12300 [linuxkpi_short_wq_8] 100060 D - 0xfffff80003a12300 [linuxkpi_short_wq_9] 100061 D - 0xfffff80003a12300 [linuxkpi_short_wq_1] 100062 D - 0xfffff80003a12300 [linuxkpi_short_wq_1] 100063 D - 0xfffff80003a12300 [linuxkpi_short_wq_1] 100064 D - 0xfffff80003a12200 [linuxkpi_long_wq_0] 100065 D - 0xfffff80003a12200 [linuxkpi_long_wq_1] 100066 D - 0xfffff80003a12200 [linuxkpi_long_wq_2] 100067 D - 0xfffff80003a12200 [linuxkpi_long_wq_3] 100068 D - 0xfffff80003a12200 [linuxkpi_long_wq_4] 100069 D - 0xfffff80003a12200 [linuxkpi_long_wq_5] 100070 D - 0xfffff80003a12200 [linuxkpi_long_wq_6] 100071 D - 0xfffff80003a12200 [linuxkpi_long_wq_7] 100072 D - 0xfffff80003a12200 [linuxkpi_long_wq_8] 100073 D - 0xfffff80003a12200 [linuxkpi_long_wq_9] 100074 D - 0xfffff80003a12200 [linuxkpi_long_wq_10] 100075 D - 0xfffff80003a12200 [linuxkpi_long_wq_11] 100076 D - 0xfffff80003a12200 [linuxkpi_long_wq_12] 100090 D - 0xfffff80003a12100 [firmware taskq] 100094 D - 0xfffff80003a11600 [crypto_0] 100095 D - 0xfffff80003a11600 [crypto_1] 100096 D - 0xfffff80003a11600 [crypto_2] 100097 D - 0xfffff80003a11600 [crypto_3] 100098 D - 0xfffff80003a11600 [crypto_4] 100099 D - 0xfffff80003a11600 [crypto_5] 100100 D - 0xfffff80003a11600 [crypto_6] 100101 D - 0xfffff80003a11600 [crypto_7] 100102 D - 0xfffff80003a11600 [crypto_8] 100103 D - 0xfffff80003a11600 [crypto_9] 100104 D - 0xfffff80003a11600 [crypto_10] 100105 D - 0xfffff80003a11600 [crypto_11] 100147 D - 0xfffff8002828d900 [acpi_task_0] 100148 D - 0xfffff8002828d900 [acpi_task_1] 100149 D - 0xfffff8002828d900 [acpi_task_2] 100150 D - 0xfffff80003a13100 [mca taskq] 100151 D - 0xfffff80003a10e00 [CAM taskq] 100155 D - 0xfffff8002828ec00 [mlx5_core0-rec] 100156 D - 0xfffff8002828eb00 [mlx5_core0-wdg] 100157 D - 0xfffff8002828ea00 [mlx5_core0-cmd] 100158 D - 0xfffff8002828e900 [mlx5_page_allocator] 100174 D - 0xfffff8002828e800 [mlx5_esw_wq] 100175 D - 0xfffff8002828e700 [mlx5_core1-rec] 100176 D - 0xfffff8002828e600 [mlx5_core1-wdg] 100177 D - 0xfffff8002828e500 [mlx5_core1-cmd] 100178 D - 0xfffff8002828e400 [mlx5_page_allocator] 100194 D - 0xfffff80003a10b00 [mlx5_esw_wq] db:0:pho> allt Tracing command kernel pid 0 tid 100000 td 0xffffffff81e8c9c0 sched_switch() at sched_switch+0x582/frame 0xffffffff82569e10 mi_switch() at mi_switch+0x157/frame 0xffffffff82569e30 sleepq_switch() at sleepq_switch+0x107/frame 0xffffffff82569e70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xffffffff82569eb0 _sleep() at _sleep+0x2d6/frame 0xffffffff82569f40 swapper() at swapper+0x75/frame 0xffffffff82569fa0 mi_startup() at mi_startup+0x30e/frame 0xffffffff82569ff0 btext() at btext+0x23 Tracing command kernel pid 0 tid 100015 td 0xfffffe00e4c57020 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4858de0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4858e00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4858e40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e4858ec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4858ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4858f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4858f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xfffffe00e4c56900 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4853de0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4853e00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4853e40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e4853ec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4853ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4853f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4853f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100017 td 0xfffffe00e4cdd720 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4862de0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4862e00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4862e40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e4862ec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4862ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4862f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4862f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100018 td 0xfffffe00e4cdd000 sched_switch() at sched_switch+0x582/frame 0xfffffe00e485dde0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e485de00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e485de40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e485dec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e485def0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e485df30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e485df30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xfffffe00e4c5e740 sched_switch() at sched_switch+0x582/frame 0xfffffe00e486cde0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e486ce00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e486ce40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e486cec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e486cef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e486cf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e486cf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xfffffe00e4c5e020 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4867de0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4867e00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4867e40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e4867ec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4867ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4867f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4867f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xfffffe00e4c5d900 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4876de0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4876e00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4876e40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e4876ec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4876ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4876f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4876f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xfffffe00e4c5d1e0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4871de0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4871e00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4871e40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e4871ec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4871ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4871f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4871f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xfffffe00e4c5cac0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4880de0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4880e00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4880e40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e4880ec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4880ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4880f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4880f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xfffffe00e4c5c3a0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e487bde0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e487be00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e487be40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e487bec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e487bef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e487bf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e487bf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xfffffe00e4ce1020 sched_switch() at sched_switch+0x582/frame 0xfffffe00e488ade0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e488ae00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e488ae40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e488aec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e488aef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e488af30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e488af30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100026 td 0xfffffe00e4ce0900 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4885de0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4885e00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4885e40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e4885ec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4885ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4885f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4885f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100027 td 0xfffffe00e4ce01e0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4894de0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4894e00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4894e40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e4894ec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4894ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4894f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4894f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100028 td 0xfffffe00e4cdfac0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e488fde0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e488fe00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e488fe40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e488fec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e488fef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e488ff30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e488ff30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100029 td 0xfffffe00e4cdf3a0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e489ede0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e489ee00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e489ee40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e489eec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e489eef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e489ef30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e489ef30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100030 td 0xfffffe00e4cdec80 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4899de0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4899e00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4899e40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e4899ec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4899ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4899f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4899f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100031 td 0xfffffe00e4cde560 sched_switch() at sched_switch+0x582/frame 0xfffffe00e48a8de0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e48a8e00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e48a8e40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e48a8ec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e48a8ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48a8f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48a8f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100032 td 0xfffffe00e4cdde40 sched_switch() at sched_switch+0x582/frame 0xfffffe00e48a3de0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e48a3e00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e48a3e40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e48a3ec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e48a3ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48a3f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48a3f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100033 td 0xfffffe00e4ceaac0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e48b2de0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e48b2e00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e48b2e40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e48b2ec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e48b2ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48b2f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48b2f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100034 td 0xfffffe00e4cea3a0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e48adde0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e48ade00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e48ade40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e48adec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e48adef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48adf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48adf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100035 td 0xfffffe00e4ce9c80 sched_switch() at sched_switch+0x582/frame 0xfffffe00e48bcde0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e48bce00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e48bce40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e48bcec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e48bcef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48bcf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48bcf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100036 td 0xfffffe00e4ce9560 sched_switch() at sched_switch+0x582/frame 0xfffffe00e48b7de0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e48b7e00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e48b7e40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e48b7ec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e48b7ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48b7f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48b7f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100037 td 0xfffffe00e4ce8e40 sched_switch() at sched_switch+0x582/frame 0xfffffe00e48c6de0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e48c6e00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e48c6e40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e48c6ec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e48c6ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48c6f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48c6f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100038 td 0xfffffe00e4ce8720 sched_switch() at sched_switch+0x582/frame 0xfffffe00e48c1de0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e48c1e00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e48c1e40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e48c1ec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e48c1ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48c1f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48c1f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100039 td 0xfffffe00e4ce8000 sched_switch() at sched_switch+0x582/frame 0xfffffe00e48d0de0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e48d0e00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e48d0e40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e48d0ec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e48d0ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48d0f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48d0f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100040 td 0xfffffe00e4ce1740 sched_switch() at sched_switch+0x582/frame 0xfffffe00e48cbdd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e48cbdf0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e48cbe30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e48cbec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48cbef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48cbf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48cbf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100042 td 0xfffffe00e4cede40 sched_switch() at sched_switch+0x582/frame 0xfffffe00e48d5dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e48d5df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e48d5e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e48d5ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48d5ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48d5f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48d5f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100043 td 0xfffffe00e4ced720 sched_switch() at sched_switch+0x582/frame 0xfffffe00e48e4dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e48e4df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e48e4e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e48e4ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48e4ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48e4f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48e4f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100044 td 0xfffffe00e4ced000 sched_switch() at sched_switch+0x582/frame 0xfffffe00e48dfdd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e48dfdf0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e48dfe30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e48dfec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48dfef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48dff30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48dff30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100046 td 0xfffffe00e4cec020 sched_switch() at sched_switch+0x582/frame 0xfffffe00e48e9dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e48e9df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e48e9e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e48e9ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48e9ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48e9f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48e9f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100048 td 0xfffffe00e4ceb1e0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e48f3dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e48f3df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e48f3e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e48f3ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48f3ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48f3f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48f3f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100049 td 0xfffffe00e4cf6000 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4902dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4902df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4902e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4902ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4902ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4902f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4902f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100050 td 0xfffffe00e4cf1740 sched_switch() at sched_switch+0x582/frame 0xfffffe00e48fdde0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e48fde00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e48fde40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e48fdec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe00e48fdef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48fdf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48fdf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100051 td 0xfffffe00e4cf1020 sched_switch() at sched_switch+0x582/frame 0xfffffe00e490cdd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e490cdf0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e490ce30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e490cec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e490cef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e490cf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e490cf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100052 td 0xfffffe00e4cf0900 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4907dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4907df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4907e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4907ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4907ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4907f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4907f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100053 td 0xfffffe00e4cf01e0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4916dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4916df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4916e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4916ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4916ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4916f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4916f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100054 td 0xfffffe00e4cefac0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4911dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4911df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4911e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4911ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4911ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4911f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4911f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100055 td 0xfffffe00e4cef3a0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4920dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4920df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4920e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4920ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4920ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4920f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4920f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100056 td 0xfffffe00e4ceec80 sched_switch() at sched_switch+0x582/frame 0xfffffe00e491bdd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e491bdf0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e491be30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e491bec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e491bef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e491bf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e491bf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100057 td 0xfffffe00e4cf9900 sched_switch() at sched_switch+0x582/frame 0xfffffe00e492add0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e492adf0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e492ae30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e492aec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e492aef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e492af30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e492af30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100058 td 0xfffffe00e4cf91e0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4925dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4925df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4925e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4925ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4925ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4925f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4925f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100059 td 0xfffffe00e4cf8ac0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4934dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4934df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4934e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4934ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4934ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4934f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4934f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100060 td 0xfffffe00e4cf83a0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e492fdd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e492fdf0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e492fe30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e492fec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e492fef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e492ff30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e492ff30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100061 td 0xfffffe00e4cf7c80 sched_switch() at sched_switch+0x582/frame 0xfffffe00e493edd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e493edf0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e493ee30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e493eec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e493eef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e493ef30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e493ef30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100062 td 0xfffffe00e4cf7560 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4939dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4939df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4939e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4939ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4939ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4939f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4939f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100063 td 0xfffffe00e4cf6e40 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4948dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4948df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4948e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4948ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4948ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4948f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4948f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100064 td 0xfffffe00e4cf6720 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4943dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4943df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4943e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4943ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4943ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4943f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4943f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100065 td 0xfffffe00e4cff3a0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4952dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4952df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4952e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4952ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4952ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4952f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4952f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100066 td 0xfffffe00e4cfec80 sched_switch() at sched_switch+0x582/frame 0xfffffe00e494ddd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e494ddf0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e494de30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e494dec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e494def0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e494df30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e494df30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100067 td 0xfffffe00e4cfe560 sched_switch() at sched_switch+0x582/frame 0xfffffe00e495cdd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e495cdf0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e495ce30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e495cec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e495cef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e495cf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e495cf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100068 td 0xfffffe00e4cfde40 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4957dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4957df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4957e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4957ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4957ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4957f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4957f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100069 td 0xfffffe00e4cfd720 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4966dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4966df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4966e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4966ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4966ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4966f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4966f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100070 td 0xfffffe00e4cfd000 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4961dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4961df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4961e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4961ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4961ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4961f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4961f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100071 td 0xfffffe00e4cfa740 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4970dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4970df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4970e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4970ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4970ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4970f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4970f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100072 td 0xfffffe00e4cfa020 sched_switch() at sched_switch+0x582/frame 0xfffffe00e496bdd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e496bdf0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e496be30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e496bec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e496bef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e496bf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e496bf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100073 td 0xfffffe00e4d02e40 sched_switch() at sched_switch+0x582/frame 0xfffffe00e497add0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e497adf0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e497ae30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e497aec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e497aef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e497af30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e497af30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100074 td 0xfffffe00e4d02720 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4975dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4975df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4975e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4975ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4975ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4975f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4975f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100075 td 0xfffffe00e4d02000 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4984dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4984df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4984e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4984ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4984ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4984f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4984f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100076 td 0xfffffe00e4d01740 sched_switch() at sched_switch+0x582/frame 0xfffffe00e497fdd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e497fdf0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e497fe30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e497fec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e497fef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e497ff30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e497ff30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100090 td 0xfffffe00e4d0eac0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e49f5dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e49f5df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e49f5e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e49f5ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e49f5ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49f5f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49f5f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100094 td 0xfffffe00e4d0ce40 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4e39dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4e39df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4e39e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4e39ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e39ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e39f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e39f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100095 td 0xfffffe00e4d0c720 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4e48dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4e48df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4e48e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4e48ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e48ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e48f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e48f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100096 td 0xfffffe00e4d0c000 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4e43dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4e43df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4e43e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4e43ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e43ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e43f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e43f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100097 td 0xfffffe00e4d27c80 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4e52dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4e52df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4e52e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4e52ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e52ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e52f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e52f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100098 td 0xfffffe00e4d27560 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4e4ddd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4e4ddf0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4e4de30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4e4dec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e4def0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e4df30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e4df30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100099 td 0xfffffe00e4d26e40 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4e5cdd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4e5cdf0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4e5ce30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4e5cec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e5cef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e5cf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e5cf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100100 td 0xfffffe00e4d26720 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4e57dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4e57df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4e57e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4e57ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e57ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e57f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e57f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100101 td 0xfffffe00e4d26000 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4e66dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4e66df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4e66e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4e66ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e66ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e66f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e66f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100102 td 0xfffffe00e4d10740 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4e61dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4e61df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4e61e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4e61ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e61ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e61f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e61f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100103 td 0xfffffe00e4d10020 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4e70dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4e70df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4e70e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4e70ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e70ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e70f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e70f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100104 td 0xfffffe00e4d0f900 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4e6bdd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4e6bdf0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4e6be30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4e6bec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e6bef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e6bf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e6bf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100105 td 0xfffffe00e4d2b720 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4e7add0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4e7adf0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4e7ae30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4e7aec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e7aef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e7af30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e7af30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100147 td 0xfffffe010dc29c80 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4ffade0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4ffae00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4ffae40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e4ffaec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe00e4ffaef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ffaf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ffaf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100148 td 0xfffffe010dc29560 sched_switch() at sched_switch+0x582/frame 0xfffffe010da45de0 mi_switch() at mi_switch+0x157/frame 0xfffffe010da45e00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe010da45e40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe010da45ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe010da45ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe010da45f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010da45f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100149 td 0xfffffe010dc28e40 sched_switch() at sched_switch+0x582/frame 0xfffffe010da40de0 mi_switch() at mi_switch+0x157/frame 0xfffffe010da40e00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe010da40e40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe010da40ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe010da40ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe010da40f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010da40f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100150 td 0xfffffe010dc28720 sched_switch() at sched_switch+0x582/frame 0xfffffe010da4fde0 mi_switch() at mi_switch+0x157/frame 0xfffffe010da4fe00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe010da4fe40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe010da4fec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe010da4fef0 fork_exit() at fork_exit+0x80/frame 0xfffffe010da4ff30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010da4ff30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100151 td 0xfffffe010dc28000 sched_switch() at sched_switch+0x582/frame 0xfffffe010da4add0 mi_switch() at mi_switch+0x157/frame 0xfffffe010da4adf0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe010da4ae30 _sleep() at _sleep+0x2ef/frame 0xfffffe010da4aec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe010da4aef0 fork_exit() at fork_exit+0x80/frame 0xfffffe010da4af30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010da4af30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100155 td 0xfffffe010a1a8020 sched_switch() at sched_switch+0x582/frame 0xfffffe010da54dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe010da54df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe010da54e30 _sleep() at _sleep+0x2ef/frame 0xfffffe010da54ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe010da54ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe010da54f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010da54f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100156 td 0xfffffe010dde9740 sched_switch() at sched_switch+0x582/frame 0xfffffe010da6ddd0 mi_switch() at mi_switch+0x157/frame 0xfffffe010da6ddf0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe010da6de30 _sleep() at _sleep+0x2ef/frame 0xfffffe010da6dec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe010da6def0 fork_exit() at fork_exit+0x80/frame 0xfffffe010da6df30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010da6df30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100157 td 0xfffffe010dde9020 sched_switch() at sched_switch+0x582/frame 0xfffffe010da68dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe010da68df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe010da68e30 _sleep() at _sleep+0x2ef/frame 0xfffffe010da68ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe010da68ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe010da68f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010da68f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100158 td 0xfffffe010dde8900 sched_switch() at sched_switch+0x582/frame 0xfffffe010da77dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe010da77df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe010da77e30 _sleep() at _sleep+0x2ef/frame 0xfffffe010da77ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe010da77ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe010da77f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010da77f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100174 td 0xfffffe010e0d1e40 sched_switch() at sched_switch+0x582/frame 0xfffffe010dac7dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe010dac7df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe010dac7e30 _sleep() at _sleep+0x2ef/frame 0xfffffe010dac7ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe010dac7ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe010dac7f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010dac7f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100175 td 0xfffffe010e0d1720 sched_switch() at sched_switch+0x582/frame 0xfffffe010dac2dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe010dac2df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe010dac2e30 _sleep() at _sleep+0x2ef/frame 0xfffffe010dac2ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe010dac2ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe010dac2f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010dac2f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100176 td 0xfffffe010e0d1000 sched_switch() at sched_switch+0x582/frame 0xfffffe010dad1dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe010dad1df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe010dad1e30 _sleep() at _sleep+0x2ef/frame 0xfffffe010dad1ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe010dad1ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe010dad1f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010dad1f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100177 td 0xfffffe010dfcc740 sched_switch() at sched_switch+0x582/frame 0xfffffe010daccdd0 mi_switch() at mi_switch+0x157/frame 0xfffffe010daccdf0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe010dacce30 _sleep() at _sleep+0x2ef/frame 0xfffffe010daccec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe010daccef0 fork_exit() at fork_exit+0x80/frame 0xfffffe010daccf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010daccf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100178 td 0xfffffe010dfcc020 sched_switch() at sched_switch+0x582/frame 0xfffffe010dadbdd0 mi_switch() at mi_switch+0x157/frame 0xfffffe010dadbdf0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe010dadbe30 _sleep() at _sleep+0x2ef/frame 0xfffffe010dadbec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe010dadbef0 fork_exit() at fork_exit+0x80/frame 0xfffffe010dadbf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010dadbf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100194 td 0xfffffe010e246900 sched_switch() at sched_switch+0x582/frame 0xfffffe010db1cdd0 mi_switch() at mi_switch+0x157/frame 0xfffffe010db1cdf0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe010db1ce30 _sleep() at _sleep+0x2ef/frame 0xfffffe010db1cec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe010db1cef0 fork_exit() at fork_exit+0x80/frame 0xfffffe010db1cf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010db1cf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command init pid 1 tid 100002 td 0xfffffe00e4c55ac0 sched_switch() at sched_switch+0x582/frame 0xfffffe00377e1a20 mi_switch() at mi_switch+0x157/frame 0xfffffe00377e1a40 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00377e1a80 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe00377e1ad0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00377e1ae0 _sleep() at _sleep+0x2e5/frame 0xfffffe00377e1b70 kern_wait6() at kern_wait6+0x474/frame 0xfffffe00377e1c00 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe00377e1e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe00377e1f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00377e1f30 --- syscall (7, FreeBSD ELF64, wait4), rip = 0x28e87a, rsp = 0x821211d48, rbp = 0x821211db0 --- Tracing command clock pid 2 tid 100077 td 0xfffffe00e4d01020 sched_switch() at sched_switch+0x582/frame 0xfffffe00e49beea0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e49beec0 softclock_thread() at softclock_thread+0x6e/frame 0xfffffe00e49beef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49bef30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49bef30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command clock pid 2 tid 100078 td 0xfffffe00e4d00900 sched_switch() at sched_switch+0x582/frame 0xfffffe00e49b9ea0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e49b9ec0 softclock_thread() at softclock_thread+0x6e/frame 0xfffffe00e49b9ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49b9f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49b9f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command clock pid 2 tid 100079 td 0xfffffe00e4d001e0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e49c8ea0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e49c8ec0 softclock_thread() at softclock_thread+0x6e/frame 0xfffffe00e49c8ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49c8f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49c8f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command clock pid 2 tid 100080 td 0xfffffe00e4cffac0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e49c3ea0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e49c3ec0 softclock_thread() at softclock_thread+0x6e/frame 0xfffffe00e49c3ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49c3f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49c3f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command clock pid 2 tid 100081 td 0xfffffe00e4d06740 sched_switch() at sched_switch+0x582/frame 0xfffffe00e49d2ea0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e49d2ec0 softclock_thread() at softclock_thread+0x6e/frame 0xfffffe00e49d2ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49d2f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49d2f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command clock pid 2 tid 100082 td 0xfffffe00e4d06020 sched_switch() at sched_switch+0x582/frame 0xfffffe00e49cdea0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e49cdec0 softclock_thread() at softclock_thread+0x6e/frame 0xfffffe00e49cdef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49cdf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49cdf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command clock pid 2 tid 100083 td 0xfffffe00e4d05900 fork_trampoline() at fork_trampoline Tracing command clock pid 2 tid 100084 td 0xfffffe00e4d051e0 fork_trampoline() at fork_trampoline Tracing command clock pid 2 tid 100085 td 0xfffffe00e4d04ac0 fork_trampoline() at fork_trampoline Tracing command clock pid 2 tid 100086 td 0xfffffe00e4d043a0 fork_trampoline() at fork_trampoline Tracing command clock pid 2 tid 100087 td 0xfffffe00e4d03c80 fork_trampoline() at fork_trampoline Tracing command clock pid 2 tid 100088 td 0xfffffe00e4d03560 fork_trampoline() at fork_trampoline Tracing command crypto pid 3 tid 100106 td 0xfffffe00e4d2b000 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4e75dc0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4e75de0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4e75e20 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4e75eb0 crypto_dispatch_thread() at crypto_dispatch_thread+0xbf/frame 0xfffffe00e4e75ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e75f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e75f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 3 tid 100107 td 0xfffffe00e4d2a740 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4e84dc0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4e84de0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4e84e20 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4e84eb0 crypto_ret_thread() at crypto_ret_thread+0x134/frame 0xfffffe00e4e84ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e84f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e84f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 3 tid 100108 td 0xfffffe00e4d2a020 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4e7fdc0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4e7fde0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4e7fe20 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4e7feb0 crypto_ret_thread() at crypto_ret_thread+0x134/frame 0xfffffe00e4e7fef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e7ff30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e7ff30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 3 tid 100109 td 0xfffffe00e4d29900 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4e8edc0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4e8ede0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4e8ee20 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4e8eeb0 crypto_ret_thread() at crypto_ret_thread+0x134/frame 0xfffffe00e4e8eef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e8ef30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e8ef30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 3 tid 100110 td 0xfffffe00e4d291e0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4e89dc0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4e89de0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4e89e20 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4e89eb0 crypto_ret_thread() at crypto_ret_thread+0x134/frame 0xfffffe00e4e89ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e89f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e89f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 3 tid 100111 td 0xfffffe00e4d28ac0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4e98dc0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4e98de0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4e98e20 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4e98eb0 crypto_ret_thread() at crypto_ret_thread+0x134/frame 0xfffffe00e4e98ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e98f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e98f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 3 tid 100112 td 0xfffffe00e4d283a0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4e93dc0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4e93de0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4e93e20 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4e93eb0 crypto_ret_thread() at crypto_ret_thread+0x134/frame 0xfffffe00e4e93ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e93f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e93f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 3 tid 100113 td 0xfffffe00e4d2f020 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4ea2dc0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4ea2de0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4ea2e20 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4ea2eb0 crypto_ret_thread() at crypto_ret_thread+0x134/frame 0xfffffe00e4ea2ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ea2f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ea2f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 3 tid 100114 td 0xfffffe00e4d2e900 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4e9ddc0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4e9dde0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4e9de20 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4e9deb0 crypto_ret_thread() at crypto_ret_thread+0x134/frame 0xfffffe00e4e9def0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e9df30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e9df30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 3 tid 100115 td 0xfffffe00e4d2e1e0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4eacdc0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4eacde0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4eace20 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4eaceb0 crypto_ret_thread() at crypto_ret_thread+0x134/frame 0xfffffe00e4eacef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4eacf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4eacf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 3 tid 100116 td 0xfffffe00e4d2dac0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4ea7dc0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4ea7de0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4ea7e20 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4ea7eb0 crypto_ret_thread() at crypto_ret_thread+0x134/frame 0xfffffe00e4ea7ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ea7f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ea7f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 3 tid 100117 td 0xfffffe00e4d2d3a0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4eb6dc0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4eb6de0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4eb6e20 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4eb6eb0 crypto_ret_thread() at crypto_ret_thread+0x134/frame 0xfffffe00e4eb6ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4eb6f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4eb6f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 3 tid 100118 td 0xfffffe00e4d2cc80 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4eb1dc0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4eb1de0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4eb1e20 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4eb1eb0 crypto_ret_thread() at crypto_ret_thread+0x134/frame 0xfffffe00e4eb1ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4eb1f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4eb1f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 4 tid 100120 td 0xfffffe00e4d2be40 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4ebbdb0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4ebbdd0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4ebbe10 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4ebbea0 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe00e4ebbef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ebbf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ebbf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 4 tid 100121 td 0xfffffe00e4d42ac0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4ecadb0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4ecadd0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4ecae10 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4ecaea0 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe00e4ecaef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ecaf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ecaf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 4 tid 100122 td 0xfffffe00e4d423a0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4ec5db0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4ec5dd0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4ec5e10 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4ec5ea0 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe00e4ec5ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ec5f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ec5f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 4 tid 100123 td 0xfffffe00e4d41c80 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4ed4db0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4ed4dd0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4ed4e10 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4ed4ea0 xpt_async_td() at xpt_async_td+0x74/frame 0xfffffe00e4ed4ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ed4f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ed4f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 4 tid 100152 td 0xfffffe010a1a8740 sched_switch() at sched_switch+0x582/frame 0xfffffe010da59dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe010da59df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe010da59e30 _sleep() at _sleep+0x2ef/frame 0xfffffe010da59ec0 xpt_scanner_thread() at xpt_scanner_thread+0x154/frame 0xfffffe010da59ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe010da59f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010da59f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command busdma pid 5 tid 100126 td 0xfffffe00e4d7d720 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4ee8dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4ee8df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4ee8e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4ee8ec0 busdma_thread() at busdma_thread+0xb5/frame 0xfffffe00e4ee8ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ee8f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ee8f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command rand_harvestq pid 6 tid 100146 td 0xfffffe010dc2a3a0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4fffd30 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4fffd50 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4fffd90 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e4fffdd0 _sleep() at _sleep+0x2d6/frame 0xfffffe00e4fffe60 random_kthread() at random_kthread+0xa7/frame 0xfffffe00e4fffef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ffff30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ffff30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon0 pid 7 tid 100153 td 0xfffffe010dde5e40 sched_switch() at sched_switch+0x582/frame 0xfffffe010da63d90 mi_switch() at mi_switch+0x157/frame 0xfffffe010da63db0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe010da63df0 _sleep() at _sleep+0x2ef/frame 0xfffffe010da63e80 enc_daemon() at enc_daemon+0xde/frame 0xfffffe010da63ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe010da63f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010da63f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon1 pid 8 tid 100154 td 0xfffffe010dde5720 sched_switch() at sched_switch+0x582/frame 0xfffffe010da5ed90 mi_switch() at mi_switch+0x157/frame 0xfffffe010da5edb0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe010da5edf0 _sleep() at _sleep+0x2ef/frame 0xfffffe010da5ee80 enc_daemon() at enc_daemon+0xde/frame 0xfffffe010da5eef0 fork_exit() at fork_exit+0x80/frame 0xfffffe010da5ef30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010da5ef30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 9 tid 100195 td 0xfffffe010e3a6ac0 (CPU 11) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003cea2df0 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003cea2e00 trap() at trap+0x57/frame 0xfffffe003cea2f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003cea2f20 --- trap 0x13, rip = 0xffffffff80f9448b, rsp = 0xfffffe010db43c00, rbp = 0xfffffe010db43eb0 --- vm_pageout_worker() at vm_pageout_worker+0x9ab/frame 0xfffffe010db43eb0 vm_pageout() at vm_pageout+0x1d7/frame 0xfffffe010db43ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe010db43f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010db43f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 9 tid 100205 td 0xfffffe0113238000 sched_switch() at sched_switch+0x582/frame 0xfffffe0118156770 mi_switch() at mi_switch+0x157/frame 0xfffffe0118156790 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe01181567d0 _sleep() at _sleep+0x2ef/frame 0xfffffe0118156860 swap_pager_putpages() at swap_pager_putpages+0x176/frame 0xfffffe0118156900 vm_pageout_flush() at vm_pageout_flush+0xf6/frame 0xfffffe0118156a00 vm_pageout_cluster() at vm_pageout_cluster+0x34b/frame 0xfffffe0118156c70 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0x749/frame 0xfffffe0118156ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe0118156f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0118156f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 9 tid 100206 td 0xfffffe010e3c7740 sched_switch() at sched_switch+0x582/frame 0xfffffe0117f4fd60 mi_switch() at mi_switch+0x157/frame 0xfffffe0117f4fd80 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0117f4fdc0 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0117f4fe00 _sleep() at _sleep+0x2d6/frame 0xfffffe0117f4fe90 pause_sbt() at pause_sbt+0xff/frame 0xfffffe0117f4fec0 uma_reclaim_worker() at uma_reclaim_worker+0x61/frame 0xfffffe0117f4fef0 fork_exit() at fork_exit+0x80/frame 0xfffffe0117f4ff30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0117f4ff30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command audit pid 10 tid 100001 td 0xfffffe00e4c561e0 sched_switch() at sched_switch+0x582/frame 0xfffffe00377e6dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00377e6df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00377e6e30 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00377e6e90 audit_worker() at audit_worker+0x9f/frame 0xfffffe00377e6ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377e6f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377e6f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100003 td 0xfffffe00e4c553a0 sched_switch() at sched_switch+0x582/frame 0xfffffe00377f0e30 mi_switch() at mi_switch+0x157/frame 0xfffffe00377f0e50 sched_idletd() at sched_idletd+0x4b1/frame 0xfffffe00377f0ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377f0f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377f0f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xfffffe00e4c54c80 (CPU 1) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce20df0 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce20e00 trap() at trap+0x57/frame 0xfffffe003ce20f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce20f20 --- trap 0x13, rip = 0xffffffff810965a6, rsp = 0xfffffe00377ebdd0, rbp = 0xfffffe00377ebdd0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00377ebdd0 acpi_cpu_idle() at acpi_cpu_idle+0x278/frame 0xfffffe00377ebe10 cpu_idle_acpi() at cpu_idle_acpi+0x4d/frame 0xfffffe00377ebe30 cpu_idle() at cpu_idle+0xac/frame 0xfffffe00377ebe50 sched_idletd() at sched_idletd+0x4b1/frame 0xfffffe00377ebef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377ebf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377ebf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffffe00e4c54560 (CPU 2) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce2ddf0 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce2de00 trap() at trap+0x57/frame 0xfffffe003ce2df20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce2df20 --- trap 0x13, rip = 0xffffffff810965a6, rsp = 0xfffffe00377fadd0, rbp = 0xfffffe00377fadd0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00377fadd0 acpi_cpu_idle() at acpi_cpu_idle+0x278/frame 0xfffffe00377fae10 cpu_idle_acpi() at cpu_idle_acpi+0x4d/frame 0xfffffe00377fae30 cpu_idle() at cpu_idle+0xac/frame 0xfffffe00377fae50 sched_idletd() at sched_idletd+0x4b1/frame 0xfffffe00377faef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377faf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377faf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xfffffe00e4c53e40 sched_switch() at sched_switch+0x582/frame 0xfffffe00377f5e30 mi_switch() at mi_switch+0x157/frame 0xfffffe00377f5e50 sched_idletd() at sched_idletd+0xa1/frame 0xfffffe00377f5ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377f5f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377f5f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100007 td 0xfffffe00e4c53720 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4830e30 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4830e50 sched_idletd() at sched_idletd+0xa1/frame 0xfffffe00e4830ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4830f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4830f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xfffffe00e4c53000 sched_switch() at sched_switch+0x582/frame 0xfffffe00377ffe30 mi_switch() at mi_switch+0x157/frame 0xfffffe00377ffe50 sched_idletd() at sched_idletd+0xa1/frame 0xfffffe00377ffef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377fff30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377fff30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xfffffe00e4c5bc80 sched_switch() at sched_switch+0x582/frame 0xfffffe00e483ae30 mi_switch() at mi_switch+0x157/frame 0xfffffe00e483ae50 sched_idletd() at sched_idletd+0x4b1/frame 0xfffffe00e483aef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e483af30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e483af30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xfffffe00e4c5b560 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4835e30 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4835e50 sched_idletd() at sched_idletd+0xa1/frame 0xfffffe00e4835ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4835f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4835f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffffe00e4c5ae40 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce7bdf0 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce7be00 trap() at trap+0x57/frame 0xfffffe003ce7bf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce7bf20 --- trap 0x13, rip = 0xffffffff80bfac69, rsp = 0xfffffe00e4844d30, rbp = 0xfffffe00e4844d30 --- runq_choose() at runq_choose+0x19/frame 0xfffffe00e4844d30 tdq_choose() at tdq_choose+0x2c/frame 0xfffffe00e4844d50 sched_choose() at sched_choose+0x32/frame 0xfffffe00e4844d70 choosethread() at choosethread+0x9/frame 0xfffffe00e4844d80 sched_switch() at sched_switch+0x494/frame 0xfffffe00e4844e30 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4844e50 sched_idletd() at sched_idletd+0x4b1/frame 0xfffffe00e4844ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4844f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4844f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100012 td 0xfffffe00e4c5a720 sched_switch() at sched_switch+0x582/frame 0xfffffe00e483fe30 mi_switch() at mi_switch+0x157/frame 0xfffffe00e483fe50 sched_idletd() at sched_idletd+0x4f9/frame 0xfffffe00e483fef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e483ff30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e483ff30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xfffffe00e4c5a000 sched_switch() at sched_switch+0x582/frame 0xfffffe00e484ee30 mi_switch() at mi_switch+0x157/frame 0xfffffe00e484ee50 sched_idletd() at sched_idletd+0xa1/frame 0xfffffe00e484eef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e484ef30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e484ef30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100014 td 0xfffffe00e4c57740 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4849e30 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4849e50 sched_idletd() at sched_idletd+0x4b1/frame 0xfffffe00e4849ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4849f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4849f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100041 td 0xfffffe00e4cee560 sched_switch() at sched_switch+0x582/frame 0xfffffe00e48dae40 mi_switch() at mi_switch+0x157/frame 0xfffffe00e48dae60 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e48daef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48daf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48daf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100045 td 0xfffffe00e4cec740 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100047 td 0xfffffe00e4ceb900 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100089 td 0xfffffe00e4d0f1e0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e49fae40 mi_switch() at mi_switch+0x157/frame 0xfffffe00e49fae60 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e49faef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49faf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49faf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100124 td 0xfffffe00e4d7e560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100125 td 0xfffffe00e4d7de40 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4ed9e40 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4ed9e60 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4ed9ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ed9f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ed9f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100127 td 0xfffffe00e4d7d000 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4ee3e40 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4ee3e60 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4ee3ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ee3f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ee3f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100133 td 0xfffffe00e4d81740 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4f01e40 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4f01e60 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4f01ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f01f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f01f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100144 td 0xfffffe010a1a5560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100145 td 0xfffffe010a1a4e40 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4f64e40 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4f64e60 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4f64ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f64f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f64f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100159 td 0xfffffe010dde81e0 sched_switch() at sched_switch+0x582/frame 0xfffffe010da72e40 mi_switch() at mi_switch+0x157/frame 0xfffffe010da72e60 ithread_loop() at ithread_loop+0x99/frame 0xfffffe010da72ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe010da72f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010da72f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100160 td 0xfffffe010dde7ac0 sched_switch() at sched_switch+0x582/frame 0xfffffe010da81e40 mi_switch() at mi_switch+0x157/frame 0xfffffe010da81e60 ithread_loop() at ithread_loop+0x99/frame 0xfffffe010da81ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe010da81f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010da81f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100161 td 0xfffffe010dde73a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100162 td 0xfffffe010dde6c80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100163 td 0xfffffe010dde6560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100164 td 0xfffffe010dfcb1e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100165 td 0xfffffe010dfcaac0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100166 td 0xfffffe010dfca3a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100167 td 0xfffffe010dfc9c80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100168 td 0xfffffe010dfc9560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100169 td 0xfffffe010dfc8e40 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100170 td 0xfffffe010dfc8720 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100171 td 0xfffffe010dfc8000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100172 td 0xfffffe010e0d2c80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100173 td 0xfffffe010e0d2560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100179 td 0xfffffe010dfcb900 sched_switch() at sched_switch+0x582/frame 0xfffffe010dad6e40 mi_switch() at mi_switch+0x157/frame 0xfffffe010dad6e60 ithread_loop() at ithread_loop+0x99/frame 0xfffffe010dad6ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe010dad6f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010dad6f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100180 td 0xfffffe00e4d41560 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4ecfe40 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4ecfe60 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4ecfef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ecff30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ecff30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100181 td 0xfffffe00e4d40e40 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100182 td 0xfffffe00e4d40720 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100183 td 0xfffffe00e4d40000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100184 td 0xfffffe00e4d2f740 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100185 td 0xfffffe010e243720 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100186 td 0xfffffe010e243000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100187 td 0xfffffe010e0d5740 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100188 td 0xfffffe010e0d5020 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100189 td 0xfffffe010e0d4900 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100190 td 0xfffffe010e0d41e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100191 td 0xfffffe010e0d3ac0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100192 td 0xfffffe010e0d33a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100193 td 0xfffffe010e247020 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100249 td 0xfffffe0141e43000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100250 td 0xfffffe013ba77740 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100251 td 0xfffffe013ba77020 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100252 td 0xfffffe013ba76900 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100253 td 0xfffffe013ba761e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100254 td 0xfffffe013ba75ac0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100255 td 0xfffffe0143f5dac0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100256 td 0xfffffe0143f5d3a0 fork_trampoline() at fork_trampoline Tracing command geom pid 13 tid 100091 td 0xfffffe00e4d0e3a0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4e34dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4e34df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4e34e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4e34ec0 g_run_events() at g_run_events+0x444/frame 0xfffffe00e4e34ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e34f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e34f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100092 td 0xfffffe00e4d0dc80 sched_switch() at sched_switch+0x582/frame 0xfffffe00e49ffdc0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e49ffde0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e49ffe20 _sleep() at _sleep+0x2ef/frame 0xfffffe00e49ffeb0 g_io_schedule_up() at g_io_schedule_up+0x3c/frame 0xfffffe00e49ffee0 g_up_procbody() at g_up_procbody+0x5c/frame 0xfffffe00e49ffef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49fff30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49fff30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100093 td 0xfffffe00e4d0d560 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4e3edc0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4e3ede0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4e3ee20 _sleep() at _sleep+0x2ef/frame 0xfffffe00e4e3eeb0 g_io_schedule_down() at g_io_schedule_down+0x4c/frame 0xfffffe00e4e3eee0 g_down_procbody() at g_down_procbody+0x5c/frame 0xfffffe00e4e3eef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e3ef30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e3ef30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sequencer 00 pid 14 tid 100119 td 0xfffffe00e4d2c560 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4ec0db0 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4ec0dd0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4ec0e10 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4ec0e70 seq_eventthread() at seq_eventthread+0x8c/frame 0xfffffe00e4ec0ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ec0f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ec0f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 15 tid 100128 td 0xfffffe00e4d44740 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4ef2e00 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4ef2e20 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4ef2e60 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4ef2ec0 usb_process() at usb_process+0xb3/frame 0xfffffe00e4ef2ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ef2f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ef2f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 15 tid 100129 td 0xfffffe00e4d44020 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4eede00 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4eede20 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4eede60 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4eedec0 usb_process() at usb_process+0xb3/frame 0xfffffe00e4eedef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4eedf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4eedf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 15 tid 100130 td 0xfffffe00e4d43900 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4efce00 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4efce20 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4efce60 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4efcec0 usb_process() at usb_process+0xb3/frame 0xfffffe00e4efcef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4efcf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4efcf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 15 tid 100131 td 0xfffffe00e4d431e0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4ef7e00 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4ef7e20 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4ef7e60 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4ef7ec0 usb_process() at usb_process+0xb3/frame 0xfffffe00e4ef7ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ef7f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ef7f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 15 tid 100132 td 0xfffffe010a1a4000 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4f06e00 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4f06e20 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4f06e60 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4f06ec0 usb_process() at usb_process+0xb3/frame 0xfffffe00e4f06ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f06f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f06f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 15 tid 100134 td 0xfffffe00e4d81020 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4f10e00 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4f10e20 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4f10e60 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4f10ec0 usb_process() at usb_process+0xb3/frame 0xfffffe00e4f10ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f10f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f10f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 15 tid 100135 td 0xfffffe00e4d80900 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4f0be00 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4f0be20 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4f0be60 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4f0bec0 usb_process() at usb_process+0xb3/frame 0xfffffe00e4f0bef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f0bf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f0bf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 15 tid 100136 td 0xfffffe00e4d801e0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4f1ae00 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4f1ae20 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4f1ae60 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4f1aec0 usb_process() at usb_process+0xb3/frame 0xfffffe00e4f1aef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f1af30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f1af30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 15 tid 100137 td 0xfffffe00e4d7fac0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4f15e00 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4f15e20 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4f15e60 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4f15ec0 usb_process() at usb_process+0xb3/frame 0xfffffe00e4f15ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f15f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f15f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 15 tid 100138 td 0xfffffe00e4d7f3a0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4f24e00 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4f24e20 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4f24e60 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4f24ec0 usb_process() at usb_process+0xb3/frame 0xfffffe00e4f24ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f24f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f24f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 15 tid 100139 td 0xfffffe010a1a7900 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4f46e00 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4f46e20 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4f46e60 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4f46ec0 usb_process() at usb_process+0xb3/frame 0xfffffe00e4f46ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f46f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f46f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 15 tid 100140 td 0xfffffe010a1a71e0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4f41e00 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4f41e20 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4f41e60 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4f41ec0 usb_process() at usb_process+0xb3/frame 0xfffffe00e4f41ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f41f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f41f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 15 tid 100141 td 0xfffffe010a1a6ac0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4f50e00 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4f50e20 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4f50e60 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4f50ec0 usb_process() at usb_process+0xb3/frame 0xfffffe00e4f50ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f50f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f50f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 15 tid 100142 td 0xfffffe010a1a63a0 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4f4be00 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4f4be20 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4f4be60 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4f4bec0 usb_process() at usb_process+0xb3/frame 0xfffffe00e4f4bef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f4bf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f4bf30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 15 tid 100143 td 0xfffffe010a1a5c80 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4f5ae00 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4f5ae20 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4f5ae60 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4f5aec0 usb_process() at usb_process+0xb3/frame 0xfffffe00e4f5aef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f5af30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f5af30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command vmdaemon pid 16 tid 100196 td 0xfffffe010e3a63a0 sched_switch() at sched_switch+0x582/frame 0xfffffe010db32de0 mi_switch() at mi_switch+0x157/frame 0xfffffe010db32e00 sched_bind() at sched_bind+0xbd/frame 0xfffffe010db32e20 vm_page_pqbatch_drain() at vm_page_pqbatch_drain+0x80/frame 0xfffffe010db32e80 vm_daemon() at vm_daemon+0xbe/frame 0xfffffe010db32ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe010db32f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010db32f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 17 tid 100197 td 0xfffffe010e3a5c80 sched_switch() at sched_switch+0x582/frame 0xfffffe010db4dd70 mi_switch() at mi_switch+0x157/frame 0xfffffe010db4dd90 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe010db4ddd0 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe010db4de10 _sleep() at _sleep+0x2d6/frame 0xfffffe010db4dea0 buf_daemon() at buf_daemon+0x121/frame 0xfffffe010db4def0 fork_exit() at fork_exit+0x80/frame 0xfffffe010db4df30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010db4df30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 17 tid 100198 td 0xfffffe010e3c4560 sched_switch() at sched_switch+0x582/frame 0xfffffe0110525d50 mi_switch() at mi_switch+0x157/frame 0xfffffe0110525d70 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0110525db0 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0110525df0 _sleep() at _sleep+0x2d6/frame 0xfffffe0110525e80 bufspace_daemon() at bufspace_daemon+0x312/frame 0xfffffe0110525ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe0110525f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0110525f30 --- trap 0xafafafaf, rip = 0xafafafafafafafaf, rsp = 0xafafafafafafafaf, rbp = 0xafafafafafafafaf --- Tracing command bufdaemon pid 17 tid 100199 td 0xfffffe010e3c3e40 sched_switch() at sched_switch+0x582/frame 0xfffffe010db52d50 mi_switch() at mi_switch+0x157/frame 0xfffffe010db52d70 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe010db52db0 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe010db52df0 _sleep() at _sleep+0x2d6/frame 0xfffffe010db52e80 bufspace_daemon() at bufspace_daemon+0x312/frame 0xfffffe010db52ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe010db52f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010db52f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 17 tid 100200 td 0xfffffe010e3c3720 sched_switch() at sched_switch+0x582/frame 0xfffffe0110f39d50 mi_switch() at mi_switch+0x157/frame 0xfffffe0110f39d70 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0110f39db0 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0110f39df0 _sleep() at _sleep+0x2d6/frame 0xfffffe0110f39e80 bufspace_daemon() at bufspace_daemon+0x312/frame 0xfffffe0110f39ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe0110f39f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0110f39f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 17 tid 100201 td 0xfffffe010e3c3000 sched_switch() at sched_switch+0x582/frame 0xfffffe0110f34d50 mi_switch() at mi_switch+0x157/frame 0xfffffe0110f34d70 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0110f34db0 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0110f34df0 _sleep() at _sleep+0x2d6/frame 0xfffffe0110f34e80 bufspace_daemon() at bufspace_daemon+0x312/frame 0xfffffe0110f34ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe0110f34f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0110f34f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 17 tid 100202 td 0xfffffe010e3a8740 sched_switch() at sched_switch+0x582/frame 0xfffffe010db57d50 mi_switch() at mi_switch+0x157/frame 0xfffffe010db57d70 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe010db57db0 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe010db57df0 _sleep() at _sleep+0x2d6/frame 0xfffffe010db57e80 bufspace_daemon() at bufspace_daemon+0x312/frame 0xfffffe010db57ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe010db57f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010db57f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 17 tid 100203 td 0xfffffe010e3a8020 sched_switch() at sched_switch+0x582/frame 0xfffffe0111140d50 mi_switch() at mi_switch+0x157/frame 0xfffffe0111140d70 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0111140db0 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0111140df0 _sleep() at _sleep+0x2d6/frame 0xfffffe0111140e80 bufspace_daemon() at bufspace_daemon+0x312/frame 0xfffffe0111140ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe0111140f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0111140f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 17 tid 100204 td 0xfffffe010e3a7900 sched_switch() at sched_switch+0x582/frame 0xfffffe011194dd50 mi_switch() at mi_switch+0x157/frame 0xfffffe011194dd70 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe011194ddb0 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe011194ddf0 _sleep() at _sleep+0x2d6/frame 0xfffffe011194de80 bufspace_daemon() at bufspace_daemon+0x312/frame 0xfffffe011194def0 fork_exit() at fork_exit+0x80/frame 0xfffffe011194df30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe011194df30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 17 tid 100219 td 0xfffffe010e3a4000 sched_switch() at sched_switch+0x582/frame 0xfffffe013b9d2d80 mi_switch() at mi_switch+0x157/frame 0xfffffe013b9d2da0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe013b9d2de0 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe013b9d2e20 _sleep() at _sleep+0x2d6/frame 0xfffffe013b9d2eb0 softdep_flush() at softdep_flush+0x188/frame 0xfffffe013b9d2ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe013b9d2f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe013b9d2f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 17 tid 100225 td 0xfffffe010dc2c740 sched_switch() at sched_switch+0x582/frame 0xfffffe013b9afd80 mi_switch() at mi_switch+0x157/frame 0xfffffe013b9afda0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe013b9afde0 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe013b9afe20 _sleep() at _sleep+0x2d6/frame 0xfffffe013b9afeb0 softdep_flush() at softdep_flush+0x188/frame 0xfffffe013b9afef0 fork_exit() at fork_exit+0x80/frame 0xfffffe013b9aff30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe013b9aff30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 17 tid 100742 td 0xfffffe0155f10c80 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4fcfd80 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4fcfda0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4fcfde0 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e4fcfe20 _sleep() at _sleep+0x2d6/frame 0xfffffe00e4fcfeb0 softdep_flush() at softdep_flush+0x188/frame 0xfffffe00e4fcfef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4fcff30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4fcff30 --- trap 0xa5a5a5a5, rip = 0xa5a5a5a5a5a5a5a5, rsp = 0xa5a5a5a5a5a5a5a5, rbp = 0xa5a5a5a5a5a5a5a5 --- Tracing command syncer pid 18 tid 100207 td 0xfffffe010e3a5560 sched_switch() at sched_switch+0x582/frame 0xfffffe010db48d40 mi_switch() at mi_switch+0x157/frame 0xfffffe010db48d60 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe010db48da0 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe010db48de0 _cv_timedwait_sbt() at _cv_timedwait_sbt+0x19a/frame 0xfffffe010db48e60 sched_sync() at sched_sync+0x62d/frame 0xfffffe010db48ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe010db48f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010db48f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command vnlru pid 19 tid 100208 td 0xfffffe010e3a4e40 sched_switch() at sched_switch+0x582/frame 0xfffffe013b996d20 mi_switch() at mi_switch+0x157/frame 0xfffffe013b996d40 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe013b996d80 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe013b996dc0 _sleep() at _sleep+0x2d6/frame 0xfffffe013b996e50 vnlru_proc() at vnlru_proc+0x99c/frame 0xfffffe013b996ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe013b996f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe013b996f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command dhclient pid 2323 tid 100239 td 0xfffffe014384c000 sched_switch() at sched_switch+0x582/frame 0xfffffe01131d6ab0 mi_switch() at mi_switch+0x157/frame 0xfffffe01131d6ad0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe01131d6b10 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01131d6b60 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01131d6b70 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01131d6bd0 kern_select() at kern_select+0xadd/frame 0xfffffe01131d6dc0 sys_select() at sys_select+0x56/frame 0xfffffe01131d6e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe01131d6f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01131d6f30 --- syscall (93, FreeBSD ELF64, select), rip = 0x31d5908706fa, rsp = 0x31d58d4601d8, rbp = 0x31d58d460290 --- Tracing command dhclient pid 2326 tid 100220 td 0xfffffe0141e47740 sched_switch() at sched_switch+0x582/frame 0xfffffe013b9dcae0 mi_switch() at mi_switch+0x157/frame 0xfffffe013b9dcb00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe013b9dcb40 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe013b9dcb90 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe013b9dcba0 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe013b9dcc00 kern_poll_kfds() at kern_poll_kfds+0x4dc/frame 0xfffffe013b9dcc90 kern_poll() at kern_poll+0x9f/frame 0xfffffe013b9dcde0 sys_poll() at sys_poll+0x50/frame 0xfffffe013b9dce00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe013b9dcf30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe013b9dcf30 --- syscall (209, FreeBSD ELF64, poll), rip = 0x31d5908705fa, rsp = 0x31d58d460418, rbp = 0x31d58d460450 --- Tracing command dhclient pid 2402 tid 100242 td 0xfffffe010e2453a0 sched_switch() at sched_switch+0x582/frame 0xfffffe01131f9a90 mi_switch() at mi_switch+0x157/frame 0xfffffe01131f9ab0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe01131f9af0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01131f9b40 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe01131f9b80 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19a/frame 0xfffffe01131f9c00 kern_poll_kfds() at kern_poll_kfds+0x4a9/frame 0xfffffe01131f9c90 kern_poll() at kern_poll+0x9f/frame 0xfffffe01131f9de0 sys_poll() at sys_poll+0x50/frame 0xfffffe01131f9e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe01131f9f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01131f9f30 --- syscall (209, FreeBSD ELF64, poll), rip = 0x31d5908705fa, rsp = 0x31d58d460418, rbp = 0x31d58d460450 --- Tracing command devd pid 2407 tid 100236 td 0xfffffe014384c720 sched_switch() at sched_switch+0x582/frame 0xfffffe01131dba60 mi_switch() at mi_switch+0x157/frame 0xfffffe01131dba80 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe01131dbac0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01131dbb10 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe01131dbb50 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19a/frame 0xfffffe01131dbbd0 kern_select() at kern_select+0xab7/frame 0xfffffe01131dbdc0 sys_select() at sys_select+0x56/frame 0xfffffe01131dbe00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe01131dbf30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01131dbf30 --- syscall (93, FreeBSD ELF64, select), rip = 0x2ac35a, rsp = 0x820a7bc28, rbp = 0x820a7ddc0 --- Tracing command syslogd pid 2594 tid 100258 td 0xfffffe0143837000 sched_switch() at sched_switch+0x582/frame 0xfffffe010dbd7ab0 mi_switch() at mi_switch+0x157/frame 0xfffffe010dbd7ad0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe010dbd7b10 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe010dbd7b60 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe010dbd7b70 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe010dbd7bd0 kern_select() at kern_select+0xadd/frame 0xfffffe010dbd7dc0 sys_select() at sys_select+0x56/frame 0xfffffe010dbd7e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe010dbd7f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe010dbd7f30 --- syscall (93, FreeBSD ELF64, select), rip = 0x21682f88a6fa, rsp = 0x21682ca1ad98, rbp = 0x21682ca1b320 --- Tracing command watchdogd pid 2602 tid 100245 td 0xfffffe014383b740 sched_switch() at sched_switch+0x582/frame 0xfffffe010db6bbc0 mi_switch() at mi_switch+0x157/frame 0xfffffe010db6bbe0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe010db6bc20 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe010db6bc70 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe010db6bcb0 _sleep() at _sleep+0x2c4/frame 0xfffffe010db6bd40 kern_clock_nanosleep() at kern_clock_nanosleep+0x1c1/frame 0xfffffe010db6bdc0 sys_nanosleep() at sys_nanosleep+0x3b/frame 0xfffffe010db6be00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe010db6bf30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe010db6bf30 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x27ee2d2cc57a, rsp = 0x27ee2ba1c7e8, rbp = 0x27ee2ba1c830 --- Tracing command powerd pid 2657 tid 100246 td 0xfffffe010e244c80 sched_switch() at sched_switch+0x582/frame 0xfffffe01131f4a60 mi_switch() at mi_switch+0x157/frame 0xfffffe01131f4a80 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe01131f4ac0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01131f4b10 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe01131f4b50 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19a/frame 0xfffffe01131f4bd0 kern_select() at kern_select+0xab7/frame 0xfffffe01131f4dc0 sys_select() at sys_select+0x56/frame 0xfffffe01131f4e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe01131f4f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01131f4f30 --- syscall (93, FreeBSD ELF64, select), rip = 0x117121a3e6fa, rsp = 0x11711f6e3f48, rbp = 0x11711f6e4080 --- Tracing command ntpd pid 2683 tid 100267 td 0xfffffe0142ae3c80 sched_switch() at sched_switch+0x582/frame 0xfffffe010dbf2ab0 mi_switch() at mi_switch+0x157/frame 0xfffffe010dbf2ad0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe010dbf2b10 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe010dbf2b60 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe010dbf2b70 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe010dbf2bd0 kern_select() at kern_select+0xadd/frame 0xfffffe010dbf2dc0 sys_select() at sys_select+0x56/frame 0xfffffe010dbf2e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe010dbf2f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe010dbf2f30 --- syscall (93, FreeBSD ELF64, select), rip = 0x1d6d3bf4f6fa, rsp = 0x1d6d3895bc58, rbp = 0x1d6d3895bc90 --- Tracing command inetd pid 2709 tid 100268 td 0xfffffe0144121560 sched_switch() at sched_switch+0x582/frame 0xfffffe0141874ab0 mi_switch() at mi_switch+0x157/frame 0xfffffe0141874ad0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0141874b10 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0141874b60 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0141874b70 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe0141874bd0 kern_select() at kern_select+0xadd/frame 0xfffffe0141874dc0 sys_select() at sys_select+0x56/frame 0xfffffe0141874e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe0141874f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0141874f30 --- syscall (93, FreeBSD ELF64, select), rip = 0x1900d2dd76fa, rsp = 0x1900cd89a8a8, rbp = 0x1900cd89b730 --- Tracing command sshd pid 2728 tid 100278 td 0xfffffe014383a900 sched_switch() at sched_switch+0x582/frame 0xfffffe01418a6ab0 mi_switch() at mi_switch+0x157/frame 0xfffffe01418a6ad0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe01418a6b10 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01418a6b60 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01418a6b70 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01418a6bd0 kern_poll_kfds() at kern_poll_kfds+0x4dc/frame 0xfffffe01418a6c60 kern_poll() at kern_poll+0x9f/frame 0xfffffe01418a6db0 sys_ppoll() at sys_ppoll+0x6f/frame 0xfffffe01418a6e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe01418a6f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01418a6f30 --- syscall (545, FreeBSD ELF64, ppoll), rip = 0x32750597461a, rsp = 0x3274fd896d98, rbp = 0x3274fd896dd0 --- Tracing command cron pid 2732 tid 100281 td 0xfffffe014443b000 --- swapped out Tracing command getty pid 2746 tid 100209 td 0xfffffe011323b900 sched_switch() at sched_switch+0x582/frame 0xfffffe013b9a0b00 mi_switch() at mi_switch+0x157/frame 0xfffffe013b9a0b20 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe013b9a0b60 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe013b9a0bb0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe013b9a0bc0 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe013b9a0c20 tty_wait() at tty_wait+0x42/frame 0xfffffe013b9a0c50 ttydisc_read() at ttydisc_read+0x208/frame 0xfffffe013b9a0cc0 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe013b9a0cf0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe013b9a0d50 dofileread() at dofileread+0x83/frame 0xfffffe013b9a0d90 sys_read() at sys_read+0xc0/frame 0xfffffe013b9a0e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe013b9a0f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe013b9a0f30 --- syscall (3, FreeBSD ELF64, read), rip = 0x1769c77c667a, rsp = 0x1769c55e8da8, rbp = 0x1769c55e8df0 --- Tracing command getty pid 2747 tid 100277 td 0xfffffe014383b020 sched_switch() at sched_switch+0x582/frame 0xfffffe010db66b00 mi_switch() at mi_switch+0x157/frame 0xfffffe010db66b20 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe010db66b60 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe010db66bb0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe010db66bc0 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe010db66c20 tty_wait() at tty_wait+0x42/frame 0xfffffe010db66c50 ttydisc_read() at ttydisc_read+0x208/frame 0xfffffe010db66cc0 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe010db66cf0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe010db66d50 dofileread() at dofileread+0x83/frame 0xfffffe010db66d90 sys_read() at sys_read+0xc0/frame 0xfffffe010db66e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe010db66f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe010db66f30 --- syscall (3, FreeBSD ELF64, read), rip = 0x13184740f67a, rsp = 0x131845c40e18, rbp = 0x131845c40e60 --- Tracing command getty pid 2748 tid 100273 td 0xfffffe0142ae2720 sched_switch() at sched_switch+0x582/frame 0xfffffe0141892b00 mi_switch() at mi_switch+0x157/frame 0xfffffe0141892b20 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0141892b60 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0141892bb0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0141892bc0 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe0141892c20 tty_wait() at tty_wait+0x42/frame 0xfffffe0141892c50 ttydisc_read() at ttydisc_read+0x208/frame 0xfffffe0141892cc0 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe0141892cf0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe0141892d50 dofileread() at dofileread+0x83/frame 0xfffffe0141892d90 sys_read() at sys_read+0xc0/frame 0xfffffe0141892e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe0141892f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0141892f30 --- syscall (3, FreeBSD ELF64, read), rip = 0x179012bd67a, rsp = 0x178ffe9f588, rbp = 0x178ffe9f5d0 --- Tracing command getty pid 2749 tid 100266 td 0xfffffe0143f5c560 sched_switch() at sched_switch+0x582/frame 0xfffffe0141865b00 mi_switch() at mi_switch+0x157/frame 0xfffffe0141865b20 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0141865b60 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0141865bb0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0141865bc0 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe0141865c20 tty_wait() at tty_wait+0x42/frame 0xfffffe0141865c50 ttydisc_read() at ttydisc_read+0x208/frame 0xfffffe0141865cc0 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe0141865cf0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe0141865d50 dofileread() at dofileread+0x83/frame 0xfffffe0141865d90 sys_read() at sys_read+0xc0/frame 0xfffffe0141865e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe0141865f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0141865f30 --- syscall (3, FreeBSD ELF64, read), rip = 0x285240ee467a, rsp = 0x28523d337918, rbp = 0x28523d337960 --- Tracing command getty pid 2750 tid 100264 td 0xfffffe013ba73720 sched_switch() at sched_switch+0x582/frame 0xfffffe0141860b00 mi_switch() at mi_switch+0x157/frame 0xfffffe0141860b20 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0141860b60 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0141860bb0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0141860bc0 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe0141860c20 tty_wait() at tty_wait+0x42/frame 0xfffffe0141860c50 ttydisc_read() at ttydisc_read+0x208/frame 0xfffffe0141860cc0 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe0141860cf0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe0141860d50 dofileread() at dofileread+0x83/frame 0xfffffe0141860d90 sys_read() at sys_read+0xc0/frame 0xfffffe0141860e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe0141860f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0141860f30 --- syscall (3, FreeBSD ELF64, read), rip = 0x366e93f1367a, rsp = 0x366e90c1ddc8, rbp = 0x366e90c1de10 --- Tracing command getty pid 2751 tid 100282 td 0xfffffe0143f5b720 sched_switch() at sched_switch+0x582/frame 0xfffffe0141879b00 mi_switch() at mi_switch+0x157/frame 0xfffffe0141879b20 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0141879b60 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0141879bb0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0141879bc0 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe0141879c20 tty_wait() at tty_wait+0x42/frame 0xfffffe0141879c50 ttydisc_read() at ttydisc_read+0x208/frame 0xfffffe0141879cc0 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe0141879cf0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe0141879d50 dofileread() at dofileread+0x83/frame 0xfffffe0141879d90 sys_read() at sys_read+0xc0/frame 0xfffffe0141879e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe0141879f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0141879f30 --- syscall (3, FreeBSD ELF64, read), rip = 0xb0832aec67a, rsp = 0xb082f9d6728, rbp = 0xb082f9d6770 --- Tracing command getty pid 2752 tid 100283 td 0xfffffe0143f5b000 sched_switch() at sched_switch+0x582/frame 0xfffffe01418c4b00 mi_switch() at mi_switch+0x157/frame 0xfffffe01418c4b20 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe01418c4b60 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01418c4bb0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01418c4bc0 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01418c4c20 tty_wait() at tty_wait+0x42/frame 0xfffffe01418c4c50 ttydisc_read() at ttydisc_read+0x208/frame 0xfffffe01418c4cc0 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01418c4cf0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe01418c4d50 dofileread() at dofileread+0x83/frame 0xfffffe01418c4d90 sys_read() at sys_read+0xc0/frame 0xfffffe01418c4e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe01418c4f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01418c4f30 --- syscall (3, FreeBSD ELF64, read), rip = 0x392e977dc67a, rsp = 0x392e95b50fd8, rbp = 0x392e95b51020 --- Tracing command getty pid 2753 tid 100284 td 0xfffffe0143850740 sched_switch() at sched_switch+0x582/frame 0xfffffe01418bfb00 mi_switch() at mi_switch+0x157/frame 0xfffffe01418bfb20 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe01418bfb60 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01418bfbb0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01418bfbc0 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01418bfc20 tty_wait() at tty_wait+0x42/frame 0xfffffe01418bfc50 ttydisc_read() at ttydisc_read+0x208/frame 0xfffffe01418bfcc0 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01418bfcf0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe01418bfd50 dofileread() at dofileread+0x83/frame 0xfffffe01418bfd90 sys_read() at sys_read+0xc0/frame 0xfffffe01418bfe00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe01418bff30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01418bff30 --- syscall (3, FreeBSD ELF64, read), rip = 0x34b61b63767a, rsp = 0x34b619a3dc48, rbp = 0x34b619a3dc90 --- Tracing command getty pid 2754 tid 100285 td 0xfffffe014443e900 sched_switch() at sched_switch+0x582/frame 0xfffffe01418ceb00 mi_switch() at mi_switch+0x157/frame 0xfffffe01418ceb20 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe01418ceb60 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01418cebb0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01418cebc0 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01418cec20 tty_wait() at tty_wait+0x42/frame 0xfffffe01418cec50 ttydisc_read() at ttydisc_read+0x208/frame 0xfffffe01418cecc0 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01418cecf0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe01418ced50 dofileread() at dofileread+0x83/frame 0xfffffe01418ced90 sys_read() at sys_read+0xc0/frame 0xfffffe01418cee00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe01418cef30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01418cef30 --- syscall (3, FreeBSD ELF64, read), rip = 0x28dbbf85f67a, rsp = 0x28dbbe820db8, rbp = 0x28dbbe820e00 --- Tracing command getty pid 2755 tid 100286 td 0xfffffe014443e1e0 sched_switch() at sched_switch+0x582/frame 0xfffffe01418c9b00 mi_switch() at mi_switch+0x157/frame 0xfffffe01418c9b20 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe01418c9b60 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01418c9bb0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01418c9bc0 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01418c9c20 tty_wait() at tty_wait+0x42/frame 0xfffffe01418c9c50 ttydisc_read() at ttydisc_read+0x208/frame 0xfffffe01418c9cc0 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01418c9cf0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe01418c9d50 dofileread() at dofileread+0x83/frame 0xfffffe01418c9d90 sys_read() at sys_read+0xc0/frame 0xfffffe01418c9e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe01418c9f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01418c9f30 --- syscall (3, FreeBSD ELF64, read), rip = 0x2afcdb48167a, rsp = 0x2afcd831de68, rbp = 0x2afcd831deb0 --- Tracing command sshd pid 2769 tid 100260 td 0xfffffe014384f900 sched_switch() at sched_switch+0x582/frame 0xfffffe01131e0ae0 mi_switch() at mi_switch+0x157/frame 0xfffffe01131e0b00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe01131e0b40 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01131e0b90 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01131e0ba0 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01131e0c00 kern_poll_kfds() at kern_poll_kfds+0x4dc/frame 0xfffffe01131e0c90 kern_poll() at kern_poll+0x9f/frame 0xfffffe01131e0de0 sys_poll() at sys_poll+0x50/frame 0xfffffe01131e0e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe01131e0f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01131e0f30 --- syscall (209, FreeBSD ELF64, poll), rip = 0x3fd0a544d5fa, rsp = 0x3fd09d0a81a8, rbp = 0x3fd09d0a81e0 --- Tracing command sshd pid 2772 tid 100234 td 0xfffffe010e3c61e0 sched_switch() at sched_switch+0x582/frame 0xfffffe01131d1ab0 mi_switch() at mi_switch+0x157/frame 0xfffffe01131d1ad0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe01131d1b10 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01131d1b60 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01131d1b70 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01131d1bd0 kern_poll_kfds() at kern_poll_kfds+0x4dc/frame 0xfffffe01131d1c60 kern_poll() at kern_poll+0x9f/frame 0xfffffe01131d1db0 sys_ppoll() at sys_ppoll+0x6f/frame 0xfffffe01131d1e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe01131d1f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01131d1f30 --- syscall (545, FreeBSD ELF64, ppoll), rip = 0x3fd0a544d61a, rsp = 0x3fd09d0a7f58, rbp = 0x3fd09d0a7f90 --- Tracing command bash pid 2773 tid 100259 td 0xfffffe010e243e40 --- swapped out Tracing command su pid 2778 tid 100218 td 0xfffffe0141e43e40 --- swapped out Tracing command csh pid 2779 tid 100274 td 0xfffffe0142ae6740 --- swapped out Tracing command bash pid 2781 tid 100243 td 0xfffffe013ba74560 --- swapped out Tracing command sh pid 3285 tid 100210 td 0xfffffe010e3a4720 --- swapped out Tracing command sh pid 3364 tid 100222 td 0xfffffe0142ae51e0 --- swapped out Tracing command sh pid 3396 tid 100521 td 0xfffffe0155f00900 --- swapped out Tracing command awk pid 3397 tid 100378 td 0xfffffe0142ae5900 sched_switch() at sched_switch+0x582/frame 0xfffffe0141964b90 mi_switch() at mi_switch+0x157/frame 0xfffffe0141964bb0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0141964bf0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0141964c40 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0141964c50 _sleep() at _sleep+0x2e5/frame 0xfffffe0141964ce0 pipe_read() at pipe_read+0x42a/frame 0xfffffe0141964d50 dofileread() at dofileread+0x83/frame 0xfffffe0141964d90 sys_read() at sys_read+0xc0/frame 0xfffffe0141964e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe0141964f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0141964f30 --- syscall (3, FreeBSD ELF64, read), rip = 0x282aae20467a, rsp = 0x282aab25c4c8, rbp = 0x282aab25c4e0 --- Tracing command sh pid 3399 tid 100280 td 0xfffffe014384ce40 sched_switch() at sched_switch+0x582/frame 0xfffffe01418abb90 mi_switch() at mi_switch+0x157/frame 0xfffffe01418abbb0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe01418abbf0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01418abc40 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01418abc50 _sleep() at _sleep+0x2e5/frame 0xfffffe01418abce0 pipe_read() at pipe_read+0x42a/frame 0xfffffe01418abd50 dofileread() at dofileread+0x83/frame 0xfffffe01418abd90 sys_read() at sys_read+0xc0/frame 0xfffffe01418abe00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe01418abf30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01418abf30 --- syscall (3, FreeBSD ELF64, read), rip = 0x186d08db067a, rsp = 0x186d05a10448, rbp = 0x186d05a10970 --- Tracing command sh pid 69469 tid 100559 td 0xfffffe014383a1e0 --- swapped out Tracing command tee pid 69470 tid 100211 td 0xfffffe013ba753a0 sched_switch() at sched_switch+0x582/frame 0xfffffe013b9aab90 mi_switch() at mi_switch+0x157/frame 0xfffffe013b9aabb0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe013b9aabf0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe013b9aac40 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe013b9aac50 _sleep() at _sleep+0x2e5/frame 0xfffffe013b9aace0 pipe_read() at pipe_read+0x42a/frame 0xfffffe013b9aad50 dofileread() at dofileread+0x83/frame 0xfffffe013b9aad90 sys_read() at sys_read+0xc0/frame 0xfffffe013b9aae00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe013b9aaf30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe013b9aaf30 --- syscall (3, FreeBSD ELF64, read), rip = 0x3cc0fa567a, rsp = 0x3cbf334d48, rbp = 0x3cbf334da0 --- Tracing command sh pid 69471 tid 100214 td 0xfffffe011323aac0 --- swapped out Tracing command md10 pid 69482 tid 100652 td 0xfffffe014f1b1740 (CPU 7) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce6edf0 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce6ee00 trap() at trap+0x57/frame 0xfffffe003ce6ef20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce6ef20 --- trap 0x13, rip = 0xffffffff810d56e6, rsp = 0xfffffe0156f86d20, rbp = 0xfffffe0156f86d20 --- memmove_erms() at memmove_erms+0x116/frame 0xfffffe0156f86d20 uiomove_fromphys() at uiomove_fromphys+0x87/frame 0xfffffe0156f86d90 physcopyin() at physcopyin+0xbe/frame 0xfffffe0156f86e20 mdstart_swap() at mdstart_swap+0x40b/frame 0xfffffe0156f86ea0 md_kthread() at md_kthread+0x20c/frame 0xfffffe0156f86ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe0156f86f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0156f86f30 --- trap 0xc, rip = 0x3e103b8e929a, rsp = 0x3e1038c7cdb8, rbp = 0x3e1038c7ce70 --- Tracing command su pid 69486 tid 100694 td 0xfffffe015b1ec020 --- swapped out Tracing command sh pid 28555 tid 100585 td 0xfffffe014f1ac020 sched_switch() at sched_switch+0x582/frame 0xfffffe0156e45b90 mi_switch() at mi_switch+0x157/frame 0xfffffe0156e45bb0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0156e45bf0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0156e45c40 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0156e45c50 _sleep() at _sleep+0x2e5/frame 0xfffffe0156e45ce0 pipe_read() at pipe_read+0x42a/frame 0xfffffe0156e45d50 dofileread() at dofileread+0x83/frame 0xfffffe0156e45d90 sys_read() at sys_read+0xc0/frame 0xfffffe0156e45e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe0156e45f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0156e45f30 --- syscall (3, FreeBSD ELF64, read), rip = 0x186d08db067a, rsp = 0x186d05a10568, rbp = 0x186d05a106d0 --- Tracing command run pid 69518 tid 100646 td 0xfffffe01575aac80 sched_switch() at sched_switch+0x582/frame 0xfffffe0156f77bc0 mi_switch() at mi_switch+0x157/frame 0xfffffe0156f77be0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0156f77c20 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0156f77c70 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe0156f77cb0 _sleep() at _sleep+0x2c4/frame 0xfffffe0156f77d40 kern_clock_nanosleep() at kern_clock_nanosleep+0x1c1/frame 0xfffffe0156f77dc0 sys_nanosleep() at sys_nanosleep+0x3b/frame 0xfffffe0156f77e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe0156f77f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0156f77f30 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x16a16ed9657a, rsp = 0x16a16d2d3ba8, rbp = 0x16a16d2d3bf0 --- Tracing command run pid 69522 tid 100586 td 0xfffffe014f1ab1e0 --- swapped out Tracing command run pid 69523 tid 100687 td 0xfffffe01571361e0 --- swapped out Tracing command sh pid 28729 tid 100235 td 0xfffffe010dc2aac0 --- swapped out Tracing command grep pid 28730 tid 100228 td 0xfffffe010dc2b1e0 sched_switch() at sched_switch+0x582/frame 0xfffffe01131a9b90 mi_switch() at mi_switch+0x157/frame 0xfffffe01131a9bb0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe01131a9bf0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01131a9c40 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01131a9c50 _sleep() at _sleep+0x2e5/frame 0xfffffe01131a9ce0 pipe_read() at pipe_read+0x42a/frame 0xfffffe01131a9d50 dofileread() at dofileread+0x83/frame 0xfffffe01131a9d90 sys_read() at sys_read+0xc0/frame 0xfffffe01131a9e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe01131a9f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01131a9f30 --- syscall (3, FreeBSD ELF64, read), rip = 0x1c71d357367a, rsp = 0x1c71d0d55738, rbp = 0x1c71d0d55850 --- Tracing command mksnap_ffs pid 28731 tid 100226 td 0xfffffe010dc2c020 --- swapped out Tracing command sh pid 28750 tid 100247 td 0xfffffe010e244560 --- swapped out Tracing command sed pid 28752 tid 100212 td 0xfffffe010dde5000 --- swapped out Tracing command creat pid 22377 tid 100657 td 0xfffffe0158d1bc80 sched_switch() at sched_switch+0x582/frame 0xfffffe0156fb3bc0 mi_switch() at mi_switch+0x157/frame 0xfffffe0156fb3be0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0156fb3c20 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0156fb3c70 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe0156fb3cb0 _sleep() at _sleep+0x2c4/frame 0xfffffe0156fb3d40 kern_clock_nanosleep() at kern_clock_nanosleep+0x1c1/frame 0xfffffe0156fb3dc0 sys_nanosleep() at sys_nanosleep+0x3b/frame 0xfffffe0156fb3e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe0156fb3f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0156fb3f30 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x598515ef57a, rsp = 0x5984eec7fd8, rbp = 0x5984eec8020 --- Tracing command mkdir pid 22378 tid 100336 td 0xfffffe014462ec80 sched_switch() at sched_switch+0x582/frame 0xfffffe0141919bc0 mi_switch() at mi_switch+0x157/frame 0xfffffe0141919be0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0141919c20 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0141919c70 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe0141919cb0 _sleep() at _sleep+0x2c4/frame 0xfffffe0141919d40 kern_clock_nanosleep() at kern_clock_nanosleep+0x1c1/frame 0xfffffe0141919dc0 sys_nanosleep() at sys_nanosleep+0x3b/frame 0xfffffe0141919e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe0141919f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0141919f30 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x2c617345157a, rsp = 0x2c617162b958, rbp = 0x2c617162b9a0 --- Tracing command rw pid 22379 tid 100555 td 0xfffffe0155f01740 sched_switch() at sched_switch+0x582/frame 0xfffffe01419c3bc0 mi_switch() at mi_switch+0x157/frame 0xfffffe01419c3be0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe01419c3c20 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01419c3c70 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe01419c3cb0 _sleep() at _sleep+0x2c4/frame 0xfffffe01419c3d40 kern_clock_nanosleep() at kern_clock_nanosleep+0x1c1/frame 0xfffffe01419c3dc0 sys_nanosleep() at sys_nanosleep+0x3b/frame 0xfffffe01419c3e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe01419c3f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01419c3f30 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x6d7af00857a, rsp = 0x6d7ada57f08, rbp = 0x6d7ada57f50 --- Tracing command swap pid 22380 tid 100261 td 0xfffffe014384f1e0 sched_switch() at sched_switch+0x582/frame 0xfffffe010dbedbc0 mi_switch() at mi_switch+0x157/frame 0xfffffe010dbedbe0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe010dbedc20 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe010dbedc70 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe010dbedcb0 _sleep() at _sleep+0x2c4/frame 0xfffffe010dbedd40 kern_clock_nanosleep() at kern_clock_nanosleep+0x1c1/frame 0xfffffe010dbeddc0 sys_nanosleep() at sys_nanosleep+0x3b/frame 0xfffffe010dbede00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe010dbedf30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe010dbedf30 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x244d7ef3957a, rsp = 0x244d7db46798, rbp = 0x244d7db467e0 --- Tracing command mkdir pid 22382 tid 100229 td 0xfffffe010e3c7020 --- swapped out Tracing command mkdir pid 22383 tid 100240 td 0xfffffe0143837e40 sched_switch() at sched_switch+0x582/frame 0xfffffe01131ef680 mi_switch() at mi_switch+0x157/frame 0xfffffe01131ef6a0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe01131ef6e0 sleeplk() at sleeplk+0x100/frame 0xfffffe01131ef740 lockmgr_xlock_hard() at lockmgr_xlock_hard+0x3ef/frame 0xfffffe01131ef7e0 __lockmgr_args() at __lockmgr_args+0x1f9/frame 0xfffffe01131ef880 getblkx() at getblkx+0x1e2/frame 0xfffffe01131ef940 breadn_flags() at breadn_flags+0x44/frame 0xfffffe01131ef9b0 ffs_breadz() at ffs_breadz+0x5a/frame 0xfffffe01131efa30 ffs_update() at ffs_update+0x140/frame 0xfffffe01131efac0 handle_workitem_remove() at handle_workitem_remove+0x463/frame 0xfffffe01131efb40 handle_workitem_remove() at handle_workitem_remove+0x55d/frame 0xfffffe01131efbc0 ufs_dirremove() at ufs_dirremove+0x372/frame 0xfffffe01131efc20 ufs_rmdir() at ufs_rmdir+0x139/frame 0xfffffe01131efc70 VOP_RMDIR_APV() at VOP_RMDIR_APV+0x85/frame 0xfffffe01131efc90 kern_frmdirat() at kern_frmdirat+0x3bd/frame 0xfffffe01131efe00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe01131eff30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01131eff30 --- syscall (137, FreeBSD ELF64, rmdir), rip = 0x2c617344ed9a, rsp = 0x2c6171629378, rbp = 0x2c61716297b0 --- Tracing command rw pid 22384 tid 100581 td 0xfffffe010e3c53a0 --- swapped out Tracing command mkdir pid 22385 tid 100217 td 0xfffffe00e4d7ec80 sched_switch() at sched_switch+0x582/frame 0xfffffe00e4f1f620 mi_switch() at mi_switch+0x157/frame 0xfffffe00e4f1f640 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00e4f1f680 sleeplk() at sleeplk+0x100/frame 0xfffffe00e4f1f6e0 lockmgr_xlock_hard() at lockmgr_xlock_hard+0x3ef/frame 0xfffffe00e4f1f780 __lockmgr_args() at __lockmgr_args+0x1f9/frame 0xfffffe00e4f1f820 getblkx() at getblkx+0x1e2/frame 0xfffffe00e4f1f8e0 breadn_flags() at breadn_flags+0x44/frame 0xfffffe00e4f1f950 ffs_breadz() at ffs_breadz+0x5a/frame 0xfffffe00e4f1f9d0 ffs_update() at ffs_update+0x140/frame 0xfffffe00e4f1fa60 ufs_mkdir() at ufs_mkdir+0x66a/frame 0xfffffe00e4f1fbe0 VOP_MKDIR_APV() at VOP_MKDIR_APV+0x65/frame 0xfffffe00e4f1fc00 kern_mkdirat() at kern_mkdirat+0x20d/frame 0xfffffe00e4f1fe00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe00e4f1ff30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00e4f1ff30 --- syscall (136, FreeBSD ELF64, mkdir), rip = 0x2c617344ed7a, rsp = 0x2c617162a478, rbp = 0x2c617162a8b0 --- Tracing command rw pid 22386 tid 100607 td 0xfffffe014f1ab900 sched_switch() at sched_switch+0x582/frame 0xfffffe0156eab520 mi_switch() at mi_switch+0x157/frame 0xfffffe0156eab540 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0156eab580 sleeplk() at sleeplk+0x100/frame 0xfffffe0156eab5e0 lockmgr_xlock_hard() at lockmgr_xlock_hard+0x3ef/frame 0xfffffe0156eab680 __lockmgr_args() at __lockmgr_args+0x1f9/frame 0xfffffe0156eab720 ffs_copyonwrite() at ffs_copyonwrite+0x1a3/frame 0xfffffe0156eab7d0 ffs_geom_strategy() at ffs_geom_strategy+0xcf/frame 0xfffffe0156eab800 ufs_strategy() at ufs_strategy+0x83/frame 0xfffffe0156eab830 VOP_STRATEGY_APV() at VOP_STRATEGY_APV+0x61/frame 0xfffffe0156eab850 bufstrategy() at bufstrategy+0x3e/frame 0xfffffe0156eab880 bufwrite() at bufwrite+0x24c/frame 0xfffffe0156eab8d0 ffs_write() at ffs_write+0x470/frame 0xfffffe0156eab980 VOP_WRITE_APV() at VOP_WRITE_APV+0xcf/frame 0xfffffe0156eaba90 vn_write() at vn_write+0x301/frame 0xfffffe0156eabb20 vn_io_fault_doio() at vn_io_fault_doio+0x43/frame 0xfffffe0156eabb80 vn_io_fault1() at vn_io_fault1+0x163/frame 0xfffffe0156eabcc0 vn_io_fault() at vn_io_fault+0x1cc/frame 0xfffffe0156eabd40 dofilewrite() at dofilewrite+0x81/frame 0xfffffe0156eabd90 sys_write() at sys_write+0xc0/frame 0xfffffe0156eabe00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe0156eabf30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0156eabf30 --- syscall (4, FreeBSD ELF64, write), rip = 0x6d7af00889a, rsp = 0x6d7ada56e98, rbp = 0x6d7ada57f50 --- Tracing command creat pid 22387 tid 100661 td 0xfffffe0155f06e40 --- swapped out Tracing command rw pid 22388 tid 100659 td 0xfffffe0158d1ae40 sched_switch() at sched_switch+0x582/frame 0xfffffe0156fbd520 mi_switch() at mi_switch+0x157/frame 0xfffffe0156fbd540 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0156fbd580 sleeplk() at sleeplk+0x100/frame 0xfffffe0156fbd5e0 lockmgr_xlock_hard() at lockmgr_xlock_hard+0x3ef/frame 0xfffffe0156fbd680 __lockmgr_args() at __lockmgr_args+0x1f9/frame 0xfffffe0156fbd720 ffs_copyonwrite() at ffs_copyonwrite+0x1a3/frame 0xfffffe0156fbd7d0 ffs_geom_strategy() at ffs_geom_strategy+0xcf/frame 0xfffffe0156fbd800 ufs_strategy() at ufs_strategy+0x83/frame 0xfffffe0156fbd830 VOP_STRATEGY_APV() at VOP_STRATEGY_APV+0x61/frame 0xfffffe0156fbd850 bufstrategy() at bufstrategy+0x3e/frame 0xfffffe0156fbd880 bufwrite() at bufwrite+0x24c/frame 0xfffffe0156fbd8d0 ffs_write() at ffs_write+0x470/frame 0xfffffe0156fbd980 VOP_WRITE_APV() at VOP_WRITE_APV+0xcf/frame 0xfffffe0156fbda90 vn_write() at vn_write+0x301/frame 0xfffffe0156fbdb20 vn_io_fault_doio() at vn_io_fault_doio+0x43/frame 0xfffffe0156fbdb80 vn_io_fault1() at vn_io_fault1+0x163/frame 0xfffffe0156fbdcc0 vn_io_fault() at vn_io_fault+0x1cc/frame 0xfffffe0156fbdd40 dofilewrite() at dofilewrite+0x81/frame 0xfffffe0156fbdd90 sys_write() at sys_write+0xc0/frame 0xfffffe0156fbde00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe0156fbdf30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0156fbdf30 --- syscall (4, FreeBSD ELF64, write), rip = 0x6d7af00889a, rsp = 0x6d7ada56e98, rbp = 0x6d7ada57f50 --- Tracing command swap pid 22389 tid 100595 td 0xfffffe0155f12900 --- swapped out Tracing command creat pid 22390 tid 100334 td 0xfffffe014462fac0 sched_switch() at sched_switch+0x582/frame 0xfffffe014190a550 mi_switch() at mi_switch+0x157/frame 0xfffffe014190a570 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe014190a5b0 sleeplk() at sleeplk+0x100/frame 0xfffffe014190a610 lockmgr_xlock_hard() at lockmgr_xlock_hard+0x3ef/frame 0xfffffe014190a6b0 __lockmgr_args() at __lockmgr_args+0x1f9/frame 0xfffffe014190a750 getblkx() at getblkx+0x1e2/frame 0xfffffe014190a810 breadn_flags() at breadn_flags+0x44/frame 0xfffffe014190a880 ffs_breadz() at ffs_breadz+0x5a/frame 0xfffffe014190a900 ffs_update() at ffs_update+0x140/frame 0xfffffe014190a990 ufs_makeinode() at ufs_makeinode+0x3e6/frame 0xfffffe014190ab00 ufs_create() at ufs_create+0x34/frame 0xfffffe014190ab20 VOP_CREATE_APV() at VOP_CREATE_APV+0x62/frame 0xfffffe014190ab40 vn_open_cred() at vn_open_cred+0x2f6/frame 0xfffffe014190acb0 kern_openat() at kern_openat+0x29c/frame 0xfffffe014190ae00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe014190af30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014190af30 --- syscall (499, FreeBSD ELF64, openat), rip = 0x598515ef5ba, rsp = 0x5984eec7f68, rbp = 0x5984eec8020 --- Tracing command rw pid 22391 tid 100587 td 0xfffffe014f1ae560 (CPU 5) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce54df0 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce54e00 trap() at trap+0x57/frame 0xfffffe003ce54f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce54f20 --- trap 0x13, rip = 0xffffffff80f6e9f3, rsp = 0xfffffe0156e40a70, rbp = 0xfffffe0156e40a80 --- mtrash_ctor() at mtrash_ctor+0x33/frame 0xfffffe0156e40a80 item_ctor() at item_ctor+0x11f/frame 0xfffffe0156e40ad0 zone_alloc_item() at zone_alloc_item+0x138/frame 0xfffffe0156e40b20 malloc() at malloc+0x86/frame 0xfffffe0156e40b60 cloneuio() at cloneuio+0x2b/frame 0xfffffe0156e40b90 vn_io_fault1() at vn_io_fault1+0x133/frame 0xfffffe0156e40cd0 vn_io_fault() at vn_io_fault+0x1cc/frame 0xfffffe0156e40d50 dofileread() at dofileread+0x83/frame 0xfffffe0156e40d90 sys_read() at sys_read+0xc0/frame 0xfffffe0156e40e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe0156e40f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0156e40f30 --- syscall (3, FreeBSD ELF64, read), rip = 0x6d7af00867a, rsp = 0x6d7ada56e98, rbp = 0x6d7ada57f50 --- Tracing command swap pid 22392 tid 100612 td 0xfffffe0144630900 --- swapped out Tracing command rw pid 22393 tid 100399 td 0xfffffe014f1ade40 sched_switch() at sched_switch+0x582/frame 0xfffffe014193c520 mi_switch() at mi_switch+0x157/frame 0xfffffe014193c540 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe014193c580 sleeplk() at sleeplk+0x100/frame 0xfffffe014193c5e0 lockmgr_xlock_hard() at lockmgr_xlock_hard+0x3ef/frame 0xfffffe014193c680 __lockmgr_args() at __lockmgr_args+0x1f9/frame 0xfffffe014193c720 ffs_copyonwrite() at ffs_copyonwrite+0x1a3/frame 0xfffffe014193c7d0 ffs_geom_strategy() at ffs_geom_strategy+0xcf/frame 0xfffffe014193c800 ufs_strategy() at ufs_strategy+0x83/frame 0xfffffe014193c830 VOP_STRATEGY_APV() at VOP_STRATEGY_APV+0x61/frame 0xfffffe014193c850 bufstrategy() at bufstrategy+0x3e/frame 0xfffffe014193c880 bufwrite() at bufwrite+0x24c/frame 0xfffffe014193c8d0 ffs_write() at ffs_write+0x470/frame 0xfffffe014193c980 VOP_WRITE_APV() at VOP_WRITE_APV+0xcf/frame 0xfffffe014193ca90 vn_write() at vn_write+0x301/frame 0xfffffe014193cb20 vn_io_fault_doio() at vn_io_fault_doio+0x43/frame 0xfffffe014193cb80 vn_io_fault1() at vn_io_fault1+0x163/frame 0xfffffe014193ccc0 vn_io_fault() at vn_io_fault+0x1cc/frame 0xfffffe014193cd40 dofilewrite() at dofilewrite+0x81/frame 0xfffffe014193cd90 sys_write() at sys_write+0xc0/frame 0xfffffe014193ce00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe014193cf30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014193cf30 --- syscall (4, FreeBSD ELF64, write), rip = 0x6d7af00889a, rsp = 0x6d7ada56e98, rbp = 0x6d7ada57f50 --- Tracing command creat pid 22394 tid 100556 td 0xfffffe01441231e0 (CPU 3) kdb_enter() at kdb_enter+0x32/frame 0xfffffe01419d24a0 vpanic() at vpanic+0x182/frame 0xfffffe01419d24f0 panic() at panic+0x43/frame 0xfffffe01419d2550 softdep_disk_io_initiation() at softdep_disk_io_initiation+0x14be/frame 0xfffffe01419d2610 ffs_geom_strategy() at ffs_geom_strategy+0x18c/frame 0xfffffe01419d2640 ufs_strategy() at ufs_strategy+0x83/frame 0xfffffe01419d2670 VOP_STRATEGY_APV() at VOP_STRATEGY_APV+0x61/frame 0xfffffe01419d2690 bufstrategy() at bufstrategy+0x3e/frame 0xfffffe01419d26c0 bufwrite() at bufwrite+0x24c/frame 0xfffffe01419d2710 vfs_bio_awrite() at vfs_bio_awrite+0x21e/frame 0xfffffe01419d2780 ffs_syncvnode() at ffs_syncvnode+0x36a/frame 0xfffffe01419d2810 ffs_copyonwrite() at ffs_copyonwrite+0x5ad/frame 0xfffffe01419d28b0 ffs_geom_strategy() at ffs_geom_strategy+0xcf/frame 0xfffffe01419d28e0 bufwrite() at bufwrite+0x24c/frame 0xfffffe01419d2930 ffs_update() at ffs_update+0x336/frame 0xfffffe01419d2990 ufs_makeinode() at ufs_makeinode+0x3e6/frame 0xfffffe01419d2b00 ufs_create() at ufs_create+0x34/frame 0xfffffe01419d2b20 VOP_CREATE_APV() at VOP_CREATE_APV+0x62/frame 0xfffffe01419d2b40 vn_open_cred() at vn_open_cred+0x2f6/frame 0xfffffe01419d2cb0 kern_openat() at kern_openat+0x29c/frame 0xfffffe01419d2e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe01419d2f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01419d2f30 --- syscall (499, FreeBSD ELF64, openat), rip = 0x598515ef5ba, rsp = 0x5984eec7f68, rbp = 0x5984eec8020 --- Tracing command swap pid 22396 tid 100584 td 0xfffffe014f1a8e40 (CPU 6) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce61df0 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce61e00 trap() at trap+0x57/frame 0xfffffe003ce61f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce61f20 --- trap 0x13, rip = 0x24455ce45755, rsp = 0x244d7db467b0, rbp = 0x244d7db467e0 --- Tracing command rw pid 22397 tid 100272 td 0xfffffe0142ae2e40 (CPU 4) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce47df0 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce47e00 trap() at trap+0x57/frame 0xfffffe003ce47f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce47f20 --- trap 0x13, rip = 0xffffffff80c6266b, rsp = 0xfffffe0141883a40, rbp = 0xfffffe0141883bf0 --- witness_checkorder() at witness_checkorder+0x1cb/frame 0xfffffe0141883bf0 __mtx_lock_spin_flags() at __mtx_lock_spin_flags+0x8e/frame 0xfffffe0141883c30 wakeup() at wakeup+0xf/frame 0xfffffe0141883c50 rangelock_calc_block() at rangelock_calc_block+0x45/frame 0xfffffe0141883c80 rangelock_enqueue() at rangelock_enqueue+0xdb/frame 0xfffffe0141883cd0 vn_io_fault() at vn_io_fault+0x19c/frame 0xfffffe0141883d50 dofileread() at dofileread+0x83/frame 0xfffffe0141883d90 sys_read() at sys_read+0xc0/frame 0xfffffe0141883e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe0141883f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0141883f30 --- syscall (3, FreeBSD ELF64, read), rip = 0x6d7af00867a, rsp = 0x6d7ada56e98, rbp = 0x6d7ada57f50 --- Tracing command rw pid 22398 tid 100653 td 0xfffffe0158d181e0 sched_switch() at sched_switch+0x582/frame 0xfffffe0156f95520 mi_switch() at mi_switch+0x157/frame 0xfffffe0156f95540 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0156f95580 sleeplk() at sleeplk+0x100/frame 0xfffffe0156f955e0 lockmgr_xlock_hard() at lockmgr_xlock_hard+0x3ef/frame 0xfffffe0156f95680 __lockmgr_args() at __lockmgr_args+0x1f9/frame 0xfffffe0156f95720 ffs_copyonwrite() at ffs_copyonwrite+0x1a3/frame 0xfffffe0156f957d0 ffs_geom_strategy() at ffs_geom_strategy+0xcf/frame 0xfffffe0156f95800 ufs_strategy() at ufs_strategy+0x83/frame 0xfffffe0156f95830 VOP_STRATEGY_APV() at VOP_STRATEGY_APV+0x61/frame 0xfffffe0156f95850 bufstrategy() at bufstrategy+0x3e/frame 0xfffffe0156f95880 bufwrite() at bufwrite+0x24c/frame 0xfffffe0156f958d0 ffs_write() at ffs_write+0x470/frame 0xfffffe0156f95980 VOP_WRITE_APV() at VOP_WRITE_APV+0xcf/frame 0xfffffe0156f95a90 vn_write() at vn_write+0x301/frame 0xfffffe0156f95b20 vn_io_fault_doio() at vn_io_fault_doio+0x43/frame 0xfffffe0156f95b80 vn_io_fault1() at vn_io_fault1+0x163/frame 0xfffffe0156f95cc0 vn_io_fault() at vn_io_fault+0x1cc/frame 0xfffffe0156f95d40 dofilewrite() at dofilewrite+0x81/frame 0xfffffe0156f95d90 sys_write() at sys_write+0xc0/frame 0xfffffe0156f95e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe0156f95f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0156f95f30 --- syscall (4, FreeBSD ELF64, write), rip = 0x6d7af00889a, rsp = 0x6d7ada56e98, rbp = 0x6d7ada57f50 --- Tracing command swap pid 22399 tid 100589 td 0xfffffe014462b900 (CPU 10) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce95df0 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce95e00 trap() at trap+0x57/frame 0xfffffe003ce95f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce95f20 --- trap 0x13, rip = 0x24455ce45757, rsp = 0x244d7db467b0, rbp = 0x244d7db467e0 --- Tracing command rw pid 22400 tid 100592 td 0xfffffe0144631740 sched_switch() at sched_switch+0x582/frame 0xfffffe0156e6f520 mi_switch() at mi_switch+0x157/frame 0xfffffe0156e6f540 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0156e6f580 sleeplk() at sleeplk+0x100/frame 0xfffffe0156e6f5e0 lockmgr_xlock_hard() at lockmgr_xlock_hard+0x3ef/frame 0xfffffe0156e6f680 __lockmgr_args() at __lockmgr_args+0x1f9/frame 0xfffffe0156e6f720 ffs_copyonwrite() at ffs_copyonwrite+0x1a3/frame 0xfffffe0156e6f7d0 ffs_geom_strategy() at ffs_geom_strategy+0xcf/frame 0xfffffe0156e6f800 ufs_strategy() at ufs_strategy+0x83/frame 0xfffffe0156e6f830 VOP_STRATEGY_APV() at VOP_STRATEGY_APV+0x61/frame 0xfffffe0156e6f850 bufstrategy() at bufstrategy+0x3e/frame 0xfffffe0156e6f880 bufwrite() at bufwrite+0x24c/frame 0xfffffe0156e6f8d0 ffs_write() at ffs_write+0x470/frame 0xfffffe0156e6f980 VOP_WRITE_APV() at VOP_WRITE_APV+0xcf/frame 0xfffffe0156e6fa90 vn_write() at vn_write+0x301/frame 0xfffffe0156e6fb20 vn_io_fault_doio() at vn_io_fault_doio+0x43/frame 0xfffffe0156e6fb80 vn_io_fault1() at vn_io_fault1+0x163/frame 0xfffffe0156e6fcc0 vn_io_fault() at vn_io_fault+0x1cc/frame 0xfffffe0156e6fd40 dofilewrite() at dofilewrite+0x81/frame 0xfffffe0156e6fd90 sys_write() at sys_write+0xc0/frame 0xfffffe0156e6fe00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe0156e6ff30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0156e6ff30 --- syscall (4, FreeBSD ELF64, write), rip = 0x6d7af00889a, rsp = 0x6d7ada56e98, rbp = 0x6d7ada57f50 --- Tracing command rw pid 22401 tid 100591 td 0xfffffe014f1b0900 (CPU 0) cpustop_handler() at cpustop_handler+0x28/frame 0xffffffff81feb300 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xffffffff81feb310 trap() at trap+0x57/frame 0xffffffff81feb430 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff81feb430 --- trap 0x13, rip = 0xffffffff80c624cc, rsp = 0xfffffe01419ebaa0, rbp = 0xfffffe01419ebc50 --- witness_checkorder() at witness_checkorder+0x2c/frame 0xfffffe01419ebc50 __mtx_lock_flags() at __mtx_lock_flags+0x94/frame 0xfffffe01419ebca0 rangelock_unlock() at rangelock_unlock+0x39/frame 0xfffffe01419ebcd0 vn_io_fault() at vn_io_fault+0x1e8/frame 0xfffffe01419ebd50 dofileread() at dofileread+0x83/frame 0xfffffe01419ebd90 sys_read() at sys_read+0xc0/frame 0xfffffe01419ebe00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe01419ebf30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01419ebf30 --- syscall (3, FreeBSD ELF64, read), rip = 0x6d7af00867a, rsp = 0x6d7ada56e98, rbp = 0x6d7ada57f50 --- Tracing command swap pid 22402 tid 100669 td 0xfffffe0158d1d1e0 (CPU 9) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce88df0 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce88e00 trap() at trap+0x57/frame 0xfffffe003ce88f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce88f20 --- trap 0x13, rip = 0x24455ce4575c, rsp = 0x244d7db467b0, rbp = 0x244d7db467e0 --- Tracing command rw pid 22403 tid 100691 td 0xfffffe015b1f1720 sched_switch() at sched_switch+0x582/frame 0xfffffe015a854310 mi_switch() at mi_switch+0x157/frame 0xfffffe015a854330 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe015a854370 sleeplk() at sleeplk+0x100/frame 0xfffffe015a8543d0 lockmgr_xlock_hard() at lockmgr_xlock_hard+0x3ef/frame 0xfffffe015a854470 __lockmgr_args() at __lockmgr_args+0x1f9/frame 0xfffffe015a854510 ffs_snapblkfree() at ffs_snapblkfree+0xb0/frame 0xfffffe015a8545d0 ffs_blkfree() at ffs_blkfree+0x62/frame 0xfffffe015a854630 handle_workitem_freefrag() at handle_workitem_freefrag+0xbe/frame 0xfffffe015a854680 ffs_reallocblks() at ffs_reallocblks+0xc46/frame 0xfffffe015a8547e0 VOP_REALLOCBLKS_APV() at VOP_REALLOCBLKS_APV+0x59/frame 0xfffffe015a854800 cluster_write() at cluster_write+0x4ad/frame 0xfffffe015a8548d0 ffs_write() at ffs_write+0x410/frame 0xfffffe015a854980 VOP_WRITE_APV() at VOP_WRITE_APV+0xcf/frame 0xfffffe015a854a90 vn_write() at vn_write+0x301/frame 0xfffffe015a854b20 vn_io_fault_doio() at vn_io_fault_doio+0x43/frame 0xfffffe015a854b80 vn_io_fault1() at vn_io_fault1+0x163/frame 0xfffffe015a854cc0 vn_io_fault() at vn_io_fault+0x1cc/frame 0xfffffe015a854d40 dofilewrite() at dofilewrite+0x81/frame 0xfffffe015a854d90 sys_write() at sys_write+0xc0/frame 0xfffffe015a854e00 amd64_syscall() at amd64_syscall+0x145/frame 0xfffffe015a854f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe015a854f30 --- syscall (4, FreeBSD ELF64, write), rip = 0x6d7af00889a, rsp = 0x6d7ada56e98, rbp = 0x6d7ada57f50 --- db> reset cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 3 59 __asm("movq %%gs:%P1,%0" : "=r" (td) : "n" (offsetof(struct pcpu, (kgdb) bt #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:59 #1 dump_savectx () at ../../../kern/kern_shutdown.c:405 #2 0xffffffff80bf0368 in dumpsys (di=0x0) at /usr/src/sys/x86/include/dump.h:87 #3 doadump (textdump=textdump@entry=0x0) at ../../../kern/kern_shutdown.c:434 #4 0xffffffff804b505a in db_dump (dummy=, dummy2=, dummy3=, dummy4=) at ../../../ddb/db_command.c:593 #5 0xffffffff804b4e60 in db_command (last_cmdp=, cmd_table=, dopager=0x0) at ../../../ddb/db_command.c:506 #6 0xffffffff804b4fa6 in db_command_script (command=command@entry=0xffffffff81bd5250 "dump") at ../../../ddb/db_command.c:571 #7 0xffffffff804ba308 in db_script_exec (scriptname=0xffffffff81bd4bc0 "dump", warnifnotfound=) at ../../../ddb/db_script.c:304 #8 0xffffffff804b4e60 in db_command (last_cmdp=, cmd_table=, dopager=0x0) at ../../../ddb/db_command.c:506 #9 0xffffffff804b4fa6 in db_command_script (command=command@entry=0xffffffff81bd51e2 "run pho1") at ../../../ddb/db_command.c:571 #10 0xffffffff804ba308 in db_script_exec (scriptname=0xffffffff81bd4bc0 "dump", warnifnotfound=) at ../../../ddb/db_script.c:304 #11 0xffffffff804b4e60 in db_command (last_cmdp=, cmd_table=, dopager=0x1) at ../../../ddb/db_command.c:506 #12 0xffffffff804b4b2d in db_command_loop () at ../../../ddb/db_command.c:553 #13 0xffffffff804b81c6 in db_trap (type=, code=) at ../../../ddb/db_main.c:270 #14 0xffffffff80c3f93e in kdb_trap (type=type@entry=0x3, code=, code@entry=0x0, tf=tf@entry=0xfffffe01419d23e0) at ../../../kern/subr_kdb.c:745 #15 0xffffffff810d95f7 in trap (frame=0xfffffe01419d23e0) at ../../../amd64/amd64/trap.c:611 #16 #17 kdb_enter (why=, msg=) at ../../../kern/subr_kdb.c:509 #18 0xffffffff80bf0512 in vpanic (fmt=, ap=ap@entry=0xfffffe01419d2530) at ../../../kern/kern_shutdown.c:967 #19 0xffffffff80bf02b3 in panic (fmt=0xffffffff81e8df30 "\201\005*\201\377\377\377\377") at ../../../kern/kern_shutdown.c:903 #20 0xffffffff80f2137e in softdep_disk_io_initiation (bp=0xfffffe003b11f970) at ../../../ufs/ffs/ffs_softdep.c:10312 #21 0xffffffff80f45b9c in buf_start (bp=0xfffffe003b11f970) at ../../../sys/buf.h:450 #22 ffs_geom_strategy (bo=0xfffffe015a49aad8, bp=0xfffffe003b11f970) at ../../../ufs/ffs/ffs_vfsops.c:2543 #23 0xffffffff80f5af73 in ufs_strategy (ap=) at ../../../ufs/ufs/ufs_vnops.c:2585 #24 0xffffffff811ad371 in VOP_STRATEGY_APV (vop=0xffffffff81b72d28 , a=a@entry=0xfffffe01419d26a0) at vnode_if.c:2425 #25 0xffffffff80cb675e in VOP_STRATEGY (vp=0xffffffff81288616, bp=0xfffffe003b11f970) at ./vnode_if.h:1222 #26 bufstrategy (bo=, bp=0xfffffe003b11f970) at ../../../kern/vfs_bio.c:5120 #27 0xffffffff80cb65fc in bstrategy (bp=0xfffffe003b11f970) at ../../../sys/buf.h:440 #28 bufwrite (bp=0xfffffe003b11f970) at ../../../kern/vfs_bio.c:2353 #29 0xffffffff80cb9b7e in bwrite (bp=0xfffffe003b11f970) at ../../../sys/buf.h:428 #30 vfs_bio_awrite (bp=bp@entry=0xfffffe003b11f970) at ../../../kern/vfs_bio.c:3286 #31 0xffffffff80f4814a in ffs_syncvnode (vp=vp@entry=0xfffffe015a3ba000, waitfor=waitfor@entry=0x1, flags=flags@entry=0x1) at ../../../ufs/ffs/ffs_vnops.c:382 #32 0xffffffff80f1eced in ffs_copyonwrite (devvp=devvp@entry=0xfffffe015a49a960, bp=bp@entry=0xfffffe003a6be8d0) at ../../../ufs/ffs/ffs_snapshot.c:2443 #33 0xffffffff80f45adf in ffs_geom_strategy (bo=0xfffffe015a49aad8, bp=0xfffffe003a6be8d0) at ../../../ufs/ffs/ffs_vfsops.c:2524 #34 0xffffffff80cb65fc in bstrategy (bp=0xfffffe003a6be8d0) at ../../../sys/buf.h:440 #35 bufwrite (bp=0xfffffe003a6be8d0) at ../../../kern/vfs_bio.c:2353 #36 0xffffffff80f17096 in ffs_update (vp=0xfffffe0158cd2770, waitfor=0x0) at ../../../ufs/ffs/ffs_inode.c:216 #37 0xffffffff80f5b616 in ufs_makeinode (mode=0x81b0, dvp=0xfffffe015ad239c8, vpp=0xfffffe01419d2d18, cnp=0xfffffe01419d2d40, callfunc=) at ../../../ufs/ufs/ufs_vnops.c:2891 #38 0xffffffff80f57834 in ufs_create (ap=0xfffffe01419d2c18) at ../../../ufs/ufs/ufs_vnops.c:257 #39 0xffffffff811a9ab2 in VOP_CREATE_APV (vop=0xffffffff81b72d28 , a=a@entry=0xfffffe01419d2c18) at vnode_if.c:244 #40 0xffffffff80cfd6b6 in VOP_CREATE (dvp=0x12, vpp=0x1, cnp=0xfffffe01419d2d40, vap=0xfffffe01419d2b58) at ./vnode_if.h:133 #41 vn_open_cred (ndp=ndp@entry=0xfffffe01419d2cc0, flagp=flagp@entry=0xfffffe01419d2dd4, cmode=cmode@entry=0x1b0, vn_open_flags=vn_open_flags@entry=0x10, cred=0xfffff804b25ede00, fp=0xfffff800320aa050) at ../../../kern/vfs_vnops.c:286 #42 0xffffffff80cf3c1c in kern_openat (td=0xfffffe01441231e0, fd=0xffffff9c, path=0x5984eec7f70 , pathseg=, flags=0x602, mode=) at ../../../kern/vfs_syscalls.c:1167 #43 0xffffffff810da455 in syscallenter (td=0xfffffe01441231e0) at ../../../amd64/amd64/../../kern/subr_syscall.c:189 #44 amd64_syscall (td=0xfffffe01441231e0, traced=0x0) at ../../../amd64/amd64/trap.c:1200 #45 #46 0x00000598515ef5ba in ?? () Backtrace stopped: Cannot access memory at address 0x5984eec7f68 (kgdb) f 20 #20 0xffffffff80f2137e in softdep_disk_io_initiation (bp=0xfffffe003b11f970) at ../../../ufs/ffs/ffs_softdep.c:10312 10312 panic("handle_disk_io_initiation: Unexpected type %s", (kgdb) l 10307 case D_FREEDEP: 10308 case D_JSEGDEP: 10309 continue; 10310 10311 default: 10312 panic("handle_disk_io_initiation: Unexpected type %s", 10313 TYPENAME(wk->wk_type)); 10314 /* NOTREACHED */ 10315 } 10316 } (kgdb) info loc marker = {wk_list = {le_next = 0x0, le_prev = 0xfffff805e5a15900}, wk_mp = 0xffffffff810d75db , wk_type = 0x1d, wk_state = 0x0, wk_all = {le_next = 0xfffff80003146558, le_prev = 0xffffffff81f33c00 }, wk_func = 0xfffffe015a49a960 "\004", wk_line = 0x81f33c00} ump = 0xfffff805cae26000 wk = 0xfffff805e5a15900 newblk = inodedep = freeblks = jblkdep = (kgdb) p *bp $1 = {b_bufobj = 0xfffffe015a3ba178, b_bcount = 0x8000, b_caller1 = 0x0, b_data = 0xfffffe004f9e8000 "", b_error = 0x0, b_iocmd = 0x2, b_ioflags = 0x0, b_iooffset = 0x2078000, b_resid = 0x0, b_iodone = 0x0, b_ckhashcalc = 0x0, b_ckhash = 0x0, b_blkno = 0x103c0, b_offset = 0xffffffff87fa0000, b_bobufs = {tqe_next = 0xfffffe003b8ca3f0, tqe_prev = 0xfffffe003ac08f90}, b_vflags = 0x1, b_qindex = 0x2, b_domain = 0x5, b_subqueue = 0xffff, b_flags = 0xa00a0024, b_xflags = 0x2, b_lock = {lock_object = { lo_name = 0xffffffff81685250 "bufwait", lo_flags = 0x26730000, lo_data = 0x0, lo_witness = 0xfffff8087fd7b600}, lk_lock = 0xffffffffffffffe0, lk_exslpfail = 0x0, lk_pri = 0x40, lk_timo = 0x0, lk_stack = {depth = 0xf, pcs = {0xffffffff80bbfa00, 0xffffffff80cb65a7, 0xffffffff80cb9b7e, 0xffffffff80f4814a, 0xffffffff80f1eced, 0xffffffff80f45adf, 0xffffffff80cb65fc, 0xffffffff80f17096, 0xffffffff80f5b616, 0xffffffff80f57834, 0xffffffff811a9ab2, 0xffffffff80cfd6b6, 0xffffffff80cf3c1c, 0xffffffff810da455, 0xffffffff810ad16b, 0x0, 0x0, 0x0}}}, b_bufsize = 0x8000, b_runningbufspace = 0x8000, b_kvasize = 0x8000, b_dirtyoff = 0x0, b_dirtyend = 0x0, b_kvabase = 0xfffffe004f9e8000 "", b_lblkno = 0xffffffffffff0ff4, b_vp = 0xfffffe015a3ba000, b_rcred = 0x0, b_wcred = 0x0, {b_freelist = {tqe_next = 0xfffffe0038d02320, tqe_prev = 0xfffffe00386a16f0}, {b_pgiodone = 0xfffffe0038d02320, b_pgbefore = 0x386a16f0, b_pgafter = 0xfffffe00}}, b_cluster = {cluster_head = {tqh_first = 0xfffffe0038b259c0, tqh_last = 0xfffffe003870db70}, cluster_entry = {tqe_next = 0xfffffe0038b259c0, tqe_prev = 0xfffffe003870db70}}, b_npages = 0x8, b_dep = { lh_first = 0xfffff805e5a15900}, b_fsprivate1 = 0x0, b_fsprivate2 = 0x0, b_fsprivate3 = 0x0, b_io_tracking = {0xffffffff812ba5dd "bqrelse", 0xffffffff8127d252 "getblkx", 0xffffffff812ba5dd "bqrelse", 0xffffffff8127d252 "getblkx", 0xffffffff812ba5dd "bqrelse", 0xffffffff8127d252 "getblkx", 0xffffffff812ba5dd "bqrelse", 0xffffffff8127d252 "getblkx", 0xffffffff812ba5dd "bqrelse", 0xffffffff8127d252 "getblkx", 0xffffffff812ba5dd "bqrelse", 0xffffffff8127d252 "getblkx", 0xffffffff812ba5dd "bqrelse", 0xffffffff8127d252 "getblkx", 0xffffffff812ba5dd "bqrelse", 0xffffffff8127d252 "getblkx", 0xffffffff812ba5dd "bqrelse", 0xffffffff8127d252 "getblkx", 0xffffffff812ba5dd "bqrelse", 0xffffffff8127d252 "getblkx", 0xffffffff812ba5dd "bqrelse", 0xffffffff8127d252 "getblkx", 0xffffffff812eb844 "bdwrite", 0xffffffff812ba5dd "bqrelse", 0xffffffff811f7a30 "bufwrite", 0xffffffff8127d252 "getblkx", 0xffffffff812ba5dd "bqrelse", 0xffffffff8127d252 "getblkx", 0xffffffff812ba5dd "bqrelse", 0xffffffff8127d252 "getblkx", 0xffffffff812ba5dd "bqrelse", 0xffffffff8127d252 "getblkx"}, b_io_tcnt = 0x7cd9, b_pages = 0xfffffe003b11fc40} (kgdb) diff --git a/sbin/fsck_ffs/dir.c b/sbin/fsck_ffs/dir.c index ba286a9655130..d09e6940f812d 100644 --- a/sbin/fsck_ffs/dir.c +++ b/sbin/fsck_ffs/dir.c @@ -679,14 +679,17 @@ expanddir(struct inode *ip, char *name) struct bufarea *bp, *nbp; struct inodesc idesc; union dinode *dp; - int indiralloced; + long cg, indiralloced; char *cp; nbp = NULL; indiralloced = newblk = indirblk = 0; + memset(&idesc, 0, sizeof(struct inodesc)); + idesc.id_type = ADDR; pwarn("NO SPACE LEFT IN %s", name); if (!preen && reply("EXPAND") == 0) return (0); + cg = ino_to_cg(&sblock, ip->i_number); dp = ip->i_dp; filesize = DIP(dp, di_size); lastlbn = lblkno(&sblock, filesize); @@ -705,7 +708,8 @@ expanddir(struct inode *ip, char *name) bp = getdirblk(oldblk, lastlbnsize); if (bp->b_errs) goto bad; - if ((newblk = allocblk(sblock.fs_frag)) == 0) + newblk = allocblk(cg, sblock.fs_frag, std_checkblkavail); + if (newblk == 0) goto bad; nbp = getdatablk(newblk, sblock.fs_bsize, BT_DIRDATA); if (nbp->b_errs) @@ -724,6 +728,7 @@ expanddir(struct inode *ip, char *name) memmove(cp, &emptydir, sizeof emptydir); dirty(nbp); brelse(nbp); + binval(bp); idesc.id_blkno = oldblk; idesc.id_numfrags = numfrags(&sblock, lastlbnsize); (void)freeblock(&idesc); @@ -731,7 +736,7 @@ expanddir(struct inode *ip, char *name) printf(" (EXPANDED)\n"); return (1); } - if ((newblk = allocblk(sblock.fs_frag)) == 0) + if ((newblk = allocblk(cg, sblock.fs_frag, std_checkblkavail)) == 0) goto bad; bp = getdirblk(newblk, sblock.fs_bsize); if (bp->b_errs) @@ -749,8 +754,12 @@ expanddir(struct inode *ip, char *name) * Allocate indirect block if needed. */ if ((indirblk = DIP(dp, di_ib[0])) == 0) { - if ((indirblk = allocblk(sblock.fs_frag)) == 0) + indirblk = allocblk(cg, sblock.fs_frag, + std_checkblkavail); + if (indirblk == 0) { + binval(bp); goto bad; + } indiralloced = 1; } nbp = getdatablk(indirblk, sblock.fs_bsize, BT_LEVEL1); @@ -774,8 +783,10 @@ expanddir(struct inode *ip, char *name) return (1); bad: pfatal(" (EXPANSION FAILED)\n"); - if (nbp != NULL) + if (nbp != NULL) { + binval(bp); brelse(nbp); + } if (newblk != 0) { idesc.id_blkno = newblk; idesc.id_numfrags = sblock.fs_frag; diff --git a/sbin/fsck_ffs/ea.c b/sbin/fsck_ffs/ea.c index 54353005bdc0d..c2429cddcff4e 100644 --- a/sbin/fsck_ffs/ea.c +++ b/sbin/fsck_ffs/ea.c @@ -74,8 +74,10 @@ eascan(struct inodesc *idesc, struct ufs2_dinode *dp) blksiz = sblock.fs_fsize; else blksiz = sblock.fs_bsize; - printf("blksiz = %ju\n", (intmax_t)blksiz); bp = getdatablk(dp->di_extb[0], blksiz, BT_EXTATTR); + if (bp->b_errs) + return (STOP); + printf("blksiz = %ju\n", (intmax_t)blksiz); cp = (u_char *)bp->b_un.b_buf; for (n = 0; n < blksiz; n++) { printf("%02x", cp[n]); diff --git a/sbin/fsck_ffs/fsck.h b/sbin/fsck_ffs/fsck.h index d1b45a0850dac..a00fedd9ef909 100644 --- a/sbin/fsck_ffs/fsck.h +++ b/sbin/fsck_ffs/fsck.h @@ -200,8 +200,7 @@ struct bufarea { #define BT_INODES 7 /* Buffer holds inodes */ #define BT_DIRDATA 8 /* Buffer holds directory data */ #define BT_DATA 9 /* Buffer holds user data */ -#define BT_EMPTY 10 /* Buffer allocated but not filled */ -#define BT_NUMBUFTYPES 11 +#define BT_NUMBUFTYPES 10 #define BT_NAMES { \ "unknown", \ "Superblock", \ @@ -212,8 +211,7 @@ struct bufarea { "External Attribute", \ "Inode Block", \ "Directory Contents", \ - "User Data", \ - "Allocated but not filled" } + "User Data" } extern char *buftype[]; #define BT_BUFTYPE(type) \ type < BT_NUMBUFTYPES ? buftype[type] : buftype[BT_UNKNOWN] @@ -234,7 +232,7 @@ extern struct bufarea *pdirbp; /* current directory contents */ (bp)->b_flags |= B_DIRTY; \ } while (0) #define initbarea(bp, type) do { \ - (bp)->b_bno = (ufs2_daddr_t)-1; \ + (bp)->b_bno = (ufs2_daddr_t)-4; \ (bp)->b_size = 0; \ (bp)->b_errs = 0; \ (bp)->b_flags = 0; \ @@ -347,6 +345,7 @@ extern char *blockmap; /* ptr to primary blk allocation map */ extern char *cdevname; /* name of device being checked */ extern char ckclean; /* only do work if not cleanly unmounted */ extern int ckhashadd; /* check hashes to be added */ +extern char *copybuf; /* buffer to copy snapshot blocks */ extern int cvtlevel; /* convert to newer file system format */ extern long dev_bsize; /* computed value of DEV_BSIZE */ extern u_int real_dev_bsize; /* actual disk sector size, not overridden */ @@ -371,6 +370,8 @@ extern char resolved; /* cleared if unresolved changes => not clean */ extern int returntosingle; /* 1 => return to single user mode on exit */ extern long secsize; /* actual disk sector size */ extern char skipclean; /* skip clean file systems if preening */ +extern int snapcnt; /* number of active snapshots */ +extern struct inode snaplist[FSMAXSNAP + 1]; /* list of active snapshots */ extern char snapname[BUFSIZ]; /* when doing snapshots, the name of the file */ extern int sujrecovery; /* 1 => doing check using the journal */ extern int surrender; /* Give up if reads fail */ @@ -441,9 +442,11 @@ struct fstab; void adjust(struct inodesc *, int lcnt); void alarmhandler(int sig); -ufs2_daddr_t allocblk(long frags); +ufs2_daddr_t allocblk(long cg, long frags, ufs2_daddr_t (*checkblkavail) + (ufs2_daddr_t blkno, long frags)); ino_t allocdir(ino_t parent, ino_t request, int mode); ino_t allocino(ino_t request, int type); +void binval(struct bufarea *); void blkerror(ino_t ino, const char *type, ufs2_daddr_t blk); char *blockcheck(char *name); int blread(int fd, char *buf, ufs2_daddr_t blk, long size); @@ -458,12 +461,15 @@ void catchquit(int); void cgdirty(struct bufarea *); struct bufarea *cglookup(int cg); int changeino(ino_t dir, const char *name, ino_t newnum); +void check_blkcnt(struct inode *ip); int check_cgmagic(int cg, struct bufarea *cgbp, int requestrebuild); int chkrange(ufs2_daddr_t blk, int cnt); void ckfini(int markclean); int ckinode(union dinode *dp, struct inodesc *); void clri(struct inodesc *, const char *type, int flag); int clearentry(struct inodesc *); +void copyonwrite(struct fs *, struct bufarea *, + ufs2_daddr_t (*checkblkavail)(long, long)); void direrror(ino_t ino, const char *errmesg); int dirscan(struct inodesc *); int dofix(struct inodesc *, const char *msg); @@ -476,6 +482,7 @@ void flush(int fd, struct bufarea *bp); int freeblock(struct inodesc *); void freeino(ino_t ino); void freeinodebuf(void); +void fsckinit(void); void fsutilinit(void); int ftypeok(union dinode *dp); void getblk(struct bufarea *bp, ufs2_daddr_t blk, long size); @@ -484,6 +491,7 @@ struct inoinfo *getinoinfo(ino_t inumber); union dinode *getnextinode(ino_t inumber, int rebuildcg); void getpathname(char *namebuf, ino_t curdir, ino_t ino); void ginode(ino_t, struct inode *); +void gjournal_check(const char *filesys); void infohandler(int sig); void irelse(struct inode *); ufs2_daddr_t ino_blkatoff(union dinode *, ino_t, ufs_lbn_t, int *, @@ -505,6 +513,7 @@ void pass4(void); void pass5(void); void pfatal(const char *fmt, ...) __printflike(1, 2); void propagate(void); +void prtbuf(struct bufarea *, const char *, ...) __printflike(2, 3); void prtinode(struct inode *); void pwarn(const char *fmt, ...) __printflike(1, 2); int readsb(void); @@ -513,9 +522,13 @@ void rwerror(const char *mesg, ufs2_daddr_t blk); void sblock_init(void); void setinodebuf(int, ino_t); int setup(char *dev); -void gjournal_check(const char *filesys); +int snapblkfree(struct fs *, ufs2_daddr_t, long, ino_t, + ufs2_daddr_t (*)(ufs2_daddr_t, long)); +void snapremove(ino_t); +void snapflush(ufs2_daddr_t (*checkblkavail)(long, long)); +ufs2_daddr_t std_checkblkavail(ufs2_daddr_t blkno, long frags); +ufs2_daddr_t suj_checkblkavail(ufs2_daddr_t, long); int suj_check(const char *filesys); void update_maps(struct cg *, struct cg*, int); -void fsckinit(void); #endif /* !_FSCK_H_ */ diff --git a/sbin/fsck_ffs/fsutil.c b/sbin/fsck_ffs/fsutil.c index f1834e1235d8c..39e708a6cb324 100644 --- a/sbin/fsck_ffs/fsutil.c +++ b/sbin/fsck_ffs/fsutil.c @@ -71,7 +71,6 @@ static void cg_write(struct bufarea *); static void slowio_start(void); static void slowio_end(void); static void printIOstats(void); -static void prtbuf(const char *, struct bufarea *); static long diskreads, totaldiskreads, totalreads; /* Disk cache statistics */ static struct timespec startpass, finishpass; @@ -79,8 +78,10 @@ struct timeval slowio_starttime; int slowio_delay_usec = 10000; /* Initial IO delay for background fsck */ int slowio_pollcnt; static struct bufarea cgblk; /* backup buffer for cylinder group blocks */ +static struct bufarea failedbuf; /* returned by failed getdatablk() */ static TAILQ_HEAD(bufqueue, bufarea) bufqueuehd; /* head of buffer cache LRU */ static LIST_HEAD(bufhash, bufarea) bufhashhd[HASHSIZE]; /* buffer hash list */ +static struct bufhash freebufs; /* unused buffers */ static int numbufs; /* size of buffer cache */ static int cachelookups; /* number of cache lookups */ static int cachereads; /* number of cache reads */ @@ -187,11 +188,15 @@ bufinit(void) { int i; + initbarea(&failedbuf, BT_UNKNOWN); + failedbuf.b_errs = -1; + failedbuf.b_un.b_buf = NULL; if ((cgblk.b_un.b_buf = Malloc((unsigned int)sblock.fs_bsize)) == NULL) errx(EEXIT, "Initial malloc(%d) failed", sblock.fs_bsize); initbarea(&cgblk, BT_CYLGRP); numbufs = cachelookups = cachereads = 0; TAILQ_INIT(&bufqueuehd); + LIST_INIT(&freebufs); for (i = 0; i < HASHSIZE; i++) LIST_INIT(&bufhashhd[i]); for (i = 0; i < BT_NUMBUFTYPES; i++) { @@ -300,7 +305,7 @@ flushentry(void) } /* - * Manage a cache of directory blocks. + * Manage a cache of filesystem disk blocks. */ struct bufarea * getdatablk(ufs2_daddr_t blkno, long size, int type) @@ -309,16 +314,19 @@ getdatablk(ufs2_daddr_t blkno, long size, int type) struct bufhash *bhdp; cachelookups++; - /* If out of range, return empty buffer with b_err == -1 */ - if (type != BT_INODES && chkrange(blkno, size / sblock.fs_fsize)) { - blkno = -1; - type = BT_EMPTY; - } + /* + * If out of range, return empty buffer with b_err == -1 + * + * Skip check for inodes because chkrange() considers + * metadata areas invalid to write data. + */ + if (type != BT_INODES && chkrange(blkno, size / sblock.fs_fsize)) + return (&failedbuf); bhdp = &bufhashhd[HASH(blkno)]; LIST_FOREACH(bp, bhdp, b_hash) if (bp->b_bno == fsbtodb(&sblock, blkno)) { if (debug && bp->b_size != size) { - prtbuf("getdatablk: size mismatch", bp); + prtbuf(bp, "getdatablk: size mismatch"); pfatal("getdatablk: b_size %d != size %ld\n", bp->b_size, size); } @@ -340,7 +348,9 @@ getdatablk(ufs2_daddr_t blkno, long size, int type) if (size > sblock.fs_bsize) errx(EEXIT, "Excessive buffer size %ld > %d\n", size, sblock.fs_bsize); - if (numbufs < MINBUFS) { + if ((bp = LIST_FIRST(&freebufs)) != NULL) { + LIST_REMOVE(bp, b_hash); + } else if (numbufs < MINBUFS) { bp = allocbuf("cannot create minimal buffer pool"); } else if (sujrecovery) { /* @@ -378,7 +388,7 @@ getdatablk(ufs2_daddr_t blkno, long size, int type) if (debug && bp->b_type != type) { printf("getdatablk: buffer type changed to %s", BT_BUFTYPE(type)); - prtbuf("", bp); + prtbuf(bp, ""); } TAILQ_REMOVE(&bufqueuehd, bp, b_list); TAILQ_INSERT_HEAD(&bufqueuehd, bp, b_list); @@ -401,11 +411,7 @@ getblk(struct bufarea *bp, ufs2_daddr_t blk, long size) readcnt[bp->b_type]++; clock_gettime(CLOCK_REALTIME_PRECISE, &start); } - if (bp->b_type != BT_EMPTY) - bp->b_errs = - blread(fsreadfd, bp->b_un.b_buf, dblk, size); - else - bp->b_errs = -1; + bp->b_errs = blread(fsreadfd, bp->b_un.b_buf, dblk, size); if (debug) { clock_gettime(CLOCK_REALTIME_PRECISE, &finish); timespecsub(&finish, &start, &finish); @@ -422,10 +428,19 @@ brelse(struct bufarea *bp) { if (bp->b_refcnt <= 0) - prtbuf("brelse: buffer with negative reference count", bp); + prtbuf(bp, "brelse: buffer with negative reference count"); bp->b_refcnt--; } +void +binval(struct bufarea *bp) +{ + + bp->b_flags &= ~B_DIRTY; + LIST_REMOVE(bp, b_hash); + LIST_INSERT_HEAD(&freebufs, bp, b_hash); +} + void flush(int fd, struct bufarea *bp) { @@ -451,10 +466,18 @@ flush(int fd, struct bufarea *bp) if (bp != &sblk) pfatal("BUFFER %p DOES NOT MATCH SBLK %p\n", bp, &sblk); + /* + * Superblocks are always pre-copied so we do not need + * to check them for copy-on-write. + */ if (sbput(fd, bp->b_un.b_fs, 0) == 0) fsmodified = 1; break; case BT_CYLGRP: + /* + * Cylinder groups are always pre-copied so we do not + * need to check them for copy-on-write. + */ if (sujrecovery) cg_write(bp); if (cgput(fswritefd, &sblock, bp->b_un.b_cg) == 0) @@ -483,11 +506,38 @@ flush(int fd, struct bufarea *bp) } /* FALLTHROUGH */ default: + copyonwrite(&sblock, bp, std_checkblkavail); blwrite(fd, bp->b_un.b_buf, bp->b_bno, bp->b_size); break; } } +/* + * If there are any snapshots, ensure that all the blocks that they + * care about have been copied, then release the snapshot inodes. + * These operations need to be done before we rebuild the cylinder + * groups so that any block allocations are properly recorded. + * Since all the cylinder group maps have already been copied in + * the snapshots, no further snapshot copies will need to be done. + */ +void +snapflush(ufs2_daddr_t (*checkblkavail)(long, long)) +{ + struct bufarea *bp; + int cnt; + + if (snapcnt > 0) { + if (debug) + printf("Check for snapshot copies\n"); + TAILQ_FOREACH_REVERSE(bp, &bufqueuehd, bufqueue, b_list) + if ((bp->b_flags & B_DIRTY) != 0) + copyonwrite(&sblock, bp, checkblkavail); + for (cnt = 0; cnt < snapcnt; cnt++) + irelse(&snaplist[cnt]); + snapcnt = 0; + } +} + /* * Journaled soft updates does not maintain cylinder group summary * information during cleanup, so this routine recalculates the summary @@ -499,6 +549,7 @@ cg_write(struct bufarea *bp) { ufs1_daddr_t fragno, cgbno, maxbno; u_int8_t *blksfree; + struct csum *csp; struct cg *cgp; int blk; int i; @@ -536,6 +587,11 @@ cg_write(struct bufarea *bp) * Update the superblock cg summary from our now correct values * before writing the block. */ + csp = &sblock.fs_cs(&sblock, cgp->cg_cgx); + sblock.fs_cstotal.cs_ndir += cgp->cg_cs.cs_ndir - csp->cs_ndir; + sblock.fs_cstotal.cs_nbfree += cgp->cg_cs.cs_nbfree - csp->cs_nbfree; + sblock.fs_cstotal.cs_nifree += cgp->cg_cs.cs_nifree - csp->cs_nifree; + sblock.fs_cstotal.cs_nffree += cgp->cg_cs.cs_nffree - csp->cs_nffree; sblock.fs_cs(&sblock, cgp->cg_cgx) = cgp->cg_cs; } @@ -587,6 +643,7 @@ ckfini(int markclean) (void)close(fsreadfd); return; } + /* * To remain idempotent with partial truncations the buffers * must be flushed in this order: @@ -629,14 +686,9 @@ ckfini(int markclean) case BT_SUPERBLK: case BT_CYLGRP: default: - prtbuf("ckfini: improper buffer type on cache list",bp); + prtbuf(bp,"ckfini: improper buffer type on cache list"); continue; /* These are the ones to flush in this step */ - case BT_EMPTY: - if (bp->b_bno >= 0) - pfatal("Unused BT_EMPTY buffer for block %jd\n", - (intmax_t)bp->b_bno); - /* FALLTHROUGH */ case BT_LEVEL1: case BT_LEVEL2: case BT_LEVEL3: @@ -649,10 +701,11 @@ ckfini(int markclean) continue; } if (debug && bp->b_refcnt != 0) { - prtbuf("ckfini: clearing in-use buffer", bp); + prtbuf(bp, "ckfini: clearing in-use buffer"); pfatal("ckfini: clearing in-use buffer\n"); } TAILQ_REMOVE(&bufqueuehd, bp, b_list); + LIST_REMOVE(bp, b_hash); cnt++; flush(fswritefd, bp); free(bp->b_un.b_buf); @@ -667,10 +720,11 @@ ckfini(int markclean) } TAILQ_FOREACH_REVERSE_SAFE(bp, &bufqueuehd, bufqueue, b_list, nbp) { if (debug && bp->b_refcnt != 0) { - prtbuf("ckfini: clearing in-use buffer", bp); + prtbuf(bp, "ckfini: clearing in-use buffer"); pfatal("ckfini: clearing in-use buffer\n"); } TAILQ_REMOVE(&bufqueuehd, bp, b_list); + LIST_REMOVE(bp, b_hash); cnt++; flush(fswritefd, bp); free(bp->b_un.b_buf); @@ -1050,45 +1104,77 @@ check_cgmagic(int cg, struct bufarea *cgbp, int request_rebuild) * allocate a data block with the specified number of fragments */ ufs2_daddr_t -allocblk(long frags) +allocblk(long startcg, long frags, + ufs2_daddr_t (*checkblkavail)(ufs2_daddr_t blkno, long frags)) { - int i, j, k, cg, baseblk; - struct bufarea *cgbp; - struct cg *cgp; + ufs2_daddr_t blkno, newblk; + if (sujrecovery && checkblkavail == std_checkblkavail) { + pfatal("allocblk: std_checkblkavail used for SUJ recovery\n"); + return (0); + } if (frags <= 0 || frags > sblock.fs_frag) return (0); - for (i = 0; i < maxfsblock - sblock.fs_frag; i += sblock.fs_frag) { - for (j = 0; j <= sblock.fs_frag - frags; j++) { - if (testbmap(i + j)) - continue; - for (k = 1; k < frags; k++) - if (testbmap(i + j + k)) - break; - if (k < frags) { - j += k; - continue; - } - cg = dtog(&sblock, i + j); - cgbp = cglookup(cg); - cgp = cgbp->b_un.b_cg; - if (!check_cgmagic(cg, cgbp, 0)) { - i = (cg + 1) * sblock.fs_fpg - sblock.fs_frag; - continue; - } - baseblk = dtogd(&sblock, i + j); - for (k = 0; k < frags; k++) { - setbmap(i + j + k); - clrbit(cg_blksfree(cgp), baseblk + k); - } - n_blks += frags; - if (frags == sblock.fs_frag) - cgp->cg_cs.cs_nbfree--; - else - cgp->cg_cs.cs_nffree -= frags; - cgdirty(cgbp); - return (i + j); + for (blkno = cgdata(&sblock, startcg); + blkno < maxfsblock - sblock.fs_frag; + blkno += sblock.fs_frag) { + if ((newblk = (*checkblkavail)(blkno, frags)) == 0) + continue; + if (newblk > 0) + return (newblk); + if (newblk < 0) + blkno = -newblk; + } + for (blkno = cgdata(&sblock, 0); + blkno < cgbase(&sblock, startcg) - sblock.fs_frag; + blkno += sblock.fs_frag) { + if ((newblk = (*checkblkavail)(blkno, frags)) == 0) + continue; + if (newblk > 0) + return (newblk); + if (newblk < 0) + blkno = -newblk; + } + return (0); +} + +ufs2_daddr_t +std_checkblkavail(blkno, frags) + ufs2_daddr_t blkno; + long frags; +{ + struct bufarea *cgbp; + struct cg *cgp; + ufs2_daddr_t j, k, baseblk; + long cg; + + for (j = 0; j <= sblock.fs_frag - frags; j++) { + if (testbmap(blkno + j)) + continue; + for (k = 1; k < frags; k++) + if (testbmap(blkno + j + k)) + break; + if (k < frags) { + j += k; + continue; + } + cg = dtog(&sblock, blkno + j); + cgbp = cglookup(cg); + cgp = cgbp->b_un.b_cg; + if (!check_cgmagic(cg, cgbp, 0)) + return (-((cg + 1) * sblock.fs_fpg - sblock.fs_frag)); + baseblk = dtogd(&sblock, blkno + j); + for (k = 0; k < frags; k++) { + setbmap(blkno + j + k); + clrbit(cg_blksfree(cgp), baseblk + k); } + n_blks += frags; + if (frags == sblock.fs_frag) + cgp->cg_cs.cs_nbfree--; + else + cgp->cg_cs.cs_nffree -= frags; + cgdirty(cgbp); + return (blkno + j); } return (0); } @@ -1261,14 +1347,19 @@ dofix(struct inodesc *idesc, const char *msg) /* * Print details about a buffer. */ -static void -prtbuf(const char *msg, struct bufarea *bp) +void +prtbuf(struct bufarea *bp, const char *fmt, ...) { - - printf("%s: bp %p, type %s, bno %jd, size %d, refcnt %d, flags %s, " - "index %jd\n", msg, bp, BT_BUFTYPE(bp->b_type), - (intmax_t) bp->b_bno, bp->b_size, bp->b_refcnt, - bp->b_flags & B_DIRTY ? "dirty" : "clean", (intmax_t) bp->b_index); + va_list ap; + va_start(ap, fmt); + if (preen) + (void)fprintf(stdout, "%s: ", cdevname); + (void)vfprintf(stdout, fmt, ap); + va_end(ap); + printf(": bp %p, type %s, bno %jd, size %d, refcnt %d, flags %s, " + "index %jd\n", bp, BT_BUFTYPE(bp->b_type), (intmax_t) bp->b_bno, + bp->b_size, bp->b_refcnt, bp->b_flags & B_DIRTY ? "dirty" : "clean", + (intmax_t) bp->b_index); } /* diff --git a/sbin/fsck_ffs/inode.c b/sbin/fsck_ffs/inode.c index c261b254ace94..1d36a6ca3e45a 100644 --- a/sbin/fsck_ffs/inode.c +++ b/sbin/fsck_ffs/inode.c @@ -38,6 +38,7 @@ static const char sccsid[] = "@(#)inode.c 8.8 (Berkeley) 4/28/95"; __FBSDID("$FreeBSD$"); #include +#include #include #include @@ -58,6 +59,9 @@ struct bufarea *icachebp; /* inode cache buffer */ static int iblock(struct inodesc *, off_t isize, int type); static ufs2_daddr_t indir_blkatoff(ufs2_daddr_t, ino_t, ufs_lbn_t, ufs_lbn_t, struct bufarea **); +static int snapclean(struct inodesc *idesc); +static void chkcopyonwrite(struct fs *, ufs2_daddr_t, + ufs2_daddr_t (*checkblkavail)(long, long)); int ckinode(union dinode *dp, struct inodesc *idesc) @@ -378,8 +382,12 @@ chkrange(ufs2_daddr_t blk, int cnt) int c; if (cnt <= 0 || blk <= 0 || blk > maxfsblock || - cnt - 1 > maxfsblock - blk) + cnt - 1 > maxfsblock - blk) { + if (debug) + printf("out of range: blk %ld, offset %i, size %d\n", + (long)blk, (int)fragnum(&sblock, blk), cnt); return (1); + } if (cnt > sblock.fs_frag || fragnum(&sblock, blk) + cnt > sblock.fs_frag) { if (debug) @@ -650,11 +658,21 @@ int freeblock(struct inodesc *idesc) { struct dups *dlp; + struct bufarea *cgbp; + struct cg *cgp; ufs2_daddr_t blkno; - long nfrags, res; + long size, nfrags, res; res = KEEPON; blkno = idesc->id_blkno; + if (idesc->id_type == SNAP) { + pfatal("clearing a snapshot dinode\n"); + return (STOP); + } + size = lfragtosize(&sblock, idesc->id_numfrags); + if (snapblkfree(&sblock, blkno, size, idesc->id_number, + std_checkblkavail)) + return (res); for (nfrags = idesc->id_numfrags; nfrags > 0; blkno++, nfrags--) { if (chkrange(blkno, 1)) { res = SKIP; @@ -674,12 +692,407 @@ freeblock(struct inodesc *idesc) } } } + /* + * If all successfully returned, account for them. + */ + if (nfrags == 0) { + cgbp = cglookup(dtog(&sblock, idesc->id_blkno)); + cgp = cgbp->b_un.b_cg; + if (idesc->id_numfrags == sblock.fs_frag) + cgp->cg_cs.cs_nbfree++; + else + cgp->cg_cs.cs_nffree += idesc->id_numfrags; + cgdirty(cgbp); + } return (res); } +/* + * Prepare a snapshot file for being removed. + */ +void +snapremove(ino_t inum) +{ + struct inodesc idesc; + struct inode ip; + int i; + + for (i = 0; i < snapcnt; i++) + if (snaplist[i].i_number == inum) + break; + if (i == snapcnt) + ginode(inum, &ip); + else + ip = snaplist[i]; + if ((DIP(ip.i_dp, di_flags) & SF_SNAPSHOT) == 0) { + printf("snapremove: inode %jd is not a snapshot\n", + (intmax_t)inum); + if (i == snapcnt) + irelse(&ip); + return; + } + if (debug) + printf("snapremove: remove %sactive snapshot %jd\n", + i == snapcnt ? "in" : "", (intmax_t)inum); + /* + * If on active snapshot list, remove it. + */ + if (i < snapcnt) { + for (i++; i < FSMAXSNAP; i++) { + if (sblock.fs_snapinum[i] == 0) + break; + snaplist[i - 1] = snaplist[i]; + sblock.fs_snapinum[i - 1] = sblock.fs_snapinum[i]; + } + sblock.fs_snapinum[i - 1] = 0; + bzero(&snaplist[i - 1], sizeof(struct inode)); + snapcnt--; + } + idesc.id_type = SNAP; + idesc.id_func = snapclean; + idesc.id_number = inum; + (void)ckinode(ip.i_dp, &idesc); + DIP_SET(ip.i_dp, di_flags, DIP(ip.i_dp, di_flags) & ~SF_SNAPSHOT); + inodirty(&ip); + irelse(&ip); +} + +static int +snapclean(struct inodesc *idesc) +{ + ufs2_daddr_t blkno; + struct bufarea *bp; + union dinode *dp; + + blkno = idesc->id_blkno; + if (blkno == 0) + return (KEEPON); + + bp = idesc->id_bp; + dp = idesc->id_dp; + if (blkno == BLK_NOCOPY || blkno == BLK_SNAP) { + if (idesc->id_lbn < UFS_NDADDR) + DIP_SET(dp, di_db[idesc->id_lbn], 0); + else + IBLK_SET(bp, bp->b_index, 0); + dirty(bp); + } + return (KEEPON); +} + +/* + * Notification that a block is being freed. Return zero if the free + * should be allowed to proceed. Return non-zero if the snapshot file + * wants to claim the block. The block will be claimed if it is an + * uncopied part of one of the snapshots. It will be freed if it is + * either a BLK_NOCOPY or has already been copied in all of the snapshots. + * If a fragment is being freed, then all snapshots that care about + * it must make a copy since a snapshot file can only claim full sized + * blocks. Note that if more than one snapshot file maps the block, + * we can pick one at random to claim it. Since none of the snapshots + * can change, we are assurred that they will all see the same unmodified + * image. When deleting a snapshot file (see ino_trunc above), we + * must push any of these claimed blocks to one of the other snapshots + * that maps it. These claimed blocks are easily identified as they will + * have a block number equal to their logical block number within the + * snapshot. A copied block can never have this property because they + * must always have been allocated from a BLK_NOCOPY location. + */ +int +snapblkfree(fs, bno, size, inum, checkblkavail) + struct fs *fs; + ufs2_daddr_t bno; + long size; + ino_t inum; + ufs2_daddr_t (*checkblkavail)(long cg, long frags); +{ + union dinode *dp; + struct inode ip; + struct bufarea *snapbp; + ufs_lbn_t lbn; + ufs2_daddr_t blkno, relblkno; + int i, frags, claimedblk, copydone; + + /* If no snapshots, nothing to do */ + if (snapcnt == 0) + return (0); + if (debug) + printf("snapblkfree: in ino %ld free blkno %ld, size %ld\n", + inum, bno, size); + relblkno = blknum(fs, bno); + lbn = fragstoblks(fs, relblkno); + /* Direct blocks are always pre-copied */ + if (lbn < UFS_NDADDR) + return (0); + copydone = 0; + claimedblk = 0; + for (i = 0; i < snapcnt; i++) { + /* + * Lookup block being freed. + */ + ip = snaplist[i]; + dp = ip.i_dp; + blkno = ino_blkatoff(dp, inum != 0 ? inum : ip.i_number, + lbn, &frags, &snapbp); + /* + * Check to see if block needs to be copied. + */ + if (blkno == 0) { + /* + * A block that we map is being freed. If it has not + * been claimed yet, we will claim or copy it (below). + */ + claimedblk = 1; + } else if (blkno == BLK_SNAP) { + /* + * No previous snapshot claimed the block, + * so it will be freed and become a BLK_NOCOPY + * (don't care) for us. + */ + if (claimedblk) + pfatal("snapblkfree: inconsistent block type"); + IBLK_SET(snapbp, snapbp->b_index, BLK_NOCOPY); + dirty(snapbp); + brelse(snapbp); + continue; + } else /* BLK_NOCOPY or default */ { + /* + * If the snapshot has already copied the block + * (default), or does not care about the block, + * it is not needed. + */ + brelse(snapbp); + continue; + } + /* + * If this is a full size block, we will just grab it + * and assign it to the snapshot inode. Otherwise we + * will proceed to copy it. See explanation for this + * routine as to why only a single snapshot needs to + * claim this block. + */ + if (size == fs->fs_bsize) { + if (debug) + printf("Grabonremove snapshot %ju lbn %jd " + "from inum %ju\n", (intmax_t)ip.i_number, + (intmax_t)lbn, (uintmax_t)inum); + IBLK_SET(snapbp, snapbp->b_index, relblkno); + dirty(snapbp); + brelse(snapbp); + DIP_SET(dp, di_blocks, + DIP(dp, di_blocks) + btodb(size)); + inodirty(&ip); + return (1); + } + + /* First time through, read the contents of the old block. */ + if (copydone == 0) { + copydone = 1; + if (blread(fsreadfd, copybuf, fsbtodb(fs, relblkno), + fs->fs_bsize) != 0) { + pfatal("Could not read snapshot %ju block " + "%jd\n", (intmax_t)ip.i_number, + (intmax_t)relblkno); + continue; + } + } + /* + * This allocation will never require any additional + * allocations for the snapshot inode. + */ + blkno = allocblk(dtog(fs, relblkno), fs->fs_frag, + checkblkavail); + if (blkno == 0) { + pfatal("Could not allocate block for snapshot %ju\n", + (intmax_t)ip.i_number); + continue; + } + if (debug) + printf("Copyonremove: snapino %jd lbn %jd for inum %ju " + "size %ld new blkno %jd\n", (intmax_t)ip.i_number, + (intmax_t)lbn, (uintmax_t)inum, size, + (intmax_t)blkno); + blwrite(fswritefd, copybuf, fsbtodb(fs, blkno), fs->fs_bsize); + IBLK_SET(snapbp, snapbp->b_index, blkno); + dirty(snapbp); + brelse(snapbp); + DIP_SET(dp, di_blocks, + DIP(dp, di_blocks) + btodb(fs->fs_bsize)); + inodirty(&ip); + } + return (0); +} + +/* + * Notification that a block is being written. Return if the block + * is part of a snapshot as snapshots never track other snapshots. + * The block will be copied in all of the snapshots that are tracking + * it and have not yet copied it. Some buffers may hold more than one + * block. Here we need to check each block in the buffer. + */ +void +copyonwrite(fs, bp, checkblkavail) + struct fs *fs; + struct bufarea *bp; + ufs2_daddr_t (*checkblkavail)(long cg, long frags); +{ + ufs2_daddr_t copyblkno; + long i, numblks; + + /* If no snapshots, nothing to do. */ + if (snapcnt == 0) + return; + numblks = blkroundup(fs, bp->b_size) / fs->fs_bsize; + if (debug) + prtbuf(bp, "copyonwrite: checking %jd block%s in buffer", + numblks, numblks > 1 ? "s" : ""); + copyblkno = blknum(fs, dbtofsb(fs, bp->b_bno)); + for (i = 0; i < numblks; i++) { + chkcopyonwrite(fs, copyblkno, checkblkavail); + copyblkno += fs->fs_frag; + } +} + +static void +chkcopyonwrite(fs, copyblkno, checkblkavail) + struct fs *fs; + ufs2_daddr_t copyblkno; + ufs2_daddr_t (*checkblkavail)(long cg, long frags); +{ + struct inode ip; + union dinode *dp; + struct bufarea *snapbp; + ufs2_daddr_t blkno; + int i, frags, copydone; + ufs_lbn_t lbn; + + lbn = fragstoblks(fs, copyblkno); + /* Direct blocks are always pre-copied */ + if (lbn < UFS_NDADDR) + return; + copydone = 0; + for (i = 0; i < snapcnt; i++) { + /* + * Lookup block being freed. + */ + ip = snaplist[i]; + dp = ip.i_dp; + blkno = ino_blkatoff(dp, ip.i_number, lbn, &frags, &snapbp); + /* + * Check to see if block needs to be copied. + */ + if (blkno != 0) { + /* + * A block that we have already copied or don't track. + */ + brelse(snapbp); + continue; + } + /* First time through, read the contents of the old block. */ + if (copydone == 0) { + copydone = 1; + if (blread(fsreadfd, copybuf, fsbtodb(fs, copyblkno), + fs->fs_bsize) != 0) { + pfatal("Could not read snapshot %ju block " + "%jd\n", (intmax_t)ip.i_number, + (intmax_t)copyblkno); + continue; + } + } + /* + * This allocation will never require any additional + * allocations for the snapshot inode. + */ + if ((blkno = allocblk(dtog(fs, copyblkno), fs->fs_frag, + checkblkavail)) == 0) { + pfatal("Could not allocate block for snapshot %ju\n", + (intmax_t)ip.i_number); + continue; + } + if (debug) + prtbuf(snapbp, "Copyonwrite: snapino %jd lbn %jd using " + "blkno %ju setting in buffer", + (intmax_t)ip.i_number, (intmax_t)lbn, + (intmax_t)blkno); + blwrite(fswritefd, copybuf, fsbtodb(fs, blkno), fs->fs_bsize); + IBLK_SET(snapbp, snapbp->b_index, blkno); + dirty(snapbp); + brelse(snapbp); + DIP_SET(dp, di_blocks, + DIP(dp, di_blocks) + btodb(fs->fs_bsize)); + inodirty(&ip); + } + return; +} + +/* + * Traverse an inode and check that its block count is correct + * fixing it if necessary. + */ +void +check_blkcnt(struct inode *ip) +{ + struct inodesc idesc; + union dinode *dp; + ufs2_daddr_t ndb; + int j, ret, offset; + + dp = ip->i_dp; + memset(&idesc, 0, sizeof(struct inodesc)); + idesc.id_func = pass1check; + idesc.id_number = ip->i_number; + idesc.id_type = (DIP(dp, di_flags) & SF_SNAPSHOT) == 0 ? ADDR : SNAP; + (void)ckinode(dp, &idesc); + if (sblock.fs_magic == FS_UFS2_MAGIC && dp->dp2.di_extsize > 0) { + ndb = howmany(dp->dp2.di_extsize, sblock.fs_bsize); + for (j = 0; j < UFS_NXADDR; j++) { + if (--ndb == 0 && + (offset = blkoff(&sblock, dp->dp2.di_extsize)) != 0) + idesc.id_numfrags = numfrags(&sblock, + fragroundup(&sblock, offset)); + else + idesc.id_numfrags = sblock.fs_frag; + if (dp->dp2.di_extb[j] == 0) + continue; + idesc.id_blkno = dp->dp2.di_extb[j]; + ret = (*idesc.id_func)(&idesc); + if (ret & STOP) + break; + } + } + idesc.id_entryno *= btodb(sblock.fs_fsize); + if (DIP(dp, di_blocks) != idesc.id_entryno) { + if (!(sujrecovery && preen)) { + pwarn("INCORRECT BLOCK COUNT I=%lu (%ju should be %ju)", + (u_long)idesc.id_number, + (uintmax_t)DIP(dp, di_blocks), + (uintmax_t)idesc.id_entryno); + if (preen) + printf(" (CORRECTED)\n"); + else if (reply("CORRECT") == 0) + return; + } + if (bkgrdflag == 0) { + DIP_SET(dp, di_blocks, idesc.id_entryno); + inodirty(ip); + } else { + cmd.value = idesc.id_number; + cmd.size = idesc.id_entryno - DIP(dp, di_blocks); + if (debug) + printf("adjblkcnt ino %ju amount %lld\n", + (uintmax_t)cmd.value, (long long)cmd.size); + if (sysctl(adjblkcnt, MIBSIZE, 0, 0, + &cmd, sizeof cmd) == -1) + rwerror("ADJUST INODE BLOCK COUNT", cmd.value); + } + } +} + void freeinodebuf(void) { + struct bufarea *bp; + int i; /* * Flush old contents in case they have been updated. @@ -689,6 +1102,14 @@ freeinodebuf(void) free((char *)inobuf.b_un.b_buf); inobuf.b_un.b_buf = NULL; firstinum = lastinum = 0; + /* + * Reload the snapshot inodes in case any of them changed. + */ + for (i = 0; i < snapcnt; i++) { + bp = snaplist[i].i_bp; + bp->b_errs = blread(fsreadfd, bp->b_un.b_buf, bp->b_bno, + bp->b_size); + } } /* @@ -720,6 +1141,7 @@ cacheino(union dinode *dp, ino_t inumber) inpp = &inphead[inumber % dirhash]; inp->i_nexthash = *inpp; *inpp = inp; + inp->i_flags = 0; inp->i_parent = inumber == UFS_ROOTINO ? UFS_ROOTINO : (ino_t)0; inp->i_dotdot = (ino_t)0; inp->i_number = inumber; @@ -803,6 +1225,10 @@ clri(struct inodesc *idesc, const char *type, int flag) printf(" (CLEARED)\n"); n_files--; if (bkgrdflag == 0) { + if (idesc->id_type == SNAP) { + snapremove(idesc->id_number); + idesc->id_type = ADDR; + } (void)ckinode(dp, idesc); inoinfo(idesc->id_number)->ino_state = USTATE; clearinode(dp); @@ -967,7 +1393,8 @@ allocino(ino_t request, int type) cgdirty(cgbp); ginode(ino, &ip); dp = ip.i_dp; - DIP_SET(dp, di_db[0], allocblk((long)1)); + DIP_SET(dp, di_db[0], allocblk(ino_to_cg(&sblock, ino), (long)1, + std_checkblkavail)); if (DIP(dp, di_db[0]) == 0) { inoinfo(ino)->ino_state = USTATE; irelse(&ip); diff --git a/sbin/fsck_ffs/main.c b/sbin/fsck_ffs/main.c index b638d31040dd0..7e4dcc3e1e32b 100644 --- a/sbin/fsck_ffs/main.c +++ b/sbin/fsck_ffs/main.c @@ -491,6 +491,7 @@ checkfilesys(char *filesys) */ if (preen == 0) printf("** Phase 5 - Check Cyl groups\n"); + snapflush(std_checkblkavail); pass5(); IOstats("Pass5"); diff --git a/sbin/fsck_ffs/setup.c b/sbin/fsck_ffs/setup.c index 69b027a3a5c82..527d125dd6705 100644 --- a/sbin/fsck_ffs/setup.c +++ b/sbin/fsck_ffs/setup.c @@ -59,6 +59,9 @@ __FBSDID("$FreeBSD$"); #include "fsck.h" struct inoinfo **inphead, **inpsort; /* info about all inodes */ +struct inode snaplist[FSMAXSNAP + 1]; /* list of active snapshots */ +int snapcnt; /* number of active snapshots */ +char *copybuf; /* buffer to copy snapshot blocks */ static int sbhashfailed; #define POWEROF2(num) (((num) & ((num) - 1)) == 0) @@ -66,6 +69,8 @@ static int sbhashfailed; static int calcsb(char *dev, int devfd, struct fs *fs); static void saverecovery(int readfd, int writefd); static int chkrecovery(int devfd); +static int getlbnblkno(struct inodesc *); +static int checksnapinfo(struct inode *); /* * Read in a superblock finding an alternate if necessary. @@ -75,7 +80,8 @@ static int chkrecovery(int devfd); int setup(char *dev) { - long bmapsize; + long i, bmapsize; + struct inode ip; /* * We are expected to have an open file descriptor and a superblock. @@ -174,6 +180,42 @@ setup(char *dev) usedsoftdep = 1; else usedsoftdep = 0; + /* + * Collect any snapshot inodes so that we can allow them to + * claim any blocks that we free. The code for doing this is + * imported here and into inode.c from sys/ufs/ffs/ffs_snapshot.c. + */ + for (snapcnt = 0; snapcnt < FSMAXSNAP; snapcnt++) { + if (sblock.fs_snapinum[snapcnt] == 0) + break; + ginode(sblock.fs_snapinum[snapcnt], &ip); + if ((DIP(ip.i_dp, di_mode) & IFMT) == IFREG && + (DIP(ip.i_dp, di_flags) & SF_SNAPSHOT) != 0 && + checksnapinfo(&ip)) { + if (debug) + printf("Load snapshot %jd\n", + (intmax_t)sblock.fs_snapinum[snapcnt]); + snaplist[snapcnt] = ip; + continue; + } + printf("Removing non-snapshot inode %ju from snapshot list\n", + (uintmax_t)sblock.fs_snapinum[snapcnt]); + irelse(&ip); + for (i = snapcnt + 1; i < FSMAXSNAP; i++) { + if (sblock.fs_snapinum[i] == 0) + break; + sblock.fs_snapinum[i - 1] = sblock.fs_snapinum[i]; + } + sblock.fs_snapinum[i - 1] = 0; + snapcnt--; + sbdirty(); + } + if (snapcnt > 0 && copybuf == NULL) { + copybuf = Malloc(sblock.fs_bsize); + if (copybuf == NULL) + errx(EEXIT, "cannot allocate space for snapshot " + "copy buffer"); + } return (1); badsb: @@ -181,6 +223,144 @@ setup(char *dev) return (0); } +/* + * Check for valid snapshot information. + * + * Each snapshot has a list of blocks that have been copied. This list + * is consulted before checking the snapshot inode. Its purpose is to + * speed checking of commonly checked blocks and to avoid recursive + * checks of the snapshot inode. In particular, the list must contain + * the superblock, the superblock summary information, and all the + * cylinder group blocks. The list may contain other commonly checked + * pointers such as those of the blocks that contain the snapshot inodes. + * The list is sorted into block order to allow binary search lookup. + * + * The twelve direct direct block pointers of the snapshot are always + * copied, so we test for them first before checking the list itself + * (i.e., they are not in the list). + * + * The checksnapinfo() routine needs to ensure that the list contains at + * least the super block, its summary information, and the cylinder groups. + * Here we check the list first for the superblock, zero or more cylinder + * groups up to the location of the superblock summary information, the + * summary group information, and any remaining cylinder group maps that + * follow it. We skip over any other entries in the list. + */ +#define CHKBLKINLIST(chkblk) \ + /* All UFS_NDADDR blocks are copied */ \ + if ((chkblk) >= UFS_NDADDR) { \ + /* Skip over blocks that are not of interest */ \ + while (*blkp < (chkblk) && blkp < lastblkp) \ + blkp++; \ + /* Fail if end of list and not all blocks found */ \ + if (blkp >= lastblkp) { \ + pwarn("UFS%d snapshot inode %jd failed: " \ + "improper block list length (%jd)\n", \ + sblock.fs_magic == FS_UFS1_MAGIC ? 1 : 2, \ + (intmax_t)snapip->i_number, \ + (intmax_t)(lastblkp - &snapblklist[0])); \ + status = 0; \ + } \ + /* Fail if block we seek is missing */ \ + else if (*blkp++ != (chkblk)) { \ + pwarn("UFS%d snapshot inode %jd failed: " \ + "block list (%jd) != %s (%jd)\n", \ + sblock.fs_magic == FS_UFS1_MAGIC ? 1 : 2, \ + (intmax_t)snapip->i_number, \ + (intmax_t)blkp[-1], #chkblk, \ + (intmax_t)chkblk); \ + status = 0; \ + } \ + } + +static int +checksnapinfo(struct inode *snapip) +{ + struct fs *fs; + struct bufarea *bp; + struct inodesc idesc; + daddr_t *snapblklist, *blkp, *lastblkp, csblkno; + int cg, loc, len, status; + ufs_lbn_t lbn; + size_t size; + + fs = &sblock; + memset(&idesc, 0, sizeof(struct inodesc)); + idesc.id_type = ADDR; + idesc.id_func = getlbnblkno; + idesc.id_number = snapip->i_number; + lbn = howmany(fs->fs_size, fs->fs_frag); + idesc.id_parent = lbn; /* sought after blkno */ + if ((ckinode(snapip->i_dp, &idesc) & FOUND) == 0) + return (0); + size = fragroundup(fs, + DIP(snapip->i_dp, di_size) - lblktosize(fs, lbn)); + bp = getdatablk(idesc.id_parent, size, BT_DATA); + snapblklist = (daddr_t *)bp->b_un.b_buf; + /* + * snapblklist[0] is the size of the list + * snapblklist[1] is the first element of the list + * + * We need to be careful to bound the size of the list and verify + * that we have not run off the end of it if it or its size has + * been corrupted. + */ + blkp = &snapblklist[1]; + lastblkp = &snapblklist[MAX(0, + MIN(snapblklist[0] + 1, size / sizeof(daddr_t)))]; + status = 1; + /* Check that the superblock is listed. */ + CHKBLKINLIST(lblkno(fs, fs->fs_sblockloc)); + if (status == 0) + goto out; + /* + * Calculate where the summary information is located. + * Usually it is in the first cylinder group, but growfs + * may move it to the first cylinder group that it adds. + * + * Check all cylinder groups up to the summary information. + */ + csblkno = fragstoblks(fs, fs->fs_csaddr); + for (cg = 0; cg < fs->fs_ncg; cg++) { + if (fragstoblks(fs, cgtod(fs, cg)) > csblkno) + break; + CHKBLKINLIST(fragstoblks(fs, cgtod(fs, cg))); + if (status == 0) + goto out; + } + /* Check the summary information block(s). */ + len = howmany(fs->fs_cssize, fs->fs_bsize); + for (loc = 0; loc < len; loc++) { + CHKBLKINLIST(csblkno + loc); + if (status == 0) + goto out; + } + /* Check the remaining cylinder groups. */ + for (; cg < fs->fs_ncg; cg++) { + CHKBLKINLIST(fragstoblks(fs, cgtod(fs, cg))); + if (status == 0) + goto out; + } +out: + brelse(bp); + return (status); +} + +/* + * Return the block number associated with a specified inode lbn. + * Requested lbn is in id_parent. If found, block is returned in + * id_parent. + */ +static int +getlbnblkno(struct inodesc *idesc) +{ + + if (idesc->id_lbn < idesc->id_parent) + return (KEEPON); + idesc->id_parent = idesc->id_blkno; + return (STOP | FOUND); +} + /* * Open a device or file to be checked by fsck. */ diff --git a/sbin/fsck_ffs/suj.c b/sbin/fsck_ffs/suj.c index 4ad86b0dc14d7..b120bd3dd38c5 100644 --- a/sbin/fsck_ffs/suj.c +++ b/sbin/fsck_ffs/suj.c @@ -321,7 +321,7 @@ blk_freemask(ufs2_daddr_t blk, ino_t ino, ufs_lbn_t lbn, int frags) * To be certain we're not freeing a reallocated block we lookup * this block in the blk hash and see if there is an allocation * journal record that overlaps with any fragments in the block - * we're concerned with. If any fragments have ben reallocated + * we're concerned with. If any fragments have been reallocated * the block has already been freed and re-used for another purpose. */ mask = 0; @@ -378,6 +378,50 @@ blk_isindir(ufs2_daddr_t blk, ino_t ino, ufs_lbn_t lbn) return (0); } +/* + * Check to see if the requested block is available. + * We can just check in the cylinder-group maps as + * they will only have usable blocks in them. + */ +ufs2_daddr_t +suj_checkblkavail(blkno, frags) + ufs2_daddr_t blkno; + long frags; +{ + struct bufarea *cgbp; + struct cg *cgp; + ufs2_daddr_t j, k, baseblk; + long cg; + + cg = dtog(&sblock, blkno); + cgbp = cglookup(cg); + cgp = cgbp->b_un.b_cg; + if (!check_cgmagic(cg, cgbp, 0)) + return (-((cg + 1) * sblock.fs_fpg - sblock.fs_frag)); + baseblk = dtogd(&sblock, blkno); + for (j = 0; j <= sblock.fs_frag - frags; j++) { + if (!isset(cg_blksfree(cgp), baseblk + j)) + continue; + for (k = 1; k < frags; k++) + if (!isset(cg_blksfree(cgp), baseblk + j + k)) + break; + if (k < frags) { + j += k; + continue; + } + for (k = 0; k < frags; k++) + clrbit(cg_blksfree(cgp), baseblk + j + k); + n_blks += frags; + if (frags == sblock.fs_frag) + cgp->cg_cs.cs_nbfree--; + else + cgp->cg_cs.cs_nffree -= frags; + cgdirty(cgbp); + return ((cg * sblock.fs_fpg) + baseblk + j); + } + return (0); +} + /* * Clear an inode from the cg bitmap. If the inode was already clear return * 0 so the caller knows it does not have to check the inode contents. @@ -420,7 +464,7 @@ ino_free(ino_t ino, int mode) * set in the mask. */ static void -blk_free(ufs2_daddr_t bno, int mask, int frags) +blk_free(ino_t ino, ufs2_daddr_t bno, int mask, int frags) { ufs1_daddr_t fragno, cgbno; struct suj_cg *sc; @@ -431,6 +475,13 @@ blk_free(ufs2_daddr_t bno, int mask, int frags) if (debug) printf("Freeing %d frags at blk %jd mask 0x%x\n", frags, bno, mask); + /* + * Check to see if the block needs to be claimed by a snapshot. + * If wanted, the snapshot references it. Otherwise we free it. + */ + if (snapblkfree(fs, bno, lfragtosize(fs, frags), ino, + suj_checkblkavail)) + return; cg = dtog(fs, bno); sc = cg_lookup(cg); cgp = sc->sc_cgp; @@ -846,7 +897,7 @@ static void blk_free_visit(ino_t ino, ufs_lbn_t lbn, ufs2_daddr_t blk, int frags) { - blk_free(blk, blk_freemask(blk, ino, lbn, frags), frags); + blk_free(ino, blk, blk_freemask(blk, ino, lbn, frags), frags); } /* @@ -865,7 +916,7 @@ blk_free_lbn(ufs2_daddr_t blk, ino_t ino, ufs_lbn_t lbn, int frags, int follow) if (lbn <= -UFS_NDADDR && follow && mask == 0) indir_visit(ino, lbn, blk, &resid, blk_free_visit, VISIT_INDIR); else - blk_free(blk, mask, frags); + blk_free(ino, blk, mask, frags); } static void @@ -997,6 +1048,8 @@ ino_reclaim(struct inode *ip, ino_t ino, int mode) if ((DIP(dp, di_mode) & IFMT) == IFDIR) ino_visit(dp, ino, ino_free_children, 0); DIP_SET(dp, di_nlink, 0); + if ((DIP(dp, di_flags) & SF_SNAPSHOT) != 0) + snapremove(ino); ino_visit(dp, ino, blk_free_visit, VISIT_EXT | VISIT_INDIR); /* Here we have to clear the inode and release any blocks it holds. */ gen = DIP(dp, di_gen); @@ -1209,7 +1262,7 @@ indir_trunc(ino_t ino, ufs_lbn_t lbn, ufs2_daddr_t blk, ufs_lbn_t lastlbn, continue; } isdirty = 1; - blk_free(nblk, 0, fs->fs_frag); + blk_free(ino, nblk, 0, fs->fs_frag); IBLK_SET(bp, i, 0); } if (isdirty) @@ -1245,6 +1298,11 @@ ino_trunc(ino_t ino, off_t size) dp = ip.i_dp; mode = DIP(dp, di_mode) & IFMT; cursize = DIP(dp, di_size); + /* If no size change, nothing to do */ + if (size == cursize) { + irelse(&ip); + return; + } if (debug) printf("Truncating ino %ju, mode %o to size %jd from size %jd\n", (uintmax_t)ino, mode, size, cursize); @@ -1264,13 +1322,14 @@ ino_trunc(ino_t ino, off_t size) if (size > 0) err_suj("Partial truncation of ino %ju snapshot file\n", (uintmax_t)ino); + snapremove(ino); } lastlbn = lblkno(fs, blkroundup(fs, size)); for (i = lastlbn; i < UFS_NDADDR; i++) { if ((bn = DIP(dp, di_db[i])) == 0) continue; blksize = sblksize(fs, cursize, i); - blk_free(bn, 0, numfrags(fs, blksize)); + blk_free(ino, bn, 0, numfrags(fs, blksize)); DIP_SET(dp, di_db[i], 0); } /* @@ -1283,13 +1342,13 @@ ino_trunc(ino_t ino, off_t size) /* If we're not freeing any in this indirect range skip it. */ if (lastlbn >= nextlbn) continue; - if (DIP(dp, di_ib[i]) == 0) - continue; - indir_trunc(ino, -lbn - i, DIP(dp, di_ib[i]), lastlbn, dp); - /* If we freed everything in this indirect free the indir. */ - if (lastlbn > lbn) - continue; - blk_free(DIP(dp, di_ib[i]), 0, fs->fs_frag); + if ((bn = DIP(dp, di_ib[i])) == 0) + continue; + indir_trunc(ino, -lbn - i, bn, lastlbn, dp); + /* If we freed everything in this indirect free the indir. */ + if (lastlbn > lbn) + continue; + blk_free(ino, bn, 0, fs->fs_frag); DIP_SET(dp, di_ib[i], 0); } /* @@ -1319,7 +1378,7 @@ ino_trunc(ino_t ino, off_t size) if (oldspace != newspace) { bn += numfrags(fs, newspace); frags = numfrags(fs, oldspace - newspace); - blk_free(bn, 0, frags); + blk_free(ino, bn, 0, frags); totalfrags -= frags; } } @@ -1468,7 +1527,7 @@ blk_check(struct suj_blk *sblk) mask >>= frags; blk += frags; frags = brec->jb_frags - frags; - blk_free(blk, mask, frags); + blk_free(brec->jb_ino, blk, mask, frags); continue; } /* @@ -2406,6 +2465,13 @@ suj_check(const char *filesys) } if (preen == 0 && (jrecs > 0 || jbytes > 0) && reply("WRITE CHANGES") == 0) return (0); + /* + * Check block counts of snapshot inodes and + * make copies of any needed snapshot blocks. + */ + for (i = 0; i < snapcnt; i++) + check_blkcnt(&snaplist[i]); + snapflush(suj_checkblkavail); /* * Recompute the fs summary info from correct cs summaries. */ diff --git a/sys/ufs/ffs/ffs_snapshot.c b/sys/ufs/ffs/ffs_snapshot.c index bcdf94a10cda4..7ee5dd2bfa43a 100644 --- a/sys/ufs/ffs/ffs_snapshot.c +++ b/sys/ufs/ffs/ffs_snapshot.c @@ -226,15 +226,6 @@ ffs_snapshot(struct mount *mp, char *snapfile) ump = VFSTOUFS(mp); fs = ump->um_fs; sn = NULL; - /* - * At the moment, journaled soft updates cannot support - * taking snapshots. - */ - if (MOUNTEDSUJ(mp)) { - vfs_mount_error(mp, "%s: Snapshots are not yet supported when " - "running with journaled soft updates", fs->fs_fsmnt); - return (EOPNOTSUPP); - } MNT_ILOCK(mp); flag = mp->mnt_flag; MNT_IUNLOCK(mp); -- Test scenario: misc/suj27.sh