panic: page fault, vmcore.377 KDB: debugger backends: ddb KDB: current backend: ddb ---<>--- Copyright (c) 1992-2021 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 14.0-CURRENT #1 main-n248167-4d1597691916-dirty: Fri Jul 23 18:15:30 CEST 2021 pho@mercat1.netperf.freebsd.org:/usr/src/sys/amd64/compile/PHO amd64 FreeBSD clang version 12.0.1 (git@github.com:llvm/llvm-project.git llvmorg-12.0.1-0-gfed41342a82f) WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. VT(vga): resolution 640x480 CPU: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz (3500.07-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306f2 Family=0x6 Model=0x3f Stepping=2 Features=0xbfebfbff Features2=0x7ffefbff AMD Features=0x2c100800 AMD Features2=0x21 Structured Extended Features=0x37ab XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics real memory = 34359738368 (32768 MB) avail memory = 33174138880 (31637 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs FreeBSD/SMP: 1 package(s) x 6 core(s) x 2 hardware threads random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" random: unblocking device. ioapic0 irqs 0-23 ioapic1 irqs 24-47 Launching APs: 1 9 5 10 3 2 4 6 8 11 7 Timecounter "TSC-low" frequency 1750035538 Hz quality 1000 random: entropy device external interface kbd1 at kbdmux0 mlx5en: Mellanox Ethernet driver 3.6.0 (December 2020) vtvga0: smbios0: at iomem 0xf0560-0xf057e smbios0: Version: 2.8, BCD Revision: 2.7 aesni0: acpi0: acpi0: Power Button (fixed) cpu0: on acpi0 atrtc0: port 0x70-0x71,0x74-0x77 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Event timer "HPET4" frequency 14318180 Hz quality 340 Event timer "HPET5" frequency 14318180 Hz quality 340 Event timer "HPET6" frequency 14318180 Hz quality 340 Event timer "HPET7" frequency 14318180 Hz quality 340 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: on acpi0 pci0: on pcib0 pci0: at device 11.1 (no driver attached) pci0: at device 11.2 (no driver attached) pci0: at device 16.1 (no driver attached) pci0: at device 16.6 (no driver attached) pci0: at device 18.1 (no driver attached) acpi_syscontainer0: on acpi0 acpi_syscontainer1: on acpi0 acpi_syscontainer2: on acpi0 acpi_syscontainer3: on acpi0 apei0: on acpi0 pcib1: port 0xcf8-0xcff on acpi0 pci1: on pcib1 pcib2: irq 26 at device 1.0 on pci1 pci2: on pcib2 pcib3: irq 32 at device 2.0 on pci1 pci3: on pcib3 pci3: at device 0.0 (no driver attached) pci3: at device 0.1 (no driver attached) pcib4: irq 40 at device 3.0 on pci1 pci4: on pcib4 pci1: at device 17.0 (no driver attached) ahci0: port 0xf110-0xf117,0xf100-0xf103,0xf0f0-0xf0f7,0xf0e0-0xf0e3,0xf020-0xf03f mem 0xfb438000-0xfb4387ff irq 16 at device 17.4 on pci1 ahci0: AHCI v1.30 with 4 6Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahciem0: on ahci0 xhci0: mem 0xfb400000-0xfb40ffff irq 19 at device 20.0 on pci1 xhci0: 32 bytes context size, 64-bit DMA usbus0 on xhci0 usbus0: 5.0Gbps Super Speed USB v3.0 pci1: at device 22.0 (no driver attached) pci1: at device 22.1 (no driver attached) ehci0: mem 0xfb434000-0xfb4343ff irq 18 at device 26.0 on pci1 usbus1: EHCI version 1.0 usbus1 on ehci0 usbus1: 480Mbps High Speed USB v2.0 pcib5: irq 16 at device 28.0 on pci1 pci5: on pcib5 pcib6: irq 18 at device 28.2 on pci1 pci6: on pcib6 pcib7: at device 0.0 on pci6 pci7: on pcib7 vgapci0: port 0xe000-0xe07f mem 0xfa000000-0xfaffffff,0xfb000000-0xfb01ffff irq 18 at device 0.0 on pci7 vgapci0: Boot video device pcib8: irq 16 at device 28.4 on pci1 pci8: on pcib8 igb0: port 0xd020-0xd03f mem 0xfb320000-0xfb33ffff,0xfb344000-0xfb347fff irq 16 at device 0.0 on pci8 igb0: Using 1024 TX descriptors and 1024 RX descriptors igb0: queue equality override not set, capping rx_queues at 6 and tx_queues at 6 igb0: Using 6 RX queues 6 TX queues igb0: Using MSI-X interrupts with 7 vectors igb0: Ethernet address: 0c:c4:7a:a8:cd:da igb0: netmap queues/slots: TX 6/1024, RX 6/1024 igb1: port 0xd000-0xd01f mem 0xfb300000-0xfb31ffff,0xfb340000-0xfb343fff irq 17 at device 0.1 on pci8 igb1: Using 1024 TX descriptors and 1024 RX descriptors igb1: queue equality override not set, capping rx_queues at 6 and tx_queues at 6 igb1: Using 6 RX queues 6 TX queues igb1: Using MSI-X interrupts with 7 vectors igb1: Ethernet address: 0c:c4:7a:a8:cd:db igb1: netmap queues/slots: TX 6/1024, RX 6/1024 ehci1: mem 0xfb433000-0xfb4333ff irq 18 at device 29.0 on pci1 usbus2: EHCI version 1.0 usbus2 on ehci1 usbus2: 480Mbps High Speed USB v2.0 isab0: at device 31.0 on pci1 isa0: on isab0 ahci1: port 0xf070-0xf077,0xf060-0xf063,0xf050-0xf057,0xf040-0xf043,0xf000-0xf01f mem 0xfb432000-0xfb4327ff irq 16 at device 31.2 on pci1 ahci1: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported ahcich4: at channel 0 on ahci1 ahcich5: at channel 1 on ahci1 ahcich6: at channel 2 on ahci1 ahcich7: at channel 3 on ahci1 ahcich8: at channel 4 on ahci1 ahcich9: at channel 5 on ahci1 ahciem1: on ahci1 acpi_button0: on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 uart1: console (115200,n,8,1) orm0: at iomem 0xc0000-0xc7fff,0xcb000-0xcbfff pnpid ORM0000 on isa0 est0: on cpu0 NULL mp in getnewvnode(9), tag crossmp Timecounters tick every 1.000 msec Expensive timeout(9) function: 0xffffffff80510d60(0xfffffe0109f8a000) 0.004847637 s ses0 at ahciem0 bus 0 scbus4 target 0 lun 0 ses0: SEMB S-E-S 2.00 device ses0: SEMB SES Device ses1 at ahciem1 bus 0 scbus11 target 0 lun 0 ses1: SEMB S-E-S 2.00 device ses1: SEMB SES Device ses0: pass0 in 'Slot 01', SATA Slot: scbus1 target 0 ada0 at ahcich1 bus 0 scbus1 target 0 lun 0 ada0: ACS-2 ATA SATA 3.x device ada0: Serial Number WD-WCC3F3KDDP98 ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 953869MB (1953525168 512 byte sectors) ada0: quirks=0x1<4K> Expensive timeout(9) function: 0xffffffff80a754c0(0xffffffff81aea0c8) 0.007400874 s mlx5_core0: mem 0xf8000000-0xf9ffffff irq 32 at device 0.0 on pci3 ugen0.1: <0x8086 XHCI root HUB> at usbus0 ugen2.1: at usbus2 uhub0 on usbus0 ugen1.1: at usbus1 uhub0: <0x8086 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0 uhub1 on usbus1 uhub1: on usbus1 uhub2 on usbus2 uhub2: on usbus2 mlx5_core0: WARN: mlx5_vsc_set_space:124:(pid 0): Space 0x7 is not supported. mlx5_core0: WARN: mlx5_fwdump_prep:99:(pid 0): VSC scan space is not supported mlx5_core0: INFO: init_one:1613:(pid 0): cannot find SR-IOV PCIe cap mlx5_core: INFO: (mlx5_core0): E-Switch: Total vports 1, l2 table size(65536), per vport: max uc(1024) max mc(16384) mlx5_core0: Failed to initialize SR-IOV support, error 2 mlx5_core1: mem 0xf6000000-0xf7ffffff irq 32 at device 0.1 on pci3 uhub2: 2 ports with 2 removable, self powered uhub1: 2 ports wi fscklock order reversal: 1st 0xfffffe010e58d0d8 ufs (ufs, lockmgr) @ kern/vfs_mount.c:2052 2nd 0xfffffe010e3f09d0 devfs (devfs, lockmgr) @ kern/vfs_subr.c:3251 lock order devfs -> ufs established at: #0 0xffffffff80c921b9 at witness_checkorder+0x459 #1 0xffffffff80bf2178 at lockmgr_lock_flags+0x178 #2 0xffffffff80f4e33d at ffs_lock+0x9d #3 0xffffffff811b413c at VOP_LOCK1_APV+0x3c #4 0xffffffff80d28414 at _vn_lock+0x54 #5 0xffffffff80d05cbb at vfs_domount+0xf5b #6 0xffffffff80d03860 at vfs_donmount+0x880 #7 0xffffffff80d08c67 at kernel_mount+0x57 #8 0xffffffff80d0b831 at parse_mount+0x4b1 #9 0xffffffff80d09f13 at vfs_mountroot+0x7b3 #10 0xffffffff80bae96f at start_init+0x1f #11 0xffffffff80bdbba0 at fork_exit+0x80 #12 0xffffffff810bb52e at fork_trampoline+0xe lock order ufs -> devfs attempted at: #0 0xffffffff80c92aef at witness_checkorder+0xd8f #1 0xffffffff80bf2178 at lockmgr_lock_flags+0x178 #2 0xffffffff811b413c at VOP_LOCK1_APV+0x3c #3 0xffffffff80d28414 at _vn_lock+0x54 #4 0xffffffff80d10832 at vput_final+0x172 #5 0xffffffff80f4bf9d at ffs_unmount+0x38d #6 0xffffffff80d07152 at dounmount+0x812 #7 0xffffffff80d068cd at kern_unmount+0x2fd #8 0xffffffff810ea9c7 at amd64_syscall+0x147 #9 0xffffffff810badce at fast_syscall_common+0xf8 ** /dev/gpt/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 132 files, 256 used, 1265151 free (87 frags, 158133 blocks, 0.0% fragmentation) ***** FILE SYSTEM MARKED CLEAN ***** Updating CPU Microcode... CPU: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz (3500.07-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306f2 Family=0x6 Model=0x3f Stepping=2 Features=0xbfebfbff Features2=0x7ffefbff AMD Features=0x2c100800 AMD Features2=0x21 Structured Extended Features=0x37ab Structured Extended Features3=0x9c000400 XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics Done. ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/lib/compat /usr/local/lib/compat/pkg /usr/local/lib/e2fsprogs /usr/local/lib/gcc10 /usr/local/lib/gcc8 /usr/local/lib/gcc9 /usr/local/lib/perl5/5.32/mach/CORE 32-bit compatibility ldconfig path: /usr/lib32 /usr/local/lib32/compat Setting hostname: mercat1.netperf.freebsd.org. Setting up harvesting: PURE_RDRAND,[UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,[NET_ETHER],NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED Feeding entropy: . Autoloading module: acpi_wmi.ko Autoloading module: ichsmb.ko Autoloading module: ioat.ko ioat0: mem 0xfb42c000-0xfb42ffff irq 31 at device 4.0 on pci1 ioat0: Capabilities: 2f7 ioat1: mem 0xfb428000-0xfb42bfff irq 39 at device 4.1 on pci1 ioat1: Capabilities: 2f7 ioat2: mem 0xfb424000-0xfb427fff irq 31 at device 4.2 on pci1 ioat2: Capabilities: f7 ioat3: mem 0xfb420000-0xfb423fff irq 39 at device 4.3 on pci1 ioat3: Capabilities: f7 ioat4: mem 0xfb41c000-0xfb41ffff irq 31 at device 4.4 on pci1 ioat4: Capabilities: f7 ioat5: mem 0xfb418000-0xfb41bfff irq 39 at device 4.5 on pci1 ioat5: Capabilities: f7 ioat6: mem 0xfb414000-0xfb417fff irq 31 at device 4.6 on pci1 ioat6: Capabilities: f7 ioat7: mem 0xfb410000-0xfb413fff irq 39 at device 4.7 on pci1 ioat7: Capabilities: f7 ichsmb0: port 0x580-0x59f mem 0xfb431000-0xfb4310ff irq 18 at device 31.3 on pci1 smbus0: on ichsmb0 acpi_wmi0: on acpi0 acpi_wmi0: cannot find EC device lo0: link state changed to UP Autoloading module: acpi_wmi.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Starting Network: igb1. igb1: flags=8822 metric 0 mtu 1500 options=4e507bb ether 0c:c4:7a:a8:cd:db media: Ethernet autoselect status: no carrier nd6 options=29 Autoloading module: ichsmb.ko Starting Network: mce0. mce0: flags=8822 metric 0 mtu 1500 options=7eed07bb ether 7c:fe:90:30:ee:38 media: Ethernet autoselect status: no carrier nd6 options=29 Starting Network: mce1. mce1: flags=8822 metric 0 mtu 1500 options=7eed07bb ether 7c:fe:90:30:ee:39 media: Ethernet autoselect status: no carrier nd6 options=29 Autoloading module: uhid.ko Autoloading module: ums.ko Autoloading module: usbhid.ko ums0 on uhub3 ums0: on usbus0 ums0: 3 buttons and [Z] coordinates ID=0 Starting dhclient. if_delmulti_locked: detaching ifnet instance 0xfffff800042f0800 DHCPDISCOVER on igb0 to 255.255.255.255 port 67 interval 8 DHCPOFFER from 192.168.5.1 DHCPREQUEST on igb0 to 255.255.255.255 port 67 if_delmulti_locked: detaching ifnet instance 0xfffff800042f0800 DHCPACK from 192.168.5.1 bound to 192.168.5.171 -- renewal in 300 seconds. add host 127.0.0.1: gateway lo0 fib 0: route already in table add host ::1: gateway lo0 fib 0: route already in table add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Creating and/or trimming log files. Recovering vi editor sessions:. Updating motd:. Clearing /tmp (X related). Starting syslogd. Starting watchdogd. No core dumps found. Setting date via ntp. 23 Jul 21:40:42 ntpdate[2630]: step time server 192.168.5.1 offset -0.028909 sec Starting rpcbind. NFS access cache time=60 Starting statd. Starting lockd. Starting powerd. Security policy loaded: MAC/ntpd (mac_ntpd) Starting ntpd. Mounting late filesystems:. Starting inetd. Configuring vt: blanktime. Performing sanity check on sshd configuration. Starting sshd. Starting cron. sysctl: unknown oid 'debug.fail_point.pmap_allocpte_page_alloc' at line 12 Fri Jul 23 21:40 FreeBSD/amd64 (mercat1.netperf.freebsd.org) (ttyu1) login: Jul 23 21:42:54 mercat1 su[2786]: pho to root on /dev/pts/0 20210723 21:43:40 all (1/78): chroot.sh 20210723 21:43:48 all (2/78): crossmp4.sh lock order reversal: 1st 0xfffffe0144369520 syncer (syncer, lockmgr) @ kern/vfs_subr.c:2617 2nd 0xfffffe01461f2c90 ufs (ufs, lockmgr) @ kern/vfs_subr.c:3251 lock order ufs -> syncer established at: #0 0xffffffff80c921b9 at witness_checkorder+0x459 #1 0xffffffff80bf2178 at lockmgr_lock_flags+0x178 #2 0xffffffff811b413c at VOP_LOCK1_APV+0x3c #3 0xffffffff80d28414 at _vn_lock+0x54 #4 0xffffffff80d10832 at vput_final+0x172 #5 0xffffffff80d07145 at dounmount+0x805 #6 0xffffffff80d068cd at kern_unmount+0x2fd #7 0xffffffff810ea9c7 at amd64_syscall+0x147 #8 0xffffffff810badce at fast_syscall_common+0xf8 lock order syncer -> ufs attempted at: #0 0xffffffff80c92aef at witness_checkorder+0xd8f #1 0xffffffff80bf2178 at lockmgr_lock_flags+0x178 #2 0xffffffff80f4e33d at ffs_lock+0x9d #3 0xffffffff811b413c at VOP_LOCK1_APV+0x3c #4 0xffffffff80d28414 at _vn_lock+0x54 #5 0xffffffff80d10832 at vput_final+0x172 #6 0xffffffff80d10162 at vget_finish+0x132 #7 0xffffffff80f4c7ad at ffs_sync+0x51d #8 0xffffffff80d1bd4b at sync_fsync+0xeb #9 0xffffffff811b2d19 at VOP_FSYNC_APV+0x59 #10 0xffffffff80d191a6 at sched_sync+0x436 #11 0xffffffff80bdbba0 at fork_exit+0x80 #12 0xffffffff810bb52e at fork_trampoline+0xe 20210723 21:52:05 all (3/78): datamove5.sh 20210723 21:52:15 all (4/78): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210723 21:52:57 all (5/78): g8a.sh GEOM_NOP: Device md10.nop created. acquiring duplicate lock of same type: "vnode interlock" 1st vnode interlock @ /usr/src/sys/fs/nullfs/null_vnops.c:372 2nd vnode interlock @ /usr/src/sys/fs/nullfs/null_vnops.c:372 stack backtrace: #0 0xffffffff80c92da1 at witness_debugger+0x71 #1 0xffffffff80bfee14 at __mtx_lock_flags+0x94 #2 0xffffffff8274c10b at null_add_writecount+0x4b #3 0xffffffff811b7599 at VOP_ADD_WRITECOUNT_APV+0x59 #4 0xffffffff8274c13f at null_add_writecount+0x7f #5 0xffffffff811b7599 at VOP_ADD_WRITECOUNT_APV+0x59 #6 0xffffffff80d2829a at vn_open_vnode+0x36a #7 0xffffffff80d27a88 at vn_open_cred+0x4d8 #8 0xffffffff80d1e70c at kern_openat+0x28c #9 0xffffffff810ea9c7 at amd64_syscall+0x147 #10 0xffffffff810badce at fast_syscall_common+0xf8 GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. g_vfs_done():md10.nop[WRITE(offset=4599971840, length=32768)]error = 6 UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt 20210723 21:55:14 all (6/78): gnop11.sh 20210723 21:55:16 all (7/78): mlockall4.sh 20210723 21:55:23 all (8/78): nullfs.sh 20210723 22:00:24 all (9/78): nullfs10.sh 20210723 22:00:26 all (10/78): nullfs11.sh 20210723 22:01:30 all (11/78): nullfs12.sh 20210723 22:01:33 all (12/78): nullfs13.sh 20210723 22:01:34 all (13/78): nullfs14.sh 20210723 22:01:36 all (14/78): nullfs15.sh 20210723 22:01:38 all (15/78): nullfs16.sh 20210723 22:01:40 all (16/78): nullfs17.sh 20210723 22:06:42 all (17/78): nullfs18.sh 20210723 22:08:51 all (18/78): nullfs19.sh 20210723 22:10:54 all (19/78): nullfs2.sh 20210723 22:21:12 all (20/78): nullfs20.sh 20210723 22:31:23 all (21/78): nullfs21.sh 20210723 22:31:24 all (22/78): nullfs22.sh 20210723 22:31:52 all (23/78): nullfs23.sh 20210723 22:31:53 all (24/78): nullfs24.sh Expensive timeout(9) function: 0xffffffff80f744a0(0) 0.011841838 s 20210723 22:36:55 all (25/78): nullfs25.sh Expensive timeout(9) function: 0xffffffff80f744a0(0) 0.059040353 s 20210723 22:41:58 all (26/78): nullfs26.sh 20210723 22:52:22 all (27/78): nullfs27.sh Jul 23 22:54:37 mercat1 kernel: pid 58062 (creat), uid 2007 inumber 131219 on /mnt: filesystem full Jul 23 22:54:42 mercat1 kernel: pid 58040 (rw), uid 2007 inumber 226106 on /mnt: filesystem full Jul 23 22:56:51 mercat1 kernel: pid 61962 (rw), uid 2007 inumber 92771 on /mnt: filesystem full 20210723 23:02:48 all (28/78): nullfs28.sh UFS: forcibly unmounting /dev/md10 from /mnt stress2: newblk leak: 3047/3051. 20210723 23:04:52 all (29/78): nullfs3.sh 20210723 23:04:55 all (30/78): nullfs4.sh stress2: mount leak: 13/44 20210723 23:04:56 all (31/78): nullfs5.sh stress2: freework leak: 135/139. 20210723 23:17:02 all (32/78): nullfs6.sh 20210723 23:17:04 all (33/78): nullfs7.sh 20210723 23:17:06 all (34/78): nullfs9.sh 20210723 23:17:08 all (35/78): pread.sh 20210723 23:17:28 all (36/78): readdir.sh 20210723 23:29:29 all (37/78): sendfile5.sh stress2: newblk leak: 32776/32781. 20210723 23:30:03 all (38/78): suj34.sh Jul 23 23:37:08 mercat1 kernel: pid 99142 (swap), jid 0, uid 0, was killed: out of swap space stress2: GEOM leak: 29/102 20210723 23:40:57 all (39/78): umountf12.sh 20210723 23:41:11 all (40/78): zz-combo04.sh 20210724 00:05:18 all (41/78): beneath4.sh 20210724 00:05:25 all (42/78): crossmp8.sh 20210724 00:06:28 all (43/78): fifo.sh 20210724 00:06:31 all (44/78): fifo2.sh 20210724 00:08:33 all (45/78): fifo3.sh 20210724 00:09:06 all (46/78): fifo4.sh 20210724 00:11:13 all (47/78): force4.sh Jul 24 00:11:24 mercat1 kernel: pid 27936 (link), uid 2007 inumber 32965 on /mnt: filesystem full Jul 24 00:11:46 mercat1 kernel: pid 27827 (swap), jid 0, uid 2007, was killed: out of swap space UFS: forcibly unmounting /dev/md10 from /mnt 20210724 00:12:05 all (48/78): force7.sh UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt lock order reversal: 1st 0xfffff803c6783c98 FFS (FFS Lock, sleep mutex) @ ufs/ffs/ffs_vfsops.c:294 2nd 0xfffffe0166b42740 struct mount mtx (struct mount mtx, sleep mutex) @ kern/vfs_mount.c:519 lock order struct mount mtx -> FFS Lock established at: #0 0xffffffff80c921b9 at witness_checkorder+0x459 #1 0xffffffff80bfee14 at __mtx_lock_flags+0x94 #2 0xffffffff80f29f3e at softdep_flushfiles+0x36e #3 0xffffffff80f4bcbb at ffs_unmount+0xab #4 0xffffffff80d07152 at dounmount+0x812 #5 0xffffffff80d068cd at kern_unmount+0x2fd #6 0xffffffff810ea9c7 at amd64_syscall+0x147 #7 0xffffffff810badce at fast_syscall_common+0xf8 lock order FFS Lock -> struct mount mtx attempted at: #0 0xffffffff80c92aef at witness_checkorder+0xd8f #1 0xffffffff80bfee14 at __mtx_lock_flags+0x94 #2 0xffffffff80d03d1b at vfs_ref+0x13b #3 0xffffffff80f48317 at ffs_fsfail_cleanup+0x67 #4 0xffffffff80f2b698 at softdep_disk_write_complete+0x58 #5 0xffffffff80ce7044 at bufdone+0x304 #6 0xffffffff80ce188c at bufwrite+0x24c #7 0xffffffff80f21246 at ffs_update+0x2f6 #8 0xffffffff80f3dc1d at softdep_sync_buf+0x3bd #9 0xffffffff80f4f65d at ffs_syncvnode+0x26d #10 0xffffffff80f4c48d at ffs_sync+0x1fd #11 0xffffffff80d2a045 at vfs_write_suspend+0x165 #12 0xffffffff80d2a205 at vfs_write_suspend_umnt+0x35 #13 0xffffffff80f4c078 at ffs_unmount+0x468 #14 0xffffffff80d07152 at dounmount+0x812 #15 0xffffffff80d068cd at kern_unmount+0x2fd #16 0xffffffff810ea9c7 at amd64_syscall+0x147 #17 0xffffffff810badce at fast_syscall_common+0xf8 UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt stress2: newblk leak: 1918/1923. 20210724 00:17:07 all (49/78): full.sh Jul 24 00:17:51 mercat1 kernel: pid 61313 (mmap), uid 2007 inumber 195309 on /mnt: filesystem full Jul 24 00:20:05 mercat1 kernel: pid 63400 (mmap), uid 2007 inumber 152833 on /mnt: filesystem full Jul 24 00:20:37 mercat1 kernel: pid 63429 (mmap), uid 2007 inumber 161159 on /mnt: filesystem full Jul 24 00:21:03 mercat1 kernel: pid 63479 (rw), uid 2007 inumber 185463 on /mnt: filesystem full Jul 24 00:27:13 mercat1 kernel: pid 72251 (rw), uid 2007 inumber 160118 on /mnt: filesystem full Jul 24 00:27:19 mercat1 kernel: pid 72320 (rw), uid 2007 inumber 144048 on /mnt: filesystem full 20210724 00:27:38 all (50/78): full.sh Jul 24 00:31:58 mercat1 kernel: pid 81926 (rw), uid 2007 inumber 103072 on /mnt: filesystem full Jul 24 00:32:03 mercat1 kernel: pid 81889 (rw), uid 2007 inumber 106799 on /mnt: filesystem full Jul 24 00:32:38 mercat1 kernel: pid 81945 (rw), uid 2007 inumber 106645 on /mnt: filesystem full Jul 24 00:33:45 mercat1 kernel: pid 81806 (creat), uid 2007 inumber 65709 on /mnt: filesystem full Jul 24 00:36:18 mercat1 kernel: pid 87922 (rw), uid 2007 inumber 85067 on /mnt: filesystem full 20210724 00:38:08 all (51/78): g8a.sh GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt 20210724 00:40:33 all (52/78): graid3.sh GEOM_RAID3: Device raid3/data launched (3/3). Jul 24 00:43:34 mercat1 kernel: pid 12684 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 00:50:14 mercat1 kernel: pid 16671 (creat), uid 2007 inumber 131328 on /mnt: filesystem full Jul 24 00:51:59 mercat1 kernel: pid 18458 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 00:52:46 mercat1 kernel: pid 18562 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 00:53:51 mercat1 kernel: pid 20050 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 00:59:58 mercat1 kernel: pid 25294 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 01:00:12 mercat1 kernel: pid 25299 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 01:00:37 mercat1 kernel: pid 25291 (rw), uid 2007 inumber 235676 on /mnt: filesystem full GEOM_RAID3: Device data: provider raid3/data destroyed. GEOM_RAID3: Device data destroyed. 20210724 01:01:33 all (53/78): marcus4.sh witness_lock_list_get: witness exhausted stress2: freework leak: 290/296. 20210724 01:18:43 all (54/78): marcus5.sh 20210724 01:36:50 all (55/78): marcus6.sh 20210724 01:38:06 all (56/78): marcus7.sh 20210724 01:50:12 all (57/78): mkfifo.sh Jul 24 01:53:51 mercat1 kernel: pid 85897 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 01:54:13 mercat1 kernel: pid 85885 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 02:01:16 mercat1 kernel: pid 92830 (swap), jid 0, uid 2007, was killed: out of swap space 20210724 02:11:10 all (58/78): mkfifo2c.sh 20210724 02:26:16 all (59/78): mkfifo2d.sh 20210724 02:32:34 all (60/78): mkfifo3.sh 20210724 02:33:00 all (61/78): mkfifo4.sh 20210724 02:40:24 all (62/78): mkfifo5.sh 20210724 02:40:25 all (63/78): mkfifo6.sh 20210724 02:40:30 all (64/78): mkfifo7.sh 20210724 02:40:32 all (65/78): mkfifo8.sh 20210724 02:41:33 all (66/78): nfs17.sh 20210724 02:41:34 all (67/78): nullfs14.sh 20210724 02:41:36 all (68/78): nullfs27.sh Jul 24 02:43:59 mercat1 kernel: pid 72184 (rw), uid 2007 inumber 72213 on /mnt: filesystem full Jul 24 02:45:07 mercat1 kernel: pid 72265 (mmap), uid 2007 inumber 65866 on /mnt: filesystem full Jul 24 02:45:51 mercat1 kernel: pid 76546 (rw), uid 2007 inumber 76361 on /mnt: filesystem full Jul 24 02:46:12 mercat1 kernel: pid 76530 (rw), uid 2007 inumber 82674 on /mnt: filesystem full Jul 24 02:50:09 mercat1 kernel: pid 84554 (rw), uid 2007 inumber 224142 on /mnt: filesystem full 20210724 02:52:04 all (69/78): nullfs28.sh UFS: forcibly unmounting /dev/md10 from /mnt stress2: newblk leak: 3459/3463. 20210724 02:54:08 all (70/78): openlock.sh 20210724 02:54:14 all (71/78): pfl2.sh Jul 24 02:54:22 mercat1 kernel: pid 96645 (rw), uid 2007 inumber 79873 on /mnt10: filesystem full Jul 24 02:54:32 mercat1 kernel: pid 96605 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 02:54:45 mercat1 kernel: pid 96712 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 02:56:38 mercat1 kernel: pid 99780 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 02:56:48 mercat1 kernel: pid 99787 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 02:58:52 mercat1 kernel: pid 1624 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 02:59:03 mercat1 kernel: pid 1610 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 02:59:17 mercat1 kernel: pid 1588 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 02:59:32 mercat1 kernel: pid 1605 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 03:00:39 mercat1 kernel: pid 2355 (rw), uid 2007 inumber 107157 on /mnt10: filesystem full Jul 24 03:00:46 mercat1 kernel: pid 3818 (rw), uid 2007 inumber 156119 on /mnt11: filesystem full Jul 24 03:02:46 mercat1 kernel: pid 5646 (mkdir), uid 2007 inumber 210567 on /mnt11: filesystem full 20210724 03:04:56 all (72/78): pfl4.sh Jul 24 03:09:24 mercat1 kernel: pid 8488 (swap), jid 0, uid 2007, was killed: out of swap space 20210724 03:15:56 all (73/78): smrstress2.sh Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Jul 24 03:18:53 mercat1 kernel: pid 23857 (link), uid 0 inumber 131457 on /mnt: filesystem full Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures 20210724 03:23:04 all (74/78): tmpfs11.sh Jul 24 03:25:42 mercat1 kernel: pid 32392 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 03:27:40 mercat1 kernel: pid 33702 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 03:28:19 mercat1 kernel: pid 33700 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 03:30:02 mercat1 kernel: pid 35680 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 03:30:12 mercat1 kernel: pid 35703 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 03:36:22 mercat1 kernel: pid 39131 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 03:39:19 mercat1 kernel: pid 39895 (swap), jid 0, uid 2007, was killed: out of swap space 20210724 03:40:09 all (75/78): tmpfs12.sh Jul 24 03:40:56 mercat1 kernel: pid 41388 (swap), jid 0, uid 0, was killed: out of swap space Jul 24 03:41:06 mercat1 kernel: pid 41401 (swap), jid 0, uid 0, was killed: out of swap space 20210724 03:41:32 all (76/78): tmpfs13.sh 20210724 03:47:29 all (77/78): tmpfs20.sh 20210724 03:57:54 all (78/78): zfs9.sh ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Jul 24 03:58:17 mercat1 kernel: pid 71306 (swap), jid 0, uid 0, was killed: out of swap space Jul 24 04:00:33 mercat1 kernel: pid 73140 (swap), jid 0, uid 0, was killed: out of swap space Jul 24 04:04:50 mercat1 kernel: pid 83806 (swap), jid 0, uid 0, was killed: out of swap space Jul 24 04:05:38 mercat1 kernel: pid 83831 (swap), jid 0, uid 0, was killed: out of swap space Expensive timeout(9) function: 0xffffffff80e19000(0xfffffe0143f68ca8) 0.089911127 s 20210724 04:09:33 all.sh done, elapsed 0 day(s), 06:25.54 20210724 04:09:37 all (1/78): chroot.sh 20210724 04:09:45 all (2/78): crossmp4.sh stress2: newblk leak: 87/122. 20210724 04:18:30 all (3/78): datamove5.sh 20210724 04:19:02 all (4/78): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt stress2: newblk leak: 338/360. 20210724 04:19:56 all (5/78): g8a.sh GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt 20210724 04:22:10 all (6/78): gnop11.sh 20210724 04:22:12 all (7/78): mlockall4.sh 20210724 04:22:19 all (8/78): nullfs.sh 20210724 04:27:21 all (9/78): nullfs10.sh 20210724 04:27:23 all (10/78): nullfs11.sh 20210724 04:28:19 all (11/78): nullfs12.sh 20210724 04:28:22 all (12/78): nullfs13.sh 20210724 04:28:24 all (13/78): nullfs14.sh 20210724 04:28:25 all (14/78): nullfs15.sh 20210724 04:28:27 all (15/78): nullfs16.sh 20210724 04:28:30 all (16/78): nullfs17.sh 20210724 04:33:32 all (17/78): nullfs18.sh 20210724 04:35:41 all (18/78): nullfs19.sh 20210724 04:37:45 all (19/78): nullfs2.sh Jul 24 04:41:38 mercat1 kernel: pid 31908 (swap), jid 0, uid 0, was killed: out of swap space 20210724 04:49:45 all (20/78): nullfs20.sh Jul 24 04:56:59 mercat1 kernel: pid 59772 (swap), jid 0, uid 2007, was killed: out of swap space 20210724 05:00:16 all (21/78): nullfs21.sh 20210724 05:00:18 all (22/78): nullfs22.sh 20210724 05:00:42 all (23/78): nullfs23.sh 20210724 05:00:44 all (24/78): nullfs24.sh Jul 24 05:01:28 mercat1 kernel: pid 78416 (swap), jid 0, uid 0, was killed: out of swap space Jul 24 05:04:43 mercat1 kernel: pid 78434 (swap), jid 0, uid 0, was killed: out of swap space 20210724 05:05:52 all (25/78): nullfs25.sh 20210724 05:10:55 all (26/78): nullfs26.sh Jul 24 05:13:16 mercat1 kernel: pid 82546 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 05:13:56 mercat1 kernel: pid 82594 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 05:14:20 mercat1 kernel: pid 82575 (swap), jid 0, uid 2007, was killed: out of swap space 20210724 05:21:18 all (27/78): nullfs27.sh Jul 24 05:23:31 mercat1 kernel: pid 92083 (rw), uid 2007 inumber 84051 on /mnt: filesystem full Jul 24 05:23:36 mercat1 kernel: pid 92167 (rw), uid 2007 inumber 100274 on /mnt: filesystem full Jul 24 05:29:50 mercat1 kernel: pid 99273 (mmap), uid 2007 inumber 168513 on /mnt: filesystem full 20210724 05:31:46 all (28/78): nullfs28.sh UFS: forcibly unmounting /dev/md10 from /mnt stress2: newblk leak: 2811/2815. 20210724 05:33:48 all (29/78): nullfs3.sh 20210724 05:33:51 all (30/78): nullfs4.sh 20210724 05:33:53 all (31/78): nullfs5.sh stress2: freework leak: 118/122. 20210724 05:45:46 all (32/78): nullfs6.sh 20210724 05:45:48 all (33/78): nullfs7.sh 20210724 05:45:50 all (34/78): nullfs9.sh 20210724 05:45:52 all (35/78): pread.sh 20210724 05:46:13 all (36/78): readdir.sh 20210724 05:58:14 all (37/78): sendfile5.sh 20210724 05:58:40 all (38/78): suj34.sh Jul 24 05:59:23 mercat1 kernel: pid 77702 (swap), jid 0, uid 0, was killed: out of swap space Jul 24 06:03:21 mercat1 kernel: pid 81764 (swap), jid 0, uid 0, was killed: out of swap space Jul 24 06:04:37 mercat1 kernel: pid 83271 (swap), jid 0, uid 0, was killed: out of swap space stress2: GEOM leak: 29/102 20210724 06:09:09 all (39/78): umountf12.sh 20210724 06:09:23 all (40/78): zz-combo04.sh Jul 24 06:12:50 mercat1 kernel: pid 31875 (swap), jid 0, uid 0, was killed: out of swap space Jul 24 06:27:26 mercat1 kernel: pid 60229 (swap), jid 0, uid 0, was killed: out of swap space Jul 24 06:27:38 mercat1 kernel: pid 60343 (swap), jid 0, uid 0, was killed: out of swap space Jul 24 06:28:06 mercat1 kernel: pid 60363 (swap), jid 0, uid 0, was killed: out of swap space Jul 24 06:30:19 mercat1 kernel: pid 66745 (swap), jid 0, uid 0, was killed: out of swap space 20210724 06:32:07 all (41/78): beneath4.sh 20210724 06:32:14 all (42/78): crossmp8.sh 20210724 06:33:17 all (43/78): fifo.sh 20210724 06:33:20 all (44/78): fifo2.sh 20210724 06:35:22 all (45/78): fifo3.sh 20210724 06:35:52 all (46/78): fifo4.sh 20210724 06:38:23 all (47/78): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210724 06:39:09 all (48/78): force7.sh UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt UFS: forcibly unmounting /dev/md10 from /mnt stress2: newblk leak: 280/284. 20210724 06:44:14 all (49/78): full.sh Jul 24 06:46:46 mercat1 kernel: pid 11645 (rw), uid 2007 inumber 210176 on /mnt: filesystem full Jul 24 06:52:48 mercat1 kernel: pid 18139 (rw), uid 2007 inumber 211840 on /mnt: filesystem full Jul 24 06:53:22 mercat1 kernel: pid 18103 (mmap), uid 2007 inumber 63354 on /mnt: filesystem full Jul 24 06:53:53 mercat1 kernel: pid 18038 (link), uid 2007 inumber 200619 on /mnt: filesystem full Jul 24 06:54:16 mercat1 kernel: pid 18099 (mmap), uid 2007 inumber 198046 on /mnt: out of inodes 20210724 06:54:46 all (50/78): full.sh Jul 24 06:57:44 mercat1 kernel: pid 25324 (rw), uid 2007 inumber 229240 on /mnt: filesystem full Jul 24 06:59:20 mercat1 kernel: pid 29788 (rw), uid 2007 inumber 10725 on /mnt: filesystem full Jul 24 07:02:41 mercat1 kernel: pid 32914 (rw), uid 2007 inumber 183913 on /mnt: filesystem full 20210724 07:05:17 all (51/78): g8a.sh GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop is still open, so it can't be definitely removed. UFS: forcibly unmounting /dev/md10.nop from /mnt 20210724 07:07:23 all (52/78): graid3.sh GEOM_RAID3: Device raid3/data launched (3/3). Jul 24 07:09:51 mercat1 kernel: pid 55621 (rw), uid 2007 inumber 161540 on /mnt: filesystem full Jul 24 07:12:05 mercat1 kernel: pid 57057 (rw), uid 2007 inumber 131107 on /mnt: filesystem full Jul 24 07:14:58 mercat1 kernel: pid 59765 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 07:16:44 mercat1 kernel: pid 61185 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 07:17:21 mercat1 kernel: pid 61172 (rw), uid 2007 inumber 86546 on /mnt: filesystem full Jul 24 07:19:05 mercat1 kernel: pid 62812 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 07:19:16 mercat1 kernel: pid 62697 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 07:19:41 mercat1 kernel: pid 62814 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 07:20:42 mercat1 kernel: pid 64268 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 07:21:05 mercat1 kernel: pid 64300 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 07:22:11 mercat1 kernel: pid 64323 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 07:23:33 mercat1 kernel: pid 65992 (rw), uid 2007 inumber 248146 on /mnt: filesystem full Jul 24 07:24:14 mercat1 kernel: pid 65982 (rw), uid 2007 inumber 119383 on /mnt: filesystem full Jul 24 07:25:05 mercat1 kernel: pid 67367 (swap), jid 0, uid 2007, was killed: out of swap space GEOM_RAID3: Device data: provider raid3/data destroyed. GEOM_RAID3: Device data destroyed. 20210724 07:28:43 all (53/78): marcus4.sh stress2: freework leak: 831/837. 20210724 07:45:48 all (54/78): marcus5.sh 20210724 08:02:10 all (55/78): marcus6.sh 20210724 08:03:17 all (56/78): marcus7.sh 20210724 08:15:20 all (57/78): mkfifo.sh Jul 24 08:22:32 mercat1 kernel: pid 71176 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 08:23:05 mercat1 kernel: pid 71268 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 08:27:18 mercat1 kernel: pid 75284 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 08:31:44 mercat1 kernel: pid 77987 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 08:32:05 mercat1 kernel: pid 78079 (swap), jid 0, uid 2007, was killed: out of swap space 20210724 08:36:12 all (58/78): mkfifo2c.sh 20210724 08:51:17 all (59/78): mkfifo2d.sh 20210724 08:57:41 all (60/78): mkfifo3.sh 20210724 08:58:06 all (61/78): mkfifo4.sh stress2: freework leak: 1463/1471. stress2: newblk leak: 17819/17827. 20210724 09:04:35 all (62/78): mkfifo5.sh 20210724 09:04:37 all (63/78): mkfifo6.sh 20210724 09:04:42 all (64/78): mkfifo7.sh 20210724 09:04:43 all (65/78): mkfifo8.sh 20210724 09:05:45 all (66/78): nfs17.sh 20210724 09:05:46 all (67/78): nullfs14.sh 20210724 09:05:48 all (68/78): nullfs27.sh Jul 24 09:12:14 mercat1 kernel: pid 55579 (mmap), uid 2007 inumber 70776 on /mnt: filesystem full Jul 24 09:12:19 mercat1 kernel: pid 55502 (rw), uid 2007 inumber 89015 on /mnt: filesystem full Jul 24 09:12:42 mercat1 kernel: pid 55581 (mmap), uid 2007 inumber 106026 on /mnt: filesystem full 20210724 09:16:13 all (69/78): nullfs28.sh g_vfs_done():md10[WRITE(offset=75661312, length=393216)]error = 6 UFS: forcibly unmounting /dev/md10 from /mnt 20210724 09:18:19 all (70/78): openlock.sh 20210724 09:18:22 all (71/78): pfl2.sh Jul 24 09:21:01 mercat1 kernel: pid 67402 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 09:21:02 mercat1 kernel: pid 67371 (rw), uid 2007 inumber 103092 on /mnt11: filesystem full Jul 24 09:21:40 mercat1 kernel: pid 67461 (creat), uid 2007 inumber 65653 on /mnt11: filesystem full Jul 24 09:24:37 mercat1 kernel: pid 71112 (rw), uid 2007 inumber 90662 on /mnt10: filesystem full Jul 24 09:25:00 mercat1 kernel: pid 71217 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 09:26:27 mercat1 kernel: pid 71200 (symlink), uid 2007 inumber 67283 on /mnt11: out of inodes Jul 24 09:26:58 mercat1 kernel: pid 72727 (rw), uid 2007 inumber 66607 on /mnt11: filesystem full Jul 24 09:27:04 mercat1 kernel: pid 72609 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 09:27:14 mercat1 kernel: pid 72679 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 09:27:31 mercat1 kernel: pid 72671 (swap), jid 0, uid 2007, was killed: out of swap space 20210724 09:28:51 all (72/78): pfl4.sh Jul 24 09:31:39 mercat1 kernel: pid 74684 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 09:34:08 mercat1 kernel: pid 74670 (swap), jid 0, uid 2007, was killed: out of swap space 20210724 09:39:41 all (73/78): smrstress2.sh Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Jul 24 09:42:14 mercat1 badcode[89370]: stack overflow detected; terminated Completed 10000000 loops with 0 failures Jul 24 09:42:31 mercat1 kernel: pid 88102 (rw), uid 0 inumber 195701 on /mnt: filesystem full Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures 20210724 09:46:39 all (74/78): tmpfs11.sh Jul 24 09:47:07 mercat1 kernel: pid 96428 (swap), jid 0, uid 2007, was killed: out of swap space Jul 24 09:51:55 mercat1 kernel: pid 769 (swap), jid 0, uid 2007, was killed: out of swap space 20210724 10:03:46 all (75/78): tmpfs12.sh Jul 24 10:04:26 mercat1 kernel: pid 8402 (swap), jid 0, uid 0, was killed: out of swap space Jul 24 10:04:37 mercat1 kernel: pid 8383 (swap), jid 0, uid 0, was killed: out of swap space Jul 24 10:04:49 mercat1 kernel: pid 8394 (swap), jid 0, uid 0, was killed: out of swap space Jul 24 10:05:02 mercat1 kernel: pid 8400 (swap), jid 0, uid 0, was killed: out of swap space 20210724 10:05:27 all (76/78): tmpfs13.sh 20210724 10:07:32 all (77/78): tmpfs20.sh 20210724 10:18:00 all (78/78): zfs9.sh ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Jul 24 10:21:13 mercat1 kernel: pid 39132 (swap), jid 0, uid 0, was killed: out of swap space Jul 24 10:27:22 mercat1 kernel: pid 48648 (swap), jid 0, uid 0, was killed: out of swap space 20210724 10:29:11 all.sh done, elapsed 0 day(s), 06:19.33 20210724 11:18:16 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210724 11:19:18 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210724 11:19:51 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210724 11:20:54 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210724 11:22:03 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210724 11:22:50 all (1/1): force4.sh Jul 24 11:22:57 mercat1 kernel: pid 82776 (rw), uid 2007 inumber 55857 on /mnt: filesystem full Jul 24 11:23:01 mercat1 kernel: pid 82785 (mkfifo), uid 2007 inumber 32896 on /mnt: out of inodes Jul 24 11:23:07 mercat1 kernel: pid 82699 (creat), uid 2007 inumber 33056 on /mnt: out of inodes Jul 24 11:23:10 mercat1 kernel: pid 82728 (openat), uid 2007 inumber 33117 on /mnt: out of inodes Jul 24 11:23:18 mercat1 kernel: pid 82657 (symlink), uid 2007 inumber 32897 on /mnt: out of inodes UFS: forcibly unmounting /dev/md10 from /mnt 20210724 11:23:28 all (1/1): force4.sh Jul 24 11:23:42 mercat1 kernel: pid 84368 (mmap), uid 2007 inumber 45907 on /mnt: filesystem full UFS: forcibly unmounting /dev/md10 from /mnt 20210724 11:24:10 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210724 11:24:44 all (1/1): force4.sh Jul 24 11:24:52 mercat1 kernel: pid 87830 (rw), uid 2007 inumber 109573 on /mnt: filesystem full Jul 24 11:25:00 mercat1 kernel: pid 87938 (mmap), uid 2007 inumber 54624 on /mnt: filesystem full Jul 24 11:25:04 mercat1 kernel: pid 87872 (mkdir), uid 2007 inumber 92199 on /mnt: out of inodes UFS: forcibly unmounting /dev/md10 from /mnt 20210724 11:25:33 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210724 11:26:32 all (1/1): force4.sh Jul 24 11:26:56 mercat1 kernel: pid 95825 (rw), uid 2007 inumber 108941 on /mnt: filesystem full UFS: forcibly unmounting /dev/md10 from /mnt 20210724 11:27:38 all (1/1): force4.sh Jul 24 11:27:54 mercat1 kernel: pid 97453 (rw), uid 2007 inumber 68224 on /mnt: filesystem full UFS: forcibly unmounting /dev/md10 from /mnt 20210724 11:28:18 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210724 11:28:58 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210724 11:29:48 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210724 11:30:43 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt stress2: freework leak: 1708/1716. stress2: newblk leak: 4257/4262. stress2: mount leak: 13/57 stress2: GEOM leak: 4/77 20210724 11:31:11 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210724 11:32:03 all (1/1): force4.sh Jul 24 11:32:15 mercat1 kernel: pid 11980 (creat), uid 2007 inumber 32899 on /mnt: filesystem full g_vfs_done():md10[WRITE(offset=8585216, length=4096)]error = 6 UFS: forcibly unmounting /dev/md10 from /mnt 20210724 11:33:09 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210724 11:33:41 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210724 11:34:48 all (1/1): force4.sh Jul 24 11:35:09 mercat1 kernel: pid 18522 (rw), uid 2007 inumber 39244 on /mnt: filesystem full UFS: forcibly unmounting /dev/md10 from /mnt 20210724 11:35:18 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt stress2: freework leak: 1111/1119. stress2: newblk leak: 947/952. stress2: mount leak: 13/57 stress2: GEOM leak: 4/77 20210724 11:35:58 all (1/1): force4.sh Jul 24 11:36:01 mercat1 kernel: pid 24603 (creat), uid 2007 inumber 32921 on /mnt: filesystem full Jul 24 11:36:10 mercat1 kernel: pid 24608 (mkdir), uid 2007 inumber 73250 on /mnt: filesystem full Jul 24 11:36:19 mercat1 kernel: pid 24575 (mmap), uid 2007 inumber 61885 on /mnt: filesystem full Jul 24 11:36:33 mercat1 kernel: pid 24577 (rw), uid 2007 inumber 33931 on /mnt: filesystem full UFS: forcibly unmounting /dev/md10 from /mnt 20210724 11:36:59 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210724 11:37:35 all (1/1): force4.sh Jul 24 11:37:47 mercat1 kernel: pid 27843 (rw), uid 2007 inumber 68101 on /mnt: filesystem full Jul 24 11:38:15 mercat1 kernel: pid 27821 (mmap), uid 2007 inumber 70154 on /mnt: filesystem full UFS: forcibly unmounting /dev/md10 from /mnt 20210724 11:39:08 all (1/1): force4.sh Jul 24 11:39:50 mercat1 kernel: pid 29505 (mmap), uid 2007 inumber 33016 on /mnt: out of inodes UFS: forcibly unmounting /dev/md10 from /mnt 20210724 11:40:06 all (1/1): force4.sh Jul 24 11:40:18 mercat1 kernel: pid 32459 (creat), uid 2007 inumber 32918 on /mnt: out of inodes UFS: forcibly unmounting /dev/md10 from /mnt 20210724 11:41:07 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210724 11:42:12 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt Fatal trap 12: page fault while in kernel mode cpuid = 5; apic id = 05 fault virtual address = 0x10 fault code = supervisor read data, page not present Fatal trap 12: page fault while in kernel mode instruction pointer = 0x20:0xffffffff8274a6be stack pointer = 0x0:0xfffffe018d9736c0 frame pointer = 0x0:0xfffffe018d9736d0 cpuid = 4; apic id = 04 fault virtual address = 0x10 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff8274a6be stack pointer = 0x28:0xfffffe018f3866c0 frame pointer = 0x28:0xfffffe018f3866d0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 37116 (dirnprename) code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 trap number = 12 panic: page fault cpuid = 4 time = 1627119779 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe018f386360 vpanic() at vpanic+0x187/frame 0xfffffe018f3863c0 panic() at panic+0x43/frame 0xfffffe018f386420 trap_fatal() at trap_fatal+0x387/frame 0xfffffe018f386480 trap_pfault() at trap_pfault+0x99/frame 0xfffffe018f3864e0 trap() at trap+0x2a7/frame 0xfffffe018f3865f0 calltrap() at calltrap+0x8/frame 0xfffffe018f3865f0 --- trap 0xc, rip = 0xffffffff8274a6be, rsp = 0xfffffe018f3866c0, rbp = 0xfffffe018f3866d0 --- null_checkvp() at null_checkvp+0xe/frame 0xfffffe018f3866d0 null_bypass() at null_bypass+0xc9/frame 0xfffffe018f386850 VOP_RENAME_APV() at VOP_RENAME_APV+0x3c/frame 0xfffffe018f386870 kern_renameat() at kern_renameat+0x3dd/frame 0xfffffe018f386ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018f386bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018f386bf0 --- syscall (128, FreeBSD ELF64, sys_rename), rip = 0x8011a68da, rsp = 0x7fffffffe208, rbp = 0x7fffffffe350 --- KDB: enter: panic [ thread pid 37116 tid 131643 ] Stopped at kdb_enter+0x37: movq $0,0x1279b5e(%rip) db> run pho db:0:pho> set $lines 20000 db:0:pho> run pho1 db:1:pho1> dump Dumping 2607 out of 32624 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Dump complete db:1:pho1> bt Tracing pid 37116 tid 131643 td 0xfffffe015e5f5560 kdb_enter() at kdb_enter+0x37/frame 0xfffffe018f386360 vpanic() at vpanic+0x1b8/frame 0xfffffe018f3863c0 panic() at panic+0x43/frame 0xfffffe018f386420 trap_fatal() at trap_fatal+0x387/frame 0xfffffe018f386480 trap_pfault() at trap_pfault+0x99/frame 0xfffffe018f3864e0 trap() at trap+0x2a7/frame 0xfffffe018f3865f0 calltrap() at calltrap+0x8/frame 0xfffffe018f3865f0 --- trap 0xc, rip = 0xffffffff8274a6be, rsp = 0xfffffe018f3866c0, rbp = 0xfffffe018f3866d0 --- null_checkvp() at null_checkvp+0xe/frame 0xfffffe018f3866d0 null_bypass() at null_bypass+0xc9/frame 0xfffffe018f386850 VOP_RENAME_APV() at VOP_RENAME_APV+0x3c/frame 0xfffffe018f386870 kern_renameat() at kern_renameat+0x3dd/frame 0xfffffe018f386ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018f386bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018f386bf0 --- syscall (128, FreeBSD ELF64, sys_rename), rip = 0x8011a68da, rsp = 0x7fffffffe208, rbp = 0x7fffffffe350 --- db:1:pho1> show allpcpu Current CPU: 4 cpuid = 0 dynamic pcpu = 0x9b4bc0 curthread = 0xfffffe01691f13a0: pid 37118 tid 127092 critnest 0 "fts" curpcb = 0xfffffe01691f18b0 fpcurthread = 0xfffffe01691f13a0: pid 37118 "fts" idlethread = 0xfffffe00e4c5b3a0: tid 100003 "idle: cpu0" self = 0xffffffff82610000 curpmap = 0xfffffe0182e36b10 tssp = 0xffffffff82610384 rsp0 = 0xfffffe018c3fccc0 kcr3 = 0x80000008142325f9 ucr3 = 0x8000000814234df9 scr3 = 0x767ee5d95 gs32p = 0xffffffff82610404 ldt = 0xffffffff82610444 tss = 0xffffffff82610434 curvnet = 0 spin locks held: cpuid = 1 dynamic pcpu = 0xfffffe00bb288bc0 curthread = 0xfffffe0176aaf3a0: pid 37099 tid 129983 critnest 0 "fts" curpcb = 0xfffffe0176aaf8b0 fpcurthread = 0xfffffe0176aaf3a0: pid 37099 "fts" idlethread = 0xfffffe00e4c5ac80: tid 100004 "idle: cpu1" self = 0xffffffff82611000 curpmap = 0xfffffe014496a120 tssp = 0xffffffff82611384 rsp0 = 0xfffffe018d31acc0 kcr3 = 0x800000016188d70a ucr3 = 0x800000016188ef0a scr3 = 0x16188ef0a gs32p = 0xffffffff82611404 ldt = 0xffffffff82611444 tss = 0xffffffff82611434 curvnet = 0 spin locks held: cpuid = 2 dynamic pcpu = 0xfffffe00bb295bc0 curthread = 0xfffffe0161395740: pid 37109 tid 127144 critnest 0 "fts" curpcb = 0xfffffe0161395c50 fpcurthread = 0xfffffe0161395740: pid 37109 "fts" idlethread = 0xfffffe00e4c5a560: tid 100005 "idle: cpu2" self = 0xffffffff82612000 curpmap = 0xfffffe0144bcbb10 tssp = 0xffffffff82612384 rsp0 = 0xfffffe018c500cc0 kcr3 = 0x800000047ee8f448 ucr3 = 0x800000047ee90c48 scr3 = 0x47ee90c48 gs32p = 0xffffffff82612404 ldt = 0xffffffff82612444 tss = 0xffffffff82612434 curvnet = 0 spin locks held: cpuid = 3 dynamic pcpu = 0xfffffe00bb2a2bc0 curthread = 0xfffffe015a5c8c80: pid 37114 tid 128293 critnest 0 "fts" curpcb = 0xfffffe015a5c9190 fpcurthread = 0xfffffe015a5c8c80: pid 37114 "fts" idlethread = 0xfffffe00e4c59e40: tid 100006 "idle: cpu3" self = 0xffffffff82613000 curpmap = 0xfffffe014568b500 tssp = 0xffffffff82613384 rsp0 = 0xfffffe0189b70cc0 kcr3 = 0x80000005606e541a ucr3 = 0x80000005606e7c1a scr3 = 0x5606e7c1a gs32p = 0xffffffff82613404 ldt = 0xffffffff82613444 tss = 0xffffffff82613434 curvnet = 0 spin locks held: cpuid = 4 dynamic pcpu = 0xfffffe00bb2afbc0 curthread = 0xfffffe015e5f5560: pid 37116 tid 131643 critnest 1 "dirnprename" curpcb = 0xfffffe015e5f5a70 fpcurthread = 0xfffffe015e5f5560: pid 37116 "dirnprename" idlethread = 0xfffffe00e4c59720: tid 100007 "idle: cpu4" self = 0xffffffff82614000 curpmap = 0xfffffe017b435120 tssp = 0xffffffff82614384 rsp0 = 0xfffffe018f386cc0 kcr3 = 0x80000004bce69017 ucr3 = 0x8000000347187817 scr3 = 0x21cc4e955 gs32p = 0xffffffff82614404 ldt = 0xffffffff82614444 tss = 0xffffffff82614434 curvnet = 0 spin locks held: cpuid = 5 dynamic pcpu = 0xfffffe00bb2bcbc0 curthread = 0xfffffe0157891ac0: pid 37129 tid 130308 critnest 0 "dirnprename" curpcb = 0xfffffe0157891fd0 fpcurthread = 0xfffffe0157891ac0: pid 37129 "dirnprename" idlethread = 0xfffffe00e4c59000: tid 100008 "idle: cpu5" self = 0xffffffff82615000 curpmap = 0xfffffe01832ed500 tssp = 0xffffffff82615384 rsp0 = 0xfffffe018d973cc0 kcr3 = 0x800000030908f070 ucr3 = 0x80000003dc32e870 scr3 = 0x82a88aa49 gs32p = 0xffffffff82615404 ldt = 0xffffffff82615444 tss = 0xffffffff82615434 curvnet = 0 spin locks held: cpuid = 6 dynamic pcpu = 0xfffffe00bb2c9bc0 curthread = 0xfffffe017ad163a0: pid 37084 tid 132078 critnest 0 "fts" curpcb = 0xfffffe017ad168b0 fpcurthread = 0xfffffe017ad163a0: pid 37084 "fts" idlethread = 0xfffffe00e4c5d740: tid 100009 "idle: cpu6" self = 0xffffffff82616000 curpmap = 0xfffffe01610a9b10 tssp = 0xffffffff82616384 rsp0 = 0xfffffe018fc05cc0 kcr3 = 0x800000070a5e16af ucr3 = 0x800000070a5e2eaf scr3 = 0x3a3c0cea2 gs32p = 0xffffffff82616404 ldt = 0xffffffff82616444 tss = 0xffffffff82616434 curvnet = 0 spin locks held: cpuid = 7 dynamic pcpu = 0xfffffe00bb2d6bc0 curthread = 0xfffffe0169201900: pid 37131 tid 128275 critnest 0 "fts" curpcb = 0xfffffe0169201e10 fpcurthread = 0xfffffe0169201900: pid 37131 "fts" idlethread = 0xfffffe00e4c5d020: tid 100010 "idle: cpu7" self = 0xffffffff82617000 curpmap = 0xfffffe015d145500 tssp = 0xffffffff82617384 rsp0 = 0xfffffe0189b16cc0 kcr3 = 0x80000007cfe92223 ucr3 = 0x800000029ad08a23 scr3 = 0x29ad08a23 gs32p = 0xffffffff82617404 ldt = 0xffffffff82617444 tss = 0xffffffff82617434 curvnet = 0 spin locks held: cpuid = 8 dynamic pcpu = 0xfffffe00bb2e3bc0 curthread = 0xfffffe0144b443a0: pid 37104 tid 137069 critnest 1 "fts" curpcb = 0xfffffe0144b448b0 fpcurthread = 0xfffffe0144b443a0: pid 37104 "fts" idlethread = 0xfffffe00e4c5c900: tid 100011 "idle: cpu8" self = 0xffffffff82618000 curpmap = 0xfffffe0183457500 tssp = 0xffffffff82618384 rsp0 = 0xfffffe0195d80cc0 kcr3 = 0x80000002eb8da5ab ucr3 = 0x80000002eb8dbdab scr3 = 0x5d7426d54 gs32p = 0xffffffff82618404 ldt = 0xffffffff82618444 tss = 0xffffffff82618434 curvnet = 0 spin locks held: cpuid = 9 dynamic pcpu = 0xfffffe00bb2f0bc0 curthread = 0xfffffe01436021e0: pid 37095 tid 100253 critnest 1 "fts" curpcb = 0xfffffe01436026f0 fpcurthread = 0xfffffe01436021e0: pid 37095 "fts" idlethread = 0xfffffe00e4c611e0: tid 100012 "idle: cpu9" self = 0xffffffff82619000 curpmap = 0xfffffe01448bb120 tssp = 0xffffffff82619384 rsp0 = 0xfffffe014167fcc0 kcr3 = 0x800000035d51d339 ucr3 = 0x8000000238572b39 scr3 = 0x238572b39 gs32p = 0xffffffff82619404 ldt = 0xffffffff82619444 tss = 0xffffffff82619434 curvnet = 0 spin locks held: cpuid = 10 dynamic pcpu = 0xfffffe00bb2fdbc0 curthread = 0xfffffe0143e12560: pid 37137 tid 100284 critnest 0 "fts" curpcb = 0xfffffe0143e12a70 fpcurthread = 0xfffffe0143e12560: pid 37137 "fts" idlethread = 0xfffffe00e4c60ac0: tid 100013 "idle: cpu10" self = 0xffffffff8261a000 curpmap = 0xfffffe010c92e120 tssp = 0xffffffff8261a384 rsp0 = 0xfffffe0141757cc0 kcr3 = 0x800000003c3ee39a ucr3 = 0x8000000381706b9a scr3 = 0x381706b9a gs32p = 0xffffffff8261a404 ldt = 0xffffffff8261a444 tss = 0xffffffff8261a434 curvnet = 0 spin locks held: cpuid = 11 dynamic pcpu = 0xfffffe00bb30abc0 curthread = 0xfffffe0149551ac0: pid 37128 tid 127851 critnest 0 "fts" curpcb = 0xfffffe0149551fd0 fpcurthread = 0xfffffe0149551ac0: pid 37128 "fts" idlethread = 0xfffffe00e4c603a0: tid 100014 "idle: cpu11" self = 0xffffffff8261b000 curpmap = 0xfffffe0182fc3500 tssp = 0xffffffff8261b384 rsp0 = 0xfffffe018b4a7cc0 kcr3 = 0x800000060a23a6ae ucr3 = 0x800000052a044eae scr3 = 0x52a044eae gs32p = 0xffffffff8261b404 ldt = 0xffffffff8261b444 tss = 0xffffffff8261b434 curvnet = 0 spin locks held: db:1:pho1> show alllocks db:1:pho1> show lockedvnods Locked vnodes vnode 0xfffffe016f593960: type VDIR usecount 1, writecount 0, refcount 2 seqc users 1 mountedhere 0xfffffe014351e700 hold count flags () flags (VIRF_MOUNTPOINT|VMP_LAZYLIST) lock type ufs: EXCL by thread 0xfffffe00e4ce5900 (pid 0, kernel, tid 100044) #0 0xffffffff80bf21e7 at lockmgr_lock_flags+0x1e7 #1 0xffffffff80f4e33d at ffs_lock+0x9d #2 0xffffffff811b413c at VOP_LOCK1_APV+0x3c #3 0xffffffff80d28414 at _vn_lock+0x54 #4 0xffffffff80d06db9 at dounmount+0x479 #5 0xffffffff80d09319 at vfs_deferred_unmount+0xf9 #6 0xffffffff80c84a3a at taskqueue_run_locked+0xaa #7 0xffffffff80c85ac2 at taskqueue_thread_loop+0xc2 #8 0xffffffff80bdbba0 at fork_exit+0x80 #9 0xffffffff810bb52e at fork_trampoline+0xe nlink=2, effnlink=2, size=512, extsize 0 generation=d4b08ae2, uid=0, gid=0, flags=0x0 ino 101764608, on dev gpt/work vnode 0xfffffe0156ecbbb8: type VDIR usecount 1, writecount 0, refcount 4 seqc users 1 mountedhere 0 hold count flags () flags (VMP_LAZYLIST) v_object 0xfffff80257fc4b58 ref 0 pages 0 cleanbuf 0 dirtybuf 1 lock type ufs: EXCL by thread 0xfffffe010cf27900 (pid 37037, openat, tid 132989) with exclusive waiters pending #0 0xffffffff80bf21e7 at lockmgr_lock_flags+0x1e7 #1 0xffffffff80f4e33d at ffs_lock+0x9d #2 0xffffffff811b413c at VOP_LOCK1_APV+0x3c #3 0xffffffff8274bbd3 at null_lock+0xd3 #4 0xffffffff811b413c at VOP_LOCK1_APV+0x3c #5 0xffffffff80d28414 at _vn_lock+0x54 #6 0xffffffff80d00c32 at lookup+0xc2 #7 0xffffffff80cffe39 at namei+0x379 #8 0xffffffff80d276ce at vn_open_cred+0x11e #9 0xffffffff80d1e70c at kern_openat+0x28c #10 0xffffffff810ea9c7 at amd64_syscall+0x147 #11 0xffffffff810badce at fast_syscall_common+0xf8 nlink=2, effnlink=2, size=512, extsize 0 generation=bbddad36, uid=2007, gid=0, flags=0x0 ino 65802, on dev md10 vnode 0xfffffe014b908000: type VDIR usecount 1, writecount 0, refcount 5 seqc users 0 mountedhere 0 hold count flags () flags (VMP_LAZYLIST) v_object 0xfffff803938ce840 ref 0 pages 6 cleanbuf 0 dirtybuf 2 lock type ufs: EXCL by thread 0xfffffe015e5f5560 (pid 37116, dirnprename, tid 131643) #0 0xffffffff80bf21e7 at lockmgr_lock_flags+0x1e7 #1 0xffffffff80f4e33d at ffs_lock+0x9d #2 0xffffffff811b413c at VOP_LOCK1_APV+0x3c #3 0xffffffff80d28414 at _vn_lock+0x54 #4 0xffffffff80d100aa at vget_finish+0x7a #5 0xffffffff80cfe55c at vfs_hash_get+0xbc #6 0xffffffff80f491b8 at ffs_vgetf+0x58 #7 0xffffffff80f58880 at ufs_lookup_ino+0xe70 #8 0xffffffff811b0afa at VOP_CACHEDLOOKUP_APV+0x5a #9 0xffffffff80ceff96 at vfs_cache_lookup+0xa6 #10 0xffffffff811b093a at VOP_LOOKUP_APV+0x5a #11 0xffffffff8274b3c9 at null_lookup+0xe9 #12 0xffffffff811b093a at VOP_LOOKUP_APV+0x5a #13 0xffffffff80d01041 at lookup+0x4d1 #14 0xffffffff80cffe39 at namei+0x379 #15 0xffffffff80d23f3b at kern_renameat+0x1db #16 0xffffffff810ea9c7 at amd64_syscall+0x147 #17 0xffffffff810badce at fast_syscall_common+0xf8 nlink=1013, effnlink=1013, size=53760, extsize 0 generation=808a681d, uid=2007, gid=0, flags=0x0 ino 65836, on dev md10 vnode 0xfffffe01460f3258: type VDIR usecount 1, writecount 0, refcount 5 seqc users 0 mountedhere 0 hold count flags () flags (VMP_LAZYLIST) v_object 0xfffff8058ca64e70 ref 0 pages 5 cleanbuf 0 dirtybuf 2 lock type ufs: EXCL by thread 0xfffffe0157891ac0 (pid 37129, dirnprename, tid 130308) #0 0xffffffff80bf21e7 at lockmgr_lock_flags+0x1e7 #1 0xffffffff80f4e33d at ffs_lock+0x9d #2 0xffffffff811b413c at VOP_LOCK1_APV+0x3c #3 0xffffffff80d28414 at _vn_lock+0x54 #4 0xffffffff80d100aa at vget_finish+0x7a #5 0xffffffff80cfe55c at vfs_hash_get+0xbc #6 0xffffffff80f491b8 at ffs_vgetf+0x58 #7 0xffffffff80f58880 at ufs_lookup_ino+0xe70 #8 0xffffffff811b0afa at VOP_CACHEDLOOKUP_APV+0x5a #9 0xffffffff80ceff96 at vfs_cache_lookup+0xa6 #10 0xffffffff811b093a at VOP_LOOKUP_APV+0x5a #11 0xffffffff8274b3c9 at null_lookup+0xe9 #12 0xffffffff811b093a at VOP_LOOKUP_APV+0x5a #13 0xffffffff80d01041 at lookup+0x4d1 #14 0xffffffff80cffe39 at namei+0x379 #15 0xffffffff80d23f3b at kern_renameat+0x1db #16 0xffffffff810ea9c7 at amd64_syscall+0x147 #17 0xffffffff810badce at fast_syscall_common+0xf8 nlink=739, effnlink=739, size=49664, extsize 0 generation=3c643799, uid=2007, gid=0, flags=0x0 ino 65867, on dev md10 vnode 0xfffffe014b6de000: type VREG usecount 1, writecount 1, refcount 1 seqc users 0 hold count flags () flags (VIRF_PGREAD|VMP_LAZYLIST) v_object 0xfffff80723f58420 ref 0 pages 0 cleanbuf 0 dirtybuf 0 lock type ufs: EXCL by thread 0xfffffe01447411e0 (pid 37064, creat, tid 136343) #0 0xffffffff80bf21e7 at lockmgr_lock_flags+0x1e7 #1 0xffffffff80f4e33d at ffs_lock+0x9d #2 0xffffffff811b413c at VOP_LOCK1_APV+0x3c #3 0xffffffff8274bbd3 at null_lock+0xd3 #4 0xffffffff811b413c at VOP_LOCK1_APV+0x3c #5 0xffffffff8274c326 at null_vput_pair+0x136 #6 0xffffffff80d27924 at vn_open_cred+0x374 #7 0xffffffff80d1e70c at kern_openat+0x28c #8 0xffffffff810ea9c7 at amd64_syscall+0x147 #9 0xffffffff810badce at fast_syscall_common+0xf8 nlink=1, effnlink=1, size=0, extsize 0 generation=10f0587b, uid=2007, gid=0, flags=0x0 ino 76573, on dev md10 vnode 0xfffffe0163c4c770: type VNON usecount 1, writecount 0, refcount 1 seqc users 0 hold count flags () flags () lock type ufs: EXCL by thread 0xfffffe010cf27900 (pid 37037, openat, tid 132989) #0 0xffffffff80bf2cda at lockmgr_xlock_hard+0x4ea #1 0xffffffff80bf319e at __lockmgr_args+0x1fe #2 0xffffffff80f492cc at ffs_vgetf+0x16c #3 0xffffffff80f1a22f at ffs_valloc+0x57f #4 0xffffffff80f635e9 at ufs_makeinode+0xe9 #5 0xffffffff80f5f2a4 at ufs_create+0x34 #6 0xffffffff811b0cc2 at VOP_CREATE_APV+0x62 #7 0xffffffff8274b16b at null_bypass+0x12b #8 0xffffffff811b0cc2 at VOP_CREATE_APV+0x62 #9 0xffffffff80d27869 at vn_open_cred+0x2b9 #10 0xffffffff80d1e70c at kern_openat+0x28c #11 0xffffffff810ea9c7 at amd64_syscall+0x147 #12 0xffffffff810badce at fast_syscall_common+0xf8 nlink=0, effnlink=0, size=0, extsize 0 generation=3296f7fa, uid=2007, gid=0, flags=0x0 ino 76423, on dev md10 vnode 0xfffffe0149d42708: type VDIR usecount 2, writecount 0, refcount 2 seqc users 1 mountedhere 0 hold count flags () flags () v_object 0xfffff80257fc4b58 ref 0 pages 0 cleanbuf 0 dirtybuf 0 lock type ufs: EXCL by thread 0xfffffe010cf27900 (pid 37037, openat, tid 132989) with exclusive waiters pending #0 0xffffffff80bf21e7 at lockmgr_lock_flags+0x1e7 #1 0xffffffff80f4e33d at ffs_lock+0x9d #2 0xffffffff811b413c at VOP_LOCK1_APV+0x3c #3 0xffffffff8274bbd3 at null_lock+0xd3 #4 0xffffffff811b413c at VOP_LOCK1_APV+0x3c #5 0xffffffff80d28414 at _vn_lock+0x54 #6 0xffffffff80d00c32 at lookup+0xc2 #7 0xffffffff80cffe39 at namei+0x379 #8 0xffffffff80d276ce at vn_open_cred+0x11e #9 0xffffffff80d1e70c at kern_openat+0x28c #10 0xffffffff810ea9c7 at amd64_syscall+0x147 #11 0xffffffff810badce at fast_syscall_common+0xf8 vp=0xfffffe0149d42708, lowervp=0xfffffe0156ecbbb8 vnode 0xfffffe01600074b0: type VDIR usecount 2, writecount 0, refcount 2 seqc users 0 mountedhere 0 hold count flags () flags () v_object 0xfffff803938ce840 ref 0 pages 6 cleanbuf 0 dirtybuf 0 lock type ufs: EXCL by thread 0xfffffe015e5f5560 (pid 37116, dirnprename, tid 131643) #0 0xffffffff80bf21e7 at lockmgr_lock_flags+0x1e7 #1 0xffffffff80f4e33d at ffs_lock+0x9d #2 0xffffffff811b413c at VOP_LOCK1_APV+0x3c #3 0xffffffff80d28414 at _vn_lock+0x54 #4 0xffffffff80d100aa at vget_finish+0x7a #5 0xffffffff80cfe55c at vfs_hash_get+0xbc #6 0xffffffff80f491b8 at ffs_vgetf+0x58 #7 0xffffffff80f58880 at ufs_lookup_ino+0xe70 #8 0xffffffff811b0afa at VOP_CACHEDLOOKUP_APV+0x5a #9 0xffffffff80ceff96 at vfs_cache_lookup+0xa6 #10 0xffffffff811b093a at VOP_LOOKUP_APV+0x5a #11 0xffffffff8274b3c9 at null_lookup+0xe9 #12 0xffffffff811b093a at VOP_LOOKUP_APV+0x5a #13 0xffffffff80d01041 at lookup+0x4d1 #14 0xffffffff80cffe39 at namei+0x379 #15 0xffffffff80d23f3b at kern_renameat+0x1db #16 0xffffffff810ea9c7 at amd64_syscall+0x147 #17 0xffffffff810badce at fast_syscall_common+0xf8 vp=0xfffffe01600074b0, lowervp=0xfffffe014b908000 vnode 0xfffffe017c66cc20: type VDIR usecount 2, writecount 0, refcount 2 seqc users 0 mountedhere 0 hold count flags () flags () v_object 0xfffff8058ca64e70 ref 0 pages 5 cleanbuf 0 dirtybuf 0 lock type ufs: EXCL by thread 0xfffffe0157891ac0 (pid 37129, dirnprename, tid 130308) #0 0xffffffff80bf21e7 at lockmgr_lock_flags+0x1e7 #1 0xffffffff80f4e33d at ffs_lock+0x9d #2 0xffffffff811b413c at VOP_LOCK1_APV+0x3c #3 0xffffffff80d28414 at _vn_lock+0x54 #4 0xffffffff80d100aa at vget_finish+0x7a #5 0xffffffff80cfe55c at vfs_hash_get+0xbc #6 0xffffffff80f491b8 at ffs_vgetf+0x58 #7 0xffffffff80f58880 at ufs_lookup_ino+0xe70 #8 0xffffffff811b0afa at VOP_CACHEDLOOKUP_APV+0x5a #9 0xffffffff80ceff96 at vfs_cache_lookup+0xa6 #10 0xffffffff811b093a at VOP_LOOKUP_APV+0x5a #11 0xffffffff8274b3c9 at null_lookup+0xe9 #12 0xffffffff811b093a at VOP_LOOKUP_APV+0x5a #13 0xffffffff80d01041 at lookup+0x4d1 #14 0xffffffff80cffe39 at namei+0x379 #15 0xffffffff80d23f3b at kern_renameat+0x1db #16 0xffffffff810ea9c7 at amd64_syscall+0x147 #17 0xffffffff810badce at fast_syscall_common+0xf8 vp=0xfffffe017c66cc20, lowervp=0xfffffe01460f3258 vnode 0xfffffe01477f54b0: type VREG usecount 1, writecount 1, refcount 1 seqc users 0 hold count flags () flags (VIRF_PGREAD) v_object 0xfffff80723f58420 ref 0 pages 0 cleanbuf 0 dirtybuf 0 lock type ufs: EXCL by thread 0xfffffe01447411e0 (pid 37064, creat, tid 136343) #0 0xffffffff80bf21e7 at lockmgr_lock_flags+0x1e7 #1 0xffffffff80f4e33d at ffs_lock+0x9d #2 0xffffffff811b413c at VOP_LOCK1_APV+0x3c #3 0xffffffff8274bbd3 at null_lock+0xd3 #4 0xffffffff811b413c at VOP_LOCK1_APV+0x3c #5 0xffffffff8274c326 at null_vput_pair+0x136 #6 0xffffffff80d27924 at vn_open_cred+0x374 #7 0xffffffff80d1e70c at kern_openat+0x28c #8 0xffffffff810ea9c7 at amd64_syscall+0x147 #9 0xffffffff810badce at fast_syscall_common+0xf8 vp=0xfffffe01477f54b0, lowervp=0xfffffe014b6de000 db:1:pho1> show mount 0xfffffe010e155700 /dev/gpt/work on / (ufs) 0xfffffe010e156280 devfs on /dev (devfs) 0xfffffe014351f280 /dev/gpt/tmp on /tmp (ufs) 0xfffffe010e154b80 procfs on /proc (procfs) 0xfffffe01484f6b80 /dev/md10 on /mnt (ufs) 0xfffffe014351e700 /mnt on /mnt2 (nullfs) More info: show mount db:1:pho1> show bufqueues bqempty: 84800 Buf domain 0 freebufs 11707 lofreebufs 219 hifreebufs 329 bufspace 33349632 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 1781 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 1570 (17911808) dirtyq count 1781 (14602240) wakeup 0 lim 25 CPU 1, 1, 0, 16, 6, 9, 6, 5, 14, 4, 5, 8, Locked buffers: 1 space 32768 Total buffers: 52246 space 33349632 Buf domain 1 freebufs 12304 lofreebufs 219 hifreebufs 329 bufspace 13557760 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 1576 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 1157 (6348800) dirtyq count 1576 (6656000) wakeup 0 lim 25 CPU 14, 9, 4, 22, 7, 1, 3, 12, 10, 4, 3, 7, Locked buffers: 0 space 0 Total buffers: 2831 space 13557760 Buf domain 2 freebufs 12457 lofreebufs 219 hifreebufs 329 bufspace 11653120 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 1512 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 1008 (4714496) dirtyq count 1512 (6299648) wakeup 0 lim 25 CPU 19, 20, 19, 21, 8, 10, 9, 14, 7, 10, 11, 8, Locked buffers: 0 space 0 Total buffers: 2680 space 11653120 Buf domain 3 freebufs 12285 lofreebufs 219 hifreebufs 329 bufspace 12791808 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 1602 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 1121 (5058560) dirtyq count 1602 (7049216) wakeup 0 lim 25 CPU 14, 15, 12, 20, 7, 1, 4, 18, 11, 13, 1, 9, Locked buffers: 0 space 0 Total buffers: 38971 space 12791808 Buf domain 4 freebufs 12390 lofreebufs 219 hifreebufs 329 bufspace 12976128 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 1577 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 1029 (5279744) dirtyq count 1577 (6619136) wakeup 0 lim 25 CPU 7, 12, 7, 21, 6, 12, 22, 13, 13, 9, 7, 8, Locked buffers: 0 space 0 Total buffers: 2743 space 12976128 Buf domain 5 freebufs 11412 lofreebufs 219 hifreebufs 329 bufspace 23257088 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 1520 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 2054 (16338944) dirtyq count 1520 (6299648) wakeup 0 lim 25 CPU 17, 12, 17, 20, 14, 11, 8, 12, 8, 8, 14, 6, Locked buffers: 0 space 0 Total buffers: 3722 space 23257088 Buf domain 6 freebufs 12397 lofreebufs 219 hifreebufs 329 bufspace 13217792 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 1542 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 1083 (6295552) dirtyq count 1542 (6438912) wakeup 0 lim 25 CPU 5, 20, 14, 1, 4, 9, 8, 17, 13, 12, 0, 8, Locked buffers: 0 space 0 Total buffers: 2738 space 13217792 db:1:pho1> show page vm_cnt.v_free_count: 1338504 vm_cnt.v_inactive_count: 7367 vm_cnt.v_active_count: 6319804 vm_cnt.v_laundry_count: 0 vm_cnt.v_wire_count: 451248 vm_cnt.v_free_reserved: 10643 vm_cnt.v_free_min: 51261 vm_cnt.v_free_target: 173115 vm_cnt.v_inactive_target: 259672 db:1:pho1> show pageq pq_free 1338504 dom 0 page_cnt 8123602 free 1338504 pq_act 6319804 pq_inact 7367 pq_laund 0 pq_unsw 0 db:0:pho> show freepages DOMAIN: 0 FREE LIST 0: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000000 | 000000 10 (004096K) | 000000 | 000000 09 (002048K) | 000003 | 000000 08 (001024K) | 001780 | 000000 07 (000512K) | 006561 | 000000 06 (000256K) | 000029 | 000000 05 (000128K) | 000053 | 000001 04 (000064K) | 000087 | 000001 03 (000032K) | 000129 | 000000 02 (000016K) | 000265 | 000000 01 (000008K) | 000364 | 000000 00 (000004K) | 001189 | 000000 FREE LIST 1: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000000 | 000000 10 (004096K) | 000000 | 000000 09 (002048K) | 000000 | 000000 08 (001024K) | 000000 | 000000 07 (000512K) | 000000 | 000000 06 (000256K) | 000000 | 000000 05 (000128K) | 000000 | 000001 04 (000064K) | 000001 | 000002 03 (000032K) | 000003 | 000000 02 (000016K) | 000003 | 000001 01 (000008K) | 000003 | 000000 00 (000004K) | 000002 | 000000 db:0:pho> show uma Zone Size Used Free Requests Sleeps Bucket Total Mem XFree VM OBJECT 264 21127 534938 985098395 0 31 146801160 0 malloc-4096 4096 34131 10 140197 0 2 139841536 0 FFS inode 1160 29144 41612 353310386 0 31 82076960 0 VNODE 600 51849 78931 1801661300 0 71 78468000 0 RADIX NODE 144 443778 92559 726495754 0 63 77232528 0 Files 80 156 724594 2478935277 0 126 57980000 0 malloc-384 384 32095 111175 799658452 0 31 55015680 0 malloc-128 128 78013 204304 561764445 0 127 36136576 0 THREAD 1808 11718 7103 71384807 0 8 34028368 0 mbuf_cluster 2048 16365 1 53484819 0 8 33517568 0 VMSPACE 2544 154 11024 35189287 0 4 28436832 0 swblk 136 2514 199558 8125402 0 62 27481792 0 FFS2 dinode 256 29144 60181 353309685 0 63 22867200 0 malloc-128 128 70057 64049 446007925 0 126 17165568 0 S VFS Cache 104 19894 141800 1385143699 0 127 16816176 0 malloc-256 256 33199 29141 800003475 0 63 15959040 0 PROC 1336 212 11305 35189534 0 8 15386712 0 BUF TRIE 144 364 105588 136304660 0 62 15257088 0 vmem btag 56 81271 101237 17680202 0 254 10220448 0 mbuf 256 16417 13748 2213182507 0 254 7722240 0 malloc-128 128 47675 6172 75196899 0 126 6892416 0 UMA Slabs 0 112 46348 3977 6174879 0 126 5636400 0 TMPFS node 224 0 19890 863975632 0 62 4455360 0 128 Bucket 1024 1776 1347 63078923 0 16 3197952 0 malloc-256 256 12038 337 93234103 0 62 3168000 0 malloc-64 64 32415 16158 615396681 0 254 3108672 0 swpctrie 144 431 17741 634008 0 62 2616768 0 TURNSTILE 136 18822 162 18822 0 62 2581824 0 malloc-256 256 3726 4974 204665608 0 62 2227200 0 ksiginfo 112 11491 7445 370245 0 126 2120832 0 pbuf 2776 0 745 0 0 0 2068120 0 64 Bucket 512 2594 1238 77075118 0 30 1961984 0 malloc-256 256 32 6943 712286886 0 62 1785600 0 g_bio 408 0 4230 1317121434 0 32 1725840 0 SLEEPQUEUE 88 18822 250 18822 0 126 1678336 0 256 Bucket 2048 575 215 95611367 0 8 1617920 0 MAP ENTRY 96 3869 10831 1250536230 0 126 1411200 0 32 Bucket 256 3420 2055 71667638 0 62 1401600 0 malloc-4096 4096 332 5 15617 0 2 1380352 0 malloc-128 128 1533 8449 12434153737 0 126 1277696 0 malloc-4096 4096 216 70 35190993 0 2 1171456 0 mbuf_jumbo_page 4096 0 285 265163689 0 16 1167360 0 malloc-128 128 4096 4987 87397145 0 126 1162624 0 mbuf_packet 256 13 4067 617252907 0 254 1044480 0 DIRHASH 1024 742 274 19852579 0 16 1040384 0 malloc-64 64 9746 3610 1306866466 0 254 854784 0 malloc-8192 8192 90 2 312 0 1 753664 0 L VFS Cache 320 1067 745 2025667 0 30 579840 0 malloc-32768 32768 10 7 8979 0 1 557056 0 malloc-65536 65536 2 6 5970 0 1 524288 0 rl_entry 40 10621 2105 10703 0 254 509040 0 UMA Zones 2048 229 6 503 0 8 481280 0 filedesc0 1072 192 249 35189535 0 8 472752 0 8 Bucket 80 125 5725 66860470 0 126 468000 0 malloc-32768 32768 13 0 20 0 1 425984 0 Mountpoints 2944 6 130 931868 0 4 400384 0 malloc-32768 32768 12 0 12 0 1 393216 0 lkpimm 160 2 2323 2 0 62 372000 0 lkpicurr 160 9 2316 9 0 62 372000 0 malloc-32 32 8648 2062 437523 0 254 342720 0 16 Bucket 144 341 1955 135694066 0 62 330624 0 malloc-65536 65536 3 2 4773 0 1 327680 0 4 Bucket 48 67 6485 4673776 0 254 314496 0 NAMEI 1024 19 285 12113480675 0 16 311296 0 malloc-64 64 4166 496 12392 0 254 298368 0 malloc-2048 2048 134 10 63111912 0 8 294912 0 malloc-8192 8192 30 4 22352319 0 1 278528 0 malloc-16 16 15370 1880 57615 0 254 276000 0 malloc-65536 65536 4 0 38 0 1 262144 0 malloc-1024 1024 203 37 21845801 0 16 245760 0 socket 944 52 200 12978 0 16 237888 0 malloc-16384 16384 14 0 4836 0 1 229376 0 malloc-64 64 56 3409 12463153646 0 254 221760 0 malloc-16384 16384 4 7 1030169 0 1 180224 0 malloc-512 512 83 261 2258720 0 30 176128 0 pipe 744 13 217 5941602 0 16 171120 0 malloc-32768 32768 2 3 53041 0 1 163840 0 malloc-8192 8192 19 0 3069 0 1 155648 0 PWD 32 40 4496 756929765 0 254 145152 0 malloc-1024 1024 70 70 418858167 0 16 143360 0 malloc-65536 65536 2 0 2 0 1 131072 0 fakepg 104 0 1248 4303 0 126 129792 0 PGRP 88 33 1255 1185 0 126 113344 0 DEVCTL 1024 0 108 3099395 0 0 110592 0 swrbuf 2776 0 37 1453918 0 2 102712 0 malloc-32 32 471 2679 13050469 0 254 100800 0 malloc-8192 8192 12 0 12 0 1 98304 0 malloc-384 384 144 96 115950 0 30 92160 0 ada_ccb 272 0 322 17816912 0 30 87584 0 malloc-4096 4096 19 2 2950 0 2 86016 0 malloc-16384 16384 5 0 25 0 1 81920 0 malloc-256 256 34 281 1899803 0 62 80640 0 malloc-256 256 181 134 51141669 0 62 80640 0 UMA Kegs 384 198 5 472 0 30 77952 0 malloc-1024 1024 15 61 7868328 0 16 77824 0 unpcb 256 16 284 9339 0 62 76800 0 ttyinq 160 330 120 327584175 0 62 72000 0 malloc-32 32 132 2010 322776031 0 254 68544 0 malloc-65536 65536 1 0 2 0 1 65536 0 malloc-65536 65536 1 0 2305 0 1 65536 0 malloc-16384 16384 4 0 43 0 1 65536 0 malloc-64 64 76 932 531324 0 254 64512 0 2 Bucket 32 204 1812 47754143 0 254 64512 0 ttyoutq 256 173 67 174711554 0 62 61440 0 malloc-4096 4096 15 0 430 0 2 61440 0 malloc-64 64 28 917 507487 0 254 60480 0 malloc-128 128 53 412 3085709 0 126 59520 0 malloc-1024 1024 17 39 21855680 0 16 57344 0 malloc-16 16 841 2659 61704936 0 254 56000 0 clpbuf 2776 0 20 3009521 0 2 55520 0 malloc-128 128 1 402 1617019 0 126 51584 0 malloc-8192 8192 1 5 8693 0 1 49152 0 malloc-32 32 614 898 108317 0 254 48384 0 pcpu-64 64 455 249 467 0 254 45056 0 malloc-32 32 141 1245 187151561 0 254 44352 0 malloc-16 16 19 2731 1375945982 0 254 44000 0 malloc-4096 4096 10 0 520539 0 2 40960 0 malloc-1024 1024 3 37 1867688 0 16 40960 0 pcpu-16 16 136 2424 136 0 254 40960 0 pcpu-8 8 3506 1614 4898 0 254 40960 0 malloc-16 16 60 2440 7466033 0 254 40000 0 malloc-16 16 727 1773 2275616 0 254 40000 0 udp_inpcb 496 22 58 2398 0 30 39680 0 cpuset 104 27 345 27 0 126 38688 0 malloc-384 384 52 48 52 0 30 38400 0 malloc-1024 1024 19 17 13192969 0 16 36864 0 malloc-512 512 33 39 20560 0 30 36864 0 malloc-32768 32768 0 1 57500 0 1 32768 0 malloc-32768 32768 1 0 1 0 1 32768 0 malloc-2048 2048 10 6 12058 0 8 32768 0 malloc-4096 4096 1 6 1959844 0 2 28672 0 malloc-4096 4096 7 0 102 0 2 28672 0 rtentry 176 12 149 16 0 62 28336 0 udpcb 32 22 860 2398 0 254 28224 0 malloc-32 32 14 868 5633 0 254 28224 0 malloc-128 128 42 175 52 0 126 27776 0 swwbuf 2776 0 10 4732049 0 1 27760 0 malloc-384 384 31 39 44245 0 30 26880 0 malloc-384 384 25 45 12257 0 30 26880 0 malloc-2048 2048 6 6 16893 0 8 24576 0 malloc-1024 1024 15 9 192 0 16 24576 0 SMR CPU 32 82 653 82 0 254 23520 0 malloc-384 384 13 47 17 0 30 23040 0 tcpcb 1080 11 10 1054 0 8 22680 0 malloc-512 512 0 40 22084 0 30 20480 0 malloc-16 16 33 1217 42397733 0 254 20000 0 smrs 8 0 2465 20000001 0 254 19720 0 SMR SHARED 24 82 653 82 0 254 17640 0 malloc-16384 16384 1 0 3545552 0 1 16384 0 malloc-8192 8192 2 0 2 0 1 16384 0 malloc-2048 2048 0 8 2080 0 8 16384 0 malloc-64 64 4 248 8 0 254 16128 0 malloc-32 32 24 480 78 0 254 16128 0 malloc-16 16 31 969 60 0 254 16000 0 routing nhops 256 9 51 16 0 62 15360 0 malloc-512 512 1 23 1931 0 30 12288 0 tcp_inpcb 496 11 13 1054 0 30 11904 0 malloc-256 256 18 27 30 0 62 11520 0 malloc-256 256 20 25 60 0 62 11520 0 malloc-2048 2048 2 2 18 0 8 8192 0 malloc-512 512 6 10 52 0 30 8192 0 KMAP ENTRY 96 48 33 304 0 0 7776 0 kenv 258 18 12 21855 0 30 7740 0 AIOCB 552 0 14 32 0 16 7728 0 AIOLIO 272 0 28 8 0 30 7616 0 FPU_save_area 832 1 8 1 0 16 7488 0 epoch_record pcpu 256 4 12 4 0 62 4096 0 malloc-2048 2048 2 0 6501260 0 8 4096 0 malloc-1024 1024 4 0 4 0 16 4096 0 hostcache 64 0 63 4 0 254 4032 0 AIOP 32 4 122 4 0 254 4032 0 malloc-64 64 8 55 24518887 0 254 4032 0 vtnet_tx_hdr 24 0 167 0 0 254 4008 0 malloc-16 16 2 248 3 0 254 4000 0 ripcb 496 1 7 12 0 30 3968 0 AIO 208 0 19 2 0 62 3952 0 UMA Slabs 1 176 9 13 10 0 62 3872 0 malloc-384 384 1 9 33 0 30 3840 0 vmem 1856 1 1 1 0 8 3712 0 FFS1 dinode 128 0 0 0 0 126 0 0 da_ccb 544 0 0 0 0 16 0 0 udplite_inpcb 496 0 0 0 0 30 0 0 tcp_log_node 120 0 0 0 0 126 0 0 tcp_log_bucket 176 0 0 0 0 62 0 0 tcp_log 416 0 0 0 0 30 0 0 tcpreass 48 0 0 399 0 254 0 0 tfo_ccache_entries 80 0 0 0 0 126 0 0 tfo 4 0 0 0 0 254 0 0 sackhole 32 0 0 405 0 254 0 0 syncache 168 0 0 342 0 62 0 0 tcptw 88 0 0 39 0 126 0 0 ipq 56 0 0 0 0 254 0 0 IPsec SA lft_c 16 0 0 0 0 254 0 0 KNOTE 160 0 0 32 0 62 0 0 itimer 352 0 0 0 0 30 0 0 NCLNODE 736 0 0 0 0 16 0 0 LTS VFS Cache 360 0 0 0 0 30 0 0 STS VFS Cache 144 0 0 0 0 62 0 0 cryptop 280 0 0 0 0 30 0 0 linux_dma_object 24 0 0 0 0 254 0 0 linux_dma_pctrie 144 0 0 0 0 62 0 0 IOMMU_MAP_ENTRY 120 0 0 0 0 126 0 0 ktls_session 192 0 0 0 0 62 0 0 mbuf_jumbo_16k 16384 0 0 0 0 1 0 0 mbuf_jumbo_9k 9216 0 0 0 0 1 0 0 audit_record 1280 0 0 0 0 8 0 0 domainset 40 0 0 0 0 254 0 0 MAC labels 40 0 0 0 0 254 0 0 vnpbuf 2776 0 0 0 0 4 0 0 mdpbuf 2776 0 0 248419 0 1 0 0 nfspbuf 2776 0 0 0 0 2 0 0 umtx_shm 88 0 0 0 0 126 0 0 umtx pi 96 0 0 0 0 126 0 0 rangeset pctrie nodes 144 0 0 0 0 62 0 0 malloc-65536 65536 0 0 0 0 1 0 0 malloc-65536 65536 0 0 16 0 1 0 0 malloc-32768 32768 0 0 8 0 1 0 0 malloc-32768 32768 0 0 0 0 1 0 0 malloc-16384 16384 0 0 12 0 1 0 0 malloc-16384 16384 0 0 0 0 1 0 0 malloc-16384 16384 0 0 0 0 1 0 0 malloc-8192 8192 0 0 404 0 1 0 0 malloc-8192 8192 0 0 106 0 1 0 0 malloc-2048 2048 0 0 0 0 8 0 0 malloc-2048 2048 0 0 223 0 8 0 0 malloc-512 512 0 0 88 0 30 0 0 malloc-512 512 0 0 0 0 30 0 0 malloc-512 512 0 0 20622253 0 30 0 0 malloc-384 384 0 0 151515236 0 30 0 0 malloc-32 32 0 0 2 0 254 0 0 pcpu-32 32 0 0 0 0 254 0 0 pcpu-4 4 0 0 0 0 254 0 0 UMA Hash 256 0 0 0 0 62 0 0 db:0:pho> show umacache Zone Size Used Free Requests Bucket debugnet_mbuf_packet 2048 0 0 0 0 debugnet_mbuf_cluster 2048 -6144 0 0 0 debugnet_mbuf 256 -10240 0 0 0 ktls_buffers 16384 0 0 0 1 vm pgcache 4096 258331 670 855159444 13 vm pgcache 4096 6439913 5472 4721748003 169 buf free cache 848 20979 152 155261978 16 buffer arena-65 65536 0 0 0 1 buffer arena-61 61440 0 0 0 1 buffer arena-57 57344 0 0 0 1 buffer arena-53 53248 0 0 0 1 buffer arena-49 49152 0 0 0 1 buffer arena-45 45056 0 0 0 1 buffer arena-40 40960 0 0 0 1 buffer arena-36 36864 0 0 0 1 buffer arena-32 32768 20242 56 55604796 3 buffer arena-28 28672 0 0 0 1 buffer arena-24 24576 0 0 0 1 buffer arena-20 20480 0 0 0 1 buffer arena-16 16384 9 31 1542892 1 buffer arena-12 12288 0 0 0 1 buffer arena-81 8192 0 0 0 1 buffer arena-40 4096 0 0 2098 2 kstack_cache 16384 11718 2 71384807 1 db:0:pho> acttrace Tracing command fts pid 37084 tid 132078 td 0xfffffe017ad163a0 (CPU 6) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce61e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce61e10 trap() at trap+0x56/frame 0xfffffe003ce61f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce61f20 --- trap 0x13, rip = 0x8011e5b4c, rsp = 0x7fffffffe240, rbp = 0x7fffffffe2e0 --- Tracing command fts pid 37095 tid 100253 td 0xfffffe01436021e0 (CPU 9) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce88e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce88e10 trap() at trap+0x56/frame 0xfffffe003ce88f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce88f20 --- trap 0x13, rip = 0xffffffff80bc6f2d, rsp = 0xfffffe014167f730, rbp = 0xfffffe014167f750 --- pwd_hold() at pwd_hold+0x9d/frame 0xfffffe014167f750 namei_setup() at namei_setup+0x88/frame 0xfffffe014167f7c0 namei() at namei+0x332/frame 0xfffffe014167f870 kern_statat() at kern_statat+0x12d/frame 0xfffffe014167f9c0 sys_fstatat() at sys_fstatat+0x2f/frame 0xfffffe014167fac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014167fbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014167fbf0 --- syscall (552, FreeBSD ELF64, sys_fstatat), rip = 0x8011a7eda, rsp = 0x7fffffffe0b8, rbp = 0x7fffffffe170 --- Tracing command fts pid 37099 tid 129983 td 0xfffffe0176aaf3a0 (CPU 1) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce20e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce20e10 trap() at trap+0x56/frame 0xfffffe003ce20f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce20f20 --- trap 0x13, rip = 0xffffffff80f766ba, rsp = 0xfffffe018d31a778, rbp = 0xfffffe018d31a7c0 --- trash_ctor() at trash_ctor+0x3a/frame 0xfffffe018d31a7c0 namei() at namei+0x108/frame 0xfffffe018d31a870 kern_statat() at kern_statat+0x12d/frame 0xfffffe018d31a9c0 sys_fstatat() at sys_fstatat+0x2f/frame 0xfffffe018d31aac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018d31abf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018d31abf0 --- syscall (552, FreeBSD ELF64, sys_fstatat), rip = 0x8011a7eda, rsp = 0x7fffffffe0b8, rbp = 0x7fffffffe170 --- Tracing command fts pid 37104 tid 137069 td 0xfffffe0144b443a0 (CPU 8) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce7be00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce7be10 trap() at trap+0x56/frame 0xfffffe003ce7bf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce7bf20 --- trap 0x13, rip = 0xffffffff80cf1a98, rsp = 0xfffffe0195d806e0, rbp = 0xfffffe0195d807c0 --- cache_fplookup() at cache_fplookup+0x2a8/frame 0xfffffe0195d807c0 namei() at namei+0x204/frame 0xfffffe0195d80870 kern_statat() at kern_statat+0x12d/frame 0xfffffe0195d809c0 sys_fstatat() at sys_fstatat+0x2f/frame 0xfffffe0195d80ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0195d80bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0195d80bf0 --- syscall (552, FreeBSD ELF64, sys_fstatat), rip = 0x8011a7eda, rsp = 0x7fffffffe0b8, rbp = 0x7fffffffe170 --- Tracing command fts pid 37109 tid 127144 td 0xfffffe0161395740 (CPU 2) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce2de00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce2de10 trap() at trap+0x56/frame 0xfffffe003ce2df20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce2df20 --- trap 0x13, rip = 0x8010fcd35, rsp = 0x7fffffffe0a8, rbp = 0x7fffffffe0b0 --- Tracing command fts pid 37114 tid 128293 td 0xfffffe015a5c8c80 (CPU 3) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce3ae00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce3ae10 trap() at trap+0x56/frame 0xfffffe003ce3af20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce3af20 --- trap 0x13, rip = 0x1023dbd, rsp = 0x7fffffffe320, rbp = 0x7fffffffe360 --- Tracing command dirnprename pid 37116 tid 131643 td 0xfffffe015e5f5560 (CPU 4) kdb_enter() at kdb_enter+0x37/frame 0xfffffe018f386360 vpanic() at vpanic+0x1b8/frame 0xfffffe018f3863c0 panic() at panic+0x43/frame 0xfffffe018f386420 trap_fatal() at trap_fatal+0x387/frame 0xfffffe018f386480 trap_pfault() at trap_pfault+0x99/frame 0xfffffe018f3864e0 trap() at trap+0x2a7/frame 0xfffffe018f3865f0 calltrap() at calltrap+0x8/frame 0xfffffe018f3865f0 --- trap 0xc, rip = 0xffffffff8274a6be, rsp = 0xfffffe018f3866c0, rbp = 0xfffffe018f3866d0 --- null_checkvp() at null_checkvp+0xe/frame 0xfffffe018f3866d0 null_bypass() at null_bypass+0xc9/frame 0xfffffe018f386850 VOP_RENAME_APV() at VOP_RENAME_APV+0x3c/frame 0xfffffe018f386870 kern_renameat() at kern_renameat+0x3dd/frame 0xfffffe018f386ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018f386bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018f386bf0 --- syscall (128, FreeBSD ELF64, sys_rename), rip = 0x8011a68da, rsp = 0x7fffffffe208, rbp = 0x7fffffffe350 --- Tracing command fts pid 37118 tid 127092 td 0xfffffe01691f13a0 (CPU 0) cpustop_handler() at cpustop_handler+0x28/frame 0xffffffff81ff9220 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xffffffff81ff9230 trap() at trap+0x56/frame 0xffffffff81ff9340 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff81ff9340 --- trap 0x13, rip = 0x8011c7345, rsp = 0x7fffffffde98, rbp = 0x7fffffffdee0 --- Tracing command fts pid 37128 tid 127851 td 0xfffffe0149551ac0 (CPU 11) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003cea2e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003cea2e10 trap() at trap+0x56/frame 0xfffffe003cea2f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003cea2f20 --- trap 0x13, rip = 0x8011c7345, rsp = 0x7fffffffde78, rbp = 0x7fffffffdec0 --- Tracing command dirnprename pid 37129 tid 130308 td 0xfffffe0157891ac0 (CPU 5) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce54e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce54e10 trap() at trap+0x56/frame 0xfffffe003ce54f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce54f20 --- trap 0x13, rip = 0xffffffff80c703f0, rsp = 0xfffffe018d973200, rbp = 0xfffffe018d973210 --- lock_delay() at lock_delay+0x20/frame 0xfffffe018d973210 _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0xc1/frame 0xfffffe018d973280 __mtx_lock_spin_flags() at __mtx_lock_spin_flags+0xd5/frame 0xfffffe018d9732c0 cnputs() at cnputs+0x56/frame 0xfffffe018d9732f0 vprintf() at vprintf+0xcd/frame 0xfffffe018d9733c0 printf() at printf+0x43/frame 0xfffffe018d973420 trap_fatal() at trap_fatal+0x224/frame 0xfffffe018d973480 trap_pfault() at trap_pfault+0x99/frame 0xfffffe018d9734e0 trap() at trap+0x2a7/frame 0xfffffe018d9735f0 calltrap() at calltrap+0x8/frame 0xfffffe018d9735f0 --- trap 0xc, rip = 0xffffffff8274a6be, rsp = 0xfffffe018d9736c0, rbp = 0xfffffe018d9736d0 --- null_checkvp() at null_checkvp+0xe/frame 0xfffffe018d9736d0 null_bypass() at null_bypass+0xc9/frame 0xfffffe018d973850 VOP_RENAME_APV() at VOP_RENAME_APV+0x3c/frame 0xfffffe018d973870 kern_renameat() at kern_renameat+0x3dd/frame 0xfffffe018d973ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018d973bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018d973bf0 --- syscall (128, FreeBSD ELF64, sys_rename), rip = 0x8011a68da, rsp = 0x7fffffffe208, rbp = 0x7fffffffe350 --- Tracing command fts pid 37131 tid 128275 td 0xfffffe0169201900 (CPU 7) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce6ee00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce6ee10 trap() at trap+0x56/frame 0xfffffe003ce6ef20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce6ef20 --- trap 0x13, rip = 0x8011e6a4e, rsp = 0x7fffffffe000, rbp = 0x7fffffffe1b0 --- Tracing command fts pid 37137 tid 100284 td 0xfffffe0143e12560 (CPU 10) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce95e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce95e10 trap() at trap+0x56/frame 0xfffffe003ce95f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce95f20 --- trap 0x13, rip = 0x8011c70c0, rsp = 0x7fffffffe2d8, rbp = 0x7fffffffe310 --- db:0:pho> ps pid ppid pgrp uid state wmesg wchan cmd 38374 38306 3116 0 R sh 38372 36992 72830 0 S+ nanslp 0xffffffff81ebc0a0 sleep 38364 37106 37014 2007 R+ lockf 38363 37087 37014 2007 R+ lockf 38354 37046 37014 2007 R+ lockf 38346 37101 37014 2007 R+ lockf 38330 37092 37014 2007 R+ lockf 38326 37056 37014 2007 R+ lockf 38306 3157 3116 0 S wait 0xfffff805c29c2000 sh 38274 37110 37014 2007 R+ lockf 37154 37057 37014 2007 R+ swap 37153 37057 37014 2007 R+ swap 37152 37057 37014 2007 R+ swap 37151 37057 37014 2007 R+ swap 37150 37057 37014 2007 R+ swap 37149 37057 37014 2007 R+ swap 37148 37057 37014 2007 R+ swap 37147 37057 37014 2007 R+ swap 37146 37057 37014 2007 R+ swap 37145 37057 37014 2007 R+ swap 37144 37057 37014 2007 R+ swap 37143 37057 37014 2007 R+ swap 37142 37057 37014 2007 R+ swap 37141 37057 37014 2007 R+ swap 37140 37057 37014 2007 R+ swap 37139 37057 37014 2007 R+ swap 37138 37075 37014 2007 R+ dirnprename 37137 37042 37014 2007 R+ CPU 10 fts 37136 37057 37014 2007 R+ swap 37134 37042 37014 2007 R+ fts 37133 37057 37014 2007 R+ swap 37131 37042 37014 2007 R+ CPU 7 fts 37130 37057 37014 2007 R+ swap 37129 37075 37014 2007 R+ CPU 5 dirnprename 37128 37042 37014 2007 R+ CPU 11 fts 37127 37057 37014 2007 R+ swap 37125 37042 37014 2007 R+ fts 37124 37057 37014 2007 R+ swap 37122 37042 37014 2007 R+ fts 37121 37057 37014 2007 R+ swap 37118 37042 37014 2007 R+ CPU 0 fts 37117 37057 37014 2007 R+ swap 37116 37075 37014 2007 R+ CPU 4 dirnprename 37114 37042 37014 2007 R+ CPU 3 fts 37113 37057 37014 2007 R+ swap 37110 37043 37014 2007 R+ lockf 37109 37042 37014 2007 R+ CPU 2 fts 37108 37057 37014 2007 R+ swap 37106 37043 37014 2007 R+ lockf 37104 37042 37014 2007 R+ CPU 8 fts 37103 37057 37014 2007 R+ swap 37101 37043 37014 2007 R+ lockf 37099 37042 37014 2007 R+ CPU 1 fts 37098 37057 37014 2007 R+ swap 37096 37043 37014 2007 R+ lockf 37095 37042 37014 2007 R+ CPU 9 fts 37093 37075 37014 2007 R+ dirnprename 37092 37043 37014 2007 R+ lockf 37091 37057 37014 2007 R+ swap 37090 37042 37014 2007 R+ fts 37087 37043 37014 2007 R+ lockf 37086 37057 37014 2007 R+ swap 37085 37058 37014 2007 R+ creat 37084 37042 37014 2007 R+ CPU 6 fts 37081 37058 37014 2007 R+ creat 37080 37042 37014 2007 R+ fts 37079 37043 37014 2007 R+ lockf 37078 37057 37014 2007 R+ swap 37075 37036 37014 2007 R+ dirnprename 37074 37058 37014 2007 R+ creat 37071 37057 37014 2007 R+ swap 37070 37058 37014 2007 R+ creat 37068 37043 37014 2007 R+ lockf 37065 37057 37014 2007 R+ swap 37064 37058 37014 2007 R+ creat 37061 37042 37014 2007 R+ fts 37060 37038 37014 2007 R+ dirrename 37058 37026 37014 2007 S+ wait 0xfffff8014f0dca70 creat 37057 37030 37014 2007 S+ wait 0xfffff80528c9da70 swap 37056 37043 37014 2007 S+ lockf 0xfffff8076c1c1a80 lockf 37055 37042 37014 2007 R+ fts 37053 37044 37014 2007 R+ link 37052 37043 37014 2007 R+ lockf 37051 37042 37014 2007 R+ fts 37050 37044 37014 2007 R+ link 37049 37043 37014 2007 R+ lockf 37048 37042 37014 2007 R+ fts 37047 37044 37014 2007 R+ link 37046 37043 37014 2007 R+ lockf 37045 37042 37014 2007 R+ fts 37044 37024 37014 2007 S+ wait 0xfffff800274e1a70 link 37043 37025 37014 2007 S+ wait 0xfffff800274a5538 lockf 37042 37023 37014 2007 S+ wait 0xfffff8048724d538 fts 37041 37032 37014 2007 R+ openat 37040 37032 37014 2007 R+ openat 37039 37032 37014 2007 R+ openat 37038 37016 37014 2007 S+ nanslp 0xffffffff81ebc0ab dirrename 37037 37032 37014 2007 R+ openat 37036 37016 37014 2007 S+ nanslp 0xffffffff81ebc0a8 dirnprename 37035 37032 37014 2007 R+ openat 37034 37028 37014 2007 R+ lockf2 37032 37019 37014 2007 S+ wait 0xfffff804f1ad4538 openat 37031 37028 37014 2007 R+ lockf2 37030 37016 37014 2007 S+ nanslp 0xffffffff81ebc0a8 swap 37028 37017 37014 2007 S+ wait 0xfffff802bfbd0000 lockf2 37026 37016 37014 2007 S+ nanslp 0xffffffff81ebc0a7 creat 37025 37016 37014 2007 S+ nanslp 0xffffffff81ebc0a4 lockf 37024 37016 37014 2007 S+ nanslp 0xffffffff81ebc0a8 link 37023 37016 37014 2007 S+ nanslp 0xffffffff81ebc0ab fts 37019 37016 37014 2007 S+ nanslp 0xffffffff81ebc0ab openat 37017 37016 37014 2007 S+ nanslp 0xffffffff81ebc0a8 lockf2 37016 37015 37014 2007 R+ run 37015 37014 37014 2007 S+ wait 0xfffff801b926d538 run 37014 37011 37014 2007 S+ nanslp 0xffffffff81ebc0a8 run 37011 36992 72830 0 S+ wait 0xfffff8048cc34000 su 36992 36990 72830 0 S+ wait 0xfffff8046e4a9000 sh 36991 72830 72830 0 S+ piperd 0xfffff806cfff5ba0 tee 36990 72830 72830 0 S+ wait 0xfffff8048f26ba70 sh 72830 2788 72830 0 S+ wait 0xfffff80267cc1a70 sh 32520 1 32520 0 Ss nanslp 0xffffffff81ebc0a5 watchdogd 71139 0 0 0 DL aiordy 0xfffff80640942a70 [aiod4] 71138 0 0 0 DL aiordy 0xfffff8077b6d9538 [aiod3] 71137 0 0 0 DL aiordy 0xfffff8077b6d9a70 [aiod2] 71136 0 0 0 DL aiordy 0xfffff8064ae62538 [aiod1] 3160 3116 3116 0 S piperd 0xfffff800272aeba0 sh 3158 3116 3116 0 S piperd 0xfffff800279c7000 awk 3157 3116 3116 0 S wait 0xfffff802a9a35538 sh 3116 1 3116 0 SWs wait 0xfffff802a9a35a70 sh 2788 2786 2788 0 SW+ wait 0xfffff800279b4538 bash 2786 2785 2786 0 SW+ pause 0xfffff802a9959b20 csh 2785 2780 2785 2006 SW+ wait 0xfffff80006041538 su 2780 2779 2780 2006 SWs+ wait 0xfffff800274d2538 bash 2779 2776 2776 2006 S select 0xfffff800271e3840 sshd 2776 2744 2776 0 Ss select 0xfffff800060c1f40 sshd 2771 1 2771 0 Ss+ ttyin 0xfffff800043a28b0 getty 2770 1 2770 0 Ss+ ttyin 0xfffff800043a2cb0 getty 2769 1 2769 0 Ss+ ttyin 0xfffff8001c3390b0 getty 2768 1 2768 0 Ss+ ttyin 0xfffff8001c3394b0 getty 2767 1 2767 0 Ss+ ttyin 0xfffff8001c3398b0 getty 2766 1 2766 0 Ss+ ttyin 0xfffff8001c339cb0 getty 2765 1 2765 0 Ss+ ttyin 0xfffff8001c3430b0 getty 2764 1 2764 0 Ss+ ttyin 0xfffff8001c3434b0 getty 2763 1 2763 0 Ss+ ttyin 0xfffff8001c3438b0 getty 2762 1 2762 0 Ss+ ttyin 0xfffff8001c343cb0 getty 2748 1 2748 0 Ss nanslp 0xffffffff81ebc0a3 cron 2744 1 2744 0 Ss select 0xfffff800272a6f40 sshd 2725 1 2725 0 Ss select 0xfffff8001c1db5c0 inetd 2699 1 2699 123 Ss select 0xfffff800271e38c0 ntpd 2673 1 2673 0 Ss select 0xfffff8001f5cce40 powerd 2668 1 2668 0 Ss rpcsvc 0xfffff8004920bd60 NLM: master 2665 1 2665 0 Ss select 0xfffff80004046ec0 rpc.statd 2640 1 2640 0 Ss select 0xfffff8001f522e40 rpcbind 2610 1 2610 0 Ss select 0xfffff8001f5ccd40 syslogd 2423 1 2423 0 Ss select 0xfffff800270e5340 devd 2418 1 2418 65 Ss select 0xfffff800041666c0 dhclient 2342 1 2342 0 Ss select 0xfffff8001f5ccdc0 dhclient 2339 1 2339 0 Ss select 0xfffff800060c1a40 dhclient 34 0 0 0 DL syncer 0xffffffff81fab480 [syncer] 33 0 0 0 DL vlruwt 0xfffff800271bb538 [vnlru] 32 0 0 0 DL (threaded) [bufdaemon] 100213 D qsleep 0xffffffff81faa520 [bufdaemon] 100214 D - 0xffffffff81a0af40 [bufspacedaemon-0] 100215 D - 0xffffffff81a13200 [bufspacedaemon-1] 100216 D - 0xffffffff81a1b4c0 [bufspacedaemon-2] 100217 D - 0xffffffff81a23780 [bufspacedaemon-3] 100218 D - 0xffffffff81a2ba40 [bufspacedaemon-4] 100219 D - 0xffffffff81a33d00 [bufspacedaemon-5] 100220 D - 0xffffffff81a3bfc0 [bufspacedaemon-6] 100236 D sdflush 0xfffff800040a08e8 [/ worker] 100243 D sdflush 0xfffff8000410d8e8 [/tmp worker] 126093 D sdflush 0xfffff8025ddac8e8 [/mnt worker] 31 0 0 0 DL psleep 0xffffffff81fd0208 [vmdaemon] 30 0 0 0 DL (threaded) [pagedaemon] 100211 D psleep 0xffffffff81fc4678 [dom0] 100221 D launds 0xffffffff81fc4684 [laundry: dom0] 100222 D umarcl 0xffffffff80f708b0 [uma] 29 0 0 0 DL idle 0xfffff80004118538 [enc_daemon1] 28 0 0 0 DL idle 0xfffff80004118a70 [enc_daemon0] 27 0 0 0 DL - 0xffffffff81cf03d8 [rand_harvestq] 26 0 0 0 DL - 0xffffffff81fa7bdc [soaiod4] 25 0 0 0 DL - 0xffffffff81fa7bdc [soaiod3] 24 0 0 0 DL - 0xffffffff81fa7bdc [soaiod2] 23 0 0 0 DL - 0xffffffff81fa7bdc [soaiod1] 22 0 0 0 DL (threaded) [usb] 100140 D - 0xfffffe010a188440 [usbus0] 100141 D - 0xfffffe010a188498 [usbus0] 100142 D - 0xfffffe010a1884f0 [usbus0] 100143 D - 0xfffffe010a188548 [usbus0] 100144 D - 0xfffffe010a1885a0 [usbus0] 100146 D - 0xfffffe00e4d66cf0 [usbus1] 100147 D - 0xfffffe00e4d66d48 [usbus1] 100148 D - 0xfffffe00e4d66da0 [usbus1] 100149 D - 0xfffffe00e4d66df8 [usbus1] 100150 D - 0xfffffe00e4d66e50 [usbus1] 100151 D - 0xfffffe010a548cf0 [usbus2] 100152 D - 0xfffffe010a548d48 [usbus2] 100153 D - 0xfffffe010a548da0 [usbus2] 100154 D - 0xfffffe010a548df8 [usbus2] 100155 D - 0xfffffe010a548e50 [usbus2] 21 0 0 0 RL (threaded) [cam] 100133 D - 0xffffffff81bc7ac0 [doneq0] 100134 RunQ [doneq1] 100135 D - 0xffffffff81bc7bc0 [doneq2] 100136 D - 0xffffffff81bc7a40 [async] 100168 D - 0xffffffff81bc7910 [scanner] 20 0 0 0 DL seqstat 0xfffff80003e0dc88 [sequencer 00] 19 0 0 0 DL crypto_ 0xfffff80003bbffa0 [crypto returns 11] 18 0 0 0 DL crypto_ 0xfffff80003bbff50 [crypto returns 10] 17 0 0 0 DL crypto_ 0xfffff80003bbff00 [crypto returns 9] 16 0 0 0 DL crypto_ 0xfffff80003bbfeb0 [crypto returns 8] 15 0 0 0 DL crypto_ 0xfffff80003bbfe60 [crypto returns 7] 14 0 0 0 DL crypto_ 0xfffff80003bbfe10 [crypto returns 6] 9 0 0 0 DL crypto_ 0xfffff80003bbfdc0 [crypto returns 5] 8 0 0 0 DL crypto_ 0xfffff80003bbfd70 [crypto returns 4] 7 0 0 0 DL crypto_ 0xfffff80003bbfd20 [crypto returns 3] 6 0 0 0 DL crypto_ 0xfffff80003bbfcd0 [crypto returns 2] 5 0 0 0 DL crypto_ 0xfffff80003bbfc80 [crypto returns 1] 4 0 0 0 DL crypto_ 0xfffff80003bbfc30 [crypto returns 0] 3 0 0 0 DL crypto_ 0xffffffff81fc1ba0 [crypto] 13 0 0 0 DL (threaded) [geom] 100104 D - 0xffffffff81e9b920 [g_event] 100105 D - 0xffffffff81e9b928 [g_up] 100106 D - 0xffffffff81e9b930 [g_down] 2 0 0 0 DL (threaded) [KTLS] 100077 D - 0xfffff80003afc800 [thr_0] 100078 D - 0xfffff80003afc880 [thr_1] 100079 D - 0xfffff80003afc900 [thr_2] 100080 D - 0xfffff80003afc980 [thr_3] 100081 D - 0xfffff80003afca00 [thr_4] 100082 D - 0xfffff80003afca80 [thr_5] 100083 D - 0xfffff80003afcb00 [thr_6] 100084 D - 0xfffff80003afcb80 [thr_7] 100085 D - 0xfffff80003afcc00 [thr_8] 100086 D - 0xfffff80003afcc80 [thr_9] 100087 D - 0xfffff80003afcd00 [thr_10] 100088 D - 0xfffff80003afcd80 [thr_11] 12 0 0 0 WL (threaded) [intr] 100046 I [swi6: task queue] 100047 I [swi6: Giant taskq] 100049 I [swi5: fast taskq] 100089 I [swi4: clock (0)] 100090 I [swi4: clock (1)] 100091 I [swi4: clock (2)] 100092 I [swi4: clock (3)] 100093 I [swi4: clock (4)] 100094 I [swi4: clock (5)] 100095 I [swi4: clock (6)] 100096 I [swi4: clock (7)] 100097 I [swi4: clock (8)] 100098 I [swi4: clock (9)] 100099 I [swi4: clock (10)] 100100 I [swi4: clock (11)] 100101 I [swi1: netisr 0] 100102 I [swi3: vm] 100137 I [swi4: apei] 100138 I [irq56: ahci0] 100139 I [irq19: xhci0] 100145 I [irq18: ehci0 ehci1+] 100156 I [irq71: ahci1] 100157 I [swi0: uart uart] 100175 I [irq73: mlx5_core0] 100176 I [irq74: mlx5_core0] 100177 I [irq72: mlx5_core0] 100178 I [irq75: mlx5_core0] 100179 I [irq76: mlx5_core0] 100180 I [irq77: mlx5_core0] 100181 I [irq78: mlx5_core0] 100182 I [irq79: mlx5_core0] 100183 I [irq80: mlx5_core0] 100184 I [irq81: mlx5_core0] 100185 I [irq82: mlx5_core0] 100186 I [irq83: mlx5_core0] 100187 I [irq84: mlx5_core0] 100188 I [irq85: mlx5_core0] 100189 I [irq86: mlx5_core0] 100195 I [irq88: mlx5_core1] 100196 I [irq89: mlx5_core1] 100197 I [irq87: mlx5_core1] 100198 I [irq90: mlx5_core1] 100199 I [irq91: mlx5_core1] 100200 I [irq92: mlx5_core1] 100201 I [irq93: mlx5_core1] 100202 I [irq94: mlx5_core1] 100203 I [irq95: mlx5_core1] 100204 I [irq96: mlx5_core1] 100205 I [irq97: mlx5_core1] 100206 I [irq98: mlx5_core1] 100207 I [irq99: mlx5_core1] 100208 I [irq100: mlx5_core1] 100209 I [irq101: mlx5_core1] 100265 I [irq102: ioat0] 100266 I [irq103: ioat1] 100267 I [irq104: ioat2] 100268 I [irq105: ioat3] 100269 I [irq106: ioat4] 100270 I [irq107: ioat5] 100271 I [irq108: ioat6] 100272 I [irq109: ioat7] 11 0 0 0 RL (threaded) [idle] 100003 CanRun [idle: cpu0] 100004 CanRun [idle: cpu1] 100005 CanRun [idle: cpu2] 100006 CanRun [idle: cpu3] 100007 CanRun [idle: cpu4] 100008 CanRun [idle: cpu5] 100009 CanRun [idle: cpu6] 100010 CanRun [idle: cpu7] 100011 CanRun [idle: cpu8] 100012 CanRun [idle: cpu9] 100013 CanRun [idle: cpu10] 100014 CanRun [idle: cpu11] 1 0 1 0 SLs wait 0xfffff80003b43538 [init] 10 0 0 0 DL audit_w 0xffffffff81fc20b0 [audit] 0 0 0 0 DLs (threaded) [kernel] 100000 D swapin 0xffffffff81e9beb0 [swapper] 100015 D - 0xfffff80003965c00 [softirq_0] 100016 D - 0xfffff80003965b00 [softirq_1] 100017 D - 0xfffff80003965a00 [softirq_2] 100018 D - 0xfffff80003965900 [softirq_3] 100019 D - 0xfffff80003965800 [softirq_4] 100020 D - 0xfffff80003965700 [softirq_5] 100021 D - 0xfffff80003965600 [softirq_6] 100022 D - 0xfffff80003965500 [softirq_7] 100023 D - 0xfffff80003965400 [softirq_8] 100024 D - 0xfffff80003965300 [softirq_9] 100025 D - 0xfffff80003965200 [softirq_10] 100026 D - 0xfffff80003965100 [softirq_11] 100027 D - 0xfffff80003965000 [if_io_tqg_0] 100028 D - 0xfffff80003b1ce00 [if_io_tqg_1] 100029 D - 0xfffff80003b1cd00 [if_io_tqg_2] 100030 D - 0xfffff80003b1cc00 [if_io_tqg_3] 100031 D - 0xfffff80003b1cb00 [if_io_tqg_4] 100032 D - 0xfffff80003b1ca00 [if_io_tqg_5] 100033 D - 0xfffff80003b1c900 [if_io_tqg_6] 100034 D - 0xfffff80003b1c800 [if_io_tqg_7] 100035 D - 0xfffff80003b1c700 [if_io_tqg_8] 100036 D - 0xfffff80003b1c600 [if_io_tqg_9] 100037 D - 0xfffff80003b1c500 [if_io_tqg_10] 100038 D - 0xfffff80003b1c400 [if_io_tqg_11] 100039 D - 0xfffff80003b1c300 [if_config_tqg_0] 100040 D - 0xfffff80003b1fb00 [pci_hp taskq] 100041 D - 0xfffff80003b1fa00 [aiod_kick taskq] 100042 D - 0xfffff80003b1f900 [kqueue_ctx taskq] 100043 D - 0xfffff80003b1f800 [inm_free taskq] 100044 D ufs 0xfffffe0156ecbc28 [deferred_unmount ta] 100045 D - 0xfffff80003b1f600 [linuxkpi_irq_wq] 100048 D - 0xfffff80003b1f100 [thread taskq] 100050 D - 0xfffff80003b12d00 [in6m_free taskq] 100051 D - 0xfffff80003b12c00 [linuxkpi_short_wq_0] 100052 D - 0xfffff80003b12c00 [linuxkpi_short_wq_1] 100053 D - 0xfffff80003b12c00 [linuxkpi_short_wq_2] 100054 D - 0xfffff80003b12c00 [linuxkpi_short_wq_3] 100055 D - 0xfffff80003b12c00 [linuxkpi_short_wq_4] 100056 D - 0xfffff80003b12c00 [linuxkpi_short_wq_5] 100057 D - 0xfffff80003b12c00 [linuxkpi_short_wq_6] 100058 D - 0xfffff80003b12c00 [linuxkpi_short_wq_7] 100059 D - 0xfffff80003b12c00 [linuxkpi_short_wq_8] 100060 D - 0xfffff80003b12c00 [linuxkpi_short_wq_9] 100061 D - 0xfffff80003b12c00 [linuxkpi_short_wq_1] 100062 D - 0xfffff80003b12c00 [linuxkpi_short_wq_1] 100063 D - 0xfffff80003b12c00 [linuxkpi_short_wq_1] 100064 D - 0xfffff80003b12b00 [linuxkpi_long_wq_0] 100065 D - 0xfffff80003b12b00 [linuxkpi_long_wq_1] 100066 D - 0xfffff80003b12b00 [linuxkpi_long_wq_2] 100067 D - 0xfffff80003b12b00 [linuxkpi_long_wq_3] 100068 D - 0xfffff80003b12b00 [linuxkpi_long_wq_4] 100069 D - 0xfffff80003b12b00 [linuxkpi_long_wq_5] 100070 D - 0xfffff80003b12b00 [linuxkpi_long_wq_6] 100071 D - 0xfffff80003b12b00 [linuxkpi_long_wq_7] 100072 D - 0xfffff80003b12b00 [linuxkpi_long_wq_8] 100073 D - 0xfffff80003b12b00 [linuxkpi_long_wq_9] 100074 D - 0xfffff80003b12b00 [linuxkpi_long_wq_10] 100075 D - 0xfffff80003b12b00 [linuxkpi_long_wq_11] 100076 D - 0xfffff80003b12b00 [linuxkpi_long_wq_12] 100103 D - 0xfffff80003af2a00 [firmware taskq] 100107 D - 0xfffff80003af2900 [crypto_0] 100108 D - 0xfffff80003af2900 [crypto_1] 100109 D - 0xfffff80003af2900 [crypto_2] 100110 D - 0xfffff80003af2900 [crypto_3] 100111 D - 0xfffff80003af2900 [crypto_4] 100112 D - 0xfffff80003af2900 [crypto_5] 100113 D - 0xfffff80003af2900 [crypto_6] 100114 D - 0xfffff80003af2900 [crypto_7] 100115 D - 0xfffff80003af2900 [crypto_8] 100116 D - 0xfffff80003af2900 [crypto_9] 100117 D - 0xfffff80003af2900 [crypto_10] 100118 D - 0xfffff80003af2900 [crypto_11] 100158 D - 0xfffff800043d8c00 [mca taskq] 100164 D - 0xfffff8001c1f5e00 [acpi_task_0] 100165 D - 0xfffff8001c1f5e00 [acpi_task_1] 100166 D - 0xfffff8001c1f5e00 [acpi_task_2] 100167 D - 0xfffff80003af2800 [CAM taskq] 100171 D - 0xfffff8001c1f5d00 [mlx5_core0-rec] 100172 D - 0xfffff8001c1f5c00 [mlx5_core0-wdg] 100173 D - 0xfffff8001c1f5b00 [mlx5_core0-cmd] 100174 D - 0xfffff8001c1f5a00 [mlx5_page_allocator] 100190 D - 0xfffff80022b07900 [mlx5_esw_wq] 100191 D - 0xfffff80022b07800 [mlx5_core1-rec] 100192 D - 0xfffff80022b07700 [mlx5_core1-wdg] 100193 D - 0xfffff80022b07600 [mlx5_core1-cmd] 100194 D - 0xfffff80022b07500 [mlx5_page_allocator] 100210 D - 0xfffff800041b0600 [mlx5_esw_wq] 37018 37016 37014 2007 Z+ symlink 37020 37016 37014 2007 Z+ socket 37021 37016 37014 2007 Z+ rw 37022 37016 37014 2007 Z+ mmap 37027 37016 37014 2007 Z+ mkdir 37029 37016 37014 2007 Z+ rename 37033 37016 37014 2007 Z+ mkfifo 37063 37043 37014 2007 Z+ lockf 37067 37060 37014 2007 Z+ dirrename 37073 37060 37014 2007 Z+ dirrename 37082 37075 37014 2007 Z+ dirnprename 37088 37075 37014 2007 Z+ dirnprename 37097 37075 37014 2007 Z+ dirnprename 37102 37075 37014 2007 Z+ dirnprename 37107 37075 37014 2007 Z+ dirnprename 37112 37075 37014 2007 Z+ dirnprename 37120 37075 37014 2007 Z+ dirnprename 37123 37075 37014 2007 Z+ dirnprename 37126 37075 37014 2007 Z+ dirnprename 37132 37075 37014 2007 Z+ dirnprename 37135 37075 37014 2007 Z+ dirnprename db:0:pho> allt Tracing command kernel pid 0 tid 100000 td 0xffffffff81e9c400 sched_switch() at sched_switch+0x5af/frame 0xffffffff8254dac0 mi_switch() at mi_switch+0x155/frame 0xffffffff8254dae0 sleepq_switch() at sleepq_switch+0x119/frame 0xffffffff8254db20 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xffffffff8254db60 _sleep() at _sleep+0x298/frame 0xffffffff8254dc00 swapper() at swapper+0x75/frame 0xffffffff8254dc60 mi_startup() at mi_startup+0x2f3/frame 0xffffffff8254dcb0 btext() at btext+0x22 Tracing command kernel pid 0 tid 100015 td 0xfffffe00e4c5fc80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377ccab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377ccad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00377ccb10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00377ccb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377ccbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377ccbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377ccbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xfffffe00e4c5f560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377c7ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377c7ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00377c7b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00377c7b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377c7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377c7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377c7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100017 td 0xfffffe00e4c5ee40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377d6ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377d6ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00377d6b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00377d6b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377d6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377d6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377d6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100018 td 0xfffffe00e4c5e720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377d1ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377d1ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00377d1b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00377d1b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377d1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377d1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377d1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xfffffe00e4c5e000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377e0ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377e0ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00377e0b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00377e0b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377e0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377e0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377e0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xfffffe00e4c62740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377dbab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377dbad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00377dbb10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00377dbb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377dbbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377dbbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377dbbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xfffffe00e4c62020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377eaab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377eaad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00377eab10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00377eab80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377eabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377eabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377eabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xfffffe00e4c61900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377e5ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377e5ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00377e5b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00377e5b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377e5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377e5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377e5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xfffffe00e4cde1e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377f4ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377f4ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00377f4b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00377f4b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377f4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377f4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377f4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xfffffe00e4cddac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377efab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377efad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00377efb10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00377efb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377efbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377efbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377efbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xfffffe00e4cdd3a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377feab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377fead0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00377feb10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00377feb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377febb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377febf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377febf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100026 td 0xfffffe00e4cdcc80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377f9ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377f9ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00377f9b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00377f9b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377f9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377f9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377f9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100027 td 0xfffffe00e4cdc560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4835ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4835ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4835b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e4835b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4835bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4835bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4835bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100028 td 0xfffffe00e4cdbe40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4830ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4830ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4830b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e4830b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4830bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4830bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4830bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100029 td 0xfffffe00e4cdb720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e483fab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e483fad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e483fb10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e483fb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e483fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e483fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e483fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100030 td 0xfffffe00e4cdb000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e483aab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e483aad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e483ab10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e483ab80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e483abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e483abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e483abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100031 td 0xfffffe00e4cdf740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4849ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4849ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4849b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e4849b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4849bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4849bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4849bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100032 td 0xfffffe00e4cdf020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4844ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4844ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4844b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e4844b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4844bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4844bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4844bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100033 td 0xfffffe00e4cde900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4853ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4853ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4853b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e4853b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4853bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4853bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4853bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100034 td 0xfffffe00e4ce51e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e484eab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e484ead0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e484eb10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e484eb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e484ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e484ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e484ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100035 td 0xfffffe00e4ce4ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e485dab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e485dad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e485db10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e485db80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e485dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e485dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e485dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100036 td 0xfffffe00e4ce43a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4858ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4858ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4858b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e4858b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4858bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4858bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4858bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100037 td 0xfffffe00e4ce3c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4867ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4867ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4867b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e4867b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4867bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4867bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4867bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100038 td 0xfffffe00e4ce3560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4862ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4862ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4862b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e4862b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4862bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4862bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4862bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100039 td 0xfffffe00e4ce2e40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4871ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4871ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4871b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e4871b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4871bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4871bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4871bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100040 td 0xfffffe00e4ce2720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e486ca80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e486caa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e486cae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e486cb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e486cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e486cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e486cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100041 td 0xfffffe00e4ce2000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e487ba80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e487baa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e487bae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e487bb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e487bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e487bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e487bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100042 td 0xfffffe00e4ce6740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4876a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4876aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4876ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4876b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4876bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4876bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4876bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100043 td 0xfffffe00e4ce6020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4885a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4885aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4885ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4885b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4885bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4885bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4885bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100044 td 0xfffffe00e4ce5900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4880610 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4880630 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4880670 sleeplk() at sleeplk+0x100/frame 0xfffffe00e48806d0 lockmgr_xlock_hard() at lockmgr_xlock_hard+0x402/frame 0xfffffe00e4880780 ffs_lock() at ffs_lock+0x9d/frame 0xfffffe00e48807c0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe00e48807e0 null_lock() at null_lock+0xd3/frame 0xfffffe00e4880840 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe00e4880860 _vn_lock() at _vn_lock+0x54/frame 0xfffffe00e48808c0 vflush() at vflush+0xd0/frame 0xfffffe00e4880a00 nullfs_unmount() at nullfs_unmount+0x51/frame 0xfffffe00e4880a40 dounmount() at dounmount+0x812/frame 0xfffffe00e4880ab0 vfs_deferred_unmount() at vfs_deferred_unmount+0xf9/frame 0xfffffe00e4880b00 taskqueue_run_locked() at taskqueue_run_locked+0xaa/frame 0xfffffe00e4880b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc2/frame 0xfffffe00e4880bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4880bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4880bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100045 td 0xfffffe00e4cec1e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e488fab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e488fad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e488fb10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e488fb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe00e488fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e488fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e488fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100048 td 0xfffffe00e4ceac80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4894a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4894aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4894ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4894b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4894bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4894bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4894bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100050 td 0xfffffe00e4ce9e40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e489ea80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e489eaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e489eae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e489eb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e489ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e489ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e489ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100051 td 0xfffffe00e4ce9720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48ada80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48adaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48adae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48adb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48adbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48adbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48adbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100052 td 0xfffffe00e4ce9000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48a8a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48a8aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48a8ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48a8b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48a8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48a8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48a8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100053 td 0xfffffe00e4ced740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48b7a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48b7aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48b7ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48b7b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48b7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48b7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48b7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100054 td 0xfffffe00e4ced020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48b2a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48b2aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48b2ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48b2b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48b2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48b2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48b2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100055 td 0xfffffe00e4cec900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48c1a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48c1aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48c1ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48c1b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48c1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48c1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48c1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100056 td 0xfffffe00e4cf11e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48bca80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48bcaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48bcae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48bcb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48bcbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48bcbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48bcbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100057 td 0xfffffe00e4cf0ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48cba80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48cbaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48cbae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48cbb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48cbbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48cbbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48cbbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100058 td 0xfffffe00e4cf03a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48c6a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48c6aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48c6ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48c6b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48c6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48c6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48c6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100059 td 0xfffffe00e4cefc80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48d5a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48d5aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48d5ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48d5b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48d5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48d5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48d5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100060 td 0xfffffe00e4cef560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48d0a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48d0aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48d0ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48d0b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48d0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48d0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48d0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100061 td 0xfffffe00e4ceee40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48dfa80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48dfaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48dfae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48dfb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48dfbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48dfbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48dfbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100062 td 0xfffffe00e4cee720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48daa80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48daaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48daae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48dab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48dabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48dabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48dabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100063 td 0xfffffe00e4cee000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48e9a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48e9aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48e9ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48e9b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48e9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48e9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48e9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100064 td 0xfffffe00e4cf2740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48e4a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48e4aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48e4ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48e4b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48e4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48e4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48e4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100065 td 0xfffffe00e4cf2020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48f3a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48f3aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48f3ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48f3b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48f3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48f3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48f3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100066 td 0xfffffe00e4cf1900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48eea80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48eeaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48eeae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48eeb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48eebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48eebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48eebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100067 td 0xfffffe00e4cf61e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48fda80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48fdaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48fdae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48fdb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48fdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48fdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48fdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100068 td 0xfffffe00e4cf5ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48f8a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48f8aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48f8ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48f8b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48f8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48f8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48f8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100069 td 0xfffffe00e4cf53a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4907a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4907aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4907ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4907b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4907bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4907bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4907bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100070 td 0xfffffe00e4cf4c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4902a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4902aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4902ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4902b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4902bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4902bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4902bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100071 td 0xfffffe00e4cf4560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4911a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4911aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4911ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4911b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4911bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4911bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4911bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100072 td 0xfffffe00e4cf3e40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e490ca80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e490caa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e490cae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e490cb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e490cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e490cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e490cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100073 td 0xfffffe00e4cf3720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e491ba80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e491baa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e491bae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e491bb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e491bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e491bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e491bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100074 td 0xfffffe00e4cf3000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4916a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4916aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4916ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4916b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4916bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4916bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4916bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100075 td 0xfffffe00e4cf7740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4925a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4925aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4925ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4925b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4925bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4925bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4925bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100076 td 0xfffffe00e4cf7020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4920a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4920aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4920ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4920b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4920bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4920bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4920bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100103 td 0xfffffe00e4d08c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49eda80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49edaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e49edae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e49edb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e49edbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49edbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49edbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100107 td 0xfffffe00e4d07000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e04a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e04aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e04ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e04b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e04bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e04bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e04bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100108 td 0xfffffe00e4d0b740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49fca80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49fcaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e49fcae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e49fcb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e49fcbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49fcbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49fcbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100109 td 0xfffffe00e4d0b020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e0ea80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e0eaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e0eae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e0eb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e0ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e0ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e0ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100110 td 0xfffffe00e4d0a900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e09a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e09aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e09ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e09b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e09bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e09bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e09bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100111 td 0xfffffe00e4d241e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e18a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e18aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e18ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e18b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e18bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e18bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e18bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100112 td 0xfffffe00e4d23ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e13a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e13aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e13ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e13b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e13bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e13bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e13bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100113 td 0xfffffe00e4d233a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e22a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e22aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e22ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e22b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e22bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e22bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e22bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100114 td 0xfffffe00e4d22c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e1da80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e1daa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e1dae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e1db80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e1dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e1dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e1dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100115 td 0xfffffe00e4d22560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e2ca80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e2caa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e2cae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e2cb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e2cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e2cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e2cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100116 td 0xfffffe00e4d21e40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e27a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e27aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e27ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e27b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e27bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e27bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e27bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100117 td 0xfffffe00e4d21720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e36a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e36aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e36ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e36b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e36bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e36bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e36bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100118 td 0xfffffe00e4d21000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e31a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e31aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e31ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e31b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e31bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e31bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e31bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100158 td 0xfffffe010a6d7000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f05ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f05ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f05b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e4f05b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe00e4f05bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f05bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f05bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100164 td 0xfffffe010d676020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f53ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f53ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f53b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e4f53b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe00e4f53bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f53bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f53bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100165 td 0xfffffe010d675900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f62ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f62ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f62b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e4f62b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe00e4f62bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f62bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f62bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100166 td 0xfffffe010d9f71e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f5dab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f5dad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f5db10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e4f5db80 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe00e4f5dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f5dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f5dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100167 td 0xfffffe010d9f6ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f6ca80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f6caa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f6cae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4f6cb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4f6cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f6cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f6cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100171 td 0xfffffe010d9f5c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f80a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f80aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f80ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4f80b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4f80bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f80bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f80bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100172 td 0xfffffe010d9f5560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f7ba80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f7baa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f7bae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4f7bb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4f7bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f7bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f7bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100173 td 0xfffffe010d9f4e40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f8aa80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f8aaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f8aae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4f8ab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4f8abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f8abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f8abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100174 td 0xfffffe010d9f4720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f85a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f85aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f85ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4f85b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4f85bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f85bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f85bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100190 td 0xfffffe010dd37c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4fd5a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4fd5aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4fd5ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4fd5b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4fd5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4fd5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4fd5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100191 td 0xfffffe010dd37560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4fe4a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4fe4aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4fe4ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4fe4b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4fe4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4fe4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4fe4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100192 td 0xfffffe010dd36e40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4fdfa80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4fdfaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4fdfae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4fdfb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4fdfbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4fdfbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4fdfbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100193 td 0xfffffe010dd36720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4feea80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4feeaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4feeae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4feeb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4feebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4feebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4feebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100194 td 0xfffffe010dd36000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4fe9a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4fe9aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4fe9ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4fe9b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4fe9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4fe9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4fe9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100210 td 0xfffffe010df2cc80 sched_switch() at sched_switch+0x5af/frame 0xfffffe010e64aa80 mi_switch() at mi_switch+0x155/frame 0xfffffe010e64aaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe010e64aae0 _sleep() at _sleep+0x32f/frame 0xfffffe010e64ab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe010e64abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe010e64abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010e64abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command init pid 1 tid 100002 td 0xfffffe00e4c5bac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377816d0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377816f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0037781730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0037781780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0037781790 _sleep() at _sleep+0x325/frame 0xfffffe0037781830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe00377818c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0037781ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0037781bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0037781bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x28e2aa, rsp = 0x7fffffffe718, rbp = 0x7fffffffe780 --- Tracing command KTLS pid 2 tid 100077 td 0xfffffe00e4cf6900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e493b9b0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e493b9d0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e493ba10 _sleep() at _sleep+0x32f/frame 0xfffffe00e493bab0 ktls_work_thread() at ktls_work_thread+0xdc/frame 0xfffffe00e493bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e493bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e493bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100078 td 0xfffffe00e4d001e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49369b0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49369d0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4936a10 _sleep() at _sleep+0x32f/frame 0xfffffe00e4936ab0 ktls_work_thread() at ktls_work_thread+0xdc/frame 0xfffffe00e4936bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4936bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4936bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100079 td 0xfffffe00e4cffac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49459b0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49459d0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4945a10 _sleep() at _sleep+0x32f/frame 0xfffffe00e4945ab0 ktls_work_thread() at ktls_work_thread+0xdc/frame 0xfffffe00e4945bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4945bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4945bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100080 td 0xfffffe00e4cff3a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49409b0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49409d0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4940a10 _sleep() at _sleep+0x32f/frame 0xfffffe00e4940ab0 ktls_work_thread() at ktls_work_thread+0xdc/frame 0xfffffe00e4940bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4940bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4940bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100081 td 0xfffffe00e4cfec80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e494f9b0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e494f9d0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e494fa10 _sleep() at _sleep+0x32f/frame 0xfffffe00e494fab0 ktls_work_thread() at ktls_work_thread+0xdc/frame 0xfffffe00e494fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e494fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e494fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100082 td 0xfffffe00e4cfe560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e494a9b0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e494a9d0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e494aa10 _sleep() at _sleep+0x32f/frame 0xfffffe00e494aab0 ktls_work_thread() at ktls_work_thread+0xdc/frame 0xfffffe00e494abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e494abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e494abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100083 td 0xfffffe00e4cfde40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49599b0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49599d0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4959a10 _sleep() at _sleep+0x32f/frame 0xfffffe00e4959ab0 ktls_work_thread() at ktls_work_thread+0xdc/frame 0xfffffe00e4959bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4959bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4959bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100084 td 0xfffffe00e4cfd720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49549b0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49549d0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4954a10 _sleep() at _sleep+0x32f/frame 0xfffffe00e4954ab0 ktls_work_thread() at ktls_work_thread+0xdc/frame 0xfffffe00e4954bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4954bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4954bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100085 td 0xfffffe00e4cfd000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49639b0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49639d0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4963a10 _sleep() at _sleep+0x32f/frame 0xfffffe00e4963ab0 ktls_work_thread() at ktls_work_thread+0xdc/frame 0xfffffe00e4963bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4963bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4963bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100086 td 0xfffffe00e4d01740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e495e9b0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e495e9d0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e495ea10 _sleep() at _sleep+0x32f/frame 0xfffffe00e495eab0 ktls_work_thread() at ktls_work_thread+0xdc/frame 0xfffffe00e495ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e495ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e495ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100087 td 0xfffffe00e4d01020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e496d9b0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e496d9d0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e496da10 _sleep() at _sleep+0x32f/frame 0xfffffe00e496dab0 ktls_work_thread() at ktls_work_thread+0xdc/frame 0xfffffe00e496dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e496dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e496dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100088 td 0xfffffe00e4d00900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49689b0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49689d0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4968a10 _sleep() at _sleep+0x32f/frame 0xfffffe00e4968ab0 ktls_work_thread() at ktls_work_thread+0xdc/frame 0xfffffe00e4968bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4968bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4968bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 3 tid 100119 td 0xfffffe00e4d25740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e40a70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e40a90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e40ad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e40b70 crypto_proc() at crypto_proc+0xbf/frame 0xfffffe00e4e40bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e40bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e40bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 0 pid 4 tid 100120 td 0xfffffe00e4d25020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e3ba70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e3ba90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e3bad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e3bb70 crypto_ret_proc() at crypto_ret_proc+0x144/frame 0xfffffe00e4e3bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e3bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e3bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 1 pid 5 tid 100121 td 0xfffffe00e4d24900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e4aa70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e4aa90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e4aad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e4ab70 crypto_ret_proc() at crypto_ret_proc+0x144/frame 0xfffffe00e4e4abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e4abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e4abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 2 pid 6 tid 100122 td 0xfffffe00e4d291e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e45a70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e45a90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e45ad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e45b70 crypto_ret_proc() at crypto_ret_proc+0x144/frame 0xfffffe00e4e45bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e45bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e45bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 3 pid 7 tid 100123 td 0xfffffe00e4d28ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e54a70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e54a90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e54ad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e54b70 crypto_ret_proc() at crypto_ret_proc+0x144/frame 0xfffffe00e4e54bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e54bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e54bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 4 pid 8 tid 100124 td 0xfffffe00e4d283a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e4fa70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e4fa90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e4fad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e4fb70 crypto_ret_proc() at crypto_ret_proc+0x144/frame 0xfffffe00e4e4fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e4fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e4fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 5 pid 9 tid 100125 td 0xfffffe00e4d27c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e5ea70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e5ea90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e5ead0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e5eb70 crypto_ret_proc() at crypto_ret_proc+0x144/frame 0xfffffe00e4e5ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e5ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e5ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command audit pid 10 tid 100001 td 0xfffffe00e4c5c1e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe0037786a90 mi_switch() at mi_switch+0x155/frame 0xfffffe0037786ab0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0037786af0 _cv_wait() at _cv_wait+0x166/frame 0xfffffe0037786b50 audit_worker() at audit_worker+0x9f/frame 0xfffffe0037786bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037786bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037786bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100003 td 0xfffffe00e4c5b3a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe0037790ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe0037790ae0 sched_idletd() at sched_idletd+0x9e/frame 0xfffffe0037790bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037790bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037790bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xfffffe00e4c5ac80 sched_switch() at sched_switch+0x5af/frame 0xfffffe003778bac0 mi_switch() at mi_switch+0x155/frame 0xfffffe003778bae0 sched_idletd() at sched_idletd+0x9e/frame 0xfffffe003778bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003778bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003778bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffffe00e4c5a560 sched_switch() at sched_switch+0x5af/frame 0xfffffe003779aac0 mi_switch() at mi_switch+0x155/frame 0xfffffe003779aae0 sched_idletd() at sched_idletd+0x9e/frame 0xfffffe003779abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003779abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003779abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xfffffe00e4c59e40 sched_switch() at sched_switch+0x5af/frame 0xfffffe0037795ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe0037795ae0 sched_idletd() at sched_idletd+0x9e/frame 0xfffffe0037795bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037795bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037795bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100007 td 0xfffffe00e4c59720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377a4ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377a4ae0 sched_idletd() at sched_idletd+0x9e/frame 0xfffffe00377a4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377a4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377a4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xfffffe00e4c59000 sched_switch() at sched_switch+0x5af/frame 0xfffffe003779fac0 mi_switch() at mi_switch+0x155/frame 0xfffffe003779fae0 sched_idletd() at sched_idletd+0x9e/frame 0xfffffe003779fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003779fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003779fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xfffffe00e4c5d740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377aeac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377aeae0 sched_idletd() at sched_idletd+0x9e/frame 0xfffffe00377aebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377aebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377aebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xfffffe00e4c5d020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377a9ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377a9ae0 sched_idletd() at sched_idletd+0x9e/frame 0xfffffe00377a9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377a9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377a9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffffe00e4c5c900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377b8ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377b8ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00377b8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377b8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377b8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100012 td 0xfffffe00e4c611e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377b3ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377b3ae0 sched_idletd() at sched_idletd+0x9e/frame 0xfffffe00377b3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377b3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377b3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xfffffe00e4c60ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377c2ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377c2ae0 sched_idletd() at sched_idletd+0x9e/frame 0xfffffe00377c2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377c2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377c2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100014 td 0xfffffe00e4c603a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377bdac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377bdae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00377bdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377bdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377bdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100046 td 0xfffffe00e4cebac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e488ab00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e488ab20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e488abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e488abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e488abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100047 td 0xfffffe00e4ceb3a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4899b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4899b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4899bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4899bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4899bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100049 td 0xfffffe00e4cea560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100089 td 0xfffffe00e4d051e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4977b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4977b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4977bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4977bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4977bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100090 td 0xfffffe00e4d04ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4972b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4972b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4972bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4972bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4972bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100091 td 0xfffffe00e4d043a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4981b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4981b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4981bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4981bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4981bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100092 td 0xfffffe00e4d03c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e497cb00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e497cb20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e497cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e497cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e497cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100093 td 0xfffffe00e4d03560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e498bb00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e498bb20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e498bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e498bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e498bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100094 td 0xfffffe00e4d02e40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4986b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4986b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4986bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4986bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4986bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100095 td 0xfffffe00e4d02720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4995b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4995b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4995bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4995bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4995bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100096 td 0xfffffe00e4d02000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4990b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4990b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4990bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4990bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4990bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100097 td 0xfffffe00e4d06740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e499fb00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e499fb20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e499fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e499fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e499fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100098 td 0xfffffe00e4d06020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e499ab00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e499ab20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e499abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e499abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e499abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100099 td 0xfffffe00e4d05900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49a9b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49a9b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e49a9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49a9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49a9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100100 td 0xfffffe00e4d0a1e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49a4b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49a4b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e49a4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49a4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49a4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100101 td 0xfffffe00e4d09ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49b3b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49b3b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e49b3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49b3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49b3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100102 td 0xfffffe00e4d093a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100137 td 0xfffffe00e4d3f740 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100138 td 0xfffffe00e4d3f020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e95b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e95b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4e95bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e95bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e95bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100139 td 0xfffffe00e4d3e900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4ea4b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4ea4b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4ea4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ea4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ea4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100145 td 0xfffffe010a18ae40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4ec2b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4ec2b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4ec2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ec2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ec2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100156 td 0xfffffe010a6d7e40 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100157 td 0xfffffe010a6d7720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f0ab00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f0ab20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4f0abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f0abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f0abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100175 td 0xfffffe010d9f4000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f94b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f94b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4f94bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f94bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f94bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100176 td 0xfffffe010dd301e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100177 td 0xfffffe010dd2fac0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100178 td 0xfffffe010dd2f3a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100179 td 0xfffffe010dd2ec80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100180 td 0xfffffe010dd2e560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100181 td 0xfffffe010dd2de40 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100182 td 0xfffffe010dd2d720 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100183 td 0xfffffe010dd2d000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100184 td 0xfffffe010dd31740 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100185 td 0xfffffe010dd31020 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100186 td 0xfffffe010dd30900 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100187 td 0xfffffe010dd391e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100188 td 0xfffffe010dd38ac0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100189 td 0xfffffe010dd383a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100195 td 0xfffffe010dd3a740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4ff8b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4ff8b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4ff8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ff8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ff8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100196 td 0xfffffe010dd481e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100197 td 0xfffffe010dd47ac0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100198 td 0xfffffe010dd473a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100199 td 0xfffffe010dd46c80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100200 td 0xfffffe010dd46560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100201 td 0xfffffe010dd45e40 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100202 td 0xfffffe010dd45720 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100203 td 0xfffffe010dd45000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100204 td 0xfffffe010dd49740 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100205 td 0xfffffe010dd49020 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100206 td 0xfffffe010dd48900 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100207 td 0xfffffe010df2e1e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100208 td 0xfffffe010df2dac0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100209 td 0xfffffe010df2d3a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100265 td 0xfffffe01437391e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100266 td 0xfffffe0143738ac0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100267 td 0xfffffe01437383a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100268 td 0xfffffe0143737c80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100269 td 0xfffffe0143737560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100270 td 0xfffffe0143736e40 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100271 td 0xfffffe0143736720 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100272 td 0xfffffe0143736000 fork_trampoline() at fork_trampoline Tracing command geom pid 13 tid 100104 td 0xfffffe00e4d08560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49e8a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49e8aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e49e8ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e49e8b80 g_run_events() at g_run_events+0x474/frame 0xfffffe00e49e8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49e8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49e8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100105 td 0xfffffe00e4d07e40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49f7a70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49f7a90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e49f7ad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e49f7b70 g_io_schedule_up() at g_io_schedule_up+0x3c/frame 0xfffffe00e49f7ba0 g_up_procbody() at g_up_procbody+0x5c/frame 0xfffffe00e49f7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49f7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49f7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100106 td 0xfffffe00e4d07720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49f2a70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49f2a90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e49f2ad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e49f2b70 g_io_schedule_down() at g_io_schedule_down+0x4c/frame 0xfffffe00e49f2ba0 g_down_procbody() at g_down_procbody+0x5c/frame 0xfffffe00e49f2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49f2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49f2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 6 pid 14 tid 100126 td 0xfffffe00e4d27560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e59a70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e59a90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e59ad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e59b70 crypto_ret_proc() at crypto_ret_proc+0x144/frame 0xfffffe00e4e59bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e59bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e59bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 7 pid 15 tid 100127 td 0xfffffe00e4d26e40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e68a70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e68a90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e68ad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e68b70 crypto_ret_proc() at crypto_ret_proc+0x144/frame 0xfffffe00e4e68bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e68bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e68bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 8 pid 16 tid 100128 td 0xfffffe00e4d26720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e63a70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e63a90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e63ad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e63b70 crypto_ret_proc() at crypto_ret_proc+0x144/frame 0xfffffe00e4e63bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e63bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e63bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 9 pid 17 tid 100129 td 0xfffffe00e4d26000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e72a70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e72a90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e72ad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e72b70 crypto_ret_proc() at crypto_ret_proc+0x144/frame 0xfffffe00e4e72bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e72bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e72bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 10 pid 18 tid 100130 td 0xfffffe00e4d2a740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e6da70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e6da90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e6dad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e6db70 crypto_ret_proc() at crypto_ret_proc+0x144/frame 0xfffffe00e4e6dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e6dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e6dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 11 pid 19 tid 100131 td 0xfffffe00e4d2a020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e7ca70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e7ca90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e7cad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e7cb70 crypto_ret_proc() at crypto_ret_proc+0x144/frame 0xfffffe00e4e7cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e7cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e7cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sequencer 00 pid 20 tid 100132 td 0xfffffe00e4d29900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e77a70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e77a90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e77ad0 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4e77b30 seq_eventthread() at seq_eventthread+0x8c/frame 0xfffffe00e4e77bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e77bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e77bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 21 tid 100133 td 0xfffffe00e4d3e1e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e86a60 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e86a80 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e86ac0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e86b60 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe00e4e86bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e86bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e86bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 21 tid 100134 td 0xfffffe00e4d3dac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e81a60 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e81a80 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e81ac0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e81b60 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe00e4e81bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e81bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e81bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 21 tid 100135 td 0xfffffe00e4d3d3a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e90a60 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e90a80 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e90ac0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e90b60 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe00e4e90bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e90bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e90bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 21 tid 100136 td 0xfffffe00e4d3cc80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e8ba60 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e8ba80 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e8bac0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e8bb60 xpt_async_td() at xpt_async_td+0x74/frame 0xfffffe00e4e8bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e8bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e8bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 21 tid 100168 td 0xfffffe010d9f63a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f67a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f67aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f67ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4f67b80 xpt_scanner_thread() at xpt_scanner_thread+0x158/frame 0xfffffe00e4f67bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f67bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f67bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100140 td 0xfffffe010a18d1e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e9fac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e9fae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e9fb20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4e9fb80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4e9fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e9fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e9fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100141 td 0xfffffe010a18cac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4eaeac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4eaeae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4eaeb20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4eaeb80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4eaebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4eaebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4eaebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100142 td 0xfffffe010a18c3a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4ea9ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4ea9ae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4ea9b20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4ea9b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4ea9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ea9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ea9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100143 td 0xfffffe010a18bc80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4eb8ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4eb8ae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4eb8b20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4eb8b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4eb8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4eb8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4eb8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100144 td 0xfffffe010a18b560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4eb3ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4eb3ae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4eb3b20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4eb3b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4eb3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4eb3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4eb3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100146 td 0xfffffe010a18a720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4ebdac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4ebdae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4ebdb20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4ebdb80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4ebdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ebdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ebdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100147 td 0xfffffe010a18a000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4eccac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4eccae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4eccb20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4eccb80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4eccbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4eccbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4eccbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100148 td 0xfffffe010a18e740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4ec7ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4ec7ae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4ec7b20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4ec7b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4ec7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ec7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ec7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100149 td 0xfffffe010a18e020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4ed6ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4ed6ae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4ed6b20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4ed6b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4ed6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ed6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ed6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100150 td 0xfffffe010a18d900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4ed1ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4ed1ae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4ed1b20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4ed1b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4ed1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ed1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ed1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100151 td 0xfffffe010a6da1e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4eecac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4eecae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4eecb20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4eecb80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4eecbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4eecbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4eecbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100152 td 0xfffffe010a6d9ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4ee7ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4ee7ae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4ee7b20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4ee7b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4ee7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ee7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ee7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100153 td 0xfffffe010a6d93a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4ef6ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4ef6ae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4ef6b20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4ef6b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4ef6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ef6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ef6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100154 td 0xfffffe010a6d8c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4ef1ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4ef1ae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4ef1b20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4ef1b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4ef1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ef1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ef1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100155 td 0xfffffe010a6d8560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f00ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f00ae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f00b20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4f00b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4f00bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f00bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f00bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod1 pid 23 tid 100159 td 0xfffffe010a6db740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f2ca30 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f2ca50 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f2ca90 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e4f2cad0 _sleep() at _sleep+0x298/frame 0xfffffe00e4f2cb70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe00e4f2cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f2cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f2cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod2 pid 24 tid 100160 td 0xfffffe010a6db020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f27a30 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f27a50 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f27a90 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e4f27ad0 _sleep() at _sleep+0x298/frame 0xfffffe00e4f27b70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe00e4f27bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f27bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f27bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod3 pid 25 tid 100161 td 0xfffffe010a6da900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f36a30 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f36a50 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f36a90 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e4f36ad0 _sleep() at _sleep+0x298/frame 0xfffffe00e4f36b70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe00e4f36bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f36bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f36bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod4 pid 26 tid 100162 td 0xfffffe010d6751e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f31a30 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f31a50 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f31a90 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e4f31ad0 _sleep() at _sleep+0x298/frame 0xfffffe00e4f31b70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe00e4f31bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f31bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f31bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command rand_harvestq pid 27 tid 100163 td 0xfffffe010d676740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f589e0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f58a00 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f58a40 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e4f58a80 _sleep() at _sleep+0x298/frame 0xfffffe00e4f58b20 random_kthread() at random_kthread+0x87/frame 0xfffffe00e4f58bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f58bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f58bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon0 pid 28 tid 100169 td 0xfffffe010d9f8740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f76a40 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f76a60 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f76aa0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4f76b40 enc_daemon() at enc_daemon+0xf6/frame 0xfffffe00e4f76bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f76bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f76bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon1 pid 29 tid 100170 td 0xfffffe010d9f8020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f71a40 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f71a60 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f71aa0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4f71b40 enc_daemon() at enc_daemon+0xf6/frame 0xfffffe00e4f71bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f71bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f71bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 30 tid 100211 td 0xfffffe010df2c560 sched_switch() at sched_switch+0x5af/frame 0xfffffe010e645770 mi_switch() at mi_switch+0x155/frame 0xfffffe010e645790 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe010e6457d0 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe010e645810 _sleep() at _sleep+0x298/frame 0xfffffe010e6458b0 vm_pageout_worker() at vm_pageout_worker+0x29c/frame 0xfffffe010e645b70 vm_pageout() at vm_pageout+0x1d7/frame 0xfffffe010e645bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe010e645bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010e645bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 30 tid 100221 td 0xfffffe010e16c740 sched_switch() at sched_switch+0x5af/frame 0xfffffe01146f2830 mi_switch() at mi_switch+0x155/frame 0xfffffe01146f2850 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01146f2890 _sleep() at _sleep+0x32f/frame 0xfffffe01146f2930 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0xb9c/frame 0xfffffe01146f2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe01146f2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01146f2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 30 tid 100222 td 0xfffffe010e16c020 sched_switch() at sched_switch+0x5af/frame 0xfffffe01145eca40 mi_switch() at mi_switch+0x155/frame 0xfffffe01145eca60 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01145ecaa0 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe01145ecae0 _sleep() at _sleep+0x298/frame 0xfffffe01145ecb80 uma_reclaim_worker() at uma_reclaim_worker+0xc8/frame 0xfffffe01145ecbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe01145ecbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01145ecbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command vmdaemon pid 31 tid 100212 td 0xfffffe010df2be40 sched_switch() at sched_switch+0x5af/frame 0xfffffe010e660a40 mi_switch() at mi_switch+0x155/frame 0xfffffe010e660a60 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe010e660aa0 _sleep() at _sleep+0x32f/frame 0xfffffe010e660b40 vm_daemon() at vm_daemon+0x82/frame 0xfffffe010e660bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe010e660bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010e660bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100213 td 0xfffffe010df2b720 sched_switch() at sched_switch+0x5af/frame 0xfffffe010e64fa20 mi_switch() at mi_switch+0x155/frame 0xfffffe010e64fa40 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe010e64fa80 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe010e64fac0 _sleep() at _sleep+0x298/frame 0xfffffe010e64fb60 buf_daemon() at buf_daemon+0x116/frame 0xfffffe010e64fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe010e64fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010e64fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100214 td 0xfffffe010df2f740 sched_switch() at sched_switch+0x5af/frame 0xfffffe010eb6fa10 mi_switch() at mi_switch+0x155/frame 0xfffffe010eb6fa30 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe010eb6fa70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe010eb6fab0 _sleep() at _sleep+0x298/frame 0xfffffe010eb6fb50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe010eb6fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe010eb6fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010eb6fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100215 td 0xfffffe010df2f020 sched_switch() at sched_switch+0x5af/frame 0xfffffe010e968a10 mi_switch() at mi_switch+0x155/frame 0xfffffe010e968a30 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe010e968a70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe010e968ab0 _sleep() at _sleep+0x298/frame 0xfffffe010e968b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe010e968bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe010e968bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010e968bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100216 td 0xfffffe010df2e900 sched_switch() at sched_switch+0x5af/frame 0xfffffe010f280a10 mi_switch() at mi_switch+0x155/frame 0xfffffe010f280a30 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe010f280a70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe010f280ab0 _sleep() at _sleep+0x298/frame 0xfffffe010f280b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe010f280bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe010f280bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010f280bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100217 td 0xfffffe010e16b1e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe010ef78a10 mi_switch() at mi_switch+0x155/frame 0xfffffe010ef78a30 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe010ef78a70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe010ef78ab0 _sleep() at _sleep+0x298/frame 0xfffffe010ef78b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe010ef78bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe010ef78bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010ef78bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100218 td 0xfffffe010e16aac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe011059da10 mi_switch() at mi_switch+0x155/frame 0xfffffe011059da30 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe011059da70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe011059dab0 _sleep() at _sleep+0x298/frame 0xfffffe011059db50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe011059dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe011059dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe011059dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100219 td 0xfffffe010e16a3a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe0110295a10 mi_switch() at mi_switch+0x155/frame 0xfffffe0110295a30 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0110295a70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0110295ab0 _sleep() at _sleep+0x298/frame 0xfffffe0110295b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe0110295bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0110295bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0110295bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100220 td 0xfffffe010e169c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe0110bada10 mi_switch() at mi_switch+0x155/frame 0xfffffe0110bada30 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0110bada70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0110badab0 _sleep() at _sleep+0x298/frame 0xfffffe0110badb50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe0110badbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0110badbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0110badbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100236 td 0xfffffe00e4d3be40 sched_switch() at sched_switch+0x5af/frame 0xfffffe01415f7a30 mi_switch() at mi_switch+0x155/frame 0xfffffe01415f7a50 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01415f7a90 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe01415f7ad0 _sleep() at _sleep+0x298/frame 0xfffffe01415f7b70 softdep_flush() at softdep_flush+0x188/frame 0xfffffe01415f7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe01415f7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01415f7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100243 td 0xfffffe010d9f7900 sched_switch() at sched_switch+0x5af/frame 0xfffffe0141664a30 mi_switch() at mi_switch+0x155/frame 0xfffffe0141664a50 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0141664a90 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0141664ad0 _sleep() at _sleep+0x298/frame 0xfffffe0141664b70 softdep_flush() at softdep_flush+0x188/frame 0xfffffe0141664bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0141664bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0141664bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 126093 td 0xfffffe019a6be3a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe018c5d7a30 mi_switch() at mi_switch+0x155/frame 0xfffffe018c5d7a50 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe018c5d7a90 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe018c5d7ad0 _sleep() at _sleep+0x298/frame 0xfffffe018c5d7b70 softdep_flush() at softdep_flush+0x188/frame 0xfffffe018c5d7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe018c5d7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe018c5d7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command vnlru pid 33 tid 100223 td 0xfffffe010df2b000 sched_switch() at sched_switch+0x5af/frame 0xfffffe01415c89d0 mi_switch() at mi_switch+0x155/frame 0xfffffe01415c89f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01415c8a30 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe01415c8a70 _sleep() at _sleep+0x298/frame 0xfffffe01415c8b10 vnlru_proc() at vnlru_proc+0xa2c/frame 0xfffffe01415c8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe01415c8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01415c8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command syncer pid 34 tid 100224 td 0xfffffe010e3c21e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe01415c3a00 mi_switch() at mi_switch+0x155/frame 0xfffffe01415c3a20 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01415c3a60 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe01415c3aa0 _cv_timedwait_sbt() at _cv_timedwait_sbt+0x19b/frame 0xfffffe01415c3b20 sched_sync() at sched_sync+0x63d/frame 0xfffffe01415c3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe01415c3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01415c3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command dhclient pid 2339 tid 100257 td 0xfffffe010e5a2020 sched_switch() at sched_switch+0x5af/frame 0xfffffe014160b720 mi_switch() at mi_switch+0x155/frame 0xfffffe014160b740 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe014160b780 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe014160b7d0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe014160b7e0 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe014160b840 seltdwait() at seltdwait+0x8d/frame 0xfffffe014160b880 kern_select() at kern_select+0xa4e/frame 0xfffffe014160ba80 sys_select() at sys_select+0x56/frame 0xfffffe014160bac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014160bbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014160bbf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8011e525a, rsp = 0x7fffffffe938, rbp = 0x7fffffffe9f0 --- Tracing command dhclient pid 2342 tid 100235 td 0xfffffe010e5b41e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe0141624760 mi_switch() at mi_switch+0x155/frame 0xfffffe0141624780 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01416247c0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0141624810 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0141624820 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe0141624880 seltdwait() at seltdwait+0x8d/frame 0xfffffe01416248c0 kern_poll_kfds() at kern_poll_kfds+0x41b/frame 0xfffffe0141624950 kern_poll() at kern_poll+0x9f/frame 0xfffffe0141624aa0 sys_poll() at sys_poll+0x50/frame 0xfffffe0141624ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0141624bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0141624bf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8011e515a, rsp = 0x7fffffffeb78, rbp = 0x7fffffffebb0 --- Tracing command dhclient pid 2418 tid 100283 td 0xfffffe0143e12c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe014175c710 mi_switch() at mi_switch+0x155/frame 0xfffffe014175c730 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe014175c770 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe014175c7c0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe014175c800 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19b/frame 0xfffffe014175c880 seltdwait() at seltdwait+0x7f/frame 0xfffffe014175c8c0 kern_poll_kfds() at kern_poll_kfds+0x41b/frame 0xfffffe014175c950 kern_poll() at kern_poll+0x9f/frame 0xfffffe014175caa0 sys_poll() at sys_poll+0x50/frame 0xfffffe014175cac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014175cbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014175cbf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8011e515a, rsp = 0x7fffffffeb78, rbp = 0x7fffffffebb0 --- Tracing command devd pid 2423 tid 100262 td 0xfffffe010e5b2c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe01416bb6d0 mi_switch() at mi_switch+0x155/frame 0xfffffe01416bb6f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01416bb730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01416bb780 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe01416bb7c0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19b/frame 0xfffffe01416bb840 seltdwait() at seltdwait+0x7f/frame 0xfffffe01416bb880 kern_select() at kern_select+0xa4e/frame 0xfffffe01416bba80 sys_select() at sys_select+0x56/frame 0xfffffe01416bbac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01416bbbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01416bbbf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x28c62a, rsp = 0x7fffffffcab8, rbp = 0x7fffffffec50 --- Tracing command syslogd pid 2610 tid 100256 td 0xfffffe0143603740 sched_switch() at sched_switch+0x5af/frame 0xfffffe014169d720 mi_switch() at mi_switch+0x155/frame 0xfffffe014169d740 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe014169d780 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe014169d7d0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe014169d7e0 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe014169d840 seltdwait() at seltdwait+0x8d/frame 0xfffffe014169d880 kern_select() at kern_select+0xa4e/frame 0xfffffe014169da80 sys_select() at sys_select+0x56/frame 0xfffffe014169dac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014169dbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014169dbf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8011ca25a, rsp = 0x7fffffffe6b8, rbp = 0x7fffffffec20 --- Tracing command rpcbind pid 2640 tid 100286 td 0xfffffe0143e11720 sched_switch() at sched_switch+0x5af/frame 0xfffffe0141761710 mi_switch() at mi_switch+0x155/frame 0xfffffe0141761730 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0141761770 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01417617c0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe0141761800 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19b/frame 0xfffffe0141761880 seltdwait() at seltdwait+0x7f/frame 0xfffffe01417618c0 kern_poll_kfds() at kern_poll_kfds+0x41b/frame 0xfffffe0141761950 kern_poll() at kern_poll+0x9f/frame 0xfffffe0141761aa0 sys_poll() at sys_poll+0x50/frame 0xfffffe0141761ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0141761bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0141761bf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8011bb15a, rsp = 0x7fffffffc978, rbp = 0x7fffffffeba0 --- Tracing command rpc.statd pid 2665 tid 100290 td 0xfffffe0143e15020 sched_switch() at sched_switch+0x5af/frame 0xfffffe014174d6d0 mi_switch() at mi_switch+0x155/frame 0xfffffe014174d6f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe014174d730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe014174d780 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe014174d7c0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19b/frame 0xfffffe014174d840 seltdwait() at seltdwait+0x7f/frame 0xfffffe014174d880 kern_select() at kern_select+0xa4e/frame 0xfffffe014174da80 sys_select() at sys_select+0x56/frame 0xfffffe014174dac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014174dbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014174dbf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8011b525a, rsp = 0x7fffffffe9f8, rbp = 0x7fffffffeb40 --- Tracing command rpc.lockd pid 2668 tid 100229 td 0xfffffe010dd3a020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4ff3640 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4ff3660 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4ff36a0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe00e4ff36f0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe00e4ff3730 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19b/frame 0xfffffe00e4ff37b0 svc_run_internal() at svc_run_internal+0xac1/frame 0xfffffe00e4ff38e0 svc_run() at svc_run+0x250/frame 0xfffffe00e4ff3940 sys_nlm_syscall() at sys_nlm_syscall+0x74a/frame 0xfffffe00e4ff3ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe00e4ff3bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00e4ff3bf0 --- syscall (154, FreeBSD ELF64, sys_nlm_syscall), rip = 0x8011bda3a, rsp = 0x7fffffffeb58, rbp = 0x7fffffffec20 --- Tracing command powerd pid 2673 tid 100259 td 0xfffffe010e5b4900 sched_switch() at sched_switch+0x5af/frame 0xfffffe01416a76d0 mi_switch() at mi_switch+0x155/frame 0xfffffe01416a76f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01416a7730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01416a7780 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe01416a77c0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19b/frame 0xfffffe01416a7840 seltdwait() at seltdwait+0x7f/frame 0xfffffe01416a7880 kern_select() at kern_select+0xa4e/frame 0xfffffe01416a7a80 sys_select() at sys_select+0x56/frame 0xfffffe01416a7ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01416a7bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01416a7bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8011c025a, rsp = 0x7fffffffeaf8, rbp = 0x7fffffffec30 --- Tracing command ntpd pid 2699 tid 100248 td 0xfffffe010e5a03a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe014168e720 mi_switch() at mi_switch+0x155/frame 0xfffffe014168e740 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe014168e780 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe014168e7d0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe014168e7e0 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe014168e840 seltdwait() at seltdwait+0x8d/frame 0xfffffe014168e880 kern_select() at kern_select+0xa4e/frame 0xfffffe014168ea80 sys_select() at sys_select+0x56/frame 0xfffffe014168eac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014168ebf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014168ebf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8015da25a, rsp = 0x7fffffffde58, rbp = 0x7fffffffde90 --- Tracing command inetd pid 2725 tid 100255 td 0xfffffe010e3bf000 sched_switch() at sched_switch+0x5af/frame 0xfffffe01416a2720 mi_switch() at mi_switch+0x155/frame 0xfffffe01416a2740 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01416a2780 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01416a27d0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01416a27e0 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01416a2840 seltdwait() at seltdwait+0x8d/frame 0xfffffe01416a2880 kern_select() at kern_select+0xa4e/frame 0xfffffe01416a2a80 sys_select() at sys_select+0x56/frame 0xfffffe01416a2ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01416a2bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01416a2bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8011df25a, rsp = 0x7fffffffdd78, rbp = 0x7fffffffec10 --- Tracing command sshd pid 2744 tid 100264 td 0xfffffe01436013a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe01416c5720 mi_switch() at mi_switch+0x155/frame 0xfffffe01416c5740 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01416c5780 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01416c57d0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01416c57e0 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01416c5840 seltdwait() at seltdwait+0x8d/frame 0xfffffe01416c5880 kern_select() at kern_select+0xa4e/frame 0xfffffe01416c5a80 sys_select() at sys_select+0x56/frame 0xfffffe01416c5ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01416c5bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01416c5bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80169825a, rsp = 0x7fffffffe5c8, rbp = 0x7fffffffe600 --- Tracing command cron pid 2748 tid 100226 td 0xfffffe010e16b900 sched_switch() at sched_switch+0x5af/frame 0xfffffe01415e8870 mi_switch() at mi_switch+0x155/frame 0xfffffe01415e8890 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01415e88d0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01415e8920 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe01415e8960 _sleep() at _sleep+0x285/frame 0xfffffe01415e8a00 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e1/frame 0xfffffe01415e8a80 sys_nanosleep() at sys_nanosleep+0x3b/frame 0xfffffe01415e8ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01415e8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01415e8bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8011d70fa, rsp = 0x7fffffffea48, rbp = 0x7fffffffec20 --- Tracing command getty pid 2762 tid 100225 td 0xfffffe010e3c3740 sched_switch() at sched_switch+0x5af/frame 0xfffffe01415de7c0 mi_switch() at mi_switch+0x155/frame 0xfffffe01415de7e0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01415de820 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01415de870 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01415de880 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01415de8e0 tty_wait() at tty_wait+0x42/frame 0xfffffe01415de910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe01415de970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01415de9a0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe01415dea00 dofileread() at dofileread+0x81/frame 0xfffffe01415dea50 sys_read() at sys_read+0xc0/frame 0xfffffe01415deac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01415debf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01415debf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c41da, rsp = 0x7fffffffeb58, rbp = 0x7fffffffeba0 --- Tracing command getty pid 2763 tid 100239 td 0xfffffe010e5b5740 sched_switch() at sched_switch+0x5af/frame 0xfffffe01416297c0 mi_switch() at mi_switch+0x155/frame 0xfffffe01416297e0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0141629820 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0141629870 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0141629880 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01416298e0 tty_wait() at tty_wait+0x42/frame 0xfffffe0141629910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe0141629970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01416299a0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe0141629a00 dofileread() at dofileread+0x81/frame 0xfffffe0141629a50 sys_read() at sys_read+0xc0/frame 0xfffffe0141629ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0141629bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0141629bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c41da, rsp = 0x7fffffffeb58, rbp = 0x7fffffffeba0 --- Tracing command getty pid 2764 tid 100237 td 0xfffffe010e3c0560 sched_switch() at sched_switch+0x5af/frame 0xfffffe01416157c0 mi_switch() at mi_switch+0x155/frame 0xfffffe01416157e0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0141615820 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0141615870 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0141615880 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01416158e0 tty_wait() at tty_wait+0x42/frame 0xfffffe0141615910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe0141615970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01416159a0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe0141615a00 dofileread() at dofileread+0x81/frame 0xfffffe0141615a50 sys_read() at sys_read+0xc0/frame 0xfffffe0141615ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0141615bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0141615bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c41da, rsp = 0x7fffffffeb58, rbp = 0x7fffffffeba0 --- Tracing command getty pid 2765 tid 100231 td 0xfffffe010e5a11e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe01416067c0 mi_switch() at mi_switch+0x155/frame 0xfffffe01416067e0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0141606820 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0141606870 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0141606880 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01416068e0 tty_wait() at tty_wait+0x42/frame 0xfffffe0141606910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe0141606970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01416069a0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe0141606a00 dofileread() at dofileread+0x81/frame 0xfffffe0141606a50 sys_read() at sys_read+0xc0/frame 0xfffffe0141606ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0141606bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0141606bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c41da, rsp = 0x7fffffffeb58, rbp = 0x7fffffffeba0 --- Tracing command getty pid 2766 tid 100230 td 0xfffffe00e4d3c560 sched_switch() at sched_switch+0x5af/frame 0xfffffe01415fc7c0 mi_switch() at mi_switch+0x155/frame 0xfffffe01415fc7e0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01415fc820 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01415fc870 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01415fc880 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01415fc8e0 tty_wait() at tty_wait+0x42/frame 0xfffffe01415fc910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe01415fc970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01415fc9a0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe01415fca00 dofileread() at dofileread+0x81/frame 0xfffffe01415fca50 sys_read() at sys_read+0xc0/frame 0xfffffe01415fcac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01415fcbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01415fcbf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c41da, rsp = 0x7fffffffeb58, rbp = 0x7fffffffeba0 --- Tracing command getty pid 2767 tid 100297 td 0xfffffe00e4d3b720 sched_switch() at sched_switch+0x5af/frame 0xfffffe01417a27c0 mi_switch() at mi_switch+0x155/frame 0xfffffe01417a27e0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01417a2820 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01417a2870 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01417a2880 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01417a28e0 tty_wait() at tty_wait+0x42/frame 0xfffffe01417a2910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe01417a2970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01417a29a0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe01417a2a00 dofileread() at dofileread+0x81/frame 0xfffffe01417a2a50 sys_read() at sys_read+0xc0/frame 0xfffffe01417a2ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01417a2bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01417a2bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c41da, rsp = 0x7fffffffeb58, rbp = 0x7fffffffeba0 --- Tracing command getty pid 2768 tid 100298 td 0xfffffe00e4d3b000 sched_switch() at sched_switch+0x5af/frame 0xfffffe014179d7c0 mi_switch() at mi_switch+0x155/frame 0xfffffe014179d7e0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe014179d820 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe014179d870 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe014179d880 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe014179d8e0 tty_wait() at tty_wait+0x42/frame 0xfffffe014179d910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe014179d970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe014179d9a0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe014179da00 dofileread() at dofileread+0x81/frame 0xfffffe014179da50 sys_read() at sys_read+0xc0/frame 0xfffffe014179dac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014179dbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014179dbf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c41da, rsp = 0x7fffffffeb58, rbp = 0x7fffffffeba0 --- Tracing command getty pid 2769 tid 100299 td 0xfffffe01440251e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe01417ac7c0 mi_switch() at mi_switch+0x155/frame 0xfffffe01417ac7e0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01417ac820 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01417ac870 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01417ac880 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01417ac8e0 tty_wait() at tty_wait+0x42/frame 0xfffffe01417ac910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe01417ac970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01417ac9a0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe01417aca00 dofileread() at dofileread+0x81/frame 0xfffffe01417aca50 sys_read() at sys_read+0xc0/frame 0xfffffe01417acac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01417acbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01417acbf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c41da, rsp = 0x7fffffffeb58, rbp = 0x7fffffffeba0 --- Tracing command getty pid 2770 tid 100300 td 0xfffffe0144024ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe01417a77c0 mi_switch() at mi_switch+0x155/frame 0xfffffe01417a77e0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01417a7820 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01417a7870 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01417a7880 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01417a78e0 tty_wait() at tty_wait+0x42/frame 0xfffffe01417a7910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe01417a7970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01417a79a0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe01417a7a00 dofileread() at dofileread+0x81/frame 0xfffffe01417a7a50 sys_read() at sys_read+0xc0/frame 0xfffffe01417a7ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01417a7bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01417a7bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c41da, rsp = 0x7fffffffeb58, rbp = 0x7fffffffeba0 --- Tracing command getty pid 2771 tid 100301 td 0xfffffe01440243a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe01417b67c0 mi_switch() at mi_switch+0x155/frame 0xfffffe01417b67e0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01417b6820 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01417b6870 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01417b6880 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01417b68e0 tty_wait() at tty_wait+0x42/frame 0xfffffe01417b6910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe01417b6970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01417b69a0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe01417b6a00 dofileread() at dofileread+0x81/frame 0xfffffe01417b6a50 sys_read() at sys_read+0xc0/frame 0xfffffe01417b6ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01417b6bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01417b6bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c41da, rsp = 0x7fffffffeb58, rbp = 0x7fffffffeba0 --- Tracing command sshd pid 2776 tid 100242 td 0xfffffe010e3c3020 sched_switch() at sched_switch+0x5af/frame 0xfffffe01415d9760 mi_switch() at mi_switch+0x155/frame 0xfffffe01415d9780 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01415d97c0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01415d9810 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01415d9820 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01415d9880 seltdwait() at seltdwait+0x8d/frame 0xfffffe01415d98c0 kern_poll_kfds() at kern_poll_kfds+0x41b/frame 0xfffffe01415d9950 kern_poll() at kern_poll+0x9f/frame 0xfffffe01415d9aa0 sys_poll() at sys_poll+0x50/frame 0xfffffe01415d9ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01415d9bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01415d9bf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x80169815a, rsp = 0x7fffffffe538, rbp = 0x7fffffffe570 --- Tracing command sshd pid 2779 tid 100260 td 0xfffffe01435abc80 sched_switch() at sched_switch+0x5af/frame 0xfffffe01416b6720 mi_switch() at mi_switch+0x155/frame 0xfffffe01416b6740 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01416b6780 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01416b67d0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01416b67e0 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01416b6840 seltdwait() at seltdwait+0x8d/frame 0xfffffe01416b6880 kern_select() at kern_select+0xa4e/frame 0xfffffe01416b6a80 sys_select() at sys_select+0x56/frame 0xfffffe01416b6ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01416b6bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01416b6bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80169825a, rsp = 0x7fffffffa4c8, rbp = 0x7fffffffa500 --- Tracing command bash pid 2780 tid 100294 td 0xfffffe0143603020 --- swapped out Tracing command su pid 2785 tid 100276 td 0xfffffe0143e141e0 --- swapped out Tracing command csh pid 2786 tid 100292 td 0xfffffe01435aa000 --- swapped out Tracing command bash pid 2788 tid 100280 td 0xfffffe01435aae40 --- swapped out Tracing command sh pid 3116 tid 100310 td 0xfffffe010d673c80 --- swapped out Tracing command sh pid 3157 tid 100311 td 0xfffffe010d673560 sched_switch() at sched_switch+0x5af/frame 0xfffffe01417e36d0 mi_switch() at mi_switch+0x155/frame 0xfffffe01417e36f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01417e3730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01417e3780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01417e3790 _sleep() at _sleep+0x325/frame 0xfffffe01417e3830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe01417e38c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe01417e3ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01417e3bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01417e3bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8012083ba, rsp = 0x7fffffffe228, rbp = 0x7fffffffe2d0 --- Tracing command awk pid 3158 tid 100312 td 0xfffffe010d672e40 sched_switch() at sched_switch+0x5af/frame 0xfffffe01417f2840 mi_switch() at mi_switch+0x155/frame 0xfffffe01417f2860 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01417f28a0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01417f28f0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01417f2900 _sleep() at _sleep+0x325/frame 0xfffffe01417f29a0 pipe_read() at pipe_read+0x4c6/frame 0xfffffe01417f2a00 dofileread() at dofileread+0x81/frame 0xfffffe01417f2a50 sys_read() at sys_read+0xc0/frame 0xfffffe01417f2ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01417f2bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01417f2bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8012151da, rsp = 0x7fffffffe1f8, rbp = 0x7fffffffe210 --- Tracing command sh pid 3160 tid 100313 td 0xfffffe010d672720 sched_switch() at sched_switch+0x5af/frame 0xfffffe01417ed840 mi_switch() at mi_switch+0x155/frame 0xfffffe01417ed860 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01417ed8a0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01417ed8f0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01417ed900 _sleep() at _sleep+0x325/frame 0xfffffe01417ed9a0 pipe_read() at pipe_read+0x4c6/frame 0xfffffe01417eda00 dofileread() at dofileread+0x81/frame 0xfffffe01417eda50 sys_read() at sys_read+0xc0/frame 0xfffffe01417edac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01417edbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01417edbf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8012081da, rsp = 0x7fffffffdcc8, rbp = 0x7fffffffe1f0 --- Tracing command sh pid 36990 tid 136340 td 0xfffffe014472b720 sched_switch() at sched_switch+0x5af/frame 0xfffffe0194f436d0 mi_switch() at mi_switch+0x155/frame 0xfffffe0194f436f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0194f43730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0194f43780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0194f43790 _sleep() at _sleep+0x325/frame 0xfffffe0194f43830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe0194f438c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0194f43ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0194f43bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0194f43bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8012083ba, rsp = 0x7fffffffe198, rbp = 0x7fffffffe240 --- Tracing command tee pid 36991 tid 132157 td 0xfffffe017ad49000 sched_switch() at sched_switch+0x5af/frame 0xfffffe018fd90840 mi_switch() at mi_switch+0x155/frame 0xfffffe018fd90860 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe018fd908a0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe018fd908f0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe018fd90900 _sleep() at _sleep+0x325/frame 0xfffffe018fd909a0 pipe_read() at pipe_read+0x4c6/frame 0xfffffe018fd90a00 dofileread() at dofileread+0x81/frame 0xfffffe018fd90a50 sys_read() at sys_read+0xc0/frame 0xfffffe018fd90ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018fd90bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018fd90bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011a61da, rsp = 0x7fffffffe5c8, rbp = 0x7fffffffe620 --- Tracing command sh pid 36992 tid 132895 td 0xfffffe010cebfac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe0190bfa6d0 mi_switch() at mi_switch+0x155/frame 0xfffffe0190bfa6f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0190bfa730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0190bfa780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0190bfa790 _sleep() at _sleep+0x325/frame 0xfffffe0190bfa830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe0190bfa8c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0190bfaac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0190bfabf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0190bfabf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8012083ba, rsp = 0x7fffffffe238, rbp = 0x7fffffffe2e0 --- Tracing command su pid 37011 tid 136858 td 0xfffffe0144a0a720 sched_switch() at sched_switch+0x5af/frame 0xfffffe01959616d0 mi_switch() at mi_switch+0x155/frame 0xfffffe01959616f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0195961730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0195961780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0195961790 _sleep() at _sleep+0x325/frame 0xfffffe0195961830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe01959618c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0195961ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0195961bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0195961bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8011ee3ba, rsp = 0x7fffffffdfa8, rbp = 0x7fffffffe500 --- Tracing command run pid 37014 tid 129732 td 0xfffffe017c411020 sched_switch() at sched_switch+0x5af/frame 0xfffffe018c0e6870 mi_switch() at mi_switch+0x155/frame 0xfffffe018c0e6890 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe018c0e68d0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe018c0e6920 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe018c0e6960 _sleep() at _sleep+0x285/frame 0xfffffe018c0e6a00 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e1/frame 0xfffffe018c0e6a80 sys_nanosleep() at sys_nanosleep+0x3b/frame 0xfffffe018c0e6ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018c0e6bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018c0e6bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8011a80fa, rsp = 0x7fffffffe188, rbp = 0x7fffffffe1d0 --- Tracing command run pid 37015 tid 131214 td 0xfffffe0166582e40 sched_switch() at sched_switch+0x5af/frame 0xfffffe018eb256d0 mi_switch() at mi_switch+0x155/frame 0xfffffe018eb256f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe018eb25730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe018eb25780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe018eb25790 _sleep() at _sleep+0x325/frame 0xfffffe018eb25830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe018eb258c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe018eb25ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018eb25bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018eb25bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8011a83ba, rsp = 0x7fffffffe198, rbp = 0x7fffffffe1d0 --- Tracing command run pid 37016 tid 134847 td 0xfffffe010c7671e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe019321a6d0 mi_switch() at mi_switch+0x155/frame 0xfffffe019321a6f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe019321a730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe019321a780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe019321a790 _sleep() at _sleep+0x325/frame 0xfffffe019321a830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe019321a8c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe019321aac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe019321abf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe019321abf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8011a83ba, rsp = 0x7fffffffdc28, rbp = 0x7fffffffe160 --- Tracing command lockf2 pid 37017 tid 129324 td 0xfffffe014cd9ee40 sched_switch() at sched_switch+0x5af/frame 0xfffffe018b8ee870 mi_switch() at mi_switch+0x155/frame 0xfffffe018b8ee890 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe018b8ee8d0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe018b8ee920 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe018b8ee960 _sleep() at _sleep+0x285/frame 0xfffffe018b8eea00 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e1/frame 0xfffffe018b8eea80 sys_nanosleep() at sys_nanosleep+0x3b/frame 0xfffffe018b8eeac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018b8eebf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018b8eebf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8011a90fa, rsp = 0x7fffffffe388, rbp = 0x7fffffffe3d0 --- Tracing command symlink pid 37018 tid 136274 td 0xfffffe01446d5020 thread_exit() at thread_exit+0x203/frame 0xfffffe0194df9a40 exit1() at exit1+0xdde/frame 0xfffffe0194df9ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe0194df9ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0194df9bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0194df9bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8011a8e3a, rsp = 0x7fffffffe4e8, rbp = 0x7fffffffe500 --- Tracing command openat pid 37019 tid 130953 td 0xfffffe015ce1ee40 sched_switch() at sched_switch+0x5af/frame 0xfffffe018e60c870 mi_switch() at mi_switch+0x155/frame 0xfffffe018e60c890 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe018e60c8d0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe018e60c920 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe018e60c960 _sleep() at _sleep+0x285/frame 0xfffffe018e60ca00 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e1/frame 0xfffffe018e60ca80 sys_nanosleep() at sys_nanosleep+0x3b/frame 0xfffffe018e60cac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018e60cbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018e60cbf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8011a90fa, rsp = 0x7fffffffe388, rbp = 0x7fffffffe3d0 --- Tracing command socket pid 37020 tid 129819 td 0xfffffe01767f1020 thread_exit() at thread_exit+0x203/frame 0xfffffe018c299a40 exit1() at exit1+0xdde/frame 0xfffffe018c299ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe018c299ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018c299bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018c299bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8011a7e3a, rsp = 0x7fffffffe4e8, rbp = 0x7fffffffe500 --- Tracing command rw pid 37021 tid 129810 td 0xfffffe0174c78c80 thread_exit() at thread_exit+0x203/frame 0xfffffe018c26ca40 exit1() at exit1+0xdde/frame 0xfffffe018c26cab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe018c26cac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018c26cbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018c26cbf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8011a8e3a, rsp = 0x7fffffffe4e8, rbp = 0x7fffffffe500 --- Tracing command mmap pid 37022 tid 1075875 td 0xfffffe01733e61e0 thread_exit() at thread_exit+0x203/frame 0xfffffe019c840a40 exit1() at exit1+0xdde/frame 0xfffffe019c840ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe019c840ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe019c840bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe019c840bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8011a8e3a, rsp = 0x7fffffffe4e8, rbp = 0x7fffffffe500 --- Tracing command fts pid 37023 tid 100325 td 0xfffffe014428a740 sched_switch() at sched_switch+0x5af/frame 0xfffffe0144431870 mi_switch() at mi_switch+0x155/frame 0xfffffe0144431890 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01444318d0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0144431920 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe0144431960 _sleep() at _sleep+0x285/frame 0xfffffe0144431a00 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e1/frame 0xfffffe0144431a80 sys_nanosleep() at sys_nanosleep+0x3b/frame 0xfffffe0144431ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0144431bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0144431bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8011a80fa, rsp = 0x7fffffffe388, rbp = 0x7fffffffe3d0 --- Tracing command link pid 37024 tid 127766 td 0xfffffe0149e34720 sched_switch() at sched_switch+0x5af/frame 0xfffffe018b2fe870 mi_switch() at mi_switch+0x155/frame 0xfffffe018b2fe890 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe018b2fe8d0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe018b2fe920 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe018b2fe960 _sleep() at _sleep+0x285/frame 0xfffffe018b2fea00 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e1/frame 0xfffffe018b2fea80 sys_nanosleep() at sys_nanosleep+0x3b/frame 0xfffffe018b2feac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018b2febf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018b2febf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8011a90fa, rsp = 0x7fffffffe388, rbp = 0x7fffffffe3d0 --- Tracing command lockf pid 37025 tid 132408 td 0xfffffe017aec0c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe0190277870 mi_switch() at mi_switch+0x155/frame 0xfffffe0190277890 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01902778d0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0190277920 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe0190277960 _sleep() at _sleep+0x285/frame 0xfffffe0190277a00 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e1/frame 0xfffffe0190277a80 sys_nanosleep() at sys_nanosleep+0x3b/frame 0xfffffe0190277ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0190277bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0190277bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8011a90fa, rsp = 0x7fffffffe388, rbp = 0x7fffffffe3d0 --- Tracing command creat pid 37026 tid 133468 td 0xfffffe010d1981e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe019172b870 mi_switch() at mi_switch+0x155/frame 0xfffffe019172b890 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe019172b8d0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe019172b920 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe019172b960 _sleep() at _sleep+0x285/frame 0xfffffe019172ba00 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e1/frame 0xfffffe019172ba80 sys_nanosleep() at sys_nanosleep+0x3b/frame 0xfffffe019172bac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe019172bbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe019172bbf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8011a90fa, rsp = 0x7fffffffe388, rbp = 0x7fffffffe3d0 --- Tracing command mkdir pid 37027 tid 132088 td 0xfffffe017ad15560 thread_exit() at thread_exit+0x203/frame 0xfffffe018fc37a40 exit1() at exit1+0xdde/frame 0xfffffe018fc37ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe018fc37ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018fc37bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018fc37bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8011a8e3a, rsp = 0x7fffffffe4e8, rbp = 0x7fffffffe500 --- Tracing command lockf2 pid 37028 tid 127098 td 0xfffffe01691f0c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe018c41a6d0 mi_switch() at mi_switch+0x155/frame 0xfffffe018c41a6f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe018c41a730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe018c41a780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe018c41a790 _sleep() at _sleep+0x325/frame 0xfffffe018c41a830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe018c41a8c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe018c41aac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018c41abf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018c41abf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8011a93ba, rsp = 0x7fffffffe398, rbp = 0x7fffffffe3d0 --- Tracing command rename pid 37029 tid 130578 td 0xfffffe014d33dc80 thread_exit() at thread_exit+0x203/frame 0xfffffe018deb9a40 exit1() at exit1+0xdde/frame 0xfffffe018deb9ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe018deb9ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018deb9bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018deb9bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8011a8e3a, rsp = 0x7fffffffe4e8, rbp = 0x7fffffffe500 --- Tracing command swap pid 37030 tid 137400 td 0xfffffe0144cfe720 sched_switch() at sched_switch+0x5af/frame 0xfffffe01963f7870 mi_switch() at mi_switch+0x155/frame 0xfffffe01963f7890 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01963f78d0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01963f7920 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe01963f7960 _sleep() at _sleep+0x285/frame 0xfffffe01963f7a00 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e1/frame 0xfffffe01963f7a80 sys_nanosleep() at sys_nanosleep+0x3b/frame 0xfffffe01963f7ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01963f7bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01963f7bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8011a90fa, rsp = 0x7fffffffe388, rbp = 0x7fffffffe3d0 --- Tracing command lockf2 pid 37031 tid 129303 td 0xfffffe016d0223a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe018b885b90 mi_switch() at mi_switch+0x155/frame 0xfffffe018b885bb0 ast() at ast+0x289/frame 0xfffffe018b885bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe330 Tracing command openat pid 37032 tid 100629 td 0xfffffe01606e6ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe01596186d0 mi_switch() at mi_switch+0x155/frame 0xfffffe01596186f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0159618730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0159618780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0159618790 _sleep() at _sleep+0x325/frame 0xfffffe0159618830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe01596188c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0159618ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0159618bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0159618bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8011a93ba, rsp = 0x7fffffffe398, rbp = 0x7fffffffe3d0 --- Tracing command mkfifo pid 37033 tid 132440 td 0xfffffe017aed3560 thread_exit() at thread_exit+0x203/frame 0xfffffe0190317a40 exit1() at exit1+0xdde/frame 0xfffffe0190317ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe0190317ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0190317bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0190317bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8011c0e3a, rsp = 0x7fffffffe4e8, rbp = 0x7fffffffe500 --- Tracing command lockf2 pid 37034 tid 129852 td 0xfffffe01767eb1e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe018c33eb90 mi_switch() at mi_switch+0x155/frame 0xfffffe018c33ebb0 ast() at ast+0x289/frame 0xfffffe018c33ebf0 fast_syscall_common() at fast_syscall_common+0x1a5/frame 0xfffffe018c33ebf0 --- syscall (0, FreeBSD ELF64, nosys), rip = 0x8011a903a, rsp = 0x7fffffffe2e8, rbp = 0x7fffffffe330 --- Tracing command openat pid 37035 tid 134349 td 0xfffffe017b548020 sched_switch() at sched_switch+0x5af/frame 0xfffffe0192860b90 mi_switch() at mi_switch+0x155/frame 0xfffffe0192860bb0 ast() at ast+0x289/frame 0xfffffe0192860bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe360 Tracing command dirnprename pid 37036 tid 130491 td 0xfffffe015fcb7e40 sched_switch() at sched_switch+0x5af/frame 0xfffffe018dd06870 mi_switch() at mi_switch+0x155/frame 0xfffffe018dd06890 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe018dd068d0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe018dd06920 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe018dd06960 _sleep() at _sleep+0x285/frame 0xfffffe018dd06a00 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e1/frame 0xfffffe018dd06a80 sys_nanosleep() at sys_nanosleep+0x3b/frame 0xfffffe018dd06ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018dd06bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018dd06bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8011a90fa, rsp = 0x7fffffffe378, rbp = 0x7fffffffe3c0 --- Tracing command openat pid 37037 tid 132989 td 0xfffffe010cf27900 sched_switch() at sched_switch+0x5af/frame 0xfffffe0190dd0210 mi_switch() at mi_switch+0x155/frame 0xfffffe0190dd0230 __rw_wlock_hard() at __rw_wlock_hard+0x461/frame 0xfffffe0190dd02e0 _rw_wlock_cookie() at _rw_wlock_cookie+0xb7/frame 0xfffffe0190dd0320 softdep_load_inodeblock() at softdep_load_inodeblock+0x45/frame 0xfffffe0190dd0350 ffs_vgetf() at ffs_vgetf+0x4b0/frame 0xfffffe0190dd03d0 ffs_valloc() at ffs_valloc+0x57f/frame 0xfffffe0190dd0480 ufs_makeinode() at ufs_makeinode+0xe9/frame 0xfffffe0190dd0610 ufs_create() at ufs_create+0x34/frame 0xfffffe0190dd0630 VOP_CREATE_APV() at VOP_CREATE_APV+0x62/frame 0xfffffe0190dd0650 null_bypass() at null_bypass+0x12b/frame 0xfffffe0190dd07d0 VOP_CREATE_APV() at VOP_CREATE_APV+0x62/frame 0xfffffe0190dd07f0 vn_open_cred() at vn_open_cred+0x2b9/frame 0xfffffe0190dd0960 kern_openat() at kern_openat+0x28c/frame 0xfffffe0190dd0ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0190dd0bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0190dd0bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x8011a913a, rsp = 0x7fffffffe138, rbp = 0x7fffffffe220 --- Tracing command dirrename pid 37038 tid 128295 td 0xfffffe015a5bde40 sched_switch() at sched_switch+0x5af/frame 0xfffffe0189b7a870 mi_switch() at mi_switch+0x155/frame 0xfffffe0189b7a890 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0189b7a8d0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0189b7a920 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe0189b7a960 _sleep() at _sleep+0x285/frame 0xfffffe0189b7aa00 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e1/frame 0xfffffe0189b7aa80 sys_nanosleep() at sys_nanosleep+0x3b/frame 0xfffffe0189b7aac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0189b7abf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0189b7abf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8011a90fa, rsp = 0x7fffffffe378, rbp = 0x7fffffffe3c0 --- Tracing command openat pid 37039 tid 132888 td 0xfffffe010cec01e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe0190bdcb90 mi_switch() at mi_switch+0x155/frame 0xfffffe0190bdcbb0 ast() at ast+0x289/frame 0xfffffe0190bdcbf0 fast_syscall_common() at fast_syscall_common+0x1a5/frame 0xfffffe0190bdcbf0 --- syscall (0, FreeBSD ELF64, nosys), rip = 0x8011a60ba, rsp = 0x7fffffffe228, rbp = 0x7fffffffe360 --- Tracing command openat pid 37040 tid 132883 td 0xfffffe010ce99000 sched_switch() at sched_switch+0x5af/frame 0xfffffe0190bbeb90 mi_switch() at mi_switch+0x155/frame 0xfffffe0190bbebb0 ast() at ast+0x289/frame 0xfffffe0190bbebf0 fast_syscall_common() at fast_syscall_common+0x1a5/frame 0xfffffe0190bbebf0 --- syscall (0, FreeBSD ELF64, nosys), rip = 0x8011a869a, rsp = 0x7fffffffe228, rbp = 0x7fffffffe360 --- Tracing command openat pid 37041 tid 136342 td 0xfffffe014472b000 sched_switch() at sched_switch+0x5af/frame 0xfffffe0194f48b90 mi_switch() at mi_switch+0x155/frame 0xfffffe0194f48bb0 ast() at ast+0x289/frame 0xfffffe0194f48bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe360 Tracing command fts pid 37042 tid 128456 td 0xfffffe01463641e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe0189e9f6d0 mi_switch() at mi_switch+0x155/frame 0xfffffe0189e9f6f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0189e9f730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0189e9f780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0189e9f790 _sleep() at _sleep+0x325/frame 0xfffffe0189e9f830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe0189e9f8c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0189e9fac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0189e9fbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0189e9fbf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8011a83ba, rsp = 0x7fffffffe398, rbp = 0x7fffffffe3d0 --- Tracing command lockf pid 37043 tid 100263 td 0xfffffe0143601ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe01416ca6d0 mi_switch() at mi_switch+0x155/frame 0xfffffe01416ca6f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01416ca730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01416ca780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01416ca790 _sleep() at _sleep+0x325/frame 0xfffffe01416ca830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe01416ca8c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe01416caac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01416cabf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01416cabf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8011a93ba, rsp = 0x7fffffffe398, rbp = 0x7fffffffe3d0 --- Tracing command link pid 37044 tid 100319 td 0xfffffe0144263020 sched_switch() at sched_switch+0x5af/frame 0xfffffe014441d6d0 mi_switch() at mi_switch+0x155/frame 0xfffffe014441d6f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe014441d730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe014441d780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe014441d790 _sleep() at _sleep+0x325/frame 0xfffffe014441d830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe014441d8c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe014441dac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014441dbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014441dbf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8011a93ba, rsp = 0x7fffffffe398, rbp = 0x7fffffffe3d0 --- Tracing command fts pid 37045 tid 100483 td 0xfffffe0145689740 sched_switch() at sched_switch+0x5af/frame 0xfffffe014593fb90 mi_switch() at mi_switch+0x155/frame 0xfffffe014593fbb0 ast() at ast+0x289/frame 0xfffffe014593fbf0 fast_syscall_common() at fast_syscall_common+0x1a5/frame 0xfffffe014593fbf0 --- syscall (20, FreeBSD ELF64, sys_getpid), rip = 0x8011a7eda, rsp = 0x7fffffffe0b8, rbp = 0x7fffffffe170 --- Tracing command lockf pid 37046 tid 134429 td 0xfffffe017b5a0560 sched_switch() at sched_switch+0x5af/frame 0xfffffe01929f0b90 mi_switch() at mi_switch+0x155/frame 0xfffffe01929f0bb0 ast() at ast+0x289/frame 0xfffffe01929f0bf0 fast_syscall_common() at fast_syscall_common+0x1a5/frame 0xfffffe01929f0bf0 --- syscall (0, FreeBSD ELF64, nosys), rip = 0x8011a903a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe2e0 --- Tracing command link pid 37047 tid 129326 td 0xfffffe014cda2740 sched_switch() at sched_switch+0x5af/frame 0xfffffe018b8f8b90 mi_switch() at mi_switch+0x155/frame 0xfffffe018b8f8bb0 ast() at ast+0x289/frame 0xfffffe018b8f8bf0 fast_syscall_common() at fast_syscall_common+0x1a5/frame 0xfffffe018b8f8bf0 --- syscall (0, FreeBSD ELF64, nosys), rip = 0x8011a60ba, rsp = 0x7fffffffe228, rbp = 0x7fffffffe360 --- Tracing command fts pid 37048 tid 134964 td 0xfffffe010c8023a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe0193463480 mi_switch() at mi_switch+0x155/frame 0xfffffe01934634a0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01934634e0 sleeplk() at sleeplk+0x100/frame 0xfffffe0193463540 lockmgr_slock_hard() at lockmgr_slock_hard+0x3d2/frame 0xfffffe01934635e0 ffs_lock() at ffs_lock+0x9d/frame 0xfffffe0193463620 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe0193463640 null_lock() at null_lock+0xd3/frame 0xfffffe01934636a0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe01934636c0 _vn_lock() at _vn_lock+0x54/frame 0xfffffe0193463720 lookup() at lookup+0xc2/frame 0xfffffe01934637c0 namei() at namei+0x379/frame 0xfffffe0193463870 kern_statat() at kern_statat+0x12d/frame 0xfffffe01934639c0 sys_fstatat() at sys_fstatat+0x2f/frame 0xfffffe0193463ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0193463bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0193463bf0 --- syscall (552, FreeBSD ELF64, sys_fstatat), rip = 0x8011a7eda, rsp = 0x7fffffffe048, rbp = 0x7fffffffe100 --- Tracing command lockf pid 37049 tid 136301 td 0xfffffe01446fdc80 sched_switch() at sched_switch+0x5af/frame 0xfffffe0194e80470 mi_switch() at mi_switch+0x155/frame 0xfffffe0194e80490 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0194e804d0 sleeplk() at sleeplk+0x100/frame 0xfffffe0194e80530 lockmgr_slock_hard() at lockmgr_slock_hard+0x3d2/frame 0xfffffe0194e805d0 ffs_lock() at ffs_lock+0x9d/frame 0xfffffe0194e80610 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe0194e80630 null_lock() at null_lock+0xd3/frame 0xfffffe0194e80690 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe0194e806b0 _vn_lock() at _vn_lock+0x54/frame 0xfffffe0194e80710 lookup() at lookup+0xc2/frame 0xfffffe0194e807b0 namei() at namei+0x379/frame 0xfffffe0194e80860 kern_funlinkat() at kern_funlinkat+0x129/frame 0xfffffe0194e80aa0 sys_unlink() at sys_unlink+0x28/frame 0xfffffe0194e80ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0194e80bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0194e80bf0 --- syscall (10, FreeBSD ELF64, sys_unlink), rip = 0x8011a60ba, rsp = 0x7fffffffe318, rbp = 0x7fffffffe360 --- Tracing command link pid 37050 tid 100232 td 0xfffffe010e5a0ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe0141601b90 mi_switch() at mi_switch+0x155/frame 0xfffffe0141601bb0 ast() at ast+0x289/frame 0xfffffe0141601bf0 fast_syscall_common() at fast_syscall_common+0x1a5/frame 0xfffffe0141601bf0 --- syscall (0, FreeBSD ELF64, nosys), rip = 0x8011a609a, rsp = 0x7fffffffe228, rbp = 0x7fffffffe360 --- Tracing command fts pid 37051 tid 134426 td 0xfffffe017b5a13a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe01929e1480 mi_switch() at mi_switch+0x155/frame 0xfffffe01929e14a0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01929e14e0 sleeplk() at sleeplk+0x100/frame 0xfffffe01929e1540 lockmgr_slock_hard() at lockmgr_slock_hard+0x3d2/frame 0xfffffe01929e15e0 ffs_lock() at ffs_lock+0x9d/frame 0xfffffe01929e1620 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe01929e1640 null_lock() at null_lock+0xd3/frame 0xfffffe01929e16a0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe01929e16c0 _vn_lock() at _vn_lock+0x54/frame 0xfffffe01929e1720 lookup() at lookup+0xc2/frame 0xfffffe01929e17c0 namei() at namei+0x379/frame 0xfffffe01929e1870 kern_statat() at kern_statat+0x12d/frame 0xfffffe01929e19c0 sys_fstatat() at sys_fstatat+0x2f/frame 0xfffffe01929e1ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01929e1bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01929e1bf0 --- syscall (552, FreeBSD ELF64, sys_fstatat), rip = 0x8011a7eda, rsp = 0x7fffffffe048, rbp = 0x7fffffffe100 --- Tracing command lockf pid 37052 tid 136026 td 0xfffffe010cde71e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe0194921470 mi_switch() at mi_switch+0x155/frame 0xfffffe0194921490 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01949214d0 sleeplk() at sleeplk+0x100/frame 0xfffffe0194921530 lockmgr_slock_hard() at lockmgr_slock_hard+0x3d2/frame 0xfffffe01949215d0 ffs_lock() at ffs_lock+0x9d/frame 0xfffffe0194921610 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe0194921630 null_lock() at null_lock+0xd3/frame 0xfffffe0194921690 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe01949216b0 _vn_lock() at _vn_lock+0x54/frame 0xfffffe0194921710 lookup() at lookup+0xc2/frame 0xfffffe01949217b0 namei() at namei+0x379/frame 0xfffffe0194921860 kern_funlinkat() at kern_funlinkat+0x129/frame 0xfffffe0194921aa0 sys_unlink() at sys_unlink+0x28/frame 0xfffffe0194921ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0194921bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0194921bf0 --- syscall (10, FreeBSD ELF64, sys_unlink), rip = 0x8011a60ba, rsp = 0x7fffffffe318, rbp = 0x7fffffffe360 --- Tracing command link pid 37053 tid 130471 td 0xfffffe01665a6c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe018dca2b90 mi_switch() at mi_switch+0x155/frame 0xfffffe018dca2bb0 ast() at ast+0x289/frame 0xfffffe018dca2bf0 fast_syscall_common() at fast_syscall_common+0x1a5/frame 0xfffffe018dca2bf0 --- syscall (0, FreeBSD ELF64, nosys), rip = 0x8011a609a, rsp = 0x7fffffffe228, rbp = 0x7fffffffe360 --- Tracing command fts pid 37055 tid 127858 td 0xfffffe014ff9b720 sched_switch() at sched_switch+0x5af/frame 0xfffffe018b4cab90 mi_switch() at mi_switch+0x155/frame 0xfffffe018b4cabb0 ast() at ast+0x289/frame 0xfffffe018b4cabf0 fast_syscall_common() at fast_syscall_common+0x1a5/frame 0xfffffe018b4cabf0 --- syscall (20, FreeBSD ELF64, sys_getpid), rip = 0x8011a7eda, rsp = 0x7fffffffe0b8, rbp = 0x7fffffffe170 --- Tracing command lockf pid 37056 tid 134353 td 0xfffffe017b54e1e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe01928795c0 mi_switch() at mi_switch+0x155/frame 0xfffffe01928795e0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0192879620 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0192879670 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0192879680 _sleep() at _sleep+0x325/frame 0xfffffe0192879720 lf_advlockasync() at lf_advlockasync+0xaa3/frame 0xfffffe01928797c0 lf_advlock() at lf_advlock+0x45/frame 0xfffffe0192879810 vop_stdadvlock() at vop_stdadvlock+0x158/frame 0xfffffe0192879920 VOP_ADVLOCK_APV() at VOP_ADVLOCK_APV+0x59/frame 0xfffffe0192879940 null_advlock() at null_advlock+0x99/frame 0xfffffe01928799a0 VOP_ADVLOCK_APV() at VOP_ADVLOCK_APV+0x59/frame 0xfffffe01928799c0 kern_fcntl() at kern_fcntl+0xb87/frame 0xfffffe0192879a50 kern_fcntl_freebsd() at kern_fcntl_freebsd+0xa1/frame 0xfffffe0192879ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0192879bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0192879bf0 --- syscall (92, FreeBSD ELF64, sys_fcntl), rip = 0x8011a903a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe2e0 --- Tracing command swap pid 37057 tid 128291 td 0xfffffe015a5c93a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe0189b666d0 mi_switch() at mi_switch+0x155/frame 0xfffffe0189b666f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0189b66730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0189b66780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0189b66790 _sleep() at _sleep+0x325/frame 0xfffffe0189b66830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe0189b668c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0189b66ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0189b66bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0189b66bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8011a93ba, rsp = 0x7fffffffe398, rbp = 0x7fffffffe3d0 --- Tracing command creat pid 37058 tid 134466 td 0xfffffe017b5b6000 sched_switch() at sched_switch+0x5af/frame 0xfffffe0192aa46d0 mi_switch() at mi_switch+0x155/frame 0xfffffe0192aa46f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0192aa4730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0192aa4780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0192aa4790 _sleep() at _sleep+0x325/frame 0xfffffe0192aa4830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe0192aa48c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0192aa4ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0192aa4bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0192aa4bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8011a93ba, rsp = 0x7fffffffe398, rbp = 0x7fffffffe3d0 --- Tracing command dirrename pid 37060 tid 100324 td 0xfffffe0144262900 sched_switch() at sched_switch+0x5af/frame 0xfffffe01444186d0 mi_switch() at mi_switch+0x155/frame 0xfffffe01444186f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0144418730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0144418780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0144418790 _sleep() at _sleep+0x325/frame 0xfffffe0144418830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe01444188c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0144418ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0144418bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0144418bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8011a93ba, rsp = 0x7fffffffe388, rbp = 0x7fffffffe3c0 --- Tracing command fts pid 37061 tid 126772 td 0xfffffe015ed5ce40 sched_switch() at sched_switch+0x5af/frame 0xfffffe018cfb9400 mi_switch() at mi_switch+0x155/frame 0xfffffe018cfb9420 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe018cfb9460 sleeplk() at sleeplk+0x100/frame 0xfffffe018cfb94c0 lockmgr_slock_hard() at lockmgr_slock_hard+0x3d2/frame 0xfffffe018cfb9560 ffs_lock() at ffs_lock+0x9d/frame 0xfffffe018cfb95a0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe018cfb95c0 null_lock() at null_lock+0xd3/frame 0xfffffe018cfb9620 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe018cfb9640 _vn_lock() at _vn_lock+0x54/frame 0xfffffe018cfb96a0 lookup() at lookup+0xc2/frame 0xfffffe018cfb9740 namei() at namei+0x379/frame 0xfffffe018cfb97f0 vn_open_cred() at vn_open_cred+0x487/frame 0xfffffe018cfb9960 kern_openat() at kern_openat+0x28c/frame 0xfffffe018cfb9ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018cfb9bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018cfb9bf0 --- syscall (5, FreeBSD ELF64, sys_open), rip = 0x8011a811a, rsp = 0x7fffffffe1f8, rbp = 0x7fffffffe220 --- Tracing command lockf pid 37063 tid 129814 td 0xfffffe0174c78560 thread_exit() at thread_exit+0x203/frame 0xfffffe018c280a40 exit1() at exit1+0xdde/frame 0xfffffe018c280ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe018c280ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018c280bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018c280bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8011a8e3a, rsp = 0x7fffffffe1c8, rbp = 0x7fffffffe1e0 --- Tracing command creat pid 37064 tid 136343 td 0xfffffe01447411e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe0194f577e0 mi_switch() at mi_switch+0x155/frame 0xfffffe0194f57800 ipi_bitmap_handler() at ipi_bitmap_handler+0x87/frame 0xfffffe0194f57830 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0xb3/frame 0xfffffe0194f57830 --- interrupt, rip = 0xffffffff80f6e06f, rsp = 0xfffffe0194f57900, rbp = 0xfffffe0194f57940 --- uma_zfree_arg() at uma_zfree_arg+0x38f/frame 0xfffffe0194f57940 NDFREE_PNBUF() at NDFREE_PNBUF+0x33/frame 0xfffffe0194f57960 kern_openat() at kern_openat+0x342/frame 0xfffffe0194f57ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0194f57bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0194f57bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x8011a913a, rsp = 0x7fffffffe2a8, rbp = 0x7fffffffe360 --- Tracing command swap pid 37065 tid 133925 td 0xfffffe017b37e900 sched_switch() at sched_switch+0x5af/frame 0xfffffe0192018b90 mi_switch() at mi_switch+0x155/frame 0xfffffe0192018bb0 ast() at ast+0x289/frame 0xfffffe0192018bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe360 Tracing command dirrename pid 37067 tid 129320 td 0xfffffe01639a41e0 thread_exit() at thread_exit+0x203/frame 0xfffffe018b8daa40 exit1() at exit1+0xdde/frame 0xfffffe018b8daab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe018b8daac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018b8dabf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018b8dabf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8011a8e3a, rsp = 0x7fffffffe0b8, rbp = 0x7fffffffe0d0 --- Tracing command lockf pid 37068 tid 137341 td 0xfffffe0144cd43a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe01962d0470 mi_switch() at mi_switch+0x155/frame 0xfffffe01962d0490 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01962d04d0 sleeplk() at sleeplk+0x100/frame 0xfffffe01962d0530 lockmgr_slock_hard() at lockmgr_slock_hard+0x3d2/frame 0xfffffe01962d05d0 ffs_lock() at ffs_lock+0x9d/frame 0xfffffe01962d0610 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe01962d0630 null_lock() at null_lock+0xd3/frame 0xfffffe01962d0690 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe01962d06b0 _vn_lock() at _vn_lock+0x54/frame 0xfffffe01962d0710 lookup() at lookup+0xc2/frame 0xfffffe01962d07b0 namei() at namei+0x379/frame 0xfffffe01962d0860 kern_funlinkat() at kern_funlinkat+0x129/frame 0xfffffe01962d0aa0 sys_unlink() at sys_unlink+0x28/frame 0xfffffe01962d0ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01962d0bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01962d0bf0 --- syscall (10, FreeBSD ELF64, sys_unlink), rip = 0x8011a60ba, rsp = 0x7fffffffe318, rbp = 0x7fffffffe360 --- Tracing command creat pid 37070 tid 128191 td 0xfffffe01532bb900 sched_switch() at sched_switch+0x5af/frame 0xfffffe018996db90 mi_switch() at mi_switch+0x155/frame 0xfffffe018996dbb0 ast() at ast+0x289/frame 0xfffffe018996dbf0 fast_syscall_common() at fast_syscall_common+0x1a5/frame 0xfffffe018996dbf0 --- syscall (0, FreeBSD ELF64, nosys), rip = 0x8011a60ba, rsp = 0x7fffffffe2a8, rbp = 0x7fffffffe360 --- Tracing command swap pid 37071 tid 136804 td 0xfffffe01449bfe40 sched_switch() at sched_switch+0x5af/frame 0xfffffe0195853b90 mi_switch() at mi_switch+0x155/frame 0xfffffe0195853bb0 ast() at ast+0x289/frame 0xfffffe0195853bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe360 Tracing command dirrename pid 37073 tid 127084 td 0xfffffe01691f1ac0 thread_exit() at thread_exit+0x203/frame 0xfffffe018c3d4a40 exit1() at exit1+0xdde/frame 0xfffffe018c3d4ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe018c3d4ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018c3d4bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018c3d4bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8011a8e3a, rsp = 0x7fffffffe0b8, rbp = 0x7fffffffe0d0 --- Tracing command creat pid 37074 tid 134470 td 0xfffffe017b5c1020 sched_switch() at sched_switch+0x5af/frame 0xfffffe0192abdba0 mi_switch() at mi_switch+0x155/frame 0xfffffe0192abdbc0 ipi_bitmap_handler() at ipi_bitmap_handler+0x87/frame 0xfffffe0192abdbf0 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0xb3/frame 0xfffffe0192abdbf0 --- interrupt, rip = 0x801220572, rsp = 0x7fffffffd040, rbp = 0x7fffffffd080 --- Tracing command dirnprename pid 37075 tid 100249 td 0xfffffe01435ac3a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe01416756d0 mi_switch() at mi_switch+0x155/frame 0xfffffe01416756f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0141675730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0141675780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0141675790 _sleep() at _sleep+0x325/frame 0xfffffe0141675830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe01416758c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0141675ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0141675bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0141675bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8011a93ba, rsp = 0x7fffffffe388, rbp = 0x7fffffffe3c0 --- Tracing command swap pid 37078 tid 129940 td 0xfffffe01504ca000 sched_switch() at sched_switch+0x5af/frame 0xfffffe018d243ba0 mi_switch() at mi_switch+0x155/frame 0xfffffe018d243bc0 ipi_bitmap_handler() at ipi_bitmap_handler+0x87/frame 0xfffffe018d243bf0 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0xb3/frame 0xfffffe018d243bf0 --- interrupt, rip = 0x10247bc, rsp = 0x7fffffffe330, rbp = 0x7fffffffe360 --- Tracing command lockf pid 37079 tid 129856 td 0xfffffe01767eaac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe018c352470 mi_switch() at mi_switch+0x155/frame 0xfffffe018c352490 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe018c3524d0 sleeplk() at sleeplk+0x100/frame 0xfffffe018c352530 lockmgr_slock_hard() at lockmgr_slock_hard+0x3d2/frame 0xfffffe018c3525d0 ffs_lock() at ffs_lock+0x9d/frame 0xfffffe018c352610 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe018c352630 null_lock() at null_lock+0xd3/frame 0xfffffe018c352690 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe018c3526b0 _vn_lock() at _vn_lock+0x54/frame 0xfffffe018c352710 lookup() at lookup+0xc2/frame 0xfffffe018c3527b0 namei() at namei+0x379/frame 0xfffffe018c352860 kern_funlinkat() at kern_funlinkat+0x129/frame 0xfffffe018c352aa0 sys_unlink() at sys_unlink+0x28/frame 0xfffffe018c352ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018c352bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018c352bf0 --- syscall (10, FreeBSD ELF64, sys_unlink), rip = 0x8011a60ba, rsp = 0x7fffffffe318, rbp = 0x7fffffffe360 --- Tracing command fts pid 37080 tid 131209 td 0xfffffe0166583560 sched_switch() at sched_switch+0x5af/frame 0xfffffe018eb0cb90 mi_switch() at mi_switch+0x155/frame 0xfffffe018eb0cbb0 ast() at ast+0x289/frame 0xfffffe018eb0cbf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffdf60 Tracing command creat pid 37081 tid 126781 td 0xfffffe01532af560 sched_switch() at sched_switch+0x5af/frame 0xfffffe018cfe6b90 mi_switch() at mi_switch+0x155/frame 0xfffffe018cfe6bb0 ast() at ast+0x289/frame 0xfffffe018cfe6bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe360 Tracing command dirnprename pid 37082 tid 130310 td 0xfffffe0178e50740 thread_exit() at thread_exit+0x203/frame 0xfffffe018d97da40 exit1() at exit1+0xdde/frame 0xfffffe018d97dab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe018d97dac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018d97dbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018d97dbf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8011a8e3a, rsp = 0x7fffffffe0b8, rbp = 0x7fffffffe0d0 --- Tracing command fts pid 37084 tid 132078 td 0xfffffe017ad163a0 (CPU 6) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce61e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce61e10 trap() at trap+0x56/frame 0xfffffe003ce61f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce61f20 --- trap 0x13, rip = 0x8011e5b4c, rsp = 0x7fffffffe240, rbp = 0x7fffffffe2e0 --- Tracing command creat pid 37085 tid 132413 td 0xfffffe017aed4ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe0190290b90 mi_switch() at mi_switch+0x155/frame 0xfffffe0190290bb0 ast() at ast+0x289/frame 0xfffffe0190290bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe360 Tracing command swap pid 37086 tid 132326 td 0xfffffe017ad44000 sched_switch() at sched_switch+0x5af/frame 0xfffffe01900ddba0 mi_switch() at mi_switch+0x155/frame 0xfffffe01900ddbc0 ipi_bitmap_handler() at ipi_bitmap_handler+0x87/frame 0xfffffe01900ddbf0 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0xb3/frame 0xfffffe01900ddbf0 --- interrupt, rip = 0x10247bc, rsp = 0x7fffffffe330, rbp = 0x7fffffffe360 --- Tracing command lockf pid 37087 tid 133477 td 0xfffffe010d1973a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe0191758b90 mi_switch() at mi_switch+0x155/frame 0xfffffe0191758bb0 ast() at ast+0x289/frame 0xfffffe0191758bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe310 Tracing command dirnprename pid 37088 tid 129235 td 0xfffffe0174f3e740 thread_exit() at thread_exit+0x203/frame 0xfffffe018b731a40 exit1() at exit1+0xdde/frame 0xfffffe018b731ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe018b731ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018b731bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018b731bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8011a8e3a, rsp = 0x7fffffffe0b8, rbp = 0x7fffffffe0d0 --- Tracing command fts pid 37090 tid 132407 td 0xfffffe017aed51e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe0190272b90 mi_switch() at mi_switch+0x155/frame 0xfffffe0190272bb0 ast() at ast+0x289/frame 0xfffffe0190272bf0 fast_syscall_common() at fast_syscall_common+0x1a5/frame 0xfffffe0190272bf0 --- syscall (20, FreeBSD ELF64, sys_getpid), rip = 0x8011a7eda, rsp = 0x7fffffffe0b8, rbp = 0x7fffffffe170 --- Tracing command swap pid 37091 tid 131651 td 0xfffffe015e5f4720 sched_switch() at sched_switch+0x5af/frame 0xfffffe018f3aeb90 mi_switch() at mi_switch+0x155/frame 0xfffffe018f3aebb0 ast() at ast+0x289/frame 0xfffffe018f3aebf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe360 Tracing command lockf pid 37092 tid 100304 td 0xfffffe0144026020 sched_switch() at sched_switch+0x5af/frame 0xfffffe01417c0b90 mi_switch() at mi_switch+0x155/frame 0xfffffe01417c0bb0 ast() at ast+0x289/frame 0xfffffe01417c0bf0 fast_syscall_common() at fast_syscall_common+0x1a5/frame 0xfffffe01417c0bf0 --- syscall (0, FreeBSD ELF64, nosys), rip = 0x8011a903a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe2e0 --- Tracing command dirnprename pid 37093 tid 127454 td 0xfffffe015bbaae40 sched_switch() at sched_switch+0x5af/frame 0xfffffe018c91a480 mi_switch() at mi_switch+0x155/frame 0xfffffe018c91a4a0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe018c91a4e0 sleeplk() at sleeplk+0x100/frame 0xfffffe018c91a540 lockmgr_slock_hard() at lockmgr_slock_hard+0x3d2/frame 0xfffffe018c91a5e0 ffs_lock() at ffs_lock+0x9d/frame 0xfffffe018c91a620 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe018c91a640 null_lock() at null_lock+0xd3/frame 0xfffffe018c91a6a0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe018c91a6c0 _vn_lock() at _vn_lock+0x54/frame 0xfffffe018c91a720 lookup() at lookup+0xc2/frame 0xfffffe018c91a7c0 namei() at namei+0x379/frame 0xfffffe018c91a870 kern_renameat() at kern_renameat+0x10c/frame 0xfffffe018c91aac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018c91abf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018c91abf0 --- syscall (128, FreeBSD ELF64, sys_rename), rip = 0x8011a68da, rsp = 0x7fffffffe208, rbp = 0x7fffffffe350 --- Tracing command fts pid 37095 tid 100253 td 0xfffffe01436021e0 (CPU 9) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce88e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce88e10 trap() at trap+0x56/frame 0xfffffe003ce88f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce88f20 --- trap 0x13, rip = 0xffffffff80bc6f2d, rsp = 0xfffffe014167f730, rbp = 0xfffffe014167f750 --- pwd_hold() at pwd_hold+0x9d/frame 0xfffffe014167f750 namei_setup() at namei_setup+0x88/frame 0xfffffe014167f7c0 namei() at namei+0x332/frame 0xfffffe014167f870 kern_statat() at kern_statat+0x12d/frame 0xfffffe014167f9c0 sys_fstatat() at sys_fstatat+0x2f/frame 0xfffffe014167fac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014167fbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014167fbf0 --- syscall (552, FreeBSD ELF64, sys_fstatat), rip = 0x8011a7eda, rsp = 0x7fffffffe0b8, rbp = 0x7fffffffe170 --- Tracing command lockf pid 37096 tid 133198 td 0xfffffe010d026720 sched_switch() at sched_switch+0x5af/frame 0xfffffe01911e53f0 mi_switch() at mi_switch+0x155/frame 0xfffffe01911e5410 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01911e5450 sleeplk() at sleeplk+0x100/frame 0xfffffe01911e54b0 lockmgr_xlock_hard() at lockmgr_xlock_hard+0x402/frame 0xfffffe01911e5560 ffs_lock() at ffs_lock+0x9d/frame 0xfffffe01911e55a0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe01911e55c0 null_lock() at null_lock+0xd3/frame 0xfffffe01911e5620 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe01911e5640 _vn_lock() at _vn_lock+0x54/frame 0xfffffe01911e56a0 lookup() at lookup+0x3dc/frame 0xfffffe01911e5740 namei() at namei+0x379/frame 0xfffffe01911e57f0 vn_open_cred() at vn_open_cred+0x11e/frame 0xfffffe01911e5960 kern_openat() at kern_openat+0x28c/frame 0xfffffe01911e5ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01911e5bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01911e5bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x8011a913a, rsp = 0x7fffffffe228, rbp = 0x7fffffffe310 --- Tracing command dirnprename pid 37097 tid 133761 td 0xfffffe017b29be40 thread_exit() at thread_exit+0x203/frame 0xfffffe0191ce4a40 exit1() at exit1+0xdde/frame 0xfffffe0191ce4ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe0191ce4ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0191ce4bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0191ce4bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8011a8e3a, rsp = 0x7fffffffe0b8, rbp = 0x7fffffffe0d0 --- Tracing command swap pid 37098 tid 128029 td 0xfffffe01532c0900 sched_switch() at sched_switch+0x5af/frame 0xfffffe0189643b90 mi_switch() at mi_switch+0x155/frame 0xfffffe0189643bb0 ast() at ast+0x289/frame 0xfffffe0189643bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe360 Tracing command fts pid 37099 tid 129983 td 0xfffffe0176aaf3a0 (CPU 1) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce20e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce20e10 trap() at trap+0x56/frame 0xfffffe003ce20f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce20f20 --- trap 0x13, rip = 0xffffffff80f766ba, rsp = 0xfffffe018d31a778, rbp = 0xfffffe018d31a7c0 --- trash_ctor() at trash_ctor+0x3a/frame 0xfffffe018d31a7c0 namei() at namei+0x108/frame 0xfffffe018d31a870 kern_statat() at kern_statat+0x12d/frame 0xfffffe018d31a9c0 sys_fstatat() at sys_fstatat+0x2f/frame 0xfffffe018d31aac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018d31abf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018d31abf0 --- syscall (552, FreeBSD ELF64, sys_fstatat), rip = 0x8011a7eda, rsp = 0x7fffffffe0b8, rbp = 0x7fffffffe170 --- Tracing command lockf pid 37101 tid 127304 td 0xfffffe0176816560 sched_switch() at sched_switch+0x5af/frame 0xfffffe018ccdab90 mi_switch() at mi_switch+0x155/frame 0xfffffe018ccdabb0 ast() at ast+0x289/frame 0xfffffe018ccdabf0 fast_syscall_common() at fast_syscall_common+0x1a5/frame 0xfffffe018ccdabf0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x8011a91da, rsp = 0x7fffffffe2e8, rbp = 0x7fffffffe310 --- Tracing command dirnprename pid 37102 tid 127136 td 0xfffffe016cffd000 thread_exit() at thread_exit+0x203/frame 0xfffffe018c4d8a40 exit1() at exit1+0xdde/frame 0xfffffe018c4d8ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe018c4d8ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018c4d8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018c4d8bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8011a8e3a, rsp = 0x7fffffffe0b8, rbp = 0x7fffffffe0d0 --- Tracing command swap pid 37103 tid 131613 td 0xfffffe016a617020 sched_switch() at sched_switch+0x5af/frame 0xfffffe018f2f0b90 mi_switch() at mi_switch+0x155/frame 0xfffffe018f2f0bb0 ast() at ast+0x289/frame 0xfffffe018f2f0bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe360 Tracing command fts pid 37104 tid 137069 td 0xfffffe0144b443a0 (CPU 8) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce7be00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce7be10 trap() at trap+0x56/frame 0xfffffe003ce7bf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce7bf20 --- trap 0x13, rip = 0xffffffff80cf1a98, rsp = 0xfffffe0195d806e0, rbp = 0xfffffe0195d807c0 --- cache_fplookup() at cache_fplookup+0x2a8/frame 0xfffffe0195d807c0 namei() at namei+0x204/frame 0xfffffe0195d80870 kern_statat() at kern_statat+0x12d/frame 0xfffffe0195d809c0 sys_fstatat() at sys_fstatat+0x2f/frame 0xfffffe0195d80ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0195d80bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0195d80bf0 --- syscall (552, FreeBSD ELF64, sys_fstatat), rip = 0x8011a7eda, rsp = 0x7fffffffe0b8, rbp = 0x7fffffffe170 --- Tracing command lockf pid 37106 tid 137070 td 0xfffffe0144b43c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe0195d85b90 mi_switch() at mi_switch+0x155/frame 0xfffffe0195d85bb0 ast() at ast+0x289/frame 0xfffffe0195d85bf0 fast_syscall_common() at fast_syscall_common+0x1a5/frame 0xfffffe0195d85bf0 --- syscall (0, FreeBSD ELF64, nosys), rip = 0x8011a903a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe2e0 --- Tracing command dirnprename pid 37107 tid 132402 td 0xfffffe017aec13a0 thread_exit() at thread_exit+0x203/frame 0xfffffe0190259a40 exit1() at exit1+0xdde/frame 0xfffffe0190259ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe0190259ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0190259bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0190259bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8011a8e3a, rsp = 0x7fffffffe0b8, rbp = 0x7fffffffe0d0 --- Tracing command swap pid 37108 tid 100244 td 0xfffffe01435ad1e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe014165fba0 mi_switch() at mi_switch+0x155/frame 0xfffffe014165fbc0 ipi_bitmap_handler() at ipi_bitmap_handler+0x87/frame 0xfffffe014165fbf0 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0xb3/frame 0xfffffe014165fbf0 --- interrupt, rip = 0x10247bc, rsp = 0x7fffffffe330, rbp = 0x7fffffffe360 --- Tracing command fts pid 37109 tid 127144 td 0xfffffe0161395740 (CPU 2) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce2de00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce2de10 trap() at trap+0x56/frame 0xfffffe003ce2df20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce2df20 --- trap 0x13, rip = 0x8010fcd35, rsp = 0x7fffffffe0a8, rbp = 0x7fffffffe0b0 --- Tracing command lockf pid 37110 tid 137071 td 0xfffffe0144b43560 sched_switch() at sched_switch+0x5af/frame 0xfffffe0195d8ab90 mi_switch() at mi_switch+0x155/frame 0xfffffe0195d8abb0 ast() at ast+0x289/frame 0xfffffe0195d8abf0 fast_syscall_common() at fast_syscall_common+0x1a5/frame 0xfffffe0195d8abf0 --- syscall (0, FreeBSD ELF64, nosys), rip = 0x8011a903a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe2e0 --- Tracing command dirnprename pid 37112 tid 131027 td 0xfffffe0144103740 thread_exit() at thread_exit+0x203/frame 0xfffffe018e77ea40 exit1() at exit1+0xdde/frame 0xfffffe018e77eab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe018e77eac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018e77ebf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018e77ebf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8011a8e3a, rsp = 0x7fffffffe0b8, rbp = 0x7fffffffe0d0 --- Tracing command swap pid 37113 tid 134467 td 0xfffffe017b5c1740 sched_switch() at sched_switch+0x5af/frame 0xfffffe0192aaeb90 mi_switch() at mi_switch+0x155/frame 0xfffffe0192aaebb0 ast() at ast+0x289/frame 0xfffffe0192aaebf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe360 Tracing command fts pid 37114 tid 128293 td 0xfffffe015a5c8c80 (CPU 3) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce3ae00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce3ae10 trap() at trap+0x56/frame 0xfffffe003ce3af20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce3af20 --- trap 0x13, rip = 0x1023dbd, rsp = 0x7fffffffe320, rbp = 0x7fffffffe360 --- Tracing command dirnprename pid 37116 tid 131643 td 0xfffffe015e5f5560 (CPU 4) kdb_enter() at kdb_enter+0x37/frame 0xfffffe018f386360 vpanic() at vpanic+0x1b8/frame 0xfffffe018f3863c0 panic() at panic+0x43/frame 0xfffffe018f386420 trap_fatal() at trap_fatal+0x387/frame 0xfffffe018f386480 trap_pfault() at trap_pfault+0x99/frame 0xfffffe018f3864e0 trap() at trap+0x2a7/frame 0xfffffe018f3865f0 calltrap() at calltrap+0x8/frame 0xfffffe018f3865f0 --- trap 0xc, rip = 0xffffffff8274a6be, rsp = 0xfffffe018f3866c0, rbp = 0xfffffe018f3866d0 --- null_checkvp() at null_checkvp+0xe/frame 0xfffffe018f3866d0 null_bypass() at null_bypass+0xc9/frame 0xfffffe018f386850 VOP_RENAME_APV() at VOP_RENAME_APV+0x3c/frame 0xfffffe018f386870 kern_renameat() at kern_renameat+0x3dd/frame 0xfffffe018f386ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018f386bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018f386bf0 --- syscall (128, FreeBSD ELF64, sys_rename), rip = 0x8011a68da, rsp = 0x7fffffffe208, rbp = 0x7fffffffe350 --- Tracing command swap pid 37117 tid 136319 td 0xfffffe0144719560 sched_switch() at sched_switch+0x5af/frame 0xfffffe0194edab90 mi_switch() at mi_switch+0x155/frame 0xfffffe0194edabb0 ast() at ast+0x289/frame 0xfffffe0194edabf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe360 Tracing command fts pid 37118 tid 127092 td 0xfffffe01691f13a0 (CPU 0) cpustop_handler() at cpustop_handler+0x28/frame 0xffffffff81ff9220 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xffffffff81ff9230 trap() at trap+0x56/frame 0xffffffff81ff9340 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff81ff9340 --- trap 0x13, rip = 0x8011c7345, rsp = 0x7fffffffde98, rbp = 0x7fffffffdee0 --- Tracing command dirnprename pid 37120 tid 135968 td 0xfffffe010cd74720 thread_exit() at thread_exit+0x203/frame 0xfffffe01947ffa40 exit1() at exit1+0xdde/frame 0xfffffe01947ffab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe01947ffac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01947ffbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01947ffbf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8011a8e3a, rsp = 0x7fffffffe0b8, rbp = 0x7fffffffe0d0 --- Tracing command swap pid 37121 tid 127861 td 0xfffffe014ff9b000 sched_switch() at sched_switch+0x5af/frame 0xfffffe018b4d9b90 mi_switch() at mi_switch+0x155/frame 0xfffffe018b4d9bb0 ast() at ast+0x289/frame 0xfffffe018b4d9bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe360 Tracing command fts pid 37122 tid 128026 td 0xfffffe01532c1020 sched_switch() at sched_switch+0x5af/frame 0xfffffe0189634b90 mi_switch() at mi_switch+0x155/frame 0xfffffe0189634bb0 ast() at ast+0x289/frame 0xfffffe0189634bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe170 Tracing command dirnprename pid 37123 tid 135965 td 0xfffffe010cd74e40 thread_exit() at thread_exit+0x203/frame 0xfffffe01947f0a40 exit1() at exit1+0xdde/frame 0xfffffe01947f0ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe01947f0ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01947f0bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01947f0bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8011a8e3a, rsp = 0x7fffffffe0b8, rbp = 0x7fffffffe0d0 --- Tracing command swap pid 37124 tid 130498 td 0xfffffe0160fbac80 sched_switch() at sched_switch+0x5af/frame 0xfffffe018dd29ba0 mi_switch() at mi_switch+0x155/frame 0xfffffe018dd29bc0 ipi_bitmap_handler() at ipi_bitmap_handler+0x87/frame 0xfffffe018dd29bf0 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0xb3/frame 0xfffffe018dd29bf0 --- interrupt, rip = 0x10247bc, rsp = 0x7fffffffe330, rbp = 0x7fffffffe360 --- Tracing command fts pid 37125 tid 137401 td 0xfffffe0144cfe000 sched_switch() at sched_switch+0x5af/frame 0xfffffe01963fcb90 mi_switch() at mi_switch+0x155/frame 0xfffffe01963fcbb0 ast() at ast+0x289/frame 0xfffffe01963fcbf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe270 Tracing command dirnprename pid 37126 tid 127148 td 0xfffffe0161395020 thread_exit() at thread_exit+0x203/frame 0xfffffe018c514a40 exit1() at exit1+0xdde/frame 0xfffffe018c514ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe018c514ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018c514bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018c514bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8011a8e3a, rsp = 0x7fffffffe0b8, rbp = 0x7fffffffe0d0 --- Tracing command swap pid 37127 tid 126733 td 0xfffffe0146e04ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe018cef6b90 mi_switch() at mi_switch+0x155/frame 0xfffffe018cef6bb0 ast() at ast+0x289/frame 0xfffffe018cef6bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe360 Tracing command fts pid 37128 tid 127851 td 0xfffffe0149551ac0 (CPU 11) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003cea2e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003cea2e10 trap() at trap+0x56/frame 0xfffffe003cea2f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003cea2f20 --- trap 0x13, rip = 0x8011c7345, rsp = 0x7fffffffde78, rbp = 0x7fffffffdec0 --- Tracing command dirnprename pid 37129 tid 130308 td 0xfffffe0157891ac0 (CPU 5) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce54e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce54e10 trap() at trap+0x56/frame 0xfffffe003ce54f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce54f20 --- trap 0x13, rip = 0xffffffff80c703f0, rsp = 0xfffffe018d973200, rbp = 0xfffffe018d973210 --- lock_delay() at lock_delay+0x20/frame 0xfffffe018d973210 _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0xc1/frame 0xfffffe018d973280 __mtx_lock_spin_flags() at __mtx_lock_spin_flags+0xd5/frame 0xfffffe018d9732c0 cnputs() at cnputs+0x56/frame 0xfffffe018d9732f0 vprintf() at vprintf+0xcd/frame 0xfffffe018d9733c0 printf() at printf+0x43/frame 0xfffffe018d973420 trap_fatal() at trap_fatal+0x224/frame 0xfffffe018d973480 trap_pfault() at trap_pfault+0x99/frame 0xfffffe018d9734e0 trap() at trap+0x2a7/frame 0xfffffe018d9735f0 calltrap() at calltrap+0x8/frame 0xfffffe018d9735f0 --- trap 0xc, rip = 0xffffffff8274a6be, rsp = 0xfffffe018d9736c0, rbp = 0xfffffe018d9736d0 --- null_checkvp() at null_checkvp+0xe/frame 0xfffffe018d9736d0 null_bypass() at null_bypass+0xc9/frame 0xfffffe018d973850 VOP_RENAME_APV() at VOP_RENAME_APV+0x3c/frame 0xfffffe018d973870 kern_renameat() at kern_renameat+0x3dd/frame 0xfffffe018d973ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018d973bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018d973bf0 --- syscall (128, FreeBSD ELF64, sys_rename), rip = 0x8011a68da, rsp = 0x7fffffffe208, rbp = 0x7fffffffe350 --- Tracing command swap pid 37130 tid 132153 td 0xfffffe017ad49720 sched_switch() at sched_switch+0x5af/frame 0xfffffe018fd7cb90 mi_switch() at mi_switch+0x155/frame 0xfffffe018fd7cbb0 ast() at ast+0x289/frame 0xfffffe018fd7cbf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe360 Tracing command fts pid 37131 tid 128275 td 0xfffffe0169201900 (CPU 7) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce6ee00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce6ee10 trap() at trap+0x56/frame 0xfffffe003ce6ef20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce6ef20 --- trap 0x13, rip = 0x8011e6a4e, rsp = 0x7fffffffe000, rbp = 0x7fffffffe1b0 --- Tracing command dirnprename pid 37132 tid 135096 td 0xfffffe010c8b8020 thread_exit() at thread_exit+0x203/frame 0xfffffe01936f7a40 exit1() at exit1+0xdde/frame 0xfffffe01936f7ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe01936f7ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01936f7bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01936f7bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8011a8e3a, rsp = 0x7fffffffe0b8, rbp = 0x7fffffffe0d0 --- Tracing command swap pid 37133 tid 100228 td 0xfffffe010e3c13a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe01415edb90 mi_switch() at mi_switch+0x155/frame 0xfffffe01415edbb0 ast() at ast+0x289/frame 0xfffffe01415edbf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe360 Tracing command fts pid 37134 tid 127312 td 0xfffffe0176815720 sched_switch() at sched_switch+0x5af/frame 0xfffffe018cd02400 mi_switch() at mi_switch+0x155/frame 0xfffffe018cd02420 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe018cd02460 sleeplk() at sleeplk+0x100/frame 0xfffffe018cd024c0 lockmgr_slock_hard() at lockmgr_slock_hard+0x3d2/frame 0xfffffe018cd02560 ffs_lock() at ffs_lock+0x9d/frame 0xfffffe018cd025a0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe018cd025c0 null_lock() at null_lock+0xd3/frame 0xfffffe018cd02620 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe018cd02640 _vn_lock() at _vn_lock+0x54/frame 0xfffffe018cd026a0 lookup() at lookup+0xc2/frame 0xfffffe018cd02740 namei() at namei+0x379/frame 0xfffffe018cd027f0 vn_open_cred() at vn_open_cred+0x487/frame 0xfffffe018cd02960 kern_openat() at kern_openat+0x28c/frame 0xfffffe018cd02ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018cd02bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018cd02bf0 --- syscall (5, FreeBSD ELF64, sys_open), rip = 0x8011a811a, rsp = 0x7fffffffe1f8, rbp = 0x7fffffffe220 --- Tracing command dirnprename pid 37135 tid 128272 td 0xfffffe0169202020 thread_exit() at thread_exit+0x203/frame 0xfffffe0189b07a40 exit1() at exit1+0xdde/frame 0xfffffe0189b07ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe0189b07ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0189b07bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0189b07bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8011a8e3a, rsp = 0x7fffffffe0b8, rbp = 0x7fffffffe0d0 --- Tracing command swap pid 37136 tid 135093 td 0xfffffe010c8b8740 sched_switch() at sched_switch+0x5af/frame 0xfffffe01936e8b90 mi_switch() at mi_switch+0x155/frame 0xfffffe01936e8bb0 ast() at ast+0x289/frame 0xfffffe01936e8bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe360 Tracing command fts pid 37137 tid 100284 td 0xfffffe0143e12560 (CPU 10) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce95e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce95e10 trap() at trap+0x56/frame 0xfffffe003ce95f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce95f20 --- trap 0x13, rip = 0x8011c70c0, rsp = 0x7fffffffe2d8, rbp = 0x7fffffffe310 --- Tracing command dirnprename pid 37138 tid 137342 td 0xfffffe0144cd3c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe01962d5480 mi_switch() at mi_switch+0x155/frame 0xfffffe01962d54a0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01962d54e0 sleeplk() at sleeplk+0x100/frame 0xfffffe01962d5540 lockmgr_slock_hard() at lockmgr_slock_hard+0x3d2/frame 0xfffffe01962d55e0 ffs_lock() at ffs_lock+0x9d/frame 0xfffffe01962d5620 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe01962d5640 null_lock() at null_lock+0xd3/frame 0xfffffe01962d56a0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe01962d56c0 _vn_lock() at _vn_lock+0x54/frame 0xfffffe01962d5720 lookup() at lookup+0xc2/frame 0xfffffe01962d57c0 namei() at namei+0x379/frame 0xfffffe01962d5870 kern_renameat() at kern_renameat+0x10c/frame 0xfffffe01962d5ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01962d5bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01962d5bf0 --- syscall (128, FreeBSD ELF64, sys_rename), rip = 0x8011a68da, rsp = 0x7fffffffe208, rbp = 0x7fffffffe350 --- Tracing command swap pid 37139 tid 134464 td 0xfffffe017b5b9900 sched_switch() at sched_switch+0x5af/frame 0xfffffe0192a9fb90 mi_switch() at mi_switch+0x155/frame 0xfffffe0192a9fbb0 ast() at ast+0x289/frame 0xfffffe0192a9fbf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe360 Tracing command swap pid 37140 tid 100277 td 0xfffffe0143e13ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe014171eba0 mi_switch() at mi_switch+0x155/frame 0xfffffe014171ebc0 ipi_bitmap_handler() at ipi_bitmap_handler+0x87/frame 0xfffffe014171ebf0 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0xb3/frame 0xfffffe014171ebf0 --- interrupt, rip = 0x10247bc, rsp = 0x7fffffffe330, rbp = 0x7fffffffe360 --- Tracing command swap pid 37141 tid 133987 td 0xfffffe017b3a6e40 sched_switch() at sched_switch+0x5af/frame 0xfffffe019214eba0 mi_switch() at mi_switch+0x155/frame 0xfffffe019214ebc0 ipi_bitmap_handler() at ipi_bitmap_handler+0x87/frame 0xfffffe019214ebf0 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0xb3/frame 0xfffffe019214ebf0 --- interrupt, rip = 0x10247bc, rsp = 0x7fffffffe330, rbp = 0x7fffffffe360 --- Tracing command swap pid 37142 tid 100247 td 0xfffffe010e3c2900 sched_switch() at sched_switch+0x5af/frame 0xfffffe0141684ba0 mi_switch() at mi_switch+0x155/frame 0xfffffe0141684bc0 ipi_bitmap_handler() at ipi_bitmap_handler+0x87/frame 0xfffffe0141684bf0 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0xb3/frame 0xfffffe0141684bf0 --- interrupt, rip = 0x10247bc, rsp = 0x7fffffffe330, rbp = 0x7fffffffe360 --- Tracing command swap pid 37143 tid 132247 td 0xfffffe017adc9720 sched_switch() at sched_switch+0x5af/frame 0xfffffe018ff52b90 mi_switch() at mi_switch+0x155/frame 0xfffffe018ff52bb0 ast() at ast+0x289/frame 0xfffffe018ff52bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe360 Tracing command swap pid 37144 tid 128814 td 0xfffffe0176824e40 sched_switch() at sched_switch+0x5af/frame 0xfffffe018a5a2ba0 mi_switch() at mi_switch+0x155/frame 0xfffffe018a5a2bc0 ipi_bitmap_handler() at ipi_bitmap_handler+0x87/frame 0xfffffe018a5a2bf0 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0xb3/frame 0xfffffe018a5a2bf0 --- interrupt, rip = 0x10247bc, rsp = 0x7fffffffe330, rbp = 0x7fffffffe360 --- Tracing command swap pid 37145 tid 100254 td 0xfffffe010e5b3ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe014161fba0 mi_switch() at mi_switch+0x155/frame 0xfffffe014161fbc0 ipi_bitmap_handler() at ipi_bitmap_handler+0x87/frame 0xfffffe014161fbf0 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0xb3/frame 0xfffffe014161fbf0 --- interrupt, rip = 0x10247bc, rsp = 0x7fffffffe330, rbp = 0x7fffffffe360 --- Tracing command swap pid 37146 tid 133752 td 0xfffffe017b29cc80 sched_switch() at sched_switch+0x5af/frame 0xfffffe0191cb7b90 mi_switch() at mi_switch+0x155/frame 0xfffffe0191cb7bb0 ast() at ast+0x289/frame 0xfffffe0191cb7bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe360 Tracing command swap pid 37147 tid 137399 td 0xfffffe0144cfee40 sched_switch() at sched_switch+0x5af/frame 0xfffffe01963f2b90 mi_switch() at mi_switch+0x155/frame 0xfffffe01963f2bb0 ast() at ast+0x289/frame 0xfffffe01963f2bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe360 Tracing command swap pid 37148 tid 136800 td 0xfffffe01449c0c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe019583fb90 mi_switch() at mi_switch+0x155/frame 0xfffffe019583fbb0 ast() at ast+0x289/frame 0xfffffe019583fbf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe360 Tracing command swap pid 37149 tid 136856 td 0xfffffe0144a0ae40 sched_switch() at sched_switch+0x5af/frame 0xfffffe0195957b90 mi_switch() at mi_switch+0x155/frame 0xfffffe0195957bb0 ast() at ast+0x289/frame 0xfffffe0195957bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe360 Tracing command swap pid 37150 tid 100238 td 0xfffffe010dd39900 sched_switch() at sched_switch+0x5af/frame 0xfffffe014162eb90 mi_switch() at mi_switch+0x155/frame 0xfffffe014162ebb0 ast() at ast+0x289/frame 0xfffffe014162ebf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe360 Tracing command swap pid 37151 tid 132431 td 0xfffffe017aed3c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe01902eab90 mi_switch() at mi_switch+0x155/frame 0xfffffe01902eabb0 ast() at ast+0x289/frame 0xfffffe01902eabf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe360 Tracing command swap pid 37152 tid 132163 td 0xfffffe017ad841e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe018fdb3b90 mi_switch() at mi_switch+0x155/frame 0xfffffe018fdb3bb0 ast() at ast+0x289/frame 0xfffffe018fdb3bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe360 Tracing command swap pid 37153 tid 136461 td 0xfffffe01447efac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe01951a0b90 mi_switch() at mi_switch+0x155/frame 0xfffffe01951a0bb0 ast() at ast+0x289/frame 0xfffffe01951a0bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe360 Tracing command swap pid 37154 tid 131032 td 0xfffffe0144103020 sched_switch() at sched_switch+0x5af/frame 0xfffffe018e797ba0 mi_switch() at mi_switch+0x155/frame 0xfffffe018e797bc0 ipi_bitmap_handler() at ipi_bitmap_handler+0x87/frame 0xfffffe018e797bf0 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0xb3/frame 0xfffffe018e797bf0 --- interrupt, rip = 0x10247bc, rsp = 0x7fffffffe330, rbp = 0x7fffffffe360 --- Tracing command lockf pid 38274 tid 137182 td 0xfffffe0144bdce40 sched_switch() at sched_switch+0x5af/frame 0xfffffe0195fb5b90 mi_switch() at mi_switch+0x155/frame 0xfffffe0195fb5bb0 ast() at ast+0x289/frame 0xfffffe0195fb5bf0 fast_syscall_common() at fast_syscall_common+0x1a5/frame 0xfffffe0195fb5bf0 --- syscall (0, FreeBSD ELF64, nosys), rip = 0x8011a903a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe2e0 --- Tracing command sh pid 38306 tid 100486 td 0xfffffe0145689020 sched_switch() at sched_switch+0x5af/frame 0xfffffe014593a6d0 mi_switch() at mi_switch+0x155/frame 0xfffffe014593a6f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe014593a730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe014593a780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe014593a790 _sleep() at _sleep+0x325/frame 0xfffffe014593a830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe014593a8c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe014593aac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014593abf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014593abf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8012083ba, rsp = 0x7fffffffdfe8, rbp = 0x7fffffffe090 --- Tracing command lockf pid 38326 tid 126802 td 0xfffffe016139b000 sched_switch() at sched_switch+0x5af/frame 0xfffffe018d04f510 mi_switch() at mi_switch+0x155/frame 0xfffffe018d04f530 kern_yield() at kern_yield+0x71/frame 0xfffffe018d04f570 uiomove_fromphys() at uiomove_fromphys+0x171/frame 0xfffffe018d04f5e0 vn_read_from_obj() at vn_read_from_obj+0x161/frame 0xfffffe018d04f6c0 VOP_READ_PGCACHE_APV() at VOP_READ_PGCACHE_APV+0x34/frame 0xfffffe018d04f6e0 null_read_pgcache() at null_read_pgcache+0x91/frame 0xfffffe018d04f740 VOP_READ_PGCACHE_APV() at VOP_READ_PGCACHE_APV+0x34/frame 0xfffffe018d04f760 vn_read() at vn_read+0x94/frame 0xfffffe018d04f7e0 vn_io_fault_doio() at vn_io_fault_doio+0x43/frame 0xfffffe018d04f840 vn_io_fault1() at vn_io_fault1+0x163/frame 0xfffffe018d04f980 vn_io_fault() at vn_io_fault+0x1ba/frame 0xfffffe018d04fa00 dofileread() at dofileread+0x81/frame 0xfffffe018d04fa50 sys_read() at sys_read+0xc0/frame 0xfffffe018d04fac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe018d04fbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe018d04fbf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011a91da, rsp = 0x7fffffffe2e8, rbp = 0x7fffffffe310 --- Tracing command lockf pid 38330 tid 126814 td 0xfffffe01738c2560 sched_switch() at sched_switch+0x5af/frame 0xfffffe018d08bb90 mi_switch() at mi_switch+0x155/frame 0xfffffe018d08bbb0 ast() at ast+0x289/frame 0xfffffe018d08bbf0 fast_syscall_common() at fast_syscall_common+0x1a5/frame 0xfffffe018d08bbf0 --- syscall (0, FreeBSD ELF64, nosys), rip = 0x8011a903a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe2e0 --- Tracing command lockf pid 38346 tid 134777 td 0xfffffe010c6f4560 sched_switch() at sched_switch+0x5af/frame 0xfffffe01930bcb90 mi_switch() at mi_switch+0x155/frame 0xfffffe01930bcbb0 ast() at ast+0x289/frame 0xfffffe01930bcbf0 fast_syscall_common() at fast_syscall_common+0x1a5/frame 0xfffffe01930bcbf0 --- syscall (0, FreeBSD ELF64, nosys), rip = 0x8011a903a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe2e0 --- Tracing command lockf pid 38354 tid 131635 td 0xfffffe015e5f5c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe018f35eb90 mi_switch() at mi_switch+0x155/frame 0xfffffe018f35ebb0 ast() at ast+0x289/frame 0xfffffe018f35ebf0 fast_syscall_common() at fast_syscall_common+0x1a5/frame 0xfffffe018f35ebf0 --- syscall (0, FreeBSD ELF64, nosys), rip = 0x8011a903a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe2e0 --- Tracing command lockf pid 38363 tid 128816 td 0xfffffe01768213a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe018a5acb90 mi_switch() at mi_switch+0x155/frame 0xfffffe018a5acbb0 ast() at ast+0x289/frame 0xfffffe018a5acbf0 fast_syscall_common() at fast_syscall_common+0x1a5/frame 0xfffffe018a5acbf0 --- syscall (0, FreeBSD ELF64, nosys), rip = 0x8011a903a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe2e0 --- Tracing command lockf pid 38364 tid 130940 td 0xfffffe015ce1f560 sched_switch() at sched_switch+0x5af/frame 0xfffffe018e5cbb90 mi_switch() at mi_switch+0x155/frame 0xfffffe018e5cbbb0 ast() at ast+0x289/frame 0xfffffe018e5cbbf0 fast_syscall_common() at fast_syscall_common+0x1a5/frame 0xfffffe018e5cbbf0 --- syscall (0, FreeBSD ELF64, nosys), rip = 0x8011a903a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe2e0 --- Tracing command aiod1 pid 71136 tid 129402 td 0xfffffe01532ac900 sched_switch() at sched_switch+0x5af/frame 0xfffffe018ba74a20 mi_switch() at mi_switch+0x155/frame 0xfffffe018ba74a40 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe018ba74a80 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe018ba74ac0 _sleep() at _sleep+0x298/frame 0xfffffe018ba74b60 aio_daemon() at aio_daemon+0x305/frame 0xfffffe018ba74bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe018ba74bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe018ba74bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command aiod2 pid 71137 tid 131064 td 0xfffffe01639b5e40 sched_switch() at sched_switch+0x5af/frame 0xfffffe018e837a20 mi_switch() at mi_switch+0x155/frame 0xfffffe018e837a40 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe018e837a80 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe018e837ac0 _sleep() at _sleep+0x298/frame 0xfffffe018e837b60 aio_daemon() at aio_daemon+0x305/frame 0xfffffe018e837bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe018e837bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe018e837bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command aiod3 pid 71138 tid 131070 td 0xfffffe01639b5720 sched_switch() at sched_switch+0x5af/frame 0xfffffe018e855a20 mi_switch() at mi_switch+0x155/frame 0xfffffe018e855a40 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe018e855a80 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe018e855ac0 _sleep() at _sleep+0x298/frame 0xfffffe018e855b60 aio_daemon() at aio_daemon+0x305/frame 0xfffffe018e855bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe018e855bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe018e855bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command aiod4 pid 71139 tid 136316 td 0xfffffe014471a3a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe0194ecba20 mi_switch() at mi_switch+0x155/frame 0xfffffe0194ecba40 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0194ecba80 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0194ecbac0 _sleep() at _sleep+0x298/frame 0xfffffe0194ecbb60 aio_daemon() at aio_daemon+0x305/frame 0xfffffe0194ecbbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0194ecbbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0194ecbbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sleep pid 38372 tid 134783 td 0xfffffe010c6f3720 sched_switch() at sched_switch+0x5af/frame 0xfffffe01930d5870 mi_switch() at mi_switch+0x155/frame 0xfffffe01930d5890 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01930d58d0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01930d5920 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe01930d5960 _sleep() at _sleep+0x285/frame 0xfffffe01930d5a00 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e1/frame 0xfffffe01930d5a80 sys_nanosleep() at sys_nanosleep+0x3b/frame 0xfffffe01930d5ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01930d5bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01930d5bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8011a60fa, rsp = 0x7fffffffe4d8, rbp = 0x7fffffffe530 --- Tracing command sh pid 38374 tid 136272 td 0xfffffe01446d2560 fork_trampoline() at fork_trampoline Tracing command sh pid 72830 tid 136235 td 0xfffffe01446aa740 sched_switch() at sched_switch+0x5af/frame 0xfffffe0194d366d0 mi_switch() at mi_switch+0x155/frame 0xfffffe0194d366f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0194d36730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0194d36780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0194d36790 _sleep() at _sleep+0x325/frame 0xfffffe0194d36830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe0194d368c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0194d36ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0194d36bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0194d36bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8012083ba, rsp = 0x7fffffffe4f8, rbp = 0x7fffffffe5a0 --- Tracing command watchdogd pid 32520 tid 132650 td 0xfffffe017b00c3a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe0190731870 mi_switch() at mi_switch+0x155/frame 0xfffffe0190731890 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01907318d0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0190731920 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe0190731960 _sleep() at _sleep+0x285/frame 0xfffffe0190731a00 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e1/frame 0xfffffe0190731a80 sys_nanosleep() at sys_nanosleep+0x3b/frame 0xfffffe0190731ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0190731bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0190731bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8011c00fa, rsp = 0x7fffffffe9e8, rbp = 0x7fffffffea30 --- db> reset (kgdb) bt #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55 #1 doadump (textdump=textdump@entry=0x0) at ../../../kern/kern_shutdown.c:399 #2 0xffffffff804ce7fa in db_dump (dummy=, dummy2=, dummy3=, dummy4=) at ../../../ddb/db_command.c:575 #3 0xffffffff804ce6b2 in db_command (last_cmdp=, cmd_table=, dopager=dopager@entry=0x0) at ../../../ddb/db_command.c:482 #4 0xffffffff804ce746 in db_command_script (command=command@entry=0xffffffff81bd27c0 "dump") at ../../../ddb/db_command.c:553 #5 0xffffffff804d3bb8 in db_script_exec (scriptname=0xffffffff81bd2130 "dump", warnifnotfound=) at ../../../ddb/db_script.c:304 #6 0xffffffff804ce6b2 in db_command (last_cmdp=, cmd_table=, dopager=dopager@entry=0x0) at ../../../ddb/db_command.c:482 #7 0xffffffff804ce746 in db_command_script (command=command@entry=0xffffffff81bd2752 "run pho1") at ../../../ddb/db_command.c:553 #8 0xffffffff804d3bb8 in db_script_exec (scriptname=0xffffffff81bd2130 "dump", warnifnotfound=) at ../../../ddb/db_script.c:304 #9 0xffffffff804ce6b2 in db_command (last_cmdp=, cmd_table=, dopager=dopager@entry=0x1) at ../../../ddb/db_command.c:482 #10 0xffffffff804ce32d in db_command_loop () at ../../../ddb/db_command.c:535 #11 0xffffffff804d1aa6 in db_trap (type=, code=) at ../../../ddb/db_main.c:270 #12 0xffffffff80c6f667 in kdb_trap (type=type@entry=0x3, code=code@entry=0x0, tf=tf@entry=0xfffffe018f386290) at ../../../kern/subr_kdb.c:727 #13 0xffffffff810e9bd3 in trap (frame=0xfffffe018f386290) at ../../../amd64/amd64/trap.c:607 #14 #15 kdb_enter (why=0xffffffff812e497a "panic", msg=) at ../../../kern/subr_kdb.c:506 #16 0xffffffff80c22b98 in vpanic (fmt=0xffffffff811f5da2 "%s", ap=, ap@entry=0xfffffe018f386400) at ../../../kern/kern_shutdown.c:907 #17 0xffffffff80c22923 in panic (fmt=0xffffffff81e9d8e8 "\377\030*\201\377\377\377\377") at ../../../kern/kern_shutdown.c:843 #18 0xffffffff810e9fd7 in trap_fatal (frame=0xfffffe018f386600, eva=0x10) at ../../../amd64/amd64/trap.c:944 #19 0xffffffff810ea079 in trap_pfault (frame=frame@entry=0xfffffe018f386600, usermode=0x0, signo=, signo@entry=0x0, ucode=, ucode@entry=0x0) at ../../../amd64/amd64/trap.c:763 #20 0xffffffff810e9677 in trap (frame=0xfffffe018f386600) at ../../../amd64/amd64/trap.c:441 #21 #22 null_checkvp (vp=0xfffffe014b3c5000, fil=0xffffffff8274d1ce "/usr/src/sys/fs/nullfs/null_vnops.c", lno=lno@entry=0x10b) at /usr/src/sys/fs/nullfs/null_subr.c:328 #23 0xffffffff8274b109 in null_bypass (ap=0xfffffe018f386a38) at /usr/src/sys/fs/nullfs/null_vnops.c:267 #24 0xffffffff811b326c in VOP_RENAME_APV (vop=0xffffffff8274e3d0 , a=a@entry=0xfffffe018f386a38) at vnode_if.c:1678 #25 0xffffffff80d2413d in VOP_RENAME (fdvp=0xffffffff81b8d898 , fvp=, fcnp=, tdvp=, tvp=0x10b, tcnp=) at ./vnode_if.h:863 #26 kern_renameat (td=0xfffffe015e5f5560, oldfd=0xffffff9c, old=0x7fffffffe2a0 , newfd=0xffffff9c, new=0x7fffffffe220 , pathseg=) at ../../../kern/vfs_syscalls.c:3743 #27 0xffffffff810ea9c7 in syscallenter (td=0xfffffe015e5f5560) at ../../../amd64/amd64/../../kern/subr_syscall.c:189 #28 amd64_syscall (td=0xfffffe015e5f5560, traced=0x0) at ../../../amd64/amd64/trap.c:1186 #29 #30 0x00000008011a68da in ?? () Backtrace stopped: Cannot access memory at address 0x7fffffffe208 (kgdb) f 22 #22 null_checkvp (vp=0xfffffe014b3c5000, fil=0xffffffff8274d1ce "/usr/src/sys/fs/nullfs/null_vnops.c", lno=lno@entry=0x10b) at /usr/src/sys/fs/nullfs/null_subr.c:328 328 if (a->null_lowervp == NULLVP) { (kgdb) l 323 if (vp->v_op != null_vnodeop_p) { 324 printf ("null_checkvp: on non-null-node\n"); 325 panic("null_checkvp"); 326 } 327 #endif 328 if (a->null_lowervp == NULLVP) { 329 /* Should never happen */ 330 panic("null_checkvp %p", vp); 331 } 332 VI_LOCK_FLAGS(a->null_lowervp, MTX_DUPOK); (kgdb) info loc a = 0x0 (kgdb) p *vp $1 = {v_type = VBAD, v_irflag = 0x1, v_seqc = 0xae8d, v_nchash = 0x40ba6ab, v_hash = 0xa59e28, v_op = 0xffffffff81aed338 , v_data = 0x0, v_mount = 0x0, v_nmntvnodes = {tqe_next = 0xffffffffffffffff, tqe_prev = 0xffffffffffffffff}, {v_mountedhere = 0x0, v_unpcb = 0x0, v_rdev = 0x0, v_fifoinfo = 0x0}, v_hashlist = {le_next = 0xffffffffffffffff, le_prev = 0xffffffffffffffff}, v_cache_src = {lh_first = 0x0}, v_cache_dst = {tqh_first = 0x0, tqh_last = 0xfffffe014b3c5058}, v_cache_dd = 0x0, v_lock = {lock_object = {lo_name = 0xffffffff8274d05e "nullfs", lo_flags = 0x6fb0008, lo_data = 0x0, lo_witness = 0x0}, lk_lock = 0x1, lk_exslpfail = 0x0, lk_pri = 0x60, lk_timo = 0x33, lk_stack = {depth = 0xb, pcs = {0xffffffff80bf2cda, 0xffffffff80bf319e, 0xffffffff80bf2145, 0xffffffff8274bc3e, 0xffffffff811b413c, 0xffffffff80d28414, 0xffffffff80d00c32, 0xffffffff80cffe39, 0xffffffff80d23f3b, 0xffffffff810ea9c7, 0xffffffff810badce, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}}, v_interlock = {lock_object = {lo_name = 0xffffffff81332525 "vnode interlock", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0x0}, mtx_lock = 0x0}, v_vnlock = 0xfffffe014b3c5070, v_vnodelist = {tqe_next = 0xfffffe016fd4d708, tqe_prev = 0xfffffe0197ae4d10}, v_lazylist = {tqe_next = 0xffffffffffffffff, tqe_prev = 0xffffffffffffffff}, v_bufobj = {bo_lock = {lock_object = {lo_name = 0xffffffff812ea94f "bufobj interlock", lo_flags = 0x5230000, lo_data = 0x0, lo_witness = 0x0}, rw_lock = 0x1}, bo_ops = 0xffffffff81b25b90 , bo_object = 0x0, bo_synclist = {le_next = 0xffffffffffffffff, le_prev = 0xffffffffffffffff}, bo_private = 0xfffffe014b3c5000, bo_clean = {bv_hd = {tqh_first = 0x0, tqh_last = 0xfffffe014b3c51c0}, bv_root = {pt_root = 0x0}, bv_cnt = 0x0}, bo_dirty = {bv_hd = {tqh_first = 0x0, tqh_last = 0xfffffe014b3c51e0}, bv_root = {pt_root = 0x0}, bv_cnt = 0x0}, bo_numoutput = 0x0, bo_flag = 0x0, bo_domain = 0x0, bo_bsize = 0x8000}, v_pollinfo = 0x0, v_label = 0x0, v_lockf = 0x0, v_rl = {rl_waiters = {tqh_first = 0x0, tqh_last = 0xfffffe014b3c5228}, rl_currdep = 0x0}, v_holdcnt = 0x3, v_usecount = 0x1b, v_iflag = 0x0, v_vflag = 0x0, v_mflag = 0x0, v_dbatchcpu = 0xffff, v_writecount = 0x0, v_seqc_users = 0x1} (kgdb) diff --git a/sys/fs/fifofs/fifo_vnops.c b/sys/fs/fifofs/fifo_vnops.c index d2a51de84fba..861f1b40a744 100644 --- a/sys/fs/fifofs/fifo_vnops.c +++ b/sys/fs/fifofs/fifo_vnops.c @@ -282,9 +282,21 @@ fifo_close(ap) struct pipe *cpipe; vp = ap->a_vp; + ASSERT_VOP_ELOCKED(vp, "fifo_close"); fip = vp->v_fifoinfo; + + /* + * During open, it is possible that the fifo vnode is relocked + * after the vnode is instantiated but before VOP_OPEN() is + * done. For instance, vn_open_vnode() might need to upgrade + * vnode lock, or ffs_vput_pair() needs to unlock vp to sync + * dvp. In this case, reclaim can observe us with v_fifoinfo + * equal to NULL. + */ + if (fip == NULL) + return (0); + cpipe = fip->fi_pipe; - ASSERT_VOP_ELOCKED(vp, "fifo_close"); if (ap->a_fflag & FREAD) { fip->fi_readers--; if (fip->fi_readers == 0) { diff --git a/sys/fs/nullfs/null.h b/sys/fs/nullfs/null.h index 6fdac4b1006e..a41625536d65 100644 --- a/sys/fs/nullfs/null.h +++ b/sys/fs/nullfs/null.h @@ -45,6 +45,8 @@ struct null_mount { struct mount *nullm_vfs; struct vnode *nullm_lowerrootvp; /* Ref to lower root vnode */ uint64_t nullm_flags; + struct mount_upper_node upper_node; + struct mount_upper_node notify_node; }; #ifdef _KERNEL diff --git a/sys/fs/nullfs/null_vfsops.c b/sys/fs/nullfs/null_vfsops.c index 4914e5fc2dbf..73301c9275d2 100644 --- a/sys/fs/nullfs/null_vfsops.c +++ b/sys/fs/nullfs/null_vfsops.c @@ -163,7 +163,8 @@ nullfs_mount(struct mount *mp) * Save pointer to underlying FS and the reference to the * lower root vnode. */ - xmp->nullm_vfs = vfs_pin_from_vp(lowerrootvp); + xmp->nullm_vfs = vfs_register_upper_from_vp(lowerrootvp, mp, + &xmp->upper_node); if (xmp->nullm_vfs == NULL) { vput(lowerrootvp); free(xmp, M_NULLFSMNT); @@ -178,7 +179,7 @@ nullfs_mount(struct mount *mp) */ error = null_nodeget(mp, lowerrootvp, &nullm_rootvp); if (error != 0) { - vfs_unpin(xmp->nullm_vfs); + vfs_unregister_upper(xmp->nullm_vfs, &xmp->upper_node); vrele(lowerrootvp); free(xmp, M_NULLFSMNT); return (error); @@ -195,6 +196,11 @@ nullfs_mount(struct mount *mp) (xmp->nullm_vfs->mnt_kern_flag & MNTK_NULL_NOCACHE) != 0) xmp->nullm_flags &= ~NULLM_CACHE; + if ((xmp->nullm_flags & NULLM_CACHE) != 0) { + vfs_register_for_notification(xmp->nullm_vfs, mp, + &xmp->notify_node); + } + MNT_ILOCK(mp); if ((xmp->nullm_flags & NULLM_CACHE) != 0) { mp->mnt_kern_flag |= lowerrootvp->v_mount->mnt_kern_flag & @@ -206,13 +212,6 @@ nullfs_mount(struct mount *mp) (MNTK_USES_BCACHE | MNTK_NO_IOPF | MNTK_UNMAPPED_BUFS); MNT_IUNLOCK(mp); vfs_getnewfsid(mp); - if ((xmp->nullm_flags & NULLM_CACHE) != 0) { - MNT_ILOCK(xmp->nullm_vfs); - TAILQ_INSERT_TAIL(&xmp->nullm_vfs->mnt_uppers, mp, - mnt_upper_link); - MNT_IUNLOCK(xmp->nullm_vfs); - } - vfs_mountedfrom(mp, target); vput(nullm_rootvp); @@ -230,7 +229,6 @@ nullfs_unmount(mp, mntflags) int mntflags; { struct null_mount *mntdata; - struct mount *ump; int error, flags; NULLFSDEBUG("nullfs_unmount: mp = %p\n", (void *)mp); @@ -259,17 +257,11 @@ nullfs_unmount(mp, mntflags) * Finally, throw away the null_mount structure */ mntdata = mp->mnt_data; - ump = mntdata->nullm_vfs; if ((mntdata->nullm_flags & NULLM_CACHE) != 0) { - MNT_ILOCK(ump); - while ((ump->mnt_kern_flag & MNTK_VGONE_UPPER) != 0) { - ump->mnt_kern_flag |= MNTK_VGONE_WAITER; - msleep(&ump->mnt_uppers, &ump->mnt_mtx, 0, "vgnupw", 0); - } - TAILQ_REMOVE(&ump->mnt_uppers, mp, mnt_upper_link); - MNT_IUNLOCK(ump); + vfs_unregister_for_notification(mntdata->nullm_vfs, + &mntdata->notify_node); } - vfs_unpin(ump); + vfs_unregister_upper(mntdata->nullm_vfs, &mntdata->upper_node); vrele(mntdata->nullm_lowerrootvp); mp->mnt_data = NULL; free(mntdata, M_NULLFSMNT); diff --git a/sys/fs/nullfs/null_vnops.c b/sys/fs/nullfs/null_vnops.c index aeebce4ed086..e6d742b9921f 100644 --- a/sys/fs/nullfs/null_vnops.c +++ b/sys/fs/nullfs/null_vnops.c @@ -223,13 +223,12 @@ int null_bypass(struct vop_generic_args *ap) { struct vnode **this_vp_p; - int error; struct vnode *old_vps[VDESC_MAX_VPS]; struct vnode **vps_p[VDESC_MAX_VPS]; struct vnode ***vppp; struct vnode *lvp; struct vnodeop_desc *descp = ap->a_desc; - int reles, i; + int error, i, reles; if (null_bug_bypass) printf ("null_bypass: %s\n", descp->vdesc_name); @@ -252,26 +251,39 @@ null_bypass(struct vop_generic_args *ap) for (i = 0; i < VDESC_MAX_VPS; reles >>= 1, i++) { if (descp->vdesc_vp_offsets[i] == VDESC_NO_OFFSET) break; /* bail out at end of list */ - vps_p[i] = this_vp_p = - VOPARG_OFFSETTO(struct vnode**,descp->vdesc_vp_offsets[i],ap); + vps_p[i] = this_vp_p = VOPARG_OFFSETTO(struct vnode **, + descp->vdesc_vp_offsets[i],ap); + /* * We're not guaranteed that any but the first vnode * are of our type. Check for and don't map any * that aren't. (We must always map first vp or vclean fails.) */ - if (i && (*this_vp_p == NULLVP || + if (i != 0 && (*this_vp_p == NULLVP || (*this_vp_p)->v_op != &null_vnodeops)) { old_vps[i] = NULLVP; } else { old_vps[i] = *this_vp_p; *(vps_p[i]) = NULLVPTOLOWERVP(*this_vp_p); + + /* + * The upper vnode reference to the lower + * vnode is the only reference that keeps our + * pointer to the lower vnode alive. If lower + * vnode is relocked during the VOP call, + * upper vnode might become unlocked and + * reclaimed, which invalidates our reference. + * Add a transient hold around VOP call. + */ + vhold(*this_vp_p); + /* * XXX - Several operations have the side effect * of vrele'ing their vp's. We must account for * that. (This should go away in the future.) */ if (reles & VDESC_VP0_WILLRELE) - VREF(*this_vp_p); + vref(*this_vp_p); } } @@ -279,9 +291,9 @@ null_bypass(struct vop_generic_args *ap) * Call the operation on the lower layer * with the modified argument structure. */ - if (vps_p[0] && *vps_p[0]) + if (vps_p[0] != NULL && *vps_p[0] != NULL) { error = VCALL(ap); - else { + } else { printf("null_bypass: no map for %s\n", descp->vdesc_name); error = EINVAL; } @@ -295,10 +307,11 @@ null_bypass(struct vop_generic_args *ap) for (i = 0; i < VDESC_MAX_VPS; reles >>= 1, i++) { if (descp->vdesc_vp_offsets[i] == VDESC_NO_OFFSET) break; /* bail out at end of list */ - if (old_vps[i]) { + if (old_vps[i] != NULL) { lvp = *(vps_p[i]); /* + * Get rid of the transient hold on lvp. * If lowervp was unlocked during VOP * operation, nullfs upper vnode could have * been reclaimed, which changes its v_vnlock @@ -306,11 +319,14 @@ null_bypass(struct vop_generic_args *ap) * must move lock ownership from lower to * upper (reclaimed) vnode. */ - if (lvp != NULLVP && - VOP_ISLOCKED(lvp) == LK_EXCLUSIVE && - old_vps[i]->v_vnlock != lvp->v_vnlock) { - VOP_UNLOCK(lvp); - VOP_LOCK(old_vps[i], LK_EXCLUSIVE | LK_RETRY); + if (lvp != NULLVP) { + if (VOP_ISLOCKED(lvp) == LK_EXCLUSIVE && + old_vps[i]->v_vnlock != lvp->v_vnlock) { + VOP_UNLOCK(lvp); + VOP_LOCK(old_vps[i], LK_EXCLUSIVE | + LK_RETRY); + } + vdrop(lvp); } *(vps_p[i]) = old_vps[i]; @@ -328,17 +344,18 @@ null_bypass(struct vop_generic_args *ap) * (Assumes that the lower layer always returns * a VREF'ed vpp unless it gets an error.) */ - if (descp->vdesc_vpp_offset != VDESC_NO_OFFSET && !error) { + if (descp->vdesc_vpp_offset != VDESC_NO_OFFSET && error == 0) { /* * XXX - even though some ops have vpp returned vp's, * several ops actually vrele this before returning. * We must avoid these ops. * (This should go away when these ops are regularized.) */ - vppp = VOPARG_OFFSETTO(struct vnode***, - descp->vdesc_vpp_offset,ap); - if (*vppp) - error = null_nodeget(old_vps[0]->v_mount, **vppp, *vppp); + vppp = VOPARG_OFFSETTO(struct vnode ***, + descp->vdesc_vpp_offset,ap); + if (*vppp != NULL) + error = null_nodeget(old_vps[0]->v_mount, **vppp, + *vppp); } return (error); @@ -419,11 +436,12 @@ null_lookup(struct vop_lookup_args *ap) * dvp to be reclaimed due to shared v_vnlock. Check for the * doomed state and return error. */ - if ((error == 0 || error == EJUSTRETURN) && - VN_IS_DOOMED(dvp)) { - error = ENOENT; - if (lvp != NULL) - vput(lvp); + if (VN_IS_DOOMED(dvp)) { + if (error == 0 || error == EJUSTRETURN) { + if (lvp != NULL) + vput(lvp); + error = ENOENT; + } /* * If vgone() did reclaimed dvp before curthread @@ -978,6 +996,28 @@ null_read_pgcache(struct vop_read_pgcache_args *ap) return (error); } +static int +null_advlock(struct vop_advlock_args *ap) +{ + struct vnode *lvp, *vp; + struct null_node *xp; + int error; + + vp = ap->a_vp; + VI_LOCK(vp); + xp = VTONULL(vp); + if (xp == NULL) { + VI_UNLOCK(vp); + return (EBADF); + } + lvp = xp->null_lowervp; + vref(lvp); + VI_UNLOCK(vp); + error = VOP_ADVLOCK(lvp, ap->a_id, ap->a_op, ap->a_fl, ap->a_flags); + vrele(lvp); + return (error); +} + /* * Avoid standard bypass, since lower dvp and vp could be no longer * valid after vput(). @@ -1050,6 +1090,7 @@ struct vop_vector null_vnodeops = { .vop_bypass = null_bypass, .vop_access = null_access, .vop_accessx = null_accessx, + .vop_advlock = null_advlock, .vop_advlockpurge = vop_stdadvlockpurge, .vop_bmap = VOP_EOPNOTSUPP, .vop_stat = null_stat, diff --git a/sys/fs/unionfs/union.h b/sys/fs/unionfs/union.h index 64706b2b21a2..96180480dbec 100644 --- a/sys/fs/unionfs/union.h +++ b/sys/fs/unionfs/union.h @@ -57,6 +57,8 @@ struct unionfs_mount { struct vnode *um_lowervp; /* VREFed once */ struct vnode *um_uppervp; /* VREFed once */ struct vnode *um_rootvp; /* ROOT vnode */ + struct mount_upper_node um_lower_link; /* node in lower FS list of uppers */ + struct mount_upper_node um_upper_link; /* node in upper FS list of uppers */ unionfs_copymode um_copymode; unionfs_whitemode um_whitemode; uid_t um_uid; diff --git a/sys/fs/unionfs/union_vfsops.c b/sys/fs/unionfs/union_vfsops.c index 96a30f0ae8b5..c17650dedc63 100644 --- a/sys/fs/unionfs/union_vfsops.c +++ b/sys/fs/unionfs/union_vfsops.c @@ -292,14 +292,16 @@ unionfs_domount(struct mount *mp) return (error); } - lowermp = vfs_pin_from_vp(ump->um_lowervp); - uppermp = vfs_pin_from_vp(ump->um_uppervp); + lowermp = vfs_register_upper_from_vp(ump->um_lowervp, mp, + &ump->um_lower_link); + uppermp = vfs_register_upper_from_vp(ump->um_uppervp, mp, + &ump->um_upper_link); if (lowermp == NULL || uppermp == NULL) { if (lowermp != NULL) - vfs_unpin(lowermp); + vfs_unregister_upper(lowermp, &ump->um_lower_link); if (uppermp != NULL) - vfs_unpin(uppermp); + vfs_unregister_upper(uppermp, &ump->um_upper_link); free(ump, M_UNIONFSMNT); mp->mnt_data = NULL; return (ENOENT); @@ -357,8 +359,8 @@ unionfs_unmount(struct mount *mp, int mntflags) if (error) return (error); - vfs_unpin(ump->um_lowervp->v_mount); - vfs_unpin(ump->um_uppervp->v_mount); + vfs_unregister_upper(ump->um_lowervp->v_mount, &ump->um_lower_link); + vfs_unregister_upper(ump->um_uppervp->v_mount, &ump->um_upper_link); free(ump, M_UNIONFSMNT); mp->mnt_data = NULL; diff --git a/sys/kern/vfs_mount.c b/sys/kern/vfs_mount.c index 8b4426209818..6d7bc181d4b0 100644 --- a/sys/kern/vfs_mount.c +++ b/sys/kern/vfs_mount.c @@ -65,6 +65,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include @@ -89,6 +90,11 @@ static bool default_autoro = false; SYSCTL_BOOL(_vfs, OID_AUTO, default_autoro, CTLFLAG_RW, &default_autoro, 0, "Retry failed r/w mount as r/o if no explicit ro/rw option is specified"); +static bool recursive_forced_unmount = false; +SYSCTL_BOOL(_vfs, OID_AUTO, recursive_forced_unmount, CTLFLAG_RW, + &recursive_forced_unmount, 0, "Recursively unmount stacked upper mounts" + " when a file system is forcibly unmounted"); + MALLOC_DEFINE(M_MOUNT, "mount", "vfs mount structure"); MALLOC_DEFINE(M_STATFS, "statfs", "statfs structure"); static uma_zone_t mount_zone; @@ -103,6 +109,16 @@ MTX_SYSINIT(mountlist, &mountlist_mtx, "mountlist", MTX_DEF); EVENTHANDLER_LIST_DEFINE(vfs_mounted); EVENTHANDLER_LIST_DEFINE(vfs_unmounted); +static void vfs_deferred_unmount(void *arg, int pending); +static struct task deferred_unmount_task = + TASK_INITIALIZER(0, vfs_deferred_unmount, NULL);; +static struct mtx deferred_unmount_lock; +MTX_SYSINIT(deferred_unmount, &deferred_unmount_lock, "deferred_unmount", + MTX_DEF); +static STAILQ_HEAD(, mount) deferred_unmount_list = + STAILQ_HEAD_INITIALIZER(deferred_unmount_list); +TASKQUEUE_DEFINE_THREAD(deferred_unmount); + static void mount_devctl_event(const char *type, struct mount *mp, bool donew); /* @@ -505,8 +521,21 @@ vfs_ref(struct mount *mp) MNT_IUNLOCK(mp); } +/* + * Register ump as an upper mount of the mount associated with + * vnode vp. This registration will be tracked through + * mount_upper_node upper, which should be allocated by the + * caller and stored in per-mount data associated with mp. + * + * If successful, this function will return the mount associated + * with vp, and will ensure that it cannot be unmounted until + * ump has been unregistered as one of its upper mounts. + * + * Upon failure this function will return NULL. + */ struct mount * -vfs_pin_from_vp(struct vnode *vp) +vfs_register_upper_from_vp(struct vnode *vp, struct mount *ump, + struct mount_upper_node *upper) { struct mount *mp; @@ -514,26 +543,81 @@ vfs_pin_from_vp(struct vnode *vp) if (mp == NULL) return (NULL); MNT_ILOCK(mp); - if (mp != vp->v_mount || (mp->mnt_kern_flag & MNTK_UNMOUNT) != 0) { + if (mp != vp->v_mount || + ((mp->mnt_kern_flag & (MNTK_UNMOUNT | MNTK_RECURSE)) != 0)) { MNT_IUNLOCK(mp); return (NULL); } + KASSERT(ump != mp, ("upper and lower mounts are identical")); + upper->mp = ump; MNT_REF(mp); - KASSERT(mp->mnt_pinned_count < INT_MAX, - ("mount pinned count overflow")); - ++mp->mnt_pinned_count; + TAILQ_INSERT_TAIL(&mp->mnt_uppers, upper, mnt_upper_link); MNT_IUNLOCK(mp); return (mp); } +/* + * Register upper mount ump to receive vnode unlink/reclaim + * notifications from lower mount mp. This registration will + * be tracked through mount_upper_node upper, which should be + * allocated by the caller and stored in per-mount data + * associated with mp. + * + * ump must already be registered as an upper mount of mp + * through a call to vfs_register_upper_from_vp(). + */ void -vfs_unpin(struct mount *mp) +vfs_register_for_notification(struct mount *mp, struct mount *ump, + struct mount_upper_node *upper) +{ + upper->mp = ump; + MNT_ILOCK(mp); + TAILQ_INSERT_TAIL(&mp->mnt_notify, upper, mnt_upper_link); + MNT_IUNLOCK(mp); +} + +static void +vfs_drain_upper_locked(struct mount *mp) +{ + mtx_assert(MNT_MTX(mp), MA_OWNED); + while (mp->mnt_upper_pending != 0) { + mp->mnt_kern_flag |= MNTK_UPPER_WAITER; + msleep(&mp->mnt_uppers, MNT_MTX(mp), 0, "mntupw", 0); + } +} + +/* + * Undo a previous call to vfs_register_for_notification(). + * The mount represented by upper must be currently registered + * as an upper mount for mp. + */ +void +vfs_unregister_for_notification(struct mount *mp, + struct mount_upper_node *upper) +{ + MNT_ILOCK(mp); + vfs_drain_upper_locked(mp); + TAILQ_REMOVE(&mp->mnt_notify, upper, mnt_upper_link); + MNT_IUNLOCK(mp); +} + +/* + * Undo a previous call to vfs_register_upper_from_vp(). + * This must be done before mp can be unmounted. + */ +void +vfs_unregister_upper(struct mount *mp, struct mount_upper_node *upper) { MNT_ILOCK(mp); - KASSERT(mp->mnt_pinned_count > 0, ("mount pinned count underflow")); KASSERT((mp->mnt_kern_flag & MNTK_UNMOUNT) == 0, - ("mount pinned with pending unmount")); - --mp->mnt_pinned_count; + ("registered upper with pending unmount")); + vfs_drain_upper_locked(mp); + TAILQ_REMOVE(&mp->mnt_uppers, upper, mnt_upper_link); + if ((mp->mnt_kern_flag & MNTK_TASKQUEUE_WAITER) != 0 && + TAILQ_EMPTY(&mp->mnt_uppers)) { + mp->mnt_kern_flag &= ~MNTK_TASKQUEUE_WAITER; + wakeup(&mp->mnt_taskqueue_link); + } MNT_REL(mp); MNT_IUNLOCK(mp); } @@ -600,8 +684,10 @@ vfs_mount_alloc(struct vnode *vp, struct vfsconf *vfsp, const char *fspath, mac_mount_create(cred, mp); #endif arc4rand(&mp->mnt_hashseed, sizeof mp->mnt_hashseed, 0); + mp->mnt_upper_pending = 0; TAILQ_INIT(&mp->mnt_uppers); - mp->mnt_pinned_count = 0; + TAILQ_INIT(&mp->mnt_notify); + mp->mnt_taskqueue_flags = 0; return (mp); } @@ -640,9 +726,9 @@ vfs_mount_destroy(struct mount *mp) vn_printf(vp, "dangling vnode "); panic("unmount: dangling vnode"); } - KASSERT(mp->mnt_pinned_count == 0, - ("mnt_pinned_count = %d", mp->mnt_pinned_count)); + KASSERT(mp->mnt_upper_pending == 0, ("mnt_upper_pending")); KASSERT(TAILQ_EMPTY(&mp->mnt_uppers), ("mnt_uppers")); + KASSERT(TAILQ_EMPTY(&mp->mnt_notify), ("mnt_notify")); if (mp->mnt_nvnodelistsize != 0) panic("vfs_mount_destroy: nonzero nvnodelistsize"); if (mp->mnt_lazyvnodelistsize != 0) @@ -1799,17 +1885,166 @@ vfs_mount_fetch_counter(struct mount *mp, enum mount_counter which) return (sum); } +static bool +deferred_unmount_enqueue(struct mount *mp, uint64_t flags, bool requeue) +{ + bool enqueued; + + enqueued = false; + mtx_lock(&deferred_unmount_lock); + if ((mp->mnt_taskqueue_flags & MNT_DEFERRED) == 0 || requeue) { + mp->mnt_taskqueue_flags = flags | MNT_DEFERRED; + STAILQ_INSERT_TAIL(&deferred_unmount_list, mp, + mnt_taskqueue_link); + enqueued = true; + } + mtx_unlock(&deferred_unmount_lock); + + if (enqueued) { + taskqueue_enqueue(taskqueue_deferred_unmount, + &deferred_unmount_task); + } + + return (enqueued); +} + +/* + * Taskqueue handler for processing async/recursive unmounts + */ +static void +vfs_deferred_unmount(void *argi __unused, int pending __unused) +{ + STAILQ_HEAD(, mount) local_unmounts; + uint64_t flags; + struct mount *mp, *tmp; + bool unmounted; + + STAILQ_INIT(&local_unmounts); + mtx_lock(&deferred_unmount_lock); + STAILQ_CONCAT(&local_unmounts, &deferred_unmount_list); + mtx_unlock(&deferred_unmount_lock); + + STAILQ_FOREACH_SAFE(mp, &local_unmounts, mnt_taskqueue_link, tmp) { + flags = mp->mnt_taskqueue_flags; + KASSERT((flags & MNT_DEFERRED) != 0, + ("taskqueue unmount without MNT_DEFERRED")); + if (dounmount(mp, flags, curthread) != 0) { + MNT_ILOCK(mp); + unmounted = ((mp->mnt_kern_flag & MNTK_REFEXPIRE) != 0); + MNT_IUNLOCK(mp); + if (!unmounted) + deferred_unmount_enqueue(mp, flags, true); + else + vfs_rel(mp); + } + } +} + /* * Do the actual filesystem unmount. */ int -dounmount(struct mount *mp, int flags, struct thread *td) +dounmount(struct mount *mp, uint64_t flags, struct thread *td) { + struct mount_upper_node *upper; struct vnode *coveredvp, *rootvp; int error; uint64_t async_flag; int mnt_gen_r; + KASSERT((flags & MNT_DEFERRED) == 0 || + (flags & (MNT_RECURSE | MNT_FORCE)) == (MNT_RECURSE | MNT_FORCE), + ("MNT_DEFERRED requires MNT_RECURSE | MNT_FORCE")); + + /* + * If the caller has explicitly requested the unmount to be handled by + * the taskqueue and we're not already in taskqueue context, queue + * up the unmount request and exit. This is done prior to any + * credential checks; MNT_DEFERRED should be used only for kernel- + * initiated unmounts and will therefore be processed with the + * (kernel) credentials of the taskqueue thread. Still, callers + * should be sure this is the behavior they want. + */ + if ((flags & MNT_DEFERRED) != 0 && + taskqueue_member(taskqueue_deferred_unmount, curthread) == 0) { + if (!deferred_unmount_enqueue(mp, flags, false)) + vfs_rel(mp); + return (EINPROGRESS); + } + + /* + * Only privileged root, or (if MNT_USER is set) the user that did the + * original mount is permitted to unmount this filesystem. + * This check should be made prior to queueing up any recursive + * unmounts of upper filesystems. Those unmounts will be executed + * with kernel thread credentials and are expected to succeed, so + * we must at least ensure the originating context has sufficient + * privilege to unmount the base filesystem before proceeding with + * the uppers. + */ + error = vfs_suser(mp, td); + if (error != 0) { + KASSERT((flags & MNT_DEFERRED) == 0, + ("taskqueue unmount with insufficient privilege")); + vfs_rel(mp); + return (error); + } + + if (recursive_forced_unmount && ((flags & MNT_FORCE) != 0)) + flags |= MNT_RECURSE; + + if ((flags & MNT_RECURSE) != 0) { + KASSERT((flags & MNT_FORCE) != 0, + ("MNT_RECURSE requires MNT_FORCE")); + + MNT_ILOCK(mp); + /* + * Set MNTK_RECURSE to prevent new upper mounts from being + * added, and note that an operation on the uppers list is in + * progress. This will ensure that unregistration from the + * uppers list, and therefore any pending unmount of the upper + * FS, can't complete until after we finish walking the list. + */ + mp->mnt_kern_flag |= MNTK_RECURSE; + mp->mnt_upper_pending++; + TAILQ_FOREACH(upper, &mp->mnt_uppers, mnt_upper_link) { + MNT_IUNLOCK(mp); + vfs_ref(upper->mp); + if (!deferred_unmount_enqueue(upper->mp, flags, false)) + vfs_rel(upper->mp); + MNT_ILOCK(mp); + } + mp->mnt_upper_pending--; + if ((mp->mnt_kern_flag & MNTK_UPPER_WAITER) != 0 && + mp->mnt_upper_pending == 0) { + mp->mnt_kern_flag &= ~MNTK_UPPER_WAITER; + wakeup(&mp->mnt_uppers); + } + /* + * If we're not on the taskqueue, wait until the uppers list + * is drained before proceeding with unmount. Otherwise, if + * we are on the taskqueue and there are still pending uppers, + * just re-enqueue on the end of the taskqueue. + */ + if ((flags & MNT_DEFERRED) == 0) { + while (!TAILQ_EMPTY(&mp->mnt_uppers)) { + mp->mnt_kern_flag |= MNTK_TASKQUEUE_WAITER; + msleep(&mp->mnt_taskqueue_link, MNT_MTX(mp), 0, + "umntqw", 0); + } + } else if (!TAILQ_EMPTY(&mp->mnt_uppers)) { + MNT_IUNLOCK(mp); + deferred_unmount_enqueue(mp, flags, true); + return (0); + } + MNT_IUNLOCK(mp); + KASSERT(TAILQ_EMPTY(&mp->mnt_uppers), ("mnt_uppers not empty")); + } + + /* Allow the taskqueue to safely re-enqueue on failure */ + if ((flags & MNT_DEFERRED) != 0) + vfs_ref(mp); + if ((coveredvp = mp->mnt_vnodecovered) != NULL) { mnt_gen_r = mp->mnt_gen; VI_LOCK(coveredvp); @@ -1828,27 +2063,13 @@ dounmount(struct mount *mp, int flags, struct thread *td) } } - /* - * Only privileged root, or (if MNT_USER is set) the user that did the - * original mount is permitted to unmount this filesystem. - */ - error = vfs_suser(mp, td); - if (error != 0) { - if (coveredvp != NULL) { - VOP_UNLOCK(coveredvp); - vdrop(coveredvp); - } - vfs_rel(mp); - return (error); - } - vfs_op_enter(mp); vn_start_write(NULL, &mp, V_WAIT | V_MNTREF); MNT_ILOCK(mp); if ((mp->mnt_kern_flag & MNTK_UNMOUNT) != 0 || (mp->mnt_flag & MNT_UPDATE) != 0 || - mp->mnt_pinned_count != 0) { + !TAILQ_EMPTY(&mp->mnt_uppers)) { dounmount_cleanup(mp, coveredvp, 0); return (EBUSY); } @@ -1952,6 +2173,7 @@ dounmount(struct mount *mp, int flags, struct thread *td) } return (error); } + mtx_lock(&mountlist_mtx); TAILQ_REMOVE(&mountlist, mp, mnt_list); mtx_unlock(&mountlist_mtx); @@ -1977,6 +2199,8 @@ dounmount(struct mount *mp, int flags, struct thread *td) } if (mp == rootdevmp) rootdevmp = NULL; + if ((flags & MNT_DEFERRED) != 0) + vfs_rel(mp); vfs_mount_destroy(mp); return (0); } diff --git a/sys/kern/vfs_subr.c b/sys/kern/vfs_subr.c index a2f25bf78495..23f6ec9cf3fd 100644 --- a/sys/kern/vfs_subr.c +++ b/sys/kern/vfs_subr.c @@ -831,9 +831,9 @@ vfs_busy(struct mount *mp, int flags) * valid. */ while (mp->mnt_kern_flag & MNTK_UNMOUNT) { - KASSERT(mp->mnt_pinned_count == 0, - ("%s: non-zero pinned count %d with pending unmount", - __func__, mp->mnt_pinned_count)); + KASSERT(TAILQ_EMPTY(&mp->mnt_uppers), + ("%s: non-empty upper mount list with pending unmount", + __func__)); if (flags & MBF_NOWAIT || mp->mnt_kern_flag & MNTK_REFEXPIRE) { MNT_REL(mp); MNT_IUNLOCK(mp); @@ -3891,61 +3891,44 @@ vgone(struct vnode *vp) VI_UNLOCK(vp); } -static void -notify_lowervp_vfs_dummy(struct mount *mp __unused, - struct vnode *lowervp __unused) -{ -} - /* * Notify upper mounts about reclaimed or unlinked vnode. */ void vfs_notify_upper(struct vnode *vp, int event) { - static struct vfsops vgonel_vfsops = { - .vfs_reclaim_lowervp = notify_lowervp_vfs_dummy, - .vfs_unlink_lowervp = notify_lowervp_vfs_dummy, - }; - struct mount *mp, *ump, *mmp; + struct mount *mp; + struct mount_upper_node *ump; - mp = vp->v_mount; + mp = atomic_load_ptr(&vp->v_mount); if (mp == NULL) return; - if (TAILQ_EMPTY(&mp->mnt_uppers)) + if (TAILQ_EMPTY(&mp->mnt_notify)) return; - mmp = malloc(sizeof(struct mount), M_TEMP, M_WAITOK | M_ZERO); - mmp->mnt_op = &vgonel_vfsops; - mmp->mnt_kern_flag |= MNTK_MARKER; MNT_ILOCK(mp); - mp->mnt_kern_flag |= MNTK_VGONE_UPPER; - for (ump = TAILQ_FIRST(&mp->mnt_uppers); ump != NULL;) { - if ((ump->mnt_kern_flag & MNTK_MARKER) != 0) { - ump = TAILQ_NEXT(ump, mnt_upper_link); - continue; - } - TAILQ_INSERT_AFTER(&mp->mnt_uppers, ump, mmp, mnt_upper_link); + mp->mnt_upper_pending++; + KASSERT(mp->mnt_upper_pending > 0, + ("%s: mnt_upper_pending %d", __func__, mp->mnt_upper_pending)); + TAILQ_FOREACH(ump, &mp->mnt_notify, mnt_upper_link) { MNT_IUNLOCK(mp); switch (event) { case VFS_NOTIFY_UPPER_RECLAIM: - VFS_RECLAIM_LOWERVP(ump, vp); + VFS_RECLAIM_LOWERVP(ump->mp, vp); break; case VFS_NOTIFY_UPPER_UNLINK: - VFS_UNLINK_LOWERVP(ump, vp); + VFS_UNLINK_LOWERVP(ump->mp, vp); break; default: KASSERT(0, ("invalid event %d", event)); break; } MNT_ILOCK(mp); - ump = TAILQ_NEXT(mmp, mnt_upper_link); - TAILQ_REMOVE(&mp->mnt_uppers, mmp, mnt_upper_link); } - free(mmp, M_TEMP); - mp->mnt_kern_flag &= ~MNTK_VGONE_UPPER; - if ((mp->mnt_kern_flag & MNTK_VGONE_WAITER) != 0) { - mp->mnt_kern_flag &= ~MNTK_VGONE_WAITER; + mp->mnt_upper_pending--; + if ((mp->mnt_kern_flag & MNTK_UPPER_WAITER) != 0 && + mp->mnt_upper_pending == 0) { + mp->mnt_kern_flag &= ~MNTK_UPPER_WAITER; wakeup(&mp->mnt_uppers); } MNT_IUNLOCK(mp); @@ -4376,12 +4359,12 @@ DB_SHOW_COMMAND(mount, db_show_mount) MNT_KERN_FLAG(MNTK_EXTENDED_SHARED); MNT_KERN_FLAG(MNTK_SHARED_WRITES); MNT_KERN_FLAG(MNTK_NO_IOPF); - MNT_KERN_FLAG(MNTK_VGONE_UPPER); - MNT_KERN_FLAG(MNTK_VGONE_WAITER); + MNT_KERN_FLAG(MNTK_RECURSE); + MNT_KERN_FLAG(MNTK_UPPER_WAITER); MNT_KERN_FLAG(MNTK_LOOKUP_EXCL_DOTDOT); - MNT_KERN_FLAG(MNTK_MARKER); MNT_KERN_FLAG(MNTK_USES_BCACHE); MNT_KERN_FLAG(MNTK_FPLOOKUP); + MNT_KERN_FLAG(MNTK_TASKQUEUE_WAITER); MNT_KERN_FLAG(MNTK_NOASYNC); MNT_KERN_FLAG(MNTK_UNMOUNT); MNT_KERN_FLAG(MNTK_MWAIT); diff --git a/sys/sys/mount.h b/sys/sys/mount.h index 693293b12370..016c7e9c188d 100644 --- a/sys/sys/mount.h +++ b/sys/sys/mount.h @@ -190,6 +190,19 @@ struct mount_pcpu { _Static_assert(sizeof(struct mount_pcpu) == 16, "the struct is allocated from pcpu 16 zone"); +/* + * Structure for tracking a stacked filesystem mounted above another + * filesystem. This is expected to be stored in the upper FS' per-mount data. + * + * Lock reference: + * i - lower mount interlock + * c - constant from node initialization + */ +struct mount_upper_node { + struct mount *mp; /* (c) mount object for upper FS */ + TAILQ_ENTRY(mount_upper_node) mnt_upper_link; /* (i) position in uppers list */ +}; + /* * Structure per mounted filesystem. Each mounted filesystem has an * array of operations and an instance record. The filesystems are @@ -199,8 +212,8 @@ _Static_assert(sizeof(struct mount_pcpu) == 16, * l - mnt_listmtx * m - mountlist_mtx * i - interlock - * i* - interlock of uppers' list head * v - vnode freelist mutex + * d - deferred unmount list mutex * * Unmarked fields are considered stable as long as a ref is held. * @@ -242,10 +255,12 @@ struct mount { struct mtx mnt_listmtx; struct vnodelst mnt_lazyvnodelist; /* (l) list of lazy vnodes */ int mnt_lazyvnodelistsize; /* (l) # of lazy vnodes */ - int mnt_pinned_count; /* (i) unmount prevented */ + int mnt_upper_pending; /* (i) # of pending ops on mnt_uppers */ struct lock mnt_explock; /* vfs_export walkers lock */ - TAILQ_ENTRY(mount) mnt_upper_link; /* (i*) we in the all uppers */ - TAILQ_HEAD(, mount) mnt_uppers; /* (i) upper mounts over us */ + TAILQ_HEAD(, mount_upper_node) mnt_uppers; /* (i) upper mounts over us */ + TAILQ_HEAD(, mount_upper_node) mnt_notify; /* (i) upper mounts for notification */ + STAILQ_ENTRY(mount) mnt_taskqueue_link; /* (d) our place in deferred unmount list */ + uint64_t mnt_taskqueue_flags; /* (d) unmount flags passed from taskqueue */ }; #endif /* _WANT_MOUNT || _KERNEL */ @@ -438,9 +453,13 @@ struct mntoptnames { #define MNT_BYFSID 0x0000000008000000ULL /* specify filesystem by ID. */ #define MNT_NOCOVER 0x0000001000000000ULL /* Do not cover a mount point */ #define MNT_EMPTYDIR 0x0000002000000000ULL /* Only mount on empty dir */ -#define MNT_CMDFLAGS (MNT_UPDATE | MNT_DELEXPORT | MNT_RELOAD | \ +#define MNT_RECURSE 0x0000100000000000ULL /* recursively unmount uppers */ +#define MNT_DEFERRED 0x0000200000000000ULL /* unmount in async context */ +#define MNT_CMDFLAGS (MNT_UPDATE | MNT_DELEXPORT | MNT_RELOAD | \ MNT_FORCE | MNT_SNAPSHOT | MNT_NONBUSY | \ - MNT_BYFSID | MNT_NOCOVER | MNT_EMPTYDIR) + MNT_BYFSID | MNT_NOCOVER | MNT_EMPTYDIR | \ + MNT_RECURSE | MNT_DEFERRED) + /* * Internal filesystem control flags stored in mnt_kern_flag. * @@ -466,10 +485,9 @@ struct mntoptnames { #define MNTK_NO_IOPF 0x00000100 /* Disallow page faults during reads and writes. Filesystem shall properly handle i/o state on EFAULT. */ -#define MNTK_VGONE_UPPER 0x00000200 -#define MNTK_VGONE_WAITER 0x00000400 +#define MNTK_RECURSE 0x00000200 /* pending recursive unmount */ +#define MNTK_UPPER_WAITER 0x00000400 /* waiting to drain MNTK_UPPER_PENDING */ #define MNTK_LOOKUP_EXCL_DOTDOT 0x00000800 -#define MNTK_MARKER 0x00001000 #define MNTK_UNMAPPED_BUFS 0x00002000 #define MNTK_USES_BCACHE 0x00004000 /* FS uses the buffer cache. */ #define MNTK_TEXT_REFS 0x00008000 /* Keep use ref for text */ @@ -477,8 +495,9 @@ struct mntoptnames { #define MNTK_UNIONFS 0x00020000 /* A hack for F_ISUNIONSTACK */ #define MNTK_FPLOOKUP 0x00040000 /* fast path lookup is supported */ #define MNTK_SUSPEND_ALL 0x00080000 /* Suspended by all-fs suspension */ -#define MNTK_NOASYNC 0x00800000 /* disable async */ -#define MNTK_UNMOUNT 0x01000000 /* unmount in progress */ +#define MNTK_TASKQUEUE_WAITER 0x00100000 /* Waiting on unmount taskqueue */ +#define MNTK_NOASYNC 0x00800000 /* disable async */ +#define MNTK_UNMOUNT 0x01000000 /* unmount in progress */ #define MNTK_MWAIT 0x02000000 /* waiting for unmount to finish */ #define MNTK_SUSPEND 0x08000000 /* request write suspension */ #define MNTK_SUSPEND2 0x04000000 /* block secondary writes */ @@ -952,7 +971,7 @@ vfs_statfs_t __vfs_statfs; * exported vnode operations */ -int dounmount(struct mount *, int, struct thread *); +int dounmount(struct mount *, uint64_t, struct thread *); int kernel_mount(struct mntarg *ma, uint64_t flags); int kernel_vmount(int flags, ...); @@ -1012,8 +1031,13 @@ struct mount *vfs_mount_alloc(struct vnode *, struct vfsconf *, const char *, int vfs_suser(struct mount *, struct thread *); void vfs_unbusy(struct mount *); void vfs_unmountall(void); -struct mount *vfs_pin_from_vp(struct vnode *); -void vfs_unpin(struct mount *); +struct mount *vfs_register_upper_from_vp(struct vnode *, + struct mount *ump, struct mount_upper_node *); +void vfs_register_for_notification(struct mount *, struct mount *, + struct mount_upper_node *); +void vfs_unregister_for_notification(struct mount *, + struct mount_upper_node *); +void vfs_unregister_upper(struct mount *, struct mount_upper_node *); extern TAILQ_HEAD(mntlist, mount) mountlist; /* mounted filesystem list */ extern struct mtx_padalign mountlist_mtx; extern struct nfs_public nfs_pub; diff --git a/sys/ufs/ffs/ffs_vfsops.c b/sys/ufs/ffs/ffs_vfsops.c index 6b7407eb88f9..2ff71cc3e4d1 100644 --- a/sys/ufs/ffs/ffs_vfsops.c +++ b/sys/ufs/ffs/ffs_vfsops.c @@ -281,28 +281,8 @@ ffs_check_blkno(struct mount *mp, ino_t inum, ufs2_daddr_t daddr, int blksize) } /* - * Initiate a forcible unmount. + * On first ENXIO error, initiate an asynchronous forcible unmount. * Used to unmount filesystems whose underlying media has gone away. - */ -static void -ffs_fsfail_unmount(void *v, int pending) -{ - struct fsfail_task *etp; - struct mount *mp; - - etp = v; - - /* - * Find our mount and get a ref on it, then try to unmount. - */ - mp = vfs_getvfs(&etp->fsid); - if (mp != NULL) - dounmount(mp, MNT_FORCE, curthread); - free(etp, M_UFSMNT); -} - -/* - * On first ENXIO error, start a task that forcibly unmounts the filesystem. * * Return true if a cleanup is in progress. */ @@ -320,25 +300,18 @@ ffs_fsfail_cleanup(struct ufsmount *ump, int error) int ffs_fsfail_cleanup_locked(struct ufsmount *ump, int error) { - struct fsfail_task *etp; - struct task *tp; - mtx_assert(UFS_MTX(ump), MA_OWNED); if (error == ENXIO && (ump->um_flags & UM_FSFAIL_CLEANUP) == 0) { ump->um_flags |= UM_FSFAIL_CLEANUP; /* * Queue an async forced unmount. */ - etp = ump->um_fsfail_task; - ump->um_fsfail_task = NULL; - if (etp != NULL) { - tp = &etp->task; - TASK_INIT(tp, 0, ffs_fsfail_unmount, etp); - taskqueue_enqueue(taskqueue_thread, tp); - printf("UFS: forcibly unmounting %s from %s\n", - ump->um_mountp->mnt_stat.f_mntfromname, - ump->um_mountp->mnt_stat.f_mntonname); - } + vfs_ref(ump->um_mountp); + dounmount(ump->um_mountp, + MNT_FORCE | MNT_RECURSE | MNT_DEFERRED, curthread); + printf("UFS: forcibly unmounting %s from %s\n", + ump->um_mountp->mnt_stat.f_mntfromname, + ump->um_mountp->mnt_stat.f_mntonname); } return ((ump->um_flags & UM_FSFAIL_CLEANUP) != 0); } @@ -1046,7 +1019,6 @@ ffs_mountfs(odevvp, mp, td) struct g_consumer *cp; struct mount *nmp; struct vnode *devvp; - struct fsfail_task *etp; int candelete, canspeedup; off_t loc; @@ -1334,9 +1306,6 @@ ffs_mountfs(odevvp, mp, td) (void) ufs_extattr_autostart(mp, td); #endif /* !UFS_EXTATTR_AUTOSTART */ #endif /* !UFS_EXTATTR */ - etp = malloc(sizeof *ump->um_fsfail_task, M_UFSMNT, M_WAITOK | M_ZERO); - etp->fsid = mp->mnt_stat.f_fsid; - ump->um_fsfail_task = etp; return (0); out: if (fs != NULL) { @@ -1583,8 +1552,6 @@ ffs_unmount(mp, mntflags) free(fs->fs_csp, M_UFSMNT); free(fs->fs_si, M_UFSMNT); free(fs, M_UFSMNT); - if (ump->um_fsfail_task != NULL) - free(ump->um_fsfail_task, M_UFSMNT); free(ump, M_UFSMNT); mp->mnt_data = NULL; MNT_ILOCK(mp); diff --git a/sys/ufs/ufs/ufsmount.h b/sys/ufs/ufs/ufsmount.h index 6071faec795c..f2951d74d44c 100644 --- a/sys/ufs/ufs/ufsmount.h +++ b/sys/ufs/ufs/ufsmount.h @@ -67,10 +67,6 @@ struct inodedep; TAILQ_HEAD(inodedeplst, inodedep); LIST_HEAD(bmsafemaphd, bmsafemap); LIST_HEAD(trimlist_hashhead, ffs_blkfree_trim_params); -struct fsfail_task { - struct task task; - fsid_t fsid; -}; #include #include @@ -123,7 +119,6 @@ struct ufsmount { struct taskqueue *um_trim_tq; /* (c) trim request queue */ struct trimlist_hashhead *um_trimhash; /* (i) trimlist hash table */ u_long um_trimlisthashsize; /* (i) trim hash table size-1 */ - struct fsfail_task *um_fsfail_task; /* (i) task for fsfail cleanup*/ /* (c) - below function ptrs */ int (*um_balloc)(struct vnode *, off_t, int, struct ucred *, int, struct buf **); diff --git a/tools/test/stress2/misc/all.exclude b/tools/test/stress2/misc/all.exclude index 9e9226c28574..d82abb3326c4 100644 --- a/tools/test/stress2/misc/all.exclude +++ b/tools/test/stress2/misc/all.exclude @@ -6,7 +6,7 @@ backingstore2.sh panic: 43 vncache entries remaining 20111220 backingstore3.sh g_vfs_done():md6a[WRITE(offset=...)]error = 28 20111230 dd.sh CAM stuck in vmwait 20200116 devfs4.sh Hang seen 20210210 -force4.sh https://people.freebsd.org/~pho/stress/log/log0082.txt 20210328 +Xforce4.sh https://people.freebsd.org/~pho/stress/log/log0082.txt 20210328 fsync.sh panic: Journal overflow 20190208 fuse.sh Memory corruption seen in log file kostik734.txt 20141114 fuse2.sh Deadlock seen 20121129 -- Test scenario: misc/force4.sh