vmcore.373 KDB: debugger backends: ddb KDB: current backend: ddb ---<>--- Copyright (c) 1992-2021 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 14.0-CURRENT #1 main-n248133-616a676a0535-dirty: Wed Jul 21 18:31:15 CEST 2021 pho@mercat1.netperf.freebsd.org:/usr/src/sys/amd64/compile/PHO amd64 FreeBSD clang version 12.0.1 (git@github.com:llvm/llvm-project.git llvmorg-12.0.1-rc2-0-ge7dac564cd0e) WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. VT(vga): resolution 640x480 CPU: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz (3500.07-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306f2 Family=0x6 Model=0x3f Stepping=2 Features=0xbfebfbff Features2=0x7ffefbff AMD Features=0x2c100800 AMD Features2=0x21 Structured Extended Features=0x37ab XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics real memory = 34359738368 (32768 MB) avail memory = 33174138880 (31637 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs FreeBSD/SMP: 1 package(s) x 6 core(s) x 2 hardware threads random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" random: unblocking device. ioapic0 irqs 0-23 ioapic1 irqs 24-47 Launching APs: 1 7 9 10 2 11 6 4 5 3 8 Timecounter "TSC-low" frequency 1750036164 Hz quality 1000 random: entropy device external interface kbd1 at kbdmux0 mlx5en: Mellanox Ethernet driver 3.6.0 (December 2020) vtvga0: smbios0: at iomem 0xf0560-0xf057e smbios0: Version: 2.8, BCD Revision: 2.7 aesni0: acpi0: acpi0: Power Button (fixed) cpu0: on acpi0 atrtc0: port 0x70-0x71,0x74-0x77 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Event timer "HPET4" frequency 14318180 Hz quality 340 Event timer "HPET5" frequency 14318180 Hz quality 340 Event timer "HPET6" frequency 14318180 Hz quality 340 Event timer "HPET7" frequency 14318180 Hz quality 340 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: on acpi0 pci0: on pcib0 pci0: at device 11.1 (no driver attached) pci0: at device 11.2 (no driver attached) pci0: at device 16.1 (no driver attached) pci0: at device 16.6 (no driver attached) pci0: at device 18.1 (no driver attached) acpi_syscontainer0: on acpi0 acpi_syscontainer1: on acpi0 acpi_syscontainer2: on acpi0 acpi_syscontainer3: on acpi0 apei0: on acpi0 pcib1: port 0xcf8-0xcff on acpi0 pci1: on pcib1 pcib2: irq 26 at device 1.0 on pci1 pci2: on pcib2 pcib3: irq 32 at device 2.0 on pci1 pci3: on pcib3 pci3: at device 0.0 (no driver attached) pci3: at device 0.1 (no driver attached) pcib4: irq 40 at device 3.0 on pci1 pci4: on pcib4 pci1: at device 17.0 (no driver attached) ahci0: port 0xf110-0xf117,0xf100-0xf103,0xf0f0-0xf0f7,0xf0e0-0xf0e3,0xf020-0xf03f mem 0xfb438000-0xfb4387ff irq 16 at device 17.4 on pci1 ahci0: AHCI v1.30 with 4 6Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahciem0: on ahci0 xhci0: mem 0xfb400000-0xfb40ffff irq 19 at device 20.0 on pci1 xhci0: 32 bytes context size, 64-bit DMA usbus0 on xhci0 usbus0: 5.0Gbps Super Speed USB v3.0 pci1: at device 22.0 (no driver attached) pci1: at device 22.1 (no driver attached) ehci0: mem 0xfb434000-0xfb4343ff irq 18 at device 26.0 on pci1 usbus1: EHCI version 1.0 usbus1 on ehci0 usbus1: 480Mbps High Speed USB v2.0 pcib5: irq 16 at device 28.0 on pci1 pci5: on pcib5 pcib6: irq 18 at device 28.2 on pci1 pci6: on pcib6 pcib7: at device 0.0 on pci6 pci7: on pcib7 vgapci0: port 0xe000-0xe07f mem 0xfa000000-0xfaffffff,0xfb000000-0xfb01ffff irq 18 at device 0.0 on pci7 vgapci0: Boot video device pcib8: irq 16 at device 28.4 on pci1 pci8: on pcib8 igb0: port 0xd020-0xd03f mem 0xfb320000-0xfb33ffff,0xfb344000-0xfb347fff irq 16 at device 0.0 on pci8 igb0: Using 1024 TX descriptors and 1024 RX descriptors igb0: queue equality override not set, capping rx_queues at 6 and tx_queues at 6 igb0: Using 6 RX queues 6 TX queues igb0: Using MSI-X interrupts uhub5 on uhub0 uhub5: on usbus2 Setting hostuuid: 00000000-0000-0000-0000-002590c99976. Setting hostid: 0x90b83241. uhub3: 4 ports with 3 removable, self powered Starting file system checks: uhub4: 6 ports with 6 removable, self powered WARNING: / was not properly dismounted uhub5: 8 ports with 8 removable, self powered Mounting local filesystems:. ugen0.3: at usbus0 ukbd0 on uhub3 ukbd0: on usbus0 kbd2 at ukbd0 fscklock order reversal: 1st 0xfffff800061e7368 ufs (ufs, lockmgr) @ kern/vfs_mount.c:2052 2nd 0xfffff8001ce72958 devfs (devfs, lockmgr) @ kern/vfs_subr.c:3254 lock order devfs -> ufs established at: #0 0xffffffff80c92149 at witness_checkorder+0x459 #1 0xffffffff80bf2178 at lockmgr_lock_flags+0x178 #2 0xffffffff80f4e30d at ffs_lock+0x9d #3 0xffffffff811b413c at VOP_LOCK1_APV+0x3c #4 0xffffffff80d283d4 at _vn_lock+0x54 #5 0xffffffff80d05c6b at vfs_domount+0xf5b #6 0xffffffff80d03810 at vfs_donmount+0x880 #7 0xffffffff80d08c17 at kernel_mount+0x57 #8 0xffffffff80d0b7e1 at parse_mount+0x4b1 #9 0xffffffff80d09ec3 at vfs_mountroot+0x7b3 #10 0xffffffff80bae96f at start_init+0x1f #11 0xffffffff80bdbba0 at fork_exit+0x80 #12 0xffffffff810bb4fe at fork_trampoline+0xe lock order ufs -> devfs attempted at: #0 0xffffffff80c92a7f at witness_checkorder+0xd8f #1 0xffffffff80bf2178 at lockmgr_lock_flags+0x178 #2 0xffffffff811b413c at VOP_LOCK1_APV+0x3c #3 0xffffffff80d283d4 at _vn_lock+0x54 #4 0xffffffff80d107f2 at vput_final+0x172 #5 0xffffffff80f4bf6d at ffs_unmount+0x38d #6 0xffffffff80d07102 at dounmount+0x812 #7 0xffffffff80d0687d at kern_unmount+0x2fd #8 0xffffffff810ea9c7 at amd64_syscall+0x147 #9 0xffffffff810bad9e at fast_syscall_common+0xf8 ** /dev/gpt/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 112 files, 220 used, 1265187 free (67 frags, 158140 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** Updating CPU Microcode... CPU: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz (3500.07-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306f2 Family=0x6 Model=0x3f Stepping=2 Features=0xbfebfbff Features2=0x7ffefbff AMD Features=0x2c100800 AMD Features2=0x21 Structured Extended Features=0x37ab Structured Extended Features3=0x9c000400 XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics Done. ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/lib/compat /usr/local/lib/compat/pkg /usr/local/lib/e2fsprogs /usr/local/lib/gcc10 /usr/local/lib/gcc8 /usr/local/lib/gcc9 /usr/local/lib/perl5/5.32/mach/CORE 32-bit compatibility ldconfig path: /usr/lib32 /usr/local/lib32/compat Setting hostname: mercat1.netperf.freebsd.org. Setting up harvesting: PURE_RDRAND,[UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,[NET_ETHER],NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED Feeding entropy: . Autoloading module: acpi_wmi.ko Autoloading module: ichsmb.ko Autoloading module: ioat.ko ioat0: mem 0xfb42c000-0xfb42ffff irq 31 at device 4.0 on pci1 ioat0: Capabilities: 2f7 ioat1: mem 0xfb428000-0xfb42bfff irq 39 at device 4.1 on pci1 ioat1: Capabilities: 2f7 ioat2: mem 0xfb424000-0xfb427fff irq 31 at device 4.2 on pci1 ioat2: Capabilities: f7 ioat3: mem 0xfb420000-0xfb423fff irq 39 at device 4.3 on pci1 ioat3: Capabilities: f7 ioat4: mem 0xfb41c000-0xfb41ffff irq 31 at device 4.4 on pci1 ioat4: Capabilities: f7 ioat5: mem 0xfb418000-0xfb41bfff irq 39 at device 4.5 on pci1 ioat5: Capabilities: f7 ioat6: mem 0xfb414000-0xfb417fff irq 31 at device 4.6 on pci1 ioat6: Capabilities: f7 ioat7: mem 0xfb410000-0xfb413fff irq 39 at device 4.7 on pci1 ioat7: Capabilities: f7 ichsmb0: port 0x580-0x59f mem 0xfb431000-0xfb4310ff irq 18 at device 31.3 on pci1 smbus0: on ichsmb0 acpi_wmi0: on acpi0 acpi_wmi0: cannot find EC device lo0: link state changed to UP Autoloading module: acpi_wmi.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Starting Network: igb1. igb1: flags=8822 metric 0 mtu 1500 options=4e507bb ether 0c:c4:7a:a8:cd:db media: Ethernet autoselect status: no carrier nd6 options=29 Autoloading module: ichsmb.ko Starting Network: mce0. mce0: flags=8822 metric 0 mtu 1500 options=7eed07bb ether 7c:fe:90:30:ee:38 media: Ethernet autoselect status: no carrier nd6 options=29 metric 0 mtu 1500 options=7eed07bb ether 7c:fe:90:30:ee:39 media: Ethernet autoselect status: no carrier nd6 options=29 Autoloading module: uhid.ko Autoloading module: ums.ko Autoloading module: usbhid.ko ums0 on uhub3 ums0: on usbus0 ums0: 3 buttons and [Z] coordinates ID=0 Starting dhclient. DHCPREQUEST on igb0 to 255.255.255.255 port 67 DHCPACK from 192if_delmulti_locked: detaching ifnet instance 0xfffff80004393800 .168.5.1 bound to 192.168.5.171 -- renewal in 300 seconds. add host 127.0.0.1: gateway lo0 fib 0: route already in table add host ::1: gateway lo0 fib 0: route already in table add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Creating and/or trimming log files. Recovering vi editor sessions:. Updating motd:. Clearing /tmp (X related). Starting syslogd. Starting watchdogd. No core dumps found. Setting date via ntp. 21 Jul 18:35:53 ntpdate[2615]: step time server 192.168.5.1 offset +0.482633 sec Starting rpcbind. NFS access cache time=60 Starting statd. Starting lockd. Starting powerd. Security policy loaded: MAC/ntpd (mac_ntpd) Starting ntpd. Mounting late filesystems:. Starting inetd. Configuring vt: blanktime. Performing sanity check on sshd configuration. Starting sshd. Starting cron. sysctl: unknown oid 'debug.fail_point.pmap_allocpte_page_alloc' at line 12 Wed Jul 21 18:35 FreeBSD/amd64 (mercat1.netperf.freebsd.org) (ttyu1) login: Jul 21 18:35:57 mercat1 su[2767]: pho to root on /dev/pts/0 20210721 18:36:12 all (1/1): force4.sh Jul 21 18:36:26 mercat1 kernel: pid 4212 (rw), uid 2007 inumber 72228 on /mnt: filesystem full UFS: forcibly unmounting /dev/md10 from /mnt 20210721 18:36:48 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 18:37:57 all (1/1): force4.sh Jul 21 18:38:42 mercat1 kernel: pid 7204 (creat), uid 2007 inumber 65815 on /mnt: out of inodes UFS: forcibly unmounting /dev/md10 from /mnt stress2: freework leak: 998/1006. stress2: newblk leak: 10711/10716. stress2: mount leak: 13/44 stress2: GEOM leak: 4/77 20210721 18:38:55 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 18:39:49 all (1/1): force4.sh g_vfs_done():md10[WRITE(offset=8585216, length=4096)]error = 6 UFS: forcibly unmounting /dev/md10 from /mnt 20210721 18:40:55 all (1/1): force4.sh Jul 21 18:41:06 mercat1 kernel: pid 14372 (mkdir), uid 2007 inumber 117790 on /mnt: filesystem full UFS: forcibly unmounting /dev/md10 from /mnt 20210721 18:41:32 all (1/1): force4.sh Jul 21 18:41:47 mercat1 kernel: pid 18209 (rw), uid 2007 inumber 65462 on /mnt: filesystem full Jul 21 18:41:57 mercat1 kernel: pid 18152 (rw), uid 2007 inumber 75631 on /mnt: filesystem full UFS: forcibly unmounting /dev/md10 from /mnt 20210721 18:42:08 all (1/1): force4.sh Jul 21 18:42:29 mercat1 kernel: pid 19749 (mmap), uid 2007 inumber 32907 on /mnt: out of inodes UFS: forcibly unmounting /dev/md10 from /mnt 20210721 18:43:05 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 18:44:04 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 18:44:43 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 18:45:35 all (1/1): force4.sh Jul 21 18:45:47 mercat1 kernel: pid 28679 (rw), uid 2007 inumber 73879 on /mnt: filesystem full Jul 21 18:45:51 mercat1 kernel: pid 28763 (mmap), uid 2007 inumber 16099 on /mnt: filesystem full UFS: forcibly unmounting /dev/md10 from /mnt 20210721 18:46:26 all (1/1): force4.sh Jul 21 18:47:02 mercat1 kernel: pid 30578 (openat), uid 2007 inumber 32905 on /mnt: out of inodes UFS: forcibly unmounting /dev/md10 from /mnt 20210721 18:47:34 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 18:48:22 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 18:48:52 all (1/1): force4.sh Jul 21 18:49:07 mercat1 kernel: pid 35644 (rw), uid 2007 inumber 48055 on /mnt: filesystem full UFS: forcibly unmounting /dev/md10 from /mnt 20210721 18:49:43 all (1/1): force4.sh Jul 21 18:50:26 mercat1 kernel: pid 37549 (openat), uid 2007 inumber 32956 on /mnt: out of inodes UFS: forcibly unmounting /dev/md10 from /mnt 20210721 18:50:46 all (1/1): force4.sh Jul 21 18:50:54 mercat1 kernel: pid 39293 (dirnprename), uid 2007 inumber 43382 on /mnt: filesystem full Jul 21 18:51:06 mercat1 kernel: pid 39215 (mmap), uid 2007 inumber 50248 on /mnt: filesystem full Jul 21 18:51:15 mercat1 kernel: pid 39224 (mmap), uid 2007 inumber 43342 on /mnt: filesystem full Jul 21 18:51:27 mercat1 kernel: pid 39237 (rw), uid 2007 inumber 57244 on /mnt: filesystem full UFS: forcibly unmounting /dev/md10 from /mnt 20210721 18:51:38 all (1/1): force4.sh Jul 21 18:51:55 mercat1 kernel: pid 42432 (rw), uid 2007 inumber 38675 on /mnt: filesystem full UFS: forcibly unmounting /dev/md10 from /mnt 20210721 18:52:17 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 18:53:16 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 18:54:24 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt stress2: mount leak: 13/44 stress2: GEOM leak: 4/77 20210721 18:55:03 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt stress2: freework leak: 255/263. 20210721 18:55:26 all (1/1): force4.sh Jul 21 18:55:39 mercat1 kernel: pid 53438 (mmap), uid 2007 inumber 72790 on /mnt: filesystem full Jul 21 18:55:52 mercat1 kernel: pid 53397 (symlink), uid 2007 inumber 33127 on /mnt: out of inodes UFS: forcibly unmounting /dev/md10 from /mnt 20210721 18:56:01 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 18:57:10 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 18:58:06 all (1/1): force4.sh Jul 21 18:58:19 mercat1 kernel: pid 59426 (rw), uid 2007 inumber 44831 on /mnt: filesystem full Jul 21 18:58:23 mercat1 kernel: pid 59422 (rw), uid 2007 inumber 50411 on /mnt: filesystem full UFS: forcibly unmounting /dev/md10 from /mnt 20210721 18:58:44 all (1/1): force4.sh Jul 21 18:59:00 mercat1 kernel: pid 60909 (rw), uid 2007 inumber 87019 on /mnt: filesystem full Jul 21 18:59:06 mercat1 kernel: pid 60945 (creat), uid 2007 inumber 33275 on /mnt: out of inodes UFS: forcibly unmounting /dev/md10 from /mnt stress2: freework leak: 121/129. stress2: newblk leak: 1404/1411. stress2: mount leak: 13/44 stress2: GEOM leak: 4/77 20210721 18:59:37 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 19:00:52 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 19:01:46 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 19:02:51 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 19:03:48 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 19:04:24 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 19:05:24 all (1/1): force4.sh Jul 21 19:05:32 mercat1 kernel: pid 74724 (rw), uid 2007 inumber 52517 on /mnt: filesystem full Jul 21 19:05:34 mercat1 kernel: pid 74736 (creat), uid 2007 inumber 32914 on /mnt: filesystem full Jul 21 19:05:42 mercat1 kernel: pid 74707 (rw), uid 2007 inumber 57506 on /mnt: filesystem full Jul 21 19:05:49 mercat1 kernel: pid 74765 (creat), uid 2007 inumber 32925 on /mnt: filesystem full UFS: forcibly unmounting /dev/md10 from /mnt stress2: freework leak: 243/251. stress2: newblk leak: 13260/13265. stress2: mount leak: 13/44 stress2: GEOM leak: 4/77 20210721 19:05:53 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 19:06:30 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 19:07:40 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 19:08:18 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 19:08:49 all (1/1): force4.sh Jul 21 19:08:58 mercat1 kernel: pid 86921 (mmap), uid 2007 inumber 43718 on /mnt: filesystem full Jul 21 19:09:02 mercat1 kernel: pid 86856 (mmap), uid 2007 inumber 41937 on /mnt: filesystem full UFS: forcibly unmounting /dev/md10 from /mnt 20210721 19:09:32 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 19:10:14 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 19:11:02 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 19:12:02 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 19:13:05 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 19:13:56 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 19:15:08 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 19:16:14 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 19:16:56 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 19:17:54 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 19:18:46 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 19:19:32 all (1/1): force4.sh Jul 21 19:19:43 mercat1 kernel: pid 9819 (creat), uid 2007 inumber 33086 on /mnt: out of inodes Jul 21 19:19:55 mercat1 kernel: pid 9854 (mkdir), uid 2007 inumber 89076 on /mnt: filesystem full UFS: forcibly unmounting /dev/md10 from /mnt 20210721 19:20:11 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 19:20:53 all (1/1): force4.sh g_vfs_done():md10[WRITE(offset=8585216, length=4096)]error = 6 UFS: forcibly unmounting /dev/md10 from /mnt stress2: pts leak: 1/2 20210721 19:21:52 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 19:22:59 all (1/1): force4.sh UFS: forcibly unmounting /dev/md10 from /mnt 20210721 19:23:46 all (1/1): force4.sh Jul 21 19:23:55 mercat1 kernel: pid 20046 (rw), uid 2007 inumber 47488 on /mnt: filesystem full Jul 21 19:24:01 mercat1 kernel: pid 20040 (mmap), uid 2007 inumber 39945 on /mnt: filesystem full Jul 21 19:24:39 mercat1 kernel: pid 20062 (openat), uid 2007 inumber 32907 on /mnt: out of inodes UFS: forcibly unmounting /dev/md10 from /mnt 20210721 19:24:46 all (1/1): force4.sh Jul 21 19:25:04 mercat1 su[22873]: pho to root on /dev/pts/1 UFS: forcibly unmounting /dev/md10 from /mnt stress2: mount leak: 13/44 stress2: GEOM leak: 4/77 root Jul 21 20:03:41 mercat1 login[2756]: ROOT LOGIN (root) ON ttyu1 Last login: Sun Jul 18 05:21:29 on ttyu1 FreeBSD 14.0-CURRENT (PHO) #1 main-n248133-616a676a0535-dirty: Wed Jul 21 18:31:15 CEST 2021 root@mercat1:~ # ps -lUroot | grep ufs 0 26527 26526 10 -52 0 13424 2984 ufs D< - 0:00.00 ls -l /tmp /dev /mnt /mnt2 /media 0 68558 68512 1 20 0 12936 2628 piperd S+ u1 0:00.00 grep ufs 0 24211 1 8 20 0 13424 2984 ufs D 0 0:12.28 ls -d /mnt /mnt2 0 47808 1 6 20 0 13424 2984 ufs D 0 0:00.00 ls -d /mnt /mnt2 0 60494 2769 2 20 0 13424 2984 ufs D+ 0 0:00.00 ls -l /mnt root@mercat1:~ # mount /dev/gpt/work on / (ufs, local, soft-updates) devfs on /dev (devfs) /dev/gpt/tmp on /tmp (ufs, local, soft-updates) root@mercat1:~ # KDB: enter: Break to debugger [ thread pid 11 tid 100003 ] Stopped at kdb_alt_break_internal+0x101: movq $0,0x1279aa4(%rip) db> run pho db:0:pho> set $lines 20000 db:0:pho> run pho1 db:1:pho1> dump Dumping 2153 out of 32624 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Dump complete db:1:pho1> bt Tracing pid 11 tid 100003 td 0xfffffe00e4c5b3a0 kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe0037790890 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00377908a0 uart_intr_rxready() at uart_intr_rxready+0xf5/frame 0xfffffe00377908d0 uart_intr() at uart_intr+0x128/frame 0xfffffe0037790910 intr_event_handle() at intr_event_handle+0xde/frame 0xfffffe0037790960 intr_execute_handlers() at intr_execute_handlers+0x66/frame 0xfffffe0037790990 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe0037790990 --- interrupt, rip = 0xffffffff810a50a6, rsp = 0xfffffe0037790a60, rbp = 0xfffffe0037790a60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0037790a60 acpi_cpu_idle() at acpi_cpu_idle+0x288/frame 0xfffffe0037790aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe0037790ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe0037790ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe0037790bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037790bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037790bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db:1:pho1> show allpcpu Current CPU: 0 cpuid = 0 dynamic pcpu = 0x9b4bc0 curthread = 0xfffffe00e4c5b3a0: pid 11 tid 100003 critnest 2 "idle: cpu0" curpcb = 0xfffffe00e4c5b8b0 fpcurthread = none idlethread = 0xfffffe00e4c5b3a0: tid 100003 "idle: cpu0" self = 0xffffffff82610000 curpmap = 0xffffffff81e9cc50 tssp = 0xffffffff82610384 rsp0 = 0xfffffe0037790cc0 kcr3 = 0x8000000002555002 ucr3 = 0xffffffffffffffff scr3 = 0x14ddbf4f gs32p = 0xffffffff82610404 ldt = 0xffffffff82610444 tss = 0xffffffff82610434 curvnet = 0 spin locks held: cpuid = 1 dynamic pcpu = 0xfffffe00bb288bc0 curthread = 0xfffffe00e4c5ac80: pid 11 tid 100004 critnest 1 "idle: cpu1" curpcb = 0xfffffe00e4c5b190 fpcurthread = none idlethread = 0xfffffe00e4c5ac80: tid 100004 "idle: cpu1" self = 0xffffffff82611000 curpmap = 0xffffffff81e9cc50 tssp = 0xffffffff82611384 rsp0 = 0xfffffe003778bcc0 kcr3 = 0x8000000002555002 ucr3 = 0xffffffffffffffff scr3 = 0x17714bc10 gs32p = 0xffffffff82611404 ldt = 0xffffffff82611444 tss = 0xffffffff82611434 curvnet = 0 spin locks held: cpuid = 2 dynamic pcpu = 0xfffffe00bb295bc0 curthread = 0xfffffe00e4c5a560: pid 11 tid 100005 critnest 1 "idle: cpu2" curpcb = 0xfffffe00e4c5aa70 fpcurthread = none idlethread = 0xfffffe00e4c5a560: tid 100005 "idle: cpu2" self = 0xffffffff82612000 curpmap = 0xffffffff81e9cc50 tssp = 0xffffffff82612384 rsp0 = 0xfffffe003779acc0 kcr3 = 0x8000000002555002 ucr3 = 0xffffffffffffffff scr3 = 0x227e34dbc gs32p = 0xffffffff82612404 ldt = 0xffffffff82612444 tss = 0xffffffff82612434 curvnet = 0 spin locks held: cpuid = 3 dynamic pcpu = 0xfffffe00bb2a2bc0 curthread = 0xfffffe00e4c59e40: pid 11 tid 100006 critnest 1 "idle: cpu3" curpcb = 0xfffffe00e4c5a350 fpcurthread = none idlethread = 0xfffffe00e4c59e40: tid 100006 "idle: cpu3" self = 0xffffffff82613000 curpmap = 0xffffffff81e9cc50 tssp = 0xffffffff82613384 rsp0 = 0xfffffe0037795cc0 kcr3 = 0x8000000002555002 ucr3 = 0xffffffffffffffff scr3 = 0x22d1ff8b9 gs32p = 0xffffffff82613404 ldt = 0xffffffff82613444 tss = 0xffffffff82613434 curvnet = 0 spin locks held: cpuid = 4 dynamic pcpu = 0xfffffe00bb2afbc0 curthread = 0xfffffe00e4ce5900: pid 0 tid 100044 critnest 0 "deferred_unmount ta" curpcb = 0xfffffe00e4ce5e10 fpcurthread = none idlethread = 0xfffffe00e4c59720: tid 100007 "idle: cpu4" self = 0xffffffff82614000 curpmap = 0xffffffff81e9cc50 tssp = 0xffffffff82614384 rsp0 = 0xfffffe00e4880cc0 kcr3 = 0x8000000002555002 ucr3 = 0xffffffffffffffff scr3 = 0x111ea69e6 gs32p = 0xffffffff82614404 ldt = 0xffffffff82614444 tss = 0xffffffff82614434 curvnet = 0 spin locks held: cpuid = 5 dynamic pcpu = 0xfffffe00bb2bcbc0 curthread = 0xfffffe00e4c59000: pid 11 tid 100008 critnest 1 "idle: cpu5" curpcb = 0xfffffe00e4c59510 fpcurthread = none idlethread = 0xfffffe00e4c59000: tid 100008 "idle: cpu5" self = 0xffffffff82615000 curpmap = 0xffffffff81e9cc50 tssp = 0xffffffff82615384 rsp0 = 0xfffffe003779fcc0 kcr3 = 0x8000000002555002 ucr3 = 0xffffffffffffffff scr3 = 0x3823e8aa1 gs32p = 0xffffffff82615404 ldt = 0xffffffff82615444 tss = 0xffffffff82615434 curvnet = 0 spin locks held: cpuid = 6 dynamic pcpu = 0xfffffe00bb2c9bc0 curthread = 0xfffffe00e4c5d740: pid 11 tid 100009 critnest 1 "idle: cpu6" curpcb = 0xfffffe00e4c5dc50 fpcurthread = none idlethread = 0xfffffe00e4c5d740: tid 100009 "idle: cpu6" self = 0xffffffff82616000 curpmap = 0xffffffff81e9cc50 tssp = 0xffffffff82616384 rsp0 = 0xfffffe00377aecc0 kcr3 = 0x8000000002555002 ucr3 = 0xffffffffffffffff scr3 = 0x1d2d7b9e8 gs32p = 0xffffffff82616404 ldt = 0xffffffff82616444 tss = 0xffffffff82616434 curvnet = 0 spin locks held: cpuid = 7 dynamic pcpu = 0xfffffe00bb2d6bc0 curthread = 0xfffffe00e4c5d020: pid 11 tid 100010 critnest 1 "idle: cpu7" curpcb = 0xfffffe00e4c5d530 fpcurthread = none idlethread = 0xfffffe00e4c5d020: tid 100010 "idle: cpu7" self = 0xffffffff82617000 curpmap = 0xffffffff81e9cc50 tssp = 0xffffffff82617384 rsp0 = 0xfffffe00377a9cc0 kcr3 = 0x8000000002555002 ucr3 = 0xffffffffffffffff scr3 = 0x43ebeaac2 gs32p = 0xffffffff82617404 ldt = 0xffffffff82617444 tss = 0xffffffff82617434 curvnet = 0 spin locks held: cpuid = 8 dynamic pcpu = 0xfffffe00bb2e3bc0 curthread = 0xfffffe00e4c5c900: pid 11 tid 100011 critnest 1 "idle: cpu8" curpcb = 0xfffffe00e4c5ce10 fpcurthread = none idlethread = 0xfffffe00e4c5c900: tid 100011 "idle: cpu8" self = 0xffffffff82618000 curpmap = 0xffffffff81e9cc50 tssp = 0xffffffff82618384 rsp0 = 0xfffffe00377b8cc0 kcr3 = 0x8000000002555002 ucr3 = 0xffffffffffffffff scr3 = 0x731de9b81 gs32p = 0xffffffff82618404 ldt = 0xffffffff82618444 tss = 0xffffffff82618434 curvnet = 0 spin locks held: cpuid = 9 dynamic pcpu = 0xfffffe00bb2f0bc0 curthread = 0xfffffe00e4c611e0: pid 11 tid 100012 critnest 1 "idle: cpu9" curpcb = 0xfffffe00e4c616f0 fpcurthread = none idlethread = 0xfffffe00e4c611e0: tid 100012 "idle: cpu9" self = 0xffffffff82619000 curpmap = 0xffffffff81e9cc50 tssp = 0xffffffff82619384 rsp0 = 0xfffffe00377b3cc0 kcr3 = 0x8000000002555002 ucr3 = 0xffffffffffffffff scr3 = 0x14ddbf39 gs32p = 0xffffffff82619404 ldt = 0xffffffff82619444 tss = 0xffffffff82619434 curvnet = 0 spin locks held: cpuid = 10 dynamic pcpu = 0xfffffe00bb2fdbc0 curthread = 0xfffffe00e4c60ac0: pid 11 tid 100013 critnest 1 "idle: cpu10" curpcb = 0xfffffe00e4c60fd0 fpcurthread = none idlethread = 0xfffffe00e4c60ac0: tid 100013 "idle: cpu10" self = 0xffffffff8261a000 curpmap = 0xffffffff81e9cc50 tssp = 0xffffffff8261a384 rsp0 = 0xfffffe00377c2cc0 kcr3 = 0x8000000002555002 ucr3 = 0xffffffffffffffff scr3 = 0x4df8e6deb gs32p = 0xffffffff8261a404 ldt = 0xffffffff8261a444 tss = 0xffffffff8261a434 curvnet = 0 spin locks held: cpuid = 11 dynamic pcpu = 0xfffffe00bb30abc0 curthread = 0xfffffe00e4c603a0: pid 11 tid 100014 critnest 1 "idle: cpu11" curpcb = 0xfffffe00e4c608b0 fpcurthread = none idlethread = 0xfffffe00e4c603a0: tid 100014 "idle: cpu11" self = 0xffffffff8261b000 curpmap = 0xffffffff81e9cc50 tssp = 0xffffffff8261b384 rsp0 = 0xfffffe00377bdcc0 kcr3 = 0x8000000002555002 ucr3 = 0xffffffffffffffff scr3 = 0x81f00f890 gs32p = 0xffffffff8261b404 ldt = 0xffffffff8261b444 tss = 0xffffffff8261b434 curvnet = 0 spin locks held: db:1:pho1> show alllocks Process 60494 (ls) thread 0xfffffe010e14d000 (100306) shared lockmgr ufs (ufs) r = 0 (0xfffff80003bcfc50) locked @ kern/vfs_lookup.c:903 Process 47808 (ls) thread 0xfffffe01471bc3a0 (100493) shared lockmgr ufs (ufs) r = 0 (0xfffff80003bcfc50) locked @ kern/vfs_lookup.c:903 Process 26527 (ls) thread 0xfffffe01496c1e40 (100551) shared lockmgr ufs (ufs) r = 0 (0xfffff80003bcfc50) locked @ kern/vfs_lookup.c:903 Process 24211 (ls) thread 0xfffffe0149780ac0 (100557) shared lockmgr ufs (ufs) r = 0 (0xfffff80003bcfc50) locked @ kern/vfs_lookup.c:903 Process 0 (kernel) thread 0xfffffe00e4ce5900 (100044) exclusive sleep mutex vnode interlock (vnode interlock) r = 0 (0xfffff8022ed43d10) locked @ kern/vfs_subr.c:6583 exclusive lockmgr ufs (ufs) r = 0 (0xfffff80111a5ac50) locked @ kern/vfs_mount.c:2052 db:1:pho1> show lockedvnods Locked vnodes vnode 0xfffff80003bcfbe0: type VDIR usecount 41, writecount 0, refcount 4 seqc users 0 mountedhere 0 hold count flags () flags (VV_ROOT|VMP_LAZYLIST) v_object 0xfffff80003bd3d68 ref 0 pages 1 cleanbuf 1 dirtybuf 0 lock type ufs: SHARED (count 4) #0 0xffffffff80bf21e7 at lockmgr_lock_flags+0x1e7 #1 0xffffffff80f4e30d at ffs_lock+0x9d #2 0xffffffff811b413c at VOP_LOCK1_APV+0x3c #3 0xffffffff80d283d4 at _vn_lock+0x54 #4 0xffffffff80d00be2 at lookup+0xc2 #5 0xffffffff80cffde9 at namei+0x379 #6 0xffffffff80d279f7 at vn_open_cred+0x487 #7 0xffffffff80d1e6cc at kern_openat+0x28c #8 0xffffffff810ea9c7 at amd64_syscall+0x147 #9 0xffffffff810bad9e at fast_syscall_common+0xf8 nlink=48, effnlink=48, size=2048, extsize 0 generation=56909bb9, uid=0, gid=0, flags=0x0 ino 2, on dev gpt/work vnode 0xfffff80111a5abe0: type VDIR usecount 5, writecount 0, refcount 2 seqc users 1 mountedhere 0xfffffe0143fe4b80 hold count flags () flags (VIRF_MOUNTPOINT) v_object 0xfffff80006194000 ref 0 pages 0 cleanbuf 0 dirtybuf 0 lock type ufs: EXCL by thread 0xfffffe00e4ce5900 (pid 0, kernel, tid 100044) with shared waiters pending #0 0xffffffff80bf21e7 at lockmgr_lock_flags+0x1e7 #1 0xffffffff80f4e30d at ffs_lock+0x9d #2 0xffffffff811b413c at VOP_LOCK1_APV+0x3c #3 0xffffffff80d283d4 at _vn_lock+0x54 #4 0xffffffff80d06d69 at dounmount+0x479 #5 0xffffffff80d092c9 at vfs_deferred_unmount+0xf9 #6 0xffffffff80c849ca at taskqueue_run_locked+0xaa #7 0xffffffff80c85a52 at taskqueue_thread_loop+0xc2 #8 0xffffffff80bdbba0 at fork_exit+0x80 #9 0xffffffff810bb4fe at fork_trampoline+0xe nlink=2, effnlink=2, size=512, extsize 0 generation=74d5cf41, uid=0, gid=0, flags=0x0 ino 37078272, on dev gpt/work db:1:pho1> show mount 0xfffffe010de94700 /dev/gpt/work on / (ufs) 0xfffffe010de95280 devfs on /dev (devfs) 0xfffffe010e5c9280 /dev/gpt/tmp on /tmp (ufs) 0xfffffe0143fe4b80 /dev/md10 on /mnt (ufs) More info: show mount db:1:pho1> show bufqueues bqempty: 71332 Buf domain 0 freebufs 13155 lofreebufs 219 hifreebufs 329 bufspace 23994368 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 1902 (22798336) dirtyq count 0 (0) wakeup 0 lim 25 CPU 10, 16, 9, 2, 0, 4, 5, 6, 4, 0, 15, 4, Locked buffers: 1 space 32768 Total buffers: 4414 space 23994368 Buf domain 1 freebufs 8027 lofreebufs 219 hifreebufs 329 bufspace 165195776 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 2 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 6978 (163266560) dirtyq count 2 (20480) wakeup 0 lim 25 CPU 3, 20, 12, 0, 17, 3, 7, 6, 19, 10, 18, 11, Locked buffers: 0 space 0 Total buffers: 10450 space 165195776 Buf domain 2 freebufs 12710 lofreebufs 219 hifreebufs 329 bufspace 11190272 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 2 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 2293 (10612736) dirtyq count 2 (32768) wakeup 0 lim 25 CPU 9, 2, 7, 9, 12, 1, 2, 13, 7, 22, 21, 23, Locked buffers: 0 space 0 Total buffers: 3541 space 11190272 Buf domain 3 freebufs 12675 lofreebufs 219 hifreebufs 329 bufspace 12988416 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 2306 (12218368) dirtyq count 0 (0) wakeup 0 lim 25 CPU 4, 12, 7, 17, 24, 19, 4, 19, 11, 16, 8, 11, Locked buffers: 0 space 0 Total buffers: 4474 space 12988416 Buf domain 4 freebufs 13872 lofreebufs 219 hifreebufs 329 bufspace 5382144 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 1116 (4571136) dirtyq count 0 (0) wakeup 0 lim 25 CPU 19, 8, 24, 0, 11, 23, 10, 12, 15, 11, 4, 8, Locked buffers: 0 space 0 Total buffers: 9568 space 5382144 Buf domain 5 freebufs 10911 lofreebufs 219 hifreebufs 329 bufspace 68767744 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 23 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 4147 (66867200) dirtyq count 23 (753664) wakeup 0 lim 25 CPU 1, 3, 0, 6, 1, 1, 5, 2, 0, 0, 16, 17, Locked buffers: 0 space 0 Total buffers: 5488 space 68767744 Buf domain 6 freebufs 14007 lofreebufs 219 hifreebufs 329 bufspace 4784128 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 1 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 1039 (4304896) dirtyq count 1 (4096) wakeup 0 lim 25 CPU 7, 17, 4, 3, 13, 9, 2, 3, 6, 7, 15, 0, Locked buffers: 0 space 0 Total buffers: 3812 space 4784128 db:1:pho1> show page vm_cnt.v_free_count: 7723478 vm_cnt.v_inactive_count: 20947 vm_cnt.v_active_count: 3025 vm_cnt.v_laundry_count: 7 vm_cnt.v_wire_count: 367738 vm_cnt.v_free_reserved: 10643 vm_cnt.v_free_min: 51261 vm_cnt.v_free_target: 173115 vm_cnt.v_inactive_target: 259672 db:1:pho1> show pageq pq_free 7723478 dom 0 page_cnt 8123602 free 7723478 pq_act 3025 pq_inact 20947 pq_laund 7 pq_unsw 0 db:0:pho> show freepages DOMAIN: 0 FREE LIST 0: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 001641 | 000000 11 (008192K) | 000204 | 000001 10 (004096K) | 000221 | 000000 09 (002048K) | 000221 | 000000 08 (001024K) | 000253 | 000001 07 (000512K) | 000270 | 000003 06 (000256K) | 000269 | 000024 05 (000128K) | 000489 | 000090 04 (000064K) | 000908 | 000195 03 (000032K) | 001450 | 000312 02 (000016K) | 000009 | 000583 01 (000008K) | 000020 | 002819 00 (000004K) | 000074 | 007150 FREE LIST 1: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000000 | 000000 10 (004096K) | 000000 | 000000 09 (002048K) | 000000 | 000000 08 (001024K) | 000000 | 000000 07 (000512K) | 000000 | 000000 06 (000256K) | 000001 | 000000 05 (000128K) | 000001 | 000000 04 (000064K) | 000002 | 000000 03 (000032K) | 000000 | 000000 02 (000016K) | 000001 | 000000 01 (000008K) | 000000 | 000000 00 (000004K) | 000000 | 000000 db:0:pho> show uma Zone Size Used Free Requests Sleeps Bucket Total Mem XFree VNODE 760 34153 177677 49698458 0 169 160990800 0 FFS inode 1160 34026 93570 18633103 0 68 148011360 0 malloc-4096 4096 34129 34 129925 0 2 139931648 0 RADIX NODE 144 14025 500668 33312288 0 199 74115792 0 malloc-384 384 491 131789 19960271 0 37 50795520 0 mbuf_cluster 2048 17013 29 17111 0 254 34902016 0 malloc-256 256 737 129058 63824190 0 84 33227520 0 FFS2 dinode 256 34026 93099 18633102 0 63 32544000 0 VM OBJECT 264 22209 93666 21291517 0 40 30591000 0 mbuf_jumbo_page 4096 0 4193 28556561 0 254 17174528 0 malloc-128 128 873 129699 48007844 0 131 16713216 0 malloc-128 128 14306 114530 36192625 0 127 16491008 0 BUF TRIE 144 693 105259 2675088 0 63 15257088 0 256 Bucket 2048 3520 1732 1017099 0 8 10756096 0 malloc-256 256 37 41678 13080041 0 63 10679040 0 S VFS Cache 104 2186 89191 10105433 0 128 9503208 0 malloc-128 128 40465 31052 7363800 0 126 9154176 0 128 Bucket 1024 2565 5078 565031 0 18 7826432 0 malloc-64 64 9714 91464 31695461 0 254 6475392 0 UMA Slabs 0 112 55585 32 55597 0 126 6229104 0 mbuf 256 16439 4261 171795635 0 254 5299200 0 malloc-256 256 14 20041 23403639 0 64 5134080 0 64 Bucket 512 2608 5336 622131 0 30 4067328 0 vmem btag 56 58472 1636 59698 0 254 3366048 0 malloc-128 128 435 17762 3199923 0 126 2329216 0 malloc-128 128 68 17633 786596638 0 126 2265728 0 pbuf 2776 0 764 0 0 0 2120864 0 DIRHASH 1024 569 1479 3328353 0 17 2097152 0 32 Bucket 256 1363 6407 304930 0 62 1989120 0 g_bio 408 0 4670 3418151 0 36 1905360 0 malloc-32768 32768 10 39 4461 0 1 1605632 0 malloc-4096 4096 84 306 370815 0 6 1597440 0 malloc-4096 4096 332 21 2745 0 2 1445888 0 THREAD 1808 676 28 734 0 8 1272832 0 mbuf_packet 256 4 4721 74262218 0 254 1209600 0 malloc-32768 32768 0 34 916 0 1 1114112 0 malloc-65536 65536 3 12 4017 0 1 983040 0 malloc-65536 65536 2 12 942 0 1 917504 0 VMSPACE 2544 48 312 370533 0 8 915840 0 MAP ENTRY 96 2248 7244 21662913 0 128 911232 0 malloc-8192 8192 90 21 148 0 1 909312 0 malloc-64 64 9505 4481 760088 0 254 895104 0 malloc-16384 16384 4 49 5470 0 1 868352 0 malloc-65536 65536 1 12 112 0 1 851968 0 L VFS Cache 320 55 2177 51368 0 30 714240 0 DEVCTL 1024 0 676 1602 0 0 692224 0 PROC 1336 81 396 370624 0 9 637272 0 malloc-65536 65536 4 5 9 0 1 589824 0 NAMEI 1024 4 544 373969465 0 22 561152 0 malloc-32768 32768 2 15 141 0 1 557056 0 filedesc0 1072 82 401 370625 0 9 517776 0 UMA Zones 2048 228 7 228 0 8 481280 0 malloc-8192 8192 29 29 145 0 1 475136 0 16 Bucket 144 218 3030 106022 0 62 467712 0 8 Bucket 80 403 5247 1251596 0 126 452000 0 malloc-32768 32768 13 0 13 0 1 425984 0 malloc-32768 32768 12 0 12 0 1 393216 0 lkpicurr 160 9 2441 9 0 62 392000 0 lkpimm 160 2 2323 2 0 62 372000 0 malloc-32 32 8631 1071 18422 0 254 310464 0 malloc-2048 2048 125 25 474 0 8 307200 0 malloc-256 256 996 159 5858 0 62 295680 0 malloc-16384 16384 14 4 4834 0 1 294912 0 socket 944 57 255 7145 0 254 294528 0 malloc-1024 1024 203 81 5166 0 16 290816 0 malloc-64 64 4149 387 4497 0 254 290304 0 malloc-16384 16384 7 10 28 0 1 278528 0 ksiginfo 112 456 2028 13875 0 126 278208 0 malloc-16 16 15320 1680 15491 0 254 272000 0 malloc-1024 1024 10 230 594154 0 16 245760 0 pipe 744 10 315 209453 0 16 241800 0 malloc-64 64 35 3556 900827203 0 254 229824 0 malloc-512 512 49 391 343997 0 30 225280 0 malloc-8192 8192 0 27 868 0 1 221184 0 malloc-128 128 1 1673 164542 0 126 214272 0 malloc-1024 1024 66 142 380777 0 16 212992 0 malloc-2048 2048 10 90 7540 0 8 204800 0 Files 80 139 2411 37126697 0 126 204000 0 malloc-8192 8192 19 5 3069 0 1 196608 0 malloc-4096 4096 19 24 2804 0 2 176128 0 malloc-384 384 137 263 6211 0 30 153600 0 Mountpoints 2944 4 48 121 0 4 153088 0 PWD 32 21 4389 4801068 0 254 141120 0 clpbuf 2776 0 48 147972 0 2 133248 0 malloc-65536 65536 2 0 2 0 1 131072 0 malloc-2048 2048 7 53 134 0 8 122880 0 ada_ccb 272 0 448 24031 0 30 121856 0 2 Bucket 32 406 3374 77992 0 254 120960 0 malloc-4096 4096 1 28 373 0 2 118784 0 TURNSTILE 136 705 156 705 0 62 117096 0 malloc-256 256 33 357 5135 0 62 99840 0 malloc-128 128 62 713 365019 0 126 99200 0 malloc-8192 8192 12 0 12 0 1 98304 0 malloc-32 32 453 2445 94331 0 254 92736 0 ttyinq 160 345 230 555 0 62 92000 0 udp_inpcb 496 22 162 374 0 254 91264 0 unpcb 256 20 325 6720 0 254 88320 0 malloc-64 64 88 1235 166315 0 254 84672 0 ttyoutq 256 181 149 290 0 62 84480 0 malloc-256 256 184 146 1014 0 62 84480 0 malloc-16384 16384 5 0 5 0 1 81920 0 SLEEPQUEUE 88 705 223 705 0 126 81664 0 malloc-384 384 31 179 1806 0 30 80640 0 UMA Kegs 384 197 6 197 0 30 77952 0 malloc-4096 4096 7 10 27 0 2 69632 0 malloc-65536 65536 1 0 1 0 1 65536 0 malloc-32768 32768 0 2 8 0 1 65536 0 malloc-4096 4096 15 1 26 0 2 65536 0 malloc-1024 1024 3 61 1191 0 16 65536 0 malloc-32 32 613 1403 27075 0 254 64512 0 4 Bucket 48 20 1324 8677 0 254 64512 0 malloc-1024 1024 18 42 724 0 16 61440 0 tcpcb 1080 12 44 22 0 254 60480 0 malloc-512 512 0 112 3183 0 30 57344 0 malloc-512 512 31 73 101 0 30 53248 0 malloc-32 32 139 1499 102513 0 254 52416 0 malloc-16 16 568 2682 1094325 0 254 52000 0 kenv 258 18 177 21840 0 30 50310 0 malloc-16384 16384 0 3 12 0 1 49152 0 malloc-1024 1024 19 29 38 0 16 49152 0 malloc-512 512 1 95 1639 0 30 49152 0 fakepg 104 0 468 932 0 126 48672 0 rl_entry 40 442 770 442 0 254 48480 0 udpcb 32 22 1490 374 0 254 48384 0 malloc-64 64 29 727 9117 0 254 48384 0 malloc-32 32 134 1378 280 0 254 48384 0 malloc-16 16 43 2957 2573 0 254 48000 0 malloc-16 16 37 2963 41195 0 254 48000 0 malloc-16 16 832 2168 45925 0 254 48000 0 malloc-384 384 25 95 10623 0 30 46080 0 malloc-2048 2048 0 22 262 0 8 45056 0 pcpu-64 64 454 250 454 0 254 45056 0 pcpu-8 8 3502 2130 3509 0 254 45056 0 PGRP 88 40 466 256 0 126 44528 0 malloc-32 32 11 1375 793 0 254 44352 0 cpuset 104 27 345 27 0 126 38688 0 malloc-384 384 52 48 52 0 30 38400 0 malloc-1024 1024 16 20 28 0 16 36864 0 pcpu-16 16 52 2252 52 0 254 36864 0 malloc-32 32 24 1110 111 0 254 36288 0 malloc-32768 32768 1 0 1 0 1 32768 0 tcp_inpcb 496 12 52 22 0 254 31744 0 malloc-384 384 13 67 14 0 30 30720 0 rtentry 176 12 149 15 0 62 28336 0 malloc-128 128 43 174 53 0 126 27776 0 routing nhops 256 9 96 15 0 62 26880 0 malloc-16 16 33 1467 47 0 254 24000 0 malloc-4096 4096 4 1 5 0 2 20480 0 malloc-512 512 6 34 64 0 30 20480 0 malloc-16384 16384 1 0 1 0 1 16384 0 malloc-8192 8192 2 0 2 0 1 16384 0 NCLNODE 736 0 22 2 0 16 16192 0 malloc-64 64 4 248 8 0 254 16128 0 malloc-256 256 18 42 30 0 62 15360 0 malloc-256 256 20 40 20 0 62 15360 0 tcptw 88 0 138 4 0 254 12144 0 KNOTE 160 0 75 6 0 62 12000 0 malloc-16 16 31 719 60 0 254 12000 0 malloc-2048 2048 2 2 2 0 8 8192 0 malloc-2048 2048 0 4 3 0 8 8192 0 syncache 168 0 48 2 0 254 8064 0 malloc-64 64 8 118 9 0 254 8064 0 malloc-16 16 2 498 3 0 254 8000 0 ripcb 496 1 15 3 0 254 7936 0 malloc-384 384 1 19 35 0 30 7680 0 FPU_save_area 832 1 8 1 0 16 7488 0 epoch_record pcpu 256 4 12 4 0 62 4096 0 malloc-2048 2048 2 0 2 0 8 4096 0 malloc-1024 1024 4 0 4 0 16 4096 0 hostcache 64 1 62 1 0 254 4032 0 STS VFS Cache 144 0 28 1 0 62 4032 0 malloc-32 32 0 126 2 0 254 4032 0 vtnet_tx_hdr 24 0 167 0 0 254 4008 0 UMA Slabs 1 176 10 12 10 0 62 3872 0 KMAP ENTRY 96 36 3 36 0 0 3744 0 vmem 1856 1 1 1 0 8 3712 0 SMR CPU 32 3 92 3 0 254 3040 0 SMR SHARED 24 3 92 3 0 254 2280 0 FFS1 dinode 128 0 0 0 0 126 0 0 swblk 136 0 0 0 0 62 0 0 swpctrie 144 0 0 0 0 62 0 0 da_ccb 544 0 0 0 0 16 0 0 udplite_inpcb 496 0 0 0 0 254 0 0 tcp_log_node 120 0 0 0 0 126 0 0 tcp_log_bucket 176 0 0 0 0 62 0 0 tcp_log 416 0 0 0 0 254 0 0 tcpreass 48 0 0 0 0 254 0 0 tfo_ccache_entries 80 0 0 0 0 126 0 0 tfo 4 0 0 0 0 254 0 0 sackhole 32 0 0 0 0 254 0 0 ipq 56 0 0 0 0 254 0 0 IPsec SA lft_c 16 0 0 0 0 254 0 0 itimer 352 0 0 0 0 30 0 0 AIOLIO 272 0 0 0 0 30 0 0 AIOCB 552 0 0 0 0 16 0 0 AIOP 32 0 0 0 0 254 0 0 AIO 208 0 0 0 0 62 0 0 TMPFS node 224 0 0 0 0 62 0 0 LTS VFS Cache 360 0 0 0 0 30 0 0 cryptop 280 0 0 0 0 30 0 0 linux_dma_object 24 0 0 0 0 254 0 0 linux_dma_pctrie 144 0 0 0 0 62 0 0 IOMMU_MAP_ENTRY 120 0 0 0 0 126 0 0 ktls_session 192 0 0 0 0 62 0 0 mbuf_jumbo_16k 16384 0 0 0 0 254 0 0 mbuf_jumbo_9k 9216 0 0 0 0 254 0 0 audit_record 1280 0 0 0 0 8 0 0 domainset 40 0 0 0 0 254 0 0 MAC labels 40 0 0 0 0 254 0 0 vnpbuf 2776 0 0 0 0 10 0 0 mdpbuf 2776 0 0 0 0 1 0 0 nfspbuf 2776 0 0 0 0 2 0 0 swwbuf 2776 0 0 0 0 1 0 0 swrbuf 2776 0 0 0 0 2 0 0 umtx_shm 88 0 0 0 0 126 0 0 umtx pi 96 0 0 0 0 126 0 0 rangeset pctrie nodes 144 0 0 0 0 62 0 0 malloc-65536 65536 0 0 0 0 1 0 0 malloc-65536 65536 0 0 0 0 1 0 0 malloc-32768 32768 0 0 0 0 1 0 0 malloc-16384 16384 0 0 0 0 1 0 0 malloc-16384 16384 0 0 0 0 1 0 0 malloc-8192 8192 0 0 0 0 1 0 0 malloc-8192 8192 0 0 0 0 1 0 0 malloc-2048 2048 0 0 0 0 8 0 0 malloc-512 512 0 0 0 0 30 0 0 malloc-512 512 0 0 0 0 30 0 0 malloc-512 512 0 0 0 0 30 0 0 malloc-384 384 0 0 0 0 30 0 0 pcpu-32 32 0 0 0 0 254 0 0 pcpu-4 4 0 0 0 0 254 0 0 UMA Hash 256 0 0 0 0 62 0 0 db:0:pho> show umacache Zone Size Used Free Requests Bucket debugnet_mbuf_packet 2048 0 0 0 0 debugnet_mbuf_cluster 2048 -6144 0 0 0 debugnet_mbuf 256 -10240 0 0 0 ktls_buffers 16384 0 0 0 1 vm pgcache 4096 199014 5973 16749150 169 vm pgcache 4096 74808 3082 59874159 169 buf free cache 848 20574 14025 10537066 29 buffer arena-65 65536 0 1 1 1 buffer arena-61 61440 0 0 0 1 buffer arena-57 57344 0 0 0 1 buffer arena-53 53248 0 0 0 1 buffer arena-49 49152 0 0 0 1 buffer arena-45 45056 0 0 0 1 buffer arena-40 40960 0 0 0 1 buffer arena-36 36864 0 0 0 1 buffer arena-32 32768 13963 9465 3873903 17 buffer arena-28 28672 0 0 0 1 buffer arena-24 24576 0 0 0 1 buffer arena-20 20480 0 0 0 1 buffer arena-16 16384 8 123 153817 6 buffer arena-12 12288 0 0 0 1 buffer arena-81 8192 0 0 0 1 buffer arena-40 4096 0 0 0 2 kstack_cache 16384 676 8 734 2 db:0:pho> acttrace Tracing command kernel pid 0 tid 100044 td 0xfffffe00e4ce5900 (CPU 4) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce47e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce47e10 trap() at trap+0x56/frame 0xfffffe003ce47f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce47f20 --- trap 0x13, rip = 0xffffffff80f4c3f6, rsp = 0xfffffe00e48808b0, rbp = 0xfffffe00e4880940 --- ffs_sync() at ffs_sync+0x196/frame 0xfffffe00e4880940 vfs_write_suspend() at vfs_write_suspend+0x165/frame 0xfffffe00e4880990 vfs_write_suspend_umnt() at vfs_write_suspend_umnt+0x35/frame 0xfffffe00e48809c0 ffs_unmount() at ffs_unmount+0x468/frame 0xfffffe00e4880a40 dounmount() at dounmount+0x812/frame 0xfffffe00e4880ab0 vfs_deferred_unmount() at vfs_deferred_unmount+0xf9/frame 0xfffffe00e4880b00 taskqueue_run_locked() at taskqueue_run_locked+0xaa/frame 0xfffffe00e4880b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc2/frame 0xfffffe00e4880bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4880bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4880bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100003 td 0xfffffe00e4c5b3a0 (CPU 0) kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe0037790890 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00377908a0 uart_intr_rxready() at uart_intr_rxready+0xf5/frame 0xfffffe00377908d0 uart_intr() at uart_intr+0x128/frame 0xfffffe0037790910 intr_event_handle() at intr_event_handle+0xde/frame 0xfffffe0037790960 intr_execute_handlers() at intr_execute_handlers+0x66/frame 0xfffffe0037790990 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe0037790990 --- interrupt, rip = 0xffffffff810a50a6, rsp = 0xfffffe0037790a60, rbp = 0xfffffe0037790a60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0037790a60 acpi_cpu_idle() at acpi_cpu_idle+0x288/frame 0xfffffe0037790aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe0037790ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe0037790ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe0037790bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037790bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037790bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xfffffe00e4c5ac80 (CPU 1) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce20e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce20e10 trap() at trap+0x56/frame 0xfffffe003ce20f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce20f20 --- trap 0x13, rip = 0xffffffff810a50a6, rsp = 0xfffffe003778ba60, rbp = 0xfffffe003778ba60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe003778ba60 acpi_cpu_idle() at acpi_cpu_idle+0x288/frame 0xfffffe003778baa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe003778bac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe003778bae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe003778bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003778bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003778bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffffe00e4c5a560 (CPU 2) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce2de00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce2de10 trap() at trap+0x56/frame 0xfffffe003ce2df20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce2df20 --- trap 0x13, rip = 0xffffffff810a50a6, rsp = 0xfffffe003779aa60, rbp = 0xfffffe003779aa60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe003779aa60 acpi_cpu_idle() at acpi_cpu_idle+0x288/frame 0xfffffe003779aaa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe003779aac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe003779aae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe003779abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003779abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003779abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xfffffe00e4c59e40 (CPU 3) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce3ae00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce3ae10 trap() at trap+0x56/frame 0xfffffe003ce3af20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce3af20 --- trap 0x13, rip = 0xffffffff810a50a6, rsp = 0xfffffe0037795a60, rbp = 0xfffffe0037795a60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0037795a60 acpi_cpu_idle() at acpi_cpu_idle+0x288/frame 0xfffffe0037795aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe0037795ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe0037795ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe0037795bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037795bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037795bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xfffffe00e4c59000 (CPU 5) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce54e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce54e10 trap() at trap+0x56/frame 0xfffffe003ce54f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce54f20 --- trap 0x13, rip = 0xffffffff810a50a6, rsp = 0xfffffe003779fa60, rbp = 0xfffffe003779fa60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe003779fa60 acpi_cpu_idle() at acpi_cpu_idle+0x288/frame 0xfffffe003779faa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe003779fac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe003779fae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe003779fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003779fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003779fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xfffffe00e4c5d740 (CPU 6) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce61e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce61e10 trap() at trap+0x56/frame 0xfffffe003ce61f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce61f20 --- trap 0x13, rip = 0xffffffff810a50a6, rsp = 0xfffffe00377aea60, rbp = 0xfffffe00377aea60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00377aea60 acpi_cpu_idle() at acpi_cpu_idle+0x288/frame 0xfffffe00377aeaa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00377aeac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00377aeae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00377aebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377aebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377aebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xfffffe00e4c5d020 (CPU 7) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce6ee00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce6ee10 trap() at trap+0x56/frame 0xfffffe003ce6ef20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce6ef20 --- trap 0x13, rip = 0xffffffff810a50a6, rsp = 0xfffffe00377a9a60, rbp = 0xfffffe00377a9a60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00377a9a60 acpi_cpu_idle() at acpi_cpu_idle+0x288/frame 0xfffffe00377a9aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00377a9ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00377a9ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00377a9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377a9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377a9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffffe00e4c5c900 (CPU 8) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce7be00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce7be10 trap() at trap+0x56/frame 0xfffffe003ce7bf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce7bf20 --- trap 0x13, rip = 0xffffffff810a50a6, rsp = 0xfffffe00377b8a60, rbp = 0xfffffe00377b8a60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00377b8a60 acpi_cpu_idle() at acpi_cpu_idle+0x288/frame 0xfffffe00377b8aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00377b8ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00377b8ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00377b8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377b8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377b8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100012 td 0xfffffe00e4c611e0 (CPU 9) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce88e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce88e10 trap() at trap+0x56/frame 0xfffffe003ce88f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce88f20 --- trap 0x13, rip = 0xffffffff810a50a6, rsp = 0xfffffe00377b3a60, rbp = 0xfffffe00377b3a60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00377b3a60 acpi_cpu_idle() at acpi_cpu_idle+0x288/frame 0xfffffe00377b3aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00377b3ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00377b3ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00377b3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377b3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377b3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xfffffe00e4c60ac0 (CPU 10) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce95e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce95e10 trap() at trap+0x56/frame 0xfffffe003ce95f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce95f20 --- trap 0x13, rip = 0xffffffff810a50a6, rsp = 0xfffffe00377c2a60, rbp = 0xfffffe00377c2a60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00377c2a60 acpi_cpu_idle() at acpi_cpu_idle+0x288/frame 0xfffffe00377c2aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00377c2ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00377c2ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00377c2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377c2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377c2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100014 td 0xfffffe00e4c603a0 (CPU 11) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003cea2e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003cea2e10 trap() at trap+0x56/frame 0xfffffe003cea2f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003cea2f20 --- trap 0x13, rip = 0xffffffff810a50a6, rsp = 0xfffffe00377bda60, rbp = 0xfffffe00377bda60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00377bda60 acpi_cpu_idle() at acpi_cpu_idle+0x288/frame 0xfffffe00377bdaa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00377bdac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00377bdae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00377bdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377bdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377bdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db:0:pho> ps pid ppid pgrp uid state wmesg wchan cmd 70861 2899 2852 0 S nanslp 0xffffffff81ebc0a2 sleep 68512 2756 68512 0 S+ ttyin 0xfffff800060ee8b0 csh 60494 2769 60494 0 D+ ufs 0xfffff80111a5ac50 ls 47808 1 47796 0 D+ ufs 0xfffff80111a5ac50 ls 26527 26526 2603 0 D ufs 0xfffff80111a5ac50 ls 26526 2603 2603 0 S wait 0xfffff806a7046000 sh 24211 1 2770 0 D+ ufs 0xfffff80111a5ac50 ls 22903 22873 22903 0 S+ select 0xfffff8000618d4c0 bash 22873 22872 22873 0 S+ pause 0xfffff804d89035e8 csh 22872 14218 22872 2006 S+ wait 0xfffff80295406a70 su 14218 14217 14218 2006 Ss+ wait 0xfffff80003baf000 bash 14217 14207 14207 2006 S select 0xfffff80347f4f9c0 sshd 14207 2729 14207 0 Ss select 0xfffff8022ed5a340 sshd 2902 2852 2852 0 S piperd 0xfffff8001cb8cba0 sh 2900 2852 2852 0 S piperd 0xfffff80009c775d0 awk 2899 2852 2852 0 S wait 0xfffff8011177b538 sh 2852 1 2852 0 Ss wait 0xfffff8003a2f8a70 sh 2769 2767 2769 0 S+ wait 0xfffff800061b3538 bash 2767 2766 2767 0 S+ pause 0xfffff8001cb9a0b0 csh 2766 2761 2766 2006 S+ wait 0xfffff800040faa70 su 2761 2760 2761 2006 Ss+ wait 0xfffff80003d98000 bash 2760 2757 2757 2006 S select 0xfffff8001c9895c0 sshd 2757 2729 2757 0 Ss select 0xfffff8000618df40 sshd 2756 1 2756 0 Ss+ wait 0xfffff801117a7538 login 2755 1 2755 0 Ss+ ttyin 0xfffff800060eecb0 getty 2754 1 2754 0 Ss+ ttyin 0xfffff80009be40b0 getty 2753 1 2753 0 Ss+ ttyin 0xfffff80009be44b0 getty 2752 1 2752 0 Ss+ ttyin 0xfffff80009be48b0 getty 2751 1 2751 0 Ss+ ttyin 0xfffff80009be4cb0 getty 2750 1 2750 0 Ss+ ttyin 0xfffff80009bee0b0 getty 2749 1 2749 0 Ss+ ttyin 0xfffff80009bee4b0 getty 2748 1 2748 0 Ss+ ttyin 0xfffff80009bee8b0 getty 2747 1 2747 0 Ss+ ttyin 0xfffff80009beecb0 getty 2733 1 2733 0 Ss nanslp 0xffffffff81ebc0a2 cron 2729 1 2729 0 Ss select 0xfffff80003c21ec0 sshd 2710 1 2710 0 Ss select 0xfffff8000680fa40 inetd 2684 1 2684 123 Ss select 0xfffff8000415ce40 ntpd 2658 1 2658 0 Ss select 0xfffff80004164ac0 powerd 2653 1 2653 0 Ss rpcsvc 0xfffff800098a0d60 NLM: master 2650 1 2650 0 Ss select 0xfffff8000415cf40 rpc.statd 2625 1 2625 0 Ss select 0xfffff8000680fac0 rpcbind 2603 1 2603 0 Ss wait 0xfffff8003a113538 watchdogd 2595 1 2595 0 Ss select 0xfffff8000680fb40 syslogd 2406 1 2406 0 Ss select 0xfffff8000680fc40 devd 2401 1 2401 65 Ss select 0xfffff80014dc5dc0 dhclient 2342 1 2342 0 Ss select 0xfffff8000439c340 dhclient 2339 1 2339 0 Ss select 0xfffff8000439c3c0 dhclient 34 0 0 0 DL syncer 0xffffffff81fab480 [syncer] 33 0 0 0 DL vlruwt 0xfffff80006024000 [vnlru] 32 0 0 0 DL (threaded) [bufdaemon] 100213 D qsleep 0xffffffff81faa520 [bufdaemon] 100214 D - 0xffffffff81a0af40 [bufspacedaemon-0] 100215 D - 0xffffffff81a13200 [bufspacedaemon-1] 100216 D - 0xffffffff81a1b4c0 [bufspacedaemon-2] 100217 D - 0xffffffff81a23780 [bufspacedaemon-3] 100218 D - 0xffffffff81a2ba40 [bufspacedaemon-4] 100219 D - 0xffffffff81a33d00 [bufspacedaemon-5] 100220 D - 0xffffffff81a3bfc0 [bufspacedaemon-6] 100234 D sdflush 0xfffff80014da8ce8 [/ worker] 100241 D sdflush 0xfffff80003be9ce8 [/tmp worker] 100734 D sdflush 0xfffff807312c34e8 [/mnt worker] 31 0 0 0 DL psleep 0xffffffff81fd0208 [vmdaemon] 30 0 0 0 DL (threaded) [pagedaemon] 100211 D psleep 0xffffffff81fc4678 [dom0] 100221 D launds 0xffffffff81fc4684 [laundry: dom0] 100222 D umarcl 0xffffffff80f70880 [uma] 29 0 0 0 DL idle 0xfffff8000609c538 [enc_daemon1] 28 0 0 0 DL idle 0xfffff8000609ca70 [enc_daemon0] 27 0 0 0 DL - 0xffffffff81cf03d8 [rand_harvestq] 26 0 0 0 DL - 0xffffffff81fa7bdc [soaiod4] 25 0 0 0 DL - 0xffffffff81fa7bdc [soaiod3] 24 0 0 0 DL - 0xffffffff81fa7bdc [soaiod2] 23 0 0 0 DL - 0xffffffff81fa7bdc [soaiod1] 22 0 0 0 DL (threaded) [usb] 100140 D - 0xfffffe010a188440 [usbus0] 100141 D - 0xfffffe010a188498 [usbus0] 100142 D - 0xfffffe010a1884f0 [usbus0] 100143 D - 0xfffffe010a188548 [usbus0] 100144 D - 0xfffffe010a1885a0 [usbus0] 100146 D - 0xfffffe00e4d66cf0 [usbus1] 100147 D - 0xfffffe00e4d66d48 [usbus1] 100148 D - 0xfffffe00e4d66da0 [usbus1] 100149 D - 0xfffffe00e4d66df8 [usbus1] 100150 D - 0xfffffe00e4d66e50 [usbus1] 100151 D - 0xfffffe010a548cf0 [usbus2] 100152 D - 0xfffffe010a548d48 [usbus2] 100153 D - 0xfffffe010a548da0 [usbus2] 100154 D - 0xfffffe010a548df8 [usbus2] 100155 D - 0xfffffe010a548e50 [usbus2] 21 0 0 0 RL (threaded) [cam] 100133 D - 0xffffffff81bc7ac0 [doneq0] 100134 RunQ [doneq1] 100135 D - 0xffffffff81bc7bc0 [doneq2] 100136 D - 0xffffffff81bc7a40 [async] 100168 D - 0xffffffff81bc7910 [scanner] 20 0 0 0 DL seqstat 0xfffff80003d64488 [sequencer 00] 19 0 0 0 DL crypto_ 0xfffff80003d43fa0 [crypto returns 11] 18 0 0 0 DL crypto_ 0xfffff80003d43f50 [crypto returns 10] 17 0 0 0 DL crypto_ 0xfffff80003d43f00 [crypto returns 9] 16 0 0 0 DL crypto_ 0xfffff80003d43eb0 [crypto returns 8] 15 0 0 0 DL crypto_ 0xfffff80003d43e60 [crypto returns 7] 14 0 0 0 DL crypto_ 0xfffff80003d43e10 [crypto returns 6] 9 0 0 0 DL crypto_ 0xfffff80003d43dc0 [crypto returns 5] 8 0 0 0 DL crypto_ 0xfffff80003d43d70 [crypto returns 4] 7 0 0 0 DL crypto_ 0xfffff80003d43d20 [crypto returns 3] 6 0 0 0 DL crypto_ 0xfffff80003d43cd0 [crypto returns 2] 5 0 0 0 DL crypto_ 0xfffff80003d43c80 [crypto returns 1] 4 0 0 0 DL crypto_ 0xfffff80003d43c30 [crypto returns 0] 3 0 0 0 DL crypto_ 0xffffffff81fc1ba0 [crypto] 13 0 0 0 DL (threaded) [geom] 100104 D - 0xffffffff81e9b920 [g_event] 100105 D - 0xffffffff81e9b928 [g_up] 100106 D - 0xffffffff81e9b930 [g_down] 2 0 0 0 DL (threaded) [KTLS] 100077 D - 0xfffff80003ccc800 [thr_0] 100078 D - 0xfffff80003ccc880 [thr_1] 100079 D - 0xfffff80003ccc900 [thr_2] 100080 D - 0xfffff80003ccc980 [thr_3] 100081 D - 0xfffff80003ccca00 [thr_4] 100082 D - 0xfffff80003ccca80 [thr_5] 100083 D - 0xfffff80003cccb00 [thr_6] 100084 D - 0xfffff80003cccb80 [thr_7] 100085 D - 0xfffff80003cccc00 [thr_8] 100086 D - 0xfffff80003cccc80 [thr_9] 100087 D - 0xfffff80003cccd00 [thr_10] 100088 D - 0xfffff80003cccd80 [thr_11] 12 0 0 0 WL (threaded) [intr] 100046 I [swi6: task queue] 100047 I [swi6: Giant taskq] 100049 I [swi5: fast taskq] 100089 I [swi4: clock (0)] 100090 I [swi4: clock (1)] 100091 I [swi4: clock (2)] 100092 I [swi4: clock (3)] 100093 I [swi4: clock (4)] 100094 I [swi4: clock (5)] 100095 I [swi4: clock (6)] 100096 I [swi4: clock (7)] 100097 I [swi4: clock (8)] 100098 I [swi4: clock (9)] 100099 I [swi4: clock (10)] 100100 I [swi4: clock (11)] 100101 I [swi1: netisr 0] 100102 I [swi3: vm] 100137 I [swi4: apei] 100138 I [irq56: ahci0] 100139 I [irq19: xhci0] 100145 I [irq18: ehci0 ehci1+] 100156 I [irq71: ahci1] 100157 I [swi0: uart uart] 100175 I [irq73: mlx5_core0] 100176 I [irq74: mlx5_core0] 100177 I [irq72: mlx5_core0] 100178 I [irq75: mlx5_core0] 100179 I [irq76: mlx5_core0] 100180 I [irq77: mlx5_core0] 100181 I [irq78: mlx5_core0] 100182 I [irq79: mlx5_core0] 100183 I [irq80: mlx5_core0] 100184 I [irq81: mlx5_core0] 100185 I [irq82: mlx5_core0] 100186 I [irq83: mlx5_core0] 100187 I [irq84: mlx5_core0] 100188 I [irq85: mlx5_core0] 100189 I [irq86: mlx5_core0] 100195 I [irq88: mlx5_core1] 100196 I [irq89: mlx5_core1] 100197 I [irq87: mlx5_core1] 100198 I [irq90: mlx5_core1] 100199 I [irq91: mlx5_core1] 100200 I [irq92: mlx5_core1] 100201 I [irq93: mlx5_core1] 100202 I [irq94: mlx5_core1] 100203 I [irq95: mlx5_core1] 100204 I [irq96: mlx5_core1] 100205 I [irq97: mlx5_core1] 100206 I [irq98: mlx5_core1] 100207 I [irq99: mlx5_core1] 100208 I [irq100: mlx5_core1] 100209 I [irq101: mlx5_core1] 100264 I [irq102: ioat0] 100265 I [irq103: ioat1] 100266 I [irq104: ioat2] 100267 I [irq105: ioat3] 100268 I [irq106: ioat4] 100269 I [irq107: ioat5] 100270 I [irq108: ioat6] 100271 I [irq109: ioat7] 11 0 0 0 RL (threaded) [idle] 100003 Run CPU 0 [idle: cpu0] 100004 Run CPU 1 [idle: cpu1] 100005 Run CPU 2 [idle: cpu2] 100006 Run CPU 3 [idle: cpu3] 100007 CanRun [idle: cpu4] 100008 Run CPU 5 [idle: cpu5] 100009 Run CPU 6 [idle: cpu6] 100010 Run CPU 7 [idle: cpu7] 100011 Run CPU 8 [idle: cpu8] 100012 Run CPU 9 [idle: cpu9] 100013 Run CPU 10 [idle: cpu10] 100014 Run CPU 11 [idle: cpu11] 1 0 1 0 SLs wait 0xfffff80003b43538 [init] 10 0 0 0 DL audit_w 0xffffffff81fc20b0 [audit] 0 0 0 0 RLs (threaded) [kernel] 100000 D swapin 0xffffffff81e9beb0 [swapper] 100015 D - 0xfffff80003965c00 [softirq_0] 100016 D - 0xfffff80003965b00 [softirq_1] 100017 D - 0xfffff80003965a00 [softirq_2] 100018 D - 0xfffff80003965900 [softirq_3] 100019 D - 0xfffff80003965800 [softirq_4] 100020 D - 0xfffff80003965700 [softirq_5] 100021 D - 0xfffff80003965600 [softirq_6] 100022 D - 0xfffff80003965500 [softirq_7] 100023 D - 0xfffff80003965400 [softirq_8] 100024 D - 0xfffff80003965300 [softirq_9] 100025 D - 0xfffff80003965200 [softirq_10] 100026 D - 0xfffff80003965100 [softirq_11] 100027 D - 0xfffff80003965000 [if_io_tqg_0] 100028 D - 0xfffff80003b1ce00 [if_io_tqg_1] 100029 D - 0xfffff80003b1cd00 [if_io_tqg_2] 100030 D - 0xfffff80003b1cc00 [if_io_tqg_3] 100031 D - 0xfffff80003b1cb00 [if_io_tqg_4] 100032 D - 0xfffff80003b1ca00 [if_io_tqg_5] 100033 D - 0xfffff80003b1c900 [if_io_tqg_6] 100034 D - 0xfffff80003b1c800 [if_io_tqg_7] 100035 D - 0xfffff80003b1c700 [if_io_tqg_8] 100036 D - 0xfffff80003b1c600 [if_io_tqg_9] 100037 D - 0xfffff80003b1c500 [if_io_tqg_10] 100038 D - 0xfffff80003b1c400 [if_io_tqg_11] 100039 D - 0xfffff80003b1c300 [if_config_tqg_0] 100040 D - 0xfffff80003b1fb00 [pci_hp taskq] 100041 D - 0xfffff80003b1fa00 [aiod_kick taskq] 100042 D - 0xfffff80003b1f900 [kqueue_ctx taskq] 100043 D - 0xfffff80003b1f800 [inm_free taskq] 100044 Run CPU 4 [deferred_unmount ta] 100045 D - 0xfffff80003b1f600 [linuxkpi_irq_wq] 100048 D - 0xfffff80003b1f100 [thread taskq] 100050 D - 0xfffff80003b12d00 [in6m_free taskq] 100051 D - 0xfffff80003b12c00 [linuxkpi_short_wq_0] 100052 D - 0xfffff80003b12c00 [linuxkpi_short_wq_1] 100053 D - 0xfffff80003b12c00 [linuxkpi_short_wq_2] 100054 D - 0xfffff80003b12c00 [linuxkpi_short_wq_3] 100055 D - 0xfffff80003b12c00 [linuxkpi_short_wq_4] 100056 D - 0xfffff80003b12c00 [linuxkpi_short_wq_5] 100057 D - 0xfffff80003b12c00 [linuxkpi_short_wq_6] 100058 D - 0xfffff80003b12c00 [linuxkpi_short_wq_7] 100059 D - 0xfffff80003b12c00 [linuxkpi_short_wq_8] 100060 D - 0xfffff80003b12c00 [linuxkpi_short_wq_9] 100061 D - 0xfffff80003b12c00 [linuxkpi_short_wq_1] 100062 D - 0xfffff80003b12c00 [linuxkpi_short_wq_1] 100063 D - 0xfffff80003b12c00 [linuxkpi_short_wq_1] 100064 D - 0xfffff80003b12b00 [linuxkpi_long_wq_0] 100065 D - 0xfffff80003b12b00 [linuxkpi_long_wq_1] 100066 D - 0xfffff80003b12b00 [linuxkpi_long_wq_2] 100067 D - 0xfffff80003b12b00 [linuxkpi_long_wq_3] 100068 D - 0xfffff80003b12b00 [linuxkpi_long_wq_4] 100069 D - 0xfffff80003b12b00 [linuxkpi_long_wq_5] 100070 D - 0xfffff80003b12b00 [linuxkpi_long_wq_6] 100071 D - 0xfffff80003b12b00 [linuxkpi_long_wq_7] 100072 D - 0xfffff80003b12b00 [linuxkpi_long_wq_8] 100073 D - 0xfffff80003b12b00 [linuxkpi_long_wq_9] 100074 D - 0xfffff80003b12b00 [linuxkpi_long_wq_10] 100075 D - 0xfffff80003b12b00 [linuxkpi_long_wq_11] 100076 D - 0xfffff80003b12b00 [linuxkpi_long_wq_12] 100103 D - 0xfffff80003c73e00 [firmware taskq] 100107 D - 0xfffff80003c73d00 [crypto_0] 100108 D - 0xfffff80003c73d00 [crypto_1] 100109 D - 0xfffff80003c73d00 [crypto_2] 100110 D - 0xfffff80003c73d00 [crypto_3] 100111 D - 0xfffff80003c73d00 [crypto_4] 100112 D - 0xfffff80003c73d00 [crypto_5] 100113 D - 0xfffff80003c73d00 [crypto_6] 100114 D - 0xfffff80003c73d00 [crypto_7] 100115 D - 0xfffff80003c73d00 [crypto_8] 100116 D - 0xfffff80003c73d00 [crypto_9] 100117 D - 0xfffff80003c73d00 [crypto_10] 100118 D - 0xfffff80003c73d00 [crypto_11] 100158 D - 0xfffff80006125c00 [mca taskq] 100164 D - 0xfffff80009aa1e00 [acpi_task_0] 100165 D - 0xfffff80009aa1e00 [acpi_task_1] 100166 D - 0xfffff80009aa1e00 [acpi_task_2] 100167 D - 0xfffff80003c73c00 [CAM taskq] 100171 D - 0xfffff80009aa1d00 [mlx5_core0-rec] 100172 D - 0xfffff80009aa1c00 [mlx5_core0-wdg] 100173 D - 0xfffff80009aa1b00 [mlx5_core0-cmd] 100174 D - 0xfffff80009aa1a00 [mlx5_page_allocator] 100190 D - 0xfffff800183ab900 [mlx5_esw_wq] 100191 D - 0xfffff800183ab800 [mlx5_core1-rec] 100192 D - 0xfffff800183ab700 [mlx5_core1-wdg] 100193 D - 0xfffff800183ab600 [mlx5_core1-cmd] 100194 D - 0xfffff800183ab500 [mlx5_page_allocator] 100210 D - 0xfffff80006056e00 [mlx5_esw_wq] db:0:pho> allt Tracing command kernel pid 0 tid 100000 td 0xffffffff81e9c400 sched_switch() at sched_switch+0x5af/frame 0xffffffff8254dac0 mi_switch() at mi_switch+0x155/frame 0xffffffff8254dae0 sleepq_switch() at sleepq_switch+0x119/frame 0xffffffff8254db20 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xffffffff8254db60 _sleep() at _sleep+0x298/frame 0xffffffff8254dc00 swapper() at swapper+0x75/frame 0xffffffff8254dc60 mi_startup() at mi_startup+0x2f3/frame 0xffffffff8254dcb0 btext() at btext+0x22 Tracing command kernel pid 0 tid 100015 td 0xfffffe00e4c5fc80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377ccab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377ccad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00377ccb10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00377ccb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377ccbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377ccbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377ccbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xfffffe00e4c5f560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377c7ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377c7ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00377c7b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00377c7b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377c7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377c7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377c7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100017 td 0xfffffe00e4c5ee40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377d6ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377d6ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00377d6b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00377d6b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377d6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377d6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377d6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100018 td 0xfffffe00e4c5e720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377d1ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377d1ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00377d1b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00377d1b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377d1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377d1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377d1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xfffffe00e4c5e000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377e0ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377e0ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00377e0b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00377e0b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377e0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377e0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377e0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xfffffe00e4c62740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377dbab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377dbad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00377dbb10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00377dbb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377dbbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377dbbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377dbbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xfffffe00e4c62020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377eaab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377eaad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00377eab10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00377eab80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377eabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377eabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377eabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xfffffe00e4c61900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377e5ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377e5ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00377e5b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00377e5b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377e5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377e5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377e5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xfffffe00e4cde1e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377f4ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377f4ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00377f4b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00377f4b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377f4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377f4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377f4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xfffffe00e4cddac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377efab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377efad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00377efb10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00377efb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377efbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377efbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377efbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xfffffe00e4cdd3a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377feab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377fead0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00377feb10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00377feb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377febb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377febf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377febf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100026 td 0xfffffe00e4cdcc80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377f9ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377f9ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00377f9b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00377f9b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377f9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377f9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377f9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100027 td 0xfffffe00e4cdc560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4835ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4835ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4835b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e4835b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4835bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4835bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4835bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100028 td 0xfffffe00e4cdbe40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4830ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4830ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4830b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e4830b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4830bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4830bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4830bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100029 td 0xfffffe00e4cdb720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e483fab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e483fad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e483fb10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e483fb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e483fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e483fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e483fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100030 td 0xfffffe00e4cdb000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e483aab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e483aad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e483ab10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e483ab80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e483abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e483abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e483abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100031 td 0xfffffe00e4cdf740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4849ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4849ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4849b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e4849b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4849bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4849bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4849bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100032 td 0xfffffe00e4cdf020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4844ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4844ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4844b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e4844b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4844bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4844bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4844bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100033 td 0xfffffe00e4cde900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4853ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4853ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4853b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e4853b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4853bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4853bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4853bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100034 td 0xfffffe00e4ce51e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e484eab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e484ead0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e484eb10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e484eb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e484ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e484ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e484ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100035 td 0xfffffe00e4ce4ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e485dab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e485dad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e485db10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e485db80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e485dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e485dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e485dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100036 td 0xfffffe00e4ce43a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4858ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4858ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4858b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e4858b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4858bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4858bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4858bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100037 td 0xfffffe00e4ce3c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4867ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4867ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4867b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e4867b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4867bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4867bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4867bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100038 td 0xfffffe00e4ce3560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4862ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4862ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4862b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e4862b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4862bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4862bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4862bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100039 td 0xfffffe00e4ce2e40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4871ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4871ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4871b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e4871b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00e4871bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4871bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4871bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100040 td 0xfffffe00e4ce2720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e486ca80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e486caa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e486cae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e486cb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e486cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e486cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e486cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100041 td 0xfffffe00e4ce2000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e487ba80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e487baa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e487bae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e487bb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e487bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e487bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e487bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100042 td 0xfffffe00e4ce6740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4876a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4876aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4876ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4876b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4876bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4876bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4876bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100043 td 0xfffffe00e4ce6020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4885a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4885aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4885ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4885b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4885bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4885bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4885bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100044 td 0xfffffe00e4ce5900 (CPU 4) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce47e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce47e10 trap() at trap+0x56/frame 0xfffffe003ce47f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce47f20 --- trap 0x13, rip = 0xffffffff80f4c3f6, rsp = 0xfffffe00e48808b0, rbp = 0xfffffe00e4880940 --- ffs_sync() at ffs_sync+0x196/frame 0xfffffe00e4880940 vfs_write_suspend() at vfs_write_suspend+0x165/frame 0xfffffe00e4880990 vfs_write_suspend_umnt() at vfs_write_suspend_umnt+0x35/frame 0xfffffe00e48809c0 ffs_unmount() at ffs_unmount+0x468/frame 0xfffffe00e4880a40 dounmount() at dounmount+0x812/frame 0xfffffe00e4880ab0 vfs_deferred_unmount() at vfs_deferred_unmount+0xf9/frame 0xfffffe00e4880b00 taskqueue_run_locked() at taskqueue_run_locked+0xaa/frame 0xfffffe00e4880b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc2/frame 0xfffffe00e4880bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4880bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4880bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100045 td 0xfffffe00e4cec1e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e488fab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e488fad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e488fb10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e488fb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe00e488fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e488fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e488fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100048 td 0xfffffe00e4ceac80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4894a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4894aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4894ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4894b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4894bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4894bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4894bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100050 td 0xfffffe00e4ce9e40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e489ea80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e489eaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e489eae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e489eb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e489ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e489ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e489ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100051 td 0xfffffe00e4ce9720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48ada80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48adaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48adae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48adb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48adbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48adbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48adbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100052 td 0xfffffe00e4ce9000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48a8a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48a8aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48a8ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48a8b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48a8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48a8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48a8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100053 td 0xfffffe00e4ced740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48b7a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48b7aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48b7ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48b7b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48b7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48b7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48b7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100054 td 0xfffffe00e4ced020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48b2a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48b2aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48b2ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48b2b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48b2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48b2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48b2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100055 td 0xfffffe00e4cec900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48c1a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48c1aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48c1ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48c1b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48c1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48c1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48c1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100056 td 0xfffffe00e4cf11e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48bca80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48bcaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48bcae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48bcb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48bcbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48bcbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48bcbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100057 td 0xfffffe00e4cf0ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48cba80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48cbaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48cbae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48cbb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48cbbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48cbbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48cbbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100058 td 0xfffffe00e4cf03a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48c6a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48c6aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48c6ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48c6b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48c6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48c6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48c6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100059 td 0xfffffe00e4cefc80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48d5a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48d5aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48d5ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48d5b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48d5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48d5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48d5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100060 td 0xfffffe00e4cef560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48d0a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48d0aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48d0ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48d0b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48d0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48d0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48d0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100061 td 0xfffffe00e4ceee40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48dfa80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48dfaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48dfae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48dfb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48dfbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48dfbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48dfbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100062 td 0xfffffe00e4cee720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48daa80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48daaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48daae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48dab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48dabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48dabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48dabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100063 td 0xfffffe00e4cee000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48e9a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48e9aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48e9ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48e9b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48e9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48e9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48e9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100064 td 0xfffffe00e4cf2740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48e4a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48e4aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48e4ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48e4b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48e4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48e4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48e4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100065 td 0xfffffe00e4cf2020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48f3a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48f3aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48f3ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48f3b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48f3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48f3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48f3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100066 td 0xfffffe00e4cf1900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48eea80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48eeaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48eeae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48eeb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48eebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48eebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48eebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100067 td 0xfffffe00e4cf61e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48fda80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48fdaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48fdae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48fdb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48fdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48fdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48fdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100068 td 0xfffffe00e4cf5ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e48f8a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e48f8aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e48f8ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e48f8b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e48f8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48f8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48f8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100069 td 0xfffffe00e4cf53a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4907a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4907aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4907ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4907b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4907bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4907bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4907bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100070 td 0xfffffe00e4cf4c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4902a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4902aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4902ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4902b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4902bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4902bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4902bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100071 td 0xfffffe00e4cf4560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4911a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4911aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4911ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4911b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4911bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4911bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4911bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100072 td 0xfffffe00e4cf3e40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e490ca80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e490caa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e490cae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e490cb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e490cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e490cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e490cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100073 td 0xfffffe00e4cf3720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e491ba80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e491baa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e491bae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e491bb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e491bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e491bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e491bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100074 td 0xfffffe00e4cf3000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4916a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4916aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4916ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4916b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4916bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4916bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4916bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100075 td 0xfffffe00e4cf7740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4925a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4925aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4925ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4925b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4925bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4925bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4925bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100076 td 0xfffffe00e4cf7020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4920a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4920aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4920ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4920b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4920bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4920bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4920bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100103 td 0xfffffe00e4d08560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49eda80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49edaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e49edae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e49edb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e49edbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49edbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49edbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100107 td 0xfffffe00e4d0b740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e04a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e04aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e04ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e04b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e04bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e04bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e04bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100108 td 0xfffffe00e4d0b020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49fca80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49fcaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e49fcae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e49fcb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e49fcbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49fcbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49fcbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100109 td 0xfffffe00e4d0a900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e0ea80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e0eaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e0eae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e0eb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e0ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e0ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e0ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100110 td 0xfffffe00e4d241e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e09a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e09aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e09ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e09b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e09bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e09bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e09bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100111 td 0xfffffe00e4d23ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e18a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e18aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e18ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e18b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e18bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e18bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e18bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100112 td 0xfffffe00e4d233a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e13a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e13aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e13ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e13b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e13bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e13bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e13bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100113 td 0xfffffe00e4d22c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e22a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e22aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e22ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e22b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e22bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e22bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e22bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100114 td 0xfffffe00e4d22560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e1da80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e1daa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e1dae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e1db80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e1dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e1dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e1dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100115 td 0xfffffe00e4d21e40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e2ca80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e2caa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e2cae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e2cb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e2cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e2cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e2cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100116 td 0xfffffe00e4d21720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e27a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e27aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e27ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e27b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e27bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e27bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e27bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100117 td 0xfffffe00e4d21000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e36a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e36aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e36ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e36b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e36bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e36bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e36bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100118 td 0xfffffe00e4d25740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e31a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e31aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e31ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e31b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4e31bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e31bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e31bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100158 td 0xfffffe010a6d7000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f05ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f05ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f05b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e4f05b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe00e4f05bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f05bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f05bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100164 td 0xfffffe010d674020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f47ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f47ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f47b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e4f47b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe00e4f47bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f47bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f47bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100165 td 0xfffffe010d673900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f56ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f56ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f56b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e4f56b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe00e4f56bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f56bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f56bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100166 td 0xfffffe010d9f51e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f51ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f51ad0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f51b10 msleep_spin_sbt() at msleep_spin_sbt+0x151/frame 0xfffffe00e4f51b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe00e4f51bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f51bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f51bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100167 td 0xfffffe010d9f4ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f60a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f60aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f60ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4f60b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4f60bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f60bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f60bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100171 td 0xfffffe010d9f3c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f80a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f80aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f80ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4f80b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4f80bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f80bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f80bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100172 td 0xfffffe010d9f3560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f7ba80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f7baa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f7bae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4f7bb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4f7bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f7bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f7bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100173 td 0xfffffe010d9f2e40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f8aa80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f8aaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f8aae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4f8ab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4f8abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f8abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f8abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100174 td 0xfffffe010d9f2720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f85a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f85aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f85ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4f85b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4f85bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f85bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f85bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100190 td 0xfffffe010e01ac80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4fd5a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4fd5aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4fd5ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4fd5b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4fd5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4fd5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4fd5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100191 td 0xfffffe010e01a560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4fe4a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4fe4aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4fe4ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4fe4b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4fe4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4fe4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4fe4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100192 td 0xfffffe010e019e40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4fdfa80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4fdfaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4fdfae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4fdfb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4fdfbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4fdfbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4fdfbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100193 td 0xfffffe010e019720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4feea80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4feeaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4feeae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4feeb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4feebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4feebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4feebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100194 td 0xfffffe010e019000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4fe9a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4fe9aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4fe9ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4fe9b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e4fe9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4fe9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4fe9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100210 td 0xfffffe010de823a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe010e64aa80 mi_switch() at mi_switch+0x155/frame 0xfffffe010e64aaa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe010e64aae0 _sleep() at _sleep+0x32f/frame 0xfffffe010e64ab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe010e64abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe010e64abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010e64abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command init pid 1 tid 100002 td 0xfffffe00e4c5bac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377816d0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377816f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0037781730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0037781780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0037781790 _sleep() at _sleep+0x325/frame 0xfffffe0037781830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe00377818c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0037781ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0037781bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0037781bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x28df8a, rsp = 0x7fffffffe718, rbp = 0x7fffffffe780 --- Tracing command KTLS pid 2 tid 100077 td 0xfffffe00e4d001e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49479b0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49479d0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4947a10 _sleep() at _sleep+0x32f/frame 0xfffffe00e4947ab0 ktls_work_thread() at ktls_work_thread+0xdc/frame 0xfffffe00e4947bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4947bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4947bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100078 td 0xfffffe00e4cffac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49429b0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49429d0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4942a10 _sleep() at _sleep+0x32f/frame 0xfffffe00e4942ab0 ktls_work_thread() at ktls_work_thread+0xdc/frame 0xfffffe00e4942bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4942bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4942bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100079 td 0xfffffe00e4cff3a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49519b0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49519d0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4951a10 _sleep() at _sleep+0x32f/frame 0xfffffe00e4951ab0 ktls_work_thread() at ktls_work_thread+0xdc/frame 0xfffffe00e4951bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4951bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4951bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100080 td 0xfffffe00e4cfec80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e494c9b0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e494c9d0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e494ca10 _sleep() at _sleep+0x32f/frame 0xfffffe00e494cab0 ktls_work_thread() at ktls_work_thread+0xdc/frame 0xfffffe00e494cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e494cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e494cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100081 td 0xfffffe00e4cfe560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e495b9b0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e495b9d0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e495ba10 _sleep() at _sleep+0x32f/frame 0xfffffe00e495bab0 ktls_work_thread() at ktls_work_thread+0xdc/frame 0xfffffe00e495bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e495bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e495bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100082 td 0xfffffe00e4cfde40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49569b0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49569d0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4956a10 _sleep() at _sleep+0x32f/frame 0xfffffe00e4956ab0 ktls_work_thread() at ktls_work_thread+0xdc/frame 0xfffffe00e4956bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4956bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4956bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100083 td 0xfffffe00e4cfd720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49659b0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49659d0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4965a10 _sleep() at _sleep+0x32f/frame 0xfffffe00e4965ab0 ktls_work_thread() at ktls_work_thread+0xdc/frame 0xfffffe00e4965bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4965bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4965bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100084 td 0xfffffe00e4cfd000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49609b0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49609d0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4960a10 _sleep() at _sleep+0x32f/frame 0xfffffe00e4960ab0 ktls_work_thread() at ktls_work_thread+0xdc/frame 0xfffffe00e4960bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4960bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4960bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100085 td 0xfffffe00e4d01740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e496f9b0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e496f9d0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e496fa10 _sleep() at _sleep+0x32f/frame 0xfffffe00e496fab0 ktls_work_thread() at ktls_work_thread+0xdc/frame 0xfffffe00e496fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e496fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e496fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100086 td 0xfffffe00e4d01020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e496a9b0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e496a9d0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e496aa10 _sleep() at _sleep+0x32f/frame 0xfffffe00e496aab0 ktls_work_thread() at ktls_work_thread+0xdc/frame 0xfffffe00e496abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e496abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e496abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100087 td 0xfffffe00e4d00900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49799b0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49799d0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4979a10 _sleep() at _sleep+0x32f/frame 0xfffffe00e4979ab0 ktls_work_thread() at ktls_work_thread+0xdc/frame 0xfffffe00e4979bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4979bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4979bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100088 td 0xfffffe00e4d051e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49749b0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49749d0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4974a10 _sleep() at _sleep+0x32f/frame 0xfffffe00e4974ab0 ktls_work_thread() at ktls_work_thread+0xdc/frame 0xfffffe00e4974bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4974bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4974bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 3 tid 100119 td 0xfffffe00e4d25020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e40a70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e40a90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e40ad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e40b70 crypto_proc() at crypto_proc+0xbf/frame 0xfffffe00e4e40bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e40bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e40bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 0 pid 4 tid 100120 td 0xfffffe00e4d24900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e3ba70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e3ba90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e3bad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e3bb70 crypto_ret_proc() at crypto_ret_proc+0x144/frame 0xfffffe00e4e3bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e3bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e3bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 1 pid 5 tid 100121 td 0xfffffe00e4d291e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e4aa70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e4aa90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e4aad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e4ab70 crypto_ret_proc() at crypto_ret_proc+0x144/frame 0xfffffe00e4e4abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e4abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e4abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 2 pid 6 tid 100122 td 0xfffffe00e4d28ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e45a70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e45a90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e45ad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e45b70 crypto_ret_proc() at crypto_ret_proc+0x144/frame 0xfffffe00e4e45bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e45bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e45bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 3 pid 7 tid 100123 td 0xfffffe00e4d283a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e54a70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e54a90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e54ad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e54b70 crypto_ret_proc() at crypto_ret_proc+0x144/frame 0xfffffe00e4e54bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e54bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e54bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 4 pid 8 tid 100124 td 0xfffffe00e4d27c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e4fa70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e4fa90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e4fad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e4fb70 crypto_ret_proc() at crypto_ret_proc+0x144/frame 0xfffffe00e4e4fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e4fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e4fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 5 pid 9 tid 100125 td 0xfffffe00e4d27560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e5ea70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e5ea90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e5ead0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e5eb70 crypto_ret_proc() at crypto_ret_proc+0x144/frame 0xfffffe00e4e5ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e5ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e5ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command audit pid 10 tid 100001 td 0xfffffe00e4c5c1e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe0037786a90 mi_switch() at mi_switch+0x155/frame 0xfffffe0037786ab0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0037786af0 _cv_wait() at _cv_wait+0x166/frame 0xfffffe0037786b50 audit_worker() at audit_worker+0x9f/frame 0xfffffe0037786bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037786bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037786bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100003 td 0xfffffe00e4c5b3a0 (CPU 0) kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe0037790890 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00377908a0 uart_intr_rxready() at uart_intr_rxready+0xf5/frame 0xfffffe00377908d0 uart_intr() at uart_intr+0x128/frame 0xfffffe0037790910 intr_event_handle() at intr_event_handle+0xde/frame 0xfffffe0037790960 intr_execute_handlers() at intr_execute_handlers+0x66/frame 0xfffffe0037790990 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe0037790990 --- interrupt, rip = 0xffffffff810a50a6, rsp = 0xfffffe0037790a60, rbp = 0xfffffe0037790a60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0037790a60 acpi_cpu_idle() at acpi_cpu_idle+0x288/frame 0xfffffe0037790aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe0037790ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe0037790ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe0037790bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037790bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037790bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xfffffe00e4c5ac80 (CPU 1) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce20e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce20e10 trap() at trap+0x56/frame 0xfffffe003ce20f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce20f20 --- trap 0x13, rip = 0xffffffff810a50a6, rsp = 0xfffffe003778ba60, rbp = 0xfffffe003778ba60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe003778ba60 acpi_cpu_idle() at acpi_cpu_idle+0x288/frame 0xfffffe003778baa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe003778bac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe003778bae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe003778bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003778bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003778bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffffe00e4c5a560 (CPU 2) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce2de00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce2de10 trap() at trap+0x56/frame 0xfffffe003ce2df20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce2df20 --- trap 0x13, rip = 0xffffffff810a50a6, rsp = 0xfffffe003779aa60, rbp = 0xfffffe003779aa60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe003779aa60 acpi_cpu_idle() at acpi_cpu_idle+0x288/frame 0xfffffe003779aaa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe003779aac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe003779aae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe003779abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003779abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003779abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xfffffe00e4c59e40 (CPU 3) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce3ae00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce3ae10 trap() at trap+0x56/frame 0xfffffe003ce3af20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce3af20 --- trap 0x13, rip = 0xffffffff810a50a6, rsp = 0xfffffe0037795a60, rbp = 0xfffffe0037795a60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0037795a60 acpi_cpu_idle() at acpi_cpu_idle+0x288/frame 0xfffffe0037795aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe0037795ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe0037795ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe0037795bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037795bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037795bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100007 td 0xfffffe00e4c59720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00377a4ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00377a4ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00377a4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377a4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377a4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xfffffe00e4c59000 (CPU 5) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce54e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce54e10 trap() at trap+0x56/frame 0xfffffe003ce54f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce54f20 --- trap 0x13, rip = 0xffffffff810a50a6, rsp = 0xfffffe003779fa60, rbp = 0xfffffe003779fa60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe003779fa60 acpi_cpu_idle() at acpi_cpu_idle+0x288/frame 0xfffffe003779faa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe003779fac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe003779fae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe003779fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003779fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003779fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xfffffe00e4c5d740 (CPU 6) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce61e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce61e10 trap() at trap+0x56/frame 0xfffffe003ce61f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce61f20 --- trap 0x13, rip = 0xffffffff810a50a6, rsp = 0xfffffe00377aea60, rbp = 0xfffffe00377aea60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00377aea60 acpi_cpu_idle() at acpi_cpu_idle+0x288/frame 0xfffffe00377aeaa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00377aeac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00377aeae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00377aebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377aebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377aebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xfffffe00e4c5d020 (CPU 7) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce6ee00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce6ee10 trap() at trap+0x56/frame 0xfffffe003ce6ef20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce6ef20 --- trap 0x13, rip = 0xffffffff810a50a6, rsp = 0xfffffe00377a9a60, rbp = 0xfffffe00377a9a60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00377a9a60 acpi_cpu_idle() at acpi_cpu_idle+0x288/frame 0xfffffe00377a9aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00377a9ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00377a9ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00377a9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377a9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377a9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffffe00e4c5c900 (CPU 8) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce7be00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce7be10 trap() at trap+0x56/frame 0xfffffe003ce7bf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce7bf20 --- trap 0x13, rip = 0xffffffff810a50a6, rsp = 0xfffffe00377b8a60, rbp = 0xfffffe00377b8a60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00377b8a60 acpi_cpu_idle() at acpi_cpu_idle+0x288/frame 0xfffffe00377b8aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00377b8ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00377b8ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00377b8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377b8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377b8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100012 td 0xfffffe00e4c611e0 (CPU 9) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce88e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce88e10 trap() at trap+0x56/frame 0xfffffe003ce88f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce88f20 --- trap 0x13, rip = 0xffffffff810a50a6, rsp = 0xfffffe00377b3a60, rbp = 0xfffffe00377b3a60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00377b3a60 acpi_cpu_idle() at acpi_cpu_idle+0x288/frame 0xfffffe00377b3aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00377b3ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00377b3ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00377b3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377b3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377b3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xfffffe00e4c60ac0 (CPU 10) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce95e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce95e10 trap() at trap+0x56/frame 0xfffffe003ce95f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce95f20 --- trap 0x13, rip = 0xffffffff810a50a6, rsp = 0xfffffe00377c2a60, rbp = 0xfffffe00377c2a60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00377c2a60 acpi_cpu_idle() at acpi_cpu_idle+0x288/frame 0xfffffe00377c2aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00377c2ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00377c2ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00377c2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377c2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377c2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100014 td 0xfffffe00e4c603a0 (CPU 11) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003cea2e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003cea2e10 trap() at trap+0x56/frame 0xfffffe003cea2f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003cea2f20 --- trap 0x13, rip = 0xffffffff810a50a6, rsp = 0xfffffe00377bda60, rbp = 0xfffffe00377bda60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00377bda60 acpi_cpu_idle() at acpi_cpu_idle+0x288/frame 0xfffffe00377bdaa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00377bdac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00377bdae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00377bdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377bdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377bdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100046 td 0xfffffe00e4cebac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e488ab00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e488ab20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e488abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e488abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e488abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100047 td 0xfffffe00e4ceb3a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4899b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4899b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4899bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4899bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4899bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100049 td 0xfffffe00e4cea560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100089 td 0xfffffe00e4d04ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4983b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4983b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4983bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4983bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4983bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100090 td 0xfffffe00e4d043a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e497eb00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e497eb20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e497ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e497ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e497ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100091 td 0xfffffe00e4d03c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e498db00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e498db20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e498dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e498dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e498dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100092 td 0xfffffe00e4d03560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4988b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4988b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4988bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4988bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4988bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100093 td 0xfffffe00e4d02e40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4997b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4997b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4997bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4997bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4997bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100094 td 0xfffffe00e4d02720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4992b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4992b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4992bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4992bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4992bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100095 td 0xfffffe00e4d02000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49a1b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49a1b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e49a1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49a1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49a1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100096 td 0xfffffe00e4d06740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e499cb00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e499cb20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e499cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e499cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e499cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100097 td 0xfffffe00e4d06020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49abb00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49abb20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e49abbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49abbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49abbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100098 td 0xfffffe00e4d05900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49a6b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49a6b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e49a6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49a6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49a6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100099 td 0xfffffe00e4d0a1e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49b5b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49b5b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e49b5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49b5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49b5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100100 td 0xfffffe00e4d09ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49b0b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49b0b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e49b0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49b0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49b0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100101 td 0xfffffe00e4d093a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49bfb00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49bfb20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e49bfbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49bfbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49bfbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100102 td 0xfffffe00e4d08c80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100137 td 0xfffffe00e4d3f740 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100138 td 0xfffffe00e4d3f020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e95b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e95b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4e95bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e95bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e95bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100139 td 0xfffffe00e4d3e900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4ea4b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4ea4b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4ea4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ea4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ea4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100145 td 0xfffffe010a18ae40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4ec2b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4ec2b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4ec2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ec2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ec2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100156 td 0xfffffe010a6d7e40 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100157 td 0xfffffe010a6d7720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f0ab00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f0ab20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4f0abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f0abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f0abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100175 td 0xfffffe010d9f2000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f94b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f94b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4f94bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f94bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f94bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100176 td 0xfffffe010df731e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100177 td 0xfffffe010df72ac0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100178 td 0xfffffe010df723a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100179 td 0xfffffe010df71c80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100180 td 0xfffffe010df71560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100181 td 0xfffffe010df70e40 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100182 td 0xfffffe010df70720 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100183 td 0xfffffe010df70000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100184 td 0xfffffe010df74740 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100185 td 0xfffffe010df74020 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100186 td 0xfffffe010df73900 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100187 td 0xfffffe010e01c1e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100188 td 0xfffffe010e01bac0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100189 td 0xfffffe010e01b3a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100195 td 0xfffffe010e01d740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4ff8b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4ff8b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4ff8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ff8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ff8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100196 td 0xfffffe010d9f5900 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100197 td 0xfffffe010ddde1e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100198 td 0xfffffe010ddddac0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100199 td 0xfffffe010dddd3a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100200 td 0xfffffe010dddcc80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100201 td 0xfffffe010dddc560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100202 td 0xfffffe010dddbe40 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100203 td 0xfffffe010dddb720 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100204 td 0xfffffe010dddb000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100205 td 0xfffffe010dddf740 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100206 td 0xfffffe010dddf020 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100207 td 0xfffffe010ddde900 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100208 td 0xfffffe010de831e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100209 td 0xfffffe010de82ac0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100264 td 0xfffffe010e3c3900 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100265 td 0xfffffe01435c01e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100266 td 0xfffffe01435bfac0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100267 td 0xfffffe01435bf3a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100268 td 0xfffffe01435bec80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100269 td 0xfffffe01435be560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100270 td 0xfffffe01435bde40 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100271 td 0xfffffe01435bd720 fork_trampoline() at fork_trampoline Tracing command geom pid 13 tid 100104 td 0xfffffe00e4d07e40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49e8a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49e8aa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e49e8ae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e49e8b80 g_run_events() at g_run_events+0x474/frame 0xfffffe00e49e8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49e8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49e8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100105 td 0xfffffe00e4d07720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49f7a70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49f7a90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e49f7ad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e49f7b70 g_io_schedule_up() at g_io_schedule_up+0x3c/frame 0xfffffe00e49f7ba0 g_up_procbody() at g_up_procbody+0x5c/frame 0xfffffe00e49f7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49f7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49f7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100106 td 0xfffffe00e4d07000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e49f2a70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e49f2a90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e49f2ad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e49f2b70 g_io_schedule_down() at g_io_schedule_down+0x4c/frame 0xfffffe00e49f2ba0 g_down_procbody() at g_down_procbody+0x5c/frame 0xfffffe00e49f2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49f2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49f2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 6 pid 14 tid 100126 td 0xfffffe00e4d26e40 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e59a70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e59a90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e59ad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e59b70 crypto_ret_proc() at crypto_ret_proc+0x144/frame 0xfffffe00e4e59bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e59bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e59bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 7 pid 15 tid 100127 td 0xfffffe00e4d26720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e68a70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e68a90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e68ad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e68b70 crypto_ret_proc() at crypto_ret_proc+0x144/frame 0xfffffe00e4e68bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e68bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e68bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 8 pid 16 tid 100128 td 0xfffffe00e4d26000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e63a70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e63a90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e63ad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e63b70 crypto_ret_proc() at crypto_ret_proc+0x144/frame 0xfffffe00e4e63bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e63bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e63bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 9 pid 17 tid 100129 td 0xfffffe00e4d2a740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e72a70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e72a90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e72ad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e72b70 crypto_ret_proc() at crypto_ret_proc+0x144/frame 0xfffffe00e4e72bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e72bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e72bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 10 pid 18 tid 100130 td 0xfffffe00e4d2a020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e6da70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e6da90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e6dad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e6db70 crypto_ret_proc() at crypto_ret_proc+0x144/frame 0xfffffe00e4e6dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e6dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e6dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 11 pid 19 tid 100131 td 0xfffffe00e4d29900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e7ca70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e7ca90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e7cad0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e7cb70 crypto_ret_proc() at crypto_ret_proc+0x144/frame 0xfffffe00e4e7cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e7cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e7cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sequencer 00 pid 20 tid 100132 td 0xfffffe00e4d3e1e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e77a70 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e77a90 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e77ad0 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4e77b30 seq_eventthread() at seq_eventthread+0x8c/frame 0xfffffe00e4e77bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e77bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e77bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 21 tid 100133 td 0xfffffe00e4d3dac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e86a60 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e86a80 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e86ac0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e86b60 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe00e4e86bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e86bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e86bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 21 tid 100134 td 0xfffffe00e4d3d3a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e81a60 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e81a80 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e81ac0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e81b60 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe00e4e81bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e81bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e81bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 21 tid 100135 td 0xfffffe00e4d3cc80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e90a60 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e90a80 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e90ac0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e90b60 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe00e4e90bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e90bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e90bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 21 tid 100136 td 0xfffffe00e4d3c560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e8ba60 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e8ba80 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e8bac0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4e8bb60 xpt_async_td() at xpt_async_td+0x74/frame 0xfffffe00e4e8bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e8bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e8bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 21 tid 100168 td 0xfffffe010d9f43a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f5ba80 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f5baa0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f5bae0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4f5bb80 xpt_scanner_thread() at xpt_scanner_thread+0x158/frame 0xfffffe00e4f5bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f5bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f5bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100140 td 0xfffffe010a18d1e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4e9fac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4e9fae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4e9fb20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4e9fb80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4e9fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e9fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e9fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100141 td 0xfffffe010a18cac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4eaeac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4eaeae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4eaeb20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4eaeb80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4eaebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4eaebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4eaebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100142 td 0xfffffe010a18c3a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4ea9ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4ea9ae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4ea9b20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4ea9b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4ea9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ea9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ea9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100143 td 0xfffffe010a18bc80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4eb8ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4eb8ae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4eb8b20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4eb8b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4eb8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4eb8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4eb8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100144 td 0xfffffe010a18b560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4eb3ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4eb3ae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4eb3b20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4eb3b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4eb3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4eb3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4eb3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100146 td 0xfffffe010a18a720 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4ebdac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4ebdae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4ebdb20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4ebdb80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4ebdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ebdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ebdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100147 td 0xfffffe010a18a000 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4eccac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4eccae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4eccb20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4eccb80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4eccbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4eccbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4eccbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100148 td 0xfffffe010a18e740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4ec7ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4ec7ae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4ec7b20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4ec7b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4ec7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ec7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ec7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100149 td 0xfffffe010a18e020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4ed6ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4ed6ae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4ed6b20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4ed6b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4ed6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ed6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ed6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100150 td 0xfffffe010a18d900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4ed1ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4ed1ae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4ed1b20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4ed1b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4ed1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ed1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ed1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100151 td 0xfffffe010a6da1e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4eecac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4eecae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4eecb20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4eecb80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4eecbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4eecbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4eecbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100152 td 0xfffffe010a6d9ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4ee7ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4ee7ae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4ee7b20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4ee7b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4ee7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ee7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ee7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100153 td 0xfffffe010a6d93a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4ef6ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4ef6ae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4ef6b20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4ef6b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4ef6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ef6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ef6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100154 td 0xfffffe010a6d8c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4ef1ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4ef1ae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4ef1b20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4ef1b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4ef1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ef1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ef1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100155 td 0xfffffe010a6d8560 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f00ac0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f00ae0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f00b20 _cv_wait() at _cv_wait+0x166/frame 0xfffffe00e4f00b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4f00bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f00bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f00bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod1 pid 23 tid 100159 td 0xfffffe010a6db740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f20a30 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f20a50 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f20a90 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e4f20ad0 _sleep() at _sleep+0x298/frame 0xfffffe00e4f20b70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe00e4f20bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f20bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f20bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod2 pid 24 tid 100160 td 0xfffffe010a6db020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f1ba30 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f1ba50 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f1ba90 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e4f1bad0 _sleep() at _sleep+0x298/frame 0xfffffe00e4f1bb70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe00e4f1bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f1bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f1bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod3 pid 25 tid 100161 td 0xfffffe010a6da900 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f2aa30 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f2aa50 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f2aa90 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e4f2aad0 _sleep() at _sleep+0x298/frame 0xfffffe00e4f2ab70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe00e4f2abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f2abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f2abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod4 pid 26 tid 100162 td 0xfffffe010d6731e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f25a30 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f25a50 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f25a90 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e4f25ad0 _sleep() at _sleep+0x298/frame 0xfffffe00e4f25b70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe00e4f25bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f25bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f25bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command rand_harvestq pid 27 tid 100163 td 0xfffffe010d674740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f4c9e0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f4ca00 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f4ca40 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e4f4ca80 _sleep() at _sleep+0x298/frame 0xfffffe00e4f4cb20 random_kthread() at random_kthread+0x87/frame 0xfffffe00e4f4cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f4cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f4cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon0 pid 28 tid 100169 td 0xfffffe010d9f6740 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f6aa40 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f6aa60 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f6aaa0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4f6ab40 enc_daemon() at enc_daemon+0xf6/frame 0xfffffe00e4f6abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f6abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f6abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon1 pid 29 tid 100170 td 0xfffffe010d9f6020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4f65a40 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4f65a60 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4f65aa0 _sleep() at _sleep+0x32f/frame 0xfffffe00e4f65b40 enc_daemon() at enc_daemon+0xf6/frame 0xfffffe00e4f65bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f65bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f65bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 30 tid 100211 td 0xfffffe010de81c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe010e645770 mi_switch() at mi_switch+0x155/frame 0xfffffe010e645790 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe010e6457d0 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe010e645810 _sleep() at _sleep+0x298/frame 0xfffffe010e6458b0 vm_pageout_worker() at vm_pageout_worker+0x29c/frame 0xfffffe010e645b70 vm_pageout() at vm_pageout+0x1d7/frame 0xfffffe010e645bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe010e645bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010e645bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 30 tid 100221 td 0xfffffe010e151740 sched_switch() at sched_switch+0x5af/frame 0xfffffe01142ee830 mi_switch() at mi_switch+0x155/frame 0xfffffe01142ee850 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01142ee890 _sleep() at _sleep+0x32f/frame 0xfffffe01142ee930 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0xb9c/frame 0xfffffe01142eebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe01142eebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01142eebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 30 tid 100222 td 0xfffffe010e151020 sched_switch() at sched_switch+0x5af/frame 0xfffffe0113ee5a40 mi_switch() at mi_switch+0x155/frame 0xfffffe0113ee5a60 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0113ee5aa0 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0113ee5ae0 _sleep() at _sleep+0x298/frame 0xfffffe0113ee5b80 uma_reclaim_worker() at uma_reclaim_worker+0xc8/frame 0xfffffe0113ee5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0113ee5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0113ee5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command vmdaemon pid 31 tid 100212 td 0xfffffe010de81560 sched_switch() at sched_switch+0x5af/frame 0xfffffe010e654a40 mi_switch() at mi_switch+0x155/frame 0xfffffe010e654a60 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe010e654aa0 _sleep() at _sleep+0x32f/frame 0xfffffe010e654b40 vm_daemon() at vm_daemon+0x82/frame 0xfffffe010e654bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe010e654bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010e654bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100213 td 0xfffffe010de80e40 sched_switch() at sched_switch+0x5af/frame 0xfffffe010e64fa20 mi_switch() at mi_switch+0x155/frame 0xfffffe010e64fa40 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe010e64fa80 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe010e64fac0 _sleep() at _sleep+0x298/frame 0xfffffe010e64fb60 buf_daemon() at buf_daemon+0x116/frame 0xfffffe010e64fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe010e64fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010e64fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100214 td 0xfffffe010de84740 sched_switch() at sched_switch+0x5af/frame 0xfffffe010e76ba10 mi_switch() at mi_switch+0x155/frame 0xfffffe010e76ba30 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe010e76ba70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe010e76bab0 _sleep() at _sleep+0x298/frame 0xfffffe010e76bb50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe010e76bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe010e76bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010e76bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100215 td 0xfffffe010de84020 sched_switch() at sched_switch+0x5af/frame 0xfffffe010e766a10 mi_switch() at mi_switch+0x155/frame 0xfffffe010e766a30 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe010e766a70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe010e766ab0 _sleep() at _sleep+0x298/frame 0xfffffe010e766b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe010e766bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe010e766bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010e766bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100216 td 0xfffffe010de83900 sched_switch() at sched_switch+0x5af/frame 0xfffffe010ee7ca10 mi_switch() at mi_switch+0x155/frame 0xfffffe010ee7ca30 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe010ee7ca70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe010ee7cab0 _sleep() at _sleep+0x298/frame 0xfffffe010ee7cb50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe010ee7cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe010ee7cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010ee7cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100217 td 0xfffffe010e1501e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe010eb74a10 mi_switch() at mi_switch+0x155/frame 0xfffffe010eb74a30 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe010eb74a70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe010eb74ab0 _sleep() at _sleep+0x298/frame 0xfffffe010eb74b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe010eb74bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe010eb74bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010eb74bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100218 td 0xfffffe010e14fac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe010fa92a10 mi_switch() at mi_switch+0x155/frame 0xfffffe010fa92a30 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe010fa92a70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe010fa92ab0 _sleep() at _sleep+0x298/frame 0xfffffe010fa92b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe010fa92bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe010fa92bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010fa92bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100219 td 0xfffffe010e14f3a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe010f98ca10 mi_switch() at mi_switch+0x155/frame 0xfffffe010f98ca30 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe010f98ca70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe010f98cab0 _sleep() at _sleep+0x298/frame 0xfffffe010f98cb50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe010f98cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe010f98cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010f98cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100220 td 0xfffffe010e14ec80 sched_switch() at sched_switch+0x5af/frame 0xfffffe01102a4a10 mi_switch() at mi_switch+0x155/frame 0xfffffe01102a4a30 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01102a4a70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe01102a4ab0 _sleep() at _sleep+0x298/frame 0xfffffe01102a4b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe01102a4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe01102a4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01102a4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100234 td 0xfffffe010e570020 sched_switch() at sched_switch+0x5af/frame 0xfffffe0141601a30 mi_switch() at mi_switch+0x155/frame 0xfffffe0141601a50 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0141601a90 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0141601ad0 _sleep() at _sleep+0x298/frame 0xfffffe0141601b70 softdep_flush() at softdep_flush+0x188/frame 0xfffffe0141601bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0141601bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0141601bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100241 td 0xfffffe010e3c2ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe01415d9a30 mi_switch() at mi_switch+0x155/frame 0xfffffe01415d9a50 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01415d9a90 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe01415d9ad0 _sleep() at _sleep+0x298/frame 0xfffffe01415d9b70 softdep_flush() at softdep_flush+0x188/frame 0xfffffe01415d9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe01415d9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01415d9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100734 td 0xfffffe0143ffe000 sched_switch() at sched_switch+0x5af/frame 0xfffffe014ea77a30 mi_switch() at mi_switch+0x155/frame 0xfffffe014ea77a50 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe014ea77a90 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe014ea77ad0 _sleep() at _sleep+0x298/frame 0xfffffe014ea77b70 softdep_flush() at softdep_flush+0x188/frame 0xfffffe014ea77bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe014ea77bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014ea77bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command vnlru pid 33 tid 100223 td 0xfffffe010de80720 sched_switch() at sched_switch+0x5af/frame 0xfffffe01415c89d0 mi_switch() at mi_switch+0x155/frame 0xfffffe01415c89f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01415c8a30 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe01415c8a70 _sleep() at _sleep+0x298/frame 0xfffffe01415c8b10 vnlru_proc() at vnlru_proc+0xa2c/frame 0xfffffe01415c8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe01415c8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01415c8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command syncer pid 34 tid 100224 td 0xfffffe010de80000 sched_switch() at sched_switch+0x5af/frame 0xfffffe01415c3a00 mi_switch() at mi_switch+0x155/frame 0xfffffe01415c3a20 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01415c3a60 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe01415c3aa0 _cv_timedwait_sbt() at _cv_timedwait_sbt+0x19b/frame 0xfffffe01415c3b20 sched_sync() at sched_sync+0x63d/frame 0xfffffe01415c3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe01415c3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01415c3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sh pid 26526 tid 100711 td 0xfffffe014e9ef560 sched_switch() at sched_switch+0x5af/frame 0xfffffe014eb216d0 mi_switch() at mi_switch+0x155/frame 0xfffffe014eb216f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe014eb21730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe014eb21780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe014eb21790 _sleep() at _sleep+0x325/frame 0xfffffe014eb21830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe014eb218c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe014eb21ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014eb21bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014eb21bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80120988a, rsp = 0x7fffffffe778, rbp = 0x7fffffffe820 --- Tracing command ls pid 26527 tid 100551 td 0xfffffe01496c1e40 sched_switch() at sched_switch+0x5af/frame 0xfffffe01498bd400 mi_switch() at mi_switch+0x155/frame 0xfffffe01498bd420 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01498bd460 sleeplk() at sleeplk+0x100/frame 0xfffffe01498bd4c0 lockmgr_slock_hard() at lockmgr_slock_hard+0x3d2/frame 0xfffffe01498bd560 ffs_lock() at ffs_lock+0x9d/frame 0xfffffe01498bd5a0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe01498bd5c0 _vn_lock() at _vn_lock+0x54/frame 0xfffffe01498bd620 vget_finish() at vget_finish+0x7a/frame 0xfffffe01498bd650 cache_lookup() at cache_lookup+0x7fe/frame 0xfffffe01498bd6b0 vfs_cache_lookup() at vfs_cache_lookup+0x7b/frame 0xfffffe01498bd700 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x5a/frame 0xfffffe01498bd720 lookup() at lookup+0x4d1/frame 0xfffffe01498bd7c0 namei() at namei+0x379/frame 0xfffffe01498bd870 kern_statat() at kern_statat+0x12d/frame 0xfffffe01498bd9c0 sys_fstatat() at sys_fstatat+0x2f/frame 0xfffffe01498bdac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01498bdbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01498bdbf0 --- syscall (552, FreeBSD ELF64, sys_fstatat), rip = 0x80123b3aa, rsp = 0x7fffffffe2d8, rbp = 0x7fffffffe390 --- Tracing command dhclient pid 2339 tid 100257 td 0xfffffe010e56f900 sched_switch() at sched_switch+0x5af/frame 0xfffffe0141698720 mi_switch() at mi_switch+0x155/frame 0xfffffe0141698740 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0141698780 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01416987d0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01416987e0 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe0141698840 seltdwait() at seltdwait+0x8d/frame 0xfffffe0141698880 kern_select() at kern_select+0xa4e/frame 0xfffffe0141698a80 sys_select() at sys_select+0x56/frame 0xfffffe0141698ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0141698bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0141698bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8011e672a, rsp = 0x7fffffffe938, rbp = 0x7fffffffe9f0 --- Tracing command dhclient pid 2342 tid 100248 td 0xfffffe010e59d1e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe0141615760 mi_switch() at mi_switch+0x155/frame 0xfffffe0141615780 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01416157c0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0141615810 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0141615820 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe0141615880 seltdwait() at seltdwait+0x8d/frame 0xfffffe01416158c0 kern_poll_kfds() at kern_poll_kfds+0x41b/frame 0xfffffe0141615950 kern_poll() at kern_poll+0x9f/frame 0xfffffe0141615aa0 sys_poll() at sys_poll+0x50/frame 0xfffffe0141615ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0141615bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0141615bf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8011e662a, rsp = 0x7fffffffeb78, rbp = 0x7fffffffebb0 --- Tracing command dhclient pid 2401 tid 100262 td 0xfffffe010d670720 sched_switch() at sched_switch+0x5af/frame 0xfffffe01416b1710 mi_switch() at mi_switch+0x155/frame 0xfffffe01416b1730 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01416b1770 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01416b17c0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe01416b1800 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19b/frame 0xfffffe01416b1880 seltdwait() at seltdwait+0x7f/frame 0xfffffe01416b18c0 kern_poll_kfds() at kern_poll_kfds+0x41b/frame 0xfffffe01416b1950 kern_poll() at kern_poll+0x9f/frame 0xfffffe01416b1aa0 sys_poll() at sys_poll+0x50/frame 0xfffffe01416b1ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01416b1bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01416b1bf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8011e662a, rsp = 0x7fffffffeb78, rbp = 0x7fffffffebb0 --- Tracing command devd pid 2406 tid 100274 td 0xfffffe0143588ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe014169d6d0 mi_switch() at mi_switch+0x155/frame 0xfffffe014169d6f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe014169d730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe014169d780 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe014169d7c0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19b/frame 0xfffffe014169d840 seltdwait() at seltdwait+0x7f/frame 0xfffffe014169d880 kern_select() at kern_select+0xa4e/frame 0xfffffe014169da80 sys_select() at sys_select+0x56/frame 0xfffffe014169dac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014169dbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014169dbf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x28c62a, rsp = 0x7fffffffcab8, rbp = 0x7fffffffec50 --- Tracing command syslogd pid 2595 tid 100290 td 0xfffffe010e56e3a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe014177a720 mi_switch() at mi_switch+0x155/frame 0xfffffe014177a740 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe014177a780 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe014177a7d0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe014177a7e0 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe014177a840 seltdwait() at seltdwait+0x8d/frame 0xfffffe014177a880 kern_select() at kern_select+0xa4e/frame 0xfffffe014177aa80 sys_select() at sys_select+0x56/frame 0xfffffe014177aac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014177abf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014177abf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8011cb72a, rsp = 0x7fffffffe6b8, rbp = 0x7fffffffec20 --- Tracing command watchdogd pid 2603 tid 100259 td 0xfffffe014353d020 sched_switch() at sched_switch+0x5af/frame 0xfffffe01416ac6d0 mi_switch() at mi_switch+0x155/frame 0xfffffe01416ac6f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01416ac730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01416ac780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01416ac790 _sleep() at _sleep+0x325/frame 0xfffffe01416ac830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe01416ac8c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe01416acac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01416acbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01416acbf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8011c188a, rsp = 0x7fffffffe968, rbp = 0x7fffffffea30 --- Tracing command rpcbind pid 2625 tid 100247 td 0xfffffe010e59cac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe0141670710 mi_switch() at mi_switch+0x155/frame 0xfffffe0141670730 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0141670770 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01416707c0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe0141670800 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19b/frame 0xfffffe0141670880 seltdwait() at seltdwait+0x7f/frame 0xfffffe01416708c0 kern_poll_kfds() at kern_poll_kfds+0x41b/frame 0xfffffe0141670950 kern_poll() at kern_poll+0x9f/frame 0xfffffe0141670aa0 sys_poll() at sys_poll+0x50/frame 0xfffffe0141670ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0141670bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0141670bf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8011bc62a, rsp = 0x7fffffffc978, rbp = 0x7fffffffeba0 --- Tracing command rpc.statd pid 2650 tid 100278 td 0xfffffe0143589900 sched_switch() at sched_switch+0x5af/frame 0xfffffe014171b6d0 mi_switch() at mi_switch+0x155/frame 0xfffffe014171b6f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe014171b730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe014171b780 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe014171b7c0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19b/frame 0xfffffe014171b840 seltdwait() at seltdwait+0x7f/frame 0xfffffe014171b880 kern_select() at kern_select+0xa4e/frame 0xfffffe014171ba80 sys_select() at sys_select+0x56/frame 0xfffffe014171bac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014171bbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014171bbf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8011b672a, rsp = 0x7fffffffe9f8, rbp = 0x7fffffffeb40 --- Tracing command rpc.lockd pid 2653 tid 100280 td 0xfffffe01435f91e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe0141725640 mi_switch() at mi_switch+0x155/frame 0xfffffe0141725660 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01417256a0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01417256f0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe0141725730 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19b/frame 0xfffffe01417257b0 svc_run_internal() at svc_run_internal+0xac1/frame 0xfffffe01417258e0 svc_run() at svc_run+0x250/frame 0xfffffe0141725940 sys_nlm_syscall() at sys_nlm_syscall+0x74a/frame 0xfffffe0141725ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0141725bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0141725bf0 --- syscall (154, FreeBSD ELF64, sys_nlm_syscall), rip = 0x8011bef0a, rsp = 0x7fffffffeb58, rbp = 0x7fffffffec20 --- Tracing command powerd pid 2658 tid 100279 td 0xfffffe010d670000 sched_switch() at sched_switch+0x5af/frame 0xfffffe014172a6d0 mi_switch() at mi_switch+0x155/frame 0xfffffe014172a6f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe014172a730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe014172a780 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe014172a7c0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19b/frame 0xfffffe014172a840 seltdwait() at seltdwait+0x7f/frame 0xfffffe014172a880 kern_select() at kern_select+0xa4e/frame 0xfffffe014172aa80 sys_select() at sys_select+0x56/frame 0xfffffe014172aac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014172abf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014172abf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8011c172a, rsp = 0x7fffffffeaf8, rbp = 0x7fffffffec30 --- Tracing command ntpd pid 2684 tid 100297 td 0xfffffe010e59a000 sched_switch() at sched_switch+0x5af/frame 0xfffffe01417a2720 mi_switch() at mi_switch+0x155/frame 0xfffffe01417a2740 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01417a2780 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01417a27d0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01417a27e0 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01417a2840 seltdwait() at seltdwait+0x8d/frame 0xfffffe01417a2880 kern_select() at kern_select+0xa4e/frame 0xfffffe01417a2a80 sys_select() at sys_select+0x56/frame 0xfffffe01417a2ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01417a2bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01417a2bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8015db72a, rsp = 0x7fffffffde58, rbp = 0x7fffffffde90 --- Tracing command inetd pid 2710 tid 100295 td 0xfffffe010e59a720 sched_switch() at sched_switch+0x5af/frame 0xfffffe014174d720 mi_switch() at mi_switch+0x155/frame 0xfffffe014174d740 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe014174d780 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe014174d7d0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe014174d7e0 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe014174d840 seltdwait() at seltdwait+0x8d/frame 0xfffffe014174d880 kern_select() at kern_select+0xa4e/frame 0xfffffe014174da80 sys_select() at sys_select+0x56/frame 0xfffffe014174dac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014174dbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014174dbf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8011e072a, rsp = 0x7fffffffdd78, rbp = 0x7fffffffec10 --- Tracing command sshd pid 2729 tid 100298 td 0xfffffe01435f83a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe0141793720 mi_switch() at mi_switch+0x155/frame 0xfffffe0141793740 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0141793780 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01417937d0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01417937e0 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe0141793840 seltdwait() at seltdwait+0x8d/frame 0xfffffe0141793880 kern_select() at kern_select+0xa4e/frame 0xfffffe0141793a80 sys_select() at sys_select+0x56/frame 0xfffffe0141793ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0141793bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0141793bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80169972a, rsp = 0x7fffffffe5c8, rbp = 0x7fffffffe600 --- Tracing command cron pid 2733 tid 100229 td 0xfffffe010e56f1e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe01415f2870 mi_switch() at mi_switch+0x155/frame 0xfffffe01415f2890 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01415f28d0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01415f2920 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe01415f2960 _sleep() at _sleep+0x285/frame 0xfffffe01415f2a00 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e1/frame 0xfffffe01415f2a80 sys_nanosleep() at sys_nanosleep+0x3b/frame 0xfffffe01415f2ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01415f2bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01415f2bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8011d85ca, rsp = 0x7fffffffea48, rbp = 0x7fffffffec20 --- Tracing command getty pid 2747 tid 100225 td 0xfffffe010e3c31e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe01415de7c0 mi_switch() at mi_switch+0x155/frame 0xfffffe01415de7e0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01415de820 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01415de870 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01415de880 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01415de8e0 tty_wait() at tty_wait+0x42/frame 0xfffffe01415de910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe01415de970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01415de9a0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe01415dea00 dofileread() at dofileread+0x81/frame 0xfffffe01415dea50 sys_read() at sys_read+0xc0/frame 0xfffffe01415deac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01415debf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01415debf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c56aa, rsp = 0x7fffffffeb58, rbp = 0x7fffffffeba0 --- Tracing command getty pid 2748 tid 100288 td 0xfffffe01435c0900 sched_switch() at sched_switch+0x5af/frame 0xfffffe01417707c0 mi_switch() at mi_switch+0x155/frame 0xfffffe01417707e0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0141770820 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0141770870 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0141770880 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01417708e0 tty_wait() at tty_wait+0x42/frame 0xfffffe0141770910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe0141770970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01417709a0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe0141770a00 dofileread() at dofileread+0x81/frame 0xfffffe0141770a50 sys_read() at sys_read+0xc0/frame 0xfffffe0141770ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0141770bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0141770bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c56aa, rsp = 0x7fffffffeb58, rbp = 0x7fffffffeba0 --- Tracing command getty pid 2749 tid 100296 td 0xfffffe01435f8ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe01417987c0 mi_switch() at mi_switch+0x155/frame 0xfffffe01417987e0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0141798820 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0141798870 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0141798880 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01417988e0 tty_wait() at tty_wait+0x42/frame 0xfffffe0141798910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe0141798970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01417989a0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe0141798a00 dofileread() at dofileread+0x81/frame 0xfffffe0141798a50 sys_read() at sys_read+0xc0/frame 0xfffffe0141798ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0141798bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0141798bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c56aa, rsp = 0x7fffffffeb58, rbp = 0x7fffffffeba0 --- Tracing command getty pid 2750 tid 100289 td 0xfffffe0143e53740 sched_switch() at sched_switch+0x5af/frame 0xfffffe014176b7c0 mi_switch() at mi_switch+0x155/frame 0xfffffe014176b7e0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe014176b820 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe014176b870 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe014176b880 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe014176b8e0 tty_wait() at tty_wait+0x42/frame 0xfffffe014176b910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe014176b970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe014176b9a0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe014176ba00 dofileread() at dofileread+0x81/frame 0xfffffe014176ba50 sys_read() at sys_read+0xc0/frame 0xfffffe014176bac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014176bbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014176bbf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c56aa, rsp = 0x7fffffffeb58, rbp = 0x7fffffffeba0 --- Tracing command getty pid 2751 tid 100284 td 0xfffffe01435fa020 sched_switch() at sched_switch+0x5af/frame 0xfffffe01417437c0 mi_switch() at mi_switch+0x155/frame 0xfffffe01417437e0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0141743820 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0141743870 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0141743880 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01417438e0 tty_wait() at tty_wait+0x42/frame 0xfffffe0141743910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe0141743970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01417439a0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe0141743a00 dofileread() at dofileread+0x81/frame 0xfffffe0141743a50 sys_read() at sys_read+0xc0/frame 0xfffffe0141743ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0141743bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0141743bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c56aa, rsp = 0x7fffffffeb58, rbp = 0x7fffffffeba0 --- Tracing command getty pid 2752 tid 100301 td 0xfffffe0143f391e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe01417b67c0 mi_switch() at mi_switch+0x155/frame 0xfffffe01417b67e0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01417b6820 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01417b6870 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01417b6880 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01417b68e0 tty_wait() at tty_wait+0x42/frame 0xfffffe01417b6910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe01417b6970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01417b69a0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe01417b6a00 dofileread() at dofileread+0x81/frame 0xfffffe01417b6a50 sys_read() at sys_read+0xc0/frame 0xfffffe01417b6ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01417b6bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01417b6bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c56aa, rsp = 0x7fffffffeb58, rbp = 0x7fffffffeba0 --- Tracing command getty pid 2753 tid 100302 td 0xfffffe0143f38ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe01417b17c0 mi_switch() at mi_switch+0x155/frame 0xfffffe01417b17e0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01417b1820 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01417b1870 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01417b1880 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01417b18e0 tty_wait() at tty_wait+0x42/frame 0xfffffe01417b1910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe01417b1970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01417b19a0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe01417b1a00 dofileread() at dofileread+0x81/frame 0xfffffe01417b1a50 sys_read() at sys_read+0xc0/frame 0xfffffe01417b1ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01417b1bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01417b1bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c56aa, rsp = 0x7fffffffeb58, rbp = 0x7fffffffeba0 --- Tracing command getty pid 2754 tid 100303 td 0xfffffe0143f383a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe01417c07c0 mi_switch() at mi_switch+0x155/frame 0xfffffe01417c07e0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01417c0820 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01417c0870 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01417c0880 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01417c08e0 tty_wait() at tty_wait+0x42/frame 0xfffffe01417c0910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe01417c0970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01417c09a0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe01417c0a00 dofileread() at dofileread+0x81/frame 0xfffffe01417c0a50 sys_read() at sys_read+0xc0/frame 0xfffffe01417c0ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01417c0bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01417c0bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c56aa, rsp = 0x7fffffffeb58, rbp = 0x7fffffffeba0 --- Tracing command getty pid 2755 tid 100304 td 0xfffffe0143f37c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe01417bb7c0 mi_switch() at mi_switch+0x155/frame 0xfffffe01417bb7e0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01417bb820 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01417bb870 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01417bb880 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01417bb8e0 tty_wait() at tty_wait+0x42/frame 0xfffffe01417bb910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe01417bb970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01417bb9a0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe01417bba00 dofileread() at dofileread+0x81/frame 0xfffffe01417bba50 sys_read() at sys_read+0xc0/frame 0xfffffe01417bbac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01417bbbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01417bbbf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c56aa, rsp = 0x7fffffffeb58, rbp = 0x7fffffffeba0 --- Tracing command login pid 2756 tid 100305 td 0xfffffe0143f37560 sched_switch() at sched_switch+0x5af/frame 0xfffffe01417ca6d0 mi_switch() at mi_switch+0x155/frame 0xfffffe01417ca6f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01417ca730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01417ca780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01417ca790 _sleep() at _sleep+0x325/frame 0xfffffe01417ca830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe01417ca8c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe01417caac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01417cabf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01417cabf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8011f188a, rsp = 0x7fffffffeab8, rbp = 0x7fffffffec90 --- Tracing command sshd pid 2757 tid 100237 td 0xfffffe010e14d720 sched_switch() at sched_switch+0x5af/frame 0xfffffe014160b760 mi_switch() at mi_switch+0x155/frame 0xfffffe014160b780 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe014160b7c0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe014160b810 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe014160b820 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe014160b880 seltdwait() at seltdwait+0x8d/frame 0xfffffe014160b8c0 kern_poll_kfds() at kern_poll_kfds+0x41b/frame 0xfffffe014160b950 kern_poll() at kern_poll+0x9f/frame 0xfffffe014160baa0 sys_poll() at sys_poll+0x50/frame 0xfffffe014160bac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014160bbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014160bbf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x80169962a, rsp = 0x7fffffffe538, rbp = 0x7fffffffe570 --- Tracing command sshd pid 2760 tid 100226 td 0xfffffe010e01d020 sched_switch() at sched_switch+0x5af/frame 0xfffffe00e4ff3720 mi_switch() at mi_switch+0x155/frame 0xfffffe00e4ff3740 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00e4ff3780 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe00e4ff37d0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e4ff37e0 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe00e4ff3840 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e4ff3880 kern_select() at kern_select+0xa4e/frame 0xfffffe00e4ff3a80 sys_select() at sys_select+0x56/frame 0xfffffe00e4ff3ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe00e4ff3bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00e4ff3bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80169972a, rsp = 0x7fffffffa4c8, rbp = 0x7fffffffa500 --- Tracing command bash pid 2761 tid 100230 td 0xfffffe00e4d3be40 sched_switch() at sched_switch+0x5af/frame 0xfffffe01415fc6d0 mi_switch() at mi_switch+0x155/frame 0xfffffe01415fc6f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01415fc730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01415fc780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01415fc790 _sleep() at _sleep+0x325/frame 0xfffffe01415fc830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe01415fc8c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe01415fcac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01415fcbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01415fcbf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80052a88a, rsp = 0x7fffffffe738, rbp = 0x7fffffffe790 --- Tracing command su pid 2766 tid 100231 td 0xfffffe00e4d3b720 sched_switch() at sched_switch+0x5af/frame 0xfffffe01415f76d0 mi_switch() at mi_switch+0x155/frame 0xfffffe01415f76f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01415f7730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01415f7780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01415f7790 _sleep() at _sleep+0x325/frame 0xfffffe01415f7830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe01415f78c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe01415f7ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01415f7bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01415f7bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8011ef88a, rsp = 0x7fffffffe578, rbp = 0x7fffffffead0 --- Tracing command csh pid 2767 tid 100244 td 0xfffffe010e56eac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe01415ed860 mi_switch() at mi_switch+0x155/frame 0xfffffe01415ed880 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01415ed8c0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01415ed910 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01415ed920 _sleep() at _sleep+0x325/frame 0xfffffe01415ed9c0 kern_sigsuspend() at kern_sigsuspend+0x1ec/frame 0xfffffe01415eda90 sys_sigsuspend() at sys_sigsuspend+0x31/frame 0xfffffe01415edac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01415edbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01415edbf0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x8012937ea, rsp = 0x7fffffffdf78, rbp = 0x7fffffffe000 --- Tracing command bash pid 2769 tid 100256 td 0xfffffe010d671560 sched_switch() at sched_switch+0x5af/frame 0xfffffe01416896d0 mi_switch() at mi_switch+0x155/frame 0xfffffe01416896f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0141689730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0141689780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0141689790 _sleep() at _sleep+0x325/frame 0xfffffe0141689830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe01416898c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0141689ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0141689bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0141689bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80052a88a, rsp = 0x7fffffffe5f8, rbp = 0x7fffffffe650 --- Tracing command sh pid 2852 tid 100314 td 0xfffffe0143f36720 sched_switch() at sched_switch+0x5af/frame 0xfffffe01417f26d0 mi_switch() at mi_switch+0x155/frame 0xfffffe01417f26f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01417f2730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01417f2780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01417f2790 _sleep() at _sleep+0x325/frame 0xfffffe01417f2830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe01417f28c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe01417f2ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01417f2bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01417f2bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80120988a, rsp = 0x7fffffffe3d8, rbp = 0x7fffffffe480 --- Tracing command sh pid 2899 tid 100315 td 0xfffffe0143f5d740 sched_switch() at sched_switch+0x5af/frame 0xfffffe01417fc6d0 mi_switch() at mi_switch+0x155/frame 0xfffffe01417fc6f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01417fc730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01417fc780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01417fc790 _sleep() at _sleep+0x325/frame 0xfffffe01417fc830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe01417fc8c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe01417fcac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01417fcbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01417fcbf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80120988a, rsp = 0x7fffffffe108, rbp = 0x7fffffffe1b0 --- Tracing command awk pid 2900 tid 100293 td 0xfffffe010e56dc80 sched_switch() at sched_switch+0x5af/frame 0xfffffe0141775840 mi_switch() at mi_switch+0x155/frame 0xfffffe0141775860 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01417758a0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01417758f0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0141775900 _sleep() at _sleep+0x325/frame 0xfffffe01417759a0 pipe_read() at pipe_read+0x4c6/frame 0xfffffe0141775a00 dofileread() at dofileread+0x81/frame 0xfffffe0141775a50 sys_read() at sys_read+0xc0/frame 0xfffffe0141775ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0141775bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0141775bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8012166aa, rsp = 0x7fffffffe1f8, rbp = 0x7fffffffe210 --- Tracing command sh pid 2902 tid 100312 td 0xfffffe010e3c1560 sched_switch() at sched_switch+0x5af/frame 0xfffffe01417e8840 mi_switch() at mi_switch+0x155/frame 0xfffffe01417e8860 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01417e88a0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01417e88f0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01417e8900 _sleep() at _sleep+0x325/frame 0xfffffe01417e89a0 pipe_read() at pipe_read+0x4c6/frame 0xfffffe01417e8a00 dofileread() at dofileread+0x81/frame 0xfffffe01417e8a50 sys_read() at sys_read+0xc0/frame 0xfffffe01417e8ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01417e8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01417e8bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8012096aa, rsp = 0x7fffffffdcc8, rbp = 0x7fffffffe1f0 --- Tracing command csh pid 68512 tid 100371 td 0xfffffe014406fe40 sched_switch() at sched_switch+0x5af/frame 0xfffffe01445457c0 mi_switch() at mi_switch+0x155/frame 0xfffffe01445457e0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0144545820 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0144545870 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0144545880 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01445458e0 tty_wait() at tty_wait+0x42/frame 0xfffffe0144545910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe0144545970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01445459a0 devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe0144545a00 dofileread() at dofileread+0x81/frame 0xfffffe0144545a50 sys_read() at sys_read+0xc0/frame 0xfffffe0144545ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0144545bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0144545bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8012936aa, rsp = 0x7fffffffe158, rbp = 0x7fffffffe180 --- Tracing command ls pid 60494 tid 100306 td 0xfffffe010e14d000 sched_switch() at sched_switch+0x5af/frame 0xfffffe01417d4400 mi_switch() at mi_switch+0x155/frame 0xfffffe01417d4420 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01417d4460 sleeplk() at sleeplk+0x100/frame 0xfffffe01417d44c0 lockmgr_slock_hard() at lockmgr_slock_hard+0x3d2/frame 0xfffffe01417d4560 ffs_lock() at ffs_lock+0x9d/frame 0xfffffe01417d45a0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe01417d45c0 _vn_lock() at _vn_lock+0x54/frame 0xfffffe01417d4620 vget_finish() at vget_finish+0x7a/frame 0xfffffe01417d4650 cache_lookup() at cache_lookup+0x7fe/frame 0xfffffe01417d46b0 vfs_cache_lookup() at vfs_cache_lookup+0x7b/frame 0xfffffe01417d4700 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x5a/frame 0xfffffe01417d4720 lookup() at lookup+0x4d1/frame 0xfffffe01417d47c0 namei() at namei+0x379/frame 0xfffffe01417d4870 kern_statat() at kern_statat+0x12d/frame 0xfffffe01417d49c0 sys_fstatat() at sys_fstatat+0x2f/frame 0xfffffe01417d4ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01417d4bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01417d4bf0 --- syscall (552, FreeBSD ELF64, sys_fstatat), rip = 0x80123b3aa, rsp = 0x7fffffffe088, rbp = 0x7fffffffe140 --- Tracing command sleep pid 70861 tid 100378 td 0xfffffe0144091ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe014455e870 mi_switch() at mi_switch+0x155/frame 0xfffffe014455e890 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe014455e8d0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe014455e920 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe014455e960 _sleep() at _sleep+0x285/frame 0xfffffe014455ea00 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e1/frame 0xfffffe014455ea80 sys_nanosleep() at sys_nanosleep+0x3b/frame 0xfffffe014455eac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014455ebf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014455ebf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8011a75ca, rsp = 0x7fffffffe5b8, rbp = 0x7fffffffe610 --- Tracing command sshd pid 14207 tid 100349 td 0xfffffe0144061c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe01444c8760 mi_switch() at mi_switch+0x155/frame 0xfffffe01444c8780 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01444c87c0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01444c8810 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01444c8820 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01444c8880 seltdwait() at seltdwait+0x8d/frame 0xfffffe01444c88c0 kern_poll_kfds() at kern_poll_kfds+0x41b/frame 0xfffffe01444c8950 kern_poll() at kern_poll+0x9f/frame 0xfffffe01444c8aa0 sys_poll() at sys_poll+0x50/frame 0xfffffe01444c8ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01444c8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01444c8bf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x80169962a, rsp = 0x7fffffffe538, rbp = 0x7fffffffe570 --- Tracing command sshd pid 14217 tid 100445 td 0xfffffe014410a740 sched_switch() at sched_switch+0x5af/frame 0xfffffe01456a9720 mi_switch() at mi_switch+0x155/frame 0xfffffe01456a9740 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01456a9780 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01456a97d0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01456a97e0 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe01456a9840 seltdwait() at seltdwait+0x8d/frame 0xfffffe01456a9880 kern_select() at kern_select+0xa4e/frame 0xfffffe01456a9a80 sys_select() at sys_select+0x56/frame 0xfffffe01456a9ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01456a9bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01456a9bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80169972a, rsp = 0x7fffffffa4c8, rbp = 0x7fffffffa500 --- Tracing command bash pid 14218 tid 100307 td 0xfffffe010e3c1c80 sched_switch() at sched_switch+0x5af/frame 0xfffffe01416576d0 mi_switch() at mi_switch+0x155/frame 0xfffffe01416576f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0141657730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0141657780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0141657790 _sleep() at _sleep+0x325/frame 0xfffffe0141657830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe01416578c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0141657ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0141657bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0141657bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80052a88a, rsp = 0x7fffffffe738, rbp = 0x7fffffffe790 --- Tracing command su pid 22872 tid 100367 td 0xfffffe0144072900 sched_switch() at sched_switch+0x5af/frame 0xfffffe01445226d0 mi_switch() at mi_switch+0x155/frame 0xfffffe01445226f0 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0144522730 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0144522780 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0144522790 _sleep() at _sleep+0x325/frame 0xfffffe0144522830 kern_wait6() at kern_wait6+0x474/frame 0xfffffe01445228c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0144522ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0144522bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0144522bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8011ef88a, rsp = 0x7fffffffe578, rbp = 0x7fffffffead0 --- Tracing command csh pid 22873 tid 100530 td 0xfffffe01497811e0 sched_switch() at sched_switch+0x5af/frame 0xfffffe0149854860 mi_switch() at mi_switch+0x155/frame 0xfffffe0149854880 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01498548c0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0149854910 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0149854920 _sleep() at _sleep+0x325/frame 0xfffffe01498549c0 kern_sigsuspend() at kern_sigsuspend+0x1ec/frame 0xfffffe0149854a90 sys_sigsuspend() at sys_sigsuspend+0x31/frame 0xfffffe0149854ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0149854bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0149854bf0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x8012937ea, rsp = 0x7fffffffdf78, rbp = 0x7fffffffe000 --- Tracing command bash pid 22903 tid 100328 td 0xfffffe0143f59720 sched_switch() at sched_switch+0x5af/frame 0xfffffe01444366e0 mi_switch() at mi_switch+0x155/frame 0xfffffe0144436700 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0144436740 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0144436790 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01444367a0 _cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe0144436800 seltdwait() at seltdwait+0x8d/frame 0xfffffe0144436840 kern_select() at kern_select+0xa4e/frame 0xfffffe0144436a40 sys_pselect() at sys_pselect+0x127/frame 0xfffffe0144436ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0144436bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0144436bf0 --- syscall (522, FreeBSD ELF64, sys_pselect), rip = 0x80052a66a, rsp = 0x7fffffffd5a8, rbp = 0x7fffffffd670 --- Tracing command ls pid 47808 tid 100493 td 0xfffffe01471bc3a0 sched_switch() at sched_switch+0x5af/frame 0xfffffe0145749400 mi_switch() at mi_switch+0x155/frame 0xfffffe0145749420 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0145749460 sleeplk() at sleeplk+0x100/frame 0xfffffe01457494c0 lockmgr_slock_hard() at lockmgr_slock_hard+0x3d2/frame 0xfffffe0145749560 ffs_lock() at ffs_lock+0x9d/frame 0xfffffe01457495a0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe01457495c0 _vn_lock() at _vn_lock+0x54/frame 0xfffffe0145749620 vget_finish() at vget_finish+0x7a/frame 0xfffffe0145749650 cache_lookup() at cache_lookup+0x7fe/frame 0xfffffe01457496b0 vfs_cache_lookup() at vfs_cache_lookup+0x7b/frame 0xfffffe0145749700 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x5a/frame 0xfffffe0145749720 lookup() at lookup+0x4d1/frame 0xfffffe01457497c0 namei() at namei+0x379/frame 0xfffffe0145749870 kern_statat() at kern_statat+0x12d/frame 0xfffffe01457499c0 sys_fstatat() at sys_fstatat+0x2f/frame 0xfffffe0145749ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0145749bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0145749bf0 --- syscall (552, FreeBSD ELF64, sys_fstatat), rip = 0x80123b3aa, rsp = 0x7fffffffdd48, rbp = 0x7fffffffde00 --- Tracing command ls pid 24211 tid 100557 td 0xfffffe0149780ac0 sched_switch() at sched_switch+0x5af/frame 0xfffffe01498db400 mi_switch() at mi_switch+0x155/frame 0xfffffe01498db420 sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe01498db460 sleeplk() at sleeplk+0x100/frame 0xfffffe01498db4c0 lockmgr_slock_hard() at lockmgr_slock_hard+0x3d2/frame 0xfffffe01498db560 ffs_lock() at ffs_lock+0x9d/frame 0xfffffe01498db5a0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x3c/frame 0xfffffe01498db5c0 _vn_lock() at _vn_lock+0x54/frame 0xfffffe01498db620 vget_finish() at vget_finish+0x7a/frame 0xfffffe01498db650 cache_lookup() at cache_lookup+0x7fe/frame 0xfffffe01498db6b0 vfs_cache_lookup() at vfs_cache_lookup+0x7b/frame 0xfffffe01498db700 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x5a/frame 0xfffffe01498db720 lookup() at lookup+0x4d1/frame 0xfffffe01498db7c0 namei() at namei+0x379/frame 0xfffffe01498db870 kern_statat() at kern_statat+0x12d/frame 0xfffffe01498db9c0 sys_fstatat() at sys_fstatat+0x2f/frame 0xfffffe01498dbac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01498dbbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01498dbbf0 --- syscall (552, FreeBSD ELF64, sys_fstatat), rip = 0x80123b3aa, rsp = 0x7fffffffdd08, rbp = 0x7fffffffddc0 --- db> (kgdb) thread 31 [Switching to thread 31 (Thread 100044)] #0 cpustop_handler () at ../../../x86/x86/mp_x86.c:1479 1479 use_mwait = (stop_mwait && (cpu_feature2 & CPUID2_MON) != 0 && (kgdb) bt #0 cpustop_handler () at ../../../x86/x86/mp_x86.c:1479 #1 0xffffffff810af619 in ipi_nmi_handler () at ../../../x86/x86/mp_x86.c:1436 #2 0xffffffff810e9426 in trap (frame=0xfffffe003ce47f30) at ../../../amd64/amd64/trap.c:238 #3 #4 ffs_sync (mp=0xfffffe0143fe4b80, waitfor=0x1) at ../../../ufs/ffs/ffs_vfsops.c:1842 #5 0xffffffff80d2a005 in vfs_write_suspend (mp=mp@entry=0xfffffe0143fe4b80, flags=flags@entry=0x0) at ../../../kern/vfs_vnops.c:2097 #6 0xffffffff80d2a1c5 in vfs_write_suspend_umnt (mp=mp@entry=0xfffffe0143fe4b80) at ../../../kern/vfs_vnops.c:2151 #7 0xffffffff80f4c048 in ffs_unmount (mp=0xfffffe0143fe4b80, mntflags=0x80000) at ../../../ufs/ffs/ffs_vfsops.c:1488 #8 0xffffffff80d07102 in dounmount (mp=0x2, mp@entry=0xfffffe0143fe4b80, flags=flags@entry=0x300000080000, td=0xfffffe00e4ce5900) at ../../../kern/vfs_mount.c:2139 #9 0xffffffff80d092c9 in vfs_deferred_unmount (argi=, pending=) at ../../../kern/vfs_mount.c:1931 #10 0xffffffff80c849ca in taskqueue_run_locked (queue=queue@entry=0xfffff80003b1f700) at ../../../kern/subr_taskqueue.c:476 #11 0xffffffff80c85a52 in taskqueue_thread_loop (arg=arg@entry=0xffffffff81faaad8 ) at ../../../kern/subr_taskqueue.c:793 #12 0xffffffff80bdbba0 in fork_exit (callout=0xffffffff80c85990 , arg=0xffffffff81faaad8 , frame=0xfffffe00e4880c00) at ../../../kern/kern_fork.c:1083 #13 (kgdb) f 5 #5 0xffffffff80d2a005 in vfs_write_suspend (mp=mp@entry=0xfffffe0143fe4b80, flags=flags@entry=0x0) at ../../../kern/vfs_vnops.c:2097 2097 if ((error = VFS_SYNC(mp, MNT_SUSPEND)) != 0) { (kgdb) l 2092 if (mp->mnt_writeopcount > 0) 2093 (void) msleep(&mp->mnt_writeopcount, 2094 MNT_MTX(mp), (PUSER - 1)|PDROP, "suspwt", 0); 2095 else 2096 MNT_IUNLOCK(mp); 2097 if ((error = VFS_SYNC(mp, MNT_SUSPEND)) != 0) { 2098 vfs_write_resume(mp, 0); 2099 /* vfs_write_resume does vfs_op_exit() for us */ 2100 } 2101 return (error); (kgdb) info loc _rc = error = (kgdb) p *mp $1 = {mnt_vfs_ops = 0x2, mnt_kern_flag = 0x49846345, mnt_flag = 0x100201000, mnt_pcpu = 0xfffffe01c3171fe0, mnt_rootvnode = 0x0, mnt_vnodecovered = 0xfffff80111a5abe0, mnt_op = 0xffffffff81b64eb0 , mnt_vfc = 0xffffffff81b64fd0 , mnt_mtx = {lock_object = { lo_name = 0xffffffff811c07ec "struct mount mtx", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xfffff8087fd75b80}, mtx_lock = 0x0}, mnt_gen = 0x5, mnt_list = {tqe_next = 0x0, tqe_prev = 0xfffffe010e5c92e8}, mnt_syncer = 0x0, mnt_ref = 0x803e, mnt_nvnodelist = { tqh_first = 0xfffff805b7b93be0, tqh_last = 0xfffff804d8ad7028}, mnt_nvnodelistsize = 0x803c, mnt_writeopcount = 0x0, mnt_opt = 0xfffff800099daf80, mnt_optnew = 0x0, mnt_stat = {f_version = 0x20140518, f_type = 0x35, f_flags = 0x100201000, f_bsize = 0x1000, f_iosize = 0x8000, f_blocks = 0x3df87, f_bfree = 0x3d776, f_bavail = 0x38824, f_files = 0x201fe, f_ffree = 0x201fb, f_syncwrites = 0x5991, f_asyncwrites = 0x8bfc, f_syncreads = 0x7cc, f_asyncreads = 0x0, f_spare = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, f_namemax = 0xff, f_owner = 0x0, f_fsid = { val = {0x60f8585f, 0x8858d83c}}, f_charspare = '\000' , f_fstypename = "ufs", '\000' , f_mntfromname = "/dev/md10", '\000' , f_mntonname = "/mnt", '\000' }, mnt_cred = 0xfffff806948ed300, mnt_data = 0xfffff80003bd0200, mnt_time = 0x0, mnt_iosize_max = 0x100000, mnt_export = 0x0, mnt_label = 0x0, mnt_hashseed = 0x4ba9eaef, mnt_lockref = 0x0, mnt_secondary_writes = 0x0, mnt_secondary_accwrites = 0x1e4b30, mnt_susp_owner = 0xfffffe00e4ce5900, mnt_gjprovider = 0x0, mnt_listmtx = { lock_object = {lo_name = 0xffffffff8125bc69 "struct mount vlist mtx", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xfffff8087fd83a80}, mtx_lock = 0x0}, mnt_lazyvnodelist = {tqh_first = 0xfffff805b7b93be0, tqh_last = 0xfffff804df96dd48}, mnt_lazyvnodelistsize = 0x33cf, mnt_upper_pending = 0x0, mnt_explock = {lock_object = {lo_name = 0xffffffff81294e11 "explock", lo_flags = 0x6730000, lo_data = 0x0, lo_witness = 0xfffff8087fd81500}, lk_lock = 0x1, lk_exslpfail = 0x0, lk_pri = 0x60, lk_timo = 0x0, lk_stack = {depth = 0x0, pcs = {0x0 }}}, mnt_uppers = {tqh_first = 0x0, tqh_last = 0xfffffe0143fe56a0}, mnt_notify = {tqh_first = 0x0, tqh_last = 0xfffffe0143fe56b0}, mnt_taskqueue_link = {stqe_next = 0x0}, mnt_taskqueue_flags = 0x300000080000} (kgdb) f 7 #7 0xffffffff80f4c048 in ffs_unmount (mp=0xfffffe0143fe4b80, mntflags=0x80000) at ../../../ufs/ffs/ffs_vfsops.c:1488 1488 error = vfs_write_suspend_umnt(mp); (kgdb) l 1483 ufs_extattr_uepm_destroy(&ump->um_extattr); 1484 e_restart = 1; 1485 } 1486 #endif 1487 if (susp) { 1488 error = vfs_write_suspend_umnt(mp); 1489 if (error != 0) 1490 goto fail1; 1491 } 1492 if (MOUNTEDSOFTDEP(mp)) (kgdb) info loc flags = ump = 0xfffff80003bd0200 td = 0xfffffe00e4ce5900 fs = 0xfffffe0143fe0000 susp = error = 0x2d e_restart = (kgdb) p *ump $2 = {um_mountp = 0xfffffe0143fe4b80, um_dev = 0xfffff801d2b8cc00, um_cp = 0xfffff80731e9b680, um_bo = 0xfffff8022e60bd58, um_odevvp = 0xfffff8043ec318e8, um_devvp = 0xfffff8022e60bbe0, um_fstype = 0x2, um_fs = 0xfffffe0143fe0000, um_extattr = {uepm_lock = {lock_object = { lo_name = 0xffffffff811cf540 "ufs_extattr_sx", lo_flags = 0x2330000, lo_data = 0x0, lo_witness = 0xfffff8087fd83d80}, sx_lock = 0x1}, uepm_list = {lh_first = 0x0}, uepm_ucred = 0x0, uepm_flags = 0x1}, um_nindir = 0x1000, um_bptrtodb = 0x3, um_seqinc = 0x8, um_maxsymlinklen = 0x78, um_lock = {lock_object = {lo_name = 0xffffffff811e72c2 "FFS", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xfffff8087fd83d00}, mtx_lock = 0x0}, um_fsckpid = 0x0, um_softdep = 0xfffff807312c3400, um_quotas = {0x0, 0x0}, um_cred = {0x0, 0x0}, um_btime = {0x0, 0x0}, um_itime = {0x0, 0x0}, um_qflags = "\000", um_savedmaxfilesize = 0x0, um_flags = 0x8, um_last_fullmsg = {tv_sec = 0x0, tv_usec = 0x0}, um_secs_fullmsg = 0x0, um_last_integritymsg = {tv_sec = 0x0, tv_usec = 0x0}, um_secs_integritymsg = 0x0, um_trim_inflight = 0x0, um_trim_inflight_blks = 0x0, um_trim_total = 0x0, um_trim_total_blks = 0x0, um_trim_tq = 0x0, um_trimhash = 0x0, um_trimlisthashsize = 0x0, um_balloc = 0xffffffff80f1f460 , um_blkatoff = 0xffffffff80f4caf0 , um_truncate = 0xffffffff80f212d0 , um_update = 0xffffffff80f20f20 , um_valloc = 0xffffffff80f19c80 , um_vfree = 0xffffffff80f1ab00 , um_ifree = 0xffffffff80f4cbb0 , um_rdonly = 0xffffffff80f22fc0 , um_snapgone = 0xffffffff80f26bf0 , um_check_blkno = 0x0} (kgdb) p *ump->um_softdep $3 = {sd_fslock = {lock_object = {lo_name = 0xffffffff811e711c "SUrw", lo_flags = 0x5230000, lo_data = 0x0, lo_witness = 0xfffff8087fd84080}, rw_lock = 0x1}, sd_workitem_pending = {lh_first = 0x0}, sd_worklist_tail = 0xfffff807312c3420, sd_journal_pending = {lh_first = 0x0}, sd_journal_tail = 0xfffff807312c3430, sd_jblocks = 0xfffff804dfd52f00, sd_unlinked = {tqh_first = 0x0, tqh_last = 0xfffff807312c3448}, sd_dirtycg = {lh_first = 0x0}, sd_mkdirlisthd = {lh_first = 0x0}, sd_pdhash = 0xfffffe014e956000, sd_pdhashsize = 0xffff, sd_pdnextclean = 0x0, sd_idhash = 0xfffffe0144600000, sd_idhashsize = 0x7ffff, sd_idnextclean = 0x0, sd_newblkhash = 0xfffffe0144a00000, sd_newblkhashsize = 0xfffff, sd_bmhash = 0xfffffe014e2cb000, sd_bmhashsize = 0x3ff, sd_indirhash = 0xfffff80003c1be60, sd_indirhashsize = 0x0, sd_on_journal = 0x0, sd_on_worklist = 0x0, sd_deps = 0x1, sd_accdeps = 0x6608f3, sd_req = 0x0, sd_flags = 0x0, sd_cleanups = 0x0, sd_flushtd = 0xfffffe0143ffe000, sd_next = {tqe_next = 0x0, tqe_prev = 0xfffff80003be9cf0}, sd_ump = 0xfffff80003bd0200, sd_curdeps = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1, 0x0 }, sd_alldeps = {{lh_first = 0x0}, {lh_first = 0x0}, {lh_first = 0x0}, {lh_first = 0x0}, {lh_first = 0x0}, {lh_first = 0x0}, {lh_first = 0xfffff805651ef800}, {lh_first = 0x0} }} (kgdb) diff --git a/sys/fs/nullfs/null.h b/sys/fs/nullfs/null.h index 6fdac4b1006e..a41625536d65 100644 --- a/sys/fs/nullfs/null.h +++ b/sys/fs/nullfs/null.h @@ -45,6 +45,8 @@ struct null_mount { struct mount *nullm_vfs; struct vnode *nullm_lowerrootvp; /* Ref to lower root vnode */ uint64_t nullm_flags; + struct mount_upper_node upper_node; + struct mount_upper_node notify_node; }; #ifdef _KERNEL diff --git a/sys/fs/nullfs/null_subr.c b/sys/fs/nullfs/null_subr.c index fb6a13449df7..e4ce525ebe03 100644 --- a/sys/fs/nullfs/null_subr.c +++ b/sys/fs/nullfs/null_subr.c @@ -287,6 +287,7 @@ null_nodeget(mp, lowervp, vpp) null_destroy_proto(vp, xp); return (0); } +lowervp->v_upper_vnode = vp; *vpp = vp; return (0); diff --git a/sys/fs/nullfs/null_vfsops.c b/sys/fs/nullfs/null_vfsops.c index 4914e5fc2dbf..73301c9275d2 100644 --- a/sys/fs/nullfs/null_vfsops.c +++ b/sys/fs/nullfs/null_vfsops.c @@ -163,7 +163,8 @@ nullfs_mount(struct mount *mp) * Save pointer to underlying FS and the reference to the * lower root vnode. */ - xmp->nullm_vfs = vfs_pin_from_vp(lowerrootvp); + xmp->nullm_vfs = vfs_register_upper_from_vp(lowerrootvp, mp, + &xmp->upper_node); if (xmp->nullm_vfs == NULL) { vput(lowerrootvp); free(xmp, M_NULLFSMNT); @@ -178,7 +179,7 @@ nullfs_mount(struct mount *mp) */ error = null_nodeget(mp, lowerrootvp, &nullm_rootvp); if (error != 0) { - vfs_unpin(xmp->nullm_vfs); + vfs_unregister_upper(xmp->nullm_vfs, &xmp->upper_node); vrele(lowerrootvp); free(xmp, M_NULLFSMNT); return (error); @@ -195,6 +196,11 @@ nullfs_mount(struct mount *mp) (xmp->nullm_vfs->mnt_kern_flag & MNTK_NULL_NOCACHE) != 0) xmp->nullm_flags &= ~NULLM_CACHE; + if ((xmp->nullm_flags & NULLM_CACHE) != 0) { + vfs_register_for_notification(xmp->nullm_vfs, mp, + &xmp->notify_node); + } + MNT_ILOCK(mp); if ((xmp->nullm_flags & NULLM_CACHE) != 0) { mp->mnt_kern_flag |= lowerrootvp->v_mount->mnt_kern_flag & @@ -206,13 +212,6 @@ nullfs_mount(struct mount *mp) (MNTK_USES_BCACHE | MNTK_NO_IOPF | MNTK_UNMAPPED_BUFS); MNT_IUNLOCK(mp); vfs_getnewfsid(mp); - if ((xmp->nullm_flags & NULLM_CACHE) != 0) { - MNT_ILOCK(xmp->nullm_vfs); - TAILQ_INSERT_TAIL(&xmp->nullm_vfs->mnt_uppers, mp, - mnt_upper_link); - MNT_IUNLOCK(xmp->nullm_vfs); - } - vfs_mountedfrom(mp, target); vput(nullm_rootvp); @@ -230,7 +229,6 @@ nullfs_unmount(mp, mntflags) int mntflags; { struct null_mount *mntdata; - struct mount *ump; int error, flags; NULLFSDEBUG("nullfs_unmount: mp = %p\n", (void *)mp); @@ -259,17 +257,11 @@ nullfs_unmount(mp, mntflags) * Finally, throw away the null_mount structure */ mntdata = mp->mnt_data; - ump = mntdata->nullm_vfs; if ((mntdata->nullm_flags & NULLM_CACHE) != 0) { - MNT_ILOCK(ump); - while ((ump->mnt_kern_flag & MNTK_VGONE_UPPER) != 0) { - ump->mnt_kern_flag |= MNTK_VGONE_WAITER; - msleep(&ump->mnt_uppers, &ump->mnt_mtx, 0, "vgnupw", 0); - } - TAILQ_REMOVE(&ump->mnt_uppers, mp, mnt_upper_link); - MNT_IUNLOCK(ump); + vfs_unregister_for_notification(mntdata->nullm_vfs, + &mntdata->notify_node); } - vfs_unpin(ump); + vfs_unregister_upper(mntdata->nullm_vfs, &mntdata->upper_node); vrele(mntdata->nullm_lowerrootvp); mp->mnt_data = NULL; free(mntdata, M_NULLFSMNT); diff --git a/sys/fs/nullfs/null_vnops.c b/sys/fs/nullfs/null_vnops.c index aeebce4ed086..8d735623c3aa 100644 --- a/sys/fs/nullfs/null_vnops.c +++ b/sys/fs/nullfs/null_vnops.c @@ -223,13 +223,12 @@ int null_bypass(struct vop_generic_args *ap) { struct vnode **this_vp_p; - int error; struct vnode *old_vps[VDESC_MAX_VPS]; struct vnode **vps_p[VDESC_MAX_VPS]; struct vnode ***vppp; struct vnode *lvp; struct vnodeop_desc *descp = ap->a_desc; - int reles, i; + int error, i, reles; if (null_bug_bypass) printf ("null_bypass: %s\n", descp->vdesc_name); @@ -252,26 +251,39 @@ null_bypass(struct vop_generic_args *ap) for (i = 0; i < VDESC_MAX_VPS; reles >>= 1, i++) { if (descp->vdesc_vp_offsets[i] == VDESC_NO_OFFSET) break; /* bail out at end of list */ - vps_p[i] = this_vp_p = - VOPARG_OFFSETTO(struct vnode**,descp->vdesc_vp_offsets[i],ap); + vps_p[i] = this_vp_p = VOPARG_OFFSETTO(struct vnode **, + descp->vdesc_vp_offsets[i],ap); + /* * We're not guaranteed that any but the first vnode * are of our type. Check for and don't map any * that aren't. (We must always map first vp or vclean fails.) */ - if (i && (*this_vp_p == NULLVP || + if (i != 0 && (*this_vp_p == NULLVP || (*this_vp_p)->v_op != &null_vnodeops)) { old_vps[i] = NULLVP; } else { old_vps[i] = *this_vp_p; *(vps_p[i]) = NULLVPTOLOWERVP(*this_vp_p); + + /* + * The upper vnode reference to the lower + * vnode is the only reference that keeps our + * pointer to the lower vnode alive. If lower + * vnode is relocked during the VOP call, + * upper vnode might become unlocked and + * reclaimed, which invalidates our reference. + * Add a transient hold around VOP call. + */ + vhold(*this_vp_p); + /* * XXX - Several operations have the side effect * of vrele'ing their vp's. We must account for * that. (This should go away in the future.) */ if (reles & VDESC_VP0_WILLRELE) - VREF(*this_vp_p); + vref(*this_vp_p); } } @@ -279,9 +291,9 @@ null_bypass(struct vop_generic_args *ap) * Call the operation on the lower layer * with the modified argument structure. */ - if (vps_p[0] && *vps_p[0]) + if (vps_p[0] != NULL && *vps_p[0] != NULL) { error = VCALL(ap); - else { + } else { printf("null_bypass: no map for %s\n", descp->vdesc_name); error = EINVAL; } @@ -295,10 +307,11 @@ null_bypass(struct vop_generic_args *ap) for (i = 0; i < VDESC_MAX_VPS; reles >>= 1, i++) { if (descp->vdesc_vp_offsets[i] == VDESC_NO_OFFSET) break; /* bail out at end of list */ - if (old_vps[i]) { + if (old_vps[i] != NULL) { lvp = *(vps_p[i]); /* + * Get rid of the transient hold on lvp. * If lowervp was unlocked during VOP * operation, nullfs upper vnode could have * been reclaimed, which changes its v_vnlock @@ -306,11 +319,14 @@ null_bypass(struct vop_generic_args *ap) * must move lock ownership from lower to * upper (reclaimed) vnode. */ - if (lvp != NULLVP && - VOP_ISLOCKED(lvp) == LK_EXCLUSIVE && - old_vps[i]->v_vnlock != lvp->v_vnlock) { - VOP_UNLOCK(lvp); - VOP_LOCK(old_vps[i], LK_EXCLUSIVE | LK_RETRY); + if (lvp != NULLVP) { + if (VOP_ISLOCKED(lvp) == LK_EXCLUSIVE && + old_vps[i]->v_vnlock != lvp->v_vnlock) { + VOP_UNLOCK(lvp); + VOP_LOCK(old_vps[i], LK_EXCLUSIVE | + LK_RETRY); + } + vdrop(lvp); } *(vps_p[i]) = old_vps[i]; @@ -328,17 +344,18 @@ null_bypass(struct vop_generic_args *ap) * (Assumes that the lower layer always returns * a VREF'ed vpp unless it gets an error.) */ - if (descp->vdesc_vpp_offset != VDESC_NO_OFFSET && !error) { + if (descp->vdesc_vpp_offset != VDESC_NO_OFFSET && error == 0) { /* * XXX - even though some ops have vpp returned vp's, * several ops actually vrele this before returning. * We must avoid these ops. * (This should go away when these ops are regularized.) */ - vppp = VOPARG_OFFSETTO(struct vnode***, - descp->vdesc_vpp_offset,ap); - if (*vppp) - error = null_nodeget(old_vps[0]->v_mount, **vppp, *vppp); + vppp = VOPARG_OFFSETTO(struct vnode ***, + descp->vdesc_vpp_offset,ap); + if (*vppp != NULL) + error = null_nodeget(old_vps[0]->v_mount, **vppp, + *vppp); } return (error); @@ -419,11 +436,11 @@ null_lookup(struct vop_lookup_args *ap) * dvp to be reclaimed due to shared v_vnlock. Check for the * doomed state and return error. */ - if ((error == 0 || error == EJUSTRETURN) && - VN_IS_DOOMED(dvp)) { - error = ENOENT; - if (lvp != NULL) + if (VN_IS_DOOMED(dvp)) { + if ((error == 0 || error == EJUSTRETURN) && lvp != NULL) vput(lvp); + if (error == 0) + error = ENOENT; /* * If vgone() did reclaimed dvp before curthread @@ -849,6 +866,7 @@ null_reclaim(struct vop_reclaim_args *ap) vp->v_data = NULL; vp->v_object = NULL; vp->v_vnlock = &vp->v_lock; +lowervp->v_upper_vnode = NULL; /* * If we were opened for write, we leased the write reference @@ -1050,6 +1068,7 @@ struct vop_vector null_vnodeops = { .vop_bypass = null_bypass, .vop_access = null_access, .vop_accessx = null_accessx, + .vop_advlock = vop_stdadvlock, .vop_advlockpurge = vop_stdadvlockpurge, .vop_bmap = VOP_EOPNOTSUPP, .vop_stat = null_stat, diff --git a/sys/fs/unionfs/union.h b/sys/fs/unionfs/union.h index 64706b2b21a2..96180480dbec 100644 --- a/sys/fs/unionfs/union.h +++ b/sys/fs/unionfs/union.h @@ -57,6 +57,8 @@ struct unionfs_mount { struct vnode *um_lowervp; /* VREFed once */ struct vnode *um_uppervp; /* VREFed once */ struct vnode *um_rootvp; /* ROOT vnode */ + struct mount_upper_node um_lower_link; /* node in lower FS list of uppers */ + struct mount_upper_node um_upper_link; /* node in upper FS list of uppers */ unionfs_copymode um_copymode; unionfs_whitemode um_whitemode; uid_t um_uid; diff --git a/sys/fs/unionfs/union_vfsops.c b/sys/fs/unionfs/union_vfsops.c index 96a30f0ae8b5..c17650dedc63 100644 --- a/sys/fs/unionfs/union_vfsops.c +++ b/sys/fs/unionfs/union_vfsops.c @@ -292,14 +292,16 @@ unionfs_domount(struct mount *mp) return (error); } - lowermp = vfs_pin_from_vp(ump->um_lowervp); - uppermp = vfs_pin_from_vp(ump->um_uppervp); + lowermp = vfs_register_upper_from_vp(ump->um_lowervp, mp, + &ump->um_lower_link); + uppermp = vfs_register_upper_from_vp(ump->um_uppervp, mp, + &ump->um_upper_link); if (lowermp == NULL || uppermp == NULL) { if (lowermp != NULL) - vfs_unpin(lowermp); + vfs_unregister_upper(lowermp, &ump->um_lower_link); if (uppermp != NULL) - vfs_unpin(uppermp); + vfs_unregister_upper(uppermp, &ump->um_upper_link); free(ump, M_UNIONFSMNT); mp->mnt_data = NULL; return (ENOENT); @@ -357,8 +359,8 @@ unionfs_unmount(struct mount *mp, int mntflags) if (error) return (error); - vfs_unpin(ump->um_lowervp->v_mount); - vfs_unpin(ump->um_uppervp->v_mount); + vfs_unregister_upper(ump->um_lowervp->v_mount, &ump->um_lower_link); + vfs_unregister_upper(ump->um_uppervp->v_mount, &ump->um_upper_link); free(ump, M_UNIONFSMNT); mp->mnt_data = NULL; diff --git a/sys/kern/vfs_default.c b/sys/kern/vfs_default.c index 63bca7810847..bd6a0f769c8d 100644 --- a/sys/kern/vfs_default.c +++ b/sys/kern/vfs_default.c @@ -556,7 +556,7 @@ vop_stdunlock(ap) } */ *ap; { struct vnode *vp = ap->a_vp; - +if (vp->v_upper_vnode != NULL) stack_save(&vp->v_upper_vnode->v_unlock_stack); return (lockmgr_unlock(vp->v_vnlock)); } diff --git a/sys/kern/vfs_mount.c b/sys/kern/vfs_mount.c index 8b4426209818..6d7bc181d4b0 100644 --- a/sys/kern/vfs_mount.c +++ b/sys/kern/vfs_mount.c @@ -65,6 +65,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include @@ -89,6 +90,11 @@ static bool default_autoro = false; SYSCTL_BOOL(_vfs, OID_AUTO, default_autoro, CTLFLAG_RW, &default_autoro, 0, "Retry failed r/w mount as r/o if no explicit ro/rw option is specified"); +static bool recursive_forced_unmount = false; +SYSCTL_BOOL(_vfs, OID_AUTO, recursive_forced_unmount, CTLFLAG_RW, + &recursive_forced_unmount, 0, "Recursively unmount stacked upper mounts" + " when a file system is forcibly unmounted"); + MALLOC_DEFINE(M_MOUNT, "mount", "vfs mount structure"); MALLOC_DEFINE(M_STATFS, "statfs", "statfs structure"); static uma_zone_t mount_zone; @@ -103,6 +109,16 @@ MTX_SYSINIT(mountlist, &mountlist_mtx, "mountlist", MTX_DEF); EVENTHANDLER_LIST_DEFINE(vfs_mounted); EVENTHANDLER_LIST_DEFINE(vfs_unmounted); +static void vfs_deferred_unmount(void *arg, int pending); +static struct task deferred_unmount_task = + TASK_INITIALIZER(0, vfs_deferred_unmount, NULL);; +static struct mtx deferred_unmount_lock; +MTX_SYSINIT(deferred_unmount, &deferred_unmount_lock, "deferred_unmount", + MTX_DEF); +static STAILQ_HEAD(, mount) deferred_unmount_list = + STAILQ_HEAD_INITIALIZER(deferred_unmount_list); +TASKQUEUE_DEFINE_THREAD(deferred_unmount); + static void mount_devctl_event(const char *type, struct mount *mp, bool donew); /* @@ -505,8 +521,21 @@ vfs_ref(struct mount *mp) MNT_IUNLOCK(mp); } +/* + * Register ump as an upper mount of the mount associated with + * vnode vp. This registration will be tracked through + * mount_upper_node upper, which should be allocated by the + * caller and stored in per-mount data associated with mp. + * + * If successful, this function will return the mount associated + * with vp, and will ensure that it cannot be unmounted until + * ump has been unregistered as one of its upper mounts. + * + * Upon failure this function will return NULL. + */ struct mount * -vfs_pin_from_vp(struct vnode *vp) +vfs_register_upper_from_vp(struct vnode *vp, struct mount *ump, + struct mount_upper_node *upper) { struct mount *mp; @@ -514,26 +543,81 @@ vfs_pin_from_vp(struct vnode *vp) if (mp == NULL) return (NULL); MNT_ILOCK(mp); - if (mp != vp->v_mount || (mp->mnt_kern_flag & MNTK_UNMOUNT) != 0) { + if (mp != vp->v_mount || + ((mp->mnt_kern_flag & (MNTK_UNMOUNT | MNTK_RECURSE)) != 0)) { MNT_IUNLOCK(mp); return (NULL); } + KASSERT(ump != mp, ("upper and lower mounts are identical")); + upper->mp = ump; MNT_REF(mp); - KASSERT(mp->mnt_pinned_count < INT_MAX, - ("mount pinned count overflow")); - ++mp->mnt_pinned_count; + TAILQ_INSERT_TAIL(&mp->mnt_uppers, upper, mnt_upper_link); MNT_IUNLOCK(mp); return (mp); } +/* + * Register upper mount ump to receive vnode unlink/reclaim + * notifications from lower mount mp. This registration will + * be tracked through mount_upper_node upper, which should be + * allocated by the caller and stored in per-mount data + * associated with mp. + * + * ump must already be registered as an upper mount of mp + * through a call to vfs_register_upper_from_vp(). + */ void -vfs_unpin(struct mount *mp) +vfs_register_for_notification(struct mount *mp, struct mount *ump, + struct mount_upper_node *upper) +{ + upper->mp = ump; + MNT_ILOCK(mp); + TAILQ_INSERT_TAIL(&mp->mnt_notify, upper, mnt_upper_link); + MNT_IUNLOCK(mp); +} + +static void +vfs_drain_upper_locked(struct mount *mp) +{ + mtx_assert(MNT_MTX(mp), MA_OWNED); + while (mp->mnt_upper_pending != 0) { + mp->mnt_kern_flag |= MNTK_UPPER_WAITER; + msleep(&mp->mnt_uppers, MNT_MTX(mp), 0, "mntupw", 0); + } +} + +/* + * Undo a previous call to vfs_register_for_notification(). + * The mount represented by upper must be currently registered + * as an upper mount for mp. + */ +void +vfs_unregister_for_notification(struct mount *mp, + struct mount_upper_node *upper) +{ + MNT_ILOCK(mp); + vfs_drain_upper_locked(mp); + TAILQ_REMOVE(&mp->mnt_notify, upper, mnt_upper_link); + MNT_IUNLOCK(mp); +} + +/* + * Undo a previous call to vfs_register_upper_from_vp(). + * This must be done before mp can be unmounted. + */ +void +vfs_unregister_upper(struct mount *mp, struct mount_upper_node *upper) { MNT_ILOCK(mp); - KASSERT(mp->mnt_pinned_count > 0, ("mount pinned count underflow")); KASSERT((mp->mnt_kern_flag & MNTK_UNMOUNT) == 0, - ("mount pinned with pending unmount")); - --mp->mnt_pinned_count; + ("registered upper with pending unmount")); + vfs_drain_upper_locked(mp); + TAILQ_REMOVE(&mp->mnt_uppers, upper, mnt_upper_link); + if ((mp->mnt_kern_flag & MNTK_TASKQUEUE_WAITER) != 0 && + TAILQ_EMPTY(&mp->mnt_uppers)) { + mp->mnt_kern_flag &= ~MNTK_TASKQUEUE_WAITER; + wakeup(&mp->mnt_taskqueue_link); + } MNT_REL(mp); MNT_IUNLOCK(mp); } @@ -600,8 +684,10 @@ vfs_mount_alloc(struct vnode *vp, struct vfsconf *vfsp, const char *fspath, mac_mount_create(cred, mp); #endif arc4rand(&mp->mnt_hashseed, sizeof mp->mnt_hashseed, 0); + mp->mnt_upper_pending = 0; TAILQ_INIT(&mp->mnt_uppers); - mp->mnt_pinned_count = 0; + TAILQ_INIT(&mp->mnt_notify); + mp->mnt_taskqueue_flags = 0; return (mp); } @@ -640,9 +726,9 @@ vfs_mount_destroy(struct mount *mp) vn_printf(vp, "dangling vnode "); panic("unmount: dangling vnode"); } - KASSERT(mp->mnt_pinned_count == 0, - ("mnt_pinned_count = %d", mp->mnt_pinned_count)); + KASSERT(mp->mnt_upper_pending == 0, ("mnt_upper_pending")); KASSERT(TAILQ_EMPTY(&mp->mnt_uppers), ("mnt_uppers")); + KASSERT(TAILQ_EMPTY(&mp->mnt_notify), ("mnt_notify")); if (mp->mnt_nvnodelistsize != 0) panic("vfs_mount_destroy: nonzero nvnodelistsize"); if (mp->mnt_lazyvnodelistsize != 0) @@ -1799,17 +1885,166 @@ vfs_mount_fetch_counter(struct mount *mp, enum mount_counter which) return (sum); } +static bool +deferred_unmount_enqueue(struct mount *mp, uint64_t flags, bool requeue) +{ + bool enqueued; + + enqueued = false; + mtx_lock(&deferred_unmount_lock); + if ((mp->mnt_taskqueue_flags & MNT_DEFERRED) == 0 || requeue) { + mp->mnt_taskqueue_flags = flags | MNT_DEFERRED; + STAILQ_INSERT_TAIL(&deferred_unmount_list, mp, + mnt_taskqueue_link); + enqueued = true; + } + mtx_unlock(&deferred_unmount_lock); + + if (enqueued) { + taskqueue_enqueue(taskqueue_deferred_unmount, + &deferred_unmount_task); + } + + return (enqueued); +} + +/* + * Taskqueue handler for processing async/recursive unmounts + */ +static void +vfs_deferred_unmount(void *argi __unused, int pending __unused) +{ + STAILQ_HEAD(, mount) local_unmounts; + uint64_t flags; + struct mount *mp, *tmp; + bool unmounted; + + STAILQ_INIT(&local_unmounts); + mtx_lock(&deferred_unmount_lock); + STAILQ_CONCAT(&local_unmounts, &deferred_unmount_list); + mtx_unlock(&deferred_unmount_lock); + + STAILQ_FOREACH_SAFE(mp, &local_unmounts, mnt_taskqueue_link, tmp) { + flags = mp->mnt_taskqueue_flags; + KASSERT((flags & MNT_DEFERRED) != 0, + ("taskqueue unmount without MNT_DEFERRED")); + if (dounmount(mp, flags, curthread) != 0) { + MNT_ILOCK(mp); + unmounted = ((mp->mnt_kern_flag & MNTK_REFEXPIRE) != 0); + MNT_IUNLOCK(mp); + if (!unmounted) + deferred_unmount_enqueue(mp, flags, true); + else + vfs_rel(mp); + } + } +} + /* * Do the actual filesystem unmount. */ int -dounmount(struct mount *mp, int flags, struct thread *td) +dounmount(struct mount *mp, uint64_t flags, struct thread *td) { + struct mount_upper_node *upper; struct vnode *coveredvp, *rootvp; int error; uint64_t async_flag; int mnt_gen_r; + KASSERT((flags & MNT_DEFERRED) == 0 || + (flags & (MNT_RECURSE | MNT_FORCE)) == (MNT_RECURSE | MNT_FORCE), + ("MNT_DEFERRED requires MNT_RECURSE | MNT_FORCE")); + + /* + * If the caller has explicitly requested the unmount to be handled by + * the taskqueue and we're not already in taskqueue context, queue + * up the unmount request and exit. This is done prior to any + * credential checks; MNT_DEFERRED should be used only for kernel- + * initiated unmounts and will therefore be processed with the + * (kernel) credentials of the taskqueue thread. Still, callers + * should be sure this is the behavior they want. + */ + if ((flags & MNT_DEFERRED) != 0 && + taskqueue_member(taskqueue_deferred_unmount, curthread) == 0) { + if (!deferred_unmount_enqueue(mp, flags, false)) + vfs_rel(mp); + return (EINPROGRESS); + } + + /* + * Only privileged root, or (if MNT_USER is set) the user that did the + * original mount is permitted to unmount this filesystem. + * This check should be made prior to queueing up any recursive + * unmounts of upper filesystems. Those unmounts will be executed + * with kernel thread credentials and are expected to succeed, so + * we must at least ensure the originating context has sufficient + * privilege to unmount the base filesystem before proceeding with + * the uppers. + */ + error = vfs_suser(mp, td); + if (error != 0) { + KASSERT((flags & MNT_DEFERRED) == 0, + ("taskqueue unmount with insufficient privilege")); + vfs_rel(mp); + return (error); + } + + if (recursive_forced_unmount && ((flags & MNT_FORCE) != 0)) + flags |= MNT_RECURSE; + + if ((flags & MNT_RECURSE) != 0) { + KASSERT((flags & MNT_FORCE) != 0, + ("MNT_RECURSE requires MNT_FORCE")); + + MNT_ILOCK(mp); + /* + * Set MNTK_RECURSE to prevent new upper mounts from being + * added, and note that an operation on the uppers list is in + * progress. This will ensure that unregistration from the + * uppers list, and therefore any pending unmount of the upper + * FS, can't complete until after we finish walking the list. + */ + mp->mnt_kern_flag |= MNTK_RECURSE; + mp->mnt_upper_pending++; + TAILQ_FOREACH(upper, &mp->mnt_uppers, mnt_upper_link) { + MNT_IUNLOCK(mp); + vfs_ref(upper->mp); + if (!deferred_unmount_enqueue(upper->mp, flags, false)) + vfs_rel(upper->mp); + MNT_ILOCK(mp); + } + mp->mnt_upper_pending--; + if ((mp->mnt_kern_flag & MNTK_UPPER_WAITER) != 0 && + mp->mnt_upper_pending == 0) { + mp->mnt_kern_flag &= ~MNTK_UPPER_WAITER; + wakeup(&mp->mnt_uppers); + } + /* + * If we're not on the taskqueue, wait until the uppers list + * is drained before proceeding with unmount. Otherwise, if + * we are on the taskqueue and there are still pending uppers, + * just re-enqueue on the end of the taskqueue. + */ + if ((flags & MNT_DEFERRED) == 0) { + while (!TAILQ_EMPTY(&mp->mnt_uppers)) { + mp->mnt_kern_flag |= MNTK_TASKQUEUE_WAITER; + msleep(&mp->mnt_taskqueue_link, MNT_MTX(mp), 0, + "umntqw", 0); + } + } else if (!TAILQ_EMPTY(&mp->mnt_uppers)) { + MNT_IUNLOCK(mp); + deferred_unmount_enqueue(mp, flags, true); + return (0); + } + MNT_IUNLOCK(mp); + KASSERT(TAILQ_EMPTY(&mp->mnt_uppers), ("mnt_uppers not empty")); + } + + /* Allow the taskqueue to safely re-enqueue on failure */ + if ((flags & MNT_DEFERRED) != 0) + vfs_ref(mp); + if ((coveredvp = mp->mnt_vnodecovered) != NULL) { mnt_gen_r = mp->mnt_gen; VI_LOCK(coveredvp); @@ -1828,27 +2063,13 @@ dounmount(struct mount *mp, int flags, struct thread *td) } } - /* - * Only privileged root, or (if MNT_USER is set) the user that did the - * original mount is permitted to unmount this filesystem. - */ - error = vfs_suser(mp, td); - if (error != 0) { - if (coveredvp != NULL) { - VOP_UNLOCK(coveredvp); - vdrop(coveredvp); - } - vfs_rel(mp); - return (error); - } - vfs_op_enter(mp); vn_start_write(NULL, &mp, V_WAIT | V_MNTREF); MNT_ILOCK(mp); if ((mp->mnt_kern_flag & MNTK_UNMOUNT) != 0 || (mp->mnt_flag & MNT_UPDATE) != 0 || - mp->mnt_pinned_count != 0) { + !TAILQ_EMPTY(&mp->mnt_uppers)) { dounmount_cleanup(mp, coveredvp, 0); return (EBUSY); } @@ -1952,6 +2173,7 @@ dounmount(struct mount *mp, int flags, struct thread *td) } return (error); } + mtx_lock(&mountlist_mtx); TAILQ_REMOVE(&mountlist, mp, mnt_list); mtx_unlock(&mountlist_mtx); @@ -1977,6 +2199,8 @@ dounmount(struct mount *mp, int flags, struct thread *td) } if (mp == rootdevmp) rootdevmp = NULL; + if ((flags & MNT_DEFERRED) != 0) + vfs_rel(mp); vfs_mount_destroy(mp); return (0); } diff --git a/sys/kern/vfs_subr.c b/sys/kern/vfs_subr.c index a2f25bf78495..dfef7257deb9 100644 --- a/sys/kern/vfs_subr.c +++ b/sys/kern/vfs_subr.c @@ -831,9 +831,9 @@ vfs_busy(struct mount *mp, int flags) * valid. */ while (mp->mnt_kern_flag & MNTK_UNMOUNT) { - KASSERT(mp->mnt_pinned_count == 0, - ("%s: non-zero pinned count %d with pending unmount", - __func__, mp->mnt_pinned_count)); + KASSERT(TAILQ_EMPTY(&mp->mnt_uppers), + ("%s: non-empty upper mount list with pending unmount", + __func__)); if (flags & MBF_NOWAIT || mp->mnt_kern_flag & MNTK_REFEXPIRE) { MNT_REL(mp); MNT_IUNLOCK(mp); @@ -1842,6 +1842,9 @@ getnewvnode(const char *tag, struct mount *mp, struct vop_vector *vops, */ vp->v_hash = (uintptr_t)vp >> vnsz2log; +vp->v_upper_vnode = NULL; +stack_zero(&vp->v_unlock_stack); + *vpp = vp; return (0); } @@ -3891,61 +3894,44 @@ vgone(struct vnode *vp) VI_UNLOCK(vp); } -static void -notify_lowervp_vfs_dummy(struct mount *mp __unused, - struct vnode *lowervp __unused) -{ -} - /* * Notify upper mounts about reclaimed or unlinked vnode. */ void vfs_notify_upper(struct vnode *vp, int event) { - static struct vfsops vgonel_vfsops = { - .vfs_reclaim_lowervp = notify_lowervp_vfs_dummy, - .vfs_unlink_lowervp = notify_lowervp_vfs_dummy, - }; - struct mount *mp, *ump, *mmp; + struct mount *mp; + struct mount_upper_node *ump; - mp = vp->v_mount; + mp = atomic_load_ptr(&vp->v_mount); if (mp == NULL) return; - if (TAILQ_EMPTY(&mp->mnt_uppers)) + if (TAILQ_EMPTY(&mp->mnt_notify)) return; - mmp = malloc(sizeof(struct mount), M_TEMP, M_WAITOK | M_ZERO); - mmp->mnt_op = &vgonel_vfsops; - mmp->mnt_kern_flag |= MNTK_MARKER; MNT_ILOCK(mp); - mp->mnt_kern_flag |= MNTK_VGONE_UPPER; - for (ump = TAILQ_FIRST(&mp->mnt_uppers); ump != NULL;) { - if ((ump->mnt_kern_flag & MNTK_MARKER) != 0) { - ump = TAILQ_NEXT(ump, mnt_upper_link); - continue; - } - TAILQ_INSERT_AFTER(&mp->mnt_uppers, ump, mmp, mnt_upper_link); + mp->mnt_upper_pending++; + KASSERT(mp->mnt_upper_pending > 0, + ("%s: mnt_upper_pending %d", __func__, mp->mnt_upper_pending)); + TAILQ_FOREACH(ump, &mp->mnt_notify, mnt_upper_link) { MNT_IUNLOCK(mp); switch (event) { case VFS_NOTIFY_UPPER_RECLAIM: - VFS_RECLAIM_LOWERVP(ump, vp); + VFS_RECLAIM_LOWERVP(ump->mp, vp); break; case VFS_NOTIFY_UPPER_UNLINK: - VFS_UNLINK_LOWERVP(ump, vp); + VFS_UNLINK_LOWERVP(ump->mp, vp); break; default: KASSERT(0, ("invalid event %d", event)); break; } MNT_ILOCK(mp); - ump = TAILQ_NEXT(mmp, mnt_upper_link); - TAILQ_REMOVE(&mp->mnt_uppers, mmp, mnt_upper_link); } - free(mmp, M_TEMP); - mp->mnt_kern_flag &= ~MNTK_VGONE_UPPER; - if ((mp->mnt_kern_flag & MNTK_VGONE_WAITER) != 0) { - mp->mnt_kern_flag &= ~MNTK_VGONE_WAITER; + mp->mnt_upper_pending--; + if ((mp->mnt_kern_flag & MNTK_UPPER_WAITER) != 0 && + mp->mnt_upper_pending == 0) { + mp->mnt_kern_flag &= ~MNTK_UPPER_WAITER; wakeup(&mp->mnt_uppers); } MNT_IUNLOCK(mp); @@ -4376,12 +4362,12 @@ DB_SHOW_COMMAND(mount, db_show_mount) MNT_KERN_FLAG(MNTK_EXTENDED_SHARED); MNT_KERN_FLAG(MNTK_SHARED_WRITES); MNT_KERN_FLAG(MNTK_NO_IOPF); - MNT_KERN_FLAG(MNTK_VGONE_UPPER); - MNT_KERN_FLAG(MNTK_VGONE_WAITER); + MNT_KERN_FLAG(MNTK_RECURSE); + MNT_KERN_FLAG(MNTK_UPPER_WAITER); MNT_KERN_FLAG(MNTK_LOOKUP_EXCL_DOTDOT); - MNT_KERN_FLAG(MNTK_MARKER); MNT_KERN_FLAG(MNTK_USES_BCACHE); MNT_KERN_FLAG(MNTK_FPLOOKUP); + MNT_KERN_FLAG(MNTK_TASKQUEUE_WAITER); MNT_KERN_FLAG(MNTK_NOASYNC); MNT_KERN_FLAG(MNTK_UNMOUNT); MNT_KERN_FLAG(MNTK_MWAIT); diff --git a/sys/sys/mount.h b/sys/sys/mount.h index 693293b12370..016c7e9c188d 100644 --- a/sys/sys/mount.h +++ b/sys/sys/mount.h @@ -190,6 +190,19 @@ struct mount_pcpu { _Static_assert(sizeof(struct mount_pcpu) == 16, "the struct is allocated from pcpu 16 zone"); +/* + * Structure for tracking a stacked filesystem mounted above another + * filesystem. This is expected to be stored in the upper FS' per-mount data. + * + * Lock reference: + * i - lower mount interlock + * c - constant from node initialization + */ +struct mount_upper_node { + struct mount *mp; /* (c) mount object for upper FS */ + TAILQ_ENTRY(mount_upper_node) mnt_upper_link; /* (i) position in uppers list */ +}; + /* * Structure per mounted filesystem. Each mounted filesystem has an * array of operations and an instance record. The filesystems are @@ -199,8 +212,8 @@ _Static_assert(sizeof(struct mount_pcpu) == 16, * l - mnt_listmtx * m - mountlist_mtx * i - interlock - * i* - interlock of uppers' list head * v - vnode freelist mutex + * d - deferred unmount list mutex * * Unmarked fields are considered stable as long as a ref is held. * @@ -242,10 +255,12 @@ struct mount { struct mtx mnt_listmtx; struct vnodelst mnt_lazyvnodelist; /* (l) list of lazy vnodes */ int mnt_lazyvnodelistsize; /* (l) # of lazy vnodes */ - int mnt_pinned_count; /* (i) unmount prevented */ + int mnt_upper_pending; /* (i) # of pending ops on mnt_uppers */ struct lock mnt_explock; /* vfs_export walkers lock */ - TAILQ_ENTRY(mount) mnt_upper_link; /* (i*) we in the all uppers */ - TAILQ_HEAD(, mount) mnt_uppers; /* (i) upper mounts over us */ + TAILQ_HEAD(, mount_upper_node) mnt_uppers; /* (i) upper mounts over us */ + TAILQ_HEAD(, mount_upper_node) mnt_notify; /* (i) upper mounts for notification */ + STAILQ_ENTRY(mount) mnt_taskqueue_link; /* (d) our place in deferred unmount list */ + uint64_t mnt_taskqueue_flags; /* (d) unmount flags passed from taskqueue */ }; #endif /* _WANT_MOUNT || _KERNEL */ @@ -438,9 +453,13 @@ struct mntoptnames { #define MNT_BYFSID 0x0000000008000000ULL /* specify filesystem by ID. */ #define MNT_NOCOVER 0x0000001000000000ULL /* Do not cover a mount point */ #define MNT_EMPTYDIR 0x0000002000000000ULL /* Only mount on empty dir */ -#define MNT_CMDFLAGS (MNT_UPDATE | MNT_DELEXPORT | MNT_RELOAD | \ +#define MNT_RECURSE 0x0000100000000000ULL /* recursively unmount uppers */ +#define MNT_DEFERRED 0x0000200000000000ULL /* unmount in async context */ +#define MNT_CMDFLAGS (MNT_UPDATE | MNT_DELEXPORT | MNT_RELOAD | \ MNT_FORCE | MNT_SNAPSHOT | MNT_NONBUSY | \ - MNT_BYFSID | MNT_NOCOVER | MNT_EMPTYDIR) + MNT_BYFSID | MNT_NOCOVER | MNT_EMPTYDIR | \ + MNT_RECURSE | MNT_DEFERRED) + /* * Internal filesystem control flags stored in mnt_kern_flag. * @@ -466,10 +485,9 @@ struct mntoptnames { #define MNTK_NO_IOPF 0x00000100 /* Disallow page faults during reads and writes. Filesystem shall properly handle i/o state on EFAULT. */ -#define MNTK_VGONE_UPPER 0x00000200 -#define MNTK_VGONE_WAITER 0x00000400 +#define MNTK_RECURSE 0x00000200 /* pending recursive unmount */ +#define MNTK_UPPER_WAITER 0x00000400 /* waiting to drain MNTK_UPPER_PENDING */ #define MNTK_LOOKUP_EXCL_DOTDOT 0x00000800 -#define MNTK_MARKER 0x00001000 #define MNTK_UNMAPPED_BUFS 0x00002000 #define MNTK_USES_BCACHE 0x00004000 /* FS uses the buffer cache. */ #define MNTK_TEXT_REFS 0x00008000 /* Keep use ref for text */ @@ -477,8 +495,9 @@ struct mntoptnames { #define MNTK_UNIONFS 0x00020000 /* A hack for F_ISUNIONSTACK */ #define MNTK_FPLOOKUP 0x00040000 /* fast path lookup is supported */ #define MNTK_SUSPEND_ALL 0x00080000 /* Suspended by all-fs suspension */ -#define MNTK_NOASYNC 0x00800000 /* disable async */ -#define MNTK_UNMOUNT 0x01000000 /* unmount in progress */ +#define MNTK_TASKQUEUE_WAITER 0x00100000 /* Waiting on unmount taskqueue */ +#define MNTK_NOASYNC 0x00800000 /* disable async */ +#define MNTK_UNMOUNT 0x01000000 /* unmount in progress */ #define MNTK_MWAIT 0x02000000 /* waiting for unmount to finish */ #define MNTK_SUSPEND 0x08000000 /* request write suspension */ #define MNTK_SUSPEND2 0x04000000 /* block secondary writes */ @@ -952,7 +971,7 @@ vfs_statfs_t __vfs_statfs; * exported vnode operations */ -int dounmount(struct mount *, int, struct thread *); +int dounmount(struct mount *, uint64_t, struct thread *); int kernel_mount(struct mntarg *ma, uint64_t flags); int kernel_vmount(int flags, ...); @@ -1012,8 +1031,13 @@ struct mount *vfs_mount_alloc(struct vnode *, struct vfsconf *, const char *, int vfs_suser(struct mount *, struct thread *); void vfs_unbusy(struct mount *); void vfs_unmountall(void); -struct mount *vfs_pin_from_vp(struct vnode *); -void vfs_unpin(struct mount *); +struct mount *vfs_register_upper_from_vp(struct vnode *, + struct mount *ump, struct mount_upper_node *); +void vfs_register_for_notification(struct mount *, struct mount *, + struct mount_upper_node *); +void vfs_unregister_for_notification(struct mount *, + struct mount_upper_node *); +void vfs_unregister_upper(struct mount *, struct mount_upper_node *); extern TAILQ_HEAD(mntlist, mount) mountlist; /* mounted filesystem list */ extern struct mtx_padalign mountlist_mtx; extern struct nfs_public nfs_pub; diff --git a/sys/sys/vnode.h b/sys/sys/vnode.h index 64fb00c5845e..9b79f855927b 100644 --- a/sys/sys/vnode.h +++ b/sys/sys/vnode.h @@ -100,6 +100,7 @@ struct vpollinfo { #if defined(_KERNEL) || defined(_KVM_VNODE) +#include struct vnode { /* * Fields which define the identity of the vnode. These fields are @@ -173,6 +174,9 @@ struct vnode { int v_writecount; /* I ref count of writers or (negative) text users */ int v_seqc_users; /* i modifications pending */ + +struct stack v_unlock_stack; +struct vnode *v_upper_vnode; }; #ifndef DEBUG_LOCKS @@ -182,7 +186,7 @@ struct vnode { * to the structure and there is nothing which can be done to prevent growth * then so be it. But don't grow it without a good reason. */ -_Static_assert(sizeof(struct vnode) <= 448, "vnode size crosses 448 bytes"); +//_Static_assert(sizeof(struct vnode) <= 448, "vnode size crosses 448 bytes"); #endif #endif diff --git a/sys/ufs/ffs/ffs_vfsops.c b/sys/ufs/ffs/ffs_vfsops.c index 6b7407eb88f9..2ff71cc3e4d1 100644 --- a/sys/ufs/ffs/ffs_vfsops.c +++ b/sys/ufs/ffs/ffs_vfsops.c @@ -281,28 +281,8 @@ ffs_check_blkno(struct mount *mp, ino_t inum, ufs2_daddr_t daddr, int blksize) } /* - * Initiate a forcible unmount. + * On first ENXIO error, initiate an asynchronous forcible unmount. * Used to unmount filesystems whose underlying media has gone away. - */ -static void -ffs_fsfail_unmount(void *v, int pending) -{ - struct fsfail_task *etp; - struct mount *mp; - - etp = v; - - /* - * Find our mount and get a ref on it, then try to unmount. - */ - mp = vfs_getvfs(&etp->fsid); - if (mp != NULL) - dounmount(mp, MNT_FORCE, curthread); - free(etp, M_UFSMNT); -} - -/* - * On first ENXIO error, start a task that forcibly unmounts the filesystem. * * Return true if a cleanup is in progress. */ @@ -320,25 +300,18 @@ ffs_fsfail_cleanup(struct ufsmount *ump, int error) int ffs_fsfail_cleanup_locked(struct ufsmount *ump, int error) { - struct fsfail_task *etp; - struct task *tp; - mtx_assert(UFS_MTX(ump), MA_OWNED); if (error == ENXIO && (ump->um_flags & UM_FSFAIL_CLEANUP) == 0) { ump->um_flags |= UM_FSFAIL_CLEANUP; /* * Queue an async forced unmount. */ - etp = ump->um_fsfail_task; - ump->um_fsfail_task = NULL; - if (etp != NULL) { - tp = &etp->task; - TASK_INIT(tp, 0, ffs_fsfail_unmount, etp); - taskqueue_enqueue(taskqueue_thread, tp); - printf("UFS: forcibly unmounting %s from %s\n", - ump->um_mountp->mnt_stat.f_mntfromname, - ump->um_mountp->mnt_stat.f_mntonname); - } + vfs_ref(ump->um_mountp); + dounmount(ump->um_mountp, + MNT_FORCE | MNT_RECURSE | MNT_DEFERRED, curthread); + printf("UFS: forcibly unmounting %s from %s\n", + ump->um_mountp->mnt_stat.f_mntfromname, + ump->um_mountp->mnt_stat.f_mntonname); } return ((ump->um_flags & UM_FSFAIL_CLEANUP) != 0); } @@ -1046,7 +1019,6 @@ ffs_mountfs(odevvp, mp, td) struct g_consumer *cp; struct mount *nmp; struct vnode *devvp; - struct fsfail_task *etp; int candelete, canspeedup; off_t loc; @@ -1334,9 +1306,6 @@ ffs_mountfs(odevvp, mp, td) (void) ufs_extattr_autostart(mp, td); #endif /* !UFS_EXTATTR_AUTOSTART */ #endif /* !UFS_EXTATTR */ - etp = malloc(sizeof *ump->um_fsfail_task, M_UFSMNT, M_WAITOK | M_ZERO); - etp->fsid = mp->mnt_stat.f_fsid; - ump->um_fsfail_task = etp; return (0); out: if (fs != NULL) { @@ -1583,8 +1552,6 @@ ffs_unmount(mp, mntflags) free(fs->fs_csp, M_UFSMNT); free(fs->fs_si, M_UFSMNT); free(fs, M_UFSMNT); - if (ump->um_fsfail_task != NULL) - free(ump->um_fsfail_task, M_UFSMNT); free(ump, M_UFSMNT); mp->mnt_data = NULL; MNT_ILOCK(mp); diff --git a/sys/ufs/ufs/ufsmount.h b/sys/ufs/ufs/ufsmount.h index 6071faec795c..f2951d74d44c 100644 --- a/sys/ufs/ufs/ufsmount.h +++ b/sys/ufs/ufs/ufsmount.h @@ -67,10 +67,6 @@ struct inodedep; TAILQ_HEAD(inodedeplst, inodedep); LIST_HEAD(bmsafemaphd, bmsafemap); LIST_HEAD(trimlist_hashhead, ffs_blkfree_trim_params); -struct fsfail_task { - struct task task; - fsid_t fsid; -}; #include #include @@ -123,7 +119,6 @@ struct ufsmount { struct taskqueue *um_trim_tq; /* (c) trim request queue */ struct trimlist_hashhead *um_trimhash; /* (i) trimlist hash table */ u_long um_trimlisthashsize; /* (i) trim hash table size-1 */ - struct fsfail_task *um_fsfail_task; /* (i) task for fsfail cleanup*/ /* (c) - below function ptrs */ int (*um_balloc)(struct vnode *, off_t, int, struct ucred *, int, struct buf **); -- Test scenario: misc/force4.sh (interrupted)