KDB: enter: watchdog timeout, vmcore.298 KDB: debugger backends: ddb KDB: current backend: ddb ---<>--- Copyright (c) 1992-2021 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 14.0-CURRENT #2 main-n245255-483c6da3a20b-dirty: Thu Mar 4 20:13:53 CET 2021 pho@mercat1.netperf.freebsd.org:/usr/src/sys/amd64/compile/PHO amd64 FreeBSD clang version 11.0.1 (git@github.com:llvm/llvm-project.git llvmorg-11.0.1-0-g43ff75f2c3fe) WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. VT(vga): resolution 640x480 CPU: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz (3500.08-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306f2 Family=0x6 Model=0x3f Stepping=2 Features=0xbfebfbff Features2=0x7ffefbff AMD Features=0x2c100800 AMD Features2=0x21 Structured Extended Features=0x37ab XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics real memory = 34359738368 (32768 MB) avail memory = 33176981504 (31640 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs FreeBSD/SMP: 1 package(s) x 6 core(s) x 2 hardware threads random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" random: unblocking device. ioapic0 irqs 0-23 ioapic1 irqs 24-47 Launching APs: 1 11 3 2 8 10 9 4 6 7 5 Timecounter "TSC-low" frequency 1750038412 Hz quality 1000 KTLS: Initialized 12 threads random: entropy device external interface WARNING: Device "kbd" is Giant locked and may be deleted before FreeBSD 14.0. kbd1 at kbdmux0 000.000048 [4350] netmap_init netmap: loaded module [ath_hal] loaded mlx5en: Mellanox Ethernet driver 3.6.0 (December 2020) nexus0 vtvga0: smbios0: at iomem 0xf0560-0xf057e smbios0: Version: 2.8, BCD Revision: 2.7 cryptosoft0: aesni0: acpi0: acpi0: Power Button (fixed) cpu0: on acpi0 atrtc0: port 0x70-0x71,0x74-0x77 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Event timer "HPET4" frequency 14318180 Hz quality 340 Event timer "HPET5" frequency 14318180 Hz quality 340 Event timer "HPET6" frequency 14318180 Hz quality 340 Event timer "HPET7" frequency 14318180 Hz quality 340 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: on acpi0 pci0: on pcib0 pci0: at device 11.1 (no driver attached) pci0: at device 11.2 (no driver attached) pci0: at device 16.1 (no driver attached) pci0: at device 16.6 (no driver attached) pci0: at device 18.1 (no driver attached) acpi_syscontainer0: on acpi0 acpi_syscontainer1: on acpi0 acpi_syscontainer2: on acpi0 acpi_syscontainer3: on acpi0 apei0: on acpi0 pcib1: port 0xcf8-0xcff on acpi0 pci1: on pcib1 pcib2: irq 26 at device 1.0 on pci1 pci2: on pcib2 pcib3: irq 32 at device 2.0 on pci1 pci3: on pcib3 pci3: at device 0.0 (no driver attached) pci3: at device 0.1 (no driver attached) pcib4: irq 40 at device 3.0 on pci1 pci4: on pcib4 pci1: at device 17.0 (no driver attached) ahci0: port 0xf110-0xf117,0xf100-0xf103,0xf0f0-0xf0f7,0xf0e0-0xf0e3,0xf020-0xf03f mem 0xfb438000-0xfb4387ff irq 16 at device 17.4 on pci1 ahci0: AHCI v1.30 with 4 6Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahciem0: on ahci0 xhci0: mem 0xfb400000-0xfb40ffff irq 19 at device 20.0 on pci1 xhci0: 32 bytes context size, 64-bit DMA usbus0 on xhci0 usbus0: 5.0Gbps Super Speed USB v3.0 pci1: at device 22.0 (no driver attached) pci1: at device 22.1 (no driver attached) ehci0: mem 0xfb434000-0xfb4343ff irq 18 at device 26.0 on pci1 usbus1: EHCI version 1.0 usbus1 on ehci0 usbus1: 480Mbps High Speed USB v2.0 pcib5: irq 16 at device 28.0 on pci1 pci5: on pcib5 pcib6: irq 18 at device 28.2 on pci1 pci6: on pcib6 pcib7: at device 0.0 on pci6 pci7: on pcib7 vgapci0: port 0xe000-0xe07f mem 0xfa000000-0xfaffffff,0xfb000000-0xfb01ffff irq 18 at device 0.0 on pci7 vgapci0: Boot video device pcib8: irq 16 at device 28.4 on pci1 pci8: on pcib8 igb0: port 0xd020-0xd03f mem 0xfb320000-0xfb33ffff,0xfb344000-0xfb347fff irq 16 at device 0.0 on pci8 igb0: Using 1024 TX descriptors and 1024 RX descriptors igb0: queue equality override not set, capping rx_queues at 6 and tx_queues at 6 igb0: Using 6 RX queues 6 TX queues igb0: Using MSI-X interrupts with 7 vectors igb0:uhub3: 4 ports with 3 removable, self powered uhub4: 6 ports with 6 removable, self powered fscklock order reversal: 1st 0xfffffe010e3c20d8 ufs (ufs, lockmgr) @ kern/vfs_mount.c:1780 2nd 0xfffffe010e48e0d8 devfs (devfs, lockmgr) @ kern/vfs_subr.c:3132 lock order devfs -> ufs established at: #0 0xffffffff80c8044d at witness_checkorder+0x46d #1 0xffffffff80be0c32 at lockmgr_lock_flags+0x172 #2 0xffffffff80f42aad at ffs_lock+0x9d #3 0xffffffff811a33a0 at VOP_LOCK1_APV+0x40 #4 0xffffffff80d14bf4 at _vn_lock+0x54 #5 0xffffffff80cf2f21 at vfs_domount+0xf01 #6 0xffffffff80cf1062 at vfs_donmount+0x872 #7 0xffffffff80cf5ae7 at kernel_mount+0x57 #8 0xffffffff80cf84b1 at parse_mount+0x4a1 #9 0xffffffff80cf6917 at vfs_mountroot+0x587 #10 0xffffffff80b9e54f at start_init+0x1f #11 0xffffffff80bcb190 at fork_exit+0x80 #12 0xffffffff810b03ce at fork_trampoline+0xe lock order ufs -> devfs attempted at: #0 0xffffffff80c80dac at witness_checkorder+0xdcc #1 0xffffffff80be0c32 at lockmgr_lock_flags+0x172 #2 0xffffffff811a33a0 at VOP_LOCK1_APV+0x40 #3 0xffffffff80d14bf4 at _vn_lock+0x54 #4 0xffffffff80cfd116 at vput_final+0x11t ffs_unmount+0x54a #6 0xffffffff80cf400b at dounmount+0x43b #7 0xffffffff80cf3b5c at kern_unmount+0x2fc #8 0xffffffff810dd0a7 at amd64_syscall+0x147 #9 0xffffffff810afc6e at fast_syscall_common+0xf8 ** /dev/gpt/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ugen0.3: at usbus0 ukbd0 on uhub3 ukbd0: on usbus0 kbd2 at ukbd0 ugen2.2: at usbus2 uhub5 on uhub1 uhub5: on usbus2 ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 209 files, 26736 used, 1238671 free (111 frags, 154820 blocks, 0.0% fragmentation) ***** FILE SYSTEM MARKED CLEAN ***** uhub5: 8 ports with 8 removable, self powered Updating CPU Microcode... CPU: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz (3500.08-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306f2 Family=0x6 Model=0x3f Stepping=2 Features=0xbfebfbff Features2=0x7ffefbff AMD Features=0x2c100800 AMD Features2=0x21 Structured Extended Features=0x37ab Structured Extended Features3=0x9c000400 XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics Done. Setting up harvesting: PURE_RDRAND,[UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,[NET_ETHER],NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED Feeding entropy: . ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/lib/compat /usr/local/lib/compat/pkg /usr/local/lib/e2fsprogs /usr/local/lib/gcc8 /usr/local/lib/gcc9 /usr/local/lib/perl5/5.30/mach/CORE 32-bit compatibility ldconfig path: /usr/lib32 /usr/local/lib32/compat Setting hostname: mercat1.netperf.freebsd.org. Autoloading module: acpi_wmi.ko Autoloading module: ichsmb.ko Autoloading module: ioat.ko ioat0: mem 0xfb42c000-0xfb42ffff irq 31 at device 4.0 on pci1 ioat0: Capabilities: 2f7 ioat1: mem 0xfb428000-0xfb42bfff irq 39 at device 4.1 on pci1 ioat1: Capabilities: 2f7 ioat2: mem 0xfb424000-0xfb427fff irq 31 at device 4.2 on pci1 ioat2: Capabilities: f7 ioat3: mem 0xfb420000-0xfb423fff irq 39 at device 4.3 on pci1 ioat3: Capabilities: f7 ioat4: mem 0xfb41c000-0xfb41ffff irq 31 at device 4.4 on pci1 ioat4: Capabilities: f7 ioat5: mem 0xfb418000-0xfb41bfff irq 39 at device 4.5 on pci1 ioat5: Capabilities: f7 ioat6: mem 0xfb414000-0xfb417fff irq 31 at device 4.6 on pci1 ioat6: Capabilities: f7 ioat7: mem 0xfb410000-0xfb413fff irq 39 at device 4.7 on pci1 ioat7: Capabilities: f7 ichsmb0: port 0x580-0x59f mem 0xfb431000-0xfb4310ff irq 18 at device 31.3 on pci1 smbus0: on ichsmb0 acpi_wmi0: on acpi0 acpi_wmi0: cannot find EC device lo0: link state changed to UP Starting Network: lo0 igb0 igb1 mce0 mce1. lo0: flags=8049 metric 0 mtu 16384 options=680003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=21 igb0: flags=8863 metric 0 mtu 1500 options=4e507bb ether 0c:c4:7a:a8:cd:da inet6 fe80::ec4:7aff:fea8:cdda%igb0 prefixlen 64 scopeid 0x1 media: Ethernet autoselect status: no carrier nd6 options=21 igb1: flags=8822 metric 0 mtu 1500 options=4e507bb ether 0c:c4:7a:a8:cd:db media: Ethernet autoselect status: no carrier nd6 options=29 mce0: flags=8822 metric 0 mtu 1500 options=1eed07bb ether 7c:fe:90:30:ee:38 media: Ethernet autoselect status: no carrier nd6 options=29 mce1: flags=8822 metric 0 mtu 1500 options=1eed07bb ether 7c:fe:90:30:ee:39 media: Ethernet autoselect status: no carrier nd6 options=29 Starting devd. igb0: link state changed to UP Autoloading module: acpi_wmi.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Starting Network: igb1. igb1: flags=8822 metric 0 mtu 1500 options=4e507bb ether 0c:c4:7a:a8:cd:db media: Ethernet autoselect status: no carrier nd6 options=29 Autoloading module: ichsmb.ko Starting Network: mce0. mce0: flags=8822 metric 0 mtu 1500 options=1eed07bb ether 7c:fe:90:30:ee:38 media: Ethernet autoselect status: no carrier nd6 options=29 Starting Network: mce1. mce1: flags=8822 metric 0 mtu 1500 options=1eed07bb ether 7c:fe:90:30:ee:39 media: Ethernet autoselect status: no carrier nd6 options=29 Autoloading module: uhid.ko Autoloading module: ums.ko Autoloading module: usbhid.ko ums0 on uhub3 ums0: on usbus0 ums0: 3 buttons and [Z] coordinates ID=0 Starting dhclient. if_delmulti_locked: detaching ifnet instance 0xfffff80004857800 DHCPDISCOVER on igb0 to 255.255.255.255 port 67 interval 8 DHCPOFFER from 192.168.5.1 DHCPREQUEST on igb0 to 255.255.255.255 port 67 if_delmulti_locked: detaching ifnet instance 0xfffff80004857800 DHCPACK from 192.168.5.1 bound to 192.168.5.171 -- renewal in 300 seconds. add host 127.0.0.1: gateway lo0 fib 0: route already in table add host ::1: gateway lo0 fib 0: route already in table add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Updating motd:. Recovering vi editor sessions:. Creating and/or trimming log files. Clearing /tmp (X related). Starting syslogd. Setting date via ntp. 5 Mar 09:00:47 ntpdate[2617]: step time server 192.168.5.1 offset -0.423986 sec Starting watchdogd. No core dumps found. Starting rpcbind. NFS access cache time=60 Starting statd. Starting lockd. Mounting late filesystems:. Security policy loaded: MAC/ntpd (mac_ntpd) Starting ntpd. Starting powerd. Performing sanity check on sshd configuration. Starting sshd. Configuring vt: blanktime. Starting cron. Starting inetd. sysctl: unknown oid 'debug.fail_point.pmap_allocpte_page_alloc' at line 12 Fri Mar 5 09:00 FreeBSD/amd64 (mercat1.netperf.freebsd.org) (ttyu1) login: Mar 5 09:08:17 mercat1 su[2807]: pho to root on /dev/pts/0 20210305 09:09:48 all (1/162): bio.sh Mar 5 09:11:00 mercat1 kernel: pid 4613 (swap), jid 0, uid 0, was killed: out of swap space 20210305 09:15:31 all (2/162): procstat2.sh Mar 5 09:17:56 mercat1 kernel: pid 15981 (rw), uid 2007 inumber 98399 on /mnt: filesystem full Mar 5 09:18:15 mercat1 kernel: pid 15957 (swap), jid 0, uid 2007, was killed: out of swap space Mar 5 09:18:31 mercat1 kernel: pid 15984 (mkdir), uid 2007 inumber 76464 on /mnt: filesystem full Mar 5 09:18:59 mercat1 kernel: pid 16013 (link), uid 2007 inumber 98354 on /mnt: filesystem full 20210305 09:20:54 all (3/162): ext4fs.sh 20210305 09:26:02 all (4/162): fullpath2.sh 20210305 09:31:12 all (5/162): sendfile13.sh 20210305 09:36:38 all (6/162): ext3fs.sh Mar 5 09:37:01 mercat1 kernel: pid 4531 (swap), jid 0, uid 2007, was killed: out of swap space Mar 5 09:37:28 mercat1 kernel: pid 4544 (swap), jid 0, uid 2007, was killed: out of swap space 20210305 09:41:47 all (7/162): geli.sh GEOM_ELI: Device md10.eli created. GEOM_ELI: Encryption: AES-XTS 128 GEOM_ELI: Crypto: accelerated software GEOM_ELI: md10 has been killed. GEOM_ELI: Device md10.eli destroyed. stress2: newblk leak: 30404/30408. 20210305 09:47:47 all (8/162): crossmp10.sh 20210305 09:52:59 all (9/162): rename.sh 20210305 10:08:58 all (10/162): kevent5.sh lock order reversal: 1st 0xfffffe01487ca450 filedesc structure (filedesc structure, sx) @ kern/kern_descrip.c:1356 2nd 0xfffffe014a69d7e0 ufs (ufs, lockmgr) @ kern/vfs_subr.c:6093 lock order filedesc structure -> ufs attempted at: #0 0xffffffff80c80dac at witness_checkorder+0xdcc #1 0xffffffff80be0c32 at lockmgr_lock_flags+0x172 #2 0xffffffff80f42aad at ffs_lock+0x9d #3 0xffffffff811a33a0 at VOP_LOCK1_APV+0x40 #4 0xffffffff80d14bf4 at _vn_lock+0x54 #5 0xffffffff80bc025b at knlist_remove_kq+0x8b #6 0xffffffff80d08924 at filt_vfsdetach+0x24 #7 0xffffffff80bc0cc1 at knote_fdclose+0x181 #8 0xffffffff80bb940c at closefp_impl+0x9c #9 0xffffffff810dd0a7 at amd64_syscall+0x147 #10 0xffffffff810afc6e at fast_syscall_common+0xf8 20210305 10:15:00 all (11/162): sort2.sh Mar 5 10:15:14 mercat1 kernel: pid 5610 (sort), jid 0, uid 0, was killed: out of swap space Mar 5 10:15:25 mercat1 kernel: pid 5592 (sort), jid 0, uid 0, was killed: out of swap space 20210305 10:21:58 all (12/162): pmc3.sh hwpmc: SOFT/16/64/0x67 TSC/1/64/0x20 IAP/4/48/0x3ff IAF/3/48/0x67 lock order reversal: (sleepable after non-sleepable) 1st 0xfffff80003f38c08 seqflq (seqflq, sleep mutex) @ dev/sound/midi/sequencer.c:757 2nd 0xffffffff81cf3080 midistat lock (midistat lock, sx) @ dev/sound/midi/midi.c:1481 lock order seqflq -> midistat lock attempted at: #0 0xffffffff80c80dac at witness_checkorder+0xdcc #1 0xffffffff80c1b607 at _sx_xlock+0x67 #2 0xffffffff809adcd2 at midimapper_open+0x22 #3 0xffffffff809b0b62 at mseq_open+0xf2 #4 0xffffffff80aaa810 at devfs_open+0x130 #5 0xffffffff811a0566 at VOP_OPEN_APV+0x56 #6 0xffffffff80d14992 at vn_open_vnode+0x1a2 #7 0xffffffff80d143ed at vn_open_cred+0x54d #8 0xffffffff80d0af7f at kern_openat+0x26f #9 0xffffffff810dd0a7 at amd64_syscall+0x147 #10 0xffffffff810afc6e at fast_syscall_common+0xf8 20210305 10:28:01 all (13/162): aesni.sh GEOM_ELI: Device md10.eli created. GEOM_ELI: Encryption: AES-XTS 128 GEOM_ELI: Crypto: accelerated software GEOM_ELI: md10 has been killed. GEOM_ELI: Device md10.eli destroyed. 20210305 10:33:35 all (14/162): swap.sh 20210305 10:40:16 all (15/162): crossmp11.sh Mar 5 10:40:19 mercat1 kernel: pid 99314 (rw), uid 2007 inumber 49154 on /mnt10: filesystem full Mar 5 10:40:19 mercat1 kernel: pid 99324 (rw), uid 2007 inumber 24597 on /mnt12: filesystem full Mar 5 10:40:25 mercat1 kernel: pid 99357 (creat), uid 2007 inumber 49162 on /mnt12: out of inodes Mar 5 10:40:27 mercat1 kernel: pid 99390 (mkdir), uid 2007 inumber 98301 on /mnt11: out of inodes Mar 5 10:40:39 mercat1 kernel: pid 99412 (mkdir), uid 2007 inumber 24544 on /mnt10: out of inodes Mar 5 10:40:39 mercat1 kernel: pid 99381 (creat), uid 2007 inumber 49167 on /mnt12: out of inodes Mar 5 10:40:40 mercat1 kernel: pid 99416 (mkdir), uid 2007 inumber 24537 on /mnt11: out of inodes Mar 5 10:40:45 mercat1 kernel: pid 99307 (rw), uid 2007 inumber 29718 on /mnt11: filesystem full Mar 5 10:40:50 mercat1 kernel: pid 99385 (creat), uid 2007 inumber 49171 on /mnt12: out of inodes Mar 5 10:40:51 mercat1 kernel: pid 99325 (rw), uid 2007 inumber 67864 on /mnt13: filesystem full Mar 5 10:40:55 mercat1 kernel: pid 99322 (rw), uid 2007 inumber 57730 on /mnt13: filesystem full Mar 5 10:40:57 mercat1 kernel: pid 99378 (creat), uid 2007 inumber 49166 on /mnt12: out of inodes Mar 5 10:41:01 mercat1 kernel: pid 99372 (creat), uid 2007 inumber 49159 on /mnt12: out of inodes Mar 5 10:41:08 mercat1 kernel: pid 99414 (mkdir), uid 2007 inumber 49072 on /mnt10: out of inodes Mar 5 10:41:17 mercat1 kernel: pid 99378 (creat), uid 2007 inumber 49166 on /mnt12: out of inodes Mar 5 10:41:20 mercat1 kernel: pid 99355 (creat), uid 2007 inumber 24613 on /mnt12: out of inodes Mar 5 10:41:25 mercat1 kernel: pid 99333 (rw), uid 2007 inumber 82938 on /mnt13: filesystem full Mar 5 10:41:29 mercat1 kernel: pid 99328 (rw), uid 2007 inumber 10463 on /mnt13: filesystem full Mar 5 10:41:33 mercat1 kernel: pid 99360 (creat), uid 2007 inumber 4 on /mnt12: out of inodes Mar 5 10:41:37 mercat1 kernel: pid 99371 (creat), uid 2007 inumber 49161 on /mnt12: out of inodes Mar 5 10:41:40 mercat1 kernel: pid 99369 (creat), uid 2007 inumber 49170 on /mnt12: out of inodes Mar 5 10:41:42 mercat1 kernel: pid 99392 (mkdir), uid 2007 inumber 49152 on /mnt10: out of inodes Mar 5 10:41:44 mercat1 kernel: pid 99317 (rw), uid 2007 inumber 73792 on /mnt10: filesystem full Mar 5 10:41:51 mercat1 kernel: pid 99385 (creat), uid 2007 inumber 49171 on /mnt12: out of inodes Mar 5 10:42:08 mercat1 kernel: pid 99384 (creat), uid 2007 inumber 49163 on /mnt12: out of inodes Mar 5 10:42:18 mercat1 kernel: pid 99376 (creat), uid 2007 inumber 49169 on /mnt12: out of inodes Mar 5 10:42:20 mercat1 kernel: pid 12236 (rw), uid 2007 inumber 24594 on /mnt11: filesystem full Mar 5 10:42:20 mercat1 kernel: pid 12217 (rw), uid 2007 inumber 24597 on /mnt13: filesystem full Mar 5 10:42:21 mercat1 kernel: pid 12306 (rw), uid 2007 inumber 73742 on /mnt12: filesystem full Mar 5 10:42:21 mercat1 kernel: pid 12378 (rw), uid 2007 inumber 35 on /mnt10: filesystem full Mar 5 10:42:23 mercat1 kernel: pid 12211 (mkdir), uid 2007 inumber 7 on /mnt13: filesystem full Mar 5 10:42:24 mercat1 kernel: pid 12367 (rw), uid 2007 inumber 18741 on /mnt10: filesystem full Mar 5 10:42:25 mercat1 kernel: pid 12312 (mkdir), uid 2007 inumber 73727 on /mnt12: out of inodes Mar 5 10:42:26 mercat1 kernel: pid 12388 (mkdir), uid 2007 inumber 61202 on /mnt10: filesystem full Mar 5 10:42:28 mercat1 kernel: pid 12304 (creat), uid 2007 inumber 6 on /mnt12: out of inodes Mar 5 10:42:41 mercat1 kernel: pid 12321 (creat), uid 2007 inumber 11 on /mnt12: out of inodes Mar 5 10:42:45 mercat1 kernel: pid 12298 (creat), uid 2007 inumber 7 on /mnt12: out of inodes Mar 5 10:42:45 mercat1 kernel: pid 12200 (creat), uid 2007 inumber 24603 on /mnt13: out of inodes Mar 5 10:42:48 mercat1 kernel: pid 12245 (creat), uid 2007 inumber 24626 on /mnt11: out of inodes Mar 5 10:42:54 mercat1 kernel: pid 12323 (creat), uid 2007 inumber 49158 on /mnt12: out of inodes Mar 5 10:42:56 mercat1 kernel: pid 12370 (mkdir), uid 2007 inumber 73726 on /mnt10: out of inodes Mar 5 10:42:59 mercat1 kernel: pid 12239 (creat), uid 2007 inumber 24611 on /mnt11: out of inodes Mar 5 10:43:09 mercat1 kernel: pid 12377 (creat), uid 2007 inumber 28 on /mnt10: out of inodes Mar 5 10:43:15 mercat1 kernel: pid 12323 (creat), uid 2007 inumber 49158 on /mnt12: out of inodes Mar 5 10:43:25 mercat1 kernel: pid 12243 (creat), uid 2007 inumber 24627 on /mnt11: out of inodes Mar 5 10:43:42 mercat1 kernel: pid 12239 (creat), uid 2007 inumber 24611 on /mnt11: out of inodes Mar 5 10:43:43 mercat1 kernel: pid 12395 (creat), uid 2007 inumber 41 on /mnt10: out of inodes Mar 5 10:43:47 mercat1 kernel: pid 12320 (creat), uid 2007 inumber 24585 on /mnt12: out of inodes Mar 5 10:43:59 mercat1 kernel: pid 12202 (mkdir), uid 2007 inumber 23294 on /mnt13: out of inodes Mar 5 10:44:03 mercat1 kernel: pid 12203 (creat), uid 2007 inumber 24608 on /mnt13: out of inodes Mar 5 10:44:09 mercat1 kernel: pid 12200 (creat), uid 2007 inumber 24603 on /mnt13: out of inodes Mar 5 10:44:19 mercat1 kernel: pid 12245 (creat), uid 2007 inumber 24626 on /mnt11: out of inodes Mar 5 10:44:21 mercat1 kernel: pid 24148 (rw), uid 2007 inumber 49165 on /mnt13: filesystem full Mar 5 10:44:21 mercat1 kernel: pid 24056 (rw), uid 2007 inumber 24609 on /mnt12: filesystem full Mar 5 10:44:24 mercat1 kernel: pid 23961 (mkdir), uid 2007 inumber 84425 on /mnt10: filesystem full Mar 5 10:44:29 mercat1 kernel: pid 24083 (mkdir), uid 2007 inumber 97674 on /mnt12: out of inodes Mar 5 10:44:31 mercat1 kernel: pid 23959 (mkdir), uid 2007 inumber 21468 on /mnt10: out of inodes Mar 5 10:44:32 mercat1 kernel: pid 24102 (mkdir), uid 2007 inumber 98285 on /mnt12: out of inodes Mar 5 10:44:34 mercat1 kernel: pid 24009 (creat), uid 2007 inumber 24584 on /mnt11: filesystem full Mar 5 10:44:35 mercat1 kernel: pid 24103 (mkdir), uid 2007 inumber 23382 on /mnt12: out of inodes Mar 5 10:44:36 mercat1 kernel: pid 24025 (rw), uid 2007 inumber 74789 on /mnt11: filesystem full Mar 5 10:44:37 mercat1 kernel: pid 24081 (creat), uid 2007 inumber 24622 on /mnt12: out of inodes Mar 5 10:44:38 mercat1 kernel: pid 23962 (creat), uid 2007 inumber 24584 on /mnt10: out of inodes Mar 5 10:44:40 mercat1 kernel: pid 24087 (creat), uid 2007 inumber 24619 on /mnt12: out of inodes Mar 5 10:44:45 mercat1 kernel: pid 24072 (creat), uid 2007 inumber 24611 on /mnt12: out of inodes Mar 5 10:44:48 mercat1 kernel: pid 24070 (creat), uid 2007 inumber 24606 on /mnt12: out of inodes Mar 5 10:44:53 mercat1 kernel: pid 23958 (creat), uid 2007 inumber 24582 on /mnt10: out of inodes Mar 5 10:44:54 mercat1 kernel: pid 24008 (rw), uid 2007 inumber 27739 on /mnt11: filesystem full Mar 5 10:44:55 mercat1 kernel: pid 24094 (creat), uid 2007 inumber 24621 on /mnt12: out of inodes Mar 5 10:45:00 mercat1 kernel: pid 24100 (mkdir), uid 2007 inumber 98055 on /mnt12: out of inodes Mar 5 10:45:02 mercat1 kernel: pid 24018 (rw), uid 2007 inumber 79745 on /mnt11: filesystem full Mar 5 10:45:03 mercat1 kernel: pid 23947 (rw), uid 2007 inumber 49282 on /mnt10: filesystem full Mar 5 10:45:06 mercat1 kernel: pid 23970 (mkdir), uid 2007 inumber 83390 on /mnt10: filesystem full Mar 5 10:45:07 mercat1 kernel: pid 24090 (creat), uid 2007 inumber 24613 on /mnt12: out of inodes Mar 5 10:45:12 mercat1 kernel: pid 24029 (rw), uid 2007 inumber 78222 on /mnt11: filesystem full Mar 5 10:45:23 mercat1 kernel: pid 23960 (creat), uid 2007 inumber 24583 on /mnt10: out of inodes Mar 5 10:45:29 mercat1 kernel: pid 24026 (rw), uid 2007 inumber 87048 on /mnt11: filesystem full Mar 5 10:45:30 mercat1 kernel: pid 24141 (creat), uid 2007 inumber 73730 on /mnt13: out of inodes Mar 5 10:45:35 mercat1 kernel: pid 24023 (rw), uid 2007 inumber 85947 on /mnt11: filesystem full Mar 5 10:46:05 mercat1 kernel: pid 24007 (creat), uid 2007 inumber 73761 on /mnt11: out of inodes Mar 5 10:46:07 mercat1 kernel: pid 23962 (creat), uid 2007 inumber 24584 on /mnt10: out of inodes Mar 5 10:46:08 mercat1 kernel: pid 24058 (mkdir), uid 2007 inumber 98282 on /mnt12: out of inodes 20210305 10:46:21 all (16/162): stealer.sh Mar 5 10:46:51 mercat1 kernel: pid 35043 (stealer), jid 0, uid 2007, was killed: out of swap space Mar 5 10:47:10 mercat1 kernel: pid 35065 (stealer), jid 0, uid 2007, was killed: out of swap space Mar 5 10:47:33 mercat1 kernel: pid 35185 (stealer), jid 0, uid 2007, was killed: out of swap space swp_pager_getswapspace(2): failed stress2: newblk leak: 32763/32767. 20210305 10:50:37 all (17/162): suj3.sh GEOM_ELI: Device md10.eli created. GEOM_ELI: Encryption: AES-XTS 128 GEOM_ELI: Crypto: accelerated software GEOM_ELI: md10 has been killed. GEOM_ELI: Device md10.eli destroyed. 20210305 10:56:52 all (18/162): audit2.sh Mar 5 11:01:44 mercat1 kernel: pid 83040 (openat), uid 2007 inumber 32894 on /mnt: out of inodes Mar 5 11:03:04 mercat1 kernel: pid 83032 (rw), uid 2007 inumber 64384 on /mnt: filesystem full 20210305 11:03:14 all (19/162): isofs3.sh lock order reversal: 1st 0xfffffe014d6102c8 isofs (isofs, lockmgr) @ kern/vfs_mount.c:1071 2nd 0xfffffe015aeba070 ufs (ufs, lockmgr) @ kern/vfs_mount.c:1083 lock order ufs -> isofs established at: #0 0xffffffff80c8044d at witness_checkorder+0x46d #1 0xffffffff80be0c32 at lockmgr_lock_flags+0x172 #2 0xffffffff811a33a0 at VOP_LOCK1_APV+0x40 #3 0xffffffff80d14bf4 at _vn_lock+0x54 #4 0xffffffff80cf2f21 at vfs_domount+0xf01 #5 0xffffffff80cf1062 at vfs_donmount+0x872 #6 0xffffffff80cf07b9 at sys_nmount+0x69 #7 0xffffffff810dd0a7 at amd64_syscall+0x147 #8 0xffffffff810afc6e at fast_syscall_common+0xf8 lock order isofs -> ufs attempted at: #0 0xffffffff80c80dac at witness_checkorder+0xdcc #1 0xffffffff80be0c32 at lockmgr_lock_flags+0x172 #2 0xffffffff80f42aad at ffs_lock+0x9d #3 0xffffffff811a33a0 at VOP_LOCK1_APV+0x40 #4 0xffffffff80d14bf4 at _vn_lock+0x54 #5 0xffffffff80cf2f21 at vfs_domount+0xf01 #6 0xffffffff80cf1062 at vfs_donmount+0x872 #7 0xffffffff80cf07b9 at sys_nmount+0x69 #8 0xffffffff810dd0a7 at amd64_syscall+0x147 #9 0xffffffff810afc6e at fast_syscall_common+0xf8 Mar 5 11:03:28 mercat1 kernel: pid 86399 (rw), uid 0 inumber 109221 on /mnt: filesystem full Mar 5 11:06:48 mercat1 kernel: pid 88272 (mkdir), uid 0 inumber 92034 on /mnt: out of inodes Mar 5 11:07:00 mercat1 kernel: pid 88242 (symlink), uid 0 inumber 98341 on /mnt: out of inodes lock order reversal: 1st 0xfffffe014d6102c8 isofs (isofs, lockmgr) @ kern/vfs_mount.c:1780 2nd 0xfffffe01560a1a38 devfs (devfs, lockmgr) @ kern/vfs_subr.c:3132 lock order isofs -> devfs attempted at: #0 0xffffffff80c80dac at witness_checkorder+0xdcc #1 0xffffffff80be0c32 at lockmgr_lock_flags+0x172 #2 0xffffffff811a33a0 at VOP_LOCK1_APV+0x40 #3 0xffffffff80d14bf4 at _vn_lock+0x54 #4 0xffffffff80cfd116 at vput_final+0x116 #5 0xffffffff80f3e32a at ffs_unmount+0x54a #6 0xffffffff80cf400b at dounmount+0x43b #7 0xffffffff80cf3b5c at kern_unmount+0x2fc #8 0xffffffff810dd0a7 at amd64_syscall+0x147 #9 0xffffffff810afc6e at fast_syscall_common+0xf8 stress2: freework leak: 1084/1092. 20210305 11:09:34 all (20/162): umount2.sh 20210305 11:16:11 all (21/162): graid1_5.sh GEOM_NOP: Device md11.nop created. GEOM_NOP: Device md12.nop created. Mar 5 11:21:56 mercat1 kernel: pid 25035 (swap), jid 0, uid 2007, was killed: out of swap space GEOM_NOP: Device md11.nop removed. GEOM_NOP: Device md12.nop removed. 20210305 11:22:30 all (22/162): mkfifo2d.sh lock order reversal: 1st 0xfffffe017652d7e0 syncer (syncer, lockmgr) @ kern/vfs_subr.c:2523 2nd 0xfffffe01634c0c28 ufs (ufs, lockmgr) @ kern/vfs_subr.c:3132 lock order ufs -> syncer established at: #0 0xffffffff80c8044d at witness_checkorder+0x46d #1 0xffffffff80be0c32 at lockmgr_lock_flags+0x172 #2 0xffffffff811a33a0 at VOP_LOCK1_APV+0x40 #3 0xffffffff80d14bf4 at _vn_lock+0x54 #4 0xffffffff80cfd116 at vput_final+0x116 #5 0xffffffff80cf3ffd at dounmount+0x42d #6 0xffffffff80cf3b5c at kern_unmount+0x2fc #7 0xffffffff810dd0a7 at amd64_syscall+0x147 #8 0xffffffff810afc6e at fast_syscall_common+0xf8 lock order syncer -> ufs attempted at: #0 0xffffffff80c80dac at witness_checkorder+0xdcc #1 0xffffffff80be0c32 at lockmgr_lock_flags+0x172 #2 0xffffffff80f42aad at ffs_lock+0x9d #3 0xffffffff811a33a0 at VOP_LOCK1_APV+0x40 #4 0xffffffff80d14bf4 at _vn_lock+0x54 #5 0xffffffff80cfd116 at vput_final+0x116 #6 0xffffffff80cfca0b at vget_finish+0x9b #7 0xffffffff80f41b1b at ffs_sync_lazy+0x12b #8 0xffffffff80f3e813 at ffs_sync+0xc3 #9 0xffffffff80d0863b at sync_fsync+0xeb #10 0xffffffff811a1fd9 at VOP_FSYNC_APV+0x59 #11 0xffffffff80d05a14 at sched_sync+0x444 #12 0xffffffff80bcb190 at fork_exit+0x80 #13 0xffffffff810b03ce at fork_trampoline+0xe 20210305 11:28:51 all (23/162): graid1_7.sh 20210305 11:35:03 all (24/162): mmap38.sh Mar 5 11:38:26 mercat1 kernel: pid 42149 (swap), jid 0, uid 0, was killed: out of swap space Mar 5 11:39:20 mercat1 kernel: pid 42161 (swap), jid 0, uid 0, was killed: out of swap space 20210305 11:41:34 all (25/162): swap2.sh Mar 5 11:41:45 mercat1 kernel: pid 47855 (swap2), jid 0, uid 0, was killed: out of swap space Mar 5 11:41:56 mercat1 kernel: pid 47848 (swap2), jid 0, uid 0, was killed: out of swap space Mar 5 11:42:08 mercat1 kernel: pid 47876 (swap2), jid 0, uid 0, was killed: out of swap space Mar 5 11:42:18 mercat1 kernel: pid 47850 (swap2), jid 0, uid 0, was killed: out of swap space 20210305 11:47:30 all (26/162): dtrace.sh Mar 5 11:48:39 mercat1 kernel: pid 65654 (swap), jid 0, uid 2007, was killed: out of swap space 20210305 11:53:58 all (27/162): pmc.sh hwpmc: SOFT/16/64/0x67 TSC/1/64/0x20 IAP/4/48/0x3ff IAF/3/48/0x67 Mar 5 11:54:43 mercat1 kernel: pid 82203 (swap), jid 0, uid 0, was killed: out of swap space Mar 5 11:54:55 mercat1 kernel: pid 82207 (swap), jid 0, uid 0, was killed: out of swap space lock order reversal: 1st 0xffffffff81e9bd38 kernel linker (kernel linker, sx) @ kern/kern_linker.c:1136 2nd 0xffffffff81e9fcc0 pmc-sx (pmc-sx, sx) @ /usr/src/sys/dev/hwpmc/hwpmc_mod.c:5783 lock order pmc-sx -> kernel linker established at: lock order kernel linker -> pmc-sx attempted at: 20210305 12:00:22 all (28/162): suj29.sh Mar 5 12:00:57 mercat1 kernel: pid 99983 (swap), jid 0, uid 2007, was killed: out of swap space 20210305 12:06:40 all (29/162): truss.sh 20210305 12:12:23 all (30/162): smrstress2.sh Mar 5 12:12:39 mercat1 kernel: pid 57400 (rw), uid 0 inumber 96131 on /mnt: filesystem full Mar 5 12:12:46 mercat1 kernel: pid 57406 (rw), uid 0 inumber 115888 on /mnt: filesystem full Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures lock order reversal: 1st 0xffffffff81a02880 proctree (proctree, sx) @ kern/imgact_elf.c:2398 2nd 0xfffffe0149974e80 ufs (ufs, lockmgr) @ kern/vfs_vnops.c:625 lock order ufs -> proctree established at: #0 0xffffffff80c8044d at witness_checkorder+0x46d #1 0xffffffff80c1c6a7 at _sx_slock_int+0x67 #2 0xffffffff80c62dab at uprintf+0x6b #3 0xffffffff80f00986 at ffs_balloc_ufs2+0x19f6 #4 0xffffffff80f424a2 at ffs_write+0x262 #5 0xffffffff811a17af at VOP_WRITE_APV+0xcf #6 0xffffffff80d1937b at vn_write+0x2eb #7 0xffffffff80d18c63 at vn_io_fault_doio+0x43 #8 0xffffffff80d153bc at vn_io_fault1+0x16c #9 0xffffffff80d12fba at vn_io_fault+0x1ba #10 0xffffffff80c86281 at dofilewrite+0x81 #11 0xffffffff80c85dfc at sys_write+0xbc #12 0xffffffff810dd0a7 at amd64_syscall+0x147 #13 0xffffffff810afc6e at fast_syscall_common+0xf8 lock order proctree -> ufs attempted at: #0 0xffffffff80c80dac at witness_checkorder+0xdcc #1 0xffffffff80be0c32 at lockmgr_lock_flags+0x172 #2 0xffffffff80f42aad at ffs_lock+0x9d #3 0xffffffff811a33a0 at VOP_LOCK1_APV+0x40 #4 0xffffffff80d14bf4 at _vn_lock+0x54 #5 0xffffffff80d15077 at vn_rdwr+0x1a7 #6 0xffffffff80d15690 at vn_rdwr_inchunks+0x90 #7 0xffffffff80b94675 at sbuf_drain_core_output+0xa5 #8 0xffffffff80c6a6d8 at sbuf_drain+0x58 #9 0xffffffff80c69c1d at sbuf_put_bytes+0x9d #10 0xffffffff80c69b6e at sbuf_bcat+0xe #11 0xffffffff80bf57db at kern_proc_out+0xeb #12 0xffffffff80b94fc6 at elf64_note_procstat_proc+0x96 #13 0xffffffff80b93dde at elf64_coredump+0xeee #14 0xffffffff80c16135 at sigexit+0xd95 #15 0xffffffff80c1692c at postsig+0x2cc #16 0xffffffff80c75bcb at ast+0x4fb #17 0xffffffff810b1d59 at doreti_ast+0x1f Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Mar 5 12:18:21 mercat1 kernel: pid 62419 (rw), uid 0 inumber 155978 on /mnt: filesystem full Completed 10000000 loops with 0 failures Completed 10000000 loops with 0 failures Mar 5 12:18:35 mercat1 kernel: pid 62441 (rw), uid 0 inumber 28406 on /mnt: filesystem full Completed 10000000 loops with 0 failures 20210305 12:18:54 all (31/162): sysctl2.sh stress2: freework leak: 1289/1313. stress2: newblk leak: 50378/50383. 20210305 12:25:45 all (32/162): gnop6.sh GEOM_NOP: Device md10.nop created. Mar 5 12:29:20 mercat1 kernel: pid 3188 (swap), jid 0, uid 2007, was killed: out of swap space GEOM_NOP: Device md10.nop removed. 20210305 12:30:56 all (33/162): msync2.sh 20210305 12:37:41 all (34/162): mkfifo4.sh 20210305 12:45:50 all (35/162): sem_post.sh Mar 5 12:47:50 mercat1 kernel: pid 95726 (swap), jid 0, uid 0, was killed: out of swap space Mar 5 12:48:01 mercat1 kernel: pid 95728 (swap), jid 0, uid 0, was killed: out of swap space Mar 5 12:48:11 mercat1 kernel: pid 95700 (swap), jid 0, uid 0, was killed: out of swap space 20210305 12:53:51 all (36/162): mincore.sh 20210305 12:59:50 all (37/162): chain.sh swap_pager: out of swap space swp_pager_getswapspace(18): failed Mar 5 13:06:58 mercat1 kernel: pid 66395 (chain), jid 0, uid 0, was killed: out of swap space 20210305 13:07:08 all (38/162): posix_openpt2.sh stress2: pts leak: 55/56 20210305 13:13:25 all (39/162): sendfile7.sh Mar 5 13:15:10 mercat1 kernel: pid 686 (swap), jid 0, uid 0, was killed: out of swap space Mar 5 13:18:16 mercat1 kernel: pid 13766 (swap), jid 0, uid 0, was killed: out of swap space Mar 5 13:19:36 mercat1 kernel: pid 13768 (swap), jid 0, uid 0, was killed: out of swap space 20210305 13:22:03 all (40/162): datamove2.sh Mar 5 13:25:02 mercat1 kernel: pid 16444 (swap), jid 0, uid 0, was killed: out of swap space Mar 5 13:25:02 mercat1 kernel: pid 16384 (swap), jid 0, uid 0, was killed: out of swap space Mar 5 13:26:42 mercat1 kernel: pid 16397 (swap), jid 0, uid 0, was killed: out of swap space Mar 5 13:27:56 mercat1 kernel: pid 18660 (swap), jid 0, uid 0, was killed: out of swap space Mar 5 13:29:51 mercat1 kernel: pid 18741 (swap), jid 0, uid 0, was killed: out of swap space Mar 5 13:30:54 mercat1 kernel: pid 18697 (swap), jid 0, uid 0, was killed: out of swap space Mar 5 13:31:27 mercat1 kernel: pid 18708 (swap), jid 0, uid 0, was killed: out of swap space 20210305 13:32:42 all (41/162): mmap40.sh Mar 5 13:33:25 mercat1 kernel: pid 21168 (swap), jid 0, uid 0, was killed: out of swap space Mar 5 13:33:55 mercat1 kernel: pid 21173 (swap), jid 0, uid 0, was killed: out of swap space Mar 5 13:34:13 mercat1 kernel: pid 21166 (swap), jid 0, uid 0, was killed: out of swap space Mar 5 13:34:54 mercat1 kernel: pid 21161 (swap), jid 0, uid 0, was killed: out of swap space Mar 5 13:35:07 mercat1 kernel: pid 21172 (swap), jid 0, uid 0, was killed: out of swap space Mar 5 13:35:18 mercat1 kernel: pid 21163 (swap), jid 0, uid 0, was killed: out of swap space 20210305 13:40:08 all (42/162): crossmp3.sh Mar 5 13:46:48 mercat1 kernel: pid 26810 (rw), uid 2007 inumber 161785 on /mnt20: filesystem full Mar 5 13:47:36 mercat1 kernel: pid 26816 (rw), uid 2007 inumber 175305 on /mnt20: filesystem full Mar 5 13:47:47 mercat1 kernel: pid 26707 (rw), uid 2007 inumber 324376 on /mnt16: filesystem full Mar 5 13:48:11 mercat1 kernel: pid 26461 (rw), uid 2007 inumber 394238 on /mnt14: filesystem full Mar 5 13:48:13 mercat1 kernel: pid 26717 (rw), uid 2007 inumber 221345 on /mnt16: filesystem full 20210305 13:48:28 all (43/162): buildkernel.sh 20210305 13:53:16 all (44/162): crossmp4.sh interrupt total irq3: uart1 1503 irq4: uart0 5 irq9: acpi0 4 irq18: ehci0 ehci1+ 45315 irq19: xhci0 27977 cpu0:timer 17531167 cpu1:timer 16581588 cpu2:timer 17177205 cpu3:timer 17211137 cpu4:timer 17321839 cpu5:timer 17174150 cpu6:timer 17044959 cpu7:timer 16934219 cpu8:timer 17082272 cpu9:timer 17191634 cpu10:timer 17786328 cpu11:timer 17047308 irq56: ahci0 7285693 irq57: igb0:rxq0 12817 irq58: igb0:rxq1 951 irq59: igb0:rxq2 64 irq60: igb0:rxq3 191 irq62: igb0:rxq5 257 irq63: igb0:aq 2 irq73: mlx5_core0 41 irq74: mlx5_core0 1 irq88: mlx5_core1 41 irq89: mlx5_core1 1 Total 213458703 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe01507d7e80 hardclock() at hardclock+0x1bd/frame 0xfffffe01507d7ec0 handleevents() at handleevents+0xb4/frame 0xfffffe01507d7f00 timercb() at timercb+0x196/frame 0xfffffe01507d7f50 lapic_handle_timer() at lapic_handle_timer+0x9b/frame 0xfffffe01507d7f80 Xtimerint() at Xtimerint+0xb1/frame 0xfffffe01507d7f80 --- interrupt, rip = 0xffffffff80c5e9c2, rsp = 0xfffffe01507d8050, rbp = 0xfffffe01507d8060 --- lock_delay() at lock_delay+0x22/frame 0xfffffe01507d8060 __mtx_lock_sleep() at __mtx_lock_sleep+0xe8/frame 0xfffffe01507d80f0 __mtx_lock_flags() at __mtx_lock_flags+0xe5/frame 0xfffffe01507d8140 vnlru_free_locked() at vnlru_free_locked+0xae/frame 0xfffffe01507d81c0 vn_alloc_hard() at vn_alloc_hard+0xd5/frame 0xfffffe01507d81f0 getnewvnode() at getnewvnode+0x234/frame 0xfffffe01507d8230 ffs_vgetf() at ffs_vgetf+0x20b/frame 0xfffffe01507d83a0 ffs_vget() at ffs_vget+0x33/frame 0xfffffe01507d83e0 ufs_lookup_ino() at ufs_lookup_ino+0xf08/frame 0xfffffe01507d8550 VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0x56/frame 0xfffffe01507d8570 vfs_cache_lookup() at vfs_cache_lookup+0xa6/frame 0xfffffe01507d85c0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x56/frame 0xfffffe01507d85e0 cache_fplookup_noentry() at cache_fplookup_noentry+0x237/frame 0xfffffe01507d8650 cache_fplookup() at cache_fplookup+0x615/frame 0xfffffe01507d8740 namei() at namei+0x129/frame 0xfffffe01507d8800 vn_open_cred() at vn_open_cred+0x459/frame 0xfffffe01507d8970 kern_openat() at kern_openat+0x26f/frame 0xfffffe01507d8ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01507d8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01507d8bf0 --- syscall (5, FreeBSD ELF64, sys_open), rip = 0x8011af4ca, rsp = 0x7fffffffdfb8, rbp = 0x7fffffffe4f0 --- KDB: enter: watchdog timeout [ thread pid 65962 tid 101081 ] Stopped at kdb_enter+0x37: movq $0,0x1288f3e(%rip) db> run pho db:0:pho> set $lines 20000 db:0:pho> run pho1 db:1:pho1> dump Dumping 3904 out of 32626 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Dump complete db:1:pho1> bt Tracing pid 65962 tid 101081 td 0xfffffe014636aa00 kdb_enter() at kdb_enter+0x37/frame 0xfffffe01507d7e80 hardclock() at hardclock+0x1d0/frame 0xfffffe01507d7ec0 handleevents() at handleevents+0xb4/frame 0xfffffe01507d7f00 timercb() at timercb+0x196/frame 0xfffffe01507d7f50 lapic_handle_timer() at lapic_handle_timer+0x9b/frame 0xfffffe01507d7f80 Xtimerint() at Xtimerint+0xb1/frame 0xfffffe01507d7f80 --- interrupt, rip = 0xffffffff80c5e9c2, rsp = 0xfffffe01507d8050, rbp = 0xfffffe01507d8060 --- lock_delay() at lock_delay+0x22/frame 0xfffffe01507d8060 __mtx_lock_sleep() at __mtx_lock_sleep+0xe8/frame 0xfffffe01507d80f0 __mtx_lock_flags() at __mtx_lock_flags+0xe5/frame 0xfffffe01507d8140 vnlru_free_locked() at vnlru_free_locked+0xae/frame 0xfffffe01507d81c0 vn_alloc_hard() at vn_alloc_hard+0xd5/frame 0xfffffe01507d81f0 getnewvnode() at getnewvnode+0x234/frame 0xfffffe01507d8230 ffs_vgetf() at ffs_vgetf+0x20b/frame 0xfffffe01507d83a0 ffs_vget() at ffs_vget+0x33/frame 0xfffffe01507d83e0 ufs_lookup_ino() at ufs_lookup_ino+0xf08/frame 0xfffffe01507d8550 VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0x56/frame 0xfffffe01507d8570 vfs_cache_lookup() at vfs_cache_lookup+0xa6/frame 0xfffffe01507d85c0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x56/frame 0xfffffe01507d85e0 cache_fplookup_noentry() at cache_fplookup_noentry+0x237/frame 0xfffffe01507d8650 cache_fplookup() at cache_fplookup+0x615/frame 0xfffffe01507d8740 namei() at namei+0x129/frame 0xfffffe01507d8800 vn_open_cred() at vn_open_cred+0x459/frame 0xfffffe01507d8970 kern_openat() at kern_openat+0x26f/frame 0xfffffe01507d8ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01507d8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01507d8bf0 --- syscall (5, FreeBSD ELF64, sys_open), rip = 0x8011af4ca, rsp = 0x7fffffffdfb8, rbp = 0x7fffffffe4f0 --- db:1:pho1> show allpcpu Current CPU: 10 cpuid = 0 dynamic pcpu = 0x7affc0 curthread = 0xfffffe00e4c53300: pid 11 tid 100003 critnest 1 "idle: cpu0" curpcb = 0xfffffe00e4c53810 fpcurthread = none idlethread = 0xfffffe00e4c53300: tid 100003 "idle: cpu0" self = 0xffffffff82410000 curpmap = 0xffffffff81e9a5a0 tssp = 0xffffffff82410384 rsp0 = 0xfffffe0037790cc0 kcr3 = 0x800000000234e002 ucr3 = 0xffffffffffffffff scr3 = 0x24735931 gs32p = 0xffffffff82410404 ldt = 0xffffffff82410444 tss = 0xffffffff82410434 curvnet = 0 spin locks held: cpuid = 1 dynamic pcpu = 0xfffffe00bb28afc0 curthread = 0xfffffe0146807500: pid 32 tid 962695 critnest 0 "/mnt worker" curpcb = 0xfffffe0146807a10 fpcurthread = none idlethread = 0xfffffe00e4c52c00: tid 100004 "idle: cpu1" self = 0xffffffff82411000 curpmap = 0xffffffff81e9a5a0 tssp = 0xffffffff82411384 rsp0 = 0xfffffe01511c4cc0 kcr3 = 0x800000000234e002 ucr3 = 0xffffffffffffffff scr3 = 0x6f047ae07 gs32p = 0xffffffff82411404 ldt = 0xffffffff82411444 tss = 0xffffffff82411434 curvnet = 0 spin locks held: cpuid = 2 dynamic pcpu = 0xfffffe00bb297fc0 curthread = 0xfffffe010e151c00: pid 2705 tid 100236 critnest 0 "ntpd" curpcb = 0xfffffe010e152110 fpcurthread = 0xfffffe010e151c00: pid 2705 "ntpd" idlethread = 0xfffffe00e4c52500: tid 100005 "idle: cpu2" self = 0xffffffff82412000 curpmap = 0xfffffe0142797b10 tssp = 0xffffffff82412384 rsp0 = 0xfffffe01415aacc0 kcr3 = 0x8000000024f40410 ucr3 = 0x8000000024735c10 scr3 = 0x5e2ebce05 gs32p = 0xffffffff82412404 ldt = 0xffffffff82412444 tss = 0xffffffff82412434 curvnet = 0 spin locks held: cpuid = 3 dynamic pcpu = 0xfffffe00bb2a4fc0 curthread = 0xfffffe0143dd2a00: pid 65959 tid 100283 critnest 0 "umount" curpcb = 0xfffffe0143dd2f10 fpcurthread = 0xfffffe0143dd2a00: pid 65959 "umount" idlethread = 0xfffffe00e4c51e00: tid 100006 "idle: cpu3" self = 0xffffffff82413000 curpmap = 0xfffffe010e5f8120 tssp = 0xffffffff82413384 rsp0 = 0xfffffe01428dccc0 kcr3 = 0x800000077e92d241 ucr3 = 0x800000034160ca41 scr3 = 0x24735c3d gs32p = 0xffffffff82413404 ldt = 0xffffffff82413444 tss = 0xffffffff82413434 curvnet = 0 spin locks held: cpuid = 4 dynamic pcpu = 0xfffffe00bb2b1fc0 curthread = 0xfffffe01427f6100: pid 2746 tid 100259 critnest 0 "cron" curpcb = 0xfffffe01427f6610 fpcurthread = 0xfffffe01427f6100: pid 2746 "cron" idlethread = 0xfffffe00e4c51700: tid 100007 "idle: cpu4" self = 0xffffffff82414000 curpmap = 0xfffffe010e53b120 tssp = 0xffffffff82414384 rsp0 = 0xfffffe014288fcc0 kcr3 = 0x80000006f037f605 ucr3 = 0x80000006f0388e05 scr3 = 0x3876c5825 gs32p = 0xffffffff82414404 ldt = 0xffffffff82414444 tss = 0xffffffff82414434 curvnet = 0 spin locks held: cpuid = 5 dynamic pcpu = 0xfffffe00bb2befc0 curthread = 0xfffffe0143ee5700: pid 65964 tid 100385 critnest 0 "dhclient" curpcb = 0xfffffe0143ee5c10 fpcurthread = 0xfffffe0143ee5700: pid 65964 "dhclient" idlethread = 0xfffffe00e4c51000: tid 100008 "idle: cpu5" self = 0xffffffff82415000 curpmap = 0xfffffe0148994b10 tssp = 0xffffffff82415384 rsp0 = 0xfffffe01444d3cc0 kcr3 = 0x80000001c07040bb ucr3 = 0x80000003876c58bb scr3 = 0x24735821 gs32p = 0xffffffff82415404 ldt = 0xffffffff82415444 tss = 0xffffffff82415434 curvnet = 0 spin locks held: cpuid = 6 dynamic pcpu = 0xfffffe00bb2cbfc0 curthread = 0xfffffe00e4c54800: pid 11 tid 100009 critnest 1 "idle: cpu6" curpcb = 0xfffffe00e4c54d10 fpcurthread = none idlethread = 0xfffffe00e4c54800: tid 100009 "idle: cpu6" self = 0xffffffff82416000 curpmap = 0xffffffff81e9a5a0 tssp = 0xffffffff82416384 rsp0 = 0xfffffe00377aecc0 kcr3 = 0x800000000234e002 ucr3 = 0xffffffffffffffff scr3 = 0x80176eb8b gs32p = 0xffffffff82416404 ldt = 0xffffffff82416444 tss = 0xffffffff82416434 curvnet = 0 spin locks held: cpuid = 7 dynamic pcpu = 0xfffffe00bb2d8fc0 curthread = 0xfffffe00e4c5a100: pid 11 tid 100010 critnest 1 "idle: cpu7" curpcb = 0xfffffe00e4c5a610 fpcurthread = none idlethread = 0xfffffe00e4c5a100: tid 100010 "idle: cpu7" self = 0xffffffff82417000 curpmap = 0xffffffff81e9a5a0 tssp = 0xffffffff82417384 rsp0 = 0xfffffe00377a9cc0 kcr3 = 0x800000000234e002 ucr3 = 0xffffffffffffffff scr3 = 0x15f373ea6 gs32p = 0xffffffff82417404 ldt = 0xffffffff82417444 tss = 0xffffffff82417434 curvnet = 0 spin locks held: cpuid = 8 dynamic pcpu = 0xfffffe00bb2e5fc0 curthread = 0xfffffe00e4c59a00: pid 11 tid 100011 critnest 1 "idle: cpu8" curpcb = 0xfffffe00e4c59f10 fpcurthread = none idlethread = 0xfffffe00e4c59a00: tid 100011 "idle: cpu8" self = 0xffffffff82418000 curpmap = 0xffffffff81e9a5a0 tssp = 0xffffffff82418384 rsp0 = 0xfffffe00377b8cc0 kcr3 = 0x800000000234e002 ucr3 = 0xffffffffffffffff scr3 = 0x24735d3f gs32p = 0xffffffff82418404 ldt = 0xffffffff82418444 tss = 0xffffffff82418434 curvnet = 0 spin locks held: cpuid = 9 dynamic pcpu = 0xfffffe00bb2f2fc0 curthread = 0xfffffe01478c1800: pid 65961 tid 101475 critnest 0 "sed" curpcb = 0xfffffe01478c1d10 fpcurthread = 0xfffffe01478c1800: pid 65961 "sed" idlethread = 0xfffffe00e4c59300: tid 100012 "idle: cpu9" self = 0xffffffff82419000 curpmap = 0xfffffe014d787500 tssp = 0xffffffff82419384 rsp0 = 0xfffffe0150f8acc0 kcr3 = 0x80000006a37c2718 ucr3 = 0x80000006de565f18 scr3 = 0x2fccdbac0 gs32p = 0xffffffff82419404 ldt = 0xffffffff82419444 tss = 0xffffffff82419434 curvnet = 0 spin locks held: cpuid = 10 dynamic pcpu = 0xfffffe00bb2fffc0 curthread = 0xfffffe014636aa00: pid 65962 tid 101081 critnest 1 "sed" curpcb = 0xfffffe014636af10 fpcurthread = 0xfffffe014636aa00: pid 65962 "sed" idlethread = 0xfffffe00e4c58c00: tid 100013 "idle: cpu10" self = 0xffffffff8241a000 curpmap = 0xfffffe010e476500 tssp = 0xffffffff8241a384 rsp0 = 0xfffffe01507d8cc0 kcr3 = 0x80000005373c7734 ucr3 = 0x80000005e2ebcf34 scr3 = 0x3876c5aa9 gs32p = 0xffffffff8241a404 ldt = 0xffffffff8241a444 tss = 0xffffffff8241a434 curvnet = 0 spin locks held: cpuid = 11 dynamic pcpu = 0xfffffe00bb30cfc0 curthread = 0xfffffe00e4c58500: pid 11 tid 100014 critnest 1 "idle: cpu11" curpcb = 0xfffffe00e4c58a10 fpcurthread = none idlethread = 0xfffffe00e4c58500: tid 100014 "idle: cpu11" self = 0xffffffff8241b000 curpmap = 0xffffffff81e9a5a0 tssp = 0xffffffff8241b384 rsp0 = 0xfffffe00377bdcc0 kcr3 = 0x800000000234e003 ucr3 = 0xffffffffffffffff scr3 = 0x2f6c559d7 gs32p = 0xffffffff8241b404 ldt = 0xffffffff8241b444 tss = 0xffffffff8241b434 curvnet = 0 spin locks held: db:1:pho1> show alllocks Process 65966 (ls) thread 0xfffffe0148139100 (101323) shared lockmgr ufs (ufs) r = 0 (0xfffffe010e3bac90) locked @ kern/vfs_lookup.c:842 Process 65964 (dhclient) thread 0xfffffe0143ee5700 (100385) exclusive lockmgr ufs (ufs) r = 0 (0xfffffe014774f330) locked @ kern/vfs_subr.c:1531 shared lockmgr ufs (ufs) r = 0 (0xfffffe010e3ba7e0) locked @ kern/vfs_cache.c:4627 Process 65962 (sed) thread 0xfffffe014636aa00 (101081) shared lockmgr ufs (ufs) r = 0 (0xfffffe01440857e0) locked @ kern/vfs_cache.c:4627 Process 65961 (sed) thread 0xfffffe01478c1800 (101475) shared lockmgr ufs (ufs) r = 0 (0xfffffe01440857e0) locked @ kern/vfs_cache.c:4627 Process 65959 (umount) thread 0xfffffe0143dd2a00 (100283) exclusive lockmgr ufs (ufs) r = 0 (0xfffffe01612b70d8) locked @ kern/vfs_mount.c:1780 Process 2746 (cron) thread 0xfffffe01427f6100 (100259) shared lockmgr ufs (ufs) r = 0 (0xfffffe0143ec69d0) locked @ kern/vfs_cache.c:4627 Process 2705 (ntpd) thread 0xfffffe010e151c00 (100236) shared lockmgr ufs (ufs) r = 0 (0xfffffe010e3c6588) locked @ kern/vfs_lookup.c:842 Process 2609 (syslogd) thread 0xfffffe010a30d000 (100248) exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe003a588730) locked @ kern/vfs_bio.c:1674 exclusive lockmgr ufs (ufs) r = 0 (0xfffffe0143eaa9d0) locked @ kern/vfs_vnops.c:1111 Process 32 (bufdaemon) thread 0xfffffe0146807500 (962695) exclusive lockmgr ufs (ufs) r = 0 (0xfffffe018a3cd330) locked @ kern/vfs_subr.c:1531 db:1:pho1> show lockedvnods Locked vnodes vnode 0xfffffe010e3bac20: type VDIR usecount 33, writecount 0, refcount 4 seqc users 0 mountedhere 0 hold count flags () flags (VV_ROOT|VMP_LAZYLIST) v_object 0xfffff80003ba5e70 ref 0 pages 1 cleanbuf 1 dirtybuf 0 lock type ufs: SHARED (count 1) #0 0xffffffff80be0c95 at lockmgr_lock_flags+0x1d5 #1 0xffffffff80f42aad at ffs_lock+0x9d #2 0xffffffff811a33a0 at VOP_LOCK1_APV+0x40 #3 0xffffffff80d14bf4 at _vn_lock+0x54 #4 0xffffffff80cee4b5 at lookup+0xd5 #5 0xffffffff80cedb6e at namei+0x57e #6 0xffffffff80d0d32d at kern_statat+0x11d #7 0xffffffff80d0da1f at sys_fstatat+0x2f #8 0xffffffff810dd0a7 at amd64_syscall+0x147 #9 0xffffffff810afc6e at fast_syscall_common+0xf8 nlink=42, effnlink=42, size=2048, extsize 0 generation=56909bb9, uid=0, gid=0, flags=0x0 ino 2, on dev gpt/work vnode 0xfffffe010e3ba770: type VDIR usecount 1, writecount 0, refcount 4 seqc users 0 mountedhere 0 hold count flags () flags () v_object 0xfffff80003ba5c60 ref 0 pages 1 cleanbuf 1 dirtybuf 0 lock type ufs: SHARED (count 1) #0 0xffffffff80be0c95 at lockmgr_lock_flags+0x1d5 #1 0xffffffff80f42aad at ffs_lock+0x9d #2 0xffffffff811a33a0 at VOP_LOCK1_APV+0x40 #3 0xffffffff80d14bf4 at _vn_lock+0x54 #4 0xffffffff80ce3e01 at cache_fplookup_noentry+0x1d1 #5 0xffffffff80cdfa05 at cache_fplookup+0x615 #6 0xffffffff80ced719 at namei+0x129 #7 0xffffffff80bc26f1 at kern_execve+0x261 #8 0xffffffff80bc211a at sys_execve+0x5a #9 0xffffffff810dd0a7 at amd64_syscall+0x147 #10 0xffffffff810afc6e at fast_syscall_common+0xf8 nlink=2, effnlink=2, size=3072, extsize 0 generation=eae2c345, uid=0, gid=0, flags=0x0 ino 90528768, on dev gpt/work vnode 0xfffffe010e3c6518: type VDIR usecount 1, writecount 0, refcount 4 seqc users 0 mountedhere 0 hold count flags () flags () v_object 0xfffff800241ad948 ref 0 pages 1 cleanbuf 1 dirtybuf 0 lock type ufs: SHARED (count 1) #0 0xffffffff80be0c95 at lockmgr_lock_flags+0x1d5 #1 0xffffffff80f42aad at ffs_lock+0x9d #2 0xffffffff811a33a0 at VOP_LOCK1_APV+0x40 #3 0xffffffff80d14bf4 at _vn_lock+0x54 #4 0xffffffff80cee4b5 at lookup+0xd5 #5 0xffffffff80cedb6e at namei+0x57e #6 0xffffffff80d13faf at vn_open_cred+0x10f #7 0xffffffff80d0af7f at kern_openat+0x26f #8 0xffffffff810dd0a7 at amd64_syscall+0x147 #9 0xffffffff810afc6e at fast_syscall_common+0xf8 nlink=11, effnlink=11, size=512, extsize 0 generation=b684d0d6, uid=0, gid=0, flags=0x0 ino 7704602, on dev gpt/work vnode 0xfffffe0143eaa960: type VREG usecount 1, writecount 1, refcount 3 seqc users 0 hold count flags () flags (VIRF_PGREAD|VMP_LAZYLIST) v_object 0xfffff8002432b000 ref 0 pages 8 cleanbuf 1 dirtybuf 0 lock type ufs: EXCL by thread 0xfffffe010a30d000 (pid 2609, syslogd, tid 100248) #0 0xffffffff80be0c95 at lockmgr_lock_flags+0x1d5 #1 0xffffffff80f42aad at ffs_lock+0x9d #2 0xffffffff811a33a0 at VOP_LOCK1_APV+0x40 #3 0xffffffff80d14bf4 at _vn_lock+0x54 #4 0xffffffff80d19260 at vn_write+0x1d0 #5 0xffffffff80d18c63 at vn_io_fault_doio+0x43 #6 0xffffffff80d153bc at vn_io_fault1+0x16c #7 0xffffffff80d12fba at vn_io_fault+0x1ba #8 0xffffffff80c86281 at dofilewrite+0x81 #9 0xffffffff80c861ae at sys_writev+0x6e #10 0xffffffff810dd0a7 at amd64_syscall+0x147 #11 0xffffffff810afc6e at fast_syscall_common+0xf8 nlink=1, effnlink=1, size=552079, extsize 0 generation=e742c11c, uid=0, gid=0, flags=0x0 ino 9951785, on dev gpt/work vnode 0xfffffe0143ec6960: type VDIR usecount 2, writecount 0, refcount 3 seqc users 0 mountedhere 0 hold count flags () flags () v_object 0xfffff80024409a50 ref 0 pages 1 cleanbuf 1 dirtybuf 0 lock type ufs: SHARED (count 1) #0 0xffffffff80be0c95 at lockmgr_lock_flags+0x1d5 #1 0xffffffff80f42aad at ffs_lock+0x9d #2 0xffffffff811a33a0 at VOP_LOCK1_APV+0x40 #3 0xffffffff80d14bf4 at _vn_lock+0x54 #4 0xffffffff80ce3e01 at cache_fplookup_noentry+0x1d1 #5 0xffffffff80cdfa05 at cache_fplookup+0x615 #6 0xffffffff80ced719 at namei+0x129 #7 0xffffffff80d0d32d at kern_statat+0x11d #8 0xffffffff80d0da1f at sys_fstatat+0x2f #9 0xffffffff810dd0a7 at amd64_syscall+0x147 #10 0xffffffff810afc6e at fast_syscall_common+0xf8 nlink=3, effnlink=3, size=512, extsize 0 generation=cb5f56a, uid=0, gid=0, flags=0x0 ino 7704619, on dev gpt/work vnode 0xfffffe0144085770: type VDIR usecount 2, writecount 0, refcount 4 seqc users 0 mountedhere 0 hold count flags () flags () v_object 0xfffff800245d1d68 ref 0 pages 1 cleanbuf 1 dirtybuf 0 lock type ufs: SHARED (count 2) #0 0xffffffff80be0c95 at lockmgr_lock_flags+0x1d5 #1 0xffffffff80f42aad at ffs_lock+0x9d #2 0xffffffff811a33a0 at VOP_LOCK1_APV+0x40 #3 0xffffffff80d14bf4 at _vn_lock+0x54 #4 0xffffffff80ce3e01 at cache_fplookup_noentry+0x1d1 #5 0xffffffff80cdfa05 at cache_fplookup+0x615 #6 0xffffffff80ced719 at namei+0x129 #7 0xffffffff80d142f9 at vn_open_cred+0x459 #8 0xffffffff80d0af7f at kern_openat+0x26f #9 0xffffffff810dd0a7 at amd64_syscall+0x147 #10 0xffffffff810afc6e at fast_syscall_common+0xf8 nlink=2, effnlink=2, size=512, extsize 0 generation=8a1ad1b, uid=0, gid=0, flags=0x0 ino 52246850, on dev gpt/work vnode 0xfffffe01612b7068: type VDIR usecount 2, writecount 0, refcount 2 seqc users 1 mountedhere 0xfffffe0148bdab80 hold count flags () flags (VIRF_MOUNTPOINT) v_object 0xfffff807035cf108 ref 0 pages 0 cleanbuf 0 dirtybuf 0 lock type ufs: EXCL by thread 0xfffffe0143dd2a00 (pid 65959, umount, tid 100283) with shared waiters pending #0 0xffffffff80be0c95 at lockmgr_lock_flags+0x1d5 #1 0xffffffff80f42aad at ffs_lock+0x9d #2 0xffffffff811a33a0 at VOP_LOCK1_APV+0x40 #3 0xffffffff80d14bf4 at _vn_lock+0x54 #4 0xffffffff80cf3c49 at dounmount+0x79 #5 0xffffffff80cf3b5c at kern_unmount+0x2fc #6 0xffffffff810dd0a7 at amd64_syscall+0x147 #7 0xffffffff810afc6e at fast_syscall_common+0xf8 nlink=2, effnlink=2, size=1536, extsize 0 generation=74d5cf41, uid=0, gid=0, flags=0x0 ino 37078272, on dev gpt/work vnode 0xfffffe014774f2c0: type VDIR usecount 0, writecount 0, refcount 1 seqc users 0 mountedhere 0 hold count flags () flags () lock type ufs: EXCL by thread 0xfffffe0143ee5700 (pid 65964, dhclient, tid 100385) #0 0xffffffff80be0c95 at lockmgr_lock_flags+0x1d5 #1 0xffffffff80f42aad at ffs_lock+0x9d #2 0xffffffff811a33a0 at VOP_LOCK1_APV+0x40 #3 0xffffffff80cf9908 at vnlru_free_locked+0x2d8 #4 0xffffffff80d05215 at vn_alloc_hard+0xd5 #5 0xffffffff80cf9e74 at getnewvnode+0x234 #6 0xffffffff80f3b10b at ffs_vgetf+0x20b #7 0xffffffff80f3aef3 at ffs_vget+0x33 #8 0xffffffff80f4d048 at ufs_lookup_ino+0xf08 #9 0xffffffff8119fe76 at VOP_CACHEDLOOKUP_APV+0x56 #10 0xffffffff80cddb66 at vfs_cache_lookup+0xa6 #11 0xffffffff8119fcc6 at VOP_LOOKUP_APV+0x56 #12 0xffffffff80ce3e67 at cache_fplookup_noentry+0x237 #13 0xffffffff80cdfa05 at cache_fplookup+0x615 #14 0xffffffff80ced719 at namei+0x129 #15 0xffffffff80bc26f1 at kern_execve+0x261 #16 0xffffffff80bc211a at sys_execve+0x5a #17 0xffffffff810dd0a7 at amd64_syscall+0x147 nlink=1, effnlink=0, size=0, extsize 0 generation=6c89ff64, uid=2007, gid=0, flags=0x0 ino 2172628, on dev md10a vnode 0xfffffe018a3cd2c0: type VDIR usecount 0, writecount 0, refcount 1 seqc users 0 mountedhere 0 hold count flags () flags () lock type ufs: EXCL by thread 0xfffffe0146807500 (pid 32, bufdaemon, tid 962695) #0 0xffffffff80be0c95 at lockmgr_lock_flags+0x1d5 #1 0xffffffff80f42aad at ffs_lock+0x9d #2 0xffffffff811a33a0 at VOP_LOCK1_APV+0x40 #3 0xffffffff80cf9908 at vnlru_free_locked+0x2d8 #4 0xffffffff80d05215 at vn_alloc_hard+0xd5 #5 0xffffffff80cf9e74 at getnewvnode+0x234 #6 0xffffffff80f3b10b at ffs_vgetf+0x20b #7 0xffffffff80f1e8ca at handle_workitem_remove+0x8a #8 0xffffffff80f0ebc8 at process_worklist_item+0x578 #9 0xffffffff80f09a45 at softdep_process_worklist+0x105 #10 0xffffffff80f0d036 at softdep_flush+0x156 #11 0xffffffff80bcb190 at fork_exit+0x80 #12 0xffffffff810b03ce at fork_trampoline+0xe nlink=1, effnlink=0, size=0, extsize 0 generation=bed7b6e6, uid=2007, gid=0, flags=0x0 ino 2246837, on dev md10a db:1:pho1> show mount 0xfffffe010de7e700 /dev/gpt/work on / (ufs) 0xfffffe010de7f280 devfs on /dev (devfs) 0xfffffe010e5ef280 /dev/gpt/tmp on /tmp (ufs) 0xfffffe0148bdab80 /dev/md10a on /mnt (ufs) More info: show mount db:1:pho1> show bufqueues bqempty: 56016 Buf domain 0 freebufs 15101 lofreebufs 219 hifreebufs 329 bufspace 696320 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 0 (0) dirtyq count 0 (0) wakeup 0 lim 25 CPU 0, 4, 0, 5, 8, 1, 3, 3, 4, 0, 3, 1, Locked buffers: 0 space 0 Total buffers: 8163 space 696320 Buf domain 1 freebufs 14461 lofreebufs 219 hifreebufs 329 bufspace 21762048 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 613 (19886080) dirtyq count 0 (0) wakeup 0 lim 25 CPU 1, 18, 9, 5, 13, 3, 4, 1, 1, 0, 2, 2, Locked buffers: 0 space 0 Total buffers: 20818 space 21762048 Buf domain 2 freebufs 15118 lofreebufs 219 hifreebufs 329 bufspace 167936 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 0 (0) dirtyq count 0 (0) wakeup 0 lim 25 CPU 0, 0, 0, 2, 7, 0, 0, 0, 0, 0, 4, 1, Locked buffers: 1 space 32768 Total buffers: 12161 space 167936 Buf domain 3 freebufs 15120 lofreebufs 219 hifreebufs 329 bufspace 53248 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 0 (0) dirtyq count 0 (0) wakeup 0 lim 25 CPU 0, 0, 0, 0, 9, 0, 1, 0, 0, 0, 3, 0, Locked buffers: 0 space 0 Total buffers: 13524 space 53248 Buf domain 4 freebufs 15112 lofreebufs 219 hifreebufs 329 bufspace 286720 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 0 (0) dirtyq count 0 (0) wakeup 0 lim 25 CPU 3, 1, 0, 0, 11, 0, 1, 0, 1, 0, 4, 0, Locked buffers: 0 space 0 Total buffers: 16859 space 286720 Buf domain 5 freebufs 15114 lofreebufs 219 hifreebufs 329 bufspace 86016 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 0 (0) dirtyq count 0 (0) wakeup 0 lim 25 CPU 0, 0, 1, 0, 16, 0, 0, 0, 1, 0, 1, 0, Locked buffers: 0 space 0 Total buffers: 20690 space 86016 Buf domain 6 freebufs 15118 lofreebufs 219 hifreebufs 329 bufspace 98304 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 0 (0) dirtyq count 0 (0) wakeup 0 lim 25 CPU 0, 0, 2, 0, 11, 0, 0, 0, 0, 0, 2, 0, Locked buffers: 0 space 0 Total buffers: 13716 space 98304 db:1:pho1> show page vm_cnt.v_free_count: 439634 vm_cnt.v_inactive_count: 1776916 vm_cnt.v_active_count: 4536533 vm_cnt.v_laundry_count: 598116 vm_cnt.v_wire_count: 766131 vm_cnt.v_free_reserved: 10644 vm_cnt.v_free_min: 51264 vm_cnt.v_free_target: 173124 vm_cnt.v_inactive_target: 259686 db:1:pho1> show pageq pq_free 439634 dom 0 page_cnt 8124114 free 439634 pq_act 4536533 pq_inact 1776916 pq_laund 598116 pq_unsw 0 db:0:pho> show freepages DOMAIN: 0 FREE LIST 0: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000000 | 000000 10 (004096K) | 000000 | 000000 09 (002048K) | 000000 | 000000 08 (001024K) | 000000 | 000000 07 (000512K) | 000033 | 000000 06 (000256K) | 000453 | 000000 05 (000128K) | 001592 | 000001 04 (000064K) | 005764 | 000001 03 (000032K) | 009240 | 000000 02 (000016K) | 018984 | 000000 01 (000008K) | 032555 | 000000 00 (000004K) | 046570 | 000001 FREE LIST 1: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000000 | 000000 10 (004096K) | 000000 | 000000 09 (002048K) | 000000 | 000000 08 (001024K) | 000000 | 000000 07 (000512K) | 000000 | 000000 06 (000256K) | 000001 | 000000 05 (000128K) | 000001 | 000000 04 (000064K) | 000000 | 000000 03 (000032K) | 000002 | 000000 02 (000016K) | 000002 | 000000 01 (000008K) | 000002 | 000000 00 (000004K) | 000003 | 000000 db:0:pho> show uma Zone Size Used Free Requests Sleeps Bucket Total Mem XFree FFS inode 1160 619482 802 117624159 0 31 719529440 0 malloc-384 384 818952 193148 125184623 0 32 388646400 0 VNODE 600 619504 2078 163490504 0 63 372949200 0 malloc-256 256 822856 194204 263943088 0 63 260367360 0 VM OBJECT 264 382603 242822 151800154 0 30 165112200 0 FFS2 dinode 256 619475 880 117445935 0 62 158810880 0 swblk 136 63999 998619 4932681 0 62 144516048 0 malloc-4096 4096 34115 35 5750990 0 2 139878400 0 malloc-128 128 745979 6856 281666694 0 126 96362880 0 RADIX NODE 144 488010 42699 114136860 0 65 76422096 0 mbuf_cluster 2048 13538 4 66219 0 8 27734016 0 malloc-256 256 1611 61134 213241177 0 71 16062720 0 BUF TRIE 144 258 105694 77164032 0 63 15257088 0 swpctrie 144 5844 65164 367373 0 62 10225152 0 THREAD 1792 1422 4266 5855585 0 8 10192896 0 vmem btag 56 157628 18832 1889648 0 254 9881760 0 malloc-128 128 12519 62315 78215128 0 126 9578752 0 S VFS Cache 104 19218 45405 89521403 0 126 6720792 0 UMA Slabs 0 112 44235 1914 285675 0 126 5168688 0 mbuf 256 15503 2330 35095908 0 254 4565248 0 malloc-64 64 53 66979 571368248 0 254 4290048 0 malloc-128 128 26012 462 1078033385 0 126 3388672 0 VMSPACE 2544 46 1160 3162786 0 4 3068064 0 64 Bucket 512 3812 1508 35121900 0 30 2723840 0 malloc-4096 4096 420 183 3594621 0 2 2469888 0 32 Bucket 256 5346 3159 70560108 0 62 2177280 0 malloc-128 128 14424 1355 2110231 0 126 2019712 0 pbuf 2776 0 699 0 0 0 1940424 0 128 Bucket 1024 1638 85 20662409 0 16 1764352 0 malloc-32768 32768 9 42 13603 0 1 1671168 0 PROC 1320 80 1144 3237638 0 8 1615680 0 256 Bucket 2048 479 285 34003945 0 17 1564672 0 MAP ENTRY 96 1692 13470 116403349 0 126 1455552 0 malloc-65536 65536 6 12 4514 0 1 1179648 0 malloc-32768 32768 1 34 24888 0 1 1146880 0 malloc-128 128 8505 392 19385752 0 126 1138816 0 malloc-64 64 9531 5337 17340712 0 254 951552 0 malloc-8192 8192 99 0 386 0 1 811008 0 Files 80 139 9961 1306770470 0 126 808000 0 TURNSTILE 136 5689 107 5689 0 62 788256 0 g_bio 408 4 1736 454624069 0 52 709920 0 malloc-64 64 9397 872 104626 0 254 657216 0 filedesc0 1072 81 521 3237639 0 8 645344 0 mbuf_packet 256 6 2141 8281495 0 254 549632 0 SLEEPQUEUE 88 5689 231 5689 0 126 520960 0 malloc-384 384 1277 63 1678 0 30 514560 0 UMA Zones 2048 229 6 274 0 8 481280 0 malloc-1024 1024 313 151 31014856 0 16 475136 0 malloc-64 64 3925 3383 5450681 0 254 467712 0 malloc-256 256 1446 369 6045487 0 62 464640 0 malloc-32768 32768 13 0 48139 0 1 425984 0 16 Bucket 144 152 2620 59020181 0 62 399168 0 malloc-32768 32768 12 0 127395 0 1 393216 0 DIRHASH 1024 164 208 2820289 0 16 380928 0 ksiginfo 112 1326 1482 41510 0 126 314496 0 malloc-32 32 8006 1822 159901 0 254 314496 0 ttyoutq 256 613 587 18572363 0 62 307200 0 malloc-16384 16384 17 1 436713 0 1 294912 0 malloc-2048 2048 7 137 7305865 0 8 294912 0 ttyinq 160 1155 620 34837824 0 62 284000 0 malloc-16 16 14644 2106 199248 0 254 268000 0 socket 944 52 228 17150 0 16 264320 0 NAMEI 1024 6 238 2287484145 0 16 249856 0 malloc-256 256 180 765 19152773 0 62 241920 0 malloc-2048 2048 110 6 14160242 0 8 237568 0 malloc-8192 8192 28 0 65064 0 1 229376 0 malloc-1024 1024 3 221 2651158 0 16 229376 0 malloc-65536 65536 3 0 1041 0 1 196608 0 8 Bucket 80 527 1923 61411063 0 126 196000 0 malloc-1024 1024 72 116 2320882 0 16 192512 0 2 Bucket 32 302 5368 6519933 0 254 181440 0 malloc-512 512 56 264 404770 0 30 163840 0 pipe 744 11 209 668530 0 16 163680 0 PWD 32 17 4897 66969036 0 254 157248 0 4 Bucket 48 397 2879 1866736 0 254 157248 0 Mountpoints 2944 4 48 239 0 4 153088 0 rl_entry 40 1210 2527 1214 0 254 149480 0 clpbuf 2776 0 51 2743321 0 2 141576 0 malloc-65536 65536 2 0 2 0 1 131072 0 malloc-65536 65536 2 0 34 0 1 131072 0 malloc-4096 4096 32 0 5902 0 2 131072 0 malloc-256 256 30 435 71691 0 62 119040 0 malloc-128 128 579 351 31141 0 126 119040 0 malloc-64 64 21 1806 640825 0 254 116928 0 PGRP 88 32 1256 17592 0 126 113344 0 DEVCTL 1024 0 104 415479 0 0 106496 0 swrbuf 2776 0 37 1071282 0 2 102712 0 malloc-16384 16384 6 0 4468 0 1 98304 0 malloc-8192 8192 12 0 13134797 0 1 98304 0 malloc-64 64 148 1112 12153501 0 254 80640 0 malloc-32 32 576 1944 304682 0 254 80640 0 malloc-32 32 119 2401 451585 0 254 80640 0 UMA Kegs 384 198 5 243 0 30 77952 0 malloc-256 256 35 265 1469164 0 62 76800 0 malloc-128 128 85 504 5542127 0 126 75392 0 malloc-4096 4096 18 0 297 0 2 73728 0 malloc-2048 2048 32 2 200213 0 8 69632 0 swwbuf 2776 0 25 2791967 0 1 69400 0 malloc-384 384 30 150 74465 0 30 69120 0 malloc-64 64 5 1066 1396050556 0 254 68544 0 malloc-64 64 46 1025 1717345 0 254 68544 0 malloc-65536 65536 1 0 1 0 1 65536 0 malloc-65536 65536 1 0 1 0 1 65536 0 malloc-65536 65536 1 0 522434 0 1 65536 0 fakepg 104 0 585 405 0 126 60840 0 malloc-16 16 847 2903 5880204 0 254 60000 0 tcpcb 1048 11 44 144 0 8 57640 0 malloc-16 16 494 3006 1987762 0 254 56000 0 malloc-384 384 110 30 55088 0 30 53760 0 malloc-4096 4096 13 0 24378 0 2 53248 0 malloc-32 32 362 1276 227530 0 254 52416 0 malloc-8192 8192 6 0 46 0 1 49152 0 malloc-16 16 1 2749 1658965 0 254 44000 0 malloc-384 384 36 74 8056 0 30 42240 0 malloc-8192 8192 5 0 369 0 1 40960 0 pcpu-8 8 3490 1630 3704 0 254 40960 0 malloc-32 32 283 977 9496265 0 254 40320 0 malloc-128 128 58 252 317 0 126 39680 0 cpuset 104 27 345 27 0 126 38688 0 malloc-4096 4096 2 7 92978 0 2 36864 0 malloc-1024 1024 20 16 34985 0 16 36864 0 malloc-512 512 30 42 72 0 30 36864 0 pcpu-64 64 456 120 472 0 254 36864 0 malloc-32 32 73 1061 1062 0 254 36288 0 malloc-16 16 4 2246 3449151 0 254 36000 0 unpcb 256 16 119 16081 0 62 34560 0 malloc-32768 32768 1 0 1 0 1 32768 0 malloc-32768 32768 1 0 2 0 1 32768 0 malloc-32768 32768 1 0 1 0 1 32768 0 malloc-16384 16384 2 0 2 0 1 32768 0 malloc-8192 8192 4 0 4 0 1 32768 0 malloc-4096 4096 7 1 48 0 2 32768 0 malloc-32 32 692 316 1109 0 254 32256 0 udp_inpcb 488 22 34 852 0 30 27328 0 malloc-384 384 28 42 119 0 30 26880 0 malloc-256 256 2 88 17732688 0 62 23040 0 malloc-1024 1024 12 8 4482 0 16 20480 0 malloc-512 512 26 14 403094 0 30 20480 0 rtentry 176 12 103 16 0 62 20240 0 malloc-32 32 6 624 4711 0 254 20160 0 malloc-16 16 56 1194 84 0 254 20000 0 malloc-16 16 25 1225 26343 0 254 20000 0 smrs 8 0 2465 20000001 0 254 19720 0 tcp_inpcb 488 11 29 144 0 30 19520 0 routing nhops 256 9 66 16 0 62 19200 0 L VFS Cache 320 4 56 34 0 30 19200 0 malloc-16384 16384 1 0 1 0 1 16384 0 malloc-16384 16384 1 0 1 0 1 16384 0 malloc-16384 16384 1 0 8666 0 1 16384 0 pcpu-16 16 52 972 52 0 254 16384 0 udpcb 32 22 482 852 0 254 16128 0 malloc-256 256 21 39 8536467 0 62 15360 0 malloc-512 512 3 21 14487 0 30 12288 0 SMR CPU 32 46 337 46 0 254 12256 0 malloc-16 16 11 739 11 0 254 12000 0 SMR SHARED 24 46 337 46 0 254 9192 0 malloc-8192 8192 1 0 1 0 1 8192 0 malloc-2048 2048 3 1 2071 0 8 8192 0 malloc-1024 1024 2 6 26648 0 16 8192 0 malloc-1024 1024 3 5 774587 0 16 8192 0 malloc-1024 1024 3 5 532 0 16 8192 0 malloc-512 512 6 10 52 0 30 8192 0 KMAP ENTRY 96 45 36 248 0 0 7776 0 kenv 258 18 12 21843 0 30 7740 0 FPU_save_area 832 1 8 1 0 16 7488 0 epoch_record pcpu 256 4 12 4 0 62 4096 0 malloc-4096 4096 1 0 132499 0 2 4096 0 malloc-2048 2048 1 1 18251 0 8 4096 0 malloc-2048 2048 2 0 3 0 8 4096 0 TMPFS node 224 0 18 2 0 62 4032 0 vtnet_tx_hdr 24 0 167 0 0 254 4008 0 ripcb 488 1 7 3 0 30 3904 0 UMA Slabs 1 176 9 13 9 0 62 3872 0 malloc-384 384 1 9 1 0 30 3840 0 malloc-384 384 1 9 35770153 0 30 3840 0 mqnode 416 3 6 3 0 30 3744 0 vmem 1856 1 1 1 0 8 3712 0 mqnotifier 216 0 0 0 0 62 0 0 mvdata 64 0 0 0 0 254 0 0 mqueue 248 0 0 0 0 62 0 0 FFS1 dinode 128 0 0 0 0 126 0 0 IPsec SA lft_c 16 0 0 0 0 254 0 0 udplite_inpcb 488 0 0 0 0 30 0 0 tcp_log_node 120 0 0 0 0 126 0 0 tcp_log_bucket 176 0 0 0 0 62 0 0 tcp_log 416 0 0 0 0 30 0 0 tcpreass 48 0 0 0 0 254 0 0 tfo_ccache_entries 80 0 0 0 0 126 0 0 tfo 4 0 0 0 0 254 0 0 sackhole 32 0 0 0 0 254 0 0 hostcache 96 0 0 2 0 126 0 0 syncache 168 0 0 63 0 62 0 0 tcptw 88 0 0 60 0 126 0 0 ipq 56 0 0 0 0 254 0 0 KNOTE 160 0 0 403727 0 62 0 0 itimer 352 0 0 0 0 30 0 0 AIOLIO 272 0 0 0 0 30 0 0 AIOCB 552 0 0 0 0 16 0 0 AIOP 32 0 0 0 0 254 0 0 AIO 208 0 0 0 0 62 0 0 NCLNODE 736 0 0 0 0 16 0 0 LTS VFS Cache 360 0 0 0 0 30 0 0 STS VFS Cache 144 0 0 0 0 62 0 0 cryptop 280 0 0 18245763 0 30 0 0 linux_dma_object 24 0 0 0 0 254 0 0 linux_dma_pctrie 144 0 0 0 0 62 0 0 IOMMU_MAP_ENTRY 120 0 0 0 0 126 0 0 ktls_session 192 0 0 0 0 62 0 0 mbuf_jumbo_16k 16384 0 0 0 0 1 0 0 mbuf_jumbo_9k 9216 0 0 0 0 1 0 0 mbuf_jumbo_page 4096 0 0 8376622 0 2 0 0 audit_record 1280 0 0 2 0 8 0 0 domainset 40 0 0 0 0 254 0 0 MAC labels 40 0 0 0 0 254 0 0 vnpbuf 2776 0 0 11004 0 4 0 0 mdpbuf 2776 0 0 38764 0 1 0 0 nfspbuf 2776 0 0 0 0 2 0 0 umtx_shm 88 0 0 0 0 126 0 0 umtx pi 96 0 0 0 0 126 0 0 rangeset pctrie nodes 144 0 0 0 0 62 0 0 malloc-65536 65536 0 0 0 0 1 0 0 malloc-32768 32768 0 0 0 0 1 0 0 malloc-16384 16384 0 0 2 0 1 0 0 malloc-16384 16384 0 0 0 0 1 0 0 malloc-8192 8192 0 0 0 0 1 0 0 malloc-2048 2048 0 0 0 0 8 0 0 malloc-2048 2048 0 0 13311365 0 8 0 0 malloc-512 512 0 0 120 0 30 0 0 malloc-512 512 0 0 4 0 30 0 0 malloc-512 512 0 0 72574 0 30 0 0 pcpu-32 32 0 0 0 0 254 0 0 pcpu-4 4 0 0 0 0 254 0 0 UMA Hash 256 0 0 0 0 62 0 0 db:0:pho> show umacache Zone Size Used Free Requests Bucket debugnet_mbuf_packet 2048 0 0 0 0 debugnet_mbuf_cluster 2048 -6144 0 0 0 debugnet_mbuf 256 -10240 0 0 0 ktls_buffers 16384 0 0 0 1 vm pgcache 4096 365914 2030 105681139 19 vm pgcache 4096 5923118 5824 981416130 169 buf free cache 848 787 49128 141875599 61 buffer arena-65 65536 0 0 0 1 buffer arena-61 61440 0 0 0 1 buffer arena-57 57344 0 0 0 1 buffer arena-53 53248 0 0 0 1 buffer arena-49 49152 0 0 0 1 buffer arena-45 45056 0 0 0 1 buffer arena-40 40960 0 0 0 1 buffer arena-36 36864 0 0 0 1 buffer arena-32 32768 773 4433 22183752 5 buffer arena-28 28672 0 0 0 1 buffer arena-24 24576 0 0 0 1 buffer arena-20 20480 0 0 0 1 buffer arena-16 16384 1 6 59871521 1 buffer arena-12 12288 0 0 0 1 buffer arena-81 8192 0 0 0 1 buffer arena-40 4096 0 0 0 2 kstack_cache 16384 1422 0 5855585 1 db:0:pho> acttrace Tracing command idle pid 11 tid 100003 td 0xfffffe00e4c53300 (CPU 0) cpustop_handler() at cpustop_handler+0x28/frame 0xffffffff81ff6ca0 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xffffffff81ff6cb0 trap() at trap+0x56/frame 0xffffffff81ff6dc0 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff81ff6dc0 --- trap 0x13, rip = 0xffffffff81098b06, rsp = 0xfffffe0037790a50, rbp = 0xfffffe0037790a50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0037790a50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe0037790aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe0037790ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe0037790ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe0037790bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037790bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037790bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xfffffe00e4c54800 (CPU 6) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce61e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce61e10 trap() at trap+0x56/frame 0xfffffe003ce61f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce61f20 --- trap 0x13, rip = 0xffffffff81098b06, rsp = 0xfffffe00377aea50, rbp = 0xfffffe00377aea50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00377aea50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00377aeaa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00377aeac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00377aeae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00377aebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377aebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377aebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xfffffe00e4c5a100 (CPU 7) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce6ee00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce6ee10 trap() at trap+0x56/frame 0xfffffe003ce6ef20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce6ef20 --- trap 0x13, rip = 0xffffffff81098b06, rsp = 0xfffffe00377a9a50, rbp = 0xfffffe00377a9a50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00377a9a50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00377a9aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00377a9ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00377a9ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00377a9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377a9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377a9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffffe00e4c59a00 (CPU 8) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce7be00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce7be10 trap() at trap+0x56/frame 0xfffffe003ce7bf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce7bf20 --- trap 0x13, rip = 0xffffffff81098b06, rsp = 0xfffffe00377b8a50, rbp = 0xfffffe00377b8a50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00377b8a50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00377b8aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00377b8ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00377b8ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00377b8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377b8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377b8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100014 td 0xfffffe00e4c58500 (CPU 11) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003cea2e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003cea2e10 trap() at trap+0x56/frame 0xfffffe003cea2f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003cea2f20 --- trap 0x13, rip = 0xffffffff81098b06, rsp = 0xfffffe00377bda50, rbp = 0xfffffe00377bda50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00377bda50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00377bdaa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00377bdac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00377bdae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00377bdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377bdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377bdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 962695 td 0xfffffe0146807500 (CPU 1) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce20e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce20e10 trap() at trap+0x56/frame 0xfffffe003ce20f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce20f20 --- trap 0x13, rip = 0xffffffff80bed7ec, rsp = 0xfffffe01511c45a0, rbp = 0xfffffe01511c45e0 --- __mtx_lock_flags() at __mtx_lock_flags+0x4c/frame 0xfffffe01511c45e0 vfs_ref_from_vp() at vfs_ref_from_vp+0x12d/frame 0xfffffe01511c4610 vop_stdgetwritemount() at vop_stdgetwritemount+0x12/frame 0xfffffe01511c4630 VOP_GETWRITEMOUNT_APV() at VOP_GETWRITEMOUNT_APV+0x48/frame 0xfffffe01511c4650 vn_start_write() at vn_start_write+0x40/frame 0xfffffe01511c46a0 vnlru_free_locked() at vnlru_free_locked+0x32e/frame 0xfffffe01511c4720 vn_alloc_hard() at vn_alloc_hard+0xd5/frame 0xfffffe01511c4750 getnewvnode() at getnewvnode+0x234/frame 0xfffffe01511c4790 ffs_vgetf() at ffs_vgetf+0x20b/frame 0xfffffe01511c4900 handle_workitem_remove() at handle_workitem_remove+0x8a/frame 0xfffffe01511c49e0 process_worklist_item() at process_worklist_item+0x578/frame 0xfffffe01511c4ad0 softdep_process_worklist() at softdep_process_worklist+0x105/frame 0xfffffe01511c4b30 softdep_flush() at softdep_flush+0x156/frame 0xfffffe01511c4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe01511c4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01511c4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command umount pid 65959 tid 100283 td 0xfffffe0143dd2a00 (CPU 3) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce3ae00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce3ae10 trap() at trap+0x56/frame 0xfffffe003ce3af20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce3af20 --- trap 0x13, rip = 0xffffffff80c5e9c2, rsp = 0xfffffe01428dc170, rbp = 0xfffffe01428dc180 --- lock_delay() at lock_delay+0x22/frame 0xfffffe01428dc180 __mtx_lock_sleep() at __mtx_lock_sleep+0xe8/frame 0xfffffe01428dc210 __mtx_lock_flags() at __mtx_lock_flags+0xe5/frame 0xfffffe01428dc260 vnlru_free_locked() at vnlru_free_locked+0xae/frame 0xfffffe01428dc2e0 vn_alloc_hard() at vn_alloc_hard+0xd5/frame 0xfffffe01428dc310 getnewvnode() at getnewvnode+0x234/frame 0xfffffe01428dc350 ffs_vgetf() at ffs_vgetf+0x20b/frame 0xfffffe01428dc4c0 handle_workitem_remove() at handle_workitem_remove+0x8a/frame 0xfffffe01428dc5a0 process_worklist_item() at process_worklist_item+0x578/frame 0xfffffe01428dc690 softdep_process_worklist() at softdep_process_worklist+0x105/frame 0xfffffe01428dc6f0 softdep_flushworklist() at softdep_flushworklist+0x4e/frame 0xfffffe01428dc740 ffs_sync() at ffs_sync+0x3ea/frame 0xfffffe01428dc830 vfs_write_suspend() at vfs_write_suspend+0x165/frame 0xfffffe01428dc880 vfs_write_suspend_umnt() at vfs_write_suspend_umnt+0x35/frame 0xfffffe01428dc8b0 ffs_unmount() at ffs_unmount+0xf5/frame 0xfffffe01428dc940 dounmount() at dounmount+0x43b/frame 0xfffffe01428dc9b0 kern_unmount() at kern_unmount+0x2fc/frame 0xfffffe01428dcac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01428dcbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01428dcbf0 --- syscall (22, FreeBSD ELF64, sys_unmount), rip = 0x8011a758a, rsp = 0x7fffffffd228, rbp = 0x7fffffffd760 --- Tracing command sed pid 65961 tid 101475 td 0xfffffe01478c1800 (CPU 9) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce88e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce88e10 trap() at trap+0x56/frame 0xfffffe003ce88f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce88f20 --- trap 0x13, rip = 0xffffffff80c5e9c0, rsp = 0xfffffe0150f8a050, rbp = 0xfffffe0150f8a060 --- lock_delay() at lock_delay+0x20/frame 0xfffffe0150f8a060 __mtx_lock_sleep() at __mtx_lock_sleep+0xe8/frame 0xfffffe0150f8a0f0 __mtx_lock_flags() at __mtx_lock_flags+0xe5/frame 0xfffffe0150f8a140 vnlru_free_locked() at vnlru_free_locked+0xae/frame 0xfffffe0150f8a1c0 vn_alloc_hard() at vn_alloc_hard+0xd5/frame 0xfffffe0150f8a1f0 getnewvnode() at getnewvnode+0x234/frame 0xfffffe0150f8a230 ffs_vgetf() at ffs_vgetf+0x20b/frame 0xfffffe0150f8a3a0 ffs_vget() at ffs_vget+0x33/frame 0xfffffe0150f8a3e0 ufs_lookup_ino() at ufs_lookup_ino+0xf08/frame 0xfffffe0150f8a550 VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0x56/frame 0xfffffe0150f8a570 vfs_cache_lookup() at vfs_cache_lookup+0xa6/frame 0xfffffe0150f8a5c0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x56/frame 0xfffffe0150f8a5e0 cache_fplookup_noentry() at cache_fplookup_noentry+0x237/frame 0xfffffe0150f8a650 cache_fplookup() at cache_fplookup+0x615/frame 0xfffffe0150f8a740 namei() at namei+0x129/frame 0xfffffe0150f8a800 vn_open_cred() at vn_open_cred+0x459/frame 0xfffffe0150f8a970 kern_openat() at kern_openat+0x26f/frame 0xfffffe0150f8aac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0150f8abf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0150f8abf0 --- syscall (5, FreeBSD ELF64, sys_open), rip = 0x8011af4ca, rsp = 0x7fffffffdfe8, rbp = 0x7fffffffe520 --- Tracing command sed pid 65962 tid 101081 td 0xfffffe014636aa00 (CPU 10) kdb_enter() at kdb_enter+0x37/frame 0xfffffe01507d7e80 hardclock() at hardclock+0x1d0/frame 0xfffffe01507d7ec0 handleevents() at handleevents+0xb4/frame 0xfffffe01507d7f00 timercb() at timercb+0x196/frame 0xfffffe01507d7f50 lapic_handle_timer() at lapic_handle_timer+0x9b/frame 0xfffffe01507d7f80 Xtimerint() at Xtimerint+0xb1/frame 0xfffffe01507d7f80 --- interrupt, rip = 0xffffffff80c5e9c2, rsp = 0xfffffe01507d8050, rbp = 0xfffffe01507d8060 --- lock_delay() at lock_delay+0x22/frame 0xfffffe01507d8060 __mtx_lock_sleep() at __mtx_lock_sleep+0xe8/frame 0xfffffe01507d80f0 __mtx_lock_flags() at __mtx_lock_flags+0xe5/frame 0xfffffe01507d8140 vnlru_free_locked() at vnlru_free_locked+0xae/frame 0xfffffe01507d81c0 vn_alloc_hard() at vn_alloc_hard+0xd5/frame 0xfffffe01507d81f0 getnewvnode() at getnewvnode+0x234/frame 0xfffffe01507d8230 ffs_vgetf() at ffs_vgetf+0x20b/frame 0xfffffe01507d83a0 ffs_vget() at ffs_vget+0x33/frame 0xfffffe01507d83e0 ufs_lookup_ino() at ufs_lookup_ino+0xf08/frame 0xfffffe01507d8550 VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0x56/frame 0xfffffe01507d8570 vfs_cache_lookup() at vfs_cache_lookup+0xa6/frame 0xfffffe01507d85c0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x56/frame 0xfffffe01507d85e0 cache_fplookup_noentry() at cache_fplookup_noentry+0x237/frame 0xfffffe01507d8650 cache_fplookup() at cache_fplookup+0x615/frame 0xfffffe01507d8740 namei() at namei+0x129/frame 0xfffffe01507d8800 vn_open_cred() at vn_open_cred+0x459/frame 0xfffffe01507d8970 kern_openat() at kern_openat+0x26f/frame 0xfffffe01507d8ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01507d8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01507d8bf0 --- syscall (5, FreeBSD ELF64, sys_open), rip = 0x8011af4ca, rsp = 0x7fffffffdfb8, rbp = 0x7fffffffe4f0 --- Tracing command dhclient pid 65964 tid 100385 td 0xfffffe0143ee5700 (CPU 5) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce54e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce54e10 trap() at trap+0x56/frame 0xfffffe003ce54f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce54f20 --- trap 0x13, rip = 0xffffffff80c80290, rsp = 0xfffffe01444d2d50, rbp = 0xfffffe01444d2f00 --- witness_checkorder() at witness_checkorder+0x2b0/frame 0xfffffe01444d2f00 __mtx_lock_flags() at __mtx_lock_flags+0x94/frame 0xfffffe01444d2f50 vfs_ref_from_vp() at vfs_ref_from_vp+0x12d/frame 0xfffffe01444d2f80 vop_stdgetwritemount() at vop_stdgetwritemount+0x12/frame 0xfffffe01444d2fa0 VOP_GETWRITEMOUNT_APV() at VOP_GETWRITEMOUNT_APV+0x48/frame 0xfffffe01444d2fc0 vn_start_write() at vn_start_write+0x40/frame 0xfffffe01444d3010 vnlru_free_locked() at vnlru_free_locked+0x32e/frame 0xfffffe01444d3090 vn_alloc_hard() at vn_alloc_hard+0xd5/frame 0xfffffe01444d30c0 getnewvnode() at getnewvnode+0x234/frame 0xfffffe01444d3100 ffs_vgetf() at ffs_vgetf+0x20b/frame 0xfffffe01444d3270 ffs_vget() at ffs_vget+0x33/frame 0xfffffe01444d32b0 ufs_lookup_ino() at ufs_lookup_ino+0xf08/frame 0xfffffe01444d3420 VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0x56/frame 0xfffffe01444d3440 vfs_cache_lookup() at vfs_cache_lookup+0xa6/frame 0xfffffe01444d3490 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x56/frame 0xfffffe01444d34b0 cache_fplookup_noentry() at cache_fplookup_noentry+0x237/frame 0xfffffe01444d3520 cache_fplookup() at cache_fplookup+0x615/frame 0xfffffe01444d3610 namei() at namei+0x129/frame 0xfffffe01444d36d0 kern_execve() at kern_execve+0x261/frame 0xfffffe01444d3a30 sys_execve() at sys_execve+0x5a/frame 0xfffffe01444d3ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01444d3bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01444d3bf0 --- syscall (59, FreeBSD ELF64, sys_execve), rip = 0x8011e38ca, rsp = 0x7fffffffd9f8, rbp = 0x7fffffffda80 --- Tracing command ntpd pid 2705 tid 100236 td 0xfffffe010e151c00 (CPU 2) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce2de00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce2de10 trap() at trap+0x56/frame 0xfffffe003ce2df20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce2df20 --- trap 0x13, rip = 0xffffffff80c83486, rsp = 0xfffffe01415a9fe8, rbp = 0xfffffe01415a9ff0 --- _isitmyx() at _isitmyx+0x6/frame 0xfffffe01415a9ff0 witness_checkorder() at witness_checkorder+0x607/frame 0xfffffe01415aa1b0 __mtx_lock_flags() at __mtx_lock_flags+0x94/frame 0xfffffe01415aa200 vnlru_free_locked() at vnlru_free_locked+0xae/frame 0xfffffe01415aa280 vn_alloc_hard() at vn_alloc_hard+0xd5/frame 0xfffffe01415aa2b0 getnewvnode() at getnewvnode+0x234/frame 0xfffffe01415aa2f0 ffs_vgetf() at ffs_vgetf+0x20b/frame 0xfffffe01415aa460 ffs_vget() at ffs_vget+0x33/frame 0xfffffe01415aa4a0 ufs_lookup_ino() at ufs_lookup_ino+0xf08/frame 0xfffffe01415aa610 VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0x56/frame 0xfffffe01415aa630 vfs_cache_lookup() at vfs_cache_lookup+0xa6/frame 0xfffffe01415aa680 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x56/frame 0xfffffe01415aa6a0 lookup() at lookup+0x4a1/frame 0xfffffe01415aa740 namei() at namei+0x57e/frame 0xfffffe01415aa800 vn_open_cred() at vn_open_cred+0x10f/frame 0xfffffe01415aa970 kern_openat() at kern_openat+0x26f/frame 0xfffffe01415aaac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01415aabf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01415aabf0 --- syscall (5, FreeBSD ELF64, sys_open), rip = 0x8015d94ca, rsp = 0x7fffffffe4f8, rbp = 0x7fffffffe530 --- Tracing command cron pid 2746 tid 100259 td 0xfffffe01427f6100 (CPU 4) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce47e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce47e10 trap() at trap+0x56/frame 0xfffffe003ce47f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce47f20 --- trap 0x13, rip = 0xffffffff80c5e9c2, rsp = 0xfffffe014288f0c0, rbp = 0xfffffe014288f0d0 --- lock_delay() at lock_delay+0x22/frame 0xfffffe014288f0d0 __mtx_lock_sleep() at __mtx_lock_sleep+0xe8/frame 0xfffffe014288f160 __mtx_lock_flags() at __mtx_lock_flags+0xe5/frame 0xfffffe014288f1b0 vnlru_free_locked() at vnlru_free_locked+0xae/frame 0xfffffe014288f230 vn_alloc_hard() at vn_alloc_hard+0xd5/frame 0xfffffe014288f260 getnewvnode() at getnewvnode+0x234/frame 0xfffffe014288f2a0 ffs_vgetf() at ffs_vgetf+0x20b/frame 0xfffffe014288f410 ffs_vget() at ffs_vget+0x33/frame 0xfffffe014288f450 ufs_lookup_ino() at ufs_lookup_ino+0xf08/frame 0xfffffe014288f5c0 VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0x56/frame 0xfffffe014288f5e0 vfs_cache_lookup() at vfs_cache_lookup+0xa6/frame 0xfffffe014288f630 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x56/frame 0xfffffe014288f650 cache_fplookup_noentry() at cache_fplookup_noentry+0x237/frame 0xfffffe014288f6c0 cache_fplookup() at cache_fplookup+0x615/frame 0xfffffe014288f7b0 namei() at namei+0x129/frame 0xfffffe014288f870 kern_statat() at kern_statat+0x11d/frame 0xfffffe014288f9c0 sys_fstatat() at sys_fstatat+0x2f/frame 0xfffffe014288fac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014288fbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014288fbf0 --- syscall (552, FreeBSD ELF64, sys_fstatat), rip = 0x8011d828a, rsp = 0x7fffffffdfc8, rbp = 0x7fffffffe070 --- db:0:pho> ps pid ppid pgrp uid state wmesg wchan cmd 65966 65965 2624 0 D ufs 0xfffffe01612b70d8 ls 65965 2624 2624 0 S wait 0xfffff807dfff9528 sh 65964 2347 2347 0 R CPU 5 dhclient 65963 3438 3387 0 S piperd 0xfffff807024dd5d0 sh 65962 3438 3387 0 R CPU 10 sed 65961 3438 3387 0 R CPU 9 sed 65959 22390 2816 0 RL+ CPU 3 umount 22406 0 0 0 DL mdwait 0xfffff8062e1ad000 [md10] 22390 22388 2816 0 S+ wait 0xfffff8001b21a528 sh 22389 2816 2816 0 S+ piperd 0xfffff80200f09000 tee 22388 2816 2816 0 SW+ wait 0xfffff804a3cd7528 sh 3441 3387 3387 0 S piperd 0xfffff806f0a745d0 sh 3439 3387 3387 0 S piperd 0xfffff806f0a748b8 awk 3438 3387 3387 0 S wait 0xfffff80024433000 sh 3387 1 3387 0 SWs wait 0xfffff806f0452000 sh 2816 2809 2816 0 SW+ wait 0xfffff800245b2000 sh 2809 2807 2809 0 S+ select 0xfffff80022d9a040 bash 2807 2806 2807 0 SW+ pause 0xfffff80024550b00 csh 2806 2801 2806 2006 SW+ wait 0xfffff803a4ab4528 su 2801 2800 2801 2006 SWs+ wait 0xfffff806f02ef000 bash 2800 2797 2797 2006 S select 0xfffff80024251340 sshd 2797 2737 2797 0 Ss select 0xfffff800242512c0 sshd 2769 1 2769 0 Ss+ ttyin 0xfffff80003cb08b0 getty 2768 1 2768 0 Ss+ ttyin 0xfffff80003cb0cb0 getty 2767 1 2767 0 Ss+ ttyin 0xfffff800190038b0 getty 2766 1 2766 0 Ss+ ttyin 0xfffff80019003cb0 getty 2765 1 2765 0 Ss+ ttyin 0xfffff8001900a0b0 getty 2764 1 2764 0 Ss+ ttyin 0xfffff8001900a4b0 getty 2763 1 2763 0 Ss+ ttyin 0xfffff8001900a8b0 getty 2762 1 2762 0 Ss+ ttyin 0xfffff8001900acb0 getty 2761 1 2761 0 Ss+ ttyin 0xfffff8001902c0b0 getty 2760 1 2760 0 Ss+ ttyin 0xfffff8001902c4b0 getty 2749 1 2749 0 Ss select 0xfffff80006fb83c0 inetd 2746 1 2746 0 Rs CPU 4 cron 2737 1 2737 0 Ss select 0xfffff800245a5dc0 sshd 2708 1 2708 0 Ss select 0xfffff803a4e298c0 powerd 2705 1 2705 123 Rs CPU 2 ntpd 2666 1 2666 0 Ss rpcsvc 0xfffff806f0281e60 NLM: master 2663 1 2663 0 Ss select 0xfffff800242513c0 rpc.statd 2641 1 2641 0 Ss select 0xfffff8001b26c1c0 rpcbind 2624 1 2624 0 Ss wait 0xfffff803a4a6ca50 watchdogd 2609 1 2609 0 Ds biord 0xfffffe003a5886b0 syslogd 2428 1 2428 0 Ss select 0xfffff80024554d40 devd 2423 1 2423 65 Ss piperd 0xfffff80024121160 dhclient 2347 1 2347 0 Ss wait 0xfffff80024392a50 dhclient 2344 1 2344 0 Ss select 0xfffff80003bb1940 dhclient 34 0 0 0 DL syncer 0xffffffff81fa8e38 [syncer] 33 0 0 0 DL vlruwt 0xfffff80004187000 [vnlru] 32 0 0 0 RL (threaded) [bufdaemon] 100210 D qsleep 0xffffffff81fa7f00 [bufdaemon] 100211 D - 0xffffffff81a0ad80 [bufspacedaemon-0] 100212 D - 0xffffffff81a13040 [bufspacedaemon-1] 100213 D - 0xffffffff81a1b300 [bufspacedaemon-2] 100214 D - 0xffffffff81a235c0 [bufspacedaemon-3] 100215 D - 0xffffffff81a2b880 [bufspacedaemon-4] 100216 D - 0xffffffff81a33b40 [bufspacedaemon-5] 100217 D - 0xffffffff81a3be00 [bufspacedaemon-6] 100230 D sdflush 0xfffff80003cab8e8 [/ worker] 100237 D sdflush 0xfffff80019143ce8 [/tmp worker] 962695 Run CPU 1 [/mnt worker] 31 0 0 0 DL psleep 0xffffffff81fcdc48 [vmdaemon] 30 0 0 0 DL (threaded) [pagedaemon] 100208 D psleep 0xffffffff81fc20b8 [dom0] 100218 D launds 0xffffffff81fc20c4 [laundry: dom0] 100219 D umarcl 0xffffffff80f64a30 [uma] 29 0 0 0 DL idle 0xfffff80004261528 [enc_daemon1] 28 0 0 0 DL idle 0xfffff80004261a50 [enc_daemon0] 27 0 0 0 DL - 0xffffffff81cedd28 [rand_harvestq] 26 0 0 0 DL - 0xffffffff81fa555c [soaiod4] 25 0 0 0 DL - 0xffffffff81fa555c [soaiod3] 24 0 0 0 DL - 0xffffffff81fa555c [soaiod2] 23 0 0 0 DL - 0xffffffff81fa555c [soaiod1] 22 0 0 0 DL (threaded) [usb] 100137 D - 0xfffffe010a186440 [usbus0] 100138 D - 0xfffffe010a186498 [usbus0] 100139 D - 0xfffffe010a1864f0 [usbus0] 100140 D - 0xfffffe010a186548 [usbus0] 100141 D - 0xfffffe010a1865a0 [usbus0] 100143 D - 0xfffffe00e4d47cf0 [usbus1] 100144 D - 0xfffffe00e4d47d48 [usbus1] 100145 D - 0xfffffe00e4d47da0 [usbus1] 100146 D - 0xfffffe00e4d47df8 [usbus1] 100147 D - 0xfffffe00e4d47e50 [usbus1] 100148 D - 0xfffffe010a421cf0 [usbus2] 100149 D - 0xfffffe010a421d48 [usbus2] 100150 D - 0xfffffe010a421da0 [usbus2] 100151 D - 0xfffffe010a421df8 [usbus2] 100152 D - 0xfffffe010a421e50 [usbus2] 21 0 0 0 RL (threaded) [cam] 100131 D - 0xffffffff81bc5640 [doneq0] 100132 RunQ [doneq1] 100133 D - 0xffffffff81bc5740 [doneq2] 100165 D - 0xffffffff81bc5510 [scanner] 20 0 0 0 DL seqstat 0xfffff80003f38c88 [sequencer 00] 19 0 0 0 DL crypto_ 0xfffff80003db0c50 [crypto returns 11] 18 0 0 0 DL crypto_ 0xfffff80003db0bf0 [crypto returns 10] 17 0 0 0 DL crypto_ 0xfffff80003db0b90 [crypto returns 9] 16 0 0 0 DL crypto_ 0xfffff80003db0b30 [crypto returns 8] 15 0 0 0 DL crypto_ 0xfffff80003db0ad0 [crypto returns 7] 14 0 0 0 DL crypto_ 0xfffff80003db0a70 [crypto returns 6] 9 0 0 0 DL crypto_ 0xfffff80003db0a10 [crypto returns 5] 8 0 0 0 DL crypto_ 0xfffff80003db09b0 [crypto returns 4] 7 0 0 0 DL crypto_ 0xfffff80003db0950 [crypto returns 3] 6 0 0 0 DL crypto_ 0xfffff80003db08f0 [crypto returns 2] 5 0 0 0 DL crypto_ 0xfffff80003db0890 [crypto returns 1] 4 0 0 0 DL crypto_ 0xfffff80003db0830 [crypto returns 0] 3 0 0 0 DL crypto_ 0xffffffff81fbf5a0 [crypto] 13 0 0 0 DL (threaded) [geom] 100102 D - 0xffffffff81e992a0 [g_event] 100103 D - 0xffffffff81e992a8 [g_up] 100104 D - 0xffffffff81e992b0 [g_down] 2 0 0 0 DL (threaded) [KTLS] 100075 D - 0xfffff80003c93800 [thr_0] 100076 D - 0xfffff80003c93880 [thr_1] 100077 D - 0xfffff80003c93900 [thr_2] 100078 D - 0xfffff80003c93980 [thr_3] 100079 D - 0xfffff80003c93a00 [thr_4] 100080 D - 0xfffff80003c93a80 [thr_5] 100081 D - 0xfffff80003c93b00 [thr_6] 100082 D - 0xfffff80003c93b80 [thr_7] 100083 D - 0xfffff80003c93c00 [thr_8] 100084 D - 0xfffff80003c93c80 [thr_9] 100085 D - 0xfffff80003c93d00 [thr_10] 100086 D - 0xfffff80003c93d80 [thr_11] 12 0 0 0 WL (threaded) [intr] 100041 I [swi5: fast taskq] 100044 I [swi6: task queue] 100047 I [swi6: Giant taskq] 100087 I [swi4: clock (0)] 100088 I [swi4: clock (1)] 100089 I [swi4: clock (2)] 100090 I [swi4: clock (3)] 100091 I [swi4: clock (4)] 100092 I [swi4: clock (5)] 100093 I [swi4: clock (6)] 100094 I [swi4: clock (7)] 100095 I [swi4: clock (8)] 100096 I [swi4: clock (9)] 100097 I [swi4: clock (10)] 100098 I [swi4: clock (11)] 100099 I [swi1: netisr 0] 100100 I [swi3: vm] 100134 I [swi4: apei] 100135 I [irq56: ahci0] 100136 I [irq19: xhci0] 100142 I [irq18: ehci0 ehci1+] 100153 I [irq71: ahci1] 100154 I [swi0: uart uart] 100172 I [irq73: mlx5_core0] 100173 I [irq74: mlx5_core0] 100174 I [irq72: mlx5_core0] 100175 I [irq75: mlx5_core0] 100176 I [irq76: mlx5_core0] 100177 I [irq77: mlx5_core0] 100178 I [irq78: mlx5_core0] 100179 I [irq79: mlx5_core0] 100180 I [irq80: mlx5_core0] 100181 I [irq81: mlx5_core0] 100182 I [irq82: mlx5_core0] 100183 I [irq83: mlx5_core0] 100184 I [irq84: mlx5_core0] 100185 I [irq85: mlx5_core0] 100186 I [irq86: mlx5_core0] 100192 I [irq88: mlx5_core1] 100193 I [irq89: mlx5_core1] 100194 I [irq87: mlx5_core1] 100195 I [irq90: mlx5_core1] 100196 I [irq91: mlx5_core1] 100197 I [irq92: mlx5_core1] 100198 I [irq93: mlx5_core1] 100199 I [irq94: mlx5_core1] 100200 I [irq95: mlx5_core1] 100201 I [irq96: mlx5_core1] 100202 I [irq97: mlx5_core1] 100203 I [irq98: mlx5_core1] 100204 I [irq99: mlx5_core1] 100205 I [irq100: mlx5_core1] 100206 I [irq101: mlx5_core1] 100266 I [irq102: ioat0] 100267 I [irq103: ioat1] 100268 I [irq104: ioat2] 100269 I [irq105: ioat3] 100270 I [irq106: ioat4] 100271 I [irq107: ioat5] 100272 I [irq108: ioat6] 100273 I [irq109: ioat7] 11 0 0 0 RL (threaded) [idle] 100003 Run CPU 0 [idle: cpu0] 100004 CanRun [idle: cpu1] 100005 CanRun [idle: cpu2] 100006 CanRun [idle: cpu3] 100007 CanRun [idle: cpu4] 100008 CanRun [idle: cpu5] 100009 Run CPU 6 [idle: cpu6] 100010 Run CPU 7 [idle: cpu7] 100011 Run CPU 8 [idle: cpu8] 100012 CanRun [idle: cpu9] 100013 CanRun [idle: cpu10] 100014 Run CPU 11 [idle: cpu11] 1 0 1 0 SLs wait 0xfffff80003a6a528 [init] 10 0 0 0 DL audit_w 0xffffffff81fbfac0 [audit] 0 0 0 0 DLs (threaded) [kernel] 100000 D swapin 0xffffffff81e99830 [swapper] 100015 D - 0xfffff80003a4ac00 [softirq_0] 100016 D - 0xfffff80003a4ab00 [softirq_1] 100017 D - 0xfffff80003a4aa00 [softirq_2] 100018 D - 0xfffff80003a4a900 [softirq_3] 100019 D - 0xfffff80003a4a800 [softirq_4] 100020 D - 0xfffff80003a4a700 [softirq_5] 100021 D - 0xfffff80003a4a600 [softirq_6] 100022 D - 0xfffff80003a4a500 [softirq_7] 100023 D - 0xfffff80003a4a400 [softirq_8] 100024 D - 0xfffff80003a4a300 [softirq_9] 100025 D - 0xfffff80003a4a200 [softirq_10] 100026 D - 0xfffff80003a4a100 [softirq_11] 100027 D - 0xfffff80003a4a000 [if_io_tqg_0] 100028 D - 0xfffff80003a47e00 [if_io_tqg_1] 100029 D - 0xfffff80003a47d00 [if_io_tqg_2] 100030 D - 0xfffff80003a47c00 [if_io_tqg_3] 100031 D - 0xfffff80003a47b00 [if_io_tqg_4] 100032 D - 0xfffff80003a47a00 [if_io_tqg_5] 100033 D - 0xfffff80003a47900 [if_io_tqg_6] 100034 D - 0xfffff80003a47800 [if_io_tqg_7] 100035 D - 0xfffff80003a47700 [if_io_tqg_8] 100036 D - 0xfffff80003a47600 [if_io_tqg_9] 100037 D - 0xfffff80003a47500 [if_io_tqg_10] 100038 D - 0xfffff80003a47400 [if_io_tqg_11] 100039 D - 0xfffff80003a47300 [if_config_tqg_0] 100040 D - 0xfffff80003a47200 [linuxkpi_irq_wq] 100042 D - 0xfffff80003a3ee00 [kqueue_ctx taskq] 100043 D - 0xfffff80003a3ed00 [inm_free taskq] 100045 D - 0xfffff80003a3ea00 [aiod_kick taskq] 100046 D - 0xfffff80003a3e900 [in6m_free taskq] 100048 D - 0xfffff80003a3e600 [thread taskq] 100049 D - 0xfffff80003a3e500 [linuxkpi_short_wq_0] 100050 D - 0xfffff80003a3e500 [linuxkpi_short_wq_1] 100051 D - 0xfffff80003a3e500 [linuxkpi_short_wq_2] 100052 D - 0xfffff80003a3e500 [linuxkpi_short_wq_3] 100053 D - 0xfffff80003a3e500 [linuxkpi_short_wq_4] 100054 D - 0xfffff80003a3e500 [linuxkpi_short_wq_5] 100055 D - 0xfffff80003a3e500 [linuxkpi_short_wq_6] 100056 D - 0xfffff80003a3e500 [linuxkpi_short_wq_7] 100057 D - 0xfffff80003a3e500 [linuxkpi_short_wq_8] 100058 D - 0xfffff80003a3e500 [linuxkpi_short_wq_9] 100059 D - 0xfffff80003a3e500 [linuxkpi_short_wq_1] 100060 D - 0xfffff80003a3e500 [linuxkpi_short_wq_1] 100061 D - 0xfffff80003a3e500 [linuxkpi_short_wq_1] 100062 D - 0xfffff80003a3e400 [linuxkpi_long_wq_0] 100063 D - 0xfffff80003a3e400 [linuxkpi_long_wq_1] 100064 D - 0xfffff80003a3e400 [linuxkpi_long_wq_2] 100065 D - 0xfffff80003a3e400 [linuxkpi_long_wq_3] 100066 D - 0xfffff80003a3e400 [linuxkpi_long_wq_4] 100067 D - 0xfffff80003a3e400 [linuxkpi_long_wq_5] 100068 D - 0xfffff80003a3e400 [linuxkpi_long_wq_6] 100069 D - 0xfffff80003a3e400 [linuxkpi_long_wq_7] 100070 D - 0xfffff80003a3e400 [linuxkpi_long_wq_8] 100071 D - 0xfffff80003a3e400 [linuxkpi_long_wq_9] 100072 D - 0xfffff80003a3e400 [linuxkpi_long_wq_10] 100073 D - 0xfffff80003a3e400 [linuxkpi_long_wq_11] 100074 D - 0xfffff80003a3e400 [linuxkpi_long_wq_12] 100101 D - 0xfffff80003c3fe00 [firmware taskq] 100105 D - 0xfffff80003c3fd00 [crypto_0] 100106 D - 0xfffff80003c3fd00 [crypto_1] 100107 D - 0xfffff80003c3fd00 [crypto_2] 100108 D - 0xfffff80003c3fd00 [crypto_3] 100109 D - 0xfffff80003c3fd00 [crypto_4] 100110 D - 0xfffff80003c3fd00 [crypto_5] 100111 D - 0xfffff80003c3fd00 [crypto_6] 100112 D - 0xfffff80003c3fd00 [crypto_7] 100113 D - 0xfffff80003c3fd00 [crypto_8] 100114 D - 0xfffff80003c3fd00 [crypto_9] 100115 D - 0xfffff80003c3fd00 [crypto_10] 100116 D - 0xfffff80003c3fd00 [crypto_11] 100155 D - 0xfffff80003cafd00 [mca taskq] 100160 D - 0xfffff80019016e00 [acpi_task_0] 100161 D - 0xfffff80019016e00 [acpi_task_1] 100162 D - 0xfffff80019016e00 [acpi_task_2] 100164 D - 0xfffff80003c3fc00 [CAM taskq] 100168 D - 0xfffff80019016c00 [mlx5_core0-rec] 100169 D - 0xfffff80019016b00 [mlx5_core0-wdg] 100170 D - 0xfffff80019016a00 [mlx5_core0-cmd] 100171 D - 0xfffff80019016900 [mlx5_page_allocator] 100187 D - 0xfffff8001e7c5800 [mlx5_esw_wq] 100188 D - 0xfffff8001e7c5700 [mlx5_core1-rec] 100189 D - 0xfffff8001e7c5600 [mlx5_core1-wdg] 100190 D - 0xfffff8001e7c5500 [mlx5_core1-cmd] 100191 D - 0xfffff8001e7c5400 [mlx5_page_allocator] 100207 D - 0xfffff80003a3e200 [mlx5_esw_wq] db:0:pho> allt Tracing command kernel pid 0 tid 100000 td 0xffffffff81e99d60 sched_switch() at sched_switch+0x5b3/frame 0xffffffff82346ab0 mi_switch() at mi_switch+0x159/frame 0xffffffff82346ad0 sleepq_switch() at sleepq_switch+0x109/frame 0xffffffff82346b10 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xffffffff82346b50 _sleep() at _sleep+0x29d/frame 0xffffffff82346c00 swapper() at swapper+0x75/frame 0xffffffff82346c60 mi_startup() at mi_startup+0x2ed/frame 0xffffffff82346cb0 btext() at btext+0x2c Tracing command kernel pid 0 tid 100015 td 0xfffffe00e4c57e00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00377ccab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00377ccad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00377ccb10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00377ccb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00377ccbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377ccbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377ccbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xfffffe00e4c57700 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00377c7ab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00377c7ad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00377c7b10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00377c7b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00377c7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377c7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377c7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100017 td 0xfffffe00e4c57000 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00377d6ab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00377d6ad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00377d6b10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00377d6b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00377d6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377d6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377d6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100018 td 0xfffffe00e4c5a800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00377d1ab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00377d1ad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00377d1b10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00377d1b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00377d1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377d1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377d1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xfffffe00e4cdb100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00377e0ab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00377e0ad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00377e0b10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00377e0b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00377e0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377e0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377e0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xfffffe00e4cdaa00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00377dbab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00377dbad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00377dbb10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00377dbb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00377dbbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377dbbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377dbbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xfffffe00e4cda300 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00377eaab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00377eaad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00377eab10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00377eab80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00377eabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377eabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377eabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xfffffe00e4cd9c00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00377e5ab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00377e5ad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00377e5b10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00377e5b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00377e5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377e5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377e5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xfffffe00e4cd9500 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00377f4ab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00377f4ad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00377f4b10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00377f4b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00377f4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377f4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377f4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xfffffe00e4cd8e00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00377efab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00377efad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00377efb10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00377efb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00377efbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377efbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377efbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xfffffe00e4cd8700 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00377feab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00377fead0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00377feb10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00377feb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00377febb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377febf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377febf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100026 td 0xfffffe00e4cd8000 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00377f9ab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00377f9ad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00377f9b10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00377f9b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00377f9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377f9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377f9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100027 td 0xfffffe00e4cdb800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4835ab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4835ad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4835b10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e4835b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00e4835bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4835bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4835bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100028 td 0xfffffe00e4ce1100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4830ab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4830ad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4830b10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e4830b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00e4830bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4830bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4830bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100029 td 0xfffffe00e4ce0a00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e483fab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e483fad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e483fb10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e483fb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00e483fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e483fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e483fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100030 td 0xfffffe00e4ce0300 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e483aab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e483aad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e483ab10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e483ab80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00e483abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e483abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e483abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100031 td 0xfffffe00e4cdfc00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4849ab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4849ad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4849b10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e4849b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00e4849bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4849bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4849bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100032 td 0xfffffe00e4cdf500 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4844ab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4844ad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4844b10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e4844b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00e4844bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4844bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4844bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100033 td 0xfffffe00e4cdee00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4853ab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4853ad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4853b10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e4853b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00e4853bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4853bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4853bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100034 td 0xfffffe00e4cde700 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e484eab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e484ead0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e484eb10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e484eb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00e484ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e484ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e484ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100035 td 0xfffffe00e4cde000 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e485dab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e485dad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e485db10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e485db80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00e485dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e485dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e485dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100036 td 0xfffffe00e4ce1800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4858ab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4858ad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4858b10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e4858b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00e4858bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4858bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4858bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100037 td 0xfffffe00e4ce5100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4867ab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4867ad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4867b10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e4867b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00e4867bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4867bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4867bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100038 td 0xfffffe00e4ce4a00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4862ab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4862ad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4862b10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e4862b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00e4862bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4862bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4862bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100039 td 0xfffffe00e4ce4300 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4871ab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4871ad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4871b10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e4871b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00e4871bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4871bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4871bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100040 td 0xfffffe00e4ce3c00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e486cab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e486cad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e486cb10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e486cb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xef/frame 0xfffffe00e486cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e486cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e486cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100042 td 0xfffffe00e4ce2e00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4876a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4876a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4876ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4876b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e4876bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4876bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4876bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100043 td 0xfffffe00e4ce2700 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4885a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4885a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4885ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4885b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e4885bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4885bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4885bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100045 td 0xfffffe00e4ce5800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e488fa70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e488fa90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e488fad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e488fb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e488fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e488fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e488fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100046 td 0xfffffe00e4ceb100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e488aa70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e488aa90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e488aad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e488ab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e488abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e488abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e488abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100048 td 0xfffffe00e4cea300 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4894a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4894a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4894ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4894b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e4894bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4894bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4894bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100049 td 0xfffffe00e4ce9c00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e48a3a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e48a3a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e48a3ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e48a3b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e48a3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48a3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48a3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100050 td 0xfffffe00e4ce9500 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e489ea70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e489ea90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e489ead0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e489eb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e489ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e489ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e489ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100051 td 0xfffffe00e4ce8e00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e48ada70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e48ada90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e48adad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e48adb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e48adbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48adbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48adbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100052 td 0xfffffe00e4ce8700 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e48a8a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e48a8a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e48a8ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e48a8b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e48a8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48a8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48a8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100053 td 0xfffffe00e4ce8000 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e48b7a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e48b7a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e48b7ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e48b7b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e48b7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48b7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48b7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100054 td 0xfffffe00e4ceb800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e48b2a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e48b2a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e48b2ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e48b2b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e48b2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48b2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48b2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100055 td 0xfffffe00e4cef100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e48c1a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e48c1a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e48c1ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e48c1b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e48c1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48c1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48c1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100056 td 0xfffffe00e4ceea00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e48bca70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e48bca90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e48bcad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e48bcb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e48bcbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48bcbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48bcbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100057 td 0xfffffe00e4cee300 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e48cba70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e48cba90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e48cbad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e48cbb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e48cbbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48cbbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48cbbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100058 td 0xfffffe00e4cedc00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e48c6a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e48c6a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e48c6ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e48c6b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e48c6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48c6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48c6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100059 td 0xfffffe00e4ced500 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e48d5a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e48d5a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e48d5ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e48d5b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e48d5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48d5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48d5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100060 td 0xfffffe00e4cece00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e48d0a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e48d0a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e48d0ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e48d0b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e48d0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48d0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48d0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100061 td 0xfffffe00e4cec700 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e48dfa70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e48dfa90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e48dfad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e48dfb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e48dfbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48dfbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48dfbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100062 td 0xfffffe00e4cec000 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e48daa70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e48daa90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e48daad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e48dab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e48dabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48dabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48dabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100063 td 0xfffffe00e4cef800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e48e9a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e48e9a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e48e9ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e48e9b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e48e9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48e9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48e9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100064 td 0xfffffe00e4cf3100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e48e4a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e48e4a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e48e4ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e48e4b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e48e4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48e4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48e4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100065 td 0xfffffe00e4cf2a00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e48f3a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e48f3a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e48f3ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e48f3b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e48f3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48f3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48f3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100066 td 0xfffffe00e4cf2300 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e48eea70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e48eea90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e48eead0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e48eeb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e48eebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48eebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48eebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100067 td 0xfffffe00e4cf1c00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e48fda70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e48fda90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e48fdad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e48fdb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e48fdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48fdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48fdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100068 td 0xfffffe00e4cf1500 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e48f8a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e48f8a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e48f8ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e48f8b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e48f8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e48f8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48f8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100069 td 0xfffffe00e4cf0e00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4907a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4907a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4907ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4907b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e4907bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4907bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4907bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100070 td 0xfffffe00e4cf0700 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4902a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4902a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4902ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4902b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e4902bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4902bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4902bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100071 td 0xfffffe00e4cf0000 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4911a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4911a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4911ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4911b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e4911bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4911bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4911bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100072 td 0xfffffe00e4cf3800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e490ca70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e490ca90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e490cad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e490cb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e490cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e490cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e490cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100073 td 0xfffffe00e4cf7100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e491ba70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e491ba90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e491bad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e491bb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e491bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e491bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e491bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100074 td 0xfffffe00e4cf6a00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4916a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4916a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4916ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4916b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e4916bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4916bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4916bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100101 td 0xfffffe003ce75000 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e49e3a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e49e3a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e49e3ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e49e3b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e49e3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49e3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49e3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100105 td 0xfffffe003ce6a300 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e49f7a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e49f7a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e49f7ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e49f7b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e49f7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49f7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49f7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100106 td 0xfffffe003ce69c00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e49f2a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e49f2a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e49f2ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e49f2b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e49f2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49f2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49f2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100107 td 0xfffffe003ce69500 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e04a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e04a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e04ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4e04b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e4e04bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e04bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e04bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100108 td 0xfffffe003ce68e00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e49fca70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e49fca90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e49fcad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e49fcb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e49fcbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49fcbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49fcbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100109 td 0xfffffe003ce68700 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e0ea70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e0ea90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e0ead0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4e0eb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e4e0ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e0ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e0ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100110 td 0xfffffe003ce68000 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e09a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e09a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e09ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4e09b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e4e09bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e09bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e09bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100111 td 0xfffffe003ce6b800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e18a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e18a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e18ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4e18b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e4e18bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e18bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e18bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100112 td 0xfffffe003ce51100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e13a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e13a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e13ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4e13b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e4e13bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e13bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e13bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100113 td 0xfffffe003ce50a00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e22a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e22a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e22ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4e22b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e4e22bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e22bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e22bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100114 td 0xfffffe003ce50300 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e1da70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e1da90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e1dad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4e1db80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e4e1dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e1dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e1dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100115 td 0xfffffe003ce4fc00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e2ca70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e2ca90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e2cad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4e2cb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e4e2cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e2cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e2cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100116 td 0xfffffe003ce4f500 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e27a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e27a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e27ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4e27b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e4e27bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e27bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e27bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100155 td 0xfffffe010a6b6700 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4efbab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4efbad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4efbb10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e4efbb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xef/frame 0xfffffe00e4efbbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4efbbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4efbbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100160 td 0xfffffe010d651800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4f36ab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4f36ad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4f36b10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e4f36b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xef/frame 0xfffffe00e4f36bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f36bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f36bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100161 td 0xfffffe010d9d5100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4f31ab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4f31ad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4f31b10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e4f31b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xef/frame 0xfffffe00e4f31bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f31bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f31bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100162 td 0xfffffe010d9d4a00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4f40ab0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4f40ad0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4f40b10 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00e4f40b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xef/frame 0xfffffe00e4f40bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f40bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f40bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100164 td 0xfffffe010d9d3c00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4f4aa70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4f4aa90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4f4aad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4f4ab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e4f4abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f4abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f4abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100168 td 0xfffffe010d9d2e00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4f5ea70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4f5ea90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4f5ead0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4f5eb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e4f5ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f5ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f5ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100169 td 0xfffffe010d9d2700 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4f59a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4f59a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4f59ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4f59b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e4f59bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f59bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f59bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100170 td 0xfffffe010d9d2000 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4f68a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4f68a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4f68ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4f68b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e4f68bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f68bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f68bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100171 td 0xfffffe010db89800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4f63a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4f63a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4f63ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4f63b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e4f63bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f63bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f63bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100187 td 0xfffffe010e048700 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4fb3a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4fb3a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4fb3ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4fb3b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e4fb3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4fb3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4fb3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100188 td 0xfffffe010e048000 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4fc2a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4fc2a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4fc2ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4fc2b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e4fc2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4fc2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4fc2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100189 td 0xfffffe010e04b800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4fbda70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4fbda90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4fbdad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4fbdb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e4fbdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4fbdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4fbdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100190 td 0xfffffe010e0f7100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4fcca70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4fcca90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4fccad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4fccb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e4fccbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4fccbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4fccbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100191 td 0xfffffe010e0f6a00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4fc7a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4fc7a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4fc7ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4fc7b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00e4fc7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4fc7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4fc7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100207 td 0xfffffe010ddab000 sched_switch() at sched_switch+0x5b3/frame 0xfffffe010e627a70 mi_switch() at mi_switch+0x159/frame 0xfffffe010e627a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe010e627ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe010e627b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe010e627bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe010e627bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010e627bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command init pid 1 tid 100002 td 0xfffffe00e4c53a00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00377816c0 mi_switch() at mi_switch+0x159/frame 0xfffffe00377816e0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe0037781720 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0037781770 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0037781780 _sleep() at _sleep+0x2ac/frame 0xfffffe0037781830 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00377818c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0037781ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0037781bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0037781bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x28d0ba, rsp = 0x7fffffffe718, rbp = 0x7fffffffe780 --- Tracing command KTLS pid 2 tid 100075 td 0xfffffe00e4cf7800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e493d980 mi_switch() at mi_switch+0x159/frame 0xfffffe00e493d9a0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e493d9e0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e493da90 ktls_work_thread() at ktls_work_thread+0xd8/frame 0xfffffe00e493dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e493dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e493dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100076 td 0xfffffe003ce92100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4938980 mi_switch() at mi_switch+0x159/frame 0xfffffe00e49389a0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e49389e0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4938a90 ktls_work_thread() at ktls_work_thread+0xd8/frame 0xfffffe00e4938bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4938bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4938bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100077 td 0xfffffe003ce91a00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4947980 mi_switch() at mi_switch+0x159/frame 0xfffffe00e49479a0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e49479e0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4947a90 ktls_work_thread() at ktls_work_thread+0xd8/frame 0xfffffe00e4947bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4947bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4947bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100078 td 0xfffffe003ce91300 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4942980 mi_switch() at mi_switch+0x159/frame 0xfffffe00e49429a0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e49429e0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4942a90 ktls_work_thread() at ktls_work_thread+0xd8/frame 0xfffffe00e4942bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4942bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4942bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100079 td 0xfffffe003ce90c00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4951980 mi_switch() at mi_switch+0x159/frame 0xfffffe00e49519a0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e49519e0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4951a90 ktls_work_thread() at ktls_work_thread+0xd8/frame 0xfffffe00e4951bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4951bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4951bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100080 td 0xfffffe003ce90500 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e494c980 mi_switch() at mi_switch+0x159/frame 0xfffffe00e494c9a0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e494c9e0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e494ca90 ktls_work_thread() at ktls_work_thread+0xd8/frame 0xfffffe00e494cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e494cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e494cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100081 td 0xfffffe003ce8fe00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e495b980 mi_switch() at mi_switch+0x159/frame 0xfffffe00e495b9a0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e495b9e0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e495ba90 ktls_work_thread() at ktls_work_thread+0xd8/frame 0xfffffe00e495bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e495bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e495bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100082 td 0xfffffe003ce8f700 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4956980 mi_switch() at mi_switch+0x159/frame 0xfffffe00e49569a0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e49569e0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4956a90 ktls_work_thread() at ktls_work_thread+0xd8/frame 0xfffffe00e4956bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4956bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4956bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100083 td 0xfffffe003ce8f000 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4965980 mi_switch() at mi_switch+0x159/frame 0xfffffe00e49659a0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e49659e0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4965a90 ktls_work_thread() at ktls_work_thread+0xd8/frame 0xfffffe00e4965bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4965bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4965bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100084 td 0xfffffe003ce92800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4960980 mi_switch() at mi_switch+0x159/frame 0xfffffe00e49609a0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e49609e0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4960a90 ktls_work_thread() at ktls_work_thread+0xd8/frame 0xfffffe00e4960bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4960bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4960bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100085 td 0xfffffe003ce85100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e496f980 mi_switch() at mi_switch+0x159/frame 0xfffffe00e496f9a0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e496f9e0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e496fa90 ktls_work_thread() at ktls_work_thread+0xd8/frame 0xfffffe00e496fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e496fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e496fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command KTLS pid 2 tid 100086 td 0xfffffe003ce84a00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e496a980 mi_switch() at mi_switch+0x159/frame 0xfffffe00e496a9a0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e496a9e0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e496aa90 ktls_work_thread() at ktls_work_thread+0xd8/frame 0xfffffe00e496abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e496abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e496abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 3 tid 100117 td 0xfffffe003ce4ee00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e36a60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e36a80 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e36ac0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4e36b70 crypto_proc() at crypto_proc+0x22f/frame 0xfffffe00e4e36bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e36bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e36bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 0 pid 4 tid 100118 td 0xfffffe003ce4e700 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e31a60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e31a80 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e31ac0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4e31b70 crypto_ret_proc() at crypto_ret_proc+0x222/frame 0xfffffe00e4e31bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e31bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e31bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 1 pid 5 tid 100119 td 0xfffffe003ce4e000 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e40a60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e40a80 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e40ac0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4e40b70 crypto_ret_proc() at crypto_ret_proc+0x222/frame 0xfffffe00e4e40bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e40bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e40bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 2 pid 6 tid 100120 td 0xfffffe003ce51800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e3ba60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e3ba80 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e3bac0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4e3bb70 crypto_ret_proc() at crypto_ret_proc+0x222/frame 0xfffffe00e4e3bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e3bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e3bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 3 pid 7 tid 100121 td 0xfffffe003ce44100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e4aa60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e4aa80 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e4aac0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4e4ab70 crypto_ret_proc() at crypto_ret_proc+0x222/frame 0xfffffe00e4e4abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e4abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e4abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 4 pid 8 tid 100122 td 0xfffffe003ce43a00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e45a60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e45a80 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e45ac0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4e45b70 crypto_ret_proc() at crypto_ret_proc+0x222/frame 0xfffffe00e4e45bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e45bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e45bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 5 pid 9 tid 100123 td 0xfffffe003ce43300 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e54a60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e54a80 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e54ac0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4e54b70 crypto_ret_proc() at crypto_ret_proc+0x222/frame 0xfffffe00e4e54bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e54bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e54bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command audit pid 10 tid 100001 td 0xfffffe00e4c54100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe0037786a80 mi_switch() at mi_switch+0x159/frame 0xfffffe0037786aa0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe0037786ae0 _cv_wait() at _cv_wait+0x16f/frame 0xfffffe0037786b40 audit_worker() at audit_worker+0x9f/frame 0xfffffe0037786bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037786bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037786bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100003 td 0xfffffe00e4c53300 (CPU 0) cpustop_handler() at cpustop_handler+0x28/frame 0xffffffff81ff6ca0 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xffffffff81ff6cb0 trap() at trap+0x56/frame 0xffffffff81ff6dc0 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff81ff6dc0 --- trap 0x13, rip = 0xffffffff81098b06, rsp = 0xfffffe0037790a50, rbp = 0xfffffe0037790a50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0037790a50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe0037790aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe0037790ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe0037790ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe0037790bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037790bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037790bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xfffffe00e4c52c00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe003778bac0 mi_switch() at mi_switch+0x159/frame 0xfffffe003778bae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe003778bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003778bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003778bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffffe00e4c52500 sched_switch() at sched_switch+0x5b3/frame 0xfffffe003779aac0 mi_switch() at mi_switch+0x159/frame 0xfffffe003779aae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe003779abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003779abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003779abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xfffffe00e4c51e00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe0037795ac0 mi_switch() at mi_switch+0x159/frame 0xfffffe0037795ae0 sched_idletd() at sched_idletd+0x9e/frame 0xfffffe0037795bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037795bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037795bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100007 td 0xfffffe00e4c51700 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00377a4ac0 mi_switch() at mi_switch+0x159/frame 0xfffffe00377a4ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00377a4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377a4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377a4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xfffffe00e4c51000 sched_switch() at sched_switch+0x5b3/frame 0xfffffe003779fac0 mi_switch() at mi_switch+0x159/frame 0xfffffe003779fae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe003779fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003779fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003779fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xfffffe00e4c54800 (CPU 6) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce61e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce61e10 trap() at trap+0x56/frame 0xfffffe003ce61f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce61f20 --- trap 0x13, rip = 0xffffffff81098b06, rsp = 0xfffffe00377aea50, rbp = 0xfffffe00377aea50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00377aea50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00377aeaa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00377aeac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00377aeae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00377aebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377aebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377aebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xfffffe00e4c5a100 (CPU 7) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce6ee00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce6ee10 trap() at trap+0x56/frame 0xfffffe003ce6ef20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce6ef20 --- trap 0x13, rip = 0xffffffff81098b06, rsp = 0xfffffe00377a9a50, rbp = 0xfffffe00377a9a50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00377a9a50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00377a9aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00377a9ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00377a9ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00377a9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377a9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377a9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffffe00e4c59a00 (CPU 8) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce7be00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce7be10 trap() at trap+0x56/frame 0xfffffe003ce7bf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce7bf20 --- trap 0x13, rip = 0xffffffff81098b06, rsp = 0xfffffe00377b8a50, rbp = 0xfffffe00377b8a50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00377b8a50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00377b8aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00377b8ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00377b8ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00377b8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377b8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377b8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100012 td 0xfffffe00e4c59300 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00377b3ac0 mi_switch() at mi_switch+0x159/frame 0xfffffe00377b3ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00377b3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377b3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377b3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xfffffe00e4c58c00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00377c2ac0 mi_switch() at mi_switch+0x159/frame 0xfffffe00377c2ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00377c2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377c2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377c2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100014 td 0xfffffe00e4c58500 (CPU 11) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003cea2e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003cea2e10 trap() at trap+0x56/frame 0xfffffe003cea2f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003cea2f20 --- trap 0x13, rip = 0xffffffff81098b06, rsp = 0xfffffe00377bda50, rbp = 0xfffffe00377bda50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00377bda50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00377bdaa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00377bdac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00377bdae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00377bdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377bdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377bdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100041 td 0xfffffe00e4ce3500 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100044 td 0xfffffe00e4ce2000 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4880b00 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4880b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4880bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4880bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4880bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100047 td 0xfffffe00e4ceaa00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4899b00 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4899b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4899bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4899bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4899bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100087 td 0xfffffe003ce84300 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4979b00 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4979b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4979bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4979bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4979bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100088 td 0xfffffe003ce83c00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4974b00 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4974b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4974bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4974bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4974bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100089 td 0xfffffe003ce83500 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4983b00 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4983b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4983bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4983bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4983bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100090 td 0xfffffe003ce82e00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e497eb00 mi_switch() at mi_switch+0x159/frame 0xfffffe00e497eb20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e497ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e497ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e497ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100091 td 0xfffffe003ce82700 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e498db00 mi_switch() at mi_switch+0x159/frame 0xfffffe00e498db20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e498dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e498dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e498dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100092 td 0xfffffe003ce82000 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4988b00 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4988b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4988bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4988bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4988bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100093 td 0xfffffe003ce85800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4997b00 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4997b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4997bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4997bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4997bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100094 td 0xfffffe003ce78100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4992b00 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4992b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4992bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4992bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4992bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100095 td 0xfffffe003ce77a00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e49a1b00 mi_switch() at mi_switch+0x159/frame 0xfffffe00e49a1b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e49a1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49a1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49a1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100096 td 0xfffffe003ce77300 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e499cb00 mi_switch() at mi_switch+0x159/frame 0xfffffe00e499cb20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e499cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e499cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e499cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100097 td 0xfffffe003ce76c00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e49abb00 mi_switch() at mi_switch+0x159/frame 0xfffffe00e49abb20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e49abbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49abbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49abbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100098 td 0xfffffe003ce76500 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e49a6b00 mi_switch() at mi_switch+0x159/frame 0xfffffe00e49a6b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e49a6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49a6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49a6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100099 td 0xfffffe003ce75e00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e49b5b00 mi_switch() at mi_switch+0x159/frame 0xfffffe00e49b5b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e49b5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49b5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49b5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100100 td 0xfffffe003ce75700 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100134 td 0xfffffe003ce1d800 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100135 td 0xfffffe00e4d62100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e8bb00 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e8bb20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4e8bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e8bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e8bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100136 td 0xfffffe00e4d61a00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e9ab00 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e9ab20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4e9abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e9abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e9abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100142 td 0xfffffe00e4d5f000 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4eb8b00 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4eb8b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4eb8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4eb8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4eb8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100153 td 0xfffffe010a6b7500 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100154 td 0xfffffe010a6b6e00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4f00b00 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4f00b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4f00bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f00bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f00bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100172 td 0xfffffe010df5b100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4f72b00 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4f72b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4f72bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f72bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f72bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100173 td 0xfffffe010df5aa00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4f6db00 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4f6db20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4f6dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f6dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f6dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100174 td 0xfffffe010df5a300 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100175 td 0xfffffe010df59c00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100176 td 0xfffffe010df59500 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100177 td 0xfffffe010df58e00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100178 td 0xfffffe010df58700 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100179 td 0xfffffe010df58000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100180 td 0xfffffe010df5b800 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100181 td 0xfffffe010e04b100 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100182 td 0xfffffe010e04aa00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100183 td 0xfffffe010e04a300 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100184 td 0xfffffe010e049c00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100185 td 0xfffffe010e049500 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100186 td 0xfffffe010e048e00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100192 td 0xfffffe010e0f6300 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4fd6b00 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4fd6b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4fd6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4fd6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4fd6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100193 td 0xfffffe00e4cf6300 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4fe0b00 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4fe0b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00e4fe0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4fe0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4fe0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100194 td 0xfffffe00e4cf5c00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100195 td 0xfffffe00e4cf5500 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100196 td 0xfffffe00e4cf4e00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100197 td 0xfffffe00e4cf4700 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100198 td 0xfffffe00e4cf4000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100199 td 0xfffffe010e0f7800 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100200 td 0xfffffe010ddae100 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100201 td 0xfffffe010ddada00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100202 td 0xfffffe010ddad300 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100203 td 0xfffffe010ddacc00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100204 td 0xfffffe010ddac500 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100205 td 0xfffffe010ddabe00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100206 td 0xfffffe010ddab700 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100266 td 0xfffffe010db88300 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100267 td 0xfffffe010db87c00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100268 td 0xfffffe010db87500 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100269 td 0xfffffe010db86e00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100270 td 0xfffffe010db86700 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100271 td 0xfffffe010db86000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100272 td 0xfffffe01427f6800 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100273 td 0xfffffe0143dd3100 fork_trampoline() at fork_trampoline Tracing command geom pid 13 tid 100102 td 0xfffffe003ce78800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e49dea70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e49dea90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e49dead0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e49deb80 g_run_events() at g_run_events+0x54/frame 0xfffffe00e49debb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49debf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49debf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100103 td 0xfffffe003ce6b100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e49eda60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e49eda80 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e49edac0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e49edb70 g_io_schedule_up() at g_io_schedule_up+0x3c/frame 0xfffffe00e49edba0 g_up_procbody() at g_up_procbody+0x5c/frame 0xfffffe00e49edbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49edbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49edbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100104 td 0xfffffe003ce6aa00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e49e8a60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e49e8a80 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e49e8ac0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e49e8b70 g_io_schedule_down() at g_io_schedule_down+0x4c/frame 0xfffffe00e49e8ba0 g_down_procbody() at g_down_procbody+0x5c/frame 0xfffffe00e49e8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e49e8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e49e8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 6 pid 14 tid 100124 td 0xfffffe003ce42c00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e4fa60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e4fa80 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e4fac0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4e4fb70 crypto_ret_proc() at crypto_ret_proc+0x222/frame 0xfffffe00e4e4fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e4fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e4fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 7 pid 15 tid 100125 td 0xfffffe003ce42500 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e5ea60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e5ea80 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e5eac0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4e5eb70 crypto_ret_proc() at crypto_ret_proc+0x222/frame 0xfffffe00e4e5ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e5ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e5ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 8 pid 16 tid 100126 td 0xfffffe003ce41e00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e59a60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e59a80 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e59ac0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4e59b70 crypto_ret_proc() at crypto_ret_proc+0x222/frame 0xfffffe00e4e59bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e59bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e59bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 9 pid 17 tid 100127 td 0xfffffe003ce41700 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e68a60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e68a80 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e68ac0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4e68b70 crypto_ret_proc() at crypto_ret_proc+0x222/frame 0xfffffe00e4e68bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e68bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e68bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 10 pid 18 tid 100128 td 0xfffffe003ce41000 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e63a60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e63a80 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e63ac0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4e63b70 crypto_ret_proc() at crypto_ret_proc+0x222/frame 0xfffffe00e4e63bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e63bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e63bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 11 pid 19 tid 100129 td 0xfffffe003ce44800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e72a60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e72a80 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e72ac0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4e72b70 crypto_ret_proc() at crypto_ret_proc+0x222/frame 0xfffffe00e4e72bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e72bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e72bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sequencer 00 pid 20 tid 100130 td 0xfffffe003ce1d100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e6da70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e6da90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e6dad0 _cv_wait() at _cv_wait+0x16f/frame 0xfffffe00e4e6db30 seq_eventthread() at seq_eventthread+0x91/frame 0xfffffe00e4e6dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e6dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e6dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 21 tid 100131 td 0xfffffe003ce1ca00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e7ca50 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e7ca70 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e7cab0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4e7cb60 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe00e4e7cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e7cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e7cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 21 tid 100132 td 0xfffffe003ce1c300 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e77a50 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e77a70 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e77ab0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4e77b60 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe00e4e77bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e77bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e77bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 21 tid 100133 td 0xfffffe003ce1bc00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e86a50 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e86a70 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e86ab0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4e86b60 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe00e4e86bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e86bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e86bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 21 tid 100165 td 0xfffffe010d9d3500 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4f45a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4f45a90 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4f45ad0 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4f45b80 xpt_scanner_thread() at xpt_scanner_thread+0x138/frame 0xfffffe00e4f45bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f45bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f45bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100137 td 0xfffffe00e4d61300 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e95ac0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e95ae0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e95b20 _cv_wait() at _cv_wait+0x16f/frame 0xfffffe00e4e95b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4e95bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e95bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e95bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100138 td 0xfffffe00e4d60c00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4ea4ac0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4ea4ae0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4ea4b20 _cv_wait() at _cv_wait+0x16f/frame 0xfffffe00e4ea4b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4ea4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ea4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ea4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100139 td 0xfffffe00e4d60500 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e9fac0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e9fae0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e9fb20 _cv_wait() at _cv_wait+0x16f/frame 0xfffffe00e4e9fb80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4e9fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e9fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e9fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100140 td 0xfffffe00e4d5fe00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4eaeac0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4eaeae0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4eaeb20 _cv_wait() at _cv_wait+0x16f/frame 0xfffffe00e4eaeb80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4eaebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4eaebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4eaebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100141 td 0xfffffe00e4d5f700 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4ea9ac0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4ea9ae0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4ea9b20 _cv_wait() at _cv_wait+0x16f/frame 0xfffffe00e4ea9b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4ea9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ea9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ea9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100143 td 0xfffffe00e4d62800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4eb3ac0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4eb3ae0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4eb3b20 _cv_wait() at _cv_wait+0x16f/frame 0xfffffe00e4eb3b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4eb3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4eb3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4eb3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100144 td 0xfffffe010a310100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4ec2ac0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4ec2ae0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4ec2b20 _cv_wait() at _cv_wait+0x16f/frame 0xfffffe00e4ec2b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4ec2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ec2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ec2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100145 td 0xfffffe010a30fa00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4ebdac0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4ebdae0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4ebdb20 _cv_wait() at _cv_wait+0x16f/frame 0xfffffe00e4ebdb80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4ebdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ebdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ebdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100146 td 0xfffffe010a30f300 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4eccac0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4eccae0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4eccb20 _cv_wait() at _cv_wait+0x16f/frame 0xfffffe00e4eccb80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4eccbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4eccbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4eccbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100147 td 0xfffffe010a30ec00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4ec7ac0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4ec7ae0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4ec7b20 _cv_wait() at _cv_wait+0x16f/frame 0xfffffe00e4ec7b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4ec7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ec7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ec7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100148 td 0xfffffe010a310800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4ee2ac0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4ee2ae0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4ee2b20 _cv_wait() at _cv_wait+0x16f/frame 0xfffffe00e4ee2b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4ee2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ee2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ee2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100149 td 0xfffffe010a6b9100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4eddac0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4eddae0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4eddb20 _cv_wait() at _cv_wait+0x16f/frame 0xfffffe00e4eddb80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4eddbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4eddbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4eddbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100150 td 0xfffffe010a6b8a00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4eecac0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4eecae0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4eecb20 _cv_wait() at _cv_wait+0x16f/frame 0xfffffe00e4eecb80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4eecbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4eecbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4eecbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100151 td 0xfffffe010a6b8300 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4ee7ac0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4ee7ae0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4ee7b20 _cv_wait() at _cv_wait+0x16f/frame 0xfffffe00e4ee7b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4ee7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ee7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ee7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 22 tid 100152 td 0xfffffe010a6b7c00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4ef6ac0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4ef6ae0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4ef6b20 _cv_wait() at _cv_wait+0x16f/frame 0xfffffe00e4ef6b80 usb_process() at usb_process+0xb3/frame 0xfffffe00e4ef6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4ef6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4ef6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod1 pid 23 tid 100156 td 0xfffffe010a6b9800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4f16a20 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4f16a40 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4f16a80 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e4f16ac0 _sleep() at _sleep+0x29d/frame 0xfffffe00e4f16b70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe00e4f16bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f16bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f16bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod2 pid 24 tid 100157 td 0xfffffe010d651100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4f11a20 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4f11a40 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4f11a80 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e4f11ac0 _sleep() at _sleep+0x29d/frame 0xfffffe00e4f11b70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe00e4f11bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f11bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f11bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod3 pid 25 tid 100158 td 0xfffffe010d650a00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4f20a20 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4f20a40 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4f20a80 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e4f20ac0 _sleep() at _sleep+0x29d/frame 0xfffffe00e4f20b70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe00e4f20bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f20bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f20bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod4 pid 26 tid 100159 td 0xfffffe010d650300 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4f1ba20 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4f1ba40 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4f1ba80 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e4f1bac0 _sleep() at _sleep+0x29d/frame 0xfffffe00e4f1bb70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe00e4f1bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f1bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f1bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command rand_harvestq pid 27 tid 100163 td 0xfffffe010d9d4300 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4f3b9d0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4f3b9f0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4f3ba30 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e4f3ba70 _sleep() at _sleep+0x29d/frame 0xfffffe00e4f3bb20 random_kthread() at random_kthread+0x67/frame 0xfffffe00e4f3bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f3bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f3bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon0 pid 28 tid 100166 td 0xfffffe010d9d5800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4f54a30 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4f54a50 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4f54a90 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4f54b40 enc_daemon() at enc_daemon+0xe4/frame 0xfffffe00e4f54bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f54bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f54bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon1 pid 29 tid 100167 td 0xfffffe010db89100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4f4fa30 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4f4fa50 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4f4fa90 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4f4fb40 enc_daemon() at enc_daemon+0xe4/frame 0xfffffe00e4f4fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4f4fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4f4fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 30 tid 100208 td 0xfffffe003ce1b500 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4e81760 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4e81780 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4e817c0 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e4e81800 _sleep() at _sleep+0x29d/frame 0xfffffe00e4e818b0 vm_pageout_worker() at vm_pageout_worker+0x28c/frame 0xfffffe00e4e81b70 vm_pageout() at vm_pageout+0x21e/frame 0xfffffe00e4e81bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4e81bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4e81bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 30 tid 100218 td 0xfffffe010e0f5c00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe00e4fd1820 mi_switch() at mi_switch+0x159/frame 0xfffffe00e4fd1840 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe00e4fd1880 _sleep() at _sleep+0x2b6/frame 0xfffffe00e4fd1930 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0xc3c/frame 0xfffffe00e4fd1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e4fd1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e4fd1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 30 tid 100219 td 0xfffffe010e0f5500 sched_switch() at sched_switch+0x5b3/frame 0xfffffe01121aaa30 mi_switch() at mi_switch+0x159/frame 0xfffffe01121aaa50 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe01121aaa90 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe01121aaad0 _sleep() at _sleep+0x29d/frame 0xfffffe01121aab80 uma_reclaim_worker() at uma_reclaim_worker+0xb8/frame 0xfffffe01121aabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe01121aabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01121aabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command vmdaemon pid 31 tid 100209 td 0xfffffe003ce1ae00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe010e63da30 mi_switch() at mi_switch+0x159/frame 0xfffffe010e63da50 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe010e63da90 _sleep() at _sleep+0x2b6/frame 0xfffffe010e63db40 vm_daemon() at vm_daemon+0x82/frame 0xfffffe010e63dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe010e63dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010e63dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100210 td 0xfffffe003ce1a700 sched_switch() at sched_switch+0x5b3/frame 0xfffffe010e638a10 mi_switch() at mi_switch+0x159/frame 0xfffffe010e638a30 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe010e638a70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe010e638ab0 _sleep() at _sleep+0x29d/frame 0xfffffe010e638b60 buf_daemon() at buf_daemon+0x121/frame 0xfffffe010e638bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe010e638bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010e638bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100211 td 0xfffffe010d64fc00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe010ea4ba00 mi_switch() at mi_switch+0x159/frame 0xfffffe010ea4ba20 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe010ea4ba60 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe010ea4baa0 _sleep() at _sleep+0x29d/frame 0xfffffe010ea4bb50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe010ea4bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe010ea4bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010ea4bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100212 td 0xfffffe010d64f500 sched_switch() at sched_switch+0x5b3/frame 0xfffffe010e743a00 mi_switch() at mi_switch+0x159/frame 0xfffffe010e743a20 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe010e743a60 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe010e743aa0 _sleep() at _sleep+0x29d/frame 0xfffffe010e743b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe010e743bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe010e743bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010e743bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100213 td 0xfffffe010d64ee00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe010f35ea00 mi_switch() at mi_switch+0x159/frame 0xfffffe010f35ea20 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe010f35ea60 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe010f35eaa0 _sleep() at _sleep+0x29d/frame 0xfffffe010f35eb50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe010f35ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe010f35ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010f35ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100214 td 0xfffffe010d64e700 sched_switch() at sched_switch+0x5b3/frame 0xfffffe010f258a00 mi_switch() at mi_switch+0x159/frame 0xfffffe010f258a20 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe010f258a60 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe010f258aa0 _sleep() at _sleep+0x29d/frame 0xfffffe010f258b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe010f258bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe010f258bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010f258bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100215 td 0xfffffe010d64e000 sched_switch() at sched_switch+0x5b3/frame 0xfffffe010fa6fa00 mi_switch() at mi_switch+0x159/frame 0xfffffe010fa6fa20 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe010fa6fa60 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe010fa6faa0 _sleep() at _sleep+0x29d/frame 0xfffffe010fa6fb50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe010fa6fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe010fa6fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010fa6fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100216 td 0xfffffe010ddae800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe010f969a00 mi_switch() at mi_switch+0x159/frame 0xfffffe010f969a20 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe010f969a60 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe010f969aa0 _sleep() at _sleep+0x29d/frame 0xfffffe010f969b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe010f969bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe010f969bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe010f969bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100217 td 0xfffffe010e153100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe0110685a00 mi_switch() at mi_switch+0x159/frame 0xfffffe0110685a20 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe0110685a60 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0110685aa0 _sleep() at _sleep+0x29d/frame 0xfffffe0110685b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe0110685bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0110685bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0110685bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100230 td 0xfffffe010e5e4100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe01415d29e0 mi_switch() at mi_switch+0x159/frame 0xfffffe01415d2a00 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe01415d2a40 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe01415d2a80 _sleep() at _sleep+0x29d/frame 0xfffffe01415d2b30 softdep_flush() at softdep_flush+0x282/frame 0xfffffe01415d2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe01415d2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01415d2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100237 td 0xfffffe014272a100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe01428159e0 mi_switch() at mi_switch+0x159/frame 0xfffffe0142815a00 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe0142815a40 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0142815a80 _sleep() at _sleep+0x29d/frame 0xfffffe0142815b30 softdep_flush() at softdep_flush+0x282/frame 0xfffffe0142815bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0142815bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0142815bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 962695 td 0xfffffe0146807500 (CPU 1) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce20e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce20e10 trap() at trap+0x56/frame 0xfffffe003ce20f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce20f20 --- trap 0x13, rip = 0xffffffff80bed7ec, rsp = 0xfffffe01511c45a0, rbp = 0xfffffe01511c45e0 --- __mtx_lock_flags() at __mtx_lock_flags+0x4c/frame 0xfffffe01511c45e0 vfs_ref_from_vp() at vfs_ref_from_vp+0x12d/frame 0xfffffe01511c4610 vop_stdgetwritemount() at vop_stdgetwritemount+0x12/frame 0xfffffe01511c4630 VOP_GETWRITEMOUNT_APV() at VOP_GETWRITEMOUNT_APV+0x48/frame 0xfffffe01511c4650 vn_start_write() at vn_start_write+0x40/frame 0xfffffe01511c46a0 vnlru_free_locked() at vnlru_free_locked+0x32e/frame 0xfffffe01511c4720 vn_alloc_hard() at vn_alloc_hard+0xd5/frame 0xfffffe01511c4750 getnewvnode() at getnewvnode+0x234/frame 0xfffffe01511c4790 ffs_vgetf() at ffs_vgetf+0x20b/frame 0xfffffe01511c4900 handle_workitem_remove() at handle_workitem_remove+0x8a/frame 0xfffffe01511c49e0 process_worklist_item() at process_worklist_item+0x578/frame 0xfffffe01511c4ad0 softdep_process_worklist() at softdep_process_worklist+0x105/frame 0xfffffe01511c4b30 softdep_flush() at softdep_flush+0x156/frame 0xfffffe01511c4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe01511c4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01511c4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command vnlru pid 33 tid 100220 td 0xfffffe003ce1a000 sched_switch() at sched_switch+0x5b3/frame 0xfffffe01415a59c0 mi_switch() at mi_switch+0x159/frame 0xfffffe01415a59e0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe01415a5a20 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe01415a5a60 _sleep() at _sleep+0x29d/frame 0xfffffe01415a5b10 vnlru_proc() at vnlru_proc+0xaac/frame 0xfffffe01415a5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe01415a5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01415a5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command syncer pid 34 tid 100221 td 0xfffffe010e153800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe01415a0a10 mi_switch() at mi_switch+0x159/frame 0xfffffe01415a0a30 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe01415a0a70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe01415a0ab0 _cv_timedwait_sbt() at _cv_timedwait_sbt+0x190/frame 0xfffffe01415a0b20 sched_sync() at sched_sync+0x67c/frame 0xfffffe01415a0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe01415a0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01415a0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command umount pid 65959 tid 100283 td 0xfffffe0143dd2a00 (CPU 3) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce3ae00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce3ae10 trap() at trap+0x56/frame 0xfffffe003ce3af20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce3af20 --- trap 0x13, rip = 0xffffffff80c5e9c2, rsp = 0xfffffe01428dc170, rbp = 0xfffffe01428dc180 --- lock_delay() at lock_delay+0x22/frame 0xfffffe01428dc180 __mtx_lock_sleep() at __mtx_lock_sleep+0xe8/frame 0xfffffe01428dc210 __mtx_lock_flags() at __mtx_lock_flags+0xe5/frame 0xfffffe01428dc260 vnlru_free_locked() at vnlru_free_locked+0xae/frame 0xfffffe01428dc2e0 vn_alloc_hard() at vn_alloc_hard+0xd5/frame 0xfffffe01428dc310 getnewvnode() at getnewvnode+0x234/frame 0xfffffe01428dc350 ffs_vgetf() at ffs_vgetf+0x20b/frame 0xfffffe01428dc4c0 handle_workitem_remove() at handle_workitem_remove+0x8a/frame 0xfffffe01428dc5a0 process_worklist_item() at process_worklist_item+0x578/frame 0xfffffe01428dc690 softdep_process_worklist() at softdep_process_worklist+0x105/frame 0xfffffe01428dc6f0 softdep_flushworklist() at softdep_flushworklist+0x4e/frame 0xfffffe01428dc740 ffs_sync() at ffs_sync+0x3ea/frame 0xfffffe01428dc830 vfs_write_suspend() at vfs_write_suspend+0x165/frame 0xfffffe01428dc880 vfs_write_suspend_umnt() at vfs_write_suspend_umnt+0x35/frame 0xfffffe01428dc8b0 ffs_unmount() at ffs_unmount+0xf5/frame 0xfffffe01428dc940 dounmount() at dounmount+0x43b/frame 0xfffffe01428dc9b0 kern_unmount() at kern_unmount+0x2fc/frame 0xfffffe01428dcac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01428dcbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01428dcbf0 --- syscall (22, FreeBSD ELF64, sys_unmount), rip = 0x8011a758a, rsp = 0x7fffffffd228, rbp = 0x7fffffffd760 --- Tracing command sed pid 65961 tid 101475 td 0xfffffe01478c1800 (CPU 9) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce88e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce88e10 trap() at trap+0x56/frame 0xfffffe003ce88f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce88f20 --- trap 0x13, rip = 0xffffffff80c5e9c0, rsp = 0xfffffe0150f8a050, rbp = 0xfffffe0150f8a060 --- lock_delay() at lock_delay+0x20/frame 0xfffffe0150f8a060 __mtx_lock_sleep() at __mtx_lock_sleep+0xe8/frame 0xfffffe0150f8a0f0 __mtx_lock_flags() at __mtx_lock_flags+0xe5/frame 0xfffffe0150f8a140 vnlru_free_locked() at vnlru_free_locked+0xae/frame 0xfffffe0150f8a1c0 vn_alloc_hard() at vn_alloc_hard+0xd5/frame 0xfffffe0150f8a1f0 getnewvnode() at getnewvnode+0x234/frame 0xfffffe0150f8a230 ffs_vgetf() at ffs_vgetf+0x20b/frame 0xfffffe0150f8a3a0 ffs_vget() at ffs_vget+0x33/frame 0xfffffe0150f8a3e0 ufs_lookup_ino() at ufs_lookup_ino+0xf08/frame 0xfffffe0150f8a550 VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0x56/frame 0xfffffe0150f8a570 vfs_cache_lookup() at vfs_cache_lookup+0xa6/frame 0xfffffe0150f8a5c0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x56/frame 0xfffffe0150f8a5e0 cache_fplookup_noentry() at cache_fplookup_noentry+0x237/frame 0xfffffe0150f8a650 cache_fplookup() at cache_fplookup+0x615/frame 0xfffffe0150f8a740 namei() at namei+0x129/frame 0xfffffe0150f8a800 vn_open_cred() at vn_open_cred+0x459/frame 0xfffffe0150f8a970 kern_openat() at kern_openat+0x26f/frame 0xfffffe0150f8aac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0150f8abf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0150f8abf0 --- syscall (5, FreeBSD ELF64, sys_open), rip = 0x8011af4ca, rsp = 0x7fffffffdfe8, rbp = 0x7fffffffe520 --- Tracing command sed pid 65962 tid 101081 td 0xfffffe014636aa00 (CPU 10) kdb_enter() at kdb_enter+0x37/frame 0xfffffe01507d7e80 hardclock() at hardclock+0x1d0/frame 0xfffffe01507d7ec0 handleevents() at handleevents+0xb4/frame 0xfffffe01507d7f00 timercb() at timercb+0x196/frame 0xfffffe01507d7f50 lapic_handle_timer() at lapic_handle_timer+0x9b/frame 0xfffffe01507d7f80 Xtimerint() at Xtimerint+0xb1/frame 0xfffffe01507d7f80 --- interrupt, rip = 0xffffffff80c5e9c2, rsp = 0xfffffe01507d8050, rbp = 0xfffffe01507d8060 --- lock_delay() at lock_delay+0x22/frame 0xfffffe01507d8060 __mtx_lock_sleep() at __mtx_lock_sleep+0xe8/frame 0xfffffe01507d80f0 __mtx_lock_flags() at __mtx_lock_flags+0xe5/frame 0xfffffe01507d8140 vnlru_free_locked() at vnlru_free_locked+0xae/frame 0xfffffe01507d81c0 vn_alloc_hard() at vn_alloc_hard+0xd5/frame 0xfffffe01507d81f0 getnewvnode() at getnewvnode+0x234/frame 0xfffffe01507d8230 ffs_vgetf() at ffs_vgetf+0x20b/frame 0xfffffe01507d83a0 ffs_vget() at ffs_vget+0x33/frame 0xfffffe01507d83e0 ufs_lookup_ino() at ufs_lookup_ino+0xf08/frame 0xfffffe01507d8550 VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0x56/frame 0xfffffe01507d8570 vfs_cache_lookup() at vfs_cache_lookup+0xa6/frame 0xfffffe01507d85c0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x56/frame 0xfffffe01507d85e0 cache_fplookup_noentry() at cache_fplookup_noentry+0x237/frame 0xfffffe01507d8650 cache_fplookup() at cache_fplookup+0x615/frame 0xfffffe01507d8740 namei() at namei+0x129/frame 0xfffffe01507d8800 vn_open_cred() at vn_open_cred+0x459/frame 0xfffffe01507d8970 kern_openat() at kern_openat+0x26f/frame 0xfffffe01507d8ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01507d8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01507d8bf0 --- syscall (5, FreeBSD ELF64, sys_open), rip = 0x8011af4ca, rsp = 0x7fffffffdfb8, rbp = 0x7fffffffe4f0 --- Tracing command sh pid 65963 tid 101012 td 0xfffffe0149059500 sched_switch() at sched_switch+0x5b3/frame 0xfffffe015067f830 mi_switch() at mi_switch+0x159/frame 0xfffffe015067f850 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe015067f890 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe015067f8e0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe015067f8f0 _sleep() at _sleep+0x2ac/frame 0xfffffe015067f9a0 pipe_read() at pipe_read+0x4c6/frame 0xfffffe015067fa00 dofileread() at dofileread+0x81/frame 0xfffffe015067fa50 sys_read() at sys_read+0xbc/frame 0xfffffe015067fac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe015067fbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe015067fbf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80120958a, rsp = 0x7fffffffdb18, rbp = 0x7fffffffe040 --- Tracing command dhclient pid 65964 tid 100385 td 0xfffffe0143ee5700 (CPU 5) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce54e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce54e10 trap() at trap+0x56/frame 0xfffffe003ce54f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce54f20 --- trap 0x13, rip = 0xffffffff80c80290, rsp = 0xfffffe01444d2d50, rbp = 0xfffffe01444d2f00 --- witness_checkorder() at witness_checkorder+0x2b0/frame 0xfffffe01444d2f00 __mtx_lock_flags() at __mtx_lock_flags+0x94/frame 0xfffffe01444d2f50 vfs_ref_from_vp() at vfs_ref_from_vp+0x12d/frame 0xfffffe01444d2f80 vop_stdgetwritemount() at vop_stdgetwritemount+0x12/frame 0xfffffe01444d2fa0 VOP_GETWRITEMOUNT_APV() at VOP_GETWRITEMOUNT_APV+0x48/frame 0xfffffe01444d2fc0 vn_start_write() at vn_start_write+0x40/frame 0xfffffe01444d3010 vnlru_free_locked() at vnlru_free_locked+0x32e/frame 0xfffffe01444d3090 vn_alloc_hard() at vn_alloc_hard+0xd5/frame 0xfffffe01444d30c0 getnewvnode() at getnewvnode+0x234/frame 0xfffffe01444d3100 ffs_vgetf() at ffs_vgetf+0x20b/frame 0xfffffe01444d3270 ffs_vget() at ffs_vget+0x33/frame 0xfffffe01444d32b0 ufs_lookup_ino() at ufs_lookup_ino+0xf08/frame 0xfffffe01444d3420 VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0x56/frame 0xfffffe01444d3440 vfs_cache_lookup() at vfs_cache_lookup+0xa6/frame 0xfffffe01444d3490 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x56/frame 0xfffffe01444d34b0 cache_fplookup_noentry() at cache_fplookup_noentry+0x237/frame 0xfffffe01444d3520 cache_fplookup() at cache_fplookup+0x615/frame 0xfffffe01444d3610 namei() at namei+0x129/frame 0xfffffe01444d36d0 kern_execve() at kern_execve+0x261/frame 0xfffffe01444d3a30 sys_execve() at sys_execve+0x5a/frame 0xfffffe01444d3ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01444d3bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01444d3bf0 --- syscall (59, FreeBSD ELF64, sys_execve), rip = 0x8011e38ca, rsp = 0x7fffffffd9f8, rbp = 0x7fffffffda80 --- Tracing command sh pid 65965 tid 101340 td 0xfffffe0148147800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe0150ce76c0 mi_switch() at mi_switch+0x159/frame 0xfffffe0150ce76e0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe0150ce7720 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0150ce7770 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0150ce7780 _sleep() at _sleep+0x2ac/frame 0xfffffe0150ce7830 kern_wait6() at kern_wait6+0x482/frame 0xfffffe0150ce78c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0150ce7ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0150ce7bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0150ce7bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80120976a, rsp = 0x7fffffffe778, rbp = 0x7fffffffe820 --- Tracing command ls pid 65966 tid 101323 td 0xfffffe0148139100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe0150c923f0 mi_switch() at mi_switch+0x159/frame 0xfffffe0150c92410 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe0150c92450 sleeplk() at sleeplk+0x100/frame 0xfffffe0150c924b0 lockmgr_slock_hard() at lockmgr_slock_hard+0x3e2/frame 0xfffffe0150c92550 ffs_lock() at ffs_lock+0x9d/frame 0xfffffe0150c92590 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x40/frame 0xfffffe0150c925b0 _vn_lock() at _vn_lock+0x54/frame 0xfffffe0150c92610 vget_finish() at vget_finish+0x7a/frame 0xfffffe0150c92640 cache_lookup() at cache_lookup+0x3c4/frame 0xfffffe0150c926a0 vfs_cache_lookup() at vfs_cache_lookup+0x7b/frame 0xfffffe0150c926f0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x56/frame 0xfffffe0150c92710 lookup() at lookup+0x4a1/frame 0xfffffe0150c927b0 namei() at namei+0x57e/frame 0xfffffe0150c92870 kern_statat() at kern_statat+0x11d/frame 0xfffffe0150c929c0 sys_fstatat() at sys_fstatat+0x2f/frame 0xfffffe0150c92ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0150c92bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0150c92bf0 --- syscall (552, FreeBSD ELF64, sys_fstatat), rip = 0x80123b28a, rsp = 0x7fffffffe258, rbp = 0x7fffffffe310 --- Tracing command dhclient pid 2344 tid 100256 td 0xfffffe01427b3300 sched_switch() at sched_switch+0x5b3/frame 0xfffffe014287b720 mi_switch() at mi_switch+0x159/frame 0xfffffe014287b740 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe014287b780 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe014287b7d0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe014287b7e0 _cv_wait_sig() at _cv_wait_sig+0x16d/frame 0xfffffe014287b850 seltdwait() at seltdwait+0x88/frame 0xfffffe014287b890 kern_select() at kern_select+0x99c/frame 0xfffffe014287ba80 sys_select() at sys_select+0x56/frame 0xfffffe014287bac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014287bbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014287bbf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8011e660a, rsp = 0x7fffffffe948, rbp = 0x7fffffffea00 --- Tracing command dhclient pid 2347 tid 100246 td 0xfffffe01427b4100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe01428496c0 mi_switch() at mi_switch+0x159/frame 0xfffffe01428496e0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe0142849720 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0142849770 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0142849780 _sleep() at _sleep+0x2ac/frame 0xfffffe0142849830 kern_wait6() at kern_wait6+0x482/frame 0xfffffe01428498c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0142849ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0142849bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0142849bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8011e676a, rsp = 0x7fffffffd9f8, rbp = 0x7fffffffda80 --- Tracing command dhclient pid 2423 tid 100262 td 0xfffffe010e5e2500 sched_switch() at sched_switch+0x5b3/frame 0xfffffe014284e830 mi_switch() at mi_switch+0x159/frame 0xfffffe014284e850 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe014284e890 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe014284e8e0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe014284e8f0 _sleep() at _sleep+0x2ac/frame 0xfffffe014284e9a0 pipe_read() at pipe_read+0x4c6/frame 0xfffffe014284ea00 dofileread() at dofileread+0x81/frame 0xfffffe014284ea50 sys_read() at sys_read+0xbc/frame 0xfffffe014284eac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014284ebf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014284ebf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011e658a, rsp = 0x7fffffffc968, rbp = 0x7fffffffc990 --- Tracing command devd pid 2428 tid 100238 td 0xfffffe014272a800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe014281f6f0 mi_switch() at mi_switch+0x159/frame 0xfffffe014281f710 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe014281f750 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe014281f7a0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe014281f7e0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x190/frame 0xfffffe014281f850 seltdwait() at seltdwait+0x79/frame 0xfffffe014281f890 kern_select() at kern_select+0x99c/frame 0xfffffe014281fa80 sys_select() at sys_select+0x56/frame 0xfffffe014281fac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014281fbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014281fbf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x28c8ca, rsp = 0x7fffffffcab8, rbp = 0x7fffffffec50 --- Tracing command syslogd pid 2609 tid 100248 td 0xfffffe010a30d000 sched_switch() at sched_switch+0x5b3/frame 0xfffffe014283a220 mi_switch() at mi_switch+0x159/frame 0xfffffe014283a240 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe014283a280 _sleep() at _sleep+0x2b6/frame 0xfffffe014283a330 bufwait() at bufwait+0x84/frame 0xfffffe014283a370 breadn_flags() at breadn_flags+0x1ce/frame 0xfffffe014283a3d0 ffs_balloc_ufs2() at ffs_balloc_ufs2+0x6da/frame 0xfffffe014283a5a0 ffs_write() at ffs_write+0x262/frame 0xfffffe014283a650 VOP_WRITE_APV() at VOP_WRITE_APV+0xcf/frame 0xfffffe014283a760 vn_write() at vn_write+0x2eb/frame 0xfffffe014283a7f0 vn_io_fault_doio() at vn_io_fault_doio+0x43/frame 0xfffffe014283a850 vn_io_fault1() at vn_io_fault1+0x16c/frame 0xfffffe014283a9a0 vn_io_fault() at vn_io_fault+0x1ba/frame 0xfffffe014283aa20 dofilewrite() at dofilewrite+0x81/frame 0xfffffe014283aa70 sys_writev() at sys_writev+0x6e/frame 0xfffffe014283aac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014283abf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014283abf0 --- syscall (121, FreeBSD ELF64, sys_writev), rip = 0x8011cb7ca, rsp = 0x7fffffffd048, rbp = 0x7fffffffd180 --- Tracing command watchdogd pid 2624 tid 100284 td 0xfffffe0142799e00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe014291f6c0 mi_switch() at mi_switch+0x159/frame 0xfffffe014291f6e0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe014291f720 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe014291f770 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe014291f780 _sleep() at _sleep+0x2ac/frame 0xfffffe014291f830 kern_wait6() at kern_wait6+0x482/frame 0xfffffe014291f8c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe014291fac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014291fbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014291fbf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8011c176a, rsp = 0x7fffffffe988, rbp = 0x7fffffffea40 --- Tracing command rpcbind pid 2641 tid 100292 td 0xfffffe010e5e1700 sched_switch() at sched_switch+0x5b3/frame 0xfffffe014289e780 mi_switch() at mi_switch+0x159/frame 0xfffffe014289e7a0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe014289e7e0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe014289e830 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe014289e870 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x190/frame 0xfffffe014289e8e0 seltdwait() at seltdwait+0x79/frame 0xfffffe014289e920 kern_poll() at kern_poll+0x4b6/frame 0xfffffe014289eaa0 sys_poll() at sys_poll+0x50/frame 0xfffffe014289eac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014289ebf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014289ebf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8011bc50a, rsp = 0x7fffffffc988, rbp = 0x7fffffffebb0 --- Tracing command rpc.statd pid 2663 tid 100240 td 0xfffffe010e0f4000 sched_switch() at sched_switch+0x5b3/frame 0xfffffe01415c86f0 mi_switch() at mi_switch+0x159/frame 0xfffffe01415c8710 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe01415c8750 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01415c87a0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe01415c87e0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x190/frame 0xfffffe01415c8850 seltdwait() at seltdwait+0x79/frame 0xfffffe01415c8890 kern_select() at kern_select+0x99c/frame 0xfffffe01415c8a80 sys_select() at sys_select+0x56/frame 0xfffffe01415c8ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01415c8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01415c8bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8011b760a, rsp = 0x7fffffffea08, rbp = 0x7fffffffeb50 --- Tracing command rpc.lockd pid 2666 tid 100233 td 0xfffffe010e5e3300 sched_switch() at sched_switch+0x5b3/frame 0xfffffe01415f0650 mi_switch() at mi_switch+0x159/frame 0xfffffe01415f0670 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe01415f06b0 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01415f0700 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe01415f0740 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x190/frame 0xfffffe01415f07b0 svc_run_internal() at svc_run_internal+0xac1/frame 0xfffffe01415f08e0 svc_run() at svc_run+0x253/frame 0xfffffe01415f0940 sys_nlm_syscall() at sys_nlm_syscall+0x74d/frame 0xfffffe01415f0ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01415f0bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01415f0bf0 --- syscall (154, FreeBSD ELF64, sys_nlm_syscall), rip = 0x8011bfdea, rsp = 0x7fffffffeb68, rbp = 0x7fffffffec30 --- Tracing command ntpd pid 2705 tid 100236 td 0xfffffe010e151c00 (CPU 2) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce2de00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce2de10 trap() at trap+0x56/frame 0xfffffe003ce2df20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce2df20 --- trap 0x13, rip = 0xffffffff80c83486, rsp = 0xfffffe01415a9fe8, rbp = 0xfffffe01415a9ff0 --- _isitmyx() at _isitmyx+0x6/frame 0xfffffe01415a9ff0 witness_checkorder() at witness_checkorder+0x607/frame 0xfffffe01415aa1b0 __mtx_lock_flags() at __mtx_lock_flags+0x94/frame 0xfffffe01415aa200 vnlru_free_locked() at vnlru_free_locked+0xae/frame 0xfffffe01415aa280 vn_alloc_hard() at vn_alloc_hard+0xd5/frame 0xfffffe01415aa2b0 getnewvnode() at getnewvnode+0x234/frame 0xfffffe01415aa2f0 ffs_vgetf() at ffs_vgetf+0x20b/frame 0xfffffe01415aa460 ffs_vget() at ffs_vget+0x33/frame 0xfffffe01415aa4a0 ufs_lookup_ino() at ufs_lookup_ino+0xf08/frame 0xfffffe01415aa610 VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0x56/frame 0xfffffe01415aa630 vfs_cache_lookup() at vfs_cache_lookup+0xa6/frame 0xfffffe01415aa680 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x56/frame 0xfffffe01415aa6a0 lookup() at lookup+0x4a1/frame 0xfffffe01415aa740 namei() at namei+0x57e/frame 0xfffffe01415aa800 vn_open_cred() at vn_open_cred+0x10f/frame 0xfffffe01415aa970 kern_openat() at kern_openat+0x26f/frame 0xfffffe01415aaac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01415aabf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01415aabf0 --- syscall (5, FreeBSD ELF64, sys_open), rip = 0x8015d94ca, rsp = 0x7fffffffe4f8, rbp = 0x7fffffffe530 --- Tracing command powerd pid 2708 tid 100229 td 0xfffffe010a30e500 sched_switch() at sched_switch+0x5b3/frame 0xfffffe01415e16f0 mi_switch() at mi_switch+0x159/frame 0xfffffe01415e1710 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe01415e1750 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01415e17a0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe01415e17e0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x190/frame 0xfffffe01415e1850 seltdwait() at seltdwait+0x79/frame 0xfffffe01415e1890 kern_select() at kern_select+0x99c/frame 0xfffffe01415e1a80 sys_select() at sys_select+0x56/frame 0xfffffe01415e1ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01415e1bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01415e1bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8011c160a, rsp = 0x7fffffffeaf8, rbp = 0x7fffffffec40 --- Tracing command sshd pid 2737 tid 100274 td 0xfffffe01427b2500 sched_switch() at sched_switch+0x5b3/frame 0xfffffe01428eb720 mi_switch() at mi_switch+0x159/frame 0xfffffe01428eb740 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe01428eb780 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01428eb7d0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01428eb7e0 _cv_wait_sig() at _cv_wait_sig+0x16d/frame 0xfffffe01428eb850 seltdwait() at seltdwait+0x88/frame 0xfffffe01428eb890 kern_select() at kern_select+0x99c/frame 0xfffffe01428eba80 sys_select() at sys_select+0x56/frame 0xfffffe01428ebac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01428ebbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01428ebbf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80169960a, rsp = 0x7fffffffe5d8, rbp = 0x7fffffffe610 --- Tracing command cron pid 2746 tid 100259 td 0xfffffe01427f6100 (CPU 4) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe003ce47e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003ce47e10 trap() at trap+0x56/frame 0xfffffe003ce47f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003ce47f20 --- trap 0x13, rip = 0xffffffff80c5e9c2, rsp = 0xfffffe014288f0c0, rbp = 0xfffffe014288f0d0 --- lock_delay() at lock_delay+0x22/frame 0xfffffe014288f0d0 __mtx_lock_sleep() at __mtx_lock_sleep+0xe8/frame 0xfffffe014288f160 __mtx_lock_flags() at __mtx_lock_flags+0xe5/frame 0xfffffe014288f1b0 vnlru_free_locked() at vnlru_free_locked+0xae/frame 0xfffffe014288f230 vn_alloc_hard() at vn_alloc_hard+0xd5/frame 0xfffffe014288f260 getnewvnode() at getnewvnode+0x234/frame 0xfffffe014288f2a0 ffs_vgetf() at ffs_vgetf+0x20b/frame 0xfffffe014288f410 ffs_vget() at ffs_vget+0x33/frame 0xfffffe014288f450 ufs_lookup_ino() at ufs_lookup_ino+0xf08/frame 0xfffffe014288f5c0 VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0x56/frame 0xfffffe014288f5e0 vfs_cache_lookup() at vfs_cache_lookup+0xa6/frame 0xfffffe014288f630 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x56/frame 0xfffffe014288f650 cache_fplookup_noentry() at cache_fplookup_noentry+0x237/frame 0xfffffe014288f6c0 cache_fplookup() at cache_fplookup+0x615/frame 0xfffffe014288f7b0 namei() at namei+0x129/frame 0xfffffe014288f870 kern_statat() at kern_statat+0x11d/frame 0xfffffe014288f9c0 sys_fstatat() at sys_fstatat+0x2f/frame 0xfffffe014288fac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014288fbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014288fbf0 --- syscall (552, FreeBSD ELF64, sys_fstatat), rip = 0x8011d828a, rsp = 0x7fffffffdfc8, rbp = 0x7fffffffe070 --- Tracing command inetd pid 2749 tid 100239 td 0xfffffe0142729a00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe0142810720 mi_switch() at mi_switch+0x159/frame 0xfffffe0142810740 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe0142810780 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01428107d0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01428107e0 _cv_wait_sig() at _cv_wait_sig+0x16d/frame 0xfffffe0142810850 seltdwait() at seltdwait+0x88/frame 0xfffffe0142810890 kern_select() at kern_select+0x99c/frame 0xfffffe0142810a80 sys_select() at sys_select+0x56/frame 0xfffffe0142810ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0142810bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0142810bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8011e060a, rsp = 0x7fffffffdd88, rbp = 0x7fffffffec20 --- Tracing command getty pid 2760 tid 100222 td 0xfffffe010e152a00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe01106807b0 mi_switch() at mi_switch+0x159/frame 0xfffffe01106807d0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe0110680810 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0110680860 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0110680870 _cv_wait_sig() at _cv_wait_sig+0x16d/frame 0xfffffe01106808e0 tty_wait() at tty_wait+0x42/frame 0xfffffe0110680910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe0110680970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01106809a0 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe0110680a00 dofileread() at dofileread+0x81/frame 0xfffffe0110680a50 sys_read() at sys_read+0xbc/frame 0xfffffe0110680ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0110680bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0110680bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c558a, rsp = 0x7fffffffeb68, rbp = 0x7fffffffebb0 --- Tracing command getty pid 2761 tid 100279 td 0xfffffe0143dd3800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe01428807b0 mi_switch() at mi_switch+0x159/frame 0xfffffe01428807d0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe0142880810 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0142880860 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0142880870 _cv_wait_sig() at _cv_wait_sig+0x16d/frame 0xfffffe01428808e0 tty_wait() at tty_wait+0x42/frame 0xfffffe0142880910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe0142880970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01428809a0 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe0142880a00 dofileread() at dofileread+0x81/frame 0xfffffe0142880a50 sys_read() at sys_read+0xbc/frame 0xfffffe0142880ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0142880bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0142880bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c558a, rsp = 0x7fffffffeb68, rbp = 0x7fffffffebb0 --- Tracing command getty pid 2762 tid 100260 td 0xfffffe014279ac00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe01428997b0 mi_switch() at mi_switch+0x159/frame 0xfffffe01428997d0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe0142899810 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0142899860 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0142899870 _cv_wait_sig() at _cv_wait_sig+0x16d/frame 0xfffffe01428998e0 tty_wait() at tty_wait+0x42/frame 0xfffffe0142899910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe0142899970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01428999a0 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe0142899a00 dofileread() at dofileread+0x81/frame 0xfffffe0142899a50 sys_read() at sys_read+0xbc/frame 0xfffffe0142899ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0142899bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0142899bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c558a, rsp = 0x7fffffffeb68, rbp = 0x7fffffffebb0 --- Tracing command getty pid 2763 tid 100281 td 0xfffffe0143e23100 sched_switch() at sched_switch+0x5b3/frame 0xfffffe01429157b0 mi_switch() at mi_switch+0x159/frame 0xfffffe01429157d0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe0142915810 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0142915860 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0142915870 _cv_wait_sig() at _cv_wait_sig+0x16d/frame 0xfffffe01429158e0 tty_wait() at tty_wait+0x42/frame 0xfffffe0142915910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe0142915970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01429159a0 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe0142915a00 dofileread() at dofileread+0x81/frame 0xfffffe0142915a50 sys_read() at sys_read+0xbc/frame 0xfffffe0142915ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0142915bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0142915bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c558a, rsp = 0x7fffffffeb68, rbp = 0x7fffffffebb0 --- Tracing command getty pid 2764 tid 100295 td 0xfffffe0143e22a00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe01429107b0 mi_switch() at mi_switch+0x159/frame 0xfffffe01429107d0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe0142910810 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0142910860 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0142910870 _cv_wait_sig() at _cv_wait_sig+0x16d/frame 0xfffffe01429108e0 tty_wait() at tty_wait+0x42/frame 0xfffffe0142910910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe0142910970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01429109a0 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe0142910a00 dofileread() at dofileread+0x81/frame 0xfffffe0142910a50 sys_read() at sys_read+0xbc/frame 0xfffffe0142910ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0142910bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0142910bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c558a, rsp = 0x7fffffffeb68, rbp = 0x7fffffffebb0 --- Tracing command getty pid 2765 tid 100296 td 0xfffffe0143e22300 sched_switch() at sched_switch+0x5b3/frame 0xfffffe014295d7b0 mi_switch() at mi_switch+0x159/frame 0xfffffe014295d7d0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe014295d810 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe014295d860 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe014295d870 _cv_wait_sig() at _cv_wait_sig+0x16d/frame 0xfffffe014295d8e0 tty_wait() at tty_wait+0x42/frame 0xfffffe014295d910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe014295d970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe014295d9a0 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe014295da00 dofileread() at dofileread+0x81/frame 0xfffffe014295da50 sys_read() at sys_read+0xbc/frame 0xfffffe014295dac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014295dbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014295dbf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c558a, rsp = 0x7fffffffeb68, rbp = 0x7fffffffebb0 --- Tracing command getty pid 2766 tid 100297 td 0xfffffe0143e21c00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe01429587b0 mi_switch() at mi_switch+0x159/frame 0xfffffe01429587d0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe0142958810 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0142958860 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0142958870 _cv_wait_sig() at _cv_wait_sig+0x16d/frame 0xfffffe01429588e0 tty_wait() at tty_wait+0x42/frame 0xfffffe0142958910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe0142958970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01429589a0 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe0142958a00 dofileread() at dofileread+0x81/frame 0xfffffe0142958a50 sys_read() at sys_read+0xbc/frame 0xfffffe0142958ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0142958bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0142958bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c558a, rsp = 0x7fffffffeb68, rbp = 0x7fffffffebb0 --- Tracing command getty pid 2767 tid 100298 td 0xfffffe0143e21500 sched_switch() at sched_switch+0x5b3/frame 0xfffffe01429677b0 mi_switch() at mi_switch+0x159/frame 0xfffffe01429677d0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe0142967810 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0142967860 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0142967870 _cv_wait_sig() at _cv_wait_sig+0x16d/frame 0xfffffe01429678e0 tty_wait() at tty_wait+0x42/frame 0xfffffe0142967910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe0142967970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01429679a0 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe0142967a00 dofileread() at dofileread+0x81/frame 0xfffffe0142967a50 sys_read() at sys_read+0xbc/frame 0xfffffe0142967ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0142967bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0142967bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c558a, rsp = 0x7fffffffeb68, rbp = 0x7fffffffebb0 --- Tracing command getty pid 2768 tid 100299 td 0xfffffe0143e20e00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe01429627b0 mi_switch() at mi_switch+0x159/frame 0xfffffe01429627d0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe0142962810 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0142962860 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0142962870 _cv_wait_sig() at _cv_wait_sig+0x16d/frame 0xfffffe01429628e0 tty_wait() at tty_wait+0x42/frame 0xfffffe0142962910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe0142962970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01429629a0 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe0142962a00 dofileread() at dofileread+0x81/frame 0xfffffe0142962a50 sys_read() at sys_read+0xbc/frame 0xfffffe0142962ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0142962bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0142962bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c558a, rsp = 0x7fffffffeb68, rbp = 0x7fffffffebb0 --- Tracing command getty pid 2769 tid 100300 td 0xfffffe0143e20700 sched_switch() at sched_switch+0x5b3/frame 0xfffffe01429717b0 mi_switch() at mi_switch+0x159/frame 0xfffffe01429717d0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe0142971810 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0142971860 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0142971870 _cv_wait_sig() at _cv_wait_sig+0x16d/frame 0xfffffe01429718e0 tty_wait() at tty_wait+0x42/frame 0xfffffe0142971910 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe0142971970 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe01429719a0 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe0142971a00 dofileread() at dofileread+0x81/frame 0xfffffe0142971a50 sys_read() at sys_read+0xbc/frame 0xfffffe0142971ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0142971bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0142971bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011c558a, rsp = 0x7fffffffeb68, rbp = 0x7fffffffebb0 --- Tracing command sshd pid 2797 tid 100223 td 0xfffffe010e152300 sched_switch() at sched_switch+0x5b3/frame 0xfffffe01415af7b0 mi_switch() at mi_switch+0x159/frame 0xfffffe01415af7d0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe01415af810 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01415af860 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01415af870 _cv_wait_sig() at _cv_wait_sig+0x16d/frame 0xfffffe01415af8e0 seltdwait() at seltdwait+0x88/frame 0xfffffe01415af920 kern_poll() at kern_poll+0x4b6/frame 0xfffffe01415afaa0 sys_poll() at sys_poll+0x50/frame 0xfffffe01415afac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01415afbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01415afbf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x80169950a, rsp = 0x7fffffffe548, rbp = 0x7fffffffe580 --- Tracing command sshd pid 2800 tid 100291 td 0xfffffe0143e23800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe0142944720 mi_switch() at mi_switch+0x159/frame 0xfffffe0142944740 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe0142944780 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01429447d0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01429447e0 _cv_wait_sig() at _cv_wait_sig+0x16d/frame 0xfffffe0142944850 seltdwait() at seltdwait+0x88/frame 0xfffffe0142944890 kern_select() at kern_select+0x99c/frame 0xfffffe0142944a80 sys_select() at sys_select+0x56/frame 0xfffffe0142944ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0142944bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0142944bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80169960a, rsp = 0x7fffffffa4d8, rbp = 0x7fffffffa510 --- Tracing command bash pid 2801 tid 100301 td 0xfffffe0143e20000 --- swapped out Tracing command su pid 2806 tid 100293 td 0xfffffe010e5e1000 --- swapped out Tracing command csh pid 2807 tid 100264 td 0xfffffe01427b2c00 --- swapped out Tracing command bash pid 2809 tid 100302 td 0xfffffe0143ee8800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe014297b6e0 mi_switch() at mi_switch+0x159/frame 0xfffffe014297b700 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe014297b740 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe014297b790 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe014297b7a0 _cv_wait_sig() at _cv_wait_sig+0x16d/frame 0xfffffe014297b810 seltdwait() at seltdwait+0x88/frame 0xfffffe014297b850 kern_select() at kern_select+0x99c/frame 0xfffffe014297ba40 sys_pselect() at sys_pselect+0x127/frame 0xfffffe014297bac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014297bbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014297bbf0 --- syscall (522, FreeBSD ELF64, sys_pselect), rip = 0x80051154a, rsp = 0x7fffffffd588, rbp = 0x7fffffffd650 --- Tracing command sh pid 2816 tid 100261 td 0xfffffe014279a500 --- swapped out Tracing command sh pid 3387 tid 100313 td 0xfffffe0143ee6c00 --- swapped out Tracing command sh pid 3438 tid 100258 td 0xfffffe01427f0800 sched_switch() at sched_switch+0x5b3/frame 0xfffffe01428856c0 mi_switch() at mi_switch+0x159/frame 0xfffffe01428856e0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe0142885720 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0142885770 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0142885780 _sleep() at _sleep+0x2ac/frame 0xfffffe0142885830 kern_wait6() at kern_wait6+0x482/frame 0xfffffe01428858c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0142885ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0142885bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0142885bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80120976a, rsp = 0x7fffffffe238, rbp = 0x7fffffffe2e0 --- Tracing command awk pid 3439 tid 100290 td 0xfffffe0142799000 sched_switch() at sched_switch+0x5b3/frame 0xfffffe0142949830 mi_switch() at mi_switch+0x159/frame 0xfffffe0142949850 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe0142949890 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01429498e0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01429498f0 _sleep() at _sleep+0x2ac/frame 0xfffffe01429499a0 pipe_read() at pipe_read+0x4c6/frame 0xfffffe0142949a00 dofileread() at dofileread+0x81/frame 0xfffffe0142949a50 sys_read() at sys_read+0xbc/frame 0xfffffe0142949ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0142949bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0142949bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80120e58a, rsp = 0x7fffffffe238, rbp = 0x7fffffffe250 --- Tracing command sh pid 3441 tid 100287 td 0xfffffe01427ed700 sched_switch() at sched_switch+0x5b3/frame 0xfffffe0142935830 mi_switch() at mi_switch+0x159/frame 0xfffffe0142935850 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe0142935890 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01429358e0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01429358f0 _sleep() at _sleep+0x2ac/frame 0xfffffe01429359a0 pipe_read() at pipe_read+0x4c6/frame 0xfffffe0142935a00 dofileread() at dofileread+0x81/frame 0xfffffe0142935a50 sys_read() at sys_read+0xbc/frame 0xfffffe0142935ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0142935bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0142935bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80120958a, rsp = 0x7fffffffdcc8, rbp = 0x7fffffffe1f0 --- Tracing command sh pid 22388 tid 101553 td 0xfffffe014e2e0300 --- swapped out Tracing command tee pid 22389 tid 100615 td 0xfffffe014e541500 sched_switch() at sched_switch+0x5b3/frame 0xfffffe014f553830 mi_switch() at mi_switch+0x159/frame 0xfffffe014f553850 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe014f553890 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe014f5538e0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe014f5538f0 _sleep() at _sleep+0x2ac/frame 0xfffffe014f5539a0 pipe_read() at pipe_read+0x4c6/frame 0xfffffe014f553a00 dofileread() at dofileread+0x81/frame 0xfffffe014f553a50 sys_read() at sys_read+0xbc/frame 0xfffffe014f553ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe014f553bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014f553bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8011a758a, rsp = 0x7fffffffe5c8, rbp = 0x7fffffffe620 --- Tracing command sh pid 22390 tid 100232 td 0xfffffe010e5e3a00 sched_switch() at sched_switch+0x5b3/frame 0xfffffe01415f56c0 mi_switch() at mi_switch+0x159/frame 0xfffffe01415f56e0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe01415f5720 sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe01415f5770 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe01415f5780 _sleep() at _sleep+0x2ac/frame 0xfffffe01415f5830 kern_wait6() at kern_wait6+0x482/frame 0xfffffe01415f58c0 sys_wait4() at sys_wait4+0x7d/frame 0xfffffe01415f5ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe01415f5bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01415f5bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80120976a, rsp = 0x7fffffffe078, rbp = 0x7fffffffe120 --- Tracing command md10 pid 22406 tid 101339 td 0xfffffe0148144000 sched_switch() at sched_switch+0x5b3/frame 0xfffffe0150ce2a50 mi_switch() at mi_switch+0x159/frame 0xfffffe0150ce2a70 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe0150ce2ab0 _sleep() at _sleep+0x2b6/frame 0xfffffe0150ce2b60 md_kthread() at md_kthread+0xd8/frame 0xfffffe0150ce2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0150ce2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0150ce2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db> reset cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 10 (kgdb) thread 7 [Switching to thread 7 (Thread 100283)] #0 cpustop_handler () at ../../../x86/x86/mp_x86.c:1475 1475 use_mwait = (stop_mwait && (cpu_feature2 & CPUID2_MON) != 0 && (kgdb) bt #0 cpustop_handler () at ../../../x86/x86/mp_x86.c:1475 #1 0xffffffff810a29b9 in ipi_nmi_handler () at ../../../x86/x86/mp_x86.c:1432 #2 0xffffffff810dbb06 in trap (frame=0xfffffe003ce3af30) at ../../../amd64/amd64/trap.c:201 #3 #4 lock_delay (la=la@entry=0xfffffe01428dc1b8) at ../../../kern/subr_lock.c:131 #5 0xffffffff80bed9e8 in __mtx_lock_sleep (c=c@entry=0xffffffff81a442d8 , v=, opts=, opts@entry=0x0, file=, file@entry=0xffffffff8125fc5f "../../../kern/vfs_subr.c", line=, line@entry=0x4f1) at ../../../kern/kern_mutex.c:600 #6 0xffffffff80bed885 in __mtx_lock_flags (c=0xffffffff81a442d8 , opts=0x0, file=0xffffffff8125fc5f "../../../kern/vfs_subr.c", line=0x4f1) at ../../../kern/kern_mutex.c:280 #7 0xffffffff80cf96de in vnlru_free_locked (count=count@entry=0x1, mnt_op=mnt_op@entry=0x0) at ../../../kern/vfs_subr.c:1265 #8 0xffffffff80d05215 in vn_alloc_hard (mp=mp@entry=0xfffffe0148bdab80) at ../../../kern/vfs_subr.c:1618 #9 0xffffffff80cf9e74 in vn_alloc (mp=0xfffffe0148bdab80) at /usr/src/sys/amd64/include/atomic.h:451 #10 getnewvnode (tag=0xffffffff811d2140 "ufs", mp=0xfffffe0148bdab80, vops=0xffffffff81b62d98 , vpp=0xfffffe01428dc468) at ../../../kern/vfs_subr.c:1683 #11 0xffffffff80f3b10b in ffs_vgetf (mp=0xfffffe0148bdab80, ino=0x2a0109, flags=0x80200, vpp=0xfffffe01428dc558, ffs_flags=0x9) at ../../../ufs/ffs/ffs_vfsops.c:2019 #12 0xffffffff80f1e8ca in handle_workitem_remove (dirrem=0xfffff806014fc300, flags=0x80200) at ../../../ufs/ffs/ffs_softdep.c:10196 #13 0xffffffff80f0ebc8 in process_worklist_item (mp=0xfffffe0148bdab80, target=0xa, flags=0x200) at ../../../ufs/ffs/ffs_softdep.c:2010 #14 0xffffffff80f09a45 in softdep_process_worklist (mp=0xfffffe0148bdab80, full=0x1) at ../../../ufs/ffs/ffs_softdep.c:1809 #15 0xffffffff80f098ce in softdep_flushworklist (oldmnt=0xfffffe0148bdab80, countp=0xfffffe01428dc7d8, td=0xfffffe0143dd2a00) at ../../../ufs/ffs/ffs_softdep.c:2121 #16 0xffffffff80f3eb3a in ffs_sync (mp=0xfffffe0148bdab80, waitfor=0x1) at ../../../ufs/ffs/ffs_vfsops.c:1905 #17 0xffffffff80d16825 in vfs_write_suspend (mp=mp@entry=0xfffffe0148bdab80, flags=flags@entry=0x0) at ../../../kern/vfs_vnops.c:2051 #18 0xffffffff80d169e5 in vfs_write_suspend_umnt (mp=0xfffffe0148bdab80) at ../../../kern/vfs_vnops.c:2105 #19 0xffffffff80f3ded5 in ffs_unmount (mp=0xfffffe0148bdab80, mntflags=0x8000000) at ../../../ufs/ffs/ffs_vfsops.c:1519 #20 0xffffffff80cf400b in dounmount (mp=0xfffffe01428dc1b8, mp@entry=0xfffffe0148bdab80, flags=flags@entry=0x8000000, td=td@entry=0xfffffe0143dd2a00) at ../../../kern/vfs_mount.c:1881 #21 0xffffffff80cf3b5c in kern_unmount (td=0xfffffe0143dd2a00, path=, flags=0x8000000) at ../../../kern/vfs_mount.c:1512 #22 0xffffffff810dd0a7 in syscallenter (td=0xfffffe0143dd2a00) at ../../../amd64/amd64/../../kern/subr_syscall.c:189 #23 amd64_syscall (td=0xfffffe0143dd2a00, traced=0x0) at ../../../amd64/amd64/trap.c:1156 #24 #25 0x00000008011a758a in ?? () Backtrace stopped: Cannot access memory at address 0x7fffffffd228 (kgdb) f 7 #7 0xffffffff80cf96de in vnlru_free_locked (count=count@entry=0x1, mnt_op=mnt_op@entry=0x0) at ../../../kern/vfs_subr.c:1265 1265 mtx_lock(&vnode_list_mtx); (kgdb) l 1260 TAILQ_REMOVE(&vnode_list, mvp, v_vnodelist); 1261 TAILQ_INSERT_AFTER(&vnode_list, vp, mvp, v_vnodelist); 1262 mtx_unlock(&vnode_list_mtx); 1263 if (vtryrecycle(vp) == 0) 1264 count--; 1265 mtx_lock(&vnode_list_mtx); 1266 vp = mvp; 1267 } 1268 return (ocount - count); 1269 } (kgdb) p vnode_list_mtx $1 = {lock_object = {lo_name = 0xffffffff8125fd3b "vnode_list", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xfffff8087fd7fe80}, mtx_lock = 0x0} (kgdb) f 8 #8 0xffffffff80d05215 in vn_alloc_hard (mp=mp@entry=0xfffffe0148bdab80) at ../../../kern/vfs_subr.c:1618 1618 if (vnlru_free_locked(1, NULL) > 0) (kgdb) l 1613 * Otherwise, wait for some space. In all cases, schedule 1614 * vnlru_proc() if we are getting short of space. The watermarks 1615 * should be chosen so that we never wait or even reclaim from 1616 * the free list to below its target minimum. 1617 */ 1618 if (vnlru_free_locked(1, NULL) > 0) 1619 goto alloc; 1620 if (mp == NULL || (mp->mnt_kern_flag & MNTK_SUSPEND) == 0) { 1621 /* 1622 * Wait for space for a new vnode. (kgdb) info loc rnumvnodes = rfreevnodes = 0x1000 (kgdb) p *mp $2 = {mnt_vfs_ops = 0x2, mnt_kern_flag = 0x49846144, mnt_flag = 0x201000, mnt_pcpu = 0xfffffe01c202cf80, mnt_rootvnode = 0x0, mnt_vnodecovered = 0xfffffe01612b7068, mnt_op = 0xffffffff81b625d0 , mnt_vfc = 0xffffffff81b626f0 , mnt_mtx = {lock_object = { lo_name = 0xffffffff811af73d "struct mount mtx", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xfffff8087fd75b80}, mtx_lock = 0x0}, mnt_gen = 0x7, mnt_list = {tqe_next = 0x0, tqe_prev = 0xfffffe010e5ef2e8}, mnt_syncer = 0x0, mnt_ref = 0x96d3f, mnt_nvnodelist = { tqh_first = 0xfffffe014631ce10, tqh_last = 0xfffffe018858cbe0}, mnt_nvnodelistsize = 0x96d3d, mnt_writeopcount = 0x0, mnt_opt = 0xfffff8001b203090, mnt_optnew = 0x0, mnt_maxsymlinklen = 0x78, mnt_stat = {f_version = 0x20140518, f_type = 0x35, f_flags = 0x201000, f_bsize = 0x1000, f_iosize = 0x8000, f_blocks = 0x6f75e5, f_bfree = 0x6f75e3, f_bavail = 0x668b2d, f_files = 0x3aaffe, f_ffree = 0x3aaffc, f_syncwrites = 0x3a4d0b, f_asyncwrites = 0x328485, f_syncreads = 0x10066, f_asyncreads = 0xaa3f, f_spare = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, f_namemax = 0xff, f_owner = 0x0, f_fsid = {val = {0x604229bd, 0x90280eda}}, f_charspare = '\000' , f_fstypename = "ufs", '\000' , f_mntfromname = "/dev/md10a", '\000' , f_mntonname = "/mnt", '\000' }, mnt_cred = 0xfffff806df6cb200, mnt_data = 0xfffff8041bf04400, mnt_time = 0x0, mnt_iosize_max = 0x100000, mnt_export = 0x0, mnt_label = 0x0, mnt_hashseed = 0xa9cef571, mnt_lockref = 0x0, mnt_secondary_writes = 0x2, mnt_secondary_accwrites = 0xf7726f, mnt_susp_owner = 0xfffffe0143dd2a00, mnt_gjprovider = 0x0, mnt_listmtx = {lock_object = {lo_name = 0xffffffff81248465 "struct mount vlist mtx", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xfffff8087fd83500}, mtx_lock = 0x0}, mnt_lazyvnodelist = {tqh_first = 0xfffffe01740fa960, tqh_last = 0xfffffe0185494f78}, mnt_lazyvnodelistsize = 0x3e8, mnt_explock = {lock_object = {lo_name = 0xffffffff81280a0f "explock", lo_flags = 0x6730000, lo_data = 0x0, lo_witness = 0xfffff8087fd80f80}, lk_lock = 0x1, lk_exslpfail = 0x0, lk_pri = 0x60, lk_timo = 0x0, lk_stack = {depth = 0x0, pcs = { 0x0 }}}, mnt_upper_link = {tqe_next = 0x0, tqe_prev = 0x0}, mnt_uppers = {tqh_first = 0x0, tqh_last = 0xfffffe0148bdb6b8}} (kgdb) what /usr/src/sys/amd64/compile/PHO/kernel.debug /usr/src/sys/amd64/compile/PHO/kernel.debug: ls -l /usr/src/sys/amd64/compile/PHO/kernel.debug /var/crash/vmcore.298 -rwxr-xr-x 1 root wheel 100030040 Mar 4 20:14 /usr/src/sys/amd64/compile/PHO/kernel.debug -rw------- 1 root wheel 4094410752 Mar 5 19:01 /var/crash/vmcore.298 vmstat -N /usr/src/sys/amd64/compile/PHO/kernel.debug -M /var/crash/vmcore.298 -m Type InUse MemUse Requests Size(s) DEVFSP 2 1K 432 64 NFSD V4client 1 1K 1 256 NFSD lckfile 1 1K 1 256 NFSD session 1 1K 1 1024 pfs_nodes 20 8K 20 384 pfs_vncache 1 64K 1 65536 tmpfs mount 0 0K 1 128 tmpfs name 0 0K 1 16 tmpfs dir 0 0K 1 64 GEOM 96 23K 3310631 16,32,64,128,256,384,512,1024,2048,4096,8192,16384,32768 raid_data 0 0K 264108 32,128,256 isadev 6 1K 6 128 isofs_mount 0 0K 1 256 isofs_node 0 0K 135 256 SCSI ENC 48 264K 8194 16,64,256,384,1024,2048,32768,65536 nvlist 28 3K 28 16,64,128 ddb_capture 1 64K 1 65536 cdev 1 1K 1 256 filedesc 10 73K 1136 16,32,64,128,4096,8192,16384,32768,65536 pwddesc 81 6K 3237639 64 sigio 0 0K 10 64 filecaps 4 1K 34677 16,32,64 kdtrace 1504 361K 9093225 64,256 kenv 129 13K 129 16,32,64,128,8192 kqueue 81 6K 3637793 64,256,512,2048 proc-args 136 6K 681044 16,32,64,128,256 Fail Points 0 0K 14476 1024 hhook 13 4K 13 256 ithread 280 46K 280 32,128,256 prison 8 1K 8 32 KTRACE 100 13K 100 128 evdev 5 5K 5 1024 linker 422 396K 2046 16,32,64,128,256,384,512,1024,2048,4096,8192,16384,32768,65536 lockf 27 9K 1633964128 64,128,256,384,1024,2048,4096 loginclass 3 1K 3 64 devbuf 52230 202351K 224306 16,32,64,128,256,384,512,1024,2048,4096,8192,65536 temp 84 34K 54123548 16,32,64,128,256,384,512,1024,2048,4096,8192,16384,32768,65536 module 511 64K 576 128 mtx_pool 2 72K 4 8192,65536 osd 3 1K 10 16,32,64,128,256 pmchooks 1 1K 1 128 pmc 42 9985K 4076 16,32,64,128,256,384,512,1024,4096,8192,16384,32768 session 28 4K 235 128 proc 3 132K 3 4096,65536 subproc 1306 784K 3238864 384,4096 cred 29 8K 10449 256 acpiintr 1 1K 1 64 acpica 24620 2405K 3657276 16,32,64,128,256,384,512,1024,2048,4096 plimit 25 7K 2252 256 uidinfo 5 17K 155 128,16384 sendfile 0 0K 2718169 64,128 dumper 2 1K 2 128,512 kbdmux 7 22K 7 16,384,1024,2048,16384 sysctl 0 0K 5909 32,64 sysctloid 37580 2146K 42452 16,32,64,128 sysctltmp 0 0K 1798541 16,32,64,128,256,1024 LED 40 3K 40 16,128 tidhash 3 260K 3 8192 callout 12 3072K 12 umtx 11376 1422K 11376 128 p1003.1b 1 1K 1 16 SWAP 1 4164K 3 bus 1675 181K 146094 16,32,64,128,256,4096 bus-sc 128 2104K 48040 16,32,64,128,256,384,512,1024,2048,4096,8192,16384,32768,65536 md_disk 1 4K 54 4096 devstat 18 37K 18 32,4096 eventhandler 143 12K 160 64,128 gtaskqueue 78 32K 78 16,32,256,8192 kobj 339 1356K 356611 4096 Per-cpu 1 1K 1 32 rman 352 41K 739 16,32,128 sbuf 0 0K 22201719 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536 toponodes 32 4K 32 128 stack 0 0K 157943 256 taskqueue 75 9K 81 16,32,128,256 terminal 11 3K 11 256 Unitno 30 2K 9493170 32,64 vmem 3 1152K 91 256,512,1024,2048,4096,8192,16384,32768,65536 ioctlops 0 0K 658 256,384,512,1024 select 15 2K 8564 128 iov 2 1K 1396068313 16,32,64,128,256,384 selfd 47 3K 10614081 64 procdesc 1 1K 3408 128 msg 4 30K 4 2048,4096,8192,16384 sem 4 106K 4 2048,4096 shm 1 32K 21 2048,32768 tty 70 70K 2320854 1024 pts 56 14K 2320840 256 acpitask 1 64K 1 65536 acpisem 32 4K 32 128 acpidev 183 12K 183 64 ktls 1 2K 1 2048 mbuf_tag 0 0K 82 32 shmfd 1 8K 1 8192 soname 5 1K 1678182 16,32,128 pcb 30 3082K 191 16,32,64,1024,2048 CAM SIM 13 4K 13 256 entropy 1 1K 1367080 16,32,64,4096 CAM XPT 49 5K 522526 16,32,64,128,512,1024,2048,65536 CAM DEV 16 32K 35 2048 CAM CCB 0 0K 7298660 2048 CAM path 19 1K 485 32 CAM periph 8 2K 32 16,32,64,128,256 vfscache 3 8216K 3 8192,16384 feeder 7 1K 7 32 cl_savebuf 0 0K 15488536 64,128,256,384,512,1024,2048,4096,8192 vfs_hash 1 4096K 1 vnodes 1 1K 2682 32,256 namei_tracker 0 0K 234 32 mount 44 3K 3689 16,32,64,128,256,1024 statfs 0 0K 92975 4096 VN POLL 1 1K 203 128 vnodemarker 2 2K 2651101 1024 chacha20random 1 4K 1 4096 BPF 13 18K 530 128,256,512,1024,4096 ifnet 6 11K 6 128,2048 ifaddr 304 23K 306 16,32,64,128,256,384,2048,4096 ether_multi 47 4K 57 16,32,64,128 clone 8 1K 8 128 lltable 23 7K 23 256,384 tun 3 1K 3 32 iflib 153 1148K 161 64,128,512,1024,8192,16384,32768 pfil 7 1K 7 128 nhops 6 1K 6 128 routetbl 15 4K 15 32,64,128,384,1024 vnet 1 1K 1 64 vnet_data 1 168K 1 vnet_data_free 1 1K 1 32 igmp 5 1K 5 128 in_multi 2 1K 4 256 ip_moptions 1 1K 1 64 encap_export_host 8 1K 8 32,64 tfo_ccache 1 128K 1 hostcache 1 32K 1 32768 LRO 24 480K 24 8192,32768 tcpfunc 1 1K 1 64 syncache 1 68K 1 in6_mfilter 2 1K 2 64 in6_multi 27 4K 27 32,256 ip6_moptions 3 1K 3 32 ip6opt 4 1K 4 256 mld 5 1K 5 128 ip6ndp 7 1K 8 64,256 inpcbpolicy 34 2K 999 32 secasvar 1 1K 1 1024 sahead 1 1K 1 1024 ipsecpolicy 1 1K 1 1024 ipsec-saq 2 2K 2 1024 NLM 0 0K 1 32 linux 26061 4674K 26588 16,32,64,128,256,384,512,1024,8192,16384 linuxcurrent 13 3K 22 64,256 radix 2 2K 2 1024 MLX5EN 2 256K 2 crypto 4 3K 43 64,256,1024,2048 audit_data 0 0K 2 64 audit_evclass 664 21K 956 32 audit_evname 658 165K 662 256 audit_trigger 0 0K 1 32 pagedep 3 1536K 27553132 256 inodedep 818940 319390K 125184608 384 bmsafemap 3 24K 108258798 256,8192 newblk 3 24576K 70100075 256 indirdep 0 0K 74078 256,32768 freefrag 8501 1063K 15177103 128 freeblks 0 0K 16780351 256 freefile 734249 91782K 106958562 128 diradd 0 0K 174055913 128 mkdir 0 0K 40666126 128 dirrem 822623 205656K 173961260 256 newdirblk 0 0K 20821257 128 freework 3 1K 17730491 32,256 freedep 0 0K 6885 128 jaddref 0 0K 2021091 256 jremref 0 0K 2021025 256 jnewblk 0 0K 7637255 128 jfreefrag 0 0K 3833521 128 jseg 0 0K 13918 256 jsegdep 0 0K 15512892 128 sbdep 0 0K 235399 128 savedino 0 0K 87847685 256 jblocks 0 0K 8 128,256 softdep 3 3K 164 1024 ufs_dirhash 95 31K 1203187 16,32,64,128,256,384,512,1024,2048,4096 ufs_quota 1 4096K 1 ufs_mount 15 74K 1008 32,64,512,4096,8192,32768 vm_pgdata 1 1K 3 128 acpi_perf 12 5K 12 384 UMA 257 5K 302 16,32 CAM I/O Scheduler 1 1K 1 128 UART 6 5K 6 16,1024 CAM queue 29 9K 89 16,32,512 USB 53 71K 61 16,128,256,384,1024,2048,4096 USBdev 39 5K 321 32,64,128,256,384,512,4096 CAM dev queue 13 1K 13 64 rpc 54 18K 184 16,32,64,128,256,384,512,1024,4096 aesni_data 2 2K 373 128,1024,4096 pci_link 16 2K 16 64,128 apmdev 1 1K 146 128 intr 4 472K 4 65536 MCA 35 8K 35 128,384 cpus 2 1K 2 64 nexusdev 7 1K 7 16 vtbuf 24 1968K 46 4096 vt 13 41K 13 384,4096,32768 fpukern_ctx 13 13K 13 1024 memdesc 1 4K 1 4096 DEVFS3 271 68K 2385327 256 DEVFS1 215 215K 2341857 1024 madt_table 0 0K 2 128,4096 io_apic 2 4K 2 2048 local_apic 1 16K 1 16384 DEVFS_RULE 56 20K 56 64,384 msi 62 8K 62 128 DEVFS 25 1K 35 16,32,64,128 cpuctl 2 65K 7 128,65536 acpiwmi 1 1K 2 32,64 ioat 8 3584K 8 eli data 0 0K 161939 64,128,512,1024,4096,8192,16384,32768,65536 nullfs_hash 1 4096K 1 nullfs_node 0 0K 7773775 64 nullfs_mount 0 0K 49 32 vmstat -N /usr/src/sys/amd64/compile/PHO/kernel.debug -M /var/crash/vmcore.298 -z ITEM SIZE LIMIT USED FREE REQ FAILSLEEP XDOMAIN UMA Kegs: 384, 0, 198, 0, 243, 0, 0, 0 UMA Zones: 2048, 0, 229, 0, 274, 0, 0, 0 UMA Slabs 0: 112, 0, 44235, 0, 285675, 0, 0, 0 UMA Slabs 1: 176, 0, 9, 0, 9, 0, 0, 0 UMA Hash: 256, 0, 0, 0, 0, 0, 0, 0 2 Bucket: 32, 0, 41131, 3336, 6560762,29221151, 0, 0 4 Bucket: 48, 0, 8705, 2625, 1875044,25660530, 0, 0 8 Bucket: 80, 0, 16604, 1671,61427140,6507553, 0, 0 16 Bucket: 144, 0, 26941, 894,59046970,41875810, 0, 0 32 Bucket: 256, 0, 94882, 2950,70649644,14233487, 0, 0 64 Bucket: 512, 0, 153521, 579,35271609,120388434, 0, 0 128 Bucket: 1024, 0, 56258, 85,20717029,91013437, 0, 0 256 Bucket: 2048, 0, 21241, 285,34024707,167035468, 0, 0 SMR SHARED: 24, 0, 46, 0, 46, 0, 0, 0 SMR CPU: 32, 0, 46, 0, 46, 0, 0, 0 vmem: 1856, 0, 1, 0, 1, 0, 0, 0 vmem btag: 56, 0, 636685, 2820, 2368705,9875, 0, 0 VM OBJECT: 264, 0, 382603, 140411,151800154, 0, 0, 0 RADIX NODE: 144, 0, 488010, 12892,114136860, 0, 0, 0 KMAP ENTRY: 96, 0, 45, 0, 248, 0, 0, 0 MAP ENTRY: 96, 0, 1692, 5573,116403349, 0, 0, 0 VMSPACE: 2544, 0, 1390, 215, 3164130, 0, 0, 0 fakepg: 104, 0, 0, 0, 405, 0, 0, 0 pcpu-4: 4, 0, 0, 0, 0, 0, 0, 0 pcpu-8: 8, 0, 3490, 0, 3704, 0, 0, 0 pcpu-16: 16, 0, 52, 0, 52, 0, 0, 0 pcpu-32: 32, 0, 0, 0, 0, 0, 0, 0 pcpu-64: 64, 0, 456, 0, 472, 0, 0, 0 malloc-16: 16, 0, 25, 0, 26343, 0, 0, 0 malloc-16: 16, 0, 14644, 254, 199248, 0, 0, 0 malloc-16: 16, 0, 12, 2246, 3449159, 0, 0, 0 malloc-16: 16, 0, 1258, 3006, 1988526, 0, 0, 0 malloc-16: 16, 0, 56, 0, 84, 0, 0, 0 malloc-16: 16, 0, 904, 350, 5880261, 0, 0, 0 malloc-16: 16, 0, 11, 0, 11, 0, 0, 0 malloc-16: 16, 0, 1, 2749, 1658965, 0, 0, 0 malloc-32: 32, 0, 73, 254, 1062, 0, 0, 0 malloc-32: 32, 0, 8007, 1822, 159902, 0, 0, 0 malloc-32: 32, 0, 256, 2401, 451722, 0, 0, 0 malloc-32: 32, 0, 1080, 1944, 305186, 0, 0, 0 malloc-32: 32, 0, 692, 0, 1109, 0, 0, 0 malloc-32: 32, 0, 362, 266, 227530, 0, 0, 0 malloc-32: 32, 0, 293, 0, 9496275, 0, 0, 0 malloc-32: 32, 0, 7, 254, 4712, 0, 0, 0 malloc-64: 64, 0, 47, 1025, 1717346, 0, 0, 0 malloc-64: 64, 0, 3958, 3038, 5450714, 0, 0, 0 malloc-64: 64, 0, 55, 1806, 640859, 0, 0, 0 malloc-64: 64, 0, 12512, 3292,17343693, 0, 0, 0 malloc-64: 64, 0, 53, 13158,571368248, 0, 0, 0 malloc-64: 64, 0, 9408, 302, 104637, 0, 0, 0 malloc-64: 64, 0, 346, 1112,12153699, 0, 0, 0 malloc-64: 64, 0, 343, 1066,1396050894, 0, 0, 0 malloc-128: 128, 0, 58, 0, 317, 0, 0, 0 malloc-128: 128, 0, 1160069, 6856,282080784, 0, 0, 0 malloc-128: 128, 0, 8505, 392,19385752, 0, 0, 0 malloc-128: 128, 0, 15779, 1355, 2111586, 0, 0, 0 malloc-128: 128, 0, 88634, 462,1078096007, 0, 0, 0 malloc-128: 128, 0, 81928, 2823,78284537, 0, 0, 0 malloc-128: 128, 0, 127, 504, 5542169, 0, 0, 0 malloc-128: 128, 0, 579, 351, 31141, 0, 0, 0 malloc-256: 256, 0, 21, 0, 8536467, 0, 0, 0 malloc-256: 256, 0, 822856, 126904,263943088, 0, 0, 0 malloc-256: 256, 0, 30, 421, 71691, 0, 0, 0 malloc-256: 256, 0, 1769, 0, 6045810, 0, 0, 0 malloc-256: 256, 0, 15922, 765,19168515, 0, 0, 0 malloc-256: 256, 0, 1611, 58110,213241177, 0, 0, 0 malloc-256: 256, 0, 324, 265, 1469453, 0, 0, 0 malloc-256: 256, 0, 17610, 88,17750296, 0, 0, 0 malloc-384: 384, 0, 3374, 0,35773526, 0, 0, 0 malloc-384: 384, 0, 28, 0, 119, 0, 0, 0 malloc-384: 384, 0, 1, 0, 1, 0, 0, 0 malloc-384: 384, 0, 69, 0, 8089, 0, 0, 0 malloc-384: 384, 0, 1017505, 128582,125383176, 0, 0, 0 malloc-384: 384, 0, 110, 0, 55088, 0, 0, 0 malloc-384: 384, 0, 43, 150, 74478, 0, 0, 0 malloc-384: 384, 0, 1277, 30, 1678, 0, 0, 0 malloc-512: 512, 0, 6, 0, 52, 0, 0, 0 malloc-512: 512, 0, 30, 0, 72, 0, 0, 0 malloc-512: 512, 0, 2, 0, 72576, 0, 0, 0 malloc-512: 512, 0, 364, 264, 405078, 0, 0, 0 malloc-512: 512, 0, 0, 0, 4, 0, 0, 0 malloc-512: 512, 0, 30, 0, 403098, 0, 0, 0 malloc-512: 512, 0, 3, 0, 14487, 0, 0, 0 malloc-512: 512, 0, 0, 0, 120, 0, 0, 0 malloc-1024: 1024, 0, 3, 0, 532, 0, 0, 0 malloc-1024: 1024, 0, 20, 0, 34985, 0, 0, 0 malloc-1024: 1024, 0, 3, 0, 774587, 0, 0, 0 malloc-1024: 1024, 0, 39, 0, 4509, 0, 0, 0 malloc-1024: 1024, 0, 72, 0, 2320882, 0, 0, 0 malloc-1024: 1024, 0, 313, 151,31014856, 0, 0, 0 malloc-1024: 1024, 0, 2, 0, 26648, 0, 0, 0 malloc-1024: 1024, 0, 219, 221, 2651374, 0, 0, 0 malloc-2048: 2048, 0, 0, 0,13311365, 0, 0, 0 malloc-2048: 2048, 0, 32, 0, 200213, 0, 0, 0 malloc-2048: 2048, 0, 3, 0, 2071, 0, 0, 0 malloc-2048: 2048, 0, 5456, 137, 7311314, 0, 0, 0 malloc-2048: 2048, 0, 2, 0, 3, 0, 0, 0 malloc-2048: 2048, 0, 417, 0,14160549, 0, 0, 0 malloc-2048: 2048, 0, 1, 0, 18251, 0, 0, 0 malloc-2048: 2048, 0, 0, 0, 0, 0, 0, 0 malloc-4096: 4096, 0, 7, 1, 48, 0, 0, 0 malloc-4096: 4096, 0, 32, 0, 5902, 0, 0, 0 malloc-4096: 4096, 0, 5, 0, 132503, 0, 0, 0 malloc-4096: 4096, 0, 18, 0, 297, 0, 0, 0 malloc-4096: 4096, 0, 2, 7, 92978, 0, 0, 0 malloc-4096: 4096, 0, 34120, 35, 5750995, 0, 0, 0 malloc-4096: 4096, 0, 14, 0, 24379, 0, 0, 0 malloc-4096: 4096, 0, 571, 183, 3594772, 0, 0, 0 malloc-8192: 8192, 0, 0, 0, 0, 0, 0, 0 malloc-8192: 8192, 0, 99, 0, 386, 0, 0, 0 malloc-8192: 8192, 0, 46, 0,13134831, 0, 0, 0 malloc-8192: 8192, 0, 4, 0, 4, 0, 0, 0 malloc-8192: 8192, 0, 6, 0, 46, 0, 0, 0 malloc-8192: 8192, 0, 39, 0, 65075, 0, 0, 0 malloc-8192: 8192, 0, 5, 0, 369, 0, 0, 0 malloc-8192: 8192, 0, 1, 0, 1, 0, 0, 0 malloc-16384: 16384, 0, 0, 0, 0, 0, 0, 0 malloc-16384: 16384, 0, 6, 0, 4468, 0, 0, 0 malloc-16384: 16384, 0, 1, 0, 8666, 0, 0, 0 malloc-16384: 16384, 0, 2, 0, 2, 0, 0, 0 malloc-16384: 16384, 0, 0, 0, 2, 0, 0, 0 malloc-16384: 16384, 0, 40, 1, 436736, 0, 0, 0 malloc-16384: 16384, 0, 1, 0, 1, 0, 0, 0 malloc-16384: 16384, 0, 1, 0, 1, 0, 0, 0 malloc-32768: 32768, 0, 1, 0, 1, 0, 0, 0 malloc-32768: 32768, 0, 13, 0, 48139, 0, 0, 0 malloc-32768: 32768, 0, 16, 0, 127399, 0, 0, 0 malloc-32768: 32768, 0, 0, 0, 0, 0, 0, 0 malloc-32768: 32768, 0, 16, 34, 24903, 0, 0, 0 malloc-32768: 32768, 0, 10, 42, 13604, 0, 0, 0 malloc-32768: 32768, 0, 1, 0, 2, 0, 0, 0 malloc-32768: 32768, 0, 1, 0, 1, 0, 0, 0 malloc-65536: 65536, 0, 1, 0, 522434, 0, 0, 0 malloc-65536: 65536, 0, 2, 0, 34, 0, 0, 0 malloc-65536: 65536, 0, 3, 0, 1041, 0, 0, 0 malloc-65536: 65536, 0, 1, 0, 1, 0, 0, 0 malloc-65536: 65536, 0, 0, 0, 0, 0, 0, 0 malloc-65536: 65536, 0, 8, 12, 4516, 0, 0, 0 malloc-65536: 65536, 0, 2, 0, 2, 0, 0, 0 malloc-65536: 65536, 0, 1, 0, 1, 0, 0, 0 SLEEPQUEUE: 88, 0, 5689, 0, 5689, 0, 0, 0 kenv: 258, 0, 18, 0, 21843, 0, 0, 0 Files: 80, 0, 7190, 1470,1306777521, 0, 0, 0 filedesc0: 1072, 0, 1387, 279, 3238945, 0, 0, 0 PWD: 32, 0, 17, 4273,66969036, 0, 0, 0 TURNSTILE: 136, 0, 5689, 0, 5689, 0, 0, 0 rangeset pctrie nodes: 144, 0, 0, 0, 0, 0, 0, 0 rl_entry: 40, 0, 1210, 0, 1214, 0, 0, 0 umtx pi: 96, 0, 0, 0, 0, 0, 0, 0 umtx_shm: 88, 0, 0, 0, 0, 0, 0, 0 swrbuf: 2776, 128, 132, 37, 1071414, 0, 0, 0 swwbuf: 2776, 64, 713, 25, 2792680, 0, 0, 0 nfspbuf: 2776, 128, 0, 0, 0, 0, 0, 0 mdpbuf: 2776, 25, 0, 0, 38764, 0, 0, 0 vnpbuf: 2776, 512, 25, 0, 11029, 0, 0, 0 clpbuf: 2776, 128, 725, 51, 2744046,31994, 0, 0 pbuf: 2776, 16, 0, 0, 0, 0, 0, 0 MAC labels: 40, 0, 0, 0, 0, 0, 0, 0 PROC: 1320, 0, 211, 259, 3237769, 0, 0, 0 PGRP: 88, 0, 32, 390, 17592, 0, 0, 0 THREAD: 1792, 0, 1422, 0, 5855585, 0, 0, 0 cpuset: 104, 0, 27, 0, 27, 0, 0, 0 domainset: 40, 0, 0, 0, 0, 0, 0, 0 audit_record: 1280, 0, 0, 0, 2, 0, 0, 0 mbuf_packet: 256, 0, 6, 2141, 8281495, 0, 0, 0 mbuf: 256, 12998582, 15627, 2330,35096032, 0, 0, 0 mbuf_cluster: 2048, 2031028, 59327, 4, 112008, 0, 0, 0 mbuf_jumbo_page: 4096, 1015514, 0, 0, 8376622, 0, 0, 0 mbuf_jumbo_9k: 9216, 300893, 0, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 169252, 0, 0, 0, 0, 0, 0 epoch_record pcpu: 256, 0, 4, 0, 4, 0, 0, 0 ktls_session: 192, 0, 0, 0, 0, 0, 0, 0 ttyinq: 160, 0, 1155, 0,34837824, 0, 0, 0 ttyoutq: 256, 0, 613, 0,18572363, 0, 0, 0 FPU_save_area: 832, 0, 1, 0, 1, 0, 0, 0 DEVCTL: 1024, 1000, 0, 0, 415479,8721902, 0, 0 IOMMU_MAP_ENTRY: 120, 0, 0, 0, 0, 0, 0, 0 g_bio: 408, 0, 73631, 1426,454697696, 0, 0, 0 linux_dma_pctrie: 144, 0, 0, 0, 0, 0, 0, 0 linux_dma_object: 24, 0, 0, 0, 0, 0, 0, 0 vtnet_tx_hdr: 24, 0, 0, 0, 0, 0, 0, 0 cryptop: 280, 0, 0, 0,18245763, 0, 0, 0 VNODE: 600, 0, 619504, 2078,163490504, 0, 0, 0 BUF TRIE: 144, 0, 258, 5063,77164032, 0, 0, 0 S VFS Cache: 104, 0, 19218, 6136,89521403, 0, 0, 0 STS VFS Cache: 144, 0, 0, 0, 0, 0, 0, 0 L VFS Cache: 320, 0, 4, 30, 34, 0, 0, 0 LTS VFS Cache: 360, 0, 0, 0, 0, 0, 0, 0 NAMEI: 1024, 0, 7293, 238,2287491432, 0, 0, 0 TMPFS node: 224, 0, 0, 2, 2, 0, 0, 0 DIRHASH: 1024, 0, 6979, 208, 2827104, 0, 0, 0 NCLNODE: 736, 0, 0, 0, 0, 0, 0, 0 AIO: 208, 0, 0, 0, 0, 0, 0, 0 AIOP: 32, 0, 0, 0, 0, 0, 0, 0 AIOCB: 552, 0, 0, 0, 0, 0, 0, 0 AIOLIO: 272, 0, 0, 0, 0, 0, 0, 0 pipe: 744, 0, 52, 209, 668571, 0, 0, 0 Mountpoints: 2944, 0, 5, 16, 240, 0, 0, 0 ksiginfo: 112, 0, 1988, 1258, 42172, 0, 0, 0 itimer: 352, 0, 0, 0, 0, 0, 0, 0 KNOTE: 160, 0, 0, 0, 403727, 0, 0, 0 socket: 944, 1044032, 52, 48, 17150, 0, 0, 0 ipq: 56, 51200, 0, 0, 0, 0, 0, 0 udp_inpcb: 488, 1044032, 22, 30, 852, 0, 0, 0 udpcb: 32, 1044032, 22, 254, 852, 0, 0, 0 tcp_inpcb: 488, 1044032, 11, 0, 144, 0, 0, 0 tcpcb: 1048, 1044032, 11, 0, 144, 0, 0, 0 tcptw: 88, 27767, 0, 0, 60, 0, 0, 0 syncache: 168, 15360, 0, 0, 63, 0, 0, 0 hostcache: 96, 15360, 1, 0, 3, 0, 0, 0 sackhole: 32, 0, 0, 0, 0, 0, 0, 0 tfo: 4, 0, 0, 0, 0, 0, 0, 0 tfo_ccache_entries: 80, 0, 0, 0, 0, 0, 0, 0 tcpreass: 48, 126939, 0, 0, 0, 0, 0, 0 tcp_log: 416, 5000000, 0, 0, 0, 0, 0, 0 tcp_log_bucket: 176, 0, 0, 0, 0, 0, 0, 0 tcp_log_node: 120, 0, 0, 0, 0, 0, 0, 0 udplite_inpcb: 488, 1044032, 0, 0, 0, 0, 0, 0 ripcb: 488, 1044032, 1, 0, 3, 0, 0, 0 IPsec SA lft_c: 16, 0, 0, 0, 0, 0, 0, 0 unpcb: 256, 1044032, 16, 119, 16081, 0, 0, 0 routing nhops: 256, 0, 9, 0, 16, 0, 0, 0 rtentry: 176, 0, 12, 0, 16, 0, 0, 0 swpctrie: 144, 4062072, 5844, 393, 367373, 0, 0, 0 swblk: 136, 4062059, 63999, 857, 4932681, 0, 0, 0 FFS inode: 1160, 0, 619482, 802,117624159, 0, 0, 0 FFS1 dinode: 128, 0, 0, 0, 0, 0, 0, 0 FFS2 dinode: 256, 0, 619475, 880,117445935, 0, 0, 0 mqnode: 416, 0, 3, 0, 3, 0, 0, 0 mqueue: 248, 0, 0, 0, 0, 0, 0, 0 mvdata: 64, 0, 0, 0, 0, 0, 0, 0 mqnotifier: 216, 0, 0, 0, 0, 0, 0, 0 smrs: 8, 0, 0, 41,20000001, 0, 0, 0 diff --git a/sys/ufs/ffs/ffs_extern.h b/sys/ufs/ffs/ffs_extern.h index 544012089046..62486941354e 100644 --- a/sys/ufs/ffs/ffs_extern.h +++ b/sys/ufs/ffs/ffs_extern.h @@ -127,6 +127,7 @@ int ffs_breadz(struct ufsmount *, struct vnode *, daddr_t, daddr_t, int, #define FFSV_FORCEINSMQ 0x0001 #define FFSV_REPLACE 0x0002 #define FFSV_REPLACE_DOOMED 0x0004 +#define FFSV_FORCEINODEDEP 0x0008 /* * Flags to ffs_reload diff --git a/sys/ufs/ffs/ffs_softdep.c b/sys/ufs/ffs/ffs_softdep.c index 786fb43c7d81..cd00181b3c21 100644 --- a/sys/ufs/ffs/ffs_softdep.c +++ b/sys/ufs/ffs/ffs_softdep.c @@ -1471,7 +1471,7 @@ get_parent_vp(struct vnode *vp, struct mount *mp, ino_t inum, struct buf *bp, ASSERT_VOP_ELOCKED(vp, "child vnode must be locked"); for (bplocked = true, pvp = NULL;;) { error = ffs_vgetf(mp, inum, LK_EXCLUSIVE | LK_NOWAIT, &pvp, - FFSV_FORCEINSMQ); + FFSV_FORCEINSMQ | FFSV_FORCEINODEDEP); if (error == 0) { /* * Since we could have unlocked vp, the inode @@ -1512,7 +1512,7 @@ get_parent_vp(struct vnode *vp, struct mount *mp, ino_t inum, struct buf *bp, VOP_UNLOCK(vp); error = ffs_vgetf(mp, inum, LK_EXCLUSIVE, &pvp, - FFSV_FORCEINSMQ); + FFSV_FORCEINSMQ | FFSV_FORCEINODEDEP); if (error != 0) { MPASS(error != ERELOOKUP); vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); @@ -1569,6 +1569,7 @@ softdep_flush(addr) struct mount *mp; struct thread *td; struct ufsmount *ump; + int cleanups; td = curthread; td->td_pflags |= TDP_NORUNNINGBUF; @@ -1603,10 +1604,14 @@ softdep_flush(addr) continue; } ump->softdep_flags &= ~FLUSH_EXIT; + cleanups = ump->um_softdep->sd_cleanups; FREE_LOCK(ump); wakeup(&ump->softdep_flags); - if (print_threads) - printf("Stop thread %s: searchfailed %d, did cleanups %d\n", td->td_name, searchfailed, ump->um_softdep->sd_cleanups); + if (print_threads) { + printf("Stop thread %s: searchfailed %d, " + "did cleanups %d\n", + td->td_name, searchfailed, cleanups); + } atomic_subtract_int(&stat_flush_threads, 1); kthread_exit(); panic("kthread_exit failed\n"); @@ -1792,10 +1797,10 @@ softdep_process_worklist(mp, full) long starttime; KASSERT(mp != NULL, ("softdep_process_worklist: NULL mp")); - if (MOUNTEDSOFTDEP(mp) == 0) + ump = VFSTOUFS(mp); + if (ump->um_softdep == NULL) return (0); matchcnt = 0; - ump = VFSTOUFS(mp); ACQUIRE_LOCK(ump); starttime = time_second; softdep_process_journal(mp, NULL, full ? MNT_WAIT : 0); @@ -2134,6 +2139,8 @@ softdep_waitidle(struct mount *mp, int flags __unused) int error, i; ump = VFSTOUFS(mp); + KASSERT(ump->um_softdep != NULL, + ("softdep_waitidle called on non-softdep filesystem")); devvp = ump->um_devvp; td = curthread; error = 0; @@ -2171,14 +2178,15 @@ softdep_flushfiles(oldmnt, flags, td) int flags; struct thread *td; { -#ifdef QUOTA struct ufsmount *ump; +#ifdef QUOTA int i; #endif int error, early, depcount, loopcnt, retry_flush_count, retry; int morework; - KASSERT(MOUNTEDSOFTDEP(oldmnt) != 0, + ump = VFSTOUFS(oldmnt); + KASSERT(ump->um_softdep != NULL, ("softdep_flushfiles called on non-softdep filesystem")); loopcnt = 10; retry_flush_count = 3; @@ -2222,7 +2230,6 @@ softdep_flushfiles(oldmnt, flags, td) MNT_ILOCK(oldmnt); morework = oldmnt->mnt_nvnodelistsize > 0; #ifdef QUOTA - ump = VFSTOUFS(oldmnt); UFS_LOCK(ump); for (i = 0; i < MAXQUOTAS; i++) { if (ump->um_quotas[i] != NULLVP) @@ -2672,50 +2679,52 @@ softdep_mount(devvp, mp, fs, cred) u_int cyl, i; int error; + ump = VFSTOUFS(mp); + sdp = malloc(sizeof(struct mount_softdeps), M_MOUNTDATA, M_WAITOK | M_ZERO); - MNT_ILOCK(mp); - mp->mnt_flag = (mp->mnt_flag & ~MNT_ASYNC) | MNT_SOFTDEP; - if ((mp->mnt_kern_flag & MNTK_SOFTDEP) == 0) { - mp->mnt_kern_flag = (mp->mnt_kern_flag & ~MNTK_ASYNC) | - MNTK_SOFTDEP | MNTK_NOASYNC; - } - ump = VFSTOUFS(mp); - ump->um_softdep = sdp; - MNT_IUNLOCK(mp); - rw_init(LOCK_PTR(ump), "per-fs softdep"); + rw_init(&sdp->sd_fslock, "SUrw"); sdp->sd_ump = ump; - LIST_INIT(&ump->softdep_workitem_pending); - LIST_INIT(&ump->softdep_journal_pending); - TAILQ_INIT(&ump->softdep_unlinked); - LIST_INIT(&ump->softdep_dirtycg); - ump->softdep_worklist_tail = NULL; - ump->softdep_on_worklist = 0; - ump->softdep_deps = 0; - LIST_INIT(&ump->softdep_mkdirlisthd); - ump->pagedep_hashtbl = hashinit(desiredvnodes / 5, M_PAGEDEP, - &ump->pagedep_hash_size); - ump->pagedep_nextclean = 0; - ump->inodedep_hashtbl = hashinit(desiredvnodes, M_INODEDEP, - &ump->inodedep_hash_size); - ump->inodedep_nextclean = 0; - ump->newblk_hashtbl = hashinit(max_softdeps / 2, M_NEWBLK, - &ump->newblk_hash_size); - ump->bmsafemap_hashtbl = hashinit(1024, M_BMSAFEMAP, - &ump->bmsafemap_hash_size); + LIST_INIT(&sdp->sd_workitem_pending); + LIST_INIT(&sdp->sd_journal_pending); + TAILQ_INIT(&sdp->sd_unlinked); + LIST_INIT(&sdp->sd_dirtycg); + sdp->sd_worklist_tail = NULL; + sdp->sd_on_worklist = 0; + sdp->sd_deps = 0; + LIST_INIT(&sdp->sd_mkdirlisthd); + sdp->sd_pdhash = hashinit(desiredvnodes / 5, M_PAGEDEP, + &sdp->sd_pdhashsize); + sdp->sd_pdnextclean = 0; + sdp->sd_idhash = hashinit(desiredvnodes, M_INODEDEP, + &sdp->sd_idhashsize); + sdp->sd_idnextclean = 0; + sdp->sd_newblkhash = hashinit(max_softdeps / 2, M_NEWBLK, + &sdp->sd_newblkhashsize); + sdp->sd_bmhash = hashinit(1024, M_BMSAFEMAP, &sdp->sd_bmhashsize); i = 1 << (ffs(desiredvnodes / 10) - 1); - ump->indir_hashtbl = malloc(i * sizeof(struct indir_hashhead), + sdp->sd_indirhash = malloc(i * sizeof(struct indir_hashhead), M_FREEWORK, M_WAITOK); - ump->indir_hash_size = i - 1; - for (i = 0; i <= ump->indir_hash_size; i++) - TAILQ_INIT(&ump->indir_hashtbl[i]); + sdp->sd_indirhashsize = i - 1; + for (i = 0; i <= sdp->sd_indirhashsize; i++) + TAILQ_INIT(&sdp->sd_indirhash[i]); #ifdef INVARIANTS for (i = 0; i <= D_LAST; i++) - LIST_INIT(&ump->softdep_alldeps[i]); + LIST_INIT(&sdp->sd_alldeps[i]); #endif ACQUIRE_GBLLOCK(&lk); TAILQ_INSERT_TAIL(&softdepmounts, sdp, sd_next); FREE_GBLLOCK(&lk); + + ump->um_softdep = sdp; + MNT_ILOCK(mp); + mp->mnt_flag = (mp->mnt_flag & ~MNT_ASYNC) | MNT_SOFTDEP; + if ((mp->mnt_kern_flag & MNTK_SOFTDEP) == 0) { + mp->mnt_kern_flag = (mp->mnt_kern_flag & ~MNTK_ASYNC) | + MNTK_SOFTDEP | MNTK_NOASYNC; + } + MNT_IUNLOCK(mp); + if ((fs->fs_flags & FS_SUJ) && (error = journal_mount(mp, fs, cred)) != 0) { printf("Failed to start journal: %d\n", error); @@ -2776,16 +2785,14 @@ softdep_unmount(mp) struct mount *mp; { struct ufsmount *ump; -#ifdef INVARIANTS - int i; -#endif + struct mount_softdeps *ums; - KASSERT(MOUNTEDSOFTDEP(mp) != 0, - ("softdep_unmount called on non-softdep filesystem")); ump = VFSTOUFS(mp); + KASSERT(ump->um_softdep != NULL, + ("softdep_unmount called on non-softdep filesystem")); MNT_ILOCK(mp); mp->mnt_flag &= ~MNT_SOFTDEP; - if (MOUNTEDSUJ(mp) == 0) { + if ((mp->mnt_flag & MNT_SUJ) == 0) { MNT_IUNLOCK(mp); } else { mp->mnt_flag &= ~MNT_SUJ; @@ -2800,35 +2807,45 @@ softdep_unmount(mp) ACQUIRE_LOCK(ump); ump->softdep_flags |= FLUSH_EXIT; wakeup(&ump->softdep_flushtd); - msleep(&ump->softdep_flags, LOCK_PTR(ump), PVM | PDROP, - "sdwait", 0); + while ((ump->softdep_flags & FLUSH_EXIT) != 0) { + msleep(&ump->softdep_flags, LOCK_PTR(ump), PVM, + "sdwait", 0); + } KASSERT((ump->softdep_flags & FLUSH_EXIT) == 0, ("Thread shutdown failed")); + FREE_LOCK(ump); } + /* - * Free up our resources. + * We are no longer have softdep structure attached to ump. */ + ums = ump->um_softdep; ACQUIRE_GBLLOCK(&lk); - TAILQ_REMOVE(&softdepmounts, ump->um_softdep, sd_next); + TAILQ_REMOVE(&softdepmounts, ums, sd_next); FREE_GBLLOCK(&lk); - rw_destroy(LOCK_PTR(ump)); - hashdestroy(ump->pagedep_hashtbl, M_PAGEDEP, ump->pagedep_hash_size); - hashdestroy(ump->inodedep_hashtbl, M_INODEDEP, ump->inodedep_hash_size); - hashdestroy(ump->newblk_hashtbl, M_NEWBLK, ump->newblk_hash_size); - hashdestroy(ump->bmsafemap_hashtbl, M_BMSAFEMAP, - ump->bmsafemap_hash_size); - free(ump->indir_hashtbl, M_FREEWORK); + ump->um_softdep = NULL; + + /* + * Free up our resources. + */ + rw_destroy(&ums->sd_fslock); + hashdestroy(ums->sd_pdhash, M_PAGEDEP, ums->sd_pdhashsize); + hashdestroy(ums->sd_idhash, M_INODEDEP, ums->sd_idhashsize); + hashdestroy(ums->sd_newblkhash, M_NEWBLK, ums->sd_newblkhashsize); + hashdestroy(ums->sd_bmhash, M_BMSAFEMAP, ums->sd_bmhashsize); + free(ums->sd_indirhash, M_FREEWORK); #ifdef INVARIANTS - for (i = 0; i <= D_LAST; i++) { - KASSERT(ump->softdep_curdeps[i] == 0, + for (int i = 0; i <= D_LAST; i++) { + KASSERT(ums->sd_curdeps[i] == 0, ("Unmount %s: Dep type %s != 0 (%ld)", ump->um_fs->fs_fsmnt, - TYPENAME(i), ump->softdep_curdeps[i])); - KASSERT(LIST_EMPTY(&ump->softdep_alldeps[i]), - ("Unmount %s: Dep type %s not empty (%p)", ump->um_fs->fs_fsmnt, - TYPENAME(i), LIST_FIRST(&ump->softdep_alldeps[i]))); + TYPENAME(i), ums->sd_curdeps[i])); + KASSERT(LIST_EMPTY(&ums->sd_alldeps[i]), + ("Unmount %s: Dep type %s not empty (%p)", + ump->um_fs->fs_fsmnt, + TYPENAME(i), LIST_FIRST(&ums->sd_alldeps[i]))); } #endif - free(ump->um_softdep, M_MOUNTDATA); + free(ums, M_MOUNTDATA); } static struct jblocks * @@ -3016,26 +3033,25 @@ journal_mount(mp, fs, cred) jblocks->jb_low = jblocks->jb_free / 3; /* Reserve 33%. */ jblocks->jb_min = jblocks->jb_free / 10; /* Suspend at 10%. */ ump->softdep_jblocks = jblocks; -out: - if (error == 0) { - MNT_ILOCK(mp); - mp->mnt_flag |= MNT_SUJ; - mp->mnt_flag &= ~MNT_SOFTDEP; - MNT_IUNLOCK(mp); - /* - * Only validate the journal contents if the - * filesystem is clean, otherwise we write the logs - * but they'll never be used. If the filesystem was - * still dirty when we mounted it the journal is - * invalid and a new journal can only be valid if it - * starts from a clean mount. - */ - if (fs->fs_clean) { - DIP_SET(ip, i_modrev, fs->fs_mtime); - ip->i_flags |= IN_MODIFIED; - ffs_update(vp, 1); - } + + MNT_ILOCK(mp); + mp->mnt_flag |= MNT_SUJ; + MNT_IUNLOCK(mp); + + /* + * Only validate the journal contents if the + * filesystem is clean, otherwise we write the logs + * but they'll never be used. If the filesystem was + * still dirty when we mounted it the journal is + * invalid and a new journal can only be valid if it + * starts from a clean mount. + */ + if (fs->fs_clean) { + DIP_SET(ip, i_modrev, fs->fs_mtime); + ip->i_flags |= IN_MODIFIED; + ffs_update(vp, 1); } +out: vput(vp); return (error); } @@ -3702,12 +3718,12 @@ softdep_process_journal(mp, needwk, flags) int off; int devbsize; - if (MOUNTEDSUJ(mp) == 0) + ump = VFSTOUFS(mp); + if (ump->um_softdep == NULL || ump->um_softdep->sd_jblocks == NULL) return; shouldflush = softdep_flushcache; bio = NULL; jseg = NULL; - ump = VFSTOUFS(mp); LOCK_OWNED(ump); fs = ump->um_fs; jblocks = ump->softdep_jblocks; @@ -8387,7 +8403,7 @@ handle_complete_freeblocks(freeblks, flags) */ if (spare && freeblks->fb_len != 0) { if (ffs_vgetf(freeblks->fb_list.wk_mp, freeblks->fb_inum, - flags, &vp, FFSV_FORCEINSMQ) != 0) + flags, &vp, FFSV_FORCEINSMQ | FFSV_FORCEINODEDEP) != 0) return (EBUSY); ip = VTOI(vp); if (ip->i_mode == 0) { @@ -10177,7 +10193,8 @@ handle_workitem_remove(dirrem, flags) mp = dirrem->dm_list.wk_mp; ump = VFSTOUFS(mp); flags |= LK_EXCLUSIVE; - if (ffs_vgetf(mp, oldinum, flags, &vp, FFSV_FORCEINSMQ) != 0) + if (ffs_vgetf(mp, oldinum, flags, &vp, FFSV_FORCEINSMQ | + FFSV_FORCEINODEDEP) != 0) return (EBUSY); ip = VTOI(vp); MPASS(ip->i_mode != 0); @@ -14196,7 +14213,8 @@ check_clear_deps(mp) * causes deferred work to be done sooner. */ ump = VFSTOUFS(mp); - suj_susp = MOUNTEDSUJ(mp) && ump->softdep_jblocks->jb_suspended; + suj_susp = ump->um_softdep->sd_jblocks != NULL && + ump->softdep_jblocks->jb_suspended; if (req_clear_remove || req_clear_inodedeps || suj_susp) { FREE_LOCK(ump); softdep_send_speedup(ump, 0, BIO_SPEEDUP_TRIM | BIO_SPEEDUP_WRITE); @@ -14269,7 +14287,7 @@ clear_remove(mp) if (error != 0) goto finish_write; error = ffs_vgetf(mp, ino, LK_EXCLUSIVE, &vp, - FFSV_FORCEINSMQ); + FFSV_FORCEINSMQ | FFSV_FORCEINODEDEP); vfs_unbusy(mp); if (error != 0) { softdep_error("clear_remove: vget", error); @@ -14349,7 +14367,7 @@ clear_inodedeps(mp) return; } if ((error = ffs_vgetf(mp, ino, LK_EXCLUSIVE, &vp, - FFSV_FORCEINSMQ)) != 0) { + FFSV_FORCEINSMQ | FFSV_FORCEINODEDEP)) != 0) { softdep_error("clear_inodedeps: vget", error); vfs_unbusy(mp); vn_finished_write(mp); diff --git a/sys/ufs/ffs/ffs_vfsops.c b/sys/ufs/ffs/ffs_vfsops.c index 762874562082..321ed03f7f67 100644 --- a/sys/ufs/ffs/ffs_vfsops.c +++ b/sys/ufs/ffs/ffs_vfsops.c @@ -380,6 +380,7 @@ ffs_mount(struct mount *mp) accmode_t accmode; struct nameidata ndp; char *fspec; + bool mounted_softdep; td = curthread; if (vfs_filteropt(mp->mnt_optnew, ffs_opts)) @@ -491,6 +492,16 @@ ffs_mount(struct mount *mp) error = vfs_write_suspend_umnt(mp); if (error != 0) return (error); + + fs->fs_ronly = 1; + if (MOUNTEDSOFTDEP(mp)) { + MNT_ILOCK(mp); + mp->mnt_flag &= ~MNT_SOFTDEP; + MNT_IUNLOCK(mp); + mounted_softdep = true; + } else + mounted_softdep = false; + /* * Check for and optionally get rid of files open * for writing. @@ -498,15 +509,22 @@ ffs_mount(struct mount *mp) flags = WRITECLOSE; if (mp->mnt_flag & MNT_FORCE) flags |= FORCECLOSE; - if (MOUNTEDSOFTDEP(mp)) { + if (mounted_softdep) { error = softdep_flushfiles(mp, flags, td); } else { error = ffs_flushfiles(mp, flags, td); } if (error) { + fs->fs_ronly = 0; + if (mounted_softdep) { + MNT_ILOCK(mp); + mp->mnt_flag |= MNT_SOFTDEP; + MNT_IUNLOCK(mp); + } vfs_write_resume(mp, 0); return (error); } + if (fs->fs_pendingblocks != 0 || fs->fs_pendinginodes != 0) { printf("WARNING: %s Update error: blocks %jd " @@ -521,10 +539,15 @@ ffs_mount(struct mount *mp) if ((error = ffs_sbupdate(ump, MNT_WAIT, 0)) != 0) { fs->fs_ronly = 0; fs->fs_clean = 0; + if (mounted_softdep) { + MNT_ILOCK(mp); + mp->mnt_flag |= MNT_SOFTDEP; + MNT_IUNLOCK(mp); + } vfs_write_resume(mp, 0); return (error); } - if (MOUNTEDSOFTDEP(mp)) + if (mounted_softdep) softdep_unmount(mp); g_topology_lock(); /* @@ -532,7 +555,6 @@ ffs_mount(struct mount *mp) */ g_access(ump->um_cp, 0, -1, -1); g_topology_unlock(); - fs->fs_ronly = 1; MNT_ILOCK(mp); mp->mnt_flag |= MNT_RDONLY; MNT_IUNLOCK(mp); @@ -623,6 +645,8 @@ ffs_mount(struct mount *mp) fs->fs_clean = 0; if ((error = ffs_sbupdate(ump, MNT_WAIT, 0)) != 0) { fs->fs_ronly = 1; + if ((fs->fs_flags & FS_DOSOFTDEP) != 0) + softdep_unmount(mp); MNT_ILOCK(mp); mp->mnt_flag |= saved_mnt_flag; MNT_IUNLOCK(mp); @@ -1331,6 +1355,7 @@ ffs_mountfs(odevvp, mp, td) free(mp->mnt_gjprovider, M_UFSMNT); mp->mnt_gjprovider = NULL; } + MPASS(ump->um_softdep == NULL); free(ump, M_UFSMNT); mp->mnt_data = NULL; } @@ -1513,6 +1538,7 @@ ffs_unmount(mp, mntflags) UFS_UNLOCK(ump); if (MOUNTEDSOFTDEP(mp)) softdep_unmount(mp); + MPASS(ump->um_softdep == NULL); if (fs->fs_ronly == 0 || ump->um_fsckpid > 0) { fs->fs_clean = fs->fs_flags & (FS_UNCLEAN|FS_NEEDSFSCK) ? 0 : 1; error = ffs_sbupdate(ump, MNT_WAIT, 0); @@ -2074,7 +2100,8 @@ ffs_vgetf(mp, ino, flags, vpp, ffs_flags) *vpp = NULL; return (error); } - if (DOINGSOFTDEP(vp)) + if (DOINGSOFTDEP(vp) && (!fs->fs_ronly || + (ffs_flags & FFSV_FORCEINODEDEP) != 0)) softdep_load_inodeblock(ip); else ip->i_effnlink = ip->i_nlink; diff --git a/sys/ufs/ufs/inode.h b/sys/ufs/ufs/inode.h index 0169d2903108..a82b4658c0af 100644 --- a/sys/ufs/ufs/inode.h +++ b/sys/ufs/ufs/inode.h @@ -265,11 +265,11 @@ struct indir { #define ITOV(ip) ((ip)->i_vnode) /* Determine if soft dependencies are being done */ -#define DOINGSOFTDEP(vp) \ - (((vp)->v_mount->mnt_flag & (MNT_SOFTDEP | MNT_SUJ)) != 0) -#define MOUNTEDSOFTDEP(mp) (((mp)->mnt_flag & (MNT_SOFTDEP | MNT_SUJ)) != 0) -#define DOINGSUJ(vp) (((vp)->v_mount->mnt_flag & MNT_SUJ) != 0) -#define MOUNTEDSUJ(mp) (((mp)->mnt_flag & MNT_SUJ) != 0) +#define MOUNTEDSOFTDEP(mp) (((mp)->mnt_flag & MNT_SOFTDEP) != 0) +#define DOINGSOFTDEP(vp) MOUNTEDSOFTDEP((vp)->v_mount) +#define MOUNTEDSUJ(mp) (((mp)->mnt_flag & (MNT_SOFTDEP | MNT_SUJ)) == \ + (MNT_SOFTDEP | MNT_SUJ)) +#define DOINGSUJ(vp) MOUNTEDSUJ((vp)->v_mount) /* This overlays the fid structure (see mount.h). */ struct ufid { -- Test scenario: misc/crossmp4.sh