watchdogd fired with a timeout of 3600s, vmcoee.128 KDB: debugger backends: ddb KDB: current backend: ddb ---<>--- Copyright (c) 1992-2019 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 13.0-CURRENT #0 r355582M: Tue Dec 10 17:27:26 CET 2019 pho@mercat1.netperf.freebsd.org:/usr/src/sys/amd64/compile/PHO amd64 FreeBSD clang version 9.0.0 (tags/RELEASE_900/final 372316) (based on LLVM 9.0.0) WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. Entering uma_startup with 9 boot pages configured startup_alloc from "UMA Kegs", 7 boot pages left startup_alloc from "UMA Zones", 6 boot pages left startup_alloc from "UMA Zones", 5 boot pages left startup_alloc from "UMA Zones", 4 boot pages left startup_alloc from "UMA Zones", 3 boot pages left startup_alloc from "UMA Zones", 2 boot pages left startup_alloc from "UMA Zones", 1 boot pages left Entering uma_startup1 with 0 boot pages left Entering uma_startup2 with 0 boot pages left VT(vga): resolution 640x480 CPU: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz (3500.08-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306f2 Family=0x6 Model=0x3f Stepping=2 Features=0xbfebfbff Features2=0x7ffefbff AMD Features=0x2c100800 AMD Features2=0x21 Structured Extended Features=0x37ab XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics real memory = 34359738368 (32768 MB) avail memory = 33164615680 (31628 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs FreeBSD/SMP: 1 package(s) x 6 core(s) x 2 hardware threads random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" random: unblocking device. ioapic0 irqs 0-23 ioapic1 irqs 24-47 Launching APs: 1 2 11 4 8 5 6 3 7 10 9 Timecounter "TSC-low" frequency 1750037618 Hz quality 1000 random: entropy device external interface WARNING: Device "kbd" is Giant locked and may be deleted before FreeBSD 13.0. kbd1 at kbdmux0 000.000055 [4336] netmap_init netmap: loaded module [ath_hal] loaded nexus0 vtvga0: cryptosoft0: acpi0: acpi0: Power Button (fixed) cpu0: on acpi0 atrtc0: port 0x70-0x71,0x74-0x77 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Event timer "HPET4" frequency 14318180 Hz quality 340 Event timer "HPET5" frequency 14318180 Hz quality 340 Event timer "HPET6" frequency 14318180 Hz quality 340 Event timer "HPET7" frequency 14318180 Hz quality 340 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: on acpi0 pci0: on pcib0 pci0: at device 11.1 (no driver attached) pci0: at device 11.2 (no driver attached) pci0: at device 16.1 (no driver attached) pci0: at device 16.6 (no driver attached) pci0: at device 18.1 (no driver attached) acpi_syscontainer0: on acpi0 acpi_syscontainer1: on acpi0 acpi_syscontainer2: on acpi0 acpi_syscontainer3: on acpi0 pcib1: port 0xcf8-0xcff on acpi0 pci1: on pcib1 pcib2: irq 26 at device 1.0 on pci1 pci2: on pcib2 pcib3: irq 32 at device 2.0 on pci1 pci3: on pcib3 pci3: at device 0.0 (no driver attached) pci3: at device 0.1 (no driver attached) pcib4: irq 40 at device 3.0 on pci1 pci4: on pcib4 pci1: at device 17.0 (no driver attached) ahci0: port 0xf110-0xf117,0xf100-0xf103,0xf0f0-0xf0f7,0xf0e0-0xf0e3,0xf020-0xf03f mem 0xfb438000-0xfb4387ff irq 16 at device 17.4 on pci1 ahci0: AHCI v1.30 with 4 6Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahciem0: on ahci0 xhci0: mem 0xfb400000-0xfb40ffff irq 19 at device 20.0 on pci1 xhci0: 32 bytes context size, 64-bit DMA usbus0 on xhci0 usbus0: 5.0Gbps Super Speed USB v3.0 pci1: at device 22.0 (no driver attached) pci1: at device 22.1 (no driver attached) ehci0: mem 0xfb434000-0xfb4343ff irq 18 at device 26.0 on pci1 usbus1: EHCI version 1.0 usbus1 on ehci0 usbus1: 480Mbps High Speed USB v2.0 pcib5: irq 16 at device 28.0 on pci1 pci5: on pcib5 pcib6: irq 18 at device 28.2 on pci1 pci6: on pcib6 pcib7: at device 0.0 on pci6 pci7: on pcib7 vgapci0: port 0xe000-0xe07f mem 0xfa000000-0xfaffffff,0xfb000000-0xfb01ffff irq 18 at device 0.0 on pci7 vgapci0: Boot video device pcib8: irq 16 at device 28.4 on pci1 pci8: on pcib8 igb0: port 0xd020-0xd03f mem 0xfb320000-0xfb33ffff,0xfb344000-0xfb347fff irq 16 at device 0.0 on pci8 igb0: Using 1024 TX descriptors and 1024 RX descriptors igb0: queue equality override not set, capping rx_queues at 6 and tx_queues at 6 igb0: Using 6 RX queues 6 TX queues igb0: Using MSI-X interrupts with 7 vectors igb0: Ethernet address: 0c:c4:7a:a8:cd:da igb0: netmap queues/slots: TX 6/1024, RX 6/1024 igb1: port 0xd000-0xd01f mem 0xfb300000-0xfb31ffff,0xfb340000-0xfb343fff irq 17 at device 0.1 on pci8 igb1: Using 1024 TX descriptors and 1024 RX descriptors igb1: queue equality override not set, capping rx_queues at 6 and tx_queues at 6 igb1: Using 6 RX queues 6 TX queues igb1: Using MSI-X interrupts with 7 vectors igb1: Ethernet address: 0c:c4:7a:a8:cd:db igb1: netmap queues/slots: TX 6/1024, RX 6/1024 ehci1: mem 0xfb433000-0xfb4333ff irq 18 at device 29.0 on pci1 usbus2: EHCI version 1.0 usbus2 on ehci1 usbus2: 480Mbps High Speed USB v2.0 isab0: at device 31.0 on pci1 isa0: on isab0 ahci1: port 0xf070-0xf077,0xf060-0xf063,0xf050-0xf057,0xf040-0xf043,0xf000-0xf01f mem 0xfb432000-0xfb4327ff irq 16 at device 31.2 on pci1 ahci1: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported ahcich4: at channel 0 on ahci1 ahcich5: at channel 1 on ahci1 ahcich6: at channel 2 on ahci1 ahcich7: at channel 3 on ahci1 ahcich8: at channel 4 on ahci1 ahcich9: at channel 5 on ahci1 ahciem1: on ahci1 acpi_button0: on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 uart1: console (115200,n,8,1) orm0: at iomem 0xc0000-0xc7fff,0xcb000-0xcbfff pnpid ORM0000 on isa0 est0: on cpu0 p4tcc0: on cpu0 NULL mp in getnewvnode(9), tag crossmp Timecounters tick every 1.000 msec Expensive timeout(9) function: 0xffffffff804d97a0(0xfffffe00d1352000) 0.004206652 s ses0 at ahciem0 bus 0 scbus4 target 0 lun 0 ses0: SEMB S-E-S 2.00 device ses0: SEMB SES Device ses1 at ahciem1 bus 0 scbus11 target 0 lun 0 ses1: SEMB S-E-S 2.00 device ses1: SEMB SES Device ses0: pass0 in 'Slot 01', SATA Slot: scbus1 target 0 ada0 at ahcich1 bus 0 scbus1 target 0 lun 0 ada0: ACS-2 ATA SATA 3.x dev3KDDP98 ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 953869MB (1953525168 512 byte sectors) ada0: quirks=0x1<4K> Expensive timeout(9) function: 0xffffffff80a34dc0(0xffffffff818e6608) 0.007385588 s hwpmc: SOFT/16/64/0x67 TSC/1/64/0x20 IAP/4/48/0x3ff IAF/3/48/0x67 Trying to mount root from ufs:/dev/gpt/work [rw]... WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. ugen0.1: <0x8086 XHCI root HUB> at usbus0 ugen1.1: at usbus1 ugen2.1: at usbus2 uhub0 on usbus0 uhub0: <0x8086 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0 uhub1 on usbus2 uhub1: on usbus2 uhub2 on usbus1 uhub2: on usbus1 WARNING: / was not properly dismounted Interface IP-Address Broadcast . Autoloading module: ioat.ko ioat0: mem 0xfb42c000-0xfb42ffff irq 31 at device 4.0 on pci1 ioat0: Capabilities: 2f7 ioat1: mem 0xfb428000-0xfb42bfff irq 39 at device 4.1 on pci1 ioat1: Capabilities: 2f7 ioat2: mem 0xfb424000-0xfb427fff irq 31 at device 4.2 on pci1 ioat2: Capabilities: f7 ioat3: mem 0xfb420000-0xfb423fff irq 39 at device 4.3 on pci1 ioat3: Capabilities: f7 ioat4: mem 0xfb41c000-0xfb41ffff irq 31 at device 4.4 on pci1 ioat4: Capabilities: f7 ioat5: mem 0xfb418000-0xfb41bfff irq 39 at device 4.5 on pci1 ioat5: Capabilities: f7 ioat6: mem 0xfb414000-0xfb417fff irq 31 at device 4.6 on pci1 ioat6: Capabilities: f7 ioat7: mem 0xfb410000-0xfb413fff irq 39 at device 4.7 on pci1 ioat7: Capabilities: f7 Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Starting Network: igb1. igb1: flags=8802 metric 0 mtu 1500 options=4e507bb ether 0c:c4:7a:a8:cd:db media: Ethernet autoselect status: no carrier nd6 options=29 Autoloading module: uhid.ko Autoloading module: ums.ko ums0 on uhub3 ums0: on usbus0 ums0: 3 buttons and [Z] coordinates ID=0 Starting dhclient. if_delmulti_locked: detaching ifnet instance 0xfffff80005063800 DHCPDISCOVER on igb0 to 255.255.255.255 port 67 interval 5 DHCPOFFER from 192.168.5.1 DHCPREQUEST on igb0 to 255.255.255.255 port 67 DHCPACK from 192.168.5.1 bound to 192.168.5.171 -- renewal in 300 seconds. add host 127.0.0.1: gateway lo0 fib 0: route already in table add host ::1: gateway lo0 fib 0: route already in table add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Creating and/or trimming log files. Starting syslogd. Setting date via ntp. 11 Dec 12:28:17 ntpdate[2075]: step time server 192.168.5.1 offset -1.075504 sec Starting rpcbind. NFS access cache time=60 Starting watchdogd. Dec 11 12:28:17 mercat1 savecore[2174]: reboot Clearing /tmp (X related). Starting statd. Starting lockd. Recovering vi editor sessions:. fscklock order reversal: 1st 0xfffff80030d5a2e0 ufs (ufs) @ kern/vfs_mount.c:1603 2nd 0xfffff80030a5c7d0 devfs (devfs) @ ufs/ffs/ffs_softdep.c:1927 stack backtrace: #0 0xffffffff80c32581 at witness_debugger+0x71 #1 0xffffffff80c322f6 at witness_checkorder+0xad6 #2 0xffffffff80b97be8 at lockmgr_lock_fast_path+0x178 #3 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #4 0xffffffff80cb2b75 at _vn_lock+0x65 #5 0xffffffff80ec3c0d at softdep_flushworklist+0x5d #6 0xffffffff80ee4c06 at ffs_sync+0x406 #7 0xffffffff80c96cdb at dounmount+0x45b #8 0xffffffff80c9682e at sys_unmount+0x2ae #9 0xffffffff8106b901 at amd64_syscall+0x2f1 #10 0xffffffff810416c0 at fast_syscall_common+0x101 ** /dev/gpt/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 235 files, 25232 used, 1240175 free (159 frags, 155002 blocks, 0.0% fragmentation) ***** FILE SYSTEM MARKED CLEAN ***** Updating motd:. Mounting late filesystems:. Security policy loaded: MAC/ntpd (mac_ntpd) Starting ntpd. Starting powerd. Configuring vt: blanktime. Performing sanity check on sshd configuration. Starting sshd. Starting cron. Wed Dec 11 12:29 FreeBSD/amd64 (mercat1.netperf.freebsd.org) (ttyu1) login: root Dec 11 12:30:42 mercat1 login[2367]: ROOT LOGIN (root) ON ttyu1 Last login: Tue Dec 10 17:40:51 on ttyu1 FreeBSD 13.0-CURRENT (PHO) #0 r355582M: Tue Dec 10 17:27:26 CET 2019 v78root@mercat1:~ # i i: Command not found. root@mercat1:~ # :q :q: Command not found. root@mercat1:~ # Dec 11 12:30:54 mercat1 su[2388]: pho to root on /dev/pts/0 20191211 12:45:32 all (1/121): chain.sh swap_pager: out of swap space swp_pager_getswapspace(32): failed Dec 11 12:52:33 mercat1 kernel: pid 4124 (chain), jid 0, uid 0, was killed: out of swap space swap_pager: out of swap space swp_pager_getswapspace(18): failed Dec 11 12:52:34 mercat1 kernel: pid 2213 (rpc.statd), jid 0, uid 0, was killed: out of swap space Dec 11 12:52:36 mercat1 kernel: pid 2216 (rpc.lockd), jid 0, uid 0, was killed: out of swap space swap_pager: out of swap space swp_pager_getswapspace(16): failed Dec 11 12:52:38 mercat1 kernel: pid 1997 (devd), jid 0, uid 0, was killed: out of swap space swap_pager: out of swap space swp_pager_getswapspace(3): failed swp_pager_getswapspace(14): failed Dec 11 12:52:40 mercat1 kernel: pid 2277 (ntpd), jid 0, uid 123, was killed: out of swap space Dec 11 12:52:44 mercat1 kernel: pid 2390 (bash), jid 0, uid 0, was killed: out of swap space 20191211 12:52:53 all (2/121): vm_lowmem.sh 20191211 12:56:59 all (3/121): geli.sh GEOM_ELI: Device md10.eli created. GEOM_ELI: Encryption: AES-XTS 128 GEOM_ELI: Crypto: software GEOM_ELI: md10 has been killed. GEOM_ELI: Device md10.eli destroyed. newblk leak: 12697/12715. 20191211 13:02:25 all (4/121): crossmp3.sh lock order reversal: 1st 0xfffff805247ab2e0 syncer (syncer) @ kern/vfs_subr.c:2406 2nd 0xfffff808115adcc0 ufs (ufs) @ kern/vfs_subr.c:3042 stack backtrace: #0 0xffffffff80c32581 at witness_debugger+0x71 #1 0xffffffff80c322f6 at witness_checkorder+0xad6 #2 0xffffffff80b981cf at lockmgr_xlock_hard+0x6f #3 0xffffffff80b98d5d at __lockmgr_args+0x60d #4 0xffffffff80ee646c at ffs_lock+0x7c #5 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #6 0xffffffff80cb2b75 at _vn_lock+0x65 #7 0xffffffff80c9f8e4 at vputx+0x164 #8 0xffffffff80c9ecc3 at vget_finish+0xa3 #9 0xffffffff80ee4d41 at ffs_sync+0x541 #10 0xffffffff80ca759e at sync_fsync+0x11e #11 0xffffffff810e785a at VOP_FSYNC_APV+0xaa #12 0xffffffff80ca54fb at sched_sync+0x3cb #13 0xffffffff80b83060 at fork_exit+0x80 #14 0xffffffff81041dde at fork_trampoline+0xe Dec 11 13:09:12 mercat1 kernel: pid 37812 (creat), uid 2007 inumber 245131 on /mnt18: out of inodes Dec 11 13:09:29 mercat1 kernel: pid 37784 (rw), uid 2007 inumber 222853 on /mnt18: filesystem full 20191211 13:10:42 all (5/121): sem_post.sh Dec 11 13:11:46 mercat1 kernel: pid 44754 (swap), jid 0, uid 0, was killed: out of swap space Dec 11 13:12:56 mercat1 kernel: pid 44766 (swap), jid 0, uid 0, was killed: out of swap space 20191211 13:16:06 all (6/121): suj30.sh 20191211 13:23:10 all (7/121): crossmp4.sh 20191211 13:31:16 all (8/121): linger.sh 20191211 13:40:28 all (9/121): swap5.sh swp_pager_getswapspace(1): failed 20191211 13:49:43 all (10/121): syscall6.sh lock order reversal: (sleepable after non-sleepable) 1st 0xfffff80003a85808 seqflq (seqflq) @ dev/sound/midi/sequencer.c:757 2nd 0xffffffff81ae1230 midistat lock (midistat lock) @ dev/sound/midi/midi.c:1491 stack backtrace: #0 0xffffffff80c32581 at witness_debugger+0x71 #1 0xffffffff80c322f6 at witness_checkorder+0xad6 #2 0xffffffff80bce667 at _sx_xlock+0x67 #3 0xffffffff809830c2 at midimapper_open+0x22 #4 0xffffffff80985fd2 at mseq_open+0xf2 #5 0xffffffff80a7ac73 at devfs_open+0x123 #6 0xffffffff810e6494 at VOP_OPEN_APV+0xa4 #7 0xffffffff80cb2909 at vn_open_vnode+0x1a9 #8 0xffffffff80cb2249 at vn_open_cred+0x3f9 #9 0xffffffff80ca9d53 at kern_openat+0x1f3 #10 0xffffffff8106b901 at amd64_syscall+0x2f1 #11 0xffffffff810416c0 at fast_syscall_common+0x101 fsync: giving up on dirty (error = 35) 0xfffff80030a5c768: tag devfs, type VCHR usecount 4, writecount 0, refcount 12 rdev 0xfffff80030977c00 flags (VI_ACTIVE) v_object 0xfffff80030f49420 ref 0 pages 66 cleanbuf 8 dirtybuf 2 lock type devfs: EXCL by thread 0xfffff80836e02000 (pid 50760, syscall4, tid 101016) with exclusive waiters pending #0 0xffffffff80b97c50 at lockmgr_lock_fast_path+0x1e0 #1 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #2 0xffffffff80cb2b75 at _vn_lock+0x65 #3 0xffffffff80cae643 at kern_fsync+0x113 #4 0xffffffff8106b901 at amd64_syscall+0x2f1 #5 0xffffffff810416c0 at fast_syscall_common+0x101 dev gpt/tmp fsync: giving up on dirty (error = 35) 0xfffff80030a5c768: tag devfs, type VCHR usecount 4, writecount 0, refcount 12 rdev 0xfffff80030977c00 flags (VI_ACTIVE) v_object 0xfffff80030f49420 ref 0 pages 66 cleanbuf 8 dirtybuf 2 lock type devfs: EXCL by thread 0xfffff8059543d6e0 (pid 50760, syscall4, tid 101034) with exclusive waiters pending #0 0xffffffff80b9856a at lockmgr_xlock_hard+0x40a #1 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #2 0xffffffff80cb2b75 at _vn_lock+0x65 #3 0xffffffff80cae643 at kern_fsync+0x113 #4 0xffffffff8106b901 at amd64_syscall+0x2f1 #5 0xffffffff810416c0 at fast_syscall_common+0x101 dev gpt/tmp fsync: giving up on dirty (error = 35) 0xfffff80030a5c768: tag devfs, type VCHR usecount 4, writecount 0, refcount 12 rdev 0xfffff80030977c00 flags (VI_ACTIVE) v_object 0xfffff80030f49420 ref 0 pages 66 cleanbuf 8 dirtybuf 2 lock type devfs: EXCL by thread 0xfffff80139355000 (pid 50760, syscall4, tid 101043) with exclusive waiters pending #0 0xffffffff80b9856a at lockmgr_xlock_hard+0x40a #1 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #2 0xffffffff80cb2b75 at _vn_lock+0x65 #3 0xffffffff80cae643 at kern_fsync+0x113 #4 0xffffffff8106b901 at amd64_syscall+0x2f1 #5 0xffffffff810416c0 at fast_syscall_common+0x101 dev gpt/tmp fsync: giving up on dirty (error = 35) 0xfffff80030a5c768: tag devfs, type VCHR usecount 4, writecount 0, refcount 12 rdev 0xfffff80030977c00 flags (VI_ACTIVE) v_object 0xfffff80030f49420 ref 0 pages 66 cleanbuf 7 dirtybuf 3 lock type devfs: EXCL by thread 0xfffff8002ddf2000 (pid 50760, syscall4, tid 101045) with exclusive waiters pending #0 0xffffffff80b9856a at lockmgr_xlock_hard+0x40a #1 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #2 0xffffffff80cb2b75 at _vn_lock+0x65 #3 0xffffffff80cae643 at kern_fsync+0x113 #4 0xffffffff8106b901 at amd64_syscall+0x2f1 #5 0xffffffff810416c0 at fast_syscall_common+0x101 dev gpt/tmp fsync: giving up on dirty (error = 35) 0xfffff80030a5c768: tag devfs, type VCHR usecount 4, writecount 0, refcount 12 rdev 0xfffff80030977c00 flags (VI_ACTIVE) v_object 0xfffff80030f49420 ref 0 pages 66 cleanbuf 7 dirtybuf 3 lock type devfs: EXCL by thread 0xfffff801393556e0 (pid 50760, syscall4, tid 101042) with exclusive waiters pending #0 0xffffffff80b9856a at lockmgr_xlock_hard+0x40a #1 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #2 0xffffffff80cb2b75 at _vn_lock+0x65 #3 0xffffffff80cae643 at kern_fsync+0x113 #4 0xffffffff8106b901 at amd64_syscall+0x2f1 #5 0xffffffff810416c0 at fast_syscall_common+0x101 dev gpt/tmp fsync: giving up on dirty (error = 35) 0xfffff80030a5c768: tag devfs, type VCHR usecount 4, writecount 0, refcount 12 rdev 0xfffff80030977c00 flags (VI_ACTIVE) v_object 0xfffff80030f49420 ref 0 pages 66 cleanbuf 7 dirtybuf 3 lock type devfs: EXCL by thread 0xfffff8073b3e06e0 (pid 50760, syscall4, tid 101062) with exclusive waiters pending #0 0xffffffff80b9856a at lockmgr_xlock_hard+0x40a #1 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #2 0xffffffff80cb2b75 at _vn_lock+0x65 #3 0xffffffff80cae643 at kern_fsync+0x113 #4 0xffffffff8106b901 at amd64_syscall+0x2f1 #5 0xffffffff810416c0 at fast_syscall_common+0x101 dev gpt/tmp fsync: giving up on dirty (error = 35) 0xfffff80030a5c768: tag devfs, type VCHR usecount 4, writecount 0, refcount 12 rdev 0xfffff80030977c00 flags (VI_ACTIVE) v_object 0xfffff80030f49420 ref 0 pages 66 cleanbuf 9 dirtybuf 1 lock type devfs: EXCL by thread 0xfffff80273f7d6e0 (pid 50760, syscall4, tid 101032) with exclusive waiters pending #0 0xffffffff80b9856a at lockmgr_xlock_hard+0x40a #1 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #2 0xffffffff80cb2b75 at _vn_lock+0x65 #3 0xffffffff80cae643 at kern_fsync+0x113 #4 0xffffffff8106b901 at amd64_syscall+0x2f1 #5 0xffffffff810416c0 at fast_syscall_common+0x101 dev gpt/tmp fsync: giving up on dirty (error = 35) 0xfffff80030a5c768: tag devfs, type VCHR usecount 4, writecount 0, refcount 12 rdev 0xfffff80030977c00 flags (VI_ACTIVE) v_object 0xfffff80030f49420 ref 0 pages 66 cleanbuf 9 dirtybuf 1 lock type devfs: EXCL by thread 0xfffff801961046e0 (pid 50760, syscall4, tid 101050) with exclusive waiters pending #0 0xffffffff80b9856a at lockmgr_xlock_hard+0x40a #1 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #2 0xffffffff80cb2b75 at _vn_lock+0x65 #3 0xffffffff80cae643 at kern_fsync+0x113 #4 0xffffffff8106b901 at amd64_syscall+0x2f1 #5 0xffffffff810416c0 at fast_syscall_common+0x101 dev gpt/tmp fsync: giving up on dirty (error = 35) 0xfffff80030a5c768: tag devfs, type VCHR usecount 4, writecount 0, refcount 12 rdev 0xfffff80030977c00 flags (VI_ACTIVE) v_object 0xfffff80030f49420 ref 0 pages 66 cleanbuf 9 dirtybuf 1 lock type devfs: EXCL by thread 0xfffff805eb5576e0 (pid 50760, syscall4, tid 101038) with exclusive waiters pending #0 0xffffffff80b9856a at lockmgr_xlock_hard+0x40a #1 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #2 0xffffffff80cb2b75 at _vn_lock+0x65 #3 0xffffffff80cae643 at kern_fsync+0x113 #4 0xffffffff8106b901 at amd64_syscall+0x2f1 #5 0xffffffff810416c0 at fast_syscall_common+0x101 dev gpt/tmp fsync: giving up on dirty (error = 35) 0xfffff80030a5c768: tag devfs, type VCHR usecount 4, writecount 0, refcount 12 rdev 0xfffff80030977c00 flags (VI_ACTIVE) v_object 0xfffff80030f49420 ref 0 pages 66 cleanbuf 9 dirtybuf 1 lock type devfs: EXCL by thread 0xfffff8073b3e0000 (pid 50760, syscall4, tid 101063) with exclusive waiters pending #0 0xffffffff80b9856a at lockmgr_xlock_hard+0x40a #1 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #2 0xffffffff80cb2b75 at _vn_lock+0x65 #3 0xffffffff80cae643 at kern_fsync+0x113 #4 0xffffffff8106b901 at amd64_syscall+0x2f1 #5 0xffffffff810416c0 at fast_syscall_common+0x101 dev gpt/tmp fsync: giving up on dirty (error = 35) 0xfffff80030a5c768: tag devfs, type VCHR usecount 4, writecount 0, refcount 12 rdev 0xfffff80030977c00 flags (VI_ACTIVE) v_object 0xfffff80030f49420 ref 0 pages 66 cleanbuf 9 dirtybuf 1 lock type devfs: EXCL by thread 0xfffff8078bc2a000 (pid 50760, syscall4, tid 101031) with exclusive waiters pending #0 0xffffffff80b9856a at lockmgr_xlock_hard+0x40a #1 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #2 0xffffffff80cb2b75 at _vn_lock+0x65 #3 0xffffffff80cae643 at kern_fsync+0x113 #4 0xffffffff8106b901 at amd64_syscall+0x2f1 #5 0xffffffff810416c0 at fast_syscall_common+0x101 dev gpt/tmp fsync: giving up on dirty (error = 35) 0xfffff80030a5c768: tag devfs, type VCHR usecount 4, writecount 0, refcount 12 rdev 0xfffff80030977c00 flags (VI_ACTIVE) v_object 0xfffff80030f49420 ref 0 pages 66 cleanbuf 9 dirtybuf 1 lock type devfs: EXCL by thread 0xfffff802942e1000 (pid 50760, syscall4, tid 101019) with exclusive waiters pending #0 0xffffffff80b9856a at lockmgr_xlock_hard+0x40a #1 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #2 0xffffffff80cb2b75 at _vn_lock+0x65 #3 0xffffffff80cae643 at kern_fsync+0x113 #4 0xffffffff8106b901 at amd64_syscall+0x2f1 #5 0xffffffff810416c0 at fast_syscall_common+0x101 dev gpt/tmp fsync: giving up on dirty (error = 35) 0xfffff80030a5c768: tag devfs, type VCHR usecount 4, writecount 0, refcount 12 rdev 0xfffff80030977c00 flags (VI_ACTIVE) v_object 0xfffff80030f49420 ref 0 pages 66 cleanbuf 9 dirtybuf 1 lock type devfs: EXCL by thread 0xfffff8078bc2a6e0 (pid 50760, syscall4, tid 101030) with exclusive waiters pending #0 0xffffffff80b9856a at lockmgr_xlock_hard+0x40a #1 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #2 0xffffffff80cb2b75 at _vn_lock+0x65 #3 0xffffffff80cae643 at kern_fsync+0x113 #4 0xffffffff8106b901 at amd64_syscall+0x2f1 #5 0xffffffff810416c0 at fast_syscall_common+0x101 dev gpt/tmp fsync: giving up on dirty (error = 35) 0xfffff80030a5c768: tag devfs, type VCHR usecount 4, writecount 0, refcount 12 rdev 0xfffff80030977c00 flags (VI_ACTIVE) v_object 0xfffff80030f49420 ref 0 pages 66 cleanbuf 9 dirtybuf 1 lock type devfs: EXCL by thread 0xfffff80273f7d000 (pid 50760, syscall4, tid 101033) with exclusive waiters pending #0 0xffffffff80b9856a at lockmgr_xlock_hard+0x40a #1 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #2 0xffffffff80cb2b75 at _vn_lock+0x65 #3 0xffffffff80cae643 at kern_fsync+0x113 #4 0xffffffff8106b901 at amd64_syscall+0x2f1 #5 0xffffffff810416c0 at fast_syscall_common+0x101 dev gpt/tmp fsync: giving up on dirty (error = 35) 0xfffff80030a5c768: tag devfs, type VCHR usecount 4, writecount 0, refcount 12 rdev 0xfffff80030977c00 flags (VI_ACTIVE) v_object 0xfffff80030f49420 ref 0 pages 66 cleanbuf 9 dirtybuf 1 lock type devfs: EXCL by thread 0xfffff8032fd6c6e0 (pid 50760, syscall4, tid 101054) with exclusive waiters pending #0 0xffffffff80b9856a at lockmgr_xlock_hard+0x40a #1 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #2 0xffffffff80cb2b75 at _vn_lock+0x65 #3 0xffffffff80cae643 at kern_fsync+0x113 #4 0xffffffff8106b901 at amd64_syscall+0x2f1 #5 0xffffffff810416c0 at fast_syscall_common+0x101 dev gpt/tmp fsync: giving up on dirty (error = 35) 0xfffff80030a5c768: tag devfs, type VCHR usecount 4, writecount 0, refcount 12 rdev 0xfffff80030977c00 flags (VI_ACTIVE) v_object 0xfffff80030f49420 ref 0 pages 66 cleanbuf 9 dirtybuf 1 lock type devfs: EXCL by thread 0xfffff8023ab8e000 (pid 50760, syscall4, tid 101041) with exclusive waiters pending #0 0xffffffff80b9856a at lockmgr_xlock_hard+0x40a #1 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #2 0xffffffff80cb2b75 at _vn_lock+0x65 #3 0xffffffff80cae643 at kern_fsync+0x113 #4 0xffffffff8106b901 at amd64_syscall+0x2f1 #5 0xffffffff810416c0 at fast_syscall_common+0x101 dev gpt/tmp fsync: giving up on dirty (error = 35) 0xfffff80030a5c768: tag devfs, type VCHR usecount 4, writecount 0, refcount 12 rdev 0xfffff80030977c00 flags (VI_ACTIVE) v_object 0xfffff80030f49420 ref 0 pages 66 cleanbuf 9 dirtybuf 1 lock type devfs: EXCL by thread 0xfffff80233aa8000 (pid 50760, syscall4, tid 101039) with exclusive waiters pending #0 0xffffffff80b9856a at lockmgr_xlock_hard+0x40a #1 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #2 0xffffffff80cb2b75 at _vn_lock+0x65 #3 0xffffffff80cae643 at kern_fsync+0x113 #4 0xffffffff8106b901 at amd64_syscall+0x2f1 #5 0xffffffff810416c0 at fast_syscall_common+0x101 dev gpt/tmp fsync: giving up on dirty (error = 35) 0xfffff80030a5c768: tag devfs, type VCHR usecount 4, writecount 0, refcount 12 rdev 0xfffff80030977c00 flags (VI_ACTIVE) v_object 0xfffff80030f49420 ref 0 pages 66 cleanbuf 9 dirtybuf 1 lock type devfs: EXCL by thread 0xfffff804fb0a3000 (pid 50760, syscall4, tid 101049) with exclusive waiters pending #0 0xffffffff80b9856a at lockmgr_xlock_hard+0x40a #1 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #2 0xffffffff80cb2b75 at _vn_lock+0x65 #3 0xffffffff80cae643 at kern_fsync+0x113 #4 0xffffffff8106b901 at amd64_syscall+0x2f1 #5 0xffffffff810416c0 at fast_syscall_common+0x101 dev gpt/tmp fsync: giving up on dirty (error = 35) 0xfffff80030a5c768: tag devfs, type VCHR usecount 4, writecount 0, refcount 12 rdev 0xfffff80030977c00 flags (VI_ACTIVE) v_object 0xfffff80030f49420 ref 0 pages 66 cleanbuf 9 dirtybuf 1 lock type devfs: EXCL by thread 0xfffff805d63316e0 (pid 50760, syscall4, tid 101056) with exclusive waiters pending #0 0xffffffff80b9856a at lockmgr_xlock_hard+0x40a #1 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #2 0xffffffff80cb2b75 at _vn_lock+0x65 #3 0xffffffff80cae643 at kern_fsync+0x113 #4 0xffffffff8106b901 at amd64_syscall+0x2f1 #5 0xffffffff810416c0 at fast_syscall_common+0x101 dev gpt/tmp 20191211 13:59:05 all (11/121): churn.sh 20191211 14:09:06 all (12/121): core5.sh 20191211 14:19:09 all (13/121): inversion.sh 20191211 14:29:13 all (14/121): parallelmount.sh GEOM leak: 1190/1263 20191211 14:39:14 all (15/121): callout_reset_on2.sh 20191211 14:49:17 all (16/121): msdos.sh 20191211 14:59:19 all (17/121): lstat.sh 20191211 15:09:23 all (18/121): pmc2.sh Expensive timeout(9) function: 0xffffffff809eb600(0xfffffe00d1584428) 0.014085830 s 20191211 15:19:26 all (19/121): quota1.sh Dec 11 15:20:03 mercat1 kernel: pid 47687 (rw), uid 0 inumber 4097 on /mnt: out of inodes Dec 11 15:20:14 mercat1 kernel: pid 47689 (creat), uid 0 inumber 4129 on /mnt: out of inodes Dec 11 15:20:25 mercat1 kernel: pid 47694 (creat), uid 0 inumber 8202 on /mnt: out of inodes Dec 11 15:20:43 mercat1 kernel: pid 47707 (creat), uid 0 inumber 8201 on /mnt: out of inodes Dec 11 15:21:11 mercat1 kernel: pid 47697 (creat), uid 0 inumber 8193 on /mnt: out of inodes Dec 11 15:21:15 mercat1 kernel: pid 47704 (mkdir), uid 0 inumber 12028 on /mnt: out of inodes Dec 11 15:23:32 mercat1 kernel: pid 72277 (rw), uid 0 inumber 8208 on /mnt: out of inodes Dec 11 15:23:36 mercat1 kernel: pid 72269 (creat), uid 0 inumber 8200 on /mnt: out of inodes Dec 11 15:23:38 mercat1 syslogd: last message repeated 1 times Dec 11 15:23:45 mercat1 kernel: pid 72265 (mkdir), uid 0 inumber 4076 on /mnt: out of inodes Dec 11 15:23:50 mercat1 kernel: pid 72267 (creat), uid 0 inumber 8198 on /mnt: out of inodes Dec 11 15:23:54 mercat1 kernel: pid 72285 (creat), uid 0 inumber 8218 on /mnt: out of inodes Dec 11 15:23:56 mercat1 kernel: pid 72269 (creat), uid 0 inumber 8200 on /mnt: out of inodes Dec 11 15:24:06 mercat1 syslogd: last message repeated 1 times Dec 11 15:24:08 mercat1 kernel: pid 72271 (creat), uid 0 inumber 8202 on /mnt: out of inodes Dec 11 15:24:21 mercat1 syslogd: last message repeated 1 times Dec 11 15:24:36 mercat1 kernel: pid 72269 (creat), uid 0 inumber 8200 on /mnt: out of inodes Dec 11 15:24:41 mercat1 kernel: pid 72285 (creat), uid 0 inumber 8218 on /mnt: out of inodes Dec 11 15:24:46 mercat1 kernel: pid 72264 (creat), uid 0 inumber 8215 on /mnt: out of inodes Dec 11 15:24:51 mercat1 kernel: pid 72271 (creat), uid 0 inumber 8202 on /mnt: out of inodes Dec 11 15:25:39 mercat1 kernel: pid 84947 (rw), uid 0 inumber 4167 on /mnt: filesystem full Dec 11 15:25:49 mercat1 kernel: pid 84961 (creat), uid 0 inumber 4121 on /mnt: out of inodes Dec 11 15:25:56 mercat1 kernel: pid 84963 (creat), uid 0 inumber 4122 on /mnt: out of inodes Dec 11 15:26:01 mercat1 kernel: pid 84948 (creat), uid 0 inumber 4114 on /mnt: out of inodes Dec 11 15:26:06 mercat1 kernel: pid 84963 (creat), uid 0 inumber 4122 on /mnt: out of inodes Dec 11 15:26:37 mercat1 kernel: pid 84950 (creat), uid 0 inumber 4117 on /mnt: out of inodes Dec 11 15:26:46 mercat1 kernel: pid 84963 (creat), uid 0 inumber 4122 on /mnt: out of inodes Dec 11 15:26:59 mercat1 kernel: pid 84951 (creat), uid 0 inumber 4108 on /mnt: out of inodes Dec 11 15:27:03 mercat1 kernel: pid 84952 (creat), uid 0 inumber 4112 on /mnt: out of inodes Dec 11 15:27:32 mercat1 kernel: pid 97511 (rw), uid 0 inumber 12288 on /mnt: out of inodes Dec 11 15:27:33 mercat1 kernel: pid 97519 (rw), uid 0 inumber 12519 on /mnt: out of inodes Dec 11 15:27:38 mercat1 kernel: pid 97530 (creat), uid 0 inumber 12301 on /mnt: out of inodes Dec 11 15:27:43 mercat1 kernel: pid 97529 (mkdir), uid 0 inumber 8139 on /mnt: out of inodes Dec 11 15:27:51 mercat1 kernel: pid 97530 (creat), uid 0 inumber 12301 on /mnt: out of inodes Dec 11 15:27:53 mercat1 kernel: pid 97517 (rw), uid 0 inumber 12304 on /mnt: out of inodes Dec 11 15:27:54 mercat1 kernel: pid 97552 (creat), uid 0 inumber 12339 on /mnt: out of inodes Dec 11 15:27:56 mercat1 kernel: pid 97540 (creat), uid 0 inumber 12371 on /mnt: out of inodes Dec 11 15:27:58 mercat1 kernel: pid 97538 (creat), uid 0 inumber 12328 on /mnt: out of inodes Dec 11 15:27:59 mercat1 kernel: pid 97528 (creat), uid 0 inumber 12319 on /mnt: out of inodes Dec 11 15:28:01 mercat1 kernel: pid 97546 (creat), uid 0 inumber 12338 on /mnt: out of inodes Dec 11 15:28:02 mercat1 kernel: pid 97538 (creat), uid 0 inumber 12328 on /mnt: out of inodes Dec 11 15:28:06 mercat1 kernel: pid 97530 (creat), uid 0 inumber 12301 on /mnt: out of inodes Dec 11 15:28:09 mercat1 kernel: pid 97532 (creat), uid 0 inumber 12325 on /mnt: out of inodes Dec 11 15:28:10 mercat1 kernel: pid 97548 (creat), uid 0 inumber 12332 on /mnt: out of inodes Dec 11 15:28:11 mercat1 syslogd: last message repeated 1 times Dec 11 15:28:14 mercat1 kernel: pid 97546 (creat), uid 0 inumber 12338 on /mnt: out of inodes Dec 11 15:28:19 mercat1 syslogd: last message repeated 2 times Dec 11 15:28:20 mercat1 kernel: pid 97540 (creat), uid 0 inumber 12371 on /mnt: out of inodes Dec 11 15:28:22 mercat1 kernel: pid 97550 (creat), uid 0 inumber 12368 on /mnt: out of inodes Dec 11 15:28:23 mercat1 kernel: pid 97542 (creat), uid 0 inumber 12353 on /mnt: out of inodes Dec 11 15:28:26 mercat1 kernel: pid 97544 (creat), uid 0 inumber 12336 on /mnt: out of inodes Dec 11 15:28:29 mercat1 syslogd: last message repeated 1 times Dec 11 15:28:31 mercat1 kernel: pid 97552 (creat), uid 0 inumber 12339 on /mnt: out of inodes Dec 11 15:28:34 mercat1 kernel: pid 97532 (creat), uid 0 inumber 12325 on /mnt: out of inodes Dec 11 15:28:36 mercat1 kernel: pid 97525 (creat), uid 0 inumber 12292 on /mnt: out of inodes Dec 11 15:28:37 mercat1 kernel: pid 97530 (creat), uid 0 inumber 12301 on /mnt: out of inodes Dec 11 15:28:38 mercat1 syslogd: last message repeated 1 times Dec 11 15:28:39 mercat1 kernel: pid 97554 (creat), uid 0 inumber 12373 on /mnt: out of inodes Dec 11 15:28:40 mercat1 kernel: pid 97540 (creat), uid 0 inumber 12371 on /mnt: out of inodes Dec 11 15:28:42 mercat1 kernel: pid 97548 (creat), uid 0 inumber 12332 on /mnt: out of inodes Dec 11 15:28:43 mercat1 kernel: pid 97544 (creat), uid 0 inumber 12336 on /mnt: out of inodes Dec 11 15:28:45 mercat1 kernel: pid 97554 (creat), uid 0 inumber 12373 on /mnt: out of inodes Dec 11 15:28:46 mercat1 kernel: pid 97534 (creat), uid 0 inumber 12355 on /mnt: out of inodes Dec 11 15:28:48 mercat1 syslogd: last message repeated 1 times Dec 11 15:28:51 mercat1 kernel: pid 97550 (creat), uid 0 inumber 12368 on /mnt: out of inodes Dec 11 15:28:52 mercat1 kernel: pid 97538 (creat), uid 0 inumber 12328 on /mnt: out of inodes Dec 11 15:28:54 mercat1 kernel: pid 97540 (creat), uid 0 inumber 12371 on /mnt: out of inodes Dec 11 15:28:56 mercat1 kernel: pid 97528 (creat), uid 0 inumber 12319 on /mnt: out of inodes Dec 11 15:28:58 mercat1 kernel: pid 97538 (creat), uid 0 inumber 12328 on /mnt: out of inodes Dec 11 15:29:01 mercat1 kernel: pid 97534 (creat), uid 0 inumber 12355 on /mnt: out of inodes Dec 11 15:29:06 mercat1 kernel: pid 97540 (creat), uid 0 inumber 12371 on /mnt: out of inodes Dec 11 15:29:08 mercat1 kernel: pid 97544 (creat), uid 0 inumber 12336 on /mnt: out of inodes Dec 11 15:29:11 mercat1 kernel: pid 97540 (creat), uid 0 inumber 12371 on /mnt: out of inodes Dec 11 15:29:14 mercat1 syslogd: last message repeated 1 times Dec 11 15:29:15 mercat1 kernel: pid 97536 (creat), uid 0 inumber 12527 on /mnt: out of inodes Dec 11 15:29:18 mercat1 syslogd: last message repeated 1 times Dec 11 15:29:20 mercat1 kernel: pid 97530 (creat), uid 0 inumber 12301 on /mnt: out of inodes Dec 11 15:29:22 mercat1 kernel: pid 97527 (creat), uid 0 inumber 12321 on /mnt: out of inodes Dec 11 15:29:24 mercat1 kernel: pid 97538 (creat), uid 0 inumber 12328 on /mnt: out of inodes Dec 11 15:29:26 mercat1 syslogd: last message repeated 1 times Dec 11 15:29:27 mercat1 kernel: pid 97548 (creat), uid 0 inumber 12332 on /mnt: out of inodes Dec 11 15:29:28 mercat1 kernel: pid 97532 (creat), uid 0 inumber 12325 on /mnt: out of inodes 20191211 15:29:29 all (20/121): select.sh 20191211 15:39:36 all (21/121): ext2fs2.sh 20191211 15:49:51 all (22/121): mlockall6.sh Expensive timeout(9) function: 0xffffffff809eb600(0xfffffe00d1584428) 0.094527260 s Dec 11 20:24:24 mercat1 su[99828]: pho to root on /dev/pts/1 20191211 20:25:51 *** Excessive run time: mlockall6.sh, 276 min pts leak: 1/2 mount leak: 13/44 GEOM leak: 23/1286 20191211 20:25:52 all (23/121): mmap3.sh 20191211 20:36:01 all (24/121): mmap2.sh 20191211 20:46:06 all (25/121): buildworld4.sh 20191211 20:56:11 all (26/121): snap5-1.sh lock order reversal: 1st 0xfffffe003c6b3b58 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:283 2nd 0xfffff807a7852930 snaplk (snaplk) @ ufs/ffs/ffs_snapshot.c:2327 stack backtrace: #0 0xffffffff80c32581 at witness_debugger+0x71 #1 0xffffffff80c322f6 at witness_checkorder+0xad6 #2 0xffffffff80b981cf at lockmgr_xlock_hard+0x6f #3 0xffffffff80b98d5d at __lockmgr_args+0x60d #4 0xffffffff80ec2ca6 at ffs_copyonwrite+0x166 #5 0xffffffff80ee54dc at ffs_geom_strategy+0x8c #6 0xffffffff80ef8c73 at ufs_strategy+0x83 #7 0xffffffff810e8f12 at VOP_STRATEGY_APV+0xb2 #8 0xffffffff80c79e4e at bufstrategy+0x3e #9 0xffffffff80c79cea at bufwrite+0x24a #10 0xffffffff80ee72ed at ffs_syncvnode+0x2bd #11 0xffffffff80ee6312 at ffs_fsync+0x22 #12 0xffffffff810e785a at VOP_FSYNC_APV+0xaa #13 0xffffffff80ef68af at ufs_remove+0xdf #14 0xffffffff810e79ee at VOP_REMOVE_APV+0xce #15 0xffffffff80cab37e at kern_funlinkat+0x2be #16 0xffffffff80cab0b8 at sys_unlink+0x28 #17 0xffffffff8106b901 at amd64_syscall+0x2f1 lock order reversal: 1st 0xfffff807a7852930 snaplk (snaplk) @ kern/vfs_subr.c:3046 2nd 0xfffff806521282e0 ufs (ufs) @ ufs/ffs/ffs_snapshot.c:1650 stack backtrace: #0 0xffffffff80c32581 at witness_debugger+0x71 #1 0xffffffff80c322f6 at witness_checkorder+0xad6 #2 0xffffffff80b981cf at lockmgr_xlock_hard+0x6f #3 0xffffffff80b98d5d at __lockmgr_args+0x60d #4 0xffffffff80ec120d at ffs_snapremove+0xed #5 0xffffffff80ebb90a at ffs_truncate+0x26a #6 0xffffffff80eeed39 at ufs_inactive+0x1c9 #7 0xffffffff810e86ea at VOP_INACTIVE_APV+0xaa #8 0xffffffff80c9f1b4 at vinactive+0xf4 #9 0xffffffff80c9f9f5 at vputx+0x275 #10 0xffffffff80c9fab9 at vput+0x39 #11 0xffffffff80cab3d3 at kern_funlinkat+0x313 #12 0xffffffff80cab0b8 at sys_unlink+0x28 #13 0xffffffff8106b901 at amd64_syscall+0x2f1 #14 0xffffffff810416c0 at fast_syscall_common+0x101 20191211 21:06:23 all (27/121): alternativeFlushPath.sh Dec 11 21:06:55 mercat1 kernel: pid 88184 (alternativeFlushPat), uid 0 inumber 401283 on /tmp: out of inodes 20191211 21:07:00 all (28/121): tmpfs.sh Dec 11 21:07:30 mercat1 kernel: pid 92120 (swap), jid 0, uid 0, was killed: out of swap space Dec 11 21:12:03 mercat1 kernel: pid 97287 (swap), jid 0, uid 0, was killed: out of swap space Dec 11 21:12:25 mercat1 kernel: pid 97304 (swap), jid 0, uid 0, was killed: out of swap space 20191211 21:17:27 all (29/121): graid1.sh Dec 11 21:22:05 mercat1 kernel: pid 12141 (swap), jid 0, uid 2007, was killed: out of swap space Dec 11 21:22:28 mercat1 kernel: pid 12152 (rw), uid 2007 inumber 65819 on /mnt: filesystem full Dec 11 21:22:32 mercat1 kernel: pid 12082 (mkdir), uid 2007 inumber 78360 on /mnt: filesystem full Dec 11 21:23:05 mercat1 kernel: pid 12099 (creat), uid 2007 inumber 65545 on /mnt: out of inodes Dec 11 21:23:39 mercat1 kernel: pid 12122 (link), uid 2007 inumber 65650 on /mnt: out of inodes Dec 11 21:24:37 mercat1 kernel: pid 14220 (swap), jid 0, uid 2007, was killed: out of swap space Dec 11 21:25:56 mercat1 kernel: pid 15699 (rw), uid 2007 inumber 81851 on /mnt: filesystem full Dec 11 21:26:33 mercat1 kernel: pid 15676 (rw), uid 2007 inumber 90879 on /mnt: filesystem full 20191211 21:27:54 all (30/121): tmpfs3.sh Dec 11 21:28:49 mercat1 kernel: pid 22499 (swap), jid 0, uid 0, was killed: out of swap space 20191211 21:38:18 all (31/121): nullfs26.sh Dec 11 21:39:09 mercat1 kernel: pid 35711 (swap), jid 0, uid 2007, was killed: out of swap space Dec 11 21:42:05 mercat1 kernel: pid 37172 (swap), jid 0, uid 2007, was killed: out of swap space Dec 11 21:44:53 mercat1 kernel: pid 40472 (swap), jid 0, uid 2007, was killed: out of swap space Dec 11 21:45:04 mercat1 kernel: pid 40384 (swap), jid 0, uid 2007, was killed: out of swap space 20191211 21:48:45 all (32/121): badcode.sh Dec 11 21:48:49 mercat1 badcode[43929]: stack overflow detected; terminated Dec 11 21:48:49 mercat1 badcode[43951]: stack overflow detected; terminated Dec 11 21:48:49 mercat1 badcode[43963]: stack overflow detected; terminated Dec 11 21:48:50 mercat1 badcode[43998]: stack overflow detected; terminated Dec 11 21:48:50 mercat1 badcode[44003]: stack overflow detected; terminated Dec 11 21:48:50 mercat1 badcode[44004]: stack overflow detected; terminated Dec 11 21:48:50 mercat1 badcode[44015]: stack overflow detected; terminated Dec 11 21:48:50 mercat1 badcode[44031]: stack overflow detected; terminated Dec 11 21:48:50 mercat1 badcode[44027]: stack overflow detected; terminated Dec 11 21:48:50 mercat1 badcode[44037]: stack overflow detected; terminated Dec 11 21:48:50 mercat1 badcode[44034]: stack overflow detected; terminated Dec 11 21:48:50 mercat1 badcode[44050]: stack overflow detected; terminated Dec 11 21:48:50 mercat1 badcode[44063]: stack overflow detected; terminated Dec 11 21:48:50 mercat1 badcode[44065]: stack overflow detected; terminated Dec 11 21:48:51 mercat1 badcode[44075]: stack overflow detected; terminated Dec 11 21:48:51 mercat1 badcode[44091]: stack overflow detected; terminated Dec 11 21:48:51 mercat1 badcode[44097]: stack overflow detected; terminated Dec 11 21:48:51 mercat1 badcode[44095]: stack overflow detected; terminated Dec 11 21:48:51 mercat1 badcode[44106]: stack overflow detected; terminated Dec 11 21:48:51 mercat1 badcode[44108]: stack overflow detected; terminated Dec 11 21:48:52 mercat1 badcode[44121]: stack overflow detected; terminated Dec 11 21:48:52 mercat1 badcode[44134]: stack overflow detected; terminated Dec 11 21:48:52 mercat1 badcode[44133]: stack overflow detected; terminated Dec 11 21:48:52 mercat1 badcode[44136]: stack overflow detected; terminated Dec 11 21:48:52 mercat1 badcode[44137]: stack overflow detected; terminated Dec 11 21:48:52 mercat1 badcode[44149]: stack overflow detected; terminated Dec 11 21:48:52 mercat1 badcode[44156]: stack overflow detected; terminated Dec 11 21:48:52 mercat1 badcode[44160]: stack overflow detected; terminated Dec 11 21:48:53 mercat1 badcode[44191]: stack overflow detected; terminated Dec 11 21:48:53 mercat1 badcode[44194]: stack overflow detected; terminated Dec 11 21:48:53 mercat1 badcode[44198]: stack overflow detected; terminated Dec 11 21:48:53 mercat1 badcode[44211]: stack overflow detected; terminated Dec 11 21:48:53 mercat1 badcode[44232]: stack overflow detected; terminated Dec 11 21:48:53 mercat1 badcode[44235]: stack overflow detected; terminated Dec 11 21:48:53 mercat1 badcode[44252]: stack overflow detected; terminated Dec 11 21:48:53 mercat1 badcode[44260]: stack overflow detected; terminated Dec 11 21:48:53 mercat1 badcode[44273]: stack overflow detected; terminated Dec 11 21:48:53 mercat1 badcode[44281]: stack overflow detected; terminated Dec 11 21:48:53 mercat1 badcode[44283]: stack overflow detected; terminated Dec 11 21:48:53 mercat1 badcode[44290]: stack overflow detected; terminated Dec 11 21:48:53 mercat1 badcode[44299]: stack overflow detected; terminated Dec 11 21:48:54 mercat1 badcode[44304]: stack overflow detected; terminated Dec 11 21:48:54 mercat1 badcode[44308]: stack overflow detected; terminated Dec 11 21:48:54 mercat1 badcode[44323]: stack overflow detected; terminated Dec 11 21:48:54 mercat1 badcode[44326]: stack overflow detected; terminated Dec 11 21:48:54 mercat1 badcode[44334]: stack overflow detected; terminated Dec 11 21:48:54 mercat1 badcode[44333]: stack overflow detected; terminated Dec 11 21:48:54 mercat1 badcode[44350]: stack overflow detected; terminated Dec 11 21:48:55 mercat1 badcode[44384]: stack overflow detected; terminated Dec 11 21:48:56 mercat1 badcode[44353]: stack overflow detected; terminated Dec 11 21:48:56 mercat1 badcode[44416]: stack overflow detected; terminated Dec 11 21:48:56 mercat1 badcode[44426]: stack overflow detected; terminated Dec 11 21:48:56 mercat1 badcode[44432]: stack overflow detected; terminated Dec 11 21:48:56 mercat1 badcode[44450]: stack overflow detected; terminated Dec 11 21:48:56 mercat1 badcode[44458]: stack overflow detected; terminated Dec 11 21:48:56 mercat1 badcode[44461]: stack overflow detected; terminated Dec 11 21:48:56 mercat1 badcode[44464]: stack overflow detected; terminated Dec 11 21:48:56 mercat1 badcode[44488]: stack overflow detected; terminated Dec 11 21:48:56 mercat1 badcode[44492]: stack overflow detected; terminated Dec 11 21:48:57 mercat1 badcode[44501]: stack overflow detected; terminated Dec 11 21:48:57 mercat1 badcode[44494]: stack overflow detected; terminated Dec 11 21:48:57 mercat1 badcode[44502]: stack overflow detected; terminated Dec 11 21:49:08 mercat1 badcode[44503]: stack overflow detected; terminated Dec 11 21:49:28 mercat1 kernel: pid 43835 (swap), jid 0, uid 2007, was killed: out of swap space Dec 11 21:50:53 mercat1 badcode[45804]: stack overflow detected; terminated Dec 11 21:50:53 mercat1 badcode[45812]: stack overflow detected; terminated Dec 11 21:50:54 mercat1 badcode[45849]: stack overflow detected; terminated Dec 11 21:50:54 mercat1 badcode[45852]: stack overflow detected; terminated Dec 11 21:50:54 mercat1 badcode[45857]: stack overflow detected; terminated Dec 11 21:50:54 mercat1 badcode[45869]: stack overflow detected; terminated Dec 11 21:50:54 mercat1 badcode[45876]: stack overflow detected; terminated Dec 11 21:50:54 mercat1 badcode[45878]: stack overflow detected; terminated Dec 11 21:50:54 mercat1 badcode[45882]: stack overflow detected; terminated Dec 11 21:50:54 mercat1 badcode[45891]: stack overflow detected; terminated Dec 11 21:50:54 mercat1 badcode[45896]: stack overflow detected; terminated Dec 11 21:50:54 mercat1 badcode[45904]: stack overflow detected; terminated Dec 11 21:50:54 mercat1 badcode[45905]: stack overflow detected; terminated Dec 11 21:50:54 mercat1 badcode[45923]: stack overflow detected; terminated Dec 11 21:50:54 mercat1 badcode[45931]: stack overflow detected; terminated Dec 11 21:50:54 mercat1 badcode[45945]: stack overflow detected; terminated Dec 11 21:50:54 mercat1 badcode[45950]: stack overflow detected; terminated Dec 11 21:50:54 mercat1 badcode[45956]: stack overflow detected; terminated Dec 11 21:50:56 mercat1 badcode[45978]: stack overflow detected; terminated Dec 11 21:50:56 mercat1 badcode[45979]: stack overflow detected; terminated Dec 11 21:50:56 mercat1 badcode[46011]: stack overflow detected; terminated Dec 11 21:50:56 mercat1 badcode[46017]: stack overflow detected; terminated Dec 11 21:50:56 mercat1 badcode[46033]: stack overflow detected; terminated Dec 11 21:50:56 mercat1 badcode[46041]: stack overflow detected; terminated Dec 11 21:50:56 mercat1 badcode[46044]: stack overflow detected; terminated Dec 11 21:50:56 mercat1 badcode[46051]: stack overflow detected; terminated Dec 11 21:50:56 mercat1 badcode[46060]: stack overflow detected; terminated Dec 11 21:50:56 mercat1 badcode[46085]: stack overflow detected; terminated Dec 11 21:50:56 mercat1 badcode[46099]: stack overflow detected; terminated Dec 11 21:50:56 mercat1 badcode[46104]: stack overflow detected; terminated Dec 11 21:50:56 mercat1 badcode[46108]: stack overflow detected; terminated Dec 11 21:50:56 mercat1 badcode[46121]: stack overflow detected; terminated Dec 11 21:50:56 mercat1 badcode[46151]: stack overflow detected; terminated Dec 11 21:50:56 mercat1 badcode[46165]: stack overflow detected; terminated Dec 11 21:50:56 mercat1 badcode[46167]: stack overflow detected; terminated Dec 11 21:50:56 mercat1 badcode[46168]: stack overflow detected; terminated Dec 11 21:50:58 mercat1 badcode[46209]: stack overflow detected; terminated Dec 11 21:50:59 mercat1 badcode[46218]: stack overflow detected; terminated Dec 11 21:50:59 mercat1 badcode[46222]: stack overflow detected; terminated Dec 11 21:50:59 mercat1 badcode[46245]: stack overflow detected; terminated Dec 11 21:50:59 mercat1 badcode[46251]: stack overflow detected; terminated Dec 11 21:50:59 mercat1 badcode[46268]: stack overflow detected; terminated Dec 11 21:50:59 mercat1 badcode[46273]: stack overflow detected; terminated Dec 11 21:51:00 mercat1 badcode[46285]: stack overflow detected; terminated Dec 11 21:51:01 mercat1 badcode[46315]: stack overflow detected; terminated Dec 11 21:51:01 mercat1 badcode[46323]: stack overflow detected; terminated Dec 11 21:51:01 mercat1 badcode[46329]: stack overflow detected; terminated Dec 11 21:51:01 mercat1 badcode[46347]: stack overflow detected; terminated Dec 11 21:51:01 mercat1 badcode[46365]: stack overflow detected; terminated Dec 11 21:51:01 mercat1 badcode[46385]: stack overflow detected; terminated Dec 11 21:51:01 mercat1 badcode[46393]: stack overflow detected; terminated Dec 11 21:51:01 mercat1 badcode[46396]: stack overflow detected; terminated Dec 11 21:51:01 mercat1 badcode[46400]: stack overflow detected; terminated Dec 11 21:51:01 mercat1 badcode[46406]: stack overflow detected; terminated Dec 11 21:51:01 mercat1 badcode[46407]: stack overflow detected; terminated Dec 11 21:51:01 mercat1 badcode[46418]: stack overflow detected; terminated Dec 11 21:51:01 mercat1 badcode[46424]: stack overflow detected; terminated Dec 11 21:51:02 mercat1 badcode[46452]: stack overflow detected; terminated Dec 11 21:51:02 mercat1 badcode[46457]: stack overflow detected; terminated Dec 11 21:51:02 mercat1 badcode[46469]: stack overflow detected; terminated Dec 11 21:51:02 mercat1 badcode[46475]: stack overflow detected; terminated Dec 11 21:51:02 mercat1 badcode[46477]: stack overflow detected; terminated Dec 11 21:51:02 mercat1 badcode[46481]: stack overflow detected; terminated Dec 11 21:51:02 mercat1 badcode[46503]: stack overflow detected; terminated Dec 11 21:51:03 mercat1 badcode[46550]: stack overflow detected; terminated Dec 11 21:51:03 mercat1 badcode[46561]: stack overflow detected; terminated Dec 11 21:51:03 mercat1 badcode[46560]: stack overflow detected; terminated Dec 11 21:51:03 mercat1 badcode[46563]: stack overflow detected; terminated Dec 11 21:51:03 mercat1 badcode[46575]: stack overflow detected; terminated Dec 11 21:51:03 mercat1 badcode[46584]: stack overflow detected; terminated Dec 11 21:51:03 mercat1 badcode[46586]: stack overflow detected; terminated Dec 11 21:51:03 mercat1 badcode[46604]: stack overflow detected; terminated Dec 11 21:51:03 mercat1 badcode[46610]: stack overflow detected; terminated Dec 11 21:51:04 mercat1 badcode[46632]: stack overflow detected; terminated Dec 11 21:51:04 mercat1 badcode[46627]: stack overflow detected; terminated Dec 11 21:51:04 mercat1 badcode[46650]: stack overflow detected; terminated Dec 11 21:51:04 mercat1 badcode[46663]: stack overflow detected; terminated Dec 11 21:51:04 mercat1 badcode[46700]: stack overflow detected; terminated Dec 11 21:51:04 mercat1 badcode[46702]: stack overflow detected; terminated Dec 11 21:51:04 mercat1 badcode[46709]: stack overflow detected; terminated Dec 11 21:51:05 mercat1 badcode[46744]: stack overflow detected; terminated Dec 11 21:51:05 mercat1 badcode[46773]: stack overflow detected; terminated Dec 11 21:51:06 mercat1 badcode[46805]: stack overflow detected; terminated Dec 11 21:51:06 mercat1 badcode[46810]: stack overflow detected; terminated Dec 11 21:51:06 mercat1 badcode[46830]: stack overflow detected; terminated Dec 11 21:51:06 mercat1 badcode[46845]: stack overflow detected; terminated Dec 11 21:51:06 mercat1 badcode[46860]: stack overflow detected; terminated Dec 11 21:51:07 mercat1 badcode[46886]: stack overflow detected; terminated Dec 11 21:51:07 mercat1 badcode[46895]: stack overflow detected; terminated Dec 11 21:51:07 mercat1 badcode[46929]: stack overflow detected; terminated Dec 11 21:51:07 mercat1 badcode[46937]: stack overflow detected; terminated Dec 11 21:51:08 mercat1 badcode[46960]: stack overflow detected; terminated Dec 11 21:51:08 mercat1 badcode[46958]: stack overflow detected; terminated Dec 11 21:51:08 mercat1 badcode[46983]: stack overflow detected; terminated Dec 11 21:51:09 mercat1 badcode[47012]: stack overflow detected; terminated Dec 11 21:51:12 mercat1 badcode[47085]: stack overflow detected; terminated Dec 11 21:51:12 mercat1 badcode[47086]: stack overflow detected; terminated Dec 11 21:51:12 mercat1 badcode[47089]: stack overflow detected; terminated Dec 11 21:51:12 mercat1 badcode[47104]: stack overflow detected; terminated Dec 11 21:51:13 mercat1 badcode[47121]: stack overflow detected; terminated Dec 11 21:51:13 mercat1 badcode[47125]: stack overflow detected; terminated Dec 11 21:51:13 mercat1 badcode[47140]: stack overflow detected; terminated Dec 11 21:51:13 mercat1 badcode[47166]: stack overflow detected; terminated Dec 11 21:51:13 mercat1 badcode[47185]: stack overflow detected; terminated Dec 11 21:51:13 mercat1 badcode[47187]: stack overflow detected; terminated Dec 11 21:51:13 mercat1 badcode[47209]: stack overflow detected; terminated Dec 11 21:51:13 mercat1 badcode[47216]: stack overflow detected; terminated Dec 11 21:51:13 mercat1 badcode[47220]: stack overflow detected; terminated Dec 11 21:51:13 mercat1 badcode[47246]: stack overflow detected; terminated Dec 11 21:51:13 mercat1 badcode[47248]: stack overflow detected; terminated Dec 11 21:51:13 mercat1 badcode[47251]: stack overflow detected; terminated Dec 11 21:51:13 mercat1 badcode[47257]: stack overflow detected; terminated Dec 11 21:51:13 mercat1 badcode[47260]: stack overflow detected; terminated Dec 11 21:51:13 mercat1 badcode[47263]: stack overflow detected; terminated Dec 11 21:51:13 mercat1 badcode[47279]: stack overflow detected; terminated Dec 11 21:51:13 mercat1 badcode[47288]: stack overflow detected; terminated Dec 11 21:51:13 mercat1 badcode[47294]: stack overflow detected; terminated Dec 11 21:51:13 mercat1 badcode[47303]: stack overflow detected; terminated Dec 11 21:51:21 mercat1 kernel: pid 45776 (swap), jid 0, uid 2007, was killed: out of swap space Dec 11 21:52:58 mercat1 badcode[49948]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[49949]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[49952]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[49959]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[49958]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[49962]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50010]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50014]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50022]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50032]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50038]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50041]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50059]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50082]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50112]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50134]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50135]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50137]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50138]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50139]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50144]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50149]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50156]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50159]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50205]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50206]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50209]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50217]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50230]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50235]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50256]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50258]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50263]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50268]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50269]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50279]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50286]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50295]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50301]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50304]: stack overflow detected; terminated Dec 11 21:52:58 mercat1 badcode[50318]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50337]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50346]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50369]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50386]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50408]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50414]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50423]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50439]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50508]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50525]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50549]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50547]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50550]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50579]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50588]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50596]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50601]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50603]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50605]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50615]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50620]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50632]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50645]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50659]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50676]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50685]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50688]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50698]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50701]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50720]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50729]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50774]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50782]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50788]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50792]: stack overflow detected; terminated Dec 11 21:53:00 mercat1 badcode[50821]: stack overflow detected; terminated Dec 11 21:53:01 mercat1 badcode[50840]: stack overflow detected; terminated Dec 11 21:53:03 mercat1 badcode[50850]: stack overflow detected; terminated Dec 11 21:53:03 mercat1 badcode[50872]: stack overflow detected; terminated Dec 11 21:53:03 mercat1 badcode[50899]: stack overflow detected; terminated Dec 11 21:53:03 mercat1 badcode[50906]: stack overflow detected; terminated Dec 11 21:53:03 mercat1 badcode[50910]: stack overflow detected; terminated Dec 11 21:53:03 mercat1 badcode[50923]: stack overflow detected; terminated Dec 11 21:53:03 mercat1 badcode[50924]: stack overflow detected; terminated Dec 11 21:53:03 mercat1 badcode[50939]: stack overflow detected; terminated Dec 11 21:53:03 mercat1 badcode[50940]: stack overflow detected; terminated Dec 11 21:53:03 mercat1 badcode[50948]: stack overflow detected; terminated Dec 11 21:53:03 mercat1 badcode[50949]: stack overflow detected; terminated Dec 11 21:53:03 mercat1 badcode[50964]: stack overflow detected; terminated Dec 11 21:53:06 mercat1 badcode[51070]: stack overflow detected; terminated Dec 11 21:53:06 mercat1 badcode[51084]: stack overflow detected; terminated Dec 11 21:53:06 mercat1 badcode[51086]: stack overflow detected; terminated Dec 11 21:53:06 mercat1 badcode[51091]: stack overflow detected; terminated Dec 11 21:53:06 mercat1 badcode[51098]: stack overflow detected; terminated Dec 11 21:53:06 mercat1 badcode[51123]: stack overflow detected; terminated Dec 11 21:53:06 mercat1 badcode[51129]: stack overflow detected; terminated Dec 11 21:53:06 mercat1 badcode[51153]: stack overflow detected; terminated Dec 11 21:53:06 mercat1 badcode[51165]: stack overflow detected; terminated Dec 11 21:53:06 mercat1 badcode[51169]: stack overflow detected; terminated Dec 11 21:53:06 mercat1 badcode[51172]: stack overflow detected; terminated Dec 11 21:53:06 mercat1 badcode[51176]: stack overflow detected; terminated Dec 11 21:53:06 mercat1 badcode[51205]: stack overflow detected; terminated Dec 11 21:53:06 mercat1 badcode[51208]: stack overflow detected; terminated Dec 11 21:53:06 mercat1 badcode[51218]: stack overflow detected; terminated Dec 11 21:53:06 mercat1 badcode[51220]: stack overflow detected; terminated Dec 11 21:53:06 mercat1 badcode[51254]: stack overflow detected; terminated Dec 11 21:53:06 mercat1 badcode[51256]: stack overflow detected; terminated Dec 11 21:53:06 mercat1 badcode[51258]: stack overflow detected; terminated Dec 11 21:53:06 mercat1 badcode[51262]: stack overflow detected; terminated Dec 11 21:53:06 mercat1 badcode[51263]: stack overflow detected; terminated Dec 11 21:53:06 mercat1 badcode[51273]: stack overflow detected; terminated Dec 11 21:53:06 mercat1 badcode[51294]: stack overflow detected; terminated Dec 11 21:53:07 mercat1 badcode[51304]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52533]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52534]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52535]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52549]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52557]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52559]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52589]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52590]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52591]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52593]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52599]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52600]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52610]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52616]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52617]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52627]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52634]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52647]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52653]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52660]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52668]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52676]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52704]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52713]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52725]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52731]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52735]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52747]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52758]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52761]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52764]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52765]: stack overflow detected; terminated Dec 11 21:55:15 mercat1 badcode[52767]: stack overflow detected; terminated Dec 11 21:55:16 mercat1 badcode[52770]: stack overflow detected; terminated Dec 11 21:55:16 mercat1 badcode[52771]: stack overflow detected; terminated Dec 11 21:55:16 mercat1 badcode[52779]: stack overflow detected; terminated Dec 11 21:55:16 mercat1 badcode[52782]: stack overflow detected; terminated Dec 11 21:55:16 mercat1 badcode[52793]: stack overflow detected; terminated Dec 11 21:55:16 mercat1 badcode[52812]: stack overflow detected; terminated Dec 11 21:55:16 mercat1 badcode[52813]: stack overflow detected; terminated Dec 11 21:55:16 mercat1 badcode[52841]: stack overflow detected; terminated Dec 11 21:55:16 mercat1 badcode[52864]: stack overflow detected; terminated Dec 11 21:55:16 mercat1 badcode[52865]: stack overflow detected; terminated Dec 11 21:55:16 mercat1 badcode[52896]: stack overflow detected; terminated Dec 11 21:55:16 mercat1 badcode[52901]: stack overflow detected; terminated Dec 11 21:55:16 mercat1 badcode[52903]: stack overflow detected; terminated Dec 11 21:55:16 mercat1 badcode[52909]: stack overflow detected; terminated Dec 11 21:55:16 mercat1 badcode[52914]: stack overflow detected; terminated Dec 11 21:55:16 mercat1 badcode[52919]: stack overflow detected; terminated Dec 11 21:55:16 mercat1 badcode[52922]: stack overflow detected; terminated Dec 11 21:55:16 mercat1 badcode[52925]: stack overflow detected; terminated Dec 11 21:55:16 mercat1 badcode[52926]: stack overflow detected; terminated Dec 11 21:55:16 mercat1 badcode[52935]: stack overflow detected; terminated Dec 11 21:55:16 mercat1 badcode[52936]: stack overflow detected; terminated Dec 11 21:55:16 mercat1 badcode[52940]: stack overflow detected; terminated Dec 11 21:55:17 mercat1 badcode[52981]: stack overflow detected; terminated Dec 11 21:55:17 mercat1 badcode[52982]: stack overflow detected; terminated Dec 11 21:55:17 mercat1 badcode[53004]: stack overflow detected; terminated Dec 11 21:55:18 mercat1 badcode[53016]: stack overflow detected; terminated Dec 11 21:55:18 mercat1 badcode[53023]: stack overflow detected; terminated Dec 11 21:55:18 mercat1 badcode[53036]: stack overflow detected; terminated Dec 11 21:55:18 mercat1 badcode[53041]: stack overflow detected; terminated Dec 11 21:55:18 mercat1 badcode[53048]: stack overflow detected; terminated Dec 11 21:55:18 mercat1 badcode[53051]: stack overflow detected; terminated Dec 11 21:55:19 mercat1 badcode[53072]: stack overflow detected; terminated Dec 11 21:55:19 mercat1 badcode[53082]: stack overflow detected; terminated Dec 11 21:55:19 mercat1 badcode[53080]: stack overflow detected; terminated Dec 11 21:55:19 mercat1 badcode[53112]: stack overflow detected; terminated Dec 11 21:55:19 mercat1 badcode[53114]: stack overflow detected; terminated Dec 11 21:55:19 mercat1 badcode[53119]: stack overflow detected; terminated Dec 11 21:55:19 mercat1 badcode[53129]: stack overflow detected; terminated Dec 11 21:55:19 mercat1 badcode[53132]: stack overflow detected; terminated Dec 11 21:55:20 mercat1 badcode[53138]: stack overflow detected; terminated Dec 11 21:55:20 mercat1 badcode[53140]: stack overflow detected; terminated Dec 11 21:55:20 mercat1 badcode[53146]: stack overflow detected; terminated Dec 11 21:55:20 mercat1 badcode[53148]: stack overflow detected; terminated Dec 11 21:55:20 mercat1 badcode[53154]: stack overflow detected; terminated Dec 11 21:55:21 mercat1 badcode[53161]: stack overflow detected; terminated Dec 11 21:55:21 mercat1 badcode[53162]: stack overflow detected; terminated Dec 11 21:55:21 mercat1 badcode[53166]: stack overflow detected; terminated Dec 11 21:55:22 mercat1 badcode[53172]: stack overflow detected; terminated Dec 11 21:55:24 mercat1 badcode[53199]: stack overflow detected; terminated Dec 11 21:55:25 mercat1 badcode[53212]: stack overflow detected; terminated Dec 11 21:55:25 mercat1 badcode[53216]: stack overflow detected; terminated Dec 11 21:55:25 mercat1 badcode[53218]: stack overflow detected; terminated Dec 11 21:55:26 mercat1 badcode[53248]: stack overflow detected; terminated Dec 11 21:55:27 mercat1 badcode[53279]: stack overflow detected; terminated Dec 11 21:55:27 mercat1 badcode[53280]: stack overflow detected; terminated Dec 11 21:55:28 mercat1 badcode[53320]: stack overflow detected; terminated Dec 11 21:55:28 mercat1 badcode[53338]: stack overflow detected; terminated Dec 11 21:55:28 mercat1 badcode[53343]: stack overflow detected; terminated Dec 11 21:55:28 mercat1 badcode[53345]: stack overflow detected; terminated Dec 11 21:55:29 mercat1 badcode[53364]: stack overflow detected; terminated Dec 11 21:55:29 mercat1 badcode[53374]: stack overflow detected; terminated Dec 11 21:55:29 mercat1 badcode[53380]: stack overflow detected; terminated Dec 11 21:55:30 mercat1 badcode[53389]: stack overflow detected; terminated Dec 11 21:55:32 mercat1 badcode[53483]: stack overflow detected; terminated Dec 11 21:55:33 mercat1 badcode[53484]: stack overflow detected; terminated Dec 11 21:55:35 mercat1 badcode[53596]: stack overflow detected; terminated Dec 11 21:57:20 mercat1 badcode[55873]: stack overflow detected; terminated Dec 11 21:57:20 mercat1 badcode[55895]: stack overflow detected; terminated Dec 11 21:57:20 mercat1 badcode[55898]: stack overflow detected; terminated Dec 11 21:57:21 mercat1 badcode[56229]: stack overflow detected; terminated Dec 11 21:57:21 mercat1 badcode[56234]: stack overflow detected; terminated Dec 11 21:57:21 mercat1 badcode[56238]: stack overflow detected; terminated Dec 11 21:57:21 mercat1 badcode[56244]: stack overflow detected; terminated Dec 11 21:57:21 mercat1 badcode[56270]: stack overflow detected; terminated Dec 11 21:57:21 mercat1 badcode[56288]: stack overflow detected; terminated Dec 11 21:57:21 mercat1 badcode[56300]: stack overflow detected; terminated Dec 11 21:57:21 mercat1 badcode[56308]: stack overflow detected; terminated Dec 11 21:57:21 mercat1 badcode[56323]: stack overflow detected; terminated Dec 11 21:57:21 mercat1 badcode[56346]: stack overflow detected; terminated Dec 11 21:57:21 mercat1 badcode[56360]: stack overflow detected; terminated Dec 11 21:57:21 mercat1 badcode[56365]: stack overflow detected; terminated Dec 11 21:57:21 mercat1 badcode[56367]: stack overflow detected; terminated Dec 11 21:57:21 mercat1 badcode[56550]: stack overflow detected; terminated Dec 11 21:57:21 mercat1 badcode[56555]: stack overflow detected; terminated Dec 11 21:57:21 mercat1 badcode[56560]: stack overflow detected; terminated Dec 11 21:57:21 mercat1 badcode[56565]: stack overflow detected; terminated Dec 11 21:57:21 mercat1 badcode[56572]: stack overflow detected; terminated Dec 11 21:57:22 mercat1 badcode[56606]: stack overflow detected; terminated Dec 11 21:57:26 mercat1 badcode[56685]: stack overflow detected; terminated Dec 11 21:57:27 mercat1 badcode[56708]: stack overflow detected; terminated Dec 11 21:57:28 mercat1 badcode[56736]: stack overflow detected; terminated Dec 11 21:57:28 mercat1 badcode[56742]: stack overflow detected; terminated Dec 11 21:59:06 mercat1 kernel: pid 55711 (swap), jid 0, uid 2007, was killed: out of swap space Dec 11 21:59:20 mercat1 kernel: pid 55701 (swap), jid 0, uid 2007, was killed: out of swap space 20191211 21:59:30 all (33/121): nullfs27.sh Dec 11 22:03:55 mercat1 kernel: pid 62678 (rw), uid 2007 inumber 112677 on /mnt: filesystem full Dec 11 22:04:01 mercat1 kernel: pid 62686 (rw), uid 2007 inumber 87746 on /mnt: filesystem full Dec 11 22:06:06 mercat1 kernel: pid 66445 (rw), uid 2007 inumber 168805 on /mnt: filesystem full 20191211 22:10:09 all (34/121): md7.sh Dec 11 22:12:26 mercat1 kernel: pid 77320 (rw), uid 0 inumber 155874 on /mnt: filesystem full Dec 11 22:12:31 mercat1 kernel: pid 77368 (rw), uid 0 inumber 15809 on /mnt: filesystem full Dec 11 22:12:39 mercat1 kernel: pid 77288 (creat), uid 0 inumber 134435 on /mnt: out of inodes Dec 11 22:12:51 mercat1 kernel: pid 77298 (mkdir), uid 0 inumber 89600 on /mnt: out of inodes Dec 11 22:13:14 mercat1 kernel: pid 77318 (creat), uid 0 inumber 134436 on /mnt: out of inodes Dec 11 22:13:46 mercat1 kernel: pid 77346 (creat), uid 0 inumber 134441 on /mnt: out of inodes Dec 11 22:13:59 mercat1 kernel: pid 77283 (creat), uid 0 inumber 134421 on /mnt: out of inodes Dec 11 22:14:13 mercat1 syslogd: last message repeated 1 times Dec 11 22:16:37 mercat1 kernel: pid 83278 (rw), uid 0 inumber 146644 on /mnt: filesystem full Dec 11 22:18:40 mercat1 kernel: pid 85912 (rw), uid 0 inumber 131082 on /mnt: filesystem full 20191211 22:20:34 all (35/121): extattr2.sh Dec 11 22:21:03 mercat1 kernel: pid 88138 (swap), jid 0, uid 2007, was killed: out of swap space Dec 11 22:21:29 mercat1 kernel: pid 88151 (swap), jid 0, uid 2007, was killed: out of swap space Dec 11 22:22:49 mercat1 kernel: pid 91158 (rw), uid 2007 inumber 117521 on /mnt: filesystem full Dec 11 22:22:55 mercat1 kernel: pid 91163 (rw), uid 2007 inumber 111840 on /mnt: filesystem full Dec 11 22:23:07 mercat1 kernel: pid 91160 (rw), uid 2007 inumber 182962 on /mnt: filesystem full Dec 11 22:25:32 mercat1 kernel: pid 95019 (swap), jid 0, uid 2007, was killed: out of swap space Dec 11 22:25:43 mercat1 kernel: pid 95052 (swap), jid 0, uid 2007, was killed: out of swap space Dec 11 22:27:29 mercat1 kernel: pid 96591 (swap), jid 0, uid 2007, was killed: out of swap space Dec 11 22:29:28 mercat1 kernel: pid 98042 (swap), jid 0, uid 2007, was killed: out of swap space Dec 11 22:29:49 mercat1 kernel: pid 98053 (swap), jid 0, uid 2007, was killed: out of swap space Dec 11 22:30:02 mercat1 kernel: pid 98068 (swap), jid 0, uid 2007, was killed: out of swap space 20191211 22:31:13 all (36/121): oovm2.sh freework leak: 14/45. 20191211 22:41:44 all (37/121): sendfile14.sh lock order reversal: 1st 0xfffff800446af988 so_snd_sx (so_snd_sx) @ kern/uipc_sockbuf.c:390 2nd 0xfffff80652608068 ufs (ufs) @ kern/kern_sendfile.c:762 stack backtrace: #0 0xffffffff80c32581 at witness_debugger+0x71 #1 0xffffffff80c322f6 at witness_checkorder+0xad6 #2 0xffffffff80b97d8e at lockmgr_slock_hard+0x6e #3 0xffffffff80b98eb2 at __lockmgr_args+0x762 #4 0xffffffff80ee646c at ffs_lock+0x7c #5 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #6 0xffffffff80cb2b75 at _vn_lock+0x65 #7 0xffffffff80bc2023 at vn_sendfile+0x6b3 #8 0xffffffff80bc3b37 at sendfile+0x127 #9 0xffffffff8106b901 at amd64_syscall+0x2f1 #10 0xffffffff810416c0 at fast_syscall_common+0x101 20191211 22:52:21 all (38/121): full.sh Dec 11 22:52:25 mercat1 badcode[64350]: stack overflow detected; terminated Dec 11 22:52:26 mercat1 badcode[64374]: stack overflow detected; terminated Dec 11 22:52:26 mercat1 badcode[64386]: stack overflow detected; terminated Dec 11 22:52:26 mercat1 badcode[64422]: stack overflow detected; terminated Dec 11 22:52:27 mercat1 badcode[64437]: stack overflow detected; terminated Dec 11 22:52:27 mercat1 badcode[64448]: stack overflow detected; terminated Dec 11 22:52:27 mercat1 badcode[64461]: stack overflow detected; terminated Dec 11 22:52:30 mercat1 badcode[64473]: stack overflow detected; terminated Dec 11 22:52:30 mercat1 badcode[64474]: stack overflow detected; terminated Dec 11 22:52:30 mercat1 badcode[64479]: stack overflow detected; terminated Dec 11 22:52:30 mercat1 badcode[64496]: stack overflow detected; terminated Dec 11 22:52:31 mercat1 badcode[64494]: stack overflow detected; terminated Dec 11 22:52:32 mercat1 badcode[64526]: stack overflow detected; terminated Dec 11 22:52:32 mercat1 badcode[64555]: stack overflow detected; terminated Dec 11 22:52:34 mercat1 badcode[64583]: stack overflow detected; terminated Dec 11 22:52:34 mercat1 badcode[64605]: stack overflow detected; terminated Dec 11 22:52:35 mercat1 badcode[64613]: stack overflow detected; terminated Dec 11 22:52:35 mercat1 badcode[64614]: stack overflow detected; terminated Dec 11 22:52:35 mercat1 badcode[64623]: stack overflow detected; terminated Dec 11 22:52:35 mercat1 badcode[64640]: stack overflow detected; terminated Dec 11 22:52:35 mercat1 badcode[64644]: stack overflow detected; terminated Dec 11 22:52:36 mercat1 badcode[64582]: stack overflow detected; terminated Dec 11 22:52:36 mercat1 badcode[64660]: stack overflow detected; terminated Dec 11 22:52:37 mercat1 badcode[64665]: stack overflow detected; terminated Dec 11 22:52:38 mercat1 badcode[64674]: stack overflow detected; terminated Dec 11 22:52:38 mercat1 badcode[64667]: stack overflow detected; terminated Dec 11 22:52:38 mercat1 badcode[64661]: stack overflow detected; terminated Dec 11 22:52:39 mercat1 badcode[64685]: stack overflow detected; terminated Dec 11 22:52:39 mercat1 badcode[64675]: stack overflow detected; terminated Dec 11 22:52:40 mercat1 badcode[64694]: stack overflow detected; terminated Dec 11 22:52:40 mercat1 badcode[64695]: stack overflow detected; terminated Dec 11 22:52:41 mercat1 badcode[64706]: stack overflow detected; terminated Dec 11 22:52:41 mercat1 badcode[64707]: stack overflow detected; terminated Dec 11 22:52:42 mercat1 badcode[64719]: stack overflow detected; terminated Dec 11 22:52:43 mercat1 badcode[64724]: stack overflow detected; terminated Dec 11 22:52:43 mercat1 badcode[64730]: stack overflow detected; terminated Dec 11 22:52:45 mercat1 badcode[64738]: stack overflow detected; terminated Dec 11 22:52:47 mercat1 badcode[64774]: stack overflow detected; terminated Dec 11 22:52:47 mercat1 badcode[64775]: stack overflow detected; terminated Dec 11 22:52:48 mercat1 badcode[64794]: stack overflow detected; terminated Dec 11 22:52:48 mercat1 badcode[64798]: stack overflow detected; terminated Dec 11 22:52:49 mercat1 badcode[64799]: stack overflow detected; terminated Dec 11 22:52:49 mercat1 badcode[64802]: stack overflow detected; terminated Dec 11 22:52:49 mercat1 badcode[64814]: stack overflow detected; terminated Dec 11 22:52:51 mercat1 badcode[64862]: stack overflow detected; terminated Dec 11 22:52:51 mercat1 badcode[64872]: stack overflow detected; terminated Dec 11 22:52:52 mercat1 badcode[64881]: stack overflow detected; terminated Dec 11 22:52:52 mercat1 badcode[64882]: stack overflow detected; terminated Dec 11 22:52:52 mercat1 badcode[64889]: stack overflow detected; terminated Dec 11 22:52:52 mercat1 badcode[64903]: stack overflow detected; terminated Dec 11 22:52:52 mercat1 badcode[64912]: stack overflow detected; terminated Dec 11 22:52:53 mercat1 badcode[64913]: stack overflow detected; terminated Dec 11 22:52:54 mercat1 badcode[64946]: stack overflow detected; terminated Dec 11 22:52:54 mercat1 badcode[64943]: stack overflow detected; terminated Dec 11 22:52:55 mercat1 badcode[64960]: stack overflow detected; terminated Dec 11 22:52:56 mercat1 badcode[64998]: stack overflow detected; terminated Dec 11 22:52:57 mercat1 badcode[65029]: stack overflow detected; terminated Dec 11 22:52:57 mercat1 badcode[65036]: stack overflow detected; terminated Dec 11 22:52:57 mercat1 badcode[65032]: stack overflow detected; terminated Dec 11 22:52:58 mercat1 badcode[65069]: stack overflow detected; terminated Dec 11 22:52:59 mercat1 badcode[65078]: stack overflow detected; terminated Dec 11 22:53:01 mercat1 badcode[65116]: stack overflow detected; terminated Dec 11 22:53:01 mercat1 badcode[65123]: stack overflow detected; terminated Dec 11 22:53:02 mercat1 badcode[65131]: stack overflow detected; terminated Dec 11 22:53:03 mercat1 badcode[65145]: stack overflow detected; terminated Dec 11 22:53:03 mercat1 badcode[65150]: stack overflow detected; terminated Dec 11 22:53:03 mercat1 badcode[65154]: stack overflow detected; terminated Dec 11 22:53:04 mercat1 badcode[65175]: stack overflow detected; terminated Dec 11 22:53:05 mercat1 badcode[65211]: stack overflow detected; terminated Dec 11 22:53:05 mercat1 badcode[65225]: stack overflow detected; terminated Dec 11 22:53:06 mercat1 badcode[65244]: stack overflow detected; terminated Dec 11 22:53:06 mercat1 badcode[65251]: stack overflow detected; terminated Dec 11 22:53:07 mercat1 badcode[65283]: stack overflow detected; terminated Dec 11 22:53:09 mercat1 badcode[65330]: stack overflow detected; terminated Dec 11 22:53:09 mercat1 badcode[65337]: stack overflow detected; terminated Dec 11 22:53:09 mercat1 badcode[65341]: stack overflow detected; terminated Dec 11 22:53:09 mercat1 badcode[65345]: stack overflow detected; terminated Dec 11 22:53:09 mercat1 badcode[65351]: stack overflow detected; terminated Dec 11 22:53:09 mercat1 badcode[65352]: stack overflow detected; terminated Dec 11 22:53:11 mercat1 badcode[65385]: stack overflow detected; terminated Dec 11 22:53:11 mercat1 badcode[65393]: stack overflow detected; terminated Dec 11 22:53:11 mercat1 badcode[65394]: stack overflow detected; terminated Dec 11 22:53:12 mercat1 badcode[65403]: stack overflow detected; terminated Dec 11 22:53:13 mercat1 badcode[65418]: stack overflow detected; terminated Dec 11 22:53:14 mercat1 badcode[65430]: stack overflow detected; terminated Dec 11 22:53:15 mercat1 badcode[65442]: stack overflow detected; terminated Dec 11 22:53:15 mercat1 badcode[65451]: stack overflow detected; terminated Dec 11 22:53:16 mercat1 badcode[65462]: stack overflow detected; terminated Dec 11 22:53:19 mercat1 badcode[65507]: stack overflow detected; terminated Dec 11 22:53:19 mercat1 badcode[65522]: stack overflow detected; terminated Dec 11 22:56:45 mercat1 badcode[69418]: stack overflow detected; terminated Dec 11 22:56:45 mercat1 badcode[69451]: stack overflow detected; terminated Dec 11 22:56:45 mercat1 badcode[69455]: stack overflow detected; terminated Dec 11 22:56:45 mercat1 badcode[69456]: stack overflow detected; terminated Dec 11 22:56:45 mercat1 badcode[69477]: stack overflow detected; terminated Dec 11 22:56:46 mercat1 badcode[69497]: stack overflow detected; terminated Dec 11 22:56:46 mercat1 badcode[69500]: stack overflow detected; terminated Dec 11 22:56:46 mercat1 badcode[69504]: stack overflow detected; terminated Dec 11 22:56:46 mercat1 badcode[69507]: stack overflow detected; terminated Dec 11 22:56:46 mercat1 badcode[69530]: stack overflow detected; terminated Dec 11 22:56:46 mercat1 badcode[69531]: stack overflow detected; terminated Dec 11 22:56:47 mercat1 badcode[69568]: stack overflow detected; terminated Dec 11 22:56:49 mercat1 badcode[69593]: stack overflow detected; terminated Dec 11 22:56:49 mercat1 badcode[69596]: stack overflow detected; terminated Dec 11 22:56:49 mercat1 badcode[69613]: stack overflow detected; terminated Dec 11 22:56:51 mercat1 badcode[69642]: stack overflow detected; terminated Dec 11 22:56:52 mercat1 badcode[69655]: stack overflow detected; terminated Dec 11 22:56:52 mercat1 badcode[69691]: stack overflow detected; terminated Dec 11 22:56:52 mercat1 badcode[69699]: stack overflow detected; terminated Dec 11 22:56:52 mercat1 badcode[69706]: stack overflow detected; terminated Dec 11 22:56:52 mercat1 badcode[69712]: stack overflow detected; terminated Dec 11 22:56:52 mercat1 badcode[69709]: stack overflow detected; terminated Dec 11 22:56:52 mercat1 badcode[69719]: stack overflow detected; terminated Dec 11 22:56:52 mercat1 badcode[69732]: stack overflow detected; terminated Dec 11 22:56:56 mercat1 badcode[69752]: stack overflow detected; terminated Dec 11 22:56:56 mercat1 badcode[69753]: stack overflow detected; terminated Dec 11 22:56:56 mercat1 badcode[69758]: stack overflow detected; terminated Dec 11 22:56:56 mercat1 badcode[69763]: stack overflow detected; terminated Dec 11 22:56:59 mercat1 badcode[69795]: stack overflow detected; terminated Dec 11 22:57:02 mercat1 badcode[69820]: stack overflow detected; terminated Dec 11 22:57:05 mercat1 badcode[69854]: stack overflow detected; terminated Dec 11 22:57:05 mercat1 badcode[69859]: stack overflow detected; terminated Dec 11 22:57:06 mercat1 badcode[69866]: stack overflow detected; terminated Dec 11 22:57:06 mercat1 badcode[69871]: stack overflow detected; terminated Dec 11 22:57:08 mercat1 badcode[69891]: stack overflow detected; terminated Dec 11 22:57:12 mercat1 badcode[69923]: stack overflow detected; terminated Dec 11 22:57:16 mercat1 badcode[69970]: stack overflow detected; terminated Dec 11 22:57:16 mercat1 badcode[69980]: stack overflow detected; terminated Dec 11 22:57:17 mercat1 badcode[69982]: stack overflow detected; terminated Dec 11 22:57:19 mercat1 badcode[70001]: stack overflow detected; terminated Dec 11 22:57:21 mercat1 badcode[70031]: stack overflow detected; terminated Dec 11 22:57:24 mercat1 kernel: pid 69276 (rw), uid 2007 inumber 211974 on /mnt: filesystem full Expensive timeout(9) function: 0xffffffff80e03aa0(0xfffff801e4bf63e8) 0.434994423 s Dec 11 22:57:53 mercat1 kernel: pid 69241 (mmap), uid 2007 inumber 214669 on /mnt: filesystem full Dec 11 23:00:56 mercat1 badcode[73469]: stack overflow detected; terminated Dec 11 23:00:56 mercat1 badcode[73487]: stack overflow detected; terminated Dec 11 23:00:57 mercat1 badcode[73510]: stack overflow detected; terminated Dec 11 23:00:57 mercat1 badcode[73524]: stack overflow detected; terminated Dec 11 23:00:57 mercat1 badcode[73535]: stack overflow detected; terminated Dec 11 23:00:57 mercat1 badcode[73539]: stack overflow detected; terminated Dec 11 23:00:58 mercat1 badcode[73570]: stack overflow detected; terminated Dec 11 23:00:58 mercat1 badcode[73579]: stack overflow detected; terminated Dec 11 23:00:58 mercat1 badcode[73585]: stack overflow detected; terminated Dec 11 23:00:58 mercat1 badcode[73625]: stack overflow detected; terminated Dec 11 23:00:59 mercat1 badcode[73693]: stack overflow detected; terminated Dec 11 23:00:59 mercat1 badcode[73713]: stack overflow detected; terminated Dec 11 23:00:59 mercat1 badcode[73708]: stack overflow detected; terminated Dec 11 23:00:59 mercat1 badcode[73714]: stack overflow detected; terminated Dec 11 23:00:59 mercat1 badcode[73701]: stack overflow detected; terminated Dec 11 23:00:59 mercat1 badcode[73712]: stack overflow detected; terminated Dec 11 23:00:59 mercat1 badcode[73724]: stack overflow detected; terminated Dec 11 23:00:59 mercat1 badcode[73727]: stack overflow detected; terminated Dec 11 23:00:59 mercat1 badcode[73750]: stack overflow detected; terminated Dec 11 23:00:59 mercat1 badcode[73754]: stack overflow detected; terminated Dec 11 23:00:59 mercat1 badcode[73761]: stack overflow detected; terminated Dec 11 23:00:59 mercat1 badcode[73763]: stack overflow detected; terminated Dec 11 23:00:59 mercat1 badcode[73771]: stack overflow detected; terminated Dec 11 23:01:00 mercat1 badcode[73816]: stack overflow detected; terminated Dec 11 23:01:00 mercat1 badcode[73832]: stack overflow detected; terminated Dec 11 23:01:00 mercat1 badcode[73847]: stack overflow detected; terminated Dec 11 23:01:00 mercat1 badcode[73860]: stack overflow detected; terminated Dec 11 23:01:00 mercat1 badcode[73865]: stack overflow detected; terminated Dec 11 23:01:00 mercat1 badcode[73875]: stack overflow detected; terminated Dec 11 23:01:01 mercat1 badcode[73912]: stack overflow detected; terminated Dec 11 23:01:01 mercat1 badcode[73919]: stack overflow detected; terminated Dec 11 23:01:01 mercat1 badcode[73920]: stack overflow detected; terminated Dec 11 23:01:01 mercat1 badcode[73925]: stack overflow detected; terminated Dec 11 23:01:02 mercat1 badcode[73936]: stack overflow detected; terminated Dec 11 23:01:02 mercat1 badcode[73931]: stack overflow detected; terminated Dec 11 23:01:02 mercat1 badcode[73968]: stack overflow detected; terminated Dec 11 23:01:02 mercat1 badcode[73976]: stack overflow detected; terminated Dec 11 23:01:02 mercat1 badcode[73982]: stack overflow detected; terminated Dec 11 23:01:03 mercat1 badcode[74037]: stack overflow detected; terminated Dec 11 23:01:03 mercat1 badcode[74050]: stack overflow detected; terminated Dec 11 23:01:03 mercat1 badcode[74053]: stack overflow detected; terminated Dec 11 23:01:03 mercat1 badcode[74060]: stack overflow detected; terminated Dec 11 23:01:04 mercat1 badcode[74111]: stack overflow detected; terminated Dec 11 23:01:07 mercat1 badcode[74161]: stack overflow detected; terminated Dec 11 23:01:07 mercat1 badcode[74181]: stack overflow detected; terminated Dec 11 23:01:08 mercat1 badcode[74218]: stack overflow detected; terminated Dec 11 23:01:08 mercat1 badcode[74220]: stack overflow detected; terminated Dec 11 23:01:08 mercat1 badcode[74228]: stack overflow detected; terminated Dec 11 23:01:08 mercat1 badcode[74233]: stack overflow detected; terminated Dec 11 23:01:08 mercat1 badcode[74249]: stack overflow detected; terminated Dec 11 23:01:08 mercat1 badcode[74244]: stack overflow detected; terminated Dec 11 23:01:09 mercat1 badcode[74263]: stack overflow detected; terminated Dec 11 23:01:09 mercat1 badcode[74289]: stack overflow detected; terminated Dec 11 23:01:10 mercat1 badcode[74306]: stack overflow detected; terminated Dec 11 23:01:10 mercat1 badcode[74316]: stack overflow detected; terminated Dec 11 23:01:10 mercat1 badcode[74301]: stack overflow detected; terminated Dec 11 23:01:12 mercat1 badcode[74380]: stack overflow detected; terminated Dec 11 23:01:12 mercat1 badcode[74381]: stack overflow detected; terminated Dec 11 23:01:14 mercat1 badcode[74448]: stack overflow detected; terminated Dec 11 23:01:15 mercat1 badcode[74472]: stack overflow detected; terminated Dec 11 23:01:15 mercat1 badcode[74498]: stack overflow detected; terminated Dec 11 23:01:16 mercat1 badcode[74504]: stack overflow detected; terminated Dec 11 23:01:17 mercat1 badcode[74536]: stack overflow detected; terminated Dec 11 23:01:17 mercat1 badcode[74543]: stack overflow detected; terminated Dec 11 23:01:17 mercat1 badcode[74544]: stack overflow detected; terminated Dec 11 23:01:17 mercat1 badcode[74549]: stack overflow detected; terminated Dec 11 23:01:17 mercat1 badcode[74551]: stack overflow detected; terminated Dec 11 23:01:18 mercat1 badcode[74568]: stack overflow detected; terminated Dec 11 23:01:18 mercat1 badcode[74576]: stack overflow detected; terminated Dec 11 23:01:19 mercat1 badcode[74598]: stack overflow detected; terminated Dec 11 23:01:19 mercat1 badcode[74619]: stack overflow detected; terminated Dec 11 23:01:20 mercat1 badcode[74631]: stack overflow detected; terminated Dec 11 23:01:21 mercat1 badcode[74674]: stack overflow detected; terminated Dec 11 23:01:22 mercat1 badcode[74692]: stack overflow detected; terminated Dec 11 23:01:23 mercat1 badcode[74750]: stack overflow detected; terminated Dec 11 23:01:23 mercat1 badcode[74761]: stack overflow detected; terminated Dec 11 23:01:24 mercat1 badcode[74776]: stack overflow detected; terminated Dec 11 23:01:24 mercat1 badcode[74792]: stack overflow detected; terminated Dec 11 23:01:24 mercat1 badcode[74797]: stack overflow detected; terminated Dec 11 23:01:24 mercat1 badcode[74806]: stack overflow detected; terminated Dec 11 23:01:24 mercat1 badcode[74805]: stack overflow detected; terminated Dec 11 23:01:24 mercat1 badcode[74812]: stack overflow detected; terminated Dec 11 23:01:24 mercat1 badcode[74815]: stack overflow detected; terminated Dec 11 23:01:24 mercat1 badcode[74819]: stack overflow detected; terminated Dec 11 23:01:24 mercat1 badcode[74826]: stack overflow detected; terminated Dec 11 23:01:25 mercat1 badcode[74845]: stack overflow detected; terminated Dec 11 23:01:25 mercat1 badcode[74860]: stack overflow detected; terminated Dec 11 23:01:26 mercat1 badcode[74878]: stack overflow detected; terminated Dec 11 23:01:27 mercat1 badcode[74922]: stack overflow detected; terminated pts leak: 1/3 20191211 23:03:05 all (39/121): nullfs20.sh 20191211 23:13:23 all (40/121): pfl2.sh Dec 11 23:14:07 mercat1 kernel: pid 24373 (swap), jid 0, uid 2007, was killed: out of swap space Dec 11 23:16:56 mercat1 kernel: pid 27173 (swap), jid 0, uid 2007, was killed: out of swap space Dec 11 23:20:03 mercat1 kernel: pid 33367 (swap), jid 0, uid 2007, was killed: out of swap space Dec 11 23:20:41 mercat1 kernel: pid 33279 (swap), jid 0, uid 2007, was killed: out of swap space 20191211 23:24:00 all (41/121): datamove2.sh Dec 11 23:24:46 mercat1 kernel: pid 42891 (swap), jid 0, uid 0, was killed: out of swap space Dec 11 23:25:10 mercat1 kernel: pid 42990 (swap), jid 0, uid 0, was killed: out of swap space Dec 11 23:28:57 mercat1 kernel: pid 42914 (swap), jid 0, uid 0, was killed: out of swap space 20191211 23:29:32 all (42/121): ext2fs.sh Dec 11 23:32:36 mercat1 kernel: pid 49442 (swap), jid 0, uid 2007, was killed: out of swap space Dec 11 23:33:08 mercat1 kernel: pid 49460 (swap), jid 0, uid 2007, was killed: out of swap space Dec 11 23:36:22 mercat1 kernel: pid 55570 (swap), jid 0, uid 2007, was killed: out of swap space Dec 11 23:36:49 mercat1 kernel: pid 55649 (swap), jid 0, uid 2007, was killed: out of swap space 20191211 23:40:04 all (43/121): tmpfs20.sh Dec 11 23:40:09 mercat1 badcode[59097]: stack overflow detected; terminated Dec 11 23:40:09 mercat1 badcode[59104]: stack overflow detected; terminated Dec 11 23:40:09 mercat1 badcode[59106]: stack overflow detected; terminated Dec 11 23:40:10 mercat1 badcode[59117]: stack overflow detected; terminated Dec 11 23:40:10 mercat1 badcode[59120]: stack overflow detected; terminated Dec 11 23:40:10 mercat1 badcode[59123]: stack overflow detected; terminated Dec 11 23:40:10 mercat1 badcode[59144]: stack overflow detected; terminated Dec 11 23:40:10 mercat1 badcode[59147]: stack overflow detected; terminated Dec 11 23:40:10 mercat1 badcode[59174]: stack overflow detected; terminated Dec 11 23:40:10 mercat1 badcode[59177]: stack overflow detected; terminated Dec 11 23:40:10 mercat1 badcode[59181]: stack overflow detected; terminated Dec 11 23:40:10 mercat1 badcode[59182]: stack overflow detected; terminated Dec 11 23:40:10 mercat1 badcode[59186]: stack overflow detected; terminated Dec 11 23:40:10 mercat1 badcode[59198]: stack overflow detected; terminated Dec 11 23:40:10 mercat1 badcode[59200]: stack overflow detected; terminated Dec 11 23:40:10 mercat1 badcode[59208]: stack overflow detected; terminated Dec 11 23:42:13 mercat1 badcode[62625]: stack overflow detected; terminated Dec 11 23:42:13 mercat1 badcode[62659]: stack overflow detected; terminated Dec 11 23:42:13 mercat1 badcode[62666]: stack overflow detected; terminated Dec 11 23:42:17 mercat1 badcode[62713]: stack overflow detected; terminated Dec 11 23:42:18 mercat1 badcode[62742]: stack overflow detected; terminated Dec 11 23:42:18 mercat1 badcode[62743]: stack overflow detected; terminated Dec 11 23:42:19 mercat1 badcode[62752]: stack overflow detected; terminated Dec 11 23:42:19 mercat1 badcode[62755]: stack overflow detected; terminated Dec 11 23:42:22 mercat1 badcode[62830]: stack overflow detected; terminated Dec 11 23:42:23 mercat1 badcode[62835]: stack overflow detected; terminated Dec 11 23:42:23 mercat1 badcode[62862]: stack overflow detected; terminated Dec 11 23:42:23 mercat1 badcode[62860]: stack overflow detected; terminated Dec 11 23:42:23 mercat1 badcode[62868]: stack overflow detected; terminated Dec 11 23:42:24 mercat1 badcode[62887]: stack overflow detected; terminated Dec 11 23:42:24 mercat1 badcode[62892]: stack overflow detected; terminated Dec 11 23:42:25 mercat1 badcode[62902]: stack overflow detected; terminated Dec 11 23:42:25 mercat1 badcode[62908]: stack overflow detected; terminated Dec 11 23:42:26 mercat1 badcode[62922]: stack overflow detected; terminated Dec 11 23:42:26 mercat1 badcode[62925]: stack overflow detected; terminated Dec 11 23:42:27 mercat1 badcode[62934]: stack overflow detected; terminated Dec 11 23:42:28 mercat1 badcode[62949]: stack overflow detected; terminated Dec 11 23:42:29 mercat1 badcode[62958]: stack overflow detected; terminated Dec 11 23:42:29 mercat1 badcode[62966]: stack overflow detected; terminated Dec 11 23:42:38 mercat1 badcode[63083]: stack overflow detected; terminated Dec 11 23:42:41 mercat1 badcode[63131]: stack overflow detected; terminated Dec 11 23:42:41 mercat1 badcode[63136]: stack overflow detected; terminated Dec 11 23:42:41 mercat1 badcode[63139]: stack overflow detected; terminated Dec 11 23:42:42 mercat1 badcode[63160]: stack overflow detected; terminated Dec 11 23:42:43 mercat1 badcode[63171]: stack overflow detected; terminated Dec 11 23:42:43 mercat1 badcode[63181]: stack overflow detected; terminated Dec 11 23:42:48 mercat1 badcode[63263]: stack overflow detected; terminated witness_lock_list_get: witness exhausted Dec 11 23:44:19 mercat1 badcode[64718]: stack overflow detected; terminated Dec 11 23:44:19 mercat1 badcode[64742]: stack overflow detected; terminated Dec 11 23:44:19 mercat1 badcode[64754]: stack overflow detected; terminated Dec 11 23:44:19 mercat1 badcode[64765]: stack overflow detected; terminated Dec 11 23:44:19 mercat1 badcode[64773]: stack overflow detected; terminated Dec 11 23:44:19 mercat1 badcode[64774]: stack overflow detected; terminated Dec 11 23:44:21 mercat1 badcode[64794]: stack overflow detected; terminated Dec 11 23:44:22 mercat1 badcode[64802]: stack overflow detected; terminated Dec 11 23:44:22 mercat1 badcode[64810]: stack overflow detected; terminated Dec 11 23:44:22 mercat1 badcode[64817]: stack overflow detected; terminated Dec 11 23:44:22 mercat1 badcode[64813]: stack overflow detected; terminated Dec 11 23:44:25 mercat1 badcode[64913]: stack overflow detected; terminated Dec 11 23:44:25 mercat1 badcode[64915]: stack overflow detected; terminated Dec 11 23:44:25 mercat1 badcode[64917]: stack overflow detected; terminated Dec 11 23:44:25 mercat1 badcode[64918]: stack overflow detected; terminated Dec 11 23:44:29 mercat1 badcode[64971]: stack overflow detected; terminated Dec 11 23:44:29 mercat1 badcode[64974]: stack overflow detected; terminated Dec 11 23:44:29 mercat1 badcode[64992]: stack overflow detected; terminated Dec 11 23:44:29 mercat1 badcode[64978]: stack overflow detected; terminated Dec 11 23:44:30 mercat1 badcode[65000]: stack overflow detected; terminated Dec 11 23:44:31 mercat1 badcode[65028]: stack overflow detected; terminated Dec 11 23:44:31 mercat1 badcode[65029]: stack overflow detected; terminated Dec 11 23:44:31 mercat1 badcode[65030]: stack overflow detected; terminated Dec 11 23:44:31 mercat1 badcode[65039]: stack overflow detected; terminated Dec 11 23:44:31 mercat1 badcode[65042]: stack overflow detected; terminated Dec 11 23:44:31 mercat1 badcode[65049]: stack overflow detected; terminated Dec 11 23:44:31 mercat1 badcode[65052]: stack overflow detected; terminated Dec 11 23:44:31 mercat1 badcode[65054]: stack overflow detected; terminated Dec 11 23:44:32 mercat1 badcode[65022]: stack overflow detected; terminated Dec 11 23:44:32 mercat1 badcode[65064]: stack overflow detected; terminated Dec 11 23:44:32 mercat1 badcode[65065]: stack overflow detected; terminated Dec 11 23:44:34 mercat1 badcode[65087]: stack overflow detected; terminated Dec 11 23:44:34 mercat1 badcode[65091]: stack overflow detected; terminated Dec 11 23:44:35 mercat1 badcode[65117]: stack overflow detected; terminated Dec 11 23:44:36 mercat1 badcode[65130]: stack overflow detected; terminated Dec 11 23:44:37 mercat1 badcode[65169]: stack overflow detected; terminated Dec 11 23:44:38 mercat1 badcode[65182]: stack overflow detected; terminated Dec 11 23:44:38 mercat1 badcode[65185]: stack overflow detected; terminated Dec 11 23:44:38 mercat1 badcode[65198]: stack overflow detected; terminated Dec 11 23:44:38 mercat1 badcode[65205]: stack overflow detected; terminated Dec 11 23:44:39 mercat1 badcode[65239]: stack overflow detected; terminated Dec 11 23:44:39 mercat1 badcode[65236]: stack overflow detected; terminated Dec 11 23:44:40 mercat1 badcode[65272]: stack overflow detected; terminated Dec 11 23:44:40 mercat1 badcode[65286]: stack overflow detected; terminated Dec 11 23:44:41 mercat1 badcode[65307]: stack overflow detected; terminated Dec 11 23:44:41 mercat1 badcode[65322]: stack overflow detected; terminated Dec 11 23:44:41 mercat1 badcode[65338]: stack overflow detected; terminated Dec 11 23:44:42 mercat1 badcode[65363]: stack overflow detected; terminated Dec 11 23:44:42 mercat1 badcode[65370]: stack overflow detected; terminated Dec 11 23:46:24 mercat1 badcode[67856]: stack overflow detected; terminated Dec 11 23:46:24 mercat1 badcode[67866]: stack overflow detected; terminated Dec 11 23:46:24 mercat1 badcode[67869]: stack overflow detected; terminated Dec 11 23:46:24 mercat1 badcode[67872]: stack overflow detected; terminated Dec 11 23:46:24 mercat1 badcode[67878]: stack overflow detected; terminated Dec 11 23:46:24 mercat1 badcode[67885]: stack overflow detected; terminated Dec 11 23:46:24 mercat1 badcode[67909]: stack overflow detected; terminated Dec 11 23:46:24 mercat1 badcode[67914]: stack overflow detected; terminated Dec 11 23:46:24 mercat1 badcode[67915]: stack overflow detected; terminated Dec 11 23:46:24 mercat1 badcode[67923]: stack overflow detected; terminated Dec 11 23:46:24 mercat1 badcode[67930]: stack overflow detected; terminated Dec 11 23:46:24 mercat1 badcode[67940]: stack overflow detected; terminated Dec 11 23:46:24 mercat1 badcode[67952]: stack overflow detected; terminated Dec 11 23:46:24 mercat1 badcode[67953]: stack overflow detected; terminated Dec 11 23:46:24 mercat1 badcode[67954]: stack overflow detected; terminated Dec 11 23:46:24 mercat1 badcode[67962]: stack overflow detected; terminated Dec 11 23:46:24 mercat1 badcode[67980]: stack overflow detected; terminated Dec 11 23:46:24 mercat1 badcode[68000]: stack overflow detected; terminated Dec 11 23:46:24 mercat1 badcode[68002]: stack overflow detected; terminated Dec 11 23:46:24 mercat1 badcode[68003]: stack overflow detected; terminated Dec 11 23:46:24 mercat1 badcode[68019]: stack overflow detected; terminated Dec 11 23:46:24 mercat1 badcode[68027]: stack overflow detected; terminated Dec 11 23:46:24 mercat1 badcode[68034]: stack overflow detected; terminated Dec 11 23:46:24 mercat1 badcode[68051]: stack overflow detected; terminated Dec 11 23:46:24 mercat1 badcode[68056]: stack overflow detected; terminated Dec 11 23:46:24 mercat1 badcode[68060]: stack overflow detected; terminated Dec 11 23:46:25 mercat1 badcode[68090]: stack overflow detected; terminated Dec 11 23:46:25 mercat1 badcode[68099]: stack overflow detected; terminated Dec 11 23:46:25 mercat1 badcode[68110]: stack overflow detected; terminated Dec 11 23:46:27 mercat1 badcode[68192]: stack overflow detected; terminated Dec 11 23:46:29 mercat1 badcode[68255]: stack overflow detected; terminated Dec 11 23:46:29 mercat1 badcode[68259]: stack overflow detected; terminated Dec 11 23:46:29 mercat1 badcode[68265]: stack overflow detected; terminated Dec 11 23:46:29 mercat1 badcode[68283]: stack overflow detected; terminated Dec 11 23:46:29 mercat1 badcode[68285]: stack overflow detected; terminated Dec 11 23:46:29 mercat1 badcode[68291]: stack overflow detected; terminated Dec 11 23:46:32 mercat1 badcode[68311]: stack overflow detected; terminated Dec 11 23:46:32 mercat1 badcode[68314]: stack overflow detected; terminated Dec 11 23:46:37 mercat1 badcode[68359]: stack overflow detected; terminated Dec 11 23:46:39 mercat1 badcode[68417]: stack overflow detected; terminated Dec 11 23:46:41 mercat1 badcode[68488]: stack overflow detected; terminated Dec 11 23:46:43 mercat1 badcode[68540]: stack overflow detected; terminated Dec 11 23:46:43 mercat1 badcode[68542]: stack overflow detected; terminated Dec 11 23:46:46 mercat1 badcode[68621]: stack overflow detected; terminated Dec 11 23:46:46 mercat1 badcode[68633]: stack overflow detected; terminated Dec 11 23:46:46 mercat1 badcode[68639]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71315]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71319]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71323]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71324]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71341]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71345]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71347]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71350]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71352]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71361]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71363]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71371]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71393]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71403]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71412]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71414]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71450]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71472]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71491]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71498]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71499]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71501]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71502]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71503]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71513]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71518]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71524]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71528]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71537]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71555]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71556]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71570]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71582]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71592]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71653]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71654]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71669]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71679]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71681]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71735]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71738]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71751]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71758]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71765]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71766]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71779]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71797]: stack overflow detected; terminated Dec 11 23:48:30 mercat1 badcode[71800]: stack overflow detected; terminated Dec 11 23:48:31 mercat1 badcode[71818]: stack overflow detected; terminated Dec 11 23:48:31 mercat1 badcode[71823]: stack overflow detected; terminated Dec 11 23:48:31 mercat1 badcode[71831]: stack overflow detected; terminated Dec 11 23:48:31 mercat1 badcode[71850]: stack overflow detected; terminated Dec 11 23:48:31 mercat1 badcode[71854]: stack overflow detected; terminated Dec 11 23:48:31 mercat1 badcode[71856]: stack overflow detected; terminated Dec 11 23:48:31 mercat1 badcode[71858]: stack overflow detected; terminated Dec 11 23:48:31 mercat1 badcode[71875]: stack overflow detected; terminated Dec 11 23:48:31 mercat1 badcode[71882]: stack overflow detected; terminated Dec 11 23:48:31 mercat1 badcode[71896]: stack overflow detected; terminated Dec 11 23:48:31 mercat1 badcode[71898]: stack overflow detected; terminated Dec 11 23:48:31 mercat1 badcode[71935]: stack overflow detected; terminated Dec 11 23:48:31 mercat1 badcode[71937]: stack overflow detected; terminated Dec 11 23:48:32 mercat1 badcode[71949]: stack overflow detected; terminated Dec 11 23:48:32 mercat1 badcode[71961]: stack overflow detected; terminated Dec 11 23:48:32 mercat1 badcode[72012]: stack overflow detected; terminated Dec 11 23:48:33 mercat1 badcode[72028]: stack overflow detected; terminated Dec 11 23:48:33 mercat1 badcode[72032]: stack overflow detected; terminated Dec 11 23:48:33 mercat1 badcode[72054]: stack overflow detected; terminated Dec 11 23:48:33 mercat1 badcode[72065]: stack overflow detected; terminated Dec 11 23:48:33 mercat1 badcode[72058]: stack overflow detected; terminated Dec 11 23:48:33 mercat1 badcode[72080]: stack overflow detected; terminated Dec 11 23:48:33 mercat1 badcode[72102]: stack overflow detected; terminated Dec 11 23:48:35 mercat1 badcode[72140]: stack overflow detected; terminated Dec 11 23:48:35 mercat1 badcode[72142]: stack overflow detected; terminated Dec 11 23:48:35 mercat1 badcode[72152]: stack overflow detected; terminated Dec 11 23:48:35 mercat1 badcode[72160]: stack overflow detected; terminated Dec 11 23:48:37 mercat1 badcode[72218]: stack overflow detected; terminated Dec 11 23:48:37 mercat1 badcode[72217]: stack overflow detected; terminated Dec 11 23:48:37 mercat1 badcode[72226]: stack overflow detected; terminated Dec 11 23:48:37 mercat1 badcode[72232]: stack overflow detected; terminated Dec 11 23:48:37 mercat1 badcode[72254]: stack overflow detected; terminated Dec 11 23:48:37 mercat1 badcode[72268]: stack overflow detected; terminated Dec 11 23:48:37 mercat1 badcode[72285]: stack overflow detected; terminated Dec 11 23:48:38 mercat1 badcode[72302]: stack overflow detected; terminated Dec 11 23:48:38 mercat1 badcode[72323]: stack overflow detected; terminated Dec 11 23:48:40 mercat1 badcode[72393]: stack overflow detected; terminated Dec 11 23:48:41 mercat1 badcode[72397]: stack overflow detected; terminated Dec 11 23:48:41 mercat1 badcode[72417]: stack overflow detected; terminated Dec 11 23:48:42 mercat1 badcode[72447]: stack overflow detected; terminated Dec 11 23:48:42 mercat1 badcode[72473]: stack overflow detected; terminated Dec 11 23:48:43 mercat1 badcode[72446]: stack overflow detected; terminated Dec 11 23:48:43 mercat1 badcode[72498]: stack overflow detected; terminated Dec 11 23:48:43 mercat1 badcode[72506]: stack overflow detected; terminated Dec 11 23:48:43 mercat1 badcode[72514]: stack overflow detected; terminated Dec 11 23:48:43 mercat1 badcode[72437]: stack overflow detected; terminated Dec 11 23:48:44 mercat1 badcode[72523]: stack overflow detected; terminated Dec 11 23:48:45 mercat1 badcode[72518]: stack overflow detected; terminated Dec 11 23:48:46 mercat1 badcode[72515]: stack overflow detected; terminated Dec 11 23:48:48 mercat1 badcode[72566]: stack overflow detected; terminated Dec 11 23:48:48 mercat1 badcode[72564]: stack overflow detected; terminated Dec 11 23:48:48 mercat1 badcode[72575]: stack overflow detected; terminated Dec 11 23:48:48 mercat1 badcode[72573]: stack overflow detected; terminated Dec 11 23:48:50 mercat1 badcode[72617]: stack overflow detected; terminated Dec 11 23:48:50 mercat1 badcode[72619]: stack overflow detected; terminated Dec 11 23:48:51 mercat1 badcode[72637]: stack overflow detected; terminated Dec 11 23:48:51 mercat1 badcode[72645]: stack overflow detected; terminated Dec 11 23:48:52 mercat1 badcode[72657]: stack overflow detected; terminated Dec 11 23:48:52 mercat1 badcode[72676]: stack overflow detected; terminated Dec 11 23:48:54 mercat1 badcode[72712]: stack overflow detected; terminated Dec 11 23:48:56 mercat1 badcode[72743]: stack overflow detected; terminated Dec 11 23:48:57 mercat1 badcode[72759]: stack overflow detected; terminated Dec 11 23:48:57 mercat1 badcode[72762]: stack overflow detected; terminated Dec 11 23:48:58 mercat1 badcode[72807]: stack overflow detected; terminated Dec 11 23:48:59 mercat1 badcode[72820]: stack overflow detected; terminated Dec 11 23:48:59 mercat1 badcode[72833]: stack overflow detected; terminated Dec 11 23:49:00 mercat1 badcode[72856]: stack overflow detected; terminated Dec 11 23:49:01 mercat1 badcode[72877]: stack overflow detected; terminated Dec 11 23:49:01 mercat1 badcode[72880]: stack overflow detected; terminated Dec 11 23:49:01 mercat1 badcode[72891]: stack overflow detected; terminated Dec 11 23:49:01 mercat1 badcode[72894]: stack overflow detected; terminated Dec 11 23:49:01 mercat1 badcode[72901]: stack overflow detected; terminated Dec 11 23:49:02 mercat1 badcode[72923]: stack overflow detected; terminated Dec 11 23:49:02 mercat1 badcode[72936]: stack overflow detected; terminated Dec 11 23:49:03 mercat1 badcode[72941]: stack overflow detected; terminated Dec 11 23:49:03 mercat1 badcode[72952]: stack overflow detected; terminated Dec 11 23:49:04 mercat1 badcode[72977]: stack overflow detected; terminated Dec 11 23:49:04 mercat1 badcode[72975]: stack overflow detected; terminated Dec 11 23:49:04 mercat1 badcode[72976]: stack overflow detected; terminated Dec 11 23:49:05 mercat1 badcode[72991]: stack overflow detected; terminated Dec 11 23:49:05 mercat1 badcode[72983]: stack overflow detected; terminated Dec 11 23:49:06 mercat1 badcode[73005]: stack overflow detected; terminated Dec 11 23:49:06 mercat1 badcode[73016]: stack overflow detected; terminated Dec 11 23:49:07 mercat1 badcode[73021]: stack overflow detected; terminated Dec 11 23:49:07 mercat1 badcode[73024]: stack overflow detected; terminated Dec 11 23:49:09 mercat1 badcode[73081]: stack overflow detected; terminated Dec 11 23:49:09 mercat1 badcode[73090]: stack overflow detected; terminated Dec 11 23:49:10 mercat1 badcode[73112]: stack overflow detected; terminated Dec 11 23:49:12 mercat1 badcode[73148]: stack overflow detected; terminated Dec 11 23:49:13 mercat1 badcode[73169]: stack overflow detected; terminated Dec 11 23:49:13 mercat1 badcode[73176]: stack overflow detected; terminated Dec 11 23:49:19 mercat1 badcode[73250]: stack overflow detected; terminated Dec 11 23:49:19 mercat1 badcode[73255]: stack overflow detected; terminated Dec 11 23:49:22 mercat1 badcode[73308]: stack overflow detected; terminated Dec 11 23:49:23 mercat1 badcode[73317]: stack overflow detected; terminated Dec 11 23:49:23 mercat1 badcode[73318]: stack overflow detected; terminated Dec 11 23:49:25 mercat1 badcode[73353]: stack overflow detected; terminated pts leak: 1/4 20191211 23:50:35 all (44/121): msdos4.sh Dec 11 23:58:53 mercat1 kernel: pid 117 (swap), jid 0, uid 0, was killed: out of swap space Dec 11 23:59:32 mercat1 kernel: pid 1786 (swap), jid 0, uid 0, was killed: out of swap space 20191212 00:01:06 all (45/121): rw.sh Dec 12 00:07:45 mercat1 kernel: pid 15645 (rw), uid 2007 inumber 196611 on /mnt: filesystem full Dec 12 00:08:01 mercat1 kernel: pid 15651 (swap), jid 0, uid 2007, was killed: out of swap space Dec 12 00:08:32 mercat1 kernel: pid 15670 (swap), jid 0, uid 2007, was killed: out of swap space 20191212 00:11:59 all (46/121): pfl4.sh 20191212 00:22:22 all (47/121): dangling.sh 20191212 00:34:08 all (48/121): sysctl3.sh Dec 12 00:37:54 mercat1 kernel: pid 5987 (swap), jid 0, uid 2007, was killed: out of swap space Dec 12 00:38:01 mercat1 kernel: pid 5992 (swap), jid 0, uid 2007, was killed: out of swap space Dec 12 00:39:11 mercat1 kernel: pid 9459 (swap), jid 0, uid 2007, was killed: out of swap space Dec 12 00:40:00 mercat1 kernel: pid 9444 (swap), jid 0, uid 2007, was killed: out of swap space Dec 12 00:44:52 mercat1 kernel: pid 13240 (swap), jid 0, uid 2007, was killed: out of swap space 20191212 00:44:56 all (49/121): union.sh Dec 12 00:54:18 mercat1 kernel: pid 42311 (swap), jid 0, uid 0, was killed: out of swap space 20191212 00:55:18 all (50/121): nullfs5.sh Dec 12 00:58:37 mercat1 kernel: pid 44733 (swap), jid 0, uid 0, was killed: out of swap space 20191212 01:07:36 all (51/121): fdgrowtable.sh Expensive timeout(9) function: 0xffffffff80cf0b70(0xfffffe00d16b5000) 0.507848232 s Dec 12 01:15:37 mercat1 kernel: pid 38596 (swap), jid 0, uid 0, was killed: out of swap space Dec 12 01:16:03 mercat1 kernel: pid 38618 (swap), jid 0, uid 0, was killed: out of swap space 20191212 01:18:41 all (52/121): md3.sh Dec 12 01:18:53 mercat1 kernel: pid 41485 (rw), uid 0 inumber 91386 on /mnt: filesystem full Dec 12 01:19:26 mercat1 kernel: pid 41459 (swap), jid 0, uid 0, was killed: out of swap space Dec 12 01:19:52 mercat1 kernel: pid 41502 (rw), uid 0 inumber 110969 on /mnt: filesystem full Dec 12 01:20:28 mercat1 kernel: pid 41498 (rw), uid 0 inumber 111131 on /mnt: filesystem full Dec 12 01:20:37 mercat1 kernel: pid 41504 (rw), uid 0 inumber 110592 on /mnt: filesystem full Dec 12 01:21:11 mercat1 kernel: pid 44491 (swap), jid 0, uid 0, was killed: out of swap space Dec 12 01:21:58 mercat1 kernel: pid 44548 (swap), jid 0, uid 0, was killed: out of swap space Dec 12 01:22:00 mercat1 kernel: pid 44489 (rw), uid 0 inumber 148441 on /mnt: filesystem full Dec 12 01:22:13 mercat1 kernel: pid 44513 (rw), uid 0 inumber 165092 on /mnt: filesystem full Dec 12 01:22:48 mercat1 kernel: pid 44497 (rw), uid 0 inumber 141018 on /mnt: filesystem full Dec 12 01:23:11 mercat1 kernel: pid 46371 (symlink), uid 0 inumber 110 on /mnt: out of inodes Dec 12 01:25:06 mercat1 kernel: pid 53625 (creat), uid 0 inumber 290 on /mnt: out of inodes Dec 12 01:25:48 mercat1 kernel: pid 53658 (swap), jid 0, uid 0, was killed: out of swap space Dec 12 01:26:02 mercat1 kernel: pid 53654 (swap), jid 0, uid 0, was killed: out of swap space Dec 12 01:27:10 mercat1 kernel: pid 54754 (rw), uid 0 inumber 124349 on /mnt: filesystem full Dec 12 01:27:15 mercat1 kernel: pid 54756 (rw), uid 0 inumber 128174 on /mnt: filesystem full Dec 12 01:27:22 mercat1 kernel: pid 54770 (creat), uid 0 inumber 89748 on /mnt: out of inodes Dec 12 01:27:28 mercat1 kernel: pid 54768 (creat), uid 0 inumber 89747 on /mnt: out of inodes Dec 12 01:27:34 mercat1 kernel: pid 54770 (creat), uid 0 inumber 89748 on /mnt: out of inodes Dec 12 01:27:41 mercat1 kernel: pid 54764 (creat), uid 0 inumber 89835 on /mnt: out of inodes Dec 12 01:27:49 mercat1 kernel: pid 54785 (openat), uid 0 inumber 89764 on /mnt: out of inodes Dec 12 01:27:58 mercat1 kernel: pid 54766 (creat), uid 0 inumber 89745 on /mnt: out of inodes Dec 12 01:28:09 mercat1 kernel: pid 54764 (creat), uid 0 inumber 89835 on /mnt: out of inodes Dec 12 01:28:17 mercat1 kernel: pid 54762 (creat), uid 0 inumber 89741 on /mnt: out of inodes Dec 12 01:28:26 mercat1 kernel: pid 54770 (creat), uid 0 inumber 89748 on /mnt: out of inodes Dec 12 01:28:39 mercat1 kernel: pid 54768 (creat), uid 0 inumber 89747 on /mnt: out of inodes Dec 12 01:28:48 mercat1 kernel: pid 54766 (creat), uid 0 inumber 89745 on /mnt: out of inodes 20191212 01:29:10 all (53/121): zfs5.sh ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Dec 12 01:36:35 mercat1 kernel: pid 68078 (swap), jid 0, uid 0, was killed: out of swap space Warning: memory type solaris leaked memory on destroy (4 allocations, 2112 bytes leaked). 20191212 01:40:01 all (54/121): oovm.sh 20191212 01:51:04 all (55/121): nullfs2.sh freework leak: 55/100. 20191212 02:02:37 all (56/121): zfs9.sh ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Expensive timeout(9) function: 0xffffffff809eb600(0xfffffe00d16a7cd8) 23.430390588 s Dec 12 02:04:26 mercat1 kernel: pid 19910 (swap), jid 0, uid 0, was killed: out of swap space root@mercat1:~ # ps auxwwl | grep zfs root 19655 0.0 0.0 0 880 - DL 02:02 0:05.90 [zfskern] 0 0 0 -8 0 t->zthr_ root 30739 0.0 0.0 13248 4936 - D 03:01 0:15.10 find -sx / /tmp 0 30735 0 26 0 zfs root 21442 0.0 0.0 11364 2764 u1 S+ 03:27 0:00.00 grep zfs 0 2372 0 20 0 piperd root 19629 0.0 0.0 11708 0 0 IW - 0:00.00 /bin/sh ./zfs9.s 0 19627 0 52 0 wait root 19878 0.0 0.0 11028 2640 0 D 02:02 0:01.49 lockf 0 19876 0 25 0 zfs root 19880 0.0 0.0 11028 2644 0 D 02:02 0:01.38 lockf 0 19876 0 24 0 zfs root 19881 0.0 0.0 11028 2644 0 D 02:02 0:09.16 link 0 19875 0 52 0 zfs root 19882 0.0 0.0 11028 2640 0 D 02:02 0:01.10 lockf 0 19876 0 20 0 zfs root 19884 0.0 0.0 11028 2640 0 D 02:02 0:01.43 lockf 0 19876 0 26 0 zfs root 19885 0.0 0.0 11028 2644 0 D 02:02 0:09.60 link 0 19875 0 52 0 zfs root 19892 0.0 0.0 11036 2656 0 D 02:02 0:00.53 mkdir 0 19891 0 20 0 zfs root 19893 0.0 0.0 11036 2656 0 D 02:02 0:01.24 mkdir 0 19891 0 21 0 zfs root 19894 0.0 0.0 11036 2656 0 D 02:02 0:00.31 mkdir 0 19891 0 20 0 zfs root 19895 0.0 0.0 11036 2656 0 D 02:02 0:00.32 mkdir 0 19891 0 21 0 zfs root 19896 0.0 0.0 11036 2656 0 D 02:02 0:00.31 mkdir 0 19891 0 20 0 zfs root 19897 0.0 0.0 11036 2656 0 D 02:02 0:00.44 mkdir 0 19891 0 20 0 zfs root 19899 0.0 0.0 11036 2656 0 D 02:02 0:00.48 mkdir 0 19891 0 20 0 zfs root 19900 0.0 0.0 11036 2656 0 D 02:02 0:00.38 mkdir 0 19891 0 20 0 zfs root 19902 0.0 0.0 11036 2656 0 D 02:02 0:00.35 mkdir 0 19891 0 20 0 zfs root 19904 0.0 0.0 11036 2656 0 D 02:02 0:00.42 mkdir 0 19891 0 20 0 zfs root 19906 0.0 0.0 11036 2656 0 D 02:02 0:01.09 mkdir 0 19891 0 20 0 zfs root 19909 0.0 0.0 11036 2656 0 D 02:02 0:00.92 mkdir 0 19891 0 20 0 zfs root 19943 0.0 0.0 11024 2644 0 D 02:02 0:00.71 dirrename 0 19942 0 20 0 zfs root 19944 0.0 0.0 11024 2644 0 D 02:02 0:00.14 dirrename 0 19942 0 20 0 zfs root 19962 0.0 0.0 11044 2648 0 D 02:02 0:00.17 dirnprename 0 19960 0 20 0 zfs root 19963 0.0 0.0 11044 2648 0 D 02:02 0:00.22 dirnprename 0 19960 0 20 0 zfs root 19964 0.0 0.0 11044 2648 0 D 02:02 0:00.22 dirnprename 0 19960 0 20 0 zfs root 19965 0.0 0.0 11044 2648 0 D 02:02 0:00.68 dirnprename 0 19960 0 20 0 zfs root 19966 0.0 0.0 11044 2648 0 D 02:02 0:00.19 dirnprename 0 19960 0 20 0 zfs root 19967 0.0 0.0 11044 2648 0 D 02:02 0:00.21 dirnprename 0 19960 0 20 0 zfs root 19968 0.0 0.0 11044 2648 0 D 02:02 0:00.17 dirnprename 0 19960 0 20 0 zfs root 19969 0.0 0.0 11044 2648 0 D 02:02 0:00.21 dirnprename 0 19960 0 20 0 zfs root 19970 0.0 0.0 11044 2648 0 D 02:02 0:00.13 dirnprename 0 19960 0 20 0 zfs root 19971 0.0 0.0 11044 2648 0 D 02:02 0:00.16 dirnprename 0 19960 0 20 0 zfs root 19972 0.0 0.0 11044 2648 0 D 02:02 0:00.14 dirnprename 0 19960 0 20 0 zfs root 19973 0.0 0.0 11044 2648 0 D 02:02 0:00.21 dirnprename 0 19960 0 20 0 zfs root 19974 0.0 0.0 11044 2648 0 D 02:02 0:00.12 dirnprename 0 19960 0 20 0 zfs root 19975 0.0 0.0 11044 2648 0 D 02:02 0:00.13 dirnprename 0 19960 0 20 0 zfs root 19976 0.0 0.0 11044 2648 0 D 02:02 0:00.16 dirnprename 0 19960 0 20 0 zfs root 19977 0.0 0.0 11044 2648 0 D 02:02 0:00.16 dirnprename 0 19960 0 20 0 zfs root 20230 0.0 0.0 12424 3184 0 D 02:02 0:00.20 fstat 0 19954 0 20 0 zfs root 20237 0.0 0.0 12508 3224 0 D 02:02 0:00.19 fstat 0 19948 0 20 0 zfs root 20241 0.0 0.0 12424 3184 0 D 02:02 0:00.18 fstat 0 19952 0 20 0 zfs root 20250 0.0 0.0 12368 3184 0 D 02:02 0:00.05 fstat 0 19953 0 20 0 zfs root 20344 0.0 0.0 11952 1236 0 D 02:02 0:00.01 fstat 0 19950 0 20 0 zfs root 20345 0.0 0.0 11764 304 0 D 02:02 0:00.01 fstat 0 19951 0 20 0 zfs root 20380 0.0 0.0 11540 1172 0 D 02:02 0:00.00 sh -c fstat > /d 0 19955 0 20 0 zfs root 20406 0.0 0.0 11540 1172 0 D 02:02 0:00.00 sh -c fstat > /d 0 19947 0 20 0 zfs root 20407 0.0 0.0 11764 304 0 D 02:02 0:00.01 fstat 0 19949 0 20 0 zfs root 20414 0.0 0.0 11540 1172 0 D 02:02 0:00.00 sh -c fstat > /d 0 19957 0 20 0 zfs root 20486 0.0 0.0 11540 1172 0 D 02:03 0:00.00 sh -c fstat > /d 0 19956 0 20 0 zfs root 20682 0.0 0.0 11532 2828 0 D 02:04 0:00.00 sh -c fstat > /d 0 19946 0 20 0 zfs root@mercat1:~ # pgrep watchdogd 2157 root@mercat1:~ # pgrep watchdogds auxwwl | grep zfs root 19655 0.0 0.0 0 880 - DL 02:02 0:06.00 [zfskern] 0 0 0 -8 0 t->zthr_ root 30739 0.0 0.0 13248 4936 - D 03:01 0:15.10 find -sx / /tmp 0 30735 0 26 0 zfs root 8536 0.0 0.0 11364 2764 u1 S+ 05:02 0:00.00 grep zfs 0 2372 0 20 0 piperd root 19629 0.0 0.0 11708 0 0 IW - 0:00.00 /bin/sh ./zfs9.s 0 19627 0 52 0 wait root 19878 0.0 0.0 11028 2640 0 D 02:02 0:01.49 lockf 0 19876 0 25 0 zfs root 19880 0.0 0.0 11028 2644 0 D 02:02 0:01.38 lockf 0 19876 0 24 0 zfs root 19881 0.0 0.0 11028 2644 0 D 02:02 0:09.16 link 0 19875 0 52 0 zfs root 19882 0.0 0.0 11028 2640 0 D 02:02 0:01.10 lockf 0 19876 0 20 0 zfs root 19884 0.0 0.0 11028 2640 0 D 02:02 0:01.43 lockf 0 19876 0 26 0 zfs root 19885 0.0 0.0 11028 2644 0 D 02:02 0:09.60 link 0 19875 0 52 0 zfs root 19892 0.0 0.0 11036 2656 0 D 02:02 0:00.53 mkdir 0 19891 0 20 0 zfs root 19893 0.0 0.0 11036 2656 0 D 02:02 0:01.24 mkdir 0 19891 0 21 0 zfs root 19894 0.0 0.0 11036 2656 0 D 02:02 0:00.31 mkdir 0 19891 0 20 0 zfs root 19895 0.0 0.0 11036 2656 0 D 02:02 0:00.32 mkdir 0 19891 0 21 0 zfs root 19896 0.0 0.0 11036 2656 0 D 02:02 0:00.31 mkdir 0 19891 0 20 0 zfs root 19897 0.0 0.0 11036 2656 0 D 02:02 0:00.44 mkdir 0 19891 0 20 0 zfs root 19899 0.0 0.0 11036 2656 0 D 02:02 0:00.48 mkdir 0 19891 0 20 0 zfs root 19900 0.0 0.0 11036 2656 0 D 02:02 0:00.38 mkdir 0 19891 0 20 0 zfs root 19902 0.0 0.0 11036 2656 0 D 02:02 0:00.35 mkdir 0 19891 0 20 0 zfs root 19904 0.0 0.0 11036 2656 0 D 02:02 0:00.42 mkdir 0 19891 0 20 0 zfs root 19906 0.0 0.0 11036 2656 0 D 02:02 0:01.09 mkdir 0 19891 0 20 0 zfs root 19909 0.0 0.0 11036 2656 0 D 02:02 0:00.92 mkdir 0 19891 0 20 0 zfs root 19943 0.0 0.0 11024 2644 0 D 02:02 0:00.71 dirrename 0 19942 0 20 0 zfs root 19944 0.0 0.0 11024 2644 0 D 02:02 0:00.14 dirrename 0 19942 0 20 0 zfs root 19962 0.0 0.0 11044 2648 0 D 02:02 0:00.17 dirnprename 0 19960 0 20 0 zfs root 19963 0.0 0.0 11044 2648 0 D 02:02 0:00.22 dirnprename 0 19960 0 20 0 zfs root 19964 0.0 0.0 11044 2648 0 D 02:02 0:00.22 dirnprename 0 19960 0 20 0 zfs root 19965 0.0 0.0 11044 2648 0 D 02:02 0:00.68 dirnprename 0 19960 0 20 0 zfs root 19966 0.0 0.0 11044 2648 0 D 02:02 0:00.19 dirnprename 0 19960 0 20 0 zfs root 19967 0.0 0.0 11044 2648 0 D 02:02 0:00.21 dirnprename 0 19960 0 20 0 zfs root 19968 0.0 0.0 11044 2648 0 D 02:02 0:00.17 dirnprename 0 19960 0 20 0 zfs root 19969 0.0 0.0 11044 2648 0 D 02:02 0:00.21 dirnprename 0 19960 0 20 0 zfs root 19970 0.0 0.0 11044 2648 0 D 02:02 0:00.13 dirnprename 0 19960 0 20 0 zfs root 19971 0.0 0.0 11044 2648 0 D 02:02 0:00.16 dirnprename 0 19960 0 20 0 zfs root 19972 0.0 0.0 11044 2648 0 D 02:02 0:00.14 dirnprename 0 19960 0 20 0 zfs root 19973 0.0 0.0 11044 2648 0 D 02:02 0:00.21 dirnprename 0 19960 0 20 0 zfs root 19974 0.0 0.0 11044 2648 0 D 02:02 0:00.12 dirnprename 0 19960 0 20 0 zfs root 19975 0.0 0.0 11044 2648 0 D 02:02 0:00.13 dirnprename 0 19960 0 20 0 zfs root 19976 0.0 0.0 11044 2648 0 D 02:02 0:00.16 dirnprename 0 19960 0 20 0 zfs root 19977 0.0 0.0 11044 2648 0 D 02:02 0:00.16 dirnprename 0 19960 0 20 0 zfs root 20230 0.0 0.0 12424 3184 0 D 02:02 0:00.20 fstat 0 19954 0 20 0 zfs root 20237 0.0 0.0 12508 3224 0 D 02:02 0:00.19 fstat 0 19948 0 20 0 zfs root 20241 0.0 0.0 12424 3184 0 D 02:02 0:00.18 fstat 0 19952 0 20 0 zfs root 20250 0.0 0.0 12368 3184 0 D 02:02 0:00.05 fstat 0 19953 0 20 0 zfs root 20344 0.0 0.0 11952 1236 0 D 02:02 0:00.01 fstat 0 19950 0 20 0 zfs root 20345 0.0 0.0 11764 304 0 D 02:02 0:00.01 fstat 0 19951 0 20 0 zfs root 20380 0.0 0.0 11540 1172 0 D 02:02 0:00.00 sh -c fstat > /d 0 19955 0 20 0 zfs root 20406 0.0 0.0 11540 1172 0 D 02:02 0:00.00 sh -c fstat > /d 0 19947 0 20 0 zfs root 20407 0.0 0.0 11764 304 0 D 02:02 0:00.01 fstat 0 19949 0 20 0 zfs root 20414 0.0 0.0 11540 1172 0 D 02:02 0:00.00 sh -c fstat > /d 0 19957 0 20 0 zfs root 20486 0.0 0.0 11540 1172 0 D 02:03 0:00.00 sh -c fstat > /d 0 19956 0 20 0 zfs root 20682 0.0 0.0 11532 2828 0 D 02:04 0:00.00 sh -c fstat > /d 0 19946 0 20 0 zfs root@mercat1:~ # procstat -k 19962 PID TID COMM TDNAME KSTACK 19962 102150 dirnprename - mi_switch sleepq_switch sleepq_wait sleeplk lockmgr_slock_hard zfs_lock VOP_LOCK1_APV _vn_lock lookup namei kern_renameat amd64_syscall fast_syscall_common root@mercat1:~ # procstat -k 19962 PID TID COMM TDNAME KSTACK 19962 102150 dirnprename - mi_switch sleepq_switch sleepq_wait sleeplk lockmgr_slock_hard zfs_lock VOP_LOCK1_APV _vn_lock lookup namei kern_renameat amd64_syscall fast_syscall_common root@mercat1:~ # KDB: enter: Break to debugger [ thread pid 19871 tid 101955 ] Stopped at kdb_alt_break_internal+0x101: movq $0,0x10870bc(%rip) db> run pho db:0:pho> set $lines 20000 db:0:pho> run pho1 db:1:pho1> bt Tracing pid 19871 tid 101955 td 0xfffff80278cd96e0 kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe00eb5750a0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00eb5750b0 uart_intr_rxready() at uart_intr_rxready+0xa5/frame 0xfffffe00eb5750e0 uart_intr() at uart_intr+0x138/frame 0xfffffe00eb575120 intr_event_handle() at intr_event_handle+0xce/frame 0xfffffe00eb575170 intr_execute_handlers() at intr_execute_handlers+0x75/frame 0xfffffe00eb5751b0 lapic_handle_intr() at lapic_handle_intr+0x5b/frame 0xfffffe00eb5751d0 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00eb5751d0 --- interrupt, rip = 0xffffffff80c11a92, rsp = 0xfffffe00eb5752a0, rbp = 0xfffffe00eb5752b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00eb5752b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00eb575370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00eb5753b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00eb5753f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00eb5754d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5755c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb575700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5757d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb575800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb575950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb575ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb575bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb575bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- db:1:pho1> show allpcpu Current CPU: 0 cpuid = 0 dynamic pcpu = 0x976e00 curthread = 0xfffff80278cd96e0: pid 19871 tid 101955 "openat" curpcb = 0xfffff80278cd9bd0 fpcurthread = 0xfffff80278cd96e0: pid 19871 "openat" idlethread = 0xfffff80003868000: tid 100003 "idle: cpu0" self = 0xffffffff82410000 curpmap = 0xfffff807ae46f120 tssp = 0xffffffff82410384 rsp0 = 0xfffffe00eb575cc0 kcr3 = 0x80000007d617e03e ucr3 = 0x80000007d617d83e scr3 = 0x165dd0c04 gs32p = 0xffffffff82410404 ldt = 0xffffffff82410444 tss = 0xffffffff82410434 tlb gen = 664213802 curvnet = 0 spin locks held: cpuid = 1 dynamic pcpu = 0xfffffe00bc7e7e00 curthread = 0xfffff801ac8d86e0: pid 19862 tid 100566 "openat" curpcb = 0xfffff801ac8d8bd0 fpcurthread = 0xfffff801ac8d86e0: pid 19862 "openat" idlethread = 0xfffff800038596e0: tid 100004 "idle: cpu1" self = 0xffffffff82411000 curpmap = 0xfffff80191ca6120 tssp = 0xffffffff82411384 rsp0 = 0xfffffe00e4fdacc0 kcr3 = 0x80000008006d01b3 ucr3 = 0x80000008007a39b3 scr3 = 0x165e80ad9 gs32p = 0xffffffff82411404 ldt = 0xffffffff82411444 tss = 0xffffffff82411434 tlb gen = 664213802 curvnet = 0 spin locks held: cpuid = 2 dynamic pcpu = 0xfffffe00bc7f3e00 curthread = 0xfffff805466d7000: pid 19961 tid 102163 "dirnprename" curpcb = 0xfffff805466d74f0 fpcurthread = 0xfffff805466d7000: pid 19961 "dirnprename" idlethread = 0xfffff80003859000: tid 100005 "idle: cpu2" self = 0xffffffff82412000 curpmap = 0xfffff8026b1ed120 tssp = 0xffffffff82412384 rsp0 = 0xfffffe00ef988cc0 kcr3 = 0x80000007d6404010 ucr3 = 0x80000007d6401810 scr3 = 0x165f11b6f gs32p = 0xffffffff82412404 ldt = 0xffffffff82412444 tss = 0xffffffff82412434 tlb gen = 664213802 curvnet = 0 spin locks held: cpuid = 3 dynamic pcpu = 0xfffffe00bc7ffe00 curthread = 0xfffff801ca3b5000: pid 19866 tid 102244 "openat" curpcb = 0xfffff801ca3b54f0 fpcurthread = 0xfffff801ca3b5000: pid 19866 "openat" idlethread = 0xfffff8000388a6e0: tid 100006 "idle: cpu3" self = 0xffffffff82413000 curpmap = 0xfffff803b437f120 tssp = 0xffffffff82413384 rsp0 = 0xfffffe00e8321cc0 kcr3 = 0x80000007d6112330 ucr3 = 0x80000007d6111b30 scr3 = 0x165d79f20 gs32p = 0xffffffff82413404 ldt = 0xffffffff82413444 tss = 0xffffffff82413434 tlb gen = 664213802 curvnet = 0 spin locks held: cpuid = 4 dynamic pcpu = 0xfffffe00bc80be00 curthread = 0xfffff807b86976e0: pid 19874 tid 100341 "openat" curpcb = 0xfffff807b8697bd0 fpcurthread = 0xfffff807b86976e0: pid 19874 "openat" idlethread = 0xfffff8000388a000: tid 100007 "idle: cpu4" self = 0xffffffff82414000 curpmap = 0xfffff807cc620120 tssp = 0xffffffff82414384 rsp0 = 0xfffffe00e3b72cc0 kcr3 = 0x80000007d61cf372 ucr3 = 0x80000007d61ceb72 scr3 = 0x165e48b6f gs32p = 0xffffffff82414404 ldt = 0xffffffff82414444 tss = 0xffffffff82414434 tlb gen = 664213799 curvnet = 0 spin locks held: cpuid = 5 dynamic pcpu = 0xfffffe00bc817e00 curthread = 0xfffff801ac17d6e0: pid 19873 tid 100616 "openat" curpcb = 0xfffff801ac17dbd0 fpcurthread = 0xfffff801ac17d6e0: pid 19873 "openat" idlethread = 0xfffff800038896e0: tid 100008 "idle: cpu5" self = 0xffffffff82415000 curpmap = 0xfffff80507945120 tssp = 0xffffffff82415384 rsp0 = 0xfffffe00ee2d6cc0 kcr3 = 0x80000007d61b4578 ucr3 = 0x80000007d61b3d78 scr3 = 0x165ec7d77 gs32p = 0xffffffff82415404 ldt = 0xffffffff82415444 tss = 0xffffffff82415434 tlb gen = 664213802 curvnet = 0 spin locks held: cpuid = 6 dynamic pcpu = 0xfffffe00bc823e00 curthread = 0xfffff8057d8e46e0: pid 19864 tid 100245 "openat" curpcb = 0xfffff8057d8e4bd0 fpcurthread = 0xfffff8057d8e46e0: pid 19864 "openat" idlethread = 0xfffff80003889000: tid 100009 "idle: cpu6" self = 0xffffffff82416000 curpmap = 0xfffff8062fdbf120 tssp = 0xffffffff82416384 rsp0 = 0xfffffe00e3a0ecc0 kcr3 = 0x80000007d60f7290 ucr3 = 0x80000007d60f6a90 scr3 = 0x165cafa9b gs32p = 0xffffffff82416404 ldt = 0xffffffff82416444 tss = 0xffffffff82416434 tlb gen = 664213802 curvnet = 0 spin locks held: cpuid = 7 dynamic pcpu = 0xfffffe00bc82fe00 curthread = 0xfffff800038696e0: pid 11 tid 100010 "idle: cpu7" curpcb = 0xfffff80003869bd0 fpcurthread = none idlethread = 0xfffff800038696e0: tid 100010 "idle: cpu7" self = 0xffffffff82417000 curpmap = 0xffffffff81d9d170 tssp = 0xffffffff82417384 rsp0 = 0xfffffe00adda1cc0 kcr3 = 0x8000000002306002 ucr3 = 0xffffffffffffffff scr3 = 0x4f088ab3c gs32p = 0xffffffff82417404 ldt = 0xffffffff82417444 tss = 0xffffffff82417434 tlb gen = 664213802 curvnet = 0 spin locks held: cpuid = 8 dynamic pcpu = 0xfffffe00bc83be00 curthread = 0xfffff8000388d000: pid 11 tid 100011 "idle: cpu8" curpcb = 0xfffff8000388d4f0 fpcurthread = none idlethread = 0xfffff8000388d000: tid 100011 "idle: cpu8" self = 0xffffffff82418000 curpmap = 0xffffffff81d9d170 tssp = 0xffffffff82418384 rsp0 = 0xfffffe00adda6cc0 kcr3 = 0x8000000002306002 ucr3 = 0xffffffffffffffff scr3 = 0x165d2dece gs32p = 0xffffffff82418404 ldt = 0xffffffff82418444 tss = 0xffffffff82418434 tlb gen = 664213802 curvnet = 0 spin locks held: cpuid = 9 dynamic pcpu = 0xfffffe00bc847e00 curthread = 0xfffff8019603a6e0: pid 19867 tid 101982 "openat" curpcb = 0xfffff8019603abd0 fpcurthread = 0xfffff8019603a6e0: pid 19867 "openat" idlethread = 0xfffff8000388c6e0: tid 100012 "idle: cpu9" self = 0xffffffff82419000 curpmap = 0xfffff8027370a120 tssp = 0xffffffff82419384 rsp0 = 0xfffffe00eb5fccc0 kcr3 = 0x80000007d612d29e ucr3 = 0x80000007d612ca9e scr3 = 0x165dedcac gs32p = 0xffffffff82419404 ldt = 0xffffffff82419444 tss = 0xffffffff82419434 tlb gen = 664213802 curvnet = 0 spin locks held: cpuid = 10 dynamic pcpu = 0xfffffe00bc853e00 curthread = 0xfffff8038d13c000: pid 19869 tid 101965 "openat" curpcb = 0xfffff8038d13c4f0 fpcurthread = 0xfffff8038d13c000: pid 19869 "openat" idlethread = 0xfffff8000388c000: tid 100013 "idle: cpu10" self = 0xffffffff8241a000 curpmap = 0xfffff8055bf79120 tssp = 0xffffffff8241a384 rsp0 = 0xfffffe00eb5a7cc0 kcr3 = 0x80000007d615108f ucr3 = 0x80000007d615088f scr3 = 0x165d4eb17 gs32p = 0xffffffff8241a404 ldt = 0xffffffff8241a444 tss = 0xffffffff8241a434 tlb gen = 664213802 curvnet = 0 spin locks held: cpuid = 11 dynamic pcpu = 0xfffffe00bc85fe00 curthread = 0xfffff8057d8f86e0: pid 19872 tid 100290 "openat" curpcb = 0xfffff8057d8f8bd0 fpcurthread = 0xfffff8057d8f86e0: pid 19872 "openat" idlethread = 0xfffff8000388b6e0: tid 100014 "idle: cpu11" self = 0xffffffff8241b000 curpmap = 0xfffff80284438120 tssp = 0xffffffff8241b384 rsp0 = 0xfffffe00e2bc1cc0 kcr3 = 0x80000007d619926d ucr3 = 0x80000007d6198a6d scr3 = 0x4f088aa71 gs32p = 0xffffffff8241b404 ldt = 0xffffffff8241b444 tss = 0xffffffff8241b434 tlb gen = 664213802 curvnet = 0 spin locks held: db:1:pho1> show alllocks db:1:pho1> show lockedvnods Locked vnodes vnode 0xfffff801efd5a768: tag zfs, type VDIR usecount 4, writecount 0, refcount 1 mountedhere 0 flags (VV_ROOT|VI_ACTIVE) v_object 0xfffff806fecc6108 ref 0 pages 0 cleanbuf 0 dirtybuf 0 lock type zfs: SHARED (count 1) #0 0xffffffff80b97c50 at lockmgr_lock_fast_path+0x1e0 #1 0xffffffff82672842 at zfs_lock+0x12 #2 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #3 0xffffffff80cb2b75 at _vn_lock+0x65 #4 0xffffffff80c91961 at lookup+0xe1 #5 0xffffffff80c91321 at namei+0x4f1 #6 0xffffffff80cabb8f at kern_statat+0x7f #7 0xffffffff80cac35f at sys_fstatat+0x2f #8 0xffffffff8106b901 at amd64_syscall+0x2f1 #9 0xffffffff810416c0 at fast_syscall_common+0x101 vnode 0xfffff8070b24ec58: tag zfs, type VDIR usecount 134, writecount 0, refcount 2 mountedhere 0 flags (VI_ACTIVE) lock type zfs: EXCL by thread 0xfffff805466d7000 (pid 19961, dirnprename, tid 102163) with exclusive waiters pending with shared waiters pending #0 0xffffffff80b97c50 at lockmgr_lock_fast_path+0x1e0 #1 0xffffffff82672842 at zfs_lock+0x12 #2 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #3 0xffffffff80cb2b75 at _vn_lock+0x65 #4 0xffffffff80c91961 at lookup+0xe1 #5 0xffffffff80c91321 at namei+0x4f1 #6 0xffffffff80caed12 at kern_mkdirat+0x92 #7 0xffffffff8106b901 at amd64_syscall+0x2f1 #8 0xffffffff810416c0 at fast_syscall_common+0x101 vnode 0xfffff805fea1dc58: tag zfs, type VDIR usecount 2, writecount 0, refcount 1 mountedhere 0 flags (VI_ACTIVE) v_object 0xfffff8055bb81630 ref 0 pages 0 cleanbuf 0 dirtybuf 0 lock type zfs: EXCL by thread 0xfffff8038d13c000 (pid 19869, openat, tid 101965) #0 0xffffffff80b97c50 at lockmgr_lock_fast_path+0x1e0 #1 0xffffffff82672842 at zfs_lock+0x12 #2 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #3 0xffffffff80cb2b75 at _vn_lock+0x65 #4 0xffffffff80c91961 at lookup+0xe1 #5 0xffffffff80c91321 at namei+0x4f1 #6 0xffffffff80cb1f41 at vn_open_cred+0xf1 #7 0xffffffff80ca9d53 at kern_openat+0x1f3 #8 0xffffffff8106b901 at amd64_syscall+0x2f1 #9 0xffffffff810416c0 at fast_syscall_common+0x101 vnode 0xfffff8001a5124f0: tag zfs, type VDIR usecount 2, writecount 0, refcount 1 mountedhere 0 flags (VI_ACTIVE) lock type zfs: EXCL by thread 0xfffff80030f7c000 (pid 19883, link, tid 100160) #0 0xffffffff80b97c50 at lockmgr_lock_fast_path+0x1e0 #1 0xffffffff82672842 at zfs_lock+0x12 #2 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #3 0xffffffff80cb2b75 at _vn_lock+0x65 #4 0xffffffff80c91961 at lookup+0xe1 #5 0xffffffff80c91321 at namei+0x4f1 #6 0xffffffff80cb1f41 at vn_open_cred+0xf1 #7 0xffffffff80ca9d53 at kern_openat+0x1f3 #8 0xffffffff8106b901 at amd64_syscall+0x2f1 #9 0xffffffff810416c0 at fast_syscall_common+0x101 vnode 0xfffff8000a084278: tag zfs, type VDIR usecount 2, writecount 0, refcount 1 mountedhere 0 flags (VI_ACTIVE) lock type zfs: EXCL by thread 0xfffff803dc08e6e0 (pid 19877, link, tid 102406) #0 0xffffffff80b97c50 at lockmgr_lock_fast_path+0x1e0 #1 0xffffffff82672842 at zfs_lock+0x12 #2 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #3 0xffffffff80cb2b75 at _vn_lock+0x65 #4 0xffffffff80c91961 at lookup+0xe1 #5 0xffffffff80c91321 at namei+0x4f1 #6 0xffffffff80cb1f41 at vn_open_cred+0xf1 #7 0xffffffff80ca9d53 at kern_openat+0x1f3 #8 0xffffffff8106b901 at amd64_syscall+0x2f1 #9 0xffffffff810416c0 at fast_syscall_common+0x101 vnode 0xfffff804bde2e278: tag zfs, type VDIR usecount 2, writecount 0, refcount 1 mountedhere 0 flags (VI_ACTIVE) v_object 0xfffff802ecf06d68 ref 0 pages 0 cleanbuf 0 dirtybuf 0 lock type zfs: EXCL by thread 0xfffff801ac17d6e0 (pid 19873, openat, tid 100616) #0 0xffffffff80b97c50 at lockmgr_lock_fast_path+0x1e0 #1 0xffffffff82672842 at zfs_lock+0x12 #2 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #3 0xffffffff80cb2b75 at _vn_lock+0x65 #4 0xffffffff80c91961 at lookup+0xe1 #5 0xffffffff80c91321 at namei+0x4f1 #6 0xffffffff80cb1f41 at vn_open_cred+0xf1 #7 0xffffffff80ca9d53 at kern_openat+0x1f3 #8 0xffffffff8106b901 at amd64_syscall+0x2f1 #9 0xffffffff810416c0 at fast_syscall_common+0x101 vnode 0xfffff801f9875768: tag zfs, type VDIR usecount 2, writecount 0, refcount 1 mountedhere 0 flags (VI_ACTIVE) v_object 0xfffff801605e9318 ref 0 pages 0 cleanbuf 0 dirtybuf 0 lock type zfs: EXCL by thread 0xfffff801ca3b5000 (pid 19866, openat, tid 102244) #0 0xffffffff80b97c50 at lockmgr_lock_fast_path+0x1e0 #1 0xffffffff82672842 at zfs_lock+0x12 #2 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #3 0xffffffff80cb2b75 at _vn_lock+0x65 #4 0xffffffff80c91961 at lookup+0xe1 #5 0xffffffff80c91321 at namei+0x4f1 #6 0xffffffff80cb1f41 at vn_open_cred+0xf1 #7 0xffffffff80ca9d53 at kern_openat+0x1f3 #8 0xffffffff8106b901 at amd64_syscall+0x2f1 #9 0xffffffff810416c0 at fast_syscall_common+0x101 vnode 0xfffff8070b6ff9e0: tag zfs, type VDIR usecount 2, writecount 0, refcount 1 mountedhere 0 flags (VI_ACTIVE) lock type zfs: EXCL by thread 0xfffff8028b921000 (pid 19887, link, tid 102281) #0 0xffffffff80b97c50 at lockmgr_lock_fast_path+0x1e0 #1 0xffffffff82672842 at zfs_lock+0x12 #2 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #3 0xffffffff80cb2b75 at _vn_lock+0x65 #4 0xffffffff80c91961 at lookup+0xe1 #5 0xffffffff80c91321 at namei+0x4f1 #6 0xffffffff80cb1f41 at vn_open_cred+0xf1 #7 0xffffffff80ca9d53 at kern_openat+0x1f3 #8 0xffffffff8106b901 at amd64_syscall+0x2f1 #9 0xffffffff810416c0 at fast_syscall_common+0x101 vnode 0xfffff80073118278: tag zfs, type VDIR usecount 2, writecount 0, refcount 1 mountedhere 0 flags (VI_ACTIVE) lock type zfs: EXCL by thread 0xfffff802f9f8c6e0 (pid 19889, link, tid 102381) #0 0xffffffff80b97c50 at lockmgr_lock_fast_path+0x1e0 #1 0xffffffff82672842 at zfs_lock+0x12 #2 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #3 0xffffffff80cb2b75 at _vn_lock+0x65 #4 0xffffffff80c91961 at lookup+0xe1 #5 0xffffffff80c91321 at namei+0x4f1 #6 0xffffffff80cb1f41 at vn_open_cred+0xf1 #7 0xffffffff80ca9d53 at kern_openat+0x1f3 #8 0xffffffff8106b901 at amd64_syscall+0x2f1 #9 0xffffffff810416c0 at fast_syscall_common+0x101 vnode 0xfffff8011e298278: tag zfs, type VDIR usecount 2, writecount 0, refcount 1 mountedhere 0 flags (VI_ACTIVE) v_object 0xfffff80514942948 ref 0 pages 0 cleanbuf 0 dirtybuf 0 lock type zfs: EXCL by thread 0xfffff80278cd96e0 (pid 19871, openat, tid 101955) #0 0xffffffff80b97c50 at lockmgr_lock_fast_path+0x1e0 #1 0xffffffff82672842 at zfs_lock+0x12 #2 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #3 0xffffffff80cb2b75 at _vn_lock+0x65 #4 0xffffffff80c91961 at lookup+0xe1 #5 0xffffffff80c91321 at namei+0x4f1 #6 0xffffffff80cb1f41 at vn_open_cred+0xf1 #7 0xffffffff80ca9d53 at kern_openat+0x1f3 #8 0xffffffff8106b901 at amd64_syscall+0x2f1 #9 0xffffffff810416c0 at fast_syscall_common+0x101 vnode 0xfffff800668d59e0: tag zfs, type VDIR usecount 2, writecount 0, refcount 1 mountedhere 0 flags (VI_ACTIVE) v_object 0xfffff8045053de70 ref 0 pages 0 cleanbuf 0 dirtybuf 0 lock type zfs: EXCL by thread 0xfffff807b86976e0 (pid 19874, openat, tid 100341) #0 0xffffffff80b97c50 at lockmgr_lock_fast_path+0x1e0 #1 0xffffffff82672842 at zfs_lock+0x12 #2 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #3 0xffffffff80cb2b75 at _vn_lock+0x65 #4 0xffffffff80c91961 at lookup+0xe1 #5 0xffffffff80c91321 at namei+0x4f1 #6 0xffffffff80cb1f41 at vn_open_cred+0xf1 #7 0xffffffff80ca9d53 at kern_openat+0x1f3 #8 0xffffffff8106b901 at amd64_syscall+0x2f1 #9 0xffffffff810416c0 at fast_syscall_common+0x101 vnode 0xfffff802d1688000: tag zfs, type VDIR usecount 2, writecount 0, refcount 1 mountedhere 0 flags (VI_ACTIVE) lock type zfs: EXCL by thread 0xfffff8073be1b6e0 (pid 19890, link, tid 102533) #0 0xffffffff80b97c50 at lockmgr_lock_fast_path+0x1e0 #1 0xffffffff82672842 at zfs_lock+0x12 #2 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #3 0xffffffff80cb2b75 at _vn_lock+0x65 #4 0xffffffff80c91961 at lookup+0xe1 #5 0xffffffff80c91321 at namei+0x4f1 #6 0xffffffff80cb1f41 at vn_open_cred+0xf1 #7 0xffffffff80ca9d53 at kern_openat+0x1f3 #8 0xffffffff8106b901 at amd64_syscall+0x2f1 #9 0xffffffff810416c0 at fast_syscall_common+0x101 vnode 0xfffff80340451768: tag zfs, type VDIR usecount 2, writecount 0, refcount 1 mountedhere 0 flags (VI_ACTIVE) v_object 0xfffff8061b3ce000 ref 0 pages 0 cleanbuf 0 dirtybuf 0 lock type zfs: EXCL by thread 0xfffff8019603a6e0 (pid 19867, openat, tid 101982) #0 0xffffffff80b97c50 at lockmgr_lock_fast_path+0x1e0 #1 0xffffffff82672842 at zfs_lock+0x12 #2 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #3 0xffffffff80cb2b75 at _vn_lock+0x65 #4 0xffffffff80c91961 at lookup+0xe1 #5 0xffffffff80c91321 at namei+0x4f1 #6 0xffffffff80cb1f41 at vn_open_cred+0xf1 #7 0xffffffff80ca9d53 at kern_openat+0x1f3 #8 0xffffffff8106b901 at amd64_syscall+0x2f1 #9 0xffffffff810416c0 at fast_syscall_common+0x101 vnode 0xfffff802a24a3768: tag zfs, type VDIR usecount 2, writecount 0, refcount 1 mountedhere 0 flags (VI_ACTIVE) v_object 0xfffff80750075528 ref 0 pages 0 cleanbuf 0 dirtybuf 0 lock type zfs: EXCL by thread 0xfffff8057d8f86e0 (pid 19872, openat, tid 100290) #0 0xffffffff80b97c50 at lockmgr_lock_fast_path+0x1e0 #1 0xffffffff82672842 at zfs_lock+0x12 #2 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #3 0xffffffff80cb2b75 at _vn_lock+0x65 #4 0xffffffff80c91961 at lookup+0xe1 #5 0xffffffff80c91321 at namei+0x4f1 #6 0xffffffff80cb1f41 at vn_open_cred+0xf1 #7 0xffffffff80ca9d53 at kern_openat+0x1f3 #8 0xffffffff8106b901 at amd64_syscall+0x2f1 #9 0xffffffff810416c0 at fast_syscall_common+0x101 vnode 0xfffff801afca69e0: tag zfs, type VDIR usecount 2, writecount 0, refcount 1 mountedhere 0 flags (VI_ACTIVE) v_object 0xfffff802426cf630 ref 0 pages 0 cleanbuf 0 dirtybuf 0 lock type zfs: EXCL by thread 0xfffff801ac8d86e0 (pid 19862, openat, tid 100566) #0 0xffffffff80b97c50 at lockmgr_lock_fast_path+0x1e0 #1 0xffffffff82672842 at zfs_lock+0x12 #2 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #3 0xffffffff80cb2b75 at _vn_lock+0x65 #4 0xffffffff80c91961 at lookup+0xe1 #5 0xffffffff80c91321 at namei+0x4f1 #6 0xffffffff80cb1f41 at vn_open_cred+0xf1 #7 0xffffffff80ca9d53 at kern_openat+0x1f3 #8 0xffffffff8106b901 at amd64_syscall+0x2f1 #9 0xffffffff810416c0 at fast_syscall_common+0x101 vnode 0xfffff8063b188000: tag zfs, type VDIR usecount 2, writecount 0, refcount 1 mountedhere 0 flags (VI_ACTIVE) v_object 0xfffff8049cb02e70 ref 0 pages 0 cleanbuf 0 dirtybuf 0 lock type zfs: EXCL by thread 0xfffff8057d8e46e0 (pid 19864, openat, tid 100245) #0 0xffffffff80b97c50 at lockmgr_lock_fast_path+0x1e0 #1 0xffffffff82672842 at zfs_lock+0x12 #2 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #3 0xffffffff80cb2b75 at _vn_lock+0x65 #4 0xffffffff80c91961 at lookup+0xe1 #5 0xffffffff80c91321 at namei+0x4f1 #6 0xffffffff80cb1f41 at vn_open_cred+0xf1 #7 0xffffffff80ca9d53 at kern_openat+0x1f3 #8 0xffffffff8106b901 at amd64_syscall+0x2f1 #9 0xffffffff810416c0 at fast_syscall_common+0x101 vnode 0xfffff803f3541768: tag zfs, type VDIR usecount 2, writecount 0, refcount 1 mountedhere 0 flags (VI_ACTIVE) lock type zfs: EXCL by thread 0xfffff80642884000 (pid 19879, link, tid 100679) #0 0xffffffff80b97c50 at lockmgr_lock_fast_path+0x1e0 #1 0xffffffff82672842 at zfs_lock+0x12 #2 0xffffffff810e8b02 at VOP_LOCK1_APV+0x92 #3 0xffffffff80cb2b75 at _vn_lock+0x65 #4 0xffffffff80c91961 at lookup+0xe1 #5 0xffffffff80c91321 at namei+0x4f1 #6 0xffffffff80cb1f41 at vn_open_cred+0xf1 #7 0xffffffff80ca9d53 at kern_openat+0x1f3 #8 0xffffffff8106b901 at amd64_syscall+0x2f1 #9 0xffffffff810416c0 at fast_syscall_common+0x101 db:1:pho1> show mount 0xfffff800309b9000 /dev/gpt/work on / (ufs) 0xfffff8003070d000 devfs on /dev (devfs) 0xfffff801c0997000 /dev/gpt/tmp on /tmp (ufs) 0xfffff80049256000 procfs on /proc (procfs) 0xfffff80044012000 stress2_tank on /stress2_tank (zfs) 0xfffff80779d4d000 stress2_tank/test on /stress2_tank/test (zfs) More info: show mount db:1:pho1> show bufqueues bqempty: 49577 Buf domain 0 freebufs 4856 lofreebufs 219 hifreebufs 329 bufspace 46804992 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 2 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 10142 (45490176) dirtyq count 2 (20480) wakeup 0 lim 25 CPU 5, 13, 9, 5, 21, 15, 1, 23, 2, 2, 17, 20, Locked buffers: 0 space 0 Total buffers: 23378 space 46804992 Buf domain 1 freebufs 8877 lofreebufs 219 hifreebufs 329 bufspace 27320320 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 6099 (26669056) dirtyq count 0 (0) wakeup 0 lim 25 CPU 14, 23, 3, 24, 22, 14, 0, 21, 10, 2, 18, 6, Locked buffers: 0 space 0 Total buffers: 8174 space 27320320 Buf domain 2 freebufs 1878 lofreebufs 219 hifreebufs 329 bufspace 228892672 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 27 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 13218 (227741696) dirtyq count 27 (884736) wakeup 0 lim 25 CPU 0, 1, 0, 1, 0, 2, 2, 2, 1, 0, 1, 0, Locked buffers: 0 space 0 Total buffers: 31144 space 228892672 Buf domain 3 freebufs 8374 lofreebufs 219 hifreebufs 329 bufspace 33296384 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 6677 (32530432) dirtyq count 0 (0) wakeup 0 lim 25 CPU 11, 2, 2, 0, 4, 10, 18, 0, 7, 12, 11, 5, Locked buffers: 0 space 0 Total buffers: 8503 space 33296384 Buf domain 4 freebufs 8908 lofreebufs 219 hifreebufs 329 bufspace 26918912 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 6080 (26189824) dirtyq count 0 (0) wakeup 0 lim 25 CPU 8, 15, 2, 6, 13, 1, 22, 2, 24, 21, 23, 8, Locked buffers: 0 space 0 Total buffers: 9930 space 26918912 Buf domain 5 freebufs 8516 lofreebufs 219 hifreebufs 329 bufspace 28262400 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 6455 (27394048) dirtyq count 0 (0) wakeup 0 lim 25 CPU 12, 8, 3, 13, 22, 16, 17, 10, 16, 10, 19, 16, Locked buffers: 0 space 0 Total buffers: 8437 space 28262400 Buf domain 6 freebufs 8692 lofreebufs 219 hifreebufs 329 bufspace 28688384 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 1 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 6331 (27549696) dirtyq count 1 (4096) wakeup 0 lim 25 CPU 11, 10, 3, 8, 6, 10, 8, 3, 7, 23, 9, 11, Locked buffers: 0 space 0 Total buffers: 16365 space 28688384 db:1:pho1> show page vm_cnt.v_free_count: 7326379 vm_cnt.v_inactive_count: 170326 vm_cnt.v_active_count: 3462 vm_cnt.v_laundry_count: 3123 vm_cnt.v_wire_count: 620443 vm_cnt.v_free_reserved: 10644 vm_cnt.v_free_min: 51264 vm_cnt.v_free_target: 173124 vm_cnt.v_inactive_target: 259686 db:1:pho1> show pageq pq_free 7326379 dom 0 page_cnt 8124093 free 7326379 pq_act 3462 pq_inact 170326 pq_laund 3123 pq_unsw 0 db:0:pho> show freepages DOMAIN: 0 FREE LIST 0: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000026 | 000000 10 (004096K) | 000969 | 000000 09 (002048K) | 003620 | 000000 08 (001024K) | 003778 | 000000 07 (000512K) | 008363 | 000001 06 (000256K) | 011638 | 000001 05 (000128K) | 016317 | 000000 04 (000064K) | 027527 | 000000 03 (000032K) | 043160 | 000002 02 (000016K) | 064416 | 000002 01 (000008K) | 012707 | 000021 00 (000004K) | 000150 | 000039 FREE LIST 1: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000000 | 000000 10 (004096K) | 000000 | 000000 09 (002048K) | 000000 | 000000 08 (001024K) | 000000 | 000000 07 (000512K) | 000000 | 000000 06 (000256K) | 000000 | 000000 05 (000128K) | 000001 | 000000 04 (000064K) | 000002 | 000000 03 (000032K) | 000003 | 000000 02 (000016K) | 000005 | 000000 01 (000008K) | 000005 | 000000 00 (000004K) | 000006 | 000000 db:0:pho> show uma Zone Size Used Free Requests Sleeps Bucket Total Mem XFree VNODE 632 619485 111 602060148 0 13 391584672 0 VM OBJECT 264 117998 560616 460309179 0 29 179154096 0 FFS2 dinode 256 619331 499 195858513 0 61 158676480 0 swblk 136 1863 1060639 4199809 0 61 144500272 0 FFS inode 160 619331 569 195859237 0 61 99184000 0 zio_buf_16384 16384 5582 0 230363 0 1 91455488 0 RADIX NODE 144 24639 465123 470643728 0 79 70525728 0 128 128 472727 1363 3384134 0 125 60683520 0 dnode_t 960 49941 963 163851 0 13 48867840 0 Files 80 174 500926 896247682 0 125 40088000 0 abd_chunk 4096 9623 0 549113 0 1 39415808 0 mbuf_cluster 2048 14933 55 9231221 0 5 30695424 0 BUF TRIE 144 2023 103929 168544402 0 61 15257088 0 DIRHASH 1024 14266 82 4947230 0 13 14692352 0 zio_buf_512 512 21288 2160 443114 0 29 12005376 0 THREAD 1760 2355 3847 7916180 0 10 10915520 0 swpctrie 144 255 70753 335209 0 61 10225152 0 dmu_buf_impl_t 352 24313 2373 299062 0 29 9393472 0 zio_buf_131072 131072 56 0 5968 0 1 7340032 0 zio_buf_4096 4096 1765 0 140260 0 1 7229440 0 vmem btag 56 73226 46070 4612846 0 253 6680576 0 4096 4096 1578 12 27131156 0 1 6512640 0 UMA Slabs 112 37143 15822 2551425 0 125 5932080 0 zfs_znode_cache 288 37 19829 163794 0 29 5721408 0 256 256 7463 12771 11207041 0 61 5179904 0 mbuf 256 16356 3028 758155050 0 253 4962304 0 512 512 4989 3135 169496304 0 29 4159488 0 sa_cache 152 37 19931 165272 0 61 3035136 0 arc_buf_hdr_t_full 376 6354 1666 229538 0 29 3015520 0 socket 904 31 3005 7909 0 13 2744544 0 PROC 1328 203 1723 13598050 0 9 2557728 0 1024 1024 905 1303 4834259 0 13 2260992 0 128 128 15212 1529 9284193 0 125 2142848 0 512 512 3801 214 578185 0 29 2055680 0 128 128 14669 1109 1831639 0 125 2019584 0 VMSPACE 2536 151 563 13597935 0 3 1810704 0 128 128 12458 252 12600 0 125 1626880 0 4096 4096 359 0 701998 0 1 1470464 0 MAP ENTRY 96 4709 7891 536063610 0 127 1209600 0 256 Bucket 2048 501 57 39818274 0 5 1142784 0 32768 32768 9 25 8305 0 1 1114112 0 S VFS Cache 108 934 8282 52248870 0 125 995328 0 16 16 56722 4778 7225134 0 253 984000 0 zio_buf_3584 3584 260 0 84106 0 1 931840 0 4096 4096 203 22 13598974 0 1 921600 0 65536 65536 3 11 4498 0 1 917504 0 32 32 1413 27108 34358682 0 253 912672 0 TURNSTILE 136 6202 307 6202 0 61 885224 0 zio_buf_1024 1024 632 164 20964 0 13 815104 0 16384 16384 49 0 317205 0 1 802816 0 zio_cache 1064 16 734 2727357 0 48 798000 0 128 Bucket 1024 542 230 8281373 0 13 790528 0 pbuf 984 0 777 0 0 0 764568 0 64 64 9523 2321 370454 0 253 758016 0 64 64 8429 3036 4531101772 0 253 733760 0 8192 8192 88 0 88 0 1 720896 0 64 Bucket 512 706 694 6702430 0 30 716800 0 256 256 736 2038 21512408 0 61 710144 0 mbuf_packet 256 45 2726 82023179 0 253 709376 0 64 64 676 9086 64439530 0 253 624768 0 arc_buf_t 64 5736 3714 229771 0 253 604800 0 range_seg_cache 72 3638 4370 435441 0 125 576576 0 SLEEPQUEUE 88 6202 245 6202 0 125 567336 0 128 128 3189 1024 3371606517 0 125 539264 0 32 Bucket 256 839 1216 9031097 0 61 526080 0 32768 32768 14 0 12448 0 1 458752 0 UMA Zones 1968 220 1 362 0 5 434928 0 32768 32768 13 0 32 0 1 425984 0 32 32 4408 8592 74334881 0 253 416000 0 32768 32768 12 0 12 0 1 393216 0 filedesc0 1104 190 163 13598050 0 9 389712 0 32 32 9205 2030 225680 0 253 359520 0 ksiginfo 112 1950 1254 9414738 0 125 358848 0 16 16 17648 3063 82837285 0 253 331376 0 selfd 64 1462 3704 1547924 0 253 330624 0 65536 65536 4 1 1660 0 1 327680 0 64 64 240 4862 13600049 0 253 326528 0 256 256 541 673 133080262 0 61 310784 0 64 64 256 4594 33216516 0 253 310400 0 64 64 1480 3217 7461434 0 253 300608 0 2048 2048 18 124 15274853 0 5 290816 0 256 256 430 677 80519741 0 61 283392 0 16 Bucket 128 282 1919 5315947 0 125 281728 0 zio_link_cache 48 0 5810 2525449 0 253 278880 0 4096 4096 66 0 14905 0 1 270336 0 g_bio 408 0 657 997254136 0 29 268056 0 NAMEI 1024 56 200 3024751119 0 13 262144 0 65536 65536 4 0 4 0 1 262144 0 Mountpoints 3008 6 78 97722 0 3 252672 0 128 128 451 1500 3965594 0 125 249728 0 64 64 3306 517 11934349 0 253 244672 0 1024 1024 209 27 3628227 0 13 241664 0 128 128 477 1259 404764766 0 125 222208 0 8 Bucket 64 104 3361 6999947 0 253 221760 0 zio_buf_1536 1536 95 47 8301 0 9 218112 0 2048 2048 99 2 1134 0 5 206848 0 256 256 384 396 68127 0 61 199680 0 65536 65536 1 2 81 0 1 196608 0 16384 16384 8 4 1730 0 1 196608 0 8192 8192 23 0 3403703 0 1 188416 0 1024 1024 21 159 387177 0 13 184320 0 256 256 35 685 67179981 0 61 184320 0 64 64 1204 1631 36627003 0 253 181440 0 256 256 22 683 17490717 0 61 180480 0 1024 1024 32 144 428962 0 13 180224 0 512 512 3 349 210978284 0 29 180224 0 ttyinq 160 375 675 54314565 0 61 168000 0 1024 1024 8 156 104419578 0 13 167936 0 pipe 760 13 207 3343463 0 13 167200 0 256 256 64 551 160665622 0 61 157440 0 rl_entry 40 2087 1813 2087 0 253 156000 0 512 512 153 151 11237 0 29 155648 0 12 Bucket 96 58 1496 3382718 0 125 149184 0 unpcb 256 14 556 6752 0 61 145920 0 ttyoutq 256 197 328 28967762 0 61 134400 0 6 Bucket 48 29 2710 1490923 0 253 131472 0 8192 8192 16 0 446014 0 1 131072 0 32 32 1263 2612 375707 0 253 124000 0 fakepg 104 0 1140 968 0 125 118560 0 4 Bucket 32 157 3343 3692443 0 253 112000 0 2048 2048 23 23 40252916 0 5 94208 0 512 512 144 33 5212 0 29 90624 0 4096 4096 15 7 189 0 1 90112 0 32 32 131 2611 959119 0 253 87744 0 32 32 79 2545 300 0 253 83968 0 udp_inpcb 488 9 151 408 0 29 78080 0 reference_cache 40 24 1876 533168 0 253 76000 0 16 16 2510 2240 2072457 0 253 76000 0 1024 1024 19 53 11971 0 13 73728 0 zio_buf_3072 3072 23 0 1795 0 3 70656 0 32 32 85 2039 531 0 253 67968 0 16384 16384 4 0 4 0 1 65536 0 8192 8192 8 0 1798 0 1 65536 0 swrbuf 984 0 65 440274 4 5 63960 0 16 16 330 2907 333676111 0 253 51792 0 tcpcb 1000 6 42 538 0 13 48000 0 16 16 177 2816 589114 0 253 47888 0 tcp_inpcb 488 6 90 538 0 29 46848 0 UMA Kegs 216 209 6 351 0 61 46440 0 2048 2048 12 10 7376472 0 5 45056 0 128 128 19 289 4852023 0 125 39424 0 cpuset 104 24 345 24 0 125 38376 0 32768 32768 1 0 2058 0 1 32768 0 32768 32768 1 0 9 0 1 32768 0 32768 32768 1 0 4 0 1 32768 0 16384 16384 2 0 1330692 0 1 32768 0 512 512 49 15 49 0 29 32768 0 KMAP ENTRY 96 26 305 88 0 125 31776 0 rtentry 208 12 140 16 0 61 31616 0 64 pcpu 8 3096 836 4691 0 253 31456 0 zio_buf_2048 2048 8 6 5666 0 5 28672 0 zio_data_buf_512 512 8 48 387 0 29 28672 0 512 512 26 30 778556 0 29 28672 0 512 512 17 39 57 0 29 28672 0 udpcb 32 9 866 408 0 253 28000 0 16 16 22 1728 214775 0 253 28000 0 zio_buf_2560 2560 10 0 4482 0 3 25600 0 8192 8192 3 0 49176 0 1 24576 0 8192 8192 3 0 3 0 1 24576 0 8192 8192 3 0 758 0 1 24576 0 reference_history_cache 8 24 2476 533168 0 253 20000 0 16 16 33 1217 76 0 253 20000 0 16 16 27 1223 31 0 253 20000 0 clpbuf 984 0 20 12260978 1907185 5 19680 0 16384 16384 1 0 22 0 1 16384 0 16384 16384 1 0 4 0 1 16384 0 2048 2048 5 3 3289381 0 5 16384 0 mt_stats_zone 64 7 204 47 0 253 13504 0 2048 2048 4 2 4 0 5 12288 0 1024 1024 6 6 24 0 13 12288 0 L VFS Cache 328 1 35 26769 0 29 11808 0 swwbuf 984 0 11 2464690 0 1 10824 0 mt_zone 24 7 399 47 0 253 9744 0 mbuf_jumbo_page 4096 0 2 53417947 0 1 8192 0 4096 4096 2 0 16572 0 1 8192 0 4096 4096 2 0 185235649 0 1 8192 0 1024 1024 2 6 2645 0 13 8192 0 int pcpu 4 336 1712 336 0 253 8192 0 32 32 3 247 25884 0 253 8000 0 mqnode 416 3 15 3 0 29 7488 0 epoch_record pcpu 256 4 12 4 0 61 4096 0 4096 4096 1 0 61284 0 1 4096 0 2048 2048 1 1 100915 0 5 4096 0 2048 2048 1 1 40 0 5 4096 0 umtx_shm 88 1 44 1 0 125 3960 0 procdesc 136 1 28 42 0 61 3944 0 ripcb 488 1 7 3 0 29 3904 0 MAP 216 0 15 0 0 61 3240 0 UMA Hash 256 2 9 2 0 61 2816 0 vmem 1856 0 1 0 0 5 1856 0 sio_cache 128 0 0 0 0 125 0 0 zil_zcw_cache 80 0 0 0 0 125 0 0 zil_lwb_cache 320 0 0 0 0 29 0 0 arc_buf_hdr_t_l2only 96 0 0 0 0 125 0 0 lz4_ctx 16384 0 0 158187 0 1 0 0 zio_data_buf_16777216 16777216 0 0 0 0 1 0 0 zio_buf_16777216 16777216 0 0 0 0 1 0 0 zio_data_buf_14680064 14680064 0 0 0 0 1 0 0 zio_buf_14680064 14680064 0 0 0 0 1 0 0 zio_data_buf_12582912 12582912 0 0 0 0 1 0 0 zio_buf_12582912 12582912 0 0 0 0 1 0 0 zio_data_buf_10485760 10485760 0 0 0 0 1 0 0 zio_buf_10485760 10485760 0 0 0 0 1 0 0 zio_data_buf_8388608 8388608 0 0 0 0 1 0 0 zio_buf_8388608 8388608 0 0 0 0 1 0 0 zio_data_buf_7340032 7340032 0 0 0 0 1 0 0 zio_buf_7340032 7340032 0 0 0 0 1 0 0 zio_data_buf_6291456 6291456 0 0 0 0 1 0 0 zio_buf_6291456 6291456 0 0 0 0 1 0 0 zio_data_buf_5242880 5242880 0 0 0 0 1 0 0 zio_buf_5242880 5242880 0 0 0 0 1 0 0 zio_data_buf_4194304 4194304 0 0 0 0 1 0 0 zio_buf_4194304 4194304 0 0 0 0 1 0 0 zio_data_buf_3670016 3670016 0 0 0 0 1 0 0 zio_buf_3670016 3670016 0 0 0 0 1 0 0 zio_data_buf_3145728 3145728 0 0 0 0 1 0 0 zio_buf_3145728 3145728 0 0 0 0 1 0 0 zio_data_buf_2621440 2621440 0 0 0 0 1 0 0 zio_buf_2621440 2621440 0 0 0 0 1 0 0 zio_data_buf_2097152 2097152 0 0 0 0 1 0 0 zio_buf_2097152 2097152 0 0 0 0 1 0 0 zio_data_buf_1835008 1835008 0 0 0 0 1 0 0 zio_buf_1835008 1835008 0 0 0 0 1 0 0 zio_data_buf_1572864 1572864 0 0 0 0 1 0 0 zio_buf_1572864 1572864 0 0 0 0 1 0 0 zio_data_buf_1310720 1310720 0 0 0 0 1 0 0 zio_buf_1310720 1310720 0 0 0 0 1 0 0 zio_data_buf_1048576 1048576 0 0 0 0 1 0 0 zio_buf_1048576 1048576 0 0 21 0 1 0 0 zio_data_buf_917504 917504 0 0 0 0 1 0 0 zio_buf_917504 917504 0 0 4 0 1 0 0 zio_data_buf_786432 786432 0 0 0 0 1 0 0 zio_buf_786432 786432 0 0 3 0 1 0 0 zio_data_buf_655360 655360 0 0 0 0 1 0 0 zio_buf_655360 655360 0 0 16 0 1 0 0 zio_data_buf_524288 524288 0 0 0 0 1 0 0 zio_buf_524288 524288 0 0 9 0 1 0 0 zio_data_buf_458752 458752 0 0 0 0 1 0 0 zio_buf_458752 458752 0 0 40 0 1 0 0 zio_data_buf_393216 393216 0 0 0 0 1 0 0 zio_buf_393216 393216 0 0 204 0 1 0 0 zio_data_buf_327680 327680 0 0 0 0 1 0 0 zio_buf_327680 327680 0 0 549 0 1 0 0 zio_data_buf_262144 262144 0 0 0 0 1 0 0 zio_buf_262144 262144 0 0 902 0 1 0 0 zio_data_buf_229376 229376 0 0 0 0 1 0 0 zio_buf_229376 229376 0 0 1417 0 1 0 0 zio_data_buf_196608 196608 0 0 0 0 1 0 0 zio_buf_196608 196608 0 0 2039 0 1 0 0 zio_data_buf_163840 163840 0 0 0 0 1 0 0 zio_buf_163840 163840 0 0 1605 0 1 0 0 zio_data_buf_131072 131072 0 0 0 0 1 0 0 zio_data_buf_114688 114688 0 0 0 0 1 0 0 zio_buf_114688 114688 0 0 1431 0 1 0 0 zio_data_buf_98304 98304 0 0 0 0 1 0 0 zio_buf_98304 98304 0 0 1569 0 1 0 0 zio_data_buf_81920 81920 0 0 0 0 1 0 0 zio_buf_81920 81920 0 0 1984 0 1 0 0 zio_data_buf_65536 65536 0 0 0 0 1 0 0 zio_buf_65536 65536 0 0 1062 0 1 0 0 zio_data_buf_57344 57344 0 0 0 0 1 0 0 zio_buf_57344 57344 0 0 1231 0 1 0 0 zio_data_buf_49152 49152 0 0 0 0 1 0 0 zio_buf_49152 49152 0 0 1112 0 1 0 0 zio_data_buf_40960 40960 0 0 0 0 1 0 0 zio_buf_40960 40960 0 0 1233 0 1 0 0 zio_data_buf_32768 32768 0 0 0 0 1 0 0 zio_buf_32768 32768 0 0 871 0 1 0 0 zio_data_buf_28672 28672 0 0 0 0 1 0 0 zio_buf_28672 28672 0 0 971 0 1 0 0 zio_data_buf_24576 24576 0 0 0 0 1 0 0 zio_buf_24576 24576 0 0 1338 0 1 0 0 zio_data_buf_20480 20480 0 0 0 0 1 0 0 zio_buf_20480 20480 0 0 1210 0 1 0 0 zio_data_buf_16384 16384 0 0 0 0 1 0 0 zio_data_buf_14336 14336 0 0 0 0 1 0 0 zio_buf_14336 14336 0 0 920 0 1 0 0 zio_data_buf_12288 12288 0 0 0 0 1 0 0 zio_buf_12288 12288 0 0 2575 0 1 0 0 zio_data_buf_10240 10240 0 0 0 0 1 0 0 zio_buf_10240 10240 0 0 1650 0 1 0 0 zio_data_buf_8192 8192 0 0 0 0 1 0 0 zio_buf_8192 8192 0 0 2559 0 1 0 0 zio_data_buf_7168 7168 0 0 0 0 1 0 0 zio_buf_7168 7168 0 0 3606 0 1 0 0 zio_data_buf_6144 6144 0 0 0 0 1 0 0 zio_buf_6144 6144 0 0 13206 0 1 0 0 zio_data_buf_5120 5120 0 0 0 0 1 0 0 zio_buf_5120 5120 0 0 76705 0 1 0 0 zio_data_buf_4096 4096 0 0 0 0 1 0 0 zio_data_buf_3584 3584 0 0 0 0 1 0 0 zio_data_buf_3072 3072 0 0 0 0 3 0 0 zio_data_buf_2560 2560 0 0 0 0 3 0 0 zio_data_buf_2048 2048 0 0 0 0 5 0 0 zio_data_buf_1536 1536 0 0 0 0 9 0 0 zio_data_buf_1024 1024 0 0 0 0 13 0 0 taskq_zone 48 0 0 6256 0 253 0 0 mqnotifier 216 0 0 0 0 61 0 0 mvdata 64 0 0 0 0 253 0 0 mqueue 264 0 0 0 0 29 0 0 FFS1 dinode 128 0 0 0 0 125 0 0 udplite_inpcb 488 0 0 0 0 29 0 0 sctp_asconf_ack 48 0 0 0 0 253 0 0 sctp_asconf 40 0 0 0 0 253 0 0 sctp_stream_msg_out 112 0 0 0 0 125 0 0 sctp_readq 152 0 0 0 0 61 0 0 sctp_chunk 152 0 0 0 0 61 0 0 sctp_raddr 736 0 0 0 0 13 0 0 sctp_laddr 48 0 0 4 0 253 0 0 sctp_asoc 2408 0 0 0 0 3 0 0 sctp_ep 1280 0 0 0 0 9 0 0 tcp_log_node 120 0 0 0 0 125 0 0 tcp_log_bucket 144 0 0 0 0 61 0 0 tcp_log 408 0 0 0 0 29 0 0 tcpreass 48 0 0 40 0 253 0 0 tfo_ccache_entries 80 0 0 0 0 125 0 0 tfo 4 0 0 0 0 253 0 0 sackhole 32 0 0 40 0 253 0 0 hostcache 96 0 0 3 0 125 0 0 syncache 168 0 0 189 0 61 0 0 tcptw 88 0 0 120 0 125 0 0 ipq 56 0 0 0 0 253 0 0 IPsec SA lft_c 16 0 0 0 0 253 0 0 KNOTE 160 0 0 2959 0 61 0 0 itimer 352 0 0 0 0 29 0 0 AIOLIO 280 0 0 0 0 29 0 0 AIOCB 752 0 0 0 0 13 0 0 AIOP 32 0 0 0 0 253 0 0 AIO 208 0 0 0 0 61 0 0 NCLNODE 744 0 0 76 0 13 0 0 LTS VFS Cache 368 0 0 12 0 29 0 0 STS VFS Cache 148 0 0 288 0 61 0 0 rentr 24 0 0 49258 0 253 0 0 VNODEPOLL 120 0 0 0 0 125 0 0 vtnet_tx_hdr 24 0 0 0 0 253 0 0 crypto_session 24 0 0 13 0 253 0 0 cryptodesc 120 0 0 0 0 125 0 0 cryptop 128 0 0 0 0 125 0 0 nvme_request 128 0 0 0 0 125 0 0 FPU_save_area 832 0 0 0 0 13 0 0 DMAR_MAP_ENTRY 120 0 0 0 0 125 0 0 mbuf_extpgs 256 0 0 0 0 61 0 0 mbuf_jumbo_16k 16384 0 0 0 0 1 0 0 mbuf_jumbo_9k 9216 0 0 0 0 1 0 0 audit_record 1280 0 0 0 0 9 0 0 domainset 40 0 0 0 0 253 0 0 MAC labels 40 0 0 0 0 253 0 0 vnpbuf 984 0 0 87337 0 13 0 0 mdpbuf 984 0 0 162075 0 1 0 0 nfspbuf 984 0 0 0 0 5 0 0 umtx pi 96 0 0 0 0 125 0 0 rangeset pctrie nodes 144 0 0 0 0 61 0 0 65536 65536 0 0 10 0 1 0 0 65536 65536 0 0 3 0 1 0 0 65536 65536 0 0 0 0 1 0 0 65536 65536 0 0 3141 0 1 0 0 32768 32768 0 0 2267931 0 1 0 0 16384 16384 0 0 0 0 1 0 0 16384 16384 0 0 129 0 1 0 0 8192 8192 0 0 104 0 1 0 0 SMR CPU 16 0 0 0 0 253 0 0 SMR GLOBAL 16 0 0 0 0 253 0 0 db:0:pho> show umacache Zone Size Used Free Requests Bucket debugnet_mbuf_packet 2048 0 0 0 0 debugnet_mbuf_cluster 2048 -6144 0 0 0 debugnet_mbuf 256 -10240 0 0 0 kstack_cache 16384 2227 7 7916051 1 vm pgcache 4096 325756 431 366822674 15 vm pgcache 4096 300011 1193 2673589097 38 buf free cache 984 55830 524 206726597 13 buffer arena-65 65536 0 0 4 1 buffer arena-61 61440 0 0 0 1 buffer arena-57 57344 0 0 0 1 buffer arena-53 53248 0 0 0 1 buffer arena-49 49152 0 0 0 1 buffer arena-45 45056 0 0 0 1 buffer arena-40 40960 0 0 0 1 buffer arena-36 36864 0 0 0 1 buffer arena-32 32768 52906 20 39228710 1 buffer arena-28 28672 0 0 0 1 buffer arena-24 24576 0 0 0 1 buffer arena-20 20480 0 0 0 1 buffer arena-16 16384 -5 1 115967341 1 buffer arena-12 12288 0 0 0 1 buffer arena-81 8192 0 0 0 1 buffer arena-40 4096 0 0 1624 1 db:0:pho> acttrace Tracing command dirnprename pid 19961 tid 102163 td 0xfffff805466d7000 (CPU 2) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e17bdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e17bdf0 trap() at trap+0x61/frame 0xfffffe003e17bf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e17bf20 --- trap 0x13, rip = 0xffffffff80c11a92, rsp = 0xfffffe00ef988320, rbp = 0xfffffe00ef988330 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00ef988330 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00ef9883f0 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00ef988430 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00ef988470 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00ef988550 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00ef988640 zap_create_norm_dnsize() at zap_create_norm_dnsize+0x26/frame 0xfffffe00ef988680 zfs_mknode() at zfs_mknode+0x1f0/frame 0xfffffe00ef9887c0 zfs_freebsd_mkdir() at zfs_freebsd_mkdir+0x48d/frame 0xfffffe00ef988870 VOP_MKDIR_APV() at VOP_MKDIR_APV+0xae/frame 0xfffffe00ef9888a0 kern_mkdirat() at kern_mkdirat+0x1b3/frame 0xfffffe00ef988ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00ef988bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ef988bf0 --- syscall (136, FreeBSD ELF64, sys_mkdir), rip = 0x80042df3a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe3d0 --- Tracing command openat pid 19874 tid 100341 td 0xfffff807b86976e0 (CPU 4) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e193dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e193df0 trap() at trap+0x61/frame 0xfffffe003e193f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e193f20 --- trap 0x13, rip = 0xffffffff8259ac56, rsp = 0xfffffe00e3b723d0, rbp = 0xfffffe00e3b723f0 --- dnode_check_slots_free() at dnode_check_slots_free+0x126/frame 0xfffffe00e3b723f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00e3b724d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00e3b725c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00e3b72700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00e3b727d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00e3b72800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00e3b72950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00e3b72ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e3b72bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e3b72bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command openat pid 19873 tid 100616 td 0xfffff801ac17d6e0 (CPU 5) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e19fdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e19fdf0 trap() at trap+0x61/frame 0xfffffe003e19ff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e19ff20 --- trap 0x13, rip = 0xffffffff80c11a90, rsp = 0xfffffe00ee2d62a0, rbp = 0xfffffe00ee2d62b0 --- lock_delay() at lock_delay+0x30/frame 0xfffffe00ee2d62b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00ee2d6370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00ee2d63b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00ee2d63f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00ee2d64d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00ee2d65c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00ee2d6700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00ee2d67d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00ee2d6800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00ee2d6950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00ee2d6ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00ee2d6bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ee2d6bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command openat pid 19872 tid 100290 td 0xfffff8057d8f86e0 (CPU 11) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1e7dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1e7df0 trap() at trap+0x61/frame 0xfffffe003e1e7f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1e7f20 --- trap 0x13, rip = 0xffffffff80c11a92, rsp = 0xfffffe00e2bc12a0, rbp = 0xfffffe00e2bc12b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00e2bc12b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00e2bc1370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00e2bc13b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00e2bc13f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00e2bc14d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00e2bc15c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00e2bc1700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00e2bc17d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00e2bc1800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00e2bc1950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00e2bc1ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2bc1bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2bc1bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command openat pid 19871 tid 101955 td 0xfffff80278cd96e0 (CPU 0) kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe00eb5750a0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00eb5750b0 uart_intr_rxready() at uart_intr_rxready+0xa5/frame 0xfffffe00eb5750e0 uart_intr() at uart_intr+0x138/frame 0xfffffe00eb575120 intr_event_handle() at intr_event_handle+0xce/frame 0xfffffe00eb575170 intr_execute_handlers() at intr_execute_handlers+0x75/frame 0xfffffe00eb5751b0 lapic_handle_intr() at lapic_handle_intr+0x5b/frame 0xfffffe00eb5751d0 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00eb5751d0 --- interrupt, rip = 0xffffffff80c11a92, rsp = 0xfffffe00eb5752a0, rbp = 0xfffffe00eb5752b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00eb5752b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00eb575370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00eb5753b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00eb5753f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00eb5754d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5755c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb575700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5757d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb575800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb575950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb575ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb575bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb575bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command openat pid 19869 tid 101965 td 0xfffff8038d13c000 (CPU 10) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1dbdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1dbdf0 trap() at trap+0x61/frame 0xfffffe003e1dbf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1dbf20 --- trap 0x13, rip = 0xffffffff82641c07, rsp = 0xfffffe00eb5a73e0, rbp = 0xfffffe00eb5a73f0 --- zrl_tryenter() at zrl_tryenter+0x7/frame 0xfffffe00eb5a73f0 dnode_hold_impl() at dnode_hold_impl+0xbb8/frame 0xfffffe00eb5a74d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5a75c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb5a7700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5a77d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb5a7800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb5a7950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb5a7ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb5a7bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb5a7bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command openat pid 19867 tid 101982 td 0xfffff8019603a6e0 (CPU 9) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1cfdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1cfdf0 trap() at trap+0x61/frame 0xfffffe003e1cff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1cff20 --- trap 0x13, rip = 0xffffffff80c11a92, rsp = 0xfffffe00eb5fc2a0, rbp = 0xfffffe00eb5fc2b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00eb5fc2b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00eb5fc370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00eb5fc3b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00eb5fc3f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00eb5fc4d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5fc5c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb5fc700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5fc7d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb5fc800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb5fc950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb5fcac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb5fcbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb5fcbf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command openat pid 19866 tid 102244 td 0xfffff801ca3b5000 (CPU 3) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e187dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e187df0 trap() at trap+0x61/frame 0xfffffe003e187f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e187f20 --- trap 0x13, rip = 0xffffffff80c11a92, rsp = 0xfffffe00e83212a0, rbp = 0xfffffe00e83212b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00e83212b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00e8321370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00e83213b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00e83213f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00e83214d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00e83215c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00e8321700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00e83217d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00e8321800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00e8321950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00e8321ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e8321bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e8321bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command openat pid 19864 tid 100245 td 0xfffff8057d8e46e0 (CPU 6) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1abdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1abdf0 trap() at trap+0x61/frame 0xfffffe003e1abf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1abf20 --- trap 0x13, rip = 0xffffffff80c11a92, rsp = 0xfffffe00e3a0e2a0, rbp = 0xfffffe00e3a0e2b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00e3a0e2b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00e3a0e370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00e3a0e3b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00e3a0e3f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00e3a0e4d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00e3a0e5c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00e3a0e700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00e3a0e7d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00e3a0e800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00e3a0e950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00e3a0eac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e3a0ebf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e3a0ebf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command openat pid 19862 tid 100566 td 0xfffff801ac8d86e0 (CPU 1) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e16fdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e16fdf0 trap() at trap+0x61/frame 0xfffffe003e16ff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e16ff20 --- trap 0x13, rip = 0xffffffff80c11a90, rsp = 0xfffffe00e4fda2a0, rbp = 0xfffffe00e4fda2b0 --- lock_delay() at lock_delay+0x30/frame 0xfffffe00e4fda2b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00e4fda370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00e4fda3b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00e4fda3f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00e4fda4d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00e4fda5c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00e4fda700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00e4fda7d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00e4fda800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00e4fda950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00e4fdaac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e4fdabf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e4fdabf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command idle pid 11 tid 100010 td 0xfffff800038696e0 (CPU 7) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1b7dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1b7df0 trap() at trap+0x61/frame 0xfffffe003e1b7f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1b7f20 --- trap 0x13, rip = 0xffffffff8102a0b6, rsp = 0xfffffe00adda1a30, rbp = 0xfffffe00adda1a30 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00adda1a30 acpi_cpu_idle() at acpi_cpu_idle+0x23d/frame 0xfffffe00adda1a90 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00adda1ab0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00adda1ad0 sched_idletd() at sched_idletd+0x401/frame 0xfffffe00adda1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adda1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adda1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffff8000388d000 (CPU 8) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1c3dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1c3df0 trap() at trap+0x61/frame 0xfffffe003e1c3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1c3f20 --- trap 0x13, rip = 0xffffffff8102a0b6, rsp = 0xfffffe00adda6a30, rbp = 0xfffffe00adda6a30 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00adda6a30 acpi_cpu_idle() at acpi_cpu_idle+0x23d/frame 0xfffffe00adda6a90 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00adda6ab0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00adda6ad0 sched_idletd() at sched_idletd+0x401/frame 0xfffffe00adda6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adda6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adda6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db:0:pho> ps pid ppid pgrp uid state wmesg wchan cmd 14234 2997 2930 0 S nanslp 0xffffffff81c80274 sleep 30742 30740 27773 0 S piperd 0xfffff801a09155f0 cat 30740 30735 27773 0 S wait 0xfffff80546669530 sh 30739 30735 27773 0 D zfs 0xfffff8070b24ecc0 find 30735 30733 27773 0 S wait 0xfffff8004d2e4a60 sh 30734 30726 27773 0 S piperd 0xfffff800441922f8 mail 30733 30726 27773 0 S wait 0xfffff801ac3d5000 sh 30726 30725 27773 0 S wait 0xfffff8023b0e3000 sh 30725 30723 27773 0 S wait 0xfffff803dc042a60 lockf 30723 30722 27773 0 S wait 0xfffff80190c2e000 sh 30722 27784 27773 0 S wait 0xfffff802a0751000 sh 27785 27776 27773 0 S piperd 0xfffff806f8a87000 mail 27784 27776 27773 0 S wait 0xfffff8077378a000 sh 27776 27775 27773 0 S wait 0xfffff80796c1da60 sh 27775 27773 27773 0 S wait 0xfffff801b1e93000 lockf 27773 27771 27773 0 Ss wait 0xfffff801b195a000 sh 27771 2312 2312 0 S piperd 0xfffff8021727b2f8 cron 20682 19946 2490 0 D+ zfs 0xfffff8070b24ecc0 sh 20486 19956 2490 0 D+ zfs 0xfffff8070b24ecc0 sh 20414 19957 2490 0 D+ zfs 0xfffff8070b24ecc0 sh 20407 19949 2490 0 D+ zfs 0xfffff8070b24ecc0 fstat 20406 19947 2490 0 D+ zfs 0xfffff8070b24ecc0 sh 20380 19955 2490 0 D+ zfs 0xfffff8070b24ecc0 sh 20345 19951 2490 0 D+ zfs 0xfffff8070b24ecc0 fstat 20344 19950 2490 0 D+ zfs 0xfffff8070b24ecc0 fstat 20250 19953 2490 0 D+ zfs 0xfffff8070b24ecc0 fstat 20241 19952 2490 0 D+ zfs 0xfffff8070b24ecc0 fstat 20237 19948 2490 0 D+ zfs 0xfffff8070b24ecc0 fstat 20230 19954 2490 0 D+ zfs 0xfffff8070b24ecc0 fstat 19977 19960 2490 0 D+ zfs 0xfffff8070b24ecc0 dirnprename 19976 19960 2490 0 D+ zfs 0xfffff8070b24ecc0 dirnprename 19975 19960 2490 0 D+ zfs 0xfffff8070b24ecc0 dirnprename 19974 19960 2490 0 D+ zfs 0xfffff8070b24ecc0 dirnprename 19973 19960 2490 0 D+ zfs 0xfffff8070b24ecc0 dirnprename 19972 19960 2490 0 D+ zfs 0xfffff8070b24ecc0 dirnprename 19971 19960 2490 0 D+ zfs 0xfffff8070b24ecc0 dirnprename 19970 19960 2490 0 D+ zfs 0xfffff8070b24ecc0 dirnprename 19969 19960 2490 0 D+ zfs 0xfffff8070b24ecc0 dirnprename 19968 19960 2490 0 D+ zfs 0xfffff8070b24ecc0 dirnprename 19967 19960 2490 0 D+ zfs 0xfffff8070b24ecc0 dirnprename 19966 19960 2490 0 D+ zfs 0xfffff8070b24ecc0 dirnprename 19965 19960 2490 0 D+ zfs 0xfffff8070b24ecc0 dirnprename 19964 19960 2490 0 D+ zfs 0xfffff8070b24ecc0 dirnprename 19963 19960 2490 0 D+ zfs 0xfffff8070b24ecc0 dirnprename 19962 19960 2490 0 D+ zfs 0xfffff8070b24ecc0 dirnprename 19961 19960 2490 0 R+ CPU 2 dirnprename 19960 19868 2490 0 S+ wait 0xfffff80044126000 dirnprename 19957 19945 2490 0 S+ wait 0xfffff80030527530 sysctl 19956 19945 2490 0 S+ wait 0xfffff804f36ba000 sysctl 19955 19945 2490 0 S+ wait 0xfffff804f36ba530 sysctl 19954 19945 2490 0 S+ wait 0xfffff804f36baa60 sysctl 19953 19945 2490 0 S+ wait 0xfffff804f36d6a60 sysctl 19952 19945 2490 0 S+ wait 0xfffff80050d1f530 sysctl 19951 19945 2490 0 S+ wait 0xfffff8002e156a60 sysctl 19950 19945 2490 0 S+ wait 0xfffff806e40e1530 sysctl 19949 19945 2490 0 S+ wait 0xfffff80109e04530 sysctl 19948 19945 2490 0 S+ wait 0xfffff80796c1d000 sysctl 19947 19945 2490 0 S+ wait 0xfffff804f36a3530 sysctl 19946 19945 2490 0 S+ wait 0xfffff8053f130530 sysctl 19945 19843 2490 0 S+ wait 0xfffff8039a4d5a60 sysctl 19944 19942 2490 0 D+ zfs 0xfffff8070b24ecc0 dirrename 19943 19942 2490 0 D+ zfs 0xfffff8070b24ecc0 dirrename 19942 19870 2490 0 S+ wait 0xfffff80189fbb530 dirrename 19909 19891 2490 0 D+ zfs 0xfffff8070b24ecc0 mkdir 19906 19891 2490 0 D+ zfs 0xfffff8070b24ecc0 mkdir 19904 19891 2490 0 D+ zfs 0xfffff8070b24ecc0 mkdir 19902 19891 2490 0 D+ zfs 0xfffff8070b24ecc0 mkdir 19900 19891 2490 0 D+ zfs 0xfffff8070b24ecc0 mkdir 19899 19891 2490 0 D+ zfs 0xfffff8070b24ecc0 mkdir 19897 19891 2490 0 D+ zfs 0xfffff8070b24ecc0 mkdir 19896 19891 2490 0 D+ zfs 0xfffff8070b24ecc0 mkdir 19895 19891 2490 0 D+ zfs 0xfffff8070b24ecc0 mkdir 19894 19891 2490 0 D+ zfs 0xfffff8070b24ecc0 mkdir 19893 19891 2490 0 D+ zfs 0xfffff8070b24ecc0 mkdir 19892 19891 2490 0 D+ zfs 0xfffff8070b24ecc0 mkdir 19891 19857 2490 0 S+ wait 0xfffff802f9fa2000 mkdir 19890 19875 2490 0 D+ dn->dn_ 0xfffff808006e41d0 link 19889 19875 2490 0 D+ dn->dn_ 0xfffff808006e41d0 link 19887 19875 2490 0 D+ dn->dn_ 0xfffff808006e41d0 link 19885 19875 2490 0 D+ zfs 0xfffff8070b24ecc0 link 19884 19876 2490 0 D+ zfs 0xfffff8070b24ecc0 lockf 19883 19875 2490 0 D+ dn->dn_ 0xfffff808006e41d0 link 19882 19876 2490 0 D+ zfs 0xfffff8070b24ecc0 lockf 19881 19875 2490 0 D+ zfs 0xfffff8070b24ecc0 link 19880 19876 2490 0 D+ zfs 0xfffff8070b24ecc0 lockf 19879 19875 2490 0 D+ dn->dn_ 0xfffff808006e41d0 link 19878 19876 2490 0 D+ zfs 0xfffff8070b24ecc0 lockf 19877 19875 2490 0 D+ dn->dn_ 0xfffff808006e41d0 link 19876 19855 2490 0 S+ wait 0xfffff801b11fe000 lockf 19875 19854 2490 0 S+ wait 0xfffff80174c75a60 link 19874 19859 2490 0 R+ CPU 4 openat 19873 19859 2490 0 R+ CPU 5 openat 19872 19859 2490 0 R+ CPU 11 openat 19871 19859 2490 0 R+ CPU 0 openat 19870 19838 2490 0 S+ wait 0xfffff803dc09e000 dirrename 19869 19859 2490 0 R+ CPU 10 openat 19868 19838 2490 0 S+ wait 0xfffff80432c95000 dirnprename 19867 19859 2490 0 R+ CPU 9 openat 19866 19859 2490 0 R+ CPU 3 openat 19864 19859 2490 0 R+ CPU 6 openat 19862 19859 2490 0 R+ CPU 1 openat 19859 19847 2490 0 S+ wait 0xfffff804f3605a60 openat 19857 19838 2490 0 S+ wait 0xfffff80191cd4530 mkdir 19855 19838 2490 0 S+ wait 0xfffff8004d208a60 lockf 19854 19838 2490 0 S+ wait 0xfffff806bdf2a000 link 19847 19838 2490 0 S+ wait 0xfffff80109ef8000 openat 19843 19838 2490 0 S+ wait 0xfffff80044126a60 sysctl 19838 19836 2490 0 S+ wait 0xfffff8015cff8000 run 19836 19834 2490 0 S+ wait 0xfffff80030527000 run 19834 19810 2490 0 S+ wait 0xfffff802b8f2d530 run 19810 19629 2490 0 SW+ wait 0xfffff802a0751530 sh 19734 0 0 0 DL (threaded) [zpool-stress2_tank] 101835 D spa->sp 0xfffffe00d9fe6cd8 [stress2_tank] 100995 D - 0xfffff805f1e98100 [zio_null_issue] 100998 D - 0xfffff80205a0d000 [zio_null_intr] 101000 D - 0xfffff80205a0d500 [zio_read_issue_0] 101003 D - 0xfffff80205a0d500 [zio_read_issue_1] 101007 D - 0xfffff80205a0d500 [zio_read_issue_2] 101009 D - 0xfffff80205a0d500 [zio_read_issue_3] 101010 D - 0xfffff80205a0d500 [zio_read_issue_4] 101012 D - 0xfffff80205a0d500 [zio_read_issue_5] 101019 D - 0xfffff80205a0d500 [zio_read_issue_6] 101022 D - 0xfffff80205a0d500 [zio_read_issue_7] 101024 D - 0xfffff80808394600 [zio_read_intr_0_0] 101026 D - 0xfffff80808394600 [zio_read_intr_0_1] 101027 D - 0xfffff80808394600 [zio_read_intr_0_2] 101036 D - 0xfffff80808394600 [zio_read_intr_0_3] 101039 D - 0xfffff80808394600 [zio_read_intr_0_4] 101040 D - 0xfffff80808394600 [zio_read_intr_0_5] 101042 D - 0xfffff80808394600 [zio_read_intr_0_6] 101043 D - 0xfffff80808394600 [zio_read_intr_0_7] 101044 D - 0xfffff80808394600 [zio_read_intr_0_8] 101051 D - 0xfffff80808394600 [zio_read_intr_0_9] 101053 D - 0xfffff80808394600 [zio_read_intr_0_10] 101057 D - 0xfffff80808394600 [zio_read_intr_0_11] 101060 D - 0xfffff805be212100 [zio_read_intr_1_0] 101061 D - 0xfffff805be212100 [zio_read_intr_1_1] 101070 D - 0xfffff805be212100 [zio_read_intr_1_2] 101071 D - 0xfffff805be212100 [zio_read_intr_1_3] 101073 D - 0xfffff805be212100 [zio_read_intr_1_4] 101074 D - 0xfffff805be212100 [zio_read_intr_1_5] 101075 D - 0xfffff805be212100 [zio_read_intr_1_6] 101077 D - 0xfffff805be212100 [zio_read_intr_1_7] 101082 D - 0xfffff805be212100 [zio_read_intr_1_8] 101083 D - 0xfffff805be212100 [zio_read_intr_1_9] 101087 D - 0xfffff805be212100 [zio_read_intr_1_10] 101088 D - 0xfffff805be212100 [zio_read_intr_1_11] 101089 D - 0xfffff801f90ca000 [zio_read_intr_2_0] 101092 D - 0xfffff801f90ca000 [zio_read_intr_2_1] 101096 D - 0xfffff801f90ca000 [zio_read_intr_2_2] 101098 D - 0xfffff801f90ca000 [zio_read_intr_2_3] 101102 D - 0xfffff801f90ca000 [zio_read_intr_2_4] 101107 D - 0xfffff801f90ca000 [zio_read_intr_2_5] 101108 D - 0xfffff801f90ca000 [zio_read_intr_2_6] 101110 D - 0xfffff801f90ca000 [zio_read_intr_2_7] 101113 D - 0xfffff801f90ca000 [zio_read_intr_2_8] 101116 D - 0xfffff801f90ca000 [zio_read_intr_2_9] 101117 D - 0xfffff801f90ca000 [zio_read_intr_2_10] 101119 D - 0xfffff801f90ca000 [zio_read_intr_2_11] 101121 D - 0xfffff80808393d00 [zio_read_intr_3_0] 101128 D - 0xfffff80808393d00 [zio_read_intr_3_1] 101129 D - 0xfffff80808393d00 [zio_read_intr_3_2] 101132 D - 0xfffff80808393d00 [zio_read_intr_3_3] 101137 D - 0xfffff80808393d00 [zio_read_intr_3_4] 101138 D - 0xfffff80808393d00 [zio_read_intr_3_5] 101143 D - 0xfffff80808393d00 [zio_read_intr_3_6] 101145 D - 0xfffff80808393d00 [zio_read_intr_3_7] 101146 D - 0xfffff80808393d00 [zio_read_intr_3_8] 101147 D - 0xfffff80808393d00 [zio_read_intr_3_9] 101148 D - 0xfffff80808393d00 [zio_read_intr_3_10] 101152 D - 0xfffff80808393d00 [zio_read_intr_3_11] 101159 D - 0xfffff80464585c00 [zio_read_intr_4_0] 101161 D - 0xfffff80464585c00 [zio_read_intr_4_1] 101163 D - 0xfffff80464585c00 [zio_read_intr_4_2] 101164 D - 0xfffff80464585c00 [zio_read_intr_4_3] 101168 D - 0xfffff80464585c00 [zio_read_intr_4_4] 101169 D - 0xfffff80464585c00 [zio_read_intr_4_5] 101174 D - 0xfffff80464585c00 [zio_read_intr_4_6] 101176 D - 0xfffff80464585c00 [zio_read_intr_4_7] 101177 D - 0xfffff80464585c00 [zio_read_intr_4_8] 101180 D - 0xfffff80464585c00 [zio_read_intr_4_9] 101182 D - 0xfffff80464585c00 [zio_read_intr_4_10] 101183 D - 0xfffff80464585c00 [zio_read_intr_4_11] 101184 D - 0xfffff8049069e000 [zio_read_intr_5_0] 101189 D - 0xfffff8049069e000 [zio_read_intr_5_1] 101191 D - 0xfffff8049069e000 [zio_read_intr_5_2] 101194 D - 0xfffff8049069e000 [zio_read_intr_5_3] 101198 D - 0xfffff8049069e000 [zio_read_intr_5_4] 101199 D - 0xfffff8049069e000 [zio_read_intr_5_5] 101200 D - 0xfffff8049069e000 [zio_read_intr_5_6] 101202 D - 0xfffff8049069e000 [zio_read_intr_5_7] 101203 D - 0xfffff8049069e000 [zio_read_intr_5_8] 101204 D - 0xfffff8049069e000 [zio_read_intr_5_9] 101206 D - 0xfffff8049069e000 [zio_read_intr_5_10] 101208 D - 0xfffff8049069e000 [zio_read_intr_5_11] 101209 D - 0xfffff807fd530e00 [zio_read_intr_6_0] 101210 D - 0xfffff807fd530e00 [zio_read_intr_6_1] 101213 D - 0xfffff807fd530e00 [zio_read_intr_6_2] 101214 D - 0xfffff807fd530e00 [zio_read_intr_6_3] 101215 D - 0xfffff807fd530e00 [zio_read_intr_6_4] 101218 D - 0xfffff807fd530e00 [zio_read_intr_6_5] 101219 D - 0xfffff807fd530e00 [zio_read_intr_6_6] 101220 D - 0xfffff807fd530e00 [zio_read_intr_6_7] 101221 D - 0xfffff807fd530e00 [zio_read_intr_6_8] 101222 D - 0xfffff807fd530e00 [zio_read_intr_6_9] 101224 D - 0xfffff807fd530e00 [zio_read_intr_6_10] 101225 D - 0xfffff807fd530e00 [zio_read_intr_6_11] 101226 D - 0xfffff804b891a700 [zio_read_intr_7_0] 101230 D - 0xfffff804b891a700 [zio_read_intr_7_1] 101234 D - 0xfffff804b891a700 [zio_read_intr_7_2] 101235 D - 0xfffff804b891a700 [zio_read_intr_7_3] 101236 D - 0xfffff804b891a700 [zio_read_intr_7_4] 101237 D - 0xfffff804b891a700 [zio_read_intr_7_5] 101238 D - 0xfffff804b891a700 [zio_read_intr_7_6] 101252 D - 0xfffff804b891a700 [zio_read_intr_7_7] 101253 D - 0xfffff804b891a700 [zio_read_intr_7_8] 101263 D - 0xfffff804b891a700 [zio_read_intr_7_9] 101264 D - 0xfffff804b891a700 [zio_read_intr_7_10] 101265 D - 0xfffff804b891a700 [zio_read_intr_7_11] 101266 D - 0xfffff80166778c00 [zio_write_issue_0] 101268 D - 0xfffff80166778c00 [zio_write_issue_1] 101270 D - 0xfffff80166778c00 [zio_write_issue_2] 101271 D - 0xfffff80166778c00 [zio_write_issue_3] 101273 D - 0xfffff80166778c00 [zio_write_issue_4] 101274 D - 0xfffff80166778c00 [zio_write_issue_5] 101275 D - 0xfffff80166778c00 [zio_write_issue_6] 101277 D - 0xfffff80166778c00 [zio_write_issue_7] 101279 D - 0xfffff80166778c00 [zio_write_issue_8] 101280 D - 0xfffff8064008dd00 [zio_write_issue_hig] 101281 D - 0xfffff8064008dd00 [zio_write_issue_hig] 101283 D - 0xfffff8064008dd00 [zio_write_issue_hig] 101284 D - 0xfffff8064008dd00 [zio_write_issue_hig] 101286 D - 0xfffff8064008dd00 [zio_write_issue_hig] 101287 D - 0xfffff805be212c00 [zio_write_intr_0] 101288 D - 0xfffff805be212c00 [zio_write_intr_1] 101291 D - 0xfffff805be212c00 [zio_write_intr_2] 101293 D - 0xfffff805be212c00 [zio_write_intr_3] 101294 D - 0xfffff805be212c00 [zio_write_intr_4] 101295 D - 0xfffff805be212c00 [zio_write_intr_5] 101297 D - 0xfffff805be212c00 [zio_write_intr_6] 101298 D - 0xfffff805be212c00 [zio_write_intr_7] 101299 D - 0xfffff8047ed3bc00 [zio_write_intr_high] 101301 D - 0xfffff8047ed3bc00 [zio_write_intr_high] 101302 D - 0xfffff8047ed3bc00 [zio_write_intr_high] 101303 D - 0xfffff8047ed3bc00 [zio_write_intr_high] 101304 D - 0xfffff8047ed3bc00 [zio_write_intr_high] 101306 D - 0xfffff800038cb200 [zio_free_issue_0_0] 101307 D - 0xfffff800038cb200 [zio_free_issue_0_1] 101308 D - 0xfffff800038cb200 [zio_free_issue_0_2] 101310 D - 0xfffff800038cb200 [zio_free_issue_0_3] 101311 D - 0xfffff800038cb200 [zio_free_issue_0_4] 101312 D - 0xfffff800038cb200 [zio_free_issue_0_5] 101314 D - 0xfffff800038cb200 [zio_free_issue_0_6] 101315 D - 0xfffff800038cb200 [zio_free_issue_0_7] 101316 D - 0xfffff800038cb200 [zio_free_issue_0_8] 101317 D - 0xfffff800038cb200 [zio_free_issue_0_9] 101319 D - 0xfffff800038cb200 [zio_free_issue_0_10] 101320 D - 0xfffff800038cb200 [zio_free_issue_0_11] 101321 D - 0xfffff8080841ac00 [zio_free_issue_1_0] 101323 D - 0xfffff8080841ac00 [zio_free_issue_1_1] 101326 D - 0xfffff8080841ac00 [zio_free_issue_1_2] 101327 D - 0xfffff8080841ac00 [zio_free_issue_1_3] 101329 D - 0xfffff8080841ac00 [zio_free_issue_1_4] 101335 D - 0xfffff8080841ac00 [zio_free_issue_1_5] 101336 D - 0xfffff8080841ac00 [zio_free_issue_1_6] 101340 D - 0xfffff8080841ac00 [zio_free_issue_1_7] 101343 D - 0xfffff8080841ac00 [zio_free_issue_1_8] 101347 D - 0xfffff8080841ac00 [zio_free_issue_1_9] 101349 D - 0xfffff8080841ac00 [zio_free_issue_1_10] 101351 D - 0xfffff8080841ac00 [zio_free_issue_1_11] 101356 D - 0xfffff80808392500 [zio_free_issue_2_0] 101361 D - 0xfffff80808392500 [zio_free_issue_2_1] 101365 D - 0xfffff80808392500 [zio_free_issue_2_2] 101366 D - 0xfffff80808392500 [zio_free_issue_2_3] 101367 D - 0xfffff80808392500 [zio_free_issue_2_4] 101368 D - 0xfffff80808392500 [zio_free_issue_2_5] 101369 D - 0xfffff80808392500 [zio_free_issue_2_6] 101370 D - 0xfffff80808392500 [zio_free_issue_2_7] 101371 D - 0xfffff80808392500 [zio_free_issue_2_8] 101372 D - 0xfffff80808392500 [zio_free_issue_2_9] 101373 D - 0xfffff80808392500 [zio_free_issue_2_10] 101374 D - 0xfffff80808392500 [zio_free_issue_2_11] 101375 D - 0xfffff803ed7bab00 [zio_free_issue_3_0] 101376 D - 0xfffff803ed7bab00 [zio_free_issue_3_1] 101377 D - 0xfffff803ed7bab00 [zio_free_issue_3_2] 101378 D - 0xfffff803ed7bab00 [zio_free_issue_3_3] 101379 D - 0xfffff803ed7bab00 [zio_free_issue_3_4] 101380 D - 0xfffff803ed7bab00 [zio_free_issue_3_5] 101381 D - 0xfffff803ed7bab00 [zio_free_issue_3_6] 101382 D - 0xfffff803ed7bab00 [zio_free_issue_3_7] 101383 D - 0xfffff803ed7bab00 [zio_free_issue_3_8] 101384 D - 0xfffff803ed7bab00 [zio_free_issue_3_9] 101385 D - 0xfffff803ed7bab00 [zio_free_issue_3_10] 101386 D - 0xfffff803ed7bab00 [zio_free_issue_3_11] 101388 D - 0xfffff800038cb900 [zio_free_issue_4_0] 101389 D - 0xfffff800038cb900 [zio_free_issue_4_1] 101390 D - 0xfffff800038cb900 [zio_free_issue_4_2] 101391 D - 0xfffff800038cb900 [zio_free_issue_4_3] 101392 D - 0xfffff800038cb900 [zio_free_issue_4_4] 101393 D - 0xfffff800038cb900 [zio_free_issue_4_5] 101394 D - 0xfffff800038cb900 [zio_free_issue_4_6] 101395 D - 0xfffff800038cb900 [zio_free_issue_4_7] 101397 D - 0xfffff800038cb900 [zio_free_issue_4_8] 101399 D - 0xfffff800038cb900 [zio_free_issue_4_9] 101401 D - 0xfffff800038cb900 [zio_free_issue_4_10] 101403 D - 0xfffff800038cb900 [zio_free_issue_4_11] 101404 D - 0xfffff805998b1a00 [zio_free_issue_5_0] 101406 D - 0xfffff805998b1a00 [zio_free_issue_5_1] 101408 D - 0xfffff805998b1a00 [zio_free_issue_5_2] 101409 D - 0xfffff805998b1a00 [zio_free_issue_5_3] 101414 D - 0xfffff805998b1a00 [zio_free_issue_5_4] 101415 D - 0xfffff805998b1a00 [zio_free_issue_5_5] 101416 D - 0xfffff805998b1a00 [zio_free_issue_5_6] 101417 D - 0xfffff805998b1a00 [zio_free_issue_5_7] 101421 D - 0xfffff805998b1a00 [zio_free_issue_5_8] 101424 D - 0xfffff805998b1a00 [zio_free_issue_5_9] 101429 D - 0xfffff805998b1a00 [zio_free_issue_5_10] 101430 D - 0xfffff805998b1a00 [zio_free_issue_5_11] 101431 D - 0xfffff801f90c5a00 [zio_free_issue_6_0] 101433 D - 0xfffff801f90c5a00 [zio_free_issue_6_1] 101437 D - 0xfffff801f90c5a00 [zio_free_issue_6_2] 101439 D - 0xfffff801f90c5a00 [zio_free_issue_6_3] 101444 D - 0xfffff801f90c5a00 [zio_free_issue_6_4] 101445 D - 0xfffff801f90c5a00 [zio_free_issue_6_5] 101446 D - 0xfffff801f90c5a00 [zio_free_issue_6_6] 101449 D - 0xfffff801f90c5a00 [zio_free_issue_6_7] 101452 D - 0xfffff801f90c5a00 [zio_free_issue_6_8] 101453 D - 0xfffff801f90c5a00 [zio_free_issue_6_9] 101454 D - 0xfffff801f90c5a00 [zio_free_issue_6_10] 101456 D - 0xfffff801f90c5a00 [zio_free_issue_6_11] 101464 D - 0xfffff80808417800 [zio_free_issue_7_0] 101465 D - 0xfffff80808417800 [zio_free_issue_7_1] 101467 D - 0xfffff80808417800 [zio_free_issue_7_2] 101468 D - 0xfffff80808417800 [zio_free_issue_7_3] 101470 D - 0xfffff80808417800 [zio_free_issue_7_4] 101471 D - 0xfffff80808417800 [zio_free_issue_7_5] 101472 D - 0xfffff80808417800 [zio_free_issue_7_6] 101475 D - 0xfffff80808417800 [zio_free_issue_7_7] 101479 D - 0xfffff80808417800 [zio_free_issue_7_8] 101480 D - 0xfffff80808417800 [zio_free_issue_7_9] 101483 D - 0xfffff80808417800 [zio_free_issue_7_10] 101484 D - 0xfffff80808417800 [zio_free_issue_7_11] 101486 D - 0xfffff805f1e98300 [zio_free_intr] 101489 D - 0xfffff804b891c700 [zio_claim_issue] 101491 D - 0xfffff804b891c600 [zio_claim_intr] 101492 D - 0xfffff805199f6800 [zio_ioctl_issue] 101495 D - 0xfffff808083bfb00 [zio_ioctl_intr] 101565 D tx->tx_ 0xfffff80454bf23c8 [txg_thread_enter] 101566 D tx->tx_ 0xfffff80800462ab8 [txg_thread_enter] 19688 0 0 0 DL mdwait 0xfffff804e0a7e000 [md11] 19681 0 0 0 DL mdwait 0xfffff804e0c4a000 [md10] 19655 0 0 0 DL (threaded) [zfskern] 102267 D t->zthr 0xfffff8004418a948 [solthread 0xfffffff] 100932 D t->zthr 0xfffff803b26a0c48 [solthread 0xfffffff] 100934 D arc_dnl 0xffffffff8273e848 [arc_dnlc_evicts_thr] 100935 D - 0xfffff80205a0c800 [dbu_evict] 100936 D dbuf_ev 0xffffffff82741fc8 [dbuf_evict_thread] 100940 D - 0xfffff80808418a00 [z_vdev_file_0] 100941 D - 0xfffff80808418a00 [z_vdev_file_1] 100945 D - 0xfffff80808418a00 [z_vdev_file_2] 100946 D - 0xfffff80808418a00 [z_vdev_file_3] 100950 D - 0xfffff80808418a00 [z_vdev_file_4] 100953 D - 0xfffff80808418a00 [z_vdev_file_5] 100957 D - 0xfffff80808418a00 [z_vdev_file_6] 100958 D - 0xfffff80808418a00 [z_vdev_file_7] 100961 D - 0xfffff80808418a00 [z_vdev_file_8] 100965 D - 0xfffff80808418a00 [z_vdev_file_9] 100966 D - 0xfffff80808418a00 [z_vdev_file_10] 100970 D - 0xfffff80808418a00 [z_vdev_file_11] 100979 D - 0xfffff80808418a00 [z_vdev_file_12] 100980 D - 0xfffff80808418a00 [z_vdev_file_13] 100981 D - 0xfffff80808418a00 [z_vdev_file_14] 100983 D - 0xfffff80808418a00 [z_vdev_file_15] 100986 D l2arc_f 0xffffffff8273e9a0 [l2arc_feed_thread] 100993 D - 0xfffff80685ceeb00 [zfsvfs] 101500 D spa->sp 0xfffffe00d9fe6d28 [trim stress2_tank] 101502 D - 0xfffff8064008d400 [metaslab_group_task] 101503 D - 0xfffff8064008d400 [metaslab_group_task] 101507 D - 0xfffff8064008d400 [metaslab_group_task] 101508 D - 0xfffff8064008d400 [metaslab_group_task] 101509 D - 0xfffff8064008d400 [metaslab_group_task] 101510 D - 0xfffff8064008d400 [metaslab_group_task] 101514 D - 0xfffff808083bec00 [dp_sync_taskq_0] 101519 D - 0xfffff808083bec00 [dp_sync_taskq_1] 101520 D - 0xfffff808083bec00 [dp_sync_taskq_2] 101524 D - 0xfffff808083bec00 [dp_sync_taskq_3] 101525 D - 0xfffff808083bec00 [dp_sync_taskq_4] 101529 D - 0xfffff808083bec00 [dp_sync_taskq_5] 101533 D - 0xfffff808083bec00 [dp_sync_taskq_6] 101534 D - 0xfffff808083bec00 [dp_sync_taskq_7] 101538 D - 0xfffff808083bec00 [dp_sync_taskq_8] 101541 D - 0xfffff80808392a00 [dp_zil_clean_taskq_] 101542 D - 0xfffff80808392a00 [dp_zil_clean_taskq_] 101543 D - 0xfffff80808392a00 [dp_zil_clean_taskq_] 101544 D - 0xfffff80808392a00 [dp_zil_clean_taskq_] 101545 D - 0xfffff80808392a00 [dp_zil_clean_taskq_] 101546 D - 0xfffff80808392a00 [dp_zil_clean_taskq_] 101552 D - 0xfffff80808392a00 [dp_zil_clean_taskq_] 101554 D - 0xfffff80808392a00 [dp_zil_clean_taskq_] 101557 D - 0xfffff80808392a00 [dp_zil_clean_taskq_] 101558 D - 0xfffff80808392a00 [dp_zil_clean_taskq_] 101560 D - 0xfffff80808392a00 [dp_zil_clean_taskq_] 101561 D - 0xfffff80808392a00 [dp_zil_clean_taskq_] 101562 D - 0xfffff807a66f4800 [zfs_vn_rele_taskq] 101570 S mmp->mm 0xfffffe00d9fe6f00 [mmp_thread_enter] 101571 D t->zthr 0xfffff808088ac748 [solthread 0xfffffff] 101574 D t->zthr 0xfffff808088ac4c8 [solthread 0xfffffff] 19629 19627 2490 0 SW+ wait 0xfffff80109e04000 sh 19628 2490 2490 0 S+ piperd 0xfffff807080c5be0 tee 19627 2490 2490 0 SW+ wait 0xfffff801a79be530 sh 124 99828 124 0 S+ select 0xfffff800303878c0 bash 99828 99824 99828 0 SW+ pause 0xfffff8053f1d40a8 csh 99824 61272 99824 2006 SW+ wait 0xfffff80174cfd000 su 61272 61259 61272 2006 SWs+ wait 0xfffff80030ca5530 bash 61259 61026 61026 2006 S select 0xfffff800308ec3c0 sshd 61026 2308 61026 0 Ss select 0xfffff8031802dc40 sshd 3000 2930 2930 0 S piperd 0xfffff80030ac65f0 sh 2998 2930 2930 0 S piperd 0xfffff80030d7f8e8 awk 2997 2930 2930 0 S wait 0xfffff80030d7e530 sh 2930 1 2930 0 SWs wait 0xfffff80030f02000 sh 2490 1 2490 0 SW+ wait 0xfffff80030e92530 sh 2388 2387 2388 0 S+ ttyin 0xfffff801c098f8b0 csh 2387 2382 2387 2006 SW+ wait 0xfffff80030d85000 su 2382 2381 2382 2006 SWs+ wait 0xfffff804f074da60 bash 2381 2378 2378 2006 S select 0xfffff800306cb240 sshd 2378 2308 2378 0 Ss select 0xfffff8003061db40 sshd 2372 2367 2372 0 S+ ttyin 0xfffff800054310b0 csh 2367 1 2367 0 SWs+ wait 0xfffff800306a9000 login 2366 1 2366 0 Ss+ ttyin 0xfffff800054314b0 getty 2365 1 2365 0 Ss+ ttyin 0xfffff80003bad0b0 getty 2364 1 2364 0 Ss+ ttyin 0xfffff80003bad4b0 getty 2363 1 2363 0 Ss+ ttyin 0xfffff80003bad8b0 getty 2362 1 2362 0 Ss+ ttyin 0xfffff80003badcb0 getty 2361 1 2361 0 Ss+ ttyin 0xfffff80003bac0b0 getty 2360 1 2360 0 Ss+ ttyin 0xfffff80003bac4b0 getty 2359 1 2359 0 Ss+ ttyin 0xfffff80003bac8b0 getty 2358 1 2358 0 Ss+ ttyin 0xfffff80003baccb0 getty 2312 1 2312 0 Ss nanslp 0xffffffff81c80275 cron 2308 1 2308 0 Ss select 0xfffff80030742c40 sshd 2280 1 2280 0 Ss select 0xfffff80003a34bc0 powerd 2157 1 2157 0 Ss nanslp 0xffffffff81c80279 watchdogd 2078 1 2078 0 Ss select 0xfffff800306cb440 rpcbind 2068 1 2068 0 Ss select 0xfffff80044875040 syslogd 1992 1 1992 65 Ss select 0xfffff8003061de40 dhclient 1928 1 1928 0 Ss select 0xfffff800309b4740 dhclient 1925 1 1925 0 Ss select 0xfffff800309b47c0 dhclient 34 0 0 0 DL vlruwt 0xfffff800306a9530 [vnlru] 33 0 0 0 DL syncer 0xffffffff81d56720 [syncer] 32 0 0 0 DL (threaded) [bufdaemon] 100130 D qsleep 0xffffffff81d55bd8 [bufdaemon] 100131 D - 0xffffffff8180a980 [bufspacedaemon-0] 100132 D - 0xffffffff81812c40 [bufspacedaemon-1] 100133 D - 0xffffffff8181af00 [bufspacedaemon-2] 100134 D - 0xffffffff818231c0 [bufspacedaemon-3] 100135 D - 0xffffffff8182b480 [bufspacedaemon-4] 100136 D - 0xffffffff81833740 [bufspacedaemon-5] 100138 D - 0xffffffff8183ba00 [bufspacedaemon-6] 100152 D sdflush 0xfffff80030a2fce8 [/ worker] 100203 D sdflush 0xfffff80030caaee8 [/tmp worker] 31 0 0 0 DL psleep 0xffffffff81d715c8 [vmdaemon] 30 0 0 0 DL (threaded) [pagedaemon] 100128 D psleep 0xffffffff81d9e218 [dom0] 100137 D launds 0xffffffff81d9e224 [laundry: dom0] 100139 D umarcl 0xffffffff80f04b90 [uma] 29 0 0 0 DL idle 0xfffff80005005530 [enc_daemon1] 28 0 0 0 DL idle 0xfffff80030527a60 [enc_daemon0] 27 0 0 0 DL - 0xffffffff81adbf18 [rand_harvestq] 26 0 0 0 DL waiting 0xffffffff81de37e0 [sctp_iterator] 25 0 0 0 DL - 0xffffffff81d555dc [soaiod4] 24 0 0 0 DL - 0xffffffff81d555dc [soaiod3] 23 0 0 0 DL - 0xffffffff81d555dc [soaiod2] 22 0 0 0 DL - 0xffffffff81d555dc [soaiod1] 21 0 0 0 DL (threaded) [usb] 100096 D - 0xfffffe00d1584460 [usbus0] 100097 D - 0xfffffe00d15844b8 [usbus0] 100098 D - 0xfffffe00d1584510 [usbus0] 100099 D - 0xfffffe00d1584568 [usbus0] 100100 D - 0xfffffe00d15845c0 [usbus0] 100102 D - 0xfffffe0037b56d10 [usbus1] 100103 D - 0xfffffe0037b56d68 [usbus1] 100104 D - 0xfffffe0037b56dc0 [usbus1] 100105 D - 0xfffffe0037b56e18 [usbus1] 100106 D - 0xfffffe0037b56e70 [usbus1] 100107 D - 0xfffffe00d16a7d10 [usbus2] 100108 D - 0xfffffe00d16a7d68 [usbus2] 100109 D - 0xfffffe00d16a7dc0 [usbus2] 100110 D - 0xfffffe00d16a7e18 [usbus2] 100111 D - 0xfffffe00d16a7e70 [usbus2] 20 0 0 0 DL (threaded) [cam] 100091 D - 0xffffffff819b72c0 [doneq0] 100092 D - 0xffffffff819b7340 [doneq1] 100093 D - 0xffffffff819b73c0 [doneq2] 100125 D - 0xffffffff819b7188 [scanner] 19 0 0 0 DL crypto_ 0xfffff80003bb2c50 [crypto returns 11] 18 0 0 0 DL crypto_ 0xfffff80003bb2bf0 [crypto returns 10] 17 0 0 0 DL crypto_ 0xfffff80003bb2b90 [crypto returns 9] 16 0 0 0 DL crypto_ 0xfffff80003bb2b30 [crypto returns 8] 15 0 0 0 DL crypto_ 0xfffff80003bb2ad0 [crypto returns 7] 9 0 0 0 DL crypto_ 0xfffff80003bb2a70 [crypto returns 6] 8 0 0 0 DL crypto_ 0xfffff80003bb2a10 [crypto returns 5] 7 0 0 0 DL crypto_ 0xfffff80003bb29b0 [crypto returns 4] 6 0 0 0 DL crypto_ 0xfffff80003bb2950 [crypto returns 3] 5 0 0 0 DL crypto_ 0xfffff80003bb28f0 [crypto returns 2] 4 0 0 0 DL crypto_ 0xfffff80003bb2890 [crypto returns 1] 3 0 0 0 DL crypto_ 0xfffff80003bb2830 [crypto returns 0] 2 0 0 0 DL crypto_ 0xffffffff81d6b6a8 [crypto] 14 0 0 0 DL seqstat 0xfffff80003a85888 [sequencer 00] 13 0 0 0 DL (threaded) [geom] 100062 D - 0xffffffff81d9c848 [g_event] 100063 D - 0xffffffff81d9c858 [g_up] 100064 D - 0xffffffff81d9c850 [g_down] 12 0 0 0 WL (threaded) [intr] 100017 I [swi6: task queue] 100018 I [swi6: Giant taskq] 100020 I [swi5: fast taskq] 100047 I [swi3: vm] 100048 I [swi1: netisr 0] 100049 I [swi4: clock (0)] 100050 I [swi4: clock (1)] 100051 I [swi4: clock (2)] 100052 I [swi4: clock (3)] 100053 I [swi4: clock (4)] 100054 I [swi4: clock (5)] 100055 I [swi4: clock (6)] 100056 I [swi4: clock (7)] 100057 I [swi4: clock (8)] 100058 I [swi4: clock (9)] 100059 I [swi4: clock (10)] 100060 I [swi4: clock (11)] 100094 I [irq56: ahci0] 100095 I [irq19: xhci0] 100101 I [irq18: ehci0 ehci1] 100112 I [irq71: ahci1] 100113 I [swi0: uart uart] 100176 I [irq72: ioat0] 100177 I [irq73: ioat1] 100178 I [irq74: ioat2] 100179 I [irq75: ioat3] 100180 I [irq76: ioat4] 100181 I [irq77: ioat5] 100182 I [irq78: ioat6] 100183 I [irq79: ioat7] 11 0 0 0 RL (threaded) [idle] 100003 CanRun [idle: cpu0] 100004 CanRun [idle: cpu1] 100005 CanRun [idle: cpu2] 100006 CanRun [idle: cpu3] 100007 CanRun [idle: cpu4] 100008 CanRun [idle: cpu5] 100009 CanRun [idle: cpu6] 100010 Run CPU 7 [idle: cpu7] 100011 Run CPU 8 [idle: cpu8] 100012 CanRun [idle: cpu9] 100013 CanRun [idle: cpu10] 100014 CanRun [idle: cpu11] 1 0 1 0 SLs wait 0xfffff80003850530 [init] 10 0 0 0 DL audit_w 0xffffffff81de4470 [audit] 0 0 0 0 RLs (threaded) [kernel] 100000 D swapin 0xffffffff81d8b138 [swapper] 100015 D - 0xfffff800038cb000 [kqueue_ctx taskq] 100016 D - 0xfffff800038cae00 [aiod_kick taskq] 100019 D - 0xfffff800038cab00 [thread taskq] 100021 D - 0xfffff80003886000 [config_0] 100022 D - 0xfffff80003885e00 [if_config_tqg_0] 100023 D - 0xfffff80003885d00 [softirq_0] 100024 D - 0xfffff80003885c00 [softirq_1] 100025 D - 0xfffff80003885b00 [softirq_2] 100026 D - 0xfffff80003885a00 [softirq_3] 100027 D - 0xfffff80003885900 [softirq_4] 100028 D - 0xfffff80003885800 [softirq_5] 100029 D - 0xfffff80003885700 [softirq_6] 100030 D - 0xfffff80003885600 [softirq_7] 100031 D - 0xfffff80003885500 [softirq_8] 100032 D - 0xfffff80003885400 [softirq_9] 100033 D - 0xfffff80003885300 [softirq_10] 100034 D - 0xfffff80003885200 [softirq_11] 100035 RunQ [if_io_tqg_0] 100036 D - 0xfffff80003885000 [if_io_tqg_1] 100037 D - 0xfffff80003884e00 [if_io_tqg_2] 100038 D - 0xfffff80003884d00 [if_io_tqg_3] 100039 D - 0xfffff80003884c00 [if_io_tqg_4] 100040 D - 0xfffff80003884b00 [if_io_tqg_5] 100041 D - 0xfffff80003884a00 [if_io_tqg_6] 100042 D - 0xfffff80003884900 [if_io_tqg_7] 100043 D - 0xfffff80003884800 [if_io_tqg_8] 100044 D - 0xfffff80003884700 [if_io_tqg_9] 100045 D - 0xfffff80003884600 [if_io_tqg_10] 100046 D - 0xfffff80003884500 [if_io_tqg_11] 100061 D - 0xfffff80003b0c100 [firmware taskq] 100066 D - 0xfffff80003b0c000 [crypto_0] 100067 D - 0xfffff80003b0c000 [crypto_1] 100068 D - 0xfffff80003b0c000 [crypto_2] 100069 D - 0xfffff80003b0c000 [crypto_3] 100070 D - 0xfffff80003b0c000 [crypto_4] 100071 D - 0xfffff80003b0c000 [crypto_5] 100072 D - 0xfffff80003b0c000 [crypto_6] 100073 D - 0xfffff80003b0c000 [crypto_7] 100074 D - 0xfffff80003b0c000 [crypto_8] 100075 D - 0xfffff80003b0c000 [crypto_9] 100076 D - 0xfffff80003b0c000 [crypto_10] 100077 D - 0xfffff80003b0c000 [crypto_11] 100114 D - 0xfffff8000546b200 [mca taskq] 100121 D - 0xfffff80003b0dd00 [acpi_task_0] 100122 D - 0xfffff80003b0dd00 [acpi_task_1] 100123 D - 0xfffff80003b0dd00 [acpi_task_2] 100124 D - 0xfffff80003b0de00 [CAM taskq] 100821 D - 0xfffff80685cefd00 [system_taskq_0] 100843 D - 0xfffff80685cefd00 [system_taskq_1] 100847 D - 0xfffff80685cefd00 [system_taskq_2] 100848 D - 0xfffff80685cefd00 [system_taskq_3] 100849 D - 0xfffff80685cefd00 [system_taskq_4] 100850 D - 0xfffff80685cefd00 [system_taskq_5] 100862 D - 0xfffff80685cefd00 [system_taskq_6] 100863 D - 0xfffff80685cefd00 [system_taskq_7] 100867 D - 0xfffff80685cefd00 [system_taskq_8] 100869 D - 0xfffff80685cefd00 [system_taskq_9] 100871 D - 0xfffff80685cefd00 [system_taskq_10] 100878 D - 0xfffff80685cefd00 [system_taskq_11] 100879 D - 0xfffff80041008d00 [arc_prune_0] 100884 D - 0xfffff80041008d00 [arc_prune_1] 100885 D - 0xfffff80041008d00 [arc_prune_2] 100890 D - 0xfffff80041008d00 [arc_prune_3] 100893 D - 0xfffff80041008d00 [arc_prune_4] 100896 D - 0xfffff80041008d00 [arc_prune_5] 100897 D - 0xfffff80041008d00 [arc_prune_6] 100898 D - 0xfffff80041008d00 [arc_prune_7] 100902 D - 0xfffff80041008d00 [arc_prune_8] 100914 D - 0xfffff80041008d00 [arc_prune_9] 100920 D - 0xfffff80041008d00 [arc_prune_10] 100921 D - 0xfffff80041008d00 [arc_prune_11] 19844 19838 2490 0 Z+ badcode 19845 19838 2490 0 Z+ tcp 19846 19838 2490 0 Z+ symlink 19848 19838 2490 0 Z+ shm 19849 19838 2490 0 Z+ udp 19850 19838 2490 0 Z+ socket 19851 19838 2490 0 Z+ rw 19852 19838 2490 0 Z+ mmap 19853 19838 2490 0 Z+ fts 19856 19838 2490 0 Z+ creat 19858 19838 2490 0 Z+ rename 19860 19838 2490 0 Z+ swap 19861 19838 2490 0 Z+ mkfifo 19863 19838 2490 0 Z+ pty 19865 19838 2490 0 Z+ thr1 db:0:pho> allt Tracing command sleep pid 14234 tid 100312 td 0xfffff80601c2b000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3abd830 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3abd870 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e3abd8b0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e3abd910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00e3abd950 _sleep() at _sleep+0x258/frame 0xfffffe00e3abd9f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe00e3abda80 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe00e3abdac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e3abdbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e3abdbf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003a8a7a, rsp = 0x7fffffffe6a8, rbp = 0x7fffffffe6f0 --- Tracing command cat pid 30742 tid 100190 td 0xfffff801c07c86e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2ad87d0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2ad8810 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2ad8850 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e2ad88b0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e2ad88e0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e2ad8980 pipe_read() at pipe_read+0x428/frame 0xfffffe00e2ad89e0 dofileread() at dofileread+0x81/frame 0xfffffe00e2ad8a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e2ad8a70 sys_read() at sys_read+0x83/frame 0xfffffe00e2ad8ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2ad8bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2ad8bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800419b1a, rsp = 0x7fffffffe708, rbp = 0x7fffffffeb90 --- Tracing command sh pid 30740 tid 101842 td 0xfffff80211032000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00eb340690 mi_switch() at mi_switch+0x182/frame 0xfffffe00eb3406d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00eb340710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00eb340770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00eb3407a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00eb340840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00eb3408d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00eb340ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb340bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb340bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80040791a, rsp = 0x7fffffffe448, rbp = 0x7fffffffe500 --- Tracing command find pid 30739 tid 102340 td 0xfffff8073be4a6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e85011d0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e8501210 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e8501250 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e8501280 sleeplk() at sleeplk+0x107/frame 0xfffffe00e85012e0 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00e8501390 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e85013c0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e85013e0 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e8501440 zfs_lookup() at zfs_lookup+0x669/frame 0xfffffe00e8501530 zfs_freebsd_lookup() at zfs_freebsd_lookup+0x91/frame 0xfffffe00e8501670 VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xa8/frame 0xfffffe00e8501690 vfs_cache_lookup() at vfs_cache_lookup+0xd2/frame 0xfffffe00e85016f0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xa8/frame 0xfffffe00e8501710 lookup() at lookup+0x611/frame 0xfffffe00e85017b0 namei() at namei+0x4f1/frame 0xfffffe00e85018a0 kern_statat() at kern_statat+0x7f/frame 0xfffffe00e85019c0 sys_fstatat() at sys_fstatat+0x2f/frame 0xfffffe00e8501ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e8501bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e8501bf0 --- syscall (552, FreeBSD ELF64, sys_fstatat), rip = 0x800420d8a, rsp = 0x7fffffffe6b8, rbp = 0x7fffffffe770 --- Tracing command sh pid 30735 tid 102688 td 0xfffff80394ba7000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00efbd5690 mi_switch() at mi_switch+0x182/frame 0xfffffe00efbd56d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00efbd5710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00efbd5770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00efbd57a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00efbd5840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00efbd58d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00efbd5ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00efbd5bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00efbd5bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80040791a, rsp = 0x7fffffffe8d8, rbp = 0x7fffffffe990 --- Tracing command mail pid 30734 tid 100695 td 0xfffff8078047b000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00ec05e7d0 mi_switch() at mi_switch+0x182/frame 0xfffffe00ec05e810 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00ec05e850 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00ec05e8b0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ec05e8e0 _sleep() at _sleep+0x2c3/frame 0xfffffe00ec05e980 pipe_read() at pipe_read+0x428/frame 0xfffffe00ec05e9e0 dofileread() at dofileread+0x81/frame 0xfffffe00ec05ea30 kern_readv() at kern_readv+0x49/frame 0xfffffe00ec05ea70 sys_read() at sys_read+0x83/frame 0xfffffe00ec05eac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00ec05ebf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ec05ebf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80042fb1a, rsp = 0x7fffffffdcf8, rbp = 0x7fffffffdd10 --- Tracing command sh pid 30733 tid 100610 td 0xfffff801ac11d6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00ee2b8690 mi_switch() at mi_switch+0x182/frame 0xfffffe00ee2b86d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00ee2b8710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00ee2b8770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ee2b87a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00ee2b8840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00ee2b88d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00ee2b8ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00ee2b8bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ee2b8bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80040791a, rsp = 0x7fffffffe528, rbp = 0x7fffffffe5e0 --- Tracing command sh pid 30726 tid 100473 td 0xfffff803ba3cf6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e4e09690 mi_switch() at mi_switch+0x182/frame 0xfffffe00e4e096d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e4e09710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e4e09770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e4e097a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e4e09840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00e4e098d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00e4e09ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e4e09bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e4e09bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80040791a, rsp = 0x7fffffffe968, rbp = 0x7fffffffea20 --- Tracing command lockf pid 30725 tid 102266 td 0xfffff80190cfb000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e838f690 mi_switch() at mi_switch+0x182/frame 0xfffffe00e838f6d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e838f710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e838f770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e838f7a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e838f840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00e838f8d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00e838fac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e838fbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e838fbf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003a891a, rsp = 0x7fffffffeae8, rbp = 0x7fffffffeb50 --- Tracing command sh pid 30723 tid 102417 td 0xfffff801e4159000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e7686690 mi_switch() at mi_switch+0x182/frame 0xfffffe00e76866d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e7686710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e7686770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e76867a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e7686840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00e76868d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00e7686ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e7686bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e7686bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80040791a, rsp = 0x7fffffffe658, rbp = 0x7fffffffe710 --- Tracing command sh pid 30722 tid 100778 td 0xfffff802ecadd000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00ec1e5690 mi_switch() at mi_switch+0x182/frame 0xfffffe00ec1e56d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00ec1e5710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00ec1e5770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ec1e57a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00ec1e5840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00ec1e58d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00ec1e5ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00ec1e5bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ec1e5bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80040791a, rsp = 0x7fffffffe798, rbp = 0x7fffffffe850 --- Tracing command mail pid 27785 tid 102268 td 0xfffff803dc06f000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e83997d0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e8399810 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e8399850 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e83998b0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e83998e0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e8399980 pipe_read() at pipe_read+0x428/frame 0xfffffe00e83999e0 dofileread() at dofileread+0x81/frame 0xfffffe00e8399a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e8399a70 sys_read() at sys_read+0x83/frame 0xfffffe00e8399ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e8399bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e8399bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80042fb1a, rsp = 0x7fffffffdd28, rbp = 0x7fffffffdd40 --- Tracing command sh pid 27784 tid 102829 td 0xfffff801104e4000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00efe9a690 mi_switch() at mi_switch+0x182/frame 0xfffffe00efe9a6d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00efe9a710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00efe9a770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00efe9a7a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00efe9a840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00efe9a8d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00efe9aac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00efe9abf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00efe9abf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80040791a, rsp = 0x7fffffffe5b8, rbp = 0x7fffffffe670 --- Tracing command sh pid 27776 tid 102234 td 0xfffff801af2ca000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e82ef690 mi_switch() at mi_switch+0x182/frame 0xfffffe00e82ef6d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e82ef710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e82ef770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e82ef7a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e82ef840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00e82ef8d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00e82efac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e82efbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e82efbf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80040791a, rsp = 0x7fffffffe9f8, rbp = 0x7fffffffeab0 --- Tracing command lockf pid 27775 tid 100555 td 0xfffff807b32e16e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e4fa3690 mi_switch() at mi_switch+0x182/frame 0xfffffe00e4fa36d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e4fa3710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e4fa3770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e4fa37a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e4fa3840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00e4fa38d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00e4fa3ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e4fa3bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e4fa3bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003a891a, rsp = 0x7fffffffeb78, rbp = 0x7fffffffebe0 --- Tracing command sh pid 27773 tid 100574 td 0xfffff801ac8d96e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00ee204690 mi_switch() at mi_switch+0x182/frame 0xfffffe00ee2046d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00ee204710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00ee204770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ee2047a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00ee204840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00ee2048d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00ee204ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00ee204bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ee204bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80040791a, rsp = 0x7fffffffe708, rbp = 0x7fffffffe7c0 --- Tracing command cron pid 27771 tid 102531 td 0xfffff805b0b77000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e78c07d0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e78c0810 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e78c0850 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e78c08b0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e78c08e0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e78c0980 pipe_read() at pipe_read+0x428/frame 0xfffffe00e78c09e0 dofileread() at dofileread+0x81/frame 0xfffffe00e78c0a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e78c0a70 sys_read() at sys_read+0x83/frame 0xfffffe00e78c0ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e78c0bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e78c0bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800447b1a, rsp = 0x7fffffffe498, rbp = 0x7fffffffe4b0 --- Tracing command sh pid 20682 tid 101994 td 0xfffff80796c776e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00ef63b4a0 mi_switch() at mi_switch+0x182/frame 0xfffffe00ef63b4e0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00ef63b520 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00ef63b550 sleeplk() at sleeplk+0x107/frame 0xfffffe00ef63b5b0 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00ef63b660 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00ef63b690 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00ef63b6b0 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00ef63b710 lookup() at lookup+0xe1/frame 0xfffffe00ef63b7b0 namei() at namei+0x4f1/frame 0xfffffe00ef63b8a0 kern_statat() at kern_statat+0x7f/frame 0xfffffe00ef63b9c0 sys_fstatat() at sys_fstatat+0x2f/frame 0xfffffe00ef63bac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00ef63bbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ef63bbf0 --- syscall (552, FreeBSD ELF64, sys_fstatat), rip = 0x800473d8a, rsp = 0x7fffffffe268, rbp = 0x7fffffffe310 --- Tracing command sh pid 20486 tid 100145 td 0xfffff80003a4a6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e0b9f700 mi_switch() at mi_switch+0x182/frame 0xfffffe00e0b9f740 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e0b9f780 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e0b9f7b0 sleeplk() at sleeplk+0x107/frame 0xfffffe00e0b9f810 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00e0b9f8c0 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e0b9f8f0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e0b9f910 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e0b9f970 vn_vptocnp() at vn_vptocnp+0xb3/frame 0xfffffe00e0b9f9f0 vn_fullpath1() at vn_fullpath1+0xcf/frame 0xfffffe00e0b9fa60 kern___getcwd() at kern___getcwd+0xc3/frame 0xfffffe00e0b9fac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e0b9fbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e0b9fbf0 --- syscall (326, FreeBSD ELF64, sys___getcwd), rip = 0x8003854ba, rsp = 0x7fffffffe198, rbp = 0x7fffffffe2f0 --- Tracing command sh pid 20414 tid 100547 td 0xfffff801ac0cf6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e4f7b700 mi_switch() at mi_switch+0x182/frame 0xfffffe00e4f7b740 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e4f7b780 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e4f7b7b0 sleeplk() at sleeplk+0x107/frame 0xfffffe00e4f7b810 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00e4f7b8c0 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e4f7b8f0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e4f7b910 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e4f7b970 vn_vptocnp() at vn_vptocnp+0xb3/frame 0xfffffe00e4f7b9f0 vn_fullpath1() at vn_fullpath1+0xcf/frame 0xfffffe00e4f7ba60 kern___getcwd() at kern___getcwd+0xc3/frame 0xfffffe00e4f7bac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e4f7bbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e4f7bbf0 --- syscall (326, FreeBSD ELF64, sys___getcwd), rip = 0x8003854ba, rsp = 0x7fffffffe198, rbp = 0x7fffffffe2f0 --- Tracing command fstat pid 20407 tid 100339 td 0xfffff807b8696000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3b682c0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3b68300 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e3b68340 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e3b68370 sleeplk() at sleeplk+0x107/frame 0xfffffe00e3b683d0 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00e3b68480 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e3b684b0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e3b684d0 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e3b68530 vn_vptocnp() at vn_vptocnp+0xb3/frame 0xfffffe00e3b685b0 vn_fullpath1() at vn_fullpath1+0xcf/frame 0xfffffe00e3b68620 vn_fullpath() at vn_fullpath+0xb0/frame 0xfffffe00e3b68670 vn_fill_kinfo_vnode() at vn_fill_kinfo_vnode+0x52/frame 0xfffffe00e3b687a0 kern_proc_filedesc_out() at kern_proc_filedesc_out+0x506/frame 0xfffffe00e3b68820 sysctl_kern_proc_filedesc() at sysctl_kern_proc_filedesc+0x7d/frame 0xfffffe00e3b688a0 sysctl_root_handler_locked() at sysctl_root_handler_locked+0x7b/frame 0xfffffe00e3b688e0 sysctl_root() at sysctl_root+0x20a/frame 0xfffffe00e3b68960 userland_sysctl() at userland_sysctl+0x17b/frame 0xfffffe00e3b68a10 sys___sysctl() at sys___sysctl+0x5f/frame 0xfffffe00e3b68ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e3b68bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e3b68bf0 --- syscall (202, FreeBSD ELF64, sys___sysctl), rip = 0x8004274ea, rsp = 0x7fffffffc1b8, rbp = 0x7fffffffc1f0 --- Tracing command sh pid 20406 tid 101844 td 0xfffff802942e1000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00eb34a700 mi_switch() at mi_switch+0x182/frame 0xfffffe00eb34a740 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00eb34a780 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00eb34a7b0 sleeplk() at sleeplk+0x107/frame 0xfffffe00eb34a810 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00eb34a8c0 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00eb34a8f0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00eb34a910 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00eb34a970 vn_vptocnp() at vn_vptocnp+0xb3/frame 0xfffffe00eb34a9f0 vn_fullpath1() at vn_fullpath1+0xcf/frame 0xfffffe00eb34aa60 kern___getcwd() at kern___getcwd+0xc3/frame 0xfffffe00eb34aac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb34abf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb34abf0 --- syscall (326, FreeBSD ELF64, sys___getcwd), rip = 0x8003854ba, rsp = 0x7fffffffe198, rbp = 0x7fffffffe2f0 --- Tracing command sh pid 20380 tid 101960 td 0xfffff80109ecb6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00eb58e700 mi_switch() at mi_switch+0x182/frame 0xfffffe00eb58e740 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00eb58e780 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00eb58e7b0 sleeplk() at sleeplk+0x107/frame 0xfffffe00eb58e810 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00eb58e8c0 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00eb58e8f0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00eb58e910 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00eb58e970 vn_vptocnp() at vn_vptocnp+0xb3/frame 0xfffffe00eb58e9f0 vn_fullpath1() at vn_fullpath1+0xcf/frame 0xfffffe00eb58ea60 kern___getcwd() at kern___getcwd+0xc3/frame 0xfffffe00eb58eac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb58ebf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb58ebf0 --- syscall (326, FreeBSD ELF64, sys___getcwd), rip = 0x8003854ba, rsp = 0x7fffffffe198, rbp = 0x7fffffffe2f0 --- Tracing command fstat pid 20345 tid 102304 td 0xfffff805466bb000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e844d2c0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e844d300 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e844d340 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e844d370 sleeplk() at sleeplk+0x107/frame 0xfffffe00e844d3d0 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00e844d480 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e844d4b0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e844d4d0 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e844d530 vn_vptocnp() at vn_vptocnp+0xb3/frame 0xfffffe00e844d5b0 vn_fullpath1() at vn_fullpath1+0xcf/frame 0xfffffe00e844d620 vn_fullpath() at vn_fullpath+0xb0/frame 0xfffffe00e844d670 vn_fill_kinfo_vnode() at vn_fill_kinfo_vnode+0x52/frame 0xfffffe00e844d7a0 kern_proc_filedesc_out() at kern_proc_filedesc_out+0x506/frame 0xfffffe00e844d820 sysctl_kern_proc_filedesc() at sysctl_kern_proc_filedesc+0x7d/frame 0xfffffe00e844d8a0 sysctl_root_handler_locked() at sysctl_root_handler_locked+0x7b/frame 0xfffffe00e844d8e0 sysctl_root() at sysctl_root+0x20a/frame 0xfffffe00e844d960 userland_sysctl() at userland_sysctl+0x17b/frame 0xfffffe00e844da10 sys___sysctl() at sys___sysctl+0x5f/frame 0xfffffe00e844dac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e844dbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e844dbf0 --- syscall (202, FreeBSD ELF64, sys___sysctl), rip = 0x8004274ea, rsp = 0x7fffffffc1b8, rbp = 0x7fffffffc1f0 --- Tracing command fstat pid 20344 tid 100609 td 0xfffff801ac11c000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00ee2b32c0 mi_switch() at mi_switch+0x182/frame 0xfffffe00ee2b3300 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00ee2b3340 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00ee2b3370 sleeplk() at sleeplk+0x107/frame 0xfffffe00ee2b33d0 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00ee2b3480 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00ee2b34b0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00ee2b34d0 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00ee2b3530 vn_vptocnp() at vn_vptocnp+0xb3/frame 0xfffffe00ee2b35b0 vn_fullpath1() at vn_fullpath1+0xcf/frame 0xfffffe00ee2b3620 vn_fullpath() at vn_fullpath+0xb0/frame 0xfffffe00ee2b3670 vn_fill_kinfo_vnode() at vn_fill_kinfo_vnode+0x52/frame 0xfffffe00ee2b37a0 kern_proc_filedesc_out() at kern_proc_filedesc_out+0x506/frame 0xfffffe00ee2b3820 sysctl_kern_proc_filedesc() at sysctl_kern_proc_filedesc+0x7d/frame 0xfffffe00ee2b38a0 sysctl_root_handler_locked() at sysctl_root_handler_locked+0x7b/frame 0xfffffe00ee2b38e0 sysctl_root() at sysctl_root+0x20a/frame 0xfffffe00ee2b3960 userland_sysctl() at userland_sysctl+0x17b/frame 0xfffffe00ee2b3a10 sys___sysctl() at sys___sysctl+0x5f/frame 0xfffffe00ee2b3ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00ee2b3bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ee2b3bf0 --- syscall (202, FreeBSD ELF64, sys___sysctl), rip = 0x8004274ea, rsp = 0x7fffffffc1b8, rbp = 0x7fffffffc1f0 --- Tracing command fstat pid 20250 tid 100550 td 0xfffff801ac0cf000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e4f8a2c0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e4f8a300 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e4f8a340 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e4f8a370 sleeplk() at sleeplk+0x107/frame 0xfffffe00e4f8a3d0 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00e4f8a480 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e4f8a4b0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e4f8a4d0 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e4f8a530 vn_vptocnp() at vn_vptocnp+0xb3/frame 0xfffffe00e4f8a5b0 vn_fullpath1() at vn_fullpath1+0xcf/frame 0xfffffe00e4f8a620 vn_fullpath() at vn_fullpath+0xb0/frame 0xfffffe00e4f8a670 vn_fill_kinfo_vnode() at vn_fill_kinfo_vnode+0x52/frame 0xfffffe00e4f8a7a0 kern_proc_filedesc_out() at kern_proc_filedesc_out+0x506/frame 0xfffffe00e4f8a820 sysctl_kern_proc_filedesc() at sysctl_kern_proc_filedesc+0x7d/frame 0xfffffe00e4f8a8a0 sysctl_root_handler_locked() at sysctl_root_handler_locked+0x7b/frame 0xfffffe00e4f8a8e0 sysctl_root() at sysctl_root+0x20a/frame 0xfffffe00e4f8a960 userland_sysctl() at userland_sysctl+0x17b/frame 0xfffffe00e4f8aa10 sys___sysctl() at sys___sysctl+0x5f/frame 0xfffffe00e4f8aac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e4f8abf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e4f8abf0 --- syscall (202, FreeBSD ELF64, sys___sysctl), rip = 0x8004274ea, rsp = 0x7fffffffc1b8, rbp = 0x7fffffffc1f0 --- Tracing command fstat pid 20241 tid 102164 td 0xfffff808246af6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00ef98d2c0 mi_switch() at mi_switch+0x182/frame 0xfffffe00ef98d300 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00ef98d340 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00ef98d370 sleeplk() at sleeplk+0x107/frame 0xfffffe00ef98d3d0 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00ef98d480 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00ef98d4b0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00ef98d4d0 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00ef98d530 vn_vptocnp() at vn_vptocnp+0xb3/frame 0xfffffe00ef98d5b0 vn_fullpath1() at vn_fullpath1+0xcf/frame 0xfffffe00ef98d620 vn_fullpath() at vn_fullpath+0xb0/frame 0xfffffe00ef98d670 vn_fill_kinfo_vnode() at vn_fill_kinfo_vnode+0x52/frame 0xfffffe00ef98d7a0 kern_proc_filedesc_out() at kern_proc_filedesc_out+0x506/frame 0xfffffe00ef98d820 sysctl_kern_proc_filedesc() at sysctl_kern_proc_filedesc+0x7d/frame 0xfffffe00ef98d8a0 sysctl_root_handler_locked() at sysctl_root_handler_locked+0x7b/frame 0xfffffe00ef98d8e0 sysctl_root() at sysctl_root+0x20a/frame 0xfffffe00ef98d960 userland_sysctl() at userland_sysctl+0x17b/frame 0xfffffe00ef98da10 sys___sysctl() at sys___sysctl+0x5f/frame 0xfffffe00ef98dac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00ef98dbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ef98dbf0 --- syscall (202, FreeBSD ELF64, sys___sysctl), rip = 0x8004274ea, rsp = 0x7fffffffc1b8, rbp = 0x7fffffffc1f0 --- Tracing command fstat pid 20237 tid 100608 td 0xfffff801ac4586e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00ee2ae2c0 mi_switch() at mi_switch+0x182/frame 0xfffffe00ee2ae300 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00ee2ae340 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00ee2ae370 sleeplk() at sleeplk+0x107/frame 0xfffffe00ee2ae3d0 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00ee2ae480 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00ee2ae4b0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00ee2ae4d0 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00ee2ae530 vn_vptocnp() at vn_vptocnp+0xb3/frame 0xfffffe00ee2ae5b0 vn_fullpath1() at vn_fullpath1+0xcf/frame 0xfffffe00ee2ae620 vn_fullpath() at vn_fullpath+0xb0/frame 0xfffffe00ee2ae670 vn_fill_kinfo_vnode() at vn_fill_kinfo_vnode+0x52/frame 0xfffffe00ee2ae7a0 kern_proc_filedesc_out() at kern_proc_filedesc_out+0x506/frame 0xfffffe00ee2ae820 sysctl_kern_proc_filedesc() at sysctl_kern_proc_filedesc+0x7d/frame 0xfffffe00ee2ae8a0 sysctl_root_handler_locked() at sysctl_root_handler_locked+0x7b/frame 0xfffffe00ee2ae8e0 sysctl_root() at sysctl_root+0x20a/frame 0xfffffe00ee2ae960 userland_sysctl() at userland_sysctl+0x17b/frame 0xfffffe00ee2aea10 sys___sysctl() at sys___sysctl+0x5f/frame 0xfffffe00ee2aeac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00ee2aebf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ee2aebf0 --- syscall (202, FreeBSD ELF64, sys___sysctl), rip = 0x8004274ea, rsp = 0x7fffffffc1b8, rbp = 0x7fffffffc1f0 --- Tracing command fstat pid 20230 tid 102491 td 0xfffff8053f1986e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e77f8400 mi_switch() at mi_switch+0x182/frame 0xfffffe00e77f8440 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e77f8480 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e77f84b0 sleeplk() at sleeplk+0x107/frame 0xfffffe00e77f8510 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00e77f85c0 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e77f85f0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e77f8610 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e77f8670 vn_fill_kinfo_vnode() at vn_fill_kinfo_vnode+0xbd/frame 0xfffffe00e77f87a0 kern_proc_filedesc_out() at kern_proc_filedesc_out+0x506/frame 0xfffffe00e77f8820 sysctl_kern_proc_filedesc() at sysctl_kern_proc_filedesc+0x7d/frame 0xfffffe00e77f88a0 sysctl_root_handler_locked() at sysctl_root_handler_locked+0x7b/frame 0xfffffe00e77f88e0 sysctl_root() at sysctl_root+0x20a/frame 0xfffffe00e77f8960 userland_sysctl() at userland_sysctl+0x17b/frame 0xfffffe00e77f8a10 sys___sysctl() at sys___sysctl+0x5f/frame 0xfffffe00e77f8ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e77f8bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e77f8bf0 --- syscall (202, FreeBSD ELF64, sys___sysctl), rip = 0x8004274ea, rsp = 0x7fffffffc1b8, rbp = 0x7fffffffc1f0 --- Tracing command dirnprename pid 19977 tid 101893 td 0xfffff801e414f000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00eb43f440 mi_switch() at mi_switch+0x182/frame 0xfffffe00eb43f480 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00eb43f4c0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00eb43f4f0 sleeplk() at sleeplk+0x107/frame 0xfffffe00eb43f550 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00eb43f600 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00eb43f630 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00eb43f650 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00eb43f6b0 lookup() at lookup+0xe1/frame 0xfffffe00eb43f750 namei() at namei+0x4f1/frame 0xfffffe00eb43f840 kern_renameat() at kern_renameat+0x9d/frame 0xfffffe00eb43fac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb43fbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb43fbf0 --- syscall (128, FreeBSD ELF64, sys_rename), rip = 0x8002f654a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe3d0 --- Tracing command dirnprename pid 19976 tid 100407 td 0xfffff80434ba36e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3cbc440 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3cbc480 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e3cbc4c0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e3cbc4f0 sleeplk() at sleeplk+0x107/frame 0xfffffe00e3cbc550 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00e3cbc600 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e3cbc630 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e3cbc650 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e3cbc6b0 lookup() at lookup+0xe1/frame 0xfffffe00e3cbc750 namei() at namei+0x4f1/frame 0xfffffe00e3cbc840 kern_renameat() at kern_renameat+0x9d/frame 0xfffffe00e3cbcac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e3cbcbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e3cbcbf0 --- syscall (128, FreeBSD ELF64, sys_rename), rip = 0x8002f654a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe3d0 --- Tracing command dirnprename pid 19975 tid 102149 td 0xfffff805d84b4000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00ef9424a0 mi_switch() at mi_switch+0x182/frame 0xfffffe00ef9424e0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00ef942520 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00ef942550 sleeplk() at sleeplk+0x107/frame 0xfffffe00ef9425b0 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00ef942660 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00ef942690 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00ef9426b0 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00ef942710 lookup() at lookup+0xe1/frame 0xfffffe00ef9427b0 namei() at namei+0x4f1/frame 0xfffffe00ef9428a0 kern_mkdirat() at kern_mkdirat+0x92/frame 0xfffffe00ef942ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00ef942bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ef942bf0 --- syscall (136, FreeBSD ELF64, sys_mkdir), rip = 0x80042df3a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe3d0 --- Tracing command dirnprename pid 19974 tid 102190 td 0xfffff80278cfe000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e82134a0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e82134e0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e8213520 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e8213550 sleeplk() at sleeplk+0x107/frame 0xfffffe00e82135b0 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00e8213660 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e8213690 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e82136b0 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e8213710 lookup() at lookup+0xe1/frame 0xfffffe00e82137b0 namei() at namei+0x4f1/frame 0xfffffe00e82138a0 kern_mkdirat() at kern_mkdirat+0x92/frame 0xfffffe00e8213ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e8213bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e8213bf0 --- syscall (136, FreeBSD ELF64, sys_mkdir), rip = 0x80042df3a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe3d0 --- Tracing command dirnprename pid 19973 tid 100749 td 0xfffff8026d3e56e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00ec159440 mi_switch() at mi_switch+0x182/frame 0xfffffe00ec159480 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00ec1594c0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00ec1594f0 sleeplk() at sleeplk+0x107/frame 0xfffffe00ec159550 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00ec159600 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00ec159630 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00ec159650 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00ec1596b0 lookup() at lookup+0xe1/frame 0xfffffe00ec159750 namei() at namei+0x4f1/frame 0xfffffe00ec159840 kern_renameat() at kern_renameat+0x9d/frame 0xfffffe00ec159ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00ec159bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ec159bf0 --- syscall (128, FreeBSD ELF64, sys_rename), rip = 0x8002f654a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe3d0 --- Tracing command dirnprename pid 19972 tid 100185 td 0xfffff8004420a000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2abf4a0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2abf4e0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2abf520 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e2abf550 sleeplk() at sleeplk+0x107/frame 0xfffffe00e2abf5b0 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00e2abf660 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e2abf690 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e2abf6b0 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e2abf710 lookup() at lookup+0xe1/frame 0xfffffe00e2abf7b0 namei() at namei+0x4f1/frame 0xfffffe00e2abf8a0 kern_mkdirat() at kern_mkdirat+0x92/frame 0xfffffe00e2abfac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2abfbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2abfbf0 --- syscall (136, FreeBSD ELF64, sys_mkdir), rip = 0x80042df3a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe3d0 --- Tracing command dirnprename pid 19971 tid 100321 td 0xfffff80030ecc000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3b0e4a0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3b0e4e0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e3b0e520 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e3b0e550 sleeplk() at sleeplk+0x107/frame 0xfffffe00e3b0e5b0 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00e3b0e660 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e3b0e690 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e3b0e6b0 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e3b0e710 lookup() at lookup+0xe1/frame 0xfffffe00e3b0e7b0 namei() at namei+0x4f1/frame 0xfffffe00e3b0e8a0 kern_mkdirat() at kern_mkdirat+0x92/frame 0xfffffe00e3b0eac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e3b0ebf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e3b0ebf0 --- syscall (136, FreeBSD ELF64, sys_mkdir), rip = 0x80042df3a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe3d0 --- Tracing command dirnprename pid 19970 tid 102513 td 0xfffff8053f143000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e78664a0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e78664e0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e7866520 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e7866550 sleeplk() at sleeplk+0x107/frame 0xfffffe00e78665b0 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00e7866660 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e7866690 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e78666b0 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e7866710 lookup() at lookup+0xe1/frame 0xfffffe00e78667b0 namei() at namei+0x4f1/frame 0xfffffe00e78668a0 kern_mkdirat() at kern_mkdirat+0x92/frame 0xfffffe00e7866ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e7866bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e7866bf0 --- syscall (136, FreeBSD ELF64, sys_mkdir), rip = 0x80042df3a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe3d0 --- Tracing command dirnprename pid 19969 tid 102516 td 0xfffff8053f1426e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e7875440 mi_switch() at mi_switch+0x182/frame 0xfffffe00e7875480 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e78754c0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e78754f0 sleeplk() at sleeplk+0x107/frame 0xfffffe00e7875550 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00e7875600 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e7875630 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e7875650 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e78756b0 lookup() at lookup+0xe1/frame 0xfffffe00e7875750 namei() at namei+0x4f1/frame 0xfffffe00e7875840 kern_renameat() at kern_renameat+0x9d/frame 0xfffffe00e7875ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e7875bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e7875bf0 --- syscall (128, FreeBSD ELF64, sys_rename), rip = 0x8002f654a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe3d0 --- Tracing command dirnprename pid 19968 tid 102236 td 0xfffff80190cfe000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e82f9440 mi_switch() at mi_switch+0x182/frame 0xfffffe00e82f9480 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e82f94c0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e82f94f0 sleeplk() at sleeplk+0x107/frame 0xfffffe00e82f9550 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00e82f9600 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e82f9630 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e82f9650 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e82f96b0 lookup() at lookup+0xe1/frame 0xfffffe00e82f9750 namei() at namei+0x4f1/frame 0xfffffe00e82f9840 kern_renameat() at kern_renameat+0x9d/frame 0xfffffe00e82f9ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e82f9bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e82f9bf0 --- syscall (128, FreeBSD ELF64, sys_rename), rip = 0x8002f654a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe3d0 --- Tracing command dirnprename pid 19967 tid 100675 td 0xfffff806e40a7000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00ee3f83e0 mi_switch() at mi_switch+0x182/frame 0xfffffe00ee3f8420 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00ee3f8460 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00ee3f8490 sleeplk() at sleeplk+0x107/frame 0xfffffe00ee3f84f0 lockmgr_xlock_hard() at lockmgr_xlock_hard+0x225/frame 0xfffffe00ee3f85b0 lockmgr_upgrade() at lockmgr_upgrade+0x10f/frame 0xfffffe00ee3f8600 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00ee3f8630 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00ee3f8650 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00ee3f86b0 lookup() at lookup+0x51c/frame 0xfffffe00ee3f8750 namei() at namei+0x4f1/frame 0xfffffe00ee3f8840 kern_renameat() at kern_renameat+0x9d/frame 0xfffffe00ee3f8ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00ee3f8bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ee3f8bf0 --- syscall (128, FreeBSD ELF64, sys_rename), rip = 0x8002f654a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe3d0 --- Tracing command dirnprename pid 19966 tid 102298 td 0xfffff807633c3000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e842f4c0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e842f500 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e842f540 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e842f570 sleeplk() at sleeplk+0x107/frame 0xfffffe00e842f5d0 lockmgr_xlock_hard() at lockmgr_xlock_hard+0x225/frame 0xfffffe00e842f690 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e842f6c0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e842f6e0 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e842f740 zfs_freebsd_rename() at zfs_freebsd_rename+0x244/frame 0xfffffe00e842f820 VOP_RENAME_APV() at VOP_RENAME_APV+0x92/frame 0xfffffe00e842f840 kern_renameat() at kern_renameat+0x35a/frame 0xfffffe00e842fac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e842fbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e842fbf0 --- syscall (128, FreeBSD ELF64, sys_rename), rip = 0x8002f654a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe3d0 --- Tracing command dirnprename pid 19965 tid 100298 td 0xfffff8057d996000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2a71440 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2a71480 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2a714c0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e2a714f0 sleeplk() at sleeplk+0x107/frame 0xfffffe00e2a71550 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00e2a71600 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e2a71630 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e2a71650 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e2a716b0 lookup() at lookup+0xe1/frame 0xfffffe00e2a71750 namei() at namei+0x4f1/frame 0xfffffe00e2a71840 kern_renameat() at kern_renameat+0x183/frame 0xfffffe00e2a71ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2a71bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2a71bf0 --- syscall (128, FreeBSD ELF64, sys_rename), rip = 0x8002f654a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe3d0 --- Tracing command dirnprename pid 19964 tid 102371 td 0xfffff80299b446e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e859c440 mi_switch() at mi_switch+0x182/frame 0xfffffe00e859c480 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e859c4c0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e859c4f0 sleeplk() at sleeplk+0x107/frame 0xfffffe00e859c550 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00e859c600 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e859c630 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e859c650 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e859c6b0 lookup() at lookup+0xe1/frame 0xfffffe00e859c750 namei() at namei+0x4f1/frame 0xfffffe00e859c840 kern_renameat() at kern_renameat+0x9d/frame 0xfffffe00e859cac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e859cbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e859cbf0 --- syscall (128, FreeBSD ELF64, sys_rename), rip = 0x8002f654a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe3d0 --- Tracing command dirnprename pid 19963 tid 100614 td 0xfffff801ac17c6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00ee2cc440 mi_switch() at mi_switch+0x182/frame 0xfffffe00ee2cc480 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00ee2cc4c0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00ee2cc4f0 sleeplk() at sleeplk+0x107/frame 0xfffffe00ee2cc550 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00ee2cc600 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00ee2cc630 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00ee2cc650 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00ee2cc6b0 lookup() at lookup+0xe1/frame 0xfffffe00ee2cc750 namei() at namei+0x4f1/frame 0xfffffe00ee2cc840 kern_renameat() at kern_renameat+0x9d/frame 0xfffffe00ee2ccac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00ee2ccbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ee2ccbf0 --- syscall (128, FreeBSD ELF64, sys_rename), rip = 0x8002f654a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe3d0 --- Tracing command dirnprename pid 19962 tid 102150 td 0xfffff808246ad000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00ef947440 mi_switch() at mi_switch+0x182/frame 0xfffffe00ef947480 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00ef9474c0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00ef9474f0 sleeplk() at sleeplk+0x107/frame 0xfffffe00ef947550 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00ef947600 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00ef947630 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00ef947650 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00ef9476b0 lookup() at lookup+0xe1/frame 0xfffffe00ef947750 namei() at namei+0x4f1/frame 0xfffffe00ef947840 kern_renameat() at kern_renameat+0x9d/frame 0xfffffe00ef947ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00ef947bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ef947bf0 --- syscall (128, FreeBSD ELF64, sys_rename), rip = 0x8002f654a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe3d0 --- Tracing command dirnprename pid 19961 tid 102163 td 0xfffff805466d7000 (CPU 2) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e17bdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e17bdf0 trap() at trap+0x61/frame 0xfffffe003e17bf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e17bf20 --- trap 0x13, rip = 0xffffffff80c11a92, rsp = 0xfffffe00ef988320, rbp = 0xfffffe00ef988330 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00ef988330 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00ef9883f0 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00ef988430 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00ef988470 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00ef988550 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00ef988640 zap_create_norm_dnsize() at zap_create_norm_dnsize+0x26/frame 0xfffffe00ef988680 zfs_mknode() at zfs_mknode+0x1f0/frame 0xfffffe00ef9887c0 zfs_freebsd_mkdir() at zfs_freebsd_mkdir+0x48d/frame 0xfffffe00ef988870 VOP_MKDIR_APV() at VOP_MKDIR_APV+0xae/frame 0xfffffe00ef9888a0 kern_mkdirat() at kern_mkdirat+0x1b3/frame 0xfffffe00ef988ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00ef988bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ef988bf0 --- syscall (136, FreeBSD ELF64, sys_mkdir), rip = 0x80042df3a, rsp = 0x7fffffffe298, rbp = 0x7fffffffe3d0 --- Tracing command dirnprename pid 19960 tid 100268 td 0xfffff8057d8e4000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3a81690 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3a816d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e3a81710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e3a81770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e3a817a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e3a81840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00e3a818d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00e3a81ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e3a81bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e3a81bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c391a, rsp = 0x7fffffffe3f8, rbp = 0x7fffffffe430 --- Tracing command sysctl pid 19957 tid 100162 td 0xfffff80030746000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2a04690 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2a046d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2a04710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e2a04770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e2a047a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e2a04840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00e2a048d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00e2a04ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2a04bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2a04bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c191a, rsp = 0x7fffffffe308, rbp = 0x7fffffffe3c0 --- Tracing command sysctl pid 19956 tid 102765 td 0xfffff805d38d1000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00efd58690 mi_switch() at mi_switch+0x182/frame 0xfffffe00efd586d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00efd58710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00efd58770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00efd587a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00efd58840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00efd588d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00efd58ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00efd58bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00efd58bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c191a, rsp = 0x7fffffffe308, rbp = 0x7fffffffe3c0 --- Tracing command sysctl pid 19955 tid 102763 td 0xfffff805d38d16e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00efd4e690 mi_switch() at mi_switch+0x182/frame 0xfffffe00efd4e6d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00efd4e710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00efd4e770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00efd4e7a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00efd4e840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00efd4e8d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00efd4eac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00efd4ebf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00efd4ebf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c191a, rsp = 0x7fffffffe308, rbp = 0x7fffffffe3c0 --- Tracing command sysctl pid 19954 tid 102761 td 0xfffff805d38d0000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00efd44690 mi_switch() at mi_switch+0x182/frame 0xfffffe00efd446d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00efd44710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00efd44770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00efd447a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00efd44840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00efd448d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00efd44ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00efd44bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00efd44bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c191a, rsp = 0x7fffffffe308, rbp = 0x7fffffffe3c0 --- Tracing command sysctl pid 19953 tid 103055 td 0xfffff80550eb4000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f0308690 mi_switch() at mi_switch+0x182/frame 0xfffffe00f03086d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f0308710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00f0308770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00f03087a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00f0308840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00f03088d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00f0308ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00f0308bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00f0308bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c191a, rsp = 0x7fffffffe308, rbp = 0x7fffffffe3c0 --- Tracing command sysctl pid 19952 tid 100387 td 0xfffff8002d3eb000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3c58690 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3c586d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e3c58710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e3c58770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e3c587a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e3c58840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00e3c588d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00e3c58ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e3c58bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e3c58bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c191a, rsp = 0x7fffffffe308, rbp = 0x7fffffffe3c0 --- Tracing command sysctl pid 19951 tid 102902 td 0xfffff8039a4df000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f0009690 mi_switch() at mi_switch+0x182/frame 0xfffffe00f00096d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f0009710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00f0009770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00f00097a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00f0009840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00f00098d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00f0009ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00f0009bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00f0009bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c191a, rsp = 0x7fffffffe308, rbp = 0x7fffffffe3c0 --- Tracing command sysctl pid 19950 tid 100676 td 0xfffff8032c3916e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00ee3fd690 mi_switch() at mi_switch+0x182/frame 0xfffffe00ee3fd6d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00ee3fd710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00ee3fd770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ee3fd7a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00ee3fd840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00ee3fd8d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00ee3fdac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00ee3fdbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ee3fdbf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c191a, rsp = 0x7fffffffe308, rbp = 0x7fffffffe3c0 --- Tracing command sysctl pid 19949 tid 102119 td 0xfffff80455d066e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00ef8ac690 mi_switch() at mi_switch+0x182/frame 0xfffffe00ef8ac6d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00ef8ac710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00ef8ac770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ef8ac7a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00ef8ac840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00ef8ac8d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00ef8acac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00ef8acbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ef8acbf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c191a, rsp = 0x7fffffffe308, rbp = 0x7fffffffe3c0 --- Tracing command sysctl pid 19948 tid 102241 td 0xfffff808246cb6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e8312690 mi_switch() at mi_switch+0x182/frame 0xfffffe00e83126d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e8312710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e8312770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e83127a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e8312840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00e83128d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00e8312ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e8312bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e8312bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c191a, rsp = 0x7fffffffe308, rbp = 0x7fffffffe3c0 --- Tracing command sysctl pid 19947 tid 102937 td 0xfffff807d069d000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f00b8690 mi_switch() at mi_switch+0x182/frame 0xfffffe00f00b86d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f00b8710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00f00b8770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00f00b87a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00f00b8840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00f00b88d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00f00b8ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00f00b8bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00f00b8bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c191a, rsp = 0x7fffffffe308, rbp = 0x7fffffffe3c0 --- Tracing command sysctl pid 19946 tid 102259 td 0xfffff8053f191000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e836c690 mi_switch() at mi_switch+0x182/frame 0xfffffe00e836c6d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e836c710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e836c770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e836c7a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e836c840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00e836c8d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00e836cac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e836cbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e836cbf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c191a, rsp = 0x7fffffffe308, rbp = 0x7fffffffe3c0 --- Tracing command sysctl pid 19945 tid 102290 td 0xfffff802c304e000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e8407690 mi_switch() at mi_switch+0x182/frame 0xfffffe00e84076d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e8407710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e8407770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e84077a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e8407840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00e84078d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00e8407ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e8407bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e8407bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c191a, rsp = 0x7fffffffe408, rbp = 0x7fffffffe440 --- Tracing command dirrename pid 19944 tid 100789 td 0xfffff806e997f000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e4e72440 mi_switch() at mi_switch+0x182/frame 0xfffffe00e4e72480 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e4e724c0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e4e724f0 sleeplk() at sleeplk+0x107/frame 0xfffffe00e4e72550 lockmgr_xlock_hard() at lockmgr_xlock_hard+0x225/frame 0xfffffe00e4e72610 lockmgr_upgrade() at lockmgr_upgrade+0x10f/frame 0xfffffe00e4e72660 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e4e72690 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e4e726b0 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e4e72710 lookup() at lookup+0x51c/frame 0xfffffe00e4e727b0 namei() at namei+0x4f1/frame 0xfffffe00e4e728a0 kern_mkdirat() at kern_mkdirat+0x92/frame 0xfffffe00e4e72ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e4e72bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e4e72bf0 --- syscall (136, FreeBSD ELF64, sys_mkdir), rip = 0x80042df3a, rsp = 0x7fffffffe2a8, rbp = 0x7fffffffe3e0 --- Tracing command dirrename pid 19943 tid 102488 td 0xfffff802c9ea46e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e77e9440 mi_switch() at mi_switch+0x182/frame 0xfffffe00e77e9480 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e77e94c0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e77e94f0 sleeplk() at sleeplk+0x107/frame 0xfffffe00e77e9550 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00e77e9600 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e77e9630 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e77e9650 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e77e96b0 lookup() at lookup+0xe1/frame 0xfffffe00e77e9750 namei() at namei+0x4f1/frame 0xfffffe00e77e9840 kern_renameat() at kern_renameat+0x9d/frame 0xfffffe00e77e9ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e77e9bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e77e9bf0 --- syscall (128, FreeBSD ELF64, sys_rename), rip = 0x8002f654a, rsp = 0x7fffffffe2a8, rbp = 0x7fffffffe3e0 --- Tracing command dirrename pid 19942 tid 102590 td 0xfffff804f09c56e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e79e7690 mi_switch() at mi_switch+0x182/frame 0xfffffe00e79e76d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e79e7710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e79e7770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e79e77a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e79e7840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00e79e78d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00e79e7ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e79e7bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e79e7bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c391a, rsp = 0x7fffffffe408, rbp = 0x7fffffffe440 --- Tracing command mkdir pid 19909 tid 100144 td 0xfffff80030aa16e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e0b9a220 mi_switch() at mi_switch+0x182/frame 0xfffffe00e0b9a260 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e0b9a2a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e0b9a2d0 sleeplk() at sleeplk+0x107/frame 0xfffffe00e0b9a330 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00e0b9a3e0 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e0b9a410 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e0b9a430 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e0b9a490 zfs_dd_callback() at zfs_dd_callback+0x23/frame 0xfffffe00e0b9a4b0 vn_vget_ino_gen() at vn_vget_ino_gen+0x11f/frame 0xfffffe00e0b9a520 zfs_lookup() at zfs_lookup+0x750/frame 0xfffffe00e0b9a610 zfs_freebsd_lookup() at zfs_freebsd_lookup+0x91/frame 0xfffffe00e0b9a750 VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xa8/frame 0xfffffe00e0b9a770 vfs_cache_lookup() at vfs_cache_lookup+0xd2/frame 0xfffffe00e0b9a7d0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xa8/frame 0xfffffe00e0b9a7f0 lookup() at lookup+0x611/frame 0xfffffe00e0b9a890 namei() at namei+0x4f1/frame 0xfffffe00e0b9a980 kern_chdir() at kern_chdir+0x4d/frame 0xfffffe00e0b9aac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e0b9abf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e0b9abf0 --- syscall (12, FreeBSD ELF64, sys_chdir), rip = 0x8002f68ca, rsp = 0x7fffffffdb88, rbp = 0x7fffffffdfc0 --- Tracing command mkdir pid 19906 tid 102349 td 0xfffff805269fb6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e852e550 mi_switch() at mi_switch+0x182/frame 0xfffffe00e852e590 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e852e5d0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e852e600 sleeplk() at sleeplk+0x107/frame 0xfffffe00e852e660 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00e852e710 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e852e740 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e852e760 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e852e7c0 lookup() at lookup+0xe1/frame 0xfffffe00e852e860 namei() at namei+0x4f1/frame 0xfffffe00e852e950 kern_frmdirat() at kern_frmdirat+0xb5/frame 0xfffffe00e852eac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e852ebf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e852ebf0 --- syscall (137, FreeBSD ELF64, sys_rmdir), rip = 0x80032941a, rsp = 0x7fffffffdb88, rbp = 0x7fffffffdfc0 --- Tracing command mkdir pid 19904 tid 102732 td 0xfffff80190c4a6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00efcb3580 mi_switch() at mi_switch+0x182/frame 0xfffffe00efcb35c0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00efcb3600 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00efcb3630 sleeplk() at sleeplk+0x107/frame 0xfffffe00efcb3690 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00efcb3740 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00efcb3770 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00efcb3790 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00efcb37f0 lookup() at lookup+0xe1/frame 0xfffffe00efcb3890 namei() at namei+0x4f1/frame 0xfffffe00efcb3980 kern_chdir() at kern_chdir+0x4d/frame 0xfffffe00efcb3ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00efcb3bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00efcb3bf0 --- syscall (12, FreeBSD ELF64, sys_chdir), rip = 0x8002f68ca, rsp = 0x7fffffffdb98, rbp = 0x7fffffffdfc0 --- Tracing command mkdir pid 19902 tid 101957 td 0xfffff80109ee76e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00eb57f550 mi_switch() at mi_switch+0x182/frame 0xfffffe00eb57f590 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00eb57f5d0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00eb57f600 sleeplk() at sleeplk+0x107/frame 0xfffffe00eb57f660 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00eb57f710 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00eb57f740 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00eb57f760 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00eb57f7c0 lookup() at lookup+0xe1/frame 0xfffffe00eb57f860 namei() at namei+0x4f1/frame 0xfffffe00eb57f950 kern_frmdirat() at kern_frmdirat+0xb5/frame 0xfffffe00eb57fac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb57fbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb57fbf0 --- syscall (137, FreeBSD ELF64, sys_rmdir), rip = 0x80032941a, rsp = 0x7fffffffdb88, rbp = 0x7fffffffdfc0 --- Tracing command mkdir pid 19900 tid 101956 td 0xfffff80546619000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00eb57a4f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00eb57a530 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00eb57a570 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00eb57a5a0 sleeplk() at sleeplk+0x107/frame 0xfffffe00eb57a600 lockmgr_xlock_hard() at lockmgr_xlock_hard+0x225/frame 0xfffffe00eb57a6c0 lockmgr_upgrade() at lockmgr_upgrade+0x10f/frame 0xfffffe00eb57a710 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00eb57a740 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00eb57a760 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00eb57a7c0 lookup() at lookup+0x51c/frame 0xfffffe00eb57a860 namei() at namei+0x4f1/frame 0xfffffe00eb57a950 kern_frmdirat() at kern_frmdirat+0xb5/frame 0xfffffe00eb57aac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb57abf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb57abf0 --- syscall (137, FreeBSD ELF64, sys_rmdir), rip = 0x80032941a, rsp = 0x7fffffffdb88, rbp = 0x7fffffffdfc0 --- Tracing command mkdir pid 19899 tid 100554 td 0xfffff801b1bd36e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e4f9e220 mi_switch() at mi_switch+0x182/frame 0xfffffe00e4f9e260 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e4f9e2a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e4f9e2d0 sleeplk() at sleeplk+0x107/frame 0xfffffe00e4f9e330 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00e4f9e3e0 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e4f9e410 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e4f9e430 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e4f9e490 zfs_dd_callback() at zfs_dd_callback+0x23/frame 0xfffffe00e4f9e4b0 vn_vget_ino_gen() at vn_vget_ino_gen+0x11f/frame 0xfffffe00e4f9e520 zfs_lookup() at zfs_lookup+0x750/frame 0xfffffe00e4f9e610 zfs_freebsd_lookup() at zfs_freebsd_lookup+0x91/frame 0xfffffe00e4f9e750 VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xa8/frame 0xfffffe00e4f9e770 vfs_cache_lookup() at vfs_cache_lookup+0xd2/frame 0xfffffe00e4f9e7d0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xa8/frame 0xfffffe00e4f9e7f0 lookup() at lookup+0x611/frame 0xfffffe00e4f9e890 namei() at namei+0x4f1/frame 0xfffffe00e4f9e980 kern_chdir() at kern_chdir+0x4d/frame 0xfffffe00e4f9eac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e4f9ebf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e4f9ebf0 --- syscall (12, FreeBSD ELF64, sys_chdir), rip = 0x8002f68ca, rsp = 0x7fffffffdb88, rbp = 0x7fffffffdfc0 --- Tracing command mkdir pid 19897 tid 102065 td 0xfffff807989b6000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00ef79e580 mi_switch() at mi_switch+0x182/frame 0xfffffe00ef79e5c0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00ef79e600 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00ef79e630 sleeplk() at sleeplk+0x107/frame 0xfffffe00ef79e690 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00ef79e740 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00ef79e770 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00ef79e790 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00ef79e7f0 lookup() at lookup+0xe1/frame 0xfffffe00ef79e890 namei() at namei+0x4f1/frame 0xfffffe00ef79e980 kern_chdir() at kern_chdir+0x4d/frame 0xfffffe00ef79eac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00ef79ebf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ef79ebf0 --- syscall (12, FreeBSD ELF64, sys_chdir), rip = 0x8002f68ca, rsp = 0x7fffffffdb98, rbp = 0x7fffffffdfc0 --- Tracing command mkdir pid 19896 tid 100342 td 0xfffff8080baf7000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3b77580 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3b775c0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e3b77600 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e3b77630 sleeplk() at sleeplk+0x107/frame 0xfffffe00e3b77690 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00e3b77740 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e3b77770 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e3b77790 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e3b777f0 lookup() at lookup+0xe1/frame 0xfffffe00e3b77890 namei() at namei+0x4f1/frame 0xfffffe00e3b77980 kern_chdir() at kern_chdir+0x4d/frame 0xfffffe00e3b77ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e3b77bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e3b77bf0 --- syscall (12, FreeBSD ELF64, sys_chdir), rip = 0x8002f68ca, rsp = 0x7fffffffdb98, rbp = 0x7fffffffdfc0 --- Tracing command mkdir pid 19895 tid 102376 td 0xfffff803dc0bb000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e85b5220 mi_switch() at mi_switch+0x182/frame 0xfffffe00e85b5260 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e85b52a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e85b52d0 sleeplk() at sleeplk+0x107/frame 0xfffffe00e85b5330 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00e85b53e0 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e85b5410 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e85b5430 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e85b5490 zfs_dd_callback() at zfs_dd_callback+0x23/frame 0xfffffe00e85b54b0 vn_vget_ino_gen() at vn_vget_ino_gen+0x11f/frame 0xfffffe00e85b5520 zfs_lookup() at zfs_lookup+0x750/frame 0xfffffe00e85b5610 zfs_freebsd_lookup() at zfs_freebsd_lookup+0x91/frame 0xfffffe00e85b5750 VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xa8/frame 0xfffffe00e85b5770 vfs_cache_lookup() at vfs_cache_lookup+0xd2/frame 0xfffffe00e85b57d0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xa8/frame 0xfffffe00e85b57f0 lookup() at lookup+0x611/frame 0xfffffe00e85b5890 namei() at namei+0x4f1/frame 0xfffffe00e85b5980 kern_chdir() at kern_chdir+0x4d/frame 0xfffffe00e85b5ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e85b5bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e85b5bf0 --- syscall (12, FreeBSD ELF64, sys_chdir), rip = 0x8002f68ca, rsp = 0x7fffffffdb88, rbp = 0x7fffffffdfc0 --- Tracing command mkdir pid 19894 tid 102347 td 0xfffff8053f1566e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e8524220 mi_switch() at mi_switch+0x182/frame 0xfffffe00e8524260 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e85242a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e85242d0 sleeplk() at sleeplk+0x107/frame 0xfffffe00e8524330 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00e85243e0 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e8524410 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e8524430 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e8524490 zfs_dd_callback() at zfs_dd_callback+0x23/frame 0xfffffe00e85244b0 vn_vget_ino_gen() at vn_vget_ino_gen+0x11f/frame 0xfffffe00e8524520 zfs_lookup() at zfs_lookup+0x750/frame 0xfffffe00e8524610 zfs_freebsd_lookup() at zfs_freebsd_lookup+0x91/frame 0xfffffe00e8524750 VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xa8/frame 0xfffffe00e8524770 vfs_cache_lookup() at vfs_cache_lookup+0xd2/frame 0xfffffe00e85247d0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xa8/frame 0xfffffe00e85247f0 lookup() at lookup+0x611/frame 0xfffffe00e8524890 namei() at namei+0x4f1/frame 0xfffffe00e8524980 kern_chdir() at kern_chdir+0x4d/frame 0xfffffe00e8524ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e8524bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e8524bf0 --- syscall (12, FreeBSD ELF64, sys_chdir), rip = 0x8002f68ca, rsp = 0x7fffffffdb88, rbp = 0x7fffffffdfc0 --- Tracing command mkdir pid 19893 tid 103019 td 0xfffff804fe2a9000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f02544a0 mi_switch() at mi_switch+0x182/frame 0xfffffe00f02544e0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f0254520 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f0254550 sleeplk() at sleeplk+0x107/frame 0xfffffe00f02545b0 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00f0254660 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00f0254690 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00f02546b0 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00f0254710 lookup() at lookup+0xe1/frame 0xfffffe00f02547b0 namei() at namei+0x4f1/frame 0xfffffe00f02548a0 kern_mkdirat() at kern_mkdirat+0x92/frame 0xfffffe00f0254ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00f0254bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00f0254bf0 --- syscall (136, FreeBSD ELF64, sys_mkdir), rip = 0x80042df3a, rsp = 0x7fffffffdb98, rbp = 0x7fffffffdfc0 --- Tracing command mkdir pid 19892 tid 101598 td 0xfffff801393556e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00ec12c4f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00ec12c530 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00ec12c570 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00ec12c5a0 sleeplk() at sleeplk+0x107/frame 0xfffffe00ec12c600 lockmgr_xlock_hard() at lockmgr_xlock_hard+0x225/frame 0xfffffe00ec12c6c0 lockmgr_upgrade() at lockmgr_upgrade+0x10f/frame 0xfffffe00ec12c710 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00ec12c740 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00ec12c760 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00ec12c7c0 lookup() at lookup+0x51c/frame 0xfffffe00ec12c860 namei() at namei+0x4f1/frame 0xfffffe00ec12c950 kern_frmdirat() at kern_frmdirat+0xb5/frame 0xfffffe00ec12cac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00ec12cbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ec12cbf0 --- syscall (137, FreeBSD ELF64, sys_rmdir), rip = 0x80032941a, rsp = 0x7fffffffdb88, rbp = 0x7fffffffdfc0 --- Tracing command mkdir pid 19891 tid 101916 td 0xfffff80482c2b000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00eb4b2690 mi_switch() at mi_switch+0x182/frame 0xfffffe00eb4b26d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00eb4b2710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00eb4b2770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00eb4b27a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00eb4b2840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00eb4b28d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00eb4b2ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb4b2bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb4b2bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c391a, rsp = 0x7fffffffe418, rbp = 0x7fffffffe450 --- Tracing command link pid 19890 tid 102533 td 0xfffff8073be1b6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e78ca550 mi_switch() at mi_switch+0x182/frame 0xfffffe00e78ca590 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e78ca5d0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e78ca600 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00e78ca660 dmu_tx_wait() at dmu_tx_wait+0x31b/frame 0xfffffe00e78ca6b0 dmu_tx_assign() at dmu_tx_assign+0x47d/frame 0xfffffe00e78ca710 zfs_freebsd_create() at zfs_freebsd_create+0x4fe/frame 0xfffffe00e78ca7d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00e78ca800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00e78ca950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00e78caac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e78cabf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e78cabf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe2b8, rbp = 0x7fffffffe3f0 --- Tracing command link pid 19889 tid 102381 td 0xfffff802f9f8c6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e85ce550 mi_switch() at mi_switch+0x182/frame 0xfffffe00e85ce590 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e85ce5d0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e85ce600 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00e85ce660 dmu_tx_wait() at dmu_tx_wait+0x31b/frame 0xfffffe00e85ce6b0 dmu_tx_assign() at dmu_tx_assign+0x47d/frame 0xfffffe00e85ce710 zfs_freebsd_create() at zfs_freebsd_create+0x4fe/frame 0xfffffe00e85ce7d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00e85ce800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00e85ce950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00e85ceac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e85cebf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e85cebf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe2b8, rbp = 0x7fffffffe3f0 --- Tracing command link pid 19887 tid 102281 td 0xfffff8028b921000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e83da550 mi_switch() at mi_switch+0x182/frame 0xfffffe00e83da590 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e83da5d0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e83da600 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00e83da660 dmu_tx_wait() at dmu_tx_wait+0x31b/frame 0xfffffe00e83da6b0 dmu_tx_assign() at dmu_tx_assign+0x47d/frame 0xfffffe00e83da710 zfs_freebsd_create() at zfs_freebsd_create+0x4fe/frame 0xfffffe00e83da7d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00e83da800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00e83da950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00e83daac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e83dabf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e83dabf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe2b8, rbp = 0x7fffffffe3f0 --- Tracing command link pid 19885 tid 102213 td 0xfffff803dc09c000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e8286220 mi_switch() at mi_switch+0x182/frame 0xfffffe00e8286260 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e82862a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e82862d0 sleeplk() at sleeplk+0x107/frame 0xfffffe00e8286330 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00e82863e0 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e8286410 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e8286430 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e8286490 zfs_dd_callback() at zfs_dd_callback+0x23/frame 0xfffffe00e82864b0 vn_vget_ino_gen() at vn_vget_ino_gen+0x11f/frame 0xfffffe00e8286520 zfs_lookup() at zfs_lookup+0x750/frame 0xfffffe00e8286610 zfs_freebsd_lookup() at zfs_freebsd_lookup+0x91/frame 0xfffffe00e8286750 VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xa8/frame 0xfffffe00e8286770 vfs_cache_lookup() at vfs_cache_lookup+0xd2/frame 0xfffffe00e82867d0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xa8/frame 0xfffffe00e82867f0 lookup() at lookup+0x611/frame 0xfffffe00e8286890 namei() at namei+0x4f1/frame 0xfffffe00e8286980 kern_chdir() at kern_chdir+0x4d/frame 0xfffffe00e8286ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e8286bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e8286bf0 --- syscall (12, FreeBSD ELF64, sys_chdir), rip = 0x8002f68ca, rsp = 0x7fffffffe3d8, rbp = 0x7fffffffe3e0 --- Tracing command lockf pid 19884 tid 102161 td 0xfffff805466d76e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00ef97e460 mi_switch() at mi_switch+0x182/frame 0xfffffe00ef97e4a0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00ef97e4e0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00ef97e510 sleeplk() at sleeplk+0x107/frame 0xfffffe00ef97e570 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00ef97e620 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00ef97e650 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00ef97e670 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00ef97e6d0 lookup() at lookup+0xe1/frame 0xfffffe00ef97e770 namei() at namei+0x4f1/frame 0xfffffe00ef97e860 kern_funlinkat() at kern_funlinkat+0xb9/frame 0xfffffe00ef97eaa0 sys_unlink() at sys_unlink+0x28/frame 0xfffffe00ef97eac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00ef97ebf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ef97ebf0 --- syscall (10, FreeBSD ELF64, sys_unlink), rip = 0x80042a5ba, rsp = 0x7fffffffe3c8, rbp = 0x7fffffffe3f0 --- Tracing command link pid 19883 tid 100160 td 0xfffff80030f7c000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e0bf6550 mi_switch() at mi_switch+0x182/frame 0xfffffe00e0bf6590 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e0bf65d0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e0bf6600 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00e0bf6660 dmu_tx_wait() at dmu_tx_wait+0x31b/frame 0xfffffe00e0bf66b0 dmu_tx_assign() at dmu_tx_assign+0x47d/frame 0xfffffe00e0bf6710 zfs_freebsd_create() at zfs_freebsd_create+0x4fe/frame 0xfffffe00e0bf67d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00e0bf6800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00e0bf6950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00e0bf6ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e0bf6bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e0bf6bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe2b8, rbp = 0x7fffffffe3f0 --- Tracing command lockf pid 19882 tid 102373 td 0xfffff802f9f926e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e85a6460 mi_switch() at mi_switch+0x182/frame 0xfffffe00e85a64a0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e85a64e0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e85a6510 sleeplk() at sleeplk+0x107/frame 0xfffffe00e85a6570 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00e85a6620 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e85a6650 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e85a6670 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e85a66d0 lookup() at lookup+0xe1/frame 0xfffffe00e85a6770 namei() at namei+0x4f1/frame 0xfffffe00e85a6860 kern_funlinkat() at kern_funlinkat+0xb9/frame 0xfffffe00e85a6aa0 sys_unlink() at sys_unlink+0x28/frame 0xfffffe00e85a6ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e85a6bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e85a6bf0 --- syscall (10, FreeBSD ELF64, sys_unlink), rip = 0x80042a5ba, rsp = 0x7fffffffe3c8, rbp = 0x7fffffffe3f0 --- Tracing command link pid 19881 tid 102938 td 0xfffff804b6c666e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f00bd220 mi_switch() at mi_switch+0x182/frame 0xfffffe00f00bd260 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f00bd2a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f00bd2d0 sleeplk() at sleeplk+0x107/frame 0xfffffe00f00bd330 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00f00bd3e0 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00f00bd410 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00f00bd430 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00f00bd490 zfs_dd_callback() at zfs_dd_callback+0x23/frame 0xfffffe00f00bd4b0 vn_vget_ino_gen() at vn_vget_ino_gen+0x11f/frame 0xfffffe00f00bd520 zfs_lookup() at zfs_lookup+0x750/frame 0xfffffe00f00bd610 zfs_freebsd_lookup() at zfs_freebsd_lookup+0x91/frame 0xfffffe00f00bd750 VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xa8/frame 0xfffffe00f00bd770 vfs_cache_lookup() at vfs_cache_lookup+0xd2/frame 0xfffffe00f00bd7d0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xa8/frame 0xfffffe00f00bd7f0 lookup() at lookup+0x611/frame 0xfffffe00f00bd890 namei() at namei+0x4f1/frame 0xfffffe00f00bd980 kern_chdir() at kern_chdir+0x4d/frame 0xfffffe00f00bdac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00f00bdbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00f00bdbf0 --- syscall (12, FreeBSD ELF64, sys_chdir), rip = 0x8002f68ca, rsp = 0x7fffffffe3d8, rbp = 0x7fffffffe3e0 --- Tracing command lockf pid 19880 tid 102489 td 0xfffff801e4163000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e77ee460 mi_switch() at mi_switch+0x182/frame 0xfffffe00e77ee4a0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e77ee4e0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e77ee510 sleeplk() at sleeplk+0x107/frame 0xfffffe00e77ee570 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00e77ee620 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00e77ee650 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00e77ee670 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00e77ee6d0 lookup() at lookup+0xe1/frame 0xfffffe00e77ee770 namei() at namei+0x4f1/frame 0xfffffe00e77ee860 kern_funlinkat() at kern_funlinkat+0xb9/frame 0xfffffe00e77eeaa0 sys_unlink() at sys_unlink+0x28/frame 0xfffffe00e77eeac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e77eebf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e77eebf0 --- syscall (10, FreeBSD ELF64, sys_unlink), rip = 0x80042a5ba, rsp = 0x7fffffffe3c8, rbp = 0x7fffffffe3f0 --- Tracing command link pid 19879 tid 100679 td 0xfffff80642884000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00ec00e550 mi_switch() at mi_switch+0x182/frame 0xfffffe00ec00e590 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00ec00e5d0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00ec00e600 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00ec00e660 dmu_tx_wait() at dmu_tx_wait+0x31b/frame 0xfffffe00ec00e6b0 dmu_tx_assign() at dmu_tx_assign+0x47d/frame 0xfffffe00ec00e710 zfs_freebsd_create() at zfs_freebsd_create+0x4fe/frame 0xfffffe00ec00e7d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00ec00e800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00ec00e950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00ec00eac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00ec00ebf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ec00ebf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe2b8, rbp = 0x7fffffffe3f0 --- Tracing command lockf pid 19878 tid 101843 td 0xfffff802942e16e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00eb345460 mi_switch() at mi_switch+0x182/frame 0xfffffe00eb3454a0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00eb3454e0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00eb345510 sleeplk() at sleeplk+0x107/frame 0xfffffe00eb345570 lockmgr_slock_hard() at lockmgr_slock_hard+0x1d6/frame 0xfffffe00eb345620 zfs_lock() at zfs_lock+0x12/frame 0xfffffe00eb345650 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x92/frame 0xfffffe00eb345670 _vn_lock() at _vn_lock+0x65/frame 0xfffffe00eb3456d0 lookup() at lookup+0xe1/frame 0xfffffe00eb345770 namei() at namei+0x4f1/frame 0xfffffe00eb345860 kern_funlinkat() at kern_funlinkat+0xb9/frame 0xfffffe00eb345aa0 sys_unlink() at sys_unlink+0x28/frame 0xfffffe00eb345ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb345bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb345bf0 --- syscall (10, FreeBSD ELF64, sys_unlink), rip = 0x80042a5ba, rsp = 0x7fffffffe3c8, rbp = 0x7fffffffe3f0 --- Tracing command link pid 19877 tid 102406 td 0xfffff803dc08e6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e764f550 mi_switch() at mi_switch+0x182/frame 0xfffffe00e764f590 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e764f5d0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e764f600 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00e764f660 dmu_tx_wait() at dmu_tx_wait+0x31b/frame 0xfffffe00e764f6b0 dmu_tx_assign() at dmu_tx_assign+0x47d/frame 0xfffffe00e764f710 zfs_freebsd_create() at zfs_freebsd_create+0x4fe/frame 0xfffffe00e764f7d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00e764f800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00e764f950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00e764fac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e764fbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e764fbf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe2b8, rbp = 0x7fffffffe3f0 --- Tracing command lockf pid 19876 tid 100528 td 0xfffff80191f62000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e4f1c690 mi_switch() at mi_switch+0x182/frame 0xfffffe00e4f1c6d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e4f1c710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e4f1c770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e4f1c7a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e4f1c840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00e4f1c8d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00e4f1cac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e4f1cbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e4f1cbf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c391a, rsp = 0x7fffffffe418, rbp = 0x7fffffffe450 --- Tracing command link pid 19875 tid 101597 td 0xfffff801961046e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00ec127690 mi_switch() at mi_switch+0x182/frame 0xfffffe00ec1276d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00ec127710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00ec127770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ec1277a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00ec127840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00ec1278d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00ec127ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00ec127bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ec127bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c391a, rsp = 0x7fffffffe418, rbp = 0x7fffffffe450 --- Tracing command openat pid 19874 tid 100341 td 0xfffff807b86976e0 (CPU 4) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e193dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e193df0 trap() at trap+0x61/frame 0xfffffe003e193f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e193f20 --- trap 0x13, rip = 0xffffffff8259ac56, rsp = 0xfffffe00e3b723d0, rbp = 0xfffffe00e3b723f0 --- dnode_check_slots_free() at dnode_check_slots_free+0x126/frame 0xfffffe00e3b723f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00e3b724d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00e3b725c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00e3b72700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00e3b727d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00e3b72800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00e3b72950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00e3b72ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e3b72bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e3b72bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command openat pid 19873 tid 100616 td 0xfffff801ac17d6e0 (CPU 5) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e19fdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e19fdf0 trap() at trap+0x61/frame 0xfffffe003e19ff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e19ff20 --- trap 0x13, rip = 0xffffffff80c11a90, rsp = 0xfffffe00ee2d62a0, rbp = 0xfffffe00ee2d62b0 --- lock_delay() at lock_delay+0x30/frame 0xfffffe00ee2d62b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00ee2d6370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00ee2d63b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00ee2d63f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00ee2d64d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00ee2d65c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00ee2d6700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00ee2d67d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00ee2d6800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00ee2d6950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00ee2d6ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00ee2d6bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ee2d6bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command openat pid 19872 tid 100290 td 0xfffff8057d8f86e0 (CPU 11) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1e7dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1e7df0 trap() at trap+0x61/frame 0xfffffe003e1e7f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1e7f20 --- trap 0x13, rip = 0xffffffff80c11a92, rsp = 0xfffffe00e2bc12a0, rbp = 0xfffffe00e2bc12b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00e2bc12b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00e2bc1370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00e2bc13b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00e2bc13f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00e2bc14d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00e2bc15c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00e2bc1700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00e2bc17d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00e2bc1800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00e2bc1950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00e2bc1ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2bc1bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2bc1bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command openat pid 19871 tid 101955 td 0xfffff80278cd96e0 (CPU 0) kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe00eb5750a0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00eb5750b0 uart_intr_rxready() at uart_intr_rxready+0xa5/frame 0xfffffe00eb5750e0 uart_intr() at uart_intr+0x138/frame 0xfffffe00eb575120 intr_event_handle() at intr_event_handle+0xce/frame 0xfffffe00eb575170 intr_execute_handlers() at intr_execute_handlers+0x75/frame 0xfffffe00eb5751b0 lapic_handle_intr() at lapic_handle_intr+0x5b/frame 0xfffffe00eb5751d0 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00eb5751d0 --- interrupt, rip = 0xffffffff80c11a92, rsp = 0xfffffe00eb5752a0, rbp = 0xfffffe00eb5752b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00eb5752b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00eb575370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00eb5753b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00eb5753f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00eb5754d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5755c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb575700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5757d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb575800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb575950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb575ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb575bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb575bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command dirrename pid 19870 tid 102470 td 0xfffff805466bb6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e778f690 mi_switch() at mi_switch+0x182/frame 0xfffffe00e778f6d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e778f710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e778f770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e778f7a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e778f840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00e778f8d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00e778fac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e778fbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e778fbf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c391a, rsp = 0x7fffffffe448, rbp = 0x7fffffffe560 --- Tracing command openat pid 19869 tid 101965 td 0xfffff8038d13c000 (CPU 10) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1dbdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1dbdf0 trap() at trap+0x61/frame 0xfffffe003e1dbf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1dbf20 --- trap 0x13, rip = 0xffffffff82641c07, rsp = 0xfffffe00eb5a73e0, rbp = 0xfffffe00eb5a73f0 --- zrl_tryenter() at zrl_tryenter+0x7/frame 0xfffffe00eb5a73f0 dnode_hold_impl() at dnode_hold_impl+0xbb8/frame 0xfffffe00eb5a74d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5a75c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb5a7700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5a77d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb5a7800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb5a7950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb5a7ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb5a7bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb5a7bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command dirnprename pid 19868 tid 102021 td 0xfffff80014c50000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00ef6c2690 mi_switch() at mi_switch+0x182/frame 0xfffffe00ef6c26d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00ef6c2710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00ef6c2770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ef6c27a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00ef6c2840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00ef6c28d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00ef6c2ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00ef6c2bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ef6c2bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c391a, rsp = 0x7fffffffe438, rbp = 0x7fffffffe550 --- Tracing command openat pid 19867 tid 101982 td 0xfffff8019603a6e0 (CPU 9) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1cfdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1cfdf0 trap() at trap+0x61/frame 0xfffffe003e1cff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1cff20 --- trap 0x13, rip = 0xffffffff80c11a92, rsp = 0xfffffe00eb5fc2a0, rbp = 0xfffffe00eb5fc2b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00eb5fc2b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00eb5fc370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00eb5fc3b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00eb5fc3f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00eb5fc4d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5fc5c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb5fc700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5fc7d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb5fc800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb5fc950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb5fcac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb5fcbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb5fcbf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command openat pid 19866 tid 102244 td 0xfffff801ca3b5000 (CPU 3) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e187dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e187df0 trap() at trap+0x61/frame 0xfffffe003e187f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e187f20 --- trap 0x13, rip = 0xffffffff80c11a92, rsp = 0xfffffe00e83212a0, rbp = 0xfffffe00e83212b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00e83212b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00e8321370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00e83213b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00e83213f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00e83214d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00e83215c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00e8321700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00e83217d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00e8321800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00e8321950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00e8321ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e8321bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e8321bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command openat pid 19864 tid 100245 td 0xfffff8057d8e46e0 (CPU 6) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1abdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1abdf0 trap() at trap+0x61/frame 0xfffffe003e1abf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1abf20 --- trap 0x13, rip = 0xffffffff80c11a92, rsp = 0xfffffe00e3a0e2a0, rbp = 0xfffffe00e3a0e2b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00e3a0e2b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00e3a0e370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00e3a0e3b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00e3a0e3f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00e3a0e4d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00e3a0e5c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00e3a0e700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00e3a0e7d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00e3a0e800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00e3a0e950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00e3a0eac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e3a0ebf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e3a0ebf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command openat pid 19862 tid 100566 td 0xfffff801ac8d86e0 (CPU 1) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e16fdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e16fdf0 trap() at trap+0x61/frame 0xfffffe003e16ff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e16ff20 --- trap 0x13, rip = 0xffffffff80c11a90, rsp = 0xfffffe00e4fda2a0, rbp = 0xfffffe00e4fda2b0 --- lock_delay() at lock_delay+0x30/frame 0xfffffe00e4fda2b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00e4fda370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00e4fda3b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00e4fda3f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00e4fda4d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00e4fda5c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00e4fda700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00e4fda7d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00e4fda800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00e4fda950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00e4fdaac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e4fdabf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e4fdabf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command openat pid 19859 tid 102445 td 0xfffff8073beb76e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e7712690 mi_switch() at mi_switch+0x182/frame 0xfffffe00e77126d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e7712710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e7712770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e77127a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e7712840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00e77128d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00e7712ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e7712bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e7712bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c391a, rsp = 0x7fffffffe408, rbp = 0x7fffffffe440 --- Tracing command mkdir pid 19857 tid 100602 td 0xfffff80191bb1000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00ee290690 mi_switch() at mi_switch+0x182/frame 0xfffffe00ee2906d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00ee290710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00ee290770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ee2907a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00ee290840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00ee2908d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00ee290ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00ee290bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ee290bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c391a, rsp = 0x7fffffffe458, rbp = 0x7fffffffe570 --- Tracing command lockf pid 19855 tid 101981 td 0xfffff806bdf2b000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00eb5f7690 mi_switch() at mi_switch+0x182/frame 0xfffffe00eb5f76d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00eb5f7710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00eb5f7770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00eb5f77a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00eb5f7840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00eb5f78d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00eb5f7ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb5f7bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb5f7bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c391a, rsp = 0x7fffffffe458, rbp = 0x7fffffffe570 --- Tracing command link pid 19854 tid 101958 td 0xfffff80109ee7000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00eb584690 mi_switch() at mi_switch+0x182/frame 0xfffffe00eb5846d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00eb584710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00eb584770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00eb5847a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00eb584840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00eb5848d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00eb584ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb584bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb584bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c391a, rsp = 0x7fffffffe458, rbp = 0x7fffffffe570 --- Tracing command openat pid 19847 tid 101959 td 0xfffff80278cd9000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00eb589690 mi_switch() at mi_switch+0x182/frame 0xfffffe00eb5896d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00eb589710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00eb589770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00eb5897a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00eb589840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00eb5898d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00eb589ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb589bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb589bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c391a, rsp = 0x7fffffffe448, rbp = 0x7fffffffe560 --- Tracing command sysctl pid 19843 tid 100221 td 0xfffff80044018000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2b97690 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2b976d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2b97710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e2b97770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e2b977a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e2b97840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00e2b978d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00e2b97ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2b97bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2b97bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c191a, rsp = 0x7fffffffe448, rbp = 0x7fffffffe560 --- Tracing command run pid 19838 tid 101846 td 0xfffff802604806e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00eb354690 mi_switch() at mi_switch+0x182/frame 0xfffffe00eb3546d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00eb354710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00eb354770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00eb3547a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00eb354840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00eb3548d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00eb354ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb354bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb354bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c191a, rsp = 0x7fffffffdbd8, rbp = 0x7fffffffe100 --- Tracing command run pid 19836 tid 100187 td 0xfffff800307256e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2ac9690 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2ac96d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2ac9710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e2ac9770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e2ac97a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e2ac9840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00e2ac98d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00e2ac9ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2ac9bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2ac9bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c191a, rsp = 0x7fffffffe128, rbp = 0x7fffffffe160 --- Tracing command run pid 19834 tid 100332 td 0xfffff806ad1556e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3b45690 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3b456d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e3b45710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e3b45770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e3b457a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e3b45840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00e3b458d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00e3b45ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e3b45bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e3b45bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c191a, rsp = 0x7fffffffe168, rbp = 0x7fffffffe280 --- Tracing command sh pid 19810 tid 100777 td 0xfffff802ecadd6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00ec1e0690 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00eb5746f0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00eb574700 trap() at trap+0x73/frame 0xfffffe00eb574830 calltrap() at calltrap+0x8/frame 0xfffffe00eb574830 --- trap 0xc, rip = 0xffffffff8103e0a0, rsp = 0xfffffe00eb574900, rbp = 0xfffffe00eb574990 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe00eb574990 db_get_value() at db_get_value+0x32/frame 0xfffffe00eb5749d0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00eb574a60 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00eb574a80 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00eb574b20 db_command() at db_command+0x25c/frame 0xfffffe00eb574bf0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00eb574c90 db_command() at db_command+0x25c/frame 0xfffffe00eb574d60 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00eb574d70 db_trap() at db_trap+0xe8/frame 0xfffffe00eb574e00 kdb_trap() at kdb_trap+0x157/frame 0xfffffe00eb574e80 trap() at trap+0x7d4/frame 0xfffffe00eb574fb0 calltrap() at calltrap+0x8/frame 0xfffffe00eb574fb0 --- trap 0x3, rip = 0xffffffff80c105a1, rsp = 0xfffffe00eb575080, rbp = 0xfffffe00eb5750a0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe00eb5750a0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00eb5750b0 uart_intr_rxready() at uart_intr_rxready+0xa5/frame 0xfffffe00eb5750e0 uart_intr() at uart_intr+0x138/frame 0xfffffe00eb575120 intr_event_handle() at intr_event_handle+0xce/frame 0xfffffe00eb575170 intr_execute_handlers() at intr_execute_handlers+0x75/frame 0xfffffe00eb5751b0 lapic_handle_intr() at lapic_handle_intr+0x5b/frame 0xfffffe00eb5751d0 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00eb5751d0 --- interrupt, rip = 0xffffffff80c11a92, rsp = 0xfffffe00eb5752a0, rbp = 0xfffffe00eb5752b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00eb5752b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00eb575370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00eb5753b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00eb5753f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00eb5754d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5755c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb575700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5757d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb575800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb575950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb575ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb575bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb575bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- *** error reading from address fffffe00ec1e0698 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00eb574980 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00eb574990 db_get_value() at db_get_value+0x4c/frame 0xfffffe00eb5749d0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00eb574a60 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00eb574a80 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00eb574b20 db_command() at db_command+0x25c/frame 0xfffffe00eb574bf0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00eb574c90 db_command() at db_command+0x25c/frame 0xfffffe00eb574d60 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00eb574d70 db_trap() at db_trap+0xe8/frame 0xfffffe00eb574e00 kdb_trap() at kdb_trap+0x157/frame 0xfffffe00eb574e80 trap() at trap+0x7d4/frame 0xfffffe00eb574fb0 calltrap() at calltrap+0x8/frame 0xfffffe00eb574fb0 --- trap 0x3, rip = 0xffffffff80c105a1, rsp = 0xfffffe00eb575080, rbp = 0xfffffe00eb5750a0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe00eb5750a0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00eb5750b0 uart_intr_rxready() at uart_intr_rxready+0xa5/frame 0xfffffe00eb5750e0 uart_intr() at uart_intr+0x138/frame 0xfffffe00eb575120 intr_event_handle() at intr_event_handle+0xce/frame 0xfffffe00eb575170 intr_execute_handlers() at intr_execute_handlers+0x75/frame 0xfffffe00eb5751b0 lapic_handle_intr() at lapic_handle_intr+0x5b/frame 0xfffffe00eb5751d0 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00eb5751d0 --- interrupt, rip = 0xffffffff80c11a92, rsp = 0xfffffe00eb5752a0, rbp = 0xfffffe00eb5752b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00eb5752b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00eb575370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00eb5753b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00eb5753f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00eb5754d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5755c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb575700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5757d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb575800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb575950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb575ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb575bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb575bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command zpool-stress2_tank pid 19734 tid 101835 td 0xfffff806fe6e9000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00eb31da30 mi_switch() at mi_switch+0x182/frame 0xfffffe00eb31da70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00eb31dab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00eb31dae0 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00eb31db40 spa_thread() at spa_thread+0x19b/frame 0xfffffe00eb31dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00eb31dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00eb31dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 100995 td 0xfffff805606446e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c3ba30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c3ba70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c3bab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c3bae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c3bb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c3bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c3bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c3bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 100998 td 0xfffff80560644000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c40a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c40a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c40ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c40ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c40b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c40bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c40bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c40bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101000 td 0xfffff807c237f6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c45a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c45a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c45ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c45ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c45b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c45bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c45bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c45bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101003 td 0xfffff807c237f000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c4aa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c4aa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c4aab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c4aae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c4ab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c4abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c4abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c4abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101007 td 0xfffff803bdcd66e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c4fa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c4fa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c4fab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c4fae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c4fb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c4fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c4fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c4fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101009 td 0xfffff8081dd12000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c54a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c54a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c54ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c54ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c54b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c54bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c54bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c54bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101010 td 0xfffff8081d3766e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c59a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c59a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c59ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c59ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c59b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c59bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c59bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c59bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101012 td 0xfffff8081d376000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c5ea30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c5ea70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c5eab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c5eae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c5eb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c5ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c5ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c5ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101019 td 0xfffff803c9ae06e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c63a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c63a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c63ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c63ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c63b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c63bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c63bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c63bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101022 td 0xfffff803c9ae0000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c68a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c68a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c68ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c68ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c68b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c68bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c68bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c68bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101024 td 0xfffff804e8acc6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c6da30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c6da70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c6dab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c6dae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c6db80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c6dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c6dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c6dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101026 td 0xfffff804e8acc000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c72a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c72a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c72ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c72ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c72b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c72bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c72bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c72bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101027 td 0xfffff802da0b36e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c77a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c77a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c77ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c77ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c77b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c77bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c77bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c77bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101036 td 0xfffff802da0b3000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c7ca30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c7ca70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c7cab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c7cae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c7cb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c7cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c7cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c7cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101039 td 0xfffff8081dd126e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c81a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c81a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c81ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c81ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c81b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c81bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c81bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c81bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101040 td 0xfffff8027da79000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c86a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c86a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c86ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c86ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c86b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c86bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c86bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c86bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101042 td 0xfffff802dace96e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c8ba30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c8ba70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c8bab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c8bae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c8bb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c8bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c8bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c8bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101043 td 0xfffff802dace9000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c90a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c90a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c90ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c90ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c90b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c90bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c90bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c90bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101044 td 0xfffff807b1bdc6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c95a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c95a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c95ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c95ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c95b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c95bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c95bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c95bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101051 td 0xfffff807b1bdc000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c9aa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c9aa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c9aab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c9aae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c9ab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c9abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c9abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c9abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101053 td 0xfffff804e8aaf6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c9fa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c9fa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c9fab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c9fae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c9fb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c9fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c9fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c9fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101057 td 0xfffff804e8aaf000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4ca4a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4ca4a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4ca4ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4ca4ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4ca4b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4ca4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4ca4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4ca4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101060 td 0xfffff802da0b76e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4ca9a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4ca9a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4ca9ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4ca9ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4ca9b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4ca9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4ca9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4ca9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101061 td 0xfffff802da0b7000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4caea30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4caea70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4caeab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4caeae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4caeb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4caebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4caebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4caebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101070 td 0xfffff8027da796e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4cb3a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4cb3a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4cb3ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4cb3ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4cb3b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4cb3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4cb3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4cb3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101071 td 0xfffff8037faa0000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4cb8a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4cb8a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4cb8ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4cb8ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4cb8b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4cb8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4cb8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4cb8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101073 td 0xfffff8063d4ce6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4cbda30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4cbda70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4cbdab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4cbdae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4cbdb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4cbdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4cbdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4cbdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101074 td 0xfffff8063d4ce000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4cc2a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4cc2a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4cc2ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4cc2ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4cc2b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4cc2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4cc2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4cc2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101075 td 0xfffff807236526e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4cc7a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4cc7a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4cc7ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4cc7ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4cc7b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4cc7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4cc7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4cc7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101077 td 0xfffff80723652000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4ccca30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4ccca70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4cccab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4cccae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4cccb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4cccbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4cccbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4cccbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101082 td 0xfffff8063d4a86e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4cd1a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4cd1a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4cd1ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4cd1ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4cd1b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4cd1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4cd1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4cd1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101083 td 0xfffff8063d4a8000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4cd6a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4cd6a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4cd6ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4cd6ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4cd6b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4cd6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4cd6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4cd6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101087 td 0xfffff802c7cde6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4cdba30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4cdba70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4cdbab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4cdbae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4cdbb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4cdbbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4cdbbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4cdbbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101088 td 0xfffff802c7cde000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4ce0a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4ce0a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4ce0ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4ce0ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4ce0b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4ce0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4ce0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4ce0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101089 td 0xfffff8037faa06e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4ce5a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4ce5a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4ce5ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4ce5ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4ce5b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4ce5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4ce5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4ce5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101092 td 0xfffff804334fa000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4ceaa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4ceaa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4ceaab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4ceaae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4ceab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4ceabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4ceabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4ceabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101096 td 0xfffff8056066f6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4cefa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4cefa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4cefab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4cefae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4cefb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4cefbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4cefbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4cefbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101098 td 0xfffff8056066f000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4cf4a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4cf4a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4cf4ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4cf4ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4cf4b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4cf4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4cf4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4cf4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101102 td 0xfffff802dad106e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4cf9a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4cf9a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4cf9ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4cf9ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4cf9b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4cf9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4cf9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4cf9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101107 td 0xfffff802dad10000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4cfea30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4cfea70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4cfeab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4cfeae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4cfeb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4cfebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4cfebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4cfebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101108 td 0xfffff8012ee4e6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d03a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d03a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d03ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d03ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d03b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d03bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d03bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d03bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101110 td 0xfffff8012ee4e000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d08a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d08a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d08ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d08ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d08b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d08bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d08bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d08bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101113 td 0xfffff80051f846e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d0da30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d0da70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d0dab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d0dae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d0db80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d0dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d0dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d0dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101116 td 0xfffff80051f84000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d12a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d12a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d12ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d12ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d12b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d12bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d12bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d12bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101117 td 0xfffff804334fa6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d17a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d17a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d17ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d17ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d17b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d17bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d17bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d17bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101119 td 0xfffff806994b3000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d1ca30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d1ca70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d1cab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d1cae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d1cb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d1cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d1cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d1cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101121 td 0xfffff807236426e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d21a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d21a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d21ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d21ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d21b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d21bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d21bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d21bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101128 td 0xfffff80723642000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d26a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d26a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d26ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d26ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d26b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d26bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d26bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d26bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101129 td 0xfffff80194f986e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d2ba30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d2ba70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d2bab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d2bae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d2bb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d2bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d2bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d2bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101132 td 0xfffff80194f98000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d30a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d30a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d30ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d30ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d30b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d30bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d30bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d30bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101137 td 0xfffff806994056e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d35a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d35a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d35ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d35ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d35b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d35bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d35bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d35bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101138 td 0xfffff80699405000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d3aa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d3aa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d3aab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d3aae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d3ab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d3abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d3abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d3abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101143 td 0xfffff8037fa016e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d3fa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d3fa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d3fab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d3fae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d3fb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d3fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d3fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d3fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101145 td 0xfffff8037fa01000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d44a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d44a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d44ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d44ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d44b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d44bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d44bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d44bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101146 td 0xfffff806994b36e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d49a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d49a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d49ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d49ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d49b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d49bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d49bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d49bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101147 td 0xfffff802dac25000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d4ea30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d4ea70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d4eab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d4eae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d4eb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d4ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d4ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d4ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101148 td 0xfffff802daccc6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d53a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d53a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d53ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d53ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d53b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d53bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d53bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d53bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101152 td 0xfffff802daccc000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d58a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d58a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d58ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d58ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d58b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d58bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d58bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d58bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101159 td 0xfffff805606086e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d5da30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d5da70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d5dab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d5dae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d5db80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d5dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d5dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d5dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101161 td 0xfffff80560608000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d62a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d62a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d62ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d62ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d62b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d62bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d62bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d62bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101163 td 0xfffff801dea356e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d67a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d67a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d67ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d67ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d67b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d67bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d67bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d67bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101164 td 0xfffff801dea35000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d6ca30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d6ca70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d6cab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d6cae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d6cb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d6cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d6cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d6cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101168 td 0xfffff802dacfa6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d71a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d71a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d71ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d71ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d71b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d71bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d71bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d71bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101169 td 0xfffff802dacfa000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d76a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d76a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d76ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d76ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d76b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d76bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d76bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d76bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101174 td 0xfffff802dac256e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d7ba30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d7ba70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d7bab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d7bae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d7bb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d7bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d7bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d7bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101176 td 0xfffff8047964d000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d80a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d80a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d80ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d80ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d80b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d80bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d80bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d80bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101177 td 0xfffff8081d3726e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d85a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d85a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d85ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d85ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d85b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d85bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d85bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d85bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101180 td 0xfffff8081d372000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d8aa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d8aa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d8aab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d8aae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d8ab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d8abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d8abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d8abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101182 td 0xfffff805606aa6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d8fa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d8fa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d8fab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d8fae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d8fb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d8fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d8fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d8fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101183 td 0xfffff805606aa000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d94a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d94a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d94ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d94ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d94b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d94bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d94bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d94bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101184 td 0xfffff8034c4416e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d99a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d99a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d99ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d99ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d99b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d99bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d99bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d99bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101189 td 0xfffff8034c441000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4d9ea30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4d9ea70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4d9eab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4d9eae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4d9eb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4d9ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4d9ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4d9ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101191 td 0xfffff805a9c9c6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4da3a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4da3a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4da3ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4da3ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4da3b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4da3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4da3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4da3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101194 td 0xfffff805a9c9c000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4da8a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4da8a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4da8ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4da8ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4da8b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4da8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4da8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4da8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101198 td 0xfffff8047964d6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4dada30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4dada70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4dadab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4dadae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4dadb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4dadbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4dadbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4dadbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101199 td 0xfffff804a37aa000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4db2a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4db2a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4db2ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4db2ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4db2b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4db2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4db2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4db2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101200 td 0xfffff8081dcfd6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4db7a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4db7a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4db7ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4db7ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4db7b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4db7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4db7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4db7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101202 td 0xfffff8081dcfd000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4dbca30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4dbca70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4dbcab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4dbcae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4dbcb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4dbcbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4dbcbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4dbcbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101203 td 0xfffff8034c4606e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4dc1a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4dc1a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4dc1ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4dc1ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4dc1b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4dc1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4dc1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4dc1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101204 td 0xfffff8034c460000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4dc6a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4dc6a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4dc6ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4dc6ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4dc6b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4dc6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4dc6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4dc6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101206 td 0xfffff804312a26e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4dcba30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4dcba70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4dcbab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4dcbae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4dcbb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4dcbbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4dcbbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4dcbbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101208 td 0xfffff804312a2000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4dd0a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4dd0a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4dd0ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4dd0ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4dd0b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4dd0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4dd0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4dd0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101209 td 0xfffff804e8a606e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4dd5a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4dd5a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4dd5ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4dd5ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4dd5b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4dd5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4dd5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4dd5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101210 td 0xfffff804e8a60000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4ddaa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4ddaa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4ddaab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4ddaae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4ddab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4ddabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4ddabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4ddabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101213 td 0xfffff804a37aa6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4ddfa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4ddfa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4ddfab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4ddfae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4ddfb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4ddfbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4ddfbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4ddfbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101214 td 0xfffff806107b3000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4de4a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4de4a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4de4ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4de4ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4de4b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4de4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4de4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4de4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101215 td 0xfffff8081dd176e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4de9a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4de9a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4de9ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4de9ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4de9b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4de9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4de9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4de9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101218 td 0xfffff8081dd17000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4deea30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4deea70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4deeab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4deeae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4deeb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4deebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4deebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4deebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101219 td 0xfffff804e8af06e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4df3a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4df3a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4df3ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4df3ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4df3b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4df3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4df3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4df3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101220 td 0xfffff804e8af0000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4df8a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4df8a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4df8ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4df8ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4df8b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4df8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4df8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4df8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101221 td 0xfffff806994a06e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4dfda30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4dfda70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4dfdab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4dfdae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4dfdb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4dfdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4dfdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4dfdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101222 td 0xfffff806994a0000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1404a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1404a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1404ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1404ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1404b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1404bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1404bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1404bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101224 td 0xfffff80841f776e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1409a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1409a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1409ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1409ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1409b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1409bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1409bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1409bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101225 td 0xfffff80841f77000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f140ea30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f140ea70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f140eab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f140eae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f140eb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f140ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f140ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f140ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101226 td 0xfffff806107b36e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1413a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1413a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1413ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1413ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1413b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1413bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1413bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1413bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101230 td 0xfffff803aa0af000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1418a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1418a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1418ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1418ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1418b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1418bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1418bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1418bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101234 td 0xfffff8069947d6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f141da30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f141da70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f141dab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f141dae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f141db80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f141dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f141dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f141dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101235 td 0xfffff8069947d000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1422a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1422a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1422ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1422ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1422b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1422bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1422bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1422bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101236 td 0xfffff802dac356e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1427a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1427a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1427ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1427ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1427b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1427bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1427bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1427bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101237 td 0xfffff802dac35000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f142ca30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f142ca70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f142cab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f142cae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f142cb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f142cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f142cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f142cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101238 td 0xfffff8011d2766e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1431a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1431a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1431ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1431ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1431b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1431bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1431bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1431bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101252 td 0xfffff8011d276000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1436a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1436a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1436ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1436ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1436b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1436bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1436bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1436bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101253 td 0xfffff802ba1fa6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f143ba30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f143ba70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f143bab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f143bae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f143bb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f143bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f143bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f143bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101263 td 0xfffff802ba1fa000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1440a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1440a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1440ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1440ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1440b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1440bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1440bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1440bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101264 td 0xfffff803aa0af6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1445a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1445a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1445ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1445ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1445b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1445bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1445bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1445bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101265 td 0xfffff802ba11c000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f144aa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f144aa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f144aab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f144aae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f144ab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f144abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f144abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f144abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101266 td 0xfffff8063d4a66e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f144fa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f144fa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f144fab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f144fae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f144fb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f144fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f144fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f144fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101268 td 0xfffff8063d4a6000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1454a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1454a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1454ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1454ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1454b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1454bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1454bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1454bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101270 td 0xfffff802dac536e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1459a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1459a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1459ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1459ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1459b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1459bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1459bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1459bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101271 td 0xfffff802dac53000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f145ea30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f145ea70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f145eab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f145eae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f145eb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f145ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f145ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f145ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101273 td 0xfffff803aa0ad6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1463a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1463a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1463ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1463ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1463b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1463bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1463bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1463bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101274 td 0xfffff803aa0ad000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1468a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1468a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1468ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1468ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1468b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1468bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1468bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1468bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101275 td 0xfffff8034c4906e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f146da30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f146da70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f146dab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f146dae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f146db80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f146dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f146dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f146dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101277 td 0xfffff8034c490000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1472a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1472a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1472ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1472ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1472b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1472bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1472bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1472bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101279 td 0xfffff802ba11c6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1477a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1477a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1477ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1477ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1477b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1477bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1477bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1477bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101280 td 0xfffff8063d421000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f147ca30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f147ca70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f147cab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f147cae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f147cb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f147cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f147cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f147cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101281 td 0xfffff801ca3106e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1481a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1481a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1481ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1481ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1481b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1481bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1481bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1481bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101283 td 0xfffff801ca310000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1486a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1486a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1486ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1486ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1486b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1486bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1486bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1486bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101284 td 0xfffff8054f9fc6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f148ba30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f148ba70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f148bab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f148bae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f148bb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f148bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f148bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f148bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101286 td 0xfffff8054f9fc000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1490a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1490a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1490ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1490ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1490b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1490bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1490bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1490bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101287 td 0xfffff80785a216e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1495a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1495a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1495ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1495ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1495b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1495bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1495bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1495bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101288 td 0xfffff80785a21000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f149aa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f149aa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f149aab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f149aae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f149ab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f149abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f149abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f149abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101291 td 0xfffff802f3fc26e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f149fa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f149fa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f149fab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f149fae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f149fb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f149fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f149fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f149fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101293 td 0xfffff802f3fc2000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f14a4a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f14a4a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f14a4ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f14a4ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f14a4b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f14a4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f14a4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f14a4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101294 td 0xfffff8063d4216e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f14a9a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f14a9a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f14a9ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f14a9ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f14a9b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f14a9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f14a9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f14a9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101295 td 0xfffff8011d21e000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f14aea30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f14aea70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f14aeab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f14aeae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f14aeb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f14aebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f14aebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f14aebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101297 td 0xfffff802dabf16e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f14b3a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f14b3a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f14b3ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f14b3ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f14b3b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f14b3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f14b3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f14b3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101298 td 0xfffff802dabf1000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f14b8a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f14b8a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f14b8ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f14b8ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f14b8b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f14b8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f14b8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f14b8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101299 td 0xfffff806994d26e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f14bda30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f14bda70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f14bdab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f14bdae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f14bdb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f14bdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f14bdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f14bdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101301 td 0xfffff806994d2000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f14c2a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f14c2a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f14c2ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f14c2ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f14c2b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f14c2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f14c2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f14c2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101302 td 0xfffff8081d36f6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f14c7a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f14c7a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f14c7ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f14c7ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f14c7b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f14c7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f14c7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f14c7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101303 td 0xfffff8081d36f000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f14cca30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f14cca70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f14ccab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f14ccae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f14ccb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f14ccbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f14ccbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f14ccbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101304 td 0xfffff8011d2c36e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f14d1a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f14d1a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f14d1ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f14d1ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f14d1b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f14d1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f14d1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f14d1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101306 td 0xfffff8011d2c3000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f14d6a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f14d6a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f14d6ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f14d6ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f14d6b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f14d6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f14d6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f14d6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101307 td 0xfffff8011d21e6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f14dba30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f14dba70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f14dbab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f14dbae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f14dbb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f14dbbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f14dbbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f14dbbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101308 td 0xfffff8046ce0c000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f14e0a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f14e0a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f14e0ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f14e0ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f14e0b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f14e0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f14e0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f14e0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101310 td 0xfffff806994206e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f14e5a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f14e5a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f14e5ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f14e5ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f14e5b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f14e5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f14e5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f14e5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101311 td 0xfffff80699420000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f14eaa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f14eaa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f14eaab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f14eaae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f14eab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f14eabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f14eabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f14eabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101312 td 0xfffff807236776e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f14efa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f14efa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f14efab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f14efae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f14efb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f14efbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f14efbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f14efbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101314 td 0xfffff80723677000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f14f4a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f14f4a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f14f4ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f14f4ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f14f4b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f14f4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f14f4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f14f4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101315 td 0xfffff8052ae8a6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f14f9a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f14f9a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f14f9ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f14f9ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f14f9b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f14f9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f14f9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f14f9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101316 td 0xfffff8052ae8a000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f14fea30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f14fea70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f14feab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f14feae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f14feb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f14febb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f14febf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f14febf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101317 td 0xfffff8034c44d6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1503a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1503a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1503ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1503ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1503b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1503bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1503bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1503bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101319 td 0xfffff8034c44d000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1508a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1508a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1508ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1508ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1508b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1508bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1508bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1508bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101320 td 0xfffff8046ce0c6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f150da30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f150da70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f150dab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f150dae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f150db80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f150dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f150dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f150dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101321 td 0xfffff803123ec000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1512a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1512a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1512ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1512ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1512b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1512bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1512bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1512bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101323 td 0xfffff802dacf96e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1517a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1517a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1517ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1517ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1517b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1517bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1517bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1517bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101326 td 0xfffff802dacf9000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f151ca30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f151ca70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f151cab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f151cae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f151cb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f151cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f151cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f151cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101327 td 0xfffff804989066e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1521a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1521a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1521ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1521ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1521b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1521bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1521bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1521bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101329 td 0xfffff80498906000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1526a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1526a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1526ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1526ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1526b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1526bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1526bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1526bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101335 td 0xfffff804e8a486e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f152ba30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f152ba70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f152bab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f152bae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f152bb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f152bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f152bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f152bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101336 td 0xfffff804e8a48000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1530a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1530a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1530ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1530ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1530b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1530bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1530bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1530bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101340 td 0xfffff8063d4996e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1535a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1535a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1535ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1535ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1535b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1535bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1535bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1535bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101343 td 0xfffff8063d499000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f153aa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f153aa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f153aab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f153aae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f153ab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f153abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f153abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f153abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101347 td 0xfffff803123ec6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f153fa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f153fa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f153fab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f153fae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f153fb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f153fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f153fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f153fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101349 td 0xfffff804e8a9a000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1544a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1544a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1544ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1544ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1544b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1544bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1544bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1544bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101351 td 0xfffff806994866e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1549a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1549a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1549ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1549ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1549b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1549bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1549bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1549bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101356 td 0xfffff80699486000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f154ea30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f154ea70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f154eab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f154eae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f154eb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f154ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f154ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f154ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101361 td 0xfffff805d492f6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1553a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1553a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1553ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1553ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1553b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1553bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1553bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1553bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101365 td 0xfffff805d492f000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1558a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1558a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1558ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1558ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1558b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1558bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1558bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1558bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101366 td 0xfffff8081d38d6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f155da30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f155da70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f155dab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f155dae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f155db80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f155dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f155dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f155dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101367 td 0xfffff8081d38d000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1562a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1562a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1562ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1562ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1562b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1562bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1562bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1562bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101368 td 0xfffff8034c4fc6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1567a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1567a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1567ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1567ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1567b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1567bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1567bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1567bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101369 td 0xfffff8034c4fc000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f156ca30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f156ca70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f156cab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f156cae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f156cb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f156cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f156cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f156cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101370 td 0xfffff804e8a9a6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1571a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1571a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1571ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1571ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1571b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1571bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1571bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1571bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101371 td 0xfffff80699441000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1576a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1576a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1576ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1576ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1576b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1576bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1576bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1576bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101372 td 0xfffff801e6a1e6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f157ba30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f157ba70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f157bab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f157bae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f157bb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f157bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f157bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f157bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101373 td 0xfffff801e6a1e000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1580a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1580a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1580ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1580ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1580b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1580bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1580bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1580bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101374 td 0xfffff8018739b6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1585a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1585a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1585ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1585ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1585b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1585bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1585bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1585bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101375 td 0xfffff8018739b000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f158aa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f158aa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f158aab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f158aae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f158ab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f158abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f158abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f158abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101376 td 0xfffff8070810d6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f158fa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f158fa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f158fab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f158fae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f158fb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f158fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f158fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f158fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101377 td 0xfffff8070810d000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1594a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1594a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1594ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1594ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1594b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1594bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1594bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1594bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101378 td 0xfffff8063d4556e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1599a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1599a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1599ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1599ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1599b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1599bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1599bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1599bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101379 td 0xfffff8063d455000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f159ea30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f159ea70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f159eab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f159eae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f159eb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f159ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f159ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f159ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101380 td 0xfffff806994416e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f15a3a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f15a3a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f15a3ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f15a3ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f15a3b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f15a3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f15a3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f15a3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101381 td 0xfffff806994d6000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f15a8a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f15a8a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f15a8ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f15a8ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f15a8b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f15a8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f15a8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f15a8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101382 td 0xfffff8081da3f6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f15ada30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f15ada70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f15adab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f15adae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f15adb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f15adbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f15adbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f15adbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101383 td 0xfffff8081da3f000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f15b2a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f15b2a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f15b2ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f15b2ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f15b2b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f15b2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f15b2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f15b2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101384 td 0xfffff803739b06e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f15b7a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f15b7a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f15b7ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f15b7ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f15b7b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f15b7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f15b7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f15b7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101385 td 0xfffff803739b0000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f15bca30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f15bca70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f15bcab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f15bcae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f15bcb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f15bcbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f15bcbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f15bcbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101386 td 0xfffff803ec4636e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f15c1a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f15c1a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f15c1ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f15c1ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f15c1b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f15c1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f15c1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f15c1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101388 td 0xfffff803ec463000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f15c6a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f15c6a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f15c6ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f15c6ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f15c6b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f15c6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f15c6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f15c6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101389 td 0xfffff8034c4fa6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f15cba30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f15cba70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f15cbab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f15cbae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f15cbb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f15cbbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f15cbbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f15cbbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101390 td 0xfffff8034c4fa000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f15d0a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f15d0a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f15d0ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f15d0ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f15d0b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f15d0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f15d0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f15d0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101391 td 0xfffff806994d66e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f15d5a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f15d5a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f15d5ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f15d5ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f15d5b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f15d5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f15d5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f15d5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101392 td 0xfffff8081d3af000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f15daa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f15daa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f15daab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f15daae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f15dab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f15dabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f15dabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f15dabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101393 td 0xfffff806994cc6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f15dfa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f15dfa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f15dfab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f15dfae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f15dfb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f15dfbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f15dfbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f15dfbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101394 td 0xfffff806994cc000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f15e4a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f15e4a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f15e4ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f15e4ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f15e4b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f15e4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f15e4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f15e4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101395 td 0xfffff8081d35f6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f15e9a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f15e9a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f15e9ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f15e9ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f15e9b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f15e9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f15e9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f15e9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101397 td 0xfffff8081d35f000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f15eea30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f15eea70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f15eeab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f15eeae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f15eeb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f15eebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f15eebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f15eebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101399 td 0xfffff8054f9376e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f15f3a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f15f3a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f15f3ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f15f3ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f15f3b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f15f3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f15f3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f15f3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101401 td 0xfffff8054f937000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f15f8a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f15f8a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f15f8ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f15f8ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f15f8b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f15f8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f15f8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f15f8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101403 td 0xfffff8063d4986e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f15fda30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f15fda70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f15fdab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f15fdae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f15fdb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f15fdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f15fdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f15fdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101404 td 0xfffff8063d498000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1604a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1604a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1604ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1604ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1604b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1604bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1604bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1604bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101406 td 0xfffff8081d3af6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1609a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1609a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1609ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1609ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1609b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1609bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1609bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1609bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101408 td 0xfffff8063d4fb000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f160ea30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f160ea70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f160eab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f160eae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f160eb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f160ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f160ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f160ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101409 td 0xfffff8081dfd16e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1613a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1613a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1613ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1613ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1613b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1613bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1613bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1613bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101414 td 0xfffff8081dfd1000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1618a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1618a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1618ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1618ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1618b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1618bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1618bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1618bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101415 td 0xfffff8034c4206e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f161da30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f161da70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f161dab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f161dae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f161db80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f161dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f161dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f161dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101416 td 0xfffff8034c420000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1622a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1622a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1622ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1622ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1622b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1622bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1622bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1622bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101417 td 0xfffff8011d2846e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1627a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1627a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1627ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1627ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1627b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1627bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1627bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1627bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101421 td 0xfffff8011d284000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f162ca30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f162ca70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f162cab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f162cae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f162cb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f162cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f162cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f162cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101424 td 0xfffff8048cdd36e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1631a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1631a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1631ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1631ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1631b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1631bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1631bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1631bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101429 td 0xfffff8048cdd3000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1636a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1636a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1636ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1636ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1636b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1636bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1636bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1636bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101430 td 0xfffff8063d4fb6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f163ba30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f163ba70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f163bab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f163bae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f163bb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f163bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f163bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f163bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101431 td 0xfffff80071e83000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1640a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1640a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1640ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1640ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1640b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1640bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1640bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1640bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101433 td 0xfffff8054f9666e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1645a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1645a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1645ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1645ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1645b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1645bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1645bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1645bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101437 td 0xfffff8054f966000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f164aa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f164aa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f164aab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f164aae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f164ab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f164abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f164abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f164abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101439 td 0xfffff8037fa726e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f164fa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f164fa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f164fab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f164fae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f164fb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f164fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f164fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f164fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101444 td 0xfffff8037fa72000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1654a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1654a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1654ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1654ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1654b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1654bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1654bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1654bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101445 td 0xfffff8081dd046e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1659a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1659a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1659ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1659ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1659b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1659bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1659bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1659bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101446 td 0xfffff8081dd04000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f165ea30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f165ea70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f165eab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f165eae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f165eb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f165ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f165ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f165ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101449 td 0xfffff8069941f6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1663a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1663a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1663ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1663ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1663b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1663bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1663bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1663bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101452 td 0xfffff8069941f000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1668a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1668a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1668ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1668ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1668b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1668bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1668bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1668bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101453 td 0xfffff80071e836e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f166da30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f166da70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f166dab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f166dae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f166db80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f166dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f166dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f166dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101454 td 0xfffff806bd946000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1672a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1672a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1672ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1672ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1672b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1672bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1672bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1672bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101456 td 0xfffff804e8a806e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1677a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1677a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1677ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1677ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1677b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1677bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1677bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1677bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101464 td 0xfffff804e8a80000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f167ca30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f167ca70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f167cab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f167cae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f167cb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f167cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f167cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f167cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101465 td 0xfffff806994e46e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1681a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1681a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1681ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1681ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1681b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1681bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1681bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1681bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101467 td 0xfffff806994e4000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1686a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1686a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1686ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1686ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1686b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1686bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1686bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1686bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101468 td 0xfffff8063d4f26e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f168ba30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f168ba70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f168bab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f168bae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f168bb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f168bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f168bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f168bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101470 td 0xfffff8063d4f2000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1690a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1690a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1690ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1690ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1690b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1690bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1690bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1690bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101471 td 0xfffff807236706e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1695a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1695a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1695ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1695ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1695b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1695bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1695bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1695bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101472 td 0xfffff80723670000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f169aa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f169aa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f169aab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f169aae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f169ab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f169abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f169abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f169abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101475 td 0xfffff806bd9466e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f169fa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f169fa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f169fab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f169fae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f169fb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f169fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f169fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f169fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101479 td 0xfffff8063d4cf000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f16a4a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f16a4a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f16a4ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f16a4ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f16a4b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f16a4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f16a4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f16a4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101480 td 0xfffff80071e696e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f16a9a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f16a9a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f16a9ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f16a9ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f16a9b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f16a9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f16a9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f16a9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101483 td 0xfffff80071e69000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f16aea30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f16aea70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f16aeab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f16aeae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f16aeb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f16aebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f16aebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f16aebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101484 td 0xfffff806a79336e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f16b3a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f16b3a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f16b3ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f16b3ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f16b3b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f16b3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f16b3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f16b3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101486 td 0xfffff806a7933000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f16b8a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f16b8a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f16b8ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f16b8ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f16b8b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f16b8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f16b8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f16b8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101489 td 0xfffff802836f86e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f16bda30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f16bda70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f16bdab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f16bdae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f16bdb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f16bdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f16bdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f16bdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101491 td 0xfffff802836f8000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f16c2a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f16c2a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f16c2ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f16c2ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f16c2b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f16c2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f16c2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f16c2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101492 td 0xfffff804a7ce66e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f16c7a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f16c7a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f16c7ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f16c7ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f16c7b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f16c7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f16c7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f16c7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101495 td 0xfffff804a7ce6000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f16cca30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f16cca70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f16ccab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f16ccae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f16ccb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f16ccbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f16ccbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f16ccbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101565 td 0xfffff804e8ac06e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f17629a0 mi_switch() at mi_switch+0x182/frame 0xfffffe00f17629e0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1762a20 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1762a50 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00f1762ab0 txg_quiesce() at txg_quiesce+0x1db/frame 0xfffffe00f1762b10 txg_quiesce_thread() at txg_quiesce_thread+0x1f1/frame 0xfffffe00f1762bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1762bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1762bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zpool-stress2_tank pid 19734 tid 101566 td 0xfffff801e1b53000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f176b970 mi_switch() at mi_switch+0x182/frame 0xfffffe00f176b9b0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f176b9f0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f176ba20 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00f176ba80 txg_thread_wait() at txg_thread_wait+0x99/frame 0xfffffe00f176bac0 txg_sync_thread() at txg_sync_thread+0x1df/frame 0xfffffe00f176bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f176bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f176bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command md11 pid 19688 tid 102523 td 0xfffff802c30506e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e7898a00 mi_switch() at mi_switch+0x182/frame 0xfffffe00e7898a40 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e7898a80 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e7898ab0 _sleep() at _sleep+0x2cd/frame 0xfffffe00e7898b50 md_kthread() at md_kthread+0xe5/frame 0xfffffe00e7898bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e7898bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e7898bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command md10 pid 19681 tid 102214 td 0xfffff80190c4e6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e828ba00 mi_switch() at mi_switch+0x182/frame 0xfffffe00e828ba40 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e828ba80 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e828bab0 _sleep() at _sleep+0x2cd/frame 0xfffffe00e828bb50 md_kthread() at md_kthread+0xe5/frame 0xfffffe00e828bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e828bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e828bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 102267 td 0xfffff8019606f6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e8394a40 mi_switch() at mi_switch+0x182/frame 0xfffffe00e8394a80 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e8394ac0 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00e8394b00 _cv_timedwait_sbt() at _cv_timedwait_sbt+0x184/frame 0xfffffe00e8394b70 zthr_procedure() at zthr_procedure+0x154/frame 0xfffffe00e8394bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e8394bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e8394bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 100932 td 0xfffff805606526e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f03a5a40 mi_switch() at mi_switch+0x182/frame 0xfffffe00f03a5a80 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f03a5ac0 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00f03a5b00 _cv_timedwait_sbt() at _cv_timedwait_sbt+0x184/frame 0xfffffe00f03a5b70 zthr_procedure() at zthr_procedure+0x154/frame 0xfffffe00f03a5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f03a5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f03a5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 100934 td 0xfffff802dabdb000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f03aaa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f03aaa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f03aaab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f03aaae0 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00f03aab40 arc_dnlc_evicts_thread() at arc_dnlc_evicts_thread+0x140/frame 0xfffffe00f03aabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f03aabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f03aabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 100935 td 0xfffff803fa7236e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f03afa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f03afa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f03afab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f03afae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f03afb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f03afbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f03afbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f03afbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 100936 td 0xfffff803fa723000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f03b4a10 mi_switch() at mi_switch+0x182/frame 0xfffffe00f03b4a50 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f03b4a90 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00f03b4ad0 _cv_timedwait_sbt() at _cv_timedwait_sbt+0x184/frame 0xfffffe00f03b4b40 dbuf_evict_thread() at dbuf_evict_thread+0x296/frame 0xfffffe00f03b4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f03b4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f03b4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 100940 td 0xfffff807d755b6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f03b9a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f03b9a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f03b9ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f03b9ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f03b9b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f03b9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f03b9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f03b9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 100941 td 0xfffff807d755b000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f03bea30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f03bea70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f03beab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f03beae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f03beb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f03bebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f03bebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f03bebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 100945 td 0xfffff80658a8e6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f03c3a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f03c3a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f03c3ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f03c3ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f03c3b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f03c3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f03c3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f03c3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 100946 td 0xfffff80658a8e000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f03c8a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f03c8a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f03c8ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f03c8ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f03c8b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f03c8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f03c8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f03c8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 100950 td 0xfffff806994626e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f03cda30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f03cda70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f03cdab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f03cdae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f03cdb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f03cdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f03cdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f03cdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 100953 td 0xfffff80699462000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f03d2a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f03d2a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f03d2ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f03d2ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f03d2b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f03d2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f03d2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f03d2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 100957 td 0xfffff802dabdb6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f03d7a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f03d7a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f03d7ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f03d7ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f03d7b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f03d7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f03d7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f03d7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 100958 td 0xfffff8034c42c000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c04a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c04a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c04ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c04ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c04b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c04bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c04bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c04bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 100961 td 0xfffff806bd97a6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c09a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c09a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c09ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c09ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c09b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c09bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c09bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c09bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 100965 td 0xfffff806bd97a000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c0ea30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c0ea70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c0eab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c0eae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c0eb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c0ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c0ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c0ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 100966 td 0xfffff8063d4a96e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c13a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c13a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c13ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c13ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c13b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c13bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c13bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c13bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 100970 td 0xfffff8063d4a9000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c18a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c18a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c18ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c18ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c18b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c18bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c18bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c18bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 100979 td 0xfffff8037fab46e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c1da30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c1da70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c1dab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c1dae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c1db80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c1dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c1dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c1dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 100980 td 0xfffff8037fab4000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c22a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c22a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c22ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c22ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c22b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c22bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c22bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c22bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 100981 td 0xfffff803177cb6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c27a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c27a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c27ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c27ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c27b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c27bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c27bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c27bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 100983 td 0xfffff803177cb000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c2ca30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c2ca70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c2cab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c2cae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c2cb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c2cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c2cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c2cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 100986 td 0xfffff8034c42c6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c31950 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c31990 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c319d0 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00f4c31a10 _cv_timedwait_sbt() at _cv_timedwait_sbt+0x184/frame 0xfffffe00f4c31a80 l2arc_feed_thread() at l2arc_feed_thread+0x214/frame 0xfffffe00f4c31bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c31bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c31bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 100993 td 0xfffff803bdcd6000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f4c36a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f4c36a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f4c36ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f4c36ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f4c36b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f4c36bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f4c36bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f4c36bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101500 td 0xfffff8063d4a4000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f16d1a40 mi_switch() at mi_switch+0x182/frame 0xfffffe00f16d1a80 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f16d1ac0 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00f16d1b00 _cv_timedwait_sbt() at _cv_timedwait_sbt+0x184/frame 0xfffffe00f16d1b70 trim_thread() at trim_thread+0xa8/frame 0xfffffe00f16d1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f16d1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f16d1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101502 td 0xfffff804e8aca6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f16d6a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f16d6a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f16d6ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f16d6ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f16d6b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f16d6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f16d6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f16d6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101503 td 0xfffff804e8aca000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f16dba30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f16dba70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f16dbab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f16dbae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f16dbb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f16dbbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f16dbbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f16dbbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101507 td 0xfffff8081d39d6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f16e0a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f16e0a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f16e0ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f16e0ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f16e0b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f16e0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f16e0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f16e0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101508 td 0xfffff8081d39d000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f16e5a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f16e5a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f16e5ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f16e5ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f16e5b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f16e5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f16e5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f16e5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101509 td 0xfffff8020ff1a6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f16eaa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f16eaa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f16eaab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f16eaae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f16eab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f16eabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f16eabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f16eabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101510 td 0xfffff8020ff1a000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f16efa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f16efa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f16efab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f16efae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f16efb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f16efbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f16efbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f16efbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101514 td 0xfffff8011d2376e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f16f4a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f16f4a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f16f4ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f16f4ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f16f4b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f16f4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f16f4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f16f4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101519 td 0xfffff8011d237000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f16f9a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f16f9a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f16f9ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f16f9ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f16f9b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f16f9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f16f9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f16f9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101520 td 0xfffff8063d4a46e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f16fea30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f16fea70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f16feab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f16feae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f16feb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f16febb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f16febf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f16febf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101524 td 0xfffff80582595000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1703a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1703a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1703ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1703ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1703b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1703bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1703bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1703bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101525 td 0xfffff8081d3ae6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1708a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1708a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1708ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1708ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1708b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1708bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1708bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1708bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101529 td 0xfffff8081d3ae000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f170da30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f170da70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f170dab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f170dae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f170db80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f170dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f170dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f170dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101533 td 0xfffff8034c4bd6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1712a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1712a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1712ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1712ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1712b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1712bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1712bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1712bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101534 td 0xfffff8034c4bd000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1717a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1717a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1717ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1717ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1717b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1717bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1717bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1717bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101538 td 0xfffff8063d4f16e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f171ca30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f171ca70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f171cab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f171cae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f171cb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f171cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f171cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f171cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101541 td 0xfffff8063d4f1000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1721a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1721a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1721ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1721ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1721b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1721bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1721bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1721bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101542 td 0xfffff802dacc56e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1726a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1726a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1726ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1726ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1726b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1726bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1726bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1726bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101543 td 0xfffff802dacc5000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f172ba30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f172ba70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f172bab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f172bae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f172bb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f172bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f172bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f172bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101544 td 0xfffff805825956e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1730a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1730a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1730ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1730ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1730b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1730bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1730bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1730bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101545 td 0xfffff804e8ac0000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1735a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1735a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1735ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1735ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1735b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1735bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1735bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1735bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101546 td 0xfffff80071e466e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f173aa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f173aa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f173aab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f173aae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f173ab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f173abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f173abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f173abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101552 td 0xfffff80071e46000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f173fa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f173fa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f173fab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f173fae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f173fb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f173fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f173fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f173fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101554 td 0xfffff804e8a6b6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1744a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1744a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1744ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1744ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1744b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1744bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1744bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1744bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101557 td 0xfffff804e8a6b000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1749a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1749a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1749ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1749ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1749b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1749bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1749bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1749bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101558 td 0xfffff804e8adc6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f174ea30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f174ea70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f174eab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f174eae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f174eb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f174ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f174ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f174ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101560 td 0xfffff804e8adc000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1753a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1753a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1753ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1753ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1753b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1753bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1753bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1753bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101561 td 0xfffff8081dd0f6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1758a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1758a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1758ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1758ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f1758b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f1758bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1758bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1758bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101562 td 0xfffff8081dd0f000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f175da30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f175da70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f175dab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f175dae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f175db80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f175dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f175dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f175dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101570 td 0xfffff80071ea56e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1770900 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1770940 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1770980 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00f17709e0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00f1770a20 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x184/frame 0xfffffe00f1770a90 mmp_thread() at mmp_thread+0xccf/frame 0xfffffe00f1770bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1770bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1770bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101571 td 0xfffff80071ea5000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f1775a60 mi_switch() at mi_switch+0x182/frame 0xfffffe00f1775aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f1775ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f1775b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00f1775b70 zthr_procedure() at zthr_procedure+0x120/frame 0xfffffe00f1775bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1775bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1775bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command zfskern pid 19655 tid 101574 td 0xfffff8037fa1f6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f177aa60 mi_switch() at mi_switch+0x182/frame 0xfffffe00f177aaa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f177aae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f177ab10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00f177ab70 zthr_procedure() at zthr_procedure+0x120/frame 0xfffffe00f177abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f177abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f177abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sh pid 19629 tid 102125 td 0xfffff80455d06000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00ef8ca690 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00eb5746f0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00eb574700 trap() at trap+0x73/frame 0xfffffe00eb574830 calltrap() at calltrap+0x8/frame 0xfffffe00eb574830 --- trap 0xc, rip = 0xffffffff8103e0a0, rsp = 0xfffffe00eb574900, rbp = 0xfffffe00eb574990 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe00eb574990 db_get_value() at db_get_value+0x32/frame 0xfffffe00eb5749d0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00eb574a60 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00eb574a80 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00eb574b20 db_command() at db_command+0x25c/frame 0xfffffe00eb574bf0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00eb574c90 db_command() at db_command+0x25c/frame 0xfffffe00eb574d60 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00eb574d70 db_trap() at db_trap+0xe8/frame 0xfffffe00eb574e00 kdb_trap() at kdb_trap+0x157/frame 0xfffffe00eb574e80 trap() at trap+0x7d4/frame 0xfffffe00eb574fb0 calltrap() at calltrap+0x8/frame 0xfffffe00eb574fb0 --- trap 0x3, rip = 0xffffffff80c105a1, rsp = 0xfffffe00eb575080, rbp = 0xfffffe00eb5750a0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe00eb5750a0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00eb5750b0 uart_intr_rxready() at uart_intr_rxready+0xa5/frame 0xfffffe00eb5750e0 uart_intr() at uart_intr+0x138/frame 0xfffffe00eb575120 intr_event_handle() at intr_event_handle+0xce/frame 0xfffffe00eb575170 intr_execute_handlers() at intr_execute_handlers+0x75/frame 0xfffffe00eb5751b0 lapic_handle_intr() at lapic_handle_intr+0x5b/frame 0xfffffe00eb5751d0 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00eb5751d0 --- interrupt, rip = 0xffffffff80c11a92, rsp = 0xfffffe00eb5752a0, rbp = 0xfffffe00eb5752b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00eb5752b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00eb575370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00eb5753b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00eb5753f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00eb5754d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5755c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb575700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5757d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb575800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb575950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb575ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb575bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb575bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- *** error reading from address fffffe00ef8ca698 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00eb574980 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00eb574990 db_get_value() at db_get_value+0x4c/frame 0xfffffe00eb5749d0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00eb574a60 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00eb574a80 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00eb574b20 db_command() at db_command+0x25c/frame 0xfffffe00eb574bf0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00eb574c90 db_command() at db_command+0x25c/frame 0xfffffe00eb574d60 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00eb574d70 db_trap() at db_trap+0xe8/frame 0xfffffe00eb574e00 kdb_trap() at kdb_trap+0x157/frame 0xfffffe00eb574e80 trap() at trap+0x7d4/frame 0xfffffe00eb574fb0 calltrap() at calltrap+0x8/frame 0xfffffe00eb574fb0 --- trap 0x3, rip = 0xffffffff80c105a1, rsp = 0xfffffe00eb575080, rbp = 0xfffffe00eb5750a0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe00eb5750a0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00eb5750b0 uart_intr_rxready() at uart_intr_rxready+0xa5/frame 0xfffffe00eb5750e0 uart_intr() at uart_intr+0x138/frame 0xfffffe00eb575120 intr_event_handle() at intr_event_handle+0xce/frame 0xfffffe00eb575170 intr_execute_handlers() at intr_execute_handlers+0x75/frame 0xfffffe00eb5751b0 lapic_handle_intr() at lapic_handle_intr+0x5b/frame 0xfffffe00eb5751d0 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00eb5751d0 --- interrupt, rip = 0xffffffff80c11a92, rsp = 0xfffffe00eb5752a0, rbp = 0xfffffe00eb5752b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00eb5752b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00eb575370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00eb5753b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00eb5753f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00eb5754d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5755c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb575700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5757d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb575800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb575950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb575ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb575bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb575bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command tee pid 19628 tid 102698 td 0xfffff80394ba76e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00efc097d0 mi_switch() at mi_switch+0x182/frame 0xfffffe00efc09810 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00efc09850 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00efc098b0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00efc098e0 _sleep() at _sleep+0x2c3/frame 0xfffffe00efc09980 pipe_read() at pipe_read+0x428/frame 0xfffffe00efc099e0 dofileread() at dofileread+0x81/frame 0xfffffe00efc09a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00efc09a70 sys_read() at sys_read+0x83/frame 0xfffffe00efc09ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00efc09bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00efc09bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800417b1a, rsp = 0x7fffffffe6a8, rbp = 0x7fffffffe6f0 --- Tracing command sh pid 19627 tid 102247 td 0xfffff80278c176e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e8330690 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00eb5746f0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00eb574700 trap() at trap+0x73/frame 0xfffffe00eb574830 calltrap() at calltrap+0x8/frame 0xfffffe00eb574830 --- trap 0xc, rip = 0xffffffff8103e0a0, rsp = 0xfffffe00eb574900, rbp = 0xfffffe00eb574990 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe00eb574990 db_get_value() at db_get_value+0x32/frame 0xfffffe00eb5749d0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00eb574a60 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00eb574a80 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00eb574b20 db_command() at db_command+0x25c/frame 0xfffffe00eb574bf0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00eb574c90 db_command() at db_command+0x25c/frame 0xfffffe00eb574d60 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00eb574d70 db_trap() at db_trap+0xe8/frame 0xfffffe00eb574e00 kdb_trap() at kdb_trap+0x157/frame 0xfffffe00eb574e80 trap() at trap+0x7d4/frame 0xfffffe00eb574fb0 calltrap() at calltrap+0x8/frame 0xfffffe00eb574fb0 --- trap 0x3, rip = 0xffffffff80c105a1, rsp = 0xfffffe00eb575080, rbp = 0xfffffe00eb5750a0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe00eb5750a0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00eb5750b0 uart_intr_rxready() at uart_intr_rxready+0xa5/frame 0xfffffe00eb5750e0 uart_intr() at uart_intr+0x138/frame 0xfffffe00eb575120 intr_event_handle() at intr_event_handle+0xce/frame 0xfffffe00eb575170 intr_execute_handlers() at intr_execute_handlers+0x75/frame 0xfffffe00eb5751b0 lapic_handle_intr() at lapic_handle_intr+0x5b/frame 0xfffffe00eb5751d0 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00eb5751d0 --- interrupt, rip = 0xffffffff80c11a92, rsp = 0xfffffe00eb5752a0, rbp = 0xfffffe00eb5752b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00eb5752b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00eb575370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00eb5753b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00eb5753f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00eb5754d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5755c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb575700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5757d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb575800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb575950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb575ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb575bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb575bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- *** error reading from address fffffe00e8330698 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00eb574980 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00eb574990 db_get_value() at db_get_value+0x4c/frame 0xfffffe00eb5749d0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00eb574a60 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00eb574a80 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00eb574b20 db_command() at db_command+0x25c/frame 0xfffffe00eb574bf0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00eb574c90 db_command() at db_command+0x25c/frame 0xfffffe00eb574d60 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00eb574d70 db_trap() at db_trap+0xe8/frame 0xfffffe00eb574e00 kdb_trap() at kdb_trap+0x157/frame 0xfffffe00eb574e80 trap() at trap+0x7d4/frame 0xfffffe00eb574fb0 calltrap() at calltrap+0x8/frame 0xfffffe00eb574fb0 --- trap 0x3, rip = 0xffffffff80c105a1, rsp = 0xfffffe00eb575080, rbp = 0xfffffe00eb5750a0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe00eb5750a0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00eb5750b0 uart_intr_rxready() at uart_intr_rxready+0xa5/frame 0xfffffe00eb5750e0 uart_intr() at uart_intr+0x138/frame 0xfffffe00eb575120 intr_event_handle() at intr_event_handle+0xce/frame 0xfffffe00eb575170 intr_execute_handlers() at intr_execute_handlers+0x75/frame 0xfffffe00eb5751b0 lapic_handle_intr() at lapic_handle_intr+0x5b/frame 0xfffffe00eb5751d0 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00eb5751d0 --- interrupt, rip = 0xffffffff80c11a92, rsp = 0xfffffe00eb5752a0, rbp = 0xfffffe00eb5752b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00eb5752b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00eb575370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00eb5753b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00eb5753f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00eb5754d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5755c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb575700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5757d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb575800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb575950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb575ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb575bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb575bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command bash pid 124 tid 102534 td 0xfffff8073be1b000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e78cf6b0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e78cf6f0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e78cf730 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e78cf790 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e78cf7c0 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e78cf820 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e78cf860 kern_select() at kern_select+0x962/frame 0xfffffe00e78cfa40 sys_pselect() at sys_pselect+0x127/frame 0xfffffe00e78cfac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e78cfbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e78cfbf0 --- syscall (522, FreeBSD ELF64, sys_pselect), rip = 0x800517a3a, rsp = 0x7fffffffd638, rbp = 0x7fffffffd700 --- Tracing command csh pid 99828 tid 102816 td 0xfffff8033f842000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00efe59880 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00eb5746f0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00eb574700 trap() at trap+0x73/frame 0xfffffe00eb574830 calltrap() at calltrap+0x8/frame 0xfffffe00eb574830 --- trap 0xc, rip = 0xffffffff8103e0a0, rsp = 0xfffffe00eb574900, rbp = 0xfffffe00eb574990 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe00eb574990 db_get_value() at db_get_value+0x32/frame 0xfffffe00eb5749d0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00eb574a60 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00eb574a80 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00eb574b20 db_command() at db_command+0x25c/frame 0xfffffe00eb574bf0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00eb574c90 db_command() at db_command+0x25c/frame 0xfffffe00eb574d60 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00eb574d70 db_trap() at db_trap+0xe8/frame 0xfffffe00eb574e00 kdb_trap() at kdb_trap+0x157/frame 0xfffffe00eb574e80 trap() at trap+0x7d4/frame 0xfffffe00eb574fb0 calltrap() at calltrap+0x8/frame 0xfffffe00eb574fb0 --- trap 0x3, rip = 0xffffffff80c105a1, rsp = 0xfffffe00eb575080, rbp = 0xfffffe00eb5750a0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe00eb5750a0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00eb5750b0 uart_intr_rxready() at uart_intr_rxready+0xa5/frame 0xfffffe00eb5750e0 uart_intr() at uart_intr+0x138/frame 0xfffffe00eb575120 intr_event_handle() at intr_event_handle+0xce/frame 0xfffffe00eb575170 intr_execute_handlers() at intr_execute_handlers+0x75/frame 0xfffffe00eb5751b0 lapic_handle_intr() at lapic_handle_intr+0x5b/frame 0xfffffe00eb5751d0 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00eb5751d0 --- interrupt, rip = 0xffffffff80c11a92, rsp = 0xfffffe00eb5752a0, rbp = 0xfffffe00eb5752b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00eb5752b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00eb575370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00eb5753b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00eb5753f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00eb5754d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5755c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb575700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5757d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb575800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb575950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb575ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb575bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb575bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- *** error reading from address fffffe00efe59888 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00eb574980 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00eb574990 db_get_value() at db_get_value+0x4c/frame 0xfffffe00eb5749d0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00eb574a60 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00eb574a80 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00eb574b20 db_command() at db_command+0x25c/frame 0xfffffe00eb574bf0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00eb574c90 db_command() at db_command+0x25c/frame 0xfffffe00eb574d60 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00eb574d70 db_trap() at db_trap+0xe8/frame 0xfffffe00eb574e00 kdb_trap() at kdb_trap+0x157/frame 0xfffffe00eb574e80 trap() at trap+0x7d4/frame 0xfffffe00eb574fb0 calltrap() at calltrap+0x8/frame 0xfffffe00eb574fb0 --- trap 0x3, rip = 0xffffffff80c105a1, rsp = 0xfffffe00eb575080, rbp = 0xfffffe00eb5750a0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe00eb5750a0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00eb5750b0 uart_intr_rxready() at uart_intr_rxready+0xa5/frame 0xfffffe00eb5750e0 uart_intr() at uart_intr+0x138/frame 0xfffffe00eb575120 intr_event_handle() at intr_event_handle+0xce/frame 0xfffffe00eb575170 intr_execute_handlers() at intr_execute_handlers+0x75/frame 0xfffffe00eb5751b0 lapic_handle_intr() at lapic_handle_intr+0x5b/frame 0xfffffe00eb5751d0 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00eb5751d0 --- interrupt, rip = 0xffffffff80c11a92, rsp = 0xfffffe00eb5752a0, rbp = 0xfffffe00eb5752b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00eb5752b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00eb575370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00eb5753b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00eb5753f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00eb5754d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5755c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb575700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5757d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb575800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb575950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb575ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb575bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb575bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command su pid 99824 tid 102762 td 0xfffff8015cf086e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00efd49690 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00eb5746f0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00eb574700 trap() at trap+0x73/frame 0xfffffe00eb574830 calltrap() at calltrap+0x8/frame 0xfffffe00eb574830 --- trap 0xc, rip = 0xffffffff8103e0a0, rsp = 0xfffffe00eb574900, rbp = 0xfffffe00eb574990 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe00eb574990 db_get_value() at db_get_value+0x32/frame 0xfffffe00eb5749d0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00eb574a60 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00eb574a80 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00eb574b20 db_command() at db_command+0x25c/frame 0xfffffe00eb574bf0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00eb574c90 db_command() at db_command+0x25c/frame 0xfffffe00eb574d60 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00eb574d70 db_trap() at db_trap+0xe8/frame 0xfffffe00eb574e00 kdb_trap() at kdb_trap+0x157/frame 0xfffffe00eb574e80 trap() at trap+0x7d4/frame 0xfffffe00eb574fb0 calltrap() at calltrap+0x8/frame 0xfffffe00eb574fb0 --- trap 0x3, rip = 0xffffffff80c105a1, rsp = 0xfffffe00eb575080, rbp = 0xfffffe00eb5750a0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe00eb5750a0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00eb5750b0 uart_intr_rxready() at uart_intr_rxready+0xa5/frame 0xfffffe00eb5750e0 uart_intr() at uart_intr+0x138/frame 0xfffffe00eb575120 intr_event_handle() at intr_event_handle+0xce/frame 0xfffffe00eb575170 intr_execute_handlers() at intr_execute_handlers+0x75/frame 0xfffffe00eb5751b0 lapic_handle_intr() at lapic_handle_intr+0x5b/frame 0xfffffe00eb5751d0 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00eb5751d0 --- interrupt, rip = 0xffffffff80c11a92, rsp = 0xfffffe00eb5752a0, rbp = 0xfffffe00eb5752b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00eb5752b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00eb575370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00eb5753b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00eb5753f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00eb5754d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5755c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb575700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5757d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb575800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb575950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb575ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb575bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb575bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- *** error reading from address fffffe00efd49698 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00eb574980 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00eb574990 db_get_value() at db_get_value+0x4c/frame 0xfffffe00eb5749d0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00eb574a60 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00eb574a80 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00eb574b20 db_command() at db_command+0x25c/frame 0xfffffe00eb574bf0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00eb574c90 db_command() at db_command+0x25c/frame 0xfffffe00eb574d60 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00eb574d70 db_trap() at db_trap+0xe8/frame 0xfffffe00eb574e00 kdb_trap() at kdb_trap+0x157/frame 0xfffffe00eb574e80 trap() at trap+0x7d4/frame 0xfffffe00eb574fb0 calltrap() at calltrap+0x8/frame 0xfffffe00eb574fb0 --- trap 0x3, rip = 0xffffffff80c105a1, rsp = 0xfffffe00eb575080, rbp = 0xfffffe00eb5750a0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe00eb5750a0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00eb5750b0 uart_intr_rxready() at uart_intr_rxready+0xa5/frame 0xfffffe00eb5750e0 uart_intr() at uart_intr+0x138/frame 0xfffffe00eb575120 intr_event_handle() at intr_event_handle+0xce/frame 0xfffffe00eb575170 intr_execute_handlers() at intr_execute_handlers+0x75/frame 0xfffffe00eb5751b0 lapic_handle_intr() at lapic_handle_intr+0x5b/frame 0xfffffe00eb5751d0 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00eb5751d0 --- interrupt, rip = 0xffffffff80c11a92, rsp = 0xfffffe00eb5752a0, rbp = 0xfffffe00eb5752b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00eb5752b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00eb575370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00eb5753b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00eb5753f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00eb5754d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5755c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb575700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5757d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb575800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb575950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb575ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb575bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb575bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command bash pid 61272 tid 100193 td 0xfffff80030ed66e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2ae7690 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00eb5746f0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00eb574700 trap() at trap+0x73/frame 0xfffffe00eb574830 calltrap() at calltrap+0x8/frame 0xfffffe00eb574830 --- trap 0xc, rip = 0xffffffff8103e0a0, rsp = 0xfffffe00eb574900, rbp = 0xfffffe00eb574990 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe00eb574990 db_get_value() at db_get_value+0x32/frame 0xfffffe00eb5749d0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00eb574a60 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00eb574a80 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00eb574b20 db_command() at db_command+0x25c/frame 0xfffffe00eb574bf0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00eb574c90 db_command() at db_command+0x25c/frame 0xfffffe00eb574d60 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00eb574d70 db_trap() at db_trap+0xe8/frame 0xfffffe00eb574e00 kdb_trap() at kdb_trap+0x157/frame 0xfffffe00eb574e80 trap() at trap+0x7d4/frame 0xfffffe00eb574fb0 calltrap() at calltrap+0x8/frame 0xfffffe00eb574fb0 --- trap 0x3, rip = 0xffffffff80c105a1, rsp = 0xfffffe00eb575080, rbp = 0xfffffe00eb5750a0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe00eb5750a0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00eb5750b0 uart_intr_rxready() at uart_intr_rxready+0xa5/frame 0xfffffe00eb5750e0 uart_intr() at uart_intr+0x138/frame 0xfffffe00eb575120 intr_event_handle() at intr_event_handle+0xce/frame 0xfffffe00eb575170 intr_execute_handlers() at intr_execute_handlers+0x75/frame 0xfffffe00eb5751b0 lapic_handle_intr() at lapic_handle_intr+0x5b/frame 0xfffffe00eb5751d0 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00eb5751d0 --- interrupt, rip = 0xffffffff80c11a92, rsp = 0xfffffe00eb5752a0, rbp = 0xfffffe00eb5752b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00eb5752b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00eb575370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00eb5753b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00eb5753f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00eb5754d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5755c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb575700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5757d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb575800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb575950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb575ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb575bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb575bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- *** error reading from address fffffe00e2ae7698 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00eb574980 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00eb574990 db_get_value() at db_get_value+0x4c/frame 0xfffffe00eb5749d0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00eb574a60 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00eb574a80 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00eb574b20 db_command() at db_command+0x25c/frame 0xfffffe00eb574bf0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00eb574c90 db_command() at db_command+0x25c/frame 0xfffffe00eb574d60 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00eb574d70 db_trap() at db_trap+0xe8/frame 0xfffffe00eb574e00 kdb_trap() at kdb_trap+0x157/frame 0xfffffe00eb574e80 trap() at trap+0x7d4/frame 0xfffffe00eb574fb0 calltrap() at calltrap+0x8/frame 0xfffffe00eb574fb0 --- trap 0x3, rip = 0xffffffff80c105a1, rsp = 0xfffffe00eb575080, rbp = 0xfffffe00eb5750a0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe00eb5750a0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00eb5750b0 uart_intr_rxready() at uart_intr_rxready+0xa5/frame 0xfffffe00eb5750e0 uart_intr() at uart_intr+0x138/frame 0xfffffe00eb575120 intr_event_handle() at intr_event_handle+0xce/frame 0xfffffe00eb575170 intr_execute_handlers() at intr_execute_handlers+0x75/frame 0xfffffe00eb5751b0 lapic_handle_intr() at lapic_handle_intr+0x5b/frame 0xfffffe00eb5751d0 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00eb5751d0 --- interrupt, rip = 0xffffffff80c11a92, rsp = 0xfffffe00eb5752a0, rbp = 0xfffffe00eb5752b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00eb5752b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00eb575370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00eb5753b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00eb5753f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00eb5754d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5755c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb575700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5757d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb575800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb575950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb575ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb575bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb575bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command sshd pid 61259 tid 100197 td 0xfffff800440216e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2b136f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2b13730 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2b13770 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e2b137d0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e2b13800 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e2b13860 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e2b138a0 kern_select() at kern_select+0x962/frame 0xfffffe00e2b13a80 sys_select() at sys_select+0x56/frame 0xfffffe00e2b13ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2b13bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2b13bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8008f5afa, rsp = 0x7fffffffa578, rbp = 0x7fffffffa5b0 --- Tracing command sshd pid 61026 tid 100343 td 0xfffff807b8697000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3b7c770 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3b7c7b0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e3b7c7f0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e3b7c850 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e3b7c880 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e3b7c8e0 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e3b7c920 kern_poll() at kern_poll+0x48b/frame 0xfffffe00e3b7caa0 sys_poll() at sys_poll+0x50/frame 0xfffffe00e3b7cac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e3b7cbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e3b7cbf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x80089913a, rsp = 0x7fffffffe5d8, rbp = 0x7fffffffe610 --- Tracing command sh pid 3000 tid 100248 td 0xfffff804f0b75000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3a1d7d0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3a1d810 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e3a1d850 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e3a1d8b0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e3a1d8e0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e3a1d980 pipe_read() at pipe_read+0x428/frame 0xfffffe00e3a1d9e0 dofileread() at dofileread+0x81/frame 0xfffffe00e3a1da30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e3a1da70 sys_read() at sys_read+0x83/frame 0xfffffe00e3a1dac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e3a1dbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e3a1dbf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800476b1a, rsp = 0x7fffffffe238, rbp = 0x7fffffffe330 --- Tracing command awk pid 2998 tid 100227 td 0xfffff804f0b756e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2bbc7d0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2bbc810 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2bbc850 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e2bbc8b0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e2bbc8e0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e2bbc980 pipe_read() at pipe_read+0x428/frame 0xfffffe00e2bbc9e0 dofileread() at dofileread+0x81/frame 0xfffffe00e2bbca30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e2bbca70 sys_read() at sys_read+0x83/frame 0xfffffe00e2bbcac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2bbcbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2bbcbf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80047bb1a, rsp = 0x7fffffffe318, rbp = 0x7fffffffe330 --- Tracing command sh pid 2997 tid 100237 td 0xfffff80030d53000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2bb7690 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2bb76d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2bb7710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e2bb7770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e2bb77a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e2bb7840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00e2bb78d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00e2bb7ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2bb7bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2bb7bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80040791a, rsp = 0x7fffffffe238, rbp = 0x7fffffffe2f0 --- Tracing command sh pid 2930 tid 100201 td 0xfffff803b95b36e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2b27690 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00eb5746f0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00eb574700 trap() at trap+0x73/frame 0xfffffe00eb574830 calltrap() at calltrap+0x8/frame 0xfffffe00eb574830 --- trap 0xc, rip = 0xffffffff8103e0a0, rsp = 0xfffffe00eb574900, rbp = 0xfffffe00eb574990 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe00eb574990 db_get_value() at db_get_value+0x32/frame 0xfffffe00eb5749d0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00eb574a60 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00eb574a80 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00eb574b20 db_command() at db_command+0x25c/frame 0xfffffe00eb574bf0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00eb574c90 db_command() at db_command+0x25c/frame 0xfffffe00eb574d60 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00eb574d70 db_trap() at db_trap+0xe8/frame 0xfffffe00eb574e00 kdb_trap() at kdb_trap+0x157/frame 0xfffffe00eb574e80 trap() at trap+0x7d4/frame 0xfffffe00eb574fb0 calltrap() at calltrap+0x8/frame 0xfffffe00eb574fb0 --- trap 0x3, rip = 0xffffffff80c105a1, rsp = 0xfffffe00eb575080, rbp = 0xfffffe00eb5750a0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe00eb5750a0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00eb5750b0 uart_intr_rxready() at uart_intr_rxready+0xa5/frame 0xfffffe00eb5750e0 uart_intr() at uart_intr+0x138/frame 0xfffffe00eb575120 intr_event_handle() at intr_event_handle+0xce/frame 0xfffffe00eb575170 intr_execute_handlers() at intr_execute_handlers+0x75/frame 0xfffffe00eb5751b0 lapic_handle_intr() at lapic_handle_intr+0x5b/frame 0xfffffe00eb5751d0 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00eb5751d0 --- interrupt, rip = 0xffffffff80c11a92, rsp = 0xfffffe00eb5752a0, rbp = 0xfffffe00eb5752b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00eb5752b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00eb575370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00eb5753b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00eb5753f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00eb5754d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5755c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb575700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5757d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb575800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb575950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb575ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb575bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb575bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- *** error reading from address fffffe00e2b27698 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00eb574980 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00eb574990 db_get_value() at db_get_value+0x4c/frame 0xfffffe00eb5749d0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00eb574a60 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00eb574a80 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00eb574b20 db_command() at db_command+0x25c/frame 0xfffffe00eb574bf0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00eb574c90 db_command() at db_command+0x25c/frame 0xfffffe00eb574d60 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00eb574d70 db_trap() at db_trap+0xe8/frame 0xfffffe00eb574e00 kdb_trap() at kdb_trap+0x157/frame 0xfffffe00eb574e80 trap() at trap+0x7d4/frame 0xfffffe00eb574fb0 calltrap() at calltrap+0x8/frame 0xfffffe00eb574fb0 --- trap 0x3, rip = 0xffffffff80c105a1, rsp = 0xfffffe00eb575080, rbp = 0xfffffe00eb5750a0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe00eb5750a0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00eb5750b0 uart_intr_rxready() at uart_intr_rxready+0xa5/frame 0xfffffe00eb5750e0 uart_intr() at uart_intr+0x138/frame 0xfffffe00eb575120 intr_event_handle() at intr_event_handle+0xce/frame 0xfffffe00eb575170 intr_execute_handlers() at intr_execute_handlers+0x75/frame 0xfffffe00eb5751b0 lapic_handle_intr() at lapic_handle_intr+0x5b/frame 0xfffffe00eb5751d0 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00eb5751d0 --- interrupt, rip = 0xffffffff80c11a92, rsp = 0xfffffe00eb5752a0, rbp = 0xfffffe00eb5752b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00eb5752b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00eb575370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00eb5753b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00eb5753f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00eb5754d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5755c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb575700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5757d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb575800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb575950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb575ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb575bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb575bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command sh pid 2490 tid 100171 td 0xfffff80030e1f000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2a3d690 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00eb5746f0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00eb574700 trap() at trap+0x73/frame 0xfffffe00eb574830 calltrap() at calltrap+0x8/frame 0xfffffe00eb574830 --- trap 0xc, rip = 0xffffffff8103e0a0, rsp = 0xfffffe00eb574900, rbp = 0xfffffe00eb574990 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe00eb574990 db_get_value() at db_get_value+0x32/frame 0xfffffe00eb5749d0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00eb574a60 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00eb574a80 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00eb574b20 db_command() at db_command+0x25c/frame 0xfffffe00eb574bf0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00eb574c90 db_command() at db_command+0x25c/frame 0xfffffe00eb574d60 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00eb574d70 db_trap() at db_trap+0xe8/frame 0xfffffe00eb574e00 kdb_trap() at kdb_trap+0x157/frame 0xfffffe00eb574e80 trap() at trap+0x7d4/frame 0xfffffe00eb574fb0 calltrap() at calltrap+0x8/frame 0xfffffe00eb574fb0 --- trap 0x3, rip = 0xffffffff80c105a1, rsp = 0xfffffe00eb575080, rbp = 0xfffffe00eb5750a0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe00eb5750a0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00eb5750b0 uart_intr_rxready() at uart_intr_rxready+0xa5/frame 0xfffffe00eb5750e0 uart_intr() at uart_intr+0x138/frame 0xfffffe00eb575120 intr_event_handle() at intr_event_handle+0xce/frame 0xfffffe00eb575170 intr_execute_handlers() at intr_execute_handlers+0x75/frame 0xfffffe00eb5751b0 lapic_handle_intr() at lapic_handle_intr+0x5b/frame 0xfffffe00eb5751d0 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00eb5751d0 --- interrupt, rip = 0xffffffff80c11a92, rsp = 0xfffffe00eb5752a0, rbp = 0xfffffe00eb5752b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00eb5752b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00eb575370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00eb5753b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00eb5753f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00eb5754d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5755c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb575700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5757d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb575800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb575950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb575ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb575bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb575bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- *** error reading from address fffffe00e2a3d698 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00eb574980 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00eb574990 db_get_value() at db_get_value+0x4c/frame 0xfffffe00eb5749d0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00eb574a60 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00eb574a80 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00eb574b20 db_command() at db_command+0x25c/frame 0xfffffe00eb574bf0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00eb574c90 db_command() at db_command+0x25c/frame 0xfffffe00eb574d60 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00eb574d70 db_trap() at db_trap+0xe8/frame 0xfffffe00eb574e00 kdb_trap() at kdb_trap+0x157/frame 0xfffffe00eb574e80 trap() at trap+0x7d4/frame 0xfffffe00eb574fb0 calltrap() at calltrap+0x8/frame 0xfffffe00eb574fb0 --- trap 0x3, rip = 0xffffffff80c105a1, rsp = 0xfffffe00eb575080, rbp = 0xfffffe00eb5750a0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe00eb5750a0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00eb5750b0 uart_intr_rxready() at uart_intr_rxready+0xa5/frame 0xfffffe00eb5750e0 uart_intr() at uart_intr+0x138/frame 0xfffffe00eb575120 intr_event_handle() at intr_event_handle+0xce/frame 0xfffffe00eb575170 intr_execute_handlers() at intr_execute_handlers+0x75/frame 0xfffffe00eb5751b0 lapic_handle_intr() at lapic_handle_intr+0x5b/frame 0xfffffe00eb5751d0 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00eb5751d0 --- interrupt, rip = 0xffffffff80c11a92, rsp = 0xfffffe00eb5752a0, rbp = 0xfffffe00eb5752b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00eb5752b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00eb575370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00eb5753b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00eb5753f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00eb5754d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5755c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb575700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5757d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb575800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb575950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb575ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb575bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb575bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command csh pid 2388 tid 100189 td 0xfffff80030ecc6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2ad3750 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2ad3790 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2ad37d0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e2ad3830 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e2ad3860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e2ad38c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e2ad38f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e2ad3950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e2ad3980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e2ad39e0 dofileread() at dofileread+0x81/frame 0xfffffe00e2ad3a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e2ad3a70 sys_read() at sys_read+0x83/frame 0xfffffe00e2ad3ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2ad3bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2ad3bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004fbb1a, rsp = 0x7fffffffe0e8, rbp = 0x7fffffffe110 --- Tracing command su pid 2387 tid 100219 td 0xfffff804f0b74000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2b8d690 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00eb5746f0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00eb574700 trap() at trap+0x73/frame 0xfffffe00eb574830 calltrap() at calltrap+0x8/frame 0xfffffe00eb574830 --- trap 0xc, rip = 0xffffffff8103e0a0, rsp = 0xfffffe00eb574900, rbp = 0xfffffe00eb574990 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe00eb574990 db_get_value() at db_get_value+0x32/frame 0xfffffe00eb5749d0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00eb574a60 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00eb574a80 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00eb574b20 db_command() at db_command+0x25c/frame 0xfffffe00eb574bf0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00eb574c90 db_command() at db_command+0x25c/frame 0xfffffe00eb574d60 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00eb574d70 db_trap() at db_trap+0xe8/frame 0xfffffe00eb574e00 kdb_trap() at kdb_trap+0x157/frame 0xfffffe00eb574e80 trap() at trap+0x7d4/frame 0xfffffe00eb574fb0 calltrap() at calltrap+0x8/frame 0xfffffe00eb574fb0 --- trap 0x3, rip = 0xffffffff80c105a1, rsp = 0xfffffe00eb575080, rbp = 0xfffffe00eb5750a0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe00eb5750a0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00eb5750b0 uart_intr_rxready() at uart_intr_rxready+0xa5/frame 0xfffffe00eb5750e0 uart_intr() at uart_intr+0x138/frame 0xfffffe00eb575120 intr_event_handle() at intr_event_handle+0xce/frame 0xfffffe00eb575170 intr_execute_handlers() at intr_execute_handlers+0x75/frame 0xfffffe00eb5751b0 lapic_handle_intr() at lapic_handle_intr+0x5b/frame 0xfffffe00eb5751d0 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00eb5751d0 --- interrupt, rip = 0xffffffff80c11a92, rsp = 0xfffffe00eb5752a0, rbp = 0xfffffe00eb5752b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00eb5752b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00eb575370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00eb5753b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00eb5753f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00eb5754d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5755c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb575700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5757d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb575800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb575950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb575ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb575bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb575bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- *** error reading from address fffffe00e2b8d698 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00eb574980 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00eb574990 db_get_value() at db_get_value+0x4c/frame 0xfffffe00eb5749d0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00eb574a60 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00eb574a80 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00eb574b20 db_command() at db_command+0x25c/frame 0xfffffe00eb574bf0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00eb574c90 db_command() at db_command+0x25c/frame 0xfffffe00eb574d60 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00eb574d70 db_trap() at db_trap+0xe8/frame 0xfffffe00eb574e00 kdb_trap() at kdb_trap+0x157/frame 0xfffffe00eb574e80 trap() at trap+0x7d4/frame 0xfffffe00eb574fb0 calltrap() at calltrap+0x8/frame 0xfffffe00eb574fb0 --- trap 0x3, rip = 0xffffffff80c105a1, rsp = 0xfffffe00eb575080, rbp = 0xfffffe00eb5750a0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe00eb5750a0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00eb5750b0 uart_intr_rxready() at uart_intr_rxready+0xa5/frame 0xfffffe00eb5750e0 uart_intr() at uart_intr+0x138/frame 0xfffffe00eb575120 intr_event_handle() at intr_event_handle+0xce/frame 0xfffffe00eb575170 intr_execute_handlers() at intr_execute_handlers+0x75/frame 0xfffffe00eb5751b0 lapic_handle_intr() at lapic_handle_intr+0x5b/frame 0xfffffe00eb5751d0 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00eb5751d0 --- interrupt, rip = 0xffffffff80c11a92, rsp = 0xfffffe00eb5752a0, rbp = 0xfffffe00eb5752b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00eb5752b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00eb575370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00eb5753b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00eb5753f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00eb5754d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5755c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb575700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5757d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb575800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb575950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb575ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb575bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb575bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command bash pid 2382 tid 100220 td 0xfffff804f07356e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2b92690 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00eb5746f0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00eb574700 trap() at trap+0x73/frame 0xfffffe00eb574830 calltrap() at calltrap+0x8/frame 0xfffffe00eb574830 --- trap 0xc, rip = 0xffffffff8103e0a0, rsp = 0xfffffe00eb574900, rbp = 0xfffffe00eb574990 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe00eb574990 db_get_value() at db_get_value+0x32/frame 0xfffffe00eb5749d0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00eb574a60 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00eb574a80 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00eb574b20 db_command() at db_command+0x25c/frame 0xfffffe00eb574bf0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00eb574c90 db_command() at db_command+0x25c/frame 0xfffffe00eb574d60 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00eb574d70 db_trap() at db_trap+0xe8/frame 0xfffffe00eb574e00 kdb_trap() at kdb_trap+0x157/frame 0xfffffe00eb574e80 trap() at trap+0x7d4/frame 0xfffffe00eb574fb0 calltrap() at calltrap+0x8/frame 0xfffffe00eb574fb0 --- trap 0x3, rip = 0xffffffff80c105a1, rsp = 0xfffffe00eb575080, rbp = 0xfffffe00eb5750a0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe00eb5750a0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00eb5750b0 uart_intr_rxready() at uart_intr_rxready+0xa5/frame 0xfffffe00eb5750e0 uart_intr() at uart_intr+0x138/frame 0xfffffe00eb575120 intr_event_handle() at intr_event_handle+0xce/frame 0xfffffe00eb575170 intr_execute_handlers() at intr_execute_handlers+0x75/frame 0xfffffe00eb5751b0 lapic_handle_intr() at lapic_handle_intr+0x5b/frame 0xfffffe00eb5751d0 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00eb5751d0 --- interrupt, rip = 0xffffffff80c11a92, rsp = 0xfffffe00eb5752a0, rbp = 0xfffffe00eb5752b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00eb5752b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00eb575370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00eb5753b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00eb5753f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00eb5754d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5755c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb575700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5757d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb575800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb575950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb575ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb575bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb575bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- *** error reading from address fffffe00e2b92698 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00eb574980 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00eb574990 db_get_value() at db_get_value+0x4c/frame 0xfffffe00eb5749d0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00eb574a60 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00eb574a80 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00eb574b20 db_command() at db_command+0x25c/frame 0xfffffe00eb574bf0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00eb574c90 db_command() at db_command+0x25c/frame 0xfffffe00eb574d60 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00eb574d70 db_trap() at db_trap+0xe8/frame 0xfffffe00eb574e00 kdb_trap() at kdb_trap+0x157/frame 0xfffffe00eb574e80 trap() at trap+0x7d4/frame 0xfffffe00eb574fb0 calltrap() at calltrap+0x8/frame 0xfffffe00eb574fb0 --- trap 0x3, rip = 0xffffffff80c105a1, rsp = 0xfffffe00eb575080, rbp = 0xfffffe00eb5750a0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe00eb5750a0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00eb5750b0 uart_intr_rxready() at uart_intr_rxready+0xa5/frame 0xfffffe00eb5750e0 uart_intr() at uart_intr+0x138/frame 0xfffffe00eb575120 intr_event_handle() at intr_event_handle+0xce/frame 0xfffffe00eb575170 intr_execute_handlers() at intr_execute_handlers+0x75/frame 0xfffffe00eb5751b0 lapic_handle_intr() at lapic_handle_intr+0x5b/frame 0xfffffe00eb5751d0 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00eb5751d0 --- interrupt, rip = 0xffffffff80c11a92, rsp = 0xfffffe00eb5752a0, rbp = 0xfffffe00eb5752b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00eb5752b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00eb575370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00eb5753b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00eb5753f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00eb5754d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5755c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb575700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5757d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb575800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb575950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb575ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb575bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb575bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command sshd pid 2381 tid 100200 td 0xfffff80030f2a6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2b226f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2b22730 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2b22770 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e2b227d0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e2b22800 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e2b22860 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e2b228a0 kern_select() at kern_select+0x962/frame 0xfffffe00e2b22a80 sys_select() at sys_select+0x56/frame 0xfffffe00e2b22ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2b22bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2b22bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8008f5afa, rsp = 0x7fffffffa578, rbp = 0x7fffffffa5b0 --- Tracing command sshd pid 2378 tid 100173 td 0xfffff80030fc66e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2a47770 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2a477b0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2a477f0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e2a47850 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e2a47880 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e2a478e0 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e2a47920 kern_poll() at kern_poll+0x48b/frame 0xfffffe00e2a47aa0 sys_poll() at sys_poll+0x50/frame 0xfffffe00e2a47ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2a47bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2a47bf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x80089913a, rsp = 0x7fffffffe5d8, rbp = 0x7fffffffe610 --- Tracing command csh pid 2372 tid 100184 td 0xfffff8004420a6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2aba750 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2aba790 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2aba7d0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e2aba830 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e2aba860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e2aba8c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e2aba8f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e2aba950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e2aba980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e2aba9e0 dofileread() at dofileread+0x81/frame 0xfffffe00e2abaa30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e2abaa70 sys_read() at sys_read+0x83/frame 0xfffffe00e2abaac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2ababf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2ababf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004fbb1a, rsp = 0x7fffffffe1e8, rbp = 0x7fffffffe210 --- Tracing command login pid 2367 tid 100142 td 0xfffff800306f9000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e0b90690 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00eb5746f0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00eb574700 trap() at trap+0x73/frame 0xfffffe00eb574830 calltrap() at calltrap+0x8/frame 0xfffffe00eb574830 --- trap 0xc, rip = 0xffffffff8103e0a0, rsp = 0xfffffe00eb574900, rbp = 0xfffffe00eb574990 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe00eb574990 db_get_value() at db_get_value+0x32/frame 0xfffffe00eb5749d0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00eb574a60 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00eb574a80 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00eb574b20 db_command() at db_command+0x25c/frame 0xfffffe00eb574bf0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00eb574c90 db_command() at db_command+0x25c/frame 0xfffffe00eb574d60 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00eb574d70 db_trap() at db_trap+0xe8/frame 0xfffffe00eb574e00 kdb_trap() at kdb_trap+0x157/frame 0xfffffe00eb574e80 trap() at trap+0x7d4/frame 0xfffffe00eb574fb0 calltrap() at calltrap+0x8/frame 0xfffffe00eb574fb0 --- trap 0x3, rip = 0xffffffff80c105a1, rsp = 0xfffffe00eb575080, rbp = 0xfffffe00eb5750a0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe00eb5750a0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00eb5750b0 uart_intr_rxready() at uart_intr_rxready+0xa5/frame 0xfffffe00eb5750e0 uart_intr() at uart_intr+0x138/frame 0xfffffe00eb575120 intr_event_handle() at intr_event_handle+0xce/frame 0xfffffe00eb575170 intr_execute_handlers() at intr_execute_handlers+0x75/frame 0xfffffe00eb5751b0 lapic_handle_intr() at lapic_handle_intr+0x5b/frame 0xfffffe00eb5751d0 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00eb5751d0 --- interrupt, rip = 0xffffffff80c11a92, rsp = 0xfffffe00eb5752a0, rbp = 0xfffffe00eb5752b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00eb5752b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00eb575370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00eb5753b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00eb5753f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00eb5754d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5755c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb575700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5757d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb575800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb575950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb575ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb575bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb575bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- *** error reading from address fffffe00e0b90698 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00eb574980 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00eb574990 db_get_value() at db_get_value+0x4c/frame 0xfffffe00eb5749d0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00eb574a60 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00eb574a80 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00eb574b20 db_command() at db_command+0x25c/frame 0xfffffe00eb574bf0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00eb574c90 db_command() at db_command+0x25c/frame 0xfffffe00eb574d60 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00eb574d70 db_trap() at db_trap+0xe8/frame 0xfffffe00eb574e00 kdb_trap() at kdb_trap+0x157/frame 0xfffffe00eb574e80 trap() at trap+0x7d4/frame 0xfffffe00eb574fb0 calltrap() at calltrap+0x8/frame 0xfffffe00eb574fb0 --- trap 0x3, rip = 0xffffffff80c105a1, rsp = 0xfffffe00eb575080, rbp = 0xfffffe00eb5750a0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x101/frame 0xfffffe00eb5750a0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00eb5750b0 uart_intr_rxready() at uart_intr_rxready+0xa5/frame 0xfffffe00eb5750e0 uart_intr() at uart_intr+0x138/frame 0xfffffe00eb575120 intr_event_handle() at intr_event_handle+0xce/frame 0xfffffe00eb575170 intr_execute_handlers() at intr_execute_handlers+0x75/frame 0xfffffe00eb5751b0 lapic_handle_intr() at lapic_handle_intr+0x5b/frame 0xfffffe00eb5751d0 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00eb5751d0 --- interrupt, rip = 0xffffffff80c11a92, rsp = 0xfffffe00eb5752a0, rbp = 0xfffffe00eb5752b0 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00eb5752b0 _sx_xlock_hard() at _sx_xlock_hard+0x1b9/frame 0xfffffe00eb575370 _sx_xlock() at _sx_xlock+0xbf/frame 0xfffffe00eb5753b0 dnode_check_slots_free() at dnode_check_slots_free+0xb5/frame 0xfffffe00eb5753f0 dnode_hold_impl() at dnode_hold_impl+0xb4b/frame 0xfffffe00eb5754d0 dmu_object_alloc_impl() at dmu_object_alloc_impl+0x2c1/frame 0xfffffe00eb5755c0 zfs_mknode() at zfs_mknode+0x20d/frame 0xfffffe00eb575700 zfs_freebsd_create() at zfs_freebsd_create+0x54c/frame 0xfffffe00eb5757d0 VOP_CREATE_APV() at VOP_CREATE_APV+0xa8/frame 0xfffffe00eb575800 vn_open_cred() at vn_open_cred+0x36c/frame 0xfffffe00eb575950 kern_openat() at kern_openat+0x1f3/frame 0xfffffe00eb575ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00eb575bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00eb575bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x80042deda, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe2a0 --- Tracing command getty pid 2366 tid 100217 td 0xfffff804f0734000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2b83750 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2b83790 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2b837d0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e2b83830 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e2b83860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e2b838c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e2b838f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e2b83950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e2b83980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e2b839e0 dofileread() at dofileread+0x81/frame 0xfffffe00e2b83a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e2b83a70 sys_read() at sys_read+0x83/frame 0xfffffe00e2b83ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2b83bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2b83bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800433b1a, rsp = 0x7fffffffebd8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2365 tid 100216 td 0xfffff804f04e06e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2b7e750 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2b7e790 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2b7e7d0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e2b7e830 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e2b7e860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e2b7e8c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e2b7e8f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e2b7e950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e2b7e980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e2b7e9e0 dofileread() at dofileread+0x81/frame 0xfffffe00e2b7ea30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e2b7ea70 sys_read() at sys_read+0x83/frame 0xfffffe00e2b7eac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2b7ebf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2b7ebf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800433b1a, rsp = 0x7fffffffebd8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2364 tid 100215 td 0xfffff8048eb9c000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2b79750 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2b79790 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2b797d0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e2b79830 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e2b79860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e2b798c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e2b798f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e2b79950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e2b79980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e2b799e0 dofileread() at dofileread+0x81/frame 0xfffffe00e2b79a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e2b79a70 sys_read() at sys_read+0x83/frame 0xfffffe00e2b79ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2b79bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2b79bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800433b1a, rsp = 0x7fffffffebd8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2363 tid 100214 td 0xfffff8048eb9c6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2b74750 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2b74790 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2b747d0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e2b74830 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e2b74860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e2b748c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e2b748f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e2b74950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e2b74980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e2b749e0 dofileread() at dofileread+0x81/frame 0xfffffe00e2b74a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e2b74a70 sys_read() at sys_read+0x83/frame 0xfffffe00e2b74ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2b74bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2b74bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800433b1a, rsp = 0x7fffffffebd8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2362 tid 100213 td 0xfffff804f06b3000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2b6f750 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2b6f790 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2b6f7d0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e2b6f830 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e2b6f860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e2b6f8c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e2b6f8f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e2b6f950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e2b6f980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e2b6f9e0 dofileread() at dofileread+0x81/frame 0xfffffe00e2b6fa30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e2b6fa70 sys_read() at sys_read+0x83/frame 0xfffffe00e2b6fac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2b6fbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2b6fbf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800433b1a, rsp = 0x7fffffffebd8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2361 tid 100212 td 0xfffff804f06b36e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2b6a750 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2b6a790 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2b6a7d0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e2b6a830 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e2b6a860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e2b6a8c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e2b6a8f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e2b6a950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e2b6a980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e2b6a9e0 dofileread() at dofileread+0x81/frame 0xfffffe00e2b6aa30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e2b6aa70 sys_read() at sys_read+0x83/frame 0xfffffe00e2b6aac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2b6abf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2b6abf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800433b1a, rsp = 0x7fffffffebd8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2360 tid 100211 td 0xfffff80044332000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2b65750 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2b65790 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2b657d0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e2b65830 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e2b65860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e2b658c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e2b658f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e2b65950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e2b65980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e2b659e0 dofileread() at dofileread+0x81/frame 0xfffffe00e2b65a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e2b65a70 sys_read() at sys_read+0x83/frame 0xfffffe00e2b65ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2b65bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2b65bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800433b1a, rsp = 0x7fffffffebd8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2359 tid 100210 td 0xfffff800443326e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2b60750 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2b60790 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2b607d0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e2b60830 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e2b60860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e2b608c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e2b608f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e2b60950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e2b60980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e2b609e0 dofileread() at dofileread+0x81/frame 0xfffffe00e2b60a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e2b60a70 sys_read() at sys_read+0x83/frame 0xfffffe00e2b60ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2b60bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2b60bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800433b1a, rsp = 0x7fffffffebd8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2358 tid 100186 td 0xfffff801c083d000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2ac4750 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2ac4790 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2ac47d0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e2ac4830 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e2ac4860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e2ac48c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e2ac48f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e2ac4950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e2ac4980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e2ac49e0 dofileread() at dofileread+0x81/frame 0xfffffe00e2ac4a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e2ac4a70 sys_read() at sys_read+0x83/frame 0xfffffe00e2ac4ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2ac4bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2ac4bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800433b1a, rsp = 0x7fffffffebd8, rbp = 0x7fffffffec10 --- Tracing command cron pid 2312 tid 100206 td 0xfffff80030d536e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2b4c830 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2b4c870 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2b4c8b0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e2b4c910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00e2b4c950 _sleep() at _sleep+0x258/frame 0xfffffe00e2b4c9f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe00e2b4ca80 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe00e2b4cac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2b4cbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2b4cbf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003d8a7a, rsp = 0x7fffffffead8, rbp = 0x7fffffffecc0 --- Tracing command sshd pid 2308 tid 100166 td 0xfffff80030aa1000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2a186f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2a18730 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2a18770 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e2a187d0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e2a18800 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e2a18860 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e2a188a0 kern_select() at kern_select+0x962/frame 0xfffffe00e2a18a80 sys_select() at sys_select+0x56/frame 0xfffffe00e2a18ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2a18bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2a18bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8008f5afa, rsp = 0x7fffffffe668, rbp = 0x7fffffffe6a0 --- Tracing command powerd pid 2280 tid 100161 td 0xfffff80030d906e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e0bfb6d0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e0bfb710 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e0bfb750 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e0bfb7b0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00e0bfb7f0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x184/frame 0xfffffe00e0bfb860 seltdwait() at seltdwait+0x6b/frame 0xfffffe00e0bfb8a0 kern_select() at kern_select+0x962/frame 0xfffffe00e0bfba80 sys_select() at sys_select+0x56/frame 0xfffffe00e0bfbac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e0bfbbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e0bfbbf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800430afa, rsp = 0x7fffffffeb78, rbp = 0x7fffffffecc0 --- Tracing command watchdogd pid 2157 tid 100195 td 0xfffff80030f3f000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2b09830 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2b09870 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2b098b0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e2b09910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00e2b09950 _sleep() at _sleep+0x258/frame 0xfffffe00e2b099f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe00e2b09a80 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe00e2b09ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2b09bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2b09bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003c1a7a, rsp = 0x7fffffffea98, rbp = 0x7fffffffeae0 --- Tracing command rpcbind pid 2078 tid 100170 td 0xfffff800441e6000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2a38750 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2a38790 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2a387d0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e2a38830 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00e2a38870 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x184/frame 0xfffffe00e2a388e0 seltdwait() at seltdwait+0x6b/frame 0xfffffe00e2a38920 kern_poll() at kern_poll+0x48b/frame 0xfffffe00e2a38aa0 sys_poll() at sys_poll+0x50/frame 0xfffffe00e2a38ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2a38bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2a38bf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8003cf13a, rsp = 0x7fffffffca38, rbp = 0x7fffffffec50 --- Tracing command syslogd pid 2068 tid 100159 td 0xfffff80030f2b6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e0bf16f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e0bf1730 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e0bf1770 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e0bf17d0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e0bf1800 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e0bf1860 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e0bf18a0 kern_select() at kern_select+0x962/frame 0xfffffe00e0bf1a80 sys_select() at sys_select+0x56/frame 0xfffffe00e0bf1ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e0bf1bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e0bf1bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800438afa, rsp = 0x7fffffffe748, rbp = 0x7fffffffecc0 --- Tracing command dhclient pid 1992 tid 100191 td 0xfffff801c07c8000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2add750 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2add790 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2add7d0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e2add830 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00e2add870 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x184/frame 0xfffffe00e2add8e0 seltdwait() at seltdwait+0x6b/frame 0xfffffe00e2add920 kern_poll() at kern_poll+0x48b/frame 0xfffffe00e2addaa0 sys_poll() at sys_poll+0x50/frame 0xfffffe00e2addac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e2addbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e2addbf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8003f213a, rsp = 0x7fffffffec28, rbp = 0x7fffffffec60 --- Tracing command dhclient pid 1928 tid 100154 td 0xfffff80030ed6000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e0bcc770 mi_switch() at mi_switch+0x182/frame 0xfffffe00e0bcc7b0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e0bcc7f0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e0bcc850 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e0bcc880 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e0bcc8e0 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e0bcc920 kern_poll() at kern_poll+0x48b/frame 0xfffffe00e0bccaa0 sys_poll() at sys_poll+0x50/frame 0xfffffe00e0bccac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e0bccbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e0bccbf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8003f213a, rsp = 0x7fffffffec38, rbp = 0x7fffffffec60 --- Tracing command dhclient pid 1925 tid 100149 td 0xfffff800051ee6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e0bb36f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e0bb3730 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e0bb3770 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e0bb37d0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e0bb3800 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e0bb3860 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e0bb38a0 kern_select() at kern_select+0x962/frame 0xfffffe00e0bb3a80 sys_select() at sys_select+0x56/frame 0xfffffe00e0bb3ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e0bb3bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e0bb3bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80044eafa, rsp = 0x7fffffffe9e8, rbp = 0x7fffffffeaa0 --- Tracing command vnlru pid 34 tid 100141 td 0xfffff800306f96e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00da9fd9b0 mi_switch() at mi_switch+0x182/frame 0xfffffe00da9fd9f0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00da9fda30 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00da9fda70 _sleep() at _sleep+0x26c/frame 0xfffffe00da9fdb10 vnlru_proc() at vnlru_proc+0x149/frame 0xfffffe00da9fdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00da9fdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00da9fdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command syncer pid 33 tid 100140 td 0xfffff80030726000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00da9f89f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00da9f8a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00da9f8a70 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00da9f8ab0 _cv_timedwait_sbt() at _cv_timedwait_sbt+0x184/frame 0xfffffe00da9f8b20 sched_sync() at sched_sync+0x5ce/frame 0xfffffe00da9f8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00da9f8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00da9f8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100130 td 0xfffff800307266e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00d3424a00 mi_switch() at mi_switch+0x182/frame 0xfffffe00d3424a40 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00d3424a80 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00d3424ac0 _sleep() at _sleep+0x26c/frame 0xfffffe00d3424b60 buf_daemon() at buf_daemon+0x112/frame 0xfffffe00d3424bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00d3424bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d3424bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100131 td 0xfffff8003053e000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00d35e99f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00d35e9a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00d35e9a70 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00d35e9ab0 _sleep() at _sleep+0x26c/frame 0xfffffe00d35e9b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe00d35e9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00d35e9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d35e9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100132 td 0xfffff800305226e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00d35ee9f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00d35eea30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00d35eea70 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00d35eeab0 _sleep() at _sleep+0x26c/frame 0xfffffe00d35eeb50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe00d35eebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00d35eebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d35eebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100133 td 0xfffff80030522000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00d35f39f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00d35f3a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00d35f3a70 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00d35f3ab0 _sleep() at _sleep+0x26c/frame 0xfffffe00d35f3b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe00d35f3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00d35f3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d35f3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100134 td 0xfffff800304806e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00d35f89f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00d35f8a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00d35f8a70 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00d35f8ab0 _sleep() at _sleep+0x26c/frame 0xfffffe00d35f8b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe00d35f8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00d35f8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d35f8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100135 td 0xfffff800307456e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00d35fd9f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00d35fda30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00d35fda70 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00d35fdab0 _sleep() at _sleep+0x26c/frame 0xfffffe00d35fdb50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe00d35fdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00d35fdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d35fdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100136 td 0xfffff80030745000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00da8c49f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00da8c4a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00da8c4a70 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00da8c4ab0 _sleep() at _sleep+0x26c/frame 0xfffffe00da8c4b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe00da8c4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00da8c4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00da8c4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100138 td 0xfffff800307466e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00da9ee9f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00da9eea30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00da9eea70 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00da9eeab0 _sleep() at _sleep+0x26c/frame 0xfffffe00da9eeb50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe00da9eebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00da9eebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00da9eebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100152 td 0xfffff80003a4a000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e0bc2a20 mi_switch() at mi_switch+0x182/frame 0xfffffe00e0bc2a60 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e0bc2aa0 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00e0bc2ae0 _sleep() at _sleep+0x26c/frame 0xfffffe00e0bc2b80 softdep_flush() at softdep_flush+0x175/frame 0xfffffe00e0bc2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e0bc2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e0bc2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100203 td 0xfffff804f04aa6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e2b3da20 mi_switch() at mi_switch+0x182/frame 0xfffffe00e2b3da60 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e2b3daa0 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00e2b3dae0 _sleep() at _sleep+0x26c/frame 0xfffffe00e2b3db80 softdep_flush() at softdep_flush+0x175/frame 0xfffffe00e2b3dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e2b3dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e2b3dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command vmdaemon pid 31 tid 100129 td 0xfffff80030725000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00d341f9f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00d341fa30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00d341fa70 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d341faa0 _sleep() at _sleep+0x2cd/frame 0xfffffe00d341fb40 vm_daemon() at vm_daemon+0x82/frame 0xfffffe00d341fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00d341fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d341fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 30 tid 100128 td 0xfffff8003046e000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00d340e610 mi_switch() at mi_switch+0x182/frame 0xfffffe00d340e650 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00d340e690 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00d340e6d0 _sleep() at _sleep+0x26c/frame 0xfffffe00d340e770 vm_pageout_worker() at vm_pageout_worker+0x279/frame 0xfffffe00d340eb70 vm_pageout() at vm_pageout+0x193/frame 0xfffffe00d340ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00d340ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d340ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 30 tid 100137 td 0xfffff8000503c000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00da9e97e0 mi_switch() at mi_switch+0x182/frame 0xfffffe00da9e9820 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00da9e9860 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00da9e9890 _sleep() at _sleep+0x2cd/frame 0xfffffe00da9e9930 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0xe1b/frame 0xfffffe00da9e9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00da9e9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00da9e9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 30 tid 100139 td 0xfffff8003077d000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00da9f3a20 mi_switch() at mi_switch+0x182/frame 0xfffffe00da9f3a60 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00da9f3aa0 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00da9f3ae0 _sleep() at _sleep+0x26c/frame 0xfffffe00da9f3b80 uma_reclaim_worker() at uma_reclaim_worker+0xb8/frame 0xfffffe00da9f3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00da9f3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00da9f3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon1 pid 29 tid 100127 td 0xfffff8000503c6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00d34099e0 mi_switch() at mi_switch+0x182/frame 0xfffffe00d3409a20 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00d3409a60 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d3409a90 _sleep() at _sleep+0x2cd/frame 0xfffffe00d3409b30 enc_daemon() at enc_daemon+0xf6/frame 0xfffffe00d3409bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00d3409bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d3409bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon0 pid 28 tid 100126 td 0xfffff8003053e6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00d34049e0 mi_switch() at mi_switch+0x182/frame 0xfffffe00d3404a20 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00d3404a60 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d3404a90 _sleep() at _sleep+0x2cd/frame 0xfffffe00d3404b30 enc_daemon() at enc_daemon+0xf6/frame 0xfffffe00d3404bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00d3404bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d3404bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command rand_harvestq pid 27 tid 100120 td 0xfffff80003d22000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5e49f0 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5e4a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5e4a70 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe003e5e4ab0 _sleep() at _sleep+0x26c/frame 0xfffffe003e5e4b50 random_kthread() at random_kthread+0x67/frame 0xfffffe003e5e4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5e4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5e4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sctp_iterator pid 26 tid 100119 td 0xfffff80003d226e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5dfa40 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5dfa80 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5dfac0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e5dfaf0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e5dfb90 sctp_iterator_thread() at sctp_iterator_thread+0x59/frame 0xfffffe003e5dfbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5dfbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5dfbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod4 pid 25 tid 100118 td 0xfffff80030392000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5cea10 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5cea50 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5cea90 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe003e5cead0 _sleep() at _sleep+0x26c/frame 0xfffffe003e5ceb70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe003e5cebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5cebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5cebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod3 pid 24 tid 100117 td 0xfffff800303926e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5c9a10 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5c9a50 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5c9a90 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe003e5c9ad0 _sleep() at _sleep+0x26c/frame 0xfffffe003e5c9b70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe003e5c9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5c9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5c9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod2 pid 23 tid 100116 td 0xfffff800303aa000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5c4a10 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5c4a50 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5c4a90 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe003e5c4ad0 _sleep() at _sleep+0x26c/frame 0xfffffe003e5c4b70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe003e5c4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5c4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5c4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod1 pid 22 tid 100115 td 0xfffff800303aa6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5bfa10 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5bfa50 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5bfa90 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe003e5bfad0 _sleep() at _sleep+0x26c/frame 0xfffffe003e5bfb70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe003e5bfbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5bfbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5bfbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100096 td 0xfffff800040c1000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e554a60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e554aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e554ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e554b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e554b70 usb_process() at usb_process+0xc3/frame 0xfffffe003e554bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e554bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e554bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100097 td 0xfffff800040b96e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e559a60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e559aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e559ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e559b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e559b70 usb_process() at usb_process+0xc3/frame 0xfffffe003e559bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e559bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e559bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100098 td 0xfffff800040b9000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e55ea60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e55eaa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e55eae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e55eb10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e55eb70 usb_process() at usb_process+0xc3/frame 0xfffffe003e55ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e55ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e55ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100099 td 0xfffff8000500f6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e563a60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e563aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e563ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e563b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e563b70 usb_process() at usb_process+0xc3/frame 0xfffffe003e563bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e563bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e563bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100100 td 0xfffff8000500f000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e568a60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e568aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e568ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e568b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e568b70 usb_process() at usb_process+0xc3/frame 0xfffffe003e568bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e568bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e568bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100102 td 0xfffff80005010000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e572a60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e572aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e572ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e572b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e572b70 usb_process() at usb_process+0xc3/frame 0xfffffe003e572bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e572bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e572bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100103 td 0xfffff800040c06e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e577a60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e577aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e577ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e577b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e577b70 usb_process() at usb_process+0xc3/frame 0xfffffe003e577bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e577bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e577bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100104 td 0xfffff8000503a000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e57ca60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e57caa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e57cae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e57cb10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e57cb70 usb_process() at usb_process+0xc3/frame 0xfffffe003e57cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e57cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e57cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100105 td 0xfffff8000503b6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e581a60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e581aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e581ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e581b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e581b70 usb_process() at usb_process+0xc3/frame 0xfffffe003e581bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e581bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e581bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100106 td 0xfffff8000503b000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e586a60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e586aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e586ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e586b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e586b70 usb_process() at usb_process+0xc3/frame 0xfffffe003e586bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e586bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e586bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100107 td 0xfffff800051d16e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e597a60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e597aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e597ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e597b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e597b70 usb_process() at usb_process+0xc3/frame 0xfffffe003e597bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e597bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e597bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100108 td 0xfffff800051d1000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e59ca60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e59caa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e59cae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e59cb10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e59cb70 usb_process() at usb_process+0xc3/frame 0xfffffe003e59cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e59cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e59cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100109 td 0xfffff800051c96e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5a1a60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5a1aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5a1ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e5a1b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e5a1b70 usb_process() at usb_process+0xc3/frame 0xfffffe003e5a1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5a1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5a1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100110 td 0xfffff800051c9000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5a6a60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5a6aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5a6ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e5a6b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e5a6b70 usb_process() at usb_process+0xc3/frame 0xfffffe003e5a6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5a6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5a6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100111 td 0xfffff8000503a6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5aba60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5abaa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5abae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e5abb10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e5abb70 usb_process() at usb_process+0xc3/frame 0xfffffe003e5abbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5abbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5abbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100091 td 0xfffff80003bc8000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e53ba10 mi_switch() at mi_switch+0x182/frame 0xfffffe003e53ba50 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e53ba90 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e53bac0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e53bb60 xpt_done_td() at xpt_done_td+0x8b/frame 0xfffffe003e53bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e53bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e53bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100092 td 0xfffff80003d356e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e540a10 mi_switch() at mi_switch+0x182/frame 0xfffffe003e540a50 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e540a90 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e540ac0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e540b60 xpt_done_td() at xpt_done_td+0x8b/frame 0xfffffe003e540bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e540bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e540bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100093 td 0xfffff80003d35000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e545a10 mi_switch() at mi_switch+0x182/frame 0xfffffe003e545a50 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e545a90 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e545ac0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e545b60 xpt_done_td() at xpt_done_td+0x8b/frame 0xfffffe003e545bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e545bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e545bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100125 td 0xfffff8003046e6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5fda00 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5fda40 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5fda80 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e5fdab0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e5fdb50 xpt_scanner_thread() at xpt_scanner_thread+0x92/frame 0xfffffe003e5fdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5fdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5fdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 11 pid 19 tid 100090 td 0xfffff80003bc86e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5369f0 mi_switch() at mi_switch+0x182/frame 0xfffffe003e536a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e536a70 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e536aa0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e536b40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe003e536bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e536bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e536bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 10 pid 18 tid 100089 td 0xfffff80003bc7000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5319f0 mi_switch() at mi_switch+0x182/frame 0xfffffe003e531a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e531a70 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e531aa0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e531b40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe003e531bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e531bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e531bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 9 pid 17 tid 100088 td 0xfffff80003bc76e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e52c9f0 mi_switch() at mi_switch+0x182/frame 0xfffffe003e52ca30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e52ca70 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e52caa0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e52cb40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe003e52cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e52cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e52cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 8 pid 16 tid 100087 td 0xfffff80003bc6000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5279f0 mi_switch() at mi_switch+0x182/frame 0xfffffe003e527a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e527a70 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e527aa0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e527b40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe003e527bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e527bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e527bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 7 pid 15 tid 100086 td 0xfffff80003bb56e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5229f0 mi_switch() at mi_switch+0x182/frame 0xfffffe003e522a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e522a70 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e522aa0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e522b40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe003e522bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e522bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e522bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 6 pid 9 tid 100085 td 0xfffff80003bc1000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e51d9f0 mi_switch() at mi_switch+0x182/frame 0xfffffe003e51da30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e51da70 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e51daa0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e51db40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe003e51dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e51dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e51dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 5 pid 8 tid 100084 td 0xfffff80003bc16e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5189f0 mi_switch() at mi_switch+0x182/frame 0xfffffe003e518a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e518a70 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e518aa0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e518b40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe003e518bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e518bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e518bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 4 pid 7 tid 100083 td 0xfffff80003bc0000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5139f0 mi_switch() at mi_switch+0x182/frame 0xfffffe003e513a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e513a70 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e513aa0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e513b40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe003e513bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e513bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e513bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 3 pid 6 tid 100082 td 0xfffff80003bc06e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e50e9f0 mi_switch() at mi_switch+0x182/frame 0xfffffe003e50ea30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e50ea70 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e50eaa0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e50eb40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe003e50ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e50ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e50ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 2 pid 5 tid 100081 td 0xfffff80003bb7000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5099f0 mi_switch() at mi_switch+0x182/frame 0xfffffe003e509a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e509a70 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e509aa0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e509b40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe003e509bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e509bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e509bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 1 pid 4 tid 100080 td 0xfffff80003bb76e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5049f0 mi_switch() at mi_switch+0x182/frame 0xfffffe003e504a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e504a70 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e504aa0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e504b40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe003e504bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e504bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e504bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 0 pid 3 tid 100079 td 0xfffff80003bb6000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4ff9f0 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4ffa30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4ffa70 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4ffaa0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4ffb40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe003e4ffbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4ffbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4ffbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 2 tid 100078 td 0xfffff80003bb66e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4faa20 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4faa60 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4faaa0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4faad0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4fab70 crypto_proc() at crypto_proc+0x2bf/frame 0xfffffe003e4fabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4fabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4fabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sequencer 00 pid 14 tid 100065 td 0xfffff80003b28000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4b9a20 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4b9a60 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4b9aa0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4b9ad0 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e4b9b30 seq_eventthread() at seq_eventthread+0x8c/frame 0xfffffe003e4b9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4b9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4b9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100062 td 0xfffff80003b276e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e49ea20 mi_switch() at mi_switch+0x182/frame 0xfffffe003e49ea60 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e49eaa0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e49ead0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e49eb70 g_run_events() at g_run_events+0x47d/frame 0xfffffe003e49ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e49ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e49ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100063 td 0xfffff80003b27000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4a3a20 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4a3a60 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4a3aa0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4a3ad0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4a3b70 g_io_schedule_up() at g_io_schedule_up+0x4c/frame 0xfffffe003e4a3ba0 g_up_procbody() at g_up_procbody+0x5c/frame 0xfffffe003e4a3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4a3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4a3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100064 td 0xfffff80003b286e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4a8a20 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4a8a60 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4a8aa0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4a8ad0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4a8b70 g_io_schedule_down() at g_io_schedule_down+0x4c/frame 0xfffffe003e4a8ba0 g_down_procbody() at g_down_procbody+0x5c/frame 0xfffffe003e4a8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4a8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4a8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100017 td 0xfffff800038d3000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00addc4b10 mi_switch() at mi_switch+0x182/frame 0xfffffe00addc4b50 ithread_loop() at ithread_loop+0x39e/frame 0xfffffe00addc4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00addc4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00addc4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100018 td 0xfffff800038d26e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00addc9b10 mi_switch() at mi_switch+0x182/frame 0xfffffe00addc9b50 ithread_loop() at ithread_loop+0x39e/frame 0xfffffe00addc9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00addc9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00addc9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100020 td 0xfffff8000388d6e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100047 td 0xfffff80003a74000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100048 td 0xfffff80003a756e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e404b10 mi_switch() at mi_switch+0x182/frame 0xfffffe003e404b50 ithread_loop() at ithread_loop+0x39e/frame 0xfffffe003e404bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e404bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e404bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100049 td 0xfffff80003a75000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e409b10 mi_switch() at mi_switch+0x182/frame 0xfffffe003e409b50 ithread_loop() at ithread_loop+0x39e/frame 0xfffffe003e409bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e409bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e409bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100050 td 0xfffff80003a6d6e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100051 td 0xfffff80003a6d000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100052 td 0xfffff80003a7e6e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100053 td 0xfffff80003a7e000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100054 td 0xfffff80003a7f6e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100055 td 0xfffff80003a7f000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100056 td 0xfffff80003a746e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100057 td 0xfffff80003a7b000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100058 td 0xfffff80003a7c6e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100059 td 0xfffff80003a7c000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100060 td 0xfffff80003a7d6e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100094 td 0xfffff800040c0000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e54ab10 mi_switch() at mi_switch+0x182/frame 0xfffffe003e54ab50 ithread_loop() at ithread_loop+0x39e/frame 0xfffffe003e54abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e54abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e54abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100095 td 0xfffff800040c16e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e54fb10 mi_switch() at mi_switch+0x182/frame 0xfffffe003e54fb50 ithread_loop() at ithread_loop+0x39e/frame 0xfffffe003e54fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e54fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e54fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100101 td 0xfffff800050106e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e56db10 mi_switch() at mi_switch+0x182/frame 0xfffffe003e56db50 ithread_loop() at ithread_loop+0x39e/frame 0xfffffe003e56dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e56dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e56dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100112 td 0xfffff800051ee000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100113 td 0xfffff800051ef6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5b5b10 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5b5b50 ithread_loop() at ithread_loop+0x39e/frame 0xfffffe003e5b5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5b5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5b5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100176 td 0xfffff80030f2b000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100177 td 0xfffff80030f216e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100178 td 0xfffff80030f21000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100179 td 0xfffff801c087b6e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100180 td 0xfffff801c087b000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100181 td 0xfffff801c087c6e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100182 td 0xfffff801c087c000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100183 td 0xfffff800443306e0 fork_trampoline() at fork_trampoline Tracing command idle pid 11 tid 100003 td 0xfffff80003868000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00add7ea90 mi_switch() at mi_switch+0x182/frame 0xfffffe00add7ead0 sched_idletd() at sched_idletd+0xa3/frame 0xfffffe00add7ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add7ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add7ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xfffff800038596e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00add83a70 mi_switch() at mi_switch+0x182/frame 0xfffffe00add83ab0 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00add83ad0 sched_idletd() at sched_idletd+0x401/frame 0xfffffe00add83bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add83bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add83bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffff80003859000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00add88a90 mi_switch() at mi_switch+0x182/frame 0xfffffe00add88ad0 sched_idletd() at sched_idletd+0xa3/frame 0xfffffe00add88bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add88bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add88bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xfffff8000388a6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00add8da70 mi_switch() at mi_switch+0x182/frame 0xfffffe00add8dab0 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00add8dad0 sched_idletd() at sched_idletd+0x401/frame 0xfffffe00add8dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add8dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add8dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100007 td 0xfffff8000388a000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00add92a70 mi_switch() at mi_switch+0x182/frame 0xfffffe00add92ab0 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00add92ad0 sched_idletd() at sched_idletd+0x401/frame 0xfffffe00add92bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add92bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add92bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xfffff800038896e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00add97a70 mi_switch() at mi_switch+0x182/frame 0xfffffe00add97ab0 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00add97ad0 sched_idletd() at sched_idletd+0x401/frame 0xfffffe00add97bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add97bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add97bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xfffff80003889000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00add9ca70 mi_switch() at mi_switch+0x182/frame 0xfffffe00add9cab0 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00add9cad0 sched_idletd() at sched_idletd+0x401/frame 0xfffffe00add9cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add9cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add9cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xfffff800038696e0 (CPU 7) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1b7dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1b7df0 trap() at trap+0x61/frame 0xfffffe003e1b7f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1b7f20 --- trap 0x13, rip = 0xffffffff8102a0b6, rsp = 0xfffffe00adda1a30, rbp = 0xfffffe00adda1a30 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00adda1a30 acpi_cpu_idle() at acpi_cpu_idle+0x23d/frame 0xfffffe00adda1a90 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00adda1ab0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00adda1ad0 sched_idletd() at sched_idletd+0x401/frame 0xfffffe00adda1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adda1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adda1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffff8000388d000 (CPU 8) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1c3dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1c3df0 trap() at trap+0x61/frame 0xfffffe003e1c3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1c3f20 --- trap 0x13, rip = 0xffffffff8102a0b6, rsp = 0xfffffe00adda6a30, rbp = 0xfffffe00adda6a30 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00adda6a30 acpi_cpu_idle() at acpi_cpu_idle+0x23d/frame 0xfffffe00adda6a90 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00adda6ab0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00adda6ad0 sched_idletd() at sched_idletd+0x401/frame 0xfffffe00adda6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adda6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adda6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100012 td 0xfffff8000388c6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00addaba70 mi_switch() at mi_switch+0x182/frame 0xfffffe00addabab0 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00addabad0 sched_idletd() at sched_idletd+0x401/frame 0xfffffe00addabbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00addabbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00addabbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xfffff8000388c000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00addb0a70 mi_switch() at mi_switch+0x182/frame 0xfffffe00addb0ab0 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00addb0ad0 sched_idletd() at sched_idletd+0x401/frame 0xfffffe00addb0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00addb0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00addb0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100014 td 0xfffff8000388b6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00addb5a70 mi_switch() at mi_switch+0x182/frame 0xfffffe00addb5ab0 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00addb5ad0 sched_idletd() at sched_idletd+0x401/frame 0xfffffe00addb5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00addb5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00addb5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command init pid 1 tid 100002 td 0xfffff800038686e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00add79690 mi_switch() at mi_switch+0x182/frame 0xfffffe00add796d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00add79710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00add79770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00add797a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00add79840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00add798d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00add79ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00add79bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00add79bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x2b0c2a, rsp = 0x7fffffffe778, rbp = 0x7fffffffe7e0 --- Tracing command audit pid 10 tid 100001 td 0xfffff80003869000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00add74a40 mi_switch() at mi_switch+0x182/frame 0xfffffe00add74a80 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00add74ac0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00add74af0 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00add74b50 audit_worker() at audit_worker+0x9f/frame 0xfffffe00add74bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add74bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add74bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xffffffff81d9c940 sched_switch() at sched_switch+0x38f/frame 0xffffffff822ffbf0 mi_switch() at mi_switch+0x182/frame 0xffffffff822ffc30 sleepq_switch() at sleepq_switch+0x10e/frame 0xffffffff822ffc70 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xffffffff822ffcb0 _sleep() at _sleep+0x26c/frame 0xffffffff822ffd50 swapper() at swapper+0x75/frame 0xffffffff822ffda0 mi_startup() at mi_startup+0x2ed/frame 0xffffffff822ffdf0 btext() at btext+0x2c Tracing command kernel pid 0 tid 100015 td 0xfffff8000388b000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00addbaa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00addbaa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00addbaab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00addbaae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00addbab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00addbabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00addbabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00addbabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xfffff800038d36e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00addbfa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00addbfa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00addbfab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00addbfae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00addbfb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00addbfbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00addbfbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00addbfbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xfffff800038d2000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00addcea30 mi_switch() at mi_switch+0x182/frame 0xfffffe00addcea70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00addceab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00addceae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00addceb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00addcebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00addcebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00addcebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xfffff800038dd000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00addd8a60 mi_switch() at mi_switch+0x182/frame 0xfffffe00addd8aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00addd8ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00addd8b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00addd8b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00addd8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00addd8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00addd8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xfffff800038dc6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe0037781a60 mi_switch() at mi_switch+0x182/frame 0xfffffe0037781aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe0037781ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0037781b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe0037781b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe0037781bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037781bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037781bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xfffff800038dc000 sched_switch() at sched_switch+0x38f/frame 0xfffffe0037786a60 mi_switch() at mi_switch+0x182/frame 0xfffffe0037786aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe0037786ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0037786b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe0037786b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe0037786bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037786bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037786bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xfffff800038db6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003778ba60 mi_switch() at mi_switch+0x182/frame 0xfffffe003778baa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003778bae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003778bb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe003778bb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe003778bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003778bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003778bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xfffff800038db000 sched_switch() at sched_switch+0x38f/frame 0xfffffe0037790a60 mi_switch() at mi_switch+0x182/frame 0xfffffe0037790aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe0037790ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0037790b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe0037790b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe0037790bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037790bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037790bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100026 td 0xfffff800039126e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe0037795a60 mi_switch() at mi_switch+0x182/frame 0xfffffe0037795aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe0037795ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0037795b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe0037795b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe0037795bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037795bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037795bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100027 td 0xfffff80003912000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003779aa60 mi_switch() at mi_switch+0x182/frame 0xfffffe003779aaa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003779aae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003779ab10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe003779ab80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe003779abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003779abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003779abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100028 td 0xfffff800039116e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003779fa60 mi_switch() at mi_switch+0x182/frame 0xfffffe003779faa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003779fae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003779fb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe003779fb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe003779fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003779fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003779fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100029 td 0xfffff80003911000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377a4a60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377a4aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377a4ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377a4b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377a4b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377a4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377a4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377a4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100030 td 0xfffff800038dd6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377a9a60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377a9aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377a9ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377a9b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377a9b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377a9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377a9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377a9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100031 td 0xfffff80003917000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377aea60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377aeaa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377aeae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377aeb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377aeb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377aebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377aebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377aebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100032 td 0xfffff800039166e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377b3a60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377b3aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377b3ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377b3b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377b3b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377b3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377b3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377b3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100033 td 0xfffff80003916000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377b8a60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377b8aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377b8ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377b8b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377b8b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377b8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377b8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377b8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100034 td 0xfffff800039156e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377bda60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377bdaa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377bdae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377bdb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377bdb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377bdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377bdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377bdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100035 td 0xfffff80003915000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377c2a60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377c2aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377c2ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377c2b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377c2b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377c2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377c2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377c2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100036 td 0xfffff80003a506e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377c7a60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377c7aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377c7ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377c7b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377c7b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377c7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377c7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377c7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100037 td 0xfffff80003a50000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377cca60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377ccaa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377ccae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377ccb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377ccb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377ccbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377ccbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377ccbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100038 td 0xfffff80003a516e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377d1a60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377d1aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377d1ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377d1b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377d1b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377d1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377d1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377d1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100039 td 0xfffff80003a51000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377d6a60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377d6aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377d6ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377d6b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377d6b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377d6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377d6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377d6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100040 td 0xfffff800039176e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377dba60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377dbaa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377dbae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377dbb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377dbb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377dbbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377dbbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377dbbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100041 td 0xfffff80003a4d000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377e0a60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377e0aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377e0ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377e0b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377e0b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377e0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377e0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377e0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100042 td 0xfffff80003a4e6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377e5a60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377e5aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377e5ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377e5b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377e5b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377e5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377e5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377e5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100043 td 0xfffff80003a4e000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377eaa60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377eaaa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377eaae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377eab10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377eab80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377eabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377eabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377eabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100044 td 0xfffff80003a4f6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377efa60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377efaa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377efae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377efb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377efb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377efbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377efbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377efbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100045 td 0xfffff80003a4f000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377f4a60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377f4aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377f4ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377f4b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377f4b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377f4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377f4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377f4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100046 td 0xfffff80003a496e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377f9a60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377f9aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377f9ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377f9b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377f9b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377f9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377f9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377f9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100061 td 0xfffff80003a7d000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e499a30 mi_switch() at mi_switch+0x182/frame 0xfffffe003e499a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e499ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e499ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e499b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe003e499bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e499bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e499bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100066 td 0xfffff80003a7b6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4bea30 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4bea70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4beab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4beae0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4beb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe003e4bebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4bebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4bebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100067 td 0xfffff80003ba8000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4c3a30 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4c3a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4c3ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4c3ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4c3b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe003e4c3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4c3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4c3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100068 td 0xfffff80003ba96e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4c8a30 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4c8a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4c8ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4c8ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4c8b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe003e4c8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4c8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4c8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100069 td 0xfffff80003ba9000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4cda30 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4cda70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4cdab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4cdae0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4cdb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe003e4cdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4cdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4cdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100070 td 0xfffff80003baa6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4d2a30 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4d2a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4d2ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4d2ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4d2b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe003e4d2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4d2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4d2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100071 td 0xfffff80003baa000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4d7a30 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4d7a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4d7ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4d7ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4d7b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe003e4d7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4d7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4d7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100072 td 0xfffff80003ba66e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4dca30 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4dca70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4dcab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4dcae0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4dcb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe003e4dcbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4dcbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4dcbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100073 td 0xfffff80003ba6000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4e1a30 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4e1a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4e1ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4e1ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4e1b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe003e4e1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4e1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4e1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100074 td 0xfffff80003ba76e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4e6a30 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4e6a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4e6ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4e6ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4e6b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe003e4e6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4e6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4e6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100075 td 0xfffff80003ba7000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4eba30 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4eba70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4ebab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4ebae0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4ebb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe003e4ebbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4ebbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4ebbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100076 td 0xfffff80003ba86e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4f0a30 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4f0a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4f0ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4f0ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4f0b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe003e4f0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4f0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4f0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100077 td 0xfffff80003bb5000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4f5a30 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4f5a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4f5ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4f5ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4f5b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe003e4f5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4f5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4f5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100114 td 0xfffff800051ef000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5baa60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5baaa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5baae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e5bab10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe003e5bab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe003e5babb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5babf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5babf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100121 td 0xfffff80003bc66e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5e9a60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5e9aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5e9ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e5e9b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe003e5e9b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe003e5e9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5e9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5e9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100122 td 0xfffff80030480000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5eea60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5eeaa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5eeae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e5eeb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe003e5eeb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe003e5eebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5eebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5eebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100123 td 0xfffff800304816e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5f3a60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5f3aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5f3ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e5f3b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe003e5f3b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe003e5f3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5f3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5f3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100124 td 0xfffff80030481000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5f8a30 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5f8a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5f8ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e5f8ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e5f8b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe003e5f8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5f8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5f8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100821 td 0xfffff80190c9a000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00eb502a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00eb502a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00eb502ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00eb502ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00eb502b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00eb502bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00eb502bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00eb502bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100843 td 0xfffff801351746e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00eb255a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00eb255a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00eb255ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00eb255ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00eb255b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00eb255bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00eb255bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00eb255bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100847 td 0xfffff806b3a8c000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f03efa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f03efa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f03efab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f03efae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f03efb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f03efbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f03efbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f03efbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100848 td 0xfffff80550eb46e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f03f4a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f03f4a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f03f4ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f03f4ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f03f4b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f03f4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f03f4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f03f4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100849 td 0xfffff806a7921000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f03f9a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f03f9a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f03f9ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f03f9ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f03f9b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f03f9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f03f9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f03f9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100850 td 0xfffff80299b256e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f03fea30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f03fea70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f03feab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f03feae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f03feb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f03febb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f03febf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f03febf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100862 td 0xfffff80332d636e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f034ba30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f034ba70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f034bab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f034bae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f034bb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f034bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f034bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f034bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100863 td 0xfffff80273f7d000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f0350a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f0350a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f0350ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f0350ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f0350b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f0350bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f0350bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f0350bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100867 td 0xfffff80690c35000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f0355a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f0355a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f0355ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f0355ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f0355b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f0355bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f0355bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f0355bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100869 td 0xfffff807507cf000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f035aa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f035aa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f035aab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f035aae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f035ab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f035abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f035abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f035abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100871 td 0xfffff8063d4d56e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f035fa30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f035fa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f035fab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f035fae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f035fb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f035fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f035fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f035fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100878 td 0xfffff8063d4d5000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f0364a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f0364a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f0364ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f0364ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f0364b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f0364bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f0364bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f0364bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100879 td 0xfffff800763d66e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f0369a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f0369a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f0369ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f0369ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f0369b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f0369bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f0369bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f0369bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100884 td 0xfffff804fe225000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f036ea30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f036ea70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f036eab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f036eae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f036eb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f036ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f036ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f036ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100885 td 0xfffff80690c97000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f0373a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f0373a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f0373ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f0373ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f0373b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f0373bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f0373bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f0373bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100890 td 0xfffff80560652000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f0378a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f0378a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f0378ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f0378ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f0378b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f0378bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f0378bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f0378bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100893 td 0xfffff806994a36e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f037da30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f037da70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f037dab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f037dae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f037db80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f037dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f037dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f037dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100896 td 0xfffff806994a3000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f0382a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f0382a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f0382ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f0382ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f0382b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f0382bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f0382bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f0382bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100897 td 0xfffff80122f7e6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f0387a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f0387a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f0387ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f0387ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f0387b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f0387bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f0387bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f0387bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100898 td 0xfffff80122f7e000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f038ca30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f038ca70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f038cab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f038cae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f038cb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f038cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f038cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f038cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100902 td 0xfffff802dac656e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f0391a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f0391a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f0391ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f0391ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f0391b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f0391bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f0391bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f0391bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100914 td 0xfffff802dac65000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f0396a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f0396a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f0396ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f0396ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f0396b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f0396bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f0396bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f0396bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100920 td 0xfffff8033c5ff6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f039ba30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f039ba70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f039bab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f039bae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f039bb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f039bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f039bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f039bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100921 td 0xfffff8033c5ff000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00f03a0a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00f03a0a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00f03a0ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00f03a0ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f03a0b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00f03a0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f03a0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f03a0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db:0:pho> dump Dumping 3281 out of 32626 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Dump complete db:0:pho> show allchains chain 1: thread 100312 (pid 14234, sleep) sleeping on 0xffffffff81c80274 "nanslp" chain 2: thread 100190 (pid 30742, cat) sleeping on 0xfffff801a09155f0 "piperd" chain 3: thread 101842 (pid 30740, sh) sleeping on 0xfffff80546669530 "wait" chain 4: thread 102340 (pid 30739, find) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 5: thread 102688 (pid 30735, sh) sleeping on 0xfffff8004d2e4a60 "wait" chain 6: thread 100695 (pid 30734, mail) sleeping on 0xfffff800441922f8 "piperd" chain 7: thread 100610 (pid 30733, sh) sleeping on 0xfffff801ac3d5000 "wait" chain 8: thread 100473 (pid 30726, sh) sleeping on 0xfffff8023b0e3000 "wait" chain 9: thread 102266 (pid 30725, lockf) sleeping on 0xfffff803dc042a60 "wait" chain 10: thread 102417 (pid 30723, sh) sleeping on 0xfffff80190c2e000 "wait" chain 11: thread 100778 (pid 30722, sh) sleeping on 0xfffff802a0751000 "wait" chain 12: thread 102268 (pid 27785, mail) sleeping on 0xfffff806f8a87000 "piperd" chain 13: thread 102829 (pid 27784, sh) sleeping on 0xfffff8077378a000 "wait" chain 14: thread 102234 (pid 27776, sh) sleeping on 0xfffff80796c1da60 "wait" chain 15: thread 100555 (pid 27775, lockf) blocked on lockmgr `EXCL thread 0 (pid -512, ) is inactive chain 16: thread 100574 (pid 27773, sh) sleeping on 0xfffff801b195a000 "wait" chain 17: thread 102531 (pid 27771, cron) sleeping on 0xfffff8021727b2f8 "piperd" chain 18: thread 101994 (pid 20682, sh) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 19: thread 100145 (pid 20486, sh) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 20: thread 100547 (pid 20414, sh) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 21: thread 100339 (pid 20407, fstat) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 22: thread 101844 (pid 20406, sh) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 23: thread 101960 (pid 20380, sh) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 24: thread 102304 (pid 20345, fstat) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 25: thread 100609 (pid 20344, fstat) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 26: thread 100550 (pid 20250, fstat) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 27: thread 102164 (pid 20241, fstat) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 28: thread 100608 (pid 20237, fstat) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 29: thread 102491 (pid 20230, fstat) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 30: thread 101893 (pid 19977, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 31: thread 100407 (pid 19976, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 32: thread 102149 (pid 19975, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 33: thread 102190 (pid 19974, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 34: thread 100749 (pid 19973, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 35: thread 100185 (pid 19972, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 36: thread 100321 (pid 19971, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 37: thread 102513 (pid 19970, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 38: thread 102516 (pid 19969, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 39: thread 102236 (pid 19968, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 40: thread 100675 (pid 19967, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 41: thread 102298 (pid 19966, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 42: thread 100298 (pid 19965, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 43: thread 102371 (pid 19964, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 44: thread 100614 (pid 19963, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 45: thread 102150 (pid 19962, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 46: thread 100268 (pid 19960, dirnprename) sleeping on 0xfffff80044126000 "wait" chain 47: thread 100162 (pid 19957, sysctl) sleeping on 0xfffff80030527530 "wait" chain 48: thread 102765 (pid 19956, sysctl) sleeping on 0xfffff804f36ba000 "wait" chain 49: thread 102763 (pid 19955, sysctl) sleeping on 0xfffff804f36ba530 "wait" chain 50: thread 102761 (pid 19954, sysctl) sleeping on 0xfffff804f36baa60 "wait" chain 51: thread 103055 (pid 19953, sysctl) sleeping on 0xfffff804f36d6a60 "wait" chain 52: thread 100387 (pid 19952, sysctl) sleeping on 0xfffff80050d1f530 "wait" chain 53: thread 102902 (pid 19951, sysctl) sleeping on 0xfffff8002e156a60 "wait" chain 54: thread 100676 (pid 19950, sysctl) sleeping on 0xfffff806e40e1530 "wait" chain 55: thread 102119 (pid 19949, sysctl) sleeping on 0xfffff80109e04530 "wait" chain 56: thread 102241 (pid 19948, sysctl) sleeping on 0xfffff80796c1d000 "wait" chain 57: thread 102937 (pid 19947, sysctl) blocked on lockmgr `ZM05jEXCL thread 0 (pid -512, ) is inactive chain 58: thread 102259 (pid 19946, sysctl) sleeping on 0xfffff8053f130530 "wait" chain 59: thread 102290 (pid 19945, sysctl) sleeping on 0xfffff8039a4d5a60 "wait" chain 60: thread 100789 (pid 19944, dirrename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 61: thread 102488 (pid 19943, dirrename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 62: thread 102590 (pid 19942, dirrename) sleeping on 0xfffff80189fbb530 "wait" chain 63: thread 100144 (pid 19909, mkdir) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 64: thread 102349 (pid 19906, mkdir) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 65: thread 102732 (pid 19904, mkdir) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 66: thread 101957 (pid 19902, mkdir) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 67: thread 101956 (pid 19900, mkdir) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 68: thread 100554 (pid 19899, mkdir) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 69: thread 102065 (pid 19897, mkdir) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 70: thread 100342 (pid 19896, mkdir) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 71: thread 102376 (pid 19895, mkdir) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 72: thread 102347 (pid 19894, mkdir) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 73: thread 103019 (pid 19893, mkdir) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 74: thread 101598 (pid 19892, mkdir) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 75: thread 101916 (pid 19891, mkdir) sleeping on 0xfffff802f9fa2000 "wait" chain 76: thread 102533 (pid 19890, link) sleeping on 0xfffff808006e41d0 "dn->dn_notxholds" chain 77: thread 102381 (pid 19889, link) sleeping on 0xfffff808006e41d0 "dn->dn_notxholds" chain 78: thread 102281 (pid 19887, link) sleeping on 0xfffff808006e41d0 "dn->dn_notxholds" chain 79: thread 102213 (pid 19885, link) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 80: thread 102161 (pid 19884, lockf) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 81: thread 100160 (pid 19883, link) sleeping on 0xfffff808006e41d0 "dn->dn_notxholds" chain 82: thread 102373 (pid 19882, lockf) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 83: thread 102938 (pid 19881, link) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 84: thread 102489 (pid 19880, lockf) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 85: thread 100679 (pid 19879, link) sleeping on 0xfffff808006e41d0 "dn->dn_notxholds" chain 86: thread 101843 (pid 19878, lockf) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 87: thread 102406 (pid 19877, link) sleeping on 0xfffff808006e41d0 "dn->dn_notxholds" chain 88: thread 100528 (pid 19876, lockf) sleeping on 0xfffff801b11fe000 "wait" chain 89: thread 101597 (pid 19875, link) sleeping on 0xfffff80174c75a60 "wait" chain 90: thread 102470 (pid 19870, dirrename) sleeping on 0xfffff803dc09e000 "wait" chain 91: thread 102021 (pid 19868, dirnprename) sleeping on 0xfffff80432c95000 "wait" chain 92: thread 102445 (pid 19859, openat) sleeping on 0xfffff804f3605a60 "wait" chain 93: thread 100602 (pid 19857, mkdir) sleeping on 0xfffff80191cd4530 "wait" chain 94: thread 101981 (pid 19855, lockf) sleeping on 0xfffff8004d208a60 "wait" chain 95: thread 101958 (pid 19854, link) sleeping on 0xfffff806bdf2a000 "wait" chain 96: thread 101959 (pid 19847, openat) sleeping on 0xfffff80109ef8000 "wait" chain 97: thread 100221 (pid 19843, sysctl) sleeping on 0xfffff80044126a60 "wait" chain 98: thread 101846 (pid 19838, run) sleeping on 0xfffff8015cff8000 "wait" chain 99: thread 100187 (pid 19836, run) sleeping on 0xfffff80030527000 "wait" chain 100: thread 100332 (pid 19834, run) sleeping on 0xfffff802b8f2d530 "wait" chain 101: thread 100777 (pid 19810, sh) sleeping on 0xfffff802a0751530 "wait" chain 102: thread 101835 (pid 19734, stress2_tank) sleeping on 0xfffffe00d9fe6cd8 "spa->spa_proc_cv" chain 103: thread 100995 (pid 19734, zio_null_issue) sleeping on 0xfffff805f1e98100 "-" chain 104: thread 100998 (pid 19734, zio_null_intr) sleeping on 0xfffff80205a0d000 "-" chain 105: thread 101000 (pid 19734, zio_read_issue_0) sleeping on 0xfffff80205a0d500 "-" chain 106: thread 101003 (pid 19734, zio_read_issue_1) sleeping on 0xfffff80205a0d500 "-" chain 107: thread 101007 (pid 19734, zio_read_issue_2) sleeping on 0xfffff80205a0d500 "-" chain 108: thread 101009 (pid 19734, zio_read_issue_3) sleeping on 0xfffff80205a0d500 "-" chain 109: thread 101010 (pid 19734, zio_read_issue_4) sleeping on 0xfffff80205a0d500 "-" chain 110: thread 101012 (pid 19734, zio_read_issue_5) sleeping on 0xfffff80205a0d500 "-" chain 111: thread 101019 (pid 19734, zio_read_issue_6) sleeping on 0xfffff80205a0d500 "-" chain 112: thread 101022 (pid 19734, zio_read_issue_7) sleeping on 0xfffff80205a0d500 "-" chain 113: thread 101024 (pid 19734, zio_read_intr_0_0) sleeping on 0xfffff80808394600 "-" chain 114: thread 101026 (pid 19734, zio_read_intr_0_1) sleeping on 0xfffff80808394600 "-" chain 115: thread 101027 (pid 19734, zio_read_intr_0_2) sleeping on 0xfffff80808394600 "-" chain 116: thread 101036 (pid 19734, zio_read_intr_0_3) sleeping on 0xfffff80808394600 "-" chain 117: thread 101039 (pid 19734, zio_read_intr_0_4) sleeping on 0xfffff80808394600 "-" chain 118: thread 101040 (pid 19734, zio_read_intr_0_5) sleeping on 0xfffff80808394600 "-" chain 119: thread 101042 (pid 19734, zio_read_intr_0_6) sleeping on 0xfffff80808394600 "-" chain 120: thread 101043 (pid 19734, zio_read_intr_0_7) sleeping on 0xfffff80808394600 "-" chain 121: thread 101044 (pid 19734, zio_read_intr_0_8) sleeping on 0xfffff80808394600 "-" chain 122: thread 101051 (pid 19734, zio_read_intr_0_9) sleeping on 0xfffff80808394600 "-" chain 123: thread 101053 (pid 19734, zio_read_intr_0_10) sleeping on 0xfffff80808394600 "-" chain 124: thread 101057 (pid 19734, zio_read_intr_0_11) sleeping on 0xfffff80808394600 "-" chain 125: thread 101060 (pid 19734, zio_read_intr_1_0) sleeping on 0xfffff805be212100 "-" chain 126: thread 101061 (pid 19734, zio_read_intr_1_1) sleeping on 0xfffff805be212100 "-" chain 127: thread 101070 (pid 19734, zio_read_intr_1_2) sleeping on 0xfffff805be212100 "-" chain 128: thread 101071 (pid 19734, zio_read_intr_1_3) sleeping on 0xfffff805be212100 "-" chain 129: thread 101073 (pid 19734, zio_read_intr_1_4) sleeping on 0xfffff805be212100 "-" chain 130: thread 101074 (pid 19734, zio_read_intr_1_5) sleeping on 0xfffff805be212100 "-" chain 131: thread 101075 (pid 19734, zio_read_intr_1_6) sleeping on 0xfffff805be212100 "-" chain 132: thread 101077 (pid 19734, zio_read_intr_1_7) sleeping on 0xfffff805be212100 "-" chain 133: thread 101082 (pid 19734, zio_read_intr_1_8) sleeping on 0xfffff805be212100 "-" chain 134: thread 101083 (pid 19734, zio_read_intr_1_9) sleeping on 0xfffff805be212100 "-" chain 135: thread 101087 (pid 19734, zio_read_intr_1_10) sleeping on 0xfffff805be212100 "-" chain 136: thread 101088 (pid 19734, zio_read_intr_1_11) sleeping on 0xfffff805be212100 "-" chain 137: thread 101089 (pid 19734, zio_read_intr_2_0) sleeping on 0xfffff801f90ca000 "-" chain 138: thread 101092 (pid 19734, zio_read_intr_2_1) sleeping on 0xfffff801f90ca000 "-" chain 139: thread 101096 (pid 19734, zio_read_intr_2_2) sleeping on 0xfffff801f90ca000 "-" chain 140: thread 101098 (pid 19734, zio_read_intr_2_3) sleeping on 0xfffff801f90ca000 "-" chain 141: thread 101102 (pid 19734, zio_read_intr_2_4) sleeping on 0xfffff801f90ca000 "-" chain 142: thread 101107 (pid 19734, zio_read_intr_2_5) sleeping on 0xfffff801f90ca000 "-" chain 143: thread 101108 (pid 19734, zio_read_intr_2_6) sleeping on 0xfffff801f90ca000 "-" chain 144: thread 101110 (pid 19734, zio_read_intr_2_7) sleeping on 0xfffff801f90ca000 "-" chain 145: thread 101113 (pid 19734, zio_read_intr_2_8) sleeping on 0xfffff801f90ca000 "-" chain 146: thread 101116 (pid 19734, zio_read_intr_2_9) sleeping on 0xfffff801f90ca000 "-" chain 147: thread 101117 (pid 19734, zio_read_intr_2_10) sleeping on 0xfffff801f90ca000 "-" chain 148: thread 101119 (pid 19734, zio_read_intr_2_11) sleeping on 0xfffff801f90ca000 "-" chain 149: thread 101121 (pid 19734, zio_read_intr_3_0) sleeping on 0xfffff80808393d00 "-" chain 150: thread 101128 (pid 19734, zio_read_intr_3_1) sleeping on 0xfffff80808393d00 "-" chain 151: thread 101129 (pid 19734, zio_read_intr_3_2) sleeping on 0xfffff80808393d00 "-" chain 152: thread 101132 (pid 19734, zio_read_intr_3_3) sleeping on 0xfffff80808393d00 "-" chain 153: thread 101137 (pid 19734, zio_read_intr_3_4) sleeping on 0xfffff80808393d00 "-" chain 154: thread 101138 (pid 19734, zio_read_intr_3_5) sleeping on 0xfffff80808393d00 "-" chain 155: thread 101143 (pid 19734, zio_read_intr_3_6) sleeping on 0xfffff80808393d00 "-" chain 156: thread 101145 (pid 19734, zio_read_intr_3_7) sleeping on 0xfffff80808393d00 "-" chain 157: thread 101146 (pid 19734, zio_read_intr_3_8) sleeping on 0xfffff80808393d00 "-" chain 158: thread 101147 (pid 19734, zio_read_intr_3_9) sleeping on 0xfffff80808393d00 "-" chain 159: thread 101148 (pid 19734, zio_read_intr_3_10) sleeping on 0xfffff80808393d00 "-" chain 160: thread 101152 (pid 19734, zio_read_intr_3_11) sleeping on 0xfffff80808393d00 "-" chain 161: thread 101159 (pid 19734, zio_read_intr_4_0) sleeping on 0xfffff80464585c00 "-" chain 162: thread 101161 (pid 19734, zio_read_intr_4_1) sleeping on 0xfffff80464585c00 "-" chain 163: thread 101163 (pid 19734, zio_read_intr_4_2) sleeping on 0xfffff80464585c00 "-" chain 164: thread 101164 (pid 19734, zio_read_intr_4_3) sleeping on 0xfffff80464585c00 "-" chain 165: thread 101168 (pid 19734, zio_read_intr_4_4) sleeping on 0xfffff80464585c00 "-" chain 166: thread 101169 (pid 19734, zio_read_intr_4_5) sleeping on 0xfffff80464585c00 "-" chain 167: thread 101174 (pid 19734, zio_read_intr_4_6) sleeping on 0xfffff80464585c00 "-" chain 168: thread 101176 (pid 19734, zio_read_intr_4_7) sleeping on 0xfffff80464585c00 "-" chain 169: thread 101177 (pid 19734, zio_read_intr_4_8) sleeping on 0xfffff80464585c00 "-" chain 170: thread 101180 (pid 19734, zio_read_intr_4_9) sleeping on 0xfffff80464585c00 "-" chain 171: thread 101182 (pid 19734, zio_read_intr_4_10) sleeping on 0xfffff80464585c00 "-" chain 172: thread 101183 (pid 19734, zio_read_intr_4_11) sleeping on 0xfffff80464585c00 "-" chain 173: thread 101184 (pid 19734, zio_read_intr_5_0) sleeping on 0xfffff8049069e000 "-" chain 174: thread 101189 (pid 19734, zio_read_intr_5_1) sleeping on 0xfffff8049069e000 "-" chain 175: thread 101191 (pid 19734, zio_read_intr_5_2) sleeping on 0xfffff8049069e000 "-" chain 176: thread 101194 (pid 19734, zio_read_intr_5_3) sleeping on 0xfffff8049069e000 "-" chain 177: thread 101198 (pid 19734, zio_read_intr_5_4) sleeping on 0xfffff8049069e000 "-" chain 178: thread 101199 (pid 19734, zio_read_intr_5_5) sleeping on 0xfffff8049069e000 "-" chain 179: thread 101200 (pid 19734, zio_read_intr_5_6) sleeping on 0xfffff8049069e000 "-" chain 180: thread 101202 (pid 19734, zio_read_intr_5_7) sleeping on 0xfffff8049069e000 "-" chain 181: thread 101203 (pid 19734, zio_read_intr_5_8) sleeping on 0xfffff8049069e000 "-" chain 182: thread 101204 (pid 19734, zio_read_intr_5_9) sleeping on 0xfffff8049069e000 "-" chain 183: thread 101206 (pid 19734, zio_read_intr_5_10) sleeping on 0xfffff8049069e000 "-" chain 184: thread 101208 (pid 19734, zio_read_intr_5_11) sleeping on 0xfffff8049069e000 "-" chain 185: thread 101209 (pid 19734, zio_read_intr_6_0) sleeping on 0xfffff807fd530e00 "-" chain 186: thread 101210 (pid 19734, zio_read_intr_6_1) sleeping on 0xfffff807fd530e00 "-" chain 187: thread 101213 (pid 19734, zio_read_intr_6_2) sleeping on 0xfffff807fd530e00 "-" chain 188: thread 101214 (pid 19734, zio_read_intr_6_3) sleeping on 0xfffff807fd530e00 "-" chain 189: thread 101215 (pid 19734, zio_read_intr_6_4) sleeping on 0xfffff807fd530e00 "-" chain 190: thread 101218 (pid 19734, zio_read_intr_6_5) sleeping on 0xfffff807fd530e00 "-" chain 191: thread 101219 (pid 19734, zio_read_intr_6_6) sleeping on 0xfffff807fd530e00 "-" chain 192: thread 101220 (pid 19734, zio_read_intr_6_7) sleeping on 0xfffff807fd530e00 "-" chain 193: thread 101221 (pid 19734, zio_read_intr_6_8) sleeping on 0xfffff807fd530e00 "-" chain 194: thread 101222 (pid 19734, zio_read_intr_6_9) sleeping on 0xfffff807fd530e00 "-" chain 195: thread 101224 (pid 19734, zio_read_intr_6_10) sleeping on 0xfffff807fd530e00 "-" chain 196: thread 101225 (pid 19734, zio_read_intr_6_11) sleeping on 0xfffff807fd530e00 "-" chain 197: thread 101226 (pid 19734, zio_read_intr_7_0) sleeping on 0xfffff804b891a700 "-" chain 198: thread 101230 (pid 19734, zio_read_intr_7_1) sleeping on 0xfffff804b891a700 "-" chain 199: thread 101234 (pid 19734, zio_read_intr_7_2) sleeping on 0xfffff804b891a700 "-" chain 200: thread 101235 (pid 19734, zio_read_intr_7_3) sleeping on 0xfffff804b891a700 "-" chain 201: thread 101236 (pid 19734, zio_read_intr_7_4) sleeping on 0xfffff804b891a700 "-" chain 202: thread 101237 (pid 19734, zio_read_intr_7_5) sleeping on 0xfffff804b891a700 "-" chain 203: thread 101238 (pid 19734, zio_read_intr_7_6) sleeping on 0xfffff804b891a700 "-" chain 204: thread 101252 (pid 19734, zio_read_intr_7_7) sleeping on 0xfffff804b891a700 "-" chain 205: thread 101253 (pid 19734, zio_read_intr_7_8) sleeping on 0xfffff804b891a700 "-" chain 206: thread 101263 (pid 19734, zio_read_intr_7_9) sleeping on 0xfffff804b891a700 "-" chain 207: thread 101264 (pid 19734, zio_read_intr_7_10) sleeping on 0xfffff804b891a700 "-" chain 208: thread 101265 (pid 19734, zio_read_intr_7_11) sleeping on 0xfffff804b891a700 "-" chain 209: thread 101266 (pid 19734, zio_write_issue_0) blocked on lockmgr (null)EXCL chain 210: thread 101268 (pid 19734, zio_write_issue_1) blocked on lockmgr (null)EXCL chain 211: thread 101270 (pid 19734, zio_write_issue_2) blocked on lockmgr (null)EXCL chain 212: thread 101271 (pid 19734, zio_write_issue_3) blocked on lockmgr (null)EXCL chain 213: thread 101273 (pid 19734, zio_write_issue_4) blocked on lockmgr (null)EXCL chain 214: thread 101274 (pid 19734, zio_write_issue_5) blocked on lockmgr (null)EXCL chain 215: thread 101275 (pid 19734, zio_write_issue_6) blocked on lockmgr (null)EXCL chain 216: thread 101277 (pid 19734, zio_write_issue_7) blocked on lockmgr (null)EXCL chain 217: thread 101279 (pid 19734, zio_write_issue_8) blocked on lockmgr (null)EXCL chain 218: thread 101280 (pid 19734, zio_write_issue_hig) sleeping on 0xfffff8064008dd00 "-" chain 219: thread 101281 (pid 19734, zio_write_issue_hig) sleeping on 0xfffff8064008dd00 "-" chain 220: thread 101283 (pid 19734, zio_write_issue_hig) sleeping on 0xfffff8064008dd00 "-" chain 221: thread 101284 (pid 19734, zio_write_issue_hig) sleeping on 0xfffff8064008dd00 "-" chain 222: thread 101286 (pid 19734, zio_write_issue_hig) sleeping on 0xfffff8064008dd00 "-" chain 223: thread 101287 (pid 19734, zio_write_intr_0) sleeping on 0xfffff805be212c00 "-" chain 224: thread 101288 (pid 19734, zio_write_intr_1) sleeping on 0xfffff805be212c00 "-" chain 225: thread 101291 (pid 19734, zio_write_intr_2) sleeping on 0xfffff805be212c00 "-" chain 226: thread 101293 (pid 19734, zio_write_intr_3) sleeping on 0xfffff805be212c00 "-" chain 227: thread 101294 (pid 19734, zio_write_intr_4) sleeping on 0xfffff805be212c00 "-" chain 228: thread 101295 (pid 19734, zio_write_intr_5) sleeping on 0xfffff805be212c00 "-" chain 229: thread 101297 (pid 19734, zio_write_intr_6) sleeping on 0xfffff805be212c00 "-" chain 230: thread 101298 (pid 19734, zio_write_intr_7) sleeping on 0xfffff805be212c00 "-" chain 231: thread 101299 (pid 19734, zio_write_intr_high) sleeping on 0xfffff8047ed3bc00 "-" chain 232: thread 101301 (pid 19734, zio_write_intr_high) sleeping on 0xfffff8047ed3bc00 "-" chain 233: thread 101302 (pid 19734, zio_write_intr_high) sleeping on 0xfffff8047ed3bc00 "-" chain 234: thread 101303 (pid 19734, zio_write_intr_high) sleeping on 0xfffff8047ed3bc00 "-" chain 235: thread 101304 (pid 19734, zio_write_intr_high) sleeping on 0xfffff8047ed3bc00 "-" chain 236: thread 101306 (pid 19734, zio_free_issue_0_0) sleeping on 0xfffff800038cb200 "-" chain 237: thread 101307 (pid 19734, zio_free_issue_0_1) sleeping on 0xfffff800038cb200 "-" chain 238: thread 101308 (pid 19734, zio_free_issue_0_2) sleeping on 0xfffff800038cb200 "-" chain 239: thread 101310 (pid 19734, zio_free_issue_0_3) sleeping on 0xfffff800038cb200 "-" chain 240: thread 101311 (pid 19734, zio_free_issue_0_4) sleeping on 0xfffff800038cb200 "-" chain 241: thread 101312 (pid 19734, zio_free_issue_0_5) sleeping on 0xfffff800038cb200 "-" chain 242: thread 101314 (pid 19734, zio_free_issue_0_6) sleeping on 0xfffff800038cb200 "-" chain 243: thread 101315 (pid 19734, zio_free_issue_0_7) sleeping on 0xfffff800038cb200 "-" chain 244: thread 101316 (pid 19734, zio_free_issue_0_8) sleeping on 0xfffff800038cb200 "-" chain 245: thread 101317 (pid 19734, zio_free_issue_0_9) sleeping on 0xfffff800038cb200 "-" chain 246: thread 101319 (pid 19734, zio_free_issue_0_10) sleeping on 0xfffff800038cb200 "-" chain 247: thread 101320 (pid 19734, zio_free_issue_0_11) sleeping on 0xfffff800038cb200 "-" chain 248: thread 101321 (pid 19734, zio_free_issue_1_0) sleeping on 0xfffff8080841ac00 "-" chain 249: thread 101323 (pid 19734, zio_free_issue_1_1) sleeping on 0xfffff8080841ac00 "-" chain 250: thread 101326 (pid 19734, zio_free_issue_1_2) sleeping on 0xfffff8080841ac00 "-" chain 251: thread 101327 (pid 19734, zio_free_issue_1_3) sleeping on 0xfffff8080841ac00 "-" chain 252: thread 101329 (pid 19734, zio_free_issue_1_4) sleeping on 0xfffff8080841ac00 "-" chain 253: thread 101335 (pid 19734, zio_free_issue_1_5) sleeping on 0xfffff8080841ac00 "-" chain 254: thread 101336 (pid 19734, zio_free_issue_1_6) sleeping on 0xfffff8080841ac00 "-" chain 255: thread 101340 (pid 19734, zio_free_issue_1_7) sleeping on 0xfffff8080841ac00 "-" chain 256: thread 101343 (pid 19734, zio_free_issue_1_8) sleeping on 0xfffff8080841ac00 "-" chain 257: thread 101347 (pid 19734, zio_free_issue_1_9) sleeping on 0xfffff8080841ac00 "-" chain 258: thread 101349 (pid 19734, zio_free_issue_1_10) sleeping on 0xfffff8080841ac00 "-" chain 259: thread 101351 (pid 19734, zio_free_issue_1_11) sleeping on 0xfffff8080841ac00 "-" chain 260: thread 101356 (pid 19734, zio_free_issue_2_0) sleeping on 0xfffff80808392500 "-" chain 261: thread 101361 (pid 19734, zio_free_issue_2_1) sleeping on 0xfffff80808392500 "-" chain 262: thread 101365 (pid 19734, zio_free_issue_2_2) sleeping on 0xfffff80808392500 "-" chain 263: thread 101366 (pid 19734, zio_free_issue_2_3) sleeping on 0xfffff80808392500 "-" chain 264: thread 101367 (pid 19734, zio_free_issue_2_4) sleeping on 0xfffff80808392500 "-" chain 265: thread 101368 (pid 19734, zio_free_issue_2_5) sleeping on 0xfffff80808392500 "-" chain 266: thread 101369 (pid 19734, zio_free_issue_2_6) sleeping on 0xfffff80808392500 "-" chain 267: thread 101370 (pid 19734, zio_free_issue_2_7) sleeping on 0xfffff80808392500 "-" chain 268: thread 101371 (pid 19734, zio_free_issue_2_8) sleeping on 0xfffff80808392500 "-" chain 269: thread 101372 (pid 19734, zio_free_issue_2_9) sleeping on 0xfffff80808392500 "-" chain 270: thread 101373 (pid 19734, zio_free_issue_2_10) sleeping on 0xfffff80808392500 "-" chain 271: thread 101374 (pid 19734, zio_free_issue_2_11) sleeping on 0xfffff80808392500 "-" chain 272: thread 101375 (pid 19734, zio_free_issue_3_0) sleeping on 0xfffff803ed7bab00 "-" chain 273: thread 101376 (pid 19734, zio_free_issue_3_1) sleeping on 0xfffff803ed7bab00 "-" chain 274: thread 101377 (pid 19734, zio_free_issue_3_2) sleeping on 0xfffff803ed7bab00 "-" chain 275: thread 101378 (pid 19734, zio_free_issue_3_3) sleeping on 0xfffff803ed7bab00 "-" chain 276: thread 101379 (pid 19734, zio_free_issue_3_4) sleeping on 0xfffff803ed7bab00 "-" chain 277: thread 101380 (pid 19734, zio_free_issue_3_5) sleeping on 0xfffff803ed7bab00 "-" chain 278: thread 101381 (pid 19734, zio_free_issue_3_6) sleeping on 0xfffff803ed7bab00 "-" chain 279: thread 101382 (pid 19734, zio_free_issue_3_7) sleeping on 0xfffff803ed7bab00 "-" chain 280: thread 101383 (pid 19734, zio_free_issue_3_8) sleeping on 0xfffff803ed7bab00 "-" chain 281: thread 101384 (pid 19734, zio_free_issue_3_9) sleeping on 0xfffff803ed7bab00 "-" chain 282: thread 101385 (pid 19734, zio_free_issue_3_10) sleeping on 0xfffff803ed7bab00 "-" chain 283: thread 101386 (pid 19734, zio_free_issue_3_11) sleeping on 0xfffff803ed7bab00 "-" chain 284: thread 101388 (pid 19734, zio_free_issue_4_0) sleeping on 0xfffff800038cb900 "-" chain 285: thread 101389 (pid 19734, zio_free_issue_4_1) sleeping on 0xfffff800038cb900 "-" chain 286: thread 101390 (pid 19734, zio_free_issue_4_2) sleeping on 0xfffff800038cb900 "-" chain 287: thread 101391 (pid 19734, zio_free_issue_4_3) sleeping on 0xfffff800038cb900 "-" chain 288: thread 101392 (pid 19734, zio_free_issue_4_4) sleeping on 0xfffff800038cb900 "-" chain 289: thread 101393 (pid 19734, zio_free_issue_4_5) sleeping on 0xfffff800038cb900 "-" chain 290: thread 101394 (pid 19734, zio_free_issue_4_6) sleeping on 0xfffff800038cb900 "-" chain 291: thread 101395 (pid 19734, zio_free_issue_4_7) sleeping on 0xfffff800038cb900 "-" chain 292: thread 101397 (pid 19734, zio_free_issue_4_8) sleeping on 0xfffff800038cb900 "-" chain 293: thread 101399 (pid 19734, zio_free_issue_4_9) sleeping on 0xfffff800038cb900 "-" chain 294: thread 101401 (pid 19734, zio_free_issue_4_10) sleeping on 0xfffff800038cb900 "-" chain 295: thread 101403 (pid 19734, zio_free_issue_4_11) sleeping on 0xfffff800038cb900 "-" chain 296: thread 101404 (pid 19734, zio_free_issue_5_0) sleeping on 0xfffff805998b1a00 "-" chain 297: thread 101406 (pid 19734, zio_free_issue_5_1) sleeping on 0xfffff805998b1a00 "-" chain 298: thread 101408 (pid 19734, zio_free_issue_5_2) sleeping on 0xfffff805998b1a00 "-" chain 299: thread 101409 (pid 19734, zio_free_issue_5_3) sleeping on 0xfffff805998b1a00 "-" chain 300: thread 101414 (pid 19734, zio_free_issue_5_4) sleeping on 0xfffff805998b1a00 "-" chain 301: thread 101415 (pid 19734, zio_free_issue_5_5) sleeping on 0xfffff805998b1a00 "-" chain 302: thread 101416 (pid 19734, zio_free_issue_5_6) sleeping on 0xfffff805998b1a00 "-" chain 303: thread 101417 (pid 19734, zio_free_issue_5_7) sleeping on 0xfffff805998b1a00 "-" chain 304: thread 101421 (pid 19734, zio_free_issue_5_8) sleeping on 0xfffff805998b1a00 "-" chain 305: thread 101424 (pid 19734, zio_free_issue_5_9) sleeping on 0xfffff805998b1a00 "-" chain 306: thread 101429 (pid 19734, zio_free_issue_5_10) sleeping on 0xfffff805998b1a00 "-" chain 307: thread 101430 (pid 19734, zio_free_issue_5_11) sleeping on 0xfffff805998b1a00 "-" chain 308: thread 101431 (pid 19734, zio_free_issue_6_0) sleeping on 0xfffff801f90c5a00 "-" chain 309: thread 101433 (pid 19734, zio_free_issue_6_1) sleeping on 0xfffff801f90c5a00 "-" chain 310: thread 101437 (pid 19734, zio_free_issue_6_2) sleeping on 0xfffff801f90c5a00 "-" chain 311: thread 101439 (pid 19734, zio_free_issue_6_3) sleeping on 0xfffff801f90c5a00 "-" chain 312: thread 101444 (pid 19734, zio_free_issue_6_4) sleeping on 0xfffff801f90c5a00 "-" chain 313: thread 101445 (pid 19734, zio_free_issue_6_5) sleeping on 0xfffff801f90c5a00 "-" chain 314: thread 101446 (pid 19734, zio_free_issue_6_6) sleeping on 0xfffff801f90c5a00 "-" chain 315: thread 101449 (pid 19734, zio_free_issue_6_7) sleeping on 0xfffff801f90c5a00 "-" chain 316: thread 101452 (pid 19734, zio_free_issue_6_8) sleeping on 0xfffff801f90c5a00 "-" chain 317: thread 101453 (pid 19734, zio_free_issue_6_9) sleeping on 0xfffff801f90c5a00 "-" chain 318: thread 101454 (pid 19734, zio_free_issue_6_10) sleeping on 0xfffff801f90c5a00 "-" chain 319: thread 101456 (pid 19734, zio_free_issue_6_11) sleeping on 0xfffff801f90c5a00 "-" chain 320: thread 101464 (pid 19734, zio_free_issue_7_0) sleeping on 0xfffff80808417800 "-" chain 321: thread 101465 (pid 19734, zio_free_issue_7_1) sleeping on 0xfffff80808417800 "-" chain 322: thread 101467 (pid 19734, zio_free_issue_7_2) sleeping on 0xfffff80808417800 "-" chain 323: thread 101468 (pid 19734, zio_free_issue_7_3) sleeping on 0xfffff80808417800 "-" chain 324: thread 101470 (pid 19734, zio_free_issue_7_4) sleeping on 0xfffff80808417800 "-" chain 325: thread 101471 (pid 19734, zio_free_issue_7_5) sleeping on 0xfffff80808417800 "-" chain 326: thread 101472 (pid 19734, zio_free_issue_7_6) sleeping on 0xfffff80808417800 "-" chain 327: thread 101475 (pid 19734, zio_free_issue_7_7) sleeping on 0xfffff80808417800 "-" chain 328: thread 101479 (pid 19734, zio_free_issue_7_8) sleeping on 0xfffff80808417800 "-" chain 329: thread 101480 (pid 19734, zio_free_issue_7_9) sleeping on 0xfffff80808417800 "-" chain 330: thread 101483 (pid 19734, zio_free_issue_7_10) sleeping on 0xfffff80808417800 "-" chain 331: thread 101484 (pid 19734, zio_free_issue_7_11) sleeping on 0xfffff80808417800 "-" chain 332: thread 101486 (pid 19734, zio_free_intr) sleeping on 0xfffff805f1e98300 "-" chain 333: thread 101489 (pid 19734, zio_claim_issue) sleeping on 0xfffff804b891c700 "-" chain 334: thread 101491 (pid 19734, zio_claim_intr) sleeping on 0xfffff804b891c600 "-" chain 335: thread 101492 (pid 19734, zio_ioctl_issue) sleeping on 0xfffff805199f6800 "-" chain 336: thread 101495 (pid 19734, zio_ioctl_intr) sleeping on 0xfffff808083bfb00 "-" chain 337: thread 101565 (pid 19734, txg_thread_enter) sleeping on 0xfffff80454bf23c8 "tx->tx_cpu[c].tc_cv[i]" chain 338: thread 101566 (pid 19734, txg_thread_enter) sleeping on 0xfffff80800462ab8 "tx->tx_quiesce_done_cv" chain 339: thread 102523 (pid 19688, md11) sleeping on 0xfffff804e0a7e000 "mdwait" chain 340: thread 102214 (pid 19681, md10) sleeping on 0xfffff804e0c4a000 "mdwait" chain 341: thread 102267 (pid 19655, solthread 0xfffffff) sleeping on 0xfffff8004418a948 "t->zthr_cv" chain 342: thread 100932 (pid 19655, solthread 0xfffffff) sleeping on 0xfffff803b26a0c48 "t->zthr_cv" chain 343: thread 100934 (pid 19655, arc_dnlc_evicts_thr) sleeping on 0xffffffff8273e848 "arc_dnlc_evicts_cv" chain 344: thread 100935 (pid 19655, dbu_evict) sleeping on 0xfffff80205a0c800 "-" chain 345: thread 100936 (pid 19655, dbuf_evict_thread) sleeping on 0xffffffff82741fc8 "dbuf_evict_cv" chain 346: thread 100940 (pid 19655, z_vdev_file_0) sleeping on 0xfffff80808418a00 "-" chain 347: thread 100941 (pid 19655, z_vdev_file_1) sleeping on 0xfffff80808418a00 "-" chain 348: thread 100945 (pid 19655, z_vdev_file_2) sleeping on 0xfffff80808418a00 "-" chain 349: thread 100946 (pid 19655, z_vdev_file_3) sleeping on 0xfffff80808418a00 "-" chain 350: thread 100950 (pid 19655, z_vdev_file_4) sleeping on 0xfffff80808418a00 "-" chain 351: thread 100953 (pid 19655, z_vdev_file_5) sleeping on 0xfffff80808418a00 "-" chain 352: thread 100957 (pid 19655, z_vdev_file_6) sleeping on 0xfffff80808418a00 "-" chain 353: thread 100958 (pid 19655, z_vdev_file_7) sleeping on 0xfffff80808418a00 "-" chain 354: thread 100961 (pid 19655, z_vdev_file_8) sleeping on 0xfffff80808418a00 "-" chain 355: thread 100965 (pid 19655, z_vdev_file_9) sleeping on 0xfffff80808418a00 "-" chain 356: thread 100966 (pid 19655, z_vdev_file_10) sleeping on 0xfffff80808418a00 "-" chain 357: thread 100970 (pid 19655, z_vdev_file_11) sleeping on 0xfffff80808418a00 "-" chain 358: thread 100979 (pid 19655, z_vdev_file_12) sleeping on 0xfffff80808418a00 "-" chain 359: thread 100980 (pid 19655, z_vdev_file_13) sleeping on 0xfffff80808418a00 "-" chain 360: thread 100981 (pid 19655, z_vdev_file_14) sleeping on 0xfffff80808418a00 "-" chain 361: thread 100983 (pid 19655, z_vdev_file_15) sleeping on 0xfffff80808418a00 "-" chain 362: thread 100986 (pid 19655, l2arc_feed_thread) sleeping on 0xffffffff8273e9a0 "l2arc_feed_thr_cv" chain 363: thread 100993 (pid 19655, zfsvfs) sleeping on 0xfffff80685ceeb00 "-" chain 364: thread 101500 (pid 19655, trim stress2_tank) sleeping on 0xfffffe00d9fe6d28 "spa->spa_trim_cv" chain 365: thread 101502 (pid 19655, metaslab_group_task) sleeping on 0xfffff8064008d400 "-" chain 366: thread 101503 (pid 19655, metaslab_group_task) sleeping on 0xfffff8064008d400 "-" chain 367: thread 101507 (pid 19655, metaslab_group_task) sleeping on 0xfffff8064008d400 "-" chain 368: thread 101508 (pid 19655, metaslab_group_task) sleeping on 0xfffff8064008d400 "-" chain 369: thread 101509 (pid 19655, metaslab_group_task) sleeping on 0xfffff8064008d400 "-" chain 370: thread 101510 (pid 19655, metaslab_group_task) sleeping on 0xfffff8064008d400 "-" chain 371: thread 101514 (pid 19655, dp_sync_taskq_0) sleeping on 0xfffff808083bec00 "-" chain 372: thread 101519 (pid 19655, dp_sync_taskq_1) sleeping on 0xfffff808083bec00 "-" chain 373: thread 101520 (pid 19655, dp_sync_taskq_2) sleeping on 0xfffff808083bec00 "-" chain 374: thread 101524 (pid 19655, dp_sync_taskq_3) sleeping on 0xfffff808083bec00 "-" chain 375: thread 101525 (pid 19655, dp_sync_taskq_4) sleeping on 0xfffff808083bec00 "-" chain 376: thread 101529 (pid 19655, dp_sync_taskq_5) sleeping on 0xfffff808083bec00 "-" chain 377: thread 101533 (pid 19655, dp_sync_taskq_6) sleeping on 0xfffff808083bec00 "-" chain 378: thread 101534 (pid 19655, dp_sync_taskq_7) sleeping on 0xfffff808083bec00 "-" chain 379: thread 101538 (pid 19655, dp_sync_taskq_8) sleeping on 0xfffff808083bec00 "-" chain 380: thread 101541 (pid 19655, dp_zil_clean_taskq_) sleeping on 0xfffff80808392a00 "-" chain 381: thread 101542 (pid 19655, dp_zil_clean_taskq_) sleeping on 0xfffff80808392a00 "-" chain 382: thread 101543 (pid 19655, dp_zil_clean_taskq_) sleeping on 0xfffff80808392a00 "-" chain 383: thread 101544 (pid 19655, dp_zil_clean_taskq_) sleeping on 0xfffff80808392a00 "-" chain 384: thread 101545 (pid 19655, dp_zil_clean_taskq_) sleeping on 0xfffff80808392a00 "-" chain 385: thread 101546 (pid 19655, dp_zil_clean_taskq_) sleeping on 0xfffff80808392a00 "-" chain 386: thread 101552 (pid 19655, dp_zil_clean_taskq_) sleeping on 0xfffff80808392a00 "-" chain 387: thread 101554 (pid 19655, dp_zil_clean_taskq_) sleeping on 0xfffff80808392a00 "-" chain 388: thread 101557 (pid 19655, dp_zil_clean_taskq_) sleeping on 0xfffff80808392a00 "-" chain 389: thread 101558 (pid 19655, dp_zil_clean_taskq_) sleeping on 0xfffff80808392a00 "-" chain 390: thread 101560 (pid 19655, dp_zil_clean_taskq_) sleeping on 0xfffff80808392a00 "-" chain 391: thread 101561 (pid 19655, dp_zil_clean_taskq_) sleeping on 0xfffff80808392a00 "-" chain 392: thread 101562 (pid 19655, zfs_vn_rele_taskq) blocked on lockmgr (null)EXCL chain 393: thread 101570 (pid 19655, mmp_thread_enter) sleeping on 0xfffffe00d9fe6f00 "mmp->mmp_thread_cv" chain 394: thread 101571 (pid 19655, solthread 0xfffffff) sleeping on 0xfffff808088ac748 "t->zthr_cv" chain 395: thread 101574 (pid 19655, solthread 0xfffffff) sleeping on 0xfffff808088ac4c8 "t->zthr_cv" chain 396: thread 102125 (pid 19629, sh) sleeping on 0xfffff80109e04000 "wait" chain 397: thread 102698 (pid 19628, tee) sleeping on 0xfffff807080c5be0 "piperd" chain 398: thread 102247 (pid 19627, sh) sleeping on 0xfffff801a79be530 "wait" chain 399: thread 102534 (pid 124, bash) sleeping on 0xfffff800303878c0 "select" chain 400: thread 102816 (pid 99828, csh) sleeping on 0xfffff8053f1d40a8 "pause" chain 401: thread 102762 (pid 99824, su) sleeping on 0xfffff80174cfd000 "wait" chain 402: thread 100193 (pid 61272, bash) sleeping on 0xfffff80030ca5530 "wait" chain 403: thread 100197 (pid 61259, sshd) sleeping on 0xfffff800308ec3c0 "select" chain 404: thread 100343 (pid 61026, sshd) sleeping on 0xfffff8031802dc40 "select" chain 405: thread 100248 (pid 3000, sh) sleeping on 0xfffff80030ac65f0 "piperd" chain 406: thread 100227 (pid 2998, awk) sleeping on 0xfffff80030d7f8e8 "piperd" chain 407: thread 100237 (pid 2997, sh) sleeping on 0xfffff80030d7e530 "wait" chain 408: thread 100201 (pid 2930, sh) sleeping on 0xfffff80030f02000 "wait" chain 409: thread 100171 (pid 2490, sh) sleeping on 0xfffff80030e92530 "wait" chain 410: thread 100189 (pid 2388, csh) sleeping on 0xfffff801c098f8b0 "ttyin" chain 411: thread 100219 (pid 2387, su) sleeping on 0xfffff80030d85000 "wait" chain 412: thread 100220 (pid 2382, bash) sleeping on 0xfffff804f074da60 "wait" chain 413: thread 100200 (pid 2381, sshd) sleeping on 0xfffff800306cb240 "select" chain 414: thread 100173 (pid 2378, sshd) sleeping on 0xfffff8003061db40 "select" chain 415: thread 100184 (pid 2372, csh) sleeping on 0xfffff800054310b0 "ttyin" chain 416: thread 100142 (pid 2367, login) sleeping on 0xfffff800306a9000 "wait" chain 417: thread 100217 (pid 2366, getty) sleeping on 0xfffff800054314b0 "ttyin" chain 418: thread 100216 (pid 2365, getty) sleeping on 0xfffff80003bad0b0 "ttyin" chain 419: thread 100215 (pid 2364, getty) sleeping on 0xfffff80003bad4b0 "ttyin" chain 420: thread 100214 (pid 2363, getty) sleeping on 0xfffff80003bad8b0 "ttyin" chain 421: thread 100213 (pid 2362, getty) sleeping on 0xfffff80003badcb0 "ttyin" chain 422: thread 100212 (pid 2361, getty) sleeping on 0xfffff80003bac0b0 "ttyin" chain 423: thread 100211 (pid 2360, getty) sleeping on 0xfffff80003bac4b0 "ttyin" chain 424: thread 100210 (pid 2359, getty) sleeping on 0xfffff80003bac8b0 "ttyin" chain 425: thread 100186 (pid 2358, getty) sleeping on 0xfffff80003baccb0 "ttyin" chain 426: thread 100206 (pid 2312, cron) sleeping on 0xffffffff81c80275 "nanslp" chain 427: thread 100166 (pid 2308, sshd) sleeping on 0xfffff80030742c40 "select" chain 428: thread 100161 (pid 2280, powerd) sleeping on 0xfffff80003a34bc0 "select" chain 429: thread 100195 (pid 2157, watchdogd) sleeping on 0xffffffff81c80279 "nanslp" chain 430: thread 100170 (pid 2078, rpcbind) sleeping on 0xfffff800306cb440 "select" chain 431: thread 100159 (pid 2068, syslogd) sleeping on 0xfffff80044875040 "select" chain 432: thread 100191 (pid 1992, dhclient) sleeping on 0xfffff8003061de40 "select" chain 433: thread 100154 (pid 1928, dhclient) sleeping on 0xfffff800309b4740 "select" chain 434: thread 100149 (pid 1925, dhclient) sleeping on 0xfffff800309b47c0 "select" chain 435: thread 100141 (pid 34, vnlru) sleeping on 0xfffff800306a9530 "vlruwt" chain 436: thread 100140 (pid 33, syncer) sleeping on 0xffffffff81d56720 "syncer" chain 437: thread 100130 (pid 32, bufdaemon) sleeping on 0xffffffff81d55bd8 "qsleep" chain 438: thread 100131 (pid 32, bufspacedaemon-0) sleeping on 0xffffffff8180a980 "-" chain 439: thread 100132 (pid 32, bufspacedaemon-1) sleeping on 0xffffffff81812c40 "-" chain 440: thread 100133 (pid 32, bufspacedaemon-2) sleeping on 0xffffffff8181af00 "-" chain 441: thread 100134 (pid 32, bufspacedaemon-3) sleeping on 0xffffffff818231c0 "-" chain 442: thread 100135 (pid 32, bufspacedaemon-4) sleeping on 0xffffffff8182b480 "-" chain 443: thread 100136 (pid 32, bufspacedaemon-5) sleeping on 0xffffffff81833740 "-" chain 444: thread 100138 (pid 32, bufspacedaemon-6) sleeping on 0xffffffff8183ba00 "-" chain 445: thread 100152 (pid 32, / worker) sleeping on 0xfffff80030a2fce8 "sdflush" chain 446: thread 100203 (pid 32, /tmp worker) sleeping on 0xfffff80030caaee8 "sdflush" chain 447: thread 100129 (pid 31, vmdaemon) sleeping on 0xffffffff81d715c8 "psleep" chain 448: thread 100128 (pid 30, dom0) sleeping on 0xffffffff81d9e218 "psleep" chain 449: thread 100137 (pid 30, laundry: dom0) sleeping on 0xffffffff81d9e224 "launds" chain 450: thread 100139 (pid 30, uma) sleeping on 0xffffffff80f04b90 "umarcl" chain 451: thread 100127 (pid 29, enc_daemon1) sleeping on 0xfffff80005005530 "idle" chain 452: thread 100126 (pid 28, enc_daemon0) sleeping on 0xfffff80030527a60 "idle" chain 453: thread 100120 (pid 27, rand_harvestq) sleeping on 0xffffffff81adbf18 "-" chain 454: thread 100119 (pid 26, sctp_iterator) sleeping on 0xffffffff81de37e0 "waiting_for_work" chain 455: thread 100118 (pid 25, soaiod4) sleeping on 0xffffffff81d555dc "-" chain 456: thread 100117 (pid 24, soaiod3) sleeping on 0xffffffff81d555dc "-" chain 457: thread 100116 (pid 23, soaiod2) sleeping on 0xffffffff81d555dc "-" chain 458: thread 100115 (pid 22, soaiod1) sleeping on 0xffffffff81d555dc "-" chain 459: thread 100096 (pid 21, usbus0) sleeping on 0xfffffe00d1584460 "-" chain 460: thread 100097 (pid 21, usbus0) sleeping on 0xfffffe00d15844b8 "-" chain 461: thread 100098 (pid 21, usbus0) sleeping on 0xfffffe00d1584510 "-" chain 462: thread 100099 (pid 21, usbus0) sleeping on 0xfffffe00d1584568 "-" chain 463: thread 100100 (pid 21, usbus0) sleeping on 0xfffffe00d15845c0 "-" chain 464: thread 100102 (pid 21, usbus1) sleeping on 0xfffffe0037b56d10 "-" chain 465: thread 100103 (pid 21, usbus1) sleeping on 0xfffffe0037b56d68 "-" chain 466: thread 100104 (pid 21, usbus1) sleeping on 0xfffffe0037b56dc0 "-" chain 467: thread 100105 (pid 21, usbus1) sleeping on 0xfffffe0037b56e18 "-" chain 468: thread 100106 (pid 21, usbus1) sleeping on 0xfffffe0037b56e70 "-" chain 469: thread 100107 (pid 21, usbus2) sleeping on 0xfffffe00d16a7d10 "-" chain 470: thread 100108 (pid 21, usbus2) sleeping on 0xfffffe00d16a7d68 "-" chain 471: thread 100109 (pid 21, usbus2) sleeping on 0xfffffe00d16a7dc0 "-" chain 472: thread 100110 (pid 21, usbus2) sleeping on 0xfffffe00d16a7e18 "-" chain 473: thread 100111 (pid 21, usbus2) sleeping on 0xfffffe00d16a7e70 "-" chain 474: thread 100091 (pid 20, doneq0) sleeping on 0xffffffff819b72c0 "-" chain 475: thread 100093 (pid 20, doneq2) sleeping on 0xffffffff819b73c0 "-" chain 476: thread 100125 (pid 20, scanner) sleeping on 0xffffffff819b7188 "-" chain 477: thread 100090 (pid 19, crypto returns 11) sleeping on 0xfffff80003bb2c50 "crypto_ret_wait" chain 478: thread 100089 (pid 18, crypto returns 10) sleeping on 0xfffff80003bb2bf0 "crypto_ret_wait" chain 479: thread 100088 (pid 17, crypto returns 9) sleeping on 0xfffff80003bb2b90 "crypto_ret_wait" chain 480: thread 100087 (pid 16, crypto returns 8) sleeping on 0xfffff80003bb2b30 "crypto_ret_wait" chain 481: thread 100086 (pid 15, crypto returns 7) sleeping on 0xfffff80003bb2ad0 "crypto_ret_wait" chain 482: thread 100085 (pid 9, crypto returns 6) sleeping on 0xfffff80003bb2a70 "crypto_ret_wait" chain 483: thread 100084 (pid 8, crypto returns 5) sleeping on 0xfffff80003bb2a10 "crypto_ret_wait" chain 484: thread 100083 (pid 7, crypto returns 4) sleeping on 0xfffff80003bb29b0 "crypto_ret_wait" chain 485: thread 100082 (pid 6, crypto returns 3) sleeping on 0xfffff80003bb2950 "crypto_ret_wait" chain 486: thread 100081 (pid 5, crypto returns 2) sleeping on 0xfffff80003bb28f0 "crypto_ret_wait" chain 487: thread 100080 (pid 4, crypto returns 1) sleeping on 0xfffff80003bb2890 "crypto_ret_wait" chain 488: thread 100079 (pid 3, crypto returns 0) sleeping on 0xfffff80003bb2830 "crypto_ret_wait" chain 489: thread 100078 (pid 2, crypto) sleeping on 0xffffffff81d6b6a8 "crypto_wait" chain 490: thread 100065 (pid 14, sequencer 00) sleeping on 0xfffff80003a85888 "seqstate" chain 491: thread 100062 (pid 13, g_event) sleeping on 0xffffffff81d9c848 "-" chain 492: thread 100063 (pid 13, g_up) sleeping on 0xffffffff81d9c858 "-" chain 493: thread 100064 (pid 13, g_down) sleeping on 0xffffffff81d9c850 "-" chain 494: thread 100002 (pid 1, init) sleeping on 0xfffff80003850530 "wait" chain 495: thread 100001 (pid 10, audit) sleeping on 0xffffffff81de4470 "audit_worker_cv" chain 496: thread 100000 (pid 0, swapper) sleeping on 0xffffffff81d8b138 "swapin" chain 497: thread 100015 (pid 0, kqueue_ctx taskq) sleeping on 0xfffff800038cb000 "-" chain 498: thread 100016 (pid 0, aiod_kick taskq) sleeping on 0xfffff800038cae00 "-" chain 499: thread 100019 (pid 0, thread taskq) sleeping on 0xfffff800038cab00 "-" chain 500: thread 100021 (pid 0, config_0) sleeping on 0xfffff80003886000 "-" chain 501: thread 100022 (pid 0, if_config_tqg_0) sleeping on 0xfffff80003885e00 "-" chain 502: thread 100023 (pid 0, softirq_0) sleeping on 0xfffff80003885d00 "-" chain 503: thread 100024 (pid 0, softirq_1) sleeping on 0xfffff80003885c00 "-" chain 504: thread 100025 (pid 0, softirq_2) sleeping on 0xfffff80003885b00 "-" chain 505: thread 100026 (pid 0, softirq_3) sleeping on 0xfffff80003885a00 "-" chain 506: thread 100027 (pid 0, softirq_4) sleeping on 0xfffff80003885900 "-" chain 507: thread 100028 (pid 0, softirq_5) sleeping on 0xfffff80003885800 "-" chain 508: thread 100029 (pid 0, softirq_6) sleeping on 0xfffff80003885700 "-" chain 509: thread 100030 (pid 0, softirq_7) sleeping on 0xfffff80003885600 "-" chain 510: thread 100031 (pid 0, softirq_8) sleeping on 0xfffff80003885500 "-" chain 511: thread 100032 (pid 0, softirq_9) sleeping on 0xfffff80003885400 "-" chain 512: thread 100033 (pid 0, softirq_10) sleeping on 0xfffff80003885300 "-" chain 513: thread 100034 (pid 0, softirq_11) sleeping on 0xfffff80003885200 "-" chain 514: thread 100036 (pid 0, if_io_tqg_1) sleeping on 0xfffff80003885000 "-" chain 515: thread 100037 (pid 0, if_io_tqg_2) sleeping on 0xfffff80003884e00 "-" chain 516: thread 100038 (pid 0, if_io_tqg_3) sleeping on 0xfffff80003884d00 "-" chain 517: thread 100039 (pid 0, if_io_tqg_4) sleeping on 0xfffff80003884c00 "-" chain 518: thread 100040 (pid 0, if_io_tqg_5) sleeping on 0xfffff80003884b00 "-" chain 519: thread 100041 (pid 0, if_io_tqg_6) sleeping on 0xfffff80003884a00 "-" chain 520: thread 100042 (pid 0, if_io_tqg_7) sleeping on 0xfffff80003884900 "-" chain 521: thread 100043 (pid 0, if_io_tqg_8) sleeping on 0xfffff80003884800 "-" chain 522: thread 100044 (pid 0, if_io_tqg_9) sleeping on 0xfffff80003884700 "-" chain 523: thread 100045 (pid 0, if_io_tqg_10) sleeping on 0xfffff80003884600 "-" chain 524: thread 100046 (pid 0, if_io_tqg_11) sleeping on 0xfffff80003884500 "-" chain 525: thread 100061 (pid 0, firmware taskq) sleeping on 0xfffff80003b0c100 "-" chain 526: thread 100066 (pid 0, crypto_0) sleeping on 0xfffff80003b0c000 "-" chain 527: thread 100067 (pid 0, crypto_1) sleeping on 0xfffff80003b0c000 "-" chain 528: thread 100068 (pid 0, crypto_2) sleeping on 0xfffff80003b0c000 "-" chain 529: thread 100069 (pid 0, crypto_3) sleeping on 0xfffff80003b0c000 "-" chain 530: thread 100070 (pid 0, crypto_4) sleeping on 0xfffff80003b0c000 "-" chain 531: thread 100071 (pid 0, crypto_5) sleeping on 0xfffff80003b0c000 "-" chain 532: thread 100072 (pid 0, crypto_6) sleeping on 0xfffff80003b0c000 "-" chain 533: thread 100073 (pid 0, crypto_7) sleeping on 0xfffff80003b0c000 "-" chain 534: thread 100074 (pid 0, crypto_8) sleeping on 0xfffff80003b0c000 "-" chain 535: thread 100075 (pid 0, crypto_9) sleeping on 0xfffff80003b0c000 "-" chain 536: thread 100076 (pid 0, crypto_10) sleeping on 0xfffff80003b0c000 "-" chain 537: thread 100077 (pid 0, crypto_11) sleeping on 0xfffff80003b0c000 "-" chain 538: thread 100114 (pid 0, mca taskq) sleeping on 0xfffff8000546b200 "-" chain 539: thread 100121 (pid 0, acpi_task_0) sleeping on 0xfffff80003b0dd00 "-" chain 540: thread 100122 (pid 0, acpi_task_1) sleeping on 0xfffff80003b0dd00 "-" chain 541: thread 100123 (pid 0, acpi_task_2) sleeping on 0xfffff80003b0dd00 "-" chain 542: thread 100124 (pid 0, CAM taskq) sleeping on 0xfffff80003b0de00 "-" chain 543: thread 100821 (pid 0, system_taskq_0) sleeping on 0xfffff80685cefd00 "-" chain 544: thread 100843 (pid 0, system_taskq_1) sleeping on 0xfffff80685cefd00 "-" chain 545: thread 100847 (pid 0, system_taskq_2) sleeping on 0xfffff80685cefd00 "-" chain 546: thread 100848 (pid 0, system_taskq_3) sleeping on 0xfffff80685cefd00 "-" chain 547: thread 100849 (pid 0, system_taskq_4) sleeping on 0xfffff80685cefd00 "-" chain 548: thread 100850 (pid 0, system_taskq_5) sleeping on 0xfffff80685cefd00 "-" chain 549: thread 100862 (pid 0, system_taskq_6) sleeping on 0xfffff80685cefd00 "-" chain 550: thread 100863 (pid 0, system_taskq_7) sleeping on 0xfffff80685cefd00 "-" chain 551: thread 100867 (pid 0, system_taskq_8) sleeping on 0xfffff80685cefd00 "-" chain 552: thread 100869 (pid 0, system_taskq_9) sleeping on 0xfffff80685cefd00 "-" chain 553: thread 100871 (pid 0, system_taskq_10) sleeping on 0xfffff80685cefd00 "-" chain 554: thread 100878 (pid 0, system_taskq_11) sleeping on 0xfffff80685cefd00 "-" chain 555: thread 100879 (pid 0, arc_prune_0) sleeping on 0xfffff80041008d00 "-" chain 556: thread 100884 (pid 0, arc_prune_1) sleeping on 0xfffff80041008d00 "-" chain 557: thread 100885 (pid 0, arc_prune_2) sleeping on 0xfffff80041008d00 "-" chain 558: thread 100890 (pid 0, arc_prune_3) sleeping on 0xfffff80041008d00 "-" chain 559: thread 100893 (pid 0, arc_prune_4) sleeping on 0xfffff80041008d00 "-" chain 560: thread 100896 (pid 0, arc_prune_5) sleeping on 0xfffff80041008d00 "-" chain 561: thread 100897 (pid 0, arc_prune_6) sleeping on 0xfffff80041008d00 "-" chain 562: thread 100898 (pid 0, arc_prune_7) sleeping on 0xfffff80041008d00 "-" chain 563: thread 100902 (pid 0, arc_prune_8) sleeping on 0xfffff80041008d00 "-" chain 564: thread 100914 (pid 0, arc_prune_9) sleeping on 0xfffff80041008d00 "-" chain 565: thread 100920 (pid 0, arc_prune_10) sleeping on 0xfffff80041008d00 "-" chain 566: thread 100921 (pid 0, arc_prune_11) sleeping on 0xfffff80041008d00 "-" db:0:pho> chain 1: thread 100312 (pid 14234, sleep) sleeping on 0xffffffff81c80274 "nanslp" chain 2: thread 100190 (pid 30742, cat) sleeping on 0xfffff801a09155f0 "piperd" chain 3: thread 101842 (pid 30740, sh) sleeping on 0xfffff80546669530 "wait" chain 4: thread 102340 (pid 30739, find) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 5: thread 102688 (pid 30735, sh) sleeping on 0xfffff8004d2e4a60 "wait" chain 6: thread 100695 (pid 30734, mail) sleeping on 0xfffff800441922f8 "piperd" chain 7: thread 100610 (pid 30733, sh) sleeping on 0xfffff801ac3d5000 "wait" chain 8: thread 100473 (pid 30726, sh) sleeping on 0xfffff8023b0e3000 "wait" chain 9: thread 102266 (pid 30725, lockf) sleeping on 0xfffff803dc042a60 "wait" chain 10: thread 102417 (pid 30723, sh) sleeping on 0xfffff80190c2e000 "wait" chain 11: thread 100778 (pid 30722, sh) sleeping on 0xfffff802a0751000 "wait" chain 12: thread 102268 (pid 27785, mail) sleeping on 0xfffff806f8a87000 "piperd" chain 13: thread 102829 (pid 27784, sh) sleeping on 0xfffff8077378a000 "wait" chain 14: thread 102234 (pid 27776, sh) sleeping on 0xfffff80796c1da60 "wait" chain 15: thread 100555 (pid 27775, lockf) blocked on lockmgr `EXCL thread 0 (pid -512, ) is inactive chain 16: thread 100574 (pid 27773, sh) sleeping on 0xfffff801b195a000 "wait" chain 17: thread 102531 (pid 27771, cron) sleeping on 0xfffff8021727b2f8 "piperd" chain 18: thread 101994 (pid 20682, sh) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 19: thread 100145 (pid 20486, sh) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 20: thread 100547 (pid 20414, sh) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 21: thread 100339 (pid 20407, fstat) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 22: thread 101844 (pid 20406, sh) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 23: thread 101960 (pid 20380, sh) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 24: thread 102304 (pid 20345, fstat) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 25: thread 100609 (pid 20344, fstat) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 26: thread 100550 (pid 20250, fstat) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 27: thread 102164 (pid 20241, fstat) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 28: thread 100608 (pid 20237, fstat) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 29: thread 102491 (pid 20230, fstat) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 30: thread 101893 (pid 19977, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 31: thread 100407 (pid 19976, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 32: thread 102149 (pid 19975, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 33: thread 102190 (pid 19974, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 34: thread 100749 (pid 19973, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 35: thread 100185 (pid 19972, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 36: thread 100321 (pid 19971, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 37: thread 102513 (pid 19970, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 38: thread 102516 (pid 19969, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 39: thread 102236 (pid 19968, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 40: thread 100675 (pid 19967, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 41: thread 102298 (pid 19966, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 42: thread 100298 (pid 19965, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 43: thread 102371 (pid 19964, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 44: thread 100614 (pid 19963, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 45: thread 102150 (pid 19962, dirnprename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 46: thread 100268 (pid 19960, dirnprename) sleeping on 0xfffff80044126000 "wait" chain 47: thread 100162 (pid 19957, sysctl) sleeping on 0xfffff80030527530 "wait" chain 48: thread 102765 (pid 19956, sysctl) sleeping on 0xfffff804f36ba000 "wait" chain 49: thread 102763 (pid 19955, sysctl) sleeping on 0xfffff804f36ba530 "wait" chain 50: thread 102761 (pid 19954, sysctl) sleeping on 0xfffff804f36baa60 "wait" chain 51: thread 103055 (pid 19953, sysctl) sleeping on 0xfffff804f36d6a60 "wait" chain 52: thread 100387 (pid 19952, sysctl) sleeping on 0xfffff80050d1f530 "wait" chain 53: thread 102902 (pid 19951, sysctl) sleeping on 0xfffff8002e156a60 "wait" chain 54: thread 100676 (pid 19950, sysctl) sleeping on 0xfffff806e40e1530 "wait" chain 55: thread 102119 (pid 19949, sysctl) sleeping on 0xfffff80109e04530 "wait" chain 56: thread 102241 (pid 19948, sysctl) sleeping on 0xfffff80796c1d000 "wait" chain 57: thread 102937 (pid 19947, sysctl) blocked on lockmgr `ZM05jEXCL thread 0 (pid -512, ) is inactive chain 58: thread 102259 (pid 19946, sysctl) sleeping on 0xfffff8053f130530 "wait" chain 59: thread 102290 (pid 19945, sysctl) sleeping on 0xfffff8039a4d5a60 "wait" chain 60: thread 100789 (pid 19944, dirrename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 61: thread 102488 (pid 19943, dirrename) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 62: thread 102590 (pid 19942, dirrename) sleeping on 0xfffff80189fbb530 "wait" chain 63: thread 100144 (pid 19909, mkdir) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 64: thread 102349 (pid 19906, mkdir) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 65: thread 102732 (pid 19904, mkdir) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 66: thread 101957 (pid 19902, mkdir) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 67: thread 101956 (pid 19900, mkdir) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 68: thread 100554 (pid 19899, mkdir) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 69: thread 102065 (pid 19897, mkdir) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 70: thread 100342 (pid 19896, mkdir) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 71: thread 102376 (pid 19895, mkdir) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 72: thread 102347 (pid 19894, mkdir) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 73: thread 103019 (pid 19893, mkdir) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 74: thread 101598 (pid 19892, mkdir) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 75: thread 101916 (pid 19891, mkdir) sleeping on 0xfffff802f9fa2000 "wait" chain 76: thread 102533 (pid 19890, link) sleeping on 0xfffff808006e41d0 "dn->dn_notxholds" chain 77: thread 102381 (pid 19889, link) sleeping on 0xfffff808006e41d0 "dn->dn_notxholds" chain 78: thread 102281 (pid 19887, link) sleeping on 0xfffff808006e41d0 "dn->dn_notxholds" chain 79: thread 102213 (pid 19885, link) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 80: thread 102161 (pid 19884, lockf) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 81: thread 100160 (pid 19883, link) sleeping on 0xfffff808006e41d0 "dn->dn_notxholds" chain 82: thread 102373 (pid 19882, lockf) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 83: thread 102938 (pid 19881, link) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 84: thread 102489 (pid 19880, lockf) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 85: thread 100679 (pid 19879, link) sleeping on 0xfffff808006e41d0 "dn->dn_notxholds" chain 86: thread 101843 (pid 19878, lockf) blocked on lockmgr zfsEXCL thread 102163 (pid 19961, dirnprename) running on CPU 2 chain 87: thread 102406 (pid 19877, link) sleeping on 0xfffff808006e41d0 "dn->dn_notxholds" chain 88: thread 100528 (pid 19876, lockf) sleeping on 0xfffff801b11fe000 "wait" chain 89: thread 101597 (pid 19875, link) sleeping on 0xfffff80174c75a60 "wait" chain 90: thread 102470 (pid 19870, dirrename) sleeping on 0xfffff803dc09e000 "wait" chain 91: thread 102021 (pid 19868, dirnprename) sleeping on 0xfffff80432c95000 "wait" chain 92: thread 102445 (pid 19859, openat) sleeping on 0xfffff804f3605a60 "wait" chain 93: thread 100602 (pid 19857, mkdir) sleeping on 0xfffff80191cd4530 "wait" chain 94: thread 101981 (pid 19855, lockf) sleeping on 0xfffff8004d208a60 "wait" chain 95: thread 101958 (pid 19854, link) sleeping on 0xfffff806bdf2a000 "wait" chain 96: thread 101959 (pid 19847, openat) sleeping on 0xfffff80109ef8000 "wait" chain 97: thread 100221 (pid 19843, sysctl) sleeping on 0xfffff80044126a60 "wait" chain 98: thread 101846 (pid 19838, run) sleeping on 0xfffff8015cff8000 "wait" chain 99: thread 100187 (pid 19836, run) sleeping on 0xfffff80030527000 "wait" chain 100: thread 100332 (pid 19834, run) sleeping on 0xfffff802b8f2d530 "wait" chain 101: thread 100777 (pid 19810, sh) sleeping on 0xfffff802a0751530 "wait" chain 102: thread 101835 (pid 19734, stress2_tank) sleeping on 0xfffffe00d9fe6cd8 "spa->spa_proc_cv" chain 103: thread 100995 (pid 19734, zio_null_issue) sleeping on 0xfffff805f1e98100 "-" chain 104: thread 100998 (pid 19734, zio_null_intr) sleeping on 0xfffff80205a0d000 "-" chain 105: thread 101000 (pid 19734, zio_read_issue_0) sleeping on 0xfffff80205a0d500 "-" chain 106: thread 101003 (pid 19734, zio_read_issue_1) sleeping on 0xfffff80205a0d500 "-" chain 107: thread 101007 (pid 19734, zio_read_issue_2) sleeping on 0xfffff80205a0d500 "-" chain 108: thread 101009 (pid 19734, zio_read_issue_3) sleeping on 0xfffff80205a0d500 "-" chain 109: thread 101010 (pid 19734, zio_read_issue_4) sleeping on 0xfffff80205a0d500 "-" chain 110: thread 101012 (pid 19734, zio_read_issue_5) sleeping on 0xfffff80205a0d500 "-" chain 111: thread 101019 (pid 19734, zio_read_issue_6) sleeping on 0xfffff80205a0d500 "-" chain 112: thread 101022 (pid 19734, zio_read_issue_7) sleeping on 0xfffff80205a0d500 "-" chain 113: thread 101024 (pid 19734, zio_read_intr_0_0) sleeping on 0xfffff80808394600 "-" chain 114: thread 101026 (pid 19734, zio_read_intr_0_1) sleeping on 0xfffff80808394600 "-" chain 115: thread 101027 (pid 19734, zio_read_intr_0_2) sleeping on 0xfffff80808394600 "-" chain 116: thread 101036 (pid 19734, zio_read_intr_0_3) sleeping on 0xfffff80808394600 "-" chain 117: thread 101039 (pid 19734, zio_read_intr_0_4) sleeping on 0xfffff80808394600 "-" chain 118: thread 101040 (pid 19734, zio_read_intr_0_5) sleeping on 0xfffff80808394600 "-" chain 119: thread 101042 (pid 19734, zio_read_intr_0_6) sleeping on 0xfffff80808394600 "-" chain 120: thread 101043 (pid 19734, zio_read_intr_0_7) sleeping on 0xfffff80808394600 "-" chain 121: thread 101044 (pid 19734, zio_read_intr_0_8) sleeping on 0xfffff80808394600 "-" chain 122: thread 101051 (pid 19734, zio_read_intr_0_9) sleeping on 0xfffff80808394600 "-" chain 123: thread 101053 (pid 19734, zio_read_intr_0_10) sleeping on 0xfffff80808394600 "-" chain 124: thread 101057 (pid 19734, zio_read_intr_0_11) sleeping on 0xfffff80808394600 "-" chain 125: thread 101060 (pid 19734, zio_read_intr_1_0) sleeping on 0xfffff805be212100 "-" chain 126: thread 101061 (pid 19734, zio_read_intr_1_1) sleeping on 0xfffff805be212100 "-" chain 127: thread 101070 (pid 19734, zio_read_intr_1_2) sleeping on 0xfffff805be212100 "-" chain 128: thread 101071 (pid 19734, zio_read_intr_1_3) sleeping on 0xfffff805be212100 "-" chain 129: thread 101073 (pid 19734, zio_read_intr_1_4) sleeping on 0xfffff805be212100 "-" chain 130: thread 101074 (pid 19734, zio_read_intr_1_5) sleeping on 0xfffff805be212100 "-" chain 131: thread 101075 (pid 19734, zio_read_intr_1_6) sleeping on 0xfffff805be212100 "-" chain 132: thread 101077 (pid 19734, zio_read_intr_1_7) sleeping on 0xfffff805be212100 "-" chain 133: thread 101082 (pid 19734, zio_read_intr_1_8) sleeping on 0xfffff805be212100 "-" chain 134: thread 101083 (pid 19734, zio_read_intr_1_9) sleeping on 0xfffff805be212100 "-" chain 135: thread 101087 (pid 19734, zio_read_intr_1_10) sleeping on 0xfffff805be212100 "-" chain 136: thread 101088 (pid 19734, zio_read_intr_1_11) sleeping on 0xfffff805be212100 "-" chain 137: thread 101089 (pid 19734, zio_read_intr_2_0) sleeping on 0xfffff801f90ca000 "-" chain 138: thread 101092 (pid 19734, zio_read_intr_2_1) sleeping on 0xfffff801f90ca000 "-" chain 139: thread 101096 (pid 19734, zio_read_intr_2_2) sleeping on 0xfffff801f90ca000 "-" chain 140: thread 101098 (pid 19734, zio_read_intr_2_3) sleeping on 0xfffff801f90ca000 "-" chain 141: thread 101102 (pid 19734, zio_read_intr_2_4) sleeping on 0xfffff801f90ca000 "-" chain 142: thread 101107 (pid 19734, zio_read_intr_2_5) sleeping on 0xfffff801f90ca000 "-" chain 143: thread 101108 (pid 19734, zio_read_intr_2_6) sleeping on 0xfffff801f90ca000 "-" chain 144: thread 101110 (pid 19734, zio_read_intr_2_7) sleeping on 0xfffff801f90ca000 "-" chain 145: thread 101113 (pid 19734, zio_read_intr_2_8) sleeping on 0xfffff801f90ca000 "-" chain 146: thread 101116 (pid 19734, zio_read_intr_2_9) sleeping on 0xfffff801f90ca000 "-" chain 147: thread 101117 (pid 19734, zio_read_intr_2_10) sleeping on 0xfffff801f90ca000 "-" chain 148: thread 101119 (pid 19734, zio_read_intr_2_11) sleeping on 0xfffff801f90ca000 "-" chain 149: thread 101121 (pid 19734, zio_read_intr_3_0) sleeping on 0xfffff80808393d00 "-" chain 150: thread 101128 (pid 19734, zio_read_intr_3_1) sleeping on 0xfffff80808393d00 "-" chain 151: thread 101129 (pid 19734, zio_read_intr_3_2) sleeping on 0xfffff80808393d00 "-" chain 152: thread 101132 (pid 19734, zio_read_intr_3_3) sleeping on 0xfffff80808393d00 "-" chain 153: thread 101137 (pid 19734, zio_read_intr_3_4) sleeping on 0xfffff80808393d00 "-" chain 154: thread 101138 (pid 19734, zio_read_intr_3_5) sleeping on 0xfffff80808393d00 "-" chain 155: thread 101143 (pid 19734, zio_read_intr_3_6) sleeping on 0xfffff80808393d00 "-" chain 156: thread 101145 (pid 19734, zio_read_intr_3_7) sleeping on 0xfffff80808393d00 "-" chain 157: thread 101146 (pid 19734, zio_read_intr_3_8) sleeping on 0xfffff80808393d00 "-" chain 158: thread 101147 (pid 19734, zio_read_intr_3_9) sleeping on 0xfffff80808393d00 "-" chain 159: thread 101148 (pid 19734, zio_read_intr_3_10) sleeping on 0xfffff80808393d00 "-" chain 160: thread 101152 (pid 19734, zio_read_intr_3_11) sleeping on 0xfffff80808393d00 "-" chain 161: thread 101159 (pid 19734, zio_read_intr_4_0) sleeping on 0xfffff80464585c00 "-" chain 162: thread 101161 (pid 19734, zio_read_intr_4_1) sleeping on 0xfffff80464585c00 "-" chain 163: thread 101163 (pid 19734, zio_read_intr_4_2) sleeping on 0xfffff80464585c00 "-" chain 164: thread 101164 (pid 19734, zio_read_intr_4_3) sleeping on 0xfffff80464585c00 "-" chain 165: thread 101168 (pid 19734, zio_read_intr_4_4) sleeping on 0xfffff80464585c00 "-" chain 166: thread 101169 (pid 19734, zio_read_intr_4_5) sleeping on 0xfffff80464585c00 "-" chain 167: thread 101174 (pid 19734, zio_read_intr_4_6) sleeping on 0xfffff80464585c00 "-" chain 168: thread 101176 (pid 19734, zio_read_intr_4_7) sleeping on 0xfffff80464585c00 "-" chain 169: thread 101177 (pid 19734, zio_read_intr_4_8) sleeping on 0xfffff80464585c00 "-" chain 170: thread 101180 (pid 19734, zio_read_intr_4_9) sleeping on 0xfffff80464585c00 "-" chain 171: thread 101182 (pid 19734, zio_read_intr_4_10) sleeping on 0xfffff80464585c00 "-" chain 172: thread 101183 (pid 19734, zio_read_intr_4_11) sleeping on 0xfffff80464585c00 "-" chain 173: thread 101184 (pid 19734, zio_read_intr_5_0) sleeping on 0xfffff8049069e000 "-" chain 174: thread 101189 (pid 19734, zio_read_intr_5_1) sleeping on 0xfffff8049069e000 "-" chain 175: thread 101191 (pid 19734, zio_read_intr_5_2) sleeping on 0xfffff8049069e000 "-" chain 176: thread 101194 (pid 19734, zio_read_intr_5_3) sleeping on 0xfffff8049069e000 "-" chain 177: thread 101198 (pid 19734, zio_read_intr_5_4) sleeping on 0xfffff8049069e000 "-" chain 178: thread 101199 (pid 19734, zio_read_intr_5_5) sleeping on 0xfffff8049069e000 "-" chain 179: thread 101200 (pid 19734, zio_read_intr_5_6) sleeping on 0xfffff8049069e000 "-" chain 180: thread 101202 (pid 19734, zio_read_intr_5_7) sleeping on 0xfffff8049069e000 "-" chain 181: thread 101203 (pid 19734, zio_read_intr_5_8) sleeping on 0xfffff8049069e000 "-" chain 182: thread 101204 (pid 19734, zio_read_intr_5_9) sleeping on 0xfffff8049069e000 "-" chain 183: thread 101206 (pid 19734, zio_read_intr_5_10) sleeping on 0xfffff8049069e000 "-" chain 184: thread 101208 (pid 19734, zio_read_intr_5_11) sleeping on 0xfffff8049069e000 "-" chain 185: thread 101209 (pid 19734, zio_read_intr_6_0) sleeping on 0xfffff807fd530e00 "-" chain 186: thread 101210 (pid 19734, zio_read_intr_6_1) sleeping on 0xfffff807fd530e00 "-" chain 187: thread 101213 (pid 19734, zio_read_intr_6_2) sleeping on 0xfffff807fd530e00 "-" chain 188: thread 101214 (pid 19734, zio_read_intr_6_3) sleeping on 0xfffff807fd530e00 "-" chain 189: thread 101215 (pid 19734, zio_read_intr_6_4) sleeping on 0xfffff807fd530e00 "-" chain 190: thread 101218 (pid 19734, zio_read_intr_6_5) sleeping on 0xfffff807fd530e00 "-" chain 191: thread 101219 (pid 19734, zio_read_intr_6_6) sleeping on 0xfffff807fd530e00 "-" chain 192: thread 101220 (pid 19734, zio_read_intr_6_7) sleeping on 0xfffff807fd530e00 "-" chain 193: thread 101221 (pid 19734, zio_read_intr_6_8) sleeping on 0xfffff807fd530e00 "-" chain 194: thread 101222 (pid 19734, zio_read_intr_6_9) sleeping on 0xfffff807fd530e00 "-" chain 195: thread 101224 (pid 19734, zio_read_intr_6_10) sleeping on 0xfffff807fd530e00 "-" chain 196: thread 101225 (pid 19734, zio_read_intr_6_11) sleeping on 0xfffff807fd530e00 "-" chain 197: thread 101226 (pid 19734, zio_read_intr_7_0) sleeping on 0xfffff804b891a700 "-" chain 198: thread 101230 (pid 19734, zio_read_intr_7_1) sleeping on 0xfffff804b891a700 "-" chain 199: thread 101234 (pid 19734, zio_read_intr_7_2) sleeping on 0xfffff804b891a700 "-" chain 200: thread 101235 (pid 19734, zio_read_intr_7_3) sleeping on 0xfffff804b891a700 "-" chain 201: thread 101236 (pid 19734, zio_read_intr_7_4) sleeping on 0xfffff804b891a700 "-" chain 202: thread 101237 (pid 19734, zio_read_intr_7_5) sleeping on 0xfffff804b891a700 "-" chain 203: thread 101238 (pid 19734, zio_read_intr_7_6) sleeping on 0xfffff804b891a700 "-" chain 204: thread 101252 (pid 19734, zio_read_intr_7_7) sleeping on 0xfffff804b891a700 "-" chain 205: thread 101253 (pid 19734, zio_read_intr_7_8) sleeping on 0xfffff804b891a700 "-" chain 206: thread 101263 (pid 19734, zio_read_intr_7_9) sleeping on 0xfffff804b891a700 "-" chain 207: thread 101264 (pid 19734, zio_read_intr_7_10) sleeping on 0xfffff804b891a700 "-" chain 208: thread 101265 (pid 19734, zio_read_intr_7_11) sleeping on 0xfffff804b891a700 "-" chain 209: thread 101266 (pid 19734, zio_write_issue_0) blocked on lockmgr (null)EXCL chain 210: thread 101268 (pid 19734, zio_write_issue_1) blocked on lockmgr (null)EXCL chain 211: thread 101270 (pid 19734, zio_write_issue_2) blocked on lockmgr (null)EXCL chain 212: thread 101271 (pid 19734, zio_write_issue_3) blocked on lockmgr (null)EXCL chain 213: thread 101273 (pid 19734, zio_write_issue_4) blocked on lockmgr (null)EXCL chain 214: thread 101274 (pid 19734, zio_write_issue_5) blocked on lockmgr (null)EXCL chain 215: thread 101275 (pid 19734, zio_write_issue_6) blocked on lockmgr (null)EXCL chain 216: thread 101277 (pid 19734, zio_write_issue_7) blocked on lockmgr (null)EXCL chain 217: thread 101279 (pid 19734, zio_write_issue_8) blocked on lockmgr (null)EXCL chain 218: thread 101280 (pid 19734, zio_write_issue_hig) sleeping on 0xfffff8064008dd00 "-" chain 219: thread 101281 (pid 19734, zio_write_issue_hig) sleeping on 0xfffff8064008dd00 "-" chain 220: thread 101283 (pid 19734, zio_write_issue_hig) sleeping on 0xfffff8064008dd00 "-" chain 221: thread 101284 (pid 19734, zio_write_issue_hig) sleeping on 0xfffff8064008dd00 "-" chain 222: thread 101286 (pid 19734, zio_write_issue_hig) sleeping on 0xfffff8064008dd00 "-" chain 223: thread 101287 (pid 19734, zio_write_intr_0) sleeping on 0xfffff805be212c00 "-" chain 224: thread 101288 (pid 19734, zio_write_intr_1) sleeping on 0xfffff805be212c00 "-" chain 225: thread 101291 (pid 19734, zio_write_intr_2) sleeping on 0xfffff805be212c00 "-" chain 226: thread 101293 (pid 19734, zio_write_intr_3) sleeping on 0xfffff805be212c00 "-" chain 227: thread 101294 (pid 19734, zio_write_intr_4) sleeping on 0xfffff805be212c00 "-" chain 228: thread 101295 (pid 19734, zio_write_intr_5) sleeping on 0xfffff805be212c00 "-" chain 229: thread 101297 (pid 19734, zio_write_intr_6) sleeping on 0xfffff805be212c00 "-" chain 230: thread 101298 (pid 19734, zio_write_intr_7) sleeping on 0xfffff805be212c00 "-" chain 231: thread 101299 (pid 19734, zio_write_intr_high) sleeping on 0xfffff8047ed3bc00 "-" chain 232: thread 101301 (pid 19734, zio_write_intr_high) sleeping on 0xfffff8047ed3bc00 "-" chain 233: thread 101302 (pid 19734, zio_write_intr_high) sleeping on 0xfffff8047ed3bc00 "-" chain 234: thread 101303 (pid 19734, zio_write_intr_high) sleeping on 0xfffff8047ed3bc00 "-" chain 235: thread 101304 (pid 19734, zio_write_intr_high) sleeping on 0xfffff8047ed3bc00 "-" chain 236: thread 101306 (pid 19734, zio_free_issue_0_0) sleeping on 0xfffff800038cb200 "-" chain 237: thread 101307 (pid 19734, zio_free_issue_0_1) sleeping on 0xfffff800038cb200 "-" chain 238: thread 101308 (pid 19734, zio_free_issue_0_2) sleeping on 0xfffff800038cb200 "-" chain 239: thread 101310 (pid 19734, zio_free_issue_0_3) sleeping on 0xfffff800038cb200 "-" chain 240: thread 101311 (pid 19734, zio_free_issue_0_4) sleeping on 0xfffff800038cb200 "-" chain 241: thread 101312 (pid 19734, zio_free_issue_0_5) sleeping on 0xfffff800038cb200 "-" chain 242: thread 101314 (pid 19734, zio_free_issue_0_6) sleeping on 0xfffff800038cb200 "-" chain 243: thread 101315 (pid 19734, zio_free_issue_0_7) sleeping on 0xfffff800038cb200 "-" chain 244: thread 101316 (pid 19734, zio_free_issue_0_8) sleeping on 0xfffff800038cb200 "-" chain 245: thread 101317 (pid 19734, zio_free_issue_0_9) sleeping on 0xfffff800038cb200 "-" chain 246: thread 101319 (pid 19734, zio_free_issue_0_10) sleeping on 0xfffff800038cb200 "-" chain 247: thread 101320 (pid 19734, zio_free_issue_0_11) sleeping on 0xfffff800038cb200 "-" chain 248: thread 101321 (pid 19734, zio_free_issue_1_0) sleeping on 0xfffff8080841ac00 "-" chain 249: thread 101323 (pid 19734, zio_free_issue_1_1) sleeping on 0xfffff8080841ac00 "-" chain 250: thread 101326 (pid 19734, zio_free_issue_1_2) sleeping on 0xfffff8080841ac00 "-" chain 251: thread 101327 (pid 19734, zio_free_issue_1_3) sleeping on 0xfffff8080841ac00 "-" chain 252: thread 101329 (pid 19734, zio_free_issue_1_4) sleeping on 0xfffff8080841ac00 "-" chain 253: thread 101335 (pid 19734, zio_free_issue_1_5) sleeping on 0xfffff8080841ac00 "-" chain 254: thread 101336 (pid 19734, zio_free_issue_1_6) sleeping on 0xfffff8080841ac00 "-" chain 255: thread 101340 (pid 19734, zio_free_issue_1_7) sleeping on 0xfffff8080841ac00 "-" chain 256: thread 101343 (pid 19734, zio_free_issue_1_8) sleeping on 0xfffff8080841ac00 "-" chain 257: thread 101347 (pid 19734, zio_free_issue_1_9) sleeping on 0xfffff8080841ac00 "-" chain 258: thread 101349 (pid 19734, zio_free_issue_1_10) sleeping on 0xfffff8080841ac00 "-" chain 259: thread 101351 (pid 19734, zio_free_issue_1_11) sleeping on 0xfffff8080841ac00 "-" chain 260: thread 101356 (pid 19734, zio_free_issue_2_0) sleeping on 0xfffff80808392500 "-" chain 261: thread 101361 (pid 19734, zio_free_issue_2_1) sleeping on 0xfffff80808392500 "-" chain 262: thread 101365 (pid 19734, zio_free_issue_2_2) sleeping on 0xfffff80808392500 "-" chain 263: thread 101366 (pid 19734, zio_free_issue_2_3) sleeping on 0xfffff80808392500 "-" chain 264: thread 101367 (pid 19734, zio_free_issue_2_4) sleeping on 0xfffff80808392500 "-" chain 265: thread 101368 (pid 19734, zio_free_issue_2_5) sleeping on 0xfffff80808392500 "-" chain 266: thread 101369 (pid 19734, zio_free_issue_2_6) sleeping on 0xfffff80808392500 "-" chain 267: thread 101370 (pid 19734, zio_free_issue_2_7) sleeping on 0xfffff80808392500 "-" chain 268: thread 101371 (pid 19734, zio_free_issue_2_8) sleeping on 0xfffff80808392500 "-" chain 269: thread 101372 (pid 19734, zio_free_issue_2_9) sleeping on 0xfffff80808392500 "-" chain 270: thread 101373 (pid 19734, zio_free_issue_2_10) sleeping on 0xfffff80808392500 "-" chain 271: thread 101374 (pid 19734, zio_free_issue_2_11) sleeping on 0xfffff80808392500 "-" chain 272: thread 101375 (pid 19734, zio_free_issue_3_0) sleeping on 0xfffff803ed7bab00 "-" chain 273: thread 101376 (pid 19734, zio_free_issue_3_1) sleeping on 0xfffff803ed7bab00 "-" chain 274: thread 101377 (pid 19734, zio_free_issue_3_2) sleeping on 0xfffff803ed7bab00 "-" chain 275: thread 101378 (pid 19734, zio_free_issue_3_3) sleeping on 0xfffff803ed7bab00 "-" chain 276: thread 101379 (pid 19734, zio_free_issue_3_4) sleeping on 0xfffff803ed7bab00 "-" chain 277: thread 101380 (pid 19734, zio_free_issue_3_5) sleeping on 0xfffff803ed7bab00 "-" chain 278: thread 101381 (pid 19734, zio_free_issue_3_6) sleeping on 0xfffff803ed7bab00 "-" chain 279: thread 101382 (pid 19734, zio_free_issue_3_7) sleeping on 0xfffff803ed7bab00 "-" chain 280: thread 101383 (pid 19734, zio_free_issue_3_8) sleeping on 0xfffff803ed7bab00 "-" chain 281: thread 101384 (pid 19734, zio_free_issue_3_9) sleeping on 0xfffff803ed7bab00 "-" chain 282: thread 101385 (pid 19734, zio_free_issue_3_10) sleeping on 0xfffff803ed7bab00 "-" chain 283: thread 101386 (pid 19734, zio_free_issue_3_11) sleeping on 0xfffff803ed7bab00 "-" chain 284: thread 101388 (pid 19734, zio_free_issue_4_0) sleeping on 0xfffff800038cb900 "-" chain 285: thread 101389 (pid 19734, zio_free_issue_4_1) sleeping on 0xfffff800038cb900 "-" chain 286: thread 101390 (pid 19734, zio_free_issue_4_2) sleeping on 0xfffff800038cb900 "-" chain 287: thread 101391 (pid 19734, zio_free_issue_4_3) sleeping on 0xfffff800038cb900 "-" chain 288: thread 101392 (pid 19734, zio_free_issue_4_4) sleeping on 0xfffff800038cb900 "-" chain 289: thread 101393 (pid 19734, zio_free_issue_4_5) sleeping on 0xfffff800038cb900 "-" chain 290: thread 101394 (pid 19734, zio_free_issue_4_6) sleeping on 0xfffff800038cb900 "-" chain 291: thread 101395 (pid 19734, zio_free_issue_4_7) sleeping on 0xfffff800038cb900 "-" chain 292: thread 101397 (pid 19734, zio_free_issue_4_8) sleeping on 0xfffff800038cb900 "-" chain 293: thread 101399 (pid 19734, zio_free_issue_4_9) sleeping on 0xfffff800038cb900 "-" chain 294: thread 101401 (pid 19734, zio_free_issue_4_10) sleeping on 0xfffff800038cb900 "-" chain 295: thread 101403 (pid 19734, zio_free_issue_4_11) sleeping on 0xfffff800038cb900 "-" chain 296: thread 101404 (pid 19734, zio_free_issue_5_0) sleeping on 0xfffff805998b1a00 "-" chain 297: thread 101406 (pid 19734, zio_free_issue_5_1) sleeping on 0xfffff805998b1a00 "-" chain 298: thread 101408 (pid 19734, zio_free_issue_5_2) sleeping on 0xfffff805998b1a00 "-" chain 299: thread 101409 (pid 19734, zio_free_issue_5_3) sleeping on 0xfffff805998b1a00 "-" chain 300: thread 101414 (pid 19734, zio_free_issue_5_4) sleeping on 0xfffff805998b1a00 "-" chain 301: thread 101415 (pid 19734, zio_free_issue_5_5) sleeping on 0xfffff805998b1a00 "-" chain 302: thread 101416 (pid 19734, zio_free_issue_5_6) sleeping on 0xfffff805998b1a00 "-" chain 303: thread 101417 (pid 19734, zio_free_issue_5_7) sleeping on 0xfffff805998b1a00 "-" chain 304: thread 101421 (pid 19734, zio_free_issue_5_8) sleeping on 0xfffff805998b1a00 "-" chain 305: thread 101424 (pid 19734, zio_free_issue_5_9) sleeping on 0xfffff805998b1a00 "-" chain 306: thread 101429 (pid 19734, zio_free_issue_5_10) sleeping on 0xfffff805998b1a00 "-" chain 307: thread 101430 (pid 19734, zio_free_issue_5_11) sleeping on 0xfffff805998b1a00 "-" chain 308: thread 101431 (pid 19734, zio_free_issue_6_0) sleeping on 0xfffff801f90c5a00 "-" chain 309: thread 101433 (pid 19734, zio_free_issue_6_1) sleeping on 0xfffff801f90c5a00 "-" chain 310: thread 101437 (pid 19734, zio_free_issue_6_2) sleeping on 0xfffff801f90c5a00 "-" chain 311: thread 101439 (pid 19734, zio_free_issue_6_3) sleeping on 0xfffff801f90c5a00 "-" chain 312: thread 101444 (pid 19734, zio_free_issue_6_4) sleeping on 0xfffff801f90c5a00 "-" chain 313: thread 101445 (pid 19734, zio_free_issue_6_5) sleeping on 0xfffff801f90c5a00 "-" chain 314: thread 101446 (pid 19734, zio_free_issue_6_6) sleeping on 0xfffff801f90c5a00 "-" chain 315: thread 101449 (pid 19734, zio_free_issue_6_7) sleeping on 0xfffff801f90c5a00 "-" chain 316: thread 101452 (pid 19734, zio_free_issue_6_8) sleeping on 0xfffff801f90c5a00 "-" chain 317: thread 101453 (pid 19734, zio_free_issue_6_9) sleeping on 0xfffff801f90c5a00 "-" chain 318: thread 101454 (pid 19734, zio_free_issue_6_10) sleeping on 0xfffff801f90c5a00 "-" chain 319: thread 101456 (pid 19734, zio_free_issue_6_11) sleeping on 0xfffff801f90c5a00 "-" chain 320: thread 101464 (pid 19734, zio_free_issue_7_0) sleeping on 0xfffff80808417800 "-" chain 321: thread 101465 (pid 19734, zio_free_issue_7_1) sleeping on 0xfffff80808417800 "-" chain 322: thread 101467 (pid 19734, zio_free_issue_7_2) sleeping on 0xfffff80808417800 "-" chain 323: thread 101468 (pid 19734, zio_free_issue_7_3) sleeping on 0xfffff80808417800 "-" chain 324: thread 101470 (pid 19734, zio_free_issue_7_4) sleeping on 0xfffff80808417800 "-" chain 325: thread 101471 (pid 19734, zio_free_issue_7_5) sleeping on 0xfffff80808417800 "-" chain 326: thread 101472 (pid 19734, zio_free_issue_7_6) sleeping on 0xfffff80808417800 "-" chain 327: thread 101475 (pid 19734, zio_free_issue_7_7) sleeping on 0xfffff80808417800 "-" chain 328: thread 101479 (pid 19734, zio_free_issue_7_8) sleeping on 0xfffff80808417800 "-" chain 329: thread 101480 (pid 19734, zio_free_issue_7_9) sleeping on 0xfffff80808417800 "-" chain 330: thread 101483 (pid 19734, zio_free_issue_7_10) sleeping on 0xfffff80808417800 "-" chain 331: thread 101484 (pid 19734, zio_free_issue_7_11) sleeping on 0xfffff80808417800 "-" chain 332: thread 101486 (pid 19734, zio_free_intr) sleeping on 0xfffff805f1e98300 "-" chain 333: thread 101489 (pid 19734, zio_claim_issue) sleeping on 0xfffff804b891c700 "-" chain 334: thread 101491 (pid 19734, zio_claim_intr) sleeping on 0xfffff804b891c600 "-" chain 335: thread 101492 (pid 19734, zio_ioctl_issue) sleeping on 0xfffff805199f6800 "-" chain 336: thread 101495 (pid 19734, zio_ioctl_intr) sleeping on 0xfffff808083bfb00 "-" chain 337: thread 101565 (pid 19734, txg_thread_enter) sleeping on 0xfffff80454bf23c8 "tx->tx_cpu[c].tc_cv[i]" chain 338: thread 101566 (pid 19734, txg_thread_enter) sleeping on 0xfffff80800462ab8 "tx->tx_quiesce_done_cv" chain 339: thread 102523 (pid 19688, md11) sleeping on 0xfffff804e0a7e000 "mdwait" chain 340: thread 102214 (pid 19681, md10) sleeping on 0xfffff804e0c4a000 "mdwait" chain 341: thread 102267 (pid 19655, solthread 0xfffffff) sleeping on 0xfffff8004418a948 "t->zthr_cv" chain 342: thread 100932 (pid 19655, solthread 0xfffffff) sleeping on 0xfffff803b26a0c48 "t->zthr_cv" chain 343: thread 100934 (pid 19655, arc_dnlc_evicts_thr) sleeping on 0xffffffff8273e848 "arc_dnlc_evicts_cv" chain 344: thread 100935 (pid 19655, dbu_evict) sleeping on 0xfffff80205a0c800 "-" chain 345: thread 100936 (pid 19655, dbuf_evict_thread) sleeping on 0xffffffff82741fc8 "dbuf_evict_cv" chain 346: thread 100940 (pid 19655, z_vdev_file_0) sleeping on 0xfffff80808418a00 "-" chain 347: thread 100941 (pid 19655, z_vdev_file_1) sleeping on 0xfffff80808418a00 "-" chain 348: thread 100945 (pid 19655, z_vdev_file_2) sleeping on 0xfffff80808418a00 "-" chain 349: thread 100946 (pid 19655, z_vdev_file_3) sleeping on 0xfffff80808418a00 "-" chain 350: thread 100950 (pid 19655, z_vdev_file_4) sleeping on 0xfffff80808418a00 "-" chain 351: thread 100953 (pid 19655, z_vdev_file_5) sleeping on 0xfffff80808418a00 "-" chain 352: thread 100957 (pid 19655, z_vdev_file_6) sleeping on 0xfffff80808418a00 "-" chain 353: thread 100958 (pid 19655, z_vdev_file_7) sleeping on 0xfffff80808418a00 "-" chain 354: thread 100961 (pid 19655, z_vdev_file_8) sleeping on 0xfffff80808418a00 "-" chain 355: thread 100965 (pid 19655, z_vdev_file_9) sleeping on 0xfffff80808418a00 "-" chain 356: thread 100966 (pid 19655, z_vdev_file_10) sleeping on 0xfffff80808418a00 "-" chain 357: thread 100970 (pid 19655, z_vdev_file_11) sleeping on 0xfffff80808418a00 "-" chain 358: thread 100979 (pid 19655, z_vdev_file_12) sleeping on 0xfffff80808418a00 "-" chain 359: thread 100980 (pid 19655, z_vdev_file_13) sleeping on 0xfffff80808418a00 "-" chain 360: thread 100981 (pid 19655, z_vdev_file_14) sleeping on 0xfffff80808418a00 "-" chain 361: thread 100983 (pid 19655, z_vdev_file_15) sleeping on 0xfffff80808418a00 "-" chain 362: thread 100986 (pid 19655, l2arc_feed_thread) sleeping on 0xffffffff8273e9a0 "l2arc_feed_thr_cv" chain 363: thread 100993 (pid 19655, zfsvfs) sleeping on 0xfffff80685ceeb00 "-" chain 364: thread 101500 (pid 19655, trim stress2_tank) sleeping on 0xfffffe00d9fe6d28 "spa->spa_trim_cv" chain 365: thread 101502 (pid 19655, metaslab_group_task) sleeping on 0xfffff8064008d400 "-" chain 366: thread 101503 (pid 19655, metaslab_group_task) sleeping on 0xfffff8064008d400 "-" chain 367: thread 101507 (pid 19655, metaslab_group_task) sleeping on 0xfffff8064008d400 "-" chain 368: thread 101508 (pid 19655, metaslab_group_task) sleeping on 0xfffff8064008d400 "-" chain 369: thread 101509 (pid 19655, metaslab_group_task) sleeping on 0xfffff8064008d400 "-" chain 370: thread 101510 (pid 19655, metaslab_group_task) sleeping on 0xfffff8064008d400 "-" chain 371: thread 101514 (pid 19655, dp_sync_taskq_0) sleeping on 0xfffff808083bec00 "-" chain 372: thread 101519 (pid 19655, dp_sync_taskq_1) sleeping on 0xfffff808083bec00 "-" chain 373: thread 101520 (pid 19655, dp_sync_taskq_2) sleeping on 0xfffff808083bec00 "-" chain 374: thread 101524 (pid 19655, dp_sync_taskq_3) sleeping on 0xfffff808083bec00 "-" chain 375: thread 101525 (pid 19655, dp_sync_taskq_4) sleeping on 0xfffff808083bec00 "-" chain 376: thread 101529 (pid 19655, dp_sync_taskq_5) sleeping on 0xfffff808083bec00 "-" chain 377: thread 101533 (pid 19655, dp_sync_taskq_6) sleeping on 0xfffff808083bec00 "-" chain 378: thread 101534 (pid 19655, dp_sync_taskq_7) sleeping on 0xfffff808083bec00 "-" chain 379: thread 101538 (pid 19655, dp_sync_taskq_8) sleeping on 0xfffff808083bec00 "-" chain 380: thread 101541 (pid 19655, dp_zil_clean_taskq_) sleeping on 0xfffff80808392a00 "-" chain 381: thread 101542 (pid 19655, dp_zil_clean_taskq_) sleeping on 0xfffff80808392a00 "-" chain 382: thread 101543 (pid 19655, dp_zil_clean_taskq_) sleeping on 0xfffff80808392a00 "-" chain 383: thread 101544 (pid 19655, dp_zil_clean_taskq_) sleeping on 0xfffff80808392a00 "-" chain 384: thread 101545 (pid 19655, dp_zil_clean_taskq_) sleeping on 0xfffff80808392a00 "-" chain 385: thread 101546 (pid 19655, dp_zil_clean_taskq_) sleeping on 0xfffff80808392a00 "-" chain 386: thread 101552 (pid 19655, dp_zil_clean_taskq_) sleeping on 0xfffff80808392a00 "-" chain 387: thread 101554 (pid 19655, dp_zil_clean_taskq_) sleeping on 0xfffff80808392a00 "-" chain 388: thread 101557 (pid 19655, dp_zil_clean_taskq_) sleeping on 0xfffff80808392a00 "-" chain 389: thread 101558 (pid 19655, dp_zil_clean_taskq_) sleeping on 0xfffff80808392a00 "-" chain 390: thread 101560 (pid 19655, dp_zil_clean_taskq_) sleeping on 0xfffff80808392a00 "-" chain 391: thread 101561 (pid 19655, dp_zil_clean_taskq_) sleeping on 0xfffff80808392a00 "-" chain 392: thread 101562 (pid 19655, zfs_vn_rele_taskq) blocked on lockmgr (null)EXCL chain 393: thread 101570 (pid 19655, mmp_thread_enter) sleeping on 0xfffffe00d9fe6f00 "mmp->mmp_thread_cv" chain 394: thread 101571 (pid 19655, solthread 0xfffffff) sleeping on 0xfffff808088ac748 "t->zthr_cv" chain 395: thread 101574 (pid 19655, solthread 0xfffffff) sleeping on 0xfffff808088ac4c8 "t->zthr_cv" chain 396: thread 102125 (pid 19629, sh) sleeping on 0xfffff80109e04000 "wait" chain 397: thread 102698 (pid 19628, tee) sleeping on 0xfffff807080c5be0 "piperd" chain 398: thread 102247 (pid 19627, sh) sleeping on 0xfffff801a79be530 "wait" chain 399: thread 102534 (pid 124, bash) sleeping on 0xfffff800303878c0 "select" chain 400: thread 102816 (pid 99828, csh) sleeping on 0xfffff8053f1d40a8 "pause" chain 401: thread 102762 (pid 99824, su) sleeping on 0xfffff80174cfd000 "wait" chain 402: thread 100193 (pid 61272, bash) sleeping on 0xfffff80030ca5530 "wait" chain 403: thread 100197 (pid 61259, sshd) sleeping on 0xfffff800308ec3c0 "select" chain 404: thread 100343 (pid 61026, sshd) sleeping on 0xfffff8031802dc40 "select" chain 405: thread 100248 (pid 3000, sh) sleeping on 0xfffff80030ac65f0 "piperd" chain 406: thread 100227 (pid 2998, awk) sleeping on 0xfffff80030d7f8e8 "piperd" chain 407: thread 100237 (pid 2997, sh) sleeping on 0xfffff80030d7e530 "wait" chain 408: thread 100201 (pid 2930, sh) sleeping on 0xfffff80030f02000 "wait" chain 409: thread 100171 (pid 2490, sh) sleeping on 0xfffff80030e92530 "wait" chain 410: thread 100189 (pid 2388, csh) sleeping on 0xfffff801c098f8b0 "ttyin" chain 411: thread 100219 (pid 2387, su) sleeping on 0xfffff80030d85000 "wait" chain 412: thread 100220 (pid 2382, bash) sleeping on 0xfffff804f074da60 "wait" chain 413: thread 100200 (pid 2381, sshd) sleeping on 0xfffff800306cb240 "select" chain 414: thread 100173 (pid 2378, sshd) sleeping on 0xfffff8003061db40 "select" chain 415: thread 100184 (pid 2372, csh) sleeping on 0xfffff800054310b0 "ttyin" chain 416: thread 100142 (pid 2367, login) sleeping on 0xfffff800306a9000 "wait" chain 417: thread 100217 (pid 2366, getty) sleeping on 0xfffff800054314b0 "ttyin" chain 418: thread 100216 (pid 2365, getty) sleeping on 0xfffff80003bad0b0 "ttyin" chain 419: thread 100215 (pid 2364, getty) sleeping on 0xfffff80003bad4b0 "ttyin" chain 420: thread 100214 (pid 2363, getty) sleeping on 0xfffff80003bad8b0 "ttyin" chain 421: thread 100213 (pid 2362, getty) sleeping on 0xfffff80003badcb0 "ttyin" chain 422: thread 100212 (pid 2361, getty) sleeping on 0xfffff80003bac0b0 "ttyin" chain 423: thread 100211 (pid 2360, getty) sleeping on 0xfffff80003bac4b0 "ttyin" chain 424: thread 100210 (pid 2359, getty) sleeping on 0xfffff80003bac8b0 "ttyin" chain 425: thread 100186 (pid 2358, getty) sleeping on 0xfffff80003baccb0 "ttyin" chain 426: thread 100206 (pid 2312, cron) sleeping on 0xffffffff81c80275 "nanslp" chain 427: thread 100166 (pid 2308, sshd) sleeping on 0xfffff80030742c40 "select" chain 428: thread 100161 (pid 2280, powerd) sleeping on 0xfffff80003a34bc0 "select" chain 429: thread 100195 (pid 2157, watchdogd) sleeping on 0xffffffff81c80279 "nanslp" chain 430: thread 100170 (pid 2078, rpcbind) sleeping on 0xfffff800306cb440 "select" chain 431: thread 100159 (pid 2068, syslogd) sleeping on 0xfffff80044875040 "select" chain 432: thread 100191 (pid 1992, dhclient) sleeping on 0xfffff8003061de40 "select" chain 433: thread 100154 (pid 1928, dhclient) sleeping on 0xfffff800309b4740 "select" chain 434: thread 100149 (pid 1925, dhclient) sleeping on 0xfffff800309b47c0 "select" chain 435: thread 100141 (pid 34, vnlru) sleeping on 0xfffff800306a9530 "vlruwt" chain 436: thread 100140 (pid 33, syncer) sleeping on 0xffffffff81d56720 "syncer" chain 437: thread 100130 (pid 32, bufdaemon) sleeping on 0xffffffff81d55bd8 "qsleep" chain 438: thread 100131 (pid 32, bufspacedaemon-0) sleeping on 0xffffffff8180a980 "-" chain 439: thread 100132 (pid 32, bufspacedaemon-1) sleeping on 0xffffffff81812c40 "-" chain 440: thread 100133 (pid 32, bufspacedaemon-2) sleeping on 0xffffffff8181af00 "-" chain 441: thread 100134 (pid 32, bufspacedaemon-3) sleeping on 0xffffffff818231c0 "-" chain 442: thread 100135 (pid 32, bufspacedaemon-4) sleeping on 0xffffffff8182b480 "-" chain 443: thread 100136 (pid 32, bufspacedaemon-5) sleeping on 0xffffffff81833740 "-" chain 444: thread 100138 (pid 32, bufspacedaemon-6) sleeping on 0xffffffff8183ba00 "-" chain 445: thread 100152 (pid 32, / worker) sleeping on 0xfffff80030a2fce8 "sdflush" chain 446: thread 100203 (pid 32, /tmp worker) sleeping on 0xfffff80030caaee8 "sdflush" chain 447: thread 100129 (pid 31, vmdaemon) sleeping on 0xffffffff81d715c8 "psleep" chain 448: thread 100128 (pid 30, dom0) sleeping on 0xffffffff81d9e218 "psleep" chain 449: thread 100137 (pid 30, laundry: dom0) sleeping on 0xffffffff81d9e224 "launds" chain 450: thread 100139 (pid 30, uma) sleeping on 0xffffffff80f04b90 "umarcl" chain 451: thread 100127 (pid 29, enc_daemon1) sleeping on 0xfffff80005005530 "idle" chain 452: thread 100126 (pid 28, enc_daemon0) sleeping on 0xfffff80030527a60 "idle" chain 453: thread 100120 (pid 27, rand_harvestq) sleeping on 0xffffffff81adbf18 "-" chain 454: thread 100119 (pid 26, sctp_iterator) sleeping on 0xffffffff81de37e0 "waiting_for_work" chain 455: thread 100118 (pid 25, soaiod4) sleeping on 0xffffffff81d555dc "-" chain 456: thread 100117 (pid 24, soaiod3) sleeping on 0xffffffff81d555dc "-" chain 457: thread 100116 (pid 23, soaiod2) sleeping on 0xffffffff81d555dc "-" chain 458: thread 100115 (pid 22, soaiod1) sleeping on 0xffffffff81d555dc "-" chain 459: thread 100096 (pid 21, usbus0) sleeping on 0xfffffe00d1584460 "-" chain 460: thread 100097 (pid 21, usbus0) sleeping on 0xfffffe00d15844b8 "-" chain 461: thread 100098 (pid 21, usbus0) sleeping on 0xfffffe00d1584510 "-" chain 462: thread 100099 (pid 21, usbus0) sleeping on 0xfffffe00d1584568 "-" chain 463: thread 100100 (pid 21, usbus0) sleeping on 0xfffffe00d15845c0 "-" chain 464: thread 100102 (pid 21, usbus1) sleeping on 0xfffffe0037b56d10 "-" chain 465: thread 100103 (pid 21, usbus1) sleeping on 0xfffffe0037b56d68 "-" chain 466: thread 100104 (pid 21, usbus1) sleeping on 0xfffffe0037b56dc0 "-" chain 467: thread 100105 (pid 21, usbus1) sleeping on 0xfffffe0037b56e18 "-" chain 468: thread 100106 (pid 21, usbus1) sleeping on 0xfffffe0037b56e70 "-" chain 469: thread 100107 (pid 21, usbus2) sleeping on 0xfffffe00d16a7d10 "-" chain 470: thread 100108 (pid 21, usbus2) sleeping on 0xfffffe00d16a7d68 "-" chain 471: thread 100109 (pid 21, usbus2) sleeping on 0xfffffe00d16a7dc0 "-" chain 472: thread 100110 (pid 21, usbus2) sleeping on 0xfffffe00d16a7e18 "-" chain 473: thread 100111 (pid 21, usbus2) sleeping on 0xfffffe00d16a7e70 "-" chain 474: thread 100091 (pid 20, doneq0) sleeping on 0xffffffff819b72c0 "-" chain 475: thread 100093 (pid 20, doneq2) sleeping on 0xffffffff819b73c0 "-" chain 476: thread 100125 (pid 20, scanner) sleeping on 0xffffffff819b7188 "-" chain 477: thread 100090 (pid 19, crypto returns 11) sleeping on 0xfffff80003bb2c50 "crypto_ret_wait" chain 478: thread 100089 (pid 18, crypto returns 10) sleeping on 0xfffff80003bb2bf0 "crypto_ret_wait" chain 479: thread 100088 (pid 17, crypto returns 9) sleeping on 0xfffff80003bb2b90 "crypto_ret_wait" chain 480: thread 100087 (pid 16, crypto returns 8) sleeping on 0xfffff80003bb2b30 "crypto_ret_wait" chain 481: thread 100086 (pid 15, crypto returns 7) sleeping on 0xfffff80003bb2ad0 "crypto_ret_wait" chain 482: thread 100085 (pid 9, crypto returns 6) sleeping on 0xfffff80003bb2a70 "crypto_ret_wait" chain 483: thread 100084 (pid 8, crypto returns 5) sleeping on 0xfffff80003bb2a10 "crypto_ret_wait" chain 484: thread 100083 (pid 7, crypto returns 4) sleeping on 0xfffff80003bb29b0 "crypto_ret_wait" chain 485: thread 100082 (pid 6, crypto returns 3) sleeping on 0xfffff80003bb2950 "crypto_ret_wait" chain 486: thread 100081 (pid 5, crypto returns 2) sleeping on 0xfffff80003bb28f0 "crypto_ret_wait" chain 487: thread 100080 (pid 4, crypto returns 1) sleeping on 0xfffff80003bb2890 "crypto_ret_wait" chain 488: thread 100079 (pid 3, crypto returns 0) sleeping on 0xfffff80003bb2830 "crypto_ret_wait" chain 489: thread 100078 (pid 2, crypto) sleeping on 0xffffffff81d6b6a8 "crypto_wait" chain 490: thread 100065 (pid 14, sequencer 00) sleeping on 0xfffff80003a85888 "seqstate" chain 491: thread 100062 (pid 13, g_event) sleeping on 0xffffffff81d9c848 "-" chain 492: thread 100063 (pid 13, g_up) sleeping on 0xffffffff81d9c858 "-" chain 493: thread 100064 (pid 13, g_down) sleeping on 0xffffffff81d9c850 "-" chain 494: thread 100002 (pid 1, init) sleeping on 0xfffff80003850530 "wait" chain 495: thread 100001 (pid 10, audit) sleeping on 0xffffffff81de4470 "audit_worker_cv" chain 496: thread 100000 (pid 0, swapper) sleeping on 0xffffffff81d8b138 "swapin" chain 497: thread 100015 (pid 0, kqueue_ctx taskq) sleeping on 0xfffff800038cb000 "-" chain 498: thread 100016 (pid 0, aiod_kick taskq) sleeping on 0xfffff800038cae00 "-" chain 499: thread 100019 (pid 0, thread taskq) sleeping on 0xfffff800038cab00 "-" chain 500: thread 100021 (pid 0, config_0) sleeping on 0xfffff80003886000 "-" chain 501: thread 100022 (pid 0, if_config_tqg_0) sleeping on 0xfffff80003885e00 "-" chain 502: thread 100023 (pid 0, softirq_0) sleeping on 0xfffff80003885d00 "-" chain 503: thread 100024 (pid 0, softirq_1) sleeping on 0xfffff80003885c00 "-" chain 504: thread 100025 (pid 0, softirq_2) sleeping on 0xfffff80003885b00 "-" chain 505: thread 100026 (pid 0, softirq_3) sleeping on 0xfffff80003885a00 "-" chain 506: thread 100027 (pid 0, softirq_4) sleeping on 0xfffff80003885900 "-" chain 507: thread 100028 (pid 0, softirq_5) sleeping on 0xfffff80003885800 "-" chain 508: thread 100029 (pid 0, softirq_6) sleeping on 0xfffff80003885700 "-" chain 509: thread 100030 (pid 0, softirq_7) sleeping on 0xfffff80003885600 "-" chain 510: thread 100031 (pid 0, softirq_8) sleeping on 0xfffff80003885500 "-" chain 511: thread 100032 (pid 0, softirq_9) sleeping on 0xfffff80003885400 "-" chain 512: thread 100033 (pid 0, softirq_10) sleeping on 0xfffff80003885300 "-" chain 513: thread 100034 (pid 0, softirq_11) sleeping on 0xfffff80003885200 "-" chain 514: thread 100036 (pid 0, if_io_tqg_1) sleeping on 0xfffff80003885000 "-" chain 515: thread 100037 (pid 0, if_io_tqg_2) sleeping on 0xfffff80003884e00 "-" chain 516: thread 100038 (pid 0, if_io_tqg_3) sleeping on 0xfffff80003884d00 "-" chain 517: thread 100039 (pid 0, if_io_tqg_4) sleeping on 0xfffff80003884c00 "-" chain 518: thread 100040 (pid 0, if_io_tqg_5) sleeping on 0xfffff80003884b00 "-" chain 519: thread 100041 (pid 0, if_io_tqg_6) sleeping on 0xfffff80003884a00 "-" chain 520: thread 100042 (pid 0, if_io_tqg_7) sleeping on 0xfffff80003884900 "-" chain 521: thread 100043 (pid 0, if_io_tqg_8) sleeping on 0xfffff80003884800 "-" chain 522: thread 100044 (pid 0, if_io_tqg_9) sleeping on 0xfffff80003884700 "-" chain 523: thread 100045 (pid 0, if_io_tqg_10) sleeping on 0xfffff80003884600 "-" chain 524: thread 100046 (pid 0, if_io_tqg_11) sleeping on 0xfffff80003884500 "-" chain 525: thread 100061 (pid 0, firmware taskq) sleeping on 0xfffff80003b0c100 "-" chain 526: thread 100066 (pid 0, crypto_0) sleeping on 0xfffff80003b0c000 "-" chain 527: thread 100067 (pid 0, crypto_1) sleeping on 0xfffff80003b0c000 "-" chain 528: thread 100068 (pid 0, crypto_2) sleeping on 0xfffff80003b0c000 "-" chain 529: thread 100069 (pid 0, crypto_3) sleeping on 0xfffff80003b0c000 "-" chain 530: thread 100070 (pid 0, crypto_4) sleeping on 0xfffff80003b0c000 "-" chain 531: thread 100071 (pid 0, crypto_5) sleeping on 0xfffff80003b0c000 "-" chain 532: thread 100072 (pid 0, crypto_6) sleeping on 0xfffff80003b0c000 "-" chain 533: thread 100073 (pid 0, crypto_7) sleeping on 0xfffff80003b0c000 "-" chain 534: thread 100074 (pid 0, crypto_8) sleeping on 0xfffff80003b0c000 "-" chain 535: thread 100075 (pid 0, crypto_9) sleeping on 0xfffff80003b0c000 "-" chain 536: thread 100076 (pid 0, crypto_10) sleeping on 0xfffff80003b0c000 "-" chain 537: thread 100077 (pid 0, crypto_11) sleeping on 0xfffff80003b0c000 "-" chain 538: thread 100114 (pid 0, mca taskq) sleeping on 0xfffff8000546b200 "-" chain 539: thread 100121 (pid 0, acpi_task_0) sleeping on 0xfffff80003b0dd00 "-" chain 540: thread 100122 (pid 0, acpi_task_1) sleeping on 0xfffff80003b0dd00 "-" chain 541: thread 100123 (pid 0, acpi_task_2) sleeping on 0xfffff80003b0dd00 "-" chain 542: thread 100124 (pid 0, CAM taskq) sleeping on 0xfffff80003b0de00 "-" chain 543: thread 100821 (pid 0, system_taskq_0) sleeping on 0xfffff80685cefd00 "-" chain 544: thread 100843 (pid 0, system_taskq_1) sleeping on 0xfffff80685cefd00 "-" chain 545: thread 100847 (pid 0, system_taskq_2) sleeping on 0xfffff80685cefd00 "-" chain 546: thread 100848 (pid 0, system_taskq_3) sleeping on 0xfffff80685cefd00 "-" chain 547: thread 100849 (pid 0, system_taskq_4) sleeping on 0xfffff80685cefd00 "-" chain 548: thread 100850 (pid 0, system_taskq_5) sleeping on 0xfffff80685cefd00 "-" chain 549: thread 100862 (pid 0, system_taskq_6) sleeping on 0xfffff80685cefd00 "-" chain 550: thread 100863 (pid 0, system_taskq_7) sleeping on 0xfffff80685cefd00 "-" chain 551: thread 100867 (pid 0, system_taskq_8) sleeping on 0xfffff80685cefd00 "-" chain 552: thread 100869 (pid 0, system_taskq_9) sleeping on 0xfffff80685cefd00 "-" chain 553: thread 100871 (pid 0, system_taskq_10) sleeping on 0xfffff80685cefd00 "-" chain 554: thread 100878 (pid 0, system_taskq_11) sleeping on 0xfffff80685cefd00 "-" chain 555: thread 100879 (pid 0, arc_prune_0) sleeping on 0xfffff80041008d00 "-" chain 556: thread 100884 (pid 0, arc_prune_1) sleeping on 0xfffff80041008d00 "-" chain 557: thread 100885 (pid 0, arc_prune_2) sleeping on 0xfffff80041008d00 "-" chain 558: thread 100890 (pid 0, arc_prune_3) sleeping on 0xfffff80041008d00 "-" chain 559: thread 100893 (pid 0, arc_prune_4) sleeping on 0xfffff80041008d00 "-" chain 560: thread 100896 (pid 0, arc_prune_5) sleeping on 0xfffff80041008d00 "-" chain 561: thread 100897 (pid 0, arc_prune_6) sleeping on 0xfffff80041008d00 "-" chain 562: thread 100898 (pid 0, arc_prune_7) sleeping on 0xfffff80041008d00 "-" chain 563: thread 100902 (pid 0, arc_prune_8) sleeping on 0xfffff80041008d00 "-" chain 564: thread 100914 (pid 0, arc_prune_9) sleeping on 0xfffff80041008d00 "-" chain 565: thread 100920 (pid 0, arc_prune_10) sleeping on 0xfffff80041008d00 "-" chain 566: thread 100921 (pid 0, arc_prune_11) sleeping on 0xfffff80041008d00 "-" db> (kgdb) proc 30739 [Switching to thread 617 (Thread 102340)]#0 sched_switch (td=0xfffff8073be4a6e0, newtd=0xfffff8000388c000, flags=) at ../../../kern/sched_ule.c:2165 2165 cpuid = PCPU_GET(cpuid); Current language: auto; currently minimal (kgdb) bt #0 sched_switch (td=0xfffff8073be4a6e0, newtd=0xfffff8000388c000, flags=) at ../../../kern/sched_ule.c:2165 #1 0xffffffff80bd0e12 in mi_switch (flags=0x104, newtd=0x0) at ../../../kern/kern_synch.c:524 #2 0xffffffff80c1fdde in sleepq_switch (wchan=0xfffff8070b24ecc0, pri=0x60) at ../../../kern/subr_sleepqueue.c:633 #3 0xffffffff80c1fc83 in sleepq_wait (wchan=0xfffff8070b24ecc0, pri=0x60) at ../../../kern/subr_sleepqueue.c:712 #4 0xffffffff80b99ad7 in sleeplk (lk=0xfffff8070b24ecc0, flags=0x202000, ilk=, wmesg=, pri=, timo=0x33, queue=0x1) at ../../../kern/kern_lock.c:295 #5 0xffffffff80b97ef6 in lockmgr_slock_hard (lk=0xfffff8070b24ecc0, flags=0x202000, ilk=, file=0xffffffff826a1fe3 "/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c", line=, lwa=) at ../../../kern/kern_lock.c:649 #6 0xffffffff82672842 in zfs_lock (ap=0xfffffe00e85013f0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:5935 #7 0xffffffff810e8b02 in VOP_LOCK1_APV (vop=0xffffffff827391d0, a=0xfffffe00e85013f0) at vnode_if.c:2118 #8 0xffffffff80cb2b75 in _vn_lock (vp=0xfffff8070b24ec58, flags=0x202000, file=0xffffffff826a1fe3 "/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c", line=0x5a6) at vnode_if.h:879 #9 0xffffffff82669a49 in zfs_lookup (dvp=, nm=, vpp=, cnp=0xfffffe00e8501940, nameiop=0x0, cr=, flags=0x0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1446 #10 0xffffffff8266a131 in zfs_freebsd_lookup (ap=0xfffffe00e85016a8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:4907 #11 0xffffffff810e5e08 in VOP_CACHEDLOOKUP_APV (vop=0xffffffff827391d0, a=0xfffffe00e85016a8) at vnode_if.c:193 #12 0xffffffff80c87a32 in vfs_cache_lookup (ap=) at vnode_if.h:80 #13 0xffffffff810e5c68 in VOP_LOOKUP_APV (vop=0xffffffff827391d0, a=0xfffffe00e8501760) at vnode_if.c:126 #14 0xffffffff80c91e91 in lookup (ndp=0xfffffe00e85018b0) at vnode_if.h:54 #15 0xffffffff80c91321 in namei (ndp=0xfffffe00e85018b0) at ../../../kern/vfs_lookup.c:488 #16 0xffffffff80cabb8f in kern_statat (td=0xfffff8073be4a6e0, flag=, fd=, path=0x8006d22f8
, pathseg=UIO_USERSPACE, sbp=0xfffffe00e85019d8, hook=0) at ../../../kern/vfs_syscalls.c:2331 #17 0xffffffff80cac35f in sys_fstatat (td=, uap=0xfffff8073be4aaa8) at ../../../kern/vfs_syscalls.c:2308 #18 0xffffffff8106b901 in amd64_syscall (td=0xfffff8073be4a6e0, traced=) at subr_syscall.c:144 #19 0xffffffff810416c0 in fast_syscall_common () at ../../../amd64/amd64/exception.S:584 #20 0x0000000800420d8a in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) f 6 #6 0xffffffff82672842 in zfs_lock (ap=0xfffffe00e85013f0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:5935 5935 err = vop_stdlock(ap); (kgdb) l 5930 { 5931 vnode_t *vp; 5932 znode_t *zp; 5933 int err; 5934 5935 err = vop_stdlock(ap); 5936 if (err == 0 && (ap->a_flags & LK_NOWAIT) == 0) { 5937 vp = ap->a_vp; 5938 zp = vp->v_data; 5939 if (vp->v_mount != NULL && !VN_IS_DOOMED(vp) && (kgdb) p *ap $1 = {a_gen = {a_desc = 0xffffffff8197f278}, a_vp = 0xfffff8070b24ec58, a_flags = 0x202000, a_file = 0xffffffff826a1fe3 "/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c", a_line = 0x5a6} (kgdb) p *ap.a_vp $2 = {v_type = VDIR, v_irflag = 0x0, v_op = 0xffffffff827391d0, v_data = 0xfffff808006c6360, v_mount = 0xfffff80779d4d000, v_nmntvnodes = {tqe_next = 0xfffff806fdfcc000, tqe_prev = 0xfffff802cff1dc78}, {v_mountedhere = 0x0, v_unpcb = 0x0, v_rdev = 0x0, v_fifoinfo = 0x0}, v_hashlist = {le_next = 0x0, le_prev = 0xfffffe00d1fceac0}, v_cache_src = {lh_first = 0x0}, v_cache_dst = {tqh_first = 0x0, tqh_last = 0xfffff8070b24eca8}, v_cache_dd = 0x0, v_lock = {lock_object = {lo_name = 0xffffffff826b6d7c "zfs", lo_flags = 0x6fb0000, lo_data = 0x0, lo_witness = 0x0}, lk_lock = 0xfffff805466d7006, lk_exslpfail = 0x0, lk_timo = 0x33, lk_pri = 0x60, lk_stack = {depth = 0x9, pcs = 0xfffff8070b24ecf8}}, v_interlock = {lock_object = {lo_name = 0xffffffff8123d3a3 "vnode interlock", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0x0}, mtx_lock = 0x0}, v_vnlock = 0xfffff8070b24ecc0, v_actfreelist = {tqe_next = 0xfffff801efd5a768, tqe_prev = 0xfffff8000a0843d0}, v_bufobj = {bo_lock = {lock_object = {lo_name = 0xffffffff811fd255 "bufobj interlock", lo_flags = 0x5230000, lo_data = 0x0, lo_witness = 0x0}, rw_lock = 0x1}, bo_ops = 0xffffffff8191db20, bo_object = 0x0, bo_synclist = {le_next = 0xfffff805fadea3e0, le_prev = 0xfffff8070b269410}, bo_private = 0xfffff8070b24ec58, bo_clean = {bv_hd = {tqh_first = 0x0, tqh_last = 0xfffff8070b24ee08}, bv_root = { pt_root = 0x0}, bv_cnt = 0x0}, bo_dirty = {bv_hd = {tqh_first = 0x0, tqh_last = 0xfffff8070b24ee28}, bv_root = {pt_root = 0x0}, bv_cnt = 0x0}, bo_numoutput = 0x0, bo_flag = 0x0, bo_domain = 0x4, bo_bsize = 0x20000}, v_pollinfo = 0x0, v_label = 0x0, v_lockf = 0x0, v_rl = { rl_waiters = {tqh_first = 0x0, tqh_last = 0xfffff8070b24ee78}, rl_currdep = 0x0}, v_cstart = 0x0, v_lasta = 0x0, v_lastw = 0x0, v_clen = 0x0, v_holdcnt = 0x2, v_usecount = 0x86, v_iflag = 0x200, v_vflag = 0x0, v_mflag = 0x0, v_writecount = 0x0, v_hash = 0x3859276, v_tag = 0xffffffff826b6d7c "zfs"} (kgdb) (kgdb) p *ap.a_vp->v_vnlock $4 = {lock_object = {lo_name = 0xffffffff826b6d7c "zfs", lo_flags = 0x6fb0000, lo_data = 0x0, lo_witness = 0x0}, lk_lock = 0xfffff805466d7006, lk_exslpfail = 0x0, lk_timo = 0x33, lk_pri = 0x60, lk_stack = {depth = 0x9, pcs = 0xfffff8070b24ecf8}} (kgdb) (kgdb) tid 102163 [Switching to thread 575 (Thread 102163)]#0 cpustop_handler () at ../../../x86/x86/mp_x86.c:1434 1434 use_mwait = (stop_mwait && (cpu_feature2 & CPUID2_MON) != 0 && (kgdb) bt #0 cpustop_handler () at ../../../x86/x86/mp_x86.c:1434 #1 0xffffffff81033fed in ipi_nmi_handler () at ../../../x86/x86/mp_x86.c:1391 #2 0xffffffff8106a1a1 in trap (frame=0xfffffe003e17bf30) at ../../../amd64/amd64/trap.c:203 #3 0xffffffff81041b03 in nmi_calltrap () at ../../../amd64/amd64/exception.S:863 #4 0xffffffff80c11a92 in lock_delay (la=0xfffffe00ef988398) at include/cpufunc.h:348 #5 0xffffffff80bce8d9 in _sx_xlock_hard (sx=0xfffff806130e20f0, x=, opts=, file=, line=) at ../../../kern/kern_sx.c:688 #6 0xffffffff80bce6bf in _sx_xlock (sx=0xfffff806130e20f0, opts=0x0, file=0xffffffff826a28fe "/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c", line=0x457) at ../../../kern/kern_sx.c:325 #7 0xffffffff8259abe5 in dnode_check_slots_free (children=, idx=0xf, slots=0x1) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c:1111 #8 0xffffffff8259a32b in dnode_hold_impl (os=0xfffff80800533800, object=, flag=, slots=, tag=0xffffffff8269edce, dnp=0xfffffe00ef988610) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c:1463 #9 0xffffffff8258cfc1 in dmu_object_alloc_impl (os=0xfffff80800533800, ot=DMU_OT_DIRECTORY_CONTENTS, blocksize=0x0, bonustype=DMU_OT_SA, bonuslen=0x140, dnodesize=, tx=0xfffff806ea290d00) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_object.c:170 #10 0xffffffff8261ce06 in zap_create_norm_dnsize (os=0xfffff80800533800, normflags=0x0, ot=, bonustype=, bonuslen=, dnodesize=, tx=0xfffff806ea290d00) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap_micro.c:768 #11 0xffffffff82628f10 in zfs_mknode (dzp=0xfffff808006c6360, vap=0xfffffe00ef9888b8, tx=0xfffff806ea290d00, cr=0xfffff80004057a00, flag=0x0, zpp=0xfffffe00ef988830, acl_ids=0xfffffe00ef9887e0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c:846 #12 0xffffffff826714fd in zfs_freebsd_mkdir (ap=) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:2124 #13 0xffffffff810e7eae in VOP_MKDIR_APV (vop=0xffffffff827391d0, a=0xfffffe00ef988a50) at vnode_if.c:1587 #14 0xffffffff80caee33 in kern_mkdirat (td=0xfffff805466d7000, fd=0xffffff9c, path=0x7fffffffe320
, segflg=UIO_USERSPACE, mode=0x1b0) at vnode_if.h:665 #15 0xffffffff8106b901 in amd64_syscall (td=0xfffff805466d7000, traced=) at subr_syscall.c:144 #16 0xffffffff810416c0 in fast_syscall_common () at ../../../amd64/amd64/exception.S:584 #17 0x000000080042df3a in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) f 6 #6 0xffffffff80bce6bf in _sx_xlock (sx=0xfffff806130e20f0, opts=0x0, file=0xffffffff826a28fe "/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c", line=0x457) at ../../../kern/kern_sx.c:325 325 error = _sx_xlock_hard(sx, x, opts LOCK_FILE_LINE_ARG); (kgdb) p *sx $5 = {lock_object = {lo_name = 0xffffffff826c065a "dn->dn_mtx", lo_flags = 0x22710000, lo_data = 0x0, lo_witness = 0x0}, sx_lock = 0x1} (kgdb) Index: src/sys/conf/files =================================================================== --- src/sys/conf/files (revision 355582) +++ src/sys/conf/files (working copy) @@ -4935,6 +4935,7 @@ vm/sg_pager.c standard vm/swap_pager.c standard vm/uma_core.c standard vm/uma_dbg.c standard +vm/uma_smr.c standard vm/memguard.c optional DEBUG_MEMGUARD vm/vm_domainset.c standard vm/vm_fault.c standard Index: src/sys/vm/uma.h =================================================================== --- src/sys/vm/uma.h (revision 355582) +++ src/sys/vm/uma.h (working copy) @@ -273,6 +273,17 @@ uma_zone_t uma_zcache_create(char *name, int size, * effort first-touch policy. */ #define UMA_ZONE_MINBUCKET 0x20000 /* Use smallest buckets. */ +#define UMA_ZONE_SMR 0x40000 /* + * Safe memory reclamation defers + * frees until all read sections + * have exited. This flag creates + * a unique SMR context for this + * zone. To share contexts see + * uma_zone_set_smr() below. + * + * See sys/vm/uma_smr.h for more + * details. + */ /* * These flags are shared between the keg and zone. In zones wishing to add @@ -281,7 +292,8 @@ uma_zone_t uma_zcache_create(char *name, int size, */ #define UMA_ZONE_INHERIT \ (UMA_ZONE_OFFPAGE | UMA_ZONE_MALLOC | UMA_ZONE_NOFREE | \ - UMA_ZONE_HASH | UMA_ZONE_VTOSLAB | UMA_ZONE_PCPU | UMA_ZONE_NUMA) + UMA_ZONE_HASH | UMA_ZONE_VTOSLAB | UMA_ZONE_PCPU | UMA_ZONE_NUMA | \ + UMA_ZONE_SMR) /* Definitions for align */ #define UMA_ALIGN_PTR (sizeof(void *) - 1) /* Alignment fit for ptr */ @@ -609,6 +621,17 @@ void uma_zone_set_allocf(uma_zone_t zone, uma_allo void uma_zone_set_freef(uma_zone_t zone, uma_free freef); /* + * Associate a zone with a smr context that is allocated after creation + * so that multiple zones may share the same context. + */ + +struct uma_smr; +typedef struct uma_smr * uma_smr_t; + +void uma_zone_set_smr(uma_zone_t zone, uma_smr_t smr); +uma_smr_t uma_zone_get_smr(uma_zone_t zone); + +/* * These flags are setable in the allocf and visible in the freef. */ #define UMA_SLAB_BOOT 0x01 /* Slab alloced from boot pages */ Index: src/sys/vm/uma_core.c =================================================================== --- src/sys/vm/uma_core.c (revision 355582) +++ src/sys/vm/uma_core.c (working copy) @@ -91,6 +91,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include @@ -419,11 +420,9 @@ bucket_alloc(uma_zone_t zone, void *udata, int fla /* * This is to stop us from allocating per cpu buckets while we're - * running out of vm.boot_pages. Otherwise, we would exhaust the - * boot pages. This also prevents us from allocating buckets in - * low memory situations. + * allocating from vm.boot_pages. */ - if (bucketdisable) + if (__predict_false(booted != BOOT_RUNNING)) return (NULL); /* * To limit bucket recursion we store the original zone flags @@ -454,6 +453,8 @@ bucket_alloc(uma_zone_t zone, void *udata, int fla #endif bucket->ub_cnt = 0; bucket->ub_entries = ubz->ubz_entries; + CTR3(KTR_UMA, "bucket_alloc: zone %s(%p) allocated bucket %p", + zone->uz_name, zone, bucket); } return (bucket); @@ -493,6 +494,11 @@ zone_fetch_bucket(uma_zone_t zone, uma_zone_domain ZONE_LOCK_ASSERT(zone); if ((bucket = TAILQ_FIRST(&zdom->uzd_buckets)) != NULL) { + if ((zone->uz_flags & UMA_ZONE_SMR) != 0 && bucket->ub_epoch) { + if (!uma_smr_poll(zone->uz_smr, bucket->ub_epoch, false)) + return (NULL); + bucket->ub_epoch = 0; + } MPASS(zdom->uzd_nitems >= bucket->ub_cnt); TAILQ_REMOVE(&zdom->uzd_buckets, bucket, ub_link); zdom->uzd_nitems -= bucket->ub_cnt; @@ -517,10 +523,7 @@ zone_put_bucket(uma_zone_t zone, uma_zone_domain_t KASSERT(!ws || zone->uz_bkt_count < zone->uz_bkt_max, ("%s: zone %p overflow", __func__, zone)); - if (ws) - TAILQ_INSERT_HEAD(&zdom->uzd_buckets, bucket, ub_link); - else - TAILQ_INSERT_TAIL(&zdom->uzd_buckets, bucket, ub_link); + TAILQ_INSERT_TAIL(&zdom->uzd_buckets, bucket, ub_link); zdom->uzd_nitems += bucket->ub_cnt; if (ws && zdom->uzd_imax < zdom->uzd_nitems) zdom->uzd_imax = zdom->uzd_nitems; @@ -764,10 +767,15 @@ bucket_drain(uma_zone_t zone, uma_bucket_t bucket) if (bucket == NULL) return; + if ((zone->uz_flags & UMA_ZONE_SMR) != 0 && bucket->ub_epoch) + uma_smr_wait(zone->uz_smr, bucket->ub_epoch); if (zone->uz_fini) for (i = 0; i < bucket->ub_cnt; i++) zone->uz_fini(bucket->ub_bucket[i], zone->uz_size); zone->uz_release(zone->uz_arg, bucket->ub_bucket, bucket->ub_cnt); +#ifdef INVARIANTS + bzero(bucket->ub_bucket, sizeof(void *) * bucket->ub_cnt); +#endif if (zone->uz_max_items > 0) { ZONE_LOCK(zone); zone->uz_items -= bucket->ub_cnt; @@ -868,6 +876,14 @@ cache_drain_safe_cpu(uma_zone_t zone, void *unused b1 = cache->uc_allocbucket; cache->uc_allocbucket = NULL; } + + /* + * Don't flush SMR zone buckets. This leaves the zone without a + * bucket and forces every free to synchronize(). + */ + if ((zone->uz_flags & UMA_ZONE_SMR) != 0) + goto out; + if (cache->uc_freebucket) { if (cache->uc_freebucket->ub_cnt != 0) zone_put_bucket(zone, &zone->uz_domain[domain], @@ -878,6 +894,7 @@ cache_drain_safe_cpu(uma_zone_t zone, void *unused } b3 = cache->uc_crossbucket; cache->uc_crossbucket = NULL; +out: critical_exit(); ZONE_UNLOCK(zone); if (b1) @@ -2115,6 +2132,8 @@ zone_ctor(void *mem, int size, void *udata, int fl ZONE_LOCK(zone); LIST_FOREACH(z, &keg->uk_zones, uz_link) { if (LIST_NEXT(z, uz_link) == NULL) { + /* Keg zones share smr state. */ + zone->uz_smr = z->uz_smr; LIST_INSERT_AFTER(z, zone, uz_link); break; } @@ -2158,6 +2177,11 @@ out: zone->uz_fails = EARLY_COUNTER; } + /* Caller requests a private SMR context. */ + if ((zone->uz_flags & (UMA_ZONE_SMR | UMA_ZONE_SECONDARY)) == + UMA_ZONE_SMR) + zone->uz_smr = uma_smr_create(zone->uz_name); + KASSERT((arg->flags & (UMA_ZONE_MAXBUCKET | UMA_ZONE_NOBUCKET)) != (UMA_ZONE_MAXBUCKET | UMA_ZONE_NOBUCKET), ("Invalid zone flag combination")); @@ -2417,6 +2441,8 @@ uma_startup1(void) printf("Entering %s with %d boot pages left\n", __func__, boot_pages); #endif booted = BOOT_PAGEALLOC; + + uma_smr_init(); } void @@ -2826,9 +2852,13 @@ cache_alloc(uma_zone_t zone, uma_cache_t cache, vo /* * If we have run out of items in our alloc bucket see * if we can switch with the free bucket. + * + * SMR Zones can't re-use the free bucket until the epoch has + * expired. */ bucket = cache->uc_freebucket; - if (bucket != NULL && bucket->ub_cnt != 0) { + if ((zone->uz_flags & UMA_ZONE_SMR) == 0 && bucket != NULL && + bucket->ub_cnt != 0) { cache->uc_freebucket = cache->uc_allocbucket; cache->uc_allocbucket = bucket; return (true); @@ -2905,8 +2935,6 @@ cache_alloc(uma_zone_t zone, uma_cache_t cache, vo * Fill a bucket and attempt to use it as the alloc bucket. */ bucket = zone_alloc_bucket(zone, udata, domain, flags); - CTR3(KTR_UMA, "uma_zalloc: zone %s(%p) bucket zone returned %p", - zone->uz_name, zone, bucket); critical_enter(); if (bucket == NULL) return (false); @@ -3390,7 +3418,6 @@ uma_zfree_arg(uma_zone_t zone, void *item, void *u do { cpu = curcpu; cache = &zone->uz_cpu[cpu]; - bucket = cache->uc_allocbucket; #ifdef UMA_XDOMAIN if ((zone->uz_flags & UMA_ZONE_NUMA) != 0) { itemdomain = _vm_phys_domain(pmap_kextract((vm_offset_t)item)); @@ -3401,13 +3428,17 @@ uma_zfree_arg(uma_zone_t zone, void *item, void *u bucket = cache->uc_crossbucket; } else #endif - /* * Try to free into the allocbucket first to give LIFO ordering * for cache-hot datastructures. Spill over into the freebucket * if necessary. Alloc will swap them if one runs dry. */ - if (bucket == NULL || bucket->ub_cnt >= bucket->ub_entries) + if ((zone->uz_flags & UMA_ZONE_SMR) == 0) { + bucket = cache->uc_allocbucket; + if (bucket == NULL || + bucket->ub_cnt >= bucket->ub_entries) + bucket = cache->uc_freebucket; + } else bucket = cache->uc_freebucket; if (__predict_true(bucket != NULL && bucket->ub_cnt < bucket->ub_entries)) { @@ -3489,16 +3520,17 @@ static __noinline bool cache_free(uma_zone_t zone, uma_cache_t cache, void *udata, void *item, int itemdomain) { - uma_bucket_t bucket; + uma_bucket_t bucket, newbucket; int cpu, domain; CRITICAL_ASSERT(curthread); - if (zone->uz_bucket_size == 0 || bucketdisable) + if (zone->uz_bucket_size == 0) return false; cpu = curcpu; cache = &zone->uz_cpu[cpu]; + newbucket = NULL; /* * NUMA domains need to free to the correct zdom. When XDOMAIN @@ -3530,14 +3562,29 @@ cache_free(uma_zone_t zone, uma_cache_t cache, voi /* We are no longer associated with this CPU. */ critical_exit(); + /* + * Don't let SMR zones operate without a free bucket. Force + * a synchronize and re-use this one. We will only degrade + * to a synchronize every bucket_size items rather than every + * item if we fail to allocate a bucket. + */ + if ((zone->uz_flags & UMA_ZONE_SMR) != 0) { + if (bucket != NULL) + bucket->ub_epoch = uma_smr_advance(zone->uz_smr); + newbucket = bucket_alloc(zone, udata, M_NOWAIT); + if (newbucket == NULL && bucket != NULL) { + bucket_drain(zone, bucket); + newbucket = bucket; + bucket = NULL; + } + } else if (!bucketdisable) + newbucket = bucket_alloc(zone, udata, M_NOWAIT); + if (bucket != NULL) zone_free_bucket(zone, bucket, udata, domain, itemdomain); - bucket = bucket_alloc(zone, udata, M_NOWAIT); - CTR3(KTR_UMA, "uma_zfree: zone %s(%p) allocated bucket %p", - zone->uz_name, zone, bucket); critical_enter(); - if (bucket == NULL) + if ((bucket = newbucket) == NULL) return (false); cpu = curcpu; cache = &zone->uz_cpu[cpu]; @@ -3661,6 +3708,15 @@ static void zone_free_item(uma_zone_t zone, void *item, void *udata, enum zfreeskip skip) { + /* + * If a free is sent directly to an SMR zone we have to + * synchronize immediately because the item can instantly + * be reallocated. This should only happen in degenerate + * cases when no memory is available for per-cpu caches. + */ + if ((zone->uz_flags & UMA_ZONE_SMR) != 0 && skip == SKIP_DTOR) + uma_smr_synchronize(zone->uz_smr); + item_dtor(zone, item, udata, skip); if (skip < SKIP_FINI && zone->uz_fini) @@ -3918,6 +3974,29 @@ uma_zone_set_allocf(uma_zone_t zone, uma_alloc all /* See uma.h */ void +uma_zone_set_smr(uma_zone_t zone, uma_smr_t smr) +{ + uma_keg_t keg; + + ZONE_LOCK(zone); + zone->uz_flags |= UMA_ZONE_SMR; + zone->uz_smr = smr; + ZONE_UNLOCK(zone); + KEG_GET(zone, keg); + KEG_LOCK(keg); + keg->uk_flags |= UMA_ZONE_SMR; + KEG_UNLOCK(keg); +} + +uma_smr_t +uma_zone_get_smr(uma_zone_t zone) +{ + + return (zone->uz_smr); +} + +/* See uma.h */ +void uma_zone_reserve(uma_zone_t zone, int items) { uma_keg_t keg; Index: src/sys/vm/uma_int.h =================================================================== --- src/sys/vm/uma_int.h (revision 355582) +++ src/sys/vm/uma_int.h (working copy) @@ -182,6 +182,7 @@ struct uma_bucket { TAILQ_ENTRY(uma_bucket) ub_link; /* Link into the zone */ int16_t ub_cnt; /* Count of items in bucket. */ int16_t ub_entries; /* Max items. */ + uint32_t ub_epoch; /* epoch value for SMR. */ void *ub_bucket[]; /* actual allocation storage */ }; @@ -398,6 +399,8 @@ struct uma_zone { struct sysctl_oid *uz_oid; /* sysctl oid pointer. */ int uz_namecnt; /* duplicate name count. */ + uma_smr_t uz_smr; /* Safe memory reclaim context. */ + /* * This HAS to be the last item because we adjust the zone size * based on NCPU and then allocate the space for the zones. -- Test scenario: misc/zfs9.sh