Memory modified after free 0xfffff80379858800(2040) val=0 @ 0xfffff80379858890 panic: Most recently used by CAM CCB, vmcore.36 KDB: debugger backends: ddb KDB: current backend: ddb ---<>--- Copyright (c) 1992-2019 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 13.0-CURRENT #1 r347243M: Wed May 8 13:00:56 CEST 2019 pho@mercat1.netperf.freebsd.org:/usr/src/sys/amd64/compile/PHO amd64 FreeBSD clang version 8.0.0 (tags/RELEASE_800/final 356365) (based on LLVM 8.0.0) WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. Entering uma_startup with 10 boot pages configured startup_alloc from "UMA Kegs", 8 boot pages left startup_alloc from "UMA Zones", 7 boot pages left startup_alloc from "UMA Zones", 6 boot pages left startup_alloc from "UMA Zones", 5 boot pages left startup_alloc from "UMA Zones", 4 boot pages left startup_alloc from "UMA Zones", 3 boot pages left startup_alloc from "UMA Hash", 2 boot pages left startup_alloc from "UMA Zones", 1 boot pages left Entering uma_startup1 with 0 boot pages left Entering uma_startup2 with 0 boot pages left VT(vga): resolution 640x480 CPU: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz (3500.07-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306f2 Family=0x6 Model=0x3f Stepping=2 Features=0xbfebfbff Features2=0x7ffefbff AMD Features=0x2c100800 AMD Features2=0x21 Structured Extended Features=0x37ab XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics real memory = 34359738368 (32768 MB) avail memory = 33182699520 (31645 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs FreeBSD/SMP: 1 package(s) x 6 core(s) x 2 hardware threads __stack_chk_init: WARNING: Initializing stack protection with non-random cookies! __stack_chk_init: WARNING: This severely limits the benefit of -fstack-protector! ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard Launching APs: 1 10 8 2 7 11 9 3 4 5 6 Timecounter "TSC-low" frequency 1750036976 Hz quality 1000 random: entropy device external interface [ath_hal] loaded kbd1 at kbdmux0 module_register_init: MOD_LOAD (vesa, 0xffffffff81149030, 0) error 19 random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" 000.000061 [4254] netmap_init netmap: loaded module nexus0 vtvga0: on motherboard cryptosoft0: on motherboard acpi0: on motherboard acpi0: Power Button (fixed) cpu0: on acpi0 atrtc0: port 0x70-0x71,0x74-0x77 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Event timer "HPET4" frequency 14318180 Hz quality 340 Event timer "HPET5" frequency 14318180 Hz quality 340 Event timer "HPET6" frequency 14318180 Hz quality 340 Event timer "HPET7" frequency 14318180 Hz quality 340 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: on acpi0 pci0: on pcib0 pci0: at device 11.1 (no driver attached) pci0: at device 11.2 (no driver attached) pci0: at device 16.1 (no driver attached) pci0: at device 16.6 (no driver attached) pci0: at device 18.1 (no driver attached) acpi_syscontainer0: on acpi0 acpi_syscontainer1: on acpi0 acpi_syscontainer2: on acpi0 acpi_syscontainer3: on acpi0 pcib1: port 0xcf8-0xcff on acpi0 pci1: on pcib1 pcib2: irq 26 at device 1.0 on pci1 pci2: on pcib2 pcib3: irq 32 at device 2.0 on pci1 pci3: on pcib3 pci3: at device 0.0 (no driver attached) pci3: at device 0.1 (no driver attached) pcib4: irq 40 at device 3.0 on pci1 pci4: on pcib4 pci1: at device 17.0 (no driver attached) ahci0: port 0xf110-0xf117,0xf100-0xf103,0xf0f0-0xf0f7,0xf0e0-0xf0e3,0xf020-0xf03f mem 0xfb438000-0xfb4387ff irq 16 at device 17.4 on pci1 ahci0: AHCI v1.30 with 4 6Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahciem0: on ahci0 xhci0: mem 0xfb400000-0xfb40ffff irq 19 at device 20.0 on pci1 xhci0: 32 bytes context size, 64-bit DMA usbus0 on xhci0 usbus0: 5.0Gbps Super Speed USB v3.0 pci1: at device 22.0 (no driver attached) pci1: at device 22.1 (no driver attached) ehci0: mem 0xfb434000-0xfb4343ff irq 18 at device 26.0 on pci1 usbus1: EHCI version 1.0 usbus1 on ehci0 usbus1: 480Mbps High Speed USB v2.0 pcib5: irq 16 at device 28.0 on pci1 pci5: on pcib5 pcib6: irq 18 at device 28.2 on pci1 pci6: on pcib6 pcib7: at device 0.0 on pci6 pci7: on pcib7 vgapci0: port 0xe000-0xe07f mem 0xfa000000-0xfaffffff,0xfb000000-0xfb01ffff irq 18 at device 0.0 on pci7 vgapci0: Boot video device pcib8: irq 16 at device 28.4 on pci1 pci8: on pcib8 igb0: port 0xd020-0xd03f mem 0xfb320000-0xfb33ffff,0xfb344000-0xfb347fff irq 16 at device 0.0 on pci8 igb0: Using 1024 TX descriptors and 1024 RX descriptors igb0: queue equality override not set, capping rx_queues at 6 and tx_queues at 6 igb0: Using 6 RX queues 6 TX queues igb0: Using MSI-X interrupts with 7 vectors igb0: Ethernet address: 0c:c4:7a:a8:cd:da igb0: netmap queues/slots: TX 6/1024, RX 6/1024 igb1: port 0xd000-0xd01f mem 0xfb300000-0xfb31ffff,0xfb340000-0xfb343fff irq 17 at device 0.1 on pci8 igb1: Using 1024 TX descriptors and 1024 RX descriptors igb1: queue equality override not set, capping rx_queues at 6 and tx_queues at 6 igb1: Using 6 RX queues 6 TX queues igb1: Using MSI-X interrupts with 7 vectors igb1: Ethernet address: 0c:c4:7a:a8:cd:db igb1: netmap queues/slots: TX 6/1024, RX 6/1024 ehci1: mem 0xfb433000-0xfb4333ff irq 18 at device 29.0 on pci1 usbus2: EHCI version 1.0 usbus2 on ehci1 usbus2: 480Mbps High Speed USB v2.0 isab0: at device 31.0 on pci1 isa0: on isab0 ahci1: port 0xf070-0xf077,0xf060-0xf063,0xf050-0xf057,0xf040-0xf043,0xf000-0xf01f mem 0xfb432000-0xfb4327ff irq 16 at device 31.2 on pci1 ahci1: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported ahcich4: at channel 0 on ahci1 ahcich5: at channel 1 on ahci1 ahcich6: at channel 2 on ahci1 ahcich7: at channel 3 on ahci1 ahcich8: at channel 4 on ahci1 ahcich9: at channel 5 on ahci1 ahciem1: on ahci1 acpi_button0: on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 uart1: console (115200,n,8,1) orm0: at iomem 0xc0000-0xc7fff,0xcb000-0xcbfff pnpid ORM0000 on isa0 est0: on cpu0 p4tcc0: on cpu0 NULL mp in getnewvnode(9), tag crossmp Timecounters tick every 1.000 msec arc4random: WARNING: initial seeding bypassed the cryptographic random device because it was not yet seeded and the knob 'bypass_before_seeding' was enabled. ugen1.1: at usbus1 ugen0.1: <0x8086 XHCI root HUB> at usbus0 ugen2.1: at usbus2 uhub0: on usbus1 uhub1: on usbus2 uhub2: <0x8086 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0 Expensive timeout(9) function: 0xffffffff804dafe0(0xfffffe0099fdc000) 0.004668419 s ses0 at ahciem0 bus 0 scbus4 target 0 lun 0 ses0: SEMB S-E-S 2.00 device ses0: SEMB SES Device ses1 at ahciem1 bus 0 scbus11 target 0 lun 0 ses1: SEMB S-E-S 2.00 device ses1: SEMB SES Device ada0 at ahcich1 bus 0 scbus1 target 0 lun 0 ada0: ACS-2 ATA SATA 3.x device ada0: Serial Number WD-WCC3F3KDDP98 ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 953869MB (1953525168 512 byte sectors) ada0: quirks=0x1<4K> hwpmc: SOFT/16/64/0x67 TSC/1/64/0x20 IAP/4/48/0x3ff IAF/3/48/0x67 Trying to mount root from ufs:/dev/gpt/work [rw]... WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. Expensive timeout(9) function: 0xffffffff80a53070(0xffffffff81af3f70) 0.007391339 s WARNING: / was not properly dismounted Interface IP-Address Broadcast ugen0.3: at usbus0 ukbd0 on uhub5 ukbd0: on usbus0 kbd2 at ukbd0 . lo0: link state changed to UP Starting Network: lo0 igb0 igb1. lo0: flags=8049 metric 0 mtu 16384 options=680003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=21 igb0: flags=8843 metric 0 mtu 1500 options=e507bb ether 0c:c4:7a:a8:cd:da inet6 fe80::ec4:7aff:fea8:cdda%igb0 prefixlen 64 scopeid 0x1 media: Ethernet autoselect status: no carrier nd6 options=21 igb1: flags=8802 metric 0 mtu 1500 options=e507bb ether 0c:c4:7a:a8:cd:db media: Ethernet autoselect status: no carrier nd6 options=29 Starting devd. igb0: link state changed to UP Autoloading module: ioat.ko ioat0: mem 0xfb42c000-0xfb42ffff irq 31 at device 4.0 on pci1 ioat0: Capabilities: 2f7 ioat1: mem 0xfb428000-0xfb42bfff irq 39 at device 4.1 on pci1 ioat1: Capabilities: 2f7 ioat2: mem 0xfb424000-0xfb427fff irq 31 at device 4.2 on pci1 ioat2: Capabilities: f7 ioat3: mem 0xfb420000-0xfb423fff irq 39 at device 4.3 on pci1 ioat3: Capabilities: f7 ioat4: mem 0xfb41c000-0xfb41ffff irq 31 at device 4.4 on pci1 ioat4: Capabilities: f7 ioat5: mem 0xfb418000-0xfb41bfff irq 39 at device 4.5 on pci1 ioat5: Capabilities: f7 ioat6: mem 0xfb414000-0xfb417fff irq 31 at device 4.6 on pci1 ioat6: Capabilities: f7 ioat7: mem 0xfb410000-0xfb413fff irq 39 at device 4.7 on pci1 ioat7: Capabilities: f7 Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Starting Network: igb1. igb1: flags=8802 metric 0 mtu 1500 options=e507bb ether 0c:c4:7a:a8:cd:db media: Ethernet autoselect status: no carrier nd6 options=29 Autoloading module: uhid.ko Autoloading module: ums.ko ums0 on uhub5 ums0: on usbus0 ums0: 3 buttons and [Z] coordinates ID=0 Starting dhclient. DHCPREQUEST on igb0 to 255.255.255.255 port 67 DHCPACK from 192if_delmulti_locked: detaching ifnet instance 0xfffff80003e85000 .168.5.1 bound to 192.168.5.171 -- renewal in 300 seconds. add host 127.0.0.1: gateway lo0 fib 0: route already in table add host ::1: gateway lo0 fib 0: route already in table add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Creating and/or trimming log files. Starting syslogd. Setting date via ntp. 8 May 13:04:56 ntpdate[2051]: step time server 192.168.5.1 offset 0.301033 sec Starting rpcbind. NFS access cache time=60 Starting watchdogd. No core dumps found. lock order reversal: 1st 0xfffffe00008ba368 bufwait (bufwait) @ kern/vfs_bio.c:3903 2nd 0xfffff80007a9da00 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:289 stack backtrace: #0 0xffffffff80c4d0b3 at witness_debugger+0x73 #1 0xffffffff80c4cdfd at witness_checkorder+0xa7d #2 0xffffffff80becb48 at _sx_xlock+0x68 #3 0xffffffff80f00b7d at ufsdirhash_add+0x4d #4 0xffffffff80f05516 at ufs_direnter+0x446 #5 0xffffffff80f0d729 at ufs_mkdir+0x8c9 #6 0xffffffff8124725c at VOP_MKDIR_APV+0x8c #7 0xffffffff80cc5a63 at kern_mkdirat+0x1b3 #8 0xffffffff810c41f1 at amd64_syscall+0x291 #9 0xffffffff81095fbd at fast_syscall_common+0x101 Clearing /tmp (X related). Starting statd. Starting lockd. fscklock order reversal: 1st 0xfffff80016301068 ufs (ufs) @ kern/vfs_subr.c:2598 2nd 0xfffffe00008b9f90 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:283 3rd 0xfffff80231876558 ufs (ufs) @ kern/vfs_subr.c:2598 stack backtrace: #0 0xffffffff80c4d0b3 at witness_debugger+0x73 #1 0xffffffff80c4cdfd at witness_checkorder+0xa7d #2 0xffffffff80bb7d5b at lockmgr_xlock_hard+0x6b #3 0xffffffff80bb87ce at __lockmgr_args+0x5fe #4 0xffffffff80efb4fc at ffs_lock+0x7c #5 0xffffffff81247db0 at VOP_LOCK1_APV+0x70 #6 0xffffffff80cc8e95 at _vn_lock+0x65 #7 0xffffffff80cb70d6 at vget+0xa6 #8 0xffffffff80ca8e9c at vfs_hash_get+0xcc #9 0xffffffff80ef6d7f at ffs_vgetf+0x3f #10 0xffffffff80eec4ff at softdep_sync_buf+0x55f #11 0xffffffff80efc314 at ffs_syncvnode+0x234 #12 0xffffffff80ef9abd at ffs_sync+0x20d #13 0xffffffff80caf7df at dounmount+0x4bf #14 0xffffffff80caf280 at sys_unmount+0x300 #15 0xffffffff810c41f1 at amd64_syscall+0x291 #16 0xffffffff81095fbd at fast_syscall_common+0x101 lock order reversal: 1st 0xfffff80007eb72e0 ufs (ufs) @ kern/vfs_mount.c:1353 2nd 0xfffff80007e1ccc0 devfs (devfs) @ ufs/ffs/ffs_softdep.c:1914 stack backtrace: #0 0xffffffff80c4d0b3 at witness_debugger+0x73 #1 0xffffffff80c4cdfd at witness_checkorder+0xa7d #2 0xffffffff80bb785b at lockmgr_lock_fast_path+0x16b #3 0xffffffff81247db0 at VOP_LOCK1_APV+0x70 #4 0xffffffff80cc8e95 at _vn_lock+0x65 #5 0xffffffff80ed9cdd at softdep_flushworklist+0x5d #6 0xffffffff80ef9cb6 at ffs_sync+0x406 #7 0xffffffff80caf7df at dounmount+0x4bf #8 0xffffffff80caf280 at sys_unmount+0x300 #9 0xffffffff810c41f1 at amd64_syscall+0x291 #10 0xffffffff81095fbd at fast_syscall_common+0x101 ** /dev/gpt/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 1785 files, 124669 used, 1140738 free (138 frags, 142575 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** Updating motd:. Mounting late filesystems:. Security policy loaded: MAC/ntpd (mac_ntpd) Starting ntpd. Configuring vt: blanktime. Performing sanity check on sshd configuration. May 8 13:04:59 mercat1 ntpd[2243]: leapsecond file ('/var/db/ntpd.leap-seconds.list'): expired less than 497 days ago Starting sshd. Starting cron. Wed May 8 13:05 FreeBSD/amd64 (mercat1.netperf.freebsd.org) (ttyu1) login: root May 8 13:05:21 mercat1 login[2331]: ROOT LOGIN (root) ON ttyu1 Last login: Wed May 8 11:42:23 on ttyu1 FreeBSD 13.0-CURRENT (PHO) #1 r347243M: Wed May 8 13:00:56 CEST 2019 root@mercat1:~ # May 8 13:05:27 mercat1 su[2342]: pho to root on /dev/pts/0 20190508 13:06:20 all (1/1): callout_reset_on.sh Expensive timeout(9) function: 0xffffffff80bfa970(0xfffff802be8dea60) 0.010913955 s 20190508 13:08:58 all (1/1): callout_reset_on.sh 20190508 13:12:46 all (1/1): callout_reset_on.sh 20190508 13:14:49 all (1/1): callout_reset_on.sh 20190508 13:16:58 all (1/1): callout_reset_on.sh 20190508 13:18:30 all (1/1): callout_reset_on.sh 20190508 13:20:21 all (1/1): callout_reset_on.sh 20190508 13:23:01 all (1/1): callout_reset_on.sh 20190508 13:25:47 all (1/1): callout_reset_on.sh 20190508 13:27:55 all (1/1): callout_reset_on.sh Expensive timeout(9) function: 0xffffffff80e10b00(0xfffff802649edb70) 0.018988774 s 20190508 13:29:32 all (1/1): callout_reset_on.sh 20190508 13:32:46 all (1/1): callout_reset_on.sh 20190508 13:34:53 all (1/1): callout_reset_on.sh 20190508 13:36:09 all (1/1): callout_reset_on.sh 20190508 13:38:15 all (1/1): callout_reset_on.sh 20190508 13:39:36 all (1/1): callout_reset_on.sh 20190508 13:40:58 all (1/1): callout_reset_on.sh 20190508 13:43:18 all (1/1): callout_reset_on.sh 20190508 13:45:14 all (1/1): callout_reset_on.sh 20190508 13:47:31 all (1/1): callout_reset_on.sh 20190508 13:49:47 all (1/1): callout_reset_on.sh 20190508 13:51:18 all (1/1): callout_reset_on.sh 20190508 13:53:03 all (1/1): callout_reset_on.sh 20190508 13:54:44 all (1/1): callout_reset_on.sh 20190508 13:56:20 all (1/1): callout_reset_on.sh 20190508 13:59:06 all (1/1): callout_reset_on.sh Expensive timeout(9) function: 0xffffffff80bfa970(0xfffff802bee84000) 0.021306699 s 20190508 14:01:42 all (1/1): callout_reset_on.sh 20190508 14:04:22 all (1/1): callout_reset_on.sh 20190508 14:05:38 all (1/1): callout_reset_on.sh 20190508 14:07:30 all (1/1): callout_reset_on.sh 20190508 14:09:20 all (1/1): callout_reset_on.sh 20190508 14:10:56 all (1/1): callout_reset_on.sh 20190508 14:12:33 all (1/1): callout_reset_on.sh 20190508 14:14:10 all (1/1): callout_reset_on.sh 20190508 14:16:14 all (1/1): callout_reset_on.sh 20190508 14:17:31 all (1/1): callout_reset_on.sh 20190508 14:18:47 all (1/1): callout_reset_on.sh 20190508 14:20:54 all (1/1): callout_reset_on.sh 20190508 14:22:25 all (1/1): callout_reset_on.sh 20190508 14:24:21 all (1/1): callout_reset_on.sh 20190508 14:25:36 all (1/1): callout_reset_on.sh 20190508 14:27:43 all (1/1): callout_reset_on.sh 20190508 14:29:50 all (1/1): callout_reset_on.sh 20190508 14:31:05 all (1/1): callout_reset_on.sh 20190508 14:32:36 all (1/1): callout_reset_on.sh 20190508 14:34:16 all (1/1): callout_reset_on.sh 20190508 14:36:23 all (1/1): callout_reset_on.sh 20190508 14:38:43 all (1/1): callout_reset_on.sh 20190508 14:41:23 all (1/1): callout_reset_on.sh 20190508 14:42:59 all (1/1): callout_reset_on.sh 20190508 14:44:45 all (1/1): callout_reset_on.sh 20190508 14:45:46 all (1/1): callout_reset_on.sh 20190508 14:47:37 all (1/1): callout_reset_on.sh 20190508 14:49:14 all (1/1): callout_reset_on.sh 20190508 14:51:05 all (1/1): callout_reset_on.sh 20190508 14:54:02 all (1/1): callout_reset_on.sh 20190508 14:55:49 all (1/1): callout_reset_on.sh 20190508 14:58:11 all (1/1): callout_reset_on.sh 20190508 15:00:31 all (1/1): callout_reset_on.sh 20190508 15:03:03 all (1/1): callout_reset_on.sh 20190508 15:04:20 all (1/1): callout_reset_on.sh 20190508 15:05:36 all (1/1): callout_reset_on.sh 20190508 15:08:17 all (1/1): callout_reset_on.sh 20190508 15:09:18 all (1/1): callout_reset_on.sh 20190508 15:11:35 all (1/1): callout_reset_on.sh pts leak: 1/2 20190508 15:13:23 all (1/1): callout_reset_on.sh 20190508 15:14:39 all (1/1): callout_reset_on.sh 20190508 15:16:58 all (1/1): callout_reset_on.sh 20190508 15:19:55 all (1/1): callout_reset_on.sh 20190508 15:21:47 all (1/1): callout_reset_on.sh 20190508 15:22:48 all (1/1): callout_reset_on.sh 20190508 15:25:10 all (1/1): callout_reset_on.sh 20190508 15:27:21 all (1/1): callout_reset_on.sh 20190508 15:29:36 all (1/1): callout_reset_on.sh 20190508 15:31:57 all (1/1): callout_reset_on.sh 20190508 15:33:14 all (1/1): callout_reset_on.sh 20190508 15:34:46 all (1/1): callout_reset_on.sh 20190508 15:36:41 all (1/1): callout_reset_on.sh 20190508 15:38:13 all (1/1): callout_reset_on.sh 20190508 15:40:08 all (1/1): callout_reset_on.sh 20190508 15:42:18 all (1/1): callout_reset_on.sh 20190508 15:44:17 all (1/1): callout_reset_on.sh 20190508 15:46:24 all (1/1): callout_reset_on.sh 20190508 15:48:45 all (1/1): callout_reset_on.sh 20190508 15:50:31 all (1/1): callout_reset_on.sh 20190508 15:53:06 all (1/1): callout_reset_on.sh 20190508 15:54:57 all (1/1): callout_reset_on.sh 20190508 15:56:39 all (1/1): callout_reset_on.sh 20190508 15:58:17 all (1/1): callout_reset_on.sh 20190508 15:59:18 all (1/1): callout_reset_on.sh 20190508 16:00:34 all (1/1): callout_reset_on.sh 20190508 16:02:46 all (1/1): callout_reset_on.sh 20190508 16:04:23 all (1/1): callout_reset_on.sh 20190508 16:05:24 all (1/1): callout_reset_on.sh 20190508 16:07:31 all (1/1): callout_reset_on.sh 20190508 16:09:42 all (1/1): callout_reset_on.sh 20190508 16:12:33 all (1/1): callout_reset_on.sh 20190508 16:14:23 all (1/1): callout_reset_on.sh 20190508 16:15:44 all (1/1): callout_reset_on.sh 20190508 16:17:45 all (1/1): callout_reset_on.sh 20190508 16:19:35 all (1/1): callout_reset_on.sh 20190508 16:20:37 all (1/1): callout_reset_on.sh 20190508 16:22:22 all (1/1): callout_reset_on.sh 20190508 16:25:52 all (1/1): callout_reset_on.sh 20190508 16:27:53 all (1/1): callout_reset_on.sh 20190508 16:29:24 all (1/1): callout_reset_on.sh 20190508 16:30:55 all (1/1): callout_reset_on.sh 20190508 16:32:31 all (1/1): callout_reset_on.sh 20190508 16:35:01 all (1/1): callout_reset_on.sh 20190508 16:36:31 all (1/1): callout_reset_on.sh 20190508 16:38:33 all (1/1): callout_reset_on.sh 20190508 16:40:35 all (1/1): callout_reset_on.sh 20190508 16:41:51 all (1/1): callout_reset_on.sh 20190508 16:43:41 all (1/1): callout_reset_on.sh 20190508 16:45:41 all (1/1): callout_reset_on.sh 20190508 16:47:32 all (1/1): callout_reset_on.sh 20190508 16:49:43 all (1/1): callout_reset_on.sh 20190508 16:51:47 all (1/1): callout_reset_on.sh 20190508 16:53:19 all (1/1): callout_reset_on.sh 20190508 16:55:11 all (1/1): callout_reset_on.sh 20190508 16:57:11 all (1/1): callout_reset_on.sh 20190508 16:59:22 all (1/1): callout_reset_on.sh 20190508 17:02:07 all (1/1): callout_reset_on.sh 20190508 17:04:20 all (1/1): callout_reset_on.sh 20190508 17:06:55 all (1/1): callout_reset_on.sh 20190508 17:08:51 all (1/1): callout_reset_on.sh 20190508 17:11:19 all (1/1): callout_reset_on.sh 20190508 17:12:36 all (1/1): callout_reset_on.sh 20190508 17:13:52 all (1/1): callout_reset_on.sh 20190508 17:15:22 all (1/1): callout_reset_on.sh 20190508 17:16:58 all (1/1): callout_reset_on.sh 20190508 17:18:20 all (1/1): callout_reset_on.sh 20190508 17:20:20 all (1/1): callout_reset_on.sh 20190508 17:21:51 all (1/1): callout_reset_on.sh 20190508 17:24:31 all (1/1): callout_reset_on.sh 20190508 17:26:57 all (1/1): callout_reset_on.sh 20190508 17:28:47 all (1/1): callout_reset_on.sh 20190508 17:31:59 all (1/1): callout_reset_on.sh 20190508 17:33:51 all (1/1): callout_reset_on.sh 20190508 17:36:36 all (1/1): callout_reset_on.sh 20190508 17:38:54 all (1/1): callout_reset_on.sh 20190508 17:41:14 all (1/1): callout_reset_on.sh 20190508 17:44:10 all (1/1): callout_reset_on.sh 20190508 17:45:46 all (1/1): callout_reset_on.sh 20190508 17:47:21 all (1/1): callout_reset_on.sh 20190508 17:50:45 all (1/1): callout_reset_on.sh 20190508 17:53:40 all (1/1): callout_reset_on.sh 20190508 17:56:30 all (1/1): callout_reset_on.sh 20190508 17:58:15 all (1/1): callout_reset_on.sh 20190508 18:00:55 all (1/1): callout_reset_on.sh 20190508 18:02:46 all (1/1): callout_reset_on.sh 20190508 18:04:58 all (1/1): callout_reset_on.sh 20190508 18:06:59 all (1/1): callout_reset_on.sh 20190508 18:09:12 all (1/1): callout_reset_on.sh 20190508 18:12:14 all (1/1): callout_reset_on.sh 20190508 18:14:40 all (1/1): callout_reset_on.sh 20190508 18:15:41 all (1/1): callout_reset_on.sh 20190508 18:17:33 all (1/1): callout_reset_on.sh 20190508 18:18:55 all (1/1): callout_reset_on.sh 20190508 18:21:12 all (1/1): callout_reset_on.sh 20190508 18:23:52 all (1/1): callout_reset_on.sh 20190508 18:26:59 all (1/1): callout_reset_on.sh 20190508 18:29:23 all (1/1): callout_reset_on.sh 20190508 18:31:24 all (1/1): callout_reset_on.sh 20190508 18:33:55 all (1/1): callout_reset_on.sh 20190508 18:36:01 all (1/1): callout_reset_on.sh 20190508 18:38:07 all (1/1): callout_reset_on.sh 20190508 18:40:44 all (1/1): callout_reset_on.sh 20190508 18:42:21 all (1/1): callout_reset_on.sh 20190508 18:44:03 all (1/1): callout_reset_on.sh 20190508 18:46:39 all (1/1): callout_reset_on.sh 20190508 18:47:56 all (1/1): callout_reset_on.sh 20190508 18:49:49 all (1/1): callout_reset_on.sh 20190508 18:51:24 all (1/1): callout_reset_on.sh 20190508 18:53:36 all (1/1): callout_reset_on.sh 20190508 18:55:03 all (1/1): callout_reset_on.sh 20190508 18:57:43 all (1/1): callout_reset_on.sh 20190508 18:59:35 all (1/1): callout_reset_on.sh 20190508 19:01:07 all (1/1): callout_reset_on.sh 20190508 19:03:37 all (1/1): callout_reset_on.sh 20190508 19:06:52 all (1/1): callout_reset_on.sh 20190508 19:09:08 all (1/1): callout_reset_on.sh 20190508 19:10:59 all (1/1): callout_reset_on.sh 20190508 19:12:20 all (1/1): callout_reset_on.sh 20190508 19:13:56 all (1/1): callout_reset_on.sh 20190508 19:15:12 all (1/1): callout_reset_on.sh 20190508 19:16:43 all (1/1): callout_reset_on.sh 20190508 19:19:33 all (1/1): callout_reset_on.sh 20190508 19:21:44 all (1/1): callout_reset_on.sh 20190508 19:23:29 all (1/1): callout_reset_on.sh 20190508 19:25:21 all (1/1): callout_reset_on.sh 20190508 19:27:08 all (1/1): callout_reset_on.sh 20190508 19:29:19 all (1/1): callout_reset_on.sh 20190508 19:31:15 all (1/1): callout_reset_on.sh 20190508 19:34:38 all (1/1): callout_reset_on.sh 20190508 19:35:54 all (1/1): callout_reset_on.sh 20190508 19:37:52 all (1/1): callout_reset_on.sh 20190508 19:39:13 all (1/1): callout_reset_on.sh 20190508 19:40:54 all (1/1): callout_reset_on.sh 20190508 19:43:35 all (1/1): callout_reset_on.sh 20190508 19:45:07 all (1/1): callout_reset_on.sh 20190508 19:46:47 all (1/1): callout_reset_on.sh 20190508 19:48:39 all (1/1): callout_reset_on.sh 20190508 19:50:40 all (1/1): callout_reset_on.sh 20190508 19:52:56 all (1/1): callout_reset_on.sh 20190508 19:54:48 all (1/1): callout_reset_on.sh 20190508 19:58:00 all (1/1): callout_reset_on.sh 20190508 20:00:11 all (1/1): callout_reset_on.sh 20190508 20:02:53 all (1/1): callout_reset_on.sh 20190508 20:04:43 all (1/1): callout_reset_on.sh 20190508 20:06:00 all (1/1): callout_reset_on.sh 20190508 20:07:45 all (1/1): callout_reset_on.sh 20190508 20:09:52 all (1/1): callout_reset_on.sh 20190508 20:11:22 all (1/1): callout_reset_on.sh 20190508 20:13:55 all (1/1): callout_reset_on.sh 20190508 20:15:11 all (1/1): callout_reset_on.sh 20190508 20:16:48 all (1/1): callout_reset_on.sh 20190508 20:18:39 all (1/1): callout_reset_on.sh 20190508 20:20:16 all (1/1): callout_reset_on.sh 20190508 20:21:50 all (1/1): callout_reset_on.sh 20190508 20:23:56 all (1/1): callout_reset_on.sh 20190508 20:26:08 all (1/1): callout_reset_on.sh 20190508 20:29:06 all (1/1): callout_reset_on.sh 20190508 20:30:22 all (1/1): callout_reset_on.sh 20190508 20:32:03 all (1/1): callout_reset_on.sh 20190508 20:33:04 all (1/1): callout_reset_on.sh 20190508 20:35:23 all (1/1): callout_reset_on.sh 20190508 20:38:04 all (1/1): callout_reset_on.sh 20190508 20:39:55 all (1/1): callout_reset_on.sh 20190508 20:42:25 all (1/1): callout_reset_on.sh 20190508 20:44:02 all (1/1): callout_reset_on.sh 20190508 20:45:47 all (1/1): callout_reset_on.sh 20190508 20:47:52 all (1/1): callout_reset_on.sh 20190508 20:49:09 all (1/1): callout_reset_on.sh 20190508 20:50:10 all (1/1): callout_reset_on.sh 20190508 20:51:42 all (1/1): callout_reset_on.sh 20190508 20:53:18 all (1/1): callout_reset_on.sh 20190508 20:54:40 all (1/1): callout_reset_on.sh 20190508 20:56:16 all (1/1): callout_reset_on.sh 20190508 20:57:57 all (1/1): callout_reset_on.sh 20190508 21:00:12 all (1/1): callout_reset_on.sh 20190508 21:02:12 all (1/1): callout_reset_on.sh 20190508 21:03:42 all (1/1): callout_reset_on.sh 20190508 21:05:12 all (1/1): callout_reset_on.sh 20190508 21:07:07 all (1/1): callout_reset_on.sh 20190508 21:08:24 all (1/1): callout_reset_on.sh 20190508 21:10:46 all (1/1): callout_reset_on.sh 20190508 21:13:22 all (1/1): callout_reset_on.sh 20190508 21:15:21 all (1/1): callout_reset_on.sh 20190508 21:16:23 all (1/1): callout_reset_on.sh 20190508 21:17:55 all (1/1): callout_reset_on.sh 20190508 21:19:47 all (1/1): callout_reset_on.sh 20190508 21:22:34 all (1/1): callout_reset_on.sh 20190508 21:24:10 all (1/1): callout_reset_on.sh 20190508 21:25:31 all (1/1): callout_reset_on.sh 20190508 21:27:59 all (1/1): callout_reset_on.sh 20190508 21:30:41 all (1/1): callout_reset_on.sh 20190508 21:33:08 all (1/1): callout_reset_on.sh 20190508 21:35:13 all (1/1): callout_reset_on.sh 20190508 21:37:44 all (1/1): callout_reset_on.sh 20190508 21:40:25 all (1/1): callout_reset_on.sh 20190508 21:43:31 all (1/1): callout_reset_on.sh 20190508 21:45:14 all (1/1): callout_reset_on.sh 20190508 21:46:50 all (1/1): callout_reset_on.sh 20190508 21:49:17 all (1/1): callout_reset_on.sh 20190508 21:50:33 all (1/1): callout_reset_on.sh 20190508 21:52:29 all (1/1): callout_reset_on.sh 20190508 21:56:00 all (1/1): callout_reset_on.sh 20190508 21:57:56 all (1/1): callout_reset_on.sh 20190508 21:59:33 all (1/1): callout_reset_on.sh 20190508 22:00:49 all (1/1): callout_reset_on.sh 20190508 22:02:26 all (1/1): callout_reset_on.sh 20190508 22:03:27 all (1/1): callout_reset_on.sh 20190508 22:05:39 all (1/1): callout_reset_on.sh 20190508 22:08:25 all (1/1): callout_reset_on.sh 20190508 22:11:42 all (1/1): callout_reset_on.sh 20190508 22:13:39 all (1/1): callout_reset_on.sh 20190508 22:16:49 all (1/1): callout_reset_on.sh 20190508 22:19:12 all (1/1): callout_reset_on.sh 20190508 22:21:18 all (1/1): callout_reset_on.sh 20190508 22:24:00 all (1/1): callout_reset_on.sh 20190508 22:26:15 all (1/1): callout_reset_on.sh 20190508 22:28:01 all (1/1): callout_reset_on.sh 20190508 22:30:24 all (1/1): callout_reset_on.sh 20190508 22:32:38 all (1/1): callout_reset_on.sh 20190508 22:35:26 all (1/1): callout_reset_on.sh 20190508 22:37:31 all (1/1): callout_reset_on.sh 20190508 22:40:06 all (1/1): callout_reset_on.sh 20190508 22:42:21 all (1/1): callout_reset_on.sh 20190508 22:43:53 all (1/1): callout_reset_on.sh 20190508 22:45:14 all (1/1): callout_reset_on.sh 20190508 22:46:31 all (1/1): callout_reset_on.sh 20190508 22:48:36 all (1/1): callout_reset_on.sh 20190508 22:51:18 all (1/1): callout_reset_on.sh 20190508 22:53:28 all (1/1): callout_reset_on.sh 20190508 22:54:44 all (1/1): callout_reset_on.sh 20190508 22:56:51 all (1/1): callout_reset_on.sh 20190508 22:57:53 all (1/1): callout_reset_on.sh 20190508 22:59:30 all (1/1): callout_reset_on.sh 20190508 23:01:55 all (1/1): callout_reset_on.sh 20190508 23:03:31 all (1/1): callout_reset_on.sh 20190508 23:05:27 all (1/1): callout_reset_on.sh 20190508 23:07:53 all (1/1): callout_reset_on.sh 20190508 23:10:40 all (1/1): callout_reset_on.sh 20190508 23:13:38 all (1/1): callout_reset_on.sh 20190508 23:16:31 all (1/1): callout_reset_on.sh 20190508 23:18:27 all (1/1): callout_reset_on.sh 20190508 23:20:02 all (1/1): callout_reset_on.sh 20190508 23:22:03 all (1/1): callout_reset_on.sh 20190508 23:24:23 all (1/1): callout_reset_on.sh 20190508 23:25:24 all (1/1): callout_reset_on.sh 20190508 23:26:40 all (1/1): callout_reset_on.sh 20190508 23:28:16 all (1/1): callout_reset_on.sh 20190508 23:29:38 all (1/1): callout_reset_on.sh 20190508 23:33:19 all (1/1): callout_reset_on.sh 20190508 23:34:49 all (1/1): callout_reset_on.sh 20190508 23:36:21 all (1/1): callout_reset_on.sh 20190508 23:38:18 all (1/1): callout_reset_on.sh 20190508 23:40:14 all (1/1): callout_reset_on.sh 20190508 23:42:00 all (1/1): callout_reset_on.sh 20190508 23:43:41 all (1/1): callout_reset_on.sh 20190508 23:45:28 all (1/1): callout_reset_on.sh 20190508 23:46:59 all (1/1): callout_reset_on.sh 20190508 23:48:47 all (1/1): callout_reset_on.sh 20190508 23:50:17 all (1/1): callout_reset_on.sh 20190508 23:52:23 all (1/1): callout_reset_on.sh 20190508 23:55:08 all (1/1): callout_reset_on.sh 20190508 23:57:13 all (1/1): callout_reset_on.sh 20190508 23:59:03 all (1/1): callout_reset_on.sh 20190509 00:01:04 all (1/1): callout_reset_on.sh 20190509 00:02:19 all (1/1): callout_reset_on.sh 20190509 00:04:15 all (1/1): callout_reset_on.sh 20190509 00:07:10 all (1/1): callout_reset_on.sh 20190509 00:09:17 all (1/1): callout_reset_on.sh 20190509 00:11:24 all (1/1): callout_reset_on.sh 20190509 00:14:28 all (1/1): callout_reset_on.sh 20190509 00:15:45 all (1/1): callout_reset_on.sh 20190509 00:17:01 all (1/1): callout_reset_on.sh 20190509 00:18:57 all (1/1): callout_reset_on.sh 20190509 00:21:27 all (1/1): callout_reset_on.sh 20190509 00:23:02 all (1/1): callout_reset_on.sh 20190509 00:24:53 all (1/1): callout_reset_on.sh 20190509 00:26:45 all (1/1): callout_reset_on.sh 20190509 00:28:02 all (1/1): callout_reset_on.sh 20190509 00:30:03 all (1/1): callout_reset_on.sh 20190509 00:31:40 all (1/1): callout_reset_on.sh 20190509 00:34:12 all (1/1): callout_reset_on.sh 20190509 00:36:56 all (1/1): callout_reset_on.sh 20190509 00:38:32 all (1/1): callout_reset_on.sh 20190509 00:40:59 all (1/1): callout_reset_on.sh 20190509 00:42:31 all (1/1): callout_reset_on.sh 20190509 00:43:32 all (1/1): callout_reset_on.sh 20190509 00:45:09 all (1/1): callout_reset_on.sh 20190509 00:46:46 all (1/1): callout_reset_on.sh 20190509 00:49:44 all (1/1): callout_reset_on.sh 20190509 00:51:06 all (1/1): callout_reset_on.sh 20190509 00:52:36 all (1/1): callout_reset_on.sh 20190509 00:54:36 all (1/1): callout_reset_on.sh 20190509 00:56:27 all (1/1): callout_reset_on.sh 20190509 00:58:04 all (1/1): callout_reset_on.sh 20190509 00:59:50 all (1/1): callout_reset_on.sh 20190509 01:02:26 all (1/1): callout_reset_on.sh 20190509 01:03:27 all (1/1): callout_reset_on.sh 20190509 01:05:05 all (1/1): callout_reset_on.sh 20190509 01:07:21 all (1/1): callout_reset_on.sh 20190509 01:08:52 all (1/1): callout_reset_on.sh 20190509 01:12:03 all (1/1): callout_reset_on.sh 20190509 01:14:20 all (1/1): callout_reset_on.sh 20190509 01:15:51 all (1/1): callout_reset_on.sh 20190509 01:17:23 all (1/1): callout_reset_on.sh 20190509 01:19:39 all (1/1): callout_reset_on.sh 20190509 01:22:00 all (1/1): callout_reset_on.sh 20190509 01:23:16 all (1/1): callout_reset_on.sh 20190509 01:24:32 all (1/1): callout_reset_on.sh 20190509 01:27:12 all (1/1): callout_reset_on.sh 20190509 01:29:28 all (1/1): callout_reset_on.sh 20190509 01:31:15 all (1/1): callout_reset_on.sh 20190509 01:33:17 all (1/1): callout_reset_on.sh 20190509 01:35:50 all (1/1): callout_reset_on.sh 20190509 01:37:21 all (1/1): callout_reset_on.sh 20190509 01:39:05 all (1/1): callout_reset_on.sh 20190509 01:41:53 all (1/1): callout_reset_on.sh 20190509 01:44:30 all (1/1): callout_reset_on.sh 20190509 01:46:22 all (1/1): callout_reset_on.sh 20190509 01:48:52 all (1/1): callout_reset_on.sh 20190509 01:50:45 all (1/1): callout_reset_on.sh 20190509 01:53:10 all (1/1): callout_reset_on.sh 20190509 01:55:42 all (1/1): callout_reset_on.sh 20190509 01:57:54 all (1/1): callout_reset_on.sh 20190509 01:59:25 all (1/1): callout_reset_on.sh 20190509 02:01:32 all (1/1): callout_reset_on.sh 20190509 02:03:49 all (1/1): callout_reset_on.sh 20190509 02:05:11 all (1/1): callout_reset_on.sh 20190509 02:06:12 all (1/1): callout_reset_on.sh 20190509 02:07:44 all (1/1): callout_reset_on.sh 20190509 02:09:01 all (1/1): callout_reset_on.sh 20190509 02:11:12 all (1/1): callout_reset_on.sh 20190509 02:13:04 all (1/1): callout_reset_on.sh 20190509 02:14:20 all (1/1): callout_reset_on.sh 20190509 02:16:44 all (1/1): callout_reset_on.sh 20190509 02:18:49 all (1/1): callout_reset_on.sh 20190509 02:21:01 all (1/1): callout_reset_on.sh 20190509 02:23:55 all (1/1): callout_reset_on.sh 20190509 02:25:42 all (1/1): callout_reset_on.sh 20190509 02:27:57 all (1/1): callout_reset_on.sh 20190509 02:29:49 all (1/1): callout_reset_on.sh 20190509 02:31:39 all (1/1): callout_reset_on.sh 20190509 02:32:56 all (1/1): callout_reset_on.sh 20190509 02:35:40 all (1/1): callout_reset_on.sh 20190509 02:38:22 all (1/1): callout_reset_on.sh 20190509 02:39:52 all (1/1): callout_reset_on.sh 20190509 02:42:23 all (1/1): callout_reset_on.sh 20190509 02:45:14 all (1/1): callout_reset_on.sh 20190509 02:47:35 all (1/1): callout_reset_on.sh 20190509 02:50:35 all (1/1): callout_reset_on.sh 20190509 02:52:11 all (1/1): callout_reset_on.sh 20190509 02:54:28 all (1/1): callout_reset_on.sh 20190509 02:56:34 all (1/1): callout_reset_on.sh 20190509 02:58:20 all (1/1): callout_reset_on.sh newblk leak: 4/12. 20190509 03:01:11 all (1/1): callout_reset_on.sh 20190509 03:02:58 all (1/1): callout_reset_on.sh 20190509 03:05:15 all (1/1): callout_reset_on.sh 20190509 03:07:32 all (1/1): callout_reset_on.sh 20190509 03:09:09 all (1/1): callout_reset_on.sh freework leak: 1368/1376. newblk leak: 41/53. 20190509 03:11:36 all (1/1): callout_reset_on.sh 20190509 03:14:47 all (1/1): callout_reset_on.sh 20190509 03:16:40 all (1/1): callout_reset_on.sh 20190509 03:17:57 all (1/1): callout_reset_on.sh 20190509 03:19:44 all (1/1): callout_reset_on.sh 20190509 03:21:31 all (1/1): callout_reset_on.sh 20190509 03:22:54 all (1/1): callout_reset_on.sh 20190509 03:24:25 all (1/1): callout_reset_on.sh 20190509 03:26:16 all (1/1): callout_reset_on.sh 20190509 03:27:48 all (1/1): callout_reset_on.sh 20190509 03:30:44 all (1/1): callout_reset_on.sh 20190509 03:32:01 all (1/1): callout_reset_on.sh 20190509 03:34:12 all (1/1): callout_reset_on.sh 20190509 03:35:29 all (1/1): callout_reset_on.sh 20190509 03:36:30 all (1/1): callout_reset_on.sh 20190509 03:38:33 all (1/1): callout_reset_on.sh 20190509 03:40:45 all (1/1): callout_reset_on.sh 20190509 03:42:21 all (1/1): callout_reset_on.sh 20190509 03:43:54 all (1/1): callout_reset_on.sh 20190509 03:45:50 all (1/1): callout_reset_on.sh 20190509 03:48:12 all (1/1): callout_reset_on.sh 20190509 03:50:04 all (1/1): callout_reset_on.sh 20190509 03:52:11 all (1/1): callout_reset_on.sh 20190509 03:54:24 all (1/1): callout_reset_on.sh 20190509 03:57:41 all (1/1): callout_reset_on.sh 20190509 03:59:57 all (1/1): callout_reset_on.sh 20190509 04:02:13 all (1/1): callout_reset_on.sh 20190509 04:03:45 all (1/1): callout_reset_on.sh 20190509 04:06:55 all (1/1): callout_reset_on.sh 20190509 04:08:31 all (1/1): callout_reset_on.sh 20190509 04:10:42 all (1/1): callout_reset_on.sh 20190509 04:12:28 all (1/1): callout_reset_on.sh 20190509 04:14:01 all (1/1): callout_reset_on.sh 20190509 04:16:10 all (1/1): callout_reset_on.sh 20190509 04:18:03 all (1/1): callout_reset_on.sh 20190509 04:19:05 all (1/1): callout_reset_on.sh 20190509 04:21:05 all (1/1): callout_reset_on.sh 20190509 04:22:47 all (1/1): callout_reset_on.sh 20190509 04:24:25 all (1/1): callout_reset_on.sh 20190509 04:26:22 all (1/1): callout_reset_on.sh 20190509 04:28:08 all (1/1): callout_reset_on.sh 20190509 04:29:25 all (1/1): callout_reset_on.sh 20190509 04:30:43 all (1/1): callout_reset_on.sh 20190509 04:33:00 all (1/1): callout_reset_on.sh 20190509 04:34:52 all (1/1): callout_reset_on.sh 20190509 04:35:54 all (1/1): callout_reset_on.sh 20190509 04:37:44 all (1/1): callout_reset_on.sh 20190509 04:39:35 all (1/1): callout_reset_on.sh 20190509 04:41:13 all (1/1): callout_reset_on.sh 20190509 04:42:31 all (1/1): callout_reset_on.sh 20190509 04:45:01 all (1/1): callout_reset_on.sh 20190509 04:47:08 all (1/1): callout_reset_on.sh 20190509 04:48:24 all (1/1): callout_reset_on.sh 20190509 04:49:55 all (1/1): callout_reset_on.sh 20190509 04:51:32 all (1/1): callout_reset_on.sh 20190509 04:53:20 all (1/1): callout_reset_on.sh 20190509 04:55:33 all (1/1): callout_reset_on.sh 20190509 04:57:45 all (1/1): callout_reset_on.sh 20190509 05:01:11 all (1/1): callout_reset_on.sh 20190509 05:03:19 all (1/1): callout_reset_on.sh 20190509 05:05:01 all (1/1): callout_reset_on.sh 20190509 05:06:38 all (1/1): callout_reset_on.sh 20190509 05:08:01 all (1/1): callout_reset_on.sh 20190509 05:10:02 all (1/1): callout_reset_on.sh 20190509 05:11:57 all (1/1): callout_reset_on.sh 20190509 05:14:14 all (1/1): callout_reset_on.sh 20190509 05:15:47 all (1/1): callout_reset_on.sh 20190509 05:17:24 all (1/1): callout_reset_on.sh 20190509 05:19:39 all (1/1): callout_reset_on.sh 20190509 05:21:57 all (1/1): callout_reset_on.sh 20190509 05:23:53 all (1/1): callout_reset_on.sh 20190509 05:25:10 all (1/1): callout_reset_on.sh 20190509 05:27:03 all (1/1): callout_reset_on.sh 20190509 05:29:08 all (1/1): callout_reset_on.sh 20190509 05:30:50 all (1/1): callout_reset_on.sh 20190509 05:32:57 all (1/1): callout_reset_on.sh 20190509 05:34:20 all (1/1): callout_reset_on.sh 20190509 05:36:06 all (1/1): callout_reset_on.sh 20190509 05:37:41 all (1/1): callout_reset_on.sh 20190509 05:39:41 all (1/1): callout_reset_on.sh 20190509 05:42:08 all (1/1): callout_reset_on.sh 20190509 05:43:10 all (1/1): callout_reset_on.sh 20190509 05:44:32 all (1/1): callout_reset_on.sh 20190509 05:46:02 all (1/1): callout_reset_on.sh 20190509 05:48:15 all (1/1): callout_reset_on.sh 20190509 05:50:32 all (1/1): callout_reset_on.sh 20190509 05:52:20 all (1/1): callout_reset_on.sh 20190509 05:54:07 all (1/1): callout_reset_on.sh 20190509 05:55:40 all (1/1): callout_reset_on.sh 20190509 05:57:16 all (1/1): callout_reset_on.sh 20190509 05:58:33 all (1/1): callout_reset_on.sh 20190509 06:00:19 all (1/1): callout_reset_on.sh 20190509 06:02:41 all (1/1): callout_reset_on.sh 20190509 06:05:02 all (1/1): callout_reset_on.sh 20190509 06:07:33 all (1/1): callout_reset_on.sh 20190509 06:11:18 all (1/1): callout_reset_on.sh 20190509 06:13:43 all (1/1): callout_reset_on.sh 20190509 06:15:04 all (1/1): callout_reset_on.sh 20190509 06:17:06 all (1/1): callout_reset_on.sh 20190509 06:19:23 all (1/1): callout_reset_on.sh 20190509 06:21:10 all (1/1): callout_reset_on.sh 20190509 06:22:51 all (1/1): callout_reset_on.sh 20190509 06:25:23 all (1/1): callout_reset_on.sh 20190509 06:27:25 all (1/1): callout_reset_on.sh 20190509 06:29:03 all (1/1): callout_reset_on.sh 20190509 06:30:54 all (1/1): callout_reset_on.sh 20190509 06:34:00 all (1/1): callout_reset_on.sh 20190509 06:37:23 all (1/1): callout_reset_on.sh 20190509 06:39:21 all (1/1): callout_reset_on.sh 20190509 06:41:23 all (1/1): callout_reset_on.sh 20190509 06:44:05 all (1/1): callout_reset_on.sh 20190509 06:46:18 all (1/1): callout_reset_on.sh 20190509 06:47:56 all (1/1): callout_reset_on.sh 20190509 06:50:58 all (1/1): callout_reset_on.sh 20190509 06:53:14 all (1/1): callout_reset_on.sh 20190509 06:54:30 all (1/1): callout_reset_on.sh 20190509 06:57:05 all (1/1): callout_reset_on.sh 20190509 06:59:20 all (1/1): callout_reset_on.sh 20190509 07:00:38 all (1/1): callout_reset_on.sh 20190509 07:03:11 all (1/1): callout_reset_on.sh 20190509 07:05:07 all (1/1): callout_reset_on.sh 20190509 07:06:44 all (1/1): callout_reset_on.sh 20190509 07:08:51 all (1/1): callout_reset_on.sh 20190509 07:10:37 all (1/1): callout_reset_on.sh 20190509 07:11:39 all (1/1): callout_reset_on.sh 20190509 07:13:02 all (1/1): callout_reset_on.sh 20190509 07:14:39 all (1/1): callout_reset_on.sh 20190509 07:16:11 all (1/1): callout_reset_on.sh 20190509 07:18:06 all (1/1): callout_reset_on.sh 20190509 07:19:38 all (1/1): callout_reset_on.sh 20190509 07:21:00 all (1/1): callout_reset_on.sh 20190509 07:22:23 all (1/1): callout_reset_on.sh 20190509 07:25:09 all (1/1): callout_reset_on.sh 20190509 07:27:02 all (1/1): callout_reset_on.sh 20190509 07:28:24 all (1/1): callout_reset_on.sh 20190509 07:29:56 all (1/1): callout_reset_on.sh 20190509 07:32:37 all (1/1): callout_reset_on.sh 20190509 07:35:05 all (1/1): callout_reset_on.sh 20190509 07:36:42 all (1/1): callout_reset_on.sh 20190509 07:37:45 all (1/1): callout_reset_on.sh 20190509 07:38:44 all (1/2): su.sh witness_lock_list_get: witness exhausted 20190509 07:43:53 all (2/2): callout_reset_on.sh 20190509 07:48:04 all (1/2): su.sh 20190509 07:53:22 all (2/2): callout_reset_on.sh 20190509 07:56:48 all (1/2): su.sh 20190509 08:01:55 all (2/2): callout_reset_on.sh 20190509 08:03:41 all (1/2): su.sh May 9 08:07:12 mercat1 kernel: pid 61489 (swap), jid 0, uid 0, was killed: out of swap space 20190509 08:08:48 all (2/2): callout_reset_on.sh 20190509 08:12:20 all (1/2): su.sh 20190509 08:17:29 all (2/2): callout_reset_on.sh 20190509 08:20:15 all (1/2): su.sh May 9 08:24:32 mercat1 kernel: pid 79874 (swap), jid 0, uid 0, was killed: out of swap space 20190509 08:25:21 all (2/2): callout_reset_on.sh 20190509 08:28:52 all (1/2): su.sh 20190509 08:33:59 all (2/2): callout_reset_on.sh 20190509 08:36:51 all (1/2): su.sh May 9 08:40:33 mercat1 kernel: pid 12131 (swap), jid 0, uid 0, was killed: out of swap space 20190509 08:41:58 all (2/2): callout_reset_on.sh 20190509 08:45:00 all (1/2): su.sh 20190509 08:50:06 all (2/2): callout_reset_on.sh 20190509 08:53:33 all (1/2): su.sh 20190509 08:58:41 all (2/2): callout_reset_on.sh 20190509 09:00:42 all (1/2): su.sh freework leak: 1/5. 20190509 09:05:49 all (2/2): callout_reset_on.sh 20190509 09:08:05 all (1/2): su.sh May 9 09:10:43 mercat1 kernel: pid 25652 (swap), jid 0, uid 0, was killed: out of swap space 20190509 09:13:07 all (2/2): callout_reset_on.sh 20190509 09:15:12 all (1/2): su.sh 20190509 09:20:19 all (2/2): callout_reset_on.sh 20190509 09:22:45 all (1/2): su.sh 20190509 09:27:55 all (2/2): callout_reset_on.sh 20190509 09:31:10 all (1/2): su.sh 20190509 09:36:18 all (2/2): callout_reset_on.sh 20190509 09:39:18 all (1/2): su.sh 20190509 09:44:25 all (2/2): callout_reset_on.sh 20190509 09:48:49 all (1/2): su.sh 20190509 09:53:55 all (2/2): callout_reset_on.sh 20190509 09:56:11 all (1/2): su.sh 20190509 10:01:18 all (2/2): callout_reset_on.sh 20190509 10:04:38 all (1/2): su.sh 20190509 10:10:36 all (2/2): callout_reset_on.sh 20190509 10:14:34 all (1/2): su.sh 20190509 10:19:40 all (2/2): callout_reset_on.sh 20190509 10:22:23 all (1/2): su.sh 20190509 10:27:33 all (2/2): callout_reset_on.sh 20190509 10:29:48 all (1/2): su.sh 20190509 10:34:54 all (2/2): callout_reset_on.sh 20190509 10:38:40 all (1/2): su.sh 20190509 10:43:46 all (2/2): callout_reset_on.sh 20190509 10:46:47 all (1/2): su.sh 20190509 10:51:53 all (2/2): callout_reset_on.sh 20190509 10:54:21 all (1/2): su.sh 20190509 10:59:30 all (2/2): callout_reset_on.sh 20190509 11:01:55 all (1/2): su.sh 20190509 11:07:01 all (2/2): callout_reset_on.sh 20190509 11:11:21 all (1/2): su.sh 20190509 11:17:47 all (2/2): callout_reset_on.sh 20190509 11:19:33 all (1/2): su.sh 20190509 11:24:43 all (2/2): callout_reset_on.sh 20190509 11:28:11 all (1/2): su.sh 20190509 11:33:53 all (2/2): callout_reset_on.sh 20190509 11:36:58 all (1/2): su.sh 20190509 11:42:32 all (2/2): callout_reset_on.sh 20190509 11:45:52 all (1/2): su.sh 20190509 11:50:58 all (2/2): callout_reset_on.sh 20190509 11:53:51 all (1/2): su.sh 20190509 11:58:58 all (2/2): callout_reset_on.sh 20190509 12:01:45 all (1/2): su.sh root@mercat1:~ # init 6 root@mercat1:~ # Stopping cron. Waiting for PIDS: 2276. Stopping sshd. Waiting for PIDS: 2272, 2272. Stopping ntpd. Waiting for PIDS: 2243. Stopping lockd. Waiting for PIDS: 2182. Stopping statd. Waiting for PIDS: 2179. Stopping watchdogd. Waiting for PIDS: 2133. Stopping rpcbind. Waiting for PIDS: 2054if_delmulti_locked: detaching ifnet instance 0xfffff800077d8000 . Stopping devd. Waiting for PIDS: 1973. Writing entropy file:. Writing early boot entropy file:. if_delmulti_locked: detaching ifnet instance 0xfffff80003e85000 Terminated . May 9 12:04:27 mercat1 syslogd: exiting on signal 15 Memory modified after free 0xfffff80379858800(2040) val=0 @ 0xfffff80379858890 panic: Most recently used by CAM CCB cpuid = 7 time = 1557396268 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0076761380 vpanic() at vpanic+0x19d/frame 0xfffffe00767613d0 panic() at panic+0x43/frame 0xfffffe0076761430 mtrash_ctor() at mtrash_ctor+0x7e/frame 0xfffffe0076761450 uma_zalloc_arg() at uma_zalloc_arg+0x9ff/frame 0xfffffe00767614e0 malloc() at malloc+0x9b/frame 0xfffffe0076761530 xpt_run_allocq() at xpt_run_allocq+0xcb/frame 0xfffffe0076761580 adastrategy() at adastrategy+0x6e/frame 0xfffffe00767615b0 g_disk_start() at g_disk_start+0x351/frame 0xfffffe0076761610 g_io_request() at g_io_request+0x3f5/frame 0xfffffe0076761660 g_part_start() at g_part_start+0x120/frame 0xfffffe00767616e0 g_io_request() at g_io_request+0x3f5/frame 0xfffffe0076761730 g_io_request() at g_io_request+0x3f5/frame 0xfffffe0076761780 bufwrite() at bufwrite+0x24d/frame 0xfffffe00767617d0 vfs_bio_awrite() at vfs_bio_awrite+0x212/frame 0xfffffe0076761830 vn_fsync_buf() at vn_fsync_buf+0x21f/frame 0xfffffe0076761890 VOP_FSYNC_APV() at VOP_FSYNC_APV+0x88/frame 0xfffffe00767618b0 ffs_sync() at ffs_sync+0x2f9/frame 0xfffffe0076761960 sys_sync() at sys_sync+0xbe/frame 0xfffffe0076761990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe0076761ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0076761ab0 --- syscall (36, FreeBSD ELF64, sys_sync), rip = 0x24dbca, rsp = 0x7fffffffe6f8, rbp = 0x7fffffffe7e0 --- KDB: enter: panic [ thread pid 1 tid 100002 ] Stopped at kdb_enter+0x3b: movq $0,kdb_why db> run pho db:0:pho> set $lines 20000 db:0:pho> run pho1 db:1:pho1> bt Tracing pid 1 tid 100002 td 0xfffff800036b85a0 kdb_enter() at kdb_enter+0x3b/frame 0xfffffe0076761380 vpanic() at vpanic+0x1ba/frame 0xfffffe00767613d0 panic() at panic+0x43/frame 0xfffffe0076761430 mtrash_ctor() at mtrash_ctor+0x7e/frame 0xfffffe0076761450 uma_zalloc_arg() at uma_zalloc_arg+0x9ff/frame 0xfffffe00767614e0 malloc() at malloc+0x9b/frame 0xfffffe0076761530 xpt_run_allocq() at xpt_run_allocq+0xcb/frame 0xfffffe0076761580 adastrategy() at adastrategy+0x6e/frame 0xfffffe00767615b0 g_disk_start() at g_disk_start+0x351/frame 0xfffffe0076761610 g_io_request() at g_io_request+0x3f5/frame 0xfffffe0076761660 g_part_start() at g_part_start+0x120/frame 0xfffffe00767616e0 g_io_request() at g_io_request+0x3f5/frame 0xfffffe0076761730 g_io_request() at g_io_request+0x3f5/frame 0xfffffe0076761780 bufwrite() at bufwrite+0x24d/frame 0xfffffe00767617d0 vfs_bio_awrite() at vfs_bio_awrite+0x212/frame 0xfffffe0076761830 vn_fsync_buf() at vn_fsync_buf+0x21f/frame 0xfffffe0076761890 VOP_FSYNC_APV() at VOP_FSYNC_APV+0x88/frame 0xfffffe00767618b0 ffs_sync() at ffs_sync+0x2f9/frame 0xfffffe0076761960 sys_sync() at sys_sync+0xbe/frame 0xfffffe0076761990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe0076761ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0076761ab0 --- syscall (36, FreeBSD ELF64, sys_sync), rip = 0x24dbca, rsp = 0x7fffffffe6f8, rbp = 0x7fffffffe7e0 --- db:1:pho1> show allpcpu Current CPU: 7 cpuid = 0 dynamic pcpu = 0xd68900 curthread = 0xfffff800036b8000: pid 11 tid 100003 "idle: cpu0" curpcb = 0xfffffe0076766b80 fpcurthread = none idlethread = 0xfffff800036b8000: tid 100003 "idle: cpu0" curpmap = 0xffffffff82095a78 tssp = 0xffffffff821cd1a0 commontssp = 0xffffffff821cd1a0 rsp0 = 0xfffffe0076766b80 gs32p = 0xffffffff821d3dd8 ldt = 0xffffffff821d3e18 tss = 0xffffffff821d3e08 tlb gen = 774637941 curvnet = 0 spin locks held: cpuid = 1 dynamic pcpu = 0xfffffe0084fdc900 curthread = 0xfffff800036b75a0: pid 11 tid 100004 "idle: cpu1" curpcb = 0xfffffe007676bb80 fpcurthread = none idlethread = 0xfffff800036b75a0: tid 100004 "idle: cpu1" curpmap = 0xffffffff82095a78 tssp = 0xffffffff821cd208 commontssp = 0xffffffff821cd208 rsp0 = 0xfffffe007676bb80 gs32p = 0xffffffff821d3e40 ldt = 0xffffffff821d3e80 tss = 0xffffffff821d3e70 tlb gen = 774637941 curvnet = 0 spin locks held: cpuid = 2 dynamic pcpu = 0xfffffe0084fe8900 curthread = 0xfffff800036b7000: pid 11 tid 100005 "idle: cpu2" curpcb = 0xfffffe0076770b80 fpcurthread = none idlethread = 0xfffff800036b7000: tid 100005 "idle: cpu2" curpmap = 0xffffffff82095a78 tssp = 0xffffffff821cd270 commontssp = 0xffffffff821cd270 rsp0 = 0xfffffe0076770b80 gs32p = 0xffffffff821d3ea8 ldt = 0xffffffff821d3ee8 tss = 0xffffffff821d3ed8 tlb gen = 774637941 curvnet = 0 spin locks held: cpuid = 3 dynamic pcpu = 0xfffffe0084ff4900 curthread = 0xfffff800036b65a0: pid 11 tid 100006 "idle: cpu3" curpcb = 0xfffffe0076775b80 fpcurthread = none idlethread = 0xfffff800036b65a0: tid 100006 "idle: cpu3" curpmap = 0xffffffff82095a78 tssp = 0xffffffff821cd2d8 commontssp = 0xffffffff821cd2d8 rsp0 = 0xfffffe0076775b80 gs32p = 0xffffffff821d3f10 ldt = 0xffffffff821d3f50 tss = 0xffffffff821d3f40 tlb gen = 774637941 curvnet = 0 spin locks held: cpuid = 4 dynamic pcpu = 0xfffffe0085000900 curthread = 0xfffff800036b6000: pid 11 tid 100007 "idle: cpu4" curpcb = 0xfffffe007677ab80 fpcurthread = none idlethread = 0xfffff800036b6000: tid 100007 "idle: cpu4" curpmap = 0xffffffff82095a78 tssp = 0xffffffff821cd340 commontssp = 0xffffffff821cd340 rsp0 = 0xfffffe007677ab80 gs32p = 0xffffffff821d3f78 ldt = 0xffffffff821d3fb8 tss = 0xffffffff821d3fa8 tlb gen = 774637940 curvnet = 0 spin locks held: cpuid = 5 dynamic pcpu = 0xfffffe008500c900 curthread = 0xfffff800036b05a0: pid 11 tid 100008 "idle: cpu5" curpcb = 0xfffffe007677fb80 fpcurthread = none idlethread = 0xfffff800036b05a0: tid 100008 "idle: cpu5" curpmap = 0xffffffff82095a78 tssp = 0xffffffff821cd3a8 commontssp = 0xffffffff821cd3a8 rsp0 = 0xfffffe007677fb80 gs32p = 0xffffffff821d3fe0 ldt = 0xffffffff821d4020 tss = 0xffffffff821d4010 tlb gen = 774637941 curvnet = 0 spin locks held: cpuid = 6 dynamic pcpu = 0xfffffe0085018900 curthread = 0xfffff800036b0000: pid 11 tid 100009 "idle: cpu6" curpcb = 0xfffffe0076784b80 fpcurthread = none idlethread = 0xfffff800036b0000: tid 100009 "idle: cpu6" curpmap = 0xffffffff82095a78 tssp = 0xffffffff821cd410 commontssp = 0xffffffff821cd410 rsp0 = 0xfffffe0076784b80 gs32p = 0xffffffff821d4048 ldt = 0xffffffff821d4088 tss = 0xffffffff821d4078 tlb gen = 774637941 curvnet = 0 spin locks held: cpuid = 7 dynamic pcpu = 0xfffffe0085024900 curthread = 0xfffff800036b85a0: pid 1 tid 100002 "init" curpcb = 0xfffffe0076761b80 fpcurthread = 0xfffff800036b85a0: pid 1 "init" idlethread = 0xfffff800036c95a0: tid 100010 "idle: cpu7" curpmap = 0xfffff800036c0130 tssp = 0xffffffff821cd478 commontssp = 0xffffffff821cd478 rsp0 = 0xfffffe0076761b80 gs32p = 0xffffffff821d40b0 ldt = 0xffffffff821d40f0 tss = 0xffffffff821d40e0 tlb gen = 774637941 curvnet = 0 spin locks held: cpuid = 8 dynamic pcpu = 0xfffffe0085030900 curthread = 0xfffff800036c9000: pid 11 tid 100011 "idle: cpu8" curpcb = 0xfffffe007678eb80 fpcurthread = none idlethread = 0xfffff800036c9000: tid 100011 "idle: cpu8" curpmap = 0xffffffff82095a78 tssp = 0xffffffff821cd4e0 commontssp = 0xffffffff821cd4e0 rsp0 = 0xfffffe007678eb80 gs32p = 0xffffffff821d4118 ldt = 0xffffffff821d4158 tss = 0xffffffff821d4148 tlb gen = 774637941 curvnet = 0 spin locks held: cpuid = 9 dynamic pcpu = 0xfffffe008503c900 curthread = 0xfffff80003aea5a0: pid 20 tid 100092 "doneq1" curpcb = 0xfffffe0006e4ab80 fpcurthread = none idlethread = 0xfffff800036c85a0: tid 100012 "idle: cpu9" curpmap = 0xffffffff82095a78 tssp = 0xffffffff821cd548 commontssp = 0xffffffff821cd548 rsp0 = 0xfffffe0006e4ab80 gs32p = 0xffffffff821d4180 ldt = 0xffffffff821d41c0 tss = 0xffffffff821d41b0 tlb gen = 774637941 curvnet = 0 spin locks held: cpuid = 10 dynamic pcpu = 0xfffffe0085048900 curthread = 0xfffff800036c8000: pid 11 tid 100013 "idle: cpu10" curpcb = 0xfffffe0076798b80 fpcurthread = none idlethread = 0xfffff800036c8000: tid 100013 "idle: cpu10" curpmap = 0xffffffff82095a78 tssp = 0xffffffff821cd5b0 commontssp = 0xffffffff821cd5b0 rsp0 = 0xfffffe0076798b80 gs32p = 0xffffffff821d41e8 ldt = 0xffffffff821d4228 tss = 0xffffffff821d4218 tlb gen = 774637941 curvnet = 0 spin locks held: cpuid = 11 dynamic pcpu = 0xfffffe0085054900 curthread = 0xfffff800036c75a0: pid 11 tid 100014 "idle: cpu11" curpcb = 0xfffffe007679db80 fpcurthread = none idlethread = 0xfffff800036c75a0: tid 100014 "idle: cpu11" curpmap = 0xffffffff82095a78 tssp = 0xffffffff821cd618 commontssp = 0xffffffff821cd618 rsp0 = 0xfffffe007679db80 gs32p = 0xffffffff821d4250 ldt = 0xffffffff821d4290 tss = 0xffffffff821d4280 tlb gen = 774637941 curvnet = 0 spin locks held: db:1:pho1> show alllocks db:1:pho1> show lockedvnods Locked vnodes vnode 0xfffff80007b6d9e0: tag devfs, type VCHR usecount 1, writecount 0, refcount 3446 rdev 0xfffff800078fb400 flags (VI_ACTIVE) v_object 0xfffff80003661a00 ref 0 pages 27695 cleanbuf 3425 dirtybuf 19 lock type devfs: EXCL by thread 0xfffff800036b85a0 (pid 1, init, tid 100002) #0 0xffffffff80bb78b6 at lockmgr_lock_fast_path+0x1c6 #1 0xffffffff81247db0 at VOP_LOCK1_APV+0x70 #2 0xffffffff80cc8e95 at _vn_lock+0x65 #3 0xffffffff80ef9b7f at ffs_sync+0x2cf #4 0xffffffff80cbe92e at sys_sync+0xbe #5 0xffffffff810c41f1 at amd64_syscall+0x291 #6 0xffffffff81095fbd at fast_syscall_common+0x101 dev gpt/work db:1:pho1> show allchains chain 1: thread 100141 (pid 34, vnlru) sleeping on 0xfffff80007884a60 "vlruwt" chain 2: thread 100140 (pid 33, syncer) sleeping on 0xffffffff820401c0 "syncer" chain 3: thread 100130 (pid 32, bufdaemon) sleeping on 0xffffffff8203f658 "qsleep" chain 4: thread 100131 (pid 32, bufspacedaemon-0) sleeping on 0xffffffff81a0a8c0 "-" chain 5: thread 100132 (pid 32, bufspacedaemon-1) sleeping on 0xffffffff81a12b80 "-" chain 6: thread 100133 (pid 32, bufspacedaemon-2) sleeping on 0xffffffff81a1ae40 "-" chain 7: thread 100135 (pid 32, bufspacedaemon-3) sleeping on 0xffffffff81a23100 "-" chain 8: thread 100136 (pid 32, bufspacedaemon-4) sleeping on 0xffffffff81a2b3c0 "-" chain 9: thread 100137 (pid 32, bufspacedaemon-5) sleeping on 0xffffffff81a33680 "-" chain 10: thread 100138 (pid 32, bufspacedaemon-6) sleeping on 0xffffffff81a3b940 "-" chain 11: thread 100153 (pid 32, / worker) sleeping on 0xfffff800162626e8 "sdflush" chain 12: thread 100199 (pid 32, /tmp worker) sleeping on 0xfffff80007a29ae8 "sdflush" chain 13: thread 100129 (pid 31, vmdaemon) sleeping on 0xffffffff8205c548 "psleep" chain 14: thread 100128 (pid 30, dom0) sleeping on 0xffffffff820c6298 "psleep" chain 15: thread 100134 (pid 30, laundry: dom0) sleeping on 0xffffffff820c62a4 "launds" chain 16: thread 100139 (pid 30, uma) sleeping on 0xffffffff80f19660 "umarcl" chain 17: thread 100127 (pid 29, enc_daemon1) sleeping on 0xfffff80007786000 "idle" chain 18: thread 100126 (pid 28, enc_daemon0) sleeping on 0xfffff80007786530 "idle" chain 19: thread 100123 (pid 27, rand_harvestq) sleeping on 0xffffffff81cfd430 "-" chain 20: thread 100119 (pid 26, sctp_iterator) sleeping on 0xffffffff820c4970 "waiting_for_work" chain 21: thread 100118 (pid 25, soaiod4) sleeping on 0xffffffff8203f05c "-" chain 22: thread 100117 (pid 24, soaiod3) sleeping on 0xffffffff8203f05c "-" chain 23: thread 100116 (pid 23, soaiod2) sleeping on 0xffffffff8203f05c "-" chain 24: thread 100115 (pid 22, soaiod1) sleeping on 0xffffffff8203f05c "-" chain 25: thread 100096 (pid 21, usbus0) sleeping on 0xfffffe009a1aa460 "-" chain 26: thread 100097 (pid 21, usbus0) sleeping on 0xfffffe009a1aa4b8 "-" chain 27: thread 100098 (pid 21, usbus0) sleeping on 0xfffffe009a1aa510 "-" chain 28: thread 100099 (pid 21, usbus0) sleeping on 0xfffffe009a1aa568 "-" chain 29: thread 100100 (pid 21, usbus0) sleeping on 0xfffffe009a1aa5c0 "-" chain 30: thread 100102 (pid 21, usbus1) sleeping on 0xfffffe0006dd6d10 "-" chain 31: thread 100103 (pid 21, usbus1) sleeping on 0xfffffe0006dd6d68 "-" chain 32: thread 100104 (pid 21, usbus1) sleeping on 0xfffffe0006dd6dc0 "-" chain 33: thread 100105 (pid 21, usbus1) sleeping on 0xfffffe0006dd6e18 "-" chain 34: thread 100106 (pid 21, usbus1) sleeping on 0xfffffe0006dd6e70 "-" chain 35: thread 100107 (pid 21, usbus2) sleeping on 0xfffffe009a63bd10 "-" chain 36: thread 100108 (pid 21, usbus2) sleeping on 0xfffffe009a63bd68 "-" chain 37: thread 100109 (pid 21, usbus2) sleeping on 0xfffffe009a63bdc0 "-" chain 38: thread 100110 (pid 21, usbus2) sleeping on 0xfffffe009a63be18 "-" chain 39: thread 100111 (pid 21, usbus2) sleeping on 0xfffffe009a63be70 "-" chain 40: thread 100091 (pid 20, doneq0) sleeping on 0xffffffff81bd86c0 "-" chain 41: thread 100093 (pid 20, doneq2) sleeping on 0xffffffff81bd87c0 "-" chain 42: thread 100125 (pid 20, scanner) sleeping on 0xffffffff81bd8588 "-" chain 43: thread 100090 (pid 19, crypto returns 11) sleeping on 0xfffff80003978c50 "crypto_ret_wait" chain 44: thread 100089 (pid 18, crypto returns 10) sleeping on 0xfffff80003978bf0 "crypto_ret_wait" chain 45: thread 100088 (pid 17, crypto returns 9) sleeping on 0xfffff80003978b90 "crypto_ret_wait" chain 46: thread 100087 (pid 16, crypto returns 8) sleeping on 0xfffff80003978b30 "crypto_ret_wait" chain 47: thread 100086 (pid 15, crypto returns 7) sleeping on 0xfffff80003978ad0 "crypto_ret_wait" chain 48: thread 100085 (pid 9, crypto returns 6) sleeping on 0xfffff80003978a70 "crypto_ret_wait" chain 49: thread 100084 (pid 8, crypto returns 5) sleeping on 0xfffff80003978a10 "crypto_ret_wait" chain 50: thread 100083 (pid 7, crypto returns 4) sleeping on 0xfffff800039789b0 "crypto_ret_wait" chain 51: thread 100082 (pid 6, crypto returns 3) sleeping on 0xfffff80003978950 "crypto_ret_wait" chain 52: thread 100081 (pid 5, crypto returns 2) sleeping on 0xfffff800039788f0 "crypto_ret_wait" chain 53: thread 100080 (pid 4, crypto returns 1) sleeping on 0xfffff80003978890 "crypto_ret_wait" chain 54: thread 100079 (pid 3, crypto returns 0) sleeping on 0xfffff80003978830 "crypto_ret_wait" chain 55: thread 100078 (pid 2, crypto) sleeping on 0xffffffff82056b40 "crypto_wait" chain 56: thread 100065 (pid 14, sequencer 00) sleeping on 0xfffff80003893c88 "seqstate" chain 57: thread 100062 (pid 13, g_event) sleeping on 0xffffffff82094e20 "-" chain 58: thread 100063 (pid 13, g_up) sleeping on 0xffffffff82094e30 "-" chain 59: thread 100064 (pid 13, g_down) sleeping on 0xffffffff82094e18 "-" chain 60: thread 100001 (pid 10, audit) sleeping on 0xffffffff820c5690 "audit_worker_cv" chain 61: thread 100000 (pid 0, swapper) sleeping on 0xffffffff82094e60 "swapin" chain 62: thread 100015 (pid 0, config_0) sleeping on 0xfffff80003121c00 "-" chain 63: thread 100016 (pid 0, kqueue_ctx taskq) sleeping on 0xfffff80003677d00 "-" chain 64: thread 100017 (pid 0, aiod_kick taskq) sleeping on 0xfffff80003677c00 "-" chain 65: thread 100020 (pid 0, thread taskq) sleeping on 0xfffff80003677900 "-" chain 66: thread 100022 (pid 0, if_config_tqg_0) sleeping on 0xfffff80003121b00 "-" chain 67: thread 100023 (pid 0, if_io_tqg_0) sleeping on 0xfffff80003121a00 "-" chain 68: thread 100024 (pid 0, if_io_tqg_1) sleeping on 0xfffff80003121900 "-" chain 69: thread 100025 (pid 0, if_io_tqg_2) sleeping on 0xfffff80003121800 "-" chain 70: thread 100026 (pid 0, if_io_tqg_3) sleeping on 0xfffff80003121700 "-" chain 71: thread 100027 (pid 0, if_io_tqg_4) sleeping on 0xfffff80003121600 "-" chain 72: thread 100028 (pid 0, if_io_tqg_5) sleeping on 0xfffff80003121500 "-" chain 73: thread 100029 (pid 0, if_io_tqg_6) sleeping on 0xfffff80003121400 "-" chain 74: thread 100030 (pid 0, if_io_tqg_7) sleeping on 0xfffff80003121300 "-" chain 75: thread 100031 (pid 0, if_io_tqg_8) sleeping on 0xfffff80003121200 "-" chain 76: thread 100032 (pid 0, if_io_tqg_9) sleeping on 0xfffff80003121100 "-" chain 77: thread 100033 (pid 0, if_io_tqg_10) sleeping on 0xfffff80003121000 "-" chain 78: thread 100034 (pid 0, if_io_tqg_11) sleeping on 0xfffff80003120e00 "-" chain 79: thread 100035 (pid 0, softirq_0) sleeping on 0xfffff800037b9100 "-" chain 80: thread 100036 (pid 0, softirq_1) sleeping on 0xfffff800037b9000 "-" chain 81: thread 100037 (pid 0, softirq_2) sleeping on 0xfffff800037b8e00 "-" chain 82: thread 100038 (pid 0, softirq_3) sleeping on 0xfffff800037b8d00 "-" chain 83: thread 100039 (pid 0, softirq_4) sleeping on 0xfffff800037b8c00 "-" chain 84: thread 100040 (pid 0, softirq_5) sleeping on 0xfffff800037b8b00 "-" chain 85: thread 100041 (pid 0, softirq_6) sleeping on 0xfffff800037b8a00 "-" chain 86: thread 100042 (pid 0, softirq_7) sleeping on 0xfffff800037b8900 "-" chain 87: thread 100043 (pid 0, softirq_8) sleeping on 0xfffff800037b8800 "-" chain 88: thread 100044 (pid 0, softirq_9) sleeping on 0xfffff800037b8700 "-" chain 89: thread 100045 (pid 0, softirq_10) sleeping on 0xfffff800037b8600 "-" chain 90: thread 100046 (pid 0, softirq_11) sleeping on 0xfffff800037b8500 "-" chain 91: thread 100061 (pid 0, firmware taskq) sleeping on 0xfffff800038b1100 "-" chain 92: thread 100066 (pid 0, crypto_0) sleeping on 0xfffff800038b1000 "-" chain 93: thread 100067 (pid 0, crypto_1) sleeping on 0xfffff800038b1000 "-" chain 94: thread 100068 (pid 0, crypto_2) sleeping on 0xfffff800038b1000 "-" chain 95: thread 100069 (pid 0, crypto_3) sleeping on 0xfffff800038b1000 "-" chain 96: thread 100070 (pid 0, crypto_4) sleeping on 0xfffff800038b1000 "-" chain 97: thread 100071 (pid 0, crypto_5) sleeping on 0xfffff800038b1000 "-" chain 98: thread 100072 (pid 0, crypto_6) sleeping on 0xfffff800038b1000 "-" chain 99: thread 100073 (pid 0, crypto_7) sleeping on 0xfffff800038b1000 "-" chain 100: thread 100074 (pid 0, crypto_8) sleeping on 0xfffff800038b1000 "-" chain 101: thread 100075 (pid 0, crypto_9) sleeping on 0xfffff800038b1000 "-" chain 102: thread 100076 (pid 0, crypto_10) sleeping on 0xfffff800038b1000 "-" chain 103: thread 100077 (pid 0, crypto_11) sleeping on 0xfffff800038b1000 "-" chain 104: thread 100114 (pid 0, mca taskq) blocked on lockmgr (null)EXCL KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0076760a80 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0076760a90 trap() at trap+0x5b/frame 0xfffffe0076760ba0 calltrap() at calltrap+0x8/frame 0xfffffe0076760ba0 --- trap 0xc, rip = 0xffffffff80c44f5a, rsp = 0xfffffe0076760c70, rbp = 0xfffffe0076760c90 --- print_lockchain() at print_lockchain+0x3a/frame 0xfffffe0076760c90 db_show_allchains() at db_show_allchains+0x80/frame 0xfffffe0076760cc0 db_command() at db_command+0x279/frame 0xfffffe0076760d90 db_script_exec() at db_script_exec+0x188/frame 0xfffffe0076760e30 db_command() at db_command+0x279/frame 0xfffffe0076760f00 db_script_exec() at db_script_exec+0x188/frame 0xfffffe0076760fa0 db_command() at db_command+0x279/frame 0xfffffe0076761070 db_command_loop() at db_command_loop+0x64/frame 0xfffffe0076761080 db_trap() at db_trap+0xef/frame 0xfffffe0076761110 kdb_trap() at kdb_trap+0x16c/frame 0xfffffe0076761190 trap() at trap+0x811/frame 0xfffffe00767612a0 calltrap() at calltrap+0x8/frame 0xfffffe00767612a0 --- trap 0x3, rip = 0xffffffff80c2bdbb, rsp = 0xfffffe0076761370, rbp = 0xfffffe0076761380 --- kdb_enter() at kdb_enter+0x3b/frame 0xfffffe0076761380 vpanic() at vpanic+0x1ba/frame 0xfffffe00767613d0 panic() at panic+0x43/frame 0xfffffe0076761430 mtrash_ctor() at mtrash_ctor+0x7e/frame 0xfffffe0076761450 uma_zalloc_arg() at uma_zalloc_arg+0x9ff/frame 0xfffffe00767614e0 malloc() at malloc+0x9b/frame 0xfffffe0076761530 xpt_run_allocq() at xpt_run_allocq+0xcb/frame 0xfffffe0076761580 adastrategy() at adastrategy+0x6e/frame 0xfffffe00767615b0 g_disk_start() at g_disk_start+0x351/frame 0xfffffe0076761610 g_io_request() at g_io_request+0x3f5/frame 0xfffffe0076761660 g_part_start() at g_part_start+0x120/frame 0xfffffe00767616e0 g_io_request() at g_io_request+0x3f5/frame 0xfffffe0076761730 g_io_request() at g_io_request+0x3f5/frame 0xfffffe0076761780 bufwrite() at bufwrite+0x24d/frame 0xfffffe00767617d0 vfs_bio_awrite() at vfs_bio_awrite+0x212/frame 0xfffffe0076761830 vn_fsync_buf() at vn_fsync_buf+0x21f/frame 0xfffffe0076761890 VOP_FSYNC_APV() at VOP_FSYNC_APV+0x88/frame 0xfffffe00767618b0 ffs_sync() at ffs_sync+0x2f9/frame 0xfffffe0076761960 sys_sync() at sys_sync+0xbe/frame 0xfffffe0076761990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe0076761ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0076761ab0 --- syscall (36, FreeBSD ELF64, sys_sync), rip = 0x24dbca, rsp = 0x7fffffffe6f8, rbp = 0x7fffffffe7e0 --- Script command 'show allchains' returned error db:1:pho1> show mount 0xfffff80007b6a000 /dev/gpt/work on / (ufs) 0xfffff80007b6b000 devfs on /dev (devfs) 0xfffff80231a42000 /dev/gpt/tmp on /tmp (ufs) More info: show mount db:1:pho1> show bufqueues bqempty: 21167 Buf domain 0 freebufs 3431 lofreebufs 219 hifreebufs 329 bufspace 88289280 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 1 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 11572 (85422080) dirtyq count 1 (4096) wakeup 0 lim 25 CPU 8, 3, 20, 9, 6, 0, 0, 15, 18, 21, 9, 17, Locked buffers: 3 space 98304 Total buffers: 11822 space 88289280 Buf domain 1 freebufs 1718 lofreebufs 219 hifreebufs 329 bufspace 235466752 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 19 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 13303 (232591360) dirtyq count 36 (1179648) wakeup 0 lim 25 CPU 4, 4, 6, 2, 2, 6, 8, 7, 13, 1, 3, 20, Locked buffers: 17 space 557056 Total buffers: 13779 space 235466752 Buf domain 2 freebufs 3510 lofreebufs 219 hifreebufs 329 bufspace 87150592 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 11520 (85114880) dirtyq count 0 (0) wakeup 0 lim 25 CPU 1, 15, 11, 0, 3, 3, 15, 13, 3, 12, 19, 8, Locked buffers: 0 space 0 Total buffers: 11628 space 87150592 Buf domain 3 freebufs 3577 lofreebufs 219 hifreebufs 329 bufspace 87781376 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 1 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 11473 (86470656) dirtyq count 1 (4096) wakeup 0 lim 25 CPU 0, 0, 4, 0, 3, 23, 14, 13, 6, 0, 8, 11, Locked buffers: 0 space 0 Total buffers: 11558 space 87781376 Buf domain 4 freebufs 2737 lofreebufs 219 hifreebufs 329 bufspace 109621248 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 12242 (106475520) dirtyq count 0 (0) wakeup 0 lim 25 CPU 3, 2, 17, 10, 0, 17, 22, 19, 8, 17, 17, 22, Locked buffers: 0 space 0 Total buffers: 12396 space 109621248 Buf domain 5 freebufs 3260 lofreebufs 219 hifreebufs 329 bufspace 96665600 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 11785 (94834688) dirtyq count 0 (0) wakeup 0 lim 25 CPU 16, 9, 8, 7, 9, 7, 4, 19, 0, 1, 1, 7, Locked buffers: 0 space 0 Total buffers: 11880 space 96665600 Buf domain 6 freebufs 3438 lofreebufs 219 hifreebufs 329 bufspace 88981504 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 11581 (86478848) dirtyq count 0 (0) wakeup 0 lim 25 CPU 13, 19, 20, 0, 5, 10, 0, 0, 13, 20, 2, 12, Locked buffers: 0 space 0 Total buffers: 11701 space 88981504 db:1:pho1> show page vm_cnt.v_free_count: 7575845 vm_cnt.v_inactive_count: 294 vm_cnt.v_active_count: 981 vm_cnt.v_laundry_count: 854 vm_cnt.v_wire_count: 547411 vm_cnt.v_free_reserved: 10649 vm_cnt.v_free_min: 51289 vm_cnt.v_free_target: 173209 vm_cnt.v_inactive_target: 259813 db:1:pho1> show pageq pq_free 7575845 dom 0 page_cnt 8128423 free 7575845 pq_act 981 pq_inact 294 pq_laund 854 pq_unsw 0 db:0:pho> show freepages DOMAIN: 0 FREE LIST 0: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000017 | 000000 10 (004096K) | 000457 | 000000 09 (002048K) | 002747 | 000000 08 (001024K) | 004690 | 000031 07 (000512K) | 013153 | 000007 06 (000256K) | 016632 | 000030 05 (000128K) | 020936 | 000187 04 (000064K) | 025427 | 000503 03 (000032K) | 030658 | 002089 02 (000016K) | 036721 | 005494 01 (000008K) | 047529 | 008256 00 (000004K) | 062131 | 008161 FREE LIST 1: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000000 | 000000 10 (004096K) | 000000 | 000000 09 (002048K) | 000000 | 000000 08 (001024K) | 000000 | 000000 07 (000512K) | 000000 | 000000 06 (000256K) | 000000 | 000000 05 (000128K) | 000000 | 000000 04 (000064K) | 000005 | 000000 03 (000032K) | 000003 | 000000 02 (000016K) | 000004 | 000000 01 (000008K) | 000003 | 000000 00 (000004K) | 000004 | 000000 db:0:pho> show uma Zone Size Used Free Requests Sleeps Bucket FFS2 dinode 256 619511 919 6480692 0 61 FFS1 dinode 128 0 0 0 0 125 FFS inode 160 619511 1033 6480693 0 61 swblk 136 3643 216235 3207876 0 61 swpctrie 144 3 15603 258480 0 61 selfd 64 237 2119 849688 0 253 rtentry 208 12 121 15 0 61 ripcb 488 0 8 6 0 29 udplite_inpcb 488 0 0 0 0 29 sctp_asconf_ack 48 0 0 0 0 253 sctp_asconf 40 0 0 0 0 253 sctp_stream_msg_out 112 0 0 0 0 125 sctp_readq 152 0 0 0 0 61 sctp_chunk 152 0 0 0 0 61 sctp_raddr 736 0 0 0 0 13 sctp_laddr 48 0 0 4 0 253 sctp_asoc 2408 0 0 0 0 3 sctp_ep 1280 0 0 0 0 9 tcp_log_node 120 0 0 0 0 125 tcp_log_bucket 144 0 0 0 0 61 tcp_log 408 0 0 0 0 29 tcpreass 48 0 0 0 0 253 tfo_ccache_entries 80 0 0 0 0 125 tfo 4 0 0 0 0 253 sackhole 32 0 0 0 0 253 hostcache 96 2 162 2 0 125 syncache 168 0 736 4763302 0 61 tcptw 88 0 1530 76574 0 125 tcpcb 976 0 376 16696796 0 13 tcp_inpcb 488 0 648 16696796 0 30 udpcb 32 2 1858 6412 0 253 udp_inpcb 488 2 254 6412 0 29 ipq 56 0 0 0 0 253 IPsec SA lft_c 16 0 0 0 0 253 unpcb 256 0 390 6456 0 61 socket 872 2 4302 16709954 0 13 KNOTE 160 0 264 13 0 61 itimer 352 0 0 0 0 29 ksiginfo 112 3698 2532 128104915 0 126 procdesc 136 0 29 10 0 61 pipe 760 0 730 13943867 0 14 AIOLIO 280 0 0 0 0 29 AIOCB 752 0 0 0 0 13 AIOP 32 0 0 0 0 253 AIO 208 0 0 0 0 61 Mountpoints 2896 3 6 4 0 3 DIRHASH 1024 26 26 12078 0 13 NCLNODE 744 0 0 0 0 13 rentr 24 0 664 5 0 253 NAMEI 1024 0 460 667589200 0 15 LTS VFS Cache 368 0 0 0 0 29 L VFS Cache 328 178566 3582 374848 0 29 STS VFS Cache 148 0 0 0 0 61 S VFS Cache 108 449758 104117 10351384 0 125 BUF TRIE 144 3513 102435 126988 0 61 VNODEPOLL 120 0 0 0 0 125 VNODE 632 619618 500 6480813 0 13 vtnet_tx_hdr 24 0 0 0 0 253 crypto_session 24 0 0 0 0 253 cryptodesc 120 0 0 0 0 125 cryptop 128 0 0 0 0 125 nvme_request 128 0 0 0 0 125 g_bio 408 80 568 14596456 0 29 ttyoutq 256 0 345 476 0 61 ttyinq 160 0 624 915 0 61 DMAR_MAP_ENTRY 120 0 0 0 0 125 FPU_save_area 832 0 0 0 0 13 epoch_record pcpu 256 4 12 4 0 61 mbuf_jumbo_16k 16384 0 0 0 0 1 mbuf_jumbo_9k 9216 0 0 0 0 1 mbuf_jumbo_page 4096 0 0 4 0 1 mbuf_cluster 2048 8198 30 80167 0 9 mbuf 256 6262 9682 20264063071 0 253 mbuf_packet 256 28 2028 6847 0 253 audit_record 1280 0 0 0 0 9 domainset 40 0 0 0 0 253 cpuset 104 27 500 27 0 125 THREAD 1416 3840 72 18461 0 9 PROC 1328 34 3689 33621870 0 10 MAC labels 40 0 0 0 0 253 pbuf 984 0 915 0 0 1 clpbuf 984 0 0 1939 0 5 vnpbuf 984 0 0 0 0 13 mdpbuf 984 0 0 0 0 1 nfspbuf 984 0 0 0 0 5 swwbuf 984 0 23 1888734 0 1 swrbuf 984 0 73 1472853 0 5 umtx_shm 88 0 0 0 0 125 umtx pi 96 0 0 0 0 125 TURNSTILE 136 3913 287 3913 0 61 rangeset pctrie nodes 144 0 0 0 0 61 rl_entry 40 3721 2318 3721 0 253 filedesc0 1104 35 2815 33621871 0 14 Files 80 0 7301 365034061 0 127 SLEEPQUEUE 88 3913 582 3913 0 125 65536 65536 0 0 0 0 1 65536 65536 0 0 0 0 1 65536 65536 6 0 2161 0 1 65536 65536 1 0 1 0 1 65536 65536 3 0 3 0 1 65536 65536 3 0 3 0 1 65536 65536 1 0 1 0 1 65536 65536 0 2 3150 0 1 32768 32768 0 0 0 0 1 32768 32768 0 0 3 0 1 32768 32768 2 0 2502 0 1 32768 32768 12 0 12 0 1 32768 32768 8 17 5528 0 2 32768 32768 0 0 0 0 1 32768 32768 14 0 15 0 1 32768 32768 1 3 2387 0 1 16384 16384 0 0 0 0 1 16384 16384 1 3 1265 0 1 16384 16384 15 0 3020 0 1 16384 16384 0 0 0 0 1 16384 16384 0 5 10 0 1 16384 16384 0 0 0 0 1 16384 16384 5 0 8 0 1 16384 16384 1 0 258 0 1 8192 8192 0 0 0 0 1 8192 8192 2 1 1266 0 1 8192 8192 14 0 2210 0 1 8192 8192 13 0 13 0 1 8192 8192 8 0 8 0 1 8192 8192 3 0 4 0 1 8192 8192 87 0 90 0 1 8192 8192 4 0 6 0 1 4096 4096 0 12 4768490 0 1 4096 4096 0 1 1264 0 1 4096 4096 31 0 2213 0 1 4096 4096 6 20 21665788 0 1 4096 4096 8 2 10 0 1 4096 4096 418 2931 33712518 0 4 4096 4096 2 0 18 0 1 4096 4096 0 4 3785 0 1 2048 2048 0 0 0 0 5 2048 2048 0 6 1264 0 5 2048 2048 12 4 697 0 5 2048 2048 17 5 36 0 5 2048 2048 29 91 3653631 0 5 2048 2048 90 2 91 0 5 2048 2048 2 2 8 0 5 2048 2048 4 4 50 0 5 1024 1024 0 0 0 0 13 1024 1024 137 27 1416 0 13 1024 1024 36 20 2480 0 13 1024 1024 19 13 21 0 13 1024 1024 4 156 3485 0 13 1024 1024 50 162 17973 0 13 1024 1024 21 19 5664 0 13 1024 1024 7 185 19862886 0 13 512 512 0 0 0 0 29 512 512 52 74 1320 0 29 512 512 22 118 6529 0 29 512 512 29 41 30 0 29 512 512 26 58 1738 0 29 512 512 3831 54 3912 0 29 512 512 15 20 37 0 29 512 512 1166 542 22253298 0 29 256 256 14 376 82252 0 61 256 256 170 550 2118 0 61 256 256 133 1562 35835134 0 61 256 256 74 11521 28747294 0 61 256 256 52 743 9579391 0 61 256 256 613 677 4801380 0 61 256 256 182 3508 33653622 0 61 256 256 9 381 3655 0 61 128 128 0 0 1278 0 125 128 128 343 1517 79956 0 125 128 128 897 405 1844 0 125 128 128 97 399 101 0 125 128 128 4043 2684 37083 0 125 128 128 732 1500 4781121 0 125 128 128 14413 1552 5688432 0 125 128 128 7864 2211 19865616 0 125 64 64 0 124 4 0 253 64 64 129 1049 2403 0 253 64 64 61 4837 38402250 0 253 64 64 243 625 246 0 253 64 64 727 1071 25837 0 253 64 64 9015 657 9411 0 253 64 64 9357 14699 270642294 0 253 64 64 3 4212 30112931 0 253 32 32 3 1857 4101 0 253 32 32 142 602 3993 0 253 32 32 66 802 1414 0 253 32 32 64 1300 27286 0 253 32 32 7156 3136 234869 0 253 32 32 186 930 9856 0 253 32 32 788 3180 16726472 0 253 32 32 134 3834 23090423 0 253 16 16 0 0 4 0 253 16 16 55 1937 59661 0 253 16 16 52 2189 1760 0 253 16 16 30 1215 30 0 253 16 16 2832 2397 3202 0 253 16 16 436 2054 21092 0 253 16 16 469 3266 17550554 0 253 16 16 82 1412 2742 0 253 mt_zone 24 428 402 429 0 253 mt_stats_zone 64 428 84 429 0 253 64 pcpu 8 3343 2289 3730 0 253 fakepg 104 0 1520 2520 0 125 VMSPACE 2552 1 3636 33621838 0 8 MAP ENTRY 112 12 229133 2208125656 0 253 KMAP ENTRY 112 7 166 7 0 125 MAP 232 2 65 2 0 61 RADIX NODE 144 22151 528907 883677632 0 253 VM OBJECT 256 110555 107485 666943415 0 139 vmem btag 56 67789 2127 68406 0 253 vmem 1792 0 1 0 0 5 256 Bucket 2048 2094 241 16884308 0 253 128 Bucket 1024 320 1180 1359405 0 15 64 Bucket 512 602 259 1505748 0 29 32 Bucket 256 297 542 3643354 0 61 16 Bucket 128 488 1588 2847299 0 125 12 Bucket 96 386 1500 974907 0 125 8 Bucket 64 699 3392 1213263 0 253 6 Bucket 48 42 3029 1396344 0 253 4 Bucket 32 2240 3464 2207192 0 253 UMA Hash 256 1 10 7 0 0 UMA Slabs 112 9849 13 463646 0 0 UMA Zones 1960 80 0 80 0 0 UMA Kegs 216 74 5 74 0 0 db:0:pho> show umacache Zone Size Used Free Requests Bucket vm pgcache 104 -87995172 4443 2806897111 253 buf free cache 984 84256 504 249941 13 buffer arena-65 65536 0 0 7040 1 buffer arena-61 61440 0 0 7040 1 buffer arena-57 57344 0 0 7040 1 buffer arena-53 53248 0 0 7040 1 buffer arena-49 49152 0 0 7040 1 buffer arena-45 45056 0 0 7040 1 buffer arena-40 40960 0 0 7040 1 buffer arena-36 36864 0 0 7040 1 buffer arena-32 32768 50323 2222 199803 2 buffer arena-28 28672 0 0 7040 1 buffer arena-24 24576 0 0 7040 1 buffer arena-20 20480 0 0 7040 1 buffer arena-16 16384 3 6 7057 1 buffer arena-12 12288 0 0 7040 1 buffer arena-81 8192 0 0 7040 1 buffer arena-40 4096 0 0 7040 1 db:0:pho> acttrace Tracing command cam pid 20 tid 100092 td 0xfffff80003aea5a0 (CPU 9) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bd3df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bd3e10 trap() at trap+0x48/frame 0xfffffe0006bd3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bd3f20 --- trap 0x13, rip = 0xffffffff80c2d400, rsp = 0xfffffe0006e4a890, rbp = 0xfffffe0006e4a8a0 --- lock_delay() at lock_delay+0x40/frame 0xfffffe0006e4a8a0 __mtx_lock_sleep() at __mtx_lock_sleep+0xe8/frame 0xfffffe0006e4a930 __mtx_lock_flags() at __mtx_lock_flags+0xee/frame 0xfffffe0006e4a980 adadone() at adadone+0x68/frame 0xfffffe0006e4a9e0 xpt_done_process() at xpt_done_process+0x388/frame 0xfffffe0006e4aa20 xpt_done_td() at xpt_done_td+0xf6/frame 0xfffffe0006e4aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e4aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e4aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100003 td 0xfffff800036b8000 (CPU 0) cpustop_handler() at cpustop_handler+0x2d/frame 0xffffffff820608b0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xffffffff820608d0 trap() at trap+0x48/frame 0xffffffff820609e0 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff820609e0 --- trap 0x13, rip = 0xffffffff8122aef6, rsp = 0xfffffe0076766900, rbp = 0xfffffe0076766900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076766900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076766950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076766970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe0076766990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe0076766a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076766ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076766ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xfffff800036b75a0 (CPU 1) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006b73df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006b73e10 trap() at trap+0x48/frame 0xfffffe0006b73f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b73f20 --- trap 0x13, rip = 0xffffffff8122aef6, rsp = 0xfffffe007676b900, rbp = 0xfffffe007676b900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007676b900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe007676b950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007676b970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe007676b990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe007676ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe007676bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007676bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffff800036b7000 (CPU 2) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006b7fdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006b7fe10 trap() at trap+0x48/frame 0xfffffe0006b7ff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b7ff20 --- trap 0x13, rip = 0xffffffff8122aef6, rsp = 0xfffffe0076770900, rbp = 0xfffffe0076770900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076770900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076770950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076770970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe0076770990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe0076770a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076770ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076770ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xfffff800036b65a0 (CPU 3) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006b8bdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006b8be10 trap() at trap+0x48/frame 0xfffffe0006b8bf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b8bf20 --- trap 0x13, rip = 0xffffffff8122aef6, rsp = 0xfffffe0076775900, rbp = 0xfffffe0076775900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076775900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076775950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076775970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe0076775990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe0076775a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076775ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076775ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100007 td 0xfffff800036b6000 (CPU 4) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006b97df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006b97e10 trap() at trap+0x48/frame 0xfffffe0006b97f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b97f20 --- trap 0x13, rip = 0xffffffff8122aef6, rsp = 0xfffffe007677a900, rbp = 0xfffffe007677a900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007677a900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe007677a950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007677a970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe007677a990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe007677aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007677aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007677aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xfffff800036b05a0 (CPU 5) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006ba3df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006ba3e10 trap() at trap+0x48/frame 0xfffffe0006ba3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006ba3f20 --- trap 0x13, rip = 0xffffffff8122aef6, rsp = 0xfffffe007677f900, rbp = 0xfffffe007677f900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007677f900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe007677f950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007677f970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe007677f990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe007677fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007677fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007677fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xfffff800036b0000 (CPU 6) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bafdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bafe10 trap() at trap+0x48/frame 0xfffffe0006baff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006baff20 --- trap 0x13, rip = 0xffffffff8122aef6, rsp = 0xfffffe0076784900, rbp = 0xfffffe0076784900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076784900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076784950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076784970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe0076784990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe0076784a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076784ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076784ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffff800036c9000 (CPU 8) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bc7df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bc7e10 trap() at trap+0x48/frame 0xfffffe0006bc7f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bc7f20 --- trap 0x13, rip = 0xffffffff8122aef6, rsp = 0xfffffe007678e900, rbp = 0xfffffe007678e900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007678e900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe007678e950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007678e970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe007678e990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe007678ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe007678eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007678eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xfffff800036c8000 (CPU 10) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bdfdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bdfe10 trap() at trap+0x48/frame 0xfffffe0006bdff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bdff20 --- trap 0x13, rip = 0xffffffff8122aef6, rsp = 0xfffffe0076798900, rbp = 0xfffffe0076798900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076798900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076798950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076798970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe0076798990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe0076798a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076798ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076798ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100014 td 0xfffff800036c75a0 (CPU 11) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bebdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bebe10 trap() at trap+0x48/frame 0xfffffe0006bebf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bebf20 --- trap 0x13, rip = 0xffffffff8122aef6, rsp = 0xfffffe007679d900, rbp = 0xfffffe007679d900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007679d900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe007679d950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007679d970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe007679d990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe007679da70 fork_exit() at fork_exit+0x84/frame 0xfffffe007679dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007679dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command init pid 1 tid 100002 td 0xfffff800036b85a0 (CPU 7) kdb_enter() at kdb_enter+0x3b/frame 0xfffffe0076761380 vpanic() at vpanic+0x1ba/frame 0xfffffe00767613d0 panic() at panic+0x43/frame 0xfffffe0076761430 mtrash_ctor() at mtrash_ctor+0x7e/frame 0xfffffe0076761450 uma_zalloc_arg() at uma_zalloc_arg+0x9ff/frame 0xfffffe00767614e0 malloc() at malloc+0x9b/frame 0xfffffe0076761530 xpt_run_allocq() at xpt_run_allocq+0xcb/frame 0xfffffe0076761580 adastrategy() at adastrategy+0x6e/frame 0xfffffe00767615b0 g_disk_start() at g_disk_start+0x351/frame 0xfffffe0076761610 g_io_request() at g_io_request+0x3f5/frame 0xfffffe0076761660 g_part_start() at g_part_start+0x120/frame 0xfffffe00767616e0 g_io_request() at g_io_request+0x3f5/frame 0xfffffe0076761730 g_io_request() at g_io_request+0x3f5/frame 0xfffffe0076761780 bufwrite() at bufwrite+0x24d/frame 0xfffffe00767617d0 vfs_bio_awrite() at vfs_bio_awrite+0x212/frame 0xfffffe0076761830 vn_fsync_buf() at vn_fsync_buf+0x21f/frame 0xfffffe0076761890 VOP_FSYNC_APV() at VOP_FSYNC_APV+0x88/frame 0xfffffe00767618b0 ffs_sync() at ffs_sync+0x2f9/frame 0xfffffe0076761960 sys_sync() at sys_sync+0xbe/frame 0xfffffe0076761990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe0076761ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0076761ab0 --- syscall (36, FreeBSD ELF64, sys_sync), rip = 0x24dbca, rsp = 0x7fffffffe6f8, rbp = 0x7fffffffe7e0 --- db:0:pho> ps pid ppid pgrp uid state wmesg wchan cmd 34 0 0 0 DL vlruwt 0xfffff80007884a60 [vnlru] 33 0 0 0 DL syncer 0xffffffff820401c0 [syncer] 32 0 0 0 DL (threaded) [bufdaemon] 100130 D qsleep 0xffffffff8203f658 [bufdaemon] 100131 D - 0xffffffff81a0a8c0 [bufspacedaemon-0] 100132 D - 0xffffffff81a12b80 [bufspacedaemon-1] 100133 D - 0xffffffff81a1ae40 [bufspacedaemon-2] 100135 D - 0xffffffff81a23100 [bufspacedaemon-3] 100136 D - 0xffffffff81a2b3c0 [bufspacedaemon-4] 100137 D - 0xffffffff81a33680 [bufspacedaemon-5] 100138 D - 0xffffffff81a3b940 [bufspacedaemon-6] 100153 D sdflush 0xfffff800162626e8 [/ worker] 100199 D sdflush 0xfffff80007a29ae8 [/tmp worker] 31 0 0 0 DL psleep 0xffffffff8205c548 [vmdaemon] 30 0 0 0 DL (threaded) [pagedaemon] 100128 D psleep 0xffffffff820c6298 [dom0] 100134 D launds 0xffffffff820c62a4 [laundry: dom0] 100139 D umarcl 0xffffffff80f19660 [uma] 29 0 0 0 DL idle 0xfffff80007786000 [enc_daemon1] 28 0 0 0 DL idle 0xfffff80007786530 [enc_daemon0] 27 0 0 0 DL - 0xffffffff81cfd430 [rand_harvestq] 26 0 0 0 DL waiting 0xffffffff820c4970 [sctp_iterator] 25 0 0 0 DL - 0xffffffff8203f05c [soaiod4] 24 0 0 0 DL - 0xffffffff8203f05c [soaiod3] 23 0 0 0 DL - 0xffffffff8203f05c [soaiod2] 22 0 0 0 DL - 0xffffffff8203f05c [soaiod1] 21 0 0 0 DL (threaded) [usb] 100096 D - 0xfffffe009a1aa460 [usbus0] 100097 D - 0xfffffe009a1aa4b8 [usbus0] 100098 D - 0xfffffe009a1aa510 [usbus0] 100099 D - 0xfffffe009a1aa568 [usbus0] 100100 D - 0xfffffe009a1aa5c0 [usbus0] 100102 D - 0xfffffe0006dd6d10 [usbus1] 100103 D - 0xfffffe0006dd6d68 [usbus1] 100104 D - 0xfffffe0006dd6dc0 [usbus1] 100105 D - 0xfffffe0006dd6e18 [usbus1] 100106 D - 0xfffffe0006dd6e70 [usbus1] 100107 D - 0xfffffe009a63bd10 [usbus2] 100108 D - 0xfffffe009a63bd68 [usbus2] 100109 D - 0xfffffe009a63bdc0 [usbus2] 100110 D - 0xfffffe009a63be18 [usbus2] 100111 D - 0xfffffe009a63be70 [usbus2] 20 0 0 0 RL (threaded) [cam] 100091 D - 0xffffffff81bd86c0 [doneq0] 100092 Run CPU 9 [doneq1] 100093 D - 0xffffffff81bd87c0 [doneq2] 100125 D - 0xffffffff81bd8588 [scanner] 19 0 0 0 DL crypto_ 0xfffff80003978c50 [crypto returns 11] 18 0 0 0 DL crypto_ 0xfffff80003978bf0 [crypto returns 10] 17 0 0 0 DL crypto_ 0xfffff80003978b90 [crypto returns 9] 16 0 0 0 DL crypto_ 0xfffff80003978b30 [crypto returns 8] 15 0 0 0 DL crypto_ 0xfffff80003978ad0 [crypto returns 7] 9 0 0 0 DL crypto_ 0xfffff80003978a70 [crypto returns 6] 8 0 0 0 DL crypto_ 0xfffff80003978a10 [crypto returns 5] 7 0 0 0 DL crypto_ 0xfffff800039789b0 [crypto returns 4] 6 0 0 0 DL crypto_ 0xfffff80003978950 [crypto returns 3] 5 0 0 0 DL crypto_ 0xfffff800039788f0 [crypto returns 2] 4 0 0 0 DL crypto_ 0xfffff80003978890 [crypto returns 1] 3 0 0 0 DL crypto_ 0xfffff80003978830 [crypto returns 0] 2 0 0 0 DL crypto_ 0xffffffff82056b40 [crypto] 14 0 0 0 DL seqstat 0xfffff80003893c88 [sequencer 00] 13 0 0 0 DL (threaded) [geom] 100062 D - 0xffffffff82094e20 [g_event] 100063 D - 0xffffffff82094e30 [g_up] 100064 D - 0xffffffff82094e18 [g_down] 12 0 0 0 WL (threaded) [intr] 100018 I [swi6: task queue] 100019 I [swi6: Giant taskq] 100021 I [swi5: fast taskq] 100047 I [swi4: clock (0)] 100048 I [swi4: clock (1)] 100049 I [swi4: clock (2)] 100050 I [swi4: clock (3)] 100051 I [swi4: clock (4)] 100052 I [swi4: clock (5)] 100053 I [swi4: clock (6)] 100054 I [swi4: clock (7)] 100055 I [swi4: clock (8)] 100056 I [swi4: clock (9)] 100057 I [swi4: clock (10)] 100058 I [swi4: clock (11)] 100059 I [swi3: vm] 100060 I [swi1: netisr 0] 100094 I [irq56: ahci0] 100095 I [irq19: xhci0] 100101 I [irq18: ehci0 ehci1] 100112 I [irq71: ahci1] 100113 I [swi0: uart uart] 100181 I [irq72: ioat0] 100182 I [irq73: ioat1] 100183 I [irq74: ioat2] 100184 I [irq75: ioat3] 100185 I [irq76: ioat4] 100186 I [irq77: ioat5] 100187 I [irq78: ioat6] 100188 I [irq79: ioat7] 11 0 0 0 RL (threaded) [idle] 100003 Run CPU 0 [idle: cpu0] 100004 Run CPU 1 [idle: cpu1] 100005 Run CPU 2 [idle: cpu2] 100006 Run CPU 3 [idle: cpu3] 100007 Run CPU 4 [idle: cpu4] 100008 Run CPU 5 [idle: cpu5] 100009 Run CPU 6 [idle: cpu6] 100010 CanRun [idle: cpu7] 100011 Run CPU 8 [idle: cpu8] 100012 CanRun [idle: cpu9] 100013 Run CPU 10 [idle: cpu10] 100014 Run CPU 11 [idle: cpu11] 1 0 1 0 RLs CPU 7 [init] 10 0 0 0 DL audit_w 0xffffffff820c5690 [audit] 0 0 0 0 DLs (threaded) [kernel] 100000 D swapin 0xffffffff82094e60 [swapper] 100015 D - 0xfffff80003121c00 [config_0] 100016 D - 0xfffff80003677d00 [kqueue_ctx taskq] 100017 D - 0xfffff80003677c00 [aiod_kick taskq] 100020 D - 0xfffff80003677900 [thread taskq] 100022 D - 0xfffff80003121b00 [if_config_tqg_0] 100023 D - 0xfffff80003121a00 [if_io_tqg_0] 100024 D - 0xfffff80003121900 [if_io_tqg_1] 100025 D - 0xfffff80003121800 [if_io_tqg_2] 100026 D - 0xfffff80003121700 [if_io_tqg_3] 100027 D - 0xfffff80003121600 [if_io_tqg_4] 100028 D - 0xfffff80003121500 [if_io_tqg_5] 100029 D - 0xfffff80003121400 [if_io_tqg_6] 100030 D - 0xfffff80003121300 [if_io_tqg_7] 100031 D - 0xfffff80003121200 [if_io_tqg_8] 100032 D - 0xfffff80003121100 [if_io_tqg_9] 100033 D - 0xfffff80003121000 [if_io_tqg_10] 100034 D - 0xfffff80003120e00 [if_io_tqg_11] 100035 D - 0xfffff800037b9100 [softirq_0] 100036 D - 0xfffff800037b9000 [softirq_1] 100037 D - 0xfffff800037b8e00 [softirq_2] 100038 D - 0xfffff800037b8d00 [softirq_3] 100039 D - 0xfffff800037b8c00 [softirq_4] 100040 D - 0xfffff800037b8b00 [softirq_5] 100041 D - 0xfffff800037b8a00 [softirq_6] 100042 D - 0xfffff800037b8900 [softirq_7] 100043 D - 0xfffff800037b8800 [softirq_8] 100044 D - 0xfffff800037b8700 [softirq_9] 100045 D - 0xfffff800037b8600 [softirq_10] 100046 D - 0xfffff800037b8500 [softirq_11] 100061 D - 0xfffff800038b1100 [firmware taskq] 100066 D - 0xfffff800038b1000 [crypto_0] 100067 D - 0xfffff800038b1000 [crypto_1] 100068 D - 0xfffff800038b1000 [crypto_2] 100069 D - 0xfffff800038b1000 [crypto_3] 100070 D - 0xfffff800038b1000 [crypto_4] 100071 D - 0xfffff800038b1000 [crypto_5] 100072 D - 0xfffff800038b1000 [crypto_6] 100073 D - 0xfffff800038b1000 [crypto_7] 100074 D - 0xfffff800038b1000 [crypto_8] 100075 D - 0xfffff800038b1000 [crypto_9] 100076 D - 0xfffff800038b1000 [crypto_10] 100077 D - 0xfffff800038b1000 [crypto_11] 100114 D - 0xfffff800060c0000 [mca taskq] 100120 D - 0xfffff8000781ee00 [acpi_task_0] 100121 D - 0xfffff8000781ee00 [acpi_task_1] 100122 D - 0xfffff8000781ee00 [acpi_task_2] 100124 D - 0xfffff800038b0e00 [CAM taskq] db:0:pho> allt Tracing command vnlru pid 34 tid 100141 td 0xfffff80007b855a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00a31fd890 mi_switch() at mi_switch+0x174/frame 0xfffffe00a31fd8c0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00a31fd900 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00a31fd940 _sleep() at _sleep+0x279/frame 0xfffffe00a31fd9e0 vnlru_proc() at vnlru_proc+0xa4/frame 0xfffffe00a31fda70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a31fdab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a31fdab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command syncer pid 33 tid 100140 td 0xfffff80007b87000 sched_switch() at sched_switch+0x713/frame 0xfffffe00a31f88c0 mi_switch() at mi_switch+0x174/frame 0xfffffe00a31f88f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00a31f8930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00a31f8970 _cv_timedwait_sbt() at _cv_timedwait_sbt+0x185/frame 0xfffffe00a31f89e0 sched_sync() at sched_sync+0x609/frame 0xfffffe00a31f8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a31f8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a31f8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100130 td 0xfffff80007b875a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006f448d0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006f44900 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006f44940 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006f44980 _sleep() at _sleep+0x279/frame 0xfffffe0006f44a20 buf_daemon() at buf_daemon+0xc8/frame 0xfffffe0006f44a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f44ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f44ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100131 td 0xfffff80003ae95a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006fe98c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006fe98f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006fe9930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006fe9970 _sleep() at _sleep+0x279/frame 0xfffffe0006fe9a10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe0006fe9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006fe9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006fe9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100132 td 0xfffff80003ae9000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006fee8c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006fee8f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006fee930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006fee970 _sleep() at _sleep+0x279/frame 0xfffffe0006feea10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe0006feea70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006feeab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006feeab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100133 td 0xfffff80003ae85a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006ff38c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ff38f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ff3930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006ff3970 _sleep() at _sleep+0x279/frame 0xfffffe0006ff3a10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe0006ff3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ff3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ff3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100135 td 0xfffff80003ae8000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006ff88c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ff88f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ff8930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006ff8970 _sleep() at _sleep+0x279/frame 0xfffffe0006ff8a10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe0006ff8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ff8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ff8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100136 td 0xfffff80003ae75a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006ffd8c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ffd8f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ffd930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006ffd970 _sleep() at _sleep+0x279/frame 0xfffffe0006ffda10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe0006ffda70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ffdab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ffdab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100137 td 0xfffff80003ae7000 sched_switch() at sched_switch+0x713/frame 0xfffffe00a30648c0 mi_switch() at mi_switch+0x174/frame 0xfffffe00a30648f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00a3064930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00a3064970 _sleep() at _sleep+0x279/frame 0xfffffe00a3064a10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe00a3064a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a3064ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a3064ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100138 td 0xfffff800039815a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00a31ae8c0 mi_switch() at mi_switch+0x174/frame 0xfffffe00a31ae8f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00a31ae930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00a31ae970 _sleep() at _sleep+0x279/frame 0xfffffe00a31aea10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe00a31aea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a31aeab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a31aeab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100153 td 0xfffff80016281000 sched_switch() at sched_switch+0x713/frame 0xfffffe00aa81d8f0 mi_switch() at mi_switch+0x174/frame 0xfffffe00aa81d920 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00aa81d960 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00aa81d9a0 _sleep() at _sleep+0x279/frame 0xfffffe00aa81da40 softdep_flush() at softdep_flush+0x17f/frame 0xfffffe00aa81da70 fork_exit() at fork_exit+0x84/frame 0xfffffe00aa81dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00aa81dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100199 td 0xfffff80016c955a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00aa8368f0 mi_switch() at mi_switch+0x174/frame 0xfffffe00aa836920 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00aa836960 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00aa8369a0 _sleep() at _sleep+0x279/frame 0xfffffe00aa836a40 softdep_flush() at softdep_flush+0x17f/frame 0xfffffe00aa836a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00aa836ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00aa836ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command vmdaemon pid 31 tid 100129 td 0xfffff80007893000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006f338c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006f338f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006f33930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f33960 _sleep() at _sleep+0x2da/frame 0xfffffe0006f33a00 vm_daemon() at vm_daemon+0xabf/frame 0xfffffe0006f33a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f33ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f33ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 30 tid 100128 td 0xfffff800078935a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006f2e560 mi_switch() at mi_switch+0x174/frame 0xfffffe0006f2e590 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006f2e5d0 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006f2e610 _sleep() at _sleep+0x279/frame 0xfffffe0006f2e6b0 vm_pageout_worker() at vm_pageout_worker+0x25e/frame 0xfffffe0006f2ea30 vm_pageout() at vm_pageout+0x156/frame 0xfffffe0006f2ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f2eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f2eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 30 tid 100134 td 0xfffff80007baa000 sched_switch() at sched_switch+0x713/frame 0xfffffe00a30e96b0 mi_switch() at mi_switch+0x174/frame 0xfffffe00a30e96e0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00a30e9720 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a30e9750 _sleep() at _sleep+0x2da/frame 0xfffffe00a30e97f0 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0xbcb/frame 0xfffffe00a30e9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a30e9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a30e9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 30 tid 100139 td 0xfffff80007ba75a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00a31f38f0 mi_switch() at mi_switch+0x174/frame 0xfffffe00a31f3920 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00a31f3960 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00a31f39a0 _sleep() at _sleep+0x279/frame 0xfffffe00a31f3a40 uma_reclaim_worker() at uma_reclaim_worker+0xda/frame 0xfffffe00a31f3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a31f3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a31f3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon1 pid 29 tid 100127 td 0xfffff8000779d000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006f298b0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006f298e0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006f29920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f29950 _sleep() at _sleep+0x2da/frame 0xfffffe0006f299f0 enc_daemon() at enc_daemon+0x1cf/frame 0xfffffe0006f29a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f29ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f29ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon0 pid 28 tid 100126 td 0xfffff8000779d5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006f248b0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006f248e0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006f24920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f24950 _sleep() at _sleep+0x2da/frame 0xfffffe0006f249f0 enc_daemon() at enc_daemon+0x1cf/frame 0xfffffe0006f24a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f24ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f24ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command rand_harvestq pid 27 tid 100123 td 0xfffff800078a2000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006f158c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006f158f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006f15930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006f15970 _sleep() at _sleep+0x279/frame 0xfffffe0006f15a10 random_kthread() at random_kthread+0x27a/frame 0xfffffe0006f15a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f15ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f15ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sctp_iterator pid 26 tid 100119 td 0xfffff800078a4000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006ef5910 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ef5940 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ef5980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006ef59b0 _sleep() at _sleep+0x2da/frame 0xfffffe0006ef5a50 sctp_iterator_thread() at sctp_iterator_thread+0x59/frame 0xfffffe0006ef5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ef5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ef5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod4 pid 25 tid 100118 td 0xfffff8000779e000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006ee48e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ee4910 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ee4950 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006ee4990 _sleep() at _sleep+0x279/frame 0xfffffe0006ee4a30 soaio_kproc_loop() at soaio_kproc_loop+0x170/frame 0xfffffe0006ee4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ee4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ee4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod3 pid 24 tid 100117 td 0xfffff8000779e5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006edf8e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006edf910 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006edf950 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006edf990 _sleep() at _sleep+0x279/frame 0xfffffe0006edfa30 soaio_kproc_loop() at soaio_kproc_loop+0x170/frame 0xfffffe0006edfa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006edfab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006edfab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod2 pid 23 tid 100116 td 0xfffff8000779f000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006eda8e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006eda910 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006eda950 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006eda990 _sleep() at _sleep+0x279/frame 0xfffffe0006edaa30 soaio_kproc_loop() at soaio_kproc_loop+0x170/frame 0xfffffe0006edaa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006edaab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006edaab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod1 pid 22 tid 100115 td 0xfffff8000779f5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006ed58e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ed5910 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ed5950 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006ed5990 _sleep() at _sleep+0x279/frame 0xfffffe0006ed5a30 soaio_kproc_loop() at soaio_kproc_loop+0x170/frame 0xfffffe0006ed5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ed5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ed5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100096 td 0xfffff80003ec7000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e5e930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e5e960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e5e9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e5e9d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e5ea30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e5ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e5eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e5eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100097 td 0xfffff80003ec65a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e63930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e63960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e639a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e639d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e63a30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e63a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e63ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e63ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100098 td 0xfffff80003ec6000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e68930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e68960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e689a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e689d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e68a30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e68a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e68ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e68ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100099 td 0xfffff80003ec55a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e6d930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e6d960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e6d9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e6d9d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e6da30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e6da70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e6dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e6dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100100 td 0xfffff80003ec5000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e72930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e72960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e729a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e729d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e72a30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e72a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e72ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e72ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100102 td 0xfffff80003eb9000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e7c930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e7c960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e7c9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e7c9d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e7ca30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e7ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e7cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e7cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100103 td 0xfffff8000605f5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e81930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e81960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e819a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e819d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e81a30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e81a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e81ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e81ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100104 td 0xfffff8000605f000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e86930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e86960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e869a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e869d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e86a30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e86a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e86ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e86ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100105 td 0xfffff8000605e5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e8b930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e8b960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e8b9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e8b9d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e8ba30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e8ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e8bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e8bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100106 td 0xfffff8000605e000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e90930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e90960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e909a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e909d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e90a30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e90a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e90ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e90ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100107 td 0xfffff8000616b000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006ead930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ead960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ead9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006ead9d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006eada30 usb_process() at usb_process+0x13b/frame 0xfffffe0006eada70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006eadab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006eadab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100108 td 0xfffff8000616a5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006eb2930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006eb2960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006eb29a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006eb29d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006eb2a30 usb_process() at usb_process+0x13b/frame 0xfffffe0006eb2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006eb2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006eb2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100109 td 0xfffff8000616a000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006eb7930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006eb7960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006eb79a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006eb79d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006eb7a30 usb_process() at usb_process+0x13b/frame 0xfffffe0006eb7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006eb7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006eb7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100110 td 0xfffff800061695a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006ebc930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ebc960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ebc9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006ebc9d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006ebca30 usb_process() at usb_process+0x13b/frame 0xfffffe0006ebca70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ebcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ebcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100111 td 0xfffff80006169000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006ec1930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ec1960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ec19a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006ec19d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006ec1a30 usb_process() at usb_process+0x13b/frame 0xfffffe0006ec1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ec1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ec1ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100091 td 0xfffff8000397d000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e458e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e45910 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e45950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e45980 _sleep() at _sleep+0x2da/frame 0xfffffe0006e45a20 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe0006e45a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e45ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e45ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100092 td 0xfffff80003aea5a0 (CPU 9) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bd3df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bd3e10 trap() at trap+0x48/frame 0xfffffe0006bd3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bd3f20 --- trap 0x13, rip = 0xffffffff80c2d400, rsp = 0xfffffe0006e4a890, rbp = 0xfffffe0006e4a8a0 --- lock_delay() at lock_delay+0x40/frame 0xfffffe0006e4a8a0 __mtx_lock_sleep() at __mtx_lock_sleep+0xe8/frame 0xfffffe0006e4a930 __mtx_lock_flags() at __mtx_lock_flags+0xee/frame 0xfffffe0006e4a980 adadone() at adadone+0x68/frame 0xfffffe0006e4a9e0 xpt_done_process() at xpt_done_process+0x388/frame 0xfffffe0006e4aa20 xpt_done_td() at xpt_done_td+0xf6/frame 0xfffffe0006e4aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e4aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e4aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100093 td 0xfffff80003aea000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e4f8e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e4f910 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e4f950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e4f980 _sleep() at _sleep+0x2da/frame 0xfffffe0006e4fa20 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe0006e4fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e4fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e4fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100125 td 0xfffff800078a1000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006f1f8d0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006f1f900 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006f1f940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f1f970 _sleep() at _sleep+0x2da/frame 0xfffffe0006f1fa10 xpt_scanner_thread() at xpt_scanner_thread+0x93/frame 0xfffffe0006f1fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f1fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f1fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 11 pid 19 tid 100090 td 0xfffff8000397d5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e408c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e408f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e40930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e40960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e40a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e40a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e40ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e40ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 10 pid 18 tid 100089 td 0xfffff8000397e000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e3b8c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e3b8f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e3b930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e3b960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e3ba00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e3ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e3bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e3bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 9 pid 17 tid 100088 td 0xfffff8000397e5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e368c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e368f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e36930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e36960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e36a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e36a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e36ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e36ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 8 pid 16 tid 100087 td 0xfffff8000397f000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e318c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e318f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e31930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e31960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e31a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e31a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e31ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e31ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 7 pid 15 tid 100086 td 0xfffff8000397f5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e2c8c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e2c8f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e2c930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e2c960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e2ca00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e2ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e2cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e2cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 6 pid 9 tid 100085 td 0xfffff80003980000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e278c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e278f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e27930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e27960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e27a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e27a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e27ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e27ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 5 pid 8 tid 100084 td 0xfffff800039805a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e228c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e228f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e22930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e22960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e22a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e22a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e22ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e22ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 4 pid 7 tid 100083 td 0xfffff80003981000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e1d8c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e1d8f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e1d930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e1d960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e1da00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e1da70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e1dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e1dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 3 pid 6 tid 100082 td 0xfffff8000395e5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e188c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e188f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e18930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e18960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e18a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e18a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e18ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e18ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 2 pid 5 tid 100081 td 0xfffff8000396e000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e138c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e138f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e13930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e13960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e13a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e13a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e13ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e13ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 1 pid 4 tid 100080 td 0xfffff8000396e5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e0e8c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e0e8f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e0e930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e0e960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e0ea00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e0ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e0eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e0eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 0 pid 3 tid 100079 td 0xfffff8000396f000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e098c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e098f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e09930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e09960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e09a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e09a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e09ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e09ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 2 tid 100078 td 0xfffff8000396f5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e048f0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e04920 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e04960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e04990 _sleep() at _sleep+0x2da/frame 0xfffffe0006e04a30 crypto_proc() at crypto_proc+0x2c4/frame 0xfffffe0006e04a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e04ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e04ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sequencer 00 pid 14 tid 100065 td 0xfffff8000395e000 sched_switch() at sched_switch+0x713/frame 0xfffffe00005c08f0 mi_switch() at mi_switch+0x174/frame 0xfffffe00005c0920 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005c0960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005c0990 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe00005c09f0 seq_eventthread() at seq_eventthread+0xdc/frame 0xfffffe00005c0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005c0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005c0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100062 td 0xfffff800038155a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00005a58f0 mi_switch() at mi_switch+0x174/frame 0xfffffe00005a5920 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005a5960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005a5990 _sleep() at _sleep+0x2da/frame 0xfffffe00005a5a30 g_run_events() at g_run_events+0x4f/frame 0xfffffe00005a5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005a5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005a5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100063 td 0xfffff80003815000 sched_switch() at sched_switch+0x713/frame 0xfffffe00005aa8f0 mi_switch() at mi_switch+0x174/frame 0xfffffe00005aa920 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005aa960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005aa990 _sleep() at _sleep+0x2da/frame 0xfffffe00005aaa30 g_io_schedule_up() at g_io_schedule_up+0xa5/frame 0xfffffe00005aaa60 g_up_procbody() at g_up_procbody+0x6d/frame 0xfffffe00005aaa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005aaab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005aaab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100064 td 0xfffff800038065a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00005af8f0 mi_switch() at mi_switch+0x174/frame 0xfffffe00005af920 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005af960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005af990 _sleep() at _sleep+0x2da/frame 0xfffffe00005afa30 g_io_schedule_down() at g_io_schedule_down+0x14c/frame 0xfffffe00005afa60 g_down_procbody() at g_down_procbody+0x6d/frame 0xfffffe00005afa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005afab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005afab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100018 td 0xfffff800036b95a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00767bd9e0 mi_switch() at mi_switch+0x174/frame 0xfffffe00767bda10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe00767bda70 fork_exit() at fork_exit+0x84/frame 0xfffffe00767bdab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00767bdab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100019 td 0xfffff800036ff000 sched_switch() at sched_switch+0x713/frame 0xfffffe00767c29e0 mi_switch() at mi_switch+0x174/frame 0xfffffe00767c2a10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe00767c2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00767c2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00767c2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100021 td 0xfffff800036fe000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100047 td 0xfffff80003806000 sched_switch() at sched_switch+0x713/frame 0xfffffe00004ee9e0 mi_switch() at mi_switch+0x174/frame 0xfffffe00004eea10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe00004eea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004eeab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004eeab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100048 td 0xfffff800038055a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100049 td 0xfffff80003805000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100050 td 0xfffff800038045a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100051 td 0xfffff80003804000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100052 td 0xfffff800038035a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100053 td 0xfffff80003803000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100054 td 0xfffff800037f55a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100055 td 0xfffff800037f5000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100056 td 0xfffff800038185a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100057 td 0xfffff80003818000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100058 td 0xfffff800038175a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100059 td 0xfffff80003817000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100060 td 0xfffff800038165a0 sched_switch() at sched_switch+0x713/frame 0xfffffe000052f9e0 mi_switch() at mi_switch+0x174/frame 0xfffffe000052fa10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe000052fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe000052fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000052fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100094 td 0xfffff80003ec8000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e549e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e54a10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe0006e54a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e54ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e54ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100095 td 0xfffff80003ec75a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e599e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e59a10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe0006e59a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e59ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e59ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100101 td 0xfffff80003eb95a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006e779e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e77a10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe0006e77a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e77ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e77ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100112 td 0xfffff800061685a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100113 td 0xfffff80006168000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006ecb9e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ecba10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe0006ecba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ecbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ecbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100181 td 0xfffff80003ec85a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100182 td 0xfffff80016c99000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100183 td 0xfffff80016c985a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100184 td 0xfffff80016c98000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100185 td 0xfffff80016c975a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100186 td 0xfffff80016c97000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100187 td 0xfffff80016c965a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100188 td 0xfffff80016c96000 fork_trampoline() at fork_trampoline Tracing command idle pid 11 tid 100003 td 0xfffff800036b8000 (CPU 0) cpustop_handler() at cpustop_handler+0x2d/frame 0xffffffff820608b0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xffffffff820608d0 trap() at trap+0x48/frame 0xffffffff820609e0 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff820609e0 --- trap 0x13, rip = 0xffffffff8122aef6, rsp = 0xfffffe0076766900, rbp = 0xfffffe0076766900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076766900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076766950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076766970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe0076766990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe0076766a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076766ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076766ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xfffff800036b75a0 (CPU 1) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006b73df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006b73e10 trap() at trap+0x48/frame 0xfffffe0006b73f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b73f20 --- trap 0x13, rip = 0xffffffff8122aef6, rsp = 0xfffffe007676b900, rbp = 0xfffffe007676b900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007676b900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe007676b950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007676b970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe007676b990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe007676ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe007676bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007676bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffff800036b7000 (CPU 2) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006b7fdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006b7fe10 trap() at trap+0x48/frame 0xfffffe0006b7ff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b7ff20 --- trap 0x13, rip = 0xffffffff8122aef6, rsp = 0xfffffe0076770900, rbp = 0xfffffe0076770900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076770900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076770950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076770970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe0076770990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe0076770a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076770ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076770ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xfffff800036b65a0 (CPU 3) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006b8bdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006b8be10 trap() at trap+0x48/frame 0xfffffe0006b8bf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b8bf20 --- trap 0x13, rip = 0xffffffff8122aef6, rsp = 0xfffffe0076775900, rbp = 0xfffffe0076775900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076775900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076775950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076775970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe0076775990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe0076775a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076775ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076775ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100007 td 0xfffff800036b6000 (CPU 4) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006b97df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006b97e10 trap() at trap+0x48/frame 0xfffffe0006b97f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b97f20 --- trap 0x13, rip = 0xffffffff8122aef6, rsp = 0xfffffe007677a900, rbp = 0xfffffe007677a900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007677a900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe007677a950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007677a970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe007677a990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe007677aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007677aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007677aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xfffff800036b05a0 (CPU 5) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006ba3df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006ba3e10 trap() at trap+0x48/frame 0xfffffe0006ba3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006ba3f20 --- trap 0x13, rip = 0xffffffff8122aef6, rsp = 0xfffffe007677f900, rbp = 0xfffffe007677f900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007677f900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe007677f950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007677f970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe007677f990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe007677fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007677fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007677fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xfffff800036b0000 (CPU 6) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bafdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bafe10 trap() at trap+0x48/frame 0xfffffe0006baff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006baff20 --- trap 0x13, rip = 0xffffffff8122aef6, rsp = 0xfffffe0076784900, rbp = 0xfffffe0076784900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076784900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076784950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076784970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe0076784990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe0076784a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076784ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076784ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xfffff800036c95a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0076789940 mi_switch() at mi_switch+0x174/frame 0xfffffe0076789970 critical_exit_preempt() at critical_exit_preempt+0x65/frame 0xfffffe0076789990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe0076789a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076789ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076789ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffff800036c9000 (CPU 8) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bc7df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bc7e10 trap() at trap+0x48/frame 0xfffffe0006bc7f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bc7f20 --- trap 0x13, rip = 0xffffffff8122aef6, rsp = 0xfffffe007678e900, rbp = 0xfffffe007678e900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007678e900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe007678e950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007678e970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe007678e990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe007678ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe007678eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007678eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100012 td 0xfffff800036c85a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0076793940 mi_switch() at mi_switch+0x174/frame 0xfffffe0076793970 critical_exit_preempt() at critical_exit_preempt+0x65/frame 0xfffffe0076793990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe0076793a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076793ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076793ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xfffff800036c8000 (CPU 10) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bdfdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bdfe10 trap() at trap+0x48/frame 0xfffffe0006bdff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bdff20 --- trap 0x13, rip = 0xffffffff8122aef6, rsp = 0xfffffe0076798900, rbp = 0xfffffe0076798900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076798900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076798950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076798970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe0076798990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe0076798a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076798ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076798ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100014 td 0xfffff800036c75a0 (CPU 11) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bebdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bebe10 trap() at trap+0x48/frame 0xfffffe0006bebf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bebf20 --- trap 0x13, rip = 0xffffffff8122aef6, rsp = 0xfffffe007679d900, rbp = 0xfffffe007679d900 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007679d900 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe007679d950 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007679d970 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe007679d990 sched_idletd() at sched_idletd+0x472/frame 0xfffffe007679da70 fork_exit() at fork_exit+0x84/frame 0xfffffe007679dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007679dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command init pid 1 tid 100002 td 0xfffff800036b85a0 (CPU 7) kdb_enter() at kdb_enter+0x3b/frame 0xfffffe0076761380 vpanic() at vpanic+0x1ba/frame 0xfffffe00767613d0 panic() at panic+0x43/frame 0xfffffe0076761430 mtrash_ctor() at mtrash_ctor+0x7e/frame 0xfffffe0076761450 uma_zalloc_arg() at uma_zalloc_arg+0x9ff/frame 0xfffffe00767614e0 malloc() at malloc+0x9b/frame 0xfffffe0076761530 xpt_run_allocq() at xpt_run_allocq+0xcb/frame 0xfffffe0076761580 adastrategy() at adastrategy+0x6e/frame 0xfffffe00767615b0 g_disk_start() at g_disk_start+0x351/frame 0xfffffe0076761610 g_io_request() at g_io_request+0x3f5/frame 0xfffffe0076761660 g_part_start() at g_part_start+0x120/frame 0xfffffe00767616e0 g_io_request() at g_io_request+0x3f5/frame 0xfffffe0076761730 g_io_request() at g_io_request+0x3f5/frame 0xfffffe0076761780 bufwrite() at bufwrite+0x24d/frame 0xfffffe00767617d0 vfs_bio_awrite() at vfs_bio_awrite+0x212/frame 0xfffffe0076761830 vn_fsync_buf() at vn_fsync_buf+0x21f/frame 0xfffffe0076761890 VOP_FSYNC_APV() at VOP_FSYNC_APV+0x88/frame 0xfffffe00767618b0 ffs_sync() at ffs_sync+0x2f9/frame 0xfffffe0076761960 sys_sync() at sys_sync+0xbe/frame 0xfffffe0076761990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe0076761ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0076761ab0 --- syscall (36, FreeBSD ELF64, sys_sync), rip = 0x24dbca, rsp = 0x7fffffffe6f8, rbp = 0x7fffffffe7e0 --- Tracing command audit pid 10 tid 100001 td 0xfffff800036b9000 sched_switch() at sched_switch+0x713/frame 0xfffffe007675c910 mi_switch() at mi_switch+0x174/frame 0xfffffe007675c940 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe007675c980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe007675c9b0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe007675ca10 audit_worker() at audit_worker+0x93/frame 0xfffffe007675ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe007675cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007675cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xffffffff820953a0 sched_switch() at sched_switch+0x713/frame 0xffffffff82900980 mi_switch() at mi_switch+0x174/frame 0xffffffff829009b0 sleepq_switch() at sleepq_switch+0x110/frame 0xffffffff829009f0 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xffffffff82900a30 _sleep() at _sleep+0x279/frame 0xffffffff82900ad0 swapper() at swapper+0x77/frame 0xffffffff82900b20 mi_startup() at mi_startup+0x2ee/frame 0xffffffff82900b70 btext() at btext+0x2c Tracing command kernel pid 0 tid 100015 td 0xfffff800036c7000 sched_switch() at sched_switch+0x713/frame 0xfffffe00767ae930 mi_switch() at mi_switch+0x174/frame 0xfffffe00767ae960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00767ae9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00767ae9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00767aea40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00767aea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00767aeab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00767aeab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xfffff800036c65a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00767b3900 mi_switch() at mi_switch+0x174/frame 0xfffffe00767b3930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00767b3970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00767b39a0 _sleep() at _sleep+0x2da/frame 0xfffffe00767b3a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00767b3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00767b3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00767b3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100017 td 0xfffff800036c6000 sched_switch() at sched_switch+0x713/frame 0xfffffe00767b8900 mi_switch() at mi_switch+0x174/frame 0xfffffe00767b8930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00767b8970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00767b89a0 _sleep() at _sleep+0x2da/frame 0xfffffe00767b8a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00767b8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00767b8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00767b8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xfffff800036fe5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00767c7900 mi_switch() at mi_switch+0x174/frame 0xfffffe00767c7930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00767c7970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00767c79a0 _sleep() at _sleep+0x2da/frame 0xfffffe00767c7a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00767c7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00767c7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00767c7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xfffff800036fd5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0000471930 mi_switch() at mi_switch+0x174/frame 0xfffffe0000471960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004719a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004719d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0000471a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000471a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000471ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000471ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xfffff800036fd000 sched_switch() at sched_switch+0x713/frame 0xfffffe0000476930 mi_switch() at mi_switch+0x174/frame 0xfffffe0000476960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004769a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004769d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0000476a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000476a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000476ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000476ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xfffff800036fc5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe000047b930 mi_switch() at mi_switch+0x174/frame 0xfffffe000047b960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe000047b9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000047b9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe000047ba40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000047ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe000047bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000047bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xfffff800036fc000 sched_switch() at sched_switch+0x713/frame 0xfffffe0000480930 mi_switch() at mi_switch+0x174/frame 0xfffffe0000480960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004809a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004809d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0000480a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000480a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000480ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000480ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100026 td 0xfffff800036fb5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0000485930 mi_switch() at mi_switch+0x174/frame 0xfffffe0000485960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004859a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004859d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0000485a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000485a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000485ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000485ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100027 td 0xfffff800036fb000 sched_switch() at sched_switch+0x713/frame 0xfffffe000048a930 mi_switch() at mi_switch+0x174/frame 0xfffffe000048a960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe000048a9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000048a9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe000048aa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000048aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe000048aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000048aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100028 td 0xfffff800037355a0 sched_switch() at sched_switch+0x713/frame 0xfffffe000048f930 mi_switch() at mi_switch+0x174/frame 0xfffffe000048f960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe000048f9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000048f9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe000048fa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000048fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe000048fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000048fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100029 td 0xfffff80003735000 sched_switch() at sched_switch+0x713/frame 0xfffffe0000494930 mi_switch() at mi_switch+0x174/frame 0xfffffe0000494960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004949a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004949d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0000494a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000494a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000494ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000494ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100030 td 0xfffff800037345a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0000499930 mi_switch() at mi_switch+0x174/frame 0xfffffe0000499960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004999a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004999d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0000499a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000499a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000499ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000499ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100031 td 0xfffff80003734000 sched_switch() at sched_switch+0x713/frame 0xfffffe000049e930 mi_switch() at mi_switch+0x174/frame 0xfffffe000049e960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe000049e9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000049e9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe000049ea40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000049ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe000049eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000049eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100032 td 0xfffff800037335a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00004a3930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004a3960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004a39a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004a39d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004a3a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004a3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004a3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004a3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100033 td 0xfffff80003733000 sched_switch() at sched_switch+0x713/frame 0xfffffe00004a8930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004a8960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004a89a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004a89d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004a8a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004a8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004a8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004a8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100034 td 0xfffff800037325a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00004ad930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004ad960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004ad9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004ad9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004ada40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004ada70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004adab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004adab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100035 td 0xfffff800037cc000 sched_switch() at sched_switch+0x713/frame 0xfffffe00004b2930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004b2960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004b29a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004b29d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004b2a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004b2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004b2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004b2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100036 td 0xfffff800037cb5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00004b7930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004b7960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004b79a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004b79d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004b7a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004b7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004b7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004b7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100037 td 0xfffff800037cb000 sched_switch() at sched_switch+0x713/frame 0xfffffe00004bc930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004bc960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004bc9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004bc9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004bca40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004bca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004bcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004bcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100038 td 0xfffff800037ca5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00004c1930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004c1960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004c19a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004c19d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004c1a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004c1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004c1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004c1ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100039 td 0xfffff800037ca000 sched_switch() at sched_switch+0x713/frame 0xfffffe00004c6930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004c6960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004c69a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004c69d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004c6a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004c6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004c6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004c6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100040 td 0xfffff800037c95a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00004cb930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004cb960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004cb9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004cb9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004cba40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004cba70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004cbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004cbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100041 td 0xfffff800037c9000 sched_switch() at sched_switch+0x713/frame 0xfffffe00004d0930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004d0960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004d09a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004d09d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004d0a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004d0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004d0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004d0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100042 td 0xfffff800037bd5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00004d5930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004d5960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004d59a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004d59d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004d5a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004d5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004d5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004d5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100043 td 0xfffff800037bd000 sched_switch() at sched_switch+0x713/frame 0xfffffe00004da930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004da960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004da9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004da9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004daa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004daa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004daab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004daab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100044 td 0xfffff800037da5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00004df930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004df960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004df9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004df9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004dfa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004dfa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004dfab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004dfab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100045 td 0xfffff800037da000 sched_switch() at sched_switch+0x713/frame 0xfffffe00004e4930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004e4960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004e49a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004e49d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004e4a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004e4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004e4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004e4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100046 td 0xfffff800037d95a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00004e9930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004e9960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004e99a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004e99d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004e9a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004e9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004e9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004e9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100061 td 0xfffff80003816000 sched_switch() at sched_switch+0x713/frame 0xfffffe0000594900 mi_switch() at mi_switch+0x174/frame 0xfffffe0000594930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0000594970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005949a0 _sleep() at _sleep+0x2da/frame 0xfffffe0000594a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0000594a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000594ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000594ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100066 td 0xfffff8000395d5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00005c5900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005c5930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005c5970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005c59a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005c5a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005c5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005c5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005c5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100067 td 0xfffff8000395d000 sched_switch() at sched_switch+0x713/frame 0xfffffe00005ca900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005ca930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005ca970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005ca9a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005caa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005caa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005caab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005caab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100068 td 0xfffff8000395c5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00005cf900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005cf930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005cf970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005cf9a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005cfa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005cfa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005cfab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005cfab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100069 td 0xfffff8000395c000 sched_switch() at sched_switch+0x713/frame 0xfffffe00005d4900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005d4930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005d4970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005d49a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005d4a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005d4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005d4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005d4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100070 td 0xfffff8000395b5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00005d9900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005d9930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005d9970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005d99a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005d9a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005d9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005d9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005d9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100071 td 0xfffff8000395b000 sched_switch() at sched_switch+0x713/frame 0xfffffe00005de900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005de930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005de970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005de9a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005dea40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005dea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005deab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005deab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100072 td 0xfffff8000395a5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00005e3900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005e3930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005e3970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005e39a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005e3a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005e3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005e3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005e3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100073 td 0xfffff8000395a000 sched_switch() at sched_switch+0x713/frame 0xfffffe00005e8900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005e8930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005e8970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005e89a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005e8a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005e8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005e8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005e8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100074 td 0xfffff800039715a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00005ed900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005ed930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005ed970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005ed9a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005eda40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005eda70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005edab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005edab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100075 td 0xfffff80003971000 sched_switch() at sched_switch+0x713/frame 0xfffffe00005f2900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005f2930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005f2970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005f29a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005f2a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005f2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005f2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005f2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100076 td 0xfffff800039705a0 sched_switch() at sched_switch+0x713/frame 0xfffffe00005f7900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005f7930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005f7970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005f79a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005f7a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005f7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005f7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005f7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100077 td 0xfffff80003970000 sched_switch() at sched_switch+0x713/frame 0xfffffe00005fc900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005fc930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005fc970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005fc9a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005fca40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005fca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005fcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005fcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100114 td 0xfffff8000615c5a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006ed0930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ed0960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ed09a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006ed09d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0006ed0a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0006ed0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ed0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ed0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100120 td 0xfffff800078a35a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006f06930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006f06960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006f069a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f069d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0006f06a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0006f06a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f06ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f06ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100121 td 0xfffff800078a3000 sched_switch() at sched_switch+0x713/frame 0xfffffe0006f0b930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006f0b960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006f0b9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f0b9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0006f0ba40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0006f0ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f0bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f0bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100122 td 0xfffff800078a25a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006f10930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006f10960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006f109a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f109d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0006f10a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0006f10a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f10ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f10ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100124 td 0xfffff800078a15a0 sched_switch() at sched_switch+0x713/frame 0xfffffe0006f1a900 mi_switch() at mi_switch+0x174/frame 0xfffffe0006f1a930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006f1a970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f1a9a0 _sleep() at _sleep+0x2da/frame 0xfffffe0006f1aa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006f1aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f1aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f1aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db:0:pho> dump Dumping 2600 out of 32621 MB:..1%..11%..21%..31% (kgdb) bt #0 doadump (textdump=0x0) at include/pcpu.h:241 #1 0xffffffff8049c87b in db_dump (dummy=, dummy3=, dummy4=) at ../../../ddb/db_command.c:574 #2 0xffffffff8049c649 in db_command (cmd_table=, dopager=0x0) at ../../../ddb/db_command.c:481 #3 0xffffffff804a15b8 in db_script_exec (scriptname=, warnifnotfound=) at ../../../ddb/db_script.c:304 #4 0xffffffff8049c649 in db_command (cmd_table=, dopager=0x1) at ../../../ddb/db_command.c:481 #5 0xffffffff8049c3c4 in db_command_loop () at ../../../ddb/db_command.c:534 #6 0xffffffff8049f56f in db_trap (type=, code=) at ../../../ddb/db_main.c:252 #7 0xffffffff80c2c6ac in kdb_trap (type=0x3, code=0x0, tf=) at ../../../kern/subr_kdb.c:692 #8 0xffffffff810c3281 in trap (frame=0xfffffe00767612b0) at ../../../amd64/amd64/trap.c:619 #9 0xffffffff810956d5 in calltrap () at ../../../amd64/amd64/exception.S:232 #10 0xffffffff80c2bdbb in kdb_enter (why=0xffffffff81355615 "panic", msg=) at include/cpufunc.h:65 #11 0xffffffff80be38ea in vpanic (fmt=, ap=) at ../../../kern/kern_shutdown.c:885 #12 0xffffffff80be3663 in panic (fmt=) at ../../../kern/kern_shutdown.c:823 #13 0xffffffff80f1ccce in mtrash_ctor (mem=, size=) at ../../../vm/uma_dbg.c:162 #14 0xffffffff80f1733f in uma_zalloc_arg (zone=0xfffff8000304e000, udata=0x0, flags=0x1) at ../../../vm/uma_core.c:2417 #15 0xffffffff80bbd18b in malloc (size=0x4e0, mtp=0xffffffff81a4a4f0, flags=0x1) at uma.h:347 #16 0xffffffff80377aab in xpt_run_allocq (periph=0xfffff800079de200, sleep=0x0) at ../../../cam/cam_xpt.c:4729 #17 0xffffffff803a0c0e in adastrategy (bp=0xfffff807ace9d7f8) at ../../../cam/ata/ata_da.c:1018 #18 0xffffffff80b25cd1 in g_disk_start (bp=) at ../../../geom/geom_disk.c:478 #19 0xffffffff80b29255 in g_io_request (bp=0xfffff80777d484c8, cp=0xfffff800037eea00) at ../../../geom/geom_io.c:635 #20 0xffffffff80b37640 in g_part_start (bp=0xfffff80817a08b28) at bio.h:163 #21 0xffffffff80b29255 in g_io_request (bp=0xfffff80817a08b28, cp=0xfffff800060a5e80) at ../../../geom/geom_io.c:635 #22 0xffffffff80b29255 in g_io_request (bp=0xfffff801250527f8, cp=0xfffff80003135880) at ../../../geom/geom_io.c:635 #23 0xffffffff80c9363d in bufwrite (bp=0xfffffe0002592068) at buf.h:425 #24 0xffffffff80c96ab2 in vfs_bio_awrite (bp=0xfffffe0002592068) at buf.h:413 #25 0xffffffff80ccb55f in vn_fsync_buf (vp=, waitfor=0x2) at ../../../kern/vfs_vnops.c:2549 #26 0xffffffff81246be8 in VOP_FSYNC_APV (vop=0xffffffff81af8478, a=0xfffffe0076761910) at vnode_if.c:1312 #27 0xffffffff80ef9ba9 in ffs_sync (mp=0xfffff80007b6a000, waitfor=0x2) at vnode_if.h:549 #28 0xffffffff80cbe92e in sys_sync (td=, uap=) at ../../../kern/vfs_syscalls.c:142 #29 0xffffffff810c41f1 in amd64_syscall (td=0xfffff800036b85a0, traced=0x0) at subr_syscall.c:135 #30 0xffffffff81095fbd in fast_syscall_common () at ../../../amd64/amd64/exception.S:504 #31 0x000000000024dbca in ?? () Previous frame inner to this frame (corrupt stack?) Current language: auto; currently minimal (kgdb) f 14 #14 0xffffffff80f1733f in uma_zalloc_arg (zone=0xfffff8000304e000, udata=0x0, flags=0x1) at ../../../vm/uma_core.c:2417 2417 zone->uz_ctor(item, zone->uz_size, udata, flags) != 0) { (kgdb) l 2412 if (zone->uz_ctor != NULL && 2413 #ifdef INVARIANTS 2414 (!skipdbg || zone->uz_ctor != trash_ctor || 2415 zone->uz_dtor != trash_dtor) && 2416 #endif 2417 zone->uz_ctor(item, zone->uz_size, udata, flags) != 0) { 2418 counter_u64_add(zone->uz_fails, 1); 2419 zone_free_item(zone, item, udata, SKIP_DTOR | SKIP_CNT); 2420 return (NULL); 2421 } (kgdb) info loc zdom = 0x8 bucket = cache = lockfail = domain = maxbucket = item = (void *) 0xfffff80379858800 (kgdb) p *zone $1 = {{uz_keg = 0xfffff80003046d20, uz_lockptr = 0xfffff80003046d20}, uz_domain = 0xfffff8000304e780, uz_flags = 0x2018, uz_size = 0x800, uz_ctor = 0xffffffff80f1cc50 , uz_dtor = 0xffffffff80f1ccd0 , uz_items = 0x0, uz_max_items = 0x0, uz_sleepers = 0x0, uz_count = 0x5, uz_count_max = 0xfd, uz_import = 0xffffffff80f1bc50 , uz_release = 0xffffffff80f1c220 , uz_arg = 0xfffff8000304e000, uz_init = 0, uz_fini = 0, uz_spare = 0x0, uz_bkt_count = 0x32, uz_bkt_max = 0xffffffffffffffff, uz_lock = {lock_object = {lo_name = 0x0, lo_flags = 0x0, lo_data = 0x0, lo_witness = 0x0}, mtx_lock = 0x0}, uz_link = {le_next = 0x0, le_prev = 0xfffff80003046d50}, uz_name = 0xffffffff8133ab38 "2048", uz_warning = 0x0, uz_ratecheck = {tv_sec = 0x0, tv_usec = 0x0}, uz_maxaction = {ta_link = {stqe_next = 0x0}, ta_pending = 0x0, ta_priority = 0x0, ta_func = 0, ta_context = 0x0}, uz_count_min = 0x5, uz_allocs = 0xfffffe0000401150, uz_frees = 0xfffffe0000401140, uz_fails = 0xfffffe0000401130, uz_sleeps = 0x0, uz_cpu = 0xfffff8000304e180} (kgdb) -- Test scenario: misc/su.sh + misc/callout_reset_on.sh