panic: vnode_pager_generic_getpages: sector size 8192 too large, vmcore.16 KDB: debugger backends: ddb KDB: current backend: ddb ---<>--- Copyright (c) 1992-2019 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 13.0-CURRENT #1 r344045: Wed Feb 13 08:42:31 CET 2019 pho@mercat1.netperf.freebsd.org:/usr/src/sys/amd64/compile/PHO amd64 FreeBSD clang version 7.0.1 (tags/RELEASE_701/final 349250) (based on LLVM 7.0.1) WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. Entering uma_startup with 10 boot pages configured startup_alloc from "UMA Kegs", 8 boot pages left startup_alloc from "UMA Zones", 7 boot pages left startup_alloc from "UMA Zones", 6 boot pages left startup_alloc from "UMA Zones", 5 boot pages left startup_alloc from "UMA Zones", 4 boot pages left startup_alloc from "UMA Zones", 3 boot pages left startup_alloc from "UMA Hash", 2 boot pages left startup_alloc from "UMA Zones", 1 boot pages left Entering uma_startup1 with 0 boot pages left Entering uma_startup2 with 0 boot pages left VT(vga): resolution 640x480 CPU: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz (3500.07-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306f2 Family=0x6 Model=0x3f Stepping=2 Features=0xbfebfbff Features2=0x7ffefbff AMD Features=0x2c100800 AMD Features2=0x21 Structured Extended Features=0x37ab XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics real memory = 34359738368 (32768 MB) avail memory = 33182662656 (31645 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs FreeBSD/SMP: 1 package(s) x 6 core(s) x 2 hardware threads random: unblocking device. ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard Launching APs: 1 9 2 8 10 7 6 4 11 3 5 Timecounter "TSC-low" frequency 1750033032 Hz quality 1000 random: entropy device external interface [ath_hal] loaded kbd1 at kbdmux0 module_register_init: MOD_LOAD (vesa, 0xffffffff81140690, 0) error 19 random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" 000.000061 [4212] netmap_init netmap: loaded module nexus0 vtvga0: on motherboard cryptosoft0: on motherboard acpi0: on motherboard acpi0: Power Button (fixed) cpu0: on acpi0 atrtc0: port 0x70-0x71,0x74-0x77 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Event timer "HPET4" frequency 14318180 Hz quality 340 Event timer "HPET5" frequency 14318180 Hz quality 340 Event timer "HPET6" frequency 14318180 Hz quality 340 Event timer "HPET7" frequency 14318180 Hz quality 340 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: on acpi0 pci0: on pcib0 pci0: at device 11.1 (no driver attached) pci0: at device 11.2 (no driver attached) pci0: at device 16.1 (no driver attached) pci0: at device 16.6 (no driver attached) pci0: at device 18.1 (no driver attached) acpi_syscontainer0: on acpi0 acpi_syscontainer1: on acpi0 acpi_syscontainer2: on acpi0 acpi_syscontainer3: on acpi0 pcib1: port 0xcf8-0xcff on acpi0 pci1: on pcib1 pcib2: irq 26 at device 1.0 on pci1 pci2: on pcib2 pcib3: irq 32 at device 2.0 on pci1 pci3: on pcib3 pci3: at device 0.0 (no driver attached) pci3: at device 0.1 (no driver attached) pcib4: irq 40 at device 3.0 on pci1 pci4: on pcib4 pci1: at device 17.0 (no driver attached) ahci0: port 0xf110-0xf117,0xf100-0xf103,0xf0f0-0xf0f7,0xf0e0-0xf0e3,0xf020-0xf03f mem 0xfb438000-0xfb4387ff irq 16 at device 17.4 on pci1 ahci0: AHCI v1.30 with 4 6Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahciem0: on ahci0 xhci0: mem 0xfb400000-0xfb40ffff irq 19 at device 20.0 on pci1 xhci0: 32 bytes context size, 64-bit DMA usbus0 on xhci0 usbus0: 5.0Gbps Super Speed USB v3.0 pci1: at device 22.0 (no driver attached) pci1: at device 22.1 (no driver attached) ehci0: mem 0xfb434000-0xfb4343ff irq 18 at device 26.0 on pci1 usbus1: EHCI version 1.0 usbus1 on ehci0 usbus1: 480Mbps High Speed USB v2.0 pcib5: irq 16 at device 28.0 on pci1 pci5: on pcib5 pcib6: irq 18 at device 28.2 on pci1 pci6: on pcib6 pcib7: at device 0.0 on pci6 pci7: on pcib7 vgapci0: port 0xe000-0xe07f mem 0xfa000000-0xfaffffff,0xfb000000-0xfb01ffff irq 18 at device 0.0 on pci7 vgapci0: Boot video device pcib8: irq 16 at device 28.4 on pci1 pci8: on pcib8 igb0: port 0xd020-0xd03f mem 0xfb320000-0xfb33ffff,0xfb344000-0xfb347fff irq 16 at device 0.0 on pci8 igb0: Using 1024 tx descriptors and 1024 rx descriptors igb0: queue equality override not set, capping rx_queues at 6 and tx_queues at 6 igb0: Using 6 rx queues 6 tx queues igb0: Using MSI-X interrupts with 7 vectors igb0:Autoloading module: ioat.ko ioat0: mem 0xfb42c000-0xfb42ffff irq 31 at device 4.0 on pci1 ioat0: Capabilities: 2f7 ioat1: mem 0xfb428000-0xfb42bfff irq 39 at device 4.1 on pci1 ioat1: Capabilities: 2f7 ioat2: mem 0xfb424000-0xfb427fff irq 31 at device 4.2 on pci1 ioat2: Capabilities: f7 ioat3: mem 0xfb420000-0xfb423fff irq 39 at device 4.3 on pci1 ioat3: Capabilities: f7 ioat4: mem 0xfb41c000-0xfb41ffff irq 31 at device 4.4 on pci1 ioat4: Capabilities: f7 ioat5: mem 0xfb418000-0xfb41bfff irq 39 at device 4.5 on pci1 ioat5: Capabilities: f7 ioat6: mem 0xfb414000-0xfb417fff irq 31 at device 4.6 on pci1 ioat6: Capabilities: f7 ioat7: mem 0xfb410000-0xfb413fff irq 39 at device 4.7 on pci1 ioat7: Capabilities: f7 Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Starting Network: igb1. igb1: flags=8802 metric 0 mtu 1500 options=e507bb ether 0c:c4:7a:a8:cd:db media: Ethernet autoselect status: no carrier nd6 options=29 Autoloading module: uhid.ko Autoloading module: ums.ko ums0 on uhub3 ums0: on usbus0 ums0: 3 buttons and [Z] coordinates ID=0 Starting dhclient. DHCPREQUEST on igb0 to 255.255.255.255 port 67 DHCPACK from 192if_delmulti_locked: detaching ifnet instance 0xfffff80006088000 .168.5.1 bound to 192.168.5.171 -- renewal in 300 seconds. add host 127.0.0.1: gateway lo0 fib 0: route already in table add host ::1: gateway lo0 fib 0: route already in table add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Creating and/or trimming log files. Starting syslogd. Setting date via ntp. 13 Feb 08:49:55 ntpdate[2037]: step time server 192.168.5.1 offset -1.329016 sec Starting rpcbind. NFS access cache time=60 Starting watchdogd. No core dumps found. lock order reversal: 1st 0xfffffe00008b4728 bufwait (bufwait) @ kern/vfs_bio.c:3906 2nd 0xfffff8001039a800 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:289 stack backtrace: #0 0xffffffff80c4bf83 at witness_debugger+0x73 #1 0xffffffff80c4bcd3 at witness_checkorder+0xac3 #2 0xffffffff80bec808 at _sx_xlock+0x68 #3 0xffffffff80efe7cd at ufsdirhash_add+0x4d #4 0xffffffff80f030e6 at ufs_direnter+0x446 #5 0xffffffff80f0b369 at ufs_mkdir+0x8c9 #6 0xffffffff81242345 at VOP_MKDIR_APV+0x1f5 #7 0xffffffff80cc4fa3 at kern_mkdirat+0x1b3 #8 0xffffffff810baeb1 at amd64_syscall+0x291 #9 0xffffffff81094b1d at fast_syscall_common+0x101 Clearing /tmp (X related). Starting statd. Starting lockd. fscklock order reversal: 1st 0xfffff800102332e0 ufs (ufs) @ kern/vfs_subr.c:2590 2nd 0xfffffe00008b4728 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:282 3rd 0xfffff8031d690cc0 ufs (ufs) @ kern/vfs_subr.c:2590 stack backtrace: #0 0xffffffff80c4bf83 at witness_debugger+0x73 #1 0xffffffff80c4bcd3 at witness_checkorder+0xac3 #2 0xffffffff80bb67fb at lockmgr_xlock_hard+0x6b #3 0xffffffff80bb7283 at __lockmgr_args+0x5f3 #4 0xffffffff80ef96cc at ffs_lock+0x7c #5 0xffffffff81243a79 at VOP_LOCK1_APV+0x1d9 #6 0xffffffff80cc8125 at _vn_lock+0x65 #7 0xffffffff80cb6566 at vget+0xa6 #8 0xffffffff80ca829c at vfs_hash_get+0xcc #9 0xffffffff80ef500f at ffs_vgetf+0x3f #10 0xffffffff80eea682 at softdep_sync_buf+0x382 #11 0xffffffff80efa454 at ffs_syncvnode+0x224 #12 0xffffffff80ef7c9d at ffs_sync+0x20d #13 0xffffffff80caeba2 at dounmount+0x462 #14 0xffffffff80cae6a0 at sys_unmount+0x300 #15 0xffffffff810baeb1 at amd64_syscall+0x291 #16 0xffffffff81094b1d at fast_syscall_common+0x101 lock order reversal: 1st 0xfffff800076cdcc0 ufs (ufs) @ kern/vfs_mount.c:1353 2nd 0xfffff80010233558 devfs (devfs) @ ufs/ffs/ffs_softdep.c:1914 stack backtrace: #0 0xffffffff80c4bf83 at witness_debugger+0x73 #1 0xffffffff80c4bcd3 at witness_checkorder+0xac3 #2 0xffffffff80bb62fb at lockmgr_lock_fast_path+0x16b #3 0xffffffff81243a79 at VOP_LOCK1_APV+0x1d9 #4 0xffffffff80cc8125 at _vn_lock+0x65 #5 0xffffffff80ed7c7d at softdep_flushworklist+0x5d #6 0xffffffff80ef7e96 at ffs_sync+0x406 #7 0xffffffff80caeba2 at dounmount+0x462 #8 0xffffffff80cae6a0 at sys_unmount+0x300 #9 0xffffffff810baeb1 at amd64_syscall+0x291 #10 0xffffffff81094b1d at fast_syscall_common+0x101 ** /dev/gpt/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 1683 files, 56221 used, 1209186 free (106 frags, 151135 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** Updating motd:. Mounting late filesystems:. Configuring vt: blanktime. Performing sanity check on sshd configuration. Starting sshd. Starting cron. Wed Feb 13 08:49 FreeBSD/amd64 (mercat1.netperf.freebsd.org) (ttyu1) login: root Feb 13 08:50:14 mercat1 login[2306]: ROOT LOGIN (root) ON ttyu1 Last login: Wed Feb 13 08:39:25 on ttyu1 FreeBSD 13.0-CURRENT (PHO) #1 r344045: Wed Feb 13 08:42:31 CET 2019 root@mercat1:~ # sh # cd /stress2/misc # cat sendfile11.sh #!/bin/sh # # Copyright (c) 2017 Dell EMC Isilon # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # # THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE # ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT # LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY # OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # SUCH DAMAGE. # # $FreeBSD$ # [ `id -u ` -ne 0 ] && echo "Must be root!" && exit 1 # sendfile(2) && block size > page size: # panic: vnode_pager_generic_getpages: sector size 8192 too large . ../default.cfg odir=`pwd` cd /tmp sed '1,/^EOF/d' < $odir/$0 > sendfile11.c mycc -o sendfile11 -Wall sendfile11.c -pthread || exit 1 rm -f sendfile11.c set -e mount | grep $mntpoint | grep -q /dev/md && umount -f $mntpoint [ -c /dev/md$mdstart ] && mdconfig -d -u $mdstart mdconfig -a -t swap -s 2g -u $mdstart dd if=/dev/random of=/tmp/geli.key bs=64 count=1 status=none echo test | geli init -s 8192 -J - -K /tmp/geli.key /dev/md$mdstart > /dev/null echo test | geli attach -j - -k /tmp/geli.key /dev/md$mdstart newfs $newfs_flags /dev/md$mdstart.eli > /dev/null mount /dev/md${mdstart}.eli $mntpoint chmod 777 $mntpoint set +e cd $mntpoint dd if=/dev/zero of=file bs=1m count=512 status=none (cd $odir/../testcases/swap; ./swap -t 5m -i 20 -h -l 100) & sleep 5 /tmp/sendfile11 file output 12345; s=$? cd $odir while pkill swap; do sleep 1 done wait while mount | grep $mntpoint | grep -q /dev/md; do umount $mntpoint || sleep 1 done checkfs /dev/md${mdstart}.eli || s=$? geli kill /dev/md$mdstart.eli mdconfig -d -u $mdstart rm -f /tmp/geli.key /tmp/sendfile11 exit $s EOF #include #include #include #include #include #include #include #include #include #include #include #include #include #define BUFSIZE 8192 static int port; static char *inputFile; static char *outputFile; static void reader(void) { struct sockaddr_in inetaddr, inetpeer; socklen_t len; int tcpsock, msgsock; int *buf, fd, n, on, t; on = 1; if ((tcpsock = socket(AF_INET, SOCK_STREAM, 0)) < 0) err(1, "socket(), %s:%d", __FILE__, __LINE__); if (setsockopt(tcpsock, SOL_SOCKET, SO_REUSEADDR, (char *)&on, sizeof(on)) < 0) err(1, "setsockopt(), %s:%d", __FILE__, __LINE__); inetaddr.sin_family = AF_INET; inetaddr.sin_addr.s_addr = INADDR_ANY; inetaddr.sin_port = htons(port); inetaddr.sin_len = sizeof(inetaddr); if (bind(tcpsock, (struct sockaddr *)&inetaddr, sizeof (inetaddr)) < 0) err(1, "bind(), %s:%d", __FILE__, __LINE__); if (listen(tcpsock, 5) < 0) err(1, "listen(), %s:%d", __FILE__, __LINE__); len = sizeof(inetpeer); if ((msgsock = accept(tcpsock, (struct sockaddr *)&inetpeer, &len)) < 0) err(1, "accept(), %s:%d", __FILE__, __LINE__); t = 0; if ((buf = malloc(BUFSIZE)) == NULL) err(1, "malloc(%d), %s:%d", BUFSIZE, __FILE__, __LINE__); if ((fd = open(outputFile, O_RDWR | O_CREAT | O_TRUNC, 0640)) == -1) err(1, "open(%s)", outputFile); for (;;) { if ((n = read(msgsock, buf, BUFSIZE)) < 0) err(1, "read(), %s:%d", __FILE__, __LINE__); t += n; if (n == 0) break; if ((write(fd, buf, n)) != n) err(1, "write"); } close(msgsock); close(fd); return; } static void writer(void) { struct sockaddr_in inetaddr; struct hostent *hostent; struct stat statb; off_t off = 0; size_t size; int i, fd, on, r, tcpsock; on = 1; for (i = 1; i < 5; i++) { if ((tcpsock = socket(AF_INET, SOCK_STREAM, 0)) < 0) err(1, "socket(), %s:%d", __FILE__, __LINE__); if (setsockopt(tcpsock, SOL_SOCKET, SO_REUSEADDR, (char *)&on, sizeof(on)) < 0) err(1, "setsockopt(), %s:%d", __FILE__, __LINE__); size = getpagesize() -4; if (setsockopt(tcpsock, SOL_SOCKET, SO_SNDBUF, (void *)&size, sizeof(size)) < 0) err(1, "setsockopt(SO_SNDBUF), %s:%d", __FILE__, __LINE__); hostent = gethostbyname ("localhost"); memcpy (&inetaddr.sin_addr.s_addr, hostent->h_addr, sizeof (struct in_addr)); inetaddr.sin_family = AF_INET; inetaddr.sin_addr.s_addr = INADDR_ANY; inetaddr.sin_port = htons(port); inetaddr.sin_len = sizeof(inetaddr); r = connect(tcpsock, (struct sockaddr *) &inetaddr, sizeof(inetaddr)); if (r == 0) break; sleep(1); close(tcpsock); } if (r < 0) err(1, "connect(), %s:%d", __FILE__, __LINE__); if (stat(inputFile, &statb) != 0) err(1, "stat(%s)", inputFile); if ((fd = open(inputFile, O_RDONLY)) == -1) err(1, "open(%s)", inputFile); if (sendfile(fd, tcpsock, 0, statb.st_size, NULL, &off, 0) == -1) err(1, "sendfile"); return; } int main(int argc, char **argv) { pid_t pid; int e, s; if (argc != 4) { fprintf(stderr, "Usage: %s 0) { reader(); kill(pid, SIGINT); waitpid(pid, &s, 0); if (s != 0) e = 1; } else err(1, "fork(), %s:%d", __FILE__, __LINE__); return (e); } # ./sendfile11.sh warning: Using sectorsize bigger than the page size! GEOM_ELI: Device md10.eli created. GEOM_ELI: Encryption: AES-XTS 128 GEOM_ELI: Crypto: software swap: run time 0+00:05:00, incarnations 20, load 100, verbose 1 panic: vnode_pager_generic_getpages: sector size 8192 too large cpuid = 8 time = 1550044270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00b6976050 vpanic() at vpanic+0x1b4/frame 0xfffffe00b69760b0 panic() at panic+0x43/frame 0xfffffe00b6976110 vnode_pager_generic_getpages() at vnode_pager_generic_getpages+0xf58/frame 0xfffffe00b6976240 vnode_pager_local_getpages_async() at vnode_pager_local_getpages_async+0x4e/frame 0xfffffe00b6976260 VOP_GETPAGES_ASYNC_APV() at VOP_GETPAGES_ASYNC_APV+0x1f5/frame 0xfffffe00b69762d0 VOP_GETPAGES_ASYNC() at VOP_GETPAGES_ASYNC+0x79/frame 0xfffffe00b6976360 vnode_pager_getpages_async() at vnode_pager_getpages_async+0x78/frame 0xfffffe00b69763c0 sendfile_swapin() at sendfile_swapin+0x594/frame 0xfffffe00b69764d0 vn_sendfile() at vn_sendfile+0x995/frame 0xfffffe00b6976800 fo_sendfile() at fo_sendfile+0x8e/frame 0xfffffe00b69768a0 sendfile() at sendfile+0x1f2/frame 0xfffffe00b6976960 sys_sendfile() at sys_sendfile+0x24/frame 0xfffffe00b6976990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b6976ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b6976ab0 --- syscall (393, FreeBSD ELF64, sys_sendfile), rip = 0x80030744a, rsp = 0x7fffffffe688, rbp = 0x7fffffffe7e0 --- KDB: enter: panic [ thread pid 2377 tid 100211 ] Stopped at kdb_enter+0x3b: movq $0,kdb_why db> run pho db:0:pho> set $lines 20000 db:0:pho> run pho1 db:1:pho1> bt Tracing pid 2377 tid 100211 td 0xfffff800106c65a0 kdb_enter() at kdb_enter+0x3b/frame 0xfffffe00b6976050 vpanic() at vpanic+0x1d1/frame 0xfffffe00b69760b0 panic() at panic+0x43/frame 0xfffffe00b6976110 vnode_pager_generic_getpages() at vnode_pager_generic_getpages+0xf58/frame 0xfffffe00b6976240 vnode_pager_local_getpages_async() at vnode_pager_local_getpages_async+0x4e/frame 0xfffffe00b6976260 VOP_GETPAGES_ASYNC_APV() at VOP_GETPAGES_ASYNC_APV+0x1f5/frame 0xfffffe00b69762d0 VOP_GETPAGES_ASYNC() at VOP_GETPAGES_ASYNC+0x79/frame 0xfffffe00b6976360 vnode_pager_getpages_async() at vnode_pager_getpages_async+0x78/frame 0xfffffe00b69763c0 sendfile_swapin() at sendfile_swapin+0x594/frame 0xfffffe00b69764d0 vn_sendfile() at vn_sendfile+0x995/frame 0xfffffe00b6976800 fo_sendfile() at fo_sendfile+0x8e/frame 0xfffffe00b69768a0 sendfile() at sendfile+0x1f2/frame 0xfffffe00b6976960 sys_sendfile() at sys_sendfile+0x24/frame 0xfffffe00b6976990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b6976ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b6976ab0 --- syscall (393, FreeBSD ELF64, sys_sendfile), rip = 0x80030744a, rsp = 0x7fffffffe688, rbp = 0x7fffffffe7e0 --- db:1:pho1> show allpcpu Current CPU: 8 cpuid = 0 dynamic pcpu = 0xd64680 curthread = 0xfffff8031dc765a0: pid 2375 tid 100253 "swap" curpcb = 0xfffffe00b8a40b80 fpcurthread = 0xfffff8031dc765a0: pid 2375 "swap" idlethread = 0xfffff800036b8000: tid 100003 "idle: cpu0" curpmap = 0xfffff803a44f0138 tssp = 0xffffffff821cc420 commontssp = 0xffffffff821cc420 rsp0 = 0xfffffe00b8a40b80 gs32p = 0xffffffff821d3058 ldt = 0xffffffff821d3098 tss = 0xffffffff821d3088 tlb gen = 32174 curvnet = 0 spin locks held: cpuid = 1 dynamic pcpu = 0xfffffe0084fdd680 curthread = 0xfffff800104265a0: pid 2372 tid 100250 "swap" curpcb = 0xfffffe00b8a31b80 fpcurthread = 0xfffff800104265a0: pid 2372 "swap" idlethread = 0xfffff800036b75a0: tid 100004 "idle: cpu1" curpmap = 0xfffff803a44aa138 tssp = 0xffffffff821cc488 commontssp = 0xffffffff821cc488 rsp0 = 0xfffffe00b8a31b80 gs32p = 0xffffffff821d30c0 ldt = 0xffffffff821d3100 tss = 0xffffffff821d30f0 tlb gen = 32174 curvnet = 0 spin locks held: cpuid = 2 dynamic pcpu = 0xfffffe0084fe9680 curthread = 0xfffff8000616c5a0: pid 2365 tid 100142 "swap" curpcb = 0xfffffe00b67e4b80 fpcurthread = 0xfffff8000616c5a0: pid 2365 "swap" idlethread = 0xfffff800036b7000: tid 100005 "idle: cpu2" curpmap = 0xfffff803a441d138 tssp = 0xffffffff821cc4f0 commontssp = 0xffffffff821cc4f0 rsp0 = 0xfffffe00b67e4b80 gs32p = 0xffffffff821d3128 ldt = 0xffffffff821d3168 tss = 0xffffffff821d3158 tlb gen = 32172 curvnet = 0 spin locks held: cpuid = 3 dynamic pcpu = 0xfffffe0084ff5680 curthread = 0xfffff800104c55a0: pid 2363 tid 100246 "swap" curpcb = 0xfffffe00b8a1db80 fpcurthread = 0xfffff800104c55a0: pid 2363 "swap" idlethread = 0xfffff800036b65a0: tid 100006 "idle: cpu3" curpmap = 0xfffff8001048e138 tssp = 0xffffffff821cc558 commontssp = 0xffffffff821cc558 rsp0 = 0xfffffe00b8a1db80 gs32p = 0xffffffff821d3190 ldt = 0xffffffff821d31d0 tss = 0xffffffff821d31c0 tlb gen = 32174 curvnet = 0 spin locks held: cpuid = 4 dynamic pcpu = 0xfffffe0085001680 curthread = 0xfffff800104c7000: pid 2356 tid 100168 "swap" curpcb = 0xfffffe00b6868b80 fpcurthread = 0xfffff800104c7000: pid 2356 "swap" idlethread = 0xfffff800036b6000: tid 100007 "idle: cpu4" curpmap = 0xfffff8031d80f138 tssp = 0xffffffff821cc5c0 commontssp = 0xffffffff821cc5c0 rsp0 = 0xfffffe00b6868b80 gs32p = 0xffffffff821d31f8 ldt = 0xffffffff821d3238 tss = 0xffffffff821d3228 tlb gen = 32174 curvnet = 0 spin locks held: cpuid = 5 dynamic pcpu = 0xfffffe008500d680 curthread = 0xfffff8031dc76000: pid 2374 tid 100252 "swap" curpcb = 0xfffffe00b8a3bb80 fpcurthread = 0xfffff8031dc76000: pid 2374 "swap" idlethread = 0xfffff800036b55a0: tid 100008 "idle: cpu5" curpmap = 0xfffff803a44f1138 tssp = 0xffffffff821cc628 commontssp = 0xffffffff821cc628 rsp0 = 0xfffffe00b8a3bb80 gs32p = 0xffffffff821d3260 ldt = 0xffffffff821d32a0 tss = 0xffffffff821d3290 tlb gen = 32174 curvnet = 0 spin locks held: cpuid = 6 dynamic pcpu = 0xfffffe0085019680 curthread = 0xfffff8031dc775a0: pid 2361 tid 100244 "swap" curpcb = 0xfffffe00b8a13b80 fpcurthread = 0xfffff8031dc775a0: pid 2361 "swap" idlethread = 0xfffff800036b5000: tid 100009 "idle: cpu6" curpmap = 0xfffff80010484138 tssp = 0xffffffff821cc690 commontssp = 0xffffffff821cc690 rsp0 = 0xfffffe00b8a13b80 gs32p = 0xffffffff821d32c8 ldt = 0xffffffff821d3308 tss = 0xffffffff821d32f8 tlb gen = 32208 curvnet = 0 spin locks held: cpuid = 7 dynamic pcpu = 0xfffffe0085025680 curthread = 0xfffff8001072e000: pid 2373 tid 100251 "swap" curpcb = 0xfffffe00b8a36b80 fpcurthread = 0xfffff8001072e000: pid 2373 "swap" idlethread = 0xfffff800036c95a0: tid 100010 "idle: cpu7" curpmap = 0xfffff803a44a9138 tssp = 0xffffffff821cc6f8 commontssp = 0xffffffff821cc6f8 rsp0 = 0xfffffe00b8a36b80 gs32p = 0xffffffff821d3330 ldt = 0xffffffff821d3370 tss = 0xffffffff821d3360 tlb gen = 32174 curvnet = 0 spin locks held: cpuid = 8 dynamic pcpu = 0xfffffe0085031680 curthread = 0xfffff800106c65a0: pid 2377 tid 100211 "sendfile11" curpcb = 0xfffffe00b6976b80 fpcurthread = 0xfffff800106c65a0: pid 2377 "sendfile11" idlethread = 0xfffff800036c9000: tid 100011 "idle: cpu8" curpmap = 0xfffff800100f3138 tssp = 0xffffffff821cc760 commontssp = 0xffffffff821cc760 rsp0 = 0xfffffe00b6976b80 gs32p = 0xffffffff821d3398 ldt = 0xffffffff821d33d8 tss = 0xffffffff821d33c8 tlb gen = 32207 curvnet = 0 spin locks held: cpuid = 9 dynamic pcpu = 0xfffffe008503d680 curthread = 0xfffff8001070d000: pid 2369 tid 100174 "swap" curpcb = 0xfffffe00b6892b80 fpcurthread = 0xfffff8001070d000: pid 2369 "swap" idlethread = 0xfffff800036c85a0: tid 100012 "idle: cpu9" curpmap = 0xfffff803a4461138 tssp = 0xffffffff821cc7c8 commontssp = 0xffffffff821cc7c8 rsp0 = 0xfffffe00b6892b80 gs32p = 0xffffffff821d3400 ldt = 0xffffffff821d3440 tss = 0xffffffff821d3430 tlb gen = 32174 curvnet = 0 spin locks held: cpuid = 10 dynamic pcpu = 0xfffffe0085049680 curthread = 0xfffff80010bb35a0: pid 2368 tid 100172 "swap" curpcb = 0xfffffe00b6888b80 fpcurthread = 0xfffff80010bb35a0: pid 2368 "swap" idlethread = 0xfffff800036c8000: tid 100013 "idle: cpu10" curpmap = 0xfffff803a4462138 tssp = 0xffffffff821cc830 commontssp = 0xffffffff821cc830 rsp0 = 0xfffffe00b6888b80 gs32p = 0xffffffff821d3468 ldt = 0xffffffff821d34a8 tss = 0xffffffff821d3498 tlb gen = 32174 curvnet = 0 spin locks held: cpuid = 11 dynamic pcpu = 0xfffffe0085055680 curthread = 0xfffff8000782d5a0: pid 30 tid 100128 "dom0" curpcb = 0xfffffe0006f2eb80 fpcurthread = none idlethread = 0xfffff800036c75a0: tid 100014 "idle: cpu11" curpmap = 0xffffffff82094d20 tssp = 0xffffffff821cc898 commontssp = 0xffffffff821cc898 rsp0 = 0xfffffe0006f2eb80 gs32p = 0xffffffff821d34d0 ldt = 0xffffffff821d3510 tss = 0xffffffff821d3500 tlb gen = 32174 curvnet = 0 spin locks held: db:1:pho1> show alllocks Process 2377 (sendfile11) thread 0xfffff800106c65a0 (100211) exclusive lockmgr bufwait (bufwait) r = 0 (0xfffff80007ff6080) locked @ vm/vm_pager.c:379 shared lockmgr ufs (ufs) r = 0 (0xfffff8031dbc27d0) locked @ kern/kern_sendfile.c:699 exclusive sx so_snd_sx (so_snd_sx) r = 0 (0xfffff80010c8e5d0) locked @ kern/uipc_sockbuf.c:282 Process 2376 (sendfile11) thread 0xfffff8001070c5a0 (100175) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xfffff80010d3d178) locked @ kern/uipc_sockbuf.c:285 Process 30 (pagedaemon) thread 0xfffff8000782d5a0 (100128) exclusive sleep mutex vnode interlock (vnode interlock) r = 0 (0xfffff800103bf898) locked @ kern/vfs_subr.c:2902 exclusive rw vm object (vm object) r = 0 (0xfffff800103b3600) locked @ vm/vm_pageout.c:1456 exclusive sleep mutex vm page (vm page) r = 0 (0xffffffff81a46580) locked @ vm/vm_page.c:1103 db:1:pho1> show lockedvnods Locked vnodes vnode 0xfffff8031dbc2768: tag ufs, type VREG usecount 2, writecount 0, refcount 7187 flags (VI_ACTIVE) v_object 0xfffff800103e3000 ref 1 pages 111804 cleanbuf 7179 dirtybuf 5 lock type ufs: SHARED (count 1) #0 0xffffffff80bb672c at lockmgr_slock_hard+0x34c #1 0xffffffff80bb73d5 at __lockmgr_args+0x745 #2 0xffffffff80ef96cc at ffs_lock+0x7c #3 0xffffffff81243a79 at VOP_LOCK1_APV+0x1d9 #4 0xffffffff80cc8125 at _vn_lock+0x65 #5 0xffffffff80bdf58c at vn_sendfile+0x6bc #6 0xffffffff80be25be at fo_sendfile+0x8e #7 0xffffffff80be1d92 at sendfile+0x1f2 #8 0xffffffff80be1b94 at sys_sendfile+0x24 #9 0xffffffff810baeb1 at amd64_syscall+0x291 #10 0xffffffff81094b1d at fast_syscall_common+0x101 ino 4, on dev md10.eli db:1:pho1> show allchains chain 1: thread 100175 (pid 2376, sendfile11) sleeping on 0xfffff80010d3d1d4 "sbwait" chain 2: thread 100206 (pid 2355, swap) sleeping on 0xfffff800103f1000 "wait" chain 3: thread 100230 (pid 2353, swap) sleeping on 0xffffffff81e9cfb3 "nanslp" chain 4: thread 100239 (pid 2347, g_eli[10] md10) sleeping on 0xfffff80010364000 "geli:w" chain 5: thread 100238 (pid 2346, g_eli[9] md10) sleeping on 0xfffff80010364000 "geli:w" chain 6: thread 100237 (pid 2345, g_eli[8] md10) sleeping on 0xfffff80010364000 "geli:w" chain 7: thread 100236 (pid 2344, g_eli[7] md10) sleeping on 0xfffff80010364000 "geli:w" chain 8: thread 100155 (pid 2343, g_eli[6] md10) sleeping on 0xfffff80010364000 "geli:w" chain 9: thread 100235 (pid 2342, g_eli[5] md10) sleeping on 0xfffff80010364000 "geli:w" chain 10: thread 100234 (pid 2341, g_eli[4] md10) sleeping on 0xfffff80010364000 "geli:w" chain 11: thread 100233 (pid 2340, g_eli[3] md10) sleeping on 0xfffff80010364000 "geli:w" chain 12: thread 100232 (pid 2339, g_eli[2] md10) sleeping on 0xfffff80010364000 "geli:w" chain 13: thread 100231 (pid 2338, g_eli[1] md10) sleeping on 0xfffff80010364000 "geli:w" chain 14: thread 100217 (pid 2337, g_eli[0] md10) sleeping on 0xfffff80010364000 "geli:w" chain 15: thread 100212 (pid 2331, md10) sleeping on 0xfffff8031d9f3000 "mdwait" chain 16: thread 100188 (pid 2315, sh) sleeping on 0xfffff800106aa000 "wait" chain 17: thread 100207 (pid 2313, sh) sleeping on 0xfffff80010705000 "wait" chain 18: thread 100197 (pid 2309, csh) sleeping on 0xfffff80010deb0a8 "pause" chain 19: thread 100216 (pid 2306, login) sleeping on 0xfffff8031d9c7000 "wait" chain 20: thread 100215 (pid 2305, getty) sleeping on 0xfffff8000612b8b0 "ttyin" chain 21: thread 100214 (pid 2304, getty) sleeping on 0xfffff800079c7cb0 "ttyin" chain 22: thread 100213 (pid 2303, getty) sleeping on 0xfffff800079c80b0 "ttyin" chain 23: thread 100169 (pid 2302, getty) sleeping on 0xfffff800079c84b0 "ttyin" chain 24: thread 100170 (pid 2301, getty) sleeping on 0xfffff800079c88b0 "ttyin" chain 25: thread 100191 (pid 2300, getty) sleeping on 0xfffff800078764b0 "ttyin" chain 26: thread 100161 (pid 2299, getty) sleeping on 0xfffff800078758b0 "ttyin" chain 27: thread 100152 (pid 2298, getty) sleeping on 0xfffff80007874cb0 "ttyin" chain 28: thread 100199 (pid 2297, getty) sleeping on 0xfffff800078740b0 "ttyin" chain 29: thread 100204 (pid 2251, cron) sleeping on 0xffffffff81e9cfb3 "nanslp" chain 30: thread 100203 (pid 2247, sshd) sleeping on 0xfffff8000313e4c0 "select" chain 31: thread 100166 (pid 2168, NLM: master) sleeping on 0xfffff80010e58360 "rpcsvc" chain 32: thread 100196 (pid 2165, rpc.statd) sleeping on 0xfffff800078463c0 "select" chain 33: thread 100150 (pid 2119, watchdogd) sleeping on 0xffffffff81e9cfb2 "nanslp" chain 34: thread 100153 (pid 2040, rpcbind) sleeping on 0xfffff8031d23a740 "select" chain 35: thread 100194 (pid 2030, syslogd) sleeping on 0xfffff80003985b40 "select" chain 36: thread 100164 (pid 1959, devd) sleeping on 0xfffff80007974040 "select" chain 37: thread 100177 (pid 1954, dhclient) sleeping on 0xfffff8031d23a7c0 "select" chain 38: thread 100165 (pid 1910, dhclient) sleeping on 0xfffff80003e669c0 "select" chain 39: thread 100149 (pid 1907, dhclient) sleeping on 0xfffff8000781a040 "select" chain 40: thread 100141 (pid 34, syncer) sleeping on 0xffffffff8203f400 "syncer" chain 41: thread 100140 (pid 33, vnlru) sleeping on 0xfffff800077e9000 "vlruwt" chain 42: thread 100130 (pid 32, bufdaemon) sleeping on 0xffffffff8203e8a8 "qsleep" chain 43: thread 100133 (pid 32, bufspacedaemon-0) sleeping on 0xffffffff81a0a880 "-" chain 44: thread 100134 (pid 32, bufspacedaemon-1) sleeping on 0xffffffff81a12b40 "-" chain 45: thread 100135 (pid 32, bufspacedaemon-2) sleeping on 0xffffffff81a1ae00 "-" chain 46: thread 100136 (pid 32, bufspacedaemon-3) sleeping on 0xffffffff81a230c0 "-" chain 47: thread 100137 (pid 32, bufspacedaemon-4) sleeping on 0xffffffff81a2b380 "-" chain 48: thread 100138 (pid 32, bufspacedaemon-5) sleeping on 0xffffffff81a33640 "-" chain 49: thread 100139 (pid 32, bufspacedaemon-6) sleeping on 0xffffffff81a3b900 "-" chain 50: thread 100151 (pid 32, / worker) blocked on lockmgr Hy0~EXCL thread 0 (pid -1, ) is inactive chain 51: thread 100202 (pid 32, /tmp worker) sleeping on 0xfffff80006133ce8 "sdflush" chain 52: thread 100241 (pid 32, /mnt worker) sleeping on 0xfffff80007ae06e8 "sdflush" chain 53: thread 100129 (pid 31, vmdaemon) sleeping on 0xffffffff8205b748 "psleep" chain 54: thread 100131 (pid 30, laundry: dom0) sleeping on 0xffffffff820c5524 "launds" chain 55: thread 100132 (pid 30, uma) sleeping on 0xffffffff80f17280 "umarcl" chain 56: thread 100127 (pid 29, enc_daemon1) sleeping on 0xfffff800076f7000 "idle" chain 57: thread 100126 (pid 28, enc_daemon0) sleeping on 0xfffff800076f7530 "idle" chain 58: thread 100123 (pid 27, rand_harvestq) sleeping on 0xffffffff81cfc628 "-" chain 59: thread 100119 (pid 26, sctp_iterator) sleeping on 0xffffffff820c3c00 "waiting_for_work" chain 60: thread 100118 (pid 25, soaiod4) sleeping on 0xffffffff8203e29c "-" chain 61: thread 100117 (pid 24, soaiod3) sleeping on 0xffffffff8203e29c "-" chain 62: thread 100116 (pid 23, soaiod2) sleeping on 0xffffffff8203e29c "-" chain 63: thread 100115 (pid 22, soaiod1) sleeping on 0xffffffff8203e29c "-" chain 64: thread 100096 (pid 21, usbus0) sleeping on 0xfffffe009a1aa460 "-" chain 65: thread 100097 (pid 21, usbus0) sleeping on 0xfffffe009a1aa4b8 "-" chain 66: thread 100098 (pid 21, usbus0) sleeping on 0xfffffe009a1aa510 "-" chain 67: thread 100099 (pid 21, usbus0) sleeping on 0xfffffe009a1aa568 "-" chain 68: thread 100100 (pid 21, usbus0) sleeping on 0xfffffe009a1aa5c0 "-" chain 69: thread 100102 (pid 21, usbus1) sleeping on 0xfffffe0006dd6d10 "-" chain 70: thread 100103 (pid 21, usbus1) sleeping on 0xfffffe0006dd6d68 "-" chain 71: thread 100104 (pid 21, usbus1) sleeping on 0xfffffe0006dd6dc0 "-" chain 72: thread 100105 (pid 21, usbus1) sleeping on 0xfffffe0006dd6e18 "-" chain 73: thread 100106 (pid 21, usbus1) sleeping on 0xfffffe0006dd6e70 "-" chain 74: thread 100107 (pid 21, usbus2) sleeping on 0xfffffe00007f3d10 "-" chain 75: thread 100108 (pid 21, usbus2) sleeping on 0xfffffe00007f3d68 "-" chain 76: thread 100109 (pid 21, usbus2) sleeping on 0xfffffe00007f3dc0 "-" chain 77: thread 100110 (pid 21, usbus2) sleeping on 0xfffffe00007f3e18 "-" chain 78: thread 100111 (pid 21, usbus2) sleeping on 0xfffffe00007f3e70 "-" chain 79: thread 100091 (pid 20, doneq0) sleeping on 0xffffffff81bd78c0 "-" chain 80: thread 100092 (pid 20, doneq1) sleeping on 0xffffffff81bd7940 "-" chain 81: thread 100093 (pid 20, doneq2) sleeping on 0xffffffff81bd79c0 "-" chain 82: thread 100125 (pid 20, scanner) sleeping on 0xffffffff81bd7788 "-" chain 83: thread 100090 (pid 19, crypto returns 11) sleeping on 0xfffff8000398f450 "crypto_ret_wait" chain 84: thread 100089 (pid 18, crypto returns 10) sleeping on 0xfffff8000398f3f0 "crypto_ret_wait" chain 85: thread 100088 (pid 17, crypto returns 9) sleeping on 0xfffff8000398f390 "crypto_ret_wait" chain 86: thread 100087 (pid 16, crypto returns 8) sleeping on 0xfffff8000398f330 "crypto_ret_wait" chain 87: thread 100086 (pid 15, crypto returns 7) sleeping on 0xfffff8000398f2d0 "crypto_ret_wait" chain 88: thread 100085 (pid 9, crypto returns 6) sleeping on 0xfffff8000398f270 "crypto_ret_wait" chain 89: thread 100084 (pid 8, crypto returns 5) sleeping on 0xfffff8000398f210 "crypto_ret_wait" chain 90: thread 100083 (pid 7, crypto returns 4) sleeping on 0xfffff8000398f1b0 "crypto_ret_wait" chain 91: thread 100082 (pid 6, crypto returns 3) sleeping on 0xfffff8000398f150 "crypto_ret_wait" chain 92: thread 100081 (pid 5, crypto returns 2) sleeping on 0xfffff8000398f0f0 "crypto_ret_wait" chain 93: thread 100080 (pid 4, crypto returns 1) sleeping on 0xfffff8000398f090 "crypto_ret_wait" chain 94: thread 100079 (pid 3, crypto returns 0) sleeping on 0xfffff8000398f030 "crypto_ret_wait" chain 95: thread 100078 (pid 2, crypto) sleeping on 0xffffffff82055d10 "crypto_wait" chain 96: thread 100065 (pid 14, sequencer 00) sleeping on 0xfffff800038fac88 "seqstate" chain 97: thread 100061 (pid 13, g_event) sleeping on 0xffffffff820940c0 "-" chain 98: thread 100062 (pid 13, g_up) sleeping on 0xffffffff820940d0 "-" chain 99: thread 100063 (pid 13, g_down) sleeping on 0xffffffff820940b8 "-" chain 100: thread 100002 (pid 1, init) sleeping on 0xfffff800036b4530 "wait" chain 101: thread 100001 (pid 10, audit) sleeping on 0xffffffff820c4920 "audit_worker_cv" chain 102: thread 100000 (pid 0, swapper) sleeping on 0xffffffff82094100 "swapin" chain 103: thread 100015 (pid 0, thread taskq) sleeping on 0xfffff800036a2e00 "-" chain 104: thread 100017 (pid 0, config_0) sleeping on 0xfffff80003693a00 "-" chain 105: thread 100018 (pid 0, kqueue_ctx taskq) sleeping on 0xfffff800036a2900 "-" chain 106: thread 100019 (pid 0, aiod_kick taskq) sleeping on 0xfffff800036a2700 "-" chain 107: thread 100022 (pid 0, if_config_tqg_0) sleeping on 0xfffff80003693700 "-" chain 108: thread 100023 (pid 0, softirq_0) sleeping on 0xfffff80003693600 "-" chain 109: thread 100024 (pid 0, softirq_1) sleeping on 0xfffff80003693500 "-" chain 110: thread 100025 (pid 0, softirq_2) sleeping on 0xfffff80003693400 "-" chain 111: thread 100026 (pid 0, softirq_3) sleeping on 0xfffff80003693300 "-" chain 112: thread 100027 (pid 0, softirq_4) sleeping on 0xfffff80003693200 "-" chain 113: thread 100028 (pid 0, softirq_5) sleeping on 0xfffff80003693100 "-" chain 114: thread 100029 (pid 0, softirq_6) sleeping on 0xfffff80003693000 "-" chain 115: thread 100030 (pid 0, softirq_7) sleeping on 0xfffff80003747100 "-" chain 116: thread 100031 (pid 0, softirq_8) sleeping on 0xfffff80003747000 "-" chain 117: thread 100032 (pid 0, softirq_9) sleeping on 0xfffff80003746e00 "-" chain 118: thread 100033 (pid 0, softirq_10) sleeping on 0xfffff80003746d00 "-" chain 119: thread 100034 (pid 0, softirq_11) sleeping on 0xfffff80003746c00 "-" chain 120: thread 100035 (pid 0, if_io_tqg_0) sleeping on 0xfffff800037d8200 "-" chain 121: thread 100036 (pid 0, if_io_tqg_1) sleeping on 0xfffff800037d8100 "-" chain 122: thread 100037 (pid 0, if_io_tqg_2) sleeping on 0xfffff800037d8000 "-" chain 123: thread 100038 (pid 0, if_io_tqg_3) sleeping on 0xfffff800037d7e00 "-" chain 124: thread 100039 (pid 0, if_io_tqg_4) sleeping on 0xfffff800037d7d00 "-" chain 125: thread 100040 (pid 0, if_io_tqg_5) sleeping on 0xfffff800037d7c00 "-" chain 126: thread 100041 (pid 0, if_io_tqg_6) sleeping on 0xfffff800037d7b00 "-" chain 127: thread 100042 (pid 0, if_io_tqg_7) sleeping on 0xfffff800037d7a00 "-" chain 128: thread 100043 (pid 0, if_io_tqg_8) sleeping on 0xfffff800037d7900 "-" chain 129: thread 100044 (pid 0, if_io_tqg_9) sleeping on 0xfffff800037d7800 "-" chain 130: thread 100045 (pid 0, if_io_tqg_10) sleeping on 0xfffff800037d7700 "-" chain 131: thread 100046 (pid 0, if_io_tqg_11) sleeping on 0xfffff800037d7600 "-" chain 132: thread 100064 (pid 0, firmware taskq) sleeping on 0xfffff80003827a00 "-" chain 133: thread 100066 (pid 0, crypto_0) sleeping on 0xfffff80003827300 "-" chain 134: thread 100067 (pid 0, crypto_1) sleeping on 0xfffff80003827300 "-" chain 135: thread 100068 (pid 0, crypto_2) sleeping on 0xfffff80003827300 "-" chain 136: thread 100069 (pid 0, crypto_3) sleeping on 0xfffff80003827300 "-" chain 137: thread 100070 (pid 0, crypto_4) sleeping on 0xfffff80003827300 "-" chain 138: thread 100071 (pid 0, crypto_5) sleeping on 0xfffff80003827300 "-" chain 139: thread 100072 (pid 0, crypto_6) sleeping on 0xfffff80003827300 "-" chain 140: thread 100073 (pid 0, crypto_7) sleeping on 0xfffff80003827300 "-" chain 141: thread 100074 (pid 0, crypto_8) sleeping on 0xfffff80003827300 "-" chain 142: thread 100075 (pid 0, crypto_9) sleeping on 0xfffff80003827300 "-" chain 143: thread 100076 (pid 0, crypto_10) sleeping on 0xfffff80003827300 "-" chain 144: thread 100077 (pid 0, crypto_11) sleeping on 0xfffff80003827300 "-" chain 145: thread 100114 (pid 0, mca taskq) sleeping on 0xfffff80003aeed00 "-" chain 146: thread 100120 (pid 0, acpi_task_0) sleeping on 0xfffff800037f5600 "-" chain 147: thread 100121 (pid 0, acpi_task_1) sleeping on 0xfffff800037f5600 "-" chain 148: thread 100122 (pid 0, acpi_task_2) sleeping on 0xfffff800037f5600 "-" chain 149: thread 100124 (pid 0, CAM taskq) sleeping on 0xfffff800037f7200 "-" db:1:pho1> show mount 0xfffff80007bc4000 /dev/gpt/work on / (ufs) 0xfffff80007bc5000 devfs on /dev (devfs) 0xfffff8031d80d000 /dev/gpt/tmp on /tmp (ufs) 0xfffff8031db77000 /dev/md10.eli on /mnt (ufs) More info: show mount db:1:pho1> show bufqueues bqempty: 91609 Buf domain 0 freebufs 15001 lofreebufs 219 hifreebufs 329 bufspace 2281472 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 1 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 24 (106496) dirtyq count 1 (4096) wakeup 0 lim 25 CPU 5, 2, 5, 4, 5, 17, 9, 16, 3, 3, 22, 16, Locked buffers: 0 space 0 Total buffers: 132 space 2281472 Buf domain 1 freebufs 9873 lofreebufs 219 hifreebufs 329 bufspace 170479616 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 8 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 5140 (166576128) dirtyq count 8 (233472) wakeup 0 lim 25 CPU 17, 4, 15, 8, 1, 1, 14, 13, 9, 0, 21, 9, Locked buffers: 0 space 0 Total buffers: 5260 space 170479616 Buf domain 2 freebufs 13907 lofreebufs 219 hifreebufs 329 bufspace 38289408 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 6 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 1150 (36487168) dirtyq count 6 (139264) wakeup 0 lim 25 CPU 5, 6, 9, 2, 2, 5, 7, 8, 7, 9, 8, 2, Locked buffers: 0 space 0 Total buffers: 1226 space 38289408 Buf domain 3 freebufs 15003 lofreebufs 219 hifreebufs 329 bufspace 2183168 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 5 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 25 (131072) dirtyq count 5 (98304) wakeup 0 lim 25 CPU 1, 2, 5, 4, 7, 3, 12, 12, 5, 6, 23, 12, Locked buffers: 8 space 262144 Total buffers: 130 space 2183168 Buf domain 4 freebufs 14991 lofreebufs 219 hifreebufs 329 bufspace 2539520 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 1 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 50 (954368) dirtyq count 1 (4096) wakeup 0 lim 25 CPU 7, 3, 13, 0, 2, 6, 5, 15, 5, 18, 9, 8, Locked buffers: 0 space 0 Total buffers: 142 space 2539520 Buf domain 5 freebufs 7946 lofreebufs 219 hifreebufs 329 bufspace 235446272 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 6 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 7045 (230850560) dirtyq count 6 (167936) wakeup 0 lim 25 CPU 7, 14, 24, 11, 7, 18, 11, 18, 7, 0, 4, 15, Locked buffers: 0 space 0 Total buffers: 7187 space 235446272 Buf domain 6 freebufs 14932 lofreebufs 219 hifreebufs 329 bufspace 4562944 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 3 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 100 (2564096) dirtyq count 3 (16384) wakeup 0 lim 25 CPU 10, 13, 20, 4, 2, 6, 2, 0, 6, 4, 20, 11, Locked buffers: 0 space 0 Total buffers: 201 space 4562944 db:1:pho1> show page vm_cnt.v_free_count: 52753 vm_cnt.v_inactive_count: 54504 vm_cnt.v_active_count: 7770582 vm_cnt.v_laundry_count: 0 vm_cnt.v_wire_count: 249414 vm_cnt.v_free_reserved: 10649 vm_cnt.v_free_min: 51289 vm_cnt.v_free_target: 173209 vm_cnt.v_inactive_target: 259813 db:1:pho1> show pageq pq_free 52753 dom 0 page_cnt 8128423 free 52753 pq_act 7770582 pq_inact 54504 pq_laund 0 pq_unsw 0 db:0:pho> show freepages DOMAIN: 0 FREE LIST 0: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000000 | 000000 10 (004096K) | 000000 | 000000 09 (002048K) | 000000 | 000000 08 (001024K) | 000001 | 000000 07 (000512K) | 000024 | 000000 06 (000256K) | 000079 | 000000 05 (000128K) | 000139 | 000000 04 (000064K) | 000189 | 000000 03 (000032K) | 000229 | 000000 02 (000016K) | 000238 | 000159 01 (000008K) | 000265 | 000551 00 (000004K) | 000264 | 000891 FREE LIST 1: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000000 | 000000 10 (004096K) | 000000 | 000000 09 (002048K) | 000000 | 000000 08 (001024K) | 000000 | 000000 07 (000512K) | 000000 | 000000 06 (000256K) | 000000 | 000000 05 (000128K) | 000000 | 000000 04 (000064K) | 000000 | 000000 03 (000032K) | 000000 | 000000 02 (000016K) | 000000 | 000000 01 (000008K) | 000000 | 000000 00 (000004K) | 000000 | 000000 db:0:pho> acttrace Tracing command sendfile11 pid 2377 tid 100211 td 0xfffff800106c65a0 (CPU 8) kdb_enter() at kdb_enter+0x3b/frame 0xfffffe00b6976050 vpanic() at vpanic+0x1d1/frame 0xfffffe00b69760b0 panic() at panic+0x43/frame 0xfffffe00b6976110 vnode_pager_generic_getpages() at vnode_pager_generic_getpages+0xf58/frame 0xfffffe00b6976240 vnode_pager_local_getpages_async() at vnode_pager_local_getpages_async+0x4e/frame 0xfffffe00b6976260 VOP_GETPAGES_ASYNC_APV() at VOP_GETPAGES_ASYNC_APV+0x1f5/frame 0xfffffe00b69762d0 VOP_GETPAGES_ASYNC() at VOP_GETPAGES_ASYNC+0x79/frame 0xfffffe00b6976360 vnode_pager_getpages_async() at vnode_pager_getpages_async+0x78/frame 0xfffffe00b69763c0 sendfile_swapin() at sendfile_swapin+0x594/frame 0xfffffe00b69764d0 vn_sendfile() at vn_sendfile+0x995/frame 0xfffffe00b6976800 fo_sendfile() at fo_sendfile+0x8e/frame 0xfffffe00b69768a0 sendfile() at sendfile+0x1f2/frame 0xfffffe00b6976960 sys_sendfile() at sys_sendfile+0x24/frame 0xfffffe00b6976990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b6976ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b6976ab0 --- syscall (393, FreeBSD ELF64, sys_sendfile), rip = 0x80030744a, rsp = 0x7fffffffe688, rbp = 0x7fffffffe7e0 --- Tracing command swap pid 2375 tid 100253 td 0xfffff8031dc765a0 (CPU 0) cpustop_handler() at cpustop_handler+0x28/frame 0xffffffff8205fab0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xffffffff8205fad0 trap() at trap+0x48/frame 0xffffffff8205fbe0 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff8205fbe0 --- trap 0x13, rip = 0x203709, rsp = 0x7fffffffe640, rbp = 0x7fffffffe660 --- Tracing command swap pid 2374 tid 100252 td 0xfffff8031dc76000 (CPU 5) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006ba3df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006ba3e10 trap() at trap+0x48/frame 0xfffffe0006ba3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006ba3f20 --- trap 0x13, rip = 0x203709, rsp = 0x7fffffffe640, rbp = 0x7fffffffe660 --- Tracing command swap pid 2373 tid 100251 td 0xfffff8001072e000 (CPU 7) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006bbbdf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006bbbe10 trap() at trap+0x48/frame 0xfffffe0006bbbf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bbbf20 --- trap 0x13, rip = 0x203707, rsp = 0x7fffffffe640, rbp = 0x7fffffffe660 --- Tracing command swap pid 2372 tid 100250 td 0xfffff800104265a0 (CPU 1) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006b73df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006b73e10 trap() at trap+0x48/frame 0xfffffe0006b73f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b73f20 --- trap 0x13, rip = 0x203707, rsp = 0x7fffffffe640, rbp = 0x7fffffffe660 --- Tracing command swap pid 2369 tid 100174 td 0xfffff8001070d000 (CPU 9) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006bd3df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006bd3e10 trap() at trap+0x48/frame 0xfffffe0006bd3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bd3f20 --- trap 0x13, rip = 0x203707, rsp = 0x7fffffffe640, rbp = 0x7fffffffe660 --- Tracing command swap pid 2368 tid 100172 td 0xfffff80010bb35a0 (CPU 10) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006bdfdf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006bdfe10 trap() at trap+0x48/frame 0xfffffe0006bdff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bdff20 --- trap 0x13, rip = 0x203707, rsp = 0x7fffffffe640, rbp = 0x7fffffffe660 --- Tracing command swap pid 2365 tid 100142 td 0xfffff8000616c5a0 (CPU 2) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006b7fdf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006b7fe10 trap() at trap+0x48/frame 0xfffffe0006b7ff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b7ff20 --- trap 0x13, rip = 0x203709, rsp = 0x7fffffffe640, rbp = 0x7fffffffe660 --- Tracing command swap pid 2363 tid 100246 td 0xfffff800104c55a0 (CPU 3) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006b8bdf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006b8be10 trap() at trap+0x48/frame 0xfffffe0006b8bf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b8bf20 --- trap 0x13, rip = 0x203707, rsp = 0x7fffffffe640, rbp = 0x7fffffffe660 --- Tracing command swap pid 2361 tid 100244 td 0xfffff8031dc775a0 (CPU 6) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006bafdf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006bafe10 trap() at trap+0x48/frame 0xfffffe0006baff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006baff20 --- trap 0x13, rip = 0x203707, rsp = 0x7fffffffe640, rbp = 0x7fffffffe660 --- Tracing command swap pid 2356 tid 100168 td 0xfffff800104c7000 (CPU 4) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006b97df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006b97e10 trap() at trap+0x48/frame 0xfffffe0006b97f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b97f20 --- trap 0x13, rip = 0x203707, rsp = 0x7fffffffe640, rbp = 0x7fffffffe660 --- Tracing command pagedaemon pid 30 tid 100128 td 0xfffff8000782d5a0 (CPU 11) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006bebdf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006bebe10 trap() at trap+0x48/frame 0xfffffe0006bebf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bebf20 --- trap 0x13, rip = 0xffffffff80cb58d2, rsp = 0xfffffe0006f2e5e0, rbp = 0xfffffe0006f2e600 --- _vdrop() at _vdrop+0xc2/frame 0xfffffe0006f2e600 vm_page_remove() at vm_page_remove+0xdd/frame 0xfffffe0006f2e620 vm_page_free_prep() at vm_page_free_prep+0x105/frame 0xfffffe0006f2e640 vm_page_free_toq() at vm_page_free_toq+0x12/frame 0xfffffe0006f2e670 vm_pageout_worker() at vm_pageout_worker+0xa36/frame 0xfffffe0006f2ea30 vm_pageout() at vm_pageout+0x156/frame 0xfffffe0006f2ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f2eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f2eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db:0:pho> ps pid ppid pgrp uid state wmesg wchan cmd 2377 2376 2315 0 R+ CPU 8 sendfile11 2376 2315 2315 0 S+ sbwait 0xfffff80010d3d1d4 sendfile11 2375 2355 2315 0 R+ CPU 0 swap 2374 2355 2315 0 R+ CPU 5 swap 2373 2355 2315 0 R+ CPU 7 swap 2372 2355 2315 0 R+ CPU 1 swap 2371 2355 2315 0 R+ swap 2370 2355 2315 0 R+ swap 2369 2355 2315 0 R+ CPU 9 swap 2368 2355 2315 0 R+ CPU 10 swap 2367 2355 2315 0 R+ swap 2366 2355 2315 0 R+ swap 2365 2355 2315 0 R+ CPU 2 swap 2364 2355 2315 0 R+ swap 2363 2355 2315 0 R+ CPU 3 swap 2362 2355 2315 0 R+ swap 2361 2355 2315 0 R+ CPU 6 swap 2360 2355 2315 0 R+ swap 2359 2355 2315 0 R+ swap 2358 2355 2315 0 R+ swap 2357 2355 2315 0 R+ swap 2356 2355 2315 0 R+ CPU 4 swap 2355 2353 2315 0 S+ wait 0xfffff800103f1000 swap 2353 2315 2315 0 S+ nanslp 0xffffffff81e9cfb3 swap 2348 0 0 0 RL [g_eli[11] md10] 2347 0 0 0 DL geli:w 0xfffff80010364000 [g_eli[10] md10] 2346 0 0 0 DL geli:w 0xfffff80010364000 [g_eli[9] md10] 2345 0 0 0 DL geli:w 0xfffff80010364000 [g_eli[8] md10] 2344 0 0 0 DL geli:w 0xfffff80010364000 [g_eli[7] md10] 2343 0 0 0 DL geli:w 0xfffff80010364000 [g_eli[6] md10] 2342 0 0 0 DL geli:w 0xfffff80010364000 [g_eli[5] md10] 2341 0 0 0 DL geli:w 0xfffff80010364000 [g_eli[4] md10] 2340 0 0 0 DL geli:w 0xfffff80010364000 [g_eli[3] md10] 2339 0 0 0 DL geli:w 0xfffff80010364000 [g_eli[2] md10] 2338 0 0 0 DL geli:w 0xfffff80010364000 [g_eli[1] md10] 2337 0 0 0 DL geli:w 0xfffff80010364000 [g_eli[0] md10] 2331 0 0 0 DL mdwait 0xfffff8031d9f3000 [md10] 2315 2313 2315 0 S+ wait 0xfffff800106aa000 sh 2313 2309 2313 0 S+ wait 0xfffff80010705000 sh 2309 2306 2309 0 S+ pause 0xfffff80010deb0a8 csh 2306 1 2306 0 Ss+ wait 0xfffff8031d9c7000 login 2305 1 2305 0 Ss+ ttyin 0xfffff8000612b8b0 getty 2304 1 2304 0 Ss+ ttyin 0xfffff800079c7cb0 getty 2303 1 2303 0 Ss+ ttyin 0xfffff800079c80b0 getty 2302 1 2302 0 Ss+ ttyin 0xfffff800079c84b0 getty 2301 1 2301 0 Ss+ ttyin 0xfffff800079c88b0 getty 2300 1 2300 0 Ss+ ttyin 0xfffff800078764b0 getty 2299 1 2299 0 Ss+ ttyin 0xfffff800078758b0 getty 2298 1 2298 0 Ss+ ttyin 0xfffff80007874cb0 getty 2297 1 2297 0 Ss+ ttyin 0xfffff800078740b0 getty 2251 1 2251 0 Ss nanslp 0xffffffff81e9cfb3 cron 2247 1 2247 0 Ss select 0xfffff8000313e4c0 sshd 2168 1 2168 0 Ss rpcsvc 0xfffff80010e58360 NLM: master 2165 1 2165 0 Ss select 0xfffff800078463c0 rpc.statd 2119 1 2119 0 Ss nanslp 0xffffffff81e9cfb2 watchdogd 2040 1 2040 0 Ss select 0xfffff8031d23a740 rpcbind 2030 1 2030 0 Ss select 0xfffff80003985b40 syslogd 1959 1 1959 0 Ss select 0xfffff80007974040 devd 1954 1 1954 65 Ss select 0xfffff8031d23a7c0 dhclient 1910 1 1910 0 Ss select 0xfffff80003e669c0 dhclient 1907 1 1907 0 Ss select 0xfffff8000781a040 dhclient 34 0 0 0 DL syncer 0xffffffff8203f400 [syncer] 33 0 0 0 DL vlruwt 0xfffff800077e9000 [vnlru] 32 0 0 0 DL (threaded) [bufdaemon] 100130 D qsleep 0xffffffff8203e8a8 [bufdaemon] 100133 D - 0xffffffff81a0a880 [bufspacedaemon-0] 100134 D - 0xffffffff81a12b40 [bufspacedaemon-1] 100135 D - 0xffffffff81a1ae00 [bufspacedaemon-2] 100136 D - 0xffffffff81a230c0 [bufspacedaemon-3] 100137 D - 0xffffffff81a2b380 [bufspacedaemon-4] 100138 D - 0xffffffff81a33640 [bufspacedaemon-5] 100139 D - 0xffffffff81a3b900 [bufspacedaemon-6] 100151 D sdflush 0xfffff80007b120e8 [/ worker] 100202 D sdflush 0xfffff80006133ce8 [/tmp worker] 100241 D sdflush 0xfffff80007ae06e8 [/mnt worker] 31 0 0 0 DL psleep 0xffffffff8205b748 [vmdaemon] 30 0 0 0 RL (threaded) [pagedaemon] 100128 Run CPU 11 [dom0] 100131 D launds 0xffffffff820c5524 [laundry: dom0] 100132 D umarcl 0xffffffff80f17280 [uma] 29 0 0 0 DL idle 0xfffff800076f7000 [enc_daemon1] 28 0 0 0 DL idle 0xfffff800076f7530 [enc_daemon0] 27 0 0 0 DL - 0xffffffff81cfc628 [rand_harvestq] 26 0 0 0 DL waiting 0xffffffff820c3c00 [sctp_iterator] 25 0 0 0 DL - 0xffffffff8203e29c [soaiod4] 24 0 0 0 DL - 0xffffffff8203e29c [soaiod3] 23 0 0 0 DL - 0xffffffff8203e29c [soaiod2] 22 0 0 0 DL - 0xffffffff8203e29c [soaiod1] 21 0 0 0 DL (threaded) [usb] 100096 D - 0xfffffe009a1aa460 [usbus0] 100097 D - 0xfffffe009a1aa4b8 [usbus0] 100098 D - 0xfffffe009a1aa510 [usbus0] 100099 D - 0xfffffe009a1aa568 [usbus0] 100100 D - 0xfffffe009a1aa5c0 [usbus0] 100102 D - 0xfffffe0006dd6d10 [usbus1] 100103 D - 0xfffffe0006dd6d68 [usbus1] 100104 D - 0xfffffe0006dd6dc0 [usbus1] 100105 D - 0xfffffe0006dd6e18 [usbus1] 100106 D - 0xfffffe0006dd6e70 [usbus1] 100107 D - 0xfffffe00007f3d10 [usbus2] 100108 D - 0xfffffe00007f3d68 [usbus2] 100109 D - 0xfffffe00007f3dc0 [usbus2] 100110 D - 0xfffffe00007f3e18 [usbus2] 100111 D - 0xfffffe00007f3e70 [usbus2] 20 0 0 0 DL (threaded) [cam] 100091 D - 0xffffffff81bd78c0 [doneq0] 100092 D - 0xffffffff81bd7940 [doneq1] 100093 D - 0xffffffff81bd79c0 [doneq2] 100125 D - 0xffffffff81bd7788 [scanner] 19 0 0 0 DL crypto_ 0xfffff8000398f450 [crypto returns 11] 18 0 0 0 DL crypto_ 0xfffff8000398f3f0 [crypto returns 10] 17 0 0 0 DL crypto_ 0xfffff8000398f390 [crypto returns 9] 16 0 0 0 DL crypto_ 0xfffff8000398f330 [crypto returns 8] 15 0 0 0 DL crypto_ 0xfffff8000398f2d0 [crypto returns 7] 9 0 0 0 DL crypto_ 0xfffff8000398f270 [crypto returns 6] 8 0 0 0 DL crypto_ 0xfffff8000398f210 [crypto returns 5] 7 0 0 0 DL crypto_ 0xfffff8000398f1b0 [crypto returns 4] 6 0 0 0 DL crypto_ 0xfffff8000398f150 [crypto returns 3] 5 0 0 0 DL crypto_ 0xfffff8000398f0f0 [crypto returns 2] 4 0 0 0 DL crypto_ 0xfffff8000398f090 [crypto returns 1] 3 0 0 0 DL crypto_ 0xfffff8000398f030 [crypto returns 0] 2 0 0 0 DL crypto_ 0xffffffff82055d10 [crypto] 14 0 0 0 DL seqstat 0xfffff800038fac88 [sequencer 00] 13 0 0 0 DL (threaded) [geom] 100061 D - 0xffffffff820940c0 [g_event] 100062 D - 0xffffffff820940d0 [g_up] 100063 D - 0xffffffff820940b8 [g_down] 12 0 0 0 WL (threaded) [intr] 100016 I [swi5: fast taskq] 100020 I [swi6: task queue] 100021 I [swi6: Giant taskq] 100047 I [swi4: clock (0)] 100048 I [swi4: clock (1)] 100049 I [swi4: clock (2)] 100050 I [swi4: clock (3)] 100051 I [swi4: clock (4)] 100052 I [swi4: clock (5)] 100053 I [swi4: clock (6)] 100054 I [swi4: clock (7)] 100055 I [swi4: clock (8)] 100056 I [swi4: clock (9)] 100057 I [swi4: clock (10)] 100058 I [swi4: clock (11)] 100059 I [swi3: vm] 100060 I [swi1: netisr 0] 100094 I [irq56: ahci0] 100095 I [irq19: xhci0] 100101 I [irq18: ehci0 ehci1] 100112 I [irq71: ahci1] 100113 I [swi0: uart uart] 100180 I [irq72: ioat0] 100181 I [irq73: ioat1] 100182 I [irq74: ioat2] 100183 I [irq75: ioat3] 100184 I [irq76: ioat4] 100185 I [irq77: ioat5] 100186 I [irq78: ioat6] 100187 I [irq79: ioat7] 11 0 0 0 RL (threaded) [idle] 100003 CanRun [idle: cpu0] 100004 CanRun [idle: cpu1] 100005 CanRun [idle: cpu2] 100006 CanRun [idle: cpu3] 100007 CanRun [idle: cpu4] 100008 CanRun [idle: cpu5] 100009 CanRun [idle: cpu6] 100010 CanRun [idle: cpu7] 100011 CanRun [idle: cpu8] 100012 CanRun [idle: cpu9] 100013 CanRun [idle: cpu10] 100014 CanRun [idle: cpu11] 1 0 1 0 SLs wait 0xfffff800036b4530 [init] 10 0 0 0 DL audit_w 0xffffffff820c4920 [audit] 0 0 0 0 DLs (threaded) [kernel] 100000 D swapin 0xffffffff82094100 [swapper] 100015 D - 0xfffff800036a2e00 [thread taskq] 100017 D - 0xfffff80003693a00 [config_0] 100018 D - 0xfffff800036a2900 [kqueue_ctx taskq] 100019 D - 0xfffff800036a2700 [aiod_kick taskq] 100022 D - 0xfffff80003693700 [if_config_tqg_0] 100023 D - 0xfffff80003693600 [softirq_0] 100024 D - 0xfffff80003693500 [softirq_1] 100025 D - 0xfffff80003693400 [softirq_2] 100026 D - 0xfffff80003693300 [softirq_3] 100027 D - 0xfffff80003693200 [softirq_4] 100028 D - 0xfffff80003693100 [softirq_5] 100029 D - 0xfffff80003693000 [softirq_6] 100030 D - 0xfffff80003747100 [softirq_7] 100031 D - 0xfffff80003747000 [softirq_8] 100032 D - 0xfffff80003746e00 [softirq_9] 100033 D - 0xfffff80003746d00 [softirq_10] 100034 D - 0xfffff80003746c00 [softirq_11] 100035 D - 0xfffff800037d8200 [if_io_tqg_0] 100036 D - 0xfffff800037d8100 [if_io_tqg_1] 100037 D - 0xfffff800037d8000 [if_io_tqg_2] 100038 D - 0xfffff800037d7e00 [if_io_tqg_3] 100039 D - 0xfffff800037d7d00 [if_io_tqg_4] 100040 D - 0xfffff800037d7c00 [if_io_tqg_5] 100041 D - 0xfffff800037d7b00 [if_io_tqg_6] 100042 D - 0xfffff800037d7a00 [if_io_tqg_7] 100043 D - 0xfffff800037d7900 [if_io_tqg_8] 100044 D - 0xfffff800037d7800 [if_io_tqg_9] 100045 D - 0xfffff800037d7700 [if_io_tqg_10] 100046 D - 0xfffff800037d7600 [if_io_tqg_11] 100064 D - 0xfffff80003827a00 [firmware taskq] 100066 D - 0xfffff80003827300 [crypto_0] 100067 D - 0xfffff80003827300 [crypto_1] 100068 D - 0xfffff80003827300 [crypto_2] 100069 D - 0xfffff80003827300 [crypto_3] 100070 D - 0xfffff80003827300 [crypto_4] 100071 D - 0xfffff80003827300 [crypto_5] 100072 D - 0xfffff80003827300 [crypto_6] 100073 D - 0xfffff80003827300 [crypto_7] 100074 D - 0xfffff80003827300 [crypto_8] 100075 D - 0xfffff80003827300 [crypto_9] 100076 D - 0xfffff80003827300 [crypto_10] 100077 D - 0xfffff80003827300 [crypto_11] 100114 D - 0xfffff80003aeed00 [mca taskq] 100120 D - 0xfffff800037f5600 [acpi_task_0] 100121 D - 0xfffff800037f5600 [acpi_task_1] 100122 D - 0xfffff800037f5600 [acpi_task_2] 100124 D - 0xfffff800037f7200 [CAM taskq] db:0:pho> allt Tracing command sendfile11 pid 2377 tid 100211 td 0xfffff800106c65a0 (CPU 8) kdb_enter() at kdb_enter+0x3b/frame 0xfffffe00b6976050 vpanic() at vpanic+0x1d1/frame 0xfffffe00b69760b0 panic() at panic+0x43/frame 0xfffffe00b6976110 vnode_pager_generic_getpages() at vnode_pager_generic_getpages+0xf58/frame 0xfffffe00b6976240 vnode_pager_local_getpages_async() at vnode_pager_local_getpages_async+0x4e/frame 0xfffffe00b6976260 VOP_GETPAGES_ASYNC_APV() at VOP_GETPAGES_ASYNC_APV+0x1f5/frame 0xfffffe00b69762d0 VOP_GETPAGES_ASYNC() at VOP_GETPAGES_ASYNC+0x79/frame 0xfffffe00b6976360 vnode_pager_getpages_async() at vnode_pager_getpages_async+0x78/frame 0xfffffe00b69763c0 sendfile_swapin() at sendfile_swapin+0x594/frame 0xfffffe00b69764d0 vn_sendfile() at vn_sendfile+0x995/frame 0xfffffe00b6976800 fo_sendfile() at fo_sendfile+0x8e/frame 0xfffffe00b69768a0 sendfile() at sendfile+0x1f2/frame 0xfffffe00b6976960 sys_sendfile() at sys_sendfile+0x24/frame 0xfffffe00b6976990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b6976ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b6976ab0 --- syscall (393, FreeBSD ELF64, sys_sendfile), rip = 0x80030744a, rsp = 0x7fffffffe688, rbp = 0x7fffffffe7e0 --- Tracing command sendfile11 pid 2376 tid 100175 td 0xfffff8001070c5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b6897610 mi_switch() at mi_switch+0x174/frame 0xfffffe00b6897640 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b6897680 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00b68976e0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00b6897710 _sleep() at _sleep+0x2d0/frame 0xfffffe00b68977b0 sbwait() at sbwait+0x67/frame 0xfffffe00b68977d0 soreceive_generic() at soreceive_generic+0x292/frame 0xfffffe00b6897890 soreceive() at soreceive+0x59/frame 0xfffffe00b68978b0 dofileread() at dofileread+0x81/frame 0xfffffe00b6897900 kern_readv() at kern_readv+0x4a/frame 0xfffffe00b6897940 sys_read() at sys_read+0x86/frame 0xfffffe00b6897990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b6897ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b6897ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80044815a, rsp = 0x7fffffffe738, rbp = 0x7fffffffe770 --- Tracing command swap pid 2375 tid 100253 td 0xfffff8031dc765a0 (CPU 0) cpustop_handler() at cpustop_handler+0x28/frame 0xffffffff8205fab0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xffffffff8205fad0 trap() at trap+0x48/frame 0xffffffff8205fbe0 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff8205fbe0 --- trap 0x13, rip = 0x203709, rsp = 0x7fffffffe640, rbp = 0x7fffffffe660 --- Tracing command swap pid 2374 tid 100252 td 0xfffff8031dc76000 (CPU 5) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006ba3df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006ba3e10 trap() at trap+0x48/frame 0xfffffe0006ba3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006ba3f20 --- trap 0x13, rip = 0x203709, rsp = 0x7fffffffe640, rbp = 0x7fffffffe660 --- Tracing command swap pid 2373 tid 100251 td 0xfffff8001072e000 (CPU 7) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006bbbdf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006bbbe10 trap() at trap+0x48/frame 0xfffffe0006bbbf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bbbf20 --- trap 0x13, rip = 0x203707, rsp = 0x7fffffffe640, rbp = 0x7fffffffe660 --- Tracing command swap pid 2372 tid 100250 td 0xfffff800104265a0 (CPU 1) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006b73df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006b73e10 trap() at trap+0x48/frame 0xfffffe0006b73f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b73f20 --- trap 0x13, rip = 0x203707, rsp = 0x7fffffffe640, rbp = 0x7fffffffe660 --- Tracing command swap pid 2371 tid 100249 td 0xfffff80010465000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b8a2ca40 mi_switch() at mi_switch+0x174/frame 0xfffffe00b8a2ca70 ast() at ast+0x28d/frame 0xfffffe00b8a2cab0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe660 Tracing command swap pid 2370 tid 100248 td 0xfffff800104655a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b8a27670 mi_switch() at mi_switch+0x174/frame 0xfffffe00b8a276a0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b8a276e0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b8a27710 _sleep() at _sleep+0x2da/frame 0xfffffe00b8a277b0 vm_waitpfault() at vm_waitpfault+0x6b/frame 0xfffffe00b8a277d0 vm_fault_hold() at vm_fault_hold+0xb00/frame 0xfffffe00b8a27910 vm_fault() at vm_fault+0x60/frame 0xfffffe00b8a27950 trap_pfault() at trap_pfault+0x189/frame 0xfffffe00b8a279a0 trap() at trap+0x460/frame 0xfffffe00b8a27ab0 calltrap() at calltrap+0x8/frame 0xfffffe00b8a27ab0 --- trap 0xc, rip = 0x8004377b5, rsp = 0x7fffffffe2e8, rbp = 0x7fffffffe330 --- Tracing command swap pid 2369 tid 100174 td 0xfffff8001070d000 (CPU 9) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006bd3df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006bd3e10 trap() at trap+0x48/frame 0xfffffe0006bd3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bd3f20 --- trap 0x13, rip = 0x203707, rsp = 0x7fffffffe640, rbp = 0x7fffffffe660 --- Tracing command swap pid 2368 tid 100172 td 0xfffff80010bb35a0 (CPU 10) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006bdfdf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006bdfe10 trap() at trap+0x48/frame 0xfffffe0006bdff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bdff20 --- trap 0x13, rip = 0x203707, rsp = 0x7fffffffe640, rbp = 0x7fffffffe660 --- Tracing command swap pid 2367 tid 100198 td 0xfffff80010bb25a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b693a670 mi_switch() at mi_switch+0x174/frame 0xfffffe00b693a6a0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b693a6e0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b693a710 _sleep() at _sleep+0x2da/frame 0xfffffe00b693a7b0 vm_waitpfault() at vm_waitpfault+0x6b/frame 0xfffffe00b693a7d0 vm_fault_hold() at vm_fault_hold+0xb00/frame 0xfffffe00b693a910 vm_fault() at vm_fault+0x60/frame 0xfffffe00b693a950 trap_pfault() at trap_pfault+0x189/frame 0xfffffe00b693a9a0 trap() at trap+0x460/frame 0xfffffe00b693aab0 calltrap() at calltrap+0x8/frame 0xfffffe00b693aab0 --- trap 0xc, rip = 0x8004377b5, rsp = 0x7fffffffe2e8, rbp = 0x7fffffffe330 --- Tracing command swap pid 2366 tid 100146 td 0xfffff80010426000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b67f8670 mi_switch() at mi_switch+0x174/frame 0xfffffe00b67f86a0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b67f86e0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b67f8710 _sleep() at _sleep+0x2da/frame 0xfffffe00b67f87b0 vm_waitpfault() at vm_waitpfault+0x6b/frame 0xfffffe00b67f87d0 vm_fault_hold() at vm_fault_hold+0xb00/frame 0xfffffe00b67f8910 vm_fault() at vm_fault+0x60/frame 0xfffffe00b67f8950 trap_pfault() at trap_pfault+0x189/frame 0xfffffe00b67f89a0 trap() at trap+0x460/frame 0xfffffe00b67f8ab0 calltrap() at calltrap+0x8/frame 0xfffffe00b67f8ab0 --- trap 0xc, rip = 0x8004377b5, rsp = 0x7fffffffe2e8, rbp = 0x7fffffffe330 --- Tracing command swap pid 2365 tid 100142 td 0xfffff8000616c5a0 (CPU 2) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006b7fdf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006b7fe10 trap() at trap+0x48/frame 0xfffffe0006b7ff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b7ff20 --- trap 0x13, rip = 0x203709, rsp = 0x7fffffffe640, rbp = 0x7fffffffe660 --- Tracing command swap pid 2364 tid 100247 td 0xfffff800104c5000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b8a22670 mi_switch() at mi_switch+0x174/frame 0xfffffe00b8a226a0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b8a226e0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b8a22710 _sleep() at _sleep+0x2da/frame 0xfffffe00b8a227b0 vm_waitpfault() at vm_waitpfault+0x6b/frame 0xfffffe00b8a227d0 vm_fault_hold() at vm_fault_hold+0xb00/frame 0xfffffe00b8a22910 vm_fault() at vm_fault+0x60/frame 0xfffffe00b8a22950 trap_pfault() at trap_pfault+0x189/frame 0xfffffe00b8a229a0 trap() at trap+0x460/frame 0xfffffe00b8a22ab0 calltrap() at calltrap+0x8/frame 0xfffffe00b8a22ab0 --- trap 0xc, rip = 0x8004377b5, rsp = 0x7fffffffe2e8, rbp = 0x7fffffffe330 --- Tracing command swap pid 2363 tid 100246 td 0xfffff800104c55a0 (CPU 3) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006b8bdf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006b8be10 trap() at trap+0x48/frame 0xfffffe0006b8bf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b8bf20 --- trap 0x13, rip = 0x203707, rsp = 0x7fffffffe640, rbp = 0x7fffffffe660 --- Tracing command swap pid 2362 tid 100245 td 0xfffff800104c6000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b8a18670 mi_switch() at mi_switch+0x174/frame 0xfffffe00b8a186a0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b8a186e0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b8a18710 _sleep() at _sleep+0x2da/frame 0xfffffe00b8a187b0 vm_waitpfault() at vm_waitpfault+0x6b/frame 0xfffffe00b8a187d0 vm_fault_hold() at vm_fault_hold+0xb00/frame 0xfffffe00b8a18910 vm_fault() at vm_fault+0x60/frame 0xfffffe00b8a18950 trap_pfault() at trap_pfault+0x189/frame 0xfffffe00b8a189a0 trap() at trap+0x460/frame 0xfffffe00b8a18ab0 calltrap() at calltrap+0x8/frame 0xfffffe00b8a18ab0 --- trap 0xc, rip = 0x8004377b5, rsp = 0x7fffffffe2e8, rbp = 0x7fffffffe330 --- Tracing command swap pid 2361 tid 100244 td 0xfffff8031dc775a0 (CPU 6) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006bafdf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006bafe10 trap() at trap+0x48/frame 0xfffffe0006baff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006baff20 --- trap 0x13, rip = 0x203707, rsp = 0x7fffffffe640, rbp = 0x7fffffffe660 --- Tracing command swap pid 2360 tid 100243 td 0xfffff8031dc785a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b8a0ea30 mi_switch() at mi_switch+0x174/frame 0xfffffe00b8a0ea60 critical_exit_preempt() at critical_exit_preempt+0x66/frame 0xfffffe00b8a0ea80 ipi_bitmap_handler() at ipi_bitmap_handler+0x9d/frame 0xfffffe00b8a0eab0 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0xb0/frame 0xfffffe00b8a0eab0 --- interrupt, rip = 0x203707, rsp = 0x7fffffffe640, rbp = 0x7fffffffe660 --- Tracing command swap pid 2359 tid 100242 td 0xfffff8031dc7a000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b8a09670 mi_switch() at mi_switch+0x174/frame 0xfffffe00b8a096a0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b8a096e0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b8a09710 _sleep() at _sleep+0x2da/frame 0xfffffe00b8a097b0 vm_waitpfault() at vm_waitpfault+0x6b/frame 0xfffffe00b8a097d0 vm_fault_hold() at vm_fault_hold+0xb00/frame 0xfffffe00b8a09910 vm_fault() at vm_fault+0x60/frame 0xfffffe00b8a09950 trap_pfault() at trap_pfault+0x189/frame 0xfffffe00b8a099a0 trap() at trap+0x460/frame 0xfffffe00b8a09ab0 calltrap() at calltrap+0x8/frame 0xfffffe00b8a09ab0 --- trap 0xc, rip = 0x8004377b5, rsp = 0x7fffffffe2e8, rbp = 0x7fffffffe330 --- Tracing command swap pid 2358 tid 100190 td 0xfffff800104c65a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b6912670 mi_switch() at mi_switch+0x174/frame 0xfffffe00b69126a0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b69126e0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b6912710 _sleep() at _sleep+0x2da/frame 0xfffffe00b69127b0 vm_waitpfault() at vm_waitpfault+0x6b/frame 0xfffffe00b69127d0 vm_fault_hold() at vm_fault_hold+0xb00/frame 0xfffffe00b6912910 vm_fault() at vm_fault+0x60/frame 0xfffffe00b6912950 trap_pfault() at trap_pfault+0x189/frame 0xfffffe00b69129a0 trap() at trap+0x460/frame 0xfffffe00b6912ab0 calltrap() at calltrap+0x8/frame 0xfffffe00b6912ab0 --- trap 0xc, rip = 0x8004377b5, rsp = 0x7fffffffe2e8, rbp = 0x7fffffffe330 --- Tracing command swap pid 2357 tid 100162 td 0xfffff800106d45a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b684a670 mi_switch() at mi_switch+0x174/frame 0xfffffe00b684a6a0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b684a6e0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b684a710 _sleep() at _sleep+0x2da/frame 0xfffffe00b684a7b0 vm_waitpfault() at vm_waitpfault+0x6b/frame 0xfffffe00b684a7d0 vm_fault_hold() at vm_fault_hold+0xb00/frame 0xfffffe00b684a910 vm_fault() at vm_fault+0x60/frame 0xfffffe00b684a950 trap_pfault() at trap_pfault+0x189/frame 0xfffffe00b684a9a0 trap() at trap+0x460/frame 0xfffffe00b684aab0 calltrap() at calltrap+0x8/frame 0xfffffe00b684aab0 --- trap 0xc, rip = 0x8004377b5, rsp = 0x7fffffffe2e8, rbp = 0x7fffffffe330 --- Tracing command swap pid 2356 tid 100168 td 0xfffff800104c7000 (CPU 4) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006b97df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006b97e10 trap() at trap+0x48/frame 0xfffffe0006b97f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b97f20 --- trap 0x13, rip = 0x203707, rsp = 0x7fffffffe640, rbp = 0x7fffffffe660 --- Tracing command swap pid 2355 tid 100206 td 0xfffff800103fb000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b695d570 mi_switch() at mi_switch+0x174/frame 0xfffffe00b695d5a0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b695d5e0 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00b695d640 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00b695d670 _sleep() at _sleep+0x2d0/frame 0xfffffe00b695d710 kern_wait6() at kern_wait6+0x48f/frame 0xfffffe00b695d7a0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00b695d990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b695dab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b695dab0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c61ca, rsp = 0x7fffffffe688, rbp = 0x7fffffffe6c0 --- Tracing command swap pid 2353 tid 100230 td 0xfffff80010bb15a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b6999710 mi_switch() at mi_switch+0x174/frame 0xfffffe00b6999740 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b6999780 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00b69997e0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00b6999820 _sleep() at _sleep+0x265/frame 0xfffffe00b69998c0 kern_clock_nanosleep() at kern_clock_nanosleep+0x207/frame 0xfffffe00b6999950 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe00b6999990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b6999ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b6999ab0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003c634a, rsp = 0x7fffffffe678, rbp = 0x7fffffffe6c0 --- Tracing command g_eli[11] md10 pid 2348 tid 100240 td 0xfffff8031dbbd000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b69fb8d0 mi_switch() at mi_switch+0x174/frame 0xfffffe00b69fb900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b69fb940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b69fb970 _sleep() at _sleep+0x2da/frame 0xfffffe00b69fba10 g_eli_worker() at g_eli_worker+0x42e/frame 0xfffffe00b69fba70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b69fbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b69fbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command g_eli[10] md10 pid 2347 tid 100239 td 0xfffff8031dbbd5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b69f68d0 mi_switch() at mi_switch+0x174/frame 0xfffffe00b69f6900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b69f6940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b69f6970 _sleep() at _sleep+0x2da/frame 0xfffffe00b69f6a10 g_eli_worker() at g_eli_worker+0x42e/frame 0xfffffe00b69f6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b69f6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b69f6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command g_eli[9] md10 pid 2346 tid 100238 td 0xfffff8031dbbe000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b69f18d0 mi_switch() at mi_switch+0x174/frame 0xfffffe00b69f1900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b69f1940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b69f1970 _sleep() at _sleep+0x2da/frame 0xfffffe00b69f1a10 g_eli_worker() at g_eli_worker+0x42e/frame 0xfffffe00b69f1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b69f1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b69f1ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command g_eli[8] md10 pid 2345 tid 100237 td 0xfffff8031dbbe5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b69ec8d0 mi_switch() at mi_switch+0x174/frame 0xfffffe00b69ec900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b69ec940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b69ec970 _sleep() at _sleep+0x2da/frame 0xfffffe00b69eca10 g_eli_worker() at g_eli_worker+0x42e/frame 0xfffffe00b69eca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b69ecab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b69ecab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command g_eli[7] md10 pid 2344 tid 100236 td 0xfffff80236176000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b69e78d0 mi_switch() at mi_switch+0x174/frame 0xfffffe00b69e7900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b69e7940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b69e7970 _sleep() at _sleep+0x2da/frame 0xfffffe00b69e7a10 g_eli_worker() at g_eli_worker+0x42e/frame 0xfffffe00b69e7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b69e7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b69e7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command g_eli[6] md10 pid 2343 tid 100155 td 0xfffff8000370f5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b68278d0 mi_switch() at mi_switch+0x174/frame 0xfffffe00b6827900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b6827940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b6827970 _sleep() at _sleep+0x2da/frame 0xfffffe00b6827a10 g_eli_worker() at g_eli_worker+0x42e/frame 0xfffffe00b6827a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b6827ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b6827ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command g_eli[5] md10 pid 2342 tid 100235 td 0xfffff802361765a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b69e28d0 mi_switch() at mi_switch+0x174/frame 0xfffffe00b69e2900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b69e2940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b69e2970 _sleep() at _sleep+0x2da/frame 0xfffffe00b69e2a10 g_eli_worker() at g_eli_worker+0x42e/frame 0xfffffe00b69e2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b69e2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b69e2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command g_eli[4] md10 pid 2341 tid 100234 td 0xfffff80236177000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b69dd8d0 mi_switch() at mi_switch+0x174/frame 0xfffffe00b69dd900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b69dd940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b69dd970 _sleep() at _sleep+0x2da/frame 0xfffffe00b69dda10 g_eli_worker() at g_eli_worker+0x42e/frame 0xfffffe00b69dda70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b69ddab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b69ddab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command g_eli[3] md10 pid 2340 tid 100233 td 0xfffff802361775a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b69d88d0 mi_switch() at mi_switch+0x174/frame 0xfffffe00b69d8900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b69d8940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b69d8970 _sleep() at _sleep+0x2da/frame 0xfffffe00b69d8a10 g_eli_worker() at g_eli_worker+0x42e/frame 0xfffffe00b69d8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b69d8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b69d8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command g_eli[2] md10 pid 2339 tid 100232 td 0xfffff80236178000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b69d38d0 mi_switch() at mi_switch+0x174/frame 0xfffffe00b69d3900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b69d3940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b69d3970 _sleep() at _sleep+0x2da/frame 0xfffffe00b69d3a10 g_eli_worker() at g_eli_worker+0x42e/frame 0xfffffe00b69d3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b69d3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b69d3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command g_eli[1] md10 pid 2338 tid 100231 td 0xfffff802361785a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b69ce8d0 mi_switch() at mi_switch+0x174/frame 0xfffffe00b69ce900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b69ce940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b69ce970 _sleep() at _sleep+0x2da/frame 0xfffffe00b69cea10 g_eli_worker() at g_eli_worker+0x42e/frame 0xfffffe00b69cea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b69ceab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b69ceab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command g_eli[0] md10 pid 2337 tid 100217 td 0xfffff802361795a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b69948d0 mi_switch() at mi_switch+0x174/frame 0xfffffe00b6994900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b6994940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b6994970 _sleep() at _sleep+0x2da/frame 0xfffffe00b6994a10 g_eli_worker() at g_eli_worker+0x42e/frame 0xfffffe00b6994a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b6994ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b6994ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command md10 pid 2331 tid 100212 td 0xfffff800100e95a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b697b8d0 mi_switch() at mi_switch+0x174/frame 0xfffffe00b697b900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b697b940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b697b970 _sleep() at _sleep+0x2da/frame 0xfffffe00b697ba10 md_kthread() at md_kthread+0x1e5/frame 0xfffffe00b697ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b697bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b697bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sh pid 2315 tid 100188 td 0xfffff80010732000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b6908570 mi_switch() at mi_switch+0x174/frame 0xfffffe00b69085a0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b69085e0 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00b6908640 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00b6908670 _sleep() at _sleep+0x2d0/frame 0xfffffe00b6908710 kern_wait6() at kern_wait6+0x48f/frame 0xfffffe00b69087a0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00b6908990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b6908ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b6908ab0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80040d1ca, rsp = 0x7fffffffe708, rbp = 0x7fffffffe7c0 --- Tracing command sh pid 2313 tid 100207 td 0xfffff800103fb5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b6962570 mi_switch() at mi_switch+0x174/frame 0xfffffe00b69625a0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b69625e0 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00b6962640 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00b6962670 _sleep() at _sleep+0x2d0/frame 0xfffffe00b6962710 kern_wait6() at kern_wait6+0x48f/frame 0xfffffe00b69627a0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00b6962990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b6962ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b6962ab0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80040d1ca, rsp = 0x7fffffffe7c8, rbp = 0x7fffffffe880 --- Tracing command csh pid 2309 tid 100197 td 0xfffff802360365a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b6935760 mi_switch() at mi_switch+0x174/frame 0xfffffe00b6935790 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b69357d0 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00b6935830 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00b6935860 _sleep() at _sleep+0x2d0/frame 0xfffffe00b6935900 kern_sigsuspend() at kern_sigsuspend+0xb4/frame 0xfffffe00b6935960 sys_sigsuspend() at sys_sigsuspend+0x31/frame 0xfffffe00b6935990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b6935ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b6935ab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x80049124a, rsp = 0x7fffffffe0f8, rbp = 0x7fffffffe180 --- Tracing command login pid 2306 tid 100216 td 0xfffff80010b7c5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b698f570 mi_switch() at mi_switch+0x174/frame 0xfffffe00b698f5a0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b698f5e0 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00b698f640 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00b698f670 _sleep() at _sleep+0x2d0/frame 0xfffffe00b698f710 kern_wait6() at kern_wait6+0x48f/frame 0xfffffe00b698f7a0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00b698f990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b698fab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b698fab0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003f31ca, rsp = 0x7fffffffeb48, rbp = 0x7fffffffed00 --- Tracing command getty pid 2305 tid 100215 td 0xfffff80010b7d000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b698a630 mi_switch() at mi_switch+0x174/frame 0xfffffe00b698a660 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b698a6a0 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00b698a700 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00b698a730 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00b698a790 tty_wait() at tty_wait+0x42/frame 0xfffffe00b698a7c0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00b698a820 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00b698a850 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00b698a8b0 dofileread() at dofileread+0x81/frame 0xfffffe00b698a900 kern_readv() at kern_readv+0x4a/frame 0xfffffe00b698a940 sys_read() at sys_read+0x86/frame 0xfffffe00b698a990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b698aab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b698aab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80043815a, rsp = 0x7fffffffebc8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2304 tid 100214 td 0xfffff80010b7d5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b6985630 mi_switch() at mi_switch+0x174/frame 0xfffffe00b6985660 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b69856a0 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00b6985700 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00b6985730 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00b6985790 tty_wait() at tty_wait+0x42/frame 0xfffffe00b69857c0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00b6985820 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00b6985850 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00b69858b0 dofileread() at dofileread+0x81/frame 0xfffffe00b6985900 kern_readv() at kern_readv+0x4a/frame 0xfffffe00b6985940 sys_read() at sys_read+0x86/frame 0xfffffe00b6985990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b6985ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b6985ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80043815a, rsp = 0x7fffffffebc8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2303 tid 100213 td 0xfffff80010b7e000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b6980630 mi_switch() at mi_switch+0x174/frame 0xfffffe00b6980660 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b69806a0 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00b6980700 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00b6980730 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00b6980790 tty_wait() at tty_wait+0x42/frame 0xfffffe00b69807c0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00b6980820 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00b6980850 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00b69808b0 dofileread() at dofileread+0x81/frame 0xfffffe00b6980900 kern_readv() at kern_readv+0x4a/frame 0xfffffe00b6980940 sys_read() at sys_read+0x86/frame 0xfffffe00b6980990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b6980ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b6980ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80043815a, rsp = 0x7fffffffebc8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2302 tid 100169 td 0xfffff80010b7e5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b6879630 mi_switch() at mi_switch+0x174/frame 0xfffffe00b6879660 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b68796a0 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00b6879700 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00b6879730 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00b6879790 tty_wait() at tty_wait+0x42/frame 0xfffffe00b68797c0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00b6879820 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00b6879850 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00b68798b0 dofileread() at dofileread+0x81/frame 0xfffffe00b6879900 kern_readv() at kern_readv+0x4a/frame 0xfffffe00b6879940 sys_read() at sys_read+0x86/frame 0xfffffe00b6879990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b6879ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b6879ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80043815a, rsp = 0x7fffffffebc8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2301 tid 100170 td 0xfffff8000782a000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b687e630 mi_switch() at mi_switch+0x174/frame 0xfffffe00b687e660 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b687e6a0 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00b687e700 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00b687e730 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00b687e790 tty_wait() at tty_wait+0x42/frame 0xfffffe00b687e7c0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00b687e820 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00b687e850 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00b687e8b0 dofileread() at dofileread+0x81/frame 0xfffffe00b687e900 kern_readv() at kern_readv+0x4a/frame 0xfffffe00b687e940 sys_read() at sys_read+0x86/frame 0xfffffe00b687e990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b687eab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b687eab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80043815a, rsp = 0x7fffffffebc8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2300 tid 100191 td 0xfffff80003eef5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b6917630 mi_switch() at mi_switch+0x174/frame 0xfffffe00b6917660 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b69176a0 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00b6917700 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00b6917730 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00b6917790 tty_wait() at tty_wait+0x42/frame 0xfffffe00b69177c0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00b6917820 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00b6917850 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00b69178b0 dofileread() at dofileread+0x81/frame 0xfffffe00b6917900 kern_readv() at kern_readv+0x4a/frame 0xfffffe00b6917940 sys_read() at sys_read+0x86/frame 0xfffffe00b6917990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b6917ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b6917ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80043815a, rsp = 0x7fffffffebc8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2299 tid 100161 td 0xfffff800106d5000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b6845630 mi_switch() at mi_switch+0x174/frame 0xfffffe00b6845660 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b68456a0 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00b6845700 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00b6845730 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00b6845790 tty_wait() at tty_wait+0x42/frame 0xfffffe00b68457c0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00b6845820 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00b6845850 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00b68458b0 dofileread() at dofileread+0x81/frame 0xfffffe00b6845900 kern_readv() at kern_readv+0x4a/frame 0xfffffe00b6845940 sys_read() at sys_read+0x86/frame 0xfffffe00b6845990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b6845ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b6845ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80043815a, rsp = 0x7fffffffebc8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2298 tid 100152 td 0xfffff800100ea000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b6818630 mi_switch() at mi_switch+0x174/frame 0xfffffe00b6818660 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b68186a0 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00b6818700 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00b6818730 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00b6818790 tty_wait() at tty_wait+0x42/frame 0xfffffe00b68187c0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00b6818820 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00b6818850 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00b68188b0 dofileread() at dofileread+0x81/frame 0xfffffe00b6818900 kern_readv() at kern_readv+0x4a/frame 0xfffffe00b6818940 sys_read() at sys_read+0x86/frame 0xfffffe00b6818990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b6818ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b6818ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80043815a, rsp = 0x7fffffffebc8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2297 tid 100199 td 0xfffff80010bb2000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b693f630 mi_switch() at mi_switch+0x174/frame 0xfffffe00b693f660 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b693f6a0 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00b693f700 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00b693f730 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00b693f790 tty_wait() at tty_wait+0x42/frame 0xfffffe00b693f7c0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00b693f820 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00b693f850 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00b693f8b0 dofileread() at dofileread+0x81/frame 0xfffffe00b693f900 kern_readv() at kern_readv+0x4a/frame 0xfffffe00b693f940 sys_read() at sys_read+0x86/frame 0xfffffe00b693f990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b693fab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b693fab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80043815a, rsp = 0x7fffffffebc8, rbp = 0x7fffffffec10 --- Tracing command cron pid 2251 tid 100204 td 0xfffff800104245a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b6953710 mi_switch() at mi_switch+0x174/frame 0xfffffe00b6953740 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b6953780 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00b69537e0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00b6953820 _sleep() at _sleep+0x265/frame 0xfffffe00b69538c0 kern_clock_nanosleep() at kern_clock_nanosleep+0x207/frame 0xfffffe00b6953950 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe00b6953990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b6953ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b6953ab0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003db34a, rsp = 0x7fffffffead8, rbp = 0x7fffffffecc0 --- Tracing command sshd pid 2247 tid 100203 td 0xfffff802360355a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b694e5d0 mi_switch() at mi_switch+0x174/frame 0xfffffe00b694e600 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b694e640 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00b694e6a0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00b694e6d0 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00b694e730 seltdwait() at seltdwait+0x8d/frame 0xfffffe00b694e770 kern_select() at kern_select+0x953/frame 0xfffffe00b694e950 sys_select() at sys_select+0x56/frame 0xfffffe00b694e990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b694eab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b694eab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80090b13a, rsp = 0x7fffffffe668, rbp = 0x7fffffffe6a0 --- Tracing command rpc.lockd pid 2168 tid 100166 td 0xfffff800104255a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b685e4f0 mi_switch() at mi_switch+0x174/frame 0xfffffe00b685e520 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b685e560 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00b685e5c0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00b685e600 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x185/frame 0xfffffe00b685e670 svc_run_internal() at svc_run_internal+0xa94/frame 0xfffffe00b685e7b0 svc_run() at svc_run+0x1ee/frame 0xfffffe00b685e810 sys_nlm_syscall() at sys_nlm_syscall+0x75b/frame 0xfffffe00b685e990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b685eab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b685eab0 --- syscall (154, FreeBSD ELF64, sys_nlm_syscall), rip = 0x8002f60aa, rsp = 0x7fffffffec08, rbp = 0x7fffffffecc0 --- Tracing command rpc.statd pid 2165 tid 100196 td 0xfffff80010731000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b69305b0 mi_switch() at mi_switch+0x174/frame 0xfffffe00b69305e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b6930620 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00b6930680 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00b69306c0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x185/frame 0xfffffe00b6930730 seltdwait() at seltdwait+0x6b/frame 0xfffffe00b6930770 kern_select() at kern_select+0x953/frame 0xfffffe00b6930950 sys_select() at sys_select+0x56/frame 0xfffffe00b6930990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b6930ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b6930ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80042c13a, rsp = 0x7fffffffea98, rbp = 0x7fffffffebe0 --- Tracing command watchdogd pid 2119 tid 100150 td 0xfffff800104c75a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b680e710 mi_switch() at mi_switch+0x174/frame 0xfffffe00b680e740 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b680e780 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00b680e7e0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00b680e820 _sleep() at _sleep+0x265/frame 0xfffffe00b680e8c0 kern_clock_nanosleep() at kern_clock_nanosleep+0x207/frame 0xfffffe00b680e950 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe00b680e990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b680eab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b680eab0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003c434a, rsp = 0x7fffffffea98, rbp = 0x7fffffffeae0 --- Tracing command rpcbind pid 2040 tid 100153 td 0xfffff800103fc000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b681d630 mi_switch() at mi_switch+0x174/frame 0xfffffe00b681d660 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b681d6a0 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00b681d700 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00b681d740 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x185/frame 0xfffffe00b681d7b0 seltdwait() at seltdwait+0x6b/frame 0xfffffe00b681d7f0 kern_poll() at kern_poll+0x439/frame 0xfffffe00b681d970 sys_poll() at sys_poll+0x50/frame 0xfffffe00b681d990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b681dab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b681dab0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8003d28ca, rsp = 0x7fffffffca38, rbp = 0x7fffffffec50 --- Tracing command syslogd pid 2030 tid 100194 td 0xfffff800106d4000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b69265d0 mi_switch() at mi_switch+0x174/frame 0xfffffe00b6926600 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b6926640 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00b69266a0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00b69266d0 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00b6926730 seltdwait() at seltdwait+0x8d/frame 0xfffffe00b6926770 kern_select() at kern_select+0x953/frame 0xfffffe00b6926950 sys_select() at sys_select+0x56/frame 0xfffffe00b6926990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b6926ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b6926ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80043d13a, rsp = 0x7fffffffe6a8, rbp = 0x7fffffffecc0 --- Tracing command devd pid 1959 tid 100164 td 0xfffff80010733000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b68545b0 mi_switch() at mi_switch+0x174/frame 0xfffffe00b68545e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b6854620 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00b6854680 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00b68546c0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x185/frame 0xfffffe00b6854730 seltdwait() at seltdwait+0x6b/frame 0xfffffe00b6854770 kern_select() at kern_select+0x953/frame 0xfffffe00b6854950 sys_select() at sys_select+0x56/frame 0xfffffe00b6854990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b6854ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b6854ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x2fd3ca, rsp = 0x7fffffffcb48, rbp = 0x7fffffffece0 --- Tracing command dhclient pid 1954 tid 100177 td 0xfffff8000616b000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b68a1630 mi_switch() at mi_switch+0x174/frame 0xfffffe00b68a1660 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b68a16a0 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00b68a1700 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00b68a1740 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x185/frame 0xfffffe00b68a17b0 seltdwait() at seltdwait+0x6b/frame 0xfffffe00b68a17f0 kern_poll() at kern_poll+0x439/frame 0xfffffe00b68a1970 sys_poll() at sys_poll+0x50/frame 0xfffffe00b68a1990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b68a1ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b68a1ab0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8003f58ca, rsp = 0x7fffffffec28, rbp = 0x7fffffffec60 --- Tracing command dhclient pid 1910 tid 100165 td 0xfffff800107325a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b6859650 mi_switch() at mi_switch+0x174/frame 0xfffffe00b6859680 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b68596c0 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00b6859720 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00b6859750 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00b68597b0 seltdwait() at seltdwait+0x8d/frame 0xfffffe00b68597f0 kern_poll() at kern_poll+0x439/frame 0xfffffe00b6859970 sys_poll() at sys_poll+0x50/frame 0xfffffe00b6859990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b6859ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b6859ab0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8003f58ca, rsp = 0x7fffffffec38, rbp = 0x7fffffffec60 --- Tracing command dhclient pid 1907 tid 100149 td 0xfffff800061745a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b68095d0 mi_switch() at mi_switch+0x174/frame 0xfffffe00b6809600 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b6809640 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe00b68096a0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00b68096d0 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00b6809730 seltdwait() at seltdwait+0x8d/frame 0xfffffe00b6809770 kern_select() at kern_select+0x953/frame 0xfffffe00b6809950 sys_select() at sys_select+0x56/frame 0xfffffe00b6809990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe00b6809ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00b6809ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80045413a, rsp = 0x7fffffffe9e8, rbp = 0x7fffffffeaa0 --- Tracing command syncer pid 34 tid 100141 td 0xfffff8000782b5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00a3dfd8c0 mi_switch() at mi_switch+0x174/frame 0xfffffe00a3dfd8f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a3dfd930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00a3dfd970 _cv_timedwait_sbt() at _cv_timedwait_sbt+0x185/frame 0xfffffe00a3dfd9e0 sched_sync() at sched_sync+0x609/frame 0xfffffe00a3dfda70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a3dfdab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a3dfdab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command vnlru pid 33 tid 100140 td 0xfffff8000782c000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00a3df8890 mi_switch() at mi_switch+0x174/frame 0xfffffe00a3df88c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a3df8900 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00a3df8940 _sleep() at _sleep+0x279/frame 0xfffffe00a3df89e0 vnlru_proc() at vnlru_proc+0x94/frame 0xfffffe00a3df8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a3df8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a3df8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100130 td 0xfffff8000782c5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006f448d0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006f44900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006f44940 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006f44980 _sleep() at _sleep+0x279/frame 0xfffffe0006f44a20 buf_daemon() at buf_daemon+0xc8/frame 0xfffffe0006f44a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f44ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f44ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100133 td 0xfffff80007c155a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006ff38c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ff38f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006ff3930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006ff3970 _sleep() at _sleep+0x279/frame 0xfffffe0006ff3a10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe0006ff3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ff3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ff3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100134 td 0xfffff80007c15000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006ff88c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ff88f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006ff8930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006ff8970 _sleep() at _sleep+0x279/frame 0xfffffe0006ff8a10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe0006ff8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ff8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ff8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100135 td 0xfffff80007c135a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006ffd8c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ffd8f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006ffd930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006ffd970 _sleep() at _sleep+0x279/frame 0xfffffe0006ffda10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe0006ffda70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ffdab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ffdab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100136 td 0xfffff80007c13000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00a3c648c0 mi_switch() at mi_switch+0x174/frame 0xfffffe00a3c648f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a3c64930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00a3c64970 _sleep() at _sleep+0x279/frame 0xfffffe00a3c64a10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe00a3c64a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a3c64ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a3c64ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100137 td 0xfffff80007c115a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00a3ca98c0 mi_switch() at mi_switch+0x174/frame 0xfffffe00a3ca98f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a3ca9930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00a3ca9970 _sleep() at _sleep+0x279/frame 0xfffffe00a3ca9a10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe00a3ca9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a3ca9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a3ca9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100138 td 0xfffff80007c11000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00a3cae8c0 mi_switch() at mi_switch+0x174/frame 0xfffffe00a3cae8f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a3cae930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00a3cae970 _sleep() at _sleep+0x279/frame 0xfffffe00a3caea10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe00a3caea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a3caeab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a3caeab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100139 td 0xfffff80007bf65a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00a3cd38c0 mi_switch() at mi_switch+0x174/frame 0xfffffe00a3cd38f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a3cd3930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00a3cd3970 _sleep() at _sleep+0x279/frame 0xfffffe00a3cd3a10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe00a3cd3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a3cd3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a3cd3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100151 td 0xfffff800100ea5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b68138f0 mi_switch() at mi_switch+0x174/frame 0xfffffe00b6813920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b6813960 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00b68139a0 _sleep() at _sleep+0x279/frame 0xfffffe00b6813a40 softdep_flush() at softdep_flush+0x17f/frame 0xfffffe00b6813a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b6813ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b6813ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100202 td 0xfffff80236036000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b682c8f0 mi_switch() at mi_switch+0x174/frame 0xfffffe00b682c920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b682c960 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00b682c9a0 _sleep() at _sleep+0x279/frame 0xfffffe00b682ca40 softdep_flush() at softdep_flush+0x17f/frame 0xfffffe00b682ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b682cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b682cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100241 td 0xfffff80010b7c000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00b8a048f0 mi_switch() at mi_switch+0x174/frame 0xfffffe00b8a04920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00b8a04960 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00b8a049a0 _sleep() at _sleep+0x279/frame 0xfffffe00b8a04a40 softdep_flush() at softdep_flush+0x17f/frame 0xfffffe00b8a04a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b8a04ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b8a04ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command vmdaemon pid 31 tid 100129 td 0xfffff8000782d000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006f338b0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006f338e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006f33920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f33950 _sleep() at _sleep+0x2da/frame 0xfffffe0006f339f0 vm_daemon() at vm_daemon+0xaa6/frame 0xfffffe0006f33a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f33ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f33ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 30 tid 100128 td 0xfffff8000782d5a0 (CPU 11) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0006bebdf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0006bebe10 trap() at trap+0x48/frame 0xfffffe0006bebf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bebf20 --- trap 0x13, rip = 0xffffffff80cb58d2, rsp = 0xfffffe0006f2e5e0, rbp = 0xfffffe0006f2e600 --- _vdrop() at _vdrop+0xc2/frame 0xfffffe0006f2e600 vm_page_remove() at vm_page_remove+0xdd/frame 0xfffffe0006f2e620 vm_page_free_prep() at vm_page_free_prep+0x105/frame 0xfffffe0006f2e640 vm_page_free_toq() at vm_page_free_toq+0x12/frame 0xfffffe0006f2e670 vm_pageout_worker() at vm_pageout_worker+0xa36/frame 0xfffffe0006f2ea30 vm_pageout() at vm_pageout+0x156/frame 0xfffffe0006f2ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f2eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f2eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 30 tid 100131 td 0xfffff800076fd5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006fa96b0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006fa96e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006fa9720 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006fa9750 _sleep() at _sleep+0x2da/frame 0xfffffe0006fa97f0 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0xc60/frame 0xfffffe0006fa9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006fa9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006fa9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 30 tid 100132 td 0xfffff800076fd000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006fee8f0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006fee920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006fee960 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006fee9a0 _sleep() at _sleep+0x279/frame 0xfffffe0006feea40 uma_reclaim_worker() at uma_reclaim_worker+0xda/frame 0xfffffe0006feea70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006feeab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006feeab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon1 pid 29 tid 100127 td 0xfffff80007711000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006f1d8b0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006f1d8e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006f1d920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f1d950 _sleep() at _sleep+0x2da/frame 0xfffffe0006f1d9f0 enc_daemon() at enc_daemon+0x1cf/frame 0xfffffe0006f1da70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f1dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f1dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon0 pid 28 tid 100126 td 0xfffff800077115a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006f188b0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006f188e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006f18920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f18950 _sleep() at _sleep+0x2da/frame 0xfffffe0006f189f0 enc_daemon() at enc_daemon+0x1cf/frame 0xfffffe0006f18a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f18ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f18ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command rand_harvestq pid 27 tid 100123 td 0xfffff800037fe000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006f098c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006f098f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006f09930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006f09970 _sleep() at _sleep+0x279/frame 0xfffffe0006f09a10 random_kthread() at random_kthread+0x27a/frame 0xfffffe0006f09a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f09ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f09ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sctp_iterator pid 26 tid 100119 td 0xfffff80003800000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006ee9910 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ee9940 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006ee9980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006ee99b0 _sleep() at _sleep+0x2da/frame 0xfffffe0006ee9a50 sctp_iterator_thread() at sctp_iterator_thread+0x59/frame 0xfffffe0006ee9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ee9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ee9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod4 pid 25 tid 100118 td 0xfffff80007712000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006ed88e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ed8910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006ed8950 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006ed8990 _sleep() at _sleep+0x279/frame 0xfffffe0006ed8a30 soaio_kproc_loop() at soaio_kproc_loop+0x170/frame 0xfffffe0006ed8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ed8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ed8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod3 pid 24 tid 100117 td 0xfffff800077125a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006ed38e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ed3910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006ed3950 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006ed3990 _sleep() at _sleep+0x279/frame 0xfffffe0006ed3a30 soaio_kproc_loop() at soaio_kproc_loop+0x170/frame 0xfffffe0006ed3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ed3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ed3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod2 pid 23 tid 100116 td 0xfffff80007713000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006ece8e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ece910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006ece950 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006ece990 _sleep() at _sleep+0x279/frame 0xfffffe0006ecea30 soaio_kproc_loop() at soaio_kproc_loop+0x170/frame 0xfffffe0006ecea70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006eceab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006eceab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod1 pid 22 tid 100115 td 0xfffff800077135a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006ec98e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ec9910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006ec9950 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006ec9990 _sleep() at _sleep+0x279/frame 0xfffffe0006ec9a30 soaio_kproc_loop() at soaio_kproc_loop+0x170/frame 0xfffffe0006ec9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ec9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ec9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100096 td 0xfffff80003eee000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e5e930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e5e960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e5e9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e5e9d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e5ea30 usb_process() at usb_process+0x12b/frame 0xfffffe0006e5ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e5eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e5eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100097 td 0xfffff80003eed5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e63930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e63960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e639a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e639d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e63a30 usb_process() at usb_process+0x12b/frame 0xfffffe0006e63a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e63ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e63ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100098 td 0xfffff80003eed000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e68930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e68960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e689a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e689d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e68a30 usb_process() at usb_process+0x12b/frame 0xfffffe0006e68a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e68ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e68ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100099 td 0xfffff80003eec5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e6d930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e6d960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e6d9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e6d9d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e6da30 usb_process() at usb_process+0x12b/frame 0xfffffe0006e6da70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e6dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e6dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100100 td 0xfffff80003eec000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e72930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e72960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e729a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e729d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e72a30 usb_process() at usb_process+0x12b/frame 0xfffffe0006e72a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e72ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e72ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100102 td 0xfffff80003ede000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e7c930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e7c960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e7c9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e7c9d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e7ca30 usb_process() at usb_process+0x12b/frame 0xfffffe0006e7ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e7cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e7cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100103 td 0xfffff800060785a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e81930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e81960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e819a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e819d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e81a30 usb_process() at usb_process+0x12b/frame 0xfffffe0006e81a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e81ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e81ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100104 td 0xfffff80006078000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e86930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e86960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e869a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e869d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e86a30 usb_process() at usb_process+0x12b/frame 0xfffffe0006e86a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e86ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e86ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100105 td 0xfffff800060775a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e8b930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e8b960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e8b9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e8b9d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e8ba30 usb_process() at usb_process+0x12b/frame 0xfffffe0006e8ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e8bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e8bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100106 td 0xfffff80006077000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e90930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e90960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e909a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e909d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e90a30 usb_process() at usb_process+0x12b/frame 0xfffffe0006e90a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e90ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e90ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100107 td 0xfffff80003afc5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006ea1930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ea1960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006ea19a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006ea19d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006ea1a30 usb_process() at usb_process+0x12b/frame 0xfffffe0006ea1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ea1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ea1ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100108 td 0xfffff80003afc000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006ea6930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ea6960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006ea69a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006ea69d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006ea6a30 usb_process() at usb_process+0x12b/frame 0xfffffe0006ea6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ea6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ea6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100109 td 0xfffff80003afb5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006eab930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006eab960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006eab9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006eab9d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006eaba30 usb_process() at usb_process+0x12b/frame 0xfffffe0006eaba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006eabab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006eabab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100110 td 0xfffff80003afb000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006eb0930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006eb0960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006eb09a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006eb09d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006eb0a30 usb_process() at usb_process+0x12b/frame 0xfffffe0006eb0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006eb0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006eb0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100111 td 0xfffff80003afa5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006eb5930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006eb5960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006eb59a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006eb59d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006eb5a30 usb_process() at usb_process+0x12b/frame 0xfffffe0006eb5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006eb5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006eb5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100091 td 0xfffff80003994000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e458e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e45910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e45950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e45980 _sleep() at _sleep+0x2da/frame 0xfffffe0006e45a20 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe0006e45a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e45ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e45ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100092 td 0xfffff80003afd5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e4a8e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e4a910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e4a950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e4a980 _sleep() at _sleep+0x2da/frame 0xfffffe0006e4aa20 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe0006e4aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e4aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e4aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100093 td 0xfffff80003afd000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e4f8e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e4f910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e4f950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e4f980 _sleep() at _sleep+0x2da/frame 0xfffffe0006e4fa20 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe0006e4fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e4fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e4fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100125 td 0xfffff8000782e000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006f138d0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006f13900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006f13940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f13970 _sleep() at _sleep+0x2da/frame 0xfffffe0006f13a10 xpt_scanner_thread() at xpt_scanner_thread+0x93/frame 0xfffffe0006f13a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f13ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f13ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 11 pid 19 tid 100090 td 0xfffff800039945a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e408c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e408f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e40930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e40960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e40a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e40a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e40ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e40ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 10 pid 18 tid 100089 td 0xfffff80003995000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e3b8c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e3b8f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e3b930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e3b960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e3ba00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e3ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e3bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e3bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 9 pid 17 tid 100088 td 0xfffff800039955a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e368c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e368f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e36930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e36960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e36a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e36a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e36ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e36ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 8 pid 16 tid 100087 td 0xfffff80003996000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e318c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e318f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e31930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e31960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e31a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e31a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e31ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e31ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 7 pid 15 tid 100086 td 0xfffff800039965a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e2c8c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e2c8f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e2c930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e2c960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e2ca00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e2ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e2cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e2cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 6 pid 9 tid 100085 td 0xfffff80003997000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e278c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e278f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e27930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e27960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e27a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e27a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e27ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e27ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 5 pid 8 tid 100084 td 0xfffff800039975a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e228c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e228f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e22930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e22960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e22a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e22a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e22ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e22ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 4 pid 7 tid 100083 td 0xfffff80003998000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e1d8c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e1d8f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e1d930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e1d960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e1da00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e1da70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e1dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e1dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 3 pid 6 tid 100082 td 0xfffff800039775a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e188c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e188f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e18930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e18960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e18a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e18a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e18ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e18ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 2 pid 5 tid 100081 td 0xfffff80003988000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e138c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e138f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e13930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e13960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e13a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e13a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e13ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e13ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 1 pid 4 tid 100080 td 0xfffff800039885a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e0e8c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e0e8f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e0e930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e0e960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e0ea00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e0ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e0eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e0eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 0 pid 3 tid 100079 td 0xfffff80003989000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e098c0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e098f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e09930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e09960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e09a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e09a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e09ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e09ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 2 tid 100078 td 0xfffff800039895a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e048f0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e04920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006e04960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e04990 _sleep() at _sleep+0x2da/frame 0xfffffe0006e04a30 crypto_proc() at crypto_proc+0x2c5/frame 0xfffffe0006e04a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e04ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e04ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sequencer 00 pid 14 tid 100065 td 0xfffff80003977000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00005c08f0 mi_switch() at mi_switch+0x174/frame 0xfffffe00005c0920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005c0960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005c0990 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe00005c09f0 seq_eventthread() at seq_eventthread+0xdc/frame 0xfffffe00005c0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005c0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005c0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100061 td 0xfffff80003839000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00005948f0 mi_switch() at mi_switch+0x174/frame 0xfffffe0000594920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000594960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0000594990 _sleep() at _sleep+0x2da/frame 0xfffffe0000594a30 g_run_events() at g_run_events+0x4f/frame 0xfffffe0000594a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000594ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000594ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100062 td 0xfffff800038385a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00005998f0 mi_switch() at mi_switch+0x174/frame 0xfffffe0000599920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000599960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0000599990 _sleep() at _sleep+0x2da/frame 0xfffffe0000599a30 g_io_schedule_up() at g_io_schedule_up+0xa5/frame 0xfffffe0000599a60 g_up_procbody() at g_up_procbody+0x6d/frame 0xfffffe0000599a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000599ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000599ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100063 td 0xfffff80003838000 sched_switch() at sched_switch+0x71a/frame 0xfffffe000059e8f0 mi_switch() at mi_switch+0x174/frame 0xfffffe000059e920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000059e960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000059e990 _sleep() at _sleep+0x2da/frame 0xfffffe000059ea30 g_io_schedule_down() at g_io_schedule_down+0x14c/frame 0xfffffe000059ea60 g_down_procbody() at g_down_procbody+0x6d/frame 0xfffffe000059ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe000059eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000059eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100016 td 0xfffff800036c65a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100020 td 0xfffff8000370e5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00767c79e0 mi_switch() at mi_switch+0x174/frame 0xfffffe00767c7a10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe00767c7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00767c7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00767c7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100021 td 0xfffff8000370e000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00767cc9e0 mi_switch() at mi_switch+0x174/frame 0xfffffe00767cca10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe00767cca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00767ccab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00767ccab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100047 td 0xfffff80003825000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00004ee9e0 mi_switch() at mi_switch+0x174/frame 0xfffffe00004eea10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe00004eea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004eeab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004eeab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100048 td 0xfffff800038245a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100049 td 0xfffff80003824000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100050 td 0xfffff800038235a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100051 td 0xfffff80003823000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100052 td 0xfffff800038225a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100053 td 0xfffff80003822000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100054 td 0xfffff8000380f5a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100055 td 0xfffff8000380f000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100056 td 0xfffff8000383b5a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100057 td 0xfffff8000383b000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100058 td 0xfffff8000383a5a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100059 td 0xfffff8000383a000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100060 td 0xfffff800038395a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe000052f9e0 mi_switch() at mi_switch+0x174/frame 0xfffffe000052fa10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe000052fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe000052fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000052fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100094 td 0xfffff80003eef000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e549e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e54a10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe0006e54a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e54ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e54ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100095 td 0xfffff80003eee5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e599e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e59a10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe0006e59a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e59ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e59ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100101 td 0xfffff80003ede5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006e779e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006e77a10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe0006e77a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e77ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e77ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100112 td 0xfffff80003afa000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100113 td 0xfffff800039985a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006ebf9e0 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ebfa10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe0006ebfa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ebfab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ebfab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100180 td 0xfffff8000616a5a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100181 td 0xfffff8000616a000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100182 td 0xfffff800061695a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100183 td 0xfffff80006169000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100184 td 0xfffff802360385a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100185 td 0xfffff80236038000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100186 td 0xfffff802360375a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100187 td 0xfffff80236037000 fork_trampoline() at fork_trampoline Tracing command idle pid 11 tid 100003 td 0xfffff800036b8000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0076766940 mi_switch() at mi_switch+0x174/frame 0xfffffe0076766970 critical_exit_preempt() at critical_exit_preempt+0x66/frame 0xfffffe0076766990 sched_idletd() at sched_idletd+0x482/frame 0xfffffe0076766a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076766ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076766ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xfffff800036b75a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe007676b940 mi_switch() at mi_switch+0x174/frame 0xfffffe007676b970 critical_exit_preempt() at critical_exit_preempt+0x66/frame 0xfffffe007676b990 sched_idletd() at sched_idletd+0x482/frame 0xfffffe007676ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe007676bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007676bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffff800036b7000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0076770960 mi_switch() at mi_switch+0x174/frame 0xfffffe0076770990 sched_idletd() at sched_idletd+0xb6/frame 0xfffffe0076770a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076770ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076770ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xfffff800036b65a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0076775960 mi_switch() at mi_switch+0x174/frame 0xfffffe0076775990 sched_idletd() at sched_idletd+0x4d4/frame 0xfffffe0076775a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076775ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076775ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100007 td 0xfffff800036b6000 sched_switch() at sched_switch+0x71a/frame 0xfffffe007677a960 mi_switch() at mi_switch+0x174/frame 0xfffffe007677a990 sched_idletd() at sched_idletd+0xb6/frame 0xfffffe007677aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007677aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007677aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xfffff800036b55a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe007677f940 mi_switch() at mi_switch+0x174/frame 0xfffffe007677f970 critical_exit_preempt() at critical_exit_preempt+0x66/frame 0xfffffe007677f990 sched_idletd() at sched_idletd+0x482/frame 0xfffffe007677fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007677fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007677fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xfffff800036b5000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0076784940 mi_switch() at mi_switch+0x174/frame 0xfffffe0076784970 critical_exit_preempt() at critical_exit_preempt+0x66/frame 0xfffffe0076784990 sched_idletd() at sched_idletd+0x482/frame 0xfffffe0076784a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076784ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076784ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xfffff800036c95a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0076789960 mi_switch() at mi_switch+0x174/frame 0xfffffe0076789990 sched_idletd() at sched_idletd+0xb6/frame 0xfffffe0076789a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076789ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076789ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffff800036c9000 sched_switch() at sched_switch+0x71a/frame 0xfffffe007678e940 mi_switch() at mi_switch+0x174/frame 0xfffffe007678e970 critical_exit_preempt() at critical_exit_preempt+0x66/frame 0xfffffe007678e990 sched_idletd() at sched_idletd+0x482/frame 0xfffffe007678ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe007678eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007678eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100012 td 0xfffff800036c85a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0076793940 mi_switch() at mi_switch+0x174/frame 0xfffffe0076793970 critical_exit_preempt() at critical_exit_preempt+0x66/frame 0xfffffe0076793990 sched_idletd() at sched_idletd+0x482/frame 0xfffffe0076793a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076793ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076793ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xfffff800036c8000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0076798940 mi_switch() at mi_switch+0x174/frame 0xfffffe0076798970 critical_exit_preempt() at critical_exit_preempt+0x66/frame 0xfffffe0076798990 sched_idletd() at sched_idletd+0x482/frame 0xfffffe0076798a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076798ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076798ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100014 td 0xfffff800036c75a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe007679d940 mi_switch() at mi_switch+0x174/frame 0xfffffe007679d970 critical_exit_preempt() at critical_exit_preempt+0x66/frame 0xfffffe007679d990 sched_idletd() at sched_idletd+0x482/frame 0xfffffe007679da70 fork_exit() at fork_exit+0x84/frame 0xfffffe007679dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007679dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command init pid 1 tid 100002 td 0xfffff800036b85a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0076761570 mi_switch() at mi_switch+0x174/frame 0xfffffe00767615a0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00767615e0 sleepq_catch_signals() at sleepq_catch_signals+0x3e7/frame 0xfffffe0076761640 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0076761670 _sleep() at _sleep+0x2d0/frame 0xfffffe0076761710 kern_wait6() at kern_wait6+0x48f/frame 0xfffffe00767617a0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe0076761990 amd64_syscall() at amd64_syscall+0x291/frame 0xfffffe0076761ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0076761ab0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x2b249a, rsp = 0x7fffffffe778, rbp = 0x7fffffffe7e0 --- Tracing command audit pid 10 tid 100001 td 0xfffff800036b9000 sched_switch() at sched_switch+0x71a/frame 0xfffffe007675c910 mi_switch() at mi_switch+0x174/frame 0xfffffe007675c940 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe007675c980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe007675c9b0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe007675ca10 audit_worker() at audit_worker+0x93/frame 0xfffffe007675ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe007675cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007675cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xffffffff82094640 sched_switch() at sched_switch+0x71a/frame 0xffffffff828fb980 mi_switch() at mi_switch+0x174/frame 0xffffffff828fb9b0 sleepq_switch() at sleepq_switch+0x10d/frame 0xffffffff828fb9f0 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xffffffff828fba30 _sleep() at _sleep+0x279/frame 0xffffffff828fbad0 swapper() at swapper+0x77/frame 0xffffffff828fbb20 mi_startup() at mi_startup+0x2e7/frame 0xffffffff828fbb70 btext() at btext+0x2c Tracing command kernel pid 0 tid 100015 td 0xfffff800036c7000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00767ae900 mi_switch() at mi_switch+0x174/frame 0xfffffe00767ae930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00767ae970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00767ae9a0 _sleep() at _sleep+0x2da/frame 0xfffffe00767aea40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00767aea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00767aeab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00767aeab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100017 td 0xfffff800036c6000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00767b8930 mi_switch() at mi_switch+0x174/frame 0xfffffe00767b8960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00767b89a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00767b89d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe00767b8a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00767b8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00767b8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00767b8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100018 td 0xfffff800036b95a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00767bd900 mi_switch() at mi_switch+0x174/frame 0xfffffe00767bd930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00767bd970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00767bd9a0 _sleep() at _sleep+0x2da/frame 0xfffffe00767bda40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00767bda70 fork_exit() at fork_exit+0x84/frame 0xfffffe00767bdab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00767bdab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xfffff8000370f000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00767c2900 mi_switch() at mi_switch+0x174/frame 0xfffffe00767c2930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00767c2970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00767c29a0 _sleep() at _sleep+0x2da/frame 0xfffffe00767c2a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00767c2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00767c2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00767c2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xfffff8000370d5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0000471930 mi_switch() at mi_switch+0x174/frame 0xfffffe0000471960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004719a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004719d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe0000471a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000471a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000471ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000471ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xfffff8000370d000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0000476930 mi_switch() at mi_switch+0x174/frame 0xfffffe0000476960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004769a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004769d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe0000476a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000476a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000476ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000476ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xfffff8000370c5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe000047b930 mi_switch() at mi_switch+0x174/frame 0xfffffe000047b960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000047b9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000047b9d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe000047ba40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000047ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe000047bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000047bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xfffff8000370c000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0000480930 mi_switch() at mi_switch+0x174/frame 0xfffffe0000480960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004809a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004809d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe0000480a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000480a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000480ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000480ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100026 td 0xfffff8000370b5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0000485930 mi_switch() at mi_switch+0x174/frame 0xfffffe0000485960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004859a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004859d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe0000485a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000485a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000485ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000485ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100027 td 0xfffff8000370b000 sched_switch() at sched_switch+0x71a/frame 0xfffffe000048a930 mi_switch() at mi_switch+0x174/frame 0xfffffe000048a960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000048a9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000048a9d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe000048aa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000048aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe000048aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000048aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100028 td 0xfffff800037435a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe000048f930 mi_switch() at mi_switch+0x174/frame 0xfffffe000048f960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000048f9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000048f9d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe000048fa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000048fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe000048fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000048fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100029 td 0xfffff80003743000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0000494930 mi_switch() at mi_switch+0x174/frame 0xfffffe0000494960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004949a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004949d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe0000494a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000494a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000494ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000494ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100030 td 0xfffff800037425a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0000499930 mi_switch() at mi_switch+0x174/frame 0xfffffe0000499960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004999a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004999d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe0000499a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000499a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000499ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000499ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100031 td 0xfffff80003742000 sched_switch() at sched_switch+0x71a/frame 0xfffffe000049e930 mi_switch() at mi_switch+0x174/frame 0xfffffe000049e960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000049e9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000049e9d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe000049ea40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000049ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe000049eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000049eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100032 td 0xfffff800037415a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00004a3930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004a3960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004a39a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004a39d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe00004a3a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004a3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004a3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004a3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100033 td 0xfffff80003741000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00004a8930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004a8960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004a89a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004a89d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe00004a8a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004a8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004a8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004a8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100034 td 0xfffff800037405a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00004ad930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004ad960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004ad9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004ad9d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe00004ada40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004ada70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004adab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004adab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100035 td 0xfffff800037f3000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00004b2930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004b2960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004b29a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004b29d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe00004b2a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004b2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004b2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004b2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100036 td 0xfffff800037f25a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00004b7930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004b7960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004b79a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004b79d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe00004b7a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004b7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004b7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004b7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100037 td 0xfffff800037f2000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00004bc930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004bc960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004bc9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004bc9d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe00004bca40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004bca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004bcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004bcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100038 td 0xfffff800037f15a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00004c1930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004c1960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004c19a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004c19d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe00004c1a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004c1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004c1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004c1ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100039 td 0xfffff800037f1000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00004c6930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004c6960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004c69a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004c69d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe00004c6a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004c6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004c6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004c6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100040 td 0xfffff800037f05a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00004cb930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004cb960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004cb9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004cb9d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe00004cba40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004cba70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004cbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004cbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100041 td 0xfffff800037f0000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00004d0930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004d0960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004d09a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004d09d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe00004d0a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004d0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004d0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004d0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100042 td 0xfffff800037dd5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00004d5930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004d5960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004d59a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004d59d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe00004d5a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004d5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004d5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004d5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100043 td 0xfffff800037dd000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00004da930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004da960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004da9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004da9d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe00004daa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004daa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004daab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004daab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100044 td 0xfffff800038015a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00004df930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004df960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004df9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004df9d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe00004dfa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004dfa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004dfab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004dfab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100045 td 0xfffff80003801000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00004e4930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004e4960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004e49a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004e49d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe00004e4a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004e4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004e4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004e4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100046 td 0xfffff800038005a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00004e9930 mi_switch() at mi_switch+0x174/frame 0xfffffe00004e9960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004e99a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004e99d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe00004e9a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004e9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004e9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004e9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100064 td 0xfffff800038255a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00005af900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005af930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005af970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005af9a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005afa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005afa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005afab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005afab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100066 td 0xfffff800039765a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00005c5900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005c5930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005c5970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005c59a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005c5a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005c5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005c5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005c5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100067 td 0xfffff80003976000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00005ca900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005ca930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005ca970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005ca9a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005caa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005caa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005caab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005caab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100068 td 0xfffff800039755a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00005cf900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005cf930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005cf970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005cf9a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005cfa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005cfa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005cfab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005cfab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100069 td 0xfffff80003975000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00005d4900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005d4930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005d4970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005d49a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005d4a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005d4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005d4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005d4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100070 td 0xfffff800039745a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00005d9900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005d9930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005d9970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005d99a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005d9a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005d9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005d9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005d9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100071 td 0xfffff80003974000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00005de900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005de930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005de970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005de9a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005dea40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005dea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005deab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005deab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100072 td 0xfffff800039735a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00005e3900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005e3930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005e3970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005e39a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005e3a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005e3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005e3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005e3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100073 td 0xfffff80003973000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00005e8900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005e8930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005e8970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005e89a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005e8a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005e8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005e8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005e8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100074 td 0xfffff8000398b5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00005ed900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005ed930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005ed970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005ed9a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005eda40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005eda70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005edab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005edab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100075 td 0xfffff8000398b000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00005f2900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005f2930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005f2970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005f29a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005f2a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005f2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005f2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005f2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100076 td 0xfffff8000398a5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe00005f7900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005f7930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005f7970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005f79a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005f7a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005f7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005f7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005f7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100077 td 0xfffff8000398a000 sched_switch() at sched_switch+0x71a/frame 0xfffffe00005fc900 mi_switch() at mi_switch+0x174/frame 0xfffffe00005fc930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005fc970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005fc9a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005fca40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005fca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005fcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005fcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100114 td 0xfffff80006174000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006ec4930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006ec4960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006ec49a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006ec49d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe0006ec4a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0006ec4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ec4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ec4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100120 td 0xfffff800037ff5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006efa930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006efa960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006efa9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006efa9d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe0006efaa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0006efaa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006efaab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006efaab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100121 td 0xfffff800037ff000 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006eff930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006eff960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006eff9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006eff9d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe0006effa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0006effa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006effab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006effab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100122 td 0xfffff800037fe5a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006f04930 mi_switch() at mi_switch+0x174/frame 0xfffffe0006f04960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006f049a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f049d0 msleep_spin_sbt() at msleep_spin_sbt+0x143/frame 0xfffffe0006f04a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0006f04a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f04ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f04ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100124 td 0xfffff800037f35a0 sched_switch() at sched_switch+0x71a/frame 0xfffffe0006f0e900 mi_switch() at mi_switch+0x174/frame 0xfffffe0006f0e930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0006f0e970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f0e9a0 _sleep() at _sleep+0x2da/frame 0xfffffe0006f0ea40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006f0ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f0eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f0eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db:0:pho> dump Dumping 1392 out of 32621 MB:..2%..11%..21%..32%..41%..51%..61%..71%..81%..91% Dump complete db> (kgdb) bt #0 doadump (textdump=) at pcpu.h:230 #1 0xffffffff8049bb5b in db_dump (dummy=, dummy2=, dummy3=, dummy4=) at ../../../ddb/db_command.c:574 #2 0xffffffff8049b929 in db_command (cmd_table=, dopager=0x0) at ../../../ddb/db_command.c:481 #3 0xffffffff804a0848 in db_script_exec (scriptname=, warnifnotfound=) at ../../../ddb/db_script.c:304 #4 0xffffffff8049b929 in db_command (cmd_table=, dopager=0x1) at ../../../ddb/db_command.c:481 #5 0xffffffff8049b6a4 in db_command_loop () at ../../../ddb/db_command.c:534 #6 0xffffffff8049e85f in db_trap (type=, code=) at ../../../ddb/db_main.c:252 #7 0xffffffff80c2be14 in kdb_trap (type=0x3, code=0x0, tf=) at ../../../kern/subr_kdb.c:692 #8 0xffffffff810b9f9b in trap (frame=0xfffffe00b6975f80) at ../../../amd64/amd64/trap.c:619 #9 0xffffffff81094235 in calltrap () at ../../../amd64/amd64/exception.S:232 #10 0xffffffff80c2b51b in kdb_enter (why=0xffffffff81353c85 "panic", msg=) at cpufunc.h:65 #11 0xffffffff80be38e1 in vpanic (fmt=, ap=0xfffffe00b69760f0) at ../../../kern/kern_shutdown.c:866 #12 0xffffffff80be3673 in panic (fmt=) at ../../../kern/kern_shutdown.c:804 #13 0xffffffff80f479e8 in vnode_pager_generic_getpages (vp=0xfffff8031dbc2768, m=0xfffff8000778c2e8, count=0x1, a_rbehind=0x0, a_rahead=0xfffffe00b69764a8, iodone=0xffffffff80be1760 , arg=0xfffff8000778c2c0) at ../../../vm/vnode_pager.c:1007 #14 0xffffffff80f47e2e in vnode_pager_local_getpages_async (ap=0xfffffe00b69762f0) at ../../../vm/vnode_pager.c:750 #15 0xffffffff81245f55 in VOP_GETPAGES_ASYNC_APV (vop=0xffffffff81b64680, a=0xfffffe00b69762f0) at vnode_if.c:2822 #16 0xffffffff80f4a019 in VOP_GETPAGES_ASYNC (vp=0xfffff8031dbc2768, m=0xfffff8000778c2e8, count=0x1, rbehind=0x0, rahead=0xfffffe00b69764a8, iodone=0xffffffff80be1760 , arg=0xfffff8000778c2c0) at vnode_if.h:1192 #17 0xffffffff80f45db8 in vnode_pager_getpages_async (object=0xfffff800103e3000, m=0xfffff8000778c2e8, count=0x1, rbehind=0x0, rahead=0xfffffe00b69764a8, iodone=0xffffffff80be1760 , arg=0xfffff8000778c2c0) at ../../../vm/vnode_pager.c:726 #18 0xffffffff80be0fb4 in sendfile_swapin (obj=0xfffff800103e3000, sfio=0xfffff8000778c2c0, off=0x8eb7404, len=0xffc, npages=0x2, rhpages=0x1f, flags=0x0) at ../../../kern/kern_sendfile.c:404 #19 0xffffffff80bdf865 in vn_sendfile (fp=0xfffff8001047b280, sockfd=0x3, hdr_uio=0x0, trl_uio=0x0, offset=0x0, nbytes=0x20000000, sent=0xfffffe00b6976900, flags=0x0, td=0xfffff800106c65a0) at ../../../kern/kern_sendfile.c:746 #20 0xffffffff80be25be in fo_sendfile (fp=0xfffff8001047b280, sockfd=0x3, hdr_uio=0x0, trl_uio=0x0, offset=0x0, nbytes=0x20000000, sent=0xfffffe00b6976900, flags=0x0, td=0xfffff800106c65a0) at file.h:385 #21 0xffffffff80be1d92 in sendfile (td=0xfffff800106c65a0, uap=0xfffff800106c6968, compat=0x0) at ../../../kern/kern_sendfile.c:1006 #22 0xffffffff80be1b94 in sys_sendfile (td=0xfffff800106c65a0, uap=0xfffff800106c6968) at ../../../kern/kern_sendfile.c:1034 #23 0xffffffff810baeb1 in amd64_syscall (td=0xfffff800106c65a0, traced=0x0) at subr_syscall.c:135 #24 0xffffffff81094b1d in fast_syscall_common () at ../../../amd64/amd64/exception.S:504 #25 0x000000080030744a in ?? () Previous frame inner to this frame (corrupt stack?) Current language: auto; currently minimal (kgdb) f 13 #13 0xffffffff80f479e8 in vnode_pager_generic_getpages (vp=0xfffff8031dbc2768, m=0xfffff8000778c2e8, count=0x1, a_rbehind=0x0, a_rahead=0xfffffe00b69764a8, iodone=0xffffffff80be1760 , arg=0xfffff8000778c2c0) at ../../../vm/vnode_pager.c:1007 1007 KASSERT(secmask < PAGE_SIZE && secmask > 0, (kgdb) l 1002 foff = IDX_TO_OFF(bp->b_pages[0]->pindex); 1003 bytecount = bp->b_npages << PAGE_SHIFT; 1004 if ((foff + bytecount) > object->un_pager.vnp.vnp_size) 1005 bytecount = object->un_pager.vnp.vnp_size - foff; 1006 secmask = bo->bo_bsize - 1; 1007 KASSERT(secmask < PAGE_SIZE && secmask > 0, 1008 ("%s: sector size %d too large", __func__, secmask + 1)); 1009 bytecount = (bytecount + secmask) & ~secmask; 1010 1011 /* (kgdb) info loc object = 0xfffff800103e3000 bo = (struct bufobj *) 0xfffff8031d5f9168 bp = (struct buf *) 0xfffff80007ff6000 foff = 0x8eb8000 blkno0 = 0x1157c0 bsize = 0x8000 pagesperblock = 0x8 error = 0x0 before = 0x18 after = 0x1f rbehind = 0x0 rahead = 0x1f poff = 0x0 i = 0x20 bytecount = 0x20000 secmask = 0x1fff (kgdb) p *vp $1 = {v_tag = 0xffffffff81270ef9 "ufs", v_op = 0xffffffff81b64680, v_data = 0xfffff80010056140, v_mount = 0xfffff8031db77000, v_nmntvnodes = {tqe_next = 0xfffff800105454f0, tqe_prev = 0xfffff800105a7020}, {v_mountedhere = 0x0, v_unpcb = 0x0, v_rdev = 0x0, v_fifoinfo = 0x0}, v_hashlist = {le_next = 0x0, le_prev = 0xfffffe009bd14b30}, v_cache_src = {lh_first = 0x0}, v_cache_dst = {tqh_first = 0xfffff800103dc5b0, tqh_last = 0xfffff800103dc5d0}, v_cache_dd = 0x0, v_lock = {lock_object = { lo_name = 0xffffffff81270ef9 "ufs", lo_flags = 0x6fb0000, lo_data = 0x0, lo_witness = 0xfffff8087fd78680}, lk_lock = 0x11, lk_exslpfail = 0x0, lk_timo = 0x33, lk_pri = 0x60, lk_stack = {depth = 0xb, pcs = 0xfffff8031dbc2808}}, v_interlock = {lock_object = {lo_name = 0xffffffff813991bb "vnode interlock", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xfffff8087fd6b680}, mtx_lock = 0x0}, v_vnlock = 0xfffff8031dbc27d0, v_actfreelist = {tqe_next = 0xfffff800105dfc58, tqe_prev = 0xfffff80010545648}, v_bufobj = {bo_lock = {lock_object = {lo_name = 0xffffffff813598f8 "bufobj interlock", lo_flags = 0x5230000, lo_data = 0x0, lo_witness = 0xfffff8087fd75800}, rw_lock = 0x1}, bo_ops = 0xffffffff81b29270, bo_object = 0xfffff800103e3000, bo_synclist = {le_next = 0x0, le_prev = 0xfffff80003835900}, bo_private = 0xfffff8031dbc2768, bo_clean = {bv_hd = {tqh_first = 0xfffffe0000ba0440, tqh_last = 0xfffffe0000b24540}, bv_root = { pt_root = 0xfffff800076b2090}, bv_cnt = 0x1c0b}, bo_dirty = {bv_hd = {tqh_first = 0xfffffe0000ee6af0, tqh_last = 0xfffffe0000889eb0}, bv_root = { pt_root = 0xfffff800076c13f0}, bv_cnt = 0x5}, bo_numoutput = 0x0, bo_flag = 0x1, bo_domain = 0x5, bo_bsize = 0x8000}, v_pollinfo = 0x0, v_label = 0x0, v_lockf = 0x0, v_rl = {rl_waiters = {tqh_first = 0x0, tqh_last = 0xfffff8031dbc2988}, rl_currdep = 0x0}, v_cstart = 0x4000, v_lasta = 0x34e1c0, v_lastw = 0x3fff, v_clen = 0x0, v_holdcnt = 0x1c13, v_usecount = 0x2, v_iflag = 0x200, v_vflag = 0x0, v_mflag = 0x0, v_writecount = 0x0, v_hash = 0x4, v_type = VREG} (kgdb) -- Test scenario: misc/sendfile11.sh