vmcore.vmcore.73 KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2018 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 12.0-CURRENT #0 r332299M: Mon Apr 9 20:32:57 CEST 2018 pho@mercat1.netperf.freebsd.org:/usr/src/sys/amd64/compile/PHO amd64 FreeBSD clang version 6.0.0 (tags/RELEASE_600/final 326565) (based on LLVM 6.0.0) WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. Entering uma_startup with 7 boot pages configured startup_alloc from "UMA Kegs", 6 boot pages left startup_alloc from "UMA Zones", 5 boot pages left startup_alloc from "UMA Zones", 4 boot pages left startup_alloc from "UMA Zones", 3 boot pages left startup_alloc from "UMA Zones", 2 boot pages left startup_alloc from "UMA Hash", 1 boot pages left Entering uma_startup1 with 0 boot pages left Entering uma_startup2 with 0 boot pages left VT(vga): resolution 640x480 CPU: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz (3500.08-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306f2 Family=0x6 Model=0x3f Stepping=2 Features=0xbfebfbff Features2=0x7ffefbff AMD Features=0x2c100800 AMD Features2=0x21 Structured Extended Features=0x37ab XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics real memory = 34359738368 (32768 MB) avail memory = 33179697152 (31642 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs FreeBSD/SMP: 1 package(s) x 6 core(s) x 2 hardware threads arc4random: no preloaded entropy cache ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard SMP: AP CPU #1 Launched! SMP: AP CPU #10 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #6 Launched! SMP: AP CPU #11 Launched! SMP: AP CPU #9 Launched! SMP: AP CPU #4 Launched! SMP: AP CPU #5 Launched! SMP: AP CPU #8 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #7 Launched! Timecounter "TSC-low" frequency 1750037604 Hz quality 1000 random: entropy device external interface module_register_init: MOD_LOAD (vesa, 0xffffffff810854e0, 0) error 19 random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" kbd1 at kbdmux0 netmap: loaded module [ath_hal] loaded nexus0 vtvga0: on motherboard cryptosoft0: on motherboard acpi0: on motherboard acpi0: Power Button (fixed) cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 cpu4: on acpi0 cpu5: on acpi0 cpu6: on acpi0 cpu7: on acpi0 cpu8: on acpi0 cpu9: on acpi0 cpu10: on acpi0 cpu11: on acpi0 atrtc0: port 0x70-0x71,0x74-0x77 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Event timer "HPET4" frequency 14318180 Hz quality 340 Event timer "HPET5" frequency 14318180 Hz quality 340 Event timer "HPET6" frequency 14318180 Hz quality 340 Event timer "HPET7" frequency 14318180 Hz quality 340 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: on acpi0 pci0: on pcib0 pci0: at device 11.1 (no driver attached) pci0: at device 11.2 (no driver attached) pci0: at device 16.1 (no driver attached) pci0: at device 16.6 (no driver attached) pci0: at device 18.1 (no driver attached) acpi_syscontainer0: on acpi0 acpi_syscontainer1: on acpi0 acpi_syscontainer2: on acpi0 acpi_syscontainer3: on acpi0 pcib1: port 0xcf8-0xcff on acpi0 pci1: on pcib1 pcib2: irq 26 at device 1.0 on pci1 pci2: on pcib2 pcib3: irq 32 at device 2.0 on pci1 pci3: on pcib3 pci3: at device 0.0 (no driver attached) pci3: at device 0.1 (no driver attached) pcib4: irq 40 at device 3.0 on pci1 pci4: on pcib4 pci1: at device 17.0 (no driver attached) ahci0: port 0xf110-0xf117,0xf100-0xf103,0xf0f0-0xf0f7,0xf0e0-0xf0e3,0xf020-0xf03f mem 0xfb438000-0xfb4387ff irq 16 at device 17.4 on pci1 ahci0: AHCI v1.30 with 4 6Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahciem0: on ahci0 xhci0: mem 0xfb400000-0xfb40ffff irq 19 at device 20.0 on pci1 xhci0: 32 bytes context size, 64-bit DMA usbus0 on xhci0 usbus0: 5.0Gbps Super Speed USB v3.0 pci1: at device 22.0 (no driver attached) pci1: at device 22.1 (no driver attached) ehci0: mem 0xfb434000-0xfb4343ff irq 18 at device 26.0 on pci1 usbus1: EHCI version 1.0 usbus1 on ehci0 usbus1: 480Mbps High Speed USB v2.0 pcib5: irq 16 at device 28.0 on pci1 pci5: on pcib5 pcib6: irq 18 at device 28.2 on pci1 pci6: on pcib6 pcib7: at device 0.0 on pci6 pci7: on pcib7 vgapci0: port 0xe000-0xe07f mem 0xfa000000-0xfaffffff,0xfb000000-0xfb01ffff irq 18 at device 0.0 on pci7 vgapci0: Boot video device pcib8: irq 16 at device 28.4 on pci1 pci8: on pcib8 igb0: port 0xd020-0xd03f mem 0xfb320000-0xfb33ffff,0xfb344000-0xfb347fff irq 16 at device 0.0 on pci8 igb0: attach_pre capping queues at 8 igb0: using 1024 tx descriptors and 1024 rx descriptors igb0: msix_init qsets capped at 8 igb0: pxm cpus: 6 queue msgs: 9 admincnt: 1 igb0: queue equality override not set, capping rx_queues at 6 and tx_queues at 6 igb0: using 6 rx queues 6 tx queues igb0: Using MSIX interrupts with 7 vectors igb0: allocated for 6 tx_queues igb0: allocated for 6 rx_queues igb0: Ethernet address: 0c:c4:7a:a8:cd:da igb0: netmap queues/slots: TX 6/1024, RX 6/1024 igb1: port 0xd000-0xd01f mem 0xfb300000-0xfb31ffff,0xfb340000-0xfb343fff irq 17 at device 0.1 on pci8 igb1: attach_pre capping queues at 8 igb1: using 1024 tx descriptors and 1024 rx descriptors igb1: msix_init qsets capped at 8 igb1: pxm cpus: 6 queue msgs: 9 admincnt: 1 igb1: queue equality override not set, capping rx_queues at 6 and tx_queues at 6 igb1: using 6 rx queues 6 tx queues igb1: Using MSIX interrupts with 7 vectors igb1: allocated for 6 tx_queues igb1: allocated for 6 rx_queues igb1: Ethernet address: 0c:c4:7a:a8:cd:db igb1: netmap queues/slots: TX 6/1024, RX 6/1024 ehci1: mem 0xfb433000-0xfb4333ff irq 18 at device 29.0 on pci1 usbus2: EHCI version 1.0 usbus2 on ehci1 usbus2: 480Mbps High Speed USB v2.0 isab0: at device 31.0 on pci1 isa0: on isab0 ahci1: port 0xf070-0xf077,0xf060-0xf063,0xf050-0xf057,0xf040-0xf043,0xf000-0xf01f mem 0xfb432000-0xfb4327ff irq 16 at device 31.2 on pci1 ahci1: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported ahcich4: at channel 0 on ahci1 ahcich5: at channel 1 on ahci1 ahcich6: at channel 2 on ahci1 ahcich7: at channel 3 on ahci1 ahcich8: at channel 4 on ahci1 ahcich9: at channel 5 on ahci1 ahciem1: on ahci1 acpi_button0: on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 uart1: console (115200,n,8,1) orm0: at iomem 0xc0000-0xc7fff,0xcb000-0xcbfff pnpid ORM0000 on isa0 est0: on cpu0 p4tcc0: on cpu0 est1: on cpu1 p4tcc1: on cpu1 est2: on cpu2 p4tcc2: on cpu2 est3: on cpu3 p4tcc3: on cpu3 est4: on cpu4 p4tcc4: on cpu4 est5: on cpu5 p4tcc5: on cpu5 est6: on cpu6 p4tcc6: on cpu6 est7: on cpu7 p4tcc7: on cpu7 est8: on cpu8 p4tcc8: on cpu8 est9: on cpu9 p4tcc9: on cpu9 est10: on cpu10 p4tcc10: on cpu10 est11: on cpu11 p4tcc11: on cpu11 NULL mp in getnewvnode(9), tag crossmp Timecounters tick every 1.000 msec arc4random: no preloaded entropy cache ugen0.1: <0x8086 XHCI root HUB> at usbus0 ugen2.1: at usbus2 ugen1.1: at usbus1 uhub0: <0x8086 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0 uhub1: on usbus1 uhub2: 0 uhub3: 6 ports with 6 removable, self powered uhub5: 4 ports with 3 removable, self powered uhub4: 8 ports with 8 removable, self powered Writing entropy file:. Setting hostname: mercat1.netperf.freebsd.org. ugen0.3: at usbus0 ukbd0 on uhub5 ukbd0: on usbus0 kbd2 at ukbd0 Starting Network: lo0 igb0 igb1. lo0: flags=8049 metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 nd6 options=21 groups: lo igb0: flags=8843 metric 0 mtu 1500 options=e505bb ether 0c:c4:7a:a8:cd:da inet6 fe80::ec4:7aff:fea8:cdda%igb0 prefixlen 64 scopeid 0x1 nd6 options=21 media: Ethernet autoselect status: no carrier igb1: flags=8802 metric 0 mtu 1500 options=e505bb ether 0c:c4:7a:a8:cd:db nd6 options=29 media: Ethernet autoselect status: no carrier Starting devd. Starting Network: igb1. igb1: flags=8802 metric 0 mtu 1500 options=e505bb ether 0c:c4:7a:a8:cd:db nd6 options=29 media: Ethernet autoselect status: no carrier ums0 on uhub5 ums0: on usbus0 Starting pflogd:ums0: 3 buttons and [Z] coordinates ID=0 add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Creating and/or trimming log files. Starting syslogd. Setting date via ntp. 9 Apr 20:43:48 ntpdate[529]: no servers can be used, exiting Starting rpcbind. No core dumps found. ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib 32-bit compatibility ldconfig path: /usr/lib32 NFS access cache time=60 rpc.umntall: 192.168.5.1: MOUNTPROG: RPC: Port mapper failure - RPC: Unable to send Starting casperd. lock order reversal: 1st 0xfffffe0000ed51e8 bufwait (bufwait) @ kern/vfs_bio.c:3893 2nd 0xfffff800134f3400 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:289 stack backtrace: #0 0xffffffff80ba7753 at witness_debugger+0x73 #1 0xffffffff80ba75d4 at witness_checkorder+0xe34 #2 0xffffffff80b49688 at _sx_xlock+0x68 #3 0xffffffff80e4d3dd at ufsdirhash_add+0x4d #4 0xffffffff80e51eff at ufs_direnter+0x49f #5 0xffffffff80e5a4fc at ufs_mkdir+0x8dc #6 0xffffffff8117be8d at VOP_MKDIR_APV+0xfd #7 0xffffffff80c20fe6 at kern_mkdirat+0x1d6 #8 0xffffffff8100607b at amd64_syscall+0x7ab #9 0xffffffff80fe0e0d at fast_syscall_common+0x101 Clearing /tmp (X related). Starting statd. Starting lockd. Updating motd:Link state changed to up . Mounting late file systems:. Configuring syscons: blanktime. Performing sanitStarting sshd. Starting cron. Mon Apr 9 20:43 FreeBSD/amd64 (mercat1.netperf.freebsd.org) (ttyu1) login: root Apr 9 20:43:51 mercat1 login: ROOT LOGIN (root) ON ttyu1 Last login: Sun Apr 8 22:47:12 on ttyu1 FreeBSD 12.0-CURRENT (PHO) #0 r332299M: Mon Apr 9 20:32:57 CEST 2018 root@mercat1:~ # Apr 9 20:44:03 mercat1 su: pho to root on /dev/pts/0 lock order reversal: 1st 0xfffff80013edea48 ufs (ufs) @ kern/vfs_subr.c:2570 2nd 0xfffffe0000ed51e8 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:282 3rd 0xfffff800213a3a48 ufs (ufs) @ kern/vfs_subr.c:2570 stack backtrace: #0 0xffffffff80ba7753 at witness_debugger+0x73 #1 0xffffffff80ba75d4 at witness_checkorder+0xe34 #2 0xffffffff80b15d4b at lockmgr_xlock_hard+0x6b #3 0xffffffff80b1672a at __lockmgr_args+0x51a #4 0xffffffff80e4819f at ffs_lock+0xaf #5 0xffffffff8117cba1 at VOP_LOCK1_APV+0xe1 #6 0xffffffff80c23ee6 at _vn_lock+0x66 #7 0xffffffff80c12288 at vget+0x98 #8 0xffffffff80c041f1 at vfs_hash_get+0xd1 #9 0xffffffff80e43a4f at ffs_vgetf+0x3f #10 0xffffffff80e391cd at softdep_sync_buf+0x5dd #11 0xffffffff80e48fa4 at ffs_syncvnode+0x294 #12 0xffffffff80e4673d at ffs_sync+0x20d #13 0xffffffff80c0a5b5 at dounmount+0x555 #14 0xffffffff80c09fd0 at sys_unmount+0x310 #15 0xffffffff8100607b at amd64_syscall+0x7ab #16 0xffffffff80fe0e0d at fast_syscall_common+0x101 lock order reversal: 1st 0xfffff80013bf9cc0 ufs (ufs) @ kern/vfs_mount.c:1335 2nd 0xfffff80013edecc0 devfs (devfs) @ ufs/ffs/ffs_softdep.c:1914 stack backtrace: #0 0xffffffff80ba7753 at witness_debugger+0x73 #1 0xffffffff80ba75d4 at witness_checkorder+0xe34 #2 0xffffffff80b1587e at lockmgr_lock_fast_path+0x17e #3 0xffffffff8117cba1 at VOP_LOCK1_APV+0xe1 #4 0xffffffff80c23ee6 at _vn_lock+0x66 #5 0xffffffff80e2692c at softdep_flushworklist+0x6c #6 0xffffffff80e46936 at ffs_sync+0x406 #7 0xffffffff80c0a5b5 at dounmount+0x555 #8 0xffffffff80c09fd0 at sys_unmount+0x310 #9 0xffffffff8100607b at amd64_syscall+0x7ab #10 0xffffffff80fe0e0d at fast_syscall_common+0x101 20180409 20:45:58 all (1/281): nullfs.sh 20180409 20:50:59 all (2/281): nullfs10.sh 20180409 20:51:01 all (3/281): nullfs11.sh 20180409 20:52:12 all (4/281): nullfs12.sh 20180409 20:52:14 all (5/281): nullfs13.sh 20180409 20:52:15 all (6/281): nullfs14.sh 20180409 20:52:15 all (7/281): nullfs15.sh newblk leak: 295/304. 20180409 20:52:16 all (8/281): nullfs16.sh 20180409 20:52:17 all (9/281): nullfs17.sh 20180409 20:57:18 all (10/281): nullfs18.sh 20180409 20:59:28 all (11/281): nullfs19.sh 20180409 21:02:22 all (12/281): nullfs2.sh Apr 9 21:05:21 mercat1 kernel: pid 56750 (swap), uid 0, was killed: out of swap space Apr 9 21:05:24 mercat1 kernel: pid 56804 (swap), uid 0, was killed: out of swap space Apr 9 21:12:44 mercat1 kernel: pid 58678 (swap), uid 0, was killed: out of swap space Apr 9 21:12:59 mercat1 kernel: pid 58656 (swap), uid 0, was killed: out of swap space Apr 9 21:13:01 mercat1 kernel: pid 58663 (swap), uid 0, was killed: out of swap space 20180409 21:13:28 all (13/281): nullfs20.sh 20180409 21:23:53 all (14/281): nullfs21.sh 20180409 21:23:53 all (15/281): nullfs22.sh 20180409 21:24:38 all (16/281): nullfs23.sh 20180409 21:24:38 all (17/281): nullfs3.sh 20180409 21:24:40 all (18/281): nullfs4.sh mount leak: 13/44 20180409 21:24:40 all (19/281): nullfs5.sh Apr 9 21:34:19 mercat1 kernel: pid 28763 (swap), uid 0, was killed: out of swap space Apr 9 21:34:31 mercat1 kernel: pid 28785 (swap), uid 0, was killed: out of swap space Apr 9 21:34:31 mercat1 kernel: pid 28769 (swap), uid 0, was killed: out of swap space Apr 9 21:34:31 mercat1 kernel: pid 28860 (swap), uid 0, was killed: out of swap space 20180409 21:37:00 all (20/281): nullfs6.sh 20180409 21:37:01 all (21/281): nullfs7.sh 20180409 21:37:02 all (22/281): nullfs9.sh 20180409 21:37:02 all (23/281): oom.sh 20180409 21:37:03 all (24/281): oovm.sh lock order reversal: 1st 0xfffff80021f987d0 syncer (syncer) @ kern/vfs_subr.c:2151 2nd 0xfffff803efd38068 ufs (ufs) @ kern/vfs_subr.c:2570 stack backtrace: #0 0xffffffff80ba7753 at witness_debugger+0x73 #1 0xffffffff80ba75d4 at witness_checkorder+0xe34 #2 0xffffffff80b15d4b at lockmgr_xlock_hard+0x6b #3 0xffffffff80b1672a at __lockmgr_args+0x51a #4 0xffffffff80e4819f at ffs_lock+0xaf #5 0xffffffff8117cba1 at VOP_LOCK1_APV+0xe1 #6 0xffffffff80c23ee6 at _vn_lock+0x66 #7 0xffffffff80c12288 at vget+0x98 #8 0xffffffff80c145d6 at vfs_msync+0xa6 #9 0xffffffff80c195f6 at sync_fsync+0xc6 #10 0xffffffff8117b71d at VOP_FSYNC_APV+0xfd #11 0xffffffff80c17464 at sched_sync+0x284 #12 0xffffffff80b01494 at fork_exit+0x84 #13 0xffffffff80fe12de at fork_trampoline+0xe 20180409 21:47:55 all (25/281): oovm2.sh 20180409 21:58:21 all (26/281): open.sh 20180409 21:58:21 all (27/281): openlock.sh 20180409 21:58:23 all (28/281): overcommit.sh Apr 9 21:59:24 mercat1 kernel: pid 56725 (swap), uid 2007, was killed: out of swap space Apr 9 22:03:36 mercat1 kernel: pid 57552 (swap), uid 2007, was killed: out of swap space Apr 9 22:06:41 mercat1 kernel: pid 57646 (swap), uid 2007, was killed: out of swap space Apr 9 22:08:53 mercat1 kernel: pid 57972 (swap), uid 2007, was killed: out of swap space Apr 9 22:08:57 mercat1 kernel: pid 57989 (swap), uid 2007, was killed: out of swap space 20180409 22:09:43 all (29/281): overcommit2.sh Apr 9 22:11:12 mercat1 kernel: pid 58393 (dd), uid 0 inumber 10 on /mnt: filesystem full 20180409 22:11:17 all (30/281): overlap.sh 20180409 22:16:20 all (31/281): parallelmount.sh GEOM leak: 1410/1483 20180409 22:26:20 all (32/281): parallelmount2.sh 20180409 22:26:21 all (33/281): pathconf.sh 20180409 22:26:21 all (34/281): pathconf2.sh 20180409 22:26:22 all (35/281): pause.sh 20180409 22:31:27 all (36/281): pcatch.sh 20180409 22:31:28 all (37/281): pcatch2.sh 20180409 22:31:28 all (38/281): pfl.sh 20180409 22:35:12 all (39/281): pfl2.sh 20180409 22:46:38 all (40/281): pfl4.sh 20180409 22:57:07 all (41/281): pipe.sh 20180409 23:02:08 all (42/281): pipe2.sh 20180409 23:02:22 all (43/281): pmc.sh freework leak: 542/718. 20180409 23:08:29 all (44/281): pmc2.sh newblk leak: 15642/15946. 20180409 23:18:33 all (45/281): pmc3.sh 20180409 23:24:34 all (46/281): pmc4.sh lock order reversal: 1st 0xffffffff824cf620 pmc-kthread (pmc-sleep) @ /usr/src/sys/dev/hwpmc/hwpmc_logging.c:613 2nd 0xfffff8020052f120 process lock (process lock) @ /usr/src/sys/dev/hwpmc/hwpmc_logging.c:616 stack backtrace: #0 0xffffffff80ba7753 at witness_debugger+0x73 #1 0xffffffff80ba75d4 at witness_checkorder+0xe34 #2 0xffffffff80b2001c at __mtx_lock_flags+0x9c #3 0xffffffff824bd4fe at pmclog_deconfigure_log+0x7e #4 0xffffffff824b68f6 at pmc_process_exit+0x4e6 #5 0xffffffff80afc11e at exit1+0x39e #6 0xffffffff80afbd7d at sys_sys_exit+0xd #7 0xffffffff8100607b at amd64_syscall+0x7ab #8 0xffffffff80fe0e0d at fast_syscall_common+0x101 lock order reversal: 1st 0xffffffff81a03d80 proctree (proctree) @ kern/imgact_elf.c:2099 2nd 0xfffff801a5045558 ufs (ufs) @ kern/vfs_vnops.c:568 stack backtrace: #0 0xffffffff80ba7753 at witness_debugger+0x73 #1 0xffffffff80ba75d4 at witness_checkorder+0xe34 #2 0xffffffff80b15d4b at lockmgr_xlock_hard+0x6b #3 0xffffffff80b1672a at __lockmgr_args+0x51a #4 0xffffffff80e4819f at ffs_lock+0xaf #5 0xffffffff8117cba1 at VOP_LOCK1_APV+0xe1 #6 0xffffffff80c23ee6 at _vn_lock+0x66 #7 0xffffffff80c24269 at vn_rdwr+0x129 #8 0xffffffff80c24883 at vn_rdwr_inchunks+0x93 #9 0xffffffff80acf7d8 at sbuf_drain_core_output+0xa8 #10 0xffffffff80b91ea8 at sbuf_drain+0x58 #11 0xffffffff80b9135f at sbuf_put_bytes+0x8f #12 0xffffffff80b912be at sbuf_bcat+0xe #13 0xffffffff80b27c6b at kern_proc_out+0xeb #14 0xffffffff80ad00c3 at elf64_note_procstat_proc+0x93 #15 0xffffffff80acf116 at elf64_coredump+0xc06 #16 0xffffffff80b44ee2 at sigexit+0x7e2 #17 0xffffffff80b45760 at postsig+0x1c0 20180409 23:24:38 all (47/281): pmc5.sh 20180409 23:24:40 all (48/281): posix_fadvise.sh 20180409 23:24:41 all (49/281): posix_fadvise2.sh 20180409 23:24:41 all (50/281): posix_fadvise3.sh 20180409 23:28:36 all (51/281): posix_openpt.sh 20180409 23:29:22 all (52/281): posix_openpt2.sh pts leak: 56/57 20180409 23:35:38 all (53/281): pread.sh 20180409 23:35:49 all (54/281): procfs.sh 20180409 23:35:58 all (55/281): procfs2.sh 20180409 23:36:00 all (56/281): procfs3.sh 20180409 23:36:09 all (57/281): procfs4.sh 20180409 23:42:24 all (58/281): procfs5.sh 20180409 23:42:25 all (59/281): pshared.sh 20180409 23:42:30 all (60/281): pthread.sh 20180409 23:42:34 all (61/281): pthread2.sh 20180409 23:44:33 all (62/281): pthread3.sh 20180409 23:51:01 all (63/281): pthread4.sh 20180409 23:54:16 all (64/281): pthread5.sh witness_lock_list_get: witness exhausted 20180409 23:55:18 all (65/281): pthread6.sh 20180409 23:59:24 all (66/281): pthread7.sh 20180410 00:01:28 all (67/281): pthread8.sh 20180410 00:04:28 all (68/281): pthread9.sh 20180410 00:04:29 all (69/281): ptrace.sh 20180410 00:04:29 all (70/281): ptrace10.sh 20180410 00:06:17 all (71/281): ptrace11.sh 20180410 00:06:27 all (72/281): ptrace2.sh 20180410 00:06:31 all (73/281): ptrace3.sh 20180410 00:06:32 all (74/281): ptrace4.sh 20180410 00:11:00 all (75/281): ptrace5.sh 20180410 00:16:00 all (76/281): ptrace6.sh 20180410 00:20:01 all (77/281): ptrace7.sh 20180410 00:24:02 all (78/281): ptrace8.sh 20180410 00:24:03 all (79/281): ptrace9.sh 20180410 00:24:09 all (80/281): pts.sh pts leak: 54/111 20180410 00:36:51 all (81/281): pts2.sh 20180410 00:36:51 all (82/281): pty.sh pts leak: 14/125 20180410 00:41:52 all (83/281): pty2.sh 20180410 00:44:52 all (84/281): quota1.sh 20180410 00:55:27 all (85/281): quota10.sh 20180410 00:56:03 all (86/281): quota11.sh 20180410 00:56:12 all (87/281): quota4.sh 20180410 00:57:19 all (88/281): quota5.sh 20180410 00:57:19 all (89/281): quota8.sh 20180410 01:00:43 all (90/281): quota9.sh 20180410 01:00:45 all (91/281): radix.sh 20180410 01:11:14 all (92/281): rdgsbase.sh 20180410 01:11:26 all (93/281): rdwr.sh 20180410 01:11:57 all (94/281): readdir.sh 20180410 01:25:00 all (95/281): recursiveflushes.sh 20180410 02:00:00 *** Excessive run time: recursiveflushes.sh, 35 min 20180410 02:05:00 all (96/281): rename.sh 20180410 02:16:00 all (97/281): rename10.sh 20180410 02:22:09 all (98/281): rename11.sh 20180410 02:30:00 all (99/281): rename12.sh 20180410 03:30:02 *** Excessive run time: rename12.sh, 60 min mount leak: 13/57 GEOM leak: 23/1506 20180410 04:11:27 all (100/281): rename13.sh 20180410 04:11:28 all (101/281): rename2.sh 20180410 04:36:28 all (102/281): rename3.sh 20180410 05:36:29 *** Excessive run time: rename3.sh, 60 min 20180410 05:36:30 all (103/281): rename4.sh 20180410 05:36:31 all (104/281): rename5.sh 20180410 05:46:32 all (105/281): rename6.sh 20180410 05:50:00 all (106/281): rename7.sh 20180410 05:55:00 all (107/281): rename8.sh 20180410 06:00:00 all (108/281): rename9.sh 20180410 06:01:32 all (109/281): revoke.sh 20180410 06:06:32 all (110/281): rot.sh 20180410 08:50:41 *** Excessive run time: rot.sh, 164 min root@mercat1:~ # KDB: enter: Break to debugger [ thread pid 11 tid 100003 ] Stopped at kdb_alt_break_internal+0x106: movq $0,kdb_why db> ps pid ppid pgrp uid state wmesg wchan cmd 84544 84536 805 0 R+ CPU 4 sh 84537 84535 805 0 S+ piperd 0xfffff807966815f0 grep 84536 84535 805 0 S+ wait 0xfffff8049b177a70 sh 84535 84532 805 0 S+ wait 0xfffff808175c7538 sh 84532 84531 805 0 S+ piperd 0xfffff8011a5b35f0 sh 84531 2109 805 0 S+ wait 0xfffff807953ac000 sh 84347 0 0 0 DL mdwait 0xfffff8062b079000 [md10] 2109 805 805 0 S+ wait 0xfffff8002112b538 sh 2108 789 2108 0 S+ select 0xfffff800135c4340 top 805 789 805 0 SW+ wait 0xfffff8000362c538 sh 789 787 789 0 SW+ wait 0xfffff80013dae538 bash 787 786 787 0 SW+ pause 0xfffff80021129b18 csh 786 785 786 2006 SW+ wait 0xfffff80013fd7538 su 785 784 785 2006 SWs+ wait 0xfffff80013d75a70 bash 784 777 777 2006 S select 0xfffff8000306c6c0 sshd 777 726 777 0 Ss select 0xfffff800060af2c0 sshd 774 773 774 0 S+ ttyin 0xfffff80006194cb0 csh 773 1 773 0 SWs+ wait 0xfffff80013d76538 login 772 1 772 0 Ss+ ttyin 0xfffff800061950b0 getty --More-- 771 1 771 0 Ss+ ttyin 0xfffff800135318b0 getty 770 1 770 0 Ss+ ttyin 0xfffff80013531cb0 getty 769 1 769 0 Ss+ ttyin 0xfffff800135320b0 getty 768 1 768 0 Ss+ ttyin 0xfffff800135324b0 getty 767 1 767 0 Ss+ ttyin 0xfffff800135328b0 getty 766 1 766 0 Ss+ ttyin 0xfffff80013532cb0 getty 765 1 765 0 Ss+ ttyin 0xfffff800135330b0 getty 764 1 764 0 Ss+ ttyin 0xfffff800135334b0 getty 730 1 730 0 Ss nanslp 0xffffffff81df2a2a cron 726 1 726 0 Ss select 0xfffff80006070d40 sshd 724 1 724 65 Ss select 0xfffff800135f0340 dhclient 667 1 667 0 Ss select 0xfffff80013585b40 dhclient 658 1 658 0 Ss select 0xfffff8000306d4c0 dhclient 602 1 602 0 Ss rpcsvc 0xfffff800212e2560 NLM: master 599 1 599 0 Ss select 0xfffff80003901540 rpc.statd 574 1 574 0 Ss select 0xfffff800060af340 casperd 573 1 573 0 Ss select 0xfffff8000306d240 casperd 532 1 532 0 Ss select 0xfffff800135856c0 rpcbind 527 1 527 0 Ss select 0xfffff80013585540 syslogd 452 1 452 0 Ss select 0xfffff800135af1c0 devd --More-- db> show page vm_cnt.v_free_count: 7552197 vm_cnt.v_inactive_count: 101999 vm_cnt.v_active_count: 30 vm_cnt.v_laundry_count: 137 vm_cnt.v_wire_count: 521178 vm_cnt.v_free_reserved: 10651 vm_cnt.v_free_min: 51298 vm_cnt.v_free_target: 173239 vm_cnt.v_inactive_target: 259858 db> c /root: Permission denied. root@mercat1:~ # root@mercat1:~ # 20180410 08:51:26 all (111/281): routetbl.sh 20180410 08:51:27 all (112/281): ruby.sh 20180410 08:55:44 all (113/281): rw.sh 20180410 09:11:01 all (114/281): rwlock_ronly.sh 20180410 09:25:00 all (115/281): sched.sh 20180410 09:28:32 all (116/281): schedfuzz.sh 20180410 09:28:33 all (117/281): seekdir.sh mount leak: 3/60 20180410 09:28:36 all (118/281): segnp.sh 20180410 09:28:37 all (119/281): segregs.sh 20180410 09:28:38 all (120/281): select.sh 20180410 09:40:01 all (121/281): select2.sh 20180410 09:40:09 all (122/281): select3.sh 20180410 09:44:00 all (123/281): selfd.sh 20180410 09:54:03 all (124/281): sem.sh 20180410 13:46:09 *** Excessive run time: sem.sh, 232 min routetbl leak: 1/27 20180410 13:46:11 all (125/281): sem_timedwait.sh Apr 10 13:46:17 mercat1 date: date set by pho Apr 10 13:46:28 mercat1 last message repeated 60 times Apr 10 13:46:28 mercat1 su: pho to root on /dev/pts/92 Apr 10 13:46:30 mercat1 date: date set by pho pts leak: 1/126 Apr 10 13:46:29 mercat1 last message repeated 8 times 20180410 13:50:00 all (126/281): sendfile.sh 20180410 13:50:01 all (127/281): sendfile10.sh 20180410 13:56:12 all (128/281): sendfile12.sh Expensive timeout(9) function: 0xffffffff80d63a30(0xfffff806343e8000) 1.891559838 s 20180410 14:16:11 all (129/281): sendfile14.sh root@mercat1:~ # pgrep sendfile14 | xargs ps -lp KDB: enter: Break to debugger [ thread pid 11 tid 100003 ] Stopped at kdb_alt_break_internal+0x106: movq $0,kdb_why db> ps pid ppid pgrp uid state wmesg wchan cmd 92137 92090 805 0 D+ pfault 0xffffffff81a49880 sleep 92127 92090 805 0 D+ pfault 0xffffffff81a49880 sendfile14 92126 92090 805 0 D+ pfault 0xffffffff81a49880 sendfile14 92118 0 0 0 DL mdwait 0xfffff8071790c000 [md11] 92109 0 0 0 DL mdwait 0xfffff8030a17e000 [md10] 92090 92088 805 0 S+ pause 0xfffff80834a6d0a8 timeout 92089 2109 805 0 S+ piperd 0xfffff805d46808e8 tee 92088 2109 805 0 S+ wait 0xfffff802e71e6000 sh 90589 90571 90589 0 S+ select 0xfffff80110da7740 bash 90571 90570 90571 0 S+ pause 0xfffff807c868a5e0 csh 90570 90198 90570 2006 S+ wait 0xfffff80326152538 su 90198 90184 90198 2006 Ss+ wait 0xfffff802c5d46000 bash 90184 89765 89765 2006 S select 0xfffff8032d8b74c0 sshd 89765 726 89765 0 Ss select 0xfffff804245f60c0 sshd 85572 789 85572 0 D+ pfault 0xffffffff81a49880 top 2109 805 805 0 S+ wait 0xfffff8002112b538 sh 805 789 805 0 SW+ wait 0xfffff8000362c538 sh 789 787 789 0 S+ wait 0xfffff80013dae538 bash 787 786 787 0 SW+ pause 0xfffff80021129b18 csh --More-- db> show page vm_cnt.v_free_count: 9511 vm_cnt.v_inactive_count: 7276652 vm_cnt.v_active_count: 28 vm_cnt.v_laundry_count: 14 vm_cnt.v_wire_count: 843078 vm_cnt.v_free_reserved: 10651 vm_cnt.v_free_min: 51298 vm_cnt.v_free_target: 173239 vm_cnt.v_inactive_target: 259858 db> run pho db:0:pho> run pho1 db:1:pho1> bt Tracing pid 11 tid 100003 td 0xfffff80003632000 kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db:1:pho1> show allpcpu Current CPU: 0 cpuid = 0 dynamic pcpu = 0xa1b180 curthread = 0xfffff80003632000: pid 11 tid 100003 "idle: cpu0" curpcb = 0xfffffe007895ab80 fpcurthread = none idlethread = 0xfffff80003632000: tid 100003 "idle: cpu0" curpmap = 0xffffffff81febf20 tssp = 0xffffffff82016810 commontssp = 0xffffffff82016810 rsp0 = 0xfffffe007895ab80 gs32p = 0xffffffff8201d058 ldt = 0xffffffff8201d098 tss = 0xffffffff8201d088 curvnet = 0 spin locks held: cpuid = 1 dynamic pcpu = 0xfffffe008561a180 curthread = 0xfffff80003631560: pid 11 tid 100004 "idle: cpu1" curpcb = 0xfffffe007895fb80 fpcurthread = none idlethread = 0xfffff80003631560: tid 100004 "idle: cpu1" curpmap = 0xffffffff81febf20 tssp = 0xffffffff82016878 commontssp = 0xffffffff82016878 rsp0 = 0xfffffe007895fb80 gs32p = 0xffffffff8201d0c0 ldt = 0xffffffff8201d100 tss = 0xffffffff8201d0f0 curvnet = 0 spin locks held: cpuid = 2 dynamic pcpu = 0xfffffe0085625180 curthread = 0xfffff80003631000: pid 11 tid 100005 "idle: cpu2" curpcb = 0xfffffe0078964b80 fpcurthread = none idlethread = 0xfffff80003631000: tid 100005 "idle: cpu2" curpmap = 0xffffffff81febf20 tssp = 0xffffffff820168e0 commontssp = 0xffffffff820168e0 rsp0 = 0xfffffe0078964b80 gs32p = 0xffffffff8201d128 ldt = 0xffffffff8201d168 tss = 0xffffffff8201d158 curvnet = 0 spin locks held: cpuid = 3 dynamic pcpu = 0xfffffe0085630180 curthread = 0xfffff80003630560: pid 11 tid 100006 "idle: cpu3" curpcb = 0xfffffe0078969b80 fpcurthread = none idlethread = 0xfffff80003630560: tid 100006 "idle: cpu3" curpmap = 0xffffffff81febf20 tssp = 0xffffffff82016948 commontssp = 0xffffffff82016948 rsp0 = 0xfffffe0078969b80 gs32p = 0xffffffff8201d190 ldt = 0xffffffff8201d1d0 tss = 0xffffffff8201d1c0 curvnet = 0 spin locks held: cpuid = 4 dynamic pcpu = 0xfffffe008563b180 curthread = 0xfffff80003630000: pid 11 tid 100007 "idle: cpu4" curpcb = 0xfffffe007896eb80 fpcurthread = none idlethread = 0xfffff80003630000: tid 100007 "idle: cpu4" curpmap = 0xffffffff81febf20 tssp = 0xffffffff820169b0 commontssp = 0xffffffff820169b0 rsp0 = 0xfffffe007896eb80 gs32p = 0xffffffff8201d1f8 ldt = 0xffffffff8201d238 tss = 0xffffffff8201d228 curvnet = 0 spin locks held: cpuid = 5 dynamic pcpu = 0xfffffe0085646180 curthread = 0xfffff8000362f560: pid 11 tid 100008 "idle: cpu5" curpcb = 0xfffffe0078973b80 fpcurthread = none idlethread = 0xfffff8000362f560: tid 100008 "idle: cpu5" curpmap = 0xffffffff81febf20 tssp = 0xffffffff82016a18 commontssp = 0xffffffff82016a18 rsp0 = 0xfffffe0078973b80 gs32p = 0xffffffff8201d260 ldt = 0xffffffff8201d2a0 tss = 0xffffffff8201d290 curvnet = 0 spin locks held: cpuid = 6 dynamic pcpu = 0xfffffe0085651180 curthread = 0xfffff8000362f000: pid 11 tid 100009 "idle: cpu6" curpcb = 0xfffffe0078978b80 fpcurthread = none idlethread = 0xfffff8000362f000: tid 100009 "idle: cpu6" curpmap = 0xffffffff81febf20 tssp = 0xffffffff82016a80 commontssp = 0xffffffff82016a80 rsp0 = 0xfffffe0078978b80 gs32p = 0xffffffff8201d2c8 ldt = 0xffffffff8201d308 tss = 0xffffffff8201d2f8 curvnet = 0 spin locks held: cpuid = 7 dynamic pcpu = 0xfffffe008565c180 curthread = 0xfffff80003644560: pid 11 tid 100010 "idle: cpu7" curpcb = 0xfffffe007897db80 fpcurthread = none idlethread = 0xfffff80003644560: tid 100010 "idle: cpu7" curpmap = 0xffffffff81febf20 tssp = 0xffffffff82016ae8 commontssp = 0xffffffff82016ae8 rsp0 = 0xfffffe007897db80 gs32p = 0xffffffff8201d330 ldt = 0xffffffff8201d370 tss = 0xffffffff8201d360 curvnet = 0 spin locks held: cpuid = 8 dynamic pcpu = 0xfffffe0085667180 curthread = 0xfffff80003644000: pid 11 tid 100011 "idle: cpu8" curpcb = 0xfffffe0078982b80 fpcurthread = none idlethread = 0xfffff80003644000: tid 100011 "idle: cpu8" curpmap = 0xffffffff81febf20 tssp = 0xffffffff82016b50 commontssp = 0xffffffff82016b50 rsp0 = 0xfffffe0078982b80 gs32p = 0xffffffff8201d398 ldt = 0xffffffff8201d3d8 tss = 0xffffffff8201d3c8 curvnet = 0 spin locks held: cpuid = 9 dynamic pcpu = 0xfffffe007f11a180 curthread = 0xfffff80003643560: pid 11 tid 100012 "idle: cpu9" curpcb = 0xfffffe0078987b80 fpcurthread = none idlethread = 0xfffff80003643560: tid 100012 "idle: cpu9" curpmap = 0xffffffff81febf20 tssp = 0xffffffff82016bb8 commontssp = 0xffffffff82016bb8 rsp0 = 0xfffffe0078987b80 gs32p = 0xffffffff8201d400 ldt = 0xffffffff8201d440 tss = 0xffffffff8201d430 curvnet = 0 spin locks held: cpuid = 10 dynamic pcpu = 0xfffffe007f125180 curthread = 0xfffff80003643000: pid 11 tid 100013 "idle: cpu10" curpcb = 0xfffffe007898cb80 fpcurthread = none idlethread = 0xfffff80003643000: tid 100013 "idle: cpu10" curpmap = 0xffffffff81febf20 tssp = 0xffffffff82016c20 commontssp = 0xffffffff82016c20 rsp0 = 0xfffffe007898cb80 gs32p = 0xffffffff8201d468 ldt = 0xffffffff8201d4a8 tss = 0xffffffff8201d498 curvnet = 0 spin locks held: cpuid = 11 dynamic pcpu = 0xfffffe007f130180 curthread = 0xfffff800133f0560: pid 29 tid 100126 "dom0" curpcb = 0xfffffe00005c3b80 fpcurthread = none idlethread = 0xfffff80003642560: tid 100014 "idle: cpu11" curpmap = 0xffffffff81febf20 tssp = 0xffffffff82016c88 commontssp = 0xffffffff82016c88 rsp0 = 0xfffffe00005c3b80 gs32p = 0xffffffff8201d4d0 ldt = 0xffffffff8201d510 tss = 0xffffffff8201d500 curvnet = 0 spin locks held: db:1:pho1> show alllocks db:1:pho1> show lockedvnods Locked vnodes db:1:pho1> show allchains chain 1: thread 104710 (pid 92137, sleep) sleeping on 0xffffffff81a49880 "pfault" chain 2: thread 104565 (pid 92127, sendfile14) sleeping on 0xffffffff81a49880 "pfault" chain 3: thread 105410 (pid 92126, sendfile14) sleeping on 0xffffffff81a49880 "pfault" chain 4: thread 100154 (pid 92118, md11) sleeping on 0xfffff8071790c000 "mdwait" chain 5: thread 104679 (pid 92109, md10) sleeping on 0xfffff8030a17e000 "mdwait" chain 6: thread 102999 (pid 92090, timeout) sleeping on 0xfffff80834a6d0a8 "pause" chain 7: thread 104958 (pid 92089, tee) sleeping on 0xfffff805d46808e8 "piperd" chain 8: thread 103323 (pid 92088, sh) sleeping on 0xfffff802e71e6000 "wait" chain 9: thread 105387 (pid 90589, bash) sleeping on 0xfffff80110da7740 "select" chain 10: thread 105316 (pid 90571, csh) sleeping on 0xfffff807c868a5e0 "pause" chain 11: thread 104671 (pid 90570, su) sleeping on 0xfffff80326152538 "wait" chain 12: thread 105394 (pid 90198, bash) blocked on lockmgr 8UmEXCL thread 0 (pid -512, ) ??? (0x6) chain 13: thread 104451 (pid 90184, sshd) sleeping on 0xfffff8032d8b74c0 "select" chain 14: thread 105178 (pid 89765, sshd) sleeping on 0xfffff804245f60c0 "select" chain 15: thread 103304 (pid 85572, top) sleeping on 0xffffffff81a49880 "pfault" chain 16: thread 100168 (pid 2109, sh) sleeping on 0xfffff8002112b538 "wait" chain 17: thread 100179 (pid 805, sh) sleeping on 0xfffff8000362c538 "wait" chain 18: thread 100201 (pid 789, bash) sleeping on 0xfffff80013dae538 "wait" chain 19: thread 100190 (pid 787, csh) sleeping on 0xfffff80021129b18 "pause" chain 20: thread 100160 (pid 786, su) sleeping on 0xfffff80013fd7538 "wait" chain 21: thread 100200 (pid 785, bash) sleeping on 0xfffff80013d75a70 "wait" chain 22: thread 100198 (pid 784, sshd) sleeping on 0xfffff8000306c6c0 "select" chain 23: thread 100182 (pid 777, sshd) sleeping on 0xfffff800060af2c0 "select" chain 24: thread 100170 (pid 774, csh) sleeping on 0xffffffff81a49880 "pfault" chain 25: thread 100157 (pid 773, login) sleeping on 0xfffff80013d76538 "wait" chain 26: thread 100197 (pid 772, getty) sleeping on 0xfffff800061950b0 "ttyin" chain 27: thread 100196 (pid 771, getty) sleeping on 0xfffff800135318b0 "ttyin" chain 28: thread 100195 (pid 770, getty) sleeping on 0xfffff80013531cb0 "ttyin" chain 29: thread 100194 (pid 769, getty) sleeping on 0xfffff800135320b0 "ttyin" chain 30: thread 100193 (pid 768, getty) sleeping on 0xfffff800135324b0 "ttyin" chain 31: thread 100192 (pid 767, getty) sleeping on 0xfffff800135328b0 "ttyin" chain 32: thread 100165 (pid 766, getty) sleeping on 0xfffff80013532cb0 "ttyin" chain 33: thread 100175 (pid 765, getty) sleeping on 0xfffff800135330b0 "ttyin" chain 34: thread 100140 (pid 764, getty) sleeping on 0xfffff800135334b0 "ttyin" chain 35: thread 100183 (pid 730, cron) sleeping on 0xffffffff81a49880 "vmwait" chain 36: thread 100185 (pid 726, sshd) sleeping on 0xfffff80006070d40 "select" chain 37: thread 100172 (pid 724, dhclient) sleeping on 0xfffff80013f45a50 "piperd" chain 38: thread 100153 (pid 667, dhclient) sleeping on 0xffffffff81a49880 "vmwait" chain 39: thread 100178 (pid 658, dhclient) sleeping on 0xfffff8000306d4c0 "select" chain 40: thread 100156 (pid 602, NLM: master) sleeping on 0xfffff800212e2560 "rpcsvc" chain 41: thread 100151 (pid 599, rpc.statd) sleeping on 0xfffff80003901540 "select" chain 42: thread 100176 (pid 574, casperd) sleeping on 0xfffff800060af340 "select" chain 43: thread 100152 (pid 573, casperd) sleeping on 0xfffff8000306d240 "select" chain 44: thread 100149 (pid 532, rpcbind) sleeping on 0xfffff800135856c0 "select" chain 45: thread 100164 (pid 527, syslogd) sleeping on 0xfffff80013585540 "select" chain 46: thread 100143 (pid 452, devd) sleeping on 0xfffff800135af1c0 "select" chain 47: thread 100131 (pid 33, vnlru) sleeping on 0xfffff800133e5a70 "vlruwt" chain 48: thread 100129 (pid 32, syncer) sleeping on 0xffffffff81f94e40 "syncer" chain 49: thread 100128 (pid 31, bufdaemon) sleeping on 0xffffffff81f942e0 "qsleep" chain 50: thread 100132 (pid 31, bufspacedaemon-0) sleeping on 0xffffffff81a0c1c0 "-" chain 51: thread 100134 (pid 31, bufspacedaemon-1) sleeping on 0xffffffff81a14480 "-" chain 52: thread 100135 (pid 31, bufspacedaemon-2) sleeping on 0xffffffff81a1c740 "-" chain 53: thread 100136 (pid 31, bufspacedaemon-3) sleeping on 0xffffffff81a24a00 "-" chain 54: thread 100137 (pid 31, bufspacedaemon-4) sleeping on 0xffffffff81a2ccc0 "-" chain 55: thread 100138 (pid 31, bufspacedaemon-5) sleeping on 0xffffffff81a34f80 "-" chain 56: thread 100139 (pid 31, bufspacedaemon-6) sleeping on 0xffffffff81a3d240 "-" chain 57: thread 100150 (pid 31, / worker) sleeping on 0xfffff80013eeb4e8 "sdflush" chain 58: thread 100203 (pid 31, /tmp worker) sleeping on 0xfffff80021f878e8 "sdflush" chain 59: thread 100494 (pid 31, /mnt worker) sleeping on 0xfffff80705eb02e8 "sdflush" chain 60: thread 100496 (pid 31, /mnt2 worker) sleeping on 0xfffff803d2c288e8 "sdflush" chain 61: thread 100127 (pid 30, vmdaemon) sleeping on 0xffffffff81fa0bc0 "psleep" chain 62: thread 100130 (pid 29, laundry: dom0) sleeping on 0xffffffff82010064 "launds" chain 63: thread 100133 (pid 29, uma) sleeping on 0xffffffff80e66400 "umarcl" chain 64: thread 100125 (pid 28, enc_daemon1) sleeping on 0xfffff80013348000 "idle" chain 65: thread 100124 (pid 27, enc_daemon0) sleeping on 0xfffff80013348538 "idle" chain 66: thread 100118 (pid 26, rand_harvestq) sleeping on 0xffffffff81cc5958 "-" chain 67: thread 100117 (pid 25, sctp_iterator) sleeping on 0xffffffff8200e880 "waiting_for_work" chain 68: thread 100116 (pid 24, soaiod4) sleeping on 0xffffffff81f93cdc "-" chain 69: thread 100115 (pid 23, soaiod3) sleeping on 0xffffffff81f93cdc "-" chain 70: thread 100114 (pid 22, soaiod2) sleeping on 0xffffffff81f93cdc "-" chain 71: thread 100113 (pid 21, soaiod1) sleeping on 0xffffffff81f93cdc "-" chain 72: thread 100094 (pid 20, usbus0) sleeping on 0xfffffe009c7cb460 "-" chain 73: thread 100095 (pid 20, usbus0) sleeping on 0xfffffe009c7cb4b8 "-" chain 74: thread 100096 (pid 20, usbus0) sleeping on 0xfffffe009c7cb510 "-" chain 75: thread 100097 (pid 20, usbus0) sleeping on 0xfffffe009c7cb568 "-" chain 76: thread 100098 (pid 20, usbus0) sleeping on 0xfffffe009c7cb5c0 "-" chain 77: thread 100100 (pid 20, usbus1) sleeping on 0xfffffe009a5d8d10 "-" chain 78: thread 100101 (pid 20, usbus1) sleeping on 0xfffffe009a5d8d68 "-" chain 79: thread 100102 (pid 20, usbus1) sleeping on 0xfffffe009a5d8dc0 "-" chain 80: thread 100103 (pid 20, usbus1) sleeping on 0xfffffe009a5d8e18 "-" chain 81: thread 100104 (pid 20, usbus1) sleeping on 0xfffffe009a5d8e70 "-" chain 82: thread 100105 (pid 20, usbus2) sleeping on 0xfffffe009ca4cd10 "-" chain 83: thread 100106 (pid 20, usbus2) sleeping on 0xfffffe009ca4cd68 "-" chain 84: thread 100107 (pid 20, usbus2) sleeping on 0xfffffe009ca4cdc0 "-" chain 85: thread 100108 (pid 20, usbus2) sleeping on 0xfffffe009ca4ce18 "-" chain 86: thread 100109 (pid 20, usbus2) sleeping on 0xfffffe009ca4ce70 "-" chain 87: thread 100089 (pid 19, doneq0) sleeping on 0xffffffff81ba0b40 "-" chain 88: thread 100090 (pid 19, doneq1) sleeping on 0xffffffff81ba0bc0 "-" chain 89: thread 100091 (pid 19, doneq2) sleeping on 0xffffffff81ba0c40 "-" chain 90: thread 100123 (pid 19, scanner) sleeping on 0xffffffff81ba0a08 "-" chain 91: thread 100088 (pid 18, crypto returns 11) sleeping on 0xfffff800037f3450 "crypto_ret_wait" chain 92: thread 100087 (pid 17, crypto returns 10) sleeping on 0xfffff800037f33f0 "crypto_ret_wait" chain 93: thread 100086 (pid 16, crypto returns 9) sleeping on 0xfffff800037f3390 "crypto_ret_wait" chain 94: thread 100085 (pid 15, crypto returns 8) sleeping on 0xfffff800037f3330 "crypto_ret_wait" chain 95: thread 100084 (pid 14, crypto returns 7) sleeping on 0xfffff800037f32d0 "crypto_ret_wait" chain 96: thread 100083 (pid 9, crypto returns 6) sleeping on 0xfffff800037f3270 "crypto_ret_wait" chain 97: thread 100082 (pid 8, crypto returns 5) sleeping on 0xfffff800037f3210 "crypto_ret_wait" chain 98: thread 100081 (pid 7, crypto returns 4) sleeping on 0xfffff800037f31b0 "crypto_ret_wait" chain 99: thread 100080 (pid 6, crypto returns 3) sleeping on 0xfffff800037f3150 "crypto_ret_wait" chain 100: thread 100079 (pid 5, crypto returns 2) sleeping on 0xfffff800037f30f0 "crypto_ret_wait" chain 101: thread 100078 (pid 4, crypto returns 1) sleeping on 0xfffff800037f3090 "crypto_ret_wait" chain 102: thread 100077 (pid 3, crypto returns 0) sleeping on 0xfffff800037f3030 "crypto_ret_wait" chain 103: thread 100076 (pid 2, crypto) sleeping on 0xffffffff81f9b1a8 "crypto_wait" chain 104: thread 100060 (pid 13, g_event) sleeping on 0xffffffff81feb2f0 "-" chain 105: thread 100061 (pid 13, g_up) sleeping on 0xffffffff81feb2f8 "-" chain 106: thread 100062 (pid 13, g_down) sleeping on 0xffffffff81feb300 "-" chain 107: thread 100002 (pid 1, init) sleeping on 0xfffff8000362e538 "wait" chain 108: thread 100001 (pid 10, audit) sleeping on 0xffffffff8200f568 "audit_worker_cv" chain 109: thread 100000 (pid 0, swapper) sleeping on 0xffffffff81a49880 "vmwait" chain 110: thread 100015 (pid 0, thread taskq) sleeping on 0xfffff800030b3c00 "-" chain 111: thread 100017 (pid 0, aiod_kick taskq) sleeping on 0xfffff800030b3a00 "-" chain 112: thread 100018 (pid 0, kqueue_ctx taskq) sleeping on 0xfffff800030b3900 "-" chain 113: thread 100021 (pid 0, softirq_0) sleeping on 0xfffff8000361d800 "-" chain 114: thread 100022 (pid 0, softirq_1) sleeping on 0xfffff8000361d600 "-" chain 115: thread 100023 (pid 0, softirq_2) sleeping on 0xfffff8000361d400 "-" chain 116: thread 100024 (pid 0, softirq_3) sleeping on 0xfffff8000361d200 "-" chain 117: thread 100025 (pid 0, softirq_4) sleeping on 0xfffff8000361d000 "-" chain 118: thread 100026 (pid 0, softirq_5) sleeping on 0xfffff8000361cd00 "-" chain 119: thread 100027 (pid 0, softirq_6) sleeping on 0xfffff8000361cb00 "-" chain 120: thread 100028 (pid 0, softirq_7) sleeping on 0xfffff8000361c900 "-" chain 121: thread 100029 (pid 0, softirq_8) sleeping on 0xfffff8000361c700 "-" chain 122: thread 100030 (pid 0, softirq_9) sleeping on 0xfffff8000361c500 "-" chain 123: thread 100031 (pid 0, softirq_10) sleeping on 0xfffff8000361c300 "-" chain 124: thread 100032 (pid 0, softirq_11) sleeping on 0xfffff8000361c100 "-" chain 125: thread 100033 (pid 0, if_config_tqg_0) sleeping on 0xfffff8000361be00 "-" chain 126: thread 100034 (pid 0, if_io_tqg_0) sleeping on 0xfffff80003723100 "-" chain 127: thread 100035 (pid 0, if_io_tqg_1) sleeping on 0xfffff80003722e00 "-" chain 128: thread 100036 (pid 0, if_io_tqg_2) sleeping on 0xfffff80003722c00 "-" chain 129: thread 100037 (pid 0, if_io_tqg_3) sleeping on 0xfffff80003722a00 "-" chain 130: thread 100038 (pid 0, if_io_tqg_4) sleeping on 0xfffff80003722800 "-" chain 131: thread 100039 (pid 0, if_io_tqg_5) sleeping on 0xfffff80003722600 "-" chain 132: thread 100040 (pid 0, if_io_tqg_6) sleeping on 0xfffff80003722400 "-" chain 133: thread 100041 (pid 0, if_io_tqg_7) sleeping on 0xfffff80003722200 "-" chain 134: thread 100042 (pid 0, if_io_tqg_8) sleeping on 0xfffff80003722000 "-" chain 135: thread 100043 (pid 0, if_io_tqg_9) sleeping on 0xfffff80003721d00 "-" chain 136: thread 100044 (pid 0, if_io_tqg_10) sleeping on 0xfffff80003721b00 "-" chain 137: thread 100045 (pid 0, if_io_tqg_11) sleeping on 0xfffff80003721900 "-" chain 138: thread 100063 (pid 0, firmware taskq) sleeping on 0xfffff800037a1100 "-" chain 139: thread 100064 (pid 0, crypto_0) sleeping on 0xfffff800037a1000 "-" chain 140: thread 100065 (pid 0, crypto_1) sleeping on 0xfffff800037a1000 "-" chain 141: thread 100066 (pid 0, crypto_2) sleeping on 0xfffff800037a1000 "-" chain 142: thread 100067 (pid 0, crypto_3) sleeping on 0xfffff800037a1000 "-" chain 143: thread 100068 (pid 0, crypto_4) sleeping on 0xfffff800037a1000 "-" chain 144: thread 100069 (pid 0, crypto_5) sleeping on 0xfffff800037a1000 "-" chain 145: thread 100070 (pid 0, crypto_6) sleeping on 0xfffff800037a1000 "-" chain 146: thread 100071 (pid 0, crypto_7) sleeping on 0xfffff800037a1000 "-" chain 147: thread 100072 (pid 0, crypto_8) sleeping on 0xfffff800037a1000 "-" chain 148: thread 100073 (pid 0, crypto_9) sleeping on 0xfffff800037a1000 "-" chain 149: thread 100074 (pid 0, crypto_10) sleeping on 0xfffff800037a1000 "-" chain 150: thread 100075 (pid 0, crypto_11) sleeping on 0xfffff800037a1000 "-" chain 151: thread 100112 (pid 0, mca taskq) blocked on lockmgr (null)EXCL KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959e00 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959e10 trap() at trap+0x5c/frame 0xfffffe0078959f20 calltrap() at calltrap+0x8/frame 0xfffffe0078959f20 --- trap 0xc, rip = 0xffffffff80b9edda, rsp = 0xfffffe0078959ff0, rbp = 0xfffffe007895a010 --- print_lockchain() at print_lockchain+0x3a/frame 0xfffffe007895a010 db_show_allchains() at db_show_allchains+0x80/frame 0xfffffe007895a040 db_command() at db_command+0x279/frame 0xfffffe007895a110 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80b85fd6, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Script command 'show allchains' returned error db:1:pho1> show mount 0xfffff80013a26000 /dev/gpt/work on / (ufs) 0xfffff80013a27000 devfs on /dev (devfs) 0xfffff80021f81000 /dev/gpt/tmp on /tmp (ufs) 0xfffff80013f5f000 procfs on /proc (procfs) 0xfffff8015578f000 /dev/md10a on /mnt (ufs) 0xfffff80155154000 /dev/md11a on /mnt2 (ufs) More info: show mount db:1:pho1> show page vm_cnt.v_free_count: 9511 vm_cnt.v_inactive_count: 7276652 vm_cnt.v_active_count: 28 vm_cnt.v_laundry_count: 14 vm_cnt.v_wire_count: 843078 vm_cnt.v_free_reserved: 10651 vm_cnt.v_free_min: 51298 vm_cnt.v_free_target: 173239 vm_cnt.v_inactive_target: 259858 db:1:pho1> show freepages DOMAIN: 0 FREE LIST 0: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000000 | 000000 10 (004096K) | 000000 | 000000 09 (002048K) | 000006 | 000000 08 (001024K) | 000000 | 000000 07 (000512K) | 000000 | 000000 06 (000256K) | 000000 | 000000 05 (000128K) | 000001 | 000000 04 (000064K) | 000000 | 000002 03 (000032K) | 000001 | 000002 02 (000016K) | 000000 | 000002 01 (000008K) | 000000 | 000002 00 (000004K) | 000000 | 000020 FREE LIST 1: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000000 | 000000 10 (004096K) | 000000 | 000000 09 (002048K) | 000000 | 000000 08 (001024K) | 000000 | 000000 07 (000512K) | 000000 | 000000 06 (000256K) | 000000 | 000000 05 (000128K) | 000000 | 000000 04 (000064K) | 000000 | 000000 03 (000032K) | 000000 | 000000 02 (000016K) | 000000 | 000000 01 (000008K) | 000000 | 000000 00 (000004K) | 000000 | 000000 db:1:pho1> show pageq pq_free 9511 dom 0 page_cnt 8129739 free 9511 pq_act 28 pq_inact 7276652 pq_laund 14 pq_unsw 0 db:1:pho1> show bufqueues bqempty: 23653 Buf domain 0 freebufs 8019 lofreebufs 219 hifreebufs 329 bufspace 232996864 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 7013 (229801984) dirtyq count 0 (0) wakeup 0 lim 25 CPU 15, 0, 4, 23, 15, 7, 4, 2, 1, 3, 6, 21, Locked buffers: 0 space 0 Total buffers: 15743 space 232996864 Buf domain 1 freebufs 7944 lofreebufs 219 hifreebufs 329 bufspace 235421696 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 7155 (234332160) dirtyq count 0 (0) wakeup 0 lim 25 CPU 4, 0, 1, 13, 0, 2, 1, 2, 1, 1, 1, 8, Locked buffers: 0 space 0 Total buffers: 18199 space 235421696 Buf domain 2 freebufs 7931 lofreebufs 219 hifreebufs 329 bufspace 235433984 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 7090 (232325120) dirtyq count 0 (0) wakeup 0 lim 25 CPU 3, 4, 11, 12, 21, 19, 6, 17, 3, 8, 0, 8, Locked buffers: 0 space 0 Total buffers: 13814 space 235433984 Buf domain 3 freebufs 7941 lofreebufs 219 hifreebufs 329 bufspace 235425792 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 7094 (232370176) dirtyq count 0 (0) wakeup 0 lim 25 CPU 0, 1, 20, 22, 6, 17, 1, 6, 1, 0, 4, 20, Locked buffers: 0 space 0 Total buffers: 11476 space 235425792 Buf domain 4 freebufs 8016 lofreebufs 219 hifreebufs 329 bufspace 233050112 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 7026 (230170624) dirtyq count 0 (0) wakeup 0 lim 25 CPU 15, 2, 20, 7, 8, 6, 8, 3, 1, 7, 4, 10, Locked buffers: 0 space 0 Total buffers: 14847 space 233050112 Buf domain 5 freebufs 7936 lofreebufs 219 hifreebufs 329 bufspace 235446272 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 7104 (232734720) dirtyq count 0 (0) wakeup 0 lim 25 CPU 0, 0, 15, 11, 14, 8, 0, 2, 11, 0, 20, 12, Locked buffers: 0 space 0 Total buffers: 7643 space 235446272 Buf domain 6 freebufs 7944 lofreebufs 219 hifreebufs 329 bufspace 235429888 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 7144 (234094592) dirtyq count 0 (0) wakeup 0 lim 25 CPU 0, 1, 11, 4, 9, 7, 0, 1, 3, 0, 8, 1, Locked buffers: 0 space 0 Total buffers: 24209 space 235429888 db:1:pho1> ps pid ppid pgrp uid state wmesg wchan cmd 92137 92090 805 0 D+ pfault 0xffffffff81a49880 sleep 92127 92090 805 0 D+ pfault 0xffffffff81a49880 sendfile14 92126 92090 805 0 D+ pfault 0xffffffff81a49880 sendfile14 92118 0 0 0 DL mdwait 0xfffff8071790c000 [md11] 92109 0 0 0 DL mdwait 0xfffff8030a17e000 [md10] 92090 92088 805 0 S+ pause 0xfffff80834a6d0a8 timeout 92089 2109 805 0 S+ piperd 0xfffff805d46808e8 tee 92088 2109 805 0 S+ wait 0xfffff802e71e6000 sh 90589 90571 90589 0 S+ select 0xfffff80110da7740 bash 90571 90570 90571 0 S+ pause 0xfffff807c868a5e0 csh 90570 90198 90570 2006 S+ wait 0xfffff80326152538 su 90198 90184 90198 2006 Ss+ wait 0xfffff802c5d46000 bash 90184 89765 89765 2006 S select 0xfffff8032d8b74c0 sshd 89765 726 89765 0 Ss select 0xfffff804245f60c0 sshd 85572 789 85572 0 D+ pfault 0xffffffff81a49880 top 2109 805 805 0 S+ wait 0xfffff8002112b538 sh 805 789 805 0 SW+ wait 0xfffff8000362c538 sh 789 787 789 0 S+ wait 0xfffff80013dae538 bash 787 786 787 0 SW+ pause 0xfffff80021129b18 csh 786 785 786 2006 SW+ wait 0xfffff80013fd7538 su 785 784 785 2006 SWs+ wait 0xfffff80013d75a70 bash 784 777 777 2006 S select 0xfffff8000306c6c0 sshd 777 726 777 0 Ss select 0xfffff800060af2c0 sshd 774 773 774 0 D+ pfault 0xffffffff81a49880 csh 773 1 773 0 SWs+ wait 0xfffff80013d76538 login 772 1 772 0 Ss+ ttyin 0xfffff800061950b0 getty 771 1 771 0 Ss+ ttyin 0xfffff800135318b0 getty 770 1 770 0 Ss+ ttyin 0xfffff80013531cb0 getty 769 1 769 0 Ss+ ttyin 0xfffff800135320b0 getty 768 1 768 0 Ss+ ttyin 0xfffff800135324b0 getty 767 1 767 0 Ss+ ttyin 0xfffff800135328b0 getty 766 1 766 0 Ss+ ttyin 0xfffff80013532cb0 getty 765 1 765 0 Ss+ ttyin 0xfffff800135330b0 getty 764 1 764 0 Ss+ ttyin 0xfffff800135334b0 getty 730 1 730 0 Ds vmwait 0xffffffff81a49880 cron 726 1 726 0 Ss select 0xfffff80006070d40 sshd 724 1 724 65 Ss piperd 0xfffff80013f45a50 dhclient 667 1 667 0 Ds vmwait 0xffffffff81a49880 dhclient 658 1 658 0 Ss select 0xfffff8000306d4c0 dhclient 602 1 602 0 Ss rpcsvc 0xfffff800212e2560 NLM: master 599 1 599 0 Ss select 0xfffff80003901540 rpc.statd 574 1 574 0 Ss select 0xfffff800060af340 casperd 573 1 573 0 Ss select 0xfffff8000306d240 casperd 532 1 532 0 Ss select 0xfffff800135856c0 rpcbind 527 1 527 0 Ss select 0xfffff80013585540 syslogd 452 1 452 0 Ss select 0xfffff800135af1c0 devd 33 0 0 0 DL vlruwt 0xfffff800133e5a70 [vnlru] 32 0 0 0 DL syncer 0xffffffff81f94e40 [syncer] 31 0 0 0 DL (threaded) [bufdaemon] 100128 D qsleep 0xffffffff81f942e0 [bufdaemon] 100132 D - 0xffffffff81a0c1c0 [bufspacedaemon-0] 100134 D - 0xffffffff81a14480 [bufspacedaemon-1] 100135 D - 0xffffffff81a1c740 [bufspacedaemon-2] 100136 D - 0xffffffff81a24a00 [bufspacedaemon-3] 100137 D - 0xffffffff81a2ccc0 [bufspacedaemon-4] 100138 D - 0xffffffff81a34f80 [bufspacedaemon-5] 100139 D - 0xffffffff81a3d240 [bufspacedaemon-6] 100150 D sdflush 0xfffff80013eeb4e8 [/ worker] 100203 D sdflush 0xfffff80021f878e8 [/tmp worker] 100494 D sdflush 0xfffff80705eb02e8 [/mnt worker] 100496 D sdflush 0xfffff803d2c288e8 [/mnt2 worker] 30 0 0 0 DL psleep 0xffffffff81fa0bc0 [vmdaemon] 29 0 0 0 RL (threaded) [pagedaemon] 100126 Run CPU 11 [dom0] 100130 D launds 0xffffffff82010064 [laundry: dom0] 100133 D umarcl 0xffffffff80e66400 [uma] 28 0 0 0 DL idle 0xfffff80013348000 [enc_daemon1] 27 0 0 0 DL idle 0xfffff80013348538 [enc_daemon0] 26 0 0 0 DL - 0xffffffff81cc5958 [rand_harvestq] 25 0 0 0 DL waiting 0xffffffff8200e880 [sctp_iterator] 24 0 0 0 DL - 0xffffffff81f93cdc [soaiod4] 23 0 0 0 DL - 0xffffffff81f93cdc [soaiod3] 22 0 0 0 DL - 0xffffffff81f93cdc [soaiod2] 21 0 0 0 DL - 0xffffffff81f93cdc [soaiod1] 20 0 0 0 DL (threaded) [usb] 100094 D - 0xfffffe009c7cb460 [usbus0] 100095 D - 0xfffffe009c7cb4b8 [usbus0] 100096 D - 0xfffffe009c7cb510 [usbus0] 100097 D - 0xfffffe009c7cb568 [usbus0] 100098 D - 0xfffffe009c7cb5c0 [usbus0] 100100 D - 0xfffffe009a5d8d10 [usbus1] 100101 D - 0xfffffe009a5d8d68 [usbus1] 100102 D - 0xfffffe009a5d8dc0 [usbus1] 100103 D - 0xfffffe009a5d8e18 [usbus1] 100104 D - 0xfffffe009a5d8e70 [usbus1] 100105 D - 0xfffffe009ca4cd10 [usbus2] 100106 D - 0xfffffe009ca4cd68 [usbus2] 100107 D - 0xfffffe009ca4cdc0 [usbus2] 100108 D - 0xfffffe009ca4ce18 [usbus2] 100109 D - 0xfffffe009ca4ce70 [usbus2] 19 0 0 0 DL (threaded) [cam] 100089 D - 0xffffffff81ba0b40 [doneq0] 100090 D - 0xffffffff81ba0bc0 [doneq1] 100091 D - 0xffffffff81ba0c40 [doneq2] 100123 D - 0xffffffff81ba0a08 [scanner] 18 0 0 0 DL crypto_ 0xfffff800037f3450 [crypto returns 11] 17 0 0 0 DL crypto_ 0xfffff800037f33f0 [crypto returns 10] 16 0 0 0 DL crypto_ 0xfffff800037f3390 [crypto returns 9] 15 0 0 0 DL crypto_ 0xfffff800037f3330 [crypto returns 8] 14 0 0 0 DL crypto_ 0xfffff800037f32d0 [crypto returns 7] 9 0 0 0 DL crypto_ 0xfffff800037f3270 [crypto returns 6] 8 0 0 0 DL crypto_ 0xfffff800037f3210 [crypto returns 5] 7 0 0 0 DL crypto_ 0xfffff800037f31b0 [crypto returns 4] 6 0 0 0 DL crypto_ 0xfffff800037f3150 [crypto returns 3] 5 0 0 0 DL crypto_ 0xfffff800037f30f0 [crypto returns 2] 4 0 0 0 DL crypto_ 0xfffff800037f3090 [crypto returns 1] 3 0 0 0 DL crypto_ 0xfffff800037f3030 [crypto returns 0] 2 0 0 0 DL crypto_ 0xffffffff81f9b1a8 [crypto] 13 0 0 0 DL (threaded) [geom] 100060 D - 0xffffffff81feb2f0 [g_event] 100061 D - 0xffffffff81feb2f8 [g_up] 100062 D - 0xffffffff81feb300 [g_down] 12 0 0 0 WL (threaded) [intr] 100016 I [swi5: fast taskq] 100019 I [swi6: task queue] 100020 I [swi6: Giant taskq] 100046 I [swi1: netisr 0] 100047 I [swi4: clock (0)] 100048 I [swi4: clock (1)] 100049 I [swi4: clock (2)] 100050 I [swi4: clock (3)] 100051 I [swi4: clock (4)] 100052 I [swi4: clock (5)] 100053 I [swi4: clock (6)] 100054 I [swi4: clock (7)] 100055 I [swi4: clock (8)] 100056 I [swi4: clock (9)] 100057 I [swi4: clock (10)] 100058 I [swi4: clock (11)] 100059 I [swi3: vm] 100092 I [irq264: ahci0] 100093 I [irq19: xhci0] 100099 I [irq18: ehci0 ehci1] 100110 I [irq279: ahci1] 100111 I [swi0: uart uart] 11 0 0 0 RL (threaded) [idle] 100003 Run CPU 0 [idle: cpu0] 100004 Run CPU 1 [idle: cpu1] 100005 Run CPU 2 [idle: cpu2] 100006 Run CPU 3 [idle: cpu3] 100007 Run CPU 4 [idle: cpu4] 100008 Run CPU 5 [idle: cpu5] 100009 Run CPU 6 [idle: cpu6] 100010 Run CPU 7 [idle: cpu7] 100011 Run CPU 8 [idle: cpu8] 100012 Run CPU 9 [idle: cpu9] 100013 Run CPU 10 [idle: cpu10] 100014 CanRun [idle: cpu11] 1 0 1 0 SLs wait 0xfffff8000362e538 [init] 10 0 0 0 DL audit_w 0xffffffff8200f568 [audit] 0 0 0 0 DLs (threaded) [kernel] 100000 D vmwait 0xffffffff81a49880 [swapper] 100015 D - 0xfffff800030b3c00 [thread taskq] 100017 D - 0xfffff800030b3a00 [aiod_kick taskq] 100018 D - 0xfffff800030b3900 [kqueue_ctx taskq] 100021 D - 0xfffff8000361d800 [softirq_0] 100022 D - 0xfffff8000361d600 [softirq_1] 100023 D - 0xfffff8000361d400 [softirq_2] 100024 D - 0xfffff8000361d200 [softirq_3] 100025 D - 0xfffff8000361d000 [softirq_4] 100026 D - 0xfffff8000361cd00 [softirq_5] 100027 D - 0xfffff8000361cb00 [softirq_6] 100028 D - 0xfffff8000361c900 [softirq_7] 100029 D - 0xfffff8000361c700 [softirq_8] 100030 D - 0xfffff8000361c500 [softirq_9] 100031 D - 0xfffff8000361c300 [softirq_10] 100032 D - 0xfffff8000361c100 [softirq_11] 100033 D - 0xfffff8000361be00 [if_config_tqg_0] 100034 D - 0xfffff80003723100 [if_io_tqg_0] 100035 D - 0xfffff80003722e00 [if_io_tqg_1] 100036 D - 0xfffff80003722c00 [if_io_tqg_2] 100037 D - 0xfffff80003722a00 [if_io_tqg_3] 100038 D - 0xfffff80003722800 [if_io_tqg_4] 100039 D - 0xfffff80003722600 [if_io_tqg_5] 100040 D - 0xfffff80003722400 [if_io_tqg_6] 100041 D - 0xfffff80003722200 [if_io_tqg_7] 100042 D - 0xfffff80003722000 [if_io_tqg_8] 100043 D - 0xfffff80003721d00 [if_io_tqg_9] 100044 D - 0xfffff80003721b00 [if_io_tqg_10] 100045 D - 0xfffff80003721900 [if_io_tqg_11] 100063 D - 0xfffff800037a1100 [firmware taskq] 100064 D - 0xfffff800037a1000 [crypto_0] 100065 D - 0xfffff800037a1000 [crypto_1] 100066 D - 0xfffff800037a1000 [crypto_2] 100067 D - 0xfffff800037a1000 [crypto_3] 100068 D - 0xfffff800037a1000 [crypto_4] 100069 D - 0xfffff800037a1000 [crypto_5] 100070 D - 0xfffff800037a1000 [crypto_6] 100071 D - 0xfffff800037a1000 [crypto_7] 100072 D - 0xfffff800037a1000 [crypto_8] 100073 D - 0xfffff800037a1000 [crypto_9] 100074 D - 0xfffff800037a1000 [crypto_10] 100075 D - 0xfffff800037a1000 [crypto_11] 100112 D - 0xfffff800061d1200 [mca taskq] 100119 D - 0xfffff8001343e300 [acpi_task_0] 100120 D - 0xfffff8001343e300 [acpi_task_1] 100121 D - 0xfffff8001343e300 [acpi_task_2] 100122 D - 0xfffff8000379fe00 [CAM taskq] oops, ran out of processes early! db:0:pho> allt Tracing command sleep pid 92137 tid 104710 td 0xfffff80401226000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a7194670 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a71946a0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a71946e0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a7194710 _sleep() at _sleep+0x367/frame 0xfffffe00a71947c0 vm_waitpfault() at vm_waitpfault+0x61/frame 0xfffffe00a71947e0 vm_fault_hold() at vm_fault_hold+0xb47/frame 0xfffffe00a7194910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a7194950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a71949a0 trap() at trap+0x3be/frame 0xfffffe00a7194ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a7194ab0 --- trap 0xc, rip = 0x80021d2d4, rsp = 0x7fffffffdb78, rbp = 0x7fffffffdb90 --- Tracing command sendfile14 pid 92127 tid 104565 td 0xfffff804c4f8a000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a6ebd3c0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a6ebd3f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a6ebd430 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a6ebd460 _sleep() at _sleep+0x367/frame 0xfffffe00a6ebd510 vm_waitpfault() at vm_waitpfault+0x61/frame 0xfffffe00a6ebd530 vm_fault_hold() at vm_fault_hold+0xb47/frame 0xfffffe00a6ebd660 vm_fault_quick_hold_pages() at vm_fault_quick_hold_pages+0x159/frame 0xfffffe00a6ebd6e0 vn_io_fault1() at vn_io_fault1+0x2ca/frame 0xfffffe00a6ebd820 vn_io_fault() at vn_io_fault+0x195/frame 0xfffffe00a6ebd890 dofilewrite() at dofilewrite+0xa7/frame 0xfffffe00a6ebd8e0 kern_writev() at kern_writev+0x68/frame 0xfffffe00a6ebd930 sys_write() at sys_write+0x86/frame 0xfffffe00a6ebd980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe00a6ebdab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a6ebdab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x80083ba3a, rsp = 0x7fffffffe4c8, rbp = 0x7fffffffe500 --- Tracing command sendfile14 pid 92126 tid 105410 td 0xfffff803cad4a560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a7f4e3c0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a7f4e3f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a7f4e430 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a7f4e460 _sleep() at _sleep+0x367/frame 0xfffffe00a7f4e510 vm_waitpfault() at vm_waitpfault+0x61/frame 0xfffffe00a7f4e530 vm_fault_hold() at vm_fault_hold+0xb47/frame 0xfffffe00a7f4e660 vm_fault_quick_hold_pages() at vm_fault_quick_hold_pages+0x159/frame 0xfffffe00a7f4e6e0 vn_io_fault1() at vn_io_fault1+0x2ca/frame 0xfffffe00a7f4e820 vn_io_fault() at vn_io_fault+0x195/frame 0xfffffe00a7f4e890 dofilewrite() at dofilewrite+0xa7/frame 0xfffffe00a7f4e8e0 kern_writev() at kern_writev+0x68/frame 0xfffffe00a7f4e930 sys_write() at sys_write+0x86/frame 0xfffffe00a7f4e980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe00a7f4eab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a7f4eab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x80083ba3a, rsp = 0x7fffffffe4c8, rbp = 0x7fffffffe500 --- Tracing command md11 pid 92118 tid 100154 td 0xfffff80003679560 sched_switch() at sched_switch+0x942/frame 0xfffffe009ec4f8c0 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ec4f8f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ec4f930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009ec4f960 _sleep() at _sleep+0x367/frame 0xfffffe009ec4fa10 md_kthread() at md_kthread+0x1e5/frame 0xfffffe009ec4fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe009ec4fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009ec4fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command md10 pid 92109 tid 104679 td 0xfffff802a4769560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a70f98c0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a70f98f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a70f9930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a70f9960 _sleep() at _sleep+0x367/frame 0xfffffe00a70f9a10 md_kthread() at md_kthread+0x1e5/frame 0xfffffe00a70f9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a70f9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a70f9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command timeout pid 92090 tid 102999 td 0xfffff806fe3ca560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a5009740 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a5009770 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a50097b0 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe00a5009810 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00a5009840 _sleep() at _sleep+0x359/frame 0xfffffe00a50098f0 kern_sigsuspend() at kern_sigsuspend+0xb4/frame 0xfffffe00a5009950 sys_sigsuspend() at sys_sigsuspend+0x31/frame 0xfffffe00a5009980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe00a5009ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a5009ab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x80039eb6a, rsp = 0x7fffffffe4d8, rbp = 0x7fffffffe6f0 --- Tracing command tee pid 92089 tid 104958 td 0xfffff8052ff8e560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a7672690 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a76726c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a7672700 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe00a7672760 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00a7672790 _sleep() at _sleep+0x359/frame 0xfffffe00a7672840 pipe_read() at pipe_read+0x345/frame 0xfffffe00a76728a0 dofileread() at dofileread+0xba/frame 0xfffffe00a76728e0 kern_readv() at kern_readv+0x68/frame 0xfffffe00a7672930 sys_read() at sys_read+0x86/frame 0xfffffe00a7672980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe00a7672ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a7672ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800411a7a, rsp = 0x7fffffffe6c8, rbp = 0x7fffffffe710 --- Tracing command sh pid 92088 tid 103323 td 0xfffff8074d8a7000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a5663550 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a5663580 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a56635c0 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe00a5663620 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00a5663650 _sleep() at _sleep+0x359/frame 0xfffffe00a5663700 kern_wait6() at kern_wait6+0x4c4/frame 0xfffffe00a5663790 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00a5663980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe00a5663ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a5663ab0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003feaea, rsp = 0x7fffffffcdc8, rbp = 0x7fffffffce80 --- Tracing command bash pid 90589 tid 105387 td 0xfffff805ba775000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a7edb570 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a7edb5a0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a7edb5e0 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe00a7edb640 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00a7edb670 _cv_wait_sig() at _cv_wait_sig+0x1fe/frame 0xfffffe00a7edb6d0 seltdwait() at seltdwait+0x8d/frame 0xfffffe00a7edb710 kern_select() at kern_select+0x886/frame 0xfffffe00a7edb900 sys_pselect() at sys_pselect+0x127/frame 0xfffffe00a7edb980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe00a7edbab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a7edbab0 --- syscall (522, FreeBSD ELF64, sys_pselect), rip = 0x800afac0a, rsp = 0x7fffffffd618, rbp = 0x7fffffffd6e0 --- Tracing command csh pid 90571 tid 105316 td 0xfffff807bd2fe560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a7d76740 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a7d76770 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a7d767b0 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe00a7d76810 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00a7d76840 _sleep() at _sleep+0x359/frame 0xfffffe00a7d768f0 kern_sigsuspend() at kern_sigsuspend+0xb4/frame 0xfffffe00a7d76950 sys_sigsuspend() at sys_sigsuspend+0x31/frame 0xfffffe00a7d76980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe00a7d76ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a7d76ab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x800481b6a, rsp = 0x7fffffffdff8, rbp = 0x7fffffffe080 --- Tracing command su pid 90570 tid 104671 td 0xfffff8024a2bb000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a70d1550 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a70d1580 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a70d15c0 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe00a70d1620 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00a70d1650 _sleep() at _sleep+0x359/frame 0xfffffe00a70d1700 kern_wait6() at kern_wait6+0x4c4/frame 0xfffffe00a70d1790 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00a70d1980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe00a70d1ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a70d1ab0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003e2aea, rsp = 0x7fffffffe608, rbp = 0x7fffffffeb50 --- Tracing command bash pid 90198 tid 105394 td 0xfffff80451b8b560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a7efe550 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a7efe580 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a7efe5c0 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe00a7efe620 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00a7efe650 _sleep() at _sleep+0x359/frame 0xfffffe00a7efe700 kern_wait6() at kern_wait6+0x4c4/frame 0xfffffe00a7efe790 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00a7efe980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe00a7efeab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a7efeab0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x800afaaea, rsp = 0x7fffffffe7b8, rbp = 0x7fffffffe800 --- Tracing command sshd pid 90184 tid 104451 td 0xfffff802597c4560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a6c815b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a6c815e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a6c81620 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe00a6c81680 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00a6c816b0 _cv_wait_sig() at _cv_wait_sig+0x1fe/frame 0xfffffe00a6c81710 seltdwait() at seltdwait+0x8d/frame 0xfffffe00a6c81750 kern_select() at kern_select+0x886/frame 0xfffffe00a6c81940 sys_select() at sys_select+0x56/frame 0xfffffe00a6c81980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe00a6c81ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a6c81ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800874a5a, rsp = 0x7fffffffa5e8, rbp = 0x7fffffffa620 --- Tracing command sshd pid 89765 tid 105178 td 0xfffff80313693000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a7ac2620 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a7ac2650 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a7ac2690 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe00a7ac26f0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00a7ac2720 _cv_wait_sig() at _cv_wait_sig+0x1fe/frame 0xfffffe00a7ac2780 seltdwait() at seltdwait+0x8d/frame 0xfffffe00a7ac27c0 kern_poll() at kern_poll+0x3f9/frame 0xfffffe00a7ac2960 sys_poll() at sys_poll+0x50/frame 0xfffffe00a7ac2980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe00a7ac2ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00a7ac2ab0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x80081343a, rsp = 0x7fffffffe5c8, rbp = 0x7fffffffe600 --- Tracing command top pid 85572 tid 103304 td 0xfffff808405f0560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a5604670 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a56046a0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a56046e0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00a5604710 _sleep() at _sleep+0x367/frame 0xfffffe00a56047c0 vm_waitpfault() at vm_waitpfault+0x61/frame 0xfffffe00a56047e0 vm_fault_hold() at vm_fault_hold+0xb47/frame 0xfffffe00a5604910 vm_fault() at vm_fault+0x75/frame 0xfffffe00a5604950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe00a56049a0 trap() at trap+0x3be/frame 0xfffffe00a5604ab0 calltrap() at calltrap+0x8/frame 0xfffffe00a5604ab0 --- trap 0xc, rip = 0x8004360af, rsp = 0x7fffffffe830, rbp = 0x7fffffffe840 --- Tracing command sh pid 2109 tid 100168 td 0xfffff80013a31560 sched_switch() at sched_switch+0x942/frame 0xfffffe009ec95550 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ec95580 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ec955c0 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe009ec95620 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe009ec95650 _sleep() at _sleep+0x359/frame 0xfffffe009ec95700 kern_wait6() at kern_wait6+0x4c4/frame 0xfffffe009ec95790 sys_wait4() at sys_wait4+0x78/frame 0xfffffe009ec95980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ec95ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ec95ab0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003feaea, rsp = 0x7fffffffd0e8, rbp = 0x7fffffffd1a0 --- Tracing command sh pid 805 tid 100179 td 0xfffff8002185b560 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80fdf186, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80b85fd6, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bash pid 789 tid 100201 td 0xfffff80013fb2000 sched_switch() at sched_switch+0x942/frame 0xfffffe009ed3a550 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ed3a580 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ed3a5c0 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe009ed3a620 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe009ed3a650 _sleep() at _sleep+0x359/frame 0xfffffe009ed3a700 kern_wait6() at kern_wait6+0x4c4/frame 0xfffffe009ed3a790 sys_wait4() at sys_wait4+0x78/frame 0xfffffe009ed3a980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ed3aab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ed3aab0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x800afaaea, rsp = 0x7fffffffe678, rbp = 0x7fffffffe6c0 --- Tracing command csh pid 787 tid 100190 td 0xfffff800217ba000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80fdf186, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80b85fd6, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command su pid 786 tid 100160 td 0xfffff80013a48000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80fdf186, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80b85fd6, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bash pid 785 tid 100200 td 0xfffff80013d85000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80fdf186, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80b85fd6, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sshd pid 784 tid 100198 td 0xfffff800210d2560 sched_switch() at sched_switch+0x942/frame 0xfffffe009ed2b5b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ed2b5e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ed2b620 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe009ed2b680 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe009ed2b6b0 _cv_wait_sig() at _cv_wait_sig+0x1fe/frame 0xfffffe009ed2b710 seltdwait() at seltdwait+0x8d/frame 0xfffffe009ed2b750 kern_select() at kern_select+0x886/frame 0xfffffe009ed2b940 sys_select() at sys_select+0x56/frame 0xfffffe009ed2b980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ed2bab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ed2bab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800874a5a, rsp = 0x7fffffffa5e8, rbp = 0x7fffffffa620 --- Tracing command sshd pid 777 tid 100182 td 0xfffff80013fb3000 sched_switch() at sched_switch+0x942/frame 0xfffffe009ecdb620 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ecdb650 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ecdb690 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe009ecdb6f0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe009ecdb720 _cv_wait_sig() at _cv_wait_sig+0x1fe/frame 0xfffffe009ecdb780 seltdwait() at seltdwait+0x8d/frame 0xfffffe009ecdb7c0 kern_poll() at kern_poll+0x3f9/frame 0xfffffe009ecdb960 sys_poll() at sys_poll+0x50/frame 0xfffffe009ecdb980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ecdbab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ecdbab0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x80081343a, rsp = 0x7fffffffe5c8, rbp = 0x7fffffffe600 --- Tracing command csh pid 774 tid 100170 td 0xfffff80013d85560 sched_switch() at sched_switch+0x942/frame 0xfffffe009ec9f670 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ec9f6a0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ec9f6e0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009ec9f710 _sleep() at _sleep+0x367/frame 0xfffffe009ec9f7c0 vm_waitpfault() at vm_waitpfault+0x61/frame 0xfffffe009ec9f7e0 vm_fault_hold() at vm_fault_hold+0xb47/frame 0xfffffe009ec9f910 vm_fault() at vm_fault+0x75/frame 0xfffffe009ec9f950 trap_pfault() at trap_pfault+0x171/frame 0xfffffe009ec9f9a0 trap() at trap+0x3be/frame 0xfffffe009ec9fab0 calltrap() at calltrap+0x8/frame 0xfffffe009ec9fab0 --- trap 0xc, rip = 0x8004f6044, rsp = 0x7fffffffe0c8, rbp = 0x7fffffffe2b0 --- Tracing command login pid 773 tid 100157 td 0xfffff80013d86000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0078959f10 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe0078959f20 trap() at trap+0x5c/frame 0xfffffe007895a030 calltrap() at calltrap+0x8/frame 0xfffffe007895a030 --- trap 0xc, rip = 0xffffffff80fdf186, rsp = 0xfffffe007895a100, rbp = 0xfffffe007895a110 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe007895a110 _db_stack_trace_all() at _db_stack_trace_all+0xf4/frame 0xfffffe007895a1b0 db_command() at db_command+0x279/frame 0xfffffe007895a280 db_script_exec() at db_script_exec+0x188/frame 0xfffffe007895a320 db_command() at db_command+0x279/frame 0xfffffe007895a3f0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe007895a400 db_trap() at db_trap+0xef/frame 0xfffffe007895a490 kdb_trap() at kdb_trap+0x1c3/frame 0xfffffe007895a500 trap() at trap+0x328/frame 0xfffffe007895a610 calltrap() at calltrap+0x8/frame 0xfffffe007895a610 --- trap 0x3, rip = 0xffffffff80b85fd6, rsp = 0xfffffe007895a6e0, rbp = 0xfffffe007895a700 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command getty pid 772 tid 100197 td 0xfffff800210d3000 sched_switch() at sched_switch+0x942/frame 0xfffffe009ed26610 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ed26640 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ed26680 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe009ed266e0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe009ed26710 _cv_wait_sig() at _cv_wait_sig+0x1fe/frame 0xfffffe009ed26770 tty_wait() at tty_wait+0x42/frame 0xfffffe009ed267a0 ttydisc_read() at ttydisc_read+0x22c/frame 0xfffffe009ed26810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe009ed26840 devfs_read_f() at devfs_read_f+0xe0/frame 0xfffffe009ed268a0 dofileread() at dofileread+0xba/frame 0xfffffe009ed268e0 kern_readv() at kern_readv+0x68/frame 0xfffffe009ed26930 sys_read() at sys_read+0x86/frame 0xfffffe009ed26980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ed26ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ed26ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80042ca7a, rsp = 0x7fffffffebe8, rbp = 0x7fffffffec30 --- Tracing command getty pid 771 tid 100196 td 0xfffff800210d3560 sched_switch() at sched_switch+0x942/frame 0xfffffe009ed21610 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ed21640 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ed21680 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe009ed216e0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe009ed21710 _cv_wait_sig() at _cv_wait_sig+0x1fe/frame 0xfffffe009ed21770 tty_wait() at tty_wait+0x42/frame 0xfffffe009ed217a0 ttydisc_read() at ttydisc_read+0x22c/frame 0xfffffe009ed21810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe009ed21840 devfs_read_f() at devfs_read_f+0xe0/frame 0xfffffe009ed218a0 dofileread() at dofileread+0xba/frame 0xfffffe009ed218e0 kern_readv() at kern_readv+0x68/frame 0xfffffe009ed21930 sys_read() at sys_read+0x86/frame 0xfffffe009ed21980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ed21ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ed21ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80042ca7a, rsp = 0x7fffffffebe8, rbp = 0x7fffffffec30 --- Tracing command getty pid 770 tid 100195 td 0xfffff800210d4000 sched_switch() at sched_switch+0x942/frame 0xfffffe009ed1c610 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ed1c640 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ed1c680 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe009ed1c6e0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe009ed1c710 _cv_wait_sig() at _cv_wait_sig+0x1fe/frame 0xfffffe009ed1c770 tty_wait() at tty_wait+0x42/frame 0xfffffe009ed1c7a0 ttydisc_read() at ttydisc_read+0x22c/frame 0xfffffe009ed1c810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe009ed1c840 devfs_read_f() at devfs_read_f+0xe0/frame 0xfffffe009ed1c8a0 dofileread() at dofileread+0xba/frame 0xfffffe009ed1c8e0 kern_readv() at kern_readv+0x68/frame 0xfffffe009ed1c930 sys_read() at sys_read+0x86/frame 0xfffffe009ed1c980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ed1cab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ed1cab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80042ca7a, rsp = 0x7fffffffebe8, rbp = 0x7fffffffec30 --- Tracing command getty pid 769 tid 100194 td 0xfffff800210d4560 sched_switch() at sched_switch+0x942/frame 0xfffffe009ed17610 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ed17640 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ed17680 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe009ed176e0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe009ed17710 _cv_wait_sig() at _cv_wait_sig+0x1fe/frame 0xfffffe009ed17770 tty_wait() at tty_wait+0x42/frame 0xfffffe009ed177a0 ttydisc_read() at ttydisc_read+0x22c/frame 0xfffffe009ed17810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe009ed17840 devfs_read_f() at devfs_read_f+0xe0/frame 0xfffffe009ed178a0 dofileread() at dofileread+0xba/frame 0xfffffe009ed178e0 kern_readv() at kern_readv+0x68/frame 0xfffffe009ed17930 sys_read() at sys_read+0x86/frame 0xfffffe009ed17980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ed17ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ed17ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80042ca7a, rsp = 0x7fffffffebe8, rbp = 0x7fffffffec30 --- Tracing command getty pid 768 tid 100193 td 0xfffff800210d5000 sched_switch() at sched_switch+0x942/frame 0xfffffe009ed12610 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ed12640 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ed12680 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe009ed126e0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe009ed12710 _cv_wait_sig() at _cv_wait_sig+0x1fe/frame 0xfffffe009ed12770 tty_wait() at tty_wait+0x42/frame 0xfffffe009ed127a0 ttydisc_read() at ttydisc_read+0x22c/frame 0xfffffe009ed12810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe009ed12840 devfs_read_f() at devfs_read_f+0xe0/frame 0xfffffe009ed128a0 dofileread() at dofileread+0xba/frame 0xfffffe009ed128e0 kern_readv() at kern_readv+0x68/frame 0xfffffe009ed12930 sys_read() at sys_read+0x86/frame 0xfffffe009ed12980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ed12ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ed12ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80042ca7a, rsp = 0x7fffffffebe8, rbp = 0x7fffffffec30 --- Tracing command getty pid 767 tid 100192 td 0xfffff800210d5560 sched_switch() at sched_switch+0x942/frame 0xfffffe009ed0d610 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ed0d640 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ed0d680 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe009ed0d6e0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe009ed0d710 _cv_wait_sig() at _cv_wait_sig+0x1fe/frame 0xfffffe009ed0d770 tty_wait() at tty_wait+0x42/frame 0xfffffe009ed0d7a0 ttydisc_read() at ttydisc_read+0x22c/frame 0xfffffe009ed0d810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe009ed0d840 devfs_read_f() at devfs_read_f+0xe0/frame 0xfffffe009ed0d8a0 dofileread() at dofileread+0xba/frame 0xfffffe009ed0d8e0 kern_readv() at kern_readv+0x68/frame 0xfffffe009ed0d930 sys_read() at sys_read+0x86/frame 0xfffffe009ed0d980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ed0dab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ed0dab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80042ca7a, rsp = 0x7fffffffebe8, rbp = 0x7fffffffec30 --- Tracing command getty pid 766 tid 100165 td 0xfffff80013bb3560 sched_switch() at sched_switch+0x942/frame 0xfffffe009ec86610 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ec86640 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ec86680 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe009ec866e0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe009ec86710 _cv_wait_sig() at _cv_wait_sig+0x1fe/frame 0xfffffe009ec86770 tty_wait() at tty_wait+0x42/frame 0xfffffe009ec867a0 ttydisc_read() at ttydisc_read+0x22c/frame 0xfffffe009ec86810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe009ec86840 devfs_read_f() at devfs_read_f+0xe0/frame 0xfffffe009ec868a0 dofileread() at dofileread+0xba/frame 0xfffffe009ec868e0 kern_readv() at kern_readv+0x68/frame 0xfffffe009ec86930 sys_read() at sys_read+0x86/frame 0xfffffe009ec86980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ec86ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ec86ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80042ca7a, rsp = 0x7fffffffebe8, rbp = 0x7fffffffec30 --- Tracing command getty pid 765 tid 100175 td 0xfffff800210d6000 sched_switch() at sched_switch+0x942/frame 0xfffffe009ecb8610 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ecb8640 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ecb8680 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe009ecb86e0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe009ecb8710 _cv_wait_sig() at _cv_wait_sig+0x1fe/frame 0xfffffe009ecb8770 tty_wait() at tty_wait+0x42/frame 0xfffffe009ecb87a0 ttydisc_read() at ttydisc_read+0x22c/frame 0xfffffe009ecb8810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe009ecb8840 devfs_read_f() at devfs_read_f+0xe0/frame 0xfffffe009ecb88a0 dofileread() at dofileread+0xba/frame 0xfffffe009ecb88e0 kern_readv() at kern_readv+0x68/frame 0xfffffe009ecb8930 sys_read() at sys_read+0x86/frame 0xfffffe009ecb8980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ecb8ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ecb8ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80042ca7a, rsp = 0x7fffffffebe8, rbp = 0x7fffffffec30 --- Tracing command getty pid 764 tid 100140 td 0xfffff80003683560 sched_switch() at sched_switch+0x942/frame 0xfffffe009ec09610 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ec09640 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ec09680 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe009ec096e0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe009ec09710 _cv_wait_sig() at _cv_wait_sig+0x1fe/frame 0xfffffe009ec09770 tty_wait() at tty_wait+0x42/frame 0xfffffe009ec097a0 ttydisc_read() at ttydisc_read+0x22c/frame 0xfffffe009ec09810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe009ec09840 devfs_read_f() at devfs_read_f+0xe0/frame 0xfffffe009ec098a0 dofileread() at dofileread+0xba/frame 0xfffffe009ec098e0 kern_readv() at kern_readv+0x68/frame 0xfffffe009ec09930 sys_read() at sys_read+0x86/frame 0xfffffe009ec09980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ec09ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ec09ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80042ca7a, rsp = 0x7fffffffebe8, rbp = 0x7fffffffec30 --- Tracing command cron pid 730 tid 100183 td 0xfffff80013fb2560 sched_switch() at sched_switch+0x942/frame 0xfffffe009ece0520 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ece0550 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ece0590 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009ece05c0 _sleep() at _sleep+0x367/frame 0xfffffe009ece0670 vm_wait_doms() at vm_wait_doms+0x85/frame 0xfffffe009ece0690 vm_wait_domain() at vm_wait_domain+0x68/frame 0xfffffe009ece06c0 vm_domain_alloc_fail() at vm_domain_alloc_fail+0x94/frame 0xfffffe009ece06f0 vm_page_alloc_domain_after() at vm_page_alloc_domain_after+0x264/frame 0xfffffe009ece0760 vm_page_alloc() at vm_page_alloc+0x74/frame 0xfffffe009ece07d0 pmap_pinit_type() at pmap_pinit_type+0x28/frame 0xfffffe009ece0810 pmap_pinit() at pmap_pinit+0x10/frame 0xfffffe009ece0820 vmspace_fork() at vmspace_fork+0x55/frame 0xfffffe009ece0880 fork1() at fork1+0x4ce/frame 0xfffffe009ece0930 sys_fork() at sys_fork+0x4c/frame 0xfffffe009ece0980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ece0ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ece0ab0 --- syscall (2, FreeBSD ELF64, sys_fork), rip = 0x8003cccaa, rsp = 0x7fffffffea98, rbp = 0x7fffffffeab0 --- Tracing command sshd pid 726 tid 100185 td 0xfffff8000606e560 sched_switch() at sched_switch+0x942/frame 0xfffffe009ecea5b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ecea5e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ecea620 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe009ecea680 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe009ecea6b0 _cv_wait_sig() at _cv_wait_sig+0x1fe/frame 0xfffffe009ecea710 seltdwait() at seltdwait+0x8d/frame 0xfffffe009ecea750 kern_select() at kern_select+0x886/frame 0xfffffe009ecea940 sys_select() at sys_select+0x56/frame 0xfffffe009ecea980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009eceaab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009eceaab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800874a5a, rsp = 0x7fffffffe698, rbp = 0x7fffffffe6d0 --- Tracing command dhclient pid 724 tid 100172 td 0xfffff80013e41000 sched_switch() at sched_switch+0x942/frame 0xfffffe009eca9690 mi_switch() at mi_switch+0x18c/frame 0xfffffe009eca96c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009eca9700 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe009eca9760 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe009eca9790 _sleep() at _sleep+0x359/frame 0xfffffe009eca9840 pipe_read() at pipe_read+0x345/frame 0xfffffe009eca98a0 dofileread() at dofileread+0xba/frame 0xfffffe009eca98e0 kern_readv() at kern_readv+0x68/frame 0xfffffe009eca9930 sys_read() at sys_read+0x86/frame 0xfffffe009eca9980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009eca9ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009eca9ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800448a7a, rsp = 0x7fffffffca48, rbp = 0x7fffffffca70 --- Tracing command dhclient pid 667 tid 100153 td 0xfffff80003683000 sched_switch() at sched_switch+0x942/frame 0xfffffe009ec4a520 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ec4a550 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ec4a590 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe009ec4a5c0 _sleep() at _sleep+0x367/frame 0xfffffe009ec4a670 vm_wait_doms() at vm_wait_doms+0x85/frame 0xfffffe009ec4a690 vm_wait_domain() at vm_wait_domain+0x68/frame 0xfffffe009ec4a6c0 vm_domain_alloc_fail() at vm_domain_alloc_fail+0x94/frame 0xfffffe009ec4a6f0 vm_page_alloc_domain_after() at vm_page_alloc_domain_after+0x264/frame 0xfffffe009ec4a760 vm_page_alloc() at vm_page_alloc+0x74/frame 0xfffffe009ec4a7d0 pmap_pinit_type() at pmap_pinit_type+0x28/frame 0xfffffe009ec4a810 pmap_pinit() at pmap_pinit+0x10/frame 0xfffffe009ec4a820 vmspace_fork() at vmspace_fork+0x55/frame 0xfffffe009ec4a880 fork1() at fork1+0x4ce/frame 0xfffffe009ec4a930 sys_fork() at sys_fork+0x4c/frame 0xfffffe009ec4a980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ec4aab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ec4aab0 --- syscall (2, FreeBSD ELF64, sys_fork), rip = 0x8003d4caa, rsp = 0x7fffffffdac8, rbp = 0x7fffffffdb50 --- Tracing command dhclient pid 658 tid 100178 td 0xfffff80013fb3560 sched_switch() at sched_switch+0x942/frame 0xfffffe009ecc75b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ecc75e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ecc7620 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe009ecc7680 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe009ecc76b0 _cv_wait_sig() at _cv_wait_sig+0x1fe/frame 0xfffffe009ecc7710 seltdwait() at seltdwait+0x8d/frame 0xfffffe009ecc7750 kern_select() at kern_select+0x886/frame 0xfffffe009ecc7940 sys_select() at sys_select+0x56/frame 0xfffffe009ecc7980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ecc7ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ecc7ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800448a5a, rsp = 0x7fffffffea08, rbp = 0x7fffffffeac0 --- Tracing command rpc.lockd pid 602 tid 100156 td 0xfffff80013a4b000 sched_switch() at sched_switch+0x942/frame 0xfffffe009ec594d0 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ec59500 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ec59540 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe009ec595a0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe009ec595e0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x221/frame 0xfffffe009ec59650 svc_run_internal() at svc_run_internal+0x38e/frame 0xfffffe009ec59790 svc_run() at svc_run+0x1fe/frame 0xfffffe009ec597f0 sys_nlm_syscall() at sys_nlm_syscall+0x776/frame 0xfffffe009ec59980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ec59ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ec59ab0 --- syscall (154, FreeBSD ELF64, sys_nlm_syscall), rip = 0x8002ee50a, rsp = 0x7fffffffec28, rbp = 0x7fffffffece0 --- Tracing command rpc.statd pid 599 tid 100151 td 0xfffff80013a34000 sched_switch() at sched_switch+0x942/frame 0xfffffe009ec40590 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ec405c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ec40600 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe009ec40660 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe009ec406a0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x221/frame 0xfffffe009ec40710 seltdwait() at seltdwait+0x6b/frame 0xfffffe009ec40750 kern_select() at kern_select+0x886/frame 0xfffffe009ec40940 sys_select() at sys_select+0x56/frame 0xfffffe009ec40980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ec40ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ec40ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800421a5a, rsp = 0x7fffffffeab8, rbp = 0x7fffffffec00 --- Tracing command casperd pid 574 tid 100176 td 0xfffff80013e6b560 sched_switch() at sched_switch+0x942/frame 0xfffffe009ecbd5b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ecbd5e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ecbd620 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe009ecbd680 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe009ecbd6b0 _cv_wait_sig() at _cv_wait_sig+0x1fe/frame 0xfffffe009ecbd710 seltdwait() at seltdwait+0x8d/frame 0xfffffe009ecbd750 kern_select() at kern_select+0x886/frame 0xfffffe009ecbd940 sys_select() at sys_select+0x56/frame 0xfffffe009ecbd980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ecbdab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ecbdab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800851a5a, rsp = 0x7fffffffe7f8, rbp = 0x7fffffffece0 --- Tracing command casperd pid 573 tid 100152 td 0xfffff80013e98000 sched_switch() at sched_switch+0x942/frame 0xfffffe009ec455b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ec455e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ec45620 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe009ec45680 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe009ec456b0 _cv_wait_sig() at _cv_wait_sig+0x1fe/frame 0xfffffe009ec45710 seltdwait() at seltdwait+0x8d/frame 0xfffffe009ec45750 kern_select() at kern_select+0x886/frame 0xfffffe009ec45940 sys_select() at sys_select+0x56/frame 0xfffffe009ec45980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ec45ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ec45ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800851a5a, rsp = 0x7fffffffe658, rbp = 0x7fffffffe720 --- Tracing command rpcbind pid 532 tid 100149 td 0xfffff80013bb4000 sched_switch() at sched_switch+0x942/frame 0xfffffe009ec36600 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ec36630 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ec36670 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe009ec366d0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe009ec36710 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x221/frame 0xfffffe009ec36780 seltdwait() at seltdwait+0x6b/frame 0xfffffe009ec367c0 kern_poll() at kern_poll+0x3f9/frame 0xfffffe009ec36960 sys_poll() at sys_poll+0x50/frame 0xfffffe009ec36980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ec36ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ec36ab0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8003c543a, rsp = 0x7fffffffca58, rbp = 0x7fffffffec70 --- Tracing command syslogd pid 527 tid 100164 td 0xfffff800210d6560 sched_switch() at sched_switch+0x942/frame 0xfffffe009ec815b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ec815e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ec81620 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe009ec81680 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe009ec816b0 _cv_wait_sig() at _cv_wait_sig+0x1fe/frame 0xfffffe009ec81710 seltdwait() at seltdwait+0x8d/frame 0xfffffe009ec81750 kern_select() at kern_select+0x886/frame 0xfffffe009ec81940 sys_select() at sys_select+0x56/frame 0xfffffe009ec81980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ec81ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ec81ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800430a5a, rsp = 0x7fffffffe6c8, rbp = 0x7fffffffece0 --- Tracing command devd pid 452 tid 100143 td 0xfffff80013d86560 sched_switch() at sched_switch+0x942/frame 0xfffffe009ec18590 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ec185c0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ec18600 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe009ec18660 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe009ec186a0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x221/frame 0xfffffe009ec18710 seltdwait() at seltdwait+0x6b/frame 0xfffffe009ec18750 kern_select() at kern_select+0x886/frame 0xfffffe009ec18940 sys_select() at sys_select+0x56/frame 0xfffffe009ec18980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe009ec18ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe009ec18ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x2f5aea, rsp = 0x7fffffffcb58, rbp = 0x7fffffffed00 --- Tracing command vnlru pid 33 tid 100131 td 0xfffff80013a36000 sched_switch() at sched_switch+0x942/frame 0xfffffe00005d7870 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005d78a0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005d78e0 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe00005d7920 _sleep() at _sleep+0x307/frame 0xfffffe00005d79d0 vnlru_proc() at vnlru_proc+0xa4/frame 0xfffffe00005d7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005d7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005d7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command syncer pid 32 tid 100129 td 0xfffff80013a36560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005d28c0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005d28f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005d2930 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe00005d2970 _cv_timedwait_sbt() at _cv_timedwait_sbt+0x221/frame 0xfffffe00005d29e0 sched_sync() at sched_sync+0x609/frame 0xfffffe00005d2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005d2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005d2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100128 td 0xfffff80013a38000 sched_switch() at sched_switch+0x942/frame 0xfffffe00005cd8c0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005cd8f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005cd930 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe00005cd970 _sleep() at _sleep+0x307/frame 0xfffffe00005cda20 buf_daemon() at buf_daemon+0xe8/frame 0xfffffe00005cda70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005cdab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005cdab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100132 td 0xfffff80013a6a560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005e18b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005e18e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005e1920 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe00005e1960 _sleep() at _sleep+0x307/frame 0xfffffe00005e1a10 bufspace_daemon() at bufspace_daemon+0x2db/frame 0xfffffe00005e1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005e1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005e1ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100134 td 0xfffff80013a6a000 sched_switch() at sched_switch+0x942/frame 0xfffffe00005eb8b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005eb8e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005eb920 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe00005eb960 _sleep() at _sleep+0x307/frame 0xfffffe00005eba10 bufspace_daemon() at bufspace_daemon+0x2db/frame 0xfffffe00005eba70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005ebab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005ebab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100135 td 0xfffff80013a69560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005f08b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005f08e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005f0920 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe00005f0960 _sleep() at _sleep+0x307/frame 0xfffffe00005f0a10 bufspace_daemon() at bufspace_daemon+0x2db/frame 0xfffffe00005f0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005f0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005f0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100136 td 0xfffff80013a69000 sched_switch() at sched_switch+0x942/frame 0xfffffe00005f58b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005f58e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005f5920 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe00005f5960 _sleep() at _sleep+0x307/frame 0xfffffe00005f5a10 bufspace_daemon() at bufspace_daemon+0x2db/frame 0xfffffe00005f5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005f5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005f5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100137 td 0xfffff80013a67560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005fa8b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005fa8e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005fa920 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe00005fa960 _sleep() at _sleep+0x307/frame 0xfffffe00005faa10 bufspace_daemon() at bufspace_daemon+0x2db/frame 0xfffffe00005faa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005faab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005faab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100138 td 0xfffff80013a67000 sched_switch() at sched_switch+0x942/frame 0xfffffe00005ff8b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005ff8e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005ff920 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe00005ff960 _sleep() at _sleep+0x307/frame 0xfffffe00005ffa10 bufspace_daemon() at bufspace_daemon+0x2db/frame 0xfffffe00005ffa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005ffab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005ffab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100139 td 0xfffff80013a65560 sched_switch() at sched_switch+0x942/frame 0xfffffe009ec048b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ec048e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ec04920 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe009ec04960 _sleep() at _sleep+0x307/frame 0xfffffe009ec04a10 bufspace_daemon() at bufspace_daemon+0x2db/frame 0xfffffe009ec04a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009ec04ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009ec04ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100150 td 0xfffff80013a4b560 sched_switch() at sched_switch+0x942/frame 0xfffffe009ec3b8e0 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ec3b910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ec3b950 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe009ec3b990 _sleep() at _sleep+0x307/frame 0xfffffe009ec3ba40 softdep_flush() at softdep_flush+0x17f/frame 0xfffffe009ec3ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe009ec3bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009ec3bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100203 td 0xfffff80013e03000 sched_switch() at sched_switch+0x942/frame 0xfffffe009ec548e0 mi_switch() at mi_switch+0x18c/frame 0xfffffe009ec54910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe009ec54950 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe009ec54990 _sleep() at _sleep+0x307/frame 0xfffffe009ec54a40 softdep_flush() at softdep_flush+0x17f/frame 0xfffffe009ec54a70 fork_exit() at fork_exit+0x84/frame 0xfffffe009ec54ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009ec54ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100494 td 0xfffff8024a18e560 sched_switch() at sched_switch+0x942/frame 0xfffffe00a817b8e0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a817b910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a817b950 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe00a817b990 _sleep() at _sleep+0x307/frame 0xfffffe00a817ba40 softdep_flush() at softdep_flush+0x17f/frame 0xfffffe00a817ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a817bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a817bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100496 td 0xfffff80175ca1000 sched_switch() at sched_switch+0x942/frame 0xfffffe00a76778e0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00a7677910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00a7677950 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe00a7677990 _sleep() at _sleep+0x307/frame 0xfffffe00a7677a40 softdep_flush() at softdep_flush+0x17f/frame 0xfffffe00a7677a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a7677ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a7677ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command vmdaemon pid 30 tid 100127 td 0xfffff800133f0000 sched_switch() at sched_switch+0x942/frame 0xfffffe00005c88a0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005c88d0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005c8910 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005c8940 _sleep() at _sleep+0x367/frame 0xfffffe00005c89f0 vm_daemon() at vm_daemon+0x85/frame 0xfffffe00005c8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005c8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005c8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 29 tid 100126 td 0xfffff800133f0560 (CPU 11) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0000cc3df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0000cc3e10 trap() at trap+0x49/frame 0xfffffe0000cc3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0000cc3f20 --- trap 0x13, rip = 0xffffffff80e89511, rsp = 0xfffffe00005c3680, rbp = 0xfffffe00005c3a50 --- vm_pageout_worker() at vm_pageout_worker+0x1231/frame 0xfffffe00005c3a50 vm_pageout() at vm_pageout+0x133/frame 0xfffffe00005c3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005c3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005c3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 29 tid 100130 td 0xfffff80013a4a560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005dc6a0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005dc6d0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005dc710 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005dc740 _sleep() at _sleep+0x367/frame 0xfffffe00005dc7f0 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0xe09/frame 0xfffffe00005dca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005dcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005dcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 29 tid 100133 td 0xfffff80013a4a000 sched_switch() at sched_switch+0x942/frame 0xfffffe00005e68e0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005e6910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005e6950 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe00005e6990 _sleep() at _sleep+0x307/frame 0xfffffe00005e6a40 uma_reclaim_worker() at uma_reclaim_worker+0xda/frame 0xfffffe00005e6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005e6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005e6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon1 pid 28 tid 100125 td 0xfffff80013518560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005be8a0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005be8d0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005be910 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005be940 _sleep() at _sleep+0x367/frame 0xfffffe00005be9f0 enc_daemon() at enc_daemon+0x1ca/frame 0xfffffe00005bea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005beab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005beab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon0 pid 27 tid 100124 td 0xfffff8000380e000 sched_switch() at sched_switch+0x942/frame 0xfffffe00005b98a0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005b98d0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005b9910 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005b9940 _sleep() at _sleep+0x367/frame 0xfffffe00005b99f0 enc_daemon() at enc_daemon+0x1ca/frame 0xfffffe00005b9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005b9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005b9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command rand_harvestq pid 26 tid 100118 td 0xfffff800133fe560 sched_switch() at sched_switch+0x942/frame 0xfffffe000059b8b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe000059b8e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000059b920 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe000059b960 _sleep() at _sleep+0x307/frame 0xfffffe000059ba10 random_kthread() at random_kthread+0x57/frame 0xfffffe000059ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe000059bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000059bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sctp_iterator pid 25 tid 100117 td 0xfffff800133ff000 sched_switch() at sched_switch+0x942/frame 0xfffffe0000596900 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000596930 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000596970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005969a0 _sleep() at _sleep+0x367/frame 0xfffffe0000596a50 sctp_iterator_thread() at sctp_iterator_thread+0x59/frame 0xfffffe0000596a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000596ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000596ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod4 pid 24 tid 100116 td 0xfffff8000380e560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005908d0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000590900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000590940 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe0000590980 _sleep() at _sleep+0x307/frame 0xfffffe0000590a30 soaio_kproc_loop() at soaio_kproc_loop+0x196/frame 0xfffffe0000590a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000590ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000590ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod3 pid 23 tid 100115 td 0xfffff8000380f000 sched_switch() at sched_switch+0x942/frame 0xfffffe000058b8d0 mi_switch() at mi_switch+0x18c/frame 0xfffffe000058b900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000058b940 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe000058b980 _sleep() at _sleep+0x307/frame 0xfffffe000058ba30 soaio_kproc_loop() at soaio_kproc_loop+0x196/frame 0xfffffe000058ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe000058bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000058bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod2 pid 22 tid 100114 td 0xfffff8000380f560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005868d0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000586900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000586940 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe0000586980 _sleep() at _sleep+0x307/frame 0xfffffe0000586a30 soaio_kproc_loop() at soaio_kproc_loop+0x196/frame 0xfffffe0000586a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000586ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000586ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod1 pid 21 tid 100113 td 0xfffff80003810000 sched_switch() at sched_switch+0x942/frame 0xfffffe00005818d0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000581900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000581940 sleepq_timedwait() at sleepq_timedwait+0x50/frame 0xfffffe0000581980 _sleep() at _sleep+0x307/frame 0xfffffe0000581a30 soaio_kproc_loop() at soaio_kproc_loop+0x196/frame 0xfffffe0000581a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000581ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000581ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100094 td 0xfffff80003cdf560 sched_switch() at sched_switch+0x942/frame 0xfffffe0000522930 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000522960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005229a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005229d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe0000522a30 usb_process() at usb_process+0x12b/frame 0xfffffe0000522a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000522ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000522ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100095 td 0xfffff80003cdf000 sched_switch() at sched_switch+0x942/frame 0xfffffe0000527930 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000527960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005279a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005279d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe0000527a30 usb_process() at usb_process+0x12b/frame 0xfffffe0000527a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000527ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000527ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100096 td 0xfffff80003cde560 sched_switch() at sched_switch+0x942/frame 0xfffffe000052c930 mi_switch() at mi_switch+0x18c/frame 0xfffffe000052c960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000052c9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000052c9d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe000052ca30 usb_process() at usb_process+0x12b/frame 0xfffffe000052ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe000052cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000052cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100097 td 0xfffff80003cde000 sched_switch() at sched_switch+0x942/frame 0xfffffe0000531930 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000531960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005319a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005319d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe0000531a30 usb_process() at usb_process+0x12b/frame 0xfffffe0000531a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000531ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000531ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100098 td 0xfffff80003cd6560 sched_switch() at sched_switch+0x942/frame 0xfffffe0000536930 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000536960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005369a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005369d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe0000536a30 usb_process() at usb_process+0x12b/frame 0xfffffe0000536a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000536ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000536ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100100 td 0xfffff80003815560 sched_switch() at sched_switch+0x942/frame 0xfffffe0000540930 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000540960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005409a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005409d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe0000540a30 usb_process() at usb_process+0x12b/frame 0xfffffe0000540a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000540ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000540ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100101 td 0xfffff80006075000 sched_switch() at sched_switch+0x942/frame 0xfffffe0000545930 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000545960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005459a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005459d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe0000545a30 usb_process() at usb_process+0x12b/frame 0xfffffe0000545a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000545ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000545ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100102 td 0xfffff80006074560 sched_switch() at sched_switch+0x942/frame 0xfffffe000054a930 mi_switch() at mi_switch+0x18c/frame 0xfffffe000054a960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000054a9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000054a9d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe000054aa30 usb_process() at usb_process+0x12b/frame 0xfffffe000054aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe000054aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000054aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100103 td 0xfffff80006074000 sched_switch() at sched_switch+0x942/frame 0xfffffe000054f930 mi_switch() at mi_switch+0x18c/frame 0xfffffe000054f960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000054f9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000054f9d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe000054fa30 usb_process() at usb_process+0x12b/frame 0xfffffe000054fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe000054fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000054fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100104 td 0xfffff80006073560 sched_switch() at sched_switch+0x942/frame 0xfffffe0000554930 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000554960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005549a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005549d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe0000554a30 usb_process() at usb_process+0x12b/frame 0xfffffe0000554a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000554ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000554ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100105 td 0xfffff8000614a560 sched_switch() at sched_switch+0x942/frame 0xfffffe0000559930 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000559960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005599a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005599d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe0000559a30 usb_process() at usb_process+0x12b/frame 0xfffffe0000559a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000559ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000559ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100106 td 0xfffff8000614a000 sched_switch() at sched_switch+0x942/frame 0xfffffe000055e930 mi_switch() at mi_switch+0x18c/frame 0xfffffe000055e960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000055e9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000055e9d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe000055ea30 usb_process() at usb_process+0x12b/frame 0xfffffe000055ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe000055eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000055eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100107 td 0xfffff80006149560 sched_switch() at sched_switch+0x942/frame 0xfffffe0000563930 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000563960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005639a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005639d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe0000563a30 usb_process() at usb_process+0x12b/frame 0xfffffe0000563a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000563ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000563ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100108 td 0xfffff80006149000 sched_switch() at sched_switch+0x942/frame 0xfffffe0000568930 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000568960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005689a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005689d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe0000568a30 usb_process() at usb_process+0x12b/frame 0xfffffe0000568a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000568ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000568ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 20 tid 100109 td 0xfffff80006148560 sched_switch() at sched_switch+0x942/frame 0xfffffe000056d930 mi_switch() at mi_switch+0x18c/frame 0xfffffe000056d960 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000056d9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000056d9d0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe000056da30 usb_process() at usb_process+0x12b/frame 0xfffffe000056da70 fork_exit() at fork_exit+0x84/frame 0xfffffe000056dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000056dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 19 tid 100089 td 0xfffff80003811560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005098d0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000509900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000509940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0000509970 _sleep() at _sleep+0x367/frame 0xfffffe0000509a20 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe0000509a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000509ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000509ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 19 tid 100090 td 0xfffff80003811000 sched_switch() at sched_switch+0x942/frame 0xfffffe000050e8d0 mi_switch() at mi_switch+0x18c/frame 0xfffffe000050e900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000050e940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000050e970 _sleep() at _sleep+0x367/frame 0xfffffe000050ea20 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe000050ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe000050eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000050eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 19 tid 100091 td 0xfffff80003810560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005138d0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000513900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000513940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0000513970 _sleep() at _sleep+0x367/frame 0xfffffe0000513a20 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe0000513a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000513ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000513ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 19 tid 100123 td 0xfffff800133fc000 sched_switch() at sched_switch+0x942/frame 0xfffffe00005b48c0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005b48f0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005b4930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005b4960 _sleep() at _sleep+0x367/frame 0xfffffe00005b4a10 xpt_scanner_thread() at xpt_scanner_thread+0x93/frame 0xfffffe00005b4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005b4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005b4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 11 pid 18 tid 100088 td 0xfffff80003815000 sched_switch() at sched_switch+0x942/frame 0xfffffe00005048b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005048e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000504920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0000504950 _sleep() at _sleep+0x367/frame 0xfffffe0000504a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0000504a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000504ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000504ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 10 pid 17 tid 100087 td 0xfffff800037e7560 sched_switch() at sched_switch+0x942/frame 0xfffffe00004ff8b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004ff8e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004ff920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004ff950 _sleep() at _sleep+0x367/frame 0xfffffe00004ffa00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00004ffa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004ffab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004ffab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 9 pid 16 tid 100086 td 0xfffff800037f6000 sched_switch() at sched_switch+0x942/frame 0xfffffe00004fa8b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004fa8e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004fa920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004fa950 _sleep() at _sleep+0x367/frame 0xfffffe00004faa00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00004faa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004faab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004faab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 8 pid 15 tid 100085 td 0xfffff800037f6560 sched_switch() at sched_switch+0x942/frame 0xfffffe00004f58b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004f58e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004f5920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004f5950 _sleep() at _sleep+0x367/frame 0xfffffe00004f5a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00004f5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004f5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004f5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 7 pid 14 tid 100084 td 0xfffff800037f7000 sched_switch() at sched_switch+0x942/frame 0xfffffe00004f08b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004f08e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004f0920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004f0950 _sleep() at _sleep+0x367/frame 0xfffffe00004f0a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00004f0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004f0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004f0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 6 pid 9 tid 100083 td 0xfffff800037f7560 sched_switch() at sched_switch+0x942/frame 0xfffffe00004eb8b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004eb8e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004eb920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004eb950 _sleep() at _sleep+0x367/frame 0xfffffe00004eba00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00004eba70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004ebab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004ebab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 5 pid 8 tid 100082 td 0xfffff800037f8000 sched_switch() at sched_switch+0x942/frame 0xfffffe00004e68b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004e68e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004e6920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004e6950 _sleep() at _sleep+0x367/frame 0xfffffe00004e6a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00004e6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004e6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004e6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 4 pid 7 tid 100081 td 0xfffff800037f8560 sched_switch() at sched_switch+0x942/frame 0xfffffe00004e18b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004e18e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004e1920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004e1950 _sleep() at _sleep+0x367/frame 0xfffffe00004e1a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00004e1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004e1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004e1ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 3 pid 6 tid 100080 td 0xfffff800037f9000 sched_switch() at sched_switch+0x942/frame 0xfffffe00004dc8b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004dc8e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004dc920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004dc950 _sleep() at _sleep+0x367/frame 0xfffffe00004dca00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00004dca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004dcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004dcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 2 pid 5 tid 100079 td 0xfffff800037f9560 sched_switch() at sched_switch+0x942/frame 0xfffffe00004d78b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004d78e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004d7920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004d7950 _sleep() at _sleep+0x367/frame 0xfffffe00004d7a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00004d7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004d7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004d7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 1 pid 4 tid 100078 td 0xfffff800037e3000 sched_switch() at sched_switch+0x942/frame 0xfffffe00004d28b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004d28e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004d2920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004d2950 _sleep() at _sleep+0x367/frame 0xfffffe00004d2a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00004d2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004d2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004d2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 0 pid 3 tid 100077 td 0xfffff800037e3560 sched_switch() at sched_switch+0x942/frame 0xfffffe00004cd8b0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004cd8e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004cd920 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004cd950 _sleep() at _sleep+0x367/frame 0xfffffe00004cda00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00004cda70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004cdab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004cdab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 2 tid 100076 td 0xfffff800037e4000 sched_switch() at sched_switch+0x942/frame 0xfffffe00004c88e0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004c8910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004c8950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004c8980 _sleep() at _sleep+0x367/frame 0xfffffe00004c8a30 crypto_proc() at crypto_proc+0x2d4/frame 0xfffffe00004c8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004c8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004c8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100060 td 0xfffff80003756000 sched_switch() at sched_switch+0x942/frame 0xfffffe00004788e0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000478910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000478950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0000478980 _sleep() at _sleep+0x367/frame 0xfffffe0000478a30 g_run_events() at g_run_events+0x4f/frame 0xfffffe0000478a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000478ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000478ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100061 td 0xfffff80003797560 sched_switch() at sched_switch+0x942/frame 0xfffffe000047d8e0 mi_switch() at mi_switch+0x18c/frame 0xfffffe000047d910 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000047d950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000047d980 _sleep() at _sleep+0x367/frame 0xfffffe000047da30 g_io_schedule_up() at g_io_schedule_up+0xa5/frame 0xfffffe000047da60 g_up_procbody() at g_up_procbody+0x6d/frame 0xfffffe000047da70 fork_exit() at fork_exit+0x84/frame 0xfffffe000047dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000047dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100062 td 0xfffff80003797000 sched_switch() at sched_switch+0x942/frame 0xfffffe00004828d0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000482900 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000482940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0000482970 _sleep() at _sleep+0x367/frame 0xfffffe0000482a20 g_io_schedule_down() at g_io_schedule_down+0x14c/frame 0xfffffe0000482a60 g_down_procbody() at g_down_procbody+0x6d/frame 0xfffffe0000482a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000482ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000482ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100016 td 0xfffff80003641560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100019 td 0xfffff80003679000 sched_switch() at sched_switch+0x942/frame 0xfffffe00789aa9f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789aaa20 ithread_loop() at ithread_loop+0x1f1/frame 0xfffffe00789aaa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789aaab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789aaab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100020 td 0xfffff80003678560 sched_switch() at sched_switch+0x942/frame 0xfffffe00789af9f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789afa20 ithread_loop() at ithread_loop+0x1f1/frame 0xfffffe00789afa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789afab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789afab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100046 td 0xfffff80003740000 sched_switch() at sched_switch+0x942/frame 0xfffffe00004329f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000432a20 ithread_loop() at ithread_loop+0x1f1/frame 0xfffffe0000432a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000432ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000432ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100047 td 0xfffff8000373f560 sched_switch() at sched_switch+0x942/frame 0xfffffe00004379f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000437a20 ithread_loop() at ithread_loop+0x1f1/frame 0xfffffe0000437a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000437ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000437ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100048 td 0xfffff8000373f000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100049 td 0xfffff8000373e560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100050 td 0xfffff8000373e000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100051 td 0xfffff80003737560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100052 td 0xfffff8000375a000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100053 td 0xfffff80003759560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100054 td 0xfffff80003759000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100055 td 0xfffff80003758560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100056 td 0xfffff80003758000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100057 td 0xfffff80003757560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100058 td 0xfffff80003757000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100059 td 0xfffff80003756560 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100092 td 0xfffff80003ce0560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005189f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000518a20 ithread_loop() at ithread_loop+0x1f1/frame 0xfffffe0000518a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000518ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000518ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100093 td 0xfffff80003ce0000 sched_switch() at sched_switch+0x942/frame 0xfffffe000051d9f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe000051da20 ithread_loop() at ithread_loop+0x1f1/frame 0xfffffe000051da70 fork_exit() at fork_exit+0x84/frame 0xfffffe000051dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000051dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100099 td 0xfffff80003cd6000 sched_switch() at sched_switch+0x942/frame 0xfffffe000053b9f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe000053ba20 ithread_loop() at ithread_loop+0x1f1/frame 0xfffffe000053ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe000053bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000053bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100110 td 0xfffff80006148000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100111 td 0xfffff8000613c560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005779f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000577a20 ithread_loop() at ithread_loop+0x1f1/frame 0xfffffe0000577a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000577ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000577ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100003 td 0xfffff80003632000 (CPU 0) kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe007895a700 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe007895a710 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe007895a740 uart_intr() at uart_intr+0x138/frame 0xfffffe007895a780 intr_event_handle() at intr_event_handle+0xbc/frame 0xfffffe007895a7d0 intr_execute_handlers() at intr_execute_handlers+0x58/frame 0xfffffe007895a800 lapic_handle_intr() at lapic_handle_intr+0x6d/frame 0xfffffe007895a840 apic_isr1_u() at apic_isr1_u+0xc1/frame 0xfffffe007895a910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895a980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895a9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xfffff80003631560 (CPU 1) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00007ffdf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00007ffe10 trap() at trap+0x49/frame 0xfffffe00007fff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00007fff20 --- trap 0x13, rip = 0xffffffff81160ce6, rsp = 0xfffffe007895f910, rbp = 0xfffffe007895f910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007895f910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007895f960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007895f980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007895f9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007895fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007895fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007895fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffff80003631000 (CPU 2) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00071b8df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00071b8e10 trap() at trap+0x49/frame 0xfffffe00071b8f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00071b8f20 --- trap 0x13, rip = 0xffffffff81160ce6, rsp = 0xfffffe0078964910, rbp = 0xfffffe0078964910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0078964910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe0078964960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078964980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe00789649a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078964a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078964ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078964ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xfffff80003630560 (CPU 3) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00071c3df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00071c3e10 trap() at trap+0x49/frame 0xfffffe00071c3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00071c3f20 --- trap 0x13, rip = 0xffffffff81160ce6, rsp = 0xfffffe0078969910, rbp = 0xfffffe0078969910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0078969910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe0078969960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078969980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe00789699a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078969a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078969ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078969ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100007 td 0xfffff80003630000 (CPU 4) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00071cedf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00071cee10 trap() at trap+0x49/frame 0xfffffe00071cef20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00071cef20 --- trap 0x13, rip = 0xffffffff81160ce6, rsp = 0xfffffe007896e910, rbp = 0xfffffe007896e910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007896e910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007896e960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007896e980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007896e9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007896ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe007896eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007896eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xfffff8000362f560 (CPU 5) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00071d9df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00071d9e10 trap() at trap+0x49/frame 0xfffffe00071d9f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00071d9f20 --- trap 0x13, rip = 0xffffffff81160ce6, rsp = 0xfffffe0078973910, rbp = 0xfffffe0078973910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0078973910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe0078973960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078973980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe00789739a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078973a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078973ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078973ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xfffff8000362f000 (CPU 6) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00071e4df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00071e4e10 trap() at trap+0x49/frame 0xfffffe00071e4f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00071e4f20 --- trap 0x13, rip = 0xffffffff81160ce6, rsp = 0xfffffe0078978910, rbp = 0xfffffe0078978910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0078978910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe0078978960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078978980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe00789789a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078978a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078978ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078978ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xfffff80003644560 (CPU 7) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00071efdf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00071efe10 trap() at trap+0x49/frame 0xfffffe00071eff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00071eff20 --- trap 0x13, rip = 0xffffffff81160ce6, rsp = 0xfffffe007897d910, rbp = 0xfffffe007897d910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007897d910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007897d960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007897d980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007897d9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007897da70 fork_exit() at fork_exit+0x84/frame 0xfffffe007897dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007897dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffff80003644000 (CPU 8) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00071fadf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe00071fae10 trap() at trap+0x49/frame 0xfffffe00071faf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00071faf20 --- trap 0x13, rip = 0xffffffff81160ce6, rsp = 0xfffffe0078982910, rbp = 0xfffffe0078982910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0078982910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe0078982960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078982980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe00789829a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078982a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078982ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078982ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100012 td 0xfffff80003643560 (CPU 9) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0000caddf0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0000cade10 trap() at trap+0x49/frame 0xfffffe0000cadf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0000cadf20 --- trap 0x13, rip = 0xffffffff81160ce6, rsp = 0xfffffe0078987910, rbp = 0xfffffe0078987910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0078987910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe0078987960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0078987980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe00789879a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078987a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078987ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078987ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xfffff80003643000 (CPU 10) cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0000cb8df0 ipi_nmi_handler() at ipi_nmi_handler+0x44/frame 0xfffffe0000cb8e10 trap() at trap+0x49/frame 0xfffffe0000cb8f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0000cb8f20 --- trap 0x13, rip = 0xffffffff81160ce6, rsp = 0xfffffe007898c910, rbp = 0xfffffe007898c910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007898c910 acpi_cpu_idle() at acpi_cpu_idle+0x2ee/frame 0xfffffe007898c960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007898c980 cpu_idle() at cpu_idle+0x8f/frame 0xfffffe007898c9a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe007898ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe007898cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007898cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100014 td 0xfffff80003642560 sched_switch() at sched_switch+0x942/frame 0xfffffe0078991950 mi_switch() at mi_switch+0x18c/frame 0xfffffe0078991980 critical_exit() at critical_exit+0x8c/frame 0xfffffe00789919a0 sched_idletd() at sched_idletd+0x517/frame 0xfffffe0078991a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078991ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078991ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command init pid 1 tid 100002 td 0xfffff80003632560 sched_switch() at sched_switch+0x942/frame 0xfffffe0078955550 mi_switch() at mi_switch+0x18c/frame 0xfffffe0078955580 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789555c0 sleepq_catch_signals() at sleepq_catch_signals+0x300/frame 0xfffffe0078955620 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0078955650 _sleep() at _sleep+0x359/frame 0xfffffe0078955700 kern_wait6() at kern_wait6+0x4c4/frame 0xfffffe0078955790 sys_wait4() at sys_wait4+0x78/frame 0xfffffe0078955980 amd64_syscall() at amd64_syscall+0x7ab/frame 0xfffffe0078955ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0078955ab0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x2a71ca, rsp = 0x7fffffffe788, rbp = 0x7fffffffe7f0 --- Tracing command audit pid 10 tid 100001 td 0xfffff80003633000 sched_switch() at sched_switch+0x942/frame 0xfffffe0078950910 mi_switch() at mi_switch+0x18c/frame 0xfffffe0078950940 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0078950980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789509b0 _cv_wait() at _cv_wait+0x203/frame 0xfffffe0078950a10 audit_worker() at audit_worker+0x93/frame 0xfffffe0078950a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078950ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078950ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xffffffff81feb860 sched_switch() at sched_switch+0x942/frame 0xffffffff825ae9b0 mi_switch() at mi_switch+0x18c/frame 0xffffffff825ae9e0 sleepq_switch() at sleepq_switch+0x10d/frame 0xffffffff825aea20 sleepq_wait() at sleepq_wait+0x43/frame 0xffffffff825aea50 _sleep() at _sleep+0x367/frame 0xffffffff825aeb00 vm_wait_min() at vm_wait_min+0x64/frame 0xffffffff825aeb20 swapper() at swapper+0x25/frame 0xffffffff825aeb70 btext() at btext+0x2c Tracing command kernel pid 0 tid 100015 td 0xfffff80003642000 sched_switch() at sched_switch+0x942/frame 0xfffffe00789968f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0078996920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0078996960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0078996990 _sleep() at _sleep+0x367/frame 0xfffffe0078996a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0078996a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0078996ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0078996ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100017 td 0xfffff80003641000 sched_switch() at sched_switch+0x942/frame 0xfffffe00789a08f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789a0920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789a0960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789a0990 _sleep() at _sleep+0x367/frame 0xfffffe00789a0a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00789a0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789a0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789a0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100018 td 0xfffff80003633560 sched_switch() at sched_switch+0x942/frame 0xfffffe00789a58f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789a5920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789a5960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789a5990 _sleep() at _sleep+0x367/frame 0xfffffe00789a5a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00789a5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789a5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789a5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xfffff80003678000 sched_switch() at sched_switch+0x942/frame 0xfffffe00789b4920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789b4950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789b4990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789b49c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789b4a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789b4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789b4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789b4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xfffff80003677560 sched_switch() at sched_switch+0x942/frame 0xfffffe00789b9920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789b9950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789b9990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789b99c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789b9a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789b9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789b9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789b9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xfffff80003677000 sched_switch() at sched_switch+0x942/frame 0xfffffe00789be920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789be950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789be990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789be9c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789bea40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789bea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789beab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789beab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xfffff80003676560 sched_switch() at sched_switch+0x942/frame 0xfffffe00789c3920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789c3950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789c3990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789c39c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789c3a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789c3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789c3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789c3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xfffff80003676000 sched_switch() at sched_switch+0x942/frame 0xfffffe00789c8920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789c8950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789c8990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789c89c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789c8a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789c8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789c8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789c8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100026 td 0xfffff80003675560 sched_switch() at sched_switch+0x942/frame 0xfffffe00789cd920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789cd950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789cd990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789cd9c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789cda40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789cda70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789cdab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789cdab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100027 td 0xfffff80003675000 sched_switch() at sched_switch+0x942/frame 0xfffffe00789d2920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789d2950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789d2990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789d29c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789d2a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789d2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789d2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789d2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100028 td 0xfffff80003686560 sched_switch() at sched_switch+0x942/frame 0xfffffe00789d7920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789d7950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789d7990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789d79c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789d7a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789d7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789d7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789d7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100029 td 0xfffff80003686000 sched_switch() at sched_switch+0x942/frame 0xfffffe00789dc920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789dc950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789dc990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789dc9c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789dca40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789dca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789dcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789dcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100030 td 0xfffff80003685560 sched_switch() at sched_switch+0x942/frame 0xfffffe00789e1920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789e1950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789e1990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789e19c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789e1a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789e1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789e1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789e1ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100031 td 0xfffff80003685000 sched_switch() at sched_switch+0x942/frame 0xfffffe00789e6920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789e6950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789e6990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789e69c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789e6a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789e6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789e6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789e6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100032 td 0xfffff80003684560 sched_switch() at sched_switch+0x942/frame 0xfffffe00789eb920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789eb950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789eb990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789eb9c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789eba40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789eba70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789ebab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789ebab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100033 td 0xfffff80003684000 sched_switch() at sched_switch+0x942/frame 0xfffffe00789f0920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789f0950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789f0990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789f09c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789f0a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789f0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789f0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789f0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100034 td 0xfffff80003737000 sched_switch() at sched_switch+0x942/frame 0xfffffe00789f5920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789f5950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789f5990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789f59c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789f5a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789f5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789f5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789f5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100035 td 0xfffff80003736560 sched_switch() at sched_switch+0x942/frame 0xfffffe00789fa920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789fa950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789fa990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789fa9c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789faa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789faa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789faab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789faab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100036 td 0xfffff80003736000 sched_switch() at sched_switch+0x942/frame 0xfffffe00789ff920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00789ff950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00789ff990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00789ff9c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00789ffa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00789ffa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00789ffab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00789ffab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100037 td 0xfffff80003735560 sched_switch() at sched_switch+0x942/frame 0xfffffe0000405920 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000405950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000405990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004059c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe0000405a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000405a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000405ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000405ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100038 td 0xfffff80003735000 sched_switch() at sched_switch+0x942/frame 0xfffffe000040a920 mi_switch() at mi_switch+0x18c/frame 0xfffffe000040a950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000040a990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000040a9c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe000040aa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000040aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe000040aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000040aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100039 td 0xfffff80003734560 sched_switch() at sched_switch+0x942/frame 0xfffffe000040f920 mi_switch() at mi_switch+0x18c/frame 0xfffffe000040f950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000040f990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000040f9c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe000040fa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000040fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe000040fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000040fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100040 td 0xfffff80003734000 sched_switch() at sched_switch+0x942/frame 0xfffffe0000414920 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000414950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000414990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004149c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe0000414a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000414a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000414ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000414ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100041 td 0xfffff80003728560 sched_switch() at sched_switch+0x942/frame 0xfffffe0000419920 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000419950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000419990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004199c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe0000419a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000419a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000419ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000419ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100042 td 0xfffff80003728000 sched_switch() at sched_switch+0x942/frame 0xfffffe000041e920 mi_switch() at mi_switch+0x18c/frame 0xfffffe000041e950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000041e990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000041e9c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe000041ea40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000041ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe000041eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000041eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100043 td 0xfffff80003741560 sched_switch() at sched_switch+0x942/frame 0xfffffe0000423920 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000423950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000423990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004239c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe0000423a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000423a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000423ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000423ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100044 td 0xfffff80003741000 sched_switch() at sched_switch+0x942/frame 0xfffffe0000428920 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000428950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000428990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004289c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe0000428a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000428a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000428ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000428ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100045 td 0xfffff80003740560 sched_switch() at sched_switch+0x942/frame 0xfffffe000042d920 mi_switch() at mi_switch+0x18c/frame 0xfffffe000042d950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000042d990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000042d9c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe000042da40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000042da70 fork_exit() at fork_exit+0x84/frame 0xfffffe000042dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000042dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100063 td 0xfffff80003796560 sched_switch() at sched_switch+0x942/frame 0xfffffe00004878f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000487920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000487960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0000487990 _sleep() at _sleep+0x367/frame 0xfffffe0000487a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0000487a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000487ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000487ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100064 td 0xfffff80003796000 sched_switch() at sched_switch+0x942/frame 0xfffffe000048c8f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe000048c920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000048c960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000048c990 _sleep() at _sleep+0x367/frame 0xfffffe000048ca40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe000048ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe000048cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000048cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100065 td 0xfffff80003795560 sched_switch() at sched_switch+0x942/frame 0xfffffe00004918f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000491920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000491960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0000491990 _sleep() at _sleep+0x367/frame 0xfffffe0000491a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0000491a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000491ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000491ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100066 td 0xfffff80003795000 sched_switch() at sched_switch+0x942/frame 0xfffffe00004968f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe0000496920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe0000496960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0000496990 _sleep() at _sleep+0x367/frame 0xfffffe0000496a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0000496a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000496ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000496ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100067 td 0xfffff80003794560 sched_switch() at sched_switch+0x942/frame 0xfffffe000049b8f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe000049b920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000049b960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000049b990 _sleep() at _sleep+0x367/frame 0xfffffe000049ba40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe000049ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe000049bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000049bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100068 td 0xfffff80003794000 sched_switch() at sched_switch+0x942/frame 0xfffffe00004a08f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004a0920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004a0960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004a0990 _sleep() at _sleep+0x367/frame 0xfffffe00004a0a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00004a0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004a0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004a0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100069 td 0xfffff8000375a560 sched_switch() at sched_switch+0x942/frame 0xfffffe00004a58f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004a5920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004a5960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004a5990 _sleep() at _sleep+0x367/frame 0xfffffe00004a5a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00004a5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004a5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004a5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100070 td 0xfffff800037e7000 sched_switch() at sched_switch+0x942/frame 0xfffffe00004aa8f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004aa920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004aa960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004aa990 _sleep() at _sleep+0x367/frame 0xfffffe00004aaa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00004aaa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004aaab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004aaab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100071 td 0xfffff800037e6560 sched_switch() at sched_switch+0x942/frame 0xfffffe00004af8f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004af920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004af960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004af990 _sleep() at _sleep+0x367/frame 0xfffffe00004afa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00004afa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004afab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004afab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100072 td 0xfffff800037e6000 sched_switch() at sched_switch+0x942/frame 0xfffffe00004b48f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004b4920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004b4960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004b4990 _sleep() at _sleep+0x367/frame 0xfffffe00004b4a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00004b4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004b4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004b4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100073 td 0xfffff800037e5560 sched_switch() at sched_switch+0x942/frame 0xfffffe00004b98f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004b9920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004b9960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004b9990 _sleep() at _sleep+0x367/frame 0xfffffe00004b9a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00004b9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004b9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004b9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100074 td 0xfffff800037e5000 sched_switch() at sched_switch+0x942/frame 0xfffffe00004be8f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004be920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004be960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004be990 _sleep() at _sleep+0x367/frame 0xfffffe00004bea40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00004bea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004beab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004beab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100075 td 0xfffff800037e4560 sched_switch() at sched_switch+0x942/frame 0xfffffe00004c38f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00004c3920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00004c3960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004c3990 _sleep() at _sleep+0x367/frame 0xfffffe00004c3a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00004c3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004c3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004c3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100112 td 0xfffff8000613c000 sched_switch() at sched_switch+0x942/frame 0xfffffe000057c920 mi_switch() at mi_switch+0x18c/frame 0xfffffe000057c950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe000057c990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000057c9c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe000057ca40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe000057ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe000057cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000057cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100119 td 0xfffff800133fe000 sched_switch() at sched_switch+0x942/frame 0xfffffe00005a0920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005a0950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005a0990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005a09c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00005a0a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00005a0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005a0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005a0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100120 td 0xfffff800133fd560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005a5920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005a5950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005a5990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005a59c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00005a5a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00005a5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005a5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005a5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100121 td 0xfffff800133fd000 sched_switch() at sched_switch+0x942/frame 0xfffffe00005aa920 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005aa950 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005aa990 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005aa9c0 msleep_spin_sbt() at msleep_spin_sbt+0x1eb/frame 0xfffffe00005aaa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00005aaa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005aaab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005aaab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100122 td 0xfffff800133fc560 sched_switch() at sched_switch+0x942/frame 0xfffffe00005af8f0 mi_switch() at mi_switch+0x18c/frame 0xfffffe00005af920 sleepq_switch() at sleepq_switch+0x10d/frame 0xfffffe00005af960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005af990 _sleep() at _sleep+0x367/frame 0xfffffe00005afa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005afa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005afab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005afab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db:0:pho> dump Dumping 2644 out of 32625 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Dump complete db> reset (kgdb) proc 92126 [Switching to thread 188 (Thread 105410)]#0 sched_switch (td=0xfffff803cad4a560, newtd=0xfffff80003643560, flags=) at ../../../kern/sched_ule.c:2115 2115 cpuid = PCPU_GET(cpuid); Current language: auto; currently minimal (kgdb) bt #0 sched_switch (td=0xfffff803cad4a560, newtd=0xfffff80003643560, flags=) at ../../../kern/sched_ule.c:2115 #1 0xffffffff80b4bd1c in mi_switch (flags=0x104, newtd=0x0) at ../../../kern/kern_synch.c:437 #2 0xffffffff80b9506d in sleepq_switch (wchan=0xffffffff81a49880, pri=0x78) at ../../../kern/subr_sleepqueue.c:613 #3 0xffffffff80b94f13 in sleepq_wait (wchan=0xffffffff81a49880, pri=0x78) at ../../../kern/subr_sleepqueue.c:692 #4 0xffffffff80b4b567 in _sleep (ident=0xffffffff81a49880, lock=, priority=0x78, wmesg=0xffffffff815e9a0d "pfault", sbt=0x0, pr=0x0, flags=0x100) at ../../../kern/kern_synch.c:216 #5 0xffffffff80e83871 in vm_waitpfault () at ../../../vm/vm_page.c:3034 #6 0xffffffff80e6a1b7 in vm_fault_hold (map=0xfffff80447388000, vaddr=, fault_type=0x1, fault_flags=, m_hold=0xfffffe00a7f4e748) at ../../../vm/vm_fault.c:787 #7 0xffffffff80e6c2c9 in vm_fault_quick_hold_pages (map=0xfffff80447388000, addr=0x8313ed000, len=, prot=0x1, ma=, max_count=) at ../../../vm/vm_fault.c:1540 #8 0xffffffff80c246da in vn_io_fault1 () at ../../../kern/vfs_vnops.c:1119 #9 0xffffffff80c22935 in vn_io_fault (fp=, uio=0xfffff80495195c00, active_cred=0xfffff8001de5e410, flags=, td=) at ../../../kern/vfs_vnops.c:1187 #10 0xffffffff80bac7f7 in dofilewrite (td=0xfffff803cad4a560, fd=0x4, fp=0xfffff80013c154b0, auio=0xfffffe00a7f4e940, offset=, flags=0x0) at file.h:309 #11 0xffffffff80bac3e8 in kern_writev (td=0xfffff803cad4a560, fd=0x4, auio=0xfffffe00a7f4e940) at ../../../kern/sys_generic.c:507 #12 0xffffffff80bac376 in sys_write (td=, uap=) at ../../../kern/sys_generic.c:421 #13 0xffffffff8100607b in amd64_syscall (td=0xfffff803cad4a560, traced=0x0) at subr_syscall.c:134 #14 0xffffffff80fe0e0d in fast_syscall_common () at ../../../amd64/amd64/exception.S:479 #15 0x000000080083ba3a in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) f 5 #5 0xffffffff80e83871 in vm_waitpfault () at ../../../vm/vm_page.c:3034 3034 msleep(&vm_min_domains, &vm_domainset_lock, PUSER, "pfault", 0); (kgdb) l 3029 { 3030 3031 mtx_lock(&vm_domainset_lock); 3032 if (vm_page_count_min()) { 3033 vm_min_waiters++; 3034 msleep(&vm_min_domains, &vm_domainset_lock, PUSER, "pfault", 0); 3035 } 3036 mtx_unlock(&vm_domainset_lock); 3037 } 3038 (kgdb) info loc No locals. (kgdb) p vm_min_waiters $1 = 0x8 (kgdb) what /usr/src/sys/amd64/compile/PHO/kernel.debug /usr/src/sys/amd64/compile/PHO/kernel.debug: ls -l /usr/src/sys/amd64/compile/PHO/kernel.debug /var/crash/vmcore.73 -rwxr-xr-x 1 root wheel 85617992 Apr 10 18:31 /usr/src/sys/amd64/compile/PHO/kernel.debug -rw------- 1 root wheel 2772611072 Apr 10 16:30 /var/crash/vmcore.73 vmstat -N /usr/src/sys/amd64/compile/PHO/kernel.debug -M /var/crash/vmcore.73 -m Type InUse MemUse HighUse Requests Size(s) feeder 7 1K - 7 32 CAM I/O Scheduler 1 1K - 1 128 CAM queue 29 9K - 89 16,32,512 UART 6 5K - 6 16,1024 USB 53 72K - 58 16,128,256,512,1024,2048,4096 USBdev 39 6K - 3974 32,64,128,256,512,4096 CAM dev queue 13 1K - 13 64 vtbuf 24 1968K - 46 4096 vt 11 6K - 11 512 DEVFS2 184 3K - 184 16 DEVFS3 294 74K - 2504823 256 DEVFS1 271 136K - 2495883 512 DEVFS 17 1K - 33 16,128 DEVFSP 2 1K - 5938 64 NFSD V4client 1 1K - 1 256 NFSD lckfile 1 1K - 1 256 NFSD session 1 1K - 1 1024 pfs_nodes 20 5K - 20 256 pfs_vncache 1 1K - 10651867 64 GEOM 1529 122K - 2105953 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768 raid_data 0 0K - 555012 32,128,256 isadev 7 1K - 7 128 SCSI ENC 46 199K - 21272 16,64,256,2048,32768 ddb_capture 1 64K - 1 65536 cdev 3 1K - 3 256 filedesc 17 133K - 10018524 16,32,64,128,256,4096,8192,16384,32768,65536 sigio 0 0K - 19 64 filecaps 4 1K - 16465 16,32,64 kdtrace 274 54K - 23051259 64,256 kenv 146 14K - 160 16,32,64,128,8192 kqueue 80 5K - 11388456 64,256,512,2048 proc-args 139 5K - 5928447 16,32,64,128,256 hhook 13 4K - 13 256 ithread 162 30K - 162 32,128,256 KTRACE 100 13K - 100 128 linker 300 219K - 384 16,32,64,128,256,512,1024,2048,4096,8192,16384,65536 lockf 20 3K - 617011702 64,128,256,512 loginclass 3 1K - 3 64 cache 1 1K - 1 32 devbuf 17817 34761K - 65048 16,32,64,128,256,512,1024,2048,4096,8192,32768 temp 80 18K - 94747689 16,32,64,128,256,512,1024,2048,4096,16384,32768,65536 module 523 66K - 524 128 mtx_pool 3 84K - 3 8192 osd 3 1K - 10 16,32,64,128,256 pmchooks 1 1K - 1 128 pgrp 36 5K - 844 128 session 27 4K - 769 128 proc 2 128K - 2 65536 subproc 4464 2512K - 11392290 512,4096 cred 41 11K - 313625 256 acpiintr 1 1K - 1 64 acpica 24513 2400K - 329438 16,32,64,128,256,512,1024,2048,4096 acpitask 1 64K - 1 65536 plimit 17 5K - 3246 256 uidinfo 6 17K - 155 128,16384 dumper 1 1K - 1 512 kbdmux 7 22K - 7 16,512,1024,2048,16384 sysctl 0 0K - 3 64 sysctloid 7407 377K - 7550 16,32,64,128 sysctltmp 0 0K - 5517 16,64,256,512,1024 LED 40 3K - 40 16,128 tidhash 1 128K - 1 callout 13 4232K - 13 umtx 138844 17356K - 138844 128 p1003.1b 1 1K - 1 16 SWAP 1 4372K - 1 bus 1661 181K - 10381633 16,32,64,128,256,1024 bus-sc 124 1911K - 17715 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536 devstat 26 53K - 26 32,4096 eventhandler 126 11K - 126 64,128 gtaskqueue 78 32K - 78 16,32,256,8192 kobj 356 1424K - 555763 4096 Per-cpu 1 1K - 1 32 rman 307 35K - 665 16,32,128 sbuf 0 0K - 6131902 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768 toponodes 32 4K - 32 128 md_disk 2 8K - 69 4096 taskqueue 30 4K - 30 16,32,128,256 terminal 11 3K - 11 256 Unitno 50 3K - 9960535 32,64 vmem 3 704K - 21 2048,4096,8192,16384,32768,65536 ioctlops 0 0K - 237 256,512,1024 select 104 13K - 104 128 iov 2 1K - 563414658 16,32,64,128,256,512 msg 4 30K - 4 2048,4096,8192,16384 sem 4 106K - 4 2048,4096 shm 1 32K - 1 32768 tty 140 140K - 1398871 1024 pts 126 32K - 1398857 256 mbuf_tag 0 0K - 270 32 shmfd 3 9K - 41 256,8192 soname 6 1K - 21915572 16,32,128 pcb 28 4247K - 338 16,32,128,1024,2048,8192 vfscache 4 8289K - 4 256,32768,65536 cl_savebuf 0 0K - 2065045 64 vfs_hash 1 4096K - 1 vnodes 1 1K - 1454 32,256 mount 70 4K - 10055994 16,32,64,128,256 statfs 0 0K - 61272764 4096 vnodemarker 0 0K - 6470402 1024 fadvise 0 0K - 8117 32 chacha20random 1 4K - 1 4096 BPF 11 18K - 11 128,512,4096 ifnet 4 7K - 4 128,2048 ifaddr 50 14K - 59 16,32,64,128,256,512,2048,4096 ether_multi 47 3K - 92 16,32,64 clone 8 1K - 8 128 ipsec 1 1K - 1 256 lltable 15 5K - 23 256,512 iflib 128 793K - 136 64,128,1024,8192,16384,32768 routetbl 26 4K - 45 32,64,128,512 vnet 1 1K - 1 64 vnet_data 1 64K - 1 65536 vnet_data_free 1 1K - 1 32 igmp 3 1K - 3 128 in_multi 2 1K - 11 256 encap_export_host 2 2K - 2 1024 newreno data 103 2K - 103 16 sctp_a_it 0 0K - 12 16 sctp_vrf 1 1K - 1 64 sctp_ifa 5 1K - 9 128 sctp_ifn 2 1K - 2 128 sctp_iter 0 0K - 12 256 tfo_ccache 1 128K - 1 hostcache 1 32K - 1 32768 LRO 24 480K - 24 8192,32768 tcpfunc 1 1K - 1 64 syncache 1 68K - 1 in6_mfilter 1 1K - 1 1024 in6_multi 27 4K - 27 32,256 ip6_moptions 2 1K - 2 32,256 mld 3 1K - 3 128 ip6ndp 5 1K - 6 64,256 inpcbpolicy 24 1K - 533 32 secasvar 1 1K - 1 1024 sahead 1 1K - 1 1024 ipsecpolicy 2 2K - 2 256,1024 ipsec-saq 2 2K - 2 1024 NLM 0 0K - 1 32 crypto 2 3K - 2 1024,2048 rpc 54 18K - 184 16,32,64,128,256,512,1024,4096 audit_evclass 230 8K - 286 32 pagedep 4 2048K - 4186773 256 inodedep 4 16384K - 38634747 512 bmsafemap 4 32K - 29962542 256,8192 newblk 4 32768K - 19915759 256 indirdep 0 0K - 165280 128,32768 freefrag 0 0K - 3882067 128 freeblks 0 0K - 4865814 256 freefile 0 0K - 29781568 64 diradd 0 0K - 106792776 128 mkdir 0 0K - 5982498 128 dirrem 0 0K - 107000257 128 newdirblk 0 0K - 3128010 64 freework 4 1K - 5175913 16,128 freedep 0 0K - 505 64 jaddref 0 0K - 13431577 128 jremref 0 0K - 13429832 128 jmvref 0 0K - 490106 128 jnewblk 0 0K - 2347364 128 jfreefrag 0 0K - 246655 128 jseg 0 0K - 1030868 128 jsegdep 0 0K - 29455428 64 sbdep 0 0K - 783364 64 savedino 0 0K - 14956056 256 jblocks 4 1K - 12 128,256 softdep 4 2K - 38611 512 ufs_dirhash 1999 494K - 404854 16,32,64,128,256,512,1024,2048,4096,8192 ufs_quota 11 4099K - 11 256 ufs_mount 14 91K - 166619 256,512,1024,2048,4096,8192,32768,65536 vm_pgdata 1 1K - 1 128 UMAHash 7 169K - 40 512,1024,2048,4096,8192,16384,32768,65536 memdesc 1 4K - 1 4096 pci_link 16 2K - 16 64,128 acpisem 31 4K - 31 128 acpidev 183 12K - 183 64 CAM SIM 13 4K - 13 256 apmdev 1 1K - 1992 128 madt_table 0 0K - 2 128,4096 CAM XPT 48 5K - 1143 16,32,64,128,256,512,1024,2048,65536 entropy 1 1K - 1048752 32,4096 CAM DEV 16 32K - 35 2048 CAM CCB 0 0K - 10189171 2048 CAM path 19 1K - 112 32 io_apic 2 4K - 2 2048 local_apic 1 16K - 1 16384 CAM periph 8 2K - 32 16,32,64,128,256 MCA 35 9K - 35 128,512 cpus 2 1K - 2 64 msi 24 3K - 24 128 nexusdev 5 1K - 5 16 acpi_perf 12 6K - 12 512 pmc 1142 11812K - 1577 16,32,128,256,512,1024,4096,8192,65536 nullfs_hash 1 4096K - 1 nullfs_node 0 0K - 28302313 64 nullfs_mount 0 0K - 213178 32 tmpfs mount 0 0K - 6 128 tmpfs name 0 0K - 5035 16,32,64 fdesc_mount 0 0K - 2 16 vmstat -N /usr/src/sys/amd64/compile/PHO/kernel.debug -M /var/crash/vmcore.73 -z ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP -- Test scenario_ misc/sendfile14.sh