panic: page fault, vmcore.121 KDB: debugger backends: ddb KDB: current backend: ddb ---<>--- Copyright (c) 1992-2019 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 13.0-CURRENT #0 r355146+3caf100893e-c245580(cpuctl): Thu Nov 28 19:08:17 CET 2019 pho@mercat1.netperf.freebsd.org:/var/tmp/deviant2/sys/amd64/compile/PHO amd64 FreeBSD clang version 9.0.0 (tags/RELEASE_900/final 372316) (based on LLVM 9.0.0) WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. Entering uma_startup with 10 boot pages configured startup_alloc from "UMA Kegs", 8 boot pages left startup_alloc from "UMA Zones", 7 boot pages left startup_alloc from "UMA Zones", 6 boot pages left startup_alloc from "UMA Zones", 5 boot pages left startup_alloc from "UMA Zones", 4 boot pages left startup_alloc from "UMA Zones", 3 boot pages left startup_alloc from "UMA Zones", 2 boot pages left Entering uma_startup1 with 1 boot pages left Entering uma_startup2 with 1 boot pages left VT(vga): resolution 640x480 CPU: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz (3500.07-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306f2 Family=0x6 Model=0x3f Stepping=2 Features=0xbfebfbff Features2=0x7ffefbff AMD Features=0x2c100800 AMD Features2=0x21 Structured Extended Features=0x37ab XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics real memory = 34359738368 (32768 MB) avail memory = 33164181504 (31627 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs FreeBSD/SMP: 1 package(s) x 6 core(s) x 2 hardware threads random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" random: unblocking device. ioapic0 irqs 0-23 ioapic1 irqs 24-47 Launching APs: 1 2 8 11 5 4 7 10 3 9 6 Timecounter "TSC-low" frequency 1750037358 Hz quality 1000 random: entropy device external interface WARNING: Device "kbd" is Giant locked and may be deleted before FreeBSD 13.0. kbd1 at kbdmux0 000.000055 [4336] netmap_init netmap: loaded module [ath_hal] loaded nexus0 vtvga0: cryptosoft0: acpi0: acpi0: Power Button (fixed) cpu0: on acpi0 atrtc0: port 0x70-0x71,0x74-0x77 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Event timer "HPET4" frequency 14318180 Hz quality 340 Event timer "HPET5" frequency 14318180 Hz quality 340 Event timer "HPET6" frequency 14318180 Hz quality 340 Event timer "HPET7" frequency 14318180 Hz quality 340 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: on acpi0 pci0: on pcib0 pci0: at device 11.1 (no driver attached) pci0: at device 11.2 (no driver attached) pci0: at device 16.1 (no driver attached) pci0: at device 16.6 (no driver attached) pci0: at device 18.1 (no driver attached) acpi_syscontainer0: on acpi0 acpi_syscontainer1: on acpi0 acpi_syscontainer2: on acpi0 acpi_syscontainer3: on acpi0 pcib1: port 0xcf8-0xcff on acpi0 pci1: on pcib1 pcib2: irq 26 at device 1.0 on pci1 pci2: on pcib2 pcib3: irq 32 at device 2.0 on pci1 pci3: on pcib3 pci3: at device 0.0 (no driver attached) pci3: at device 0.1 (no driver attached) pcib4: irq 40 at device 3.0 on pci1 pci4: on pcib4 pci1: at device 17.0 (no driver attached) ahci0: port 0xf110-0xf117,0xf100-0xf103,0xf0f0-0xf0f7,0xf0e0-0xf0e3,0xf020-0xf03f mem 0xfb438000-0xfb4387ff irq 16 at device 17.4 on pci1 ahci0: AHCI v1.30 with 4 6Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahciem0: on ahci0 xhci0: mem 0xfb400000-0xfb40ffff irq 19 at device 20.0 on pci1 xhci0: 32 bytes context size, 64-bit DMA usbus0 on xhci0 usbus0: 5.0Gbps Super Speed USB v3.0 pci1: at device 22.0 (no driver attached) pci1: at device 22.1 (no driver attached) ehci0: mem 0xfb434000-0xfb4343ff irq 18 at device 26.0 on pci1 usbus1: EHCI version 1.0 usbus1 on ehci0 usbus1: 480Mbps High Speed USB v2.0 pcib5: irq 16 at device 28.0 on pci1 pci5: on pcib5 pcib6: irq 18 at device 28.2 on pci1 pci6: on pcib6 pcib7: at device 0.0 on pci6 pci7: on pcib7 vgapci0: port 0xe000-0xe07f mem 0xfa000000-0xfaffffff,0xfb000000-0xfb01ffff irq 18 at device 0.0 on pci7 vgapci0: Boot video device pcib8: irq 16 at device 28.4 on pci1 pci8: on pcib8 igb0: port 0xd020-0xd03f mem 0xfb320000-0xfb33ffff,0xfb344000-0xfb347fff irq 16 at device 0.0 on pci8 igb0: Using 1024 TX descriptors and 1024 RX descriptors igb0: queue equality override not set, capping rx_queues at 6 and tx_queues at 6 igb0: Using 6 RX queues 6 TX queues igb0: Using MSI-X interrupts with 7 vectors igb0:Autoloading module: ioat.ko ioat0: mem 0xfb42c000-0xfb42ffff irq 31 at device 4.0 on pci1 ioat0: Capabilities: 2f7 ioat1: mem 0xfb428000-0xfb42bfff irq 39 at device 4.1 on pci1 ioat1: Capabilities: 2f7 ioat2: mem 0xfb424000-0xfb427fff irq 31 at device 4.2 on pci1 ioat2: Capabilities: f7 ioat3: mem 0xfb420000-0xfb423fff irq 39 at device 4.3 on pci1 ioat3: Capabilities: f7 ioat4: mem 0xfb41c000-0xfb41ffff irq 31 at device 4.4 on pci1 ioat4: Capabilities: f7 ioat5: mem 0xfb418000-0xfb41bfff irq 39 at device 4.5 on pci1 ioat5: Capabilities: f7 ioat6: mem 0xfb414000-0xfb417fff irq 31 at device 4.6 on pci1 ioat6: Capabilities: f7 ioat7: mem 0xfb410000-0xfb413fff irq 39 at device 4.7 on pci1 ioat7: Capabilities: f7 Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Starting Network: igb1. igb1: flags=8802 metric 0 mtu 1500 options=4e507bb ether 0c:c4:7a:a8:cd:db media: Ethernet autoselect status: no carrier nd6 options=29 Autoloading module: uhid.ko Autoloading module: ums.ko ums0 on uhub3 ums0: on usbus0 ums0: 3 buttons and [Z] coordinates ID=0 Starting dhclient. DHCPREQUEST on igb0 to 255.255.255.255 port 67 DHCPACK from 192if_delmulti_locked: detaching ifnet instance 0xfffff80004141800 .168.5.1 bound to 192.168.5.171 -- renewal in 300 seconds. add host 127.0.0.1: gateway lo0 fib 0: route already in table add host ::1: gateway lo0 fib 0: route already in table add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Creating and/or trimming log files. Starting syslogd. Setting date via ntp. 28 Nov 19:16:05 ntpdate[2050]: step time server 192.168.5.1 offset 0.008048 sec Starting rpcbind. NFS access cache time=60 Starting watchdogd. No core dumps found. Clearing /tmp (X related). Starting statd. Starting lockd. Recovering vi editor sessions:. fscklock order reversal: 1st 0xfffff80026b8b068 ufs (ufs) @ kern/vfs_mount.c:1603 2nd 0xfffff80026b03558 devfs (devfs) @ ufs/ffs/ffs_softdep.c:1927 stack backtrace: #0 0xffffffff80c36b11 at witness_debugger+0x71 #1 0xffffffff80c36886 at witness_checkorder+0xad6 #2 0xffffffff80b9c2f8 at lockmgr_lock_fast_path+0x178 #3 0xffffffff810eaac2 at VOP_LOCK1_APV+0x92 #4 0xffffffff80cb6ec5 at _vn_lock+0x65 #5 0xffffffff80ec7c9d at softdep_flushworklist+0x5d #6 0xffffffff80ee8c76 at ffs_sync+0x406 #7 0xffffffff80c9b18a at dounmount+0x46a #8 0xffffffff80c9acce at sys_unmount+0x2ae #9 0xffffffff8106d8c1 at amd64_syscall+0x2f1 #10 0xffffffff810436c0 at fast_syscall_common+0x101 ** /dev/gpt/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 254 files, 23033 used, 1242374 free (158 frags, 155277 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** Updating motd:. Mounting late filesystems:. Security policy loaded: MAC/ntpd (mac_ntpd) Starting ntpd. Starting powerd. Configuring vt: blanktime. Performing sanity check on sshd configuration. Starting sshd. Starting cron. Thu Nov 28 19:16 FreeBSD/amd64 (mercat1.netperf.freebsd.org) (ttyu1) login: root Nov 28 19:16:12 mercat1 login[2334]: ROOT LOGIN (root) ON ttyu1 Last login: Thu Nov 28 18:40:18 on ttyu1 FreeBSD 13.0-CURRENT (PHO) #0 r355146+3caf100893e-c245580(cpuctl): Thu Nov 28 19:08:17 CET 2019 root@mercat1:~ # Nov 28 19:16:26 mercat1 su[2349]: pho to root on /dev/pts/0 20191128 19:16:44 all (1/1): radix.sh 20191128 19:17:12 all.sh done, elapsed 0 day(s), 00:00.28 20191128 19:17:44 all (1/680): 1st.sh 20191128 19:17:46 all (2/680): 256m.sh 20191128 19:17:47 all (3/680): audit.sh 20191128 19:17:49 all (4/680): beneath2.sh 20191128 19:17:50 all (5/680): beneath3.sh 20191128 19:17:52 all (6/680): buildworld.sh 20191128 19:17:53 all (7/680): buildworld2.sh 20191128 19:17:55 all (8/680): cluster.sh 20191128 19:17:56 all (9/680): cpuset.sh 20191128 19:17:57 all (10/680): crossmp2.sh 20191128 19:17:59 all (11/680): crossmp6.sh 20191128 19:18:00 all (12/680): datagram2.sh 20191128 19:18:02 all (13/680): datagram3.sh 20191128 19:18:04 all (14/680): datamove.sh freework leak: 24/42. 20191128 19:18:05 all (15/680): datamove3.sh 20191128 19:18:07 all (16/680): devfs2.sh 20191128 19:18:08 all (17/680): devfs5.sh 20191128 19:18:09 all (18/680): dtrace_fault.sh 20191128 19:18:11 all (19/680): dumpfs.sh newblk leak: 294/312. 20191128 19:18:12 all (20/680): dup2.sh 20191128 19:18:14 all (21/680): execi386.sh 20191128 19:18:16 all (22/680): extattr_set_fd.sh 20191128 19:18:17 all (23/680): extattrctl.sh lock order reversal: 1st 0xfffff80288381838 ufs_extattr_sx (ufs_extattr_sx) @ ufs/ufs/ufs_extattr.c:119 2nd 0xfffff803029e12e0 ufs (ufs) @ ufs/ffs/ffs_vfsops.c:1718 stack backtrace: #0 0xffffffff80c36b11 at witness_debugger+0x71 #1 0xffffffff80c36886 at witness_checkorder+0xad6 #2 0xffffffff80b9c8df at lockmgr_xlock_hard+0x6f #3 0xffffffff80b9d46d at __lockmgr_args+0x60d #4 0xffffffff80ee5d91 at ffs_vgetf+0x161 #5 0xffffffff80ef8bcb at ufs_root+0x1b #6 0xffffffff80ca7c28 at vfs_cache_root_fallback+0x128 #7 0xffffffff80ca79a5 at vfs_cache_root+0x1b5 #8 0xffffffff80ef103f at ufs_extattr_autostart+0x5f #9 0xffffffff80ee815a at ffs_mount+0x1c6a #10 0xffffffff80c99e69 at vfs_domount+0x819 #11 0xffffffff80c98801 at vfs_donmount+0x911 #12 0xffffffff80c97eb9 at sys_nmount+0x69 #13 0xffffffff8106d8c1 at amd64_syscall+0x2f1 #14 0xffffffff810436c0 at fast_syscall_common+0x101 20191128 19:18:19 all (24/680): fcntl3.sh 20191128 19:18:20 all (25/680): fifo.sh 20191128 19:18:22 all (26/680): forkbomb.sh 20191128 19:18:23 all (27/680): fpclone2.sh 20191128 19:18:27 all (28/680): fsck2.sh 20191128 19:18:28 all (29/680): fsck3.sh 20191128 19:18:30 all (30/680): fsck4.sh 20191128 19:18:31 all (31/680): geomleak.sh 20191128 19:18:33 all (32/680): getrandom.sh 20191128 19:18:35 all (33/680): gnop3.sh 20191128 19:18:36 all (34/680): holdcnt0.sh 20191128 19:18:37 all (35/680): holdcnt02.sh 20191128 19:18:39 all (36/680): holdcnt03.sh 20191128 19:18:40 all (37/680): holdcnt04.sh 20191128 19:18:42 all (38/680): holdcnt05.sh 20191128 19:18:43 all (39/680): isofs2.sh 20191128 19:18:45 all (40/680): jail.sh 20191128 19:18:46 all (41/680): jail2.sh 20191128 19:18:47 all (42/680): jail3.sh 20191128 19:18:49 all (43/680): jail4.sh 20191128 19:18:50 all (44/680): kern_umtx_inf_loop.sh 20191128 19:18:52 all (45/680): kevent10.sh 20191128 19:18:53 all (46/680): kevent3.sh 20191128 19:18:55 all (47/680): ldt.sh 20191128 19:18:57 all (48/680): ldt2.sh 20191128 19:18:58 all (49/680): libMicro.sh 20191128 19:18:59 all (50/680): linux.sh 20191128 19:19:01 all (51/680): lockd.sh 20191128 19:19:02 all (52/680): lockf.sh 20191128 19:19:03 all (53/680): lookup_shared.sh 20191128 19:19:05 all (54/680): mac.sh 20191128 19:19:06 all (55/680): mac_chkexec.sh 20191128 19:19:08 all (56/680): machipc.sh 20191128 19:19:09 all (57/680): machipc2.sh 20191128 19:19:10 all (58/680): maxmemdom.sh 20191128 19:19:12 all (59/680): maxproc.sh 20191128 19:19:13 all (60/680): md4.sh 20191128 19:19:15 all (61/680): md5.sh 20191128 19:19:17 all (62/680): md6.sh 20191128 19:19:18 all (63/680): mdconfig.sh 20191128 19:19:19 all (64/680): mdconfig3.sh 20191128 19:19:21 all (65/680): memguard4.sh 20191128 19:19:22 all (66/680): mkfifo5.sh 20191128 19:19:24 all (67/680): mkfifo7.sh mount leak: 16/47 20191128 19:19:25 all (68/680): mlockall5.sh 20191128 19:19:27 all (69/680): mmap.sh 20191128 19:19:28 all (70/680): mmap12.sh 20191128 19:19:30 all (71/680): mmap16.sh 20191128 19:19:31 all (72/680): mmap17.sh 20191128 19:19:33 all (73/680): mmap20.sh 20191128 19:19:34 all (74/680): mmap24.sh 20191128 19:19:36 all (75/680): mmap30.sh 20191128 19:19:37 all (76/680): mmap33.sh 20191128 19:19:39 all (77/680): mmap35.sh 20191128 19:19:40 all (78/680): mmap36.sh 20191128 19:19:41 all (79/680): mount.sh 20191128 19:19:43 all (80/680): mprotect.sh 20191128 19:19:44 all (81/680): msdos2.sh 20191128 19:19:46 all (82/680): msdos3.sh 20191128 19:19:47 all (83/680): msdos7.sh lock order reversal: 1st 0xfffff801b533d7d0 msdosfs (msdosfs) @ kern/vfs_syscalls.c:3424 2nd 0xfffff802f86f97d0 devfs (devfs) @ fs/msdosfs/msdosfs_vnops.c:853 stack backtrace: #0 0xffffffff80c36b11 at witness_debugger+0x71 #1 0xffffffff80c36886 at witness_checkorder+0xad6 #2 0xffffffff80b9c2f8 at lockmgr_lock_fast_path+0x178 #3 0xffffffff810eaac2 at VOP_LOCK1_APV+0x92 #4 0xffffffff80cb6ec5 at _vn_lock+0x65 #5 0xffffffff80a89b49 at msdosfs_fsync+0x49 #6 0xffffffff810e981a at VOP_FSYNC_APV+0xaa #7 0xffffffff80cb29e9 at kern_fsync+0x199 #8 0xffffffff8106d8c1 at amd64_syscall+0x2f1 #9 0xffffffff810436c0 at fast_syscall_common+0x101 20191128 19:19:49 all (84/680): msdos8.sh 20191128 19:19:50 all (85/680): msetdomain.sh 20191128 19:19:52 all (86/680): nanosleep.sh 20191128 19:19:53 all (87/680): nbufkv.sh 20191128 19:19:55 all (88/680): nfs.sh 20191128 19:19:56 all (89/680): nfs12.sh 20191128 19:19:58 all (90/680): nfs14.sh 20191128 19:19:59 all (91/680): nfs15.sh 20191128 19:20:00 all (92/680): nfs15lockd.sh 20191128 19:20:02 all (93/680): nfs15lockd3.sh 20191128 19:20:03 all (94/680): nfs3.sh 20191128 19:20:05 all (95/680): nfs4.sh 20191128 19:20:06 all (96/680): nfs7.sh 20191128 19:20:07 all (97/680): nfs8.sh 20191128 19:20:09 all (98/680): nfs9.sh 20191128 19:20:10 all (99/680): nfs_halfpage.sh 20191128 19:20:12 all (100/680): nfs_halfpage2.sh 20191128 19:20:13 all (101/680): nfsdelegation.sh 20191128 19:20:14 all (102/680): nfsdepth.sh 20191128 19:20:16 all (103/680): nfsrename.sh 20191128 19:20:17 all (104/680): nfssillyrename.sh 20191128 19:20:19 all (105/680): nullfs13.sh 20191128 19:20:20 all (106/680): nullfs14.sh 20191128 19:20:22 all (107/680): nullfs15.sh 20191128 19:20:23 all (108/680): nullfs21.sh 20191128 19:20:25 all (109/680): nullfs23.sh 20191128 19:20:26 all (110/680): nullfs4.sh 20191128 19:20:27 all (111/680): nullfs6.sh 20191128 19:20:29 all (112/680): nullfs7.sh 20191128 19:20:30 all (113/680): nullfs9.sh 20191128 19:20:32 all (114/680): numa.sh 20191128 19:20:33 all (115/680): oom.sh 20191128 19:20:35 all (116/680): open.sh 20191128 19:20:36 all (117/680): pagefault.sh 20191128 19:20:38 all (118/680): parallelmount2.sh 20191128 19:20:39 all (119/680): pathconf.sh 20191128 19:20:40 all (120/680): pathconf2.sh 20191128 19:20:42 all (121/680): pcatch.sh 20191128 19:20:43 all (122/680): pcatch2.sh 20191128 19:20:45 all (123/680): ping.sh 20191128 19:20:46 all (124/680): pkru.sh 20191128 19:20:47 all (125/680): pkru2.sh 20191128 19:20:49 all (126/680): pmc4.sh 20191128 19:20:51 all (127/680): pmc5.sh 20191128 19:20:52 all (128/680): pmc8.sh 20191128 19:20:54 all (129/680): poll.sh 20191128 19:20:55 all (130/680): posix_fadvise.sh 20191128 19:20:56 all (131/680): posix_fadvise2.sh 20191128 19:20:58 all (132/680): procfs5.sh 20191128 19:20:59 all (133/680): procfs6.sh 20191128 19:21:01 all (134/680): procstat.sh 20191128 19:21:02 all (135/680): pthread9.sh 20191128 19:21:04 all (136/680): ptrace.sh 20191128 19:21:05 all (137/680): ptrace3.sh 20191128 19:21:07 all (138/680): pts2.sh 20191128 19:21:08 all (139/680): quota5.sh 20191128 19:21:10 all (140/680): rename13.sh 20191128 19:21:11 all (141/680): rename4.sh 20191128 19:21:12 all (142/680): revoke.sh 20191128 19:21:14 all (143/680): routetbl.sh 20191128 19:21:15 all (144/680): ruby.sh 20191128 19:21:17 all (145/680): rwlock_ronly.sh 20191128 19:21:18 all (146/680): schedfuzz.sh 20191128 19:21:20 all (147/680): seekdir.sh 20191128 19:21:21 all (148/680): segnp.sh 20191128 19:21:23 all (149/680): segregs.sh 20191128 19:21:24 all (150/680): select3.sh 20191128 19:21:26 all (151/680): sendfile4.sh 20191128 19:21:27 all (152/680): sendfile6.sh 20191128 19:21:29 all (153/680): setuid.sh 20191128 19:21:31 all (154/680): shm_super.sh 20191128 19:21:32 all (155/680): sigaltstack.sh 20191128 19:21:34 all (156/680): sigreturn.sh 20191128 19:21:35 all (157/680): sigstop.sh 20191128 19:21:37 all (158/680): snap.sh lock order reversal: 1st 0xfffffe00381cf9f8 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:283 2nd 0xfffff8002628b530 snaplk (snaplk) @ ufs/ffs/ffs_snapshot.c:2326 stack backtrace: #0 0xffffffff80c36b11 at witness_debugger+0x71 #1 0xffffffff80c36886 at witness_checkorder+0xad6 #2 0xffffffff80b9c8df at lockmgr_xlock_hard+0x6f #3 0xffffffff80b9d46d at __lockmgr_args+0x60d #4 0xffffffff80ec6d26 at ffs_copyonwrite+0x166 #5 0xffffffff80ee954c at ffs_geom_strategy+0x8c #6 0xffffffff80efcda3 at ufs_strategy+0x83 #7 0xffffffff810eaed2 at VOP_STRATEGY_APV+0xb2 #8 0xffffffff80c7e224 at bufstrategy+0x44 #9 0xffffffff80c7e0ba at bufwrite+0x24a #10 0xffffffff80eeb35d at ffs_syncvnode+0x2bd #11 0xffffffff80eea362 at ffs_fsync+0x22 #12 0xffffffff810e981a at VOP_FSYNC_APV+0xaa #13 0xffffffff80efa9cf at ufs_remove+0xdf #14 0xffffffff810e99ae at VOP_REMOVE_APV+0xce #15 0xffffffff80caf6a1 at kern_funlinkat+0x2c1 #16 0xffffffff80caf3d8 at sys_unlink+0x28 #17 0xffffffff8106d8c1 at amd64_syscall+0x2f1 lock order reversal: 1st 0xfffff8002628b530 snaplk (snaplk) @ kern/vfs_subr.c:3014 2nd 0xfffff80232e67a48 ufs (ufs) @ ufs/ffs/ffs_snapshot.c:1649 stack backtrace: #0 0xffffffff80c36b11 at witness_debugger+0x71 #1 0xffffffff80c36886 at witness_checkorder+0xad6 #2 0xffffffff80b9c8df at lockmgr_xlock_hard+0x6f #3 0xffffffff80b9d46d at __lockmgr_args+0x60d #4 0xffffffff80ec527d at ffs_snapremove+0xed #5 0xffffffff80ebf85e at ffs_truncate+0x26e #6 0xffffffff80ef2e0c at ufs_inactive+0x1cc #7 0xffffffff810ea6aa at VOP_INACTIVE_APV+0xaa #8 0xffffffff80ca3fc4 at vinactive+0xf4 #9 0xffffffff80ca454e at vputx+0x2ae #10 0xffffffff80caf6f9 at kern_funlinkat+0x319 #11 0xffffffff80caf3d8 at sys_unlink+0x28 #12 0xffffffff8106d8c1 at amd64_syscall+0x2f1 #13 0xffffffff810436c0 at fast_syscall_common+0x101 20191128 19:21:39 all (159/680): snap12.sh 20191128 19:21:40 all (160/680): snap2-1.sh 20191128 19:21:42 all (161/680): snap2.sh 20191128 19:21:43 all (162/680): sndstat.sh 20191128 19:21:44 all (163/680): socketpair.sh 20191128 19:21:46 all (164/680): socketpair2.sh 20191128 19:21:47 all (165/680): suj2.sh 20191128 19:21:49 all (166/680): suj32.sh 20191128 19:21:51 all (167/680): suj35.sh Nov 28 19:21:52 mercat1 kernel: pid 67220 (mkdir), uid 0 inumber 255 on /mnt: out of inodes 20191128 19:21:52 all (168/680): suj7.sh Failed to find journal. Use tunefs to create one Failed to start journal: 2 20191128 19:21:54 all (169/680): tar.sh 20191128 19:21:55 all (170/680): temp.sh 20191128 19:21:57 all (171/680): tmpfs15.sh 20191128 19:21:58 all (172/680): tmpfs4.sh 20191128 19:22:00 all (173/680): tmpfs7.sh 20191128 19:22:01 all (174/680): trim2.sh 20191128 19:22:03 all (175/680): trim5.sh 20191128 19:22:05 all (176/680): trim6.sh 20191128 19:22:06 all (177/680): trim7.sh 20191128 19:22:08 all (178/680): trim8.sh 20191128 19:22:09 all (179/680): truncate4.sh 20191128 19:22:11 all (180/680): truncate6.sh 20191128 19:22:12 all (181/680): tvnlru.sh 20191128 19:22:14 all (182/680): ufssuspend.sh 20191128 19:22:15 all (183/680): uma_zalloc_arg.sh 20191128 19:22:17 all (184/680): unlink.sh 20191128 19:22:18 all (185/680): unload.sh 20191128 19:22:20 all (186/680): unload2.sh 20191128 19:22:21 all (187/680): vfork.sh 20191128 19:22:23 all (188/680): vmio.sh 20191128 19:22:24 all (189/680): watchman.sh 20191128 19:22:26 all (190/680): write.sh 20191128 19:22:27 all (191/680): scp.sh 20191128 19:22:29 all (192/680): sendfile.sh 20191128 19:22:32 all (193/680): sendfile2.sh lock order reversal: 1st 0xfffff801b5e30490 filedesc structure (filedesc structure) @ kern/sys_generic.c:1516 2nd 0xfffff802aefa22e0 ufs (ufs) @ kern/vfs_vnops.c:1551 stack backtrace: #0 0xffffffff80c36b11 at witness_debugger+0x71 #1 0xffffffff80c36886 at witness_checkorder+0xad6 #2 0xffffffff80b9c8df at lockmgr_xlock_hard+0x6f #3 0xffffffff80b9d46d at __lockmgr_args+0x60d #4 0xffffffff80eea4bc at ffs_lock+0x7c #5 0xffffffff810eaac2 at VOP_LOCK1_APV+0x92 #6 0xffffffff80cb6ec5 at _vn_lock+0x65 #7 0xffffffff80cb597a at vn_poll+0x3a #8 0xffffffff80c3cff9 at kern_poll+0x419 #9 0xffffffff80c3cbd0 at sys_poll+0x50 #10 0xffffffff8106d8c1 at amd64_syscall+0x2f1 #11 0xffffffff810436c0 at fast_syscall_common+0x101 20191128 19:22:59 all (194/680): contigmalloc2.sh 20191128 19:23:01 all (195/680): contigmalloc3.sh 20191128 19:23:03 all (196/680): datagram.sh 20191128 19:23:06 all (197/680): ext2fs3.sh 20191128 19:23:08 all (198/680): extattr.sh 20191128 19:23:11 all (199/680): fcntl.sh 20191128 19:23:13 all (200/680): gnop5.sh GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. 20191128 19:23:15 all (201/680): isofs.sh 20191128 19:23:17 all (202/680): killpg.sh 20191128 19:23:20 all (203/680): md9.sh 20191128 19:23:22 all (204/680): mmap25.sh 20191128 19:23:24 all (205/680): mmap26.sh 20191128 19:23:27 all (206/680): mmap28.sh Nov 28 19:23:30 mercat1 kernel: Failed to fully fault in a core file segment at VA 0x800e00000 with size 0x71e5000 to be written at offset 0x86b000 for process mmap28 20191128 19:23:30 all (207/680): mmap8.sh 20191128 19:23:33 all (208/680): mmap9.sh 20191128 19:23:35 all (209/680): mountro2.sh 20191128 19:23:37 all (210/680): mountro3.sh newblk leak: 1743/2055. 20191128 19:23:39 all (211/680): msdos9.sh 20191128 19:23:42 all (212/680): newfs5.sh 20191128 19:23:45 all (213/680): nullfs10.sh 20191128 19:23:47 all (214/680): nullfs12.sh 20191128 19:23:49 all (215/680): nullfs16.sh 20191128 19:23:51 all (216/680): nullfs3.sh 20191128 19:23:54 all (217/680): openlock.sh 20191128 19:23:56 all (218/680): procfs2.sh 20191128 19:23:59 all (219/680): ptrace11.sh 20191128 19:24:02 all (220/680): ptrace8.sh 20191128 19:24:04 all (221/680): quota9.sh lock order reversal: 1st 0xfffff802b88cf288 FFS (FFS Lock) @ ufs/ufs/ufs_quota.c:759 2nd 0xfffff802f8778000 struct mount mtx (struct mount mtx) @ ufs/ufs/ufs_quota.c:765 stack backtrace: #0 0xffffffff80c36b11 at witness_debugger+0x71 #1 0xffffffff80c36886 at witness_checkorder+0xad6 #2 0xffffffff80ba8244 at __mtx_lock_flags+0x94 #3 0xffffffff80ef77ad at quotaoff_inchange+0xbd #4 0xffffffff80ee5bad at ffs_flushfiles+0x12d #5 0xffffffff80ec7f5d at softdep_flushfiles+0xad #6 0xffffffff80ee8367 at ffs_unmount+0xd7 #7 0xffffffff80c9b179 at dounmount+0x459 #8 0xffffffff80c9acce at sys_unmount+0x2ae #9 0xffffffff8106d8c1 at amd64_syscall+0x2f1 #10 0xffffffff810436c0 at fast_syscall_common+0x101 20191128 19:24:06 all (222/680): random.sh 20191128 19:24:08 all (223/680): rename6.sh 20191128 19:24:11 all (224/680): snap10.sh 20191128 19:24:13 all (225/680): spin.sh 20191128 19:24:16 all (226/680): suj8.sh Nov 28 19:24:17 mercat1 kernel: pid 93307 (dd), uid 0 inumber 4 on /mnt: filesystem full Nov 28 19:24:17 mercat1 kernel: pid 93323 (dd), uid 0 inumber 5 on /mnt: filesystem full 20191128 19:24:18 all (227/680): tmpfs6.sh 20191128 19:24:19 all (228/680): truncate5.sh 20191128 19:24:22 all (229/680): umtx_suspend.sh 20191128 19:24:25 all (230/680): vm_fault_dontneed.sh 20191128 19:24:27 all (231/680): datamove4.sh 20191128 19:24:30 all (232/680): extattr3.sh 20191128 19:24:34 all (233/680): kevent2.sh 20191128 19:24:37 all (234/680): kevent8.sh 20191128 19:24:40 all (235/680): mlockall7.sh 20191128 19:24:44 all (236/680): mmap23.sh lock order reversal: 1st 0xfffff80149344068 syncer (syncer) @ kern/vfs_subr.c:2349 2nd 0xfffff804ec986558 ufs (ufs) @ kern/vfs_subr.c:2763 stack backtrace: #0 0xffffffff80c36b11 at witness_debugger+0x71 #1 0xffffffff80c36886 at witness_checkorder+0xad6 #2 0xffffffff80b9c8df at lockmgr_xlock_hard+0x6f #3 0xffffffff80b9d46d at __lockmgr_args+0x60d #4 0xffffffff80eea4bc at ffs_lock+0x7c #5 0xffffffff810eaac2 at VOP_LOCK1_APV+0x92 #6 0xffffffff80cb6ec5 at _vn_lock+0x65 #7 0xffffffff80ca3aa9 at vget_finish+0x89 #8 0xffffffff80ca5e44 at vfs_msync+0x154 #9 0xffffffff80cab88e at sync_fsync+0x10e #10 0xffffffff810e981a at VOP_FSYNC_APV+0xaa #11 0xffffffff80ca9802 at sched_sync+0x3d2 #12 0xffffffff80b877d0 at fork_exit+0x80 #13 0xffffffff81043dde at fork_trampoline+0xe 20191128 19:24:48 all (237/680): mmap4.sh vnode_pager_putpages: I/O error 28 0xfffff806ff073000: tag ufs, type VREG usecount 0, writecount 0, refcount 3203 flags (VI_ACTIVE|VI_DOINGINACT) v_object 0xfffff805aa4d2738 ref 0 pages 25608 cleanbuf 3200 dirtybuf 1 lock type ufs: EXCL by thread 0xfffff8002658a6e0 (pid 3322, mmap4, tid 100146) #0 0xffffffff80b9cc7a at lockmgr_xlock_hard+0x40a #1 0xffffffff80b9d46d at __lockmgr_args+0x60d #2 0xffffffff80eea4bc at ffs_lock+0x7c #3 0xffffffff810eaac2 at VOP_LOCK1_APV+0x92 #4 0xffffffff80cb6ec5 at _vn_lock+0x65 #5 0xffffffff80ca4456 at vputx+0x1b6 #6 0xffffffff80f1382d at vm_map_process_deferred+0xad #7 0xffffffff80f1978b at vm_map_remove+0xeb #8 0xffffffff80f13155 at vmspace_exit+0xd5 #9 0xffffffff80b822c2 at exit1+0x5a2 #10 0xffffffff80b81d1d at sys_sys_exit+0xd #11 0xffffffff8106d8c1 at amd64_syscall+0x2f1 #12 0xffffffff810436c0 at fast_syscall_common+0x101 nlink=1, effnlink=1, size=104857600, extsize 0 generation=8ef72699, uid=0, gid=0, flags=0x0 ino 4, on dev md10a vnode_pager_putpages: residual I/O 32768 at 9760 0xfffff806ff073000: tag ufs, type VREG usecount 0, writecount 0, refcount 3203 flags (VI_ACTIVE|VI_DOINGINACT) v_object 0xfffff805aa4d2738 ref 0 pages 25608 cleanbuf 3200 dirtybuf 1 lock type ufs: EXCL by thread 0xfffff8002658a6e0 (pid 3322, mmap4, tid 100146) #0 0xffffffff80b9cc7a at lockmgr_xlock_hard+0x40a #1 0xffffffff80b9d46d at __lockmgr_args+0x60d #2 0xffffffff80eea4bc at ffs_lock+0x7c #3 0xffffffff810eaac2 at VOP_LOCK1_APV+0x92 #4 0xffffffff80cb6ec5 at _vn_lock+0x65 #5 0xffffffff80ca4456 at vputx+0x1b6 #6 0xffffffff80f1382d at vm_map_procesNov 28 19:24:49 s_deferred+0xad mercat1 kernel: #7 0xffffffff80f1978b at vm_map_remove+0xeb pid 3322 (mmap4)#8 0xffffffff80f13155 at vmspace_exit+0xd5 , uid 0 inumber #9 0xffffffff80b822c2 at exit1+0x5a2 4 on /mnt: files#10 0xffffffff80b81d1d at sys_sys_exit+0xd #11 0xffffffff8106d8c1 at amd64_syscall+0x2f1 ystem full #12 0xffffffff810436c0 at fast_syscall_common+0x101 nlink=1, effnlink=1, size=104857600, extsize 0 generation=8ef72699, uid=0, gid=0, flags=0x0 ino 4, on dev md10a vnode_pager_putpages: zero-length write at 40075264 resid 131072 0xfffff806ff073000: tag ufs, type VREG usecount 1, writecount 0, refcount 3203 flags (VI_ACTIVE) v_object 0xfffff805aa4d2738 ref 0 pages 25608 cleanbuf 3201 dirtybuf 0 lock type ufs: EXCL by thread 0xfffff801b5c39000 (pid 3330, umount, tid 100341) #0 0xffffffff80b9cc7a at lockmgr_xlock_hard+0x40a #1 0xffffffff80b9d46d at __lockmgr_args+0x60d #2 0xffffffff80eea4bc at ffs_lock+0x7c #3 0xffffffff810eaac2 at VOP_LOCK1_APV+0x92 #4 0xffffffff80cb6ec5 at _vn_lock+0x65 #5 0xffffffff80ca3aa9 at vget_finish+0x89 #6 0xffffffff80ca5e44 at vfs_msync+0x154 #7 0xffffffff80c9b0f9 at dounmount+0x3d9 Nov 28 19:24:50 #8 0xffffffff80c9acce at sys_unmount+0x2ae mercat1 kernel: #9 0xffffffff8106d8c1 at amd64_syscall+0x2f1 pid 3330 (umount#10 0xffffffff810436c0 at fast_syscall_common+0x101 nlink=1, effnlink=1, size=104857600, extsize 0 generation=8ef72699, uid=0, gid=0, flags=0x0 ino 4, on dev md10a ), uid 0 inumber 4 on /mnt: filesystem full 20191128 19:24:51 all (238/680): pthread.sh 20191128 19:24:54 all (239/680): tmpfs5.sh 20191128 19:24:58 all (240/680): udp.sh 20191128 19:25:01 all (241/680): unix_socket.sh 20191128 19:25:05 all (242/680): devfd.sh 20191128 19:25:09 all (243/680): dup.sh 20191128 19:25:13 all (244/680): linger3.sh 20191128 19:25:18 all (245/680): mkfifo6.sh 20191128 19:25:23 all (246/680): mmap13.sh 20191128 19:25:27 all (247/680): mountro4.sh 20191128 19:25:31 all (248/680): procfs3.sh 20191128 19:25:36 all (249/680): ptrace2.sh 20191128 19:25:41 all (250/680): rename11.sh 20191128 19:25:45 all (251/680): truncate7.sh 20191128 19:25:49 all (252/680): symlink4.sh 20191128 19:25:54 all (253/680): datamove5.sh 20191128 19:25:59 all (254/680): growfs.sh g_access(997): provider ufsid/5de011397daa4a47 has error 6 set g_access(997): provider ufsid/5de011397daa4a47 has error 6 set g_access(997): provider ufsid/5de011397daa4a47 has error 6 set g_access(997): provider gptid/877fba83-120c-11ea-a39a-0cc47aa8cdda has error 6 set g_access(997): provider gptid/877fba83-120c-11ea-a39a-0cc47aa8cdda has error 6 set g_access(997): provider gptid/877fba83-120c-11ea-a39a-0cc47aa8cdda has error 6 set g_dev_taste: make_dev_p() failed (gp->name=gptid/877fba83-120c-11ea-a39a-0cc47aa8cdda, error=17) g_dev_taste: make_dev_p() failed (gp->name=gptid/877fba83-120c-11ea-a39a-0cc47aa8cdda, error=17) g_dev_taste: make_dev_p() failed (gp->name=ufsid/5de011397daa4a47, error=17) g_dev_taste: make_dev_p() failed (gp->name=gptid/877fba83-120c-11ea-a39a-0cc47aa8cdda, error=17) g_dev_taste: make_dev_p() failed (gp->name=ufsid/5de011397daa4a47, error=17) g_dev_taste: make_dev_p() failed (gp->name=gptid/877fba83-120c-11ea-a39a-0cc47aa8cdda, error=17) g_dev_taste: make_dev_p() failed (gp->name=ufsid/5de011397daa4a47, error=17) GEOM leak: 14/87 20191128 19:26:05 all (255/680): procfs.sh 20191128 19:26:10 all (256/680): pshared.sh 20191128 19:26:15 all (257/680): tmpfs10.sh 20191128 19:26:21 all (258/680): truncate3.sh 20191128 19:26:26 all (259/680): linger2.sh Nov 28 19:26:29 mercat1 kernel: pid 46257 (linger2), uid 2007 inumber 888 on /mnt: filesystem full 20191128 19:26:33 all (260/680): linger4.sh Nov 28 19:26:37 mercat1 kernel: pid 47556 (linger4), uid 2007 inumber 47391 on /mnt: out of inodes 20191128 19:26:39 all (261/680): mlockall4.sh 20191128 19:26:46 all (262/680): ptrace9.sh 20191128 19:26:52 all (263/680): sparse.sh 20191128 19:26:59 all (264/680): fork.sh 20191128 19:27:06 all (265/680): symlink2.sh 20191128 19:27:12 all (266/680): sysctl.sh 20191128 19:27:20 all (267/680): contigmalloc.sh 20191128 19:29:21 all (268/680): quota11.sh 20191128 19:29:29 all (269/680): execve.sh Fatal trap 12: page fault while in kernel mode cpuid = 8; apic id = 08 fault virtual address = 0x0 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff80f01279 stack pointer = 0x28:0xfffffe02f94a48a0 frame pointer = 0x28:0xfffffe02f94a48d0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 78999 (cron) trap number = 12 panic: page fault cpuid = 8 time = 1574965860 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe02f94a4500 vpanic() at vpanic+0x17e/frame 0xfffffe02f94a4560 panic() at panic+0x43/frame 0xfffffe02f94a45c0 trap_fatal() at trap_fatal+0x386/frame 0xfffffe02f94a4620 trap_pfault() at trap_pfault+0x99/frame 0xfffffe02f94a46a0 trap() at trap+0x2c0/frame 0xfffffe02f94a47d0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a47d0 --- trap 0xc, rip = 0xffffffff80f01279, rsp = 0xfffffe02f94a48a0, rbp = 0xfffffe02f94a48d0 --- swap_pager_copy() at swap_pager_copy+0xf9/frame 0xfffffe02f94a48d0 vm_object_collapse() at vm_object_collapse+0x181/frame 0xfffffe02f94a4960 vm_object_deallocate() at vm_object_deallocate+0x41d/frame 0xfffffe02f94a49b0 vm_map_process_deferred() at vm_map_process_deferred+0xad/frame 0xfffffe02f94a49d0 vm_map_remove() at vm_map_remove+0xeb/frame 0xfffffe02f94a4a00 vmspace_exit() at vmspace_exit+0xd5/frame 0xfffffe02f94a4a40 exit1() at exit1+0x5a2/frame 0xfffffe02f94a4ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe02f94a4ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f94a4bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f94a4bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8003d880a, rsp = 0x7fffffffeb08, rbp = 0x7fffffffeb30 --- KDB: enter: panic [ thread pid 78999 tid 100164 ] Stopped at kdb_enter+0x37: movq $0,0x1081ba6(%rip) db> run pho db:0:pho> set $lines 20000 db:0:pho> run pho1 db:1:pho1> bt Tracing pid 78999 tid 100164 td 0xfffff806ba27c000 kdb_enter() at kdb_enter+0x37/frame 0xfffffe02f94a4500 vpanic() at vpanic+0x19a/frame 0xfffffe02f94a4560 panic() at panic+0x43/frame 0xfffffe02f94a45c0 trap_fatal() at trap_fatal+0x386/frame 0xfffffe02f94a4620 trap_pfault() at trap_pfault+0x99/frame 0xfffffe02f94a46a0 trap() at trap+0x2c0/frame 0xfffffe02f94a47d0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a47d0 --- trap 0xc, rip = 0xffffffff80f01279, rsp = 0xfffffe02f94a48a0, rbp = 0xfffffe02f94a48d0 --- swap_pager_copy() at swap_pager_copy+0xf9/frame 0xfffffe02f94a48d0 vm_object_collapse() at vm_object_collapse+0x181/frame 0xfffffe02f94a4960 vm_object_deallocate() at vm_object_deallocate+0x41d/frame 0xfffffe02f94a49b0 vm_map_process_deferred() at vm_map_process_deferred+0xad/frame 0xfffffe02f94a49d0 vm_map_remove() at vm_map_remove+0xeb/frame 0xfffffe02f94a4a00 vmspace_exit() at vmspace_exit+0xd5/frame 0xfffffe02f94a4a40 exit1() at exit1+0x5a2/frame 0xfffffe02f94a4ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe02f94a4ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f94a4bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f94a4bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8003d880a, rsp = 0x7fffffffeb08, rbp = 0x7fffffffeb30 --- db:1:pho1> show allpcpu Current CPU: 8 cpuid = 0 dynamic pcpu = 0x972e00 curthread = 0xfffff806baa7e000: pid 79083 tid 101212 "execve" curpcb = 0xfffff806baa7e4f0 fpcurthread = 0xfffff806baa7e000: pid 79083 "execve" idlethread = 0xfffff800037d8000: tid 100003 "idle: cpu0" self = 0xffffffff82410000 curpmap = 0xfffff804e62ec130 tssp = 0xffffffff82410384 rsp0 = 0xfffffe02f93a3cc0 kcr3 = 0x80000005065121fd ucr3 = 0x80000001e4d939fd scr3 = 0x1e4d939fd gs32p = 0xffffffff82410404 ldt = 0xffffffff82410444 tss = 0xffffffff82410434 tlb gen = 2927498 curvnet = 0 spin locks held: cpuid = 1 dynamic pcpu = 0xfffffe00bc7ece00 curthread = 0xfffff806ba6856e0: pid 79080 tid 100179 "execve" curpcb = 0xfffff806ba685bd0 fpcurthread = 0xfffff806ba6856e0: pid 79080 "execve" idlethread = 0xfffff800037d76e0: tid 100004 "idle: cpu1" self = 0xffffffff82411000 curpmap = 0xfffff801ef23e130 tssp = 0xffffffff82411384 rsp0 = 0xfffffe02f92bdcc0 kcr3 = 0x800000050605404c ucr3 = 0x80000005063d584c scr3 = 0x5063d584c gs32p = 0xffffffff82411404 ldt = 0xffffffff82411444 tss = 0xffffffff82411434 tlb gen = 2927498 curvnet = 0 spin locks held: cpuid = 2 dynamic pcpu = 0xfffffe00bc7f8e00 curthread = 0xfffff806f4784000: pid 79078 tid 101055 "execve" curpcb = 0xfffff806f47844f0 fpcurthread = 0xfffff806f4784000: pid 79078 "execve" idlethread = 0xfffff800037d7000: tid 100005 "idle: cpu2" self = 0xffffffff82412000 curpmap = 0xfffff806ba6a2130 tssp = 0xffffffff82412384 rsp0 = 0xfffffe00e16aacc0 kcr3 = 0x80000004ec3ae472 ucr3 = 0x80000002a9183c72 scr3 = 0x2a9183c72 gs32p = 0xffffffff82412404 ldt = 0xffffffff82412444 tss = 0xffffffff82412434 tlb gen = 2927498 curvnet = 0 spin locks held: cpuid = 3 dynamic pcpu = 0xfffffe00bc804e00 curthread = 0xfffff806b3d69000: pid 79088 tid 101050 "execve" curpcb = 0xfffff806b3d694f0 fpcurthread = 0xfffff806b3d69000: pid 79088 "execve" idlethread = 0xfffff800037eb6e0: tid 100006 "idle: cpu3" self = 0xffffffff82413000 curpmap = 0xfffff806ace6e130 tssp = 0xffffffff82413384 rsp0 = 0xfffffe00e35f2cc0 kcr3 = 0x800000025c63d00e ucr3 = 0x80000002f3edf80e scr3 = 0x2f3edf80e gs32p = 0xffffffff82413404 ldt = 0xffffffff82413444 tss = 0xffffffff82413434 tlb gen = 2927497 curvnet = 0 spin locks held: cpuid = 4 dynamic pcpu = 0xfffffe00bc810e00 curthread = 0xfffff806b3d6c000: pid 79076 tid 100566 "execve" curpcb = 0xfffff806b3d6c4f0 fpcurthread = 0xfffff806b3d6c000: pid 79076 "execve" idlethread = 0xfffff800037eb000: tid 100007 "idle: cpu4" self = 0xffffffff82414000 curpmap = 0xfffff801efd84130 tssp = 0xffffffff82414384 rsp0 = 0xfffffe02f933fcc0 kcr3 = 0x8000000369203694 ucr3 = 0x8000000373190e94 scr3 = 0x373190e94 gs32p = 0xffffffff82414404 ldt = 0xffffffff82414444 tss = 0xffffffff82414434 tlb gen = 2927498 curvnet = 0 spin locks held: cpuid = 5 dynamic pcpu = 0xfffffe00bc81ce00 curthread = 0xfffff80506061000: pid 79077 tid 101035 "execve" curpcb = 0xfffff805060614f0 fpcurthread = 0xfffff80506061000: pid 79077 "execve" idlethread = 0xfffff800037e66e0: tid 100008 "idle: cpu5" self = 0xffffffff82415000 curpmap = 0xfffff80755663130 tssp = 0xffffffff82415384 rsp0 = 0xfffffe00e3303cc0 kcr3 = 0x800000026958042a ucr3 = 0x800000065d0e4c2a scr3 = 0x65d0e4c2a gs32p = 0xffffffff82415404 ldt = 0xffffffff82415444 tss = 0xffffffff82415434 tlb gen = 2927498 curvnet = 0 spin locks held: cpuid = 6 dynamic pcpu = 0xfffffe00bc828e00 curthread = 0xfffff806ffd9a000: pid 79072 tid 101314 "execve" curpcb = 0xfffff806ffd9a4f0 fpcurthread = 0xfffff806ffd9a000: pid 79072 "execve" idlethread = 0xfffff800037e6000: tid 100009 "idle: cpu6" self = 0xffffffff82416000 curpmap = 0xfffff806bade0130 tssp = 0xffffffff82416384 rsp0 = 0xfffffe00e03fecc0 kcr3 = 0x80000003794540df ucr3 = 0x80000006f4b848df scr3 = 0x6f4b848df gs32p = 0xffffffff82416404 ldt = 0xffffffff82416444 tss = 0xffffffff82416434 tlb gen = 2927498 curvnet = 0 spin locks held: cpuid = 7 dynamic pcpu = 0xfffffe00bc834e00 curthread = 0xfffff8065d88f6e0: pid 79086 tid 100397 "execve" curpcb = 0xfffff8065d88fbd0 fpcurthread = 0xfffff8065d88f6e0: pid 79086 "execve" idlethread = 0xfffff800037d96e0: tid 100010 "idle: cpu7" self = 0xffffffff82417000 curpmap = 0xfffff806ffb89130 tssp = 0xffffffff82417384 rsp0 = 0xfffffe00e324acc0 kcr3 = 0x800000025de25584 ucr3 = 0x800000050644ad84 scr3 = 0x50644ad84 gs32p = 0xffffffff82417404 ldt = 0xffffffff82417444 tss = 0xffffffff82417434 tlb gen = 2927498 curvnet = 0 spin locks held: cpuid = 8 dynamic pcpu = 0xfffffe00bc840e00 curthread = 0xfffff806ba27c000: pid 78999 tid 100164 "cron" curpcb = 0xfffff806ba27c4f0 fpcurthread = 0xfffff806ba27c000: pid 78999 "cron" idlethread = 0xfffff800037ee000: tid 100011 "idle: cpu8" self = 0xffffffff82418000 curpmap = 0xffffffff81d9c420 tssp = 0xffffffff82418384 rsp0 = 0xfffffe02f94a4cc0 kcr3 = 0x8000000002301005 ucr3 = 0xffffffffffffffff scr3 = 0x6ac259a02 gs32p = 0xffffffff82418404 ldt = 0xffffffff82418444 tss = 0xffffffff82418434 tlb gen = 2927497 curvnet = 0 spin locks held: cpuid = 9 dynamic pcpu = 0xfffffe00bc84ce00 curthread = 0xfffff804ec3106e0: pid 79094 tid 100914 "execve" curpcb = 0xfffff804ec310bd0 fpcurthread = 0xfffff804ec3106e0: pid 79094 "execve" idlethread = 0xfffff800037ed6e0: tid 100012 "idle: cpu9" self = 0xffffffff82419000 curpmap = 0xfffff804e9ba5130 tssp = 0xffffffff82419384 rsp0 = 0xfffffe00e3213cc0 kcr3 = 0x8000000226008611 ucr3 = 0x800000022600ce11 scr3 = 0x22600ce11 gs32p = 0xffffffff82419404 ldt = 0xffffffff82419444 tss = 0xffffffff82419434 tlb gen = 2927498 curvnet = 0 spin locks held: cpuid = 10 dynamic pcpu = 0xfffffe00bc858e00 curthread = 0xfffff806ffccd000: pid 79096 tid 102484 "execve" curpcb = 0xfffff806ffccd4f0 fpcurthread = 0xfffff806ffccd000: pid 79096 "execve" idlethread = 0xfffff800037ed000: tid 100013 "idle: cpu10" self = 0xffffffff8241a000 curpmap = 0xfffff806ba55b130 tssp = 0xffffffff8241a384 rsp0 = 0xfffffe02f9349cc0 kcr3 = 0x80000001abdbe611 ucr3 = 0x80000001e4ae7e11 scr3 = 0x1e4ae7e11 gs32p = 0xffffffff8241a404 ldt = 0xffffffff8241a444 tss = 0xffffffff8241a434 tlb gen = 2927497 curvnet = 0 spin locks held: cpuid = 11 dynamic pcpu = 0xfffffe00bc864e00 curthread = 0xfffff800261bd000: pid 30 tid 100128 "dom0" curpcb = 0xfffff800261bd4f0 fpcurthread = none idlethread = 0xfffff800037ec6e0: tid 100014 "idle: cpu11" self = 0xffffffff8241b000 curpmap = 0xffffffff81d9c420 tssp = 0xffffffff8241b384 rsp0 = 0xfffffe00d3218cc0 kcr3 = 0x800000000230100b ucr3 = 0xffffffffffffffff scr3 = 0x679be9c13 gs32p = 0xffffffff8241b404 ldt = 0xffffffff8241b444 tss = 0xffffffff8241b434 tlb gen = 2927497 curvnet = 0 spin locks held: db:1:pho1> show alllocks Process 78999 (cron) thread 0xfffff806ba27c000 (100164) exclusive sx swspsx (swspsx) r = 0 (0xffffffff81d6bfe0) locked @ vm/swap_pager.c:1003 Process 2043 (syslogd) thread 0xfffff80026b83000 (100163) exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe003b02f0b8) locked @ kern/vfs_bio.c:1666 exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe0038a2b640) locked @ kern/vfs_bio.c:3873 exclusive lockmgr ufs (ufs) r = 0 (0xfffff8003c4f7a48) locked @ kern/vfs_vnops.c:874 Process 30 (pagedaemon) thread 0xfffff800261bd000 (100128) exclusive sleep mutex pmap (pmap) r = 0 (0xfffff804ecfc1130) locked @ amd64/amd64/pmap.c:7763 exclusive rw pmap pv list (pmap pv list) r = 0 (0xfffffe0037656ec0) locked @ amd64/amd64/pmap.c:7648 exclusive sleep mutex vm page (vm page) r = 0 (0xffffffff81846d80) locked @ vm/vm_page.c:1164 db:1:pho1> show lockedvnods Locked vnodes vnode 0xfffff8003c4f79e0: tag ufs, type VREG usecount 1, writecount 1, refcount 5 flags (VI_ACTIVE) v_object 0xfffff80026eaf840 ref 0 pages 18 cleanbuf 2 dirtybuf 1 lock type ufs: EXCL by thread 0xfffff80026b83000 (pid 2043, syslogd, tid 100163) #0 0xffffffff80b9cc7a at lockmgr_xlock_hard+0x40a #1 0xffffffff80b9d46d at __lockmgr_args+0x60d #2 0xffffffff80eea4bc at ffs_lock+0x7c #3 0xffffffff810eaac2 at VOP_LOCK1_APV+0x92 #4 0xffffffff80cb6ec5 at _vn_lock+0x65 #5 0xffffffff80cbb165 at vn_write+0x1e5 #6 0xffffffff80cbabd3 at vn_io_fault_doio+0x43 #7 0xffffffff80cb767c at vn_io_fault1+0x16c #8 0xffffffff80cb5546 at vn_io_fault+0x186 #9 0xffffffff80c3b391 at dofilewrite+0x81 #10 0xffffffff80c3afc9 at kern_writev+0x49 #11 0xffffffff80c3b2e5 at sys_writev+0x35 #12 0xffffffff8106d8c1 at amd64_syscall+0x2f1 #13 0xffffffff810436c0 at fast_syscall_common+0x101 nlink=1, effnlink=1, size=237504, extsize 0 generation=364dad8c, uid=0, gid=0, flags=0x0 ino 9951777, on dev gpt/work db:1:pho1> show mount 0xfffff800263e2000 /dev/gpt/work on / (ufs) 0xfffff800263e1000 devfs on /dev (devfs) 0xfffff80149478000 /dev/gpt/tmp on /tmp (ufs) 0xfffff802ad683000 procfs on /proc (procfs) More info: show mount db:1:pho1> show bufqueues bqempty: 97255 Buf domain 0 freebufs 14195 lofreebufs 219 hifreebufs 329 bufspace 8261632 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 896 (7991296) dirtyq count 0 (0) wakeup 0 lim 25 CPU 0, 0, 0, 0, 1, 11, 0, 7, 22, 1, 0, 0, Locked buffers: 0 space 0 Total buffers: 6547 space 8261632 Buf domain 1 freebufs 12221 lofreebufs 219 hifreebufs 329 bufspace 59486208 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 22 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 2857 (57708544) dirtyq count 22 (696320) wakeup 0 lim 25 CPU 12, 0, 0, 4, 0, 0, 0, 4, 2, 10, 0, 0, Locked buffers: 2 space 40960 Total buffers: 8527 space 59486208 Buf domain 2 freebufs 14646 lofreebufs 219 hifreebufs 329 bufspace 2150400 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 2 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 424 (1736704) dirtyq count 2 (8192) wakeup 0 lim 25 CPU 8, 0, 3, 3, 0, 16, 1, 6, 21, 2, 0, 1, Locked buffers: 0 space 0 Total buffers: 6202 space 2150400 Buf domain 3 freebufs 14230 lofreebufs 219 hifreebufs 329 bufspace 7020544 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 854 (6725632) dirtyq count 0 (0) wakeup 0 lim 25 CPU 1, 1, 1, 0, 0, 15, 0, 8, 22, 0, 0, 1, Locked buffers: 0 space 0 Total buffers: 6598 space 7020544 Buf domain 4 freebufs 14112 lofreebufs 219 hifreebufs 329 bufspace 9359360 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 995 (9109504) dirtyq count 0 (0) wakeup 0 lim 25 CPU 0, 0, 0, 0, 2, 3, 0, 0, 19, 1, 0, 1, Locked buffers: 0 space 0 Total buffers: 6413 space 9359360 Buf domain 5 freebufs 13127 lofreebufs 219 hifreebufs 329 bufspace 30330880 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 1988 (29904896) dirtyq count 0 (0) wakeup 0 lim 25 CPU 0, 0, 2, 1, 4, 0, 1, 1, 0, 8, 0, 0, Locked buffers: 1 space 32768 Total buffers: 7959 space 30330880 Buf domain 6 freebufs 15112 lofreebufs 219 hifreebufs 329 bufspace 249856 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 2 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 2 (8192) dirtyq count 2 (36864) wakeup 0 lim 25 CPU 1, 0, 0, 2, 3, 3, 0, 0, 1, 7, 0, 0, Locked buffers: 0 space 0 Total buffers: 13342 space 249856 db:1:pho1> show page vm_cnt.v_free_count: 160755 vm_cnt.v_inactive_count: 1061 vm_cnt.v_active_count: 7335429 vm_cnt.v_laundry_count: 396860 vm_cnt.v_wire_count: 229205 vm_cnt.v_free_reserved: 10644 vm_cnt.v_free_min: 51264 vm_cnt.v_free_target: 173124 vm_cnt.v_inactive_target: 259686 db:1:pho1> show pageq pq_free 160755 dom 0 page_cnt 8124092 free 160755 pq_act 7335429 pq_inact 1061 pq_laund 396860 pq_unsw 0 db:0:pho> show freepages DOMAIN: 0 FREE LIST 0: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000000 | 000000 10 (004096K) | 000000 | 000000 09 (002048K) | 000000 | 000000 08 (001024K) | 000000 | 000000 07 (000512K) | 000000 | 000000 06 (000256K) | 000000 | 000000 05 (000128K) | 000000 | 000000 04 (000064K) | 000000 | 000000 03 (000032K) | 000000 | 000000 02 (000016K) | 000000 | 000000 01 (000008K) | 000000 | 000000 00 (000004K) | 000000 | 001316 FREE LIST 1: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000000 | 000000 10 (004096K) | 000000 | 000000 09 (002048K) | 000000 | 000000 08 (001024K) | 000000 | 000000 07 (000512K) | 000000 | 000000 06 (000256K) | 000000 | 000000 05 (000128K) | 000000 | 000000 04 (000064K) | 000000 | 000000 03 (000032K) | 000000 | 000000 02 (000016K) | 000000 | 000000 01 (000008K) | 000000 | 000000 00 (000004K) | 000000 | 000000 db:0:pho> show uma Zone Size Used Free Requests Sleeps Bucket Total Mem XFree RADIX NODE 144 528022 8948 34170151 0 249 77323680 0 VM OBJECT 264 12196 221249 2780920 0 54 61629480 0 VNODE 632 39956 11464 1284104 0 156 32497440 0 mbuf_cluster 2048 13123 179 16890 0 5 27242496 0 BUF TRIE 144 663 105285 455848 0 61 15256512 0 256 Bucket 2048 920 5601 212276 0 245 13355008 0 FFS2 dinode 256 39173 9562 518177 0 69 12476160 0 FFS inode 160 39173 11899 518190 0 70 8171520 0 128 Bucket 1024 429 5355 134313 0 61 5922816 0 S VFS Cache 108 39860 7845 1096607 0 125 5152140 0 swblk 136 10165 24722 44864 0 61 4744632 0 mbuf 256 14736 3688 68088 0 253 4716544 0 128 128 4683 26627 722952 0 125 4007680 0 vmem btag 56 19189 51869 104227 0 253 3979248 0 128 128 14455 12670 1388971 0 125 3472000 0 THREAD 1760 1371 511 258736 0 19 3312320 0 VMSPACE 2552 97 1117 178864 0 3 3098128 0 64 Bucket 512 330 5249 68636 0 46 2856448 0 socket 904 51 2733 6214 0 13 2516736 0 UMA Slabs 112 13888 4512 28234 0 0 2060800 0 256 256 943 6692 1375897 0 61 1954560 0 MAP ENTRY 112 3039 13831 9638009 0 146 1889440 0 128 128 2553 11056 209530 0 125 1741952 0 32 Bucket 256 212 6387 162949 0 61 1689344 0 PROC 1328 131 1075 179036 0 9 1601568 0 4096 4096 348 10 7415 0 1 1466368 0 128 128 140 11237 628383 0 125 1456256 0 65536 65536 4 13 3534 0 1 1114112 0 256 256 3 4242 187894 0 61 1086720 0 256 256 81 3834 175085 0 61 1002240 0 DIRHASH 1024 387 565 11257 0 13 974848 0 g_bio 408 8 2323 1788170 0 62 951048 0 256 256 18 3537 90658 0 61 910080 0 16384 16384 50 4 2886 0 1 884736 0 32768 32768 10 14 9408 0 1 786432 0 512 512 1249 277 7313 0 29 781312 0 64 64 9480 2548 18022 0 253 769792 0 32768 32768 12 11 287 0 1 753664 0 64 64 65 11653 249095 0 253 749952 0 2048 2048 3 363 126374 0 36 749568 0 64 64 8448 3084 2386211 0 253 738048 0 4096 4096 138 40 362403 0 1 729088 0 512 512 131 1283 1338868 0 44 723968 0 8192 8192 87 0 87 0 1 712704 0 65536 65536 1 9 92 0 1 655360 0 65536 65536 2 8 20 0 1 655360 0 mbuf_packet 256 9 2522 11688 0 253 647936 0 128 128 51 5002 479956 0 125 646784 0 16 Bucket 128 149 4655 31799 0 125 614912 0 pbuf 984 0 531 0 0 0 522504 0 filedesc0 1104 130 305 179037 0 9 480240 0 16384 16384 7 22 41708 0 1 475136 0 Files 80 127 5753 3511728 0 130 470400 0 256 256 35 1795 138093 0 61 468480 0 swpctrie 144 1292 1732 4268 0 61 435456 0 pipe 760 9 561 439876 0 13 433200 0 32768 32768 12 0 12 0 1 393216 0 Mountpoints 3008 4 116 2151 0 3 360960 0 1024 1024 170 166 677 0 13 344064 0 64 64 1272 3997 185346 0 253 337216 0 32768 32768 3 7 2342 0 1 327680 0 12 Bucket 96 63 3299 45003 0 125 322752 0 ksiginfo 112 1181 1689 107546 0 125 321440 0 TURNSTILE 136 1883 297 1883 0 61 296480 0 512 512 112 441 1041844 0 29 283136 0 KNOTE 160 0 1752 368092 0 61 280320 0 16384 16384 0 17 406 0 1 278528 0 65536 65536 4 0 4 0 1 262144 0 128 128 443 1510 1755 0 125 249984 0 256 256 235 740 8314 0 61 249600 0 64 64 227 3617 7608 0 253 246016 0 256 256 122 838 18684 0 61 245760 0 64 64 224 3558 193721 0 253 242048 0 SLEEPQUEUE 88 1883 783 1883 0 125 234608 0 ttyoutq 256 173 742 2762 0 61 234240 0 32 32 3348 3968 86994 0 253 234112 0 4 Bucket 32 362 6954 184517 0 253 234112 0 16 16 140 14302 1771899 0 253 231072 0 8 Bucket 64 120 3475 50471 0 253 230080 0 8192 8192 18 10 1933 0 1 229376 0 256 256 53 802 11777 0 61 218880 0 unpcb 256 19 821 5907 0 61 215040 0 512 512 71 349 1166758 0 29 215040 0 NAMEI 1024 0 208 7422027 0 13 212992 0 tcpcb 976 9 207 65 0 13 210816 0 selfd 64 144 3018 62890 0 253 202368 0 128 128 60 1521 272431 0 125 202368 0 64 64 16 3084 16039 0 253 198400 0 32768 32768 0 6 3126 0 1 196608 0 8192 8192 3 21 7657 0 1 196608 0 2048 2048 91 5 91 0 5 196608 0 1024 1024 33 159 3976 0 13 196608 0 tcp_inpcb 488 9 391 65 0 29 195200 0 1024 1024 2 182 522369 0 13 188416 0 6 Bucket 48 45 3690 32744 0 253 179280 0 2048 2048 13 73 399576 0 5 176128 0 mt_stats_zone 64 430 2322 509 0 253 176128 0 ttyinq 160 330 750 5190 0 61 172800 0 4096 4096 42 0 46 0 1 172032 0 UMA Zones 1968 86 1 120 0 0 171216 0 64 64 103 2563 100181 0 253 170624 0 1024 1024 1 159 71 0 13 163840 0 4096 4096 22 17 7092 0 1 159744 0 fakepg 104 0 1520 1281 0 125 158080 0 8192 8192 6 13 275 0 1 155648 0 udp_inpcb 488 20 292 214 0 29 152256 0 512 512 0 294 334 0 29 150528 0 512 512 37 257 141 0 29 150528 0 1024 1024 12 128 876 0 13 143360 0 32 32 795 3669 69005 0 253 142848 0 vnpbuf 984 0 143 2484 0 13 140712 0 32 32 135 4081 738918 0 253 134912 0 32768 32768 1 3 9 0 1 131072 0 2048 2048 4 60 6138 0 5 131072 0 KMAP ENTRY 112 18 1134 160 0 125 129024 0 rl_entry 40 934 2234 934 0 253 126720 0 1024 1024 2 118 69 0 13 122880 0 32 32 164 3556 22916 0 253 119040 0 32 32 585 3135 51098 0 253 119040 0 4096 4096 16 13 500649 0 1 118784 0 8192 8192 14 0 14 0 1 114688 0 16 16 3094 3629 184217 0 253 107568 0 32 32 298 3050 20198 0 253 107136 0 32 32 88 3136 1651 0 253 103168 0 L VFS Cache 328 106 206 151 0 29 102336 0 procdesc 136 1 637 40 0 61 86768 0 clpbuf 984 0 87 29445 1047 5 85608 0 FFS1 dinode 128 0 651 11 0 125 83328 0 4096 4096 5 15 24447 0 1 81920 0 2048 2048 0 40 20 0 5 81920 0 1024 1024 67 13 70 0 13 81920 0 tcptw 88 0 900 17 0 125 79200 0 syncache 168 0 460 18 0 61 77280 0 rentr 24 0 3154 2092 0 253 75696 0 udpcb 32 20 2336 214 0 253 75392 0 2048 2048 21 13 84802 0 5 69632 0 mt_zone 24 430 2392 509 0 253 67728 0 ripcb 488 1 135 15 0 29 66368 0 65536 65536 1 0 1 0 1 65536 0 65536 65536 1 0 1 0 1 65536 0 16384 16384 4 0 4 0 1 65536 0 16 16 470 3265 893775 0 253 59760 0 umtx_shm 88 0 630 35 0 125 55440 0 16 16 28 3209 17933 0 253 51792 0 16 16 159 3078 102079 0 253 51792 0 16 16 43 3194 12049 0 253 51792 0 cpuset 104 27 469 27 0 125 51584 0 16 16 79 2909 292 0 253 47808 0 VNODEPOLL 120 0 396 4 0 125 47520 0 16 16 2 2737 100012 0 253 43824 0 128 128 85 256 87 0 125 43648 0 8192 8192 4 1 5 0 1 40960 0 64 pcpu 8 3085 2035 3549 0 253 40960 0 swrbuf 984 1 36 175 0 5 36408 0 16384 16384 2 0 2 0 1 32768 0 rtentry 208 12 121 15 0 61 27664 0 umtx pi 96 0 287 2 0 125 27552 0 512 512 13 29 14 0 29 21504 0 UMA Kegs 216 75 3 109 0 0 16848 0 16384 16384 1 0 1 0 1 16384 0 8192 8192 1 1 2 0 1 16384 0 2048 2048 2 6 3 0 5 16384 0 1024 1024 1 15 3 0 13 16384 0 hostcache 96 1 163 1 0 125 15744 0 MAP 232 2 65 2 0 61 15544 0 int pcpu 4 480 2592 480 0 253 12288 0 mdpbuf 984 0 12 12200 0 1 11808 0 512 512 15 6 30 0 29 10752 0 2048 2048 2 2 2 0 5 8192 0 epoch_record pcpu 256 4 12 4 0 61 4096 0 4096 4096 1 0 1 0 1 4096 0 32 32 1 123 1 0 253 3968 0 UMA Hash 256 1 11 2 0 0 3072 0 vmem 1856 0 1 0 0 5 1856 0 IPsec SA lft_c 16 0 0 0 0 253 0 0 udplite_inpcb 488 0 0 0 0 29 0 0 sctp_asconf_ack 48 0 0 0 0 253 0 0 sctp_asconf 40 0 0 0 0 253 0 0 sctp_stream_msg_out 112 0 0 0 0 125 0 0 sctp_readq 152 0 0 0 0 61 0 0 sctp_chunk 152 0 0 0 0 61 0 0 sctp_raddr 736 0 0 0 0 13 0 0 sctp_laddr 48 0 0 4 0 253 0 0 sctp_asoc 2408 0 0 0 0 3 0 0 sctp_ep 1280 0 0 0 0 9 0 0 tcp_log_node 120 0 0 0 0 125 0 0 tcp_log_bucket 144 0 0 0 0 61 0 0 tcp_log 408 0 0 0 0 29 0 0 tcpreass 48 0 0 0 0 253 0 0 tfo_ccache_entries 80 0 0 0 0 125 0 0 tfo 4 0 0 0 0 253 0 0 sackhole 32 0 0 0 0 253 0 0 ipq 56 0 0 0 0 253 0 0 itimer 352 0 0 0 0 29 0 0 AIOLIO 280 0 0 0 0 29 0 0 AIOCB 752 0 0 0 0 13 0 0 AIOP 32 0 0 0 0 253 0 0 AIO 208 0 0 0 0 61 0 0 NCLNODE 744 0 0 0 0 13 0 0 LTS VFS Cache 368 0 0 0 0 29 0 0 STS VFS Cache 148 0 0 0 0 61 0 0 vtnet_tx_hdr 24 0 0 0 0 253 0 0 crypto_session 24 0 0 0 0 253 0 0 cryptodesc 120 0 0 0 0 125 0 0 cryptop 128 0 0 0 0 125 0 0 nvme_request 128 0 0 0 0 125 0 0 FPU_save_area 832 0 0 0 0 13 0 0 DMAR_MAP_ENTRY 120 0 0 0 0 125 0 0 mbuf_extpgs 256 0 0 0 0 61 0 0 mbuf_jumbo_16k 16384 0 0 0 0 1 0 0 mbuf_jumbo_9k 9216 0 0 0 0 1 0 0 mbuf_jumbo_page 4096 0 0 2 0 1 0 0 audit_record 1280 0 0 0 0 9 0 0 domainset 40 0 0 0 0 253 0 0 MAC labels 40 0 0 0 0 253 0 0 nfspbuf 984 0 0 0 0 5 0 0 swwbuf 984 0 0 25712 0 1 0 0 rangeset pctrie nodes 144 0 0 0 0 61 0 0 65536 65536 0 0 0 0 1 0 0 65536 65536 0 0 0 0 1 0 0 32768 32768 0 0 0 0 1 0 0 32768 32768 0 0 0 0 1 0 0 16384 16384 0 0 0 0 1 0 0 16384 16384 0 0 0 0 1 0 0 8192 8192 0 0 0 0 1 0 0 4096 4096 0 0 13 0 1 0 0 db:0:pho> show umacache Zone Size Used Free Requests Bucket debugnet_mbuf_packet 2048 0 0 0 0 debugnet_mbuf_cluster 2048 -6144 0 0 0 debugnet_mbuf 256 -10240 0 0 0 kstack_cache 16384 1233 10 258598 1 vm pgcache 4096 108835 43 5577926 20 vm pgcache 4096 918370 160 17615460 88 buf free cache 984 8288 388 585604 13 buffer arena-65 65536 0 5 133 1 buffer arena-61 61440 0 0 0 1 buffer arena-57 57344 0 0 0 1 buffer arena-53 53248 0 0 0 1 buffer arena-49 49152 0 0 0 1 buffer arena-45 45056 0 0 0 1 buffer arena-40 40960 0 0 0 1 buffer arena-36 36864 0 0 0 1 buffer arena-32 32768 4257 41 246034 1 buffer arena-28 28672 0 0 0 1 buffer arena-24 24576 0 0 0 1 buffer arena-20 20480 0 0 0 1 buffer arena-16 16384 3 15 80702 1 buffer arena-12 12288 0 0 0 1 buffer arena-81 8192 0 0 0 1 buffer arena-40 4096 0 0 0 1 db:0:pho> acttrace Tracing command execve pid 79096 tid 102484 td 0xfffff806ffccd000 (CPU 10) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1dfdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1dfdf0 trap() at trap+0x61/frame 0xfffffe003e1dff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1dff20 --- trap 0x13, rip = 0xffffffff80c16112, rsp = 0xfffffe02f9349650, rbp = 0xfffffe02f9349660 --- lock_delay() at lock_delay+0x32/frame 0xfffffe02f9349660 _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0xd1/frame 0xfffffe02f93496f0 __mtx_lock_spin_flags() at __mtx_lock_spin_flags+0xda/frame 0xfffffe02f9349730 smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0xca/frame 0xfffffe02f93497b0 pmap_invalidate_range() at pmap_invalidate_range+0x1b8/frame 0xfffffe02f9349830 vm_thread_stack_dispose() at vm_thread_stack_dispose+0x25/frame 0xfffffe02f9349880 kstack_release() at kstack_release+0x46/frame 0xfffffe02f93498b0 zone_free_item() at zone_free_item+0x4b/frame 0xfffffe02f93498f0 uma_zfree_arg() at uma_zfree_arg+0x1a4/frame 0xfffffe02f9349980 thread_reap() at thread_reap+0x102/frame 0xfffffe02f93499a0 thread_alloc() at thread_alloc+0xf/frame 0xfffffe02f93499c0 thread_create() at thread_create+0x10f/frame 0xfffffe02f9349a30 sys_thr_new() at sys_thr_new+0xd2/frame 0xfffffe02f9349ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f9349bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f9349bf0 --- syscall (455, FreeBSD ELF64, sys_thr_new), rip = 0x8003054ca, rsp = 0x7fffffffe648, rbp = 0x7fffffffe730 --- Tracing command execve pid 79094 tid 100914 td 0xfffff804ec3106e0 (CPU 9) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1d3dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1d3df0 trap() at trap+0x61/frame 0xfffffe003e1d3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1d3f20 --- trap 0x13, rip = 0xffffffff80c16112, rsp = 0xfffffe00e3213650, rbp = 0xfffffe00e3213660 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00e3213660 _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0xd1/frame 0xfffffe00e32136f0 __mtx_lock_spin_flags() at __mtx_lock_spin_flags+0xda/frame 0xfffffe00e3213730 smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0xca/frame 0xfffffe00e32137b0 pmap_invalidate_range() at pmap_invalidate_range+0x1b8/frame 0xfffffe00e3213830 vm_thread_stack_dispose() at vm_thread_stack_dispose+0x25/frame 0xfffffe00e3213880 kstack_release() at kstack_release+0x46/frame 0xfffffe00e32138b0 zone_free_item() at zone_free_item+0x4b/frame 0xfffffe00e32138f0 uma_zfree_arg() at uma_zfree_arg+0x1a4/frame 0xfffffe00e3213980 thread_reap() at thread_reap+0x102/frame 0xfffffe00e32139a0 thread_alloc() at thread_alloc+0xf/frame 0xfffffe00e32139c0 thread_create() at thread_create+0x10f/frame 0xfffffe00e3213a30 sys_thr_new() at sys_thr_new+0xd2/frame 0xfffffe00e3213ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e3213bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e3213bf0 --- syscall (455, FreeBSD ELF64, sys_thr_new), rip = 0x8003054ca, rsp = 0x7fffffffe648, rbp = 0x7fffffffe730 --- Tracing command execve pid 79088 tid 101050 td 0xfffff806b3d69000 (CPU 3) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e18bdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e18bdf0 trap() at trap+0x61/frame 0xfffffe003e18bf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e18bf20 --- trap 0x13, rip = 0xffffffff810e3375, rsp = 0xfffffe00e35f2640, rbp = 0xfffffe00e35f2640 --- native_lapic_ipi_raw() at native_lapic_ipi_raw+0x105/frame 0xfffffe00e35f2640 smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0x158/frame 0xfffffe00e35f26c0 pmap_invalidate_range() at pmap_invalidate_range+0x1b8/frame 0xfffffe00e35f2740 vm_thread_stack_create() at vm_thread_stack_create+0x79/frame 0xfffffe00e35f2880 kstack_import() at kstack_import+0x51/frame 0xfffffe00e35f28c0 zone_alloc_item_locked() at zone_alloc_item_locked+0xae/frame 0xfffffe00e35f2920 uma_zalloc_arg() at uma_zalloc_arg+0x10c/frame 0xfffffe00e35f2960 vm_thread_new() at vm_thread_new+0x4b/frame 0xfffffe00e35f29a0 thread_alloc() at thread_alloc+0x3a/frame 0xfffffe00e35f29c0 thread_create() at thread_create+0x10f/frame 0xfffffe00e35f2a30 sys_thr_new() at sys_thr_new+0xd2/frame 0xfffffe00e35f2ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e35f2bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e35f2bf0 --- syscall (455, FreeBSD ELF64, sys_thr_new), rip = 0x8003054ca, rsp = 0x7fffffffe648, rbp = 0x7fffffffe730 --- Tracing command execve pid 79086 tid 100397 td 0xfffff8065d88f6e0 (CPU 7) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1bbdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1bbdf0 trap() at trap+0x61/frame 0xfffffe003e1bbf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1bbf20 --- trap 0x13, rip = 0xffffffff80c16112, rsp = 0xfffffe00e324a560, rbp = 0xfffffe00e324a570 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00e324a570 _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0xd1/frame 0xfffffe00e324a600 __mtx_lock_spin_flags() at __mtx_lock_spin_flags+0xda/frame 0xfffffe00e324a640 smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0xca/frame 0xfffffe00e324a6c0 pmap_invalidate_range() at pmap_invalidate_range+0x1b8/frame 0xfffffe00e324a740 vm_thread_stack_create() at vm_thread_stack_create+0x79/frame 0xfffffe00e324a880 kstack_import() at kstack_import+0x51/frame 0xfffffe00e324a8c0 zone_alloc_item_locked() at zone_alloc_item_locked+0xae/frame 0xfffffe00e324a920 uma_zalloc_arg() at uma_zalloc_arg+0x10c/frame 0xfffffe00e324a960 vm_thread_new() at vm_thread_new+0x4b/frame 0xfffffe00e324a9a0 thread_alloc() at thread_alloc+0x3a/frame 0xfffffe00e324a9c0 thread_create() at thread_create+0x10f/frame 0xfffffe00e324aa30 sys_thr_new() at sys_thr_new+0xd2/frame 0xfffffe00e324aac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e324abf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e324abf0 --- syscall (455, FreeBSD ELF64, sys_thr_new), rip = 0x8003054ca, rsp = 0x7fffffffe648, rbp = 0x7fffffffe730 --- Tracing command execve pid 79083 tid 101212 td 0xfffff806baa7e000 (CPU 0) cpustop_handler() at cpustop_handler+0x2a/frame 0xffffffff81d83f80 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xffffffff81d83fa0 trap() at trap+0x61/frame 0xffffffff81d840d0 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff81d840d0 --- trap 0x13, rip = 0xffffffff80c36cda, rsp = 0xfffffe02f93a36d0, rbp = 0xfffffe02f93a3700 --- witness_lock() at witness_lock+0xba/frame 0xfffffe02f93a3700 _rw_wlock_cookie() at _rw_wlock_cookie+0xcf/frame 0xfffffe02f93a3740 vm_thread_stack_create() at vm_thread_stack_create+0x98/frame 0xfffffe02f93a3880 kstack_import() at kstack_import+0x51/frame 0xfffffe02f93a38c0 zone_alloc_item_locked() at zone_alloc_item_locked+0xae/frame 0xfffffe02f93a3920 uma_zalloc_arg() at uma_zalloc_arg+0x10c/frame 0xfffffe02f93a3960 vm_thread_new() at vm_thread_new+0x4b/frame 0xfffffe02f93a39a0 thread_alloc() at thread_alloc+0x3a/frame 0xfffffe02f93a39c0 thread_create() at thread_create+0x10f/frame 0xfffffe02f93a3a30 sys_thr_new() at sys_thr_new+0xd2/frame 0xfffffe02f93a3ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f93a3bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f93a3bf0 --- syscall (455, FreeBSD ELF64, sys_thr_new), rip = 0x8003054ca, rsp = 0x7fffffffe648, rbp = 0x7fffffffe730 --- Tracing command execve pid 79080 tid 100179 td 0xfffff806ba6856e0 (CPU 1) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e173dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e173df0 trap() at trap+0x61/frame 0xfffffe003e173f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e173f20 --- trap 0x13, rip = 0xffffffff80c16112, rsp = 0xfffffe02f92bd560, rbp = 0xfffffe02f92bd570 --- lock_delay() at lock_delay+0x32/frame 0xfffffe02f92bd570 _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0xd1/frame 0xfffffe02f92bd600 __mtx_lock_spin_flags() at __mtx_lock_spin_flags+0xda/frame 0xfffffe02f92bd640 smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0xca/frame 0xfffffe02f92bd6c0 pmap_invalidate_range() at pmap_invalidate_range+0x1b8/frame 0xfffffe02f92bd740 vm_thread_stack_create() at vm_thread_stack_create+0x79/frame 0xfffffe02f92bd880 kstack_import() at kstack_import+0x51/frame 0xfffffe02f92bd8c0 zone_alloc_item_locked() at zone_alloc_item_locked+0xae/frame 0xfffffe02f92bd920 uma_zalloc_arg() at uma_zalloc_arg+0x10c/frame 0xfffffe02f92bd960 vm_thread_new() at vm_thread_new+0x4b/frame 0xfffffe02f92bd9a0 thread_alloc() at thread_alloc+0x3a/frame 0xfffffe02f92bd9c0 thread_create() at thread_create+0x10f/frame 0xfffffe02f92bda30 sys_thr_new() at sys_thr_new+0xd2/frame 0xfffffe02f92bdac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f92bdbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f92bdbf0 --- syscall (455, FreeBSD ELF64, sys_thr_new), rip = 0x8003054ca, rsp = 0x7fffffffe648, rbp = 0x7fffffffe730 --- Tracing command execve pid 79078 tid 101055 td 0xfffff806f4784000 (CPU 2) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e17fdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e17fdf0 trap() at trap+0x61/frame 0xfffffe003e17ff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e17ff20 --- trap 0x13, rip = 0xffffffff80c16112, rsp = 0xfffffe00e16aa560, rbp = 0xfffffe00e16aa570 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00e16aa570 _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0xd1/frame 0xfffffe00e16aa600 __mtx_lock_spin_flags() at __mtx_lock_spin_flags+0xda/frame 0xfffffe00e16aa640 smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0xca/frame 0xfffffe00e16aa6c0 pmap_invalidate_range() at pmap_invalidate_range+0x1b8/frame 0xfffffe00e16aa740 vm_thread_stack_create() at vm_thread_stack_create+0x79/frame 0xfffffe00e16aa880 kstack_import() at kstack_import+0x51/frame 0xfffffe00e16aa8c0 zone_alloc_item_locked() at zone_alloc_item_locked+0xae/frame 0xfffffe00e16aa920 uma_zalloc_arg() at uma_zalloc_arg+0x10c/frame 0xfffffe00e16aa960 vm_thread_new() at vm_thread_new+0x4b/frame 0xfffffe00e16aa9a0 thread_alloc() at thread_alloc+0x3a/frame 0xfffffe00e16aa9c0 thread_create() at thread_create+0x10f/frame 0xfffffe00e16aaa30 sys_thr_new() at sys_thr_new+0xd2/frame 0xfffffe00e16aaac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e16aabf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e16aabf0 --- syscall (455, FreeBSD ELF64, sys_thr_new), rip = 0x8003054ca, rsp = 0x7fffffffe648, rbp = 0x7fffffffe730 --- Tracing command execve pid 79077 tid 101035 td 0xfffff80506061000 (CPU 5) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1a3dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1a3df0 trap() at trap+0x61/frame 0xfffffe003e1a3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1a3f20 --- trap 0x13, rip = 0xffffffff80c16112, rsp = 0xfffffe00e3303560, rbp = 0xfffffe00e3303570 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00e3303570 _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0xd1/frame 0xfffffe00e3303600 __mtx_lock_spin_flags() at __mtx_lock_spin_flags+0xda/frame 0xfffffe00e3303640 smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0xca/frame 0xfffffe00e33036c0 pmap_invalidate_range() at pmap_invalidate_range+0x1b8/frame 0xfffffe00e3303740 vm_thread_stack_create() at vm_thread_stack_create+0x79/frame 0xfffffe00e3303880 kstack_import() at kstack_import+0x51/frame 0xfffffe00e33038c0 zone_alloc_item_locked() at zone_alloc_item_locked+0xae/frame 0xfffffe00e3303920 uma_zalloc_arg() at uma_zalloc_arg+0x10c/frame 0xfffffe00e3303960 vm_thread_new() at vm_thread_new+0x4b/frame 0xfffffe00e33039a0 thread_alloc() at thread_alloc+0x3a/frame 0xfffffe00e33039c0 thread_create() at thread_create+0x10f/frame 0xfffffe00e3303a30 sys_thr_new() at sys_thr_new+0xd2/frame 0xfffffe00e3303ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e3303bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e3303bf0 --- syscall (455, FreeBSD ELF64, sys_thr_new), rip = 0x8003054ca, rsp = 0x7fffffffe648, rbp = 0x7fffffffe730 --- Tracing command execve pid 79076 tid 100566 td 0xfffff806b3d6c000 (CPU 4) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e197dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e197df0 trap() at trap+0x61/frame 0xfffffe003e197f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e197f20 --- trap 0x13, rip = 0xffffffff80c16110, rsp = 0xfffffe02f933f560, rbp = 0xfffffe02f933f570 --- lock_delay() at lock_delay+0x30/frame 0xfffffe02f933f570 _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0xd1/frame 0xfffffe02f933f600 __mtx_lock_spin_flags() at __mtx_lock_spin_flags+0xda/frame 0xfffffe02f933f640 smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0xca/frame 0xfffffe02f933f6c0 pmap_invalidate_range() at pmap_invalidate_range+0x1b8/frame 0xfffffe02f933f740 vm_thread_stack_create() at vm_thread_stack_create+0x79/frame 0xfffffe02f933f880 kstack_import() at kstack_import+0x51/frame 0xfffffe02f933f8c0 zone_alloc_item_locked() at zone_alloc_item_locked+0xae/frame 0xfffffe02f933f920 uma_zalloc_arg() at uma_zalloc_arg+0x10c/frame 0xfffffe02f933f960 vm_thread_new() at vm_thread_new+0x4b/frame 0xfffffe02f933f9a0 thread_alloc() at thread_alloc+0x3a/frame 0xfffffe02f933f9c0 thread_create() at thread_create+0x10f/frame 0xfffffe02f933fa30 sys_thr_new() at sys_thr_new+0xd2/frame 0xfffffe02f933fac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f933fbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f933fbf0 --- syscall (455, FreeBSD ELF64, sys_thr_new), rip = 0x8003054ca, rsp = 0x7fffffffe648, rbp = 0x7fffffffe730 --- Tracing command execve pid 79072 tid 101314 td 0xfffff806ffd9a000 (CPU 6) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1afdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1afdf0 trap() at trap+0x61/frame 0xfffffe003e1aff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1aff20 --- trap 0x13, rip = 0xffffffff80c16112, rsp = 0xfffffe00e03fe560, rbp = 0xfffffe00e03fe570 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00e03fe570 _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0xd1/frame 0xfffffe00e03fe600 __mtx_lock_spin_flags() at __mtx_lock_spin_flags+0xda/frame 0xfffffe00e03fe640 smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0xca/frame 0xfffffe00e03fe6c0 pmap_invalidate_range() at pmap_invalidate_range+0x1b8/frame 0xfffffe00e03fe740 vm_thread_stack_create() at vm_thread_stack_create+0x79/frame 0xfffffe00e03fe880 kstack_import() at kstack_import+0x51/frame 0xfffffe00e03fe8c0 zone_alloc_item_locked() at zone_alloc_item_locked+0xae/frame 0xfffffe00e03fe920 uma_zalloc_arg() at uma_zalloc_arg+0x10c/frame 0xfffffe00e03fe960 vm_thread_new() at vm_thread_new+0x4b/frame 0xfffffe00e03fe9a0 thread_alloc() at thread_alloc+0x3a/frame 0xfffffe00e03fe9c0 thread_create() at thread_create+0x10f/frame 0xfffffe00e03fea30 sys_thr_new() at sys_thr_new+0xd2/frame 0xfffffe00e03feac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e03febf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e03febf0 --- syscall (455, FreeBSD ELF64, sys_thr_new), rip = 0x8003054ca, rsp = 0x7fffffffe648, rbp = 0x7fffffffe730 --- Tracing command cron pid 78999 tid 100164 td 0xfffff806ba27c000 (CPU 8) kdb_enter() at kdb_enter+0x37/frame 0xfffffe02f94a4500 vpanic() at vpanic+0x19a/frame 0xfffffe02f94a4560 panic() at panic+0x43/frame 0xfffffe02f94a45c0 trap_fatal() at trap_fatal+0x386/frame 0xfffffe02f94a4620 trap_pfault() at trap_pfault+0x99/frame 0xfffffe02f94a46a0 trap() at trap+0x2c0/frame 0xfffffe02f94a47d0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a47d0 --- trap 0xc, rip = 0xffffffff80f01279, rsp = 0xfffffe02f94a48a0, rbp = 0xfffffe02f94a48d0 --- swap_pager_copy() at swap_pager_copy+0xf9/frame 0xfffffe02f94a48d0 vm_object_collapse() at vm_object_collapse+0x181/frame 0xfffffe02f94a4960 vm_object_deallocate() at vm_object_deallocate+0x41d/frame 0xfffffe02f94a49b0 vm_map_process_deferred() at vm_map_process_deferred+0xad/frame 0xfffffe02f94a49d0 vm_map_remove() at vm_map_remove+0xeb/frame 0xfffffe02f94a4a00 vmspace_exit() at vmspace_exit+0xd5/frame 0xfffffe02f94a4a40 exit1() at exit1+0x5a2/frame 0xfffffe02f94a4ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe02f94a4ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f94a4bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f94a4bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8003d880a, rsp = 0x7fffffffeb08, rbp = 0x7fffffffeb30 --- Tracing command pagedaemon pid 30 tid 100128 td 0xfffff800261bd000 (CPU 11) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1ebdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1ebdf0 trap() at trap+0x61/frame 0xfffffe003e1ebf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1ebf20 --- trap 0x13, rip = 0xffffffff810530ba, rsp = 0xfffffe00d3218650, rbp = 0xfffffe00d3218680 --- pmap_invalidate_page_pcid_invpcid() at pmap_invalidate_page_pcid_invpcid+0xca/frame 0xfffffe00d3218680 pmap_invalidate_page() at pmap_invalidate_page+0x64/frame 0xfffffe00d32186e0 pmap_ts_referenced() at pmap_ts_referenced+0x7dd/frame 0xfffffe00d3218770 vm_pageout_worker() at vm_pageout_worker+0x1079/frame 0xfffffe00d3218b70 vm_pageout() at vm_pageout+0x193/frame 0xfffffe00d3218bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00d3218bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d3218bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db:0:pho> ps pid ppid pgrp uid state wmesg wchan cmd 79106 76488 8322 0 R+ execve 79105 76488 8322 0 R+ execve 79104 76488 8322 0 R+ execve 79103 76488 8322 0 R+ execve 79102 76488 8322 0 R+ execve 79101 76488 8322 0 R+ execve 79100 76488 8322 0 R+ execve 79099 76488 8322 0 R+ execve 79098 76488 8322 0 R+ execve 79097 76488 8322 0 R+ execve 79096 76488 8322 0 R+ (threaded) execve 102484 Run CPU 10 execve 100975 RunQ execve 79095 76488 8322 0 R+ (threaded) execve 101382 RunQ execve 101021 RunQ execve 79094 76488 8322 0 R+ (threaded) execve 100914 Run CPU 9 execve 100741 RunQ execve 79093 76488 8322 0 R+ execve 79092 76488 8322 0 R+ execve 79091 76488 8322 0 R+ execve 79090 76488 8322 0 R+ execve 79089 76488 8322 0 R+ (threaded) execve 100587 RunQ execve 100758 RunQ execve 79088 76488 8322 0 R+ CPU 3 execve 79087 76488 8322 0 R+ (threaded) execve 100557 RunQ execve 100884 RunQ execve 100913 RunQ execve 79086 76488 8322 0 R+ CPU 7 execve 79085 76488 8322 0 R+ execve 79084 76488 8322 0 R+ execve 79083 76488 8322 0 R+ CPU 0 execve 79082 76488 8322 0 R+ execve 79081 76488 8322 0 R+ (threaded) execve 100514 RunQ execve 100895 RunQ execve 79080 76488 8322 0 R+ CPU 1 execve 79079 76488 8322 0 R+ (threaded) execve 100853 RunQ execve 100629 RunQ execve 79078 76488 8322 0 R+ CPU 2 execve 79077 76488 8322 0 R+ (threaded) execve 101035 Run CPU 5 execve 100750 RunQ execve 100892 RunQ execve 79076 76488 8322 0 R+ (threaded) execve 100566 Run CPU 4 execve 100837 RunQ execve 79075 76488 8322 0 R+ (threaded) execve 100520 RunQ execve 100609 RunQ execve 100610 RunQ execve 100621 RunQ execve 100622 RunQ execve 100626 RunQ execve 79074 76488 8322 0 R+ (threaded) execve 101103 RunQ execve 100607 RunQ execve 100733 RunQ execve 79073 76488 8322 0 R+ (threaded) execve 101292 RunQ execve 100589 RunQ execve 79072 76488 8322 0 R+ (threaded) execve 101314 Run CPU 6 execve 100580 RunQ execve 79071 76488 8322 0 R+ (threaded) execve 101337 RunQ execve 100579 RunQ execve 100878 RunQ execve 79070 76488 8322 0 R+ (threaded) execve 101000 RunQ execve 100576 RunQ execve 100693 RunQ execve 79069 76488 8322 0 R+ (threaded) execve 100212 RunQ execve 100561 RunQ execve 100684 RunQ execve 79068 76488 8322 0 T+ (threaded) true 100422 RunQ execve 100544 s true 100550 RunQ execve 100552 RunQ execve 79067 76488 8322 0 R+ (threaded) execve 100831 RunQ execve 100175 RunQ execve 100492 RunQ execve 100519 RunQ execve 100530 RunQ execve 100531 RunQ execve 79031 77773 2434 0 R sh 78999 2279 2279 0 RE CPU 8 cron 77773 2469 2434 0 S piperd 0xfffff801ef1778e8 sh 76488 75328 8322 0 R+ execve 75355 75343 75340 0 R swap 75354 75343 75340 0 R swap 75353 75343 75340 0 D swread 0xfffff807a6ce9e30 swap 75352 75343 75340 0 R swap 75351 75343 75340 0 R swap 75350 75343 75340 0 R swap 75349 75343 75340 0 R swap 75348 75343 75340 0 R swap 75347 75343 75340 0 R swap 75346 75343 75340 0 R swap 75345 75343 75340 0 R swap 75344 75343 75340 0 R swap 75343 75340 75340 0 SW wait 0xfffff806ba658530 swap 75340 1 75340 0 Ss nanslp 0xffffffff81c7f2a7 swap 75328 75326 8322 0 SW+ wait 0xfffff80004115530 sh 75327 8322 8322 0 S+ piperd 0xfffff8025fd895f0 tee 75326 8322 8322 0 SW+ wait 0xfffff806ba407530 sh 8322 2351 8322 0 SW+ wait 0xfffff801b530f000 sh 2472 2434 2434 0 S piperd 0xfffff80026d92000 sh 2470 2434 2434 0 S piperd 0xfffff80026d922f8 awk 2469 2434 2434 0 SW wait 0xfffff80004117000 sh 2434 1 2434 0 SWs wait 0xfffff80026165000 sh 2351 2349 2351 0 S+ select 0xfffff800260c1ac0 bash 2349 2348 2349 0 SW+ pause 0xfffff80026c475d8 csh 2348 2343 2348 2006 SW+ wait 0xfffff80026728000 su 2343 2342 2343 2006 SWs+ wait 0xfffff80026165530 bash 2342 2339 2339 2006 S select 0xfffff8002508d640 sshd 2339 2275 2339 0 Ss select 0xfffff80026cfacc0 sshd 2335 2334 2335 0 S+ ttyin 0xfffff8002509b8b0 csh 2334 1 2334 0 SWs+ wait 0xfffff80026737000 login 2333 1 2333 0 Ss+ ttyin 0xfffff800250a3cb0 getty 2332 1 2332 0 Ss+ ttyin 0xfffff800261d88b0 getty 2331 1 2331 0 Ss+ ttyin 0xfffff800261d70b0 getty 2330 1 2330 0 Ss+ ttyin 0xfffff800261d78b0 getty 2329 1 2329 0 Ss+ ttyin 0xfffff800250a68b0 getty 2328 1 2328 0 Ss+ ttyin 0xfffff800261220b0 getty 2327 1 2327 0 Ss+ ttyin 0xfffff800261228b0 getty 2326 1 2326 0 Ss+ ttyin 0xfffff800261210b0 getty 2325 1 2325 0 Ss+ ttyin 0xfffff800261218b0 getty 2279 1 2279 0 Ss nanslp 0xffffffff81c7f2a0 cron 2275 1 2275 0 Ss select 0xfffff80026b98f40 sshd 2247 1 2247 0 Ss select 0xfffff800039e8b40 powerd 2244 1 2244 123 Ss (threaded) ntpd 100177 S select 0xfffff80026c3edc0 ntpd 100677 S usem 0xfffff80195411500 ntpd 2181 1 2181 0 Ss rpcsvc 0xfffff80026177a60 NLM: master 2178 1 2178 0 Ss select 0xfffff800260c1bc0 rpc.statd 2132 1 2132 0 Ss nanslp 0xffffffff81c7f2a2 watchdogd 2053 1 2053 0 Ss select 0xfffff800269911c0 rpcbind 2043 1 2043 0 Ds biord 0xfffffe003b02f038 syslogd 1972 1 1972 0 Ss select 0xfffff80026d1a1c0 devd 1967 1 1967 65 Ss select 0xfffff80026da5240 dhclient 1920 1 1920 0 Ss select 0xfffff80026991340 dhclient 1917 1 1917 0 Ss select 0xfffff8003c4ef2c0 dhclient 34 0 0 0 DL syncer 0xffffffff81d55760 [syncer] 33 0 0 0 DL vlruwt 0xfffff8002616a000 [vnlru] 32 0 0 0 DL (threaded) [bufdaemon] 100130 D qsleep 0xffffffff81d54c18 [bufdaemon] 100132 D - 0xffffffff8180a900 [bufspacedaemon-0] 100134 D - 0xffffffff81812bc0 [bufspacedaemon-1] 100135 D - 0xffffffff8181ae80 [bufspacedaemon-2] 100136 D - 0xffffffff81823140 [bufspacedaemon-3] 100137 D - 0xffffffff8182b400 [bufspacedaemon-4] 100139 D - 0xffffffff818336c0 [bufspacedaemon-5] 100140 D - 0xffffffff8183b980 [bufspacedaemon-6] 100152 D sdflush 0xfffff8002632c4e8 [/ worker] 100200 D sdflush 0xfffff800262fa6e8 [/tmp worker] 31 0 0 0 RL [vmdaemon] 30 0 0 0 RL (threaded) [pagedaemon] 100128 Run CPU 11 [dom0] 100131 D launds 0xffffffff81d9d4e4 [laundry: dom0] 100133 D umarcl 0xffffffff80f08830 [uma] 29 0 0 0 DL idle 0xfffff80026275530 [enc_daemon1] 28 0 0 0 DL idle 0xfffff80026275a60 [enc_daemon0] 27 0 0 0 DL - 0xffffffff81adaf18 [rand_harvestq] 26 0 0 0 DL waiting 0xffffffff81de2ab0 [sctp_iterator] 25 0 0 0 DL - 0xffffffff81d5461c [soaiod4] 24 0 0 0 DL - 0xffffffff81d5461c [soaiod3] 23 0 0 0 DL - 0xffffffff81d5461c [soaiod2] 22 0 0 0 DL - 0xffffffff81d5461c [soaiod1] 21 0 0 0 DL (threaded) [usb] 100096 D - 0xfffffe00d1584460 [usbus0] 100097 D - 0xfffffe00d15844b8 [usbus0] 100098 D - 0xfffffe00d1584510 [usbus0] 100099 D - 0xfffffe00d1584568 [usbus0] 100100 D - 0xfffffe00d15845c0 [usbus0] 100102 D - 0xfffffe0037b66d10 [usbus1] 100103 D - 0xfffffe0037b66d68 [usbus1] 100104 D - 0xfffffe0037b66dc0 [usbus1] 100105 D - 0xfffffe0037b66e18 [usbus1] 100106 D - 0xfffffe0037b66e70 [usbus1] 100107 D - 0xfffffe00d16a7d10 [usbus2] 100108 D - 0xfffffe00d16a7d68 [usbus2] 100109 D - 0xfffffe00d16a7dc0 [usbus2] 100110 D - 0xfffffe00d16a7e18 [usbus2] 100111 D - 0xfffffe00d16a7e70 [usbus2] 20 0 0 0 DL (threaded) [cam] 100091 D - 0xffffffff819b62c0 [doneq0] 100092 D - 0xffffffff819b6340 [doneq1] 100093 D - 0xffffffff819b63c0 [doneq2] 100125 D - 0xffffffff819b6188 [scanner] 19 0 0 0 DL crypto_ 0xfffff80003a74450 [crypto returns 11] 18 0 0 0 DL crypto_ 0xfffff80003a743f0 [crypto returns 10] 17 0 0 0 DL crypto_ 0xfffff80003a74390 [crypto returns 9] 16 0 0 0 DL crypto_ 0xfffff80003a74330 [crypto returns 8] 15 0 0 0 DL crypto_ 0xfffff80003a742d0 [crypto returns 7] 9 0 0 0 DL crypto_ 0xfffff80003a74270 [crypto returns 6] 8 0 0 0 DL crypto_ 0xfffff80003a74210 [crypto returns 5] 7 0 0 0 DL crypto_ 0xfffff80003a741b0 [crypto returns 4] 6 0 0 0 DL crypto_ 0xfffff80003a74150 [crypto returns 3] 5 0 0 0 DL crypto_ 0xfffff80003a740f0 [crypto returns 2] 4 0 0 0 DL crypto_ 0xfffff80003a74090 [crypto returns 1] 3 0 0 0 DL crypto_ 0xfffff80003a74030 [crypto returns 0] 2 0 0 0 DL crypto_ 0xffffffff81d6a6f8 [crypto] 14 0 0 0 DL seqstat 0xfffff800039f7c88 [sequencer 00] 13 0 0 0 DL (threaded) [geom] 100062 D - 0xffffffff81d9bad0 [g_event] 100063 D - 0xffffffff81d9bae0 [g_up] 100064 D - 0xffffffff81d9bad8 [g_down] 12 0 0 0 WL (threaded) [intr] 100018 I [swi6: task queue] 100019 I [swi6: Giant taskq] 100021 I [swi5: fast taskq] 100047 I [swi4: clock (0)] 100048 I [swi4: clock (1)] 100049 I [swi4: clock (2)] 100050 I [swi4: clock (3)] 100051 I [swi4: clock (4)] 100052 I [swi4: clock (5)] 100053 I [swi4: clock (6)] 100054 I [swi4: clock (7)] 100055 I [swi4: clock (8)] 100056 I [swi4: clock (9)] 100057 I [swi4: clock (10)] 100058 I [swi4: clock (11)] 100059 I [swi3: vm] 100060 I [swi1: netisr 0] 100094 I [irq56: ahci0] 100095 I [irq19: xhci0] 100101 I [irq18: ehci0 ehci1] 100112 I [irq71: ahci1] 100113 I [swi0: uart uart] 100180 I [irq72: ioat0] 100181 I [irq73: ioat1] 100182 I [irq74: ioat2] 100183 I [irq75: ioat3] 100184 I [irq76: ioat4] 100185 I [irq77: ioat5] 100186 I [irq78: ioat6] 100187 I [irq79: ioat7] 11 0 0 0 RL (threaded) [idle] 100003 CanRun [idle: cpu0] 100004 CanRun [idle: cpu1] 100005 CanRun [idle: cpu2] 100006 CanRun [idle: cpu3] 100007 CanRun [idle: cpu4] 100008 CanRun [idle: cpu5] 100009 CanRun [idle: cpu6] 100010 CanRun [idle: cpu7] 100011 CanRun [idle: cpu8] 100012 CanRun [idle: cpu9] 100013 CanRun [idle: cpu10] 100014 CanRun [idle: cpu11] 1 0 1 0 SLs wait 0xfffff800037d6530 [init] 10 0 0 0 DL audit_w 0xffffffff81de3740 [audit] 0 0 0 0 DLs (threaded) [kernel] 100000 D swapin 0xffffffff81d8a1c8 [swapper] 100015 D - 0xfffff800037afe00 [config_0] 100016 D - 0xfffff800037b8e00 [kqueue_ctx taskq] 100017 D - 0xfffff800037b8c00 [aiod_kick taskq] 100020 D - 0xfffff800037b8600 [thread taskq] 100022 D - 0xfffff800037afd00 [if_config_tqg_0] 100023 D - 0xfffff800037afc00 [softirq_0] 100024 D - 0xfffff800037afb00 [softirq_1] 100025 D - 0xfffff800037afa00 [softirq_2] 100026 D - 0xfffff800037af900 [softirq_3] 100027 D - 0xfffff800037af800 [softirq_4] 100028 D - 0xfffff800037af700 [softirq_5] 100029 D - 0xfffff800037af600 [softirq_6] 100030 D - 0xfffff800037af500 [softirq_7] 100031 D - 0xfffff800037af400 [softirq_8] 100032 D - 0xfffff800037af300 [softirq_9] 100033 D - 0xfffff800037af200 [softirq_10] 100034 D - 0xfffff800037af100 [softirq_11] 100035 D - 0xfffff800037af000 [if_io_tqg_0] 100036 D - 0xfffff800037aee00 [if_io_tqg_1] 100037 D - 0xfffff800037aed00 [if_io_tqg_2] 100038 D - 0xfffff800037aec00 [if_io_tqg_3] 100039 D - 0xfffff800037aeb00 [if_io_tqg_4] 100040 D - 0xfffff800037aea00 [if_io_tqg_5] 100041 D - 0xfffff800037ae900 [if_io_tqg_6] 100042 D - 0xfffff800037ae800 [if_io_tqg_7] 100043 D - 0xfffff800037ae700 [if_io_tqg_8] 100044 D - 0xfffff800037ae600 [if_io_tqg_9] 100045 D - 0xfffff800037ae500 [if_io_tqg_10] 100046 D - 0xfffff800037ae400 [if_io_tqg_11] 100061 D - 0xfffff8000398fa00 [firmware taskq] 100066 D - 0xfffff8000398f300 [crypto_0] 100067 D - 0xfffff8000398f300 [crypto_1] 100068 D - 0xfffff8000398f300 [crypto_2] 100069 D - 0xfffff8000398f300 [crypto_3] 100070 D - 0xfffff8000398f300 [crypto_4] 100071 D - 0xfffff8000398f300 [crypto_5] 100072 D - 0xfffff8000398f300 [crypto_6] 100073 D - 0xfffff8000398f300 [crypto_7] 100074 D - 0xfffff8000398f300 [crypto_8] 100075 D - 0xfffff8000398f300 [crypto_9] 100076 D - 0xfffff8000398f300 [crypto_10] 100077 D - 0xfffff8000398f300 [crypto_11] 100114 D - 0xfffff800250b9d00 [mca taskq] 100121 D - 0xfffff80026176c00 [acpi_task_0] 100122 D - 0xfffff80026176c00 [acpi_task_1] 100123 D - 0xfffff80026176c00 [acpi_task_2] 100124 D - 0xfffff80003991600 [CAM taskq] 79033 79031 2434 0 Z sed db:0:pho> allt Tracing command execve pid 79106 tid 100204 td 0xfffff80026736000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e1773b60 mi_switch() at mi_switch+0x182/frame 0xfffffe00e1773ba0 ast() at ast+0x2a7/frame 0xfffffe00e1773bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe770 Tracing command execve pid 79105 tid 100235 td 0xfffff801b53a46e0 fork_trampoline() at fork_trampoline Tracing command execve pid 79104 tid 101074 td 0xfffff80755e8a6e0 fork_trampoline() at fork_trampoline Tracing command execve pid 79103 tid 102249 td 0xfffff806baa77000 fork_trampoline() at fork_trampoline Tracing command execve pid 79102 tid 102307 td 0xfffff806ba687000 fork_trampoline() at fork_trampoline Tracing command execve pid 79101 tid 101288 td 0xfffff806f48e26e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3f2bb60 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3f2bba0 ast() at ast+0x2a7/frame 0xfffffe00e3f2bbf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe6d0 Tracing command execve pid 79100 tid 100380 td 0xfffff8068c71b000 fork_trampoline() at fork_trampoline Tracing command execve pid 79099 tid 100467 td 0xfffff803796a06e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e33b7b60 mi_switch() at mi_switch+0x182/frame 0xfffffe00e33b7ba0 ast() at ast+0x2a7/frame 0xfffffe00e33b7bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe770 Tracing command execve pid 79098 tid 100799 td 0xfffff806bae7f6e0 fork_trampoline() at fork_trampoline Tracing command execve pid 79097 tid 100495 td 0xfffff801b58626e0 fork_trampoline() at fork_trampoline Tracing command execve pid 79096 tid 102484 td 0xfffff806ffccd000 (CPU 10) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1dfdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1dfdf0 trap() at trap+0x61/frame 0xfffffe003e1dff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1dff20 --- trap 0x13, rip = 0xffffffff80c16112, rsp = 0xfffffe02f9349650, rbp = 0xfffffe02f9349660 --- lock_delay() at lock_delay+0x32/frame 0xfffffe02f9349660 _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0xd1/frame 0xfffffe02f93496f0 __mtx_lock_spin_flags() at __mtx_lock_spin_flags+0xda/frame 0xfffffe02f9349730 smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0xca/frame 0xfffffe02f93497b0 pmap_invalidate_range() at pmap_invalidate_range+0x1b8/frame 0xfffffe02f9349830 vm_thread_stack_dispose() at vm_thread_stack_dispose+0x25/frame 0xfffffe02f9349880 kstack_release() at kstack_release+0x46/frame 0xfffffe02f93498b0 zone_free_item() at zone_free_item+0x4b/frame 0xfffffe02f93498f0 uma_zfree_arg() at uma_zfree_arg+0x1a4/frame 0xfffffe02f9349980 thread_reap() at thread_reap+0x102/frame 0xfffffe02f93499a0 thread_alloc() at thread_alloc+0xf/frame 0xfffffe02f93499c0 thread_create() at thread_create+0x10f/frame 0xfffffe02f9349a30 sys_thr_new() at sys_thr_new+0xd2/frame 0xfffffe02f9349ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f9349bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f9349bf0 --- syscall (455, FreeBSD ELF64, sys_thr_new), rip = 0x8003054ca, rsp = 0x7fffffffe648, rbp = 0x7fffffffe730 --- Tracing command execve pid 79096 tid 100975 td 0xfffff802260b1000 fork_trampoline() at fork_trampoline Tracing command execve pid 79095 tid 101382 td 0xfffff806ba29d6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe02f9413b60 mi_switch() at mi_switch+0x182/frame 0xfffffe02f9413ba0 ast() at ast+0x2a7/frame 0xfffffe02f9413bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe730 Tracing command execve pid 79095 tid 101021 td 0xfffff8037969f000 fork_trampoline() at fork_trampoline Tracing command execve pid 79094 tid 100914 td 0xfffff804ec3106e0 (CPU 9) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1d3dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1d3df0 trap() at trap+0x61/frame 0xfffffe003e1d3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1d3f20 --- trap 0x13, rip = 0xffffffff80c16112, rsp = 0xfffffe00e3213650, rbp = 0xfffffe00e3213660 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00e3213660 _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0xd1/frame 0xfffffe00e32136f0 __mtx_lock_spin_flags() at __mtx_lock_spin_flags+0xda/frame 0xfffffe00e3213730 smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0xca/frame 0xfffffe00e32137b0 pmap_invalidate_range() at pmap_invalidate_range+0x1b8/frame 0xfffffe00e3213830 vm_thread_stack_dispose() at vm_thread_stack_dispose+0x25/frame 0xfffffe00e3213880 kstack_release() at kstack_release+0x46/frame 0xfffffe00e32138b0 zone_free_item() at zone_free_item+0x4b/frame 0xfffffe00e32138f0 uma_zfree_arg() at uma_zfree_arg+0x1a4/frame 0xfffffe00e3213980 thread_reap() at thread_reap+0x102/frame 0xfffffe00e32139a0 thread_alloc() at thread_alloc+0xf/frame 0xfffffe00e32139c0 thread_create() at thread_create+0x10f/frame 0xfffffe00e3213a30 sys_thr_new() at sys_thr_new+0xd2/frame 0xfffffe00e3213ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e3213bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e3213bf0 --- syscall (455, FreeBSD ELF64, sys_thr_new), rip = 0x8003054ca, rsp = 0x7fffffffe648, rbp = 0x7fffffffe730 --- Tracing command execve pid 79094 tid 100741 td 0xfffff801b57f7000 fork_trampoline() at fork_trampoline Tracing command execve pid 79093 tid 100755 td 0xfffff804e6dff000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00adde03f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00adde0430 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00adde0450 lapic_handle_timer() at lapic_handle_timer+0xd7/frame 0xfffffe00adde0490 Xtimerint() at Xtimerint+0xb1/frame 0xfffffe00adde0490 --- interrupt, rip = 0xffffffff80c16112, rsp = 0xfffffe00adde0560, rbp = 0xfffffe00adde0570 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00adde0570 _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0xd1/frame 0xfffffe00adde0600 __mtx_lock_spin_flags() at __mtx_lock_spin_flags+0xda/frame 0xfffffe00adde0640 smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0xca/frame 0xfffffe00adde06c0 pmap_invalidate_range() at pmap_invalidate_range+0x1b8/frame 0xfffffe00adde0740 vm_thread_stack_create() at vm_thread_stack_create+0x79/frame 0xfffffe00adde0880 kstack_import() at kstack_import+0x51/frame 0xfffffe00adde08c0 zone_alloc_item_locked() at zone_alloc_item_locked+0xae/frame 0xfffffe00adde0920 uma_zalloc_arg() at uma_zalloc_arg+0x10c/frame 0xfffffe00adde0960 vm_thread_new() at vm_thread_new+0x4b/frame 0xfffffe00adde09a0 thread_alloc() at thread_alloc+0x3a/frame 0xfffffe00adde09c0 thread_create() at thread_create+0x10f/frame 0xfffffe00adde0a30 sys_thr_new() at sys_thr_new+0xd2/frame 0xfffffe00adde0ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00adde0bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00adde0bf0 --- syscall (455, FreeBSD ELF64, sys_thr_new), rip = 0x8003054ca, rsp = 0x7fffffffe648, rbp = 0x7fffffffe730 --- Tracing command execve pid 79092 tid 102492 td 0xfffff806ff0b5000 fork_trampoline() at fork_trampoline Tracing command execve pid 79091 tid 102233 td 0xfffff800598ea000 fork_trampoline() at fork_trampoline Tracing command execve pid 79090 tid 100860 td 0xfffff806f4f7c000 fork_trampoline() at fork_trampoline Tracing command execve pid 79089 tid 100587 td 0xfffff806ba480000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e35cd450 mi_switch() at mi_switch+0x182/frame 0xfffffe00e35cd490 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00e35cd4b0 intr_event_handle() at intr_event_handle+0x18b/frame 0xfffffe00e35cd500 intr_execute_handlers() at intr_execute_handlers+0x75/frame 0xfffffe00e35cd540 lapic_handle_intr() at lapic_handle_intr+0x5b/frame 0xfffffe00e35cd560 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00e35cd560 --- interrupt, rip = 0xffffffff8104cd4a, rsp = 0xfffffe00e35cd630, rbp = 0xfffffe00e35cd640 --- spinlock_exit() at spinlock_exit+0x3a/frame 0xfffffe00e35cd640 smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0x37d/frame 0xfffffe00e35cd6c0 pmap_invalidate_range() at pmap_invalidate_range+0x1b8/frame 0xfffffe00e35cd740 vm_thread_stack_create() at vm_thread_stack_create+0x79/frame 0xfffffe00e35cd880 kstack_import() at kstack_import+0x51/frame 0xfffffe00e35cd8c0 zone_alloc_item_locked() at zone_alloc_item_locked+0xae/frame 0xfffffe00e35cd920 uma_zalloc_arg() at uma_zalloc_arg+0x10c/frame 0xfffffe00e35cd960 vm_thread_new() at vm_thread_new+0x4b/frame 0xfffffe00e35cd9a0 thread_alloc() at thread_alloc+0x3a/frame 0xfffffe00e35cd9c0 thread_create() at thread_create+0x10f/frame 0xfffffe00e35cda30 sys_thr_new() at sys_thr_new+0xd2/frame 0xfffffe00e35cdac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e35cdbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e35cdbf0 --- syscall (455, FreeBSD ELF64, sys_thr_new), rip = 0x8003054ca, rsp = 0x7fffffffe648, rbp = 0x7fffffffe730 --- Tracing command execve pid 79089 tid 100758 td 0xfffff804e92446e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e35ed830 mi_switch() at mi_switch+0x182/frame 0xfffffe00e35ed870 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e35ed8b0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e35ed910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00e35ed950 _sleep() at _sleep+0x258/frame 0xfffffe00e35ed9f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe00e35eda80 sys_nanosleep() at sys_nanosleep+0x97/frame 0xfffffe00e35edac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e35edbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e35edbf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003d3a7a, rsp = 0x7fffdfffdf38, rbp = 0x7fffdfffdf60 --- Tracing command execve pid 79088 tid 101050 td 0xfffff806b3d69000 (CPU 3) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e18bdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e18bdf0 trap() at trap+0x61/frame 0xfffffe003e18bf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e18bf20 --- trap 0x13, rip = 0xffffffff810e3375, rsp = 0xfffffe00e35f2640, rbp = 0xfffffe00e35f2640 --- native_lapic_ipi_raw() at native_lapic_ipi_raw+0x105/frame 0xfffffe00e35f2640 smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0x158/frame 0xfffffe00e35f26c0 pmap_invalidate_range() at pmap_invalidate_range+0x1b8/frame 0xfffffe00e35f2740 vm_thread_stack_create() at vm_thread_stack_create+0x79/frame 0xfffffe00e35f2880 kstack_import() at kstack_import+0x51/frame 0xfffffe00e35f28c0 zone_alloc_item_locked() at zone_alloc_item_locked+0xae/frame 0xfffffe00e35f2920 uma_zalloc_arg() at uma_zalloc_arg+0x10c/frame 0xfffffe00e35f2960 vm_thread_new() at vm_thread_new+0x4b/frame 0xfffffe00e35f29a0 thread_alloc() at thread_alloc+0x3a/frame 0xfffffe00e35f29c0 thread_create() at thread_create+0x10f/frame 0xfffffe00e35f2a30 sys_thr_new() at sys_thr_new+0xd2/frame 0xfffffe00e35f2ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e35f2bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e35f2bf0 --- syscall (455, FreeBSD ELF64, sys_thr_new), rip = 0x8003054ca, rsp = 0x7fffffffe648, rbp = 0x7fffffffe730 --- Tracing command execve pid 79087 tid 100557 td 0xfffff803795a6000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e41a6b60 mi_switch() at mi_switch+0x182/frame 0xfffffe00e41a6ba0 ast() at ast+0x2a7/frame 0xfffffe00e41a6bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe730 Tracing command execve pid 79087 tid 100884 td 0xfffff804e9404000 sched_switch() at sched_switch+0x38f/frame 0xfffffe02f924a830 mi_switch() at mi_switch+0x182/frame 0xfffffe02f924a870 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe02f924a8b0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe02f924a910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe02f924a950 _sleep() at _sleep+0x258/frame 0xfffffe02f924a9f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe02f924aa80 sys_nanosleep() at sys_nanosleep+0x97/frame 0xfffffe02f924aac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f924abf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f924abf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003d3a7a, rsp = 0x7fffdfffdf38, rbp = 0x7fffdfffdf60 --- Tracing command execve pid 79087 tid 100913 td 0xfffff8065d88f000 sched_switch() at sched_switch+0x38f/frame 0xfffffe02f9367830 mi_switch() at mi_switch+0x182/frame 0xfffffe02f9367870 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe02f93678b0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe02f9367910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe02f9367950 _sleep() at _sleep+0x258/frame 0xfffffe02f93679f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe02f9367a80 sys_nanosleep() at sys_nanosleep+0x97/frame 0xfffffe02f9367ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f9367bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f9367bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003d3a7a, rsp = 0x7fffdfdfcf38, rbp = 0x7fffdfdfcf60 --- Tracing command execve pid 79086 tid 100397 td 0xfffff8065d88f6e0 (CPU 7) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1bbdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1bbdf0 trap() at trap+0x61/frame 0xfffffe003e1bbf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1bbf20 --- trap 0x13, rip = 0xffffffff80c16112, rsp = 0xfffffe00e324a560, rbp = 0xfffffe00e324a570 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00e324a570 _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0xd1/frame 0xfffffe00e324a600 __mtx_lock_spin_flags() at __mtx_lock_spin_flags+0xda/frame 0xfffffe00e324a640 smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0xca/frame 0xfffffe00e324a6c0 pmap_invalidate_range() at pmap_invalidate_range+0x1b8/frame 0xfffffe00e324a740 vm_thread_stack_create() at vm_thread_stack_create+0x79/frame 0xfffffe00e324a880 kstack_import() at kstack_import+0x51/frame 0xfffffe00e324a8c0 zone_alloc_item_locked() at zone_alloc_item_locked+0xae/frame 0xfffffe00e324a920 uma_zalloc_arg() at uma_zalloc_arg+0x10c/frame 0xfffffe00e324a960 vm_thread_new() at vm_thread_new+0x4b/frame 0xfffffe00e324a9a0 thread_alloc() at thread_alloc+0x3a/frame 0xfffffe00e324a9c0 thread_create() at thread_create+0x10f/frame 0xfffffe00e324aa30 sys_thr_new() at sys_thr_new+0xd2/frame 0xfffffe00e324aac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e324abf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e324abf0 --- syscall (455, FreeBSD ELF64, sys_thr_new), rip = 0x8003054ca, rsp = 0x7fffffffe648, rbp = 0x7fffffffe730 --- Tracing command execve pid 79085 tid 101211 td 0xfffff806ff0b56e0 fork_trampoline() at fork_trampoline Tracing command execve pid 79084 tid 101332 td 0xfffff805998106e0 fork_trampoline() at fork_trampoline Tracing command execve pid 79083 tid 101212 td 0xfffff806baa7e000 (CPU 0) cpustop_handler() at cpustop_handler+0x2a/frame 0xffffffff81d83f80 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xffffffff81d83fa0 trap() at trap+0x61/frame 0xffffffff81d840d0 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff81d840d0 --- trap 0x13, rip = 0xffffffff80c36cda, rsp = 0xfffffe02f93a36d0, rbp = 0xfffffe02f93a3700 --- witness_lock() at witness_lock+0xba/frame 0xfffffe02f93a3700 _rw_wlock_cookie() at _rw_wlock_cookie+0xcf/frame 0xfffffe02f93a3740 vm_thread_stack_create() at vm_thread_stack_create+0x98/frame 0xfffffe02f93a3880 kstack_import() at kstack_import+0x51/frame 0xfffffe02f93a38c0 zone_alloc_item_locked() at zone_alloc_item_locked+0xae/frame 0xfffffe02f93a3920 uma_zalloc_arg() at uma_zalloc_arg+0x10c/frame 0xfffffe02f93a3960 vm_thread_new() at vm_thread_new+0x4b/frame 0xfffffe02f93a39a0 thread_alloc() at thread_alloc+0x3a/frame 0xfffffe02f93a39c0 thread_create() at thread_create+0x10f/frame 0xfffffe02f93a3a30 sys_thr_new() at sys_thr_new+0xd2/frame 0xfffffe02f93a3ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f93a3bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f93a3bf0 --- syscall (455, FreeBSD ELF64, sys_thr_new), rip = 0x8003054ca, rsp = 0x7fffffffe648, rbp = 0x7fffffffe730 --- Tracing command execve pid 79082 tid 101156 td 0xfffff801b5729000 fork_trampoline() at fork_trampoline Tracing command execve pid 79081 tid 100514 td 0xfffff803795a66e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3f21b60 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3f21ba0 ast() at ast+0x2a7/frame 0xfffffe00e3f21bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe730 Tracing command execve pid 79081 tid 100895 td 0xfffff806ba481000 fork_trampoline() at fork_trampoline Tracing command execve pid 79080 tid 100179 td 0xfffff806ba6856e0 (CPU 1) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e173dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e173df0 trap() at trap+0x61/frame 0xfffffe003e173f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e173f20 --- trap 0x13, rip = 0xffffffff80c16112, rsp = 0xfffffe02f92bd560, rbp = 0xfffffe02f92bd570 --- lock_delay() at lock_delay+0x32/frame 0xfffffe02f92bd570 _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0xd1/frame 0xfffffe02f92bd600 __mtx_lock_spin_flags() at __mtx_lock_spin_flags+0xda/frame 0xfffffe02f92bd640 smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0xca/frame 0xfffffe02f92bd6c0 pmap_invalidate_range() at pmap_invalidate_range+0x1b8/frame 0xfffffe02f92bd740 vm_thread_stack_create() at vm_thread_stack_create+0x79/frame 0xfffffe02f92bd880 kstack_import() at kstack_import+0x51/frame 0xfffffe02f92bd8c0 zone_alloc_item_locked() at zone_alloc_item_locked+0xae/frame 0xfffffe02f92bd920 uma_zalloc_arg() at uma_zalloc_arg+0x10c/frame 0xfffffe02f92bd960 vm_thread_new() at vm_thread_new+0x4b/frame 0xfffffe02f92bd9a0 thread_alloc() at thread_alloc+0x3a/frame 0xfffffe02f92bd9c0 thread_create() at thread_create+0x10f/frame 0xfffffe02f92bda30 sys_thr_new() at sys_thr_new+0xd2/frame 0xfffffe02f92bdac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f92bdbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f92bdbf0 --- syscall (455, FreeBSD ELF64, sys_thr_new), rip = 0x8003054ca, rsp = 0x7fffffffe648, rbp = 0x7fffffffe730 --- Tracing command execve pid 79079 tid 100853 td 0xfffff801497f26e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe02f93d0b60 mi_switch() at mi_switch+0x182/frame 0xfffffe02f93d0ba0 ast() at ast+0x2a7/frame 0xfffffe02f93d0bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe730 Tracing command execve pid 79079 tid 100629 td 0xfffff806b3d616e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e41f6830 mi_switch() at mi_switch+0x182/frame 0xfffffe00e41f6870 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e41f68b0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e41f6910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00e41f6950 _sleep() at _sleep+0x258/frame 0xfffffe00e41f69f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe00e41f6a80 sys_nanosleep() at sys_nanosleep+0x97/frame 0xfffffe00e41f6ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e41f6bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e41f6bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003d3a7a, rsp = 0x7fffdfffdf38, rbp = 0x7fffdfffdf60 --- Tracing command execve pid 79078 tid 101055 td 0xfffff806f4784000 (CPU 2) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e17fdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e17fdf0 trap() at trap+0x61/frame 0xfffffe003e17ff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e17ff20 --- trap 0x13, rip = 0xffffffff80c16112, rsp = 0xfffffe00e16aa560, rbp = 0xfffffe00e16aa570 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00e16aa570 _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0xd1/frame 0xfffffe00e16aa600 __mtx_lock_spin_flags() at __mtx_lock_spin_flags+0xda/frame 0xfffffe00e16aa640 smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0xca/frame 0xfffffe00e16aa6c0 pmap_invalidate_range() at pmap_invalidate_range+0x1b8/frame 0xfffffe00e16aa740 vm_thread_stack_create() at vm_thread_stack_create+0x79/frame 0xfffffe00e16aa880 kstack_import() at kstack_import+0x51/frame 0xfffffe00e16aa8c0 zone_alloc_item_locked() at zone_alloc_item_locked+0xae/frame 0xfffffe00e16aa920 uma_zalloc_arg() at uma_zalloc_arg+0x10c/frame 0xfffffe00e16aa960 vm_thread_new() at vm_thread_new+0x4b/frame 0xfffffe00e16aa9a0 thread_alloc() at thread_alloc+0x3a/frame 0xfffffe00e16aa9c0 thread_create() at thread_create+0x10f/frame 0xfffffe00e16aaa30 sys_thr_new() at sys_thr_new+0xd2/frame 0xfffffe00e16aaac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e16aabf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e16aabf0 --- syscall (455, FreeBSD ELF64, sys_thr_new), rip = 0x8003054ca, rsp = 0x7fffffffe648, rbp = 0x7fffffffe730 --- Tracing command execve pid 79077 tid 101035 td 0xfffff80506061000 (CPU 5) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1a3dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1a3df0 trap() at trap+0x61/frame 0xfffffe003e1a3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1a3f20 --- trap 0x13, rip = 0xffffffff80c16112, rsp = 0xfffffe00e3303560, rbp = 0xfffffe00e3303570 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00e3303570 _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0xd1/frame 0xfffffe00e3303600 __mtx_lock_spin_flags() at __mtx_lock_spin_flags+0xda/frame 0xfffffe00e3303640 smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0xca/frame 0xfffffe00e33036c0 pmap_invalidate_range() at pmap_invalidate_range+0x1b8/frame 0xfffffe00e3303740 vm_thread_stack_create() at vm_thread_stack_create+0x79/frame 0xfffffe00e3303880 kstack_import() at kstack_import+0x51/frame 0xfffffe00e33038c0 zone_alloc_item_locked() at zone_alloc_item_locked+0xae/frame 0xfffffe00e3303920 uma_zalloc_arg() at uma_zalloc_arg+0x10c/frame 0xfffffe00e3303960 vm_thread_new() at vm_thread_new+0x4b/frame 0xfffffe00e33039a0 thread_alloc() at thread_alloc+0x3a/frame 0xfffffe00e33039c0 thread_create() at thread_create+0x10f/frame 0xfffffe00e3303a30 sys_thr_new() at sys_thr_new+0xd2/frame 0xfffffe00e3303ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e3303bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e3303bf0 --- syscall (455, FreeBSD ELF64, sys_thr_new), rip = 0x8003054ca, rsp = 0x7fffffffe648, rbp = 0x7fffffffe730 --- Tracing command execve pid 79077 tid 100750 td 0xfffff806ff70f000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e55ec830 mi_switch() at mi_switch+0x182/frame 0xfffffe00e55ec870 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e55ec8b0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e55ec910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00e55ec950 _sleep() at _sleep+0x258/frame 0xfffffe00e55ec9f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe00e55eca80 sys_nanosleep() at sys_nanosleep+0x97/frame 0xfffffe00e55ecac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e55ecbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e55ecbf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003d3a7a, rsp = 0x7fffdfffdf38, rbp = 0x7fffdfffdf60 --- Tracing command execve pid 79077 tid 100892 td 0xfffff806ffc086e0 fork_trampoline() at fork_trampoline Tracing command execve pid 79076 tid 100566 td 0xfffff806b3d6c000 (CPU 4) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e197dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e197df0 trap() at trap+0x61/frame 0xfffffe003e197f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e197f20 --- trap 0x13, rip = 0xffffffff80c16110, rsp = 0xfffffe02f933f560, rbp = 0xfffffe02f933f570 --- lock_delay() at lock_delay+0x30/frame 0xfffffe02f933f570 _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0xd1/frame 0xfffffe02f933f600 __mtx_lock_spin_flags() at __mtx_lock_spin_flags+0xda/frame 0xfffffe02f933f640 smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0xca/frame 0xfffffe02f933f6c0 pmap_invalidate_range() at pmap_invalidate_range+0x1b8/frame 0xfffffe02f933f740 vm_thread_stack_create() at vm_thread_stack_create+0x79/frame 0xfffffe02f933f880 kstack_import() at kstack_import+0x51/frame 0xfffffe02f933f8c0 zone_alloc_item_locked() at zone_alloc_item_locked+0xae/frame 0xfffffe02f933f920 uma_zalloc_arg() at uma_zalloc_arg+0x10c/frame 0xfffffe02f933f960 vm_thread_new() at vm_thread_new+0x4b/frame 0xfffffe02f933f9a0 thread_alloc() at thread_alloc+0x3a/frame 0xfffffe02f933f9c0 thread_create() at thread_create+0x10f/frame 0xfffffe02f933fa30 sys_thr_new() at sys_thr_new+0xd2/frame 0xfffffe02f933fac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f933fbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f933fbf0 --- syscall (455, FreeBSD ELF64, sys_thr_new), rip = 0x8003054ca, rsp = 0x7fffffffe648, rbp = 0x7fffffffe730 --- Tracing command execve pid 79076 tid 100837 td 0xfffff806acb76000 fork_trampoline() at fork_trampoline Tracing command execve pid 79075 tid 100520 td 0xfffff806b3c5b6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3e3b830 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3e3b870 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e3e3b8b0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e3e3b910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00e3e3b950 _sleep() at _sleep+0x258/frame 0xfffffe00e3e3b9f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe00e3e3ba80 sys_nanosleep() at sys_nanosleep+0x97/frame 0xfffffe00e3e3bac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e3e3bbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e3e3bbf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003d3a7a, rsp = 0x7fffffffe6d8, rbp = 0x7fffffffe700 --- Tracing command execve pid 79075 tid 100609 td 0xfffff806bac72000 sched_switch() at sched_switch+0x38f/frame 0xfffffe02f9521830 mi_switch() at mi_switch+0x182/frame 0xfffffe02f9521870 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe02f95218b0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe02f9521910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe02f9521950 _sleep() at _sleep+0x258/frame 0xfffffe02f95219f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe02f9521a80 sys_nanosleep() at sys_nanosleep+0x97/frame 0xfffffe02f9521ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f9521bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f9521bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003d3a7a, rsp = 0x7fffdfffdf38, rbp = 0x7fffdfffdf60 --- Tracing command execve pid 79075 tid 100610 td 0xfffff806bae756e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3fd5830 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3fd5870 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e3fd58b0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e3fd5910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00e3fd5950 _sleep() at _sleep+0x258/frame 0xfffffe00e3fd59f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe00e3fd5a80 sys_nanosleep() at sys_nanosleep+0x97/frame 0xfffffe00e3fd5ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e3fd5bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e3fd5bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003d3a7a, rsp = 0x7fffdfdfcf38, rbp = 0x7fffdfdfcf60 --- Tracing command execve pid 79075 tid 100621 td 0xfffff806ba47a000 sched_switch() at sched_switch+0x38f/frame 0xfffffe02f9394830 mi_switch() at mi_switch+0x182/frame 0xfffffe02f9394870 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe02f93948b0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe02f9394910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe02f9394950 _sleep() at _sleep+0x258/frame 0xfffffe02f93949f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe02f9394a80 sys_nanosleep() at sys_nanosleep+0x97/frame 0xfffffe02f9394ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f9394bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f9394bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003d3a7a, rsp = 0x7fffdfbfbf38, rbp = 0x7fffdfbfbf60 --- Tracing command execve pid 79075 tid 100622 td 0xfffff801ef4786e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00addd6830 mi_switch() at mi_switch+0x182/frame 0xfffffe00addd6870 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00addd68b0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00addd6910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00addd6950 _sleep() at _sleep+0x258/frame 0xfffffe00addd69f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe00addd6a80 sys_nanosleep() at sys_nanosleep+0x97/frame 0xfffffe00addd6ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00addd6bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00addd6bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003d3a7a, rsp = 0x7fffdf9faf38, rbp = 0x7fffdf9faf60 --- Tracing command execve pid 79075 tid 100626 td 0xfffff806aca7f000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3e22830 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3e22870 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e3e228b0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e3e22910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00e3e22950 _sleep() at _sleep+0x258/frame 0xfffffe00e3e229f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe00e3e22a80 sys_nanosleep() at sys_nanosleep+0x97/frame 0xfffffe00e3e22ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e3e22bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e3e22bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003d3a7a, rsp = 0x7fffdf7f9f38, rbp = 0x7fffdf7f9f60 --- Tracing command execve pid 79074 tid 101103 td 0xfffff806ac2ca6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe02f93e4b60 mi_switch() at mi_switch+0x182/frame 0xfffffe02f93e4ba0 ast() at ast+0x2a7/frame 0xfffffe02f93e4bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe730 Tracing command execve pid 79074 tid 100607 td 0xfffff806b3c5d000 fork_trampoline() at fork_trampoline Tracing command execve pid 79074 tid 100733 td 0xfffff806ff06d000 fork_trampoline() at fork_trampoline Tracing command execve pid 79073 tid 101292 td 0xfffff801b5311000 sched_switch() at sched_switch+0x38f/frame 0xfffffe02f929a400 mi_switch() at mi_switch+0x182/frame 0xfffffe02f929a440 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe02f929a460 ipi_bitmap_handler() at ipi_bitmap_handler+0x9f/frame 0xfffffe02f929a490 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0xb3/frame 0xfffffe02f929a490 --- interrupt, rip = 0xffffffff80c16112, rsp = 0xfffffe02f929a560, rbp = 0xfffffe02f929a570 --- lock_delay() at lock_delay+0x32/frame 0xfffffe02f929a570 _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0xd1/frame 0xfffffe02f929a600 __mtx_lock_spin_flags() at __mtx_lock_spin_flags+0xda/frame 0xfffffe02f929a640 smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0xca/frame 0xfffffe02f929a6c0 pmap_invalidate_range() at pmap_invalidate_range+0x1b8/frame 0xfffffe02f929a740 vm_thread_stack_create() at vm_thread_stack_create+0x79/frame 0xfffffe02f929a880 kstack_import() at kstack_import+0x51/frame 0xfffffe02f929a8c0 zone_alloc_item_locked() at zone_alloc_item_locked+0xae/frame 0xfffffe02f929a920 uma_zalloc_arg() at uma_zalloc_arg+0x10c/frame 0xfffffe02f929a960 vm_thread_new() at vm_thread_new+0x4b/frame 0xfffffe02f929a9a0 thread_alloc() at thread_alloc+0x3a/frame 0xfffffe02f929a9c0 thread_create() at thread_create+0x10f/frame 0xfffffe02f929aa30 sys_thr_new() at sys_thr_new+0xd2/frame 0xfffffe02f929aac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f929abf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f929abf0 --- syscall (455, FreeBSD ELF64, sys_thr_new), rip = 0x8003054ca, rsp = 0x7fffffffe648, rbp = 0x7fffffffe730 --- Tracing command execve pid 79073 tid 100589 td 0xfffff806b3d50000 fork_trampoline() at fork_trampoline Tracing command execve pid 79072 tid 101314 td 0xfffff806ffd9a000 (CPU 6) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1afdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1afdf0 trap() at trap+0x61/frame 0xfffffe003e1aff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1aff20 --- trap 0x13, rip = 0xffffffff80c16112, rsp = 0xfffffe00e03fe560, rbp = 0xfffffe00e03fe570 --- lock_delay() at lock_delay+0x32/frame 0xfffffe00e03fe570 _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0xd1/frame 0xfffffe00e03fe600 __mtx_lock_spin_flags() at __mtx_lock_spin_flags+0xda/frame 0xfffffe00e03fe640 smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0xca/frame 0xfffffe00e03fe6c0 pmap_invalidate_range() at pmap_invalidate_range+0x1b8/frame 0xfffffe00e03fe740 vm_thread_stack_create() at vm_thread_stack_create+0x79/frame 0xfffffe00e03fe880 kstack_import() at kstack_import+0x51/frame 0xfffffe00e03fe8c0 zone_alloc_item_locked() at zone_alloc_item_locked+0xae/frame 0xfffffe00e03fe920 uma_zalloc_arg() at uma_zalloc_arg+0x10c/frame 0xfffffe00e03fe960 vm_thread_new() at vm_thread_new+0x4b/frame 0xfffffe00e03fe9a0 thread_alloc() at thread_alloc+0x3a/frame 0xfffffe00e03fe9c0 thread_create() at thread_create+0x10f/frame 0xfffffe00e03fea30 sys_thr_new() at sys_thr_new+0xd2/frame 0xfffffe00e03feac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e03febf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e03febf0 --- syscall (455, FreeBSD ELF64, sys_thr_new), rip = 0x8003054ca, rsp = 0x7fffffffe648, rbp = 0x7fffffffe730 --- Tracing command execve pid 79072 tid 100580 td 0xfffff806ba8766e0 fork_trampoline() at fork_trampoline Tracing command execve pid 79071 tid 101337 td 0xfffff806b3e406e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e55fbb60 mi_switch() at mi_switch+0x182/frame 0xfffffe00e55fbba0 ast() at ast+0x2a7/frame 0xfffffe00e55fbbf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe730 Tracing command execve pid 79071 tid 100579 td 0xfffff80755e206e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e1696830 mi_switch() at mi_switch+0x182/frame 0xfffffe00e1696870 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e16968b0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e1696910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00e1696950 _sleep() at _sleep+0x258/frame 0xfffffe00e16969f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe00e1696a80 sys_nanosleep() at sys_nanosleep+0x97/frame 0xfffffe00e1696ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e1696bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e1696bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003d3a7a, rsp = 0x7fffdfffdf38, rbp = 0x7fffdfffdf60 --- Tracing command execve pid 79071 tid 100878 td 0xfffff801ef81b6e0 fork_trampoline() at fork_trampoline Tracing command execve pid 79070 tid 101000 td 0xfffff806ba887000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3f71b60 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3f71ba0 ast() at ast+0x2a7/frame 0xfffffe00e3f71bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe730 Tracing command execve pid 79070 tid 100576 td 0xfffff801b579a000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3e27830 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3e27870 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e3e278b0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e3e27910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00e3e27950 _sleep() at _sleep+0x258/frame 0xfffffe00e3e279f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe00e3e27a80 sys_nanosleep() at sys_nanosleep+0x97/frame 0xfffffe00e3e27ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e3e27bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e3e27bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003d3a7a, rsp = 0x7fffdfffdf38, rbp = 0x7fffdfffdf60 --- Tracing command execve pid 79070 tid 100693 td 0xfffff806ba49b6e0 fork_trampoline() at fork_trampoline Tracing command execve pid 79069 tid 100212 td 0xfffff806b3a03000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e32bdb60 mi_switch() at mi_switch+0x182/frame 0xfffffe00e32bdba0 ast() at ast+0x2a7/frame 0xfffffe00e32bdbf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe730 Tracing command execve pid 79069 tid 100561 td 0xfffff806ff06d6e0 fork_trampoline() at fork_trampoline Tracing command execve pid 79069 tid 100684 td 0xfffff8002673e6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe02f9312830 mi_switch() at mi_switch+0x182/frame 0xfffffe02f9312870 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe02f93128b0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe02f9312910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe02f9312950 _sleep() at _sleep+0x258/frame 0xfffffe02f93129f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe02f9312a80 sys_nanosleep() at sys_nanosleep+0x97/frame 0xfffffe02f9312ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f9312bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f9312bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003d3a7a, rsp = 0x7fffdfdfcf38, rbp = 0x7fffdfdfcf60 --- Tracing command true pid 79068 tid 100422 td 0xfffff806bac7c000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e350ab20 mi_switch() at mi_switch+0x182/frame 0xfffffe00e350ab60 thread_suspend_check() at thread_suspend_check+0xd0/frame 0xfffffe00e350aba0 ast() at ast+0x54a/frame 0xfffffe00e350abf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe730 Tracing command true pid 79068 tid 100544 td 0xfffff806bac806e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe02f92a9910 mi_switch() at mi_switch+0x182/frame 0xfffffe02f92a9950 thread_suspend_switch() at thread_suspend_switch+0x10d/frame 0xfffffe02f92a99a0 thread_single() at thread_single+0x4bb/frame 0xfffffe02f92a9a00 post_execve() at post_execve+0x8e/frame 0xfffffe02f92a9a40 sys_execve() at sys_execve+0x5c/frame 0xfffffe02f92a9ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f92a9bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f92a9bf0 --- syscall (0, FreeBSD ELF64, nosys), rip = 0x800209000, rsp = 0x7fffffffee28, rbp = 0 --- Tracing command true pid 79068 tid 100550 td 0xfffff801b5ca26e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3268b20 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3268b60 thread_suspend_check() at thread_suspend_check+0xd0/frame 0xfffffe00e3268ba0 ast() at ast+0x54a/frame 0xfffffe00e3268bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffdf9faf60 Tracing command true pid 79068 tid 100552 td 0xfffff806ba27d000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e41ceb20 mi_switch() at mi_switch+0x182/frame 0xfffffe00e41ceb60 thread_suspend_check() at thread_suspend_check+0xd0/frame 0xfffffe00e41ceba0 ast() at ast+0x54a/frame 0xfffffe00e41cebf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffdf7f9f60 Tracing command execve pid 79067 tid 100831 td 0xfffff806ba29d000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e17e6930 mi_switch() at mi_switch+0x182/frame 0xfffffe00e17e6970 kern_yield() at kern_yield+0x73/frame 0xfffffe00e17e69b0 uiomove_faultflag() at uiomove_faultflag+0xee/frame 0xfffffe00e17e69f0 read_random_uio() at read_random_uio+0x118/frame 0xfffffe00e17e6a40 sys_getrandom() at sys_getrandom+0x78/frame 0xfffffe00e17e6ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e17e6bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e17e6bf0 --- syscall (563, FreeBSD ELF64, sys_getrandom), rip = 0x80043fdaa, rsp = 0x7fffffffe688, rbp = 0x7fffffffe6d0 --- Tracing command execve pid 79067 tid 100175 td 0xfffff806b3a176e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe02f949f830 mi_switch() at mi_switch+0x182/frame 0xfffffe02f949f870 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe02f949f8b0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe02f949f910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe02f949f950 _sleep() at _sleep+0x258/frame 0xfffffe02f949f9f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe02f949fa80 sys_nanosleep() at sys_nanosleep+0x97/frame 0xfffffe02f949fac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f949fbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f949fbf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003d3a7a, rsp = 0x7fffdfffdf38, rbp = 0x7fffdfffdf60 --- Tracing command execve pid 79067 tid 100492 td 0xfffff806ba879000 sched_switch() at sched_switch+0x38f/frame 0xfffffe02f9422830 mi_switch() at mi_switch+0x182/frame 0xfffffe02f9422870 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe02f94228b0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe02f9422910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe02f9422950 _sleep() at _sleep+0x258/frame 0xfffffe02f94229f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe02f9422a80 sys_nanosleep() at sys_nanosleep+0x97/frame 0xfffffe02f9422ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f9422bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f9422bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003d3a7a, rsp = 0x7fffdfdfcf38, rbp = 0x7fffdfdfcf60 --- Tracing command execve pid 79067 tid 100519 td 0xfffff804e61c56e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3209830 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3209870 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e32098b0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e3209910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00e3209950 _sleep() at _sleep+0x258/frame 0xfffffe00e32099f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe00e3209a80 sys_nanosleep() at sys_nanosleep+0x97/frame 0xfffffe00e3209ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e3209bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e3209bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003d3a7a, rsp = 0x7fffdfbfbf38, rbp = 0x7fffdfbfbf60 --- Tracing command execve pid 79067 tid 100530 td 0xfffff806ba49b000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e401b830 mi_switch() at mi_switch+0x182/frame 0xfffffe00e401b870 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e401b8b0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e401b910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00e401b950 _sleep() at _sleep+0x258/frame 0xfffffe00e401b9f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe00e401ba80 sys_nanosleep() at sys_nanosleep+0x97/frame 0xfffffe00e401bac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e401bbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e401bbf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003d3a7a, rsp = 0x7fffdf9faf38, rbp = 0x7fffdf9faf60 --- Tracing command execve pid 79067 tid 100531 td 0xfffff80599f36000 sched_switch() at sched_switch+0x38f/frame 0xfffffe02f9371830 mi_switch() at mi_switch+0x182/frame 0xfffffe02f9371870 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe02f93718b0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe02f9371910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe02f9371950 _sleep() at _sleep+0x258/frame 0xfffffe02f93719f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe02f9371a80 sys_nanosleep() at sys_nanosleep+0x97/frame 0xfffffe02f9371ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f9371bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f9371bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003d3a7a, rsp = 0x7fffdf7f9f38, rbp = 0x7fffdf7f9f60 --- Tracing command sh pid 79031 tid 100602 td 0xfffff801ef3ba000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e41d3690 mi_switch() at mi_switch+0x182/frame 0xfffffe00e41d36d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e41d3710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e41d3770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e41d37a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e41d3840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00e41d38d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00e41d3ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e41d3bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e41d3bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80040791a, rsp = 0x7fffffffdd98, rbp = 0x7fffffffde50 --- Tracing command cron pid 78999 tid 100164 td 0xfffff806ba27c000 (CPU 8) kdb_enter() at kdb_enter+0x37/frame 0xfffffe02f94a4500 vpanic() at vpanic+0x19a/frame 0xfffffe02f94a4560 panic() at panic+0x43/frame 0xfffffe02f94a45c0 trap_fatal() at trap_fatal+0x386/frame 0xfffffe02f94a4620 trap_pfault() at trap_pfault+0x99/frame 0xfffffe02f94a46a0 trap() at trap+0x2c0/frame 0xfffffe02f94a47d0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a47d0 --- trap 0xc, rip = 0xffffffff80f01279, rsp = 0xfffffe02f94a48a0, rbp = 0xfffffe02f94a48d0 --- swap_pager_copy() at swap_pager_copy+0xf9/frame 0xfffffe02f94a48d0 vm_object_collapse() at vm_object_collapse+0x181/frame 0xfffffe02f94a4960 vm_object_deallocate() at vm_object_deallocate+0x41d/frame 0xfffffe02f94a49b0 vm_map_process_deferred() at vm_map_process_deferred+0xad/frame 0xfffffe02f94a49d0 vm_map_remove() at vm_map_remove+0xeb/frame 0xfffffe02f94a4a00 vmspace_exit() at vmspace_exit+0xd5/frame 0xfffffe02f94a4a40 exit1() at exit1+0x5a2/frame 0xfffffe02f94a4ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe02f94a4ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f94a4bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f94a4bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8003d880a, rsp = 0x7fffffffeb08, rbp = 0x7fffffffeb30 --- Tracing command sh pid 77773 tid 100821 td 0xfffff806b3a99000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e35d77d0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e35d7810 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e35d7850 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e35d78b0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e35d78e0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e35d7980 pipe_read() at pipe_read+0x428/frame 0xfffffe00e35d79e0 dofileread() at dofileread+0x81/frame 0xfffffe00e35d7a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e35d7a70 sys_read() at sys_read+0x83/frame 0xfffffe00e35d7ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e35d7bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e35d7bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800476b1a, rsp = 0x7fffffffdfe8, rbp = 0x7fffffffe130 --- Tracing command execve pid 76488 tid 100260 td 0xfffff801b52e7000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3286b60 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3286ba0 ast() at ast+0x2a7/frame 0xfffffe00e3286bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe770 Tracing command swap pid 75355 tid 102209 td 0xfffff806ba67e000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3f35b60 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3f35ba0 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00e3f35bc0 ipi_bitmap_handler() at ipi_bitmap_handler+0x9f/frame 0xfffffe00e3f35bf0 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0xb3/frame 0xfffffe00e3f35bf0 --- interrupt, rip = 0x203687, rsp = 0x7fffffffe5a0, rbp = 0x7fffffffe5c0 --- Tracing command swap pid 75354 tid 100317 td 0xfffff801b5a19000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3371b60 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3371ba0 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00e3371bc0 ipi_bitmap_handler() at ipi_bitmap_handler+0x9f/frame 0xfffffe00e3371bf0 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0xb3/frame 0xfffffe00e3371bf0 --- interrupt, rip = 0x203687, rsp = 0x7fffffffe5a0, rbp = 0x7fffffffe5c0 --- Tracing command swap pid 75353 tid 102157 td 0xfffff806ba29c000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3e45650 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3e45690 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e3e456d0 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00e3e45710 _sleep() at _sleep+0x26c/frame 0xfffffe00e3e457b0 swap_pager_getpages() at swap_pager_getpages+0x508/frame 0xfffffe00e3e45840 vm_pager_get_pages() at vm_pager_get_pages+0x50/frame 0xfffffe00e3e45890 vm_fault() at vm_fault+0x896/frame 0xfffffe00e3e45a00 vm_fault_trap() at vm_fault_trap+0x6e/frame 0xfffffe00e3e45a40 trap_pfault() at trap_pfault+0x1f3/frame 0xfffffe00e3e45ac0 trap() at trap+0x470/frame 0xfffffe00e3e45bf0 calltrap() at calltrap+0x8/frame 0xfffffe00e3e45bf0 --- trap 0xc, rip = 0x203689, rsp = 0x7fffffffe5a0, rbp = 0x7fffffffe5c0 --- Tracing command swap pid 75352 tid 100229 td 0xfffff801b539d6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e17f0b60 mi_switch() at mi_switch+0x182/frame 0xfffffe00e17f0ba0 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00e17f0bc0 ipi_bitmap_handler() at ipi_bitmap_handler+0x9f/frame 0xfffffe00e17f0bf0 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0xb3/frame 0xfffffe00e17f0bf0 --- interrupt, rip = 0x203687, rsp = 0x7fffffffe5a0, rbp = 0x7fffffffe5c0 --- Tracing command swap pid 75351 tid 102206 td 0xfffff806ba67d6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3f12b60 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3f12ba0 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00e3f12bc0 ipi_bitmap_handler() at ipi_bitmap_handler+0x9f/frame 0xfffffe00e3f12bf0 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0xb3/frame 0xfffffe00e3f12bf0 --- interrupt, rip = 0x203687, rsp = 0x7fffffffe5a0, rbp = 0x7fffffffe5c0 --- Tracing command swap pid 75350 tid 102210 td 0xfffff806ba67f6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3f30b60 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3f30ba0 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00e3f30bc0 ipi_bitmap_handler() at ipi_bitmap_handler+0x9f/frame 0xfffffe00e3f30bf0 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0xb3/frame 0xfffffe00e3f30bf0 --- interrupt, rip = 0x203687, rsp = 0x7fffffffe5a0, rbp = 0x7fffffffe5c0 --- Tracing command swap pid 75349 tid 102213 td 0xfffff806ba680000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3f53b60 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3f53ba0 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00e3f53bc0 ipi_bitmap_handler() at ipi_bitmap_handler+0x9f/frame 0xfffffe00e3f53bf0 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0xb3/frame 0xfffffe00e3f53bf0 --- interrupt, rip = 0x203687, rsp = 0x7fffffffe5a0, rbp = 0x7fffffffe5c0 --- Tracing command swap pid 75348 tid 100466 td 0xfffff802c07826e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e34e8b60 mi_switch() at mi_switch+0x182/frame 0xfffffe00e34e8ba0 ast() at ast+0x2a7/frame 0xfffffe00e34e8bf0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe290 Tracing command swap pid 75347 tid 102238 td 0xfffff806ba8796e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3fd0b60 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3fd0ba0 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00e3fd0bc0 ipi_bitmap_handler() at ipi_bitmap_handler+0x9f/frame 0xfffffe00e3fd0bf0 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0xb3/frame 0xfffffe00e3fd0bf0 --- interrupt, rip = 0x203689, rsp = 0x7fffffffe5a0, rbp = 0x7fffffffe5c0 --- Tracing command swap pid 75346 tid 102251 td 0xfffff806baa78000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3ff3b60 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3ff3ba0 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00e3ff3bc0 ipi_bitmap_handler() at ipi_bitmap_handler+0x9f/frame 0xfffffe00e3ff3bf0 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0xb3/frame 0xfffffe00e3ff3bf0 --- interrupt, rip = 0x203687, rsp = 0x7fffffffe5a0, rbp = 0x7fffffffe5c0 --- Tracing command swap pid 75345 tid 102287 td 0xfffff806bac7f000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e40cfb60 mi_switch() at mi_switch+0x182/frame 0xfffffe00e40cfba0 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00e40cfbc0 ipi_bitmap_handler() at ipi_bitmap_handler+0x9f/frame 0xfffffe00e40cfbf0 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0xb3/frame 0xfffffe00e40cfbf0 --- interrupt, rip = 0x203687, rsp = 0x7fffffffe5a0, rbp = 0x7fffffffe5c0 --- Tracing command swap pid 75344 tid 102302 td 0xfffff806bae776e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e411ab60 mi_switch() at mi_switch+0x182/frame 0xfffffe00e411aba0 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00e411abc0 ipi_bitmap_handler() at ipi_bitmap_handler+0x9f/frame 0xfffffe00e411abf0 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0xb3/frame 0xfffffe00e411abf0 --- interrupt, rip = 0x203687, rsp = 0x7fffffffe5a0, rbp = 0x7fffffffe5c0 --- Tracing command swap pid 75343 tid 102200 td 0xfffff806ba67a6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3efe690 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe02f94a3b60 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe02f94a3b70 trap() at trap+0x73/frame 0xfffffe02f94a3ca0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a3ca0 --- trap 0xc, rip = 0xffffffff810400a0, rsp = 0xfffffe02f94a3d70, rbp = 0xfffffe02f94a3e00 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe02f94a3e00 db_get_value() at db_get_value+0x32/frame 0xfffffe02f94a3e40 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe02f94a3ed0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe02f94a3ef0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe02f94a3f90 db_command() at db_command+0x25c/frame 0xfffffe02f94a4060 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe02f94a4100 db_command() at db_command+0x25c/frame 0xfffffe02f94a41d0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe02f94a41e0 db_trap() at db_trap+0xe8/frame 0xfffffe02f94a4270 kdb_trap() at kdb_trap+0x157/frame 0xfffffe02f94a42f0 trap() at trap+0x7d4/frame 0xfffffe02f94a4420 calltrap() at calltrap+0x8/frame 0xfffffe02f94a4420 --- trap 0x3, rip = 0xffffffff80c14af7, rsp = 0xfffffe02f94a44f0, rbp = 0xfffffe02f94a4500 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe02f94a4500 vpanic() at vpanic+0x19a/frame 0xfffffe02f94a4560 panic() at panic+0x43/frame 0xfffffe02f94a45c0 trap_fatal() at trap_fatal+0x386/frame 0xfffffe02f94a4620 trap_pfault() at trap_pfault+0x99/frame 0xfffffe02f94a46a0 trap() at trap+0x2c0/frame 0xfffffe02f94a47d0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a47d0 --- trap 0xc, rip = 0xffffffff80f01279, rsp = 0xfffffe02f94a48a0, rbp = 0xfffffe02f94a48d0 --- swap_pager_copy() at swap_pager_copy+0xf9/frame 0xfffffe02f94a48d0 vm_object_collapse() at vm_object_collapse+0x181/frame 0xfffffe02f94a4960 vm_object_deallocate() at vm_object_deallocate+0x41d/frame 0xfffffe02f94a49b0 vm_map_process_deferred() at vm_map_process_deferred+0xad/frame 0xfffffe02f94a49d0 vm_map_remove() at vm_map_remove+0xeb/frame 0xfffffe02f94a4a00 vmspace_exit() at vmspace_exit+0xd5/frame 0xfffffe02f94a4a40 exit1() at exit1+0x5a2/frame 0xfffffe02f94a4ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe02f94a4ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f94a4bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f94a4bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8003d880a, rsp = 0x7fffffffeb08, rbp = 0x7fffffffeb30 --- *** error reading from address fffffe00e3efe698 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe02f94a3df0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe02f94a3e00 db_get_value() at db_get_value+0x4c/frame 0xfffffe02f94a3e40 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe02f94a3ed0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe02f94a3ef0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe02f94a3f90 db_command() at db_command+0x25c/frame 0xfffffe02f94a4060 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe02f94a4100 db_command() at db_command+0x25c/frame 0xfffffe02f94a41d0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe02f94a41e0 db_trap() at db_trap+0xe8/frame 0xfffffe02f94a4270 kdb_trap() at kdb_trap+0x157/frame 0xfffffe02f94a42f0 trap() at trap+0x7d4/frame 0xfffffe02f94a4420 calltrap() at calltrap+0x8/frame 0xfffffe02f94a4420 --- trap 0x3, rip = 0xffffffff80c14af7, rsp = 0xfffffe02f94a44f0, rbp = 0xfffffe02f94a4500 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe02f94a4500 vpanic() at vpanic+0x19a/frame 0xfffffe02f94a4560 panic() at panic+0x43/frame 0xfffffe02f94a45c0 trap_fatal() at trap_fatal+0x386/frame 0xfffffe02f94a4620 trap_pfault() at trap_pfault+0x99/frame 0xfffffe02f94a46a0 trap() at trap+0x2c0/frame 0xfffffe02f94a47d0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a47d0 --- trap 0xc, rip = 0xffffffff80f01279, rsp = 0xfffffe02f94a48a0, rbp = 0xfffffe02f94a48d0 --- swap_pager_copy() at swap_pager_copy+0xf9/frame 0xfffffe02f94a48d0 vm_object_collapse() at vm_object_collapse+0x181/frame 0xfffffe02f94a4960 vm_object_deallocate() at vm_object_deallocate+0x41d/frame 0xfffffe02f94a49b0 vm_map_process_deferred() at vm_map_process_deferred+0xad/frame 0xfffffe02f94a49d0 vm_map_remove() at vm_map_remove+0xeb/frame 0xfffffe02f94a4a00 vmspace_exit() at vmspace_exit+0xd5/frame 0xfffffe02f94a4a40 exit1() at exit1+0x5a2/frame 0xfffffe02f94a4ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe02f94a4ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f94a4bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f94a4bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8003d880a, rsp = 0x7fffffffeb08, rbp = 0x7fffffffeb30 --- Tracing command swap pid 75340 tid 102152 td 0xfffff806ba2966e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3e2c830 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3e2c870 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e3e2c8b0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e3e2c910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00e3e2c950 _sleep() at _sleep+0x258/frame 0xfffffe00e3e2c9f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe00e3e2ca80 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe00e3e2cac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e3e2cbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e3e2cbf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003c3a7a, rsp = 0x7fffffffe5d8, rbp = 0x7fffffffe620 --- Tracing command sh pid 75328 tid 100145 td 0xfffff80004122000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e03c7690 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe02f94a3b60 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe02f94a3b70 trap() at trap+0x73/frame 0xfffffe02f94a3ca0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a3ca0 --- trap 0xc, rip = 0xffffffff810400a0, rsp = 0xfffffe02f94a3d70, rbp = 0xfffffe02f94a3e00 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe02f94a3e00 db_get_value() at db_get_value+0x32/frame 0xfffffe02f94a3e40 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe02f94a3ed0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe02f94a3ef0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe02f94a3f90 db_command() at db_command+0x25c/frame 0xfffffe02f94a4060 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe02f94a4100 db_command() at db_command+0x25c/frame 0xfffffe02f94a41d0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe02f94a41e0 db_trap() at db_trap+0xe8/frame 0xfffffe02f94a4270 kdb_trap() at kdb_trap+0x157/frame 0xfffffe02f94a42f0 trap() at trap+0x7d4/frame 0xfffffe02f94a4420 calltrap() at calltrap+0x8/frame 0xfffffe02f94a4420 --- trap 0x3, rip = 0xffffffff80c14af7, rsp = 0xfffffe02f94a44f0, rbp = 0xfffffe02f94a4500 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe02f94a4500 vpanic() at vpanic+0x19a/frame 0xfffffe02f94a4560 panic() at panic+0x43/frame 0xfffffe02f94a45c0 trap_fatal() at trap_fatal+0x386/frame 0xfffffe02f94a4620 trap_pfault() at trap_pfault+0x99/frame 0xfffffe02f94a46a0 trap() at trap+0x2c0/frame 0xfffffe02f94a47d0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a47d0 --- trap 0xc, rip = 0xffffffff80f01279, rsp = 0xfffffe02f94a48a0, rbp = 0xfffffe02f94a48d0 --- swap_pager_copy() at swap_pager_copy+0xf9/frame 0xfffffe02f94a48d0 vm_object_collapse() at vm_object_collapse+0x181/frame 0xfffffe02f94a4960 vm_object_deallocate() at vm_object_deallocate+0x41d/frame 0xfffffe02f94a49b0 vm_map_process_deferred() at vm_map_process_deferred+0xad/frame 0xfffffe02f94a49d0 vm_map_remove() at vm_map_remove+0xeb/frame 0xfffffe02f94a4a00 vmspace_exit() at vmspace_exit+0xd5/frame 0xfffffe02f94a4a40 exit1() at exit1+0x5a2/frame 0xfffffe02f94a4ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe02f94a4ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f94a4bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f94a4bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8003d880a, rsp = 0x7fffffffeb08, rbp = 0x7fffffffeb30 --- *** error reading from address fffffe00e03c7698 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe02f94a3df0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe02f94a3e00 db_get_value() at db_get_value+0x4c/frame 0xfffffe02f94a3e40 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe02f94a3ed0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe02f94a3ef0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe02f94a3f90 db_command() at db_command+0x25c/frame 0xfffffe02f94a4060 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe02f94a4100 db_command() at db_command+0x25c/frame 0xfffffe02f94a41d0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe02f94a41e0 db_trap() at db_trap+0xe8/frame 0xfffffe02f94a4270 kdb_trap() at kdb_trap+0x157/frame 0xfffffe02f94a42f0 trap() at trap+0x7d4/frame 0xfffffe02f94a4420 calltrap() at calltrap+0x8/frame 0xfffffe02f94a4420 --- trap 0x3, rip = 0xffffffff80c14af7, rsp = 0xfffffe02f94a44f0, rbp = 0xfffffe02f94a4500 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe02f94a4500 vpanic() at vpanic+0x19a/frame 0xfffffe02f94a4560 panic() at panic+0x43/frame 0xfffffe02f94a45c0 trap_fatal() at trap_fatal+0x386/frame 0xfffffe02f94a4620 trap_pfault() at trap_pfault+0x99/frame 0xfffffe02f94a46a0 trap() at trap+0x2c0/frame 0xfffffe02f94a47d0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a47d0 --- trap 0xc, rip = 0xffffffff80f01279, rsp = 0xfffffe02f94a48a0, rbp = 0xfffffe02f94a48d0 --- swap_pager_copy() at swap_pager_copy+0xf9/frame 0xfffffe02f94a48d0 vm_object_collapse() at vm_object_collapse+0x181/frame 0xfffffe02f94a4960 vm_object_deallocate() at vm_object_deallocate+0x41d/frame 0xfffffe02f94a49b0 vm_map_process_deferred() at vm_map_process_deferred+0xad/frame 0xfffffe02f94a49d0 vm_map_remove() at vm_map_remove+0xeb/frame 0xfffffe02f94a4a00 vmspace_exit() at vmspace_exit+0xd5/frame 0xfffffe02f94a4a40 exit1() at exit1+0x5a2/frame 0xfffffe02f94a4ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe02f94a4ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f94a4bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f94a4bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8003d880a, rsp = 0x7fffffffeb08, rbp = 0x7fffffffeb30 --- Tracing command tee pid 75327 tid 102241 td 0xfffff806ba87a000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3fc17d0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e3fc1810 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e3fc1850 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e3fc18b0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e3fc18e0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e3fc1980 pipe_read() at pipe_read+0x428/frame 0xfffffe00e3fc19e0 dofileread() at dofileread+0x81/frame 0xfffffe00e3fc1a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e3fc1a70 sys_read() at sys_read+0x83/frame 0xfffffe00e3fc1ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e3fc1bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e3fc1bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800417b1a, rsp = 0x7fffffffe738, rbp = 0x7fffffffe780 --- Tracing command sh pid 75326 tid 102170 td 0xfffff806ba2846e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e3e68690 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe02f94a3b60 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe02f94a3b70 trap() at trap+0x73/frame 0xfffffe02f94a3ca0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a3ca0 --- trap 0xc, rip = 0xffffffff810400a0, rsp = 0xfffffe02f94a3d70, rbp = 0xfffffe02f94a3e00 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe02f94a3e00 db_get_value() at db_get_value+0x32/frame 0xfffffe02f94a3e40 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe02f94a3ed0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe02f94a3ef0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe02f94a3f90 db_command() at db_command+0x25c/frame 0xfffffe02f94a4060 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe02f94a4100 db_command() at db_command+0x25c/frame 0xfffffe02f94a41d0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe02f94a41e0 db_trap() at db_trap+0xe8/frame 0xfffffe02f94a4270 kdb_trap() at kdb_trap+0x157/frame 0xfffffe02f94a42f0 trap() at trap+0x7d4/frame 0xfffffe02f94a4420 calltrap() at calltrap+0x8/frame 0xfffffe02f94a4420 --- trap 0x3, rip = 0xffffffff80c14af7, rsp = 0xfffffe02f94a44f0, rbp = 0xfffffe02f94a4500 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe02f94a4500 vpanic() at vpanic+0x19a/frame 0xfffffe02f94a4560 panic() at panic+0x43/frame 0xfffffe02f94a45c0 trap_fatal() at trap_fatal+0x386/frame 0xfffffe02f94a4620 trap_pfault() at trap_pfault+0x99/frame 0xfffffe02f94a46a0 trap() at trap+0x2c0/frame 0xfffffe02f94a47d0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a47d0 --- trap 0xc, rip = 0xffffffff80f01279, rsp = 0xfffffe02f94a48a0, rbp = 0xfffffe02f94a48d0 --- swap_pager_copy() at swap_pager_copy+0xf9/frame 0xfffffe02f94a48d0 vm_object_collapse() at vm_object_collapse+0x181/frame 0xfffffe02f94a4960 vm_object_deallocate() at vm_object_deallocate+0x41d/frame 0xfffffe02f94a49b0 vm_map_process_deferred() at vm_map_process_deferred+0xad/frame 0xfffffe02f94a49d0 vm_map_remove() at vm_map_remove+0xeb/frame 0xfffffe02f94a4a00 vmspace_exit() at vmspace_exit+0xd5/frame 0xfffffe02f94a4a40 exit1() at exit1+0x5a2/frame 0xfffffe02f94a4ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe02f94a4ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f94a4bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f94a4bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8003d880a, rsp = 0x7fffffffeb08, rbp = 0x7fffffffeb30 --- *** error reading from address fffffe00e3e68698 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe02f94a3df0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe02f94a3e00 db_get_value() at db_get_value+0x4c/frame 0xfffffe02f94a3e40 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe02f94a3ed0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe02f94a3ef0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe02f94a3f90 db_command() at db_command+0x25c/frame 0xfffffe02f94a4060 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe02f94a4100 db_command() at db_command+0x25c/frame 0xfffffe02f94a41d0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe02f94a41e0 db_trap() at db_trap+0xe8/frame 0xfffffe02f94a4270 kdb_trap() at kdb_trap+0x157/frame 0xfffffe02f94a42f0 trap() at trap+0x7d4/frame 0xfffffe02f94a4420 calltrap() at calltrap+0x8/frame 0xfffffe02f94a4420 --- trap 0x3, rip = 0xffffffff80c14af7, rsp = 0xfffffe02f94a44f0, rbp = 0xfffffe02f94a4500 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe02f94a4500 vpanic() at vpanic+0x19a/frame 0xfffffe02f94a4560 panic() at panic+0x43/frame 0xfffffe02f94a45c0 trap_fatal() at trap_fatal+0x386/frame 0xfffffe02f94a4620 trap_pfault() at trap_pfault+0x99/frame 0xfffffe02f94a46a0 trap() at trap+0x2c0/frame 0xfffffe02f94a47d0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a47d0 --- trap 0xc, rip = 0xffffffff80f01279, rsp = 0xfffffe02f94a48a0, rbp = 0xfffffe02f94a48d0 --- swap_pager_copy() at swap_pager_copy+0xf9/frame 0xfffffe02f94a48d0 vm_object_collapse() at vm_object_collapse+0x181/frame 0xfffffe02f94a4960 vm_object_deallocate() at vm_object_deallocate+0x41d/frame 0xfffffe02f94a49b0 vm_map_process_deferred() at vm_map_process_deferred+0xad/frame 0xfffffe02f94a49d0 vm_map_remove() at vm_map_remove+0xeb/frame 0xfffffe02f94a4a00 vmspace_exit() at vmspace_exit+0xd5/frame 0xfffffe02f94a4a40 exit1() at exit1+0x5a2/frame 0xfffffe02f94a4ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe02f94a4ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f94a4bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f94a4bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8003d880a, rsp = 0x7fffffffeb08, rbp = 0x7fffffffeb30 --- Tracing command sh pid 8322 tid 100396 td 0xfffff80026ee4000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e16a5690 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe02f94a3b60 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe02f94a3b70 trap() at trap+0x73/frame 0xfffffe02f94a3ca0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a3ca0 --- trap 0xc, rip = 0xffffffff810400a0, rsp = 0xfffffe02f94a3d70, rbp = 0xfffffe02f94a3e00 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe02f94a3e00 db_get_value() at db_get_value+0x32/frame 0xfffffe02f94a3e40 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe02f94a3ed0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe02f94a3ef0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe02f94a3f90 db_command() at db_command+0x25c/frame 0xfffffe02f94a4060 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe02f94a4100 db_command() at db_command+0x25c/frame 0xfffffe02f94a41d0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe02f94a41e0 db_trap() at db_trap+0xe8/frame 0xfffffe02f94a4270 kdb_trap() at kdb_trap+0x157/frame 0xfffffe02f94a42f0 trap() at trap+0x7d4/frame 0xfffffe02f94a4420 calltrap() at calltrap+0x8/frame 0xfffffe02f94a4420 --- trap 0x3, rip = 0xffffffff80c14af7, rsp = 0xfffffe02f94a44f0, rbp = 0xfffffe02f94a4500 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe02f94a4500 vpanic() at vpanic+0x19a/frame 0xfffffe02f94a4560 panic() at panic+0x43/frame 0xfffffe02f94a45c0 trap_fatal() at trap_fatal+0x386/frame 0xfffffe02f94a4620 trap_pfault() at trap_pfault+0x99/frame 0xfffffe02f94a46a0 trap() at trap+0x2c0/frame 0xfffffe02f94a47d0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a47d0 --- trap 0xc, rip = 0xffffffff80f01279, rsp = 0xfffffe02f94a48a0, rbp = 0xfffffe02f94a48d0 --- swap_pager_copy() at swap_pager_copy+0xf9/frame 0xfffffe02f94a48d0 vm_object_collapse() at vm_object_collapse+0x181/frame 0xfffffe02f94a4960 vm_object_deallocate() at vm_object_deallocate+0x41d/frame 0xfffffe02f94a49b0 vm_map_process_deferred() at vm_map_process_deferred+0xad/frame 0xfffffe02f94a49d0 vm_map_remove() at vm_map_remove+0xeb/frame 0xfffffe02f94a4a00 vmspace_exit() at vmspace_exit+0xd5/frame 0xfffffe02f94a4a40 exit1() at exit1+0x5a2/frame 0xfffffe02f94a4ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe02f94a4ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f94a4bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f94a4bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8003d880a, rsp = 0x7fffffffeb08, rbp = 0x7fffffffeb30 --- *** error reading from address fffffe00e16a5698 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe02f94a3df0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe02f94a3e00 db_get_value() at db_get_value+0x4c/frame 0xfffffe02f94a3e40 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe02f94a3ed0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe02f94a3ef0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe02f94a3f90 db_command() at db_command+0x25c/frame 0xfffffe02f94a4060 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe02f94a4100 db_command() at db_command+0x25c/frame 0xfffffe02f94a41d0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe02f94a41e0 db_trap() at db_trap+0xe8/frame 0xfffffe02f94a4270 kdb_trap() at kdb_trap+0x157/frame 0xfffffe02f94a42f0 trap() at trap+0x7d4/frame 0xfffffe02f94a4420 calltrap() at calltrap+0x8/frame 0xfffffe02f94a4420 --- trap 0x3, rip = 0xffffffff80c14af7, rsp = 0xfffffe02f94a44f0, rbp = 0xfffffe02f94a4500 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe02f94a4500 vpanic() at vpanic+0x19a/frame 0xfffffe02f94a4560 panic() at panic+0x43/frame 0xfffffe02f94a45c0 trap_fatal() at trap_fatal+0x386/frame 0xfffffe02f94a4620 trap_pfault() at trap_pfault+0x99/frame 0xfffffe02f94a46a0 trap() at trap+0x2c0/frame 0xfffffe02f94a47d0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a47d0 --- trap 0xc, rip = 0xffffffff80f01279, rsp = 0xfffffe02f94a48a0, rbp = 0xfffffe02f94a48d0 --- swap_pager_copy() at swap_pager_copy+0xf9/frame 0xfffffe02f94a48d0 vm_object_collapse() at vm_object_collapse+0x181/frame 0xfffffe02f94a4960 vm_object_deallocate() at vm_object_deallocate+0x41d/frame 0xfffffe02f94a49b0 vm_map_process_deferred() at vm_map_process_deferred+0xad/frame 0xfffffe02f94a49d0 vm_map_remove() at vm_map_remove+0xeb/frame 0xfffffe02f94a4a00 vmspace_exit() at vmspace_exit+0xd5/frame 0xfffffe02f94a4a40 exit1() at exit1+0x5a2/frame 0xfffffe02f94a4ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe02f94a4ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f94a4bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f94a4bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8003d880a, rsp = 0x7fffffffeb08, rbp = 0x7fffffffeb30 --- Tracing command sh pid 2472 tid 100147 td 0xfffff80026432000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e03d17d0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e03d1810 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e03d1850 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e03d18b0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e03d18e0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e03d1980 pipe_read() at pipe_read+0x428/frame 0xfffffe00e03d19e0 dofileread() at dofileread+0x81/frame 0xfffffe00e03d1a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e03d1a70 sys_read() at sys_read+0x83/frame 0xfffffe00e03d1ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e03d1bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e03d1bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800476b1a, rsp = 0x7fffffffe2c8, rbp = 0x7fffffffe3c0 --- Tracing command awk pid 2470 tid 100216 td 0xfffff80026ef26e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e17af7d0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e17af810 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e17af850 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e17af8b0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e17af8e0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e17af980 pipe_read() at pipe_read+0x428/frame 0xfffffe00e17af9e0 dofileread() at dofileread+0x81/frame 0xfffffe00e17afa30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e17afa70 sys_read() at sys_read+0x83/frame 0xfffffe00e17afac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e17afbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e17afbf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80047bb1a, rsp = 0x7fffffffe3b8, rbp = 0x7fffffffe3d0 --- Tracing command sh pid 2469 tid 100217 td 0xfffff801b52b96e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e17b4690 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe02f94a3b60 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe02f94a3b70 trap() at trap+0x73/frame 0xfffffe02f94a3ca0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a3ca0 --- trap 0xc, rip = 0xffffffff810400a0, rsp = 0xfffffe02f94a3d70, rbp = 0xfffffe02f94a3e00 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe02f94a3e00 db_get_value() at db_get_value+0x32/frame 0xfffffe02f94a3e40 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe02f94a3ed0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe02f94a3ef0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe02f94a3f90 db_command() at db_command+0x25c/frame 0xfffffe02f94a4060 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe02f94a4100 db_command() at db_command+0x25c/frame 0xfffffe02f94a41d0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe02f94a41e0 db_trap() at db_trap+0xe8/frame 0xfffffe02f94a4270 kdb_trap() at kdb_trap+0x157/frame 0xfffffe02f94a42f0 trap() at trap+0x7d4/frame 0xfffffe02f94a4420 calltrap() at calltrap+0x8/frame 0xfffffe02f94a4420 --- trap 0x3, rip = 0xffffffff80c14af7, rsp = 0xfffffe02f94a44f0, rbp = 0xfffffe02f94a4500 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe02f94a4500 vpanic() at vpanic+0x19a/frame 0xfffffe02f94a4560 panic() at panic+0x43/frame 0xfffffe02f94a45c0 trap_fatal() at trap_fatal+0x386/frame 0xfffffe02f94a4620 trap_pfault() at trap_pfault+0x99/frame 0xfffffe02f94a46a0 trap() at trap+0x2c0/frame 0xfffffe02f94a47d0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a47d0 --- trap 0xc, rip = 0xffffffff80f01279, rsp = 0xfffffe02f94a48a0, rbp = 0xfffffe02f94a48d0 --- swap_pager_copy() at swap_pager_copy+0xf9/frame 0xfffffe02f94a48d0 vm_object_collapse() at vm_object_collapse+0x181/frame 0xfffffe02f94a4960 vm_object_deallocate() at vm_object_deallocate+0x41d/frame 0xfffffe02f94a49b0 vm_map_process_deferred() at vm_map_process_deferred+0xad/frame 0xfffffe02f94a49d0 vm_map_remove() at vm_map_remove+0xeb/frame 0xfffffe02f94a4a00 vmspace_exit() at vmspace_exit+0xd5/frame 0xfffffe02f94a4a40 exit1() at exit1+0x5a2/frame 0xfffffe02f94a4ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe02f94a4ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f94a4bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f94a4bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8003d880a, rsp = 0x7fffffffeb08, rbp = 0x7fffffffeb30 --- *** error reading from address fffffe00e17b4698 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe02f94a3df0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe02f94a3e00 db_get_value() at db_get_value+0x4c/frame 0xfffffe02f94a3e40 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe02f94a3ed0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe02f94a3ef0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe02f94a3f90 db_command() at db_command+0x25c/frame 0xfffffe02f94a4060 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe02f94a4100 db_command() at db_command+0x25c/frame 0xfffffe02f94a41d0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe02f94a41e0 db_trap() at db_trap+0xe8/frame 0xfffffe02f94a4270 kdb_trap() at kdb_trap+0x157/frame 0xfffffe02f94a42f0 trap() at trap+0x7d4/frame 0xfffffe02f94a4420 calltrap() at calltrap+0x8/frame 0xfffffe02f94a4420 --- trap 0x3, rip = 0xffffffff80c14af7, rsp = 0xfffffe02f94a44f0, rbp = 0xfffffe02f94a4500 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe02f94a4500 vpanic() at vpanic+0x19a/frame 0xfffffe02f94a4560 panic() at panic+0x43/frame 0xfffffe02f94a45c0 trap_fatal() at trap_fatal+0x386/frame 0xfffffe02f94a4620 trap_pfault() at trap_pfault+0x99/frame 0xfffffe02f94a46a0 trap() at trap+0x2c0/frame 0xfffffe02f94a47d0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a47d0 --- trap 0xc, rip = 0xffffffff80f01279, rsp = 0xfffffe02f94a48a0, rbp = 0xfffffe02f94a48d0 --- swap_pager_copy() at swap_pager_copy+0xf9/frame 0xfffffe02f94a48d0 vm_object_collapse() at vm_object_collapse+0x181/frame 0xfffffe02f94a4960 vm_object_deallocate() at vm_object_deallocate+0x41d/frame 0xfffffe02f94a49b0 vm_map_process_deferred() at vm_map_process_deferred+0xad/frame 0xfffffe02f94a49d0 vm_map_remove() at vm_map_remove+0xeb/frame 0xfffffe02f94a4a00 vmspace_exit() at vmspace_exit+0xd5/frame 0xfffffe02f94a4a40 exit1() at exit1+0x5a2/frame 0xfffffe02f94a4ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe02f94a4ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f94a4bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f94a4bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8003d880a, rsp = 0x7fffffffeb08, rbp = 0x7fffffffeb30 --- Tracing command sh pid 2434 tid 100153 td 0xfffff800265846e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e03ef690 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe02f94a3b60 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe02f94a3b70 trap() at trap+0x73/frame 0xfffffe02f94a3ca0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a3ca0 --- trap 0xc, rip = 0xffffffff810400a0, rsp = 0xfffffe02f94a3d70, rbp = 0xfffffe02f94a3e00 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe02f94a3e00 db_get_value() at db_get_value+0x32/frame 0xfffffe02f94a3e40 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe02f94a3ed0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe02f94a3ef0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe02f94a3f90 db_command() at db_command+0x25c/frame 0xfffffe02f94a4060 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe02f94a4100 db_command() at db_command+0x25c/frame 0xfffffe02f94a41d0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe02f94a41e0 db_trap() at db_trap+0xe8/frame 0xfffffe02f94a4270 kdb_trap() at kdb_trap+0x157/frame 0xfffffe02f94a42f0 trap() at trap+0x7d4/frame 0xfffffe02f94a4420 calltrap() at calltrap+0x8/frame 0xfffffe02f94a4420 --- trap 0x3, rip = 0xffffffff80c14af7, rsp = 0xfffffe02f94a44f0, rbp = 0xfffffe02f94a4500 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe02f94a4500 vpanic() at vpanic+0x19a/frame 0xfffffe02f94a4560 panic() at panic+0x43/frame 0xfffffe02f94a45c0 trap_fatal() at trap_fatal+0x386/frame 0xfffffe02f94a4620 trap_pfault() at trap_pfault+0x99/frame 0xfffffe02f94a46a0 trap() at trap+0x2c0/frame 0xfffffe02f94a47d0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a47d0 --- trap 0xc, rip = 0xffffffff80f01279, rsp = 0xfffffe02f94a48a0, rbp = 0xfffffe02f94a48d0 --- swap_pager_copy() at swap_pager_copy+0xf9/frame 0xfffffe02f94a48d0 vm_object_collapse() at vm_object_collapse+0x181/frame 0xfffffe02f94a4960 vm_object_deallocate() at vm_object_deallocate+0x41d/frame 0xfffffe02f94a49b0 vm_map_process_deferred() at vm_map_process_deferred+0xad/frame 0xfffffe02f94a49d0 vm_map_remove() at vm_map_remove+0xeb/frame 0xfffffe02f94a4a00 vmspace_exit() at vmspace_exit+0xd5/frame 0xfffffe02f94a4a40 exit1() at exit1+0x5a2/frame 0xfffffe02f94a4ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe02f94a4ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f94a4bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f94a4bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8003d880a, rsp = 0x7fffffffeb08, rbp = 0x7fffffffeb30 --- *** error reading from address fffffe00e03ef698 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe02f94a3df0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe02f94a3e00 db_get_value() at db_get_value+0x4c/frame 0xfffffe02f94a3e40 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe02f94a3ed0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe02f94a3ef0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe02f94a3f90 db_command() at db_command+0x25c/frame 0xfffffe02f94a4060 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe02f94a4100 db_command() at db_command+0x25c/frame 0xfffffe02f94a41d0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe02f94a41e0 db_trap() at db_trap+0xe8/frame 0xfffffe02f94a4270 kdb_trap() at kdb_trap+0x157/frame 0xfffffe02f94a42f0 trap() at trap+0x7d4/frame 0xfffffe02f94a4420 calltrap() at calltrap+0x8/frame 0xfffffe02f94a4420 --- trap 0x3, rip = 0xffffffff80c14af7, rsp = 0xfffffe02f94a44f0, rbp = 0xfffffe02f94a4500 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe02f94a4500 vpanic() at vpanic+0x19a/frame 0xfffffe02f94a4560 panic() at panic+0x43/frame 0xfffffe02f94a45c0 trap_fatal() at trap_fatal+0x386/frame 0xfffffe02f94a4620 trap_pfault() at trap_pfault+0x99/frame 0xfffffe02f94a46a0 trap() at trap+0x2c0/frame 0xfffffe02f94a47d0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a47d0 --- trap 0xc, rip = 0xffffffff80f01279, rsp = 0xfffffe02f94a48a0, rbp = 0xfffffe02f94a48d0 --- swap_pager_copy() at swap_pager_copy+0xf9/frame 0xfffffe02f94a48d0 vm_object_collapse() at vm_object_collapse+0x181/frame 0xfffffe02f94a4960 vm_object_deallocate() at vm_object_deallocate+0x41d/frame 0xfffffe02f94a49b0 vm_map_process_deferred() at vm_map_process_deferred+0xad/frame 0xfffffe02f94a49d0 vm_map_remove() at vm_map_remove+0xeb/frame 0xfffffe02f94a4a00 vmspace_exit() at vmspace_exit+0xd5/frame 0xfffffe02f94a4a40 exit1() at exit1+0x5a2/frame 0xfffffe02f94a4ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe02f94a4ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f94a4bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f94a4bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8003d880a, rsp = 0x7fffffffeb08, rbp = 0x7fffffffeb30 --- Tracing command bash pid 2351 tid 100214 td 0xfffff80026d506e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e17a56b0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e17a56f0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e17a5730 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e17a5790 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e17a57c0 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e17a5820 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e17a5860 kern_select() at kern_select+0x962/frame 0xfffffe00e17a5a40 sys_pselect() at sys_pselect+0x127/frame 0xfffffe00e17a5ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e17a5bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e17a5bf0 --- syscall (522, FreeBSD ELF64, sys_pselect), rip = 0x800517a3a, rsp = 0x7fffffffd6c8, rbp = 0x7fffffffd790 --- Tracing command csh pid 2349 tid 100213 td 0xfffff801b5196000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e17a0880 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe02f94a3b60 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe02f94a3b70 trap() at trap+0x73/frame 0xfffffe02f94a3ca0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a3ca0 --- trap 0xc, rip = 0xffffffff810400a0, rsp = 0xfffffe02f94a3d70, rbp = 0xfffffe02f94a3e00 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe02f94a3e00 db_get_value() at db_get_value+0x32/frame 0xfffffe02f94a3e40 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe02f94a3ed0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe02f94a3ef0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe02f94a3f90 db_command() at db_command+0x25c/frame 0xfffffe02f94a4060 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe02f94a4100 db_command() at db_command+0x25c/frame 0xfffffe02f94a41d0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe02f94a41e0 db_trap() at db_trap+0xe8/frame 0xfffffe02f94a4270 kdb_trap() at kdb_trap+0x157/frame 0xfffffe02f94a42f0 trap() at trap+0x7d4/frame 0xfffffe02f94a4420 calltrap() at calltrap+0x8/frame 0xfffffe02f94a4420 --- trap 0x3, rip = 0xffffffff80c14af7, rsp = 0xfffffe02f94a44f0, rbp = 0xfffffe02f94a4500 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe02f94a4500 vpanic() at vpanic+0x19a/frame 0xfffffe02f94a4560 panic() at panic+0x43/frame 0xfffffe02f94a45c0 trap_fatal() at trap_fatal+0x386/frame 0xfffffe02f94a4620 trap_pfault() at trap_pfault+0x99/frame 0xfffffe02f94a46a0 trap() at trap+0x2c0/frame 0xfffffe02f94a47d0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a47d0 --- trap 0xc, rip = 0xffffffff80f01279, rsp = 0xfffffe02f94a48a0, rbp = 0xfffffe02f94a48d0 --- swap_pager_copy() at swap_pager_copy+0xf9/frame 0xfffffe02f94a48d0 vm_object_collapse() at vm_object_collapse+0x181/frame 0xfffffe02f94a4960 vm_object_deallocate() at vm_object_deallocate+0x41d/frame 0xfffffe02f94a49b0 vm_map_process_deferred() at vm_map_process_deferred+0xad/frame 0xfffffe02f94a49d0 vm_map_remove() at vm_map_remove+0xeb/frame 0xfffffe02f94a4a00 vmspace_exit() at vmspace_exit+0xd5/frame 0xfffffe02f94a4a40 exit1() at exit1+0x5a2/frame 0xfffffe02f94a4ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe02f94a4ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f94a4bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f94a4bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8003d880a, rsp = 0x7fffffffeb08, rbp = 0x7fffffffeb30 --- *** error reading from address fffffe00e17a0888 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe02f94a3df0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe02f94a3e00 db_get_value() at db_get_value+0x4c/frame 0xfffffe02f94a3e40 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe02f94a3ed0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe02f94a3ef0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe02f94a3f90 db_command() at db_command+0x25c/frame 0xfffffe02f94a4060 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe02f94a4100 db_command() at db_command+0x25c/frame 0xfffffe02f94a41d0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe02f94a41e0 db_trap() at db_trap+0xe8/frame 0xfffffe02f94a4270 kdb_trap() at kdb_trap+0x157/frame 0xfffffe02f94a42f0 trap() at trap+0x7d4/frame 0xfffffe02f94a4420 calltrap() at calltrap+0x8/frame 0xfffffe02f94a4420 --- trap 0x3, rip = 0xffffffff80c14af7, rsp = 0xfffffe02f94a44f0, rbp = 0xfffffe02f94a4500 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe02f94a4500 vpanic() at vpanic+0x19a/frame 0xfffffe02f94a4560 panic() at panic+0x43/frame 0xfffffe02f94a45c0 trap_fatal() at trap_fatal+0x386/frame 0xfffffe02f94a4620 trap_pfault() at trap_pfault+0x99/frame 0xfffffe02f94a46a0 trap() at trap+0x2c0/frame 0xfffffe02f94a47d0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a47d0 --- trap 0xc, rip = 0xffffffff80f01279, rsp = 0xfffffe02f94a48a0, rbp = 0xfffffe02f94a48d0 --- swap_pager_copy() at swap_pager_copy+0xf9/frame 0xfffffe02f94a48d0 vm_object_collapse() at vm_object_collapse+0x181/frame 0xfffffe02f94a4960 vm_object_deallocate() at vm_object_deallocate+0x41d/frame 0xfffffe02f94a49b0 vm_map_process_deferred() at vm_map_process_deferred+0xad/frame 0xfffffe02f94a49d0 vm_map_remove() at vm_map_remove+0xeb/frame 0xfffffe02f94a4a00 vmspace_exit() at vmspace_exit+0xd5/frame 0xfffffe02f94a4a40 exit1() at exit1+0x5a2/frame 0xfffffe02f94a4ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe02f94a4ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f94a4bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f94a4bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8003d880a, rsp = 0x7fffffffeb08, rbp = 0x7fffffffeb30 --- Tracing command su pid 2348 tid 100205 td 0xfffff801496f9000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e1778690 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe02f94a3b60 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe02f94a3b70 trap() at trap+0x73/frame 0xfffffe02f94a3ca0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a3ca0 --- trap 0xc, rip = 0xffffffff810400a0, rsp = 0xfffffe02f94a3d70, rbp = 0xfffffe02f94a3e00 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe02f94a3e00 db_get_value() at db_get_value+0x32/frame 0xfffffe02f94a3e40 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe02f94a3ed0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe02f94a3ef0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe02f94a3f90 db_command() at db_command+0x25c/frame 0xfffffe02f94a4060 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe02f94a4100 db_command() at db_command+0x25c/frame 0xfffffe02f94a41d0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe02f94a41e0 db_trap() at db_trap+0xe8/frame 0xfffffe02f94a4270 kdb_trap() at kdb_trap+0x157/frame 0xfffffe02f94a42f0 trap() at trap+0x7d4/frame 0xfffffe02f94a4420 calltrap() at calltrap+0x8/frame 0xfffffe02f94a4420 --- trap 0x3, rip = 0xffffffff80c14af7, rsp = 0xfffffe02f94a44f0, rbp = 0xfffffe02f94a4500 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe02f94a4500 vpanic() at vpanic+0x19a/frame 0xfffffe02f94a4560 panic() at panic+0x43/frame 0xfffffe02f94a45c0 trap_fatal() at trap_fatal+0x386/frame 0xfffffe02f94a4620 trap_pfault() at trap_pfault+0x99/frame 0xfffffe02f94a46a0 trap() at trap+0x2c0/frame 0xfffffe02f94a47d0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a47d0 --- trap 0xc, rip = 0xffffffff80f01279, rsp = 0xfffffe02f94a48a0, rbp = 0xfffffe02f94a48d0 --- swap_pager_copy() at swap_pager_copy+0xf9/frame 0xfffffe02f94a48d0 vm_object_collapse() at vm_object_collapse+0x181/frame 0xfffffe02f94a4960 vm_object_deallocate() at vm_object_deallocate+0x41d/frame 0xfffffe02f94a49b0 vm_map_process_deferred() at vm_map_process_deferred+0xad/frame 0xfffffe02f94a49d0 vm_map_remove() at vm_map_remove+0xeb/frame 0xfffffe02f94a4a00 vmspace_exit() at vmspace_exit+0xd5/frame 0xfffffe02f94a4a40 exit1() at exit1+0x5a2/frame 0xfffffe02f94a4ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe02f94a4ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f94a4bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f94a4bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8003d880a, rsp = 0x7fffffffeb08, rbp = 0x7fffffffeb30 --- *** error reading from address fffffe00e1778698 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe02f94a3df0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe02f94a3e00 db_get_value() at db_get_value+0x4c/frame 0xfffffe02f94a3e40 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe02f94a3ed0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe02f94a3ef0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe02f94a3f90 db_command() at db_command+0x25c/frame 0xfffffe02f94a4060 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe02f94a4100 db_command() at db_command+0x25c/frame 0xfffffe02f94a41d0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe02f94a41e0 db_trap() at db_trap+0xe8/frame 0xfffffe02f94a4270 kdb_trap() at kdb_trap+0x157/frame 0xfffffe02f94a42f0 trap() at trap+0x7d4/frame 0xfffffe02f94a4420 calltrap() at calltrap+0x8/frame 0xfffffe02f94a4420 --- trap 0x3, rip = 0xffffffff80c14af7, rsp = 0xfffffe02f94a44f0, rbp = 0xfffffe02f94a4500 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe02f94a4500 vpanic() at vpanic+0x19a/frame 0xfffffe02f94a4560 panic() at panic+0x43/frame 0xfffffe02f94a45c0 trap_fatal() at trap_fatal+0x386/frame 0xfffffe02f94a4620 trap_pfault() at trap_pfault+0x99/frame 0xfffffe02f94a46a0 trap() at trap+0x2c0/frame 0xfffffe02f94a47d0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a47d0 --- trap 0xc, rip = 0xffffffff80f01279, rsp = 0xfffffe02f94a48a0, rbp = 0xfffffe02f94a48d0 --- swap_pager_copy() at swap_pager_copy+0xf9/frame 0xfffffe02f94a48d0 vm_object_collapse() at vm_object_collapse+0x181/frame 0xfffffe02f94a4960 vm_object_deallocate() at vm_object_deallocate+0x41d/frame 0xfffffe02f94a49b0 vm_map_process_deferred() at vm_map_process_deferred+0xad/frame 0xfffffe02f94a49d0 vm_map_remove() at vm_map_remove+0xeb/frame 0xfffffe02f94a4a00 vmspace_exit() at vmspace_exit+0xd5/frame 0xfffffe02f94a4a40 exit1() at exit1+0x5a2/frame 0xfffffe02f94a4ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe02f94a4ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f94a4bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f94a4bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8003d880a, rsp = 0x7fffffffeb08, rbp = 0x7fffffffeb30 --- Tracing command bash pid 2343 tid 100143 td 0xfffff80026583000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e03bd690 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe02f94a3b60 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe02f94a3b70 trap() at trap+0x73/frame 0xfffffe02f94a3ca0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a3ca0 --- trap 0xc, rip = 0xffffffff810400a0, rsp = 0xfffffe02f94a3d70, rbp = 0xfffffe02f94a3e00 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe02f94a3e00 db_get_value() at db_get_value+0x32/frame 0xfffffe02f94a3e40 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe02f94a3ed0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe02f94a3ef0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe02f94a3f90 db_command() at db_command+0x25c/frame 0xfffffe02f94a4060 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe02f94a4100 db_command() at db_command+0x25c/frame 0xfffffe02f94a41d0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe02f94a41e0 db_trap() at db_trap+0xe8/frame 0xfffffe02f94a4270 kdb_trap() at kdb_trap+0x157/frame 0xfffffe02f94a42f0 trap() at trap+0x7d4/frame 0xfffffe02f94a4420 calltrap() at calltrap+0x8/frame 0xfffffe02f94a4420 --- trap 0x3, rip = 0xffffffff80c14af7, rsp = 0xfffffe02f94a44f0, rbp = 0xfffffe02f94a4500 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe02f94a4500 vpanic() at vpanic+0x19a/frame 0xfffffe02f94a4560 panic() at panic+0x43/frame 0xfffffe02f94a45c0 trap_fatal() at trap_fatal+0x386/frame 0xfffffe02f94a4620 trap_pfault() at trap_pfault+0x99/frame 0xfffffe02f94a46a0 trap() at trap+0x2c0/frame 0xfffffe02f94a47d0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a47d0 --- trap 0xc, rip = 0xffffffff80f01279, rsp = 0xfffffe02f94a48a0, rbp = 0xfffffe02f94a48d0 --- swap_pager_copy() at swap_pager_copy+0xf9/frame 0xfffffe02f94a48d0 vm_object_collapse() at vm_object_collapse+0x181/frame 0xfffffe02f94a4960 vm_object_deallocate() at vm_object_deallocate+0x41d/frame 0xfffffe02f94a49b0 vm_map_process_deferred() at vm_map_process_deferred+0xad/frame 0xfffffe02f94a49d0 vm_map_remove() at vm_map_remove+0xeb/frame 0xfffffe02f94a4a00 vmspace_exit() at vmspace_exit+0xd5/frame 0xfffffe02f94a4a40 exit1() at exit1+0x5a2/frame 0xfffffe02f94a4ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe02f94a4ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f94a4bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f94a4bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8003d880a, rsp = 0x7fffffffeb08, rbp = 0x7fffffffeb30 --- *** error reading from address fffffe00e03bd698 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe02f94a3df0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe02f94a3e00 db_get_value() at db_get_value+0x4c/frame 0xfffffe02f94a3e40 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe02f94a3ed0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe02f94a3ef0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe02f94a3f90 db_command() at db_command+0x25c/frame 0xfffffe02f94a4060 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe02f94a4100 db_command() at db_command+0x25c/frame 0xfffffe02f94a41d0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe02f94a41e0 db_trap() at db_trap+0xe8/frame 0xfffffe02f94a4270 kdb_trap() at kdb_trap+0x157/frame 0xfffffe02f94a42f0 trap() at trap+0x7d4/frame 0xfffffe02f94a4420 calltrap() at calltrap+0x8/frame 0xfffffe02f94a4420 --- trap 0x3, rip = 0xffffffff80c14af7, rsp = 0xfffffe02f94a44f0, rbp = 0xfffffe02f94a4500 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe02f94a4500 vpanic() at vpanic+0x19a/frame 0xfffffe02f94a4560 panic() at panic+0x43/frame 0xfffffe02f94a45c0 trap_fatal() at trap_fatal+0x386/frame 0xfffffe02f94a4620 trap_pfault() at trap_pfault+0x99/frame 0xfffffe02f94a46a0 trap() at trap+0x2c0/frame 0xfffffe02f94a47d0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a47d0 --- trap 0xc, rip = 0xffffffff80f01279, rsp = 0xfffffe02f94a48a0, rbp = 0xfffffe02f94a48d0 --- swap_pager_copy() at swap_pager_copy+0xf9/frame 0xfffffe02f94a48d0 vm_object_collapse() at vm_object_collapse+0x181/frame 0xfffffe02f94a4960 vm_object_deallocate() at vm_object_deallocate+0x41d/frame 0xfffffe02f94a49b0 vm_map_process_deferred() at vm_map_process_deferred+0xad/frame 0xfffffe02f94a49d0 vm_map_remove() at vm_map_remove+0xeb/frame 0xfffffe02f94a4a00 vmspace_exit() at vmspace_exit+0xd5/frame 0xfffffe02f94a4a40 exit1() at exit1+0x5a2/frame 0xfffffe02f94a4ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe02f94a4ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f94a4bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f94a4bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8003d880a, rsp = 0x7fffffffeb08, rbp = 0x7fffffffeb30 --- Tracing command sshd pid 2342 tid 100157 td 0xfffff80026c15000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e16046f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e1604730 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e1604770 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e16047d0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e1604800 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e1604860 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e16048a0 kern_select() at kern_select+0x962/frame 0xfffffe00e1604a80 sys_select() at sys_select+0x56/frame 0xfffffe00e1604ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e1604bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e1604bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8008f5afa, rsp = 0x7fffffffa608, rbp = 0x7fffffffa640 --- Tracing command sshd pid 2339 tid 100202 td 0xfffff80026c146e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e1769770 mi_switch() at mi_switch+0x182/frame 0xfffffe00e17697b0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e17697f0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e1769850 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e1769880 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e17698e0 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e1769920 kern_poll() at kern_poll+0x48b/frame 0xfffffe00e1769aa0 sys_poll() at sys_poll+0x50/frame 0xfffffe00e1769ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e1769bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e1769bf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x80089913a, rsp = 0x7fffffffe668, rbp = 0x7fffffffe6a0 --- Tracing command csh pid 2335 tid 100171 td 0xfffff80026dc56e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e1656750 mi_switch() at mi_switch+0x182/frame 0xfffffe00e1656790 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e16567d0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e1656830 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e1656860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e16568c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e16568f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e1656950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e1656980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e16569e0 dofileread() at dofileread+0x81/frame 0xfffffe00e1656a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e1656a70 sys_read() at sys_read+0x83/frame 0xfffffe00e1656ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e1656bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e1656bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004fbb1a, rsp = 0x7fffffffe278, rbp = 0x7fffffffe2a0 --- Tracing command login pid 2334 tid 100196 td 0xfffff8002673e000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e173f690 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe02f94a3b60 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe02f94a3b70 trap() at trap+0x73/frame 0xfffffe02f94a3ca0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a3ca0 --- trap 0xc, rip = 0xffffffff810400a0, rsp = 0xfffffe02f94a3d70, rbp = 0xfffffe02f94a3e00 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe02f94a3e00 db_get_value() at db_get_value+0x32/frame 0xfffffe02f94a3e40 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe02f94a3ed0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe02f94a3ef0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe02f94a3f90 db_command() at db_command+0x25c/frame 0xfffffe02f94a4060 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe02f94a4100 db_command() at db_command+0x25c/frame 0xfffffe02f94a41d0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe02f94a41e0 db_trap() at db_trap+0xe8/frame 0xfffffe02f94a4270 kdb_trap() at kdb_trap+0x157/frame 0xfffffe02f94a42f0 trap() at trap+0x7d4/frame 0xfffffe02f94a4420 calltrap() at calltrap+0x8/frame 0xfffffe02f94a4420 --- trap 0x3, rip = 0xffffffff80c14af7, rsp = 0xfffffe02f94a44f0, rbp = 0xfffffe02f94a4500 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe02f94a4500 vpanic() at vpanic+0x19a/frame 0xfffffe02f94a4560 panic() at panic+0x43/frame 0xfffffe02f94a45c0 trap_fatal() at trap_fatal+0x386/frame 0xfffffe02f94a4620 trap_pfault() at trap_pfault+0x99/frame 0xfffffe02f94a46a0 trap() at trap+0x2c0/frame 0xfffffe02f94a47d0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a47d0 --- trap 0xc, rip = 0xffffffff80f01279, rsp = 0xfffffe02f94a48a0, rbp = 0xfffffe02f94a48d0 --- swap_pager_copy() at swap_pager_copy+0xf9/frame 0xfffffe02f94a48d0 vm_object_collapse() at vm_object_collapse+0x181/frame 0xfffffe02f94a4960 vm_object_deallocate() at vm_object_deallocate+0x41d/frame 0xfffffe02f94a49b0 vm_map_process_deferred() at vm_map_process_deferred+0xad/frame 0xfffffe02f94a49d0 vm_map_remove() at vm_map_remove+0xeb/frame 0xfffffe02f94a4a00 vmspace_exit() at vmspace_exit+0xd5/frame 0xfffffe02f94a4a40 exit1() at exit1+0x5a2/frame 0xfffffe02f94a4ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe02f94a4ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f94a4bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f94a4bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8003d880a, rsp = 0x7fffffffeb08, rbp = 0x7fffffffeb30 --- *** error reading from address fffffe00e173f698 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe02f94a3df0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe02f94a3e00 db_get_value() at db_get_value+0x4c/frame 0xfffffe02f94a3e40 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe02f94a3ed0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe02f94a3ef0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe02f94a3f90 db_command() at db_command+0x25c/frame 0xfffffe02f94a4060 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe02f94a4100 db_command() at db_command+0x25c/frame 0xfffffe02f94a41d0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe02f94a41e0 db_trap() at db_trap+0xe8/frame 0xfffffe02f94a4270 kdb_trap() at kdb_trap+0x157/frame 0xfffffe02f94a42f0 trap() at trap+0x7d4/frame 0xfffffe02f94a4420 calltrap() at calltrap+0x8/frame 0xfffffe02f94a4420 --- trap 0x3, rip = 0xffffffff80c14af7, rsp = 0xfffffe02f94a44f0, rbp = 0xfffffe02f94a4500 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe02f94a4500 vpanic() at vpanic+0x19a/frame 0xfffffe02f94a4560 panic() at panic+0x43/frame 0xfffffe02f94a45c0 trap_fatal() at trap_fatal+0x386/frame 0xfffffe02f94a4620 trap_pfault() at trap_pfault+0x99/frame 0xfffffe02f94a46a0 trap() at trap+0x2c0/frame 0xfffffe02f94a47d0 calltrap() at calltrap+0x8/frame 0xfffffe02f94a47d0 --- trap 0xc, rip = 0xffffffff80f01279, rsp = 0xfffffe02f94a48a0, rbp = 0xfffffe02f94a48d0 --- swap_pager_copy() at swap_pager_copy+0xf9/frame 0xfffffe02f94a48d0 vm_object_collapse() at vm_object_collapse+0x181/frame 0xfffffe02f94a4960 vm_object_deallocate() at vm_object_deallocate+0x41d/frame 0xfffffe02f94a49b0 vm_map_process_deferred() at vm_map_process_deferred+0xad/frame 0xfffffe02f94a49d0 vm_map_remove() at vm_map_remove+0xeb/frame 0xfffffe02f94a4a00 vmspace_exit() at vmspace_exit+0xd5/frame 0xfffffe02f94a4a40 exit1() at exit1+0x5a2/frame 0xfffffe02f94a4ab0 sys_sys_exit() at sys_sys_exit+0xd/frame 0xfffffe02f94a4ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f94a4bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f94a4bf0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8003d880a, rsp = 0x7fffffffeb08, rbp = 0x7fffffffeb30 --- Tracing command getty pid 2333 tid 100167 td 0xfffff80026eca6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e1642750 mi_switch() at mi_switch+0x182/frame 0xfffffe00e1642790 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e16427d0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e1642830 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e1642860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e16428c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e16428f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e1642950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e1642980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e16429e0 dofileread() at dofileread+0x81/frame 0xfffffe00e1642a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e1642a70 sys_read() at sys_read+0x83/frame 0xfffffe00e1642ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e1642bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e1642bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800433b1a, rsp = 0x7fffffffec68, rbp = 0x7fffffffeca0 --- Tracing command getty pid 2332 tid 100211 td 0xfffff8003c4dc6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e1796750 mi_switch() at mi_switch+0x182/frame 0xfffffe00e1796790 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e17967d0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e1796830 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e1796860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e17968c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e17968f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e1796950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e1796980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e17969e0 dofileread() at dofileread+0x81/frame 0xfffffe00e1796a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e1796a70 sys_read() at sys_read+0x83/frame 0xfffffe00e1796ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e1796bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e1796bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800433b1a, rsp = 0x7fffffffec68, rbp = 0x7fffffffeca0 --- Tracing command getty pid 2331 tid 100210 td 0xfffff80059f8e000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e1791750 mi_switch() at mi_switch+0x182/frame 0xfffffe00e1791790 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e17917d0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e1791830 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e1791860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e17918c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e17918f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e1791950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e1791980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e17919e0 dofileread() at dofileread+0x81/frame 0xfffffe00e1791a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e1791a70 sys_read() at sys_read+0x83/frame 0xfffffe00e1791ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e1791bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e1791bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800433b1a, rsp = 0x7fffffffec68, rbp = 0x7fffffffeca0 --- Tracing command getty pid 2330 tid 100209 td 0xfffff80059f8e6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e178c750 mi_switch() at mi_switch+0x182/frame 0xfffffe00e178c790 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e178c7d0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e178c830 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e178c860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e178c8c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e178c8f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e178c950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e178c980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e178c9e0 dofileread() at dofileread+0x81/frame 0xfffffe00e178ca30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e178ca70 sys_read() at sys_read+0x83/frame 0xfffffe00e178cac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e178cbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e178cbf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800433b1a, rsp = 0x7fffffffec68, rbp = 0x7fffffffeca0 --- Tracing command getty pid 2329 tid 100208 td 0xfffff80026d90000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e1787750 mi_switch() at mi_switch+0x182/frame 0xfffffe00e1787790 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e17877d0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e1787830 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e1787860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e17878c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e17878f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e1787950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e1787980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e17879e0 dofileread() at dofileread+0x81/frame 0xfffffe00e1787a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e1787a70 sys_read() at sys_read+0x83/frame 0xfffffe00e1787ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e1787bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e1787bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800433b1a, rsp = 0x7fffffffec68, rbp = 0x7fffffffeca0 --- Tracing command getty pid 2328 tid 100207 td 0xfffff80026d906e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e1782750 mi_switch() at mi_switch+0x182/frame 0xfffffe00e1782790 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e17827d0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e1782830 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e1782860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e17828c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e17828f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e1782950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e1782980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e17829e0 dofileread() at dofileread+0x81/frame 0xfffffe00e1782a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e1782a70 sys_read() at sys_read+0x83/frame 0xfffffe00e1782ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e1782bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e1782bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800433b1a, rsp = 0x7fffffffec68, rbp = 0x7fffffffeca0 --- Tracing command getty pid 2327 tid 100206 td 0xfffff80026d8f000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e177d750 mi_switch() at mi_switch+0x182/frame 0xfffffe00e177d790 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e177d7d0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e177d830 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e177d860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e177d8c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e177d8f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e177d950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e177d980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e177d9e0 dofileread() at dofileread+0x81/frame 0xfffffe00e177da30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e177da70 sys_read() at sys_read+0x83/frame 0xfffffe00e177dac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e177dbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e177dbf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800433b1a, rsp = 0x7fffffffec68, rbp = 0x7fffffffeca0 --- Tracing command getty pid 2326 tid 100170 td 0xfffff80026d8f6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e1651750 mi_switch() at mi_switch+0x182/frame 0xfffffe00e1651790 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e16517d0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e1651830 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e1651860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e16518c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e16518f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e1651950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e1651980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e16519e0 dofileread() at dofileread+0x81/frame 0xfffffe00e1651a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e1651a70 sys_read() at sys_read+0x83/frame 0xfffffe00e1651ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e1651bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e1651bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800433b1a, rsp = 0x7fffffffec68, rbp = 0x7fffffffeca0 --- Tracing command getty pid 2325 tid 100142 td 0xfffff80003bea6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e03b8750 mi_switch() at mi_switch+0x182/frame 0xfffffe00e03b8790 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e03b87d0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e03b8830 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e03b8860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e03b88c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e03b88f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e03b8950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e03b8980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e03b89e0 dofileread() at dofileread+0x81/frame 0xfffffe00e03b8a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e03b8a70 sys_read() at sys_read+0x83/frame 0xfffffe00e03b8ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e03b8bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e03b8bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800433b1a, rsp = 0x7fffffffec68, rbp = 0x7fffffffeca0 --- Tracing command cron pid 2279 tid 100176 td 0xfffff8003c50a000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e1687830 mi_switch() at mi_switch+0x182/frame 0xfffffe00e1687870 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e16878b0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e1687910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00e1687950 _sleep() at _sleep+0x258/frame 0xfffffe00e16879f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe00e1687a80 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe00e1687ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e1687bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e1687bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003d8a7a, rsp = 0x7fffffffeb68, rbp = 0x7fffffffed50 --- Tracing command sshd pid 2275 tid 100166 td 0xfffff800041196e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e163d6f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e163d730 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e163d770 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e163d7d0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e163d800 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e163d860 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e163d8a0 kern_select() at kern_select+0x962/frame 0xfffffe00e163da80 sys_select() at sys_select+0x56/frame 0xfffffe00e163dac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e163dbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e163dbf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8008f5afa, rsp = 0x7fffffffe6f8, rbp = 0x7fffffffe730 --- Tracing command powerd pid 2247 tid 100201 td 0xfffff80026c5b000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e17646d0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e1764710 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e1764750 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e17647b0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00e17647f0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x184/frame 0xfffffe00e1764860 seltdwait() at seltdwait+0x6b/frame 0xfffffe00e17648a0 kern_select() at kern_select+0x962/frame 0xfffffe00e1764a80 sys_select() at sys_select+0x56/frame 0xfffffe00e1764ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e1764bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e1764bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800430afa, rsp = 0x7fffffffec08, rbp = 0x7fffffffed50 --- Tracing command ntpd pid 2244 tid 100177 td 0xfffff8003c51f6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e168c6f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e168c730 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e168c770 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e168c7d0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e168c800 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e168c860 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e168c8a0 kern_select() at kern_select+0x962/frame 0xfffffe00e168ca80 sys_select() at sys_select+0x56/frame 0xfffffe00e168cac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e168cbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e168cbf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800827afa, rsp = 0x7fffffffdf88, rbp = 0x7fffffffdfc0 --- Tracing command ntpd pid 2244 tid 100677 td 0xfffff804e6fe06e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe02f92d67e0 mi_switch() at mi_switch+0x182/frame 0xfffffe02f92d6820 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe02f92d6860 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe02f92d68c0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe02f92d68f0 _sleep() at _sleep+0x2c3/frame 0xfffffe02f92d6990 umtxq_sleep() at umtxq_sleep+0x19e/frame 0xfffffe02f92d69f0 do_sem2_wait() at do_sem2_wait+0x422/frame 0xfffffe02f92d6a70 __umtx_op_sem2_wait() at __umtx_op_sem2_wait+0x76/frame 0xfffffe02f92d6ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe02f92d6bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02f92d6bf0 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x80071df7a, rsp = 0x7fffdff3aed8, rbp = 0x7fffdff3af50 --- Tracing command rpc.lockd pid 2181 tid 100188 td 0xfffff8005954a6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e16f3610 mi_switch() at mi_switch+0x182/frame 0xfffffe00e16f3650 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e16f3690 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e16f36f0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00e16f3730 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x184/frame 0xfffffe00e16f37a0 svc_run_internal() at svc_run_internal+0xabe/frame 0xfffffe00e16f38e0 svc_run() at svc_run+0x253/frame 0xfffffe00e16f3940 sys_nlm_syscall() at sys_nlm_syscall+0x745/frame 0xfffffe00e16f3ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e16f3bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e16f3bf0 --- syscall (154, FreeBSD ELF64, sys_nlm_syscall), rip = 0x8002f54aa, rsp = 0x7fffffffec88, rbp = 0x7fffffffed50 --- Tracing command rpc.statd pid 2178 tid 100198 td 0xfffff8005954a000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e17496d0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e1749710 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e1749750 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e17497b0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00e17497f0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x184/frame 0xfffffe00e1749860 seltdwait() at seltdwait+0x6b/frame 0xfffffe00e17498a0 kern_select() at kern_select+0x962/frame 0xfffffe00e1749a80 sys_select() at sys_select+0x56/frame 0xfffffe00e1749ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e1749bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e1749bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800427afa, rsp = 0x7fffffffeb38, rbp = 0x7fffffffec80 --- Tracing command watchdogd pid 2132 tid 100193 td 0xfffff800269fc000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e1730830 mi_switch() at mi_switch+0x182/frame 0xfffffe00e1730870 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e17308b0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e1730910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00e1730950 _sleep() at _sleep+0x258/frame 0xfffffe00e17309f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe00e1730a80 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe00e1730ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e1730bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e1730bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003c1a7a, rsp = 0x7fffffffeb28, rbp = 0x7fffffffeb70 --- Tracing command rpcbind pid 2053 tid 100190 td 0xfffff80026c0d000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e1709750 mi_switch() at mi_switch+0x182/frame 0xfffffe00e1709790 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e17097d0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e1709830 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00e1709870 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x184/frame 0xfffffe00e17098e0 seltdwait() at seltdwait+0x6b/frame 0xfffffe00e1709920 kern_poll() at kern_poll+0x48b/frame 0xfffffe00e1709aa0 sys_poll() at sys_poll+0x50/frame 0xfffffe00e1709ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e1709bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e1709bf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8003cf13a, rsp = 0x7fffffffcac8, rbp = 0x7fffffffece0 --- Tracing command syslogd pid 2043 tid 100163 td 0xfffff80026b83000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e162e070 mi_switch() at mi_switch+0x182/frame 0xfffffe00e162e0b0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e162e0f0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00e162e120 _sleep() at _sleep+0x2cd/frame 0xfffffe00e162e1c0 bufwait() at bufwait+0x84/frame 0xfffffe00e162e200 breadn_flags() at breadn_flags+0x1ae/frame 0xfffffe00e162e270 ffs_getcg() at ffs_getcg+0xd7/frame 0xfffffe00e162e2e0 ffs_realloccg() at ffs_realloccg+0x386/frame 0xfffffe00e162e3f0 ffs_balloc_ufs2() at ffs_balloc_ufs2+0x578/frame 0xfffffe00e162e5b0 ffs_write() at ffs_write+0x217/frame 0xfffffe00e162e650 VOP_WRITE_APV() at VOP_WRITE_APV+0x113/frame 0xfffffe00e162e760 vn_write() at vn_write+0x293/frame 0xfffffe00e162e7e0 vn_io_fault_doio() at vn_io_fault_doio+0x43/frame 0xfffffe00e162e840 vn_io_fault1() at vn_io_fault1+0x16c/frame 0xfffffe00e162e990 vn_io_fault() at vn_io_fault+0x186/frame 0xfffffe00e162ea00 dofilewrite() at dofilewrite+0x81/frame 0xfffffe00e162ea50 kern_writev() at kern_writev+0x49/frame 0xfffffe00e162ea90 sys_writev() at sys_writev+0x35/frame 0xfffffe00e162eac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e162ebf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e162ebf0 --- syscall (121, FreeBSD ELF64, sys_writev), rip = 0x800429b4a, rsp = 0x7fffffffd1d8, rbp = 0x7fffffffd310 --- Tracing command devd pid 1972 tid 100159 td 0xfffff800265826e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e160e6d0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e160e710 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e160e750 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e160e7b0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00e160e7f0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x184/frame 0xfffffe00e160e860 seltdwait() at seltdwait+0x6b/frame 0xfffffe00e160e8a0 kern_select() at kern_select+0x962/frame 0xfffffe00e160ea80 sys_select() at sys_select+0x56/frame 0xfffffe00e160eac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e160ebf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e160ebf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x2f5caa, rsp = 0x7fffffffcbe8, rbp = 0x7fffffffed70 --- Tracing command dhclient pid 1967 tid 100191 td 0xfffff80026ee3000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e170e750 mi_switch() at mi_switch+0x182/frame 0xfffffe00e170e790 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e170e7d0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e170e830 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00e170e870 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x184/frame 0xfffffe00e170e8e0 seltdwait() at seltdwait+0x6b/frame 0xfffffe00e170e920 kern_poll() at kern_poll+0x48b/frame 0xfffffe00e170eaa0 sys_poll() at sys_poll+0x50/frame 0xfffffe00e170eac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e170ebf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e170ebf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8003f213a, rsp = 0x7fffffffecb8, rbp = 0x7fffffffecf0 --- Tracing command dhclient pid 1920 tid 100189 td 0xfffff80026ee36e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e16f8770 mi_switch() at mi_switch+0x182/frame 0xfffffe00e16f87b0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e16f87f0 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e16f8850 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e16f8880 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e16f88e0 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e16f8920 kern_poll() at kern_poll+0x48b/frame 0xfffffe00e16f8aa0 sys_poll() at sys_poll+0x50/frame 0xfffffe00e16f8ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e16f8bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e16f8bf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8003f213a, rsp = 0x7fffffffecc8, rbp = 0x7fffffffecf0 --- Tracing command dhclient pid 1917 tid 100168 td 0xfffff8002504a6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e16476f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e1647730 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e1647770 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00e16477d0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00e1647800 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e1647860 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e16478a0 kern_select() at kern_select+0x962/frame 0xfffffe00e1647a80 sys_select() at sys_select+0x56/frame 0xfffffe00e1647ac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00e1647bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e1647bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80044eafa, rsp = 0x7fffffffea78, rbp = 0x7fffffffeb30 --- Tracing command syncer pid 34 tid 100141 td 0xfffff800265836e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e03b39f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e03b3a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e03b3a70 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00e03b3ab0 _cv_timedwait_sbt() at _cv_timedwait_sbt+0x184/frame 0xfffffe00e03b3b20 sched_sync() at sched_sync+0x5ce/frame 0xfffffe00e03b3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e03b3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e03b3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command vnlru pid 33 tid 100138 td 0xfffff80026582000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e03a49b0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e03a49f0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e03a4a30 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00e03a4a70 _sleep() at _sleep+0x26c/frame 0xfffffe00e03a4b10 vnlru_proc() at vnlru_proc+0x149/frame 0xfffffe00e03a4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e03a4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e03a4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100130 td 0xfffff800261be000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00d322ea00 mi_switch() at mi_switch+0x182/frame 0xfffffe00d322ea40 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00d322ea80 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00d322eac0 _sleep() at _sleep+0x26c/frame 0xfffffe00d322eb60 buf_daemon() at buf_daemon+0x112/frame 0xfffffe00d322ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00d322ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d322ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100132 td 0xfffff800264366e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00dabe99f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00dabe9a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00dabe9a70 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00dabe9ab0 _sleep() at _sleep+0x26c/frame 0xfffffe00dabe9b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe00dabe9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00dabe9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00dabe9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100134 td 0xfffff80026436000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00dabee9f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00dabeea30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00dabeea70 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00dabeeab0 _sleep() at _sleep+0x26c/frame 0xfffffe00dabeeb50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe00dabeebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00dabeebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00dabeebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100135 td 0xfffff800264346e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00dabf39f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00dabf3a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00dabf3a70 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00dabf3ab0 _sleep() at _sleep+0x26c/frame 0xfffffe00dabf3b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe00dabf3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00dabf3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00dabf3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100136 td 0xfffff8002640b6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00dabf89f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00dabf8a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00dabf8a70 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00dabf8ab0 _sleep() at _sleep+0x26c/frame 0xfffffe00dabf8b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe00dabf8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00dabf8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00dabf8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100137 td 0xfffff8002640b000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00dabfd9f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00dabfda30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00dabfda70 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00dabfdab0 _sleep() at _sleep+0x26c/frame 0xfffffe00dabfdb50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe00dabfdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00dabfdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00dabfdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100139 td 0xfffff800265936e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e03a99f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e03a9a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e03a9a70 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00e03a9ab0 _sleep() at _sleep+0x26c/frame 0xfffffe00e03a9b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe00e03a9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e03a9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e03a9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100140 td 0xfffff80026593000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e03ae9f0 mi_switch() at mi_switch+0x182/frame 0xfffffe00e03aea30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e03aea70 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00e03aeab0 _sleep() at _sleep+0x26c/frame 0xfffffe00e03aeb50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe00e03aebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e03aebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e03aebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100152 td 0xfffff80026c14000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e03eaa20 mi_switch() at mi_switch+0x182/frame 0xfffffe00e03eaa60 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e03eaaa0 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00e03eaae0 _sleep() at _sleep+0x26c/frame 0xfffffe00e03eab80 softdep_flush() at softdep_flush+0x175/frame 0xfffffe00e03eabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e03eabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e03eabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100200 td 0xfffff80026c5b6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00e175fa20 mi_switch() at mi_switch+0x182/frame 0xfffffe00e175fa60 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00e175faa0 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00e175fae0 _sleep() at _sleep+0x26c/frame 0xfffffe00e175fb80 softdep_flush() at softdep_flush+0x175/frame 0xfffffe00e175fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e175fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e175fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command vmdaemon pid 31 tid 100129 td 0xfffff800261be6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00d3229a70 mi_switch() at mi_switch+0x182/frame 0xfffffe00d3229ab0 sched_bind() at sched_bind+0xd8/frame 0xfffffe00d3229ae0 vm_page_pqbatch_drain() at vm_page_pqbatch_drain+0x80/frame 0xfffffe00d3229b40 vm_daemon() at vm_daemon+0xc1/frame 0xfffffe00d3229bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00d3229bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d3229bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 30 tid 100128 td 0xfffff800261bd000 (CPU 11) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1ebdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1ebdf0 trap() at trap+0x61/frame 0xfffffe003e1ebf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1ebf20 --- trap 0x13, rip = 0xffffffff810530ba, rsp = 0xfffffe00d3218650, rbp = 0xfffffe00d3218680 --- pmap_invalidate_page_pcid_invpcid() at pmap_invalidate_page_pcid_invpcid+0xca/frame 0xfffffe00d3218680 pmap_invalidate_page() at pmap_invalidate_page+0x64/frame 0xfffffe00d32186e0 pmap_ts_referenced() at pmap_ts_referenced+0x7dd/frame 0xfffffe00d3218770 vm_pageout_worker() at vm_pageout_worker+0x1079/frame 0xfffffe00d3218b70 vm_pageout() at vm_pageout+0x193/frame 0xfffffe00d3218bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00d3218bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d3218bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 30 tid 100131 td 0xfffff80026434000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00d33ff7e0 mi_switch() at mi_switch+0x182/frame 0xfffffe00d33ff820 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00d33ff860 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d33ff890 _sleep() at _sleep+0x2cd/frame 0xfffffe00d33ff930 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0xe1b/frame 0xfffffe00d33ffbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00d33ffbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d33ffbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 30 tid 100133 td 0xfffff800264326e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00dab84a20 mi_switch() at mi_switch+0x182/frame 0xfffffe00dab84a60 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00dab84aa0 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00dab84ae0 _sleep() at _sleep+0x26c/frame 0xfffffe00dab84b80 uma_reclaim_worker() at uma_reclaim_worker+0xb8/frame 0xfffffe00dab84bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00dab84bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00dab84bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon1 pid 29 tid 100127 td 0xfffff80026286000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00d32139e0 mi_switch() at mi_switch+0x182/frame 0xfffffe00d3213a20 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00d3213a60 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d3213a90 _sleep() at _sleep+0x2cd/frame 0xfffffe00d3213b30 enc_daemon() at enc_daemon+0xf6/frame 0xfffffe00d3213bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00d3213bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d3213bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon0 pid 28 tid 100126 td 0xfffff800262866e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00d320e9e0 mi_switch() at mi_switch+0x182/frame 0xfffffe00d320ea20 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00d320ea60 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d320ea90 _sleep() at _sleep+0x2cd/frame 0xfffffe00d320eb30 enc_daemon() at enc_daemon+0xf6/frame 0xfffffe00d320ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00d320ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d320ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command rand_harvestq pid 27 tid 100120 td 0xfffff80026170000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5f09f0 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5f0a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5f0a70 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe003e5f0ab0 _sleep() at _sleep+0x26c/frame 0xfffffe003e5f0b50 random_kthread() at random_kthread+0x67/frame 0xfffffe003e5f0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5f0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5f0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sctp_iterator pid 26 tid 100119 td 0xfffff800261706e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5dfa40 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5dfa80 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5dfac0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e5dfaf0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e5dfb90 sctp_iterator_thread() at sctp_iterator_thread+0x59/frame 0xfffffe003e5dfbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5dfbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5dfbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod4 pid 25 tid 100118 td 0xfffff800260836e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5cea10 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5cea50 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5cea90 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe003e5cead0 _sleep() at _sleep+0x26c/frame 0xfffffe003e5ceb70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe003e5cebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5cebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5cebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod3 pid 24 tid 100117 td 0xfffff80026091000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5c9a10 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5c9a50 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5c9a90 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe003e5c9ad0 _sleep() at _sleep+0x26c/frame 0xfffffe003e5c9b70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe003e5c9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5c9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5c9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod2 pid 23 tid 100116 td 0xfffff800260916e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5c4a10 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5c4a50 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5c4a90 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe003e5c4ad0 _sleep() at _sleep+0x26c/frame 0xfffffe003e5c4b70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe003e5c4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5c4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5c4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod1 pid 22 tid 100115 td 0xfffff80026090000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5bfa10 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5bfa50 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5bfa90 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe003e5bfad0 _sleep() at _sleep+0x26c/frame 0xfffffe003e5bfb70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe003e5bfbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5bfbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5bfbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100096 td 0xfffff80003f996e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e548a60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e548aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e548ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e548b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e548b70 usb_process() at usb_process+0xc3/frame 0xfffffe003e548bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e548bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e548bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100097 td 0xfffff80003f99000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e54da60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e54daa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e54dae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e54db10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e54db70 usb_process() at usb_process+0xc3/frame 0xfffffe003e54dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e54dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e54dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100098 td 0xfffff80003be86e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e552a60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e552aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e552ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e552b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e552b70 usb_process() at usb_process+0xc3/frame 0xfffffe003e552bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e552bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e552bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100099 td 0xfffff80004119000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e557a60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e557aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e557ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e557b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e557b70 usb_process() at usb_process+0xc3/frame 0xfffffe003e557bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e557bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e557bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100100 td 0xfffff8000411a6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e55ca60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e55caa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e55cae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e55cb10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e55cb70 usb_process() at usb_process+0xc3/frame 0xfffffe003e55cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e55cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e55cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100102 td 0xfffff8000411b6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e566a60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e566aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e566ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e566b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e566b70 usb_process() at usb_process+0xc3/frame 0xfffffe003e566bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e566bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e566bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100103 td 0xfffff8000411b000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e56ba60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e56baa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e56bae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e56bb10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e56bb70 usb_process() at usb_process+0xc3/frame 0xfffffe003e56bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e56bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e56bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100104 td 0xfffff8000412b6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e570a60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e570aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e570ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e570b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e570b70 usb_process() at usb_process+0xc3/frame 0xfffffe003e570bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e570bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e570bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100105 td 0xfffff8000412b000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e575a60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e575aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e575ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e575b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e575b70 usb_process() at usb_process+0xc3/frame 0xfffffe003e575bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e575bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e575bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100106 td 0xfffff800041226e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e57aa60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e57aaa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e57aae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e57ab10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e57ab70 usb_process() at usb_process+0xc3/frame 0xfffffe003e57abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e57abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e57abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100107 td 0xfffff8002504a000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e597a60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e597aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e597ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e597b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e597b70 usb_process() at usb_process+0xc3/frame 0xfffffe003e597bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e597bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e597bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100108 td 0xfffff8002504b6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e59ca60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e59caa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e59cae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e59cb10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e59cb70 usb_process() at usb_process+0xc3/frame 0xfffffe003e59cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e59cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e59cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100109 td 0xfffff8002504b000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5a1a60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5a1aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5a1ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e5a1b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e5a1b70 usb_process() at usb_process+0xc3/frame 0xfffffe003e5a1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5a1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5a1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100110 td 0xfffff800250496e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5a6a60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5a6aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5a6ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e5a6b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e5a6b70 usb_process() at usb_process+0xc3/frame 0xfffffe003e5a6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5a6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5a6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100111 td 0xfffff80025049000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5aba60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5abaa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5abae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e5abb10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e5abb70 usb_process() at usb_process+0xc3/frame 0xfffffe003e5abbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5abbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5abbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100091 td 0xfffff80003be8000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e52fa10 mi_switch() at mi_switch+0x182/frame 0xfffffe003e52fa50 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e52fa90 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e52fac0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e52fb60 xpt_done_td() at xpt_done_td+0x8b/frame 0xfffffe003e52fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e52fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e52fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100092 td 0xfffff80003be96e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e534a10 mi_switch() at mi_switch+0x182/frame 0xfffffe003e534a50 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e534a90 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e534ac0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e534b60 xpt_done_td() at xpt_done_td+0x8b/frame 0xfffffe003e534bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e534bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e534bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100093 td 0xfffff80003be9000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e539a10 mi_switch() at mi_switch+0x182/frame 0xfffffe003e539a50 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e539a90 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e539ac0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e539b60 xpt_done_td() at xpt_done_td+0x8b/frame 0xfffffe003e539bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e539bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e539bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100125 td 0xfffff800261bd6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00d3209a00 mi_switch() at mi_switch+0x182/frame 0xfffffe00d3209a40 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00d3209a80 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d3209ab0 _sleep() at _sleep+0x2cd/frame 0xfffffe00d3209b50 xpt_scanner_thread() at xpt_scanner_thread+0x92/frame 0xfffffe00d3209bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00d3209bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d3209bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 11 pid 19 tid 100090 td 0xfffff80003a876e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e52a9f0 mi_switch() at mi_switch+0x182/frame 0xfffffe003e52aa30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e52aa70 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e52aaa0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e52ab40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe003e52abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e52abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e52abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 10 pid 18 tid 100089 td 0xfffff80003a8b000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5259f0 mi_switch() at mi_switch+0x182/frame 0xfffffe003e525a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e525a70 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e525aa0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e525b40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe003e525bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e525bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e525bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 9 pid 17 tid 100088 td 0xfffff80003a8b6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5209f0 mi_switch() at mi_switch+0x182/frame 0xfffffe003e520a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e520a70 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e520aa0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e520b40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe003e520bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e520bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e520bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 8 pid 16 tid 100087 td 0xfffff80003a8a000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e51b9f0 mi_switch() at mi_switch+0x182/frame 0xfffffe003e51ba30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e51ba70 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e51baa0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e51bb40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe003e51bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e51bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e51bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 7 pid 15 tid 100086 td 0xfffff80003a8a6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5169f0 mi_switch() at mi_switch+0x182/frame 0xfffffe003e516a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e516a70 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e516aa0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e516b40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe003e516bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e516bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e516bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 6 pid 9 tid 100085 td 0xfffff80003a7f000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5119f0 mi_switch() at mi_switch+0x182/frame 0xfffffe003e511a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e511a70 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e511aa0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e511b40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe003e511bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e511bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e511bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 5 pid 8 tid 100084 td 0xfffff80003a7f6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e50c9f0 mi_switch() at mi_switch+0x182/frame 0xfffffe003e50ca30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e50ca70 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e50caa0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e50cb40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe003e50cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e50cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e50cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 4 pid 7 tid 100083 td 0xfffff80003a7e000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5079f0 mi_switch() at mi_switch+0x182/frame 0xfffffe003e507a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e507a70 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e507aa0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e507b40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe003e507bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e507bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e507bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 3 pid 6 tid 100082 td 0xfffff80003a7e6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5029f0 mi_switch() at mi_switch+0x182/frame 0xfffffe003e502a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e502a70 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e502aa0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e502b40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe003e502bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e502bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e502bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 2 pid 5 tid 100081 td 0xfffff80003a87000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4fd9f0 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4fda30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4fda70 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4fdaa0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4fdb40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe003e4fdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4fdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4fdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 1 pid 4 tid 100080 td 0xfffff80003a706e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4f89f0 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4f8a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4f8a70 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4f8aa0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4f8b40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe003e4f8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4f8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4f8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 0 pid 3 tid 100079 td 0xfffff80003a6f000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4f39f0 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4f3a30 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4f3a70 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4f3aa0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4f3b40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe003e4f3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4f3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4f3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 2 tid 100078 td 0xfffff80003a6f6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4eea20 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4eea60 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4eeaa0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4eead0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4eeb70 crypto_proc() at crypto_proc+0x2bf/frame 0xfffffe003e4eebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4eebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4eebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sequencer 00 pid 14 tid 100065 td 0xfffff800039ef000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4ada20 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4ada60 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4adaa0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4adad0 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe003e4adb30 seq_eventthread() at seq_eventthread+0x8c/frame 0xfffffe003e4adbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4adbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4adbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100062 td 0xfffff800039f56e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e492a20 mi_switch() at mi_switch+0x182/frame 0xfffffe003e492a60 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e492aa0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e492ad0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e492b70 g_run_events() at g_run_events+0x45d/frame 0xfffffe003e492bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e492bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e492bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100063 td 0xfffff800039f5000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e497a20 mi_switch() at mi_switch+0x182/frame 0xfffffe003e497a60 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e497aa0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e497ad0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e497b70 g_io_schedule_up() at g_io_schedule_up+0x4c/frame 0xfffffe003e497ba0 g_up_procbody() at g_up_procbody+0x5c/frame 0xfffffe003e497bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e497bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e497bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100064 td 0xfffff800039ef6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e49ca20 mi_switch() at mi_switch+0x182/frame 0xfffffe003e49ca60 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e49caa0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e49cad0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e49cb70 g_io_schedule_down() at g_io_schedule_down+0x4c/frame 0xfffffe003e49cba0 g_down_procbody() at g_down_procbody+0x5c/frame 0xfffffe003e49cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e49cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e49cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100018 td 0xfffff800038136e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00addbdb10 mi_switch() at mi_switch+0x182/frame 0xfffffe00addbdb50 ithread_loop() at ithread_loop+0x39e/frame 0xfffffe00addbdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00addbdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00addbdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100019 td 0xfffff80003813000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00addc2b10 mi_switch() at mi_switch+0x182/frame 0xfffffe00addc2b50 ithread_loop() at ithread_loop+0x39e/frame 0xfffffe00addc2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00addc2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00addc2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100021 td 0xfffff8000381a000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100047 td 0xfffff800038e5000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377feb10 mi_switch() at mi_switch+0x182/frame 0xfffffe00377feb50 ithread_loop() at ithread_loop+0x39e/frame 0xfffffe00377febb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377febf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377febf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100048 td 0xfffff800038e66e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100049 td 0xfffff800038e6000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100050 td 0xfffff800038e96e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100051 td 0xfffff8000398b000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100052 td 0xfffff8000398c6e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100053 td 0xfffff8000398c000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100054 td 0xfffff8000398d6e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100055 td 0xfffff8000398d000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100056 td 0xfffff800039966e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100057 td 0xfffff80003996000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100058 td 0xfffff800039976e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100059 td 0xfffff80003997000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100060 td 0xfffff8000398b6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e440b10 mi_switch() at mi_switch+0x182/frame 0xfffffe003e440b50 ithread_loop() at ithread_loop+0x39e/frame 0xfffffe003e440bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e440bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e440bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100094 td 0xfffff80003fa56e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e53eb10 mi_switch() at mi_switch+0x182/frame 0xfffffe003e53eb50 ithread_loop() at ithread_loop+0x39e/frame 0xfffffe003e53ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e53ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e53ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100095 td 0xfffff80003fa5000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e543b10 mi_switch() at mi_switch+0x182/frame 0xfffffe003e543b50 ithread_loop() at ithread_loop+0x39e/frame 0xfffffe003e543bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e543bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e543bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100101 td 0xfffff8000411a000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e561b10 mi_switch() at mi_switch+0x182/frame 0xfffffe003e561b50 ithread_loop() at ithread_loop+0x39e/frame 0xfffffe003e561bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e561bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e561bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100112 td 0xfffff800250666e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100113 td 0xfffff80025066000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5b5b10 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5b5b50 ithread_loop() at ithread_loop+0x39e/frame 0xfffffe003e5b5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5b5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5b5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100180 td 0xfffff800261bc6e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100181 td 0xfffff8005958d6e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100182 td 0xfffff8005958d000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100183 td 0xfffff8005958e6e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100184 td 0xfffff8005958e000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100185 td 0xfffff8003c50a6e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100186 td 0xfffff80059591000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100187 td 0xfffff800595926e0 fork_trampoline() at fork_trampoline Tracing command idle pid 11 tid 100003 td 0xfffff800037d8000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00add72a70 mi_switch() at mi_switch+0x182/frame 0xfffffe00add72ab0 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00add72ad0 sched_idletd() at sched_idletd+0x401/frame 0xfffffe00add72bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add72bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add72bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xfffff800037d76e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00add77a70 mi_switch() at mi_switch+0x182/frame 0xfffffe00add77ab0 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00add77ad0 sched_idletd() at sched_idletd+0x401/frame 0xfffffe00add77bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add77bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add77bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffff800037d7000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00add7ca70 mi_switch() at mi_switch+0x182/frame 0xfffffe00add7cab0 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00add7cad0 sched_idletd() at sched_idletd+0x401/frame 0xfffffe00add7cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add7cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add7cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xfffff800037eb6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00add81a70 mi_switch() at mi_switch+0x182/frame 0xfffffe00add81ab0 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00add81ad0 sched_idletd() at sched_idletd+0x401/frame 0xfffffe00add81bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add81bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add81bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100007 td 0xfffff800037eb000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00add86a70 mi_switch() at mi_switch+0x182/frame 0xfffffe00add86ab0 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00add86ad0 sched_idletd() at sched_idletd+0x401/frame 0xfffffe00add86bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add86bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add86bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xfffff800037e66e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00add8ba90 mi_switch() at mi_switch+0x182/frame 0xfffffe00add8bad0 sched_idletd() at sched_idletd+0x459/frame 0xfffffe00add8bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add8bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add8bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xfffff800037e6000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00add90a70 mi_switch() at mi_switch+0x182/frame 0xfffffe00add90ab0 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00add90ad0 sched_idletd() at sched_idletd+0x401/frame 0xfffffe00add90bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add90bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add90bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xfffff800037d96e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00add95a70 mi_switch() at mi_switch+0x182/frame 0xfffffe00add95ab0 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00add95ad0 sched_idletd() at sched_idletd+0x401/frame 0xfffffe00add95bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add95bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add95bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffff800037ee000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00add9aa90 mi_switch() at mi_switch+0x182/frame 0xfffffe00add9aad0 sched_idletd() at sched_idletd+0xa3/frame 0xfffffe00add9abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add9abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add9abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100012 td 0xfffff800037ed6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00add9fa90 mi_switch() at mi_switch+0x182/frame 0xfffffe00add9fad0 sched_idletd() at sched_idletd+0x459/frame 0xfffffe00add9fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add9fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add9fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xfffff800037ed000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00adda4a70 mi_switch() at mi_switch+0x182/frame 0xfffffe00adda4ab0 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00adda4ad0 sched_idletd() at sched_idletd+0x401/frame 0xfffffe00adda4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adda4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adda4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100014 td 0xfffff800037ec6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00adda9a70 mi_switch() at mi_switch+0x182/frame 0xfffffe00adda9ab0 critical_exit_preempt() at critical_exit_preempt+0x63/frame 0xfffffe00adda9ad0 sched_idletd() at sched_idletd+0x401/frame 0xfffffe00adda9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adda9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adda9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command init pid 1 tid 100002 td 0xfffff800037d86e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00add6d690 mi_switch() at mi_switch+0x182/frame 0xfffffe00add6d6d0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00add6d710 sleepq_catch_signals() at sleepq_catch_signals+0x415/frame 0xfffffe00add6d770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00add6d7a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00add6d840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00add6d8d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00add6dac0 amd64_syscall() at amd64_syscall+0x2f1/frame 0xfffffe00add6dbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00add6dbf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x2b0c2a, rsp = 0x7fffffffe808, rbp = 0x7fffffffe870 --- Tracing command audit pid 10 tid 100001 td 0xfffff800037d9000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00add68a40 mi_switch() at mi_switch+0x182/frame 0xfffffe00add68a80 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00add68ac0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00add68af0 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00add68b50 audit_worker() at audit_worker+0x9f/frame 0xfffffe00add68bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add68bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add68bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xffffffff81d9bbe0 sched_switch() at sched_switch+0x38f/frame 0xffffffff822fabf0 mi_switch() at mi_switch+0x182/frame 0xffffffff822fac30 sleepq_switch() at sleepq_switch+0x10e/frame 0xffffffff822fac70 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xffffffff822facb0 _sleep() at _sleep+0x26c/frame 0xffffffff822fad50 swapper() at swapper+0x75/frame 0xffffffff822fada0 mi_startup() at mi_startup+0x2ed/frame 0xffffffff822fadf0 btext() at btext+0x2c Tracing command kernel pid 0 tid 100015 td 0xfffff800037ec000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00addaea60 mi_switch() at mi_switch+0x182/frame 0xfffffe00addaeaa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00addaeae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00addaeb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00addaeb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00addaebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00addaebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00addaebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xfffff800038146e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00addb3a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00addb3a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00addb3ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00addb3ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00addb3b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00addb3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00addb3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00addb3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100017 td 0xfffff80003814000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00addb8a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00addb8a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00addb8ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00addb8ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00addb8b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00addb8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00addb8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00addb8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xfffff800037ee6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00addc7a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00addc7a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00addc7ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00addc7ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00addc7b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00addc7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00addc7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00addc7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xfffff800038196e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe0037781a60 mi_switch() at mi_switch+0x182/frame 0xfffffe0037781aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe0037781ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0037781b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe0037781b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe0037781bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037781bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037781bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xfffff80003819000 sched_switch() at sched_switch+0x38f/frame 0xfffffe0037786a60 mi_switch() at mi_switch+0x182/frame 0xfffffe0037786aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe0037786ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0037786b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe0037786b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe0037786bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037786bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037786bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xfffff800038186e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003778ba60 mi_switch() at mi_switch+0x182/frame 0xfffffe003778baa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003778bae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003778bb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe003778bb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe003778bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003778bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003778bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xfffff80003818000 sched_switch() at sched_switch+0x38f/frame 0xfffffe0037790a60 mi_switch() at mi_switch+0x182/frame 0xfffffe0037790aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe0037790ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0037790b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe0037790b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe0037790bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037790bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037790bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100026 td 0xfffff8000384d6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe0037795a60 mi_switch() at mi_switch+0x182/frame 0xfffffe0037795aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe0037795ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0037795b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe0037795b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe0037795bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037795bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037795bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100027 td 0xfffff8000384d000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003779aa60 mi_switch() at mi_switch+0x182/frame 0xfffffe003779aaa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003779aae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003779ab10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe003779ab80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe003779abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003779abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003779abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100028 td 0xfffff8000384c6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003779fa60 mi_switch() at mi_switch+0x182/frame 0xfffffe003779faa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003779fae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003779fb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe003779fb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe003779fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003779fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003779fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100029 td 0xfffff8000384c000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377a4a60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377a4aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377a4ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377a4b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377a4b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377a4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377a4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377a4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100030 td 0xfffff8000381a6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377a9a60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377a9aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377a9ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377a9b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377a9b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377a9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377a9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377a9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100031 td 0xfffff80003852000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377aea60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377aeaa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377aeae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377aeb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377aeb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377aebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377aebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377aebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100032 td 0xfffff800038516e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377b3a60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377b3aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377b3ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377b3b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377b3b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377b3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377b3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377b3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100033 td 0xfffff80003851000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377b8a60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377b8aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377b8ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377b8b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377b8b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377b8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377b8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377b8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100034 td 0xfffff800038506e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377bda60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377bdaa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377bdae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377bdb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377bdb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377bdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377bdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377bdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100035 td 0xfffff80003850000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377c2a60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377c2aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377c2ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377c2b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377c2b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377c2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377c2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377c2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100036 td 0xfffff800038706e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377c7a60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377c7aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377c7ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377c7b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377c7b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377c7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377c7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377c7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100037 td 0xfffff80003870000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377cca60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377ccaa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377ccae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377ccb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377ccb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377ccbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377ccbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377ccbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100038 td 0xfffff800038716e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377d1a60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377d1aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377d1ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377d1b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377d1b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377d1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377d1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377d1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100039 td 0xfffff80003871000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377d6a60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377d6aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377d6ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377d6b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377d6b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377d6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377d6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377d6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100040 td 0xfffff800038526e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377dba60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377dbaa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377dbae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377dbb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377dbb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377dbbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377dbbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377dbbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100041 td 0xfffff800038e9000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377e0a60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377e0aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377e0ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377e0b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377e0b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377e0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377e0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377e0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100042 td 0xfffff8000386e6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377e5a60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377e5aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377e5ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377e5b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377e5b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377e5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377e5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377e5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100043 td 0xfffff8000386e000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377eaa60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377eaaa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377eaae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377eab10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377eab80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377eabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377eabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377eabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100044 td 0xfffff8000386f6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377efa60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377efaa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377efae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377efb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377efb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377efbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377efbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377efbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100045 td 0xfffff8000386f000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377f4a60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377f4aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377f4ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377f4b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377f4b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377f4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377f4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377f4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100046 td 0xfffff800038e56e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe00377f9a60 mi_switch() at mi_switch+0x182/frame 0xfffffe00377f9aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00377f9ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00377f9b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377f9b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377f9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377f9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377f9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100061 td 0xfffff800039f4000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e48da30 mi_switch() at mi_switch+0x182/frame 0xfffffe003e48da70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e48dab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e48dae0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e48db80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe003e48dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e48dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e48dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100066 td 0xfffff80003a6d6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4b2a30 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4b2a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4b2ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4b2ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4b2b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe003e4b2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4b2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4b2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100067 td 0xfffff80003a6d000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4b7a30 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4b7a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4b7ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4b7ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4b7b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe003e4b7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4b7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4b7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100068 td 0xfffff80003a6e6e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4bca30 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4bca70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4bcab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4bcae0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4bcb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe003e4bcbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4bcbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4bcbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100069 td 0xfffff80003a6e000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4c1a30 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4c1a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4c1ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4c1ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4c1b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe003e4c1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4c1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4c1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100070 td 0xfffff800039f46e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4c6a30 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4c6a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4c6ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4c6ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4c6b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe003e4c6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4c6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4c6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100071 td 0xfffff80003a70000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4cba30 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4cba70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4cbab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4cbae0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4cbb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe003e4cbbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4cbbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4cbbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100072 td 0xfffff80003a716e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4d0a30 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4d0a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4d0ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4d0ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4d0b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe003e4d0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4d0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4d0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100073 td 0xfffff80003a71000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4d5a30 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4d5a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4d5ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4d5ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4d5b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe003e4d5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4d5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4d5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100074 td 0xfffff80003a726e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4daa30 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4daa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4daab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4daae0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4dab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe003e4dabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4dabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4dabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100075 td 0xfffff80003a72000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4dfa30 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4dfa70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4dfab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4dfae0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4dfb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe003e4dfbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4dfbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4dfbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100076 td 0xfffff80003a786e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4e4a30 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4e4a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4e4ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4e4ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4e4b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe003e4e4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4e4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4e4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100077 td 0xfffff80003a78000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e4e9a30 mi_switch() at mi_switch+0x182/frame 0xfffffe003e4e9a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e4e9ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e4e9ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe003e4e9b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe003e4e9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e4e9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e4e9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100114 td 0xfffff800250676e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5baa60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5baaa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5baae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e5bab10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe003e5bab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe003e5babb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5babf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5babf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100121 td 0xfffff800261686e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5f5a60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5f5aa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5f5ae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e5f5b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe003e5f5b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe003e5f5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5f5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5f5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100122 td 0xfffff80026168000 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5faa60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5faaa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5faae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e5fab10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe003e5fab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe003e5fabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5fabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5fabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100123 td 0xfffff800260906e0 sched_switch() at sched_switch+0x38f/frame 0xfffffe003e5ffa60 mi_switch() at mi_switch+0x182/frame 0xfffffe003e5ffaa0 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe003e5ffae0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe003e5ffb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe003e5ffb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe003e5ffbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003e5ffbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003e5ffbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100124 td 0xfffff800261bc000 sched_switch() at sched_switch+0x38f/frame 0xfffffe00d3204a30 mi_switch() at mi_switch+0x182/frame 0xfffffe00d3204a70 sleepq_switch() at sleepq_switch+0x10e/frame 0xfffffe00d3204ab0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d3204ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00d3204b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00d3204bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00d3204bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d3204bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db:0:pho> dump Dumping 1688 out of 32626 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Dump complete db:0:pho> show allchains chain 1: thread 100821 (pid 77773, sh) sleeping on 0xfffff801ef1778e8 "piperd" chain 2: thread 102157 (pid 75353, swap) sleeping on 0xfffff807a6ce9e30 "swread" chain 3: thread 102200 (pid 75343, swap) sleeping on 0xfffff806ba658530 "wait" chain 4: thread 102152 (pid 75340, swap) sleeping on 0xffffffff81c7f2a7 "nanslp" chain 5: thread 100145 (pid 75328, sh) sleeping on 0xfffff80004115530 "wait" chain 6: thread 102241 (pid 75327, tee) sleeping on 0xfffff8025fd895f0 "piperd" chain 7: thread 102170 (pid 75326, sh) sleeping on 0xfffff806ba407530 "wait" chain 8: thread 100396 (pid 8322, sh) sleeping on 0xfffff801b530f000 "wait" chain 9: thread 100147 (pid 2472, sh) sleeping on 0xfffff80026d92000 "piperd" chain 10: thread 100216 (pid 2470, awk) sleeping on 0xfffff80026d922f8 "piperd" chain 11: thread 100217 (pid 2469, sh) sleeping on 0xfffff80004117000 "wait" chain 12: thread 100153 (pid 2434, sh) sleeping on 0xfffff80026165000 "wait" chain 13: thread 100214 (pid 2351, bash) sleeping on 0xfffff800260c1ac0 "select" chain 14: thread 100213 (pid 2349, csh) sleeping on 0xfffff80026c475d8 "pause" chain 15: thread 100205 (pid 2348, su) blocked on lockmgr 0&EXCL thread 0 (pid -512, ) can run chain 16: thread 100143 (pid 2343, bash) blocked on lockmgr `z&EXCL thread 0 (pid -512, ) ??? (0x6) chain 17: thread 100157 (pid 2342, sshd) sleeping on 0xfffff8002508d640 "select" chain 18: thread 100202 (pid 2339, sshd) sleeping on 0xfffff80026cfacc0 "select" chain 19: thread 100171 (pid 2335, csh) sleeping on 0xfffff8002509b8b0 "ttyin" chain 20: thread 100196 (pid 2334, login) sleeping on 0xfffff80026737000 "wait" chain 21: thread 100167 (pid 2333, getty) sleeping on 0xfffff800250a3cb0 "ttyin" chain 22: thread 100211 (pid 2332, getty) sleeping on 0xfffff800261d88b0 "ttyin" chain 23: thread 100210 (pid 2331, getty) sleeping on 0xfffff800261d70b0 "ttyin" chain 24: thread 100209 (pid 2330, getty) sleeping on 0xfffff800261d78b0 "ttyin" chain 25: thread 100208 (pid 2329, getty) sleeping on 0xfffff800250a68b0 "ttyin" chain 26: thread 100207 (pid 2328, getty) sleeping on 0xfffff800261220b0 "ttyin" chain 27: thread 100206 (pid 2327, getty) sleeping on 0xfffff800261228b0 "ttyin" chain 28: thread 100170 (pid 2326, getty) sleeping on 0xfffff800261210b0 "ttyin" chain 29: thread 100142 (pid 2325, getty) sleeping on 0xfffff800261218b0 "ttyin" chain 30: thread 100176 (pid 2279, cron) sleeping on 0xffffffff81c7f2a0 "nanslp" chain 31: thread 100166 (pid 2275, sshd) sleeping on 0xfffff80026b98f40 "select" chain 32: thread 100201 (pid 2247, powerd) sleeping on 0xfffff800039e8b40 "select" chain 33: thread 100177 (pid 2244, ntpd) sleeping on 0xfffff80026c3edc0 "select" chain 34: thread 100677 (pid 2244, ntpd) blocked on lockmgr (null)EXCL chain 35: thread 100188 (pid 2181, NLM: master) sleeping on 0xfffff80026177a60 "rpcsvc" chain 36: thread 100198 (pid 2178, rpc.statd) sleeping on 0xfffff800260c1bc0 "select" chain 37: thread 100193 (pid 2132, watchdogd) sleeping on 0xffffffff81c7f2a2 "nanslp" chain 38: thread 100190 (pid 2053, rpcbind) sleeping on 0xfffff800269911c0 "select" chain 39: thread 100163 (pid 2043, syslogd) sleeping on 0xfffffe003b02f038 "biord" chain 40: thread 100159 (pid 1972, devd) sleeping on 0xfffff80026d1a1c0 "select" chain 41: thread 100191 (pid 1967, dhclient) sleeping on 0xfffff80026da5240 "select" chain 42: thread 100189 (pid 1920, dhclient) sleeping on 0xfffff80026991340 "select" chain 43: thread 100168 (pid 1917, dhclient) sleeping on 0xfffff8003c4ef2c0 "select" chain 44: thread 100141 (pid 34, syncer) sleeping on 0xffffffff81d55760 "syncer" chain 45: thread 100138 (pid 33, vnlru) blocked on lockmgr `&EXCL thread 0 (pid -512, ) is inactive chain 46: thread 100130 (pid 32, bufdaemon) sleeping on 0xffffffff81d54c18 "qsleep" chain 47: thread 100132 (pid 32, bufspacedaemon-0) sleeping on 0xffffffff8180a900 "-" chain 48: thread 100134 (pid 32, bufspacedaemon-1) sleeping on 0xffffffff81812bc0 "-" chain 49: thread 100135 (pid 32, bufspacedaemon-2) sleeping on 0xffffffff8181ae80 "-" chain 50: thread 100136 (pid 32, bufspacedaemon-3) sleeping on 0xffffffff81823140 "-" chain 51: thread 100137 (pid 32, bufspacedaemon-4) sleeping on 0xffffffff8182b400 "-" chain 52: thread 100139 (pid 32, bufspacedaemon-5) sleeping on 0xffffffff818336c0 "-" chain 53: thread 100140 (pid 32, bufspacedaemon-6) sleeping on 0xffffffff8183b980 "-" chain 54: thread 100152 (pid 32, / worker) blocked on lockmgr 0&EXCL thread 0 (pid -1, ) is inactive chain 55: thread 100200 (pid 32, /tmp worker) sleeping on 0xfffff800262fa6e8 "sdflush" chain 56: thread 100131 (pid 30, laundry: dom0) sleeping on 0xffffffff81d9d4e4 "launds" chain 57: thread 100133 (pid 30, uma) sleeping on 0xffffffff80f08830 "umarcl" chain 58: thread 100127 (pid 29, enc_daemon1) blocked on lockmgr 0&EXCL thread 0 (pid -512, ) is inactive chain 59: thread 100126 (pid 28, enc_daemon0) blocked on lockmgr `&EXCL thread 0 (pid -512, ) is inactive chain 60: thread 100120 (pid 27, rand_harvestq) sleeping on 0xffffffff81adaf18 "-" chain 61: thread 100119 (pid 26, sctp_iterator) sleeping on 0xffffffff81de2ab0 "waiting_for_work" chain 62: thread 100118 (pid 25, soaiod4) sleeping on 0xffffffff81d5461c "-" chain 63: thread 100117 (pid 24, soaiod3) sleeping on 0xffffffff81d5461c "-" chain 64: thread 100116 (pid 23, soaiod2) sleeping on 0xffffffff81d5461c "-" chain 65: thread 100115 (pid 22, soaiod1) sleeping on 0xffffffff81d5461c "-" chain 66: thread 100096 (pid 21, usbus0) sleeping on 0xfffffe00d1584460 "-" chain 67: thread 100097 (pid 21, usbus0) sleeping on 0xfffffe00d15844b8 "-" chain 68: thread 100098 (pid 21, usbus0) sleeping on 0xfffffe00d1584510 "-" chain 69: thread 100099 (pid 21, usbus0) sleeping on 0xfffffe00d1584568 "-" chain 70: thread 100100 (pid 21, usbus0) sleeping on 0xfffffe00d15845c0 "-" chain 71: thread 100102 (pid 21, usbus1) sleeping on 0xfffffe0037b66d10 "-" chain 72: thread 100103 (pid 21, usbus1) sleeping on 0xfffffe0037b66d68 "-" chain 73: thread 100104 (pid 21, usbus1) sleeping on 0xfffffe0037b66dc0 "-" chain 74: thread 100105 (pid 21, usbus1) sleeping on 0xfffffe0037b66e18 "-" chain 75: thread 100106 (pid 21, usbus1) sleeping on 0xfffffe0037b66e70 "-" chain 76: thread 100107 (pid 21, usbus2) sleeping on 0xfffffe00d16a7d10 "-" chain 77: thread 100108 (pid 21, usbus2) sleeping on 0xfffffe00d16a7d68 "-" chain 78: thread 100109 (pid 21, usbus2) sleeping on 0xfffffe00d16a7dc0 "-" chain 79: thread 100110 (pid 21, usbus2) sleeping on 0xfffffe00d16a7e18 "-" chain 80: thread 100111 (pid 21, usbus2) sleeping on 0xfffffe00d16a7e70 "-" chain 81: thread 100091 (pid 20, doneq0) sleeping on 0xffffffff819b62c0 "-" chain 82: thread 100093 (pid 20, doneq2) sleeping on 0xffffffff819b63c0 "-" chain 83: thread 100125 (pid 20, scanner) sleeping on 0xffffffff819b6188 "-" chain 84: thread 100090 (pid 19, crypto returns 11) sleeping on 0xfffff80003a74450 "crypto_ret_wait" chain 85: thread 100089 (pid 18, crypto returns 10) sleeping on 0xfffff80003a743f0 "crypto_ret_wait" chain 86: thread 100088 (pid 17, crypto returns 9) sleeping on 0xfffff80003a74390 "crypto_ret_wait" chain 87: thread 100087 (pid 16, crypto returns 8) sleeping on 0xfffff80003a74330 "crypto_ret_wait" chain 88: thread 100086 (pid 15, crypto returns 7) sleeping on 0xfffff80003a742d0 "crypto_ret_wait" chain 89: thread 100085 (pid 9, crypto returns 6) sleeping on 0xfffff80003a74270 "crypto_ret_wait" chain 90: thread 100084 (pid 8, crypto returns 5) sleeping on 0xfffff80003a74210 "crypto_ret_wait" chain 91: thread 100083 (pid 7, crypto returns 4) sleeping on 0xfffff80003a741b0 "crypto_ret_wait" chain 92: thread 100082 (pid 6, crypto returns 3) sleeping on 0xfffff80003a74150 "crypto_ret_wait" chain 93: thread 100081 (pid 5, crypto returns 2) sleeping on 0xfffff80003a740f0 "crypto_ret_wait" chain 94: thread 100080 (pid 4, crypto returns 1) sleeping on 0xfffff80003a74090 "crypto_ret_wait" chain 95: thread 100079 (pid 3, crypto returns 0) sleeping on 0xfffff80003a74030 "crypto_ret_wait" chain 96: thread 100078 (pid 2, crypto) sleeping on 0xffffffff81d6a6f8 "crypto_wait" chain 97: thread 100065 (pid 14, sequencer 00) sleeping on 0xfffff800039f7c88 "seqstate" chain 98: thread 100062 (pid 13, g_event) sleeping on 0xffffffff81d9bad0 "-" chain 99: thread 100063 (pid 13, g_up) sleeping on 0xffffffff81d9bae0 "-" chain 100: thread 100064 (pid 13, g_down) sleeping on 0xffffffff81d9bad8 "-" chain 101: thread 100002 (pid 1, init) sleeping on 0xfffff800037d6530 "wait" chain 102: thread 100001 (pid 10, audit) sleeping on 0xffffffff81de3740 "audit_worker_cv" chain 103: thread 100000 (pid 0, swapper) sleeping on 0xffffffff81d8a1c8 "swapin" chain 104: thread 100015 (pid 0, config_0) sleeping on 0xfffff800037afe00 "-" chain 105: thread 100016 (pid 0, kqueue_ctx taskq) sleeping on 0xfffff800037b8e00 "-" chain 106: thread 100017 (pid 0, aiod_kick taskq) sleeping on 0xfffff800037b8c00 "-" chain 107: thread 100020 (pid 0, thread taskq) sleeping on 0xfffff800037b8600 "-" chain 108: thread 100022 (pid 0, if_config_tqg_0) sleeping on 0xfffff800037afd00 "-" chain 109: thread 100023 (pid 0, softirq_0) sleeping on 0xfffff800037afc00 "-" chain 110: thread 100024 (pid 0, softirq_1) sleeping on 0xfffff800037afb00 "-" chain 111: thread 100025 (pid 0, softirq_2) sleeping on 0xfffff800037afa00 "-" chain 112: thread 100026 (pid 0, softirq_3) sleeping on 0xfffff800037af900 "-" chain 113: thread 100027 (pid 0, softirq_4) sleeping on 0xfffff800037af800 "-" chain 114: thread 100028 (pid 0, softirq_5) sleeping on 0xfffff800037af700 "-" chain 115: thread 100029 (pid 0, softirq_6) sleeping on 0xfffff800037af600 "-" chain 116: thread 100030 (pid 0, softirq_7) sleeping on 0xfffff800037af500 "-" chain 117: thread 100031 (pid 0, softirq_8) sleeping on 0xfffff800037af400 "-" chain 118: thread 100032 (pid 0, softirq_9) sleeping on 0xfffff800037af300 "-" chain 119: thread 100033 (pid 0, softirq_10) sleeping on 0xfffff800037af200 "-" chain 120: thread 100034 (pid 0, softirq_11) sleeping on 0xfffff800037af100 "-" chain 121: thread 100035 (pid 0, if_io_tqg_0) sleeping on 0xfffff800037af000 "-" chain 122: thread 100036 (pid 0, if_io_tqg_1) sleeping on 0xfffff800037aee00 "-" chain 123: thread 100037 (pid 0, if_io_tqg_2) sleeping on 0xfffff800037aed00 "-" chain 124: thread 100038 (pid 0, if_io_tqg_3) sleeping on 0xfffff800037aec00 "-" chain 125: thread 100039 (pid 0, if_io_tqg_4) sleeping on 0xfffff800037aeb00 "-" chain 126: thread 100040 (pid 0, if_io_tqg_5) sleeping on 0xfffff800037aea00 "-" chain 127: thread 100041 (pid 0, if_io_tqg_6) sleeping on 0xfffff800037ae900 "-" chain 128: thread 100042 (pid 0, if_io_tqg_7) sleeping on 0xfffff800037ae800 "-" chain 129: thread 100043 (pid 0, if_io_tqg_8) sleeping on 0xfffff800037ae700 "-" chain 130: thread 100044 (pid 0, if_io_tqg_9) sleeping on 0xfffff800037ae600 "-" chain 131: thread 100045 (pid 0, if_io_tqg_10) sleeping on 0xfffff800037ae500 "-" chain 132: thread 100046 (pid 0, if_io_tqg_11) sleeping on 0xfffff800037ae400 "-" chain 133: thread 100061 (pid 0, firmware taskq) sleeping on 0xfffff8000398fa00 "-" chain 134: thread 100066 (pid 0, crypto_0) sleeping on 0xfffff8000398f300 "-" chain 135: thread 100067 (pid 0, crypto_1) sleeping on 0xfffff8000398f300 "-" chain 136: thread 100068 (pid 0, crypto_2) sleeping on 0xfffff8000398f300 "-" chain 137: thread 100069 (pid 0, crypto_3) sleeping on 0xfffff8000398f300 "-" chain 138: thread 100070 (pid 0, crypto_4) sleeping on 0xfffff8000398f300 "-" chain 139: thread 100071 (pid 0, crypto_5) sleeping on 0xfffff8000398f300 "-" chain 140: thread 100072 (pid 0, crypto_6) sleeping on 0xfffff8000398f300 "-" chain 141: thread 100073 (pid 0, crypto_7) sleeping on 0xfffff8000398f300 "-" chain 142: thread 100074 (pid 0, crypto_8) sleeping on 0xfffff8000398f300 "-" chain 143: thread 100075 (pid 0, crypto_9) sleeping on 0xfffff8000398f300 "-" chain 144: thread 100076 (pid 0, crypto_10) sleeping on 0xfffff8000398f300 "-" chain 145: thread 100077 (pid 0, crypto_11) sleeping on 0xfffff8000398f300 "-" chain 146: thread 100114 (pid 0, mca taskq) sleeping on 0xfffff800250b9d00 "-" chain 147: thread 100121 (pid 0, acpi_task_0) blocked on lockmgr (null)EXCL chain 148: thread 100122 (pid 0, acpi_task_1) blocked on lockmgr (null)EXCL chain 149: thread 100123 (pid 0, acpi_task_2) blocked on lockmgr (null)EXCL chain 150: thread 100124 (pid 0, CAM taskq) sleeping on 0xfffff80003991600 "-" db:0:pho> chain 1: thread 100821 (pid 77773, sh) sleeping on 0xfffff801ef1778e8 "piperd" chain 2: thread 102157 (pid 75353, swap) sleeping on 0xfffff807a6ce9e30 "swread" chain 3: thread 102200 (pid 75343, swap) sleeping on 0xfffff806ba658530 "wait" chain 4: thread 102152 (pid 75340, swap) sleeping on 0xffffffff81c7f2a7 "nanslp" chain 5: thread 100145 (pid 75328, sh) sleeping on 0xfffff80004115530 "wait" chain 6: thread 102241 (pid 75327, tee) sleeping on 0xfffff8025fd895f0 "piperd" chain 7: thread 102170 (pid 75326, sh) sleeping on 0xfffff806ba407530 "wait" chain 8: thread 100396 (pid 8322, sh) sleeping on 0xfffff801b530f000 "wait" chain 9: thread 100147 (pid 2472, sh) sleeping on 0xfffff80026d92000 "piperd" chain 10: thread 100216 (pid 2470, awk) sleeping on 0xfffff80026d922f8 "piperd" chain 11: thread 100217 (pid 2469, sh) sleeping on 0xfffff80004117000 "wait" chain 12: thread 100153 (pid 2434, sh) sleeping on 0xfffff80026165000 "wait" chain 13: thread 100214 (pid 2351, bash) sleeping on 0xfffff800260c1ac0 "select" chain 14: thread 100213 (pid 2349, csh) sleeping on 0xfffff80026c475d8 "pause" chain 15: thread 100205 (pid 2348, su) blocked on lockmgr 0&EXCL thread 0 (pid -512, ) can run chain 16: thread 100143 (pid 2343, bash) blocked on lockmgr `z&EXCL thread 0 (pid -512, ) ??? (0x6) chain 17: thread 100157 (pid 2342, sshd) sleeping on 0xfffff8002508d640 "select" chain 18: thread 100202 (pid 2339, sshd) sleeping on 0xfffff80026cfacc0 "select" chain 19: thread 100171 (pid 2335, csh) sleeping on 0xfffff8002509b8b0 "ttyin" chain 20: thread 100196 (pid 2334, login) sleeping on 0xfffff80026737000 "wait" chain 21: thread 100167 (pid 2333, getty) sleeping on 0xfffff800250a3cb0 "ttyin" chain 22: thread 100211 (pid 2332, getty) sleeping on 0xfffff800261d88b0 "ttyin" chain 23: thread 100210 (pid 2331, getty) sleeping on 0xfffff800261d70b0 "ttyin" chain 24: thread 100209 (pid 2330, getty) sleeping on 0xfffff800261d78b0 "ttyin" chain 25: thread 100208 (pid 2329, getty) sleeping on 0xfffff800250a68b0 "ttyin" chain 26: thread 100207 (pid 2328, getty) sleeping on 0xfffff800261220b0 "ttyin" chain 27: thread 100206 (pid 2327, getty) sleeping on 0xfffff800261228b0 "ttyin" chain 28: thread 100170 (pid 2326, getty) sleeping on 0xfffff800261210b0 "ttyin" chain 29: thread 100142 (pid 2325, getty) sleeping on 0xfffff800261218b0 "ttyin" chain 30: thread 100176 (pid 2279, cron) sleeping on 0xffffffff81c7f2a0 "nanslp" chain 31: thread 100166 (pid 2275, sshd) sleeping on 0xfffff80026b98f40 "select" chain 32: thread 100201 (pid 2247, powerd) sleeping on 0xfffff800039e8b40 "select" chain 33: thread 100177 (pid 2244, ntpd) sleeping on 0xfffff80026c3edc0 "select" chain 34: thread 100677 (pid 2244, ntpd) blocked on lockmgr (null)EXCL chain 35: thread 100188 (pid 2181, NLM: master) sleeping on 0xfffff80026177a60 "rpcsvc" chain 36: thread 100198 (pid 2178, rpc.statd) sleeping on 0xfffff800260c1bc0 "select" chain 37: thread 100193 (pid 2132, watchdogd) sleeping on 0xffffffff81c7f2a2 "nanslp" chain 38: thread 100190 (pid 2053, rpcbind) sleeping on 0xfffff800269911c0 "select" chain 39: thread 100163 (pid 2043, syslogd) sleeping on 0xfffffe003b02f038 "biord" chain 40: thread 100159 (pid 1972, devd) sleeping on 0xfffff80026d1a1c0 "select" chain 41: thread 100191 (pid 1967, dhclient) sleeping on 0xfffff80026da5240 "select" chain 42: thread 100189 (pid 1920, dhclient) sleeping on 0xfffff80026991340 "select" chain 43: thread 100168 (pid 1917, dhclient) sleeping on 0xfffff8003c4ef2c0 "select" chain 44: thread 100141 (pid 34, syncer) sleeping on 0xffffffff81d55760 "syncer" chain 45: thread 100138 (pid 33, vnlru) blocked on lockmgr `&EXCL thread 0 (pid -512, ) is inactive chain 46: thread 100130 (pid 32, bufdaemon) sleeping on 0xffffffff81d54c18 "qsleep" chain 47: thread 100132 (pid 32, bufspacedaemon-0) sleeping on 0xffffffff8180a900 "-" chain 48: thread 100134 (pid 32, bufspacedaemon-1) sleeping on 0xffffffff81812bc0 "-" chain 49: thread 100135 (pid 32, bufspacedaemon-2) sleeping on 0xffffffff8181ae80 "-" chain 50: thread 100136 (pid 32, bufspacedaemon-3) sleeping on 0xffffffff81823140 "-" chain 51: thread 100137 (pid 32, bufspacedaemon-4) sleeping on 0xffffffff8182b400 "-" chain 52: thread 100139 (pid 32, bufspacedaemon-5) sleeping on 0xffffffff818336c0 "-" chain 53: thread 100140 (pid 32, bufspacedaemon-6) sleeping on 0xffffffff8183b980 "-" chain 54: thread 100152 (pid 32, / worker) blocked on lockmgr 0&EXCL thread 0 (pid -1, ) is inactive chain 55: thread 100200 (pid 32, /tmp worker) sleeping on 0xfffff800262fa6e8 "sdflush" chain 56: thread 100131 (pid 30, laundry: dom0) sleeping on 0xffffffff81d9d4e4 "launds" chain 57: thread 100133 (pid 30, uma) sleeping on 0xffffffff80f08830 "umarcl" chain 58: thread 100127 (pid 29, enc_daemon1) blocked on lockmgr 0&EXCL thread 0 (pid -512, ) is inactive chain 59: thread 100126 (pid 28, enc_daemon0) blocked on lockmgr `&EXCL thread 0 (pid -512, ) is inactive chain 60: thread 100120 (pid 27, rand_harvestq) sleeping on 0xffffffff81adaf18 "-" chain 61: thread 100119 (pid 26, sctp_iterator) sleeping on 0xffffffff81de2ab0 "waiting_for_work" chain 62: thread 100118 (pid 25, soaiod4) sleeping on 0xffffffff81d5461c "-" chain 63: thread 100117 (pid 24, soaiod3) sleeping on 0xffffffff81d5461c "-" chain 64: thread 100116 (pid 23, soaiod2) sleeping on 0xffffffff81d5461c "-" chain 65: thread 100115 (pid 22, soaiod1) sleeping on 0xffffffff81d5461c "-" chain 66: thread 100096 (pid 21, usbus0) sleeping on 0xfffffe00d1584460 "-" chain 67: thread 100097 (pid 21, usbus0) sleeping on 0xfffffe00d15844b8 "-" chain 68: thread 100098 (pid 21, usbus0) sleeping on 0xfffffe00d1584510 "-" chain 69: thread 100099 (pid 21, usbus0) sleeping on 0xfffffe00d1584568 "-" chain 70: thread 100100 (pid 21, usbus0) sleeping on 0xfffffe00d15845c0 "-" chain 71: thread 100102 (pid 21, usbus1) sleeping on 0xfffffe0037b66d10 "-" chain 72: thread 100103 (pid 21, usbus1) sleeping on 0xfffffe0037b66d68 "-" chain 73: thread 100104 (pid 21, usbus1) sleeping on 0xfffffe0037b66dc0 "-" chain 74: thread 100105 (pid 21, usbus1) sleeping on 0xfffffe0037b66e18 "-" chain 75: thread 100106 (pid 21, usbus1) sleeping on 0xfffffe0037b66e70 "-" chain 76: thread 100107 (pid 21, usbus2) sleeping on 0xfffffe00d16a7d10 "-" chain 77: thread 100108 (pid 21, usbus2) sleeping on 0xfffffe00d16a7d68 "-" chain 78: thread 100109 (pid 21, usbus2) sleeping on 0xfffffe00d16a7dc0 "-" chain 79: thread 100110 (pid 21, usbus2) sleeping on 0xfffffe00d16a7e18 "-" chain 80: thread 100111 (pid 21, usbus2) sleeping on 0xfffffe00d16a7e70 "-" chain 81: thread 100091 (pid 20, doneq0) sleeping on 0xffffffff819b62c0 "-" chain 82: thread 100093 (pid 20, doneq2) sleeping on 0xffffffff819b63c0 "-" chain 83: thread 100125 (pid 20, scanner) sleeping on 0xffffffff819b6188 "-" chain 84: thread 100090 (pid 19, crypto returns 11) sleeping on 0xfffff80003a74450 "crypto_ret_wait" chain 85: thread 100089 (pid 18, crypto returns 10) sleeping on 0xfffff80003a743f0 "crypto_ret_wait" chain 86: thread 100088 (pid 17, crypto returns 9) sleeping on 0xfffff80003a74390 "crypto_ret_wait" chain 87: thread 100087 (pid 16, crypto returns 8) sleeping on 0xfffff80003a74330 "crypto_ret_wait" chain 88: thread 100086 (pid 15, crypto returns 7) sleeping on 0xfffff80003a742d0 "crypto_ret_wait" chain 89: thread 100085 (pid 9, crypto returns 6) sleeping on 0xfffff80003a74270 "crypto_ret_wait" chain 90: thread 100084 (pid 8, crypto returns 5) sleeping on 0xfffff80003a74210 "crypto_ret_wait" chain 91: thread 100083 (pid 7, crypto returns 4) sleeping on 0xfffff80003a741b0 "crypto_ret_wait" chain 92: thread 100082 (pid 6, crypto returns 3) sleeping on 0xfffff80003a74150 "crypto_ret_wait" chain 93: thread 100081 (pid 5, crypto returns 2) sleeping on 0xfffff80003a740f0 "crypto_ret_wait" chain 94: thread 100080 (pid 4, crypto returns 1) sleeping on 0xfffff80003a74090 "crypto_ret_wait" chain 95: thread 100079 (pid 3, crypto returns 0) sleeping on 0xfffff80003a74030 "crypto_ret_wait" chain 96: thread 100078 (pid 2, crypto) sleeping on 0xffffffff81d6a6f8 "crypto_wait" chain 97: thread 100065 (pid 14, sequencer 00) sleeping on 0xfffff800039f7c88 "seqstate" chain 98: thread 100062 (pid 13, g_event) sleeping on 0xffffffff81d9bad0 "-" chain 99: thread 100063 (pid 13, g_up) sleeping on 0xffffffff81d9bae0 "-" chain 100: thread 100064 (pid 13, g_down) sleeping on 0xffffffff81d9bad8 "-" chain 101: thread 100002 (pid 1, init) sleeping on 0xfffff800037d6530 "wait" chain 102: thread 100001 (pid 10, audit) sleeping on 0xffffffff81de3740 "audit_worker_cv" chain 103: thread 100000 (pid 0, swapper) sleeping on 0xffffffff81d8a1c8 "swapin" chain 104: thread 100015 (pid 0, config_0) sleeping on 0xfffff800037afe00 "-" chain 105: thread 100016 (pid 0, kqueue_ctx taskq) sleeping on 0xfffff800037b8e00 "-" chain 106: thread 100017 (pid 0, aiod_kick taskq) sleeping on 0xfffff800037b8c00 "-" chain 107: thread 100020 (pid 0, thread taskq) sleeping on 0xfffff800037b8600 "-" chain 108: thread 100022 (pid 0, if_config_tqg_0) sleeping on 0xfffff800037afd00 "-" chain 109: thread 100023 (pid 0, softirq_0) sleeping on 0xfffff800037afc00 "-" chain 110: thread 100024 (pid 0, softirq_1) sleeping on 0xfffff800037afb00 "-" chain 111: thread 100025 (pid 0, softirq_2) sleeping on 0xfffff800037afa00 "-" chain 112: thread 100026 (pid 0, softirq_3) sleeping on 0xfffff800037af900 "-" chain 113: thread 100027 (pid 0, softirq_4) sleeping on 0xfffff800037af800 "-" chain 114: thread 100028 (pid 0, softirq_5) sleeping on 0xfffff800037af700 "-" chain 115: thread 100029 (pid 0, softirq_6) sleeping on 0xfffff800037af600 "-" chain 116: thread 100030 (pid 0, softirq_7) sleeping on 0xfffff800037af500 "-" chain 117: thread 100031 (pid 0, softirq_8) sleeping on 0xfffff800037af400 "-" chain 118: thread 100032 (pid 0, softirq_9) sleeping on 0xfffff800037af300 "-" chain 119: thread 100033 (pid 0, softirq_10) sleeping on 0xfffff800037af200 "-" chain 120: thread 100034 (pid 0, softirq_11) sleeping on 0xfffff800037af100 "-" chain 121: thread 100035 (pid 0, if_io_tqg_0) sleeping on 0xfffff800037af000 "-" chain 122: thread 100036 (pid 0, if_io_tqg_1) sleeping on 0xfffff800037aee00 "-" chain 123: thread 100037 (pid 0, if_io_tqg_2) sleeping on 0xfffff800037aed00 "-" chain 124: thread 100038 (pid 0, if_io_tqg_3) sleeping on 0xfffff800037aec00 "-" chain 125: thread 100039 (pid 0, if_io_tqg_4) sleeping on 0xfffff800037aeb00 "-" chain 126: thread 100040 (pid 0, if_io_tqg_5) sleeping on 0xfffff800037aea00 "-" chain 127: thread 100041 (pid 0, if_io_tqg_6) sleeping on 0xfffff800037ae900 "-" chain 128: thread 100042 (pid 0, if_io_tqg_7) sleeping on 0xfffff800037ae800 "-" chain 129: thread 100043 (pid 0, if_io_tqg_8) sleeping on 0xfffff800037ae700 "-" chain 130: thread 100044 (pid 0, if_io_tqg_9) sleeping on 0xfffff800037ae600 "-" chain 131: thread 100045 (pid 0, if_io_tqg_10) sleeping on 0xfffff800037ae500 "-" chain 132: thread 100046 (pid 0, if_io_tqg_11) sleeping on 0xfffff800037ae400 "-" chain 133: thread 100061 (pid 0, firmware taskq) sleeping on 0xfffff8000398fa00 "-" chain 134: thread 100066 (pid 0, crypto_0) sleeping on 0xfffff8000398f300 "-" chain 135: thread 100067 (pid 0, crypto_1) sleeping on 0xfffff8000398f300 "-" chain 136: thread 100068 (pid 0, crypto_2) sleeping on 0xfffff8000398f300 "-" chain 137: thread 100069 (pid 0, crypto_3) sleeping on 0xfffff8000398f300 "-" chain 138: thread 100070 (pid 0, crypto_4) sleeping on 0xfffff8000398f300 "-" chain 139: thread 100071 (pid 0, crypto_5) sleeping on 0xfffff8000398f300 "-" chain 140: thread 100072 (pid 0, crypto_6) sleeping on 0xfffff8000398f300 "-" chain 141: thread 100073 (pid 0, crypto_7) sleeping on 0xfffff8000398f300 "-" chain 142: thread 100074 (pid 0, crypto_8) sleeping on 0xfffff8000398f300 "-" chain 143: thread 100075 (pid 0, crypto_9) sleeping on 0xfffff8000398f300 "-" chain 144: thread 100076 (pid 0, crypto_10) sleeping on 0xfffff8000398f300 "-" chain 145: thread 100077 (pid 0, crypto_11) sleeping on 0xfffff8000398f300 "-" chain 146: thread 100114 (pid 0, mca taskq) sleeping on 0xfffff800250b9d00 "-" chain 147: thread 100121 (pid 0, acpi_task_0) blocked on lockmgr (null)EXCL chain 148: thread 100122 (pid 0, acpi_task_1) blocked on lockmgr (null)EXCL chain 149: thread 100123 (pid 0, acpi_task_2) blocked on lockmgr (null)EXCL chain 150: thread 100124 (pid 0, CAM taskq) sleeping on 0xfffff80003991600 "-" db> (kgdb) bt #0 __curthread () at /var/tmp/deviant2/sys/amd64/include/pcpu_aux.h:55 #1 doadump (textdump=0x0) at ../../../kern/kern_shutdown.c:392 #2 0xffffffff8049a9ea in db_dump (dummy=, dummy2=, dummy3=, dummy4=) at ../../../ddb/db_command.c:575 #3 0xffffffff8049a7ac in db_command (last_cmdp=, cmd_table=, dopager=0x0) at ../../../ddb/db_command.c:482 #4 0xffffffff8049f778 in db_script_exec (scriptname=, warnifnotfound=) at ../../../ddb/db_script.c:304 #5 0xffffffff8049a7ac in db_command (last_cmdp=, cmd_table=, dopager=0x1) at ../../../ddb/db_command.c:482 #6 0xffffffff8049a51d in db_command_loop () at ../../../ddb/db_command.c:535 #7 0xffffffff8049d748 in db_trap (type=, code=) at ../../../ddb/db_main.c:252 #8 0xffffffff80c153a7 in kdb_trap (type=0x3, code=0x0, tf=) at ../../../kern/subr_kdb.c:691 #9 0xffffffff8106c8d4 in trap (frame=0xfffffe02f94a4430) at ../../../amd64/amd64/trap.c:585 #10 #11 kdb_enter (why=0xffffffff811f956c "panic", msg=) at ../../../kern/subr_kdb.c:478 #12 0xffffffff80bc98aa in vpanic (fmt=, ap=) at ../../../kern/kern_shutdown.c:897 #13 0xffffffff80bc9643 in panic (fmt=0xffffffff81c78a48 "\366\333\033\201\377\377\377\377") at ../../../kern/kern_shutdown.c:835 #14 0xffffffff8106cd16 in trap_fatal (frame=0xfffffe02f94a47e0, eva=0x0) at ../../../amd64/amd64/trap.c:926 #15 0xffffffff8106cdb9 in trap_pfault (frame=, usermode=, signo=, ucode=0x0) at ../../../amd64/amd64/trap.c:743 #16 0xffffffff8106c3c0 in trap (frame=0xfffffe02f94a47e0) at ../../../amd64/amd64/trap.c:407 #17 #18 0xffffffff80f01279 in swap_pager_copy (srcobject=0xfffff8003c593630, dstobject=0xfffff801a0440210, offset=0x0, destroysource=) at ../../../vm/swap_pager.c:1004 #19 0xffffffff80f20051 in vm_object_collapse (object=0xfffff801a0440210) at ../../../vm/vm_object.c:1841 #20 0xffffffff80f1fccd in vm_object_deallocate (object=0xfffff801a0440210) at ../../../vm/vm_object.c:662 #21 0xffffffff80f1382d in vm_map_entry_deallocate (entry=, system_map=) at ../../../vm/vm_map.c:3484 #22 vm_map_process_deferred () at ../../../vm/vm_map.c:612 #23 0xffffffff80f1978b in _vm_map_unlock (map=, file=, line=) at ../../../vm/vm_map.c:669 #24 vm_map_remove (map=, start=0x1000, end=0x800000000000) at ../../../vm/vm_map.c:3686 #25 0xffffffff80f13155 in vmspace_dofree (vm=) at ../../../vm/vm_map.c:336 #26 vmspace_exit (td=0xfffff806ba27c000) at ../../../vm/vm_map.c:417 #27 0xffffffff80b822c2 in exit1 (td=, rval=, signo=0x0) at ../../../kern/kern_exit.c:416 #28 0xffffffff80b81d1d in sys_sys_exit (td=0xffffffff81d6bfe0 , uap=) at ../../../kern/kern_exit.c:195 #29 0xffffffff8106d8c1 in syscallenter (td=0xfffff806ba27c000) at ../../../amd64/amd64/../../kern/subr_syscall.c:144 #30 amd64_syscall (td=0xfffff806ba27c000, traced=) at ../../../amd64/amd64/trap.c:1163 #31 #32 0x00000008003d880a in ?? () Backtrace stopped: Cannot access memory at address 0x7fffffffeb08 (kgdb) f 18 #18 0xffffffff80f01279 in swap_pager_copy (srcobject=0xfffff8003c593630, dstobject=0xfffff801a0440210, offset=0x0, destroysource=) at ../../../vm/swap_pager.c:1004 1004 TAILQ_REMOVE(NOBJLIST(srcobject->handle), srcobject, (kgdb) l 999 vm_object_pip_add(srcobject, 1); 1000 VM_OBJECT_WUNLOCK(srcobject); 1001 vm_object_pip_add(dstobject, 1); 1002 VM_OBJECT_WUNLOCK(dstobject); 1003 sx_xlock(&sw_alloc_sx); 1004 TAILQ_REMOVE(NOBJLIST(srcobject->handle), srcobject, 1005 pager_object_list); 1006 sx_xunlock(&sw_alloc_sx); 1007 VM_OBJECT_WLOCK(dstobject); 1008 vm_object_pip_wakeup(dstobject); (kgdb) info loc No locals. (kgdb) p *srcobject $1 = {lock = {lock_object = {lo_name = 0xffffffff81154eac "vm object", lo_flags = 0x25630000, lo_data = 0x0, lo_witness = 0xfffff8087fd6b780}, rw_lock = 0x1}, object_list = {tqe_next = 0xfffff8003c593738, tqe_prev = 0xfffff8003c593548}, shadow_head = {lh_first = 0xfffff801a0440210}, shadow_list = {le_next = 0x0, le_prev = 0xfffff8005982a660}, memq = {tqh_first = 0x0, tqh_last = 0xfffff8003c593678}, rtree = {rt_root = 0x0}, size = 0x6, domain = {dr_policy = 0x0, dr_iter = 0x0}, generation = 0x1, cleangeneration = 0x1, ref_count = 0x1, shadow_count = 0x1, memattr = 0x6, type = 0x1, flags = 0x1018, pg_color = 0x1e3, paging_in_progress = 0x2, busy = 0x0, resident_page_count = 0x0, backing_object = 0xfffff80026648108, backing_object_offset = 0x1e3000, pager_object_list = {tqe_next = 0x0, tqe_prev = 0x0}, rvq = {lh_first = 0x0}, handle = 0xfffff80026648108, un_pager = {vnp = {vnp_size = 0x0, writemappings = 0xfffff800262b82a9}, devp = {devp_pglist = {tqh_first = 0x0, tqh_last = 0xfffff800262b82a9}, ops = 0x0, dev = 0x0}, sgp = {sgp_pglist = {tqh_first = 0x0, tqh_last = 0xfffff800262b82a9}}, swp = { swp_tmpfs = 0x0, swp_blks = {pt_root = 0xfffff800262b82a9}, writemappings = 0x0}}, cred = 0xfffff800265d6000, charge = 0x6000, umtx_data = 0x0} (kgdb) p *dstobject $2 = {lock = {lock_object = {lo_name = 0xffffffff81154eac "vm object", lo_flags = 0x25630000, lo_data = 0x0, lo_witness = 0xfffff8087fd6b780}, rw_lock = 0x1}, object_list = {tqe_next = 0xfffff801a0440318, tqe_prev = 0xfffff801a0440128}, shadow_head = {lh_first = 0x0}, shadow_list = { le_next = 0x0, le_prev = 0xfffff8003c593660}, memq = {tqh_first = 0xfffffe000a5a3558, tqh_last = 0xfffffe000a5a3290}, rtree = {rt_root = 0xfffff8022d86f7e0}, size = 0x6, domain = {dr_policy = 0x0, dr_iter = 0x0}, generation = 0x1, cleangeneration = 0x1, ref_count = 0x2, shadow_count = 0x0, memattr = 0x6, type = 0x0, flags = 0x7010, pg_color = 0x1e3, paging_in_progress = 0x2, busy = 0x0, resident_page_count = 0x5, backing_object = 0xfffff8003c593630, backing_object_offset = 0x0, pager_object_list = {tqe_next = 0x0, tqe_prev = 0x0}, rvq = { lh_first = 0x0}, handle = 0xfffff80026648108, un_pager = {vnp = {vnp_size = 0x0, writemappings = 0x0}, devp = {devp_pglist = {tqh_first = 0x0, tqh_last = 0x0}, ops = 0x0, dev = 0x0}, sgp = {sgp_pglist = {tqh_first = 0x0, tqh_last = 0x0}}, swp = {swp_tmpfs = 0x0, swp_blks = { pt_root = 0x0}, writemappings = 0x0}}, cred = 0xfffff800265d6000, charge = 0x6000, umtx_data = 0x0} (kgdb) -- Test scenario: misc/execve.sh