panic: vm_fault_lookup: fault on nofault entry, addr: 0xfffffe02ba488000, vmcore.164 KDB: debugger backends: ddb KDB: current backend: ddb ---<>--- Copyright (c) 1992-2020 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 13.0-CURRENT #1 r357853M: Thu Feb 13 08:19:46 CET 2020 pho@mercat1.netperf.freebsd.org:/usr/src/sys/amd64/compile/PHO amd64 FreeBSD clang version 9.0.1 (git@github.com:llvm/llvm-project.git c1a0a213378a458fbea1a5c77b315c7dce08fd05) (based on LLVM 9.0.1) WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. VT(vga): resolution 640x480 CPU: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz (3500.07-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306f2 Family=0x6 Model=0x3f Stepping=2 Features=0xbfebfbff Features2=0x7ffefbff AMD Features=0x2c100800 AMD Features2=0x21 Structured Extended Features=0x37ab XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics real memory = 34359738368 (32768 MB) avail memory = 33164111872 (31627 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs FreeBSD/SMP: 1 package(s) x 6 core(s) x 2 hardware threads random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" random: unblocking device. ioapic0 irqs 0-23 ioapic1 irqs 24-47 Launching APs: 1 9 5 2 8 11 3 7 10 6 4 Timecounter "TSC-low" frequency 1750034486 Hz quality 1000 random: entropy device external interface WARNING: Device "kbd" is Giant locked and may be deleted before FreeBSD 13.0. kbd1 at kbdmux0 000.000055 [4341] netmap_init netmap: loaded module [ath_hal] loaded nexus0 vtvga0: cryptosoft0: acpi0: acpi0: Power Button (fixed) cpu0: on acpi0 atrtc0: port 0x70-0x71,0x74-0x77 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Event timer "HPET4" frequency 14318180 Hz quality 340 Event timer "HPET5" frequency 14318180 Hz quality 340 Event timer "HPET6" frequency 14318180 Hz quality 340 Event timer "HPET7" frequency 14318180 Hz quality 340 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: on acpi0 pci0: on pcib0 pci0: at device 11.1 (no driver attached) pci0: at device 11.2 (no driver attached) pci0: at device 16.1 (no driver attached) pci0: at device 16.6 (no driver attached) pci0: at device 18.1 (no driver attached) acpi_syscontainer0: on acpi0 acpi_syscontainer1: on acpi0 acpi_syscontainer2: on acpi0 acpi_syscontainer3: on acpi0 pcib1: port 0xcf8-0xcff on acpi0 pci1: on pcib1 pcib2: irq 26 at device 1.0 on pci1 pci2: on pcib2 pcib3: irq 32 at device 2.0 on pci1 pci3: on pcib3 pci3: at device 0.0 (no driver attached) pci3: at device 0.1 (no driver attached) pcib4: irq 40 at device 3.0 on pci1 pci4: on pcib4 pci1: at device 17.0 (no driver attached) ahci0: port 0xf110-0xf117,0xf100-0xf103,0xf0f0-0xf0f7,0xf0e0-0xf0e3,0xf020-0xf03f mem 0xfb438000-0xfb4387ff irq 16 at device 17.4 on pci1 ahci0: AHCI v1.30 with 4 6Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahciem0: on ahci0 xhci0: mem 0xfb400000-0xfb40ffff irq 19 at device 20.0 on pci1 xhci0: 32 bytes context size, 64-bit DMA usbus0 on xhci0 usbus0: 5.0Gbps Super Speed USB v3.0 pci1: at device 22.0 (no driver attached) pci1: at device 22.1 (no driver attached) ehci0: mem 0xfb434000-0xfb4343ff irq 18 at device 26.0 on pci1 usbus1: EHCI version 1.0 usbus1 on ehci0 usbus1: 480Mbps High Speed USB v2.0 pcib5: irq 16 at device 28.0 on pci1 pci5: on pcib5 pcib6: irq 18 at device 28.2 on pci1 pci6: on pcib6 pcib7: at device 0.0 on pci6 pci7: on pcib7 vgapci0: port 0xe000-0xe07f mem 0xfa000000-0xfaffffff,0xfb000000-0xfb01ffff irq 18 at device 0.0 on pci7 vgapci0: Boot video device pcib8: irq 16 at device 28.4 on pci1 pci8: on pcib8 igb0: port 0xd020-0xd03f mem 0xfb320000-0xfb33ffff,0xfb344000-0xfb347fff irq 16 at device 0.0 on pci8 igb0: Using 1024 TX descriptors and 1024 RX descriptors igb0: queue equality overrideugen0.3: at usbus0 ukbd0 on uhub3 ukbd0: on usbus0 kbd2 at ukbd0 . lo0: link state changed to UP Autoloading module: ioat.ko ioat0: mem 0xfb42c000-0xfb42ffff irq 31 at device 4.0 on pci1 ioat0: Capabilities: 2f7 ioat1: mem 0xfb428000-0xfb42bfff irq 39 at device 4.1 on pci1 ioat1: Capabilities: 2f7 ioat2: mem 0xfb424000-0xfb427fff irq 31 at device 4.2 on pci1 ioat2: Capabilities: f7 ioat3: mem 0xfb420000-0xfb423fff irq 39 at device 4.3 on pci1 ioat3: Capabilities: f7 ioat4: mem 0xfb41c000-0xfb41ffff irq 31 at device 4.4 on pci1 ioat4: Capabilities: f7 ioat5: mem 0xfb418000-0xfb41bfff irq 39 at device 4.5 on pci1 ioat5: Capabilities: f7 ioat6: mem 0xfb414000-0xfb417fff irq 31 at device 4.6 on pci1 ioat6: Capabilities: f7 ioat7: mem 0xfb410000-0xfb413fff irq 39 at device 4.7 on pci1 ioat7: Capabilities: f7 Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Starting Network: igb1. igb1: flags=8822 metric 0 mtu 1500 options=4e507bb ether 0c:c4:7a:a8:cd:db media: Ethernet autoselect status: no carrier nd6 options=29 Autoloading module: uhid.ko Autoloading module: ums.ko ums0 on uhub3 ums0: on usbus0 ums0: 3 buttons and [Z] coordinates ID=0 Starting dhclient. if_delmulti_locked: detaching ifnet instance 0xfffff80004196800 DHCPDISCOVER on igb0 to 255.255.255.255 port 67 interval 4 DHCPOFFER from 192.168.5.1 DHCPREQUEST on igb0 to 255.255.255.255 port 67 DHCPACK from 192.168.5.1 bound to 192.168.5.171 -- renewal in 300 seconds. add host 127.0.0.1: gateway lo0 fib 0: route already in table add host ::1: gateway lo0 fib 0: route already in table add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Creating and/or trimming log files. Starting syslogd. Setting date via ntp. 13 Feb 17:41:41 ntpdate[2075]: step time server 192.168.5.1 offset 0.120021 sec Starting rpcbind. NFS access cache time=60 Starting watchdogd. Feb 13 17:41:42 mercat1 savecore[2174]: reboot Clearing /tmp (X related). Starting statd. Starting lockd. Recovering vi editor sessions:. fscklock order reversal: 1st 0xfffff80020771a68 ufs (ufs) @ kern/vfs_mount.c:1628 2nd 0xfffff800206d2a68 devfs (devfs) @ ufs/ffs/ffs_vfsops.c:1404 stack backtrace: #0 0xffffffff80c36c81 at witness_debugger+0x71 #1 0xffffffff80c369f6 at witness_checkorder+0xad6 #2 0xffffffff80b9ba08 at lockmgr_lock_fast_path+0x178 #3 0xffffffff810f1ec0 at VOP_LOCK1_APV+0x40 #4 0xffffffff80cb9804 at _vn_lock+0x54 #5 0xffffffff80ee72d3 at ffs_flushfiles+0xa3 #6 0xffffffff80ec911d at softdep_flushfiles+0xad #7 0xffffffff80ee9b17 at ffs_unmount+0xd7 #8 0xffffffff80c9b73a at dounmount+0x42a #9 0xffffffff80c9b2b9 at kern_unmount+0x2b9 #10 0xffffffff81074afe at amd64_syscall+0x2ee #11 0xffffffff8104a460 at fast_syscall_common+0x101 ** /dev/gpt/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes PARTIALLY TRUNCATED INODE I=223 SALVAGE? yes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts UNREF FILE I=223 OWNER=root MODE=100644 SIZE=3339703 MTIME=Feb 13 16:18 2020 CLEAR? yes ** Phase 5 - Check Cyl groups FREE BLK COUNT(S) WRONG IN SUPERBLK SALVAGE? yes SUMMARY INFORMATION BAD SALVAGE? yes BLK(S) MISSING IN BIT MAPS SALVAGE? yes 293 files, 1028 used, 1264379 free (139 frags, 158030 blocks, 0.0% fragmentation) ***** FILE SYSTEM MARKED CLEAN ***** ***** FILE SYSTEM WAS MODIFIED ***** Updating motd:. Mounting late filesystems:. Security policy loaded: MAC/ntpd (mac_ntpd) Starting ntpd. Starting powerd. Configuring vt: blanktime. Performing sanity check on sshd configuration. Starting sshd. Starting cron. Thu Feb 13 17:42 FreeBSD/amd64 (mercat1.netperf.freebsd.org) (ttyu1) login: root Feb 13 17:43:30 mercat1 login[2367]: ROOT LOGIN (root) ON ttyu1 Last login: Thu Feb 13 11:50:41 on ttyu1 FreeBSD 13.0-CURRENT (PHO) #1 r357853M: Thu Feb 13 08:19:46 CET 2020 root@mercat1:~ # Feb 13 17:43:38 mercat1 su[2384]: pho to root on /dev/pts/0 20200213 17:51:05 all (1/610): nfs.sh 20200213 17:51:07 all (2/610): nfs12.sh 20200213 17:51:08 all (3/610): nfs14.sh 20200213 17:51:10 all (4/610): nfs15.sh 20200213 17:51:11 all (5/610): nfs15lockd.sh 20200213 17:51:12 all (6/610): nfs15lockd3.sh 20200213 17:51:14 all (7/610): nfs3.sh 20200213 17:51:15 all (8/610): nfs4.sh 20200213 17:51:17 all (9/610): nfs7.sh 20200213 17:51:18 all (10/610): nfs8.sh 20200213 17:51:19 all (11/610): nfs9.sh 20200213 17:51:21 all (12/610): nfs_halfpage.sh 20200213 17:51:24 all (13/610): nfs_halfpage2.sh 20200213 17:51:26 all (14/610): nfsdelegation.sh 20200213 17:51:27 all (15/610): nfsdepth.sh 20200213 17:51:28 all (16/610): nfsrename.sh 20200213 17:51:30 all (17/610): nfssillyrename.sh 20200213 17:51:31 all (18/610): nullfs13.sh 20200213 17:51:33 all (19/610): nullfs14.sh 20200213 17:51:34 all (20/610): nullfs15.sh 20200213 17:51:36 all (21/610): nullfs21.sh 20200213 17:51:37 all (22/610): nullfs23.sh 20200213 17:51:38 all (23/610): nullfs4.sh mount leak: 13/44 20200213 17:51:40 all (24/610): nullfs6.sh 20200213 17:51:41 all (25/610): nullfs7.sh 20200213 17:51:43 all (26/610): nullfs9.sh 20200213 17:51:44 all (27/610): numa.sh 20200213 17:51:45 all (28/610): oom.sh 20200213 17:51:47 all (29/610): open.sh 20200213 17:51:48 all (30/610): pagefault.sh 20200213 17:51:50 all (31/610): parallelmount2.sh 20200213 17:51:51 all (32/610): pathconf.sh 20200213 17:51:52 all (33/610): pathconf2.sh 20200213 17:51:54 all (34/610): pcatch.sh 20200213 17:51:55 all (35/610): pcatch2.sh 20200213 17:51:57 all (36/610): ping.sh 20200213 17:51:58 all (37/610): pkru.sh 20200213 17:51:59 all (38/610): pkru2.sh 20200213 17:52:01 all (39/610): pmc4.sh 20200213 17:52:02 all (40/610): pmc5.sh 20200213 17:52:04 all (41/610): pmc8.sh 20200213 17:52:05 all (42/610): poll.sh 20200213 17:52:07 all (43/610): posix_fadvise.sh 20200213 17:52:09 all (44/610): posix_fadvise2.sh 20200213 17:52:10 all (45/610): procfs5.sh 20200213 17:52:12 all (46/610): procfs6.sh 20200213 17:52:13 all (47/610): procstat.sh 20200213 17:52:14 all (48/610): pthread9.sh 20200213 17:52:16 all (49/610): ptrace.sh 20200213 17:52:17 all (50/610): ptrace3.sh 20200213 17:52:19 all (51/610): pts2.sh 20200213 17:52:20 all (52/610): quota5.sh 20200213 17:52:22 all (53/610): rename13.sh 20200213 17:52:23 all (54/610): rename4.sh 20200213 17:52:24 all (55/610): revoke.sh 20200213 17:52:26 all (56/610): routetbl.sh 20200213 17:52:27 all (57/610): ruby.sh 20200213 17:52:29 all (58/610): rwlock_ronly.sh 20200213 17:52:30 all (59/610): schedfuzz.sh 20200213 17:52:32 all (60/610): seekdir.sh 20200213 17:52:33 all (61/610): segnp.sh 20200213 17:52:35 all (62/610): segregs.sh 20200213 17:52:36 all (63/610): select3.sh 20200213 17:52:37 all (64/610): sendfile.sh 20200213 17:52:39 all (65/610): sendfile4.sh 20200213 17:52:41 all (66/610): setuid.sh newblk leak: 8/33. 20200213 17:52:42 all (67/610): shm_super.sh 20200213 17:52:44 all (68/610): sigaltstack.sh 20200213 17:52:45 all (69/610): sigreturn.sh 20200213 17:52:47 all (70/610): sigstop.sh 20200213 17:52:48 all (71/610): snap.sh lock order reversal: 1st 0xfffffe0038434760 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:302 2nd 0xfffff80004106330 snaplk (snaplk) @ ufs/ffs/ffs_snapshot.c:2327 stack backtrace: #0 0xffffffff80c36c81 at witness_debugger+0x71 #1 0xffffffff80c369f6 at witness_checkorder+0xad6 #2 0xffffffff80b9bfee at lockmgr_xlock_hard+0x6e #3 0xffffffff80b9cbec at __lockmgr_args+0x62c #4 0xffffffff80ec7f06 at ffs_copyonwrite+0x166 #5 0xffffffff80eeacec at ffs_geom_strategy+0x8c #6 0xffffffff80efe6a3 at ufs_strategy+0x83 #7 0xffffffff810f2331 at VOP_STRATEGY_APV+0x61 #8 0xffffffff80c7e9ae at bufstrategy+0x3e #9 0xffffffff80c7e84a at bufwrite+0x24a #10 0xffffffff80eecc3d at ffs_syncvnode+0x2bd #11 0xffffffff80eebb62 at ffs_fsync+0x22 #12 0xffffffff810f0b59 at VOP_FSYNC_APV+0x59 #13 0xffffffff80efc3d8 at ufs_remove+0xd8 #14 0xffffffff810f0d09 at VOP_REMOVE_APV+0x79 #15 0xffffffff80cb223a at kern_funlinkat+0x2ba #16 0xffffffff80cb1f78 at sys_unlink+0x28 #17 0xffffffff81074afe at amd64_syscall+0x2ee lock order reversal: 1st 0xfffff80004106330 snaplk (snaplk) @ ufs/ufs/ufs_vnops.c:918 2nd 0xfffff8031b88f068 ufs (ufs) @ ufs/ffs/ffs_snapshot.c:1650 stack backtrace: #0 0xffffffff80c36c81 at witness_debugger+0x71 #1 0xffffffff80c369f6 at witness_checkorder+0xad6 #2 0xffffffff80b9bfee at lockmgr_xlock_hard+0x6e #3 0xffffffff80b9cbec at __lockmgr_args+0x62c #4 0xffffffff80ec647d at ffs_snapremove+0xed #5 0xffffffff80ec0bcf at ffs_truncate+0x25f #6 0xffffffff80ef46ef at ufs_inactive+0x1cf #7 0xffffffff810f1a59 at VOP_INACTIVE_APV+0x59 #8 0xffffffff80ca4fe7 at vinactivef+0x107 #9 0xffffffff80ca430d at vput_final+0x28d #10 0xffffffff80cb2290 at kern_funlinkat+0x310 #11 0xffffffff80cb1f78 at sys_unlink+0x28 #12 0xffffffff81074afe at amd64_syscall+0x2ee #13 0xffffffff8104a460 at fast_syscall_common+0x101 20200213 17:52:50 all (72/610): snap12.sh 20200213 17:52:51 all (73/610): snap2-1.sh 20200213 17:52:53 all (74/610): snap2.sh 20200213 17:52:54 all (75/610): sndstat.sh 20200213 17:52:56 all (76/610): socketpair.sh 20200213 17:52:57 all (77/610): socketpair2.sh 20200213 17:52:59 all (78/610): suj32.sh 20200213 17:53:00 all (79/610): suj7.sh Failed to find journal. Use tunefs to create one Failed to start journal: 2 20200213 17:53:02 all (80/610): suj8.sh Feb 13 17:53:03 mercat1 kernel: pid 29405 (dd), uid 0 inumber 4 on /mnt: filesystem full Feb 13 17:53:03 mercat1 kernel: pid 29421 (dd), uid 0 inumber 5 on /mnt: filesystem full 20200213 17:53:03 all (81/610): syzkaller1.sh 20200213 17:53:05 all (82/610): syzkaller2.sh 20200213 17:53:06 all (83/610): syzkaller3.sh 20200213 17:53:08 all (84/610): tar.sh freework leak: 207122/207145. newblk leak: 10624/10657. 20200213 17:59:42 all (85/610): temp.sh 20200213 17:59:45 all (86/610): tmpfs15.sh 20200213 17:59:48 all (87/610): tmpfs4.sh 20200213 17:59:50 all (88/610): tmpfs6.sh 20200213 17:59:52 all (89/610): tmpfs7.sh 20200213 17:59:53 all (90/610): trim2.sh 20200213 17:59:55 all (91/610): trim5.sh 20200213 17:59:57 all (92/610): trim6.sh 20200213 17:59:58 all (93/610): trim7.sh 20200213 18:00:00 all (94/610): trim8.sh 20200213 18:00:01 all (95/610): truncate4.sh 20200213 18:00:02 all (96/610): truncate6.sh 20200213 18:00:04 all (97/610): tvnlru.sh 20200213 18:00:05 all (98/610): uma_zalloc_arg.sh 20200213 18:00:07 all (99/610): unlink.sh 20200213 18:00:08 all (100/610): unload.sh 20200213 18:00:10 all (101/610): unload2.sh 20200213 18:00:11 all (102/610): vfork.sh 20200213 18:00:13 all (103/610): vmio.sh 20200213 18:00:14 all (104/610): watchman.sh 20200213 18:00:15 all (105/610): write.sh newblk leak: 54446/65103. 20200213 18:18:46 all (106/610): beneath2.sh 20200213 18:18:55 all (107/610): execi386.sh 20200213 18:18:58 all (108/610): sendfile2.sh lock order reversal: 1st 0xfffffe00d39d4180 filedesc structure (filedesc structure) @ kern/sys_generic.c:1556 2nd 0xfffff806287a92e8 ufs (ufs) @ kern/vfs_vnops.c:1553 stack backtrace: #0 0xffffffff80c36c81 at witness_debugger+0x71 #1 0xffffffff80c369f6 at witness_checkorder+0xad6 #2 0xffffffff80b9bfee at lockmgr_xlock_hard+0x6e #3 0xffffffff80b9cbec at __lockmgr_args+0x62c #4 0xffffffff80eebcbc at ffs_lock+0x7c #5 0xffffffff810f1ec0 at VOP_LOCK1_APV+0x40 #6 0xffffffff80cb9804 at _vn_lock+0x54 #7 0xffffffff80cb80da at vn_poll+0x3a #8 0xffffffff80c3d339 at kern_poll+0x419 #9 0xffffffff80c3cf10 at sys_poll+0x50 #10 0xffffffff81074afe at amd64_syscall+0x2ee #11 0xffffffff8104a460 at fast_syscall_common+0x101 20200213 18:19:00 all (109/610): sendfile6.sh 20200213 18:19:02 all (110/610): ufssuspend.sh 20200213 18:19:03 all (111/610): contigmalloc3.sh 20200213 18:19:12 all (112/610): datagram.sh 20200213 18:19:14 all (113/610): ext2fs3.sh 20200213 18:19:17 all (114/610): extattr.sh 20200213 18:19:19 all (115/610): fcntl.sh 20200213 18:19:22 all (116/610): fpclone2.sh 20200213 18:19:24 all (117/610): geomleak.sh 20200213 18:19:25 all (118/610): isofs.sh 20200213 18:19:28 all (119/610): kevent2.sh 20200213 18:19:31 all (120/610): killpg.sh 20200213 18:19:33 all (121/610): md9.sh lock order reversal: 1st 0xfffff80020496080 bufwait (bufwait) @ vm/vm_pager.c:391 2nd 0xfffff80692d66068 tmpfs (tmpfs) @ dev/md/md.c:987 stack backtrace: #0 0xffffffff80c36c81 at witness_debugger+0x71 #1 0xffffffff80c369f6 at witness_checkorder+0xad6 #2 0xffffffff80b9d745 at lockmgr_xlock+0x55 #3 0xffffffff810f1ec0 at VOP_LOCK1_APV+0x40 #4 0xffffffff80cb9804 at _vn_lock+0x54 #5 0xffffffff806eca72 at mdstart_vnode+0x462 #6 0xffffffff806ea80c at md_kthread+0x20c #7 0xffffffff80b87170 at fork_exit+0x80 #8 0xffffffff8104ab7e at fork_trampoline+0xe 20200213 18:19:35 all (122/610): mmap25.sh 20200213 18:19:38 all (123/610): mmap26.sh 20200213 18:19:40 all (124/610): mmap28.sh Feb 13 18:19:42 mercat1 kernel: Failed to fully fault in a core file segment at VA 0x800e00000 with size 0x3b5000 to be written at offset 0x86b000 for process mmap28 20200213 18:19:42 all (125/610): mmap8.sh 20200213 18:19:44 all (126/610): mmap9.sh 20200213 18:19:47 all (127/610): mountro2.sh 20200213 18:19:49 all (128/610): msdos9.sh lock order reversal: 1st 0xfffff80781f887e8 msdosfs (msdosfs) @ kern/vfs_subr.c:2922 2nd 0xfffff80781f88a68 devfs (devfs) @ fs/msdosfs/msdosfs_vnops.c:853 stack backtrace: #0 0xffffffff80c36c81 at witness_debugger+0x71 #1 0xffffffff80c369f6 at witness_checkorder+0xad6 #2 0xffffffff80b9ba08 at lockmgr_lock_fast_path+0x178 #3 0xffffffff810f1ec0 at VOP_LOCK1_APV+0x40 #4 0xffffffff80cb9804 at _vn_lock+0x54 #5 0xffffffff80a82349 at msdosfs_fsync+0x49 #6 0xffffffff810f0b59 at VOP_FSYNC_APV+0x59 #7 0xffffffff80a80dc8 at msdosfs_sync+0x148 #8 0xffffffff80a80a0f at msdosfs_unmount+0x2f #9 0xffffffff80c9b73a at dounmount+0x42a #10 0xffffffff80c9b2b9 at kern_unmount+0x2b9 #11 0xffffffff81074afe at amd64_syscall+0x2ee #12 0xffffffff8104a460 at fast_syscall_common+0x101 20200213 18:19:52 all (129/610): newfs5.sh 20200213 18:19:54 all (130/610): nullfs10.sh 20200213 18:19:56 all (131/610): nullfs12.sh 20200213 18:19:59 all (132/610): nullfs16.sh 20200213 18:20:01 all (133/610): nullfs3.sh 20200213 18:20:03 all (134/610): openlock.sh 20200213 18:20:06 all (135/610): ptrace11.sh 20200213 18:20:08 all (136/610): ptrace8.sh 20200213 18:20:11 all (137/610): quota9.sh lock order reversal: 1st 0xfffff8002025b288 FFS (FFS Lock) @ ufs/ufs/ufs_quota.c:759 2nd 0xfffffe00e64df700 struct mount mtx (struct mount mtx) @ ufs/ufs/ufs_quota.c:765 stack backtrace: #0 0xffffffff80c36c81 at witness_debugger+0x71 #1 0xffffffff80c369f6 at witness_checkorder+0xad6 #2 0xffffffff80ba7e24 at __mtx_lock_flags+0x94 #3 0xffffffff80ef913d at quotaoff_inchange+0xbd #4 0xffffffff80ee7356 at ffs_flushfiles+0x126 #5 0xffffffff80ec911d at softdep_flushfiles+0xad #6 0xffffffff80ee9b17 at ffs_unmount+0xd7 #7 0xffffffff80c9b73a at dounmount+0x42a #8 0xffffffff80c9b2b9 at kern_unmount+0x2b9 #9 0xffffffff81074afe at amd64_syscall+0x2ee #10 0xffffffff8104a460 at fast_syscall_common+0x101 20200213 18:20:13 all (138/610): random.sh 20200213 18:20:15 all (139/610): rename6.sh 20200213 18:20:18 all (140/610): scp.sh 20200213 18:20:20 all (141/610): spin.sh 20200213 18:20:22 all (142/610): suj2.sh 20200213 18:20:24 all (143/610): suj35.sh Feb 13 18:20:25 mercat1 kernel: pid 11330 (mkdir), uid 0 inumber 255 on /mnt: out of inodes 20200213 18:20:25 all (144/610): umtx_suspend.sh 20200213 18:20:28 all (145/610): vm_fault_dontneed.sh 20200213 18:20:30 all (146/610): contigmalloc2.sh 20200213 18:20:32 all (147/610): procfs2.sh 20200213 18:20:35 all (148/610): snap10.sh 20200213 18:20:38 all (149/610): datamove4.sh 20200213 18:20:41 all (150/610): devfd.sh 20200213 18:20:44 all (151/610): dup.sh 20200213 18:20:48 all (152/610): extattr3.sh 20200213 18:20:52 all (153/610): mlockall7.sh 20200213 18:20:55 all (154/610): mmap23.sh 20200213 18:20:59 all (155/610): pthread.sh 20200213 18:21:02 all (156/610): rename11.sh 20200213 18:21:06 all (157/610): tmpfs5.sh 20200213 18:21:10 all (158/610): truncate5.sh 20200213 18:21:12 all (159/610): udp.sh 20200213 18:21:16 all (160/610): unix_socket.sh 20200213 18:21:19 all (161/610): truncate7.sh 20200213 18:21:23 all (162/610): kevent8.sh 20200213 18:21:27 all (163/610): linger3.sh 20200213 18:21:31 all (164/610): mkfifo6.sh 20200213 18:21:36 all (165/610): mmap13.sh 20200213 18:21:40 all (166/610): mountro4.sh 20200213 18:21:44 all (167/610): procfs3.sh 20200213 18:21:49 all (168/610): symlink4.sh 20200213 18:21:54 all (169/610): tmpfs10.sh 20200213 18:21:59 all (170/610): procfs.sh 20200213 18:22:04 all (171/610): ptrace2.sh 20200213 18:22:08 all (172/610): datamove5.sh 20200213 18:22:13 all (173/610): growfs.sh g_access(1003): provider ufsid/5e4585c723a04732 has error 6 set g_access(1003): provider ufsid/5e4585c723a04732 has error 6 set g_access(1003): provider ufsid/5e4585c723a04732 has error 6 set g_access(1003): provider gptid/60dfbd31-4e85-11ea-a9e0-0cc47aa8cdda has error 6 set g_access(1003): provider gptid/60dfbd31-4e85-11ea-a9e0-0cc47aa8cdda has error 6 set g_access(1003): provider gptid/60dfbd31-4e85-11ea-a9e0-0cc47aa8cdda has error 6 set g_dev_taste: make_dev_p() failed (gp->name=gptid/60dfbd31-4e85-11ea-a9e0-0cc47aa8cdda, error=17) g_dev_taste: make_dev_p() failed (gp->name=gptid/60dfbd31-4e85-11ea-a9e0-0cc47aa8cdda, error=17) g_dev_taste: make_dev_p() failed (gp->name=ufsid/5e4585c723a04732, error=17) g_dev_taste: make_dev_p() failed (gp->name=gptid/60dfbd31-4e85-11ea-a9e0-0cc47aa8cdda, error=17) g_dev_taste: make_dev_p() failed (gp->name=ufsid/5e4585c723a04732, error=17) g_dev_taste: make_dev_p() failed (gp->name=gptid/60dfbd31-4e85-11ea-a9e0-0cc47aa8cdda, error=17) g_dev_taste: make_dev_p() failed (gp->name=ufsid/5e4585c723a04732, error=17) GEOM leak: 14/87 20200213 18:22:19 all (174/610): pshared.sh 20200213 18:22:25 all (175/610): truncate3.sh 20200213 18:22:30 all (176/610): fork.sh 20200213 18:22:36 all (177/610): linger2.sh Feb 13 18:22:39 mercat1 kernel: pid 65800 (linger2), uid 2007 inumber 887 on /mnt: filesystem full 20200213 18:22:43 all (178/610): linger4.sh Feb 13 18:22:48 mercat1 kernel: pid 67129 (linger4), uid 2007 inumber 103173 on /mnt: out of inodes 20200213 18:22:50 all (179/610): mlockall4.sh 20200213 18:22:56 all (180/610): ptrace9.sh 20200213 18:23:03 all (181/610): quota11.sh 20200213 18:23:09 all (182/610): sparse.sh 20200213 18:23:15 all (183/610): symlink2.sh 20200213 18:23:22 all (184/610): sysctl.sh 20200213 18:23:29 all (185/610): ffs_blkfree.sh 20200213 18:23:39 all (186/610): md8.sh 20200213 18:23:49 all (187/610): tmpfs8.sh 20200213 18:23:58 all (188/610): rename8.sh 20200213 18:24:07 all (189/610): thr.sh witness_lock_list_get: witness exhausted 20200213 18:24:17 all (190/610): compare.sh 20200213 18:24:27 all (191/610): fsck5.sh 20200213 18:24:38 all (192/610): umountf4.sh 20200213 18:24:52 all (193/610): execve.sh 20200213 18:27:05 all (194/610): nfs15lockd2.sh 20200213 18:27:16 all (195/610): posix_fadvise3.sh 20200213 18:27:30 all (196/610): proccontrol.sh 20200213 18:27:41 all (197/610): rdgsbase.sh 20200213 18:27:53 all (198/610): rename9.sh 20200213 18:28:06 all (199/610): mprotect2.sh 20200213 18:28:17 all (200/610): pmc7.sh 20200213 18:28:29 all (201/610): shm_open.sh 20200213 18:28:41 all (202/610): tmpfs14.sh 20200213 18:28:53 all (203/610): umountf6.sh 20200213 18:29:06 all (204/610): pipe2.sh 20200213 18:29:19 all (205/610): umountf11.sh 20200213 18:29:32 all (206/610): umountf12.sh 20200213 18:29:46 all (207/610): rdwr.sh 20200213 18:30:01 all (208/610): sendfile_shm.sh 20200213 18:30:25 all (209/610): smrstress.sh panic: vm_fault_lookup: fault on nofault entry, addr: 0xfffffe02ba488000 cpuid = 0 time = 1581615032 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0337fbc060 vpanic() at vpanic+0x185/frame 0xfffffe0337fbc0c0 panic() at panic+0x43/frame 0xfffffe0337fbc120 vm_fault() at vm_fault+0x1a98/frame 0xfffffe0337fbc250 vm_fault_trap() at vm_fault_trap+0x6e/frame 0xfffffe0337fbc290 trap_pfault() at trap_pfault+0x1f3/frame 0xfffffe0337fbc310 trap() at trap+0x2a7/frame 0xfffffe0337fbc440 calltrap() at calltrap+0x8/frame 0xfffffe0337fbc440 --- trap 0xc, rip = 0xffffffff80c27d2d, rsp = 0xfffffe0337fbc510, rbp = 0xfffffe0337fbc530 --- smr_create() at smr_create+0x5d/frame 0xfffffe0337fbc530 zone_ctor() at zone_ctor+0x38a/frame 0xfffffe0337fbc5a0 item_ctor() at item_ctor+0x2d9/frame 0xfffffe0337fbc600 zone_alloc_item() at zone_alloc_item+0x134/frame 0xfffffe0337fbc650 uma_zcreate() at uma_zcreate+0xc3/frame 0xfffffe0337fbc6c0 smrs_modevent() at smrs_modevent+0x44/frame 0xfffffe0337fbc6f0 module_register_init() at module_register_init+0xbd/frame 0xfffffe0337fbc720 linker_load_module() at linker_load_module+0xc25/frame 0xfffffe0337fbca40 kern_kldload() at kern_kldload+0xef/frame 0xfffffe0337fbca90 sys_kldload() at sys_kldload+0x5b/frame 0xfffffe0337fbcac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe0337fbcbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0337fbcbf0 --- syscall (304, FreeBSD ELF64, sys_kldload), rip = 0x8002dbeaa, rsp = 0x7fffffffe158, rbp = 0x7fffffffe6d0 --- KDB: enter: panic [ thread pid 69162 tid 104200 ] Stopped at kdb_enter+0x37: movq $0,0x10855a6(%rip) db> run pho db:0:pho> set $lines 20000 db:0:pho> run pho1 db:1:pho1> dump Dumping 2547 out of 32626 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Dump complete db:1:pho1> bt Tracing pid 69162 tid 104200 td 0xfffffe014b82e100 kdb_enter() at kdb_enter+0x37/frame 0xfffffe0337fbc060 vpanic() at vpanic+0x1a1/frame 0xfffffe0337fbc0c0 panic() at panic+0x43/frame 0xfffffe0337fbc120 vm_fault() at vm_fault+0x1a98/frame 0xfffffe0337fbc250 vm_fault_trap() at vm_fault_trap+0x6e/frame 0xfffffe0337fbc290 trap_pfault() at trap_pfault+0x1f3/frame 0xfffffe0337fbc310 trap() at trap+0x2a7/frame 0xfffffe0337fbc440 calltrap() at calltrap+0x8/frame 0xfffffe0337fbc440 --- trap 0xc, rip = 0xffffffff80c27d2d, rsp = 0xfffffe0337fbc510, rbp = 0xfffffe0337fbc530 --- smr_create() at smr_create+0x5d/frame 0xfffffe0337fbc530 zone_ctor() at zone_ctor+0x38a/frame 0xfffffe0337fbc5a0 item_ctor() at item_ctor+0x2d9/frame 0xfffffe0337fbc600 zone_alloc_item() at zone_alloc_item+0x134/frame 0xfffffe0337fbc650 uma_zcreate() at uma_zcreate+0xc3/frame 0xfffffe0337fbc6c0 smrs_modevent() at smrs_modevent+0x44/frame 0xfffffe0337fbc6f0 module_register_init() at module_register_init+0xbd/frame 0xfffffe0337fbc720 linker_load_module() at linker_load_module+0xc25/frame 0xfffffe0337fbca40 kern_kldload() at kern_kldload+0xef/frame 0xfffffe0337fbca90 sys_kldload() at sys_kldload+0x5b/frame 0xfffffe0337fbcac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe0337fbcbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0337fbcbf0 --- syscall (304, FreeBSD ELF64, sys_kldload), rip = 0x8002dbeaa, rsp = 0x7fffffffe158, rbp = 0x7fffffffe6d0 --- db:1:pho1> show allpcpu Current CPU: 0 cpuid = 0 dynamic pcpu = 0x975d80 curthread = 0xfffffe014b82e100: pid 69162 tid 104200 critnest 1 "kldload" curpcb = 0xfffffe014b82e600 fpcurthread = 0xfffffe014b82e100: pid 69162 "kldload" idlethread = 0xfffffe003e3e4300: tid 100003 "idle: cpu0" self = 0xffffffff82410000 curpmap = 0xfffffe015e35bb08 tssp = 0xffffffff82410384 rsp0 = 0xfffffe0337fbccc0 kcr3 = 0x80000006af0172cf ucr3 = 0x800000075c35cacf scr3 = 0x75c35cacf gs32p = 0xffffffff82410404 ldt = 0xffffffff82410444 tss = 0xffffffff82410434 tlb gen = 5604746 curvnet = 0xfffff80003761040 spin locks held: cpuid = 1 dynamic pcpu = 0xfffffe00bc7f6d80 curthread = 0xfffffe003e3e3c00: pid 11 tid 100004 critnest 1 "idle: cpu1" curpcb = 0xfffffe003e3e4100 fpcurthread = none idlethread = 0xfffffe003e3e3c00: tid 100004 "idle: cpu1" self = 0xffffffff82411000 curpmap = 0xffffffff81d9ea00 tssp = 0xffffffff82411384 rsp0 = 0xfffffe00add77cc0 kcr3 = 0x8000000002306005 ucr3 = 0xffffffffffffffff scr3 = 0x75c86ccf3 gs32p = 0xffffffff82411404 ldt = 0xffffffff82411444 tss = 0xffffffff82411434 tlb gen = 5604893 curvnet = 0 spin locks held: cpuid = 2 dynamic pcpu = 0xfffffe00bc802d80 curthread = 0xfffffe011d435500: pid 69155 tid 104091 critnest 0 "swap" curpcb = 0xfffffe011d435a00 fpcurthread = 0xfffffe011d435500: pid 69155 "swap" idlethread = 0xfffffe003e3e3500: tid 100005 "idle: cpu2" self = 0xffffffff82412000 curpmap = 0xfffffe0113957120 tssp = 0xffffffff82412384 rsp0 = 0xfffffe0334971cc0 kcr3 = 0x800000063786568f ucr3 = 0x8000000637864e8f scr3 = 0x637864e8f gs32p = 0xffffffff82412404 ldt = 0xffffffff82412444 tss = 0xffffffff82412434 tlb gen = 5604893 curvnet = 0 spin locks held: cpuid = 3 dynamic pcpu = 0xfffffe00bc80ed80 curthread = 0xfffffe014b818c00: pid 69144 tid 104265 critnest 0 "swap" curpcb = 0xfffffe014b819100 fpcurthread = 0xfffffe014b818c00: pid 69144 "swap" idlethread = 0xfffffe003e3e2e00: tid 100006 "idle: cpu3" self = 0xffffffff82413000 curpmap = 0xfffffe00dd1e4120 tssp = 0xffffffff82413384 rsp0 = 0xfffffe0336708cc0 kcr3 = 0x80000005272bd05b ucr3 = 0x800000075c86c85b scr3 = 0x75c86c85b gs32p = 0xffffffff82413404 ldt = 0xffffffff82413444 tss = 0xffffffff82413434 tlb gen = 5604893 curvnet = 0 spin locks held: cpuid = 4 dynamic pcpu = 0xfffffe00bc81ad80 curthread = 0xfffffe0168e67000: pid 69143 tid 104011 critnest 0 "swap" curpcb = 0xfffffe0168e67500 fpcurthread = 0xfffffe0168e67000: pid 69143 "swap" idlethread = 0xfffffe003e3e2700: tid 100007 "idle: cpu4" self = 0xffffffff82414000 curpmap = 0xfffffe010e653120 tssp = 0xffffffff82414384 rsp0 = 0xfffffe03365dfcc0 kcr3 = 0x80000005272ca06c ucr3 = 0x80000002dcc9f86c scr3 = 0x2dcc9f86c gs32p = 0xffffffff82414404 ldt = 0xffffffff82414444 tss = 0xffffffff82414434 tlb gen = 5604893 curvnet = 0 spin locks held: cpuid = 5 dynamic pcpu = 0xfffffe00bc826d80 curthread = 0xfffffe0169e2f800: pid 69154 tid 104224 critnest 0 "swap" curpcb = 0xfffffe0169e2fd00 fpcurthread = 0xfffffe0169e2f800: pid 69154 "swap" idlethread = 0xfffffe003e3e2000: tid 100008 "idle: cpu5" self = 0xffffffff82415000 curpmap = 0xfffffe00dd1e94f0 tssp = 0xffffffff82415384 rsp0 = 0xfffffe0336636cc0 kcr3 = 0x800000075c47905b ucr3 = 0x800000075c47885b scr3 = 0x75c47885b gs32p = 0xffffffff82415404 ldt = 0xffffffff82415444 tss = 0xffffffff82415434 tlb gen = 5604893 curvnet = 0 spin locks held: cpuid = 6 dynamic pcpu = 0xfffffe00bc832d80 curthread = 0xfffffe0168e80c00: pid 69147 tid 104096 critnest 0 "swap" curpcb = 0xfffffe0168e81100 fpcurthread = 0xfffffe0168e80c00: pid 69147 "swap" idlethread = 0xfffffe003e3eca00: tid 100009 "idle: cpu6" self = 0xffffffff82416000 curpmap = 0xfffffe014b9d3120 tssp = 0xffffffff82416384 rsp0 = 0xfffffe03349a8cc0 kcr3 = 0x80000006af00518d ucr3 = 0x80000006af00498d scr3 = 0x6af00498d gs32p = 0xffffffff82416404 ldt = 0xffffffff82416444 tss = 0xffffffff82416434 tlb gen = 5604893 curvnet = 0 spin locks held: cpuid = 7 dynamic pcpu = 0xfffffe00bc83ed80 curthread = 0xfffffe011d441500: pid 69160 tid 104332 critnest 0 "swap" curpcb = 0xfffffe011d441a00 fpcurthread = 0xfffffe011d441500: pid 69160 "swap" idlethread = 0xfffffe003e3ec300: tid 100010 "idle: cpu7" self = 0xffffffff82417000 curpmap = 0xfffffe01287e34f0 tssp = 0xffffffff82417384 rsp0 = 0xfffffe0334a54cc0 kcr3 = 0x800000075cdd548c ucr3 = 0x800000075cdd4c8c scr3 = 0x75cdd4c8c gs32p = 0xffffffff82417404 ldt = 0xffffffff82417444 tss = 0xffffffff82417434 tlb gen = 5604893 curvnet = 0 spin locks held: cpuid = 8 dynamic pcpu = 0xfffffe00bc84ad80 curthread = 0xfffffe003e3ebc00: pid 11 tid 100011 critnest 1 "idle: cpu8" curpcb = 0xfffffe003e3ec100 fpcurthread = none idlethread = 0xfffffe003e3ebc00: tid 100011 "idle: cpu8" self = 0xffffffff82418000 curpmap = 0xffffffff81d9ea00 tssp = 0xffffffff82418384 rsp0 = 0xfffffe00add9acc0 kcr3 = 0x8000000002306003 ucr3 = 0xffffffffffffffff scr3 = 0x75cdd4979 gs32p = 0xffffffff82418404 ldt = 0xffffffff82418444 tss = 0xffffffff82418434 tlb gen = 5604893 curvnet = 0 spin locks held: cpuid = 9 dynamic pcpu = 0xfffffe00bc856d80 curthread = 0xfffffe011dff2500: pid 69159 tid 104214 critnest 0 "swap" curpcb = 0xfffffe011dff2a00 fpcurthread = 0xfffffe011dff2500: pid 69159 "swap" idlethread = 0xfffffe003e3eb500: tid 100012 "idle: cpu9" self = 0xffffffff82419000 curpmap = 0xfffffe00d39df120 tssp = 0xffffffff82419384 rsp0 = 0xfffffe0337fe9cc0 kcr3 = 0x80000002dc5bd781 ucr3 = 0x80000002dc5bcf81 scr3 = 0x2dc5bcf81 gs32p = 0xffffffff82419404 ldt = 0xffffffff82419444 tss = 0xffffffff82419434 tlb gen = 5604893 curvnet = 0 spin locks held: cpuid = 10 dynamic pcpu = 0xfffffe00bc862d80 curthread = 0xfffffe003e3eae00: pid 11 tid 100013 critnest 1 "idle: cpu10" curpcb = 0xfffffe003e3eb300 fpcurthread = none idlethread = 0xfffffe003e3eae00: tid 100013 "idle: cpu10" self = 0xffffffff8241a000 curpmap = 0xffffffff81d9ea00 tssp = 0xffffffff8241a384 rsp0 = 0xfffffe00adda4cc0 kcr3 = 0x8000000002306002 ucr3 = 0xffffffffffffffff scr3 = 0x75c4a6dcc gs32p = 0xffffffff8241a404 ldt = 0xffffffff8241a444 tss = 0xffffffff8241a434 tlb gen = 5604893 curvnet = 0 spin locks held: cpuid = 11 dynamic pcpu = 0xfffffe00bc86ed80 curthread = 0xfffffe016fa6c700: pid 69145 tid 104134 critnest 0 "swap" curpcb = 0xfffffe016fa6cc00 fpcurthread = 0xfffffe016fa6c700: pid 69145 "swap" idlethread = 0xfffffe003e3ea700: tid 100014 "idle: cpu11" self = 0xffffffff8241b000 curpmap = 0xfffffe0128868b08 tssp = 0xffffffff8241b384 rsp0 = 0xfffffe0337e68cc0 kcr3 = 0x800000026094d669 ucr3 = 0x80000002dc113e69 scr3 = 0x2dc113e69 gs32p = 0xffffffff8241b404 ldt = 0xffffffff8241b444 tss = 0xffffffff8241b434 tlb gen = 5604893 curvnet = 0 spin locks held: db:1:pho1> show alllocks db:1:pho1> show lockedvnods Locked vnodes db:1:pho1> show mount 0xfffffe00d3def700 /dev/gpt/work on / (ufs) 0xfffffe00d3df0280 devfs on /dev (devfs) 0xfffffe00dd1f3700 /dev/gpt/tmp on /tmp (ufs) 0xfffffe00e648e700 procfs on /proc (procfs) More info: show mount db:1:pho1> show bufqueues bqempty: 87108 Buf domain 0 freebufs 11938 lofreebufs 219 hifreebufs 329 bufspace 73994240 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 80 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 2992 (67653632) dirtyq count 80 (2531328) wakeup 0 lim 25 CPU 19, 2, 12, 14, 4, 16, 12, 2, 13, 15, 2, 12, Locked buffers: 0 space 0 Total buffers: 4440 space 73994240 Buf domain 1 freebufs 12945 lofreebufs 219 hifreebufs 329 bufspace 50397184 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 1 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 2150 (49307648) dirtyq count 1 (4096) wakeup 0 lim 25 CPU 1, 0, 0, 1, 21, 0, 1, 0, 0, 5, 3, 5, Locked buffers: 0 space 0 Total buffers: 4068 space 50397184 Buf domain 2 freebufs 15012 lofreebufs 219 hifreebufs 329 bufspace 1781760 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 1 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 82 (1159168) dirtyq count 1 (4096) wakeup 0 lim 25 CPU 0, 1, 1, 0, 5, 0, 6, 1, 0, 14, 3, 7, Locked buffers: 0 space 0 Total buffers: 20060 space 1781760 Buf domain 3 freebufs 13871 lofreebufs 219 hifreebufs 329 bufspace 11485184 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 4 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 1244 (11169792) dirtyq count 4 (20480) wakeup 0 lim 25 CPU 1, 1, 2, 0, 0, 0, 0, 0, 1, 3, 4, 2, Locked buffers: 0 space 0 Total buffers: 4660 space 11485184 Buf domain 4 freebufs 13778 lofreebufs 219 hifreebufs 329 bufspace 14045184 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 2 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 1345 (13844480) dirtyq count 2 (32768) wakeup 0 lim 25 CPU 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 4, 1, Locked buffers: 0 space 0 Total buffers: 25188 space 14045184 Buf domain 5 freebufs 14492 lofreebufs 219 hifreebufs 329 bufspace 19456000 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 9 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 578 (18210816) dirtyq count 9 (266240) wakeup 0 lim 25 CPU 0, 1, 1, 0, 17, 0, 5, 0, 0, 17, 4, 9, Locked buffers: 0 space 0 Total buffers: 26511 space 19456000 Buf domain 6 freebufs 13879 lofreebufs 219 hifreebufs 329 bufspace 10764288 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 1239 (10461184) dirtyq count 0 (0) wakeup 0 lim 25 CPU 0, 1, 0, 0, 3, 0, 0, 0, 0, 2, 9, 0, Locked buffers: 0 space 0 Total buffers: 3699 space 10764288 db:1:pho1> show page vm_cnt.v_free_count: 4929965 vm_cnt.v_inactive_count: 680 vm_cnt.v_active_count: 2734834 vm_cnt.v_laundry_count: 58 vm_cnt.v_wire_count: 454498 vm_cnt.v_free_reserved: 10644 vm_cnt.v_free_min: 51264 vm_cnt.v_free_target: 173124 vm_cnt.v_inactive_target: 259686 db:1:pho1> show pageq pq_free 4929965 dom 0 page_cnt 8124099 free 4929965 pq_act 2734834 pq_inact 680 pq_laund 58 pq_unsw 0 db:0:pho> show freepages DOMAIN: 0 FREE LIST 0: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000666 | 000000 11 (008192K) | 000076 | 000000 10 (004096K) | 000001 | 000000 09 (002048K) | 000001 | 000006 08 (001024K) | 002700 | 000048 07 (000512K) | 003549 | 000019 06 (000256K) | 004369 | 000024 05 (000128K) | 005183 | 000001 04 (000064K) | 007607 | 000000 03 (000032K) | 012951 | 000000 02 (000016K) | 020722 | 000001 01 (000008K) | 026506 | 000000 00 (000004K) | 025214 | 000001 FREE LIST 1: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000000 | 000000 10 (004096K) | 000000 | 000000 09 (002048K) | 000000 | 000000 08 (001024K) | 000000 | 000000 07 (000512K) | 000000 | 000000 06 (000256K) | 000001 | 000000 05 (000128K) | 000001 | 000000 04 (000064K) | 000002 | 000000 03 (000032K) | 000000 | 000000 02 (000016K) | 000001 | 000000 01 (000008K) | 000000 | 000000 00 (000004K) | 000001 | 000000 db:0:pho> show uma Zone Size Used Free Requests Sleeps Bucket Total Mem XFree 1024 1024 35 500177 1901899 0 16 512217088 0 VM OBJECT 264 13604 498781 7471036 0 31 135269640 0 256 256 109 500591 701836 0 62 128179200 0 RADIX NODE 144 197984 334622 14853182 0 67 76695264 0 mbuf_cluster 2048 15844 38 20467 0 8 32526336 0 64 64 207 502407 943754 0 254 32167296 0 MAP ENTRY 96 2536 263828 35800996 0 126 25570944 0 VNODE 640 26149 10457 3035366 0 16 23427840 0 UMA Slabs 0 112 144003 9 251472 0 126 16129344 0 BUF TRIE 144 840 105112 3791695 0 62 15257088 0 4096 4096 159 3113 465954 0 6 13402112 0 THREAD 1776 3394 4085 281749 0 8 13282704 0 swblk 136 1557 92664 99541 0 62 12814056 0 64 Bucket 512 16907 2677 2328686 0 254 10027008 0 FFS2 dinode 256 25301 11269 1840801 0 62 9361920 0 128 128 67173 1523 230277 0 126 8793088 0 128 128 65083 1536 146335 0 126 8527232 0 FFS inode 160 25301 15949 1840854 0 62 6600000 0 256 Bucket 2048 2593 49 1271809 0 12 5410816 0 mbuf 256 16437 3562 261368 0 254 5119744 0 16 Bucket 144 32640 651 583480 0 62 4793904 0 128 Bucket 1024 2425 1923 2295636 0 254 4452352 0 PROC 1328 100 3167 368934 0 9 4338576 0 vmem btag 56 23657 49545 117914 0 254 4099312 0 S VFS Cache 108 35672 2020 3205746 0 126 4070736 0 socket 904 55 3509 6389 0 16 3221856 0 VMSPACE 2536 67 1079 368754 0 4 2906256 0 512 512 47 5505 3171361 0 30 2842624 0 16 16 162675 1575 162743 0 254 2628000 0 256 256 645 9345 16190730 0 62 2557440 0 32 32 70611 3099 641643 0 254 2358720 0 128 128 16338 1518 278306 0 126 2285568 0 128 128 14424 3277 4546175 0 126 2265728 0 512 512 3381 115 11225 0 30 1789952 0 4096 4096 345 48 80118 0 2 1609728 0 128 128 11042 1482 289575 0 126 1603072 0 pipe 760 8 1842 541704 0 16 1406000 0 16384 16384 23 53 64620 0 1 1245184 0 TURNSTILE 136 7480 458 7480 0 62 1079568 0 65536 65536 5 11 3463 0 1 1048576 0 256 256 306 3654 5530319 0 62 1013760 0 swpctrie 144 249 6611 6999 0 62 987840 0 mbuf_packet 256 24 3557 124353 0 254 916736 0 32768 32768 9 17 11061 0 1 851968 0 8192 8192 98 0 102 0 1 802816 0 8192 8192 22 75 69853 0 1 794624 0 64 64 9294 3054 79489076 0 254 790272 0 32768 32768 24 0 3030 0 1 786432 0 64 64 9027 3069 848277 0 254 774144 0 65536 65536 4 7 20 0 1 720896 0 SLEEPQUEUE 88 7480 296 7480 0 126 684288 0 16384 16384 37 0 37 0 1 606208 0 pbuf 984 0 584 0 0 0 574656 0 64 64 3253 5126 401427 0 254 536256 0 ksiginfo 112 3238 1082 114698 0 126 483840 0 Files 80 137 5813 11189046 0 126 476000 0 1024 1024 157 275 5235 0 16 442368 0 64 64 3548 3067 141726 0 254 423360 0 g_bio 408 0 1030 9251947 0 31 420240 0 2048 2048 27 153 1982413 0 8 368640 0 Mountpoints 2944 4 120 5137 0 4 365056 0 8192 8192 3 41 1661 0 1 360448 0 32 Bucket 256 285 1049 2132562 0 62 341504 0 filedesc0 1104 101 207 368935 0 8 340032 0 32 32 7074 3132 26919 0 254 326592 0 128 128 712 1644 2741008 0 126 301568 0 4096 4096 21 46 34455 0 2 274432 0 16 16 14149 2601 14745 0 254 268000 0 NAMEI 1024 0 256 28392538 0 16 262144 0 65536 65536 4 0 4 0 1 262144 0 256 256 232 758 4280 0 62 253440 0 256 256 28 932 6461419 0 62 245760 0 rl_entry 40 3089 2971 3089 0 254 242400 0 512 512 76 396 339711 0 30 241664 0 selfd 64 505 3086 648217 0 254 229824 0 256 256 113 757 7944 0 62 222720 0 1024 1024 15 193 885 0 16 212992 0 64 64 218 3057 3816164 0 254 209600 0 UMA Zones 2176 90 5 92 0 4 206720 0 256 256 36 744 128249 0 62 199680 0 1024 1024 8 176 21 0 16 188416 0 2048 2048 88 2 423750 0 8 184320 0 256 256 37 683 72206 0 62 184320 0 128 128 111 1284 3401 0 126 178560 0 512 512 0 336 385 0 30 172032 0 unpcb 256 22 623 5901 0 62 165120 0 fakepg 104 0 1521 1402 0 126 158184 0 4 Bucket 48 160 2948 320495 0 254 149184 0 16384 16384 0 9 347 0 1 147456 0 1024 1024 0 144 86 0 16 147456 0 512 512 179 101 572 0 30 143360 0 8 Bucket 80 512 1187 123287 0 126 135920 0 65536 65536 2 0 2 0 1 131072 0 32768 32768 4 0 4 0 1 131072 0 32768 32768 0 4 300 0 1 131072 0 32 32 875 3157 213130 0 254 129024 0 clpbuf 984 0 128 687751 0 4 125952 0 8192 8192 15 0 16 0 1 122880 0 TMPFS node 232 0 510 1070339 0 62 118320 0 TMPFS dirent 64 0 1827 1070323 0 254 116928 0 32 32 574 3080 43534 0 254 116928 0 DIRHASH 1024 47 65 11432 0 16 114688 0 16384 16384 7 0 9 0 1 114688 0 16384 16384 7 0 7 0 1 114688 0 32 32 238 3164 19234 0 254 108864 0 32 32 185 3091 76772 0 254 104832 0 32 32 8 3142 2143 0 254 100800 0 2048 2048 42 6 69 0 8 98304 0 4096 4096 22 0 25 0 2 90112 0 2 Bucket 32 367 2405 381263 0 254 88704 0 ttyinq 160 345 155 5205 0 62 80000 0 rentr 24 0 3173 4499 0 254 76152 0 1024 1024 49 23 2137 0 16 73728 0 ttyoutq 256 181 104 2770 0 62 72960 0 udp_inpcb 488 20 124 386 0 30 70272 0 65536 65536 1 0 1 0 1 65536 0 65536 65536 1 0 1 0 1 65536 0 65536 65536 1 0 1 0 1 65536 0 32768 32768 2 0 2 0 1 65536 0 16 16 450 3550 645331 0 254 64000 0 16 16 471 3279 2940452 0 254 60000 0 swrbuf 984 0 60 3113 0 4 59040 0 512 512 33 79 34 0 30 57344 0 16 16 435 3065 158820 0 254 56000 0 128 128 52 382 3702 0 126 55552 0 4096 4096 12 1 12 0 2 53248 0 64 64 0 819 34 0 254 52416 0 16 16 18 3232 71 0 254 52000 0 1024 1024 37 11 571 0 16 49152 0 64 pcpu 8 3138 3006 3583 0 254 49152 0 mt_stats_zone 64 442 262 443 0 254 45056 0 KMAP ENTRY 96 63 396 76 0 126 44064 0 L VFS Cache 328 60 72 148 0 30 43296 0 swwbuf 984 0 40 55640 0 2 39360 0 udpcb 32 20 1114 386 0 254 36288 0 cpuset 104 27 314 27 0 126 35464 0 8192 8192 4 0 4 0 1 32768 0 UMA Kegs 384 80 5 82 0 30 32640 0 32 32 20 988 21 0 254 32256 0 mt_zone 24 442 894 443 0 254 32064 0 rtentry 208 12 140 16 0 62 31616 0 4096 4096 1 6 4737 0 2 28672 0 procdesc 136 1 202 38 0 62 27608 0 1024 1024 11 9 14 0 16 20480 0 KNOTE 160 0 125 365044 0 62 20000 0 16 16 17 1233 18 0 254 20000 0 tcpcb 1008 10 6 67 0 16 16128 0 64 64 9 243 100091 0 254 16128 0 VNODEPOLL 120 1 131 7 0 126 15840 0 tcp_inpcb 488 10 22 67 0 30 15616 0 MAP 216 2 69 2 0 62 15336 0 4096 4096 1 2 3 0 2 12288 0 2048 2048 3 3 3 0 8 12288 0 512 512 15 9 1763904 0 30 12288 0 int pcpu 4 498 2574 498 0 254 12288 0 8192 8192 1 0 1 0 1 8192 0 2048 2048 2 2 5636 0 8 8192 0 2048 2048 2 2 2 0 8 8192 0 SMR CPU 24 1 255 1 0 254 6144 0 epoch_record pcpu 256 4 12 4 0 62 4096 0 4096 4096 1 0 500038 0 2 4096 0 2048 2048 2 0 2 0 8 4096 0 2048 2048 1 1 513 0 8 4096 0 SMR SHARED 16 1 255 1 0 254 4096 0 hostcache 96 1 41 1 0 126 4032 0 ripcb 488 1 7 13 0 30 3904 0 UMA Slabs 1 176 16 5 16 0 62 3696 0 vmem 1856 0 1 0 0 8 1856 0 smrs 8 0 0 0 0 0 0 0 taskq_zone 48 0 0 0 0 254 0 0 dtrace_state_cache 16384 0 0 1 0 1 0 0 FFS1 dinode 128 0 0 0 0 126 0 0 IPsec SA lft_c 16 0 0 0 0 254 0 0 udplite_inpcb 488 0 0 0 0 30 0 0 sctp_asconf_ack 48 0 0 0 0 254 0 0 sctp_asconf 40 0 0 0 0 254 0 0 sctp_stream_msg_out 112 0 0 0 0 126 0 0 sctp_readq 152 0 0 0 0 62 0 0 sctp_chunk 152 0 0 0 0 62 0 0 sctp_raddr 736 0 0 0 0 16 0 0 sctp_laddr 48 0 0 4 0 254 0 0 sctp_asoc 2288 0 0 0 0 4 0 0 sctp_ep 1280 0 0 0 0 8 0 0 tcp_log_node 120 0 0 0 0 126 0 0 tcp_log_bucket 184 0 0 0 0 62 0 0 tcp_log 408 0 0 0 0 30 0 0 tcpreass 48 0 0 0 0 254 0 0 tfo_ccache_entries 80 0 0 0 0 126 0 0 tfo 4 0 0 0 0 254 0 0 sackhole 32 0 0 0 0 254 0 0 syncache 168 0 0 19 0 62 0 0 tcptw 88 0 0 17 0 126 0 0 ipq 56 0 0 0 0 254 0 0 itimer 352 0 0 1 0 30 0 0 AIOLIO 280 0 0 0 0 30 0 0 AIOCB 752 0 0 0 0 16 0 0 AIOP 32 0 0 0 0 254 0 0 AIO 208 0 0 0 0 62 0 0 NCLNODE 744 0 0 0 0 16 0 0 LTS VFS Cache 368 0 0 0 0 30 0 0 STS VFS Cache 148 0 0 0 0 62 0 0 vtnet_tx_hdr 24 0 0 0 0 254 0 0 crypto_session 24 0 0 0 0 254 0 0 cryptodesc 120 0 0 0 0 126 0 0 cryptop 128 0 0 0 0 126 0 0 nvme_request 128 0 0 0 0 126 0 0 FPU_save_area 832 0 0 0 0 16 0 0 DMAR_MAP_ENTRY 128 0 0 0 0 126 0 0 mbuf_extpgs 256 0 0 0 0 62 0 0 mbuf_jumbo_16k 16384 0 0 0 0 1 0 0 mbuf_jumbo_9k 9216 0 0 0 0 1 0 0 mbuf_jumbo_page 4096 0 0 321 0 2 0 0 audit_record 1280 0 0 0 0 8 0 0 domainset 40 0 0 0 0 254 0 0 MAC labels 40 0 0 0 0 254 0 0 vnpbuf 984 0 0 2484 0 16 0 0 mdpbuf 984 0 0 22730 0 0 0 0 nfspbuf 984 0 0 0 0 4 0 0 umtx_shm 88 0 0 35 0 126 0 0 umtx pi 96 0 0 1 0 126 0 0 rangeset pctrie nodes 144 0 0 0 0 62 0 0 65536 65536 0 0 3 0 1 0 0 32768 32768 0 0 0 0 1 0 0 32768 32768 0 0 0 0 1 0 0 32768 32768 0 0 0 0 1 0 0 16384 16384 0 0 0 0 1 0 0 16384 16384 0 0 0 0 1 0 0 16384 16384 0 0 0 0 1 0 0 8192 8192 0 0 0 0 1 0 0 8192 8192 0 0 1 0 1 0 0 512 512 0 0 0 0 30 0 0 16 16 0 0 4 0 254 0 0 UMA Hash 256 0 0 0 0 62 0 0 db:0:pho> show umacache Zone Size Used Free Requests Bucket debugnet_mbuf_packet 2048 0 0 0 0 debugnet_mbuf_cluster 2048 -6144 0 0 0 debugnet_mbuf 256 -10240 0 0 0 kstack_cache 16384 3267 8 281622 2 vm pgcache 4096 291193 179 11057850 19 vm pgcache 4096 42346 3305 80647981 129 buf free cache 984 10016 8807 10181325 16 buffer arena-65 65536 0 0 106 1 buffer arena-61 61440 0 0 0 1 buffer arena-57 57344 0 0 0 1 buffer arena-53 53248 0 0 0 1 buffer arena-49 49152 0 0 0 1 buffer arena-45 45056 0 0 0 1 buffer arena-40 40960 0 0 0 1 buffer arena-36 36864 0 0 0 1 buffer arena-32 32768 3941 60 360134 2 buffer arena-28 28672 0 0 0 1 buffer arena-24 24576 0 0 0 1 buffer arena-20 20480 0 0 0 1 buffer arena-16 16384 3 73 98390 1 buffer arena-12 12288 0 0 0 1 buffer arena-81 8192 0 0 0 1 buffer arena-40 4096 0 0 0 2 db:0:pho> acttrace Tracing command kldload pid 69162 tid 104200 td 0xfffffe014b82e100 (CPU 0) kdb_enter() at kdb_enter+0x37/frame 0xfffffe0337fbc060 vpanic() at vpanic+0x1a1/frame 0xfffffe0337fbc0c0 panic() at panic+0x43/frame 0xfffffe0337fbc120 vm_fault() at vm_fault+0x1a98/frame 0xfffffe0337fbc250 vm_fault_trap() at vm_fault_trap+0x6e/frame 0xfffffe0337fbc290 trap_pfault() at trap_pfault+0x1f3/frame 0xfffffe0337fbc310 trap() at trap+0x2a7/frame 0xfffffe0337fbc440 calltrap() at calltrap+0x8/frame 0xfffffe0337fbc440 --- trap 0xc, rip = 0xffffffff80c27d2d, rsp = 0xfffffe0337fbc510, rbp = 0xfffffe0337fbc530 --- smr_create() at smr_create+0x5d/frame 0xfffffe0337fbc530 zone_ctor() at zone_ctor+0x38a/frame 0xfffffe0337fbc5a0 item_ctor() at item_ctor+0x2d9/frame 0xfffffe0337fbc600 zone_alloc_item() at zone_alloc_item+0x134/frame 0xfffffe0337fbc650 uma_zcreate() at uma_zcreate+0xc3/frame 0xfffffe0337fbc6c0 smrs_modevent() at smrs_modevent+0x44/frame 0xfffffe0337fbc6f0 module_register_init() at module_register_init+0xbd/frame 0xfffffe0337fbc720 linker_load_module() at linker_load_module+0xc25/frame 0xfffffe0337fbca40 kern_kldload() at kern_kldload+0xef/frame 0xfffffe0337fbca90 sys_kldload() at sys_kldload+0x5b/frame 0xfffffe0337fbcac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe0337fbcbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0337fbcbf0 --- syscall (304, FreeBSD ELF64, sys_kldload), rip = 0x8002dbeaa, rsp = 0x7fffffffe158, rbp = 0x7fffffffe6d0 --- Tracing command swap pid 69160 tid 104332 td 0xfffffe011d441500 (CPU 7) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1c7dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1c7df0 trap() at trap+0x56/frame 0xfffffe003e1c7f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1c7f20 --- trap 0x13, rip = 0xffffffff80c37387, rsp = 0xfffffe0334a54880, rbp = 0xfffffe0334a548b0 --- witness_unlock() at witness_unlock+0x47/frame 0xfffffe0334a548b0 _rw_wunlock_cookie() at _rw_wunlock_cookie+0x59/frame 0xfffffe0334a548d0 vm_fault() at vm_fault+0xbb0/frame 0xfffffe0334a54a00 vm_fault_trap() at vm_fault_trap+0x6e/frame 0xfffffe0334a54a40 trap_pfault() at trap_pfault+0x1f3/frame 0xfffffe0334a54ac0 trap() at trap+0x457/frame 0xfffffe0334a54bf0 calltrap() at calltrap+0x8/frame 0xfffffe0334a54bf0 --- trap 0xc, rip = 0x800437015, rsp = 0x7fffffffe198, rbp = 0x7fffffffe1e0 --- Tracing command swap pid 69159 tid 104214 td 0xfffffe011dff2500 (CPU 9) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1dfdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1dfdf0 trap() at trap+0x56/frame 0xfffffe003e1dff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1dff20 --- trap 0x13, rip = 0x203689, rsp = 0x7fffffffe4f0, rbp = 0x7fffffffe510 --- Tracing command swap pid 69155 tid 104091 td 0xfffffe011d435500 (CPU 2) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e18bdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e18bdf0 trap() at trap+0x56/frame 0xfffffe003e18bf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e18bf20 --- trap 0x13, rip = 0x203687, rsp = 0x7fffffffe4f0, rbp = 0x7fffffffe510 --- Tracing command swap pid 69154 tid 104224 td 0xfffffe0169e2f800 (CPU 5) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1afdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1afdf0 trap() at trap+0x56/frame 0xfffffe003e1aff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1aff20 --- trap 0x13, rip = 0xffffffff80f2f10c, rsp = 0xfffffe03366367e0, rbp = 0xfffffe03366367e0 --- vm_page_alloc_check() at vm_page_alloc_check+0xc/frame 0xfffffe03366367e0 vm_page_alloc_domain_after() at vm_page_alloc_domain_after+0x332/frame 0xfffffe0336636870 vm_page_alloc() at vm_page_alloc+0x74/frame 0xfffffe03366368d0 vm_fault() at vm_fault+0x531/frame 0xfffffe0336636a00 vm_fault_trap() at vm_fault_trap+0x6e/frame 0xfffffe0336636a40 trap_pfault() at trap_pfault+0x1f3/frame 0xfffffe0336636ac0 trap() at trap+0x457/frame 0xfffffe0336636bf0 calltrap() at calltrap+0x8/frame 0xfffffe0336636bf0 --- trap 0xc, rip = 0x800437015, rsp = 0x7fffffffe198, rbp = 0x7fffffffe1e0 --- Tracing command swap pid 69147 tid 104096 td 0xfffffe0168e80c00 (CPU 6) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1bbdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1bbdf0 trap() at trap+0x56/frame 0xfffffe003e1bbf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1bbf20 --- trap 0x13, rip = 0x203687, rsp = 0x7fffffffe4f0, rbp = 0x7fffffffe510 --- Tracing command swap pid 69145 tid 104134 td 0xfffffe016fa6c700 (CPU 11) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1f7dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1f7df0 trap() at trap+0x56/frame 0xfffffe003e1f7f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1f7f20 --- trap 0x13, rip = 0x203689, rsp = 0x7fffffffe4f0, rbp = 0x7fffffffe510 --- Tracing command swap pid 69144 tid 104265 td 0xfffffe014b818c00 (CPU 3) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e197dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e197df0 trap() at trap+0x56/frame 0xfffffe003e197f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e197f20 --- trap 0x13, rip = 0x203687, rsp = 0x7fffffffe4f0, rbp = 0x7fffffffe510 --- Tracing command swap pid 69143 tid 104011 td 0xfffffe0168e67000 (CPU 4) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1a3dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1a3df0 trap() at trap+0x56/frame 0xfffffe003e1a3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1a3f20 --- trap 0x13, rip = 0x203687, rsp = 0x7fffffffe4f0, rbp = 0x7fffffffe510 --- Tracing command idle pid 11 tid 100004 td 0xfffffe003e3e3c00 (CPU 1) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e17fdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e17fdf0 trap() at trap+0x56/frame 0xfffffe003e17ff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e17ff20 --- trap 0x13, rip = 0xffffffff81032d36, rsp = 0xfffffe00add77a30, rbp = 0xfffffe00add77a30 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00add77a30 acpi_cpu_idle() at acpi_cpu_idle+0x23d/frame 0xfffffe00add77a90 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00add77ab0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00add77ad0 sched_idletd() at sched_idletd+0x3e1/frame 0xfffffe00add77bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add77bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add77bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffffe003e3ebc00 (CPU 8) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1d3dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1d3df0 trap() at trap+0x56/frame 0xfffffe003e1d3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1d3f20 --- trap 0x13, rip = 0xffffffff81032d36, rsp = 0xfffffe00add9aa30, rbp = 0xfffffe00add9aa30 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00add9aa30 acpi_cpu_idle() at acpi_cpu_idle+0x23d/frame 0xfffffe00add9aa90 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00add9aab0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00add9aad0 sched_idletd() at sched_idletd+0x3e1/frame 0xfffffe00add9abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add9abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add9abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xfffffe003e3eae00 (CPU 10) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1ebdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1ebdf0 trap() at trap+0x56/frame 0xfffffe003e1ebf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1ebf20 --- trap 0x13, rip = 0xffffffff81032d36, rsp = 0xfffffe00adda4a30, rbp = 0xfffffe00adda4a30 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00adda4a30 acpi_cpu_idle() at acpi_cpu_idle+0x23d/frame 0xfffffe00adda4a90 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00adda4ab0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00adda4ad0 sched_idletd() at sched_idletd+0x3e1/frame 0xfffffe00adda4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adda4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adda4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db:0:pho> ps pid ppid pgrp uid state wmesg wchan cmd 69162 69086 2440 0 R+ CPU 0 kldload 69161 69142 2440 0 S+ nanslp 0xffffffff81c82710 swap 69160 69142 2440 0 R+ CPU 7 swap 69159 69142 2440 0 R+ CPU 9 swap 69158 69142 2440 0 S+ nanslp 0xffffffff81c82713 swap 69157 69142 2440 0 S+ nanslp 0xffffffff81c82719 swap 69156 69142 2440 0 S+ nanslp 0xffffffff81c82712 swap 69155 69142 2440 0 R+ CPU 2 swap 69154 69142 2440 0 R+ CPU 5 swap 69153 69142 2440 0 S+ nanslp 0xffffffff81c82717 swap 69152 69142 2440 0 S+ nanslp 0xffffffff81c82714 swap 69151 69142 2440 0 S+ nanslp 0xffffffff81c82711 swap 69150 69142 2440 0 S+ nanslp 0xffffffff81c82715 swap 69149 69142 2440 0 S+ nanslp 0xffffffff81c82718 swap 69148 69142 2440 0 S+ nanslp 0xffffffff81c82714 swap 69147 69142 2440 0 R+ CPU 6 swap 69146 69142 2440 0 S+ nanslp 0xffffffff81c82714 swap 69145 69142 2440 0 R+ CPU 11 swap 69144 69142 2440 0 R+ CPU 3 swap 69143 69142 2440 0 R+ CPU 4 swap 69142 69140 2440 0 S+ wait 0xfffff8075c306a60 swap 69140 69086 2440 0 S+ nanslp 0xffffffff81c8271a swap 69086 69084 2440 0 S+ wait 0xfffff8075c6e7a60 sh 69085 2440 2440 0 S+ piperd 0xfffff802dd9d45f0 tee 69084 2440 2440 0 S+ wait 0xfffff8075c51e530 sh 68971 4402 4348 0 S nanslp 0xffffffff81c82716 sleep 11521 0 0 0 DL ftcl 0xffffffff82e1d4f4 [ftcleanup] 4405 4348 4348 0 S piperd 0xfffff80020728be0 sh 4403 4348 4348 0 S piperd 0xfffff80020a26000 awk 4402 4348 4348 0 S wait 0xfffff80110307a60 sh 4348 1 4348 0 SWs wait 0xfffff800205fe530 sh 2440 2386 2440 0 S+ wait 0xfffff800209e2000 sh 2429 2424 2429 2006 S+ ttyin 0xfffff80003af64b0 vi 2424 2423 2424 2006 SWs+ wait 0xfffff800206be530 bash 2423 2420 2420 2006 S select 0xfffff800041be040 sshd 2420 2308 2420 0 Ss select 0xfffff80003e95740 sshd 2386 2384 2386 0 S+ select 0xfffff80003e959c0 bash 2384 2383 2384 0 SW+ pause 0xfffff800209a80a8 csh 2383 2378 2383 2006 SW+ wait 0xfffff800207a9530 su 2378 2377 2378 2006 SWs+ wait 0xfffff800206d6000 bash 2377 2374 2374 2006 S select 0xfffff800041689c0 sshd 2374 2308 2374 0 Ss select 0xfffff80003edd3c0 sshd 2370 2367 2370 0 S+ ttyin 0xfffff80017038cb0 csh 2367 1 2367 0 SWs+ wait 0xfffff80296fed000 login 2366 1 2366 0 Ss+ ttyin 0xfffff800170370b0 getty 2365 1 2365 0 Ss+ ttyin 0xfffff8002018f8b0 getty 2364 1 2364 0 Ss+ ttyin 0xfffff8002018fcb0 getty 2363 1 2363 0 Ss+ ttyin 0xfffff800201a00b0 getty 2362 1 2362 0 Ss+ ttyin 0xfffff800201a04b0 getty 2361 1 2361 0 Ss+ ttyin 0xfffff800201a08b0 getty 2360 1 2360 0 Ss+ ttyin 0xfffff800201a0cb0 getty 2359 1 2359 0 Ss+ ttyin 0xfffff8002019f0b0 getty 2358 1 2358 0 Ss+ ttyin 0xfffff8002019f4b0 getty 2312 1 2312 0 Ss nanslp 0xffffffff81c82713 cron 2308 1 2308 0 Ss select 0xfffff800203a5e40 sshd 2280 1 2280 0 Ss select 0xfffff800039f45c0 powerd 2277 1 2277 123 Ss (threaded) ntpd 100166 S select 0xfffff80003e95a40 ntpd 100621 S usem 0xfffff80246d49b80 ntpd 2214 1 2214 0 Ss rpcsvc 0xfffff80020b31d60 NLM: master 2211 1 2211 0 Ss select 0xfffff80003e95ac0 rpc.statd 2157 1 2157 0 Ss nanslp 0xffffffff81c82710 watchdogd 2078 1 2078 0 Ss select 0xfffff80003e95bc0 rpcbind 2068 1 2068 0 Ss select 0xfffff80003edd540 syslogd 1997 1 1997 0 Ss select 0xfffff80003edd5c0 devd 1992 1 1992 65 Ss select 0xfffff800201d30c0 dhclient 1928 1 1928 0 Ss select 0xfffff800201d3740 dhclient 1925 1 1925 0 Ss select 0xfffff800041be2c0 dhclient 34 0 0 0 DL vlruwt 0xfffff80020056a60 [vnlru] 33 0 0 0 DL syncer 0xffffffff81d58b98 [syncer] 32 0 0 0 DL (threaded) [bufdaemon] 100130 D qsleep 0xffffffff81d58098 [bufdaemon] 100131 D - 0xffffffff8180a980 [bufspacedaemon-0] 100134 D - 0xffffffff81812c40 [bufspacedaemon-1] 100135 D - 0xffffffff8181af00 [bufspacedaemon-2] 100137 D - 0xffffffff818231c0 [bufspacedaemon-3] 100138 D - 0xffffffff8182b480 [bufspacedaemon-4] 100139 D - 0xffffffff81833740 [bufspacedaemon-5] 100140 D - 0xffffffff8183ba00 [bufspacedaemon-6] 100152 D sdflush 0xfffff80003a728e8 [/ worker] 100202 D sdflush 0xfffff8001708fce8 [/tmp worker] 31 0 0 0 DL psleep 0xffffffff81d73b08 [vmdaemon] 30 0 0 0 DL (threaded) [pagedaemon] 100128 D psleep 0xffffffff81d9fa98 [dom0] 100132 D launds 0xffffffff81d9faa4 [laundry: dom0] 100133 D umarcl 0xffffffff80f0dc20 [uma] 29 0 0 0 DL idle 0xfffff8002024b000 [enc_daemon1] 28 0 0 0 DL idle 0xfffff8002024b530 [enc_daemon0] 27 0 0 0 DL - 0xffffffff81adcfb0 [rand_harvestq] 26 0 0 0 DL waiting 0xffffffff81de5060 [sctp_iterator] 25 0 0 0 DL - 0xffffffff81d57a9c [soaiod4] 24 0 0 0 DL - 0xffffffff81d57a9c [soaiod3] 23 0 0 0 DL - 0xffffffff81d57a9c [soaiod2] 22 0 0 0 DL - 0xffffffff81d57a9c [soaiod1] 21 0 0 0 DL (threaded) [usb] 100096 D - 0xfffffe00d2d84440 [usbus0] 100097 D - 0xfffffe00d2d84498 [usbus0] 100098 D - 0xfffffe00d2d844f0 [usbus0] 100099 D - 0xfffffe00d2d84548 [usbus0] 100100 D - 0xfffffe00d2d845a0 [usbus0] 100102 D - 0xfffffe003e49dcf0 [usbus1] 100103 D - 0xfffffe003e49dd48 [usbus1] 100104 D - 0xfffffe003e49dda0 [usbus1] 100105 D - 0xfffffe003e49ddf8 [usbus1] 100106 D - 0xfffffe003e49de50 [usbus1] 100107 D - 0xfffffe00d2e27cf0 [usbus2] 100108 D - 0xfffffe00d2e27d48 [usbus2] 100109 D - 0xfffffe00d2e27da0 [usbus2] 100110 D - 0xfffffe00d2e27df8 [usbus2] 100111 D - 0xfffffe00d2e27e50 [usbus2] 20 0 0 0 RL (threaded) [cam] 100091 D - 0xffffffff819b83c0 [doneq0] 100092 RunQ [doneq1] 100093 D - 0xffffffff819b84c0 [doneq2] 100125 D - 0xffffffff819b8288 [scanner] 19 0 0 0 DL crypto_ 0xfffff80003aafc50 [crypto returns 11] 18 0 0 0 DL crypto_ 0xfffff80003aafbf0 [crypto returns 10] 17 0 0 0 DL crypto_ 0xfffff80003aafb90 [crypto returns 9] 16 0 0 0 DL crypto_ 0xfffff80003aafb30 [crypto returns 8] 15 0 0 0 DL crypto_ 0xfffff80003aafad0 [crypto returns 7] 9 0 0 0 DL crypto_ 0xfffff80003aafa70 [crypto returns 6] 8 0 0 0 DL crypto_ 0xfffff80003aafa10 [crypto returns 5] 7 0 0 0 DL crypto_ 0xfffff80003aaf9b0 [crypto returns 4] 6 0 0 0 DL crypto_ 0xfffff80003aaf950 [crypto returns 3] 5 0 0 0 DL crypto_ 0xfffff80003aaf8f0 [crypto returns 2] 4 0 0 0 DL crypto_ 0xfffff80003aaf890 [crypto returns 1] 3 0 0 0 DL crypto_ 0xfffff80003aaf830 [crypto returns 0] 2 0 0 0 DL crypto_ 0xffffffff81d6db78 [crypto] 14 0 0 0 DL seqstat 0xfffff80003a4b888 [sequencer 00] 13 0 0 0 DL (threaded) [geom] 100062 D - 0xffffffff81d9e0c8 [g_event] 100063 D - 0xffffffff81d9e0d8 [g_up] 100064 D - 0xffffffff81d9e0d0 [g_down] 12 0 0 0 WL (threaded) [intr] 100016 I [swi6: task queue] 100017 I [swi6: Giant taskq] 100019 I [swi5: fast taskq] 100047 I [swi1: netisr 0] 100048 I [swi4: clock (0)] 100049 I [swi4: clock (1)] 100050 I [swi4: clock (2)] 100051 I [swi4: clock (3)] 100052 I [swi4: clock (4)] 100053 I [swi4: clock (5)] 100054 I [swi4: clock (6)] 100055 I [swi4: clock (7)] 100056 I [swi4: clock (8)] 100057 I [swi4: clock (9)] 100058 I [swi4: clock (10)] 100059 I [swi4: clock (11)] 100060 I [swi3: vm] 100094 I [irq56: ahci0] 100095 I [irq19: xhci0] 100101 I [irq18: ehci0 ehci1] 100112 I [irq71: ahci1] 100113 I [swi0: uart uart] 100180 I [irq72: ioat0] 100181 I [irq73: ioat1] 100182 I [irq74: ioat2] 100183 I [irq75: ioat3] 100184 I [irq76: ioat4] 100185 I [irq77: ioat5] 100186 I [irq78: ioat6] 100187 I [irq79: ioat7] 11 0 0 0 RL (threaded) [idle] 100003 CanRun [idle: cpu0] 100004 Run CPU 1 [idle: cpu1] 100005 CanRun [idle: cpu2] 100006 CanRun [idle: cpu3] 100007 CanRun [idle: cpu4] 100008 CanRun [idle: cpu5] 100009 CanRun [idle: cpu6] 100010 CanRun [idle: cpu7] 100011 Run CPU 8 [idle: cpu8] 100012 CanRun [idle: cpu9] 100013 Run CPU 10 [idle: cpu10] 100014 CanRun [idle: cpu11] 1 0 1 0 SLs wait 0xfffff80003896000 [init] 10 0 0 0 DL audit_w 0xffffffff81de5cf0 [audit] 0 0 0 0 DLs (threaded) [kernel] 100000 D swapin 0xffffffff81d8d688 [swapper] 100015 D - 0xfffff8000377f100 [aiod_kick taskq] 100018 D - 0xfffff8000377ea00 [thread taskq] 100020 D - 0xfffff80003777000 [config_0] 100021 D - 0xfffff8000377e500 [kqueue_ctx taskq] 100022 D - 0xfffff80003776e00 [if_io_tqg_0] 100023 D - 0xfffff80003776d00 [if_io_tqg_1] 100024 D - 0xfffff80003776c00 [if_io_tqg_2] 100025 D - 0xfffff80003776b00 [if_io_tqg_3] 100026 D - 0xfffff80003776a00 [if_io_tqg_4] 100027 D - 0xfffff80003776900 [if_io_tqg_5] 100028 D - 0xfffff80003776800 [if_io_tqg_6] 100029 D - 0xfffff80003776700 [if_io_tqg_7] 100030 D - 0xfffff80003776600 [if_io_tqg_8] 100031 D - 0xfffff80003776500 [if_io_tqg_9] 100032 D - 0xfffff80003776400 [if_io_tqg_10] 100033 D - 0xfffff80003776300 [if_io_tqg_11] 100034 D - 0xfffff800039e8300 [if_config_tqg_0] 100035 D - 0xfffff800039e8200 [softirq_0] 100036 D - 0xfffff800039e8100 [softirq_1] 100037 D - 0xfffff800039e8000 [softirq_2] 100038 D - 0xfffff800039e7e00 [softirq_3] 100039 D - 0xfffff800039e7d00 [softirq_4] 100040 D - 0xfffff800039e7c00 [softirq_5] 100041 D - 0xfffff800039e7b00 [softirq_6] 100042 D - 0xfffff800039e7a00 [softirq_7] 100043 D - 0xfffff800039e7900 [softirq_8] 100044 D - 0xfffff800039e7800 [softirq_9] 100045 D - 0xfffff800039e7700 [softirq_10] 100046 D - 0xfffff800039e7600 [softirq_11] 100061 D - 0xfffff8000377c700 [firmware taskq] 100066 D - 0xfffff8000377c100 [crypto_0] 100067 D - 0xfffff8000377c100 [crypto_1] 100068 D - 0xfffff8000377c100 [crypto_2] 100069 D - 0xfffff8000377c100 [crypto_3] 100070 D - 0xfffff8000377c100 [crypto_4] 100071 D - 0xfffff8000377c100 [crypto_5] 100072 D - 0xfffff8000377c100 [crypto_6] 100073 D - 0xfffff8000377c100 [crypto_7] 100074 D - 0xfffff8000377c100 [crypto_8] 100075 D - 0xfffff8000377c100 [crypto_9] 100076 D - 0xfffff8000377c100 [crypto_10] 100077 D - 0xfffff8000377c100 [crypto_11] 100114 D - 0xfffff800041f2000 [mca taskq] 100120 D - 0xfffff800200deb00 [acpi_task_0] 100121 D - 0xfffff800200deb00 [acpi_task_1] 100122 D - 0xfffff800200deb00 [acpi_task_2] 100124 D - 0xfffff80003aaaa00 [CAM taskq] 100308 D - 0xfffff80695c7f500 [dtrace_taskq] 100673 D - 0xfffff80418ef0000 [system_taskq_0] 100674 D - 0xfffff80418ef0000 [system_taskq_1] 100680 D - 0xfffff80418ef0000 [system_taskq_2] 100692 D - 0xfffff80418ef0000 [system_taskq_3] 100694 D - 0xfffff80418ef0000 [system_taskq_4] 100695 D - 0xfffff80418ef0000 [system_taskq_5] 100699 D - 0xfffff80418ef0000 [system_taskq_6] 100700 D - 0xfffff80418ef0000 [system_taskq_7] 100703 D - 0xfffff80418ef0000 [system_taskq_8] 100717 D - 0xfffff80418ef0000 [system_taskq_9] 101351 D - 0xfffff80418ef0000 [system_taskq_10] 101356 D - 0xfffff80418ef0000 [system_taskq_11] db:0:pho> allt Tracing command kldload pid 69162 tid 104200 td 0xfffffe014b82e100 (CPU 0) kdb_enter() at kdb_enter+0x37/frame 0xfffffe0337fbc060 vpanic() at vpanic+0x1a1/frame 0xfffffe0337fbc0c0 panic() at panic+0x43/frame 0xfffffe0337fbc120 vm_fault() at vm_fault+0x1a98/frame 0xfffffe0337fbc250 vm_fault_trap() at vm_fault_trap+0x6e/frame 0xfffffe0337fbc290 trap_pfault() at trap_pfault+0x1f3/frame 0xfffffe0337fbc310 trap() at trap+0x2a7/frame 0xfffffe0337fbc440 calltrap() at calltrap+0x8/frame 0xfffffe0337fbc440 --- trap 0xc, rip = 0xffffffff80c27d2d, rsp = 0xfffffe0337fbc510, rbp = 0xfffffe0337fbc530 --- smr_create() at smr_create+0x5d/frame 0xfffffe0337fbc530 zone_ctor() at zone_ctor+0x38a/frame 0xfffffe0337fbc5a0 item_ctor() at item_ctor+0x2d9/frame 0xfffffe0337fbc600 zone_alloc_item() at zone_alloc_item+0x134/frame 0xfffffe0337fbc650 uma_zcreate() at uma_zcreate+0xc3/frame 0xfffffe0337fbc6c0 smrs_modevent() at smrs_modevent+0x44/frame 0xfffffe0337fbc6f0 module_register_init() at module_register_init+0xbd/frame 0xfffffe0337fbc720 linker_load_module() at linker_load_module+0xc25/frame 0xfffffe0337fbca40 kern_kldload() at kern_kldload+0xef/frame 0xfffffe0337fbca90 sys_kldload() at sys_kldload+0x5b/frame 0xfffffe0337fbcac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe0337fbcbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0337fbcbf0 --- syscall (304, FreeBSD ELF64, sys_kldload), rip = 0x8002dbeaa, rsp = 0x7fffffffe158, rbp = 0x7fffffffe6d0 --- Tracing command swap pid 69161 tid 104277 td 0xfffffe0168e86100 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0336744830 mi_switch() at mi_switch+0x159/frame 0xfffffe0336744870 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe03367448b0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe0336744910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe0336744950 _sleep() at _sleep+0x258/frame 0xfffffe03367449f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe0336744a80 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe0336744ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe0336744bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0336744bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003c6dca, rsp = 0x7fffffffe4c8, rbp = 0x7fffffffe510 --- Tracing command swap pid 69160 tid 104332 td 0xfffffe011d441500 (CPU 7) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1c7dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1c7df0 trap() at trap+0x56/frame 0xfffffe003e1c7f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1c7f20 --- trap 0x13, rip = 0xffffffff80c37387, rsp = 0xfffffe0334a54880, rbp = 0xfffffe0334a548b0 --- witness_unlock() at witness_unlock+0x47/frame 0xfffffe0334a548b0 _rw_wunlock_cookie() at _rw_wunlock_cookie+0x59/frame 0xfffffe0334a548d0 vm_fault() at vm_fault+0xbb0/frame 0xfffffe0334a54a00 vm_fault_trap() at vm_fault_trap+0x6e/frame 0xfffffe0334a54a40 trap_pfault() at trap_pfault+0x1f3/frame 0xfffffe0334a54ac0 trap() at trap+0x457/frame 0xfffffe0334a54bf0 calltrap() at calltrap+0x8/frame 0xfffffe0334a54bf0 --- trap 0xc, rip = 0x800437015, rsp = 0x7fffffffe198, rbp = 0x7fffffffe1e0 --- Tracing command swap pid 69159 tid 104214 td 0xfffffe011dff2500 (CPU 9) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1dfdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1dfdf0 trap() at trap+0x56/frame 0xfffffe003e1dff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1dff20 --- trap 0x13, rip = 0x203689, rsp = 0x7fffffffe4f0, rbp = 0x7fffffffe510 --- Tracing command swap pid 69158 tid 103674 td 0xfffffe011d3f2300 sched_switch() at sched_switch+0x5ba/frame 0xfffffe033616c830 mi_switch() at mi_switch+0x159/frame 0xfffffe033616c870 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe033616c8b0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe033616c910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe033616c950 _sleep() at _sleep+0x258/frame 0xfffffe033616c9f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe033616ca80 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe033616cac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe033616cbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe033616cbf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003c6dca, rsp = 0x7fffffffe4c8, rbp = 0x7fffffffe510 --- Tracing command swap pid 69157 tid 103523 td 0xfffffe0168e3d300 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0335e09830 mi_switch() at mi_switch+0x159/frame 0xfffffe0335e09870 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0335e098b0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe0335e09910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe0335e09950 _sleep() at _sleep+0x258/frame 0xfffffe0335e099f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe0335e09a80 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe0335e09ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe0335e09bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0335e09bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003c6dca, rsp = 0x7fffffffe4c8, rbp = 0x7fffffffe510 --- Tracing command swap pid 69156 tid 104146 td 0xfffffe016b53a000 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0337e95830 mi_switch() at mi_switch+0x159/frame 0xfffffe0337e95870 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0337e958b0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe0337e95910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe0337e95950 _sleep() at _sleep+0x258/frame 0xfffffe0337e959f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe0337e95a80 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe0337e95ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe0337e95bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0337e95bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003c6dca, rsp = 0x7fffffffe4c8, rbp = 0x7fffffffe510 --- Tracing command swap pid 69155 tid 104091 td 0xfffffe011d435500 (CPU 2) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e18bdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e18bdf0 trap() at trap+0x56/frame 0xfffffe003e18bf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e18bf20 --- trap 0x13, rip = 0x203687, rsp = 0x7fffffffe4f0, rbp = 0x7fffffffe510 --- Tracing command swap pid 69154 tid 104224 td 0xfffffe0169e2f800 (CPU 5) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1afdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1afdf0 trap() at trap+0x56/frame 0xfffffe003e1aff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1aff20 --- trap 0x13, rip = 0xffffffff80f2f10c, rsp = 0xfffffe03366367e0, rbp = 0xfffffe03366367e0 --- vm_page_alloc_check() at vm_page_alloc_check+0xc/frame 0xfffffe03366367e0 vm_page_alloc_domain_after() at vm_page_alloc_domain_after+0x332/frame 0xfffffe0336636870 vm_page_alloc() at vm_page_alloc+0x74/frame 0xfffffe03366368d0 vm_fault() at vm_fault+0x531/frame 0xfffffe0336636a00 vm_fault_trap() at vm_fault_trap+0x6e/frame 0xfffffe0336636a40 trap_pfault() at trap_pfault+0x1f3/frame 0xfffffe0336636ac0 trap() at trap+0x457/frame 0xfffffe0336636bf0 calltrap() at calltrap+0x8/frame 0xfffffe0336636bf0 --- trap 0xc, rip = 0x800437015, rsp = 0x7fffffffe198, rbp = 0x7fffffffe1e0 --- Tracing command swap pid 69153 tid 104163 td 0xfffffe0168e67700 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0337ee5830 mi_switch() at mi_switch+0x159/frame 0xfffffe0337ee5870 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0337ee58b0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe0337ee5910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe0337ee5950 _sleep() at _sleep+0x258/frame 0xfffffe0337ee59f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe0337ee5a80 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe0337ee5ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe0337ee5bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0337ee5bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003c6dca, rsp = 0x7fffffffe4c8, rbp = 0x7fffffffe510 --- Tracing command swap pid 69152 tid 104225 td 0xfffffe0168e73000 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0337ffd830 mi_switch() at mi_switch+0x159/frame 0xfffffe0337ffd870 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0337ffd8b0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe0337ffd910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe0337ffd950 _sleep() at _sleep+0x258/frame 0xfffffe0337ffd9f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe0337ffda80 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe0337ffdac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe0337ffdbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0337ffdbf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003c6dca, rsp = 0x7fffffffe4c8, rbp = 0x7fffffffe510 --- Tracing command swap pid 69151 tid 104222 td 0xfffffe011d41e300 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0336609830 mi_switch() at mi_switch+0x159/frame 0xfffffe0336609870 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe03366098b0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe0336609910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe0336609950 _sleep() at _sleep+0x258/frame 0xfffffe03366099f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe0336609a80 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe0336609ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe0336609bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0336609bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003c6dca, rsp = 0x7fffffffe4c8, rbp = 0x7fffffffe510 --- Tracing command swap pid 69150 tid 104257 td 0xfffffe0168e6c500 sched_switch() at sched_switch+0x5ba/frame 0xfffffe033669f830 mi_switch() at mi_switch+0x159/frame 0xfffffe033669f870 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe033669f8b0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe033669f910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe033669f950 _sleep() at _sleep+0x258/frame 0xfffffe033669f9f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe033669fa80 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe033669fac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe033669fbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe033669fbf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003c6dca, rsp = 0x7fffffffe4c8, rbp = 0x7fffffffe510 --- Tracing command swap pid 69149 tid 104374 td 0xfffffe014b85da00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0334b1c830 mi_switch() at mi_switch+0x159/frame 0xfffffe0334b1c870 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0334b1c8b0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe0334b1c910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe0334b1c950 _sleep() at _sleep+0x258/frame 0xfffffe0334b1c9f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe0334b1ca80 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe0334b1cac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe0334b1cbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0334b1cbf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003c6dca, rsp = 0x7fffffffe4c8, rbp = 0x7fffffffe510 --- Tracing command swap pid 69148 tid 104311 td 0xfffffe0168e9b700 sched_switch() at sched_switch+0x5ba/frame 0xfffffe03367b7830 mi_switch() at mi_switch+0x159/frame 0xfffffe03367b7870 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe03367b78b0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe03367b7910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe03367b7950 _sleep() at _sleep+0x258/frame 0xfffffe03367b79f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe03367b7a80 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe03367b7ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe03367b7bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe03367b7bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003c6dca, rsp = 0x7fffffffe4c8, rbp = 0x7fffffffe510 --- Tracing command swap pid 69147 tid 104096 td 0xfffffe0168e80c00 (CPU 6) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1bbdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1bbdf0 trap() at trap+0x56/frame 0xfffffe003e1bbf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1bbf20 --- trap 0x13, rip = 0x203687, rsp = 0x7fffffffe4f0, rbp = 0x7fffffffe510 --- Tracing command swap pid 69146 tid 104333 td 0xfffffe00e79fc800 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0334a3b830 mi_switch() at mi_switch+0x159/frame 0xfffffe0334a3b870 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0334a3b8b0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe0334a3b910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe0334a3b950 _sleep() at _sleep+0x258/frame 0xfffffe0334a3b9f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe0334a3ba80 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe0334a3bac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe0334a3bbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0334a3bbf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003c6dca, rsp = 0x7fffffffe4c8, rbp = 0x7fffffffe510 --- Tracing command swap pid 69145 tid 104134 td 0xfffffe016fa6c700 (CPU 11) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1f7dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1f7df0 trap() at trap+0x56/frame 0xfffffe003e1f7f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1f7f20 --- trap 0x13, rip = 0x203689, rsp = 0x7fffffffe4f0, rbp = 0x7fffffffe510 --- Tracing command swap pid 69144 tid 104265 td 0xfffffe014b818c00 (CPU 3) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e197dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e197df0 trap() at trap+0x56/frame 0xfffffe003e197f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e197f20 --- trap 0x13, rip = 0x203687, rsp = 0x7fffffffe4f0, rbp = 0x7fffffffe510 --- Tracing command swap pid 69143 tid 104011 td 0xfffffe0168e67000 (CPU 4) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1a3dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1a3df0 trap() at trap+0x56/frame 0xfffffe003e1a3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1a3f20 --- trap 0x13, rip = 0x203687, rsp = 0x7fffffffe4f0, rbp = 0x7fffffffe510 --- Tracing command swap pid 69142 tid 103986 td 0xfffffe014b9db800 sched_switch() at sched_switch+0x5ba/frame 0xfffffe033658a6b0 mi_switch() at mi_switch+0x159/frame 0xfffffe033658a6f0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe033658a730 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe033658a790 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe033658a7a0 _sleep() at _sleep+0x2c3/frame 0xfffffe033658a840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe033658a8d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe033658aac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe033658abf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe033658abf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003c6c6a, rsp = 0x7fffffffe538, rbp = 0x7fffffffe570 --- Tracing command swap pid 69140 tid 104181 td 0xfffffe011e019000 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0337f62830 mi_switch() at mi_switch+0x159/frame 0xfffffe0337f62870 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0337f628b0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe0337f62910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe0337f62950 _sleep() at _sleep+0x258/frame 0xfffffe0337f629f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe0337f62a80 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe0337f62ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe0337f62bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0337f62bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003c6dca, rsp = 0x7fffffffe528, rbp = 0x7fffffffe570 --- Tracing command sh pid 69086 tid 104313 td 0xfffffe011d443800 sched_switch() at sched_switch+0x5ba/frame 0xfffffe03367f86b0 mi_switch() at mi_switch+0x159/frame 0xfffffe03367f86f0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe03367f8730 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe03367f8790 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe03367f87a0 _sleep() at _sleep+0x2c3/frame 0xfffffe03367f8840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe03367f88d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe03367f8ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe03367f8bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe03367f8bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80040ac6a, rsp = 0x7fffffffe328, rbp = 0x7fffffffe3e0 --- Tracing command tee pid 69085 tid 104025 td 0xfffffe0168e6a800 sched_switch() at sched_switch+0x5ba/frame 0xfffffe03348277f0 mi_switch() at mi_switch+0x159/frame 0xfffffe0334827830 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0334827870 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe03348278d0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe03348278e0 _sleep() at _sleep+0x2c3/frame 0xfffffe0334827980 pipe_read() at pipe_read+0x428/frame 0xfffffe03348279e0 dofileread() at dofileread+0x81/frame 0xfffffe0334827a30 kern_readv() at kern_readv+0x49/frame 0xfffffe0334827a70 sys_read() at sys_read+0x83/frame 0xfffffe0334827ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe0334827bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0334827bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80041b8ea, rsp = 0x7fffffffe688, rbp = 0x7fffffffe6d0 --- Tracing command sh pid 69084 tid 104312 td 0xfffffe00d3df2500 sched_switch() at sched_switch+0x5ba/frame 0xfffffe03367ee6b0 mi_switch() at mi_switch+0x159/frame 0xfffffe03367ee6f0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe03367ee730 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe03367ee790 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe03367ee7a0 _sleep() at _sleep+0x2c3/frame 0xfffffe03367ee840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe03367ee8d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe03367eeac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe03367eebf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe03367eebf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80040ac6a, rsp = 0x7fffffffb528, rbp = 0x7fffffffb5e0 --- Tracing command sleep pid 68971 tid 103776 td 0xfffffe01201ff800 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0336376830 mi_switch() at mi_switch+0x159/frame 0xfffffe0336376870 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe03363768b0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe0336376910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe0336376950 _sleep() at _sleep+0x258/frame 0xfffffe03363769f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe0336376a80 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe0336376ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe0336376bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0336376bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003abdca, rsp = 0x7fffffffe678, rbp = 0x7fffffffe6c0 --- Tracing command ftcleanup pid 11521 tid 100232 td 0xfffffe00e63e0700 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e57c2a40 mi_switch() at mi_switch+0x159/frame 0xfffffe00e57c2a80 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e57c2ac0 _sleep() at _sleep+0x2cd/frame 0xfffffe00e57c2b60 fasttrap_pid_cleanup_cb() at fasttrap_pid_cleanup_cb+0x27c/frame 0xfffffe00e57c2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e57c2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e57c2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sh pid 4405 tid 100231 td 0xfffffe00dd1ed500 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e57bd7f0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e57bd830 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e57bd870 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e57bd8d0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e57bd8e0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e57bd980 pipe_read() at pipe_read+0x428/frame 0xfffffe00e57bd9e0 dofileread() at dofileread+0x81/frame 0xfffffe00e57bda30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e57bda70 sys_read() at sys_read+0x83/frame 0xfffffe00e57bdac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e57bdbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e57bdbf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80047a8ea, rsp = 0x7fffffffe1f8, rbp = 0x7fffffffe300 --- Tracing command awk pid 4403 tid 100146 td 0xfffffe00dd1d5500 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e37b67f0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e37b6830 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e37b6870 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e37b68d0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e37b68e0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e37b6980 pipe_read() at pipe_read+0x428/frame 0xfffffe00e37b69e0 dofileread() at dofileread+0x81/frame 0xfffffe00e37b6a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e37b6a70 sys_read() at sys_read+0x83/frame 0xfffffe00e37b6ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e37b6bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e37b6bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80047f8ea, rsp = 0x7fffffffe2f8, rbp = 0x7fffffffe310 --- Tracing command sh pid 4402 tid 100238 td 0xfffffe00e5be3800 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e57e06b0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e57e06f0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e57e0730 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e57e0790 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e57e07a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e57e0840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00e57e08d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00e57e0ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e57e0bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e57e0bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80040ac6a, rsp = 0x7fffffffe208, rbp = 0x7fffffffe2c0 --- Tracing command sh pid 4348 tid 100156 td 0xfffffe00d39ce500 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e37e86b0 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0337fbb6c0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe0337fbb6d0 trap() at trap+0x6c/frame 0xfffffe0337fbb800 calltrap() at calltrap+0x8/frame 0xfffffe0337fbb800 --- trap 0xc, rip = 0xffffffff81046e40, rsp = 0xfffffe0337fbb8d0, rbp = 0xfffffe0337fbb960 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe0337fbb960 db_get_value() at db_get_value+0x32/frame 0xfffffe0337fbb9a0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe0337fbba30 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe0337fbba50 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe0337fbbaf0 db_command() at db_command+0x25c/frame 0xfffffe0337fbbbc0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe0337fbbc60 db_command() at db_command+0x25c/frame 0xfffffe0337fbbd30 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe0337fbbd40 db_trap() at db_trap+0xe8/frame 0xfffffe0337fbbdd0 kdb_trap() at kdb_trap+0x157/frame 0xfffffe0337fbbe50 trap() at trap+0x7ae/frame 0xfffffe0337fbbf80 calltrap() at calltrap+0x8/frame 0xfffffe0337fbbf80 --- trap 0x3, rip = 0xffffffff80c14557, rsp = 0xfffffe0337fbc050, rbp = 0xfffffe0337fbc060 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe0337fbc060 vpanic() at vpanic+0x1a1/frame 0xfffffe0337fbc0c0 panic() at panic+0x43/frame 0xfffffe0337fbc120 vm_fault() at vm_fault+0x1a98/frame 0xfffffe0337fbc250 vm_fault_trap() at vm_fault_trap+0x6e/frame 0xfffffe0337fbc290 trap_pfault() at trap_pfault+0x1f3/frame 0xfffffe0337fbc310 trap() at trap+0x2a7/frame 0xfffffe0337fbc440 calltrap() at calltrap+0x8/frame 0xfffffe0337fbc440 --- trap 0xc, rip = 0xffffffff80c27d2d, rsp = 0xfffffe0337fbc510, rbp = 0xfffffe0337fbc530 --- smr_create() at smr_create+0x5d/frame 0xfffffe0337fbc530 zone_ctor() at zone_ctor+0x38a/frame 0xfffffe0337fbc5a0 item_ctor() at item_ctor+0x2d9/frame 0xfffffe0337fbc600 zone_alloc_item() at zone_alloc_item+0x134/frame 0xfffffe0337fbc650 uma_zcreate() at uma_zcreate+0xc3/frame 0xfffffe0337fbc6c0 smrs_modevent() at smrs_modevent+0x44/frame 0xfffffe0337fbc6f0 module_register_init() at module_register_init+0xbd/frame 0xfffffe0337fbc720 linker_load_module() at linker_load_module+0xc25/frame 0xfffffe0337fbca40 kern_kldload() at kern_kldload+0xef/frame 0xfffffe0337fbca90 sys_kldload() at sys_kldload+0x5b/frame 0xfffffe0337fbcac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe0337fbcbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0337fbcbf0 --- syscall (304, FreeBSD ELF64, sys_kldload), rip = 0x8002dbeaa, rsp = 0x7fffffffe158, rbp = 0x7fffffffe6d0 --- *** error reading from address fffffe00e37e86b8 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0337fbb950 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe0337fbb960 db_get_value() at db_get_value+0x4c/frame 0xfffffe0337fbb9a0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe0337fbba30 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe0337fbba50 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe0337fbbaf0 db_command() at db_command+0x25c/frame 0xfffffe0337fbbbc0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe0337fbbc60 db_command() at db_command+0x25c/frame 0xfffffe0337fbbd30 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe0337fbbd40 db_trap() at db_trap+0xe8/frame 0xfffffe0337fbbdd0 kdb_trap() at kdb_trap+0x157/frame 0xfffffe0337fbbe50 trap() at trap+0x7ae/frame 0xfffffe0337fbbf80 calltrap() at calltrap+0x8/frame 0xfffffe0337fbbf80 --- trap 0x3, rip = 0xffffffff80c14557, rsp = 0xfffffe0337fbc050, rbp = 0xfffffe0337fbc060 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe0337fbc060 vpanic() at vpanic+0x1a1/frame 0xfffffe0337fbc0c0 panic() at panic+0x43/frame 0xfffffe0337fbc120 vm_fault() at vm_fault+0x1a98/frame 0xfffffe0337fbc250 vm_fault_trap() at vm_fault_trap+0x6e/frame 0xfffffe0337fbc290 trap_pfault() at trap_pfault+0x1f3/frame 0xfffffe0337fbc310 trap() at trap+0x2a7/frame 0xfffffe0337fbc440 calltrap() at calltrap+0x8/frame 0xfffffe0337fbc440 --- trap 0xc, rip = 0xffffffff80c27d2d, rsp = 0xfffffe0337fbc510, rbp = 0xfffffe0337fbc530 --- smr_create() at smr_create+0x5d/frame 0xfffffe0337fbc530 zone_ctor() at zone_ctor+0x38a/frame 0xfffffe0337fbc5a0 item_ctor() at item_ctor+0x2d9/frame 0xfffffe0337fbc600 zone_alloc_item() at zone_alloc_item+0x134/frame 0xfffffe0337fbc650 uma_zcreate() at uma_zcreate+0xc3/frame 0xfffffe0337fbc6c0 smrs_modevent() at smrs_modevent+0x44/frame 0xfffffe0337fbc6f0 module_register_init() at module_register_init+0xbd/frame 0xfffffe0337fbc720 linker_load_module() at linker_load_module+0xc25/frame 0xfffffe0337fbca40 kern_kldload() at kern_kldload+0xef/frame 0xfffffe0337fbca90 sys_kldload() at sys_kldload+0x5b/frame 0xfffffe0337fbcac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe0337fbcbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0337fbcbf0 --- syscall (304, FreeBSD ELF64, sys_kldload), rip = 0x8002dbeaa, rsp = 0x7fffffffe158, rbp = 0x7fffffffe6d0 --- Tracing command sh pid 2440 tid 100147 td 0xfffffe00dd1dce00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e37bb6b0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e37bb6f0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e37bb730 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e37bb790 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e37bb7a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e37bb840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00e37bb8d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00e37bbac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e37bbbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e37bbbf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80040ac6a, rsp = 0x7fffffffb848, rbp = 0x7fffffffb900 --- Tracing command vi pid 2429 tid 100188 td 0xfffffe00e596ac00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e56c7770 mi_switch() at mi_switch+0x159/frame 0xfffffe00e56c77b0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e56c77f0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e56c7850 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e56c7860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e56c78c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e56c78f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e56c7950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e56c7980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e56c79e0 dofileread() at dofileread+0x81/frame 0xfffffe00e56c7a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e56c7a70 sys_read() at sys_read+0x83/frame 0xfffffe00e56c7ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e56c7bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e56c7bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004ff8ea, rsp = 0x7fffffffd3f8, rbp = 0x7fffffffe520 --- Tracing command bash pid 2424 tid 100218 td 0xfffffe00d39cfa00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e57816b0 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0337fbb6c0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe0337fbb6d0 trap() at trap+0x6c/frame 0xfffffe0337fbb800 calltrap() at calltrap+0x8/frame 0xfffffe0337fbb800 --- trap 0xc, rip = 0xffffffff81046e40, rsp = 0xfffffe0337fbb8d0, rbp = 0xfffffe0337fbb960 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe0337fbb960 db_get_value() at db_get_value+0x32/frame 0xfffffe0337fbb9a0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe0337fbba30 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe0337fbba50 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe0337fbbaf0 db_command() at db_command+0x25c/frame 0xfffffe0337fbbbc0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe0337fbbc60 db_command() at db_command+0x25c/frame 0xfffffe0337fbbd30 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe0337fbbd40 db_trap() at db_trap+0xe8/frame 0xfffffe0337fbbdd0 kdb_trap() at kdb_trap+0x157/frame 0xfffffe0337fbbe50 trap() at trap+0x7ae/frame 0xfffffe0337fbbf80 calltrap() at calltrap+0x8/frame 0xfffffe0337fbbf80 --- trap 0x3, rip = 0xffffffff80c14557, rsp = 0xfffffe0337fbc050, rbp = 0xfffffe0337fbc060 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe0337fbc060 vpanic() at vpanic+0x1a1/frame 0xfffffe0337fbc0c0 panic() at panic+0x43/frame 0xfffffe0337fbc120 vm_fault() at vm_fault+0x1a98/frame 0xfffffe0337fbc250 vm_fault_trap() at vm_fault_trap+0x6e/frame 0xfffffe0337fbc290 trap_pfault() at trap_pfault+0x1f3/frame 0xfffffe0337fbc310 trap() at trap+0x2a7/frame 0xfffffe0337fbc440 calltrap() at calltrap+0x8/frame 0xfffffe0337fbc440 --- trap 0xc, rip = 0xffffffff80c27d2d, rsp = 0xfffffe0337fbc510, rbp = 0xfffffe0337fbc530 --- smr_create() at smr_create+0x5d/frame 0xfffffe0337fbc530 zone_ctor() at zone_ctor+0x38a/frame 0xfffffe0337fbc5a0 item_ctor() at item_ctor+0x2d9/frame 0xfffffe0337fbc600 zone_alloc_item() at zone_alloc_item+0x134/frame 0xfffffe0337fbc650 uma_zcreate() at uma_zcreate+0xc3/frame 0xfffffe0337fbc6c0 smrs_modevent() at smrs_modevent+0x44/frame 0xfffffe0337fbc6f0 module_register_init() at module_register_init+0xbd/frame 0xfffffe0337fbc720 linker_load_module() at linker_load_module+0xc25/frame 0xfffffe0337fbca40 kern_kldload() at kern_kldload+0xef/frame 0xfffffe0337fbca90 sys_kldload() at sys_kldload+0x5b/frame 0xfffffe0337fbcac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe0337fbcbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0337fbcbf0 --- syscall (304, FreeBSD ELF64, sys_kldload), rip = 0x8002dbeaa, rsp = 0x7fffffffe158, rbp = 0x7fffffffe6d0 --- *** error reading from address fffffe00e57816b8 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0337fbb950 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe0337fbb960 db_get_value() at db_get_value+0x4c/frame 0xfffffe0337fbb9a0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe0337fbba30 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe0337fbba50 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe0337fbbaf0 db_command() at db_command+0x25c/frame 0xfffffe0337fbbbc0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe0337fbbc60 db_command() at db_command+0x25c/frame 0xfffffe0337fbbd30 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe0337fbbd40 db_trap() at db_trap+0xe8/frame 0xfffffe0337fbbdd0 kdb_trap() at kdb_trap+0x157/frame 0xfffffe0337fbbe50 trap() at trap+0x7ae/frame 0xfffffe0337fbbf80 calltrap() at calltrap+0x8/frame 0xfffffe0337fbbf80 --- trap 0x3, rip = 0xffffffff80c14557, rsp = 0xfffffe0337fbc050, rbp = 0xfffffe0337fbc060 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe0337fbc060 vpanic() at vpanic+0x1a1/frame 0xfffffe0337fbc0c0 panic() at panic+0x43/frame 0xfffffe0337fbc120 vm_fault() at vm_fault+0x1a98/frame 0xfffffe0337fbc250 vm_fault_trap() at vm_fault_trap+0x6e/frame 0xfffffe0337fbc290 trap_pfault() at trap_pfault+0x1f3/frame 0xfffffe0337fbc310 trap() at trap+0x2a7/frame 0xfffffe0337fbc440 calltrap() at calltrap+0x8/frame 0xfffffe0337fbc440 --- trap 0xc, rip = 0xffffffff80c27d2d, rsp = 0xfffffe0337fbc510, rbp = 0xfffffe0337fbc530 --- smr_create() at smr_create+0x5d/frame 0xfffffe0337fbc530 zone_ctor() at zone_ctor+0x38a/frame 0xfffffe0337fbc5a0 item_ctor() at item_ctor+0x2d9/frame 0xfffffe0337fbc600 zone_alloc_item() at zone_alloc_item+0x134/frame 0xfffffe0337fbc650 uma_zcreate() at uma_zcreate+0xc3/frame 0xfffffe0337fbc6c0 smrs_modevent() at smrs_modevent+0x44/frame 0xfffffe0337fbc6f0 module_register_init() at module_register_init+0xbd/frame 0xfffffe0337fbc720 linker_load_module() at linker_load_module+0xc25/frame 0xfffffe0337fbca40 kern_kldload() at kern_kldload+0xef/frame 0xfffffe0337fbca90 sys_kldload() at sys_kldload+0x5b/frame 0xfffffe0337fbcac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe0337fbcbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0337fbcbf0 --- syscall (304, FreeBSD ELF64, sys_kldload), rip = 0x8002dbeaa, rsp = 0x7fffffffe158, rbp = 0x7fffffffe6d0 --- Tracing command sshd pid 2423 tid 100177 td 0xfffffe00e596b300 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e5660710 mi_switch() at mi_switch+0x159/frame 0xfffffe00e5660750 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e5660790 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e56607f0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e5660800 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e5660860 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e56608a0 kern_select() at kern_select+0x952/frame 0xfffffe00e5660a80 sys_select() at sys_select+0x56/frame 0xfffffe00e5660ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e5660bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e5660bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8008f98ca, rsp = 0x7fffffffa538, rbp = 0x7fffffffa570 --- Tracing command sshd pid 2420 tid 100206 td 0xfffffe00dd1d4000 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e5745790 mi_switch() at mi_switch+0x159/frame 0xfffffe00e57457d0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e5745810 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e5745870 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e5745880 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e57458e0 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e5745920 kern_poll() at kern_poll+0x48b/frame 0xfffffe00e5745aa0 sys_poll() at sys_poll+0x50/frame 0xfffffe00e5745ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e5745bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e5745bf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x80089c48a, rsp = 0x7fffffffe598, rbp = 0x7fffffffe5d0 --- Tracing command bash pid 2386 tid 100189 td 0xfffffe003e4b7300 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e56cc6d0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e56cc710 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e56cc750 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e56cc7b0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e56cc7c0 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e56cc820 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e56cc860 kern_select() at kern_select+0x952/frame 0xfffffe00e56cca40 sys_pselect() at sys_pselect+0x127/frame 0xfffffe00e56ccac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e56ccbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e56ccbf0 --- syscall (522, FreeBSD ELF64, sys_pselect), rip = 0x800528d8a, rsp = 0x7fffffffd5d8, rbp = 0x7fffffffd6a0 --- Tracing command csh pid 2384 tid 100207 td 0xfffffe00dd1d3800 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e574a8a0 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0337fbb6c0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe0337fbb6d0 trap() at trap+0x6c/frame 0xfffffe0337fbb800 calltrap() at calltrap+0x8/frame 0xfffffe0337fbb800 --- trap 0xc, rip = 0xffffffff81046e40, rsp = 0xfffffe0337fbb8d0, rbp = 0xfffffe0337fbb960 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe0337fbb960 db_get_value() at db_get_value+0x32/frame 0xfffffe0337fbb9a0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe0337fbba30 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe0337fbba50 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe0337fbbaf0 db_command() at db_command+0x25c/frame 0xfffffe0337fbbbc0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe0337fbbc60 db_command() at db_command+0x25c/frame 0xfffffe0337fbbd30 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe0337fbbd40 db_trap() at db_trap+0xe8/frame 0xfffffe0337fbbdd0 kdb_trap() at kdb_trap+0x157/frame 0xfffffe0337fbbe50 trap() at trap+0x7ae/frame 0xfffffe0337fbbf80 calltrap() at calltrap+0x8/frame 0xfffffe0337fbbf80 --- trap 0x3, rip = 0xffffffff80c14557, rsp = 0xfffffe0337fbc050, rbp = 0xfffffe0337fbc060 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe0337fbc060 vpanic() at vpanic+0x1a1/frame 0xfffffe0337fbc0c0 panic() at panic+0x43/frame 0xfffffe0337fbc120 vm_fault() at vm_fault+0x1a98/frame 0xfffffe0337fbc250 vm_fault_trap() at vm_fault_trap+0x6e/frame 0xfffffe0337fbc290 trap_pfault() at trap_pfault+0x1f3/frame 0xfffffe0337fbc310 trap() at trap+0x2a7/frame 0xfffffe0337fbc440 calltrap() at calltrap+0x8/frame 0xfffffe0337fbc440 --- trap 0xc, rip = 0xffffffff80c27d2d, rsp = 0xfffffe0337fbc510, rbp = 0xfffffe0337fbc530 --- smr_create() at smr_create+0x5d/frame 0xfffffe0337fbc530 zone_ctor() at zone_ctor+0x38a/frame 0xfffffe0337fbc5a0 item_ctor() at item_ctor+0x2d9/frame 0xfffffe0337fbc600 zone_alloc_item() at zone_alloc_item+0x134/frame 0xfffffe0337fbc650 uma_zcreate() at uma_zcreate+0xc3/frame 0xfffffe0337fbc6c0 smrs_modevent() at smrs_modevent+0x44/frame 0xfffffe0337fbc6f0 module_register_init() at module_register_init+0xbd/frame 0xfffffe0337fbc720 linker_load_module() at linker_load_module+0xc25/frame 0xfffffe0337fbca40 kern_kldload() at kern_kldload+0xef/frame 0xfffffe0337fbca90 sys_kldload() at sys_kldload+0x5b/frame 0xfffffe0337fbcac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe0337fbcbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0337fbcbf0 --- syscall (304, FreeBSD ELF64, sys_kldload), rip = 0x8002dbeaa, rsp = 0x7fffffffe158, rbp = 0x7fffffffe6d0 --- *** error reading from address fffffe00e574a8a8 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0337fbb950 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe0337fbb960 db_get_value() at db_get_value+0x4c/frame 0xfffffe0337fbb9a0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe0337fbba30 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe0337fbba50 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe0337fbbaf0 db_command() at db_command+0x25c/frame 0xfffffe0337fbbbc0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe0337fbbc60 db_command() at db_command+0x25c/frame 0xfffffe0337fbbd30 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe0337fbbd40 db_trap() at db_trap+0xe8/frame 0xfffffe0337fbbdd0 kdb_trap() at kdb_trap+0x157/frame 0xfffffe0337fbbe50 trap() at trap+0x7ae/frame 0xfffffe0337fbbf80 calltrap() at calltrap+0x8/frame 0xfffffe0337fbbf80 --- trap 0x3, rip = 0xffffffff80c14557, rsp = 0xfffffe0337fbc050, rbp = 0xfffffe0337fbc060 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe0337fbc060 vpanic() at vpanic+0x1a1/frame 0xfffffe0337fbc0c0 panic() at panic+0x43/frame 0xfffffe0337fbc120 vm_fault() at vm_fault+0x1a98/frame 0xfffffe0337fbc250 vm_fault_trap() at vm_fault_trap+0x6e/frame 0xfffffe0337fbc290 trap_pfault() at trap_pfault+0x1f3/frame 0xfffffe0337fbc310 trap() at trap+0x2a7/frame 0xfffffe0337fbc440 calltrap() at calltrap+0x8/frame 0xfffffe0337fbc440 --- trap 0xc, rip = 0xffffffff80c27d2d, rsp = 0xfffffe0337fbc510, rbp = 0xfffffe0337fbc530 --- smr_create() at smr_create+0x5d/frame 0xfffffe0337fbc530 zone_ctor() at zone_ctor+0x38a/frame 0xfffffe0337fbc5a0 item_ctor() at item_ctor+0x2d9/frame 0xfffffe0337fbc600 zone_alloc_item() at zone_alloc_item+0x134/frame 0xfffffe0337fbc650 uma_zcreate() at uma_zcreate+0xc3/frame 0xfffffe0337fbc6c0 smrs_modevent() at smrs_modevent+0x44/frame 0xfffffe0337fbc6f0 module_register_init() at module_register_init+0xbd/frame 0xfffffe0337fbc720 linker_load_module() at linker_load_module+0xc25/frame 0xfffffe0337fbca40 kern_kldload() at kern_kldload+0xef/frame 0xfffffe0337fbca90 sys_kldload() at sys_kldload+0x5b/frame 0xfffffe0337fbcac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe0337fbcbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0337fbcbf0 --- syscall (304, FreeBSD ELF64, sys_kldload), rip = 0x8002dbeaa, rsp = 0x7fffffffe158, rbp = 0x7fffffffe6d0 --- Tracing command su pid 2383 tid 100174 td 0xfffffe00e5863a00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e56516b0 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0337fbb6c0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe0337fbb6d0 trap() at trap+0x6c/frame 0xfffffe0337fbb800 calltrap() at calltrap+0x8/frame 0xfffffe0337fbb800 --- trap 0xc, rip = 0xffffffff81046e40, rsp = 0xfffffe0337fbb8d0, rbp = 0xfffffe0337fbb960 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe0337fbb960 db_get_value() at db_get_value+0x32/frame 0xfffffe0337fbb9a0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe0337fbba30 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe0337fbba50 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe0337fbbaf0 db_command() at db_command+0x25c/frame 0xfffffe0337fbbbc0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe0337fbbc60 db_command() at db_command+0x25c/frame 0xfffffe0337fbbd30 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe0337fbbd40 db_trap() at db_trap+0xe8/frame 0xfffffe0337fbbdd0 kdb_trap() at kdb_trap+0x157/frame 0xfffffe0337fbbe50 trap() at trap+0x7ae/frame 0xfffffe0337fbbf80 calltrap() at calltrap+0x8/frame 0xfffffe0337fbbf80 --- trap 0x3, rip = 0xffffffff80c14557, rsp = 0xfffffe0337fbc050, rbp = 0xfffffe0337fbc060 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe0337fbc060 vpanic() at vpanic+0x1a1/frame 0xfffffe0337fbc0c0 panic() at panic+0x43/frame 0xfffffe0337fbc120 vm_fault() at vm_fault+0x1a98/frame 0xfffffe0337fbc250 vm_fault_trap() at vm_fault_trap+0x6e/frame 0xfffffe0337fbc290 trap_pfault() at trap_pfault+0x1f3/frame 0xfffffe0337fbc310 trap() at trap+0x2a7/frame 0xfffffe0337fbc440 calltrap() at calltrap+0x8/frame 0xfffffe0337fbc440 --- trap 0xc, rip = 0xffffffff80c27d2d, rsp = 0xfffffe0337fbc510, rbp = 0xfffffe0337fbc530 --- smr_create() at smr_create+0x5d/frame 0xfffffe0337fbc530 zone_ctor() at zone_ctor+0x38a/frame 0xfffffe0337fbc5a0 item_ctor() at item_ctor+0x2d9/frame 0xfffffe0337fbc600 zone_alloc_item() at zone_alloc_item+0x134/frame 0xfffffe0337fbc650 uma_zcreate() at uma_zcreate+0xc3/frame 0xfffffe0337fbc6c0 smrs_modevent() at smrs_modevent+0x44/frame 0xfffffe0337fbc6f0 module_register_init() at module_register_init+0xbd/frame 0xfffffe0337fbc720 linker_load_module() at linker_load_module+0xc25/frame 0xfffffe0337fbca40 kern_kldload() at kern_kldload+0xef/frame 0xfffffe0337fbca90 sys_kldload() at sys_kldload+0x5b/frame 0xfffffe0337fbcac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe0337fbcbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0337fbcbf0 --- syscall (304, FreeBSD ELF64, sys_kldload), rip = 0x8002dbeaa, rsp = 0x7fffffffe158, rbp = 0x7fffffffe6d0 --- *** error reading from address fffffe00e56516b8 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0337fbb950 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe0337fbb960 db_get_value() at db_get_value+0x4c/frame 0xfffffe0337fbb9a0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe0337fbba30 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe0337fbba50 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe0337fbbaf0 db_command() at db_command+0x25c/frame 0xfffffe0337fbbbc0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe0337fbbc60 db_command() at db_command+0x25c/frame 0xfffffe0337fbbd30 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe0337fbbd40 db_trap() at db_trap+0xe8/frame 0xfffffe0337fbbdd0 kdb_trap() at kdb_trap+0x157/frame 0xfffffe0337fbbe50 trap() at trap+0x7ae/frame 0xfffffe0337fbbf80 calltrap() at calltrap+0x8/frame 0xfffffe0337fbbf80 --- trap 0x3, rip = 0xffffffff80c14557, rsp = 0xfffffe0337fbc050, rbp = 0xfffffe0337fbc060 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe0337fbc060 vpanic() at vpanic+0x1a1/frame 0xfffffe0337fbc0c0 panic() at panic+0x43/frame 0xfffffe0337fbc120 vm_fault() at vm_fault+0x1a98/frame 0xfffffe0337fbc250 vm_fault_trap() at vm_fault_trap+0x6e/frame 0xfffffe0337fbc290 trap_pfault() at trap_pfault+0x1f3/frame 0xfffffe0337fbc310 trap() at trap+0x2a7/frame 0xfffffe0337fbc440 calltrap() at calltrap+0x8/frame 0xfffffe0337fbc440 --- trap 0xc, rip = 0xffffffff80c27d2d, rsp = 0xfffffe0337fbc510, rbp = 0xfffffe0337fbc530 --- smr_create() at smr_create+0x5d/frame 0xfffffe0337fbc530 zone_ctor() at zone_ctor+0x38a/frame 0xfffffe0337fbc5a0 item_ctor() at item_ctor+0x2d9/frame 0xfffffe0337fbc600 zone_alloc_item() at zone_alloc_item+0x134/frame 0xfffffe0337fbc650 uma_zcreate() at uma_zcreate+0xc3/frame 0xfffffe0337fbc6c0 smrs_modevent() at smrs_modevent+0x44/frame 0xfffffe0337fbc6f0 module_register_init() at module_register_init+0xbd/frame 0xfffffe0337fbc720 linker_load_module() at linker_load_module+0xc25/frame 0xfffffe0337fbca40 kern_kldload() at kern_kldload+0xef/frame 0xfffffe0337fbca90 sys_kldload() at sys_kldload+0x5b/frame 0xfffffe0337fbcac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe0337fbcbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0337fbcbf0 --- syscall (304, FreeBSD ELF64, sys_kldload), rip = 0x8002dbeaa, rsp = 0x7fffffffe158, rbp = 0x7fffffffe6d0 --- Tracing command bash pid 2378 tid 100175 td 0xfffffe00dd1d0700 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e56566b0 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0337fbb6c0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe0337fbb6d0 trap() at trap+0x6c/frame 0xfffffe0337fbb800 calltrap() at calltrap+0x8/frame 0xfffffe0337fbb800 --- trap 0xc, rip = 0xffffffff81046e40, rsp = 0xfffffe0337fbb8d0, rbp = 0xfffffe0337fbb960 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe0337fbb960 db_get_value() at db_get_value+0x32/frame 0xfffffe0337fbb9a0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe0337fbba30 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe0337fbba50 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe0337fbbaf0 db_command() at db_command+0x25c/frame 0xfffffe0337fbbbc0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe0337fbbc60 db_command() at db_command+0x25c/frame 0xfffffe0337fbbd30 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe0337fbbd40 db_trap() at db_trap+0xe8/frame 0xfffffe0337fbbdd0 kdb_trap() at kdb_trap+0x157/frame 0xfffffe0337fbbe50 trap() at trap+0x7ae/frame 0xfffffe0337fbbf80 calltrap() at calltrap+0x8/frame 0xfffffe0337fbbf80 --- trap 0x3, rip = 0xffffffff80c14557, rsp = 0xfffffe0337fbc050, rbp = 0xfffffe0337fbc060 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe0337fbc060 vpanic() at vpanic+0x1a1/frame 0xfffffe0337fbc0c0 panic() at panic+0x43/frame 0xfffffe0337fbc120 vm_fault() at vm_fault+0x1a98/frame 0xfffffe0337fbc250 vm_fault_trap() at vm_fault_trap+0x6e/frame 0xfffffe0337fbc290 trap_pfault() at trap_pfault+0x1f3/frame 0xfffffe0337fbc310 trap() at trap+0x2a7/frame 0xfffffe0337fbc440 calltrap() at calltrap+0x8/frame 0xfffffe0337fbc440 --- trap 0xc, rip = 0xffffffff80c27d2d, rsp = 0xfffffe0337fbc510, rbp = 0xfffffe0337fbc530 --- smr_create() at smr_create+0x5d/frame 0xfffffe0337fbc530 zone_ctor() at zone_ctor+0x38a/frame 0xfffffe0337fbc5a0 item_ctor() at item_ctor+0x2d9/frame 0xfffffe0337fbc600 zone_alloc_item() at zone_alloc_item+0x134/frame 0xfffffe0337fbc650 uma_zcreate() at uma_zcreate+0xc3/frame 0xfffffe0337fbc6c0 smrs_modevent() at smrs_modevent+0x44/frame 0xfffffe0337fbc6f0 module_register_init() at module_register_init+0xbd/frame 0xfffffe0337fbc720 linker_load_module() at linker_load_module+0xc25/frame 0xfffffe0337fbca40 kern_kldload() at kern_kldload+0xef/frame 0xfffffe0337fbca90 sys_kldload() at sys_kldload+0x5b/frame 0xfffffe0337fbcac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe0337fbcbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0337fbcbf0 --- syscall (304, FreeBSD ELF64, sys_kldload), rip = 0x8002dbeaa, rsp = 0x7fffffffe158, rbp = 0x7fffffffe6d0 --- *** error reading from address fffffe00e56566b8 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0337fbb950 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe0337fbb960 db_get_value() at db_get_value+0x4c/frame 0xfffffe0337fbb9a0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe0337fbba30 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe0337fbba50 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe0337fbbaf0 db_command() at db_command+0x25c/frame 0xfffffe0337fbbbc0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe0337fbbc60 db_command() at db_command+0x25c/frame 0xfffffe0337fbbd30 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe0337fbbd40 db_trap() at db_trap+0xe8/frame 0xfffffe0337fbbdd0 kdb_trap() at kdb_trap+0x157/frame 0xfffffe0337fbbe50 trap() at trap+0x7ae/frame 0xfffffe0337fbbf80 calltrap() at calltrap+0x8/frame 0xfffffe0337fbbf80 --- trap 0x3, rip = 0xffffffff80c14557, rsp = 0xfffffe0337fbc050, rbp = 0xfffffe0337fbc060 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe0337fbc060 vpanic() at vpanic+0x1a1/frame 0xfffffe0337fbc0c0 panic() at panic+0x43/frame 0xfffffe0337fbc120 vm_fault() at vm_fault+0x1a98/frame 0xfffffe0337fbc250 vm_fault_trap() at vm_fault_trap+0x6e/frame 0xfffffe0337fbc290 trap_pfault() at trap_pfault+0x1f3/frame 0xfffffe0337fbc310 trap() at trap+0x2a7/frame 0xfffffe0337fbc440 calltrap() at calltrap+0x8/frame 0xfffffe0337fbc440 --- trap 0xc, rip = 0xffffffff80c27d2d, rsp = 0xfffffe0337fbc510, rbp = 0xfffffe0337fbc530 --- smr_create() at smr_create+0x5d/frame 0xfffffe0337fbc530 zone_ctor() at zone_ctor+0x38a/frame 0xfffffe0337fbc5a0 item_ctor() at item_ctor+0x2d9/frame 0xfffffe0337fbc600 zone_alloc_item() at zone_alloc_item+0x134/frame 0xfffffe0337fbc650 uma_zcreate() at uma_zcreate+0xc3/frame 0xfffffe0337fbc6c0 smrs_modevent() at smrs_modevent+0x44/frame 0xfffffe0337fbc6f0 module_register_init() at module_register_init+0xbd/frame 0xfffffe0337fbc720 linker_load_module() at linker_load_module+0xc25/frame 0xfffffe0337fbca40 kern_kldload() at kern_kldload+0xef/frame 0xfffffe0337fbca90 sys_kldload() at sys_kldload+0x5b/frame 0xfffffe0337fbcac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe0337fbcbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0337fbcbf0 --- syscall (304, FreeBSD ELF64, sys_kldload), rip = 0x8002dbeaa, rsp = 0x7fffffffe158, rbp = 0x7fffffffe6d0 --- Tracing command sshd pid 2377 tid 100208 td 0xfffffe00dd1d3100 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e574f710 mi_switch() at mi_switch+0x159/frame 0xfffffe00e574f750 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e574f790 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e574f7f0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e574f800 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e574f860 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e574f8a0 kern_select() at kern_select+0x952/frame 0xfffffe00e574fa80 sys_select() at sys_select+0x56/frame 0xfffffe00e574fac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e574fbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e574fbf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8008f98ca, rsp = 0x7fffffffa538, rbp = 0x7fffffffa570 --- Tracing command sshd pid 2374 tid 100209 td 0xfffffe00dd1d2a00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e5754790 mi_switch() at mi_switch+0x159/frame 0xfffffe00e57547d0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e5754810 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e5754870 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e5754880 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e57548e0 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e5754920 kern_poll() at kern_poll+0x48b/frame 0xfffffe00e5754aa0 sys_poll() at sys_poll+0x50/frame 0xfffffe00e5754ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e5754bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e5754bf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x80089c48a, rsp = 0x7fffffffe598, rbp = 0x7fffffffe5d0 --- Tracing command csh pid 2370 tid 100173 td 0xfffffe00d3df1000 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e564c770 mi_switch() at mi_switch+0x159/frame 0xfffffe00e564c7b0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e564c7f0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e564c850 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e564c860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e564c8c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e564c8f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e564c950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e564c980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e564c9e0 dofileread() at dofileread+0x81/frame 0xfffffe00e564ca30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e564ca70 sys_read() at sys_read+0x83/frame 0xfffffe00e564cac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e564cbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e564cbf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004ff8ea, rsp = 0x7fffffffe1d8, rbp = 0x7fffffffe200 --- Tracing command login pid 2367 tid 100214 td 0xfffffe00e3cf6500 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e576d6b0 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0337fbb6c0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe0337fbb6d0 trap() at trap+0x6c/frame 0xfffffe0337fbb800 calltrap() at calltrap+0x8/frame 0xfffffe0337fbb800 --- trap 0xc, rip = 0xffffffff81046e40, rsp = 0xfffffe0337fbb8d0, rbp = 0xfffffe0337fbb960 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe0337fbb960 db_get_value() at db_get_value+0x32/frame 0xfffffe0337fbb9a0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe0337fbba30 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe0337fbba50 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe0337fbbaf0 db_command() at db_command+0x25c/frame 0xfffffe0337fbbbc0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe0337fbbc60 db_command() at db_command+0x25c/frame 0xfffffe0337fbbd30 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe0337fbbd40 db_trap() at db_trap+0xe8/frame 0xfffffe0337fbbdd0 kdb_trap() at kdb_trap+0x157/frame 0xfffffe0337fbbe50 trap() at trap+0x7ae/frame 0xfffffe0337fbbf80 calltrap() at calltrap+0x8/frame 0xfffffe0337fbbf80 --- trap 0x3, rip = 0xffffffff80c14557, rsp = 0xfffffe0337fbc050, rbp = 0xfffffe0337fbc060 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe0337fbc060 vpanic() at vpanic+0x1a1/frame 0xfffffe0337fbc0c0 panic() at panic+0x43/frame 0xfffffe0337fbc120 vm_fault() at vm_fault+0x1a98/frame 0xfffffe0337fbc250 vm_fault_trap() at vm_fault_trap+0x6e/frame 0xfffffe0337fbc290 trap_pfault() at trap_pfault+0x1f3/frame 0xfffffe0337fbc310 trap() at trap+0x2a7/frame 0xfffffe0337fbc440 calltrap() at calltrap+0x8/frame 0xfffffe0337fbc440 --- trap 0xc, rip = 0xffffffff80c27d2d, rsp = 0xfffffe0337fbc510, rbp = 0xfffffe0337fbc530 --- smr_create() at smr_create+0x5d/frame 0xfffffe0337fbc530 zone_ctor() at zone_ctor+0x38a/frame 0xfffffe0337fbc5a0 item_ctor() at item_ctor+0x2d9/frame 0xfffffe0337fbc600 zone_alloc_item() at zone_alloc_item+0x134/frame 0xfffffe0337fbc650 uma_zcreate() at uma_zcreate+0xc3/frame 0xfffffe0337fbc6c0 smrs_modevent() at smrs_modevent+0x44/frame 0xfffffe0337fbc6f0 module_register_init() at module_register_init+0xbd/frame 0xfffffe0337fbc720 linker_load_module() at linker_load_module+0xc25/frame 0xfffffe0337fbca40 kern_kldload() at kern_kldload+0xef/frame 0xfffffe0337fbca90 sys_kldload() at sys_kldload+0x5b/frame 0xfffffe0337fbcac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe0337fbcbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0337fbcbf0 --- syscall (304, FreeBSD ELF64, sys_kldload), rip = 0x8002dbeaa, rsp = 0x7fffffffe158, rbp = 0x7fffffffe6d0 --- *** error reading from address fffffe00e576d6b8 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0337fbb950 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe0337fbb960 db_get_value() at db_get_value+0x4c/frame 0xfffffe0337fbb9a0 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe0337fbba30 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe0337fbba50 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe0337fbbaf0 db_command() at db_command+0x25c/frame 0xfffffe0337fbbbc0 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe0337fbbc60 db_command() at db_command+0x25c/frame 0xfffffe0337fbbd30 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe0337fbbd40 db_trap() at db_trap+0xe8/frame 0xfffffe0337fbbdd0 kdb_trap() at kdb_trap+0x157/frame 0xfffffe0337fbbe50 trap() at trap+0x7ae/frame 0xfffffe0337fbbf80 calltrap() at calltrap+0x8/frame 0xfffffe0337fbbf80 --- trap 0x3, rip = 0xffffffff80c14557, rsp = 0xfffffe0337fbc050, rbp = 0xfffffe0337fbc060 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe0337fbc060 vpanic() at vpanic+0x1a1/frame 0xfffffe0337fbc0c0 panic() at panic+0x43/frame 0xfffffe0337fbc120 vm_fault() at vm_fault+0x1a98/frame 0xfffffe0337fbc250 vm_fault_trap() at vm_fault_trap+0x6e/frame 0xfffffe0337fbc290 trap_pfault() at trap_pfault+0x1f3/frame 0xfffffe0337fbc310 trap() at trap+0x2a7/frame 0xfffffe0337fbc440 calltrap() at calltrap+0x8/frame 0xfffffe0337fbc440 --- trap 0xc, rip = 0xffffffff80c27d2d, rsp = 0xfffffe0337fbc510, rbp = 0xfffffe0337fbc530 --- smr_create() at smr_create+0x5d/frame 0xfffffe0337fbc530 zone_ctor() at zone_ctor+0x38a/frame 0xfffffe0337fbc5a0 item_ctor() at item_ctor+0x2d9/frame 0xfffffe0337fbc600 zone_alloc_item() at zone_alloc_item+0x134/frame 0xfffffe0337fbc650 uma_zcreate() at uma_zcreate+0xc3/frame 0xfffffe0337fbc6c0 smrs_modevent() at smrs_modevent+0x44/frame 0xfffffe0337fbc6f0 module_register_init() at module_register_init+0xbd/frame 0xfffffe0337fbc720 linker_load_module() at linker_load_module+0xc25/frame 0xfffffe0337fbca40 kern_kldload() at kern_kldload+0xef/frame 0xfffffe0337fbca90 sys_kldload() at sys_kldload+0x5b/frame 0xfffffe0337fbcac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe0337fbcbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0337fbcbf0 --- syscall (304, FreeBSD ELF64, sys_kldload), rip = 0x8002dbeaa, rsp = 0x7fffffffe158, rbp = 0x7fffffffe6d0 --- Tracing command getty pid 2366 tid 100153 td 0xfffffe00d31aa300 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e37d9770 mi_switch() at mi_switch+0x159/frame 0xfffffe00e37d97b0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e37d97f0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e37d9850 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e37d9860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e37d98c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e37d98f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e37d9950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e37d9980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e37d99e0 dofileread() at dofileread+0x81/frame 0xfffffe00e37d9a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e37d9a70 sys_read() at sys_read+0x83/frame 0xfffffe00e37d9ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e37d9bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e37d9bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004378ea, rsp = 0x7fffffffebb8, rbp = 0x7fffffffec00 --- Tracing command getty pid 2365 tid 100179 td 0xfffffe00d31a9500 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e566a770 mi_switch() at mi_switch+0x159/frame 0xfffffe00e566a7b0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e566a7f0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e566a850 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e566a860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e566a8c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e566a8f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e566a950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e566a980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e566a9e0 dofileread() at dofileread+0x81/frame 0xfffffe00e566aa30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e566aa70 sys_read() at sys_read+0x83/frame 0xfffffe00e566aac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e566abf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e566abf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004378ea, rsp = 0x7fffffffebb8, rbp = 0x7fffffffec00 --- Tracing command getty pid 2364 tid 100213 td 0xfffffe00d3df8100 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e5768770 mi_switch() at mi_switch+0x159/frame 0xfffffe00e57687b0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e57687f0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e5768850 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e5768860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e57688c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e57688f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e5768950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e5768980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e57689e0 dofileread() at dofileread+0x81/frame 0xfffffe00e5768a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e5768a70 sys_read() at sys_read+0x83/frame 0xfffffe00e5768ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e5768bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e5768bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004378ea, rsp = 0x7fffffffebb8, rbp = 0x7fffffffec00 --- Tracing command getty pid 2363 tid 100212 td 0xfffffe00d3df8800 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e5763770 mi_switch() at mi_switch+0x159/frame 0xfffffe00e57637b0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e57637f0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e5763850 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e5763860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e57638c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e57638f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e5763950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e5763980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e57639e0 dofileread() at dofileread+0x81/frame 0xfffffe00e5763a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e5763a70 sys_read() at sys_read+0x83/frame 0xfffffe00e5763ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e5763bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e5763bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004378ea, rsp = 0x7fffffffebb8, rbp = 0x7fffffffec00 --- Tracing command getty pid 2362 tid 100211 td 0xfffffe00d39cd000 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e575e770 mi_switch() at mi_switch+0x159/frame 0xfffffe00e575e7b0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e575e7f0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e575e850 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e575e860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e575e8c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e575e8f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e575e950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e575e980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e575e9e0 dofileread() at dofileread+0x81/frame 0xfffffe00e575ea30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e575ea70 sys_read() at sys_read+0x83/frame 0xfffffe00e575eac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e575ebf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e575ebf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004378ea, rsp = 0x7fffffffebb8, rbp = 0x7fffffffec00 --- Tracing command getty pid 2361 tid 100210 td 0xfffffe00d39cd700 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e5759770 mi_switch() at mi_switch+0x159/frame 0xfffffe00e57597b0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e57597f0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e5759850 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e5759860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e57598c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e57598f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e5759950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e5759980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e57599e0 dofileread() at dofileread+0x81/frame 0xfffffe00e5759a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e5759a70 sys_read() at sys_read+0x83/frame 0xfffffe00e5759ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e5759bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e5759bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004378ea, rsp = 0x7fffffffebb8, rbp = 0x7fffffffec00 --- Tracing command getty pid 2360 tid 100171 td 0xfffffe00d39cde00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e5642770 mi_switch() at mi_switch+0x159/frame 0xfffffe00e56427b0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e56427f0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e5642850 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e5642860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e56428c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e56428f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e5642950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e5642980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e56429e0 dofileread() at dofileread+0x81/frame 0xfffffe00e5642a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e5642a70 sys_read() at sys_read+0x83/frame 0xfffffe00e5642ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e5642bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e5642bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004378ea, rsp = 0x7fffffffebb8, rbp = 0x7fffffffec00 --- Tracing command getty pid 2359 tid 100168 td 0xfffffe00d3df1e00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e5627770 mi_switch() at mi_switch+0x159/frame 0xfffffe00e56277b0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e56277f0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e5627850 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e5627860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e56278c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e56278f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e5627950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e5627980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e56279e0 dofileread() at dofileread+0x81/frame 0xfffffe00e5627a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e5627a70 sys_read() at sys_read+0x83/frame 0xfffffe00e5627ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e5627bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e5627bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004378ea, rsp = 0x7fffffffebb8, rbp = 0x7fffffffec00 --- Tracing command getty pid 2358 tid 100142 td 0xfffffe00d39cf300 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e37a2770 mi_switch() at mi_switch+0x159/frame 0xfffffe00e37a27b0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e37a27f0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e37a2850 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e37a2860 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e37a28c0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e37a28f0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e37a2950 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e37a2980 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e37a29e0 dofileread() at dofileread+0x81/frame 0xfffffe00e37a2a30 kern_readv() at kern_readv+0x49/frame 0xfffffe00e37a2a70 sys_read() at sys_read+0x83/frame 0xfffffe00e37a2ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e37a2bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e37a2bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004378ea, rsp = 0x7fffffffebb8, rbp = 0x7fffffffec00 --- Tracing command cron pid 2312 tid 100144 td 0xfffffe00d31aaa00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e37ac830 mi_switch() at mi_switch+0x159/frame 0xfffffe00e37ac870 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e37ac8b0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e37ac910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe00e37ac950 _sleep() at _sleep+0x258/frame 0xfffffe00e37ac9f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe00e37aca80 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe00e37acac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e37acbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e37acbf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003dbdca, rsp = 0x7fffffffea98, rbp = 0x7fffffffec80 --- Tracing command sshd pid 2308 tid 100165 td 0xfffffe00d5cd1300 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e5618710 mi_switch() at mi_switch+0x159/frame 0xfffffe00e5618750 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e5618790 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e56187f0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e5618800 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e5618860 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e56188a0 kern_select() at kern_select+0x952/frame 0xfffffe00e5618a80 sys_select() at sys_select+0x56/frame 0xfffffe00e5618ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e5618bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e5618bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8008f98ca, rsp = 0x7fffffffe628, rbp = 0x7fffffffe660 --- Tracing command powerd pid 2280 tid 100160 td 0xfffffe00dd1ec000 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e37fc6d0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e37fc710 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e37fc750 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e37fc7b0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe00e37fc7f0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x184/frame 0xfffffe00e37fc860 seltdwait() at seltdwait+0x6b/frame 0xfffffe00e37fc8a0 kern_select() at kern_select+0x952/frame 0xfffffe00e37fca80 sys_select() at sys_select+0x56/frame 0xfffffe00e37fcac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e37fcbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e37fcbf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8004348ca, rsp = 0x7fffffffeb48, rbp = 0x7fffffffec90 --- Tracing command ntpd pid 2277 tid 100166 td 0xfffffe00e5864800 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e561d710 mi_switch() at mi_switch+0x159/frame 0xfffffe00e561d750 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e561d790 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e561d7f0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e561d800 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e561d860 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e561d8a0 kern_select() at kern_select+0x952/frame 0xfffffe00e561da80 sys_select() at sys_select+0x56/frame 0xfffffe00e561dac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e561dbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e561dbf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80082c8ca, rsp = 0x7fffffffded8, rbp = 0x7fffffffdf10 --- Tracing command ntpd pid 2277 tid 100621 td 0xfffffe014b82a100 sched_switch() at sched_switch+0x5ba/frame 0xfffffe03367e97f0 mi_switch() at mi_switch+0x159/frame 0xfffffe03367e9830 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe03367e9870 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe03367e98d0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe03367e98e0 _sleep() at _sleep+0x2c3/frame 0xfffffe03367e9980 umtxq_sleep() at umtxq_sleep+0x19e/frame 0xfffffe03367e99e0 do_sem2_wait() at do_sem2_wait+0x38f/frame 0xfffffe03367e9a70 __umtx_op_sem2_wait() at __umtx_op_sem2_wait+0x76/frame 0xfffffe03367e9ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe03367e9bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe03367e9bf0 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x800720a2a, rsp = 0x7fffdffbced8, rbp = 0x7fffdffbcf50 --- Tracing command rpc.lockd pid 2214 tid 100197 td 0xfffffe00dd1eea00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e5718610 mi_switch() at mi_switch+0x159/frame 0xfffffe00e5718650 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e5718690 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e57186f0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe00e5718730 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x184/frame 0xfffffe00e57187a0 svc_run_internal() at svc_run_internal+0xabe/frame 0xfffffe00e57188e0 svc_run() at svc_run+0x253/frame 0xfffffe00e5718940 sys_nlm_syscall() at sys_nlm_syscall+0x745/frame 0xfffffe00e5718ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e5718bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e5718bf0 --- syscall (154, FreeBSD ELF64, sys_nlm_syscall), rip = 0x8002f65ea, rsp = 0x7fffffffebb8, rbp = 0x7fffffffec80 --- Tracing command rpc.statd pid 2211 tid 100195 td 0xfffffe00e58dba00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e570e6d0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e570e710 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e570e750 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e570e7b0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe00e570e7f0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x184/frame 0xfffffe00e570e860 seltdwait() at seltdwait+0x6b/frame 0xfffffe00e570e8a0 kern_select() at kern_select+0x952/frame 0xfffffe00e570ea80 sys_select() at sys_select+0x56/frame 0xfffffe00e570eac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e570ebf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e570ebf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80042b8ca, rsp = 0x7fffffffea68, rbp = 0x7fffffffebb0 --- Tracing command watchdogd pid 2157 tid 100192 td 0xfffffe00e5be4e00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e56e7830 mi_switch() at mi_switch+0x159/frame 0xfffffe00e56e7870 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e56e78b0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e56e7910 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe00e56e7950 _sleep() at _sleep+0x258/frame 0xfffffe00e56e79f0 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe00e56e7a80 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe00e56e7ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e56e7bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e56e7bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003c4dca, rsp = 0x7fffffffea58, rbp = 0x7fffffffeaa0 --- Tracing command rpcbind pid 2078 tid 100194 td 0xfffffe00e5be4000 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e56f1750 mi_switch() at mi_switch+0x159/frame 0xfffffe00e56f1790 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e56f17d0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e56f1830 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe00e56f1870 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x184/frame 0xfffffe00e56f18e0 seltdwait() at seltdwait+0x6b/frame 0xfffffe00e56f1920 kern_poll() at kern_poll+0x48b/frame 0xfffffe00e56f1aa0 sys_poll() at sys_poll+0x50/frame 0xfffffe00e56f1ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e56f1bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e56f1bf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8003d248a, rsp = 0x7fffffffc9f8, rbp = 0x7fffffffec10 --- Tracing command syslogd pid 2068 tid 100164 td 0xfffffe00dd1ef800 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e5613710 mi_switch() at mi_switch+0x159/frame 0xfffffe00e5613750 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e5613790 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e56137f0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e5613800 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e5613860 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e56138a0 kern_select() at kern_select+0x952/frame 0xfffffe00e5613a80 sys_select() at sys_select+0x56/frame 0xfffffe00e5613ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e5613bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e5613bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80043c8ca, rsp = 0x7fffffffe708, rbp = 0x7fffffffec80 --- Tracing command devd pid 1997 tid 100167 td 0xfffffe00e5864100 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e56226d0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e5622710 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e5622750 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e56227b0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe00e56227f0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x184/frame 0xfffffe00e5622860 seltdwait() at seltdwait+0x6b/frame 0xfffffe00e56228a0 kern_select() at kern_select+0x952/frame 0xfffffe00e5622a80 sys_select() at sys_select+0x56/frame 0xfffffe00e5622ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e5622bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e5622bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x2f803a, rsp = 0x7fffffffcb08, rbp = 0x7fffffffeca0 --- Tracing command dhclient pid 1992 tid 100161 td 0xfffffe00dd1d4e00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e5604750 mi_switch() at mi_switch+0x159/frame 0xfffffe00e5604790 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e56047d0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e5604830 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe00e5604870 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x184/frame 0xfffffe00e56048e0 seltdwait() at seltdwait+0x6b/frame 0xfffffe00e5604920 kern_poll() at kern_poll+0x48b/frame 0xfffffe00e5604aa0 sys_poll() at sys_poll+0x50/frame 0xfffffe00e5604ac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e5604bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e5604bf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8003f548a, rsp = 0x7fffffffebe8, rbp = 0x7fffffffec20 --- Tracing command dhclient pid 1928 tid 100154 td 0xfffffe00d31a9c00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e37de790 mi_switch() at mi_switch+0x159/frame 0xfffffe00e37de7d0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e37de810 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e37de870 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e37de880 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e37de8e0 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e37de920 kern_poll() at kern_poll+0x48b/frame 0xfffffe00e37deaa0 sys_poll() at sys_poll+0x50/frame 0xfffffe00e37deac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e37debf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e37debf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8003f548a, rsp = 0x7fffffffebf8, rbp = 0x7fffffffec20 --- Tracing command dhclient pid 1925 tid 100157 td 0xfffffe003e4b8100 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e37ed710 mi_switch() at mi_switch+0x159/frame 0xfffffe00e37ed750 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e37ed790 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e37ed7f0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e37ed800 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e37ed860 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e37ed8a0 kern_select() at kern_select+0x952/frame 0xfffffe00e37eda80 sys_select() at sys_select+0x56/frame 0xfffffe00e37edac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00e37edbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e37edbf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8004528ca, rsp = 0x7fffffffe9a8, rbp = 0x7fffffffea60 --- Tracing command vnlru pid 34 tid 100141 td 0xfffffe00d3df2c00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e379d9b0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e379d9f0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e379da30 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e379da70 _sleep() at _sleep+0x26c/frame 0xfffffe00e379db10 vnlru_proc() at vnlru_proc+0xaed/frame 0xfffffe00e379dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e379dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e379dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command syncer pid 33 tid 100136 td 0xfffffe00d3df3300 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e37849f0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e3784a30 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e3784a70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e3784ab0 _cv_timedwait_sbt() at _cv_timedwait_sbt+0x184/frame 0xfffffe00e3784b20 sched_sync() at sched_sync+0x69d/frame 0xfffffe00e3784bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e3784bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e3784bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100130 td 0xfffffe00d3df3a00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037be6a00 mi_switch() at mi_switch+0x159/frame 0xfffffe0037be6a40 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037be6a80 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0037be6ac0 _sleep() at _sleep+0x26c/frame 0xfffffe0037be6b60 buf_daemon() at buf_daemon+0x112/frame 0xfffffe0037be6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037be6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037be6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100131 td 0xfffffe00d3df7a00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037beb9f0 mi_switch() at mi_switch+0x159/frame 0xfffffe0037beba30 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037beba70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0037bebab0 _sleep() at _sleep+0x26c/frame 0xfffffe0037bebb50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe0037bebbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037bebbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037bebbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100134 td 0xfffffe00d3df7300 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037bfa9f0 mi_switch() at mi_switch+0x159/frame 0xfffffe0037bfaa30 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037bfaa70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0037bfaab0 _sleep() at _sleep+0x26c/frame 0xfffffe0037bfab50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe0037bfabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037bfabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037bfabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100135 td 0xfffffe00d3df6c00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037bff9f0 mi_switch() at mi_switch+0x159/frame 0xfffffe0037bffa30 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037bffa70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0037bffab0 _sleep() at _sleep+0x26c/frame 0xfffffe0037bffb50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe0037bffbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037bffbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037bffbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100137 td 0xfffffe00d3df6500 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e37899f0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e3789a30 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e3789a70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e3789ab0 _sleep() at _sleep+0x26c/frame 0xfffffe00e3789b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe00e3789bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e3789bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e3789bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100138 td 0xfffffe00d3df5e00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e378e9f0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e378ea30 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e378ea70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e378eab0 _sleep() at _sleep+0x26c/frame 0xfffffe00e378eb50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe00e378ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e378ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e378ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100139 td 0xfffffe00d3df5700 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e37939f0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e3793a30 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e3793a70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e3793ab0 _sleep() at _sleep+0x26c/frame 0xfffffe00e3793b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe00e3793bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e3793bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e3793bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100140 td 0xfffffe00d3df5000 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e37989f0 mi_switch() at mi_switch+0x159/frame 0xfffffe00e3798a30 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e3798a70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e3798ab0 _sleep() at _sleep+0x26c/frame 0xfffffe00e3798b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe00e3798bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e3798bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e3798bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100152 td 0xfffffe00dd1ec700 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e37d4a20 mi_switch() at mi_switch+0x159/frame 0xfffffe00e37d4a60 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e37d4aa0 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e37d4ae0 _sleep() at _sleep+0x26c/frame 0xfffffe00e37d4b80 softdep_flush() at softdep_flush+0x175/frame 0xfffffe00e37d4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e37d4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e37d4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 32 tid 100202 td 0xfffffe00dd1d0000 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e5731a20 mi_switch() at mi_switch+0x159/frame 0xfffffe00e5731a60 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e5731aa0 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e5731ae0 _sleep() at _sleep+0x26c/frame 0xfffffe00e5731b80 softdep_flush() at softdep_flush+0x175/frame 0xfffffe00e5731bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e5731bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e5731bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command vmdaemon pid 31 tid 100129 td 0xfffffe00d3df4100 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037be1a20 mi_switch() at mi_switch+0x159/frame 0xfffffe0037be1a60 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037be1aa0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037be1b40 vm_daemon() at vm_daemon+0x82/frame 0xfffffe0037be1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037be1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037be1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 30 tid 100128 td 0xfffffe00d6099e00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037bdc610 mi_switch() at mi_switch+0x159/frame 0xfffffe0037bdc650 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037bdc690 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0037bdc6d0 _sleep() at _sleep+0x26c/frame 0xfffffe0037bdc770 vm_pageout_worker() at vm_pageout_worker+0x279/frame 0xfffffe0037bdcb70 vm_pageout() at vm_pageout+0x193/frame 0xfffffe0037bdcbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037bdcbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037bdcbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 30 tid 100132 td 0xfffffe003e1db300 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037bf0810 mi_switch() at mi_switch+0x159/frame 0xfffffe0037bf0850 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037bf0890 _sleep() at _sleep+0x2cd/frame 0xfffffe0037bf0930 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0xc1b/frame 0xfffffe0037bf0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037bf0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037bf0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 30 tid 100133 td 0xfffffe003e1dac00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037bf5a20 mi_switch() at mi_switch+0x159/frame 0xfffffe0037bf5a60 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037bf5aa0 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0037bf5ae0 _sleep() at _sleep+0x26c/frame 0xfffffe0037bf5b80 uma_reclaim_worker() at uma_reclaim_worker+0xb8/frame 0xfffffe0037bf5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037bf5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037bf5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon1 pid 29 tid 100127 td 0xfffffe00d5cd2100 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037bd7a10 mi_switch() at mi_switch+0x159/frame 0xfffffe0037bd7a50 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037bd7a90 _sleep() at _sleep+0x2cd/frame 0xfffffe0037bd7b30 enc_daemon() at enc_daemon+0xf6/frame 0xfffffe0037bd7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037bd7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037bd7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon0 pid 28 tid 100126 td 0xfffffe00d5cd2800 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037bd2a10 mi_switch() at mi_switch+0x159/frame 0xfffffe0037bd2a50 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037bd2a90 _sleep() at _sleep+0x2cd/frame 0xfffffe0037bd2b30 enc_daemon() at enc_daemon+0xf6/frame 0xfffffe0037bd2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037bd2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037bd2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command rand_harvestq pid 27 tid 100123 td 0xfffffe00d609b300 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037bc39d0 mi_switch() at mi_switch+0x159/frame 0xfffffe0037bc3a10 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037bc3a50 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0037bc3a90 _sleep() at _sleep+0x26c/frame 0xfffffe0037bc3b30 random_kthread() at random_kthread+0x67/frame 0xfffffe0037bc3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037bc3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037bc3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sctp_iterator pid 26 tid 100119 td 0xfffffe00d5ccf000 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037ba3a70 mi_switch() at mi_switch+0x159/frame 0xfffffe0037ba3ab0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037ba3af0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037ba3b90 sctp_iterator_thread() at sctp_iterator_thread+0x59/frame 0xfffffe0037ba3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037ba3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037ba3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod4 pid 25 tid 100118 td 0xfffffe00d31ab100 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037b9ea10 mi_switch() at mi_switch+0x159/frame 0xfffffe0037b9ea50 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037b9ea90 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0037b9ead0 _sleep() at _sleep+0x26c/frame 0xfffffe0037b9eb70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe0037b9ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037b9ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037b9ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod3 pid 24 tid 100117 td 0xfffffe00d31ab800 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037b99a10 mi_switch() at mi_switch+0x159/frame 0xfffffe0037b99a50 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037b99a90 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0037b99ad0 _sleep() at _sleep+0x26c/frame 0xfffffe0037b99b70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe0037b99bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037b99bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037b99bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod2 pid 23 tid 100116 td 0xfffffe00d6099000 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037b94a10 mi_switch() at mi_switch+0x159/frame 0xfffffe0037b94a50 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037b94a90 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0037b94ad0 _sleep() at _sleep+0x26c/frame 0xfffffe0037b94b70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe0037b94bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037b94bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037b94bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod1 pid 22 tid 100115 td 0xfffffe00d6099700 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037b8fa10 mi_switch() at mi_switch+0x159/frame 0xfffffe0037b8fa50 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037b8fa90 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0037b8fad0 _sleep() at _sleep+0x26c/frame 0xfffffe0037b8fb70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe0037b8fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037b8fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037b8fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100096 td 0xfffffe003e4b5e00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037b24a90 mi_switch() at mi_switch+0x159/frame 0xfffffe0037b24ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037b24b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe0037b24b70 usb_process() at usb_process+0xc3/frame 0xfffffe0037b24bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037b24bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037b24bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100097 td 0xfffffe003e4b5700 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037b29a90 mi_switch() at mi_switch+0x159/frame 0xfffffe0037b29ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037b29b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe0037b29b70 usb_process() at usb_process+0xc3/frame 0xfffffe0037b29bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037b29bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037b29bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100098 td 0xfffffe003e4b5000 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037b2ea90 mi_switch() at mi_switch+0x159/frame 0xfffffe0037b2ead0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037b2eb10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe0037b2eb70 usb_process() at usb_process+0xc3/frame 0xfffffe0037b2ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037b2ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037b2ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100099 td 0xfffffe003e17c800 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037b33a90 mi_switch() at mi_switch+0x159/frame 0xfffffe0037b33ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037b33b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe0037b33b70 usb_process() at usb_process+0xc3/frame 0xfffffe0037b33bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037b33bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037b33bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100100 td 0xfffffe003e17c100 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037b38a90 mi_switch() at mi_switch+0x159/frame 0xfffffe0037b38ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037b38b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe0037b38b70 usb_process() at usb_process+0xc3/frame 0xfffffe0037b38bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037b38bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037b38bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100102 td 0xfffffe00d2e1e500 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037b42a90 mi_switch() at mi_switch+0x159/frame 0xfffffe0037b42ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037b42b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe0037b42b70 usb_process() at usb_process+0xc3/frame 0xfffffe0037b42bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037b42bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037b42bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100103 td 0xfffffe00d2e1de00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037b47a90 mi_switch() at mi_switch+0x159/frame 0xfffffe0037b47ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037b47b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe0037b47b70 usb_process() at usb_process+0xc3/frame 0xfffffe0037b47bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037b47bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037b47bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100104 td 0xfffffe00d2e1d700 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037b4ca90 mi_switch() at mi_switch+0x159/frame 0xfffffe0037b4cad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037b4cb10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe0037b4cb70 usb_process() at usb_process+0xc3/frame 0xfffffe0037b4cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037b4cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037b4cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100105 td 0xfffffe00d2e1d000 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037b51a90 mi_switch() at mi_switch+0x159/frame 0xfffffe0037b51ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037b51b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe0037b51b70 usb_process() at usb_process+0xc3/frame 0xfffffe0037b51bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037b51bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037b51bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100106 td 0xfffffe003e4b8800 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037b56a90 mi_switch() at mi_switch+0x159/frame 0xfffffe0037b56ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037b56b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe0037b56b70 usb_process() at usb_process+0xc3/frame 0xfffffe0037b56bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037b56bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037b56bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100107 td 0xfffffe00d31a8e00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037b67a90 mi_switch() at mi_switch+0x159/frame 0xfffffe0037b67ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037b67b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe0037b67b70 usb_process() at usb_process+0xc3/frame 0xfffffe0037b67bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037b67bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037b67bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100108 td 0xfffffe00d31a8700 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037b6ca90 mi_switch() at mi_switch+0x159/frame 0xfffffe0037b6cad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037b6cb10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe0037b6cb70 usb_process() at usb_process+0xc3/frame 0xfffffe0037b6cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037b6cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037b6cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100109 td 0xfffffe00d31a8000 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037b71a90 mi_switch() at mi_switch+0x159/frame 0xfffffe0037b71ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037b71b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe0037b71b70 usb_process() at usb_process+0xc3/frame 0xfffffe0037b71bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037b71bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037b71bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100110 td 0xfffffe00d2e20800 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037b76a90 mi_switch() at mi_switch+0x159/frame 0xfffffe0037b76ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037b76b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe0037b76b70 usb_process() at usb_process+0xc3/frame 0xfffffe0037b76bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037b76bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037b76bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100111 td 0xfffffe00d2e20100 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037b7ba90 mi_switch() at mi_switch+0x159/frame 0xfffffe0037b7bad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037b7bb10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe0037b7bb70 usb_process() at usb_process+0xc3/frame 0xfffffe0037b7bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037b7bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037b7bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100091 td 0xfffffe003e179000 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037b0ba40 mi_switch() at mi_switch+0x159/frame 0xfffffe0037b0ba80 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037b0bac0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037b0bb60 xpt_done_td() at xpt_done_td+0x8b/frame 0xfffffe0037b0bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037b0bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037b0bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100092 td 0xfffffe003e188800 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037b10a40 mi_switch() at mi_switch+0x159/frame 0xfffffe0037b10a80 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037b10ac0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037b10b60 xpt_done_td() at xpt_done_td+0x8b/frame 0xfffffe0037b10bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037b10bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037b10bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100093 td 0xfffffe003e188100 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037b15a40 mi_switch() at mi_switch+0x159/frame 0xfffffe0037b15a80 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037b15ac0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037b15b60 xpt_done_td() at xpt_done_td+0x8b/frame 0xfffffe0037b15bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037b15bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037b15bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100125 td 0xfffffe00d609a500 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037bcda30 mi_switch() at mi_switch+0x159/frame 0xfffffe0037bcda70 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037bcdab0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037bcdb50 xpt_scanner_thread() at xpt_scanner_thread+0x92/frame 0xfffffe0037bcdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037bcdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037bcdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 11 pid 19 tid 100090 td 0xfffffe003e179700 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037afaa20 mi_switch() at mi_switch+0x159/frame 0xfffffe0037afaa60 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037afaaa0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037afab40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe0037afabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037afabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037afabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 10 pid 18 tid 100089 td 0xfffffe003e179e00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037af5a20 mi_switch() at mi_switch+0x159/frame 0xfffffe0037af5a60 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037af5aa0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037af5b40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe0037af5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037af5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037af5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 9 pid 17 tid 100088 td 0xfffffe003e17a500 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037af0a20 mi_switch() at mi_switch+0x159/frame 0xfffffe0037af0a60 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037af0aa0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037af0b40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe0037af0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037af0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037af0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 8 pid 16 tid 100087 td 0xfffffe003e17ac00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037aeba20 mi_switch() at mi_switch+0x159/frame 0xfffffe0037aeba60 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037aebaa0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037aebb40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe0037aebbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037aebbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037aebbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 7 pid 15 tid 100086 td 0xfffffe003e17b300 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037ae6a20 mi_switch() at mi_switch+0x159/frame 0xfffffe0037ae6a60 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037ae6aa0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037ae6b40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe0037ae6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037ae6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037ae6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 6 pid 9 tid 100085 td 0xfffffe003e194800 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037ae1a20 mi_switch() at mi_switch+0x159/frame 0xfffffe0037ae1a60 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037ae1aa0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037ae1b40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe0037ae1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037ae1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037ae1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 5 pid 8 tid 100084 td 0xfffffe003e185000 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037adca20 mi_switch() at mi_switch+0x159/frame 0xfffffe0037adca60 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037adcaa0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037adcb40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe0037adcbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037adcbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037adcbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 4 pid 7 tid 100083 td 0xfffffe003e185700 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037ad7a20 mi_switch() at mi_switch+0x159/frame 0xfffffe0037ad7a60 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037ad7aa0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037ad7b40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe0037ad7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037ad7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037ad7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 3 pid 6 tid 100082 td 0xfffffe003e185e00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037ad2a20 mi_switch() at mi_switch+0x159/frame 0xfffffe0037ad2a60 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037ad2aa0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037ad2b40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe0037ad2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037ad2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037ad2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 2 pid 5 tid 100081 td 0xfffffe003e186500 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037acda20 mi_switch() at mi_switch+0x159/frame 0xfffffe0037acda60 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037acdaa0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037acdb40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe0037acdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037acdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037acdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 1 pid 4 tid 100080 td 0xfffffe003e186c00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037ac8a20 mi_switch() at mi_switch+0x159/frame 0xfffffe0037ac8a60 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037ac8aa0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037ac8b40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe0037ac8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037ac8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037ac8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 0 pid 3 tid 100079 td 0xfffffe003e187300 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037ac3a20 mi_switch() at mi_switch+0x159/frame 0xfffffe0037ac3a60 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037ac3aa0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037ac3b40 crypto_ret_proc() at crypto_ret_proc+0x23d/frame 0xfffffe0037ac3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037ac3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037ac3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 2 tid 100078 td 0xfffffe003e187a00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037abea50 mi_switch() at mi_switch+0x159/frame 0xfffffe0037abea90 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037abead0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037abeb70 crypto_proc() at crypto_proc+0x2bf/frame 0xfffffe0037abebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037abebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037abebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sequencer 00 pid 14 tid 100065 td 0xfffffe003e1c3300 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037a7da50 mi_switch() at mi_switch+0x159/frame 0xfffffe0037a7da90 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037a7dad0 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe0037a7db30 seq_eventthread() at seq_eventthread+0x8c/frame 0xfffffe0037a7dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037a7dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037a7dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100062 td 0xfffffe003e1c4800 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037a6ea50 mi_switch() at mi_switch+0x159/frame 0xfffffe0037a6ea90 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037a6ead0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037a6eb70 g_run_events() at g_run_events+0x64/frame 0xfffffe0037a6ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037a6ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037a6ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100063 td 0xfffffe003e1c4100 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037a73a50 mi_switch() at mi_switch+0x159/frame 0xfffffe0037a73a90 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037a73ad0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037a73b70 g_io_schedule_up() at g_io_schedule_up+0x4c/frame 0xfffffe0037a73ba0 g_up_procbody() at g_up_procbody+0x5c/frame 0xfffffe0037a73bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037a73bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037a73bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100064 td 0xfffffe003e1c3a00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037a78a50 mi_switch() at mi_switch+0x159/frame 0xfffffe0037a78a90 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037a78ad0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037a78b70 g_io_schedule_down() at g_io_schedule_down+0x4c/frame 0xfffffe0037a78ba0 g_down_procbody() at g_down_procbody+0x5c/frame 0xfffffe0037a78bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037a78bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037a78bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100016 td 0xfffffe003e3e5800 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00addb3ae0 mi_switch() at mi_switch+0x159/frame 0xfffffe00addb3b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00addb3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00addb3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00addb3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100017 td 0xfffffe003e3fc300 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00addb8ae0 mi_switch() at mi_switch+0x159/frame 0xfffffe00addb8b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00addb8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00addb8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00addb8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100019 td 0xfffffe003e3fb500 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100047 td 0xfffffe003e1e5e00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00377feae0 mi_switch() at mi_switch+0x159/frame 0xfffffe00377feb20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00377febb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377febf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377febf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100048 td 0xfffffe003e1e5700 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037a04ae0 mi_switch() at mi_switch+0x159/frame 0xfffffe0037a04b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe0037a04bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037a04bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037a04bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100049 td 0xfffffe003e1e5000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100050 td 0xfffffe003e1f4800 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100051 td 0xfffffe003e1f4100 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100052 td 0xfffffe003e1f3a00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100053 td 0xfffffe003e1f3300 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100054 td 0xfffffe003e1c1000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100055 td 0xfffffe003e1d0800 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100056 td 0xfffffe003e1d0100 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100057 td 0xfffffe003e1cfa00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100058 td 0xfffffe003e1cf300 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100059 td 0xfffffe003e1cec00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100060 td 0xfffffe003e1ce500 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100094 td 0xfffffe003e4b6c00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037b1aae0 mi_switch() at mi_switch+0x159/frame 0xfffffe0037b1ab20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe0037b1abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037b1abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037b1abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100095 td 0xfffffe003e4b6500 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037b1fae0 mi_switch() at mi_switch+0x159/frame 0xfffffe0037b1fb20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe0037b1fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037b1fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037b1fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100101 td 0xfffffe003e17ba00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037b3dae0 mi_switch() at mi_switch+0x159/frame 0xfffffe0037b3db20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe0037b3dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037b3dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037b3dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100112 td 0xfffffe00d2e1fa00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100113 td 0xfffffe00d2e1f300 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037b85ae0 mi_switch() at mi_switch+0x159/frame 0xfffffe0037b85b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe0037b85bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037b85bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037b85bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100180 td 0xfffffe00dd1df100 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100181 td 0xfffffe00dd1dea00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100182 td 0xfffffe00dd1de300 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100183 td 0xfffffe00dd1ddc00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100184 td 0xfffffe00dd1dd500 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100185 td 0xfffffe00e5be2300 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100186 td 0xfffffe00e5be1c00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100187 td 0xfffffe00e5be1500 fork_trampoline() at fork_trampoline Tracing command idle pid 11 tid 100003 td 0xfffffe003e3e4300 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00add72a90 mi_switch() at mi_switch+0x159/frame 0xfffffe00add72ad0 sched_idletd() at sched_idletd+0x3e1/frame 0xfffffe00add72bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add72bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add72bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xfffffe003e3e3c00 (CPU 1) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e17fdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e17fdf0 trap() at trap+0x56/frame 0xfffffe003e17ff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e17ff20 --- trap 0x13, rip = 0xffffffff81032d36, rsp = 0xfffffe00add77a30, rbp = 0xfffffe00add77a30 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00add77a30 acpi_cpu_idle() at acpi_cpu_idle+0x23d/frame 0xfffffe00add77a90 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00add77ab0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00add77ad0 sched_idletd() at sched_idletd+0x3e1/frame 0xfffffe00add77bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add77bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add77bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffffe003e3e3500 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00add7ca90 mi_switch() at mi_switch+0x159/frame 0xfffffe00add7cad0 sched_idletd() at sched_idletd+0x3e1/frame 0xfffffe00add7cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add7cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add7cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xfffffe003e3e2e00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00add81a90 mi_switch() at mi_switch+0x159/frame 0xfffffe00add81ad0 sched_idletd() at sched_idletd+0x3e1/frame 0xfffffe00add81bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add81bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add81bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100007 td 0xfffffe003e3e2700 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00add86a90 mi_switch() at mi_switch+0x159/frame 0xfffffe00add86ad0 sched_idletd() at sched_idletd+0x3e1/frame 0xfffffe00add86bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add86bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add86bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xfffffe003e3e2000 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00add8ba90 mi_switch() at mi_switch+0x159/frame 0xfffffe00add8bad0 sched_idletd() at sched_idletd+0x3e1/frame 0xfffffe00add8bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add8bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add8bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xfffffe003e3eca00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00add90a90 mi_switch() at mi_switch+0x159/frame 0xfffffe00add90ad0 sched_idletd() at sched_idletd+0x3e1/frame 0xfffffe00add90bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add90bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add90bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xfffffe003e3ec300 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00add95a90 mi_switch() at mi_switch+0x159/frame 0xfffffe00add95ad0 sched_idletd() at sched_idletd+0x3e1/frame 0xfffffe00add95bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add95bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add95bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffffe003e3ebc00 (CPU 8) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1d3dd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1d3df0 trap() at trap+0x56/frame 0xfffffe003e1d3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1d3f20 --- trap 0x13, rip = 0xffffffff81032d36, rsp = 0xfffffe00add9aa30, rbp = 0xfffffe00add9aa30 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00add9aa30 acpi_cpu_idle() at acpi_cpu_idle+0x23d/frame 0xfffffe00add9aa90 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00add9aab0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00add9aad0 sched_idletd() at sched_idletd+0x3e1/frame 0xfffffe00add9abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add9abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add9abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100012 td 0xfffffe003e3eb500 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00add9fa90 mi_switch() at mi_switch+0x159/frame 0xfffffe00add9fad0 sched_idletd() at sched_idletd+0x9e/frame 0xfffffe00add9fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add9fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add9fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xfffffe003e3eae00 (CPU 10) cpustop_handler() at cpustop_handler+0x2a/frame 0xfffffe003e1ebdd0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xfffffe003e1ebdf0 trap() at trap+0x56/frame 0xfffffe003e1ebf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003e1ebf20 --- trap 0x13, rip = 0xffffffff81032d36, rsp = 0xfffffe00adda4a30, rbp = 0xfffffe00adda4a30 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00adda4a30 acpi_cpu_idle() at acpi_cpu_idle+0x23d/frame 0xfffffe00adda4a90 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00adda4ab0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00adda4ad0 sched_idletd() at sched_idletd+0x3e1/frame 0xfffffe00adda4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adda4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adda4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100014 td 0xfffffe003e3ea700 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00adda9a90 mi_switch() at mi_switch+0x159/frame 0xfffffe00adda9ad0 sched_idletd() at sched_idletd+0x9e/frame 0xfffffe00adda9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adda9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adda9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command init pid 1 tid 100002 td 0xfffffe003e3e4a00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00add6d6b0 mi_switch() at mi_switch+0x159/frame 0xfffffe00add6d6f0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00add6d730 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00add6d790 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00add6d7a0 _sleep() at _sleep+0x2c3/frame 0xfffffe00add6d840 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00add6d8d0 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00add6dac0 amd64_syscall() at amd64_syscall+0x2ee/frame 0xfffffe00add6dbf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00add6dbf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x2b2bfa, rsp = 0x7fffffffe768, rbp = 0x7fffffffe7d0 --- Tracing command audit pid 10 tid 100001 td 0xfffffe003e3e5100 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00add68a70 mi_switch() at mi_switch+0x159/frame 0xfffffe00add68ab0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00add68af0 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00add68b50 audit_worker() at audit_worker+0x9f/frame 0xfffffe00add68bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00add68bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00add68bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xffffffff81d9e1c0 sched_switch() at sched_switch+0x5ba/frame 0xffffffff822ffbf0 mi_switch() at mi_switch+0x159/frame 0xffffffff822ffc30 sleepq_switch() at sleepq_switch+0x11a/frame 0xffffffff822ffc70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xffffffff822ffcb0 _sleep() at _sleep+0x26c/frame 0xffffffff822ffd50 swapper() at swapper+0x75/frame 0xffffffff822ffda0 mi_startup() at mi_startup+0x2ed/frame 0xffffffff822ffdf0 btext() at btext+0x2c Tracing command kernel pid 0 tid 100015 td 0xfffffe003e3ea000 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00addaea60 mi_switch() at mi_switch+0x159/frame 0xfffffe00addaeaa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00addaeae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00addaeb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00addaebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00addaebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00addaebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100018 td 0xfffffe003e3fbc00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00addbda60 mi_switch() at mi_switch+0x159/frame 0xfffffe00addbdaa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00addbdae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00addbdb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00addbdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00addbdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00addbdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xfffffe003e3fae00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00addc7a90 mi_switch() at mi_switch+0x159/frame 0xfffffe00addc7ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00addc7b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00addc7b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00addc7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00addc7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00addc7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xfffffe003e3fa700 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00addcca60 mi_switch() at mi_switch+0x159/frame 0xfffffe00addccaa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00addccae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00addccb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00addccbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00addccbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00addccbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xfffffe003e3fa000 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037781a90 mi_switch() at mi_switch+0x159/frame 0xfffffe0037781ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037781b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe0037781b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe0037781bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037781bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037781bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xfffffe003e3ed800 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037786a90 mi_switch() at mi_switch+0x159/frame 0xfffffe0037786ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037786b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe0037786b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe0037786bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037786bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037786bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xfffffe003e3ed100 sched_switch() at sched_switch+0x5ba/frame 0xfffffe003778ba90 mi_switch() at mi_switch+0x159/frame 0xfffffe003778bad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe003778bb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe003778bb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe003778bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003778bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003778bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xfffffe003e1f2c00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037790a90 mi_switch() at mi_switch+0x159/frame 0xfffffe0037790ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037790b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe0037790b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe0037790bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037790bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037790bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100026 td 0xfffffe003e1f2500 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037795a90 mi_switch() at mi_switch+0x159/frame 0xfffffe0037795ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037795b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe0037795b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe0037795bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037795bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037795bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100027 td 0xfffffe003e1f1e00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe003779aa90 mi_switch() at mi_switch+0x159/frame 0xfffffe003779aad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe003779ab10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe003779ab80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe003779abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003779abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003779abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100028 td 0xfffffe003e1f1700 sched_switch() at sched_switch+0x5ba/frame 0xfffffe003779fa90 mi_switch() at mi_switch+0x159/frame 0xfffffe003779fad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe003779fb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe003779fb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe003779fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe003779fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe003779fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100029 td 0xfffffe003e1f1000 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00377a4a90 mi_switch() at mi_switch+0x159/frame 0xfffffe00377a4ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00377a4b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377a4b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377a4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377a4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377a4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100030 td 0xfffffe003e3fd800 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00377a9a90 mi_switch() at mi_switch+0x159/frame 0xfffffe00377a9ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00377a9b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377a9b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377a9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377a9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377a9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100031 td 0xfffffe003e3fd100 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00377aea90 mi_switch() at mi_switch+0x159/frame 0xfffffe00377aead0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00377aeb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377aeb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377aebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377aebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377aebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100032 td 0xfffffe003e3fca00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00377b3a90 mi_switch() at mi_switch+0x159/frame 0xfffffe00377b3ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00377b3b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377b3b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377b3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377b3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377b3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100033 td 0xfffffe003e1e6500 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00377b8a90 mi_switch() at mi_switch+0x159/frame 0xfffffe00377b8ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00377b8b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377b8b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377b8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377b8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377b8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100034 td 0xfffffe003e1d9e00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00377bda90 mi_switch() at mi_switch+0x159/frame 0xfffffe00377bdad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00377bdb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377bdb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377bdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377bdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377bdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100035 td 0xfffffe003e1d9700 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00377c2a90 mi_switch() at mi_switch+0x159/frame 0xfffffe00377c2ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00377c2b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377c2b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377c2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377c2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377c2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100036 td 0xfffffe003e1d9000 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00377c7a90 mi_switch() at mi_switch+0x159/frame 0xfffffe00377c7ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00377c7b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377c7b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377c7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377c7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377c7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100037 td 0xfffffe003e1e8800 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00377cca90 mi_switch() at mi_switch+0x159/frame 0xfffffe00377ccad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00377ccb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377ccb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377ccbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377ccbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377ccbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100038 td 0xfffffe003e1e8100 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00377d1a90 mi_switch() at mi_switch+0x159/frame 0xfffffe00377d1ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00377d1b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377d1b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377d1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377d1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377d1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100039 td 0xfffffe003e1e7a00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00377d6a90 mi_switch() at mi_switch+0x159/frame 0xfffffe00377d6ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00377d6b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377d6b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377d6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377d6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377d6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100040 td 0xfffffe003e1e7300 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00377dba90 mi_switch() at mi_switch+0x159/frame 0xfffffe00377dbad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00377dbb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377dbb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377dbbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377dbbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377dbbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100041 td 0xfffffe003e1e6c00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00377e0a90 mi_switch() at mi_switch+0x159/frame 0xfffffe00377e0ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00377e0b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377e0b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377e0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377e0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377e0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100042 td 0xfffffe003e1cd700 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00377e5a90 mi_switch() at mi_switch+0x159/frame 0xfffffe00377e5ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00377e5b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377e5b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377e5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377e5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377e5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100043 td 0xfffffe003e1cd000 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00377eaa90 mi_switch() at mi_switch+0x159/frame 0xfffffe00377eaad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00377eab10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377eab80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377eabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377eabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377eabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100044 td 0xfffffe003e1dc800 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00377efa90 mi_switch() at mi_switch+0x159/frame 0xfffffe00377efad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00377efb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377efb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377efbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377efbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377efbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100045 td 0xfffffe003e1dc100 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00377f4a90 mi_switch() at mi_switch+0x159/frame 0xfffffe00377f4ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00377f4b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377f4b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377f4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377f4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377f4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100046 td 0xfffffe003e1dba00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00377f9a90 mi_switch() at mi_switch+0x159/frame 0xfffffe00377f9ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00377f9b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00377f9b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00377f9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00377f9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00377f9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100061 td 0xfffffe003e1cde00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037a69a60 mi_switch() at mi_switch+0x159/frame 0xfffffe0037a69aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037a69ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037a69b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0037a69bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037a69bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037a69bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100066 td 0xfffffe003e1c2c00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037a82a60 mi_switch() at mi_switch+0x159/frame 0xfffffe0037a82aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037a82ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037a82b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0037a82bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037a82bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037a82bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100067 td 0xfffffe003e1c2500 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037a87a60 mi_switch() at mi_switch+0x159/frame 0xfffffe0037a87aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037a87ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037a87b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0037a87bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037a87bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037a87bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100068 td 0xfffffe003e1c1e00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037a8ca60 mi_switch() at mi_switch+0x159/frame 0xfffffe0037a8caa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037a8cae0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037a8cb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0037a8cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037a8cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037a8cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100069 td 0xfffffe003e1c1700 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037a91a60 mi_switch() at mi_switch+0x159/frame 0xfffffe0037a91aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037a91ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037a91b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0037a91bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037a91bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037a91bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100070 td 0xfffffe003e194100 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037a96a60 mi_switch() at mi_switch+0x159/frame 0xfffffe0037a96aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037a96ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037a96b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0037a96bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037a96bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037a96bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100071 td 0xfffffe003e193a00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037a9ba60 mi_switch() at mi_switch+0x159/frame 0xfffffe0037a9baa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037a9bae0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037a9bb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0037a9bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037a9bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037a9bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100072 td 0xfffffe003e193300 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037aa0a60 mi_switch() at mi_switch+0x159/frame 0xfffffe0037aa0aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037aa0ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037aa0b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0037aa0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037aa0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037aa0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100073 td 0xfffffe003e192c00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037aa5a60 mi_switch() at mi_switch+0x159/frame 0xfffffe0037aa5aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037aa5ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037aa5b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0037aa5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037aa5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037aa5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100074 td 0xfffffe003e192500 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037aaaa60 mi_switch() at mi_switch+0x159/frame 0xfffffe0037aaaaa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037aaaae0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037aaab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0037aaabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037aaabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037aaabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100075 td 0xfffffe003e191e00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037aafa60 mi_switch() at mi_switch+0x159/frame 0xfffffe0037aafaa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037aafae0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037aafb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0037aafbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037aafbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037aafbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100076 td 0xfffffe003e191700 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037ab4a60 mi_switch() at mi_switch+0x159/frame 0xfffffe0037ab4aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037ab4ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037ab4b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0037ab4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037ab4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037ab4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100077 td 0xfffffe003e191000 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037ab9a60 mi_switch() at mi_switch+0x159/frame 0xfffffe0037ab9aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037ab9ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037ab9b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0037ab9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037ab9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037ab9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100114 td 0xfffffe00d2e1ec00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037b8aa90 mi_switch() at mi_switch+0x159/frame 0xfffffe0037b8aad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037b8ab10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe0037b8ab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe0037b8abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037b8abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037b8abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100120 td 0xfffffe00d609c800 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037bb4a90 mi_switch() at mi_switch+0x159/frame 0xfffffe0037bb4ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037bb4b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe0037bb4b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe0037bb4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037bb4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037bb4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100121 td 0xfffffe00d609c100 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037bb9a90 mi_switch() at mi_switch+0x159/frame 0xfffffe0037bb9ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037bb9b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe0037bb9b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe0037bb9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037bb9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037bb9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100122 td 0xfffffe00d609ba00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037bbea90 mi_switch() at mi_switch+0x159/frame 0xfffffe0037bbead0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037bbeb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe0037bbeb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe0037bbebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037bbebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037bbebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100124 td 0xfffffe00d609ac00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe0037bc8a60 mi_switch() at mi_switch+0x159/frame 0xfffffe0037bc8aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe0037bc8ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe0037bc8b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0037bc8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe0037bc8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0037bc8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100308 td 0xfffffe00e3cac500 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e6f62a60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e6f62aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e6f62ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00e6f62b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e6f62bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e6f62bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e6f62bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100673 td 0xfffffe00e6782100 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e6f80a60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e6f80aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e6f80ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00e6f80b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e6f80bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e6f80bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e6f80bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100674 td 0xfffffe00e644ee00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e6f26a60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e6f26aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e6f26ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00e6f26b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e6f26bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e6f26bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e6f26bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100680 td 0xfffffe00e643c300 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e6f30a60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e6f30aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e6f30ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00e6f30b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e6f30bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e6f30bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e6f30bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100692 td 0xfffffe00e677b700 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e6f71a60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e6f71aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e6f71ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00e6f71b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e6f71bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e6f71bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e6f71bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100694 td 0xfffffe00e678da00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e6eaea60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e6eaeaa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e6eaeae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00e6eaeb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e6eaebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e6eaebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e6eaebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100695 td 0xfffffe00e6793000 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e6f5da60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e6f5daa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e6f5dae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00e6f5db80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e6f5dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e6f5dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e6f5dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100699 td 0xfffffe00e3cd5e00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e6ec2a60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e6ec2aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e6ec2ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00e6ec2b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e6ec2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e6ec2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e6ec2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100700 td 0xfffffe00e644fc00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e6e59a60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e6e59aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e6e59ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00e6e59b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e6e59bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e6e59bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e6e59bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100703 td 0xfffffe00e66e3a00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e6f67a60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e6f67aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e6f67ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00e6f67b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e6f67bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e6f67bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e6f67bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100717 td 0xfffffe00e6693a00 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e6f8aa60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e6f8aaa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e6f8aae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00e6f8ab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e6f8abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e6f8abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e6f8abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 101351 td 0xfffffe00e678e800 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e6f7ba60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e6f7baa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e6f7bae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00e6f7bb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e6f7bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e6f7bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e6f7bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 101356 td 0xfffffe00e6785300 sched_switch() at sched_switch+0x5ba/frame 0xfffffe00e6ff9a60 mi_switch() at mi_switch+0x159/frame 0xfffffe00e6ff9aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e6ff9ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00e6ff9b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00e6ff9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e6ff9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e6ff9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db:0:pho> show allchains chain 1: thread 104277 (pid 69161, swap) sleeping on 0xffffffff81c82710 "nanslp" chain 2: thread 103674 (pid 69158, swap) sleeping on 0xffffffff81c82713 "nanslp" chain 3: thread 103523 (pid 69157, swap) sleeping on 0xffffffff81c82719 "nanslp" chain 4: thread 104146 (pid 69156, swap) sleeping on 0xffffffff81c82712 "nanslp" chain 5: thread 104163 (pid 69153, swap) sleeping on 0xffffffff81c82717 "nanslp" chain 6: thread 104225 (pid 69152, swap) sleeping on 0xffffffff81c82714 "nanslp" chain 7: thread 104222 (pid 69151, swap) sleeping on 0xffffffff81c82711 "nanslp" chain 8: thread 104257 (pid 69150, swap) sleeping on 0xffffffff81c82715 "nanslp" chain 9: thread 104374 (pid 69149, swap) sleeping on 0xffffffff81c82718 "nanslp" chain 10: thread 104311 (pid 69148, swap) sleeping on 0xffffffff81c82714 "nanslp" chain 11: thread 104333 (pid 69146, swap) sleeping on 0xffffffff81c82714 "nanslp" chain 12: thread 103986 (pid 69142, swap) sleeping on 0xfffff8075c306a60 "wait" chain 13: thread 104181 (pid 69140, swap) sleeping on 0xffffffff81c8271a "nanslp" chain 14: thread 104313 (pid 69086, sh) sleeping on 0xfffff8075c6e7a60 "wait" chain 15: thread 104025 (pid 69085, tee) sleeping on 0xfffff802dd9d45f0 "piperd" chain 16: thread 104312 (pid 69084, sh) sleeping on 0xfffff8075c51e530 "wait" chain 17: thread 103776 (pid 68971, sleep) sleeping on 0xffffffff81c82716 "nanslp" chain 18: thread 100232 (pid 11521, ftcleanup) sleeping on 0xffffffff82e1d4f4 "ftcl" chain 19: thread 100231 (pid 4405, sh) sleeping on 0xfffff80020728be0 "piperd" chain 20: thread 100146 (pid 4403, awk) sleeping on 0xfffff80020a26000 "piperd" chain 21: thread 100238 (pid 4402, sh) sleeping on 0xfffff80110307a60 "wait" chain 22: thread 100156 (pid 4348, sh) sleeping on 0xfffff800205fe530 "wait" chain 23: thread 100147 (pid 2440, sh) sleeping on 0xfffff800209e2000 "wait" chain 24: thread 100188 (pid 2429, vi) sleeping on 0xfffff80003af64b0 "ttyin" chain 25: thread 100218 (pid 2424, bash) sleeping on 0xfffff800206be530 "wait" chain 26: thread 100177 (pid 2423, sshd) sleeping on 0xfffff800041be040 "select" chain 27: thread 100206 (pid 2420, sshd) sleeping on 0xfffff80003e95740 "select" chain 28: thread 100189 (pid 2386, bash) sleeping on 0xfffff80003e959c0 "select" chain 29: thread 100207 (pid 2384, csh) sleeping on 0xfffff800209a80a8 "pause" chain 30: thread 100174 (pid 2383, su) sleeping on 0xfffff800207a9530 "wait" chain 31: thread 100175 (pid 2378, bash) sleeping on 0xfffff800206d6000 "wait" chain 32: thread 100208 (pid 2377, sshd) sleeping on 0xfffff800041689c0 "select" chain 33: thread 100209 (pid 2374, sshd) sleeping on 0xfffff80003edd3c0 "select" chain 34: thread 100173 (pid 2370, csh) sleeping on 0xfffff80017038cb0 "ttyin" chain 35: thread 100214 (pid 2367, login) sleeping on 0xfffff80296fed000 "wait" chain 36: thread 100153 (pid 2366, getty) sleeping on 0xfffff800170370b0 "ttyin" chain 37: thread 100179 (pid 2365, getty) sleeping on 0xfffff8002018f8b0 "ttyin" chain 38: thread 100213 (pid 2364, getty) sleeping on 0xfffff8002018fcb0 "ttyin" chain 39: thread 100212 (pid 2363, getty) sleeping on 0xfffff800201a00b0 "ttyin" chain 40: thread 100211 (pid 2362, getty) sleeping on 0xfffff800201a04b0 "ttyin" chain 41: thread 100210 (pid 2361, getty) sleeping on 0xfffff800201a08b0 "ttyin" chain 42: thread 100171 (pid 2360, getty) sleeping on 0xfffff800201a0cb0 "ttyin" chain 43: thread 100168 (pid 2359, getty) sleeping on 0xfffff8002019f0b0 "ttyin" chain 44: thread 100142 (pid 2358, getty) sleeping on 0xfffff8002019f4b0 "ttyin" chain 45: thread 100144 (pid 2312, cron) sleeping on 0xffffffff81c82713 "nanslp" chain 46: thread 100165 (pid 2308, sshd) sleeping on 0xfffff800203a5e40 "select" chain 47: thread 100160 (pid 2280, powerd) sleeping on 0xfffff800039f45c0 "select" chain 48: thread 100166 (pid 2277, ntpd) sleeping on 0xfffff80003e95a40 "select" chain 49: thread 100621 (pid 2277, ntpd) blocked on lockmgr (null)EXCL chain 50: thread 100197 (pid 2214, NLM: master) sleeping on 0xfffff80020b31d60 "rpcsvc" chain 51: thread 100195 (pid 2211, rpc.statd) sleeping on 0xfffff80003e95ac0 "select" chain 52: thread 100192 (pid 2157, watchdogd) sleeping on 0xffffffff81c82710 "nanslp" chain 53: thread 100194 (pid 2078, rpcbind) sleeping on 0xfffff80003e95bc0 "select" chain 54: thread 100164 (pid 2068, syslogd) sleeping on 0xfffff80003edd540 "select" chain 55: thread 100167 (pid 1997, devd) sleeping on 0xfffff80003edd5c0 "select" chain 56: thread 100161 (pid 1992, dhclient) sleeping on 0xfffff800201d30c0 "select" chain 57: thread 100154 (pid 1928, dhclient) sleeping on 0xfffff800201d3740 "select" chain 58: thread 100157 (pid 1925, dhclient) sleeping on 0xfffff800041be2c0 "select" chain 59: thread 100141 (pid 34, vnlru) sleeping on 0xfffff80020056a60 "vlruwt" chain 60: thread 100136 (pid 33, syncer) sleeping on 0xffffffff81d58b98 "syncer" chain 61: thread 100130 (pid 32, bufdaemon) sleeping on 0xffffffff81d58098 "qsleep" chain 62: thread 100131 (pid 32, bufspacedaemon-0) sleeping on 0xffffffff8180a980 "-" chain 63: thread 100134 (pid 32, bufspacedaemon-1) sleeping on 0xffffffff81812c40 "-" chain 64: thread 100135 (pid 32, bufspacedaemon-2) sleeping on 0xffffffff8181af00 "-" chain 65: thread 100137 (pid 32, bufspacedaemon-3) sleeping on 0xffffffff818231c0 "-" chain 66: thread 100138 (pid 32, bufspacedaemon-4) sleeping on 0xffffffff8182b480 "-" chain 67: thread 100139 (pid 32, bufspacedaemon-5) sleeping on 0xffffffff81833740 "-" chain 68: thread 100140 (pid 32, bufspacedaemon-6) sleeping on 0xffffffff8183ba00 "-" chain 69: thread 100152 (pid 32, / worker) sleeping on 0xfffff80003a728e8 "sdflush" chain 70: thread 100202 (pid 32, /tmp worker) sleeping on 0xfffff8001708fce8 "sdflush" chain 71: thread 100129 (pid 31, vmdaemon) sleeping on 0xffffffff81d73b08 "psleep" chain 72: thread 100128 (pid 30, dom0) sleeping on 0xffffffff81d9fa98 "psleep" chain 73: thread 100132 (pid 30, laundry: dom0) sleeping on 0xffffffff81d9faa4 "launds" chain 74: thread 100133 (pid 30, uma) sleeping on 0xffffffff80f0dc20 "umarcl" chain 75: thread 100127 (pid 29, enc_daemon1) sleeping on 0xfffff8002024b000 "idle" chain 76: thread 100126 (pid 28, enc_daemon0) sleeping on 0xfffff8002024b530 "idle" chain 77: thread 100123 (pid 27, rand_harvestq) sleeping on 0xffffffff81adcfb0 "-" chain 78: thread 100119 (pid 26, sctp_iterator) sleeping on 0xffffffff81de5060 "waiting_for_work" chain 79: thread 100118 (pid 25, soaiod4) sleeping on 0xffffffff81d57a9c "-" chain 80: thread 100117 (pid 24, soaiod3) sleeping on 0xffffffff81d57a9c "-" chain 81: thread 100116 (pid 23, soaiod2) sleeping on 0xffffffff81d57a9c "-" chain 82: thread 100115 (pid 22, soaiod1) sleeping on 0xffffffff81d57a9c "-" chain 83: thread 100096 (pid 21, usbus0) sleeping on 0xfffffe00d2d84440 "-" chain 84: thread 100097 (pid 21, usbus0) sleeping on 0xfffffe00d2d84498 "-" chain 85: thread 100098 (pid 21, usbus0) sleeping on 0xfffffe00d2d844f0 "-" chain 86: thread 100099 (pid 21, usbus0) sleeping on 0xfffffe00d2d84548 "-" chain 87: thread 100100 (pid 21, usbus0) sleeping on 0xfffffe00d2d845a0 "-" chain 88: thread 100102 (pid 21, usbus1) sleeping on 0xfffffe003e49dcf0 "-" chain 89: thread 100103 (pid 21, usbus1) sleeping on 0xfffffe003e49dd48 "-" chain 90: thread 100104 (pid 21, usbus1) sleeping on 0xfffffe003e49dda0 "-" chain 91: thread 100105 (pid 21, usbus1) sleeping on 0xfffffe003e49ddf8 "-" chain 92: thread 100106 (pid 21, usbus1) sleeping on 0xfffffe003e49de50 "-" chain 93: thread 100107 (pid 21, usbus2) sleeping on 0xfffffe00d2e27cf0 "-" chain 94: thread 100108 (pid 21, usbus2) sleeping on 0xfffffe00d2e27d48 "-" chain 95: thread 100109 (pid 21, usbus2) sleeping on 0xfffffe00d2e27da0 "-" chain 96: thread 100110 (pid 21, usbus2) sleeping on 0xfffffe00d2e27df8 "-" chain 97: thread 100111 (pid 21, usbus2) sleeping on 0xfffffe00d2e27e50 "-" chain 98: thread 100091 (pid 20, doneq0) sleeping on 0xffffffff819b83c0 "-" chain 99: thread 100093 (pid 20, doneq2) sleeping on 0xffffffff819b84c0 "-" chain 100: thread 100125 (pid 20, scanner) sleeping on 0xffffffff819b8288 "-" chain 101: thread 100090 (pid 19, crypto returns 11) sleeping on 0xfffff80003aafc50 "crypto_ret_wait" chain 102: thread 100089 (pid 18, crypto returns 10) sleeping on 0xfffff80003aafbf0 "crypto_ret_wait" chain 103: thread 100088 (pid 17, crypto returns 9) sleeping on 0xfffff80003aafb90 "crypto_ret_wait" chain 104: thread 100087 (pid 16, crypto returns 8) sleeping on 0xfffff80003aafb30 "crypto_ret_wait" chain 105: thread 100086 (pid 15, crypto returns 7) sleeping on 0xfffff80003aafad0 "crypto_ret_wait" chain 106: thread 100085 (pid 9, crypto returns 6) sleeping on 0xfffff80003aafa70 "crypto_ret_wait" chain 107: thread 100084 (pid 8, crypto returns 5) sleeping on 0xfffff80003aafa10 "crypto_ret_wait" chain 108: thread 100083 (pid 7, crypto returns 4) sleeping on 0xfffff80003aaf9b0 "crypto_ret_wait" chain 109: thread 100082 (pid 6, crypto returns 3) sleeping on 0xfffff80003aaf950 "crypto_ret_wait" chain 110: thread 100081 (pid 5, crypto returns 2) sleeping on 0xfffff80003aaf8f0 "crypto_ret_wait" chain 111: thread 100080 (pid 4, crypto returns 1) sleeping on 0xfffff80003aaf890 "crypto_ret_wait" chain 112: thread 100079 (pid 3, crypto returns 0) sleeping on 0xfffff80003aaf830 "crypto_ret_wait" chain 113: thread 100078 (pid 2, crypto) sleeping on 0xffffffff81d6db78 "crypto_wait" chain 114: thread 100065 (pid 14, sequencer 00) sleeping on 0xfffff80003a4b888 "seqstate" chain 115: thread 100062 (pid 13, g_event) sleeping on 0xffffffff81d9e0c8 "-" chain 116: thread 100063 (pid 13, g_up) sleeping on 0xffffffff81d9e0d8 "-" chain 117: thread 100064 (pid 13, g_down) sleeping on 0xffffffff81d9e0d0 "-" chain 118: thread 100002 (pid 1, init) sleeping on 0xfffff80003896000 "wait" chain 119: thread 100001 (pid 10, audit) sleeping on 0xffffffff81de5cf0 "audit_worker_cv" chain 120: thread 100000 (pid 0, swapper) sleeping on 0xffffffff81d8d688 "swapin" chain 121: thread 100015 (pid 0, aiod_kick taskq) sleeping on 0xfffff8000377f100 "-" chain 122: thread 100018 (pid 0, thread taskq) sleeping on 0xfffff8000377ea00 "-" chain 123: thread 100020 (pid 0, config_0) sleeping on 0xfffff80003777000 "-" chain 124: thread 100021 (pid 0, kqueue_ctx taskq) sleeping on 0xfffff8000377e500 "-" chain 125: thread 100022 (pid 0, if_io_tqg_0) sleeping on 0xfffff80003776e00 "-" chain 126: thread 100023 (pid 0, if_io_tqg_1) sleeping on 0xfffff80003776d00 "-" chain 127: thread 100024 (pid 0, if_io_tqg_2) sleeping on 0xfffff80003776c00 "-" chain 128: thread 100025 (pid 0, if_io_tqg_3) sleeping on 0xfffff80003776b00 "-" chain 129: thread 100026 (pid 0, if_io_tqg_4) sleeping on 0xfffff80003776a00 "-" chain 130: thread 100027 (pid 0, if_io_tqg_5) sleeping on 0xfffff80003776900 "-" chain 131: thread 100028 (pid 0, if_io_tqg_6) sleeping on 0xfffff80003776800 "-" chain 132: thread 100029 (pid 0, if_io_tqg_7) sleeping on 0xfffff80003776700 "-" chain 133: thread 100030 (pid 0, if_io_tqg_8) sleeping on 0xfffff80003776600 "-" chain 134: thread 100031 (pid 0, if_io_tqg_9) sleeping on 0xfffff80003776500 "-" chain 135: thread 100032 (pid 0, if_io_tqg_10) sleeping on 0xfffff80003776400 "-" chain 136: thread 100033 (pid 0, if_io_tqg_11) sleeping on 0xfffff80003776300 "-" chain 137: thread 100034 (pid 0, if_config_tqg_0) sleeping on 0xfffff800039e8300 "-" chain 138: thread 100035 (pid 0, softirq_0) sleeping on 0xfffff800039e8200 "-" chain 139: thread 100036 (pid 0, softirq_1) sleeping on 0xfffff800039e8100 "-" chain 140: thread 100037 (pid 0, softirq_2) sleeping on 0xfffff800039e8000 "-" chain 141: thread 100038 (pid 0, softirq_3) sleeping on 0xfffff800039e7e00 "-" chain 142: thread 100039 (pid 0, softirq_4) sleeping on 0xfffff800039e7d00 "-" chain 143: thread 100040 (pid 0, softirq_5) sleeping on 0xfffff800039e7c00 "-" chain 144: thread 100041 (pid 0, softirq_6) sleeping on 0xfffff800039e7b00 "-" chain 145: thread 100042 (pid 0, softirq_7) sleeping on 0xfffff800039e7a00 "-" chain 146: thread 100043 (pid 0, softirq_8) sleeping on 0xfffff800039e7900 "-" chain 147: thread 100044 (pid 0, softirq_9) sleeping on 0xfffff800039e7800 "-" chain 148: thread 100045 (pid 0, softirq_10) sleeping on 0xfffff800039e7700 "-" chain 149: thread 100046 (pid 0, softirq_11) sleeping on 0xfffff800039e7600 "-" chain 150: thread 100061 (pid 0, firmware taskq) sleeping on 0xfffff8000377c700 "-" chain 151: thread 100066 (pid 0, crypto_0) sleeping on 0xfffff8000377c100 "-" chain 152: thread 100067 (pid 0, crypto_1) sleeping on 0xfffff8000377c100 "-" chain 153: thread 100068 (pid 0, crypto_2) sleeping on 0xfffff8000377c100 "-" chain 154: thread 100069 (pid 0, crypto_3) sleeping on 0xfffff8000377c100 "-" chain 155: thread 100070 (pid 0, crypto_4) sleeping on 0xfffff8000377c100 "-" chain 156: thread 100071 (pid 0, crypto_5) sleeping on 0xfffff8000377c100 "-" chain 157: thread 100072 (pid 0, crypto_6) sleeping on 0xfffff8000377c100 "-" chain 158: thread 100073 (pid 0, crypto_7) sleeping on 0xfffff8000377c100 "-" chain 159: thread 100074 (pid 0, crypto_8) sleeping on 0xfffff8000377c100 "-" chain 160: thread 100075 (pid 0, crypto_9) sleeping on 0xfffff8000377c100 "-" chain 161: thread 100076 (pid 0, crypto_10) sleeping on 0xfffff8000377c100 "-" chain 162: thread 100077 (pid 0, crypto_11) sleeping on 0xfffff8000377c100 "-" chain 163: thread 100114 (pid 0, mca taskq) sleeping on 0xfffff800041f2000 "-" chain 164: thread 100120 (pid 0, acpi_task_0) sleeping on 0xfffff800200deb00 "-" chain 165: thread 100121 (pid 0, acpi_task_1) sleeping on 0xfffff800200deb00 "-" chain 166: thread 100122 (pid 0, acpi_task_2) sleeping on 0xfffff800200deb00 "-" chain 167: thread 100124 (pid 0, CAM taskq) sleeping on 0xfffff80003aaaa00 "-" chain 168: thread 100308 (pid 0, dtrace_taskq) sleeping on 0xfffff80695c7f500 "-" chain 169: thread 100673 (pid 0, system_taskq_0) sleeping on 0xfffff80418ef0000 "-" chain 170: thread 100674 (pid 0, system_taskq_1) sleeping on 0xfffff80418ef0000 "-" chain 171: thread 100680 (pid 0, system_taskq_2) sleeping on 0xfffff80418ef0000 "-" chain 172: thread 100692 (pid 0, system_taskq_3) sleeping on 0xfffff80418ef0000 "-" chain 173: thread 100694 (pid 0, system_taskq_4) sleeping on 0xfffff80418ef0000 "-" chain 174: thread 100695 (pid 0, system_taskq_5) sleeping on 0xfffff80418ef0000 "-" chain 175: thread 100699 (pid 0, system_taskq_6) sleeping on 0xfffff80418ef0000 "-" chain 176: thread 100700 (pid 0, system_taskq_7) sleeping on 0xfffff80418ef0000 "-" chain 177: thread 100703 (pid 0, system_taskq_8) sleeping on 0xfffff80418ef0000 "-" chain 178: thread 100717 (pid 0, system_taskq_9) sleeping on 0xfffff80418ef0000 "-" chain 179: thread 101351 (pid 0, system_taskq_10) sleeping on 0xfffff80418ef0000 "-" chain 180: thread 101356 (pid 0, system_taskq_11) sleeping on 0xfffff80418ef0000 "-" db:0:pho> chain 1: thread 104277 (pid 69161, swap) sleeping on 0xffffffff81c82710 "nanslp" chain 2: thread 103674 (pid 69158, swap) sleeping on 0xffffffff81c82713 "nanslp" chain 3: thread 103523 (pid 69157, swap) sleeping on 0xffffffff81c82719 "nanslp" chain 4: thread 104146 (pid 69156, swap) sleeping on 0xffffffff81c82712 "nanslp" chain 5: thread 104163 (pid 69153, swap) sleeping on 0xffffffff81c82717 "nanslp" chain 6: thread 104225 (pid 69152, swap) sleeping on 0xffffffff81c82714 "nanslp" chain 7: thread 104222 (pid 69151, swap) sleeping on 0xffffffff81c82711 "nanslp" chain 8: thread 104257 (pid 69150, swap) sleeping on 0xffffffff81c82715 "nanslp" chain 9: thread 104374 (pid 69149, swap) sleeping on 0xffffffff81c82718 "nanslp" chain 10: thread 104311 (pid 69148, swap) sleeping on 0xffffffff81c82714 "nanslp" chain 11: thread 104333 (pid 69146, swap) sleeping on 0xffffffff81c82714 "nanslp" chain 12: thread 103986 (pid 69142, swap) sleeping on 0xfffff8075c306a60 "wait" chain 13: thread 104181 (pid 69140, swap) sleeping on 0xffffffff81c8271a "nanslp" chain 14: thread 104313 (pid 69086, sh) sleeping on 0xfffff8075c6e7a60 "wait" chain 15: thread 104025 (pid 69085, tee) sleeping on 0xfffff802dd9d45f0 "piperd" chain 16: thread 104312 (pid 69084, sh) sleeping on 0xfffff8075c51e530 "wait" chain 17: thread 103776 (pid 68971, sleep) sleeping on 0xffffffff81c82716 "nanslp" chain 18: thread 100232 (pid 11521, ftcleanup) sleeping on 0xffffffff82e1d4f4 "ftcl" chain 19: thread 100231 (pid 4405, sh) sleeping on 0xfffff80020728be0 "piperd" chain 20: thread 100146 (pid 4403, awk) sleeping on 0xfffff80020a26000 "piperd" chain 21: thread 100238 (pid 4402, sh) sleeping on 0xfffff80110307a60 "wait" chain 22: thread 100156 (pid 4348, sh) sleeping on 0xfffff800205fe530 "wait" chain 23: thread 100147 (pid 2440, sh) sleeping on 0xfffff800209e2000 "wait" chain 24: thread 100188 (pid 2429, vi) sleeping on 0xfffff80003af64b0 "ttyin" chain 25: thread 100218 (pid 2424, bash) sleeping on 0xfffff800206be530 "wait" chain 26: thread 100177 (pid 2423, sshd) sleeping on 0xfffff800041be040 "select" chain 27: thread 100206 (pid 2420, sshd) sleeping on 0xfffff80003e95740 "select" chain 28: thread 100189 (pid 2386, bash) sleeping on 0xfffff80003e959c0 "select" chain 29: thread 100207 (pid 2384, csh) sleeping on 0xfffff800209a80a8 "pause" chain 30: thread 100174 (pid 2383, su) sleeping on 0xfffff800207a9530 "wait" chain 31: thread 100175 (pid 2378, bash) sleeping on 0xfffff800206d6000 "wait" chain 32: thread 100208 (pid 2377, sshd) sleeping on 0xfffff800041689c0 "select" chain 33: thread 100209 (pid 2374, sshd) sleeping on 0xfffff80003edd3c0 "select" chain 34: thread 100173 (pid 2370, csh) sleeping on 0xfffff80017038cb0 "ttyin" chain 35: thread 100214 (pid 2367, login) sleeping on 0xfffff80296fed000 "wait" chain 36: thread 100153 (pid 2366, getty) sleeping on 0xfffff800170370b0 "ttyin" chain 37: thread 100179 (pid 2365, getty) sleeping on 0xfffff8002018f8b0 "ttyin" chain 38: thread 100213 (pid 2364, getty) sleeping on 0xfffff8002018fcb0 "ttyin" chain 39: thread 100212 (pid 2363, getty) sleeping on 0xfffff800201a00b0 "ttyin" chain 40: thread 100211 (pid 2362, getty) sleeping on 0xfffff800201a04b0 "ttyin" chain 41: thread 100210 (pid 2361, getty) sleeping on 0xfffff800201a08b0 "ttyin" chain 42: thread 100171 (pid 2360, getty) sleeping on 0xfffff800201a0cb0 "ttyin" chain 43: thread 100168 (pid 2359, getty) sleeping on 0xfffff8002019f0b0 "ttyin" chain 44: thread 100142 (pid 2358, getty) sleeping on 0xfffff8002019f4b0 "ttyin" chain 45: thread 100144 (pid 2312, cron) sleeping on 0xffffffff81c82713 "nanslp" chain 46: thread 100165 (pid 2308, sshd) sleeping on 0xfffff800203a5e40 "select" chain 47: thread 100160 (pid 2280, powerd) sleeping on 0xfffff800039f45c0 "select" chain 48: thread 100166 (pid 2277, ntpd) sleeping on 0xfffff80003e95a40 "select" chain 49: thread 100621 (pid 2277, ntpd) blocked on lockmgr (null)EXCL chain 50: thread 100197 (pid 2214, NLM: master) sleeping on 0xfffff80020b31d60 "rpcsvc" chain 51: thread 100195 (pid 2211, rpc.statd) sleeping on 0xfffff80003e95ac0 "select" chain 52: thread 100192 (pid 2157, watchdogd) sleeping on 0xffffffff81c82710 "nanslp" chain 53: thread 100194 (pid 2078, rpcbind) sleeping on 0xfffff80003e95bc0 "select" chain 54: thread 100164 (pid 2068, syslogd) sleeping on 0xfffff80003edd540 "select" chain 55: thread 100167 (pid 1997, devd) sleeping on 0xfffff80003edd5c0 "select" chain 56: thread 100161 (pid 1992, dhclient) sleeping on 0xfffff800201d30c0 "select" chain 57: thread 100154 (pid 1928, dhclient) sleeping on 0xfffff800201d3740 "select" chain 58: thread 100157 (pid 1925, dhclient) sleeping on 0xfffff800041be2c0 "select" chain 59: thread 100141 (pid 34, vnlru) sleeping on 0xfffff80020056a60 "vlruwt" chain 60: thread 100136 (pid 33, syncer) sleeping on 0xffffffff81d58b98 "syncer" chain 61: thread 100130 (pid 32, bufdaemon) sleeping on 0xffffffff81d58098 "qsleep" chain 62: thread 100131 (pid 32, bufspacedaemon-0) sleeping on 0xffffffff8180a980 "-" chain 63: thread 100134 (pid 32, bufspacedaemon-1) sleeping on 0xffffffff81812c40 "-" chain 64: thread 100135 (pid 32, bufspacedaemon-2) sleeping on 0xffffffff8181af00 "-" chain 65: thread 100137 (pid 32, bufspacedaemon-3) sleeping on 0xffffffff818231c0 "-" chain 66: thread 100138 (pid 32, bufspacedaemon-4) sleeping on 0xffffffff8182b480 "-" chain 67: thread 100139 (pid 32, bufspacedaemon-5) sleeping on 0xffffffff81833740 "-" chain 68: thread 100140 (pid 32, bufspacedaemon-6) sleeping on 0xffffffff8183ba00 "-" chain 69: thread 100152 (pid 32, / worker) sleeping on 0xfffff80003a728e8 "sdflush" chain 70: thread 100202 (pid 32, /tmp worker) sleeping on 0xfffff8001708fce8 "sdflush" chain 71: thread 100129 (pid 31, vmdaemon) sleeping on 0xffffffff81d73b08 "psleep" chain 72: thread 100128 (pid 30, dom0) sleeping on 0xffffffff81d9fa98 "psleep" chain 73: thread 100132 (pid 30, laundry: dom0) sleeping on 0xffffffff81d9faa4 "launds" chain 74: thread 100133 (pid 30, uma) sleeping on 0xffffffff80f0dc20 "umarcl" chain 75: thread 100127 (pid 29, enc_daemon1) sleeping on 0xfffff8002024b000 "idle" chain 76: thread 100126 (pid 28, enc_daemon0) sleeping on 0xfffff8002024b530 "idle" chain 77: thread 100123 (pid 27, rand_harvestq) sleeping on 0xffffffff81adcfb0 "-" chain 78: thread 100119 (pid 26, sctp_iterator) sleeping on 0xffffffff81de5060 "waiting_for_work" chain 79: thread 100118 (pid 25, soaiod4) sleeping on 0xffffffff81d57a9c "-" chain 80: thread 100117 (pid 24, soaiod3) sleeping on 0xffffffff81d57a9c "-" chain 81: thread 100116 (pid 23, soaiod2) sleeping on 0xffffffff81d57a9c "-" chain 82: thread 100115 (pid 22, soaiod1) sleeping on 0xffffffff81d57a9c "-" chain 83: thread 100096 (pid 21, usbus0) sleeping on 0xfffffe00d2d84440 "-" chain 84: thread 100097 (pid 21, usbus0) sleeping on 0xfffffe00d2d84498 "-" chain 85: thread 100098 (pid 21, usbus0) sleeping on 0xfffffe00d2d844f0 "-" chain 86: thread 100099 (pid 21, usbus0) sleeping on 0xfffffe00d2d84548 "-" chain 87: thread 100100 (pid 21, usbus0) sleeping on 0xfffffe00d2d845a0 "-" chain 88: thread 100102 (pid 21, usbus1) sleeping on 0xfffffe003e49dcf0 "-" chain 89: thread 100103 (pid 21, usbus1) sleeping on 0xfffffe003e49dd48 "-" chain 90: thread 100104 (pid 21, usbus1) sleeping on 0xfffffe003e49dda0 "-" chain 91: thread 100105 (pid 21, usbus1) sleeping on 0xfffffe003e49ddf8 "-" chain 92: thread 100106 (pid 21, usbus1) sleeping on 0xfffffe003e49de50 "-" chain 93: thread 100107 (pid 21, usbus2) sleeping on 0xfffffe00d2e27cf0 "-" chain 94: thread 100108 (pid 21, usbus2) sleeping on 0xfffffe00d2e27d48 "-" chain 95: thread 100109 (pid 21, usbus2) sleeping on 0xfffffe00d2e27da0 "-" chain 96: thread 100110 (pid 21, usbus2) sleeping on 0xfffffe00d2e27df8 "-" chain 97: thread 100111 (pid 21, usbus2) sleeping on 0xfffffe00d2e27e50 "-" chain 98: thread 100091 (pid 20, doneq0) sleeping on 0xffffffff819b83c0 "-" chain 99: thread 100093 (pid 20, doneq2) sleeping on 0xffffffff819b84c0 "-" chain 100: thread 100125 (pid 20, scanner) sleeping on 0xffffffff819b8288 "-" chain 101: thread 100090 (pid 19, crypto returns 11) sleeping on 0xfffff80003aafc50 "crypto_ret_wait" chain 102: thread 100089 (pid 18, crypto returns 10) sleeping on 0xfffff80003aafbf0 "crypto_ret_wait" chain 103: thread 100088 (pid 17, crypto returns 9) sleeping on 0xfffff80003aafb90 "crypto_ret_wait" chain 104: thread 100087 (pid 16, crypto returns 8) sleeping on 0xfffff80003aafb30 "crypto_ret_wait" chain 105: thread 100086 (pid 15, crypto returns 7) sleeping on 0xfffff80003aafad0 "crypto_ret_wait" chain 106: thread 100085 (pid 9, crypto returns 6) sleeping on 0xfffff80003aafa70 "crypto_ret_wait" chain 107: thread 100084 (pid 8, crypto returns 5) sleeping on 0xfffff80003aafa10 "crypto_ret_wait" chain 108: thread 100083 (pid 7, crypto returns 4) sleeping on 0xfffff80003aaf9b0 "crypto_ret_wait" chain 109: thread 100082 (pid 6, crypto returns 3) sleeping on 0xfffff80003aaf950 "crypto_ret_wait" chain 110: thread 100081 (pid 5, crypto returns 2) sleeping on 0xfffff80003aaf8f0 "crypto_ret_wait" chain 111: thread 100080 (pid 4, crypto returns 1) sleeping on 0xfffff80003aaf890 "crypto_ret_wait" chain 112: thread 100079 (pid 3, crypto returns 0) sleeping on 0xfffff80003aaf830 "crypto_ret_wait" chain 113: thread 100078 (pid 2, crypto) sleeping on 0xffffffff81d6db78 "crypto_wait" chain 114: thread 100065 (pid 14, sequencer 00) sleeping on 0xfffff80003a4b888 "seqstate" chain 115: thread 100062 (pid 13, g_event) sleeping on 0xffffffff81d9e0c8 "-" chain 116: thread 100063 (pid 13, g_up) sleeping on 0xffffffff81d9e0d8 "-" chain 117: thread 100064 (pid 13, g_down) sleeping on 0xffffffff81d9e0d0 "-" chain 118: thread 100002 (pid 1, init) sleeping on 0xfffff80003896000 "wait" chain 119: thread 100001 (pid 10, audit) sleeping on 0xffffffff81de5cf0 "audit_worker_cv" chain 120: thread 100000 (pid 0, swapper) sleeping on 0xffffffff81d8d688 "swapin" chain 121: thread 100015 (pid 0, aiod_kick taskq) sleeping on 0xfffff8000377f100 "-" chain 122: thread 100018 (pid 0, thread taskq) sleeping on 0xfffff8000377ea00 "-" chain 123: thread 100020 (pid 0, config_0) sleeping on 0xfffff80003777000 "-" chain 124: thread 100021 (pid 0, kqueue_ctx taskq) sleeping on 0xfffff8000377e500 "-" chain 125: thread 100022 (pid 0, if_io_tqg_0) sleeping on 0xfffff80003776e00 "-" chain 126: thread 100023 (pid 0, if_io_tqg_1) sleeping on 0xfffff80003776d00 "-" chain 127: thread 100024 (pid 0, if_io_tqg_2) sleeping on 0xfffff80003776c00 "-" chain 128: thread 100025 (pid 0, if_io_tqg_3) sleeping on 0xfffff80003776b00 "-" chain 129: thread 100026 (pid 0, if_io_tqg_4) sleeping on 0xfffff80003776a00 "-" chain 130: thread 100027 (pid 0, if_io_tqg_5) sleeping on 0xfffff80003776900 "-" chain 131: thread 100028 (pid 0, if_io_tqg_6) sleeping on 0xfffff80003776800 "-" chain 132: thread 100029 (pid 0, if_io_tqg_7) sleeping on 0xfffff80003776700 "-" chain 133: thread 100030 (pid 0, if_io_tqg_8) sleeping on 0xfffff80003776600 "-" chain 134: thread 100031 (pid 0, if_io_tqg_9) sleeping on 0xfffff80003776500 "-" chain 135: thread 100032 (pid 0, if_io_tqg_10) sleeping on 0xfffff80003776400 "-" chain 136: thread 100033 (pid 0, if_io_tqg_11) sleeping on 0xfffff80003776300 "-" chain 137: thread 100034 (pid 0, if_config_tqg_0) sleeping on 0xfffff800039e8300 "-" chain 138: thread 100035 (pid 0, softirq_0) sleeping on 0xfffff800039e8200 "-" chain 139: thread 100036 (pid 0, softirq_1) sleeping on 0xfffff800039e8100 "-" chain 140: thread 100037 (pid 0, softirq_2) sleeping on 0xfffff800039e8000 "-" chain 141: thread 100038 (pid 0, softirq_3) sleeping on 0xfffff800039e7e00 "-" chain 142: thread 100039 (pid 0, softirq_4) sleeping on 0xfffff800039e7d00 "-" chain 143: thread 100040 (pid 0, softirq_5) sleeping on 0xfffff800039e7c00 "-" chain 144: thread 100041 (pid 0, softirq_6) sleeping on 0xfffff800039e7b00 "-" chain 145: thread 100042 (pid 0, softirq_7) sleeping on 0xfffff800039e7a00 "-" chain 146: thread 100043 (pid 0, softirq_8) sleeping on 0xfffff800039e7900 "-" chain 147: thread 100044 (pid 0, softirq_9) sleeping on 0xfffff800039e7800 "-" chain 148: thread 100045 (pid 0, softirq_10) sleeping on 0xfffff800039e7700 "-" chain 149: thread 100046 (pid 0, softirq_11) sleeping on 0xfffff800039e7600 "-" chain 150: thread 100061 (pid 0, firmware taskq) sleeping on 0xfffff8000377c700 "-" chain 151: thread 100066 (pid 0, crypto_0) sleeping on 0xfffff8000377c100 "-" chain 152: thread 100067 (pid 0, crypto_1) sleeping on 0xfffff8000377c100 "-" chain 153: thread 100068 (pid 0, crypto_2) sleeping on 0xfffff8000377c100 "-" chain 154: thread 100069 (pid 0, crypto_3) sleeping on 0xfffff8000377c100 "-" chain 155: thread 100070 (pid 0, crypto_4) sleeping on 0xfffff8000377c100 "-" chain 156: thread 100071 (pid 0, crypto_5) sleeping on 0xfffff8000377c100 "-" chain 157: thread 100072 (pid 0, crypto_6) sleeping on 0xfffff8000377c100 "-" chain 158: thread 100073 (pid 0, crypto_7) sleeping on 0xfffff8000377c100 "-" chain 159: thread 100074 (pid 0, crypto_8) sleeping on 0xfffff8000377c100 "-" chain 160: thread 100075 (pid 0, crypto_9) sleeping on 0xfffff8000377c100 "-" chain 161: thread 100076 (pid 0, crypto_10) sleeping on 0xfffff8000377c100 "-" chain 162: thread 100077 (pid 0, crypto_11) sleeping on 0xfffff8000377c100 "-" chain 163: thread 100114 (pid 0, mca taskq) sleeping on 0xfffff800041f2000 "-" chain 164: thread 100120 (pid 0, acpi_task_0) sleeping on 0xfffff800200deb00 "-" chain 165: thread 100121 (pid 0, acpi_task_1) sleeping on 0xfffff800200deb00 "-" chain 166: thread 100122 (pid 0, acpi_task_2) sleeping on 0xfffff800200deb00 "-" chain 167: thread 100124 (pid 0, CAM taskq) sleeping on 0xfffff80003aaaa00 "-" chain 168: thread 100308 (pid 0, dtrace_taskq) sleeping on 0xfffff80695c7f500 "-" chain 169: thread 100673 (pid 0, system_taskq_0) sleeping on 0xfffff80418ef0000 "-" chain 170: thread 100674 (pid 0, system_taskq_1) sleeping on 0xfffff80418ef0000 "-" chain 171: thread 100680 (pid 0, system_taskq_2) sleeping on 0xfffff80418ef0000 "-" chain 172: thread 100692 (pid 0, system_taskq_3) sleeping on 0xfffff80418ef0000 "-" chain 173: thread 100694 (pid 0, system_taskq_4) sleeping on 0xfffff80418ef0000 "-" chain 174: thread 100695 (pid 0, system_taskq_5) sleeping on 0xfffff80418ef0000 "-" chain 175: thread 100699 (pid 0, system_taskq_6) sleeping on 0xfffff80418ef0000 "-" chain 176: thread 100700 (pid 0, system_taskq_7) sleeping on 0xfffff80418ef0000 "-" chain 177: thread 100703 (pid 0, system_taskq_8) sleeping on 0xfffff80418ef0000 "-" chain 178: thread 100717 (pid 0, system_taskq_9) sleeping on 0xfffff80418ef0000 "-" chain 179: thread 101351 (pid 0, system_taskq_10) sleeping on 0xfffff80418ef0000 "-" chain 180: thread 101356 (pid 0, system_taskq_11) sleeping on 0xfffff80418ef0000 "-" db> (kgdb) bt #0 doadump (textdump=0x0) at include/pcpu_aux.h:55 #1 0xffffffff8049bb3a in db_dump (dummy=, dummy2=, dummy3=, dummy4=) at ../../../ddb/db_command.c:575 #2 0xffffffff8049b8fc in db_command (cmd_table=, dopager=0x0) at ../../../ddb/db_command.c:482 #3 0xffffffff804a08c8 in db_script_exec (scriptname=, warnifnotfound=) at ../../../ddb/db_script.c:304 #4 0xffffffff8049b8fc in db_command (cmd_table=, dopager=0x0) at ../../../ddb/db_command.c:482 #5 0xffffffff804a08c8 in db_script_exec (scriptname=, warnifnotfound=) at ../../../ddb/db_script.c:304 #6 0xffffffff8049b8fc in db_command (cmd_table=, dopager=0x1) at ../../../ddb/db_command.c:482 #7 0xffffffff8049b66d in db_command_loop () at ../../../ddb/db_command.c:535 #8 0xffffffff8049e898 in db_trap (type=, code=) at ../../../ddb/db_main.c:252 #9 0xffffffff80c14df7 in kdb_trap (type=0x3, code=0x0, tf=) at ../../../kern/subr_kdb.c:689 #10 0xffffffff81073aee in trap (frame=0xfffffe0337fbbf90) at ../../../amd64/amd64/trap.c:579 #11 0xffffffff81049b2c in calltrap () at ../../../amd64/amd64/exception.S:289 #12 0xffffffff80c14557 in kdb_enter (why=0xffffffff81203cba "panic", msg=) at include/cpufunc.h:65 #13 0xffffffff80bca111 in vpanic (fmt=, ap=) at ../../../kern/kern_shutdown.c:899 #14 0xffffffff80bc9ea3 in panic (fmt=) at ../../../kern/kern_shutdown.c:836 #15 0xffffffff80f15218 in vm_fault (map=, vaddr=, fault_type=, fault_flags=, m_hold=) at ../../../vm/vm_fault.c:769 #16 0xffffffff80f1362e in vm_fault_trap (map=0xfffff80003002000, vaddr=, fault_type=, fault_flags=0x0, signo=0x0, ucode=) at ../../../vm/vm_fault.c:589 #17 0xffffffff81074143 in trap_pfault (frame=, usermode=, signo=, ucode=0x0) at ../../../amd64/amd64/trap.c:822 #18 0xffffffff810735e7 in trap (frame=0xfffffe0337fbc450) at ../../../amd64/amd64/trap.c:401 #19 0xffffffff81049b2c in calltrap () at ../../../amd64/amd64/exception.S:289 #20 0xffffffff80c27d2d in smr_create (name=0xffffffff82e37075 "smrs") at ../../../kern/subr_smr.c:423 #21 0xffffffff80f0904a in zone_ctor (mem=0xfffffe00e67e5880, size=, udata=0xfffffe0337fbc660, flags=0x2) at ../../../vm/uma_core.c:2530 #22 0xffffffff80f0b129 in item_ctor (zone=0xfffff8007982a000, uz_flags=, size=0x880, udata=, flags=, item=0xfffffe00e67e5880) at ../../../vm/uma_core.c:2998 #23 0xffffffff80f09d24 in zone_alloc_item (zone=0xfffff8007982a000, udata=0xfffffe0337fbc660, domain=, flags=0x2) at ../../../vm/uma_core.c:3833 #24 0xffffffff80f099a3 in uma_zcreate (name=, size=, ctor=, dtor=, uminit=, fini=, align=0x7, flags=0x40002) at ../../../vm/uma_core.c:2860 #25 0xffffffff82e36044 in ?? () #26 0x0000000000000007 in ?? () #27 0x0000000000040002 in ?? () #28 0xfffff802494d8980 in ?? () #29 0xffffffff82e38000 in ?? () #30 0xfffffe0337fbc720 in ?? () #31 0xffffffff80ba69ed in module_register_init (arg=0xffffffff82e38000) at ../../../kern/kern_module.c:123 Previous frame identical to this frame (corrupt stack?) Current language: auto; currently minimal (kgdb) f 20 #20 0xffffffff80c27d2d in smr_create (name=0xffffffff82e37075 "smrs") at ../../../kern/subr_smr.c:423 423 c = zpcpu_get_cpu(smr, i); (kgdb) l 418 s->s_name = name; 419 s->s_rd_seq = s->s_wr_seq = SMR_SEQ_INIT; 420 421 /* Initialize all CPUS, not just those running. */ 422 for (i = 0; i <= mp_maxid; i++) { 423 c = zpcpu_get_cpu(smr, i); 424 c->c_seq = SMR_SEQ_INVALID; 425 c->c_shared = s; 426 } 427 atomic_thread_fence_seq_cst(); (kgdb) info loc s = 0xfffff8075c23ae80 i = 0x0 smr = (kgdb) Index: src/sys/vm/swap_pager.c =================================================================== --- src/sys/vm/swap_pager.c (revision 357853) +++ src/sys/vm/swap_pager.c (working copy) @@ -1188,8 +1188,8 @@ swap_pager_unswapped(vm_page_t m) * The pages in "ma" must be busied and will remain busied upon return. */ static int -swap_pager_getpages(vm_object_t object, vm_page_t *ma, int count, int *rbehind, - int *rahead) +swap_pager_getpages_locked(vm_object_t object, vm_page_t *ma, int count, + int *rbehind, int *rahead) { struct buf *bp; vm_page_t bm, mpred, msucc, p; @@ -1197,7 +1197,7 @@ static int daddr_t blk; int i, maxahead, maxbehind, reqcount; - VM_OBJECT_WLOCK(object); + VM_OBJECT_ASSERT_WLOCKED(object); reqcount = count; KASSERT(object->type == OBJT_SWAP, @@ -1352,6 +1352,15 @@ static int */ } +static int +swap_pager_getpages(vm_object_t object, vm_page_t *ma, int count, + int *rbehind, int *rahead) +{ + + VM_OBJECT_WLOCK(object); + return (swap_pager_getpages_locked(object, ma, count, rbehind, rahead)); +} + /* * swap_pager_getpages_async(): * @@ -1444,18 +1453,6 @@ swap_pager_putpages(vm_object_t object, vm_page_t /* Maximum I/O size is limited by maximum swap block size. */ n = min(count - i, nsw_cluster_max); - /* Get a block of swap of size up to size n. */ - blk = swp_pager_getswapspace(&n, 4); - if (blk == SWAPBLK_NONE) { - for (j = 0; j < n; ++j) - rtvals[i + j] = VM_PAGER_FAIL; - continue; - } - - /* - * All I/O parameters have been satisfied. Build the I/O - * request and assign the swap space. - */ if (async) { mtx_lock(&swbuf_mtx); while (nsw_wcount_async == 0) @@ -1464,6 +1461,7 @@ swap_pager_putpages(vm_object_t object, vm_page_t nsw_wcount_async--; mtx_unlock(&swbuf_mtx); } + bp = uma_zalloc(swwbuf_zone, M_WAITOK); if (async) bp->b_flags = B_ASYNC; @@ -1470,13 +1468,19 @@ swap_pager_putpages(vm_object_t object, vm_page_t bp->b_flags |= B_PAGING; bp->b_iocmd = BIO_WRITE; - bp->b_rcred = crhold(thread0.td_ucred); - bp->b_wcred = crhold(thread0.td_ucred); - bp->b_bcount = PAGE_SIZE * n; - bp->b_bufsize = PAGE_SIZE * n; - bp->b_blkno = blk; - + /* Get a block of swap of size up to size n. */ VM_OBJECT_WLOCK(object); + blk = swp_pager_getswapspace(&n, 4); + if (blk == SWAPBLK_NONE) { + VM_OBJECT_WUNLOCK(object); + for (j = 0; j < n; ++j) + rtvals[i + j] = VM_PAGER_FAIL; + mtx_lock(&swbuf_mtx); + nsw_wcount_async++; + mtx_unlock(&swbuf_mtx); + uma_zfree(swwbuf_zone, bp); + continue; + } for (j = 0; j < n; ++j) { mreq = ma[i + j]; vm_page_aflag_clear(mreq, PGA_SWAP_FREE); @@ -1490,7 +1494,14 @@ swap_pager_putpages(vm_object_t object, vm_page_t bp->b_pages[j] = mreq; } VM_OBJECT_WUNLOCK(object); + + bp->b_rcred = crhold(thread0.td_ucred); + bp->b_wcred = crhold(thread0.td_ucred); + bp->b_bcount = PAGE_SIZE * n; + bp->b_bufsize = PAGE_SIZE * n; + bp->b_blkno = blk; bp->b_npages = n; + /* * Must set dirty range for NFS to work. */ @@ -1712,72 +1723,12 @@ swp_pager_force_dirty(vm_page_t m) { vm_page_dirty(m); -#ifdef INVARIANTS - if (!vm_page_wired(m) && m->a.queue == PQ_NONE) - panic("page %p is neither wired nor queued", m); -#endif vm_page_xunbusy(m); swap_pager_unswapped(m); -} - -static void -swp_pager_force_launder(vm_page_t m) -{ - - vm_page_dirty(m); vm_page_launder(m); - vm_page_xunbusy(m); - swap_pager_unswapped(m); } /* - * SWP_PAGER_FORCE_PAGEIN() - force swap blocks to be paged in - * - * This routine dissociates pages starting at the given index within an - * object from their backing store, paging them in if they do not reside - * in memory. Pages that are paged in are marked dirty and placed in the - * laundry queue. Pages are marked dirty because they no longer have - * backing store. They are placed in the laundry queue because they have - * not been accessed recently. Otherwise, they would already reside in - * memory. - */ -static void -swp_pager_force_pagein(vm_object_t object, vm_pindex_t pindex, int npages) -{ - vm_page_t ma[npages]; - int i, j; - - KASSERT(npages > 0, ("%s: No pages", __func__)); - KASSERT(npages <= MAXPHYS / PAGE_SIZE, - ("%s: Too many pages: %d", __func__, npages)); - KASSERT(object->type == OBJT_SWAP, - ("%s: Object not swappable", __func__)); - vm_object_pip_add(object, npages); - vm_page_grab_pages(object, pindex, VM_ALLOC_NORMAL, ma, npages); - for (i = j = 0;; i++) { - /* Count nonresident pages, to page-in all at once. */ - if (i < npages && ma[i]->valid != VM_PAGE_BITS_ALL) - continue; - if (j < i) { - VM_OBJECT_WUNLOCK(object); - /* Page-in nonresident pages. Mark for laundering. */ - if (swap_pager_getpages(object, &ma[j], i - j, NULL, - NULL) != VM_PAGER_OK) - panic("%s: read from swap failed", __func__); - VM_OBJECT_WLOCK(object); - do { - swp_pager_force_launder(ma[j]); - } while (++j < i); - } - if (i == npages) - break; - /* Mark dirty a resident page. */ - swp_pager_force_dirty(ma[j++]); - } - vm_object_pip_wakeupn(object, npages); -} - -/* * swap_pager_swapoff_object: * * Page in all of the pages that have been paged out for an object @@ -1787,62 +1738,80 @@ static void swap_pager_swapoff_object(struct swdevt *sp, vm_object_t object) { struct swblk *sb; - vm_pindex_t pi, s_pindex; - daddr_t blk, n_blks, s_blk; - int i; + vm_page_t m, ma[SWAP_META_PAGES]; + vm_pindex_t pi; + daddr_t blk; + int i, j, npages, nv, rahead, rv; KASSERT(object->type == OBJT_SWAP, ("%s: Object not swappable", __func__)); - n_blks = 0; + for (pi = 0; (sb = SWAP_PCTRIE_LOOKUP_GE( &object->un_pager.swp.swp_blks, pi)) != NULL; ) { + if ((object->flags & OBJ_DEAD) != 0) { + swp_pager_meta_free_all(object); + break; + } for (i = 0; i < SWAP_META_PAGES; i++) { - blk = sb->d[i]; - if (!swp_pager_isondev(blk, sp)) - blk = SWAPBLK_NONE; - /* - * If there are no blocks/pages accumulated, start a new - * accumulation here. + * Count the number of contiguous valid blocks. */ - if (n_blks == 0) { - if (blk != SWAPBLK_NONE) { - s_blk = blk; - s_pindex = sb->p + i; - n_blks = 1; - } + for (nv = 0; nv < SWAP_META_PAGES - i; nv++) { + blk = sb->d[i + nv]; + if (!swp_pager_isondev(blk, sp) || + blk == SWAPBLK_NONE) + break; + } + if (nv == 0) continue; - } /* - * If the accumulation can be extended without breaking - * the sequence of consecutive blocks and pages that - * swp_pager_force_pagein() depends on, do so. + * Grab the page for the first valid block. If the page + * is valid, mark it dirty and free its swap block. Try + * to batch this operation since it may cause sp to be + * freed, meaning that we must restart the scan. + * Invalid pages will be revisited in future iterations. */ - if (n_blks < MAXPHYS / PAGE_SIZE && - s_blk + n_blks == blk && - s_pindex + n_blks == sb->p + i) { - ++n_blks; - continue; + m = vm_page_grab(object, sb->p + i, VM_ALLOC_NORMAL | + VM_ALLOC_WAITFAIL); + if (m == NULL) + break; + if (vm_page_all_valid(m)) { + npages = vm_page_grab_pages(object, + m->pindex + 1, + VM_ALLOC_NOCREAT | VM_ALLOC_NOWAIT, ma, + nv - 1); + swp_pager_force_dirty(m); + for (j = 0; j < npages; j++) { + if (vm_page_all_valid(ma[j])) + swp_pager_force_dirty(ma[j]); + else + vm_page_xunbusy(ma[j]); + } + break; } + vm_object_pip_add(object, 1); + rahead = SWAP_META_PAGES; + rv = swap_pager_getpages_locked(object, &m, 1, NULL, + &rahead); + if (rv != VM_PAGER_OK) + panic("%s: read from swap failed: %d", + __func__, rv); + vm_object_pip_wakeupn(object, 1); + VM_OBJECT_WLOCK(object); + vm_page_xunbusy(m); + /* - * The sequence of consecutive blocks and pages cannot - * be extended, so page them all in here. Then, - * because doing so involves releasing and reacquiring - * a lock that protects the swap block pctrie, do not - * rely on the current swap block. Break this loop and - * re-fetch the same pindex from the pctrie again. + * The object lock was dropped so we must restart the + * scan of this swap block. Pages paged in during this + * iteration will be marked dirty in a future iteration. */ - swp_pager_force_pagein(object, s_pindex, n_blks); - n_blks = 0; break; } if (i == SWAP_META_PAGES) pi = sb->p + SWAP_META_PAGES; } - if (n_blks > 0) - swp_pager_force_pagein(object, s_pindex, n_blks); } /* @@ -2078,7 +2047,7 @@ allocated: * Free the swblk if we end up with the empty page run. */ if (swapblk == SWAPBLK_NONE) - swp_pager_free_empty_swblk(object, sb); + swp_pager_free_empty_swblk(object, sb); return (prev_swapblk); } @@ -2350,7 +2319,7 @@ swaponsomething(struct vnode *vp, void *id, u_long sw_strategy_t *strategy, sw_close_t *close, dev_t dev, int flags) { struct swdevt *sp, *tsp; - swblk_t dvbase; + daddr_t dvbase; u_long mblocks; /* Index: src/sys/vm/swap_pager.h =================================================================== --- src/sys/vm/swap_pager.h (revision 357853) +++ src/sys/vm/swap_pager.h (working copy) @@ -38,14 +38,9 @@ */ #ifndef _VM_SWAP_PAGER_H_ -#define _VM_SWAP_PAGER_H_ 1 +#define _VM_SWAP_PAGER_H_ -typedef int32_t swblk_t; /* - * swap offset. This is the type used to - * address the "virtual swap device" and - * therefore the maximum swap space is - * 2^32 pages. - */ +#include struct buf; struct swdevt; @@ -62,8 +57,8 @@ struct swdevt { dev_t sw_dev; struct vnode *sw_vp; void *sw_id; - swblk_t sw_first; - swblk_t sw_end; + __daddr_t sw_first; + __daddr_t sw_end; struct blist *sw_blist; TAILQ_ENTRY(swdevt) sw_list; sw_strategy_t *sw_strategy; -- Test scenario: misc/smrstress.sh