panic: ASan: Invalid access 4-byte read at 0xfffffe01293cb1e0, RedZonePartial, vmcore.264 KDB: debugger backends: ddb KDB: current backend: ddb ---<>--- Copyright (c) 1992-2020 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 13.0-CURRENT #0 r368820M: Tue Dec 22 19:33:25 CET 2020 pho@mercat1.netperf.freebsd.org:/usr/src/sys/amd64/compile/PHO-KASAN amd64 FreeBSD clang version 11.0.0 (git@github.com:llvm/llvm-project.git llvmorg-11.0.0-0-g176249bd673) WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. VT(vga): resolution 640x480 CPU: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz (3500.07-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306f2 Family=0x6 Model=0x3f Stepping=2 Features=0xbfebfbff Features2=0x7ffefbff AMD Features=0x2c100800 AMD Features2=0x21 Structured Extended Features=0x37ab XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics real memory = 34359738368 (32768 MB) avail memory = 32548745216 (31040 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs FreeBSD/SMP: 1 package(s) x 6 core(s) x 2 hardware threads random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" random: unblocking device. ioapic0 irqs 0-23 ioapic1 irqs 24-47 Launching APs: 1 9 3 4 5 6 8 10 11 7 2 Timecounter "TSC-low" frequency 1750034708 Hz quality 1000 random: entropy device external interface WARNING: Device "kbd" is Giant locked and may be deleted before FreeBSD 13.0. kbd1 at kbdmux0 000.000053 [4346] netmap_init netmap: loaded module [ath_hal] loaded nexus0 vtvga0: cryptosoft0: acpi0: acpi0: Power Button (fixed) cpu0: on acpi0 atrtc0: port 0x70-0x71,0x74-0x77 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Event timer "HPET4" frequency 14318180 Hz quality 340 Event timer "HPET5" frequency 14318180 Hz quality 340 Event timer "HPET6" frequency 14318180 Hz quality 340 Event timer "HPET7" frequency 14318180 Hz quality 340 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: on acpi0 pci0: on pcib0 pci0: at device 11.1 (no driver attached) pci0: at device 11.2 (no driver attached) pci0: at device 16.1 (no driver attached) pci0: at device 16.6 (no driver attached) pci0: at device 18.1 (no driver attached) acpi_syscontainer0: on acpi0 acpi_syscontainer1: on acpi0 acpi_syscontainer2: on acpi0 acpi_syscontainer3: on acpi0 apei0: on acpi0 pcib1: port 0xcf8-0xcff on acpi0 pci1: on pcib1 pcib2: irq 26 at device 1.0 on pci1 pci2: on pcib2 pcib3: irq 32 at device 2.0 on pci1 pci3: on pcib3 pci3: at device 0.0 (no driver attached) pci3: at device 0.1 (no driver attached) pcib4: irq 40 at device 3.0 on pci1 pci4: on pcib4 pci1: at device 17.0 (no driver attached) ahci0: port 0xf110-0xf117,0xf100-0xf103,0xf0f0-0xf0f7,0xf0e0-0xf0e3,0xf020-0xf03f mem 0xfb438000-0xfb4387ff irq 16 at device 17.4 on pci1 ahci0: AHCI v1.30 with 4 6Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahciem0: on ahci0 xhci0: mem 0xfb400000-0xfb40ffff irq 19 at device 20.0 on pci1 xhci0: 32 bytes context size, 64-bit DMA usbus0 on xhci0 usbus0: 5.0Gbps Super Speed USB v3.0 pci1: at device 22.0 (no driver attached) pci1: at device 22.1 (no driver attached) ehci0: mem 0xfb434000-0xfb4343ff irq 18 at device 26.0 on pci1 usbus1: EHCI version 1.0 usbus1 on ehci0 usbus1: 480Mbps High Speed USB v2.0 pcib5: irq 16 at device 28.0 on pci1 pci5: on pcib5 pcib6: irq 18 at device 28.2 on pci1 pci6: on pcib6 pcib7: at device 0.0 on pci6 pci7: on pcib7 vgapci0: port 0xe000-0xe07f mem 0xfa000000-0xfaffffff,0xfb000000-0xfb01ffff irq 18 at device 0.0 on pci7 vgapci0: Boot video device pcib8: irq 16 at device 28.4 on pci1 pci8: on pcib8 igb0: port 0xd020-0xd03f mem 0xfb320000-0xfb33ffff,0xfb344000-0xfb347fff irq 16 at device 0.0 on pci8 igb0: Using 1024 TX descriptors and 1024 RX descriptors igb0: queue equality override not set, capping rx_queues at 6 and tx_queues at 6 igb0: Using 6 RX queues 6 TX queues igb0: Using MSI-X interrupts with 7 vectors igb0: Ethernet address: 0c:c4Blocks and Sizes uhub3: 4 ports with 3 removable, self powered uhub4: 6 ports with 6 removable, self powered uhub5: 8 ports with 8 removable, self powered ugen0.3: at usbus0 ukbd0 on uhub3 ukbd0: on usbus0 kbd2 at ukbd0 ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 490 files, 15263 used, 1250144 free (96 frags, 156256 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** Updating CPU Microcode... CPU: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz (3500.07-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306f2 Family=0x6 Model=0x3f Stepping=2 Features=0xbfebfbff Features2=0x7ffefbff AMD Features=0x2c100800 AMD Features2=0x21 Structured Extended Features=0x37ab Structured Extended Features3=0x9c000400 XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics Done. Setting up harvesting: PURE_RDRAND,[UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,[NET_ETHER],NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED Feeding entropy: . ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat /usr/local/lib/e2fsprogs /usr/local/lib/gcc8 /usr/local/lib/gcc9 /usr/local/lib/perl5/5.30/mach/CORE 32-bit compatibility ldconfig path: /usr/lib32 /usr/local/lib32/compat Setting hostname: mercat1.netperf.freebsd.org. lo0: link state changed to UP Starting Network: lo0 igb0 igb1. lo0: flags=8049 metric 0 mtu 16384 options=680003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=21 igb0: flags=8863 metric 0 mtu 1500 options=4e507bb ether 0c:c4:7a:a8:cd:da inet6 fe80::ec4:7aff:fea8:cdda%igb0 prefixlen 64 scopeid 0x1 media: Ethernet autoselect status: no carrier nd6 options=21 igb1: flags=8822 metric 0 mtu 1500 options=4e507bb ether 0c:c4:7a:a8:cd:db media: Ethernet autoselect status: no carrier nd6 options=29 Starting devd. igb0: link state changed to UP Autoloading module: acpi_wmi.ko acpi_wmi0: on acpi0 acpi_wmi0: cannot find EC device device_attach: acpi_wmi0 attach returned 6 Autoloading module: ioat.ko ioat0: mem 0xfb42c000-0xfb42ffff irq 31 at device 4.0 on pci1 ioat0: Capabilities: 2f7 ioat1: mem 0xfb428000-0xfb42bfff irq 39 at device 4.1 on pci1 ioat1: Capabilities: 2f7 ioat2: mem 0xfb424000-0xfb427fff irq 31 at device 4.2 on pci1 ioat2: Capabilities: f7 ioat3: mem 0xfb420000-0xfb423fff irq 39 at device 4.3 on pci1 ioat3: Capabilities: f7 ioat4: mem 0xfb41c000-0xfb41ffff irq 31 at device 4.4 on pci1 ioat4: Capabilities: f7 ioat5: mem 0xfb418000-0xfb41bfff irq 39 at device 4.5 on pci1 ioat5: Capabilities: f7 ioat6: mem 0xfb414000-0xfb417fff irq 31 at device 4.6 on pci1 ioat6: Capabilities: f7 ioat7: mem 0xfb410000-0xfb413fff irq 39 at device 4.7 on pci1 ioat7: Capabilities: f7 Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Starting Network: igb1. igb1: flags=8822 metric 0 mtu 1500 options=4e507bb ether 0c:c4:7a:a8:cd:db media: Ethernet autoselect status: no carrier nd6 options=29 Autoloading module: ichsmb.ko ichsmb0: port 0x580-0x59f mem 0xfb431000-0xfb4310ff irq 18 at device 31.3 on pci1 smbus0: on ichsmb0 Autoloading module: uhid.ko Autoloading module: ums.ko ums0 on uhub3 ums0: on usbus0 ums0: 3 buttons and [Z] coordinates ID=0 Starting dhclient. if_delmulti_locked: detaching ifnet instance 0xfffffe014ee2d800 DHCPDISCOVER on igb0 to 255.255.255.255 port 67 interval 3 DHCPOFFER from 192.168.5.1 DHCPREQUEST on igb0 to 255.255.255.255 port 67 DHCPACK from 192.168.5.1 if_delmulti_locked: detaching ifnet instance 0xfffffe014ee2d800 bound to 192.168.5.171 -- renewal in 300 seconds. add host 127.0.0.1: gateway lo0 fib 0: route already in table add host ::1: gateway lo0 fib 0: route already in table add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Updating motd:. Recovering vi editor sessions:. Creating and/or trimming log files. Clearing /tmp (X related). Starting syslogd. Setting date via ntp. 22 Dec 19:45:00 ntpdate[2357]: step time server 192.168.5.1 offset -1.014189 sec Starting watchdogd. No core dumps found. Starting rpcbind. NFS access cache time=60 Starting statd. Starting lockd. Mounting late filesystems:. Security policy loaded: MAC/ntpd (mac_ntpd) Starting ntpd. Starting powerd. Performing sanity check on sshd configuration. Starting sshd. Configuring vt: blanktime. Starting cron. Starting inetd. sysctl: unknown oid 'debug.fail_point.pmap_allocpte_page_alloc' at line 12 Tue Dec 22 19:45 FreeBSD/amd64 (mercat1.netperf.freebsd.org) (ttyu1) login: root Dec 22 19:45:50 mercat1 login[2509]: ROOT LOGIN (root) ON ttyu1 Last login: Tue Dec 22 19:17:53 on ttyu1 FreeBSD 13.0-CURRENT (PHO-KASAN) #0 r368820M: Tue Dec 22 19:33:25 CET 2020 78root@mercat1:~ # Dec 22 19:46:14 mercat1 su[2528]: pho to root on /dev/pts/0 20201222 19:47:17 all (1/735): 256m.sh 20201222 19:47:21 all (2/735): buildworld.sh 20201222 19:47:25 all (3/735): cpuset.sh 20201222 19:47:29 all (4/735): crossmp2.sh 20201222 19:47:33 all (5/735): devfs5.sh 20201222 19:47:38 all (6/735): holdcnt03.sh 20201222 19:47:42 all (7/735): holdcnt04.sh 20201222 19:47:46 all (8/735): jexec.sh 20201222 19:47:50 all (9/735): ldt.sh 20201222 19:47:54 all (10/735): ldt2.sh 20201222 19:47:59 all (11/735): libMicro.sh 20201222 19:48:03 all (12/735): lockd.sh 20201222 19:48:07 all (13/735): lockf.sh 20201222 19:48:11 all (14/735): lookup_shared.sh 20201222 19:48:15 all (15/735): mac.sh 20201222 19:48:19 all (16/735): nfs.sh 20201222 19:48:24 all (17/735): nfs12.sh 20201222 19:48:28 all (18/735): nfs14.sh 20201222 19:48:32 all (19/735): nfs8.sh 20201222 19:48:36 all (20/735): nfsdelegation.sh 20201222 19:48:40 all (21/735): nfsdepth.sh 20201222 19:48:44 all (22/735): nfsrename.sh 20201222 19:48:49 all (23/735): numa.sh 20201222 19:48:53 all (24/735): pagefault.sh 20201222 19:48:57 all (25/735): pathconf2.sh 20201222 19:49:01 all (26/735): pcatch.sh 20201222 19:49:05 all (27/735): pcatch2.sh 20201222 19:49:09 all (28/735): pkru.sh 20201222 19:49:13 all (29/735): pkru2.sh 20201222 19:49:18 all (30/735): quota5.sh 20201222 19:49:22 all (31/735): schedfuzz.sh 20201222 19:49:26 all (32/735): vmio.sh 20201222 19:49:30 all (33/735): watchman.sh 20201222 19:49:34 all (34/735): poll.sh 20201222 19:49:38 all (35/735): crossmp6.sh 20201222 19:49:43 all (36/735): holdcnt02.sh 20201222 19:49:47 all (37/735): holdcnt05.sh 20201222 19:49:51 all (38/735): linux.sh 20201222 19:49:55 all (39/735): machipc2.sh 20201222 19:49:59 all (40/735): mmap16.sh 20201222 19:50:03 all (41/735): audit.sh 20201222 19:50:07 all (42/735): buildworld2.sh 20201222 19:50:12 all (43/735): gnop3.sh 20201222 19:50:16 all (44/735): mac_chkexec.sh 20201222 19:50:20 all (45/735): md4.sh 20201222 19:50:24 all (46/735): mdconfig.sh 20201222 19:50:29 all (47/735): mount.sh 20201222 19:50:33 all (48/735): msdos3.sh 20201222 19:50:38 all (49/735): nfs15.sh 20201222 19:50:42 all (50/735): nullfs4.sh stress2: mount leak: 13/44 20201222 19:50:46 all (51/735): nullfs6.sh 20201222 19:50:51 all (52/735): oom.sh 20201222 19:50:55 all (53/735): nfs3.sh 20201222 19:50:59 all (54/735): nfs9.sh 20201222 19:51:03 all (55/735): nfssillyrename.sh 20201222 19:51:07 all (56/735): parallelmount2.sh 20201222 19:51:11 all (57/735): pathconf.sh 20201222 19:51:16 all (58/735): procfs6.sh 20201222 19:51:20 all (59/735): segregs.sh 20201222 19:51:24 all (60/735): sendfile19.sh 20201222 19:51:28 all (61/735): sendfile20.sh 20201222 19:51:32 all (62/735): tmpfs15.sh 20201222 19:51:37 all (63/735): tvnlru.sh 20201222 19:51:41 all (64/735): fsck2.sh 20201222 19:51:45 all (65/735): machipc.sh 20201222 19:51:49 all (66/735): maxmemdom.sh 20201222 19:51:53 all (67/735): mmap20.sh 20201222 19:52:01 all (68/735): nullfs14.sh 20201222 19:52:06 all (69/735): nullfs23.sh 20201222 19:52:10 all (70/735): ptrace3.sh 20201222 19:52:14 all (71/735): open.sh 20201222 19:52:19 all (72/735): routetbl.sh 20201222 19:52:23 all (73/735): syzkaller6.sh 20201222 19:52:28 all (74/735): uma_zalloc_arg.sh 20201222 19:52:32 all (75/735): trim5.sh panic: ASan: Invalid access 4-byte read at 0xfffffe01293cb1e0, RedZonePartial cpuid = 8 time = 1608663155 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe01d50a8ed0 vpanic() at vpanic+0x1f0/frame 0xfffffe01d50a8f30 panic() at panic+0x4f/frame 0xfffffe01d50a8f90 __asan_load4_noabort() at __asan_load4_noabort+0x1db/frame 0xfffffe01d50a9050 cgaccount() at cgaccount+0xfa/frame 0xfffffe01d50a9110 ffs_snapshot() at ffs_snapshot+0x1494/frame 0xfffffe01d50a94f0 ffs_mount() at ffs_mount+0x14e6/frame 0xfffffe01d50a96f0 vfs_domount() at vfs_domount+0xb7f/frame 0xfffffe01d50a99b0 vfs_donmount() at vfs_donmount+0xac7/frame 0xfffffe01d50a9a50 sys_nmount() at sys_nmount+0xa9/frame 0xfffffe01d50a9aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d50a9bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d50a9bf0 --- syscall (378, FreeBSD ELF64, sys_nmount), rip = 0x80038468a, rsp = 0x7fffffffce38, rbp = 0x7fffffffe690 --- KDB: enter: panic [ thread pid 40197 tid 100252 ] Stopped at kdb_enter+0x37: movq $0,0x239bc0e(%rip) db> run pho db:0:pho> set $lines 20000 db:0:pho> run pho1 db:1:pho1> dump Dumping 1673 out of 32584 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Dump complete db:1:pho1> bt Tracing pid 40197 tid 100252 td 0xfffffe01d1916000 kdb_enter() at kdb_enter+0x37/frame 0xfffffe01d50a8ed0 vpanic() at vpanic+0x229/frame 0xfffffe01d50a8f30 panic() at panic+0x4f/frame 0xfffffe01d50a8f90 __asan_load4_noabort() at __asan_load4_noabort+0x1db/frame 0xfffffe01d50a9050 cgaccount() at cgaccount+0xfa/frame 0xfffffe01d50a9110 ffs_snapshot() at ffs_snapshot+0x1494/frame 0xfffffe01d50a94f0 ffs_mount() at ffs_mount+0x14e6/frame 0xfffffe01d50a96f0 vfs_domount() at vfs_domount+0xb7f/frame 0xfffffe01d50a99b0 vfs_donmount() at vfs_donmount+0xac7/frame 0xfffffe01d50a9a50 sys_nmount() at sys_nmount+0xa9/frame 0xfffffe01d50a9aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d50a9bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d50a9bf0 --- syscall (378, FreeBSD ELF64, sys_nmount), rip = 0x80038468a, rsp = 0x7fffffffce38, rbp = 0x7fffffffe690 --- db:1:pho1> show allpcpu Current CPU: 8 cpuid = 0 dynamic pcpu = 0xb44840 curthread = 0xfffffe012928a300: pid 11 tid 100003 critnest 1 "idle: cpu0" curpcb = 0xfffffe012928a810 fpcurthread = none idlethread = 0xfffffe012928a300: tid 100003 "idle: cpu0" self = 0xffffffff83e10000 curpmap = 0xffffffff83558180 tssp = 0xffffffff83e10384 rsp0 = 0xfffffe0128f35cc0 kcr3 = 0x8000000003cdc002 ucr3 = 0xffffffffffffffff scr3 = 0x6b2118804 gs32p = 0xffffffff83e10404 ldt = 0xffffffff83e10444 tss = 0xffffffff83e10434 curvnet = 0 spin locks held: cpuid = 1 dynamic pcpu = 0xfffffe00fe504840 curthread = 0xfffffe0129289c00: pid 11 tid 100004 critnest 1 "idle: cpu1" curpcb = 0xfffffe012928a110 fpcurthread = none idlethread = 0xfffffe0129289c00: tid 100004 "idle: cpu1" self = 0xffffffff83e11000 curpmap = 0xffffffff83558180 tssp = 0xffffffff83e11384 rsp0 = 0xfffffe0128f30cc0 kcr3 = 0x8000000003cdc002 ucr3 = 0xffffffffffffffff scr3 = 0x2b3e92fe2 gs32p = 0xffffffff83e11404 ldt = 0xffffffff83e11444 tss = 0xffffffff83e11434 curvnet = 0 spin locks held: cpuid = 2 dynamic pcpu = 0xfffffe00fe510840 curthread = 0xfffffe0129289500: pid 11 tid 100005 critnest 1 "idle: cpu2" curpcb = 0xfffffe0129289a10 fpcurthread = none idlethread = 0xfffffe0129289500: tid 100005 "idle: cpu2" self = 0xffffffff83e12000 curpmap = 0xffffffff83558180 tssp = 0xffffffff83e12384 rsp0 = 0xfffffe0128f3fcc0 kcr3 = 0x8000000003cdc002 ucr3 = 0xffffffffffffffff scr3 = 0x2b3b55fa9 gs32p = 0xffffffff83e12404 ldt = 0xffffffff83e12444 tss = 0xffffffff83e12434 curvnet = 0 spin locks held: cpuid = 3 dynamic pcpu = 0xfffffe00fe51c840 curthread = 0xfffffe0129288e00: pid 11 tid 100006 critnest 1 "idle: cpu3" curpcb = 0xfffffe0129289310 fpcurthread = none idlethread = 0xfffffe0129288e00: tid 100006 "idle: cpu3" self = 0xffffffff83e13000 curpmap = 0xffffffff83558180 tssp = 0xffffffff83e13384 rsp0 = 0xfffffe0128f3acc0 kcr3 = 0x8000000003cdc002 ucr3 = 0xffffffffffffffff scr3 = 0x2b39dcf53 gs32p = 0xffffffff83e13404 ldt = 0xffffffff83e13444 tss = 0xffffffff83e13434 curvnet = 0 spin locks held: cpuid = 4 dynamic pcpu = 0xfffffe00fe528840 curthread = 0xfffffe0129288700: pid 11 tid 100007 critnest 1 "idle: cpu4" curpcb = 0xfffffe0129288c10 fpcurthread = none idlethread = 0xfffffe0129288700: tid 100007 "idle: cpu4" self = 0xffffffff83e14000 curpmap = 0xffffffff83558180 tssp = 0xffffffff83e14384 rsp0 = 0xfffffe0128f49cc0 kcr3 = 0x8000000003cdc002 ucr3 = 0xffffffffffffffff scr3 = 0x6b207786a gs32p = 0xffffffff83e14404 ldt = 0xffffffff83e14444 tss = 0xffffffff83e14434 curvnet = 0 spin locks held: cpuid = 5 dynamic pcpu = 0xfffffe00fe534840 curthread = 0xfffffe0129288000: pid 11 tid 100008 critnest 1 "idle: cpu5" curpcb = 0xfffffe0129288510 fpcurthread = none idlethread = 0xfffffe0129288000: tid 100008 "idle: cpu5" self = 0xffffffff83e15000 curpmap = 0xffffffff83558180 tssp = 0xffffffff83e15384 rsp0 = 0xfffffe0128f44cc0 kcr3 = 0x8000000003cdc002 ucr3 = 0xffffffffffffffff scr3 = 0x2b3b0f853 gs32p = 0xffffffff83e15404 ldt = 0xffffffff83e15444 tss = 0xffffffff83e15434 curvnet = 0 spin locks held: cpuid = 6 dynamic pcpu = 0xfffffe00fe540840 curthread = 0xfffffe01d083f100: pid 40193 tid 100150 critnest 1 "md10" curpcb = 0xfffffe01d083f610 fpcurthread = none idlethread = 0xfffffe012928b800: tid 100009 "idle: cpu6" self = 0xffffffff83e16000 curpmap = 0xffffffff83558180 tssp = 0xffffffff83e16384 rsp0 = 0xfffffe01d054bcc0 kcr3 = 0x8000000003cdc002 ucr3 = 0xffffffffffffffff scr3 = 0x2a848e99a gs32p = 0xffffffff83e16404 ldt = 0xffffffff83e16444 tss = 0xffffffff83e16434 curvnet = 0 spin locks held: cpuid = 7 dynamic pcpu = 0xfffffe00fe54c840 curthread = 0xfffffe0129296100: pid 11 tid 100010 critnest 1 "idle: cpu7" curpcb = 0xfffffe0129296610 fpcurthread = none idlethread = 0xfffffe0129296100: tid 100010 "idle: cpu7" self = 0xffffffff83e17000 curpmap = 0xffffffff83558180 tssp = 0xffffffff83e17384 rsp0 = 0xfffffe0128f4ecc0 kcr3 = 0x8000000003cdc002 ucr3 = 0xffffffffffffffff scr3 = 0x2b3b21a66 gs32p = 0xffffffff83e17404 ldt = 0xffffffff83e17444 tss = 0xffffffff83e17434 curvnet = 0 spin locks held: cpuid = 8 dynamic pcpu = 0xfffffe00fe558840 curthread = 0xfffffe01d1916000: pid 40197 tid 100252 critnest 1 "mksnap_ffs" curpcb = 0xfffffe01d1916510 fpcurthread = 0xfffffe01d1916000: pid 40197 "mksnap_ffs" idlethread = 0xfffffe0129295a00: tid 100011 "idle: cpu8" self = 0xffffffff83e18000 curpmap = 0xfffffe01e71d7500 tssp = 0xffffffff83e18384 rsp0 = 0xfffffe01d50a9cc0 kcr3 = 0x80000002b364d1d6 ucr3 = 0x80000002b3b979d6 scr3 = 0x2b3b979d6 gs32p = 0xffffffff83e18404 ldt = 0xffffffff83e18444 tss = 0xffffffff83e18434 curvnet = 0 spin locks held: cpuid = 9 dynamic pcpu = 0xfffffe00fe564840 curthread = 0xfffffe0129295300: pid 11 tid 100012 critnest 1 "idle: cpu9" curpcb = 0xfffffe0129295810 fpcurthread = none idlethread = 0xfffffe0129295300: tid 100012 "idle: cpu9" self = 0xffffffff83e19000 curpmap = 0xffffffff83558180 tssp = 0xffffffff83e19384 rsp0 = 0xfffffe0128f58cc0 kcr3 = 0x8000000003cdc002 ucr3 = 0xffffffffffffffff scr3 = 0x2b3e31929 gs32p = 0xffffffff83e19404 ldt = 0xffffffff83e19444 tss = 0xffffffff83e19434 curvnet = 0 spin locks held: cpuid = 10 dynamic pcpu = 0xfffffe00fe571840 curthread = 0xfffffe0129294c00: pid 11 tid 100013 critnest 1 "idle: cpu10" curpcb = 0xfffffe0129295110 fpcurthread = none idlethread = 0xfffffe0129294c00: tid 100013 "idle: cpu10" self = 0xffffffff83e1a000 curpmap = 0xffffffff83558180 tssp = 0xffffffff83e1a384 rsp0 = 0xfffffe0128f67cc0 kcr3 = 0x8000000003cdc002 ucr3 = 0xffffffffffffffff scr3 = 0x2b3b0f9c5 gs32p = 0xffffffff83e1a404 ldt = 0xffffffff83e1a444 tss = 0xffffffff83e1a434 curvnet = 0 spin locks held: cpuid = 11 dynamic pcpu = 0xfffffe00fe57d840 curthread = 0xfffffe0129294500: pid 11 tid 100014 critnest 1 "idle: cpu11" curpcb = 0xfffffe0129294a10 fpcurthread = none idlethread = 0xfffffe0129294500: tid 100014 "idle: cpu11" self = 0xffffffff83e1b000 curpmap = 0xffffffff83558180 tssp = 0xffffffff83e1b384 rsp0 = 0xfffffe0128f62cc0 kcr3 = 0x8000000003cdc002 ucr3 = 0xffffffffffffffff scr3 = 0x2b3a5b8c5 gs32p = 0xffffffff83e1b404 ldt = 0xffffffff83e1b444 tss = 0xffffffff83e1b434 curvnet = 0 spin locks held: db:1:pho1> show alllocks Process 40197 (mksnap_ffs) thread 0xfffffe01d1916000 (100252) exclusive sleep mutex FFS (FFS Lock) r = 0 (0xfffffe01d27cca90) locked @ ufs/ffs/ffs_snapshot.c:917 exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe007c496860) locked @ kern/vfs_bio.c:3895 exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe007c86d608) locked @ kern/vfs_bio.c:3895 exclusive lockmgr ufs (ufs) r = 0 (0xfffffe01e7567570) locked @ ufs/ffs/ffs_vfsops.c:1999 db:1:pho1> show lockedvnods Locked vnodes vnode 0xfffffe01e7567500: type VREG usecount 1, writecount 0, refcount 10 seqc users 0 hold count flags () flags (VV_SYSTEM|VMP_LAZYLIST) v_object 0xfffffe01e777b840 ref 0 pages 64 cleanbuf 7 dirtybuf 1 lock type ufs: EXCL by thread 0xfffffe01d1916000 (pid 40197, mksnap_ffs, tid 100252) #0 0xffffffff8115b92e at lockmgr_xlock_hard+0x66e #1 0xffffffff8115c703 at __lockmgr_args+0x9c3 #2 0xffffffff8160629f at ffs_vgetf+0x1bf #3 0xffffffff815b56aa at ffs_valloc+0x92a #4 0xffffffff81631e47 at ufs_makeinode+0x1c7 #5 0xffffffff8162a1b9 at ufs_create+0x99 #6 0xffffffff819b6eea at VOP_CREATE_APV+0x9a #7 0xffffffff815c54e3 at ffs_snapshot+0x783 #8 0xffffffff816085d6 at ffs_mount+0x14e6 #9 0xffffffff812ef16f at vfs_domount+0xb7f #10 0xffffffff812ed137 at vfs_donmount+0xac7 #11 0xffffffff812ec629 at sys_nmount+0xa9 #12 0xffffffff8187edb2 at amd64_syscall+0x282 #13 0xffffffff8184159e at fast_syscall_common+0xf8 nlink=1, effnlink=1, size=134217728, extsize 0 generation=e464635, uid=0, gid=5, flags=0x0 ino 4, on dev md10a db:1:pho1> show mount 0xfffffe015391d700 /dev/gpt/work on / (ufs) 0xfffffe015391e280 devfs on /dev (devfs) 0xfffffe01d27c1280 /dev/gpt/tmp on /tmp (ufs) 0xfffffe01e7741280 procfs on /proc (procfs) 0xfffffe01d27bfb80 /dev/md10a on /mnt (ufs) More info: show mount db:1:pho1> show bufqueues bqempty: 93531 Buf domain 0 freebufs 12124 lofreebufs 219 hifreebufs 329 bufspace 67145728 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 17 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 2927 (64897024) dirtyq count 17 (528384) wakeup 0 lim 25 CPU 4, 3, 2, 1, 14, 9, 2, 6, 0, 9, 6, 9, Locked buffers: 0 space 0 Total buffers: 3045 space 67145728 Buf domain 1 freebufs 13391 lofreebufs 219 hifreebufs 329 bufspace 25210880 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 1600 (22757376) dirtyq count 0 (0) wakeup 0 lim 25 CPU 5, 9, 19, 16, 5, 24, 2, 18, 17, 18, 4, 5, Locked buffers: 0 space 0 Total buffers: 1744 space 25210880 Buf domain 2 freebufs 12571 lofreebufs 219 hifreebufs 329 bufspace 52518912 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 2416 (49893376) dirtyq count 0 (0) wakeup 0 lim 25 CPU 15, 7, 16, 12, 1, 18, 10, 18, 2, 10, 22, 15, Locked buffers: 0 space 0 Total buffers: 2562 space 52518912 Buf domain 3 freebufs 13978 lofreebufs 219 hifreebufs 329 bufspace 6471680 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 1 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 1046 (5664768) dirtyq count 1 (4096) wakeup 0 lim 25 CPU 7, 3, 11, 10, 4, 17, 10, 9, 1, 21, 13, 2, Locked buffers: 0 space 0 Total buffers: 1158 space 6471680 Buf domain 4 freebufs 13871 lofreebufs 219 hifreebufs 329 bufspace 9723904 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 1 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 1148 (8753152) dirtyq count 1 (32768) wakeup 0 lim 25 CPU 0, 3, 6, 3, 19, 23, 12, 15, 3, 18, 0, 8, Locked buffers: 4 space 131072 Total buffers: 1262 space 9723904 Buf domain 5 freebufs 13856 lofreebufs 219 hifreebufs 329 bufspace 9805824 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 3 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 1158 (8675328) dirtyq count 3 (98304) wakeup 0 lim 25 CPU 4, 18, 13, 9, 13, 9, 5, 8, 11, 3, 19, 4, Locked buffers: 0 space 0 Total buffers: 1278 space 9805824 Buf domain 6 freebufs 13882 lofreebufs 219 hifreebufs 329 bufspace 8859648 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 3 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 1145 (7335936) dirtyq count 3 (12288) wakeup 0 lim 25 CPU 19, 4, 10, 18, 17, 0, 11, 3, 0, 13, 1, 6, Locked buffers: 2 space 36864 Total buffers: 1254 space 8859648 db:1:pho1> show page vm_cnt.v_free_count: 7666106 vm_cnt.v_inactive_count: 1080 vm_cnt.v_active_count: 10091 vm_cnt.v_laundry_count: 0 vm_cnt.v_wire_count: 394718 vm_cnt.v_free_reserved: 10630 vm_cnt.v_free_min: 51197 vm_cnt.v_free_target: 172898 vm_cnt.v_inactive_target: 259347 db:1:pho1> show pageq pq_free 7666106 dom 0 page_cnt 8113516 free 7666106 pq_act 10091 pq_inact 1080 pq_laund 0 pq_unsw 0 db:0:pho> show freepages DOMAIN: 0 FREE LIST 0: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 001830 | 000000 11 (008192K) | 000001 | 000001 10 (004096K) | 000000 | 000001 09 (002048K) | 000000 | 000000 08 (001024K) | 000001 | 000001 07 (000512K) | 000001 | 000001 06 (000256K) | 000001 | 000001 05 (000128K) | 000001 | 000001 04 (000064K) | 000001 | 000000 03 (000032K) | 000001 | 000000 02 (000016K) | 000001 | 000000 01 (000008K) | 000000 | 000013 00 (000004K) | 000000 | 000108 FREE LIST 1: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000000 | 000000 10 (004096K) | 000000 | 000000 09 (002048K) | 000000 | 000000 08 (001024K) | 000000 | 000000 07 (000512K) | 000000 | 000000 06 (000256K) | 000001 | 000000 05 (000128K) | 000001 | 000000 04 (000064K) | 000002 | 000000 03 (000032K) | 000000 | 000000 02 (000016K) | 000001 | 000000 01 (000008K) | 000000 | 000000 00 (000004K) | 000000 | 000000 db:0:pho> show uma Zone Size Used Free Requests Sleeps Bucket Total Mem XFree FFS inode 1128 132916 126 133219 0 8 150071376 0 VNODE 640 133046 184 133397 0 16 85267200 0 FFS2 dinode 256 132916 134 133219 0 62 34060800 0 mbuf_cluster 2048 11034 20 309777 0 254 22638592 0 BUF TRIE 144 1115 104837 6165 0 62 15257088 0 S VFS Cache 104 133482 1575 134776 0 126 14045928 0 vmem btag 56 91571 493 91571 0 254 5155584 0 mbuf 256 14662 578 317543 0 254 3901440 0 RADIX NODE 144 15645 2411 1974436 0 63 2600064 0 VM OBJECT 264 8837 508 593781 0 30 2467080 0 pbuf 2784 0 864 0 0 0 2405376 0 malloc-128 128 14560 258 1123554 0 126 1896704 0 malloc-4096 4096 359 18 2790 0 2 1544192 0 malloc-128 128 10995 258 11075 0 126 1440384 0 256 Bucket 2048 574 96 50227 0 8 1372160 0 malloc-16384 16384 10 49 5477 0 1 966656 0 malloc-65536 65536 2 12 330 0 1 917504 0 UMA Slabs 0 112 7931 22 7931 0 126 890736 0 malloc-32768 32768 9 18 61 0 1 884736 0 malloc-65536 65536 2 11 67 0 1 851968 0 malloc-8192 8192 89 2 93 0 1 745472 0 DIRHASH 1024 660 20 660 0 16 696320 0 THREAD 1792 316 62 366 0 8 677376 0 malloc-4096 4096 130 35 133882 0 2 675840 0 DEVCTL 1024 0 652 679 0 0 667648 0 malloc-65536 65536 4 6 13 0 1 655360 0 malloc-64 64 9524 619 19009 0 254 649152 0 malloc-64 64 9343 737 49872 0 254 645120 0 MAP ENTRY 96 2002 4466 2021340 0 127 620928 0 malloc-65536 65536 3 6 3090 0 1 589824 0 malloc-32768 32768 12 5 21 0 1 557056 0 128 Bucket 1024 252 219 21421 0 16 482304 0 UMA Zones 2048 230 5 230 0 8 481280 0 malloc-32768 32768 12 1 13 0 1 425984 0 malloc-16384 16384 14 12 4046 0 1 425984 0 malloc-65536 65536 6 0 6 0 1 393216 0 md10 512 697 7 697 0 30 360448 0 malloc-2048 2048 16 142 21837 0 8 323584 0 VMSPACE 2544 45 81 40159 0 4 320544 0 malloc-256 256 442 608 2229 0 62 268800 0 PROC 1320 79 116 40197 0 8 257400 0 malloc-16 16 14156 1844 14370 0 254 256000 0 32 Bucket 256 203 787 5777 0 62 253440 0 filedesc0 1072 80 151 40198 0 8 247632 0 malloc-32 32 6742 944 6834 0 254 245952 0 malloc-256 256 627 333 4428 0 62 245760 0 malloc-64 64 3308 472 3317 0 254 241920 0 malloc-1024 1024 200 36 5574 0 16 241664 0 malloc-384 384 337 283 1290 0 30 238080 0 malloc-2048 2048 112 4 112 0 8 237568 0 malloc-32768 32768 1 6 3254 0 1 229376 0 malloc-128 128 1549 218 17483 0 126 226176 0 g_bio 408 6 534 91492 0 30 220320 0 malloc-8192 8192 25 1 27 0 1 212992 0 malloc-256 256 287 538 8223 0 62 211200 0 pipe 744 9 266 24195 0 16 204600 0 malloc-64 64 99 2925 241688 0 254 193536 0 malloc-256 256 47 673 2449 0 62 184320 0 malloc-256 256 1 689 1488 0 62 176640 0 socket 944 57 123 6092 0 254 169920 0 ksiginfo 112 170 1306 889 0 126 165312 0 malloc-32768 32768 1 4 9 0 1 163840 0 malloc-16384 16384 10 0 10 0 1 163840 0 malloc-8192 8192 1 19 265 0 1 163840 0 Files 80 140 1860 306459 0 126 160000 0 64 Bucket 512 238 74 14668 0 30 159744 0 malloc-128 128 674 504 27055 0 126 150784 0 malloc-8192 8192 14 3 2336 0 1 139264 0 mbuf_packet 256 15 525 728 0 254 138240 0 malloc-64 64 16 1937 8506 0 254 124992 0 8 Bucket 80 195 1305 214132 0 126 120000 0 PWD 32 20 3508 29676 0 254 112896 0 malloc-256 256 94 311 5006 0 62 103680 0 malloc-1024 1024 63 37 774 0 16 102400 0 malloc-16384 16384 0 6 12 0 1 98304 0 malloc-8192 8192 12 0 12 0 1 98304 0 ttyinq 160 345 255 555 0 62 96000 0 NAMEI 1024 1 91 668681 0 16 94208 0 malloc-8192 8192 5 6 11 0 1 90112 0 malloc-512 512 124 52 1130 0 30 90112 0 ttyoutq 256 181 164 290 0 62 88320 0 16 Bucket 144 194 394 23827 0 62 84672 0 malloc-128 128 283 368 1467 0 126 83328 0 tcpcb 1048 12 65 17 0 254 80696 0 UMA Kegs 384 200 3 200 0 30 77952 0 malloc-2048 2048 10 28 6626 0 8 77824 0 malloc-4096 4096 4 14 816 0 2 73728 0 4 Bucket 48 43 1469 3832 0 254 72576 0 Mountpoints 2944 5 19 15 0 4 70656 0 malloc-1024 1024 23 45 50598 0 16 69632 0 malloc-64 64 293 778 53985 0 254 68544 0 TURNSTILE 136 379 104 379 0 62 65688 0 malloc-65536 65536 0 1 9 0 1 65536 0 malloc-65536 65536 0 1 1 0 1 65536 0 malloc-65536 65536 1 0 1 0 1 65536 0 malloc-32768 32768 1 1 3 0 1 65536 0 malloc-32 32 761 1255 39994 0 254 64512 0 malloc-32 32 153 1863 15508 0 254 64512 0 udp_inpcb 488 22 106 167 0 254 62464 0 unpcb 256 20 205 5891 0 254 57600 0 malloc-4096 4096 13 1 24 0 2 57344 0 malloc-2048 2048 2 26 56 0 8 57344 0 malloc-16 16 677 2823 1077138 0 254 56000 0 malloc-4096 4096 10 3 22 0 2 53248 0 SLEEPQUEUE 88 379 197 379 0 126 50688 0 malloc-384 384 83 47 83 0 30 49920 0 malloc-8192 8192 5 1 6 0 1 49152 0 malloc-1024 1024 5 43 120 0 16 49152 0 rl_entry 40 163 1049 164 0 254 48480 0 udpcb 32 22 1490 167 0 254 48384 0 malloc-32 32 29 1483 162 0 254 48384 0 malloc-32 32 254 1258 14651 0 254 48384 0 malloc-32 32 309 1203 5333 0 254 48384 0 2 Bucket 32 343 1169 13714 0 254 48384 0 malloc-16 16 752 2248 27512 0 254 48000 0 malloc-16 16 74 2926 196 0 254 48000 0 malloc-128 128 77 295 315 0 126 47616 0 kenv 258 0 180 21352 0 30 46440 0 L VFS Cache 320 113 31 113 0 30 46080 0 malloc-1024 1024 1 43 354 0 16 45056 0 fakepg 104 0 429 316 0 126 44616 0 malloc-16 16 23 2727 833 0 254 44000 0 malloc-128 128 4 337 314 0 126 43648 0 malloc-128 128 45 296 161 0 126 43648 0 malloc-384 384 39 71 7912 0 30 42240 0 malloc-4096 4096 0 10 32 0 2 40960 0 pcpu-64 64 449 191 449 0 254 40960 0 malloc-64 64 31 599 150 0 254 40320 0 malloc-32 32 25 1235 138 0 254 40320 0 cpuset 104 27 345 27 0 126 38688 0 malloc-384 384 13 87 54 0 30 38400 0 malloc-512 512 12 60 27 0 30 36864 0 malloc-512 512 16 56 47 0 30 36864 0 pcpu-8 8 3331 1277 3335 0 254 36864 0 malloc-32 32 11 1123 97 0 254 36288 0 clpbuf 2784 0 12 19 0 2 33408 0 malloc-32768 32768 1 0 1 0 1 32768 0 malloc-16384 16384 2 0 2 0 1 32768 0 malloc-4096 4096 8 0 8 0 2 32768 0 malloc-512 512 0 64 575 0 30 32768 0 malloc-512 512 15 49 15 0 30 32768 0 malloc-1024 1024 4 24 13 0 16 28672 0 malloc-16 16 4 1746 10 0 254 28000 0 tcp_inpcb 488 12 44 17 0 254 27328 0 malloc-384 384 0 70 101 0 30 26880 0 pcpu-16 16 24 1512 24 0 254 24576 0 rtentry 176 12 126 16 0 62 24288 0 routing nhops 256 9 81 16 0 62 23040 0 malloc-2048 2048 2 8 295 0 8 20480 0 malloc-16 16 16 1234 17 0 254 20000 0 malloc-256 256 9 66 17 0 62 19200 0 mbuf_jumbo_page 4096 0 4 4 0 254 16384 0 malloc-16384 16384 1 0 1 0 1 16384 0 malloc-2048 2048 3 5 4 0 8 16384 0 malloc-512 512 3 29 6 0 30 16384 0 TMPFS node 224 0 72 5 0 62 16128 0 malloc-64 64 49 203 177 0 254 16128 0 malloc-512 512 6 18 40 0 30 12288 0 malloc-16 16 24 726 56 0 254 12000 0 malloc-256 256 12 33 12 0 62 11520 0 malloc-8192 8192 1 0 1 0 1 8192 0 syncache 168 0 48 2 0 254 8064 0 ripcb 488 1 15 3 0 254 7808 0 malloc-384 384 3 17 43 0 30 7680 0 FPU_save_area 832 1 8 1 0 16 7488 0 epoch_record pcpu 256 4 12 4 0 62 4096 0 malloc-2048 2048 2 0 2 0 8 4096 0 vtnet_tx_hdr 24 0 167 0 0 254 4008 0 UMA Slabs 1 176 8 14 8 0 62 3872 0 sctp_ep 1280 0 3 1 0 254 3840 0 KMAP ENTRY 96 37 2 45 0 0 3744 0 vmem 1856 1 1 1 0 8 3712 0 SMR CPU 32 2 61 2 0 254 2016 0 SMR SHARED 24 2 61 2 0 254 1512 0 sctp_asconf_ack 48 0 0 0 0 254 0 0 sctp_asconf 40 0 0 0 0 254 0 0 sctp_stream_msg_out 112 0 0 0 0 254 0 0 sctp_readq 152 0 0 0 0 254 0 0 sctp_chunk 152 0 0 0 0 254 0 0 sctp_raddr 736 0 0 0 0 254 0 0 sctp_laddr 48 0 0 0 0 254 0 0 sctp_asoc 2288 0 0 0 0 254 0 0 FFS1 dinode 128 0 0 0 0 126 0 0 swblk 136 0 0 0 0 62 0 0 swpctrie 144 0 0 0 0 62 0 0 IPsec SA lft_c 16 0 0 0 0 254 0 0 udplite_inpcb 488 0 0 0 0 254 0 0 tcp_log_node 120 0 0 0 0 126 0 0 tcp_log_bucket 176 0 0 0 0 62 0 0 tcp_log 416 0 0 0 0 254 0 0 tcpreass 48 0 0 0 0 254 0 0 tfo_ccache_entries 80 0 0 0 0 126 0 0 tfo 4 0 0 0 0 254 0 0 sackhole 32 0 0 0 0 254 0 0 hostcache 96 0 0 0 0 254 0 0 tcptw 88 0 0 0 0 254 0 0 ipq 56 0 0 0 0 254 0 0 KNOTE 160 0 0 0 0 62 0 0 itimer 352 0 0 0 0 30 0 0 AIOLIO 272 0 0 0 0 30 0 0 AIOCB 488 0 0 0 0 30 0 0 AIOP 32 0 0 0 0 254 0 0 AIO 208 0 0 0 0 62 0 0 NCLNODE 744 0 0 0 0 16 0 0 LTS VFS Cache 360 0 0 0 0 30 0 0 STS VFS Cache 144 0 0 0 0 62 0 0 crypto_session 72 0 0 0 0 126 0 0 cryptop 280 0 0 0 0 30 0 0 IOMMU_MAP_ENTRY 120 0 0 0 0 126 0 0 mbuf_jumbo_16k 16384 0 0 0 0 254 0 0 mbuf_jumbo_9k 9216 0 0 0 0 254 0 0 audit_record 1280 0 0 0 0 8 0 0 domainset 40 0 0 0 0 254 0 0 MAC labels 40 0 0 0 0 254 0 0 vnpbuf 2784 0 0 0 0 10 0 0 mdpbuf 2784 0 0 0 0 1 0 0 nfspbuf 2784 0 0 0 0 2 0 0 swwbuf 2784 0 0 0 0 1 0 0 swrbuf 2784 0 0 0 0 2 0 0 umtx_shm 88 0 0 0 0 126 0 0 umtx pi 96 0 0 0 0 126 0 0 rangeset pctrie nodes 144 0 0 0 0 62 0 0 malloc-32768 32768 0 0 0 0 1 0 0 malloc-16384 16384 0 0 0 0 1 0 0 malloc-16384 16384 0 0 0 0 1 0 0 malloc-4096 4096 0 0 0 0 2 0 0 malloc-2048 2048 0 0 0 0 8 0 0 malloc-1024 1024 0 0 0 0 16 0 0 malloc-1024 1024 0 0 0 0 16 0 0 malloc-512 512 0 0 0 0 30 0 0 malloc-384 384 0 0 0 0 30 0 0 malloc-384 384 0 0 0 0 30 0 0 pcpu-32 32 0 0 0 0 254 0 0 pcpu-4 4 0 0 0 0 254 0 0 UMA Hash 256 0 0 0 0 62 0 0 db:0:pho> show umacache Zone Size Used Free Requests Bucket debugnet_mbuf_packet 2048 0 0 0 0 debugnet_mbuf_cluster 2048 -6144 0 0 0 debugnet_mbuf 256 -10240 0 0 0 vm pgcache 4096 47644 2575 1299394 169 vm pgcache 4096 33757 2948 2031147 169 buf free cache 856 12258 142 13756 16 buffer arena-65 65536 0 0 0 1 buffer arena-61 61440 0 0 0 1 buffer arena-57 57344 0 0 0 1 buffer arena-53 53248 0 0 0 1 buffer arena-49 49152 0 0 0 1 buffer arena-45 45056 0 0 0 1 buffer arena-40 40960 0 0 0 1 buffer arena-36 36864 0 0 0 1 buffer arena-32 32768 8864 18 9420 1 buffer arena-28 28672 0 0 0 1 buffer arena-24 24576 0 0 0 1 buffer arena-20 20480 0 0 0 1 buffer arena-16 16384 6 34 57 1 buffer arena-12 12288 0 0 0 1 buffer arena-81 8192 0 0 0 1 buffer arena-40 4096 0 0 0 2 kstack_cache 16384 316 20 366 2 db:0:pho> acttrace Tracing command mksnap_ffs pid 40197 tid 100252 td 0xfffffe01d1916000 (CPU 8) kdb_enter() at kdb_enter+0x37/frame 0xfffffe01d50a8ed0 vpanic() at vpanic+0x229/frame 0xfffffe01d50a8f30 panic() at panic+0x4f/frame 0xfffffe01d50a8f90 __asan_load4_noabort() at __asan_load4_noabort+0x1db/frame 0xfffffe01d50a9050 cgaccount() at cgaccount+0xfa/frame 0xfffffe01d50a9110 ffs_snapshot() at ffs_snapshot+0x1494/frame 0xfffffe01d50a94f0 ffs_mount() at ffs_mount+0x14e6/frame 0xfffffe01d50a96f0 vfs_domount() at vfs_domount+0xb7f/frame 0xfffffe01d50a99b0 vfs_donmount() at vfs_donmount+0xac7/frame 0xfffffe01d50a9a50 sys_nmount() at sys_nmount+0xa9/frame 0xfffffe01d50a9aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d50a9bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d50a9bf0 --- syscall (378, FreeBSD ELF64, sys_nmount), rip = 0x80038468a, rsp = 0x7fffffffce38, rbp = 0x7fffffffe690 --- Tracing command md10 pid 40193 tid 100150 td 0xfffffe01d083f100 (CPU 6) cpustop_handler() at cpustop_handler+0x32/frame 0xfffffe00816d0dd0 ipi_nmi_handler() at ipi_nmi_handler+0x4f/frame 0xfffffe00816d0e00 trap() at trap+0x78/frame 0xfffffe00816d0f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00816d0f20 --- trap 0x13, rip = 0xffffffff811f5136, rsp = 0xfffffe01d054b9e8, rbp = 0xfffffe01d054b9f0 --- __asan_load4_noabort() at __asan_load4_noabort+0x6/frame 0xfffffe01d054b9f0 uma_zalloc_arg() at uma_zalloc_arg+0x1b1/frame 0xfffffe01d054ba50 mdstart_malloc() at mdstart_malloc+0x569/frame 0xfffffe01d054bb30 md_kthread() at md_kthread+0x276/frame 0xfffffe01d054bbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe01d054bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01d054bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100003 td 0xfffffe012928a300 (CPU 0) cpustop_handler() at cpustop_handler+0x32/frame 0xffffffff8370fad0 ipi_nmi_handler() at ipi_nmi_handler+0x4f/frame 0xffffffff8370fb00 trap() at trap+0x78/frame 0xffffffff8370fc20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff8370fc20 --- trap 0x13, rip = 0xffffffff81822836, rsp = 0xfffffe0128f359f0, rbp = 0xfffffe0128f359f0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0128f359f0 acpi_cpu_idle() at acpi_cpu_idle+0x3b0/frame 0xfffffe0128f35a50 cpu_idle_acpi() at cpu_idle_acpi+0x46/frame 0xfffffe0128f35a70 cpu_idle() at cpu_idle+0xdb/frame 0xfffffe0128f35aa0 sched_idletd() at sched_idletd+0x5c9/frame 0xfffffe0128f35bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f35bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f35bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xfffffe0129289c00 (CPU 1) cpustop_handler() at cpustop_handler+0x32/frame 0xfffffe0081694dd0 ipi_nmi_handler() at ipi_nmi_handler+0x4f/frame 0xfffffe0081694e00 trap() at trap+0x78/frame 0xfffffe0081694f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0081694f20 --- trap 0x13, rip = 0xffffffff81822836, rsp = 0xfffffe0128f309f0, rbp = 0xfffffe0128f309f0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0128f309f0 acpi_cpu_idle() at acpi_cpu_idle+0x3b0/frame 0xfffffe0128f30a50 cpu_idle_acpi() at cpu_idle_acpi+0x46/frame 0xfffffe0128f30a70 cpu_idle() at cpu_idle+0xdb/frame 0xfffffe0128f30aa0 sched_idletd() at sched_idletd+0x5c9/frame 0xfffffe0128f30bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f30bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f30bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffffe0129289500 (CPU 2) cpustop_handler() at cpustop_handler+0x32/frame 0xfffffe00816a0dd0 ipi_nmi_handler() at ipi_nmi_handler+0x4f/frame 0xfffffe00816a0e00 trap() at trap+0x78/frame 0xfffffe00816a0f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00816a0f20 --- trap 0x13, rip = 0xffffffff81822836, rsp = 0xfffffe0128f3f9f0, rbp = 0xfffffe0128f3f9f0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0128f3f9f0 acpi_cpu_idle() at acpi_cpu_idle+0x3b0/frame 0xfffffe0128f3fa50 cpu_idle_acpi() at cpu_idle_acpi+0x46/frame 0xfffffe0128f3fa70 cpu_idle() at cpu_idle+0xdb/frame 0xfffffe0128f3faa0 sched_idletd() at sched_idletd+0x5c9/frame 0xfffffe0128f3fbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f3fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f3fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xfffffe0129288e00 (CPU 3) cpustop_handler() at cpustop_handler+0x32/frame 0xfffffe00816acdd0 ipi_nmi_handler() at ipi_nmi_handler+0x4f/frame 0xfffffe00816ace00 trap() at trap+0x78/frame 0xfffffe00816acf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00816acf20 --- trap 0x13, rip = 0xffffffff81822836, rsp = 0xfffffe0128f3a9f0, rbp = 0xfffffe0128f3a9f0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0128f3a9f0 acpi_cpu_idle() at acpi_cpu_idle+0x3b0/frame 0xfffffe0128f3aa50 cpu_idle_acpi() at cpu_idle_acpi+0x46/frame 0xfffffe0128f3aa70 cpu_idle() at cpu_idle+0xdb/frame 0xfffffe0128f3aaa0 sched_idletd() at sched_idletd+0x5c9/frame 0xfffffe0128f3abb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f3abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f3abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100007 td 0xfffffe0129288700 (CPU 4) cpustop_handler() at cpustop_handler+0x32/frame 0xfffffe00816b8dd0 ipi_nmi_handler() at ipi_nmi_handler+0x4f/frame 0xfffffe00816b8e00 trap() at trap+0x78/frame 0xfffffe00816b8f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00816b8f20 --- trap 0x13, rip = 0xffffffff81822836, rsp = 0xfffffe0128f499f0, rbp = 0xfffffe0128f499f0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0128f499f0 acpi_cpu_idle() at acpi_cpu_idle+0x3b0/frame 0xfffffe0128f49a50 cpu_idle_acpi() at cpu_idle_acpi+0x46/frame 0xfffffe0128f49a70 cpu_idle() at cpu_idle+0xdb/frame 0xfffffe0128f49aa0 sched_idletd() at sched_idletd+0x5c9/frame 0xfffffe0128f49bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f49bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f49bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xfffffe0129288000 (CPU 5) cpustop_handler() at cpustop_handler+0x32/frame 0xfffffe00816c4dd0 ipi_nmi_handler() at ipi_nmi_handler+0x4f/frame 0xfffffe00816c4e00 trap() at trap+0x78/frame 0xfffffe00816c4f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00816c4f20 --- trap 0x13, rip = 0xffffffff81822836, rsp = 0xfffffe0128f449f0, rbp = 0xfffffe0128f449f0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0128f449f0 acpi_cpu_idle() at acpi_cpu_idle+0x3b0/frame 0xfffffe0128f44a50 cpu_idle_acpi() at cpu_idle_acpi+0x46/frame 0xfffffe0128f44a70 cpu_idle() at cpu_idle+0xdb/frame 0xfffffe0128f44aa0 sched_idletd() at sched_idletd+0x5c9/frame 0xfffffe0128f44bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f44bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f44bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xfffffe0129296100 (CPU 7) cpustop_handler() at cpustop_handler+0x32/frame 0xfffffe00816dcdd0 ipi_nmi_handler() at ipi_nmi_handler+0x4f/frame 0xfffffe00816dce00 trap() at trap+0x78/frame 0xfffffe00816dcf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00816dcf20 --- trap 0x13, rip = 0xffffffff81822836, rsp = 0xfffffe0128f4e9f0, rbp = 0xfffffe0128f4e9f0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0128f4e9f0 acpi_cpu_idle() at acpi_cpu_idle+0x3b0/frame 0xfffffe0128f4ea50 cpu_idle_acpi() at cpu_idle_acpi+0x46/frame 0xfffffe0128f4ea70 cpu_idle() at cpu_idle+0xdb/frame 0xfffffe0128f4eaa0 sched_idletd() at sched_idletd+0x5c9/frame 0xfffffe0128f4ebb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f4ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f4ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100012 td 0xfffffe0129295300 (CPU 9) cpustop_handler() at cpustop_handler+0x32/frame 0xfffffe00816f4dd0 ipi_nmi_handler() at ipi_nmi_handler+0x4f/frame 0xfffffe00816f4e00 trap() at trap+0x78/frame 0xfffffe00816f4f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00816f4f20 --- trap 0x13, rip = 0xffffffff81822836, rsp = 0xfffffe0128f589f0, rbp = 0xfffffe0128f589f0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0128f589f0 acpi_cpu_idle() at acpi_cpu_idle+0x3b0/frame 0xfffffe0128f58a50 cpu_idle_acpi() at cpu_idle_acpi+0x46/frame 0xfffffe0128f58a70 cpu_idle() at cpu_idle+0xdb/frame 0xfffffe0128f58aa0 sched_idletd() at sched_idletd+0x5c9/frame 0xfffffe0128f58bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f58bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f58bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xfffffe0129294c00 (CPU 10) cpustop_handler() at cpustop_handler+0x32/frame 0xfffffe0081701dd0 ipi_nmi_handler() at ipi_nmi_handler+0x4f/frame 0xfffffe0081701e00 trap() at trap+0x78/frame 0xfffffe0081701f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0081701f20 --- trap 0x13, rip = 0xffffffff81822836, rsp = 0xfffffe0128f679f0, rbp = 0xfffffe0128f679f0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0128f679f0 acpi_cpu_idle() at acpi_cpu_idle+0x3b0/frame 0xfffffe0128f67a50 cpu_idle_acpi() at cpu_idle_acpi+0x46/frame 0xfffffe0128f67a70 cpu_idle() at cpu_idle+0xdb/frame 0xfffffe0128f67aa0 sched_idletd() at sched_idletd+0x5c9/frame 0xfffffe0128f67bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f67bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f67bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100014 td 0xfffffe0129294500 (CPU 11) cpustop_handler() at cpustop_handler+0x32/frame 0xfffffe008170ddd0 ipi_nmi_handler() at ipi_nmi_handler+0x4f/frame 0xfffffe008170de00 trap() at trap+0x78/frame 0xfffffe008170df20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe008170df20 --- trap 0x13, rip = 0xffffffff81822836, rsp = 0xfffffe0128f629f0, rbp = 0xfffffe0128f629f0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0128f629f0 acpi_cpu_idle() at acpi_cpu_idle+0x3b0/frame 0xfffffe0128f62a50 cpu_idle_acpi() at cpu_idle_acpi+0x46/frame 0xfffffe0128f62a70 cpu_idle() at cpu_idle+0xdb/frame 0xfffffe0128f62aa0 sched_idletd() at sched_idletd+0x5c9/frame 0xfffffe0128f62bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f62bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f62bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db:0:pho> ps pid ppid pgrp uid state wmesg wchan cmd 40197 40182 2531 0 R+ CPU 8 mksnap_ffs 40193 0 0 0 RL CPU 6 [md10] 40183 2531 2531 0 S+ piperd 0xfffffe01e72a42e8 tee 40182 40181 2531 0 S+ wait 0xfffffe01e7303000 sh 40181 2531 2531 0 S+ wait 0xfffffe01d28c4528 sh 40048 11088 11074 0 S nanslp 0xffffffff83582bc1 sleep 39510 0 0 0 DL waiting 0xffffffff8404ebc0 [sctp_iterator] 11091 11074 11074 0 S piperd 0xfffffe01e717dba0 sh 11089 11074 11074 0 S piperd 0xfffffe01e71352e8 awk 11088 11074 11074 0 S wait 0xfffffe01e7160000 sh 11074 1 11074 0 Ss wait 0xfffffe01d4b41000 sh 5779 5770 5779 2006 Ss+ select 0xfffffe01539dbf40 bash 5770 5495 5495 2006 S select 0xfffffe014c882a40 sshd 5495 2477 5495 0 Ss select 0xfffffe014f38a940 sshd 2531 2530 2531 0 S+ wait 0xfffffe01d297da50 sh 2530 2528 2530 0 S+ select 0xfffffe014c849540 bash 2528 2527 2528 0 S+ pause 0xfffffe01d083a5d8 csh 2527 2517 2527 2006 S+ wait 0xfffffe01d1913528 su 2517 2516 2517 2006 Ss+ wait 0xfffffe01d0605528 bash 2516 2513 2513 2006 S select 0xfffffe01539dbe40 sshd 2513 2477 2513 0 Ss select 0xfffffe014ee20440 sshd 2510 2509 2510 0 S+ ttyin 0xfffffe014ef7f0b0 csh 2509 1 2509 0 Ss+ wait 0xfffffe01d28c4000 login 2508 1 2508 0 Ss+ ttyin 0xfffffe014ef7f4b0 getty 2507 1 2507 0 Ss+ ttyin 0xfffffe01535f48b0 getty 2506 1 2506 0 Ss+ ttyin 0xfffffe01535f4cb0 getty 2505 1 2505 0 Ss+ ttyin 0xfffffe01535c50b0 getty 2504 1 2504 0 Ss+ ttyin 0xfffffe01535c54b0 getty 2503 1 2503 0 Ss+ ttyin 0xfffffe01535c58b0 getty 2502 1 2502 0 Ss+ ttyin 0xfffffe01535c5cb0 getty 2501 1 2501 0 Ss+ ttyin 0xfffffe014c4790b0 getty 2500 1 2500 0 Ss+ ttyin 0xfffffe014c4794b0 getty 2489 1 2489 0 Ss select 0xfffffe014ef3cd40 inetd 2486 1 2486 0 Ss nanslp 0xffffffff83582bc9 cron 2477 1 2477 0 Ss select 0xfffffe014ef36940 sshd 2448 1 2448 0 Ss select 0xfffffe014f38a6c0 powerd 2445 1 2445 123 Ss select 0xfffffe014c882ac0 ntpd 2406 1 2406 0 Ss rpcsvc 0xfffffe01d4cd4e60 NLM: master 2403 1 2403 0 Ss select 0xfffffe014c849740 rpc.statd 2381 1 2381 0 Ss select 0xfffffe014f38a7c0 rpcbind 2364 1 2364 0 Ss nanslp 0xffffffff83582bca watchdogd 2349 1 2349 0 Ss select 0xfffffe0153562640 syslogd 2170 1 2170 0 Ss select 0xfffffe01d272bdc0 devd 2165 1 2165 65 Ss select 0xfffffe014ee204c0 dhclient 2089 1 2089 0 Ss select 0xfffffe014ee20540 dhclient 2086 1 2086 0 Ss select 0xfffffe014c849940 dhclient 33 0 0 0 DL vlruwt 0xfffffe01d0605a50 [vnlru] 32 0 0 0 DL syncer 0xffffffff836a5dc0 [syncer] 31 0 0 0 DL (threaded) [bufdaemon] 100131 D qsleep 0xffffffff836a3d20 [bufdaemon] 100132 D - 0xffffffff82a11cc0 [bufspacedaemon-0] 100133 D - 0xffffffff82a19f80 [bufspacedaemon-1] 100134 D - 0xffffffff82a22240 [bufspacedaemon-2] 100135 D - 0xffffffff82a2a500 [bufspacedaemon-3] 100136 D - 0xffffffff82a327c0 [bufspacedaemon-4] 100137 D - 0xffffffff82a3aa80 [bufspacedaemon-5] 100138 D - 0xffffffff82a42d40 [bufspacedaemon-6] 100152 D sdflush 0xfffffe01d0c72ce8 [/ worker] 100160 D sdflush 0xfffffe01d27cdce8 [/tmp worker] 100366 D sdflush 0xfffffe01d27cd4e8 [/mnt worker] 30 0 0 0 DL psleep 0xffffffff836d7800 [vmdaemon] 29 0 0 0 DL (threaded) [pagedaemon] 100129 D psleep 0xffffffff836cb378 [dom0] 100139 D launds 0xffffffff836cb384 [laundry: dom0] 100140 D umarcl 0xffffffff81646260 [uma] 28 0 0 0 DL idle 0xfffffe0153170000 [enc_daemon1] 27 0 0 0 DL idle 0xfffffe0153170528 [enc_daemon0] 26 0 0 0 DL - 0xffffffff83339c80 [rand_harvestq] 25 0 0 0 DL - 0xffffffff836a1e40 [soaiod4] 24 0 0 0 DL - 0xffffffff836a1e40 [soaiod3] 23 0 0 0 DL - 0xffffffff836a1e40 [soaiod2] 22 0 0 0 DL - 0xffffffff836a1e40 [soaiod1] 21 0 0 0 DL (threaded) [usb] 100098 D - 0xfffffe014ec5d440 [usbus0] 100099 D - 0xfffffe014ec5d498 [usbus0] 100100 D - 0xfffffe014ec5d4f0 [usbus0] 100101 D - 0xfffffe014ec5d548 [usbus0] 100102 D - 0xfffffe014ec5d5a0 [usbus0] 100104 D - 0xfffffe014c8becf0 [usbus1] 100105 D - 0xfffffe014c8bed48 [usbus1] 100106 D - 0xfffffe014c8beda0 [usbus1] 100107 D - 0xfffffe014c8bedf8 [usbus1] 100108 D - 0xfffffe014c8bee50 [usbus1] 100109 D - 0xfffffe014ef87cf0 [usbus2] 100110 D - 0xfffffe014ef87d48 [usbus2] 100111 D - 0xfffffe014ef87da0 [usbus2] 100112 D - 0xfffffe014ef87df8 [usbus2] 100113 D - 0xfffffe014ef87e50 [usbus2] 20 0 0 0 RL (threaded) [cam] 100092 D - 0xffffffff831bfc40 [doneq0] 100093 RunQ [doneq1] 100094 D - 0xffffffff831bfd40 [doneq2] 100126 D - 0xffffffff831bfac0 [scanner] 19 0 0 0 DL crypto_ 0xfffffe014c428c50 [crypto returns 11] 18 0 0 0 DL crypto_ 0xfffffe014c428bf0 [crypto returns 10] 17 0 0 0 DL crypto_ 0xfffffe014c428b90 [crypto returns 9] 16 0 0 0 DL crypto_ 0xfffffe014c428b30 [crypto returns 8] 15 0 0 0 DL crypto_ 0xfffffe014c428ad0 [crypto returns 7] 9 0 0 0 DL crypto_ 0xfffffe014c428a70 [crypto returns 6] 8 0 0 0 DL crypto_ 0xfffffe014c428a10 [crypto returns 5] 7 0 0 0 DL crypto_ 0xfffffe014c4289b0 [crypto returns 4] 6 0 0 0 DL crypto_ 0xfffffe014c428950 [crypto returns 3] 5 0 0 0 DL crypto_ 0xfffffe014c4288f0 [crypto returns 2] 4 0 0 0 DL crypto_ 0xfffffe014c428890 [crypto returns 1] 3 0 0 0 DL crypto_ 0xfffffe014c428830 [crypto returns 0] 2 0 0 0 DL crypto_ 0xffffffff836c55e0 [crypto] 14 0 0 0 DL seqstat 0xfffffe01293f4c88 [sequencer 00] 13 0 0 0 DL (threaded) [geom] 100063 D - 0xffffffff83556200 [g_event] 100064 D - 0xffffffff83556240 [g_up] 100065 D - 0xffffffff83556280 [g_down] 12 0 0 0 WL (threaded) [intr] 100042 I [swi6: Giant taskq] 100044 I [swi5: fast taskq] 100047 I [swi6: task queue] 100048 I [swi3: vm] 100049 I [swi1: netisr 0] 100050 I [swi4: clock (0)] 100051 I [swi4: clock (1)] 100052 I [swi4: clock (2)] 100053 I [swi4: clock (3)] 100054 I [swi4: clock (4)] 100055 I [swi4: clock (5)] 100056 I [swi4: clock (6)] 100057 I [swi4: clock (7)] 100058 I [swi4: clock (8)] 100059 I [swi4: clock (9)] 100060 I [swi4: clock (10)] 100061 I [swi4: clock (11)] 100095 I [swi4: apei] 100096 I [irq56: ahci0] 100097 I [irq19: xhci0] 100103 I [irq18: ehci0 ehci1+] 100114 I [irq71: ahci1] 100115 I [swi0: uart uart] 100195 I [irq72: ioat0] 100196 I [irq73: ioat1] 100197 I [irq74: ioat2] 100198 I [irq75: ioat3] 100199 I [irq76: ioat4] 100200 I [irq77: ioat5] 100201 I [irq78: ioat6] 100202 I [irq79: ioat7] 11 0 0 0 RL (threaded) [idle] 100003 Run CPU 0 [idle: cpu0] 100004 Run CPU 1 [idle: cpu1] 100005 Run CPU 2 [idle: cpu2] 100006 Run CPU 3 [idle: cpu3] 100007 Run CPU 4 [idle: cpu4] 100008 Run CPU 5 [idle: cpu5] 100009 CanRun [idle: cpu6] 100010 Run CPU 7 [idle: cpu7] 100011 CanRun [idle: cpu8] 100012 Run CPU 9 [idle: cpu9] 100013 Run CPU 10 [idle: cpu10] 100014 Run CPU 11 [idle: cpu11] 1 0 1 0 SLs wait 0xfffffe0129287528 [init] 10 0 0 0 DL audit_w 0xffffffff836c63c0 [audit] 0 0 0 0 DLs (threaded) [kernel] 100000 D swapin 0xffffffff83557100 [swapper] 100015 D - 0xfffffe0129154d00 [softirq_0] 100016 D - 0xfffffe0129154c00 [softirq_1] 100017 D - 0xfffffe0129154b00 [softirq_2] 100018 D - 0xfffffe0129154a00 [softirq_3] 100019 D - 0xfffffe0129154900 [softirq_4] 100020 D - 0xfffffe0129154800 [softirq_5] 100021 D - 0xfffffe0129154700 [softirq_6] 100022 D - 0xfffffe0129154600 [softirq_7] 100023 D - 0xfffffe0129154500 [softirq_8] 100024 D - 0xfffffe0129154400 [softirq_9] 100025 D - 0xfffffe0129154300 [softirq_10] 100026 D - 0xfffffe0129154200 [softirq_11] 100027 D - 0xfffffe0129154100 [if_io_tqg_0] 100028 D - 0xfffffe0129154000 [if_io_tqg_1] 100029 D - 0xfffffe0129336e00 [if_io_tqg_2] 100030 D - 0xfffffe0129336d00 [if_io_tqg_3] 100031 D - 0xfffffe0129336c00 [if_io_tqg_4] 100032 D - 0xfffffe0129336b00 [if_io_tqg_5] 100033 D - 0xfffffe0129336a00 [if_io_tqg_6] 100034 D - 0xfffffe0129336900 [if_io_tqg_7] 100035 D - 0xfffffe0129336800 [if_io_tqg_8] 100036 D - 0xfffffe0129336700 [if_io_tqg_9] 100037 D - 0xfffffe0129336600 [if_io_tqg_10] 100038 D - 0xfffffe0129336500 [if_io_tqg_11] 100039 D - 0xfffffe0129336400 [if_config_tqg_0] 100040 D - 0xfffffe0129336300 [in6m_free taskq] 100041 D - 0xfffffe0129336200 [aiod_kick taskq] 100043 D - 0xfffffe0129336000 [thread taskq] 100045 D - 0xfffffe0129342d00 [inm_free taskq] 100046 D - 0xfffffe0129342c00 [kqueue_ctx taskq] 100062 D - 0xfffffe0129371e00 [firmware taskq] 100067 D - 0xfffffe0129371d00 [crypto_0] 100068 D - 0xfffffe0129371d00 [crypto_1] 100069 D - 0xfffffe0129371d00 [crypto_2] 100070 D - 0xfffffe0129371d00 [crypto_3] 100071 D - 0xfffffe0129371d00 [crypto_4] 100072 D - 0xfffffe0129371d00 [crypto_5] 100073 D - 0xfffffe0129371d00 [crypto_6] 100074 D - 0xfffffe0129371d00 [crypto_7] 100075 D - 0xfffffe0129371d00 [crypto_8] 100076 D - 0xfffffe0129371d00 [crypto_9] 100077 D - 0xfffffe0129371d00 [crypto_10] 100078 D - 0xfffffe0129371d00 [crypto_11] 100116 D - 0xfffffe014f284700 [mca taskq] 100121 D - 0xfffffe0129371a00 [acpi_task_0] 100122 D - 0xfffffe0129371a00 [acpi_task_1] 100123 D - 0xfffffe0129371a00 [acpi_task_2] 100125 D - 0xfffffe0129371c00 [CAM taskq] 100365 D - 0xfffffe01e7e08300 [/mnt trim] db:0:pho> allt Tracing command mksnap_ffs pid 40197 tid 100252 td 0xfffffe01d1916000 (CPU 8) kdb_enter() at kdb_enter+0x37/frame 0xfffffe01d50a8ed0 vpanic() at vpanic+0x229/frame 0xfffffe01d50a8f30 panic() at panic+0x4f/frame 0xfffffe01d50a8f90 __asan_load4_noabort() at __asan_load4_noabort+0x1db/frame 0xfffffe01d50a9050 cgaccount() at cgaccount+0xfa/frame 0xfffffe01d50a9110 ffs_snapshot() at ffs_snapshot+0x1494/frame 0xfffffe01d50a94f0 ffs_mount() at ffs_mount+0x14e6/frame 0xfffffe01d50a96f0 vfs_domount() at vfs_domount+0xb7f/frame 0xfffffe01d50a99b0 vfs_donmount() at vfs_donmount+0xac7/frame 0xfffffe01d50a9a50 sys_nmount() at sys_nmount+0xa9/frame 0xfffffe01d50a9aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d50a9bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d50a9bf0 --- syscall (378, FreeBSD ELF64, sys_nmount), rip = 0x80038468a, rsp = 0x7fffffffce38, rbp = 0x7fffffffe690 --- Tracing command md10 pid 40193 tid 100150 td 0xfffffe01d083f100 (CPU 6) cpustop_handler() at cpustop_handler+0x32/frame 0xfffffe00816d0dd0 ipi_nmi_handler() at ipi_nmi_handler+0x4f/frame 0xfffffe00816d0e00 trap() at trap+0x78/frame 0xfffffe00816d0f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00816d0f20 --- trap 0x13, rip = 0xffffffff811f5136, rsp = 0xfffffe01d054b9e8, rbp = 0xfffffe01d054b9f0 --- __asan_load4_noabort() at __asan_load4_noabort+0x6/frame 0xfffffe01d054b9f0 uma_zalloc_arg() at uma_zalloc_arg+0x1b1/frame 0xfffffe01d054ba50 mdstart_malloc() at mdstart_malloc+0x569/frame 0xfffffe01d054bb30 md_kthread() at md_kthread+0x276/frame 0xfffffe01d054bbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe01d054bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01d054bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command tee pid 40183 tid 100210 td 0xfffffe01d2c63c00 sched_switch() at sched_switch+0x90f/frame 0xfffffe01537d3750 mi_switch() at mi_switch+0x268/frame 0xfffffe01537d3780 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01537d37d0 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01537d3820 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01537d3840 _sleep() at _sleep+0x324/frame 0xfffffe01537d38f0 pipe_read() at pipe_read+0x79b/frame 0xfffffe01537d39b0 dofileread() at dofileread+0x112/frame 0xfffffe01537d3a20 sys_read() at sys_read+0x143/frame 0xfffffe01537d3aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01537d3bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01537d3bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800385e4a, rsp = 0x7fffffffe648, rbp = 0x7fffffffe6a0 --- Tracing command sh pid 40182 tid 100281 td 0xfffffe01e711d000 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d5144650 mi_switch() at mi_switch+0x268/frame 0xfffffe01d5144680 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d51446d0 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01d5144720 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01d5144740 _sleep() at _sleep+0x324/frame 0xfffffe01d51447f0 kern_wait6() at kern_wait6+0x5df/frame 0xfffffe01d51448a0 sys_wait4() at sys_wait4+0xa3/frame 0xfffffe01d5144aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d5144bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d5144bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003e602a, rsp = 0x7fffffffe2e8, rbp = 0x7fffffffe390 --- Tracing command sh pid 40181 tid 100167 td 0xfffffe01d07d4500 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d05bf650 mi_switch() at mi_switch+0x268/frame 0xfffffe01d05bf680 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d05bf6d0 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01d05bf720 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01d05bf740 _sleep() at _sleep+0x324/frame 0xfffffe01d05bf7f0 kern_wait6() at kern_wait6+0x5df/frame 0xfffffe01d05bf8a0 sys_wait4() at sys_wait4+0xa3/frame 0xfffffe01d05bfaa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d05bfbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d05bfbf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003e602a, rsp = 0x7fffffffa328, rbp = 0x7fffffffa3d0 --- Tracing command sleep pid 40048 tid 100303 td 0xfffffe01e7527800 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d519e7e0 mi_switch() at mi_switch+0x268/frame 0xfffffe01d519e810 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d519e860 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01d519e8b0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x16/frame 0xfffffe01d519e900 _sleep() at _sleep+0x2ff/frame 0xfffffe01d519e9b0 kern_clock_nanosleep() at kern_clock_nanosleep+0x2a1/frame 0xfffffe01d519ea50 sys_nanosleep() at sys_nanosleep+0x56/frame 0xfffffe01d519eaa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d519ebf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d519ebf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x800385d6a, rsp = 0x7fffffffe648, rbp = 0x7fffffffe690 --- Tracing command sctp_iterator pid 39510 tid 100305 td 0xfffffe01e7538a00 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d51b2a60 mi_switch() at mi_switch+0x268/frame 0xfffffe01d51b2a90 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d51b2ae0 _sleep() at _sleep+0x31a/frame 0xfffffe01d51b2b90 sctp_iterator_thread() at sctp_iterator_thread+0x59/frame 0xfffffe01d51b2bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe01d51b2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01d51b2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sh pid 11091 tid 100185 td 0xfffffe01d07fec00 sched_switch() at sched_switch+0x90f/frame 0xfffffe015374d750 mi_switch() at mi_switch+0x268/frame 0xfffffe015374d780 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe015374d7d0 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe015374d820 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe015374d840 _sleep() at _sleep+0x324/frame 0xfffffe015374d8f0 pipe_read() at pipe_read+0x79b/frame 0xfffffe015374d9b0 dofileread() at dofileread+0x112/frame 0xfffffe015374da20 sys_read() at sys_read+0x143/frame 0xfffffe015374daa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe015374dbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe015374dbf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8003e5e4a, rsp = 0x7fffffffdd78, rbp = 0x7fffffffe2a0 --- Tracing command awk pid 11089 tid 100204 td 0xfffffe01d07d3000 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d05f6750 mi_switch() at mi_switch+0x268/frame 0xfffffe01d05f6780 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d05f67d0 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01d05f6820 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01d05f6840 _sleep() at _sleep+0x324/frame 0xfffffe01d05f68f0 pipe_read() at pipe_read+0x79b/frame 0xfffffe01d05f69b0 dofileread() at dofileread+0x112/frame 0xfffffe01d05f6a20 sys_read() at sys_read+0x143/frame 0xfffffe01d05f6aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d05f6bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d05f6bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8003eae4a, rsp = 0x7fffffffe2b8, rbp = 0x7fffffffe2d0 --- Tracing command sh pid 11088 tid 100289 td 0xfffffe01e7294c00 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d516c650 mi_switch() at mi_switch+0x268/frame 0xfffffe01d516c680 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d516c6d0 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01d516c720 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01d516c740 _sleep() at _sleep+0x324/frame 0xfffffe01d516c7f0 kern_wait6() at kern_wait6+0x5df/frame 0xfffffe01d516c8a0 sys_wait4() at sys_wait4+0xa3/frame 0xfffffe01d516caa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d516cbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d516cbf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003e602a, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe260 --- Tracing command sh pid 11074 tid 100235 td 0xfffffe01d2949e00 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d500e650 mi_switch() at mi_switch+0x268/frame 0xfffffe01d500e680 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d500e6d0 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01d500e720 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01d500e740 _sleep() at _sleep+0x324/frame 0xfffffe01d500e7f0 kern_wait6() at kern_wait6+0x5df/frame 0xfffffe01d500e8a0 sys_wait4() at sys_wait4+0xa3/frame 0xfffffe01d500eaa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d500ebf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d500ebf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003e602a, rsp = 0x7fffffffe478, rbp = 0x7fffffffe520 --- Tracing command bash pid 5779 tid 100154 td 0xfffffe014f23a000 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d055f640 mi_switch() at mi_switch+0x268/frame 0xfffffe01d055f670 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d055f6c0 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01d055f710 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01d055f730 _cv_wait_sig() at _cv_wait_sig+0x205/frame 0xfffffe01d055f7a0 seltdwait() at seltdwait+0x9d/frame 0xfffffe01d055f7e0 kern_select() at kern_select+0xa13/frame 0xfffffe01d055fa20 sys_pselect() at sys_pselect+0x1a0/frame 0xfffffe01d055faa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d055fbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d055fbf0 --- syscall (522, FreeBSD ELF64, sys_pselect), rip = 0x80050ee0a, rsp = 0x7fffffffd6c8, rbp = 0x7fffffffd790 --- Tracing command sshd pid 5770 tid 100242 td 0xfffffe0153929700 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d05ce660 mi_switch() at mi_switch+0x268/frame 0xfffffe01d05ce690 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d05ce6e0 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01d05ce730 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01d05ce750 _cv_wait_sig() at _cv_wait_sig+0x205/frame 0xfffffe01d05ce7c0 seltdwait() at seltdwait+0x9d/frame 0xfffffe01d05ce800 kern_select() at kern_select+0xa13/frame 0xfffffe01d05cea40 sys_select() at sys_select+0x92/frame 0xfffffe01d05ceaa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d05cebf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d05cebf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800872eca, rsp = 0x7fffffffa4e8, rbp = 0x7fffffffa520 --- Tracing command sshd pid 5495 tid 100215 td 0xfffffe01d49f8c00 sched_switch() at sched_switch+0x90f/frame 0xfffffe01537e76e0 mi_switch() at mi_switch+0x268/frame 0xfffffe01537e7710 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01537e7760 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01537e77b0 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01537e77d0 _cv_wait_sig() at _cv_wait_sig+0x205/frame 0xfffffe01537e7840 seltdwait() at seltdwait+0x9d/frame 0xfffffe01537e7880 kern_poll() at kern_poll+0x57b/frame 0xfffffe01537e7a50 sys_poll() at sys_poll+0x94/frame 0xfffffe01537e7aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01537e7bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01537e7bf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x800872dca, rsp = 0x7fffffffe548, rbp = 0x7fffffffe580 --- Tracing command sh pid 2531 tid 100182 td 0xfffffe0129344000 sched_switch() at sched_switch+0x90f/frame 0xfffffe0153739650 mi_switch() at mi_switch+0x268/frame 0xfffffe0153739680 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01537396d0 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe0153739720 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe0153739740 _sleep() at _sleep+0x324/frame 0xfffffe01537397f0 kern_wait6() at kern_wait6+0x5df/frame 0xfffffe01537398a0 sys_wait4() at sys_wait4+0xa3/frame 0xfffffe0153739aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe0153739bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0153739bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003e602a, rsp = 0x7fffffffa678, rbp = 0x7fffffffa720 --- Tracing command bash pid 2530 tid 100178 td 0xfffffe01d07d3700 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d05fb640 mi_switch() at mi_switch+0x268/frame 0xfffffe01d05fb670 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d05fb6c0 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01d05fb710 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01d05fb730 _cv_wait_sig() at _cv_wait_sig+0x205/frame 0xfffffe01d05fb7a0 seltdwait() at seltdwait+0x9d/frame 0xfffffe01d05fb7e0 kern_select() at kern_select+0xa13/frame 0xfffffe01d05fba20 sys_pselect() at sys_pselect+0x1a0/frame 0xfffffe01d05fbaa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d05fbbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d05fbbf0 --- syscall (522, FreeBSD ELF64, sys_pselect), rip = 0x80050ee0a, rsp = 0x7fffffffd588, rbp = 0x7fffffffd650 --- Tracing command csh pid 2528 tid 100220 td 0xfffffe01d49f1a00 sched_switch() at sched_switch+0x90f/frame 0xfffffe01537a6860 mi_switch() at mi_switch+0x268/frame 0xfffffe01537a6890 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01537a68e0 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01537a6930 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01537a6950 _sleep() at _sleep+0x324/frame 0xfffffe01537a6a00 kern_sigsuspend() at kern_sigsuspend+0x15b/frame 0xfffffe01537a6a70 sys_sigsuspend() at sys_sigsuspend+0x53/frame 0xfffffe01537a6aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01537a6bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01537a6bf0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x80047cf8a, rsp = 0x7fffffffdf88, rbp = 0x7fffffffe010 --- Tracing command su pid 2527 tid 100155 td 0xfffffe01d1919100 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d055a650 mi_switch() at mi_switch+0x268/frame 0xfffffe01d055a680 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d055a6d0 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01d055a720 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01d055a740 _sleep() at _sleep+0x324/frame 0xfffffe01d055a7f0 kern_wait6() at kern_wait6+0x5df/frame 0xfffffe01d055a8a0 sys_wait4() at sys_wait4+0xa3/frame 0xfffffe01d055aaa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d055abf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d055abf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003ce02a, rsp = 0x7fffffffe5a8, rbp = 0x7fffffffeae0 --- Tracing command bash pid 2517 tid 100144 td 0xfffffe015392ac00 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d0519650 mi_switch() at mi_switch+0x268/frame 0xfffffe01d0519680 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d05196d0 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01d0519720 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01d0519740 _sleep() at _sleep+0x324/frame 0xfffffe01d05197f0 kern_wait6() at kern_wait6+0x5df/frame 0xfffffe01d05198a0 sys_wait4() at sys_wait4+0xa3/frame 0xfffffe01d0519aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d0519bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d0519bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80050f02a, rsp = 0x7fffffffe718, rbp = 0x7fffffffe770 --- Tracing command sshd pid 2516 tid 100148 td 0xfffffe01d0800100 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d051e660 mi_switch() at mi_switch+0x268/frame 0xfffffe01d051e690 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d051e6e0 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01d051e730 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01d051e750 _cv_wait_sig() at _cv_wait_sig+0x205/frame 0xfffffe01d051e7c0 seltdwait() at seltdwait+0x9d/frame 0xfffffe01d051e800 kern_select() at kern_select+0xa13/frame 0xfffffe01d051ea40 sys_select() at sys_select+0x92/frame 0xfffffe01d051eaa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d051ebf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d051ebf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800872eca, rsp = 0x7fffffffa4e8, rbp = 0x7fffffffa520 --- Tracing command sshd pid 2513 tid 100188 td 0xfffffe01d294b300 sched_switch() at sched_switch+0x90f/frame 0xfffffe01537576e0 mi_switch() at mi_switch+0x268/frame 0xfffffe0153757710 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0153757760 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01537577b0 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01537577d0 _cv_wait_sig() at _cv_wait_sig+0x205/frame 0xfffffe0153757840 seltdwait() at seltdwait+0x9d/frame 0xfffffe0153757880 kern_poll() at kern_poll+0x57b/frame 0xfffffe0153757a50 sys_poll() at sys_poll+0x94/frame 0xfffffe0153757aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe0153757bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0153757bf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x800872dca, rsp = 0x7fffffffe548, rbp = 0x7fffffffe580 --- Tracing command csh pid 2510 tid 100161 td 0xfffffe01d07d5a00 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d05286e0 mi_switch() at mi_switch+0x268/frame 0xfffffe01d0528710 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d0528760 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01d05287b0 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01d05287d0 _cv_wait_sig() at _cv_wait_sig+0x205/frame 0xfffffe01d0528840 tty_wait() at tty_wait+0x6e/frame 0xfffffe01d0528880 ttydisc_read() at ttydisc_read+0x2a2/frame 0xfffffe01d0528900 ttydev_read() at ttydev_read+0x6b/frame 0xfffffe01d0528930 devfs_read_f() at devfs_read_f+0x16f/frame 0xfffffe01d05289b0 dofileread() at dofileread+0x112/frame 0xfffffe01d0528a20 sys_read() at sys_read+0x143/frame 0xfffffe01d0528aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d0528bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d0528bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80047ce4a, rsp = 0x7fffffffe178, rbp = 0x7fffffffe1a0 --- Tracing command login pid 2509 tid 100169 td 0xfffffe01d07d3e00 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d05ba650 mi_switch() at mi_switch+0x268/frame 0xfffffe01d05ba680 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d05ba6d0 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01d05ba720 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01d05ba740 _sleep() at _sleep+0x324/frame 0xfffffe01d05ba7f0 kern_wait6() at kern_wait6+0x5df/frame 0xfffffe01d05ba8a0 sys_wait4() at sys_wait4+0xa3/frame 0xfffffe01d05baaa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d05babf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d05babf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003d002a, rsp = 0x7fffffffeae8, rbp = 0x7fffffffeca0 --- Tracing command getty pid 2508 tid 100177 td 0xfffffe01d294c100 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d05f16e0 mi_switch() at mi_switch+0x268/frame 0xfffffe01d05f1710 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d05f1760 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01d05f17b0 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01d05f17d0 _cv_wait_sig() at _cv_wait_sig+0x205/frame 0xfffffe01d05f1840 tty_wait() at tty_wait+0x6e/frame 0xfffffe01d05f1880 ttydisc_read() at ttydisc_read+0x2a2/frame 0xfffffe01d05f1900 ttydev_read() at ttydev_read+0x6b/frame 0xfffffe01d05f1930 devfs_read_f() at devfs_read_f+0x16f/frame 0xfffffe01d05f19b0 dofileread() at dofileread+0x112/frame 0xfffffe01d05f1a20 sys_read() at sys_read+0x143/frame 0xfffffe01d05f1aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d05f1bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d05f1bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8003a2e4a, rsp = 0x7fffffffeb68, rbp = 0x7fffffffebb0 --- Tracing command getty pid 2507 tid 100203 td 0xfffffe01d49f2800 sched_switch() at sched_switch+0x90f/frame 0xfffffe01537b56e0 mi_switch() at mi_switch+0x268/frame 0xfffffe01537b5710 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01537b5760 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01537b57b0 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01537b57d0 _cv_wait_sig() at _cv_wait_sig+0x205/frame 0xfffffe01537b5840 tty_wait() at tty_wait+0x6e/frame 0xfffffe01537b5880 ttydisc_read() at ttydisc_read+0x2a2/frame 0xfffffe01537b5900 ttydev_read() at ttydev_read+0x6b/frame 0xfffffe01537b5930 devfs_read_f() at devfs_read_f+0x16f/frame 0xfffffe01537b59b0 dofileread() at dofileread+0x112/frame 0xfffffe01537b5a20 sys_read() at sys_read+0x143/frame 0xfffffe01537b5aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01537b5bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01537b5bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8003a2e4a, rsp = 0x7fffffffeb68, rbp = 0x7fffffffebb0 --- Tracing command getty pid 2506 tid 100171 td 0xfffffe015393c700 sched_switch() at sched_switch+0x90f/frame 0xfffffe0153d236e0 mi_switch() at mi_switch+0x268/frame 0xfffffe0153d23710 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0153d23760 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe0153d237b0 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe0153d237d0 _cv_wait_sig() at _cv_wait_sig+0x205/frame 0xfffffe0153d23840 tty_wait() at tty_wait+0x6e/frame 0xfffffe0153d23880 ttydisc_read() at ttydisc_read+0x2a2/frame 0xfffffe0153d23900 ttydev_read() at ttydev_read+0x6b/frame 0xfffffe0153d23930 devfs_read_f() at devfs_read_f+0x16f/frame 0xfffffe0153d239b0 dofileread() at dofileread+0x112/frame 0xfffffe0153d23a20 sys_read() at sys_read+0x143/frame 0xfffffe0153d23aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe0153d23bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0153d23bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8003a2e4a, rsp = 0x7fffffffeb68, rbp = 0x7fffffffebb0 --- Tracing command getty pid 2505 tid 100193 td 0xfffffe01d18f8000 sched_switch() at sched_switch+0x90f/frame 0xfffffe01537776e0 mi_switch() at mi_switch+0x268/frame 0xfffffe0153777710 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0153777760 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01537777b0 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01537777d0 _cv_wait_sig() at _cv_wait_sig+0x205/frame 0xfffffe0153777840 tty_wait() at tty_wait+0x6e/frame 0xfffffe0153777880 ttydisc_read() at ttydisc_read+0x2a2/frame 0xfffffe0153777900 ttydev_read() at ttydev_read+0x6b/frame 0xfffffe0153777930 devfs_read_f() at devfs_read_f+0x16f/frame 0xfffffe01537779b0 dofileread() at dofileread+0x112/frame 0xfffffe0153777a20 sys_read() at sys_read+0x143/frame 0xfffffe0153777aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe0153777bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0153777bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8003a2e4a, rsp = 0x7fffffffeb68, rbp = 0x7fffffffebb0 --- Tracing command getty pid 2504 tid 100168 td 0xfffffe015392a500 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d05146e0 mi_switch() at mi_switch+0x268/frame 0xfffffe01d0514710 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d0514760 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01d05147b0 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01d05147d0 _cv_wait_sig() at _cv_wait_sig+0x205/frame 0xfffffe01d0514840 tty_wait() at tty_wait+0x6e/frame 0xfffffe01d0514880 ttydisc_read() at ttydisc_read+0x2a2/frame 0xfffffe01d0514900 ttydev_read() at ttydev_read+0x6b/frame 0xfffffe01d0514930 devfs_read_f() at devfs_read_f+0x16f/frame 0xfffffe01d05149b0 dofileread() at dofileread+0x112/frame 0xfffffe01d0514a20 sys_read() at sys_read+0x143/frame 0xfffffe01d0514aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d0514bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d0514bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8003a2e4a, rsp = 0x7fffffffeb68, rbp = 0x7fffffffebb0 --- Tracing command getty pid 2503 tid 100183 td 0xfffffe01d18f9c00 sched_switch() at sched_switch+0x90f/frame 0xfffffe01537436e0 mi_switch() at mi_switch+0x268/frame 0xfffffe0153743710 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0153743760 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01537437b0 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01537437d0 _cv_wait_sig() at _cv_wait_sig+0x205/frame 0xfffffe0153743840 tty_wait() at tty_wait+0x6e/frame 0xfffffe0153743880 ttydisc_read() at ttydisc_read+0x2a2/frame 0xfffffe0153743900 ttydev_read() at ttydev_read+0x6b/frame 0xfffffe0153743930 devfs_read_f() at devfs_read_f+0x16f/frame 0xfffffe01537439b0 dofileread() at dofileread+0x112/frame 0xfffffe0153743a20 sys_read() at sys_read+0x143/frame 0xfffffe0153743aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe0153743bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0153743bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8003a2e4a, rsp = 0x7fffffffeb68, rbp = 0x7fffffffebb0 --- Tracing command getty pid 2502 tid 100209 td 0xfffffe01d2c64300 sched_switch() at sched_switch+0x90f/frame 0xfffffe01537c46e0 mi_switch() at mi_switch+0x268/frame 0xfffffe01537c4710 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01537c4760 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01537c47b0 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01537c47d0 _cv_wait_sig() at _cv_wait_sig+0x205/frame 0xfffffe01537c4840 tty_wait() at tty_wait+0x6e/frame 0xfffffe01537c4880 ttydisc_read() at ttydisc_read+0x2a2/frame 0xfffffe01537c4900 ttydev_read() at ttydev_read+0x6b/frame 0xfffffe01537c4930 devfs_read_f() at devfs_read_f+0x16f/frame 0xfffffe01537c49b0 dofileread() at dofileread+0x112/frame 0xfffffe01537c4a20 sys_read() at sys_read+0x143/frame 0xfffffe01537c4aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01537c4bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01537c4bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8003a2e4a, rsp = 0x7fffffffeb68, rbp = 0x7fffffffebb0 --- Tracing command getty pid 2501 tid 100172 td 0xfffffe0129344700 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d05c46e0 mi_switch() at mi_switch+0x268/frame 0xfffffe01d05c4710 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d05c4760 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01d05c47b0 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01d05c47d0 _cv_wait_sig() at _cv_wait_sig+0x205/frame 0xfffffe01d05c4840 tty_wait() at tty_wait+0x6e/frame 0xfffffe01d05c4880 ttydisc_read() at ttydisc_read+0x2a2/frame 0xfffffe01d05c4900 ttydev_read() at ttydev_read+0x6b/frame 0xfffffe01d05c4930 devfs_read_f() at devfs_read_f+0x16f/frame 0xfffffe01d05c49b0 dofileread() at dofileread+0x112/frame 0xfffffe01d05c4a20 sys_read() at sys_read+0x143/frame 0xfffffe01d05c4aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d05c4bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d05c4bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8003a2e4a, rsp = 0x7fffffffeb68, rbp = 0x7fffffffebb0 --- Tracing command getty pid 2500 tid 100143 td 0xfffffe0153161000 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d050f6e0 mi_switch() at mi_switch+0x268/frame 0xfffffe01d050f710 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d050f760 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01d050f7b0 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01d050f7d0 _cv_wait_sig() at _cv_wait_sig+0x205/frame 0xfffffe01d050f840 tty_wait() at tty_wait+0x6e/frame 0xfffffe01d050f880 ttydisc_read() at ttydisc_read+0x2a2/frame 0xfffffe01d050f900 ttydev_read() at ttydev_read+0x6b/frame 0xfffffe01d050f930 devfs_read_f() at devfs_read_f+0x16f/frame 0xfffffe01d050f9b0 dofileread() at dofileread+0x112/frame 0xfffffe01d050fa20 sys_read() at sys_read+0x143/frame 0xfffffe01d050faa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d050fbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d050fbf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8003a2e4a, rsp = 0x7fffffffeb68, rbp = 0x7fffffffebb0 --- Tracing command inetd pid 2489 tid 100164 td 0xfffffe0129345500 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d05a6660 mi_switch() at mi_switch+0x268/frame 0xfffffe01d05a6690 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d05a66e0 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01d05a6730 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01d05a6750 _cv_wait_sig() at _cv_wait_sig+0x205/frame 0xfffffe01d05a67c0 seltdwait() at seltdwait+0x9d/frame 0xfffffe01d05a6800 kern_select() at kern_select+0xa13/frame 0xfffffe01d05a6a40 sys_select() at sys_select+0x92/frame 0xfffffe01d05a6aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d05a6bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d05a6bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8003bdeca, rsp = 0x7fffffffdd88, rbp = 0x7fffffffec20 --- Tracing command cron pid 2486 tid 100181 td 0xfffffe01d07ff300 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d05907e0 mi_switch() at mi_switch+0x268/frame 0xfffffe01d0590810 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d0590860 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01d05908b0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x16/frame 0xfffffe01d0590900 _sleep() at _sleep+0x2ff/frame 0xfffffe01d05909b0 kern_clock_nanosleep() at kern_clock_nanosleep+0x2a1/frame 0xfffffe01d0590a50 sys_nanosleep() at sys_nanosleep+0x56/frame 0xfffffe01d0590aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d0590bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d0590bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003b6d6a, rsp = 0x7fffffffea58, rbp = 0x7fffffffec30 --- Tracing command sshd pid 2477 tid 100219 td 0xfffffe01d4a0ba00 sched_switch() at sched_switch+0x90f/frame 0xfffffe01537ba660 mi_switch() at mi_switch+0x268/frame 0xfffffe01537ba690 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01537ba6e0 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01537ba730 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01537ba750 _cv_wait_sig() at _cv_wait_sig+0x205/frame 0xfffffe01537ba7c0 seltdwait() at seltdwait+0x9d/frame 0xfffffe01537ba800 kern_select() at kern_select+0xa13/frame 0xfffffe01537baa40 sys_select() at sys_select+0x92/frame 0xfffffe01537baaa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01537babf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01537babf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800872eca, rsp = 0x7fffffffe5d8, rbp = 0x7fffffffe610 --- Tracing command powerd pid 2448 tid 100149 td 0xfffffe01d0800800 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d0541620 mi_switch() at mi_switch+0x268/frame 0xfffffe01d0541650 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d05416a0 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01d05416f0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x16/frame 0xfffffe01d0541740 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x223/frame 0xfffffe01d05417c0 seltdwait() at seltdwait+0x8f/frame 0xfffffe01d0541800 kern_select() at kern_select+0xa13/frame 0xfffffe01d0541a40 sys_select() at sys_select+0x92/frame 0xfffffe01d0541aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d0541bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d0541bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80039feca, rsp = 0x7fffffffeaf8, rbp = 0x7fffffffec40 --- Tracing command ntpd pid 2445 tid 100165 td 0xfffffe01d07d5300 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d05b5660 mi_switch() at mi_switch+0x268/frame 0xfffffe01d05b5690 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d05b56e0 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01d05b5730 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01d05b5750 _cv_wait_sig() at _cv_wait_sig+0x205/frame 0xfffffe01d05b57c0 seltdwait() at seltdwait+0x9d/frame 0xfffffe01d05b5800 kern_select() at kern_select+0xa13/frame 0xfffffe01d05b5a40 sys_select() at sys_select+0x92/frame 0xfffffe01d05b5aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d05b5bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d05b5bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8007a2eca, rsp = 0x7fffffffde68, rbp = 0x7fffffffdea0 --- Tracing command rpc.lockd pid 2406 tid 100194 td 0xfffffe01d2c65800 sched_switch() at sched_switch+0x90f/frame 0xfffffe0153772500 mi_switch() at mi_switch+0x268/frame 0xfffffe0153772530 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0153772580 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01537725d0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x16/frame 0xfffffe0153772620 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x223/frame 0xfffffe01537726a0 svc_run_internal() at svc_run_internal+0x13f5/frame 0xfffffe0153772880 svc_run() at svc_run+0x3ff/frame 0xfffffe0153772900 sys_nlm_syscall() at sys_nlm_syscall+0x98f/frame 0xfffffe0153772aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe0153772bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0153772bf0 --- syscall (154, FreeBSD ELF64, sys_nlm_syscall), rip = 0x80039d70a, rsp = 0x7fffffffeb78, rbp = 0x7fffffffec30 --- Tracing command rpc.statd pid 2403 tid 100186 td 0xfffffe01d07fe500 sched_switch() at sched_switch+0x90f/frame 0xfffffe0153748620 mi_switch() at mi_switch+0x268/frame 0xfffffe0153748650 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01537486a0 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01537486f0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x16/frame 0xfffffe0153748740 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x223/frame 0xfffffe01537487c0 seltdwait() at seltdwait+0x8f/frame 0xfffffe0153748800 kern_select() at kern_select+0xa13/frame 0xfffffe0153748a40 sys_select() at sys_select+0x92/frame 0xfffffe0153748aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe0153748bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0153748bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800394eca, rsp = 0x7fffffffea18, rbp = 0x7fffffffeb60 --- Tracing command rpcbind pid 2381 tid 100173 td 0xfffffe0153929e00 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d05d36a0 mi_switch() at mi_switch+0x268/frame 0xfffffe01d05d36d0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d05d3720 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01d05d3770 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x16/frame 0xfffffe01d05d37c0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x223/frame 0xfffffe01d05d3840 seltdwait() at seltdwait+0x8f/frame 0xfffffe01d05d3880 kern_poll() at kern_poll+0x57b/frame 0xfffffe01d05d3a50 sys_poll() at sys_poll+0x94/frame 0xfffffe01d05d3aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d05d3bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d05d3bf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x80039adca, rsp = 0x7fffffffc9a8, rbp = 0x7fffffffebc0 --- Tracing command watchdogd pid 2364 tid 100206 td 0xfffffe01d49fa800 sched_switch() at sched_switch+0x90f/frame 0xfffffe01537b07e0 mi_switch() at mi_switch+0x268/frame 0xfffffe01537b0810 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01537b0860 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01537b08b0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x16/frame 0xfffffe01537b0900 _sleep() at _sleep+0x2ff/frame 0xfffffe01537b09b0 kern_clock_nanosleep() at kern_clock_nanosleep+0x2a1/frame 0xfffffe01537b0a50 sys_nanosleep() at sys_nanosleep+0x56/frame 0xfffffe01537b0aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01537b0bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01537b0bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x80039fd6a, rsp = 0x7fffffffea08, rbp = 0x7fffffffea50 --- Tracing command syslogd pid 2349 tid 100212 td 0xfffffe01d294ac00 sched_switch() at sched_switch+0x90f/frame 0xfffffe0153752660 mi_switch() at mi_switch+0x268/frame 0xfffffe0153752690 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01537526e0 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe0153752730 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe0153752750 _cv_wait_sig() at _cv_wait_sig+0x205/frame 0xfffffe01537527c0 seltdwait() at seltdwait+0x9d/frame 0xfffffe0153752800 kern_select() at kern_select+0xa13/frame 0xfffffe0153752a40 sys_select() at sys_select+0x92/frame 0xfffffe0153752aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe0153752bf0 fast_syscall_common() at fast_syscall_co--- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8003a8eca, rsp = 0x7fffffffe6b8, rbp = 0x7fffffffec30 --- Tracing command devd pid 2170 tid 100163 td 0xfffffe0129345c00 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d05ab620 mi_switch() at mi_switch+0x268/frame 0xfffffe01d05ab650 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d05ab6a0 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01d05ab6f0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x16/frame 0xfffffe01d05ab740 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x223/frame 0xfffffe01d05ab7c0 seltdwait() at seltdwait+0x8f/frame 0xfffffe01d05ab800 kern_select() at kern_select+0xa13/frame 0xfffffe01d05aba40 sys_select() at sys_select+0x92/frame 0xfffffe01d05abaa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d05abbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d05abbf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x28c28a, rsp = 0x7fffffffcab8, rbp = 0x7fffffffec50 --- Tracing command dhclient pid 2165 tid 100190 td 0xfffffe01d2c65100 sched_switch() at sched_switch+0x90f/frame 0xfffffe01537346a0 mi_switch() at mi_switch+0x268/frame 0xfffffe01537346d0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0153734720 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe0153734770 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x16/frame 0xfffffe01537347c0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x223/frame 0xfffffe0153734840 seltdwait() at seltdwait+0x8f/frame 0xfffffe0153734880 kern_poll() at kern_poll+0x57b/frame 0xfffffe0153734a50 sys_poll() at sys_poll+0x94/frame 0xfffffe0153734aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe0153734bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0153734bf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8003bfdca, rsp = 0x7fffffffeb98, rbp = 0x7fffffffebd0 --- Tracing command dhclient pid 2089 tid 100208 td 0xfffffe01d2c64a00 sched_switch() at sched_switch+0x90f/frame 0xfffffe01537c96e0 mi_switch() at mi_switch+0x268/frame 0xfffffe01537c9710 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01537c9760 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01537c97b0 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01537c97d0 _cv_wait_sig() at _cv_wait_sig+0x205/frame 0xfffffe01537c9840 seltdwait() at seltdwait+0x9d/frame 0xfffffe01537c9880 kern_poll() at kern_poll+0x57b/frame 0xfffffe01537c9a50 sys_poll() at sys_poll+0x94/frame 0xfffffe01537c9aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01537c9bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01537c9bf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8003bfdca, rsp = 0x7fffffffeba8, rbp = 0x7fffffffebd0 --- Tracing command dhclient pid 2086 tid 100187 td 0xfffffe01d294ba00 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d05ec660 mi_switch() at mi_switch+0x268/frame 0xfffffe01d05ec690 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d05ec6e0 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe01d05ec730 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe01d05ec750 _cv_wait_sig() at _cv_wait_sig+0x205/frame 0xfffffe01d05ec7c0 seltdwait() at seltdwait+0x9d/frame 0xfffffe01d05ec800 kern_select() at kern_select+0xa13/frame 0xfffffe01d05eca40 sys_select() at sys_select+0x92/frame 0xfffffe01d05ecaa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe01d05ecbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d05ecbf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8003bfeca, rsp = 0x7fffffffe958, rbp = 0x7fffffffea10 --- Tracing command vnlru pid 33 tid 100142 td 0xfffffe015392b300 sched_switch() at sched_switch+0x90f/frame 0xfffffe01af5f3950 mi_switch() at mi_switch+0x268/frame 0xfffffe01af5f3980 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01af5f39d0 sleepq_timedwait() at sleepq_timedwait+0x60/frame 0xfffffe01af5f3a20 _sleep() at _sleep+0x30e/frame 0xfffffe01af5f3ad0 vnlru_proc() at vnlru_proc+0x101f/frame 0xfffffe01af5f3bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe01af5f3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01af5f3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command syncer pid 32 tid 100141 td 0xfffffe015392ba00 sched_switch() at sched_switch+0x90f/frame 0xfffffe01af5f89c0 mi_switch() at mi_switch+0x268/frame 0xfffffe01af5f89f0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01af5f8a40 sleepq_timedwait() at sleepq_timedwait+0x60/frame 0xfffffe01af5f8a90 _cv_timedwait_sbt() at _cv_timedwait_sbt+0x223/frame 0xfffffe01af5f8b10 sched_sync() at sched_sync+0x891/frame 0xfffffe01af5f8bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe01af5f8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01af5f8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100131 td 0xfffffe015392c100 sched_switch() at sched_switch+0x90f/frame 0xfffffe015361f9e0 mi_switch() at mi_switch+0x268/frame 0xfffffe015361fa10 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe015361fa60 sleepq_timedwait() at sleepq_timedwait+0x60/frame 0xfffffe015361fab0 _sleep() at _sleep+0x30e/frame 0xfffffe015361fb60 buf_daemon() at buf_daemon+0x151/frame 0xfffffe015361fbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe015361fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe015361fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100132 td 0xfffffe015392c800 sched_switch() at sched_switch+0x90f/frame 0xfffffe0153d0a990 mi_switch() at mi_switch+0x268/frame 0xfffffe0153d0a9c0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0153d0aa10 sleepq_timedwait() at sleepq_timedwait+0x60/frame 0xfffffe0153d0aa60 _sleep() at _sleep+0x30e/frame 0xfffffe0153d0ab10 bufspace_daemon() at bufspace_daemon+0x41d/frame 0xfffffe0153d0abb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0153d0abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0153d0abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100133 td 0xfffffe015393f100 sched_switch() at sched_switch+0x90f/frame 0xfffffe0153d05990 mi_switch() at mi_switch+0x268/frame 0xfffffe0153d059c0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0153d05a10 sleepq_timedwait() at sleepq_timedwait+0x60/frame 0xfffffe0153d05a60 _sleep() at _sleep+0x30e/frame 0xfffffe0153d05b10 bufspace_daemon() at bufspace_daemon+0x41d/frame 0xfffffe0153d05bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0153d05bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0153d05bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100134 td 0xfffffe015393ea00 sched_switch() at sched_switch+0x90f/frame 0xfffffe0153d14990 mi_switch() at mi_switch+0x268/frame 0xfffffe0153d149c0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0153d14a10 sleepq_timedwait() at sleepq_timedwait+0x60/frame 0xfffffe0153d14a60 _sleep() at _sleep+0x30e/frame 0xfffffe0153d14b10 bufspace_daemon() at bufspace_daemon+0x41d/frame 0xfffffe0153d14bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0153d14bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0153d14bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100135 td 0xfffffe015393e300 sched_switch() at sched_switch+0x90f/frame 0xfffffe0153d0f990 mi_switch() at mi_switch+0x268/frame 0xfffffe0153d0f9c0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0153d0fa10 sleepq_timedwait() at sleepq_timedwait+0x60/frame 0xfffffe0153d0fa60 _sleep() at _sleep+0x30e/frame 0xfffffe0153d0fb10 bufspace_daemon() at bufspace_daemon+0x41d/frame 0xfffffe0153d0fbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0153d0fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0153d0fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100136 td 0xfffffe015393dc00 sched_switch() at sched_switch+0x90f/frame 0xfffffe0153d1e990 mi_switch() at mi_switch+0x268/frame 0xfffffe0153d1e9c0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0153d1ea10 sleepq_timedwait() at sleepq_timedwait+0x60/frame 0xfffffe0153d1ea60 _sleep() at _sleep+0x30e/frame 0xfffffe0153d1eb10 bufspace_daemon() at bufspace_daemon+0x41d/frame 0xfffffe0153d1ebb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0153d1ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0153d1ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100137 td 0xfffffe015393d500 sched_switch() at sched_switch+0x90f/frame 0xfffffe0153d19990 mi_switch() at mi_switch+0x268/frame 0xfffffe0153d199c0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0153d19a10 sleepq_timedwait() at sleepq_timedwait+0x60/frame 0xfffffe0153d19a60 _sleep() at _sleep+0x30e/frame 0xfffffe0153d19b10 bufspace_daemon() at bufspace_daemon+0x41d/frame 0xfffffe0153d19bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0153d19bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0153d19bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100138 td 0xfffffe015393ce00 sched_switch() at sched_switch+0x90f/frame 0xfffffe0153d28990 mi_switch() at mi_switch+0x268/frame 0xfffffe0153d289c0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0153d28a10 sleepq_timedwait() at sleepq_timedwait+0x60/frame 0xfffffe0153d28a60 _sleep() at _sleep+0x30e/frame 0xfffffe0153d28b10 bufspace_daemon() at bufspace_daemon+0x41d/frame 0xfffffe0153d28bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0153d28bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0153d28bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100152 td 0xfffffe01d18fb100 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d05329d0 mi_switch() at mi_switch+0x268/frame 0xfffffe01d0532a00 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d0532a50 sleepq_timedwait() at sleepq_timedwait+0x60/frame 0xfffffe01d0532aa0 _sleep() at _sleep+0x30e/frame 0xfffffe01d0532b50 softdep_flush() at softdep_flush+0x250/frame 0xfffffe01d0532bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe01d0532bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01d0532bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100160 td 0xfffffe01d07ffa00 sched_switch() at sched_switch+0x90f/frame 0xfffffe01d05959d0 mi_switch() at mi_switch+0x268/frame 0xfffffe01d0595a00 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01d0595a50 sleepq_timedwait() at sleepq_timedwait+0x60/frame 0xfffffe01d0595aa0 _sleep() at _sleep+0x30e/frame 0xfffffe01d0595b50 softdep_flush() at softdep_flush+0x250/frame 0xfffffe01d0595bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe01d0595bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01d0595bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100366 td 0xfffffe01e7f57800 sched_switch() at sched_switch+0x90f/frame 0xfffffe01e785d9d0 mi_switch() at mi_switch+0x268/frame 0xfffffe01e785da00 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01e785da50 sleepq_timedwait() at sleepq_timedwait+0x60/frame 0xfffffe01e785daa0 _sleep() at _sleep+0x30e/frame 0xfffffe01e785db50 softdep_flush() at softdep_flush+0x250/frame 0xfffffe01e785dbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe01e785dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01e785dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command vmdaemon pid 30 tid 100130 td 0xfffffe0153164800 sched_switch() at sched_switch+0x90f/frame 0xfffffe01536249f0 mi_switch() at mi_switch+0x268/frame 0xfffffe0153624a20 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0153624a70 _sleep() at _sleep+0x31a/frame 0xfffffe0153624b20 vm_daemon() at vm_daemon+0xab/frame 0xfffffe0153624bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0153624bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0153624bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 29 tid 100129 td 0xfffffe014c595000 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c3fe670 mi_switch() at mi_switch+0x268/frame 0xfffffe014c3fe6a0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c3fe6f0 sleepq_timedwait() at sleepq_timedwait+0x60/frame 0xfffffe014c3fe740 _sleep() at _sleep+0x30e/frame 0xfffffe014c3fe7f0 vm_pageout_worker() at vm_pageout_worker+0x417/frame 0xfffffe014c3feb60 vm_pageout() at vm_pageout+0x295/frame 0xfffffe014c3febb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c3febf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c3febf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 29 tid 100139 td 0xfffffe0153161e00 sched_switch() at sched_switch+0x90f/frame 0xfffffe015372f7c0 mi_switch() at mi_switch+0x268/frame 0xfffffe015372f7f0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe015372f840 _sleep() at _sleep+0x31a/frame 0xfffffe015372f8f0 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0x11b4/frame 0xfffffe015372fbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe015372fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe015372fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 29 tid 100140 td 0xfffffe0153161700 sched_switch() at sched_switch+0x90f/frame 0xfffffe015372a9f0 mi_switch() at mi_switch+0x268/frame 0xfffffe015372aa20 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe015372aa70 sleepq_timedwait() at sleepq_timedwait+0x60/frame 0xfffffe015372aac0 _sleep() at _sleep+0x30e/frame 0xfffffe015372ab70 uma_reclaim_worker() at uma_reclaim_worker+0xf3/frame 0xfffffe015372abb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe015372abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe015372abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon1 pid 28 tid 100128 td 0xfffffe0153162500 sched_switch() at sched_switch+0x90f/frame 0xfffffe01536099d0 mi_switch() at mi_switch+0x268/frame 0xfffffe0153609a00 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0153609a50 _sleep() at _sleep+0x31a/frame 0xfffffe0153609b00 enc_daemon() at enc_daemon+0x1a4/frame 0xfffffe0153609bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0153609bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0153609bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon0 pid 27 tid 100127 td 0xfffffe0153162c00 sched_switch() at sched_switch+0x90f/frame 0xfffffe015360e9d0 mi_switch() at mi_switch+0x268/frame 0xfffffe015360ea00 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe015360ea50 _sleep() at _sleep+0x31a/frame 0xfffffe015360eb00 enc_daemon() at enc_daemon+0x1a4/frame 0xfffffe015360ebb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe015360ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe015360ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command rand_harvestq pid 26 tid 100124 td 0xfffffe014c596500 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c3f99a0 mi_switch() at mi_switch+0x268/frame 0xfffffe014c3f99d0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c3f9a20 sleepq_timedwait() at sleepq_timedwait+0x60/frame 0xfffffe014c3f9a70 _sleep() at _sleep+0x30e/frame 0xfffffe014c3f9b20 random_kthread() at random_kthread+0x57/frame 0xfffffe014c3f9bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c3f9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c3f9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod4 pid 25 tid 100120 td 0xfffffe0153163300 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c3d49e0 mi_switch() at mi_switch+0x268/frame 0xfffffe014c3d4a10 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c3d4a60 sleepq_timedwait() at sleepq_timedwait+0x60/frame 0xfffffe014c3d4ab0 _sleep() at _sleep+0x30e/frame 0xfffffe014c3d4b60 soaio_kproc_loop() at soaio_kproc_loop+0x1c5/frame 0xfffffe014c3d4bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c3d4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c3d4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod3 pid 24 tid 100119 td 0xfffffe0153163a00 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c3d99e0 mi_switch() at mi_switch+0x268/frame 0xfffffe014c3d9a10 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c3d9a60 sleepq_timedwait() at sleepq_timedwait+0x60/frame 0xfffffe014c3d9ab0 _sleep() at _sleep+0x30e/frame 0xfffffe014c3d9b60 soaio_kproc_loop() at soaio_kproc_loop+0x1c5/frame 0xfffffe014c3d9bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c3d9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c3d9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod2 pid 23 tid 100118 td 0xfffffe0153164100 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c3ca9e0 mi_switch() at mi_switch+0x268/frame 0xfffffe014c3caa10 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c3caa60 sleepq_timedwait() at sleepq_timedwait+0x60/frame 0xfffffe014c3caab0 _sleep() at _sleep+0x30e/frame 0xfffffe014c3cab60 soaio_kproc_loop() at soaio_kproc_loop+0x1c5/frame 0xfffffe014c3cabb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c3cabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c3cabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod1 pid 22 tid 100117 td 0xfffffe014f23d800 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c3cf9e0 mi_switch() at mi_switch+0x268/frame 0xfffffe014c3cfa10 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c3cfa60 sleepq_timedwait() at sleepq_timedwait+0x60/frame 0xfffffe014c3cfab0 _sleep() at _sleep+0x30e/frame 0xfffffe014c3cfb60 soaio_kproc_loop() at soaio_kproc_loop+0x1c5/frame 0xfffffe014c3cfbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c3cfbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c3cfbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100098 td 0xfffffe014c91e300 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c34ea50 mi_switch() at mi_switch+0x268/frame 0xfffffe014c34ea80 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c34ead0 _cv_wait() at _cv_wait+0x1fc/frame 0xfffffe014c34eb40 usb_process() at usb_process+0x127/frame 0xfffffe014c34ebb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c34ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c34ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100099 td 0xfffffe014c91dc00 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c35da50 mi_switch() at mi_switch+0x268/frame 0xfffffe014c35da80 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c35dad0 _cv_wait() at _cv_wait+0x1fc/frame 0xfffffe014c35db40 usb_process() at usb_process+0x127/frame 0xfffffe014c35dbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c35dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c35dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100100 td 0xfffffe014c91d500 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c358a50 mi_switch() at mi_switch+0x268/frame 0xfffffe014c358a80 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c358ad0 _cv_wait() at _cv_wait+0x1fc/frame 0xfffffe014c358b40 usb_process() at usb_process+0x127/frame 0xfffffe014c358bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c358bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c358bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100101 td 0xfffffe014c91ce00 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c367a50 mi_switch() at mi_switch+0x268/frame 0xfffffe014c367a80 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c367ad0 _cv_wait() at _cv_wait+0x1fc/frame 0xfffffe014c367b40 usb_process() at usb_process+0x127/frame 0xfffffe014c367bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c367bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c367bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100102 td 0xfffffe014c91c700 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c362a50 mi_switch() at mi_switch+0x268/frame 0xfffffe014c362a80 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c362ad0 _cv_wait() at _cv_wait+0x1fc/frame 0xfffffe014c362b40 usb_process() at usb_process+0x127/frame 0xfffffe014c362bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c362bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c362bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100104 td 0xfffffe014c91f800 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c36ca50 mi_switch() at mi_switch+0x268/frame 0xfffffe014c36ca80 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c36cad0 _cv_wait() at _cv_wait+0x1fc/frame 0xfffffe014c36cb40 usb_process() at usb_process+0x127/frame 0xfffffe014c36cbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c36cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c36cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100105 td 0xfffffe014ee0a100 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c37ba50 mi_switch() at mi_switch+0x268/frame 0xfffffe014c37ba80 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c37bad0 _cv_wait() at _cv_wait+0x1fc/frame 0xfffffe014c37bb40 usb_process() at usb_process+0x127/frame 0xfffffe014c37bbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c37bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c37bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100106 td 0xfffffe014ee09a00 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c376a50 mi_switch() at mi_switch+0x268/frame 0xfffffe014c376a80 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c376ad0 _cv_wait() at _cv_wait+0x1fc/frame 0xfffffe014c376b40 usb_process() at usb_process+0x127/frame 0xfffffe014c376bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c376bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c376bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100107 td 0xfffffe014ee09300 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c385a50 mi_switch() at mi_switch+0x268/frame 0xfffffe014c385a80 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c385ad0 _cv_wait() at _cv_wait+0x1fc/frame 0xfffffe014c385b40 usb_process() at usb_process+0x127/frame 0xfffffe014c385bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c385bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c385bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100108 td 0xfffffe014ee08c00 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c380a50 mi_switch() at mi_switch+0x268/frame 0xfffffe014c380a80 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c380ad0 _cv_wait() at _cv_wait+0x1fc/frame 0xfffffe014c380b40 usb_process() at usb_process+0x127/frame 0xfffffe014c380bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c380bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c380bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100109 td 0xfffffe014ee0a800 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c39ba50 mi_switch() at mi_switch+0x268/frame 0xfffffe014c39ba80 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c39bad0 _cv_wait() at _cv_wait+0x1fc/frame 0xfffffe014c39bb40 usb_process() at usb_process+0x127/frame 0xfffffe014c39bbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c39bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c39bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100110 td 0xfffffe014f23d100 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c396a50 mi_switch() at mi_switch+0x268/frame 0xfffffe014c396a80 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c396ad0 _cv_wait() at _cv_wait+0x1fc/frame 0xfffffe014c396b40 usb_process() at usb_process+0x127/frame 0xfffffe014c396bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c396bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c396bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100111 td 0xfffffe014f23ca00 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c3a5a50 mi_switch() at mi_switch+0x268/frame 0xfffffe014c3a5a80 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c3a5ad0 _cv_wait() at _cv_wait+0x1fc/frame 0xfffffe014c3a5b40 usb_process() at usb_process+0x127/frame 0xfffffe014c3a5bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c3a5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c3a5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100112 td 0xfffffe014f23c300 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c3a0a50 mi_switch() at mi_switch+0x268/frame 0xfffffe014c3a0a80 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c3a0ad0 _cv_wait() at _cv_wait+0x1fc/frame 0xfffffe014c3a0b40 usb_process() at usb_process+0x127/frame 0xfffffe014c3a0bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c3a0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c3a0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100113 td 0xfffffe014f23bc00 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c3afa50 mi_switch() at mi_switch+0x268/frame 0xfffffe014c3afa80 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c3afad0 _cv_wait() at _cv_wait+0x1fc/frame 0xfffffe014c3afb40 usb_process() at usb_process+0x127/frame 0xfffffe014c3afbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c3afbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c3afbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100092 td 0xfffffe014c432000 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c335a00 mi_switch() at mi_switch+0x268/frame 0xfffffe014c335a30 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c335a80 _sleep() at _sleep+0x31a/frame 0xfffffe014c335b30 xpt_done_td() at xpt_done_td+0x105/frame 0xfffffe014c335bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c335bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c335bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100093 td 0xfffffe014c435800 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c330a00 mi_switch() at mi_switch+0x268/frame 0xfffffe014c330a30 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c330a80 _sleep() at _sleep+0x31a/frame 0xfffffe014c330b30 xpt_done_td() at xpt_done_td+0x105/frame 0xfffffe014c330bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c330bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c330bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100094 td 0xfffffe014c598100 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c33fa00 mi_switch() at mi_switch+0x268/frame 0xfffffe014c33fa30 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c33fa80 _sleep() at _sleep+0x31a/frame 0xfffffe014c33fb30 xpt_done_td() at xpt_done_td+0x105/frame 0xfffffe014c33fbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c33fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c33fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100126 td 0xfffffe014c595700 sched_switch() at sched_switch+0x90f/frame 0xfffffe0153604a40 mi_switch() at mi_switch+0x268/frame 0xfffffe0153604a70 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0153604ac0 _sleep() at _sleep+0x31a/frame 0xfffffe0153604b70 xpt_scanner_thread() at xpt_scanner_thread+0x1e8/frame 0xfffffe0153604bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0153604bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0153604bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 11 pid 19 tid 100091 td 0xfffffe014c432700 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c3269f0 mi_switch() at mi_switch+0x268/frame 0xfffffe014c326a20 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c326a70 _sleep() at _sleep+0x31a/frame 0xfffffe014c326b20 crypto_ret_proc() at crypto_ret_proc+0x42b/frame 0xfffffe014c326bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c326bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c326bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 10 pid 18 tid 100090 td 0xfffffe014c432e00 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c32b9f0 mi_switch() at mi_switch+0x268/frame 0xfffffe014c32ba20 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c32ba70 _sleep() at _sleep+0x31a/frame 0xfffffe014c32bb20 crypto_ret_proc() at crypto_ret_proc+0x42b/frame 0xfffffe014c32bbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c32bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c32bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 9 pid 17 tid 100089 td 0xfffffe014c433500 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c31c9f0 mi_switch() at mi_switch+0x268/frame 0xfffffe014c31ca20 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c31ca70 _sleep() at _sleep+0x31a/frame 0xfffffe014c31cb20 crypto_ret_proc() at crypto_ret_proc+0x42b/frame 0xfffffe014c31cbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c31cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c31cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 8 pid 16 tid 100088 td 0xfffffe014c433c00 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c3219f0 mi_switch() at mi_switch+0x268/frame 0xfffffe014c321a20 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c321a70 _sleep() at _sleep+0x31a/frame 0xfffffe014c321b20 crypto_ret_proc() at crypto_ret_proc+0x42b/frame 0xfffffe014c321bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c321bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c321bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 7 pid 15 tid 100087 td 0xfffffe014c434300 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c3129f0 mi_switch() at mi_switch+0x268/frame 0xfffffe014c312a20 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c312a70 _sleep() at _sleep+0x31a/frame 0xfffffe014c312b20 crypto_ret_proc() at crypto_ret_proc+0x42b/frame 0xfffffe014c312bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c312bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c312bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 6 pid 9 tid 100086 td 0xfffffe014c434a00 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c3179f0 mi_switch() at mi_switch+0x268/frame 0xfffffe014c317a20 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c317a70 _sleep() at _sleep+0x31a/frame 0xfffffe014c317b20 crypto_ret_proc() at crypto_ret_proc+0x42b/frame 0xfffffe014c317bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c317bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c317bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 5 pid 8 tid 100085 td 0xfffffe014c435100 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c3089f0 mi_switch() at mi_switch+0x268/frame 0xfffffe014c308a20 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c308a70 _sleep() at _sleep+0x31a/frame 0xfffffe014c308b20 crypto_ret_proc() at crypto_ret_proc+0x42b/frame 0xfffffe014c308bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c308bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c308bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 4 pid 7 tid 100084 td 0xfffffe014c422800 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c30d9f0 mi_switch() at mi_switch+0x268/frame 0xfffffe014c30da20 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c30da70 _sleep() at _sleep+0x31a/frame 0xfffffe014c30db20 crypto_ret_proc() at crypto_ret_proc+0x42b/frame 0xfffffe014c30dbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c30dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c30dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 3 pid 6 tid 100083 td 0xfffffe014c41f000 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c2fe9f0 mi_switch() at mi_switch+0x268/frame 0xfffffe014c2fea20 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c2fea70 _sleep() at _sleep+0x31a/frame 0xfffffe014c2feb20 crypto_ret_proc() at crypto_ret_proc+0x42b/frame 0xfffffe014c2febb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c2febf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c2febf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 2 pid 5 tid 100082 td 0xfffffe014c41f700 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c3039f0 mi_switch() at mi_switch+0x268/frame 0xfffffe014c303a20 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c303a70 _sleep() at _sleep+0x31a/frame 0xfffffe014c303b20 crypto_ret_proc() at crypto_ret_proc+0x42b/frame 0xfffffe014c303bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c303bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c303bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 1 pid 4 tid 100081 td 0xfffffe014c41fe00 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c2f49f0 mi_switch() at mi_switch+0x268/frame 0xfffffe014c2f4a20 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c2f4a70 _sleep() at _sleep+0x31a/frame 0xfffffe014c2f4b20 crypto_ret_proc() at crypto_ret_proc+0x42b/frame 0xfffffe014c2f4bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c2f4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c2f4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 0 pid 3 tid 100080 td 0xfffffe014c420500 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c2f99f0 mi_switch() at mi_switch+0x268/frame 0xfffffe014c2f9a20 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c2f9a70 _sleep() at _sleep+0x31a/frame 0xfffffe014c2f9b20 crypto_ret_proc() at crypto_ret_proc+0x42b/frame 0xfffffe014c2f9bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c2f9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c2f9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 2 tid 100079 td 0xfffffe014c420c00 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c2eaa30 mi_switch() at mi_switch+0x268/frame 0xfffffe014c2eaa60 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c2eaab0 _sleep() at _sleep+0x31a/frame 0xfffffe014c2eab60 crypto_proc() at crypto_proc+0x3ab/frame 0xfffffe014c2eabb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c2eabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c2eabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sequencer 00 pid 14 tid 100066 td 0xfffffe0129361800 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c2b3a10 mi_switch() at mi_switch+0x268/frame 0xfffffe014c2b3a40 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c2b3a90 _cv_wait() at _cv_wait+0x1fc/frame 0xfffffe014c2b3b00 seq_eventthread() at seq_eventthread+0xf7/frame 0xfffffe014c2b3bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c2b3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c2b3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100063 td 0xfffffe012935ee00 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c29aa20 mi_switch() at mi_switch+0x268/frame 0xfffffe014c29aa50 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c29aaa0 _sleep() at _sleep+0x31a/frame 0xfffffe014c29ab50 g_run_events() at g_run_events+0x64/frame 0xfffffe014c29abb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c29abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c29abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100064 td 0xfffffe012935e700 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c2a9a30 mi_switch() at mi_switch+0x268/frame 0xfffffe014c2a9a60 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c2a9ab0 _sleep() at _sleep+0x31a/frame 0xfffffe014c2a9b60 g_io_schedule_up() at g_io_schedule_up+0x3c/frame 0xfffffe014c2a9b90 g_up_procbody() at g_up_procbody+0x6c/frame 0xfffffe014c2a9bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c2a9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c2a9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100065 td 0xfffffe012935e000 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c2a4a20 mi_switch() at mi_switch+0x268/frame 0xfffffe014c2a4a50 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c2a4aa0 _sleep() at _sleep+0x31a/frame 0xfffffe014c2a4b50 g_io_schedule_down() at g_io_schedule_down+0x4c/frame 0xfffffe014c2a4b90 g_down_procbody() at g_down_procbody+0x6c/frame 0xfffffe014c2a4bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c2a4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c2a4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100042 td 0xfffffe0129337e00 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128feea90 mi_switch() at mi_switch+0x268/frame 0xfffffe0128feeac0 ithread_loop() at ithread_loop+0x693/frame 0xfffffe0128feebb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128feebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128feebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100044 td 0xfffffe0129337000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100047 td 0xfffffe0129346a00 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c213a90 mi_switch() at mi_switch+0x268/frame 0xfffffe014c213ac0 ithread_loop() at ithread_loop+0x693/frame 0xfffffe014c213bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c213bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c213bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100048 td 0xfffffe0129347800 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100049 td 0xfffffe0129358100 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c224a90 mi_switch() at mi_switch+0x268/frame 0xfffffe014c224ac0 ithread_loop() at ithread_loop+0x693/frame 0xfffffe014c224bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c224bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c224bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100050 td 0xfffffe0129357a00 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c233a90 mi_switch() at mi_switch+0x268/frame 0xfffffe014c233ac0 ithread_loop() at ithread_loop+0x693/frame 0xfffffe014c233bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c233bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c233bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100051 td 0xfffffe0129357300 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c22ea90 mi_switch() at mi_switch+0x268/frame 0xfffffe014c22eac0 ithread_loop() at ithread_loop+0x693/frame 0xfffffe014c22ebb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c22ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c22ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100052 td 0xfffffe0129356c00 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c23da90 mi_switch() at mi_switch+0x268/frame 0xfffffe014c23dac0 ithread_loop() at ithread_loop+0x693/frame 0xfffffe014c23dbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c23dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c23dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100053 td 0xfffffe0129356500 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c238a90 mi_switch() at mi_switch+0x268/frame 0xfffffe014c238ac0 ithread_loop() at ithread_loop+0x693/frame 0xfffffe014c238bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c238bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c238bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100054 td 0xfffffe0129355e00 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c247a90 mi_switch() at mi_switch+0x268/frame 0xfffffe014c247ac0 ithread_loop() at ithread_loop+0x693/frame 0xfffffe014c247bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c247bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c247bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100055 td 0xfffffe0129355700 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c242a90 mi_switch() at mi_switch+0x268/frame 0xfffffe014c242ac0 ithread_loop() at ithread_loop+0x693/frame 0xfffffe014c242bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c242bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c242bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100056 td 0xfffffe0129355000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100057 td 0xfffffe0129358800 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100058 td 0xfffffe0129361100 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100059 td 0xfffffe0129360a00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100060 td 0xfffffe0129360300 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100061 td 0xfffffe012935fc00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100095 td 0xfffffe014c598800 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100096 td 0xfffffe014c91f100 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c344a90 mi_switch() at mi_switch+0x268/frame 0xfffffe014c344ac0 ithread_loop() at ithread_loop+0x693/frame 0xfffffe014c344bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c344bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c344bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100097 td 0xfffffe014c91ea00 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c353a90 mi_switch() at mi_switch+0x268/frame 0xfffffe014c353ac0 ithread_loop() at ithread_loop+0x693/frame 0xfffffe014c353bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c353bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c353bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100103 td 0xfffffe014c91c000 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c371a90 mi_switch() at mi_switch+0x268/frame 0xfffffe014c371ac0 ithread_loop() at ithread_loop+0x693/frame 0xfffffe014c371bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c371bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c371bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100114 td 0xfffffe014f23b500 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100115 td 0xfffffe014f23ae00 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c3b9a90 mi_switch() at mi_switch+0x268/frame 0xfffffe014c3b9ac0 ithread_loop() at ithread_loop+0x693/frame 0xfffffe014c3b9bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c3b9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c3b9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100195 td 0xfffffe01d083ea00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100196 td 0xfffffe01d083e300 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100197 td 0xfffffe01d083dc00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100198 td 0xfffffe01d083d500 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100199 td 0xfffffe01d083ce00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100200 td 0xfffffe01d083c700 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100201 td 0xfffffe01d083c000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100202 td 0xfffffe01d49f2100 fork_trampoline() at fork_trampoline Tracing command idle pid 11 tid 100003 td 0xfffffe012928a300 (CPU 0) cpustop_handler() at cpustop_handler+0x32/frame 0xffffffff8370fad0 ipi_nmi_handler() at ipi_nmi_handler+0x4f/frame 0xffffffff8370fb00 trap() at trap+0x78/frame 0xffffffff8370fc20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff8370fc20 --- trap 0x13, rip = 0xffffffff81822836, rsp = 0xfffffe0128f359f0, rbp = 0xfffffe0128f359f0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0128f359f0 acpi_cpu_idle() at acpi_cpu_idle+0x3b0/frame 0xfffffe0128f35a50 cpu_idle_acpi() at cpu_idle_acpi+0x46/frame 0xfffffe0128f35a70 cpu_idle() at cpu_idle+0xdb/frame 0xfffffe0128f35aa0 sched_idletd() at sched_idletd+0x5c9/frame 0xfffffe0128f35bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f35bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f35bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xfffffe0129289c00 (CPU 1) cpustop_handler() at cpustop_handler+0x32/frame 0xfffffe0081694dd0 ipi_nmi_handler() at ipi_nmi_handler+0x4f/frame 0xfffffe0081694e00 trap() at trap+0x78/frame 0xfffffe0081694f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0081694f20 --- trap 0x13, rip = 0xffffffff81822836, rsp = 0xfffffe0128f309f0, rbp = 0xfffffe0128f309f0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0128f309f0 acpi_cpu_idle() at acpi_cpu_idle+0x3b0/frame 0xfffffe0128f30a50 cpu_idle_acpi() at cpu_idle_acpi+0x46/frame 0xfffffe0128f30a70 cpu_idle() at cpu_idle+0xdb/frame 0xfffffe0128f30aa0 sched_idletd() at sched_idletd+0x5c9/frame 0xfffffe0128f30bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f30bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f30bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffffe0129289500 (CPU 2) cpustop_handler() at cpustop_handler+0x32/frame 0xfffffe00816a0dd0 ipi_nmi_handler() at ipi_nmi_handler+0x4f/frame 0xfffffe00816a0e00 trap() at trap+0x78/frame 0xfffffe00816a0f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00816a0f20 --- trap 0x13, rip = 0xffffffff81822836, rsp = 0xfffffe0128f3f9f0, rbp = 0xfffffe0128f3f9f0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0128f3f9f0 acpi_cpu_idle() at acpi_cpu_idle+0x3b0/frame 0xfffffe0128f3fa50 cpu_idle_acpi() at cpu_idle_acpi+0x46/frame 0xfffffe0128f3fa70 cpu_idle() at cpu_idle+0xdb/frame 0xfffffe0128f3faa0 sched_idletd() at sched_idletd+0x5c9/frame 0xfffffe0128f3fbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f3fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f3fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xfffffe0129288e00 (CPU 3) cpustop_handler() at cpustop_handler+0x32/frame 0xfffffe00816acdd0 ipi_nmi_handler() at ipi_nmi_handler+0x4f/frame 0xfffffe00816ace00 trap() at trap+0x78/frame 0xfffffe00816acf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00816acf20 --- trap 0x13, rip = 0xffffffff81822836, rsp = 0xfffffe0128f3a9f0, rbp = 0xfffffe0128f3a9f0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0128f3a9f0 acpi_cpu_idle() at acpi_cpu_idle+0x3b0/frame 0xfffffe0128f3aa50 cpu_idle_acpi() at cpu_idle_acpi+0x46/frame 0xfffffe0128f3aa70 cpu_idle() at cpu_idle+0xdb/frame 0xfffffe0128f3aaa0 sched_idletd() at sched_idletd+0x5c9/frame 0xfffffe0128f3abb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f3abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f3abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100007 td 0xfffffe0129288700 (CPU 4) cpustop_handler() at cpustop_handler+0x32/frame 0xfffffe00816b8dd0 ipi_nmi_handler() at ipi_nmi_handler+0x4f/frame 0xfffffe00816b8e00 trap() at trap+0x78/frame 0xfffffe00816b8f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00816b8f20 --- trap 0x13, rip = 0xffffffff81822836, rsp = 0xfffffe0128f499f0, rbp = 0xfffffe0128f499f0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0128f499f0 acpi_cpu_idle() at acpi_cpu_idle+0x3b0/frame 0xfffffe0128f49a50 cpu_idle_acpi() at cpu_idle_acpi+0x46/frame 0xfffffe0128f49a70 cpu_idle() at cpu_idle+0xdb/frame 0xfffffe0128f49aa0 sched_idletd() at sched_idletd+0x5c9/frame 0xfffffe0128f49bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f49bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f49bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xfffffe0129288000 (CPU 5) cpustop_handler() at cpustop_handler+0x32/frame 0xfffffe00816c4dd0 ipi_nmi_handler() at ipi_nmi_handler+0x4f/frame 0xfffffe00816c4e00 trap() at trap+0x78/frame 0xfffffe00816c4f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00816c4f20 --- trap 0x13, rip = 0xffffffff81822836, rsp = 0xfffffe0128f449f0, rbp = 0xfffffe0128f449f0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0128f449f0 acpi_cpu_idle() at acpi_cpu_idle+0x3b0/frame 0xfffffe0128f44a50 cpu_idle_acpi() at cpu_idle_acpi+0x46/frame 0xfffffe0128f44a70 cpu_idle() at cpu_idle+0xdb/frame 0xfffffe0128f44aa0 sched_idletd() at sched_idletd+0x5c9/frame 0xfffffe0128f44bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f44bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f44bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xfffffe012928b800 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128f53a70 mi_switch() at mi_switch+0x268/frame 0xfffffe0128f53aa0 sched_idletd() at sched_idletd+0x5c9/frame 0xfffffe0128f53bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f53bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f53bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xfffffe0129296100 (CPU 7) cpustop_handler() at cpustop_handler+0x32/frame 0xfffffe00816dcdd0 ipi_nmi_handler() at ipi_nmi_handler+0x4f/frame 0xfffffe00816dce00 trap() at trap+0x78/frame 0xfffffe00816dcf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00816dcf20 --- trap 0x13, rip = 0xffffffff81822836, rsp = 0xfffffe0128f4e9f0, rbp = 0xfffffe0128f4e9f0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0128f4e9f0 acpi_cpu_idle() at acpi_cpu_idle+0x3b0/frame 0xfffffe0128f4ea50 cpu_idle_acpi() at cpu_idle_acpi+0x46/frame 0xfffffe0128f4ea70 cpu_idle() at cpu_idle+0xdb/frame 0xfffffe0128f4eaa0 sched_idletd() at sched_idletd+0x5c9/frame 0xfffffe0128f4ebb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f4ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f4ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffffe0129295a00 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128f5da70 mi_switch() at mi_switch+0x268/frame 0xfffffe0128f5daa0 sched_idletd() at sched_idletd+0x5c9/frame 0xfffffe0128f5dbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f5dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f5dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100012 td 0xfffffe0129295300 (CPU 9) cpustop_handler() at cpustop_handler+0x32/frame 0xfffffe00816f4dd0 ipi_nmi_handler() at ipi_nmi_handler+0x4f/frame 0xfffffe00816f4e00 trap() at trap+0x78/frame 0xfffffe00816f4f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00816f4f20 --- trap 0x13, rip = 0xffffffff81822836, rsp = 0xfffffe0128f589f0, rbp = 0xfffffe0128f589f0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0128f589f0 acpi_cpu_idle() at acpi_cpu_idle+0x3b0/frame 0xfffffe0128f58a50 cpu_idle_acpi() at cpu_idle_acpi+0x46/frame 0xfffffe0128f58a70 cpu_idle() at cpu_idle+0xdb/frame 0xfffffe0128f58aa0 sched_idletd() at sched_idletd+0x5c9/frame 0xfffffe0128f58bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f58bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f58bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xfffffe0129294c00 (CPU 10) cpustop_handler() at cpustop_handler+0x32/frame 0xfffffe0081701dd0 ipi_nmi_handler() at ipi_nmi_handler+0x4f/frame 0xfffffe0081701e00 trap() at trap+0x78/frame 0xfffffe0081701f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0081701f20 --- trap 0x13, rip = 0xffffffff81822836, rsp = 0xfffffe0128f679f0, rbp = 0xfffffe0128f679f0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0128f679f0 acpi_cpu_idle() at acpi_cpu_idle+0x3b0/frame 0xfffffe0128f67a50 cpu_idle_acpi() at cpu_idle_acpi+0x46/frame 0xfffffe0128f67a70 cpu_idle() at cpu_idle+0xdb/frame 0xfffffe0128f67aa0 sched_idletd() at sched_idletd+0x5c9/frame 0xfffffe0128f67bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f67bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f67bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100014 td 0xfffffe0129294500 (CPU 11) cpustop_handler() at cpustop_handler+0x32/frame 0xfffffe008170ddd0 ipi_nmi_handler() at ipi_nmi_handler+0x4f/frame 0xfffffe008170de00 trap() at trap+0x78/frame 0xfffffe008170df20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe008170df20 --- trap 0x13, rip = 0xffffffff81822836, rsp = 0xfffffe0128f629f0, rbp = 0xfffffe0128f629f0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0128f629f0 acpi_cpu_idle() at acpi_cpu_idle+0x3b0/frame 0xfffffe0128f62a50 cpu_idle_acpi() at cpu_idle_acpi+0x46/frame 0xfffffe0128f62a70 cpu_idle() at cpu_idle+0xdb/frame 0xfffffe0128f62aa0 sched_idletd() at sched_idletd+0x5c9/frame 0xfffffe0128f62bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f62bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f62bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command init pid 1 tid 100002 td 0xfffffe012928aa00 sched_switch() at sched_switch+0x90f/frame 0xfffffe007bff9650 mi_switch() at mi_switch+0x268/frame 0xfffffe007bff9680 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe007bff96d0 sleepq_catch_signals() at sleepq_catch_signals+0x2fe/frame 0xfffffe007bff9720 sleepq_wait_sig() at sleepq_wait_sig+0xb/frame 0xfffffe007bff9740 _sleep() at _sleep+0x324/frame 0xfffffe007bff97f0 kern_wait6() at kern_wait6+0x5df/frame 0xfffffe007bff98a0 sys_wait4() at sys_wait4+0xa3/frame 0xfffffe007bff9aa0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe007bff9bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe007bff9bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x28d18a, rsp = 0x7fffffffe718, rbp = 0x7fffffffe780 --- Tracing command audit pid 10 tid 100001 td 0xfffffe012928b100 sched_switch() at sched_switch+0x90f/frame 0xfffffe007bffea40 mi_switch() at mi_switch+0x268/frame 0xfffffe007bffea70 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe007bffeac0 _cv_wait() at _cv_wait+0x1fc/frame 0xfffffe007bffeb30 audit_worker() at audit_worker+0xbb/frame 0xfffffe007bffebb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe007bffebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007bffebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xffffffff83557780 sched_switch() at sched_switch+0x90f/frame 0xffffffff83cd5a70 mi_switch() at mi_switch+0x268/frame 0xffffffff83cd5aa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xffffffff83cd5af0 sleepq_timedwait() at sleepq_timedwait+0x60/frame 0xffffffff83cd5b40 _sleep() at _sleep+0x30e/frame 0xffffffff83cd5bf0 swapper() at swapper+0x90/frame 0xffffffff83cd5c60 mi_startup() at mi_startup+0x402/frame 0xffffffff83cd5cb0 btext() at btext+0x2c Tracing command kernel pid 0 tid 100015 td 0xfffffe0129293e00 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128f71a70 mi_switch() at mi_switch+0x268/frame 0xfffffe0128f71aa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128f71af0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe0128f71b70 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x12a/frame 0xfffffe0128f71bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f71bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f71bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xfffffe0129293700 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128f6ca70 mi_switch() at mi_switch+0x268/frame 0xfffffe0128f6caa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128f6caf0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe0128f6cb70 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x12a/frame 0xfffffe0128f6cbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f6cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f6cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100017 td 0xfffffe0129293000 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128f7ba70 mi_switch() at mi_switch+0x268/frame 0xfffffe0128f7baa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128f7baf0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe0128f7bb70 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x12a/frame 0xfffffe0128f7bbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f7bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f7bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100018 td 0xfffffe0129296800 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128f76a70 mi_switch() at mi_switch+0x268/frame 0xfffffe0128f76aa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128f76af0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe0128f76b70 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x12a/frame 0xfffffe0128f76bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f76bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f76bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xfffffe012932b100 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128f85a70 mi_switch() at mi_switch+0x268/frame 0xfffffe0128f85aa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128f85af0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe0128f85b70 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x12a/frame 0xfffffe0128f85bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f85bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f85bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xfffffe012932aa00 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128f80a70 mi_switch() at mi_switch+0x268/frame 0xfffffe0128f80aa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128f80af0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe0128f80b70 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x12a/frame 0xfffffe0128f80bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f80bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f80bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xfffffe012932a300 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128f8fa70 mi_switch() at mi_switch+0x268/frame 0xfffffe0128f8faa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128f8faf0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe0128f8fb70 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x12a/frame 0xfffffe0128f8fbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f8fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f8fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xfffffe0129329c00 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128f8aa70 mi_switch() at mi_switch+0x268/frame 0xfffffe0128f8aaa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128f8aaf0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe0128f8ab70 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x12a/frame 0xfffffe0128f8abb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f8abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f8abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xfffffe0129329500 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128f99a70 mi_switch() at mi_switch+0x268/frame 0xfffffe0128f99aa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128f99af0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe0128f99b70 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x12a/frame 0xfffffe0128f99bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f99bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f99bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xfffffe0129328e00 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128f94a70 mi_switch() at mi_switch+0x268/frame 0xfffffe0128f94aa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128f94af0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe0128f94b70 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x12a/frame 0xfffffe0128f94bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f94bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f94bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xfffffe0129328700 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128fa3a70 mi_switch() at mi_switch+0x268/frame 0xfffffe0128fa3aa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128fa3af0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe0128fa3b70 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x12a/frame 0xfffffe0128fa3bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128fa3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128fa3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100026 td 0xfffffe0129328000 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128f9ea70 mi_switch() at mi_switch+0x268/frame 0xfffffe0128f9eaa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128f9eaf0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe0128f9eb70 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x12a/frame 0xfffffe0128f9ebb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128f9ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128f9ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100027 td 0xfffffe012932b800 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128fada70 mi_switch() at mi_switch+0x268/frame 0xfffffe0128fadaa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128fadaf0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe0128fadb70 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x12a/frame 0xfffffe0128fadbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128fadbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128fadbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100028 td 0xfffffe0129332100 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128fa8a70 mi_switch() at mi_switch+0x268/frame 0xfffffe0128fa8aa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128fa8af0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe0128fa8b70 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x12a/frame 0xfffffe0128fa8bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128fa8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128fa8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100029 td 0xfffffe0129331a00 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128fb7a70 mi_switch() at mi_switch+0x268/frame 0xfffffe0128fb7aa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128fb7af0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe0128fb7b70 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x12a/frame 0xfffffe0128fb7bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128fb7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128fb7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100030 td 0xfffffe0129331300 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128fb2a70 mi_switch() at mi_switch+0x268/frame 0xfffffe0128fb2aa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128fb2af0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe0128fb2b70 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x12a/frame 0xfffffe0128fb2bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128fb2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128fb2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100031 td 0xfffffe0129330c00 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128fc1a70 mi_switch() at mi_switch+0x268/frame 0xfffffe0128fc1aa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128fc1af0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe0128fc1b70 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x12a/frame 0xfffffe0128fc1bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128fc1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128fc1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100032 td 0xfffffe0129330500 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128fbca70 mi_switch() at mi_switch+0x268/frame 0xfffffe0128fbcaa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128fbcaf0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe0128fbcb70 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x12a/frame 0xfffffe0128fbcbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128fbcbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128fbcbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100033 td 0xfffffe012932fe00 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128fcba70 mi_switch() at mi_switch+0x268/frame 0xfffffe0128fcbaa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128fcbaf0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe0128fcbb70 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x12a/frame 0xfffffe0128fcbbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128fcbbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128fcbbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100034 td 0xfffffe012932f700 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128fc6a70 mi_switch() at mi_switch+0x268/frame 0xfffffe0128fc6aa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128fc6af0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe0128fc6b70 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x12a/frame 0xfffffe0128fc6bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128fc6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128fc6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100035 td 0xfffffe012932f000 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128fd5a70 mi_switch() at mi_switch+0x268/frame 0xfffffe0128fd5aa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128fd5af0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe0128fd5b70 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x12a/frame 0xfffffe0128fd5bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128fd5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128fd5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100036 td 0xfffffe0129332800 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128fd0a70 mi_switch() at mi_switch+0x268/frame 0xfffffe0128fd0aa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128fd0af0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe0128fd0b70 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x12a/frame 0xfffffe0128fd0bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128fd0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128fd0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100037 td 0xfffffe012933a100 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128fdfa70 mi_switch() at mi_switch+0x268/frame 0xfffffe0128fdfaa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128fdfaf0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe0128fdfb70 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x12a/frame 0xfffffe0128fdfbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128fdfbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128fdfbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100038 td 0xfffffe0129339a00 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128fdaa70 mi_switch() at mi_switch+0x268/frame 0xfffffe0128fdaaa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128fdaaf0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe0128fdab70 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x12a/frame 0xfffffe0128fdabb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128fdabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128fdabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100039 td 0xfffffe0129339300 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128fe9a70 mi_switch() at mi_switch+0x268/frame 0xfffffe0128fe9aa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128fe9af0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe0128fe9b70 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x12a/frame 0xfffffe0128fe9bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128fe9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128fe9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100040 td 0xfffffe0129338c00 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128fe4a40 mi_switch() at mi_switch+0x268/frame 0xfffffe0128fe4a70 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128fe4ac0 _sleep() at _sleep+0x31a/frame 0xfffffe0128fe4b70 taskqueue_thread_loop() at taskqueue_thread_loop+0x10b/frame 0xfffffe0128fe4bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128fe4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128fe4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100041 td 0xfffffe0129338500 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128ff3a40 mi_switch() at mi_switch+0x268/frame 0xfffffe0128ff3a70 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128ff3ac0 _sleep() at _sleep+0x31a/frame 0xfffffe0128ff3b70 taskqueue_thread_loop() at taskqueue_thread_loop+0x10b/frame 0xfffffe0128ff3bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128ff3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128ff3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100043 td 0xfffffe0129337700 sched_switch() at sched_switch+0x90f/frame 0xfffffe0128ffda40 mi_switch() at mi_switch+0x268/frame 0xfffffe0128ffda70 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe0128ffdac0 _sleep() at _sleep+0x31a/frame 0xfffffe0128ffdb70 taskqueue_thread_loop() at taskqueue_thread_loop+0x10b/frame 0xfffffe0128ffdbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe0128ffdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0128ffdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100045 td 0xfffffe012933a800 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c209a40 mi_switch() at mi_switch+0x268/frame 0xfffffe014c209a70 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c209ac0 _sleep() at _sleep+0x31a/frame 0xfffffe014c209b70 taskqueue_thread_loop() at taskqueue_thread_loop+0x10b/frame 0xfffffe014c209bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c209bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c209bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100046 td 0xfffffe0129347100 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c204a40 mi_switch() at mi_switch+0x268/frame 0xfffffe014c204a70 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c204ac0 _sleep() at _sleep+0x31a/frame 0xfffffe014c204b70 taskqueue_thread_loop() at taskqueue_thread_loop+0x10b/frame 0xfffffe014c204bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c204bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c204bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100062 td 0xfffffe012935f500 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c29fa40 mi_switch() at mi_switch+0x268/frame 0xfffffe014c29fa70 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c29fac0 _sleep() at _sleep+0x31a/frame 0xfffffe014c29fb70 taskqueue_thread_loop() at taskqueue_thread_loop+0x10b/frame 0xfffffe014c29fbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c29fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c29fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100067 td 0xfffffe014c41d100 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c2aea40 mi_switch() at mi_switch+0x268/frame 0xfffffe014c2aea70 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c2aeac0 _sleep() at _sleep+0x31a/frame 0xfffffe014c2aeb70 taskqueue_thread_loop() at taskqueue_thread_loop+0x10b/frame 0xfffffe014c2aebb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c2aebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c2aebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100068 td 0xfffffe014c41ca00 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c2bda40 mi_switch() at mi_switch+0x268/frame 0xfffffe014c2bda70 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c2bdac0 _sleep() at _sleep+0x31a/frame 0xfffffe014c2bdb70 taskqueue_thread_loop() at taskqueue_thread_loop+0x10b/frame 0xfffffe014c2bdbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c2bdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c2bdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100069 td 0xfffffe014c41c300 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c2b8a40 mi_switch() at mi_switch+0x268/frame 0xfffffe014c2b8a70 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c2b8ac0 _sleep() at _sleep+0x31a/frame 0xfffffe014c2b8b70 taskqueue_thread_loop() at taskqueue_thread_loop+0x10b/frame 0xfffffe014c2b8bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c2b8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c2b8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100070 td 0xfffffe014c41bc00 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c2c7a40 mi_switch() at mi_switch+0x268/frame 0xfffffe014c2c7a70 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c2c7ac0 _sleep() at _sleep+0x31a/frame 0xfffffe014c2c7b70 taskqueue_thread_loop() at taskqueue_thread_loop+0x10b/frame 0xfffffe014c2c7bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c2c7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c2c7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100071 td 0xfffffe014c41b500 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c2c2a40 mi_switch() at mi_switch+0x268/frame 0xfffffe014c2c2a70 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c2c2ac0 _sleep() at _sleep+0x31a/frame 0xfffffe014c2c2b70 taskqueue_thread_loop() at taskqueue_thread_loop+0x10b/frame 0xfffffe014c2c2bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c2c2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c2c2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100072 td 0xfffffe014c41ae00 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c2d1a40 mi_switch() at mi_switch+0x268/frame 0xfffffe014c2d1a70 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c2d1ac0 _sleep() at _sleep+0x31a/frame 0xfffffe014c2d1b70 taskqueue_thread_loop() at taskqueue_thread_loop+0x10b/frame 0xfffffe014c2d1bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c2d1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c2d1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100073 td 0xfffffe014c41a700 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c2cca40 mi_switch() at mi_switch+0x268/frame 0xfffffe014c2cca70 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c2ccac0 _sleep() at _sleep+0x31a/frame 0xfffffe014c2ccb70 taskqueue_thread_loop() at taskqueue_thread_loop+0x10b/frame 0xfffffe014c2ccbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c2ccbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c2ccbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100074 td 0xfffffe014c41a000 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c2dba40 mi_switch() at mi_switch+0x268/frame 0xfffffe014c2dba70 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c2dbac0 _sleep() at _sleep+0x31a/frame 0xfffffe014c2dbb70 taskqueue_thread_loop() at taskqueue_thread_loop+0x10b/frame 0xfffffe014c2dbbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c2dbbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c2dbbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100075 td 0xfffffe014c41d800 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c2d6a40 mi_switch() at mi_switch+0x268/frame 0xfffffe014c2d6a70 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c2d6ac0 _sleep() at _sleep+0x31a/frame 0xfffffe014c2d6b70 taskqueue_thread_loop() at taskqueue_thread_loop+0x10b/frame 0xfffffe014c2d6bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c2d6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c2d6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100076 td 0xfffffe014c422100 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c2e5a40 mi_switch() at mi_switch+0x268/frame 0xfffffe014c2e5a70 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c2e5ac0 _sleep() at _sleep+0x31a/frame 0xfffffe014c2e5b70 taskqueue_thread_loop() at taskqueue_thread_loop+0x10b/frame 0xfffffe014c2e5bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c2e5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c2e5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100077 td 0xfffffe014c421a00 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c2e0a40 mi_switch() at mi_switch+0x268/frame 0xfffffe014c2e0a70 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c2e0ac0 _sleep() at _sleep+0x31a/frame 0xfffffe014c2e0b70 taskqueue_thread_loop() at taskqueue_thread_loop+0x10b/frame 0xfffffe014c2e0bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c2e0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c2e0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100078 td 0xfffffe014c421300 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c2efa40 mi_switch() at mi_switch+0x268/frame 0xfffffe014c2efa70 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c2efac0 _sleep() at _sleep+0x31a/frame 0xfffffe014c2efb70 taskqueue_thread_loop() at taskqueue_thread_loop+0x10b/frame 0xfffffe014c2efbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c2efbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c2efbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100116 td 0xfffffe014f23a700 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c3b4a70 mi_switch() at mi_switch+0x268/frame 0xfffffe014c3b4aa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c3b4af0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe014c3b4b70 taskqueue_thread_loop() at taskqueue_thread_loop+0x12a/frame 0xfffffe014c3b4bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c3b4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c3b4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100121 td 0xfffffe014c597a00 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c33aa70 mi_switch() at mi_switch+0x268/frame 0xfffffe014c33aaa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c33aaf0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe014c33ab70 taskqueue_thread_loop() at taskqueue_thread_loop+0x12a/frame 0xfffffe014c33abb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c33abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c33abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100122 td 0xfffffe014c597300 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c3efa70 mi_switch() at mi_switch+0x268/frame 0xfffffe014c3efaa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c3efaf0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe014c3efb70 taskqueue_thread_loop() at taskqueue_thread_loop+0x12a/frame 0xfffffe014c3efbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c3efbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c3efbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100123 td 0xfffffe014c596c00 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c3eaa70 mi_switch() at mi_switch+0x268/frame 0xfffffe014c3eaaa0 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c3eaaf0 msleep_spin_sbt() at msleep_spin_sbt+0x18c/frame 0xfffffe014c3eab70 taskqueue_thread_loop() at taskqueue_thread_loop+0x12a/frame 0xfffffe014c3eabb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c3eabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c3eabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100125 td 0xfffffe014c595e00 sched_switch() at sched_switch+0x90f/frame 0xfffffe014c3f4a40 mi_switch() at mi_switch+0x268/frame 0xfffffe014c3f4a70 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe014c3f4ac0 _sleep() at _sleep+0x31a/frame 0xfffffe014c3f4b70 taskqueue_thread_loop() at taskqueue_thread_loop+0x10b/frame 0xfffffe014c3f4bb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe014c3f4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe014c3f4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100365 td 0xfffffe01e7e3c000 sched_switch() at sched_switch+0x90f/frame 0xfffffe01e783fa40 mi_switch() at mi_switch+0x268/frame 0xfffffe01e783fa70 sleepq_switch() at sleepq_switch+0x1b1/frame 0xfffffe01e783fac0 _sleep() at _sleep+0x31a/frame 0xfffffe01e783fb70 taskqueue_thread_loop() at taskqueue_thread_loop+0x10b/frame 0xfffffe01e783fbb0 fork_exit() at fork_exit+0xa1/frame 0xfffffe01e783fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01e783fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db> reset cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 8 (kgdb) bt #0 doadump (textdump=0x0) at include/pcpu_aux.h:55 #1 0xffffffff80564606 in db_dump (dummy=Unhandled dwarf expression opcode 0xa3 ) at ../../../ddb/db_command.c:575 #2 0xffffffff805643c4 in db_command (cmd_table=, dopager=0x0) at ../../../ddb/db_command.c:482 #3 0xffffffff8056ad88 in db_script_exec (scriptname=, warnifnotfound=) at ../../../ddb/db_script.c:304 #4 0xffffffff805643c4 in db_command (cmd_table=, dopager=0x0) at ../../../ddb/db_command.c:482 #5 0xffffffff8056ad88 in db_script_exec (scriptname=, warnifnotfound=) at ../../../ddb/db_script.c:304 #6 0xffffffff805643c4 in db_command (cmd_table=, dopager=0x1) at ../../../ddb/db_command.c:482 #7 0xffffffff80563fbd in db_command_loop () at ../../../ddb/db_command.c:535 #8 0xffffffff805681e0 in db_trap (type=, code=) at ../../../ddb/db_main.c:270 #9 0xffffffff8121accf in kdb_trap (type=0x3, code=0x0, tf=0xffffffff805645d0) at ../../../kern/subr_kdb.c:699 #10 0xffffffff8187d8ad in trap (frame=0x0) at ../../../amd64/amd64/trap.c:576 #11 0xffffffff81840c78 in calltrap () at ../../../amd64/amd64/exception.S:290 #12 0xffffffff8121a027 in kdb_enter (why=0xffffffff822073c0 "panic", msg=) at include/cpufunc.h:65 #13 0xffffffff811a2819 in vpanic (fmt=0xffffffff81a284fd "ASan: Invalid access %zu-byte %s at %#lx, %s", ap=) at include/pcpu_aux.h:55 #14 0xffffffff811a24ff in panic (fmt=) at ../../../kern/kern_shutdown.c:843 #15 0xffffffff811f530b in __asan_load4_noabort (addr=) at ../../../kern/subr_asan.c:187 #16 0xffffffff815c8bfa in cgaccount (cg=0x0, vp=0xfffffe01e7567500, nbp=0xfffffe007c86d588, passno=0x1) at ../../../ufs/ffs/ffs_snapshot.c:918 #17 0xffffffff815c61f4 in ffs_snapshot (mp=, snapfile=0xfffffe01e7567500 "\001") at ../../../ufs/ffs/ffs_snapshot.c:398 #18 0xffffffff816085d6 in ffs_mount (mp=0x0) at ../../../ufs/ffs/ffs_vfsops.c:703 #19 0xffffffff812ef16f in vfs_domount (td=0xfffffe01293cb270, fstype=, fspath=, fsflags=, optlist=) at ../../../kern/vfs_mount.c:1152 #20 0xffffffff812ed137 in vfs_donmount (td=, fsflags=0x0, fsoptions=0xfffffe01d1916000) at ../../../kern/vfs_mount.c:787 #21 0xffffffff812ec629 in sys_nmount (td=0xfffffe01d1916000, uap=0xfffffe01e7fc1350) at ../../../kern/vfs_mount.c:441 #22 0xffffffff8187edb2 in amd64_syscall (td=0xffffffff81a284fd, traced=0x0) at subr_syscall.c:189 #23 0xffffffff8184159e in fast_syscall_common () at ../../../amd64/amd64/exception.S:584 #24 0x000000080038468a in ?? () Previous frame inner to this frame (corrupt stack?) Current language: auto; currently minimal (kgdb) f 17 #17 0xffffffff815c61f4 in ffs_snapshot (mp=, snapfile=0xfffffe01e7567500 "\001") at ../../../ufs/ffs/ffs_snapshot.c:398 398 error = cgaccount(cg, vp, nbp, 1); (kgdb) l 393 for (cg = 0; cg < fs->fs_ncg; cg++) { 394 error = UFS_BALLOC(vp, lfragtosize(fs, cgtod(fs, cg)), 395 fs->fs_bsize, KERNCRED, 0, &nbp); 396 if (error) 397 goto out; 398 error = cgaccount(cg, vp, nbp, 1); 399 bawrite(nbp); 400 if (cg % 10 == 0 && error == 0) 401 error = ffs_syncvnode(vp, MNT_WAIT, 0); 402 if (error) (kgdb) info loc nd = {ni_dirp = 0xfffffe01293cb230 "/mnt/.snap/snap", ni_segflg = UIO_SYSSPACE, ni_rightsneeded = 0xffffffff82a08708, ni_startdir = 0x0, ni_rootdir = 0xfffffe01d0603c80, ni_topdir = 0x0, ni_dirfd = 0xffffff9c, ni_lcf = 0x0, ni_filecaps = {fc_rights = { cr_rights = 0xfffffe01d50a9210}, fc_ioctls = 0x0, fc_nioctls = 0xffff, fc_fcntls = 0x0}, ni_vp = 0xfffffe01e7567500, ni_dvp = 0xfffffe01e74e3280, ni_resflags = 0x1, ni_debugflags = 0x3, ni_loopcnt = 0x0, ni_pathlen = 0x1, ni_next = 0xfffffe01d06ffc0f "", ni_cnd = { cn_origflags = 0xd, cn_flags = 0x1100040d, cn_thread = 0xfffffe01d1916000, cn_cred = 0xfffffe01d4a05800, cn_nameiop = CREATE, cn_lkflags = 0x80000, cn_pnbuf = 0xfffffe01d06ffc00 "/mnt/.snap/snap", cn_nameptr = 0xfffffe01d06ffc0b "snap", cn_namelen = 0x4}, ni_cap_tracker = { tqh_first = 0x0, tqh_last = 0xfffffe01d50a9298}, ni_beneath_latch = 0xffffffffffffffff} vat = {va_type = VREG, va_mode = 0x100, va_padding0 = 0xffff, va_uid = 0xffffffff, va_gid = 0xffffffff, va_nlink = 0xffffffffffffffff, va_fsid = 0xffffffffffffffff, va_fileid = 0xffffffffffffffff, va_size = 0xffffffffffffffff, va_blocksize = 0xffffffffffffffff, va_atime = { tv_sec = 0xffffffffffffffff, tv_nsec = 0xffffffffffffffff}, va_mtime = {tv_sec = 0xffffffffffffffff, tv_nsec = 0xffffffffffffffff}, va_ctime = {tv_sec = 0xffffffffffffffff, tv_nsec = 0xffffffffffffffff}, va_birthtime = {tv_sec = 0xffffffffffffffff, tv_nsec = 0xffffffffffffffff}, va_gen = 0xffffffffffffffff, va_flags = 0xffffffffffffffff, va_rdev = 0xffffffffffffffff, va_bytes = 0xffffffffffffffff, va_filerev = 0xffffffff835d8e60, va_vaflags = 0x2, va_spare = 0xfffffe01d50a9240} auio = {uio_iov = 0xffffffff831718c0, uio_iovcnt = 0xe74e3280, uio_offset = 0xfffffe01d50a9230, uio_resid = 0xfffffe01d50a9258, uio_segflg = 3574239528, uio_rw = 4294966785, uio_td = 0x10000} aiov = {iov_base = 0xffffffff8115a9a8, iov_len = 0x17fd70001} starttime = {tv_sec = 0x0, tv_nsec = 0x0} endtime = {tv_sec = 0xfffffe01d50a93d0, tv_nsec = 0xffffffff8164d0ad} copy_fs = (struct fs *) Cannot access memory at address 0x0 (kgdb) Index: src/sys/amd64/amd64/machdep.c =================================================================== --- src/sys/amd64/amd64/machdep.c (revision 368820) +++ src/sys/amd64/amd64/machdep.c (working copy) @@ -58,6 +58,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -1934,6 +1935,10 @@ hammer_time(u_int64_t modulep, u_int64_t physfree) #endif thread0.td_critnest = 0; +#ifdef KASAN + kasan_init(); +#endif + TSEXIT(); /* Location of kernel stack for locore */ Index: src/sys/amd64/amd64/pmap.c =================================================================== --- src/sys/amd64/amd64/pmap.c (revision 368820) +++ src/sys/amd64/amd64/pmap.c (working copy) @@ -112,6 +112,7 @@ __FBSDID("$FreeBSD$"); #include "opt_vm.h" #include +#include #include #include #include @@ -419,11 +420,15 @@ static int pat_index[PAT_INDEX_SIZE]; /* cache mod static u_int64_t KPTphys; /* phys addr of kernel level 1 */ static u_int64_t KPDphys; /* phys addr of kernel level 2 */ -u_int64_t KPDPphys; /* phys addr of kernel level 3 */ +static u_int64_t KPDPphys; /* phys addr of kernel level 3 */ u_int64_t KPML4phys; /* phys addr of kernel level 4 */ u_int64_t KPML5phys; /* phys addr of kernel level 5, if supported */ +#ifdef KASAN +static uint64_t KASANPDPphys; +#endif + static pml4_entry_t *kernel_pml4; static u_int64_t DMPDphys; /* phys addr of direct mapped level 2 */ static u_int64_t DMPDPphys; /* phys addr of direct mapped level 3 */ @@ -1608,11 +1613,16 @@ bootaddr_rwx(vm_paddr_t pa) static void create_pagetables(vm_paddr_t *firstaddr) { - int i, j, ndm1g, nkpdpe, nkdmpde; pd_entry_t *pd_p; pdp_entry_t *pdp_p; pml4_entry_t *p4_p; uint64_t DMPDkernphys; +#ifdef KASAN + pt_entry_t *pt_p; + uint64_t KASANPDphys, KASANPTphys, KASANphys; + int nkasanpte; +#endif + int i, j, ndm1g, nkpdpe, nkdmpde; /* Allocate page table pages for the direct map */ ndmpdp = howmany(ptoa(Maxmem), NBPDP); @@ -1652,6 +1662,10 @@ create_pagetables(vm_paddr_t *firstaddr) /* Allocate pages */ KPML4phys = allocpages(firstaddr, 1); KPDPphys = allocpages(firstaddr, NKPML4E); +#ifdef KASAN + KASANPDPphys = allocpages(firstaddr, NKASANPML4E); + KASANPDphys = allocpages(firstaddr, 1); +#endif /* * Allocate the initial number of kernel page table pages required to @@ -1669,6 +1683,12 @@ create_pagetables(vm_paddr_t *firstaddr) KPTphys = allocpages(firstaddr, nkpt); KPDphys = allocpages(firstaddr, nkpdpe); +#ifdef KASAN + nkasanpte = howmany(nkpt, KASAN_SHADOW_SCALE); + KASANPTphys = allocpages(firstaddr, nkasanpte); + KASANphys = allocpages(firstaddr, nkasanpte * NPTEPG); +#endif + /* * Connect the zero-filled PT pages to their PD entries. This * implicitly maps the PT pages at their correct locations within @@ -1701,6 +1721,21 @@ create_pagetables(vm_paddr_t *firstaddr) for (i = 0; i < nkpdpe; i++) pdp_p[i + KPDPI] = (KPDphys + ptoa(i)) | X86_PG_RW | X86_PG_V; +#ifdef KASAN + pdp_p = (pdp_entry_t *)KASANPDPphys; + pdp_p[255] = (KASANPDphys | X86_PG_RW | X86_PG_V | pg_nx); + + pd_p = (pd_entry_t *)KASANPDphys; + for (i = 0; i < nkasanpte; i++) + pd_p[i + KASANPDI] = (KASANPTphys + ptoa(i)) | X86_PG_RW | + X86_PG_V | pg_nx; + + pt_p = (pt_entry_t *)KASANPTphys; + for (i = 0; i < nkasanpte * NPTEPG; i++) + pt_p[i] = (KASANphys + ptoa(i)) | X86_PG_RW | X86_PG_V | + X86_PG_M | X86_PG_A | pg_nx; +#endif + /* * Now, set up the direct map region using 2MB and/or 1GB pages. If * the end of physical memory is not aligned to a 1GB page boundary, @@ -1749,7 +1784,15 @@ create_pagetables(vm_paddr_t *firstaddr) p4_p[PML4PML4I] = KPML4phys; p4_p[PML4PML4I] |= X86_PG_RW | X86_PG_V | pg_nx; - /* Connect the Direct Map slot(s) up to the PML4. */ +#ifdef KASAN + /* Connect the KASAN shadow map slots up to the PML4. */ + for (i = 0; i < NKASANPML4E; i++) { + p4_p[KASANPML4I + i] = KASANPDPphys + ptoa(i); + p4_p[KASANPML4I + i] |= X86_PG_RW | X86_PG_V | pg_nx; + } +#endif + + /* Connect the Direct Map slots up to the PML4. */ for (i = 0; i < ndmpdpphys; i++) { p4_p[DMPML4I + i] = DMPDPphys + ptoa(i); p4_p[DMPML4I + i] |= X86_PG_RW | X86_PG_V | pg_nx; @@ -3984,6 +4027,12 @@ pmap_pinit_pml4(vm_page_t pml4pg) pm_pml4[KPML4BASE + i] = (KPDPphys + ptoa(i)) | X86_PG_RW | X86_PG_V; } +#ifdef KASAN + for (i = 0; i < NKASANPML4E; i++) { + pm_pml4[KASANPML4I + i] = (KASANPDPphys + ptoa(i)) | X86_PG_RW | + X86_PG_V | pg_nx; + } +#endif for (i = 0; i < ndmpdpphys; i++) { pm_pml4[DMPML4I + i] = (DMPDPphys + ptoa(i)) | X86_PG_RW | X86_PG_V; @@ -4533,6 +4582,10 @@ pmap_release(pmap_t pmap) } else { for (i = 0; i < NKPML4E; i++) /* KVA */ pmap->pm_pmltop[KPML4BASE + i] = 0; +#ifdef KASAN + for (i = 0; i < NKASANPML4E; i++) /* KASAN shadow map */ + pmap->pm_pmltop[KASANPML4I + i] = 0; +#endif for (i = 0; i < ndmpdpphys; i++)/* Direct Map */ pmap->pm_pmltop[DMPML4I + i] = 0; pmap->pm_pmltop[PML4PML4I] = 0; /* Recursive Mapping */ @@ -4616,6 +4669,34 @@ pmap_page_array_startup(long pages) pde_store(pde, newpdir); } vm_page_array = (vm_page_t)start; +#ifdef KASAN + vm_paddr_t dummypa = vm_phys_early_alloc(-1, PAGE_SIZE); + pagezero((void *)PHYS_TO_DMAP(dummypa)); + pt_entry_t *pte; + + start = KASAN_MIN_ADDRESS; + end = start + ((pages * sizeof(struct vm_page)) >> KASAN_SHADOW_SCALE_SHIFT); + for (va = start; va < end; va += PAGE_SIZE) { + pdpe = pmap_pdpe(kernel_pmap, va); + if ((*pdpe & X86_PG_V) == 0) { + pa = vm_phys_early_alloc(-1, PAGE_SIZE); + pagezero((void *)PHYS_TO_DMAP(pa)); + *pdpe = (pdp_entry_t)(pa | X86_PG_V | X86_PG_RW | + X86_PG_A | X86_PG_M | pg_nx); + } + pde = pmap_pdpe_to_pde(pdpe, va); + if ((*pde & X86_PG_V) == 0) { + pa = vm_phys_early_alloc(-1, PAGE_SIZE); + pagezero((void *)PHYS_TO_DMAP(pa)); + *pde = (pd_entry_t)(pa | X86_PG_V | X86_PG_RW | + X86_PG_A | X86_PG_M | pg_nx); + } + pte = pmap_pde_to_pte(pde, va); + *pte = (pt_entry_t)(dummypa | X86_PG_V | X86_PG_RW | X86_PG_A | + X86_PG_M | pg_nx); + } + /* XXXMJ missing gap at the end of the array */ +#endif } /* @@ -4652,6 +4733,10 @@ pmap_growkernel(vm_offset_t addr) addr = roundup2(addr, NBPDR); if (addr - 1 >= vm_map_max(kernel_map)) addr = vm_map_max(kernel_map); +#ifdef KASAN + if (kernel_vm_end < addr) + kasan_shadow_map((void *)kernel_vm_end, addr - kernel_vm_end); +#endif while (kernel_vm_end < addr) { pdpe = pmap_pdpe(kernel_pmap, kernel_vm_end); if ((*pdpe & X86_PG_V) == 0) { @@ -11006,7 +11091,59 @@ pmap_pkru_clear(pmap_t pmap, vm_offset_t sva, vm_o return (error); } +#ifdef KASAN +static vm_page_t +pmap_kasan_enter_alloc(void) +{ + vm_page_t m; + + m = vm_page_alloc(NULL, 0, VM_ALLOC_INTERRUPT | VM_ALLOC_NOOBJ | + VM_ALLOC_WIRED | VM_ALLOC_ZERO); + if (m == NULL) + panic("%s: no memory to grow shadow map", __func__); + if ((m->flags & PG_ZERO) == 0) + pmap_zero_page(m); + return (m); +} + /* + * Grow the shadow map. + */ +void __nosanitizeaddress +pmap_kasan_enter(vm_offset_t va) +{ + pdp_entry_t *pdpe; + pd_entry_t *pde; + pt_entry_t *pte; + vm_page_t m; + + pdpe = pmap_pdpe(kernel_pmap, va); + if ((*pdpe & X86_PG_V) == 0) { + m = pmap_kasan_enter_alloc(); + *pdpe = (pdp_entry_t)(VM_PAGE_TO_PHYS(m) | X86_PG_RW | + X86_PG_V | pg_nx); + } + pde = pmap_pdpe_to_pde(pdpe, va); + if ((*pde & X86_PG_V) == 0) { + m = pmap_kasan_enter_alloc(); + *pde = (pd_entry_t)(VM_PAGE_TO_PHYS(m) | X86_PG_RW | + X86_PG_V | pg_nx); + } + pte = pmap_pde_to_pte(pde, va); + /* XXXMJ double mapping for vm_page_array[] */ + if ((*pte & X86_PG_V) != 0) + return; +#if 0 + KASSERT((*pte & X86_PG_V) == 0, + ("%s: shadow address %#lx is already mapped", __func__, va)); +#endif + m = pmap_kasan_enter_alloc(); + *pte = (pt_entry_t)(VM_PAGE_TO_PHYS(m) | X86_PG_RW | X86_PG_V | + X86_PG_M | X86_PG_A | pg_nx); +} +#endif + +/* * Track a range of the kernel's virtual address space that is contiguous * in various mapping attributes. */ @@ -11183,6 +11320,11 @@ sysctl_kmaps(SYSCTL_HANDLER_ARGS) case DMPML4I: sbuf_printf(sb, "\nDirect map:\n"); break; +#ifdef KASAN + case KASANPML4I: + sbuf_printf(sb, "\nKASAN shadow map:\n"); + break; +#endif case KPML4BASE: sbuf_printf(sb, "\nKernel map:\n"); break; Index: src/sys/amd64/include/pmap.h =================================================================== --- src/sys/amd64/include/pmap.h (revision 368820) +++ src/sys/amd64/include/pmap.h (working copy) @@ -196,6 +196,12 @@ #define NKPML4E 4 /* + * Number of PML4 slots for the KASAN shadow map. It requires 1 byte of memory + * for every 8 bytes of the kernel address space. + */ +#define NKASANPML4E ((NKPML4E + 7) / 8) + +/* * We use the same numbering of the page table pages for 5-level and * 4-level paging structures. */ @@ -239,13 +245,16 @@ #define KPML4BASE (NPML4EPG-NKPML4E) /* KVM at highest addresses */ #define DMPML4I rounddown(KPML4BASE-NDMPML4E, NDMPML4E) /* Below KVM */ +#define KASANPML4I (DMPML4I - NKASANPML4E) /* Below the direct map */ +/* XXXMJ why not put it directly below the kernel map? */ #define KPML4I (NPML4EPG-1) #define KPDPI (NPDPEPG-2) /* kernbase at -2GB */ +#define KASANPDI (NPDEPG - 128) /* XXXMJ comment */ /* Large map: index of the first and max last pml4 entry */ #define LMSPML4I (PML4PML4I + 1) -#define LMEPML4I (DMPML4I - 1) +#define LMEPML4I (DMPML4I - 2) /* XXXMJ */ /* * XXX doesn't really belong here I guess... @@ -303,7 +312,6 @@ typedef u_int64_t pml5_entry_t; #define P5Dmap ((pd_entry_t *)(addr_P5Dmap)) extern int nkpt; /* Initial number of kernel page tables */ -extern u_int64_t KPDPphys; /* physical address of kernel level 3 */ extern u_int64_t KPML4phys; /* physical address of kernel level 4 */ extern u_int64_t KPML5phys; /* physical address of kernel level 5 */ @@ -502,6 +510,11 @@ int pmap_pkru_set(pmap_t pmap, vm_offset_t sva, vm void pmap_thread_init_invl_gen(struct thread *td); int pmap_vmspace_copy(pmap_t dst_pmap, pmap_t src_pmap); void pmap_page_array_startup(long count); + +#ifdef KASAN +void pmap_kasan_enter(vm_offset_t); +#endif + #endif /* _KERNEL */ /* Return various clipped indexes for a given VA */ Index: src/sys/amd64/include/vmparam.h =================================================================== --- src/sys/amd64/include/vmparam.h (revision 368820) +++ src/sys/amd64/include/vmparam.h (working copy) @@ -75,7 +75,9 @@ * of the direct mapped segment. This uses 2MB pages for reduced * TLB pressure. */ +#ifndef KASAN #define UMA_MD_SMALL_ALLOC +#endif /* * The physical address space is densely populated. @@ -165,7 +167,8 @@ * 0xffff800000000000 - 0xffff804020100fff recursive page table (512GB slot) * 0xffff804020100fff - 0xffff807fffffffff unused * 0xffff808000000000 - 0xffff847fffffffff large map (can be tuned up) - * 0xffff848000000000 - 0xfffff7ffffffffff unused (large map extends there) + * 0xffff848000000000 - 0xfffff77fffffffff unused (large map extends there) + * 0xfffff78000000000 - 0xfffff7ffffffffff 512GB KASAN shadow map * 0xfffff80000000000 - 0xfffffbffffffffff 4TB direct map * 0xfffffc0000000000 - 0xfffffdffffffffff unused * 0xfffffe0000000000 - 0xffffffffffffffff 2TB kernel map @@ -183,6 +186,9 @@ #define DMAP_MIN_ADDRESS KV4ADDR(DMPML4I, 0, 0, 0) #define DMAP_MAX_ADDRESS KV4ADDR(DMPML4I + NDMPML4E, 0, 0, 0) +#define KASAN_MIN_ADDRESS KV4ADDR(KASANPML4I, 0, 0, 0) +#define KASAN_MAX_ADDRESS KV4ADDR(KASANPML4I + NKASANPML4E, 0, 0, 0) + #define LARGEMAP_MIN_ADDRESS KV4ADDR(LMSPML4I, 0, 0, 0) #define LARGEMAP_MAX_ADDRESS KV4ADDR(LMEPML4I + 1, 0, 0, 0) Index: src/sys/cddl/dev/fbt/fbt.c =================================================================== --- src/sys/cddl/dev/fbt/fbt.c (revision 368820) +++ src/sys/cddl/dev/fbt/fbt.c (working copy) @@ -136,7 +136,16 @@ fbt_excluded(const char *name) strcmp(name, "owner_sx") == 0) return (1); +#ifdef KASAN /* + * dtrace_probe() can be instrumented by KASAN. + */ + if (strncmp(name, "__asan_", 7) == 0 || + strncmp(name, "kasan_", 6) == 0) + return (1); +#endif + + /* * Stack unwinders may be called from probe context on some * platforms. */ Index: src/sys/conf/files =================================================================== --- src/sys/conf/files (revision 368820) +++ src/sys/conf/files (working copy) @@ -3871,6 +3871,8 @@ kern/stack_protector.c standard \ compile-with "${NORMAL_C:N-fstack-protector*}" kern/subr_acl_nfs4.c optional ufs_acl | zfs kern/subr_acl_posix1e.c optional ufs_acl +kern/subr_asan.c optional kasan \ + compile-with "${NORMAL_C:N-fsanitize*}" kern/subr_autoconf.c standard kern/subr_blist.c standard kern/subr_boot.c standard @@ -3903,6 +3905,8 @@ kern/subr_lock.c standard kern/subr_log.c standard kern/subr_mchain.c optional libmchain kern/subr_module.c standard +kern/subr_msan.c optional kmsan \ + compile-with "${NORMAL_C:N-fsanitize*}" kern/subr_msgbuf.c standard kern/subr_param.c standard kern/subr_pcpu.c standard Index: src/sys/conf/kern.pre.mk =================================================================== --- src/sys/conf/kern.pre.mk (revision 368820) +++ src/sys/conf/kern.pre.mk (working copy) @@ -107,11 +107,22 @@ PROF= -pg .endif DEFINED_PROF= ${PROF} +KASAN_ENABLED!= grep KASAN opt_global.h || true ; echo +.if !empty(KASAN_ENABLED) +SAN_CFLAGS+= -fsanitize=kernel-address -mllvm -asan-stack=false \ + -mllvm -asan-instrumentation-with-call-threshold=0 +.endif + KCSAN_ENABLED!= grep KCSAN opt_global.h || true ; echo .if !empty(KCSAN_ENABLED) SAN_CFLAGS+= -fsanitize=thread .endif +KMSAN_ENABLED!= grep KMSAN opt_global.h || true ; echo +.if !empty(KMSAN_ENABLED) +SAN_CFLAGS+= -fsanitize=kernel-memory +.endif + KUBSAN_ENABLED!= grep KUBSAN opt_global.h || true ; echo .if !empty(KUBSAN_ENABLED) SAN_CFLAGS+= -fsanitize=undefined Index: src/sys/conf/options =================================================================== --- src/sys/conf/options (revision 368820) +++ src/sys/conf/options (working copy) @@ -229,8 +229,10 @@ ZSTDIO opt_zstdio.h # Sanitizers COVERAGE opt_global.h +KASAN opt_global.h KCOV KCSAN opt_global.h +KMSAN opt_global.h KUBSAN opt_global.h # POSIX kernel options Index: src/sys/contrib/zstd/lib/compress/zstd_cwksp.h =================================================================== --- src/sys/contrib/zstd/lib/compress/zstd_cwksp.h (revision 368820) +++ src/sys/contrib/zstd/lib/compress/zstd_cwksp.h (working copy) @@ -178,7 +178,7 @@ MEM_STATIC size_t ZSTD_cwksp_align(size_t size, si * else is though. */ MEM_STATIC size_t ZSTD_cwksp_alloc_size(size_t size) { -#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) +#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) && !defined(_KERNEL) return size + 2 * ZSTD_CWKSP_ASAN_REDZONE_SIZE; #else return size; @@ -228,7 +228,7 @@ MEM_STATIC void* ZSTD_cwksp_reserve_internal( ZSTD_cwksp_internal_advance_phase(ws, phase); alloc = (BYTE *)ws->allocStart - bytes; -#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) +#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) && !defined(_KERNEL) /* over-reserve space */ alloc = (BYTE *)alloc - 2 * ZSTD_CWKSP_ASAN_REDZONE_SIZE; #endif @@ -247,7 +247,7 @@ MEM_STATIC void* ZSTD_cwksp_reserve_internal( } ws->allocStart = alloc; -#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) +#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) && !defined(_KERNEL) /* Move alloc so there's ZSTD_CWKSP_ASAN_REDZONE_SIZE unused space on * either size. */ alloc = (BYTE *)alloc + ZSTD_CWKSP_ASAN_REDZONE_SIZE; @@ -296,7 +296,7 @@ MEM_STATIC void* ZSTD_cwksp_reserve_table(ZSTD_cwk } ws->tableEnd = end; -#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) +#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) && !defined(_KERNEL) __asan_unpoison_memory_region(alloc, bytes); #endif @@ -311,7 +311,7 @@ MEM_STATIC void* ZSTD_cwksp_reserve_object(ZSTD_cw void* alloc = ws->objectEnd; void* end = (BYTE*)alloc + roundedBytes; -#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) +#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) && !defined(_KERNEL) /* over-reserve space */ end = (BYTE *)end + 2 * ZSTD_CWKSP_ASAN_REDZONE_SIZE; #endif @@ -332,7 +332,7 @@ MEM_STATIC void* ZSTD_cwksp_reserve_object(ZSTD_cw ws->tableEnd = end; ws->tableValidEnd = end; -#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) +#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) && !defined(_KERNEL) /* Move alloc so there's ZSTD_CWKSP_ASAN_REDZONE_SIZE unused space on * either size. */ alloc = (BYTE *)alloc + ZSTD_CWKSP_ASAN_REDZONE_SIZE; @@ -392,7 +392,7 @@ MEM_STATIC void ZSTD_cwksp_clean_tables(ZSTD_cwksp MEM_STATIC void ZSTD_cwksp_clear_tables(ZSTD_cwksp* ws) { DEBUGLOG(4, "cwksp: clearing tables!"); -#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) +#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) && !defined(_KERNEL) { size_t size = (BYTE*)ws->tableValidEnd - (BYTE*)ws->objectEnd; __asan_poison_memory_region(ws->objectEnd, size); @@ -421,7 +421,7 @@ MEM_STATIC void ZSTD_cwksp_clear(ZSTD_cwksp* ws) { } #endif -#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) +#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) && !defined(_KERNEL) { size_t size = (BYTE*)ws->workspaceEnd - (BYTE*)ws->objectEnd; __asan_poison_memory_region(ws->objectEnd, size); Index: src/sys/dev/md/md.c =================================================================== --- src/sys/dev/md/md.c (revision 368820) +++ src/sys/dev/md/md.c (working copy) @@ -1012,11 +1012,11 @@ unmapped_step: goto unmapped_step; } uma_zfree(md_pbuf_zone, pb); + } else { + bp->bio_resid = auio.uio_resid; } free(piov, M_MD); - if (pb == NULL) - bp->bio_resid = auio.uio_resid; return (error); } @@ -1185,6 +1185,23 @@ mdstart_null(struct md_s *sc, struct bio *bp) } static void +md_handleattr(struct md_s *sc, struct bio *bp) +{ + if (sc->fwsectors && sc->fwheads && + (g_handleattr_int(bp, "GEOM::fwsectors", sc->fwsectors) || + g_handleattr_int(bp, "GEOM::fwheads", sc->fwheads))) + return; + if (g_handleattr_int(bp, "GEOM::candelete", 1)) + return; + if (sc->ident[0] != '\0' && + g_handleattr_str(bp, "GEOM::ident", sc->ident)) + return; + if (g_handleattr_int(bp, "MNT::verified", (sc->flags & MD_VERIFY) != 0)) + return; + g_io_deliver(bp, EOPNOTSUPP); +} + +static void md_kthread(void *arg) { struct md_s *sc; @@ -1212,39 +1229,23 @@ md_kthread(void *arg) } mtx_unlock(&sc->queue_mtx); if (bp->bio_cmd == BIO_GETATTR) { - int isv = ((sc->flags & MD_VERIFY) != 0); - - if ((sc->fwsectors && sc->fwheads && - (g_handleattr_int(bp, "GEOM::fwsectors", - sc->fwsectors) || - g_handleattr_int(bp, "GEOM::fwheads", - sc->fwheads))) || - g_handleattr_int(bp, "GEOM::candelete", 1)) - error = -1; - else if (sc->ident[0] != '\0' && - g_handleattr_str(bp, "GEOM::ident", sc->ident)) - error = -1; - else if (g_handleattr_int(bp, "MNT::verified", isv)) - error = -1; - else - error = EOPNOTSUPP; + md_handleattr(sc, bp); } else { error = sc->start(sc, bp); - } - - if (bp->bio_cmd == BIO_READ || bp->bio_cmd == BIO_WRITE) { - /* - * Devstat uses (bio_bcount, bio_resid) for - * determining the length of the completed part of - * the i/o. g_io_deliver() will translate from - * bio_completed to that, but it also destroys the - * bio so we must do our own translation. - */ - bp->bio_bcount = bp->bio_length; - bp->bio_resid = (error == -1 ? bp->bio_bcount : 0); - devstat_end_transaction_bio(sc->devstat, bp); - } - if (error != -1) { + if (bp->bio_cmd == BIO_READ || bp->bio_cmd == BIO_WRITE) { + /* + * Devstat uses (bio_bcount, bio_resid) for + * determining the length of the completed part + * of the i/o. g_io_deliver() will translate + * from bio_completed to that, but it also + * destroys the bio so we must do our own + * translation. + */ + bp->bio_bcount = bp->bio_length; + bp->bio_resid = + (error != 0 ? bp->bio_bcount : 0); + devstat_end_transaction_bio(sc->devstat, bp); + } bp->bio_completed = bp->bio_length; g_io_deliver(bp, error); } Index: src/sys/kern/kern_malloc.c =================================================================== --- src/sys/kern/kern_malloc.c (revision 368820) +++ src/sys/kern/kern_malloc.c (working copy) @@ -53,6 +53,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include #include #include @@ -110,7 +111,7 @@ dtrace_malloc_probe_func_t __read_mostly dtrace_ma #define MALLOC_DEBUG 1 #endif -#ifdef DEBUG_REDZONE +#if defined(KASAN) || defined(DEBUG_REDZONE) #define DEBUG_REDZONE_ARG_DEF , unsigned long osize #define DEBUG_REDZONE_ARG , osize #else @@ -492,6 +493,7 @@ void contigfree(void *addr, unsigned long size, struct malloc_type *type) { + kasan_mark(addr, size, size, 0); kmem_free((vm_offset_t)addr, size); malloc_type_freed(type, round_page(size)); } @@ -608,6 +610,10 @@ malloc_large(size_t *size, struct malloc_type *mtp if (va != NULL) va = redzone_setup(va, osize); #endif +#ifdef KASAN + if (va != NULL) + kasan_mark((void *)va, osize, sz, KASAN_MALLOC_REDZONE); +#endif return (va); } @@ -615,6 +621,7 @@ static void free_large(void *addr, size_t size) { + kasan_mark(addr, size, size, 0); kmem_free((vm_offset_t)addr, size); uma_total_dec(size); } @@ -633,7 +640,7 @@ void * int indx; caddr_t va; uma_zone_t zone; -#ifdef DEBUG_REDZONE +#if defined(DEBUG_REDZONE) || defined(KASAN) unsigned long osize = size; #endif @@ -665,6 +672,10 @@ void * if (va != NULL) va = redzone_setup(va, osize); #endif +#ifdef KASAN + if (va != NULL) + kasan_mark((void *)va, osize, size, KASAN_MALLOC_REDZONE); +#endif return ((void *) va); } @@ -699,7 +710,7 @@ malloc_domainset(size_t size, struct malloc_type * caddr_t va; int domain; int indx; -#ifdef DEBUG_REDZONE +#if defined(KASAN) || defined(DEBUG_REDZONE) unsigned long osize = size; #endif @@ -728,6 +739,10 @@ malloc_domainset(size_t size, struct malloc_type * if (va != NULL) va = redzone_setup(va, osize); #endif +#ifdef KASAN + if (va != NULL) + kasan_mark((void *)va, osize, size, KASAN_MALLOC_REDZONE); +#endif return (va); } @@ -745,7 +760,7 @@ void * malloc_domainset_exec(size_t size, struct malloc_type *mtp, struct domainset *ds, int flags) { -#ifdef DEBUG_REDZONE +#if defined(DEBUG_REDZONE) || defined(KASAN) unsigned long osize = size; #endif #ifdef MALLOC_DEBUG @@ -773,7 +788,7 @@ mallocarray(size_t nmemb, size_t size, struct mall return (malloc(size * nmemb, type, flags)); } -#ifdef INVARIANTS +#if defined(INVARIANTS) && !defined(KASAN) static void free_save_type(void *addr, struct malloc_type *mtp, u_long size) { @@ -854,9 +869,10 @@ free(void *addr, struct malloc_type *mtp) if (__predict_true(!malloc_large_slab(slab))) { size = zone->uz_size; -#ifdef INVARIANTS +#if defined(INVARIANTS) && !defined(KASAN) free_save_type(addr, mtp, size); #endif + kasan_mark(addr, size, size, 0); uma_zfree_arg(zone, addr, slab); } else { size = malloc_large_size(slab); @@ -894,9 +910,10 @@ zfree(void *addr, struct malloc_type *mtp) if (__predict_true(!malloc_large_slab(slab))) { size = zone->uz_size; -#ifdef INVARIANTS +#if defined(INVARIANTS) && !defined(KASAN) free_save_type(addr, mtp, size); #endif + kasan_mark(addr, size, size, 0); explicit_bzero(addr, size); uma_zfree_arg(zone, addr, slab); } else { @@ -955,9 +972,11 @@ realloc(void *addr, size_t size, struct malloc_typ alloc = malloc_large_size(slab); /* Reuse the original block if appropriate */ - if (size <= alloc - && (size > (alloc >> REALLOC_FRACTION) || alloc == MINALLOCSIZE)) + if (size <= alloc && + (size > (alloc >> REALLOC_FRACTION) || alloc == MINALLOCSIZE)) { + kasan_mark((void *)addr, size, alloc, KASAN_MALLOC_REDZONE); return (addr); + } #endif /* !DEBUG_REDZONE */ /* Allocate a new, bigger (or smaller) block */ @@ -965,6 +984,7 @@ realloc(void *addr, size_t size, struct malloc_typ return (NULL); /* Copy over original contents */ + kasan_mark(addr, size, size, alloc); bcopy(addr, newaddr, min(size, alloc)); free(addr, mtp); return (newaddr); @@ -1151,7 +1171,7 @@ mallocinit(void *dummy) for (subzone = 0; subzone < numzones; subzone++) { kmemzones[indx].kz_zone[subzone] = uma_zcreate(name, size, -#ifdef INVARIANTS +#if defined(INVARIANTS) && !defined(KASAN) mtrash_ctor, mtrash_dtor, mtrash_init, mtrash_fini, #else NULL, NULL, NULL, NULL, Index: src/sys/kern/kern_sendfile.c =================================================================== --- src/sys/kern/kern_sendfile.c (revision 368820) +++ src/sys/kern/kern_sendfile.c (working copy) @@ -905,6 +905,7 @@ retry_space: sfio->obj = obj; sfio->error = 0; sfio->m = NULL; + sfio->npages = npages; #ifdef KERN_TLS /* * This doesn't use ktls_hold() because sfio->m will @@ -1152,7 +1153,6 @@ prepend_header: } else { sfio->so = so; sfio->m = m0; - sfio->npages = npages; soref(so); error = (*so->so_proto->pr_usrreqs->pru_send) (so, PRUS_NOTREADY, m, NULL, NULL, td); Index: src/sys/kern/link_elf_obj.c =================================================================== --- src/sys/kern/link_elf_obj.c (revision 368820) +++ src/sys/kern/link_elf_obj.c (working copy) @@ -385,6 +385,8 @@ link_elf_link_preload(linker_class_t cls, const ch #ifdef __amd64__ case SHT_X86_64_UNWIND: #endif + case SHT_INIT_ARRAY: + case SHT_FINI_ARRAY: /* Ignore sections not loaded by the loader. */ if (shdr[i].sh_addr == 0) break; @@ -469,6 +471,7 @@ link_elf_link_preload(linker_class_t cls, const ch #ifdef __amd64__ case SHT_X86_64_UNWIND: #endif + case SHT_FINI_ARRAY: if (shdr[i].sh_addr == 0) break; ef->progtab[pb].addr = (void *)shdr[i].sh_addr; @@ -478,6 +481,10 @@ link_elf_link_preload(linker_class_t cls, const ch else if (shdr[i].sh_type == SHT_X86_64_UNWIND) ef->progtab[pb].name = "<>"; #endif + else if (shdr[i].sh_type == SHT_INIT_ARRAY) + ef->progtab[pb].name = "<>"; + else if (shdr[i].sh_type == SHT_FINI_ARRAY) + ef->progtab[pb].name = "<>"; else ef->progtab[pb].name = "<>"; ef->progtab[pb].size = shdr[i].sh_size; @@ -524,8 +531,9 @@ link_elf_link_preload(linker_class_t cls, const ch vnet_data_copy(vnet_data, shdr[i].sh_size); ef->progtab[pb].addr = vnet_data; #endif - } else if (ef->progtab[pb].name != NULL && - !strcmp(ef->progtab[pb].name, ".ctors")) { + } else if ((ef->progtab[pb].name != NULL && + strcmp(ef->progtab[pb].name, ".ctors") == 0) || + shdr[i].sh_type == SHT_INIT_ARRAY) { lf->ctors_addr = ef->progtab[pb].addr; lf->ctors_size = shdr[i].sh_size; } @@ -772,6 +780,8 @@ link_elf_load_file(linker_class_t cls, const char #ifdef __amd64__ case SHT_X86_64_UNWIND: #endif + case SHT_INIT_ARRAY: + case SHT_FINI_ARRAY: if ((shdr[i].sh_flags & SHF_ALLOC) == 0) break; ef->nprogtab++; @@ -893,6 +903,8 @@ link_elf_load_file(linker_class_t cls, const char #ifdef __amd64__ case SHT_X86_64_UNWIND: #endif + case SHT_INIT_ARRAY: + case SHT_FINI_ARRAY: if ((shdr[i].sh_flags & SHF_ALLOC) == 0) break; alignmask = shdr[i].sh_addralign - 1; @@ -970,6 +982,8 @@ link_elf_load_file(linker_class_t cls, const char #ifdef __amd64__ case SHT_X86_64_UNWIND: #endif + case SHT_INIT_ARRAY: + case SHT_FINI_ARRAY: if ((shdr[i].sh_flags & SHF_ALLOC) == 0) break; alignmask = shdr[i].sh_addralign - 1; @@ -978,7 +992,8 @@ link_elf_load_file(linker_class_t cls, const char if (ef->shstrtab != NULL && shdr[i].sh_name != 0) { ef->progtab[pb].name = ef->shstrtab + shdr[i].sh_name; - if (!strcmp(ef->progtab[pb].name, ".ctors")) { + if (!strcmp(ef->progtab[pb].name, ".ctors") || + shdr[i].sh_type == SHT_INIT_ARRAY) { lf->ctors_addr = (caddr_t)mapbase; lf->ctors_size = shdr[i].sh_size; } Index: src/sys/kern/vfs_subr.c =================================================================== --- src/sys/kern/vfs_subr.c (revision 368820) +++ src/sys/kern/vfs_subr.c (working copy) @@ -48,6 +48,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include #include #include @@ -525,6 +526,54 @@ vn_free_marker(struct vnode *vp) free(vp, M_VNODE_MARKER); } +#ifdef KASAN +static int +vnode_ctor(void *mem, int size, void *arg __unused, int flags __unused) +{ + kasan_mark(mem, size, roundup2(size, UMA_ALIGN_PTR + 1), 0); + return (0); +} + +static void +vnode_dtor(void *mem, int size, void *arg __unused) +{ + size_t end1, end2, off1, off2; + + _Static_assert(offsetof(struct vnode, v_vnodelist) < + offsetof(struct vnode, v_dbatchcpu), + "KASAN marks require updating"); + + off1 = offsetof(struct vnode, v_vnodelist); + off2 = offsetof(struct vnode, v_dbatchcpu); + end1 = off1 + sizeof(((struct vnode *)NULL)->v_vnodelist); + end2 = off2 + sizeof(((struct vnode *)NULL)->v_dbatchcpu); + + /* + * Access to the v_vnodelist and v_dbatchcpu fields are permitted even + * after the vnode has been freed. Try to get some KASAN coverage by + * marking everything except those two fields as invalid. Because + * KASAN's tracking is not byte-granular, any preceding fields sharing + * the same 8-byte aligned word must also be marked valid. + */ + + /* Handle the area from the start until v_vnodelist... */ + off1 = rounddown2(off1, KASAN_SHADOW_SCALE); + kasan_mark(mem, off1, off1, KASAN_UMA_FREED); + + /* ... then the area between v_vnodelist and v_dbatchcpu ... */ + off1 = roundup2(end1, KASAN_SHADOW_SCALE); + off2 = rounddown2(off2, KASAN_SHADOW_SCALE); + if (off2 > off1) + kasan_mark((void *)((char *)mem + off1), off2 - off1, + off2 - off1, KASAN_UMA_FREED); + + /* ... and finally the area from v_dbatchcpu to the end. */ + off2 = roundup2(end2, KASAN_SHADOW_SCALE); + kasan_mark((void *)((char *)mem + off2), size - off2, size - off2, + KASAN_UMA_FREED); +} +#endif /* KASAN */ + /* * Initialize a vnode as it first enters the zone. */ @@ -590,6 +639,8 @@ vnode_fini(void *mem, int size) mtx_destroy(&vp->v_interlock); bo = &vp->v_bufobj; rw_destroy(BO_LOCKPTR(bo)); + + kasan_mark(mem, size, size, 0); } /* @@ -617,6 +668,8 @@ static void vntblinit(void *dummy __unused) { struct vdbatch *vd; + uma_ctor ctor; + uma_dtor dtor; int cpu, physvnodes, virtvnodes; u_int i; @@ -656,9 +709,18 @@ vntblinit(void *dummy __unused) TAILQ_INSERT_HEAD(&vnode_list, vnode_list_free_marker, v_vnodelist); vnode_list_reclaim_marker = vn_alloc_marker(NULL); TAILQ_INSERT_HEAD(&vnode_list, vnode_list_reclaim_marker, v_vnodelist); - vnode_zone = uma_zcreate("VNODE", sizeof (struct vnode), NULL, NULL, - vnode_init, vnode_fini, UMA_ALIGN_PTR, 0); + +#ifdef KASAN + ctor = vnode_ctor; + dtor = vnode_dtor; +#else + ctor = NULL; + dtor = NULL; +#endif + vnode_zone = uma_zcreate("VNODE", sizeof(struct vnode), ctor, dtor, + vnode_init, vnode_fini, UMA_ALIGN_PTR, UMA_ZONE_NOKASAN); uma_zone_set_smr(vnode_zone, vfs_smr); + /* * Preallocate enough nodes to support one-per buf so that * we can not fail an insert. reassignbuf() callers can not Index: src/sys/net/rtsock.c =================================================================== --- src/sys/net/rtsock.c (revision 368820) +++ src/sys/net/rtsock.c (working copy) @@ -1258,12 +1258,12 @@ rtsock_fix_netmask(const struct sockaddr *dst, con static struct mbuf * rtsock_msg_mbuf(int type, struct rt_addrinfo *rtinfo) { + struct sockaddr_storage ss; struct rt_msghdr *rtm; struct mbuf *m; int i; struct sockaddr *sa; #ifdef INET6 - struct sockaddr_storage ss; struct sockaddr_in6 *sin6; #endif int len, dlen; @@ -1308,13 +1308,17 @@ rtsock_msg_mbuf(int type, struct rt_addrinfo *rtin if ((sa = rtinfo->rti_info[i]) == NULL) continue; rtinfo->rti_addrs |= (1 << i); + dlen = SA_SIZE(sa); + if (dlen != sa->sa_len) { + bzero(&ss, sizeof(ss)); + bcopy(sa, &ss, sa->sa_len); + sa = (struct sockaddr *)&ss; + } #ifdef INET6 if (sa->sa_family == AF_INET6) { - sin6 = (struct sockaddr_in6 *)&ss; - bcopy(sa, sin6, sizeof(*sin6)); - if (sa6_recoverscope(sin6) == 0) - sa = (struct sockaddr *)sin6; + sin6 = (struct sockaddr_in6 *)sa; + (void)sa6_recoverscope(sin6); } #endif m_copyback(m, len, dlen, (caddr_t)sa); @@ -1342,12 +1346,11 @@ rtsock_msg_mbuf(int type, struct rt_addrinfo *rtin static int rtsock_msg_buffer(int type, struct rt_addrinfo *rtinfo, struct walkarg *w, int *plen) { - int i; - int len, buflen = 0, dlen; + struct sockaddr_storage ss; + int len, buflen = 0, dlen, i; caddr_t cp = NULL; struct rt_msghdr *rtm = NULL; #ifdef INET6 - struct sockaddr_storage ss; struct sockaddr_in6 *sin6; #endif #ifdef COMPAT_FREEBSD32 @@ -1414,12 +1417,15 @@ rtsock_msg_buffer(int type, struct rt_addrinfo *rt #endif dlen = SA_SIZE(sa); if (cp != NULL && buflen >= dlen) { + if (dlen != sa->sa_len) { + bzero(&ss, sizeof(ss)); + bcopy(sa, &ss, sa->sa_len); + sa = (struct sockaddr *)&ss; + } #ifdef INET6 if (sa->sa_family == AF_INET6) { - sin6 = (struct sockaddr_in6 *)&ss; - bcopy(sa, sin6, sizeof(*sin6)); - if (sa6_recoverscope(sin6) == 0) - sa = (struct sockaddr *)sin6; + sin6 = (struct sockaddr_in6 *)sa; + (void)sa6_recoverscope(sin6); } #endif bcopy((caddr_t)sa, cp, (unsigned)dlen); Index: src/sys/netgraph/ng_base.c =================================================================== --- src/sys/netgraph/ng_base.c (revision 368820) +++ src/sys/netgraph/ng_base.c (working copy) @@ -3166,12 +3166,10 @@ vnet_netgraph_uninit(const void *unused __unused) /* Attempt to kill it only if it is a regular node */ if (node != NULL) { if (node == last_killed) { - /* This should never happen */ - printf("ng node %s needs NGF_REALLY_DIE\n", - node->nd_name); if (node->nd_flags & NGF_REALLY_DIE) panic("ng node %s won't die", node->nd_name); + /* The node persisted itself. Try again. */ node->nd_flags |= NGF_REALLY_DIE; } ng_rmnode(node, NULL, NULL, 0); Index: src/sys/netgraph/ng_ether.c =================================================================== --- src/sys/netgraph/ng_ether.c (revision 368820) +++ src/sys/netgraph/ng_ether.c (working copy) @@ -414,8 +414,7 @@ ng_ether_ifnet_arrival_event(void *arg __unused, s node_p node; /* Only ethernet interfaces are of interest. */ - if (ifp->if_type != IFT_ETHER - && ifp->if_type != IFT_L2VLAN) + if (ifp->if_type != IFT_ETHER && ifp->if_type != IFT_L2VLAN) return; /* @@ -753,13 +752,13 @@ ng_ether_shutdown(node_p node) if (node->nd_flags & NGF_REALLY_DIE) { /* - * WE came here because the ethernet card is being unloaded, - * so stop being persistent. - * Actually undo all the things we did on creation. - * Assume the ifp has already been freed. + * The ifnet is going away, perhaps because the driver was + * unloaded or its vnet is being torn down. */ NG_NODE_SET_PRIVATE(node, NULL); - free(priv, M_NETGRAPH); + if (priv->ifp != NULL) + IFP2NG(priv->ifp) = NULL; + free(priv, M_NETGRAPH); NG_NODE_UNREF(node); /* free node itself */ return (0); } Index: src/sys/netinet/in_pcb.c =================================================================== --- src/sys/netinet/in_pcb.c (revision 368820) +++ src/sys/netinet/in_pcb.c (working copy) @@ -645,6 +645,7 @@ out: int in_pcbbind(struct inpcb *inp, struct sockaddr *nam, struct ucred *cred) { + struct sockaddr_in *sin; int anonport, error; INP_WLOCK_ASSERT(inp); @@ -652,7 +653,16 @@ in_pcbbind(struct inpcb *inp, struct sockaddr *nam if (inp->inp_lport != 0 || inp->inp_laddr.s_addr != INADDR_ANY) return (EINVAL); - anonport = nam == NULL || ((struct sockaddr_in *)nam)->sin_port == 0; + if (nam != NULL) { + sin = (struct sockaddr_in *)nam; + if (sin->sin_len != sizeof(*sin)) + return (EINVAL); + if (sin->sin_family != AF_INET) + return (EAFNOSUPPORT); + anonport = sin->sin_port == 0; + } else { + anonport = 0; + } error = in_pcbbind_setup(inp, nam, &inp->inp_laddr.s_addr, &inp->inp_lport, cred); if (error) Index: src/sys/netinet/ip_divert.c =================================================================== --- src/sys/netinet/ip_divert.c (revision 368820) +++ src/sys/netinet/ip_divert.c (working copy) @@ -609,15 +609,12 @@ div_bind(struct socket *so, struct sockaddr *nam, inp = sotoinpcb(so); KASSERT(inp != NULL, ("div_bind: inp == NULL")); - /* in_pcbbind assumes that nam is a sockaddr_in - * and in_pcbbind requires a valid address. Since divert - * sockets don't we need to make sure the address is - * filled in properly. + /* + * in_pcbbind() requires a valid address. Since divert sockets do not + * we need to make sure the address is filled in properly. * XXX -- divert should not be abusing in_pcbind * and should probably have its own family. */ - if (nam->sa_family != AF_INET) - return EAFNOSUPPORT; ((struct sockaddr_in *)nam)->sin_addr.s_addr = INADDR_ANY; INP_INFO_WLOCK(&V_divcbinfo); INP_WLOCK(inp); Index: src/sys/netinet/raw_ip.c =================================================================== --- src/sys/netinet/raw_ip.c (revision 368820) +++ src/sys/netinet/raw_ip.c (working copy) @@ -1086,6 +1086,10 @@ rip_send(struct socket *so, int flags, struct mbuf m_freem(m); return (ENOTCONN); } + if (nam->sa_len != sizeof(struct sockaddr_in)) { + m_freem(m); + return (EINVAL); + } dst = ((struct sockaddr_in *)nam)->sin_addr.s_addr; } return (rip_output(m, so, dst)); Index: src/sys/netinet/tcp_usrreq.c =================================================================== --- src/sys/netinet/tcp_usrreq.c (revision 368820) +++ src/sys/netinet/tcp_usrreq.c (working copy) @@ -449,7 +449,7 @@ tcp_usr_listen(struct socket *so, int backlog, str error = solisten_proto_check(so); INP_HASH_WLOCK(&V_tcbinfo); if (error == 0 && inp->inp_lport == 0) - error = in_pcbbind(inp, (struct sockaddr *)0, td->td_ucred); + error = in_pcbbind(inp, NULL, td->td_ucred); INP_HASH_WUNLOCK(&V_tcbinfo); if (error == 0) { tcp_state_change(tp, TCPS_LISTEN); Index: src/sys/netinet/udp_usrreq.c =================================================================== --- src/sys/netinet/udp_usrreq.c (revision 368820) +++ src/sys/netinet/udp_usrreq.c (working copy) @@ -1623,6 +1623,7 @@ udp_bind(struct socket *so, struct sockaddr *nam, pcbinfo = udp_get_inpcbinfo(so->so_proto->pr_protocol); inp = sotoinpcb(so); KASSERT(inp != NULL, ("udp_bind: inp == NULL")); + INP_WLOCK(inp); INP_HASH_WLOCK(pcbinfo); error = in_pcbbind(inp, nam, td->td_ucred); @@ -1663,12 +1664,18 @@ udp_connect(struct socket *so, struct sockaddr *na pcbinfo = udp_get_inpcbinfo(so->so_proto->pr_protocol); inp = sotoinpcb(so); KASSERT(inp != NULL, ("udp_connect: inp == NULL")); + + sin = (struct sockaddr_in *)nam; + if (sin->sin_len != sizeof(*sin)) + return (EINVAL); + if (sin->sin_family != AF_INET) + return (EAFNOSUPPORT); + INP_WLOCK(inp); if (inp->inp_faddr.s_addr != INADDR_ANY) { INP_WUNLOCK(inp); return (EISCONN); } - sin = (struct sockaddr_in *)nam; error = prison_remote_ip4(td->td_ucred, &sin->sin_addr); if (error != 0) { INP_WUNLOCK(inp); @@ -1738,9 +1745,17 @@ udp_send(struct socket *so, int flags, struct mbuf struct mbuf *control, struct thread *td) { struct inpcb *inp; + struct sockaddr_in *sin; inp = sotoinpcb(so); KASSERT(inp != NULL, ("udp_send: inp == NULL")); + if (addr != NULL) { + sin = (struct sockaddr_in *)addr; + if (sin->sin_len != sizeof(*sin)) + return (EINVAL); + if (sin->sin_family != AF_INET) + return (EAFNOSUPPORT); + } return (udp_output(inp, m, addr, control, td, flags)); } #endif /* INET */ Index: src/sys/netinet6/udp6_usrreq.c =================================================================== --- src/sys/netinet6/udp6_usrreq.c (revision 368820) +++ src/sys/netinet6/udp6_usrreq.c (working copy) @@ -1089,6 +1089,7 @@ udp6_bind(struct socket *so, struct sockaddr *nam, { struct inpcb *inp; struct inpcbinfo *pcbinfo; + struct sockaddr_in6 *sin6; int error; u_char vflagsav; @@ -1102,17 +1103,23 @@ udp6_bind(struct socket *so, struct sockaddr *nam, inp->inp_vflag &= ~INP_IPV4; inp->inp_vflag |= INP_IPV6; if ((inp->inp_flags & IN6P_IPV6_V6ONLY) == 0) { - struct sockaddr_in6 *sin6_p; + sin6 = (struct sockaddr_in6 *)nam; + if (sin6->sin6_len != sizeof(*sin6)) { + error = EINVAL; + goto out; + } + if (sin6->sin6_family != AF_INET6) { + error = EAFNOSUPPORT; + goto out; + } - sin6_p = (struct sockaddr_in6 *)nam; - - if (IN6_IS_ADDR_UNSPECIFIED(&sin6_p->sin6_addr)) + if (IN6_IS_ADDR_UNSPECIFIED(&sin6->sin6_addr)) inp->inp_vflag |= INP_IPV4; #ifdef INET - else if (IN6_IS_ADDR_V4MAPPED(&sin6_p->sin6_addr)) { + else if (IN6_IS_ADDR_V4MAPPED(&sin6->sin6_addr)) { struct sockaddr_in sin; - in6_sin6_2_sin(&sin, sin6_p); + in6_sin6_2_sin(&sin, sin6); inp->inp_vflag |= INP_IPV4; inp->inp_vflag &= ~INP_IPV6; error = in_pcbbind(inp, (struct sockaddr *)&sin, @@ -1123,9 +1130,7 @@ udp6_bind(struct socket *so, struct sockaddr *nam, } error = in6_pcbbind(inp, nam, td->td_ucred); -#ifdef INET out: -#endif if (error != 0) inp->inp_vflag = vflagsav; INP_HASH_WUNLOCK(pcbinfo); @@ -1181,9 +1186,12 @@ udp6_connect(struct socket *so, struct sockaddr *n pcbinfo = udp_get_inpcbinfo(so->so_proto->pr_protocol); inp = sotoinpcb(so); - sin6 = (struct sockaddr_in6 *)nam; KASSERT(inp != NULL, ("udp6_connect: inp == NULL")); + sin6 = (struct sockaddr_in6 *)nam; + if (sin6->sin6_len != sizeof(*sin6)) + return (EINVAL); + /* * XXXRW: Need to clarify locking of v4/v6 flags. */ Index: src/sys/sys/libkern.h =================================================================== --- src/sys/sys/libkern.h (revision 368820) +++ src/sys/sys/libkern.h (working copy) @@ -191,13 +191,16 @@ size_t strspn(const char *, const char *); char *strstr(const char *, const char *); int strvalid(const char *, size_t); -#ifdef KCSAN +#if defined(KASAN) +size_t kasan_strlen(const char *); +#define strlen(s) kasan_strlen(s) +#elif defined(KCSAN) char *kcsan_strcpy(char *, const char *); int kcsan_strcmp(const char *, const char *); size_t kcsan_strlen(const char *); -#define strcpy(d, s) kcsan_strcpy((d), (s)) -#define strcmp(s1, s2) kcsan_strcmp((s1), (s2)) -#define strlen(s) kcsan_strlen((s)) +#define strcpy(d, s) kcsan_strcpy((d), (s)) +#define strcmp(s1, s2) kcsan_strcmp((s1), (s2)) +#define strlen(s) kcsan_strlen((s)) #endif static __inline char * Index: src/sys/sys/systm.h =================================================================== --- src/sys/sys/systm.h (revision 368820) +++ src/sys/sys/systm.h (working copy) @@ -351,7 +351,19 @@ void *memcpy(void * _Nonnull to, const void * _Non void *memmove(void * _Nonnull dest, const void * _Nonnull src, size_t n); int memcmp(const void *b1, const void *b2, size_t len); -#ifdef KCSAN +#if defined(KASAN) +void *kasan_memset(void *, int, size_t); +void *kasan_memcpy(void *, const void *, size_t); +void *kasan_memmove(void *, const void *, size_t); +int kasan_memcmp(const void *, const void *, size_t); +#define bcopy(from, to, len) kasan_memmove((to), (from), (len)) +#define bzero(buf, len) kasan_memset((buf), 0, (len)) +#define bcmp(b1, b2, len) kasan_memcmp((b1), (b2), (len)) +#define memset(buf, c, len) kasan_memset((buf), (c), (len)) +#define memcpy(to, from, len) kasan_memcpy((to), (from), (len)) +#define memmove(dest, src, n) kasan_memmove((dest), (src), (n)) +#define memcmp(b1, b2, len) kasan_memcmp((b1), (b2), (len)) +#elif defined(KCSAN) void *kcsan_memset(void *, int, size_t); void *kcsan_memcpy(void *, const void *, size_t); void *kcsan_memmove(void *, const void *, size_t); @@ -402,13 +414,20 @@ int copyout(const void * _Nonnull __restrict kaddr int copyout_nofault(const void * _Nonnull __restrict kaddr, void * __restrict udaddr, size_t len); -#ifdef KCSAN +#if defined(KASAN) +int kasan_copyin(const void *, void *, size_t); +int kasan_copyinstr(const void *, void *, size_t, size_t *); +int kasan_copyout(const void *, void *, size_t); +#define copyin(u, k, l) kasan_copyin((u), (k), (l)) +#define copyinstr(u, k, l, lc) kasan_copyinstr((u), (k), (l), (lc)) +#define copyout(k, u, l) kasan_copyout((k), (u), (l)) +#elif defined(KCSAN) int kcsan_copyin(const void *, void *, size_t); int kcsan_copyinstr(const void *, void *, size_t, size_t *); int kcsan_copyout(const void *, void *, size_t); -#define copyin(u, k, l) kcsan_copyin((u), (k), (l)) -#define copyinstr(u, k, l, lc) kcsan_copyinstr((u), (k), (l), (lc)) -#define copyout(k, u, l) kcsan_copyout((k), (u), (l)) +#define copyin(u, k, l) kcsan_copyin((u), (k), (l)) +#define copyinstr(u, k, l, lc) kcsan_copyinstr((u), (k), (l), (lc)) +#define copyout(k, u, l) kcsan_copyout((k), (u), (l)) #endif int fubyte(volatile const void *base); Index: src/sys/vm/uma.h =================================================================== --- src/sys/vm/uma.h (revision 368820) +++ src/sys/vm/uma.h (working copy) @@ -276,6 +276,7 @@ uma_zone_t uma_zcache_create(const char *name, int * * See sys/smr.h for more details. */ +#define UMA_ZONE_NOKASAN 0x80000 /* XXXMJ */ /* In use by UMA_ZFLAGs: 0xffe00000 */ /* @@ -286,7 +287,7 @@ uma_zone_t uma_zcache_create(const char *name, int #define UMA_ZONE_INHERIT \ (UMA_ZONE_NOTOUCH | UMA_ZONE_MALLOC | UMA_ZONE_NOFREE | \ UMA_ZONE_VM | UMA_ZONE_NOTPAGE | UMA_ZONE_PCPU | \ - UMA_ZONE_FIRSTTOUCH | UMA_ZONE_ROUNDROBIN) + UMA_ZONE_FIRSTTOUCH | UMA_ZONE_ROUNDROBIN | UMA_ZONE_NOKASAN) /* Definitions for align */ #define UMA_ALIGN_PTR (sizeof(void *) - 1) /* Alignment fit for ptr */ Index: src/sys/vm/uma_core.c =================================================================== --- src/sys/vm/uma_core.c (revision 368820) +++ src/sys/vm/uma_core.c (working copy) @@ -58,6 +58,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include #include #include @@ -318,6 +319,7 @@ static void bucket_zone_drain(void); static uma_bucket_t zone_alloc_bucket(uma_zone_t, void *, int, int); static void *slab_alloc_item(uma_keg_t keg, uma_slab_t slab); static void slab_free_item(uma_zone_t zone, uma_slab_t slab, void *item); +static size_t slab_sizeof(int nitems); static uma_keg_t uma_kcreate(uma_zone_t zone, size_t size, uma_init uminit, uma_fini fini, int align, uint32_t flags); static int zone_import(void *, void **, int, int, int); @@ -532,6 +534,73 @@ bucket_zone_drain(void) } /* + * Mark a single item as valid. + */ +static void +kasan_mark_item_valid(uma_zone_t zone, void *item) +{ +#ifdef KASAN + void *pcpu_item; + size_t n; + int i; + + if ((zone->uz_flags & UMA_ZONE_NOKASAN) == 0) { + n = zone->uz_size; + if ((zone->uz_flags & UMA_ZONE_PCPU) != 0) { + pcpu_item = zpcpu_base_to_offset(item); + for (i = 0; i <= mp_maxid; i++) + kasan_mark(zpcpu_get_cpu(pcpu_item, i), + n, roundup2(n, KASAN_SHADOW_SCALE), 0); + } else { + kasan_mark(item, n, roundup2(n, KASAN_SHADOW_SCALE), 0); + } + } +#endif +} + +static void +kasan_mark_item_invalid(uma_zone_t zone, void *item) +{ +#ifdef KASAN + size_t n; + + if ((zone->uz_flags & UMA_ZONE_NOKASAN) == 0) { + n = roundup2(zone->uz_size, KASAN_SHADOW_SCALE); + kasan_mark(item, 0, n, KASAN_UMA_FREED); + } +#endif +} + +static void +kasan_mark_slab_valid(uma_keg_t keg, void *mem) +{ +#ifdef KASAN + size_t n; + + if ((keg->uk_flags & UMA_ZONE_NOKASAN) == 0) { + n = keg->uk_ppera * PAGE_SIZE; + kasan_mark(mem, n, n, 0); + } +#endif +} + +static void +kasan_mark_slab_invalid(uma_keg_t keg, void *mem) +{ +#ifdef KASAN + size_t n; + + if ((keg->uk_flags & UMA_ZONE_NOKASAN) == 0) { + if ((keg->uk_flags & UMA_ZFLAG_OFFPAGE) != 0) + n = keg->uk_ppera * PAGE_SIZE; + else + n = keg->uk_pgoff; + kasan_mark(mem, 0, n, KASAN_UMA_FREED); + } +#endif +} + +/* * Acquire the domain lock and record contention. */ static uma_zone_domain_t @@ -1156,8 +1225,11 @@ bucket_drain(uma_zone_t zone, uma_bucket_t bucket) zone->uz_size, NULL, SKIP_NONE); } if (zone->uz_fini) - for (i = 0; i < bucket->ub_cnt; i++) + for (i = 0; i < bucket->ub_cnt; i++) { + kasan_mark_item_valid(zone, bucket->ub_bucket[i]); zone->uz_fini(bucket->ub_bucket[i], zone->uz_size); + kasan_mark_item_invalid(zone, bucket->ub_bucket[i]); + } zone->uz_release(zone->uz_arg, bucket->ub_bucket, bucket->ub_cnt); if (zone->uz_max_items > 0) zone_free_limit(zone, bucket->ub_cnt); @@ -1362,6 +1434,7 @@ static void keg_free_slab(uma_keg_t keg, uma_slab_t slab, int start) { uint8_t *mem; + size_t size; int i; uint8_t flags; @@ -1369,10 +1442,11 @@ keg_free_slab(uma_keg_t keg, uma_slab_t slab, int keg->uk_name, keg, slab, PAGE_SIZE * keg->uk_ppera); mem = slab_data(slab, keg); - flags = slab->us_flags; - i = start; + size = PAGE_SIZE * keg->uk_ppera; + + kasan_mark_slab_valid(keg, mem); if (keg->uk_fini != NULL) { - for (i--; i > -1; i--) + for (i = start - 1; i > -1; i--) #ifdef INVARIANTS /* * trash_fini implies that dtor was trash_dtor. trash_fini @@ -1387,11 +1461,13 @@ keg_free_slab(uma_keg_t keg, uma_slab_t slab, int #endif keg->uk_fini(slab_item(slab, keg, i), keg->uk_size); } - if (keg->uk_flags & UMA_ZFLAG_OFFPAGE) + flags = slab->us_flags; + if (keg->uk_flags & UMA_ZFLAG_OFFPAGE) { zone_free_item(slabzone(keg->uk_ipers), slab_tohashslab(slab), NULL, SKIP_NONE); - keg->uk_freef(mem, PAGE_SIZE * keg->uk_ppera, flags); - uma_total_dec(PAGE_SIZE * keg->uk_ppera); + } + keg->uk_freef(mem, size, flags); + uma_total_dec(size); } static void @@ -1535,7 +1611,6 @@ keg_alloc_slab(uma_keg_t keg, uma_zone_t zone, int int aflags) { uma_domain_t dom; - uma_alloc allocf; uma_slab_t slab; unsigned long size; uint8_t *mem; @@ -1545,7 +1620,6 @@ keg_alloc_slab(uma_keg_t keg, uma_zone_t zone, int KASSERT(domain >= 0 && domain < vm_ndomains, ("keg_alloc_slab: domain %d out of range", domain)); - allocf = keg->uk_allocf; slab = NULL; mem = NULL; if (keg->uk_flags & UMA_ZFLAG_OFFPAGE) { @@ -1574,7 +1648,7 @@ keg_alloc_slab(uma_keg_t keg, uma_zone_t zone, int /* zone is passed for legacy reasons. */ size = keg->uk_ppera * PAGE_SIZE; - mem = allocf(zone, size, domain, &sflags, aflags); + mem = keg->uk_allocf(zone, size, domain, &sflags, aflags); if (mem == NULL) { if (keg->uk_flags & UMA_ZFLAG_OFFPAGE) zone_free_item(slabzone(keg->uk_ipers), @@ -1588,10 +1662,11 @@ keg_alloc_slab(uma_keg_t keg, uma_zone_t zone, int domain = 0; /* Point the slab into the allocated memory */ - if (!(keg->uk_flags & UMA_ZFLAG_OFFPAGE)) - slab = (uma_slab_t )(mem + keg->uk_pgoff); - else + if ((keg->uk_flags & UMA_ZFLAG_OFFPAGE) == 0) { + slab = (uma_slab_t)(mem + keg->uk_pgoff); + } else { slab_tohashslab(slab)->uhs_data = mem; + } if (keg->uk_flags & UMA_ZFLAG_VTOSLAB) for (i = 0; i < keg->uk_ppera; i++) @@ -1617,6 +1692,7 @@ keg_alloc_slab(uma_keg_t keg, uma_zone_t zone, int goto fail; } } + kasan_mark_slab_invalid(keg, mem); KEG_LOCK(keg, domain); CTR3(KTR_UMA, "keg_alloc_slab: allocated slab %p for %s(%p)", @@ -2598,9 +2674,12 @@ zone_ctor(void *mem, int size, void *udata, int fl STAILQ_INIT(&zdom->uzd_buckets); } -#ifdef INVARIANTS +#if defined(INVARIANTS) && !defined(KASAN) if (arg->uminit == trash_init && arg->fini == trash_fini) zone->uz_flags |= UMA_ZFLAG_TRASH | UMA_ZFLAG_CTORDTOR; +#elif defined(KASAN) + if ((arg->flags & (UMA_ZONE_NOFREE | UMA_ZFLAG_CACHE)) != 0) + arg->flags |= UMA_ZONE_NOKASAN; #endif /* @@ -3023,7 +3102,7 @@ uma_zcreate(const char *name, size_t size, uma_cto args.dtor = dtor; args.uminit = uminit; args.fini = fini; -#ifdef INVARIANTS +#if defined(INVARIANTS) && !defined(KASAN) /* * Inject procedures which check for memory use after free if we are * allowed to scramble the memory while it is not allocated. This @@ -3175,12 +3254,17 @@ item_ctor(uma_zone_t zone, int uz_flags, int size, { #ifdef INVARIANTS bool skipdbg; +#endif + kasan_mark_item_valid(zone, item); + +#ifdef INVARIANTS skipdbg = uma_dbg_zskip(zone, item); - if (!skipdbg && (zone->uz_flags & UMA_ZFLAG_TRASH) != 0 && + if (!skipdbg && (uz_flags & UMA_ZFLAG_TRASH) != 0 && zone->uz_ctor != trash_ctor) trash_ctor(item, size, udata, flags); #endif + /* Check flags before loading ctor pointer. */ if (__predict_false((uz_flags & UMA_ZFLAG_CTORDTOR) != 0) && __predict_false(zone->uz_ctor != NULL) && @@ -3223,6 +3307,7 @@ item_dtor(uma_zone_t zone, void *item, int size, v trash_dtor(item, size, udata); #endif } + kasan_mark_item_invalid(zone, item); } #ifdef NUMA @@ -3944,7 +4029,7 @@ static uma_bucket_t zone_alloc_bucket(uma_zone_t zone, void *udata, int domain, int flags) { uma_bucket_t bucket; - int maxbucket, cnt; + int error, maxbucket, cnt; CTR3(KTR_UMA, "zone_alloc_bucket zone %s(%p) domain %d", zone->uz_name, zone, domain); @@ -3979,10 +4064,15 @@ zone_alloc_bucket(uma_zone_t zone, void *udata, in if (bucket->ub_cnt != 0 && zone->uz_init != NULL) { int i; - for (i = 0; i < bucket->ub_cnt; i++) - if (zone->uz_init(bucket->ub_bucket[i], zone->uz_size, - flags) != 0) + for (i = 0; i < bucket->ub_cnt; i++) { + kasan_mark_item_valid(zone, bucket->ub_bucket[i]); + error = zone->uz_init(bucket->ub_bucket[i], + zone->uz_size, flags); + kasan_mark_item_invalid(zone, bucket->ub_bucket[i]); + if (error != 0) break; + } + /* * If we couldn't initialize the whole bucket, put the * rest back onto the freelist. @@ -4049,7 +4139,12 @@ zone_alloc_item(uma_zone_t zone, void *udata, int * to be both zone-init'd as well as zone-ctor'd. */ if (zone->uz_init != NULL) { - if (zone->uz_init(item, zone->uz_size, flags) != 0) { + int error; + + kasan_mark_item_valid(zone, item); + error = zone->uz_init(item, zone->uz_size, flags); + kasan_mark_item_invalid(zone, item); + if (error != 0) { zone_free_item(zone, item, udata, SKIP_FINI | SKIP_CNT); goto fail_cnt; } @@ -4531,8 +4626,11 @@ zone_free_item(uma_zone_t zone, void *item, void * item_dtor(zone, item, zone->uz_size, udata, skip); - if (skip < SKIP_FINI && zone->uz_fini) + if (skip < SKIP_FINI && zone->uz_fini) { + kasan_mark_item_valid(zone, item); zone->uz_fini(item, zone->uz_size); + kasan_mark_item_invalid(zone, item); + } zone->uz_release(zone->uz_arg, &item, 1); Index: src/sys/vm/vm_glue.c =================================================================== --- src/sys/vm/vm_glue.c (revision 368820) +++ src/sys/vm/vm_glue.c (working copy) @@ -68,6 +68,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include #include #include @@ -86,7 +87,6 @@ __FBSDID("$FreeBSD$"); #include #include #include -#include #include #include #include @@ -385,6 +385,7 @@ vm_thread_new(struct thread *td, int pages) return (0); td->td_kstack = ks; td->td_kstack_pages = pages; + kasan_mark((void *)ks, ptoa(pages), ptoa(pages), 0); return (1); } @@ -401,6 +402,7 @@ vm_thread_dispose(struct thread *td) ks = td->td_kstack; td->td_kstack = 0; td->td_kstack_pages = 0; + kasan_mark((void *)ks, 0, ptoa(pages), KASAN_KSTACK_FREED); if (pages == kstack_pages) uma_zfree(kstack_cache, (void *)ks); else Index: src/sys/vm/vm_page.c =================================================================== --- src/sys/vm/vm_page.c (revision 368820) +++ src/sys/vm/vm_page.c (working copy) @@ -653,7 +653,9 @@ vm_page_startup(vm_offset_t vaddr) * In other words, solve * "available physical memory" - round_page(page_range * * sizeof(struct vm_page)) = page_range * PAGE_SIZE - * for page_range. + * for page_range. + * + * XXXMJ needs update for KASAN */ low_avail = phys_avail[0]; high_avail = phys_avail[1]; -- Test scenario: misc/trim5.sh