panic: unp_pcb_lock_peer: socket 0xfffff80771055d00 is connecting, vmcore.226 KDB: debugger backends: ddb KDB: current backend: ddb ---<>--- Copyright (c) 1992-2020 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 13.0-CURRENT #2 r365247M: Wed Sep 2 20:14:16 CEST 2020 pho@mercat1.netperf.freebsd.org:/usr/src/sys/amd64/compile/PHO amd64 FreeBSD clang version 10.0.1 (git@github.com:llvm/llvm-project.git llvmorg-10.0.1-0-gef32c611aa2) WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. VT(vga): resolution 640x480 CPU: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz (3500.06-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306f2 Family=0x6 Model=0x3f Stepping=2 Features=0xbfebfbff Features2=0x7ffefbff AMD Features=0x2c100800 AMD Features2=0x21 Structured Extended Features=0x37ab XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics real memory = 34359738368 (32768 MB) avail memory = 33165787136 (31629 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs FreeBSD/SMP: 1 package(s) x 6 core(s) x 2 hardware threads random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" arc4random: WARNING: initial seeding bypassed the cryptographic random device because it was not yet seeded and the knob 'bypass_before_seeding' was enabled. ioapic0 irqs 0-23 ioapic1 irqs 24-47 Launching APs: 1 2 3 9 11 8 4 10 7 5 6 Timecounter "TSC-low" frequency 1750032320 Hz quality 1000 random: entropy device external interface WARNING: Device "kbd" is Giant locked and may be deleted before FreeBSD 13.0. kbd1 at kbdmux0 000.000052 [4346] netmap_init netmap: loaded module [ath_hal] loaded nexus0 vtvga0: cryptosoft0: acpi0: acpi0: Power Button (fixed) cpu0: on acpi0 atrtc0: port 0x70-0x71,0x74-0x77 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Event timer "HPET4" frequency 14318180 Hz quality 340 Event timer "HPET5" frequency 14318180 Hz quality 340 Event timer "HPET6" frequency 14318180 Hz quality 340 Event timer "HPET7" frequency 14318180 Hz quality 340 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: on acpi0 pci0: on pcib0 pci0: at device 11.1 (no driver attached) pci0: at device 11.2 (no driver attached) pci0: at device 16.1 (no driver attached) pci0: at device 16.6 (no driver attached) pci0: at device 18.1 (no driver attached) acpi_syscontainer0: on acpi0 acpi_syscontainer1: on acpi0 acpi_syscontainer2: on acpi0 acpi_syscontainer3: on acpi0 apei0: on acpi0 pcib1: port 0xcf8-0xcff on acpi0 pci1: on pcib1 pcib2: irq 26 at device 1.0 on pci1 pci2: on pcib2 pcib3: irq 32 at device 2.0 on pci1 pci3: on pcib3 pci3: at device 0.0 (no driver attached) pci3: at device 0.1 (no driver attached) pcib4: irq 40 at device 3.0 on pci1 pci4: on pcib4 pci1: at device 17.0 (no driver attached) ahci0: port 0xf110-0xf117,0xf100-0xf103,0xf0f0-0xf0f7,0xf0e0-0xf0e3,0xf020-0xf03f mem 0xfb438000-0xfb4387ff irq 16 at device 17.4 on pci1 ahci0: AHCI v1.30 with 4 6Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahciem0: on ahci0 xhci0: mem 0xfb400000-0xfb40ffff irq 19 at device 20.0 on pci1 xhci0: 32 bytes context size, 64-bit DMA usbus0 on xhci0 usbus0: 5.0Gbps Super Speed USB v3.0 pci1: at device 22.0 (no driver attached) pci1: at device 22.1 (no driver attached) ehci0: mem 0xfb434000-0xfb4343ff irq 18 at device 26.0 on pci1 usbus1: EHCI version 1.0 usbus1 on ehci0 usbus1: 480Mbps High Speed USB v2.0 pcib5: irq 16 at device 28.0 on pci1 pci5: on pcib5 pcib6: irq 18 at device 28.2 on pci1 pci6: on pcib6 pcib7: at device 0.0 on pci6 pci7: on pcib7 vgapci0: port 0xe000-0xe07f mem 0xfa000000-0xfaffffff,0xfb000000-0xfb01ffff irq 18 at device 0.0 on pci7 vgapci0: Boot video device pcib8: irq 16 at device 28.4 on pci1 pci8: on pcib8 igb0: port 0xd020-0xd03f mem 0xfb320000-0xfb33ffff,0xfb344000-0xfb347fff irq 16 at device 0.0 on pci8 igb0: Using 1024 TX descriptors and 1024 RX descriptors igb0: queue equality override not set, capping rx_queues at 6 and tx_queues at 6 igb0: Using 6 RX queues 6 TX queues igb0: U0 options=4e507bb ether 0c:c4:7a:a8:cd:db media: Ethernet autoselect status: no carrier nd6 options=29 Starting devd. igb0: link state changed to UP Autoloading module: ioat.ko ioat0: mem 0xfb42c000-0xfb42ffff irq 31 at device 4.0 on pci1 ioat0: Capabilities: 2f7 ioat1: mem 0xfb428000-0xfb42bfff irq 39 at device 4.1 on pci1 ioat1: Capabilities: 2f7 ioat2: mem 0xfb424000-0xfb427fff irq 31 at device 4.2 on pci1 ioat2: Capabilities: f7 ioat3: mem 0xfb420000-0xfb423fff irq 39 at device 4.3 on pci1 ioat3: Capabilities: f7 ioat4: mem 0xfb41c000-0xfb41ffff irq 31 at device 4.4 on pci1 ioat4: Capabilities: f7 ioat5: mem 0xfb418000-0xfb41bfff irq 39 at device 4.5 on pci1 ioat5: Capabilities: f7 ioat6: mem 0xfb414000-0xfb417fff irq 31 at device 4.6 on pci1 ioat6: Capabilities: f7 ioat7: mem 0xfb410000-0xfb413fff irq 39 at device 4.7 on pci1 ioat7: Capabilities: f7 Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Starting Network: igb1. igb1: flags=8822 metric 0 mtu 1500 options=4e507bb ether 0c:c4:7a:a8:cd:db media: Ethernet autoselect status: no carrier nd6 options=29 Autoloading module: uhid.ko Autoloading module: ums.ko ums0 on uhub3 ums0: on usbus0 ums0: 3 buttons and [Z] coordinates ID=0 Starting dhclient. DHCPREQUEST on igb0 to 255.255.255.255 port 67 DHCPACK from 192.168.5.1 if_delmulti_locked: detaching ifnet instance 0xfffff80039054000 bound to 192.168.5.171 -- renewal in 300 seconds. add host 127.0.0.1: gateway lo0 fib 0: route already in table add host ::1: gateway lo0 fib 0: route already in table add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Creating and/or trimming log files. Starting syslogd. Setting date via ntp. 2 Sep 20:24:01 ntpdate[2060]: step time server 192.168.5.1 offset -0.742537 sec Starting rpcbind. NFS access cache time=60 Updating CPU Microcode... CPU: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz (3500.06-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306f2 Family=0x6 Model=0x3f Stepping=2 Features=0xbfebfbff Features2=0x7ffefbff AMD Features=0x2c100800 AMD Features2=0x21 Structured Extended Features=0x37ab Structured Extended Features3=0x9c000400 XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics Done. Starting watchdogd. No core dumps found. Clearing /tmp (X related). Starting statd. Starting lockd. Recovering vi editor sessions:. fscklock order reversal: 1st 0xfffff800459b4070 ufs (ufs, lockmgr) @ kern/vfs_mount.c:1711 2nd 0xfffff800456d02f8 devfs (devfs, lockmgr) @ kern/vfs_subr.c:3118 lock order devfs -> ufs established at: #0 0xffffffff80c44b8c at witness_checkorder+0x46c #1 0xffffffff80ba7ea2 at lockmgr_lock_flags+0x172 #2 0xffffffff80e9828d at ffs_lock+0x8d #3 0xffffffff810d66c0 at VOP_LOCK1_APV+0x40 #4 0xffffffff80cd2b24 at _vn_lock+0x54 #5 0xffffffff80cb1f60 at vfs_domount+0xe80 #6 0xffffffff80cb0321 at vfs_donmount+0x871 #7 0xffffffff80cb4a37 at kernel_mount+0x57 #8 0xffffffff80cb7131 at parse_mount+0x4a1 #9 0xffffffff80cb5659 at vfs_mountroot+0x589 #10 0xffffffff80b69df8 at start_init+0x28 #11 0xffffffff80b932d0 at fork_exit+0x80 #12 0xffffffff80ff8b9e at fork_trampoline+0xe lock order ufs -> devfs attempted at: #0 0xffffffff80c454d8 at witness_checkorder+0xdb8 #1 0xffffffff80ba7ea2 at lockmgr_lock_flags+0x172 #2 0xffffffff810d66c0 at VOP_LOCK1_APV+0x40 #3 0xffffffff80cd2b24 at _vn_lock+0x54 #4 0xffffffff80cbbdf4 at vput_final+0x114 #5 0xffffffff80e95f1f at ffs_unmount+0x3af #6 0xffffffff80cb304d at dounmount+0x43d #7 0xffffffff80cb2bb9 at kern_unmount+0x329 #8 0xffffffff81024d19 at amd64_syscall+0x159 #9 0xffffffff80ff846e at fast_syscall_common+0xf8 ** /dev/gpt/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 536 files, 178051 used, 1087356 free (76 frags, 135910 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** Updating motd:. Mounting late filesystems:. Security policy loaded: MAC/ntpd (mac_ntpd) Starting ntpd. Starting powerd. Configuring vt: blanktime. Performing sanity check on sshd configuration. Starting sshd. Starting cron. sysctl: unknown oid 'debug.fail_point.pmap_allocpte_page_alloc' at line 12 Wed Sep 2 20:24 FreeBSD/amd64 (mercat1.netperf.freebsd.org) (ttyu1) login: root Sep 2 20:24:08 mercat1 login[2420]: ROOT LOGIN (root) ON ttyu1 Last login: Wed Sep 2 19:24:05 on ttyu1 FreeBSD 13.0-CURRENT (PHO) #2 r365247M: Wed Sep 2 20:14:16 CEST 2020 root@mercat1:~ # Sep 2 20:24:15 mercat1 sshd[2425]: error: PAM: Authentication error for pho from zoo.netperf.freebsd.org Sep 2 20:24:16 mercat1 syslogd: last message repeated 1 times Sep 2 20:24:31 mercat1 su[2440]: pho to root on /dev/pts/0 20200902 20:24:36 all (1/49): callout_reset_on.sh Expensive timeout(9) function: 0xffffffff80bf0d70(0xfffff805b1309520) 0.010283303 s 20200902 20:25:38 all (2/49): callout_reset_on2.sh Expensive timeout(9) function: 0xffffffff80db5720(0xfffffe00f00abc70) 3.486998641 s 20200902 20:35:40 all (3/49): datagram.sh stress2: newblk leak: 2/6. 20200902 20:35:43 all (4/49): datagram2.sh 20200902 20:35:44 all (5/49): datagram3.sh 20200902 20:35:46 all (6/49): jumbo.sh 20200902 20:36:47 all (7/49): kevent7.sh lock order reversal: (sleepable after non-sleepable) 1st 0xfffff80003962808 seqflq (seqflq, sleep mutex) @ dev/sound/midi/sequencer.c:757 2nd 0xffffffff81ae5ce0 midistat lock (midistat lock, sx) @ dev/sound/midi/midi.c:1481 lock order seqflq -> midistat lock attempted at: #0 0xffffffff80c454d8 at witness_checkorder+0xdb8 #1 0xffffffff80be1687 at _sx_xlock+0x67 #2 0xffffffff80980b52 at midimapper_open+0x22 #3 0xffffffff80983a32 at mseq_open+0xf2 #4 0xffffffff80a7861f at devfs_open+0x12f #5 0xffffffff810d3b36 at VOP_OPEN_APV+0x56 #6 0xffffffff80cd28be at vn_open_vnode+0x18e #7 0xffffffff80cd2255 at vn_open_cred+0x395 #8 0xffffffff80cc95fa at kern_openat+0x25a #9 0xffffffff81024d19 at amd64_syscall+0x159 #10 0xffffffff80ff846e at fast_syscall_common+0xf8 20200902 20:41:55 all (8/49): multicast.sh stress2: freework leak: 1/6. stress2: newblk leak: 2/6. 20200902 20:42:16 all (9/49): overflow3.sh if_delmulti_locked: detaching ifnet instance 0xfffff80039054000 20200902 20:44:18 all (10/49): sctp.sh 20200902 20:45:25 all (11/49): sendfile.sh stress2: newblk leak: 2/6. 20200902 20:45:27 all (12/49): sendfile10.sh 20200902 20:47:29 all (13/49): sendfile11.sh GEOM_ELI: Device md10.eli created. GEOM_ELI: Encryption: AES-XTS 128 GEOM_ELI: Crypto: software GEOM_ELI: md10 has been killed. GEOM_ELI: Device md10.eli destroyed. stress2: freework leak: 1/6. 20200902 20:51:21 all (14/49): sendfile12.sh 20200902 20:56:32 all (15/49): sendfile13.sh 20200902 21:02:30 all (16/49): sendfile14.sh 20200902 21:13:00 all (17/49): sendfile15.sh 20200902 21:18:10 all (18/49): sendfile16.sh 20200902 21:23:17 all (19/49): sendfile17.sh 20200902 21:28:20 all (20/49): sendfile18.sh GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. stress2: newblk leak: 2/6. 20200902 21:28:28 all (21/49): sendfile19.sh 20200902 21:28:29 all (22/49): sendfile2.sh stress2: freework leak: 2/6. stress2: newblk leak: 1/6. 20200902 21:28:31 all (23/49): sendfile20.sh 20200902 21:28:33 all (24/49): sendfile21.sh 20200902 21:32:28 all (25/49): sendfile22.sh 20200902 21:36:22 all (26/49): sendfile23.sh 20200902 21:40:13 all (27/49): sendfile24.sh GEOM_NOP: Device md10.nop created. g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 g_vfs_done():md10.nop[READ(offset=16977920, length=4096)]error = 5 GEOM_NOP: Device md10.nop removed. stress2: newblk leak: 2/6. 20200902 21:40:25 all (28/49): sendfile3.sh 20200902 21:40:55 all (29/49): sendfile4.sh stress2: freework leak: 2/6. stress2: newblk leak: 2/6. 20200902 21:40:57 all (30/49): sendfile5.sh 20200902 21:41:24 all (31/49): sendfile6.sh stress2: freework leak: 3/11. stress2: newblk leak: 3/11. 20200902 21:41:27 all (32/49): sendfile8.sh 20200902 21:42:28 all (33/49): sendfile9.sh 20200902 21:43:30 all (34/49): sendfile_shm.sh stress2: freework leak: 1/6. stress2: newblk leak: 3/7. 20200902 21:43:46 all (35/49): sendmsg.sh 20200902 21:48:50 all (36/49): sendmsg2.sh 20200902 21:49:22 all (37/49): setsockopt.sh 20200902 21:52:24 all (38/49): setsockopt2.sh 20200902 21:54:26 all (39/49): socketpair.sh stress2: newblk leak: 2/6. 20200902 21:54:28 all (40/49): socketpair2.sh 20200902 21:54:30 all (41/49): socketpair3.sh 20200902 21:55:31 all (42/49): socketpair4.sh 20200902 21:58:33 all (43/49): syscall4.sh WARNING pid 43497 (calls): ioctl sign-extension ioctl 488203bed4f8 WARNING pid 43566 (calls): ioctl sign-extension ioctl 4f2fc7689c4a WARNING pid 43643 (calls): ioctl sign-extension ioctl 20f0051d15fdd69f WARNING pid 44887 (calls): ioctl sign-extension ioctl 636520db2581 WARNING pid 44910 (calls): ioctl sign-extension ioctl ca745158d2a1c803 WARNING pid 45002 (calls): ioctl sign-extension ioctl 371dfba6eb0c9747 WARNING pid 45113 (calls): ioctl sign-extension ioctl 3ca706d9feb7 WARNING pid 45158 (calls): ioctl sign-extension ioctl 44934e3e1b68 WARNING pid 45182 (calls): ioctl sign-extension ioctl 7d2ba9a8da8d9c83 WARNING pid 45192 (calls): ioctl sign-extension ioctl b795107534beb3cf WARNING pid 45306 (calls): ioctl sign-extension ioctl 56a44c9af721 WARNING pid 45950 (calls): ioctl sign-extension ioctl df81636378e89a89 WARNING pid 46306 (calls): ioctl sign-extension ioctl 5ccf110924f WARNING pid 46306 (calls): ioctl sign-extension ioctl 994620528b6874b8 WARNING pid 46490 (calls): ioctl sign-extension ioctl 37f64f42b4dd735d WARNING pid 46753 (calls): ioctl sign-extension ioctl b737589209711bad WARNING pid 46770 (calls): ioctl sign-extension ioctl 14254af8a5bbe4fd WARNING pid 46821 (calls): ioctl sign-extension ioctl 7c4614c4ca8f WARNING pid 46896 (calls): ioctl sign-extension ioctl 2be448799eca7c2 WARNING pid 46979 (calls): ioctl sign-extension ioctl 3eb2593442bb WARNING pid 47035 (calls): ioctl sign-extension ioctl ad057b0406318d55 WARNING pid 47186 (calls): ioctl sign-extension ioctl 8601e28f7cb16ae1 WARNING pid 48067 (calls): ioctl sign-extension ioctl a8c64ae73b0088e1 WARNING pid 48356 (calls): ioctl sign-extension ioctl aeed0aabdb6da738 WARNING pid 48417 (calls): ioctl sign-extension ioctl 6d4e913e4c39 WARNING pid 48508 (calls): ioctl sign-extension ioctl e76206fc18b1ae51 WARNING pid 48569 (calls): ioctl sign-extension ioctl 3b67cde5dbeb WARNING pid 49669 (calls): ioctl sign-extension ioctl eea98baec7c67d6c WARNING pid 49970 (calls): ioctl sign-extension ioctl 2a1ebf30bcb4 WARNING pid 50034 (calls): ioctl sign-extension ioctl 7b2af76aca2e WARNING pid 50082 (calls): ioctl sign-extension ioctl 7e6286990415a399 WARNING pid 50167 (calls): ioctl sign-extension ioctl 6448ed33cb80 WARNING pid 50233 (calls): ioctl sign-extension ioctl 442350eac07d WARNING pid 50281 (calls): ioctl sign-extension ioctl 289c45628204 WARNING pid 50312 (calls): ioctl sign-extension ioctl 4c98fddd903a2d79 WARNING pid 51379 (calls): ioctl sign-extension ioctl 3fc9742c88f0 WARNING pid 51676 (calls): ioctl sign-extension ioctl 6f36a8d95383 WARNING pid 51681 (calls): ioctl sign-extension ioctl db31fbf3890a8e30 WARNING pid 51698 (calls): ioctl sign-extension ioctl 1d95700f13a56838 WARNING pid 51701 (calls): ioctl sign-extension ioctl b1feaa7282bb1273 WARNING pid 51801 (calls): ioctl sign-extension ioctl 4013e2de96e9bc33 WARNING pid 52720 (calls): ioctl sign-extension ioctl 5b1f135ae2bf WARNING pid 53306 (calls): ioctl sign-extension ioctl dd54582828e618d6 WARNING pid 53313 (calls): ioctl sign-extension ioctl 410a0f006d57 WARNING pid 53369 (calls): ioctl sign-extension ioctl 2c2b36553d000adb WARNING pid 53479 (calls): ioctl sign-extension ioctl 10c031804a0cbcf4 WARNING pid 54099 (calls): ioctl sign-extension ioctl 273c6a1bff22 WARNING pid 54240 (calls): ioctl sign-extension ioctl 565eacd7970 WARNING pid 54920 (calls): ioctl sign-extension ioctl 5a5881bee72a WARNING pid 54922 (calls): ioctl sign-extension ioctl 48b8d66431a6 WARNING pid 54939 (calls): ioctl sign-extension ioctl f1ce78387a7c1cbb WARNING pid 55039 (calls): ioctl sign-extension ioctl a4ea1895bc6da496 WARNING pid 55152 (calls): ioctl sign-extension ioctl 3a7cdc1d25c6da67 WARNING pid 56369 (calls): ioctl sign-extension ioctl 13d5ca142dc3 WARNING pid 56408 (calls): ioctl sign-extension ioctl dd39f9ba9910f0b0 WARNING pid 56502 (calls): ioctl sign-extension ioctl 8286b82cd4316d0b WARNING pid 56502 (calls): ioctl sign-extension ioctl 687eddecd8badee4 WARNING pid 56512 (calls): ioctl sign-extension ioctl a5722473ae1e02c6 WARNING pid 56518 (calls): ioctl sign-extension ioctl 2b8a0d3ebb48 WARNING pid 56569 (calls): ioctl sign-extension ioctl 67cd1906f089 WARNING pid 56639 (calls): ioctl sign-extension ioctl 1cacbcaa1086 WARNING pid 56657 (calls): ioctl sign-extension ioctl 2428cb1d5b8c9fe8 WARNING pid 56667 (calls): ioctl sign-extension ioctl 58313c062277 WARNING pid 56669 (calls): ioctl sign-extension ioctl 845701c02bb3bbe9 WARNING pid 57351 (calls): ioctl sign-extension ioctl 5555c03aca9 WARNING pid 58094 (calls): ioctl sign-extension ioctl 850371213555e9cd WARNING pid 58118 (calls): ioctl sign-extension ioctl 28957e01888fd8bf WARNING pid 58175 (calls): ioctl sign-extension ioctl 3587254573b5 WARNING pid 58278 (calls): ioctl sign-extension ioctl 2e5146d038526faf WARNING pid 58384 (calls): ioctl sign-extension ioctl 3541e1c5f578f939 WARNING pid 58403 (calls): ioctl sign-extension ioctl 618d0d0fd3a2 WARNING pid 58406 (calls): ioctl sign-extension ioctl 26a038ac95a WARNING pid 58447 (calls): ioctl sign-extension ioctl 32eb8548cb820167 WARNING pid 59507 (calls): ioctl sign-extension ioctl 9cf88687d7b6d869 WARNING pid 59507 (calls): ioctl sign-extension ioctl 130fc543a393 WARNING pid 59733 (calls): ioctl sign-extension ioctl 3f8be6ce09fc WARNING pid 59742 (calls): ioctl sign-extension ioctl 7ba0f8f8042926c3 WARNING pid 59836 (calls): ioctl sign-extension ioctl 2b00bc3ee675 WARNING pid 59904 (calls): ioctl sign-extension ioctl 7ea3aa38c897 WARNING pid 59927 (calls): ioctl sign-extension ioctl 45b01dae1266 WARNING pid 60519 (calls): ioctl sign-extension ioctl 14b350b1f1fb WARNING pid 61125 (calls): ioctl sign-extension ioctl 665e15cfefbe WARNING pid 61177 (calls): ioctl sign-extension ioctl 602314759477 20200902 22:00:35 all (44/49): tcp3.sh 20200902 22:01:53 all (45/49): tcp4.sh witness_lock_list_get: witness exhausted 20200902 22:03:31 all (46/49): tmpfs7.sh stress2: freework leak: 5/11. stress2: newblk leak: 6/11. 20200902 22:03:33 all (47/49): unix_socket.sh 20200902 22:03:37 all (48/49): unix_socket_detach.sh panic: unp_pcb_lock_peer: socket 0xfffff80771055d00 is connecting cpuid = 11 time = 1599077019 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00f0ce87a0 vpanic() at vpanic+0x182/frame 0xfffffe00f0ce87f0 panic() at panic+0x43/frame 0xfffffe00f0ce8850 unp_pcb_lock_peer() at unp_pcb_lock_peer+0xd6/frame 0xfffffe00f0ce88c0 unp_drop() at unp_drop+0x63/frame 0xfffffe00f0ce8900 uipc_abort() at uipc_abort+0xf7/frame 0xfffffe00f0ce8950 soabort() at soabort+0x4c/frame 0xfffffe00f0ce8970 soclose() at soclose+0x3e3/frame 0xfffffe00f0ce89e0 _fdrop() at _fdrop+0x1a/frame 0xfffffe00f0ce8a00 closef() at closef+0x1ea/frame 0xfffffe00f0ce8a90 closefp() at closefp+0x96/frame 0xfffffe00f0ce8ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f0ce8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f0ce8bf0 --- syscall (6, FreeBSD ELF64, sys_close), rip = 0x8004290fa, rsp = 0x7fffffffe618, rbp = 0x7fffffffe6b0 --- KDB: enter: panic [ thread pid 70162 tid 101705 ] Stopped at kdb_enter+0x37: movq $0,0x10b7396(%rip) db> run pho db:0:pho> set $lines 20000 db:0:pho> run pho1 db:1:pho1> dump Dumping 3944 out of 32628 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Dump complete db:1:pho1> bt Tracing pid 70162 tid 101705 td 0xfffffe00f1111c00 kdb_enter() at kdb_enter+0x37/frame 0xfffffe00f0ce87a0 vpanic() at vpanic+0x19e/frame 0xfffffe00f0ce87f0 panic() at panic+0x43/frame 0xfffffe00f0ce8850 unp_pcb_lock_peer() at unp_pcb_lock_peer+0xd6/frame 0xfffffe00f0ce88c0 unp_drop() at unp_drop+0x63/frame 0xfffffe00f0ce8900 uipc_abort() at uipc_abort+0xf7/frame 0xfffffe00f0ce8950 soabort() at soabort+0x4c/frame 0xfffffe00f0ce8970 soclose() at soclose+0x3e3/frame 0xfffffe00f0ce89e0 _fdrop() at _fdrop+0x1a/frame 0xfffffe00f0ce8a00 closef() at closef+0x1ea/frame 0xfffffe00f0ce8a90 closefp() at closefp+0x96/frame 0xfffffe00f0ce8ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f0ce8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f0ce8bf0 --- syscall (6, FreeBSD ELF64, sys_close), rip = 0x8004290fa, rsp = 0x7fffffffe618, rbp = 0x7fffffffe6b0 --- db:1:pho1> show allpcpu Current CPU: 11 cpuid = 0 dynamic pcpu = 0x796b00 curthread = 0xfffffe00ad7d9300: pid 11 tid 100003 critnest 1 "idle: cpu0" curpcb = 0xfffffe00ad7d9810 fpcurthread = none idlethread = 0xfffffe00ad7d9300: tid 100003 "idle: cpu0" self = 0xffffffff82210000 curpmap = 0xffffffff81c8d060 tssp = 0xffffffff82210384 rsp0 = 0xfffffe00ad546cc0 kcr3 = 0x8000000002127002 ucr3 = 0xffffffffffffffff scr3 = 0x6a72e9e51 gs32p = 0xffffffff82210404 ldt = 0xffffffff82210444 tss = 0xffffffff82210434 curvnet = 0 spin locks held: cpuid = 1 dynamic pcpu = 0xfffffe00bc247b00 curthread = 0xfffffe00ad7d8c00: pid 11 tid 100004 critnest 1 "idle: cpu1" curpcb = 0xfffffe00ad7d9110 fpcurthread = none idlethread = 0xfffffe00ad7d8c00: tid 100004 "idle: cpu1" self = 0xffffffff82211000 curpmap = 0xffffffff81c8d060 tssp = 0xffffffff82211384 rsp0 = 0xfffffe00ad54bcc0 kcr3 = 0x8000000002127004 ucr3 = 0xffffffffffffffff scr3 = 0x51dde6b61 gs32p = 0xffffffff82211404 ldt = 0xffffffff82211444 tss = 0xffffffff82211434 curvnet = 0 spin locks held: cpuid = 2 dynamic pcpu = 0xfffffe00bc253b00 curthread = 0xfffffe00eddb7300: pid 70161 tid 104251 critnest 0 "unix_socket_detach" curpcb = 0xfffffe00eddb7810 fpcurthread = 0xfffffe00eddb7300: pid 70161 "unix_socket_detach" idlethread = 0xfffffe00ad7d8500: tid 100005 "idle: cpu2" self = 0xffffffff82212000 curpmap = 0xfffffe00f01874f0 tssp = 0xffffffff82212384 rsp0 = 0xfffffe00f5e4acc0 kcr3 = 0x8000000763e0952f ucr3 = 0x800000040f1b1d2f scr3 = 0x40f1b1d2f gs32p = 0xffffffff82212404 ldt = 0xffffffff82212444 tss = 0xffffffff82212434 curvnet = 0xfffff800037b6040 spin locks held: cpuid = 3 dynamic pcpu = 0xfffffe00bc25fb00 curthread = 0xfffffe00ad7d7e00: pid 11 tid 100006 critnest 1 "idle: cpu3" curpcb = 0xfffffe00ad7d8310 fpcurthread = none idlethread = 0xfffffe00ad7d7e00: tid 100006 "idle: cpu3" self = 0xffffffff82213000 curpmap = 0xffffffff81c8d060 tssp = 0xffffffff82213384 rsp0 = 0xfffffe00ad555cc0 kcr3 = 0x8000000002127004 ucr3 = 0xffffffffffffffff scr3 = 0x51dde6905 gs32p = 0xffffffff82213404 ldt = 0xffffffff82213444 tss = 0xffffffff82213434 curvnet = 0 spin locks held: cpuid = 4 dynamic pcpu = 0xfffffe00bc26bb00 curthread = 0xfffffe00ad7d7700: pid 11 tid 100007 critnest 1 "idle: cpu4" curpcb = 0xfffffe00ad7d7c10 fpcurthread = none idlethread = 0xfffffe00ad7d7700: tid 100007 "idle: cpu4" self = 0xffffffff82214000 curpmap = 0xffffffff81c8d060 tssp = 0xffffffff82214384 rsp0 = 0xfffffe00ad55acc0 kcr3 = 0x8000000002127002 ucr3 = 0xffffffffffffffff scr3 = 0x51dde6c0f gs32p = 0xffffffff82214404 ldt = 0xffffffff82214444 tss = 0xffffffff82214434 curvnet = 0 spin locks held: cpuid = 5 dynamic pcpu = 0xfffffe00bc277b00 curthread = 0xfffffe00ad7d7000: pid 11 tid 100008 critnest 1 "idle: cpu5" curpcb = 0xfffffe00ad7d7510 fpcurthread = none idlethread = 0xfffffe00ad7d7000: tid 100008 "idle: cpu5" self = 0xffffffff82215000 curpmap = 0xffffffff81c8d060 tssp = 0xffffffff82215384 rsp0 = 0xfffffe00ad55fcc0 kcr3 = 0x8000000002127003 ucr3 = 0xffffffffffffffff scr3 = 0x51dde6db1 gs32p = 0xffffffff82215404 ldt = 0xffffffff82215444 tss = 0xffffffff82215434 curvnet = 0 spin locks held: cpuid = 6 dynamic pcpu = 0xfffffe00bc283b00 curthread = 0xfffffe00ad7e1a00: pid 11 tid 100009 critnest 1 "idle: cpu6" curpcb = 0xfffffe00ad7e1f10 fpcurthread = none idlethread = 0xfffffe00ad7e1a00: tid 100009 "idle: cpu6" self = 0xffffffff82216000 curpmap = 0xffffffff81c8d060 tssp = 0xffffffff82216384 rsp0 = 0xfffffe00ad564cc0 kcr3 = 0x8000000002127002 ucr3 = 0xffffffffffffffff scr3 = 0x51dde6c0c gs32p = 0xffffffff82216404 ldt = 0xffffffff82216444 tss = 0xffffffff82216434 curvnet = 0 spin locks held: cpuid = 7 dynamic pcpu = 0xfffffe00bc28fb00 curthread = 0xfffffe00ad7e1300: pid 11 tid 100010 critnest 1 "idle: cpu7" curpcb = 0xfffffe00ad7e1810 fpcurthread = none idlethread = 0xfffffe00ad7e1300: tid 100010 "idle: cpu7" self = 0xffffffff82217000 curpmap = 0xffffffff81c8d060 tssp = 0xffffffff82217384 rsp0 = 0xfffffe00ad569cc0 kcr3 = 0x8000000002127002 ucr3 = 0xffffffffffffffff scr3 = 0x51dde684e gs32p = 0xffffffff82217404 ldt = 0xffffffff82217444 tss = 0xffffffff82217434 curvnet = 0 spin locks held: cpuid = 8 dynamic pcpu = 0xfffffe00bc29bb00 curthread = 0xfffffe00ad7e0c00: pid 11 tid 100011 critnest 1 "idle: cpu8" curpcb = 0xfffffe00ad7e1110 fpcurthread = none idlethread = 0xfffffe00ad7e0c00: tid 100011 "idle: cpu8" self = 0xffffffff82218000 curpmap = 0xffffffff81c8d060 tssp = 0xffffffff82218384 rsp0 = 0xfffffe00ad56ecc0 kcr3 = 0x8000000002127003 ucr3 = 0xffffffffffffffff scr3 = 0x51dde6cf2 gs32p = 0xffffffff82218404 ldt = 0xffffffff82218444 tss = 0xffffffff82218434 curvnet = 0 spin locks held: cpuid = 9 dynamic pcpu = 0xfffffe00bc2a7b00 curthread = 0xfffffe00ad7e0500: pid 11 tid 100012 critnest 1 "idle: cpu9" curpcb = 0xfffffe00ad7e0a10 fpcurthread = none idlethread = 0xfffffe00ad7e0500: tid 100012 "idle: cpu9" self = 0xffffffff82219000 curpmap = 0xffffffff81c8d060 tssp = 0xffffffff82219384 rsp0 = 0xfffffe00ad573cc0 kcr3 = 0x8000000002127003 ucr3 = 0xffffffffffffffff scr3 = 0x79c047c3a gs32p = 0xffffffff82219404 ldt = 0xffffffff82219444 tss = 0xffffffff82219434 curvnet = 0 spin locks held: cpuid = 10 dynamic pcpu = 0xfffffe00bc2b3b00 curthread = 0xfffffe00ad7dfe00: pid 11 tid 100013 critnest 1 "idle: cpu10" curpcb = 0xfffffe00ad7e0310 fpcurthread = none idlethread = 0xfffffe00ad7dfe00: tid 100013 "idle: cpu10" self = 0xffffffff8221a000 curpmap = 0xffffffff81c8d060 tssp = 0xffffffff8221a384 rsp0 = 0xfffffe00ad578cc0 kcr3 = 0x8000000002127008 ucr3 = 0xffffffffffffffff scr3 = 0x51dde6dc7 gs32p = 0xffffffff8221a404 ldt = 0xffffffff8221a444 tss = 0xffffffff8221a434 curvnet = 0 spin locks held: cpuid = 11 dynamic pcpu = 0xfffffe00bc2bfb00 curthread = 0xfffffe00f1111c00: pid 70162 tid 101705 critnest 1 "unix_socket_detach" curpcb = 0xfffffe00f1112110 fpcurthread = 0xfffffe00f1111c00: pid 70162 "unix_socket_detach" idlethread = 0xfffffe00ad7df700: tid 100014 "idle: cpu11" self = 0xffffffff8221b000 curpmap = 0xfffffe00f10254f0 tssp = 0xffffffff8221b384 rsp0 = 0xfffffe00f0ce8cc0 kcr3 = 0x80000007fac96403 ucr3 = 0x800000004788ac03 scr3 = 0x4788ac03 gs32p = 0xffffffff8221b404 ldt = 0xffffffff8221b444 tss = 0xffffffff8221b434 curvnet = 0xfffff800037b6040 spin locks held: db:1:pho1> show alllocks db:1:pho1> show lockedvnods Locked vnodes db:1:pho1> show mount 0xfffffe00dfef5700 /dev/gpt/work on / (ufs) 0xfffffe00dfef6280 devfs on /dev (devfs) 0xfffffe00dffef700 /dev/gpt/tmp on /tmp (ufs) More info: show mount db:1:pho1> show bufqueues bqempty: 102405 Buf domain 0 freebufs 15071 lofreebufs 219 hifreebufs 329 bufspace 1241088 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 29 (774144) dirtyq count 0 (0) wakeup 0 lim 25 CPU 1, 0, 2, 2, 5, 3, 5, 2, 8, 1, 2, 2, Locked buffers: 0 space 0 Total buffers: 8456 space 1241088 Buf domain 1 freebufs 15071 lofreebufs 219 hifreebufs 329 bufspace 1277952 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 1 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 15 (331776) dirtyq count 1 (32768) wakeup 0 lim 25 CPU 4, 4, 0, 13, 4, 1, 1, 2, 6, 0, 8, 3, Locked buffers: 0 space 0 Total buffers: 7979 space 1277952 Buf domain 2 freebufs 14161 lofreebufs 219 hifreebufs 329 bufspace 30928896 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 7 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 926 (29978624) dirtyq count 7 (184320) wakeup 0 lim 25 CPU 2, 4, 1, 8, 4, 0, 8, 3, 7, 0, 2, 0, Locked buffers: 0 space 0 Total buffers: 12851 space 30928896 Buf domain 3 freebufs 14546 lofreebufs 219 hifreebufs 329 bufspace 18370560 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 537 (17391616) dirtyq count 0 (0) wakeup 0 lim 25 CPU 6, 7, 3, 9, 6, 3, 0, 3, 5, 0, 6, 2, Locked buffers: 0 space 0 Total buffers: 8919 space 18370560 Buf domain 4 freebufs 15084 lofreebufs 219 hifreebufs 329 bufspace 749568 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 17 (372736) dirtyq count 0 (0) wakeup 0 lim 25 CPU 3, 1, 2, 4, 2, 1, 2, 3, 6, 3, 3, 2, Locked buffers: 0 space 0 Total buffers: 8334 space 749568 Buf domain 5 freebufs 13642 lofreebufs 219 hifreebufs 329 bufspace 47886336 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 1 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 1409 (45711360) dirtyq count 1 (4096) wakeup 0 lim 25 CPU 0, 3, 7, 21, 6, 8, 5, 3, 1, 4, 16, 7, Locked buffers: 0 space 0 Total buffers: 17487 space 47886336 Buf domain 6 freebufs 14986 lofreebufs 219 hifreebufs 329 bufspace 3747840 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 21 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 107 (2621440) dirtyq count 21 (688128) wakeup 0 lim 25 CPU 0, 0, 1, 0, 2, 1, 6, 2, 1, 0, 3, 3, Locked buffers: 0 space 0 Total buffers: 41689 space 3747840 db:1:pho1> show page vm_cnt.v_free_count: 7321384 vm_cnt.v_inactive_count: 206 vm_cnt.v_active_count: 1008 vm_cnt.v_laundry_count: 105 vm_cnt.v_wire_count: 798515 vm_cnt.v_free_reserved: 10644 vm_cnt.v_free_min: 51267 vm_cnt.v_free_target: 173136 vm_cnt.v_inactive_target: 259704 db:1:pho1> show pageq pq_free 7321384 dom 0 page_cnt 8124610 free 7321384 pq_act 1008 pq_inact 206 pq_laund 105 pq_unsw 0 db:0:pho> show freepages DOMAIN: 0 FREE LIST 0: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000201 | 000000 11 (008192K) | 000787 | 000000 10 (004096K) | 001888 | 000001 09 (002048K) | 002590 | 000000 08 (001024K) | 001629 | 000001 07 (000512K) | 003236 | 000001 06 (000256K) | 004040 | 000001 05 (000128K) | 005000 | 000002 04 (000064K) | 006114 | 000442 03 (000032K) | 006958 | 002752 02 (000016K) | 009475 | 005657 01 (000008K) | 027504 | 007509 00 (000004K) | 033809 | 009700 FREE LIST 1: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000000 | 000000 10 (004096K) | 000000 | 000000 09 (002048K) | 000000 | 000000 08 (001024K) | 000000 | 000000 07 (000512K) | 000000 | 000000 06 (000256K) | 000001 | 000000 05 (000128K) | 000001 | 000000 04 (000064K) | 000002 | 000000 03 (000032K) | 000000 | 000000 02 (000016K) | 000001 | 000000 01 (000008K) | 000000 | 000000 00 (000004K) | 000000 | 000000 db:0:pho> show uma Zone Size Used Free Requests Sleeps Bucket Total Mem XFree socket 960 74 939814 12132622 0 16 902292480 0 mbuf_cluster 2048 267166 48 1993598 0 8 547254272 0 mbuf 256 17313 1873753 1117694646 0 254 484112896 0 mbuf_jumbo_page 4096 187 61067 16585919 0 58 250896384 0 Files 80 124 941426 413930432 0 126 75324000 0 RADIX NODE 144 9083 512219 98205643 0 75 75067488 0 mbuf_packet 256 372 254507 68568440 0 254 65249024 0 VNODE 648 89416 284 145929 0 16 58125600 0 UMA Slabs 0 112 206776 2 293666 0 126 23159136 0 FFS2 dinode 256 89235 660 141913 0 62 23013120 0 swblk 136 1487 148617 1285297 0 62 20414144 0 4096 4096 81 4664 4864277 0 3 19435520 0 256 Bucket 2048 8784 66 2147378 0 10 18124800 0 BUF TRIE 144 395 105557 2817957 0 62 15257088 0 FFS inode 160 89235 1065 141915 0 62 14448000 0 VM OBJECT 264 7791 42459 43036888 0 30 13266000 0 VMSPACE 2536 43 4697 4864205 0 4 12020640 0 S VFS Cache 104 110833 2696 364514 0 126 11807016 0 THREAD 1792 4895 82 76757 0 8 8918784 0 MAP ENTRY 96 1676 89212 111852190 0 126 8725248 0 PROC 1312 80 4714 4864276 0 8 6289728 0 tcpcb 1040 31 5205 826592 0 8 5445440 0 filedesc0 1080 81 4686 4864277 0 8 5148360 0 tcp_inpcb 488 559 5953 826592 0 31 3177856 0 512 512 4816 72 4829 0 30 2502656 0 128 128 14453 1388 4678492 0 126 2027648 0 128 Bucket 1024 1480 147 2170645 0 16 1666048 0 128 128 11059 1341 301723 0 126 1587200 0 swpctrie 144 226 10750 95657 0 62 1580544 0 256 256 932 5008 9657795 0 62 1520640 0 128 128 10872 412 11327 0 126 1444352 0 256 256 4722 693 82771 0 62 1386240 0 4096 4096 336 0 2287 0 2 1376256 0 vmem btag 56 19153 4235 36392 0 254 1309728 0 DEVCTL 1024 0 1000 1357 0 16 1024000 0 64 64 9103 5765 5856204 0 254 951552 0 64 Bucket 512 1377 271 3005159 0 30 843776 0 64 64 9494 3106 22759174 0 254 806400 0 ksiginfo 112 4751 2269 5426814 0 126 786240 0 pbuf 984 0 788 0 0 0 775392 0 TURNSTILE 136 4978 356 4978 0 62 725424 0 8192 8192 86 0 365 0 1 704512 0 32 Bucket 256 1660 710 2590930 0 62 606720 0 32768 32768 12 6 610 0 1 589824 0 SLEEPQUEUE 88 4978 814 4978 0 126 509696 0 UMA Zones 2048 233 5 233 0 8 487424 0 64 64 590 6025 657771 0 254 423360 0 65536 65536 5 1 110 0 1 393216 0 32768 32768 12 0 7688 0 1 393216 0 32768 32768 12 0 15 0 1 393216 0 64 64 134 5977 20480002 0 254 391104 0 1024 1024 158 146 18493 0 16 311296 0 32 32 6877 2825 904048 0 254 310464 0 128 128 765 1405 774748 0 126 277760 0 16 16 14021 2979 1155923 0 254 272000 0 64 64 3290 931 315323 0 254 270144 0 2048 2048 18 112 2093926 0 8 266240 0 512 512 152 368 49749 0 30 266240 0 32 32 592 7724 2518267 0 254 266112 0 65536 65536 4 0 2483 0 1 262144 0 16384 16384 15 1 187621 0 1 262144 0 1024 1024 38 194 502848 0 16 237568 0 32768 32768 2 5 13310 0 1 229376 0 pipe 760 8 292 257310 0 16 228000 0 8 Bucket 80 1902 898 962156 0 126 224000 0 4096 4096 47 6 97728 0 2 217088 0 rl_entry 40 2520 2833 2520 0 254 214120 0 NAMEI 1024 0 192 6836030 0 16 196608 0 65536 65536 3 0 3 0 1 196608 0 65536 65536 3 0 3 0 1 196608 0 g_bio 408 0 480 7691713 0 30 195840 0 32 32 133 5915 2802445 0 254 193536 0 2048 2048 88 2 89 0 8 184320 0 16 Bucket 144 499 761 3929884 0 62 181440 0 selfd 64 330 2442 2138788 0 254 177408 0 4 Bucket 48 1429 2183 453748 0 254 173376 0 4096 4096 30 12 2093 0 2 172032 0 16384 16384 10 0 1649033 0 1 163840 0 sackhole 32 0 4914 56430 0 254 157248 0 1024 1024 7 145 43069 0 16 155648 0 2 Bucket 32 1957 2705 1131365 0 254 149184 0 AIOCB 752 0 195 897 0 16 146640 0 tcpreass 48 2 3022 41516 0 254 145152 0 AIO 208 0 684 728 0 62 142272 0 PWD 32 18 4392 65358 0 254 141120 0 8192 8192 17 0 3100 0 1 139264 0 unpcb 256 19 506 7804821 0 62 134400 0 tcptw 88 528 990 5417 0 126 133584 0 65536 65536 2 0 11 0 1 131072 0 syncache 168 1 767 14685 0 62 129024 0 fakepg 104 0 1170 204 0 126 121680 0 32 32 578 3076 30630 0 254 116928 0 256 256 70 380 61517 0 62 115200 0 256 256 43 377 3244 0 62 107520 0 8192 8192 13 0 55 0 1 106496 0 128 128 40 766 982 0 126 103168 0 ttyinq 160 330 270 540 0 62 96000 0 256 256 220 155 13757 0 62 96000 0 8192 8192 11 0 11 0 1 90112 0 ttyoutq 256 173 142 282 0 62 80640 0 UMA Kegs 384 203 3 203 0 30 79104 0 4096 4096 18 0 20 0 2 73728 0 L VFS Cache 320 159 69 160 0 30 72960 0 Mountpoints 2944 3 21 34 0 4 70656 0 64 64 154 917 682 0 254 68544 0 32 32 301 1841 142837 0 254 68544 0 65536 65536 0 1 272 0 1 65536 0 512 512 113 15 372 0 30 65536 0 128 128 142 354 1623 0 126 63488 0 DIRHASH 1024 46 14 920 0 16 61440 0 1024 1024 49 11 55 0 16 61440 0 512 512 44 76 1034993 0 30 61440 0 128 128 41 393 3446985 0 126 55552 0 2048 2048 18 8 192 0 8 53248 0 16 16 394 2856 3948 0 254 52000 0 128 128 18 385 141 0 126 51584 0 16384 16384 0 3 68 0 1 49152 0 16 16 472 2528 307358545 0 254 48000 0 cpuset 104 27 407 402 0 126 45136 0 512 512 22 66 10917 0 30 45056 0 mt_stats_zone 64 443 261 443 0 254 45056 0 16 16 434 2316 21650 0 254 44000 0 2048 2048 13 7 1263 0 8 40960 0 64 pcpu 8 3263 1857 3264 0 254 40960 0 32 32 216 1044 10270 0 254 40320 0 udp_inpcb 488 20 60 755054 0 30 39040 0 64 64 25 542 5918 0 254 36288 0 procdesc 136 1 260 1030806 0 62 35496 0 256 256 22 113 824443 0 62 34560 0 32768 32768 1 0 1 0 1 32768 0 16384 16384 2 0 4 0 1 32768 0 512 512 24 40 25 0 30 32768 0 512 512 30 34 469899 0 30 32768 0 32 32 10 998 11 0 254 32256 0 KMAP ENTRY 96 31 302 38 0 126 31968 0 TMPFS node 232 0 136 4 0 62 31552 0 256 256 25 95 325 0 62 30720 0 1024 1024 17 11 44 0 16 28672 0 rtentry 176 12 149 15 0 62 28336 0 AIOP 32 4 878 4 0 254 28224 0 32 32 181 701 1736 0 254 28224 0 16 16 42 1708 468 0 254 28000 0 16 16 9 1741 152 0 254 28000 0 8192 8192 3 0 30 0 1 24576 0 udpcb 32 20 736 755054 0 254 24192 0 mt_zone 24 443 559 443 0 254 24048 0 16 16 32 1468 34 0 254 24000 0 swrbuf 984 0 24 492295 0 2 23616 0 1024 1024 13 7 42 0 16 20480 0 TMPFS dirent 64 0 315 2 0 254 20160 0 routing nhops 256 7 68 15 0 62 19200 0 16384 16384 1 0 1 0 1 16384 0 8192 8192 2 0 2 0 1 16384 0 8192 8192 2 0 2 0 1 16384 0 256 256 18 42 98 0 62 15360 0 2048 2048 4 2 5893 0 8 12288 0 4096 4096 2 0 1468 0 2 8192 0 4096 4096 2 0 1952 0 2 8192 0 1024 1024 7 1 31 0 16 8192 0 rentr 24 0 334 35 0 254 8016 0 FPU_save_area 832 1 8 1 0 16 7488 0 int pcpu 4 98 1438 98 0 254 6144 0 epoch_record pcpu 256 4 12 4 0 62 4096 0 2048 2048 1 1 1 0 8 4096 0 1024 1024 2 2 31 0 16 4096 0 512 512 1 7 197 0 30 4096 0 hostcache 96 1 41 1 0 126 4032 0 64 64 1 62 2 0 254 4032 0 ripcb 488 1 7 936549 0 30 3904 0 UMA Slabs 1 176 12 10 12 0 62 3872 0 mqnode 416 3 6 3 0 30 3744 0 vmem 1856 1 1 1 0 8 3712 0 SMR CPU 32 2 93 2 0 254 3040 0 SMR SHARED 24 2 93 2 0 254 2280 0 mqnotifier 216 0 0 0 0 62 0 0 mvdata 64 0 0 0 0 254 0 0 mqueue 264 0 0 0 0 30 0 0 sctp_asconf_ack 48 0 0 0 0 254 0 0 sctp_asconf 40 0 0 0 0 254 0 0 sctp_stream_msg_out 112 0 0 35000 0 126 0 0 sctp_readq 152 0 0 35000 0 62 0 0 sctp_chunk 152 0 0 140000 0 62 0 0 sctp_raddr 736 0 0 140930 0 16 0 0 sctp_laddr 48 0 0 0 0 254 0 0 sctp_asoc 2288 0 0 70930 0 4 0 0 sctp_ep 1280 0 0 823893 0 8 0 0 FFS1 dinode 128 0 0 0 0 126 0 0 IPsec SA lft_c 16 0 0 0 0 254 0 0 udplite_inpcb 488 0 0 0 0 30 0 0 tcp_log_node 120 0 0 0 0 126 0 0 tcp_log_bucket 176 0 0 0 0 62 0 0 tcp_log 416 0 0 0 0 30 0 0 tfo_ccache_entries 80 0 0 0 0 126 0 0 tfo 4 0 0 0 0 254 0 0 ipq 56 0 0 0 0 254 0 0 KNOTE 160 0 0 2510 0 62 0 0 itimer 352 0 0 0 0 30 0 0 AIOLIO 280 0 0 0 0 30 0 0 NCLNODE 744 0 0 0 0 16 0 0 LTS VFS Cache 360 0 0 0 0 30 0 0 STS VFS Cache 144 0 0 0 0 62 0 0 VNODEPOLL 120 0 0 0 0 126 0 0 vtnet_tx_hdr 24 0 0 0 0 254 0 0 crypto_session 72 0 0 13 0 126 0 0 cryptop 280 0 0 197156 0 30 0 0 nvme_request 128 0 0 0 0 126 0 0 IOMMU_MAP_ENTRY 120 0 0 0 0 126 0 0 mbuf_jumbo_16k 16384 0 0 0 0 1 0 0 mbuf_jumbo_9k 9216 0 0 0 0 1 0 0 audit_record 1280 0 0 0 0 8 0 0 domainset 40 0 0 225 0 254 0 0 MAC labels 40 0 0 0 0 254 0 0 vnpbuf 984 0 0 166177 0 8 0 0 clpbuf 984 0 0 1019810 0 2 0 0 mdpbuf 984 0 0 0 0 0 0 0 nfspbuf 984 0 0 0 0 2 0 0 swwbuf 984 0 0 684389 0 0 0 0 umtx_shm 88 0 0 0 0 126 0 0 umtx pi 96 0 0 0 0 126 0 0 rangeset pctrie nodes 144 0 0 0 0 62 0 0 kenv 258 0 0 19248 0 30 0 0 65536 65536 0 0 0 0 1 0 0 65536 65536 0 0 2 0 1 0 0 32768 32768 0 0 0 0 1 0 0 32768 32768 0 0 19504 0 1 0 0 32768 32768 0 0 0 0 1 0 0 16384 16384 0 0 0 0 1 0 0 16384 16384 0 0 23 0 1 0 0 16384 16384 0 0 0 0 1 0 0 8192 8192 0 0 17 0 1 0 0 4096 4096 0 0 0 0 2 0 0 2048 2048 0 0 4 0 8 0 0 2048 2048 0 0 0 0 8 0 0 16 16 0 0 4 0 254 0 0 UMA Hash 256 0 0 0 0 62 0 0 db:0:pho> show umacache Zone Size Used Free Requests Bucket debugnet_mbuf_packet 2048 0 0 0 0 debugnet_mbuf_cluster 2048 -6144 0 0 0 debugnet_mbuf 256 -10240 0 0 0 vm pgcache 4096 674109 1340 97733390 25 vm pgcache 4096 36885 1667 224737372 28 buf free cache 984 3370 156 4168253 16 buffer arena-65 65536 0 0 0 1 buffer arena-61 61440 0 0 0 1 buffer arena-57 57344 0 0 0 1 buffer arena-53 53248 0 0 0 1 buffer arena-49 49152 0 0 0 1 buffer arena-45 45056 0 0 0 1 buffer arena-40 40960 0 0 0 1 buffer arena-36 36864 0 0 0 1 buffer arena-32 32768 163 19 48316 1 buffer arena-28 28672 0 0 0 1 buffer arena-24 24576 0 0 0 1 buffer arena-20 20480 0 0 0 1 buffer arena-16 16384 3 0 1183 1 buffer arena-12 12288 0 0 0 1 buffer arena-81 8192 0 0 0 1 buffer arena-40 4096 0 0 0 2 kstack_cache 16384 4895 22 76757 1 db:0:pho> acttrace Tracing command unix_socket_detach pid 70162 tid 101705 td 0xfffffe00f1111c00 (CPU 11) kdb_enter() at kdb_enter+0x37/frame 0xfffffe00f0ce87a0 vpanic() at vpanic+0x19e/frame 0xfffffe00f0ce87f0 panic() at panic+0x43/frame 0xfffffe00f0ce8850 unp_pcb_lock_peer() at unp_pcb_lock_peer+0xd6/frame 0xfffffe00f0ce88c0 unp_drop() at unp_drop+0x63/frame 0xfffffe00f0ce8900 uipc_abort() at uipc_abort+0xf7/frame 0xfffffe00f0ce8950 soabort() at soabort+0x4c/frame 0xfffffe00f0ce8970 soclose() at soclose+0x3e3/frame 0xfffffe00f0ce89e0 _fdrop() at _fdrop+0x1a/frame 0xfffffe00f0ce8a00 closef() at closef+0x1ea/frame 0xfffffe00f0ce8a90 closefp() at closefp+0x96/frame 0xfffffe00f0ce8ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f0ce8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f0ce8bf0 --- syscall (6, FreeBSD ELF64, sys_close), rip = 0x8004290fa, rsp = 0x7fffffffe618, rbp = 0x7fffffffe6b0 --- Tracing command unix_socket_detach pid 70161 tid 104251 td 0xfffffe00eddb7300 (CPU 2) cpustop_handler() at cpustop_handler+0x26/frame 0xfffffe003dbdce00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003dbdce10 trap() at trap+0x56/frame 0xfffffe003dbdcf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003dbdcf20 --- trap 0x13, rip = 0xffffffff80c238b0, rsp = 0xfffffe00f5e4a7d0, rbp = 0xfffffe00f5e4a7e0 --- lock_delay() at lock_delay+0x30/frame 0xfffffe00f5e4a7e0 __mtx_lock_sleep() at __mtx_lock_sleep+0xe8/frame 0xfffffe00f5e4a870 __mtx_lock_flags() at __mtx_lock_flags+0xe5/frame 0xfffffe00f5e4a8c0 unp_pcb_lock_peer() at unp_pcb_lock_peer+0x237/frame 0xfffffe00f5e4a930 uipc_disconnect() at uipc_disconnect+0x7f/frame 0xfffffe00f5e4a970 soclose() at soclose+0x9b/frame 0xfffffe00f5e4a9e0 _fdrop() at _fdrop+0x1a/frame 0xfffffe00f5e4aa00 closef() at closef+0x1ea/frame 0xfffffe00f5e4aa90 closefp() at closefp+0x96/frame 0xfffffe00f5e4aad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f5e4abf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f5e4abf0 --- syscall (6, FreeBSD ELF64, sys_close), rip = 0x8004290fa, rsp = 0x7fffffffe618, rbp = 0x7fffffffe6b0 --- Tracing command idle pid 11 tid 100003 td 0xfffffe00ad7d9300 (CPU 0) cpustop_handler() at cpustop_handler+0x26/frame 0xffffffff81de70a0 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xffffffff81de70b0 trap() at trap+0x56/frame 0xffffffff81de71c0 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff81de71c0 --- trap 0x13, rip = 0xffffffff80fe1466, rsp = 0xfffffe00ad546a50, rbp = 0xfffffe00ad546a50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00ad546a50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00ad546aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00ad546ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00ad546ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00ad546bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad546bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad546bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xfffffe00ad7d8c00 (CPU 1) cpustop_handler() at cpustop_handler+0x26/frame 0xfffffe003dbd0e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003dbd0e10 trap() at trap+0x56/frame 0xfffffe003dbd0f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003dbd0f20 --- trap 0x13, rip = 0xffffffff80fe1466, rsp = 0xfffffe00ad54ba50, rbp = 0xfffffe00ad54ba50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00ad54ba50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00ad54baa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00ad54bac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00ad54bae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00ad54bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad54bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad54bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xfffffe00ad7d7e00 (CPU 3) cpustop_handler() at cpustop_handler+0x26/frame 0xfffffe003dbe8e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003dbe8e10 trap() at trap+0x56/frame 0xfffffe003dbe8f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003dbe8f20 --- trap 0x13, rip = 0xffffffff80fe1466, rsp = 0xfffffe00ad555a50, rbp = 0xfffffe00ad555a50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00ad555a50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00ad555aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00ad555ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00ad555ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00ad555bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad555bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad555bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100007 td 0xfffffe00ad7d7700 (CPU 4) cpustop_handler() at cpustop_handler+0x26/frame 0xfffffe003dbf4e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003dbf4e10 trap() at trap+0x56/frame 0xfffffe003dbf4f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003dbf4f20 --- trap 0x13, rip = 0xffffffff80fe1466, rsp = 0xfffffe00ad55aa50, rbp = 0xfffffe00ad55aa50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00ad55aa50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00ad55aaa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00ad55aac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00ad55aae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00ad55abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad55abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad55abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xfffffe00ad7d7000 (CPU 5) cpustop_handler() at cpustop_handler+0x26/frame 0xfffffe003dc00e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003dc00e10 trap() at trap+0x56/frame 0xfffffe003dc00f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003dc00f20 --- trap 0x13, rip = 0xffffffff80fe1466, rsp = 0xfffffe00ad55fa50, rbp = 0xfffffe00ad55fa50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00ad55fa50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00ad55faa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00ad55fac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00ad55fae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00ad55fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad55fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad55fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xfffffe00ad7e1a00 (CPU 6) cpustop_handler() at cpustop_handler+0x26/frame 0xfffffe003dc0ce00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003dc0ce10 trap() at trap+0x56/frame 0xfffffe003dc0cf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003dc0cf20 --- trap 0x13, rip = 0xffffffff80fe1466, rsp = 0xfffffe00ad564a50, rbp = 0xfffffe00ad564a50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00ad564a50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00ad564aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00ad564ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00ad564ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00ad564bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad564bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad564bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xfffffe00ad7e1300 (CPU 7) cpustop_handler() at cpustop_handler+0x26/frame 0xfffffe003dc18e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003dc18e10 trap() at trap+0x56/frame 0xfffffe003dc18f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003dc18f20 --- trap 0x13, rip = 0xffffffff80fe1466, rsp = 0xfffffe00ad569a50, rbp = 0xfffffe00ad569a50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00ad569a50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00ad569aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00ad569ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00ad569ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00ad569bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad569bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad569bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffffe00ad7e0c00 (CPU 8) cpustop_handler() at cpustop_handler+0x26/frame 0xfffffe003dc24e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003dc24e10 trap() at trap+0x56/frame 0xfffffe003dc24f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003dc24f20 --- trap 0x13, rip = 0xffffffff80fe1466, rsp = 0xfffffe00ad56ea50, rbp = 0xfffffe00ad56ea50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00ad56ea50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00ad56eaa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00ad56eac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00ad56eae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00ad56ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad56ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad56ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100012 td 0xfffffe00ad7e0500 (CPU 9) cpustop_handler() at cpustop_handler+0x26/frame 0xfffffe003dc30e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003dc30e10 trap() at trap+0x56/frame 0xfffffe003dc30f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003dc30f20 --- trap 0x13, rip = 0xffffffff80fe1466, rsp = 0xfffffe00ad573a50, rbp = 0xfffffe00ad573a50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00ad573a50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00ad573aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00ad573ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00ad573ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00ad573bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad573bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad573bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xfffffe00ad7dfe00 (CPU 10) cpustop_handler() at cpustop_handler+0x26/frame 0xfffffe003dc3ce00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003dc3ce10 trap() at trap+0x56/frame 0xfffffe003dc3cf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003dc3cf20 --- trap 0x13, rip = 0xffffffff80fe1466, rsp = 0xfffffe00ad578a50, rbp = 0xfffffe00ad578a50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00ad578a50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00ad578aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00ad578ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00ad578ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00ad578bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad578bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad578bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db:0:pho> ps pid ppid pgrp uid state wmesg wchan cmd 70162 70161 2443 0 R+ CPU 11 unix_socket_detach 70161 70150 2443 0 R+ CPU 2 unix_socket_detach 70150 70148 2443 0 S+ wait 0xfffff80519520a40 sh 70149 2445 2443 0 S+ piperd 0xfffff80165cf68e8 tee 70148 2445 2443 0 S+ wait 0xfffff802cc653a40 sh 69845 2728 2678 0 S nanslp 0xffffffff81cacdc1 sleep 43502 0 0 0 DL aiordy 0xfffff803fc51fa40 [aiod4] 43501 0 0 0 DL aiordy 0xfffff8052c266520 [aiod3] 43500 0 0 0 DL aiordy 0xfffff805b1a5a520 [aiod2] 43499 0 0 0 DL aiordy 0xfffff8052c306a40 [aiod1] 52305 0 0 0 DL waiting 0xffffffff823aff98 [sctp_iterator] 2731 2678 2678 0 S piperd 0xfffff800459e12f8 sh 2729 2678 2678 0 S piperd 0xfffff800457cebe0 awk 2728 2678 2678 0 S wait 0xfffff80045763a40 sh 2678 1 2678 0 SWs wait 0xfffff803fc0bb520 sh 2445 2443 2443 0 S+ wait 0xfffff80004971a40 sh 2443 2442 2443 0 SW+ wait 0xfffff80004971000 sh 2442 2440 2442 0 SW+ wait 0xfffff8006277ba40 bash 2440 2439 2440 0 SW+ pause 0xfffff8000496eae8 csh 2439 2431 2439 2006 SW+ wait 0xfffff80045b4ba40 su 2431 2430 2431 2006 SWs+ wait 0xfffff800459eb520 bash 2430 2425 2425 2006 S select 0xfffff800455670c0 sshd 2425 2358 2425 0 Ss select 0xfffff80039099cc0 sshd 2421 2420 2421 0 S+ ttyin 0xfffff800392168b0 csh 2420 1 2420 0 SWs+ wait 0xfffff803b128b520 login 2419 1 2419 0 Ss+ ttyin 0xfffff80039216cb0 getty 2418 1 2418 0 Ss+ ttyin 0xfffff800452ea0b0 getty 2417 1 2417 0 Ss+ ttyin 0xfffff800452ea4b0 getty 2416 1 2416 0 Ss+ ttyin 0xfffff800452ea8b0 getty 2415 1 2415 0 Ss+ ttyin 0xfffff800452eacb0 getty 2414 1 2414 0 Ss+ ttyin 0xfffff800452ed0b0 getty 2413 1 2413 0 Ss+ ttyin 0xfffff800452ed4b0 getty 2412 1 2412 0 Ss+ ttyin 0xfffff800452ed8b0 getty 2411 1 2411 0 Ss+ ttyin 0xfffff800452edcb0 getty 2362 1 2362 0 Ss nanslp 0xffffffff81cacdc4 cron 2358 1 2358 0 Ss select 0xfffff80003fbc1c0 sshd 2330 1 2330 0 Ss select 0xfffff80039099e40 powerd 2327 1 2327 123 Ss (threaded) ntpd 100201 S select 0xfffff8004557b140 ntpd 2261 1 2261 0 Ss rpcsvc 0xfffff8004537b760 NLM: master 2258 1 2258 0 Ss select 0xfffff800452bd2c0 rpc.statd 2212 1 2212 0 Ss nanslp 0xffffffff81cacdc0 watchdogd 2063 1 2063 0 Ss select 0xfffff80315066640 rpcbind 2053 1 2053 0 Ss select 0xfffff80315066540 syslogd 1982 1 1982 0 Ss select 0xfffff80045c15040 devd 1977 1 1977 65 Ss select 0xfffff80039099d40 dhclient 1918 1 1918 0 Ss select 0xfffff80045c15340 dhclient 1915 1 1915 0 Ss select 0xfffff80003fbc340 dhclient 33 0 0 0 DL syncer 0xffffffff81d99ba8 [syncer] 32 0 0 0 DL vlruwt 0xfffff800452d8a40 [vnlru] 31 0 0 0 DL (threaded) [bufdaemon] 100131 D qsleep 0xffffffff81d98ef0 [bufdaemon] 100132 D - 0xffffffff8180aa00 [bufspacedaemon-0] 100136 D - 0xffffffff81812cc0 [bufspacedaemon-1] 100137 D - 0xffffffff8181af80 [bufspacedaemon-2] 100138 D - 0xffffffff81823240 [bufspacedaemon-3] 100140 D - 0xffffffff8182b500 [bufspacedaemon-4] 100141 D - 0xffffffff818337c0 [bufspacedaemon-5] 100142 D - 0xffffffff8183ba80 [bufspacedaemon-6] 100154 D sdflush 0xfffff80045af3ce8 [/ worker] 100204 D sdflush 0xfffff80045ad48e8 [/tmp worker] 30 0 0 0 DL psleep 0xffffffff81dbe048 [vmdaemon] 29 0 0 0 DL (threaded) [pagedaemon] 100129 D psleep 0xffffffff81db24f8 [dom0] 100133 D launds 0xffffffff81db2504 [laundry: dom0] 100135 D umarcl 0xffffffff80eb79b0 [uma] 28 0 0 0 DL idle 0xfffff80045379520 [enc_daemon1] 27 0 0 0 DL idle 0xfffff80045379a40 [enc_daemon0] 26 0 0 0 DL - 0xffffffff81ae0988 [rand_harvestq] 25 0 0 0 DL - 0xffffffff81d9882c [soaiod4] 24 0 0 0 DL - 0xffffffff81d9882c [soaiod3] 23 0 0 0 DL - 0xffffffff81d9882c [soaiod2] 22 0 0 0 DL - 0xffffffff81d9882c [soaiod1] 21 0 0 0 DL (threaded) [usb] 100098 D - 0xfffffe00dabfc440 [usbus0] 100099 D - 0xfffffe00dabfc498 [usbus0] 100100 D - 0xfffffe00dabfc4f0 [usbus0] 100101 D - 0xfffffe00dabfc548 [usbus0] 100102 D - 0xfffffe00dabfc5a0 [usbus0] 100104 D - 0xfffffe00db0a0cf0 [usbus1] 100105 D - 0xfffffe00db0a0d48 [usbus1] 100106 D - 0xfffffe00db0a0da0 [usbus1] 100107 D - 0xfffffe00db0a0df8 [usbus1] 100108 D - 0xfffffe00db0a0e50 [usbus1] 100109 D - 0xfffffe00dc073cf0 [usbus2] 100110 D - 0xfffffe00dc073d48 [usbus2] 100111 D - 0xfffffe00dc073da0 [usbus2] 100112 D - 0xfffffe00dc073df8 [usbus2] 100113 D - 0xfffffe00dc073e50 [usbus2] 20 0 0 0 RL (threaded) [cam] 100092 D - 0xffffffff819b86c0 [doneq0] 100093 RunQ [doneq1] 100094 D - 0xffffffff819b87c0 [doneq2] 100126 D - 0xffffffff819b8590 [scanner] 19 0 0 0 DL crypto_ 0xfffff80003ba9c50 [crypto returns 11] 18 0 0 0 DL crypto_ 0xfffff80003ba9bf0 [crypto returns 10] 17 0 0 0 DL crypto_ 0xfffff80003ba9b90 [crypto returns 9] 16 0 0 0 DL crypto_ 0xfffff80003ba9b30 [crypto returns 8] 15 0 0 0 DL crypto_ 0xfffff80003ba9ad0 [crypto returns 7] 9 0 0 0 DL crypto_ 0xfffff80003ba9a70 [crypto returns 6] 8 0 0 0 DL crypto_ 0xfffff80003ba9a10 [crypto returns 5] 7 0 0 0 DL crypto_ 0xfffff80003ba99b0 [crypto returns 4] 6 0 0 0 DL crypto_ 0xfffff80003ba9950 [crypto returns 3] 5 0 0 0 DL crypto_ 0xfffff80003ba98f0 [crypto returns 2] 4 0 0 0 DL crypto_ 0xfffff80003ba9890 [crypto returns 1] 3 0 0 0 DL crypto_ 0xfffff80003ba9830 [crypto returns 0] 2 0 0 0 DL crypto_ 0xffffffff81dafa20 [crypto] 14 0 0 0 DL seqstat 0xfffff80003962888 [sequencer 00] 13 0 0 0 DL (threaded) [geom] 100063 D - 0xffffffff81c8bd50 [g_event] 100064 D - 0xffffffff81c8bd58 [g_up] 100065 D - 0xffffffff81c8bd60 [g_down] 12 0 0 0 WL (threaded) [intr] 100041 I [swi6: task queue] 100042 I [swi6: Giant taskq] 100045 I [swi5: fast taskq] 100048 I [swi4: clock (0)] 100049 I [swi4: clock (1)] 100050 I [swi4: clock (2)] 100051 I [swi4: clock (3)] 100052 I [swi4: clock (4)] 100053 I [swi4: clock (5)] 100054 I [swi4: clock (6)] 100055 I [swi4: clock (7)] 100056 I [swi4: clock (8)] 100057 I [swi4: clock (9)] 100058 I [swi4: clock (10)] 100059 I [swi4: clock (11)] 100060 I [swi1: netisr 0] 100061 I [swi3: vm] 100095 I [swi4: apei] 100096 I [irq56: ahci0] 100097 I [irq19: xhci0] 100103 I [irq18: ehci0 ehci1] 100114 I [irq71: ahci1] 100115 I [swi0: uart uart] 100178 I [irq72: ioat0] 100179 I [irq73: ioat1] 100180 I [irq74: ioat2] 100181 I [irq75: ioat3] 100182 I [irq76: ioat4] 100183 I [irq77: ioat5] 100184 I [irq78: ioat6] 100185 I [irq79: ioat7] 11 0 0 0 RL (threaded) [idle] 100003 Run CPU 0 [idle: cpu0] 100004 Run CPU 1 [idle: cpu1] 100005 CanRun [idle: cpu2] 100006 Run CPU 3 [idle: cpu3] 100007 Run CPU 4 [idle: cpu4] 100008 Run CPU 5 [idle: cpu5] 100009 Run CPU 6 [idle: cpu6] 100010 Run CPU 7 [idle: cpu7] 100011 Run CPU 8 [idle: cpu8] 100012 Run CPU 9 [idle: cpu9] 100013 Run CPU 10 [idle: cpu10] 100014 CanRun [idle: cpu11] 1 0 1 0 SLs wait 0xfffff800038df000 [init] 10 0 0 0 DL audit_w 0xffffffff81daff08 [audit] 0 0 0 0 DLs (threaded) [kernel] 100000 D swapin 0xffffffff81c8c2f0 [swapper] 100015 D - 0xfffff80003917100 [softirq_0] 100016 D - 0xfffff80003917000 [softirq_1] 100017 D - 0xfffff80003914e00 [softirq_2] 100018 D - 0xfffff80003914d00 [softirq_3] 100019 D - 0xfffff80003914c00 [softirq_4] 100020 D - 0xfffff80003914b00 [softirq_5] 100021 D - 0xfffff80003914a00 [softirq_6] 100022 D - 0xfffff80003914900 [softirq_7] 100023 D - 0xfffff80003914800 [softirq_8] 100024 D - 0xfffff80003914700 [softirq_9] 100025 D - 0xfffff80003914600 [softirq_10] 100026 D - 0xfffff80003914500 [softirq_11] 100027 D - 0xfffff80003914400 [if_io_tqg_0] 100028 D - 0xfffff80003914300 [if_io_tqg_1] 100029 D - 0xfffff80003914200 [if_io_tqg_2] 100030 D - 0xfffff80003914100 [if_io_tqg_3] 100031 D - 0xfffff80003914000 [if_io_tqg_4] 100032 D - 0xfffff80003915e00 [if_io_tqg_5] 100033 D - 0xfffff80003915d00 [if_io_tqg_6] 100034 D - 0xfffff80003915c00 [if_io_tqg_7] 100035 D - 0xfffff80003915b00 [if_io_tqg_8] 100036 D - 0xfffff80003915a00 [if_io_tqg_9] 100037 D - 0xfffff80003915900 [if_io_tqg_10] 100038 D - 0xfffff80003915800 [if_io_tqg_11] 100039 D - 0xfffff80003915700 [if_config_tqg_0] 100040 D - 0xfffff80003920100 [kqueue_ctx taskq] 100043 D - 0xfffff80003921d00 [thread taskq] 100044 D - 0xfffff80003921c00 [inm_free taskq] 100046 D - 0xfffff80003921a00 [aiod_kick taskq] 100047 D - 0xfffff80003921900 [in6m_free taskq] 100062 D - 0xfffff80003921800 [firmware taskq] 100067 D - 0xfffff80003921700 [crypto_0] 100068 D - 0xfffff80003921700 [crypto_1] 100069 D - 0xfffff80003921700 [crypto_2] 100070 D - 0xfffff80003921700 [crypto_3] 100071 D - 0xfffff80003921700 [crypto_4] 100072 D - 0xfffff80003921700 [crypto_5] 100073 D - 0xfffff80003921700 [crypto_6] 100074 D - 0xfffff80003921700 [crypto_7] 100075 D - 0xfffff80003921700 [crypto_8] 100076 D - 0xfffff80003921700 [crypto_9] 100077 D - 0xfffff80003921700 [crypto_10] 100078 D - 0xfffff80003921700 [crypto_11] 100116 D - 0xfffff800390ae100 [mca taskq] 100121 D - 0xfffff80045240100 [acpi_task_0] 100122 D - 0xfffff80045240100 [acpi_task_1] 100123 D - 0xfffff80045240100 [acpi_task_2] 100125 D - 0xfffff80003921500 [CAM taskq] db:0:pho> allt Tracing command unix_socket_detach pid 70162 tid 101705 td 0xfffffe00f1111c00 (CPU 11) kdb_enter() at kdb_enter+0x37/frame 0xfffffe00f0ce87a0 vpanic() at vpanic+0x19e/frame 0xfffffe00f0ce87f0 panic() at panic+0x43/frame 0xfffffe00f0ce8850 unp_pcb_lock_peer() at unp_pcb_lock_peer+0xd6/frame 0xfffffe00f0ce88c0 unp_drop() at unp_drop+0x63/frame 0xfffffe00f0ce8900 uipc_abort() at uipc_abort+0xf7/frame 0xfffffe00f0ce8950 soabort() at soabort+0x4c/frame 0xfffffe00f0ce8970 soclose() at soclose+0x3e3/frame 0xfffffe00f0ce89e0 _fdrop() at _fdrop+0x1a/frame 0xfffffe00f0ce8a00 closef() at closef+0x1ea/frame 0xfffffe00f0ce8a90 closefp() at closefp+0x96/frame 0xfffffe00f0ce8ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f0ce8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f0ce8bf0 --- syscall (6, FreeBSD ELF64, sys_close), rip = 0x8004290fa, rsp = 0x7fffffffe618, rbp = 0x7fffffffe6b0 --- Tracing command unix_socket_detach pid 70161 tid 104251 td 0xfffffe00eddb7300 (CPU 2) cpustop_handler() at cpustop_handler+0x26/frame 0xfffffe003dbdce00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003dbdce10 trap() at trap+0x56/frame 0xfffffe003dbdcf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003dbdcf20 --- trap 0x13, rip = 0xffffffff80c238b0, rsp = 0xfffffe00f5e4a7d0, rbp = 0xfffffe00f5e4a7e0 --- lock_delay() at lock_delay+0x30/frame 0xfffffe00f5e4a7e0 __mtx_lock_sleep() at __mtx_lock_sleep+0xe8/frame 0xfffffe00f5e4a870 __mtx_lock_flags() at __mtx_lock_flags+0xe5/frame 0xfffffe00f5e4a8c0 unp_pcb_lock_peer() at unp_pcb_lock_peer+0x237/frame 0xfffffe00f5e4a930 uipc_disconnect() at uipc_disconnect+0x7f/frame 0xfffffe00f5e4a970 soclose() at soclose+0x9b/frame 0xfffffe00f5e4a9e0 _fdrop() at _fdrop+0x1a/frame 0xfffffe00f5e4aa00 closef() at closef+0x1ea/frame 0xfffffe00f5e4aa90 closefp() at closefp+0x96/frame 0xfffffe00f5e4aad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f5e4abf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f5e4abf0 --- syscall (6, FreeBSD ELF64, sys_close), rip = 0x8004290fa, rsp = 0x7fffffffe618, rbp = 0x7fffffffe6b0 --- Tracing command sh pid 70150 tid 104056 td 0xfffffe00ee7d7800 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00f5a7b6e0 mi_switch() at mi_switch+0x155/frame 0xfffffe00f5a7b700 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00f5a7b740 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00f5a7b7a0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00f5a7b7b0 _sleep() at _sleep+0x2c3/frame 0xfffffe00f5a7b850 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00f5a7b8e0 sys_wait4() at sys_wait4+0x7b/frame 0xfffffe00f5a7bad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f5a7bbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f5a7bbf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80041a25a, rsp = 0x7fffffffe308, rbp = 0x7fffffffe3c0 --- Tracing command tee pid 70149 tid 102205 td 0xfffffe00f1f25e00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00f3654840 mi_switch() at mi_switch+0x155/frame 0xfffffe00f3654860 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00f36548a0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00f3654900 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00f3654910 _sleep() at _sleep+0x2c3/frame 0xfffffe00f36549b0 pipe_read() at pipe_read+0x518/frame 0xfffffe00f3654a10 dofileread() at dofileread+0x81/frame 0xfffffe00f3654a60 sys_read() at sys_read+0xbc/frame 0xfffffe00f3654ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f3654bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f3654bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004290da, rsp = 0x7fffffffe658, rbp = 0x7fffffffe6b0 --- Tracing command sh pid 70148 tid 102724 td 0xfffffe00f0095800 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00f40776e0 mi_switch() at mi_switch+0x155/frame 0xfffffe00f4077700 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00f4077740 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00f40777a0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00f40777b0 _sleep() at _sleep+0x2c3/frame 0xfffffe00f4077850 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00f40778e0 sys_wait4() at sys_wait4+0x7b/frame 0xfffffe00f4077ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f4077bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f4077bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80041a25a, rsp = 0x7fffffffdda8, rbp = 0x7fffffffde60 --- Tracing command sleep pid 69845 tid 103806 td 0xfffffe00ee529700 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00f5599870 mi_switch() at mi_switch+0x155/frame 0xfffffe00f5599890 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00f55998d0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00f5599930 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe00f5599970 _sleep() at _sleep+0x258/frame 0xfffffe00f5599a10 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe00f5599a90 sys_nanosleep() at sys_nanosleep+0x3b/frame 0xfffffe00f5599ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f5599bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f5599bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003ba3ba, rsp = 0x7fffffffe658, rbp = 0x7fffffffe6a0 --- Tracing command aiod4 pid 43502 tid 100288 td 0xfffffe00f0103e00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00f0362a20 mi_switch() at mi_switch+0x155/frame 0xfffffe00f0362a40 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00f0362a80 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00f0362ac0 _sleep() at _sleep+0x26c/frame 0xfffffe00f0362b60 aio_daemon() at aio_daemon+0x2fd/frame 0xfffffe00f0362bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f0362bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f0362bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command aiod3 pid 43501 tid 100647 td 0xfffffe00f0964300 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00f0c3ea20 mi_switch() at mi_switch+0x155/frame 0xfffffe00f0c3ea40 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00f0c3ea80 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00f0c3eac0 _sleep() at _sleep+0x26c/frame 0xfffffe00f0c3eb60 aio_daemon() at aio_daemon+0x2fd/frame 0xfffffe00f0c3ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f0c3ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f0c3ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command aiod2 pid 43500 tid 100868 td 0xfffffe00f10ae300 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00f1290a20 mi_switch() at mi_switch+0x155/frame 0xfffffe00f1290a40 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00f1290a80 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00f1290ac0 _sleep() at _sleep+0x26c/frame 0xfffffe00f1290b60 aio_daemon() at aio_daemon+0x2fd/frame 0xfffffe00f1290bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f1290bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f1290bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command aiod1 pid 43499 tid 100577 td 0xfffffe00f08b9800 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00f0ae0a20 mi_switch() at mi_switch+0x155/frame 0xfffffe00f0ae0a40 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00f0ae0a80 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00f0ae0ac0 _sleep() at _sleep+0x26c/frame 0xfffffe00f0ae0b60 aio_daemon() at aio_daemon+0x2fd/frame 0xfffffe00f0ae0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f0ae0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f0ae0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sctp_iterator pid 52305 tid 100366 td 0xfffffe00f018f800 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00f04c0a90 mi_switch() at mi_switch+0x155/frame 0xfffffe00f04c0ab0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00f04c0af0 _sleep() at _sleep+0x2cd/frame 0xfffffe00f04c0b90 sctp_iterator_thread() at sctp_iterator_thread+0x59/frame 0xfffffe00f04c0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f04c0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f04c0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sh pid 2731 tid 100170 td 0xfffffe00dff04000 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e64fa840 mi_switch() at mi_switch+0x155/frame 0xfffffe00e64fa860 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e64fa8a0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e64fa900 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e64fa910 _sleep() at _sleep+0x2c3/frame 0xfffffe00e64fa9b0 pipe_read() at pipe_read+0x518/frame 0xfffffe00e64faa10 dofileread() at dofileread+0x81/frame 0xfffffe00e64faa60 sys_read() at sys_read+0xbc/frame 0xfffffe00e64faad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00e64fabf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00e64fabf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004890da, rsp = 0x7fffffffddb8, rbp = 0x7fffffffe2e0 --- Tracing command awk pid 2729 tid 100191 td 0xfffffe00dffd4a00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e6593840 mi_switch() at mi_switch+0x155/frame 0xfffffe00e6593860 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e65938a0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e6593900 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e6593910 _sleep() at _sleep+0x2c3/frame 0xfffffe00e65939b0 pipe_read() at pipe_read+0x518/frame 0xfffffe00e6593a10 dofileread() at dofileread+0x81/frame 0xfffffe00e6593a60 sys_read() at sys_read+0xbc/frame 0xfffffe00e6593ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00e6593bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00e6593bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80048d0da, rsp = 0x7fffffffe2d8, rbp = 0x7fffffffe2f0 --- Tracing command sh pid 2728 tid 100151 td 0xfffffe00dffbee00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e648f6e0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e648f700 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e648f740 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e648f7a0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e648f7b0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e648f850 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00e648f8e0 sys_wait4() at sys_wait4+0x7b/frame 0xfffffe00e648fad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00e648fbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00e648fbf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80041a25a, rsp = 0x7fffffffe1e8, rbp = 0x7fffffffe2a0 --- Tracing command sh pid 2678 tid 100220 td 0xfffffe00e82dee00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00f02636e0 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00f0ce7e50 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00f0ce7e60 trap() at trap+0x6c/frame 0xfffffe00f0ce7f70 calltrap() at calltrap+0x8/frame 0xfffffe00f0ce7f70 --- trap 0xc, rip = 0xffffffff80ff4c30, rsp = 0xfffffe00f0ce8040, rbp = 0xfffffe00f0ce80d0 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe00f0ce80d0 db_get_value() at db_get_value+0x32/frame 0xfffffe00f0ce8110 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00f0ce81a0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00f0ce81c0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00f0ce8260 db_command() at db_command+0x25c/frame 0xfffffe00f0ce8330 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00f0ce83d0 db_command() at db_command+0x25c/frame 0xfffffe00f0ce84a0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00f0ce84b0 db_trap() at db_trap+0xe8/frame 0xfffffe00f0ce8540 kdb_trap() at kdb_trap+0x154/frame 0xfffffe00f0ce85b0 trap() at trap+0x7b8/frame 0xfffffe00f0ce86c0 calltrap() at calltrap+0x8/frame 0xfffffe00f0ce86c0 --- trap 0x3, rip = 0xffffffff80c222e7, rsp = 0xfffffe00f0ce8790, rbp = 0xfffffe00f0ce87a0 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe00f0ce87a0 vpanic() at vpanic+0x19e/frame 0xfffffe00f0ce87f0 panic() at panic+0x43/frame 0xfffffe00f0ce8850 unp_pcb_lock_peer() at unp_pcb_lock_peer+0xd6/frame 0xfffffe00f0ce88c0 unp_drop() at unp_drop+0x63/frame 0xfffffe00f0ce8900 uipc_abort() at uipc_abort+0xf7/frame 0xfffffe00f0ce8950 soabort() at soabort+0x4c/frame 0xfffffe00f0ce8970 soclose() at soclose+0x3e3/frame 0xfffffe00f0ce89e0 _fdrop() at _fdrop+0x1a/frame 0xfffffe00f0ce8a00 closef() at closef+0x1ea/frame 0xfffffe00f0ce8a90 closefp() at closefp+0x96/frame 0xfffffe00f0ce8ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f0ce8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f0ce8bf0 --- syscall (6, FreeBSD ELF64, sys_close), rip = 0x8004290fa, rsp = 0x7fffffffe618, rbp = 0x7fffffffe6b0 --- *** error reading from address fffffe00f02636e8 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00f0ce80c0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00f0ce80d0 db_get_value() at db_get_value+0x4c/frame 0xfffffe00f0ce8110 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00f0ce81a0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00f0ce81c0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00f0ce8260 db_command() at db_command+0x25c/frame 0xfffffe00f0ce8330 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00f0ce83d0 db_command() at db_command+0x25c/frame 0xfffffe00f0ce84a0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00f0ce84b0 db_trap() at db_trap+0xe8/frame 0xfffffe00f0ce8540 kdb_trap() at kdb_trap+0x154/frame 0xfffffe00f0ce85b0 trap() at trap+0x7b8/frame 0xfffffe00f0ce86c0 calltrap() at calltrap+0x8/frame 0xfffffe00f0ce86c0 --- trap 0x3, rip = 0xffffffff80c222e7, rsp = 0xfffffe00f0ce8790, rbp = 0xfffffe00f0ce87a0 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe00f0ce87a0 vpanic() at vpanic+0x19e/frame 0xfffffe00f0ce87f0 panic() at panic+0x43/frame 0xfffffe00f0ce8850 unp_pcb_lock_peer() at unp_pcb_lock_peer+0xd6/frame 0xfffffe00f0ce88c0 unp_drop() at unp_drop+0x63/frame 0xfffffe00f0ce8900 uipc_abort() at uipc_abort+0xf7/frame 0xfffffe00f0ce8950 soabort() at soabort+0x4c/frame 0xfffffe00f0ce8970 soclose() at soclose+0x3e3/frame 0xfffffe00f0ce89e0 _fdrop() at _fdrop+0x1a/frame 0xfffffe00f0ce8a00 closef() at closef+0x1ea/frame 0xfffffe00f0ce8a90 closefp() at closefp+0x96/frame 0xfffffe00f0ce8ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f0ce8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f0ce8bf0 --- syscall (6, FreeBSD ELF64, sys_close), rip = 0x8004290fa, rsp = 0x7fffffffe618, rbp = 0x7fffffffe6b0 --- Tracing command sh pid 2445 tid 100167 td 0xfffffe00dfef8500 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e64eb6e0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e64eb700 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e64eb740 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e64eb7a0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e64eb7b0 _sleep() at _sleep+0x2c3/frame 0xfffffe00e64eb850 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00e64eb8e0 sys_wait4() at sys_wait4+0x7b/frame 0xfffffe00e64ebad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00e64ebbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00e64ebbf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80041a25a, rsp = 0x7fffffffe0c8, rbp = 0x7fffffffe180 --- Tracing command sh pid 2443 tid 100144 td 0xfffffe00d69d8a00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e646c6e0 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00f0ce7e50 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00f0ce7e60 trap() at trap+0x6c/frame 0xfffffe00f0ce7f70 calltrap() at calltrap+0x8/frame 0xfffffe00f0ce7f70 --- trap 0xc, rip = 0xffffffff80ff4c30, rsp = 0xfffffe00f0ce8040, rbp = 0xfffffe00f0ce80d0 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe00f0ce80d0 db_get_value() at db_get_value+0x32/frame 0xfffffe00f0ce8110 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00f0ce81a0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00f0ce81c0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00f0ce8260 db_command() at db_command+0x25c/frame 0xfffffe00f0ce8330 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00f0ce83d0 db_command() at db_command+0x25c/frame 0xfffffe00f0ce84a0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00f0ce84b0 db_trap() at db_trap+0xe8/frame 0xfffffe00f0ce8540 kdb_trap() at kdb_trap+0x154/frame 0xfffffe00f0ce85b0 trap() at trap+0x7b8/frame 0xfffffe00f0ce86c0 calltrap() at calltrap+0x8/frame 0xfffffe00f0ce86c0 --- trap 0x3, rip = 0xffffffff80c222e7, rsp = 0xfffffe00f0ce8790, rbp = 0xfffffe00f0ce87a0 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe00f0ce87a0 vpanic() at vpanic+0x19e/frame 0xfffffe00f0ce87f0 panic() at panic+0x43/frame 0xfffffe00f0ce8850 unp_pcb_lock_peer() at unp_pcb_lock_peer+0xd6/frame 0xfffffe00f0ce88c0 unp_drop() at unp_drop+0x63/frame 0xfffffe00f0ce8900 uipc_abort() at uipc_abort+0xf7/frame 0xfffffe00f0ce8950 soabort() at soabort+0x4c/frame 0xfffffe00f0ce8970 soclose() at soclose+0x3e3/frame 0xfffffe00f0ce89e0 _fdrop() at _fdrop+0x1a/frame 0xfffffe00f0ce8a00 closef() at closef+0x1ea/frame 0xfffffe00f0ce8a90 closefp() at closefp+0x96/frame 0xfffffe00f0ce8ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f0ce8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f0ce8bf0 --- syscall (6, FreeBSD ELF64, sys_close), rip = 0x8004290fa, rsp = 0x7fffffffe618, rbp = 0x7fffffffe6b0 --- *** error reading from address fffffe00e646c6e8 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00f0ce80c0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00f0ce80d0 db_get_value() at db_get_value+0x4c/frame 0xfffffe00f0ce8110 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00f0ce81a0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00f0ce81c0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00f0ce8260 db_command() at db_command+0x25c/frame 0xfffffe00f0ce8330 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00f0ce83d0 db_command() at db_command+0x25c/frame 0xfffffe00f0ce84a0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00f0ce84b0 db_trap() at db_trap+0xe8/frame 0xfffffe00f0ce8540 kdb_trap() at kdb_trap+0x154/frame 0xfffffe00f0ce85b0 trap() at trap+0x7b8/frame 0xfffffe00f0ce86c0 calltrap() at calltrap+0x8/frame 0xfffffe00f0ce86c0 --- trap 0x3, rip = 0xffffffff80c222e7, rsp = 0xfffffe00f0ce8790, rbp = 0xfffffe00f0ce87a0 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe00f0ce87a0 vpanic() at vpanic+0x19e/frame 0xfffffe00f0ce87f0 panic() at panic+0x43/frame 0xfffffe00f0ce8850 unp_pcb_lock_peer() at unp_pcb_lock_peer+0xd6/frame 0xfffffe00f0ce88c0 unp_drop() at unp_drop+0x63/frame 0xfffffe00f0ce8900 uipc_abort() at uipc_abort+0xf7/frame 0xfffffe00f0ce8950 soabort() at soabort+0x4c/frame 0xfffffe00f0ce8970 soclose() at soclose+0x3e3/frame 0xfffffe00f0ce89e0 _fdrop() at _fdrop+0x1a/frame 0xfffffe00f0ce8a00 closef() at closef+0x1ea/frame 0xfffffe00f0ce8a90 closefp() at closefp+0x96/frame 0xfffffe00f0ce8ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f0ce8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f0ce8bf0 --- syscall (6, FreeBSD ELF64, sys_close), rip = 0x8004290fa, rsp = 0x7fffffffe618, rbp = 0x7fffffffe6b0 --- Tracing command bash pid 2442 tid 100177 td 0xfffffe00dc203100 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e651d6e0 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00f0ce7e50 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00f0ce7e60 trap() at trap+0x6c/frame 0xfffffe00f0ce7f70 calltrap() at calltrap+0x8/frame 0xfffffe00f0ce7f70 --- trap 0xc, rip = 0xffffffff80ff4c30, rsp = 0xfffffe00f0ce8040, rbp = 0xfffffe00f0ce80d0 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe00f0ce80d0 db_get_value() at db_get_value+0x32/frame 0xfffffe00f0ce8110 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00f0ce81a0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00f0ce81c0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00f0ce8260 db_command() at db_command+0x25c/frame 0xfffffe00f0ce8330 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00f0ce83d0 db_command() at db_command+0x25c/frame 0xfffffe00f0ce84a0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00f0ce84b0 db_trap() at db_trap+0xe8/frame 0xfffffe00f0ce8540 kdb_trap() at kdb_trap+0x154/frame 0xfffffe00f0ce85b0 trap() at trap+0x7b8/frame 0xfffffe00f0ce86c0 calltrap() at calltrap+0x8/frame 0xfffffe00f0ce86c0 --- trap 0x3, rip = 0xffffffff80c222e7, rsp = 0xfffffe00f0ce8790, rbp = 0xfffffe00f0ce87a0 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe00f0ce87a0 vpanic() at vpanic+0x19e/frame 0xfffffe00f0ce87f0 panic() at panic+0x43/frame 0xfffffe00f0ce8850 unp_pcb_lock_peer() at unp_pcb_lock_peer+0xd6/frame 0xfffffe00f0ce88c0 unp_drop() at unp_drop+0x63/frame 0xfffffe00f0ce8900 uipc_abort() at uipc_abort+0xf7/frame 0xfffffe00f0ce8950 soabort() at soabort+0x4c/frame 0xfffffe00f0ce8970 soclose() at soclose+0x3e3/frame 0xfffffe00f0ce89e0 _fdrop() at _fdrop+0x1a/frame 0xfffffe00f0ce8a00 closef() at closef+0x1ea/frame 0xfffffe00f0ce8a90 closefp() at closefp+0x96/frame 0xfffffe00f0ce8ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f0ce8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f0ce8bf0 --- syscall (6, FreeBSD ELF64, sys_close), rip = 0x8004290fa, rsp = 0x7fffffffe618, rbp = 0x7fffffffe6b0 --- *** error reading from address fffffe00e651d6e8 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00f0ce80c0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00f0ce80d0 db_get_value() at db_get_value+0x4c/frame 0xfffffe00f0ce8110 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00f0ce81a0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00f0ce81c0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00f0ce8260 db_command() at db_command+0x25c/frame 0xfffffe00f0ce8330 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00f0ce83d0 db_command() at db_command+0x25c/frame 0xfffffe00f0ce84a0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00f0ce84b0 db_trap() at db_trap+0xe8/frame 0xfffffe00f0ce8540 kdb_trap() at kdb_trap+0x154/frame 0xfffffe00f0ce85b0 trap() at trap+0x7b8/frame 0xfffffe00f0ce86c0 calltrap() at calltrap+0x8/frame 0xfffffe00f0ce86c0 --- trap 0x3, rip = 0xffffffff80c222e7, rsp = 0xfffffe00f0ce8790, rbp = 0xfffffe00f0ce87a0 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe00f0ce87a0 vpanic() at vpanic+0x19e/frame 0xfffffe00f0ce87f0 panic() at panic+0x43/frame 0xfffffe00f0ce8850 unp_pcb_lock_peer() at unp_pcb_lock_peer+0xd6/frame 0xfffffe00f0ce88c0 unp_drop() at unp_drop+0x63/frame 0xfffffe00f0ce8900 uipc_abort() at uipc_abort+0xf7/frame 0xfffffe00f0ce8950 soabort() at soabort+0x4c/frame 0xfffffe00f0ce8970 soclose() at soclose+0x3e3/frame 0xfffffe00f0ce89e0 _fdrop() at _fdrop+0x1a/frame 0xfffffe00f0ce8a00 closef() at closef+0x1ea/frame 0xfffffe00f0ce8a90 closefp() at closefp+0x96/frame 0xfffffe00f0ce8ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f0ce8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f0ce8bf0 --- syscall (6, FreeBSD ELF64, sys_close), rip = 0x8004290fa, rsp = 0x7fffffffe618, rbp = 0x7fffffffe6b0 --- Tracing command csh pid 2440 tid 100160 td 0xfffffe00dfffc100 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e64c88d0 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00f0ce7e50 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00f0ce7e60 trap() at trap+0x6c/frame 0xfffffe00f0ce7f70 calltrap() at calltrap+0x8/frame 0xfffffe00f0ce7f70 --- trap 0xc, rip = 0xffffffff80ff4c30, rsp = 0xfffffe00f0ce8040, rbp = 0xfffffe00f0ce80d0 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe00f0ce80d0 db_get_value() at db_get_value+0x32/frame 0xfffffe00f0ce8110 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00f0ce81a0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00f0ce81c0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00f0ce8260 db_command() at db_command+0x25c/frame 0xfffffe00f0ce8330 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00f0ce83d0 db_command() at db_command+0x25c/frame 0xfffffe00f0ce84a0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00f0ce84b0 db_trap() at db_trap+0xe8/frame 0xfffffe00f0ce8540 kdb_trap() at kdb_trap+0x154/frame 0xfffffe00f0ce85b0 trap() at trap+0x7b8/frame 0xfffffe00f0ce86c0 calltrap() at calltrap+0x8/frame 0xfffffe00f0ce86c0 --- trap 0x3, rip = 0xffffffff80c222e7, rsp = 0xfffffe00f0ce8790, rbp = 0xfffffe00f0ce87a0 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe00f0ce87a0 vpanic() at vpanic+0x19e/frame 0xfffffe00f0ce87f0 panic() at panic+0x43/frame 0xfffffe00f0ce8850 unp_pcb_lock_peer() at unp_pcb_lock_peer+0xd6/frame 0xfffffe00f0ce88c0 unp_drop() at unp_drop+0x63/frame 0xfffffe00f0ce8900 uipc_abort() at uipc_abort+0xf7/frame 0xfffffe00f0ce8950 soabort() at soabort+0x4c/frame 0xfffffe00f0ce8970 soclose() at soclose+0x3e3/frame 0xfffffe00f0ce89e0 _fdrop() at _fdrop+0x1a/frame 0xfffffe00f0ce8a00 closef() at closef+0x1ea/frame 0xfffffe00f0ce8a90 closefp() at closefp+0x96/frame 0xfffffe00f0ce8ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f0ce8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f0ce8bf0 --- syscall (6, FreeBSD ELF64, sys_close), rip = 0x8004290fa, rsp = 0x7fffffffe618, rbp = 0x7fffffffe6b0 --- *** error reading from address fffffe00e64c88d8 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00f0ce80c0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00f0ce80d0 db_get_value() at db_get_value+0x4c/frame 0xfffffe00f0ce8110 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00f0ce81a0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00f0ce81c0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00f0ce8260 db_command() at db_command+0x25c/frame 0xfffffe00f0ce8330 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00f0ce83d0 db_command() at db_command+0x25c/frame 0xfffffe00f0ce84a0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00f0ce84b0 db_trap() at db_trap+0xe8/frame 0xfffffe00f0ce8540 kdb_trap() at kdb_trap+0x154/frame 0xfffffe00f0ce85b0 trap() at trap+0x7b8/frame 0xfffffe00f0ce86c0 calltrap() at calltrap+0x8/frame 0xfffffe00f0ce86c0 --- trap 0x3, rip = 0xffffffff80c222e7, rsp = 0xfffffe00f0ce8790, rbp = 0xfffffe00f0ce87a0 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe00f0ce87a0 vpanic() at vpanic+0x19e/frame 0xfffffe00f0ce87f0 panic() at panic+0x43/frame 0xfffffe00f0ce8850 unp_pcb_lock_peer() at unp_pcb_lock_peer+0xd6/frame 0xfffffe00f0ce88c0 unp_drop() at unp_drop+0x63/frame 0xfffffe00f0ce8900 uipc_abort() at uipc_abort+0xf7/frame 0xfffffe00f0ce8950 soabort() at soabort+0x4c/frame 0xfffffe00f0ce8970 soclose() at soclose+0x3e3/frame 0xfffffe00f0ce89e0 _fdrop() at _fdrop+0x1a/frame 0xfffffe00f0ce8a00 closef() at closef+0x1ea/frame 0xfffffe00f0ce8a90 closefp() at closefp+0x96/frame 0xfffffe00f0ce8ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f0ce8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f0ce8bf0 --- syscall (6, FreeBSD ELF64, sys_close), rip = 0x8004290fa, rsp = 0x7fffffffe618, rbp = 0x7fffffffe6b0 --- Tracing command su pid 2439 tid 100190 td 0xfffffe00dffe3c00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e658e6e0 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00f0ce7e50 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00f0ce7e60 trap() at trap+0x6c/frame 0xfffffe00f0ce7f70 calltrap() at calltrap+0x8/frame 0xfffffe00f0ce7f70 --- trap 0xc, rip = 0xffffffff80ff4c30, rsp = 0xfffffe00f0ce8040, rbp = 0xfffffe00f0ce80d0 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe00f0ce80d0 db_get_value() at db_get_value+0x32/frame 0xfffffe00f0ce8110 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00f0ce81a0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00f0ce81c0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00f0ce8260 db_command() at db_command+0x25c/frame 0xfffffe00f0ce8330 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00f0ce83d0 db_command() at db_command+0x25c/frame 0xfffffe00f0ce84a0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00f0ce84b0 db_trap() at db_trap+0xe8/frame 0xfffffe00f0ce8540 kdb_trap() at kdb_trap+0x154/frame 0xfffffe00f0ce85b0 trap() at trap+0x7b8/frame 0xfffffe00f0ce86c0 calltrap() at calltrap+0x8/frame 0xfffffe00f0ce86c0 --- trap 0x3, rip = 0xffffffff80c222e7, rsp = 0xfffffe00f0ce8790, rbp = 0xfffffe00f0ce87a0 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe00f0ce87a0 vpanic() at vpanic+0x19e/frame 0xfffffe00f0ce87f0 panic() at panic+0x43/frame 0xfffffe00f0ce8850 unp_pcb_lock_peer() at unp_pcb_lock_peer+0xd6/frame 0xfffffe00f0ce88c0 unp_drop() at unp_drop+0x63/frame 0xfffffe00f0ce8900 uipc_abort() at uipc_abort+0xf7/frame 0xfffffe00f0ce8950 soabort() at soabort+0x4c/frame 0xfffffe00f0ce8970 soclose() at soclose+0x3e3/frame 0xfffffe00f0ce89e0 _fdrop() at _fdrop+0x1a/frame 0xfffffe00f0ce8a00 closef() at closef+0x1ea/frame 0xfffffe00f0ce8a90 closefp() at closefp+0x96/frame 0xfffffe00f0ce8ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f0ce8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f0ce8bf0 --- syscall (6, FreeBSD ELF64, sys_close), rip = 0x8004290fa, rsp = 0x7fffffffe618, rbp = 0x7fffffffe6b0 --- *** error reading from address fffffe00e658e6e8 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00f0ce80c0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00f0ce80d0 db_get_value() at db_get_value+0x4c/frame 0xfffffe00f0ce8110 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00f0ce81a0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00f0ce81c0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00f0ce8260 db_command() at db_command+0x25c/frame 0xfffffe00f0ce8330 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00f0ce83d0 db_command() at db_command+0x25c/frame 0xfffffe00f0ce84a0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00f0ce84b0 db_trap() at db_trap+0xe8/frame 0xfffffe00f0ce8540 kdb_trap() at kdb_trap+0x154/frame 0xfffffe00f0ce85b0 trap() at trap+0x7b8/frame 0xfffffe00f0ce86c0 calltrap() at calltrap+0x8/frame 0xfffffe00f0ce86c0 --- trap 0x3, rip = 0xffffffff80c222e7, rsp = 0xfffffe00f0ce8790, rbp = 0xfffffe00f0ce87a0 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe00f0ce87a0 vpanic() at vpanic+0x19e/frame 0xfffffe00f0ce87f0 panic() at panic+0x43/frame 0xfffffe00f0ce8850 unp_pcb_lock_peer() at unp_pcb_lock_peer+0xd6/frame 0xfffffe00f0ce88c0 unp_drop() at unp_drop+0x63/frame 0xfffffe00f0ce8900 uipc_abort() at uipc_abort+0xf7/frame 0xfffffe00f0ce8950 soabort() at soabort+0x4c/frame 0xfffffe00f0ce8970 soclose() at soclose+0x3e3/frame 0xfffffe00f0ce89e0 _fdrop() at _fdrop+0x1a/frame 0xfffffe00f0ce8a00 closef() at closef+0x1ea/frame 0xfffffe00f0ce8a90 closefp() at closefp+0x96/frame 0xfffffe00f0ce8ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f0ce8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f0ce8bf0 --- syscall (6, FreeBSD ELF64, sys_close), rip = 0x8004290fa, rsp = 0x7fffffffe618, rbp = 0x7fffffffe6b0 --- Tracing command bash pid 2431 tid 100217 td 0xfffffe00e8367500 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00f02546e0 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00f0ce7e50 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00f0ce7e60 trap() at trap+0x6c/frame 0xfffffe00f0ce7f70 calltrap() at calltrap+0x8/frame 0xfffffe00f0ce7f70 --- trap 0xc, rip = 0xffffffff80ff4c30, rsp = 0xfffffe00f0ce8040, rbp = 0xfffffe00f0ce80d0 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe00f0ce80d0 db_get_value() at db_get_value+0x32/frame 0xfffffe00f0ce8110 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00f0ce81a0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00f0ce81c0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00f0ce8260 db_command() at db_command+0x25c/frame 0xfffffe00f0ce8330 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00f0ce83d0 db_command() at db_command+0x25c/frame 0xfffffe00f0ce84a0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00f0ce84b0 db_trap() at db_trap+0xe8/frame 0xfffffe00f0ce8540 kdb_trap() at kdb_trap+0x154/frame 0xfffffe00f0ce85b0 trap() at trap+0x7b8/frame 0xfffffe00f0ce86c0 calltrap() at calltrap+0x8/frame 0xfffffe00f0ce86c0 --- trap 0x3, rip = 0xffffffff80c222e7, rsp = 0xfffffe00f0ce8790, rbp = 0xfffffe00f0ce87a0 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe00f0ce87a0 vpanic() at vpanic+0x19e/frame 0xfffffe00f0ce87f0 panic() at panic+0x43/frame 0xfffffe00f0ce8850 unp_pcb_lock_peer() at unp_pcb_lock_peer+0xd6/frame 0xfffffe00f0ce88c0 unp_drop() at unp_drop+0x63/frame 0xfffffe00f0ce8900 uipc_abort() at uipc_abort+0xf7/frame 0xfffffe00f0ce8950 soabort() at soabort+0x4c/frame 0xfffffe00f0ce8970 soclose() at soclose+0x3e3/frame 0xfffffe00f0ce89e0 _fdrop() at _fdrop+0x1a/frame 0xfffffe00f0ce8a00 closef() at closef+0x1ea/frame 0xfffffe00f0ce8a90 closefp() at closefp+0x96/frame 0xfffffe00f0ce8ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f0ce8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f0ce8bf0 --- syscall (6, FreeBSD ELF64, sys_close), rip = 0x8004290fa, rsp = 0x7fffffffe618, rbp = 0x7fffffffe6b0 --- *** error reading from address fffffe00f02546e8 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00f0ce80c0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00f0ce80d0 db_get_value() at db_get_value+0x4c/frame 0xfffffe00f0ce8110 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00f0ce81a0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00f0ce81c0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00f0ce8260 db_command() at db_command+0x25c/frame 0xfffffe00f0ce8330 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00f0ce83d0 db_command() at db_command+0x25c/frame 0xfffffe00f0ce84a0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00f0ce84b0 db_trap() at db_trap+0xe8/frame 0xfffffe00f0ce8540 kdb_trap() at kdb_trap+0x154/frame 0xfffffe00f0ce85b0 trap() at trap+0x7b8/frame 0xfffffe00f0ce86c0 calltrap() at calltrap+0x8/frame 0xfffffe00f0ce86c0 --- trap 0x3, rip = 0xffffffff80c222e7, rsp = 0xfffffe00f0ce8790, rbp = 0xfffffe00f0ce87a0 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe00f0ce87a0 vpanic() at vpanic+0x19e/frame 0xfffffe00f0ce87f0 panic() at panic+0x43/frame 0xfffffe00f0ce8850 unp_pcb_lock_peer() at unp_pcb_lock_peer+0xd6/frame 0xfffffe00f0ce88c0 unp_drop() at unp_drop+0x63/frame 0xfffffe00f0ce8900 uipc_abort() at uipc_abort+0xf7/frame 0xfffffe00f0ce8950 soabort() at soabort+0x4c/frame 0xfffffe00f0ce8970 soclose() at soclose+0x3e3/frame 0xfffffe00f0ce89e0 _fdrop() at _fdrop+0x1a/frame 0xfffffe00f0ce8a00 closef() at closef+0x1ea/frame 0xfffffe00f0ce8a90 closefp() at closefp+0x96/frame 0xfffffe00f0ce8ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f0ce8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f0ce8bf0 --- syscall (6, FreeBSD ELF64, sys_close), rip = 0x8004290fa, rsp = 0x7fffffffe618, rbp = 0x7fffffffe6b0 --- Tracing command sshd pid 2430 tid 100162 td 0xfffffe00dffd8000 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e64d2740 mi_switch() at mi_switch+0x155/frame 0xfffffe00e64d2760 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e64d27a0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e64d2800 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e64d2810 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e64d2870 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e64d28b0 kern_select() at kern_select+0x8f0/frame 0xfffffe00e64d2a90 sys_select() at sys_select+0x56/frame 0xfffffe00e64d2ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00e64d2bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00e64d2bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8008f10ba, rsp = 0x7fffffffa4f8, rbp = 0x7fffffffa530 --- Tracing command sshd pid 2425 tid 100202 td 0xfffffe00dfefe100 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00f02097b0 mi_switch() at mi_switch+0x155/frame 0xfffffe00f02097d0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00f0209810 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00f0209870 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00f0209880 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00f02098e0 seltdwait() at seltdwait+0x8d/frame 0xfffffe00f0209920 kern_poll() at kern_poll+0x4b7/frame 0xfffffe00f0209ab0 sys_poll() at sys_poll+0x50/frame 0xfffffe00f0209ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f0209bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f0209bf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x80089481a, rsp = 0x7fffffffe558, rbp = 0x7fffffffe590 --- Tracing command csh pid 2421 tid 100205 td 0xfffffe00e82dfc00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00f02187c0 mi_switch() at mi_switch+0x155/frame 0xfffffe00f02187e0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00f0218820 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00f0218880 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00f0218890 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00f02188f0 tty_wait() at tty_wait+0x42/frame 0xfffffe00f0218920 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00f0218980 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00f02189b0 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00f0218a10 dofileread() at dofileread+0x81/frame 0xfffffe00f0218a60 sys_read() at sys_read+0xbc/frame 0xfffffe00f0218ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f0218bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f0218bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80051f0da, rsp = 0x7fffffffe198, rbp = 0x7fffffffe1c0 --- Tracing command login pid 2420 tid 100215 td 0xfffffe00eff54700 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00f024a6e0 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00f0ce7e50 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00f0ce7e60 trap() at trap+0x6c/frame 0xfffffe00f0ce7f70 calltrap() at calltrap+0x8/frame 0xfffffe00f0ce7f70 --- trap 0xc, rip = 0xffffffff80ff4c30, rsp = 0xfffffe00f0ce8040, rbp = 0xfffffe00f0ce80d0 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe00f0ce80d0 db_get_value() at db_get_value+0x32/frame 0xfffffe00f0ce8110 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00f0ce81a0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00f0ce81c0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00f0ce8260 db_command() at db_command+0x25c/frame 0xfffffe00f0ce8330 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00f0ce83d0 db_command() at db_command+0x25c/frame 0xfffffe00f0ce84a0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00f0ce84b0 db_trap() at db_trap+0xe8/frame 0xfffffe00f0ce8540 kdb_trap() at kdb_trap+0x154/frame 0xfffffe00f0ce85b0 trap() at trap+0x7b8/frame 0xfffffe00f0ce86c0 calltrap() at calltrap+0x8/frame 0xfffffe00f0ce86c0 --- trap 0x3, rip = 0xffffffff80c222e7, rsp = 0xfffffe00f0ce8790, rbp = 0xfffffe00f0ce87a0 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe00f0ce87a0 vpanic() at vpanic+0x19e/frame 0xfffffe00f0ce87f0 panic() at panic+0x43/frame 0xfffffe00f0ce8850 unp_pcb_lock_peer() at unp_pcb_lock_peer+0xd6/frame 0xfffffe00f0ce88c0 unp_drop() at unp_drop+0x63/frame 0xfffffe00f0ce8900 uipc_abort() at uipc_abort+0xf7/frame 0xfffffe00f0ce8950 soabort() at soabort+0x4c/frame 0xfffffe00f0ce8970 soclose() at soclose+0x3e3/frame 0xfffffe00f0ce89e0 _fdrop() at _fdrop+0x1a/frame 0xfffffe00f0ce8a00 closef() at closef+0x1ea/frame 0xfffffe00f0ce8a90 closefp() at closefp+0x96/frame 0xfffffe00f0ce8ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f0ce8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f0ce8bf0 --- syscall (6, FreeBSD ELF64, sys_close), rip = 0x8004290fa, rsp = 0x7fffffffe618, rbp = 0x7fffffffe6b0 --- *** error reading from address fffffe00f024a6e8 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00f0ce80c0 kdb_reenter() at kdb_reenter+0x2c/frame 0xfffffe00f0ce80d0 db_get_value() at db_get_value+0x4c/frame 0xfffffe00f0ce8110 db_backtrace() at db_backtrace+0x23e/frame 0xfffffe00f0ce81a0 db_trace_thread() at db_trace_thread+0x3b/frame 0xfffffe00f0ce81c0 _db_stack_trace_all() at _db_stack_trace_all+0xb6/frame 0xfffffe00f0ce8260 db_command() at db_command+0x25c/frame 0xfffffe00f0ce8330 db_script_exec() at db_script_exec+0x1a8/frame 0xfffffe00f0ce83d0 db_command() at db_command+0x25c/frame 0xfffffe00f0ce84a0 db_command_loop() at db_command_loop+0x4d/frame 0xfffffe00f0ce84b0 db_trap() at db_trap+0xe8/frame 0xfffffe00f0ce8540 kdb_trap() at kdb_trap+0x154/frame 0xfffffe00f0ce85b0 trap() at trap+0x7b8/frame 0xfffffe00f0ce86c0 calltrap() at calltrap+0x8/frame 0xfffffe00f0ce86c0 --- trap 0x3, rip = 0xffffffff80c222e7, rsp = 0xfffffe00f0ce8790, rbp = 0xfffffe00f0ce87a0 --- kdb_enter() at kdb_enter+0x37/frame 0xfffffe00f0ce87a0 vpanic() at vpanic+0x19e/frame 0xfffffe00f0ce87f0 panic() at panic+0x43/frame 0xfffffe00f0ce8850 unp_pcb_lock_peer() at unp_pcb_lock_peer+0xd6/frame 0xfffffe00f0ce88c0 unp_drop() at unp_drop+0x63/frame 0xfffffe00f0ce8900 uipc_abort() at uipc_abort+0xf7/frame 0xfffffe00f0ce8950 soabort() at soabort+0x4c/frame 0xfffffe00f0ce8970 soclose() at soclose+0x3e3/frame 0xfffffe00f0ce89e0 _fdrop() at _fdrop+0x1a/frame 0xfffffe00f0ce8a00 closef() at closef+0x1ea/frame 0xfffffe00f0ce8a90 closefp() at closefp+0x96/frame 0xfffffe00f0ce8ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f0ce8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f0ce8bf0 --- syscall (6, FreeBSD ELF64, sys_close), rip = 0x8004290fa, rsp = 0x7fffffffe618, rbp = 0x7fffffffe6b0 --- Tracing command getty pid 2419 tid 100214 td 0xfffffe00eff54e00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00f02457c0 mi_switch() at mi_switch+0x155/frame 0xfffffe00f02457e0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00f0245820 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00f0245880 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00f0245890 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00f02458f0 tty_wait() at tty_wait+0x42/frame 0xfffffe00f0245920 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00f0245980 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00f02459b0 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00f0245a10 dofileread() at dofileread+0x81/frame 0xfffffe00f0245a60 sys_read() at sys_read+0xbc/frame 0xfffffe00f0245ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f0245bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f0245bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004460da, rsp = 0x7fffffffeb78, rbp = 0x7fffffffebc0 --- Tracing command getty pid 2418 tid 100213 td 0xfffffe00dfd5e700 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00f02407c0 mi_switch() at mi_switch+0x155/frame 0xfffffe00f02407e0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00f0240820 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00f0240880 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00f0240890 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00f02408f0 tty_wait() at tty_wait+0x42/frame 0xfffffe00f0240920 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00f0240980 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00f02409b0 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00f0240a10 dofileread() at dofileread+0x81/frame 0xfffffe00f0240a60 sys_read() at sys_read+0xbc/frame 0xfffffe00f0240ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f0240bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f0240bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004460da, rsp = 0x7fffffffeb78, rbp = 0x7fffffffebc0 --- Tracing command getty pid 2417 tid 100212 td 0xfffffe00dfd5ee00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00f023b7c0 mi_switch() at mi_switch+0x155/frame 0xfffffe00f023b7e0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00f023b820 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00f023b880 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00f023b890 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00f023b8f0 tty_wait() at tty_wait+0x42/frame 0xfffffe00f023b920 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00f023b980 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00f023b9b0 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00f023ba10 dofileread() at dofileread+0x81/frame 0xfffffe00f023ba60 sys_read() at sys_read+0xbc/frame 0xfffffe00f023bad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f023bbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f023bbf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004460da, rsp = 0x7fffffffeb78, rbp = 0x7fffffffebc0 --- Tracing command getty pid 2416 tid 100192 td 0xfffffe00dfd5f500 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e65987c0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e65987e0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e6598820 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e6598880 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e6598890 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e65988f0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e6598920 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e6598980 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e65989b0 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e6598a10 dofileread() at dofileread+0x81/frame 0xfffffe00e6598a60 sys_read() at sys_read+0xbc/frame 0xfffffe00e6598ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00e6598bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00e6598bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004460da, rsp = 0x7fffffffeb78, rbp = 0x7fffffffebc0 --- Tracing command getty pid 2415 tid 100194 td 0xfffffe00ee930700 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e65ae7c0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e65ae7e0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e65ae820 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e65ae880 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e65ae890 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e65ae8f0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e65ae920 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e65ae980 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e65ae9b0 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e65aea10 dofileread() at dofileread+0x81/frame 0xfffffe00e65aea60 sys_read() at sys_read+0xbc/frame 0xfffffe00e65aead0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00e65aebf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00e65aebf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004460da, rsp = 0x7fffffffeb78, rbp = 0x7fffffffebc0 --- Tracing command getty pid 2414 tid 100198 td 0xfffffe00dff0a700 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e65f27c0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e65f27e0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e65f2820 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e65f2880 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e65f2890 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e65f28f0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e65f2920 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e65f2980 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e65f29b0 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e65f2a10 dofileread() at dofileread+0x81/frame 0xfffffe00e65f2a60 sys_read() at sys_read+0xbc/frame 0xfffffe00e65f2ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00e65f2bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00e65f2bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004460da, rsp = 0x7fffffffeb78, rbp = 0x7fffffffebc0 --- Tracing command getty pid 2413 tid 100199 td 0xfffffe00dfef7e00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e65f77c0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e65f77e0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e65f7820 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e65f7880 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e65f7890 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e65f78f0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e65f7920 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e65f7980 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e65f79b0 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e65f7a10 dofileread() at dofileread+0x81/frame 0xfffffe00e65f7a60 sys_read() at sys_read+0xbc/frame 0xfffffe00e65f7ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00e65f7bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00e65f7bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004460da, rsp = 0x7fffffffeb78, rbp = 0x7fffffffebc0 --- Tracing command getty pid 2412 tid 100193 td 0xfffffe00ee930e00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e659d7c0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e659d7e0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e659d820 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e659d880 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e659d890 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e659d8f0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e659d920 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e659d980 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e659d9b0 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e659da10 dofileread() at dofileread+0x81/frame 0xfffffe00e659da60 sys_read() at sys_read+0xbc/frame 0xfffffe00e659dad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00e659dbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00e659dbf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004460da, rsp = 0x7fffffffeb78, rbp = 0x7fffffffebc0 --- Tracing command getty pid 2411 tid 100143 td 0xfffffe00dff0bc00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e64677c0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e64677e0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e6467820 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e6467880 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e6467890 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e64678f0 tty_wait() at tty_wait+0x42/frame 0xfffffe00e6467920 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00e6467980 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00e64679b0 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00e6467a10 dofileread() at dofileread+0x81/frame 0xfffffe00e6467a60 sys_read() at sys_read+0xbc/frame 0xfffffe00e6467ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00e6467bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00e6467bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004460da, rsp = 0x7fffffffeb78, rbp = 0x7fffffffebc0 --- Tracing command cron pid 2362 tid 100150 td 0xfffffe00dffd8700 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e648a870 mi_switch() at mi_switch+0x155/frame 0xfffffe00e648a890 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e648a8d0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e648a930 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe00e648a970 _sleep() at _sleep+0x258/frame 0xfffffe00e648aa10 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe00e648aa90 sys_nanosleep() at sys_nanosleep+0x3b/frame 0xfffffe00e648aad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00e648abf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00e648abf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003eb3ba, rsp = 0x7fffffffea58, rbp = 0x7fffffffec40 --- Tracing command sshd pid 2358 tid 100200 td 0xfffffe00dc202300 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e65fc740 mi_switch() at mi_switch+0x155/frame 0xfffffe00e65fc760 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e65fc7a0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e65fc800 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e65fc810 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e65fc870 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e65fc8b0 kern_select() at kern_select+0x8f0/frame 0xfffffe00e65fca90 sys_select() at sys_select+0x56/frame 0xfffffe00e65fcad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00e65fcbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00e65fcbf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8008f10ba, rsp = 0x7fffffffe5e8, rbp = 0x7fffffffe620 --- Tracing command powerd pid 2330 tid 100175 td 0xfffffe00dfffba00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e6513700 mi_switch() at mi_switch+0x155/frame 0xfffffe00e6513720 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e6513760 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e65137c0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe00e6513800 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x184/frame 0xfffffe00e6513870 seltdwait() at seltdwait+0x6b/frame 0xfffffe00e65138b0 kern_select() at kern_select+0x8f0/frame 0xfffffe00e6513a90 sys_select() at sys_select+0x56/frame 0xfffffe00e6513ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00e6513bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00e6513bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8004430ba, rsp = 0x7fffffffeb08, rbp = 0x7fffffffec50 --- Tracing command ntpd pid 2327 tid 100201 td 0xfffffe00e8367c00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00f0204740 mi_switch() at mi_switch+0x155/frame 0xfffffe00f0204760 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00f02047a0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00f0204800 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00f0204810 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00f0204870 seltdwait() at seltdwait+0x8d/frame 0xfffffe00f02048b0 kern_select() at kern_select+0x8f0/frame 0xfffffe00f0204a90 sys_select() at sys_select+0x56/frame 0xfffffe00f0204ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00f0204bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00f0204bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8008200ba, rsp = 0x7fffffffde78, rbp = 0x7fffffffdeb0 --- Tracing command rpc.lockd pid 2261 tid 100168 td 0xfffffe00e8368300 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e64f0640 mi_switch() at mi_switch+0x155/frame 0xfffffe00e64f0660 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e64f06a0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e64f0700 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe00e64f0740 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x184/frame 0xfffffe00e64f07b0 svc_run_internal() at svc_run_internal+0xafe/frame 0xfffffe00e64f08f0 svc_run() at svc_run+0x253/frame 0xfffffe00e64f0950 sys_nlm_syscall() at sys_nlm_syscall+0x74d/frame 0xfffffe00e64f0ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00e64f0bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00e64f0bf0 --- syscall (154, FreeBSD ELF64, sys_nlm_syscall), rip = 0x8002fb4ba, rsp = 0x7fffffffeb78, rbp = 0x7fffffffec40 --- Tracing command rpc.statd pid 2258 tid 100152 td 0xfffffe00dff05500 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e6494700 mi_switch() at mi_switch+0x155/frame 0xfffffe00e6494720 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e6494760 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e64947c0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe00e6494800 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x184/frame 0xfffffe00e6494870 seltdwait() at seltdwait+0x6b/frame 0xfffffe00e64948b0 kern_select() at kern_select+0x8f0/frame 0xfffffe00e6494a90 sys_select() at sys_select+0x56/frame 0xfffffe00e6494ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00e6494bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00e6494bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8004390ba, rsp = 0x7fffffffea18, rbp = 0x7fffffffeb60 --- Tracing command watchdogd pid 2212 tid 100196 td 0xfffffe00dfefe800 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e65b8870 mi_switch() at mi_switch+0x155/frame 0xfffffe00e65b8890 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e65b88d0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e65b8930 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe00e65b8970 _sleep() at _sleep+0x258/frame 0xfffffe00e65b8a10 kern_clock_nanosleep() at kern_clock_nanosleep+0x1e2/frame 0xfffffe00e65b8a90 sys_nanosleep() at sys_nanosleep+0x3b/frame 0xfffffe00e65b8ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00e65b8bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00e65b8bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003d43ba, rsp = 0x7fffffffea18, rbp = 0x7fffffffea60 --- Tracing command rpcbind pid 2063 tid 100164 td 0xfffffe00e8368a00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e64dc770 mi_switch() at mi_switch+0x155/frame 0xfffffe00e64dc790 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e64dc7d0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e64dc830 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe00e64dc870 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x184/frame 0xfffffe00e64dc8e0 seltdwait() at seltdwait+0x6b/frame 0xfffffe00e64dc920 kern_poll() at kern_poll+0x4b7/frame 0xfffffe00e64dcab0 sys_poll() at sys_poll+0x50/frame 0xfffffe00e64dcad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00e64dcbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00e64dcbf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8003e181a, rsp = 0x7fffffffc9b8, rbp = 0x7fffffffebd0 --- Tracing command syslogd pid 2053 tid 100173 td 0xfffffe00dffd2700 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e6509740 mi_switch() at mi_switch+0x155/frame 0xfffffe00e6509760 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e65097a0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e6509800 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e6509810 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e6509870 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e65098b0 kern_select() at kern_select+0x8f0/frame 0xfffffe00e6509a90 sys_select() at sys_select+0x56/frame 0xfffffe00e6509ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00e6509bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00e6509bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80044c0ba, rsp = 0x7fffffffe6c8, rbp = 0x7fffffffec40 --- Tracing command devd pid 1982 tid 100149 td 0xfffffe00dffd2e00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e6485700 mi_switch() at mi_switch+0x155/frame 0xfffffe00e6485720 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e6485760 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e64857c0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe00e6485800 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x184/frame 0xfffffe00e6485870 seltdwait() at seltdwait+0x6b/frame 0xfffffe00e64858b0 kern_select() at kern_select+0x8f0/frame 0xfffffe00e6485a90 sys_select() at sys_select+0x56/frame 0xfffffe00e6485ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00e6485bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00e6485bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x302c0a, rsp = 0x7fffffffcac8, rbp = 0x7fffffffec60 --- Tracing command dhclient pid 1977 tid 100195 td 0xfffffe00dc202a00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e65b3770 mi_switch() at mi_switch+0x155/frame 0xfffffe00e65b3790 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e65b37d0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e65b3830 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe00e65b3870 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x184/frame 0xfffffe00e65b38e0 seltdwait() at seltdwait+0x6b/frame 0xfffffe00e65b3920 kern_poll() at kern_poll+0x4b7/frame 0xfffffe00e65b3ab0 sys_poll() at sys_poll+0x50/frame 0xfffffe00e65b3ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00e65b3bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00e65b3bf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x80040681a, rsp = 0x7fffffffeba8, rbp = 0x7fffffffebe0 --- Tracing command dhclient pid 1918 tid 100187 td 0xfffffe00dffe4300 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e657f7b0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e657f7d0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e657f810 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e657f870 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e657f880 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e657f8e0 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e657f920 kern_poll() at kern_poll+0x4b7/frame 0xfffffe00e657fab0 sys_poll() at sys_poll+0x50/frame 0xfffffe00e657fad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00e657fbf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00e657fbf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x80040681a, rsp = 0x7fffffffebb8, rbp = 0x7fffffffebe0 --- Tracing command dhclient pid 1915 tid 100174 td 0xfffffe00dffdb800 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e650e740 mi_switch() at mi_switch+0x155/frame 0xfffffe00e650e760 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e650e7a0 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00e650e800 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00e650e810 _cv_wait_sig() at _cv_wait_sig+0x161/frame 0xfffffe00e650e870 seltdwait() at seltdwait+0x8d/frame 0xfffffe00e650e8b0 kern_select() at kern_select+0x8f0/frame 0xfffffe00e650ea90 sys_select() at sys_select+0x56/frame 0xfffffe00e650ead0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00e650ebf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00e650ebf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8004630ba, rsp = 0x7fffffffe968, rbp = 0x7fffffffea20 --- Tracing command syncer pid 33 tid 100139 td 0xfffffe00dfef9300 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e6447a10 mi_switch() at mi_switch+0x155/frame 0xfffffe00e6447a30 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e6447a70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e6447ab0 _cv_timedwait_sbt() at _cv_timedwait_sbt+0x184/frame 0xfffffe00e6447b20 sched_sync() at sched_sync+0x68b/frame 0xfffffe00e6447bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e6447bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e6447bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command vnlru pid 32 tid 100134 td 0xfffffe00dfef9a00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e642e9d0 mi_switch() at mi_switch+0x155/frame 0xfffffe00e642e9f0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e642ea30 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e642ea70 _sleep() at _sleep+0x26c/frame 0xfffffe00e642eb10 vnlru_proc() at vnlru_proc+0xaac/frame 0xfffffe00e642ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e642ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e642ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100131 td 0xfffffe00dfefa100 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00de89ea20 mi_switch() at mi_switch+0x155/frame 0xfffffe00de89ea40 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00de89ea80 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00de89eac0 _sleep() at _sleep+0x26c/frame 0xfffffe00de89eb60 buf_daemon() at buf_daemon+0x112/frame 0xfffffe00de89ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00de89ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00de89ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100132 td 0xfffffe00dfefda00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e00c4a10 mi_switch() at mi_switch+0x155/frame 0xfffffe00e00c4a30 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e00c4a70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e00c4ab0 _sleep() at _sleep+0x26c/frame 0xfffffe00e00c4b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe00e00c4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e00c4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e00c4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100136 td 0xfffffe00dfefd300 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e6438a10 mi_switch() at mi_switch+0x155/frame 0xfffffe00e6438a30 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e6438a70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e6438ab0 _sleep() at _sleep+0x26c/frame 0xfffffe00e6438b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe00e6438bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e6438bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e6438bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100137 td 0xfffffe00dfefcc00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e643da10 mi_switch() at mi_switch+0x155/frame 0xfffffe00e643da30 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e643da70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e643dab0 _sleep() at _sleep+0x26c/frame 0xfffffe00e643db50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe00e643dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e643dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e643dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100138 td 0xfffffe00dfefc500 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e6442a10 mi_switch() at mi_switch+0x155/frame 0xfffffe00e6442a30 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e6442a70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e6442ab0 _sleep() at _sleep+0x26c/frame 0xfffffe00e6442b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe00e6442bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e6442bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e6442bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100140 td 0xfffffe00dfefbe00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e644ca10 mi_switch() at mi_switch+0x155/frame 0xfffffe00e644ca30 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e644ca70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e644cab0 _sleep() at _sleep+0x26c/frame 0xfffffe00e644cb50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe00e644cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e644cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e644cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100141 td 0xfffffe00dfefb700 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e6451a10 mi_switch() at mi_switch+0x155/frame 0xfffffe00e6451a30 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e6451a70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e6451ab0 _sleep() at _sleep+0x26c/frame 0xfffffe00e6451b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe00e6451bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e6451bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e6451bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100142 td 0xfffffe00dfefb000 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e6456a10 mi_switch() at mi_switch+0x155/frame 0xfffffe00e6456a30 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e6456a70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e6456ab0 _sleep() at _sleep+0x26c/frame 0xfffffe00e6456b50 bufspace_daemon() at bufspace_daemon+0x2d9/frame 0xfffffe00e6456bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e6456bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e6456bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100154 td 0xfffffe00dc203800 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e649ea40 mi_switch() at mi_switch+0x155/frame 0xfffffe00e649ea60 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e649eaa0 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e649eae0 _sleep() at _sleep+0x26c/frame 0xfffffe00e649eb80 softdep_flush() at softdep_flush+0x16e/frame 0xfffffe00e649ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e649ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e649ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 31 tid 100204 td 0xfffffe00ee930000 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00f0213a40 mi_switch() at mi_switch+0x155/frame 0xfffffe00f0213a60 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00f0213aa0 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00f0213ae0 _sleep() at _sleep+0x26c/frame 0xfffffe00f0213b80 softdep_flush() at softdep_flush+0x16e/frame 0xfffffe00f0213bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00f0213bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f0213bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command vmdaemon pid 30 tid 100130 td 0xfffffe00de7dce00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00de899a40 mi_switch() at mi_switch+0x155/frame 0xfffffe00de899a60 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00de899aa0 _sleep() at _sleep+0x2cd/frame 0xfffffe00de899b40 vm_daemon() at vm_daemon+0x82/frame 0xfffffe00de899bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00de899bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00de899bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 29 tid 100129 td 0xfffffe00de7dd500 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00de894770 mi_switch() at mi_switch+0x155/frame 0xfffffe00de894790 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00de8947d0 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00de894810 _sleep() at _sleep+0x26c/frame 0xfffffe00de8948b0 vm_pageout_worker() at vm_pageout_worker+0x28c/frame 0xfffffe00de894b70 vm_pageout() at vm_pageout+0x21e/frame 0xfffffe00de894bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00de894bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00de894bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 29 tid 100133 td 0xfffffe00dff06300 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e11e9830 mi_switch() at mi_switch+0x155/frame 0xfffffe00e11e9850 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e11e9890 _sleep() at _sleep+0x2cd/frame 0xfffffe00e11e9930 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0xc0e/frame 0xfffffe00e11e9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e11e9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e11e9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 29 tid 100135 td 0xfffffe00dff05c00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00e6433a40 mi_switch() at mi_switch+0x155/frame 0xfffffe00e6433a60 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00e6433aa0 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00e6433ae0 _sleep() at _sleep+0x26c/frame 0xfffffe00e6433b80 uma_reclaim_worker() at uma_reclaim_worker+0xb8/frame 0xfffffe00e6433bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00e6433bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e6433bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon1 pid 28 tid 100128 td 0xfffffe00dfd61100 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00de88fa40 mi_switch() at mi_switch+0x155/frame 0xfffffe00de88fa60 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00de88faa0 _sleep() at _sleep+0x2cd/frame 0xfffffe00de88fb40 enc_daemon() at enc_daemon+0xe4/frame 0xfffffe00de88fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00de88fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00de88fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command enc_daemon0 pid 27 tid 100127 td 0xfffffe00dfd61800 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00de88aa40 mi_switch() at mi_switch+0x155/frame 0xfffffe00de88aa60 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00de88aaa0 _sleep() at _sleep+0x2cd/frame 0xfffffe00de88ab40 enc_daemon() at enc_daemon+0xe4/frame 0xfffffe00de88abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00de88abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00de88abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command rand_harvestq pid 26 tid 100124 td 0xfffffe00de7dea00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00de87b9f0 mi_switch() at mi_switch+0x155/frame 0xfffffe00de87ba10 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00de87ba50 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00de87ba90 _sleep() at _sleep+0x26c/frame 0xfffffe00de87bb30 random_kthread() at random_kthread+0x67/frame 0xfffffe00de87bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00de87bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00de87bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod4 pid 25 tid 100120 td 0xfffffe00de7dc000 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00de85ba30 mi_switch() at mi_switch+0x155/frame 0xfffffe00de85ba50 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00de85ba90 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00de85bad0 _sleep() at _sleep+0x26c/frame 0xfffffe00de85bb70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe00de85bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00de85bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00de85bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod3 pid 24 tid 100119 td 0xfffffe00de7dc700 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00de856a30 mi_switch() at mi_switch+0x155/frame 0xfffffe00de856a50 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00de856a90 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00de856ad0 _sleep() at _sleep+0x26c/frame 0xfffffe00de856b70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe00de856bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00de856bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00de856bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod2 pid 23 tid 100118 td 0xfffffe003dbfd100 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00de851a30 mi_switch() at mi_switch+0x155/frame 0xfffffe00de851a50 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00de851a90 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00de851ad0 _sleep() at _sleep+0x26c/frame 0xfffffe00de851b70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe00de851bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00de851bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00de851bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod1 pid 22 tid 100117 td 0xfffffe003dbfd800 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00de84ca30 mi_switch() at mi_switch+0x155/frame 0xfffffe00de84ca50 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00de84ca90 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00de84cad0 _sleep() at _sleep+0x26c/frame 0xfffffe00de84cb70 soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe00de84cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00de84cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00de84cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100098 td 0xfffffe00d69d6700 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb81ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb81ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb81b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00adb81b70 usb_process() at usb_process+0xc3/frame 0xfffffe00adb81bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb81bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb81bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100099 td 0xfffffe00d69d6000 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb86ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb86ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb86b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00adb86b70 usb_process() at usb_process+0xc3/frame 0xfffffe00adb86bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb86bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb86bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100100 td 0xfffffe003dbf1800 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb8bab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb8bad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb8bb10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00adb8bb70 usb_process() at usb_process+0xc3/frame 0xfffffe00adb8bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb8bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb8bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100101 td 0xfffffe003dbf1100 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb90ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb90ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb90b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00adb90b70 usb_process() at usb_process+0xc3/frame 0xfffffe00adb90bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb90bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb90bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100102 td 0xfffffe003dbf0a00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb95ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb95ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb95b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00adb95b70 usb_process() at usb_process+0xc3/frame 0xfffffe00adb95bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb95bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb95bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100104 td 0xfffffe00db22de00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb9fab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb9fad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb9fb10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00adb9fb70 usb_process() at usb_process+0xc3/frame 0xfffffe00adb9fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb9fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb9fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100105 td 0xfffffe00db22d700 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adba4ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00adba4ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adba4b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00adba4b70 usb_process() at usb_process+0xc3/frame 0xfffffe00adba4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adba4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adba4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100106 td 0xfffffe00db22d000 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adba9ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00adba9ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adba9b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00adba9b70 usb_process() at usb_process+0xc3/frame 0xfffffe00adba9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adba9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adba9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100107 td 0xfffffe00d69d9800 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adbaeab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00adbaead0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adbaeb10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00adbaeb70 usb_process() at usb_process+0xc3/frame 0xfffffe00adbaebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adbaebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adbaebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100108 td 0xfffffe00d69d9100 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adbb3ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00adbb3ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adbb3b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00adbb3b70 usb_process() at usb_process+0xc3/frame 0xfffffe00adbb3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adbb3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adbb3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100109 td 0xfffffe00dc200e00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adbc4ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00adbc4ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adbc4b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00adbc4b70 usb_process() at usb_process+0xc3/frame 0xfffffe00adbc4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adbc4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adbc4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100110 td 0xfffffe00dc200700 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adbc9ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00adbc9ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adbc9b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00adbc9b70 usb_process() at usb_process+0xc3/frame 0xfffffe00adbc9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adbc9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adbc9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100111 td 0xfffffe00dc200000 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adbceab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00adbcead0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adbceb10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00adbceb70 usb_process() at usb_process+0xc3/frame 0xfffffe00adbcebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adbcebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adbcebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100112 td 0xfffffe00db230800 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adbd3ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00adbd3ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adbd3b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00adbd3b70 usb_process() at usb_process+0xc3/frame 0xfffffe00adbd3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adbd3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adbd3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100113 td 0xfffffe00db230100 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adbd8ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00adbd8ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adbd8b10 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00adbd8b70 usb_process() at usb_process+0xc3/frame 0xfffffe00adbd8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adbd8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adbd8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100092 td 0xfffffe003dbeee00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb63a60 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb63a80 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb63ac0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adb63b60 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe00adb63bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb63bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb63bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100093 td 0xfffffe003dbee700 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb68a60 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb68a80 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb68ac0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adb68b60 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe00adb68bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb68bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb68bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100094 td 0xfffffe003dbee000 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb6da60 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb6da80 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb6dac0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adb6db60 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe00adb6dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb6dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb6dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100126 td 0xfffffe00de7ddc00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00de885a70 mi_switch() at mi_switch+0x155/frame 0xfffffe00de885a90 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00de885ad0 _sleep() at _sleep+0x2cd/frame 0xfffffe00de885b70 xpt_scanner_thread() at xpt_scanner_thread+0x68/frame 0xfffffe00de885bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00de885bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00de885bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 11 pid 19 tid 100091 td 0xfffffe003dbef500 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb5ea50 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb5ea70 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb5eab0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adb5eb50 crypto_ret_proc() at crypto_ret_proc+0x21f/frame 0xfffffe00adb5ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb5ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb5ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 10 pid 18 tid 100090 td 0xfffffe003dbefc00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb59a50 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb59a70 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb59ab0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adb59b50 crypto_ret_proc() at crypto_ret_proc+0x21f/frame 0xfffffe00adb59bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb59bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb59bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 9 pid 17 tid 100089 td 0xfffffe003dbf0300 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb54a50 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb54a70 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb54ab0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adb54b50 crypto_ret_proc() at crypto_ret_proc+0x21f/frame 0xfffffe00adb54bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb54bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb54bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 8 pid 16 tid 100088 td 0xfffffe003dc15800 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb4fa50 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb4fa70 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb4fab0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adb4fb50 crypto_ret_proc() at crypto_ret_proc+0x21f/frame 0xfffffe00adb4fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb4fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb4fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 7 pid 15 tid 100087 td 0xfffffe003dbfa000 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb4aa50 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb4aa70 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb4aab0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adb4ab50 crypto_ret_proc() at crypto_ret_proc+0x21f/frame 0xfffffe00adb4abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb4abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb4abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 6 pid 9 tid 100086 td 0xfffffe003dbfa700 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb45a50 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb45a70 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb45ab0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adb45b50 crypto_ret_proc() at crypto_ret_proc+0x21f/frame 0xfffffe00adb45bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb45bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb45bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 5 pid 8 tid 100085 td 0xfffffe003dbfae00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb40a50 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb40a70 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb40ab0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adb40b50 crypto_ret_proc() at crypto_ret_proc+0x21f/frame 0xfffffe00adb40bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb40bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb40bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 4 pid 7 tid 100084 td 0xfffffe003dbfb500 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb3ba50 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb3ba70 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb3bab0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adb3bb50 crypto_ret_proc() at crypto_ret_proc+0x21f/frame 0xfffffe00adb3bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb3bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb3bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 3 pid 6 tid 100083 td 0xfffffe003dbfbc00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb36a50 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb36a70 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb36ab0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adb36b50 crypto_ret_proc() at crypto_ret_proc+0x21f/frame 0xfffffe00adb36bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb36bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb36bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 2 pid 5 tid 100082 td 0xfffffe003dbfc300 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb31a50 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb31a70 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb31ab0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adb31b50 crypto_ret_proc() at crypto_ret_proc+0x21f/frame 0xfffffe00adb31bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb31bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb31bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 1 pid 4 tid 100081 td 0xfffffe003dbfca00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb2ca50 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb2ca70 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb2cab0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adb2cb50 crypto_ret_proc() at crypto_ret_proc+0x21f/frame 0xfffffe00adb2cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb2cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb2cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 0 pid 3 tid 100080 td 0xfffffe003dc12000 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb27a50 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb27a70 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb27ab0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adb27b50 crypto_ret_proc() at crypto_ret_proc+0x21f/frame 0xfffffe00adb27bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb27bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb27bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 2 tid 100079 td 0xfffffe003dc12700 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb22a70 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb22a90 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb22ad0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adb22b70 crypto_proc() at crypto_proc+0x25f/frame 0xfffffe00adb22bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb22bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb22bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sequencer 00 pid 14 tid 100066 td 0xfffffe003dc39100 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adae1a70 mi_switch() at mi_switch+0x155/frame 0xfffffe00adae1a90 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adae1ad0 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00adae1b30 seq_eventthread() at seq_eventthread+0x91/frame 0xfffffe00adae1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adae1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adae1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100063 td 0xfffffe003dc43500 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adac6a70 mi_switch() at mi_switch+0x155/frame 0xfffffe00adac6a90 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adac6ad0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adac6b70 g_run_events() at g_run_events+0x64/frame 0xfffffe00adac6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adac6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adac6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100064 td 0xfffffe003dc42e00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adacba70 mi_switch() at mi_switch+0x155/frame 0xfffffe00adacba90 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adacbad0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adacbb70 g_io_schedule_up() at g_io_schedule_up+0x3c/frame 0xfffffe00adacbba0 g_up_procbody() at g_up_procbody+0x5c/frame 0xfffffe00adacbbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adacbbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adacbbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100065 td 0xfffffe003dc39800 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adad0a70 mi_switch() at mi_switch+0x155/frame 0xfffffe00adad0a90 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adad0ad0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adad0b70 g_io_schedule_down() at g_io_schedule_down+0x4c/frame 0xfffffe00adad0ba0 g_down_procbody() at g_down_procbody+0x5c/frame 0xfffffe00adad0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adad0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adad0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100041 td 0xfffffe00ad870e00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ada04b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00ada04b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00ada04bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ada04bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ada04bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100042 td 0xfffffe00ad870700 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ada09b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00ada09b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00ada09bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ada09bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ada09bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100045 td 0xfffffe00ad86d100 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100048 td 0xfffffe00ad86bc00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ada57b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00ada57b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00ada57bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ada57bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ada57bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100049 td 0xfffffe003dc42700 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100050 td 0xfffffe003dc42000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100051 td 0xfffffe00ad873800 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100052 td 0xfffffe00ad873100 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100053 td 0xfffffe00ad872a00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100054 td 0xfffffe00ad872300 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100055 td 0xfffffe00ad871c00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100056 td 0xfffffe00ad871500 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100057 td 0xfffffe003dc36000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100058 td 0xfffffe003dc45800 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100059 td 0xfffffe003dc45100 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100060 td 0xfffffe003dc44a00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ada93b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00ada93b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00ada93bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ada93bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ada93bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100061 td 0xfffffe003dc44300 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100095 td 0xfffffe00d69d7c00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100096 td 0xfffffe00d69d7500 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb77b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb77b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00adb77bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb77bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb77bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100097 td 0xfffffe00d69d6e00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb7cb00 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb7cb20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00adb7cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb7cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb7cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100103 td 0xfffffe00db22e500 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb9ab00 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb9ab20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00adb9abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb9abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb9abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100114 td 0xfffffe00db22fa00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100115 td 0xfffffe00db22f300 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adbe2b00 mi_switch() at mi_switch+0x155/frame 0xfffffe00adbe2b20 ithread_loop() at ithread_loop+0x99/frame 0xfffffe00adbe2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adbe2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adbe2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100178 td 0xfffffe00dfffac00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100179 td 0xfffffe00dfffa500 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100180 td 0xfffffe00dfff9e00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100181 td 0xfffffe00dfff9700 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100182 td 0xfffffe00dfff9000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100183 td 0xfffffe00ee932300 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100184 td 0xfffffe00ee931c00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100185 td 0xfffffe00ee931500 fork_trampoline() at fork_trampoline Tracing command idle pid 11 tid 100003 td 0xfffffe00ad7d9300 (CPU 0) cpustop_handler() at cpustop_handler+0x26/frame 0xffffffff81de70a0 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xffffffff81de70b0 trap() at trap+0x56/frame 0xffffffff81de71c0 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff81de71c0 --- trap 0x13, rip = 0xffffffff80fe1466, rsp = 0xfffffe00ad546a50, rbp = 0xfffffe00ad546a50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00ad546a50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00ad546aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00ad546ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00ad546ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00ad546bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad546bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad546bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xfffffe00ad7d8c00 (CPU 1) cpustop_handler() at cpustop_handler+0x26/frame 0xfffffe003dbd0e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003dbd0e10 trap() at trap+0x56/frame 0xfffffe003dbd0f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003dbd0f20 --- trap 0x13, rip = 0xffffffff80fe1466, rsp = 0xfffffe00ad54ba50, rbp = 0xfffffe00ad54ba50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00ad54ba50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00ad54baa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00ad54bac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00ad54bae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00ad54bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad54bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad54bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffffe00ad7d8500 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad5509e0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad550a00 Xtimerint() at Xtimerint+0xb1/frame 0xfffffe00ad550a00 --- interrupt, rip = 0xffffffff80fe1b3e, rsp = 0xfffffe00ad550ad0, rbp = 0xfffffe00ad550ae0 --- cpu_idle() at cpu_idle+0xee/frame 0xfffffe00ad550ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00ad550bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad550bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad550bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xfffffe00ad7d7e00 (CPU 3) cpustop_handler() at cpustop_handler+0x26/frame 0xfffffe003dbe8e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003dbe8e10 trap() at trap+0x56/frame 0xfffffe003dbe8f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003dbe8f20 --- trap 0x13, rip = 0xffffffff80fe1466, rsp = 0xfffffe00ad555a50, rbp = 0xfffffe00ad555a50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00ad555a50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00ad555aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00ad555ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00ad555ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00ad555bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad555bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad555bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100007 td 0xfffffe00ad7d7700 (CPU 4) cpustop_handler() at cpustop_handler+0x26/frame 0xfffffe003dbf4e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003dbf4e10 trap() at trap+0x56/frame 0xfffffe003dbf4f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003dbf4f20 --- trap 0x13, rip = 0xffffffff80fe1466, rsp = 0xfffffe00ad55aa50, rbp = 0xfffffe00ad55aa50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00ad55aa50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00ad55aaa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00ad55aac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00ad55aae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00ad55abb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad55abf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad55abf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xfffffe00ad7d7000 (CPU 5) cpustop_handler() at cpustop_handler+0x26/frame 0xfffffe003dc00e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003dc00e10 trap() at trap+0x56/frame 0xfffffe003dc00f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003dc00f20 --- trap 0x13, rip = 0xffffffff80fe1466, rsp = 0xfffffe00ad55fa50, rbp = 0xfffffe00ad55fa50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00ad55fa50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00ad55faa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00ad55fac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00ad55fae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00ad55fbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad55fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad55fbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xfffffe00ad7e1a00 (CPU 6) cpustop_handler() at cpustop_handler+0x26/frame 0xfffffe003dc0ce00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003dc0ce10 trap() at trap+0x56/frame 0xfffffe003dc0cf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003dc0cf20 --- trap 0x13, rip = 0xffffffff80fe1466, rsp = 0xfffffe00ad564a50, rbp = 0xfffffe00ad564a50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00ad564a50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00ad564aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00ad564ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00ad564ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00ad564bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad564bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad564bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xfffffe00ad7e1300 (CPU 7) cpustop_handler() at cpustop_handler+0x26/frame 0xfffffe003dc18e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003dc18e10 trap() at trap+0x56/frame 0xfffffe003dc18f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003dc18f20 --- trap 0x13, rip = 0xffffffff80fe1466, rsp = 0xfffffe00ad569a50, rbp = 0xfffffe00ad569a50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00ad569a50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00ad569aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00ad569ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00ad569ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00ad569bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad569bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad569bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffffe00ad7e0c00 (CPU 8) cpustop_handler() at cpustop_handler+0x26/frame 0xfffffe003dc24e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003dc24e10 trap() at trap+0x56/frame 0xfffffe003dc24f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003dc24f20 --- trap 0x13, rip = 0xffffffff80fe1466, rsp = 0xfffffe00ad56ea50, rbp = 0xfffffe00ad56ea50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00ad56ea50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00ad56eaa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00ad56eac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00ad56eae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00ad56ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad56ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad56ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100012 td 0xfffffe00ad7e0500 (CPU 9) cpustop_handler() at cpustop_handler+0x26/frame 0xfffffe003dc30e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003dc30e10 trap() at trap+0x56/frame 0xfffffe003dc30f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003dc30f20 --- trap 0x13, rip = 0xffffffff80fe1466, rsp = 0xfffffe00ad573a50, rbp = 0xfffffe00ad573a50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00ad573a50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00ad573aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00ad573ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00ad573ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00ad573bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad573bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad573bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xfffffe00ad7dfe00 (CPU 10) cpustop_handler() at cpustop_handler+0x26/frame 0xfffffe003dc3ce00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe003dc3ce10 trap() at trap+0x56/frame 0xfffffe003dc3cf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe003dc3cf20 --- trap 0x13, rip = 0xffffffff80fe1466, rsp = 0xfffffe00ad578a50, rbp = 0xfffffe00ad578a50 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00ad578a50 acpi_cpu_idle() at acpi_cpu_idle+0x232/frame 0xfffffe00ad578aa0 cpu_idle_acpi() at cpu_idle_acpi+0x3e/frame 0xfffffe00ad578ac0 cpu_idle() at cpu_idle+0x9f/frame 0xfffffe00ad578ae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00ad578bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad578bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad578bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100014 td 0xfffffe00ad7df700 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad57d9e0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad57da00 Xtimerint() at Xtimerint+0xb1/frame 0xfffffe00ad57da00 --- interrupt, rip = 0xffffffff80fe1b3e, rsp = 0xfffffe00ad57dad0, rbp = 0xfffffe00ad57dae0 --- cpu_idle() at cpu_idle+0xee/frame 0xfffffe00ad57dae0 sched_idletd() at sched_idletd+0x3d1/frame 0xfffffe00ad57dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad57dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad57dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command init pid 1 tid 100002 td 0xfffffe00ad7d9a00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad5416e0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad541700 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad541740 sleepq_catch_signals() at sleepq_catch_signals+0x3f2/frame 0xfffffe00ad5417a0 sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00ad5417b0 _sleep() at _sleep+0x2c3/frame 0xfffffe00ad541850 kern_wait6() at kern_wait6+0x482/frame 0xfffffe00ad5418e0 sys_wait4() at sys_wait4+0x7b/frame 0xfffffe00ad541ad0 amd64_syscall() at amd64_syscall+0x159/frame 0xfffffe00ad541bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00ad541bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x2bda3a, rsp = 0x7fffffffe728, rbp = 0x7fffffffe790 --- Tracing command audit pid 10 tid 100001 td 0xfffffe00ad7da100 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad53ca90 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad53cab0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad53caf0 _cv_wait() at _cv_wait+0x15a/frame 0xfffffe00ad53cb50 audit_worker() at audit_worker+0x9f/frame 0xfffffe00ad53cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad53cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad53cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xffffffff81c8c820 sched_switch() at sched_switch+0x5b2/frame 0xffffffff82120ad0 mi_switch() at mi_switch+0x155/frame 0xffffffff82120af0 sleepq_switch() at sleepq_switch+0x11a/frame 0xffffffff82120b30 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xffffffff82120b70 _sleep() at _sleep+0x26c/frame 0xffffffff82120c10 swapper() at swapper+0x75/frame 0xffffffff82120c60 mi_startup() at mi_startup+0x2ed/frame 0xffffffff82120cb0 btext() at btext+0x2c Tracing command kernel pid 0 tid 100015 td 0xfffffe00ad7df000 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad582ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad582ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad582b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00ad582b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00ad582bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad582bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad582bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xfffffe00ad7da800 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad587ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad587ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad587b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00ad587b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00ad587bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad587bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad587bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100017 td 0xfffffe00ad862300 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad58cab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad58cad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad58cb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00ad58cb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00ad58cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad58cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad58cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100018 td 0xfffffe00ad861c00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad591ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad591ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad591b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00ad591b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00ad591bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad591bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad591bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xfffffe00ad861500 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad596ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad596ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad596b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00ad596b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00ad596bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad596bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad596bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xfffffe00ad860e00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad59bab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad59bad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad59bb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00ad59bb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00ad59bbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad59bbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad59bbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xfffffe00ad860700 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad5a0ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad5a0ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad5a0b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00ad5a0b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00ad5a0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad5a0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad5a0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xfffffe00ad860000 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad5a5ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad5a5ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad5a5b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00ad5a5b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00ad5a5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad5a5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad5a5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xfffffe00ad7e2800 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad5aaab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad5aaad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad5aab10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00ad5aab80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00ad5aabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad5aabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad5aabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xfffffe00ad7e2100 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad5afab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad5afad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad5afb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00ad5afb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00ad5afbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad5afbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad5afbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xfffffe00ad865c00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad5b4ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad5b4ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad5b4b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00ad5b4b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00ad5b4bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad5b4bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad5b4bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100026 td 0xfffffe00ad865500 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad5b9ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad5b9ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad5b9b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00ad5b9b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00ad5b9bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad5b9bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad5b9bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100027 td 0xfffffe00ad864e00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad5beab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad5bead0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad5beb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00ad5beb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00ad5bebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad5bebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad5bebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100028 td 0xfffffe00ad864700 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad5c3ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad5c3ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad5c3b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00ad5c3b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00ad5c3bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad5c3bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad5c3bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100029 td 0xfffffe00ad864000 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad5c8ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad5c8ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad5c8b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00ad5c8b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00ad5c8bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad5c8bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad5c8bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100030 td 0xfffffe00ad863800 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad5cdab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad5cdad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad5cdb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00ad5cdb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00ad5cdbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad5cdbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad5cdbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100031 td 0xfffffe00ad863100 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad5d2ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad5d2ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad5d2b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00ad5d2b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00ad5d2bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad5d2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad5d2bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100032 td 0xfffffe00ad862a00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad5d7ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad5d7ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad5d7b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00ad5d7b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00ad5d7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad5d7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad5d7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100033 td 0xfffffe00ad86b500 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad5dcab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad5dcad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad5dcb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00ad5dcb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00ad5dcbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad5dcbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad5dcbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100034 td 0xfffffe00ad86ae00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad5e1ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad5e1ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad5e1b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00ad5e1b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00ad5e1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad5e1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad5e1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100035 td 0xfffffe00ad86a700 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad5e6ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad5e6ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad5e6b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00ad5e6b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00ad5e6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad5e6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad5e6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100036 td 0xfffffe00ad86a000 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad5ebab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad5ebad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad5ebb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00ad5ebb80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00ad5ebbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad5ebbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad5ebbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100037 td 0xfffffe00ad867800 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad5f0ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad5f0ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad5f0b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00ad5f0b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00ad5f0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad5f0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad5f0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100038 td 0xfffffe00ad867100 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad5f5ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad5f5ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad5f5b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00ad5f5b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00ad5f5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad5f5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad5f5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100039 td 0xfffffe00ad866a00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad5faab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad5faad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad5fab10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00ad5fab80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xef/frame 0xfffffe00ad5fabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad5fabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad5fabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100040 td 0xfffffe00ad866300 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ad5ffa80 mi_switch() at mi_switch+0x155/frame 0xfffffe00ad5ffaa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ad5ffae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00ad5ffb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00ad5ffbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ad5ffbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad5ffbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100043 td 0xfffffe00ad870000 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ada0ea80 mi_switch() at mi_switch+0x155/frame 0xfffffe00ada0eaa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ada0eae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00ada0eb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00ada0ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ada0ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ada0ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100044 td 0xfffffe00ad86d800 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ada13a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00ada13aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ada13ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00ada13b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00ada13bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ada13bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ada13bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100046 td 0xfffffe00ad86ca00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ada1da80 mi_switch() at mi_switch+0x155/frame 0xfffffe00ada1daa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ada1dae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00ada1db80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00ada1dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ada1dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ada1dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100047 td 0xfffffe00ad86c300 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00ada22a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00ada22aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00ada22ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00ada22b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00ada22bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00ada22bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ada22bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100062 td 0xfffffe003dc43c00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adac1a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00adac1aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adac1ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adac1b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00adac1bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adac1bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adac1bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100067 td 0xfffffe003dc38a00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adae6a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00adae6aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adae6ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adae6b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00adae6bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adae6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adae6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100068 td 0xfffffe003dc38300 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adaeba80 mi_switch() at mi_switch+0x155/frame 0xfffffe00adaebaa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adaebae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adaebb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00adaebbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adaebbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adaebbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100069 td 0xfffffe003dc37c00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adaf0a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00adaf0aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adaf0ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adaf0b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00adaf0bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adaf0bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adaf0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100070 td 0xfffffe003dc37500 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adaf5a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00adaf5aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adaf5ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adaf5b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00adaf5bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adaf5bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adaf5bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100071 td 0xfffffe003dc36e00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adafaa80 mi_switch() at mi_switch+0x155/frame 0xfffffe00adafaaa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adafaae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adafab80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00adafabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adafabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adafabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100072 td 0xfffffe003dc36700 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adaffa80 mi_switch() at mi_switch+0x155/frame 0xfffffe00adaffaa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adaffae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adaffb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00adaffbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adaffbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adaffbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100073 td 0xfffffe003dc15100 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb04a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb04aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb04ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adb04b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00adb04bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb04bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb04bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100074 td 0xfffffe003dc14a00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb09a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb09aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb09ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adb09b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00adb09bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb09bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb09bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100075 td 0xfffffe003dc14300 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb0ea80 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb0eaa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb0eae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adb0eb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00adb0ebb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb0ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb0ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100076 td 0xfffffe003dc13c00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb13a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb13aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb13ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adb13b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00adb13bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb13bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb13bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100077 td 0xfffffe003dc13500 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb18a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb18aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb18ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adb18b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00adb18bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb18bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb18bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100078 td 0xfffffe003dc12e00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adb1da80 mi_switch() at mi_switch+0x155/frame 0xfffffe00adb1daa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adb1dae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00adb1db80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00adb1dbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adb1dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adb1dbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100116 td 0xfffffe00db22ec00 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00adbe7ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00adbe7ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00adbe7b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00adbe7b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xef/frame 0xfffffe00adbe7bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00adbe7bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adbe7bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100121 td 0xfffffe00dfd5e000 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00de86cab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00de86cad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00de86cb10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00de86cb80 taskqueue_thread_loop() at taskqueue_thread_loop+0xef/frame 0xfffffe00de86cbb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00de86cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00de86cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100122 td 0xfffffe00de7df800 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00de871ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00de871ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00de871b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00de871b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xef/frame 0xfffffe00de871bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00de871bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00de871bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100123 td 0xfffffe00de7df100 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00de876ab0 mi_switch() at mi_switch+0x155/frame 0xfffffe00de876ad0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00de876b10 msleep_spin_sbt() at msleep_spin_sbt+0x152/frame 0xfffffe00de876b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xef/frame 0xfffffe00de876bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00de876bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00de876bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100125 td 0xfffffe00de7de300 sched_switch() at sched_switch+0x5b2/frame 0xfffffe00de880a80 mi_switch() at mi_switch+0x155/frame 0xfffffe00de880aa0 sleepq_switch() at sleepq_switch+0x11a/frame 0xfffffe00de880ae0 _sleep() at _sleep+0x2cd/frame 0xfffffe00de880b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xc7/frame 0xfffffe00de880bb0 fork_exit() at fork_exit+0x80/frame 0xfffffe00de880bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00de880bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db> (kgdb) bt #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55 #1 doadump (textdump=0x0) at ../../../kern/kern_shutdown.c:394 #2 0xffffffff8049e61a in db_dump (dummy=, dummy2=, dummy3=, dummy4=) at ../../../ddb/db_command.c:575 #3 0xffffffff8049e3dc in db_command (last_cmdp=, cmd_table=, dopager=0x0) at ../../../ddb/db_command.c:482 #4 0xffffffff804a33e8 in db_script_exec (scriptname=, warnifnotfound=) at ../../../ddb/db_script.c:304 #5 0xffffffff8049e3dc in db_command (last_cmdp=, cmd_table=, dopager=0x0) at ../../../ddb/db_command.c:482 #6 0xffffffff804a33e8 in db_script_exec (scriptname=, warnifnotfound=) at ../../../ddb/db_script.c:304 #7 0xffffffff8049e3dc in db_command (last_cmdp=, cmd_table=, dopager=0x1) at ../../../ddb/db_command.c:482 #8 0xffffffff8049e14d in db_command_loop () at ../../../ddb/db_command.c:535 #9 0xffffffff804a13f8 in db_trap (type=, code=) at ../../../ddb/db_main.c:270 #10 0xffffffff80c22b54 in kdb_trap (type=0x3, code=0x0, tf=) at ../../../kern/subr_kdb.c:699 #11 0xffffffff81023eb8 in trap (frame=0xfffffe00f0ce86d0) at ../../../amd64/amd64/trap.c:576 #12 #13 kdb_enter (why=0xffffffff811f0d15 "panic", msg=) at ../../../kern/subr_kdb.c:486 #14 0xffffffff80bd78be in vpanic (fmt=, ap=) at ../../../kern/kern_shutdown.c:902 #15 0xffffffff80bd7653 in panic (fmt=0xffffffff81c8dc48 "'4\033\201\377\377\377\377") at ../../../kern/kern_shutdown.c:839 #16 0xffffffff80c86846 in unp_pcb_lock_peer (unp=0xfffff80771055d00) at ../../../kern/uipc_usrreq.c:389 #17 0xffffffff80c866a3 in unp_drop (unp=0xfffff80771055d00) at ../../../kern/uipc_usrreq.c:1932 #18 0xffffffff80c83427 in uipc_abort (so=0xfffff8074c9b8780) at ../../../kern/uipc_usrreq.c:483 #19 0xffffffff80c76e3c in soabort (so=0xfffff8074c9b8780) at ../../../kern/uipc_socket.c:1284 #20 0xffffffff80c77c73 in soclose (so=0xfffff804f9ee63c0) at ../../../kern/uipc_socket.c:1244 #21 0xffffffff80b7ad9a in fo_close (fp=0xfffff800456c20a0, td=0x80) at ../../../sys/file.h:363 #22 _fdrop (fp=0xfffff800456c20a0, td=0x80) at ../../../kern/kern_descrip.c:3111 #23 0xffffffff80b7e01a in closef (fp=0xfffff800456c20a0, td=0xfffffe00f1111c00) at ../../../kern/kern_descrip.c:2606 #24 0xffffffff80b7b216 in closefp (fdp=0xfffffe00f23a1950, fd=0x3, fp=0xfffff800456c20a0, td=0xfffffe00f1111c00, holdleaders=0x0) at ../../../kern/kern_descrip.c:1263 #25 0xffffffff81024d19 in syscallenter (td=0xfffffe00f1111c00) at ../../../amd64/amd64/../../kern/subr_syscall.c:162 #26 amd64_syscall (td=0xfffffe00f1111c00, traced=0x0) at ../../../amd64/amd64/trap.c:1156 #27 #28 0x00000008004290fa in ?? () Backtrace stopped: Cannot access memory at address 0x7fffffffe618 (kgdb) f 16 #16 0xffffffff80c86846 in unp_pcb_lock_peer (unp=0xfffff80771055d00) at ../../../kern/uipc_usrreq.c:389 389 KASSERT((unp->unp_flags & UNP_CONNECTING) == 0, (kgdb) l 384 if (__predict_false(unp2 == NULL)) 385 return (NULL); 386 if (__predict_false(unp == unp2)) 387 return (unp); 388 389 KASSERT((unp->unp_flags & UNP_CONNECTING) == 0, 390 ("%s: socket %p is connecting", __func__, unp)); 391 UNP_PCB_UNLOCK_ASSERT(unp2); 392 393 if (__predict_true(UNP_PCB_TRYLOCK(unp2))) (kgdb) info loc unp2 = 0xfffff80370bc8d00 (kgdb) p *unp $1 = {unp_mtx = {lock_object = {lo_name = 0xffffffff8110f568 "unp", lo_flags = 0x1430000, lo_data = 0x0, lo_witness = 0x0}, mtx_lock = 0xfffffe00f1111c00}, unp_conn = 0xfffff80370bc8d00, unp_refcount = 0x3, unp_flags = 0x11, unp_gcflag = 0x0, unp_addr = 0x0, unp_socket = 0xfffff803f2a55000, unp_vnode = 0x0, unp_peercred = {cr_version = 0x0, cr_uid = 0x0, cr_ngroups = 0x3, cr_groups = {0x0, 0x0, 0x5, 0x0 }, {_cr_unused1 = 0x11212, cr_pid = 0x11212}}, unp_reflink = {le_next = 0x0, le_prev = 0x0}, unp_link = {le_next = 0xfffff8006283c500, le_prev = 0xfffff80370bc8db0}, unp_refs = {lh_first = 0x0}, unp_gencnt = 0xee2f15, unp_file = 0x0, unp_msgcount = 0x0, unp_gcrefs = 0x0, unp_ino = 0x771794, unp_dead = {le_next = 0x0, le_prev = 0x0}} (kgdb) (kgdb) f 18 #18 0xffffffff80c83427 in uipc_abort (so=0xfffff8074c9b8780) at ../../../kern/uipc_usrreq.c:483 483 unp_drop(unp2); (kgdb) l 478 UNP_PCB_LOCK(unp); 479 unp2 = unp->unp_conn; 480 if (unp2 != NULL) { 481 unp_pcb_hold(unp2); 482 UNP_PCB_UNLOCK(unp); 483 unp_drop(unp2); 484 } else 485 UNP_PCB_UNLOCK(unp); 486 } 487 (kgdb) info loc unp = 0xfffff80370bc8d00 unp2 = 0xfffff80771055d00 (kgdb) p *so $2 = {so_lock = {lock_object = {lo_name = 0xffffffff8111f003 "socket", lo_flags = 0x1430000, lo_data = 0x0, lo_witness = 0x0}, mtx_lock = 0x0}, so_count = 0x0, so_rdsel = {si_tdlist = {tqh_first = 0x0, tqh_last = 0x0}, si_note = {kl_list = {slh_first = 0x0}, kl_lock = 0xffffffff80c75b70 , kl_unlock = 0xffffffff80c75bb0 , kl_assert_locked = 0xffffffff80c75bf0 , kl_assert_unlocked = 0xffffffff80c75c40 , kl_lockarg = 0xfffff8074c9b8780, kl_autodestroy = 0x0}, si_mtx = 0x0}, so_wrsel = {si_tdlist = {tqh_first = 0x0, tqh_last = 0x0}, si_note = {kl_list = {slh_first = 0x0}, kl_lock = 0xffffffff80c75c80 , kl_unlock = 0xffffffff80c75cc0 , kl_assert_locked = 0xffffffff80c75d00 , kl_assert_unlocked = 0xffffffff80c75d50 , kl_lockarg = 0xfffff8074c9b8780, kl_autodestroy = 0x0}, si_mtx = 0x0}, so_type = 0x1, so_options = 0x4, so_linger = 0x0, so_state = 0x3, so_pcb = 0xfffff80370bc8d00, so_vnet = 0xfffff800037b6040, so_proto = 0xffffffff8191eee0 , so_timeo = 0x0, so_error = 0x0, so_sigio = 0x0, so_cred = 0xfffff800455b7b00, so_label = 0x0, so_gencnt = 0x17241d1, so_emuldata = 0x0, so_dtor = 0x0, osd = {osd_nslots = 0x0, osd_slots = 0x0, osd_next = {le_next = 0x0, le_prev = 0x0}}, so_fibnum = 0x0, so_user_cookie = 0x0, so_ts_clock = 0x0, so_max_pacing_rate = 0x0, {{so_rcv = {sb_mtx = {lock_object = {lo_name = 0xffffffff81193166 "so_rcv", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0x0}, mtx_lock = 0x0}, sb_sx = {lock_object = {lo_name = 0xffffffff811ebe6d "so_rcv_sx", lo_flags = 0x2330000, lo_data = 0x0, lo_witness = 0x0}, sx_lock = 0x1}, sb_sel = 0xfffff8074c9b87a8, sb_state = 0x0, sb_mb = 0x0, sb_mbtail = 0x0, sb_lastrecord = 0x0, sb_sndptr = 0x0, sb_fnrdy = 0x0, sb_sndptroff = 0x0, sb_acc = 0x0, sb_ccc = 0x0, sb_hiwat = 0x2000, sb_mbcnt = 0x0, sb_mcnt = 0x0, sb_ccnt = 0x0, sb_mbmax = 0x10000, sb_ctl = 0x0, sb_tlscc = 0x0, sb_tlsdcc = 0x0, sb_lowat = 0x1, sb_timeo = 0x0, sb_tls_seqno = 0x0, sb_tls_info = 0x0, sb_mtls = 0x0, sb_mtlstail = 0x0, sb_flags = 0x0, sb_upcall = 0x0, sb_upcallarg = 0x0, sb_aiojobq = {tqh_first = 0x0, tqh_last = 0xfffff8074c9b89c0}, sb_aiotask = {ta_link = {stqe_next = 0x0}, ta_pending = 0x0, ta_priority = 0x0, ta_flags = 0x0, ta_func = 0xffffffff80c539c0 , ta_context = 0xfffff8074c9b8780}}, so_snd = {sb_mtx = {lock_object = {lo_name = 0xffffffff811a8597 "so_snd", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0x0}, mtx_lock = 0x0}, sb_sx = {lock_object = {lo_name = 0xffffffff81218333 "so_snd_sx", lo_flags = 0x2330000, lo_data = 0x0, lo_witness = 0x0}, sx_lock = 0x1}, sb_sel = 0xfffff8074c9b87f8, sb_state = 0x0, sb_mb = 0x0, sb_mbtail = 0x0, sb_lastrecord = 0x0, sb_sndptr = 0x0, sb_fnrdy = 0x0, sb_sndptroff = 0x0, sb_acc = 0x0, sb_ccc = 0x0, sb_hiwat = 0x2000, sb_mbcnt = 0x0, sb_mcnt = 0x0, sb_ccnt = 0x0, sb_mbmax = 0x10000, sb_ctl = 0x0, sb_tlscc = 0x0, sb_tlsdcc = 0x0, sb_lowat = 0x800, sb_timeo = 0x0, sb_tls_seqno = 0x0, sb_tls_info = 0x0, sb_mtls = 0x0, sb_mtlstail = 0x0, sb_flags = 0x0, sb_upcall = 0x0, sb_upcallarg = 0x0, sb_aiojobq = {tqh_first = 0x0, tqh_last = 0xfffff8074c9b8ad8}, sb_aiotask = {ta_link = {stqe_next = 0x0}, ta_pending = 0x0, ta_priority = 0x0, ta_flags = 0x0, ta_func = 0xffffffff80c541d0 , ta_context = 0xfffff8074c9b8780}}, so_list = {tqe_next = 0x0, tqe_prev = 0xfffffe00f0ce89a8}, so_listen = 0x0, so_qstate = SQ_NONE, so_peerlabel = 0x0, so_oobmark = 0x0, so_ktls_rx_list = {stqe_next = 0x0}}, {sol_incomp = { tqh_first = 0xffffffff81193166, tqh_last = 0x1030000}, sol_comp = {tqh_first = 0x0, tqh_last = 0x0}, sol_qlen = 0x811ebe6d, sol_incqlen = 0xffffffff, sol_qlimit = 0x2330000, sol_accept_filter = 0x0, sol_accept_filter_arg = 0x1, sol_accept_filter_str = 0xfffff8074c9b87a8 "", sol_upcall = 0x0, sol_upcallarg = 0x0, sol_sbrcv_lowat = 0x0, sol_sbsnd_lowat = 0x0, sol_sbrcv_hiwat = 0x0, sol_sbsnd_hiwat = 0x0, sol_sbrcv_flags = 0x0, sol_sbsnd_flags = 0x0, sol_sbrcv_timeo = 0x0, sol_sbsnd_timeo = 0x0, sol_lastover = {tv_sec = 0x200000000000, tv_usec = 0x0}, sol_overcount = 0x0}}} (kgdb) Index: src/sys/kern/uipc_usrreq.c =================================================================== --- src/sys/kern/uipc_usrreq.c (revision 365247) +++ src/sys/kern/uipc_usrreq.c (working copy) @@ -65,13 +65,13 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include -#include /* XXX must be before */ -#include #include #include #include #include +#include #include #include #include @@ -106,9 +106,7 @@ __FBSDID("$FreeBSD$"); MALLOC_DECLARE(M_FILECAPS); /* - * Locking key: - * (l) Locked using list lock - * (g) Locked using linkage lock + * See unpcb.h for the locking key. */ static uma_zone_t unp_zone; @@ -196,41 +194,32 @@ SYSCTL_INT(_net_local, OID_AUTO, deferred, CTLFLAG /* * Locking and synchronization: * - * Three types of locks exist in the local domain socket implementation: a - * a global linkage rwlock, the mtxpool lock, and per-unpcb mutexes. - * The linkage lock protects the socket count, global generation number, - * and stream/datagram global lists. + * Several types of locks exist in the lock domain socket implementation: + * - a global linkage lock + * - a global connection list lock + * - the mtxpool lock + * - per-unpcb mutexes * - * The mtxpool lock protects the vnode from being modified while referenced. - * Lock ordering requires that it be acquired before any unpcb locks. + * The linkage lock protects the global socket lists, the generation number + * counter and garbage collector state. * - * The unpcb lock (unp_mtx) protects all fields in the unpcb. Of particular - * note is that this includes the unp_conn field. So long as the unpcb lock - * is held the reference to the unpcb pointed to by unp_conn is valid. If we - * require that the unpcb pointed to by unp_conn remain live in cases where - * we need to drop the unp_mtx as when we need to acquire the lock for a - * second unpcb the caller must first acquire an additional reference on the - * second unpcb and then revalidate any state (typically check that unp_conn - * is non-NULL) upon requiring the initial unpcb lock. The lock ordering - * between unpcbs is the conventional ascending address order. Two helper - * routines exist for this: + * The connection list lock protects the list of referring sockets in a datagram + * socket PCB. This lock is also overloaded to protect a global list of + * sockets whose buffers contain socket references in the form of SCM_RIGHTS + * messages. To avoid recursion, such references are released by a dedicated + * thread. * - * - unp_pcb_lock2(unp, unp2) - which just acquires the two locks in the - * safe ordering. + * The mtxpool lock protects the vnode from being modified while referenced. + * Lock ordering rules require that it be acquired before any PCB locks. * - * - unp_pcb_owned_lock2(unp, unp2, freed) - the lock for unp is held - * when called. If unp is unlocked and unp2 is subsequently freed - * freed will be set to 1. + * The unpcb lock (unp_mtx) protects the most commonly referenced fields in the + * unpcb. This includes the unp_conn field, which either links two connected + * PCBs together (for connected socket types) or points at the destination + * socket (for connectionless socket types). The operations of creating or + * destroying a connection therefore involve locking multiple PCBs. To avoid + * lock order reversals, in some cases this involves dropping a PCB lock and + * using a reference counter to maintain liveness. * - * The helper routines for references are: - * - * - unp_pcb_hold(unp): Can be called any time we currently hold a valid - * reference to unp. - * - * - unp_pcb_rele(unp): The caller must hold the unp lock. If we are - * releasing the last reference, detach must have been called thus - * unp->unp_socket be NULL. - * * UNIX domain sockets each have an unpcb hung off of their so_pcb pointer, * allocated in pru_attach() and freed in pru_detach(). The validity of that * pointer is an invariant, so no lock is required to dereference the so_pcb @@ -324,35 +313,46 @@ static void unp_process_defers(void * __unused, in static void unp_pcb_hold(struct unpcb *unp) { - MPASS(unp->unp_refcount); - refcount_acquire(&unp->unp_refcount); + u_int old __unused; + + old = refcount_acquire(&unp->unp_refcount); + KASSERT(old > 0, ("%s: unpcb %p has no references", __func__, unp)); } -static int +static __result_use_check bool unp_pcb_rele(struct unpcb *unp) { - int freed; + bool ret; UNP_PCB_LOCK_ASSERT(unp); - MPASS(unp->unp_refcount); - if ((freed = refcount_release(&unp->unp_refcount))) { - /* we got here with having detached? */ - MPASS(unp->unp_socket == NULL); + + if ((ret = refcount_release(&unp->unp_refcount))) { UNP_PCB_UNLOCK(unp); UNP_PCB_LOCK_DESTROY(unp); uma_zfree(unp_zone, unp); } - return (freed); + return (ret); } static void -unp_pcb_lock2(struct unpcb *unp, struct unpcb *unp2) +unp_pcb_rele_notlast(struct unpcb *unp) { - MPASS(unp != unp2); + bool ret __unused; + + ret = refcount_release(&unp->unp_refcount); + KASSERT(!ret, ("%s: unpcb %p has no references", __func__, unp)); +} + +static void +unp_pcb_lock_pair(struct unpcb *unp, struct unpcb *unp2) +{ UNP_PCB_UNLOCK_ASSERT(unp); UNP_PCB_UNLOCK_ASSERT(unp2); - if ((uintptr_t)unp2 > (uintptr_t)unp) { + + if (unp == unp2) { UNP_PCB_LOCK(unp); + } else if ((uintptr_t)unp2 > (uintptr_t)unp) { + UNP_PCB_LOCK(unp); UNP_PCB_LOCK(unp2); } else { UNP_PCB_LOCK(unp2); @@ -360,35 +360,64 @@ static void } } -static __noinline void -unp_pcb_owned_lock2_slowpath(struct unpcb *unp, struct unpcb **unp2p, - int *freed) +static void +unp_pcb_unlock_pair(struct unpcb *unp, struct unpcb *unp2) { + UNP_PCB_UNLOCK(unp); + if (unp != unp2) + UNP_PCB_UNLOCK(unp2); +} + +/* + * Try to lock the connected peer of an already locked socket. In some cases + * this requires that we unlock the current socket. The UNP_CONNECTING flag is + * used to block concurrent connection attempts while the lock is dropped. The + * caller must be careful to revalidate PCB state. + */ +static struct unpcb * +unp_pcb_lock_peer(struct unpcb *unp) +{ struct unpcb *unp2; - unp2 = *unp2p; + UNP_PCB_LOCK_ASSERT(unp); + unp2 = unp->unp_conn; + if (__predict_false(unp2 == NULL)) + return (NULL); + if (__predict_false(unp == unp2)) + return (unp); + + KASSERT((unp->unp_flags & UNP_CONNECTING) == 0, + ("%s: socket %p is connecting", __func__, unp)); + UNP_PCB_UNLOCK_ASSERT(unp2); + + if (__predict_true(UNP_PCB_TRYLOCK(unp2))) + return (unp2); + if ((uintptr_t)unp2 > (uintptr_t)unp) { + UNP_PCB_LOCK(unp2); + return (unp2); + } + unp->unp_flags |= UNP_CONNECTING; unp_pcb_hold(unp2); UNP_PCB_UNLOCK(unp); + UNP_PCB_LOCK(unp2); UNP_PCB_LOCK(unp); - *freed = unp_pcb_rele(unp2); - if (*freed) - *unp2p = NULL; + KASSERT(unp->unp_conn == unp2 || unp->unp_conn == NULL, + ("%s: socket %p was reconnected", __func__, unp)); + KASSERT((unp->unp_flags & UNP_CONNECTING) != 0, + ("%s: socket %p has UNP_CONNECTING clear", __func__, unp)); + unp->unp_flags &= ~UNP_CONNECTING; + if (unp_pcb_rele(unp2)) { + /* unp2 is unlocked. */ + return (NULL); + } + if (unp->unp_conn == NULL) { + UNP_PCB_UNLOCK(unp2); + return (NULL); + } + return (unp2); } -#define unp_pcb_owned_lock2(unp, unp2, freed) do { \ - freed = 0; \ - UNP_PCB_LOCK_ASSERT(unp); \ - UNP_PCB_UNLOCK_ASSERT(unp2); \ - MPASS((unp) != (unp2)); \ - if (__predict_true(UNP_PCB_TRYLOCK(unp2))) \ - break; \ - else if ((uintptr_t)(unp2) > (uintptr_t)(unp)) \ - UNP_PCB_LOCK(unp2); \ - else \ - unp_pcb_owned_lock2_slowpath((unp), &(unp2), &freed); \ -} while (0) - /* * Definitions of protocols supported in the LOCAL domain. */ @@ -470,18 +499,14 @@ uipc_accept(struct socket *so, struct sockaddr **n KASSERT(unp != NULL, ("uipc_accept: unp == NULL")); *nam = malloc(sizeof(struct sockaddr_un), M_SONAME, M_WAITOK); - UNP_LINK_RLOCK(); - unp2 = unp->unp_conn; - if (unp2 != NULL && unp2->unp_addr != NULL) { - UNP_PCB_LOCK(unp2); - sa = (struct sockaddr *) unp2->unp_addr; - bcopy(sa, *nam, sa->sa_len); - UNP_PCB_UNLOCK(unp2); - } else { + UNP_PCB_LOCK(unp); + unp2 = unp_pcb_lock_peer(unp); + if (unp2 != NULL && unp2->unp_addr != NULL) + sa = (struct sockaddr *)unp2->unp_addr; + else sa = &sun_noname; - bcopy(sa, *nam, sa->sa_len); - } - UNP_LINK_RUNLOCK(); + bcopy(sa, *nam, sa->sa_len); + unp_pcb_unlock_pair(unp, unp2); return (0); } @@ -525,7 +550,7 @@ uipc_attach(struct socket *so, int proto, struct t UNP_PCB_LOCK_INIT(unp); unp->unp_socket = so; so->so_pcb = unp; - unp->unp_refcount = 1; + refcount_init(&unp->unp_refcount, 1); if ((locked = UNP_LINK_WOWNED()) == false) UNP_LINK_WLOCK(); @@ -702,7 +727,7 @@ uipc_close(struct socket *so) struct unpcb *unp, *unp2; struct vnode *vp = NULL; struct mtx *vplock; - int freed; + unp = sotounpcb(so); KASSERT(unp != NULL, ("uipc_close: unp == NULL")); @@ -720,18 +745,9 @@ uipc_close(struct socket *so) VOP_UNP_DETACH(vp); unp->unp_vnode = NULL; } - unp2 = unp->unp_conn; - unp_pcb_hold(unp); - if (__predict_false(unp == unp2)) { + if ((unp2 = unp_pcb_lock_peer(unp)) != NULL) unp_disconnect(unp, unp2); - } else if (unp2 != NULL) { - unp_pcb_hold(unp2); - unp_pcb_owned_lock2(unp, unp2, freed); - unp_disconnect(unp, unp2); - if (unp_pcb_rele(unp2) == 0) - UNP_PCB_UNLOCK(unp2); - } - if (unp_pcb_rele(unp) == 0) + else UNP_PCB_UNLOCK(unp); if (vp) { mtx_unlock(vplock); @@ -749,14 +765,9 @@ uipc_connect2(struct socket *so1, struct socket *s KASSERT(unp != NULL, ("uipc_connect2: unp == NULL")); unp2 = so2->so_pcb; KASSERT(unp2 != NULL, ("uipc_connect2: unp2 == NULL")); - if (unp != unp2) - unp_pcb_lock2(unp, unp2); - else - UNP_PCB_LOCK(unp); + unp_pcb_lock_pair(unp, unp2); error = unp_connect2(so1, so2, PRU_CONNECT2); - if (unp != unp2) - UNP_PCB_UNLOCK(unp2); - UNP_PCB_UNLOCK(unp); + unp_pcb_unlock_pair(unp, unp2); return (error); } @@ -766,7 +777,7 @@ uipc_detach(struct socket *so) struct unpcb *unp, *unp2; struct mtx *vplock; struct vnode *vp; - int freeunp, local_unp_rights; + int local_unp_rights; unp = sotounpcb(so); KASSERT(unp != NULL, ("uipc_detach: unp == NULL")); @@ -773,7 +784,6 @@ uipc_detach(struct socket *so) vp = NULL; vplock = NULL; - local_unp_rights = 0; SOCK_LOCK(so); if (!SOLISTENING(so)) { @@ -812,24 +822,11 @@ uipc_detach(struct socket *so) VOP_UNP_DETACH(vp); unp->unp_vnode = NULL; } - if (__predict_false(unp == unp->unp_conn)) { - unp_disconnect(unp, unp); - unp2 = NULL; - } else { - if ((unp2 = unp->unp_conn) != NULL) { - unp_pcb_owned_lock2(unp, unp2, freeunp); - if (freeunp) - unp2 = NULL; - } - unp_pcb_hold(unp); - if (unp2 != NULL) { - unp_pcb_hold(unp2); - unp_disconnect(unp, unp2); - if (unp_pcb_rele(unp2) == 0) - UNP_PCB_UNLOCK(unp2); - } - } - UNP_PCB_UNLOCK(unp); + if ((unp2 = unp_pcb_lock_peer(unp)) != NULL) + unp_disconnect(unp, unp2); + else + UNP_PCB_UNLOCK(unp); + UNP_REF_LIST_LOCK(); while (!LIST_EMPTY(&unp->unp_refs)) { struct unpcb *ref = LIST_FIRST(&unp->unp_refs); @@ -842,11 +839,9 @@ uipc_detach(struct socket *so) unp_drop(ref); UNP_REF_LIST_LOCK(); } + UNP_REF_LIST_UNLOCK(); - UNP_REF_LIST_UNLOCK(); UNP_PCB_LOCK(unp); - freeunp = unp_pcb_rele(unp); - MPASS(freeunp == 0); local_unp_rights = unp_rights; unp->unp_socket->so_pcb = NULL; unp->unp_socket = NULL; @@ -866,30 +861,15 @@ static int uipc_disconnect(struct socket *so) { struct unpcb *unp, *unp2; - int freed; unp = sotounpcb(so); KASSERT(unp != NULL, ("uipc_disconnect: unp == NULL")); UNP_PCB_LOCK(unp); - if ((unp2 = unp->unp_conn) == NULL) { + if ((unp2 = unp_pcb_lock_peer(unp)) != NULL) + unp_disconnect(unp, unp2); + else UNP_PCB_UNLOCK(unp); - return (0); - } - if (__predict_true(unp != unp2)) { - unp_pcb_owned_lock2(unp, unp2, freed); - if (__predict_false(freed)) { - UNP_PCB_UNLOCK(unp); - return (0); - } - unp_pcb_hold(unp2); - } - unp_pcb_hold(unp); - unp_disconnect(unp, unp2); - if (unp_pcb_rele(unp) == 0) - UNP_PCB_UNLOCK(unp); - if ((unp != unp2) && unp_pcb_rele(unp2) == 0) - UNP_PCB_UNLOCK(unp2); return (0); } @@ -1008,27 +988,6 @@ uipc_rcvd(struct socket *so, int flags) } static int -connect_internal(struct socket *so, struct sockaddr *nam, struct thread *td) -{ - int error; - struct unpcb *unp; - - unp = so->so_pcb; - if (unp->unp_conn != NULL) - return (EISCONN); - error = unp_connect(so, nam, td); - if (error) - return (error); - UNP_PCB_LOCK(unp); - if (unp->unp_conn == NULL) { - UNP_PCB_UNLOCK(unp); - if (error == 0) - error = ENOTCONN; - } - return (error); -} - -static int uipc_send(struct socket *so, int flags, struct mbuf *m, struct sockaddr *nam, struct mbuf *control, struct thread *td) { @@ -1058,57 +1017,26 @@ uipc_send(struct socket *so, int flags, struct mbu const struct sockaddr *from; if (nam != NULL) { - /* - * We return with UNP_PCB_LOCK_HELD so we know that - * the reference is live if the pointer is valid. - */ - if ((error = connect_internal(so, nam, td))) + error = unp_connect(so, nam, td); + if (error != 0) break; - MPASS(unp->unp_conn != NULL); - unp2 = unp->unp_conn; - } else { - UNP_PCB_LOCK(unp); + } + UNP_PCB_LOCK(unp); - /* - * Because connect() and send() are non-atomic in a sendto() - * with a target address, it's possible that the socket will - * have disconnected before the send() can run. In that case - * return the slightly counter-intuitive but otherwise - * correct error that the socket is not connected. - */ - if ((unp2 = unp->unp_conn) == NULL) { - UNP_PCB_UNLOCK(unp); - error = ENOTCONN; - break; - } - } - if (__predict_false(unp == unp2)) { - if (unp->unp_socket == NULL) { - error = ENOTCONN; - break; - } - goto connect_self; - } - unp_pcb_owned_lock2(unp, unp2, freed); - if (__predict_false(freed)) { - UNP_PCB_UNLOCK(unp); - error = ENOTCONN; - break; - } /* - * The socket referencing unp2 may have been closed - * or unp may have been disconnected if the unp lock - * was dropped to acquire unp2. + * Because connect() and send() are non-atomic in a sendto() + * with a target address, it's possible that the socket will + * have disconnected before the send() can run. In that case + * return the slightly counter-intuitive but otherwise + * correct error that the socket is not connected. */ - if (__predict_false(unp->unp_conn == NULL) || - unp2->unp_socket == NULL) { + unp2 = unp_pcb_lock_peer(unp); + if (unp2 == NULL) { UNP_PCB_UNLOCK(unp); - if (unp_pcb_rele(unp2) == 0) - UNP_PCB_UNLOCK(unp2); error = ENOTCONN; break; } - connect_self: + if (unp2->unp_flags & UNP_WANTCRED) control = unp_addsockcred(td, control); if (unp->unp_addr != NULL) @@ -1128,9 +1056,8 @@ uipc_send(struct socket *so, int flags, struct mbu } if (nam != NULL) unp_disconnect(unp, unp2); - if (__predict_true(unp != unp2)) - UNP_PCB_UNLOCK(unp2); - UNP_PCB_UNLOCK(unp); + else + unp_pcb_unlock_pair(unp, unp2); break; } @@ -1138,7 +1065,7 @@ uipc_send(struct socket *so, int flags, struct mbu case SOCK_STREAM: if ((so->so_state & SS_ISCONNECTED) == 0) { if (nam != NULL) { - error = connect_internal(so, nam, td); + error = unp_connect(so, nam, td); if (error != 0) break; } else { @@ -1145,29 +1072,19 @@ uipc_send(struct socket *so, int flags, struct mbu error = ENOTCONN; break; } - } else { - UNP_PCB_LOCK(unp); } - if ((unp2 = unp->unp_conn) == NULL) { + UNP_PCB_LOCK(unp); + if ((unp2 = unp_pcb_lock_peer(unp)) == NULL) { UNP_PCB_UNLOCK(unp); error = ENOTCONN; break; } else if (so->so_snd.sb_state & SBS_CANTSENDMORE) { - UNP_PCB_UNLOCK(unp); + unp_pcb_unlock_pair(unp, unp2); error = EPIPE; break; - } else if ((unp2 = unp->unp_conn) == NULL) { - UNP_PCB_UNLOCK(unp); - error = ENOTCONN; - break; } - unp_pcb_owned_lock2(unp, unp2, freed); UNP_PCB_UNLOCK(unp); - if (__predict_false(freed)) { - error = ENOTCONN; - break; - } if ((so2 = unp2->unp_socket) == NULL) { UNP_PCB_UNLOCK(unp2); error = ENOTCONN; @@ -1302,30 +1219,19 @@ uipc_ready(struct socket *so, struct mbuf *m, int ("%s: unexpected socket type for %p", __func__, so)); UNP_PCB_LOCK(unp); - if ((unp2 = unp->unp_conn) == NULL) { + if ((unp2 = unp_pcb_lock_peer(unp)) != NULL) { UNP_PCB_UNLOCK(unp); - goto search; + so2 = unp2->unp_socket; + SOCKBUF_LOCK(&so2->so_rcv); + if ((error = sbready(&so2->so_rcv, m, count)) == 0) + sorwakeup_locked(so2); + else + SOCKBUF_UNLOCK(&so2->so_rcv); + UNP_PCB_UNLOCK(unp2); + return (error); } - if (unp != unp2) { - if (UNP_PCB_TRYLOCK(unp2) == 0) { - unp_pcb_hold(unp2); - UNP_PCB_UNLOCK(unp); - UNP_PCB_LOCK(unp2); - if (unp_pcb_rele(unp2)) - goto search; - } else - UNP_PCB_UNLOCK(unp); - } - so2 = unp2->unp_socket; - SOCKBUF_LOCK(&so2->so_rcv); - if ((error = sbready(&so2->so_rcv, m, count)) == 0) - sorwakeup_locked(so2); - else - SOCKBUF_UNLOCK(&so2->so_rcv); - UNP_PCB_UNLOCK(unp2); - return (error); + UNP_PCB_UNLOCK(unp); -search: /* * The receiving socket has been disconnected, but may still be valid. * In this case, the now-ready mbufs are still present in its socket @@ -1568,7 +1474,8 @@ static int unp_connectat(int fd, struct socket *so, struct sockaddr *nam, struct thread *td) { - struct sockaddr_un *soun = (struct sockaddr_un *)nam; + struct mtx *vplock; + struct sockaddr_un *soun; struct vnode *vp; struct socket *so2; struct unpcb *unp, *unp2, *unp3; @@ -1576,8 +1483,8 @@ unp_connectat(int fd, struct socket *so, struct so char buf[SOCK_MAXADDRLEN]; struct sockaddr *sa; cap_rights_t rights; - int error, len, freed; - struct mtx *vplock; + int error, len; + bool connreq; if (nam->sa_family != AF_UNIX) return (EAFNOSUPPORT); @@ -1586,19 +1493,28 @@ unp_connectat(int fd, struct socket *so, struct so len = nam->sa_len - offsetof(struct sockaddr_un, sun_path); if (len <= 0) return (EINVAL); + soun = (struct sockaddr_un *)nam; bcopy(soun->sun_path, buf, len); buf[len] = 0; unp = sotounpcb(so); UNP_PCB_LOCK(unp); - if (unp->unp_flags & UNP_CONNECTING) { + if (unp->unp_conn != NULL) { UNP_PCB_UNLOCK(unp); + return (EISCONN); + } + if ((unp->unp_flags & UNP_CONNECTING) != 0) { + UNP_PCB_UNLOCK(unp); return (EALREADY); } unp->unp_flags |= UNP_CONNECTING; UNP_PCB_UNLOCK(unp); - sa = malloc(sizeof(struct sockaddr_un), M_SONAME, M_WAITOK); + connreq = (so->so_proto->pr_flags & PR_CONNREQUIRED) != 0; + if (connreq) + sa = malloc(sizeof(struct sockaddr_un), M_SONAME, M_WAITOK); + else + sa = NULL; NDINIT_ATRIGHTS(&nd, LOOKUP, FOLLOW | LOCKSHARED | LOCKLEAF, UIO_SYSSPACE, buf, fd, cap_rights_init(&rights, CAP_CONNECTAT), td); error = namei(&nd); @@ -1639,7 +1555,7 @@ unp_connectat(int fd, struct socket *so, struct so error = EPROTOTYPE; goto bad2; } - if (so->so_proto->pr_flags & PR_CONNREQUIRED) { + if (connreq) { if (so2->so_options & SO_ACCEPTCONN) { CURVNET_SET(so2->so_vnet); so2 = sonewconn(so2, 0); @@ -1651,7 +1567,7 @@ unp_connectat(int fd, struct socket *so, struct so goto bad2; } unp3 = sotounpcb(so2); - unp_pcb_lock2(unp2, unp3); + unp_pcb_lock_pair(unp2, unp3); if (unp2->unp_addr != NULL) { bcopy(unp2->unp_addr, sa, unp2->unp_addr->sun_len); unp3->unp_addr = (struct sockaddr_un *) sa; @@ -1662,29 +1578,24 @@ unp_connectat(int fd, struct socket *so, struct so UNP_PCB_UNLOCK(unp2); unp2 = unp3; - unp_pcb_owned_lock2(unp2, unp, freed); - if (__predict_false(freed)) { - UNP_PCB_UNLOCK(unp2); - error = ECONNREFUSED; - goto bad2; - } + + /* + * It is safe to block on the PCB lock here since unp2 is + * nascent and cannot be connected to any other sockets. + */ + UNP_PCB_LOCK(unp); #ifdef MAC mac_socketpeer_set_from_socket(so, so2); mac_socketpeer_set_from_socket(so2, so); #endif } else { - if (unp == unp2) - UNP_PCB_LOCK(unp); - else - unp_pcb_lock2(unp, unp2); + unp_pcb_lock_pair(unp, unp2); } KASSERT(unp2 != NULL && so2 != NULL && unp2->unp_socket == so2 && sotounpcb(so2) == unp2, ("%s: unp2 %p so2 %p", __func__, unp2, so2)); error = unp_connect2(so, so2, PRU_CONNECT); - if (unp != unp2) - UNP_PCB_UNLOCK(unp2); - UNP_PCB_UNLOCK(unp); + unp_pcb_unlock_pair(unp, unp2); bad2: mtx_unlock(vplock); bad: @@ -1693,6 +1604,8 @@ bad: } free(sa, M_SONAME); UNP_PCB_LOCK(unp); + KASSERT((unp->unp_flags & UNP_CONNECTING) != 0, + ("%s: unp %p has UNP_CONNECTING clear", __func__, unp)); unp->unp_flags &= ~UNP_CONNECTING; UNP_PCB_UNLOCK(unp); return (error); @@ -1732,6 +1645,8 @@ unp_connect2(struct socket *so, struct socket *so2 UNP_PCB_LOCK_ASSERT(unp); UNP_PCB_LOCK_ASSERT(unp2); + KASSERT(unp->unp_conn == NULL, + ("%s: socket %p is already connected", __func__, unp)); if (so2->so_type != so->so_type) return (EPROTOTYPE); @@ -1748,6 +1663,8 @@ unp_connect2(struct socket *so, struct socket *so2 case SOCK_STREAM: case SOCK_SEQPACKET: + KASSERT(unp2->unp_conn == NULL, + ("%s: socket %p is already connected", __func__, unp2)); unp2->unp_conn = unp; if (req == PRU_CONNECT && ((unp->unp_flags | unp2->unp_flags) & UNP_CONNWAIT)) @@ -1767,17 +1684,15 @@ static void unp_disconnect(struct unpcb *unp, struct unpcb *unp2) { struct socket *so, *so2; - int freed __unused; +#ifdef INVARIANTS + struct unpcb *unptmp; +#endif - KASSERT(unp2 != NULL, ("unp_disconnect: unp2 == NULL")); - UNP_PCB_LOCK_ASSERT(unp); UNP_PCB_LOCK_ASSERT(unp2); + KASSERT(unp->unp_conn == unp2, + ("%s: unpcb %p is not connected to %p", __func__, unp, unp2)); - if (unp->unp_conn == NULL && unp2->unp_conn == NULL) - return; - - MPASS(unp->unp_conn == unp2); unp->unp_conn = NULL; so = unp->unp_socket; so2 = unp2->unp_socket; @@ -1784,6 +1699,14 @@ unp_disconnect(struct unpcb *unp, struct unpcb *un switch (unp->unp_socket->so_type) { case SOCK_DGRAM: UNP_REF_LIST_LOCK(); +#ifdef INVARIANTS + LIST_FOREACH(unptmp, &unp2->unp_refs, unp_reflink) { + if (unptmp == unp) + break; + } + KASSERT(unptmp != NULL, + ("%s: %p not found in reflist of %p", __func__, unp, unp2)); +#endif LIST_REMOVE(unp, unp_reflink); UNP_REF_LIST_UNLOCK(); if (so) { @@ -1803,10 +1726,17 @@ unp_disconnect(struct unpcb *unp, struct unpcb *un soisdisconnected(so2); break; } - freed = unp_pcb_rele(unp); - MPASS(freed == 0); - freed = unp_pcb_rele(unp2); - MPASS(freed == 0); + + if (unp == unp2) { + unp_pcb_rele_notlast(unp); + if (!unp_pcb_rele(unp)) + UNP_PCB_UNLOCK(unp); + } else { + if (!unp_pcb_rele(unp)) + UNP_PCB_UNLOCK(unp); + if (!unp_pcb_rele(unp2)) + UNP_PCB_UNLOCK(unp2); + } } /* @@ -1825,7 +1755,7 @@ unp_pcblist(SYSCTL_HANDLER_ARGS) struct unp_head *head; struct xunpcb *xu; u_int i; - int error, freeunp, n; + int error, n; switch ((intptr_t)arg1) { case SOCK_STREAM: @@ -1902,9 +1832,10 @@ unp_pcblist(SYSCTL_HANDLER_ARGS) for (i = 0; i < n; i++) { unp = unp_list[i]; UNP_PCB_LOCK(unp); - freeunp = unp_pcb_rele(unp); + if (unp_pcb_rele(unp)) + continue; - if (freeunp == 0 && unp->unp_gencnt <= gencnt) { + if (unp->unp_gencnt <= gencnt) { xu->xu_len = sizeof *xu; xu->xu_unpp = (uintptr_t)unp; /* @@ -1931,8 +1862,9 @@ unp_pcblist(SYSCTL_HANDLER_ARGS) sotoxsocket(unp->unp_socket, &xu->xu_socket); UNP_PCB_UNLOCK(unp); error = SYSCTL_OUT(req, xu, sizeof *xu); - } else if (freeunp == 0) + } else { UNP_PCB_UNLOCK(unp); + } } free(xu, M_TEMP); if (!error) { @@ -1987,7 +1919,6 @@ unp_drop(struct unpcb *unp) { struct socket *so = unp->unp_socket; struct unpcb *unp2; - int freed; /* * Regardless of whether the socket's peer dropped the connection @@ -1994,23 +1925,17 @@ unp_drop(struct unpcb *unp) * with this socket by aborting or disconnecting, POSIX requires * that ECONNRESET is returned. */ - /* acquire a reference so that unp isn't freed from underneath us */ UNP_PCB_LOCK(unp); if (so) so->so_error = ECONNRESET; - unp2 = unp->unp_conn; - if (unp2 == unp) { + if ((unp2 = unp_pcb_lock_peer(unp)) != NULL) { + /* Last reference dropped in unp_disconnect(). */ + unp_pcb_rele_notlast(unp); unp_disconnect(unp, unp2); - } else if (unp2 != NULL) { - unp_pcb_hold(unp2); - unp_pcb_owned_lock2(unp, unp2, freed); - unp_disconnect(unp, unp2); - if (unp_pcb_rele(unp2) == 0) - UNP_PCB_UNLOCK(unp2); + } else if (!unp_pcb_rele(unp)) { + UNP_PCB_UNLOCK(unp); } - if (unp_pcb_rele(unp) == 0) - UNP_PCB_UNLOCK(unp); } static void @@ -2148,18 +2073,44 @@ unp_zone_change(void *tag) uma_zone_set_max(unp_zone, maxsockets); } +#ifdef INVARIANTS static void +unp_zdtor(void *mem, int size __unused, void *arg __unused) +{ + struct unpcb *unp; + + unp = mem; + + KASSERT(LIST_EMPTY(&unp->unp_refs), + ("%s: unpcb %p has lingering refs", __func__, unp)); + KASSERT(unp->unp_socket == NULL, + ("%s: unpcb %p has socket backpointer", __func__, unp)); + KASSERT(unp->unp_vnode == NULL, + ("%s: unpcb %p has vnode references", __func__, unp)); + KASSERT(unp->unp_conn == NULL, + ("%s: unpcb %p is still connected", __func__, unp)); + KASSERT(unp->unp_addr == NULL, + ("%s: unpcb %p has leaked addr", __func__, unp)); +} +#endif + +static void unp_init(void) { + uma_dtor dtor; #ifdef VIMAGE if (!IS_DEFAULT_VNET(curvnet)) return; #endif - unp_zone = uma_zcreate("unpcb", sizeof(struct unpcb), NULL, NULL, + +#ifdef INVARIANTS + dtor = unp_zdtor; +#else + dtor = NULL; +#endif + unp_zone = uma_zcreate("unpcb", sizeof(struct unpcb), NULL, dtor, NULL, NULL, UMA_ALIGN_CACHE, 0); - if (unp_zone == NULL) - panic("unp_init"); uma_zone_set_max(unp_zone, maxsockets); uma_zone_set_warning(unp_zone, "kern.ipc.maxsockets limit reached"); EVENTHANDLER_REGISTER(maxsockets_change, unp_zone_change, Index: src/sys/sys/unpcb.h =================================================================== --- src/sys/sys/unpcb.h (revision 365247) +++ src/sys/sys/unpcb.h (working copy) @@ -65,30 +65,37 @@ typedef uint64_t unp_gen_t; * Stream sockets keep copies of receive sockbuf sb_cc and sb_mbcnt * so that changes in the sockbuf may be computed to modify * back pressure on the sender accordingly. + * + * Locking key: + * (a) Atomic + * (c) Constant + * (g) Locked using linkage lock + * (l) Locked using list lock + * (p) Locked using pcb lock */ LIST_HEAD(unp_head, unpcb); struct unpcb { /* Cache line 1 */ - struct mtx unp_mtx; /* mutex */ - struct unpcb *unp_conn; /* control block of connected socket */ - volatile u_int unp_refcount; - short unp_flags; /* flags */ - short unp_gcflag; /* Garbage collector flags. */ - struct sockaddr_un *unp_addr; /* bound address of socket */ - struct socket *unp_socket; /* pointer back to socket */ + struct mtx unp_mtx; /* PCB mutex */ + struct unpcb *unp_conn; /* (p) connected socket */ + volatile u_int unp_refcount; /* (a, p) atomic refcount */ + short unp_flags; /* (p) PCB flags */ + short unp_gcflag; /* (g) Garbage collector flags */ + struct sockaddr_un *unp_addr; /* (p) bound address of socket */ + struct socket *unp_socket; /* (c) pointer back to socket */ /* Cache line 2 */ - struct vnode *unp_vnode; /* if associated with file */ - struct xucred unp_peercred; /* peer credentials, if applicable */ - LIST_ENTRY(unpcb) unp_reflink; /* link in unp_refs list */ - LIST_ENTRY(unpcb) unp_link; /* glue on list of all PCBs */ - struct unp_head unp_refs; /* referencing socket linked list */ - unp_gen_t unp_gencnt; /* generation count of this instance */ - struct file *unp_file; /* back-pointer to file for gc. */ - u_int unp_msgcount; /* references from message queue */ - u_int unp_gcrefs; /* garbage collector refcount */ - ino_t unp_ino; /* fake inode number */ - LIST_ENTRY(unpcb) unp_dead; /* link in dead list */ + struct vnode *unp_vnode; /* (p) associated file if applicable */ + struct xucred unp_peercred; /* (p) peer credentials if applicable */ + LIST_ENTRY(unpcb) unp_reflink; /* (l) link in unp_refs list */ + LIST_ENTRY(unpcb) unp_link; /* (g) glue on list of all PCBs */ + struct unp_head unp_refs; /* (l) referencing socket linked list */ + unp_gen_t unp_gencnt; /* (g) generation count of this item */ + struct file *unp_file; /* (g) back-pointer to file for gc */ + u_int unp_msgcount; /* (g) references from message queue */ + u_int unp_gcrefs; /* (g) garbage collector refcount */ + ino_t unp_ino; /* (g) fake inode number */ + LIST_ENTRY(unpcb) unp_dead; /* (g) link in dead list */ } __aligned(CACHE_LINE_SIZE); /* -- Test scenario: misc/unix_socket_detach.sh