Spinning thread seen with syscall fuzz, vmcore.75 KDB: debugger backends: ddb KDB: current backend: ddb ---<>--- Copyright (c) 1992-2019 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 13.0-CURRENT #0 r351575M: Wed Aug 28 21:25:51 CEST 2019 pho@mercat1.netperf.freebsd.org:/usr/src/sys/amd64/compile/PHO amd64 FreeBSD clang version 8.0.1 (tags/RELEASE_801/final 366581) (based on LLVM 8.0.1) WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. Entering uma_startup with 10 boot pages configured startup_alloc from "UMA Kegs", 8 boot pages left startup_alloc from "UMA Zones", 7 boot pages left startup_alloc from "UMA Zones", 6 boot pages left startup_alloc from "UMA Zones", 5 boot pages left startup_alloc from "UMA Zones", 4 boot pages left startup_alloc from "UMA Zones", 3 boot pages left startup_alloc from "UMA Hash", 2 boot pages left startup_alloc from "UMA Zones", 1 boot pages left Entering uma_startup1 with 0 boot pages left Entering uma_startup2 with 0 boot pages left VT(vga): resolution 640x480 CPU: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz (3500.07-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306f2 Family=0x6 Model=0x3f Stepping=2 Features=0xbfebfbff Features2=0x7ffefbff AMD Features=0x2c100800 AMD Features2=0x21 Structured Extended Features=0x37ab XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics real memory = 34359738368 (32768 MB) avail memory = 33160597504 (31624 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs FreeBSD/SMP: 1 package(s) x 6 core(s) x 2 hardware threads random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" random: unblocking device. ioapic0 irqs 0-23 ioapic1 irqs 24-47 Launching APs: 1 4 5 2 6 10 9 8 11 3 7 Timecounter "TSC-low" frequency 1750035198 Hz quality 1000 random: entropy device external interface kbd1 at kbdmux0 000.000054 [4254] netmap_init netmap: loaded module module_register_init: MOD_LOAD (vesa, 0xffffffff811c1ab0, 0) error 19 [ath_hal] loaded nexus0 vtvga0: cryptosoft0: acpi0: acpi0: Power Button (fixed) cpu0: on acpi0 atrtc0: port 0x70-0x71,0x74-0x77 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Event timer "HPET4" frequency 14318180 Hz quality 340 Event timer "HPET5" frequency 14318180 Hz quality 340 Event timer "HPET6" frequency 14318180 Hz quality 340 Event timer "HPET7" frequency 14318180 Hz quality 340 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: on acpi0 pci0: on pcib0 pci0: at device 11.1 (no driver attached) pci0: at device 11.2 (no driver attached) pci0: at device 16.1 (no driver attached) pci0: at device 16.6 (no driver attached) pci0: at device 18.1 (no driver attached) acpi_syscontainer0: on acpi0 acpi_syscontainer1: on acpi0 acpi_syscontainer2: on acpi0 acpi_syscontainer3: on acpi0 pcib1: port 0xcf8-0xcff on acpi0 pci1: on pcib1 pcib2: irq 26 at device 1.0 on pci1 pci2: on pcib2 pcib3: irq 32 at device 2.0 on pci1 pci3: on pcib3 pci3: at device 0.0 (no driver attached) pci3: at device 0.1 (no driver attached) pcib4: irq 40 at device 3.0 on pci1 pci4: on pcib4 pci1: at device 17.0 (no driver attached) ahci0: port 0xf110-0xf117,0xf100-0xf103,0xf0f0-0xf0f7,0xf0e0-0xf0e3,0xf020-0xf03f mem 0xfb438000-0xfb4387ff irq 16 at device 17.4 on pci1 ahci0: AHCI v1.30 with 4 6Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahciem0: at channel 2147483647 on ahci0 device_attach: ahciem0 attach returned 6 xhci0: mem 0xfb400000-0xfb40ffff irq 19 at device 20.0 on pci1 xhci0: 32 bytes context size, 64-bit DMA usbus0 on xhci0 usbus0: 5.0Gbps Super Speed USB v3.0 pci1: at device 22.0 (no driver attached) pci1: at device 22.1 (no driver attached) ehci0: mem 0xfb434000-0xfb4343ff irq 18 at device 26.0 on pci1 usbus1: EHCI version 1.0 usbus1 on ehci0 usbus1: 480Mbps High Speed USB v2.0 pcib5: irq 16 at device 28.0 on pci1 pci5: on pcib5 pcib6: irq 18 at device 28.2 on pci1 pci6: on pcib6 pcib7: at device 0.0 on pci6 pci7: on pcib7 vgapci0: port 0xe000-0xe07f mem 0xfa000000-0xfaffffff,0xfb000000-0xfb01ffff irq 18 at device 0.0 on pci7 vgapci0: Boot video device pcib8: irq 16 at device 28.4 on pci1 pci8: on pcib8 igb0: port 0xd020-0xd03f mem 0xfb320000-0xfb33ffff,0xfb344000-0xfb347fff irq 16 at device 0.0 on pci8 igb0: Using 1024 TX descriptors and 1024 RX descriptors igb0: queue equality override not set, capping rx_queues at 6 and tx_queues at 6 igb0: Using 6 RX queues 6 TX queues igb0: Using MSI-X interrupts with 7 vectors igb0:ugen0.3: at usbus0 ukbd0 on uhub3 ukbd0: on usbus0 kbd2 at ukbd0 . lo0: link state changed to UP ions=e507bb ether 0c:c4:7a:a8:cd:db media: Ethernet autoselect status: no carrier nd6 options=29 Starting devd. igb0: link state changed to UP Autoloading module: ioat.ko ioat0: mem 0xfb42c000-0xfb42ffff irq 31 at device 4.0 on pci1 ioat0: Capabilities: 2f7 ioat1: mem 0xfb428000-0xfb42bfff irq 39 at device 4.1 on pci1 ioat1: Capabilities: 2f7 ioat2: mem 0xfb424000-0xfb427fff irq 31 at device 4.2 on pci1 ioat2: Capabilities: f7 ioat3: mem 0xfb420000-0xfb423fff irq 39 at device 4.3 on pci1 ioat3: Capabilities: f7 ioat4: mem 0xfb41c000-0xfb41ffff irq 31 at device 4.4 on pci1 ioat4: Capabilities: f7 ioat5: mem 0xfb418000-0xfb41bfff irq 39 at device 4.5 on pci1 ioat5: Capabilities: f7 ioat6: mem 0xfb414000-0xfb417fff irq 31 at device 4.6 on pci1 ioat6: Capabilities: f7 ioat7: mem 0xfb410000-0xfb413fff irq 39 at device 4.7 on pci1 ioat7: Capabilities: f7 Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Starting Network: igb1. igb1: flags=8802 metric 0 mtu 1500 options=e507bb ether 0c:c4:7a:a8:cd:db media: Ethernet autoselect status: no carrier nd6 options=29 Autoloading module: uhid.ko Autoloading module: ums.ko ums0 on uhub3 ums0: on usbus0 ums0: 3 buttons and [Z] coordinates ID=0 Starting dhclient. DHCPREQUEST on igb0 to 255.255.255.255 port 67 DHCPACK from 192if_delmulti_locked: detaching ifnet instance 0xfffff800060e3000 .168.5.1 bound to 192.168.5.171 -- renewal in 300 seconds. add host 127.0.0.1: gateway lo0 fib 0: route already in table add host ::1: gateway lo0 fib 0: route already in table add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Creating and/or trimming log files. Starting syslogd. Setting date via ntp. 28 Aug 21:31:13 ntpdate[2053]: step time server 192.168.5.1 offset -0.536857 sec Starting rpcbind. NFS access cache time=60 Starting watchdogd. No core dumps found. Clearing /tmp (X related). Starting statd. Starting lockd. fscklock order reversal: 1st 0xfffff80003bab068 ufs (ufs) @ kern/vfs_mount.c:1354 2nd 0xfffff80007a822e0 devfs (devfs) @ ufs/ffs/ffs_softdep.c:1926 stack backtrace: #0 0xffffffff80c424b3 at witness_debugger+0x73 #1 0xffffffff80c421fe at witness_checkorder+0xa9e #2 0xffffffff80ba8249 at lockmgr_lock_fast_path+0x179 #3 0xffffffff81237a40 at VOP_LOCK1_APV+0x70 #4 0xffffffff80cbfe95 at _vn_lock+0x65 #5 0xffffffff80eca42d at softdep_flushworklist+0x5d #6 0xffffffff80eeb4e6 at ffs_sync+0x406 #7 0xffffffff80ca5cd1 at dounmount+0x4f1 #8 0xffffffff80ca5729 at sys_unmount+0x339 #9 0xffffffff8119a254 at amd64_syscall+0x2d4 #10 0xffffffff811715a0 at fast_syscall_common+0x101 ** /dev/gpt/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 299 files, 5465 used, 1259942 free (134 frags, 157476 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** Updating motd:. Mounting late filesystems:. Security policy loaded: MAC/ntpd (mac_ntpd) Starting ntpd. Configuring vt: blanktime. Performing sanity check on sshd configuration. Starting sshd. Starting cron. Wed Aug 28 21:31 FreeBSD/amd64 (mercat1.netperf.freebsd.org) (ttyu1) login: root Aug 28 21:31:54 mercat1 login[2335]: ROOT LOGIN (root) ON ttyu1 Last login: Tue Aug 27 22:39:25 on ttyu1 FreeBSD 13.0-CURRENT (PHO) #0 r351575M: Wed Aug 28 21:25:51 CEST 2019 root@mercat1:~ # Aug 28 21:32:01 mercat1 su[2349]: pho to root on /dev/pts/0 20190828 21:32:19 all (1/668): 1st.sh 20190828 21:32:21 all (2/668): 256m.sh 20190828 21:32:22 all (3/668): audit.sh 20190828 21:32:24 all (4/668): beneath2.sh 20190828 21:32:27 all (5/668): beneath3.sh 20190828 21:32:29 all (6/668): buildworld.sh 20190828 21:32:30 all (7/668): buildworld2.sh 20190828 21:32:32 all (8/668): cluster.sh 20190828 21:32:33 all (9/668): cpuset.sh 20190828 21:32:34 all (10/668): crossmp2.sh 20190828 21:32:36 all (11/668): crossmp6.sh 20190828 21:32:37 all (12/668): datagram2.sh 20190828 21:32:39 all (13/668): datagram3.sh 20190828 21:32:41 all (14/668): datamove.sh freework leak: 32/41. newblk leak: 7/15. 20190828 21:32:42 all (15/668): datamove3.sh newblk leak: 1/16. 20190828 21:32:44 all (16/668): devfs2.sh 20190828 21:32:45 all (17/668): devfs5.sh 20190828 21:32:46 all (18/668): dtrace_fault.sh 20190828 21:32:48 all (19/668): dumpfs.sh newblk leak: 296/312. 20190828 21:32:50 all (20/668): dup2.sh 20190828 21:32:51 all (21/668): execi386.sh 20190828 21:32:53 all (22/668): extattr_set_fd.sh 20190828 21:32:54 all (23/668): extattrctl.sh lock order reversal: 1st 0xfffff800075e3238 ufs_extattr_sx (ufs_extattr_sx) @ ufs/ufs/ufs_extattr.c:119 2nd 0xfffff8015918c558 ufs (ufs) @ ufs/ffs/ffs_vfsops.c:1709 stack backtrace: #0 0xffffffff80c424b3 at witness_debugger+0x73 #1 0xffffffff80c421fe at witness_checkorder+0xa9e #2 0xffffffff80ba885d at lockmgr_xlock_hard+0x6d #3 0xffffffff80ba9415 at __lockmgr_args+0x605 #4 0xffffffff80ee8643 at ffs_vgetf+0x133 #5 0xffffffff80efb3ab at ufs_root+0x1b #6 0xffffffff80ef37af at ufs_extattr_autostart+0x5f #7 0xffffffff80eea9d6 at ffs_mount+0x1c76 #8 0xffffffff80ca4908 at vfs_domount+0x748 #9 0xffffffff80ca3bce at vfs_donmount+0x7be #10 0xffffffff80ca33d9 at sys_nmount+0x69 #11 0xffffffff8119a254 at amd64_syscall+0x2d4 #12 0xffffffff811715a0 at fast_syscall_common+0x101 20190828 21:32:56 all (24/668): fifo.sh 20190828 21:32:57 all (25/668): forkbomb.sh 20190828 21:32:59 all (26/668): fsck2.sh 20190828 21:33:00 all (27/668): fsck3.sh 20190828 21:33:02 all (28/668): fsck4.sh 20190828 21:33:03 all (29/668): geomleak.sh 20190828 21:33:05 all (30/668): getrandom.sh 20190828 21:33:06 all (31/668): gnop3.sh 20190828 21:33:08 all (32/668): gnop5.sh GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. 20190828 21:33:09 all (33/668): holdcnt0.sh 20190828 21:33:11 all (34/668): holdcnt02.sh 20190828 21:33:12 all (35/668): holdcnt03.sh 20190828 21:33:13 all (36/668): holdcnt04.sh 20190828 21:33:15 all (37/668): isofs2.sh 20190828 21:33:16 all (38/668): jail.sh 20190828 21:33:18 all (39/668): jail2.sh 20190828 21:33:19 all (40/668): jail3.sh 20190828 21:33:21 all (41/668): jail4.sh 20190828 21:33:22 all (42/668): kern_umtx_inf_loop.sh 20190828 21:33:24 all (43/668): kevent10.sh 20190828 21:33:25 all (44/668): kevent3.sh 20190828 21:33:27 all (45/668): ldt.sh 20190828 21:33:28 all (46/668): ldt2.sh 20190828 21:33:30 all (47/668): libMicro.sh 20190828 21:33:31 all (48/668): linux.sh 20190828 21:33:32 all (49/668): lockd.sh 20190828 21:33:34 all (50/668): lockf.sh 20190828 21:33:35 all (51/668): lookup_shared.sh 20190828 21:33:37 all (52/668): mac.sh 20190828 21:33:38 all (53/668): mac_chkexec.sh 20190828 21:33:39 all (54/668): machipc.sh 20190828 21:33:41 all (55/668): machipc2.sh 20190828 21:33:42 all (56/668): maxmemdom.sh 20190828 21:33:44 all (57/668): maxproc.sh 20190828 21:33:45 all (58/668): md4.sh 20190828 21:33:47 all (59/668): md5.sh 20190828 21:33:48 all (60/668): md6.sh 20190828 21:33:50 all (61/668): mdconfig.sh 20190828 21:33:51 all (62/668): mdconfig3.sh 20190828 21:33:53 all (63/668): memguard4.sh 20190828 21:33:54 all (64/668): mkfifo5.sh 20190828 21:33:55 all (65/668): mkfifo7.sh mount leak: 16/47 20190828 21:33:57 all (66/668): mlockall5.sh 20190828 21:33:58 all (67/668): mmap.sh 20190828 21:34:00 all (68/668): mmap12.sh 20190828 21:34:01 all (69/668): mmap16.sh 20190828 21:34:03 all (70/668): mmap17.sh 20190828 21:34:04 all (71/668): mmap20.sh 20190828 21:34:06 all (72/668): mmap24.sh 20190828 21:34:07 all (73/668): mmap30.sh 20190828 21:34:09 all (74/668): mmap33.sh 20190828 21:34:10 all (75/668): mmap35.sh 20190828 21:34:11 all (76/668): mmap36.sh 20190828 21:34:13 all (77/668): mount.sh 20190828 21:34:14 all (78/668): mprotect.sh 20190828 21:34:16 all (79/668): msdos2.sh 20190828 21:34:17 all (80/668): msdos3.sh 20190828 21:34:19 all (81/668): msdos7.sh lock order reversal: 1st 0xfffff80159f97a48 msdosfs (msdosfs) @ kern/vfs_syscalls.c:3424 2nd 0xfffff8002dca4cc0 devfs (devfs) @ fs/msdosfs/msdosfs_vnops.c:854 stack backtrace: #0 0xffffffff80c424b3 at witness_debugger+0x73 #1 0xffffffff80c421fe at witness_checkorder+0xa9e #2 0xffffffff80ba8249 at lockmgr_lock_fast_path+0x179 #3 0xffffffff81237a40 at VOP_LOCK1_APV+0x70 #4 0xffffffff80cbfe95 at _vn_lock+0x65 #5 0xffffffff80a957e9 at msdosfs_fsync+0x49 #6 0xffffffff81236878 at VOP_FSYNC_APV+0x88 #7 0xffffffff80cbbe19 at kern_fsync+0x199 #8 0xffffffff8119a254 at amd64_syscall+0x2d4 #9 0xffffffff811715a0 at fast_syscall_common+0x101 20190828 21:34:20 all (82/668): msdos8.sh 20190828 21:34:22 all (83/668): msetdomain.sh 20190828 21:34:23 all (84/668): nanosleep.sh 20190828 21:34:25 all (85/668): nbufkv.sh 20190828 21:34:26 all (86/668): nfs.sh 20190828 21:34:27 all (87/668): nfs12.sh 20190828 21:34:29 all (88/668): nfs13.sh 20190828 21:34:30 all (89/668): nfs14.sh 20190828 21:34:32 all (90/668): nfs15.sh 20190828 21:34:33 all (91/668): nfs15lockd.sh 20190828 21:34:34 all (92/668): nfs15lockd3.sh 20190828 21:34:36 all (93/668): nfs3.sh 20190828 21:34:37 all (94/668): nfs4.sh 20190828 21:34:39 all (95/668): nfs7.sh 20190828 21:34:40 all (96/668): nfs8.sh 20190828 21:34:41 all (97/668): nfs9.sh 20190828 21:34:43 all (98/668): nfs_halfpage.sh 20190828 21:34:44 all (99/668): nfs_halfpage2.sh 20190828 21:34:46 all (100/668): nfsdelegation.sh 20190828 21:34:47 all (101/668): nfsdepth.sh 20190828 21:34:48 all (102/668): nfsrename.sh 20190828 21:34:50 all (103/668): nfssillyrename.sh 20190828 21:34:51 all (104/668): nullfs10.sh 20190828 21:34:53 all (105/668): nullfs13.sh 20190828 21:34:55 all (106/668): nullfs14.sh 20190828 21:34:56 all (107/668): nullfs15.sh 20190828 21:34:58 all (108/668): nullfs21.sh 20190828 21:34:59 all (109/668): nullfs23.sh 20190828 21:35:00 all (110/668): nullfs4.sh 20190828 21:35:02 all (111/668): nullfs6.sh 20190828 21:35:03 all (112/668): nullfs7.sh 20190828 21:35:05 all (113/668): nullfs9.sh 20190828 21:35:06 all (114/668): numa.sh 20190828 21:35:08 all (115/668): oom.sh 20190828 21:35:09 all (116/668): open.sh 20190828 21:35:10 all (117/668): pagefault.sh 20190828 21:35:12 all (118/668): parallelmount2.sh 20190828 21:35:13 all (119/668): pathconf.sh 20190828 21:35:15 all (120/668): pathconf2.sh 20190828 21:35:16 all (121/668): pcatch.sh 20190828 21:35:17 all (122/668): pcatch2.sh 20190828 21:35:19 all (123/668): pkru.sh 20190828 21:35:20 all (124/668): pkru2.sh 20190828 21:35:21 all (125/668): pmc4.sh 20190828 21:35:23 all (126/668): pmc5.sh 20190828 21:35:25 all (127/668): poll.sh 20190828 21:35:26 all (128/668): posix_fadvise.sh 20190828 21:35:28 all (129/668): posix_fadvise2.sh 20190828 21:35:29 all (130/668): procfs5.sh 20190828 21:35:30 all (131/668): procstat.sh 20190828 21:35:32 all (132/668): pthread9.sh 20190828 21:35:33 all (133/668): ptrace.sh 20190828 21:35:35 all (134/668): ptrace3.sh 20190828 21:35:36 all (135/668): pts2.sh 20190828 21:35:38 all (136/668): quota5.sh 20190828 21:35:39 all (137/668): rename13.sh 20190828 21:35:41 all (138/668): rename4.sh 20190828 21:35:42 all (139/668): routetbl.sh 20190828 21:35:43 all (140/668): ruby.sh 20190828 21:35:45 all (141/668): rwlock_ronly.sh 20190828 21:35:46 all (142/668): schedfuzz.sh 20190828 21:35:48 all (143/668): seekdir.sh 20190828 21:35:49 all (144/668): segnp.sh 20190828 21:35:51 all (145/668): segregs.sh 20190828 21:35:52 all (146/668): select3.sh 20190828 21:35:54 all (147/668): sendfile2.sh 20190828 21:35:55 all (148/668): sendfile4.sh 20190828 21:35:57 all (149/668): setuid.sh 20190828 21:35:58 all (150/668): shm_super.sh 20190828 21:36:00 all (151/668): sigaltstack.sh 20190828 21:36:01 all (152/668): sigreturn.sh 20190828 21:36:03 all (153/668): sigstop.sh 20190828 21:36:04 all (154/668): snap.sh lock order reversal: 1st 0xfffffe0000929ae8 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:283 2nd 0xfffff80159ae7630 snaplk (snaplk) @ ufs/ffs/ffs_snapshot.c:2327 stack backtrace: #0 0xffffffff80c424b3 at witness_debugger+0x73 #1 0xffffffff80c421fe at witness_checkorder+0xa9e #2 0xffffffff80ba885d at lockmgr_xlock_hard+0x6d #3 0xffffffff80ba9415 at __lockmgr_args+0x605 #4 0xffffffff80ec94b6 at ffs_copyonwrite+0x176 #5 0xffffffff80eebdac at ffs_geom_strategy+0x8c #6 0xffffffff80eff583 at ufs_strategy+0x83 #7 0xffffffff81237e80 at VOP_STRATEGY_APV+0x90 #8 0xffffffff80c89a44 at bufstrategy+0x44 #9 0xffffffff80c898cd at bufwrite+0x24d #10 0xffffffff80eedbce at ffs_syncvnode+0x2be #11 0xffffffff80eecbe2 at ffs_fsync+0x22 #12 0xffffffff81236878 at VOP_FSYNC_APV+0x88 #13 0xffffffff80efd1bf at ufs_remove+0xdf #14 0xffffffff81236a1c at VOP_REMOVE_APV+0xac #15 0xffffffff80cb8acf at kern_funlinkat+0x2bf #16 0xffffffff80cb8808 at sys_unlink+0x28 #17 0xffffffff8119a254 at amd64_syscall+0x2d4 lock order reversal: 1st 0xfffff80159ae7630 snaplk (snaplk) @ ufs/ufs/ufs_vnops.c:918 2nd 0xfffff80159cd8a48 ufs (ufs) @ ufs/ffs/ffs_snapshot.c:1650 stack backtrace: #0 0xffffffff80c424b3 at witness_debugger+0x73 #1 0xffffffff80c421fe at witness_checkorder+0xa9e #2 0xffffffff80ba885d at lockmgr_xlock_hard+0x6d #3 0xffffffff80ba9415 at __lockmgr_args+0x605 #4 0xffffffff80ec7a0d at ffs_snapremove+0xed #5 0xffffffff80ec2010 at ffs_truncate+0x270 #6 0xffffffff80ef554d at ufs_inactive+0x1cd #7 0xffffffff81237758 at VOP_INACTIVE_APV+0x88 #8 0xffffffff80cadbe8 at vinactive+0xe8 #9 0xffffffff80cae221 at vputx+0x341 #10 0xffffffff80cb8b23 at kern_funlinkat+0x313 #11 0xffffffff80cb8808 at sys_unlink+0x28 #12 0xffffffff8119a254 at amd64_syscall+0x2d4 #13 0xffffffff811715a0 at fast_syscall_common+0x101 20190828 21:36:06 all (155/668): snap12.sh 20190828 21:36:08 all (156/668): snap2-1.sh 20190828 21:36:09 all (157/668): snap2.sh 20190828 21:36:11 all (158/668): sndstat.sh 20190828 21:36:12 all (159/668): socketpair.sh 20190828 21:36:14 all (160/668): socketpair2.sh 20190828 21:36:15 all (161/668): suj2.sh 20190828 21:36:17 all (162/668): suj32.sh 20190828 21:36:18 all (163/668): suj35.sh lock order reversal: 1st 0xfffff8052dabf558 ufs (ufs) @ kern/vfs_subr.c:2737 2nd 0xffffffff81a02400 proctree (proctree) @ kern/subr_prf.c:182 stack backtrace: #0 0xffffffff80c424b3 at witness_debugger+0x73 #1 0xffffffff80c421fe at witness_checkorder+0xa9e #2 0xffffffff80bdfd28 at _sx_slock_int+0x68 #3 0xffffffff80c2552e at uprintf+0x5e #4 0xffffffff80ebaac1 at ffs_valloc+0x541 #5 0xffffffff80efe887 at ufs_mkdir+0xc7 #6 0xffffffff81236eec at VOP_MKDIR_APV+0x8c #7 0xffffffff80cbc593 at kern_mkdirat+0x1b3 #8 0xffffffff8119a254 at amd64_syscall+0x2d4 #9 0xffffffff811715a0 at fast_syscall_common+0x101 Aug 28 21:36:20 mercat1 kernel: pid 59310 (mkdir), uid 0 inumber 255 on /mnt: out of inodes 20190828 21:36:20 all (164/668): suj7.sh Failed to find journal. Use tunefs to create one Failed to start journal: 2 20190828 21:36:21 all (165/668): suj8.sh Aug 28 21:36:23 mercat1 kernel: pid 59557 (dd), uid 0 inumber 4 on /mnt: filesystem full Aug 28 21:36:23 mercat1 kernel: pid 59573 (dd), uid 0 inumber 5 on /mnt: filesystem full 20190828 21:36:23 all (166/668): tar.sh 20190828 21:36:25 all (167/668): temp.sh 20190828 21:36:26 all (168/668): tmpfs15.sh 20190828 21:36:28 all (169/668): tmpfs18.sh 20190828 21:36:29 all (170/668): tmpfs4.sh 20190828 21:36:31 all (171/668): tmpfs6.sh 20190828 21:36:32 all (172/668): tmpfs7.sh 20190828 21:36:34 all (173/668): trim5.sh 20190828 21:36:36 all (174/668): trim6.sh 20190828 21:36:37 all (175/668): trim7.sh 20190828 21:36:39 all (176/668): trim8.sh 20190828 21:36:40 all (177/668): truncate4.sh 20190828 21:36:42 all (178/668): truncate6.sh 20190828 21:36:43 all (179/668): tvnlru.sh 20190828 21:36:44 all (180/668): ufssuspend.sh 20190828 21:36:46 all (181/668): uma_zalloc_arg.sh 20190828 21:36:47 all (182/668): unlink.sh 20190828 21:36:49 all (183/668): unload.sh 20190828 21:36:51 all (184/668): unload2.sh 20190828 21:36:52 all (185/668): vfork.sh 20190828 21:36:54 all (186/668): vmio.sh 20190828 21:36:55 all (187/668): watchman.sh 20190828 21:36:56 all (188/668): write.sh 20190828 21:36:58 all (189/668): revoke.sh 20190828 21:37:00 all (190/668): scp.sh 20190828 21:37:01 all (191/668): datagram.sh 20190828 21:37:04 all (192/668): ext2fs3.sh 20190828 21:37:06 all (193/668): extattr.sh 20190828 21:37:09 all (194/668): isofs.sh 20190828 21:37:11 all (195/668): killpg.sh 20190828 21:37:14 all (196/668): md9.sh lock order reversal: 1st 0xfffff80007d71880 bufwait (bufwait) @ vm/vm_pager.c:379 2nd 0xfffff8052dc40cc0 tmpfs (tmpfs) @ dev/md/md.c:990 stack backtrace: #0 0xffffffff80c424b3 at witness_debugger+0x73 #1 0xffffffff80c421fe at witness_checkorder+0xa9e #2 0xffffffff80ba8249 at lockmgr_lock_fast_path+0x179 #3 0xffffffff81237a40 at VOP_LOCK1_APV+0x70 #4 0xffffffff80cbfe95 at _vn_lock+0x65 #5 0xffffffff806f44ca at mdstart_vnode+0x47a #6 0xffffffff806f21bc at md_kthread+0x20c #7 0xffffffff80b93604 at fork_exit+0x84 #8 0xffffffff81171cbe at fork_trampoline+0xe 20190828 21:37:16 all (197/668): mmap25.sh 20190828 21:37:19 all (198/668): mmap26.sh 20190828 21:37:21 all (199/668): mmap28.sh Aug 28 21:37:23 mercat1 kernel: Failed to fully fault in a core file segment at VA 0x800e00000 with size 0xc3b000 to be written at offset 0x86d000 for process mmap28 20190828 21:37:23 all (200/668): mmap8.sh 20190828 21:37:25 all (201/668): mmap9.sh 20190828 21:37:28 all (202/668): mountro2.sh newblk leak: 319/631. 20190828 21:37:30 all (203/668): mountro3.sh newblk leak: 1423/2054. 20190828 21:37:32 all (204/668): nullfs12.sh 20190828 21:37:34 all (205/668): nullfs16.sh 20190828 21:37:36 all (206/668): nullfs3.sh 20190828 21:37:39 all (207/668): openlock.sh 20190828 21:37:42 all (208/668): ptrace11.sh 20190828 21:37:44 all (209/668): ptrace8.sh 20190828 21:37:47 all (210/668): quota9.sh lock order reversal: 1st 0xfffff800077d9a88 FFS (FFS Lock) @ ufs/ufs/ufs_quota.c:759 2nd 0xfffff80159e54000 struct mount mtx (struct mount mtx) @ ufs/ufs/ufs_quota.c:765 stack backtrace: #0 0xffffffff80c424b3 at witness_debugger+0x73 #1 0xffffffff80c421fe at witness_checkorder+0xa9e #2 0xffffffff80bb4125 at __mtx_lock_flags+0x95 #3 0xffffffff80ef9f6d at quotaoff_inchange+0xbd #4 0xffffffff80ee848d at ffs_flushfiles+0x12d #5 0xffffffff80eca6a3 at softdep_flushfiles+0x73 #6 0xffffffff80eeabd7 at ffs_unmount+0xd7 #7 0xffffffff80ca5cc0 at dounmount+0x4e0 #8 0xffffffff80ca5729 at sys_unmount+0x339 #9 0xffffffff8119a254 at amd64_syscall+0x2d4 #10 0xffffffff811715a0 at fast_syscall_common+0x101 20190828 21:37:49 all (211/668): random.sh 20190828 21:37:51 all (212/668): rename6.sh 20190828 21:37:54 all (213/668): sendfile.sh 20190828 21:37:56 all (214/668): sendfile6.sh 20190828 21:37:58 all (215/668): snap10.sh 20190828 21:38:00 all (216/668): spin.sh 20190828 21:38:02 all (217/668): trim2.sh 20190828 21:38:04 all (218/668): umtx_suspend.sh 20190828 21:38:07 all (219/668): vm_fault_dontneed.sh 20190828 21:38:09 all (220/668): datamove4.sh 20190828 21:38:13 all (221/668): fcntl.sh 20190828 21:38:16 all (222/668): fpclone2.sh 20190828 21:38:25 all (223/668): kevent2.sh 20190828 21:38:28 all (224/668): kevent8.sh 20190828 21:38:32 all (225/668): mlockall7.sh 20190828 21:38:35 all (226/668): mmap23.sh 20190828 21:38:39 all (227/668): mmap4.sh vnode_pager_putpages: I/O error 28 0xfffff8052d30f4f0: tag ufs, type VREG usecount 0, writecount 0, refcount 3203 flags (VI_ACTIVE|VI_DOINGINACT) v_object 0xfffff8019e50cd00 ref 0 pages 25608 cleanbuf 3200 dirtybuf 1 lock type ufs: EXCL by thread 0xfffff80159d735a0 (pid 89624, mmap4, tid 100322) #0 0xffffffff80ba8c22 at lockmgr_xlock_hard+0x432 #1 0xffffffff80ba9415 at __lockmgr_args+0x605 #2 0xffffffff80eecd2c at ffs_lock+0x7c #3 0xffffffff81237a40 at VOP_LOCK1_APV+0x70 #4 0xffffffff80cbfe95 at _vn_lock+0x65 #5 0xffffffff80cae101 at vputx+0x221 #6 0xffffffff80f213ca at vm_object_deallocate+0x58a #7 0xffffffff80f1522f at vm_map_process_deferred+0x7f #8 0xffffffff80f1ae36 at vm_map_remove+0xc6 #9 0xffffffff80f14d03 at vmspace_exit+0xd3 #10 0xffffffff80b8e28d at exit1+0x59d #11 0xffffffff80b8dced at sys_sys_exit+0xd #12 0xffffffff8119a254 at amd64_syscall+0x2d4 #13 0xffffffff811715a0 at fast_syscall_common+0x101 nlink=1, Aug 28 21:38:40 mercat1 kernel: effnlink=1, size=104857600, extsize 0 generation=3d82131f, uid=0, gid=0, flags=0x0 ino 4, on dev md10a vnode_pager_putpages: residual I/O 32768 at 9760 0xfffff8052d30f4f0: tag ufs, type VREG usecount 0, writecount 0, refcount 3203 flags (VI_ACTIVE|VI_DOINGINACT) v_object 0xfffff8019e50cd00 ref 0 pages 25608 cleanbuf 3200 dirtybuf 1 lock type ufs: EXCL by thread 0xfffff80159d735a0 (pid 89624, mmap4, tid 100322) pid 89624 (mmap4#), uid 0 inumber 4 on /mnt: filesystem full 0 0xffffffff80ba8c22 at lockmgr_xlock_hard+0x432 #1 0xffffffff80ba9415 at __lockmgr_args+0x605 #2 0xffffffff80eecd2c at ffs_lock+0x7c #3 0xffffffff81237a40 at VOP_LOCK1_APV+0x70 #4 0xffffffff80cbfe95 at _vn_lock+0x65 #5 0xffffffff80cae101 at vputx+0x221 #6 0xffffffff80f213ca at vm_object_deallocate+0x58a #7 0xffffffff80f1522f at vm_map_process_deferred+0x7f #8 0xffffffff80f1ae36 at vm_map_remove+0xc6 #9 0xffffffff80f14d03 at vmspace_exit+0xd3 #10 0xffffffff80b8e28d at exit1+0x59d #11 0xffffffff80b8dced at sys_sys_exit+0xd #12 0xffffffff8119a254 at amd64_syscall+0x2d4 #13 0xffffffff811715a0 at fast_syscall_common+0x101 nlink=1, effnlink=1, size=104857600, extsize 0 generation=3d82131f, uid=0, gid=0, flags=0x0 ino 4, on dev md10a vnode_pager_putpages: zero-length write at 40075264 resid 131072 0xfffff8052d30f4f0: tag ufs, type VREG usecount 1, writecount 0, refcount 3203 flags (VI_ACTIVE) v_object 0xfffff8019e50cd00 ref 0 pages 25608 cleanbuf 3201 dirtybuf 0 lock type ufs: EXCL by thread 0xfffff800060f15a0 (pid 89632, umount, tid 100141) #0 0xffffffff80ba8c22 at lockmgr_xlock_hard+0x432 #1 0xffffffff80ba9415 at __lockmgr_args+0x605 #2 0xffffffff80eecd2c at ffs_lock+0x7c #3 0xffffffff81237a40 at VOP_LOCK1_APV+0x70 #4 0xffffffff80cbfe95 at _vn_lock+0x65 #5 0xffffffff80cad7a6 at vget+0xa6 #6 0xffffffff80cafa46 at vfs_msync+0xa6 #7 0xffffffff80ca5c40 at dounmount+0x460 #8 0xffffffff80ca5729 at sys_unmount+0x339 #9 0xffffffff8119a254 at amd64_syscall+0x2d4 #10 0xffffffff811715a0 at fast_syscall_common+0x101 nlink=1, effnlink=1, size=104857600, extsize 0 generation=3d82131f, uid=0, gid=0, flags=0x0 ino 4, on dev md10a Aug 28 21:38:42 mercat1 kernel: pid 89632 (umount), uid 0 inumber 4 on /mnt: filesystem full 20190828 21:38:42 all (228/668): msdos9.sh 20190828 21:38:45 all (229/668): newfs5.sh 20190828 21:38:48 all (230/668): procfs2.sh lock order reversal: 1st 0xfffff8017a604558 pseudofs (pseudofs) @ kern/vfs_syscalls.c:4052 2nd 0xffffffff81a023c0 allproc (allproc) @ fs/pseudofs/pseudofs_vnops.c:794 stack backtrace: #0 0xffffffff80c424b3 at witness_debugger+0x73 #1 0xffffffff80c421fe at witness_checkorder+0xa9e #2 0xffffffff80bdfd28 at _sx_slock_int+0x68 #3 0xffffffff80b0e6f6 at pfs_readdir+0x116 #4 0xffffffff81237458 at VOP_READDIR_APV+0x88 #5 0xffffffff80cbceac at kern_getdirentries+0x21c #6 0xffffffff80cbd0d9 at sys_getdirentries+0x29 #7 0xffffffff8119a254 at amd64_syscall+0x2d4 #8 0xffffffff811715a0 at fast_syscall_common+0x101 lock order reversal: (sleepable after non-sleepable) 1st 0xfffff800077da688 pfs_node (pfs_node) @ fs/pseudofs/pseudofs_internal.h:105 2nd 0xfffff80003680340 pidhash (pidhash) @ kern/kern_proc.c:425 stack backtrace: #0 0xffffffff80c424b3 at witness_debugger+0x73 #1 0xffffffff80c421fe at witness_checkorder+0xa9e #2 0xffffffff80bdfd28 at _sx_slock_int+0x68 #3 0xffffffff80bb9a8e at pfind+0x6e #4 0xffffffff80b0e729 at pfs_readdir+0x149 #5 0xffffffff81237458 at VOP_READDIR_APV+0x88 #6 0xffffffff80cbceac at kern_getdirentries+0x21c #7 0xffffffff80cbd0d9 at sys_getdirentries+0x29 #8 0xffffffff8119a254 at amd64_syscall+0x2d4 #9 0xffffffff811715a0 at fast_syscall_common+0x101 20190828 21:38:51 all (231/668): pthread.sh 20190828 21:38:55 all (232/668): tmpfs5.sh 20190828 21:38:59 all (233/668): truncate5.sh 20190828 21:39:02 all (234/668): udp.sh 20190828 21:39:05 all (235/668): unix_socket.sh 20190828 21:39:09 all (236/668): contigmalloc2.sh 20190828 21:39:12 all (237/668): devfd.sh 20190828 21:39:16 all (238/668): dup.sh 20190828 21:39:20 all (239/668): mkfifo6.sh 20190828 21:39:25 all (240/668): mmap13.sh 20190828 21:39:30 all (241/668): mountro4.sh 20190828 21:39:34 all (242/668): contigmalloc3.sh 20190828 21:39:36 all (243/668): growfs.sh g_access(997): provider ufsid/5d66d879a3689f74 has error 6 set g_access(997): provider ufsid/5d66d879a3689f74 has error 6 set g_access(997): provider ufsid/5d66d879a3689f74 has error 6 set g_access(997): provider gptid/91ea6e07-c9cb-11e9-ae12-0cc47aa8cdda has error 6 set g_access(997): provider gptid/91ea6e07-c9cb-11e9-ae12-0cc47aa8cdda has error 6 set g_access(997): provider gptid/91ea6e07-c9cb-11e9-ae12-0cc47aa8cdda has error 6 set g_dev_taste: make_dev_p() failed (gp->name=gptid/91ea6e07-c9cb-11e9-ae12-0cc47aa8cdda, error=17) g_dev_taste: make_dev_p() failed (gp->name=gptid/91ea6e07-c9cb-11e9-ae12-0cc47aa8cdda, error=17) g_dev_taste: make_dev_p() failed (gp->name=ufsid/5d66d879a3689f74, error=17) g_dev_taste: make_dev_p() failed (gp->name=gptid/91ea6e07-c9cb-11e9-ae12-0cc47aa8cdda, error=17) g_dev_taste: make_dev_p() failed (gp->name=ufsid/5d66d879a3689f74, error=17) g_dev_taste: make_dev_p() failed (gp->name=gptid/91ea6e07-c9cb-11e9-ae12-0cc47aa8cdda, error=17) g_dev_taste: make_dev_p() failed (gp->name=ufsid/5d66d879a3689f74, error=17) GEOM leak: 14/87 20190828 21:39:41 all (244/668): procfs.sh 20190828 21:39:47 all (245/668): ptrace2.sh 20190828 21:39:52 all (246/668): rename11.sh 20190828 21:39:57 all (247/668): symlink4.sh 20190828 21:40:02 all (248/668): tmpfs10.sh 20190828 21:40:08 all (249/668): truncate3.sh 20190828 21:40:13 all (250/668): truncate7.sh 20190828 21:40:18 all (251/668): datamove5.sh 20190828 21:40:24 all (252/668): fork.sh 20190828 21:40:31 all (253/668): linger3.sh 20190828 21:40:37 all (254/668): mlockall4.sh 20190828 21:40:44 all (255/668): pshared.sh 20190828 21:40:49 all (256/668): ptrace9.sh 20190828 21:40:56 all (257/668): sparse.sh 20190828 21:41:06 all (258/668): linger2.sh Aug 28 21:41:09 mercat1 kernel: pid 40174 (linger2), uid 2007 inumber 889 on /mnt: filesystem full 20190828 21:41:43 all (259/668): linger4.sh Aug 28 21:41:47 mercat1 kernel: pid 44847 (linger4), uid 2007 inumber 37646 on /mnt: out of inodes 20190828 21:41:50 all (260/668): symlink2.sh 20190828 21:41:58 all (261/668): sysctl.sh 20190828 21:42:05 all (262/668): md8.sh 20190828 21:42:14 all (263/668): quota11.sh 20190828 21:42:23 all (264/668): shm_open.sh 20190828 21:42:31 all (265/668): compare.sh 20190828 21:42:40 all (266/668): ffs_blkfree.sh newblk leak: 26627/28681. 20190828 21:42:50 all (267/668): procfs3.sh 20190828 21:42:58 all (268/668): execve.sh 20190828 21:43:03 all (269/668): thr.sh witness_lock_list_get: witness exhausted 20190828 21:43:13 all (270/668): tmpfs8.sh 20190828 21:43:23 all (271/668): umountf4.sh freework leak: 156/197. 20190828 21:43:34 all (272/668): fsck5.sh 20190828 21:43:45 all (273/668): nfs15lockd2.sh 20190828 21:43:56 all (274/668): proccontrol.sh 20190828 21:44:07 all (275/668): rdgsbase.sh 20190828 21:44:19 all (276/668): rename8.sh 20190828 21:44:28 all (277/668): rename9.sh 20190828 21:44:39 all (278/668): mprotect2.sh 20190828 21:44:52 all (279/668): posix_fadvise3.sh 20190828 21:45:04 all (280/668): tmpfs14.sh 20190828 21:45:16 all (281/668): pipe2.sh 20190828 21:45:29 all (282/668): umountf11.sh 20190828 21:45:43 all (283/668): umountf12.sh 20190828 21:45:56 all (284/668): umountf6.sh 20190828 21:46:09 all (285/668): recursiveflushes.sh 20190828 21:46:24 all (286/668): radix.sh Aug 28 21:46:29 mercat1 kernel: pid 36285 (radix), jid 0, uid 0, was killed: out of swap space 20190828 21:46:36 all (287/668): suj10.sh 20190828 21:46:53 all (288/668): mountu.sh 20190828 21:47:10 all (289/668): rdwr.sh 20190828 21:47:28 all (290/668): sem.sh 20190828 21:47:46 all (291/668): kevent6.sh 20190828 21:48:04 all (292/668): suj14.sh Aug 28 21:48:19 mercat1 kernel: pid 73568 (cp), uid 0 inumber 41151 on /mnt: filesystem full Aug 28 21:48:20 mercat1 kernel: pid 73568 (cp), uid 0 inumber 41190 on /mnt: filesystem full Aug 28 21:48:21 mercat1 kernel: pid 73568 (cp), uid 0 inumber 41225 on /mnt: filesystem full Aug 28 21:48:22 mercat1 kernel: pid 73602 (cp), uid 0 inumber 41226 on /mnt: filesystem full Aug 28 21:48:23 mercat1 kernel: pid 73643 (mkdir), uid 0 inumber 41227 on /mnt: filesystem full 20190828 21:48:25 all (293/668): mountro6.sh 20190828 21:48:46 all (294/668): multicast.sh 20190828 21:49:08 all (295/668): rename7.sh 20190828 21:49:28 all (296/668): tmpfs19.sh 20190828 21:49:58 all (297/668): core4.sh Aug 28 21:50:02 mercat1 kernel: Failed to write core file for process core4 (error 5) 20190828 21:50:24 all (298/668): sendfile5.sh Aug 28 21:50:23 mercat1 syslogd: last message repeated 59 times 20190828 21:50:46 all (299/668): f_offset.sh 20190828 21:51:12 all (300/668): tmpfs2.sh 20190828 21:51:37 all (301/668): pread.sh 20190828 21:52:02 all (302/668): mmap7.sh 20190828 21:52:27 all (303/668): multicast2.sh Aug 28 21:52:31 mercat1 mDNSResponder[70659]: mDNSResponder (Engineering Build) (Jul 27 2019 06:17:54) starting Aug 28 21:52:31 mercat1 mDNSResponder[70659]: mDNS_AddDNSServer: Lock not held! mDNS_busy (0) mDNS_reentrancy (0) Aug 28 21:52:31 mercat1 mDNSResponder[70659]: mDNSPlatformSourceAddrForDest: connect 1.1.1.1 failed errno 51 (Network is unreachable) if_delmulti_locked: detaching ifnet instance 0xfffff800060e3000 if_delmulti_locked: detaching ifnet instance 0xfffff800060e3000 if_delmulti_locked: detaching ifnet instance 0xfffff8002d88f800 if_delmulti_locked: detaching ifnet instance 0xfffff8002d88f800 if_delmulti_locked: detaching ifnet instance 0xfffff800060e3000 if_delmulti_locked: detaching ifnet instance 0xfffff8002d88f800 if_delmulti_locked: detaching ifnet instance 0xfffff8002d88f800 if_delmulti_locked: detaching ifnet instance 0xfffff8002d88f800 if_delmulti_locked: detaching ifnet instance 0xfffff8002d88f800 if_delmulti_locked: detaching ifnet instance 0xfffff8002d88f800 if_delmulti_locked: detaching ifnet instance 0xfffff8002d88f800 if_delmulti_locked: detaching ifnet instance 0xfffff800060e3000 Aug 28 21:53:11 mercat1 mDNSResponder[70659]: mDNSResponder (Engineering Build) (Jul 27 2019 06:17:54) stopping 20190828 21:53:12 all (304/668): sendmsg2.sh if_delmulti_locked: detaching ifnet instance 0xfffff800060e3000 20190828 21:53:39 all (305/668): beneath.sh 20190828 21:54:07 all (306/668): sendfile3.sh 20190828 21:54:34 all (307/668): namecache2.sh 20190828 21:55:02 all (308/668): split.sh freework leak: 691/888. newblk leak: 52809/81490. 20190828 21:55:57 all (309/668): mkfifo3.sh 20190828 21:56:29 all (310/668): mmap37.sh 20190828 21:57:00 all (311/668): credleak.sh 20190828 21:57:34 all (312/668): pthread5.sh 20190828 21:58:06 all (313/668): tcp4.sh Expensive timeout(9) function: 0xffffffff80e08f60(0xfffff802d8ab63d0) 0.072794268 s Expensive timeout(9) function: 0xffffffff80e08f60(0xfffff805930e97a0) 0.543016693 s Expensive timeout(9) function: 0xffffffff80e09ec0(0xfffff8045fe3b3d0) 0.924353811 s Expensive timeout(9) function: 0xffffffff80e08f60(0xfffff8051a12d7a0) 1.265901689 s 20190828 21:58:42 all (314/668): msdos5.sh GEOM leak: 19/106 20190828 21:59:17 all (315/668): core2.sh 20190828 21:59:52 all (316/668): nullfs22.sh 20190828 22:00:27 all (317/668): rename3.sh 20190828 22:01:02 all (318/668): umountf5.sh Aug 28 22:01:04 mercat1 kernel: pid 79318 (mkdir), uid 0 inumber 1151 on /mnt: out of inodes Aug 28 22:01:09 mercat1 kernel: pid 79309 (creat), uid 0 inumber 768 on /mnt: out of inodes Aug 28 22:01:12 mercat1 syslogd: last message repeated 1 times Aug 28 22:01:19 mercat1 kernel: pid 79306 (creat), uid 0 inumber 385 on /mnt: out of inodes Aug 28 22:01:22 mercat1 kernel: pid 79308 (creat), uid 0 inumber 5 on /mnt: out of inodes Aug 28 22:01:26 mercat1 kernel: pid 79309 (creat), uid 0 inumber 768 on /mnt: out of inodes Aug 28 22:01:36 mercat1 kernel: pid 79310 (creat), uid 0 inumber 769 on /mnt: out of inodes Aug 28 22:01:44 mercat1 kernel: pid 79307 (creat), uid 0 inumber 1153 on /mnt: out of inodes Aug 28 22:01:48 mercat1 kernel: pid 79308 (creat), uid 0 inumber 5 on /mnt: out of inodes Aug 28 22:01:51 mercat1 kernel: pid 79312 (mkdir), uid 0 inumber 74 on /mnt: out of inodes Aug 28 22:01:56 mercat1 kernel: pid 79308 (creat), uid 0 inumber 5 on /mnt: out of inodes 20190828 22:02:17 all (319/668): umountf.sh 20190828 22:02:50 all (320/668): vnodes.sh 20190828 22:03:30 all (321/668): quota8.sh 20190828 22:04:16 all (322/668): crossmp7.sh ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Warning: memory type solaris leaked memory on destroy (4 allocations, 2112 bytes leaked). 20190828 22:05:02 all (323/668): mmap5.sh 20190828 22:05:31 all (324/668): suj33.sh 20190828 22:06:01 all (325/668): fpu.sh 20190828 22:06:54 all (326/668): ufsbench.sh 20190828 22:07:49 all (327/668): crossmp5.sh 20190828 22:08:44 all (328/668): posix_openpt.sh 20190828 22:09:31 all (329/668): nullfs11.sh 20190828 22:10:26 all (330/668): fifo3.sh 20190828 22:11:22 all (331/668): advlock.sh 20190828 22:12:23 all (332/668): flock.sh 20190828 22:13:24 all (333/668): fsgs.sh 20190828 22:14:25 all (334/668): fts2.sh 20190828 22:15:26 all (335/668): fullpath.sh 20190828 22:16:28 all (336/668): jumbo.sh 20190828 22:17:27 all (337/668): mdconfig2.sh 20190828 22:18:28 all (338/668): mknod.sh 20190828 22:19:29 all (339/668): mmap29.sh 20190828 22:20:31 all (340/668): pts3.sh 20190828 22:21:32 all (341/668): sem_wait.sh 20190828 22:22:33 all (342/668): sigstop2.sh 20190828 22:23:34 all (343/668): socketpair3.sh 20190828 22:24:35 all (344/668): bench.sh 20190828 22:25:38 all (345/668): fsck.sh Aug 28 22:26:18 mercat1 kernel: pid 27746 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:18 mercat1 kernel: pid 27769 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:18 mercat1 kernel: pid 27792 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:18 mercat1 kernel: pid 27815 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:18 mercat1 kernel: pid 27838 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:18 mercat1 kernel: pid 27861 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:18 mercat1 kernel: pid 27884 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:18 mercat1 kernel: pid 27907 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:18 mercat1 kernel: pid 27930 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:18 mercat1 kernel: pid 27953 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:18 mercat1 kernel: pid 27976 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:18 mercat1 kernel: pid 27999 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:18 mercat1 kernel: pid 28022 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:18 mercat1 kernel: pid 28045 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:18 mercat1 kernel: pid 28068 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:18 mercat1 kernel: pid 28096 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:18 mercat1 kernel: pid 28119 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:18 mercat1 kernel: pid 28142 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:18 mercat1 kernel: pid 28165 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:18 mercat1 kernel: pid 28188 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:18 mercat1 kernel: pid 28211 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:18 mercat1 kernel: pid 28234 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 28285 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 28308 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 28331 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 28354 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 28377 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 28400 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 28451 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 28478 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 28519 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 28614 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 28711 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 28807 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 28902 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 29000 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 29098 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 29193 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 29288 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 29362 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 29427 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 29492 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 29557 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 29624 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 29689 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 29756 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 29821 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 29886 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 29951 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 30018 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 30082 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 30106 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:19 mercat1 kernel: pid 30129 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30152 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30175 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30198 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30221 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30244 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30267 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30290 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30313 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30336 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30359 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30382 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30405 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30456 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30479 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30502 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30525 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30548 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30571 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30594 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30617 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30640 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30663 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30686 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30709 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30732 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30755 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30778 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30801 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30824 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:20 mercat1 kernel: pid 30847 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 30898 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 30921 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 30972 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 30995 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 31018 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 31041 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 31064 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 31115 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 31138 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 31161 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 31184 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 31207 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 31230 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 31253 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 31276 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 31299 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 31322 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 31345 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 31368 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 31391 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 31414 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 31437 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 31460 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 31483 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 31506 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 31529 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 31552 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 31575 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 31598 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:21 mercat1 kernel: pid 31621 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 31644 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 31667 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 31690 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 31713 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 31736 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 31759 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 31782 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 31805 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 31828 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 31851 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 31874 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 31897 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 31920 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 31943 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 31966 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 31989 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 32012 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 32035 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 32058 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 32086 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 32109 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 32132 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 32155 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 32178 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 32201 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 32224 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 32247 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 32270 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 32293 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 32316 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 32339 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:22 mercat1 kernel: pid 32362 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 32385 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 32408 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 32431 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 32454 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 32477 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 32500 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 32523 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 32546 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 32569 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 32620 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 32643 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 32666 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 32689 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 32712 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 32735 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 32758 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 32781 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 32804 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 32827 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 32850 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 32873 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 32896 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 32919 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 32942 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 32965 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 32988 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 33011 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 33034 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 33057 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 33080 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:23 mercat1 kernel: pid 33103 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33126 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33149 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33172 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33195 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33218 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33241 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33292 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33315 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33338 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33361 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33384 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33407 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33430 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33453 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33476 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33499 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33522 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33545 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33568 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33591 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33614 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33637 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33660 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33683 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33706 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33729 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33752 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33775 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33798 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33821 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33844 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:24 mercat1 kernel: pid 33867 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 33890 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 33913 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 33936 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 33959 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 33982 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 34005 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 34028 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 34051 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 34074 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 34097 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 34120 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 34143 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 34166 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 34189 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 34212 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 34235 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 34258 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 34281 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 34304 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 34327 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 34350 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 34373 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 34396 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 34419 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 34442 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 34465 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 34488 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 34511 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 34534 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 34557 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 34580 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:25 mercat1 kernel: pid 34603 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 34626 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 34649 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 34672 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 34695 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 34718 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 34741 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 34764 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 34792 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 34815 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 34838 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 34861 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 34884 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 34907 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 34930 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 34953 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 34976 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 34999 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 35022 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 35045 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 35068 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 35091 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 35114 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 35137 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 35160 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 35183 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 35206 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 35229 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 35252 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 35275 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 35298 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 35321 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 35344 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:26 mercat1 kernel: pid 35367 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 35390 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 35413 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 35436 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 35459 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 35482 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 35505 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 35528 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 35551 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 35574 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 35597 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 35620 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 35643 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 35666 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 35689 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 35712 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 35735 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 35758 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 35781 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 35804 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 35827 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 35850 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 35873 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 35896 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 35919 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 35942 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 35965 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 35988 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 36011 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 36034 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 36057 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 36080 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:27 mercat1 kernel: pid 36103 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36126 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36149 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36172 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36195 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36218 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36241 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36264 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36292 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36315 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36338 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36361 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36384 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36407 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36430 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36453 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36476 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36499 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36522 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36573 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36596 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36619 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36642 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36665 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36688 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36711 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36734 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36785 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36808 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:28 mercat1 kernel: pid 36831 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 36882 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 36905 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 36956 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 36979 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 37002 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 37025 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 37048 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 37071 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 37094 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 37117 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 37168 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 37191 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 37214 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 37237 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 37260 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 37311 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 37334 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 37357 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 37380 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 37403 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 37426 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 37449 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 37472 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 37495 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 37518 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 37541 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 37564 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:29 mercat1 kernel: pid 37587 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 37638 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 37665 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 37688 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 37747 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 37845 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 37940 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 38035 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 38133 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 38228 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 38323 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 38418 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 38513 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 38582 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 38649 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 38714 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 38779 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 38846 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 38911 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 38978 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 39043 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 39108 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 39175 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 39240 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 39293 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 39316 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 39339 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 39362 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 39385 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 39408 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 39431 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:30 mercat1 kernel: pid 39454 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 39477 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 39500 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 39523 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 39546 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 39569 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 39592 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 39615 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 39638 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 39661 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 39684 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 39707 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 39730 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 39753 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 39776 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 39799 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 39822 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 39845 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 39868 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 39891 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 39914 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 39937 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 39960 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 39985 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 40008 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 40031 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 40054 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 40077 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 40100 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 40123 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 40146 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 40169 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:31 mercat1 kernel: pid 40192 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40215 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40238 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40261 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40284 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40307 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40330 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40353 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40376 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40399 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40422 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40445 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40468 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40491 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40514 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40537 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40560 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40583 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40634 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40657 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40680 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40703 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40726 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40749 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40772 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40795 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40818 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40841 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40864 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40887 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40910 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:32 mercat1 kernel: pid 40933 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 40956 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 40979 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41002 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41030 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41053 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41076 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41099 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41122 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41145 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41168 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41191 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41214 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41237 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41260 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41283 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41306 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41329 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41352 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41375 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41398 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41421 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41444 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41467 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41490 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41513 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41536 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41559 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41582 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41605 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41628 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41651 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:33 mercat1 kernel: pid 41674 (touch), uid 0 inumber 2 on /mnt11: out of inodes Aug 28 22:26:34 mercat1 kernel: pid 41697 (touch), uid 0 inumberSuperblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) 2 on /mnt11: out of inodes Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) Superblock check-hash failed: recorded check-hash 0xe6aba3bb != computed check-hash 0x8d7f7c7d (Ignored) 20190828 22:26:34 all (346/668): midi2.sh 20190828 22:27:35 all (347/668): sendfile8.sh 20190828 22:28:36 all (348/668): sendfile9.sh 20190828 22:29:37 all (349/668): shm.sh 20190828 22:30:39 all (350/668): crossmp8.sh 20190828 22:31:41 all (351/668): suj5.sh 20190828 22:32:54 all (352/668): tcp3.sh 20190828 22:33:59 all (353/668): sendfile_shm.sh 20190828 22:35:45 all (354/668): exlock.sh 20190828 22:36:53 all (355/668): rename10.sh 20190828 22:38:00 all (356/668): suj6.sh 20190828 22:39:09 all (357/668): truss2.sh 20190828 22:40:39 all (358/668): stack_guard_page.sh Aug 28 22:41:00 mercat1 kernel: pid 21945 (swap), jid 0, uid 2007, was killed: out of swap space 20190828 22:42:13 all (359/668): vmstat.sh 20190828 22:43:24 all (360/668): quota4.sh 20190828 22:44:32 all (361/668): mountro5.sh 20190828 22:45:53 all (362/668): zfs6.sh ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Warning: memory type solaris leaked memory on destroy (4 allocations, 2112 bytes leaked). 20190828 22:47:16 all (363/668): overcommit2.sh Aug 28 22:48:31 mercat1 kernel: pid 42034 (dd), uid 0 inumber 5 on /mnt: filesystem full 20190828 22:48:34 all (364/668): crossmp.sh 20190828 22:49:54 all (365/668): pthread2.sh 20190828 22:51:26 all (366/668): dd.sh Aug 28 22:53:53 mercat1 kernel: pid 11866 (dd), uid 0 inumber 561804 on /tmp: filesystem full 20190828 22:54:20 all (367/668): tmpfs12.sh Aug 28 22:54:56 mercat1 kernel: pid 13223 (swap), jid 0, uid 0, was killed: out of swap space 20190828 22:55:47 all (368/668): pmc6.sh 20190828 22:57:26 all (369/668): mmap18.sh 20190828 22:59:10 all (370/668): pthread7.sh 20190828 23:00:49 all (371/668): ptrace10.sh 20190828 23:02:37 all (372/668): umountf2.sh 20190828 23:04:26 all (373/668): pthread3.sh 20190828 23:06:18 all (374/668): msdos10.sh 20190828 23:08:14 all (375/668): fifo2.sh 20190828 23:10:15 all (376/668): kevent.sh 20190828 23:12:17 all (377/668): kevent14.sh 20190828 23:14:18 all (378/668): overflow3.sh 20190828 23:16:19 all (379/668): sendfile10.sh 20190828 23:18:21 all (380/668): gnop2.sh GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. 20190828 23:20:21 all (381/668): midi.sh 20190828 23:22:22 all (382/668): mmap21.sh Aug 28 23:23:26 mercat1 kernel: pid 94969 (mmap21), uid 2007 inumber 260 on /tmp: filesystem full Aug 28 23:23:26 mercat1 kernel: Failed to write core file for process mmap21 (error 28) 20190828 23:23:27 all (383/668): ftruncate3.sh Aug 28 23:23:26 mercat1 syslogd: last message repeated 1 times GEOM leak: 10/116 20190828 23:25:29 all (384/668): mmap27.sh 20190828 23:27:42 all (385/668): poll2.sh 20190828 23:29:44 all (386/668): setsockopt2.sh 20190828 23:31:45 all (387/668): shm2.sh 20190828 23:33:50 all (388/668): snap11.sh 20190828 23:35:53 all (389/668): vm_map.sh 20190828 23:37:58 all (390/668): graid1_3.sh g_access(997): provider ufsid/5d66d879a3689f74 has error 6 set g_access(997): provider gptid/91ea6e07-c9cb-11e9-ae12-0cc47aa8cdda has error 6 set 20190828 23:40:02 all (391/668): truncate9.sh 20190828 23:42:06 all (392/668): fts3.sh 20190828 23:44:11 all (393/668): graid1_10.sh g_access(997): provider ufsid/5d66d879a3689f74 has error 6 set g_access(997): provider gptid/91ea6e07-c9cb-11e9-ae12-0cc47aa8cdda has error 6 set 20190828 23:46:12 all (394/668): mmap31.sh 20190828 23:48:18 all (395/668): suj23.sh 20190828 23:51:32 all (396/668): tmpfs13.sh 20190828 23:53:38 all (397/668): syscall4.sh WARNING pid 83260 (calls): ioctl sign-extension ioctl 47621f5d797c WARNING pid 83305 (calls): ioctl sign-extension ioctl cb24e9a1604cb455 WARNING pid 83438 (calls): ioctl sign-extension ioctl 78035118e6ea WARNING pid 83461 (calls): ioctl sign-extension ioctl 26300d310671 WARNING pid 83469 (calls): ioctl sign-extension ioctl 253b0dff43d WARNING pid 83537 (calls): ioctl sign-extension ioctl 5af525316db4 WARNING pid 83579 (calls): ioctl sign-extension ioctl c2743b71dbec4dea WARNING pid 83584 (calls): ioctl sign-extension ioctl 7c161d5a0f6 WARNING pid 83700 (calls): ioctl sign-extension ioctl 588f537dfbc7 WARNING pid 83703 (calls): ioctl sign-extension ioctl 3c61f60eed24 WARNING pid 83710 (calls): ioctl sign-extension ioctl 275cdc8556d7 WARNING pid 83774 (calls): ioctl sign-extension ioctl 5ff00312aebb3e1f WARNING pid 84699 (calls): ioctl sign-extension ioctl bd9f9ffe4dc4abf2 WARNING pid 84937 (calls): ioctl sign-extension ioctl 45700620cbee WARNING pid 85030 (calls): ioctl sign-extension ioctl 38f174534d8bba33 WARNING pid 85069 (calls): ioctl sign-extension ioctl c6a35ba39fa7bda6 WARNING pid 85093 (calls): ioctl sign-extension ioctl cc947f53e624f8b4 WARNING pid 85128 (calls): ioctl sign-extension ioctl 708a966d2047 WARNING pid 85154 (calls): ioctl sign-extension ioctl d742cb2c3a4 WARNING pid 85191 (calls): ioctl sign-extension ioctl 2c10bea5083e3764 WARNING pid 85206 (calls): ioctl sign-extension ioctl 45240b3ccc3034c7 WARNING pid 85435 (calls): ioctl sign-extension ioctl 1c0cd1a8dae WARNING pid 85453 (calls): ioctl sign-extension ioctl eda302d7f2494e7 WARNING pid 85462 (calls): ioctl sign-extension ioctl 672e3a27806f WARNING pid 85471 (calls): ioctl sign-extension ioctl a7a7106c69fc8882 WARNING pid 85474 (calls): ioctl sign-extension ioctl 11aeeda3fb89 WARNING pid 85477 (calls): ioctl sign-extension ioctl 3531d8e52b4b WARNING pid 85503 (calls): ioctl sign-extension ioctl f06c0ccfa26ed61b WARNING pid 85528 (calls): ioctl sign-extension ioctl 2a0aabdf3bd9fa8 WARNING pid 85589 (calls): ioctl sign-extension ioctl 2ff5d8d52195 WARNING pid 85610 (calls): ioctl sign-extension ioctl 3727be13f6f7 WARNING pid 85726 (calls): ioctl sign-extension ioctl 35e8099bf6dd WARNING pid 85726 (calls): ioctl sign-extension ioctl 4a550b51ba16 WARNING pid 85811 (calls): ioctl sign-extension ioctl 1324dde78d5f WARNING pid 85819 (calls): ioctl sign-extension ioctl d131256e0a6c55c6 WARNING pid 85827 (calls): ioctl sign-extension ioctl 9da24fc92c898b73 WARNING pid 85989 (calls): ioctl sign-extension ioctl e23dd3af9d6 WARNING pid 86566 (calls): ioctl sign-extension ioctl 23e1923b33ad WARNING pid 87058 (calls): ioctl sign-extension ioctl d3d5920b92dd1966 WARNING pid 87101 (calls): ioctl sign-extension ioctl 49fbe4289972 WARNING pid 87107 (calls): ioctl sign-extension ioctl 1520bd8162e884ea WARNING pid 87116 (calls): ioctl sign-extension ioctl d4a071c75102fe5a WARNING pid 87146 (calls): ioctl sign-extension ioctl 7cf879d4ebc1 WARNING pid 87153 (calls): ioctl sign-extension ioctl b4941faead3880d6 WARNING pid 87180 (calls): ioctl sign-extension ioctl 7dfd0ad6bca6 WARNING pid 87211 (calls): ioctl sign-extension ioctl 1dfbad21a0b2bba9 WARNING pid 87216 (calls): ioctl sign-extension ioctl 738ca0af183c WARNING pid 87285 (calls): ioctl sign-extension ioctl 4b9e30c16c3d WARNING pid 87326 (calls): ioctl sign-extension ioctl 2d88bf69e0f9 WARNING pid 87327 (calls): ioctl sign-extension ioctl fb44f40d68157a2e WARNING pid 87330 (calls): ioctl sign-extension ioctl 5756c147e73f WARNING pid 87345 (calls): ioctl sign-extension ioctl 61374b0457d1 WARNING pid 87470 (calls): ioctl sign-extension ioctl d82442c3e5c92b30 WARNING pid 87515 (calls): ioctl sign-extension ioctl ae5e682a12c6fd67 WARNING pid 87580 (calls): ioctl sign-extension ioctl 77a931bc01b4c0c1 WARNING pid 87791 (calls): ioctl sign-extension ioctl 591ddf41d6cc WARNING pid 87868 (calls): ioctl sign-extension ioctl 37dc5008433 WARNING pid 87893 (calls): ioctl sign-extension ioctl 517201b864e5d717 WARNING pid 88623 (calls): ioctl sign-extension ioctl 4fae0fe14c0 WARNING pid 88779 (calls): ioctl sign-extension ioctl dfd0e8170c3 WARNING pid 89226 (calls): ioctl sign-extension ioctl 65326a6b92 WARNING pid 89283 (calls): ioctl sign-extension ioctl b32f2558d97ed770 WARNING pid 89300 (calls): ioctl sign-extension ioctl 3f8e8e9ae8c03f03 WARNING pid 89363 (calls): ioctl sign-extension ioctl ea61fa5d296 WARNING pid 89431 (calls): ioctl sign-extension ioctl 3956f1360666 WARNING pid 89445 (calls): ioctl sign-extension ioctl 9de0e27784c3aa9a WARNING pid 89505 (calls): ioctl sign-extension ioctl 76b59834898cf3ab WARNING pid 89653 (calls): ioctl sign-extension ioctl 3bb9a9c4363c WARNING pid 89925 (calls): ioctl sign-extension ioctl 62e33dadf5b WARNING pid 89932 (calls): ioctl sign-extension ioctl 48c6c51403d WARNING pid 91215 (calls): ioctl sign-extension ioctl 5e85e9371ee1 WARNING pid 91303 (calls): ioctl sign-extension ioctl 48deea9648139503 WARNING pid 91307 (calls): ioctl sign-extension ioctl 6fbb79e7c367 WARNING pid 91324 (calls): ioctl sign-extension ioctl 6d293f054f9a WARNING pid 91365 (calls): ioctl sign-extension ioctl d4b97e0fff993680 WARNING pid 91425 (calls): ioctl sign-extension ioctl 254c040b5a16 WARNING pid 91430 (calls): ioctl sign-extension ioctl efc3c6f6f7d42a45 WARNING pid 91474 (calls): ioctl sign-extension ioctl 493ed7df839f WARNING pid 91474 (calls): ioctl sign-extension ioctl ae9d2541942dda08 WARNING pid 91593 (calls): ioctl sign-extension ioctl 682daeb9d0a2 WARNING pid 91603 (calls): ioctl sign-extension ioctl 125173fcfaebbb53 WARNING pid 91631 (calls): ioctl sign-extension ioctl 4d22d0a3271f25ac WARNING pid 91635 (calls): ioctl sign-extension ioctl 1354b57d62590381 WARNING pid 91729 (calls): ioctl sign-extension ioctl 91d498204c7217fa WARNING pid 91739 (calls): ioctl sign-extension ioctl 41a9dd3e343e WARNING pid 91833 (calls): ioctl sign-extension ioctl 48eeee826de36fe8 WARNING pid 91835 (calls): ioctl sign-extension ioctl b4ccbc88baefadfc WARNING pid 91892 (calls): ioctl sign-extension ioctl 90b1758d63e05a2e WARNING pid 91932 (calls): ioctl sign-extension ioctl faa5d4d6d6953953 WARNING pid 91985 (calls): ioctl sign-extension ioctl 58775134a6bf WARNING pid 91995 (calls): ioctl sign-extension ioctl 4c64511186b45904 WARNING pid 92058 (calls): ioctl sign-extension ioctl f5d76336926ee1ca WARNING pid 92472 (calls): ioctl sign-extension ioctl ed0b5ff40a4a9d23 WARNING pid 93275 (calls): ioctl sign-extension ioctl e2b38499389044a1 WARNING pid 93282 (calls): ioctl sign-extension ioctl d723cd583f69ad30 WARNING pid 93298 (calls): ioctl sign-extension ioctl 8ba64f9027e399ab WARNING pid 93332 (calls): ioctl sign-extension ioctl 29bc8ac0e90b800e WARNING pid 93341 (calls): ioctl sign-extension ioctl 25fbc2a8d9b1f751 WARNING pid 93376 (calls): ioctl sign-extension ioctl 18cdff44137ce6bc WARNING pid 93420 (calls): ioctl sign-extension ioctl 371d6266c489 WARNING pid 93558 (calls): ioctl sign-extension ioctl 2b5030e1b3779d4a WARNING pid 93586 (calls): ioctl sign-extension ioctl 519f4a5354ff WARNING pid 93615 (calls): ioctl sign-extension ioctl 9a330a650e7ee35f WARNING pid 93673 (calls): ioctl sign-extension ioctl 314797c40364 WARNING pid 93690 (calls): ioctl sign-extension ioctl 75afdc333f1176ec WARNING pid 93737 (calls): ioctl sign-extension ioctl 63eeb9752276 WARNING pid 93772 (calls): ioctl sign-extension ioctl d7480103762472c5 WARNING pid 93809 (calls): ioctl sign-extension ioctl b264111cc5c9ffda WARNING pid 93815 (calls): ioctl sign-extension ioctl 27d1485c3b5 WARNING pid 93852 (calls): ioctl sign-extension ioctl 76a40e41132cadaa WARNING pid 93856 (calls): ioctl sign-extension ioctl 426a4ec15fa0adad WARNING pid 93872 (calls): ioctl sign-extension ioctl 55cbfb5030fb7f01 WARNING pid 93943 (calls): ioctl sign-extension ioctl bafce48cb4ef2d94 WARNING pid 94004 (calls): ioctl sign-extension ioctl fff2a28dab1792 WARNING pid 94084 (calls): ioctl sign-extension ioctl f44d733634d2e5e4 WARNING pid 94093 (calls): ioctl sign-extension ioctl 4c1adadcf7da9b80 WARNING pid 94100 (calls): ioctl sign-extension ioctl 3c3bbfbd1aae WARNING pid 95419 (calls): ioctl sign-extension ioctl a50800b602c2c50e WARNING pid 95422 (calls): ioctl sign-extension ioctl 5c8f3339071a3d42 WARNING pid 95424 (calls): ioctl sign-extension ioctl dd6c226f19c05a40 WARNING pid 95424 (calls): ioctl sign-extension ioctl 9264a87e324e1f4a WARNING pid 95443 (calls): ioctl sign-extension ioctl 3d662cbe076e WARNING pid 95499 (calls): ioctl sign-extension ioctl 61a0a4ec7de1 WARNING pid 95515 (calls): ioctl sign-extension ioctl 6208c1b1cf7d WARNING pid 95566 (calls): ioctl sign-extension ioctl 5c56cd0c975650b4 WARNING pid 95582 (calls): ioctl sign-extension ioctl 7a17a3ed7913 WARNING pid 95624 (calls): ioctl sign-extension ioctl 14fdb60da2aa3bfb WARNING pid 95635 (calls): ioctl sign-extension ioctl 9814ba016139d3eb WARNING pid 95689 (calls): ioctl sign-extension ioctl 1fa30825dd24 WARNING pid 95698 (calls): ioctl sign-extension ioctl 356d837b2dab WARNING pid 95920 (calls): ioctl sign-extension ioctl 1dd1e2822cc WARNING pid 95961 (calls): ioctl sign-extension ioctl 7469a5580d8dab52 WARNING pid 95981 (calls): ioctl sign-extension ioctl bec4a5a745f14b28 WARNING pid 95982 (calls): ioctl sign-extension ioctl 56af5209a2fd23e2 WARNING pid 96059 (calls): ioctl sign-extension ioctl f100033a211 WARNING pid 96166 (calls): ioctl sign-extension ioctl fab0ce6f40cb3bf2 WARNING pid 97142 (calls): ioctl sign-extension ioctl c53ad13a1fbaabf4 WARNING pid 97371 (calls): ioctl sign-extension ioctl d1acc1668653c158 WARNING pid 97538 (calls): ioctl sign-extension ioctl 4817763af1b2 WARNING pid 97556 (calls): ioctl sign-extension ioctl c33fb5a9cb368ce9 WARNING pid 97584 (calls): ioctl sign-extension ioctl 5e18e2c2ae58 WARNING pid 97593 (calls): ioctl sign-extension ioctl 728fc3986d3 WARNING pid 97630 (calls): ioctl sign-extension ioctl 3f18cc619370 WARNING pid 97709 (calls): ioctl sign-extension ioctl 99435db70502f0bd WARNING pid 97714 (calls): ioctl sign-extension ioctl 2f8391ca6f5e WARNING pid 97789 (calls): ioctl sign-extension ioctl 706c49e2d961 WARNING pid 97803 (calls): ioctl sign-extension ioctl 8e4fb0413243dc54 WARNING pid 97856 (calls): ioctl sign-extension ioctl 400f4f2d6b68 Aug 29 08:42:11 mercat1 su[34558]: pho to root on /dev/pts/1 root@mercat1:~ # procstat -k 97867 PID TID COMM TDNAME KSTACK 97867 100382 syscall4 test mi_switch thread_suspend_check ast doreti_ast 97867 100701 syscall4 calls mi_switch thread_suspend_check ast doreti_ast 97867 103566 syscall4 - mi_switch thread_suspend_check ast doreti_ast 97867 104606 syscall4 calls __mtx_lock_sleep __mtx_lock_flags sleepq_catch_signals sleepq_wait_sig _cv_wait_sig mseq_read devfs_read_f dofileread kern_preadv freebsd6_pread amd64_syscall fast_syscall_common 97867 104634 syscall4 calls mi_switch thread_suspend_check ast fast_syscall_common 97867 104685 syscall4 calls mi_switch thread_suspend_check ast fast_syscall_common 97867 104702 syscall4 calls mi_switch thread_suspend_check ast fast_syscall_common 97867 104707 syscall4 calls mi_switch thread_suspend_check ast doreti_ast 97867 104721 syscall4 calls mi_switch thread_suspend_check ast doreti_ast 97867 104723 syscall4 calls mi_switch thread_suspend_check ast doreti_ast 97867 104816 syscall4 calls mi_switch thread_suspend_check ast doreti_ast 97867 104829 syscall4 calls mi_switch thread_suspend_check ast doreti_ast 97867 105211 syscall4 calls mi_switch thread_suspend_check ast doreti_ast 97867 105252 syscall4 calls mi_switch thread_suspend_check ast doreti_ast 97867 105391 syscall4 calls mi_switch thread_suspend_switch thread_single pre_execve sys_fexecve amd64_syscall fast_syscall_common 97867 105392 syscall4 calls mi_switch thread_suspend_check ast doreti_ast 97867 105393 syscall4 calls mi_switch thread_suspend_check ast doreti_ast 97867 105394 syscall4 - mi_switch thread_suspend_check ast doreti_ast root@mercat1:~ # ps -lHp97867 UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND 65534 97867 1 0 52 0 33004 3932 - TC - 0:00.00 /tmp/syscall4 65534 97867 1 0 52 0 33004 3932 - TC - 0:00.00 /tmp/syscall4 65534 97867 1 0 20 0 33004 3932 - TC - 0:00.00 /tmp/syscall4 65534 97867 1 0 103 0 33004 3932 seqqin RC - 528:21.86 /tmp/syscall4 65534 97867 1 0 52 0 33004 3932 - TC - 0:00.00 /tmp/syscall4 65534 97867 1 0 52 0 33004 3932 - TC - 0:00.00 /tmp/syscall4 65534 97867 1 0 52 0 33004 3932 - TC - 0:00.00 /tmp/syscall4 65534 97867 1 0 52 0 33004 3932 - TC - 0:00.00 /tmp/syscall4 65534 97867 1 0 52 0 33004 3932 - TC - 0:00.00 /tmp/syscall4 65534 97867 1 0 52 0 33004 3932 - TC - 0:00.00 /tmp/syscall4 65534 97867 1 0 52 0 33004 3932 - TC - 0:00.00 /tmp/syscall4 65534 97867 1 0 52 0 33004 3932 - TC - 0:00.00 /tmp/syscall4 65534 97867 1 0 52 0 33004 3932 - TC - 0:00.00 /tmp/syscall4 65534 97867 1 0 52 0 33004 3932 - TC - 0:00.00 /tmp/syscall4 65534 97867 1 0 52 0 33004 3932 - TC - 0:00.00 /tmp/syscall4 65534 97867 1 0 52 0 33004 3932 - TC - 0:00.00 /tmp/syscall4 65534 97867 1 0 52 0 33004 3932 - TC - 0:00.00 /tmp/syscall4 65534 97867 1 0 52 0 33004 3932 - TC - 0:00.00 /tmp/syscall4 root@mercat1:~ # root@mercat1:~ # KDB: enter: Break to debugger [ thread pid 38017 tid 101847 ] Stopped at kdb_alt_break_internal+0x106: movq $0,kdb_why db> run pho db:0:pho> set $lines 20000 db:0:pho> run pho1 db:1:pho1> bt Tracing pid 38017 tid 101847 td 0xfffff801d7085000 kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe00aed8d4e0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00aed8d4f0 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe00aed8d520 uart_intr() at uart_intr+0x138/frame 0xfffffe00aed8d560 intr_event_handle() at intr_event_handle+0xb5/frame 0xfffffe00aed8d5b0 intr_execute_handlers() at intr_execute_handlers+0x55/frame 0xfffffe00aed8d5e0 lapic_handle_intr() at lapic_handle_intr+0x5f/frame 0xfffffe00aed8d600 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00aed8d600 --- interrupt, rip = 0xffffffff80bb40a9, rsp = 0xfffffe00aed8d6d0, rbp = 0xfffffe00aed8d710 --- __mtx_lock_flags() at __mtx_lock_flags+0x19/frame 0xfffffe00aed8d710 pmap_enter() at pmap_enter+0x2ba/frame 0xfffffe00aed8d7d0 vm_fault_hold() at vm_fault_hold+0x1799/frame 0xfffffe00aed8d910 vm_fault() at vm_fault+0x60/frame 0xfffffe00aed8d950 trap_pfault() at trap_pfault+0x188/frame 0xfffffe00aed8d9a0 trap() at trap+0x469/frame 0xfffffe00aed8dab0 calltrap() at calltrap+0x8/frame 0xfffffe00aed8dab0 --- trap 0xc, rip = 0x800433855, rsp = 0x7fffffffd978, rbp = 0x7fffffffd9c0 --- db:1:pho1> show allpcpu Current CPU: 0 cpuid = 0 dynamic pcpu = 0xb98600 curthread = 0xfffff801d7085000: pid 38017 tid 101847 "pkill" curpcb = 0xfffffe00aed8db80 fpcurthread = 0xfffff801d7085000: pid 38017 "pkill" idlethread = 0xfffff800037ae000: tid 100003 "idle: cpu0" curpmap = 0xfffff80370bcb130 tssp = 0xffffffff82006f60 commontssp = 0xffffffff82006f60 rsp0 = 0xfffffe00aed8db80 gs32p = 0xffffffff8200d7a8 ldt = 0xffffffff8200d7e8 tss = 0xffffffff8200d7d8 tlb gen = 62587060 curvnet = 0 spin locks held: cpuid = 1 dynamic pcpu = 0xfffffe0084fec600 curthread = 0xfffff800037af5a0: pid 11 tid 100004 "idle: cpu1" curpcb = 0xfffffe007676bb80 fpcurthread = none idlethread = 0xfffff800037af5a0: tid 100004 "idle: cpu1" curpmap = 0xffffffff81fb9b30 tssp = 0xffffffff82006fc8 commontssp = 0xffffffff82006fc8 rsp0 = 0xfffffe007676bb80 gs32p = 0xffffffff8200d810 ldt = 0xffffffff8200d850 tss = 0xffffffff8200d840 tlb gen = 62587060 curvnet = 0 spin locks held: cpuid = 2 dynamic pcpu = 0xfffffe0084ff8600 curthread = 0xfffff800037af000: pid 11 tid 100005 "idle: cpu2" curpcb = 0xfffffe0076770b80 fpcurthread = none idlethread = 0xfffff800037af000: tid 100005 "idle: cpu2" curpmap = 0xffffffff81fb9b30 tssp = 0xffffffff82007030 commontssp = 0xffffffff82007030 rsp0 = 0xfffffe0076770b80 gs32p = 0xffffffff8200d878 ldt = 0xffffffff8200d8b8 tss = 0xffffffff8200d8a8 tlb gen = 62587060 curvnet = 0 spin locks held: cpuid = 3 dynamic pcpu = 0xfffffe0085004600 curthread = 0xfffff800037b05a0: pid 11 tid 100006 "idle: cpu3" curpcb = 0xfffffe0076775b80 fpcurthread = none idlethread = 0xfffff800037b05a0: tid 100006 "idle: cpu3" curpmap = 0xffffffff81fb9b30 tssp = 0xffffffff82007098 commontssp = 0xffffffff82007098 rsp0 = 0xfffffe0076775b80 gs32p = 0xffffffff8200d8e0 ldt = 0xffffffff8200d920 tss = 0xffffffff8200d910 tlb gen = 62587060 curvnet = 0 spin locks held: cpuid = 4 dynamic pcpu = 0xfffffe0085010600 curthread = 0xfffff800037b0000: pid 11 tid 100007 "idle: cpu4" curpcb = 0xfffffe007677ab80 fpcurthread = none idlethread = 0xfffff800037b0000: tid 100007 "idle: cpu4" curpmap = 0xffffffff81fb9b30 tssp = 0xffffffff82007100 commontssp = 0xffffffff82007100 rsp0 = 0xfffffe007677ab80 gs32p = 0xffffffff8200d948 ldt = 0xffffffff8200d988 tss = 0xffffffff8200d978 tlb gen = 62587060 curvnet = 0 spin locks held: cpuid = 5 dynamic pcpu = 0xfffffe008501c600 curthread = 0xfffff800037b15a0: pid 11 tid 100008 "idle: cpu5" curpcb = 0xfffffe007677fb80 fpcurthread = none idlethread = 0xfffff800037b15a0: tid 100008 "idle: cpu5" curpmap = 0xffffffff81fb9b30 tssp = 0xffffffff82007168 commontssp = 0xffffffff82007168 rsp0 = 0xfffffe007677fb80 gs32p = 0xffffffff8200d9b0 ldt = 0xffffffff8200d9f0 tss = 0xffffffff8200d9e0 tlb gen = 62587060 curvnet = 0 spin locks held: cpuid = 6 dynamic pcpu = 0xfffffe0085028600 curthread = 0xfffff800037b1000: pid 11 tid 100009 "idle: cpu6" curpcb = 0xfffffe0076784b80 fpcurthread = none idlethread = 0xfffff800037b1000: tid 100009 "idle: cpu6" curpmap = 0xffffffff81fb9b30 tssp = 0xffffffff820071d0 commontssp = 0xffffffff820071d0 rsp0 = 0xfffffe0076784b80 gs32p = 0xffffffff8200da18 ldt = 0xffffffff8200da58 tss = 0xffffffff8200da48 tlb gen = 62587060 curvnet = 0 spin locks held: cpuid = 7 dynamic pcpu = 0xfffffe0085034600 curthread = 0xfffff8000379c5a0: pid 11 tid 100010 "idle: cpu7" curpcb = 0xfffffe0076789b80 fpcurthread = none idlethread = 0xfffff8000379c5a0: tid 100010 "idle: cpu7" curpmap = 0xffffffff81fb9b30 tssp = 0xffffffff82007238 commontssp = 0xffffffff82007238 rsp0 = 0xfffffe0076789b80 gs32p = 0xffffffff8200da80 ldt = 0xffffffff8200dac0 tss = 0xffffffff8200dab0 tlb gen = 62587060 curvnet = 0 spin locks held: cpuid = 8 dynamic pcpu = 0xfffffe0085040600 curthread = 0xfffff8000379c000: pid 11 tid 100011 "idle: cpu8" curpcb = 0xfffffe007678eb80 fpcurthread = none idlethread = 0xfffff8000379c000: tid 100011 "idle: cpu8" curpmap = 0xffffffff81fb9b30 tssp = 0xffffffff820072a0 commontssp = 0xffffffff820072a0 rsp0 = 0xfffffe007678eb80 gs32p = 0xffffffff8200dae8 ldt = 0xffffffff8200db28 tss = 0xffffffff8200db18 tlb gen = 62587060 curvnet = 0 spin locks held: cpuid = 9 dynamic pcpu = 0xfffffe008504c600 curthread = 0xfffff8000379d5a0: pid 11 tid 100012 "idle: cpu9" curpcb = 0xfffffe0076793b80 fpcurthread = none idlethread = 0xfffff8000379d5a0: tid 100012 "idle: cpu9" curpmap = 0xffffffff81fb9b30 tssp = 0xffffffff82007308 commontssp = 0xffffffff82007308 rsp0 = 0xfffffe0076793b80 gs32p = 0xffffffff8200db50 ldt = 0xffffffff8200db90 tss = 0xffffffff8200db80 tlb gen = 62587057 curvnet = 0 spin locks held: cpuid = 10 dynamic pcpu = 0xfffffe0085058600 curthread = 0xfffff8000379d000: pid 11 tid 100013 "idle: cpu10" curpcb = 0xfffffe0076798b80 fpcurthread = none idlethread = 0xfffff8000379d000: tid 100013 "idle: cpu10" curpmap = 0xffffffff81fb9b30 tssp = 0xffffffff82007370 commontssp = 0xffffffff82007370 rsp0 = 0xfffffe0076798b80 gs32p = 0xffffffff8200dbb8 ldt = 0xffffffff8200dbf8 tss = 0xffffffff8200dbe8 tlb gen = 62587060 curvnet = 0 spin locks held: cpuid = 11 dynamic pcpu = 0xfffffe0085064600 curthread = 0xfffff8017ae7b000: pid 97867 tid 104606 "calls" curpcb = 0xfffffe00ade27b80 fpcurthread = 0xfffff8017ae7b000: pid 97867 "calls" idlethread = 0xfffff8000379e5a0: tid 100014 "idle: cpu11" curpmap = 0xfffff8024ae5d130 tssp = 0xffffffff820073d8 commontssp = 0xffffffff820073d8 rsp0 = 0xfffffe00ade27b80 gs32p = 0xffffffff8200dc20 ldt = 0xffffffff8200dc60 tss = 0xffffffff8200dc50 tlb gen = 62587060 curvnet = 0 spin locks held: db:1:pho1> show alllocks db:1:pho1> show lockedvnods Locked vnodes db:1:pho1> show allchains chain 1: thread 104510 (pid 31580, sleep) sleeping on 0xffffffff81e90956 "nanslp" chain 2: thread 103445 (pid 36335, bash) sleeping on 0xfffff802f2cc43c0 "select" chain 3: thread 104460 (pid 34558, csh) sleeping on 0xfffff8011c142b08 "pause" chain 4: thread 101336 (pid 34554, su) sleeping on 0xfffff80197542530 "wait" chain 5: thread 100849 (pid 87305, bash) sleeping on 0xfffff801d78fc530 "wait" chain 6: thread 103717 (pid 87141, sshd) sleeping on 0xfffff807c3c887c0 "select" chain 7: thread 104657 (pid 85132, sshd) sleeping on 0xfffff806235a7ec0 "select" chain 8: thread 102877 (pid 83109, aiod4) sleeping on 0xfffff801dc547530 "aiordy" chain 9: thread 101419 (pid 83108, aiod3) sleeping on 0xfffff8019bd7d000 "aiordy" chain 10: thread 101418 (pid 83107, aiod2) sleeping on 0xfffff8019bd7d530 "aiordy" chain 11: thread 103699 (pid 83106, aiod1) sleeping on 0xfffff806be8d3a60 "aiordy" chain 12: thread 104862 (pid 83084, md10) sleeping on 0xfffff8081499e000 "mdwait" chain 13: thread 100977 (pid 83053, sh) sleeping on 0xfffff807a0110530 "wait" chain 14: thread 100716 (pid 83052, tee) sleeping on 0xfffff8051ddf98e8 "piperd" chain 15: thread 101738 (pid 83051, sh) sleeping on 0xfffff801969eca60 "wait" chain 16: thread 100268 (pid 9975, sh) sleeping on 0xfffff8000774d5f0 "piperd" chain 17: thread 100145 (pid 9973, awk) sleeping on 0xfffff80012d572f8 "piperd" chain 18: thread 100253 (pid 9972, sh) sleeping on 0xfffff80012e0e530 "wait" chain 19: thread 100240 (pid 9933, sh) sleeping on 0xfffff80006601000 "wait" chain 20: thread 100204 (pid 7612, sh) blocked on lockmgr 0EEXCL thread 0 (pid -512, ) is inactive chain 21: thread 100188 (pid 2351, bash) sleeping on 0xfffff800066af000 "wait" chain 22: thread 100201 (pid 2349, csh) sleeping on 0xfffff800061145d8 "pause" chain 23: thread 100170 (pid 2348, su) blocked on lockmgr 0EXCL thread 0 (pid -512, ) can run chain 24: thread 100213 (pid 2344, bash) sleeping on 0xfffff80159080530 "wait" chain 25: thread 100154 (pid 2343, sshd) sleeping on 0xfffff80007e8d040 "select" chain 26: thread 100171 (pid 2340, sshd) sleeping on 0xfffff80003fe0cc0 "select" chain 27: thread 100192 (pid 2336, csh) sleeping on 0xfffff8000381f8b0 "ttyin" chain 28: thread 100211 (pid 2335, login) blocked on lockmgr 0-EXCL thread 0 (pid -512, ) is inactive chain 29: thread 100210 (pid 2334, getty) sleeping on 0xfffff8000381fcb0 "ttyin" chain 30: thread 100209 (pid 2333, getty) sleeping on 0xfffff8000433dcb0 "ttyin" chain 31: thread 100208 (pid 2332, getty) sleeping on 0xfffff8000433c0b0 "ttyin" chain 32: thread 100207 (pid 2331, getty) sleeping on 0xfffff8000433c4b0 "ttyin" chain 33: thread 100206 (pid 2330, getty) sleeping on 0xfffff8000433c8b0 "ttyin" chain 34: thread 100205 (pid 2329, getty) sleeping on 0xfffff8000433ccb0 "ttyin" chain 35: thread 100194 (pid 2328, getty) sleeping on 0xfffff8000433b0b0 "ttyin" chain 36: thread 100150 (pid 2327, getty) sleeping on 0xfffff8000433b4b0 "ttyin" chain 37: thread 100140 (pid 2326, getty) sleeping on 0xfffff8000433b8b0 "ttyin" chain 38: thread 100200 (pid 2280, cron) sleeping on 0xffffffff81e90958 "nanslp" chain 39: thread 100175 (pid 2276, sshd) sleeping on 0xfffff80012e85cc0 "select" chain 40: thread 100197 (pid 2247, ntpd) sleeping on 0xfffff8000387d840 "select" chain 41: thread 102739 (pid 2247, ntpd) sleeping on 0xfffff800065e7380 "usem" chain 42: thread 100191 (pid 2184, NLM: master) sleeping on 0xfffff80003813a60 "rpcsvc" chain 43: thread 100159 (pid 2181, rpc.statd) sleeping on 0xfffff80012e85dc0 "select" chain 44: thread 100163 (pid 2135, watchdogd) sleeping on 0xffffffff81e90955 "nanslp" chain 45: thread 100164 (pid 2056, rpcbind) sleeping on 0xfffff80006187d40 "select" chain 46: thread 100178 (pid 2046, syslogd) sleeping on 0xfffff80003c0adc0 "select" chain 47: thread 100174 (pid 1975, devd) sleeping on 0xfffff800079cfc40 "select" chain 48: thread 100153 (pid 1970, dhclient) sleeping on 0xfffff80003c0acc0 "select" chain 49: thread 100172 (pid 1926, dhclient) sleeping on 0xfffff8000387da40 "select" chain 50: thread 100165 (pid 1923, dhclient) sleeping on 0xfffff80007e8ee40 "select" chain 51: thread 100135 (pid 32, vnlru) blocked on lockmgr 05EXCL thread 0 (pid -512, ) is inactive chain 52: thread 100132 (pid 31, syncer) sleeping on 0xffffffff81f66e00 "syncer" chain 53: thread 100128 (pid 30, bufdaemon) sleeping on 0xffffffff81f66298 "qsleep" chain 54: thread 100131 (pid 30, bufspacedaemon-0) sleeping on 0xffffffff81a0a880 "-" chain 55: thread 100133 (pid 30, bufspacedaemon-1) sleeping on 0xffffffff81a12b40 "-" chain 56: thread 100134 (pid 30, bufspacedaemon-2) sleeping on 0xffffffff81a1ae00 "-" chain 57: thread 100136 (pid 30, bufspacedaemon-3) sleeping on 0xffffffff81a230c0 "-" chain 58: thread 100137 (pid 30, bufspacedaemon-4) sleeping on 0xffffffff81a2b380 "-" chain 59: thread 100138 (pid 30, bufspacedaemon-5) sleeping on 0xffffffff81a33640 "-" chain 60: thread 100139 (pid 30, bufspacedaemon-6) sleeping on 0xffffffff81a3b900 "-" chain 61: thread 100151 (pid 30, / worker) sleeping on 0xfffff80007a862e8 "sdflush" chain 62: thread 100196 (pid 30, /tmp worker) sleeping on 0xfffff80007a87ce8 "sdflush" chain 63: thread 100168 (pid 30, /mnt worker) sleeping on 0xfffff800075e36e8 "sdflush" chain 64: thread 100127 (pid 29, vmdaemon) sleeping on 0xffffffff81f80588 "psleep" chain 65: thread 100126 (pid 28, dom0) sleeping on 0xffffffff81fea318 "psleep" chain 66: thread 100129 (pid 28, laundry: dom0) sleeping on 0xffffffff81e8f6d8 "laundp" chain 67: thread 100130 (pid 28, uma) sleeping on 0xffffffff80f0b1c0 "umarcl" chain 68: thread 100120 (pid 27, rand_harvestq) sleeping on 0xffffffff81ced6a0 "-" chain 69: thread 100119 (pid 26, sctp_iterator) sleeping on 0xffffffff81fe8a30 "waiting_for_work" chain 70: thread 100118 (pid 25, soaiod4) sleeping on 0xffffffff81f65c9c "-" chain 71: thread 100117 (pid 24, soaiod3) sleeping on 0xffffffff81f65c9c "-" chain 72: thread 100116 (pid 23, soaiod2) sleeping on 0xffffffff81f65c9c "-" chain 73: thread 100115 (pid 22, soaiod1) sleeping on 0xffffffff81f65c9c "-" chain 74: thread 100096 (pid 21, usbus0) sleeping on 0xfffffe009a1aa460 "-" chain 75: thread 100097 (pid 21, usbus0) sleeping on 0xfffffe009a1aa4b8 "-" chain 76: thread 100098 (pid 21, usbus0) sleeping on 0xfffffe009a1aa510 "-" chain 77: thread 100099 (pid 21, usbus0) sleeping on 0xfffffe009a1aa568 "-" chain 78: thread 100100 (pid 21, usbus0) sleeping on 0xfffffe009a1aa5c0 "-" chain 79: thread 100102 (pid 21, usbus1) sleeping on 0xfffffe0006de6d10 "-" chain 80: thread 100103 (pid 21, usbus1) sleeping on 0xfffffe0006de6d68 "-" chain 81: thread 100104 (pid 21, usbus1) sleeping on 0xfffffe0006de6dc0 "-" chain 82: thread 100105 (pid 21, usbus1) sleeping on 0xfffffe0006de6e18 "-" chain 83: thread 100106 (pid 21, usbus1) sleeping on 0xfffffe0006de6e70 "-" chain 84: thread 100107 (pid 21, usbus2) sleeping on 0xfffffe009a63bd10 "-" chain 85: thread 100108 (pid 21, usbus2) sleeping on 0xfffffe009a63bd68 "-" chain 86: thread 100109 (pid 21, usbus2) sleeping on 0xfffffe009a63bdc0 "-" chain 87: thread 100110 (pid 21, usbus2) sleeping on 0xfffffe009a63be18 "-" chain 88: thread 100111 (pid 21, usbus2) sleeping on 0xfffffe009a63be70 "-" chain 89: thread 100091 (pid 20, doneq0) sleeping on 0xffffffff81bc8a40 "-" chain 90: thread 100092 (pid 20, doneq1) sleeping on 0xffffffff81bc8ac0 "-" chain 91: thread 100093 (pid 20, doneq2) sleeping on 0xffffffff81bc8b40 "-" chain 92: thread 100125 (pid 20, scanner) sleeping on 0xffffffff81bc8908 "-" chain 93: thread 100090 (pid 19, crypto returns 11) sleeping on 0xfffff80003acd450 "crypto_ret_wait" chain 94: thread 100089 (pid 18, crypto returns 10) sleeping on 0xfffff80003acd3f0 "crypto_ret_wait" chain 95: thread 100088 (pid 17, crypto returns 9) sleeping on 0xfffff80003acd390 "crypto_ret_wait" chain 96: thread 100087 (pid 16, crypto returns 8) sleeping on 0xfffff80003acd330 "crypto_ret_wait" chain 97: thread 100086 (pid 15, crypto returns 7) sleeping on 0xfffff80003acd2d0 "crypto_ret_wait" chain 98: thread 100085 (pid 9, crypto returns 6) sleeping on 0xfffff80003acd270 "crypto_ret_wait" chain 99: thread 100084 (pid 8, crypto returns 5) sleeping on 0xfffff80003acd210 "crypto_ret_wait" chain 100: thread 100083 (pid 7, crypto returns 4) sleeping on 0xfffff80003acd1b0 "crypto_ret_wait" chain 101: thread 100082 (pid 6, crypto returns 3) sleeping on 0xfffff80003acd150 "crypto_ret_wait" chain 102: thread 100081 (pid 5, crypto returns 2) sleeping on 0xfffff80003acd0f0 "crypto_ret_wait" chain 103: thread 100080 (pid 4, crypto returns 1) sleeping on 0xfffff80003acd090 "crypto_ret_wait" chain 104: thread 100079 (pid 3, crypto returns 0) sleeping on 0xfffff80003acd030 "crypto_ret_wait" chain 105: thread 100078 (pid 2, crypto) sleeping on 0xffffffff81f7abe8 "crypto_wait" chain 106: thread 100065 (pid 14, sequencer 00) sleeping on 0xfffff80003b2dc88 "seqstate" chain 107: thread 100062 (pid 13, g_event) sleeping on 0xffffffff81fb8ec0 "-" chain 108: thread 100063 (pid 13, g_up) sleeping on 0xffffffff81fb8ed0 "-" chain 109: thread 100064 (pid 13, g_down) sleeping on 0xffffffff81fb8eb8 "-" chain 110: thread 100002 (pid 1, init) sleeping on 0xfffff800037b2530 "wait" chain 111: thread 100001 (pid 10, audit) sleeping on 0xffffffff81fe9740 "audit_worker_cv" chain 112: thread 100000 (pid 0, swapper) sleeping on 0xffffffff81fb8f00 "swapin" chain 113: thread 100017 (pid 0, thread taskq) sleeping on 0xfffff8000369bc00 "-" chain 114: thread 100019 (pid 0, config_0) sleeping on 0xfffff8000369ba00 "-" chain 115: thread 100020 (pid 0, kqueue_ctx taskq) sleeping on 0xfffff8000369b900 "-" chain 116: thread 100021 (pid 0, aiod_kick taskq) sleeping on 0xfffff8000369b800 "-" chain 117: thread 100022 (pid 0, softirq_0) sleeping on 0xfffff8000369b700 "-" chain 118: thread 100023 (pid 0, softirq_1) sleeping on 0xfffff8000369b600 "-" chain 119: thread 100024 (pid 0, softirq_2) sleeping on 0xfffff8000369b500 "-" chain 120: thread 100025 (pid 0, softirq_3) sleeping on 0xfffff8000369b400 "-" chain 121: thread 100026 (pid 0, softirq_4) sleeping on 0xfffff8000369b300 "-" chain 122: thread 100027 (pid 0, softirq_5) sleeping on 0xfffff8000369b200 "-" chain 123: thread 100028 (pid 0, softirq_6) sleeping on 0xfffff8000369b100 "-" chain 124: thread 100029 (pid 0, softirq_7) sleeping on 0xfffff8000369b000 "-" chain 125: thread 100030 (pid 0, softirq_8) sleeping on 0xfffff8000369ae00 "-" chain 126: thread 100031 (pid 0, softirq_9) sleeping on 0xfffff8000369ad00 "-" chain 127: thread 100032 (pid 0, softirq_10) sleeping on 0xfffff8000369ac00 "-" chain 128: thread 100033 (pid 0, softirq_11) sleeping on 0xfffff8000369ab00 "-" chain 129: thread 100034 (pid 0, if_io_tqg_0) sleeping on 0xfffff8000369aa00 "-" chain 130: thread 100035 (pid 0, if_io_tqg_1) sleeping on 0xfffff8000369a900 "-" chain 131: thread 100036 (pid 0, if_io_tqg_2) sleeping on 0xfffff8000369a800 "-" chain 132: thread 100037 (pid 0, if_io_tqg_3) sleeping on 0xfffff8000369a700 "-" chain 133: thread 100038 (pid 0, if_io_tqg_4) sleeping on 0xfffff8000369a600 "-" chain 134: thread 100039 (pid 0, if_io_tqg_5) sleeping on 0xfffff8000369a500 "-" chain 135: thread 100040 (pid 0, if_io_tqg_6) sleeping on 0xfffff8000369a400 "-" chain 136: thread 100041 (pid 0, if_io_tqg_7) sleeping on 0xfffff8000369a300 "-" chain 137: thread 100042 (pid 0, if_io_tqg_8) sleeping on 0xfffff8000369a200 "-" chain 138: thread 100043 (pid 0, if_io_tqg_9) sleeping on 0xfffff8000369a100 "-" chain 139: thread 100044 (pid 0, if_io_tqg_10) sleeping on 0xfffff8000369a000 "-" chain 140: thread 100045 (pid 0, if_io_tqg_11) sleeping on 0xfffff80003699e00 "-" chain 141: thread 100046 (pid 0, if_config_tqg_0) sleeping on 0xfffff80003699d00 "-" chain 142: thread 100061 (pid 0, firmware taskq) sleeping on 0xfffff80003699c00 "-" chain 143: thread 100066 (pid 0, crypto_0) sleeping on 0xfffff80003699b00 "-" chain 144: thread 100067 (pid 0, crypto_1) sleeping on 0xfffff80003699b00 "-" chain 145: thread 100068 (pid 0, crypto_2) sleeping on 0xfffff80003699b00 "-" chain 146: thread 100069 (pid 0, crypto_3) sleeping on 0xfffff80003699b00 "-" chain 147: thread 100070 (pid 0, crypto_4) sleeping on 0xfffff80003699b00 "-" chain 148: thread 100071 (pid 0, crypto_5) sleeping on 0xfffff80003699b00 "-" chain 149: thread 100072 (pid 0, crypto_6) sleeping on 0xfffff80003699b00 "-" chain 150: thread 100073 (pid 0, crypto_7) sleeping on 0xfffff80003699b00 "-" chain 151: thread 100074 (pid 0, crypto_8) sleeping on 0xfffff80003699b00 "-" chain 152: thread 100075 (pid 0, crypto_9) sleeping on 0xfffff80003699b00 "-" chain 153: thread 100076 (pid 0, crypto_10) sleeping on 0xfffff80003699b00 "-" chain 154: thread 100077 (pid 0, crypto_11) sleeping on 0xfffff80003699b00 "-" chain 155: thread 100114 (pid 0, mca taskq) blocked on lockmgr (null)EXCL thread 0 (pid 0, ) is inactive chain 156: thread 100121 (pid 0, acpi_task_0) sleeping on 0xfffff800078d1200 "-" chain 157: thread 100122 (pid 0, acpi_task_1) sleeping on 0xfffff800078d1200 "-" chain 158: thread 100123 (pid 0, acpi_task_2) sleeping on 0xfffff800078d1200 "-" chain 159: thread 100124 (pid 0, CAM taskq) sleeping on 0xfffff80003699a00 "-" db:1:pho1> show mount 0xfffff80007bce000 /dev/gpt/work on / (ufs) 0xfffff80007bcd000 devfs on /dev (devfs) 0xfffff800120e5000 /dev/gpt/tmp on /tmp (ufs) 0xfffff8002dffb000 procfs on /proc (procfs) 0xfffff8017ade6000 /dev/md10a on /mnt (ufs) More info: show mount db:1:pho1> show bufqueues bqempty: 4446 Buf domain 0 freebufs 329 lofreebufs 219 hifreebufs 329 bufspace 151080960 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 14718 (149483520) dirtyq count 0 (0) wakeup 0 lim 25 CPU 8, 15, 9, 0, 10, 7, 10, 9, 10, 0, 2, 6, Locked buffers: 0 space 0 Total buffers: 15579 space 151080960 Buf domain 1 freebufs 1564 lofreebufs 219 hifreebufs 329 bufspace 235425792 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 15 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 13400 (230473728) dirtyq count 15 (491520) wakeup 0 lim 25 CPU 20, 12, 18, 17, 10, 0, 0, 24, 12, 19, 10, 12, Locked buffers: 0 space 0 Total buffers: 16193 space 235425792 Buf domain 2 freebufs 2934 lofreebufs 219 hifreebufs 329 bufspace 235433984 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 12095 (233336832) dirtyq count 0 (0) wakeup 0 lim 25 CPU 3, 16, 5, 9, 16, 8, 13, 9, 0, 3, 2, 20, Locked buffers: 0 space 0 Total buffers: 12978 space 235433984 Buf domain 3 freebufs 329 lofreebufs 219 hifreebufs 329 bufspace 193040384 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 2 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 14646 (189935616) dirtyq count 2 (65536) wakeup 0 lim 25 CPU 17, 18, 6, 0, 12, 18, 21, 8, 7, 11, 22, 15, Locked buffers: 1 space 32768 Total buffers: 15374 space 193040384 Buf domain 4 freebufs 329 lofreebufs 219 hifreebufs 329 bufspace 202915840 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 0 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 14681 (200126464) dirtyq count 0 (0) wakeup 0 lim 25 CPU 11, 2, 22, 18, 1, 1, 9, 11, 11, 3, 16, 18, Locked buffers: 0 space 0 Total buffers: 15218 space 202915840 Buf domain 5 freebufs 4292 lofreebufs 219 hifreebufs 329 bufspace 235433984 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 5 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 10703 (233041920) dirtyq count 5 (163840) wakeup 0 lim 25 CPU 1, 4, 9, 22, 7, 24, 5, 2, 14, 20, 9, 16, Locked buffers: 0 space 0 Total buffers: 13415 space 235433984 Buf domain 6 freebufs 3970 lofreebufs 219 hifreebufs 329 bufspace 235421696 maxbufspace 247939072 hibufspace 247845449 lobufspace 235453176 bufspacethresh 241649312 numdirtybuffers 1 lodirtybuffers 1893 hidirtybuffers 3786 dirtybufthresh 3407 cleanq count 11008 (232521728) dirtyq count 1 (4096) wakeup 0 lim 25 CPU 16, 23, 21, 22, 2, 1, 11, 9, 23, 1, 19, 6, Locked buffers: 0 space 0 Total buffers: 17174 space 235421696 db:1:pho1> show page vm_cnt.v_free_count: 6599035 vm_cnt.v_inactive_count: 256056 vm_cnt.v_active_count: 2497 vm_cnt.v_laundry_count: 24 vm_cnt.v_wire_count: 1283683 vm_cnt.v_free_reserved: 10642 vm_cnt.v_free_min: 51256 vm_cnt.v_free_target: 173098 vm_cnt.v_inactive_target: 259647 db:1:pho1> show pageq pq_free 6599035 dom 0 page_cnt 8123034 free 6599035 pq_act 2497 pq_inact 256056 pq_laund 24 pq_unsw 0 db:0:pho> show freepages DOMAIN: 0 FREE LIST 0: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000401 | 000000 11 (008192K) | 000505 | 000002 10 (004096K) | 001118 | 000003 09 (002048K) | 001740 | 000002 08 (001024K) | 002415 | 000011 07 (000512K) | 003117 | 000034 06 (000256K) | 004313 | 000169 05 (000128K) | 006797 | 000459 04 (000064K) | 013526 | 000637 03 (000032K) | 007210 | 000030 02 (000016K) | 000000 | 000048 01 (000008K) | 000000 | 000042 00 (000004K) | 000001 | 000406 FREE LIST 1: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 -- -- -- -- -- -- 12 (016384K) | 000000 | 000000 11 (008192K) | 000000 | 000000 10 (004096K) | 000000 | 000000 09 (002048K) | 000000 | 000000 08 (001024K) | 000000 | 000000 07 (000512K) | 000000 | 000000 06 (000256K) | 000001 | 000000 05 (000128K) | 000001 | 000000 04 (000064K) | 000002 | 000000 03 (000032K) | 000000 | 000000 02 (000016K) | 000001 | 000000 01 (000008K) | 000000 | 000000 00 (000004K) | 000001 | 000000 db:0:pho> show uma Zone Size Used Free Requests Sleeps Bucket XFree mqnotifier 216 0 0 0 0 61 0 mvdata 64 0 0 0 0 253 0 mqueue 264 0 0 0 0 29 0 mqnode 416 3 33 3 0 29 0 FFS2 dinode 256 619063 1082 24829475 0 65 0 FFS1 dinode 128 0 0 11 0 125 0 FFS inode 160 619063 833 24830568 0 61 0 swblk 136 1398 120547 874804 0 61 0 swpctrie 144 189 10341 68536 0 61 0 selfd 64 672 2738 99077658 0 253 0 rtentry 208 12 102 18 0 61 0 IPsec SA lft_c 16 0 0 0 0 253 0 ripcb 488 1 31 680509 0 29 0 udplite_inpcb 488 0 0 0 0 29 0 sctp_asconf_ack 48 0 0 0 0 253 0 sctp_asconf 40 0 0 0 0 253 0 sctp_stream_msg_out 112 0 0 0 0 125 0 sctp_readq 152 0 0 0 0 61 0 sctp_chunk 152 0 0 0 0 61 0 sctp_raddr 736 0 0 0 0 13 0 sctp_laddr 48 0 0 6 0 253 0 sctp_asoc 2408 0 0 0 0 3 0 sctp_ep 1280 0 0 557097 0 9 0 tcp_log_node 120 0 0 0 0 125 0 tcp_log_bucket 144 0 0 0 0 61 0 tcp_log 408 0 0 0 0 29 0 tcpreass 48 0 0 10880 0 253 0 tfo_ccache_entries 80 0 0 0 0 125 0 tfo 4 0 0 0 0 253 0 sackhole 32 0 0 14042 0 253 0 hostcache 96 0 41 1 0 125 0 syncache 168 0 69 3105 0 61 0 tcptw 88 0 0 1130 0 125 0 tcpcb 976 10 130 579306 0 13 0 tcp_inpcb 488 10 294 579306 0 29 0 udpcb 32 20 2832 560412 0 253 0 udp_inpcb 488 20 348 560412 0 29 0 ipq 56 0 0 0 0 253 0 unpcb 256 24 861 5536750 0 61 0 socket 904 57 939507 8539275 0 13 0 KNOTE 160 0 0 81750637 0 61 0 itimer 352 0 0 0 0 29 0 ksiginfo 112 4231 2524 1213927 0 125 0 AIOLIO 280 0 0 0 0 29 0 AIOCB 752 0 160 1857 0 13 0 AIOP 32 4 616 4 0 253 0 AIO 208 0 684 1526 0 61 0 Mountpoints 2896 5 112 62437 0 3 0 procdesc 136 1 86 935779 0 61 0 pipe 760 9 271 31371575 0 13 0 DIRHASH 1024 11533 803 163326 0 13 0 NCLNODE 744 0 0 0 0 13 0 LTS VFS Cache 368 0 0 0 0 29 0 L VFS Cache 328 53477 135355 425294 0 29 0 STS VFS Cache 148 0 0 0 0 61 0 S VFS Cache 108 566568 58882 35098963 0 160 0 rentr 24 0 2324 9656946 0 253 0 NAMEI 1024 0 976 773184020 0 26 0 BUF TRIE 144 6122 99826 28219968 0 61 0 VNODEPOLL 120 0 0 316 0 125 0 VNODE 632 619199 4813 62391641 0 253 0 vtnet_tx_hdr 24 0 0 0 0 253 0 crypto_session 24 0 0 0 0 253 0 cryptodesc 120 0 0 0 0 125 0 cryptop 128 0 0 0 0 125 0 nvme_request 128 0 0 0 0 125 0 g_bio 408 0 4707 117924926 0 159 0 FPU_save_area 832 0 0 0 0 13 0 DMAR_MAP_ENTRY 120 0 0 0 0 125 0 ttyoutq 256 181 119 2851 0 61 0 ttyinq 160 345 231 5356 0 61 0 epoch_record pcpu 256 4 12 4 0 61 0 mbuf_extpgs 256 0 0 0 0 61 0 mbuf_jumbo_16k 16384 0 0 0 0 1 0 mbuf_jumbo_9k 9216 0 0 0 0 1 0 mbuf_jumbo_page 4096 0 2 11287665 0 1 0 mbuf_cluster 2048 9190 644 225588 0 5 0 mbuf 256 6257 5502 198548450 0 253 0 mbuf_packet 256 22 3024 23226853 0 253 0 audit_record 1280 0 0 0 0 9 0 domainset 40 0 2542 288 0 253 0 cpuset 104 27 1523 649 0 125 0 THREAD 1440 4391 13565 43299649 0 9 0 PROC 1328 83 4174 45900768 0 9 0 MAC labels 40 0 0 0 0 253 0 pbuf 984 0 835 0 0 1 0 vnpbuf 984 0 0 2484 0 13 0 clpbuf 984 0 94 3849479 242343 5 0 mdpbuf 984 0 0 274353 0 1 0 nfspbuf 984 0 0 0 0 5 0 swwbuf 984 0 12 460182 0 1 0 swrbuf 984 0 70 172261 0 5 0 umtx_shm 88 0 0 35 0 125 0 umtx pi 96 0 0 44823834 0 125 0 TURNSTILE 136 17957 383 17957 0 61 0 rangeset pctrie nodes 144 0 0 0 0 61 0 rl_entry 40 3094 2945 3094 0 253 0 filedesc0 1104 84 1470 45900769 0 16 0 Files 80 148 941485 809163503 0 125 0 SLEEPQUEUE 88 17957 333 17957 0 125 0 65536 65536 2 0 15 0 1 0 65536 65536 0 0 1030 0 1 0 65536 65536 1 13 142509 0 1 0 65536 65536 3 0 26 0 1 0 65536 65536 4 0 2160 0 1 0 65536 65536 0 0 0 0 1 0 65536 65536 1 1 3 0 1 0 65536 65536 2 0 5 0 1 0 32768 32768 1 0 23 0 1 0 32768 32768 1 0 1032 0 1 0 32768 32768 2 45 1255145 0 1 0 32768 32768 12 17 582095 0 1 0 32768 32768 1 0 2492 0 1 0 32768 32768 12 0 12 0 1 0 32768 32768 1 0 4 0 1 0 32768 32768 2 0 6 0 1 0 16384 16384 0 3 80 0 1 0 16384 16384 0 0 3 0 1 0 16384 16384 39 15 1098 0 1 0 16384 16384 12 9 2435704 0 1 0 16384 16384 15 0 2845 0 1 0 16384 16384 0 0 0 0 1 0 16384 16384 0 1 5 0 1 0 16384 16384 1 0 5 0 1 0 8192 8192 1 24 820980 0 1 0 8192 8192 6 3 48865 0 1 0 8192 8192 14 19 49092 0 1 0 8192 8192 86 2 385354 0 1 0 8192 8192 13 10 1702 0 1 0 8192 8192 12 0 12 0 1 0 8192 8192 4 0 8 0 1 0 8192 8192 1 0 11 0 1 0 4096 4096 2 18 107307 0 1 0 4096 4096 13 20 3879749 0 1 0 4096 4096 51 20 734298 0 1 0 4096 4096 3 17 1176854 0 1 0 4096 4096 16 12 503714 0 1 0 4096 4096 444 1394 47215257 0 2 0 4096 4096 2 0 9 0 1 0 4096 4096 7 0 30 0 1 0 2048 2048 2 78 183122 0 5 0 2048 2048 3 3 16 0 5 0 2048 2048 112 100 17464409 0 5 0 2048 2048 2 60 20280908 0 5 0 2048 2048 10 60 557774 0 5 0 2048 2048 0 0 0 0 5 0 2048 2048 0 0 13 0 5 0 2048 2048 0 70 504 0 5 0 1024 1024 3 197 215566 0 13 0 1024 1024 0 0 1 0 13 0 1024 1024 500055 185 181339374 0 13 0 1024 1024 40 144 7098227 0 13 0 1024 1024 158 214 2234029 0 13 0 1024 1024 4 216 299863 0 13 0 1024 1024 1 183 34005089 0 13 0 1024 1024 16 68 270 0 13 0 512 512 1 4255 13514594 0 29 0 512 512 89 93 88146 0 29 0 512 512 152 345 36579398 0 29 0 512 512 14 350 16749303 0 29 0 512 512 35 350 1122455 0 29 0 512 512 4258 75 4258 0 29 0 512 512 13 36 48 0 29 0 512 512 1256 445 18145 0 29 0 256 256 56 4309 4122809 0 61 0 256 256 18 1182 4284569 0 61 0 256 256 500700 9930 39002290 0 61 0 256 256 254 7141 156492240 0 61 0 256 256 170 745 5778188 0 61 0 256 256 11 244 50375 0 61 0 256 256 64 791 7658269 0 61 0 256 256 385 455 1612 0 61 0 128 128 69 1636 934203 0 125 0 128 128 4 1763 1878270 0 125 0 128 128 873 11186 97809741 0 125 0 128 128 36463 38619 34809946 0 125 0 128 128 17305 88126 325240786 0 125 0 128 128 39 1387 106184 0 125 0 128 128 2431 1506 186039 0 125 0 128 128 404 1208 4533 0 125 0 64 64 24 3820 307580968 0 253 0 64 64 60 312 2194189 0 253 0 64 64 509443 8132 71396377 0 253 0 64 64 121 4343 91986218 0 253 0 64 64 9307 3093 233864230 0 253 0 64 64 15 3829 1176752 0 253 0 64 64 525 3071 81862 0 253 0 64 64 339 2947 4016 0 253 0 32 32 17972 3108 156143 0 253 0 32 32 147 3573 91956178 0 253 0 32 32 273 3323 5622750 0 253 0 32 32 66 3034 23316721 0 253 0 32 32 568 2904 2407123 0 253 0 32 32 118 3850 35411058 0 253 0 32 32 3493 3079 186983 0 253 0 32 32 479 52469 7456240 0 253 0 16 16 23 3214 8553642 0 253 0 16 16 29 1216 53 0 253 0 16 16 574 2912 14697017 0 253 0 16 16 3 2985 5226989 0 253 0 16 16 621 3114 1743624 0 253 0 16 16 6 2982 725 0 253 0 16 16 2813 2416 3510 0 253 0 16 16 1278 3148821 60174909 0 253 0 mt_zone 24 430 566 564 0 253 0 mt_stats_zone 64 430 530 564 0 253 0 64 pcpu 8 3016 2872 97621 0 253 0 fakepg 104 0 52554 1619991 0 125 0 VMSPACE 2552 48 4138 45880361 0 8 0 MAP ENTRY 112 2317 37128 2524557940 0 253 0 KMAP ENTRY 112 9 269 82 0 125 0 MAP 232 2 65 2 0 61 0 RADIX NODE 144 65025 303621 996041670 0 253 0 VM OBJECT 256 642058 477647 571115568 0 253 0 vmem btag 56 63608 59132 1874833 0 253 0 vmem 1856 0 1 0 0 5 0 256 Bucket 2048 15303 10518 6069822 0 253 0 128 Bucket 1024 1145 5775 1050737 0 38 0 64 Bucket 512 569 1846 1147529 0 29 0 32 Bucket 256 5062 1777 2587514 0 61 0 16 Bucket 128 303 1835 1537397 0 125 0 12 Bucket 96 309 1864 386218 0 125 0 8 Bucket 64 319 3400 1232561 0 253 0 6 Bucket 48 2831 1817 566023 0 253 0 4 Bucket 32 4202 1378 21043771 0 253 0 UMA Hash 256 0 11 196 0 0 0 UMA Slabs 112 151418 16 488094 0 0 0 UMA Zones 1960 87 21 31475 0 0 0 UMA Kegs 216 79 34 31467 0 0 0 db:0:pho> show umacache Zone Size Used Free Requests Bucket kstack_cache 16384 4266 112 43299522 1 vm pgcache 104 799360 3552 896349822 253 vm pgcache 104 608867 3720 4156883897 253 buf free cache 984 139939 9301 38435746 64 buffer arena-65 65536 0 4 3485035 1 buffer arena-61 61440 0 0 3484924 1 buffer arena-57 57344 0 0 3484924 1 buffer arena-53 53248 0 0 3484924 1 buffer arena-49 49152 0 0 3484924 1 buffer arena-45 45056 0 0 3484924 1 buffer arena-40 40960 0 0 3484924 1 buffer arena-36 36864 0 0 3484924 1 buffer arena-32 32768 240752 10141 13640313 2 buffer arena-28 28672 0 0 3484924 1 buffer arena-24 24576 0 0 3484924 1 buffer arena-20 20480 0 0 3484924 1 buffer arena-16 16384 109419 702 7268604 1 buffer arena-12 12288 0 0 3484924 1 buffer arena-81 8192 0 0 3484924 1 buffer arena-40 4096 0 10 3484957 1 db:0:pho> acttrace Tracing command pkill pid 38017 tid 101847 td 0xfffff801d7085000 (CPU 0) kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe00aed8d4e0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00aed8d4f0 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe00aed8d520 uart_intr() at uart_intr+0x138/frame 0xfffffe00aed8d560 intr_event_handle() at intr_event_handle+0xb5/frame 0xfffffe00aed8d5b0 intr_execute_handlers() at intr_execute_handlers+0x55/frame 0xfffffe00aed8d5e0 lapic_handle_intr() at lapic_handle_intr+0x5f/frame 0xfffffe00aed8d600 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00aed8d600 --- interrupt, rip = 0xffffffff80bb40a9, rsp = 0xfffffe00aed8d6d0, rbp = 0xfffffe00aed8d710 --- __mtx_lock_flags() at __mtx_lock_flags+0x19/frame 0xfffffe00aed8d710 pmap_enter() at pmap_enter+0x2ba/frame 0xfffffe00aed8d7d0 vm_fault_hold() at vm_fault_hold+0x1799/frame 0xfffffe00aed8d910 vm_fault() at vm_fault+0x60/frame 0xfffffe00aed8d950 trap_pfault() at trap_pfault+0x188/frame 0xfffffe00aed8d9a0 trap() at trap+0x469/frame 0xfffffe00aed8dab0 calltrap() at calltrap+0x8/frame 0xfffffe00aed8dab0 --- trap 0xc, rip = 0x800433855, rsp = 0x7fffffffd978, rbp = 0x7fffffffd9c0 --- Tracing command syscall4 pid 97867 tid 104606 td 0xfffff8017ae7b000 (CPU 11) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bebdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bebe10 trap() at trap+0x46/frame 0xfffffe0006bebf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bebf20 --- trap 0x13, rip = 0xffffffff80c30037, rsp = 0xfffffe00ade276f0, rbp = 0xfffffe00ade27740 --- sleepq_catch_signals() at sleepq_catch_signals+0x37/frame 0xfffffe00ade27740 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ade27770 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ade277d0 mseq_read() at mseq_read+0xff/frame 0xfffffe00ade27840 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00ade278a0 dofileread() at dofileread+0x81/frame 0xfffffe00ade278f0 kern_preadv() at kern_preadv+0x5f/frame 0xfffffe00ade27930 freebsd6_pread() at freebsd6_pread+0x8b/frame 0xfffffe00ade27980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00ade27ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ade27ab0 --- syscall (0, FreeBSD ELF64, nosys), rip = 0x8004442ba, rsp = 0x7fffdfbfbf48, rbp = 0x7fffdfbfbfb0 --- Tracing command idle pid 11 tid 100004 td 0xfffff800037af5a0 (CPU 1) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006b73df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006b73e10 trap() at trap+0x46/frame 0xfffffe0006b73f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b73f20 --- trap 0x13, rip = 0xffffffff81159db6, rsp = 0xfffffe007676b910, rbp = 0xfffffe007676b910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007676b910 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe007676b960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007676b980 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe007676b9a0 sched_idletd() at sched_idletd+0x442/frame 0xfffffe007676ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe007676bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007676bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffff800037af000 (CPU 2) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006b7fdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006b7fe10 trap() at trap+0x46/frame 0xfffffe0006b7ff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b7ff20 --- trap 0x13, rip = 0xffffffff81159db6, rsp = 0xfffffe0076770910, rbp = 0xfffffe0076770910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076770910 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076770960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076770980 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe00767709a0 sched_idletd() at sched_idletd+0x442/frame 0xfffffe0076770a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076770ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076770ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xfffff800037b05a0 (CPU 3) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006b8bdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006b8be10 trap() at trap+0x46/frame 0xfffffe0006b8bf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b8bf20 --- trap 0x13, rip = 0xffffffff81159db6, rsp = 0xfffffe0076775910, rbp = 0xfffffe0076775910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076775910 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076775960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076775980 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe00767759a0 sched_idletd() at sched_idletd+0x442/frame 0xfffffe0076775a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076775ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076775ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100007 td 0xfffff800037b0000 (CPU 4) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006b97df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006b97e10 trap() at trap+0x46/frame 0xfffffe0006b97f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b97f20 --- trap 0x13, rip = 0xffffffff81159db6, rsp = 0xfffffe007677a910, rbp = 0xfffffe007677a910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007677a910 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe007677a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007677a980 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe007677a9a0 sched_idletd() at sched_idletd+0x442/frame 0xfffffe007677aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007677aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007677aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xfffff800037b15a0 (CPU 5) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006ba3df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006ba3e10 trap() at trap+0x46/frame 0xfffffe0006ba3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006ba3f20 --- trap 0x13, rip = 0xffffffff81159db6, rsp = 0xfffffe007677f910, rbp = 0xfffffe007677f910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007677f910 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe007677f960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007677f980 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe007677f9a0 sched_idletd() at sched_idletd+0x442/frame 0xfffffe007677fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007677fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007677fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xfffff800037b1000 (CPU 6) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bafdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bafe10 trap() at trap+0x46/frame 0xfffffe0006baff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006baff20 --- trap 0x13, rip = 0xffffffff81159db6, rsp = 0xfffffe0076784910, rbp = 0xfffffe0076784910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076784910 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076784960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076784980 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe00767849a0 sched_idletd() at sched_idletd+0x442/frame 0xfffffe0076784a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076784ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076784ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xfffff8000379c5a0 (CPU 7) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bbbdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bbbe10 trap() at trap+0x46/frame 0xfffffe0006bbbf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bbbf20 --- trap 0x13, rip = 0xffffffff81159db6, rsp = 0xfffffe0076789910, rbp = 0xfffffe0076789910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076789910 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076789960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076789980 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe00767899a0 sched_idletd() at sched_idletd+0x442/frame 0xfffffe0076789a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076789ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076789ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffff8000379c000 (CPU 8) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bc7df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bc7e10 trap() at trap+0x46/frame 0xfffffe0006bc7f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bc7f20 --- trap 0x13, rip = 0xffffffff81159db6, rsp = 0xfffffe007678e910, rbp = 0xfffffe007678e910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007678e910 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe007678e960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007678e980 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe007678e9a0 sched_idletd() at sched_idletd+0x442/frame 0xfffffe007678ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe007678eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007678eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100012 td 0xfffff8000379d5a0 (CPU 9) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bd3df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bd3e10 trap() at trap+0x46/frame 0xfffffe0006bd3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bd3f20 --- trap 0x13, rip = 0xffffffff81159db6, rsp = 0xfffffe0076793910, rbp = 0xfffffe0076793910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076793910 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076793960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076793980 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe00767939a0 sched_idletd() at sched_idletd+0x442/frame 0xfffffe0076793a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076793ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076793ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xfffff8000379d000 (CPU 10) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bdfdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bdfe10 trap() at trap+0x46/frame 0xfffffe0006bdff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bdff20 --- trap 0x13, rip = 0xffffffff81159db6, rsp = 0xfffffe0076798910, rbp = 0xfffffe0076798910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076798910 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076798960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076798980 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe00767989a0 sched_idletd() at sched_idletd+0x442/frame 0xfffffe0076798a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076798ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076798ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db:0:pho> ps pid ppid pgrp uid state wmesg wchan cmd 38017 83053 7612 0 R+ CPU 0 pkill 31580 9972 9933 0 S nanslp 0xffffffff81e90956 sleep 36335 34558 36335 0 S+ select 0xfffff802f2cc43c0 bash 34558 34554 34558 0 S+ pause 0xfffff8011c142b08 csh 34554 87305 34554 2006 S+ wait 0xfffff80197542530 su 87305 87141 87305 2006 Ss+ wait 0xfffff801d78fc530 bash 87141 85132 85132 2006 S select 0xfffff807c3c887c0 sshd 85132 2276 85132 0 Ss select 0xfffff806235a7ec0 sshd 97867 1 96118 65534 T (threaded) syscall4 103566 s syscall4 100382 s test 100701 s calls 104606 Run seqqin 0xfffff80003b2dc68 calls 104634 s calls 104685 s calls 104702 s calls 104707 s calls 104721 s calls 104723 s calls 104816 s calls 104829 s calls 105211 s calls 105252 s calls 105391 s calls 105392 s calls 105393 s calls 105394 s syscall4 83109 0 0 0 DL aiordy 0xfffff801dc547530 [aiod4] 83108 0 0 0 DL aiordy 0xfffff8019bd7d000 [aiod3] 83107 0 0 0 DL aiordy 0xfffff8019bd7d530 [aiod2] 83106 0 0 0 DL aiordy 0xfffff806be8d3a60 [aiod1] 83084 0 0 0 DL mdwait 0xfffff8081499e000 [md10] 83053 83051 7612 0 S+ wait 0xfffff807a0110530 sh 83052 7612 7612 0 S+ piperd 0xfffff8051ddf98e8 tee 83051 7612 7612 0 S+ wait 0xfffff801969eca60 sh 9975 9933 9933 0 S piperd 0xfffff8000774d5f0 sh 9973 9933 9933 0 S piperd 0xfffff80012d572f8 awk 9972 9933 9933 0 S wait 0xfffff80012e0e530 sh 9933 1 9933 0 SWs wait 0xfffff80006601000 sh 7612 2351 7612 0 S+ wait 0xfffff80007eb9000 sh 2351 2349 2351 0 SW+ wait 0xfffff800066af000 bash 2349 2348 2349 0 SW+ pause 0xfffff800061145d8 csh 2348 2344 2348 2006 SW+ wait 0xfffff8000777ca60 su 2344 2343 2344 2006 SWs+ wait 0xfffff80159080530 bash 2343 2340 2340 2006 S select 0xfffff80007e8d040 sshd 2340 2276 2340 0 Ss select 0xfffff80003fe0cc0 sshd 2336 2335 2336 0 S+ ttyin 0xfffff8000381f8b0 csh 2335 1 2335 0 SWs+ wait 0xfffff80159080a60 login 2334 1 2334 0 Ss+ ttyin 0xfffff8000381fcb0 getty 2333 1 2333 0 Ss+ ttyin 0xfffff8000433dcb0 getty 2332 1 2332 0 Ss+ ttyin 0xfffff8000433c0b0 getty 2331 1 2331 0 Ss+ ttyin 0xfffff8000433c4b0 getty 2330 1 2330 0 Ss+ ttyin 0xfffff8000433c8b0 getty 2329 1 2329 0 Ss+ ttyin 0xfffff8000433ccb0 getty 2328 1 2328 0 Ss+ ttyin 0xfffff8000433b0b0 getty 2327 1 2327 0 Ss+ ttyin 0xfffff8000433b4b0 getty 2326 1 2326 0 Ss+ ttyin 0xfffff8000433b8b0 getty 2280 1 2280 0 Ss nanslp 0xffffffff81e90958 cron 2276 1 2276 0 Ss select 0xfffff80012e85cc0 sshd 2247 1 2247 123 Ss (threaded) ntpd 100197 S select 0xfffff8000387d840 ntpd 102739 S usem 0xfffff800065e7380 ntpd 2184 1 2184 0 Ss rpcsvc 0xfffff80003813a60 NLM: master 2181 1 2181 0 Ss select 0xfffff80012e85dc0 rpc.statd 2135 1 2135 0 Ss nanslp 0xffffffff81e90955 watchdogd 2056 1 2056 0 Ss select 0xfffff80006187d40 rpcbind 2046 1 2046 0 Ss select 0xfffff80003c0adc0 syslogd 1975 1 1975 0 Ss select 0xfffff800079cfc40 devd 1970 1 1970 65 Ss select 0xfffff80003c0acc0 dhclient 1926 1 1926 0 Ss select 0xfffff8000387da40 dhclient 1923 1 1923 0 Ss select 0xfffff80007e8ee40 dhclient 32 0 0 0 DL vlruwt 0xfffff80004361a60 [vnlru] 31 0 0 0 DL syncer 0xffffffff81f66e00 [syncer] 30 0 0 0 DL (threaded) [bufdaemon] 100128 D qsleep 0xffffffff81f66298 [bufdaemon] 100131 D - 0xffffffff81a0a880 [bufspacedaemon-0] 100133 D - 0xffffffff81a12b40 [bufspacedaemon-1] 100134 D - 0xffffffff81a1ae00 [bufspacedaemon-2] 100136 D - 0xffffffff81a230c0 [bufspacedaemon-3] 100137 D - 0xffffffff81a2b380 [bufspacedaemon-4] 100138 D - 0xffffffff81a33640 [bufspacedaemon-5] 100139 D - 0xffffffff81a3b900 [bufspacedaemon-6] 100151 D sdflush 0xfffff80007a862e8 [/ worker] 100196 D sdflush 0xfffff80007a87ce8 [/tmp worker] 100168 D sdflush 0xfffff800075e36e8 [/mnt worker] 29 0 0 0 DL psleep 0xffffffff81f80588 [vmdaemon] 28 0 0 0 DL (threaded) [pagedaemon] 100126 D psleep 0xffffffff81fea318 [dom0] 100129 D laundp 0xffffffff81e8f6d8 [laundry: dom0] 100130 D umarcl 0xffffffff80f0b1c0 [uma] 27 0 0 0 DL - 0xffffffff81ced6a0 [rand_harvestq] 26 0 0 0 DL waiting 0xffffffff81fe8a30 [sctp_iterator] 25 0 0 0 DL - 0xffffffff81f65c9c [soaiod4] 24 0 0 0 DL - 0xffffffff81f65c9c [soaiod3] 23 0 0 0 DL - 0xffffffff81f65c9c [soaiod2] 22 0 0 0 DL - 0xffffffff81f65c9c [soaiod1] 21 0 0 0 DL (threaded) [usb] 100096 D - 0xfffffe009a1aa460 [usbus0] 100097 D - 0xfffffe009a1aa4b8 [usbus0] 100098 D - 0xfffffe009a1aa510 [usbus0] 100099 D - 0xfffffe009a1aa568 [usbus0] 100100 D - 0xfffffe009a1aa5c0 [usbus0] 100102 D - 0xfffffe0006de6d10 [usbus1] 100103 D - 0xfffffe0006de6d68 [usbus1] 100104 D - 0xfffffe0006de6dc0 [usbus1] 100105 D - 0xfffffe0006de6e18 [usbus1] 100106 D - 0xfffffe0006de6e70 [usbus1] 100107 D - 0xfffffe009a63bd10 [usbus2] 100108 D - 0xfffffe009a63bd68 [usbus2] 100109 D - 0xfffffe009a63bdc0 [usbus2] 100110 D - 0xfffffe009a63be18 [usbus2] 100111 D - 0xfffffe009a63be70 [usbus2] 20 0 0 0 DL (threaded) [cam] 100091 D - 0xffffffff81bc8a40 [doneq0] 100092 D - 0xffffffff81bc8ac0 [doneq1] 100093 D - 0xffffffff81bc8b40 [doneq2] 100125 D - 0xffffffff81bc8908 [scanner] 19 0 0 0 DL crypto_ 0xfffff80003acd450 [crypto returns 11] 18 0 0 0 DL crypto_ 0xfffff80003acd3f0 [crypto returns 10] 17 0 0 0 DL crypto_ 0xfffff80003acd390 [crypto returns 9] 16 0 0 0 DL crypto_ 0xfffff80003acd330 [crypto returns 8] 15 0 0 0 DL crypto_ 0xfffff80003acd2d0 [crypto returns 7] 9 0 0 0 DL crypto_ 0xfffff80003acd270 [crypto returns 6] 8 0 0 0 DL crypto_ 0xfffff80003acd210 [crypto returns 5] 7 0 0 0 DL crypto_ 0xfffff80003acd1b0 [crypto returns 4] 6 0 0 0 DL crypto_ 0xfffff80003acd150 [crypto returns 3] 5 0 0 0 DL crypto_ 0xfffff80003acd0f0 [crypto returns 2] 4 0 0 0 DL crypto_ 0xfffff80003acd090 [crypto returns 1] 3 0 0 0 DL crypto_ 0xfffff80003acd030 [crypto returns 0] 2 0 0 0 DL crypto_ 0xffffffff81f7abe8 [crypto] 14 0 0 0 DL seqstat 0xfffff80003b2dc88 [sequencer 00] 13 0 0 0 DL (threaded) [geom] 100062 D - 0xffffffff81fb8ec0 [g_event] 100063 D - 0xffffffff81fb8ed0 [g_up] 100064 D - 0xffffffff81fb8eb8 [g_down] 12 0 0 0 WL (threaded) [intr] 100015 I [swi6: task queue] 100016 I [swi6: Giant taskq] 100018 I [swi5: fast taskq] 100047 I [swi4: clock (0)] 100048 I [swi4: clock (1)] 100049 I [swi4: clock (2)] 100050 I [swi4: clock (3)] 100051 I [swi4: clock (4)] 100052 I [swi4: clock (5)] 100053 I [swi4: clock (6)] 100054 I [swi4: clock (7)] 100055 I [swi4: clock (8)] 100056 I [swi4: clock (9)] 100057 I [swi4: clock (10)] 100058 I [swi4: clock (11)] 100059 I [swi3: vm] 100060 I [swi1: netisr 0] 100094 I [irq56: ahci0] 100095 I [irq19: xhci0] 100101 I [irq18: ehci0 ehci1] 100112 I [irq71: ahci1] 100113 I [swi0: uart uart] 100179 I [irq72: ioat0] 100180 I [irq73: ioat1] 100181 I [irq74: ioat2] 100182 I [irq75: ioat3] 100183 I [irq76: ioat4] 100184 I [irq77: ioat5] 100185 I [irq78: ioat6] 100186 I [irq79: ioat7] 11 0 0 0 RL (threaded) [idle] 100003 CanRun [idle: cpu0] 100004 Run CPU 1 [idle: cpu1] 100005 Run CPU 2 [idle: cpu2] 100006 Run CPU 3 [idle: cpu3] 100007 Run CPU 4 [idle: cpu4] 100008 Run CPU 5 [idle: cpu5] 100009 Run CPU 6 [idle: cpu6] 100010 Run CPU 7 [idle: cpu7] 100011 Run CPU 8 [idle: cpu8] 100012 Run CPU 9 [idle: cpu9] 100013 Run CPU 10 [idle: cpu10] 100014 CanRun [idle: cpu11] 1 0 1 0 SLs wait 0xfffff800037b2530 [init] 10 0 0 0 DL audit_w 0xffffffff81fe9740 [audit] 0 0 0 0 DLs (threaded) [kernel] 100000 D swapin 0xffffffff81fb8f00 [swapper] 100017 D - 0xfffff8000369bc00 [thread taskq] 100019 D - 0xfffff8000369ba00 [config_0] 100020 D - 0xfffff8000369b900 [kqueue_ctx taskq] 100021 D - 0xfffff8000369b800 [aiod_kick taskq] 100022 D - 0xfffff8000369b700 [softirq_0] 100023 D - 0xfffff8000369b600 [softirq_1] 100024 D - 0xfffff8000369b500 [softirq_2] 100025 D - 0xfffff8000369b400 [softirq_3] 100026 D - 0xfffff8000369b300 [softirq_4] 100027 D - 0xfffff8000369b200 [softirq_5] 100028 D - 0xfffff8000369b100 [softirq_6] 100029 D - 0xfffff8000369b000 [softirq_7] 100030 D - 0xfffff8000369ae00 [softirq_8] 100031 D - 0xfffff8000369ad00 [softirq_9] 100032 D - 0xfffff8000369ac00 [softirq_10] 100033 D - 0xfffff8000369ab00 [softirq_11] 100034 D - 0xfffff8000369aa00 [if_io_tqg_0] 100035 D - 0xfffff8000369a900 [if_io_tqg_1] 100036 D - 0xfffff8000369a800 [if_io_tqg_2] 100037 D - 0xfffff8000369a700 [if_io_tqg_3] 100038 D - 0xfffff8000369a600 [if_io_tqg_4] 100039 D - 0xfffff8000369a500 [if_io_tqg_5] 100040 D - 0xfffff8000369a400 [if_io_tqg_6] 100041 D - 0xfffff8000369a300 [if_io_tqg_7] 100042 D - 0xfffff8000369a200 [if_io_tqg_8] 100043 D - 0xfffff8000369a100 [if_io_tqg_9] 100044 D - 0xfffff8000369a000 [if_io_tqg_10] 100045 D - 0xfffff80003699e00 [if_io_tqg_11] 100046 D - 0xfffff80003699d00 [if_config_tqg_0] 100061 D - 0xfffff80003699c00 [firmware taskq] 100066 D - 0xfffff80003699b00 [crypto_0] 100067 D - 0xfffff80003699b00 [crypto_1] 100068 D - 0xfffff80003699b00 [crypto_2] 100069 D - 0xfffff80003699b00 [crypto_3] 100070 D - 0xfffff80003699b00 [crypto_4] 100071 D - 0xfffff80003699b00 [crypto_5] 100072 D - 0xfffff80003699b00 [crypto_6] 100073 D - 0xfffff80003699b00 [crypto_7] 100074 D - 0xfffff80003699b00 [crypto_8] 100075 D - 0xfffff80003699b00 [crypto_9] 100076 D - 0xfffff80003699b00 [crypto_10] 100077 D - 0xfffff80003699b00 [crypto_11] 100114 D - 0xfffff800067ae100 [mca taskq] 100121 D - 0xfffff800078d1200 [acpi_task_0] 100122 D - 0xfffff800078d1200 [acpi_task_1] 100123 D - 0xfffff800078d1200 [acpi_task_2] 100124 D - 0xfffff80003699a00 [CAM taskq] db:0:pho> allt Tracing command pkill pid 38017 tid 101847 td 0xfffff801d7085000 (CPU 0) kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe00aed8d4e0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00aed8d4f0 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe00aed8d520 uart_intr() at uart_intr+0x138/frame 0xfffffe00aed8d560 intr_event_handle() at intr_event_handle+0xb5/frame 0xfffffe00aed8d5b0 intr_execute_handlers() at intr_execute_handlers+0x55/frame 0xfffffe00aed8d5e0 lapic_handle_intr() at lapic_handle_intr+0x5f/frame 0xfffffe00aed8d600 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00aed8d600 --- interrupt, rip = 0xffffffff80bb40a9, rsp = 0xfffffe00aed8d6d0, rbp = 0xfffffe00aed8d710 --- __mtx_lock_flags() at __mtx_lock_flags+0x19/frame 0xfffffe00aed8d710 pmap_enter() at pmap_enter+0x2ba/frame 0xfffffe00aed8d7d0 vm_fault_hold() at vm_fault_hold+0x1799/frame 0xfffffe00aed8d910 vm_fault() at vm_fault+0x60/frame 0xfffffe00aed8d950 trap_pfault() at trap_pfault+0x188/frame 0xfffffe00aed8d9a0 trap() at trap+0x469/frame 0xfffffe00aed8dab0 calltrap() at calltrap+0x8/frame 0xfffffe00aed8dab0 --- trap 0xc, rip = 0x800433855, rsp = 0x7fffffffd978, rbp = 0x7fffffffd9c0 --- Tracing command sleep pid 31580 tid 104510 td 0xfffff801dc421000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe02d3090710 mi_switch() at mi_switch+0x172/frame 0xfffffe02d3090740 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe02d3090780 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe02d30907e0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe02d3090820 _sleep() at _sleep+0x265/frame 0xfffffe02d30908c0 kern_clock_nanosleep() at kern_clock_nanosleep+0x206/frame 0xfffffe02d3090940 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe02d3090980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe02d3090ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02d3090ab0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003a4a0a, rsp = 0x7fffffffe6a8, rbp = 0x7fffffffe6f0 --- Tracing command bash pid 36335 tid 103445 td 0xfffff80683a36000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe02ca60e580 mi_switch() at mi_switch+0x172/frame 0xfffffe02ca60e5b0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe02ca60e5f0 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe02ca60e650 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe02ca60e680 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe02ca60e6e0 seltdwait() at seltdwait+0x8d/frame 0xfffffe02ca60e720 kern_select() at kern_select+0x95a/frame 0xfffffe02ca60e900 sys_pselect() at sys_pselect+0x127/frame 0xfffffe02ca60e980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe02ca60eab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02ca60eab0 --- syscall (522, FreeBSD ELF64, sys_pselect), rip = 0x8005159ca, rsp = 0x7fffffffd618, rbp = 0x7fffffffd6e0 --- Tracing command csh pid 34558 tid 104460 td 0xfffff80006442000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe02c7db7750 mi_switch() at mi_switch+0x172/frame 0xfffffe02c7db7780 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe02c7db77c0 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe02c7db7820 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe02c7db7850 _sleep() at _sleep+0x2d0/frame 0xfffffe02c7db78f0 kern_sigsuspend() at kern_sigsuspend+0xb4/frame 0xfffffe02c7db7950 sys_sigsuspend() at sys_sigsuspend+0x31/frame 0xfffffe02c7db7980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe02c7db7ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02c7db7ab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x80048992a, rsp = 0x7fffffffdfd8, rbp = 0x7fffffffe060 --- Tracing command su pid 34554 tid 101336 td 0xfffff80195d3c5a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00adf2b560 mi_switch() at mi_switch+0x172/frame 0xfffffe00adf2b590 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00adf2b5d0 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00adf2b630 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00adf2b660 _sleep() at _sleep+0x2d0/frame 0xfffffe00adf2b700 kern_wait6() at kern_wait6+0x48f/frame 0xfffffe00adf2b790 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00adf2b980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00adf2bab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00adf2bab0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8003ea8aa, rsp = 0x7fffffffe5d8, rbp = 0x7fffffffeb20 --- Tracing command bash pid 87305 tid 100849 td 0xfffff801d7e82000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe02c8040560 mi_switch() at mi_switch+0x172/frame 0xfffffe02c8040590 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe02c80405d0 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe02c8040630 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe02c8040660 _sleep() at _sleep+0x2d0/frame 0xfffffe02c8040700 kern_wait6() at kern_wait6+0x48f/frame 0xfffffe02c8040790 sys_wait4() at sys_wait4+0x78/frame 0xfffffe02c8040980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe02c8040ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02c8040ab0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8005158aa, rsp = 0x7fffffffe7c8, rbp = 0x7fffffffe810 --- Tracing command sshd pid 87141 tid 103717 td 0xfffff803b41745a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe02cff535c0 mi_switch() at mi_switch+0x172/frame 0xfffffe02cff535f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe02cff53630 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe02cff53690 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe02cff536c0 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe02cff53720 seltdwait() at seltdwait+0x8d/frame 0xfffffe02cff53760 kern_select() at kern_select+0x95a/frame 0xfffffe02cff53940 sys_select() at sys_select+0x56/frame 0xfffffe02cff53980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe02cff53ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02cff53ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8009021da, rsp = 0x7fffffffa578, rbp = 0x7fffffffa5b0 --- Tracing command sshd pid 85132 tid 104657 td 0xfffff8019600a5a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe02c7f4e640 mi_switch() at mi_switch+0x172/frame 0xfffffe02c7f4e670 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe02c7f4e6b0 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe02c7f4e710 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe02c7f4e740 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe02c7f4e7a0 seltdwait() at seltdwait+0x8d/frame 0xfffffe02c7f4e7e0 kern_poll() at kern_poll+0x448/frame 0xfffffe02c7f4e960 sys_poll() at sys_poll+0x50/frame 0xfffffe02c7f4e980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe02c7f4eab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02c7f4eab0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8008a41fa, rsp = 0x7fffffffe5d8, rbp = 0x7fffffffe610 --- Tracing command syscall4 pid 97867 tid 103566 td 0xfffff80197e0e000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe02c23d8a00 mi_switch() at mi_switch+0x172/frame 0xfffffe02c23d8a30 thread_suspend_check() at thread_suspend_check+0x93/frame 0xfffffe02c23d8a70 ast() at ast+0x534/frame 0xfffffe02c23d8ab0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffe540 Tracing command syscall4 pid 97867 tid 100382 td 0xfffff8011c5a8000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00aec2fa00 mi_switch() at mi_switch+0x172/frame 0xfffffe00aec2fa30 thread_suspend_check() at thread_suspend_check+0x93/frame 0xfffffe00aec2fa70 ast() at ast+0x534/frame 0xfffffe00aec2fab0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffdfffdef0 Tracing command syscall4 pid 97867 tid 100701 td 0xfffff8011c5b55a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe02cff26a00 mi_switch() at mi_switch+0x172/frame 0xfffffe02cff26a30 thread_suspend_check() at thread_suspend_check+0x93/frame 0xfffffe02cff26a70 ast() at ast+0x534/frame 0xfffffe02cff26ab0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffdfdfc410 Tracing command syscall4 pid 97867 tid 104606 td 0xfffff8017ae7b000 (CPU 11) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bebdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bebe10 trap() at trap+0x46/frame 0xfffffe0006bebf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bebf20 --- trap 0x13, rip = 0xffffffff80c30037, rsp = 0xfffffe00ade276f0, rbp = 0xfffffe00ade27740 --- sleepq_catch_signals() at sleepq_catch_signals+0x37/frame 0xfffffe00ade27740 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ade27770 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ade277d0 mseq_read() at mseq_read+0xff/frame 0xfffffe00ade27840 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00ade278a0 dofileread() at dofileread+0x81/frame 0xfffffe00ade278f0 kern_preadv() at kern_preadv+0x5f/frame 0xfffffe00ade27930 freebsd6_pread() at freebsd6_pread+0x8b/frame 0xfffffe00ade27980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00ade27ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ade27ab0 --- syscall (0, FreeBSD ELF64, nosys), rip = 0x8004442ba, rsp = 0x7fffdfbfbf48, rbp = 0x7fffdfbfbfb0 --- Tracing command syscall4 pid 97867 tid 104634 td 0xfffff8047e7b45a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe02d3022a00 mi_switch() at mi_switch+0x172/frame 0xfffffe02d3022a30 thread_suspend_check() at thread_suspend_check+0x93/frame 0xfffffe02d3022a70 ast() at ast+0x534/frame 0xfffffe02d3022ab0 fast_syscall_common() at fast_syscall_common+0x198/frame 0xfffffe02d3022ab0 --- syscall (939510), rip = 0x8004442ba, rsp = 0x7fffdf9faf48, rbp = 0x7fffdf9fafb0 --- Tracing command syscall4 pid 97867 tid 104685 td 0xfffff806853a85a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00b7d0da00 mi_switch() at mi_switch+0x172/frame 0xfffffe00b7d0da30 thread_suspend_check() at thread_suspend_check+0x93/frame 0xfffffe00b7d0da70 ast() at ast+0x534/frame 0xfffffe00b7d0dab0 fast_syscall_common() at fast_syscall_common+0x198/frame 0xfffffe00b7d0dab0 --- syscall (0, FreeBSD ELF64, nosys), rip = 0x8003bd8aa, rsp = 0x7fffdf7f9ef8, rbp = 0x7fffdf7f9f50 --- Tracing command syscall4 pid 97867 tid 104702 td 0xfffff8052d29f000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00b7dbca00 mi_switch() at mi_switch+0x172/frame 0xfffffe00b7dbca30 thread_suspend_check() at thread_suspend_check+0x93/frame 0xfffffe00b7dbca70 ast() at ast+0x534/frame 0xfffffe00b7dbcab0 fast_syscall_common() at fast_syscall_common+0x198/frame 0xfffffe00b7dbcab0 --- syscall (0, FreeBSD ELF64, nosys), rip = 0x8003bd8aa, rsp = 0x7fffdf5f8ef8, rbp = 0x7fffdf5f8f50 --- Tracing command syscall4 pid 97867 tid 104707 td 0xfffff80683106000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00aed51a00 mi_switch() at mi_switch+0x172/frame 0xfffffe00aed51a30 thread_suspend_check() at thread_suspend_check+0x93/frame 0xfffffe00aed51a70 ast() at ast+0x534/frame 0xfffffe00aed51ab0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffdf3f79a0 Tracing command syscall4 pid 97867 tid 104721 td 0xfffff803b48725a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe02c90c2a00 mi_switch() at mi_switch+0x172/frame 0xfffffe02c90c2a30 thread_suspend_check() at thread_suspend_check+0x93/frame 0xfffffe02c90c2a70 ast() at ast+0x534/frame 0xfffffe02c90c2ab0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffdf1f6060 Tracing command syscall4 pid 97867 tid 104723 td 0xfffff8047e6af000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe02cff08a00 mi_switch() at mi_switch+0x172/frame 0xfffffe02cff08a30 thread_suspend_check() at thread_suspend_check+0x93/frame 0xfffffe02cff08a70 ast() at ast+0x534/frame 0xfffffe02cff08ab0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffdeff5bf0 Tracing command syscall4 pid 97867 tid 104816 td 0xfffff80117f58000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe02c8521a00 mi_switch() at mi_switch+0x172/frame 0xfffffe02c8521a30 thread_suspend_check() at thread_suspend_check+0x93/frame 0xfffffe02c8521a70 ast() at ast+0x534/frame 0xfffffe02c8521ab0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffdedf4bf0 Tracing command syscall4 pid 97867 tid 104829 td 0xfffff8058f186000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe02c22d9a00 mi_switch() at mi_switch+0x172/frame 0xfffffe02c22d9a30 thread_suspend_check() at thread_suspend_check+0x93/frame 0xfffffe02c22d9a70 ast() at ast+0x534/frame 0xfffffe02c22d9ab0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffdebf3bf0 Tracing command syscall4 pid 97867 tid 105211 td 0xfffff801d7b105a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe02c8acca00 mi_switch() at mi_switch+0x172/frame 0xfffffe02c8acca30 thread_suspend_check() at thread_suspend_check+0x93/frame 0xfffffe02c8acca70 ast() at ast+0x534/frame 0xfffffe02c8accab0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffde9f2bf0 Tracing command syscall4 pid 97867 tid 105252 td 0xfffff8065d00c000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00aa7d6a00 mi_switch() at mi_switch+0x172/frame 0xfffffe00aa7d6a30 thread_suspend_check() at thread_suspend_check+0x93/frame 0xfffffe00aa7d6a70 ast() at ast+0x534/frame 0xfffffe00aa7d6ab0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffde7f1bf0 Tracing command syscall4 pid 97867 tid 105391 td 0xfffff801dc1b75a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe02cff037e0 mi_switch() at mi_switch+0x172/frame 0xfffffe02cff03810 thread_suspend_switch() at thread_suspend_switch+0x10e/frame 0xfffffe02cff03860 thread_single() at thread_single+0x4bb/frame 0xfffffe02cff038c0 pre_execve() at pre_execve+0x77/frame 0xfffffe02cff03900 sys_fexecve() at sys_fexecve+0x1c/frame 0xfffffe02cff03980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe02cff03ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02cff03ab0 --- syscall (0, FreeBSD ELF64, nosys), rip = 0x8004442ba, rsp = 0x7fffde5f0f48, rbp = 0x7fffde5f0fb0 --- Tracing command syscall4 pid 97867 tid 105392 td 0xfffff803b4b7e000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00ac696a00 mi_switch() at mi_switch+0x172/frame 0xfffffe00ac696a30 thread_suspend_check() at thread_suspend_check+0x93/frame 0xfffffe00ac696a70 ast() at ast+0x534/frame 0xfffffe00ac696ab0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffde3efbf0 Tracing command syscall4 pid 97867 tid 105393 td 0xfffff805fa6035a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe02c64a4a00 mi_switch() at mi_switch+0x172/frame 0xfffffe02c64a4a30 thread_suspend_check() at thread_suspend_check+0x93/frame 0xfffffe02c64a4a70 ast() at ast+0x534/frame 0xfffffe02c64a4ab0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffde1eebf0 Tracing command syscall4 pid 97867 tid 105394 td 0xfffff8011c043000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00b2dbca00 mi_switch() at mi_switch+0x172/frame 0xfffffe00b2dbca30 thread_suspend_check() at thread_suspend_check+0x93/frame 0xfffffe00b2dbca70 ast() at ast+0x534/frame 0xfffffe00b2dbcab0 doreti_ast() at doreti_ast+0x1f Tracing command aiod4 pid 83109 tid 102877 td 0xfffff80801d35000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe02c93c68d0 mi_switch() at mi_switch+0x172/frame 0xfffffe02c93c6900 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe02c93c6940 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe02c93c6980 _sleep() at _sleep+0x279/frame 0xfffffe02c93c6a20 aio_daemon() at aio_daemon+0x2ee/frame 0xfffffe02c93c6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe02c93c6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe02c93c6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command aiod3 pid 83108 tid 101419 td 0xfffff808277455a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00b7c368d0 mi_switch() at mi_switch+0x172/frame 0xfffffe00b7c36900 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00b7c36940 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00b7c36980 _sleep() at _sleep+0x279/frame 0xfffffe00b7c36a20 aio_daemon() at aio_daemon+0x2ee/frame 0xfffffe00b7c36a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b7c36ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b7c36ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command aiod2 pid 83107 tid 101418 td 0xfffff8017a88e000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00b7d308d0 mi_switch() at mi_switch+0x172/frame 0xfffffe00b7d30900 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00b7d30940 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00b7d30980 _sleep() at _sleep+0x279/frame 0xfffffe00b7d30a20 aio_daemon() at aio_daemon+0x2ee/frame 0xfffffe00b7d30a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b7d30ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b7d30ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command aiod1 pid 83106 tid 103699 td 0xfffff8011c450000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00ad9a68d0 mi_switch() at mi_switch+0x172/frame 0xfffffe00ad9a6900 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ad9a6940 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00ad9a6980 _sleep() at _sleep+0x279/frame 0xfffffe00ad9a6a20 aio_daemon() at aio_daemon+0x2ee/frame 0xfffffe00ad9a6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00ad9a6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ad9a6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command md10 pid 83084 tid 104862 td 0xfffff801d75dd5a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe02cb60e8d0 mi_switch() at mi_switch+0x172/frame 0xfffffe02cb60e900 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe02cb60e940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe02cb60e970 _sleep() at _sleep+0x2da/frame 0xfffffe02cb60ea10 md_kthread() at md_kthread+0x1f5/frame 0xfffffe02cb60ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe02cb60eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe02cb60eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sh pid 83053 tid 100977 td 0xfffff8011bb2f5a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00ac792560 mi_switch() at mi_switch+0x172/frame 0xfffffe00ac792590 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ac7925d0 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ac792630 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ac792660 _sleep() at _sleep+0x2d0/frame 0xfffffe00ac792700 kern_wait6() at kern_wait6+0x48f/frame 0xfffffe00ac792790 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00ac792980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00ac792ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ac792ab0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8004058aa, rsp = 0x7fffffffe1b8, rbp = 0x7fffffffe270 --- Tracing command tee pid 83052 tid 100716 td 0xfffff8052dc80000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00aec2a6a0 mi_switch() at mi_switch+0x172/frame 0xfffffe00aec2a6d0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00aec2a710 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00aec2a770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00aec2a7a0 _sleep() at _sleep+0x2d0/frame 0xfffffe00aec2a840 pipe_read() at pipe_read+0x348/frame 0xfffffe00aec2a8a0 dofileread() at dofileread+0x81/frame 0xfffffe00aec2a8f0 kern_readv() at kern_readv+0x49/frame 0xfffffe00aec2a930 sys_read() at sys_read+0x84/frame 0xfffffe00aec2a980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00aec2aab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00aec2aab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004151fa, rsp = 0x7fffffffe6b8, rbp = 0x7fffffffe700 --- Tracing command sh pid 83051 tid 101738 td 0xfffff80197130000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe02c2301560 mi_switch() at mi_switch+0x172/frame 0xfffffe02c2301590 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe02c23015d0 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe02c2301630 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe02c2301660 _sleep() at _sleep+0x2d0/frame 0xfffffe02c2301700 kern_wait6() at kern_wait6+0x48f/frame 0xfffffe02c2301790 sys_wait4() at sys_wait4+0x78/frame 0xfffffe02c2301980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe02c2301ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02c2301ab0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8004058aa, rsp = 0x7fffffffab38, rbp = 0x7fffffffabf0 --- Tracing command sh pid 9975 tid 100268 td 0xfffff80006450000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00ad6a46a0 mi_switch() at mi_switch+0x172/frame 0xfffffe00ad6a46d0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ad6a4710 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ad6a4770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ad6a47a0 _sleep() at _sleep+0x2d0/frame 0xfffffe00ad6a4840 pipe_read() at pipe_read+0x348/frame 0xfffffe00ad6a48a0 dofileread() at dofileread+0x81/frame 0xfffffe00ad6a48f0 kern_readv() at kern_readv+0x49/frame 0xfffffe00ad6a4930 sys_read() at sys_read+0x84/frame 0xfffffe00ad6a4980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00ad6a4ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ad6a4ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004761fa, rsp = 0x7fffffffe228, rbp = 0x7fffffffe330 --- Tracing command awk pid 9973 tid 100145 td 0xfffff80004056000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00aa7d16a0 mi_switch() at mi_switch+0x172/frame 0xfffffe00aa7d16d0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00aa7d1710 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00aa7d1770 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00aa7d17a0 _sleep() at _sleep+0x2d0/frame 0xfffffe00aa7d1840 pipe_read() at pipe_read+0x348/frame 0xfffffe00aa7d18a0 dofileread() at dofileread+0x81/frame 0xfffffe00aa7d18f0 kern_readv() at kern_readv+0x49/frame 0xfffffe00aa7d1930 sys_read() at sys_read+0x84/frame 0xfffffe00aa7d1980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00aa7d1ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00aa7d1ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004791fa, rsp = 0x7fffffffe328, rbp = 0x7fffffffe340 --- Tracing command sh pid 9972 tid 100253 td 0xfffff80159960000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00ad659560 mi_switch() at mi_switch+0x172/frame 0xfffffe00ad659590 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ad6595d0 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ad659630 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ad659660 _sleep() at _sleep+0x2d0/frame 0xfffffe00ad659700 kern_wait6() at kern_wait6+0x48f/frame 0xfffffe00ad659790 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00ad659980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00ad659ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ad659ab0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8004058aa, rsp = 0x7fffffffe238, rbp = 0x7fffffffe2f0 --- Tracing command sh pid 9933 tid 100240 td 0xfffff800065f1000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00aed8cce0 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe00aed8ccf0 trap() at trap+0x59/frame 0xfffffe00aed8ce00 calltrap() at calltrap+0x8/frame 0xfffffe00aed8ce00 --- trap 0xc, rip = 0xffffffff8116e986, rsp = 0xfffffe00aed8ced0, rbp = 0xfffffe00aed8cee0 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe00aed8cee0 _db_stack_trace_all() at _db_stack_trace_all+0xc4/frame 0xfffffe00aed8cf80 db_command() at db_command+0x279/frame 0xfffffe00aed8d050 db_script_exec() at db_script_exec+0x188/frame 0xfffffe00aed8d0f0 db_command() at db_command+0x279/frame 0xfffffe00aed8d1c0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe00aed8d1d0 db_trap() at db_trap+0xef/frame 0xfffffe00aed8d260 kdb_trap() at kdb_trap+0x16c/frame 0xfffffe00aed8d2e0 trap() at trap+0x80f/frame 0xfffffe00aed8d3f0 calltrap() at calltrap+0x8/frame 0xfffffe00aed8d3f0 --- trap 0x3, rip = 0xffffffff80c205b6, rsp = 0xfffffe00aed8d4c0, rbp = 0xfffffe00aed8d4e0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe00aed8d4e0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00aed8d4f0 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe00aed8d520 uart_intr() at uart_intr+0x138/frame 0xfffffe00aed8d560 intr_event_handle() at intr_event_handle+0xb5/frame 0xfffffe00aed8d5b0 intr_execute_handlers() at intr_execute_handlers+0x55/frame 0xfffffe00aed8d5e0 lapic_handle_intr() at lapic_handle_intr+0x5f/frame 0xfffffe00aed8d600 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00aed8d600 --- interrupt, rip = 0xffffffff80bb40a9, rsp = 0xfffffe00aed8d6d0, rbp = 0xfffffe00aed8d710 --- __mtx_lock_flags() at __mtx_lock_flags+0x19/frame 0xfffffe00aed8d710 pmap_enter() at pmap_enter+0x2ba/frame 0xfffffe00aed8d7d0 vm_fault_hold() at vm_fault_hold+0x1799/frame 0xfffffe00aed8d910 vm_fault() at vm_fault+0x60/frame 0xfffffe00aed8d950 trap_pfault() at trap_pfault+0x188/frame 0xfffffe00aed8d9a0 trap() at trap+0x469/frame 0xfffffe00aed8dab0 calltrap() at calltrap+0x8/frame 0xfffffe00aed8dab0 --- trap 0xc, rip = 0x800433855, rsp = 0x7fffffffd978, rbp = 0x7fffffffd9c0 --- Tracing command sh pid 7612 tid 100204 td 0xfffff80007e83000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00ac760560 mi_switch() at mi_switch+0x172/frame 0xfffffe00ac760590 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ac7605d0 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ac760630 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ac760660 _sleep() at _sleep+0x2d0/frame 0xfffffe00ac760700 kern_wait6() at kern_wait6+0x48f/frame 0xfffffe00ac760790 sys_wait4() at sys_wait4+0x78/frame 0xfffffe00ac760980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00ac760ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ac760ab0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8004058aa, rsp = 0x7fffffffae58, rbp = 0x7fffffffaf10 --- Tracing command bash pid 2351 tid 100188 td 0xfffff80006699000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00aed8cce0 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe00aed8ccf0 trap() at trap+0x59/frame 0xfffffe00aed8ce00 calltrap() at calltrap+0x8/frame 0xfffffe00aed8ce00 --- trap 0xc, rip = 0xffffffff8116e986, rsp = 0xfffffe00aed8ced0, rbp = 0xfffffe00aed8cee0 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe00aed8cee0 _db_stack_trace_all() at _db_stack_trace_all+0xc4/frame 0xfffffe00aed8cf80 db_command() at db_command+0x279/frame 0xfffffe00aed8d050 db_script_exec() at db_script_exec+0x188/frame 0xfffffe00aed8d0f0 db_command() at db_command+0x279/frame 0xfffffe00aed8d1c0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe00aed8d1d0 db_trap() at db_trap+0xef/frame 0xfffffe00aed8d260 kdb_trap() at kdb_trap+0x16c/frame 0xfffffe00aed8d2e0 trap() at trap+0x80f/frame 0xfffffe00aed8d3f0 calltrap() at calltrap+0x8/frame 0xfffffe00aed8d3f0 --- trap 0x3, rip = 0xffffffff80c205b6, rsp = 0xfffffe00aed8d4c0, rbp = 0xfffffe00aed8d4e0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe00aed8d4e0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00aed8d4f0 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe00aed8d520 uart_intr() at uart_intr+0x138/frame 0xfffffe00aed8d560 intr_event_handle() at intr_event_handle+0xb5/frame 0xfffffe00aed8d5b0 intr_execute_handlers() at intr_execute_handlers+0x55/frame 0xfffffe00aed8d5e0 lapic_handle_intr() at lapic_handle_intr+0x5f/frame 0xfffffe00aed8d600 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00aed8d600 --- interrupt, rip = 0xffffffff80bb40a9, rsp = 0xfffffe00aed8d6d0, rbp = 0xfffffe00aed8d710 --- __mtx_lock_flags() at __mtx_lock_flags+0x19/frame 0xfffffe00aed8d710 pmap_enter() at pmap_enter+0x2ba/frame 0xfffffe00aed8d7d0 vm_fault_hold() at vm_fault_hold+0x1799/frame 0xfffffe00aed8d910 vm_fault() at vm_fault+0x60/frame 0xfffffe00aed8d950 trap_pfault() at trap_pfault+0x188/frame 0xfffffe00aed8d9a0 trap() at trap+0x469/frame 0xfffffe00aed8dab0 calltrap() at calltrap+0x8/frame 0xfffffe00aed8dab0 --- trap 0xc, rip = 0x800433855, rsp = 0x7fffffffd978, rbp = 0x7fffffffd9c0 --- Tracing command csh pid 2349 tid 100201 td 0xfffff800060f55a0 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00aed8cce0 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe00aed8ccf0 trap() at trap+0x59/frame 0xfffffe00aed8ce00 calltrap() at calltrap+0x8/frame 0xfffffe00aed8ce00 --- trap 0xc, rip = 0xffffffff8116e986, rsp = 0xfffffe00aed8ced0, rbp = 0xfffffe00aed8cee0 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe00aed8cee0 _db_stack_trace_all() at _db_stack_trace_all+0xc4/frame 0xfffffe00aed8cf80 db_command() at db_command+0x279/frame 0xfffffe00aed8d050 db_script_exec() at db_script_exec+0x188/frame 0xfffffe00aed8d0f0 db_command() at db_command+0x279/frame 0xfffffe00aed8d1c0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe00aed8d1d0 db_trap() at db_trap+0xef/frame 0xfffffe00aed8d260 kdb_trap() at kdb_trap+0x16c/frame 0xfffffe00aed8d2e0 trap() at trap+0x80f/frame 0xfffffe00aed8d3f0 calltrap() at calltrap+0x8/frame 0xfffffe00aed8d3f0 --- trap 0x3, rip = 0xffffffff80c205b6, rsp = 0xfffffe00aed8d4c0, rbp = 0xfffffe00aed8d4e0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe00aed8d4e0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00aed8d4f0 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe00aed8d520 uart_intr() at uart_intr+0x138/frame 0xfffffe00aed8d560 intr_event_handle() at intr_event_handle+0xb5/frame 0xfffffe00aed8d5b0 intr_execute_handlers() at intr_execute_handlers+0x55/frame 0xfffffe00aed8d5e0 lapic_handle_intr() at lapic_handle_intr+0x5f/frame 0xfffffe00aed8d600 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00aed8d600 --- interrupt, rip = 0xffffffff80bb40a9, rsp = 0xfffffe00aed8d6d0, rbp = 0xfffffe00aed8d710 --- __mtx_lock_flags() at __mtx_lock_flags+0x19/frame 0xfffffe00aed8d710 pmap_enter() at pmap_enter+0x2ba/frame 0xfffffe00aed8d7d0 vm_fault_hold() at vm_fault_hold+0x1799/frame 0xfffffe00aed8d910 vm_fault() at vm_fault+0x60/frame 0xfffffe00aed8d950 trap_pfault() at trap_pfault+0x188/frame 0xfffffe00aed8d9a0 trap() at trap+0x469/frame 0xfffffe00aed8dab0 calltrap() at calltrap+0x8/frame 0xfffffe00aed8dab0 --- trap 0xc, rip = 0x800433855, rsp = 0x7fffffffd978, rbp = 0x7fffffffd9c0 --- Tracing command su pid 2348 tid 100170 td 0xfffff8001267d5a0 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00aed8cce0 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe00aed8ccf0 trap() at trap+0x59/frame 0xfffffe00aed8ce00 calltrap() at calltrap+0x8/frame 0xfffffe00aed8ce00 --- trap 0xc, rip = 0xffffffff8116e986, rsp = 0xfffffe00aed8ced0, rbp = 0xfffffe00aed8cee0 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe00aed8cee0 _db_stack_trace_all() at _db_stack_trace_all+0xc4/frame 0xfffffe00aed8cf80 db_command() at db_command+0x279/frame 0xfffffe00aed8d050 db_script_exec() at db_script_exec+0x188/frame 0xfffffe00aed8d0f0 db_command() at db_command+0x279/frame 0xfffffe00aed8d1c0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe00aed8d1d0 db_trap() at db_trap+0xef/frame 0xfffffe00aed8d260 kdb_trap() at kdb_trap+0x16c/frame 0xfffffe00aed8d2e0 trap() at trap+0x80f/frame 0xfffffe00aed8d3f0 calltrap() at calltrap+0x8/frame 0xfffffe00aed8d3f0 --- trap 0x3, rip = 0xffffffff80c205b6, rsp = 0xfffffe00aed8d4c0, rbp = 0xfffffe00aed8d4e0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe00aed8d4e0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00aed8d4f0 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe00aed8d520 uart_intr() at uart_intr+0x138/frame 0xfffffe00aed8d560 intr_event_handle() at intr_event_handle+0xb5/frame 0xfffffe00aed8d5b0 intr_execute_handlers() at intr_execute_handlers+0x55/frame 0xfffffe00aed8d5e0 lapic_handle_intr() at lapic_handle_intr+0x5f/frame 0xfffffe00aed8d600 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00aed8d600 --- interrupt, rip = 0xffffffff80bb40a9, rsp = 0xfffffe00aed8d6d0, rbp = 0xfffffe00aed8d710 --- __mtx_lock_flags() at __mtx_lock_flags+0x19/frame 0xfffffe00aed8d710 pmap_enter() at pmap_enter+0x2ba/frame 0xfffffe00aed8d7d0 vm_fault_hold() at vm_fault_hold+0x1799/frame 0xfffffe00aed8d910 vm_fault() at vm_fault+0x60/frame 0xfffffe00aed8d950 trap_pfault() at trap_pfault+0x188/frame 0xfffffe00aed8d9a0 trap() at trap+0x469/frame 0xfffffe00aed8dab0 calltrap() at calltrap+0x8/frame 0xfffffe00aed8dab0 --- trap 0xc, rip = 0x800433855, rsp = 0x7fffffffd978, rbp = 0x7fffffffd9c0 --- Tracing command bash pid 2344 tid 100213 td 0xfffff8002de5c000 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00aed8cce0 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe00aed8ccf0 trap() at trap+0x59/frame 0xfffffe00aed8ce00 calltrap() at calltrap+0x8/frame 0xfffffe00aed8ce00 --- trap 0xc, rip = 0xffffffff8116e986, rsp = 0xfffffe00aed8ced0, rbp = 0xfffffe00aed8cee0 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe00aed8cee0 _db_stack_trace_all() at _db_stack_trace_all+0xc4/frame 0xfffffe00aed8cf80 db_command() at db_command+0x279/frame 0xfffffe00aed8d050 db_script_exec() at db_script_exec+0x188/frame 0xfffffe00aed8d0f0 db_command() at db_command+0x279/frame 0xfffffe00aed8d1c0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe00aed8d1d0 db_trap() at db_trap+0xef/frame 0xfffffe00aed8d260 kdb_trap() at kdb_trap+0x16c/frame 0xfffffe00aed8d2e0 trap() at trap+0x80f/frame 0xfffffe00aed8d3f0 calltrap() at calltrap+0x8/frame 0xfffffe00aed8d3f0 --- trap 0x3, rip = 0xffffffff80c205b6, rsp = 0xfffffe00aed8d4c0, rbp = 0xfffffe00aed8d4e0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe00aed8d4e0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00aed8d4f0 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe00aed8d520 uart_intr() at uart_intr+0x138/frame 0xfffffe00aed8d560 intr_event_handle() at intr_event_handle+0xb5/frame 0xfffffe00aed8d5b0 intr_execute_handlers() at intr_execute_handlers+0x55/frame 0xfffffe00aed8d5e0 lapic_handle_intr() at lapic_handle_intr+0x5f/frame 0xfffffe00aed8d600 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00aed8d600 --- interrupt, rip = 0xffffffff80bb40a9, rsp = 0xfffffe00aed8d6d0, rbp = 0xfffffe00aed8d710 --- __mtx_lock_flags() at __mtx_lock_flags+0x19/frame 0xfffffe00aed8d710 pmap_enter() at pmap_enter+0x2ba/frame 0xfffffe00aed8d7d0 vm_fault_hold() at vm_fault_hold+0x1799/frame 0xfffffe00aed8d910 vm_fault() at vm_fault+0x60/frame 0xfffffe00aed8d950 trap_pfault() at trap_pfault+0x188/frame 0xfffffe00aed8d9a0 trap() at trap+0x469/frame 0xfffffe00aed8dab0 calltrap() at calltrap+0x8/frame 0xfffffe00aed8dab0 --- trap 0xc, rip = 0x800433855, rsp = 0x7fffffffd978, rbp = 0x7fffffffd9c0 --- Tracing command sshd pid 2343 tid 100154 td 0xfffff800122a65a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00aa7fe5c0 mi_switch() at mi_switch+0x172/frame 0xfffffe00aa7fe5f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00aa7fe630 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00aa7fe690 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00aa7fe6c0 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00aa7fe720 seltdwait() at seltdwait+0x8d/frame 0xfffffe00aa7fe760 kern_select() at kern_select+0x95a/frame 0xfffffe00aa7fe940 sys_select() at sys_select+0x56/frame 0xfffffe00aa7fe980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00aa7feab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00aa7feab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8009021da, rsp = 0x7fffffffa578, rbp = 0x7fffffffa5b0 --- Tracing command sshd pid 2340 tid 100171 td 0xfffff8001267d000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00ac678640 mi_switch() at mi_switch+0x172/frame 0xfffffe00ac678670 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ac6786b0 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ac678710 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ac678740 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ac6787a0 seltdwait() at seltdwait+0x8d/frame 0xfffffe00ac6787e0 kern_poll() at kern_poll+0x448/frame 0xfffffe00ac678960 sys_poll() at sys_poll+0x50/frame 0xfffffe00ac678980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00ac678ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ac678ab0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8008a41fa, rsp = 0x7fffffffe5d8, rbp = 0x7fffffffe610 --- Tracing command csh pid 2336 tid 100192 td 0xfffff8000669a5a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00ac729620 mi_switch() at mi_switch+0x172/frame 0xfffffe00ac729650 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ac729690 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ac7296f0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ac729720 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ac729780 tty_wait() at tty_wait+0x42/frame 0xfffffe00ac7297b0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00ac729810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00ac729840 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00ac7298a0 dofileread() at dofileread+0x81/frame 0xfffffe00ac7298f0 kern_readv() at kern_readv+0x49/frame 0xfffffe00ac729930 sys_read() at sys_read+0x84/frame 0xfffffe00ac729980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00ac729ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ac729ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004fa1fa, rsp = 0x7fffffffe1f8, rbp = 0x7fffffffe220 --- Tracing command login pid 2335 tid 100211 td 0xfffff8002de5c5a0 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00aed8cce0 kdb_reenter() at kdb_reenter+0x2f/frame 0xfffffe00aed8ccf0 trap() at trap+0x59/frame 0xfffffe00aed8ce00 calltrap() at calltrap+0x8/frame 0xfffffe00aed8ce00 --- trap 0xc, rip = 0xffffffff8116e986, rsp = 0xfffffe00aed8ced0, rbp = 0xfffffe00aed8cee0 --- db_trace_thread() at db_trace_thread+0x26/frame 0xfffffe00aed8cee0 _db_stack_trace_all() at _db_stack_trace_all+0xc4/frame 0xfffffe00aed8cf80 db_command() at db_command+0x279/frame 0xfffffe00aed8d050 db_script_exec() at db_script_exec+0x188/frame 0xfffffe00aed8d0f0 db_command() at db_command+0x279/frame 0xfffffe00aed8d1c0 db_command_loop() at db_command_loop+0x64/frame 0xfffffe00aed8d1d0 db_trap() at db_trap+0xef/frame 0xfffffe00aed8d260 kdb_trap() at kdb_trap+0x16c/frame 0xfffffe00aed8d2e0 trap() at trap+0x80f/frame 0xfffffe00aed8d3f0 calltrap() at calltrap+0x8/frame 0xfffffe00aed8d3f0 --- trap 0x3, rip = 0xffffffff80c205b6, rsp = 0xfffffe00aed8d4c0, rbp = 0xfffffe00aed8d4e0 --- kdb_alt_break_internal() at kdb_alt_break_internal+0x106/frame 0xfffffe00aed8d4e0 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00aed8d4f0 uart_intr_rxready() at uart_intr_rxready+0x9a/frame 0xfffffe00aed8d520 uart_intr() at uart_intr+0x138/frame 0xfffffe00aed8d560 intr_event_handle() at intr_event_handle+0xb5/frame 0xfffffe00aed8d5b0 intr_execute_handlers() at intr_execute_handlers+0x55/frame 0xfffffe00aed8d5e0 lapic_handle_intr() at lapic_handle_intr+0x5f/frame 0xfffffe00aed8d600 Xapic_isr1() at Xapic_isr1+0xdc/frame 0xfffffe00aed8d600 --- interrupt, rip = 0xffffffff80bb40a9, rsp = 0xfffffe00aed8d6d0, rbp = 0xfffffe00aed8d710 --- __mtx_lock_flags() at __mtx_lock_flags+0x19/frame 0xfffffe00aed8d710 pmap_enter() at pmap_enter+0x2ba/frame 0xfffffe00aed8d7d0 vm_fault_hold() at vm_fault_hold+0x1799/frame 0xfffffe00aed8d910 vm_fault() at vm_fault+0x60/frame 0xfffffe00aed8d950 trap_pfault() at trap_pfault+0x188/frame 0xfffffe00aed8d9a0 trap() at trap+0x469/frame 0xfffffe00aed8dab0 calltrap() at calltrap+0x8/frame 0xfffffe00aed8dab0 --- trap 0xc, rip = 0x800433855, rsp = 0x7fffffffd978, rbp = 0x7fffffffd9c0 --- Tracing command getty pid 2334 tid 100210 td 0xfffff8002de5b000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00ac77e620 mi_switch() at mi_switch+0x172/frame 0xfffffe00ac77e650 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ac77e690 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ac77e6f0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ac77e720 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ac77e780 tty_wait() at tty_wait+0x42/frame 0xfffffe00ac77e7b0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00ac77e810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00ac77e840 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00ac77e8a0 dofileread() at dofileread+0x81/frame 0xfffffe00ac77e8f0 kern_readv() at kern_readv+0x49/frame 0xfffffe00ac77e930 sys_read() at sys_read+0x84/frame 0xfffffe00ac77e980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00ac77eab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ac77eab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004311fa, rsp = 0x7fffffffebc8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2333 tid 100209 td 0xfffff8002de5b5a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00ac779620 mi_switch() at mi_switch+0x172/frame 0xfffffe00ac779650 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ac779690 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ac7796f0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ac779720 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ac779780 tty_wait() at tty_wait+0x42/frame 0xfffffe00ac7797b0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00ac779810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00ac779840 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00ac7798a0 dofileread() at dofileread+0x81/frame 0xfffffe00ac7798f0 kern_readv() at kern_readv+0x49/frame 0xfffffe00ac779930 sys_read() at sys_read+0x84/frame 0xfffffe00ac779980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00ac779ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ac779ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004311fa, rsp = 0x7fffffffebc8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2332 tid 100208 td 0xfffff8002de5a000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00ac774620 mi_switch() at mi_switch+0x172/frame 0xfffffe00ac774650 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ac774690 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ac7746f0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ac774720 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ac774780 tty_wait() at tty_wait+0x42/frame 0xfffffe00ac7747b0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00ac774810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00ac774840 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00ac7748a0 dofileread() at dofileread+0x81/frame 0xfffffe00ac7748f0 kern_readv() at kern_readv+0x49/frame 0xfffffe00ac774930 sys_read() at sys_read+0x84/frame 0xfffffe00ac774980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00ac774ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ac774ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004311fa, rsp = 0x7fffffffebc8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2331 tid 100207 td 0xfffff80006122000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00ac76f620 mi_switch() at mi_switch+0x172/frame 0xfffffe00ac76f650 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ac76f690 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ac76f6f0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ac76f720 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ac76f780 tty_wait() at tty_wait+0x42/frame 0xfffffe00ac76f7b0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00ac76f810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00ac76f840 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00ac76f8a0 dofileread() at dofileread+0x81/frame 0xfffffe00ac76f8f0 kern_readv() at kern_readv+0x49/frame 0xfffffe00ac76f930 sys_read() at sys_read+0x84/frame 0xfffffe00ac76f980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00ac76fab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ac76fab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004311fa, rsp = 0x7fffffffebc8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2330 tid 100206 td 0xfffff800061225a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00ac76a620 mi_switch() at mi_switch+0x172/frame 0xfffffe00ac76a650 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ac76a690 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ac76a6f0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ac76a720 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ac76a780 tty_wait() at tty_wait+0x42/frame 0xfffffe00ac76a7b0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00ac76a810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00ac76a840 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00ac76a8a0 dofileread() at dofileread+0x81/frame 0xfffffe00ac76a8f0 kern_readv() at kern_readv+0x49/frame 0xfffffe00ac76a930 sys_read() at sys_read+0x84/frame 0xfffffe00ac76a980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00ac76aab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ac76aab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004311fa, rsp = 0x7fffffffebc8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2329 tid 100205 td 0xfffff8000613d000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00ac765620 mi_switch() at mi_switch+0x172/frame 0xfffffe00ac765650 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ac765690 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ac7656f0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ac765720 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ac765780 tty_wait() at tty_wait+0x42/frame 0xfffffe00ac7657b0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00ac765810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00ac765840 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00ac7658a0 dofileread() at dofileread+0x81/frame 0xfffffe00ac7658f0 kern_readv() at kern_readv+0x49/frame 0xfffffe00ac765930 sys_read() at sys_read+0x84/frame 0xfffffe00ac765980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00ac765ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ac765ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004311fa, rsp = 0x7fffffffebc8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2328 tid 100194 td 0xfffff8000613d5a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00ac733620 mi_switch() at mi_switch+0x172/frame 0xfffffe00ac733650 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ac733690 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ac7336f0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ac733720 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ac733780 tty_wait() at tty_wait+0x42/frame 0xfffffe00ac7337b0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00ac733810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00ac733840 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00ac7338a0 dofileread() at dofileread+0x81/frame 0xfffffe00ac7338f0 kern_readv() at kern_readv+0x49/frame 0xfffffe00ac733930 sys_read() at sys_read+0x84/frame 0xfffffe00ac733980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00ac733ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ac733ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004311fa, rsp = 0x7fffffffebc8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2327 tid 100150 td 0xfffff80007ab15a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00aa7ea620 mi_switch() at mi_switch+0x172/frame 0xfffffe00aa7ea650 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00aa7ea690 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00aa7ea6f0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00aa7ea720 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00aa7ea780 tty_wait() at tty_wait+0x42/frame 0xfffffe00aa7ea7b0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00aa7ea810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00aa7ea840 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00aa7ea8a0 dofileread() at dofileread+0x81/frame 0xfffffe00aa7ea8f0 kern_readv() at kern_readv+0x49/frame 0xfffffe00aa7ea930 sys_read() at sys_read+0x84/frame 0xfffffe00aa7ea980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00aa7eaab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00aa7eaab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004311fa, rsp = 0x7fffffffebc8, rbp = 0x7fffffffec10 --- Tracing command getty pid 2326 tid 100140 td 0xfffff800061c85a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00aa7b8620 mi_switch() at mi_switch+0x172/frame 0xfffffe00aa7b8650 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00aa7b8690 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00aa7b86f0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00aa7b8720 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00aa7b8780 tty_wait() at tty_wait+0x42/frame 0xfffffe00aa7b87b0 ttydisc_read() at ttydisc_read+0x20a/frame 0xfffffe00aa7b8810 ttydev_read() at ttydev_read+0x4b/frame 0xfffffe00aa7b8840 devfs_read_f() at devfs_read_f+0xde/frame 0xfffffe00aa7b88a0 dofileread() at dofileread+0x81/frame 0xfffffe00aa7b88f0 kern_readv() at kern_readv+0x49/frame 0xfffffe00aa7b8930 sys_read() at sys_read+0x84/frame 0xfffffe00aa7b8980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00aa7b8ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00aa7b8ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8004311fa, rsp = 0x7fffffffebc8, rbp = 0x7fffffffec10 --- Tracing command cron pid 2280 tid 100200 td 0xfffff8002da155a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00ac74c710 mi_switch() at mi_switch+0x172/frame 0xfffffe00ac74c740 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ac74c780 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ac74c7e0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00ac74c820 _sleep() at _sleep+0x265/frame 0xfffffe00ac74c8c0 kern_clock_nanosleep() at kern_clock_nanosleep+0x206/frame 0xfffffe00ac74c940 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe00ac74c980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00ac74cab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ac74cab0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003d4a0a, rsp = 0x7fffffffead8, rbp = 0x7fffffffecc0 --- Tracing command sshd pid 2276 tid 100175 td 0xfffff8000644f5a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00ac68c5c0 mi_switch() at mi_switch+0x172/frame 0xfffffe00ac68c5f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ac68c630 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ac68c690 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ac68c6c0 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ac68c720 seltdwait() at seltdwait+0x8d/frame 0xfffffe00ac68c760 kern_select() at kern_select+0x95a/frame 0xfffffe00ac68c940 sys_select() at sys_select+0x56/frame 0xfffffe00ac68c980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00ac68cab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ac68cab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8009021da, rsp = 0x7fffffffe668, rbp = 0x7fffffffe6a0 --- Tracing command ntpd pid 2247 tid 100197 td 0xfffff8002da1a5a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00ac7425c0 mi_switch() at mi_switch+0x172/frame 0xfffffe00ac7425f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ac742630 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ac742690 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ac7426c0 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ac742720 seltdwait() at seltdwait+0x8d/frame 0xfffffe00ac742760 kern_select() at kern_select+0x95a/frame 0xfffffe00ac742940 sys_select() at sys_select+0x56/frame 0xfffffe00ac742980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00ac742ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ac742ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80082e1da, rsp = 0x7fffffffdf18, rbp = 0x7fffffffdf50 --- Tracing command ntpd pid 2247 tid 102739 td 0xfffff8017a0625a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe02c7b806b0 mi_switch() at mi_switch+0x172/frame 0xfffffe02c7b806e0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe02c7b80720 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe02c7b80780 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe02c7b807b0 _sleep() at _sleep+0x2d0/frame 0xfffffe02c7b80850 umtxq_sleep() at umtxq_sleep+0x153/frame 0xfffffe02c7b808b0 do_sem2_wait() at do_sem2_wait+0x41c/frame 0xfffffe02c7b80930 __umtx_op_sem2_wait() at __umtx_op_sem2_wait+0x76/frame 0xfffffe02c7b80980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe02c7b80ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe02c7b80ab0 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x8007225aa, rsp = 0x7fffdfef9ed8, rbp = 0x7fffdfef9f50 --- Tracing command rpc.lockd pid 2184 tid 100191 td 0xfffff80004057000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00ac70c4e0 mi_switch() at mi_switch+0x172/frame 0xfffffe00ac70c510 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ac70c550 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ac70c5b0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00ac70c5f0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x185/frame 0xfffffe00ac70c660 svc_run_internal() at svc_run_internal+0xa90/frame 0xfffffe00ac70c7a0 svc_run() at svc_run+0x24e/frame 0xfffffe00ac70c800 sys_nlm_syscall() at sys_nlm_syscall+0x763/frame 0xfffffe00ac70c980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00ac70cab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ac70cab0 --- syscall (154, FreeBSD ELF64, sys_nlm_syscall), rip = 0x8002f062a, rsp = 0x7fffffffec08, rbp = 0x7fffffffecc0 --- Tracing command rpc.statd pid 2181 tid 100159 td 0xfffff80007e81000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00ac6185a0 mi_switch() at mi_switch+0x172/frame 0xfffffe00ac6185d0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ac618610 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ac618670 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00ac6186b0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x185/frame 0xfffffe00ac618720 seltdwait() at seltdwait+0x6b/frame 0xfffffe00ac618760 kern_select() at kern_select+0x95a/frame 0xfffffe00ac618940 sys_select() at sys_select+0x56/frame 0xfffffe00ac618980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00ac618ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ac618ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8004251da, rsp = 0x7fffffffea98, rbp = 0x7fffffffebe0 --- Tracing command watchdogd pid 2135 tid 100163 td 0xfffff8000644e000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00ac62c710 mi_switch() at mi_switch+0x172/frame 0xfffffe00ac62c740 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ac62c780 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ac62c7e0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00ac62c820 _sleep() at _sleep+0x265/frame 0xfffffe00ac62c8c0 kern_clock_nanosleep() at kern_clock_nanosleep+0x206/frame 0xfffffe00ac62c940 sys_nanosleep() at sys_nanosleep+0x5f/frame 0xfffffe00ac62c980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00ac62cab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ac62cab0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8003bda0a, rsp = 0x7fffffffea98, rbp = 0x7fffffffeae0 --- Tracing command rpcbind pid 2056 tid 100164 td 0xfffff800122a25a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00ac631620 mi_switch() at mi_switch+0x172/frame 0xfffffe00ac631650 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ac631690 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ac6316f0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00ac631730 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x185/frame 0xfffffe00ac6317a0 seltdwait() at seltdwait+0x6b/frame 0xfffffe00ac6317e0 kern_poll() at kern_poll+0x448/frame 0xfffffe00ac631960 sys_poll() at sys_poll+0x50/frame 0xfffffe00ac631980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00ac631ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ac631ab0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8003cc1fa, rsp = 0x7fffffffca38, rbp = 0x7fffffffec50 --- Tracing command syslogd pid 2046 tid 100178 td 0xfffff8000780f000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00ac69b5c0 mi_switch() at mi_switch+0x172/frame 0xfffffe00ac69b5f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ac69b630 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ac69b690 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ac69b6c0 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ac69b720 seltdwait() at seltdwait+0x8d/frame 0xfffffe00ac69b760 kern_select() at kern_select+0x95a/frame 0xfffffe00ac69b940 sys_select() at sys_select+0x56/frame 0xfffffe00ac69b980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00ac69bab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ac69bab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8004361da, rsp = 0x7fffffffe748, rbp = 0x7fffffffecc0 --- Tracing command devd pid 1975 tid 100174 td 0xfffff80003c675a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00ac6875a0 mi_switch() at mi_switch+0x172/frame 0xfffffe00ac6875d0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ac687610 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ac687670 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00ac6876b0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x185/frame 0xfffffe00ac687720 seltdwait() at seltdwait+0x6b/frame 0xfffffe00ac687760 kern_select() at kern_select+0x95a/frame 0xfffffe00ac687940 sys_select() at sys_select+0x56/frame 0xfffffe00ac687980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00ac687ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ac687ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x2f533a, rsp = 0x7fffffffcb48, rbp = 0x7fffffffece0 --- Tracing command dhclient pid 1970 tid 100153 td 0xfffff800060f45a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00aa7f9620 mi_switch() at mi_switch+0x172/frame 0xfffffe00aa7f9650 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00aa7f9690 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00aa7f96f0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00aa7f9730 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x185/frame 0xfffffe00aa7f97a0 seltdwait() at seltdwait+0x6b/frame 0xfffffe00aa7f97e0 kern_poll() at kern_poll+0x448/frame 0xfffffe00aa7f9960 sys_poll() at sys_poll+0x50/frame 0xfffffe00aa7f9980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00aa7f9ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00aa7f9ab0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8003ef1fa, rsp = 0x7fffffffec28, rbp = 0x7fffffffec60 --- Tracing command dhclient pid 1926 tid 100172 td 0xfffff800066995a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00ac67d640 mi_switch() at mi_switch+0x172/frame 0xfffffe00ac67d670 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ac67d6b0 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ac67d710 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ac67d740 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ac67d7a0 seltdwait() at seltdwait+0x8d/frame 0xfffffe00ac67d7e0 kern_poll() at kern_poll+0x448/frame 0xfffffe00ac67d960 sys_poll() at sys_poll+0x50/frame 0xfffffe00ac67d980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00ac67dab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ac67dab0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8003ef1fa, rsp = 0x7fffffffec38, rbp = 0x7fffffffec60 --- Tracing command dhclient pid 1923 tid 100165 td 0xfffff800122a2000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00ac6365c0 mi_switch() at mi_switch+0x172/frame 0xfffffe00ac6365f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ac636630 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe00ac636690 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00ac6366c0 _cv_wait_sig() at _cv_wait_sig+0x162/frame 0xfffffe00ac636720 seltdwait() at seltdwait+0x8d/frame 0xfffffe00ac636760 kern_select() at kern_select+0x95a/frame 0xfffffe00ac636940 sys_select() at sys_select+0x56/frame 0xfffffe00ac636980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe00ac636ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00ac636ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80044d1da, rsp = 0x7fffffffe9e8, rbp = 0x7fffffffeaa0 --- Tracing command vnlru pid 32 tid 100135 td 0xfffff8000780d000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00a2ffd890 mi_switch() at mi_switch+0x172/frame 0xfffffe00a2ffd8c0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00a2ffd900 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00a2ffd940 _sleep() at _sleep+0x279/frame 0xfffffe00a2ffd9e0 vnlru_proc() at vnlru_proc+0xa4/frame 0xfffffe00a2ffda70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a2ffdab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a2ffdab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command syncer pid 31 tid 100132 td 0xfffff8000780d5a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00a2fee8c0 mi_switch() at mi_switch+0x172/frame 0xfffffe00a2fee8f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00a2fee930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00a2fee970 _cv_timedwait_sbt() at _cv_timedwait_sbt+0x185/frame 0xfffffe00a2fee9e0 sched_sync() at sched_sync+0x609/frame 0xfffffe00a2feea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a2feeab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a2feeab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 30 tid 100128 td 0xfffff8000780c000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006f178d0 mi_switch() at mi_switch+0x172/frame 0xfffffe0006f17900 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006f17940 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006f17980 _sleep() at _sleep+0x279/frame 0xfffffe0006f17a20 buf_daemon() at buf_daemon+0xc8/frame 0xfffffe0006f17a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f17ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f17ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 30 tid 100131 td 0xfffff80007a9a000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00a2fe98c0 mi_switch() at mi_switch+0x172/frame 0xfffffe00a2fe98f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00a2fe9930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00a2fe9970 _sleep() at _sleep+0x279/frame 0xfffffe00a2fe9a10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe00a2fe9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a2fe9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a2fe9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 30 tid 100133 td 0xfffff80007a9b5a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00a2ff38c0 mi_switch() at mi_switch+0x172/frame 0xfffffe00a2ff38f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00a2ff3930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00a2ff3970 _sleep() at _sleep+0x279/frame 0xfffffe00a2ff3a10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe00a2ff3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a2ff3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a2ff3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 30 tid 100134 td 0xfffff80007a9b000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00a2ff88c0 mi_switch() at mi_switch+0x172/frame 0xfffffe00a2ff88f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00a2ff8930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00a2ff8970 _sleep() at _sleep+0x279/frame 0xfffffe00a2ff8a10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe00a2ff8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a2ff8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a2ff8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 30 tid 100136 td 0xfffff80007a9c5a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00aa7a48c0 mi_switch() at mi_switch+0x172/frame 0xfffffe00aa7a48f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00aa7a4930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00aa7a4970 _sleep() at _sleep+0x279/frame 0xfffffe00aa7a4a10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe00aa7a4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00aa7a4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00aa7a4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 30 tid 100137 td 0xfffff80007a9c000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00aa7a98c0 mi_switch() at mi_switch+0x172/frame 0xfffffe00aa7a98f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00aa7a9930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00aa7a9970 _sleep() at _sleep+0x279/frame 0xfffffe00aa7a9a10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe00aa7a9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00aa7a9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00aa7a9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 30 tid 100138 td 0xfffff80007a9d5a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00aa7ae8c0 mi_switch() at mi_switch+0x172/frame 0xfffffe00aa7ae8f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00aa7ae930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00aa7ae970 _sleep() at _sleep+0x279/frame 0xfffffe00aa7aea10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe00aa7aea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00aa7aeab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00aa7aeab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 30 tid 100139 td 0xfffff80007a9d000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00aa7b38c0 mi_switch() at mi_switch+0x172/frame 0xfffffe00aa7b38f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00aa7b3930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00aa7b3970 _sleep() at _sleep+0x279/frame 0xfffffe00aa7b3a10 bufspace_daemon() at bufspace_daemon+0x2eb/frame 0xfffffe00aa7b3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00aa7b3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00aa7b3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 30 tid 100151 td 0xfffff80007ab1000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00aa7ef8f0 mi_switch() at mi_switch+0x172/frame 0xfffffe00aa7ef920 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00aa7ef960 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00aa7ef9a0 _sleep() at _sleep+0x279/frame 0xfffffe00aa7efa40 softdep_flush() at softdep_flush+0x17f/frame 0xfffffe00aa7efa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00aa7efab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00aa7efab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 30 tid 100196 td 0xfffff800126865a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00ac73d8f0 mi_switch() at mi_switch+0x172/frame 0xfffffe00ac73d920 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00ac73d960 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00ac73d9a0 _sleep() at _sleep+0x279/frame 0xfffffe00ac73da40 softdep_flush() at softdep_flush+0x17f/frame 0xfffffe00ac73da70 fork_exit() at fork_exit+0x84/frame 0xfffffe00ac73dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ac73dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command bufdaemon pid 30 tid 100168 td 0xfffff801d7cb1000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00adfbc8f0 mi_switch() at mi_switch+0x172/frame 0xfffffe00adfbc920 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00adfbc960 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00adfbc9a0 _sleep() at _sleep+0x279/frame 0xfffffe00adfbca40 softdep_flush() at softdep_flush+0x17f/frame 0xfffffe00adfbca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00adfbcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00adfbcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command vmdaemon pid 29 tid 100127 td 0xfffff800065e05a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006f068c0 mi_switch() at mi_switch+0x172/frame 0xfffffe0006f068f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006f06930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006f06960 _sleep() at _sleep+0x2da/frame 0xfffffe0006f06a00 vm_daemon() at vm_daemon+0xa6f/frame 0xfffffe0006f06a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f06ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f06ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 28 tid 100126 td 0xfffff80004356000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006f01520 mi_switch() at mi_switch+0x172/frame 0xfffffe0006f01550 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006f01590 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006f015d0 _sleep() at _sleep+0x279/frame 0xfffffe0006f01670 vm_pageout_worker() at vm_pageout_worker+0x26b/frame 0xfffffe0006f01a30 vm_pageout() at vm_pageout+0x176/frame 0xfffffe0006f01a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006f01ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006f01ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 28 tid 100129 td 0xfffff800061c75a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006ffc660 mi_switch() at mi_switch+0x172/frame 0xfffffe0006ffc690 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ffc6d0 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006ffc710 _sleep() at _sleep+0x279/frame 0xfffffe0006ffc7b0 pause_sbt() at pause_sbt+0x10f/frame 0xfffffe0006ffc7f0 vm_pageout_laundry_worker() at vm_pageout_laundry_worker+0xb3c/frame 0xfffffe0006ffca70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ffcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ffcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command pagedaemon pid 28 tid 100130 td 0xfffff800061c7000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00a2fe48f0 mi_switch() at mi_switch+0x172/frame 0xfffffe00a2fe4920 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00a2fe4960 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe00a2fe49a0 _sleep() at _sleep+0x279/frame 0xfffffe00a2fe4a40 uma_reclaim_worker() at uma_reclaim_worker+0xda/frame 0xfffffe00a2fe4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00a2fe4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a2fe4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command rand_harvestq pid 27 tid 100120 td 0xfffff80004346000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006ee38c0 mi_switch() at mi_switch+0x172/frame 0xfffffe0006ee38f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ee3930 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006ee3970 _sleep() at _sleep+0x279/frame 0xfffffe0006ee3a10 random_kthread() at random_kthread+0x26a/frame 0xfffffe0006ee3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ee3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ee3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sctp_iterator pid 26 tid 100119 td 0xfffff800043465a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006ed2910 mi_switch() at mi_switch+0x172/frame 0xfffffe0006ed2940 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ed2980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006ed29b0 _sleep() at _sleep+0x2da/frame 0xfffffe0006ed2a50 sctp_iterator_thread() at sctp_iterator_thread+0x59/frame 0xfffffe0006ed2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ed2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ed2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod4 pid 25 tid 100118 td 0xfffff800065e25a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006ec18e0 mi_switch() at mi_switch+0x172/frame 0xfffffe0006ec1910 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ec1950 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006ec1990 _sleep() at _sleep+0x279/frame 0xfffffe0006ec1a30 soaio_kproc_loop() at soaio_kproc_loop+0x170/frame 0xfffffe0006ec1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ec1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ec1ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod3 pid 24 tid 100117 td 0xfffff800065e1000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006ebc8e0 mi_switch() at mi_switch+0x172/frame 0xfffffe0006ebc910 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ebc950 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006ebc990 _sleep() at _sleep+0x279/frame 0xfffffe0006ebca30 soaio_kproc_loop() at soaio_kproc_loop+0x170/frame 0xfffffe0006ebca70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ebcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ebcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod2 pid 23 tid 100116 td 0xfffff800065e15a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006eb78e0 mi_switch() at mi_switch+0x172/frame 0xfffffe0006eb7910 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006eb7950 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006eb7990 _sleep() at _sleep+0x279/frame 0xfffffe0006eb7a30 soaio_kproc_loop() at soaio_kproc_loop+0x170/frame 0xfffffe0006eb7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006eb7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006eb7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command soaiod1 pid 22 tid 100115 td 0xfffff800065e0000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006eb28e0 mi_switch() at mi_switch+0x172/frame 0xfffffe0006eb2910 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006eb2950 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xfffffe0006eb2990 _sleep() at _sleep+0x279/frame 0xfffffe0006eb2a30 soaio_kproc_loop() at soaio_kproc_loop+0x170/frame 0xfffffe0006eb2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006eb2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006eb2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100096 td 0xfffff800041785a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006e3b930 mi_switch() at mi_switch+0x172/frame 0xfffffe0006e3b960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e3b9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e3b9d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e3ba30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e3ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e3bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e3bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100097 td 0xfffff80004178000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006e40930 mi_switch() at mi_switch+0x172/frame 0xfffffe0006e40960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e409a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e409d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e40a30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e40a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e40ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e40ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100098 td 0xfffff800041795a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006e45930 mi_switch() at mi_switch+0x172/frame 0xfffffe0006e45960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e459a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e459d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e45a30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e45a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e45ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e45ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100099 td 0xfffff80004179000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006e4a930 mi_switch() at mi_switch+0x172/frame 0xfffffe0006e4a960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e4a9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e4a9d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e4aa30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e4aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e4aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e4aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100100 td 0xfffff800041875a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006e4f930 mi_switch() at mi_switch+0x172/frame 0xfffffe0006e4f960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e4f9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e4f9d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e4fa30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e4fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e4fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e4fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100102 td 0xfffff80003c645a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006e59930 mi_switch() at mi_switch+0x172/frame 0xfffffe0006e59960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e599a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e599d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e59a30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e59a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e59ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e59ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100103 td 0xfffff800060f1000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006e5e930 mi_switch() at mi_switch+0x172/frame 0xfffffe0006e5e960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e5e9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e5e9d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e5ea30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e5ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e5eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e5eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100104 td 0xfffff800060f25a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006e63930 mi_switch() at mi_switch+0x172/frame 0xfffffe0006e63960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e639a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e639d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e63a30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e63a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e63ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e63ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100105 td 0xfffff800060f2000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006e68930 mi_switch() at mi_switch+0x172/frame 0xfffffe0006e68960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e689a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e689d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e68a30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e68a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e68ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e68ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100106 td 0xfffff800060f35a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006e6d930 mi_switch() at mi_switch+0x172/frame 0xfffffe0006e6d960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e6d9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e6d9d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e6da30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e6da70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e6dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e6dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100107 td 0xfffff800037d15a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006e8a930 mi_switch() at mi_switch+0x172/frame 0xfffffe0006e8a960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e8a9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e8a9d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e8aa30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e8aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e8aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e8aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100108 td 0xfffff800037d1000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006e8f930 mi_switch() at mi_switch+0x172/frame 0xfffffe0006e8f960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e8f9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e8f9d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e8fa30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e8fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e8fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e8fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100109 td 0xfffff800037d25a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006e94930 mi_switch() at mi_switch+0x172/frame 0xfffffe0006e94960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e949a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e949d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e94a30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e94a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e94ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e94ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100110 td 0xfffff800037d2000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006e99930 mi_switch() at mi_switch+0x172/frame 0xfffffe0006e99960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e999a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e999d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e99a30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e99a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e99ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e99ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command usb pid 21 tid 100111 td 0xfffff800037d35a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006e9e930 mi_switch() at mi_switch+0x172/frame 0xfffffe0006e9e960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e9e9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e9e9d0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe0006e9ea30 usb_process() at usb_process+0x13b/frame 0xfffffe0006e9ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e9eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e9eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100091 td 0xfffff80003c64000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006e228e0 mi_switch() at mi_switch+0x172/frame 0xfffffe0006e22910 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e22950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e22980 _sleep() at _sleep+0x2da/frame 0xfffffe0006e22a20 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe0006e22a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e22ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e22ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100092 td 0xfffff80003c655a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006e278e0 mi_switch() at mi_switch+0x172/frame 0xfffffe0006e27910 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e27950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e27980 _sleep() at _sleep+0x2da/frame 0xfffffe0006e27a20 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe0006e27a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e27ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e27ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100093 td 0xfffff80003c65000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006e2c8e0 mi_switch() at mi_switch+0x172/frame 0xfffffe0006e2c910 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e2c950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e2c980 _sleep() at _sleep+0x2da/frame 0xfffffe0006e2ca20 xpt_done_td() at xpt_done_td+0x7b/frame 0xfffffe0006e2ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e2cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e2cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command cam pid 20 tid 100125 td 0xfffff800043565a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006efc8d0 mi_switch() at mi_switch+0x172/frame 0xfffffe0006efc900 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006efc940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006efc970 _sleep() at _sleep+0x2da/frame 0xfffffe0006efca10 xpt_scanner_thread() at xpt_scanner_thread+0x93/frame 0xfffffe0006efca70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006efcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006efcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 11 pid 19 tid 100090 td 0xfffff80003ad15a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006e1d8c0 mi_switch() at mi_switch+0x172/frame 0xfffffe0006e1d8f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e1d930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e1d960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e1da00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e1da70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e1dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e1dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 10 pid 18 tid 100089 td 0xfffff80003ac2000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006e188c0 mi_switch() at mi_switch+0x172/frame 0xfffffe0006e188f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e18930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e18960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e18a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e18a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e18ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e18ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 9 pid 17 tid 100088 td 0xfffff80003ac25a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006e138c0 mi_switch() at mi_switch+0x172/frame 0xfffffe0006e138f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e13930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e13960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e13a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e13a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e13ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e13ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 8 pid 16 tid 100087 td 0xfffff80003ac1000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006e0e8c0 mi_switch() at mi_switch+0x172/frame 0xfffffe0006e0e8f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e0e930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e0e960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e0ea00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e0ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e0eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e0eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 7 pid 15 tid 100086 td 0xfffff80003ac15a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006e098c0 mi_switch() at mi_switch+0x172/frame 0xfffffe0006e098f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e09930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e09960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e09a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e09a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e09ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e09ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 6 pid 9 tid 100085 td 0xfffff80003ac0000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006e048c0 mi_switch() at mi_switch+0x172/frame 0xfffffe0006e048f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006e04930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006e04960 _sleep() at _sleep+0x2da/frame 0xfffffe0006e04a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe0006e04a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e04ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e04ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 5 pid 8 tid 100084 td 0xfffff80003ac05a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00005fb8c0 mi_switch() at mi_switch+0x172/frame 0xfffffe00005fb8f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005fb930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005fb960 _sleep() at _sleep+0x2da/frame 0xfffffe00005fba00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00005fba70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005fbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005fbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 4 pid 7 tid 100083 td 0xfffff80003abf000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00005f68c0 mi_switch() at mi_switch+0x172/frame 0xfffffe00005f68f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005f6930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005f6960 _sleep() at _sleep+0x2da/frame 0xfffffe00005f6a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00005f6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005f6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005f6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 3 pid 6 tid 100082 td 0xfffff80003abf5a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00005f18c0 mi_switch() at mi_switch+0x172/frame 0xfffffe00005f18f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005f1930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005f1960 _sleep() at _sleep+0x2da/frame 0xfffffe00005f1a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00005f1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005f1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005f1ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 2 pid 5 tid 100081 td 0xfffff80003ad5000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00005ec8c0 mi_switch() at mi_switch+0x172/frame 0xfffffe00005ec8f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005ec930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005ec960 _sleep() at _sleep+0x2da/frame 0xfffffe00005eca00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00005eca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005ecab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005ecab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 1 pid 4 tid 100080 td 0xfffff80003ad55a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00005e78c0 mi_switch() at mi_switch+0x172/frame 0xfffffe00005e78f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005e7930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005e7960 _sleep() at _sleep+0x2da/frame 0xfffffe00005e7a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00005e7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005e7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005e7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto returns 0 pid 3 tid 100079 td 0xfffff80003ad4000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00005e28c0 mi_switch() at mi_switch+0x172/frame 0xfffffe00005e28f0 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005e2930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005e2960 _sleep() at _sleep+0x2da/frame 0xfffffe00005e2a00 crypto_ret_proc() at crypto_ret_proc+0x238/frame 0xfffffe00005e2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005e2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005e2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command crypto pid 2 tid 100078 td 0xfffff80003ad45a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00005dd8f0 mi_switch() at mi_switch+0x172/frame 0xfffffe00005dd920 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005dd960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005dd990 _sleep() at _sleep+0x2da/frame 0xfffffe00005dda30 crypto_proc() at crypto_proc+0x2c4/frame 0xfffffe00005dda70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005ddab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005ddab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sequencer 00 pid 14 tid 100065 td 0xfffff80003b25000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe000059c8f0 mi_switch() at mi_switch+0x172/frame 0xfffffe000059c920 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe000059c960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000059c990 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe000059c9f0 seq_eventthread() at seq_eventthread+0xdc/frame 0xfffffe000059ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe000059cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000059cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100062 td 0xfffff800037155a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00005818f0 mi_switch() at mi_switch+0x172/frame 0xfffffe0000581920 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0000581960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0000581990 _sleep() at _sleep+0x2da/frame 0xfffffe0000581a30 g_run_events() at g_run_events+0x4f/frame 0xfffffe0000581a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000581ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000581ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100063 td 0xfffff80003715000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00005868f0 mi_switch() at mi_switch+0x172/frame 0xfffffe0000586920 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0000586960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0000586990 _sleep() at _sleep+0x2da/frame 0xfffffe0000586a30 g_io_schedule_up() at g_io_schedule_up+0xa5/frame 0xfffffe0000586a60 g_up_procbody() at g_up_procbody+0x6d/frame 0xfffffe0000586a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000586ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000586ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command geom pid 13 tid 100064 td 0xfffff80003b255a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe000058b8f0 mi_switch() at mi_switch+0x172/frame 0xfffffe000058b920 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe000058b960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000058b990 _sleep() at _sleep+0x2da/frame 0xfffffe000058ba30 g_io_schedule_down() at g_io_schedule_down+0x14c/frame 0xfffffe000058ba60 g_down_procbody() at g_down_procbody+0x6d/frame 0xfffffe000058ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe000058bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000058bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100015 td 0xfffff8000379e000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00767ae9e0 mi_switch() at mi_switch+0x172/frame 0xfffffe00767aea10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe00767aea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00767aeab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00767aeab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100016 td 0xfffff8000379f5a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00767b39e0 mi_switch() at mi_switch+0x172/frame 0xfffffe00767b3a10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe00767b3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00767b3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00767b3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100018 td 0xfffff800037ad5a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100047 td 0xfffff80003718000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00004ee9e0 mi_switch() at mi_switch+0x172/frame 0xfffffe00004eea10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe00004eea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004eeab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004eeab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100048 td 0xfffff800037195a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00004f39e0 mi_switch() at mi_switch+0x172/frame 0xfffffe00004f3a10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe00004f3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004f3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004f3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100049 td 0xfffff80003719000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00004f89e0 mi_switch() at mi_switch+0x172/frame 0xfffffe00004f8a10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe00004f8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004f8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004f8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100050 td 0xfffff8000371a5a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00004fd9e0 mi_switch() at mi_switch+0x172/frame 0xfffffe00004fda10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe00004fda70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004fdab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004fdab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100051 td 0xfffff8000371a000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00005029e0 mi_switch() at mi_switch+0x172/frame 0xfffffe0000502a10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe0000502a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000502ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000502ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100052 td 0xfffff8000371b5a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00005079e0 mi_switch() at mi_switch+0x172/frame 0xfffffe0000507a10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe0000507a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000507ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000507ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100053 td 0xfffff8000371b000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe000050c9e0 mi_switch() at mi_switch+0x172/frame 0xfffffe000050ca10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe000050ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe000050cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000050cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100054 td 0xfffff8000371c5a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00005119e0 mi_switch() at mi_switch+0x172/frame 0xfffffe0000511a10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe0000511a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000511ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000511ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100055 td 0xfffff8000370a000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00005169e0 mi_switch() at mi_switch+0x172/frame 0xfffffe0000516a10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe0000516a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000516ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000516ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100056 td 0xfffff8000370e5a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe000051b9e0 mi_switch() at mi_switch+0x172/frame 0xfffffe000051ba10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe000051ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe000051bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000051bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100057 td 0xfffff8000370e000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00005209e0 mi_switch() at mi_switch+0x172/frame 0xfffffe0000520a10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe0000520a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000520ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000520ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100058 td 0xfffff800037135a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00005259e0 mi_switch() at mi_switch+0x172/frame 0xfffffe0000525a10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe0000525a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000525ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000525ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100059 td 0xfffff80003713000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100060 td 0xfffff800037145a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe000052f9e0 mi_switch() at mi_switch+0x172/frame 0xfffffe000052fa10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe000052fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe000052fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000052fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100094 td 0xfffff800041775a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006e319e0 mi_switch() at mi_switch+0x172/frame 0xfffffe0006e31a10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe0006e31a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e31ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e31ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100095 td 0xfffff80004177000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006e369e0 mi_switch() at mi_switch+0x172/frame 0xfffffe0006e36a10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe0006e36a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e36ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e36ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100101 td 0xfffff80004187000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006e549e0 mi_switch() at mi_switch+0x172/frame 0xfffffe0006e54a10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe0006e54a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006e54ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006e54ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100112 td 0xfffff800037d3000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100113 td 0xfffff800037dd5a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006ea89e0 mi_switch() at mi_switch+0x172/frame 0xfffffe0006ea8a10 ithread_loop() at ithread_loop+0x351/frame 0xfffffe0006ea8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ea8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ea8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100179 td 0xfffff80003c67000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100180 td 0xfffff80003c685a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100181 td 0xfffff80003c68000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100182 td 0xfffff80012248000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100183 td 0xfffff8001292f5a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100184 td 0xfffff8001292f000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100185 td 0xfffff800122f75a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100186 td 0xfffff800122f7000 fork_trampoline() at fork_trampoline Tracing command idle pid 11 tid 100003 td 0xfffff800037ae000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0076766950 mi_switch() at mi_switch+0x172/frame 0xfffffe0076766980 critical_exit_preempt() at critical_exit_preempt+0x65/frame 0xfffffe00767669a0 sched_idletd() at sched_idletd+0x442/frame 0xfffffe0076766a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076766ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076766ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xfffff800037af5a0 (CPU 1) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006b73df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006b73e10 trap() at trap+0x46/frame 0xfffffe0006b73f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b73f20 --- trap 0x13, rip = 0xffffffff81159db6, rsp = 0xfffffe007676b910, rbp = 0xfffffe007676b910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007676b910 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe007676b960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007676b980 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe007676b9a0 sched_idletd() at sched_idletd+0x442/frame 0xfffffe007676ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe007676bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007676bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffff800037af000 (CPU 2) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006b7fdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006b7fe10 trap() at trap+0x46/frame 0xfffffe0006b7ff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b7ff20 --- trap 0x13, rip = 0xffffffff81159db6, rsp = 0xfffffe0076770910, rbp = 0xfffffe0076770910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076770910 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076770960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076770980 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe00767709a0 sched_idletd() at sched_idletd+0x442/frame 0xfffffe0076770a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076770ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076770ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xfffff800037b05a0 (CPU 3) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006b8bdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006b8be10 trap() at trap+0x46/frame 0xfffffe0006b8bf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b8bf20 --- trap 0x13, rip = 0xffffffff81159db6, rsp = 0xfffffe0076775910, rbp = 0xfffffe0076775910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076775910 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076775960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076775980 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe00767759a0 sched_idletd() at sched_idletd+0x442/frame 0xfffffe0076775a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076775ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076775ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100007 td 0xfffff800037b0000 (CPU 4) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006b97df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006b97e10 trap() at trap+0x46/frame 0xfffffe0006b97f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006b97f20 --- trap 0x13, rip = 0xffffffff81159db6, rsp = 0xfffffe007677a910, rbp = 0xfffffe007677a910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007677a910 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe007677a960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007677a980 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe007677a9a0 sched_idletd() at sched_idletd+0x442/frame 0xfffffe007677aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007677aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007677aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xfffff800037b15a0 (CPU 5) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006ba3df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006ba3e10 trap() at trap+0x46/frame 0xfffffe0006ba3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006ba3f20 --- trap 0x13, rip = 0xffffffff81159db6, rsp = 0xfffffe007677f910, rbp = 0xfffffe007677f910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007677f910 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe007677f960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007677f980 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe007677f9a0 sched_idletd() at sched_idletd+0x442/frame 0xfffffe007677fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe007677fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007677fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xfffff800037b1000 (CPU 6) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bafdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bafe10 trap() at trap+0x46/frame 0xfffffe0006baff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006baff20 --- trap 0x13, rip = 0xffffffff81159db6, rsp = 0xfffffe0076784910, rbp = 0xfffffe0076784910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076784910 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076784960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076784980 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe00767849a0 sched_idletd() at sched_idletd+0x442/frame 0xfffffe0076784a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076784ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076784ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xfffff8000379c5a0 (CPU 7) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bbbdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bbbe10 trap() at trap+0x46/frame 0xfffffe0006bbbf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bbbf20 --- trap 0x13, rip = 0xffffffff81159db6, rsp = 0xfffffe0076789910, rbp = 0xfffffe0076789910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076789910 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076789960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076789980 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe00767899a0 sched_idletd() at sched_idletd+0x442/frame 0xfffffe0076789a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076789ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076789ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffff8000379c000 (CPU 8) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bc7df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bc7e10 trap() at trap+0x46/frame 0xfffffe0006bc7f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bc7f20 --- trap 0x13, rip = 0xffffffff81159db6, rsp = 0xfffffe007678e910, rbp = 0xfffffe007678e910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe007678e910 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe007678e960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe007678e980 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe007678e9a0 sched_idletd() at sched_idletd+0x442/frame 0xfffffe007678ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe007678eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007678eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100012 td 0xfffff8000379d5a0 (CPU 9) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bd3df0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bd3e10 trap() at trap+0x46/frame 0xfffffe0006bd3f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bd3f20 --- trap 0x13, rip = 0xffffffff81159db6, rsp = 0xfffffe0076793910, rbp = 0xfffffe0076793910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076793910 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076793960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076793980 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe00767939a0 sched_idletd() at sched_idletd+0x442/frame 0xfffffe0076793a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076793ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076793ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xfffff8000379d000 (CPU 10) cpustop_handler() at cpustop_handler+0x2d/frame 0xfffffe0006bdfdf0 ipi_nmi_handler() at ipi_nmi_handler+0x40/frame 0xfffffe0006bdfe10 trap() at trap+0x46/frame 0xfffffe0006bdff20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0006bdff20 --- trap 0x13, rip = 0xffffffff81159db6, rsp = 0xfffffe0076798910, rbp = 0xfffffe0076798910 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe0076798910 acpi_cpu_idle() at acpi_cpu_idle+0x235/frame 0xfffffe0076798960 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe0076798980 cpu_idle() at cpu_idle+0xa6/frame 0xfffffe00767989a0 sched_idletd() at sched_idletd+0x442/frame 0xfffffe0076798a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0076798ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0076798ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command idle pid 11 tid 100014 td 0xfffff8000379e5a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe007679d970 mi_switch() at mi_switch+0x172/frame 0xfffffe007679d9a0 sched_idletd() at sched_idletd+0xa3/frame 0xfffffe007679da70 fork_exit() at fork_exit+0x84/frame 0xfffffe007679dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007679dab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command init pid 1 tid 100002 td 0xfffff800037ae5a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0076761560 mi_switch() at mi_switch+0x172/frame 0xfffffe0076761590 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00767615d0 sleepq_catch_signals() at sleepq_catch_signals+0x417/frame 0xfffffe0076761630 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0076761660 _sleep() at _sleep+0x2d0/frame 0xfffffe0076761700 kern_wait6() at kern_wait6+0x48f/frame 0xfffffe0076761790 sys_wait4() at sys_wait4+0x78/frame 0xfffffe0076761980 amd64_syscall() at amd64_syscall+0x2d4/frame 0xfffffe0076761ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0076761ab0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x2b0f8a, rsp = 0x7fffffffe778, rbp = 0x7fffffffe7e0 --- Tracing command audit pid 10 tid 100001 td 0xfffff800037ad000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe007675c910 mi_switch() at mi_switch+0x172/frame 0xfffffe007675c940 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe007675c980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe007675c9b0 _cv_wait() at _cv_wait+0x15b/frame 0xfffffe007675ca10 audit_worker() at audit_worker+0x93/frame 0xfffffe007675ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe007675cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe007675cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xffffffff81fb9440 sched_switch() at sched_switch+0x3f0/frame 0xffffffff82720980 mi_switch() at mi_switch+0x172/frame 0xffffffff827209b0 sleepq_switch() at sleepq_switch+0x110/frame 0xffffffff827209f0 sleepq_timedwait() at sleepq_timedwait+0x4f/frame 0xffffffff82720a30 _sleep() at _sleep+0x279/frame 0xffffffff82720ad0 swapper() at swapper+0x77/frame 0xffffffff82720b20 mi_startup() at mi_startup+0x2ee/frame 0xffffffff82720b70 btext() at btext+0x2c Tracing command kernel pid 0 tid 100017 td 0xfffff8000379f000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00767b8900 mi_switch() at mi_switch+0x172/frame 0xfffffe00767b8930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00767b8970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00767b89a0 _sleep() at _sleep+0x2da/frame 0xfffffe00767b8a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00767b8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00767b8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00767b8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xfffff80003756000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00767c2930 mi_switch() at mi_switch+0x172/frame 0xfffffe00767c2960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00767c29a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00767c29d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00767c2a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00767c2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00767c2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00767c2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xfffff800037575a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00767c7900 mi_switch() at mi_switch+0x172/frame 0xfffffe00767c7930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00767c7970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00767c79a0 _sleep() at _sleep+0x2da/frame 0xfffffe00767c7a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00767c7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00767c7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00767c7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xfffff80003757000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00767cc900 mi_switch() at mi_switch+0x172/frame 0xfffffe00767cc930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00767cc970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00767cc9a0 _sleep() at _sleep+0x2da/frame 0xfffffe00767cca40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00767cca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00767ccab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00767ccab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xfffff800037585a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0000471930 mi_switch() at mi_switch+0x172/frame 0xfffffe0000471960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004719a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004719d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0000471a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000471a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000471ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000471ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xfffff80003758000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0000476930 mi_switch() at mi_switch+0x172/frame 0xfffffe0000476960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004769a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004769d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0000476a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000476a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000476ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000476ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xfffff800037595a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe000047b930 mi_switch() at mi_switch+0x172/frame 0xfffffe000047b960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe000047b9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000047b9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe000047ba40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000047ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe000047bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000047bab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xfffff80003759000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0000480930 mi_switch() at mi_switch+0x172/frame 0xfffffe0000480960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004809a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004809d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0000480a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000480a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000480ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000480ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100026 td 0xfffff8000375a5a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0000485930 mi_switch() at mi_switch+0x172/frame 0xfffffe0000485960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004859a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004859d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0000485a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000485a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000485ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000485ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100027 td 0xfffff8000375a000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe000048a930 mi_switch() at mi_switch+0x172/frame 0xfffffe000048a960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe000048a9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000048a9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe000048aa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000048aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe000048aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000048aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100028 td 0xfffff800037215a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe000048f930 mi_switch() at mi_switch+0x172/frame 0xfffffe000048f960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe000048f9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000048f9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe000048fa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000048fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe000048fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000048fab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100029 td 0xfffff80003721000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0000494930 mi_switch() at mi_switch+0x172/frame 0xfffffe0000494960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004949a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004949d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0000494a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000494a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000494ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000494ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100030 td 0xfffff800037225a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0000499930 mi_switch() at mi_switch+0x172/frame 0xfffffe0000499960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004999a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004999d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0000499a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe0000499a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000499ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000499ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100031 td 0xfffff80003722000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe000049e930 mi_switch() at mi_switch+0x172/frame 0xfffffe000049e960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe000049e9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000049e9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe000049ea40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe000049ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe000049eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000049eab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100032 td 0xfffff800037235a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00004a3930 mi_switch() at mi_switch+0x172/frame 0xfffffe00004a3960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004a39a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004a39d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004a3a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004a3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004a3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004a3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100033 td 0xfffff80003723000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00004a8930 mi_switch() at mi_switch+0x172/frame 0xfffffe00004a8960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004a89a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004a89d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004a8a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004a8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004a8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004a8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100034 td 0xfffff800037245a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00004ad930 mi_switch() at mi_switch+0x172/frame 0xfffffe00004ad960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004ad9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004ad9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004ada40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004ada70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004adab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004adab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100035 td 0xfffff80003724000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00004b2930 mi_switch() at mi_switch+0x172/frame 0xfffffe00004b2960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004b29a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004b29d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004b2a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004b2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004b2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004b2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100036 td 0xfffff800037565a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00004b7930 mi_switch() at mi_switch+0x172/frame 0xfffffe00004b7960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004b79a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004b79d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004b7a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004b7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004b7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004b7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100037 td 0xfffff8000371c000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00004bc930 mi_switch() at mi_switch+0x172/frame 0xfffffe00004bc960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004bc9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004bc9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004bca40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004bca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004bcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004bcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100038 td 0xfffff8000371d5a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00004c1930 mi_switch() at mi_switch+0x172/frame 0xfffffe00004c1960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004c19a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004c19d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004c1a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004c1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004c1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004c1ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100039 td 0xfffff8000371d000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00004c6930 mi_switch() at mi_switch+0x172/frame 0xfffffe00004c6960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004c69a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004c69d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004c6a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004c6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004c6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004c6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100040 td 0xfffff8000371e5a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00004cb930 mi_switch() at mi_switch+0x172/frame 0xfffffe00004cb960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004cb9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004cb9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004cba40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004cba70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004cbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004cbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100041 td 0xfffff8000371e000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00004d0930 mi_switch() at mi_switch+0x172/frame 0xfffffe00004d0960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004d09a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004d09d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004d0a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004d0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004d0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004d0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100042 td 0xfffff8000371f5a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00004d5930 mi_switch() at mi_switch+0x172/frame 0xfffffe00004d5960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004d59a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004d59d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004d5a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004d5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004d5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004d5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100043 td 0xfffff8000371f000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00004da930 mi_switch() at mi_switch+0x172/frame 0xfffffe00004da960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004da9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004da9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004daa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004daa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004daab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004daab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100044 td 0xfffff800037205a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00004df930 mi_switch() at mi_switch+0x172/frame 0xfffffe00004df960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004df9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004df9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004dfa40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004dfa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004dfab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004dfab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100045 td 0xfffff80003720000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00004e4930 mi_switch() at mi_switch+0x172/frame 0xfffffe00004e4960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004e49a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004e49d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004e4a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004e4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004e4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004e4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100046 td 0xfffff800037185a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00004e9930 mi_switch() at mi_switch+0x172/frame 0xfffffe00004e9960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00004e99a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00004e99d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe00004e9a40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb1/frame 0xfffffe00004e9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00004e9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004e9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100061 td 0xfffff80003714000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe000057c900 mi_switch() at mi_switch+0x172/frame 0xfffffe000057c930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe000057c970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000057c9a0 _sleep() at _sleep+0x2da/frame 0xfffffe000057ca40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe000057ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe000057cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000057cab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100066 td 0xfffff80003b265a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00005a1900 mi_switch() at mi_switch+0x172/frame 0xfffffe00005a1930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005a1970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005a19a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005a1a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005a1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005a1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005a1ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100067 td 0xfffff80003b26000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00005a6900 mi_switch() at mi_switch+0x172/frame 0xfffffe00005a6930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005a6970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005a69a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005a6a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005a6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005a6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005a6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100068 td 0xfffff80003b275a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00005ab900 mi_switch() at mi_switch+0x172/frame 0xfffffe00005ab930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005ab970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005ab9a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005aba40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005aba70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005abab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005abab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100069 td 0xfffff80003b27000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00005b0900 mi_switch() at mi_switch+0x172/frame 0xfffffe00005b0930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005b0970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005b09a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005b0a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005b0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005b0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005b0ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100070 td 0xfffff80003b285a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00005b5900 mi_switch() at mi_switch+0x172/frame 0xfffffe00005b5930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005b5970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005b59a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005b5a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005b5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005b5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005b5ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100071 td 0xfffff80003b28000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00005ba900 mi_switch() at mi_switch+0x172/frame 0xfffffe00005ba930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005ba970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005ba9a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005baa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005baa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005baab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005baab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100072 td 0xfffff8000370a5a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00005bf900 mi_switch() at mi_switch+0x172/frame 0xfffffe00005bf930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005bf970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005bf9a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005bfa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005bfa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005bfab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005bfab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100073 td 0xfffff80003ad1000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00005c4900 mi_switch() at mi_switch+0x172/frame 0xfffffe00005c4930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005c4970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005c49a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005c4a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005c4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005c4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005c4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100074 td 0xfffff80003ad25a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00005c9900 mi_switch() at mi_switch+0x172/frame 0xfffffe00005c9930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005c9970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005c99a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005c9a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005c9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005c9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005c9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100075 td 0xfffff80003ad2000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00005ce900 mi_switch() at mi_switch+0x172/frame 0xfffffe00005ce930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005ce970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005ce9a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005cea40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005cea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005ceab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005ceab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100076 td 0xfffff80003ad35a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00005d3900 mi_switch() at mi_switch+0x172/frame 0xfffffe00005d3930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005d3970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005d39a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005d3a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005d3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005d3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005d3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100077 td 0xfffff80003ad3000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe00005d8900 mi_switch() at mi_switch+0x172/frame 0xfffffe00005d8930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe00005d8970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00005d89a0 _sleep() at _sleep+0x2da/frame 0xfffffe00005d8a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe00005d8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00005d8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005d8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100114 td 0xfffff800037dd000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006ead930 mi_switch() at mi_switch+0x172/frame 0xfffffe0006ead960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ead9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006ead9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0006eada40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0006eada70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006eadab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006eadab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100121 td 0xfffff800043475a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006ee8930 mi_switch() at mi_switch+0x172/frame 0xfffffe0006ee8960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ee89a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006ee89d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0006ee8a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0006ee8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ee8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ee8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100122 td 0xfffff80004347000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006eed930 mi_switch() at mi_switch+0x172/frame 0xfffffe0006eed960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006eed9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006eed9d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0006eeda40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0006eeda70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006eedab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006eedab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100123 td 0xfffff800043485a0 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006ef2930 mi_switch() at mi_switch+0x172/frame 0xfffffe0006ef2960 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ef29a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006ef29d0 msleep_spin_sbt() at msleep_spin_sbt+0x153/frame 0xfffffe0006ef2a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe0006ef2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ef2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ef2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command kernel pid 0 tid 100124 td 0xfffff80004348000 sched_switch() at sched_switch+0x3f0/frame 0xfffffe0006ef7900 mi_switch() at mi_switch+0x172/frame 0xfffffe0006ef7930 sleepq_switch() at sleepq_switch+0x110/frame 0xfffffe0006ef7970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0006ef79a0 _sleep() at _sleep+0x2da/frame 0xfffffe0006ef7a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xe1/frame 0xfffffe0006ef7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0006ef7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0006ef7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db:0:pho> dump Dumping 4655 out of 32622 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Dump complete db> reset (kgdb) tid 104606 [Switching to thread 209 (Thread 104606)]#0 cpustop_handler () at ../../../x86/x86/mp_x86.c:1413 1413 use_mwait = (stop_mwait && (cpu_feature2 & CPUID2_MON) != 0 && Current language: auto; currently minimal (kgdb) bt #0 cpustop_handler () at ../../../x86/x86/mp_x86.c:1413 #1 0xffffffff81163e60 in ipi_nmi_handler () at ../../../x86/x86/mp_x86.c:1370 #2 0xffffffff81198af6 in trap (frame=0xfffffe0006bebf30) at ../../../amd64/amd64/trap.c:210 #3 0xffffffff811719e3 in nmi_calltrap () at ../../../amd64/amd64/exception.S:860 #4 0xffffffff80c30037 in sleepq_catch_signals (wchan=0xfffff80003b2dc68, pri=0x0) at ../../../kern/subr_sleepqueue.c:444 #5 0xffffffff80c2ff6f in sleepq_wait_sig (wchan=, pri=) at ../../../kern/subr_sleepqueue.c:718 #6 0xffffffff80b6db02 in _cv_wait_sig (cvp=0xfffff80003b2dc68, lock=0xfffff80003b2dc08) at ../../../kern/kern_condvar.c:252 #7 0xffffffff80990e3f in mseq_read (i_dev=, uio=0xfffffe00ade27940, ioflag=0x0) at ../../../dev/sound/midi/sequencer.c:914 #8 0xffffffff80a8c0ee in devfs_read_f (fp=0xfffff8002dfb3aa0, uio=0xfffffe00ade27940, cred=0xfffff8017ae7b000, flags=0x1, td=) at ../../../fs/devfs/devfs_vnops.c:1318 #9 0xffffffff80c46851 in dofileread (td=0xfffff8017ae7b000, fd=, fp=0xfffff8002dfb3aa0, auio=0xfffffe00ade27940, offset=, flags=) at file.h:313 #10 0xffffffff80c4665f in kern_preadv (td=0xfffff8017ae7b000, fd=0x10, auio=0xfffffe00ade27940, offset=0xcfdfd7c5371c927f) at ../../../kern/sys_generic.c:334 #11 0xffffffff80c4675b in freebsd6_pread (td=, uap=) at ../../../kern/sys_generic.c:243 #12 0xffffffff8119a254 in amd64_syscall (td=0xfffff8017ae7b000, traced=0x0) at subr_syscall.c:144 #13 0xffffffff811715a0 in fast_syscall_common () at ../../../amd64/amd64/exception.S:581 #14 0x00000008004442ba in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) f 7 #7 0xffffffff80990e3f in mseq_read (i_dev=, uio=0xfffffe00ade27940, ioflag=0x0) at ../../../dev/sound/midi/sequencer.c:914 914 retval = cv_wait_sig(&scp->in_cv, &scp->seq_lock); (kgdb) l 909 if (scp->fflags & O_NONBLOCK) 910 goto err1; 911 if (ioflag & O_NONBLOCK) 912 goto err1; 913 914 retval = cv_wait_sig(&scp->in_cv, &scp->seq_lock); 915 if (retval == EINTR) 916 goto err1; 917 } 918 (kgdb) info loc buf = 0xfffffe00ade277f0 "" scp = (struct seq_softc *) 0xfffff80003b2dc00 retval = 0x0 used = (kgdb) p *scp $1 = {ops = 0xfffff80003aea000, seq_lock = {lock_object = {lo_name = 0xffffffff812b3dde "seqflq", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xfffff8087fd73100}, mtx_lock = 0x0}, q_lock = {lock_object = {lo_name = 0x0, lo_flags = 0x0, lo_data = 0x0, lo_witness = 0x0}, mtx_lock = 0x0}, empty_cv = {cv_description = 0xffffffff8123fb16 "seqempty", cv_waiters = 0x0}, reset_cv = {cv_description = 0xffffffff81387faf "seqtimer", cv_waiters = 0x0}, in_cv = {cv_description = 0xffffffff812755c5 "seqqin", cv_waiters = 0x7fffffff}, out_cv = {cv_description = 0xffffffff812755bd "seqqout", cv_waiters = 0x0}, state_cv = {cv_description = 0xffffffff812fd373 "seqstate", cv_waiters = 0x1}, th_cv = {cv_description = 0xffffffff81387fb8 "seqstart", cv_waiters = 0x0}, in_q = { h = 0x0, t = 0x0, s = 0x2000, b = 0xfffffe00003fe000 ""}, out_q = {h = 0x0, t = 0x0, s = 0x2000, b = 0xfffffe0006be5000 ""}, flags = 0x0, fflags = 0x3, music = 0x0, out_water = 0x1000, sync_parm = {chan = 0x0, pos = 0x0}, sync_thread = 0x0, in_sel = {si_tdlist = { tqh_first = 0x0, tqh_last = 0x0}, si_note = {kl_list = {slh_first = 0x0}, kl_lock = 0, kl_unlock = 0, kl_assert_locked = 0, kl_assert_unlocked = 0, kl_lockarg = 0x0, kl_autodestroy = 0x0}, si_mtx = 0x0}, out_sel = {si_tdlist = {tqh_first = 0x0, tqh_last = 0x0}, si_note = {kl_list = {slh_first = 0x0}, kl_lock = 0, kl_unlock = 0, kl_assert_locked = 0, kl_assert_unlocked = 0, kl_lockarg = 0x0, kl_autodestroy = 0x0}, si_mtx = 0x0}, midi_number = 0x0, seqdev = 0xfffff80003772800, musicdev = 0xfffff80003772400, unit = 0x0, maxunits = 0x0, midis = 0xfffff80003126800, midi_flags = 0xfffff80003104a80, mapper = 0xfffff80003b2dc00, mapper_cookie = 0x0, timerstop = {tv_sec = 0x5d66f833, tv_usec = 0xb67be}, timersub = {tv_sec = 0x5d66f833, tv_usec = 0xb67be}, timerbase = 0x64, tempo = 0x3c, timerrun = 0x0, done = 0x0, playing = 0x0, recording = 0x0, busy = 0x1, pre_event_timeout = 0x0, waiting = 0x0} (kgdb) Index: src/sys/amd64/amd64/efirt_machdep.c =================================================================== --- src/sys/amd64/amd64/efirt_machdep.c (revision 351575) +++ src/sys/amd64/amd64/efirt_machdep.c (working copy) @@ -74,7 +74,7 @@ efi_destroy_1t1_map(void) if (obj_1t1_pt != NULL) { VM_OBJECT_RLOCK(obj_1t1_pt); TAILQ_FOREACH(m, &obj_1t1_pt->memq, listq) - m->wire_count = 0; + m->wire_count = VPRC_OBJREF; vm_wire_sub(obj_1t1_pt->resident_page_count); VM_OBJECT_RUNLOCK(obj_1t1_pt); vm_object_deallocate(obj_1t1_pt); Index: src/sys/amd64/amd64/pmap.c =================================================================== --- src/sys/amd64/amd64/pmap.c (revision 351575) +++ src/sys/amd64/amd64/pmap.c (working copy) @@ -3058,31 +3058,23 @@ pmap_extract_and_hold(pmap_t pmap, vm_offset_t va, m = NULL; PG_RW = pmap_rw_bit(pmap); PG_V = pmap_valid_bit(pmap); + PMAP_LOCK(pmap); -retry: pdep = pmap_pde(pmap, va); if (pdep != NULL && (pde = *pdep)) { if (pde & PG_PS) { - if ((pde & PG_RW) || (prot & VM_PROT_WRITE) == 0) { - if (vm_page_pa_tryrelock(pmap, (pde & - PG_PS_FRAME) | (va & PDRMASK), &pa)) - goto retry; - m = PHYS_TO_VM_PAGE(pa); - } + if ((pde & PG_RW) != 0 || (prot & VM_PROT_WRITE) == 0) + m = PHYS_TO_VM_PAGE((pde & PG_PS_FRAME) | + (va & PDRMASK)); } else { pte = *pmap_pde_to_pte(pdep, va); - if ((pte & PG_V) && - ((pte & PG_RW) || (prot & VM_PROT_WRITE) == 0)) { - if (vm_page_pa_tryrelock(pmap, pte & PG_FRAME, - &pa)) - goto retry; - m = PHYS_TO_VM_PAGE(pa); - } + if ((pte & PG_V) != 0 && + ((pte & PG_RW) != 0 || (prot & VM_PROT_WRITE) == 0)) + m = PHYS_TO_VM_PAGE(pte & PG_FRAME); } - if (m != NULL) - vm_page_wire(m); + if (m != NULL && !vm_page_wire_mapped(m)) + m = NULL; } - PA_UNLOCK_COND(pa); PMAP_UNLOCK(pmap); return (m); } Index: src/sys/amd64/sgx/sgx.c =================================================================== --- src/sys/amd64/sgx/sgx.c (revision 351575) +++ src/sys/amd64/sgx/sgx.c (working copy) @@ -357,9 +357,7 @@ sgx_page_remove(struct sgx_softc *sc, vm_page_t p) vm_paddr_t pa; uint64_t offs; - vm_page_lock(p); (void)vm_page_remove(p); - vm_page_unlock(p); dprintf("%s: p->pidx %ld\n", __func__, p->pindex); Index: src/sys/amd64/vmm/vmm.c =================================================================== --- src/sys/amd64/vmm/vmm.c (revision 351575) +++ src/sys/amd64/vmm/vmm.c (working copy) @@ -1002,9 +1002,7 @@ vm_gpa_release(void *cookie) { vm_page_t m = cookie; - vm_page_lock(m); vm_page_unwire(m, PQ_ACTIVE); - vm_page_unlock(m); } int Index: src/sys/arm/arm/pmap-v4.c =================================================================== --- src/sys/arm/arm/pmap-v4.c (revision 351575) +++ src/sys/arm/arm/pmap-v4.c (working copy) @@ -3415,14 +3415,14 @@ pmap_extract_and_hold(pmap_t pmap, vm_offset_t va, struct l2_dtable *l2; pd_entry_t l1pd; pt_entry_t *ptep, pte; - vm_paddr_t pa, paddr; - vm_page_t m = NULL; + vm_paddr_t pa; + vm_page_t m; u_int l1idx; + l1idx = L1_IDX(va); - paddr = 0; + m = NULL; PMAP_LOCK(pmap); -retry: l1pd = pmap->pm_l1->l1_kva[l1idx]; if (l1pte_section_p(l1pd)) { /* @@ -3434,11 +3434,10 @@ pmap_extract_and_hold(pmap_t pmap, vm_offset_t va, pa = (l1pd & L1_SUP_FRAME) | (va & L1_SUP_OFFSET); else pa = (l1pd & L1_S_FRAME) | (va & L1_S_OFFSET); - if (vm_page_pa_tryrelock(pmap, pa & PG_FRAME, &paddr)) - goto retry; if (l1pd & L1_S_PROT_W || (prot & VM_PROT_WRITE) == 0) { m = PHYS_TO_VM_PAGE(pa); - vm_page_wire(m); + if (!vm_page_wire_mapped(m)) + m = NULL; } } else { /* @@ -3466,15 +3465,12 @@ pmap_extract_and_hold(pmap_t pmap, vm_offset_t va, pa = (pte & L2_L_FRAME) | (va & L2_L_OFFSET); else pa = (pte & L2_S_FRAME) | (va & L2_S_OFFSET); - if (vm_page_pa_tryrelock(pmap, pa & PG_FRAME, &paddr)) - goto retry; m = PHYS_TO_VM_PAGE(pa); - vm_page_wire(m); + if (!vm_page_wire_mapped(m)) + m = NULL; } } - PMAP_UNLOCK(pmap); - PA_UNLOCK_COND(paddr); return (m); } Index: src/sys/arm/arm/pmap-v6.c =================================================================== --- src/sys/arm/arm/pmap-v6.c (revision 351575) +++ src/sys/arm/arm/pmap-v6.c (working copy) @@ -1986,23 +1986,20 @@ pmap_extract(pmap_t pmap, vm_offset_t va) vm_page_t pmap_extract_and_hold(pmap_t pmap, vm_offset_t va, vm_prot_t prot) { - vm_paddr_t pa, lockpa; + vm_paddr_t pa; pt1_entry_t pte1; pt2_entry_t pte2, *pte2p; vm_page_t m; - lockpa = 0; m = NULL; PMAP_LOCK(pmap); -retry: pte1 = pte1_load(pmap_pte1(pmap, va)); if (pte1_is_section(pte1)) { if (!(pte1 & PTE1_RO) || !(prot & VM_PROT_WRITE)) { pa = pte1_pa(pte1) | (va & PTE1_OFFSET); - if (vm_page_pa_tryrelock(pmap, pa, &lockpa)) - goto retry; m = PHYS_TO_VM_PAGE(pa); - vm_page_wire(m); + if (!vm_page_wire_mapped(m)) + m = NULL; } } else if (pte1_is_link(pte1)) { pte2p = pmap_pte2(pmap, va); @@ -2011,13 +2008,11 @@ pmap_extract_and_hold(pmap_t pmap, vm_offset_t va, if (pte2_is_valid(pte2) && (!(pte2 & PTE2_RO) || !(prot & VM_PROT_WRITE))) { pa = pte2_pa(pte2); - if (vm_page_pa_tryrelock(pmap, pa, &lockpa)) - goto retry; m = PHYS_TO_VM_PAGE(pa); - vm_page_wire(m); + if (!vm_page_wire_mapped(m)) + m = NULL; } } - PA_UNLOCK_COND(lockpa); PMAP_UNLOCK(pmap); return (m); } Index: src/sys/arm64/arm64/efirt_machdep.c =================================================================== --- src/sys/arm64/arm64/efirt_machdep.c (revision 351575) +++ src/sys/arm64/arm64/efirt_machdep.c (working copy) @@ -74,7 +74,7 @@ efi_destroy_1t1_map(void) if (obj_1t1_pt != NULL) { VM_OBJECT_RLOCK(obj_1t1_pt); TAILQ_FOREACH(m, &obj_1t1_pt->memq, listq) - m->wire_count = 0; + m->wire_count = VPRC_OBJREF; vm_wire_sub(obj_1t1_pt->resident_page_count); VM_OBJECT_RUNLOCK(obj_1t1_pt); vm_object_deallocate(obj_1t1_pt); Index: src/sys/arm64/arm64/pmap.c =================================================================== --- src/sys/arm64/arm64/pmap.c (revision 351575) +++ src/sys/arm64/arm64/pmap.c (working copy) @@ -1079,14 +1079,11 @@ pmap_extract_and_hold(pmap_t pmap, vm_offset_t va, { pt_entry_t *pte, tpte; vm_offset_t off; - vm_paddr_t pa; vm_page_t m; int lvl; - pa = 0; m = NULL; PMAP_LOCK(pmap); -retry: pte = pmap_pte(pmap, va, &lvl); if (pte != NULL) { tpte = pmap_load(pte); @@ -1111,14 +1108,11 @@ pmap_extract_and_hold(pmap_t pmap, vm_offset_t va, default: off = 0; } - if (vm_page_pa_tryrelock(pmap, - (tpte & ~ATTR_MASK) | off, &pa)) - goto retry; m = PHYS_TO_VM_PAGE((tpte & ~ATTR_MASK) | off); - vm_page_wire(m); + if (!vm_page_wire_mapped(m)) + m = NULL; } } - PA_UNLOCK_COND(pa); PMAP_UNLOCK(pmap); return (m); } Index: src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c =================================================================== --- src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c (revision 351575) +++ src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c (working copy) @@ -481,9 +481,7 @@ page_wire(vnode_t *vp, int64_t start) } ASSERT3U(pp->valid, ==, VM_PAGE_BITS_ALL); - vm_page_lock(pp); vm_page_wire(pp); - vm_page_unlock(pp); } else pp = NULL; break; @@ -495,9 +493,7 @@ static void page_unwire(vm_page_t pp) { - vm_page_lock(pp); vm_page_unwire(pp, PQ_ACTIVE); - vm_page_unlock(pp); } /* @@ -591,16 +587,16 @@ mappedread_sf(vnode_t *vp, int nbytes, uio_t *uio) zfs_unmap_page(sf); zfs_vmobject_wlock(obj); vm_page_sunbusy(pp); - vm_page_lock(pp); if (error) { - if (!vm_page_wired(pp) && pp->valid == 0 && - !vm_page_busied(pp)) + if (!vm_page_busied(pp) && !vm_page_wired(pp) && + pp->valid == 0) vm_page_free(pp); } else { pp->valid = VM_PAGE_BITS_ALL; + vm_page_lock(pp); vm_page_activate(pp); + vm_page_unlock(pp); } - vm_page_unlock(pp); } else { ASSERT3U(pp->valid, ==, VM_PAGE_BITS_ALL); vm_page_sunbusy(pp); Index: src/sys/compat/linuxkpi/common/include/linux/mm.h =================================================================== --- src/sys/compat/linuxkpi/common/include/linux/mm.h (revision 351575) +++ src/sys/compat/linuxkpi/common/include/linux/mm.h (working copy) @@ -227,9 +227,7 @@ mark_page_accessed(struct vm_page *page) static inline void get_page(struct vm_page *page) { - vm_page_lock(page); vm_page_wire(page); - vm_page_unlock(page); } extern long @@ -250,10 +248,7 @@ get_user_pages_remote(struct task_struct *, struct static inline void put_page(struct vm_page *page) { - vm_page_lock(page); - if (vm_page_unwire(page, PQ_ACTIVE) && page->object == NULL) - vm_page_free(page); - vm_page_unlock(page); + vm_page_unwire(page, PQ_ACTIVE); } #define copy_highpage(to, from) pmap_copy_page(from, to) Index: src/sys/compat/linuxkpi/common/src/linux_compat.c =================================================================== --- src/sys/compat/linuxkpi/common/src/linux_compat.c (revision 351575) +++ src/sys/compat/linuxkpi/common/src/linux_compat.c (working copy) @@ -511,9 +511,7 @@ linux_cdev_pager_fault(vm_object_t vm_obj, vm_ooff vm_page_replace_checked(page, vm_obj, (*mres)->pindex, *mres); - vm_page_lock(*mres); vm_page_free(*mres); - vm_page_unlock(*mres); *mres = page; } page->valid = VM_PAGE_BITS_ALL; Index: src/sys/compat/linuxkpi/common/src/linux_page.c =================================================================== --- src/sys/compat/linuxkpi/common/src/linux_page.c (revision 351575) +++ src/sys/compat/linuxkpi/common/src/linux_page.c (working copy) @@ -154,10 +154,8 @@ linux_free_pages(vm_page_t page, unsigned int orde for (x = 0; x != npages; x++) { vm_page_t pgo = page + x; - vm_page_lock(pgo); if (vm_page_unwire_noq(pgo)) vm_page_free(pgo); - vm_page_unlock(pgo); } } else { vm_offset_t vaddr; @@ -295,10 +293,8 @@ linux_shmem_read_mapping_page_gfp(vm_object_t obj, if (vm_pager_has_page(obj, pindex, NULL, NULL)) { rv = vm_pager_get_pages(obj, &page, 1, NULL, NULL); if (rv != VM_PAGER_OK) { - vm_page_lock(page); vm_page_unwire_noq(page); vm_page_free(page); - vm_page_unlock(page); VM_OBJECT_WUNLOCK(obj); return (ERR_PTR(-EINVAL)); } Index: src/sys/contrib/vchiq/interface/vchiq_arm/vchiq_2835_arm.c =================================================================== --- src/sys/contrib/vchiq/interface/vchiq_arm/vchiq_2835_arm.c (revision 351575) +++ src/sys/contrib/vchiq/interface/vchiq_arm/vchiq_2835_arm.c (working copy) @@ -377,10 +377,7 @@ vchiq_platform_handle_timeout(VCHIQ_STATE_T *state static void pagelist_page_free(vm_page_t pp) { - vm_page_lock(pp); - if (vm_page_unwire(pp, PQ_INACTIVE) && pp->object == NULL) - vm_page_free(pp); - vm_page_unlock(pp); + vm_page_unwire(pp, PQ_INACTIVE); } /* There is a potential problem with partial cache lines (pages?) Index: src/sys/dev/agp/agp.c =================================================================== --- src/sys/dev/agp/agp.c (revision 351575) +++ src/sys/dev/agp/agp.c (working copy) @@ -616,9 +616,7 @@ bad: m = vm_page_lookup(mem->am_obj, OFF_TO_IDX(k)); if (k >= i) vm_page_xunbusy(m); - vm_page_lock(m); vm_page_unwire(m, PQ_INACTIVE); - vm_page_unlock(m); } VM_OBJECT_WUNLOCK(mem->am_obj); @@ -653,9 +651,7 @@ agp_generic_unbind_memory(device_t dev, struct agp VM_OBJECT_WLOCK(mem->am_obj); for (i = 0; i < mem->am_size; i += PAGE_SIZE) { m = vm_page_lookup(mem->am_obj, atop(i)); - vm_page_lock(m); vm_page_unwire(m, PQ_INACTIVE); - vm_page_unlock(m); } VM_OBJECT_WUNLOCK(mem->am_obj); @@ -1003,7 +999,7 @@ agp_bind_pages(device_t dev, vm_page_t *pages, vm_ mtx_lock(&sc->as_lock); for (i = 0; i < size; i += PAGE_SIZE) { m = pages[OFF_TO_IDX(i)]; - KASSERT(m->wire_count > 0, + KASSERT(vm_page_wired(m), ("agp_bind_pages: page %p hasn't been wired", m)); /* Index: src/sys/dev/agp/agp_i810.c =================================================================== --- src/sys/dev/agp/agp_i810.c (revision 351575) +++ src/sys/dev/agp/agp_i810.c (working copy) @@ -1795,9 +1795,7 @@ agp_i810_free_memory(device_t dev, struct agp_memo */ VM_OBJECT_WLOCK(mem->am_obj); m = vm_page_lookup(mem->am_obj, 0); - vm_page_lock(m); vm_page_unwire(m, PQ_INACTIVE); - vm_page_unlock(m); VM_OBJECT_WUNLOCK(mem->am_obj); } else { contigfree(sc->argb_cursor, mem->am_size, M_AGP); Index: src/sys/dev/cxgbe/tom/t4_cpl_io.c =================================================================== --- src/sys/dev/cxgbe/tom/t4_cpl_io.c (revision 351575) +++ src/sys/dev/cxgbe/tom/t4_cpl_io.c (working copy) @@ -1910,7 +1910,6 @@ aiotx_free_pgs(struct mbuf *m) { struct mbuf_ext_pgs *ext_pgs; struct kaiocb *job; - struct mtx *mtx; vm_page_t pg; MBUF_EXT_PGS_ASSERT(m); @@ -1921,14 +1920,10 @@ aiotx_free_pgs(struct mbuf *m) m->m_len, jobtotid(job)); #endif - mtx = NULL; for (int i = 0; i < ext_pgs->npgs; i++) { pg = PHYS_TO_VM_PAGE(ext_pgs->pa[i]); - vm_page_change_lock(pg, &mtx); vm_page_unwire(pg, PQ_ACTIVE); } - if (mtx != NULL) - mtx_unlock(mtx); aiotx_free_job(job); } Index: src/sys/dev/cxgbe/tom/t4_ddp.c =================================================================== --- src/sys/dev/cxgbe/tom/t4_ddp.c (revision 351575) +++ src/sys/dev/cxgbe/tom/t4_ddp.c (working copy) @@ -114,9 +114,7 @@ free_pageset(struct tom_data *td, struct pageset * for (i = 0; i < ps->npages; i++) { p = ps->pages[i]; - vm_page_lock(p); vm_page_unwire(p, PQ_INACTIVE); - vm_page_unlock(p); } mtx_lock(&ddp_orphan_pagesets_lock); TAILQ_INSERT_TAIL(&ddp_orphan_pagesets, ps, link); Index: src/sys/dev/drm2/ttm/ttm_page_alloc.c =================================================================== --- src/sys/dev/drm2/ttm/ttm_page_alloc.c (revision 351575) +++ src/sys/dev/drm2/ttm/ttm_page_alloc.c (working copy) @@ -132,7 +132,7 @@ ttm_vm_page_free(vm_page_t m) { KASSERT(m->object == NULL, ("ttm page %p is owned", m)); - KASSERT(m->wire_count == 1, ("ttm lost wire %p", m)); + KASSERT(vm_page_wired(m), ("ttm lost wire %p", m)); KASSERT((m->flags & PG_FICTITIOUS) != 0, ("ttm lost fictitious %p", m)); KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("ttm got unmanaged %p", m)); m->flags &= ~PG_FICTITIOUS; Index: src/sys/dev/md/md.c =================================================================== --- src/sys/dev/md/md.c (revision 351575) +++ src/sys/dev/md/md.c (working copy) @@ -1029,9 +1029,7 @@ md_swap_page_free(vm_page_t m) { vm_page_xunbusy(m); - vm_page_lock(m); vm_page_free(m); - vm_page_unlock(m); } static int Index: src/sys/dev/xen/gntdev/gntdev.c =================================================================== --- src/sys/dev/xen/gntdev/gntdev.c (revision 351575) +++ src/sys/dev/xen/gntdev/gntdev.c (working copy) @@ -826,14 +826,12 @@ gntdev_gmap_pg_fault(vm_object_t object, vm_ooffse KASSERT((page->flags & PG_FICTITIOUS) != 0, ("not fictitious %p", page)); - KASSERT(page->wire_count == 1, ("wire_count not 1 %p", page)); - KASSERT(vm_page_busied(page) == 0, ("page %p is busy", page)); + KASSERT(vm_page_wired(page), ("page %p is not wired", page)); + KASSERT(!vm_page_busied(page), ("page %p is busy", page)); if (*mres != NULL) { oldm = *mres; - vm_page_lock(oldm); vm_page_free(oldm); - vm_page_unlock(oldm); *mres = NULL; } Index: src/sys/dev/xen/privcmd/privcmd.c =================================================================== --- src/sys/dev/xen/privcmd/privcmd.c (revision 351575) +++ src/sys/dev/xen/privcmd/privcmd.c (working copy) @@ -169,14 +169,12 @@ privcmd_pg_fault(vm_object_t object, vm_ooffset_t KASSERT((page->flags & PG_FICTITIOUS) != 0, ("not fictitious %p", page)); - KASSERT(page->wire_count == 1, ("wire_count not 1 %p", page)); - KASSERT(vm_page_busied(page) == 0, ("page %p is busy", page)); + KASSERT(vm_page_wired(page), ("page %p not wired", page)); + KASSERT(!vm_page_busied(page), ("page %p is busy", page)); if (*mres != NULL) { oldm = *mres; - vm_page_lock(oldm); vm_page_free(oldm); - vm_page_unlock(oldm); *mres = NULL; } Index: src/sys/i386/i386/pmap.c =================================================================== --- src/sys/i386/i386/pmap.c (revision 351575) +++ src/sys/i386/i386/pmap.c (working copy) @@ -1690,35 +1690,24 @@ __CONCAT(PMTYPE, extract_and_hold)(pmap_t pmap, vm pd_entry_t pde; pt_entry_t pte; vm_page_t m; - vm_paddr_t pa; - pa = 0; m = NULL; PMAP_LOCK(pmap); -retry: pde = *pmap_pde(pmap, va); if (pde != 0) { if (pde & PG_PS) { - if ((pde & PG_RW) || (prot & VM_PROT_WRITE) == 0) { - if (vm_page_pa_tryrelock(pmap, (pde & - PG_PS_FRAME) | (va & PDRMASK), &pa)) - goto retry; - m = PHYS_TO_VM_PAGE(pa); - } + if ((pde & PG_RW) || (prot & VM_PROT_WRITE) == 0) + m = PHYS_TO_VM_PAGE((pde & PG_PS_FRAME) | + (va & PDRMASK)); } else { pte = pmap_pte_ufast(pmap, va, pde); if (pte != 0 && - ((pte & PG_RW) || (prot & VM_PROT_WRITE) == 0)) { - if (vm_page_pa_tryrelock(pmap, pte & PG_FRAME, - &pa)) - goto retry; - m = PHYS_TO_VM_PAGE(pa); - } + ((pte & PG_RW) || (prot & VM_PROT_WRITE) == 0)) + m = PHYS_TO_VM_PAGE(pte & PG_FRAME); } - if (m != NULL) - vm_page_wire(m); + if (m != NULL && !vm_page_wire_mapped(m)) + m = NULL; } - PA_UNLOCK_COND(pa); PMAP_UNLOCK(pmap); return (m); } Index: src/sys/kern/kern_cpuset.c =================================================================== --- src/sys/kern/kern_cpuset.c (revision 351575) +++ src/sys/kern/kern_cpuset.c (working copy) @@ -2168,9 +2168,11 @@ kern_cpuset_setdomain(struct thread *td, cpulevel_ DOMAINSET_FILL(&domain.ds_mask); } + DOMAINSET_AND(&domain.ds_mask, &all_domains); + /* - * When given an impossible policy, fall back to interleaving - * across all domains + * When given an impossible policy, fall back to interleaving + * across all domains */ if (domainset_empty_vm(&domain)) domainset_copy(&domainset2, &domain); Index: src/sys/kern/kern_exec.c =================================================================== --- src/sys/kern/kern_exec.c (revision 351575) +++ src/sys/kern/kern_exec.c (working copy) @@ -977,10 +977,8 @@ exec_map_first_page(struct image_params *imgp) if (ma[0]->valid != VM_PAGE_BITS_ALL) { vm_page_xbusy(ma[0]); if (!vm_pager_has_page(object, 0, NULL, &after)) { - vm_page_lock(ma[0]); vm_page_unwire_noq(ma[0]); vm_page_free(ma[0]); - vm_page_unlock(ma[0]); VM_OBJECT_WUNLOCK(object); return (EIO); } @@ -1004,13 +1002,9 @@ exec_map_first_page(struct image_params *imgp) initial_pagein = i; rv = vm_pager_get_pages(object, ma, initial_pagein, NULL, NULL); if (rv != VM_PAGER_OK) { - for (i = 0; i < initial_pagein; i++) { - vm_page_lock(ma[i]); - if (i == 0) - vm_page_unwire_noq(ma[i]); + vm_page_unwire_noq(ma[0]); + for (i = 0; i < initial_pagein; i++) vm_page_free(ma[i]); - vm_page_unlock(ma[i]); - } VM_OBJECT_WUNLOCK(object); return (EIO); } @@ -1035,9 +1029,7 @@ exec_unmap_first_page(struct image_params *imgp) m = sf_buf_page(imgp->firstpage); sf_buf_free(imgp->firstpage); imgp->firstpage = NULL; - vm_page_lock(m); vm_page_unwire(m, PQ_ACTIVE); - vm_page_unlock(m); } } Index: src/sys/kern/kern_sendfile.c =================================================================== --- src/sys/kern/kern_sendfile.c (revision 351575) +++ src/sys/kern/kern_sendfile.c (working copy) @@ -415,11 +415,8 @@ sendfile_swapin(vm_object_t obj, struct sf_io *sfi &sendfile_iodone, sfio); if (rv != VM_PAGER_OK) { for (j = i; j < i + count; j++) { - if (pa[j] != bogus_page) { - vm_page_lock(pa[j]); + if (pa[j] != bogus_page) vm_page_unwire(pa[j], PQ_INACTIVE); - vm_page_unlock(pa[j]); - } } VM_OBJECT_WUNLOCK(obj); return (EIO); Index: src/sys/kern/sys_process.c =================================================================== --- src/sys/kern/sys_process.c (revision 351575) +++ src/sys/kern/sys_process.c (working copy) @@ -312,10 +312,7 @@ proc_rwmem(struct proc *p, struct uio *uio) /* * Release the page. */ - vm_page_lock(m); - if (vm_page_unwire(m, PQ_ACTIVE) && m->object == NULL) - vm_page_free(m); - vm_page_unlock(m); + vm_page_unwire(m, PQ_ACTIVE); } while (error == 0 && uio->uio_resid > 0); Index: src/sys/kern/uipc_mbuf.c =================================================================== --- src/sys/kern/uipc_mbuf.c (revision 351575) +++ src/sys/kern/uipc_mbuf.c (working copy) @@ -1621,10 +1621,6 @@ mb_free_mext_pgs(struct mbuf *m) ext_pgs = m->m_ext.ext_pgs; for (int i = 0; i < ext_pgs->npgs; i++) { pg = PHYS_TO_VM_PAGE(ext_pgs->pa[i]); - /* - * Note: page is not locked, as it has no - * object and is not on any queues. - */ vm_page_unwire_noq(pg); vm_page_free(pg); } Index: src/sys/kern/uipc_shm.c =================================================================== --- src/sys/kern/uipc_shm.c (revision 351575) +++ src/sys/kern/uipc_shm.c (working copy) @@ -198,10 +198,8 @@ uiomove_object_page(vm_object_t obj, size_t len, s printf( "uiomove_object: vm_obj %p idx %jd valid %x pager error %d\n", obj, idx, m->valid, rv); - vm_page_lock(m); vm_page_unwire_noq(m); vm_page_free(m); - vm_page_unlock(m); VM_OBJECT_WUNLOCK(obj); return (EIO); } @@ -217,9 +215,7 @@ uiomove_object_page(vm_object_t obj, size_t len, s vm_pager_page_unswapped(m); VM_OBJECT_WUNLOCK(obj); } - vm_page_lock(m); vm_page_unwire(m, PQ_ACTIVE); - vm_page_unlock(m); return (error); } @@ -474,7 +470,6 @@ retry: goto retry; rv = vm_pager_get_pages(object, &m, 1, NULL, NULL); - vm_page_lock(m); if (rv == VM_PAGER_OK) { /* * Since the page was not resident, @@ -485,11 +480,9 @@ retry: * as an access. */ vm_page_launder(m); - vm_page_unlock(m); vm_page_xunbusy(m); } else { vm_page_free(m); - vm_page_unlock(m); VM_OBJECT_WUNLOCK(object); return (EIO); } Index: src/sys/mips/mips/pmap.c =================================================================== --- src/sys/mips/mips/pmap.c (revision 351575) +++ src/sys/mips/mips/pmap.c (working copy) @@ -796,26 +796,22 @@ vm_page_t pmap_extract_and_hold(pmap_t pmap, vm_offset_t va, vm_prot_t prot) { pt_entry_t pte, *ptep; - vm_paddr_t pa, pte_pa; + vm_paddr_t pa; vm_page_t m; m = NULL; - pa = 0; PMAP_LOCK(pmap); -retry: ptep = pmap_pte(pmap, va); if (ptep != NULL) { pte = *ptep; if (pte_test(&pte, PTE_V) && (!pte_test(&pte, PTE_RO) || (prot & VM_PROT_WRITE) == 0)) { - pte_pa = TLBLO_PTE_TO_PA(pte); - if (vm_page_pa_tryrelock(pmap, pte_pa, &pa)) - goto retry; - m = PHYS_TO_VM_PAGE(pte_pa); - vm_page_wire(m); + pa = TLBLO_PTE_TO_PA(pte); + m = PHYS_TO_VM_PAGE(pa); + if (!vm_page_wire_mapped(m)) + m = NULL; } } - PA_UNLOCK_COND(pa); PMAP_UNLOCK(pmap); return (m); } Index: src/sys/net/bpf_zerocopy.c =================================================================== --- src/sys/net/bpf_zerocopy.c (revision 351575) +++ src/sys/net/bpf_zerocopy.c (working copy) @@ -115,10 +115,7 @@ static void zbuf_page_free(vm_page_t pp) { - vm_page_lock(pp); - if (vm_page_unwire(pp, PQ_INACTIVE) && pp->object == NULL) - vm_page_free(pp); - vm_page_unlock(pp); + vm_page_unwire(pp, PQ_INACTIVE); } /* Index: src/sys/powerpc/aim/mmu_oea.c =================================================================== --- src/sys/powerpc/aim/mmu_oea.c (revision 351575) +++ src/sys/powerpc/aim/mmu_oea.c (working copy) @@ -1264,22 +1264,17 @@ moea_extract_and_hold(mmu_t mmu, pmap_t pmap, vm_o { struct pvo_entry *pvo; vm_page_t m; - vm_paddr_t pa; m = NULL; - pa = 0; PMAP_LOCK(pmap); -retry: pvo = moea_pvo_find_va(pmap, va & ~ADDR_POFF, NULL); if (pvo != NULL && (pvo->pvo_pte.pte.pte_hi & PTE_VALID) && ((pvo->pvo_pte.pte.pte_lo & PTE_PP) == PTE_RW || (prot & VM_PROT_WRITE) == 0)) { - if (vm_page_pa_tryrelock(pmap, pvo->pvo_pte.pte.pte_lo & PTE_RPGN, &pa)) - goto retry; m = PHYS_TO_VM_PAGE(pvo->pvo_pte.pte.pte_lo & PTE_RPGN); - vm_page_wire(m); + if (!vm_page_wire_mapped(m)) + m = NULL; } - PA_UNLOCK_COND(pa); PMAP_UNLOCK(pmap); return (m); } Index: src/sys/powerpc/aim/mmu_oea64.c =================================================================== --- src/sys/powerpc/aim/mmu_oea64.c (revision 351575) +++ src/sys/powerpc/aim/mmu_oea64.c (working copy) @@ -1584,21 +1584,15 @@ moea64_extract_and_hold(mmu_t mmu, pmap_t pmap, vm { struct pvo_entry *pvo; vm_page_t m; - vm_paddr_t pa; m = NULL; - pa = 0; PMAP_LOCK(pmap); -retry: pvo = moea64_pvo_find_va(pmap, va & ~ADDR_POFF); if (pvo != NULL && (pvo->pvo_pte.prot & prot) == prot) { - if (vm_page_pa_tryrelock(pmap, - pvo->pvo_pte.pa & LPTE_RPGN, &pa)) - goto retry; m = PHYS_TO_VM_PAGE(pvo->pvo_pte.pa & LPTE_RPGN); - vm_page_wire(m); + if (!vm_page_wire_mapped(m)) + m = NULL; } - PA_UNLOCK_COND(pa); PMAP_UNLOCK(pmap); return (m); } Index: src/sys/powerpc/booke/pmap.c =================================================================== --- src/sys/powerpc/booke/pmap.c (revision 351575) +++ src/sys/powerpc/booke/pmap.c (working copy) @@ -2790,12 +2790,9 @@ mmu_booke_extract_and_hold(mmu_t mmu, pmap_t pmap, pte_t *pte; vm_page_t m; uint32_t pte_wbit; - vm_paddr_t pa; - + m = NULL; - pa = 0; PMAP_LOCK(pmap); -retry: pte = pte_find(mmu, pmap, va); if ((pte != NULL) && PTE_ISVALID(pte)) { if (pmap == kernel_pmap) @@ -2803,15 +2800,12 @@ mmu_booke_extract_and_hold(mmu_t mmu, pmap_t pmap, else pte_wbit = PTE_UW; - if ((*pte & pte_wbit) || ((prot & VM_PROT_WRITE) == 0)) { - if (vm_page_pa_tryrelock(pmap, PTE_PA(pte), &pa)) - goto retry; + if ((*pte & pte_wbit) != 0 || (prot & VM_PROT_WRITE) == 0) { m = PHYS_TO_VM_PAGE(PTE_PA(pte)); - vm_page_wire(m); + if (!vm_page_wire_mapped(m)) + m = NULL; } } - - PA_UNLOCK_COND(pa); PMAP_UNLOCK(pmap); return (m); } Index: src/sys/riscv/riscv/pmap.c =================================================================== --- src/sys/riscv/riscv/pmap.c (revision 351575) +++ src/sys/riscv/riscv/pmap.c (working copy) @@ -870,24 +870,19 @@ pmap_extract_and_hold(pmap_t pmap, vm_offset_t va, { pt_entry_t *l3p, l3; vm_paddr_t phys; - vm_paddr_t pa; vm_page_t m; - pa = 0; m = NULL; PMAP_LOCK(pmap); -retry: l3p = pmap_l3(pmap, va); if (l3p != NULL && (l3 = pmap_load(l3p)) != 0) { if ((l3 & PTE_W) != 0 || (prot & VM_PROT_WRITE) == 0) { phys = PTE_TO_PHYS(l3); - if (vm_page_pa_tryrelock(pmap, phys, &pa)) - goto retry; m = PHYS_TO_VM_PAGE(phys); - vm_page_wire(m); + if (!vm_page_wire_mapped(m)) + m = NULL; } } - PA_UNLOCK_COND(pa); PMAP_UNLOCK(pmap); return (m); } Index: src/sys/sparc64/sparc64/pmap.c =================================================================== --- src/sys/sparc64/sparc64/pmap.c (revision 351575) +++ src/sys/sparc64/sparc64/pmap.c (working copy) @@ -846,19 +846,15 @@ pmap_extract_and_hold(pmap_t pm, vm_offset_t va, v { struct tte *tp; vm_page_t m; - vm_paddr_t pa; m = NULL; - pa = 0; PMAP_LOCK(pm); -retry: if (pm == kernel_pmap) { if (va >= VM_MIN_DIRECT_ADDRESS) { tp = NULL; m = PHYS_TO_VM_PAGE(TLB_DIRECT_TO_PHYS(va)); - (void)vm_page_pa_tryrelock(pm, TLB_DIRECT_TO_PHYS(va), - &pa); - vm_page_wire(m); + if (!vm_page_wire_mapped(m)) + m = NULL; } else { tp = tsb_kvtotte(va); if ((tp->tte_data & TD_V) == 0) @@ -868,12 +864,10 @@ pmap_extract_and_hold(pmap_t pm, vm_offset_t va, v tp = tsb_tte_lookup(pm, va); if (tp != NULL && ((tp->tte_data & TD_SW) || (prot & VM_PROT_WRITE) == 0)) { - if (vm_page_pa_tryrelock(pm, TTE_GET_PA(tp), &pa)) - goto retry; m = PHYS_TO_VM_PAGE(TTE_GET_PA(tp)); - vm_page_wire(m); + if (!vm_page_wire_mapped(m)) + m = NULL; } - PA_UNLOCK_COND(pa); PMAP_UNLOCK(pm); return (m); } Index: src/sys/vm/device_pager.c =================================================================== --- src/sys/vm/device_pager.c (revision 351575) +++ src/sys/vm/device_pager.c (working copy) @@ -235,9 +235,7 @@ cdev_pager_free_page(vm_object_t object, vm_page_t if (object->type == OBJT_MGTDEVICE) { KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("unmanaged %p", m)); pmap_remove_all(m); - vm_page_lock(m); (void)vm_page_remove(m); - vm_page_unlock(m); } else if (object->type == OBJT_DEVICE) dev_pager_free_page(object, m); } Index: src/sys/vm/vm_fault.c =================================================================== --- src/sys/vm/vm_fault.c (revision 351575) +++ src/sys/vm/vm_fault.c (working copy) @@ -186,9 +186,7 @@ unlock_and_deallocate(struct faultstate *fs) VM_OBJECT_WUNLOCK(fs->object); if (fs->object != fs->first_object) { VM_OBJECT_WLOCK(fs->first_object); - vm_page_lock(fs->first_m); vm_page_free(fs->first_m); - vm_page_unlock(fs->first_m); vm_object_pip_wakeup(fs->first_object); VM_OBJECT_WUNLOCK(fs->first_object); fs->first_m = NULL; @@ -263,18 +261,6 @@ vm_fault_dirty(vm_map_entry_t entry, vm_page_t m, vm_pager_page_unswapped(m); } -static void -vm_fault_fill_hold(vm_page_t *m_hold, vm_page_t m) -{ - - if (m_hold != NULL) { - *m_hold = m; - vm_page_lock(m); - vm_page_wire(m); - vm_page_unlock(m); - } -} - /* * Unlocks fs.first_object and fs.map on success. */ @@ -335,7 +321,10 @@ vm_fault_soft_fast(struct faultstate *fs, vm_offse PMAP_ENTER_NOSLEEP | (wired ? PMAP_ENTER_WIRED : 0), psind); if (rv != KERN_SUCCESS) return (rv); - vm_fault_fill_hold(m_hold, m); + if (m_hold != NULL) { + *m_hold = m; + vm_page_wire(m); + } vm_fault_dirty(fs->entry, m, prot, fault_type, fault_flags, false); if (psind == 0 && !wired) vm_fault_prefault(fs, vaddr, PFBAK, PFFOR, true); @@ -511,11 +500,12 @@ vm_fault_populate(struct faultstate *fs, vm_prot_t VM_OBJECT_WLOCK(fs->first_object); m_mtx = NULL; for (i = 0; i < npages; i++) { - vm_page_change_lock(&m[i], &m_mtx); - if ((fault_flags & VM_FAULT_WIRE) != 0) + if ((fault_flags & VM_FAULT_WIRE) != 0) { vm_page_wire(&m[i]); - else + } else { + vm_page_change_lock(&m[i], &m_mtx); vm_page_activate(&m[i]); + } if (m_hold != NULL && m[i].pindex == fs->first_pindex) { *m_hold = &m[i]; vm_page_wire(&m[i]); @@ -576,7 +566,6 @@ vm_fault_hold(vm_map_t map, vm_offset_t vaddr, vm_ struct faultstate fs; struct vnode *vp; struct domainset *dset; - struct mtx *mtx; vm_object_t next_object, retry_object; vm_offset_t e_end, e_start; vm_pindex_t retry_pindex; @@ -745,9 +734,7 @@ RetryFault_oom: VM_OBJECT_WLOCK(fs.first_object); VM_OBJECT_WLOCK(fs.object); } - vm_page_lock(fs.first_m); vm_page_free(fs.first_m); - vm_page_unlock(fs.first_m); vm_object_pip_wakeup(fs.first_object); VM_OBJECT_WUNLOCK(fs.first_object); fs.first_m = NULL; @@ -1030,12 +1017,10 @@ readrest: * an error. */ if (rv == VM_PAGER_ERROR || rv == VM_PAGER_BAD) { - vm_page_lock(fs.m); if (!vm_page_wired(fs.m)) vm_page_free(fs.m); else - vm_page_xunbusy_maybelocked(fs.m); - vm_page_unlock(fs.m); + vm_page_xunbusy(fs.m); fs.m = NULL; unlock_and_deallocate(&fs); return (rv == VM_PAGER_ERROR ? KERN_FAILURE : @@ -1053,12 +1038,10 @@ readrest: * that we are. */ if (fs.object != fs.first_object) { - vm_page_lock(fs.m); if (!vm_page_wired(fs.m)) vm_page_free(fs.m); else - vm_page_xunbusy_maybelocked(fs.m); - vm_page_unlock(fs.m); + vm_page_xunbusy(fs.m); fs.m = NULL; } } @@ -1169,23 +1152,11 @@ readrest: * We don't chase down the shadow chain */ fs.object == fs.first_object->backing_object) { - /* - * Keep the page wired to ensure that it is not - * freed by another thread, such as the page - * daemon, while it is disassociated from an - * object. - */ - mtx = NULL; - vm_page_change_lock(fs.m, &mtx); - vm_page_wire(fs.m); + (void)vm_page_remove(fs.m); - vm_page_change_lock(fs.first_m, &mtx); vm_page_replace_checked(fs.m, fs.first_object, fs.first_pindex, fs.first_m); vm_page_free(fs.first_m); - vm_page_change_lock(fs.m, &mtx); - vm_page_unwire(fs.m, PQ_ACTIVE); - mtx_unlock(mtx); vm_page_dirty(fs.m); #if VM_NRESERVLEVEL > 0 /* @@ -1211,13 +1182,8 @@ readrest: fs.first_m->valid = VM_PAGE_BITS_ALL; if (wired && (fault_flags & VM_FAULT_WIRE) == 0) { - vm_page_lock(fs.first_m); vm_page_wire(fs.first_m); - vm_page_unlock(fs.first_m); - - vm_page_lock(fs.m); vm_page_unwire(fs.m, PQ_INACTIVE); - vm_page_unlock(fs.m); } /* * We no longer need the old page or object. @@ -1350,21 +1316,22 @@ readrest: faultcount > 0 ? behind : PFBAK, faultcount > 0 ? ahead : PFFOR, false); VM_OBJECT_WLOCK(fs.object); - vm_page_lock(fs.m); /* * If the page is not wired down, then put it where the pageout daemon * can find it. */ - if ((fault_flags & VM_FAULT_WIRE) != 0) + if ((fault_flags & VM_FAULT_WIRE) != 0) { vm_page_wire(fs.m); - else + } else { + vm_page_lock(fs.m); vm_page_activate(fs.m); + vm_page_unlock(fs.m); + } if (m_hold != NULL) { *m_hold = fs.m; vm_page_wire(fs.m); } - vm_page_unlock(fs.m); vm_page_xunbusy(fs.m); /* @@ -1633,13 +1600,8 @@ vm_fault_quick_hold_pages(vm_map_t map, vm_offset_ return (count); error: for (mp = ma; mp < ma + count; mp++) - if (*mp != NULL) { - vm_page_lock(*mp); - if (vm_page_unwire(*mp, PQ_INACTIVE) && - (*mp)->object == NULL) - vm_page_free(*mp); - vm_page_unlock(*mp); - } + if (*mp != NULL) + vm_page_unwire(*mp, PQ_INACTIVE); return (-1); } @@ -1835,12 +1797,8 @@ again: if (upgrade) { if (src_m != dst_m) { - vm_page_lock(src_m); vm_page_unwire(src_m, PQ_INACTIVE); - vm_page_unlock(src_m); - vm_page_lock(dst_m); vm_page_wire(dst_m); - vm_page_unlock(dst_m); } else { KASSERT(vm_page_wired(dst_m), ("dst_m %p is not wired", dst_m)); Index: src/sys/vm/vm_glue.c =================================================================== --- src/sys/vm/vm_glue.c (revision 351575) +++ src/sys/vm/vm_glue.c (working copy) @@ -229,10 +229,8 @@ vm_imgact_hold_page(vm_object_t object, vm_ooffset vm_page_xbusy(m); rv = vm_pager_get_pages(object, &m, 1, NULL, NULL); if (rv != VM_PAGER_OK) { - vm_page_lock(m); vm_page_unwire_noq(m); vm_page_free(m); - vm_page_unlock(m); m = NULL; goto out; } @@ -270,9 +268,7 @@ vm_imgact_unmap_page(struct sf_buf *sf) m = sf_buf_page(sf); sf_buf_free(sf); sched_unpin(); - vm_page_lock(m); vm_page_unwire(m, PQ_ACTIVE); - vm_page_unlock(m); } void @@ -380,10 +376,8 @@ vm_thread_stack_dispose(vm_object_t ksobj, vm_offs m = vm_page_lookup(ksobj, i); if (m == NULL) panic("vm_thread_dispose: kstack already missing?"); - vm_page_lock(m); vm_page_unwire_noq(m); vm_page_free(m); - vm_page_unlock(m); } VM_OBJECT_WUNLOCK(ksobj); vm_object_deallocate(ksobj); Index: src/sys/vm/vm_object.c =================================================================== --- src/sys/vm/vm_object.c (revision 351575) +++ src/sys/vm/vm_object.c (working copy) @@ -694,12 +694,9 @@ static void vm_object_terminate_pages(vm_object_t object) { vm_page_t p, p_next; - struct mtx *mtx; VM_OBJECT_ASSERT_WLOCKED(object); - mtx = NULL; - /* * Free any remaining pageable pages. This also removes them from the * paging queues. However, don't free wired pages, just remove them @@ -708,20 +705,16 @@ vm_object_terminate_pages(vm_object_t object) */ TAILQ_FOREACH_SAFE(p, &object->memq, listq, p_next) { vm_page_assert_unbusied(p); - if ((object->flags & OBJ_UNMANAGED) == 0) - /* - * vm_page_free_prep() only needs the page - * lock for managed pages. - */ - vm_page_change_lock(p, &mtx); + KASSERT(p->object == object && + (p->ref_count & VPRC_OBJREF) != 0, + ("vm_object_terminate_pages: page %p is inconsistent", p)); + p->object = NULL; - if (vm_page_wired(p)) - continue; - VM_CNT_INC(v_pfree); - vm_page_free(p); + if (vm_page_drop(p, VPRC_OBJREF) == VPRC_OBJREF) { + VM_CNT_INC(v_pfree); + vm_page_free(p); + } } - if (mtx != NULL) - mtx_unlock(mtx); /* * If the object contained any pages, then reset it to an empty state. @@ -1178,13 +1171,9 @@ next_page: /* * If the page is not in a normal state, skip it. */ - if (tm->valid != VM_PAGE_BITS_ALL) + if (tm->valid != VM_PAGE_BITS_ALL || + vm_page_wired(tm)) goto next_pindex; - vm_page_lock(tm); - if (vm_page_wired(tm)) { - vm_page_unlock(tm); - goto next_pindex; - } KASSERT((tm->flags & PG_FICTITIOUS) == 0, ("vm_object_madvise: page %p is fictitious", tm)); KASSERT((tm->oflags & VPO_UNMANAGED) == 0, @@ -1192,6 +1181,7 @@ next_page: if (vm_page_busied(tm)) { if (object != tobject) VM_OBJECT_WUNLOCK(tobject); + vm_page_lock(tm); VM_OBJECT_WUNLOCK(object); if (advice == MADV_WILLNEED) { /* @@ -1204,6 +1194,7 @@ next_page: vm_page_busy_sleep(tm, "madvpo", false); goto relookup; } + vm_page_lock(tm); vm_page_advise(tm, advice); vm_page_unlock(tm); vm_object_madvise_freespace(tobject, advice, tm->pindex, 1); @@ -1557,16 +1548,10 @@ vm_object_collapse_scan(vm_object_t object, int op swap_pager_freespace(backing_object, p->pindex, 1); - /* - * Page is out of the parent object's range, we can - * simply destroy it. - */ - vm_page_lock(p); KASSERT(!pmap_page_is_mapped(p), ("freeing mapped page %p", p)); if (vm_page_remove(p)) vm_page_free(p); - vm_page_unlock(p); continue; } @@ -1603,12 +1588,10 @@ vm_object_collapse_scan(vm_object_t object, int op if (backing_object->type == OBJT_SWAP) swap_pager_freespace(backing_object, p->pindex, 1); - vm_page_lock(p); KASSERT(!pmap_page_is_mapped(p), ("freeing mapped page %p", p)); if (vm_page_remove(p)) vm_page_free(p); - vm_page_unlock(p); continue; } @@ -1909,7 +1892,14 @@ again: VM_OBJECT_WLOCK(object); goto again; } + if (vm_page_busied(p)) { + VM_OBJECT_WUNLOCK(object); + vm_page_busy_sleep(p, "vmopar", false); + VM_OBJECT_WLOCK(object); + goto again; + } if (vm_page_wired(p)) { +wired: if ((options & OBJPR_NOTMAPPED) == 0 && object->ref_count != 0) pmap_remove_all(p); @@ -1919,23 +1909,19 @@ again: } continue; } - if (vm_page_busied(p)) { - VM_OBJECT_WUNLOCK(object); - vm_page_busy_sleep(p, "vmopar", false); - VM_OBJECT_WLOCK(object); - goto again; - } KASSERT((p->flags & PG_FICTITIOUS) == 0, ("vm_object_page_remove: page %p is fictitious", p)); if ((options & OBJPR_CLEANONLY) != 0 && p->valid != 0) { if ((options & OBJPR_NOTMAPPED) == 0 && - object->ref_count != 0) - pmap_remove_write(p); + object->ref_count != 0 && + !vm_page_try_remove_write(p)) + goto wired; if (p->dirty != 0) continue; } - if ((options & OBJPR_NOTMAPPED) == 0 && object->ref_count != 0) - pmap_remove_all(p); + if ((options & OBJPR_NOTMAPPED) == 0 && + object->ref_count != 0 && !vm_page_try_remove_all(p)) + goto wired; vm_page_free(p); } if (mtx != NULL) @@ -2225,8 +2211,8 @@ again: tm = m; m = TAILQ_NEXT(m, listq); } - vm_page_lock(tm); if (vm_page_xbusied(tm)) { + vm_page_lock(tm); for (tobject = object; locked_depth >= 1; locked_depth--) { t1object = tobject->backing_object; @@ -2237,7 +2223,6 @@ again: goto again; } vm_page_unwire(tm, queue); - vm_page_unlock(tm); next_page: pindex++; } Index: src/sys/vm/vm_page.c =================================================================== --- src/sys/vm/vm_page.c (revision 351575) +++ src/sys/vm/vm_page.c (working copy) @@ -168,6 +168,7 @@ static int vm_page_insert_after(vm_page_t m, vm_ob vm_pindex_t pindex, vm_page_t mpred); static void vm_page_insert_radixdone(vm_page_t m, vm_object_t object, vm_page_t mpred); +static void vm_page_mvqueue(vm_page_t m, uint8_t queue); static int vm_page_reclaim_run(int req_class, int domain, u_long npages, vm_page_t m_run, vm_paddr_t high); static int vm_domain_alloc_fail(struct vm_domain *vmd, vm_object_t object, @@ -513,7 +514,7 @@ vm_page_init_page(vm_page_t m, vm_paddr_t pa, int { m->object = NULL; - m->wire_count = 0; + m->ref_count = 0; m->busy_lock = VPB_UNBUSIED; m->flags = m->aflags = 0; m->phys_addr = pa; @@ -1107,17 +1108,11 @@ vm_page_change_lock(vm_page_t m, struct mtx **mtx) void vm_page_unhold_pages(vm_page_t *ma, int count) { - struct mtx *mtx; - mtx = NULL; for (; count != 0; count--) { - vm_page_change_lock(*ma, &mtx); - if (vm_page_unwire(*ma, PQ_ACTIVE) && (*ma)->object == NULL) - vm_page_free(*ma); + vm_page_unwire(*ma, PQ_ACTIVE); ma++; } - if (mtx != NULL) - mtx_unlock(mtx); } vm_page_t @@ -1180,7 +1175,8 @@ vm_page_initfake(vm_page_t m, vm_paddr_t paddr, vm /* Fictitious pages don't use "order" or "pool". */ m->oflags = VPO_UNMANAGED; m->busy_lock = VPB_SINGLE_EXCLUSIVER; - m->wire_count = 1; + /* Fictitious pages are unevictable. */ + m->ref_count = 1; pmap_page_init(m); memattr: pmap_page_set_memattr(m, memattr); @@ -1375,10 +1371,11 @@ vm_page_insert_after(vm_page_t m, vm_object_t obje ("vm_page_insert_after: msucc doesn't succeed pindex")); /* - * Record the object/offset pair in this page + * Record the object/offset pair in this page. */ m->object = object; m->pindex = pindex; + m->ref_count |= VPRC_OBJREF; /* * Now link into the object's ordered list of backed pages. @@ -1386,6 +1383,7 @@ vm_page_insert_after(vm_page_t m, vm_object_t obje if (vm_radix_insert(&object->rtree, m)) { m->object = NULL; m->pindex = 0; + m->ref_count &= ~VPRC_OBJREF; return (1); } vm_page_insert_radixdone(m, object, mpred); @@ -1410,11 +1408,13 @@ vm_page_insert_radixdone(vm_page_t m, vm_object_t VM_OBJECT_ASSERT_WLOCKED(object); KASSERT(object != NULL && m->object == object, ("vm_page_insert_radixdone: page %p has inconsistent object", m)); + KASSERT((m->ref_count & VPRC_OBJREF) != 0, + ("vm_page_insert_radixdone: page %p is missing object ref", m)); if (mpred != NULL) { KASSERT(mpred->object == object, - ("vm_page_insert_after: object doesn't contain mpred")); + ("vm_page_insert_radixdone: object doesn't contain mpred")); KASSERT(mpred->pindex < m->pindex, - ("vm_page_insert_after: mpred doesn't precede pindex")); + ("vm_page_insert_radixdone: mpred doesn't precede pindex")); } if (mpred != NULL) @@ -1442,25 +1442,19 @@ vm_page_insert_radixdone(vm_page_t m, vm_object_t } /* - * vm_page_remove: - * - * Removes the specified page from its containing object, but does not - * invalidate any backing storage. Return true if the page may be safely - * freed and false otherwise. - * - * The object must be locked. The page must be locked if it is managed. + * Do the work to remove a page from its object. The caller is responsible for + * updating the page's fields to reflect this removal. */ -bool -vm_page_remove(vm_page_t m) +static void +vm_page_object_remove(vm_page_t m) { vm_object_t object; vm_page_t mrem; object = m->object; - - if ((m->oflags & VPO_UNMANAGED) == 0) - vm_page_assert_locked(m); VM_OBJECT_ASSERT_WLOCKED(object); + KASSERT((m->ref_count & VPRC_OBJREF) != 0, + ("page %p is missing its object ref", m)); if (vm_page_xbusied(m)) vm_page_xunbusy_maybelocked(m); mrem = vm_radix_remove(&object->rtree, m->pindex); @@ -1481,9 +1475,24 @@ vm_page_insert_radixdone(vm_page_t m, vm_object_t */ if (object->resident_page_count == 0 && object->type == OBJT_VNODE) vdrop(object->handle); +} +/* + * vm_page_remove: + * + * Removes the specified page from its containing object, but does not + * invalidate any backing storage. Returns true if the object's reference + * was the last reference to the page, and false otherwise. + * + * The object must be locked. + */ +bool +vm_page_remove(vm_page_t m) +{ + + vm_page_object_remove(m); m->object = NULL; - return (!vm_page_wired(m)); + return (vm_page_drop(m, VPRC_OBJREF) == VPRC_OBJREF); } /* @@ -1564,8 +1573,6 @@ vm_page_prev(vm_page_t m) /* * Uses the page mnew as a replacement for an existing page at index * pindex which must be already present in the object. - * - * The existing page must not be on a paging queue. */ vm_page_t vm_page_replace(vm_page_t mnew, vm_object_t object, vm_pindex_t pindex) @@ -1573,10 +1580,8 @@ vm_page_replace(vm_page_t mnew, vm_object_t object vm_page_t mold; VM_OBJECT_ASSERT_WLOCKED(object); - KASSERT(mnew->object == NULL, + KASSERT(mnew->object == NULL && (mnew->ref_count & VPRC_OBJREF) == 0, ("vm_page_replace: page %p already in object", mnew)); - KASSERT(mnew->queue == PQ_NONE || vm_page_wired(mnew), - ("vm_page_replace: new page %p is on a paging queue", mnew)); /* * This function mostly follows vm_page_insert() and @@ -1586,6 +1591,7 @@ vm_page_replace(vm_page_t mnew, vm_object_t object mnew->object = object; mnew->pindex = pindex; + atomic_set_int(&mnew->ref_count, VPRC_OBJREF); mold = vm_radix_replace(&object->rtree, mnew); KASSERT(mold->queue == PQ_NONE, ("vm_page_replace: old page %p is on a paging queue", mold)); @@ -1595,6 +1601,7 @@ vm_page_replace(vm_page_t mnew, vm_object_t object TAILQ_REMOVE(&object->memq, mold, listq); mold->object = NULL; + atomic_clear_int(&mold->ref_count, VPRC_OBJREF); vm_page_xunbusy_maybelocked(mold); /* @@ -1632,6 +1639,7 @@ vm_page_rename(vm_page_t m, vm_object_t new_object VM_OBJECT_ASSERT_WLOCKED(new_object); + KASSERT(m->ref_count != 0, ("vm_page_rename: page %p has no refs", m)); mpred = vm_radix_lookup_le(&new_object->rtree, new_pindex); KASSERT(mpred == NULL || mpred->pindex != new_pindex, ("vm_page_rename: pindex already renamed")); @@ -1653,13 +1661,12 @@ vm_page_rename(vm_page_t m, vm_object_t new_object * the listq iterator is tainted. */ m->pindex = opidx; - vm_page_lock(m); - (void)vm_page_remove(m); + vm_page_object_remove(m); /* Return back to the new pindex to complete vm_page_insert(). */ m->pindex = new_pindex; m->object = new_object; - vm_page_unlock(m); + vm_page_insert_radixdone(m, new_object, mpred); vm_page_dirty(m); return (0); @@ -1876,7 +1883,7 @@ found: * page is inserted into the object. */ vm_wire_add(1); - m->wire_count = 1; + m->ref_count = 1; } m->act_count = 0; @@ -1884,7 +1891,7 @@ found: if (vm_page_insert_after(m, object, pindex, mpred)) { if (req & VM_ALLOC_WIRED) { vm_wire_sub(1); - m->wire_count = 0; + m->ref_count = 0; } KASSERT(m->object == NULL, ("page %p has object", m)); m->oflags = VPO_UNMANAGED; @@ -2076,7 +2083,7 @@ found: m->flags = (m->flags | PG_NODUMP) & flags; m->busy_lock = busy_lock; if ((req & VM_ALLOC_WIRED) != 0) - m->wire_count = 1; + m->ref_count = 1; m->act_count = 0; m->oflags = oflags; if (object != NULL) { @@ -2089,7 +2096,7 @@ found: for (m = m_ret; m < &m_ret[npages]; m++) { if (m <= mpred && (req & VM_ALLOC_WIRED) != 0) - m->wire_count = 0; + m->ref_count = 0; m->oflags = VPO_UNMANAGED; m->busy_lock = VPB_UNBUSIED; /* Don't change PG_ZERO. */ @@ -2123,7 +2130,7 @@ vm_page_alloc_check(vm_page_t m) KASSERT(m->queue == PQ_NONE && (m->aflags & PGA_QUEUE_STATE_MASK) == 0, ("page %p has unexpected queue %d, flags %#x", m, m->queue, (m->aflags & PGA_QUEUE_STATE_MASK))); - KASSERT(!vm_page_wired(m), ("page %p is wired", m)); + KASSERT(m->ref_count == 0, ("page %p has references", m)); KASSERT(!vm_page_busied(m), ("page %p is busy", m)); KASSERT(m->dirty == 0, ("page %p is dirty", m)); KASSERT(pmap_page_get_memattr(m) == VM_MEMATTR_DEFAULT, @@ -2207,7 +2214,7 @@ again: * not belong to an object. */ vm_wire_add(1); - m->wire_count = 1; + m->ref_count = 1; } /* Unmanaged pages don't use "act_count". */ m->oflags = VPO_UNMANAGED; @@ -2300,8 +2307,8 @@ vm_page_scan_contig(u_long npages, vm_page_t m_sta for (m = m_start; m < m_end && run_len < npages; m += m_inc) { KASSERT((m->flags & PG_MARKER) == 0, ("page %p is PG_MARKER", m)); - KASSERT((m->flags & PG_FICTITIOUS) == 0 || m->wire_count == 1, - ("fictitious page %p has invalid wire count", m)); + KASSERT((m->flags & PG_FICTITIOUS) == 0 || m->ref_count >= 1, + ("fictitious page %p has invalid ref count", m)); /* * If the current page would be the start of a run, check its @@ -2358,9 +2365,6 @@ retry: */ VM_OBJECT_RUNLOCK(object); goto retry; - } else if (vm_page_wired(m)) { - run_ext = 0; - goto unlock; } } /* Don't care: PG_NODUMP, PG_ZERO. */ @@ -2378,7 +2382,8 @@ retry: vm_reserv_size(level)) - pa); #endif } else if (object->memattr == VM_MEMATTR_DEFAULT && - vm_page_queue(m) != PQ_NONE && !vm_page_busied(m)) { + vm_page_queue(m) != PQ_NONE && !vm_page_busied(m) && + !vm_page_wired(m)) { /* * The page is allocated but eligible for * relocation. Extend the current run by one @@ -2394,7 +2399,6 @@ retry: run_ext = 1; } else run_ext = 0; -unlock: VM_OBJECT_RUNLOCK(object); #if VM_NRESERVLEVEL > 0 } else if (level >= 0) { @@ -2498,6 +2502,9 @@ vm_page_reclaim_run(int req_class, int domain, u_l */ vm_page_change_lock(m, &m_mtx); retry: + /* + * Racily check for wirings. Races are handled below. + */ if (vm_page_wired(m)) error = EBUSY; else if ((object = m->object) != NULL) { @@ -2515,9 +2522,6 @@ retry: */ VM_OBJECT_WUNLOCK(object); goto retry; - } else if (vm_page_wired(m)) { - error = EBUSY; - goto unlock; } } /* Don't care: PG_NODUMP, PG_ZERO. */ @@ -2528,7 +2532,7 @@ retry: else if (object->memattr != VM_MEMATTR_DEFAULT) error = EINVAL; else if (vm_page_queue(m) != PQ_NONE && - !vm_page_busied(m)) { + !vm_page_busied(m) && !vm_page_wired(m)) { KASSERT(pmap_page_get_memattr(m) == VM_MEMATTR_DEFAULT, ("page %p has an unexpected memattr", m)); @@ -2577,8 +2581,6 @@ retry: error = ENOMEM; goto unlock; } - KASSERT(!vm_page_wired(m_new), - ("page %p is wired", m_new)); /* * Replace "m" with the new page. For @@ -2586,8 +2588,11 @@ retry: * and dequeued. Finally, change "m" * as if vm_page_free() was called. */ - if (object->ref_count != 0) - pmap_remove_all(m); + if (object->ref_count != 0 && + !vm_page_try_remove_all(m)) { + error = EBUSY; + goto unlock; + } m_new->aflags = m->aflags & ~PGA_QUEUE_STATE_MASK; KASSERT(m_new->oflags == VPO_UNMANAGED, @@ -3134,6 +3139,13 @@ vm_pqbatch_process(struct vm_pagequeue *pq, struct vm_batchqueue_init(bq); } +/* + * vm_page_pqbatch_submit: [ internal use only ] + * + * Enqueue a page in the specified page queue's batched work queue. + * The caller must have encoded the requested operation in the page + * structure's aflags field. + */ void vm_page_pqbatch_submit(vm_page_t m, uint8_t queue) { @@ -3143,8 +3155,7 @@ vm_page_pqbatch_submit(vm_page_t m, uint8_t queue) KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("page %p is unmanaged", m)); - KASSERT(mtx_owned(vm_page_lockptr(m)) || - (m->object == NULL && (m->aflags & PGA_DEQUEUE) != 0), + KASSERT(mtx_owned(vm_page_lockptr(m)) || m->object == NULL, ("missing synchronization for page %p", m)); KASSERT(queue < PQ_COUNT, ("invalid queue %d", queue)); @@ -3255,17 +3266,26 @@ vm_page_dequeue_deferred(vm_page_t m) if ((queue = vm_page_queue(m)) == PQ_NONE) return; - vm_page_aflag_set(m, PGA_DEQUEUE); - vm_page_pqbatch_submit(m, queue); + + /* + * Set PGA_DEQUEUE if it is not already set to handle a concurrent call + * to vm_page_dequeue_deferred_free(). In particular, avoid modifying + * the page's queue state once vm_page_dequeue_deferred_free() has been + * called. In the event of a race, two batch queue entries for the page + * will be created, but the second will have no effect. + */ + if (vm_page_pqstate_cmpset(m, queue, queue, PGA_DEQUEUE, PGA_DEQUEUE)) + vm_page_pqbatch_submit(m, queue); } /* * A variant of vm_page_dequeue_deferred() that does not assert the page - * lock and is only to be called from vm_page_free_prep(). It is just an - * open-coded implementation of vm_page_dequeue_deferred(). Because the - * page is being freed, we can assume that nothing else is scheduling queue - * operations on this page, so we get for free the mutual exclusion that - * is otherwise provided by the page lock. + * lock and is only to be called from vm_page_free_prep(). Because the + * page is being freed, we can assume that nothing other than the page + * daemon is scheduling queue operations on this page, so we get for + * free the mutual exclusion that is otherwise provided by the page lock. + * To handle races, the page daemon must take care to atomically check + * for PGA_DEQUEUE when updating queue state. */ static void vm_page_dequeue_deferred_free(vm_page_t m) @@ -3272,7 +3292,7 @@ vm_page_dequeue_deferred_free(vm_page_t m) { uint8_t queue; - KASSERT(m->object == NULL, ("page %p has an object reference", m)); + KASSERT(m->ref_count == 0, ("page %p has references", m)); if ((m->aflags & PGA_DEQUEUE) != 0) return; @@ -3379,6 +3399,42 @@ vm_page_requeue(vm_page_t m) } /* + * vm_page_swapqueue: [ internal use only ] + * + * Move the page from one queue to another, or to the tail of its + * current queue, in the face of a possible concurrent call to + * vm_page_dequeue_deferred_free(). + */ +void +vm_page_swapqueue(vm_page_t m, uint8_t oldq, uint8_t newq) +{ + struct vm_pagequeue *pq; + + KASSERT(oldq < PQ_COUNT && newq < PQ_COUNT && oldq != newq, + ("vm_page_swapqueue: invalid queues (%d, %d)", oldq, newq)); + KASSERT((m->oflags & VPO_UNMANAGED) == 0, + ("vm_page_swapqueue: page %p is unmanaged", m)); + vm_page_assert_locked(m); + + /* + * Atomically update the queue field and set PGA_REQUEUE while + * ensuring that PGA_DEQUEUE has not been set. + */ + pq = &vm_pagequeue_domain(m)->vmd_pagequeues[oldq]; + vm_pagequeue_lock(pq); + if (!vm_page_pqstate_cmpset(m, oldq, newq, PGA_DEQUEUE, PGA_REQUEUE)) { + vm_pagequeue_unlock(pq); + return; + } + if ((m->aflags & PGA_ENQUEUED) != 0) { + vm_pagequeue_remove(pq, m); + vm_page_aflag_clear(m, PGA_ENQUEUED); + } + vm_pagequeue_unlock(pq); + vm_page_pqbatch_submit(m, newq); +} + +/* * vm_page_free_prep: * * Prepares the given page to be put on the free list, @@ -3392,6 +3448,12 @@ bool vm_page_free_prep(vm_page_t m) { + /* + * Synchronize with threads that have dropped a reference to this + * page. + */ + atomic_thread_fence_acq(); + #if defined(DIAGNOSTIC) && defined(PHYS_TO_DMAP) if (PMAP_HAS_DMAP && (m->flags & PG_ZERO) != 0) { uint64_t *p; @@ -3402,11 +3464,10 @@ vm_page_free_prep(vm_page_t m) m, i, (uintmax_t)*p)); } #endif - if ((m->oflags & VPO_UNMANAGED) == 0) { - vm_page_lock_assert(m, MA_OWNED); + if ((m->oflags & VPO_UNMANAGED) == 0) KASSERT(!pmap_page_is_mapped(m), ("vm_page_free_prep: freeing mapped page %p", m)); - } else + else KASSERT(m->queue == PQ_NONE, ("vm_page_free_prep: unmanaged page %p is queued", m)); VM_CNT_INC(v_tfree); @@ -3414,16 +3475,28 @@ vm_page_free_prep(vm_page_t m) if (vm_page_sbusied(m)) panic("vm_page_free_prep: freeing busy page %p", m); - if (m->object != NULL) - (void)vm_page_remove(m); + if (m->object != NULL) { + vm_page_object_remove(m); + /* + * The object reference can be released without an atomic + * operation. + */ + KASSERT((m->flags & PG_FICTITIOUS) != 0 || + m->ref_count == VPRC_OBJREF, + ("vm_page_free_prep: page %p has unexpected ref_count %u", + m, m->ref_count)); + m->object = NULL; + m->ref_count = 0; + } + /* * If fictitious remove object association and * return. */ if ((m->flags & PG_FICTITIOUS) != 0) { - KASSERT(m->wire_count == 1, - ("fictitious page %p is not wired", m)); + KASSERT(m->ref_count == 1, + ("fictitious page %p is referenced", m)); KASSERT(m->queue == PQ_NONE, ("fictitious page %p is queued", m)); return (false); @@ -3440,8 +3513,8 @@ vm_page_free_prep(vm_page_t m) m->valid = 0; vm_page_undirty(m); - if (vm_page_wired(m) != 0) - panic("vm_page_free_prep: freeing wired page %p", m); + if (m->ref_count != 0) + panic("vm_page_free_prep: page %p has references", m); /* * Restore the default memory attribute to the page. @@ -3523,161 +3596,193 @@ vm_page_free_pages_toq(struct spglist *free, bool } /* - * vm_page_wire: - * - * Mark this page as wired down. If the page is fictitious, then - * its wire count must remain one. - * - * The page must be locked. + * Mark this page as wired down, preventing reclamation by the page daemon + * or when the containing object is destroyed. */ void vm_page_wire(vm_page_t m) { + u_int old; - vm_page_assert_locked(m); - if ((m->flags & PG_FICTITIOUS) != 0) { - KASSERT(m->wire_count == 1, - ("vm_page_wire: fictitious page %p's wire count isn't one", - m)); - return; - } - if (!vm_page_wired(m)) { - KASSERT((m->oflags & VPO_UNMANAGED) == 0 || - m->queue == PQ_NONE, - ("vm_page_wire: unmanaged page %p is queued", m)); + KASSERT(m->object != NULL, + ("vm_page_wire: page %p does not belong to an object", m)); + if (!vm_page_busied(m)) + VM_OBJECT_ASSERT_LOCKED(m->object); + KASSERT((m->flags & PG_FICTITIOUS) == 0 || + VPRC_WIRE_COUNT(m->ref_count) >= 1, + ("vm_page_wire: fictitious page %p has zero wirings", m)); + + old = atomic_fetchadd_int(&m->ref_count, 1); + KASSERT(VPRC_WIRE_COUNT(old) != VPRC_WIRE_COUNT_MAX, + ("vm_page_wire: counter overflow for page %p", m)); + if (VPRC_WIRE_COUNT(old) == 0) vm_wire_add(1); - } - m->wire_count++; - KASSERT(m->wire_count != 0, ("vm_page_wire: wire_count overflow m=%p", m)); } /* - * vm_page_unwire: - * + * Attempt to wire a mapped page following a pmap lookup of that page. + * This may fail if a thread is concurrently tearing down mappings of the page. + */ +bool +vm_page_wire_mapped(vm_page_t m) +{ + u_int old; + + old = m->ref_count; + do { + KASSERT(old > 0, + ("vm_page_wire_mapped: wiring unreferenced page %p", m)); + if ((old & VPRC_BLOCKED) != 0) + return (false); + } while (!atomic_fcmpset_int(&m->ref_count, &old, old + 1)); + + if (VPRC_WIRE_COUNT(old) == 0) + vm_wire_add(1); + return (true); +} + +/* * Release one wiring of the specified page, potentially allowing it to be - * paged out. Returns TRUE if the number of wirings transitions to zero and - * FALSE otherwise. + * paged out. * * Only managed pages belonging to an object can be paged out. If the number * of wirings transitions to zero and the page is eligible for page out, then - * the page is added to the specified paging queue (unless PQ_NONE is - * specified, in which case the page is dequeued if it belongs to a paging - * queue). + * the page is added to the specified paging queue. If the released wiring + * represented the last reference to the page, the page is freed. * - * If a page is fictitious, then its wire count must always be one. - * * A managed page must be locked. */ -bool +void vm_page_unwire(vm_page_t m, uint8_t queue) { - bool unwired; + u_int old; + bool locked; - KASSERT(queue < PQ_COUNT || queue == PQ_NONE, - ("vm_page_unwire: invalid queue %u request for page %p", - queue, m)); - if ((m->oflags & VPO_UNMANAGED) == 0) - vm_page_assert_locked(m); + KASSERT(queue < PQ_COUNT, + ("vm_page_unwire: invalid queue %u request for page %p", queue, m)); - unwired = vm_page_unwire_noq(m); - if (!unwired || (m->oflags & VPO_UNMANAGED) != 0 || m->object == NULL) - return (unwired); + if ((m->oflags & VPO_UNMANAGED) != 0) { + if (vm_page_unwire_noq(m) && m->ref_count == 0) + vm_page_free(m); + return; + } - if (vm_page_queue(m) == queue) { - if (queue == PQ_ACTIVE) - vm_page_reference(m); - else if (queue != PQ_NONE) - vm_page_requeue(m); - } else { - vm_page_dequeue(m); - if (queue != PQ_NONE) { - vm_page_enqueue(m, queue); - if (queue == PQ_ACTIVE) - /* Initialize act_count. */ - vm_page_activate(m); + /* + * Update LRU state before releasing the wiring reference. + * We only need to do this once since we hold the page lock. + * Use a release store when updating the reference count to + * synchronize with vm_page_free_prep(). + */ + old = m->ref_count; + locked = false; + do { + KASSERT(VPRC_WIRE_COUNT(old) > 0, + ("vm_page_unwire: wire count underflow for page %p", m)); + if (!locked && VPRC_WIRE_COUNT(old) == 1) { + vm_page_lock(m); + locked = true; + if (queue == PQ_ACTIVE && vm_page_queue(m) == PQ_ACTIVE) + vm_page_reference(m); + else + vm_page_mvqueue(m, queue); } + } while (!atomic_fcmpset_rel_int(&m->ref_count, &old, old - 1)); + + /* + * Release the lock only after the wiring is released, to ensure that + * the page daemon does not encounter and dequeue the page while it is + * still wired. + */ + if (locked) + vm_page_unlock(m); + + if (VPRC_WIRE_COUNT(old) == 1) { + vm_wire_sub(1); + if (old == 1) + vm_page_free(m); } - return (unwired); } /* - * - * vm_page_unwire_noq: - * * Unwire a page without (re-)inserting it into a page queue. It is up * to the caller to enqueue, requeue, or free the page as appropriate. - * In most cases, vm_page_unwire() should be used instead. + * In most cases involving managed pages, vm_page_unwire() should be used + * instead. */ bool vm_page_unwire_noq(vm_page_t m) { + u_int old; - if ((m->oflags & VPO_UNMANAGED) == 0) - vm_page_assert_locked(m); - if ((m->flags & PG_FICTITIOUS) != 0) { - KASSERT(m->wire_count == 1, - ("vm_page_unwire: fictitious page %p's wire count isn't one", m)); + if ((m->oflags & VPO_UNMANAGED) != 0) + old = m->ref_count--; + else + old = vm_page_drop(m, 1); + KASSERT(VPRC_WIRE_COUNT(old) != 0, + ("vm_page_unref: counter underflow for page %p", m)); + KASSERT((m->flags & PG_FICTITIOUS) == 0 || VPRC_WIRE_COUNT(old) > 1, + ("vm_page_unref: missing ref on fictitious page %p", m)); + + if (VPRC_WIRE_COUNT(old) > 1) return (false); - } - if (!vm_page_wired(m)) - panic("vm_page_unwire: page %p's wire count is zero", m); - m->wire_count--; - if (m->wire_count == 0) { - vm_wire_sub(1); - return (true); - } else - return (false); + vm_wire_sub(1); + return (true); } /* - * vm_page_activate: + * Ensure that the page is in the specified page queue. If the page is + * active or being moved to the active queue, ensure that its act_count is + * at least ACT_INIT but do not otherwise mess with it. Otherwise, ensure that + * the page is at the tail of its page queue. * - * Put the specified page on the active list (if appropriate). - * Ensure that act_count is at least ACT_INIT but do not otherwise - * mess with it. + * The page may be wired. The caller should release its wiring reference + * before releasing the page lock, otherwise the page daemon may immediately + * dequeue the page. * - * The page must be locked. + * A managed page must be locked. */ -void -vm_page_activate(vm_page_t m) +static __always_inline void +vm_page_mvqueue(vm_page_t m, const uint8_t nqueue) { vm_page_assert_locked(m); + KASSERT((m->oflags & VPO_UNMANAGED) == 0, + ("vm_page_mvqueue: page %p is unmanaged", m)); - if (vm_page_wired(m) || (m->oflags & VPO_UNMANAGED) != 0) - return; - if (vm_page_queue(m) == PQ_ACTIVE) { - if (m->act_count < ACT_INIT) - m->act_count = ACT_INIT; - return; + if (vm_page_queue(m) != nqueue) { + vm_page_dequeue(m); + vm_page_enqueue(m, nqueue); + } else if (nqueue != PQ_ACTIVE) { + vm_page_requeue(m); } - vm_page_dequeue(m); - if (m->act_count < ACT_INIT) + if (nqueue == PQ_ACTIVE && m->act_count < ACT_INIT) m->act_count = ACT_INIT; - vm_page_enqueue(m, PQ_ACTIVE); } /* + * Put the specified page on the active list (if appropriate). + */ +void +vm_page_activate(vm_page_t m) +{ + + if ((m->oflags & VPO_UNMANAGED) != 0 || vm_page_wired(m)) + return; + vm_page_mvqueue(m, PQ_ACTIVE); +} + +/* * Move the specified page to the tail of the inactive queue, or requeue * the page if it is already in the inactive queue. - * - * The page must be locked. */ void vm_page_deactivate(vm_page_t m) { - vm_page_assert_locked(m); - - if (vm_page_wired(m) || (m->oflags & VPO_UNMANAGED) != 0) + if ((m->oflags & VPO_UNMANAGED) != 0 || vm_page_wired(m)) return; - - if (!vm_page_inactive(m)) { - vm_page_dequeue(m); - vm_page_enqueue(m, PQ_INACTIVE); - } else - vm_page_requeue(m); + vm_page_mvqueue(m, PQ_INACTIVE); } /* @@ -3685,18 +3790,13 @@ vm_page_deactivate(vm_page_t m) * bypassing LRU. A marker page is used to maintain FIFO ordering. * As with regular enqueues, we use a per-CPU batch queue to reduce * contention on the page queue lock. - * - * The page must be locked. */ -void -vm_page_deactivate_noreuse(vm_page_t m) +static void +_vm_page_deactivate_noreuse(vm_page_t m) { vm_page_assert_locked(m); - if (vm_page_wired(m) || (m->oflags & VPO_UNMANAGED) != 0) - return; - if (!vm_page_inactive(m)) { vm_page_dequeue(m); m->queue = PQ_INACTIVE; @@ -3706,31 +3806,31 @@ vm_page_deactivate(vm_page_t m) vm_page_pqbatch_submit(m, PQ_INACTIVE); } +void +vm_page_deactivate_noreuse(vm_page_t m) +{ + + KASSERT(m->object != NULL, + ("vm_page_deactivate_noreuse: page %p has no object", m)); + + if ((m->oflags & VPO_UNMANAGED) == 0 && !vm_page_wired(m)) + _vm_page_deactivate_noreuse(m); +} + /* - * vm_page_launder - * - * Put a page in the laundry, or requeue it if it is already there. + * Put a page in the laundry, or requeue it if it is already there. */ void vm_page_launder(vm_page_t m) { - vm_page_assert_locked(m); - if (vm_page_wired(m) || (m->oflags & VPO_UNMANAGED) != 0) + if ((m->oflags & VPO_UNMANAGED) != 0 || vm_page_wired(m)) return; - - if (vm_page_in_laundry(m)) - vm_page_requeue(m); - else { - vm_page_dequeue(m); - vm_page_enqueue(m, PQ_LAUNDRY); - } + vm_page_mvqueue(m, PQ_LAUNDRY); } /* - * vm_page_unswappable - * - * Put a page in the PQ_UNSWAPPABLE holding queue. + * Put a page in the PQ_UNSWAPPABLE holding queue. */ void vm_page_unswappable(vm_page_t m) @@ -3748,6 +3848,8 @@ static void vm_page_release_toq(vm_page_t m, int flags) { + vm_page_assert_locked(m); + /* * Use a check of the valid bits to determine whether we should * accelerate reclamation of the page. The object lock might not be @@ -3759,11 +3861,11 @@ vm_page_release_toq(vm_page_t m, int flags) * inactive queue so that is reclaimed sooner. */ if ((flags & (VPR_TRYFREE | VPR_NOREUSE)) != 0 || m->valid == 0) - vm_page_deactivate_noreuse(m); + _vm_page_deactivate_noreuse(m); else if (vm_page_active(m)) vm_page_reference(m); else - vm_page_deactivate(m); + vm_page_mvqueue(m, PQ_INACTIVE); } /* @@ -3773,41 +3875,63 @@ void vm_page_release(vm_page_t m, int flags) { vm_object_t object; - bool freed; + u_int old; + bool locked; KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("vm_page_release: page %p is unmanaged", m)); - vm_page_lock(m); - if (m->object != NULL) - VM_OBJECT_ASSERT_UNLOCKED(m->object); - if (vm_page_unwire_noq(m)) { - if ((object = m->object) == NULL) { - vm_page_free(m); - } else { - freed = false; - if ((flags & VPR_TRYFREE) != 0 && !vm_page_busied(m) && - /* Depends on type stability. */ - VM_OBJECT_TRYWLOCK(object)) { - /* - * Only free unmapped pages. The busy test from - * before the object was locked cannot be relied - * upon. - */ - if ((object->ref_count == 0 || - !pmap_page_is_mapped(m)) && m->dirty == 0 && - !vm_page_busied(m)) { - vm_page_free(m); - freed = true; - } - VM_OBJECT_WUNLOCK(object); + if ((flags & VPR_TRYFREE) != 0) { + for (;;) { + object = (vm_object_t)atomic_load_ptr(&m->object); + if (object == NULL) + break; + /* Depends on type-stability. */ + if (vm_page_busied(m) || !VM_OBJECT_TRYWLOCK(object)) { + object = NULL; + break; } + if (object == m->object) + break; + VM_OBJECT_WUNLOCK(object); + } + if (__predict_true(object != NULL)) { + vm_page_release_locked(m, flags); + VM_OBJECT_WUNLOCK(object); + return; + } + } - if (!freed) - vm_page_release_toq(m, flags); + /* + * Update LRU state before releasing the wiring reference. + * Use a release store when updating the reference count to + * synchronize with vm_page_free_prep(). + */ + old = m->ref_count; + locked = false; + do { + KASSERT(VPRC_WIRE_COUNT(old) > 0, + ("vm_page_unwire: wire count underflow for page %p", m)); + if (!locked && VPRC_WIRE_COUNT(old) == 1) { + vm_page_lock(m); + locked = true; + vm_page_release_toq(m, flags); } + } while (!atomic_fcmpset_rel_int(&m->ref_count, &old, old - 1)); + + /* + * Release the lock only after the wiring is released, to ensure that + * the page daemon does not encounter and dequeue the page while it is + * still wired. + */ + if (locked) + vm_page_unlock(m); + + if (VPRC_WIRE_COUNT(old) == 1) { + vm_wire_sub(1); + if (old == 1) + vm_page_free(m); } - vm_page_unlock(m); } /* See vm_page_release(). */ @@ -3819,7 +3943,6 @@ vm_page_release_locked(vm_page_t m, int flags) KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("vm_page_release_locked: page %p is unmanaged", m)); - vm_page_lock(m); if (vm_page_unwire_noq(m)) { if ((flags & VPR_TRYFREE) != 0 && (m->object->ref_count == 0 || !pmap_page_is_mapped(m)) && @@ -3826,13 +3949,67 @@ vm_page_release_locked(vm_page_t m, int flags) m->dirty == 0 && !vm_page_busied(m)) { vm_page_free(m); } else { + vm_page_lock(m); vm_page_release_toq(m, flags); + vm_page_unlock(m); } } - vm_page_unlock(m); } +static bool +vm_page_try_blocked_op(vm_page_t m, void (*op)(vm_page_t)) +{ + u_int old; + + KASSERT(m->object != NULL && (m->oflags & VPO_UNMANAGED) == 0, + ("vm_page_try_blocked_op: page %p has no object", m)); + KASSERT(!vm_page_busied(m), + ("vm_page_try_blocked_op: page %p is busy", m)); + VM_OBJECT_ASSERT_LOCKED(m->object); + + old = m->ref_count; + do { + KASSERT(old != 0, + ("vm_page_try_blocked_op: page %p has no references", m)); + if (VPRC_WIRE_COUNT(old) != 0) + return (false); + } while (!atomic_fcmpset_int(&m->ref_count, &old, old | VPRC_BLOCKED)); + + (op)(m); + + /* + * If the object is read-locked, new wirings may be created via an + * object lookup. + */ + old = vm_page_drop(m, VPRC_BLOCKED); + KASSERT(!VM_OBJECT_WOWNED(m->object) || + old == (VPRC_BLOCKED | VPRC_OBJREF), + ("vm_page_try_blocked_op: unexpected refcount value %u for %p", + old, m)); + return (true); +} + /* + * Atomically check for wirings and remove all mappings of the page. + */ +bool +vm_page_try_remove_all(vm_page_t m) +{ + + return (vm_page_try_blocked_op(m, pmap_remove_all)); +} + +/* + * Atomically check for wirings and remove all writeable mappings of the page. + */ +bool +vm_page_try_remove_write(vm_page_t m) +{ + + return (vm_page_try_blocked_op(m, pmap_remove_write)); +} + +/* * vm_page_advise * * Apply the specified advice to the given page. @@ -3926,11 +4103,8 @@ retrylookup: VM_OBJECT_WLOCK(object); goto retrylookup; } else { - if ((allocflags & VM_ALLOC_WIRED) != 0) { - vm_page_lock(m); + if ((allocflags & VM_ALLOC_WIRED) != 0) vm_page_wire(m); - vm_page_unlock(m); - } if ((allocflags & (VM_ALLOC_NOBUSY | VM_ALLOC_SBUSY)) == 0) vm_page_xbusy(m); @@ -4028,11 +4202,8 @@ retrylookup: VM_OBJECT_WLOCK(object); goto retrylookup; } - if ((allocflags & VM_ALLOC_WIRED) != 0) { - vm_page_lock(m); + if ((allocflags & VM_ALLOC_WIRED) != 0) vm_page_wire(m); - vm_page_unlock(m); - } if ((allocflags & (VM_ALLOC_NOBUSY | VM_ALLOC_SBUSY)) == 0) vm_page_xbusy(m); @@ -4561,10 +4732,10 @@ DB_SHOW_COMMAND(pginfo, vm_page_print_pginfo) else m = (vm_page_t)addr; db_printf( - "page %p obj %p pidx 0x%jx phys 0x%jx q %d wire %d\n" + "page %p obj %p pidx 0x%jx phys 0x%jx q %d ref %u\n" " af 0x%x of 0x%x f 0x%x act %d busy %x valid 0x%x dirty 0x%x\n", m, m->object, (uintmax_t)m->pindex, (uintmax_t)m->phys_addr, - m->queue, m->wire_count, m->aflags, m->oflags, + m->queue, m->ref_count, m->aflags, m->oflags, m->flags, m->act_count, m->busy_lock, m->valid, m->dirty); } #endif /* DDB */ Index: src/sys/vm/vm_page.h =================================================================== --- src/sys/vm/vm_page.h (revision 351575) +++ src/sys/vm/vm_page.h (working copy) @@ -115,25 +115,24 @@ * the implementation of read-modify-write operations on the * field is encapsulated in vm_page_clear_dirty_mask(). * - * The page structure contains two counters which prevent page reuse. - * Both counters are protected by the page lock (P). The hold - * counter counts transient references obtained via a pmap lookup, and - * is also used to prevent page reclamation in situations where it is - * undesirable to block other accesses to the page. The wire counter - * is used to implement mlock(2) and is non-zero for pages containing - * kernel memory. Pages that are wired or held will not be reclaimed - * or laundered by the page daemon, but are treated differently during - * a page queue scan: held pages remain at their position in the queue, - * while wired pages are removed from the queue and must later be - * re-enqueued appropriately by the unwiring thread. It is legal to - * call vm_page_free() on a held page; doing so causes it to be removed - * from its object and page queue, and the page is released to the - * allocator once the last hold reference is dropped. In contrast, - * wired pages may not be freed. + * The ref_count field tracks references to the page. References that + * prevent the page from being reclaimable are called wirings and are + * counted in the low bits of ref_count. The containing object's + * reference, if one exists, is counted using the VPRC_OBJREF bit in the + * ref_count field. Additionally, the VPRC_BLOCKED bit is used to + * atomically check for wirings and prevent new wirings via + * pmap_extract_and_hold(). When a page belongs to an object, it may be + * wired only when the object is locked, or the page is busy, or by + * pmap_extract_and_hold(). As a result, if the object is locked and the + * page is not busy (or is exclusively busied by the current thread), and + * the page is unmapped, its wire count will not increase. The ref_count + * field is updated using atomic operations in most cases, except when it + * is known that no other references to the page exist, such as in the page + * allocator. A page may be present in the page queues, or even actively + * scanned by the page daemon, without an explicitly counted referenced. + * The page daemon must therefore handle the possibility of a concurrent + * free of the page. * - * In some pmap implementations, the wire count of a page table page is - * used to track the number of populated entries. - * * The busy lock is an embedded reader-writer lock which protects the * page's contents and identity (i.e., its tuple) and * interlocks with the object lock (O). In particular, a page may be @@ -147,28 +146,34 @@ * sleep until the page's busy state changes, after which the caller * must re-lookup the page and re-evaluate its state. * - * The queue field is the index of the page queue containing the - * page, or PQ_NONE if the page is not enqueued. The queue lock of a - * page is the page queue lock corresponding to the page queue index, - * or the page lock (P) for the page if it is not enqueued. To modify - * the queue field, the queue lock for the old value of the field must - * be held. It is invalid for a page's queue field to transition - * between two distinct page queue indices. That is, when updating - * the queue field, either the new value or the old value must be - * PQ_NONE. + * The queue field is the index of the page queue containing the page, + * or PQ_NONE if the page is not enqueued. The queue lock of a page is + * the page queue lock corresponding to the page queue index, or the + * page lock (P) for the page if it is not enqueued. To modify the + * queue field, the queue lock for the old value of the field must be + * held. There is one exception to this rule: the page daemon may + * transition the queue field from PQ_INACTIVE to PQ_NONE immediately + * prior to freeing a page during an inactive queue scan. At that + * point the page has already been physically dequeued and no other + * references to that vm_page structure exist. * * To avoid contention on page queue locks, page queue operations - * (enqueue, dequeue, requeue) are batched using per-CPU queues. - * A deferred operation is requested by inserting an entry into a - * batch queue; the entry is simply a pointer to the page, and the - * request type is encoded in the page's aflags field using the values - * in PGA_QUEUE_STATE_MASK. The type-stability of struct vm_pages is + * (enqueue, dequeue, requeue) are batched using per-CPU queues. A + * deferred operation is requested by inserting an entry into a batch + * queue; the entry is simply a pointer to the page, and the request + * type is encoded in the page's aflags field using the values in + * PGA_QUEUE_STATE_MASK. The type-stability of struct vm_pages is * crucial to this scheme since the processing of entries in a given - * batch queue may be deferred indefinitely. In particular, a page - * may be freed before its pending batch queue entries have been - * processed. The page lock (P) must be held to schedule a batched - * queue operation, and the page queue lock must be held in order to - * process batch queue entries for the page queue. + * batch queue may be deferred indefinitely. In particular, a page may + * be freed before its pending batch queue entries have been processed. + * The page lock (P) must be held to schedule a batched queue + * operation, and the page queue lock must be held in order to process + * batch queue entries for the page queue. There is one exception to + * this rule: the thread freeing a page may schedule a dequeue without + * holding the page lock. In this scenario the only other thread which + * may hold a reference to the page is the page daemon, which is + * careful to avoid modifying the page's queue state once the dequeue + * has been requested by setting PGA_DEQUEUE. */ #if PAGE_SIZE == 4096 @@ -198,11 +203,14 @@ struct vm_page { } memguard; } plinks; TAILQ_ENTRY(vm_page) listq; /* pages in same object (O) */ - vm_object_t object; /* which object am I in (O,P) */ + vm_object_t object; /* which object am I in (O) */ vm_pindex_t pindex; /* offset into object (O,P) */ vm_paddr_t phys_addr; /* physical address of page (C) */ struct md_page md; /* machine dependent stuff */ - u_int wire_count; /* wired down maps refs (P) */ + union { + u_int wire_count; + u_int ref_count; /* page references */ + }; volatile u_int busy_lock; /* busy owners lock */ uint16_t flags; /* page PG_* flags (P) */ uint8_t order; /* index of the buddy queue (F) */ @@ -220,6 +228,26 @@ struct vm_page { }; /* + * Special bits used in the ref_count field. + * + * ref_count is normally used to count wirings that prevent the page from being + * reclaimed, but also supports several special types of references that do not + * prevent reclamation. Accesses to the ref_count field must be atomic unless + * the page is unallocated. + * + * VPRC_OBJREF is the reference held by the containing object. It can set or + * cleared only when the corresponding object's write lock is held. + * + * VPRC_BLOCKED is used to atomically block wirings via pmap lookups while + * attempting to tear down all mappings of a given page. The page lock and + * object write lock must both be held in order to set or clear this bit. + */ +#define VPRC_BLOCKED 0x40000000u /* mappings are being removed */ +#define VPRC_OBJREF 0x80000000u /* object reference, cleared with (O) */ +#define VPRC_WIRE_COUNT(c) ((c) & ~(VPRC_BLOCKED | VPRC_OBJREF)) +#define VPRC_WIRE_COUNT_MAX (~(VPRC_BLOCKED | VPRC_OBJREF)) + +/* * Page flags stored in oflags: * * Access to these page flags is synchronized by the lock on the object @@ -578,13 +606,17 @@ void vm_page_set_valid_range(vm_page_t m, int base int vm_page_sleep_if_busy(vm_page_t m, const char *msg); vm_offset_t vm_page_startup(vm_offset_t vaddr); void vm_page_sunbusy(vm_page_t m); +void vm_page_swapqueue(vm_page_t m, uint8_t oldq, uint8_t newq); +bool vm_page_try_remove_all(vm_page_t m); +bool vm_page_try_remove_write(vm_page_t m); int vm_page_trysbusy(vm_page_t m); void vm_page_unhold_pages(vm_page_t *ma, int count); void vm_page_unswappable(vm_page_t m); -bool vm_page_unwire(vm_page_t m, uint8_t queue); +void vm_page_unwire(vm_page_t m, uint8_t queue); bool vm_page_unwire_noq(vm_page_t m); void vm_page_updatefake(vm_page_t m, vm_paddr_t paddr, vm_memattr_t memattr); -void vm_page_wire (vm_page_t); +void vm_page_wire(vm_page_t); +bool vm_page_wire_mapped(vm_page_t m); void vm_page_xunbusy_hard(vm_page_t m); void vm_page_xunbusy_maybelocked(vm_page_t m); void vm_page_set_validclean (vm_page_t, int, int); @@ -667,9 +699,31 @@ void vm_page_assert_pga_writeable(vm_page_t m, uin * destinations. In order that we can easily use a 32-bit operation, we * require that the aflags field be 32-bit aligned. */ -CTASSERT(offsetof(struct vm_page, aflags) % sizeof(uint32_t) == 0); +_Static_assert(offsetof(struct vm_page, aflags) % sizeof(uint32_t) == 0, + "aflags field is not 32-bit aligned"); /* + * We want to be able to update the aflags and queue fields atomically in + * the same operation. + */ +_Static_assert(offsetof(struct vm_page, aflags) / sizeof(uint32_t) == + offsetof(struct vm_page, queue) / sizeof(uint32_t), + "aflags and queue fields do not belong to the same 32-bit word"); +_Static_assert(offsetof(struct vm_page, queue) % sizeof(uint32_t) == 2, + "queue field is at an unexpected offset"); +_Static_assert(sizeof(((struct vm_page *)NULL)->queue) == 1, + "queue field has an unexpected size"); + +#if BYTE_ORDER == LITTLE_ENDIAN +#define VM_PAGE_AFLAG_SHIFT 0 +#define VM_PAGE_QUEUE_SHIFT 16 +#else +#define VM_PAGE_AFLAG_SHIFT 24 +#define VM_PAGE_QUEUE_SHIFT 8 +#endif +#define VM_PAGE_QUEUE_MASK (0xff << VM_PAGE_QUEUE_SHIFT) + +/* * Clear the given bits in the specified page. */ static inline void @@ -689,12 +743,7 @@ vm_page_aflag_clear(vm_page_t m, uint8_t bits) * within this word are handled properly by the atomic update. */ addr = (void *)&m->aflags; - KASSERT(((uintptr_t)addr & (sizeof(uint32_t) - 1)) == 0, - ("vm_page_aflag_clear: aflags is misaligned")); - val = bits; -#if BYTE_ORDER == BIG_ENDIAN - val <<= 24; -#endif + val = bits << VM_PAGE_AFLAG_SHIFT; atomic_clear_32(addr, val); } @@ -714,16 +763,46 @@ vm_page_aflag_set(vm_page_t m, uint8_t bits) * within this word are handled properly by the atomic update. */ addr = (void *)&m->aflags; - KASSERT(((uintptr_t)addr & (sizeof(uint32_t) - 1)) == 0, - ("vm_page_aflag_set: aflags is misaligned")); - val = bits; -#if BYTE_ORDER == BIG_ENDIAN - val <<= 24; -#endif + val = bits << VM_PAGE_AFLAG_SHIFT; atomic_set_32(addr, val); -} +} /* + * Atomically update the queue state of the page. The operation fails if + * any of the queue flags in "fflags" are set or if the "queue" field of + * the page does not match the expected value; if the operation is + * successful, the flags in "nflags" are set and all other queue state + * flags are cleared. + */ +static inline bool +vm_page_pqstate_cmpset(vm_page_t m, uint32_t oldq, uint32_t newq, + uint32_t fflags, uint32_t nflags) +{ + uint32_t *addr, nval, oval, qsmask; + + vm_page_assert_locked(m); + + fflags <<= VM_PAGE_AFLAG_SHIFT; + nflags <<= VM_PAGE_AFLAG_SHIFT; + newq <<= VM_PAGE_QUEUE_SHIFT; + oldq <<= VM_PAGE_QUEUE_SHIFT; + qsmask = ((PGA_DEQUEUE | PGA_REQUEUE | PGA_REQUEUE_HEAD) << + VM_PAGE_AFLAG_SHIFT) | VM_PAGE_QUEUE_MASK; + + addr = (void *)&m->aflags; + oval = atomic_load_32(addr); + do { + if ((oval & fflags) != 0) + return (false); + if ((oval & VM_PAGE_QUEUE_MASK) != oldq) + return (false); + nval = (oval & ~qsmask) | nflags | newq; + } while (!atomic_fcmpset_32(addr, &oval, nval)); + + return (true); +} + +/* * vm_page_dirty: * * Set all bits in the page's dirty field. @@ -815,15 +894,35 @@ vm_page_in_laundry(vm_page_t m) } /* + * vm_page_drop: + * + * Release a reference to a page and return the old reference count. + */ +static inline u_int +vm_page_drop(vm_page_t m, u_int val) +{ + + /* + * Synchronize with vm_page_free_prep(): ensure that all updates to the + * page structure are visible before it is freed. + */ + atomic_thread_fence_rel(); + return (atomic_fetchadd_int(&m->ref_count, -val)); +} + +/* * vm_page_wired: * - * Return true if a reference prevents the page from being reclaimable. + * Perform a racy check to determine whether a reference prevents the page + * from being reclaimable. If the page's object is locked, and the page is + * unmapped and unbusied or exclusively busied by the current thread, no + * new wirings may be created. */ static inline bool vm_page_wired(vm_page_t m) { - return (m->wire_count > 0); + return (VPRC_WIRE_COUNT(m->ref_count) > 0); } #endif /* _KERNEL */ Index: src/sys/vm/vm_pageout.c =================================================================== --- src/sys/vm/vm_pageout.c (revision 351575) +++ src/sys/vm/vm_pageout.c (working copy) @@ -305,7 +305,9 @@ vm_pageout_collect_batch(struct scan_state *ss, co vm_pagequeue_unlock(pq); } -/* Return the next page to be scanned, or NULL if the scan is complete. */ +/* + * Return the next page to be scanned, or NULL if the scan is complete. + */ static __always_inline vm_page_t vm_pageout_next(struct scan_state *ss, const bool dequeue) { @@ -328,17 +330,12 @@ vm_pageout_cluster(vm_page_t m) vm_pindex_t pindex; int ib, is, page_base, pageout_count; - vm_page_assert_locked(m); object = m->object; VM_OBJECT_ASSERT_WLOCKED(object); pindex = m->pindex; vm_page_assert_unbusied(m); - KASSERT(!vm_page_wired(m), ("page %p is wired", m)); - pmap_remove_write(m); - vm_page_unlock(m); - mc[vm_pageout_page_count] = pb = ps = m; pageout_count = 1; page_base = vm_pageout_page_count; @@ -363,7 +360,8 @@ more: ib = 0; break; } - if ((p = vm_page_prev(pb)) == NULL || vm_page_busied(p)) { + if ((p = vm_page_prev(pb)) == NULL || vm_page_busied(p) || + vm_page_wired(p)) { ib = 0; break; } @@ -373,12 +371,11 @@ more: break; } vm_page_lock(p); - if (vm_page_wired(p) || !vm_page_in_laundry(p)) { + if (!vm_page_in_laundry(p) || !vm_page_try_remove_write(p)) { vm_page_unlock(p); ib = 0; break; } - pmap_remove_write(p); vm_page_unlock(p); mc[--page_base] = pb = p; ++pageout_count; @@ -393,17 +390,17 @@ more: } while (pageout_count < vm_pageout_page_count && pindex + is < object->size) { - if ((p = vm_page_next(ps)) == NULL || vm_page_busied(p)) + if ((p = vm_page_next(ps)) == NULL || vm_page_busied(p) || + vm_page_wired(p)) break; vm_page_test_dirty(p); if (p->dirty == 0) break; vm_page_lock(p); - if (vm_page_wired(p) || !vm_page_in_laundry(p)) { + if (!vm_page_in_laundry(p) || !vm_page_try_remove_write(p)) { vm_page_unlock(p); break; } - pmap_remove_write(p); vm_page_unlock(p); mc[page_base + pageout_count] = ps = p; ++pageout_count; @@ -648,10 +645,10 @@ vm_pageout_clean(vm_page_t m, int *numpagedout) } /* - * The page may have been busied or referenced while the object - * and page locks were released. + * The page may have been busied while the object and page + * locks were released. */ - if (vm_page_busied(m) || vm_page_wired(m)) { + if (vm_page_busied(m)) { vm_page_unlock(m); error = EBUSY; goto unlock_all; @@ -659,6 +656,16 @@ vm_pageout_clean(vm_page_t m, int *numpagedout) } /* + * Remove all writeable mappings, failing if the page is wired. + */ + if (!vm_page_try_remove_write(m)) { + vm_page_unlock(m); + error = EBUSY; + goto unlock_all; + } + vm_page_unlock(m); + + /* * If a page is dirty, then it is either being washed * (but not yet cleaned) or it is still in the * laundry. If it is still in the laundry, then we @@ -732,7 +739,9 @@ scan: recheck: /* * The page may have been disassociated from the queue - * while locks were dropped. + * or even freed while locks were dropped. We thus must be + * careful whenever modifying page state. Once the object lock + * has been acquired, we have a stable reference to the page. */ if (vm_page_queue(m) != queue) continue; @@ -742,7 +751,7 @@ recheck: * chance. */ if ((m->aflags & PGA_REQUEUE) != 0) { - vm_page_requeue(m); + vm_page_pqbatch_submit(m, queue); continue; } @@ -749,7 +758,9 @@ recheck: /* * Wired pages may not be freed. Complete their removal * from the queue now to avoid needless revisits during - * future scans. + * future scans. This check is racy and must be reverified once + * we hold the object lock and have verified that the page + * is not busy. */ if (vm_page_wired(m)) { vm_page_dequeue_deferred(m); @@ -759,8 +770,13 @@ recheck: if (object != m->object) { if (object != NULL) VM_OBJECT_WUNLOCK(object); - object = m->object; - if (!VM_OBJECT_TRYWLOCK(object)) { + + /* + * A page's object pointer may be set to NULL before + * the object lock is acquired. + */ + object = (vm_object_t)atomic_load_ptr(&m->object); + if (object != NULL && !VM_OBJECT_TRYWLOCK(object)) { mtx_unlock(mtx); /* Depends on type-stability. */ VM_OBJECT_WLOCK(object); @@ -768,11 +784,31 @@ recheck: goto recheck; } } + if (__predict_false(m->object == NULL)) + /* + * The page has been removed from its object. + */ + continue; + KASSERT(m->object == object, ("page %p does not belong to %p", + m, object)); if (vm_page_busied(m)) continue; /* + * Re-check for wirings now that we hold the object lock and + * have verified that the page is unbusied. If the page is + * mapped, it may still be wired by pmap lookups. The call to + * vm_page_try_remove_all() below atomically checks for such + * wirings and removes mappings. If the page is unmapped, the + * wire count is guaranteed not to increase. + */ + if (__predict_false(vm_page_wired(m))) { + vm_page_dequeue_deferred(m); + continue; + } + + /* * Invalid pages can be easily freed. They cannot be * mapped; vm_page_free() asserts this. */ @@ -839,8 +875,10 @@ recheck: */ if (object->ref_count != 0) { vm_page_test_dirty(m); - if (m->dirty == 0) - pmap_remove_all(m); + if (m->dirty == 0 && !vm_page_try_remove_all(m)) { + vm_page_dequeue_deferred(m); + continue; + } } /* @@ -1132,6 +1170,7 @@ vm_pageout_scan_active(struct vm_domain *vmd, int { struct scan_state ss; struct mtx *mtx; + vm_object_t object; vm_page_t m, marker; struct vm_pagequeue *pq; long min_scan; @@ -1192,7 +1231,9 @@ act_scan: /* * The page may have been disassociated from the queue - * while locks were dropped. + * or even freed while locks were dropped. We thus must be + * careful whenever modifying page state. Once the object lock + * has been acquired, we have a stable reference to the page. */ if (vm_page_queue(m) != PQ_ACTIVE) continue; @@ -1206,6 +1247,17 @@ act_scan: } /* + * A page's object pointer may be set to NULL before + * the object lock is acquired. + */ + object = (vm_object_t)atomic_load_ptr(&m->object); + if (__predict_false(object == NULL)) + /* + * The page has been removed from its object. + */ + continue; + + /* * Check to see "how much" the page has been used. * * Test PGA_REFERENCED after calling pmap_ts_referenced() so @@ -1224,7 +1276,7 @@ act_scan: * This race delays the detection of a new reference. At * worst, we will deactivate and reactivate the page. */ - if (m->object->ref_count != 0) + if (object->ref_count != 0) act_delta = pmap_ts_referenced(m); else act_delta = 0; @@ -1256,9 +1308,9 @@ act_scan: * place them directly in the laundry queue to reduce * queuing overhead. */ - if (page_shortage <= 0) - vm_page_deactivate(m); - else { + if (page_shortage <= 0) { + vm_page_swapqueue(m, PQ_ACTIVE, PQ_INACTIVE); + } else { /* * Calling vm_page_test_dirty() here would * require acquisition of the object's write @@ -1270,11 +1322,13 @@ act_scan: * dirty field by the pmap. */ if (m->dirty == 0) { - vm_page_deactivate(m); + vm_page_swapqueue(m, PQ_ACTIVE, + PQ_INACTIVE); page_shortage -= act_scan_laundry_weight; } else { - vm_page_launder(m); + vm_page_swapqueue(m, PQ_ACTIVE, + PQ_LAUNDRY); page_shortage--; } } @@ -1398,7 +1452,9 @@ vm_pageout_scan_inactive(struct vm_domain *vmd, in recheck: /* * The page may have been disassociated from the queue - * while locks were dropped. + * or even freed while locks were dropped. We thus must be + * careful whenever modifying page state. Once the object lock + * has been acquired, we have a stable reference to the page. */ if (vm_page_queue(m) != PQ_INACTIVE) { addl_page_shortage++; @@ -1417,7 +1473,9 @@ recheck: /* * Wired pages may not be freed. Complete their removal * from the queue now to avoid needless revisits during - * future scans. + * future scans. This check is racy and must be reverified once + * we hold the object lock and have verified that the page + * is not busy. */ if (vm_page_wired(m)) { vm_page_dequeue_deferred(m); @@ -1427,8 +1485,13 @@ recheck: if (object != m->object) { if (object != NULL) VM_OBJECT_WUNLOCK(object); - object = m->object; - if (!VM_OBJECT_TRYWLOCK(object)) { + + /* + * A page's object pointer may be set to NULL before + * the object lock is acquired. + */ + object = (vm_object_t)atomic_load_ptr(&m->object); + if (object != NULL && !VM_OBJECT_TRYWLOCK(object)) { mtx_unlock(mtx); /* Depends on type-stability. */ VM_OBJECT_WLOCK(object); @@ -1436,6 +1499,13 @@ recheck: goto recheck; } } + if (__predict_false(m->object == NULL)) + /* + * The page has been removed from its object. + */ + continue; + KASSERT(m->object == object, ("page %p does not belong to %p", + m, object)); if (vm_page_busied(m)) { /* @@ -1451,6 +1521,19 @@ recheck: } /* + * Re-check for wirings now that we hold the object lock and + * have verified that the page is unbusied. If the page is + * mapped, it may still be wired by pmap lookups. The call to + * vm_page_try_remove_all() below atomically checks for such + * wirings and removes mappings. If the page is unmapped, the + * wire count is guaranteed not to increase. + */ + if (__predict_false(vm_page_wired(m))) { + vm_page_dequeue_deferred(m); + continue; + } + + /* * Invalid pages can be easily freed. They cannot be * mapped, vm_page_free() asserts this. */ @@ -1506,8 +1589,10 @@ recheck: */ if (object->ref_count != 0) { vm_page_test_dirty(m); - if (m->dirty == 0) - pmap_remove_all(m); + if (m->dirty == 0 && !vm_page_try_remove_all(m)) { + vm_page_dequeue_deferred(m); + continue; + } } /* Index: src/sys/vm/vm_swapout.c =================================================================== --- src/sys/vm/vm_swapout.c (revision 351575) +++ src/sys/vm/vm_swapout.c (working copy) @@ -207,16 +207,20 @@ vm_swapout_object_deactivate_pages(pmap_t pmap, vm goto unlock_return; if (should_yield()) goto unlock_return; - if (vm_page_busied(p)) + + /* + * The page may acquire a wiring after this check. + * The page daemon handles wired pages, so there is + * no harm done if a wiring appears while we are + * attempting to deactivate the page. + */ + if (vm_page_busied(p) || vm_page_wired(p)) continue; VM_CNT_INC(v_pdpages); - vm_page_lock(p); - if (vm_page_wired(p) || - !pmap_page_exists_quick(pmap, p)) { - vm_page_unlock(p); + if (!pmap_page_exists_quick(pmap, p)) continue; - } act_delta = pmap_ts_referenced(p); + vm_page_lock(p); if ((p->aflags & PGA_REFERENCED) != 0) { if (act_delta == 0) act_delta = 1; @@ -234,7 +238,7 @@ vm_swapout_object_deactivate_pages(pmap_t pmap, vm p->act_count -= min(p->act_count, ACT_DECLINE); if (!remove_mode && p->act_count == 0) { - pmap_remove_all(p); + (void)vm_page_try_remove_all(p); vm_page_deactivate(p); } } else { @@ -244,7 +248,7 @@ vm_swapout_object_deactivate_pages(pmap_t pmap, vm p->act_count += ACT_ADVANCE; } } else if (vm_page_inactive(p)) - pmap_remove_all(p); + (void)vm_page_try_remove_all(p); vm_page_unlock(p); } if ((backing_object = object->backing_object) == NULL) @@ -556,9 +560,7 @@ vm_thread_swapout(struct thread *td) if (m == NULL) panic("vm_thread_swapout: kstack already missing?"); vm_page_dirty(m); - vm_page_lock(m); vm_page_unwire(m, PQ_LAUNDRY); - vm_page_unlock(m); } VM_OBJECT_WUNLOCK(ksobj); } -- Test scenario: misc/syscall4.sh