KDB: debugger backends: ddb KDB: current backend: ddb ---<>--- Copyright (c) 1992-2019 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 13.0-CURRENT #0 r351563M: Wed Aug 28 08:34:42 CEST 2019 pho@t2.osted.lan:/usr/src/sys/amd64/compile/PHO amd64 FreeBSD clang version 8.0.1 (tags/RELEASE_801/final 366581) (based on LLVM 8.0.1) WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. Entering uma_startup with 15 boot pages configured startup_alloc from "UMA Kegs", 13 boot pages left startup_alloc from "UMA Zones", 12 boot pages left startup_alloc from "UMA Zones", 11 boot pages left startup_alloc from "UMA Zones", 10 boot pages left startup_alloc from "UMA Zones", 9 boot pages left startup_alloc from "UMA Zones", 8 boot pages left startup_alloc from "UMA Zones", 7 boot pages left startup_alloc from "UMA Zones", 6 boot pages left startup_alloc from "UMA Zones", 5 boot pages left startup_alloc from "UMA Zones", 4 boot pages left startup_alloc from "UMA Zones", 3 boot pages left startup_alloc from "UMA Hash", 2 boot pages left startup_alloc from "UMA Zones", 1 boot pages left Entering uma_startup1 with 0 boot pages left Entering uma_startup2 with 0 boot pages left VT(vga): resolution 640x480 CPU: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz (1995.24-MHz K8-class CPU) Origin="GenuineIntel" Id=0x206d7 Family=0x6 Model=0x2d Stepping=7 Features=0xbfebfbff Features2=0x1fbee3ff AMD Features=0x2c100800 AMD Features2=0x1 XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID TSC: P-state invariant, performance statistics real memory = 68719476736 (65536 MB) avail memory = 66703433728 (63613 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 24 CPUs FreeBSD/SMP: 2 package(s) x 6 core(s) x 2 hardware threads random: unblocking device. Firmware Warning (ACPI): Invalid length for FADT/Pm1aControlBlock: 32, using default 16 (20190703/tbfadt-850) ioapic0 irqs 0-23 ioapic1 irqs 24-47 ioapic2 irqs 48-71 Launching APs: 23 8 7 11 15 1 6 18 16 10 9 5 20 21 22 3 4 19 14 13 2 12 17 Timecounter "TSC" frequency 1995236396 Hz quality 1000 random: entropy device external interface [ath_hal] loaded kbd1 at kbdmux0 000.000066 [4254] netmap_init netmap: loaded module module_register_init: MOD_LOAD (vesa, 0xffffffff811beab0, 0) error 19 nexus0 vtvga0: cryptosoft0: acpi0: acpi0: Power Button (fixed) cpu0: numa-domain 0 on acpi0 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Event timer "HPET4" frequency 14318180 Hz quality 340 Event timer "HPET5" frequency 14318180 Hz quality 340 Event timer "HPET6" frequency 14318180 Hz quality 340 Event timer "HPET7" frequency 14318180 Hz quality 340 atrtc0: port 0x70-0x77 irq 8 on acpi0 atrtc0: Warning: Couldn't map I/O. atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: port 0xcf8-0xcff numa-domain 0 on acpi0 pci0: numa-domain 0 on pcib0 pcib1: irq 47 at device 1.0 numa-domain 0 on pci0 pci1: numa-domain 0 on pcib1 pcib2: irq 47 at device 2.0 numa-domain 0 on pci0 pci2: numa-domain 0 on pcib2 pcib3: irq 47 at device 3.0 numa-domain 0 on pci0 pci3: numa-domain 0 on pcib3 pcib4: irq 16 at device 17.0 numa-domain 0 on pci0 pci4: numa-domain 0 on pcib4 isci0: port 0x2000-0x20ff mem 0xd7c00000-0xd7c03fff,0xd7800000-0xd7bfffff irq 16 at device 0.0 numa-domain 0 on pci4 pci0: at device 22.0 (no driver attached) pci0: at device 22.1 (no driver attached) ehci0: mem 0xd0420000-0xd04203ff irq 22 at device 26.0 numa-domain 0 on pci0 usbus0: waiting for BIOS to give up control usbus0: timed out waiting for BIOS usbus0: EHCI version 1.0 usbus0 numa-domain 0 on ehci0 usbus0: 480Mbps High Speed USB v2.0 hdac0: mem 0xd7f20000-0xd7f23fff irq 22 at device 27.0 numa-domain 0 on pci0 pcib5: irq 16 at device 28.0 numa-domain 0 on pci0 pci5: numa-domain 0 on pcib5 igb0: port 0x1020-0x103f mem 0xd0220000-0xd023ffff,0xd0250000-0xd0253fff irq 16 at device 0.0 numa-domain 0 on pci5 igb0: PHY reset is blocked due to SOL/IDER session. igb0: Using 1024 TX descriptors and 1024 RX descriptors igb0: queue equality override not set, capping rx_queues at 6 and tx_queues at 6 igb0: Using 6 RX queues 6 TX queues igb0: Using MSI-X interrupts with 7 vectors igb0: Ethernet address: 00:1e:67:56:b6:9b igb0: link state changed to UP igb0: netmap queues/slots: TX 6/1024, RX 6/1024 igb1: port 0x1000-0x101f mem 0xd0200000-0xd021ffff,0xd0240000-0xd0243fff irq 17 at device 0.1 numa-domain 0 on pci5 igb1: PHY reset is blocked due to SOL/IDER session. igb1: Using 1024 TX descriptors and 1024 RX descriptors igb1: queue equality override not set, capping rx_queues at 6 and tx_queues at 6 igb1: Using 6 RX queues 6 TX queues igb1: Using MSI-X interrupts with 7 vectors igb1: Ethernet address: 00:1e:67:56:b6:9c igb1: netmap queues/slots: TX 6/1024, RX 6/1024 pcib6: irq 17 at device 28.5 numa-domain 0 on pci0 pci6: numa-domain 0 on pcib6 pcib7: at device 0.0 numa-domain 0 on pci6 pci7: numa-domain 0 on pcib7 pci7: at device 0.0 (no driver attached) pcib8: irq 18 at device 28.6 numa-domain 0 on pci0 pci8: numa-domain 0 on pcib8 xhci0: mem 0xd0010000-0xd001ffff,0xd0000000-0xd0001fff irq 18 at device 0.0 numa-domain 0 on pci8 xhci0: 64 bytes context size, 64-bit DMA usbus1 numa-domain 0 on xhci0 usbus1: 5.0Gbps Super Speed USB v3.0 pcib9: irq 19 at device 28.7 numa-domain 0 on pci0 pci9: numa-domain 0 on pcib9 ehci1: mem 0xd0410000-0xd04103ff irq 20 at device 29.0 numa-domain 0 on pci0 usbus2: waiting for BIOS to give up control usbus2: timed out waiting for BIOS usbus2: EHCI version 1.0 usbus2 numa-domain 0 on ehci1 usbus2: 480Mbps High Speed USB v2.0 pcib10: at device 30.0 numa-domain 0 on pci0 pci10: numa-domain 0 on pcib10 isab0: at device 31.0 numa-domain 0 on pci0 isa0: numa-domain 0 on isab0 ahci0: port 0x3070-0x3077,0x3060-0x3063,0x3050-0x3057,0x3040-0x3043,0x3020-0x303f mem 0xd0400000-0xd04007ff irq 21 at device 31.2 numa-domain 0 on pci0 ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahcich4: at channel 4 on ahci0 ahcich5: at channel 5 on ahci0 ahciem0: at channel 2147483647 on ahci0 device_attach: ahciem0 attach returned 6 pcib11: numa-domain 1 on acpi0 pci11: numa-domain 1 on pcib11 pcib12: at device 0.0 numa-domain 1 on pci11 pci12: numa-domain 1 on pcib12 pcib13: irq 71 at device 1.0 numa-domain 1 on pci11 pci13: numa-domain 1 on pcib13 pcib14: irq 71 at device 2.0 numa-domain 1 on pci11 pci14: numa-domain 1 on pcib14 vgapci0: port 0xa000-0xa0ff mem 0xe0000000-0xefffffff,0xd8020000-0xd803ffff irq 56 at device 0.0 numa-domain 1 on pci14 vgapci0: Boot video device hdac1: mem 0xd8040000-0xd8043fff irq 60 at device 0.1 numa-domain 1 on pci14 pcib15: irq 71 at device 3.0 numa-domain 1 on pci11 pci15: numa-domain 1 on pcib15 acpi_syscontainer0: on acpi0 acpi_syscontainer1: on acpi0 pcib16: on acpi0 pci16: on pcib16 pci16: at device 14.1 (no driver attached) pci16: at device 19.1 (no driver attached) pci16: at device 19.4 (no driver attached) pci16: at device 19.5 (no driver attached) pcib17: on acpi0 pci17: on pcib17 pci17: at device 14.1 (no driver attached) pci17: at device 19.1 (no driver attached) pci17: at device 19.4 (no driver attached) pci17: at device 19.5 (no driver attached) uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 flags 0x10 on acpi0 uart1: console (115200,n,8,1) orm0: at iomem 0xc0000-0xcffff,0xda000-0xdafff,0xdb000-0xdbfff pnpid ORM0000 on isa0 est0: numa-domain 0 on cpu0 NULL mp in getnewvnode(9), tag crossmp Timecounters tick every 1.000 msec hdacc0: at cad 0 on hdac0 hdaa0: at nid 1 on hdacc0 pcm0: at nid 20,22,21,23 and 24,26 on hdaa0 pcm1: at nid 27 and 25 on hdaa0 pcm2: at nid 30 on hdaa0 hdacc1: at cad 0 on hdac1 hdaa1: at nid 1 on hdacc1 pcm3: at nid 3 on hdaa1 ugen0.1: at usbus0 ugen1.1: <0x104c XHCI root HUB> at usbus1 ugen2.1: at usbus2 uhub0 numa-domain 0 on usbus0 uhub0: on usbus0 uhub1 numa-domain 0 on usbus1 uhub1: <0x104c XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus1 uhub2 numa-domain 0 on usbus2 uhub2: on usbus2 ada0 at ahcich0 bus 0 scbus1 target 0 lun 0 ada0: ACS-2 ATA SATA 3.x device ada0: Serial Number CVCV317607P0240CGN ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 228936MB (468862128 512 byte sectors) cd0 at ahcich1 bus 0 scbus2 target 0 lun 0 cd0: Removable CD-ROM SCSI device cd0: Serial Number R93E68ED300PN5 cd0: 150.000MB/s transfers (SATA 1.x, UDMA5, ATAPI 12bytes, PIO 8192bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present - tray closed da0 at isci0 bus 0 scbus0 target 0 lun 0 da0: Fixed Direct Access SPC-3 SCSI device da0: Serial Number 6SL64N800000N339035N da0: 300.000MB/s transfers da0: Command Queueing enabled da0: 572325MB (1172123568 512 byte sectors) Trying to mount root from ufs:/dev/da0p2 [rw]... WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. Expensive timeout(9) function: 0xffffffff80a44520(0xffffffff81ae7430) 0.006694637 s uhub1: 4 ports with 4 removable, self powered uhub2: 2 ports with 2 removable, self powered uhub0: 2 ports with 2 removable, self powered Setting hostuuid: 2bde2bde-f4e2-e111-aab2-001e6756b69b. Setting hostid: 0x0035ff86. WARNING: reducing swap size to maximum of 65536MB per unit Starting file system checks: /dev/da0p2: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/da0p2: clean, 42173810 free (1093330 frags, 5135060 blocks, 1.1% fragmentation) /dev/ada0p2: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ada0p2: clean, 25956990 free (74110 frags, 3235360 blocks, 0.2% fragmentation) /dev/da0p3: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/da0p3: clean, 25133060 free (7860 frags, 3140650 blocks, 0.0% fragmentation) Mounting local filesystems:. ugen0.2: at usbus0 uhub3 numa-domain 0 on uhub0 uhub3: on usbus0 ugen2.2: at usbus2 uhub4 numa-domain 0 on uhub2 uhub4: on usbus2 ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat /usr/local/lib/e2fsprogs /usr/local/lib/gcc8 /usr/local/lib/mozjpeg /usr/local/lib/nss /usr/local/lib/perl5/5.28/mach/CORE /usr/local/lib/qt4 /usr/local/lib/qt5 32-bit compatibility ldconfig path: /usr/lib32 /usr/local/lib32/compat Setting hostname: t2.osted.lan. Setting up harvesting: [UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,[NET_ETHER],NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED Feeding entropy: uhub3: 6 ports with 6 removable, self powered . lo0: link state changed to UP uhub4: 8 ports with 8 removable, self powered ugen2.3: at usbus2 uhub5 numa-domain 0 on uhub4 uhub5: on usbus2 uhub5: MTT enabled uhub5: 4 ports with 4 removable, self powered ugen2.4: at usbus2 ukbd0 numa-domain 0 on uhub4 ukbd0: on usbus2 kbd2 at ukbd0 Starting Network: lo0 igb0 igb1. lo0: flags=8049 metric 0 mtu 16384 options=680003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=21 igb0: flags=8843 metric 0 mtu 1500 options=e527bb ether 00:1e:67:56:b6:9b inet 192.168.1.109 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::21e:67ff:fe56:b69b%igb0 prefixlen 64 scopeid 0x1 media: Ethernet autoselect (1000baseT ) status: active nd6 options=23 igb1: flags=8802 metric 0 mtu 1500 options=e527bb ether 00:1e:67:56:b6:9c media: Ethernet autoselect status: no carrier nd6 options=29 Starting devd. Autoloading module: ioat.ko ioat0: mem 0xd7fa0000-0xd7fa3fff irq 31 at device 4.0 numa-domain 0 on pci0 ioat0: Capabilities: 2f7 ioat1: mem 0xd7f90000-0xd7f93fff irq 39 at device 4.1 numa-domain 0 on pci0 ioat1: Capabilities: 2f7 ioat2: mem 0xd7f80000-0xd7f83fff irq 31 at device 4.2 numa-domain 0 on pci0 ioat2: Capabilities: f7 ioat3: mem 0xd7f70000-0xd7f73fff irq 39 at device 4.3 numa-domain 0 on pci0 ioat3: Capabilities: f7 ioat4: mem 0xd7f60000-0xd7f63fff irq 31 at device 4.4 numa-domain 0 on pci0 ioat4: Capabilities: f7 ioat5: mem 0xd7f50000-0xd7f53fff irq 39 at device 4.5 numa-domain 0 on pci0 ioat5: Capabilities: f7 ioat6: mem 0xd7f40000-0xd7f43fff irq 31 at device 4.6 numa-domain 0 on pci0 ioat6: Capabilities: f7 ioat7: mem 0xd7f30000-0xd7f33fff irq 39 at device 4.7 numa-domain 0 on pci0 ioat7: Capabilities: f7 ioat8: mem 0xfbf70000-0xfbf73fff irq 55 at device 4.0 numa-domain 1 on pci11 ioat8: Capabilities: 2f7 ioat9: mem 0xfbf60000-0xfbf63fff irq 63 at device 4.1 numa-domain 1 on pci11 ioat9: Capabilities: 2f7 ioat10: mem 0xfbf50000-0xfbf53fff irq 55 at device 4.2 numa-domain 1 on pci11 ioat10: Capabilities: f7 ioat11: mem 0xfbf40000-0xfbf43fff irq 63 at device 4.3 numa-domain 1 on pci11 ioat11: Capabilities: f7 ioat12: mem 0xfbf30000-0xfbf33fff irq 55 at device 4.4 numa-domain 1 on pci11 ioat12: Capabilities: f7 ioat13: mem 0xfbf20000-0xfbf23fff irq 63 at device 4.5 numa-domain 1 on pci11 ioat13: Capabilities: f7 ioat14: mem 0xfbf10000-0xfbf13fff irq 55 at device 4.6 numa-domain 1 on pci11 ioat14: Capabilities: f7 ioat15: mem 0xfbf00000-0xfbf03fff irq 63 at device 4.7 numa-domain 1 on pci11 ioat15: Capabilities: f7 Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Starting Network: igb1. igb1: flags=8802 metric 0 mtu 1500 options=e527bb ether 00:1e:67:56:b6:9c media: Ethernet autoselect status: no carrier nd6 options=29 Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Autoloading module: ioat.ko Configuring vt: keymap. Autoloading module: uhid.ko Autoloading module: ums.ko ums0 numa-domain 0 on uhub4 ums0: on usbus2 ums0: 3 buttons and [Z] coordinates ID=0 add host 127.0.0.1: gateway lo0 fib 0: route already in table add net default: gateway 192.168.1.1 add host ::1: gateway lo0 fib 0: route already in table add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Starting rtsold. Creating and/or trimming log files. Starting syslogd. Updating CPU Microcode... CPU: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz (1995.24-MHz K8-class CPU) Origin="GenuineIntel" Id=0x206d7 Family=0x6 Model=0x2d Stepping=7 Features=0xbfebfbff Features2=0x1fbee3ff AMD Features=0x2c100800 AMD Features2=0x1 Structured Extended Features3=0x9c000400 XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID TSC: P-state invariant, performance statistics Done. No core dumps found. Starting rpcbind. NFS access cache time=60 Clearing /tmp (X related). Starting nfsuserd. Starting mountd. Starting nfsd. Starting statd. Starting lockd. Recovering vi editor sessions:. Updating motd:. Mounting late filesystems:. Security policy loaded: MAC/ntpd (mac_ntpd) Starting ntpd. Starting powerd. Performing sanity check on sshd configuration. Starting sshd. Configuring vt: keymap blanktime. Starting sendmail_submit. Starting sendmail_msp_queue. Starting cron. Local package initialization:Aug 28 08:40:34 t2 su[2577]: pho to root on /dev/pts/0 backup. Starting nfscbd. Starting default mousedmoused: unable to open /dev/psm0: No such file or directory . Starting inetd. Wed Aug 28 08:40 FreeBSD/amd64 (t2.osted.lan) (ttyu1) login: 20190828 08:40:57 all (1/674): 256m.sh 20190828 08:41:01 all (2/674): fsck2.sh 20190828 08:41:05 all (3/674): holdcnt04.sh 20190828 08:41:09 all (4/674): ldt2.sh 20190828 08:41:13 all (5/674): linux.sh 20190828 08:41:16 all (6/674): mac_chkexec.sh 20190828 08:41:20 all (7/674): mac.sh 20190828 08:41:24 all (8/674): machipc.sh 20190828 08:41:28 all (9/674): machipc2.sh 20190828 08:41:32 all (10/674): md4.sh 20190828 08:41:36 all (11/674): mmap16.sh 20190828 08:41:39 all (12/674): nfs14.sh 20190828 08:41:43 all (13/674): numa.sh 20190828 08:41:47 all (14/674): oom.sh 20190828 08:41:51 all (15/674): pagefault.sh 20190828 08:41:54 all (16/674): parallelmount2.sh 20190828 08:41:58 all (17/674): pathconf2.sh 20190828 08:42:02 all (18/674): pcatch.sh 20190828 08:42:06 all (19/674): pcatch2.sh 20190828 08:42:09 all (20/674): pkru.sh 20190828 08:42:13 all (21/674): pkru2.sh 20190828 08:42:17 all (22/674): pmc6.sh 20190828 08:42:21 all (23/674): procstat.sh 20190828 08:42:25 all (24/674): quota5.sh 20190828 08:42:29 all (25/674): ruby.sh 20190828 08:42:33 all (26/674): schedfuzz.sh 20190828 08:42:36 all (27/674): segregs.sh 20190828 08:42:40 all (28/674): select3.sh 20190828 08:42:44 all (29/674): shm_super.sh 20190828 08:42:53 all (30/674): sigaltstack.sh 20190828 08:42:57 all (31/674): trim8.sh 20190828 08:43:00 all (32/674): open.sh 20190828 08:43:06 all (33/674): random.sh 20190828 08:43:12 all (34/674): 1st.sh 20190828 08:43:17 all (35/674): audit.sh 20190828 08:43:23 all (36/674): beneath2.sh lock order reversal: 1st 0xfffff8003b30a558 ufs (ufs) @ kern/vfs_mount.c:1354 2nd 0xfffff8009d827558 devfs (devfs) @ ufs/ffs/ffs_vfsops.c:1403 stack backtrace: #0 0xffffffff80c41713 at witness_debugger+0x73 #1 0xffffffff80c4145e at witness_checkorder+0xa9e #2 0xffffffff80ba73d9 at lockmgr_lock_fast_path+0x179 #3 0xffffffff81234a40 at VOP_LOCK1_APV+0x70 #4 0xffffffff80cbf155 at _vn_lock+0x65 #5 0xffffffff80ee76c3 at ffs_flushfiles+0xa3 #6 0xffffffff80ec9963 at softdep_flushfiles+0x73 #7 0xffffffff80ee9e21 at ffs_unmount+0x71 #8 0xffffffff80ca4f80 at dounmount+0x4e0 #9 0xffffffff80ca49e9 at sys_unmount+0x339 #10 0xffffffff81197254 at amd64_syscall+0x2d4 #11 0xffffffff8116ec10 at fast_syscall_common+0x101 mount leak: 1/65 20190828 08:43:29 all (37/674): beneath3.sh mount leak: 1/66 20190828 08:43:35 all (38/674): buildworld.sh 20190828 08:43:41 all (39/674): buildworld2.sh 20190828 08:43:47 all (40/674): datagram2.sh 20190828 08:43:53 all (41/674): datagram3.sh 20190828 08:43:59 all (42/674): datamove.sh 20190828 08:44:05 all (43/674): datamove3.sh 20190828 08:44:12 all (44/674): devfs2.sh 20190828 08:44:18 all (45/674): devfs5.sh 20190828 08:44:23 all (46/674): dtrace_fault.sh 20190828 08:44:29 all (47/674): dumpfs.sh newblk leak: 277/313. mount leak: 1/67 20190828 08:44:36 all (48/674): dup2.sh 20190828 08:44:42 all (49/674): execi386.sh 20190828 08:44:48 all (50/674): extattrctl.sh 20190828 08:44:54 all (51/674): fifo.sh mount leak: 1/68 20190828 08:45:00 all (52/674): forkbomb.sh 20190828 08:45:05 all (53/674): getrandom.sh 20190828 08:45:11 all (54/674): isofs2.sh mount leak: 1/69 20190828 08:45:17 all (55/674): jail.sh 20190828 08:45:23 all (56/674): jail2.sh 20190828 08:45:29 all (57/674): jail3.sh 20190828 08:45:35 all (58/674): jail4.sh 20190828 08:45:41 all (59/674): kern_umtx_inf_loop.sh 20190828 08:45:47 all (60/674): kevent10.sh 20190828 08:45:53 all (61/674): ldt.sh kernel trap 9 with interrupts disabled kernel trap 9 with interrupts disabled kernel trap 9 with interrupts disabled kernel trap 9 with interrupts disabled kernel trap 9 with interrupts disabled 20190828 08:45:59 all (62/674): maxmemdom.sh 20190828 08:46:05 all (63/674): maxproc.sh 20190828 08:46:11 all (64/674): md6.sh 20190828 08:46:17 all (65/674): mdconfig.sh 20190828 08:46:23 all (66/674): memguard4.sh 20190828 08:46:28 all (67/674): mkfifo5.sh mount leak: 1/70 20190828 08:46:34 all (68/674): mkfifo7.sh mount leak: 17/87 20190828 08:46:41 all (69/674): mlockall5.sh 20190828 08:46:46 all (70/674): mmap.sh 20190828 08:46:52 all (71/674): mmap12.sh 20190828 08:46:58 all (72/674): mmap17.sh 20190828 08:47:04 all (73/674): mmap20.sh 20190828 08:47:10 all (74/674): mmap24.sh 20190828 08:47:17 all (75/674): mmap30.sh 20190828 08:47:23 all (76/674): mmap33.sh 20190828 08:47:28 all (77/674): mmap35.sh 20190828 08:47:34 all (78/674): mmap36.sh 20190828 08:47:40 all (79/674): mount.sh 20190828 08:47:46 all (80/674): mprotect.sh 20190828 08:47:52 all (81/674): msdos2.sh 20190828 08:47:58 all (82/674): msdos3.sh 20190828 08:48:04 all (83/674): msdos7.sh lock order reversal: 1st 0xfffff8009d852558 msdosfs (msdosfs) @ kern/vfs_syscalls.c:3424 2nd 0xfffff8009d848a48 devfs (devfs) @ fs/msdosfs/msdosfs_vnops.c:854 stack backtrace: #0 0xffffffff80c41713 at witness_debugger+0x73 #1 0xffffffff80c4145e at witness_checkorder+0xa9e #2 0xffffffff80ba73d9 at lockmgr_lock_fast_path+0x179 #3 0xffffffff81234a40 at VOP_LOCK1_APV+0x70 #4 0xffffffff80cbf155 at _vn_lock+0x65 #5 0xffffffff80a94899 at msdosfs_fsync+0x49 #6 0xffffffff81233878 at VOP_FSYNC_APV+0x88 #7 0xffffffff80cbb0d9 at kern_fsync+0x199 #8 0xffffffff81197254 at amd64_syscall+0x2d4 #9 0xffffffff8116ec10 at fast_syscall_common+0x101 20190828 08:48:10 all (84/674): msdos8.sh 20190828 08:48:16 all (85/674): msetdomain.sh 20190828 08:48:22 all (86/674): nanosleep.sh 20190828 08:48:28 all (87/674): nfs13.sh 20190828 08:48:34 all (88/674): nullfs10.sh 20190828 08:48:41 all (89/674): nullfs13.sh 20190828 08:48:47 all (90/674): nullfs14.sh 20190828 08:48:53 all (91/674): nullfs15.sh 20190828 08:48:59 all (92/674): nullfs21.sh 20190828 08:49:05 all (93/674): nullfs4.sh 20190828 08:49:11 all (94/674): nullfs6.sh 20190828 08:49:17 all (95/674): nullfs7.sh 20190828 08:49:23 all (96/674): nullfs9.sh 20190828 08:49:29 all (97/674): pmc4.sh hwpmc: SOFT/16/64/0x67 TSC/1/64/0x20 IAP/4/48/0x3ff IAF/3/48/0x67 20190828 08:49:36 all (98/674): pmc5.sh hwpmc: SOFT/16/64/0x67 TSC/1/64/0x20 IAP/4/48/0x3ff IAF/3/48/0x67 20190828 08:49:42 all (99/674): posix_fadvise.sh 20190828 08:49:48 all (100/674): posix_fadvise2.sh 20190828 08:49:54 all (101/674): procfs5.sh 20190828 08:50:00 all (102/674): ptrace.sh 20190828 08:50:06 all (103/674): ptrace3.sh 20190828 08:50:12 all (104/674): ptrace8.sh 20190828 08:50:19 all (105/674): pts2.sh 20190828 08:50:25 all (106/674): rdgsbase.sh 20190828 08:50:31 all (107/674): rename13.sh 20190828 08:50:37 all (108/674): rename4.sh 20190828 08:50:43 all (109/674): revoke.sh 20190828 08:50:49 all (110/674): rwlock_ronly.sh 20190828 08:50:55 all (111/674): seekdir.sh 20190828 08:51:01 all (112/674): segnp.sh 20190828 08:51:08 all (113/674): sendfile4.sh lock order reversal: 1st 0xfffff8087df14278 so_snd_sx (so_snd_sx) @ kern/uipc_sockbuf.c:390 2nd 0xfffff8009d85ea48 ufs (ufs) @ kern/kern_sendfile.c:736 stack backtrace: #0 0xffffffff80c41713 at witness_debugger+0x73 #1 0xffffffff80c4145e at witness_checkorder+0xa9e #2 0xffffffff80ba758f at lockmgr_slock_hard+0x6f #3 0xffffffff80ba8728 at __lockmgr_args+0x788 #4 0xffffffff80eebf6c at ffs_lock+0x7c #5 0xffffffff81234a40 at VOP_LOCK1_APV+0x70 #6 0xffffffff80cbf155 at _vn_lock+0x65 #7 0xffffffff80bd1af0 at vn_sendfile+0x6c0 #8 0xffffffff80bd33d7 at sendfile+0x127 #9 0xffffffff81197254 at amd64_syscall+0x2d4 #10 0xffffffff8116ec10 at fast_syscall_common+0x101 20190828 08:51:14 all (114/674): sendfile6.sh mount leak: 1/88 20190828 08:51:20 all (115/674): setuid.sh mount leak: 1/89 20190828 08:51:26 all (116/674): sigreturn.sh 20190828 08:51:33 all (117/674): sigstop.sh 20190828 08:51:39 all (118/674): snap12.sh lock order reversal: 1st 0xfffff8003b30a558 ufs (ufs) @ kern/vfs_mount.c:1354 2nd 0xfffff809021c3530 snaplk (snaplk) @ ufs/ffs/ffs_snapshot.c:2132 3rd 0xfffff8009d864a48 ufs (ufs) @ ufs/ffs/ffs_snapshot.c:2133 stack backtrace: #0 0xffffffff80c41713 at witness_debugger+0x73 #1 0xffffffff80c4145e at witness_checkorder+0xa9e #2 0xffffffff80ba79ed at lockmgr_xlock_hard+0x6d #3 0xffffffff80ba85a5 at __lockmgr_args+0x605 #4 0xffffffff80ec80b9 at ffs_snapshot_unmount+0x109 #5 0xffffffff80ee7699 at ffs_flushfiles+0x79 #6 0xffffffff80ec9963 at softdep_flushfiles+0x73 #7 0xffffffff80ee9e21 at ffs_unmount+0x71 #8 0xffffffff80ca4f80 at dounmount+0x4e0 #9 0xffffffff80ca49e9 at sys_unmount+0x339 #10 0xffffffff81197254 at amd64_syscall+0x2d4 #11 0xffffffff8116ec10 at fast_syscall_common+0x101 mount leak: 1/90 20190828 08:51:45 all (119/674): sndstat.sh 20190828 08:51:51 all (120/674): socketpair.sh 20190828 08:51:57 all (121/674): socketpair2.sh 20190828 08:52:03 all (122/674): suj2.sh mount leak: 1/91 20190828 08:52:09 all (123/674): suj35.sh Aug 28 08:52:13 t2 kernel: pid 1301 (mkdir), uid 0 inumber 255 on /mnt: out of inodes mount leak: 1/92 20190828 08:52:15 all (124/674): suj7.sh Failed to find journal. Use tunefs to create one Failed to start journal: 2 mount leak: 1/93 20190828 08:52:21 all (125/674): tmpfs15.sh mount leak: 1/94 20190828 08:52:27 all (126/674): tmpfs18.sh mount leak: 1/95 20190828 08:52:33 all (127/674): tmpfs4.sh mount leak: 1/96 20190828 08:52:40 all (128/674): tmpfs6.sh mount leak: 1/97 20190828 08:52:47 all (129/674): tmpfs7.sh lock order reversal: 1st 0xfffff808836bf988 so_snd_sx (so_snd_sx) @ kern/uipc_sockbuf.c:390 2nd 0xfffff8090043f7d0 tmpfs (tmpfs) @ kern/kern_sendfile.c:736 stack backtrace: #0 0xffffffff80c41713 at witness_debugger+0x73 #1 0xffffffff80c4145e at witness_checkorder+0xa9e #2 0xffffffff80ba72de at lockmgr_lock_fast_path+0x7e #3 0xffffffff81234a40 at VOP_LOCK1_APV+0x70 #4 0xffffffff80cbf155 at _vn_lock+0x65 #5 0xffffffff80bd1af0 at vn_sendfile+0x6c0 #6 0xffffffff80bd33d7 at sendfile+0x127 #7 0xffffffff81197254 at amd64_syscall+0x2d4 #8 0xffffffff8116ec10 at fast_syscall_common+0x101 mount leak: 1/98 20190828 08:52:53 all (130/674): trim5.sh mount leak: 1/99 20190828 08:52:59 all (131/674): truncate4.sh mount leak: 2/101 20190828 08:53:05 all (132/674): truncate6.sh mount leak: 1/102 20190828 08:53:11 all (133/674): uma_zalloc_arg.sh 20190828 08:53:17 all (134/674): unlink.sh 20190828 08:53:23 all (135/674): unload.sh 20190828 08:53:29 all (136/674): unload2.sh 20190828 08:53:35 all (137/674): vfork.sh 20190828 08:53:41 all (138/674): vmio.sh 20190828 08:53:47 all (139/674): watchman.sh 20190828 08:53:53 all (140/674): datagram.sh 20190828 08:54:00 all (141/674): extattr_set_fd.sh mount leak: 1/103 20190828 08:54:06 all (142/674): fsck3.sh 20190828 08:54:13 all (143/674): geomleak.sh 20190828 08:54:19 all (144/674): gnop5.sh GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. mount leak: 4/107 20190828 08:54:26 all (145/674): kevent3.sh 20190828 08:54:33 all (146/674): killpg.sh 20190828 08:54:40 all (147/674): lockf.sh 20190828 09:01:04 all (148/674): md5.sh 20190828 09:01:11 all (149/674): md9.sh lock order reversal: 1st 0xfffff8087d968880 bufwait (bufwait) @ vm/vm_pager.c:379 2nd 0xfffff80900416068 tmpfs (tmpfs) @ dev/md/md.c:990 stack backtrace: #0 0xffffffff80c41713 at witness_debugger+0x73 #1 0xffffffff80c4145e at witness_checkorder+0xa9e #2 0xffffffff80ba73d9 at lockmgr_lock_fast_path+0x179 #3 0xffffffff81234a40 at VOP_LOCK1_APV+0x70 #4 0xffffffff80cbf155 at _vn_lock+0x65 #5 0xffffffff806f34ca at mdstart_vnode+0x47a #6 0xffffffff806f11bc at md_kthread+0x20c #7 0xffffffff80b92794 at fork_exit+0x84 #8 0xffffffff8116f32e at fork_trampoline+0xe mount leak: 1/108 20190828 09:01:18 all (150/674): mdconfig3.sh 20190828 09:01:25 all (151/674): mmap25.sh 20190828 09:01:32 all (152/674): mmap8.sh 20190828 09:01:39 all (153/674): mmap9.sh 20190828 09:01:46 all (154/674): mountro3.sh newblk leak: 1733/2046. mount leak: 1/109 20190828 09:01:53 all (155/674): nfsdelegation.sh 20190828 09:02:00 all (156/674): nfssillyrename.sh 20190828 09:03:24 all (157/674): nullfs12.sh mount leak: 1/110 20190828 09:03:31 all (158/674): nullfs16.sh mount leak: 1/111 20190828 09:03:39 all (159/674): openlock.sh 20190828 09:03:46 all (160/674): sendfile.sh 20190828 09:03:52 all (161/674): sendfile2.sh lock order reversal: 1st 0xfffff80901a8a490 filedesc structure (filedesc structure) @ kern/sys_generic.c:1519 2nd 0xfffff809003d17d0 ufs (ufs) @ kern/vfs_vnops.c:1543 stack backtrace: #0 0xffffffff80c41713 at witness_debugger+0x73 #1 0xffffffff80c4145e at witness_checkorder+0xa9e #2 0xffffffff80ba79ed at lockmgr_xlock_hard+0x6d #3 0xffffffff80ba85a5 at __lockmgr_args+0x605 #4 0xffffffff80eebf6c at ffs_lock+0x7c #5 0xffffffff81234a40 at VOP_LOCK1_APV+0x70 #6 0xffffffff80cbf155 at _vn_lock+0x65 #7 0xffffffff80cbdf8a at vn_poll+0x3a #8 0xffffffff80c47c79 at kern_poll+0x3d9 #9 0xffffffff80c47890 at sys_poll+0x50 #10 0xffffffff81197254 at amd64_syscall+0x2d4 #11 0xffffffff8116ec10 at fast_syscall_common+0x101 20190828 09:04:25 all (162/674): snap10.sh lock order reversal: 1st 0xfffff8000909d480 bufwait (bufwait) @ vm/vm_pager.c:379 2nd 0xfffff809021c3530 snaplk (snaplk) @ dev/md/md.c:990 stack backtrace: #0 0xffffffff80c41713 at witness_debugger+0x73 #1 0xffffffff80c4145e at witness_checkorder+0xa9e #2 0xffffffff80ba79ed at lockmgr_xlock_hard+0x6d #3 0xffffffff80ba85a5 at __lockmgr_args+0x605 #4 0xffffffff80eebf6c at ffs_lock+0x7c #5 0xffffffff81234a40 at VOP_LOCK1_APV+0x70 #6 0xffffffff80cbf155 at _vn_lock+0x65 #7 0xffffffff806f34ca at mdstart_vnode+0x47a #8 0xffffffff806f11bc at md_kthread+0x20c #9 0xffffffff80b92794 at fork_exit+0x84 #10 0xffffffff8116f32e at fork_trampoline+0xe mount leak: 1/112 20190828 09:04:32 all (163/674): suj8.sh Aug 28 09:04:35 t2 kernel: pid 87648 (dd), uid 0 inumber 4 on /mnt: filesystem full Aug 28 09:04:36 t2 kernel: pid 87664 (dd), uid 0 inumber 5 on /mnt: filesystem full mount leak: 2/114 20190828 09:04:39 all (164/674): trim2.sh mount leak: 1/115 20190828 09:04:46 all (165/674): umtx_suspend.sh 20190828 09:04:53 all (166/674): vm_fault_dontneed.sh 20190828 09:05:00 all (167/674): contigmalloc3.sh 20190828 09:05:21 all (168/674): ext2fs3.sh mount leak: 1/116 20190828 09:05:28 all (169/674): fpclone2.sh 20190828 09:05:37 all (170/674): isofs.sh mount leak: 1/117 20190828 09:05:45 all (171/674): mmap23.sh 20190828 09:05:53 all (172/674): mmap26.sh 20190828 09:06:00 all (173/674): mmap28.sh lock order reversal: 1st 0xfffff80003f197d0 syncer (syncer) @ kern/vfs_subr.c:2317 2nd 0xfffff801121f4a48 ufs (ufs) @ kern/vfs_subr.c:2737 stack backtrace: #0 0xffffffff80c41713 at witness_debugger+0x73 #1 0xffffffff80c4145e at witness_checkorder+0xa9e #2 0xffffffff80ba79ed at lockmgr_xlock_hard+0x6d #3 0xffffffff80ba85a5 at __lockmgr_args+0x605 #4 0xffffffff80eebf6c at ffs_lock+0x7c #5 0xffffffff81234a40 at VOP_LOCK1_APV+0x70 #6 0xffffffff80cbf155 at _vn_lock+0x65 #7 0xffffffff80caca66 at vget+0xa6 #8 0xffffffff80caed06 at vfs_msync+0xa6 #9 0xffffffff80cb3f95 at sync_fsync+0xc5 #10 0xffffffff81233878 at VOP_FSYNC_APV+0x88 #11 0xffffffff80cb1de4 at sched_sync+0x284 #12 0xffffffff80b92794 at fork_exit+0x84 #13 0xffffffff8116f32e at fork_trampoline+0xe Aug 28 09:06:05 t2 kernel: Failed to fully fault in a core file segment at VA 0x800e00000 with size 0x1199000 to be written at offset 0x86d000 for process mmap28 20190828 09:06:08 all (174/674): mmap4.sh vnode_pager_putpages: I/O error 28 0xfffff80d02737768: tag ufs, type VREG usecount 0, writecount 0, refcount 3203 flags (VI_ACTIVE|VI_DOINGINACT) v_object 0xfffff806c943f300 ref 0 pages 25608 cleanbuf 3200 dirtybuf 1 lock type ufs: EXCL by thread 0xfffff800092a9000 (pid 232, mmap4, tid 100328) #0 0xffffffff80ba7db2 at lockmgr_xlock_hard+0x432 #1 0xffffffff80ba85a5 at __lockmgr_args+0x605 Aug 28 09:06:11 #2 0xffffffff80eebf6c at ffs_lock+0x7c t2 kernel: pid 2#3 0xffffffff81234a40 at VOP_LOCK1_APV+0x70 32 (mmap4), uid #4 0xffffffff80cbf155 at _vn_lock+0x65 0 inumber 4 on /#5 0xffffffff80cad3c1 at vputx+0x221 mnt: filesystem #6 0xffffffff80f1ea3a at vm_object_deallocate+0x58a full #7 0xffffffff80f1289f at vm_map_process_deferred+0x7f #8 0xffffffff80f184a6 at vm_map_remove+0xc6 #9 0xffffffff80f12373 at vmspace_exit+0xd3 #10 0xffffffff80b8d32d at exit1+0x59d #11 0xffffffff80b8cd8d at sys_sys_exit+0xd #12 0xffffffff81197254 at amd64_syscall+0x2d4 #13 0xffffffff8116ec10 at fast_syscall_common+0x101 nlink=1, effnlink=1, size=104857600, extsize 0 generation=6453d915, uid=0, gid=0, flags=0x0 ino 4, on dev md10a vnode_pager_putpages: residual I/O 32768 at 9760 0xfffff80d02737768: tag ufs, type VREG usecount 0, writecount 0, refcount 3203 flags (VI_ACTIVE|VI_DOINGINACT) v_object 0xfffff806c943f300 ref 0 pages 25608 cleanbuf 3200 dirtybuf 1 lock type ufs: EXCL by thread 0xfffff800092a9000 (pid 232, mmap4, tid 100328) #0 0xffffffff80ba7db2 at lockmgr_xlock_hard+0x432 #1 0xffffffff80ba85a5 at __lockmgr_args+0x605 #2 0xffffffff80eebf6c at ffs_lock+0x7c #3 0xffffffff81234a40 at VOP_LOCK1_APV+0x70 #4 0xffffffff80cbf155 at _vn_lock+0x65 #5 0xffffffff80cad3c1 at vputx+0x221 #6 0xffffffff80f1ea3a at vm_object_deallocate+0x58a #7 0xffffffff80f1289f at vm_map_process_deferred+0x7f #8 0xffffffff80f184a6 at vm_map_remove+0xc6 #9 0xffffffff80f12373 at vmspace_exit+0xd3 #10 0xffffffff80b8d32d at exit1+0x59d #11 0xffffffff80b8cd8d at sys_sys_exit+0xd #12 0xffffffff81197254 at amd64_syscall+0x2d4 #13 0xffffffff8116ec10 at fast_syscall_common+0x101 nlink=1, effnlink=1, size=104857600, extsize 0 generation=6453d915, uid=0, gid=0, flags=0x0 ino 4, on dev md10a vnode_pager_putpages: zero-length write at 40075264 resid 131072 0xfffff80d02737768: tag ufs, type VREG usecount 1, writecount 0, refcount 3203 flags (VI_ACTIVE) v_object 0xfffff806c943f300 ref 0 pages 25608 cleanbuf 3201 dirtybuf 0 lock type ufs: EXCL by thread 0xfffff8009e3dd5a0 (pid 240, umount, tid 100617) #0 0xffffffff80ba7db2 at lockmgr_xlock_hard+0x432 #1 0xffffffff80ba85a5 at __lockmgr_args+0x605 #2 0xffffffff80eebf6c at ffs_lock+0x7c #3 0xffffffff81234a40 at VOP_LOCK1_APV+0x70 #4 0xffffffff80cbf155 at _vn_lock+0x65 #5 0xffffffff80caca66 at vget+0xa6 #6 0xffffffff80caed06 at vfs_msync+0xa6 #7 0xffffffff80ca4f00 at dounmount+0x460 Aug 28 09:06:13 #8 0xffffffff80ca49e9 at sys_unmount+0x339 t2 kernel: pid 2#9 0xffffffff81197254 at amd64_syscall+0x2d4 40 (umount), uid#10 0xffffffff8116ec10 at fast_syscall_common+0x101 nlink=1, effnlink=1, size=104857600, extsize 0 generation=6453d915, uid=0, gid=0, flags=0x0 ino 4, on dev md10a 0 inumber 4 on /mnt: filesystem full mount leak: 1/118 20190828 09:06:16 all (175/674): nullfs3.sh mount leak: 1/119 20190828 09:06:24 all (176/674): procfs2.sh lock order reversal: (sleepable after non-sleepable) 1st 0xfffff8000761d688 pfs_node (pfs_node) @ fs/pseudofs/pseudofs_internal.h:105 2nd 0xfffffe0006d5f560 pidhash (pidhash) @ kern/kern_proc.c:429 stack backtrace: #0 0xffffffff80c41713 at witness_debugger+0x73 #1 0xffffffff80c4145e at witness_checkorder+0xa9e #2 0xffffffff80bdef88 at _sx_slock_int+0x68 #3 0xffffffff80bb8c3e at pfind+0x6e #4 0xffffffff80b0d7d9 at pfs_readdir+0x149 #5 0xffffffff81234458 at VOP_READDIR_APV+0x88 #6 0xffffffff80cbc16c at kern_getdirentries+0x21c #7 0xffffffff80cbc399 at sys_getdirentries+0x29 #8 0xffffffff81197254 at amd64_syscall+0x2d4 #9 0xffffffff8116ec10 at fast_syscall_common+0x101 20190828 09:06:32 all (177/674): ptrace11.sh 20190828 09:06:39 all (178/674): rename6.sh mount leak: 1/120 20190828 09:06:47 all (179/674): spin.sh mount leak: 1/121 20190828 09:06:54 all (180/674): suj32.sh mount leak: 1/122 20190828 09:07:01 all (181/674): truncate5.sh mount leak: 1/123 20190828 09:07:09 all (182/674): unix_socket.sh 20190828 09:07:17 all (183/674): contigmalloc2.sh 20190828 09:07:27 all (184/674): datamove4.sh mount leak: 1/124 20190828 09:07:37 all (185/674): extattr.sh mount leak: 1/125 20190828 09:07:46 all (186/674): fcntl.sh 20190828 09:07:55 all (187/674): mkfifo6.sh mount leak: 1/126 20190828 09:08:04 all (188/674): mountro2.sh mount leak: 1/127 20190828 09:08:12 all (189/674): mountro4.sh mount leak: 3/130 20190828 09:08:21 all (190/674): msdos9.sh 20190828 09:08:30 all (191/674): newfs5.sh mount leak: 1/131 20190828 09:08:39 all (192/674): kevent2.sh 20190828 09:08:48 all (193/674): kevent8.sh mount leak: 1/132 20190828 09:08:57 all (194/674): quota9.sh lock order reversal: 1st 0xfffff8003b554688 FFS (FFS Lock) @ ufs/ufs/ufs_quota.c:759 2nd 0xfffff80902715000 struct mount mtx (struct mount mtx) @ ufs/ufs/ufs_quota.c:765 stack backtrace: #0 0xffffffff80c41713 at witness_debugger+0x73 #1 0xffffffff80c4145e at witness_checkorder+0xa9e #2 0xffffffff80bb32b5 at __mtx_lock_flags+0x95 #3 0xffffffff80ef764d at quotaoff_inchange+0xbd #4 0xffffffff80ee774d at ffs_flushfiles+0x12d #5 0xffffffff80ec9963 at softdep_flushfiles+0x73 #6 0xffffffff80ee9e21 at ffs_unmount+0x71 #7 0xffffffff80ca4f80 at dounmount+0x4e0 #8 0xffffffff80ca49e9 at sys_unmount+0x339 #9 0xffffffff81197254 at amd64_syscall+0x2d4 #10 0xffffffff8116ec10 at fast_syscall_common+0x101 mount leak: 1/133 20190828 09:09:07 all (195/674): tmpfs5.sh mount leak: 1/134 20190828 09:09:17 all (196/674): devfd.sh mount leak: 1/135 20190828 09:09:30 all (197/674): fork.sh 20190828 09:09:41 all (198/674): mlockall4.sh mount leak: 1/136 20190828 09:09:52 all (199/674): mmap13.sh 20190828 09:10:03 all (200/674): ptrace2.sh 20190828 09:10:14 all (201/674): ptrace9.sh 20190828 09:10:25 all (202/674): snap2-1.sh mount leak: 1/137 20190828 09:10:36 all (203/674): snap2.sh mount leak: 1/138 20190828 09:10:46 all (204/674): symlink2.sh mount leak: 2/140 20190828 09:10:59 all (205/674): truncate7.sh mount leak: 1/141 20190828 09:11:12 all (206/674): nfs8.sh 20190828 09:11:31 all (207/674): proccontrol.sh 20190828 09:11:44 all (208/674): pshared.sh 20190828 09:11:58 all (209/674): symlink4.sh mount leak: 1/142 20190828 09:12:11 all (210/674): truncate3.sh mount leak: 1/143 20190828 09:12:23 all (211/674): linger2.sh Aug 28 09:12:31 t2 kernel: pid 57545 (linger2), uid 1004 inumber 890 on /mnt: filesystem full mount leak: 1/144 20190828 09:12:37 all (212/674): pthread.sh 20190828 09:12:46 all (213/674): dup.sh 20190828 09:12:58 all (214/674): execve.sh 20190828 09:13:12 all (215/674): linger3.sh mount leak: 1/145 20190828 09:13:27 all (216/674): shm_open.sh 20190828 09:13:43 all (217/674): sparse.sh mount leak: 1/146 20190828 09:14:03 all (218/674): thr.sh witness_lock_list_get: witness exhausted 20190828 09:14:18 all (219/674): linger4.sh Aug 28 09:14:26 t2 kernel: pid 78998 (linger4), uid 1004 inumber 18818 on /mnt: out of inodes mount leak: 1/147 20190828 09:14:33 all (220/674): mountu2.sh mount leak: 3/150 20190828 09:16:30 all (221/674): nfs15lockd3.sh 20190828 09:16:40 all (222/674): rename11.sh mount leak: 1/151 20190828 09:16:49 all (223/674): sysctl.sh 20190828 09:17:03 all (224/674): growfs.sh g_access(997): provider ufsid/5d662a72f227264d has error 6 set g_access(997): provider ufsid/5d662a72f227264d has error 6 set g_access(997): provider ufsid/5d662a72f227264d has error 6 set g_access(997): provider gptid/d73a396d-c963-11e9-9e73-001e6756b69b has error 6 set g_access(997): provider gptid/d73a396d-c963-11e9-9e73-001e6756b69b has error 6 set g_access(997): provider gptid/d73a396d-c963-11e9-9e73-001e6756b69b has error 6 set g_dev_taste: make_dev_p() failed (gp->name=gptid/d73a396d-c963-11e9-9e73-001e6756b69b, error=17) g_dev_taste: make_dev_p() failed (gp->name=gptid/d73a396d-c963-11e9-9e73-001e6756b69b, error=17) g_dev_taste: make_dev_p() failed (gp->name=ufsid/5d662a72f227264d, error=17) g_dev_taste: make_dev_p() failed (gp->name=gptid/d73a396d-c963-11e9-9e73-001e6756b69b, error=17) g_dev_taste: make_dev_p() failed (gp->name=ufsid/5d662a72f227264d, error=17) g_dev_taste: make_dev_p() failed (gp->name=gptid/d73a396d-c963-11e9-9e73-001e6756b69b, error=17) g_dev_taste: make_dev_p() failed (gp->name=ufsid/5d662a72f227264d, error=17) mount leak: 1/152 GEOM leak: 14/174 20190828 09:17:17 all (225/674): tmpfs14.sh mount leak: 1/153 20190828 09:17:32 all (226/674): pipe2.sh 20190828 09:17:47 all (227/674): umountf11.sh mount leak: 1/154 20190828 09:18:03 all (228/674): umountf12.sh mount leak: 1/155 20190828 09:18:19 all (229/674): compare.sh 20190828 09:18:37 all (230/674): quota11.sh mount leak: 1/156 20190828 09:18:54 all (231/674): tmpfs10.sh mount leak: 1/157 20190828 09:19:03 all (232/674): procfs.sh 20190828 09:19:13 all (233/674): sendfile3.sh freework leak: 196/277. newblk leak: 65195/67241. 20190828 09:20:12 all (234/674): scp.sh 20190828 09:21:47 all (235/674): sendfile12.sh mount leak: 1/158 20190828 09:22:04 all (236/674): datamove5.sh mount leak: 1/159 20190828 09:22:16 all (237/674): rename8.sh mount leak: 1/160 20190828 09:22:38 all (238/674): tmpfs8.sh mount leak: 1/161 20190828 09:22:57 all (239/674): multicast2.sh Aug 28 09:23:01 t2 mDNSResponder[70389]: mDNSResponder (Engineering Build) (Jan 5 2019 05:09:39) starting Aug 28 09:23:01 t2 mDNSResponder[70389]: mDNS_AddDNSServer: Lock not held! mDNS_busy (0) mDNS_reentrancy (0) if_delmulti_locked: detaching ifnet instance 0xfffff8013009a800 if_delmulti_locked: detaching ifnet instance 0xfffff8013009a800 if_delmulti_locked: detaching ifnet instance 0xfffff8013009a800 if_delmulti_locked: detaching ifnet instance 0xfffff8013009a800 if_delmulti_locked: detaching ifnet instance 0xfffff8013009a800 if_delmulti_locked: detaching ifnet instance 0xfffff8013009a800 if_delmulti_locked: detaching ifnet instance 0xfffff8013009a800 Aug 28 09:23:08 t2 mDNSResponder[70389]: mDNSResponder (Engineering Build) (Jan 5 2019 05:09:39) stopping if_delmulti_locked: detaching ifnet instance 0xfffff80848c17000 20190828 09:23:13 all (240/674): fsck5.sh mount leak: 1/162 20190828 09:23:37 all (241/674): mountu.sh mount leak: 3/165 20190828 09:24:04 all (242/674): mprotect2.sh 20190828 09:24:26 all (243/674): procfs3.sh 20190828 09:24:54 all (244/674): rename9.sh mount leak: 1/166 20190828 09:25:21 all (245/674): sem.sh 20190828 09:25:43 all (246/674): core4.sh Aug 28 09:25:48 t2 kernel: Failed to write core file for process core4 (error 5) mount leak: 1/167 20190828 09:26:10 all (247/674): mountro6.sh Aug 28 09:26:09 t2 syslogd: last message repeated 59 times mount leak: 1/168 20190828 09:26:34 all (248/674): mmap19.sh 20190828 09:30:04 all (249/674): ufssuspend.sh 20190828 09:30:08 all (250/674): bug131876.sh 20190828 09:38:33 all (251/674): ffs_blkfree.sh mount leak: 11/179 20190828 09:38:59 all (252/674): pread.sh mount leak: 3/182 20190828 09:39:32 all (253/674): snap.sh mount leak: 2/184 20190828 09:40:00 all (254/674): suj10.sh mount leak: 1/185 20190828 09:40:24 all (255/674): recursiveflushes.sh mount leak: 1/186 20190828 09:40:52 all (256/674): multicast.sh 20190828 09:41:22 all (257/674): credleak.sh mount leak: 162/348 20190828 09:41:52 all (258/674): rdwr.sh 20190828 09:42:23 all (259/674): mmap37.sh mount leak: 1/349 20190828 09:42:57 all (260/674): pthread5.sh 20190828 09:43:32 all (261/674): sendfile_shm.sh 20190828 09:43:56 all (262/674): tcp4.sh Expensive timeout(9) function: 0xffffffff80e09180(0xfffff808ad36d7a0) 0.006809378 s Expensive timeout(9) function: 0xffffffff80e09180(0xfffff8009dfbcb70) 0.092760924 s Expensive timeout(9) function: 0xffffffff80e09180(0xfffff8009ed1f3d0) 0.186949753 s Expensive timeout(9) function: 0xffffffff80e08220(0xfffff80ec6bd3000) 0.368934321 s Expensive timeout(9) function: 0xffffffff80e09180(0xfffff80fd68503d0) 0.388779718 s 20190828 09:44:32 all (263/674): fsck4.sh mount leak: 1/350 20190828 09:45:13 all (264/674): rename7.sh mount leak: 1/351 20190828 09:46:00 all (265/674): umountf5.sh mount leak: 1/352 20190828 09:46:36 all (266/674): mkfifo3.sh 20190828 09:47:19 all (267/674): nfs12.sh 20190828 09:48:04 all (268/674): sendfile5.sh mount leak: 3/355 20190828 09:48:44 all (269/674): radix.sh Aug 28 09:48:54 t2 kernel: pid 21258 (radix), jid 0, uid 0, was killed: out of swap space 20190828 09:49:07 all (270/674): midistat.sh 20190828 09:50:14 all (271/674): namecache2.sh mount leak: 1/356 20190828 09:51:15 all (272/674): routetbl.sh mount leak: 2/358 20190828 09:52:19 all (273/674): fsgs.sh 20190828 09:53:22 all (274/674): fullpath.sh 20190828 09:54:26 all (275/674): libMicro.sh 20190828 09:55:30 all (276/674): mdconfig2.sh 20190828 09:56:33 all (277/674): mknod.sh mount leak: 1/359 20190828 09:57:36 all (278/674): mmap29.sh mount leak: 1/360 20190828 09:58:40 all (279/674): pts3.sh 20190828 09:59:44 all (280/674): sendfile9.sh mount leak: 1/361 20190828 10:00:48 all (281/674): shm.sh 20190828 10:01:52 all (282/674): sigstop2.sh 20190828 10:02:55 all (283/674): socketpair3.sh 20190828 10:03:58 all (284/674): flock.sh mount leak: 1/362 20190828 10:05:01 all (285/674): fts2.sh mount leak: 1/363 20190828 10:06:05 all (286/674): nfs_halfpage2.sh 20190828 10:07:10 all (287/674): sendfile8.sh mount leak: 1/364 20190828 10:08:14 all (288/674): lockd.sh 20190828 10:09:23 all (289/674): nfs_halfpage.sh 20190828 10:10:29 all (290/674): advlock.sh mount leak: 1/365 20190828 10:11:32 all (291/674): tvnlru.sh mount leak: 3/368 20190828 10:12:30 all (292/674): crossmp7.sh ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Warning: memory type solaris leaked memory on destroy (4 allocations, 4160 bytes leaked). 20190828 10:13:28 all (293/674): rename10.sh mount leak: 1/369 20190828 10:14:43 all (294/674): crossmp8.sh 20190828 10:15:50 all (295/674): tcp3.sh 20190828 10:17:06 all (296/674): truss2.sh 20190828 10:18:17 all (297/674): vmstat.sh 20190828 10:19:30 all (298/674): suj6.sh mount leak: 1/370 20190828 10:21:10 all (299/674): stack_guard_page.sh mount leak: 1/371 20190828 10:22:39 all (300/674): nullfs22.sh mount leak: 1/372 20190828 10:23:37 all (301/674): jumbo.sh 20190828 10:24:53 all (302/674): mmap5.sh 20190828 10:26:17 all (303/674): bench.sh mount leak: 5/377 20190828 10:27:36 all (304/674): callout_reset_on.sh 20190828 10:34:05 all (305/674): fifo3.sh mount leak: 1/378 20190828 10:35:32 all (306/674): exlock.sh mount leak: 1/379 20190828 10:37:18 all (307/674): ptrace10.sh 20190828 10:39:09 all (308/674): rename3.sh 20190828 10:40:15 all (309/674): kevent14.sh 20190828 10:42:19 all (310/674): fifo2.sh mount leak: 1/380 20190828 10:44:22 all (311/674): kevent.sh 20190828 10:46:26 all (312/674): midi.sh 20190828 10:48:29 all (313/674): sendfile10.sh mount leak: 1/381 20190828 10:50:34 all (314/674): poll2.sh 20190828 10:52:43 all (315/674): tmpfs2.sh 20190828 10:53:25 all (316/674): truncate9.sh mount leak: 1/382 20190828 10:55:33 all (317/674): cpuset.sh panic: stack overflow detected; backtrace may be corrupted cpuid = 12 time = 1566983248 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe096d01d800 vpanic() at vpanic+0x19d/frame 0xfffffe096d01d850 panic() at panic+0x43/frame 0xfffffe096d01d8b0 __stack_chk_fail() at __stack_chk_fail+0x12/frame 0xfffffe096d01d8c0 kern_cpuset_setdomain() at kern_cpuset_setdomain+0x40f/frame 0xfffffe096d01d960 sys_cpuset_setdomain() at sys_cpuset_setdomain+0x26/frame 0xfffffe093f3e3d3c kernel trap 12 with interrupts disabled Fatal trap 12: page fault while in kernel mode cpuid = 12; apic id = 20 fault virtual address = 0xfffffe093f3e3d44 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff8116b510 stack pointer = 0x28:0xfffffe096d01d620 frame pointer = 0x28:0xfffffe096d01d6b0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = resume, IOPL = 0 current process = 71221 (cpuset) trap number = 12 panic: page fault cpuid = 12 time = 1566983248 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe096d01d2e0 vpanic() at vpanic+0x19d/frame 0xfffffe096d01d330 panic() at panic+0x43/frame 0xfffffe096d01d390 trap_fatal() at trap_fatal+0x39c/frame 0xfffffe096d01d3f0 trap_pfault() at trap_pfault+0x62/frame 0xfffffe096d01d440 trap() at trap+0x2b2/frame 0xfffffe096d01d550 calltrap() at calltrap+0x8/frame 0xfffffe096d01d550 --- trap 0xc, rip = 0xffffffff8116b510, rsp = 0xfffffe096d01d620, rbp = 0xfffffe096d01d6b0 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe096d01d6b0 db_get_value() at db_get_value+0x33/frame 0xfffffe096d01d6f0 db_backtrace() at db_backtrace+0x240/frame 0xfffffe096d01d780 db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe096d01d800 vpanic() at vpanic+0x19d/frame 0xfffffe096d01d850 panic() at panic+0x43/frame 0xfffffe096d01d8b0 __stack_chk_fail() at __stack_chk_fail+0x12/frame 0xfffffe096d01d8c0 kern_cpuset_setdomain() at kern_cpuset_setdomain+0x40f/frame 0xfffffe096d01d960 sys_cpuset_setdomain() at sys_cpuset_setdomain+0x26/frame 0xfffffe093f3e3d3c kernel trap 12 with interrupts disabled Fatal trap 12: page fault while in kernel mode cpuid = 12; apic id = 20 fault virtual address = 0xfffffe093f3e3d44 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff8116b510 stack pointer = 0x28:0xfffffe096d01d100 frame pointer = 0x28:0xfffffe096d01d190 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = resume, IOPL = 0 current process = 71221 (cpuset) trap number = 12 panic: page fault cpuid = 12 time = 1566983248 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe096d01cdc0 vpanic() at vpanic+0x19d/frame 0xfffffe096d01ce10 panic() at panic+0x43/frame 0xfffffe096d01ce70 trap_fatal() at trap_fatal+0x39c/frame 0xfffffe096d01ced0 trap_pfault() at trap_pfault+0x62/frame 0xfffffe096d01cf20 trap() at trap+0x2b2/frame 0xfffffe096d01d030 calltrap() at calltrap+0x8/frame 0xfffffe096d01d030 --- trap 0xc, rip = 0xffffffff8116b510, rsp = 0xfffffe096d01d100, rbp = 0xfffffe096d01d190 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe096d01d190 db_get_value() at db_get_value+0x33/frame 0xfffffe096d01d1d0 db_backtrace() at db_backtrace+0x240/frame 0xfffffe096d01d260 db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe096d01d2e0 vpanic() at vpanic+0x19d/frame 0xfffffe096d01d330 panic() at panic+0x43/frame 0xfffffe096d01d390 trap_fatal() at trap_fatal+0x39c/frame 0xfffffe096d01d3f0 trap_pfault() at trap_pfault+0x62/frame 0xfffffe096d01d440 trap() at trap+0x2b2/frame 0xfffffe096d01d550 calltrap() at calltrap+0x8/frame 0xfffffe096d01d550 --- trap 0xc, rip = 0xffffffff8116b510, rsp = 0xfffffe096d01d620, rbp = 0xfffffe096d01d6b0 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe096d01d6b0 db_get_value() at db_get_value+0x33/frame 0xfffffe096d01d6f0 db_backtrace() at db_backtrace+0x240/frame 0xfffffe096d01d780 db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe096d01d800 vpanic() at vpanic+0x19d/frame 0xfffffe096d01d850 panic() at panic+0x43/frame 0xfffffe096d01d8b0 __stack_chk_fail() at __stack_chk_fail+0x12/frame 0xfffffe096d01d8c0 kern_cpuset_setdomain() at kern_cpuset_setdomain+0x40f/frame 0xfffffe096d01d960 sys_cpuset_setdomain() at sys_cpuset_setdomain+0x26/frame 0xfffffe093f3e3d3c kernel trap 12 with interrupts disabled Fatal trap 12: page fault while in kernel mode cpuid = 12; apic id = 20 fault virtual address = 0xfffffe093f3e3d44 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff8116b510 stack pointer = 0x28:0xfffffe096d01cbe0 frame pointer = 0x28:0xfffffe096d01cc70 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = resume, IOPL = 0 current process = 71221 (cpuset) trap number = 12 panic: page fault cpuid = 12 time = 1566983248 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe096d01c8a0 vpanic() at vpanic+0x19d/frame 0xfffffe096d01c8f0 panic() at panic+0x43/frame 0xfffffe096d01c950 trap_fatal() at trap_fatal+0x39c/frame 0xfffffe096d01c9b0 trap_pfault() at trap_pfault+0x62/frame 0xfffffe096d01ca00 trap() at trap+0x2b2/frame 0xfffffe096d01cb10 calltrap() at calltrap+0x8/frame 0xfffffe096d01cb10 --- trap 0xc, rip = 0xffffffff8116b510, rsp = 0xfffffe096d01cbe0, rbp = 0xfffffe096d01cc70 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe096d01cc70 db_get_value() at db_get_value+0x33/frame 0xfffffe096d01ccb0 db_backtrace() at db_backtrace+0x240/frame 0xfffffe096d01cd40 db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe096d01cdc0 vpanic() at vpanic+0x19d/frame 0xfffffe096d01ce10 panic() at panic+0x43/frame 0xfffffe096d01ce70 trap_fatal() at trap_fatal+0x39c/frame 0xfffffe096d01ced0 trap_pfault() at trap_pfault+0x62/frame 0xfffffe096d01cf20 trap() at trap+0x2b2/frame 0xfffffe096d01d030 calltrap() at calltrap+0x8/frame 0xfffffe096d01d030 --- trap 0xc, rip = 0xffffffff8116b510, rsp = 0xfffffe096d01d100, rbp = 0xfffffe096d01d190 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe096d01d190 db_get_value() at db_get_value+0x33/frame 0xfffffe096d01d1d0 db_backtrace() at db_backtrace+0x240/frame 0xfffffe096d01d260 db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe096d01d2e0 vpanic() at vpanic+0x19d/frame 0xfffffe096d01d330 panic() at panic+0x43/frame 0xfffffe096d01d390 trap_fatal() at trap_fatal+0x39c/frame 0xfffffe096d01d3f0 trap_pfault() at trap_pfault+0x62/frame 0xfffffe096d01d440 trap() at trap+0x2b2/frame 0xfffffe096d01d550 calltrap() at calltrap+0x8/frame 0xfffffe096d01d550 --- trap 0xc, rip = 0xffffffff8116b510, rsp = 0xfffffe096d01d620, rbp = 0xfffffe096d01d6b0 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe096d01d6b0 db_get_value() at db_get_value+0x33/frame 0xfffffe096d01d6f0 db_backtrace() at db_backtrace+0x240/frame 0xfffffe096d01d780 db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe096d01d800 vpanic() at vpanic+0x19d/frame 0xfffffe096d01d850 panic() at panic+0x43/frame 0xfffffe096d01d8b0 __stack_chk_fail() at __stack_chk_fail+0x12/frame 0xfffffe096d01d8c0 kern_cpuset_setdomain() at kern_cpuset_setdomain+0x40f/frame 0xfffffe096d01d960 sys_cpuset_setdomain() at sys_cpuset_setdomain+0x26/frame 0xfffffe093f3e3d3c kernel trap 12 with interrupts disabled Fatal trap 12: page fault while in kernel mode cpuid = 12; apic id = 20 fault virtual address = 0xfffffe093f3e3d44 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff8116b510 stack pointer = 0x28:0xfffffe096d01c6c0 frame pointer = 0x28:0xfffffe096d01c750 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = resume, IOPL = 0 current process = 71221 (cpuset) trap number = 12 panic: page fault cpuid = 12 time = 1566983248 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe096d01c380 vpanic() at vpanic+0x19d/frame 0xfffffe096d01c3d0 panic() at panic+0x43/frame 0xfffffe096d01c430 trap_fatal() at trap_fatal+0x39c/frame 0xfffffe096d01c490 trap_pfault() at trap_pfault+0x62/frame 0xfffffe096d01c4e0 trap() at trap+0x2b2/frame 0xfffffe096d01c5f0 calltrap() at calltrap+0x8/frame 0xfffffe096d01c5f0 --- trap 0xc, rip = 0xffffffff8116b510, rsp = 0xfffffe096d01c6c0, rbp = 0xfffffe096d01c750 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe096d01c750 db_get_value() at db_get_value+0x33/frame 0xfffffe096d01c790 db_backtrace() at db_backtrace+0x240/frame 0xfffffe096d01c820 db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe096d01c8a0 vpanic() at vpanic+0x19d/frame 0xfffffe096d01c8f0 panic() at panic+0x43/frame 0xfffffe096d01c950 trap_fatal() at trap_fatal+0x39c/frame 0xfffffe096d01c9b0 trap_pfault() at trap_pfault+0x62/frame 0xfffffe096d01ca00 trap() at trap+0x2b2/frame 0xfffffe096d01cb10 calltrap() at calltrap+0x8/frame 0xfffffe096d01cb10 --- trap 0xc, rip = 0xffffffff8116b510, rsp = 0xfffffe096d01cbe0, rbp = 0xfffffe096d01cc70 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe096d01cc70 db_get_value() at db_get_value+0x33/frame 0xfffffe096d01ccb0 db_backtrace() at db_backtrace+0x240/frame 0xfffffe096d01cd40 db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe096d01cdc0 vpanic() at vpanic+0x19d/frame 0xfffffe096d01ce10 panic() at panic+0x43/frame 0xfffffe096d01ce70 trap_fatal() at trap_fatal+0x39c/frame 0xfffffe096d01ced0 trap_pfault() at trap_pfault+0x62/frame 0xfffffe096d01cf20 trap() at trap+0x2b2/frame 0xfffffe096d01d030 calltrap() at calltrap+0x8/frame 0xfffffe096d01d030 --- trap 0xc, rip = 0xffffffff8116b510, rsp = 0xfffffe096d01d100, rbp = 0xfffffe096d01d190 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe096d01d190 db_get_value() at db_get_value+0x33/frame 0xfffffe096d01d1d0 db_backtrace() at db_backtrace+0x240/frame 0xfffffe096d01d260 db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe096d01d2e0 vpanic() at vpanic+0x19d/frame 0xfffffe096d01d330 panic() at panic+0x43/frame 0xfffffe096d01d390 trap_fatal() at trap_fatal+0x39c/frame 0xfffffe096d01d3f0 trap_pfault() at trap_pfault+0x62/frame 0xfffffe096d01d440 trap() at trap+0x2b2/frame 0xfffffe096d01d550 calltrap() at calltrap+0x8/frame 0xfffffe096d01d550 --- trap 0xc, rip = 0xffffffff8116b510, rsp = 0xfffffe096d01d620, rbp = 0xfffffe096d01d6b0 --- db_read_bytes() at db_read_bytes+0x90/frame 0xfffffe096d01d6b0 db_get_value() at db_get_value+0x33/frame 0xfffffe096d01d6f0 db_backtrace() at db_backtrace+0x240/frame 0xfffffe096d01d780 db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe096d01d800 vpanic() at vpanic+0x19d/frame 0xfffffe096d01d850 panic() at panic+0x43/frame 0xfffffe096d01d8b0 __stack_chk_fail() at __stack_chk_fail+0x12/frame 0xfffffe096d01d8c0 kern_cpuset_setdomain() at kern_cpuset_setdomain+0x40f/frame 0xfffffe096d01d960 sys_cpuset_setdomain() at sys_cpuset_setdomain+0x26/frame 0xfffffe093f3e3d3c kernel trap 12 with interrupts disabled No luck with a NMI. Had to power cycle. (kgdb) l *kern_cpuset_setdomain+0x40f 0xffffffff80b7848f is in kern_cpuset_setdomain (../../../kern/kern_cpuset.c:2208). 2203 nset = cpuset_refroot(set); 2204 else 2205 nset = cpuset_refbase(set); 2206 error = cpuset_modify_domain(nset, &domain); 2207 cpuset_rel(nset); 2208 cpuset_rel(set); 2209 break; 2210 case CPU_LEVEL_WHICH: 2211 switch (which) { 2212 case CPU_WHICH_TID: (kgdb) (kgdb) l *sys_cpuset_setdomain+0x26 0xffffffff80b78076 is in sys_cpuset_setdomain (../../../kern/kern_cpuset.c:2103). 2098 #endif 2099 int 2100 sys_cpuset_setdomain(struct thread *td, struct cpuset_setdomain_args *uap) 2101 { 2102 2103 return (kern_cpuset_setdomain(td, uap->level, uap->which, 2104 uap->id, uap->domainsetsize, uap->mask, uap->policy)); 2105 } 2106 2107 int (kgdb) Index: src/share/man/man9/Makefile =================================================================== --- src/share/man/man9/Makefile (revision 351563) +++ src/share/man/man9/Makefile (working copy) @@ -2223,7 +2223,9 @@ MLINKS+=vm_map_lookup.9 vm_map_lookup_done.9 MLINKS+=vm_map_max.9 vm_map_min.9 \ vm_map_max.9 vm_map_pmap.9 MLINKS+=vm_map_stack.9 vm_map_growstack.9 -MLINKS+=vm_map_wire.9 vm_map_unwire.9 +MLINKS+=vm_map_wire.9 vm_map_wire_mapped.9 \ + vm_page_wire.9 vm_page_unwire.9 \ + vm_page_wire.9 vm_page_unwire_noq.9 MLINKS+=vm_page_bits.9 vm_page_clear_dirty.9 \ vm_page_bits.9 vm_page_dirty.9 \ vm_page_bits.9 vm_page_is_valid.9 \ Index: src/share/man/man9/vm_page_wire.9 =================================================================== --- src/share/man/man9/vm_page_wire.9 (revision 351563) +++ src/share/man/man9/vm_page_wire.9 (working copy) @@ -26,12 +26,13 @@ .\" .\" $FreeBSD$ .\" -.Dd July 13, 2001 +.Dd June 3, 2019 .Dt VM_PAGE_WIRE 9 .Os .Sh NAME .Nm vm_page_wire , -.Nm vm_page_unwire +.Nm vm_page_unwire , +.Nm vm_page_unwire_noq .Nd "wire and unwire pages" .Sh SYNOPSIS .In sys/param.h @@ -39,29 +40,44 @@ .In vm/vm_page.h .Ft void .Fn vm_page_wire "vm_page_t m" +.Ft bool +.Fn vm_page_wire_mapped "vm_page_t m" .Ft void -.Fn vm_page_unwire "vm_page_t m" "int activate" +.Fn vm_page_unwire "vm_page_t m" "int queue" +.Ft bool +.Fn vm_page_unwire_noq "vm_page_t m" .Sh DESCRIPTION The .Fn vm_page_wire -function increments the wire count on a page, and removes it from -whatever queue it is on. +and +.Fn vm_page_wire_mapped +function wire the page, prevent it from being reclaimed by the page +daemon or when its containing object is destroyed. +Both functions require that the page belong to an object. +The +.Fn vm_page_wire_mapped +function is for use by the +.Xr pmap 9 +layer following a lookup. +This function may fail if mappings of the page are concurrently +being destroyed, in which case it will return false. .Pp The .Fn vm_page_unwire -function releases one of the wirings on the page. -When -.Va write_count -reaches zero the page is placed back onto either the active queue -(if -.Fa activate -is non-zero) or onto the inactive queue (if -.Fa activate -is zero). -If the page is unmanaged -.Dv ( PG_UNMANAGED -is set) then the page is left on -.Dv PQ_NONE . +and +.Fn vm_page_unwire_noq +functions release a wiring of a page. +The +.Fn vm_page_unwire +function takes a queue index and will insert the page into the +corresponding page queue upon releasing its last wiring. +If the page does not belong to an object and no other references +to the page exist, +.Fn vm_page_unwire +will free the page. +.Fn vm_page_unwire_noq +releases the wiring and returns true if it was the last wiring +of the page. .Sh AUTHORS This manual page was written by .An Chad David Aq Mt davidc@acns.ab.ca . Index: src/sys/amd64/amd64/pmap.c =================================================================== --- src/sys/amd64/amd64/pmap.c (revision 351563) +++ src/sys/amd64/amd64/pmap.c (working copy) @@ -3058,31 +3058,23 @@ pmap_extract_and_hold(pmap_t pmap, vm_offset_t va, m = NULL; PG_RW = pmap_rw_bit(pmap); PG_V = pmap_valid_bit(pmap); + PMAP_LOCK(pmap); -retry: pdep = pmap_pde(pmap, va); if (pdep != NULL && (pde = *pdep)) { if (pde & PG_PS) { - if ((pde & PG_RW) || (prot & VM_PROT_WRITE) == 0) { - if (vm_page_pa_tryrelock(pmap, (pde & - PG_PS_FRAME) | (va & PDRMASK), &pa)) - goto retry; - m = PHYS_TO_VM_PAGE(pa); - } + if ((pde & PG_RW) != 0 || (prot & VM_PROT_WRITE) == 0) + m = PHYS_TO_VM_PAGE((pde & PG_PS_FRAME) | + (va & PDRMASK)); } else { pte = *pmap_pde_to_pte(pdep, va); - if ((pte & PG_V) && - ((pte & PG_RW) || (prot & VM_PROT_WRITE) == 0)) { - if (vm_page_pa_tryrelock(pmap, pte & PG_FRAME, - &pa)) - goto retry; - m = PHYS_TO_VM_PAGE(pa); - } + if ((pte & PG_V) != 0 && + ((pte & PG_RW) != 0 || (prot & VM_PROT_WRITE) == 0)) + m = PHYS_TO_VM_PAGE(pte & PG_FRAME); } - if (m != NULL) - vm_page_wire(m); + if (m != NULL && !vm_page_wire_mapped(m)) + m = NULL; } - PA_UNLOCK_COND(pa); PMAP_UNLOCK(pmap); return (m); } @@ -5804,7 +5796,7 @@ retry: ("pmap_enter: no PV entry for %#lx", va)); if ((newpte & PG_MANAGED) == 0) free_pv_entry(pmap, pv); - if ((om->aflags & PGA_WRITEABLE) != 0 && + if (vm_page_aflag_isset(om, PGA_WRITEABLE) && TAILQ_EMPTY(&om->md.pv_list) && ((om->flags & PG_FICTITIOUS) != 0 || TAILQ_EMPTY(&pa_to_pvh(opa)->pv_list))) @@ -6987,7 +6979,7 @@ pmap_remove_pages(pmap_t pmap) pvh->pv_gen++; if (TAILQ_EMPTY(&pvh->pv_list)) { for (mt = m; mt < &m[NBPDR / PAGE_SIZE]; mt++) - if ((mt->aflags & PGA_WRITEABLE) != 0 && + if (vm_page_aflag_isset(mt, PGA_WRITEABLE) && TAILQ_EMPTY(&mt->md.pv_list)) vm_page_aflag_clear(mt, PGA_WRITEABLE); } @@ -7005,7 +6997,7 @@ pmap_remove_pages(pmap_t pmap) pmap_resident_count_dec(pmap, 1); TAILQ_REMOVE(&m->md.pv_list, pv, pv_next); m->md.pv_gen++; - if ((m->aflags & PGA_WRITEABLE) != 0 && + if (vm_page_aflag_isset(m, PGA_WRITEABLE) && TAILQ_EMPTY(&m->md.pv_list) && (m->flags & PG_FICTITIOUS) == 0) { pvh = pa_to_pvh(VM_PAGE_TO_PHYS(m)); @@ -7136,7 +7128,7 @@ pmap_is_modified(vm_page_t m) * is clear, no PTEs can have PG_M set. */ VM_OBJECT_ASSERT_WLOCKED(m->object); - if (!vm_page_xbusied(m) && (m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_xbusied(m) && !vm_page_aflag_isset(m, PGA_WRITEABLE)) return (FALSE); return (pmap_page_test_mappings(m, FALSE, TRUE)); } @@ -7205,7 +7197,7 @@ pmap_remove_write(vm_page_t m) * if PGA_WRITEABLE is clear, no page table entries need updating. */ VM_OBJECT_ASSERT_WLOCKED(m->object); - if (!vm_page_xbusied(m) && (m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_xbusied(m) && !vm_page_aflag_isset(m, PGA_WRITEABLE)) return; lock = VM_PAGE_TO_PV_LIST_LOCK(m); pvh = (m->flags & PG_FICTITIOUS) != 0 ? &pv_dummy : @@ -7688,7 +7680,7 @@ pmap_clear_modify(vm_page_t m) * If the object containing the page is locked and the page is not * exclusive busied, then PGA_WRITEABLE cannot be concurrently set. */ - if ((m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_aflag_isset(m, PGA_WRITEABLE)) return; pvh = (m->flags & PG_FICTITIOUS) != 0 ? &pv_dummy : pa_to_pvh(VM_PAGE_TO_PHYS(m)); Index: src/sys/amd64/vmm/vmm.c =================================================================== --- src/sys/amd64/vmm/vmm.c (revision 351563) +++ src/sys/amd64/vmm/vmm.c (working copy) @@ -1002,9 +1002,7 @@ vm_gpa_release(void *cookie) { vm_page_t m = cookie; - vm_page_lock(m); vm_page_unwire(m, PQ_ACTIVE); - vm_page_unlock(m); } int Index: src/sys/arm/arm/pmap-v4.c =================================================================== --- src/sys/arm/arm/pmap-v4.c (revision 351563) +++ src/sys/arm/arm/pmap-v4.c (working copy) @@ -3415,14 +3415,14 @@ pmap_extract_and_hold(pmap_t pmap, vm_offset_t va, struct l2_dtable *l2; pd_entry_t l1pd; pt_entry_t *ptep, pte; - vm_paddr_t pa, paddr; - vm_page_t m = NULL; + vm_paddr_t pa; + vm_page_t m; u_int l1idx; + l1idx = L1_IDX(va); - paddr = 0; + m = NULL; PMAP_LOCK(pmap); -retry: l1pd = pmap->pm_l1->l1_kva[l1idx]; if (l1pte_section_p(l1pd)) { /* @@ -3434,11 +3434,10 @@ pmap_extract_and_hold(pmap_t pmap, vm_offset_t va, pa = (l1pd & L1_SUP_FRAME) | (va & L1_SUP_OFFSET); else pa = (l1pd & L1_S_FRAME) | (va & L1_S_OFFSET); - if (vm_page_pa_tryrelock(pmap, pa & PG_FRAME, &paddr)) - goto retry; if (l1pd & L1_S_PROT_W || (prot & VM_PROT_WRITE) == 0) { m = PHYS_TO_VM_PAGE(pa); - vm_page_wire(m); + if (!vm_page_wire_mapped(m)) + m = NULL; } } else { /* @@ -3466,15 +3465,12 @@ pmap_extract_and_hold(pmap_t pmap, vm_offset_t va, pa = (pte & L2_L_FRAME) | (va & L2_L_OFFSET); else pa = (pte & L2_S_FRAME) | (va & L2_S_OFFSET); - if (vm_page_pa_tryrelock(pmap, pa & PG_FRAME, &paddr)) - goto retry; m = PHYS_TO_VM_PAGE(pa); - vm_page_wire(m); + if (!vm_page_wire_mapped(m)) + m = NULL; } } - PMAP_UNLOCK(pmap); - PA_UNLOCK_COND(paddr); return (m); } @@ -4108,7 +4104,7 @@ pmap_clear_modify(vm_page_t m) * If the object containing the page is locked and the page is not * exclusive busied, then PGA_WRITEABLE cannot be concurrently set. */ - if ((m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_aflag_isset(m, PGA_WRITEABLE)) return; if (m->md.pvh_attrs & PVF_MOD) pmap_clearbit(m, PVF_MOD); @@ -4147,7 +4143,7 @@ pmap_remove_write(vm_page_t m) * if PGA_WRITEABLE is clear, no page table entries need updating. */ VM_OBJECT_ASSERT_WLOCKED(m->object); - if (vm_page_xbusied(m) || (m->aflags & PGA_WRITEABLE) != 0) + if (vm_page_xbusied(m) || vm_page_aflag_isset(m, PGA_WRITEABLE)) pmap_clearbit(m, PVF_WRITE); } Index: src/sys/arm/arm/pmap-v6.c =================================================================== --- src/sys/arm/arm/pmap-v6.c (revision 351563) +++ src/sys/arm/arm/pmap-v6.c (working copy) @@ -1986,23 +1986,20 @@ pmap_extract(pmap_t pmap, vm_offset_t va) vm_page_t pmap_extract_and_hold(pmap_t pmap, vm_offset_t va, vm_prot_t prot) { - vm_paddr_t pa, lockpa; + vm_paddr_t pa; pt1_entry_t pte1; pt2_entry_t pte2, *pte2p; vm_page_t m; - lockpa = 0; m = NULL; PMAP_LOCK(pmap); -retry: pte1 = pte1_load(pmap_pte1(pmap, va)); if (pte1_is_section(pte1)) { if (!(pte1 & PTE1_RO) || !(prot & VM_PROT_WRITE)) { pa = pte1_pa(pte1) | (va & PTE1_OFFSET); - if (vm_page_pa_tryrelock(pmap, pa, &lockpa)) - goto retry; m = PHYS_TO_VM_PAGE(pa); - vm_page_wire(m); + if (!vm_page_wire_mapped(m)) + m = NULL; } } else if (pte1_is_link(pte1)) { pte2p = pmap_pte2(pmap, va); @@ -2011,13 +2008,11 @@ pmap_extract_and_hold(pmap_t pmap, vm_offset_t va, if (pte2_is_valid(pte2) && (!(pte2 & PTE2_RO) || !(prot & VM_PROT_WRITE))) { pa = pte2_pa(pte2); - if (vm_page_pa_tryrelock(pmap, pa, &lockpa)) - goto retry; m = PHYS_TO_VM_PAGE(pa); - vm_page_wire(m); + if (!vm_page_wire_mapped(m)) + m = NULL; } } - PA_UNLOCK_COND(lockpa); PMAP_UNLOCK(pmap); return (m); } @@ -5202,7 +5197,7 @@ pmap_is_modified(vm_page_t m) * is clear, no PTE2s can have PG_M set. */ VM_OBJECT_ASSERT_WLOCKED(m->object); - if (!vm_page_xbusied(m) && (m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_xbusied(m) && !vm_page_aflag_isset(m, PGA_WRITEABLE)) return (FALSE); rw_wlock(&pvh_global_lock); rv = pmap_is_modified_pvh(&m->md) || @@ -5545,7 +5540,7 @@ pmap_remove_write(vm_page_t m) * if PGA_WRITEABLE is clear, no page table entries need updating. */ VM_OBJECT_ASSERT_WLOCKED(m->object); - if (!vm_page_xbusied(m) && (m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_xbusied(m) && !pmap_page_is_write_mapped(m)) return; rw_wlock(&pvh_global_lock); sched_pin(); Index: src/sys/arm64/arm64/pmap.c =================================================================== --- src/sys/arm64/arm64/pmap.c (revision 351563) +++ src/sys/arm64/arm64/pmap.c (working copy) @@ -1079,14 +1079,11 @@ pmap_extract_and_hold(pmap_t pmap, vm_offset_t va, { pt_entry_t *pte, tpte; vm_offset_t off; - vm_paddr_t pa; vm_page_t m; int lvl; - pa = 0; m = NULL; PMAP_LOCK(pmap); -retry: pte = pmap_pte(pmap, va, &lvl); if (pte != NULL) { tpte = pmap_load(pte); @@ -1111,14 +1108,11 @@ pmap_extract_and_hold(pmap_t pmap, vm_offset_t va, default: off = 0; } - if (vm_page_pa_tryrelock(pmap, - (tpte & ~ATTR_MASK) | off, &pa)) - goto retry; m = PHYS_TO_VM_PAGE((tpte & ~ATTR_MASK) | off); - vm_page_wire(m); + if (!vm_page_wire_mapped(m)) + m = NULL; } } - PA_UNLOCK_COND(pa); PMAP_UNLOCK(pmap); return (m); } @@ -3339,7 +3333,7 @@ havel3: pv = pmap_pvh_remove(&om->md, pmap, va); if ((m->oflags & VPO_UNMANAGED) != 0) free_pv_entry(pmap, pv); - if ((om->aflags & PGA_WRITEABLE) != 0 && + if (vm_page_aflag_isset(om, PGA_WRITEABLE) && TAILQ_EMPTY(&om->md.pv_list) && ((om->flags & PG_FICTITIOUS) != 0 || TAILQ_EMPTY(&pa_to_pvh(opa)->pv_list))) @@ -4378,7 +4372,7 @@ pmap_remove_pages(pmap_t pmap) pvh->pv_gen++; if (TAILQ_EMPTY(&pvh->pv_list)) { for (mt = m; mt < &m[L2_SIZE / PAGE_SIZE]; mt++) - if ((mt->aflags & PGA_WRITEABLE) != 0 && + if (vm_page_aflag_isset(mt, PGA_WRITEABLE) && TAILQ_EMPTY(&mt->md.pv_list)) vm_page_aflag_clear(mt, PGA_WRITEABLE); } @@ -4400,7 +4394,7 @@ pmap_remove_pages(pmap_t pmap) TAILQ_REMOVE(&m->md.pv_list, pv, pv_next); m->md.pv_gen++; - if ((m->aflags & PGA_WRITEABLE) != 0 && + if (vm_page_aflag_isset(m, PGA_WRITEABLE) && TAILQ_EMPTY(&m->md.pv_list) && (m->flags & PG_FICTITIOUS) == 0) { pvh = pa_to_pvh( @@ -4540,7 +4534,7 @@ pmap_is_modified(vm_page_t m) * is clear, no PTEs can have PG_M set. */ VM_OBJECT_ASSERT_WLOCKED(m->object); - if (!vm_page_xbusied(m) && (m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_xbusied(m) && !vm_page_aflag_isset(m, PGA_WRITEABLE)) return (FALSE); return (pmap_page_test_mappings(m, FALSE, TRUE)); } @@ -4606,7 +4600,7 @@ pmap_remove_write(vm_page_t m) * if PGA_WRITEABLE is clear, no page table entries need updating. */ VM_OBJECT_ASSERT_WLOCKED(m->object); - if (!vm_page_xbusied(m) && (m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_xbusied(m) && !vm_page_aflag_isset(m)) return; lock = VM_PAGE_TO_PV_LIST_LOCK(m); pvh = (m->flags & PG_FICTITIOUS) != 0 ? &pv_dummy : @@ -4983,7 +4977,7 @@ pmap_clear_modify(vm_page_t m) * set. If the object containing the page is locked and the page is not * exclusive busied, then PGA_WRITEABLE cannot be concurrently set. */ - if ((m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_aflag_isset(m, PGA_WRITEABLE)) return; pvh = (m->flags & PG_FICTITIOUS) != 0 ? &pv_dummy : pa_to_pvh(VM_PAGE_TO_PHYS(m)); Index: src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c =================================================================== --- src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c (revision 351563) +++ src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c (working copy) @@ -481,9 +481,7 @@ page_wire(vnode_t *vp, int64_t start) } ASSERT3U(pp->valid, ==, VM_PAGE_BITS_ALL); - vm_page_lock(pp); vm_page_wire(pp); - vm_page_unlock(pp); } else pp = NULL; break; @@ -495,9 +493,7 @@ static void page_unwire(vm_page_t pp) { - vm_page_lock(pp); vm_page_unwire(pp, PQ_ACTIVE); - vm_page_unlock(pp); } /* Index: src/sys/compat/linuxkpi/common/include/linux/mm.h =================================================================== --- src/sys/compat/linuxkpi/common/include/linux/mm.h (revision 351563) +++ src/sys/compat/linuxkpi/common/include/linux/mm.h (working copy) @@ -227,9 +227,7 @@ mark_page_accessed(struct vm_page *page) static inline void get_page(struct vm_page *page) { - vm_page_lock(page); vm_page_wire(page); - vm_page_unlock(page); } extern long @@ -250,10 +248,7 @@ get_user_pages_remote(struct task_struct *, struct static inline void put_page(struct vm_page *page) { - vm_page_lock(page); - if (vm_page_unwire(page, PQ_ACTIVE) && page->object == NULL) - vm_page_free(page); - vm_page_unlock(page); + vm_page_unwire(page, PQ_ACTIVE); } #define copy_highpage(to, from) pmap_copy_page(from, to) Index: src/sys/compat/linuxkpi/common/src/linux_page.c =================================================================== --- src/sys/compat/linuxkpi/common/src/linux_page.c (revision 351563) +++ src/sys/compat/linuxkpi/common/src/linux_page.c (working copy) @@ -154,10 +154,8 @@ linux_free_pages(vm_page_t page, unsigned int orde for (x = 0; x != npages; x++) { vm_page_t pgo = page + x; - vm_page_lock(pgo); if (vm_page_unwire_noq(pgo)) vm_page_free(pgo); - vm_page_unlock(pgo); } } else { vm_offset_t vaddr; Index: src/sys/contrib/vchiq/interface/vchiq_arm/vchiq_2835_arm.c =================================================================== --- src/sys/contrib/vchiq/interface/vchiq_arm/vchiq_2835_arm.c (revision 351563) +++ src/sys/contrib/vchiq/interface/vchiq_arm/vchiq_2835_arm.c (working copy) @@ -377,10 +377,7 @@ vchiq_platform_handle_timeout(VCHIQ_STATE_T *state static void pagelist_page_free(vm_page_t pp) { - vm_page_lock(pp); - if (vm_page_unwire(pp, PQ_INACTIVE) && pp->object == NULL) - vm_page_free(pp); - vm_page_unlock(pp); + vm_page_unwire(pp, PQ_INACTIVE); } /* There is a potential problem with partial cache lines (pages?) Index: src/sys/dev/agp/agp.c =================================================================== --- src/sys/dev/agp/agp.c (revision 351563) +++ src/sys/dev/agp/agp.c (working copy) @@ -616,9 +616,7 @@ bad: m = vm_page_lookup(mem->am_obj, OFF_TO_IDX(k)); if (k >= i) vm_page_xunbusy(m); - vm_page_lock(m); vm_page_unwire(m, PQ_INACTIVE); - vm_page_unlock(m); } VM_OBJECT_WUNLOCK(mem->am_obj); @@ -653,9 +651,7 @@ agp_generic_unbind_memory(device_t dev, struct agp VM_OBJECT_WLOCK(mem->am_obj); for (i = 0; i < mem->am_size; i += PAGE_SIZE) { m = vm_page_lookup(mem->am_obj, atop(i)); - vm_page_lock(m); vm_page_unwire(m, PQ_INACTIVE); - vm_page_unlock(m); } VM_OBJECT_WUNLOCK(mem->am_obj); Index: src/sys/dev/agp/agp_i810.c =================================================================== --- src/sys/dev/agp/agp_i810.c (revision 351563) +++ src/sys/dev/agp/agp_i810.c (working copy) @@ -1795,9 +1795,7 @@ agp_i810_free_memory(device_t dev, struct agp_memo */ VM_OBJECT_WLOCK(mem->am_obj); m = vm_page_lookup(mem->am_obj, 0); - vm_page_lock(m); vm_page_unwire(m, PQ_INACTIVE); - vm_page_unlock(m); VM_OBJECT_WUNLOCK(mem->am_obj); } else { contigfree(sc->argb_cursor, mem->am_size, M_AGP); Index: src/sys/dev/cxgbe/tom/t4_cpl_io.c =================================================================== --- src/sys/dev/cxgbe/tom/t4_cpl_io.c (revision 351563) +++ src/sys/dev/cxgbe/tom/t4_cpl_io.c (working copy) @@ -1910,7 +1910,6 @@ aiotx_free_pgs(struct mbuf *m) { struct mbuf_ext_pgs *ext_pgs; struct kaiocb *job; - struct mtx *mtx; vm_page_t pg; MBUF_EXT_PGS_ASSERT(m); @@ -1921,14 +1920,10 @@ aiotx_free_pgs(struct mbuf *m) m->m_len, jobtotid(job)); #endif - mtx = NULL; for (int i = 0; i < ext_pgs->npgs; i++) { pg = PHYS_TO_VM_PAGE(ext_pgs->pa[i]); - vm_page_change_lock(pg, &mtx); vm_page_unwire(pg, PQ_ACTIVE); } - if (mtx != NULL) - mtx_unlock(mtx); aiotx_free_job(job); } Index: src/sys/dev/cxgbe/tom/t4_ddp.c =================================================================== --- src/sys/dev/cxgbe/tom/t4_ddp.c (revision 351563) +++ src/sys/dev/cxgbe/tom/t4_ddp.c (working copy) @@ -114,9 +114,7 @@ free_pageset(struct tom_data *td, struct pageset * for (i = 0; i < ps->npages; i++) { p = ps->pages[i]; - vm_page_lock(p); vm_page_unwire(p, PQ_INACTIVE); - vm_page_unlock(p); } mtx_lock(&ddp_orphan_pagesets_lock); TAILQ_INSERT_TAIL(&ddp_orphan_pagesets, ps, link); Index: src/sys/dev/md/md.c =================================================================== --- src/sys/dev/md/md.c (revision 351563) +++ src/sys/dev/md/md.c (working copy) @@ -1029,9 +1029,7 @@ md_swap_page_free(vm_page_t m) { vm_page_xunbusy(m); - vm_page_lock(m); vm_page_free(m); - vm_page_unlock(m); } static int Index: src/sys/dev/virtio/balloon/virtio_balloon.c =================================================================== --- src/sys/dev/virtio/balloon/virtio_balloon.c (revision 351563) +++ src/sys/dev/virtio/balloon/virtio_balloon.c (working copy) @@ -332,8 +332,6 @@ vtballoon_inflate(struct vtballoon_softc *sc, int sc->vtballoon_page_frames[i] = VM_PAGE_TO_PHYS(m) >> VIRTIO_BALLOON_PFN_SHIFT; - KASSERT(m->queue == PQ_NONE, - ("%s: allocated page %p on queue", __func__, m)); TAILQ_INSERT_TAIL(&sc->vtballoon_pages, m, plinks.q); } Index: src/sys/i386/i386/pmap.c =================================================================== --- src/sys/i386/i386/pmap.c (revision 351563) +++ src/sys/i386/i386/pmap.c (working copy) @@ -1690,35 +1690,24 @@ __CONCAT(PMTYPE, extract_and_hold)(pmap_t pmap, vm pd_entry_t pde; pt_entry_t pte; vm_page_t m; - vm_paddr_t pa; - pa = 0; m = NULL; PMAP_LOCK(pmap); -retry: pde = *pmap_pde(pmap, va); if (pde != 0) { if (pde & PG_PS) { - if ((pde & PG_RW) || (prot & VM_PROT_WRITE) == 0) { - if (vm_page_pa_tryrelock(pmap, (pde & - PG_PS_FRAME) | (va & PDRMASK), &pa)) - goto retry; - m = PHYS_TO_VM_PAGE(pa); - } + if ((pde & PG_RW) || (prot & VM_PROT_WRITE) == 0) + m = PHYS_TO_VM_PAGE((pde & PG_PS_FRAME) | + (va & PDRMASK)); } else { pte = pmap_pte_ufast(pmap, va, pde); if (pte != 0 && - ((pte & PG_RW) || (prot & VM_PROT_WRITE) == 0)) { - if (vm_page_pa_tryrelock(pmap, pte & PG_FRAME, - &pa)) - goto retry; - m = PHYS_TO_VM_PAGE(pa); - } + ((pte & PG_RW) || (prot & VM_PROT_WRITE) == 0)) + m = PHYS_TO_VM_PAGE(pte & PG_FRAME); } - if (m != NULL) - vm_page_wire(m); + if (m != NULL && !vm_page_wire_mapped(m)) + m = NULL; } - PA_UNLOCK_COND(pa); PMAP_UNLOCK(pmap); return (m); } @@ -3763,7 +3752,7 @@ __CONCAT(PMTYPE, enter)(pmap_t pmap, vm_offset_t v ("pmap_enter: no PV entry for %#x", va)); if ((newpte & PG_MANAGED) == 0) free_pv_entry(pmap, pv); - if ((om->aflags & PGA_WRITEABLE) != 0 && + if (vm_page_aflag_isset(om) && TAILQ_EMPTY(&om->md.pv_list) && ((om->flags & PG_FICTITIOUS) != 0 || TAILQ_EMPTY(&pa_to_pvh(opa)->pv_list))) @@ -4859,7 +4848,7 @@ __CONCAT(PMTYPE, is_modified)(vm_page_t m) * is clear, no PTEs can have PG_M set. */ VM_OBJECT_ASSERT_WLOCKED(m->object); - if (!vm_page_xbusied(m) && (m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_xbusied(m) && !vm_page_aflag_isset(m)) return (FALSE); rw_wlock(&pvh_global_lock); rv = pmap_is_modified_pvh(&m->md) || @@ -4990,7 +4979,7 @@ __CONCAT(PMTYPE, remove_write)(vm_page_t m) * if PGA_WRITEABLE is clear, no page table entries need updating. */ VM_OBJECT_ASSERT_WLOCKED(m->object); - if (!vm_page_xbusied(m) && (m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_xbusied(m) && !vm_page_aflag_isset(m)) return; rw_wlock(&pvh_global_lock); sched_pin(); @@ -5302,7 +5291,7 @@ __CONCAT(PMTYPE, clear_modify)(vm_page_t m) * If the object containing the page is locked and the page is not * exclusive busied, then PGA_WRITEABLE cannot be concurrently set. */ - if ((m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_aflag_isset(m)) return; rw_wlock(&pvh_global_lock); sched_pin(); Index: src/sys/kern/kern_exec.c =================================================================== --- src/sys/kern/kern_exec.c (revision 351563) +++ src/sys/kern/kern_exec.c (working copy) @@ -972,13 +972,13 @@ exec_map_first_page(struct image_params *imgp) #if VM_NRESERVLEVEL > 0 vm_object_color(object, 0); #endif - ma[0] = vm_page_grab(object, 0, VM_ALLOC_NORMAL | VM_ALLOC_NOBUSY); + ma[0] = vm_page_grab(object, 0, VM_ALLOC_NORMAL | VM_ALLOC_NOBUSY | + VM_ALLOC_WIRED); if (ma[0]->valid != VM_PAGE_BITS_ALL) { vm_page_xbusy(ma[0]); if (!vm_pager_has_page(object, 0, NULL, &after)) { - vm_page_lock(ma[0]); + vm_page_unwire_noq(ma[0]); vm_page_free(ma[0]); - vm_page_unlock(ma[0]); VM_OBJECT_WUNLOCK(object); return (EIO); } @@ -1002,11 +1002,9 @@ exec_map_first_page(struct image_params *imgp) initial_pagein = i; rv = vm_pager_get_pages(object, ma, initial_pagein, NULL, NULL); if (rv != VM_PAGER_OK) { - for (i = 0; i < initial_pagein; i++) { - vm_page_lock(ma[i]); + vm_page_unwire_noq(ma[0]); + for (i = 0; i < initial_pagein; i++) vm_page_free(ma[i]); - vm_page_unlock(ma[i]); - } VM_OBJECT_WUNLOCK(object); return (EIO); } @@ -1014,9 +1012,6 @@ exec_map_first_page(struct image_params *imgp) for (i = 1; i < initial_pagein; i++) vm_page_readahead_finish(ma[i]); } - vm_page_lock(ma[0]); - vm_page_wire(ma[0]); - vm_page_unlock(ma[0]); VM_OBJECT_WUNLOCK(object); imgp->firstpage = sf_buf_alloc(ma[0], 0); @@ -1034,9 +1029,7 @@ exec_unmap_first_page(struct image_params *imgp) m = sf_buf_page(imgp->firstpage); sf_buf_free(imgp->firstpage); imgp->firstpage = NULL; - vm_page_lock(m); vm_page_unwire(m, PQ_ACTIVE); - vm_page_unlock(m); } } Index: src/sys/kern/kern_sendfile.c =================================================================== --- src/sys/kern/kern_sendfile.c (revision 351563) +++ src/sys/kern/kern_sendfile.c (working copy) @@ -415,11 +415,8 @@ sendfile_swapin(vm_object_t obj, struct sf_io *sfi &sendfile_iodone, sfio); if (rv != VM_PAGER_OK) { for (j = i; j < i + count; j++) { - if (pa[j] != bogus_page) { - vm_page_lock(pa[j]); + if (pa[j] != bogus_page) vm_page_unwire(pa[j], PQ_INACTIVE); - vm_page_unlock(pa[j]); - } } VM_OBJECT_WUNLOCK(obj); return (EIO); Index: src/sys/kern/sys_process.c =================================================================== --- src/sys/kern/sys_process.c (revision 351563) +++ src/sys/kern/sys_process.c (working copy) @@ -312,10 +312,7 @@ proc_rwmem(struct proc *p, struct uio *uio) /* * Release the page. */ - vm_page_lock(m); - if (vm_page_unwire(m, PQ_ACTIVE) && m->object == NULL) - vm_page_free(m); - vm_page_unlock(m); + vm_page_unwire(m, PQ_ACTIVE); } while (error == 0 && uio->uio_resid > 0); Index: src/sys/kern/uipc_mbuf.c =================================================================== --- src/sys/kern/uipc_mbuf.c (revision 351563) +++ src/sys/kern/uipc_mbuf.c (working copy) @@ -1621,10 +1621,6 @@ mb_free_mext_pgs(struct mbuf *m) ext_pgs = m->m_ext.ext_pgs; for (int i = 0; i < ext_pgs->npgs; i++) { pg = PHYS_TO_VM_PAGE(ext_pgs->pa[i]); - /* - * Note: page is not locked, as it has no - * object and is not on any queues. - */ vm_page_unwire_noq(pg); vm_page_free(pg); } Index: src/sys/kern/uipc_shm.c =================================================================== --- src/sys/kern/uipc_shm.c (revision 351563) +++ src/sys/kern/uipc_shm.c (working copy) @@ -188,7 +188,8 @@ uiomove_object_page(vm_object_t obj, size_t len, s * lock to page out tobj's pages because tobj is a OBJT_SWAP * type object. */ - m = vm_page_grab(obj, idx, VM_ALLOC_NORMAL | VM_ALLOC_NOBUSY); + m = vm_page_grab(obj, idx, VM_ALLOC_NORMAL | VM_ALLOC_NOBUSY | + VM_ALLOC_WIRED); if (m->valid != VM_PAGE_BITS_ALL) { vm_page_xbusy(m); if (vm_pager_has_page(obj, idx, NULL, NULL)) { @@ -197,9 +198,8 @@ uiomove_object_page(vm_object_t obj, size_t len, s printf( "uiomove_object: vm_obj %p idx %jd valid %x pager error %d\n", obj, idx, m->valid, rv); - vm_page_lock(m); + vm_page_unwire_noq(m); vm_page_free(m); - vm_page_unlock(m); VM_OBJECT_WUNLOCK(obj); return (EIO); } @@ -207,9 +207,6 @@ uiomove_object_page(vm_object_t obj, size_t len, s vm_page_zero_invalid(m, TRUE); vm_page_xunbusy(m); } - vm_page_lock(m); - vm_page_wire(m); - vm_page_unlock(m); VM_OBJECT_WUNLOCK(obj); error = uiomove_fromphys(&m, offset, tlen, uio); if (uio->uio_rw == UIO_WRITE && error == 0) { @@ -218,9 +215,7 @@ uiomove_object_page(vm_object_t obj, size_t len, s vm_pager_page_unswapped(m); VM_OBJECT_WUNLOCK(obj); } - vm_page_lock(m); vm_page_unwire(m, PQ_ACTIVE); - vm_page_unlock(m); return (error); } Index: src/sys/mips/mips/pmap.c =================================================================== --- src/sys/mips/mips/pmap.c (revision 351563) +++ src/sys/mips/mips/pmap.c (working copy) @@ -796,26 +796,22 @@ vm_page_t pmap_extract_and_hold(pmap_t pmap, vm_offset_t va, vm_prot_t prot) { pt_entry_t pte, *ptep; - vm_paddr_t pa, pte_pa; + vm_paddr_t pa; vm_page_t m; m = NULL; - pa = 0; PMAP_LOCK(pmap); -retry: ptep = pmap_pte(pmap, va); if (ptep != NULL) { pte = *ptep; if (pte_test(&pte, PTE_V) && (!pte_test(&pte, PTE_RO) || (prot & VM_PROT_WRITE) == 0)) { - pte_pa = TLBLO_PTE_TO_PA(pte); - if (vm_page_pa_tryrelock(pmap, pte_pa, &pa)) - goto retry; - m = PHYS_TO_VM_PAGE(pte_pa); - vm_page_wire(m); + pa = TLBLO_PTE_TO_PA(pte); + m = PHYS_TO_VM_PAGE(pa); + if (!vm_page_wire_mapped(m)) + m = NULL; } } - PA_UNLOCK_COND(pa); PMAP_UNLOCK(pmap); return (m); } @@ -2168,7 +2164,7 @@ pmap_enter(pmap_t pmap, vm_offset_t va, vm_page_t pv = pmap_pvh_remove(&om->md, pmap, va); if (!pte_test(&newpte, PTE_MANAGED)) free_pv_entry(pmap, pv); - if ((om->aflags & PGA_WRITEABLE) != 0 && + if (vm_page_aflag_isset(m, PGA_WRITEABLE) && TAILQ_EMPTY(&om->md.pv_list)) vm_page_aflag_clear(om, PGA_WRITEABLE); } @@ -2938,7 +2934,7 @@ pmap_remove_write(vm_page_t m) * if PGA_WRITEABLE is clear, no page table entries need updating. */ VM_OBJECT_ASSERT_WLOCKED(m->object); - if (!vm_page_xbusied(m) && (m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_xbusied(m) && !vm_page_aflag_isset(m, PGA_WRITEABLE)) return; rw_wlock(&pvh_global_lock); TAILQ_FOREACH(pv, &m->md.pv_list, pv_list) { @@ -3003,7 +2999,7 @@ pmap_is_modified(vm_page_t m) * is clear, no PTEs can have PTE_D set. */ VM_OBJECT_ASSERT_WLOCKED(m->object); - if (!vm_page_xbusied(m) && (m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_xbusied(m) && !vm_page_aflag_isset(m, PGA_WRITEABLE)) return (FALSE); rw_wlock(&pvh_global_lock); rv = pmap_testbit(m, PTE_D); @@ -3147,7 +3143,7 @@ pmap_clear_modify(vm_page_t m) * If the object containing the page is locked and the page is not * write busied, then PGA_WRITEABLE cannot be concurrently set. */ - if ((m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_aflag_isset(m, PGA_WRITEABLE)) return; rw_wlock(&pvh_global_lock); TAILQ_FOREACH(pv, &m->md.pv_list, pv_list) { @@ -3274,7 +3270,7 @@ retry: * determine if the address is MINCORE_REFERENCED. */ m = PHYS_TO_VM_PAGE(pa); - if ((m->aflags & PGA_REFERENCED) != 0) + if (vm_page_aflag_isset(m, PGA_REFERENCED)) val |= MINCORE_REFERENCED | MINCORE_REFERENCED_OTHER; } if ((val & (MINCORE_MODIFIED_OTHER | MINCORE_REFERENCED_OTHER)) != Index: src/sys/net/bpf_zerocopy.c =================================================================== --- src/sys/net/bpf_zerocopy.c (revision 351563) +++ src/sys/net/bpf_zerocopy.c (working copy) @@ -115,10 +115,7 @@ static void zbuf_page_free(vm_page_t pp) { - vm_page_lock(pp); - if (vm_page_unwire(pp, PQ_INACTIVE) && pp->object == NULL) - vm_page_free(pp); - vm_page_unlock(pp); + vm_page_unwire(pp, PQ_INACTIVE); } /* Index: src/sys/powerpc/aim/mmu_oea.c =================================================================== --- src/sys/powerpc/aim/mmu_oea.c (revision 351563) +++ src/sys/powerpc/aim/mmu_oea.c (working copy) @@ -1264,22 +1264,17 @@ moea_extract_and_hold(mmu_t mmu, pmap_t pmap, vm_o { struct pvo_entry *pvo; vm_page_t m; - vm_paddr_t pa; m = NULL; - pa = 0; PMAP_LOCK(pmap); -retry: pvo = moea_pvo_find_va(pmap, va & ~ADDR_POFF, NULL); if (pvo != NULL && (pvo->pvo_pte.pte.pte_hi & PTE_VALID) && ((pvo->pvo_pte.pte.pte_lo & PTE_PP) == PTE_RW || (prot & VM_PROT_WRITE) == 0)) { - if (vm_page_pa_tryrelock(pmap, pvo->pvo_pte.pte.pte_lo & PTE_RPGN, &pa)) - goto retry; m = PHYS_TO_VM_PAGE(pvo->pvo_pte.pte.pte_lo & PTE_RPGN); - vm_page_wire(m); + if (!vm_page_wire_mapped(m)) + m = NULL; } - PA_UNLOCK_COND(pa); PMAP_UNLOCK(pmap); return (m); } @@ -1324,7 +1319,7 @@ moea_is_modified(mmu_t mmu, vm_page_t m) * is clear, no PTEs can have PTE_CHG set. */ VM_OBJECT_ASSERT_WLOCKED(m->object); - if (!vm_page_xbusied(m) && (m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_xbusied(m) && !vm_page_aflag_isset(m, PGA_WRITEABLE)) return (FALSE); rw_wlock(&pvh_global_lock); rv = moea_query_bit(m, PTE_CHG); @@ -1360,7 +1355,7 @@ moea_clear_modify(mmu_t mmu, vm_page_t m) * set. If the object containing the page is locked and the page is * not exclusive busied, then PGA_WRITEABLE cannot be concurrently set. */ - if ((m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_aflag_isset(m, PGA_WRITEABLE)) return; rw_wlock(&pvh_global_lock); moea_clear_bit(m, PTE_CHG); @@ -1387,7 +1382,7 @@ moea_remove_write(mmu_t mmu, vm_page_t m) * if PGA_WRITEABLE is clear, no page table entries need updating. */ VM_OBJECT_ASSERT_WLOCKED(m->object); - if (!vm_page_xbusied(m) && (m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_xbusied(m) && !vm_page_aflag_isset(m, PGA_WRITEABLE)) return; rw_wlock(&pvh_global_lock); lo = moea_attr_fetch(m); @@ -1920,7 +1915,7 @@ moea_remove_all(mmu_t mmu, vm_page_t m) moea_pvo_remove(pvo, -1); PMAP_UNLOCK(pmap); } - if ((m->aflags & PGA_WRITEABLE) && moea_query_bit(m, PTE_CHG)) { + if (vm_page_aflag_isset(m, PGA_WRITEABLE) && moea_query_bit(m, PTE_CHG)) { moea_attr_clear(m, PTE_CHG); vm_page_dirty(m); } Index: src/sys/powerpc/aim/mmu_oea64.c =================================================================== --- src/sys/powerpc/aim/mmu_oea64.c (revision 351563) +++ src/sys/powerpc/aim/mmu_oea64.c (working copy) @@ -1473,7 +1473,7 @@ out: * Flush the page from the instruction cache if this page is * mapped executable and cacheable. */ - if (pmap != kernel_pmap && !(m->aflags & PGA_EXECUTABLE) && + if (pmap != kernel_pmap && vm_page_aflag_isset(m, PGA_EXECUTABLE) && (pte_lo & (LPTE_I | LPTE_G | LPTE_NOEXEC)) == 0) { vm_page_aflag_set(m, PGA_EXECUTABLE); moea64_syncicache(mmu, pmap, va, VM_PAGE_TO_PHYS(m), PAGE_SIZE); @@ -1584,21 +1584,15 @@ moea64_extract_and_hold(mmu_t mmu, pmap_t pmap, vm { struct pvo_entry *pvo; vm_page_t m; - vm_paddr_t pa; m = NULL; - pa = 0; PMAP_LOCK(pmap); -retry: pvo = moea64_pvo_find_va(pmap, va & ~ADDR_POFF); if (pvo != NULL && (pvo->pvo_pte.prot & prot) == prot) { - if (vm_page_pa_tryrelock(pmap, - pvo->pvo_pte.pa & LPTE_RPGN, &pa)) - goto retry; m = PHYS_TO_VM_PAGE(pvo->pvo_pte.pa & LPTE_RPGN); - vm_page_wire(m); + if (!vm_page_wire_mapped(m)) + m = NULL; } - PA_UNLOCK_COND(pa); PMAP_UNLOCK(pmap); return (m); } @@ -1700,7 +1694,7 @@ moea64_is_modified(mmu_t mmu, vm_page_t m) * is clear, no PTEs can have LPTE_CHG set. */ VM_OBJECT_ASSERT_LOCKED(m->object); - if (!vm_page_xbusied(m) && (m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_xbusied(m) && !vm_page_aflag_isset(m, PGA_WRITEABLE)) return (FALSE); return (moea64_query_bit(mmu, m, LPTE_CHG)); } @@ -1734,7 +1728,7 @@ moea64_clear_modify(mmu_t mmu, vm_page_t m) * set. If the object containing the page is locked and the page is * not exclusive busied, then PGA_WRITEABLE cannot be concurrently set. */ - if ((m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_aflag_isset(m, PGA_WRITEABLE)) return; moea64_clear_bit(mmu, m, LPTE_CHG); } @@ -1758,7 +1752,7 @@ moea64_remove_write(mmu_t mmu, vm_page_t m) * if PGA_WRITEABLE is clear, no page table entries need updating. */ VM_OBJECT_ASSERT_WLOCKED(m->object); - if (!vm_page_xbusied(m) && (m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_xbusied(m) && !vm_page_aflag_isset(m, PGA_WRITEABLE)) return; powerpc_sync(); PV_PAGE_LOCK(m); @@ -2248,7 +2242,8 @@ moea64_pvo_protect(mmu_t mmu, pmap_t pm, struct p if (refchg < 0) refchg = (oldprot & VM_PROT_WRITE) ? LPTE_CHG : 0; - if (pm != kernel_pmap && pg != NULL && !(pg->aflags & PGA_EXECUTABLE) && + if (pm != kernel_pmap && pg != NULL && + !vm_page_aflag_isset(pg, PGA_EXECUTABLE) && (pvo->pvo_pte.pa & (LPTE_I | LPTE_G | LPTE_NOEXEC)) == 0) { if ((pg->oflags & VPO_UNMANAGED) == 0) vm_page_aflag_set(pg, PGA_EXECUTABLE); @@ -2462,7 +2457,8 @@ moea64_remove_all(mmu_t mmu, vm_page_t m) } KASSERT(!pmap_page_is_mapped(m), ("Page still has mappings")); - KASSERT(!(m->aflags & PGA_WRITEABLE), ("Page still writable")); + KASSERT(!vm_page_aflag_isset(m, PGA_WRITEABLE), + ("Page still writable")); PV_PAGE_UNLOCK(m); /* Clean up UMA allocations */ Index: src/sys/powerpc/booke/pmap.c =================================================================== --- src/sys/powerpc/booke/pmap.c (revision 351563) +++ src/sys/powerpc/booke/pmap.c (working copy) @@ -2694,7 +2694,7 @@ mmu_booke_remove_write(mmu_t mmu, vm_page_t m) * if PGA_WRITEABLE is clear, no page table entries need updating. */ VM_OBJECT_ASSERT_WLOCKED(m->object); - if (!vm_page_xbusied(m) && (m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_xbusied(m) && !vm_page_aflag_isset(m, PGA_WRITEABLE)) return; rw_wlock(&pvh_global_lock); TAILQ_FOREACH(pv, &m->md.pv_list, pv_link) { @@ -2790,12 +2790,9 @@ mmu_booke_extract_and_hold(mmu_t mmu, pmap_t pmap, pte_t *pte; vm_page_t m; uint32_t pte_wbit; - vm_paddr_t pa; - + m = NULL; - pa = 0; PMAP_LOCK(pmap); -retry: pte = pte_find(mmu, pmap, va); if ((pte != NULL) && PTE_ISVALID(pte)) { if (pmap == kernel_pmap) @@ -2803,15 +2800,12 @@ mmu_booke_extract_and_hold(mmu_t mmu, pmap_t pmap, else pte_wbit = PTE_UW; - if ((*pte & pte_wbit) || ((prot & VM_PROT_WRITE) == 0)) { - if (vm_page_pa_tryrelock(pmap, PTE_PA(pte), &pa)) - goto retry; + if ((*pte & pte_wbit) != 0 || (prot & VM_PROT_WRITE) == 0) { m = PHYS_TO_VM_PAGE(PTE_PA(pte)); - vm_page_wire(m); + if (!vm_page_wire_mapped(m)) + m = NULL; } } - - PA_UNLOCK_COND(pa); PMAP_UNLOCK(pmap); return (m); } @@ -3046,7 +3040,7 @@ mmu_booke_is_modified(mmu_t mmu, vm_page_t m) * is clear, no PTEs can be modified. */ VM_OBJECT_ASSERT_WLOCKED(m->object); - if (!vm_page_xbusied(m) && (m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_xbusied(m) && !vm_page_aflag_isset(m, PGA_WRITEABLE)) return (rv); rw_wlock(&pvh_global_lock); TAILQ_FOREACH(pv, &m->md.pv_list, pv_link) { @@ -3125,7 +3119,7 @@ mmu_booke_clear_modify(mmu_t mmu, vm_page_t m) * If the object containing the page is locked and the page is not * exclusive busied, then PG_AWRITEABLE cannot be concurrently set. */ - if ((m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_aflag_isset(m, PGA_WRITEABLE)) return; rw_wlock(&pvh_global_lock); TAILQ_FOREACH(pv, &m->md.pv_list, pv_link) { Index: src/sys/riscv/riscv/pmap.c =================================================================== --- src/sys/riscv/riscv/pmap.c (revision 351563) +++ src/sys/riscv/riscv/pmap.c (working copy) @@ -870,24 +870,19 @@ pmap_extract_and_hold(pmap_t pmap, vm_offset_t va, { pt_entry_t *l3p, l3; vm_paddr_t phys; - vm_paddr_t pa; vm_page_t m; - pa = 0; m = NULL; PMAP_LOCK(pmap); -retry: l3p = pmap_l3(pmap, va); if (l3p != NULL && (l3 = pmap_load(l3p)) != 0) { if ((l3 & PTE_W) != 0 || (prot & VM_PROT_WRITE) == 0) { phys = PTE_TO_PHYS(l3); - if (vm_page_pa_tryrelock(pmap, phys, &pa)) - goto retry; m = PHYS_TO_VM_PAGE(phys); - vm_page_wire(m); + if (!vm_page_wire_mapped(m)) + m = NULL; } } - PA_UNLOCK_COND(pa); PMAP_UNLOCK(pmap); return (m); } @@ -2830,7 +2825,7 @@ pmap_enter(pmap_t pmap, vm_offset_t va, vm_page_t ("pmap_enter: no PV entry for %#lx", va)); if ((new_l3 & PTE_SW_MANAGED) == 0) free_pv_entry(pmap, pv); - if ((om->aflags & PGA_WRITEABLE) != 0 && + if (!vm_page_aflag_isset(om, PGA_WRITEABLE) && TAILQ_EMPTY(&om->md.pv_list)) vm_page_aflag_clear(om, PGA_WRITEABLE); } @@ -3561,7 +3556,7 @@ pmap_remove_pages_pv(pmap_t pmap, vm_page_t m, pv_ if (TAILQ_EMPTY(&pvh->pv_list)) { for (mt = m; mt < &m[Ln_ENTRIES]; mt++) if (TAILQ_EMPTY(&mt->md.pv_list) && - (mt->aflags & PGA_WRITEABLE) != 0) + vm_page_aflag_isset(mt, PGA_WRITEABLE)) vm_page_aflag_clear(mt, PGA_WRITEABLE); } mpte = pmap_remove_pt_page(pmap, pv->pv_va); @@ -3579,7 +3574,7 @@ pmap_remove_pages_pv(pmap_t pmap, vm_page_t m, pv_ TAILQ_REMOVE(&m->md.pv_list, pv, pv_next); m->md.pv_gen++; if (TAILQ_EMPTY(&m->md.pv_list) && - (m->aflags & PGA_WRITEABLE) != 0) { + vm_page_aflag_isset(m, PGA_WRITEABLE)) { pvh = pa_to_pvh(m->phys_addr); if (TAILQ_EMPTY(&pvh->pv_list)) vm_page_aflag_clear(m, PGA_WRITEABLE); @@ -3794,7 +3789,7 @@ pmap_is_modified(vm_page_t m) * is clear, no PTEs can have PG_M set. */ VM_OBJECT_ASSERT_WLOCKED(m->object); - if (!vm_page_xbusied(m) && (m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_xbusied(m) && !vm_page_aflag_isset(m, PGA_WRITEABLE)) return (FALSE); return (pmap_page_test_mappings(m, FALSE, TRUE)); } @@ -3860,7 +3855,7 @@ pmap_remove_write(vm_page_t m) * if PGA_WRITEABLE is clear, no page table entries need updating. */ VM_OBJECT_ASSERT_WLOCKED(m->object); - if (!vm_page_xbusied(m) && (m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_xbusied(m) && !vm_page_aflag_isset(m, PGA_WRITEABLE)) return; lock = VM_PAGE_TO_PV_LIST_LOCK(m); pvh = (m->flags & PG_FICTITIOUS) != 0 ? &pv_dummy : @@ -4120,7 +4115,7 @@ pmap_clear_modify(vm_page_t m) * If the object containing the page is locked and the page is not * exclusive busied, then PGA_WRITEABLE cannot be concurrently set. */ - if ((m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_aflag_isset(m, PGA_WRITEABLE)) return; pvh = (m->flags & PG_FICTITIOUS) != 0 ? &pv_dummy : pa_to_pvh(VM_PAGE_TO_PHYS(m)); Index: src/sys/sparc64/sparc64/pmap.c =================================================================== --- src/sys/sparc64/sparc64/pmap.c (revision 351563) +++ src/sys/sparc64/sparc64/pmap.c (working copy) @@ -846,19 +846,15 @@ pmap_extract_and_hold(pmap_t pm, vm_offset_t va, v { struct tte *tp; vm_page_t m; - vm_paddr_t pa; m = NULL; - pa = 0; PMAP_LOCK(pm); -retry: if (pm == kernel_pmap) { if (va >= VM_MIN_DIRECT_ADDRESS) { tp = NULL; m = PHYS_TO_VM_PAGE(TLB_DIRECT_TO_PHYS(va)); - (void)vm_page_pa_tryrelock(pm, TLB_DIRECT_TO_PHYS(va), - &pa); - vm_page_wire(m); + if (!vm_page_wire_mapped(m)) + m = NULL; } else { tp = tsb_kvtotte(va); if ((tp->tte_data & TD_V) == 0) @@ -868,12 +864,10 @@ pmap_extract_and_hold(pmap_t pm, vm_offset_t va, v tp = tsb_tte_lookup(pm, va); if (tp != NULL && ((tp->tte_data & TD_SW) || (prot & VM_PROT_WRITE) == 0)) { - if (vm_page_pa_tryrelock(pm, TTE_GET_PA(tp), &pa)) - goto retry; m = PHYS_TO_VM_PAGE(TTE_GET_PA(tp)); - vm_page_wire(m); + if (!vm_page_wire_mapped(m)) + m = NULL; } - PA_UNLOCK_COND(pa); PMAP_UNLOCK(pm); return (m); } @@ -2127,7 +2121,7 @@ pmap_is_modified(vm_page_t m) * is clear, no TTEs can have TD_W set. */ VM_OBJECT_ASSERT_WLOCKED(m->object); - if (!vm_page_xbusied(m) && (m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_xbusied(m) && !vm_page_aflag_isset(m, PGA_WRITEABLE)) return (rv); rw_wlock(&tte_list_global_lock); TAILQ_FOREACH(tp, &m->md.tte_list, tte_link) { @@ -2210,7 +2204,7 @@ pmap_clear_modify(vm_page_t m) * If the object containing the page is locked and the page is not * exclusive busied, then PGA_WRITEABLE cannot be concurrently set. */ - if ((m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_aflag_isset(m, PGA_WRITEABLE)) return; rw_wlock(&tte_list_global_lock); TAILQ_FOREACH(tp, &m->md.tte_list, tte_link) { @@ -2238,7 +2232,7 @@ pmap_remove_write(vm_page_t m) * if PGA_WRITEABLE is clear, no page table entries need updating. */ VM_OBJECT_ASSERT_WLOCKED(m->object); - if (!vm_page_xbusied(m) && (m->aflags & PGA_WRITEABLE) == 0) + if (!vm_page_xbusied(m) && !vm_page_aflag_isset(m, PGA_WRITEABLE)) return; rw_wlock(&tte_list_global_lock); TAILQ_FOREACH(tp, &m->md.tte_list, tte_link) { Index: src/sys/vm/device_pager.c =================================================================== --- src/sys/vm/device_pager.c (revision 351563) +++ src/sys/vm/device_pager.c (working copy) @@ -235,9 +235,7 @@ cdev_pager_free_page(vm_object_t object, vm_page_t if (object->type == OBJT_MGTDEVICE) { KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("unmanaged %p", m)); pmap_remove_all(m); - vm_page_lock(m); (void)vm_page_remove(m); - vm_page_unlock(m); } else if (object->type == OBJT_DEVICE) dev_pager_free_page(object, m); } Index: src/sys/vm/swap_pager.c =================================================================== --- src/sys/vm/swap_pager.c (revision 351563) +++ src/sys/vm/swap_pager.c (working copy) @@ -1641,12 +1641,6 @@ swp_pager_force_dirty(vm_page_t m) { vm_page_dirty(m); -#ifdef INVARIANTS - vm_page_lock(m); - if (!vm_page_wired(m) && m->queue == PQ_NONE) - panic("page %p is neither wired nor queued", m); - vm_page_unlock(m); -#endif vm_page_xunbusy(m); swap_pager_unswapped(m); } Index: src/sys/vm/vm_fault.c =================================================================== --- src/sys/vm/vm_fault.c (revision 351563) +++ src/sys/vm/vm_fault.c (working copy) @@ -186,9 +186,7 @@ unlock_and_deallocate(struct faultstate *fs) VM_OBJECT_WUNLOCK(fs->object); if (fs->object != fs->first_object) { VM_OBJECT_WLOCK(fs->first_object); - vm_page_lock(fs->first_m); vm_page_free(fs->first_m); - vm_page_unlock(fs->first_m); vm_object_pip_wakeup(fs->first_object); VM_OBJECT_WUNLOCK(fs->first_object); fs->first_m = NULL; @@ -263,18 +261,6 @@ vm_fault_dirty(vm_map_entry_t entry, vm_page_t m, vm_pager_page_unswapped(m); } -static void -vm_fault_fill_hold(vm_page_t *m_hold, vm_page_t m) -{ - - if (m_hold != NULL) { - *m_hold = m; - vm_page_lock(m); - vm_page_wire(m); - vm_page_unlock(m); - } -} - /* * Unlocks fs.first_object and fs.map on success. */ @@ -335,7 +321,10 @@ vm_fault_soft_fast(struct faultstate *fs, vm_offse PMAP_ENTER_NOSLEEP | (wired ? PMAP_ENTER_WIRED : 0), psind); if (rv != KERN_SUCCESS) return (rv); - vm_fault_fill_hold(m_hold, m); + if (m_hold != NULL) { + *m_hold = m; + vm_page_wire(m); + } vm_fault_dirty(fs->entry, m, prot, fault_type, fault_flags, false); if (psind == 0 && !wired) vm_fault_prefault(fs, vaddr, PFBAK, PFFOR, true); @@ -511,11 +500,12 @@ vm_fault_populate(struct faultstate *fs, vm_prot_t VM_OBJECT_WLOCK(fs->first_object); m_mtx = NULL; for (i = 0; i < npages; i++) { - vm_page_change_lock(&m[i], &m_mtx); - if ((fault_flags & VM_FAULT_WIRE) != 0) + if ((fault_flags & VM_FAULT_WIRE) != 0) { vm_page_wire(&m[i]); - else + } else { + vm_page_change_lock(&m[i], &m_mtx); vm_page_activate(&m[i]); + } if (m_hold != NULL && m[i].pindex == fs->first_pindex) { *m_hold = &m[i]; vm_page_wire(&m[i]); @@ -576,7 +566,6 @@ vm_fault_hold(vm_map_t map, vm_offset_t vaddr, vm_ struct faultstate fs; struct vnode *vp; struct domainset *dset; - struct mtx *mtx; vm_object_t next_object, retry_object; vm_offset_t e_end, e_start; vm_pindex_t retry_pindex; @@ -745,9 +734,7 @@ RetryFault_oom: VM_OBJECT_WLOCK(fs.first_object); VM_OBJECT_WLOCK(fs.object); } - vm_page_lock(fs.first_m); vm_page_free(fs.first_m); - vm_page_unlock(fs.first_m); vm_object_pip_wakeup(fs.first_object); VM_OBJECT_WUNLOCK(fs.first_object); fs.first_m = NULL; @@ -1030,12 +1017,10 @@ readrest: * an error. */ if (rv == VM_PAGER_ERROR || rv == VM_PAGER_BAD) { - vm_page_lock(fs.m); if (!vm_page_wired(fs.m)) vm_page_free(fs.m); else - vm_page_xunbusy_maybelocked(fs.m); - vm_page_unlock(fs.m); + vm_page_xunbusy(fs.m); fs.m = NULL; unlock_and_deallocate(&fs); return (rv == VM_PAGER_ERROR ? KERN_FAILURE : @@ -1053,12 +1038,10 @@ readrest: * that we are. */ if (fs.object != fs.first_object) { - vm_page_lock(fs.m); if (!vm_page_wired(fs.m)) vm_page_free(fs.m); else - vm_page_xunbusy_maybelocked(fs.m); - vm_page_unlock(fs.m); + vm_page_xunbusy(fs.m); fs.m = NULL; } } @@ -1169,23 +1152,11 @@ readrest: * We don't chase down the shadow chain */ fs.object == fs.first_object->backing_object) { - /* - * Keep the page wired to ensure that it is not - * freed by another thread, such as the page - * daemon, while it is disassociated from an - * object. - */ - mtx = NULL; - vm_page_change_lock(fs.m, &mtx); - vm_page_wire(fs.m); + (void)vm_page_remove(fs.m); - vm_page_change_lock(fs.first_m, &mtx); vm_page_replace_checked(fs.m, fs.first_object, fs.first_pindex, fs.first_m); vm_page_free(fs.first_m); - vm_page_change_lock(fs.m, &mtx); - vm_page_unwire(fs.m, PQ_ACTIVE); - mtx_unlock(mtx); vm_page_dirty(fs.m); #if VM_NRESERVLEVEL > 0 /* @@ -1211,13 +1182,8 @@ readrest: fs.first_m->valid = VM_PAGE_BITS_ALL; if (wired && (fault_flags & VM_FAULT_WIRE) == 0) { - vm_page_lock(fs.first_m); vm_page_wire(fs.first_m); - vm_page_unlock(fs.first_m); - - vm_page_lock(fs.m); vm_page_unwire(fs.m, PQ_INACTIVE); - vm_page_unlock(fs.m); } /* * We no longer need the old page or object. @@ -1350,21 +1316,22 @@ readrest: faultcount > 0 ? behind : PFBAK, faultcount > 0 ? ahead : PFFOR, false); VM_OBJECT_WLOCK(fs.object); - vm_page_lock(fs.m); /* * If the page is not wired down, then put it where the pageout daemon * can find it. */ - if ((fault_flags & VM_FAULT_WIRE) != 0) + if ((fault_flags & VM_FAULT_WIRE) != 0) { vm_page_wire(fs.m); - else + } else { + vm_page_lock(fs.m); vm_page_activate(fs.m); + vm_page_unlock(fs.m); + } if (m_hold != NULL) { *m_hold = fs.m; vm_page_wire(fs.m); } - vm_page_unlock(fs.m); vm_page_xunbusy(fs.m); /* @@ -1633,13 +1600,8 @@ vm_fault_quick_hold_pages(vm_map_t map, vm_offset_ return (count); error: for (mp = ma; mp < ma + count; mp++) - if (*mp != NULL) { - vm_page_lock(*mp); - if (vm_page_unwire(*mp, PQ_INACTIVE) && - (*mp)->object == NULL) - vm_page_free(*mp); - vm_page_unlock(*mp); - } + if (*mp != NULL) + vm_page_unwire(*mp, PQ_INACTIVE); return (-1); } @@ -1835,12 +1797,8 @@ again: if (upgrade) { if (src_m != dst_m) { - vm_page_lock(src_m); vm_page_unwire(src_m, PQ_INACTIVE); - vm_page_unlock(src_m); - vm_page_lock(dst_m); vm_page_wire(dst_m); - vm_page_unlock(dst_m); } else { KASSERT(vm_page_wired(dst_m), ("dst_m %p is not wired", dst_m)); Index: src/sys/vm/vm_glue.c =================================================================== --- src/sys/vm/vm_glue.c (revision 351563) +++ src/sys/vm/vm_glue.c (working copy) @@ -229,10 +229,8 @@ vm_imgact_hold_page(vm_object_t object, vm_ooffset vm_page_xbusy(m); rv = vm_pager_get_pages(object, &m, 1, NULL, NULL); if (rv != VM_PAGER_OK) { - vm_page_lock(m); - vm_page_unwire(m, PQ_NONE); + vm_page_unwire_noq(m); vm_page_free(m); - vm_page_unlock(m); m = NULL; goto out; } @@ -270,9 +268,7 @@ vm_imgact_unmap_page(struct sf_buf *sf) m = sf_buf_page(sf); sf_buf_free(sf); sched_unpin(); - vm_page_lock(m); vm_page_unwire(m, PQ_ACTIVE); - vm_page_unlock(m); } void @@ -380,10 +376,8 @@ vm_thread_stack_dispose(vm_object_t ksobj, vm_offs m = vm_page_lookup(ksobj, i); if (m == NULL) panic("vm_thread_dispose: kstack already missing?"); - vm_page_lock(m); vm_page_unwire_noq(m); vm_page_free(m); - vm_page_unlock(m); } VM_OBJECT_WUNLOCK(ksobj); vm_object_deallocate(ksobj); Index: src/sys/vm/vm_object.c =================================================================== --- src/sys/vm/vm_object.c (revision 351563) +++ src/sys/vm/vm_object.c (working copy) @@ -694,12 +694,9 @@ static void vm_object_terminate_pages(vm_object_t object) { vm_page_t p, p_next; - struct mtx *mtx; VM_OBJECT_ASSERT_WLOCKED(object); - mtx = NULL; - /* * Free any remaining pageable pages. This also removes them from the * paging queues. However, don't free wired pages, just remove them @@ -708,20 +705,16 @@ vm_object_terminate_pages(vm_object_t object) */ TAILQ_FOREACH_SAFE(p, &object->memq, listq, p_next) { vm_page_assert_unbusied(p); - if ((object->flags & OBJ_UNMANAGED) == 0) - /* - * vm_page_free_prep() only needs the page - * lock for managed pages. - */ - vm_page_change_lock(p, &mtx); + KASSERT(p->object == object && + (p->ref_count & VPRC_OBJREF) != 0, + ("vm_object_terminate_pages: page %p is inconsistent", p)); + p->object = NULL; - if (vm_page_wired(p)) - continue; - VM_CNT_INC(v_pfree); - vm_page_free(p); + if (vm_page_drop(p, VPRC_OBJREF) == VPRC_OBJREF) { + VM_CNT_INC(v_pfree); + vm_page_free(p); + } } - if (mtx != NULL) - mtx_unlock(mtx); /* * If the object contained any pages, then reset it to an empty state. @@ -1178,13 +1171,9 @@ next_page: /* * If the page is not in a normal state, skip it. */ - if (tm->valid != VM_PAGE_BITS_ALL) + if (tm->valid != VM_PAGE_BITS_ALL || + vm_page_wired(tm)) goto next_pindex; - vm_page_lock(tm); - if (vm_page_wired(tm)) { - vm_page_unlock(tm); - goto next_pindex; - } KASSERT((tm->flags & PG_FICTITIOUS) == 0, ("vm_object_madvise: page %p is fictitious", tm)); KASSERT((tm->oflags & VPO_UNMANAGED) == 0, @@ -1192,6 +1181,7 @@ next_page: if (vm_page_busied(tm)) { if (object != tobject) VM_OBJECT_WUNLOCK(tobject); + vm_page_lock(tm); VM_OBJECT_WUNLOCK(object); if (advice == MADV_WILLNEED) { /* @@ -1204,6 +1194,7 @@ next_page: vm_page_busy_sleep(tm, "madvpo", false); goto relookup; } + vm_page_lock(tm); vm_page_advise(tm, advice); vm_page_unlock(tm); vm_object_madvise_freespace(tobject, advice, tm->pindex, 1); @@ -1557,16 +1548,10 @@ vm_object_collapse_scan(vm_object_t object, int op swap_pager_freespace(backing_object, p->pindex, 1); - /* - * Page is out of the parent object's range, we can - * simply destroy it. - */ - vm_page_lock(p); KASSERT(!pmap_page_is_mapped(p), ("freeing mapped page %p", p)); if (vm_page_remove(p)) vm_page_free(p); - vm_page_unlock(p); continue; } @@ -1603,12 +1588,10 @@ vm_object_collapse_scan(vm_object_t object, int op if (backing_object->type == OBJT_SWAP) swap_pager_freespace(backing_object, p->pindex, 1); - vm_page_lock(p); KASSERT(!pmap_page_is_mapped(p), ("freeing mapped page %p", p)); if (vm_page_remove(p)) vm_page_free(p); - vm_page_unlock(p); continue; } @@ -1909,7 +1892,14 @@ again: VM_OBJECT_WLOCK(object); goto again; } + if (vm_page_busied(p)) { + VM_OBJECT_WUNLOCK(object); + vm_page_busy_sleep(p, "vmopar", false); + VM_OBJECT_WLOCK(object); + goto again; + } if (vm_page_wired(p)) { +wired: if ((options & OBJPR_NOTMAPPED) == 0 && object->ref_count != 0) pmap_remove_all(p); @@ -1919,23 +1909,19 @@ again: } continue; } - if (vm_page_busied(p)) { - VM_OBJECT_WUNLOCK(object); - vm_page_busy_sleep(p, "vmopar", false); - VM_OBJECT_WLOCK(object); - goto again; - } KASSERT((p->flags & PG_FICTITIOUS) == 0, ("vm_object_page_remove: page %p is fictitious", p)); if ((options & OBJPR_CLEANONLY) != 0 && p->valid != 0) { if ((options & OBJPR_NOTMAPPED) == 0 && - object->ref_count != 0) - pmap_remove_write(p); + object->ref_count != 0 && + !vm_page_try_remove_write(p)) + goto wired; if (p->dirty != 0) continue; } - if ((options & OBJPR_NOTMAPPED) == 0 && object->ref_count != 0) - pmap_remove_all(p); + if ((options & OBJPR_NOTMAPPED) == 0 && + object->ref_count != 0 && !vm_page_try_remove_all(p)) + goto wired; vm_page_free(p); } if (mtx != NULL) @@ -2225,8 +2211,8 @@ again: tm = m; m = TAILQ_NEXT(m, listq); } - vm_page_lock(tm); if (vm_page_xbusied(tm)) { + vm_page_lock(tm); for (tobject = object; locked_depth >= 1; locked_depth--) { t1object = tobject->backing_object; @@ -2237,7 +2223,6 @@ again: goto again; } vm_page_unwire(tm, queue); - vm_page_unlock(tm); next_page: pindex++; } Index: src/sys/vm/vm_page.c =================================================================== --- src/sys/vm/vm_page.c (revision 351563) +++ src/sys/vm/vm_page.c (working copy) @@ -168,6 +168,7 @@ static int vm_page_insert_after(vm_page_t m, vm_ob vm_pindex_t pindex, vm_page_t mpred); static void vm_page_insert_radixdone(vm_page_t m, vm_object_t object, vm_page_t mpred); +static void vm_page_mvqueue(vm_page_t m, uint8_t queue); static int vm_page_reclaim_run(int req_class, int domain, u_long npages, vm_page_t m_run, vm_paddr_t high); static int vm_domain_alloc_fail(struct vm_domain *vmd, vm_object_t object, @@ -513,7 +514,7 @@ vm_page_init_page(vm_page_t m, vm_paddr_t pa, int { m->object = NULL; - m->wire_count = 0; + m->ref_count = 0; m->busy_lock = VPB_UNBUSIED; m->flags = m->aflags = 0; m->phys_addr = pa; @@ -1107,17 +1108,11 @@ vm_page_change_lock(vm_page_t m, struct mtx **mtx) void vm_page_unhold_pages(vm_page_t *ma, int count) { - struct mtx *mtx; - mtx = NULL; for (; count != 0; count--) { - vm_page_change_lock(*ma, &mtx); - if (vm_page_unwire(*ma, PQ_ACTIVE) && (*ma)->object == NULL) - vm_page_free(*ma); + vm_page_unwire(*ma, PQ_ACTIVE); ma++; } - if (mtx != NULL) - mtx_unlock(mtx); } vm_page_t @@ -1180,7 +1175,8 @@ vm_page_initfake(vm_page_t m, vm_paddr_t paddr, vm /* Fictitious pages don't use "order" or "pool". */ m->oflags = VPO_UNMANAGED; m->busy_lock = VPB_SINGLE_EXCLUSIVER; - m->wire_count = 1; + /* Fictitious pages are unevictable. */ + m->ref_count = 1; pmap_page_init(m); memattr: pmap_page_set_memattr(m, memattr); @@ -1375,10 +1371,11 @@ vm_page_insert_after(vm_page_t m, vm_object_t obje ("vm_page_insert_after: msucc doesn't succeed pindex")); /* - * Record the object/offset pair in this page + * Record the object/offset pair in this page. */ m->object = object; m->pindex = pindex; + m->ref_count |= VPRC_OBJREF; /* * Now link into the object's ordered list of backed pages. @@ -1386,6 +1383,7 @@ vm_page_insert_after(vm_page_t m, vm_object_t obje if (vm_radix_insert(&object->rtree, m)) { m->object = NULL; m->pindex = 0; + m->ref_count &= ~VPRC_OBJREF; return (1); } vm_page_insert_radixdone(m, object, mpred); @@ -1410,11 +1408,13 @@ vm_page_insert_radixdone(vm_page_t m, vm_object_t VM_OBJECT_ASSERT_WLOCKED(object); KASSERT(object != NULL && m->object == object, ("vm_page_insert_radixdone: page %p has inconsistent object", m)); + KASSERT((m->ref_count & VPRC_OBJREF) != 0, + ("vm_page_insert_radixdone: page %p is missing object ref", m)); if (mpred != NULL) { KASSERT(mpred->object == object, - ("vm_page_insert_after: object doesn't contain mpred")); + ("vm_page_insert_radixdone: object doesn't contain mpred")); KASSERT(mpred->pindex < m->pindex, - ("vm_page_insert_after: mpred doesn't precede pindex")); + ("vm_page_insert_radixdone: mpred doesn't precede pindex")); } if (mpred != NULL) @@ -1442,25 +1442,19 @@ vm_page_insert_radixdone(vm_page_t m, vm_object_t } /* - * vm_page_remove: - * - * Removes the specified page from its containing object, but does not - * invalidate any backing storage. Return true if the page may be safely - * freed and false otherwise. - * - * The object must be locked. The page must be locked if it is managed. + * Do the work to remove a page from its object. The caller is responsible for + * updating the page's fields to reflect this removal. */ -bool -vm_page_remove(vm_page_t m) +static void +vm_page_object_remove(vm_page_t m) { vm_object_t object; vm_page_t mrem; object = m->object; - - if ((m->oflags & VPO_UNMANAGED) == 0) - vm_page_assert_locked(m); VM_OBJECT_ASSERT_WLOCKED(object); + KASSERT((m->ref_count & VPRC_OBJREF) != 0, + ("page %p is missing its object ref", m)); if (vm_page_xbusied(m)) vm_page_xunbusy_maybelocked(m); mrem = vm_radix_remove(&object->rtree, m->pindex); @@ -1481,9 +1475,24 @@ vm_page_insert_radixdone(vm_page_t m, vm_object_t */ if (object->resident_page_count == 0 && object->type == OBJT_VNODE) vdrop(object->handle); +} +/* + * vm_page_remove: + * + * Removes the specified page from its containing object, but does not + * invalidate any backing storage. Returns true if the object's reference + * was the last reference to the page, and false otherwise. + * + * The object must be locked. + */ +bool +vm_page_remove(vm_page_t m) +{ + + vm_page_object_remove(m); m->object = NULL; - return (!vm_page_wired(m)); + return (vm_page_drop(m, VPRC_OBJREF) == VPRC_OBJREF); } /* @@ -1564,8 +1573,6 @@ vm_page_prev(vm_page_t m) /* * Uses the page mnew as a replacement for an existing page at index * pindex which must be already present in the object. - * - * The existing page must not be on a paging queue. */ vm_page_t vm_page_replace(vm_page_t mnew, vm_object_t object, vm_pindex_t pindex) @@ -1573,10 +1580,8 @@ vm_page_replace(vm_page_t mnew, vm_object_t object vm_page_t mold; VM_OBJECT_ASSERT_WLOCKED(object); - KASSERT(mnew->object == NULL, + KASSERT(mnew->object == NULL && (mnew->ref_count & VPRC_OBJREF) == 0, ("vm_page_replace: page %p already in object", mnew)); - KASSERT(mnew->queue == PQ_NONE || vm_page_wired(mnew), - ("vm_page_replace: new page %p is on a paging queue", mnew)); /* * This function mostly follows vm_page_insert() and @@ -1586,6 +1591,7 @@ vm_page_replace(vm_page_t mnew, vm_object_t object mnew->object = object; mnew->pindex = pindex; + atomic_set_int(&mnew->ref_count, VPRC_OBJREF); mold = vm_radix_replace(&object->rtree, mnew); KASSERT(mold->queue == PQ_NONE, ("vm_page_replace: old page %p is on a paging queue", mold)); @@ -1595,6 +1601,7 @@ vm_page_replace(vm_page_t mnew, vm_object_t object TAILQ_REMOVE(&object->memq, mold, listq); mold->object = NULL; + atomic_clear_int(&mold->ref_count, VPRC_OBJREF); vm_page_xunbusy_maybelocked(mold); /* @@ -1632,6 +1639,7 @@ vm_page_rename(vm_page_t m, vm_object_t new_object VM_OBJECT_ASSERT_WLOCKED(new_object); + KASSERT(m->ref_count != 0, ("vm_page_rename: page %p has no refs", m)); mpred = vm_radix_lookup_le(&new_object->rtree, new_pindex); KASSERT(mpred == NULL || mpred->pindex != new_pindex, ("vm_page_rename: pindex already renamed")); @@ -1653,13 +1661,12 @@ vm_page_rename(vm_page_t m, vm_object_t new_object * the listq iterator is tainted. */ m->pindex = opidx; - vm_page_lock(m); - (void)vm_page_remove(m); + vm_page_object_remove(m); /* Return back to the new pindex to complete vm_page_insert(). */ m->pindex = new_pindex; m->object = new_object; - vm_page_unlock(m); + vm_page_insert_radixdone(m, new_object, mpred); vm_page_dirty(m); return (0); @@ -1876,7 +1883,7 @@ found: * page is inserted into the object. */ vm_wire_add(1); - m->wire_count = 1; + m->ref_count = 1; } m->act_count = 0; @@ -1884,7 +1891,7 @@ found: if (vm_page_insert_after(m, object, pindex, mpred)) { if (req & VM_ALLOC_WIRED) { vm_wire_sub(1); - m->wire_count = 0; + m->ref_count = 0; } KASSERT(m->object == NULL, ("page %p has object", m)); m->oflags = VPO_UNMANAGED; @@ -2076,7 +2083,7 @@ found: m->flags = (m->flags | PG_NODUMP) & flags; m->busy_lock = busy_lock; if ((req & VM_ALLOC_WIRED) != 0) - m->wire_count = 1; + m->ref_count = 1; m->act_count = 0; m->oflags = oflags; if (object != NULL) { @@ -2089,7 +2096,7 @@ found: for (m = m_ret; m < &m_ret[npages]; m++) { if (m <= mpred && (req & VM_ALLOC_WIRED) != 0) - m->wire_count = 0; + m->ref_count = 0; m->oflags = VPO_UNMANAGED; m->busy_lock = VPB_UNBUSIED; /* Don't change PG_ZERO. */ @@ -2123,7 +2130,7 @@ vm_page_alloc_check(vm_page_t m) KASSERT(m->queue == PQ_NONE && (m->aflags & PGA_QUEUE_STATE_MASK) == 0, ("page %p has unexpected queue %d, flags %#x", m, m->queue, (m->aflags & PGA_QUEUE_STATE_MASK))); - KASSERT(!vm_page_wired(m), ("page %p is wired", m)); + KASSERT(m->ref_count == 0, ("page %p has references", m)); KASSERT(!vm_page_busied(m), ("page %p is busy", m)); KASSERT(m->dirty == 0, ("page %p is dirty", m)); KASSERT(pmap_page_get_memattr(m) == VM_MEMATTR_DEFAULT, @@ -2207,7 +2214,7 @@ again: * not belong to an object. */ vm_wire_add(1); - m->wire_count = 1; + m->ref_count = 1; } /* Unmanaged pages don't use "act_count". */ m->oflags = VPO_UNMANAGED; @@ -2300,8 +2307,8 @@ vm_page_scan_contig(u_long npages, vm_page_t m_sta for (m = m_start; m < m_end && run_len < npages; m += m_inc) { KASSERT((m->flags & PG_MARKER) == 0, ("page %p is PG_MARKER", m)); - KASSERT((m->flags & PG_FICTITIOUS) == 0 || m->wire_count == 1, - ("fictitious page %p has invalid wire count", m)); + KASSERT((m->flags & PG_FICTITIOUS) == 0 || m->ref_count >= 1, + ("fictitious page %p has invalid ref count", m)); /* * If the current page would be the start of a run, check its @@ -2358,9 +2365,6 @@ retry: */ VM_OBJECT_RUNLOCK(object); goto retry; - } else if (vm_page_wired(m)) { - run_ext = 0; - goto unlock; } } /* Don't care: PG_NODUMP, PG_ZERO. */ @@ -2378,7 +2382,8 @@ retry: vm_reserv_size(level)) - pa); #endif } else if (object->memattr == VM_MEMATTR_DEFAULT && - vm_page_queue(m) != PQ_NONE && !vm_page_busied(m)) { + vm_page_queue(m) != PQ_NONE && !vm_page_busied(m) && + !vm_page_wired(m)) { /* * The page is allocated but eligible for * relocation. Extend the current run by one @@ -2394,7 +2399,6 @@ retry: run_ext = 1; } else run_ext = 0; -unlock: VM_OBJECT_RUNLOCK(object); #if VM_NRESERVLEVEL > 0 } else if (level >= 0) { @@ -2498,6 +2502,9 @@ vm_page_reclaim_run(int req_class, int domain, u_l */ vm_page_change_lock(m, &m_mtx); retry: + /* + * Racily check for wirings. Races are handled below. + */ if (vm_page_wired(m)) error = EBUSY; else if ((object = m->object) != NULL) { @@ -2515,9 +2522,6 @@ retry: */ VM_OBJECT_WUNLOCK(object); goto retry; - } else if (vm_page_wired(m)) { - error = EBUSY; - goto unlock; } } /* Don't care: PG_NODUMP, PG_ZERO. */ @@ -2528,7 +2532,7 @@ retry: else if (object->memattr != VM_MEMATTR_DEFAULT) error = EINVAL; else if (vm_page_queue(m) != PQ_NONE && - !vm_page_busied(m)) { + !vm_page_busied(m) && !vm_page_wired(m)) { KASSERT(pmap_page_get_memattr(m) == VM_MEMATTR_DEFAULT, ("page %p has an unexpected memattr", m)); @@ -2577,8 +2581,6 @@ retry: error = ENOMEM; goto unlock; } - KASSERT(!vm_page_wired(m_new), - ("page %p is wired", m_new)); /* * Replace "m" with the new page. For @@ -2586,8 +2588,11 @@ retry: * and dequeued. Finally, change "m" * as if vm_page_free() was called. */ - if (object->ref_count != 0) - pmap_remove_all(m); + if (object->ref_count != 0 && + !vm_page_try_remove_all(m)) { + error = EBUSY; + goto unlock; + } m_new->aflags = m->aflags & ~PGA_QUEUE_STATE_MASK; KASSERT(m_new->oflags == VPO_UNMANAGED, @@ -3134,6 +3139,13 @@ vm_pqbatch_process(struct vm_pagequeue *pq, struct vm_batchqueue_init(bq); } +/* + * vm_page_pqbatch_submit: [ internal use only ] + * + * Enqueue a page in the specified page queue's batched work queue. + * The caller must have encoded the requested operation in the page + * structure's aflags field. + */ void vm_page_pqbatch_submit(vm_page_t m, uint8_t queue) { @@ -3143,8 +3155,7 @@ vm_page_pqbatch_submit(vm_page_t m, uint8_t queue) KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("page %p is unmanaged", m)); - KASSERT(mtx_owned(vm_page_lockptr(m)) || - (m->object == NULL && (m->aflags & PGA_DEQUEUE) != 0), + KASSERT(mtx_owned(vm_page_lockptr(m)) || m->object == NULL, ("missing synchronization for page %p", m)); KASSERT(queue < PQ_COUNT, ("invalid queue %d", queue)); @@ -3255,17 +3266,26 @@ vm_page_dequeue_deferred(vm_page_t m) if ((queue = vm_page_queue(m)) == PQ_NONE) return; - vm_page_aflag_set(m, PGA_DEQUEUE); - vm_page_pqbatch_submit(m, queue); + + /* + * Set PGA_DEQUEUE if it is not already set to handle a concurrent call + * to vm_page_dequeue_deferred_free(). In particular, avoid modifying + * the page's queue state once vm_page_dequeue_deferred_free() has been + * called. In the event of a race, two batch queue entries for the page + * will be created, but the second will have no effect. + */ + if (vm_page_pqstate_cmpset(m, queue, queue, PGA_DEQUEUE, PGA_DEQUEUE)) + vm_page_pqbatch_submit(m, queue); } /* * A variant of vm_page_dequeue_deferred() that does not assert the page - * lock and is only to be called from vm_page_free_prep(). It is just an - * open-coded implementation of vm_page_dequeue_deferred(). Because the - * page is being freed, we can assume that nothing else is scheduling queue - * operations on this page, so we get for free the mutual exclusion that - * is otherwise provided by the page lock. + * lock and is only to be called from vm_page_free_prep(). Because the + * page is being freed, we can assume that nothing other than the page + * daemon is scheduling queue operations on this page, so we get for + * free the mutual exclusion that is otherwise provided by the page lock. + * To handle races, the page daemon must take care to atomically check + * for PGA_DEQUEUE when updating queue state. */ static void vm_page_dequeue_deferred_free(vm_page_t m) @@ -3272,7 +3292,7 @@ vm_page_dequeue_deferred_free(vm_page_t m) { uint8_t queue; - KASSERT(m->object == NULL, ("page %p has an object reference", m)); + KASSERT(m->ref_count == 0, ("page %p has references", m)); if ((m->aflags & PGA_DEQUEUE) != 0) return; @@ -3379,6 +3399,42 @@ vm_page_requeue(vm_page_t m) } /* + * vm_page_swapqueue: [ internal use only ] + * + * Move the page from one queue to another, or to the tail of its + * current queue, in the face of a possible concurrent call to + * vm_page_dequeue_deferred_free(). + */ +void +vm_page_swapqueue(vm_page_t m, uint8_t oldq, uint8_t newq) +{ + struct vm_pagequeue *pq; + + KASSERT(oldq < PQ_COUNT && newq < PQ_COUNT && oldq != newq, + ("vm_page_swapqueue: invalid queues (%d, %d)", oldq, newq)); + KASSERT((m->oflags & VPO_UNMANAGED) == 0, + ("vm_page_swapqueue: page %p is unmanaged", m)); + vm_page_assert_locked(m); + + /* + * Atomically update the queue field and set PGA_REQUEUE while + * ensuring that PGA_DEQUEUE has not been set. + */ + pq = &vm_pagequeue_domain(m)->vmd_pagequeues[oldq]; + vm_pagequeue_lock(pq); + if (!vm_page_pqstate_cmpset(m, oldq, newq, PGA_DEQUEUE, PGA_REQUEUE)) { + vm_pagequeue_unlock(pq); + return; + } + if ((m->aflags & PGA_ENQUEUED) != 0) { + vm_pagequeue_remove(pq, m); + vm_page_aflag_clear(m, PGA_ENQUEUED); + } + vm_pagequeue_unlock(pq); + vm_page_pqbatch_submit(m, newq); +} + +/* * vm_page_free_prep: * * Prepares the given page to be put on the free list, @@ -3392,6 +3448,12 @@ bool vm_page_free_prep(vm_page_t m) { + /* + * Synchronize with threads that have dropped a reference to this + * page. + */ + atomic_thread_fence_acq(); + #if defined(DIAGNOSTIC) && defined(PHYS_TO_DMAP) if (PMAP_HAS_DMAP && (m->flags & PG_ZERO) != 0) { uint64_t *p; @@ -3402,11 +3464,10 @@ vm_page_free_prep(vm_page_t m) m, i, (uintmax_t)*p)); } #endif - if ((m->oflags & VPO_UNMANAGED) == 0) { - vm_page_lock_assert(m, MA_OWNED); + if ((m->oflags & VPO_UNMANAGED) == 0) KASSERT(!pmap_page_is_mapped(m), ("vm_page_free_prep: freeing mapped page %p", m)); - } else + else KASSERT(m->queue == PQ_NONE, ("vm_page_free_prep: unmanaged page %p is queued", m)); VM_CNT_INC(v_tfree); @@ -3414,16 +3475,28 @@ vm_page_free_prep(vm_page_t m) if (vm_page_sbusied(m)) panic("vm_page_free_prep: freeing busy page %p", m); - if (m->object != NULL) - (void)vm_page_remove(m); + if (m->object != NULL) { + vm_page_object_remove(m); + /* + * The object reference can be released without an atomic + * operation. + */ + KASSERT((m->flags & PG_FICTITIOUS) != 0 || + m->ref_count == VPRC_OBJREF, + ("vm_page_free_prep: page %p has unexpected ref_count %u", + m, m->ref_count)); + m->object = NULL; + m->ref_count = 0; + } + /* * If fictitious remove object association and * return. */ if ((m->flags & PG_FICTITIOUS) != 0) { - KASSERT(m->wire_count == 1, - ("fictitious page %p is not wired", m)); + KASSERT(m->ref_count == 1, + ("fictitious page %p is referenced", m)); KASSERT(m->queue == PQ_NONE, ("fictitious page %p is queued", m)); return (false); @@ -3440,8 +3513,8 @@ vm_page_free_prep(vm_page_t m) m->valid = 0; vm_page_undirty(m); - if (vm_page_wired(m) != 0) - panic("vm_page_free_prep: freeing wired page %p", m); + if (m->ref_count != 0) + panic("vm_page_free_prep: page %p has references", m); /* * Restore the default memory attribute to the page. @@ -3523,161 +3596,193 @@ vm_page_free_pages_toq(struct spglist *free, bool } /* - * vm_page_wire: - * - * Mark this page as wired down. If the page is fictitious, then - * its wire count must remain one. - * - * The page must be locked. + * Mark this page as wired down, preventing reclamation by the page daemon + * or when the containing object is destroyed. */ void vm_page_wire(vm_page_t m) { + u_int old; - vm_page_assert_locked(m); - if ((m->flags & PG_FICTITIOUS) != 0) { - KASSERT(m->wire_count == 1, - ("vm_page_wire: fictitious page %p's wire count isn't one", - m)); - return; - } - if (!vm_page_wired(m)) { - KASSERT((m->oflags & VPO_UNMANAGED) == 0 || - m->queue == PQ_NONE, - ("vm_page_wire: unmanaged page %p is queued", m)); + KASSERT(m->object != NULL, + ("vm_page_wire: page %p does not belong to an object", m)); + if (!vm_page_busied(m)) + VM_OBJECT_ASSERT_LOCKED(m->object); + KASSERT((m->flags & PG_FICTITIOUS) == 0 || + VPRC_WIRE_COUNT(m->ref_count) >= 1, + ("vm_page_wire: fictitious page %p has zero wirings", m)); + + old = atomic_fetchadd_int(&m->ref_count, 1); + KASSERT(VPRC_WIRE_COUNT(old) != VPRC_WIRE_COUNT_MAX, + ("vm_page_wire: counter overflow for page %p", m)); + if (VPRC_WIRE_COUNT(old) == 0) vm_wire_add(1); - } - m->wire_count++; - KASSERT(m->wire_count != 0, ("vm_page_wire: wire_count overflow m=%p", m)); } /* - * vm_page_unwire: - * + * Attempt to wire a mapped page following a pmap lookup of that page. + * This may fail if a thread is concurrently tearing down mappings of the page. + */ +bool +vm_page_wire_mapped(vm_page_t m) +{ + u_int old; + + old = m->ref_count; + do { + KASSERT(old > 0, + ("vm_page_wire_mapped: wiring unreferenced page %p", m)); + if ((old & VPRC_BLOCKED) != 0) + return (false); + } while (!atomic_fcmpset_int(&m->ref_count, &old, old + 1)); + + if (VPRC_WIRE_COUNT(old) == 0) + vm_wire_add(1); + return (true); +} + +/* * Release one wiring of the specified page, potentially allowing it to be - * paged out. Returns TRUE if the number of wirings transitions to zero and - * FALSE otherwise. + * paged out. * * Only managed pages belonging to an object can be paged out. If the number * of wirings transitions to zero and the page is eligible for page out, then - * the page is added to the specified paging queue (unless PQ_NONE is - * specified, in which case the page is dequeued if it belongs to a paging - * queue). + * the page is added to the specified paging queue. If the released wiring + * represented the last reference to the page, the page is freed. * - * If a page is fictitious, then its wire count must always be one. - * * A managed page must be locked. */ -bool +void vm_page_unwire(vm_page_t m, uint8_t queue) { - bool unwired; + u_int old; + bool locked; - KASSERT(queue < PQ_COUNT || queue == PQ_NONE, - ("vm_page_unwire: invalid queue %u request for page %p", - queue, m)); - if ((m->oflags & VPO_UNMANAGED) == 0) - vm_page_assert_locked(m); + KASSERT(queue < PQ_COUNT, + ("vm_page_unwire: invalid queue %u request for page %p", queue, m)); - unwired = vm_page_unwire_noq(m); - if (!unwired || (m->oflags & VPO_UNMANAGED) != 0 || m->object == NULL) - return (unwired); + if ((m->oflags & VPO_UNMANAGED) != 0) { + if (vm_page_unwire_noq(m) && m->ref_count == 0) + vm_page_free(m); + return; + } - if (vm_page_queue(m) == queue) { - if (queue == PQ_ACTIVE) - vm_page_reference(m); - else if (queue != PQ_NONE) - vm_page_requeue(m); - } else { - vm_page_dequeue(m); - if (queue != PQ_NONE) { - vm_page_enqueue(m, queue); - if (queue == PQ_ACTIVE) - /* Initialize act_count. */ - vm_page_activate(m); + /* + * Update LRU state before releasing the wiring reference. + * We only need to do this once since we hold the page lock. + * Use a release store when updating the reference count to + * synchronize with vm_page_free_prep(). + */ + old = m->ref_count; + locked = false; + do { + KASSERT(VPRC_WIRE_COUNT(old) > 0, + ("vm_page_unwire: wire count underflow for page %p", m)); + if (!locked && VPRC_WIRE_COUNT(old) == 1) { + vm_page_lock(m); + locked = true; + if (queue == PQ_ACTIVE && vm_page_queue(m) == PQ_ACTIVE) + vm_page_reference(m); + else + vm_page_mvqueue(m, queue); } + } while (!atomic_fcmpset_rel_int(&m->ref_count, &old, old - 1)); + + /* + * Release the lock only after the wiring is released, to ensure that + * the page daemon does not encounter and dequeue the page while it is + * still wired. + */ + if (locked) + vm_page_unlock(m); + + if (VPRC_WIRE_COUNT(old) == 1) { + vm_wire_sub(1); + if (old == 1) + vm_page_free(m); } - return (unwired); } /* - * - * vm_page_unwire_noq: - * * Unwire a page without (re-)inserting it into a page queue. It is up * to the caller to enqueue, requeue, or free the page as appropriate. - * In most cases, vm_page_unwire() should be used instead. + * In most cases involving managed pages, vm_page_unwire() should be used + * instead. */ bool vm_page_unwire_noq(vm_page_t m) { + u_int old; - if ((m->oflags & VPO_UNMANAGED) == 0) - vm_page_assert_locked(m); - if ((m->flags & PG_FICTITIOUS) != 0) { - KASSERT(m->wire_count == 1, - ("vm_page_unwire: fictitious page %p's wire count isn't one", m)); + if ((m->oflags & VPO_UNMANAGED) != 0) + old = m->ref_count--; + else + old = vm_page_drop(m, 1); + KASSERT(VPRC_WIRE_COUNT(old) != 0, + ("vm_page_unref: counter underflow for page %p", m)); + KASSERT((m->flags & PG_FICTITIOUS) == 0 || VPRC_WIRE_COUNT(old) > 1, + ("vm_page_unref: missing ref on fictitious page %p", m)); + + if (VPRC_WIRE_COUNT(old) > 1) return (false); - } - if (!vm_page_wired(m)) - panic("vm_page_unwire: page %p's wire count is zero", m); - m->wire_count--; - if (m->wire_count == 0) { - vm_wire_sub(1); - return (true); - } else - return (false); + vm_wire_sub(1); + return (true); } /* - * vm_page_activate: + * Ensure that the page is in the specified page queue. If the page is + * active or being moved to the active queue, ensure that its act_count is + * at least ACT_INIT but do not otherwise mess with it. Otherwise, ensure that + * the page is at the tail of its page queue. * - * Put the specified page on the active list (if appropriate). - * Ensure that act_count is at least ACT_INIT but do not otherwise - * mess with it. + * The page may be wired. The caller should release its wiring reference + * before releasing the page lock, otherwise the page daemon may immediately + * dequeue the page. * - * The page must be locked. + * A managed page must be locked. */ -void -vm_page_activate(vm_page_t m) +static __always_inline void +vm_page_mvqueue(vm_page_t m, const uint8_t nqueue) { vm_page_assert_locked(m); + KASSERT((m->oflags & VPO_UNMANAGED) == 0, + ("vm_page_mvqueue: page %p is unmanaged", m)); - if (vm_page_wired(m) || (m->oflags & VPO_UNMANAGED) != 0) - return; - if (vm_page_queue(m) == PQ_ACTIVE) { - if (m->act_count < ACT_INIT) - m->act_count = ACT_INIT; - return; + if (vm_page_queue(m) != nqueue) { + vm_page_dequeue(m); + vm_page_enqueue(m, nqueue); + } else if (nqueue != PQ_ACTIVE) { + vm_page_requeue(m); } - vm_page_dequeue(m); - if (m->act_count < ACT_INIT) + if (nqueue == PQ_ACTIVE && m->act_count < ACT_INIT) m->act_count = ACT_INIT; - vm_page_enqueue(m, PQ_ACTIVE); } /* + * Put the specified page on the active list (if appropriate). + */ +void +vm_page_activate(vm_page_t m) +{ + + if ((m->oflags & VPO_UNMANAGED) != 0 || vm_page_wired(m)) + return; + vm_page_mvqueue(m, PQ_ACTIVE); +} + +/* * Move the specified page to the tail of the inactive queue, or requeue * the page if it is already in the inactive queue. - * - * The page must be locked. */ void vm_page_deactivate(vm_page_t m) { - vm_page_assert_locked(m); - - if (vm_page_wired(m) || (m->oflags & VPO_UNMANAGED) != 0) + if ((m->oflags & VPO_UNMANAGED) != 0 || vm_page_wired(m)) return; - - if (!vm_page_inactive(m)) { - vm_page_dequeue(m); - vm_page_enqueue(m, PQ_INACTIVE); - } else - vm_page_requeue(m); + vm_page_mvqueue(m, PQ_INACTIVE); } /* @@ -3685,18 +3790,13 @@ vm_page_deactivate(vm_page_t m) * bypassing LRU. A marker page is used to maintain FIFO ordering. * As with regular enqueues, we use a per-CPU batch queue to reduce * contention on the page queue lock. - * - * The page must be locked. */ -void -vm_page_deactivate_noreuse(vm_page_t m) +static void +_vm_page_deactivate_noreuse(vm_page_t m) { vm_page_assert_locked(m); - if (vm_page_wired(m) || (m->oflags & VPO_UNMANAGED) != 0) - return; - if (!vm_page_inactive(m)) { vm_page_dequeue(m); m->queue = PQ_INACTIVE; @@ -3706,31 +3806,31 @@ vm_page_deactivate(vm_page_t m) vm_page_pqbatch_submit(m, PQ_INACTIVE); } +void +vm_page_deactivate_noreuse(vm_page_t m) +{ + + KASSERT(m->object != NULL, + ("vm_page_deactivate_noreuse: page %p has no object", m)); + + if ((m->oflags & VPO_UNMANAGED) == 0 && !vm_page_wired(m)) + _vm_page_deactivate_noreuse(m); +} + /* - * vm_page_launder - * - * Put a page in the laundry, or requeue it if it is already there. + * Put a page in the laundry, or requeue it if it is already there. */ void vm_page_launder(vm_page_t m) { - vm_page_assert_locked(m); - if (vm_page_wired(m) || (m->oflags & VPO_UNMANAGED) != 0) + if ((m->oflags & VPO_UNMANAGED) != 0 || vm_page_wired(m)) return; - - if (vm_page_in_laundry(m)) - vm_page_requeue(m); - else { - vm_page_dequeue(m); - vm_page_enqueue(m, PQ_LAUNDRY); - } + vm_page_mvqueue(m, PQ_LAUNDRY); } /* - * vm_page_unswappable - * - * Put a page in the PQ_UNSWAPPABLE holding queue. + * Put a page in the PQ_UNSWAPPABLE holding queue. */ void vm_page_unswappable(vm_page_t m) @@ -3748,6 +3848,8 @@ static void vm_page_release_toq(vm_page_t m, int flags) { + vm_page_assert_locked(m); + /* * Use a check of the valid bits to determine whether we should * accelerate reclamation of the page. The object lock might not be @@ -3759,11 +3861,11 @@ vm_page_release_toq(vm_page_t m, int flags) * inactive queue so that is reclaimed sooner. */ if ((flags & (VPR_TRYFREE | VPR_NOREUSE)) != 0 || m->valid == 0) - vm_page_deactivate_noreuse(m); + _vm_page_deactivate_noreuse(m); else if (vm_page_active(m)) vm_page_reference(m); else - vm_page_deactivate(m); + vm_page_mvqueue(m, PQ_INACTIVE); } /* @@ -3773,41 +3875,63 @@ void vm_page_release(vm_page_t m, int flags) { vm_object_t object; - bool freed; + u_int old; + bool locked; KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("vm_page_release: page %p is unmanaged", m)); - vm_page_lock(m); - if (m->object != NULL) - VM_OBJECT_ASSERT_UNLOCKED(m->object); - if (vm_page_unwire_noq(m)) { - if ((object = m->object) == NULL) { - vm_page_free(m); - } else { - freed = false; - if ((flags & VPR_TRYFREE) != 0 && !vm_page_busied(m) && - /* Depends on type stability. */ - VM_OBJECT_TRYWLOCK(object)) { - /* - * Only free unmapped pages. The busy test from - * before the object was locked cannot be relied - * upon. - */ - if ((object->ref_count == 0 || - !pmap_page_is_mapped(m)) && m->dirty == 0 && - !vm_page_busied(m)) { - vm_page_free(m); - freed = true; - } - VM_OBJECT_WUNLOCK(object); + if ((flags & VPR_TRYFREE) != 0) { + for (;;) { + object = (vm_object_t)atomic_load_ptr(&m->object); + if (object == NULL) + break; + /* Depends on type-stability. */ + if (vm_page_busied(m) || !VM_OBJECT_TRYWLOCK(object)) { + object = NULL; + break; } + if (object == m->object) + break; + VM_OBJECT_WUNLOCK(object); + } + if (__predict_true(object != NULL)) { + vm_page_release_locked(m, flags); + VM_OBJECT_WUNLOCK(object); + return; + } + } - if (!freed) - vm_page_release_toq(m, flags); + /* + * Update LRU state before releasing the wiring reference. + * Use a release store when updating the reference count to + * synchronize with vm_page_free_prep(). + */ + old = m->ref_count; + locked = false; + do { + KASSERT(VPRC_WIRE_COUNT(old) > 0, + ("vm_page_unwire: wire count underflow for page %p", m)); + if (!locked && VPRC_WIRE_COUNT(old) == 1) { + vm_page_lock(m); + locked = true; + vm_page_release_toq(m, flags); } + } while (!atomic_fcmpset_rel_int(&m->ref_count, &old, old - 1)); + + /* + * Release the lock only after the wiring is released, to ensure that + * the page daemon does not encounter and dequeue the page while it is + * still wired. + */ + if (locked) + vm_page_unlock(m); + + if (VPRC_WIRE_COUNT(old) == 1) { + vm_wire_sub(1); + if (old == 1) + vm_page_free(m); } - vm_page_unlock(m); } /* See vm_page_release(). */ @@ -3819,7 +3943,6 @@ vm_page_release_locked(vm_page_t m, int flags) KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("vm_page_release_locked: page %p is unmanaged", m)); - vm_page_lock(m); if (vm_page_unwire_noq(m)) { if ((flags & VPR_TRYFREE) != 0 && (m->object->ref_count == 0 || !pmap_page_is_mapped(m)) && @@ -3826,13 +3949,67 @@ vm_page_release_locked(vm_page_t m, int flags) m->dirty == 0 && !vm_page_busied(m)) { vm_page_free(m); } else { + vm_page_lock(m); vm_page_release_toq(m, flags); + vm_page_unlock(m); } } - vm_page_unlock(m); } +static bool +vm_page_try_blocked_op(vm_page_t m, void (*op)(vm_page_t)) +{ + u_int old; + + KASSERT(m->object != NULL && (m->oflags & VPO_UNMANAGED) == 0, + ("vm_page_try_blocked_op: page %p has no object", m)); + KASSERT(!vm_page_busied(m), + ("vm_page_try_blocked_op: page %p is busy", m)); + VM_OBJECT_ASSERT_LOCKED(m->object); + + old = m->ref_count; + do { + KASSERT(old != 0, + ("vm_page_try_blocked_op: page %p has no references", m)); + if (VPRC_WIRE_COUNT(old) != 0) + return (false); + } while (!atomic_fcmpset_int(&m->ref_count, &old, old | VPRC_BLOCKED)); + + (op)(m); + + /* + * If the object is read-locked, new wirings may be created via an + * object lookup. + */ + old = vm_page_drop(m, VPRC_BLOCKED); + KASSERT(!VM_OBJECT_WOWNED(m->object) || + old == (VPRC_BLOCKED | VPRC_OBJREF), + ("vm_page_try_blocked_op: unexpected refcount value %u for %p", + old, m)); + return (true); +} + /* + * Atomically check for wirings and remove all mappings of the page. + */ +bool +vm_page_try_remove_all(vm_page_t m) +{ + + return (vm_page_try_blocked_op(m, pmap_remove_all)); +} + +/* + * Atomically check for wirings and remove all writeable mappings of the page. + */ +bool +vm_page_try_remove_write(vm_page_t m) +{ + + return (vm_page_try_blocked_op(m, pmap_remove_write)); +} + +/* * vm_page_advise * * Apply the specified advice to the given page. @@ -3926,11 +4103,8 @@ retrylookup: VM_OBJECT_WLOCK(object); goto retrylookup; } else { - if ((allocflags & VM_ALLOC_WIRED) != 0) { - vm_page_lock(m); + if ((allocflags & VM_ALLOC_WIRED) != 0) vm_page_wire(m); - vm_page_unlock(m); - } if ((allocflags & (VM_ALLOC_NOBUSY | VM_ALLOC_SBUSY)) == 0) vm_page_xbusy(m); @@ -4028,11 +4202,8 @@ retrylookup: VM_OBJECT_WLOCK(object); goto retrylookup; } - if ((allocflags & VM_ALLOC_WIRED) != 0) { - vm_page_lock(m); + if ((allocflags & VM_ALLOC_WIRED) != 0) vm_page_wire(m); - vm_page_unlock(m); - } if ((allocflags & (VM_ALLOC_NOBUSY | VM_ALLOC_SBUSY)) == 0) vm_page_xbusy(m); @@ -4561,10 +4732,10 @@ DB_SHOW_COMMAND(pginfo, vm_page_print_pginfo) else m = (vm_page_t)addr; db_printf( - "page %p obj %p pidx 0x%jx phys 0x%jx q %d wire %d\n" + "page %p obj %p pidx 0x%jx phys 0x%jx q %d ref %u\n" " af 0x%x of 0x%x f 0x%x act %d busy %x valid 0x%x dirty 0x%x\n", m, m->object, (uintmax_t)m->pindex, (uintmax_t)m->phys_addr, - m->queue, m->wire_count, m->aflags, m->oflags, + m->queue, m->ref_count, m->aflags, m->oflags, m->flags, m->act_count, m->busy_lock, m->valid, m->dirty); } #endif /* DDB */ Index: src/sys/vm/vm_page.h =================================================================== --- src/sys/vm/vm_page.h (revision 351563) +++ src/sys/vm/vm_page.h (working copy) @@ -115,25 +115,24 @@ * the implementation of read-modify-write operations on the * field is encapsulated in vm_page_clear_dirty_mask(). * - * The page structure contains two counters which prevent page reuse. - * Both counters are protected by the page lock (P). The hold - * counter counts transient references obtained via a pmap lookup, and - * is also used to prevent page reclamation in situations where it is - * undesirable to block other accesses to the page. The wire counter - * is used to implement mlock(2) and is non-zero for pages containing - * kernel memory. Pages that are wired or held will not be reclaimed - * or laundered by the page daemon, but are treated differently during - * a page queue scan: held pages remain at their position in the queue, - * while wired pages are removed from the queue and must later be - * re-enqueued appropriately by the unwiring thread. It is legal to - * call vm_page_free() on a held page; doing so causes it to be removed - * from its object and page queue, and the page is released to the - * allocator once the last hold reference is dropped. In contrast, - * wired pages may not be freed. + * The ref_count field tracks references to the page. References that + * prevent the page from being reclaimable are called wirings and are + * counted in the low bits of ref_count. The containing object's + * reference, if one exists, is counted using the VPRC_OBJREF bit in the + * ref_count field. Additionally, the VPRC_BLOCKED bit is used to + * atomically check for wirings and prevent new wirings via + * pmap_extract_and_hold(). When a page belongs to an object, it may be + * wired only when the object is locked, or the page is busy, or by + * pmap_extract_and_hold(). As a result, if the object is locked and the + * page is not busy (or is exclusively busied by the current thread), and + * the page is unmapped, its wire count will not increase. The ref_count + * field is updated using atomic operations in most cases, except when it + * is known that no other references to the page exist, such as in the page + * allocator. A page may be present in the page queues, or even actively + * scanned by the page daemon, without an explicitly counted referenced. + * The page daemon must therefore handle the possibility of a concurrent + * free of the page. * - * In some pmap implementations, the wire count of a page table page is - * used to track the number of populated entries. - * * The busy lock is an embedded reader-writer lock which protects the * page's contents and identity (i.e., its tuple) and * interlocks with the object lock (O). In particular, a page may be @@ -147,28 +146,34 @@ * sleep until the page's busy state changes, after which the caller * must re-lookup the page and re-evaluate its state. * - * The queue field is the index of the page queue containing the - * page, or PQ_NONE if the page is not enqueued. The queue lock of a - * page is the page queue lock corresponding to the page queue index, - * or the page lock (P) for the page if it is not enqueued. To modify - * the queue field, the queue lock for the old value of the field must - * be held. It is invalid for a page's queue field to transition - * between two distinct page queue indices. That is, when updating - * the queue field, either the new value or the old value must be - * PQ_NONE. + * The queue field is the index of the page queue containing the page, + * or PQ_NONE if the page is not enqueued. The queue lock of a page is + * the page queue lock corresponding to the page queue index, or the + * page lock (P) for the page if it is not enqueued. To modify the + * queue field, the queue lock for the old value of the field must be + * held. There is one exception to this rule: the page daemon may + * transition the queue field from PQ_INACTIVE to PQ_NONE immediately + * prior to freeing a page during an inactive queue scan. At that + * point the page has already been physically dequeued and no other + * references to that vm_page structure exist. * * To avoid contention on page queue locks, page queue operations - * (enqueue, dequeue, requeue) are batched using per-CPU queues. - * A deferred operation is requested by inserting an entry into a - * batch queue; the entry is simply a pointer to the page, and the - * request type is encoded in the page's aflags field using the values - * in PGA_QUEUE_STATE_MASK. The type-stability of struct vm_pages is + * (enqueue, dequeue, requeue) are batched using per-CPU queues. A + * deferred operation is requested by inserting an entry into a batch + * queue; the entry is simply a pointer to the page, and the request + * type is encoded in the page's aflags field using the values in + * PGA_QUEUE_STATE_MASK. The type-stability of struct vm_pages is * crucial to this scheme since the processing of entries in a given - * batch queue may be deferred indefinitely. In particular, a page - * may be freed before its pending batch queue entries have been - * processed. The page lock (P) must be held to schedule a batched - * queue operation, and the page queue lock must be held in order to - * process batch queue entries for the page queue. + * batch queue may be deferred indefinitely. In particular, a page may + * be freed before its pending batch queue entries have been processed. + * The page lock (P) must be held to schedule a batched queue + * operation, and the page queue lock must be held in order to process + * batch queue entries for the page queue. There is one exception to + * this rule: the thread freeing a page may schedule a dequeue without + * holding the page lock. In this scenario the only other thread which + * may hold a reference to the page is the page daemon, which is + * careful to avoid modifying the page's queue state once the dequeue + * has been requested by setting PGA_DEQUEUE. */ #if PAGE_SIZE == 4096 @@ -198,11 +203,14 @@ struct vm_page { } memguard; } plinks; TAILQ_ENTRY(vm_page) listq; /* pages in same object (O) */ - vm_object_t object; /* which object am I in (O,P) */ + vm_object_t object; /* which object am I in (O) */ vm_pindex_t pindex; /* offset into object (O,P) */ vm_paddr_t phys_addr; /* physical address of page (C) */ struct md_page md; /* machine dependent stuff */ - u_int wire_count; /* wired down maps refs (P) */ + union { + u_int wire_count; + u_int ref_count; /* page references */ + }; volatile u_int busy_lock; /* busy owners lock */ uint16_t flags; /* page PG_* flags (P) */ uint8_t order; /* index of the buddy queue (F) */ @@ -220,6 +228,26 @@ struct vm_page { }; /* + * Special bits used in the ref_count field. + * + * ref_count is normally used to count wirings that prevent the page from being + * reclaimed, but also supports several special types of references that do not + * prevent reclamation. Accesses to the ref_count field must be atomic unless + * the page is unallocated. + * + * VPRC_OBJREF is the reference held by the containing object. It can set or + * cleared only when the corresponding object's write lock is held. + * + * VPRC_BLOCKED is used to atomically block wirings via pmap lookups while + * attempting to tear down all mappings of a given page. The page lock and + * object write lock must both be held in order to set or clear this bit. + */ +#define VPRC_BLOCKED 0x40000000u /* mappings are being removed */ +#define VPRC_OBJREF 0x80000000u /* object reference, cleared with (O) */ +#define VPRC_WIRE_COUNT(c) ((c) & ~(VPRC_BLOCKED | VPRC_OBJREF)) +#define VPRC_WIRE_COUNT_MAX (~(VPRC_BLOCKED | VPRC_OBJREF)) + +/* * Page flags stored in oflags: * * Access to these page flags is synchronized by the lock on the object @@ -578,13 +606,17 @@ void vm_page_set_valid_range(vm_page_t m, int base int vm_page_sleep_if_busy(vm_page_t m, const char *msg); vm_offset_t vm_page_startup(vm_offset_t vaddr); void vm_page_sunbusy(vm_page_t m); +void vm_page_swapqueue(vm_page_t m, uint8_t oldq, uint8_t newq); +bool vm_page_try_remove_all(vm_page_t m); +bool vm_page_try_remove_write(vm_page_t m); int vm_page_trysbusy(vm_page_t m); void vm_page_unhold_pages(vm_page_t *ma, int count); void vm_page_unswappable(vm_page_t m); -bool vm_page_unwire(vm_page_t m, uint8_t queue); +void vm_page_unwire(vm_page_t m, uint8_t queue); bool vm_page_unwire_noq(vm_page_t m); void vm_page_updatefake(vm_page_t m, vm_paddr_t paddr, vm_memattr_t memattr); -void vm_page_wire (vm_page_t); +void vm_page_wire(vm_page_t); +bool vm_page_wire_mapped(vm_page_t m); void vm_page_xunbusy_hard(vm_page_t m); void vm_page_xunbusy_maybelocked(vm_page_t m); void vm_page_set_validclean (vm_page_t, int, int); @@ -667,9 +699,31 @@ void vm_page_assert_pga_writeable(vm_page_t m, uin * destinations. In order that we can easily use a 32-bit operation, we * require that the aflags field be 32-bit aligned. */ -CTASSERT(offsetof(struct vm_page, aflags) % sizeof(uint32_t) == 0); +_Static_assert(offsetof(struct vm_page, aflags) % sizeof(uint32_t) == 0, + "aflags field is not 32-bit aligned"); /* + * We want to be able to update the aflags and queue fields atomically in + * the same operation. + */ +_Static_assert(offsetof(struct vm_page, aflags) / sizeof(uint32_t) == + offsetof(struct vm_page, queue) / sizeof(uint32_t), + "aflags and queue fields do not belong to the same 32-bit word"); +_Static_assert(offsetof(struct vm_page, queue) % sizeof(uint32_t) == 2, + "queue field is at an unexpected offset"); +_Static_assert(sizeof(((struct vm_page *)NULL)->queue) == 1, + "queue field has an unexpected size"); + +#if BYTE_ORDER == LITTLE_ENDIAN +#define VM_PAGE_AFLAG_SHIFT 0 +#define VM_PAGE_QUEUE_SHIFT 16 +#else +#define VM_PAGE_AFLAG_SHIFT 24 +#define VM_PAGE_QUEUE_SHIFT 8 +#endif +#define VM_PAGE_QUEUE_MASK (0xff << VM_PAGE_QUEUE_SHIFT) + +/* * Clear the given bits in the specified page. */ static inline void @@ -689,16 +743,21 @@ vm_page_aflag_clear(vm_page_t m, uint8_t bits) * within this word are handled properly by the atomic update. */ addr = (void *)&m->aflags; - KASSERT(((uintptr_t)addr & (sizeof(uint32_t) - 1)) == 0, - ("vm_page_aflag_clear: aflags is misaligned")); - val = bits; -#if BYTE_ORDER == BIG_ENDIAN - val <<= 24; -#endif + val = bits << VM_PAGE_AFLAG_SHIFT; atomic_clear_32(addr, val); } /* + * Return true if all of the specified flags are set in the page. + */ +static inline bool +vm_page_aflag_isset(vm_page_t m, uint8_t flags) +{ + + return ((m->aflags & flags) == flags); +} + +/* * Set the given bits in the specified page. */ static inline void @@ -714,16 +773,46 @@ vm_page_aflag_set(vm_page_t m, uint8_t bits) * within this word are handled properly by the atomic update. */ addr = (void *)&m->aflags; - KASSERT(((uintptr_t)addr & (sizeof(uint32_t) - 1)) == 0, - ("vm_page_aflag_set: aflags is misaligned")); - val = bits; -#if BYTE_ORDER == BIG_ENDIAN - val <<= 24; -#endif + val = bits << VM_PAGE_AFLAG_SHIFT; atomic_set_32(addr, val); -} +} /* + * Atomically update the queue state of the page. The operation fails if + * any of the queue flags in "fflags" are set or if the "queue" field of + * the page does not match the expected value; if the operation is + * successful, the flags in "nflags" are set and all other queue state + * flags are cleared. + */ +static inline bool +vm_page_pqstate_cmpset(vm_page_t m, uint32_t oldq, uint32_t newq, + uint32_t fflags, uint32_t nflags) +{ + uint32_t *addr, nval, oval, qsmask; + + vm_page_assert_locked(m); + + fflags <<= VM_PAGE_AFLAG_SHIFT; + nflags <<= VM_PAGE_AFLAG_SHIFT; + newq <<= VM_PAGE_QUEUE_SHIFT; + oldq <<= VM_PAGE_QUEUE_SHIFT; + qsmask = ((PGA_DEQUEUE | PGA_REQUEUE | PGA_REQUEUE_HEAD) << + VM_PAGE_AFLAG_SHIFT) | VM_PAGE_QUEUE_MASK; + + addr = (void *)&m->aflags; + oval = atomic_load_32(addr); + do { + if ((oval & fflags) != 0) + return (false); + if ((oval & VM_PAGE_QUEUE_MASK) != oldq) + return (false); + nval = (oval & ~qsmask) | nflags | newq; + } while (!atomic_fcmpset_32(addr, &oval, nval)); + + return (true); +} + +/* * vm_page_dirty: * * Set all bits in the page's dirty field. @@ -815,15 +904,35 @@ vm_page_in_laundry(vm_page_t m) } /* + * vm_page_drop: + * + * Release a reference to a page and return the old reference count. + */ +static inline u_int +vm_page_drop(vm_page_t m, u_int val) +{ + + /* + * Synchronize with vm_page_free_prep(): ensure that all updates to the + * page structure are visible before it is freed. + */ + atomic_thread_fence_rel(); + return (atomic_fetchadd_int(&m->ref_count, -val)); +} + +/* * vm_page_wired: * - * Return true if a reference prevents the page from being reclaimable. + * Perform a racy check to determine whether a reference prevents the page + * from being reclaimable. If the page's object is locked, and the page is + * unmapped and unbusied or exclusively busied by the current thread, no + * new wirings may be created. */ static inline bool vm_page_wired(vm_page_t m) { - return (m->wire_count > 0); + return (VPRC_WIRE_COUNT(m->ref_count) > 0); } #endif /* _KERNEL */ Index: src/sys/vm/vm_pageout.c =================================================================== --- src/sys/vm/vm_pageout.c (revision 351563) +++ src/sys/vm/vm_pageout.c (working copy) @@ -305,7 +305,9 @@ vm_pageout_collect_batch(struct scan_state *ss, co vm_pagequeue_unlock(pq); } -/* Return the next page to be scanned, or NULL if the scan is complete. */ +/* + * Return the next page to be scanned, or NULL if the scan is complete. + */ static __always_inline vm_page_t vm_pageout_next(struct scan_state *ss, const bool dequeue) { @@ -328,17 +330,12 @@ vm_pageout_cluster(vm_page_t m) vm_pindex_t pindex; int ib, is, page_base, pageout_count; - vm_page_assert_locked(m); object = m->object; VM_OBJECT_ASSERT_WLOCKED(object); pindex = m->pindex; vm_page_assert_unbusied(m); - KASSERT(!vm_page_wired(m), ("page %p is wired", m)); - pmap_remove_write(m); - vm_page_unlock(m); - mc[vm_pageout_page_count] = pb = ps = m; pageout_count = 1; page_base = vm_pageout_page_count; @@ -363,7 +360,8 @@ more: ib = 0; break; } - if ((p = vm_page_prev(pb)) == NULL || vm_page_busied(p)) { + if ((p = vm_page_prev(pb)) == NULL || vm_page_busied(p) || + vm_page_wired(p)) { ib = 0; break; } @@ -373,12 +371,11 @@ more: break; } vm_page_lock(p); - if (vm_page_wired(p) || !vm_page_in_laundry(p)) { + if (!vm_page_in_laundry(p) || !vm_page_try_remove_write(p)) { vm_page_unlock(p); ib = 0; break; } - pmap_remove_write(p); vm_page_unlock(p); mc[--page_base] = pb = p; ++pageout_count; @@ -393,17 +390,17 @@ more: } while (pageout_count < vm_pageout_page_count && pindex + is < object->size) { - if ((p = vm_page_next(ps)) == NULL || vm_page_busied(p)) + if ((p = vm_page_next(ps)) == NULL || vm_page_busied(p) || + vm_page_wired(p)) break; vm_page_test_dirty(p); if (p->dirty == 0) break; vm_page_lock(p); - if (vm_page_wired(p) || !vm_page_in_laundry(p)) { + if (!vm_page_in_laundry(p) || !vm_page_try_remove_write(p)) { vm_page_unlock(p); break; } - pmap_remove_write(p); vm_page_unlock(p); mc[page_base + pageout_count] = ps = p; ++pageout_count; @@ -648,10 +645,10 @@ vm_pageout_clean(vm_page_t m, int *numpagedout) } /* - * The page may have been busied or referenced while the object - * and page locks were released. + * The page may have been busied while the object and page + * locks were released. */ - if (vm_page_busied(m) || vm_page_wired(m)) { + if (vm_page_busied(m)) { vm_page_unlock(m); error = EBUSY; goto unlock_all; @@ -659,6 +656,16 @@ vm_pageout_clean(vm_page_t m, int *numpagedout) } /* + * Remove all writeable mappings, failing if the page is wired. + */ + if (!vm_page_try_remove_write(m)) { + vm_page_unlock(m); + error = EBUSY; + goto unlock_all; + } + vm_page_unlock(m); + + /* * If a page is dirty, then it is either being washed * (but not yet cleaned) or it is still in the * laundry. If it is still in the laundry, then we @@ -732,7 +739,9 @@ scan: recheck: /* * The page may have been disassociated from the queue - * while locks were dropped. + * or even freed while locks were dropped. We thus must be + * careful whenever modifying page state. Once the object lock + * has been acquired, we have a stable reference to the page. */ if (vm_page_queue(m) != queue) continue; @@ -742,7 +751,7 @@ recheck: * chance. */ if ((m->aflags & PGA_REQUEUE) != 0) { - vm_page_requeue(m); + vm_page_pqbatch_submit(m, queue); continue; } @@ -749,7 +758,9 @@ recheck: /* * Wired pages may not be freed. Complete their removal * from the queue now to avoid needless revisits during - * future scans. + * future scans. This check is racy and must be reverified once + * we hold the object lock and have verified that the page + * is not busy. */ if (vm_page_wired(m)) { vm_page_dequeue_deferred(m); @@ -759,8 +770,13 @@ recheck: if (object != m->object) { if (object != NULL) VM_OBJECT_WUNLOCK(object); - object = m->object; - if (!VM_OBJECT_TRYWLOCK(object)) { + + /* + * A page's object pointer may be set to NULL before + * the object lock is acquired. + */ + object = (vm_object_t)atomic_load_ptr(&m->object); + if (object != NULL && !VM_OBJECT_TRYWLOCK(object)) { mtx_unlock(mtx); /* Depends on type-stability. */ VM_OBJECT_WLOCK(object); @@ -768,11 +784,31 @@ recheck: goto recheck; } } + if (__predict_false(m->object == NULL)) + /* + * The page has been removed from its object. + */ + continue; + KASSERT(m->object == object, ("page %p does not belong to %p", + m, object)); if (vm_page_busied(m)) continue; /* + * Re-check for wirings now that we hold the object lock and + * have verified that the page is unbusied. If the page is + * mapped, it may still be wired by pmap lookups. The call to + * vm_page_try_remove_all() below atomically checks for such + * wirings and removes mappings. If the page is unmapped, the + * wire count is guaranteed not to increase. + */ + if (__predict_false(vm_page_wired(m))) { + vm_page_dequeue_deferred(m); + continue; + } + + /* * Invalid pages can be easily freed. They cannot be * mapped; vm_page_free() asserts this. */ @@ -839,8 +875,10 @@ recheck: */ if (object->ref_count != 0) { vm_page_test_dirty(m); - if (m->dirty == 0) - pmap_remove_all(m); + if (m->dirty == 0 && !vm_page_try_remove_all(m)) { + vm_page_dequeue_deferred(m); + continue; + } } /* @@ -1132,6 +1170,7 @@ vm_pageout_scan_active(struct vm_domain *vmd, int { struct scan_state ss; struct mtx *mtx; + vm_object_t object; vm_page_t m, marker; struct vm_pagequeue *pq; long min_scan; @@ -1192,7 +1231,9 @@ act_scan: /* * The page may have been disassociated from the queue - * while locks were dropped. + * or even freed while locks were dropped. We thus must be + * careful whenever modifying page state. Once the object lock + * has been acquired, we have a stable reference to the page. */ if (vm_page_queue(m) != PQ_ACTIVE) continue; @@ -1206,6 +1247,17 @@ act_scan: } /* + * A page's object pointer may be set to NULL before + * the object lock is acquired. + */ + object = (vm_object_t)atomic_load_ptr(&m->object); + if (__predict_false(object == NULL)) + /* + * The page has been removed from its object. + */ + continue; + + /* * Check to see "how much" the page has been used. * * Test PGA_REFERENCED after calling pmap_ts_referenced() so @@ -1224,7 +1276,7 @@ act_scan: * This race delays the detection of a new reference. At * worst, we will deactivate and reactivate the page. */ - if (m->object->ref_count != 0) + if (object->ref_count != 0) act_delta = pmap_ts_referenced(m); else act_delta = 0; @@ -1256,9 +1308,9 @@ act_scan: * place them directly in the laundry queue to reduce * queuing overhead. */ - if (page_shortage <= 0) - vm_page_deactivate(m); - else { + if (page_shortage <= 0) { + vm_page_swapqueue(m, PQ_ACTIVE, PQ_INACTIVE); + } else { /* * Calling vm_page_test_dirty() here would * require acquisition of the object's write @@ -1270,11 +1322,13 @@ act_scan: * dirty field by the pmap. */ if (m->dirty == 0) { - vm_page_deactivate(m); + vm_page_swapqueue(m, PQ_ACTIVE, + PQ_INACTIVE); page_shortage -= act_scan_laundry_weight; } else { - vm_page_launder(m); + vm_page_swapqueue(m, PQ_ACTIVE, + PQ_LAUNDRY); page_shortage--; } } @@ -1398,7 +1452,9 @@ vm_pageout_scan_inactive(struct vm_domain *vmd, in recheck: /* * The page may have been disassociated from the queue - * while locks were dropped. + * or even freed while locks were dropped. We thus must be + * careful whenever modifying page state. Once the object lock + * has been acquired, we have a stable reference to the page. */ if (vm_page_queue(m) != PQ_INACTIVE) { addl_page_shortage++; @@ -1417,7 +1473,9 @@ recheck: /* * Wired pages may not be freed. Complete their removal * from the queue now to avoid needless revisits during - * future scans. + * future scans. This check is racy and must be reverified once + * we hold the object lock and have verified that the page + * is not busy. */ if (vm_page_wired(m)) { vm_page_dequeue_deferred(m); @@ -1427,8 +1485,13 @@ recheck: if (object != m->object) { if (object != NULL) VM_OBJECT_WUNLOCK(object); - object = m->object; - if (!VM_OBJECT_TRYWLOCK(object)) { + + /* + * A page's object pointer may be set to NULL before + * the object lock is acquired. + */ + object = (vm_object_t)atomic_load_ptr(&m->object); + if (object != NULL && !VM_OBJECT_TRYWLOCK(object)) { mtx_unlock(mtx); /* Depends on type-stability. */ VM_OBJECT_WLOCK(object); @@ -1436,6 +1499,13 @@ recheck: goto recheck; } } + if (__predict_false(m->object == NULL)) + /* + * The page has been removed from its object. + */ + continue; + KASSERT(m->object == object, ("page %p does not belong to %p", + m, object)); if (vm_page_busied(m)) { /* @@ -1451,6 +1521,19 @@ recheck: } /* + * Re-check for wirings now that we hold the object lock and + * have verified that the page is unbusied. If the page is + * mapped, it may still be wired by pmap lookups. The call to + * vm_page_try_remove_all() below atomically checks for such + * wirings and removes mappings. If the page is unmapped, the + * wire count is guaranteed not to increase. + */ + if (__predict_false(vm_page_wired(m))) { + vm_page_dequeue_deferred(m); + continue; + } + + /* * Invalid pages can be easily freed. They cannot be * mapped, vm_page_free() asserts this. */ @@ -1506,8 +1589,10 @@ recheck: */ if (object->ref_count != 0) { vm_page_test_dirty(m); - if (m->dirty == 0) - pmap_remove_all(m); + if (m->dirty == 0 && !vm_page_try_remove_all(m)) { + vm_page_dequeue_deferred(m); + continue; + } } /* Index: src/sys/vm/vm_swapout.c =================================================================== --- src/sys/vm/vm_swapout.c (revision 351563) +++ src/sys/vm/vm_swapout.c (working copy) @@ -207,16 +207,20 @@ vm_swapout_object_deactivate_pages(pmap_t pmap, vm goto unlock_return; if (should_yield()) goto unlock_return; - if (vm_page_busied(p)) + + /* + * The page may acquire a wiring after this check. + * The page daemon handles wired pages, so there is + * no harm done if a wiring appears while we are + * attempting to deactivate the page. + */ + if (vm_page_busied(p) || vm_page_wired(p)) continue; VM_CNT_INC(v_pdpages); - vm_page_lock(p); - if (vm_page_wired(p) || - !pmap_page_exists_quick(pmap, p)) { - vm_page_unlock(p); + if (!pmap_page_exists_quick(pmap, p)) continue; - } act_delta = pmap_ts_referenced(p); + vm_page_lock(p); if ((p->aflags & PGA_REFERENCED) != 0) { if (act_delta == 0) act_delta = 1; @@ -234,7 +238,7 @@ vm_swapout_object_deactivate_pages(pmap_t pmap, vm p->act_count -= min(p->act_count, ACT_DECLINE); if (!remove_mode && p->act_count == 0) { - pmap_remove_all(p); + (void)vm_page_try_remove_all(p); vm_page_deactivate(p); } } else { @@ -244,7 +248,7 @@ vm_swapout_object_deactivate_pages(pmap_t pmap, vm p->act_count += ACT_ADVANCE; } } else if (vm_page_inactive(p)) - pmap_remove_all(p); + (void)vm_page_try_remove_all(p); vm_page_unlock(p); } if ((backing_object = object->backing_object) == NULL) @@ -556,9 +560,7 @@ vm_thread_swapout(struct thread *td) if (m == NULL) panic("vm_thread_swapout: kstack already missing?"); vm_page_dirty(m); - vm_page_lock(m); vm_page_unwire(m, PQ_LAUNDRY); - vm_page_unlock(m); } VM_OBJECT_WUNLOCK(ksobj); } -- Test scenario: misc/cpuset.sh 20190828 10:55:33 all: cpuset.sh 10:55:34 cpuset -n round-robin:0 10:57:03 cpuset -n round-robin:1 10:58:33 cpuset -n round-robin:0-1 11:00:03 cpuset -n round-robin:all 11:01:33 cpuset -n 0-1 11:03:04 cpuset -n round-robin 11:04:34 cpuset -n all 11:06:01 cpuset -n first-touch 11:07:28 cpuset -n prefer:1