KDB: debugger backends: ddb KDB: current backend: ddb ---<>--- Copyright (c) 1992-2019 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 13.0-CURRENT #0 r350211M: Mon Jul 22 17:38:45 CEST 2019 pho@x4.osted.lan:/usr/src/sys/i386/compile/PHO i386 FreeBSD clang version 8.0.1 (branches/release_80 363030) (based on LLVM 8.0.1) WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. Entering uma_startup with 9 boot pages configured startup_alloc from "UMA Kegs", 8 boot pages left startup_alloc from "UMA Zones", 7 boot pages left startup_alloc from "UMA Hash", 6 boot pages left startup_alloc from "vmem btag", 5 boot pages left startup_alloc from "UMA Zones", 4 boot pages left startup_alloc from "MAP", 3 boot pages left startup_alloc from "KMAP ENTRY", 2 boot pages left startup_alloc from "vmem", 1 boot pages left Entering uma_startup2 with 0 boot pages left VT(vga): resolution 640x480 CPU: AMD Phenom(tm) 9150e Quad-Core Processor (1800.02-MHz 686-class CPU) Origin="AuthenticAMD" Id=0x100f23 Family=0x10 Model=0x2 Stepping=3 Features=0x178bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x7ff SVM: NP,NAsids=64 TSC: P-state invariant real memory = 8589934592 (8192 MB) avail memory = 3520135168 (3357 MB) Event timer "LAPIC" quality 100 ACPI APIC Table: <060508 APIC1048> FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 4 core(s) random: unblocking device. ioapic0 irqs 0-23 on motherboard Launching APs: 1 3 2 Timecounter "TSC" frequency 1800016781 Hz quality 800 random: entropy device external interface [ath_hal] loaded kbd1 at kbdmux0 module_register_init: MOD_LOAD (vesa, 0x14a1f80, 0) error 19 nexus0 vtvga0: on motherboard cryptosoft0: on motherboard acpi0: <060508 RSDT1048> on motherboard acpi0: Power Button (fixed) cpu0: on acpi0 attimer0: port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 atrtc0: port 0x70-0x71 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: at device 1.0 on pci0 pci1: on pcib1 vgapci0: port 0xc000-0xc0ff mem 0xf0000000-0xf7ffffff,0xfe9f0000-0xfe9fffff,0xfe800000-0xfe8fffff irq 18 at device 5.0 on pci1 vgapci0: Boot video device hdac0: mem 0xfe9e8000-0xfe9ebfff irq 19 at device 5.2 on pci1 pcib2: at device 7.0 on pci0 pci2: on pcib2 re0: port 0xd800-0xd8ff mem 0xfeaff000-0xfeafffff irq 19 at device 0.0 on pci2 re0: Using 1 MSI message re0: Chip rev. 0x38000000 re0: MAC rev. 0x00000000 miibus0: on re0 rgephy0: PHY 1 on miibus0 rgephy0: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow re0: Using defaults for TSO: 65518/35/2048 re0: Ethernet address: 00:21:85:15:33:7d ahci0: port 0xb000-0xb007,0xa000-0xa003,0x9000-0x9007,0x8000-0x8003,0x7000-0x700f mem 0xfe7ff800-0xfe7ffbff irq 22 at device 18.0 on pci0 ahci0: AHCI v1.10 with 4 3Gbps ports, Port Multiplier supported ahci0: quirks=0x7000 ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ohci0: mem 0xfe7fe000-0xfe7fefff irq 16 at device 19.0 on pci0 usbus0 on ohci0 usbus0: 12Mbps Full Speed USB v1.0 ohci1: mem 0xfe7fd000-0xfe7fdfff irq 17 at device 19.1 on pci0 usbus1 on ohci1 usbus1: 12Mbps Full Speed USB v1.0 ohci2: mem 0xfe7fc000-0xfe7fcfff irq 18 at device 19.2 on pci0 usbus2 on ohci2 usbus2: 12Mbps Full Speed USB v1.0 ohci3: mem 0xfe7fb000-0xfe7fbfff irq 17 at device 19.3 on pci0 usbus3 on ohci3 usbus3: 12Mbps Full Speed USB v1.0 ohci4: mem 0xfe7fa000-0xfe7fafff irq 18 at device 19.4 on pci0 usbus4 on ohci4 usbus4: 12Mbps Full Speed USB v1.0 ehci0: mem 0xfe7ff000-0xfe7ff0ff irq 19 at device 19.5 on pci0 ehci0: AMD SB600/700 quirk applied usbus5: EHCI version 1.0 usbus5 on ehci0 usbus5: 480Mbps High Speed USB v2.0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xff00-0xff0f at device 20.1 on pci0 ata0: at channel 0 on atapci0 hdac1: mem 0xfe7f4000-0xfe7f7fff irq 16 at device 20.2 on pci0 isab0: at device 20.3 on pci0 isa0: on isab0 pcib3: at device 20.4 on pci0 pci3: on pcib3 pci3: at device 6.0 (no driver attached) acpi_button0: on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: console (115200,n,8,1) fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] ppc0: parallel port not found. hwpstate0: on cpu0 NULL mp in getnewvnode(9), tag crossmp Timecounters tick every 1.000 msec hdacc0: at cad 0 on hdac0 hdaa0: at nid 1 on hdacc0 pcm0: at nid 3 on hdaa0 hdacc1: at cad 3 on hdac1 hdaa1: at nid 1 on hdacc1 pcm1: at nid 20,22,21,23 and 24,26 on hdaa1 pcm2: at nid 27 and 25 on hdaa1 pcm3: at nid 30 on hdaa1 ugen2.1: at usbus2 ugen5.1: at usbus5 uhub0 on usbus2 ugen0.1: at usbus0 ugen1.1: at usbus1 uhub1 on usbus5 uhub2 on usbus0 uhub1: on usbus5 uhub0: on usbus2 uhub2: on usbus0 uhub3 on usbus1 uhub3: on usbus1 Expensive timeout(9) function: 0xa3aa20(0x17ae1000) 0.009480534 s ugen4.1: at usbus4 ugen3.1: at usbus3 uhub4(aprobe0:ahcich0:0:15:0): NOP FLUSHQUEUE. ACB: 00 00 00 00 00 00 00 00 00 00 00 00 on usbus4 uhub4: on usbus4 (aprobe0:ahcich0:0:15:0): CAM status: Command timeout uhub5 on usbus3 (aprobe0:ahcich0:0:15:0): Error 5, Retries exhausted uhub5: on usbus3 ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: ATA8-ACS SATA 2.x device ada0: Serial Number 9VMJ99G5 ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 476940MB (976773168 512 byte sectors) cd0 at ata0 bus 0 scbus4 target 1 lun 0 cd0: <_NEC DVD_RW ND-4550A 1.06> Removable CD-ROM SCSI device cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present Trying to mount root from ufs:/dev/ada0s1a [rw]... WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. uhub0: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered uhub4: 2 ports with 2 removable, self powered uhub5: 2 ports with 2 removable, self powered Setting hostuuid: 00000000-0000-0000-0000-00218515337d. Setting hostid: 0x6b64ac17. warning: total configured swap (4194304 pages) exceeds maximum recommended amount (2009920 pages). warning: increase kern.maxswzone or reduce amount of swap. Starting file system checks: /dev/ada0s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ada0s1a: clean, 604845 free (2293 frags, 75319 blocks, 0.2% fragmentation) /dev/ada0s1f: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ada0s1f: clean, 749767 free (11263 frags, 92313 blocks, 1.1% fragmentation) /dev/label/tmp: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/label/tmp: clean, 28199416 free (36288 frags, 3520391 blocks, 0.0% fragmentation) /dev/ada0s1d: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ada0s1d: clean, 1289223 free (244471 frags, 130594 blocks, 2.4% fragmentation) /dev/ada0s1e: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ada0s1e: clean, 6801740 free (166716 frags, 829378 blocks, 0.7% fragmentation) Mounting local filesystems:. ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/arduino/lib /usr/local/lib/compat /usr/local/lib/compat/pkg /usr/local/lib/e2fsprogs /usr/local/lib/gcc7 /usr/local/lib/gcc8 /usr/local/lib/mysql /usr/local/lib/nss /usr/local/lib/perl5/5.28/mach/CORE /usr/local/lib/pth /usr/local/lib/qt4 /usr/local/lib/qt5 /usr/local/llvm37/lib /usr/local/llvm39/lib a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout Setting hostname: x4.osted.lan. Setting up harvesting: [UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,[NET_ETHER],NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED Feeding entropy: uhub1: 10 ports with 10 removable, self powered . lo0: link state changed to UP re0: link state changed to DOWN Starting Network: lo0 re0. lo0: flags=8049 metric 0 mtu 16384 options=680003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=21 re0: flags=8843 metric 0 mtu 1500 options=8209b ether 00:21:85:15:33:7d inet 192.168.1.101 netmask 0xffffff00 broadcast 192.168.1.255 media: Ethernet autoselect (none) status: no carrier nd6 options=29 Starting devd. re0: link state changed to UP add host 127.0.0.1: gateway lo0 fib 0: route already in table add net default: gateway 192.168.1.1 add host ::1: gateway lo0 fib 0: route already in table add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Creating and/or trimming log files. Starting syslogd. Starting rpcbind. NFS access cache time=60 Starting watchdogd. No core dumps found. lock order reversal: 1st 0x9668d54 bufwait (bufwait) @ kern/vfs_bio.c:3904 2nd 0x210a6e00 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:289 stack backtrace: #0 0x102f051 at witness_debugger+0x61 #1 0x102ee29 at witness_checkorder+0x8a9 #2 0xfd709d at _sx_xlock+0x4d #3 0x12a3854 at ufsdirhash_move+0x44 #4 0x12a59c1 at ufs_direnter+0x2e1 #5 0x12ad58c at ufs_mkdir+0x78c #6 0x156192d at VOP_MKDIR_APV+0x6d #7 0x109a023 at kern_mkdirat+0x173 #8 0x1099ea9 at sys_mkdir+0x19 #9 0x15312d4 at syscall+0x304 #10 0xffc033b7 at _end+0xfdc4f0c7 Clearing /tmp (X related). Starting nfsuserd. Starting mountd. Starting nfsd. Starting statd. Starting lockd. Recovering vi editor sessions:. Updating motd:. Mounting late filesystems:. Starting ntpd. Starting powerd. Starting smartd. Jul 22 18:17:19 x4 smartd[630]: Device: /dev/ada0, WARNING: A firmware update for this drive may be available, Jul 22 18:17:19 x4 smartd[630]: see the following Seagate web pages: Jul 22 18:17:19 x4 smartd[630]: http://knowledge.seagate.com/articles/en_US/FAQ/207931en Jul 22 18:17:19 x4 smartd[630]: http://knowledge.seagate.com/articles/en_US/FAQ/213891en Configuring vt: keymap blanktime. Performing sanity check on sshd configuration. Starting sshd. Starting sendmail_submit. Starting sendmail_msp_queue. Starting cron. Local package initialization: backup. Starting nfscbd. Starting default mousedmoused: unable to open /dev/psm0: No such file or directory . Starting inetd. Mon Jul 22 18:17 FreeBSD/i386 (x4.osted.lan) (ttyu0) login: Jul 22 18:18:51 x4 su[932]: pho to root on /dev/pts/0 20190722 18:19:02 all (1/2): mprotect.sh 20190722 18:19:14 all (2/2): shm_super.sh 20190722 18:19:17 all: done 20190722 18:20:06 all (1/657): ptrace9.sh freework leak: 1/9. newblk leak: 1/9. 20190722 18:20:18 all (2/657): mmap19.sh freework leak: 9/18. newblk leak: 2/11. 20190722 18:27:02 all (3/657): unix_socket_detach.sh 20190722 18:32:08 all (4/657): marcus2.sh lock order reversal: 1st 0x2cc025e4 ufs (ufs) @ kern/vfs_lookup.c:713 2nd 0x9729748 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:283 3rd 0x228f35e4 ufs (ufs) @ kern/vfs_subr.c:2687 stack backtrace: #0 0x102f051 at witness_debugger+0x61 #1 0x102ee29 at witness_checkorder+0x8a9 #2 0xfa65e8 at lockmgr_xlock_hard+0x48 #3 0xfa6d56 at __lockmgr_args+0x376 #4 0x129dccf at ffs_lock+0x6f #5 0x15622d8 at VOP_LOCK1_APV+0x58 #6 0x109d3e8 at _vn_lock+0x58 #7 0x108c19b at vget+0x7b #8 0x107f623 at vfs_hash_get+0xa3 #9 0x12996a7 at ffs_vgetf+0x27 #10 0x128f8c1 at softdep_sync_buf+0x8d1 #11 0x129e9f1 at ffs_syncvnode+0x231 #12 0x1275a0c at ffs_truncate+0x88c #13 0x12a5c5e at ufs_direnter+0x57e #14 0x12ae20d at ufs_makeinode+0x51d #15 0x12ad82b at ufs_symlink+0x2b #16 0x1561c4d at VOP_SYMLINK_APV+0x6d #17 0x109663b at kern_symlinkat+0x1cb lock order reversal: 1st 0x1f95300 cpslk (cpslk) @ i386/i386/copyout.c:121 2nd 0x1896621c bufwait (bufwait) @ vm/vm_pager.c:379 stack backtrace: #0 0x102f051 at witness_debugger+0x61 #1 0x102ee29 at witness_checkorder+0x8a9 #2 0xfa65e8 at lockmgr_xlock_hard+0x48 #3 0xfa6d56 at __lockmgr_args+0x376 #4 0x12dc479 at pbuf_ctor+0x79 #5 0x12b80ab at zone_alloc_item_locked+0x3bb #6 0x12b6eec at uma_zalloc_arg+0x92c #7 0x12b02c2 at swap_pager_getpages+0x2a2 #8 0x12dc6a4 at vm_pager_get_pages+0x44 #9 0x12bdbc8 at vm_fault_hold+0x1938 #10 0x12bc24a at vm_fault+0x4a #11 0x1530ca3 at trap_pfault+0xf3 #12 0x15302c0 at trap+0x3c0 #13 0xffc0316d at _end+0xfdc4ee7d #14 0x150bcf3 at copyout_slow0+0x13 #15 0x150b9b3 at cp_slow0+0x153 #16 0x150bcb7 at copyout+0xa7 #17 0x10288da at uiomove_faultflag+0x11a Jul 22 18:50:43 x4 kernel: pid 43112 (swap), jid 0, uid 1004, was killed: out of swap space lock order reversal: 1st 0x21c7830c ufs (ufs) @ kern/vfs_mount.c:1354 2nd 0x2a9768bc devfs (devfs) @ ufs/ffs/ffs_softdep.c:1926 stack backtrace: #0 0x102f051 at witness_debugger+0x61 #1 0x102ee29 at witness_checkorder+0x8a9 #2 0xfa61c5 at lockmgr_lock_fast_path+0x125 #3 0x107c2f4 at vop_stdlock+0x24 #4 0x15622d8 at VOP_LOCK1_APV+0x58 #5 0x109d3e8 at _vn_lock+0x58 #6 0x127e08b at softdep_flushworklist+0x5b #7 0x129c4b1 at ffs_sync+0x411 #8 0x1085505 at dounmount+0x425 #9 0x1084feb at sys_unmount+0x26b #10 0x15312d4 at syscall+0x304 #11 0xffc033b7 at _end+0xfdc4f0c7 20190722 18:55:36 all (5/657): vmstat2.sh 20190722 18:57:47 all (6/657): pathconf.sh 20190722 18:57:53 all (7/657): pread.sh lock order reversal: (sleepable after non-sleepable) 1st 0x1839fb88 pfs_node (pfs_node) @ fs/pseudofs/pseudofs_internal.h:105 2nd 0x9984850 pidhash (pidhash) @ kern/kern_proc.c:429 stack backtrace: #0 0x102f051 at witness_debugger+0x61 #1 0x102ee29 at witness_checkorder+0x8a9 Jul 22 18:58:22 #2 0xfd80d9 atx4 mountd[554]: _sx_slock_int+0x49 #3 0xfb556ccan't export /mn at pfind+0x5c #4 0xf1e60a at pt fs_readdir+0x13a #5 0x1561dc9 at VOP_READDIR_APV+0x69 #6 0x109a7a9 at kern_getdirentries+0x1c9 #7 0x109a9e5 at sys_getdirentries+0x25 #8 0x15312d4 at syscall+0x304 #9 0xffc033b7 at fdhashtbl+0xc97eb39f Jul 22 18:58:34 x4 syslogd: last message repeated 2 times mount leak: 3/100 20190722 18:59:06 all (8/657): mprotect.sh 20190722 18:59:13 all (9/657): kevent11.sh lock order reversal: 1st 0x185fd718 filedesc structure (filedesc structure) @ kern/kern_descrip.c:1248 2nd 0x2eb59478 ufs (ufs) @ kern/vfs_subr.c:5140 stack backtrace: #0 0x102f051 at witness_debugger+0x61 #1 0x102ee29 at witness_checkorder+0x8a9 #2 0xfa65e8 at lockmgr_xlock_hard+0x48 #3 0xfa6d56 at __lockmgr_args+0x376 #4 0x129dccf at ffs_lock+0x6f #5 0x15622d8 at VOP_LOCK1_APV+0x58 #6 0x109d3e8 at _vn_lock+0x58 #7 0x108e44a at vfs_knllock+0x1a #8 0xf8a0bc at knlist_remove_kq+0x7c #9 0xf8a033 at knlist_remove+0x13 #10 0x10931e0 at filt_vfsdetach+0x20 #11 0xf8a9df at knote_fdclose+0x13f #12 0xf7eec1 at closefp+0xe1 #13 0xf7f68d at kern_close+0x12d #14 0xf7f550 at sys_close+0x10 #15 0x15312d4 at syscall+0x304 #16 0xffc033b7 at _end+0xfdc4f0c7 mount leak: 1/101 20190722 19:02:27 all (10/657): nullfs22.sh mount leak: 1/102 20190722 19:04:34 all (11/657): libMicro.sh freework leak: 21/39. newblk leak: 1315/1326. 20190722 19:05:40 all (12/657): suj7.sh Failed to find journal. Use tunefs to create one Failed to start journal: 2 mount leak: 1/103 20190722 19:05:46 all (13/657): md.sh Jul 22 19:07:53 x4 kernel: pid 13392 (rw), uid 0 inumber 11 on /mnt: filesystem full Jul 22 19:07:55 x4 kernel: pid 13394 (rw), uid 0 inumber 140 on /mnt: filesystem full Jul 22 19:07:58 x4 kernel: pid 13385 (rw), uid 0 inumber 139 on /mnt: filesystem full Jul 22 19:07:59 x4 kernel: pid 13395 (rw), uid 0 inumber 132 on /mnt: filesystem full Jul 22 19:09:51 x4 kernel: pid 23077 (rw), uid 0 inumber 131 on /mnt: filesystem full Jul 22 19:09:52 x4 kernel: pid 23081 (rw), uid 0 inumber 6 on /mnt: filesystem full Jul 22 19:09:55 x4 kernel: pid 23080 (rw), uid 0 inumber 129 on /mnt: filesystem full Jul 22 19:09:57 x4 kernel: pid 23082 (rw), uid 0 inumber 5 on /mnt: filesystem full Jul 22 19:09:59 x4 kernel: pid 23083 (rw), uid 0 inumber 129 on /mnt: filesystem full Jul 22 19:10:01 x4 kernel: pid 23078 (rw), uid 0 inumber 129 on /mnt: filesystem full Jul 22 19:10:04 x4 kernel: pid 24172 (rw), uid 0 inumber 132 on /mnt: filesystem full Jul 22 19:10:06 x4 kernel: pid 24174 (rw), uid 0 inumber 5 on /mnt: filesystem full Jul 22 19:10:07 x4 kernel: pid 24173 (rw), uid 0 inumber 130 on /mnt: filesystem full Jul 22 19:10:11 x4 kernel: pid 24171 (rw), uid 0 inumber 130 on /mnt: filesystem full Jul 22 19:10:13 x4 kernel: pid 24952 (rw), uid 0 inumber 130 on /mnt: filesystem full Jul 22 19:10:14 x4 kernel: pid 24953 (rw), uid 0 inumber 5 on /mnt: filesystem full Jul 22 19:10:17 x4 kernel: pid 25266 (rw), uid 0 inumber 135 on /mnt: filesystem full Jul 22 19:10:19 x4 kernel: pid 25267 (rw), uid 0 inumber 6 on /mnt: filesystem full Jul 22 19:10:21 x4 kernel: pid 25268 (rw), uid 0 inumber 130 on /mnt: filesystem full Jul 22 19:10:24 x4 kernel: pid 25263 (rw), uid 0 inumber 4 on /mnt: filesystem full Jul 22 19:10:28 x4 kernel: pid 26253 (rw), uid 0 inumber 138 on /mnt: filesystem full Jul 22 19:10:29 x4 kernel: pid 26257 (rw), uid 0 inumber 7 on /mnt: filesystem full Jul 22 19:10:33 x4 kernel: pid 26255 (rw), uid 0 inumber 129 on /mnt: filesystem full Jul 22 19:12:33 x4 kernel: pid 36051 (rw), uid 0 inumber 130 on /mnt: filesystem full Jul 22 19:12:36 x4 kernel: pid 37062 (rw), uid 0 inumber 135 on /mnt: filesystem full Jul 22 19:12:37 x4 kernel: pid 37061 (rw), uid 0 inumber 130 on /mnt: filesystem full Jul 22 19:12:39 x4 kernel: pid 37060 (rw), uid 0 inumber 130 on /mnt: filesystem full Jul 22 19:12:41 x4 kernel: pid 37063 (rw), uid 0 inumber 5 on /mnt: filesystem full Jul 22 19:12:43 x4 kernel: pid 37058 (rw), uid 0 inumber 130 on /mnt: filesystem full Jul 22 19:12:44 x4 kernel: pid 37059 (rw), uid 0 inumber 129 on /mnt: filesystem full Jul 22 19:12:49 x4 kernel: pid 37203 (rw), uid 0 inumber 5 on /mnt: filesystem full Jul 22 19:14:46 x4 kernel: pid 47888 (rw), uid 0 inumber 137 on /mnt: filesystem full Jul 22 19:14:48 x4 kernel: pid 47893 (rw), uid 0 inumber 130 on /mnt: filesystem full Jul 22 19:14:54 x4 kernel: pid 47892 (rw), uid 0 inumber 7 on /mnt: filesystem full Jul 22 19:14:58 x4 kernel: pid 48982 (rw), uid 0 inumber 136 on /mnt: filesystem full Jul 22 19:15:00 x4 kernel: pid 48980 (rw), uid 0 inumber 131 on /mnt: filesystem full Jul 22 19:15:02 x4 kernel: pid 48984 (rw), uid 0 inumber 6 on /mnt: filesystem full Jul 22 19:15:04 x4 kernel: pid 48981 (rw), uid 0 inumber 4 on /mnt: filesystem full mount leak: 1/104 20190722 19:16:57 all (14/657): nullfs21.sh mount leak: 1/105 20190722 19:17:03 all (15/657): md6.sh 20190722 19:17:08 all (16/657): holdcnt04.sh 20190722 19:17:12 all (17/657): churn.sh 20190722 19:27:17 all (18/657): umountf12.sh mount leak: 1/106 20190722 19:27:35 all (19/657): linger4.sh mount leak: 1/107 20190722 19:47:47 all (20/657): overflow3.sh 20190722 19:49:53 all (21/657): ufsbench.sh mount leak: 5/112 20190722 19:54:09 all (22/657): mmap18.sh 20190722 20:00:47 all (23/657): graid1_7.sh mount leak: 1/113 20190722 20:07:08 all (24/657): crossmp11.sh Jul 22 20:08:09 x4 kernel: pid 97505 (mkdir), uid 1004 inumber 49412 on /mnt10: filesystem full Jul 22 20:08:12 x4 kernel: pid 97539 (rw), uid 1004 inumber 73750 on /mnt10: filesystem full Jul 22 20:09:21 x4 kernel: pid 97482 (rw), uid 1004 inumber 24587 on /mnt11: filesystem full Jul 22 20:09:25 x4 kernel: pid 97535 (rw), uid 1004 inumber 49162 on /mnt10: filesystem full Jul 22 20:09:25 x4 kernel: pid 97561 (rw), uid 1004 inumber 73731 on /mnt13: filesystem full Jul 22 20:09:27 x4 kernel: pid 97476 (rw), uid 1004 inumber 24600 on /mnt11: filesystem full Jul 22 20:09:54 x4 kernel: pid 626 (rw), uid 1004 inumber 24586 on /mnt11: filesystem full Jul 22 20:10:01 x4 kernel: pid 623 (rw), uid 1004 inumber 24581 on /mnt11: filesystem full Jul 22 20:10:02 x4 kernel: pid 627 (rw), uid 1004 inumber 24584 on /mnt11: filesystem full Jul 22 20:10:04 x4 kernel: pid 116 (rw), uid 1004 inumber 73737 on /mnt12: filesystem full Jul 22 20:10:07 x4 kernel: pid 133 (rw), uid 1004 inumber 24586 on /mnt13: filesystem full Jul 22 20:10:08 x4 kernel: pid 109 (mkdir), uid 1004 inumber 24615 on /mnt13: filesystem full Jul 22 20:10:08 x4 kernel: pid 99979 (mkdir), uid 1004 inumber 76 on /mnt12: filesystem full Jul 22 20:10:22 x4 kernel: pid 127 (rw), uid 1004 inumber 8 on /mnt10: filesystem full Jul 22 20:10:51 x4 kernel: pid 1898 (rw), uid 1004 inumber 49161 on /mnt13: filesystem full Jul 22 20:13:50 x4 kernel: pid 3368 (rw), uid 1004 inumber 43467 on /mnt10: filesystem full Jul 22 20:13:52 x4 kernel: pid 3283 (mkdir), uid 1004 inumber 73895 on /mnt10: filesystem full Jul 22 20:13:53 x4 kernel: pid 3373 (rw), uid 1004 inumber 4120 on /mnt10: filesystem full Jul 22 20:13:55 x4 kernel: pid 3369 (rw), uid 1004 inumber 29277 on /mnt10: filesystem full Jul 22 20:13:56 x4 kernel: pid 3182 (mkdir), uid 1004 inumber 24590 on /mnt11: filesystem full Jul 22 20:13:56 x4 kernel: pid 3279 (creat), uid 1004 inumber 73741 on /mnt10: filesystem full Jul 22 20:13:58 x4 kernel: pid 3276 (creat), uid 1004 inumber 73743 on /mnt10: filesystem full Jul 22 20:13:59 x4 kernel: pid 3275 (creat), uid 1004 inumber 73737 on /mnt10: filesystem full Jul 22 20:13:59 x4 kernel: pid 3174 (rw), uid 1004 inumber 24582 on /mnt11: filesystem full Jul 22 20:14:00 x4 kernel: pid 3280 (creat), uid 1004 inumber 73738 on /mnt10: filesystem full Jul 22 20:14:00 x4 kernel: pid 3192 (creat), uid 1004 inumber 49167 on /mnt11: filesystem full mount leak: 1/114 20190722 20:15:22 all (25/657): mountro.sh pts leak: 1/2 20190722 20:19:32 all (26/657): msync.sh 20190722 20:22:46 all (27/657): truncate.sh freework leak: 12802/12841. newblk leak: 11514/12840. 20190722 20:38:18 all (28/657): sort.sh 20190722 20:40:03 all (29/657): snap2.sh lock order reversal: 1st 0x967ec2c bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:283 2nd 0x25c4a31c snaplk (snaplk) @ ufs/ffs/ffs_snapshot.c:2327 stack backtrace: #0 0x102f051 at witness_debugger+0x61 #1 0x102ee29 at witness_checkorder+0x8a9 #2 0xfa65e8 at lockmgr_xlock_hard+0x48 #3 0xfa6d56 at __lockmgr_args+0x376 #4 0x127d09f at ffs_copyonwrite+0x19f #5 0x129ccf3 at ffs_geom_strategy+0x73 #6 0x12ada47 at ufs_strategy+0x87 #7 0x1562688 at VOP_STRATEGY_APV+0x78 #8 0x106b333 at bufstrategy+0x43 #9 0x106b1da at bufwrite+0x1fa #10 0x129ea71 at ffs_syncvnode+0x2b1 #11 0x129db88 at ffs_fsync+0x18 #12 0x1561399 at VOP_FSYNC_APV+0x69 #13 0x12ab9a7 at ufs_remove+0xc7 #14 0x15614fd at VOP_REMOVE_APV+0x8d #15 0x1096ad3 at kern_funlinkat+0x273 #16 0x109684f at sys_unlink+0x1f #17 0x15312d4 at syscall+0x304 lock order reversal: 1st 0x25c4a31c snaplk (snaplk) @ ufs/ufs/ufs_vnops.c:918 2nd 0x339b8e6c ufs (ufs) @ ufs/ffs/ffs_snapshot.c:1650 stack backtrace: #0 0x102f051 at witness_debugger+0x61 #1 0x102ee29 at witness_checkorder+0x8a9 #2 0xfa65e8 at lockmgr_xlock_hard+0x48 #3 0xfa6d56 at __lockmgr_args+0x376 #4 0x127b52f at ffs_snapremove+0xcf #5 0x1275608 at ffs_truncate+0x488 #6 0x12a43b4 at ufs_inactive+0x1d4 #7 0x1562059 at VOP_INACTIVE_APV+0x69 #8 0x108c583 at vinactive+0xe3 #9 0x108cb4f at vputx+0x2ef #10 0x1096b24 at kern_funlinkat+0x2c4 #11 0x109684f at sys_unlink+0x1f #12 0x15312d4 at syscall+0x304 #13 0xffc033b7 at _end+0xfdc4f0c7 mount leak: 1/115 20190722 20:40:39 all (30/657): mountro2.sh 20190722 20:40:47 all (31/657): pathconf2.sh 20190722 20:40:50 all (32/657): pthread2.sh 20190722 21:02:47 all (33/657): rename13.sh mount leak: 1/116 20190722 21:02:52 all (34/657): nfs14.sh 20190722 21:02:56 all (35/657): tvnlru.sh 20190722 21:03:02 all (36/657): ufssuspend.sh 20190722 21:03:08 all (37/657): nfs15lockd3.sh 20190722 21:03:14 all (38/657): tcp3.sh 20190722 21:04:26 all (39/657): fpclone2.sh 20190722 21:05:01 all (40/657): sendfile.sh lock order reversal: 1st 0x218f216c so_snd_sx (so_snd_sx) @ kern/uipc_sockbuf.c:385 2nd 0x269d4a28 ufs (ufs) @ kern/kern_sendfile.c:733 stack backtrace: #0 0x102f051 at witness_debugger+0x61 #1 0x102ee29 at witness_checkorder+0x8a9 #2 0xfa62e5 at lockmgr_slock_hard+0x65 #3 0xfa6b06 at __lockmgr_args+0x126 #4 0x129dccf at ffs_lock+0x6f #5 0x15622d8 at VOP_LOCK1_APV+0x58 #6 0x109d3e8 at _vn_lock+0x58 #7 0xfcb9b7 at vn_sendfile+0x7d7 #8 0xfcd816 at sendfile+0x126 #9 0xfcd6e0 at sys_sendfile+0x10 #10 0x15312d4 at syscall+0x304 #11 0xffc033b7 at _end+0xfdc4f0c7 20190722 21:05:09 all (41/657): tmpfs11.sh mount leak: 1/117 20190722 21:20:35 all (42/657): temp.sh 20190722 21:20:45 all (43/657): tmpfs19.sh mount leak: 1/118 20190722 21:22:03 all (44/657): graid1.sh mount leak: 1/119 20190722 21:33:06 all (45/657): graid0.sh GEOM_STRIPE: Device data created (id=798237371). GEOM_STRIPE: Disk md10 attached to data. GEOM_STRIPE: Disk md11 attached to data. GEOM_STRIPE: Disk md12 attached to data. GEOM_STRIPE: Device stripe/data activated. GEOM_STRIPE: Disk md12 removed from data. GEOM_STRIPE: Device stripe/data deactivated. GEOM_STRIPE: Disk md11 removed from data. GEOM_STRIPE: Disk md10 removed from data. GEOM_STRIPE: Device data destroyed. mount leak: 1/120 20190722 21:43:49 all (46/657): zzbuildworld.sh 20190723 02:20:41 *** Excessive run time: zzbuildworld.sh, 276 min mount leak: 1/121 20190723 02:20:45 all (47/657): overlap.sh mount leak: 1/122 20190723 02:25:53 all (48/657): trim.sh Jul 23 02:36:41 x4 kernel: pid 11364 (link), uid 1004 inumber 12308 on /mnt: out of inodes Jul 23 02:36:42 x4 kernel: pid 11335 (creat), uid 1004 inumber 8219 on /mnt: out of inodes Jul 23 02:36:47 x4 kernel: pid 11323 (creat), uid 1004 inumber 8211 on /mnt: out of inodes Jul 23 02:36:49 x4 kernel: pid 11275 (symlink), uid 1004 inumber 8232 on /mnt: out of inodes Jul 23 02:36:51 x4 kernel: pid 11324 (creat), uid 1004 inumber 8218 on /mnt: out of inodes Jul 23 02:36:54 x4 kernel: pid 11363 (link), uid 1004 inumber 12435 on /mnt: out of inodes Jul 23 02:36:59 x4 kernel: pid 11319 (creat), uid 1004 inumber 8203 on /mnt: out of inodes Jul 23 02:37:01 x4 kernel: pid 11292 (openat), uid 1004 inumber 8209 on /mnt: out of inodes Jul 23 02:37:02 x4 kernel: pid 11318 (creat), uid 1004 inumber 8210 on /mnt: out of inodes Jul 23 02:37:03 x4 kernel: pid 11337 (creat), uid 1004 inumber 8214 on /mnt: out of inodes Jul 23 02:37:05 x4 kernel: pid 11355 (link), uid 1004 inumber 12298 on /mnt: out of inodes Jul 23 02:37:06 x4 kernel: pid 11326 (creat), uid 1004 inumber 12301 on /mnt: out of inodes Jul 23 02:37:07 x4 kernel: pid 11320 (creat), uid 1004 inumber 8217 on /mnt: out of inodes Jul 23 02:37:09 x4 kernel: pid 11319 (creat), uid 1004 inumber 8203 on /mnt: out of inodes Jul 23 02:37:11 x4 kernel: pid 11317 (creat), uid 1004 inumber 12425 on /mnt: out of inodes Jul 23 02:37:12 x4 kernel: pid 11339 (creat), uid 1004 inumber 12292 on /mnt: out of inodes Jul 23 02:37:14 x4 kernel: pid 11322 (creat), uid 1004 inumber 12288 on /mnt: out of inodes Jul 23 02:37:16 x4 kernel: pid 11339 (creat), uid 1004 inumber 12292 on /mnt: out of inodes Jul 23 02:37:20 x4 kernel: pid 11354 (link), uid 1004 inumber 12296 on /mnt: out of inodes Jul 23 02:37:21 x4 kernel: pid 11357 (link), uid 1004 inumber 12428 on /mnt: out of inodes Jul 23 02:37:23 x4 kernel: pid 11323 (creat), uid 1004 inumber 8211 on /mnt: out of inodes Jul 23 02:37:24 x4 kernel: pid 11339 (creat), uid 1004 inumber 12292 on /mnt: out of inodes Jul 23 02:37:28 x4 kernel: pid 11329 (creat), uid 1004 inumber 8223 on /mnt: out of inodes Jul 23 02:37:30 x4 kernel: pid 11339 (creat), uid 1004 inumber 12292 on /mnt: out of inodes Jul 23 02:37:33 x4 kernel: pid 11318 (creat), uid 1004 inumber 8210 on /mnt: out of inodes Jul 23 02:37:36 x4 kernel: pid 11324 (creat), uid 1004 inumber 8218 on /mnt: out of inodes Jul 23 02:37:38 x4 kernel: pid 11322 (creat), uid 1004 inumber 12288 on /mnt: out of inodes Jul 23 02:37:40 x4 kernel: pid 11328 (creat), uid 1004 inumber 8220 on /mnt: out of inodes Jul 23 02:37:41 x4 kernel: pid 11337 (creat), uid 1004 inumber 8214 on /mnt: out of inodes Jul 23 02:37:43 x4 kernel: pid 11322 (creat), uid 1004 inumber 12288 on /mnt: out of inodes Jul 23 02:37:45 x4 kernel: pid 11319 (creat), uid 1004 inumber 8203 on /mnt: out of inodes Jul 23 02:37:46 x4 kernel: pid 11323 (creat), uid 1004 inumber 8211 on /mnt: out of inodes Jul 23 02:37:48 x4 kernel: pid 11316 (creat), uid 1004 inumber 8202 on /mnt: out of inodes Jul 23 02:37:50 x4 kernel: pid 11339 (creat), uid 1004 inumber 12292 on /mnt: out of inodes Jul 23 02:37:54 x4 kernel: pid 11329 (creat), uid 1004 inumber 8223 on /mnt: out of inodes Jul 23 02:37:55 x4 kernel: pid 11331 (creat), uid 1004 inumber 8212 on /mnt: out of inodes Jul 23 02:37:58 x4 kernel: pid 11357 (link), uid 1004 inumber 12428 on /mnt: out of inodes Jul 23 02:38:01 x4 kernel: pid 11337 (creat), uid 1004 inumber 8214 on /mnt: out of inodes Jul 23 02:38:02 x4 kernel: pid 11361 (link), uid 1004 inumber 12307 on /mnt: out of inodes Jul 23 02:38:03 x4 syslogd: last message repeated 1 times Jul 23 02:38:06 x4 kernel: pid 11339 (creat), uid 1004 inumber 12292 on /mnt: out of inodes Jul 23 02:38:08 x4 kernel: pid 11322 (creat), uid 1004 inumber 12288 on /mnt: out of inodes Jul 23 02:38:10 x4 kernel: pid 11331 (creat), uid 1004 inumber 8212 on /mnt: out of inodes Jul 23 02:38:12 x4 kernel: pid 11326 (creat), uid 1004 inumber 12301 on /mnt: out of inodes Jul 23 02:38:14 x4 kernel: pid 11328 (creat), uid 1004 inumber 8220 on /mnt: out of inodes Jul 23 02:38:15 x4 kernel: pid 11317 (creat), uid 1004 inumber 12425 on /mnt: out of inodes Jul 23 02:38:24 x4 kernel: pid 11329 (creat), uid 1004 inumber 8223 on /mnt: out of inodes Jul 23 02:38:25 x4 kernel: pid 11339 (creat), uid 1004 inumber 12292 on /mnt: out of inodes Jul 23 02:38:26 x4 kernel: pid 11325 (creat), uid 1004 inumber 8207 on /mnt: out of inodes Jul 23 02:38:43 x4 kernel: pid 11924 (mkdir), uid 1004 inumber 11309 on /mnt: out of inodes Jul 23 02:38:52 x4 kernel: pid 11875 (mkdir), uid 1004 inumber 1463 on /mnt: filesystem full Jul 23 02:39:06 x4 kernel: pid 11891 (mkdir), uid 1004 inumber 12360 on /mnt: out of inodes Jul 23 02:39:17 x4 kernel: pid 11849 (openat), uid 1004 inumber 12332 on /mnt: out of inodes Jul 23 02:39:41 x4 kernel: pid 11868 (creat), uid 1004 inumber 8244 on /mnt: out of inodes Jul 23 02:39:46 x4 kernel: pid 11882 (mkdir), uid 1004 inumber 11874 on /mnt: out of inodes Jul 23 02:40:00 x4 kernel: pid 11874 (creat), uid 1004 inumber 8213 on /mnt: out of inodes Jul 23 02:40:17 x4 kernel: pid 11827 (symlink), uid 1004 inumber 8204 on /mnt: out of inodes Jul 23 02:40:26 x4 kernel: pid 11874 (creat), uid 1004 inumber 8213 on /mnt: out of inodes Jul 23 02:43:50 x4 kernel: pid 15368 (mkdir), uid 1004 inumber 12252 on /mnt: out of inodes Jul 23 02:43:55 x4 kernel: pid 15410 (creat), uid 1004 inumber 151 on /mnt: out of inodes Jul 23 02:44:03 x4 kernel: pid 15397 (creat), uid 1004 inumber 145 on /mnt: out of inodes Jul 23 02:44:07 x4 kernel: pid 15396 (creat), uid 1004 inumber 144 on /mnt: out of inodes Jul 23 02:44:13 x4 kernel: pid 15348 (rw), uid 1004 inumber 3033 on /mnt: filesystem full Jul 23 02:44:25 x4 kernel: pid 15418 (creat), uid 1004 inumber 149 on /mnt: out of inodes Jul 23 02:45:57 x4 kernel: pid 16682 (openat), uid 1004 inumber 11547 on /mnt: out of inodes Jul 23 02:46:09 x4 kernel: pid 16663 (openat), uid 1004 inumber 11539 on /mnt: out of inodes Jul 23 02:47:17 x4 kernel: pid 17960 (creat), uid 1004 inumber 4096 on /mnt: out of inodes Jul 23 02:47:19 x4 kernel: pid 17933 (openat), uid 1004 inumber 22 on /mnt: out of inodes Jul 23 02:48:02 x4 kernel: pid 17983 (creat), uid 1004 inumber 29 on /mnt: out of inodes Jul 23 02:48:52 x4 kernel: pid 18007 (rw), uid 1004 inumber 4103 on /mnt: out of inodes Jul 23 02:53:25 x4 kernel: pid 20765 (creat), uid 1004 inumber 4102 on /mnt: out of inodes Jul 23 02:53:30 x4 kernel: pid 20780 (creat), uid 1004 inumber 4103 on /mnt: out of inodes Jul 23 02:53:35 x4 kernel: pid 20770 (creat), uid 1004 inumber 4099 on /mnt: out of inodes Jul 23 02:53:37 x4 kernel: pid 20784 (creat), uid 1004 inumber 4104 on /mnt: out of inodes Jul 23 02:53:40 x4 kernel: pid 20789 (creat), uid 1004 inumber 12475 on /mnt: out of inodes Jul 23 02:53:41 x4 kernel: pid 20790 (creat), uid 1004 inumber 4107 on /mnt: out of inodes Jul 23 02:53:43 x4 kernel: pid 20759 (rw), uid 1004 inumber 4097 on /mnt: out of inodes Jul 23 02:53:44 x4 kernel: pid 20782 (creat), uid 1004 inumber 12318 on /mnt: out of inodes Jul 23 02:53:46 x4 kernel: pid 20776 (creat), uid 1004 inumber 12357 on /mnt: out of inodes Jul 23 02:53:49 x4 kernel: pid 20756 (creat), uid 1004 inumber 12312 on /mnt: out of inodes Jul 23 02:53:50 x4 kernel: pid 20770 (creat), uid 1004 inumber 4099 on /mnt: out of inodes Jul 23 02:53:52 x4 kernel: pid 20780 (creat), uid 1004 inumber 4103 on /mnt: out of inodes Jul 23 02:53:53 x4 kernel: pid 20770 (creat), uid 1004 inumber 4099 on /mnt: out of inodes Jul 23 02:53:54 x4 kernel: pid 20752 (creat), uid 1004 inumber 12314 on /mnt: out of inodes Jul 23 02:53:56 x4 kernel: pid 20780 (creat), uid 1004 inumber 4103 on /mnt: out of inodes Jul 23 02:53:57 x4 kernel: pid 20762 (creat), uid 1004 inumber 12315 on /mnt: out of inodes Jul 23 02:53:59 x4 kernel: pid 20786 (creat), uid 1004 inumber 4100 on /mnt: out of inodes Jul 23 02:54:01 x4 kernel: pid 20784 (creat), uid 1004 inumber 4104 on /mnt: out of inodes Jul 23 02:54:04 x4 kernel: pid 20782 (creat), uid 1004 inumber 12318 on /mnt: out of inodes Jul 23 02:54:06 x4 kernel: pid 20767 (creat), uid 1004 inumber 4105 on /mnt: out of inodes Jul 23 02:54:08 x4 kernel: pid 20790 (creat), uid 1004 inumber 4107 on /mnt: out of inodes Jul 23 02:54:10 x4 syslogd: last message repeated 1 times Jul 23 02:54:12 x4 kernel: pid 20782 (creat), uid 1004 inumber 12318 on /mnt: out of inodes Jul 23 02:54:13 x4 kernel: pid 20765 (creat), uid 1004 inumber 4102 on /mnt: out of inodes Jul 23 02:54:16 x4 kernel: pid 20758 (creat), uid 1004 inumber 12305 on /mnt: out of inodes Jul 23 02:54:18 x4 kernel: pid 20776 (creat), uid 1004 inumber 12357 on /mnt: out of inodes Jul 23 02:54:21 x4 kernel: pid 20790 (creat), uid 1004 inumber 4107 on /mnt: out of inodes Jul 23 02:54:26 x4 kernel: pid 20786 (creat), uid 1004 inumber 4100 on /mnt: out of inodes Jul 23 02:54:27 x4 kernel: pid 20773 (creat), uid 1004 inumber 4108 on /mnt: out of inodes Jul 23 02:54:31 x4 kernel: pid 20758 (creat), uid 1004 inumber 12305 on /mnt: out of inodes Jul 23 02:54:36 x4 kernel: pid 20784 (creat), uid 1004 inumber 4104 on /mnt: out of inodes Jul 23 02:54:38 x4 kernel: pid 20786 (creat), uid 1004 inumber 4100 on /mnt: out of inodes Jul 23 02:54:41 x4 kernel: pid 20758 (creat), uid 1004 inumber 12305 on /mnt: out of inodes Jul 23 02:54:42 x4 kernel: pid 20749 (creat), uid 1004 inumber 12300 on /mnt: out of inodes Jul 23 02:54:45 x4 kernel: pid 20782 (creat), uid 1004 inumber 12318 on /mnt: out of inodes Jul 23 02:54:47 x4 kernel: pid 20752 (creat), uid 1004 inumber 12314 on /mnt: out of inodes Jul 23 02:54:48 x4 kernel: pid 20770 (creat), uid 1004 inumber 4099 on /mnt: out of inodes Jul 23 02:54:50 x4 kernel: pid 20784 (creat), uid 1004 inumber 4104 on /mnt: out of inodes Jul 23 02:54:53 x4 kernel: pid 20756 (creat), uid 1004 inumber 12312 on /mnt: out of inodes Jul 23 02:54:55 x4 kernel: pid 20749 (creat), uid 1004 inumber 12300 on /mnt: out of inodes Jul 23 02:54:57 x4 kernel: pid 20765 (creat), uid 1004 inumber 4102 on /mnt: out of inodes Jul 23 02:54:58 x4 kernel: pid 20767 (creat), uid 1004 inumber 4105 on /mnt: out of inodes Jul 23 02:55:02 x4 kernel: pid 20762 (creat), uid 1004 inumber 12315 on /mnt: out of inodes Jul 23 02:55:03 x4 kernel: pid 20773 (creat), uid 1004 inumber 4108 on /mnt: out of inodes Jul 23 02:55:04 x4 kernel: pid 20758 (creat), uid 1004 inumber 12305 on /mnt: out of inodes Jul 23 02:55:06 x4 kernel: pid 20789 (creat), uid 1004 inumber 12475 on /mnt: out of inodes Jul 23 02:55:08 x4 kernel: pid 20749 (creat), uid 1004 inumber 12300 on /mnt: out of inodes Jul 23 02:55:11 x4 syslogd: last message repeated 1 times Jul 23 02:55:13 x4 kernel: pid 20765 (creat), uid 1004 inumber 4102 on /mnt: out of inodes Jul 23 02:55:15 x4 kernel: pid 20787 (creat), uid 1004 inumber 4106 on /mnt: out of inodes Jul 23 02:55:17 x4 kernel: pid 20776 (creat), uid 1004 inumber 12357 on /mnt: out of inodes Jul 23 02:55:20 x4 kernel: pid 20773 (creat), uid 1004 inumber 4108 on /mnt: out of inodes Jul 23 02:55:21 x4 kernel: pid 20782 (creat), uid 1004 inumber 12318 on /mnt: out of inodes Jul 23 02:55:41 x4 kernel: pid 22914 (creat), uid 1004 inumber 4267 on /mnt: out of inodes Jul 23 02:55:49 x4 kernel: pid 22896 (rw), uid 1004 inumber 1157 on /mnt: filesystem full Jul 23 02:55:55 x4 kernel: pid 22914 (creat), uid 1004 inumber 4267 on /mnt: out of inodes Jul 23 02:56:13 x4 kernel: pid 22910 (creat), uid 1004 inumber 4259 on /mnt: out of inodes Jul 23 02:56:30 x4 kernel: pid 22930 (creat), uid 1004 inumber 4270 on /mnt: out of inodes Jul 23 02:56:33 x4 kernel: pid 22895 (creat), uid 1004 inumber 4250 on /mnt: out of inodes Jul 23 02:56:36 x4 kernel: pid 22889 (creat), uid 1004 inumber 4255 on /mnt: out of inodes Jul 23 02:56:50 x4 kernel: pid 22892 (creat), uid 1004 inumber 4276 on /mnt: out of inodes Jul 23 02:56:56 x4 kernel: pid 22860 (link), uid 1004 inumber 4243 on /mnt: out of inodes Jul 23 02:57:02 x4 kernel: pid 22938 (creat), uid 1004 inumber 4307 on /mnt: out of inodes Jul 23 02:57:05 x4 kernel: pid 22899 (creat), uid 1004 inumber 4248 on /mnt: out of inodes Jul 23 02:57:17 x4 kernel: pid 22938 (creat), uid 1004 inumber 4307 on /mnt: out of inodes Jul 23 02:57:20 x4 kernel: pid 22861 (link), uid 1004 inumber 4247 on /mnt: out of inodes mount leak: 3/125 20190723 02:57:27 all (49/657): ffs_syncvnode2.sh Jul 23 03:00:05 x4 kernel: pid 37533 (mkdir), uid 1004 inumber 71993 on /mnt: filesystem full Jul 23 03:08:15 x4 kernel: pid 59919 (rw), uid 1004 inumber 65552 on /mnt: filesystem full Jul 23 03:11:49 x4 kernel: pid 74848 (rw), uid 1004 inumber 196652 on /mnt: filesystem full Jul 23 03:13:50 x4 kernel: pid 82345 (rw), uid 1004 inumber 68838 on /mnt: filesystem full Jul 23 03:16:05 x4 kernel: pid 90851 (rw), uid 1004 inumber 200655 on /mnt: filesystem full mount leak: 1/126 20190723 03:17:51 all (50/657): mkfifo2c.sh mount leak: 1/127 20190723 03:50:10 all (51/657): namecache2.sh mount leak: 1/128 20190723 03:53:35 all (52/657): tmpfs13.sh mount leak: 1/129 20190723 03:55:53 all (53/657): pipe.sh 20190723 04:01:00 all (54/657): machipc.sh 20190723 04:01:04 all (55/657): fsck.sh Superblock check-hash failed: recorded check-hash 0x8b4ba6d4 != computed check-hash 0x13105432 Superblock check-hash failed: recorded check-hash 0x8b4ba6d4 != computed check-hash 0x13105432 20190723 04:02:11 all (56/657): msdos4.sh 20190723 04:13:13 all (57/657): mknod.sh mount leak: 1/130 20190723 04:14:25 all (58/657): ftruncate.sh 20190723 04:17:35 all (59/657): mmap26.sh 20190723 04:17:42 all (60/657): msdos8.sh mount leak: 1/131 20190723 04:17:48 all (61/657): ptrace11.sh 20190723 04:17:55 all (62/657): indir_trunc.sh 20190723 04:18:01 all (63/657): radix.sh 20190723 04:18:31 all (64/657): jail2.sh 20190723 04:18:42 all (65/657): sendfile_shm.sh newblk leak: 2486/15326. 20190723 04:19:33 all (66/657): fcntl.sh 20190723 04:19:43 all (67/657): routetbl.sh mount leak: 2/133 20190723 04:20:48 all (68/657): sem_wait.sh 20190723 04:21:54 all (69/657): mkfifo2d.sh mount leak: 1/134 20190723 04:28:36 all (70/657): quota11.sh lock order reversal: 1st 0x33214034 syncer (syncer) @ kern/vfs_subr.c:2267 2nd 0x26cc5478 ufs (ufs) @ kern/vfs_subr.c:2687 stack backtrace: #0 0x102f051 at witness_debugger+0x61 #1 0x102ee29 at witness_checkorder+0x8a9 #2 0xfa65e8 at lockmgr_xlock_hard+0x48 #3 0xfa6d56 at __lockmgr_args+0x376 #4 0x129dccf at ffs_lock+0x6f #5 0x15622d8 at VOP_LOCK1_APV+0x58 #6 0x109d3e8 at _vn_lock+0x58 #7 0x108c19b at vget+0x7b #8 0x12a9455 at qsync+0x55 #9 0x129c625 at ffs_sync+0x585 #10 0x1092eef at sync_fsync+0xcf #11 0x1561399 at VOP_FSYNC_APV+0x69 #12 0x1090f37 at sched_sync+0x277 #13 0xf941dc at fork_exit+0x6c #14 0xffc033ca at _end+0xfdc4f0da lock order reversal: 1st 0x2d347b48 FFS (FFS Lock) @ ufs/ufs/ufs_quota.c:757 2nd 0x26932000 struct mount mtx (struct mount mtx) @ ufs/ufs/ufs_quota.c:763 stack backtrace: #0 0x102f051 at witness_debugger+0x61 #1 0x102ee29 at witness_checkorder+0x8a9 #2 0xfb0616 at __mtx_lock_flags+0x76 #3 0x12a89be at quotaoff_inchange+0x9e #4 0x12a8a93 at quotaoff+0x93 #5 0x12995ee at ffs_flushfiles+0x12e #6 0x127e2d1 at softdep_flushfiles+0x61 #7 0x129bae2 at ffs_unmount+0x72 #8 0x10854f5 at dounmount+0x415 #9 0x1084feb at sys_unmount+0x26b #10 0x15312d4 at syscall+0x304 #11 0xffc033b7 at _end+0xfdc4f0c7 mount leak: 1/135 20190723 04:29:24 all (71/657): tmpfs9.sh mount leak: 1/136 20190723 04:47:06 all (72/657): crossmp.sh mount leak: 1/137 20190723 04:53:30 all (73/657): thr2.sh 20190723 05:14:43 all (74/657): suj.sh 20190723 05:34:57 all (75/657): tmpfs15.sh mount leak: 1/138 20190723 05:35:03 all (76/657): extattr2.sh 20190723 05:46:28 all (77/657): mmap34.sh mount leak: 1/139 20190723 05:51:43 all (78/657): mmap36.sh 20190723 05:51:49 all (79/657): full.sh lock order reversal: 1st 0x219b1b94 so_snd_sx (so_snd_sx) @ kern/uipc_sockbuf.c:388 2nd 0x1895705c bufwait (bufwait) @ vm/vm_pager.c:379 stack backtrace: #0 0x102f051 at witness_debugger+0x61 #1 0x102ee29 at witness_checkorder+0x8a9 #2 0xfa65e8 at lockmgr_xlock_hard+0x48 #3 0xfa6d56 at __lockmgr_args+0x376 #4 0x12dc479 at pbuf_ctor+0x79 #5 0x12b6fcb at uma_zalloc_arg+0xa0b #6 0x12b02c2 at swap_pager_getpages+0x2a2 #7 0x12dc6a4 at vm_pager_get_pages+0x44 #8 0x12bdbc8 at vm_fault_hold+0x1938 #9 0x12bf1b4 at vm_fault_quick_hold_pages+0x114 #10 0x150b8b6 at cp_slow0+0x56 #11 0x150bbc7 at copyin+0xa7 #12 0x10288b6 at uiomove_faultflag+0xf6 #13 0x10287b3 at uiomove+0x13 #14 0x1051334 at m_uiotombuf+0x124 #15 0x1059c57 at sosend_generic+0x2d7 #16 0x1059f10 at sosend+0x50 #17 0x103a332 at soo_write+0x32 Expensive timeout(9) function: 0x11be370(0x24e272e4) 0.010204048 s mount leak: 1/140 20190723 06:04:02 all (80/657): execve.sh 20190723 06:05:58 all (81/657): ext2fs3.sh mount leak: 1/141 20190723 06:06:07 all (82/657): rdgsbase.sh 20190723 06:06:11 all (83/657): mkfifo6.sh mount leak: 1/142 20190723 06:06:21 all (84/657): crossmp5.sh mount leak: 1/143 20190723 06:07:23 all (85/657): ffs_blkfree.sh newblk leak: 50236/65562. mount leak: 10/153 20190723 06:07:46 all (86/657): graid1_3.sh 20190723 06:09:59 all (87/657): trim4.sh mount leak: 3/156 20190723 06:30:16 all (88/657): devfs3.sh mount leak: 1/157 20190723 06:35:25 all (89/657): nullfs3.sh 20190723 06:35:32 all (90/657): advlock.sh mount leak: 1/158 20190723 06:36:44 all (91/657): kinfo2.sh 20190723 06:51:51 all (92/657): mmap.sh 20190723 06:51:57 all (93/657): zz-combo01.sh witness_lock_list_get: witness exhausted Approaching the limit on PV entries, consider increasing either the vm.pmap.shpgperproc or the vm.pmap.pv_entries tunable. Jul 23 06:55:24 x4 kernel: pid 74778 (swap), jid 0, uid 0, was killed: out of swap space Jul 23 06:55:25 x4 kernel: pid 74777 (swap), jid 0, uid 0, was killed: out of swap space Approaching the limit on PV entries, consider increasing either the vm.pmap.shpgperproc or the vm.pmap.pv_entries tunable. Expensive timeout(9) function: 0x11bf6c0(0x2a719000) 0.015066528 s Approaching the limit on PV entries, consider increasing either the vm.pmap.shpgperproc or the vm.pmap.pv_entries tunable. Approaching the limit on PV entries, consider increasing either the vm.pmap.shpgperproc or the vm.pmap.pv_entries tunable. Expensive timeout(9) function: 0xfe3c90(0x2c7e2354) 0.019020260 s 20190723 07:20:40 all (94/657): crossmp9.sh mount leak: 1/159 20190723 07:25:49 all (95/657): jumbo.sh 20190723 07:27:34 all (96/657): nfsdelegation.sh 20190723 07:27:40 all (97/657): pthread4.sh 20190723 07:52:55 all (98/657): nullfs14.sh 20190723 07:53:00 all (99/657): truss.sh Jul 23 07:54:52 x4 kernel: pid 31673 (swap), jid 0, uid 0, was killed: out of swap space Jul 23 07:56:52 x4 kernel: pid 31683 (swap), jid 0, uid 0, was killed: out of swap space Jul 23 07:57:01 x4 kernel: pid 31688 (swap), jid 0, uid 0, was killed: out of swap space 20190723 07:59:30 all (100/657): nfs3.sh 20190723 07:59:35 all (101/657): nullfs11.sh mount leak: 1/160 20190723 08:04:05 all (102/657): procfs3.sh 20190723 08:04:29 all (103/657): marcus.sh Jul 23 08:15:18 x4 kernel: pid 77126 (rw), uid 1004 inumber 98579 on /mnt: filesystem full 2mount leak: 1/161 20190723 08:36:25 all (104/657): crossmp3.sh mount leak: 1/162 20190723 08:42:51 all (105/657): umountf11.sh 20190723 08:43:09 all (106/657): pshared.sh 20190723 08:43:33 all (107/657): jail4.sh 20190723 08:43:39 all (108/657): flock.sh mount leak: 1/163 20190723 08:44:45 all (109/657): gnop4.sh GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. mount leak: 1/164 20190723 09:12:54 all (110/657): dangling.sh mount leak: 1/165 20190723 09:23:04 all (111/657): trim6.sh 20190723 09:23:08 all (112/657): rename12.sh mount leak: 1/166 20190723 09:26:40 all (113/657): tmpfs.sh mount leak: 1/167 20190723 09:37:14 all (114/657): swap5.sh 20190723 09:37:20 all (115/657): umountf6.sh mount leak: 1/168 20190723 09:39:12 all (116/657): umountf8.sh 20190723 09:41:48 all (117/657): revoke.sh 20190723 09:41:58 all (118/657): beneath2.sh mount leak: 1/169 20190723 09:42:04 all (119/657): mdconfig.sh 20190723 09:42:10 all (120/657): mountu.sh mount leak: 3/172 20190723 09:42:32 all (121/657): vmstat.sh 20190723 09:43:47 all (122/657): stealer.sh warning: total configured swap (4194304 pages) exceeds maximum recommended amount (2009920 pages). warning: increase kern.maxswzone or reduce amount of swap. 20190723 09:48:00 all (123/657): fullpath2.sh mount leak: 1/173 20190723 09:54:09 all (124/657): msetdomain.sh 20190723 09:54:15 all (125/657): ruby.sh 20190723 09:54:18 all (126/657): mmap37.sh mount leak: 1/174 20190723 09:55:00 all (127/657): tmpfs5.sh mount leak: 1/175 20190723 09:55:17 all (128/657): tmpfs16.sh mount leak: 1/176 20190723 09:58:23 all (129/657): pthread7.sh 20190723 10:22:51 all (130/657): nfs12.sh 20190723 10:22:57 all (131/657): random.sh 20190723 10:23:03 all (132/657): vm_map.sh 20190723 10:25:11 all (133/657): mmap17.sh 20190723 10:25:17 all (134/657): snap11.sh mount leak: 1/177 20190723 10:27:28 all (135/657): pts.sh Jul 23 10:30:38 x4 kernel: pid 27363 (swap), jid 0, uid 0, was killed: out of swap space pts leak: 82/84 20190723 10:38:01 all (136/657): datamove.sh 20190723 10:38:08 all (137/657): sendmsg.sh 20190723 10:43:19 all (138/657): snap7.sh Jul 23 10:44:20 x4 mountd[554]: can't export /mnt Jul 23 10:45:20 x4 syslogd: last message repeated 1 times Jul 23 10:49:48 x4 syslogd: last message repeated 1 times Jul 23 11:00:10 x4 syslogd: last message repeated 6 times mount leak: 2211/2388 20190723 11:03:54 all (139/657): suj5.sh mount leak: 1/2389 20190723 11:05:11 all (140/657): posix_fadvise.sh 20190723 11:05:17 all (141/657): nbufkv.sh 20190723 11:05:23 all (142/657): zfs9.sh ZFS NOTICE: Prefetch is disabled by default on i386 -- to enable, add "vfs.zfs.prefetch_disable=0" to /boot/loader.conf. ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Jul 23 11:03:16 x4 syslogd: last message repeated 2 times Jul 23 11:08:53 x4 kernel: pid 95219 (swap), jid 0, uid 0, was killed: out of swap space Jul 23 11:08:55 x4 kernel: pid 95200 (swap), jid 0, uid 0, was killed: out of swap space Jul 23 11:14:34 x4 kernel: pid 3028 (swap), jid 0, uid 0, was killed: out of swap space Warning: memory type solaris leaked memory on destroy (4 allocations, 576 bytes leaked). 20190723 11:18:48 all (143/657): tmpfs7.sh mount leak: 1/2390 20190723 11:19:00 all (144/657): mmap13.sh 20190723 11:19:16 all (145/657): sendfile17.sh mount leak: 1/2391 20190723 11:24:25 all (146/657): mmap6.sh 20190723 11:29:30 all (147/657): suj6.sh mount leak: 1/2392 20190723 11:29:42 all (148/657): holdcnt03.sh 20190723 11:29:46 all (149/657): sem.sh 20190723 11:30:52 all (150/657): ptrace10.sh 20190723 11:32:42 all (151/657): ffs_sync.sh mount leak: 1/2393 20190723 11:35:50 all (152/657): crossmp2.sh 20190723 11:35:55 all (153/657): newfs2.sh mount leak: 28/2421 20190723 11:57:26 all (154/657): umountf3.sh mount leak: 1/2422 20190723 12:01:42 all (155/657): thr.sh 20190723 12:01:57 all (156/657): readdir.sh Jul 23 12:06:13 x4 mountd[554]: can't export /mnt mount leak: 4/2426 20190723 12:14:16 all (157/657): mmap12.sh 20190723 12:14:22 all (158/657): thr3.sh 20190723 12:17:28 all (159/657): md4.sh 20190723 12:17:32 all (160/657): 256m.sh 20190723 12:17:36 all (161/657): tmpfs12.sh 20190723 12:17:39 all (162/657): sendfile5.sh mount leak: 3/2429 20190723 12:19:21 all (163/657): rename7.sh mount leak: 1/2430 20190723 12:20:35 all (164/657): zz-combo02.sh Approaching the limit on PV entries, consider increasing either the vm.pmap.shpgperproc or the vm.pmap.pv_entries tunable. Approaching the limit on PV entries, consider increasing either the vm.pmap.shpgperproc or the vm.pmap.pv_entries tunable. 20190723 12:26:14 all (165/657): extattr_set_fd.sh mount leak: 1/2431 20190723 12:26:20 all (166/657): mmap25.sh 20190723 12:26:27 all (167/657): softupdate.sh mount leak: 2/2433 20190723 12:47:25 all (168/657): devfs.sh mount leak: 1/2434 20190723 12:52:34 all (169/657): mmap2.sh 20190723 13:04:01 all (170/657): callout_reset_on.sh 20190723 13:07:54 all (171/657): symlink4.sh 20190723 13:08:11 all (172/657): alternativeFlushPath.sh 20190723 13:28:55 all (173/657): fpu.sh 20190723 13:31:38 all (174/657): mount2.sh mount leak: 1/2435 20190723 13:36:46 all (175/657): unlink.sh 20190723 13:36:52 all (176/657): crossmp8.sh 20190723 13:38:06 all (177/657): db.sh 20190723 13:41:12 all (178/657): kevent7.sh 20190723 13:46:24 all (179/657): rename3.sh 20190723 13:49:43 all (180/657): getrandom.sh mount leak: 1/2436 20190723 13:55:06 all (181/657): kevent3.sh 20190723 13:55:19 all (182/657): tmpfs4.sh mount leak: 1/2437 20190723 13:55:26 all (183/657): ptrace4.sh Jul 23 13:57:14 x4 kernel: pid 6962 (swap), jid 0, uid 0, was killed: out of swap space Jul 23 13:57:17 x4 kernel: pid 6980 (swap), jid 0, uid 0, was killed: out of swap space mount leak: 14/2451 GEOM leak: 23/276 20190723 13:59:41 all (184/657): sysctl4.sh 20190723 14:02:46 all (185/657): mmap24.sh 20190723 14:02:52 all (186/657): quota1.sh Jul 23 14:03:16 x4 kernel: pid 22705 (creat), uid 0 inumber 8193 on /mnt: out of inodes Jul 23 14:03:18 x4 kernel: pid 22709 (creat), uid 0 inumber 12288 on /mnt: out of inodes Jul 23 14:03:31 x4 kernel: pid 22710 (creat), uid 0 inumber 8194 on /mnt: out of inodes Jul 23 14:03:35 x4 kernel: pid 22705 (creat), uid 0 inumber 8193 on /mnt: out of inodes Jul 23 14:03:38 x4 syslogd: last message repeated 1 times Jul 23 14:03:45 x4 kernel: pid 22737 (mkdir), uid 0 inumber 8282 on /mnt: out of inodes Jul 23 14:03:47 x4 kernel: pid 22708 (creat), uid 0 inumber 12291 on /mnt: out of inodes Jul 23 14:03:52 x4 kernel: pid 22711 (creat), uid 0 inumber 8195 on /mnt: out of inodes Jul 23 14:03:54 x4 kernel: pid 22733 (mkdir), uid 0 inumber 8192 on /mnt: out of inodes Jul 23 14:05:22 x4 kernel: pid 32750 (mkdir), uid 0 inumber 11908 on /mnt: out of inodes Jul 23 14:05:24 x4 kernel: pid 32755 (creat), uid 0 inumber 8254 on /mnt: out of inodes Jul 23 14:05:29 x4 kernel: pid 32763 (creat), uid 0 inumber 8257 on /mnt: out of inodes Jul 23 14:07:06 x4 kernel: pid 43459 (creat), uid 0 inumber 12424 on /mnt: out of inodes Jul 23 14:07:11 x4 kernel: pid 43463 (creat), uid 0 inumber 12431 on /mnt: out of inodes Jul 23 14:07:13 x4 kernel: pid 43458 (creat), uid 0 inumber 12427 on /mnt: out of inodes Jul 23 14:07:41 x4 kernel: pid 43456 (creat), uid 0 inumber 12416 on /mnt: out of inodes Jul 23 14:07:42 x4 kernel: pid 43463 (creat), uid 0 inumber 12431 on /mnt: out of inodes Jul 23 14:07:44 x4 kernel: pid 43459 (creat), uid 0 inumber 12424 on /mnt: out of inodes Jul 23 14:08:01 x4 syslogd: last message repeated 2 times Jul 23 14:08:04 x4 kernel: pid 43456 (creat), uid 0 inumber 12416 on /mnt: out of inodes Jul 23 14:08:20 x4 kernel: pid 43459 (creat), uid 0 inumber 12424 on /mnt: out of inodes Jul 23 14:09:08 x4 kernel: pid 53911 (mkdir), uid 0 inumber 8192 on /mnt: out of inodes Jul 23 14:09:17 x4 kernel: pid 53919 (creat), uid 0 inumber 8254 on /mnt: out of inodes Jul 23 14:09:30 x4 kernel: pid 53904 (rw), uid 0 inumber 8250 on /mnt: filesystem full Jul 23 14:13:04 x4 kernel: pid 63929 (rw), uid 0 inumber 12548 on /mnt: filesystem full 20190723 14:13:07 all (187/657): graid1_6.sh Superblock check-hash failed: recorded check-hash 0xfcf122dc != computed check-hash 0x7d63ef36 Superblock check-hash failed: recorded check-hash 0xfcf122dc != computed check-hash 0x7d63ef36 Superblock check-hash failed: recorded check-hash 0xfcf122dc != computed check-hash 0x7d63ef36 Superblock check-hash failed: recorded check-hash 0xfcf122dc != computed check-hash 0x7d63ef36 Superblock check-hash failed: recorded check-hash 0xfcf122dc != computed check-hash 0x7d63ef36 Superblock check-hash failed: recorded check-hash 0xfcf122dc != computed check-hash 0x7d63ef36 Superblock check-hash failed: recorded check-hash 0xfcf122dc != computed check-hash 0x7d63ef36 Superblock check-hash failed: recorded check-hash 0xfcf122dc != computed check-hash 0x7d63ef36 Superblock check-hash failed: recorded check-hash 0xfcf122dc != computed check-hash 0x7d63ef36 Superblock check-hash failed: recorded check-hash 0xfcf122dc != computed check-hash 0x7d63ef36 20190723 14:18:28 all (188/657): gnop.sh GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. GEOM_NOP: Device md10.nop created. Jul 23 14:35:55 x4 kernel: pid 97264 (swap), jid 0, uid 1004, was killed: out of swap space Jul 23 14:36:45 x4 kernel: pid 97279 (swap), jid 0, uid 1004, was killed: out of swap space GEOM_NOP: Device md10.nop removed. 20190723 14:37:03 all (189/657): tar.sh freework leak: 25675/38516. 20190723 14:46:56 all (190/657): tmpfs6.sh 20190723 14:47:12 all (191/657): ext2fs2.sh 20190723 14:57:25 all (192/657): buildworld2.sh 20190723 14:57:31 all (193/657): bench.sh mount leak: 1/2452 20190723 14:59:32 all (194/657): quota4.sh mount leak: 1/2453 20190723 15:00:48 all (195/657): laundry.sh mount leak: 1/2454 20190723 15:05:54 all (196/657): rename10.sh mount leak: 1/2455 20190723 15:07:12 all (197/657): suj14.sh Jul 23 15:08:44 x4 kernel: pid 16820 (cp), uid 0 inumber 47345 on /mnt: filesystem full Jul 23 15:08:45 x4 kernel: pid 16936 (cp), uid 0 inumber 47366 on /mnt: filesystem full Jul 23 15:08:46 x4 kernel: pid 17584 (cp), uid 0 inumber 47366 on /mnt: filesystem full Jul 23 15:08:47 x4 kernel: pid 17968 (cp), uid 0 inumber 47366 on /mnt: filesystem full Jul 23 15:08:48 x4 kernel: pid 17986 (cp), uid 0 inumber 47366 on /mnt: filesystem full Jul 23 15:08:49 x4 kernel: pid 18004 (cp), uid 0 inumber 47366 on /mnt: filesystem full mount leak: 1/2456 20190723 15:08:54 all (198/657): nullfs20.sh mount leak: 1/2457 20190723 15:21:42 all (199/657): nullfs13.sh mount leak: 1/2458 20190723 15:21:48 all (200/657): md9.sh mount leak: 1/2459 20190723 15:21:59 all (201/657): procfs2.sh 20190723 15:22:09 all (202/657): sigreturn.sh 20190723 15:22:21 all (203/657): mkfifo7.sh mount leak: 17/2476 20190723 15:22:28 all (204/657): fsck5.sh 20190723 15:23:05 all (205/657): gnop2.sh GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. GEOM_NOP: Device md10.nop created. GEOM_NOP: Device md10.nop removed. 20190723 15:28:52 all (206/657): f_offset.sh 20190723 15:30:21 all (207/657): pause.sh 20190723 15:35:38 all (208/657): maxproc.sh maxproc limit exceeded by uid 65534 (pid 72796); see tuning(7) and login.conf(5) 20190723 15:35:48 all (209/657): procstat.sh 20190723 15:38:00 all (210/657): pagefault.sh 20190723 15:38:04 all (211/657): signal0.sh 20190723 15:44:26 all (212/657): sigstop.sh 20190723 15:44:33 all (213/657): socketpair2.sh 20190723 15:44:39 all (214/657): numa.sh 20190723 15:44:43 all (215/657): core4.sh Jul 23 15:44:48 x4 kernel: Failed to write core file for process core4 (error 5) 20190723 15:45:11 all (216/657): symlink3.sh Jul 23 15:45:09 x4 syslogd: last message repeated 59 times 20190723 16:05:38 all (217/657): kevent4.sh newblk leak: 79592/145154. 20190723 16:08:56 all (218/657): segregs.sh 20190723 16:11:08 all (219/657): snap3.sh mount leak: 10/2486 20190723 16:31:15 all (220/657): pmc3.sh hwpmc: SOFT/16/64/0x67 TSC/1/64/0x20 K8/16/48/0x1ff mount leak: 2/2488 20190723 16:37:25 all (221/657): select.sh 20190723 16:47:52 all (222/657): umount.sh mount leak: 1/2489 20190723 16:52:16 all (223/657): setsockopt2.sh mount leak: 1/2490 20190723 16:54:37 all (224/657): gnop3.sh 20190723 16:54:42 all (225/657): nullfs9.sh mount leak: 1/2491 20190723 16:54:48 all (226/657): mmap29.sh mount leak: 1/2492 20190723 16:54:58 all (227/657): segnp.sh 20190723 16:55:05 all (228/657): fdatasync.sh 20190723 16:58:15 all (229/657): ptrace5.sh 20190723 17:03:21 all (230/657): mmap14.sh 20190723 17:05:29 all (231/657): core5.sh mount leak: 1/2493 20190723 17:15:39 all (232/657): sigaltstack.sh 20190723 17:15:44 all (233/657): lockf4.sh mount leak: 1/2494 20190723 17:35:21 all (234/657): ptrace8.sh 20190723 17:35:27 all (235/657): truncate5.sh mount leak: 1/2495 20190723 17:35:35 all (236/657): swappedout.sh warning: total configured swap (4194304 pages) exceeds maximum recommended amount (2009920 pages). warning: increase kern.maxswzone or reduce amount of swap. 20190723 17:37:45 all (237/657): newfs3.sh mount leak: 80/2575 20190723 18:05:00 all (238/657): ptrace2.sh 20190723 18:05:16 all (239/657): zfs6.sh ZFS NOTICE: Prefetch is disabled by default on i386 -- to enable, add "vfs.zfs.prefetch_disable=0" to /boot/loader.conf. ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Warning: memory type solaris leaked memory on destroy (4 allocations, 576 bytes leaked). 20190723 18:11:41 all (240/657): r335171.sh 20190723 18:16:54 all (241/657): mountro5.sh mount leak: 1/2576 20190723 18:19:09 all (242/657): audit.sh 20190723 18:19:15 all (243/657): snap5.sh mount leak: 1/2577 20190723 18:49:38 all (244/657): mlockall3.sh 20190723 18:52:50 all (245/657): mac.sh 20190723 18:52:54 all (246/657): msdos7.sh 20190723 18:53:00 all (247/657): scp.sh 20190723 18:53:08 all (248/657): procfs5.sh 20190723 18:53:14 all (249/657): dev3.sh 20190723 19:16:20 all (250/657): lockf3.sh mount leak: 1/2578 GEOM leak: 6/282 20190723 19:39:13 all (251/657): sendfile11.sh GEOM_ELI: Device md10.eli created. GEOM_ELI: Encryption: AES-XTS 128 GEOM_ELI: Crypto: software GEOM_ELI: md10 has been killed. GEOM_ELI: Device md10.eli destroyed. mount leak: 1/2579 20190723 19:45:08 all (252/657): holdcnt02.sh 20190723 19:45:12 all (253/657): snap12.sh mount leak: 1/2580 20190723 19:45:18 all (254/657): sendfile15.sh 20190723 19:50:32 all (255/657): kevent2.sh 20190723 19:50:47 all (256/657): execi386.sh 20190723 19:50:50 all (257/657): mlockall7.sh 20190723 19:53:59 all (258/657): linger2.sh Jul 23 19:54:05 x4 kernel: pid 72187 (linger2), uid 1004 inumber 890 on /mnt: filesystem full mount leak: 1/2581 20190723 19:54:11 all (259/657): unload.sh 20190723 19:54:17 all (260/657): cmp.sh mount leak: 165/2746 20190723 20:00:39 all (261/657): nullfs16.sh mount leak: 1/2747 20190723 20:00:51 all (262/657): nullfs25.sh mount leak: 1/2748 20190723 20:05:58 all (263/657): posix_fadvise2.sh 20190723 20:06:10 all (264/657): fs.sh Jul 23 20:08:17 x4 kernel: pid 23670 (rw), uid 1004 inumber 772 on /mnt: filesystem full Jul 23 20:08:26 x4 kernel: pid 23674 (creat), uid 1004 inumber 770 on /mnt: out of inodes Jul 23 20:08:29 x4 kernel: pid 23671 (creat), uid 1004 inumber 4 on /mnt: out of inodes Jul 23 20:08:30 x4 kernel: pid 23672 (creat), uid 1004 inumber 1536 on /mnt: out of inodes Jul 23 20:08:31 x4 kernel: pid 23677 (creat), uid 1004 inumber 769 on /mnt: out of inodes Jul 23 20:08:33 x4 kernel: pid 23674 (creat), uid 1004 inumber 770 on /mnt: out of inodes Jul 23 20:08:38 x4 kernel: pid 23677 (creat), uid 1004 inumber 769 on /mnt: out of inodes Jul 23 20:08:39 x4 kernel: pid 23675 (creat), uid 1004 inumber 2307 on /mnt: out of inodes Jul 23 20:08:44 x4 kernel: pid 23677 (creat), uid 1004 inumber 769 on /mnt: out of inodes Jul 23 20:08:50 x4 kernel: pid 23674 (creat), uid 1004 inumber 770 on /mnt: out of inodes Jul 23 20:08:51 x4 kernel: pid 23675 (creat), uid 1004 inumber 2307 on /mnt: out of inodes Jul 23 20:08:53 x4 kernel: pid 23671 (creat), uid 1004 inumber 4 on /mnt: out of inodes Jul 23 20:08:56 x4 syslogd: last message repeated 1 times Jul 23 20:08:57 x4 kernel: pid 23672 (creat), uid 1004 inumber 1536 on /mnt: out of inodes Jul 23 20:08:59 x4 kernel: pid 23674 (creat), uid 1004 inumber 770 on /mnt: out of inodes Jul 23 20:09:03 x4 kernel: pid 23676 (creat), uid 1004 inumber 768 on /mnt: out of inodes Jul 23 20:09:05 x4 kernel: pid 23671 (creat), uid 1004 inumber 4 on /mnt: out of inodes Jul 23 20:09:07 x4 syslogd: last message repeated 1 times Jul 23 20:09:10 x4 kernel: pid 23673 (creat), uid 1004 inumber 2308 on /mnt: out of inodes Jul 23 20:09:11 x4 kernel: pid 23671 (creat), uid 1004 inumber 4 on /mnt: out of inodes Jul 23 20:09:13 x4 syslogd: last message repeated 1 times Jul 23 20:12:47 x4 kernel: pid 39634 (rw), uid 1004 inumber 652 on /mnt: filesystem full Jul 23 20:14:40 x4 kernel: pid 46515 (creat), uid 1004 inumber 1302 on /mnt: out of inodes Jul 23 20:14:42 x4 kernel: pid 46527 (creat), uid 1004 inumber 1921 on /mnt: out of inodes Jul 23 20:14:47 x4 kernel: pid 46514 (creat), uid 1004 inumber 1301 on /mnt: out of inodes Jul 23 20:14:57 x4 kernel: pid 46516 (rw), uid 1004 inumber 1090 on /mnt: filesystem full Jul 23 20:15:23 x4 kernel: pid 49930 (rw), uid 1004 inumber 753 on /mnt: filesystem full Jul 23 20:15:34 x4 kernel: pid 49942 (rw), uid 1004 inumber 63 on /mnt: filesystem full Jul 23 20:15:36 x4 kernel: pid 49937 (creat), uid 1004 inumber 23 on /mnt: out of inodes Jul 23 20:15:37 x4 kernel: pid 49933 (creat), uid 1004 inumber 16 on /mnt: out of inodes Jul 23 20:15:38 x4 kernel: pid 49935 (creat), uid 1004 inumber 25 on /mnt: out of inodes Jul 23 20:15:41 x4 syslogd: last message repeated 2 times Jul 23 20:15:42 x4 kernel: pid 49939 (creat), uid 1004 inumber 24 on /mnt: out of inodes Jul 23 20:15:43 x4 kernel: pid 49941 (creat), uid 1004 inumber 26 on /mnt: out of inodes Jul 23 20:15:45 x4 kernel: pid 49933 (creat), uid 1004 inumber 16 on /mnt: out of inodes Jul 23 20:15:46 x4 kernel: pid 49939 (creat), uid 1004 inumber 24 on /mnt: out of inodes Jul 23 20:15:47 x4 kernel: pid 49933 (creat), uid 1004 inumber 16 on /mnt: out of inodes Jul 23 20:15:48 x4 kernel: pid 49931 (creat), uid 1004 inumber 39 on /mnt: out of inodes Jul 23 20:15:49 x4 kernel: pid 49933 (creat), uid 1004 inumber 16 on /mnt: out of inodes Jul 23 20:15:50 x4 kernel: pid 49939 (creat), uid 1004 inumber 24 on /mnt: out of inodes Jul 23 20:15:51 x4 kernel: pid 49941 (creat), uid 1004 inumber 26 on /mnt: out of inodes Jul 23 20:15:52 x4 kernel: pid 49931 (creat), uid 1004 inumber 39 on /mnt: out of inodes Jul 23 20:15:53 x4 kernel: pid 49933 (creat), uid 1004 inumber 16 on /mnt: out of inodes Jul 23 20:15:55 x4 kernel: pid 49935 (creat), uid 1004 inumber 25 on /mnt: out of inodes Jul 23 20:15:56 x4 kernel: pid 49931 (creat), uid 1004 inumber 39 on /mnt: out of inodes Jul 23 20:15:59 x4 syslogd: last message repeated 2 times Jul 23 20:16:00 x4 kernel: pid 49939 (creat), uid 1004 inumber 24 on /mnt: out of inodes Jul 23 20:16:01 x4 kernel: pid 49933 (creat), uid 1004 inumber 16 on /mnt: out of inodes Jul 23 20:16:03 x4 syslogd: last message repeated 1 times Jul 23 20:16:04 x4 kernel: pid 49941 (creat), uid 1004 inumber 26 on /mnt: out of inodes Jul 23 20:16:07 x4 syslogd: last message repeated 2 times Jul 23 20:16:08 x4 kernel: pid 49933 (creat), uid 1004 inumber 16 on /mnt: out of inodes Jul 23 20:16:09 x4 kernel: pid 49937 (creat), uid 1004 inumber 23 on /mnt: out of inodes Jul 23 20:16:10 x4 syslogd: last message repeated 1 times Jul 23 20:16:12 x4 kernel: pid 49935 (creat), uid 1004 inumber 25 on /mnt: out of inodes Jul 23 20:16:13 x4 syslogd: last message repeated 1 times Jul 23 20:16:14 x4 kernel: pid 49939 (creat), uid 1004 inumber 24 on /mnt: out of inodes Jul 23 20:16:15 x4 kernel: pid 49933 (creat), uid 1004 inumber 16 on /mnt: out of inodes Jul 23 20:16:24 x4 kernel: pid 55611 (rw), uid 1004 inumber 1573 on /mnt: filesystem full Jul 23 20:16:26 x4 kernel: pid 55632 (rw), uid 1004 inumber 1908 on /mnt: filesystem full Jul 23 20:16:27 x4 kernel: pid 55604 (mkdir), uid 1004 inumber 1139 on /mnt: out of inodes Jul 23 20:16:29 x4 kernel: pid 55617 (rw), uid 1004 inumber 1317 on /mnt: filesystem full Jul 23 20:16:30 x4 kernel: pid 55631 (rw), uid 1004 inumber 1342 on /mnt: filesystem full Jul 23 20:16:36 x4 kernel: pid 55602 (creat), uid 1004 inumber 1920 on /mnt: out of inodes Jul 23 20:16:38 x4 kernel: pid 55619 (creat), uid 1004 inumber 667 on /mnt: out of inodes Jul 23 20:16:39 x4 kernel: pid 55610 (mkdir), uid 1004 inumber 758 on /mnt: out of inodes Jul 23 20:16:40 x4 kernel: pid 55622 (creat), uid 1004 inumber 652 on /mnt: out of inodes Jul 23 20:16:42 x4 kernel: pid 55624 (creat), uid 1004 inumber 4 on /mnt: out of inodes Jul 23 20:16:44 x4 kernel: pid 55613 (creat), uid 1004 inumber 1923 on /mnt: out of inodes Jul 23 20:16:45 x4 kernel: pid 55615 (creat), uid 1004 inumber 1921 on /mnt: out of inodes Jul 23 20:16:47 x4 kernel: pid 55619 (creat), uid 1004 inumber 667 on /mnt: out of inodes Jul 23 20:16:50 x4 kernel: pid 55615 (creat), uid 1004 inumber 1921 on /mnt: out of inodes Jul 23 20:16:51 x4 kernel: pid 55624 (creat), uid 1004 inumber 4 on /mnt: out of inodes Jul 23 20:16:53 x4 kernel: pid 55619 (creat), uid 1004 inumber 667 on /mnt: out of inodes Jul 23 20:16:54 x4 kernel: pid 55624 (creat), uid 1004 inumber 4 on /mnt: out of inodes Jul 23 20:16:56 x4 kernel: pid 55606 (creat), uid 1004 inumber 1922 on /mnt: out of inodes Jul 23 20:16:57 x4 kernel: pid 55622 (creat), uid 1004 inumber 652 on /mnt: out of inodes Jul 23 20:16:59 x4 kernel: pid 55615 (creat), uid 1004 inumber 1921 on /mnt: out of inodes Jul 23 20:17:00 x4 syslogd: last message repeated 1 times Jul 23 20:17:01 x4 kernel: pid 55602 (creat), uid 1004 inumber 1920 on /mnt: out of inodes Jul 23 20:17:02 x4 kernel: pid 55606 (creat), uid 1004 inumber 1922 on /mnt: out of inodes Jul 23 20:17:03 x4 syslogd: last message repeated 1 times Jul 23 20:17:05 x4 kernel: pid 55602 (creat), uid 1004 inumber 1920 on /mnt: out of inodes Jul 23 20:17:06 x4 kernel: pid 55619 (creat), uid 1004 inumber 667 on /mnt: out of inodes Jul 23 20:17:07 x4 syslogd: last message repeated 1 times Jul 23 20:17:08 x4 kernel: pid 55609 (creat), uid 1004 inumber 668 on /mnt: out of inodes Jul 23 20:17:10 x4 kernel: pid 55624 (creat), uid 1004 inumber 4 on /mnt: out of inodes Jul 23 20:17:12 x4 kernel: pid 55606 (creat), uid 1004 inumber 1922 on /mnt: out of inodes Jul 23 20:17:13 x4 kernel: pid 55624 (creat), uid 1004 inumber 4 on /mnt: out of inodes Jul 23 20:17:14 x4 kernel: pid 55606 (creat), uid 1004 inumber 1922 on /mnt: out of inodes Jul 23 20:17:15 x4 syslogd: last message repeated 1 times Jul 23 20:18:25 x4 kernel: pid 63613 (mkdir), uid 1004 inumber 1946 on /mnt: out of inodes Jul 23 20:18:30 x4 kernel: pid 63602 (creat), uid 1004 inumber 1298 on /mnt: out of inodes Jul 23 20:18:31 x4 kernel: pid 63589 (creat), uid 1004 inumber 640 on /mnt: out of inodes Jul 23 20:18:32 x4 kernel: pid 63602 (creat), uid 1004 inumber 1298 on /mnt: out of inodes Jul 23 20:18:34 x4 kernel: pid 63593 (creat), uid 1004 inumber 642 on /mnt: out of inodes Jul 23 20:18:35 x4 kernel: pid 63611 (mkdir), uid 1004 inumber 561 on /mnt: out of inodes Jul 23 20:18:37 x4 kernel: pid 63591 (creat), uid 1004 inumber 18 on /mnt: out of inodes Jul 23 20:18:39 x4 kernel: pid 63601 (mkdir), uid 1004 inumber 304 on /mnt: out of inodes Jul 23 20:18:40 x4 kernel: pid 63600 (creat), uid 1004 inumber 19 on /mnt: out of inodes Jul 23 20:18:41 x4 kernel: pid 63605 (mkdir), uid 1004 inumber 75 on /mnt: out of inodes Jul 23 20:18:42 x4 kernel: pid 63588 (creat), uid 1004 inumber 1920 on /mnt: out of inodes Jul 23 20:18:43 x4 kernel: pid 63608 (creat), uid 1004 inumber 1923 on /mnt: out of inodes Jul 23 20:18:44 x4 kernel: pid 63598 (creat), uid 1004 inumber 641 on /mnt: out of inodes Jul 23 20:18:46 x4 kernel: pid 63594 (creat), uid 1004 inumber 1286 on /mnt: out of inodes Jul 23 20:18:47 x4 kernel: pid 63589 (creat), uid 1004 inumber 640 on /mnt: out of inodes Jul 23 20:18:49 x4 kernel: pid 63594 (creat), uid 1004 inumber 1286 on /mnt: out of inodes Jul 23 20:18:52 x4 kernel: pid 63614 (creat), uid 1004 inumber 1289 on /mnt: out of inodes Jul 23 20:18:55 x4 kernel: pid 63600 (creat), uid 1004 inumber 19 on /mnt: out of inodes Jul 23 20:18:57 x4 kernel: pid 63604 (creat), uid 1004 inumber 1921 on /mnt: out of inodes Jul 23 20:18:59 x4 kernel: pid 63598 (creat), uid 1004 inumber 641 on /mnt: out of inodes Jul 23 20:19:02 x4 syslogd: last message repeated 1 times Jul 23 20:19:04 x4 kernel: pid 63614 (creat), uid 1004 inumber 1289 on /mnt: out of inodes Jul 23 20:19:07 x4 kernel: pid 63591 (creat), uid 1004 inumber 18 on /mnt: out of inodes Jul 23 20:19:10 x4 syslogd: last message repeated 1 times Jul 23 20:19:11 x4 kernel: pid 63614 (creat), uid 1004 inumber 1289 on /mnt: out of inodes Jul 23 20:19:13 x4 kernel: pid 63598 (creat), uid 1004 inumber 641 on /mnt: out of inodes Jul 23 20:19:16 x4 kernel: pid 63610 (creat), uid 1004 inumber 1300 on /mnt: out of inodes Jul 23 20:19:25 x4 kernel: pid 68150 (rw), uid 1004 inumber 1283 on /mnt: filesystem full Jul 23 20:19:27 x4 kernel: pid 68152 (rw), uid 1004 inumber 1566 on /mnt: filesystem full Jul 23 20:19:28 x4 kernel: pid 68127 (mkdir), uid 1004 inumber 1537 on /mnt: out of inodes Jul 23 20:19:30 x4 kernel: pid 68125 (creat), uid 1004 inumber 1940 on /mnt: out of inodes Jul 23 20:19:31 x4 kernel: pid 68160 (creat), uid 1004 inumber 1955 on /mnt: out of inodes Jul 23 20:19:33 x4 kernel: pid 68157 (rw), uid 1004 inumber 1549 on /mnt: out of inodes Jul 23 20:19:34 x4 kernel: pid 68126 (creat), uid 1004 inumber 650 on /mnt: out of inodes Jul 23 20:19:35 x4 kernel: pid 68125 (creat), uid 1004 inumber 1940 on /mnt: out of inodes Jul 23 20:19:36 x4 kernel: pid 68160 (creat), uid 1004 inumber 1955 on /mnt: out of inodes Jul 23 20:19:37 x4 kernel: pid 68133 (creat), uid 1004 inumber 1942 on /mnt: out of inodes Jul 23 20:19:38 x4 kernel: pid 68139 (creat), uid 1004 inumber 1947 on /mnt: out of inodes Jul 23 20:19:39 x4 kernel: pid 68144 (creat), uid 1004 inumber 1557 on /mnt: out of inodes Jul 23 20:19:40 x4 kernel: pid 68125 (creat), uid 1004 inumber 1940 on /mnt: out of inodes Jul 23 20:19:41 x4 kernel: pid 68153 (creat), uid 1004 inumber 1950 on /mnt: out of inodes Jul 23 20:19:42 x4 kernel: pid 68151 (creat), uid 1004 inumber 1948 on /mnt: out of inodes Jul 23 20:19:44 x4 kernel: pid 68125 (creat), uid 1004 inumber 1940 on /mnt: out of inodes Jul 23 20:19:45 x4 kernel: pid 68160 (creat), uid 1004 inumber 1955 on /mnt: out of inodes Jul 23 20:19:46 x4 kernel: pid 68136 (creat), uid 1004 inumber 1558 on /mnt: out of inodes Jul 23 20:19:47 x4 syslogd: last message repeated 1 times Jul 23 20:19:48 x4 kernel: pid 68158 (creat), uid 1004 inumber 1949 on /mnt: out of inodes Jul 23 20:19:49 x4 kernel: pid 68153 (creat), uid 1004 inumber 1950 on /mnt: out of inodes Jul 23 20:19:51 x4 kernel: pid 68130 (creat), uid 1004 inumber 1937 on /mnt: out of inodes Jul 23 20:19:52 x4 kernel: pid 68153 (creat), uid 1004 inumber 1950 on /mnt: out of inodes Jul 23 20:19:53 x4 kernel: pid 68139 (creat), uid 1004 inumber 1947 on /mnt: out of inodes Jul 23 20:19:54 x4 kernel: pid 68158 (creat), uid 1004 inumber 1949 on /mnt: out of inodes Jul 23 20:19:55 x4 kernel: pid 68153 (creat), uid 1004 inumber 1950 on /mnt: out of inodes Jul 23 20:19:56 x4 kernel: pid 68147 (creat), uid 1004 inumber 1952 on /mnt: out of inodes Jul 23 20:19:58 x4 kernel: pid 68139 (creat), uid 1004 inumber 1947 on /mnt: out of inodes Jul 23 20:19:59 x4 kernel: pid 68133 (creat), uid 1004 inumber 1942 on /mnt: out of inodes Jul 23 20:20:00 x4 kernel: pid 68131 (rw), uid 1004 inumber 1945 on /mnt: out of inodes Jul 23 20:20:01 x4 kernel: pid 68126 (creat), uid 1004 inumber 650 on /mnt: out of inodes Jul 23 20:20:02 x4 kernel: pid 68133 (creat), uid 1004 inumber 1942 on /mnt: out of inodes Jul 23 20:20:03 x4 kernel: pid 68151 (creat), uid 1004 inumber 1948 on /mnt: out of inodes Jul 23 20:20:04 x4 kernel: pid 68125 (creat), uid 1004 inumber 1940 on /mnt: out of inodes Jul 23 20:20:06 x4 kernel: pid 68160 (creat), uid 1004 inumber 1955 on /mnt: out of inodes Jul 23 20:20:07 x4 kernel: pid 68151 (creat), uid 1004 inumber 1948 on /mnt: out of inodes Jul 23 20:20:08 x4 kernel: pid 68139 (creat), uid 1004 inumber 1947 on /mnt: out of inodes Jul 23 20:20:09 x4 kernel: pid 68149 (creat), uid 1004 inumber 1954 on /mnt: out of inodes Jul 23 20:20:10 x4 kernel: pid 68130 (creat), uid 1004 inumber 1937 on /mnt: out of inodes Jul 23 20:20:11 x4 kernel: pid 68161 (creat), uid 1004 inumber 1953 on /mnt: out of inodes Jul 23 20:20:12 x4 kernel: pid 68125 (creat), uid 1004 inumber 1940 on /mnt: out of inodes Jul 23 20:20:13 x4 syslogd: last message repeated 1 times Jul 23 20:20:14 x4 kernel: pid 68149 (creat), uid 1004 inumber 1954 on /mnt: out of inodes Jul 23 20:20:16 x4 kernel: pid 68151 (creat), uid 1004 inumber 1948 on /mnt: out of inodes Jul 23 20:20:17 x4 kernel: pid 68156 (creat), uid 1004 inumber 1957 on /mnt: out of inodes Jul 23 20:20:18 x4 kernel: pid 68130 (creat), uid 1004 inumber 1937 on /mnt: out of inodes Jul 23 20:20:19 x4 kernel: pid 68151 (creat), uid 1004 inumber 1948 on /mnt: out of inodes Jul 23 20:20:20 x4 kernel: pid 68144 (creat), uid 1004 inumber 1557 on /mnt: out of inodes Jul 23 20:20:29 x4 kernel: pid 71632 (creat), uid 1004 inumber 1285 on /mnt: out of inodes Jul 23 20:20:31 x4 kernel: pid 71655 (creat), uid 1004 inumber 1296 on /mnt: out of inodes Jul 23 20:20:32 x4 kernel: pid 71639 (mkdir), uid 1004 inumber 2525 on /mnt: out of inodes Jul 23 20:20:33 x4 kernel: pid 71623 (creat), uid 1004 inumber 1920 on /mnt: out of inodes Jul 23 20:20:34 x4 kernel: pid 71640 (mkdir), uid 1004 inumber 1455 on /mnt: out of inodes Jul 23 20:20:35 x4 kernel: pid 71642 (mkdir), uid 1004 inumber 613 on /mnt: out of inodes Jul 23 20:20:37 x4 kernel: pid 71623 (creat), uid 1004 inumber 1920 on /mnt: out of inodes Jul 23 20:20:38 x4 kernel: pid 71632 (creat), uid 1004 inumber 1285 on /mnt: out of inodes Jul 23 20:20:39 x4 kernel: pid 71618 (creat), uid 1004 inumber 1283 on /mnt: out of inodes Jul 23 20:20:40 x4 kernel: pid 71648 (creat), uid 1004 inumber 1290 on /mnt: out of inodes Jul 23 20:20:42 x4 kernel: pid 71645 (creat), uid 1004 inumber 1297 on /mnt: out of inodes Jul 23 20:20:43 x4 kernel: pid 71632 (creat), uid 1004 inumber 1285 on /mnt: out of inodes Jul 23 20:20:44 x4 kernel: pid 71650 (creat), uid 1004 inumber 1293 on /mnt: out of inodes Jul 23 20:20:46 x4 kernel: pid 71623 (creat), uid 1004 inumber 1920 on /mnt: out of inodes Jul 23 20:20:47 x4 kernel: pid 71645 (creat), uid 1004 inumber 1297 on /mnt: out of inodes Jul 23 20:20:48 x4 kernel: pid 71651 (creat), uid 1004 inumber 1288 on /mnt: out of inodes Jul 23 20:20:49 x4 kernel: pid 71650 (creat), uid 1004 inumber 1293 on /mnt: out of inodes Jul 23 20:20:50 x4 kernel: pid 71655 (creat), uid 1004 inumber 1296 on /mnt: out of inodes Jul 23 20:20:51 x4 kernel: pid 71632 (creat), uid 1004 inumber 1285 on /mnt: out of inodes Jul 23 20:20:52 x4 kernel: pid 71615 (creat), uid 1004 inumber 1281 on /mnt: out of inodes Jul 23 20:20:53 x4 kernel: pid 71641 (creat), uid 1004 inumber 1289 on /mnt: out of inodes Jul 23 20:20:54 x4 kernel: pid 71647 (creat), uid 1004 inumber 1287 on /mnt: out of inodes Jul 23 20:20:55 x4 kernel: pid 71650 (creat), uid 1004 inumber 1293 on /mnt: out of inodes Jul 23 20:20:56 x4 kernel: pid 71643 (creat), uid 1004 inumber 1294 on /mnt: out of inodes Jul 23 20:20:57 x4 kernel: pid 71629 (creat), uid 1004 inumber 1284 on /mnt: out of inodes Jul 23 20:20:59 x4 kernel: pid 71623 (creat), uid 1004 inumber 1920 on /mnt: out of inodes Jul 23 20:21:00 x4 kernel: pid 71632 (creat), uid 1004 inumber 1285 on /mnt: out of inodes Jul 23 20:21:01 x4 kernel: pid 71653 (creat), uid 1004 inumber 1298 on /mnt: out of inodes Jul 23 20:21:02 x4 kernel: pid 71654 (creat), uid 1004 inumber 1295 on /mnt: out of inodes Jul 23 20:21:03 x4 kernel: pid 71655 (creat), uid 1004 inumber 1296 on /mnt: out of inodes Jul 23 20:21:04 x4 kernel: pid 71623 (creat), uid 1004 inumber 1920 on /mnt: out of inodes Jul 23 20:21:05 x4 kernel: pid 71649 (creat), uid 1004 inumber 1291 on /mnt: out of inodes Jul 23 20:21:07 x4 kernel: pid 71621 (creat), uid 1004 inumber 1282 on /mnt: out of inodes Jul 23 20:21:08 x4 kernel: pid 71654 (creat), uid 1004 inumber 1295 on /mnt: out of inodes Jul 23 20:21:09 x4 syslogd: last message repeated 1 times Jul 23 20:21:10 x4 kernel: pid 71652 (creat), uid 1004 inumber 1292 on /mnt: out of inodes Jul 23 20:21:11 x4 kernel: pid 71654 (creat), uid 1004 inumber 1295 on /mnt: out of inodes Jul 23 20:21:12 x4 kernel: pid 71653 (creat), uid 1004 inumber 1298 on /mnt: out of inodes Jul 23 20:21:13 x4 kernel: pid 71629 (creat), uid 1004 inumber 1284 on /mnt: out of inodes Jul 23 20:21:14 x4 kernel: pid 71626 (creat), uid 1004 inumber 1286 on /mnt: out of inodes Jul 23 20:21:15 x4 kernel: pid 71643 (creat), uid 1004 inumber 1294 on /mnt: out of inodes Jul 23 20:21:16 x4 kernel: pid 71645 (creat), uid 1004 inumber 1297 on /mnt: out of inodes Jul 23 20:21:18 x4 kernel: pid 71629 (creat), uid 1004 inumber 1284 on /mnt: out of inodes Jul 23 20:21:19 x4 kernel: pid 71649 (creat), uid 1004 inumber 1291 on /mnt: out of inodes Jul 23 20:21:20 x4 kernel: pid 71621 (creat), uid 1004 inumber 1282 on /mnt: out of inodes Jul 23 20:21:21 x4 kernel: pid 71653 (creat), uid 1004 inumber 1298 on /mnt: out of inodes Jul 23 20:21:28 x4 kernel: pid 75069 (creat), uid 1004 inumber 1333 on /mnt: out of inodes Jul 23 20:21:33 x4 kernel: pid 75064 (rw), uid 1004 inumber 1330 on /mnt: out of inodes Jul 23 20:21:34 x4 kernel: pid 75078 (rw), uid 1004 inumber 1342 on /mnt: out of inodes Jul 23 20:21:36 x4 kernel: pid 75067 (mkdir), uid 1004 inumber 1692 on /mnt: out of inodes Jul 23 20:21:37 x4 kernel: pid 75066 (creat), uid 1004 inumber 1664 on /mnt: out of inodes Jul 23 20:21:38 x4 kernel: pid 75069 (creat), uid 1004 inumber 1333 on /mnt: out of inodes Jul 23 20:21:39 x4 kernel: pid 75066 (creat), uid 1004 inumber 1664 on /mnt: out of inodes Jul 23 20:21:40 x4 kernel: pid 75069 (creat), uid 1004 inumber 1333 on /mnt: out of inodes Jul 23 20:21:41 x4 kernel: pid 75066 (creat), uid 1004 inumber 1664 on /mnt: out of inodes Jul 23 20:21:42 x4 kernel: pid 75069 (creat), uid 1004 inumber 1333 on /mnt: out of inodes Jul 23 20:21:43 x4 syslogd: last message repeated 1 times Jul 23 20:21:44 x4 kernel: pid 75071 (creat), uid 1004 inumber 1326 on /mnt: out of inodes Jul 23 20:21:45 x4 kernel: pid 75069 (creat), uid 1004 inumber 1333 on /mnt: out of inodes Jul 23 20:21:46 x4 kernel: pid 75066 (creat), uid 1004 inumber 1664 on /mnt: out of inodes Jul 23 20:21:48 x4 kernel: pid 75071 (creat), uid 1004 inumber 1326 on /mnt: out of inodes Jul 23 20:21:50 x4 syslogd: last message repeated 2 times Jul 23 20:21:51 x4 kernel: pid 75069 (creat), uid 1004 inumber 1333 on /mnt: out of inodes Jul 23 20:21:52 x4 kernel: pid 75066 (creat), uid 1004 inumber 1664 on /mnt: out of inodes Jul 23 20:21:54 x4 syslogd: last message repeated 2 times Jul 23 20:21:55 x4 kernel: pid 75069 (creat), uid 1004 inumber 1333 on /mnt: out of inodes Jul 23 20:21:58 x4 syslogd: last message repeated 3 times Jul 23 20:21:59 x4 kernel: pid 75071 (creat), uid 1004 inumber 1326 on /mnt: out of inodes Jul 23 20:22:00 x4 kernel: pid 75069 (creat), uid 1004 inumber 1333 on /mnt: out of inodes Jul 23 20:22:01 x4 syslogd: last message repeated 1 times Jul 23 20:22:02 x4 kernel: pid 75066 (creat), uid 1004 inumber 1664 on /mnt: out of inodes Jul 23 20:22:03 x4 kernel: pid 75069 (creat), uid 1004 inumber 1333 on /mnt: out of inodes Jul 23 20:22:04 x4 syslogd: last message repeated 1 times Jul 23 20:22:05 x4 kernel: pid 75071 (creat), uid 1004 inumber 1326 on /mnt: out of inodes Jul 23 20:22:06 x4 kernel: pid 75069 (creat), uid 1004 inumber 1333 on /mnt: out of inodes Jul 23 20:22:07 x4 syslogd: last message repeated 1 times Jul 23 20:22:08 x4 kernel: pid 75071 (creat), uid 1004 inumber 1326 on /mnt: out of inodes Jul 23 20:22:09 x4 kernel: pid 75069 (creat), uid 1004 inumber 1333 on /mnt: out of inodes Jul 23 20:22:10 x4 kernel: pid 75066 (creat), uid 1004 inumber 1664 on /mnt: out of inodes Jul 23 20:22:11 x4 kernel: pid 75071 (creat), uid 1004 inumber 1326 on /mnt: out of inodes Jul 23 20:22:12 x4 kernel: pid 75066 (creat), uid 1004 inumber 1664 on /mnt: out of inodes Jul 23 20:22:13 x4 kernel: pid 75069 (creat), uid 1004 inumber 1333 on /mnt: out of inodes Jul 23 20:22:14 x4 kernel: pid 75071 (creat), uid 1004 inumber 1326 on /mnt: out of inodes Jul 23 20:22:15 x4 syslogd: last message repeated 1 times Jul 23 20:22:17 x4 kernel: pid 75069 (creat), uid 1004 inumber 1333 on /mnt: out of inodes Jul 23 20:22:18 x4 kernel: pid 75071 (creat), uid 1004 inumber 1326 on /mnt: out of inodes Jul 23 20:22:19 x4 kernel: pid 75069 (creat), uid 1004 inumber 1333 on /mnt: out of inodes Jul 23 20:22:20 x4 kernel: pid 75066 (creat), uid 1004 inumber 1664 on /mnt: out of inodes Jul 23 20:22:21 x4 kernel: pid 75071 (creat), uid 1004 inumber 1326 on /mnt: out of inodes mount leak: 8/2756 20190723 20:22:24 all (265/657): suj13.sh Jul 23 20:25:37 x4 kernel: pid 82304 (openat), uid 1004 inumber 12289 on /mnt: out of inodes Jul 23 20:25:43 x4 kernel: pid 82348 (creat), uid 1004 inumber 8205 on /mnt: out of inodes Jul 23 20:25:44 x4 syslogd: last message repeated 1 times Jul 23 20:25:45 x4 kernel: pid 82342 (creat), uid 1004 inumber 12320 on /mnt: out of inodes Jul 23 20:25:46 x4 kernel: pid 82340 (creat), uid 1004 inumber 12317 on /mnt: out of inodes Jul 23 20:25:47 x4 kernel: pid 82345 (creat), uid 1004 inumber 8207 on /mnt: out of inodes Jul 23 20:25:48 x4 kernel: pid 82339 (creat), uid 1004 inumber 12318 on /mnt: out of inodes Jul 23 20:25:49 x4 kernel: pid 82348 (creat), uid 1004 inumber 8205 on /mnt: out of inodes Jul 23 20:25:50 x4 kernel: pid 82339 (creat), uid 1004 inumber 12318 on /mnt: out of inodes Jul 23 20:25:51 x4 kernel: pid 82353 (creat), uid 1004 inumber 12323 on /mnt: out of inodes Jul 23 20:25:52 x4 kernel: pid 82352 (creat), uid 1004 inumber 12319 on /mnt: out of inodes Jul 23 20:25:53 x4 kernel: pid 82346 (creat), uid 1004 inumber 30 on /mnt: out of inodes Jul 23 20:25:54 x4 kernel: pid 82347 (creat), uid 1004 inumber 8224 on /mnt: out of inodes Jul 23 20:25:55 x4 kernel: pid 82348 (creat), uid 1004 inumber 8205 on /mnt: out of inodes Jul 23 20:25:56 x4 kernel: pid 82345 (creat), uid 1004 inumber 8207 on /mnt: out of inodes Jul 23 20:25:59 x4 kernel: pid 82353 (creat), uid 1004 inumber 12323 on /mnt: out of inodes Jul 23 20:26:02 x4 kernel: pid 82346 (creat), uid 1004 inumber 30 on /mnt: out of inodes Jul 23 20:26:09 x4 kernel: pid 82350 (creat), uid 1004 inumber 15 on /mnt: out of inodes Jul 23 20:26:11 x4 kernel: pid 82344 (creat), uid 1004 inumber 8196 on /mnt: out of inodes Jul 23 20:26:18 x4 kernel: pid 82342 (creat), uid 1004 inumber 12320 on /mnt: out of inodes Jul 23 20:26:19 x4 kernel: pid 82346 (creat), uid 1004 inumber 30 on /mnt: out of inodes Jul 23 20:26:22 x4 kernel: pid 82337 (creat), uid 1004 inumber 8223 on /mnt: out of inodes Jul 23 20:26:27 x4 kernel: pid 82347 (creat), uid 1004 inumber 8224 on /mnt: out of inodes Jul 23 20:26:29 x4 kernel: pid 82352 (creat), uid 1004 inumber 12319 on /mnt: out of inodes Jul 23 20:26:33 x4 syslogd: last message repeated 1 times Jul 23 20:26:57 x4 kernel: pid 83532 (rw), uid 1004 inumber 4385 on /mnt: filesystem full Jul 23 20:32:51 x4 kernel: pid 4787 (rw), uid 1004 inumber 12326 on /mnt: filesystem full Jul 23 20:33:03 x4 kernel: pid 4776 (creat), uid 1004 inumber 4285 on /mnt: out of inodes Jul 23 20:33:21 x4 kernel: pid 4782 (creat), uid 1004 inumber 8221 on /mnt: out of inodes Jul 23 20:33:25 x4 kernel: pid 4773 (creat), uid 1004 inumber 8223 on /mnt: out of inodes Jul 23 20:33:29 x4 kernel: pid 4776 (creat), uid 1004 inumber 4285 on /mnt: out of inodes Jul 23 20:33:34 x4 kernel: pid 4774 (creat), uid 1004 inumber 4300 on /mnt: out of inodes Jul 23 20:33:44 x4 kernel: pid 4786 (creat), uid 1004 inumber 8215 on /mnt: out of inodes Jul 23 20:33:47 x4 kernel: pid 4806 (creat), uid 1004 inumber 8220 on /mnt: out of inodes Jul 23 20:33:50 x4 kernel: pid 4777 (creat), uid 1004 inumber 8227 on /mnt: out of inodes Jul 23 20:33:54 x4 kernel: pid 4779 (creat), uid 1004 inumber 8213 on /mnt: out of inodes Jul 23 20:34:00 x4 kernel: pid 4857 (openat), uid 1004 inumber 4313 on /mnt: out of inodes Jul 23 20:34:03 x4 kernel: pid 4806 (creat), uid 1004 inumber 8220 on /mnt: out of inodes Jul 23 20:34:08 x4 kernel: pid 4782 (creat), uid 1004 inumber 8221 on /mnt: out of inodes Jul 23 20:34:11 x4 kernel: pid 4788 (creat), uid 1004 inumber 8209 on /mnt: out of inodes Jul 23 20:34:17 x4 kernel: pid 4770 (creat), uid 1004 inumber 8202 on /mnt: out of inodes Jul 23 20:34:22 x4 kernel: pid 4782 (creat), uid 1004 inumber 8221 on /mnt: out of inodes Jul 23 20:34:24 x4 kernel: pid 4776 (creat), uid 1004 inumber 4285 on /mnt: out of inodes Jul 23 20:34:26 x4 kernel: pid 4806 (creat), uid 1004 inumber 8220 on /mnt: out of inodes Jul 23 20:34:33 x4 kernel: pid 4777 (creat), uid 1004 inumber 8227 on /mnt: out of inodes Jul 23 20:34:34 x4 kernel: pid 4788 (creat), uid 1004 inumber 8209 on /mnt: out of inodes Jul 23 20:34:36 x4 kernel: pid 4774 (creat), uid 1004 inumber 4300 on /mnt: out of inodes Jul 23 20:34:38 x4 kernel: pid 4777 (creat), uid 1004 inumber 8227 on /mnt: out of inodes Jul 23 20:34:39 x4 kernel: pid 4769 (symlink), uid 1004 inumber 8219 on /mnt: out of inodes Jul 23 20:34:41 x4 kernel: pid 4776 (creat), uid 1004 inumber 4285 on /mnt: out of inodes Jul 23 20:35:35 x4 kernel: pid 6487 (openat), uid 1004 inumber 8213 on /mnt: out of inodes Jul 23 20:35:41 x4 kernel: pid 6483 (creat), uid 1004 inumber 12334 on /mnt: out of inodes Jul 23 20:35:43 x4 kernel: pid 6484 (creat), uid 1004 inumber 12326 on /mnt: out of inodes Jul 23 20:35:48 x4 kernel: pid 6519 (link), uid 1004 inumber 8229 on /mnt: out of inodes Jul 23 20:35:50 x4 kernel: pid 6483 (creat), uid 1004 inumber 12334 on /mnt: out of inodes Jul 23 20:35:55 x4 kernel: pid 6482 (creat), uid 1004 inumber 12336 on /mnt: out of inodes Jul 23 20:35:56 x4 kernel: pid 6535 (link), uid 1004 inumber 8231 on /mnt: out of inodes Jul 23 20:36:10 x4 kernel: pid 6483 (creat), uid 1004 inumber 12334 on /mnt: out of inodes Jul 23 20:36:11 x4 kernel: pid 6524 (link), uid 1004 inumber 8221 on /mnt: out of inodes Jul 23 20:36:15 x4 kernel: pid 6532 (link), uid 1004 inumber 8204 on /mnt: out of inodes Jul 23 20:36:17 x4 kernel: pid 6476 (creat), uid 1004 inumber 8203 on /mnt: out of inodes Jul 23 20:36:21 x4 kernel: pid 6484 (creat), uid 1004 inumber 12326 on /mnt: out of inodes Jul 23 20:36:22 x4 kernel: pid 6524 (link), uid 1004 inumber 8221 on /mnt: out of inodes Jul 23 20:36:23 x4 syslogd: last message repeated 1 times Jul 23 20:36:41 x4 kernel: pid 6521 (link), uid 1004 inumber 12338 on /mnt: out of inodes Jul 23 20:37:21 x4 kernel: pid 7390 (rw), uid 1004 inumber 8814 on /mnt: filesystem full Jul 23 20:39:19 x4 kernel: pid 8007 (rw), uid 1004 inumber 4988 on /mnt: filesystem full Jul 23 20:40:50 x4 kernel: pid 8012 (rw), uid 1004 inumber 4843 on /mnt: filesystem full mount leak: 1/2757 20190723 20:43:32 all (266/657): sparse.sh mount leak: 1/2758 20190723 20:43:48 all (267/657): zfs.sh ZFS NOTICE: Prefetch is disabled by default on i386 -- to enable, add "vfs.zfs.prefetch_disable=0" to /boot/loader.conf. ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Jul 23 20:54:58 x4 kernel: pid 29607 (swap), jid 0, uid 0, was killed: out of swap space swap_pager: cannot allocate bio Warning: memory type solaris leaked memory on destroy (4 allocations, 576 bytes leaked). 20190723 21:04:16 all (268/657): union.sh mount leak: 1/2759 20190723 21:14:59 all (269/657): namecache.sh 20190723 21:34:50 all (270/657): sendfile12.sh mount leak: 1/2760 20190723 21:40:15 all (271/657): vm_fault_dontneed.sh 20190723 21:40:24 all (272/657): md2.sh g_vfs_done():md10[WRITE(offset=120619008, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=120651776, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=125501440, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=125534208, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=125337600, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=77266944, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=139132928, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=82345984, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=82345984, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=82345984, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=83918848, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=83853312, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=83886080, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=83918848, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=83918848, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=83918848, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=83918848, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=77201408, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=94011392, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=110755840, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=77201408, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=94011392, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=77201408, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=94011392, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=77365248, length=12288)]error = 28 g_vfs_done():md10[WRITE(offset=112984064, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=113016832, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=113049600, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=113016832, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=113049600, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=143884288, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=113016832, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=113049600, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=143884288, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=113016832, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=113049600, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=143884288, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=103645184, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=103677952, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=130547712, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=90112000, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=107511808, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=107544576, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=107479040, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=107511808, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=107544576, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=107544576, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=94404608, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=94404608, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=94404608, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=94404608, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=94404608, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=94404608, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=107544576, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=107544576, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=107544576, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=162627584, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=174325760, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=88047616, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=88080384, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=88113152, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=157089792, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=162594816, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=162627584, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=174325760, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=225509376, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=81264640, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=88080384, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=88113152, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=157089792, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=162594816, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=162627584, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=174325760, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=225509376, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=81264640, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=88080384, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=88113152, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=162594816, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=162627584, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=174325760, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=225509376, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=81264640, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=88080384, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=88113152, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=157089792, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=174325760, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=225509376, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=81264640, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=88113152, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=174325760, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=225509376, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=81264640, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=88113152, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=174325760, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=225509376, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=136937472, length=16384)]error = 28 g_vfs_done():md10[WRITE(offset=69402624, length=8192)]error = 28 g_vfs_done():md10[WRITE(offset=136904704, length=16384)]error = 28 Jul 23 21:45:05 x4 kernel: pid 61466 (swap), jid 0, uid 0, was killed: out of swap space g_vfs_done():md10[WRITE(offset=86212608, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=94732288, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=149487616, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=149520384, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=219742208, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=219774976, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=240779264, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=78708736, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=86147072, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=86212608, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=94732288, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=219742208, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=219774976, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=240779264, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=78708736, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=86147072, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=86212608, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=94732288, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=219774976, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=240779264, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=94732288, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=219774976, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=94732288, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=94732288, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=94732288, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=100106240, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=100139008, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=157417472, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=157450240, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=157483008, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=224854016, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=89489408, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=100368384, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=100401152, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=100433920, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=157646848, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=157679616, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=225050624, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=225083392, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=242286592, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=89489408, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=100368384, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=100401152, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=106725376, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=136937472, length=20480)]error = 28 g_vfs_done():md10[WRITE(offset=149749760, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=149782528, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=156303360, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=156336128, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=163971072, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=238452736, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=238485504, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=74842112, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=95158272, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=95191040, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=118325248, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=213745664, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=145752064, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=213745664, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=145752064, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=213745664, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=145752064, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=213745664, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=213745664, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=213778432, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=147128320, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=147161088, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=233537536, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=233570304, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=73629696, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=88014848, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=147095552, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=81002496, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=129433600, length=32768)]error = 28 g_vfs_done():md10[WRITE(offset=129433600, length=32768)]error = 28 mount leak: 2/2762 20190723 22:01:40 all (273/657): marcus5.sh mount leak: 1/2763 20190723 22:18:45 all (274/657): nullfs15.sh mount leak: 1/2764 20190723 22:18:51 all (275/657): mlockall6.sh mount leak: 1/2765 20190723 22:29:26 all (276/657): graid1_10.sh mount leak: 1/2766 20190723 22:31:39 all (277/657): fdatasync2.sh 20190723 22:34:57 all (278/657): msync2.sh 20190723 22:41:43 all (279/657): rename4.sh 20190723 22:41:49 all (280/657): suj3.sh GEOM_ELI: Device md10.eli created. GEOM_ELI: Encryption: AES-XTS 128 GEOM_ELI: Crypto: software GEOM_ELI: md10 has been killed. GEOM_ELI: Device md10.eli destroyed. mount leak: 1/2767 20190723 22:48:14 all (281/657): growfs.sh g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_dev_taste: make_dev_p() failed (gp->name=gptid/32f36b0f-ad8b-11e9-bfce-00218515337d, error=17) g_dev_taste: make_dev_p() failed (gp->name=gptid/32f36b0f-ad8b-11e9-bfce-00218515337d, error=17) g_dev_taste: make_dev_p() failed (gp->name=ufsid/5d377291cfdb3c9c, error=17) g_dev_taste: make_dev_p() failed (gp->name=gptid/32f36b0f-ad8b-11e9-bfce-00218515337d, error=17) g_dev_taste: make_dev_p() failed (gp->name=ufsid/5d377291cfdb3c9c, error=17) g_dev_taste: make_dev_p() failed (gp->name=gptid/32f36b0f-ad8b-11e9-bfce-00218515337d, error=17) g_dev_taste: make_dev_p() failed (gp->name=ufsid/5d377291cfdb3c9c, error=17) mount leak: 1/2768 20190723 22:48:35 all (282/657): mmap5.sh 20190723 22:50:02 all (283/657): datamove2.sh 20190723 23:00:41 all (284/657): pmc2.sh hwpmc: SOFT/16/64/0x67 TSC/1/64/0x20 K8/16/48/0x1ff 20190723 23:00:48 all (285/657): shm_super.sh 20190723 23:00:58 all (286/657): md3.sh mount leak: 1/2769 20190723 23:11:46 all (287/657): compare.sh 20190723 23:12:22 all (288/657): snap2-1.sh mount leak: 1/2770 20190723 23:12:58 all (289/657): tmpfs8.sh 20190723 23:13:57 all (290/657): nullfs24.sh 20190723 23:19:47 all (291/657): contigmalloc3.sh 20190723 23:20:16 all (292/657): nfs15.sh 20190723 23:20:22 all (293/657): nfs.sh 20190723 23:20:28 all (294/657): truncate2.sh 20190723 23:25:02 all (295/657): mount.sh mount leak: 1/2771 20190723 23:25:08 all (296/657): syscall5.sh 23:25:12 syscall 568 (funlinkat) 23:27:33 syscall 567 (fhreadlink) 23:29:58 syscall 566 (fhlinkat) 23:32:18 syscall 565 (fhlink) 23:34:27 syscall 564 (getfhat) 23:36:47 syscall 563 (getrandom) 23:39:01 syscall 562 (cpuset_setdomain) 23:41:13 syscall 561 (cpuset_getdomain) 23:43:22 syscall 560 (kevent) 23:45:36 syscall 559 (mknodat) 23:47:49 syscall 558 (fhstatfs) 23:49:52 syscall 557 (getfsstat) 23:51:58 syscall 556 (fstatfs) 23:54:11 syscall 555 (statfs) mount leak: 13/2784 20190723 23:56:20 all (297/657): graid3.sh g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set GEOM_RAID3: Device raid3/data launched (3/3). GEOM_RAID3: Device data: provider raid3/data destroyed. GEOM_RAID3: Device data destroyed. mount leak: 1/2785 20190724 00:18:30 all (298/657): suj23.sh mount leak: 1/2786 20190724 00:20:06 all (299/657): mmap15.sh 20190724 00:26:56 all (300/657): wire_no_page.sh 20190724 00:32:01 all (301/657): mountro4.sh mount leak: 3/2789 20190724 00:32:10 all (302/657): geomleak.sh 20190724 00:32:17 all (303/657): credleak.sh mount leak: 162/2951 20190724 00:33:17 all (304/657): fsck2.sh 20190724 00:33:21 all (305/657): mmap8.sh 20190724 00:33:28 all (306/657): dup.sh 20190724 00:33:45 all (307/657): sndstat.sh 20190724 00:33:51 all (308/657): kevent6.sh mount leak: 2/2953 20190724 00:34:55 all (309/657): tmpfs2.sh mount leak: 1/2954 20190724 00:37:51 all (310/657): suj32.sh 20190724 00:37:58 all (311/657): fifo4.sh mount leak: 1/2955 20190724 00:40:08 all (312/657): pthread8.sh 20190724 00:43:15 all (313/657): sendfile14.sh 20190724 00:43:21 all (314/657): isofs.sh mount leak: 1/2956 20190724 00:43:37 all (315/657): buildworld.sh 20190724 00:43:43 all (316/657): msdos2.sh mount leak: 1/2957 20190724 00:43:48 all (317/657): sendfile13.sh 20190724 00:43:54 all (318/657): beneath.sh 20190724 00:47:19 all (319/657): pfl2.sh swap_pager: cannot allocate bio swap_pager: cannot allocate bio mount leak: 1/2958 20190724 00:57:46 all (320/657): su.sh Approaching the limit on PV entries, consider increasing either the vm.pmap.shpgperproc or the vm.pmap.pv_entries tunable. Approaching the limit on PV entries, consider increasing either the vm.pmap.shpgperproc or the vm.pmap.pv_entries tunable. 20190724 01:03:35 all (321/657): swap3.sh Jul 24 01:05:09 x4 kernel: pid 52443 (swap2), jid 0, uid 0, was killed: out of swap space Jul 24 01:05:29 x4 kernel: pid 52442 (swap2), jid 0, uid 0, was killed: out of swap space Jul 24 01:05:48 x4 kernel: pid 52441 (swap2), jid 0, uid 0, was killed: out of swap space Jul 24 01:06:15 x4 kernel: pid 52440 (swap2), jid 0, uid 0, was killed: out of swap space Jul 24 01:06:40 x4 kernel: pid 52439 (swap2), jid 0, uid 0, was killed: out of swap space Jul 24 01:07:18 x4 kernel: pid 52438 (swap2), jid 0, uid 0, was killed: out of swap space Jul 24 01:07:42 x4 kernel: pid 52437 (swap2), jid 0, uid 0, was killed: out of swap space Jul 24 01:08:00 x4 kernel: pid 52436 (swap2), jid 0, uid 0, was killed: out of swap space Jul 24 01:08:14 x4 kernel: pid 52435 (swap2), jid 0, uid 0, was killed: out of swap space Jul 24 01:08:36 x4 kernel: pid 52434 (swap2), jid 0, uid 0, was killed: out of swap space warning: total configured swap (4194304 pages) exceeds maximum recommended amount (2009920 pages). warning: increase kern.maxswzone or reduce amount of swap. 20190724 01:08:56 all (322/657): beneath3.sh mount leak: 1/2959 20190724 01:09:08 all (323/657): kevent14.sh 20190724 01:11:12 all (324/657): suj4.sh mount leak: 1/2960 20190724 01:41:24 all (325/657): openlock.sh 20190724 01:41:31 all (326/657): mlockall5.sh mount leak: 1/2961 20190724 01:41:38 all (327/657): snap5-1.sh mount leak: 7/2968 20190724 01:53:23 all (328/657): oovm2.sh 20190724 02:04:26 all (329/657): vfork.sh 20190724 02:04:32 all (330/657): geli.sh GEOM_ELI: Device md10.eli created. GEOM_ELI: Encryption: AES-XTS 128 GEOM_ELI: Crypto: software GEOM_ELI: md10 has been killed. GEOM_ELI: Device md10.eli destroyed. mount leak: 1/2969 20190724 02:09:51 all (331/657): mmap31.sh 20190724 02:09:57 all (332/657): nfs7.sh 20190724 02:10:03 all (333/657): swap2.sh Jul 24 02:14:17 x4 kernel: pid 4122 (swap2), jid 0, uid 0, was killed: out of swap space mount leak: 1/2970 20190724 02:15:54 all (334/657): rename5.sh mount leak: 1/2971 20190724 02:21:07 all (335/657): suj10.sh mount leak: 1/2972 20190724 02:22:08 all (336/657): link2.sh mount leak: 2/2974 20190724 02:24:42 all (337/657): mac_chkexec.sh 20190724 02:24:46 all (338/657): audit2.sh mount leak: 1/2975 20190724 02:31:25 all (339/657): shm2.sh 20190724 02:33:41 all (340/657): pthread9.sh 20190724 02:33:47 all (341/657): pfl4.sh 20190724 02:33:53 all (342/657): symlink2.sh mount leak: 2/2977 20190724 02:34:05 all (343/657): trim3.sh Jul 24 02:35:28 x4 kernel: pid 61328 (swap), jid 0, uid 1004, was killed: out of swap space Jul 24 02:35:29 x4 kernel: pid 61311 (swap), jid 0, uid 1004, was killed: out of swap space mount leak: 3/2980 20190724 03:00:19 all (344/657): nfs13.sh mount leak: 1/2981 20190724 03:11:39 all (345/657): syscall6.sh g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set fsync: giving up on dirty (error = 35) 0x211fdccc: tag devfs, type VCHR usecount 2, writecount 0, refcount 256 rdev 0x189d1400 flags (VI_ACTIVE) v_object 0x210ea1d4 ref 0 pages 65383 cleanbuf 251 dirtybuf 2 lock type devfs: EXCL by thread 0x29f51000 (pid 16151, syscall4, tid 106111) with exclusive waiters pending #0 0xfa6218 at lockmgr_lock_fast_path+0x178 #1 0x107c2f4 at vop_stdlock+0x24 #2 0x15622d8 at VOP_LOCK1_APV+0x58 #3 0x109d3e8 at _vn_lock+0x58 #4 0x10998bb at kern_fsync+0xdb #5 0x1099a32 at sys_fdatasync+0x12 #6 0x15312d4 at syscall+0x304 #7 0xffc033b7 at exit_tag+0xc45f6387 dev label/tmp fsync: giving up on dirty (error = 35) 0x211fdccc: tag devfs, type VCHR usecount 2, writecount 0, refcount 5 rdev 0x189d1400 flags (VI_ACTIVE) v_object 0x210ea1d4 ref 0 pages 65383 cleanbuf 1 dirtybuf 1 lock type devfs: EXCL by thread 0x28aec380 (pid 16151, syscall4, tid 106581) with exclusive waiters pending #0 0xfa68b7 at lockmgr_xlock_hard+0x317 #1 0xfa6239 at lockmgr_lock_fast_path+0x199 #2 0x107c2f4 at vop_stdlock+0x24 #3 0x15622d8 at VOP_LOCK1_APV+0x58 #4 0x109d3e8 at _vn_lock+0x58 #5 0x10998bb at kern_fsync+0xdb #6 0x1099a32 at sys_fdatasync+0x12 #7 0x15312d4 at syscall+0x304 #8 0xffc033b7 at exit_tag+0xc45f6387 dev label/tmp fsync: giving up on dirty (error = 35) 0x211fdccc: tag devfs, type VCHR usecount 2, writecount 0, refcount 5 rdev 0x189d1400 flags (VI_ACTIVE) v_object 0x210ea1d4 ref 0 pages 65383 cleanbuf 1 dirtybuf 1 lock type devfs: EXCL by thread 0x27a04a80 (pid 16151, syscall4, tid 106039) with exclusive waiters pending #0 0xfa68b7 at lockmgr_xlock_hard+0x317 #1 0xfa6239 at lockmgr_lock_fast_path+0x199 #2 0x107c2f4 at vop_stdlock+0x24 #3 0x15622d8 at VOP_LOCK1_APV+0x58 #4 0x109d3e8 at _vn_lock+0x58 #5 0x10998bb at kern_fsync+0xdb #6 0x1099a32 at sys_fdatasync+0x12 #7 0x15312d4 at syscall+0x304 #8 0xffc033b7 at exit_tag+0xc45f6387 dev label/tmp fsync: giving up on dirty (error = 35) 0x211fdccc: tag devfs, type VCHR usecount 2, writecount 0, refcount 4 rdev 0x189d1400 flags (VI_ACTIVE) v_object 0x210ea1d4 ref 0 pages 65383 cleanbuf 0 dirtybuf 1 lock type devfs: EXCL by thread 0x2c14a000 (pid 16151, syscall4, tid 106282) with exclusive waiters pending #0 0xfa68b7 at lockmgr_xlock_hard+0x317 #1 0xfa6239 at lockmgr_lock_fast_path+0x199 #2 0x107c2f4 at vop_stdlock+0x24 #3 0x15622d8 at VOP_LOCK1_APV+0x58 #4 0x109d3e8 at _vn_lock+0x58 #5 0x10998bb at kern_fsync+0xdb #6 0x1099a32 at sys_fdatasync+0x12 #7 0x15312d4 at syscall+0x304 #8 0xffc033b7 at exit_tag+0xc45f6387 dev label/tmp fsync: giving up on dirty (error = 35) 0x211fdccc: tag devfs, type VCHR usecount 2, writecount 0, refcount 4 rdev 0x189d1400 flags (VI_ACTIVE) v_object 0x210ea1d4 ref 0 pages 65383 cleanbuf 0 dirtybuf 1 lock type devfs: EXCL by thread 0x2c6d6700 (pid 16151, syscall4, tid 106052) with exclusive waiters pending #0 0xfa68b7 at lockmgr_xlock_hard+0x317 #1 0xfa6239 at lockmgr_lock_fast_path+0x199 #2 0x107c2f4 at vop_stdlock+0x24 #3 0x15622d8 at VOP_LOCK1_APV+0x58 #4 0x109d3e8 at _vn_lock+0x58 #5 0x10998bb at kern_fsync+0xdb #6 0x1099a32 at sys_fdatasync+0x12 #7 0x15312d4 at syscall+0x304 #8 0xffc033b7 at exit_tag+0xc45f6387 dev label/tmp g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider ufsid/5d377291cfdb3c9c has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set g_access(958): provider gptid/32f36b0f-ad8b-11e9-bfce-00218515337d has error 6 set mount leak: 3/2984 20190724 03:23:03 all (346/657): mmap7.sh 20190724 03:24:54 all (347/657): pts2.sh 20190724 03:25:01 all (348/657): lookup_shared.sh 20190724 03:25:05 all (349/657): nfsrename.sh 20190724 03:25:10 all (350/657): shm_open.sh 20190724 03:25:42 all (351/657): overcommit2.sh Jul 24 03:25:57 x4 kernel: pid 91592 (dd), uid 0 inumber 6 on /mnt: filesystem full mount leak: 1/2985 20190724 03:26:05 all (352/657): nfs9.sh 20190724 03:26:11 all (353/657): sem_timedwait.sh mount leak: 1/2986 20190724 03:31:48 all (354/657): extattrctl.sh 20190724 03:31:54 all (355/657): ptrace7.sh 20190724 03:36:20 all (356/657): truncate8.sh mount leak: 2/2988 20190724 03:39:29 all (357/657): kern_umtx_inf_loop.sh 20190724 03:39:36 all (358/657): nullfs4.sh mount leak: 1/2989 20190724 03:39:42 all (359/657): pkru2.sh 20190724 03:39:46 all (360/657): pcatch.sh 20190724 03:39:50 all (361/657): flock_open_close.sh mount leak: 1/2990 20190724 03:48:27 all (362/657): mlockall4.sh mount leak: 1/2991 20190724 03:48:54 all (363/657): nullfs10.sh mount leak: 1/2992 20190724 03:49:01 all (364/657): nfs_halfpage.sh 20190724 03:49:09 all (365/657): nfs15lockd2.sh 20190724 03:49:25 all (366/657): rename9.sh mount leak: 1/2993 20190724 03:50:21 all (367/657): zfs5.sh ZFS NOTICE: Prefetch is disabled by default on i386 -- to enable, add "vfs.zfs.prefetch_disable=0" to /boot/loader.conf. ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Jul 24 03:59:28 x4 kernel: pid 21321 (swap), jid 0, uid 0, was killed: out of swap space Warning: memory type solaris leaked memory on destroy (4 allocations, 576 bytes leaked). 20190724 04:02:34 all (368/657): pmc.sh hwpmc: SOFT/16/64/0x67 TSC/1/64/0x20 K8/16/48/0x1ff 20190724 04:02:41 all (369/657): select3.sh 20190724 04:02:45 all (370/657): suj17.sh mount leak: 1/2994 20190724 04:15:52 all (371/657): datamove4.sh mount leak: 1/2995 20190724 04:16:12 all (372/657): sendfile4.sh 20190724 04:16:19 all (373/657): core.sh 20190724 04:20:04 all (374/657): umountf2.sh mount leak: 1/2996 20190724 04:36:02 all (375/657): devfs2.sh 20190724 04:36:08 all (376/657): tmpfs17.sh Jul 24 04:36:12 x4 mountd[554]: can't export /mnt mount option is unknown 20190724 04:41:16 all (377/657): mmap27.sh 20190724 04:43:24 all (378/657): core2.sh mount leak: 1/2997 20190724 04:45:35 all (379/657): isofs2.sh mount leak: 1/2998 20190724 04:45:42 all (380/657): fsgs.sh 20190724 04:46:47 all (381/657): posix_openpt2.sh pts leak: 11/95 20190724 04:53:13 all (382/657): dd.sh 20190724 04:57:04 all (383/657): jail.sh 20190724 04:57:15 all (384/657): schedfuzz.sh 20190724 04:57:19 all (385/657): posix_openpt.sh 20190724 04:58:17 all (386/657): kevent9.sh 20190724 05:03:29 all (387/657): datamove5.sh mount leak: 1/2999 20190724 05:03:52 all (388/657): pfl.sh mount leak: 1/3000 20190724 05:14:23 all (389/657): kinfo.sh 20190724 05:34:31 all (390/657): mmap23.sh 20190724 05:34:39 all (391/657): fsck4.sh 20190724 05:35:31 all (392/657): sysctl2.sh Jul 24 05:37:20 x4 kernel: pid 17814 (swap), jid 0, uid 0, was killed: out of swap space Jul 24 05:37:36 x4 kernel: pid 17815 (swap), jid 0, uid 0, was killed: out of swap space 20190724 05:43:34 all (393/657): nullfs.sh 20190724 05:48:44 all (394/657): nfs8.sh 20190724 05:48:49 all (395/657): callout_reset_on2.sh 20190724 05:59:04 all (396/657): mkfifo.sh 20190724 06:20:58 all (397/657): mmap9.sh 20190724 06:21:11 all (398/657): msdos10.sh mount leak: 1/3001 20190724 06:25:11 all (399/657): ext2fs.sh swap_pager: cannot allocate bio swap_pager: cannot allocate bio swap_pager: cannot allocate bio swap_pager: cannot allocate bio swap_pager: cannot allocate bio swap_pager: cannot allocate bio swap_pager: cannot allocate bio swap_pager: cannot allocate bio swap_pager: cannot allocate bio swap_pager: cannot allocate bio swap_pager: cannot allocate bio swap_pager: cannot allocate bio swap_pager: cannot allocate bio swap_pager: cannot allocate bio swap_pager: cannot allocate bio swap_pager: cannot allocate bio swap_pager: cannot allocate bio swap_pager: cannot allocate bio swap_pager: cannot allocate bio swap_pager: cannot allocate bio swap_pager: cannot allocate bio swap_pager: cannot allocate bio swap_pager: cannot allocate bio swap_pager: cannot allocate bio swap_pager: cannot allocate bio mount leak: 1/3002 20190724 06:36:06 all (400/657): pty.sh pts leak: 4/99 20190724 06:41:20 all (401/657): ptrace.sh 20190724 06:41:26 all (402/657): nullfs19.sh mount leak: 1/3003 20190724 06:43:40 all (403/657): vnodes.sh 20190724 06:46:46 all (404/657): mmap22.sh 20190724 06:51:52 all (405/657): symlink.sh mount leak: 2/3005 20190724 07:12:26 all (406/657): posix_fadvise3.sh 20190724 07:13:21 all (407/657): fifo3.sh mount leak: 1/3006 20190724 07:16:17 all (408/657): nullfs2.sh mount leak: 1/3007 20190724 07:27:49 all (409/657): md5.sh 20190724 07:27:56 all (410/657): setsockopt.sh 20190724 07:31:16 all (411/657): split.sh 20190724 07:50:28 all (412/657): stack_guard_page.sh mount leak: 1/3008 20190724 07:51:42 all (413/657): unix_socket_detach2.sh 20190724 07:56:55 all (414/657): devfd.sh mount leak: 1/3009 20190724 07:57:10 all (415/657): core3.sh Jul 24 07:57:15 x4 kernel: Failed to write core file for process core3 (error 5) Jul 24 07:57:46 x4 syslogd: last message repeated 19579 times Jul 24 07:59:47 x4 syslogd: last message repeated 78668 times Jul 24 08:00:00 x4 syslogd: last message repeated 8539 times Jul 24 08:00:00 x4 kernel: Failed to write core file for process core3 (error 5) Jul 24 08:00:30 x4 syslogd: last message repeated 17415 times Jul 24 08:02:30 x4 syslogd: last message repeated 78708 times Jul 24 08:07:14 x4 syslogd: last message repeated 176958 times mount leak: 15412/18421 GEOM leak: 2933/3215 20190724 08:17:17 all (416/657): sysctl.sh 20190724 08:17:35 all (417/657): newfs.sh Jul 24 08:27:14 x4 kernel: pid 6896 (creat), uid 1004 inumber 8203 on /mnt: out of inodes Jul 24 08:27:44 x4 kernel: pid 9424 (creat), uid 1004 inumber 16390 on /mnt: out of inodes Jul 24 08:27:53 x4 kernel: pid 10318 (mkdir), uid 1004 inumber 8190 on /mnt: out of inodes Jul 24 08:27:54 x4 kernel: pid 10286 (mkdir), uid 1004 inumber 8095 on /mnt: out of inodes Jul 24 08:27:57 x4 kernel: pid 10306 (creat), uid 1004 inumber 12313 on /mnt: out of inodes Jul 24 08:28:22 x4 kernel: pid 12732 (creat), uid 1004 inumber 16390 on /mnt: out of inodes Jul 24 08:28:38 x4 kernel: pid 14391 (creat), uid 1004 inumber 12288 on /mnt: out of inodes Jul 24 08:32:30 x4 kernel: pid 34558 (mkdir), uid 1004 inumber 1548 on /mnt: out of inodes mount leak: 64/18485 20190724 08:32:34 all (418/657): extattr.sh mount leak: 1/18486 20190724 08:32:45 all (419/657): gbde.sh mount leak: 1/18487 20190724 08:53:37 all (420/657): pmc4.sh hwpmc: SOFT/16/64/0x67 TSC/1/64/0x20 K8/16/48/0x1ff 20190724 08:53:50 all (421/657): suj35.sh Jul 24 08:53:54 x4 kernel: pid 68661 (mkdir), uid 0 inumber 255 on /mnt: out of inodes mount leak: 1/18488 20190724 08:53:56 all (422/657): truss2.sh 20190724 08:55:03 all (423/657): suj9.sh Jul 24 08:59:44 x4 kernel: pid 73613 (creat), uid 1004 inumber 8194 on /mnt: out of inodes Jul 24 09:00:45 x4 kernel: pid 73686 (link), uid 1004 inumber 4180 on /mnt: out of inodes Jul 24 09:00:54 x4 kernel: pid 73612 (creat), uid 1004 inumber 4141 on /mnt: out of inodes Jul 24 09:01:06 x4 kernel: pid 73625 (creat), uid 1004 inumber 8197 on /mnt: out of inodes Jul 24 09:01:11 x4 kernel: pid 73612 (creat), uid 1004 inumber 4141 on /mnt: out of inodes Jul 24 09:01:15 x4 kernel: pid 73617 (creat), uid 1004 inumber 4144 on /mnt: out of inodes Jul 24 09:01:23 x4 kernel: pid 73621 (creat), uid 1004 inumber 8196 on /mnt: out of inodes Jul 24 09:01:37 x4 kernel: pid 75504 (rw), uid 1004 inumber 3207 on /mnt: filesystem full Jul 24 09:01:40 x4 kernel: pid 75523 (rw), uid 1004 inumber 8353 on /mnt: filesystem full Jul 24 09:02:25 x4 kernel: pid 75518 (creat), uid 1004 inumber 4145 on /mnt: out of inodes Jul 24 09:02:29 x4 kernel: pid 75528 (rw), uid 1004 inumber 2843 on /mnt: filesystem full Jul 24 09:02:43 x4 kernel: pid 75513 (creat), uid 1004 inumber 4138 on /mnt: out of inodes Jul 24 09:02:46 x4 kernel: pid 75515 (creat), uid 1004 inumber 4144 on /mnt: out of inodes Jul 24 09:02:51 x4 kernel: pid 75485 (openat), uid 1004 inumber 4126 on /mnt: out of inodes Jul 24 09:02:53 x4 kernel: pid 75512 (creat), uid 1004 inumber 4133 on /mnt: out of inodes Jul 24 09:02:57 x4 syslogd: last message repeated 1 times Jul 24 09:03:02 x4 kernel: pid 75500 (rw), uid 1004 inumber 4143 on /mnt: out of inodes Jul 24 09:03:09 x4 kernel: pid 75515 (creat), uid 1004 inumber 4144 on /mnt: out of inodes Jul 24 09:03:12 x4 kernel: pid 75513 (creat), uid 1004 inumber 4138 on /mnt: out of inodes Jul 24 09:03:19 x4 kernel: pid 75512 (creat), uid 1004 inumber 4133 on /mnt: out of inodes Jul 24 09:03:24 x4 kernel: pid 75488 (openat), uid 1004 inumber 4108 on /mnt: out of inodes Jul 24 09:03:26 x4 kernel: pid 75516 (creat), uid 1004 inumber 4121 on /mnt: out of inodes Jul 24 09:03:29 x4 kernel: pid 75513 (creat), uid 1004 inumber 4138 on /mnt: out of inodes Jul 24 09:04:45 x4 kernel: pid 76963 (rw), uid 1004 inumber 8518 on /mnt: filesystem full Jul 24 09:05:53 x4 kernel: pid 78324 (creat), uid 1004 inumber 8215 on /mnt: out of inodes Jul 24 09:06:10 x4 kernel: pid 78311 (creat), uid 1004 inumber 8214 on /mnt: out of inodes Jul 24 09:06:14 x4 kernel: pid 78360 (openat), uid 1004 inumber 8203 on /mnt: out of inodes Jul 24 09:06:20 x4 kernel: pid 78317 (creat), uid 1004 inumber 8212 on /mnt: out of inodes Jul 24 09:06:36 x4 kernel: pid 78328 (creat), uid 1004 inumber 8217 on /mnt: out of inodes Jul 24 09:06:41 x4 kernel: pid 78366 (symlink), uid 1004 inumber 8219 on /mnt: out of inodes Jul 24 09:06:58 x4 kernel: pid 78359 (symlink), uid 1004 inumber 8 on /mnt: out of inodes Jul 24 09:07:01 x4 kernel: pid 78372 (mkdir), uid 1004 inumber 4096 on /mnt: out of inodes Jul 24 09:07:09 x4 kernel: pid 78369 (openat), uid 1004 inumber 8206 on /mnt: out of inodes Jul 24 09:07:10 x4 kernel: pid 78314 (creat), uid 1004 inumber 8204 on /mnt: out of inodes Jul 24 09:07:13 x4 kernel: pid 78321 (creat), uid 1004 inumber 8218 on /mnt: out of inodes Jul 24 09:07:20 x4 kernel: pid 78311 (creat), uid 1004 inumber 8214 on /mnt: out of inodes Jul 24 09:07:24 x4 kernel: pid 78314 (creat), uid 1004 inumber 8204 on /mnt: out of inodes Jul 24 09:07:37 x4 kernel: pid 78311 (creat), uid 1004 inumber 8214 on /mnt: out of inodes mount leak: 4/18492 20190724 09:20:35 all (424/657): tmpfs10.sh mount leak: 1/18493 20190724 09:21:00 all (425/657): suj33.sh fsync: giving up on dirty (error = 35) 0x24cd1ccc: tag devfs, type VCHR usecount 1, writecount 0, refcount 345 rdev 0x189d2600 flags (VI_ACTIVE) v_object 0x3240130c ref 0 pages 5371 cleanbuf 258 dirtybuf 85 lock type devfs: EXCL by thread 0x2a839a80 (pid 93577, umount, tid 100915) with exclusive waiters pending #0 0xfa6218 at lockmgr_lock_fast_path+0x178 #1 0x107c2f4 at vop_stdlock+0x24 #2 0x15622d8 at VOP_LOCK1_APV+0x58 #3 0x109d3e8 at _vn_lock+0x58 #4 0x127e08b at softdep_flushworklist+0x5b #5 0x129c4b1 at ffs_sync+0x411 #6 0x1085505 at dounmount+0x425 #7 0x1084feb at sys_unmount+0x26b #8 0x15312d4 at syscall+0x304 #9 0xffc033b7 at exit_tag+0xc45f6387 dev md10a fsync: giving up on dirty (error = 35) 0x24cd1ccc: tag devfs, type VCHR usecount 1, writecount 0, refcount 345 rdev 0x189d2600 flags (VI_ACTIVE) v_object 0x3240130c ref 0 pages 5371 cleanbuf 258 dirtybuf 85 lock type devfs: EXCL by thread 0x2a839a80 (pid 93577, umount, tid 100915) with exclusive waiters pending #0 0xfa6218 at lockmgr_lock_fast_path+0x178 #1 0x107c2f4 at vop_stdlock+0x24 #2 0x15622d8 at VOP_LOCK1_APV+0x58 #3 0x109d3e8 at _vn_lock+0x58 #4 0x129c34d at ffs_sync+0x2ad #5 0x1085505 at dounmount+0x425 #6 0x1084feb at sys_unmount+0x26b #7 0x15312d4 at syscall+0x304 #8 0xffc033b7 at exit_tag+0xc45f6387 dev md10a mount leak: 1/18494 20190724 09:26:52 all (426/657): ext3fs.sh mount leak: 1/18495 20190724 09:32:59 all (427/657): machipc2.sh 20190724 09:33:02 all (428/657): kevent8.sh mount leak: 1/18496 20190724 09:33:48 all (429/657): datagram3.sh 20190724 09:33:54 all (430/657): pkru.sh 20190724 09:33:58 all (431/657): uma_zalloc_arg.sh 20190724 09:34:04 all (432/657): sendfile9.sh mount leak: 1/18497 20190724 09:35:11 all (433/657): msdos3.sh mount leak: 1/18498 20190724 09:35:17 all (434/657): nfssillyrename.sh 20190724 09:35:24 all (435/657): dfull.sh Jul 24 09:36:30 x4 kernel: pid 51351 (rw), uid 0 inumber 32782 on /mnt: filesystem full Jul 24 09:37:14 x4 kernel: pid 51352 (creat), uid 0 inumber 49153 on /mnt: out of inodes Jul 24 09:37:32 x4 kernel: pid 54946 (rw), uid 0 inumber 49161 on /mnt: filesystem full Jul 24 09:38:33 x4 kernel: pid 57311 (rw), uid 0 inumber 32797 on /mnt: filesystem full Jul 24 09:38:39 x4 kernel: pid 57314 (rw), uid 0 inumber 56629 on /mnt: filesystem full Jul 24 09:39:34 x4 kernel: pid 61552 (rw), uid 0 inumber 52089 on /mnt: filesystem full Jul 24 09:40:34 x4 kernel: pid 65504 (rw), uid 0 inumber 34635 on /mnt: filesystem full Jul 24 09:41:33 x4 kernel: pid 69004 (rw), uid 0 inumber 8580 on /mnt: filesystem full Jul 24 09:43:36 x4 kernel: pid 75374 (rw), uid 0 inumber 51307 on /mnt: filesystem full Jul 24 09:43:55 x4 kernel: pid 75380 (rw), uid 0 inumber 51200 on /mnt: filesystem full Jul 24 09:44:36 x4 kernel: pid 80426 (rw), uid 0 inumber 49163 on /mnt: filesystem full Jul 24 09:44:41 x4 kernel: pid 80436 (mkdir), uid 0 inumber 38921 on /mnt: filesystem full Jul 24 09:44:44 x4 kernel: pid 80440 (mkdir), uid 0 inumber 49188 on /mnt: filesystem full Jul 24 09:44:47 x4 kernel: pid 80438 (mkdir), uid 0 inumber 38223 on /mnt: filesystem full Jul 24 09:45:37 x4 kernel: pid 85353 (rw), uid 0 inumber 8 on /mnt: filesystem full Jul 24 09:46:38 x4 kernel: pid 89531 (rw), uid 0 inumber 52258 on /mnt: filesystem full Jul 24 09:46:40 x4 kernel: pid 89517 (mkdir), uid 0 inumber 4390 on /mnt: filesystem full Jul 24 09:47:37 x4 kernel: pid 94652 (rw), uid 0 inumber 63286 on /mnt: filesystem full Jul 24 09:48:38 x4 kernel: pid 97893 (rw), uid 0 inumber 37809 on /mnt: filesystem full Jul 24 09:49:39 x4 kernel: pid 983 (rw), uid 0 inumber 32782 on /mnt: filesystem full Jul 24 09:50:39 x4 kernel: pid 3041 (rw), uid 0 inumber 32770 on /mnt: filesystem full Jul 24 09:52:41 x4 kernel: pid 10107 (rw), uid 0 inumber 49291 on /mnt: filesystem full Jul 24 09:53:40 x4 kernel: pid 13578 (rw), uid 0 inumber 49936 on /mnt: filesystem full Jul 24 09:55:41 x4 kernel: pid 18567 (rw), uid 0 inumber 49214 on /mnt: filesystem full Jul 24 09:55:42 x4 kernel: pid 18544 (rw), uid 0 inumber 52958 on /mnt: filesystem full Jul 24 09:56:42 x4 kernel: pid 24118 (rw), uid 0 inumber 49280 on /mnt: filesystem full Jul 24 09:56:45 x4 kernel: pid 24131 (rw), uid 0 inumber 54530 on /mnt: filesystem full Jul 24 09:57:43 x4 kernel: pid 26525 (rw), uid 0 inumber 33027 on /mnt: filesystem full Jul 24 09:58:44 x4 kernel: pid 32011 (rw), uid 0 inumber 32925 on /mnt: filesystem full Jul 24 09:59:43 x4 kernel: pid 35409 (rw), uid 0 inumber 88 on /mnt: filesystem full Jul 24 09:59:46 x4 kernel: pid 35417 (rw), uid 0 inumber 90 on /mnt: filesystem full Jul 24 10:00:44 x4 kernel: pid 38161 (rw), uid 0 inumber 644 on /mnt: filesystem full Jul 24 10:01:44 x4 kernel: pid 41687 (rw), uid 0 inumber 50720 on /mnt: filesystem full Jul 24 10:02:44 x4 kernel: pid 47496 (rw), uid 0 inumber 210 on /mnt: filesystem full Jul 24 10:03:45 x4 kernel: pid 50867 (rw), uid 0 inumber 58403 on /mnt: filesystem full Jul 24 10:04:45 x4 kernel: pid 53372 (rw), uid 0 inumber 33862 on /mnt: filesystem full Jul 24 10:04:47 x4 kernel: pid 53368 (rw), uid 0 inumber 131 on /mnt: filesystem full Jul 24 10:04:51 x4 kernel: pid 53376 (mkdir), uid 0 inumber 50686 on /mnt: filesystem full mount leak: 1/18499 20190724 10:05:49 all (436/657): vmtotal.sh 20190724 10:11:04 all (437/657): nullfs26.sh Jul 24 10:14:26 x4 kernel: pid 83369 (swap), jid 0, uid 1004, was killed: out of swap space mount leak: 1/18500 20190724 10:22:11 all (438/657): datagram.sh 20190724 10:22:25 all (439/657): suj30.sh interrupt total irq4: uart0 6178 irq6: fdc0 1 irq9: acpi0 3 irq14: ata0 99364 irq16: hdac1 ohci0 388 irq17: ohci1 ohci3 4 irq18: ohci2 ohci4 3 irq22: ahci0 48015508 cpu0:timer 157732081 cpu1:timer 159554207 cpu2:timer 157805762 cpu3:timer 157632798 irq24: hdac0 5 irq25: re0 826050 Total 681672352 KDB: stack backtrace: db_trace_self_wrapper(3ded8a64,1c32914,0,1ecc690,1ecc680,...) at db_trace_self_wrapper+0x2a/frame 0x3ded8a10 kdb_backtrace(0,28a17ea0,2,9860dc0,8,...) at kdb_backtrace+0x2e/frame 0x3ded8a70 hardclock(1,0) at hardclock+0x1a2/frame 0x3ded8aa8 handleevents(0) at handleevents+0xf8/frame 0x3ded8aec timercb(1f74cdc,0) at timercb+0x207/frame 0x3ded8b20 lapic_handle_timer(3ded8b4c,8,28,28,3ded8bd0,...) at lapic_handle_timer+0x83/frame 0x3ded8b40 Xtimerint() at 0xffc048b1/frame 0x3ded8b40 --- interrupt, eip = 0x108dafe, esp = 0x3ded8b8c, ebp = 0x3ded8b98 --- __mnt_vnode_next_all(3ded8bd0,21a1c000) at __mnt_vnode_next_all+0x11e/frame 0x3ded8b98 ffs_sync(21a1c000,2) at ffs_sync+0x1fc/frame 0x3ded8c04 softdep_process_journal(0) at softdep_process_journal+0x925/frame 0x3ded8c74 softdep_process_worklist(2a924000,2a924000,184569fc,1281720,3ded8cd4,...) at softdep_process_worklist+0x68/frame 0x3ded8c9c softdep_flush(21a1c000,3ded8ce8) at softdep_flush+0xce/frame 0x3ded8cb4 fork_exit(1281720,21a1c000,3ded8ce8,0,0,...) at fork_exit+0x6c/frame 0x3ded8cd4 fork_trampoline() at 0xffc033ca/frame 0x3ded8cd4 --- trap 0, eip = 0, esp = 0x3ded8d20, ebp = 0 --- (null)() at 0 KDB: enter: watchdog timeout [ thread pid 23 tid 106217 ] Stopped at kdb_enter+0x35: movl $0,kdb_why db> run pho db:0:pho> set $lines 20000 db:0:pho> run pho1 db:1:pho1> bt Tracing pid 23 tid 106217 td 0x2a924000 kdb_enter(1595c37,1590be2) at kdb_enter+0x35/frame 0x3ded8a68 hardclock(1,0) at hardclock+0x1b1/frame 0x3ded8aa8 handleevents(0) at handleevents+0xf8/frame 0x3ded8aec timercb(1f74cdc,0) at timercb+0x207/frame 0x3ded8b20 lapic_handle_timer(3ded8b4c,8,28,28,3ded8bd0,...) at lapic_handle_timer+0x83/frame 0x3ded8b40 Xtimerint() at 0xffc048b1/frame 0x3ded8b40 --- interrupt, eip = 0x108dafe, esp = 0x3ded8b8c, ebp = 0x3ded8b98 --- __mnt_vnode_next_all(3ded8bd0,21a1c000) at __mnt_vnode_next_all+0x11e/frame 0x3ded8b98 ffs_sync(21a1c000,2) at ffs_sync+0x1fc/frame 0x3ded8c04 softdep_process_journal(0) at softdep_process_journal+0x925/frame 0x3ded8c74 softdep_process_worklist(2a924000,2a924000,184569fc,1281720,3ded8cd4,...) at softdep_process_worklist+0x68/frame 0x3ded8c9c softdep_flush(21a1c000,3ded8ce8) at softdep_flush+0xce/frame 0x3ded8cb4 fork_exit(1281720,21a1c000,3ded8ce8,0,0,...) at fork_exit+0x6c/frame 0x3ded8cd4 fork_trampoline() at 0xffc033ca/frame 0x3ded8cd4 --- trap 0, eip = 0, esp = 0x3ded8d20, ebp = 0 --- (null)() at 0 db:1:pho1> show allpcpu Current CPU: 2 cpuid = 0 dynamic pcpu = 0x8bcac0 curthread = 0x99d9700: pid 11 tid 100003 "idle: cpu0" curpcb = 0x12521d40 fpcurthread = none idlethread = 0x99d9700: tid 100003 "idle: cpu0" APIC ID = 0 currentldt = 0x50 trampstk = 0xffc07ff0 kesp0 = 0x12521d30 common_tssp = 0xffc01260 tlb gen = 2873136614 curvnet = 0 spin locks held: cpuid = 1 dynamic pcpu = 0x7c19ac0 curthread = 0x95b1000: pid 406 tid 100123 "syslogd" curpcb = 0x21334d40 fpcurthread = 0x95b1000: pid 406 "syslogd" idlethread = 0x99d9380: tid 100004 "idle: cpu1" APIC ID = 1 currentldt = 0x50 trampstk = 0xffc0aff0 kesp0 = 0x21334d30 common_tssp = 0xffc012c8 tlb gen = 2873136613 curvnet = 0 spin locks held: cpuid = 2 dynamic pcpu = 0x7c1eac0 curthread = 0x2a924000: pid 23 tid 106217 "/mnt worker" curpcb = 0x3ded8d40 fpcurthread = none idlethread = 0x99d9000: tid 100005 "idle: cpu2" APIC ID = 2 currentldt = 0x50 trampstk = 0xffc0bff0 kesp0 = 0x3ded8d30 common_tssp = 0xffc01330 tlb gen = 2873136613 curvnet = 0 spin locks held: cpuid = 3 dynamic pcpu = 0x7c23ac0 curthread = 0x99d8a80: pid 11 tid 100006 "idle: cpu3" curpcb = 0x12530d40 fpcurthread = none idlethread = 0x99d8a80: tid 100006 "idle: cpu3" APIC ID = 3 currentldt = 0x50 trampstk = 0xffc0dff0 kesp0 = 0x12530d30 common_tssp = 0xffc01398 tlb gen = 2873136614 curvnet = 0 spin locks held: db:1:pho1> show alllocks db:1:pho1> show lockedvnods Locked vnodes vnode 0x3269f444: tag ufs, type VREG usecount 1, writecount 1, refcount 3 flags (VI_ACTIVE) v_object 0x31c46a5c ref 0 pages 3 cleanbuf 0 dirtybuf 1 lock type ufs: EXCL by thread 0x95b1000 (pid 406, syslogd, tid 100123) #0 0xfa68b7 at lockmgr_xlock_hard+0x317 #1 0xfa6d56 at __lockmgr_args+0x376 #2 0x129dccf at ffs_lock+0x6f #3 0x15622d8 at VOP_LOCK1_APV+0x58 #4 0x109d3e8 at _vn_lock+0x58 #5 0x10998bb at kern_fsync+0xdb #6 0x1099a12 at sys_fsync+0x12 #7 0x15312d4 at syscall+0x304 #8 0xffc033b7 at exit_tag+0xc45f6387 nlink=1, effnlink=1, size=11496, extsize 0 generation=69df52e1, uid=0, gid=0, flags=0x0 ino 6052933, on dev ada0s1e vnode 0x2b8f09f4: tag ufs, type VDIR usecount 3, writecount 0, refcount 6 mountedhere 0 flags (VV_ROOT|VI_ACTIVE) v_object 0x33aebc30 ref 0 pages 1 cleanbuf 1 dirtybuf 0 lock type ufs: SHARED (count 1) #0 0xfa6541 at lockmgr_slock_hard+0x2c1 #1 0xfa6b06 at __lockmgr_args+0x126 #2 0x129dccf at ffs_lock+0x6f #3 0x15622d8 at VOP_LOCK1_APV+0x58 #4 0x109d3e8 at _vn_lock+0x58 #5 0x108c19b at vget+0x7b #6 0x107f623 at vfs_hash_get+0xa3 #7 0x12996a7 at ffs_vgetf+0x27 #8 0x1299679 at ffs_vget+0x19 #9 0x12a9e29 at ufs_root+0x19 #10 0x10817af at lookup+0x73f #11 0x1080bd1 at namei+0x4c1 #12 0x1099b85 at kern_renameat+0x125 #13 0x1099a59 at sys_rename+0x19 #14 0x15312d4 at syscall+0x304 #15 0xffc033b7 at exit_tag+0xc45f6387 nlink=23, effnlink=23, size=1024, extsize 0 generation=49fbface, uid=0, gid=0, flags=0x0 ino 2, on dev md10a vnode 0x2e9ac9f4: tag ufs, type VDIR usecount 2, writecount 0, refcount 6 mountedhere 0 flags (VI_ACTIVE) v_object 0x330bfa5c ref 0 pages 0 cleanbuf 0 dirtybuf 1 lock type ufs: EXCL by thread 0x3002e700 (pid 90966, suj30, tid 106274) with exclusive waiters pending with shared waiters pending #0 0xfa68b7 at lockmgr_xlock_hard+0x317 #1 0xfa6d56 at __lockmgr_args+0x376 #2 0x129dccf at ffs_lock+0x6f #3 0x15622d8 at VOP_LOCK1_APV+0x58 #4 0x109d3e8 at _vn_lock+0x58 #5 0x12abd35 at ufs_rename+0x155 #6 0x1561802 at VOP_RENAME_APV+0x52 #7 0x1099d95 at kern_renameat+0x335 #8 0x1099a59 at sys_rename+0x19 #9 0x15312d4 at syscall+0x304 #10 0xffc033b7 at exit_tag+0xc45f6387 nlink=304, effnlink=3, size=1536, extsize 0 generation=de9266eb, uid=0, gid=0, flags=0x0 ino 321028, on dev md10a vnode 0x543ff444: tag ufs, type VDIR usecount 0, writecount 0, refcount 4 mountedhere 0 flags (VI_DOOMED|VI_ACTIVE) lock type ufs: EXCL by thread 0x183d0a80 (pid 24, vnlru, tid 100103) with exclusive waiters pending #0 0xfa68b7 at lockmgr_xlock_hard+0x317 #1 0xfa6d56 at __lockmgr_args+0x376 #2 0x129dccf at ffs_lock+0x6f #3 0x15622d8 at VOP_LOCK1_APV+0x58 #4 0x10908dd at vnlru_proc+0x45d #5 0xf941dc at fork_exit+0x6c #6 0xffc033ca at exit_tag+0xc45f639a nlink=3, effnlink=2, size=512, extsize 0 generation=b5cb0863, uid=0, gid=0, flags=0x0 ino 335572, on dev md10a db:1:pho1> show allchains chain 1: thread 100958 (pid 69946, sleep) sleeping on 0x1ecd6ff "nanslp" chain 2: thread 101351 (pid 305, ls) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL thread 106274 (pid 90966, loop_rename) blocked on lockmgr ufsEXCL thread 100103 (pid 24, vnlru) blocked on lockmgr ufsEXCL