KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2016 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.0-CURRENT #0 r295557M: Fri Feb 12 07:43:00 CET 2016 pho@t2.osted.lan:/usr/src/sys/amd64/compile/PHO amd64 FreeBSD clang version 3.7.1 (tags/RELEASE_371/final 255217) 20151225 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. VT(vga): resolution 640x480 CPU: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz (1995.23-MHz K8-class CPU) Origin="GenuineIntel" Id=0x206d7 Family=0x6 Model=0x2d Stepping=7 Features=0xbfebfbff Features2=0x1fbee3ff AMD Features=0x2c100800 AMD Features2=0x1 XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID TSC: P-state invariant, performance statistics real memory = 68719476736 (65536 MB) avail memory = 32140697600 (30651 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 24 CPUs FreeBSD/SMP: 2 package(s) x 6 core(s) x 2 SMT threads cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 cpu4 (AP): APIC ID: 4 cpu5 (AP): APIC ID: 5 cpu6 (AP): APIC ID: 6 cpu7 (AP): APIC ID: 7 cpu8 (AP): APIC ID: 8 cpu9 (AP): APIC ID: 9 cpu10 (AP): APIC ID: 10 cpu11 (AP): APIC ID: 11 cpu12 (AP): APIC ID: 32 cpu13 (AP): APIC ID: 33 cpu14 (AP): APIC ID: 34 cpu15 (AP): APIC ID: 35 cpu16 (AP): APIC ID: 36 cpu17 (AP): APIC ID: 37 cpu18 (AP): APIC ID: 38 cpu19 (AP): APIC ID: 39 cpu20 (AP): APIC ID: 40 cpu21 (AP): APIC ID: 41 cpu22 (AP): APIC ID: 42 cpu23 (AP): APIC ID: 43 random: unblocking device. ACPI BIOS Warning (bug): Invalid length for FADT/Pm1aControlBlock: 32, using default 16 (20150818/tbfadt-730) ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard ioapic2 irqs 48-71 on motherboard random: entropy device external interface kbd1 at kbdmux0 netmap: loaded module module_register_init: MOD_LOAD (vesa, 0xffffffff80f0af20, 0) error 19 vtvga0: on motherboard cryptosoft0: on motherboard acpi0: on motherboard acpi0: Power Button (fixed) cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 cpu4: on acpi0 cpu5: on acpi0 cpu6: on acpi0 cpu7: on acpi0 cpu8: on acpi0 cpu9: on acpi0 cpu10: on acpi0 cpu11: on acpi0 cpu12: on acpi0 cpu13: on acpi0 cpu14: on acpi0 cpu15: on acpi0 cpu16: on acpi0 cpu17: on acpi0 cpu18: on acpi0 cpu19: on acpi0 cpu20: on acpi0 cpu21: on acpi0 cpu22: on acpi0 cpu23: on acpi0 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Event timer "HPET4" frequency 14318180 Hz quality 340 Event timer "HPET5" frequency 14318180 Hz quality 340 Event timer "HPET6" frequency 14318180 Hz quality 340 Event timer "HPET7" frequency 14318180 Hz quality 340 atrtc0: port 0x70-0x77 irq 8 on acpi0 atrtc0: Warning: Couldn't map I/O. Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: irq 47 at device 1.0 on pci0 pci1: on pcib1 pcib2: irq 47 at device 2.0 on pci0 pci2: on pcib2 pcib3: irq 47 at device 3.0 on pci0 pci3: on pcib3 pcib4: irq 16 at device 17.0 on pci0 pci4: on pcib4 isci0: port 0x2000-0x20ff mem 0xd7c00000-0xd7c03fff,0xd7800000-0xd7bfffff irq 16 at device 0.0 on pci4 pci0: at device 22.0 (no driver attached) pci0: at device 22.1 (no driver attached) ehci0: mem 0xd0420000-0xd04203ff irq 22 at device 26.0 on pci0 usbus0: waiting for BIOS to give up control usbus0: timed out waiting for BIOS usbus0: EHCI version 1.0 usbus0 on ehci0 hdac0: mem 0xd7f20000-0xd7f23fff irq 22 at device 27.0 on pci0 pcib5: irq 16 at device 28.0 on pci0 pci5: on pcib5 igb0: port 0x1020-0x103f mem 0xd0220000-0xd023ffff,0xd0250000-0xd0253fff irq 16 at device 0.0 on pci5 igb0: Using MSIX interrupts with 9 vectors igb0: Ethernet address: 00:1e:67:56:b6:9b igb0: PHY reset is blocked due to SOL/IDER session. igb0: Bound queue 0 to cpu 0 igb0: Bound queue 1 to cpu 1 igb0: Bound queue 2 to cpu 2 igb0: Bound queue 3 to cpu 3 igb0: Bound queue 4 to cpu 4 igb0: Bound queue 5 to cpu 5 igb0: Bound queue 6 to cpu 6 igb0: Bound queue 7 to cpu 7 igb0: netmap queues/slots: TX 8/1024, RX 8/1024 igb1: port 0x1000-0x101f mem 0xd0200000-0xd021ffff,0xd0240000-0xd0243fff irq 17 at device 0.1 on pci5 igb1: Using MSIX interrupts with 9 vectors igb1: Ethernet address: 00:1e:67:56:b6:9c igb1: PHY reset is blocked due to SOL/IDER session. igb1: Bound queue 0 to cpu 8 igb1: Bound queue 1 to cpu 9 igb1: Bound queue 2 to cpu 10 igb1: Bound queue 3 to cpu 11 igb1: Bound queue 4 to cpu 12 igb1: Bound queue 5 to cpu 13 igb1: Bound queue 6 to cpu 14 igb1: Bound queue 7 to cpu 15 igb1: netmap queues/slots: TX 8/1024, RX 8/1024 pcib6: irq 17 at device 28.5 on pci0 pci6: on pcib6 pcib7: at device 0.0 on pci6 pci7: on pcib7 pci7: at device 0.0 (no driver attached) pcib8: irq 18 at device 28.6 on pci0 pci8: on pcib8 xhci0: mem 0xd0010000-0xd001ffff,0xd0000000-0xd0001fff irq 18 at device 0.0 on pci8 xhci0: 64 bytes context size, 64-bit DMA usbus1 on xhci0 pcib9: irq 19 at device 28.7 on pci0 pci9: on pcib9 ehci1: mem 0xd0410000-0xd04103ff irq 20 at device 29.0 on pci0 usbus2: waiting for BIOS to give up control usbus2: timed out waiting for BIOS usbus2: EHCI version 1.0 usbus2 on ehci1 pcib10: at device 30.0 on pci0 pci10: on pcib10 isab0: at device 31.0 on pci0 isa0: on isab0 ahci0: port 0x3070-0x3077,0x3060-0x3063,0x3050-0x3057,0x3040-0x3043,0x3020-0x303f mem 0xd0400000-0xd04007ff irq 21 at device 31.2 on pci0 ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahcich4: at channel 4 on ahci0 ahcich5: at channel 5 on ahci0 ahciem0: on ahci0 pcib11: on acpi0 pci11: on pcib11 pcib12: at device 0.0 on pci11 pci12: on pcib12 pcib13: irq 71 at device 1.0 on pci11 pci13: on pcib13 pcib14: irq 71 at device 2.0 on pci11 pci14: on pcib14 vgapci0: port 0xa000-0xa0ff mem 0xe0000000-0xefffffff,0xd8020000-0xd803ffff irq 56 at device 0.0 on pci14 vgapci0: Boot video device hdac1: mem 0xd8040000-0xd8043fff irq 60 at device 0.1 on pci14 pcib15: irq 71 at device 3.0 on pci11 pci15: on pcib15 pcib16: on acpi0 pci16: on pcib16 pcib17: on acpi0 pci17: on pcib17 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 flags 0x10 on acpi0 uart1: console (115200,n,8,1) orm0: at iomem 0xc0000-0xcffff,0xda000-0xdafff,0xdb000-0xdbfff on isa0 ppc0: cannot reserve I/O port range est0: on cpu0 est1: on cpu1 est2: on cpu2 est3: on cpu3 est4: on cpu4 est5: on cpu5 est6: on cpu6 est7: on cpu7 est8: on cpu8 est9: on cpu9 est10: on cpu10 est11: on cpu11 est12: on cpu12 est13: on cpu13 est14: on cpu14 est15: on cpu15 est16: on cpu16 est17: on cpu17 est18: on cpu18 est19: on cpu19 est20: on cpu20 est21: on cpu21 est22: on cpu22 est23: on cpu23 igb0: link state changed to UP usbus0: 480Mbps High Speed USB v2.0 Timecounters tick every 1.000 msec IPsec: Initialized Security Association Processing. hdacc0: at cad 0 on hdac0 hdaa0: at nid 1 on hdacc0 pcm0: at nid 20,22,21,23 and 24,26 on hdaa0 pcm1: at nid 27 and 25 on hdaa0 pcm2: at nid 30 on hdaa0 hdacc1: at cad 0 on hdac1 hdaa1: at nid 1 on hdacc1 pcm3: at nid 3 on hdaa1 usbus1: 5.0Gbps Super Speed USB v3.0 usbus2: 480Mbps High Speed USB v2.0 ugen0.1: at usbus0 uhub0: on usbus0 ugen1.1: <0x104c> at usbus1 uhub1: <0x104c XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus1 ugen2.1: at usbus2 uhub2: on usbus2 ses0 at ahciem0 bus 0 scbus7 target 0 lun 0 ses0: SEMB S-E-S 2.00 device ses0: SEMB SES Device ada0 at ahcich0 bus 0 scbus1 target 0 lun 0 ada0: ACS-2 ATA SATA 3.x device ada0: Serial Number CVCV317607P0240CGN ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 228936MB (468862128 512 byte sectors) da0 at isci0 bus 0 scbus0 target 0 lun 0 da0: Fixed Direct Access SPC-3 SCSI device da0: Serial Number 6SL64N800000N339035N da0: 300.000MB/s transfers da0: Command Queueing enabled da0: 572325MB (1172123568 512 byte sectors) cd0 at ahcich1 bus 0 scbus2 target 0 lun 0 cd0: Removable CD-ROM SCSI device cd0: Serial Number R93E68ED300PN5 cd0: 150.000MB/s transfers (SATA 1.x, UDMA5, ATAPI 12bytes, PIO 8192bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present - tray closed SMP: AP CPU #1 Launched! SMP: AP CPU #4 Launched! SMP: AP CPU #14 Launched! SMP: AP CPU #7 Launched! SMP: AP CPU #17 Launched! SMP: AP CPU #9 Launched! SMP: AP CPU #11 Launched! SMP: AP CPU #21 Launched! SMP: AP CPU #10 Launched! SMP: AP CPU #5 Launched! SMP: AP CPU #15 Launched! SMP: AP CPU #8 Launched! SMP: AP CPU #13 Launched! SMP: AP CPU #6 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #20 Launched! SMP: AP CPU #18 Launched! SMP: AP CPU #19 Launched! SMP: AP CPU #23 Launched! SMP: AP CPU #12 Launched! SMP: AP CPU #22 Launched! SMP: AP CPU #16 Launched! Timecounter "TSC" frequency 1995234400 Hz quality 1000 Expensive timeout(9) function: 0xffffffff808b9e70(0xffffffff81887f18) 0.021048345 s WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. Trying to mount root from ufs:/dev/da0p2 [rw]... uhub1: 4 ports with 4 removable, self powered uhub2: 2 ports with 2 removable, self powered uhub0: 2 ports with 2 removable, self powered Setting hostuuid: 2bde2bde-f4e2-e111-aab2-001e6756b69b. Setting hostid: 0x0035ff86. Starting ddb. Starting file system checks: /dev/da0p2: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/da0p2: clean, 43372142 free (695630 frags, 5334564 blocks, 0.7% fragmentation) ugen0.2: at usbus0 uhub3: on usbus0 ugen2.2: at usbus2 uhub4: on usbus2 /dev/da0p3: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/da0p3: clean, 22449974 free (3406 frags, 2805821 blocks, 0.0% fragmentation) Mounting local file systems:. lock order reversal: 1st 0xfffff80010f90068 ufs (ufs) @ kern/vfs_subr.c:2476 2nd 0xfffffe0785636f50 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:263 3rd 0xfffff80012cfda28 ufs (ufs) @ kern/vfs_subr.c:2476 stack backtrace: #0 0xffffffff80a84e20 at witness_debugger+0x70 #1 0xffffffff80a84d21 at witness_checkorder+0xe71 #2 0xffffffff80a03727 at __lockmgr_args+0xd47 #3 0xffffffff80ce6610 at ffs_lock+0xb0 #4 0xffffffff80fe64f0 at VOP_LOCK1_APV+0x100 #5 0xffffffff80b0c853 at VOP_LOCK1+0x53 #6 0xffffffff80b0a741 at _vn_lock+0xc1 #7 0xffffffff80aedcfa at vget+0x14a #8 0xffffffff80ad89e1 at vfs_hash_get+0xf1 #9 0xffffffff80ce1c50 at ffs_vgetf+0x40 #10 0xffffffff80cd95b1 at softdep_sync_buf+0xad1 #11 0xffffffff80ce7216 at ffs_syncvnode+0x256 #12 0xffffffff80cbe0e0 at ffs_truncate+0x650 #13 0xffffffff80cf02cb at ufs_direnter+0x7bb #14 0xffffffff80cf9633 at ufs_makeinode+0x5f3 #15 0xffffffff80cf51dd at ufs_create+0x2d #16 0xffffffff80fe375f at VOP_CREATE_APV+0x10f #17 0xffffffff80b09f49 at VOP_CREATE+0x59 ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/gcc48 /usr/local/lib/nss /usr/local/llvm36/lib 32-bit compatibility ldconfig path: /usr/lib32 uhub3: 6 ports with 6 removable, self powered Setting hostname: t2.osted.lan. Setting up harvesting:[UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,NET_ETHER,NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED Feeding entropy:uhub4: 8 ports with 8 removable, self powered . ugen0.3: at usbus0 ukbd0: on usbus0 kbd2 at ukbd0 ugen2.3: at usbus2 uhub5: on usbus2 uhub5: MTT enabled uhub5: 4 ports with 4 removable, self powered ugen2.4: at usbus2 ukbd1: on usbus2 kbd3 at ukbd1 Starting Network: lo0 igb0 igb1. lo0: flags=8049 metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 nd6 options=21 groups: lo igb0: flags=8843 metric 0 mtu 1500 options=403bb ether 00:1e:67:56:b6:9b inet 192.168.1.109 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::21e:67ff:fe56:b69b%igb0 prefixlen 64 scopeid 0x1 nd6 options=23 media: Ethernet autoselect (1000baseT ) status: active igb1: flags=8c02 metric 0 mtu 1500 options=403bb ether 00:1e:67:56:b6:9c nd6 options=29 media: Ethernet autoselect status: no carrier Starting devd. Starting Network: igb1. igb1: flags=8c02 metric 0 mtu 1500 options=403bb ether 00:1e:67:56:b6:9c nd6 options=29 media: Ethernet autoselect status: no carrier Configuring vt: keymap. ums0: on usbus0 ums1: on usbus2 ums0: 18 buttons and [XYZT] coordinates ID=2 ums1: 3 buttons and [Z] coordinates ID=0 add net default: gateway 192.168.1.1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Starting rtsold. Creating and/or trimming log files. Starting syslogd. Starting watchdogd. No core dumps found. Starting rpcbind. NFS access cache time=60 Starting casperd. lock order reversal: 1st 0xfffffe0785662110 bufwait (bufwait) @ kern/vfs_bio.c:3488 2nd 0xfffff8005736a000 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:281 stack backtrace: #0 0xffffffff80a84e20 at witness_debugger+0x70 #1 0xffffffff80a84d21 at witness_checkorder+0xe71 #2 0xffffffff80a31ea2 at _sx_xlock+0x72 #3 0xffffffff80ceb5cd at ufsdirhash_add+0x4d #4 0xffffffff80cf013f at ufs_direnter+0x62f #5 0xffffffff80cf877a at ufs_mkdir+0x8ba #6 0xffffffff80fe58f5 at VOP_MKDIR_APV+0x115 #7 0xffffffff80b06069 at VOP_MKDIR+0x59 #8 0xffffffff80b05f4f at kern_mkdirat+0x3ef #9 0xffffffff80b05b50 at sys_mkdir+0x40 #10 0xffffffff80e97176 at amd64_syscall+0x2f6 #11 0xffffffff80e76cdb at Xfast_syscall+0xfb Clearing /tmp (X related). Starting nfsuserd. Starting mountd. Starting nfsd. Recovering vi editor sessions:. Updating motd:. Mounting late file systems:. Starting ntpd. Starting powerd. Configuring vt: keymap blanktime. Performing sanity check on sshd configuration. Starting sshd. Starting sendmail_submit. Starting sendmail_msp_queue. Starting cron. Local package initialization: backup. Starting default mousedmoused: unable to open /dev/psm0: No such file or directory . Fri Feb 12 07:47 FreeBSD/amd64 (t2.osted.lan) (ttyu1) login: Feb 12 07:54:49 t2 su: pho to root on /dev/pts/0 20160212 07:55:01 all (1/1): nullfs5.sh lock order reversal: 1st 0xfffff8008fc50490 filedesc structure (filedesc structure) @ kern/kern_descrip.c:836 2nd 0xfffff800571e42d8 ufs (ufs) @ kern/vfs_subr.c:4825 stack backtrace: #0 0xffffffff80a84e20 at witness_debugger+0x70 #1 0xffffffff80a84d21 at witness_checkorder+0xe71 #2 0xffffffff80a03727 at __lockmgr_args+0xd47 #3 0xffffffff80ce6610 at ffs_lock+0xb0 #4 0xffffffff80fe64f0 at VOP_LOCK1_APV+0x100 #5 0xffffffff80b0c853 at VOP_LOCK1+0x53 #6 0xffffffff80b0a741 at _vn_lock+0xc1 #7 0xffffffff80af13a6 at vfs_knllock+0x36 #8 0xffffffff809e4b3e at knlist_remove_kq+0x7e #9 0xffffffff80af9f0a at filt_vfsdetach+0x7a #10 0xffffffff809e5417 at knote_fdclose+0xc7 #11 0xffffffff809d95f5 at closefp+0x65 #12 0xffffffff809d7f67 at kern_dup+0x617 #13 0xffffffff80e97176 at amd64_syscall+0x2f6 #14 0xffffffff80e76cdb at Xfast_syscall+0xfb 20160212 08:07:27 all (1/1): nullfs5.sh 20160212 08:19:28 all (1/1): nullfs5.sh 20160212 08:31:19 all (1/1): nullfs5.sh 20160212 08:41:51 all (1/1): nullfs5.sh 20160212 08:52:08 all (1/1): nullfs5.sh KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe081d8bd260 vfs_badlock() at vfs_badlock+0x29/frame 0xfffffe081d8bd290 assert_vop_locked() at assert_vop_locked+0x86/frame 0xfffffe081d8bd2c0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x18b/frame 0xfffffe081d8bd2f0 VOP_LOOKUP() at VOP_LOOKUP+0x4d/frame 0xfffffe081d8bd340 lookup() at lookup+0xa17/frame 0xfffffe081d8bd490 namei() at namei+0x9e2/frame 0xfffffe081d8bd650 kern_statat() at kern_statat+0x16d/frame 0xfffffe081d8bd8d0 sys_fstatat() at sys_fstatat+0x71/frame 0xfffffe081d8bd9a0 amd64_syscall() at amd64_syscall+0x2f6/frame 0xfffffe081d8bdab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081d8bdab0 --- syscall (493, FreeBSD ELF64, sys_fstatat), rip = 0x800acb1fa, rsp = 0x7fffffffe358, rbp = 0x7fffffffe410 --- VOP_LOOKUP: 0xfffff8071ec04000 is not locked but should be KDB: enter: lock violation [ thread pid 70126 tid 100415 ] Stopped at kdb_enter+0x3b: movq $0,kdb_why db> run pho db:0:pho> bt Tracing pid 70126 tid 100415 td 0xfffff800126a44d0 kdb_enter() at kdb_enter+0x3b/frame 0xfffffe081d8bd260 vfs_badlock() at vfs_badlock+0x82/frame 0xfffffe081d8bd290 assert_vop_locked() at assert_vop_locked+0x86/frame 0xfffffe081d8bd2c0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x18b/frame 0xfffffe081d8bd2f0 VOP_LOOKUP() at VOP_LOOKUP+0x4d/frame 0xfffffe081d8bd340 lookup() at lookup+0xa17/frame 0xfffffe081d8bd490 namei() at namei+0x9e2/frame 0xfffffe081d8bd650 kern_statat() at kern_statat+0x16d/frame 0xfffffe081d8bd8d0 sys_fstatat() at sys_fstatat+0x71/frame 0xfffffe081d8bd9a0 amd64_syscall() at amd64_syscall+0x2f6/frame 0xfffffe081d8bdab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081d8bdab0 --- syscall (493, FreeBSD ELF64, sys_fstatat), rip = 0x800acb1fa, rsp = 0x7fffffffe358, rbp = 0x7fffffffe410 --- db:0:bt> show allpcpu Current CPU: 12 cpuid = 0 dynamic pcpu = 0x7b5f80 curthread = 0xfffff8021a953000: pid 22309 "kinfo" curpcb = 0xfffffe081ded1b80 fpcurthread = 0xfffff8021a953000: pid 22309 "kinfo" idlethread = 0xfffff800075669a0: tid 100004 "idle: cpu0" curpmap = 0xfffff8014394a138 tssp = 0xffffffff81d35310 commontssp = 0xffffffff81d35310 rsp0 = 0xfffffe081ded1b80 gs32p = 0xffffffff81d3bb68 ldt = 0xffffffff81d3bba8 tss = 0xffffffff81d3bb98 spin locks held: cpuid = 1 dynamic pcpu = 0xfffffe0879e3ff80 curthread = 0xfffff800075664d0: pid 11 "idle: cpu1" curpcb = 0xfffffe0785559b80 fpcurthread = none idlethread = 0xfffff800075664d0: tid 100005 "idle: cpu1" curpmap = 0xffffffff81cff580 tssp = 0xffffffff81d35378 commontssp = 0xffffffff81d35378 rsp0 = 0xfffffe0785559b80 gs32p = 0xffffffff81d3bbd0 ldt = 0xffffffff81d3bc10 tss = 0xffffffff81d3bc00 spin locks held: cpuid = 2 dynamic pcpu = 0xfffffe0879e47f80 curthread = 0xfffff80007566000: pid 11 "idle: cpu2" curpcb = 0xfffffe078555eb80 fpcurthread = none idlethread = 0xfffff80007566000: tid 100006 "idle: cpu2" curpmap = 0xffffffff81cff580 tssp = 0xffffffff81d353e0 commontssp = 0xffffffff81d353e0 rsp0 = 0xfffffe078555eb80 gs32p = 0xffffffff81d3bc38 ldt = 0xffffffff81d3bc78 tss = 0xffffffff81d3bc68 spin locks held: cpuid = 3 dynamic pcpu = 0xfffffe0879e4ff80 curthread = 0xfffff800075659a0: pid 11 "idle: cpu3" curpcb = 0xfffffe0785563b80 fpcurthread = none idlethread = 0xfffff800075659a0: tid 100007 "idle: cpu3" curpmap = 0xffffffff81cff580 tssp = 0xffffffff81d35448 commontssp = 0xffffffff81d35448 rsp0 = 0xfffffe0785563b80 gs32p = 0xffffffff81d3bca0 ldt = 0xffffffff81d3bce0 tss = 0xffffffff81d3bcd0 spin locks held: cpuid = 4 dynamic pcpu = 0xfffffe0879e57f80 curthread = 0xfffff80012143000: pid 6820 "kinfo" curpcb = 0xfffffe081d2acb80 fpcurthread = none idlethread = 0xfffff800075654d0: tid 100008 "idle: cpu4" curpmap = 0xfffff8008f862138 tssp = 0xffffffff81d354b0 commontssp = 0xffffffff81d354b0 rsp0 = 0xfffffe081d2acb80 gs32p = 0xffffffff81d3bd08 ldt = 0xffffffff81d3bd48 tss = 0xffffffff81d3bd38 spin locks held: cpuid = 5 dynamic pcpu = 0xfffffe0879e5ff80 curthread = 0xfffff8071e681000: pid 6827 "kinfo" curpcb = 0xfffffe081e072b80 fpcurthread = none idlethread = 0xfffff80007565000: tid 100009 "idle: cpu5" curpmap = 0xfffff8016eb7a138 tssp = 0xffffffff81d35518 commontssp = 0xffffffff81d35518 rsp0 = 0xfffffe081e072b80 gs32p = 0xffffffff81d3bd70 ldt = 0xffffffff81d3bdb0 tss = 0xffffffff81d3bda0 spin locks held: cpuid = 6 dynamic pcpu = 0xfffffe0879e67f80 curthread = 0xfffff8008f997000: pid 70023 "rename" curpcb = 0xfffffe081dbf8b80 fpcurthread = none idlethread = 0xfffff8000757c9a0: tid 100010 "idle: cpu6" curpmap = 0xfffff8007c5b2138 tssp = 0xffffffff81d35580 commontssp = 0xffffffff81d35580 rsp0 = 0xfffffe081dbf8b80 gs32p = 0xffffffff81d3bdd8 ldt = 0xffffffff81d3be18 tss = 0xffffffff81d3be08 spin locks held: cpuid = 7 dynamic pcpu = 0xfffffe0879e6ff80 curthread = 0xfffff8000757c4d0: pid 11 "idle: cpu7" curpcb = 0xfffffe0785577b80 fpcurthread = none idlethread = 0xfffff8000757c4d0: tid 100011 "idle: cpu7" curpmap = 0xffffffff81cff580 tssp = 0xffffffff81d355e8 commontssp = 0xffffffff81d355e8 rsp0 = 0xfffffe0785577b80 gs32p = 0xffffffff81d3be40 ldt = 0xffffffff81d3be80 tss = 0xffffffff81d3be70 spin locks held: cpuid = 8 dynamic pcpu = 0xfffffe0879e77f80 curthread = 0xfffff8008faab4d0: pid 69972 "link" curpcb = 0xfffffe081dc13b80 fpcurthread = 0xfffff8008faab4d0: pid 69972 "link" idlethread = 0xfffff8000757c000: tid 100012 "idle: cpu8" curpmap = 0xfffff8016ee84138 tssp = 0xffffffff81d35650 commontssp = 0xffffffff81d35650 rsp0 = 0xfffffe081dc13b80 gs32p = 0xffffffff81d3bea8 ldt = 0xffffffff81d3bee8 tss = 0xffffffff81d3bed8 spin locks held: cpuid = 9 dynamic pcpu = 0xfffffe0879e7ff80 curthread = 0xfffff8000757b9a0: pid 11 "idle: cpu9" curpcb = 0xfffffe0785581b80 fpcurthread = none idlethread = 0xfffff8000757b9a0: tid 100013 "idle: cpu9" curpmap = 0xffffffff81cff580 tssp = 0xffffffff81d356b8 commontssp = 0xffffffff81d356b8 rsp0 = 0xfffffe0785581b80 gs32p = 0xffffffff81d3bf10 ldt = 0xffffffff81d3bf50 tss = 0xffffffff81d3bf40 spin locks held: cpuid = 10 dynamic pcpu = 0xfffffe0879e87f80 curthread = 0xfffff8000757b4d0: pid 11 "idle: cpu10" curpcb = 0xfffffe0785586b80 fpcurthread = none idlethread = 0xfffff8000757b4d0: tid 100014 "idle: cpu10" curpmap = 0xffffffff81cff580 tssp = 0xffffffff81d35720 commontssp = 0xffffffff81d35720 rsp0 = 0xfffffe0785586b80 gs32p = 0xffffffff81d3bf78 ldt = 0xffffffff81d3bfb8 tss = 0xffffffff81d3bfa8 spin locks held: cpuid = 11 dynamic pcpu = 0xfffffe0879e8ff80 curthread = 0xfffff800129b04d0: pid 6824 "kinfo" curpcb = 0xfffffe081d4a2b80 fpcurthread = none idlethread = 0xfffff8000757b000: tid 100015 "idle: cpu11" curpmap = 0xfffff802492cc138 tssp = 0xffffffff81d35788 commontssp = 0xffffffff81d35788 rsp0 = 0xfffffe081d4a2b80 gs32p = 0xffffffff81d3bfe0 ldt = 0xffffffff81d3c020 tss = 0xffffffff81d3c010 spin locks held: cpuid = 12 dynamic pcpu = 0xfffffe0879e97f80 curthread = 0xfffff800126a44d0: pid 70126 "fts" curpcb = 0xfffffe081d8bdb80 fpcurthread = none idlethread = 0xfffff8000757a9a0: tid 100016 "idle: cpu12" curpmap = 0xfffff8008fb10138 tssp = 0xffffffff81d357f0 commontssp = 0xffffffff81d357f0 rsp0 = 0xfffffe081d8bdb80 gs32p = 0xffffffff81d3c048 ldt = 0xffffffff81d3c088 tss = 0xffffffff81d3c078 spin locks held: cpuid = 13 dynamic pcpu = 0xfffffe0879e9ff80 curthread = 0xfffff80057b14000: pid 6829 "kinfo" curpcb = 0xfffffe081db8ab80 fpcurthread = none idlethread = 0xfffff8000757a4d0: tid 100017 "idle: cpu13" curpmap = 0xfffff8016e515138 tssp = 0xffffffff81d35858 commontssp = 0xffffffff81d35858 rsp0 = 0xfffffe081db8ab80 gs32p = 0xffffffff81d3c0b0 ldt = 0xffffffff81d3c0f0 tss = 0xffffffff81d3c0e0 spin locks held: cpuid = 14 dynamic pcpu = 0xfffffe0879ea7f80 curthread = 0xfffff8002b93d000: pid 69961 "link" curpcb = 0xfffffe081db2bb80 fpcurthread = 0xfffff8002b93d000: pid 69961 "link" idlethread = 0xfffff8000757a000: tid 100018 "idle: cpu14" curpmap = 0xfffff804a8712138 tssp = 0xffffffff81d358c0 commontssp = 0xffffffff81d358c0 rsp0 = 0xfffffe081db2bb80 gs32p = 0xffffffff81d3c118 ldt = 0xffffffff81d3c158 tss = 0xffffffff81d3c148 spin locks held: cpuid = 15 dynamic pcpu = 0xfffffe0879eaff80 curthread = 0xfffff8000757f9a0: pid 11 "idle: cpu15" curpcb = 0xfffffe078559fb80 fpcurthread = none idlethread = 0xfffff8000757f9a0: tid 100019 "idle: cpu15" curpmap = 0xffffffff81cff580 tssp = 0xffffffff81d35928 commontssp = 0xffffffff81d35928 rsp0 = 0xfffffe078559fb80 gs32p = 0xffffffff81d3c180 ldt = 0xffffffff81d3c1c0 tss = 0xffffffff81d3c1b0 spin locks held: cpuid = 16 dynamic pcpu = 0xfffffe0879eb7f80 curthread = 0xfffff8000757f4d0: pid 11 "idle: cpu16" curpcb = 0xfffffe07855a4b80 fpcurthread = none idlethread = 0xfffff8000757f4d0: tid 100020 "idle: cpu16" curpmap = 0xffffffff81cff580 tssp = 0xffffffff81d35990 commontssp = 0xffffffff81d35990 rsp0 = 0xfffffe07855a4b80 gs32p = 0xffffffff81d3c1e8 ldt = 0xffffffff81d3c228 tss = 0xffffffff81d3c218 spin locks held: cpuid = 17 dynamic pcpu = 0xfffffe0879ebff80 curthread = 0xfffff8010f2c99a0: pid 69957 "rw" curpcb = 0xfffffe081dd1cb80 fpcurthread = none idlethread = 0xfffff8000757f000: tid 100021 "idle: cpu17" curpmap = 0xfffff80249296138 tssp = 0xffffffff81d359f8 commontssp = 0xffffffff81d359f8 rsp0 = 0xfffffe081dd1cb80 gs32p = 0xffffffff81d3c250 ldt = 0xffffffff81d3c290 tss = 0xffffffff81d3c280 spin locks held: cpuid = 18 dynamic pcpu = 0xfffffe0879ec7f80 curthread = 0xfffff8008fa60000: pid 69985 "rename" curpcb = 0xfffffe081dc36b80 fpcurthread = none idlethread = 0xfffff8000757e9a0: tid 100022 "idle: cpu18" curpmap = 0xfffff8008fd34138 tssp = 0xffffffff81d35a60 commontssp = 0xffffffff81d35a60 rsp0 = 0xfffffe081dc36b80 gs32p = 0xffffffff81d3c2b8 ldt = 0xffffffff81d3c2f8 tss = 0xffffffff81d3c2e8 spin locks held: cpuid = 19 dynamic pcpu = 0xfffffe0879ecff80 curthread = 0xfffff8000757e4d0: pid 11 "idle: cpu19" curpcb = 0xfffffe07855b3b80 fpcurthread = none idlethread = 0xfffff8000757e4d0: tid 100023 "idle: cpu19" curpmap = 0xffffffff81cff580 tssp = 0xffffffff81d35ac8 commontssp = 0xffffffff81d35ac8 rsp0 = 0xfffffe07855b3b80 gs32p = 0xffffffff81d3c320 ldt = 0xffffffff81d3c360 tss = 0xffffffff81d3c350 spin locks held: cpuid = 20 dynamic pcpu = 0xfffffe0879ed7f80 curthread = 0xfffff8000757e000: pid 11 "idle: cpu20" curpcb = 0xfffffe07855b8b80 fpcurthread = none idlethread = 0xfffff8000757e000: tid 100024 "idle: cpu20" curpmap = 0xffffffff81cff580 tssp = 0xffffffff81d35b30 commontssp = 0xffffffff81d35b30 rsp0 = 0xfffffe07855b8b80 gs32p = 0xffffffff81d3c388 ldt = 0xffffffff81d3c3c8 tss = 0xffffffff81d3c3b8 spin locks held: cpuid = 21 dynamic pcpu = 0xfffffe0879edff80 curthread = 0xfffff80480b7c9a0: pid 69948 "rw" curpcb = 0xfffffe081ddc6b80 fpcurthread = none idlethread = 0xfffff8000757d9a0: tid 100025 "idle: cpu21" curpmap = 0xfffff8001e26a138 tssp = 0xffffffff81d35b98 commontssp = 0xffffffff81d35b98 rsp0 = 0xfffffe081ddc6b80 gs32p = 0xffffffff81d3c3f0 ldt = 0xffffffff81d3c430 tss = 0xffffffff81d3c420 spin locks held: cpuid = 22 dynamic pcpu = 0xfffffe0879ee7f80 curthread = 0xfffff8008fa609a0: pid 69963 "rw" curpcb = 0xfffffe081dc2cb80 fpcurthread = none idlethread = 0xfffff8000757d4d0: tid 100026 "idle: cpu22" curpmap = 0xfffff8016e644138 tssp = 0xffffffff81d35c00 commontssp = 0xffffffff81d35c00 rsp0 = 0xfffffe081dc2cb80 gs32p = 0xffffffff81d3c458 ldt = 0xffffffff81d3c498 tss = 0xffffffff81d3c488 spin locks held: cpuid = 23 dynamic pcpu = 0xfffffe0803c2bf80 curthread = 0xfffff8000757d000: pid 11 "idle: cpu23" curpcb = 0xfffffe07855c7b80 fpcurthread = none idlethread = 0xfffff8000757d000: tid 100027 "idle: cpu23" curpmap = 0xffffffff81cff580 tssp = 0xffffffff81d35c68 commontssp = 0xffffffff81d35c68 rsp0 = 0xfffffe07855c7b80 gs32p = 0xffffffff81d3c4c0 ldt = 0xffffffff81d3c500 tss = 0xffffffff81d3c4f0 spin locks held: db:0:allpcpu> show alllocks Process 22272 (sed) thread 0xfffff8008f19f9a0 (100509) exclusive lockmgr ufs (ufs) r = 0 (0xfffff80012adf7b8) locked @ kern/vfs_vnops.c:442 Process 70126 (fts) thread 0xfffff800126a44d0 (100415) shared lockmgr ufs (ufs) r = 0 (0xfffff805f154ec98) locked @ kern/vfs_subr.c:2476 shared lockmgr ufs (ufs) r = 0 (0xfffff805839702d8) locked @ /usr/src/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:711 Process 70023 (rename) thread 0xfffff8008f997000 (100581) exclusive sleep mutex vnode interlock (vnode interlock) r = 0 (0xfffff801b5d88af0) locked @ /usr/src/sys/modules/nullfs/../../fs/nullfs/null_subr.c:322 exclusive sleep mutex vnode interlock (vnode interlock) r = 0 (0xfffff800a4452880) locked @ kern/vfs_subr.c:2583 Process 69985 (rename) thread 0xfffff8008fa60000 (100593) exclusive sleep mutex vnode interlock (vnode interlock) r = 0 (0xfffff80173e7cd60) locked @ /usr/src/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:965 exclusive lockmgr ufs (ufs) r = 0 (0xfffff800798e6c98) locked @ /usr/src/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:711 Process 69972 (link) thread 0xfffff8008faab4d0 (100586) shared sx dirhash (dirhash) r = 0 (0xfffff8046286ec00) locked @ ufs/ufs/ufs_dirhash.c:233 exclusive lockmgr ufs (ufs) r = 0 (0xfffff8003d570c98) locked @ /usr/src/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:711 Process 69963 (rw) thread 0xfffff8008fa609a0 (100591) exclusive sleep mutex vnode interlock (vnode interlock) r = 0 (0xfffff80187e6aaf0) locked @ /usr/src/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:965 exclusive lockmgr ufs (ufs) r = 0 (0xfffff801319f6548) locked @ /usr/src/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:711 Process 69961 (link) thread 0xfffff8002b93d000 (100540) exclusive sleep mutex vnode interlock (vnode interlock) r = 0 (0xfffff804c37a4130) locked @ /usr/src/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:640 shared lockmgr ufs (ufs) r = 0 (0xfffff802c46e07b8) locked @ /usr/src/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:711 Process 69957 (rw) thread 0xfffff8010f2c99a0 (100639) exclusive sleep mutex vm page (vm page) r = 0 (0xffffffff81d32600) locked @ kern/vfs_bio.c:2542 exclusive rw vm object (vm object) r = 0 (0xfffff8010f9f9a00) locked @ kern/vfs_bio.c:2603 exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe07882fe0d8) locked @ ufs/ffs/ffs_softdep.c:7110 exclusive lockmgr null (null) r = 0 (0xfffff8015a761068) locked @ /usr/src/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:836 exclusive lockmgr ufs (ufs) r = 0 (0xfffff807672d3a28) locked @ kern/vfs_subr.c:2476 Process 69948 (rw) thread 0xfffff80480b7c9a0 (100673) exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe0785cb2ab0) locked @ kern/vfs_bio.c:3488 exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe07874f18c8) locked @ kern/vfs_bio.c:3488 exclusive lockmgr ufs (ufs) r = 0 (0xfffff80189fbe7b8) locked @ /usr/src/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:711 Process 6829 (kinfo) thread 0xfffff80057b14000 (100559) exclusive sleep mutex pmap (pmap) r = 0 (0xfffff8016e5a6138) locked @ amd64/amd64/pmap.c:4865 exclusive sleep mutex pmap (pmap) r = 0 (0xfffff8016e515138) locked @ amd64/amd64/pmap.c:4864 shared rw pmap pv global (pmap pv global) r = 0 (0xffffffff81c8e680) locked @ amd64/amd64/pmap.c:4859 exclusive sx vm map (user) (vm map (user)) r = 0 (0xfffff8016e5a6080) locked @ vm/vm_map.c:3315 exclusive sx vm map (user) (vm map (user)) r = 0 (0xfffff8016e515080) locked @ vm/vm_map.c:3311 Process 6827 (kinfo) thread 0xfffff8071e681000 (100809) exclusive sx vm map (user) (vm map (user)) r = 0 (0xfffff8008fd14080) locked @ vm/vm_map.c:3315 exclusive sx vm map (user) (vm map (user)) r = 0 (0xfffff8016eb7a080) locked @ vm/vm_map.c:3311 Process 6824 (kinfo) thread 0xfffff800129b04d0 (100210) exclusive sleep mutex pmap (pmap) r = 0 (0xfffff8008fc96138) locked @ amd64/amd64/pmap.c:4861 shared rw pmap pv global (pmap pv global) r = 0 (0xffffffff81c8e680) locked @ amd64/amd64/pmap.c:4859 exclusive sx vm map (user) (vm map (user)) r = 0 (0xfffff8008fc96080) locked @ vm/vm_map.c:3315 exclusive sx vm map (user) (vm map (user)) r = 0 (0xfffff802492cc080) locked @ vm/vm_map.c:3311 Process 6820 (kinfo) thread 0xfffff80012143000 (100170) exclusive sx vm map (user) (vm map (user)) r = 0 (0xfffff8001e0cd080) locked @ vm/vm_map.c:3315 exclusive sx vm map (user) (vm map (user)) r = 0 (0xfffff8008f862080) locked @ vm/vm_map.c:3311 Process 6817 (kinfo) thread 0xfffff8001e1604d0 (100456) exclusive sx sysctl mem (sysctl mem) r = 0 (0xffffffff81aee880) locked @ kern/kern_sysctl.c:1936 Process 618 (syslogd) thread 0xfffff800126a49a0 (100215) exclusive lockmgr bufwait (bufwait) r = 0 (0xfffffe0785668d88) locked @ kern/vfs_bio.c:3488 exclusive lockmgr ufs (ufs) r = 0 (0xfffff805eb818548) locked @ kern/vfs_syscalls.c:3401 db:0:alllocks> show lockedvnods Locked vnodes 0xfffff80012adf750: tag ufs, type VREG usecount 0, writecount 0, refcount 3 mountedhere 0 flags (VI_ACTIVE|VI_DOINGINACT) v_object 0xfffff80012ad1c00 ref 0 pages 1 cleanbuf 1 dirtybuf 0 lock type ufs: EXCL by thread 0xfffff8008f19f9a0 (pid 22272, sed, tid 100509) #0 0xffffffff80a036b9 at __lockmgr_args+0xcd9 #1 0xffffffff80ce6610 at ffs_lock+0xb0 #2 0xffffffff80fe64f0 at VOP_LOCK1_APV+0x100 #3 0xffffffff80b0c853 at VOP_LOCK1+0x53 #4 0xffffffff80b0a741 at _vn_lock+0xc1 #5 0xffffffff80b0aaa9 at vn_close+0xa9 #6 0xffffffff80b08cd4 at vn_closefile+0x74 #7 0xffffffff809d905a at _fdrop+0x1a #8 0xffffffff809dc1a1 at closef+0x1e1 #9 0xffffffff809d962f at closefp+0x9f #10 0xffffffff80e97176 at amd64_syscall+0x2f6 #11 0xffffffff80e76cdb at Xfast_syscall+0xfb ino 43980667, on dev da0p2 0xfffff805eb8184e0: tag ufs, type VREG usecount 2, writecount 1, refcount 6 mountedhere 0 flags (VI_ACTIVE) v_object 0xfffff8008c16b600 ref 0 pages 1 cleanbuf 1 dirtybuf 0 lock type ufs: EXCL by thread 0xfffff800126a49a0 (pid 618, syslogd, tid 100215) with exclusive waiters pending #0 0xffffffff80a03a22 at __lockmgr_args+0x1042 #1 0xffffffff80ce6610 at ffs_lock+0xb0 #2 0xffffffff80fe64f0 at VOP_LOCK1_APV+0x100 #3 0xffffffff80b0c853 at VOP_LOCK1+0x53 #4 0xffffffff80b0a741 at _vn_lock+0xc1 #5 0xffffffff80b04f48 at sys_fsync+0x198 #6 0xffffffff80e97176 at amd64_syscall+0x2f6 #7 0xffffffff80e76cdb at Xfast_syscall+0xfb ino 43980583, on dev da0p2 0xfffff800798e6c30: tag ufs, type VDIR usecount 24, writecount 0, refcount 55 mountedhere 0 flags (VI_ACTIVE) v_object 0xfffff80735547e00 ref 0 pages 104 cleanbuf 1 dirtybuf 13 lock type ufs: EXCL by thread 0xfffff8008fa60000 (pid 69985, rename, tid 100593) with shared waiters pending #0 0xffffffff80a03a22 at __lockmgr_args+0x1042 #1 0xffffffff80ce6610 at ffs_lock+0xb0 #2 0xffffffff80fe64f0 at VOP_LOCK1_APV+0x100 #3 0xffffffff82226c19 at null_lock+0x1e9 #4 0xffffffff80fe64f0 at VOP_LOCK1_APV+0x100 #5 0xffffffff80b0c853 at VOP_LOCK1+0x53 #6 0xffffffff80b0a741 at _vn_lock+0xc1 #7 0xffffffff80adb976 at lookup+0x976 #8 0xffffffff80ada972 at namei+0x9e2 #9 0xffffffff80b054e5 at kern_renameat+0x345 #10 0xffffffff80b05197 at sys_rename+0x47 #11 0xffffffff80e97176 at amd64_syscall+0x2f6 #12 0xffffffff80e76cdb at Xfast_syscall+0xfb ino 3210240, on dev da0p3 0xfffff802c46e0750: tag ufs, type VDIR usecount 1, writecount 0, refcount 28 mountedhere 0 flags (VI_ACTIVE) v_object 0xfffff8054cd96000 ref 0 pages 192 cleanbuf 19 dirtybuf 6 lock type ufs: SHARED (count 1) #0 0xffffffff80a036b9 at __lockmgr_args+0xcd9 #1 0xffffffff80ce6610 at ffs_lock+0xb0 #2 0xffffffff80fe64f0 at VOP_LOCK1_APV+0x100 #3 0xffffffff82226c19 at null_lock+0x1e9 #4 0xffffffff80fe64f0 at VOP_LOCK1_APV+0x100 #5 0xffffffff80b0c853 at VOP_LOCK1+0x53 #6 0xffffffff80b0a741 at _vn_lock+0xc1 #7 0xffffffff80adb1b3 at lookup+0x1b3 #8 0xffffffff80ada972 at namei+0x9e2 #9 0xffffffff80b00207 at kern_unlinkat+0x147 #10 0xffffffff80b000b8 at sys_unlink+0x38 #11 0xffffffff80e97176 at amd64_syscall+0x2f6 #12 0xffffffff80e76cdb at Xfast_syscall+0xfb ino 3210244, on dev da0p3 0xfffff80583970270: tag ufs, type VDIR usecount 1, writecount 0, refcount 38 mountedhere 0 flags (VI_ACTIVE) v_object 0xfffff801b37e4a00 ref 0 pages 129 cleanbuf 15 dirtybuf 2 lock type ufs: SHARED (count 1) with exclusive waiters pending with shared waiters pending #0 0xffffffff80a036b9 at __lockmgr_args+0xcd9 #1 0xffffffff80ce6610 at ffs_lock+0xb0 #2 0xffffffff80fe64f0 at VOP_LOCK1_APV+0x100 #3 0xffffffff82226c19 at null_lock+0x1e9 #4 0xffffffff80fe64f0 at VOP_LOCK1_APV+0x100 #5 0xffffffff80b0c853 at VOP_LOCK1+0x53 #6 0xffffffff80b0a741 at _vn_lock+0xc1 #7 0xffffffff80adb1b3 at lookup+0x1b3 #8 0xffffffff80ada972 at namei+0x9e2 #9 0xffffffff80b013bd at kern_statat+0x16d #10 0xffffffff80b016c1 at sys_fstatat+0x71 #11 0xffffffff80e97176 at amd64_syscall+0x2f6 #12 0xffffffff80e76cdb at Xfast_syscall+0xfb ino 3210248, on dev da0p3 0xfffff8003d570c30: tag ufs, type VDIR usecount 1, writecount 0, refcount 29 mountedhere 0 flags (VI_ACTIVE) v_object 0xfffff8019308fe00 ref 0 pages 192 cleanbuf 8 dirtybuf 17 lock type ufs: EXCL by thread 0xfffff8008faab4d0 (pid 69972, link, tid 100586) #0 0xffffffff80a036b9 at __lockmgr_args+0xcd9 #1 0xffffffff80ce6610 at ffs_lock+0xb0 #2 0xffffffff80fe64f0 at VOP_LOCK1_APV+0x100 #3 0xffffffff82226c19 at null_lock+0x1e9 #4 0xffffffff80fe64f0 at VOP_LOCK1_APV+0x100 #5 0xffffffff80b0c853 at VOP_LOCK1+0x53 #6 0xffffffff80b0a741 at _vn_lock+0xc1 #7 0xffffffff80adb1b3 at lookup+0x1b3 #8 0xffffffff80ada972 at namei+0x9e2 #9 0xffffffff80b00207 at kern_unlinkat+0x147 #10 0xffffffff80b000b8 at sys_unlink+0x38 #11 0xffffffff80e97176 at amd64_syscall+0x2f6 #12 0xffffffff80e76cdb at Xfast_syscall+0xfb ino 3210250, on dev da0p3 0xfffff805f154ec30: tag ufs, type VREG usecount 1, writecount 0, refcount 1 mountedhere 0 flags (VI_ACTIVE) v_object 0xfffff8008f510900 ref 0 pages 0 cleanbuf 0 dirtybuf 0 lock type ufs: SHARED (count 1) #0 0xffffffff80a036b9 at __lockmgr_args+0xcd9 #1 0xffffffff80ce6610 at ffs_lock+0xb0 #2 0xffffffff80fe64f0 at VOP_LOCK1_APV+0x100 #3 0xffffffff80b0c853 at VOP_LOCK1+0x53 #4 0xffffffff80b0a741 at _vn_lock+0xc1 #5 0xffffffff80aedcfa at vget+0x14a #6 0xffffffff80aca679 at cache_lookup+0xc99 #7 0xffffffff80acc76a at vfs_cache_lookup+0x12a #8 0xffffffff80fe33df at VOP_LOOKUP_APV+0x10f #9 0xffffffff822264f2 at null_lookup+0xf2 #10 0xffffffff80fe33df at VOP_LOOKUP_APV+0x10f #11 0xffffffff80adc66d at VOP_LOOKUP+0x4d #12 0xffffffff80adba17 at lookup+0xa17 #13 0xffffffff80ada972 at namei+0x9e2 #14 0xffffffff80b013bd at kern_statat+0x16d #15 0xffffffff80b016c1 at sys_fstatat+0x71 #16 0xffffffff80e97176 at amd64_syscall+0x2f6 #17 0xffffffff80e76cdb at Xfast_syscall+0xfb ino 3210270, on dev da0p3 0xfffff807672d39c0: tag ufs, type VREG usecount 0, writecount 0, refcount 4159 mountedhere 0 flags (VV_NOSYNC|VI_ACTIVE|VI_DOINGINACT) v_object 0xfffff8010f9f9a00 ref 0 pages 33250 cleanbuf 4157 dirtybuf 0 lock type ufs: EXCL by thread 0xfffff8010f2c99a0 (pid 69957, rw, tid 100639) #0 0xffffffff80a03a22 at __lockmgr_args+0x1042 #1 0xffffffff80ce6610 at ffs_lock+0xb0 #2 0xffffffff80fe64f0 at VOP_LOCK1_APV+0x100 #3 0xffffffff80b0c853 at VOP_LOCK1+0x53 #4 0xffffffff80b0a741 at _vn_lock+0xc1 #5 0xffffffff80aedcfa at vget+0x14a #6 0xffffffff80ad89e1 at vfs_hash_get+0xf1 #7 0xffffffff80ce1c50 at ffs_vgetf+0x40 #8 0xffffffff80cef578 at ufs_lookup_ino+0xbe8 #9 0xffffffff80fe359f at VOP_CACHEDLOOKUP_APV+0x10f #10 0xffffffff80acc86d at VOP_CACHEDLOOKUP+0x4d #11 0xffffffff80acc788 at vfs_cache_lookup+0x148 #12 0xffffffff80fe33df at VOP_LOOKUP_APV+0x10f #13 0xffffffff822264f2 at null_lookup+0xf2 #14 0xffffffff80fe33df at VOP_LOOKUP_APV+0x10f #15 0xffffffff80adc66d at VOP_LOOKUP+0x4d #16 0xffffffff80adba17 at lookup+0xa17 #17 0xffffffff80ada972 at namei+0x9e2 ino 3211622, on dev da0p3 0xfffff80189fbe750: tag ufs, type VREG usecount 1, writecount 1, refcount 6701 mountedhere 0 flags (VI_ACTIVE) v_object 0xfffff8001c11c200 ref 0 pages 53592 cleanbuf 6696 dirtybuf 3 lock type ufs: EXCL by thread 0xfffff80480b7c9a0 (pid 69948, rw, tid 100673) #0 0xffffffff80a03a22 at __lockmgr_args+0x1042 #1 0xffffffff80ce6610 at ffs_lock+0xb0 #2 0xffffffff80fe64f0 at VOP_LOCK1_APV+0x100 #3 0xffffffff82226c19 at null_lock+0x1e9 #4 0xffffffff80fe64f0 at VOP_LOCK1_APV+0x100 #5 0xffffffff80b0c853 at VOP_LOCK1+0x53 #6 0xffffffff80b0a741 at _vn_lock+0xc1 #7 0xffffffff80b0f3a7 at vn_write+0x247 #8 0xffffffff80b086ec at vn_io_fault+0x22c #9 0xffffffff80a89c8a at dofilewrite+0x8a #10 0xffffffff80a89998 at kern_writev+0x68 #11 0xffffffff80a89920 at sys_write+0x60 #12 0xffffffff80e97176 at amd64_syscall+0x2f6 #13 0xffffffff80e76cdb at Xfast_syscall+0xfb ino 3212253, on dev da0p3 0xfffff801319f64e0: tag ufs, type VREG usecount 1, writecount 1, refcount 2692 mountedhere 0 flags (VI_ACTIVE) v_object 0xfffff80189e83900 ref 0 pages 21520 cleanbuf 2686 dirtybuf 4 lock type ufs: EXCL by thread 0xfffff8008fa609a0 (pid 69963, rw, tid 100591) #0 0xffffffff80a03a22 at __lockmgr_args+0x1042 #1 0xffffffff80ce6610 at ffs_lock+0xb0 #2 0xffffffff80fe64f0 at VOP_LOCK1_APV+0x100 #3 0xffffffff82226c19 at null_lock+0x1e9 #4 0xffffffff80fe64f0 at VOP_LOCK1_APV+0x100 #5 0xffffffff80b0c853 at VOP_LOCK1+0x53 #6 0xffffffff80b0a741 at _vn_lock+0xc1 #7 0xffffffff80b0f3a7 at vn_write+0x247 #8 0xffffffff80b086ec at vn_io_fault+0x22c #9 0xffffffff80a89c8a at dofilewrite+0x8a #10 0xffffffff80a89998 at kern_writev+0x68 #11 0xffffffff80a89920 at sys_write+0x60 #12 0xffffffff80e97176 at amd64_syscall+0x2f6 #13 0xffffffff80e76cdb at Xfast_syscall+0xfb ino 3212561, on dev da0p3 Copyright(c) 2009 - 2012 Intel Corporation.All rights reserved. Version 2.00.1201.Copyright(c) 2010 - 2012 American Megatrends,Inc. Installed BIOS: SE5C600.86B.01.08.0003 Platform ID: W2600CR