Generic HEAD from 29 15:54 UTC, vmcore.12
GDB: no debug ports present
KDB: debugger backends: ddb
KDB: current backend: ddb
786432K of memory above 4GB ignored
Copyright (c) 1992-2007 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 8.0-CURRENT #0: Fri Nov 30 16:08:00 UTC 2007
test@hydra2.netperf.freebsd.org:/m2/src/sys/i386/compile/PHO
WARNING: WITNESS option enabled, expect reduced performance.
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Xeon(R) CPU E5320 @ 1.86GHz (1866.75-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0x6f7 Stepping = 7
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x4e3bd<SSE3,RSVD2,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA>
AMD Features=0x20100000<NX,LM>
AMD Features2=0x1<LAHF>
Cores per package: 4
real memory = 3489005568 (3327 MB)
avail memory = 3409268736 (3251 MB)
MPTable: <INTEL Bridge CRB >
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
cpu0 (BSP): APIC ID: 0
cpu1 (AP): APIC ID: 1
cpu2 (AP): APIC ID: 2
cpu3 (AP): APIC ID: 3
cpu4 (AP): APIC ID: 4
cpu5 (AP): APIC ID: 5
cpu6 (AP): APIC ID: 6
cpu7 (AP): APIC ID: 7
ioapic0: Assuming intbase of 0
ioapic1: Assuming intbase of 24
:
Trying to mount root from nfs:
NFS ROOT: 192.168.5.1:/zoo/hydra2
em0: link state changed to UP
Interface em0 IP-Address 192.168.5.111 Broadcast 192.168.5.255
Loading configuration files.
No suitable dump device was found.
Entropy harvesting: interrupts ethernet point_to_point kickstart.
Starting file system checks:
mount_nfs: can't update /var/db/mounttab for 192.168.5.1:/zoo/hydra2
Setting hostuuid: 7634a964-b127-0430-c299-0030488f5548.
Setting hostid: 0x20c74c48.
Mounting local file systems:.
Setting hostname: hydra2.netperf.freebsd.org.
net.inet6.ip6.auto_linklocal: 1 -> 0
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
inet6 ::1 prefixlen 128
inet 127.0.0.1 netmask 0xff000000
Additional routing options:.
Starting devd.
em1: link state changed to UP
Additional IP options:.
Mounting NFS file systems:.
Creating and/or trimming log files:.
Starting syslogd.
/etc/rc: WARNING: Dump device does not exist. Savecore not run.
ELF ldconfig path: /lib /usr/lib /usr/lib/compat
a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout
Initial i386 initialization:.
Additional ABI support:.
Starting rpcbind.
NFS access cache time=60
Clearing /tmp (X related).
Starting statd.
Starting lockd.
Starting local daemons:.
Updating motd.
Mounting late file systems:.
Configuring syscons: blanktime.
Starting sshd.
Starting cron.
Local package initialization:.
Starting background file system checks in 60 seconds.
Fri Nov 30 18:01:54 UTC 2007
FreeBSD/i386 (hydra2.netperf.freebsd.org) (ttyd0)
login: Nov 30 18:02:09 hydra2 su: test to root on /dev/ttyp0
test
Password:
Last login: Fri Nov 30 18:02:06 from 192.168.5.1
Copyright (c) 1992-2007 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD 8.0-CURRENT (PHO) #0: Fri Nov 30 16:08:00 UTC 2007
Welcome to FreeBSD!
Before seeking technical support, please use the following resources:
o Security advisories and updated errata information for all releases are
at http://www.FreeBSD.org/releases/ - always consult the ERRATA section
for your release first as it's updated frequently.
o The Handbook and FAQ documents are at http://www.FreeBSD.org/ and,
along with the mailing lists, can be searched by going to
http://www.FreeBSD.org/search/. If the doc distribution has
been installed, they're also available formatted in /usr/share/doc.
If you still have a question or problem, please take the output of
`uname -a', along with any relevant error messages, and email it
as a question to the questions@FreeBSD.org mailing list. If you are
unfamiliar with FreeBSD's directory layout, please refer to the hier(7)
manual page. If you are not familiar with manual pages, type `man man'.
You may also use sysinstall(8) to re-enter the installation and
configuration utility. Edit /etc/motd to change this login announcement.
To see the output from when your computer started, run dmesg(8). If it has
been replaced with other messages, look at /var/run/dmesg.boot.
-- Francisco Reyes <lists@natserv.com>
> pho/keepalive.sh
Fri Nov 30 19:05:05 CET 2007 7:05PM up 3 mins, 2 users, load averages: 40.48, 12.90, 4.93
Limiting icmp unreach response from 5104 to 200 packets/sec
Limiting icmp unreach response from 5175 to 200 packets/sec
Limiting icmp unreach response from 4083 to 200 packets/sec
Limiting icmp unreach response from 5446 to 200 packets/sec
Limiting icmp unreach response from 4397 to 200 packets/sec
Limiting icmp unreach response from 2511 to 200 packets/sec
Limiting icmp unreach response from 2973 to 200 packets/sec
Limiting icmp unreach response from 2608 to 200 packets/sec
Limiting icmp unreach response from 2388 to 200 packets/sec
Limiting icmp unreach response from 2820 to 200 packets/sec
Limiting icmp unreach response from 1261 to 200 packets/sec
Nov 30 18:06:49 hydra2 kernel: pid 912 (rw), uid 1001 inumber 2154 on /tmp: filesystem full
Nov 30 18:06:49 hydra2 kernel: pid 907 (rw), uid 1001 inumber 2144 on /tmp: filesystem full
Limiting icmp unreach response from 393 to 200 packets/sec
Nov 30 18:06:50 hydra2 kernel: pid 1031 (badcode), uid 1001 inumber 1411 on /tmp: filesystem full
Limiting icmp unreach response from 3939 to 200 packets/sec
Limiting icmp unreach response from 3092 to 200 packets/sec
Limiting icmp unreach response from 577 to 200 packets/sec
:
Limiting icmp unreach response from 1313 to 200 packets/sec
Limiting icmp unreach response from 1078 to 200 packets/sec
Limiting icmp unreach response from 442 to 200 packets/sec
Limiting icmp unreach response from 1552 to 200 packets/sec
Limiting icmp unreach response from 1468 to 200 packets/sec
Limiting icmp unreach response from 990 to 200 packets/sec
Limiting icmp unreach response from 1725 to 200 packets/sec
Fri Nov 30 19:20:16 CET 2007 7:20PM up 19 mins, 2 users, load averages: 5.79, 11.00, 13.24
Fri Nov 30 19:35:18 CET 2007 7:35PM up 34 mins, 2 users, load averages: 9.85, 13.54, 14.08
Nov 30 18:41:50 hydra2 kernel: pid 4834 (lockf2), uid 1001 inumber 1435 on /tmp: filesystem full
Nov 30 18:41:50 hydra2 kernel: pid 4832 (lockf2), uid 1001 inumber 1421 on /tmp: filesystem full
Nov 30 18:41:52 hydra2 kernel: pid 4830 (lockf2), uid 1001 inumber 1436 on /tmp: filesystem full
Nov 30 18:41:52 hydra2 kernel: pid 4841 (lockf2), uid 1001 inumber 1433 on /tmp: filesystem full
Nov 30 18:41:53 hydra2 kernel: pid 4879 (lockf2), uid 1001 inumber 1439 on /tmp: filesystem full
Nov 30 18:41:53 hydra2 kernel: pid 4896 (lockf2), uid 1001 inumber 1446 on /tmp: filesystem full
Nov 30 18:41:54 hydra2 kernel: pid 4878 (lockf2), uid 1001 inumber 1439 on /tmp: filesystem full
Nov 30 18:41:54 hydra2 kernel: pid 4833 (lockf2), uid 1001 inumber 1434 on /tmp: filesystem full
Nov 30 18:41:56 hydra2 kernel: pid 4816 (lockf2), uid 1001 inumber 1439 on /tmp: filesystem full
Nov 30 18:41:56 hydra2 kernel: pid 4811 (lockf2), uid 1001 inumber 1436 on /tmp: filesystem full
Nov 30 18:41:57 hydra2 kernel: pid 4795 (lockf2), uid 1001 inumber 1439 on /tmp: filesystem full
Nov 30 18:41:58 hydra2 kernel: pid 4893 (lockf2), uid 1001 inumber 1444 on /tmp: filesystem full
Nov 30 18:44:36 hydra2 kernel: pid 5113 (lockf2), uid 1001 inumber 1427 on /tmp: filesystem full
Nov 30 18:44:38 hydra2 kernel: pid 5090 (lockf2), uid 1001 inumber 1424 on /tmp: filesystem full
Nov 30 18:44:39 hydra2 kernel: pid 5070 (lockf2), uid 1001 inumber 1425 on /tmp: filesystem full
Nov 30 18:44:41 hydra2 kernel: pid 5092 (lockf2), uid 1001 inumber 1423 on /tmp: filesystem full
panic: ffs_truncate3
KDB: stack backtrace:
cpuid = 3
dKDB: enter: panic
[thread pid 5130 tid 100065 ]
Stopped at kdb_enter+0x32: leave
db> bt
Tracing pid 5130 tid 100065 td 0xc6d04880
kdb_enter(c0ab4b70,3,c0ad3174,e93527c0,3,...) at kdb_enter+0x32
panic(c0ad3174,0,c0ad3124,1e9,0,...) at panic+0x124
ffs_truncate(c8c73000,0,0,c00,0,...) at ffs_truncate+0x1858
ufs_inactive(e9352a38,c0ae6aa9,c8c730d4,c8c730d4,c8c73000,...) at ufs_inactive+0x250
VOP_INACTIVE_APV(c0b86340,e9352a38,c0ac0353,8e3,c0b9d240,...) at VOP_INACTIVE_APV+0xc5
vinactive(c0b86340,e9352a6c,c0ac0353,86f,c8c73000,...) at vinactive+0xb1
vput(c8c73000,e9352aac,c6d04880,c0ac129b,120,...) at vput+0x1f7
vn_close(c8c73000,3,c6ec1a00,c6d04880,c6d04918,...) at vn_close+0xf0
vn_closefile(c6fc3d80,c6d04880,c0ab09dc,87b,6db,...) at vn_closefile+0xf4
fdrop(c6fc3d80,c6d04880,c07991c8,c0c2cbe8,0,c6d04918,c0b6d14c,c0ab09e5,c6c5bc2c,67a,c0ab09dc,e9352ba0,c0767810,c6c5bc2c,8,c0ab09dc,67a) at fdrop+0x11e
closef(c6fc3d80,c6d04880,67a,678,c0798cf2,...) at closef+0x254
fdfree(c6d04880,0,c0ab1279,fd,c6cdbb30,...) at fdfree+0x3a6
exit1(c6d04880,100,e9352d2c,c0a22f63,c6d04880,...) at exit1+0x573
sys_exit(c6d04880,e9352cfc,4,e9352d38,c0b65698,...) at sys_exit+0x1d
syscall(e9352d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (1, FreeBSD ELF32, sys_exit), eip = 0x280e270f, esp = 0xbfbfe4cc, ebp = 0xbfbfe4d8 ---
db> set $lines 0
db> show allpcpu
Current CPU: 3
cpuid = 0
curthread = 0xc6788220: pid 11 "idle: cpu0"
curpcb = 0xe5434d90
fpcurthread = none
idlethread = 0xc6788220: pid 11 "idle: cpu0"
APIC ID = 0
currentldt = 0x50
spin locks held:
cpuid = 1
curthread = 0xc9255660: pid 5060 "lockf2"
curpcb = 0xe9817d90
fpcurthread = none
idlethread = 0xc6788440: pid 11 "idle: cpu1"
APIC ID = 1
currentldt = 0x50
spin locks held:
cpuid = 2
curthread = 0xc739e440: pid 5071 "swap"
curpcb = 0xe94ffd90
fpcurthread = none
idlethread = 0xc677d000: pid 11 "idle: cpu2"
APIC ID = 2
currentldt = 0x50
spin locks held:
cpuid = 3
curthread = 0xc6d04880: pid 5130 "lockf2"
curpcb = 0xe9352d90
fpcurthread = none
idlethread = 0xc677d220: pid 11 "idle: cpu3"
APIC ID = 3
currentldt = 0x50
spin locks held:
cpuid = 4
curthread = 0xc91e7880: pid 5103 "lockf2"
curpcb = 0xe98b0d90
fpcurthread = none
idlethread = 0xc677d440: pid 11 "idle: cpu4"
APIC ID = 4
currentldt = 0x50
spin locks held:
cpuid = 5
curthread = 0xc677d660: pid 11 "idle: cpu5"
curpcb = 0xe5425d90
fpcurthread = none
idlethread = 0xc677d660: pid 11 "idle: cpu5"
APIC ID = 5
currentldt = 0x50
spin locks held:
cpuid = 6
curthread = 0xc677d880: pid 11 "idle: cpu6"
curpcb = 0xe5422d90
fpcurthread = none
idlethread = 0xc677d880: pid 11 "idle: cpu6"
APIC ID = 6
currentldt = 0x50
spin locks held:
cpuid = 7
curthread = 0xc677daa0: pid 11 "idle: cpu7"
curpcb = 0xe541fd90
fpcurthread = none
idlethread = 0xc677daa0: pid 11 "idle: cpu7"
APIC ID = 7
currentldt = 0x50
spin locks held:
db> show allocks^H ^H^H ^H^H ^H^H ^Hlocks
Process 5130 (lockf2) thread 0xc6d04880 (100065)
exclusive sleep mutex vnode interlock r = 0 (0xc8c730d4) locked @ ufs/ffs/ffs_inode.c:489
Process 5103 (lockf2) thread 0xc91e7880 (100384)
exclusive sleep mutex 64 (UMA zone) r = 0 (0xc1474808) locked @ vm/uma_core.c:1830
Process 5075 (swap) thread 0xc920b220 (100307)
exclusive sx user map r = 0 (0xc74d4784) locked @ vm/vm_map.c:3109
Process 5073 (swap) thread 0xc739faa0 (100150)
exclusive sleep mutex process lock r = 0 (0xc738e5e0) locked @ i386/i386/trap.c:757
Process 5071 (swap) thread 0xc739e440 (100160)
exclusive sleep mutex vm object (standard object) r = 0 (0xc8eb564c) locked @ vm/vm_fault.c:293
exclusive sx user map r = 0 (0xca143784) locked @ vm/vm_map.c:3109
Process 5062 (swap) thread 0xc74bf220 (100165)
exclusive sx user map r = 0 (0xc74d486c) locked @ vm/vm_map.c:3109
Process 5060 (lockf2) thread 0xc9255660 (100330)
exclusive sleep mutex vm object (standard object) r = 0 (0xc8ef6d14) locked @ vm/vnode_pager.c:361
Process 655 (sshd) thread 0xc7051880 (100132)
exclusive sx so_rcv_sx r = 0 (0xc6e45530) locked @ kern/uipc_sockbuf.c:145
db> show lockedvnods
Locked vnodes
0xc8cb8d98: tag ufs, type VREG
usecount 1, writecount 1, refcount 36 mountedhere 0
flags ()
v_object 0xc8ef6d14 ref 0 pages 136
lock type ufs: EXCL (count 1) by thread 0xc9255660 (pid 5060)#0 0xc074f470 at _lockmgr+0x560
#1 0xc0951f91 at ffs_lock+0x91
#2 0xc0a30945 at VOP_LOCK1_APV+0xa5
#3 0xc07ea6b8 at _vn_lock+0xf8
#4 0xc07eb57a at vn_write+0x16a
#5 0xc079b015 at dofilewrite+0x95
#6 0xc079b298 at kern_writev+0x58
#7 0xc079b30f at write+0x4f
#8 0xc0a22f63 at syscall+0x2b3
#9 0xc0a092c0 at Xint0x80_syscall+0x20
ino 1422, on dev md1
0xc8e83ae0: tag ufs, type VREG
usecount 1, writecount 1, refcount 20 mountedhere 0
flags ()
v_object 0xc86d3364 ref 0 pages 72
lock type ufs: EXCL (count 1) by thread 0xc9202440 (pid 5104)#0 0xc074f470 at _lockmgr+0x560
#1 0xc0951f91 at ffs_lock+0x91
#2 0xc0a30945 at VOP_LOCK1_APV+0xa5
#3 0xc07ea6b8 at _vn_lock+0xf8
#4 0xc07eb57a at vn_write+0x16a
#5 0xc079b015 at dofilewrite+0x95
#6 0xc079b298 at kern_writev+0x58
#7 0xc079b30f at write+0x4f
#8 0xc0a22f63 at syscall+0x2b3
#9 0xc0a092c0 at Xint0x80_syscall+0x20
ino 1426, on dev md1
0xc7be2570: tag ufs, type VREG
usecount 0, writecount 0, refcount 16 mountedhere 0
flags (VV_NOSYNC|VI_DOINGINACT)
v_object 0xc884c83c ref 0 pages 56
lock type ufs: EXCL (count 1) by thread 0xc92be660 (pid 5122)#0 0xc074f470 at _lockmgr+0x560
#1 0xc0951f91 at ffs_lock+0x91
#2 0xc0a30945 at VOP_LOCK1_APV+0xa5
#3 0xc07ea6b8 at _vn_lock+0xf8
#4 0xc07eaf0a at vn_close+0x8a
#5 0xc07eb084 at vn_closefile+0xf4
#6 0xc0733fbe at fdrop+0x11e
#7 0xc0735504 at closef+0x254
#8 0xc0736256 at fdfree+0x3a6
#9 0xc073f703 at exit1+0x573
#10 0xc074056d at sys_exit+0x1d
#11 0xc0a22f63 at syscall+0x2b3
#12 0xc0a092c0 at Xint0x80_syscall+0x20
ino 1427, on dev md1
0xc8d89570: tag ufs, type VREG
usecount 1, writecount 1, refcount 15 mountedhere 0
flags ()
v_object 0xc8d245d0 ref 0 pages 52
lock type ufs: EXCL (count 1) by thread 0xc9e69220 (pid 5107)#0 0xc074f470 at _lockmgr+0x560
#1 0xc0951f91 at ffs_lock+0x91
#2 0xc0a30945 at VOP_LOCK1_APV+0xa5
#3 0xc07ea6b8 at _vn_lock+0xf8
#4 0xc07eb57a at vn_write+0x16a
#5 0xc079b015 at dofilewrite+0x95
#6 0xc079b298 at kern_writev+0x58
#7 0xc079b30f at write+0x4f
#8 0xc0a22f63 at syscall+0x2b3
#9 0xc0a092c0 at Xint0x80_syscall+0x20
ino 1428, on dev md1
0xc8b6e000: tag ufs, type VREG
usecount 1, writecount 1, refcount 21 mountedhere 0
flags ()
v_object 0xc6fd683c ref 0 pages 76
lock type ufs: EXCL (count 1) by thread 0xc6e79660 (pid 5155)#0 0xc074f470 at _lockmgr+0x560
#1 0xc0951f91 at ffs_lock+0x91
#2 0xc0a30945 at VOP_LOCK1_APV+0xa5
#3 0xc07ea6b8 at _vn_lock+0xf8
#4 0xc07eb57a at vn_write+0x16a
#5 0xc079b015 at dofilewrite+0x95
#6 0xc079b298 at kern_writev+0x58
#7 0xc079b30f at write+0x4f
#8 0xc0a22f63 at syscall+0x2b3
#9 0xc0a092c0 at Xint0x80_syscall+0x20
ino 1430, on dev md1
0xc8c73000: tag ufs, type VREG
usecount 0, writecount 0, refcount 3 mountedhere 0
flags (VV_NOSYNC|VI_DOINGINACT)
VI_LOCKed v_object 0xc8ce0b24 ref 0 pages 4
lock type ufs: EXCL (count 1) by thread 0xc6d04880 (pid 5130)#0 0xc074f470 at _lockmgr+0x560
#1 0xc0951f91 at ffs_lock+0x91
#2 0xc0a30945 at VOP_LOCK1_APV+0xa5
#3 0xc07ea6b8 at _vn_lock+0xf8
#4 0xc07eaf0a at vn_close+0x8a
#5 0xc07eb084 at vn_closefile+0xf4
#6 0xc0733fbe at fdrop+0x11e
#7 0xc0735504 at closef+0x254
#8 0xc0736256 at fdfree+0x3a6
#9 0xc073f703 at exit1+0x573
#10 0xc074056d at sys_exit+0x1d
#11 0xc0a22f63 at syscall+0x2b3
#12 0xc0a092c0 at Xint0x80_syscall+0x20
KDB: stack backtrace:
db_trace_self_wrapper(c0ab79ab,e93523f0,c07db0d3,c0a092c0,c0d54723,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0a092c0,c0d54723,20,20,c0b86860,...) at kdb_backtrace+0x29
vfs_badlock(e9352414,c0a2d97d,c8c73000,c0ae67b0,80,...) at vfs_badlock+0x23
assert_vi_unlocked(c8c73000,c0ae67b0,80,c8c73000,c8c73000,...) at assert_vi_unlocked+0x33
VOP_PRINT_APV(c0b86340,e9352430,0,4,0,...) at VOP_PRINT_APV+0x7d
vn_printf(c8c73000,c0a7e5e6,c0aa5cd7,c0b9cee0,c8c73000,...) at vn_printf+0x4af
lockedvnodes(c0789022,0,ffffffff,e9352598,c048fd00,...) at lockedvnodes+0x71
db_command_loop(c0789022,fffffff3,c0ab37db,7,e98b0a40,...) at db_command_loop+0x2f5
db_trap(3,0,f7,3,c6d04880,...) at db_trap+0xc5
kdb_trap(3,0,e9352740,c6d04880,c6cdbaa0,...) at kdb_trap+0x96
trap(e9352740) at trap+0x56b
calltrap() at calltrap+0x6
--- trap 0x3, eip = 0xc0789022, esp = 0xe9352780, ebp = 0xe9352788 ---
kdb_enter(c0ab4b70,3,c0ad3174,e93527c0,3,...) at kdb_enter+0x32
panic(c0ad3174,0,c0ad3124,1e9,0,...) at panic+0x124
ffs_truncate(c8c73000,0,0,c00,0,...) at ffs_truncate+0x1858
ufs_inactive(e9352a38,c0ae6aa9,c8c730d4,c8c730d4,c8c73000,...) at ufs_inactive+0x250
VOP_INACTIVE_APV(c0b86340,e9352a38,c0ac0353,8e3,c0b9d240,...) at VOP_INACTIVE_APV+0xc5
vinactive(c0b86340,e9352a6c,c0ac0353,86f,c8c73000,...) at vinactive+0xb1
vput(c8c73000,e9352aac,c6d04880,c0ac129b,120,...) at vput+0x1f7
vn_close(c8c73000,3,c6ec1a00,c6d04880,c6d04918,...) at vn_close+0xf0
vn_closefile(c6fc3d80,c6d04880,c0ab09dc,87b,6db,...) at vn_closefile+0xf4
fdrop(c6fc3d80,c6d04880,c07991c8,c0c2cbe8,0,c6d04918,c0b6d14c,c0ab09e5,c6c5bc2c,67a,c0ab09dc,e9352ba0,c0767810,c6c5bc2c,8,c0ab09dc,67a) at fdrop+0x11e
closef(c6fc3d80,c6d04880,67a,678,c0798cf2,...) at closef+0x254
fdfree(c6d04880,0,c0ab1279,fd,c6cdbb30,...) at fdfree+0x3a6
exit1(c6d04880,100,e9352d2c,c0a22f63,c6d04880,...) at exit1+0x573
sys_exit(c6d04880,e9352cfc,4,e9352d38,c0b65698,...) at sys_exit+0x1d
syscall(e9352d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (1, FreeBSD ELF32, sys_exit), eip = 0x280e270f, esp = 0xbfbfe4cc, ebp = 0xbfbfe4d8 ---
VOP_PRINT: 0xc8c73000 interlock is locked but should not be
ino 1431, on dev md1
KDB: stack backtrace:
db_trace_self_wrapper(c0ab79ab,e93523f0,c07db0d3,e93523e8,c89d2dec,...) at db_trace_self_wrapper+0x26
kdb_backtrace(e93523e8,c89d2dec,e93523f8,c095d960,0,...) at kdb_backtrace+0x29
vfs_badlock(e9352414,c0a2d9ec,c8c73000,c0ae67b0,80,...) at vfs_badlock+0x23
assert_vi_unlocked(c8c73000,c0ae67b0,80,c8c73000,c8c73000,...) at assert_vi_unlocked+0x33
VOP_PRINT_APV(c0b86340,e9352430,0,4,0,...) at VOP_PRINT_APV+0xec
vn_printf(c8c73000,c0a7e5e6,c0aa5cd7,c0b9cee0,c8c73000,...) at vn_printf+0x4af
lockedvnodes(c0789022,0,ffffffff,e9352598,c048fd00,...) at lockedvnodes+0x71
db_command_loop(c0789022,fffffff3,c0ab37db,7,e98b0a40,...) at db_command_loop+0x2f5
db_trap(3,0,f7,3,c6d04880,...) at db_trap+0xc5
kdb_trap(3,0,e9352740,c6d04880,c6cdbaa0,...) at kdb_trap+0x96
trap(e9352740) at trap+0x56b
calltrap() at calltrap+0x6
--- trap 0x3, eip = 0xc0789022, esp = 0xe9352780, ebp = 0xe9352788 ---
kdb_enter(c0ab4b70,3,c0ad3174,e93527c0,3,...) at kdb_enter+0x32
panic(c0ad3174,0,c0ad3124,1e9,0,...) at panic+0x124
ffs_truncate(c8c73000,0,0,c00,0,...) at ffs_truncate+0x1858
ufs_inactive(e9352a38,c0ae6aa9,c8c730d4,c8c730d4,c8c73000,...) at ufs_inactive+0x250
VOP_INACTIVE_APV(c0b86340,e9352a38,c0ac0353,8e3,c0b9d240,...) at VOP_INACTIVE_APV+0xc5
vinactive(c0b86340,e9352a6c,c0ac0353,86f,c8c73000,...) at vinactive+0xb1
vput(c8c73000,e9352aac,c6d04880,c0ac129b,120,...) at vput+0x1f7
vn_close(c8c73000,3,c6ec1a00,c6d04880,c6d04918,...) at vn_close+0xf0
vn_closefile(c6fc3d80,c6d04880,c0ab09dc,87b,6db,...) at vn_closefile+0xf4
fdrop(c6fc3d80,c6d04880,c07991c8,c0c2cbe8,0,c6d04918,c0b6d14c,c0ab09e5,c6c5bc2c,67a,c0ab09dc,e9352ba0,c0767810,c6c5bc2c,8,c0ab09dc,67a) at fdrop+0x11e
closef(c6fc3d80,c6d04880,67a,678,c0798cf2,...) at closef+0x254
fdfree(c6d04880,0,c0ab1279,fd,c6cdbb30,...) at fdfree+0x3a6
exit1(c6d04880,100,e9352d2c,c0a22f63,c6d04880,...) at exit1+0x573
sys_exit(c6d04880,e9352cfc,4,e9352d38,c0b65698,...) at sys_exit+0x1d
syscall(e9352d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (1, FreeBSD ELF32, sys_exit), eip = 0x280e270f, esp = 0xbfbfe4cc, ebp = 0xbfbfe4d8 ---
VOP_PRINT: 0xc8c73000 interlock is locked but should not be
0xc8ca1414: tag ufs, type VREG
usecount 0, writecount 0, refcount 1 mountedhere 0
flags (VV_NOSYNC|VI_DOINGINACT)
v_object 0xc8c1945c ref 0 pages 0
lock type ufs: EXCL (count 1) by thread 0xc91eecc0 (pid 5157)#0 0xc074f470 at _lockmgr+0x560
#1 0xc0951f91 at ffs_lock+0x91
#2 0xc0a30945 at VOP_LOCK1_APV+0xa5
#3 0xc07ea6b8 at _vn_lock+0xf8
#4 0xc07eaf0a at vn_close+0x8a
#5 0xc07eb084 at vn_closefile+0xf4
#6 0xc0733fbe at fdrop+0x11e
#7 0xc0735504 at closef+0x254
#8 0xc0736256 at fdfree+0x3a6
#9 0xc073f703 at exit1+0x573
#10 0xc074056d at sys_exit+0x1d
#11 0xc0a22f63 at syscall+0x2b3
#12 0xc0a092c0 at Xint0x80_syscall+0x20
ino 1433, on dev md1
db> ps
pid ppid pgrp uid state wmesg wchan cmd
5157 5052 834 1001 RE+ lockf2
5155 5052 834 1001 R+ lockf2
5151 5052 834 1001 S+ nanslp 0xc0bdc124 lockf2
5150 5052 834 1001 S+ nanslp 0xc0bdc124 lockf2
5149 5052 834 1001 S+ nanslp 0xc0bdc124 lockf2
5148 5052 834 1001 S+ nanslp 0xc0bdc124 lockf2
5144 5052 834 1001 S+ nanslp 0xc0bdc124 lockf2
5143 5052 834 1001 R+ lockf2
5142 5052 834 1001 S+ nanslp 0xc0bdc124 lockf2
5141 5052 834 1001 S+ nanslp 0xc0bdc124 lockf2
5140 5052 834 1001 S+ nanslp 0xc0bdc124 lockf2
5139 5052 834 1001 R+ lockf2
5138 5052 834 1001 R+ lockf2
5136 5052 834 1001 S+ nanslp 0xc0bdc124 lockf2
5134 5052 834 1001 R+ lockf2
5133 5052 834 1001 S+ nanslp 0xc0bdc124 lockf2
5131 5052 834 1001 S+ nanslp 0xc0bdc124 lockf2
5130 5052 834 1001 RE+ CPU 3 lockf2
5129 5052 834 1001 S+ nanslp 0xc0bdc124 lockf2
5128 5052 834 1001 R+ lockf2
5126 5052 834 1001 S+ nanslp 0xc0bdc124 lockf2
5123 5052 834 1001 R+ lockf2
5122 5052 834 1001 SE+ getblk 0xdaa06ef0 lockf2
5118 5052 834 1001 S+ nanslp 0xc0bdc124 lockf2
5117 5052 834 1001 S+ nanslp 0xc0bdc124 lockf2
5116 5052 834 1001 R+ lockf2
5114 5052 834 1001 S+ nanslp 0xc0bdc124 lockf2
5112 5052 834 1001 S+ nanslp 0xc0bdc124 lockf2
5107 5052 834 1001 R+ lockf2
5104 5052 834 1001 R+ lockf2
5103 5052 834 1001 R+ CPU 4 lockf2
5102 5052 834 1001 R+ lockf2
5100 5052 834 1001 S+ nanslp 0xc0bdc124 lockf2
5095 5052 834 1001 R+ lockf2
5094 5052 834 1001 S+ nanslp 0xc0bdc124 lockf2
5089 5052 834 1001 R+ lockf2
5087 5052 834 1001 S+ nanslp 0xc0bdc124 lockf2
5086 5052 834 1001 S+ nanslp 0xc0bdc124 lockf2
5085 5052 834 1001 S+ nanslp 0xc0bdc124 lockf2
5081 5052 834 1001 R+ lockf2
5080 5052 834 1001 R+ lockf2
5078 5052 834 1001 R+ lockf2
5077 5052 834 1001 S+ nanslp 0xc0bdc124 lockf2
5075 5059 834 1001 R+ swap
5073 5059 834 1001 R+ swap
5071 5059 834 1001 R+ CPU 2 swap
5069 5059 834 1001 S+ nanslp 0xc0bdc124 swap
5068 5052 834 1001 R+ lockf2
5067 5059 834 1001 S+ nanslp 0xc0bdc124 swap
5065 5059 834 1001 S+ nanslp 0xc0bdc124 swap
5062 5059 834 1001 R+ swap
5060 5052 834 1001 R+ CPU 1 lockf2
5059 5051 834 1001 S+ wait 0xc6b2f2a8 swap
5058 5052 834 1001 R+ lockf2
5055 5052 834 1001 S+ nanslp 0xc0bdc124 lockf2
5052 5049 834 1001 S+ wait 0xc9310d48 lockf2
5051 4769 834 1001 S+ nanslp 0xc0bdc124 swap
5049 4769 834 1001 S+ nanslp 0xc0bdc124 lockf2
4769 4768 834 1001 S+ wait 0xc9e617f8 run
4768 4767 834 1001 S+ wait 0xca0b77f8 run
4767 4761 834 1001 S+ nanslp 0xc0bdc124 run
4761 834 834 1001 S+ wait 0xc6e5f7f8 sh
4615 807 807 1001 S+ nanslp 0xc0bdc124 sleep
834 659 834 1001 S+ wait 0xc6e687f8 sh
807 800 807 1001 S+ wait 0xc74e5550 sh
800 654 800 1001 S+ pause 0xc6ec2060 tcsh
683 1 683 0 Ss nanslp 0xc0bdc124 watchdogd
659 658 659 1001 Ss+ pause 0xc6fa4308 tcsh
658 655 655 1001 S select 0xc0c2d6bc sshd
655 592 655 0 Ss sbwait 0xc6e45560 sshd
654 1 654 0 Ss+ wait 0xc6fa5000 login
653 1 653 0 Ss+ ttyin 0xc68e5c10 getty
652 1 652 0 Ss+ ttyin 0xc68eb010 getty
651 1 651 0 Ss+ ttyin 0xc68eb410 getty
650 1 650 0 Ss+ ttyin 0xc68eb810 getty
649 1 649 0 Ss+ ttyin 0xc68ebc10 getty
648 1 648 0 Ss+ ttyin 0xc68ec010 getty
647 1 647 0 Ss+ ttyin 0xc68ec410 getty
646 1 646 0 Ss+ ttyin 0xc68ec810 getty
608 1 608 0 Ss nanslp 0xc0bdc124 cron
602 1 602 25 Ss pause 0xc6c93da8 sendmail
598 1 598 0 Ss select 0xc0c2d6bc sendmail
592 1 592 0 Ss select 0xc0c2d6bc sshd
539 533 533 0 S nfslockd 0xc0c33d08 rpc.lockd
533 1 533 0 Ss select 0xc0c2d6bc rpc.lockd
528 1 528 0 Ss select 0xc0c2d6bc rpc.statd
498 0 0 0 SL mdwait 0xc6de1000 [md1]
478 1 478 0 Ss select 0xc0c2d6bc rpcbind
456 1 456 0 Rs syslogd
410 1 410 0 Ss select 0xc0c2d6bc devd
94 0 0 0 SL mdwait 0xc6d56800 [md0]
27 0 0 0 SL sdflush 0xc0c394a4 [softdepflush]
26 0 0 0 SL syncer 0xc0bdbf4c [syncer]
25 0 0 0 SL vlruwt 0xc6af47f8 [vnlru]
24 0 0 0 SL psleep 0xc0c2db64 [bufdaemon]
23 0 0 0 SL pgzero 0xc0c3a060 [pagezero]
22 0 0 0 SL psleep 0xc0c39c78 [vmdaemon]
21 0 0 0 SL psleep 0xc0c39c40 [pagedaemon]
20 0 0 0 SL waiting_ 0xc0c2f8ec [sctp_iterator]
19 0 0 0 SL usbevt 0xc6865210 [usb3]
18 0 0 0 SL usbevt 0xc68be210 [usb2]
17 0 0 0 SL usbevt 0xc68ac210 [usb1]
16 0 0 0 SL usbtsk 0xc0bcc814 [usbtask-dr]
15 0 0 0 SL usbtsk 0xc0bcc800 [usbtask-hc]
14 0 0 0 SL usbevt 0xc6896210 [usb0]
9 0 0 0 SL - 0xc689ce00 [em1 taskq]
8 0 0 0 SL - 0xc687a980 [em0 taskq]
7 0 0 0 SL ccb_scan 0xc0b9d9d4 [xpt_thrd]
6 0 0 0 SL - 0xc67c2700 [kqueue taskq]
5 0 0 0 SL - 0xc67c2980 [thread taskq]
13 0 0 0 SL - 0xc0bdbf54 [yarrow]
4 0 0 0 SL - 0xc0bccf6c [g_down]
3 0 0 0 SL - 0xc0bccf68 [g_up]
2 0 0 0 SL - 0xc0bccf60 [g_event]
12 0 0 0 WL (threaded) intr
100038 I [swi0: sio]
100037 I [irq1: atkbd0]
100036 I [irq15: ata1]
100035 I [irq14: ata0]
100032 I [irq18: uhci2]
100030 I [irq19: uhci1+]
100026 I [irq17: uhci0 ehci0]
100023 I [swi5: +]
100022 I [swi2: cambio]
100019 I [swi6: task queue]
100018 I [swi6: Giant taskq]
100012 I [swi3: vm]
100011 I [swi4: clock sio]
100010 I [swi1: net]
11 0 0 0 RL (threaded) idle
100009 Run CPU 0 [idle: cpu0]
100008 CanRun [idle: cpu1]
100007 CanRun [idle: cpu2]
100006 CanRun [idle: cpu3]
100005 CanRun [idle: cpu4]
100004 Run CPU 5 [idle: cpu5]
100003 Run CPU 6 [idle: cpu6]
100002 Run CPU 7 [idle: cpu7]
1 0 1 0 SLs wait 0xc677bd48 [init]
10 0 0 0 SL audit_wo 0xc0c38f10 [audit]
0 0 0 0 WLs [swapper]
5127 5052 834 1001 Z+ lockf2
5119 5052 834 1001 Z+ lockf2
5076 5052 834 1001 Z+ lockf2
5099 5052 834 1001 Z+ lockf2
5132 5052 834 1001 Z+ lockf2
5124 5052 834 1001 Z+ lockf2
5108 5052 834 1001 Z+ lockf2
5092 5052 834 1001 Z+ lockf2
5072 5052 834 1001 Z+ lockf2
5145 5052 834 1001 Z+ lockf2
5084 5052 834 1001 Z+ lockf2
5121 5052 834 1001 Z+ lockf2
5063 5052 834 1001 Z+ lockf2
5082 5052 834 1001 Z+ lockf2
5137 5052 834 1001 Z+ lockf2
5056 5052 834 1001 Z+ lockf2
5057 5052 834 1001 Z+ lockf2
5110 5052 834 1001 Z+ lockf2
5088 5052 834 1001 Z+ lockf2
5070 5052 834 1001 Z+ lockf2
5064 5052 834 1001 Z+ lockf2
5125 5052 834 1001 Z+ lockf2
5066 5052 834 1001 Z+ lockf2
5061 5052 834 1001 Z+ lockf2
5152 5052 834 1001 Z+ lockf2
5160 5052 834 1001 Z+ lockf2
5105 5052 834 1001 Z+ lockf2
5096 5052 834 1001 Z+ lockf2
5090 5052 834 1001 Z+ lockf2
5115 5052 834 1001 Z+ lockf2
5091 5052 834 1001 Z+ lockf2
5074 5052 834 1001 Z+ lockf2
5093 5052 834 1001 Z+ lockf2
5146 5052 834 1001 Z+ lockf2
5153 5052 834 1001 Z+ lockf2
5083 5052 834 1001 Z+ lockf2
5098 5052 834 1001 Z+ lockf2
5111 5052 834 1001 Z+ lockf2
5101 5052 834 1001 Z+ lockf2
5159 5052 834 1001 Z+ lockf2
5147 5052 834 1001 Z+ lockf2
5156 5052 834 1001 Z+ lockf2
5106 5052 834 1001 Z+ lockf2
5158 5052 834 1001 Z+ lockf2
5109 5052 834 1001 Z+ lockf2
5154 5052 834 1001 Z+ lockf2
5120 5052 834 1001 Z+ lockf2
5113 5052 834 1001 Z+ lockf2
5097 5052 834 1001 Z+ lockf2
5135 5052 834 1001 Z+ lockf2
5079 5052 834 1001 Z+ lockf2
5050 4769 834 1001 Z+ lockf
db> allt
Tracing command lockf2 pid 5157 tid 100314 td 0xc91eecc0
sched_switch(c91eecc0,0,1,180,2aefa471,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,c91eecc0,...) at mi_switch+0x217
sleepq_switch(c91eecc0,0,c0ab8594,21d,50,...) at sleepq_switch+0xf0
sleepq_wait(daa06ef0,0,c0ab54cf,da,0,...) at sleepq_wait+0x60
_sleep(daa06ef0,c0bdb810,50,c0abea93,0,...) at _sleep+0x369
acquire(e97f388c,e97f3884,c0ab2474,e0,c91eecc0,...) at acquire+0x73
_lockmgr(daa06ef0,202122,c6e2ae6c,c91eecc0,c0abe563,...) at _lockmgr+0x4bc
getblk(c6e2ad98,5120,0,4000,0,...) at getblk+0x21f
breadn(c6e2ad98,5120,0,4000,0,...) at breadn+0x8c
bread(c6e2ad98,5120,0,4000,0,...) at bread+0x4c
ffs_update(c8ca1414,0,8180,c00,0,...) at ffs_update+0x1a4
ufs_inactive(e97f3a38,c0ae6aa9,c8ca14e8,c8ca14e8,c8ca1414,...) at ufs_inactive+0x35d
VOP_INACTIVE_APV(c0b86340,e97f3a38,c0ac0353,8e3,c0b9d240,...) at VOP_INACTIVE_APV+0xc5
vinactive(c0b86340,e97f3a6c,c0ac0353,86f,c8ca1414,...) at vinactive+0xb1
vput(c8ca1414,e97f3aac,c91eecc0,c0ac129b,120,...) at vput+0x1f7
vn_close(c8ca1414,3,c6ec1a00,c91eecc0,c91eed58,...) at vn_close+0xf0
vn_closefile(c7c967e0,c91eecc0,c0ab09dc,87b,6db,...) at vn_closefile+0xf4
fdrop(c7c967e0,c91eecc0,c07991c8,c0c2cec0,0,c91eed58,c0b6d14c,c0ab09e5,c6c3102c,67a,c0ab09dc,e97f3ba0,c0767810,c6c3102c,8,c0ab09dc,67a) at fdrop+0x11e
closef(c7c967e0,c91eecc0,67a,678,c0798cf2,...) at closef+0x254
fdfree(c91eecc0,0,c0ab1279,fd,c915f5e0,...) at fdfree+0x3a6
exit1(c91eecc0,100,e97f3d2c,c0a22f63,c91eecc0,...) at exit1+0x573
sys_exit(c91eecc0,e97f3cfc,4,e97f3d38,c0b65698,...) at sys_exit+0x1d
syscall(e97f3d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (1, FreeBSD ELF32, sys_exit), eip = 0x280e270f, esp = 0xbfbfe4cc, ebp = 0xbfbfe4d8 ---
Tracing command lockf2 pid 5155 tid 100108 td 0xc6e79660
sched_switch(c6e79660,0,1,180,249bb9e7,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,c6e79660,...) at mi_switch+0x217
sleepq_switch(c6e79660,0,c0ab8594,21d,50,...) at sleepq_switch+0xf0
sleepq_wait(da9c0450,c0bdb540,c0abea93,0,0,...) at sleepq_wait+0x60
_sleep(da9c0450,c0bdb540,50,c0abea93,0,...) at _sleep+0x369
acquire(e941c670,e941c668,c0ab2474,e0,c6e79660,...) at acquire+0x73
_lockmgr(da9c0450,202122,c6e2ae6c,c6e79660,c0abe563,...) at _lockmgr+0x4bc
getblk(c6e2ad98,5100,0,800,0,...) at getblk+0x21f
breadn(c6e2ad98,5100,0,800,0,...) at breadn+0x8c
bread(c6e2ad98,5100,0,800,0,...) at bread+0x4c
ffs_blkfree(c6cd4400,c6de0800,c6e2ad98,1900,0,...) at ffs_blkfree+0x338
ffs_reallocblks(e941ca34,c0ae6a12,0,c8b6e000,c7c73568,...) at ffs_reallocblks+0x160b
VOP_REALLOCBLKS_APV(c0b86340,e941ca34,0,4000,0,...) at VOP_REALLOCBLKS_APV+0xd6
cluster_write(c8b6e000,da9af950,48000,0,7f,...) at cluster_write+0x3d2
ffs_write(e941cbc4,c0ae739e,e941cb2c,0,c0b86340,...) at ffs_write+0x5ce
VOP_WRITE_APV(c0b86340,e941cbc4,c6e79660,c0ac129b,242,...) at VOP_WRITE_APV+0x155
vn_write(c6e46ab0,e941cc60,c6ec1a00,0,c6e79660,...) at vn_write+0x231
dofilewrite(e941cc60,ffffffff,ffffffff,0,c6e46ab0,...) at dofilewrite+0x95
kern_writev(c6e79660,3,e941cc60,bfbfe964,0,...) at kern_writev+0x58
write(c6e79660,e941ccfc,c,c0aa4b83,c0b656e0,...) at write+0x4f
syscall(e941cd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (4, FreeBSD ELF32, write), eip = 0x281547e3, esp = 0xbfbfe53c, ebp = 0xbfbfe968 ---
Tracing command lockf2 pid 5151 tid 100138 td 0xc704faa0
sched_switch(c704faa0,0,1,180,6beea3fc,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c704faa0,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e94adbc4,c075356d,c0be4100,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,2711,...) at _sleep+0x331
kern_nanosleep(c704faa0,e94adc64,e94adc6c,a,0,...) at kern_nanosleep+0xc1
nanosleep(c704faa0,e94adcfc,8,e94add38,c0b66d00,...) at nanosleep+0x6f
syscall(e94add38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813754f, esp = 0xbfbfe93c, ebp = 0xbfbfe968 ---
Tracing command lockf2 pid 5150 tid 100075 td 0xc6d32000
sched_switch(c6d32000,0,1,180,6be08c63,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c6d32000,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e937dbc4,c075356d,c0be4000,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,2329,...) at _sleep+0x331
kern_nanosleep(c6d32000,e937dc64,e937dc6c,9,0,...) at kern_nanosleep+0xc1
nanosleep(c6d32000,e937dcfc,8,e937dd38,c0b66d00,...) at nanosleep+0x6f
syscall(e937dd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813754f, esp = 0xbfbfe93c, ebp = 0xbfbfe968 ---
Tracing command lockf2 pid 5149 tid 100252 td 0xc9e68000
sched_switch(c9e68000,0,1,180,6bd2dcea,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c9e68000,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e9663bc4,c075356d,c0be3f80,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,2711,...) at _sleep+0x331
kern_nanosleep(c9e68000,e9663c64,e9663c6c,a,0,...) at kern_nanosleep+0xc1
nanosleep(c9e68000,e9663cfc,8,e9663d38,c0b66d00,...) at nanosleep+0x6f
syscall(e9663d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813754f, esp = 0xbfbfe93c, ebp = 0xbfbfe968 ---
Tracing command lockf2 pid 5148 tid 100056 td 0xc6927cc0
sched_switch(c6927cc0,0,1,180,6bbc9b8b,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c6927cc0,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e932fbc4,c075356d,c0be4100,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,2329,...) at _sleep+0x331
kern_nanosleep(c6927cc0,e932fc64,e932fc6c,9,0,...) at kern_nanosleep+0xc1
nanosleep(c6927cc0,e932fcfc,8,e932fd38,c0b66d00,...) at nanosleep+0x6f
syscall(e932fd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813754f, esp = 0xbfbfe93c, ebp = 0xbfbfe968 ---
Tracing command lockf2 pid 5144 tid 100244 td 0xc7019000
sched_switch(c7019000,0,1,180,6b7f14c7,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c7019000,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e964abc4,c075356d,c0be4000,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,2711,...) at _sleep+0x331
kern_nanosleep(c7019000,e964ac64,e964ac6c,a,0,...) at kern_nanosleep+0xc1
nanosleep(c7019000,e964acfc,8,e964ad38,c0b66d00,...) at nanosleep+0x6f
syscall(e964ad38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813754f, esp = 0xbfbfe93c, ebp = 0xbfbfe968 ---
Tracing command lockf2 pid 5143 tid 100221 td 0xc9e68880
sched_switch(c9e68880,0,1,17b,2a89ca0b,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8d64,2e2,c92c2aa0,...) at mi_switch+0x217
turnstile_wait(c92c2aa0,c9f56000,0,192,1,...) at turnstile_wait+0x4cb
_mtx_lock_sleep(c1474808,c9e68880,0,c0ad6278,8d1,...) at _mtx_lock_sleep+0x19b
_mtx_lock_flags(c1474808,0,c0ad6278,8d1,c0c2d128,...) at _mtx_lock_flags+0xef
uma_zfree_arg(c1472d20,c6fdcc80,c6fdcef8,c755b140,0,...) at uma_zfree_arg+0x58
free(c6fdcc80,c0b69ea0,c0ab2675,d8,0,...) at free+0xc8
lf_advlock(e95f2c20,c8de0034,100000,0) at lf_advlock+0x898
ufs_advlock(e95f2c20,c0ae7052,c6c31e00,c6c31e00,16,...) at ufs_advlock+0x2d
VOP_ADVLOCK_APV(c0b86340,e95f2c20,210,20f,0,...) at VOP_ADVLOCK_APV+0xd6
kern_fcntl(c9e68880,3,8,e95f2c60,30be8,...) at kern_fcntl+0x8ff
fcntl(c9e68880,e95f2cfc,c,c0aba75c,c0b65f20,...) at fcntl+0x57
syscall(e95f2d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (92, FreeBSD ELF32, fcntl), eip = 0x281433bf, esp = 0xbfbfe8ec, ebp = 0xbfbfe928 ---
Tracing command lockf2 pid 5142 tid 100224 td 0xc7501aa0
sched_switch(c7501aa0,0,1,180,6b5abfdb,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c7501aa0,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e95febc4,c075356d,c0be4100,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,2711,...) at _sleep+0x331
kern_nanosleep(c7501aa0,e95fec64,e95fec6c,a,0,...) at kern_nanosleep+0xc1
nanosleep(c7501aa0,e95fecfc,8,e95fed38,c0b66d00,...) at nanosleep+0x6f
syscall(e95fed38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813754f, esp = 0xbfbfe93c, ebp = 0xbfbfe968 ---
Tracing command lockf2 pid 5141 tid 100229 td 0xc7501880
sched_switch(c7501880,0,1,180,6b4b9d0e,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c7501880,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e9611bc4,c075356d,c0be4000,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,2711,...) at _sleep+0x331
kern_nanosleep(c7501880,e9611c64,e9611c6c,a,0,...) at kern_nanosleep+0xc1
nanosleep(c7501880,e9611cfc,8,e9611d38,c0b66d00,...) at nanosleep+0x6f
syscall(e9611d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813754f, esp = 0xbfbfe93c, ebp = 0xbfbfe968 ---
Tracing command lockf2 pid 5140 tid 100294 td 0xc7502aa0
sched_switch(c7502aa0,0,1,180,6b3ffe7e,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c7502aa0,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e9712bc4,c075356d,c0be3f80,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,2711,...) at _sleep+0x331
kern_nanosleep(c7502aa0,e9712c64,e9712c6c,a,0,...) at kern_nanosleep+0xc1
nanosleep(c7502aa0,e9712cfc,8,e9712d38,c0b66d00,...) at nanosleep+0x6f
syscall(e9712d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813754f, esp = 0xbfbfe93c, ebp = 0xbfbfe968 ---
Tracing command lockf2 pid 5139 tid 100386 td 0xc9201000
sched_switch(c9201000,0,2,17b,2bc1de62,...) at sched_switch+0x49b
mi_switch(2,0,c0ab8ccd,ec,1,...) at mi_switch+0x217
ast(e98b6d38) at ast+0x2b7
doreti_ast() at doreti_ast+0x17
Tracing command lockf2 pid 5138 tid 100133 td 0xc7051660
sched_switch(c7051660,0,1,17b,2a89eb05,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8d64,2e2,c92c2aa0,...) at mi_switch+0x217
turnstile_wait(c92c2aa0,c9f56000,0,192,1,...) at turnstile_wait+0x4cb
_mtx_lock_sleep(c1474808,c7051660,0,c0ad6278,726,...) at _mtx_lock_sleep+0x19b
_mtx_lock_flags(c1474808,0,c0ad6278,726,483,...) at _mtx_lock_flags+0xef
uma_zalloc_arg(c1472d20,0,2,2,5b801,...) at uma_zalloc_arg+0xd3
malloc(34,c0b69ea0,2,d8,0,...) at malloc+0xd2
lf_advlock(e949ec20,c8c131c0,100000,0) at lf_advlock+0x183
ufs_advlock(e949ec20,c0ae7052,c6c5b400,c6c5b400,16,...) at ufs_advlock+0x2d
VOP_ADVLOCK_APV(c0b86340,e949ec20,c0ab09dc,226,0,...) at VOP_ADVLOCK_APV+0xd6
kern_fcntl(c7051660,3,9,e949ec60,5b801,...) at kern_fcntl+0x8bf
fcntl(c7051660,e949ecfc,c,c0aba567,c0b65f20,...) at fcntl+0x57
syscall(e949ed38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (92, FreeBSD ELF32, fcntl), eip = 0x281433bf, esp = 0xbfbfe8ec, ebp = 0xbfbfe928 ---
Tracing command lockf2 pid 5136 tid 100290 td 0xc74cd000
sched_switch(c74cd000,0,1,180,6af71074,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c74cd000,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e96febc4,c075356d,c0be4100,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,2711,...) at _sleep+0x331
kern_nanosleep(c74cd000,e96fec64,e96fec6c,a,0,...) at kern_nanosleep+0xc1
nanosleep(c74cd000,e96fecfc,8,e96fed38,c0b66d00,...) at nanosleep+0x6f
syscall(e96fed38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813754f, esp = 0xbfbfe93c, ebp = 0xbfbfe968 ---
Tracing command lockf2 pid 5134 tid 100324 td 0xc6d16440
sched_switch(c6d16440,0,1,17b,2a8a67ca,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8d64,2e2,c92c2aa0,...) at mi_switch+0x217
turnstile_wait(c92c2aa0,c9f56000,0,192,1,...) at turnstile_wait+0x4cb
_mtx_lock_sleep(c1474808,c6d16440,0,c0ad6278,726,...) at _mtx_lock_sleep+0x19b
_mtx_lock_flags(c1474808,0,c0ad6278,726,2721e6,...) at _mtx_lock_flags+0xef
uma_zalloc_arg(c1472d20,0,2,2,398f2,...) at uma_zalloc_arg+0xd3
malloc(34,c0b69ea0,2,d8,0,...) at malloc+0xd2
lf_advlock(e977bc20,c8a2f1c0,100000,0) at lf_advlock+0x183
ufs_advlock(e977bc20,c0ae7052,c6c5b800,c6c5b800,16,...) at ufs_advlock+0x2d
VOP_ADVLOCK_APV(c0b86340,e977bc20,210,20f,0,...) at VOP_ADVLOCK_APV+0xd6
kern_fcntl(c6d16440,3,8,e977bc60,398f2,...) at kern_fcntl+0x8ff
fcntl(c6d16440,e977bcfc,c,c0aba75c,c0b65f20,...) at fcntl+0x57
syscall(e977bd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (92, FreeBSD ELF32, fcntl), eip = 0x281433bf, esp = 0xbfbfe8ec, ebp = 0xbfbfe928 ---
Tracing command lockf2 pid 5133 tid 100214 td 0xc9e69000
sched_switch(c9e69000,0,1,180,6ac30f00,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c9e69000,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e95d6bc4,c075356d,c0be4100,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,2329,...) at _sleep+0x331
kern_nanosleep(c9e69000,e95d6c64,e95d6c6c,9,0,...) at kern_nanosleep+0xc1
nanosleep(c9e69000,e95d6cfc,8,e95d6d38,c0b66d00,...) at nanosleep+0x6f
syscall(e95d6d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813754f, esp = 0xbfbfe93c, ebp = 0xbfbfe968 ---
Tracing command lockf2 pid 5131 tid 100052 td 0xc6c94220
sched_switch(c6c94220,0,1,180,6aa4ab0c,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c6c94220,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e931fbc4,c075356d,c0be3f80,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,2329,...) at _sleep+0x331
kern_nanosleep(c6c94220,e931fc64,e931fc6c,9,0,...) at kern_nanosleep+0xc1
nanosleep(c6c94220,e931fcfc,8,e931fd38,c0b66d00,...) at nanosleep+0x6f
syscall(e931fd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813754f, esp = 0xbfbfe93c, ebp = 0xbfbfe968 ---
Tracing command lockf2 pid 5130 tid 100065 td 0xc6d04880
kdb_enter(c0ab4b70,3,c0ad3174,e93527c0,3,...) at kdb_enter+0x32
panic(c0ad3174,0,c0ad3124,1e9,0,...) at panic+0x124
ffs_truncate(c8c73000,0,0,c00,0,...) at ffs_truncate+0x1858
ufs_inactive(e9352a38,c0ae6aa9,c8c730d4,c8c730d4,c8c73000,...) at ufs_inactive+0x250
VOP_INACTIVE_APV(c0b86340,e9352a38,c0ac0353,8e3,c0b9d240,...) at VOP_INACTIVE_APV+0xc5
vinactive(c0b86340,e9352a6c,c0ac0353,86f,c8c73000,...) at vinactive+0xb1
vput(c8c73000,e9352aac,c6d04880,c0ac129b,120,...) at vput+0x1f7
vn_close(c8c73000,3,c6ec1a00,c6d04880,c6d04918,...) at vn_close+0xf0
vn_closefile(c6fc3d80,c6d04880,c0ab09dc,87b,6db,...) at vn_closefile+0xf4
fdrop(c6fc3d80,c6d04880,c07991c8,c0c2cbe8,0,c6d04918,c0b6d14c,c0ab09e5,c6c5bc2c,67a,c0ab09dc,e9352ba0,c0767810,c6c5bc2c,8,c0ab09dc,67a) at fdrop+0x11e
closef(c6fc3d80,c6d04880,67a,678,c0798cf2,...) at closef+0x254
fdfree(c6d04880,0,c0ab1279,fd,c6cdbb30,...) at fdfree+0x3a6
exit1(c6d04880,100,e9352d2c,c0a22f63,c6d04880,...) at exit1+0x573
sys_exit(c6d04880,e9352cfc,4,e9352d38,c0b65698,...) at sys_exit+0x1d
syscall(e9352d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (1, FreeBSD ELF32, sys_exit), eip = 0x280e270f, esp = 0xbfbfe4cc, ebp = 0xbfbfe4d8 ---
Tracing command lockf2 pid 5129 tid 100282 td 0xc74bd440
sched_switch(c74bd440,0,1,180,6a7a37d9,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c74bd440,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e96dabc4,c075356d,c0be4100,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,2711,...) at _sleep+0x331
kern_nanosleep(c74bd440,e96dac64,e96dac6c,a,0,...) at kern_nanosleep+0xc1
nanosleep(c74bd440,e96dacfc,8,e96dad38,c0b66d00,...) at nanosleep+0x6f
syscall(e96dad38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813754f, esp = 0xbfbfe93c, ebp = 0xbfbfe968 ---
Tracing command lockf2 pid 5128 tid 100164 td 0xc6e66000
sched_switch(c6e66000,0,1,17b,2a8a85bb,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8d64,2e2,c92c2aa0,...) at mi_switch+0x217
turnstile_wait(c92c2aa0,c9f56000,0,192,1,...) at turnstile_wait+0x4cb
_mtx_lock_sleep(c1474808,c6e66000,0,c0ad6278,726,...) at _mtx_lock_sleep+0x19b
_mtx_lock_flags(c1474808,0,c0ad6278,726,c0be4000,...) at _mtx_lock_flags+0xef
uma_zalloc_arg(c1472d20,0,2,2,a6a75,...) at uma_zalloc_arg+0xd3
malloc(34,c0b69ea0,2,d8,0,...) at malloc+0xd2
lf_advlock(e951ac20,c8cedb8c,100000,0) at lf_advlock+0x183
ufs_advlock(e951ac20,c0ae7052,c6c5be00,c6c5be00,16,...) at ufs_advlock+0x2d
VOP_ADVLOCK_APV(c0b86340,e951ac20,210,20f,4e2000,...) at VOP_ADVLOCK_APV+0xd6
kern_fcntl(c6e66000,3,8,e951ac60,a6a75,...) at kern_fcntl+0x8ff
fcntl(c6e66000,e951acfc,c,c0aba75c,c0b65f20,...) at fcntl+0x57
syscall(e951ad38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (92, FreeBSD ELF32, fcntl), eip = 0x281433bf, esp = 0xbfbfe8ec, ebp = 0xbfbfe928 ---
Tracing command lockf2 pid 5126 tid 100336 td 0xc9f60220
sched_switch(c9f60220,0,1,180,6a47e77e,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c9f60220,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e9829bc4,c075356d,c0be4100,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,2711,...) at _sleep+0x331
kern_nanosleep(c9f60220,e9829c64,e9829c6c,a,0,...) at kern_nanosleep+0xc1
nanosleep(c9f60220,e9829cfc,8,e9829d38,c0b66d00,...) at nanosleep+0x6f
syscall(e9829d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813754f, esp = 0xbfbfe93c, ebp = 0xbfbfe968 ---
Tracing command lockf2 pid 5123 tid 100340 td 0xc92d1000
sched_switch(c92d1000,0,1,17b,2aad4ca3,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8d64,2e2,c92c2aa0,...) at mi_switch+0x217
turnstile_wait(c92c2aa0,c9f56000,0,192,1,...) at turnstile_wait+0x4cb
_mtx_lock_sleep(c1474808,c92d1000,0,c0ad6278,726,...) at _mtx_lock_sleep+0x19b
_mtx_lock_flags(c1474808,0,c0ad6278,726,c92d1000,...) at _mtx_lock_flags+0xef
uma_zalloc_arg(c1472d20,0,2,2,23862,...) at uma_zalloc_arg+0xd3
malloc(34,c0b69ea0,2,d8,0,...) at malloc+0xd2
lf_advlock(e9832c20,c8cfdb8c,100000,0) at lf_advlock+0x183
ufs_advlock(e9832c20,c0ae7052,c6c5e400,c6c5e400,16,...) at ufs_advlock+0x2d
VOP_ADVLOCK_APV(c0b86340,e9832c20,210,20f,0,...) at VOP_ADVLOCK_APV+0xd6
kern_fcntl(c92d1000,3,8,e9832c60,23862,...) at kern_fcntl+0x8ff
fcntl(c92d1000,e9832cfc,c,e9832ccc,c0b65f20,...) at fcntl+0x57
syscall(e9832d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (92, FreeBSD ELF32, fcntl), eip = 0x281433bf, esp = 0xbfbfe8ec, ebp = 0xbfbfe928 ---
Tracing command lockf2 pid 5122 tid 100341 td 0xc92be660
sched_switch(c92be660,0,1,180,2b289939,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,c92be660,...) at mi_switch+0x217
sleepq_switch(c92be660,0,c0ab8594,21d,50,...) at sleepq_switch+0xf0
sleepq_wait(daa06ef0,0,c0ab54cf,da,0,...) at sleepq_wait+0x60
_sleep(daa06ef0,c0bdb810,50,c0abea93,0,...) at _sleep+0x369
acquire(e9835674,e983566c,c0ab2474,e0,c92be660,...) at acquire+0x73
_lockmgr(daa06ef0,202122,c6e2ae6c,c92be660,c0abe563,...) at _lockmgr+0x4bc
getblk(c6e2ad98,5120,0,4000,0,...) at getblk+0x21f
breadn(c6e2ad98,5120,0,4000,0,...) at breadn+0x8c
bread(c6e2ad98,5120,0,4000,0,...) at bread+0x4c
ffs_update(c7be2570,1,0,272389,c92be660,...) at ffs_update+0x1a4
ffs_truncate(c7be2570,0,0,c00,0,...) at ffs_truncate+0xeb0
ufs_inactive(e9835a38,c0ae6aa9,c7be2644,c7be2644,c7be2570,...) at ufs_inactive+0x250
VOP_INACTIVE_APV(c0b86340,e9835a38,c0ac0353,8e3,c0b9d240,...) at VOP_INACTIVE_APV+0xc5
vinactive(c0b86340,e9835a6c,c0ac0353,86f,c7be2570,...) at vinactive+0xb1
vput(c7be2570,e9835aac,c92be660,c0ac129b,120,...) at vput+0x1f7
vn_close(c7be2570,3,c6ec1a00,c92be660,c92be6f8,...) at vn_close+0xf0
vn_closefile(c6fbfaf8,c92be660,c0ab09dc,87b,6db,...) at vn_closefile+0xf4
fdrop(c6fbfaf8,c92be660,c07991c8,c0c2d0b8,0,c92be6f8,c0b6d14c,c0ab09e5,c6c5e52c,67a,c0ab09dc,e9835ba0,c0767810,c6c5e52c,8,c0ab09dc,67a) at fdrop+0x11e
closef(c6fbfaf8,c92be660,67a,678,c0798cf2,...) at closef+0x254
fdfree(c92be660,0,c0ab1279,fd,c922a090,...) at fdfree+0x3a6
exit1(c92be660,100,e9835d2c,c0a22f63,c92be660,...) at exit1+0x573
sys_exit(c92be660,e9835cfc,4,e9835d38,c0b65698,...) at sys_exit+0x1d
syscall(e9835d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (1, FreeBSD ELF32, sys_exit), eip = 0x280e270f, esp = 0xbfbfe4cc, ebp = 0xbfbfe4d8 ---
Tracing command lockf2 pid 5118 tid 100373 td 0xc9f73000
sched_switch(c9f73000,0,1,180,69c6c430,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c9f73000,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e988fbc4,c075356d,c0be3f80,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,2711,...) at _sleep+0x331
kern_nanosleep(c9f73000,e988fc64,e988fc6c,a,0,...) at kern_nanosleep+0xc1
nanosleep(c9f73000,e988fcfc,8,e988fd38,c0b66d00,...) at nanosleep+0x6f
syscall(e988fd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813754f, esp = 0xbfbfe93c, ebp = 0xbfbfe968 ---
Tracing command lockf2 pid 5117 tid 100374 td 0xc9256440
sched_switch(c9256440,0,1,180,69b1d8f1,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c9256440,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e9892bc4,c075356d,c0be4100,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,2711,...) at _sleep+0x331
kern_nanosleep(c9256440,e9892c64,e9892c6c,a,0,...) at kern_nanosleep+0xc1
nanosleep(c9256440,e9892cfc,8,e9892d38,c0b66d00,...) at nanosleep+0x6f
syscall(e9892d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813754f, esp = 0xbfbfe93c, ebp = 0xbfbfe968 ---
Tracing command lockf2 pid 5116 tid 100375 td 0xc9f6aaa0
sched_switch(c9f6aaa0,0,1,17b,2a9124e1,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8d64,2e2,c92c2aa0,...) at mi_switch+0x217
turnstile_wait(c92c2aa0,c9f56000,0,192,1,...) at turnstile_wait+0x4cb
_mtx_lock_sleep(c1474808,c9f6aaa0,0,c0ad6278,726,...) at _mtx_lock_sleep+0x19b
_mtx_lock_flags(c1474808,0,c0ad6278,726,483,...) at _mtx_lock_flags+0xef
uma_zalloc_arg(c1472d20,0,2,2,ab281,...) at uma_zalloc_arg+0xd3
malloc(34,c0b69ea0,2,d8,0,...) at malloc+0xd2
lf_advlock(e9895c20,c8c97034,100000,0) at lf_advlock+0x183
ufs_advlock(e9895c20,c0ae7052,c6c5eb00,c6c5eb00,16,...) at ufs_advlock+0x2d
VOP_ADVLOCK_APV(c0b86340,e9895c20,210,20f,0,...) at VOP_ADVLOCK_APV+0xd6
kern_fcntl(c9f6aaa0,3,8,e9895c60,ab281,...) at kern_fcntl+0x8ff
fcntl(c9f6aaa0,e9895cfc,c,e9895ccc,c0b65f20,...) at fcntl+0x57
syscall(e9895d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (92, FreeBSD ELF32, fcntl), eip = 0x281433bf, esp = 0xbfbfe8ec, ebp = 0xbfbfe928 ---
Tracing command lockf2 pid 5114 tid 100377 td 0xc92ca880
sched_switch(c92ca880,0,1,180,69668732,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c92ca880,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e989bbc4,c075356d,c0be4000,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,2711,...) at _sleep+0x331
kern_nanosleep(c92ca880,e989bc64,e989bc6c,a,0,...) at kern_nanosleep+0xc1
nanosleep(c92ca880,e989bcfc,8,e989bd38,c0b66d00,...) at nanosleep+0x6f
syscall(e989bd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813754f, esp = 0xbfbfe93c, ebp = 0xbfbfe968 ---
Tracing command lockf2 pid 5112 tid 100379 td 0xc92decc0
sched_switch(c92decc0,0,1,180,69505626,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c92decc0,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e98a1bc4,c075356d,c0be4100,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,2711,...) at _sleep+0x331
kern_nanosleep(c92decc0,e98a1c64,e98a1c6c,a,0,...) at kern_nanosleep+0xc1
nanosleep(c92decc0,e98a1cfc,8,e98a1d38,c0b66d00,...) at nanosleep+0x6f
syscall(e98a1d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813754f, esp = 0xbfbfe93c, ebp = 0xbfbfe968 ---
Tracing command lockf2 pid 5107 tid 100213 td 0xc9e69220
sched_switch(c9e69220,0,1,180,2b23046e,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,c9e69220,...) at mi_switch+0x217
sleepq_switch(c9e69220,0,c0ab8594,21d,4c,...) at sleepq_switch+0xf0
sleepq_wait(daa06e90,0,c0ab54cf,da,0,...) at sleepq_wait+0x60
_sleep(daa06e90,c0c2dbb8,4c,c0abdf8f,0,...) at _sleep+0x369
bwait(daa06e90,4c,c0abdf8f,daa06e90,e95d2800,...) at bwait+0x60
bufwait(daa06e90,daa06e90,33d,c0abeb7e,daa06e90,...) at bufwait+0x48
bufwrite(daa06e90,0,c0ad4b2d,6ce,daa06e90,c6e2ad98,a0020020,0,0,0) at bufwrite+0x1ed
ffs_bufwrite(daa06e90,c8e76400,100,4000,0,...) at ffs_bufwrite+0x33f
ffs_update(c8d89570,1,c0ad4f93,14e,c0abe563,...) at ffs_update+0x2d5
ffs_syncvnode(c8d89570,1,0,1418,0,...) at ffs_syncvnode+0x4af
ffs_balloc_ufs2(c8d89570,30000,0,400,c6ec1a00,...) at ffs_balloc_ufs2+0x1d08
ffs_write(e95d2bc4,c0ae739e,e95d2b2c,0,c0b86340,...) at ffs_write+0x373
VOP_WRITE_APV(c0b86340,e95d2bc4,c9e69220,c0ac129b,242,...) at VOP_WRITE_APV+0x155
vn_write(c6fc4a68,e95d2c60,c6ec1a00,0,c9e69220,...) at vn_write+0x231
dofilewrite(e95d2c60,ffffffff,ffffffff,0,c6fc4a68,...) at dofilewrite+0x95
kern_writev(c9e69220,3,e95d2c60,bfbfe564,400,...) at kern_writev+0x58
write(c9e69220,e95d2cfc,c,c0aa4b83,c0b656e0,...) at write+0x4f
syscall(e95d2d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (4, FreeBSD ELF32, write), eip = 0x281547e3, esp = 0xbfbfe53c, ebp = 0xbfbfe968 ---
Tracing command lockf2 pid 5104 tid 100337 td 0xc9202440
sched_switch(c9202440,0,1,180,232ddc2e,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,c9202440,...) at mi_switch+0x217
sleepq_switch(c9202440,0,c0ab8594,21d,50,...) at sleepq_switch+0xf0
sleepq_wait(da9c0450,0,c0ab54cf,da,0,...) at sleepq_wait+0x60
_sleep(da9c0450,c0bdb540,50,c0abea93,0,...) at _sleep+0x369
acquire(e979c678,e979c670,c0ab2474,e0,c9202440,...) at acquire+0x73
_lockmgr(da9c0450,202122,c6e2ae6c,c9202440,c0abe563,...) at _lockmgr+0x4bc
getblk(c6e2ad98,5100,0,800,0,...) at getblk+0x21f
breadn(c6e2ad98,5100,0,800,0,...) at breadn+0x8c
bread(c6e2ad98,5100,0,800,0,...) at bread+0x4c
ffs_checkblk(4000,c0ab37db,fe,c0c2d128,e979c810,...) at ffs_checkblk+0x1d6
ffs_reallocblks(e979ca34,c0ae6a12,0,c8e83ae0,c8555d64,...) at ffs_reallocblks+0xc3a
VOP_REALLOCBLKS_APV(c0b86340,e979ca34,0,4000,0,...) at VOP_REALLOCBLKS_APV+0xd6
cluster_write(c8e83ae0,daa57ce0,44000,0,7f,...) at cluster_write+0x3d2
ffs_write(e979cbc4,c0ae739e,e979cb2c,0,c0b86340,...) at ffs_write+0x5ce
VOP_WRITE_APV(c0b86340,e979cbc4,c9202440,c0ac129b,242,...) at VOP_WRITE_APV+0x155
vn_write(c7c9d948,e979cc60,c6ec1a00,0,c9202440,...) at vn_write+0x231
dofilewrite(e979cc60,ffffffff,ffffffff,0,c7c9d948,...) at dofilewrite+0x95
kern_writev(c9202440,3,e979cc60,bfbfe964,0,...) at kern_writev+0x58
write(c9202440,e979ccfc,c,c0aa4b83,c0b656e0,...) at write+0x4f
syscall(e979cd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (4, FreeBSD ELF32, write), eip = 0x281547e3, esp = 0xbfbfe53c, ebp = 0xbfbfe968 ---
Tracing command lockf2 pid 5103 tid 100384 td 0xc91e7880
cpustop_handler(10,e98b064c,c0a2326d,e98b05cc,e98b05cc,...) at cpustop_handler+0x32
ipi_nmi_handler(e98b05cc,e98b05cc,c0a15a08,e98b05e4,c6b2ed48,...) at ipi_nmi_handler+0x2f
trap(e98b0658) at trap+0x2d
calltrap() at calltrap+0x6
--- trap 0x13, eip = 0xc0753a4f, esp = 0xe98b0698, ebp = 0xe98b06a8 ---
_mtx_lock_spin(c0c46a6c,c91e7880,0,c0adeb06,9fc,...) at _mtx_lock_spin+0x3f
_mtx_lock_spin_flags(c0c46a6c,0,c0adeb06,9fc,c0c2d860,...) at _mtx_lock_spin_flags+0x10e
sio_cnputc(c0b91fe0,62,e98b0754,1,0,...) at sio_cnputc+0x8d
cnputc(62,0,c0abc0ab,28e,1,...) at cnputc+0x5f
cnputs(e98b0754,c078c670,e98b07d4,a,e98b0808,...) at cnputs+0x58
printf(c0ac4fc6,62,c0ccc60f,c0ae008a,e98b0934,...) at printf+0x7e
db_putchar(62,0,c0c2d860,e98b0840,c0ccc622,...) at db_putchar+0x31
kvprintf(c0ae0088,c048fd40,0,10,e98b0964,...) at kvprintf+0x92d
db_printf(c0ae0088,c0ccc60d,e98b096c,ffffffff,c048f7c6,...) at db_printf+0x34
db_print_stack_entry(e98b0a7c,c048f7c6,0,c075356d,e98b0a74,...) at db_print_stack_entry+0xe5
db_backtrace(c048f7c6,3ff,0,e98b0a40,c0ac0543,...) at db_backtrace+0x2a7
db_trace_self(e98b0a40,0,e98b0a40,e98b0a34,e98b0a74,...) at db_trace_self+0x49
db_trace_self_wrapper(c0ab79ab,e98b0aa4,c07db0d3,0,e98b0aa4,...) at db_trace_self_wrapper+0x26
kdb_backtrace(0,e98b0aa4,c0a2d795,e98b0ab4,c8e09000,...) at kdb_backtrace+0x29
vfs_badlock(c0b86340,e98b0ab4,c0b9cee0,c8e09000,0) at vfs_badlock+0x23
assert_vop_locked(c8e09000,c0ac055f) at assert_vop_locked+0x50
vop_lock_post(e98b0b10,0,1002,1002,c8e09000,...) at vop_lock_post+0x36
VOP_LOCK1_APV(c0b86340,e98b0b10,c0ac129b,327,c8e090d4,...) at VOP_LOCK1_APV+0xc1
_vn_lock(c8e09000,1002,c91e7880,c0ac129b,120,...) at _vn_lock+0xf8
vn_close(c8e09000,3,c6ec1a00,c91e7880,c07d0801,...) at vn_close+0x8a
vn_closefile(c6ebeca8,c91e7880,c0ab09dc,87b,c8e09000,...) at vn_closefile+0xf4
fdrop(c6ebeca8,c91e7880,c0a11fc5,40,0,c0b9d360,c8e09000,c6b2ed48,2,e98b0c20,40,0,0,0,0,c0ab09dc,2) at fdrop+0x11e
closef(c6ebeca8,c91e7880,41c,401,c6ebeca8,...) at closef+0x254
kern_close(c91e7880,3,e98b0d2c,c0a22f63,c91e7880,...) at kern_close+0x11d
close(c91e7880,e98b0cfc,4,c0aba75c,c0b65710,...) at close+0x1a
syscall(e98b0d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (6, FreeBSD ELF32, close), eip = 0x281547c3, esp = 0xbfbfe92c, ebp = 0xbfbfe968 ---
Tracing command lockf2 pid 5102 tid 100195 td 0xc74be660
sched_switch(c74be660,0,1,17b,fb47d25f,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8d64,2e2,c92c2aa0,...) at mi_switch+0x217
turnstile_wait(c92c2aa0,c9e68880,0,192,1,...) at turnstile_wait+0x4cb
_mtx_lock_sleep(c1474808,c74be660,0,c0ad6278,726,...) at _mtx_lock_sleep+0x19b
_mtx_lock_flags(c1474808,0,c0ad6278,726,e958aae8,...) at _mtx_lock_flags+0xef
uma_zalloc_arg(c1472d20,0,2,2,1a23b,...) at uma_zalloc_arg+0xd3
malloc(34,c0b69ea0,2,d8,0,...) at malloc+0xd2
lf_advlock(e958ac20,c8c62e20,100000,0) at lf_advlock+0x183
ufs_advlock(e958ac20,c0ae7052,c95c6100,c95c6100,16,...) at ufs_advlock+0x2d
VOP_ADVLOCK_APV(c0b86340,e958ac20,c0ab09dc,226,0,...) at VOP_ADVLOCK_APV+0xd6
kern_fcntl(c74be660,3,9,e958ac60,1a23b,...) at kern_fcntl+0x8bf
fcntl(c74be660,e958acfc,c,c0aba567,c0b65f20,...) at fcntl+0x57
syscall(e958ad38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (92, FreeBSD ELF32, fcntl), eip = 0x281433bf, esp = 0xbfbfe8ec, ebp = 0xbfbfe928 ---
Tracing command lockf2 pid 5100 tid 100254 td 0xc9e67aa0
sched_switch(c9e67aa0,0,1,180,68915e18,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c9e67aa0,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e966bbc4,c075356d,c0be3f80,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,2329,...) at _sleep+0x331
kern_nanosleep(c9e67aa0,e966bc64,e966bc6c,9,0,...) at kern_nanosleep+0xc1
nanosleep(c9e67aa0,e966bcfc,8,e966bd38,c0b66d00,...) at nanosleep+0x6f
syscall(e966bd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813754f, esp = 0xbfbfe93c, ebp = 0xbfbfe968 ---
Tracing command lockf2 pid 5095 tid 100313 td 0xc6d16220
sched_switch(c6d16220,0,1,17b,2a8e805e,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8d64,2e2,c92c2aa0,...) at mi_switch+0x217
turnstile_wait(c92c2aa0,c9f56000,0,192,1,...) at turnstile_wait+0x4cb
_mtx_lock_sleep(c1474808,c6d16220,0,c0ad6278,726,...) at _mtx_lock_sleep+0x19b
_mtx_lock_flags(c1474808,0,c0ad6278,726,e9760ae8,...) at _mtx_lock_flags+0xef
uma_zalloc_arg(c1472d20,0,2,2,51754,...) at uma_zalloc_arg+0xd3
malloc(34,c0b69ea0,2,d8,0,...) at malloc+0xd2
lf_advlock(e9760c20,c8b4aea4,100000,0) at lf_advlock+0x183
ufs_advlock(e9760c20,c0ae7052,c9082200,c9082200,16,...) at ufs_advlock+0x2d
VOP_ADVLOCK_APV(c0b86340,e9760c20,c0ab09dc,226,783,...) at VOP_ADVLOCK_APV+0xd6
kern_fcntl(c6d16220,3,9,e9760c60,51754,...) at kern_fcntl+0x8bf
fcntl(c6d16220,e9760cfc,c,c0aba567,c0b65f20,...) at fcntl+0x57
syscall(e9760d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (92, FreeBSD ELF32, fcntl), eip = 0x281433bf, esp = 0xbfbfe8ec, ebp = 0xbfbfe928 ---
Tracing command lockf2 pid 5094 tid 100245 td 0xc7018cc0
sched_switch(c7018cc0,0,1,180,6853cfeb,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c7018cc0,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e964dbc4,c075356d,c0be3f80,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,2711,...) at _sleep+0x331
kern_nanosleep(c7018cc0,e964dc64,e964dc6c,a,0,...) at kern_nanosleep+0xc1
nanosleep(c7018cc0,e964dcfc,8,e964dd38,c0b66d00,...) at nanosleep+0x6f
syscall(e964dd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813754f, esp = 0xbfbfe93c, ebp = 0xbfbfe968 ---
Tracing command lockf2 pid 5089 tid 100067 td 0xc6d17cc0
sched_switch(c6d17cc0,0,1,17b,2a8a2ade,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8d64,2e2,c92c2aa0,...) at mi_switch+0x217
turnstile_wait(c92c2aa0,c9f56000,0,192,1,...) at turnstile_wait+0x4cb
_mtx_lock_sleep(c1474808,c6d17cc0,0,c0ad6278,8d1,...) at _mtx_lock_sleep+0x19b
_mtx_lock_flags(c1474808,0,c0ad6278,8d1,c0c2d128,...) at _mtx_lock_flags+0xef
uma_zfree_arg(c1472d20,c9614dc0,c9614ef8,c6a96100,0,...) at uma_zfree_arg+0x58
free(c9614dc0,c0b69ea0,c0ab2675,d8,0,...) at free+0xc8
lf_advlock(e935ac20,c89d52c8,100000,0) at lf_advlock+0x898
ufs_advlock(e935ac20,c0ae7052,c9095c00,c9095c00,16,...) at ufs_advlock+0x2d
VOP_ADVLOCK_APV(c0b86340,e935ac20,210,20f,c0bdee44,...) at VOP_ADVLOCK_APV+0xd6
kern_fcntl(c6d17cc0,3,8,e935ac60,27d01,...) at kern_fcntl+0x8ff
fcntl(c6d17cc0,e935acfc,c,e935accc,c0b65f20,...) at fcntl+0x57
syscall(e935ad38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (92, FreeBSD ELF32, fcntl), eip = 0x281433bf, esp = 0xbfbfe8ec, ebp = 0xbfbfe928 ---
Tracing command lockf2 pid 5087 tid 100143 td 0xc704f000
sched_switch(c704f000,0,1,180,680cad17,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c704f000,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e94bcbc4,c075356d,c0be3f80,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,2711,...) at _sleep+0x331
kern_nanosleep(c704f000,e94bcc64,e94bcc6c,a,0,...) at kern_nanosleep+0xc1
nanosleep(c704f000,e94bccfc,8,e94bcd38,c0b66d00,...) at nanosleep+0x6f
syscall(e94bcd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813754f, esp = 0xbfbfe93c, ebp = 0xbfbfe968 ---
Tracing command lockf2 pid 5086 tid 100205 td 0xc74d2220
sched_switch(c74d2220,0,1,180,68011716,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c74d2220,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e95b2bc4,c075356d,c0be4100,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,2329,...) at _sleep+0x331
kern_nanosleep(c74d2220,e95b2c64,e95b2c6c,9,0,...) at kern_nanosleep+0xc1
nanosleep(c74d2220,e95b2cfc,8,e95b2d38,c0b66d00,...) at nanosleep+0x6f
syscall(e95b2d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813754f, esp = 0xbfbfe93c, ebp = 0xbfbfe968 ---
Tracing command lockf2 pid 5085 tid 100251 td 0xc7018220
sched_switch(c7018220,0,1,180,67f880ad,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c7018220,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e9660bc4,c075356d,c0be4000,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,2329,...) at _sleep+0x331
kern_nanosleep(c7018220,e9660c64,e9660c6c,9,0,...) at kern_nanosleep+0xc1
nanosleep(c7018220,e9660cfc,8,e9660d38,c0b66d00,...) at nanosleep+0x6f
syscall(e9660d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813754f, esp = 0xbfbfe93c, ebp = 0xbfbfe968 ---
Tracing command lockf2 pid 5081 tid 100334 td 0xc9f56000
sched_switch(c9f56000,0,2,17b,2b22a24b,...) at sched_switch+0x49b
mi_switch(2,0,c0ab8ccd,ec,c0be4100,...) at mi_switch+0x217
ast(e9823d38) at ast+0x2b7
doreti_ast() at doreti_ast+0x17
Tracing command lockf2 pid 5080 tid 100333 td 0xc92b3660
sched_switch(c92b3660,0,1,17b,2a8a09e4,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8d64,2e2,c92c2aa0,...) at mi_switch+0x217
turnstile_wait(c92c2aa0,c9f56000,0,192,1,...) at turnstile_wait+0x4cb
_mtx_lock_sleep(c1474808,c92b3660,0,c0ad6278,726,...) at _mtx_lock_sleep+0x19b
_mtx_lock_flags(c1474808,0,c0ad6278,726,c92b3660,...) at _mtx_lock_flags+0xef
uma_zalloc_arg(c1472d20,0,2,2,54e6a,...) at uma_zalloc_arg+0xd3
malloc(34,c0b69ea0,2,d8,0,...) at malloc+0xd2
lf_advlock(e9820c20,c8d596e8,100000,0) at lf_advlock+0x183
ufs_advlock(e9820c20,c0ae7052,c95be200,c95be200,16,...) at ufs_advlock+0x2d
VOP_ADVLOCK_APV(c0b86340,e9820c20,c0ab09dc,226,e9820bd4,...) at VOP_ADVLOCK_APV+0xd6
kern_fcntl(c92b3660,3,9,e9820c60,54e6a,...) at kern_fcntl+0x8bf
fcntl(c92b3660,e9820cfc,c,0,c0b65f20,...) at fcntl+0x57
syscall(e9820d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (92, FreeBSD ELF32, fcntl), eip = 0x281433bf, esp = 0xbfbfe8ec, ebp = 0xbfbfe928 ---
Tracing command lockf2 pid 5078 tid 100317 td 0xc9f41880
sched_switch(c9f41880,0,2,17b,2b2912d9,...) at sched_switch+0x49b
mi_switch(2,0,c0ab8ccd,ec,c9f41880,...) at mi_switch+0x217
ast(e97fcd38) at ast+0x2b7
doreti_ast() at doreti_ast+0x17
Tracing command lockf2 pid 5077 tid 100385 td 0xc9f64000
sched_switch(c9f64000,0,1,180,67960d26,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c9f64000,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e98b3bc4,c075356d,c0be4100,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,2711,...) at _sleep+0x331
kern_nanosleep(c9f64000,e98b3c64,e98b3c6c,a,0,...) at kern_nanosleep+0xc1
nanosleep(c9f64000,e98b3cfc,8,e98b3d38,c0b66d00,...) at nanosleep+0x6f
syscall(e98b3d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813754f, esp = 0xbfbfe93c, ebp = 0xbfbfe968 ---
Tracing command swap pid 5075 tid 100307 td 0xc920b220
sched_switch(c920b220,0,2,17b,2dcc8de6,...) at sched_switch+0x49b
mi_switch(2,0,c0ab8ccd,ec,1,...) at mi_switch+0x217
ast(e976fd38) at ast+0x2b7
doreti_ast() at doreti_ast+0x17
Tracing command swap pid 5073 tid 100150 td 0xc739faa0
sched_switch(c739faa0,0,2,17b,3029451d,...) at sched_switch+0x49b
mi_switch(2,0,c0ab8ccd,ec,1,...) at mi_switch+0x217
ast(e94d7d38) at ast+0x2b7
doreti_ast() at doreti_ast+0x17
Tracing command swap pid 5071 tid 100160 td 0xc739e440
cpustop_handler(4,e94ffc24,c0a2326d,290fc000,29101000,...) at cpustop_handler+0x32
ipi_nmi_handler(290fc000,29101000,1,c8eb564c,c6fa5aa0,...) at ipi_nmi_handler+0x2f
trap(e94ffc30) at trap+0x2d
calltrap() at calltrap+0x6
--- trap 0x13, eip = 0xc09fa9bf, esp = 0xe94ffc70, ebp = 0xe94ffc94 ---
siointr1(c0c46a6c,0,c0adeb06,56f,e94ffcd4,...) at siointr1+0x8f
siointr(c691ec00,c0aafe7a,216,c739e440,c6778e80,...) at siointr+0x32
intr_execute_handlers(c676fcd0,e94ffcf4,bfbfe8e8,c0a09624,38,...) at intr_execute_handlers+0xec
lapic_handle_intr(38,e94ffcf4) at lapic_handle_intr+0x3f
Xapic_isr1() at Xapic_isr1+0x34
--- interrupt, eip = 0xc0a09b78, esp = 0xe94ffd34, ebp = 0xbfbfe8e8 ---
doreti_ast() at doreti_ast+0x12
Tracing command swap pid 5069 tid 100049 td 0xc6c94880
sched_switch(c6c94880,0,1,180,93bcb762,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c6c94880,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e9313bc4,c075356d,c0be4080,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,fa1,...) at _sleep+0x331
kern_nanosleep(c6c94880,e9313c64,e9313c6c,4,0,...) at kern_nanosleep+0xc1
nanosleep(c6c94880,e9313cfc,8,c0ae75e4,c0b66d00,...) at nanosleep+0x6f
syscall(e9313d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813854f, esp = 0xbfbfe84c, ebp = 0xbfbfe878 ---
Tracing command lockf2 pid 5068 tid 100115 td 0xc6e78660
sched_switch(c6e78660,0,1,17b,2a8a46f3,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8d64,2e2,c92c2aa0,...) at mi_switch+0x217
turnstile_wait(c92c2aa0,c9f56000,0,192,1,...) at turnstile_wait+0x4cb
_mtx_lock_sleep(c1474808,c6e78660,0,c0ad6278,726,...) at _mtx_lock_sleep+0x19b
_mtx_lock_flags(c1474808,0,c0ad6278,726,c0be4200,...) at _mtx_lock_flags+0xef
uma_zalloc_arg(c1472d20,0,2,2,ac54a,...) at uma_zalloc_arg+0xd3
malloc(34,c0b69ea0,2,d8,0,...) at malloc+0xd2
lf_advlock(e9431c20,c8e19034,100000,0) at lf_advlock+0x183
ufs_advlock(e9431c20,c0ae7052,ca13b800,ca13b800,16,...) at ufs_advlock+0x2d
VOP_ADVLOCK_APV(c0b86340,e9431c20,210,20f,0,...) at VOP_ADVLOCK_APV+0xd6
kern_fcntl(c6e78660,3,8,e9431c60,ac54a,...) at kern_fcntl+0x8ff
fcntl(c6e78660,e9431cfc,c,c0aba75c,c0b65f20,...) at fcntl+0x57
syscall(e9431d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (92, FreeBSD ELF32, fcntl), eip = 0x281433bf, esp = 0xbfbfe8ec, ebp = 0xbfbfe928 ---
Tracing command swap pid 5067 tid 100116 td 0xc6e78440
sched_switch(c6e78440,0,1,180,71d909f1,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c6e78440,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e9434bc4,c075356d,c0be4300,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,7d1,...) at _sleep+0x331
kern_nanosleep(c6e78440,e9434c64,e9434c6c,2,0,...) at kern_nanosleep+0xc1
nanosleep(c6e78440,e9434cfc,8,c0ae75e4,c0b66d00,...) at nanosleep+0x6f
syscall(e9434d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813854f, esp = 0xbfbfe84c, ebp = 0xbfbfe878 ---
Tracing command swap pid 5065 tid 100054 td 0xc68b6aa0
sched_switch(c68b6aa0,0,1,180,292ae09,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c68b6aa0,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e9327bc4,c075356d,c0be4100,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,fa1,...) at _sleep+0x331
kern_nanosleep(c68b6aa0,e9327c64,e9327c6c,4,0,...) at kern_nanosleep+0xc1
nanosleep(c68b6aa0,e9327cfc,8,c0ae75e4,c0b66d00,...) at nanosleep+0x6f
syscall(e9327d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813854f, esp = 0xbfbfe84c, ebp = 0xbfbfe878 ---
Tracing command swap pid 5062 tid 100165 td 0xc74bf220
sched_switch(c74bf220,0,2,17b,2f17471e,...) at sched_switch+0x49b
mi_switch(2,0,c0ab8ccd,ec,1,...) at mi_switch+0x217
ast(e951dd38) at ast+0x2b7
doreti_ast() at doreti_ast+0x17
Tracing command lockf2 pid 5060 tid 100330 td 0xc9255660
cpustop_handler(2,e9817c10,c0a2326d,c6dcd364,0,...) at cpustop_handler+0x32
ipi_nmi_handler(c6dcd364,0,c0ac129b,c922baa0,c922baa0,...) at ipi_nmi_handler+0x2f
trap(e9817c1c) at trap+0x2d
calltrap() at calltrap+0x6
--- trap 0x13, eip = 0xc0760076, esp = 0xe9817c5c, ebp = 0xe9817c80 ---
panic(c0ae4901,c0aa4b83,1,c0aa4b83,c0b656e0,...) at panic+0x26
syscall(e9817d38) at syscall+0x46e
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (1024), eip = 0x281547e3, esp = 0xbfbfe53c, ebp = 0xbfbfe968 ---
Tracing command swap pid 5059 tid 100382 td 0xc9f57aa0
sched_switch(c9f57aa0,0,1,180,675d33a1,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c9f57aa0,0,c0ab8594,19b,c6b2f2a8,...) at sleepq_switch+0xf0
sleepq_catch_signals(c075356d,c0be4100,4,c0ab37db,5c,...) at sleepq_catch_signals+0x21e
sleepq_wait_sig(c6b2f2a8,0,c0ab54cf,da,0,...) at sleepq_wait_sig+0x14
_sleep(c6b2f2a8,c6b2f338,15c,c0aba709,0,...) at _sleep+0x359
kern_wait(c9f57aa0,13c6,e98aac2c,0,0,...) at kern_wait+0xa66
wait4(c9f57aa0,e98aacfc,10,e98aad38,c0b65728,...) at wait4+0x3b
syscall(e98aad38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x280d902b, esp = 0xbfbfe96c, ebp = 0xbfbfe988 ---
Tracing command lockf2 pid 5058 tid 100331 td 0xc91f6440
sched_switch(c91f6440,0,6,17b,e18afff8,...) at sched_switch+0x49b
mi_switch(6,0,c0ae307c,47e,6,...) at mi_switch+0x217
ipi_bitmap_handler(e9810008,c0790028,c0ad0028,c91f6440,0,...) at ipi_bitmap_handler+0x72
Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0x2e
--- interrupt, eip = 0x20, esp = 0xc91f6440, ebp = 0xe981aa44 ---
uart_z8530_class(c0753b50,20,246,c1474808,c91f6440,...) at 0x20
Tracing command lockf2 pid 5055 tid 100107 td 0xc6e66cc0
sched_switch(c6e66cc0,0,1,180,65b07f9b,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c6e66cc0,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e9419bc4,c075356d,c0be4000,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,2711,...) at _sleep+0x331
kern_nanosleep(c6e66cc0,e9419c64,e9419c6c,a,0,...) at kern_nanosleep+0xc1
nanosleep(c6e66cc0,e9419cfc,8,e9419d38,c0b66d00,...) at nanosleep+0x6f
syscall(e9419d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813754f, esp = 0xbfbfe93c, ebp = 0xbfbfe968 ---
Tracing command lockf2 pid 5052 tid 100408 td 0xc9199880
sched_switch(c9199880,0,1,180,a875c185,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c9199880,0,c0ab8594,19b,c9310d48,...) at sleepq_switch+0xf0
sleepq_catch_signals(c0ab8594,155,0,100,5c,...) at sleepq_catch_signals+0x21e
sleepq_wait_sig(c9310d48,c9310dd8,c0aba709,100,0,...) at sleepq_wait_sig+0x14
_sleep(c9310d48,c9310dd8,15c,c0aba709,0,...) at _sleep+0x359
kern_wait(c9199880,13bf,e98f8c2c,0,0,...) at kern_wait+0xa66
wait4(c9199880,e98f8cfc,10,c0aba52a,c0b65728,...) at wait4+0x3b
syscall(e98f8d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x280d802b, esp = 0xbfbfe96c, ebp = 0xbfbfe988 ---
Tracing command swap pid 5051 tid 100407 td 0xc92de000
sched_switch(c92de000,0,1,180,e0fe2634,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c92de000,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e98f5bc4,c075356d,c0be4080,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,3e9,...) at _sleep+0x331
kern_nanosleep(c92de000,e98f5c64,e98f5c6c,1,0,...) at kern_nanosleep+0xc1
nanosleep(c92de000,e98f5cfc,8,c0aba84d,c0b66d00,...) at nanosleep+0x6f
syscall(e98f5d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813854f, esp = 0xbfbfe98c, ebp = 0xbfbfe9b8 ---
Tracing command lockf2 pid 5049 tid 100405 td 0xc92ca440
sched_switch(c92ca440,0,1,180,e0363a47,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c92ca440,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e98efbc4,c075356d,c0be4300,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,3e9,...) at _sleep+0x331
kern_nanosleep(c92ca440,e98efc64,e98efc6c,1,0,...) at kern_nanosleep+0xc1
nanosleep(c92ca440,e98efcfc,8,c0aba84d,c0b66d00,...) at nanosleep+0x6f
syscall(e98efd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813754f, esp = 0xbfbfe98c, ebp = 0xbfbfe9b8 ---
Tracing command run pid 4769 tid 100266 td 0xc9e67440
sched_switch(c9e67440,0,1,180,65c95e61,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c9e67440,0,c0ab8594,19b,c9e617f8,...) at sleepq_switch+0xf0
sleepq_catch_signals(c0ab8594,155,0,100,5c,...) at sleepq_catch_signals+0x21e
sleepq_wait_sig(c9e617f8,c9e61888,c0aba709,100,0,...) at sleepq_wait_sig+0x14
_sleep(c9e617f8,c9e61888,15c,c0aba709,0,...) at _sleep+0x359
kern_wait(c9e67440,13b9,e9698c2c,0,0,...) at kern_wait+0xa66
wait4(c9e67440,e9698cfc,10,e9698d38,c0b65728,...) at wait4+0x3b
syscall(e9698d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x280d802b, esp = 0xbfbfe45c, ebp = 0xbfbfe478 ---
Tracing command run pid 4768 tid 100328 td 0xc9f5e880
Tracing command run pid 4767 tid 100077 td 0xc6d03880
sched_switch(c6d03880,0,1,180,d085ea9c,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c6d03880,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e9384bc4,c075356d,c0be4180,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,3e9,...) at _sleep+0x331
kern_nanosleep(c6d03880,e9384c64,e9384c6c,1,0,...) at kern_nanosleep+0xc1
nanosleep(c6d03880,e9384cfc,8,c0aba84d,c0b66d00,...) at nanosleep+0x6f
syscall(e9384d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813754f, esp = 0xbfbfe92c, ebp = 0xbfbfe958 ---
Tracing command sh pid 4761 tid 100180 td 0xc6e62880
Tracing command sleep pid 4615 tid 100288 td 0xc9e64440
Tracing command sh pid 834 tid 100113 td 0xc6e78aa0
Tracing command sh pid 807 tid 100218 td 0xc7501cc0
Tracing command tcsh pid 800 tid 100212 td 0xc6ec8880
Tracing command watchdogd pid 683 tid 100137 td 0xc704fcc0
sched_switch(c704fcc0,0,1,180,dfbbbe58,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c704fcc0,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e94aabc4,c075356d,c0be4000,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,ea61,...) at _sleep+0x331
kern_nanosleep(c704fcc0,e94aac64,e94aac6c,3c,0,...) at kern_nanosleep+0xc1
nanosleep(c704fcc0,e94aacfc,8,c0ae7564,c0b66d00,...) at nanosleep+0x6f
syscall(e94aad38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2815854f, esp = 0xbfbfeb6c, ebp = 0xbfbfeb98 ---
Tracing command tcsh pid 659 tid 100136 td 0xc7051000
Tracing command sshd pid 658 tid 100135 td 0xc7051220
sched_switch(c7051220,0,1,180,2b49e453,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c7051220,0,c0ab8594,19b,c0c2d6bc,...) at sleepq_switch+0xf0
sleepq_catch_signals(c075391a,c0c2d6a4,0,c0ab37db,c7051220,...) at sleepq_catch_signals+0x21e
sleepq_wait_sig(c0c2d6bc,c0c2d6a4,c0ad238d,101,0,...) at sleepq_wait_sig+0x14
_cv_wait_sig(c0c2d6bc,c0c2d6a4,c0aba178,30f,c0acbb17,...) at _cv_wait_sig+0x1fc
kern_select(c7051220,9,285030d4,285030d8,0,0,0,280df5e8) at kern_select+0x6cc
select(c7051220,e94a4cfc,14,c0aa4b83,c0b65f38,...) at select+0x5e
syscall(e94a4d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x283e2783, esp = 0xbfbfe5fc, ebp = 0xbfbfe648 ---
Tracing command sshd pid 655 tid 100132 td 0xc7051880
Tracing command login pid 654 tid 100131 td 0xc6e67440
Tracing command getty pid 653 tid 100130 td 0xc6e67660
Tracing command getty pid 652 tid 100129 td 0xc6e67880
Tracing command getty pid 651 tid 100128 td 0xc6e67aa0
Tracing command getty pid 650 tid 100091 td 0xc6d62440
Tracing command getty pid 649 tid 100110 td 0xc6e79220
Tracing command getty pid 648 tid 100118 td 0xc6e78000
Tracing command getty pid 647 tid 100114 td 0xc6e78880
Tracing command getty pid 646 tid 100111 td 0xc6e79000
Tracing command cron pid 608 tid 100090 td 0xc6d62660
sched_switch(c6d62660,0,1,180,3bcb6786,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c6d62660,0,c0ab8594,19b,c0bdc124,...) at sleepq_switch+0xf0
sleepq_catch_signals(e93abbc4,c075356d,c0be3f80,4,5c,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bdc124,0,c0ab54cf,da,0,...) at sleepq_timedwait_sig+0x17
_sleep(c0bdc124,0,15c,c0ab5ccf,e679,...) at _sleep+0x331
kern_nanosleep(c6d62660,e93abc64,e93abc6c,3b,0,...) at kern_nanosleep+0xc1
nanosleep(c6d62660,e93abcfc,8,c0aba84d,c0b66d00,...) at nanosleep+0x6f
syscall(e93abd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2815054f, esp = 0xbfbfed0c, ebp = 0xbfbfed38 ---
Tracing command sendmail pid 602 tid 100073 td 0xc6d03cc0
Tracing command sendmail pid 598 tid 100083 td 0xc6d63660
sched_switch(c6d63660,0,1,180,76d81df6,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c6d63660,0,c0ab8594,19b,c0c2d6bc,...) at sleepq_switch+0xf0
sleepq_catch_signals(e9396a84,c075391a,c0c2d6a4,0,c6d63660,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0c2d6bc,1389,c0ad238d,101,0,...) at sleepq_timedwait_sig+0x17
_cv_timedwait_sig(c0c2d6bc,c0c2d6a4,1389,30f,c0750a9d,...) at _cv_timedwait_sig+0x20b
kern_select(c6d63660,4,bfbfd92c,0,0,e9396c70,5,0) at kern_select+0x6b4
select(c6d63660,e9396cfc,14,c0abab02,c0b65f38,...) at select+0x5e
syscall(e9396d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x2839b783, esp = 0xbfbfd09c, ebp = 0xbfbfdb38 ---
Tracing command sshd pid 592 tid 100092 td 0xc6d62220
Tracing command rpc.lockd pid 539 tid 100061 td 0xc6927440
Tracing command rpc.lockd pid 533 tid 100082 td 0xc6d63880
sched_switch(c6d63880,0,1,180,5ba5e4da,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c6d63880,0,c0ab8594,19b,c0c2d6bc,...) at sleepq_switch+0xf0
sleepq_catch_signals(e9393a84,c075391a,c0c2d6a4,0,c6d63880,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0c2d6bc,7531,c0ad238d,101,0,...) at sleepq_timedwait_sig+0x17
_cv_timedwait_sig(c0c2d6bc,c0c2d6a4,7531,30f,e9393afc,...) at _cv_timedwait_sig+0x20b
kern_select(c6d63880,7,bfbfed54,0,0,e9393c70,1e,0) at kern_select+0x6b4
select(c6d63880,e9393cfc,14,c0aba84d,c0b65f38,...) at select+0x5e
syscall(e9393d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x28170783, esp = 0xbfbfecac, ebp = 0xbfbfede8 ---
Tracing command rpc.statd pid 528 tid 100074 td 0xc6d32220
sched_switch(c6d32220,0,1,180,18e92ed7,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c6d32220,0,c0ab8594,19b,c0c2d6bc,...) at sleepq_switch+0xf0
sleepq_catch_signals(e9379a84,c075391a,c0c2d6a4,0,c6d32220,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0c2d6bc,7531,c0ad238d,101,0,...) at sleepq_timedwait_sig+0x17
_cv_timedwait_sig(c0c2d6bc,c0c2d6a4,7531,30f,e9379afc,...) at _cv_timedwait_sig+0x20b
kern_select(c6d32220,7,bfbfed64,0,0,e9379c70,1e,0) at kern_select+0x6b4
select(c6d32220,e9379cfc,14,c0aba84d,c0b65f38,...) at select+0x5e
syscall(e9379d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x2815c783, esp = 0xbfbfecbc, ebp = 0xbfbfedf8 ---
Tracing command md1 pid 498 tid 100093 td 0xc6d62000
sched_switch(c6d62000,0,1,180,2b341ef5,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,c6d62000,...) at mi_switch+0x217
sleepq_switch(c6d62000,0,c0ab8594,21d,4c,...) at sleepq_switch+0xf0
sleepq_wait(c6de1000,c6de1020,c0a91f9f,0,0,...) at sleepq_wait+0x60
_sleep(c6de1000,c6de1020,24c,c0a91f9f,0,...) at _sleep+0x369
md_kthread(c6de1000,e93e8d38,c0ab13b9,30c,c6af3000,...) at md_kthread+0x125
fork_exit(c05e59c0,c6de1000,e93e8d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe93e8d70, ebp = 0 ---
Tracing command rpcbind pid 478 tid 100069 td 0xc6d04440
sched_switch(c6d04440,0,1,180,2a0c11c3,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c6d04440,0,c0ab8594,19b,c0c2d6bc,...) at sleepq_switch+0xf0
sleepq_catch_signals(e9364ad8,c075391a,c0c2d6a4,0,c6d04440,...) at sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0c2d6bc,7531,c0ad238d,101,0,...) at sleepq_timedwait_sig+0x17
_cv_timedwait_sig(c0c2d6bc,c0c2d6a4,7531,3bb,0,...) at _cv_timedwait_sig+0x20b
poll(c6d04440,e9364cfc,c,c0aba84d,c0b66a18,...) at poll+0x554
syscall(e9364d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (209, FreeBSD ELF32, poll), eip = 0x2811b333, esp = 0xbfbfcc7c, ebp = 0xbfbfee48 ---
Tracing command syslogd pid 456 tid 100079 td 0xc6d03440
sched_switch(c6d03440,0,1,180,c248f932,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c6d03440,0,c0ab8594,19b,c0c2d6bc,...) at sleepq_switch+0xf0
sleepq_catch_signals(c075391a,c0c2d6a4,0,c0ab37db,c6d03440,...) at sleepq_catch_signals+0x21e
sleepq_wait_sig(c0c2d6bc,c0c2d6a4,c0ad238d,101,0,...) at sleepq_wait_sig+0x14
_cv_wait_sig(c0c2d6bc,c0c2d6a4,c0aba178,30f,b,...) at _cv_wait_sig+0x1fc
kern_select(c6d03440,9,282200bc,0,0,0,c6dcd5d0,28203000) at kern_select+0x6cc
select(c6d03440,e938acfc,14,c0ad3edb,c0b65f38,...) at select+0x5e
syscall(e938ad38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x28166783, esp = 0xbfbfdf3c, ebp = 0xbfbfee88 ---
Tracing command devd pid 410 tid 100071 td 0xc6d04220
Tracing command md0 pid 94 tid 100057 td 0xc6927aa0
sched_switch(c6927aa0,0,1,180,ae8d6331,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,c6927aa0,...) at mi_switch+0x217
sleepq_switch(c6927aa0,0,c0ab8594,21d,4c,...) at sleepq_switch+0xf0
sleepq_wait(c6d56800,c6d56820,c0a91f9f,0,0,...) at sleepq_wait+0x60
_sleep(c6d56800,c6d56820,24c,c0a91f9f,0,...) at _sleep+0x369
md_kthread(c6d56800,e9333d38,c0ab13b9,30c,c6c92000,...) at md_kthread+0x125
fork_exit(c05e59c0,c6d56800,e9333d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe9333d70, ebp = 0 ---
Tracing command softdepflush pid 27 tid 100046 td 0xc68b6cc0
sched_switch(c68b6cc0,0,1,180,c6d96eac,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,44,...) at mi_switch+0x217
sleepq_switch(c68b6cc0,0,c0ab8594,240,44,...) at sleepq_switch+0xf0
sleepq_timedwait(c0c394a4,3e8,c0ad4358,0,0,...) at sleepq_timedwait+0x68
_sleep(c0c394a4,c0c3947c,44,c0ad4358,3e8,...) at _sleep+0x345
softdep_flush(0,e711dd38,c0ab13b9,30c,c6af42a8,...) at softdep_flush+0x2d2
fork_exit(c0947e40,0,e711dd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe711dd70, ebp = 0 ---
Tracing command syncer pid 26 tid 100045 td 0xc6926000
sched_switch(c6926000,0,1,180,daa5f1d0,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,c6926000,...) at mi_switch+0x217
sleepq_switch(c6926000,0,c0ab8594,21d,68,...) at sleepq_switch+0xf0
sleepq_wait(c0bdbf4c,0,c0ac0dcc,0,0,...) at sleepq_wait+0x60
_sleep(c0bdbf4c,c0c2de44,68,c0ac0dcc,0,...) at _sleep+0x369
sched_sync(0,e711ad38,c0ab13b9,30c,c6af4550,...) at sched_sync+0x56e
fork_exit(c07df8e0,0,e711ad38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe711ad70, ebp = 0 ---
Tracing command vnlru pid 25 tid 100044 td 0xc6926220
sched_switch(c6926220,0,1,180,d04c500b,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,50,...) at mi_switch+0x217
sleepq_switch(c6926220,0,c0ab8594,240,50,...) at sleepq_switch+0xf0
sleepq_timedwait(c6af47f8,3e8,c0ac0e76,0,0,...) at sleepq_timedwait+0x68
_sleep(c6af47f8,c0c2de18,250,c0ac0e76,3e8,...) at _sleep+0x345
vnlru_proc(0,e7117d38,c0ab13b9,30c,c6af47f8,...) at vnlru_proc+0x114
fork_exit(c07e06b0,0,e7117d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe7117d70, ebp = 0 ---
Tracing command bufdaemon pid 24 tid 100043 td 0xc6926440
sched_switch(c6926440,0,1,180,214f1f63,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,44,...) at mi_switch+0x217
sleepq_switch(c6926440,0,c0ab8594,240,44,...) at sleepq_switch+0xf0
sleepq_timedwait(c0c2db64,3e8,c0abe936,0,0,...) at sleepq_timedwait+0x68
_sleep(c0c2db64,c0c2db68,44,c0abe936,3e8,...) at _sleep+0x345
buf_daemon(0,e7114d38,c0ab13b9,30c,c67f97f8,...) at buf_daemon+0x21e
fork_exit(c07cab10,0,e7114d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe7114d70, ebp = 0 ---
Tracing command pagezero pid 23 tid 100042 td 0xc6926660
sched_switch(c6926660,0,1,180,dbf3a9d2,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c6926660,0,c0ab8594,240,0,...) at sleepq_switch+0xf0
sleepq_timedwait(c0c3a060,493e0,c0ad903b,0,0,...) at sleepq_timedwait+0x68
_sleep(c0c3a060,c0c39c04,0,c0ad903b,493e0,...) at _sleep+0x345
vm_pagezero(0,e7111d38,c0ab13b9,30c,c67f9aa0,...) at vm_pagezero+0xdc
fork_exit(c09820d0,0,e7111d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe7111d70, ebp = 0 ---
Tracing command vmdaemon pid 22 tid 100041 td 0xc6926880
sched_switch(c6926880,0,1,180,a6292b8c,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,c6926880,...) at mi_switch+0x217
sleepq_switch(c6926880,0,c0ab8594,21d,68,...) at sleepq_switch+0xf0
sleepq_wait(c0c39c78,c0c39c7c,c0abe936,0,0,...) at sleepq_wait+0x60
_sleep(c0c39c78,c0c39c7c,68,c0abe936,0,...) at _sleep+0x369
vm_daemon(0,e710ed38,c0ab13b9,30c,c67f9d48,...) at vm_daemon+0x59
fork_exit(c097d4f0,0,e710ed38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe710ed70, ebp = 0 ---
Tracing command pagedaemon pid 21 tid 100040 td 0xc6926aa0
sched_switch(c6926aa0,0,1,180,ae3b8718,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,44,...) at mi_switch+0x217
sleepq_switch(c6926aa0,0,c0ab8594,240,44,...) at sleepq_switch+0xf0
sleepq_timedwait(c0c39c40,1388,c0abe936,0,0,...) at sleepq_timedwait+0x68
_sleep(c0c39c40,c0c39c04,44,c0abe936,1388,...) at _sleep+0x345
vm_pageout(0,e710bd38,c0ab13b9,30c,c68b7000,...) at vm_pageout+0x2bb
fork_exit(c097e0a0,0,e710bd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe710bd70, ebp = 0 ---
Tracing command sctp_iterator pid 20 tid 100039 td 0xc6926cc0
sched_switch(c6926cc0,0,1,180,36224976,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,c6926cc0,...) at mi_switch+0x217
sleepq_switch(c6926cc0,0,c0ab8594,21d,0,...) at sleepq_switch+0xf0
sleepq_wait(c0c2f8ec,c0c2f7f4,c0ac7b18,0,0,...) at sleepq_wait+0x60
_sleep(c0c2f8ec,c0c2f7f4,0,c0ac7b18,0,...) at _sleep+0x369
sctp_iterator_thread(0,e7108d38,c0ab13b9,30c,c68b72a8,...) at sctp_iterator_thread+0x60
fork_exit(c0846b80,0,e7108d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe7108d70, ebp = 0 ---
Tracing command usb3 pid 19 tid 100034 td 0xc68b5440
sched_switch(c68b5440,0,1,180,5a6987ff,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,5c,...) at mi_switch+0x217
sleepq_switch(c68b5440,0,c0ab8594,240,5c,...) at sleepq_switch+0xf0
sleepq_timedwait(c6865210,ea60,c0aa91f6,0,0,...) at sleepq_timedwait+0x68
_sleep(c6865210,0,5c,c0aa91f6,ea60,...) at _sleep+0x345
usb_event_thread(c68c7a00,e6ed8d38,c0ab13b9,30c,c68b7550,...) at usb_event_thread+0xca
fork_exit(c06da230,c68c7a00,e6ed8d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6ed8d70, ebp = 0 ---
Tracing command usb2 pid 18 tid 100033 td 0xc68b5660
sched_switch(c68b5660,0,1,180,62c1cdd3,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,5c,...) at mi_switch+0x217
sleepq_switch(c68b5660,0,c0ab8594,240,5c,...) at sleepq_switch+0xf0
sleepq_timedwait(c68be210,ea60,c0aa91f6,0,0,...) at sleepq_timedwait+0x68
_sleep(c68be210,0,5c,c0aa91f6,ea60,...) at _sleep+0x345
usb_event_thread(c68a8140,e6ec5d38,c0ab13b9,30c,c68b77f8,...) at usb_event_thread+0xca
fork_exit(c06da230,c68a8140,e6ec5d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6ec5d70, ebp = 0 ---
Tracing command usb1 pid 17 tid 100031 td 0xc68b5aa0
sched_switch(c68b5aa0,0,1,180,626c43f7,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,5c,...) at mi_switch+0x217
sleepq_switch(c68b5aa0,0,c0ab8594,240,5c,...) at sleepq_switch+0xf0
sleepq_timedwait(c68ac210,ea60,c0aa91f6,0,0,...) at sleepq_timedwait+0x68
_sleep(c68ac210,0,5c,c0aa91f6,ea60,...) at _sleep+0x345
usb_event_thread(c68a8900,e6ebed38,c0ab13b9,30c,c68b7aa0,...) at usb_event_thread+0xca
fork_exit(c06da230,c68a8900,e6ebed38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6ebed70, ebp = 0 ---
Tracing command usbtask-dr pid 16 tid 100029 td 0xc68b6000
sched_switch(c68b6000,0,1,180,c52143a0,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,c68b6000,...) at mi_switch+0x217
sleepq_switch(c68b6000,0,c0ab8594,21d,5c,...) at sleepq_switch+0xf0
sleepq_wait(c0bcc814,0,c0ab54cf,da,0,...) at sleepq_wait+0x60
_sleep(c0bcc814,0,5c,c0aa9233,0,...) at _sleep+0x369
usb_task_thread(c0bcc814,e6eb7d38,c0ab13b9,30c,c68b7d48,...) at usb_task_thread+0x62
fork_exit(c06da9b0,c0bcc814,e6eb7d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6eb7d70, ebp = 0 ---
Tracing command usbtask-hc pid 15 tid 100028 td 0xc68b6220
sched_switch(c68b6220,0,1,180,c5212c77,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,c68b6220,...) at mi_switch+0x217
sleepq_switch(c68b6220,0,c0ab8594,21d,5c,...) at sleepq_switch+0xf0
sleepq_wait(c0bcc800,0,c0ab54cf,da,0,...) at sleepq_wait+0x60
_sleep(c0bcc800,0,5c,c0aa9233,0,...) at _sleep+0x369
usb_task_thread(c0bcc800,e6eb4d38,c0ab13b9,30c,c677c2a8,...) at usb_task_thread+0x62
fork_exit(c06da9b0,c0bcc800,e6eb4d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6eb4d70, ebp = 0 ---
Tracing command usb0 pid 14 tid 100027 td 0xc68b6440
sched_switch(c68b6440,0,1,180,636ca826,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,5c,...) at mi_switch+0x217
sleepq_switch(c68b6440,0,c0ab8594,240,5c,...) at sleepq_switch+0xf0
sleepq_timedwait(c6896210,ea60,c0aa91f6,0,0,...) at sleepq_timedwait+0x68
_sleep(c6896210,0,5c,c0aa91f6,ea60,...) at _sleep+0x345
usb_event_thread(c68a9700,e6eb1d38,c0ab13b9,30c,c677c550,...) at usb_event_thread+0xca
fork_exit(c06da230,c68a9700,e6eb1d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6eb1d70, ebp = 0 ---
Tracing command em1 taskq pid 9 tid 100025 td 0xc6788880
sched_switch(c6788880,0,1,180,c51cbab8,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,c6788880,...) at mi_switch+0x217
sleepq_switch(c6788880,0,c0ab8594,21d,c6788880,...) at sleepq_switch+0xf0
sleepq_wait(c689ce00,0,c0ab55e3,c0aac335,0,...) at sleepq_wait+0x60
msleep_spin(c689ce00,c689ce1c,c0aac335,0,c0ab37db,...) at msleep_spin+0x20d
taskqueue_thread_loop(c6864b14,e6eaad38,c0ab13b9,30c,c677c7f8,...) at taskqueue_thread_loop+0x8a
fork_exit(c0792c40,c6864b14,e6eaad38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6eaad70, ebp = 0 ---
Tracing command em0 taskq pid 8 tid 100024 td 0xc6788aa0
sched_switch(c6788aa0,0,1,180,2b471ecf,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,c6788aa0,...) at mi_switch+0x217
sleepq_switch(c6788aa0,0,c0ab8594,21d,c6788aa0,...) at sleepq_switch+0xf0
sleepq_wait(c687a980,0,c0ab55e3,c0aac335,0,...) at sleepq_wait+0x60
msleep_spin(c687a980,c687a99c,c0aac335,0,c0ab37db,...) at msleep_spin+0x20d
taskqueue_thread_loop(c6862b14,e6e87d38,c0ab13b9,30c,c677caa0,...) at taskqueue_thread_loop+0x8a
fork_exit(c0792c40,c6862b14,e6e87d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6e87d70, ebp = 0 ---
Tracing command xpt_thrd pid 7 tid 100021 td 0xc686f220
sched_switch(c686f220,0,1,180,c520eea4,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,c686f220,...) at mi_switch+0x217
sleepq_switch(c686f220,0,c0ab8594,21d,4c,...) at sleepq_switch+0xf0
sleepq_wait(c0b9d9d4,0,c0ab54cf,da,0,...) at sleepq_wait+0x60
_sleep(c0b9d9d4,c0b9d9ec,4c,c0a6a7ce,0,...) at _sleep+0x369
xpt_scanner_thread(0,e6e5ed38,c0ab13b9,30c,c677cd48,...) at xpt_scanner_thread+0x41
fork_exit(c0468df0,0,e6e5ed38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6e5ed70, ebp = 0 ---
Tracing command kqueue taskq pid 6 tid 100020 td 0xc686f440
sched_switch(c686f440,0,1,180,c55ce526,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,c686f440,...) at mi_switch+0x217
sleepq_switch(c686f440,0,c0ab8594,21d,0,...) at sleepq_switch+0xf0
sleepq_wait(c67c2700,c67c271c,c0aac335,0,0,...) at sleepq_wait+0x60
_sleep(c67c2700,c67c271c,0,c0aac335,0,...) at _sleep+0x369
taskqueue_thread_loop(c0bcd76c,e6e5bd38,c0ab13b9,30c,c67f9000,...) at taskqueue_thread_loop+0xb4
fork_exit(c0792c40,c0bcd76c,e6e5bd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6e5bd70, ebp = 0 ---
Tracing command thread taskq pid 5 tid 100017 td 0xc686faa0
sched_switch(c686faa0,0,1,180,c55ccbfe,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,c686faa0,...) at mi_switch+0x217
sleepq_switch(c686faa0,0,c0ab8594,21d,0,...) at sleepq_switch+0xf0
sleepq_wait(c67c2980,c67c299c,c0aac335,0,0,...) at sleepq_wait+0x60
_sleep(c67c2980,c67c299c,0,c0aac335,0,...) at _sleep+0x369
taskqueue_thread_loop(c0be60d8,e6e52d38,c0ab13b9,30c,c67f92a8,...) at taskqueue_thread_loop+0xb4
fork_exit(c0792c40,c0be60d8,e6e52d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6e52d70, ebp = 0 ---
Tracing command yarrow pid 13 tid 100016 td 0xc677f220
sched_switch(c677f220,0,1,180,2dced1dc,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c677f220,0,c0ab8594,240,0,...) at sleepq_switch+0xf0
sleepq_timedwait(c0bdbf54,64,c0aac335,2,0,...) at sleepq_timedwait+0x68
_sleep(c0bdbf54,0,0,c0aac335,64,...) at _sleep+0x345
pause(c0aac335,64,c0aa0114,112,0,...) at pause+0x47
random_kthread(0,e6e4fd38,c0ab13b9,30c,c67f9550,...) at random_kthread+0x1d9
fork_exit(c065a170,0,e6e4fd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6e4fd70, ebp = 0 ---
Tracing command g_down pid 4 tid 100015 td 0xc677f440
sched_switch(c677f440,0,1,180,2b24de22,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,4c,...) at mi_switch+0x217
sleepq_switch(c677f440,0,c0ab8594,240,4c,...) at sleepq_switch+0xf0
sleepq_timedwait(c0bccf6c,64,c0aac335,0,0,...) at sleepq_timedwait+0x68
_sleep(c0bccf6c,c0bcce88,24c,c0aac335,64,...) at _sleep+0x345
g_io_schedule_down(c677f440,0,c0aad9f8,74,0,...) at g_io_schedule_down+0x6b
g_down_procbody(0,e6e48d38,c0ab13b9,30c,c677b000,...) at g_down_procbody+0x8d
fork_exit(c0711ec0,0,e6e48d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6e48d70, ebp = 0 ---
Tracing command g_up pid 3 tid 100014 td 0xc677f660
sched_switch(c677f660,0,1,180,2b4f1397,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,4c,...) at mi_switch+0x217
sleepq_switch(c677f660,0,c0ab8594,240,4c,...) at sleepq_switch+0xf0
sleepq_timedwait(c0bccf68,64,c0aac335,0,0,...) at sleepq_timedwait+0x68
_sleep(c0bccf68,c0bccec8,24c,c0aac335,64,...) at _sleep+0x345
g_io_schedule_up(c677f660,0,c0aad9f8,5d,0,...) at g_io_schedule_up+0x133
g_up_procbody(0,e6e45d38,c0ab13b9,30c,c677b2a8,...) at g_up_procbody+0x8d
fork_exit(c0711f50,0,e6e45d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6e45d70, ebp = 0 ---
Tracing command g_event pid 2 tid 100013 td 0xc677f880
sched_switch(c677f880,0,1,180,2dcdde8e,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,4c,...) at mi_switch+0x217
sleepq_switch(c677f880,0,c0ab8594,240,4c,...) at sleepq_switch+0xf0
sleepq_timedwait(c0bccf60,64,c0aac335,0,0,...) at sleepq_timedwait+0x68
_sleep(c0bccf60,0,4c,c0aac335,64,...) at _sleep+0x345
g_event_procbody(0,e6e42d38,c0ab13b9,30c,c677b550,...) at g_event_procbody+0xcb
fork_exit(c0711fe0,0,e6e42d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6e42d70, ebp = 0 ---
Tracing command intr pid 12 tid 100038 td 0xc6927000
sched_switch(c6927000,0,1,180,b75eda39,...) at sched_switch+0x49b
mi_switch(1,0,c0ab163b,469,c6925c64,...) at mi_switch+0x217
ithread_loop(c68ea070,e7102d38,c0ab13b9,30c,c677b7f8,...) at ithread_loop+0x34c
fork_exit(c0743070,c68ea070,e7102d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe7102d70, ebp = 0 ---
Tracing command intr pid 12 tid 100037 td 0xc686fcc0
fork_trampoline() at fork_trampoline
Tracing command intr pid 12 tid 100036 td 0xc68b5000
fork_trampoline() at fork_trampoline
Tracing command intr pid 12 tid 100035 td 0xc68b5220
sched_switch(c68b5220,0,1,180,d8755e05,...) at sched_switch+0x49b
mi_switch(1,0,c0ab163b,469,c67c2464,...) at mi_switch+0x217
ithread_loop(c68cb2a0,e6eeed38,c0ab13b9,30c,c677b7f8,...) at ithread_loop+0x34c
fork_exit(c0743070,c68cb2a0,e6eeed38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6eeed70, ebp = 0 ---
Tracing command intr pid 12 tid 100032 td 0xc68b5880
fork_trampoline() at fork_trampoline
Tracing command intr pid 12 tid 100030 td 0xc68b5cc0
sched_switch(c68b5cc0,0,1,180,81c362f3,...) at sched_switch+0x49b
mi_switch(1,0,c0ab163b,469,c67c21e4,...) at mi_switch+0x217
ithread_loop(c68b30b0,e6ebad38,c0ab13b9,30c,c677b7f8,...) at ithread_loop+0x34c
fork_exit(c0743070,c68b30b0,e6ebad38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6ebad70, ebp = 0 ---
Tracing command intr pid 12 tid 100026 td 0xc6788660
fork_trampoline() at fork_trampoline
Tracing command intr pid 12 tid 100023 td 0xc6788cc0
sched_switch(c6788cc0,0,1,180,595750c6,...) at sched_switch+0x49b
mi_switch(1,0,c0ab163b,469,c687a0e4,...) at mi_switch+0x217
ithread_loop(c6863120,e6e64d38,c0ab13b9,30c,c677b7f8,...) at ithread_loop+0x34c
fork_exit(c0743070,c6863120,e6e64d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6e64d70, ebp = 0 ---
Tracing command intr pid 12 tid 100022 td 0xc686f000
fork_trampoline() at fork_trampoline
Tracing command intr pid 12 tid 100019 td 0xc686f660
sched_switch(c686f660,0,1,180,6885fc2b,...) at sched_switch+0x49b
mi_switch(1,0,c0ab163b,469,c67c27e4,...) at mi_switch+0x217
ithread_loop(c68631c0,e6e58d38,c0ab13b9,30c,c677b7f8,...) at ithread_loop+0x34c
fork_exit(c0743070,c68631c0,e6e58d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6e58d70, ebp = 0 ---
Tracing command intr pid 12 tid 100018 td 0xc686f880
fork_trampoline() at fork_trampoline
Tracing command intr pid 12 tid 100012 td 0xc677faa0
fork_trampoline() at fork_trampoline
Tracing command intr pid 12 tid 100011 td 0xc677fcc0
sched_switch(c677fcc0,0,1,180,358cb1c7,...) at sched_switch+0x49b
mi_switch(1,0,c0ab163b,469,c67c11e4,...) at mi_switch+0x217
ithread_loop(c677a240,e6e3cd38,c0ab13b9,30c,c677b7f8,...) at ithread_loop+0x34c
fork_exit(c0743070,c677a240,e6e3cd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6e3cd70, ebp = 0 ---
Tracing command intr pid 12 tid 100010 td 0xc6788000
sched_switch(c6788000,0,1,180,e489d43,...) at sched_switch+0x49b
mi_switch(1,0,c0ab163b,469,c67c1264,...) at mi_switch+0x217
ithread_loop(c677a250,e6e39d38,c0ab13b9,30c,c677b7f8,...) at ithread_loop+0x34c
fork_exit(c0743070,c677a250,e6e39d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6e39d70, ebp = 0 ---
Tracing command idle pid 11 tid 100009 td 0xc6788220
cpustop_handler(1,e5434c24,c0a2326d,0,a0003a84,...) at cpustop_handler+0x32
ipi_nmi_handler(0,a0003a84,0,c0b6dc00,c677baa0,...) at ipi_nmi_handler+0x2f
trap(e5434c30) at trap+0x2d
calltrap() at calltrap+0x6
--- trap 0x13, eip = 0xc0753a8a, esp = 0xe5434c70, ebp = 0xe5434c80 ---
_mtx_lock_spin(c0be4180,c6788220,10,c0ab6558,26b,...) at _mtx_lock_spin+0x7a
_mtx_lock_spin_flags(c0be4180,10,c0ab6558,26b,c0bdf480,...) at _mtx_lock_spin_flags+0x10e
tdq_lock_pair(c0be3f80,0,c0ab6558,322,c0bddb80,...) at tdq_lock_pair+0x58
sched_idletd(0,e5434d38,c0ab13b9,30c,c677baa0,...) at sched_idletd+0x19d
fork_exit(c077e4a0,0,e5434d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe5434d70, ebp = 0 ---
Tracing command idle pid 11 tid 100008 td 0xc6788440
sched_switch(c6788440,0,1,17b,2a8eb60b,...) at sched_switch+0x49b
mi_switch(1,0,c0ab6558,320,c0bde1c0,...) at mi_switch+0x217
sched_idletd(0,e5431d38,c0ab13b9,30c,c677baa0,...) at sched_idletd+0x217
fork_exit(c077e4a0,0,e5431d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe5431d70, ebp = 0 ---
Tracing command idle pid 11 tid 100007 td 0xc677d000
sched_switch(c677d000,0,1,17b,d887e795,...) at sched_switch+0x49b
mi_switch(1,0,c0ab6558,320,c0bde800,...) at mi_switch+0x217
sched_idletd(0,e542ed38,c0ab13b9,30c,c677baa0,...) at sched_idletd+0x217
fork_exit(c077e4a0,0,e542ed38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe542ed70, ebp = 0 ---
Tracing command idle pid 11 tid 100006 td 0xc677d220
sched_switch(c677d220,0,1,17b,2afb7f06,...) at sched_switch+0x49b
mi_switch(1,0,c0ab6558,320,c0bdee40,...) at mi_switch+0x217
sched_idletd(0,e542bd38,c0ab13b9,30c,c677baa0,...) at sched_idletd+0x217
fork_exit(c077e4a0,0,e542bd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe542bd70, ebp = 0 ---
Tracing command idle pid 11 tid 100005 td 0xc677d440
sched_switch(c677d440,0,1,17b,299116b8,...) at sched_switch+0x49b
mi_switch(1,0,c0ab6558,320,c0bdf480,...) at mi_switch+0x217
sched_idletd(0,e5428d38,c0ab13b9,30c,c677baa0,...) at sched_idletd+0x217
fork_exit(c077e4a0,0,e5428d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe5428d70, ebp = 0 ---
Tracing command idle pid 11 tid 100004 td 0xc677d660
cpustop_handler(20,e5425c24,c0a2326d,0,c7501844,...) at cpustop_handler+0x32
ipi_nmi_handler(0,c7501844,c9e69404,c0bdfac0,c677baa0,...) at ipi_nmi_handler+0x2f
trap(e5425c30) at trap+0x2d
calltrap() at calltrap+0x6
--- trap 0x13, eip = 0xc0753a8a, esp = 0xe5425c70, ebp = 0xe5425c80 ---
_mtx_lock_spin(c0be4180,c677d660,0,c0ab6558,26d,...) at _mtx_lock_spin+0x7a
_mtx_lock_spin_flags(c0be4180,0,c0ab6558,26d,c0bdf480,...) at _mtx_lock_spin_flags+0x10e
tdq_lock_pair(c0be4200,0,c0ab6558,322,c0bdfac0,...) at tdq_lock_pair+0x82
sched_idletd(0,e5425d38,c0ab13b9,30c,c677baa0,...) at sched_idletd+0x19d
fork_exit(c077e4a0,0,e5425d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe5425d70, ebp = 0 ---
Tracing command idle pid 11 tid 100003 td 0xc677d880
cpustop_handler(40,e5422c3c,c0a2326d,c0be0514,1b,...) at cpustop_handler+0x32
ipi_nmi_handler(c0be0514,1b,c0ab6558,476,c677baa0,...) at ipi_nmi_handler+0x2f
trap(e5422c48) at trap+0x2d
calltrap() at calltrap+0x6
--- trap 0x13, eip = 0xc0753f95, esp = 0xe5422c88, ebp = 0xe5422ca8 ---
_mtx_lock_spin_flags(c0be4180,0,c0ab6558,26d,c0bdf480,...) at _mtx_lock_spin_flags+0xd5
tdq_lock_pair(c0be4280,0,c0ab6558,322,c0be0100,...) at tdq_lock_pair+0x82
sched_idletd(0,e5422d38,c0ab13b9,30c,c677baa0,...) at sched_idletd+0x19d
fork_exit(c077e4a0,0,e5422d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe5422d70, ebp = 0 ---
Tracing command idle pid 11 tid 100002 td 0xc677daa0
cpustop_handler(80,e541fc68,c0a2326d,c677baa0,c0be4300,...) at cpustop_handler+0x32
ipi_nmi_handler(c677baa0,c0be4300,0,c677daa0,c677baa0,...) at ipi_nmi_handler+0x2f
trap(e541fc74) at trap+0x2d
calltrap() at calltrap+0x6
--- trap 0x13, eip = 0xc07534ab, esp = 0xe541fcb4, ebp = 0xe541fcc8 ---
_mtx_unlock_spin_flags(c0be4180,0,c0ab6558,320,c0be0740,...) at _mtx_unlock_spin_flags+0x3b
sched_idletd(0,e541fd38,c0ab13b9,30c,c677baa0,...) at sched_idletd+0x203
fork_exit(c077e4a0,0,e541fd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe541fd70, ebp = 0 ---
Tracing command init pid 1 tid 100001 td 0xc677dcc0
sched_switch(c677dcc0,0,1,180,e1825e8a,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,0,...) at mi_switch+0x217
sleepq_switch(c677dcc0,0,c0ab8594,19b,c677bd48,...) at sleepq_switch+0xf0
sleepq_catch_signals(c075356d,c0be3f80,4,c0ab37db,5c,...) at sleepq_catch_signals+0x21e
sleepq_wait_sig(c677bd48,0,c0ab54cf,da,0,...) at sleepq_wait_sig+0x14
_sleep(c677bd48,c677bdd8,15c,c0aba709,0,...) at _sleep+0x359
kern_wait(c677dcc0,ffffffff,e541bc2c,0,0,...) at kern_wait+0xa66
wait4(c677dcc0,e541bcfc,10,c0aba52a,c0b65728,...) at wait4+0x3b
syscall(e541bd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x805456b, esp = 0xbfbfe97c, ebp = 0xbfbfe998 ---
Tracing command audit pid 10 tid 100000 td 0xc677f000
sched_switch(c677f000,0,1,180,c51dc2c0,...) at sched_switch+0x49b
mi_switch(1,0,c0ab8594,1bf,c677f000,...) at mi_switch+0x217
sleepq_switch(c677f000,0,c0ab8594,21d,c677f000,...) at sleepq_switch+0xf0
sleepq_wait(c0c38f10,c0c38ef0,c0ad1ac3,1,0,...) at sleepq_wait+0x60
_cv_wait(c0c38f10,c0c38ef0,c0ad2466,1c2,0,...) at _cv_wait+0x1fc
audit_worker(0,e5418d38,c0ab13b9,30c,c677c000,...) at audit_worker+0x99
fork_exit(c092c7d0,0,e5418d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe5418d70, ebp = 0 ---
Tracing command kernel pid 0 tid 0 td 0xc0bcd2d0
sched_switch(c0bcd2d0,0,1,180,6289368f,...) at sched_switch+0x49b
mi_switch(1,0,c0ad6c19,2e4,c6cdb2c0,...) at mi_switch+0x217
scheduler(0,101ec00,101ec00,101e000,1025000,...) at scheduler+0x2f2
mi_startup() at mi_startup+0x96
begin() at begin+0x2c
db> call doadump
Physical memory: 3314 MB
Dumping 189 MB: 174 158 142 126 110 94 78 62 46 30 14
Dump complete
= 0xf
db> c
bUptime: _42mt56s
rAutomatic reboot in 15 seconds - press a key on the console to abort
ace_self_wrapper(c0ab79ab,e98b0aa4,c07db0d3,0,e98b0aa4,...) at db_trace_self_wrapper+0x26
kdb_backtrace(0,e98b0aa4,c0a2d795,e98b0ab4,c8e09000,...) at kdb_backtrace+0x29
vfs_badlock(c0b86340,e98b0apbi4d, c500b791c e(es0w,acp8)e,0 9u0i0d0 ,100)0 1a,t wvafss _kbialdlledo:c ke+x0cxe2e3de
da smsaexritm_uvmo pC_PlUo clkiemdi(tc8
e09000,c0ac055f) at assert_vop_locked+0x50
vop_lock_post(e98b0b10,0,1002,1002,c8e09000,...) at vop_lock_post+0x36
VOP_LOCK1_APV(c0b86340,e98b0b10,c0ac129b,327,c8e090d4,...) at VOP_LOCK
1_APV+0xc1
_vn_lock(c8e09000,1002,c91e7880,c0ac129b,120,...) at _vn_lock+0xf8
vn
_close(c8e09000,3,c6ec1a00,c91e7880,c07d0801,...) at vn_close+0x8a
vn_closefile(c6e
beca8,c91e7880,c0ab09dc,87b,c8e09000,...) at vn_closefile+0xf4
fdrop(c6ebeca8,c91e7880,c0a11fc5,40,0,c0b9d360,c8e09000,c6b2ed48,2,e98b0c20,40,0,0,0,0,c0ab09dc,2) at fdro
p+0x11e
closef(c6ebeca8,c91e7880,41c,401,c6ebeca8,...) at closef+0x254
kern_close(c91e7880,3,e98b0d2c,c0a22f63,c91e7880,...) at kern_close+0x11d
close(c91e7880,e98b
0cfc,4,c0aba75c,c0b65710,...) at close+0x1a
syscall(e98b0d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (6, FreepBiSdD 5E0L7F13 2(,swcalpo)s,e )u,i de i1p0 0=1 ,0 xw2a8s1 5k4i7lcl3e,d :e sepx c=e e0dxebdf bmfaex9i2mcu,m eCbPpU =l i0mxibtfb
fe968 ---
VOP_LOCK: 0xc8e09000 is not locked but should be
KDB: enter: lock violation
[thread pid 5103 tid 100384 ]
Stopped at kdb_enter+0x32: leave
db> c
KDB: stack backtrace:
db_trace_self_wrapper(c0ab79ab,e98b0af8,c07db0d3,c91e7880,e98b0a
f8,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c91e7880,e98b0af8,c0a2d795,e98b0b08,c8e09000,...) at kdb_backtrace+0x29
vfs_badlock(c0b86340,e98b0b08,c0b9cee0,c8e09000,c91e7880) at vfs_badlock+0x23
assert_vop_elocked(c8e09000,c0ae6b31,3,3,c8e090
00,...) at assert_vop_elocked+0x53
VOP_CLOSE_APV(c0b86340,e98b0b50,c91e7880,c0ac12
9b,120,...) at VOP_CLOSE_APV+0x8a
vn_close(c8e09000,3,c6ec1a00,c91e7880,c07d0801,...) at vn_close+0xe6
vn_closefile(c6ebeca8,c91e7880,c0ab09dc,87b,c8e09000,...) at vn_closefile+0xf4
fdrop(c6ebeca8,c91e7880,c0a11fc5,40,0,c0b9d360,c8e09000,c6b2ed48,2,e98b0c20,40,0,0,0,0,c0ab09dc,2) at fdrop+0x11e
closef(c6ebeca8,c91e7880,41pci,d4 0510,7c16 e(bsewcaap8),,. .u.i)d a1t0 0c1l,o sweafs+ 0kxi2l5l4ed
:k eerxnc_eceldoesde (ma9x1iem7u8m8 0C,P3U, el9i8mbi0td2
c,c0a22f63,c91e7880,...) at kern_close+0x11d
close
(c91e7880,e98b0cfc,4,c0aba75c,c0b65710,...) at close+0x1a
syscall(e98b0d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (6, FreeBSD ELF32, close), eip = 0x281547c3, esp = 0xbfbfe92c, ebp = 0xbfbfe968 ---
VOP_CLOSE: 0xc8e09000 is not exclusive locked but should be
KDB: enter: lock violation
[thread pid 5103 tid 100384 ]
Stopped at kdb_enter+0x32: leave
db> c
DB: stack backtrace:
db_trace_self_wrapper(c0ab79ab,e98b0af8,c07db0d3,c91e7880,e98b0af8,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c91e7880,e98b0af8,c0a2d795,e98b0b0
8,c8e09000,...) at kdb_backtrace+0x29
vfs_badlock(c0b86340,e98b0b08,c0b9cee0,c8e09000,c91e7880) at vfs_badlock+0x23
assert_vop_elocked(c8e09000,c0ae6b31,3,3,c8e09000
,...) at assert_vop_elocked+0x53
VOP_CLOSE_APV(c0b86340,e98b0b50,c91e7880,c0ac129b,120,...) at VOP_CLOSE_APV+0xff
vn_clospei(dc 85e007910 0(0s,w3a,pc)6,e cu1iad0 01,c0911,e 7w8a8s0 ,kci0l7lde0d8:0 1e,x.c.e.e)d eadt mvanx_icmluoms eC+P0Ux el6im
ivtn_
closefile(c6ebeca8,c91e7880,c0ab09dc,87b,c8e09000,...) at vn_closefile+0xf4
fdrop(c6beca8,c91e7880,c0a11fc5,40,0,c0b9d360,c8e09000,c6b2ed48,2,e98b0c20,40,0,0,0,0,c0ab09dc,2) at fdrop+0x11e
closef(c6ebeca8,c91e7880,41c,401,c6ebeca8,...) at closef+0x254
kern_close(c91e7880,3,e98b0d2c,c0a22f63,c91e7880,...) at kern_close+0x11d
close(c91e78
80,e98b0cfc,4,c0aba75c,c0b65710,...) at close+0x1a
syscall(e98b0d38) at syscall+0x
2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (6, FreeBSD ELF32, c
lose), eip = 0x281547c3, esp = 0xbfbfe92c, ebp = 0xbfbfe968 ---
VOP_CLOSE: 0xc8e09000 is not exclusive locked but should be
KDB: enter: lock violation
[thread pid 5103 tid 100384 ]
Stopped at kdb_enter+0x32: leave
(kgdb) bt
#0 doadump () at pcpu.h:195
#1 0xc048da99 in db_fncall (dummy1=-1065840606, dummy2=0, dummy3=-1, dummy4=0xe9352598 "") at ../../../ddb/db_command.c:486
#2 0xc048e005 in db_command_loop () at ../../../ddb/db_command.c:401
#3 0xc048f775 in db_trap (type=3, code=0) at ../../../ddb/db_main.c:222
#4 0xc0788ea6 in kdb_trap (type=3, code=0, tf=0xe9352740) at ../../../kern/subr_kdb.c:502
#5 0xc0a237ab in trap (frame=0xe9352740) at ../../../i386/i386/trap.c:621
#6 0xc0a0925b in calltrap () at ../../../i386/i386/exception.s:139
#7 0xc0789022 in kdb_enter (msg=0xc0ab4b70 "panic") at cpufunc.h:60
#8 0xc0760174 in panic (fmt=0xc0ad3174 "ffs_truncate3") at ../../../kern/kern_shutdown.c:547
#9 0xc0938b08 in ffs_truncate (vp=0xc8c73000, length=0, flags=Variable "flags" is not available.
) at ../../../ufs/ffs/ffs_inode.c:494
#10 0xc0957d30 in ufs_inactive (ap=0xe9352a38) at ../../../ufs/ufs/ufs_inode.c:132
#11 0xc0a2e6d5 in VOP_INACTIVE_APV (vop=0xc0b86860, a=0xe9352a38) at vnode_if.c:1513
#12 0xc07de101 in vinactive (vp=0xc8c73000, td=0xc6d04880) at vnode_if.h:796
#13 0xc07de907 in vput (vp=0xc8c73000) at ../../../kern/vfs_subr.c:2197
#14 0xc07eaf70 in vn_close (vp=0xc8c73000, flags=3, file_cred=0xc6ec1a00, td=0xc6d04880) at ../../../kern/vfs_vnops.c:295
#15 0xc07eb084 in vn_closefile (fp=0xc6fc3d80, td=0xc6d04880) at ../../../kern/vfs_vnops.c:868
#16 0xc0733fbe in fdrop (fp=0xc6fc3d80, td=0xc6d04880) at file.h:297
#17 0xc0735504 in closef (fp=0xc6fc3d80, td=0xc6d04880) at ../../../kern/kern_descrip.c:1958
#18 0xc0736256 in fdfree (td=0xc6d04880) at ../../../kern/kern_descrip.c:1668
#19 0xc073f703 in exit1 (td=0xc6d04880, rv=256) at ../../../kern/kern_exit.c:271
#20 0xc074056d in sys_exit (td=Could not find the frame base for "sys_exit".
) at ../../../kern/kern_exit.c:98
#21 0xc0a22f63 in syscall (frame=0xe9352d38) at ../../../i386/i386/trap.c:1008
#22 0xc0a092c0 in Xint0x80_syscall () at ../../../i386/i386/exception.s:196
#23 0x00000033 in ?? ()
Previous frame inner to this frame (corrupt stack?)
(kgdb) f 9
#9 0xc0938b08 in ffs_truncate (vp=0xc8c73000, length=0, flags=Variable "flags" is not available.
) at ../../../ufs/ffs/ffs_inode.c:494
494 panic("ffs_truncate3");
(kgdb) p *vp
$8 = {v_type = VREG, v_tag = 0xc0aaf0bf "ufs", v_op = 0xc0b86340, v_data = 0xc89d2dec, v_mount = 0xc6dcd2e8, v_nmntvnodes = {tqe_next = 0xc8ca1414, tqe_prev = 0xc8b6e014},
v_un = {vu_mount = 0x0, vu_socket = 0x0, vu_cdev = 0x0, vu_fifoinfo = 0x0}, v_hashlist = {le_next = 0xc7f082b8, le_prev = 0xc69d1068}, v_hash = 0x597, v_cache_src = {
lh_first = 0x0}, v_cache_dst = {tqh_first = 0x0, tqh_last = 0xc8c73030}, v_dd = 0x0, v_cstart = 0x0, v_lasta = 0x0, v_lastw = 0x0, v_clen = 0x0, v_lock = {lk_object = {
lo_name = 0xc0aaf0bf "ufs", lo_type = 0xc0aaf0bf "ufs", lo_flags = 0x5390000, lo_witness_data = {lod_list = {stqe_next = 0x0}, lod_witness = 0x0}},
lk_interlock = 0xc0bdb678, lk_flags = 0x40040, lk_sharecount = 0x0, lk_waitcount = 0x0, lk_exclusivecount = 0x1, lk_prio = 0x50, lk_timo = 0x33,
lk_lockholder = 0xc6d04880, lk_newlock = 0x0, lk_stack = {depth = 0xd, pcs = {0xc074f470, 0xc0951f91, 0xc0a30945, 0xc07ea6b8, 0xc07eaf0a, 0xc07eb084, 0xc0733fbe,
0xc0735504, 0xc0736256, 0xc073f703, 0xc074056d, 0xc0a22f63, 0xc0a092c0, 0x0, 0x0, 0x0, 0x0, 0x0}}}, v_interlock = {lock_object = {
lo_name = 0xc0aba01f "vnode interlock", lo_type = 0xc0aba01f "vnode interlock", lo_flags = 0x1030000, lo_witness_data = {lod_list = {stqe_next = 0xc0bf0938},
lod_witness = 0xc0bf0938}}, mtx_lock = 0xc6d04880, mtx_recurse = 0x0}, v_vnlock = 0xc8c73058, v_holdcnt = 0x3, v_usecount = 0x0, v_iflag = 0x800, v_vflag = 0x4,
v_writecount = 0x0, v_freelist = {tqe_next = 0x0, tqe_prev = 0x0}, v_bufobj = {bo_mtx = 0xc8c730d4, bo_clean = {bv_hd = {tqh_first = 0xda9f99f0, tqh_last = 0xda9f9a28},
bv_root = 0xda9f99f0, bv_cnt = 0x1}, bo_dirty = {bv_hd = {tqh_first = 0x0, tqh_last = 0xc8c73120}, bv_root = 0x0, bv_cnt = 0x0}, bo_numoutput = 0x0, bo_flag = 0x0,
bo_ops = 0xc0b73ba0, bo_bsize = 0x4000, bo_object = 0xc8ce0b24, bo_synclist = {le_next = 0xc7be267c, le_prev = 0xc8b6e144}, bo_private = 0xc8c73000,
__bo_vnode = 0xc8c73000}, v_pollinfo = 0x0, v_label = 0x0}
(kgdb) p vp->v_bufobj.bo_clean.bv_hd.tqh_first
$9 = (struct buf *) 0xda9f99f0
(kgdb) p *vp->v_bufobj.bo_clean.bv_hd.tqh_first
$10 = {b_bufobj = 0xc8c7310c, b_bcount = 0x4000, b_caller1 = 0x0, b_data = 0xdf6ad000 "", b_error = 0x0, b_iocmd = 0x2, b_ioflags = 0x2, b_iooffset = 0xbfc000, b_resid = 0x0,
b_iodone = 0, b_blkno = 0x5fe0, b_offset = 0xfffffffffffd0000, b_bobufs = {tqe_next = 0x0, tqe_prev = 0xc8c73110}, b_left = 0x0, b_right = 0x0, b_vflags = 0x0, b_freelist = {
tqe_next = 0xda979950, tqe_prev = 0xda8b351c}, b_qindex = 0x1, b_flags = 0x20000020, b_xflags = 0x2, b_lock = {lk_object = {lo_name = 0xc0abea93 "getblk",
lo_type = 0xc0abecbd "bufwait", lo_flags = 0x5390000, lo_witness_data = {lod_list = {stqe_next = 0x0}, lod_witness = 0x0}}, lk_interlock = 0xc0bdb840, lk_flags = 0x0,
lk_sharecount = 0x0, lk_waitcount = 0x0, lk_exclusivecount = 0x0, lk_prio = 0x50, lk_timo = 0x0, lk_lockholder = 0xffffffff, lk_newlock = 0x0, lk_stack = {depth = 0xd,
pcs = {0xc074f470, 0xc07cb96f, 0xc07cc4cc, 0xc07cc65c, 0xc09348fb, 0xc0953ba3, 0xc0a31495, 0xc07eb641, 0xc079b015, 0xc079b298, 0xc079b30f, 0xc0a22f63, 0xc0a092c0, 0x0,
0x0, 0x0, 0x0, 0x0}}}, b_bufsize = 0x4000, b_runningbufspace = 0x0, b_kvabase = 0xdf6ad000 "", b_kvasize = 0x4000, b_lblkno = 0xfffffffffffffff4, b_vp = 0xc8c73000,
b_dirtyoff = 0x0, b_dirtyend = 0x0, b_rcred = 0x0, b_wcred = 0x0, b_saveaddr = 0xdf6ad000, b_pager = {pg_reqpage = 0x0}, b_cluster = {cluster_head = {tqh_first = 0xda8865f0,
tqh_last = 0xdaae9350}, cluster_entry = {tqe_next = 0xda8865f0, tqe_prev = 0xdaae9350}}, b_pages = {0xc2a7fdc0, 0xc2a7fe08, 0xc2a7fe50, 0xc48d79d8,
0x0 <repeats 28 times>}, b_npages = 0x4, b_dep = {lh_first = 0x0}, b_fsprivate1 = 0x0, b_fsprivate2 = 0x0, b_fsprivate3 = 0x0, b_pin_count = 0x0}