GENERIC HEAD from 2011-06-11 04:59:01 UTC, r222975M, vmcore.161 KDB: debugger backends: ddb KDB: current backend: ddb 524288K of memory above 4GB ignored Copyright (c) 1992-2011 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-CURRENT #0 r222975M: Sat Jun 11 23:37:27 CEST 2011 pho@x4.osted.lan:/usr/src/sys/i386/compile/PHO i386 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. CPU: AMD Phenom(tm) 9150e Quad-Core Processor (1800.02-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x100f23 Family = 10 Model = 2 Stepping = 3 Features=0x178bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x7ff TSC: P-state invariant real memory = 4294967296 (4096 MB) avail memory = 3535958016 (3372 MB) : Trying to mount root from ufs:/dev/ad4s1a [rw]... WARNING: / was not properly dismounted Setting hostuuid: 00000000-0000-0000-0000-00218515337d. Setting hostid: 0x6b64ac17. Starting ddb. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: /dev/ad4s1a: 2357 files, 678175 used, 334840 free (1488 frags, 41669 blocks, 0.1% fragmentation) /dev/ad4s1f: INCORRECT BLOCK COUNT I=191015 (268 should be 256) (CORRECTED) /dev/ad4s1f: FREE BLK COUNT(S) WRONG IN SUPERBLK (SALVAGED) /dev/ad4s1f: SUMMARY INFORMATION BAD (SALVAGED) /dev/ad4s1f: BLK(S) MISSING IN BIT MAPS (SALVAGED) /dev/ad4s1f: 15564 files, 815608 used, 197407 free (4463 frags, 24118 blocks, 0.4% fragmentation) /dev/label/tmp: FREE BLK COUNT(S) WRONG IN SUPERBLK (SALVAGED) /dev/label/tmp: SUMMARY INFORMATION BAD (SALVAGED) /dev/label/tmp: BLK(S) MISSING IN BIT MAPS (SALVAGED) /dev/label/tmp: 101363 files, 24006998 used, 94306983 free (2943 frags, 11788005 blocks, 0.0% fragmentation) /dev/ad4s1d: 765324 files, 7665107 used, 2489092 free (231292 frags, 282225 blocks, 2.3% fragmentation) /dev/ad4s1e: 620183 files, 16632693 used, 8755824 free (48384 frags, 1088430 blocks, 0.2% fragmentation) Mounting local file systems:. Setting hostname: x4.osted.lan. re0: link state changed to DOWN Starting Network: lo0 re0 fwe0 fwip0. lo0: flags=8049 metric 0 mtu 16384 options=3 inet 127.0.0.1 netmask 0xff000000 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0xa nd6 options=21 re0: flags=8843 metric 0 mtu 1500 options=389b ether 00:21:85:15:33:7d inet 192.168.1.101 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::221:85ff:fe15:337d%re0 prefixlen 64 tentative scopeid 0x1 nd6 options=29 media: Ethernet autoselect (none) status: no carrier fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb ch 1 dma -1 fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 Starting devd. Starting Network: usbus0. Starting Network: usbus1. Starting Network: usbus2. Starting Network: usbus3. Starting Network: usbus4. Starting Network: usbus5. Starting Network: fwe0. fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb ch 1 dma -1 re0: link state changed to UP Starting Network: fwip0. fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 add net default: gateway 192.168.1.1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/kde4/lib /usr/local/lib /usr/local/lib/compat /usr/local/lib/compat/pkg /usr/local/lib/nss /usr/local/lib/qt4 /usr/local/lib/virtualbox /usr/local/lib/wine a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout Creating and/or trimming log files. Starting syslogd. savecore: reboot Jun 12 08:23:0lock order reversal: 1 x4 savecore: r 1st 0xe0715460 bufwait (bufwait) @ kern/vfs_bio.c:2658 eboot 2nd 0xc7909a00 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:284 KDB: stack backtrace: db_trace_self_wrapper(c0e95b56,2e687361,38323a63,c7000a34,a7a,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c09e61bb,c0e9955f,c6d5e2a0,c6d61ff8,ef1d57e4,...) at kdb_backtrace+0x2a _witness_debugger(c0e9955f,c7909a00,c0ec8822,c6d61ff8,c0ec84b4,...) at _witness_debugger+0x25 witness_checkorder(c7909a00,9,c0ec84ab,11c,0,...) at witness_checkorder+0x839 _sx_xlock(c7909a00,0,c0ec84ab,11c,c79060e8,...) at _sx_xlock+0x85 ufsdirhash_acquire(e0715400,e12925a4,5c,e12925b8,ef1d58b4,...) at ufsdirhash_acquire+0x48 ufsdirhash_add(c79060e8,ef1d5914,25b8,ef1d58a0,ef1d58a4,...) at ufsdirhash_add+0x13 ufs_direnter(c7907ae0,c7907570,ef1d5914,ef1d5ba4,0,...) at ufs_direnter+0x749 ufs_makeinode(ef1d5ba4,c0fc7a80,ef1d5b00,ef1d5a5c,c0cfea35,...) at ufs_makeinode+0x5e5 ufs_create(ef1d5b00,c0ee696d,0,0,ef1d5b78,...) at ufs_create+0x30 VOP_CREATE_APV(c0fc7500,ef1d5b00,ef1d5ba4,ef1d5a98,0,...) at VOP_CREATE_APV+0xc5 vn_open_cred(ef1d5b78,ef1d5c2c,180,0,c6d9fe00,...) at vn_open_cred+0x205 vn_open(ef1d5b78,ef1d5c2c,180,c7413498,0,...) at vn_open+0x3b kern_openat(c743d5c0,ffffff9c,28450000,0,602,...) at kern_openat+0x12f kern_open(c743d5c0,28450000,0,601,180,...) at kern_open+0x35 open(c743d5c0,ef1d5cec,ef1d5d80,ef1d5c88,0,...) at open+0x30 syscallenter(c743d5c0,ef1d5ce4,c0cdb566,0,0,...) at syscallenter+0x263 syscall(ef1d5d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (5, FreeBSD ELF32, open), eip = 0x281896a3, esp = 0xbfbfe2ac, ebp = 0xbfbfedb8 --- savecore: writing core to vmcore.160 Unable to find matching kernel for /var/crash/vmcore.160 Additional ABI support: linux. Starting rpcbind. NFS access cache time=60 Clearing /tmp (X related). Starting mountd. NFSv4 is disabled Starting nfsd. Recovering vi editor sessions:. Updating motd:. Starting ntpd. Starting default moused. Configuring syscons: keymap blanktime. Starting sshd. Starting cron. Local package initialization: backuplock order reversal: 1st 0xc770c058 ufs (ufs) @ kern/vfs_subr.c:2134 2nd 0xe08c7c20 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:261 3rd 0xc777e880 ufs (ufs) @ kern/vfs_subr.c:2134 KDB: stack backtrace: db_trace_self_wrapper(c0e95b56,3a632e73,a313632,ef25a800,246,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c09e61bb,c0e99578,c6d5e2a0,c6d61f90,ef25a8a4,...) at kdb_backtrace+0x2a _witness_debugger(c0e99578,c777e880,c0e88914,c6d61f90,c0ea117b,...) at _witness_debugger+0x25 witness_checkorder(c777e880,9,c0ea1172,856,0,...) at witness_checkorder+0x839 __lockmgr_args(c777e880,80100,c777e8ec,0,0,...) at __lockmgr_args+0x814 ffs_lock(ef25a9c8,c09f7aab,c0ea04d7,80100,c777e828,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c0fc7500,ef25a9c8,109,c0fe2500,c777e828,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c777e828,80100,c0ea1172,856,4,...) at _vn_lock+0x78 vget(c777e828,80100,c749e2e0,50,0,...) at vget+0xbb vfs_hash_get(c77245a8,a27000,80000,c749e2e0,ef25ab0c,...) at vfs_hash_get+0xed ffs_vgetf(c77245a8,a27000,80000,ef25ab0c,1,...) at ffs_vgetf+0x49 softdep_sync_buf(c770c000,e08c7bc0,1,107,0,...) at softdep_sync_buf+0x4a3 ffs_syncvnode(c770c000,1,c749e2e0,566,0,...) at ffs_syncvnode+0x238 ffs_sync(c77245a8,1,ef25ac14,4ee,0,...) at ffs_sync+0x26f dounmount(c77245a8,8080000,c749e2e0,473,5b85c5ea,...) at dounmount+0x447 unmount(c749e2e0,ef25acec,28176705,1,0,...) at unmount+0x310 syscallenter(c749e2e0,ef25ace4,c0cdb8ad,c101d170,0,...) at syscallenter+0x263 syscall(ef25ad28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, unmount), eip = 0x280dcd5b, esp = 0xbfbfe64c, ebp = 0xbfbfe718 --- lock order reversal: 1st 0xc770c880 ufs (ufs) @ kern/vfs_mount.c:1193 2nd 0xc770d310 devfs (devfs) @ ufs/ffs/ffs_softdep.c:1827 KDB: stack backtrace: db_trace_self_wrapper(c0e95b56,38313a63,a3732,c0ec9ea1,c15b3888,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c09e61bb,c0e9955f,c6d61f90,c6d61e58,ef25aa04,...) at kdb_backtrace+0x2a _witness_debugger(c0e9955f,c770d310,c0e80787,c6d61e58,c0ec4aa9,...) at _witness_debugger+0x25 witness_checkorder(c770d310,9,c0ec4aa0,723,c770d37c,...) at witness_checkorder+0x839 __lockmgr_args(c770d310,80400,c770d37c,0,0,...) at __lockmgr_args+0x814 vop_stdlock(ef25ab24,4,c0e90227,80400,c770d2b8,...) at vop_stdlock+0x65 VOP_LOCK1_APV(c0f9d000,ef25ab24,c119b230,c0fe2500,c770d2b8,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c770d2b8,80400,c0ec4aa0,723,c77245a8,...) at _vn_lock+0x78 softdep_flushworklist(c77245a8,ef25abd0,c749e2e0,574,0,...) at softdep_flushworklist+0x47 ffs_sync(c77245a8,1,ef25ac14,4ee,0,...) at ffs_sync+0x2fd dounmount(c77245a8,8080000,c749e2e0,473,5b85c5ea,...) at dounmount+0x447 unmount(c749e2e0,ef25acec,28176705,1,0,...) at unmount+0x310 syscallenter(c749e2e0,ef25ace4,c0cdb8ad,c101d170,0,...) at syscallenter+0x263 syscall(ef25ad28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, unmount), eip = 0x280dcd5b, esp = 0xbfbfe64c, ebp = 0xbfbfe718 --- ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 101363 files, 24006998 used, 94306983 free (2943 frags, 11788005 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes Jun 12 08:25:06 x4 su: pho to root on /dev/pts/1 ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 101363 files, 24006998 used, 94306983 free (2943 frags, 11788005 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** fsck -y /tmp watchdogd. Starting inetd. Sun Jun 12 08:26:23 CEST 2011 FreeBSD/i386 (x4.osted.lan) (console) login: lock order reversal: 1st 0xe06f1ba0 bufwait (bufwait) @ kern/vfs_bio.c:2658 2nd 0xc78e6d1c snaplk (snaplk) @ ufs/ffs/ffs_snapshot.c:808 KDB: stack backtrace: db_trace_self_wrapper(c0e95b56,632e746f,3830383a,ef2e000a,c09f7aab,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c09e61bb,c0e9955f,c6d5e2a0,c6d627b0,ef2ed50c,...) at kdb_backtrace+0x2a _witness_debugger(c0e9955f,c78e6d1c,c0ec4545,c6d627b0,c0ec44cf,...) at _witness_debugger+0x25 witness_checkorder(c78e6d1c,9,c0ec44c6,328,ca313e5c,...) at witness_checkorder+0x839 __lockmgr_args(c78e6d1c,80400,ca313e5c,0,0,...) at __lockmgr_args+0x814 ffs_lock(ef2ed630,e0714900,100000,80400,ca313d98,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c0fc7500,ef2ed630,e07149a8,c0fe2500,ca313d98,...) at VOP_LOCK1_APV+0xb5 _vn_lock(ca313d98,80400,c0ec44c6,328,0,...) at _vn_lock+0x78 ffs_snapshot(c73f12d4,c7c75500,c0ec7950,174,ef2ed980,...) at ffs_snapshot+0x2946 ffs_mount(c73f12d4,c87d7000,ff,393,0,...) at ffs_mount+0x1590 vfs_donmount(c77878a0,211300,c7709580,c7709580,c7c9a834,...) at vfs_donmount+0x1147 nmount(c77878a0,ef2edcec,ef2edd28,c0e97daa,0,...) at nmount+0x84 syscallenter(c77878a0,ef2edce4,ef2edce4,0,0,...) at syscallenter+0x263 syscall(ef2edd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (378, FreeBSD ELF32, nmount), eip = 0x280da9db, esp = 0xbfbfe0ec, ebp = 0xbfbfe868 --- fsync: giving up on dirty 0xc82feae0: tag devfs, type VCHR usecount 1, writecount 0, refcount 571 mountedhere 0xc7dad400 flags (VV_COPYONWRITE) v_object 0xc8300088 ref 0 pages 2270 lock type devfs: EXCL by thread 0xc749d2e0 (pid 2457) #0 0xc099587e at __lockmgr_args+0xbfe #1 0xc0a35865 at vop_stdlock+0x65 #2 0xc0cfda65 at VOP_LOCK1_APV+0xb5 #3 0xc0a557f8 at _vn_lock+0x78 #4 0xc0bdfa17 at softdep_flushworklist+0x47 #5 0xc0be32ed at ffs_sync+0x2fd #6 0xc0a555d2 at vfs_write_suspend+0x122 #7 0xc0bc7cc1 at ffs_snapshot+0xc91 #8 0xc0be5220 at ffs_mount+0x1590 #9 0xc0a3eff7 at vfs_donmount+0x1147 #10 0xc0a3f664 at nmount+0x84 #11 0xc09f0473 at syscallenter+0x263 #12 0xc0cdb04f at syscall+0x4f #13 0xc0cc4711 at Xint0x80_syscall+0x21 dev md5a fsync: giving up on dirty 0xc82feae0: tag devfs, type VCHR usecount 1, writecount 0, refcount 568 mountedhere 0xc7dad400 flags (VV_COPYONWRITE) v_object 0xc8300088 ref 0 pages 2274 lock type devfs: EXCL by thread 0xc749d2e0 (pid 2457) #0 0xc099587e at __lockmgr_args+0xbfe #1 0xc0a35865 at vop_stdlock+0x65 #2 0xc0cfda65 at VOP_LOCK1_APV+0xb5 #3 0xc0a557f8 at _vn_lock+0x78 #4 0xc0be33c5 at ffs_sync+0x3d5 #5 0xc0a555d2 at vfs_write_suspend+0x122 #6 0xc0bc7cc1 at ffs_snapshot+0xc91 #7 0xc0be5220 at ffs_mount+0x1590 #8 0xc0a3eff7 at vfs_donmount+0x1147 #9 0xc0a3f664 at nmount+0x84 #10 0xc09f0473 at syscallenter+0x263 #11 0xc0cdb04f at syscall+0x4f #12 0xc0cc4711 at Xint0x80_syscall+0x21 dev md5a fsync: giving up on dirty 0xc82feae0: tag devfs, type VCHR usecount 1, writecount 0, refcount 90 mountedhere 0xc7dad400 flags (VV_COPYONWRITE) v_object 0xc8300088 ref 0 pages 2250 lock type devfs: EXCL by thread 0xc82748a0 (pid 2460) #0 0xc099587e at __lockmgr_args+0xbfe #1 0xc0a35865 at vop_stdlock+0x65 #2 0xc0cfda65 at VOP_LOCK1_APV+0xb5 #3 0xc0a557f8 at _vn_lock+0x78 #4 0xc0bdfa17 at softdep_flushworklist+0x47 #5 0xc0be32ed at ffs_sync+0x2fd #6 0xc0a555d2 at vfs_write_suspend+0x122 #7 0xc0bc7cc1 at ffs_snapshot+0xc91 #8 0xc0be5220 at ffs_mount+0x1590 #9 0xc0a3eff7 at vfs_donmount+0x1147 #10 0xc0a3f664 at nmount+0x84 #11 0xc09f0473 at syscallenter+0x263 #12 0xc0cdb04f at syscall+0x4f #13 0xc0cc4711 at Xint0x80_syscall+0x21 dev md5a Jun 12 08:27:20 x4 kernel: pid 2480 (suj20), uid 0 inumber 7 on /mnt: filesystem full Jun 12 08:49:49 x4 su: pho to root on /dev/pts/0 Stopping inetd. Waiting for PIDS: 2290. Shutting down local packages:. Stopping cron. Waiting for PIDS: 2132. Stopping sshd. Waiting for PIDS: 2110. Stopping watchdogd. Waiting for PIDS: 226630 second watchdog timeout expired. Shutdown terminated. Sun Jun 12 09:06:20 CEST 2011 Jun 12 09:06:20 x4 init: /bin/sh on /etc/rc.shutdown terminated abnormally, going to single user mode Jun 12 09:06:20 x4 syslogd: exiting on signal 15 Jun 12 09:06:40 init: some processes would not die; ps axl advised Enter full pathname of shell or RETURN for /bin/sh: # ps -l UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND 0 6138 1 0 20 0 9928 1816 wait Ss u0 0:00.01 -sh (sh) 0 6139 6138 0 20 0 9676 1268 - R+ u0 0:00.00 ps -l 0 5813 1 0 20 0 9876 1672 ufs D 0- 0:00.01 ls -l /mnt 0 2480 1 0 20 0 1644 764 - R 1- 39:02.24 /tmp/suj20 # kill -9 2480 # kill -9 2480 # kill -9 2480 # df -i /mnt Filesystem 512-blocks Used Avail Capacity iused ifree %iused Mounted on /dev/md5a 2025932 1835036 28824 98% 19463 121847 14% /mnt # ps -l UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND 0 6138 1 0 20 0 9928 1816 wait Ss u0 0:00.01 -sh (sh) 0 6141 6138 0 20 0 9676 1268 - R+ u0 0:00.00 ps -l 0 5813 1 0 20 0 9876 1672 ufs D 0- 0:00.01 ls -l /mnt 0 2480 1 0 20 0 1644 764 - R 1- 39:02.24 /tmp/suj20 # ~KDB: enter: Line break on console [ thread pid 11 tid 100003 ] Stopped at kdb_enter+0x3a: movl $0,kdb_why db> run pho db:0:pho> bt Tracing pid 11 tid 100003 td 0xc6dbe8a0 kdb_enter(c0d188be,c0e5ba70,0,c716a080,0,...) at kdb_enter+0x3a uart_intr(c716a000,0,c11bfba0,c6b3abe8,0,...) at uart_intr+0x126 intr_event_handle(c6d88800,c6b3ac10,c6b3ac34,0,c6fe9200,...) at intr_event_handle+0x76 intr_execute_handlers(c6e020d0,c6b3ac10,0,c6b3ac50,c0cc4a75,...) at intr_execute_handlers+0x49 lapic_handle_intr(3b,c6b3ac10) at lapic_handle_intr+0x36 Xapic_isr1() at Xapic_isr1+0x35 --- interrupt, eip = 0xc0cc0075, esp = 0xc6b3ac50, ebp = 0xc6b3ac50 --- acpi_cpu_c1(c6b3ac6c,c0fe3bd0,0,300,b89,...) at acpi_cpu_c1+0x5 acpi_cpu_idle(0,c6b3aca0,c0cce299,0,ffffffff,...) at acpi_cpu_idle+0x122 cpu_idle_acpi(0,ffffffff,c102b700,2,c6b3ace8,...) at cpu_idle_acpi+0x2f cpu_idle(0,c6b3acc4,c0e93c4b,a05,c6dbe8a0,...) at cpu_idle+0x89 sched_idletd(0,c6b3ad28,c0e8c598,390,c6dbc834,...) at sched_idletd+0x263 fork_exit(c09d3440,0,c6b3ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b3ad60, ebp = 0 --- db:0:bt> show allpcpu Current CPU: 0 cpuid = 0 dynamic pcpu = 0x442400 curthread = 0xc6dbe8a0: pid 11 "idle: cpu0" curpcb = 0xc6b3ad80 fpcurthread = none idlethread = 0xc6dbe8a0: tid 100003 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: cpuid = 1 dynamic pcpu = 0x5b47400 curthread = 0xc6dbe5c0: pid 11 "idle: cpu1" curpcb = 0xc6b3dd80 fpcurthread = none idlethread = 0xc6dbe5c0: tid 100004 "idle: cpu1" APIC ID = 1 currentldt = 0x50 spin locks held: cpuid = 2 dynamic pcpu = 0x5b4a400 curthread = 0xc74388a0: pid 2480 "suj20" curpcb = 0xef20cd80 fpcurthread = none idlethread = 0xc6dbe2e0: tid 100005 "idle: cpu2" APIC ID = 2 currentldt = 0x50 spin locks held: cpuid = 3 dynamic pcpu = 0x5b4d400 curthread = 0xc6dbe000: pid 11 "idle: cpu3" curpcb = 0xc6b43d80 fpcurthread = none idlethread = 0xc6dbe000: tid 100006 "idle: cpu3" APIC ID = 3 currentldt = 0x50 spin locks held: db:0:allpcpu> show alllocks Process 5813 (ls) thread 0xc743d8a0 (100086) shared lockmgr ufs (ufs) r = 0 (0xc82fe9dc) locked @ kern/vfs_lookup.c:501 Process 2574 (ls) thread 0xc82f78a0 (100196) shared lockmgr ufs (ufs) r = 0 (0xc82fe9dc) locked @ kern/vfs_lookup.c:501 Process 2480 (suj20) thread 0xc74388a0 (100103) exclusive lockmgr snaplk (snaplk) r = 0 (0xc78e6d1c) locked @ ufs/ffs/ffs_snapshot.c:2267 exclusive lockmgr bufwait (bufwait) r = 0 (0xe09a0300) locked @ kern/vfs_bio.c:2658 exclusive lockmgr ufs (ufs) r = 0 (0xc9bce310) locked @ kern/vfs_vnops.c:612 Process 16 (syncer) thread 0xc71752e0 (100073) exclusive lockmgr bufwait (bufwait) r = 0 (0xe07f9140) locked @ kern/vfs_bio.c:2658 exclusive lockmgr ufs (ufs) r = 0 (0xc8e31310) locked @ kern/vfs_subr.c:2134 exclusive lockmgr syncer (syncer) r = 0 (0xc82fe724) locked @ kern/vfs_subr.c:1734 Process 9 (bufdaemon) thread 0xc71755c0 (100072) exclusive lockmgr ufs (ufs) r = 0 (0xc9bef1b4) locked @ kern/vfs_bio.c:2372 db:0:alllocks> show lockedvnods Locked vnodes 0xc82fe984: 0xc82fe984: tag ufs, type VDIR tag ufs, type VDIR usecount 6, writecount 0, refcount 9 mountedhere 0 usecount 6, writecount 0, refcount 9 mountedhere 0 flags (VV_ROOT) flags (VV_ROOT) v_object 0xc8276220 ref 0 pages 1 v_object 0xc8276220 ref 0 pages 1 lock type ufs: SHARED (count 2) lock type ufs: SHARED (count 2) #0 0xc0995225 at __lockmgr_args+0x5a5 #0 0xc0995225 at __lockmgr_args+0x5a5 #1 0xc0be7e51 at ffs_lock+0xa1 #1 0xc0be7e51 at ffs_lock+0xa1 #2 0xc0cfda65 at VOP_LOCK1_APV+0xb5 #2 0xc0cfda65 at VOP_LOCK1_APV+0xb5 #3 0xc0a557f8 at _vn_lock+0x78 #3 0xc0a557f8 at _vn_lock+0x78 #4 0xc0a4892b at vget+0xbb #4 0xc0a4892b at vget+0xbb #5 0xc0a398dd at vfs_hash_get+0xed #5 0xc0a398dd at vfs_hash_get+0xed #6 0xc0be2359 at ffs_vgetf+0x49 #6 0xc0be2359 at ffs_vgetf+0x49 #7 0xc0be28be at ffs_vget+0x2e #7 0xc0be28be at ffs_vget+0x2e #8 0xc0bf6208 at ufs_root+0x28 #8 0xc0bf6208 at ufs_root+0x28 #9 0xc0a3af41 at lookup+0x9a1 #9 0xc0a3af41 at lookup+0x9a1 #10 0xc0a3ba82 at namei+0x592 #10 0xc0a3ba82 at namei+0x592 #11 0xc0a4e03c at kern_statfs+0x8c #11 0xc0a4e03c at kern_statfs+0x8c #12 0xc0a4e27b at statfs+0x3b #12 0xc0a4e27b at statfs+0x3b #13 0xc09f0473 at syscallenter+0x263 #13 0xc09f0473 at syscallenter+0x263 #14 0xc0cdb04f at syscall+0x4f #14 0xc0cdb04f at syscall+0x4f #15 0xc0cc4711 at Xint0x80_syscall+0x21 #15 0xc0cc4711 at Xint0x80_syscall+0x21 ino 2, on dev md5a ino 2, on dev md5a 0xc82fe6cc: 0xc82fe6cc: tag syncer, type VNON tag syncer, type VNON usecount 1, writecount 0, refcount 2 mountedhere 0 usecount 1, writecount 0, refcount 2 mountedhere 0 flags () flags () lock type syncer: EXCL by thread 0xc71752e0 (pid 16) lock type syncer: EXCL by thread 0xc71752e0 (pid 16) #0 0xc099587e at __lockmgr_args+0xbfe #0 0xc099587e at __lockmgr_args+0xbfe #1 0xc0a35865 at vop_stdlock+0x65 #1 0xc0a35865 at vop_stdlock+0x65 #2 0xc0cfda65 at VOP_LOCK1_APV+0xb5 #2 0xc0cfda65 at VOP_LOCK1_APV+0xb5 #3 0xc0a557f8 at _vn_lock+0x78 #3 0xc0a557f8 at _vn_lock+0x78 #4 0xc0a48032 at sync_vnode+0x142 #4 0xc0a48032 at sync_vnode+0x142 #5 0xc0a483b2 at sched_sync+0x282 #5 0xc0a483b2 at sched_sync+0x282 #6 0xc09809f8 at fork_exit+0xb8 #6 0xc09809f8 at fork_exit+0xb8 #7 0xc0cc4724 at fork_trampoline+0x8 #7 0xc0cc4724 at fork_trampoline+0x8 0xc8e312b8: 0xc8e312b8: tag ufs, type VDIR tag ufs, type VDIR usecount 1, writecount 0, refcount 6 mountedhere 0 usecount 1, writecount 0, refcount 6 mountedhere 0 flags () flags () v_object 0xc8e26110 ref 0 pages 2 v_object 0xc8e26110 ref 0 pages 2 lock type ufs: EXCL by thread 0xc71752e0 (pid 16) lock type ufs: EXCL by thread 0xc71752e0 (pid 16) with shared waiters pending with shared waiters pending #0 0xc099587e at __lockmgr_args+0xbfe #0 0xc099587e at __lockmgr_args+0xbfe #1 0xc0be7e51 at ffs_lock+0xa1 #1 0xc0be7e51 at ffs_lock+0xa1 #2 0xc0cfda65 at VOP_LOCK1_APV+0xb5 #2 0xc0cfda65 at VOP_LOCK1_APV+0xb5 #3 0xc0a557f8 at _vn_lock+0x78 #3 0xc0a557f8 at _vn_lock+0x78 #4 0xc0a4892b at vget+0xbb #4 0xc0a4892b at vget+0xbb #5 0xc0be3206 at ffs_sync+0x216 #5 0xc0be3206 at ffs_sync+0x216 #6 0xc0a4b20f at sync_fsync+0x18f #6 0xc0a4b20f at sync_fsync+0x18f #7 0xc0cfc295 at VOP_FSYNC_APV+0xc5 #7 0xc0cfc295 at VOP_FSYNC_APV+0xc5 #8 0xc0a4805b at sync_vnode+0x16b #8 0xc0a4805b at sync_vnode+0x16b #9 0xc0a483b2 at sched_sync+0x282 #9 0xc0a483b2 at sched_sync+0x282 #10 0xc09809f8 at fork_exit+0xb8 #10 0xc09809f8 at fork_exit+0xb8 #11 0xc0cc4724 at fork_trampoline+0x8 #11 0xc0cc4724 at fork_trampoline+0x8 ino 1345, on dev md5a ino 1345, on dev md5a 0xca313d98: 0xca313d98: tag ufs, type VREG tag ufs, type VREG usecount 1, writecount 0, refcount 114 mountedhere 0 usecount 1, writecount 0, refcount 114 mountedhere 0 flags (VV_SYSTEM) flags (VV_SYSTEM) v_object 0xca466dd0 ref 0 pages 51028 v_object 0xca466dd0 ref 0 pages 51028 lock type snaplk: EXCL by thread 0xc74388a0 (pid 2480) lock type snaplk: EXCL by thread 0xc74388a0 (pid 2480) with exclusive waiters pending with exclusive waiters pending #0 0xc099587e at __lockmgr_args+0xbfe #0 0xc099587e at __lockmgr_args+0xbfe #1 0xc0bc60e2 at ffs_copyonwrite+0x2c2 #1 0xc0bc60e2 at ffs_copyonwrite+0x2c2 #2 0xc0be2a2b at ffs_geom_strategy+0xeb #2 0xc0be2a2b at ffs_geom_strategy+0xeb #3 0xc0a2ebd9 at bufwrite+0x159 #3 0xc0a2ebd9 at bufwrite+0x159 #4 0xc0be20c0 at ffs_bufwrite+0x290 #4 0xc0be20c0 at ffs_bufwrite+0x290 #5 0xc0bc09bc at ffs_update+0x28c #5 0xc0bc09bc at ffs_update+0x28c #6 0xc0bde86f at softdep_request_cleanup+0x9f #6 0xc0bde86f at softdep_request_cleanup+0x9f #7 0xc0bba437 at ffs_realloccg+0xd37 #7 0xc0bba437 at ffs_realloccg+0xd37 #8 0xc0bbd485 at ffs_balloc_ufs2+0xf85 #8 0xc0bbd485 at ffs_balloc_ufs2+0xf85 #9 0xc0be741d at ffs_write+0x2ad #9 0xc0be741d at ffs_write+0x2ad #10 0xc0cfe596 at VOP_WRITE_APV+0x136 #10 0xc0cfe596 at VOP_WRITE_APV+0x136 #11 0xc0a56bda at vn_write+0x1ca #11 0xc0a56bda at vn_write+0x1ca #12 0xc09fa445 at dofilewrite+0x95 #12 0xc09fa445 at dofilewrite+0x95 #13 0xc09fbbe8 at kern_writev+0x58 #13 0xc09fbbe8 at kern_writev+0x58 #14 0xc09fbcff at write+0x4f #14 0xc09fbcff at write+0x4f #15 0xc09f0473 at syscallenter+0x263 #15 0xc09f0473 at syscallenter+0x263 #16 0xc0cdb04f at syscall+0x4f #16 0xc0cdb04f at syscall+0x4f #17 0xc0cc4711 at Xint0x80_syscall+0x21 #17 0xc0cc4711 at Xint0x80_syscall+0x21 ino 6461, on dev md5a ino 6461, on dev md5a 0xc92ab2b8: 0xc92ab2b8: tag ufs, type VREG tag ufs, type VREG usecount 1, writecount 0, refcount 389 mountedhere 0 usecount 1, writecount 0, refcount 389 mountedhere 0 flags (VV_SYSTEM) flags (VV_SYSTEM) v_object 0xc8385c38 ref 0 pages 14684 v_object 0xc8385c38 ref 0 pages 14684 lock type snaplk: EXCL by thread 0xc74388a0 (pid 2480) lock type snaplk: EXCL by thread 0xc74388a0 (pid 2480) with exclusive waiters pending with exclusive waiters pending #0 0xc099587e at __lockmgr_args+0xbfe #0 0xc099587e at __lockmgr_args+0xbfe #1 0xc0bc60e2 at ffs_copyonwrite+0x2c2 #1 0xc0bc60e2 at ffs_copyonwrite+0x2c2 #2 0xc0be2a2b at ffs_geom_strategy+0xeb #2 0xc0be2a2b at ffs_geom_strategy+0xeb #3 0xc0a2ebd9 at bufwrite+0x159 #3 0xc0a2ebd9 at bufwrite+0x159 #4 0xc0be20c0 at ffs_bufwrite+0x290 #4 0xc0be20c0 at ffs_bufwrite+0x290 #5 0xc0bc09bc at ffs_update+0x28c #5 0xc0bc09bc at ffs_update+0x28c #6 0xc0bde86f at softdep_request_cleanup+0x9f #6 0xc0bde86f at softdep_request_cleanup+0x9f #7 0xc0bba437 at ffs_realloccg+0xd37 #7 0xc0bba437 at ffs_realloccg+0xd37 #8 0xc0bbd485 at ffs_balloc_ufs2+0xf85 #8 0xc0bbd485 at ffs_balloc_ufs2+0xf85 #9 0xc0be741d at ffs_write+0x2ad #9 0xc0be741d at ffs_write+0x2ad #10 0xc0cfe596 at VOP_WRITE_APV+0x136 #10 0xc0cfe596 at VOP_WRITE_APV+0x136 #11 0xc0a56bda at vn_write+0x1ca #11 0xc0a56bda at vn_write+0x1ca #12 0xc09fa445 at dofilewrite+0x95 #12 0xc09fa445 at dofilewrite+0x95 #13 0xc09fbbe8 at kern_writev+0x58 #13 0xc09fbbe8 at kern_writev+0x58 #14 0xc09fbcff at write+0x4f #14 0xc09fbcff at write+0x4f #15 0xc09f0473 at syscallenter+0x263 #15 0xc09f0473 at syscallenter+0x263 #16 0xc0cdb04f at syscall+0x4f #16 0xc0cdb04f at syscall+0x4f #17 0xc0cc4711 at Xint0x80_syscall+0x21 #17 0xc0cc4711 at Xint0x80_syscall+0x21 ino 7, on dev md5a ino 7, on dev md5a 0xc9bef15c: 0xc9bef15c: tag ufs, type VREG tag ufs, type VREG usecount 0, writecount 0, refcount 3 mountedhere 0 usecount 0, writecount 0, refcount 3 mountedhere 0 flags () flags () v_object 0xc9be4ee0 ref 0 pages 5 v_object 0xc9be4ee0 ref 0 pages 5 lock type ufs: EXCL by thread 0xc71755c0 (pid 9) lock type ufs: EXCL by thread 0xc71755c0 (pid 9) #0 0xc099587e at __lockmgr_args+0xbfe #0 0xc099587e at __lockmgr_args+0xbfe #1 0xc0be7e51 at ffs_lock+0xa1 #1 0xc0be7e51 at ffs_lock+0xa1 #2 0xc0cfda65 at VOP_LOCK1_APV+0xb5 #2 0xc0cfda65 at VOP_LOCK1_APV+0xb5 #3 0xc0a557f8 at _vn_lock+0x78 #3 0xc0a557f8 at _vn_lock+0x78 #4 0xc0a2f4bb at flushbufqueues+0x56b #4 0xc0a2f4bb at flushbufqueues+0x56b #5 0xc0a2f7b9 at buf_do_flush+0x19 #5 0xc0a2f7b9 at buf_do_flush+0x19 #6 0xc0a2f98a at buf_daemon+0xea #6 0xc0a2f98a at buf_daemon+0xea #7 0xc09809f8 at fork_exit+0xb8 #7 0xc09809f8 at fork_exit+0xb8 #8 0xc0cc4724 at fork_trampoline+0x8 #8 0xc0cc4724 at fork_trampoline+0x8 ino 2118, on dev md5a ino 2118, on dev md5a 0xc9bce2b8: 0xc9bce2b8: tag ufs, type VREG tag ufs, type VREG usecount 1, writecount 1, refcount 3 mountedhere 0 usecount 1, writecount 1, refcount 3 mountedhere 0 flags () flags () v_object 0xc9de6660 ref 0 pages 1 v_object 0xc9de6660 ref 0 pages 1 lock type ufs: EXCL by thread 0xc74388a0 (pid 2480) lock type ufs: EXCL by thread 0xc74388a0 (pid 2480) #0 0xc099587e at __lockmgr_args+0xbfe #0 0xc099587e at __lockmgr_args+0xbfe #1 0xc0be7e51 at ffs_lock+0xa1 #1 0xc0be7e51 at ffs_lock+0xa1 #2 0xc0cfda65 at VOP_LOCK1_APV+0xb5 #2 0xc0cfda65 at VOP_LOCK1_APV+0xb5 #3 0xc0a557f8 at _vn_lock+0x78 #3 0xc0a557f8 at _vn_lock+0x78 #4 0xc0a56b66 at vn_write+0x156 #4 0xc0a56b66 at vn_write+0x156 #5 0xc09fa445 at dofilewrite+0x95 #5 0xc09fa445 at dofilewrite+0x95 #6 0xc09fbbe8 at kern_writev+0x58 #6 0xc09fbbe8 at kern_writev+0x58 #7 0xc09fbcff at write+0x4f #7 0xc09fbcff at write+0x4f #8 0xc09f0473 at syscallenter+0x263 #8 0xc09f0473 at syscallenter+0x263 #9 0xc0cdb04f at syscall+0x4f #9 0xc0cdb04f at syscall+0x4f #10 0xc0cc4711 at Xint0x80_syscall+0x21 #10 0xc0cc4711 at Xint0x80_syscall+0x21 ino 97038, on dev md5a ino 97038, on dev md5a db:0:lockedvnods> show mount 0xc73f1b50 /dev/ad4s1a on / (ufs) 0xc73f2000 devfs on /dev (devfs) 0xc73f187c /dev/ad4s1f on /home (ufs) 0xc77242d4 /dev/ad4s1d on /usr (ufs) 0xc7724000 /dev/ad4s1e on /var (ufs) 0xc73f2b50 procfs on /proc (procfs) 0xc73f15a8 /dev/label/tmp on /tmp (ufs) 0xc73f12d4 /dev/md5a on /mnt (ufs) More info: show mount db:0:mount> ps pid ppid pgrp uid state wmesg wchan cmd 6138 1 6138 0 Ss+ ttyin 0xc6f59a70 sh 5813 1 5813 0 D+ ufs 0xc8e31310 ls 2574 1 2266 0 D ufs 0xc8e31310 ls 2480 1 2406 0 R+ CPU 2 suj20 2425 0 0 0 DL mdwait 0xc7efb000 [md5] 18 0 0 0 DL snaplk 0xc78e6d1c [softdepflush] 17 0 0 0 DL vlruwt 0xc73a1578 [vnlru] 16 0 0 0 DL snaplk 0xc78e6d1c [syncer] 9 0 0 0 DL snaplk 0xc78e6d1c [bufdaemon] 8 0 0 0 DL pgzero 0xc119d0dc [pagezero] 7 0 0 0 DL psleep 0xc119ccc8 [vmdaemon] 6 0 0 0 DL psleep 0xc119cc90 [pagedaemon] 5 0 0 0 DL ccb_scan 0xc0fe3954 [xpt_thrd] 4 0 0 0 DL waiting_ 0xc1196798 [sctp_iterator] 3 0 0 0 DL - 0xc6fe9e3c [fdc0] 2 0 0 0 SL - 0xc706b000 [fw0_probe] 15 0 0 0 DL (threaded) [usb] 100057 D - 0xc704cd34 [usbus5] 100056 D - 0xc704cd04 [usbus5] 100055 D - 0xc704ccd4 [usbus5] 100054 D - 0xc704cca4 [usbus5] 100052 D - 0xc703eb5c [usbus4] 100051 D - 0xc703eb2c [usbus4] 100050 D - 0xc703eafc [usbus4] 100049 D - 0xc703eacc [usbus4] 100048 D - 0xc7038b5c [usbus3] 100047 D - 0xc7038b2c [usbus3] 100046 D - 0xc7038afc [usbus3] 100045 D - 0xc7038acc [usbus3] 100044 D - 0xc702fb5c [usbus2] 100043 D - 0xc702fb2c [usbus2] 100042 D - 0xc702fafc [usbus2] 100041 D - 0xc702facc [usbus2] 100039 D - 0xc7028b5c [usbus1] 100038 D - 0xc7028b2c [usbus1] 100037 D - 0xc7028afc [usbus1] 100036 D - 0xc7028acc [usbus1] 100034 D - 0xc701fb5c [usbus0] 100033 D - 0xc701fb2c [usbus0] 100032 D - 0xc701fafc [usbus0] 100031 D - 0xc701facc [usbus0] 14 0 0 0 DL - 0xc101d884 [yarrow] 13 0 0 0 DL (threaded) [geom] 100015 D - 0xc101b4a8 [g_down] 100014 D - 0xc101b4a4 [g_up] 100013 D - 0xc101b49c [g_event] 12 0 0 0 WL (threaded) [intr] 100065 I [irq12: psm0] 100064 I [irq1: atkbd0] 100062 I [swi0: uart] 100059 I [irq20: fwohci0] 100058 I [irq14: ata0] 100053 I [irq19: ehci0] 100040 I [irq18: ohci2 ohci4] 100035 I [irq17: ohci1 ohci3] 100030 I [irq16: ohci0] 100029 I [irq22: ahci0] 100028 I [irq256: re0] 100027 I [swi5: +] 100026 I [swi2: cambio] 100021 I [swi6: task queue] 100019 I [swi6: Giant taskq] 100012 I [swi3: vm] 100011 I [swi4: clock] 100010 I [swi4: clock] 100009 I [swi4: clock] 100008 I [swi4: clock] 100007 I [swi1: netisr 0] 11 0 0 0 RL (threaded) [idle] 100006 Run CPU 3 [idle: cpu3] 100005 CanRun [idle: cpu2] 100004 Run CPU 1 [idle: cpu1] 100003 Run CPU 0 [idle: cpu0] 1 0 1 0 SLs wait 0xc6dbcaf0 [init] 10 0 0 0 DL audit_wo 0xc119aa00 [audit] 0 0 0 0 DLs (threaded) [kernel] 100076 D - 0xc73729c0 [mca taskq] 100066 D - 0xc101d884 [deadlkres] 100060 D - 0xc7034480 [fw0_taskq] 100025 D - 0xc6f50540 [acpi_task_2] 100024 D - 0xc6f50540 [acpi_task_1] 100023 D - 0xc6f50540 [acpi_task_0] 100022 D - 0xc6f50580 [kqueue taskq] 100020 D - 0xc6f50600 [ffs_trim taskq] 100018 D - 0xc6f50680 [thread taskq] 100016 D - 0xc6d76b80 [firmware taskq] 100000 D sched 0xc101b580 [swapper] db:0:ps> allt Tracing command sh pid 6138 tid 100117 td 0xc77562e0 sched_switch(c77562e0,0,104,191,327c67ae,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c77562e0,0,c0e97420,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c099d73a,c6f59a04,0,c0e90227,c77562e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6f59a70,0,ef23ead8,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6f59a70,c6f59a04,c0e9badc,514,0,...) at _cv_wait_sig+0x243 tty_wait(c6f59a00,c6f59a70,ef23ec28,1,0,...) at tty_wait+0x71 ttydisc_read(c6f59a00,ef23ec28,0,9f,0,...) at ttydisc_read+0x22c ttydev_read(c6ec6000,ef23ec28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c7beb4d0,ef23ec28,c6d9fe00,0,c77562e0,...) at devfs_read_f+0x7e dofileread(ef23ec28,ffffffff,ffffffff,0,c7beb4d0,...) at dofileread+0x9e kern_readv(c77562e0,0,ef23ec28,ef23ec48,1,...) at kern_readv+0x58 read(c77562e0,ef23ecec,ef23ed28,c0e97daa,0,...) at read+0x4f syscallenter(c77562e0,ef23ece4,ef23ece4,0,0,...) at syscallenter+0x263 syscall(ef23ed28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x281fb643, esp = 0xbfbfecac, ebp = 0xbfbfecd8 --- Tracing command ls pid 5813 tid 100086 td 0xc743d8a0 sched_switch(c743d8a0,0,104,191,bba631fc,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,60,...) at mi_switch+0x1eb sleepq_switch(c743d8a0,0,c0e97420,268,c8e31310,...) at sleepq_switch+0x162 sleepq_wait(c8e31310,60,c0e88914,4,1,...) at sleepq_wait+0x63 __lockmgr_args(c8e31310,200100,c8e3137c,0,0,...) at __lockmgr_args+0x4cf ffs_lock(ef1c08e4,c09f7aab,c0e9fa89,200100,c8e312b8,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c0fc7500,ef1c08e4,c743d950,c0fe2500,c8e312b8,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c8e312b8,200100,c0ea1172,856,0,...) at _vn_lock+0x78 vget(c8e312b8,200100,c743d8a0,244,0,...) at vget+0xbb cache_lookup(c82fe984,ef1c0b44,ef1c0b58,ef1c0b44,c955e100,...) at cache_lookup+0x67b vfs_cache_lookup(ef1c0a20,c0ea1ee3,c0fe2140,200000,ef1c0b2c,...) at vfs_cache_lookup+0xad VOP_LOOKUP_APV(c0fc7500,ef1c0a20,ef1c0b58,1f5,ef1c0b44,...) at VOP_LOOKUP_APV+0xe5 lookup(ef1c0b2c,c0ea05bd,ee,c9,ffffff9c,...) at lookup+0x67b namei(ef1c0b2c,ef1c0acc,60,0,c743d8a0,...) at namei+0x592 kern_statat_vnhook(c743d8a0,200,ffffff9c,2842e8b8,0,...) at kern_statat_vnhook+0x72 kern_statat(c743d8a0,200,ffffff9c,2842e8b8,0,...) at kern_statat+0x3c kern_lstat(c743d8a0,2842e8b8,0,ef1c0be8,8b,...) at kern_lstat+0x36 lstat(c743d8a0,ef1c0cec,ef1c0d28,c0e97daa,0,...) at lstat+0x2f syscallenter(c743d8a0,ef1c0ce4,ef1c0ce4,0,0,...) at syscallenter+0x263 syscall(ef1c0d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (190, FreeBSD ELF32, lstat), eip = 0x281c8623, esp = 0xbfbfe2ec, ebp = 0xbfbfe378 --- Tracing command ls pid 2574 tid 100196 td 0xc82f78a0 sched_switch(c82f78a0,0,104,191,267b4673,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,60,...) at mi_switch+0x1eb sleepq_switch(c82f78a0,0,c0e97420,268,c8e31310,...) at sleepq_switch+0x162 sleepq_wait(c8e31310,60,c0e88914,4,1,...) at sleepq_wait+0x63 __lockmgr_args(c8e31310,200100,c8e3137c,0,0,...) at __lockmgr_args+0x4cf ffs_lock(ef38f8e4,c09f7aab,c0e9fa89,200100,c8e312b8,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c0fc7500,ef38f8e4,c82f7950,c0fe2500,c8e312b8,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c8e312b8,200100,c0ea1172,856,0,...) at _vn_lock+0x78 vget(c8e312b8,200100,c82f78a0,244,0,...) at vget+0xbb cache_lookup(c82fe984,ef38fb44,ef38fb58,ef38fb44,c6d9fe00,...) at cache_lookup+0x67b vfs_cache_lookup(ef38fa20,c0ea1ee3,c0fe2140,200000,ef38fb2c,...) at vfs_cache_lookup+0xad VOP_LOOKUP_APV(c0fc7500,ef38fa20,ef38fb58,1f5,ef38fb44,...) at VOP_LOOKUP_APV+0xe5 lookup(ef38fb2c,c0ea05bd,ee,c9,ffffff9c,...) at lookup+0x67b namei(ef38fb2c,ef38facc,60,0,c82f78a0,...) at namei+0x592 kern_statat_vnhook(c82f78a0,200,ffffff9c,2841f6f8,0,...) at kern_statat_vnhook+0x72 kern_statat(c82f78a0,200,ffffff9c,2841f6f8,0,...) at kern_statat+0x3c kern_lstat(c82f78a0,2841f6f8,0,ef38fbe8,8b,...) at kern_lstat+0x36 lstat(c82f78a0,ef38fcec,ef38fd28,c0e97daa,0,...) at lstat+0x2f syscallenter(c82f78a0,ef38fce4,ef38fce4,0,0,...) at syscallenter+0x263 syscall(ef38fd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (190, FreeBSD ELF32, lstat), eip = 0x281c8623, esp = 0xbfbfe57c, ebp = 0xbfbfe608 --- Tracing command suj20 pid 2480 tid 100103 td 0xc74388a0 cpustop_handler(4,ef20c33c,c0cdb166,ef20c2f8,c0ce2c2c,...) at cpustop_handler+0x49 ipi_nmi_handler(ef20c2f8,c0ce2c2c,c11f2480,ef20c2dc,c7755000,...) at ipi_nmi_handler+0x44 trap(ef20c348) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc09f59a0, esp = 0xef20c388, ebp = 0xef20c394 --- find_instance(c1167400,c0ec4aa9,c0fac4f8,c11673d0,c736c048,...) at find_instance+0x40 witness_lock(c736c048,8,c0ec4aa0,30cf,0,...) at witness_lock+0x8c _mtx_lock_flags(c736c048,0,c0ec4aa0,30cf,ef20c41c,...) at _mtx_lock_flags+0x113 softdep_request_cleanup(c7efa800,c92ab2b8,c6da0000,4,0,...) at softdep_request_cleanup+0x39b ffs_balloc_ufs2(c92ab2b8,2dfdc000,0,4000,c6da0000,...) at ffs_balloc_ufs2+0x1bc0 ffs_copyonwrite(c82feae0,e09a02a0,e09a02a0,0,0,...) at ffs_copyonwrite+0x502 ffs_geom_strategy(c82febd8,e09a02a0,df,e09a02a0,c9b7a828,...) at ffs_geom_strategy+0xeb bufwrite(e09a02a0,0,c0ec7950,768,0) at bufwrite+0x159 ffs_bufwrite(e09a02a0,c9bb7600,100,4000,0,...) at ffs_bufwrite+0x290 ffs_update(c9bce2b8,1,c0ec4aa0,308f,ef20c774,...) at ffs_update+0x28c softdep_request_cleanup(c7efa800,c9bce2b8,c701c880,4,1,...) at softdep_request_cleanup+0x9f ffs_realloccg(c9b7a828,0,0,60722,0,...) at ffs_realloccg+0xd37 ffs_balloc_ufs2(c9bce2b8,1000,0,1000,c701c880,...) at ffs_balloc_ufs2+0xf85 ffs_write(ef20cb94,c0ee68a3,c9bce2b8,0,c9bce2b8,...) at ffs_write+0x2ad VOP_WRITE_APV(c0fc7500,ef20cb94,c9bce2b8,264,0,...) at VOP_WRITE_APV+0x136 vn_write(c744cab8,ef20cc28,c701c880,0,c74388a0,...) at vn_write+0x1ca dofilewrite(ef20cc28,ffffffff,ffffffff,0,c744cab8,...) at dofilewrite+0x95 kern_writev(c74388a0,3,ef20cc28,ef20cc48,1,...) at kern_writev+0x58 write(c74388a0,ef20ccec,ef20cd28,c0e97daa,0,...) at write+0x4f syscallenter(c74388a0,ef20cce4,ef20cce4,0,c102c400,...) at syscallenter+0x263 syscall(ef20cd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x28181623, esp = 0xbfbfe7cc, ebp = 0xbfbfe878 --- Tracing command md5 pid 2425 tid 100195 td 0xc82f7b80 sched_switch(c82f7b80,0,104,191,33695009,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c82f7b80,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c7efb000,5c,c0e3da91,0,0,...) at sleepq_wait+0x63 _sleep(c7efb000,c7efb020,25c,c0e3da91,0,...) at _sleep+0x372 md_kthread(c7efb000,ef38cd28,c0e8c598,390,c826f578,...) at md_kthread+0x11a fork_exit(c072b540,c7efb000,ef38cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xef38cd60, ebp = 0 --- Tracing command softdepflush pid 18 tid 100075 td 0xc7045b80 sched_switch(c7045b80,0,104,191,b5298e5a,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,60,...) at mi_switch+0x1eb sleepq_switch(c7045b80,0,c0e97420,268,60,...) at sleepq_switch+0x162 sleepq_wait(c78e6d1c,60,c0ec4545,4,0,...) at sleepq_wait+0x63 __lockmgr_args(c78e6d1c,80900,c82feba4,0,0,...) at __lockmgr_args+0xb1f ffs_snapblkfree(c7efa800,c82feae0,6d926,0,1000,...) at ffs_snapblkfree+0xea ffs_blkfree(c736c000,c7efa800,c82feae0,6d926,0,...) at ffs_blkfree+0x55 handle_workitem_freefrag(0,ecfa6c6c,2,6b3,ecfa6c70,...) at handle_workitem_freefrag+0xe9 process_worklist_item(c119b230,0,c0ec4aa0,5e2,c7045b80,...) at process_worklist_item+0x2cd softdep_process_worklist(c73f12d4,0,c0ec4aa0,55a,3e8,...) at softdep_process_worklist+0x88 softdep_flush(0,ecfa6d28,c0e8c598,390,c73a12bc,...) at softdep_flush+0x2a0 fork_exit(c0bdfbe0,0,ecfa6d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecfa6d60, ebp = 0 --- Tracing command vnlru pid 17 tid 100074 td 0xc7175000 sched_switch(c7175000,0,104,191,2ed32346,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,60,...) at mi_switch+0x1eb sleepq_switch(c7175000,0,c0e97420,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c73a1578,60,c0ea20e4,0,0,...) at sleepq_timedwait+0x6b _sleep(c73a1578,c1194c34,260,c0ea20e4,3e8,...) at _sleep+0x342 vnlru_proc(0,ecfa3d28,c0e8c598,390,c73a1578,...) at vnlru_proc+0xe7 fork_exit(c0a4a190,0,ecfa3d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecfa3d60, ebp = 0 --- Tracing command syncer pid 16 tid 100073 td 0xc71752e0 sched_switch(c71752e0,0,104,191,339de0c5,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,60,...) at mi_switch+0x1eb sleepq_switch(c71752e0,0,c0e97420,268,60,...) at sleepq_switch+0x162 sleepq_wait(c78e6d1c,60,c0ec4545,4,0,...) at sleepq_wait+0x63 __lockmgr_args(c78e6d1c,80900,c82feba4,0,0,...) at __lockmgr_args+0xb1f ffs_copyonwrite(c82feae0,e07f90e0,e07f90e0,0,0,...) at ffs_copyonwrite+0x2c2 ffs_geom_strategy(c82febd8,e07f90e0,df,e07f90e0,c8e28570,...) at ffs_geom_strategy+0xeb bufwrite(e07f90e0,0,c0ec7950,768,0) at bufwrite+0x159 ffs_bufwrite(e07f90e0,c8e25700,100,4000,0,...) at ffs_bufwrite+0x290 ffs_update(c8e312b8,3,c0ec7f79,12f,0,...) at ffs_update+0x28c ffs_syncvnode(c8e312b8,3,c71752e0,566,c0e90227,...) at ffs_syncvnode+0x423 ffs_sync(c73f12d4,3,c0ea1172,db7,c73f12d4,...) at ffs_sync+0x26f sync_fsync(ecfa0c6c,c0ee6201,c82fe790,ecfa0c6c,c82fe6cc,...) at sync_fsync+0x18f VOP_FSYNC_APV(c0fb5d40,ecfa0c6c,c0ea1172,6c6,c71752e0,...) at VOP_FSYNC_APV+0xc5 sync_vnode(c1194c74,c1194c60,3e8,76a,4e20,...) at sync_vnode+0x16b sched_sync(0,ecfa0d28,c0e8c598,390,c73a1834,...) at sched_sync+0x282 fork_exit(c0a48130,0,ecfa0d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecfa0d60, ebp = 0 --- Tracing command bufdaemon pid 9 tid 100072 td 0xc71755c0 sched_switch(c71755c0,0,104,191,b52896f1,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,60,...) at mi_switch+0x1eb sleepq_switch(c71755c0,0,c0e97420,268,60,...) at sleepq_switch+0x162 sleepq_wait(c78e6d1c,60,c0ec4545,4,0,...) at sleepq_wait+0x63 __lockmgr_args(c78e6d1c,80900,c82feba4,0,0,...) at __lockmgr_args+0xb1f ffs_copyonwrite(c82feae0,e0720ee0,0,e0720ee0,c9bef15c,...) at ffs_copyonwrite+0x2c2 ffs_geom_strategy(c82febd8,e0720ee0,ecf9db5c,3,c71755c0,...) at ffs_geom_strategy+0xeb ufs_strategy(ecf9db98,c0ee6668,c71755c0,e0720ee0,4,...) at ufs_strategy+0xd3 VOP_STRATEGY_APV(c0fc7500,ecf9db98,c0e9e7cb,c0fe2560,c9bef15c,...) at VOP_STRATEGY_APV+0xe5 bufstrategy(c9bef254,e0720ee0,3a6,1000,0,...) at bufstrategy+0x61 bufwrite(e0720ee0,0,c0e9e7cb,6dc,c0ea13bf,...) at bufwrite+0x159 vfs_bio_awrite(e0720ee0,0,c0e9e7cb,945,0,...) at vfs_bio_awrite+0x318 flushbufqueues(d0,c0e90227,c73a1b78,391,391,...) at flushbufqueues+0x5b0 buf_do_flush(c73a1af0,0,c0e9e7cb,8ac,3e8,...) at buf_do_flush+0x19 buf_daemon(0,ecf9dd28,c0e8c598,390,c73a1af0,...) at buf_daemon+0xea fork_exit(c0a2f8a0,0,ecf9dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf9dd60, ebp = 0 --- Tracing command pagezero pid 8 tid 100071 td 0xc71758a0 sched_switch(c71758a0,0,104,191,af58941c,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c71758a0,0,c0e97420,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c119d0dc,0,c0ecd968,0,0,...) at sleepq_timedwait+0x6b _sleep(c119d0dc,c119bb80,0,c0ecd968,493e0,...) at _sleep+0x342 vm_pagezero(0,ecf9ad28,c0e8c598,390,c73a3000,...) at vm_pagezero+0xdc fork_exit(c0c239b0,0,ecf9ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf9ad60, ebp = 0 --- Tracing command vmdaemon pid 7 tid 100070 td 0xc7175b80 sched_switch(c7175b80,0,104,191,17255aaa,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,74,...) at mi_switch+0x1eb sleepq_switch(c7175b80,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c119ccc8,74,c0e9f112,0,0,...) at sleepq_wait+0x63 _sleep(c119ccc8,c119cccc,74,c0e9f112,0,...) at _sleep+0x372 vm_daemon(0,ecf97d28,c0e8c598,390,c6dbd2bc,...) at vm_daemon+0x59 fork_exit(c0c1d570,0,ecf97d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf97d60, ebp = 0 --- Tracing command pagedaemon pid 6 tid 100069 td 0xc7176000 sched_switch(c7176000,0,104,191,5807bf8f,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,54,...) at mi_switch+0x1eb sleepq_switch(c7176000,0,c0e97420,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c119cc90,54,c0e9f112,0,0,...) at sleepq_timedwait+0x6b _sleep(c119cc90,c119bb80,54,c0e9f112,1388,...) at _sleep+0x342 vm_pageout(0,ecf94d28,c0e8c598,390,c6dbd578,...) at vm_pageout+0x2c7 fork_exit(c0c1e590,0,ecf94d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf94d60, ebp = 0 --- Tracing command xpt_thrd pid 5 tid 100068 td 0xc71762e0 sched_switch(c71762e0,0,104,191,12e586a2,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c71762e0,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c0fe3954,5c,c0d02a32,0,0,...) at sleepq_wait+0x63 _sleep(c0fe3954,c0fe3998,5c,c0d02a32,0,...) at _sleep+0x372 xpt_scanner_thread(0,ecf91d28,c0e8c598,390,c6dbd834,...) at xpt_scanner_thread+0x47 fork_exit(c048c530,0,ecf91d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf91d60, ebp = 0 --- Tracing command sctp_iterator pid 4 tid 100067 td 0xc71765c0 sched_switch(c71765c0,0,104,191,1593b0e0,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c71765c0,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c1196798,0,c0eb533f,0,0,...) at sleepq_wait+0x63 _sleep(c1196798,c1196760,0,c0eb533f,0,...) at _sleep+0x372 sctp_iterator_thread(0,ecf8ed28,c0e8c598,390,c6dbdaf0,...) at sctp_iterator_thread+0x5c fork_exit(c0ad9ac0,0,ecf8ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf8ed60, ebp = 0 --- Tracing command fdc0 pid 3 tid 100063 td 0xc70372e0 sched_switch(c70372e0,0,104,191,e5a72bcc,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c70372e0,0,c0e97420,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c6fe9e3c,5c,c0e84d12,0,0,...) at sleepq_timedwait+0x6b _sleep(c6fe9e3c,c6fe9ef0,5c,c0e84d12,3e8,...) at _sleep+0x342 fdc_thread(c6fe9e00,ecf82d28,c0e8c598,390,c7024000,...) at fdc_thread+0x27d fork_exit(c0ca2bd0,c6fe9e00,ecf82d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf82d60, ebp = 0 --- Tracing command fw0_probe pid 2 tid 100061 td 0xc70378a0 sched_switch(c70378a0,0,104,191,1346b15d,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c70378a0,0,c0e97420,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c706b000,6c,c0e84d12,100,0,...) at sleepq_wait_sig+0x17 _sleep(c706b000,c706f488,16c,c0e84d12,0,...) at _sleep+0x35c fw_bus_probe_thread(c706b000,ecf72d28,c0e8c598,390,c70242bc,...) at fw_bus_probe_thread+0xa08 fork_exit(c06ae8d0,c706b000,ecf72d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf72d60, ebp = 0 --- Tracing command usb pid 15 tid 100057 td 0xc70435c0 sched_switch(c70435c0,0,104,191,4bf95f,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70435c0,0,c0e97420,268,c70435c0,...) at sleepq_switch+0x162 sleepq_wait(c704cd34,0,ecf5fcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c704cd34,c704cdd4,c0e632de,6b,c704cd3c,...) at _cv_wait+0x243 usb_process(c704cd2c,ecf5fd28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c704cd2c,ecf5fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf5fd60, ebp = 0 --- Tracing command usb pid 15 tid 100056 td 0xc70438a0 sched_switch(c70438a0,0,104,191,a8b88ef4,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70438a0,0,c0e97420,268,c70438a0,...) at sleepq_switch+0x162 sleepq_wait(c704cd04,0,ecf5ccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c704cd04,c704cdd4,c0e632de,6b,c704cd0c,...) at _cv_wait+0x243 usb_process(c704ccfc,ecf5cd28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c704ccfc,ecf5cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf5cd60, ebp = 0 --- Tracing command usb pid 15 tid 100055 td 0xc7043b80 sched_switch(c7043b80,0,104,191,ffe47967,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7043b80,0,c0e97420,268,c7043b80,...) at sleepq_switch+0x162 sleepq_wait(c704ccd4,0,ecf59cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c704ccd4,c704cdd4,c0e632de,6b,c704ccdc,...) at _cv_wait+0x243 usb_process(c704cccc,ecf59d28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c704cccc,ecf59d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf59d60, ebp = 0 --- Tracing command usb pid 15 tid 100054 td 0xc7045000 sched_switch(c7045000,0,104,191,ffe46794,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7045000,0,c0e97420,268,c7045000,...) at sleepq_switch+0x162 sleepq_wait(c704cca4,0,ecf56cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c704cca4,c704cdd4,c0e632de,6b,c704ccac,...) at _cv_wait+0x243 usb_process(c704cc9c,ecf56d28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c704cc9c,ecf56d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf56d60, ebp = 0 --- Tracing command usb pid 15 tid 100052 td 0xc70455c0 sched_switch(c70455c0,0,104,191,ffe45289,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70455c0,0,c0e97420,268,c70455c0,...) at sleepq_switch+0x162 sleepq_wait(c703eb5c,0,ecf4dcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c703eb5c,c703ebfc,c0e632de,6b,c703eb64,...) at _cv_wait+0x243 usb_process(c703eb54,ecf4dd28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c703eb54,ecf4dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf4dd60, ebp = 0 --- Tracing command usb pid 15 tid 100051 td 0xc7035000 sched_switch(c7035000,0,104,191,a3cae687,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7035000,0,c0e97420,268,c7035000,...) at sleepq_switch+0x162 sleepq_wait(c703eb2c,0,ecf4acac,1,0,...) at sleepq_wait+0x63 _cv_wait(c703eb2c,c703ebfc,c0e632de,6b,c703eb34,...) at _cv_wait+0x243 usb_process(c703eb24,ecf4ad28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c703eb24,ecf4ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf4ad60, ebp = 0 --- Tracing command usb pid 15 tid 100050 td 0xc70352e0 sched_switch(c70352e0,0,104,191,ff80888d,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70352e0,0,c0e97420,268,c70352e0,...) at sleepq_switch+0x162 sleepq_wait(c703eafc,0,ecf47cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c703eafc,c703ebfc,c0e632de,6b,c703eb04,...) at _cv_wait+0x243 usb_process(c703eaf4,ecf47d28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c703eaf4,ecf47d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf47d60, ebp = 0 --- Tracing command usb pid 15 tid 100049 td 0xc70355c0 sched_switch(c70355c0,0,104,191,ff8075d1,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70355c0,0,c0e97420,268,c70355c0,...) at sleepq_switch+0x162 sleepq_wait(c703eacc,0,ecf44cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c703eacc,c703ebfc,c0e632de,6b,c703ead4,...) at _cv_wait+0x243 usb_process(c703eac4,ecf44d28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c703eac4,ecf44d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf44d60, ebp = 0 --- Tracing command usb pid 15 tid 100048 td 0xc70358a0 sched_switch(c70358a0,0,104,191,ff805ed0,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70358a0,0,c0e97420,268,c70358a0,...) at sleepq_switch+0x162 sleepq_wait(c7038b5c,0,ecf40cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7038b5c,c7038bfc,c0e632de,6b,c7038b64,...) at _cv_wait+0x243 usb_process(c7038b54,ecf40d28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c7038b54,ecf40d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf40d60, ebp = 0 --- Tracing command usb pid 15 tid 100047 td 0xc7035b80 sched_switch(c7035b80,0,104,191,a3ca713f,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7035b80,0,c0e97420,268,c7035b80,...) at sleepq_switch+0x162 sleepq_wait(c7038b2c,0,ecf3dcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7038b2c,c7038bfc,c0e632de,6b,c7038b34,...) at _cv_wait+0x243 usb_process(c7038b24,ecf3dd28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c7038b24,ecf3dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf3dd60, ebp = 0 --- Tracing command usb pid 15 tid 100046 td 0xc7036000 sched_switch(c7036000,0,104,191,f9a6d7b4,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7036000,0,c0e97420,268,c7036000,...) at sleepq_switch+0x162 sleepq_wait(c7038afc,0,ecf3acac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7038afc,c7038bfc,c0e632de,6b,c7038b04,...) at _cv_wait+0x243 usb_process(c7038af4,ecf3ad28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c7038af4,ecf3ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf3ad60, ebp = 0 --- Tracing command usb pid 15 tid 100045 td 0xc70362e0 sched_switch(c70362e0,0,104,191,f9a6c5ca,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70362e0,0,c0e97420,268,c70362e0,...) at sleepq_switch+0x162 sleepq_wait(c7038acc,0,ecf37cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7038acc,c7038bfc,c0e632de,6b,c7038ad4,...) at _cv_wait+0x243 usb_process(c7038ac4,ecf37d28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c7038ac4,ecf37d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf37d60, ebp = 0 --- Tracing command usb pid 15 tid 100044 td 0xc70365c0 sched_switch(c70365c0,0,104,191,f9a6b329,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70365c0,0,c0e97420,268,c70365c0,...) at sleepq_switch+0x162 sleepq_wait(c702fb5c,0,ecf33cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c702fb5c,c702fbfc,c0e632de,6b,c702fb64,...) at _cv_wait+0x243 usb_process(c702fb54,ecf33d28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c702fb54,ecf33d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf33d60, ebp = 0 --- Tracing command usb pid 15 tid 100043 td 0xc70368a0 sched_switch(c70368a0,0,104,191,a3c9dc42,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70368a0,0,c0e97420,268,c70368a0,...) at sleepq_switch+0x162 sleepq_wait(c702fb2c,0,e0bffcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c702fb2c,c702fbfc,c0e632de,6b,c702fb34,...) at _cv_wait+0x243 usb_process(c702fb24,e0bffd28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c702fb24,e0bffd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0bffd60, ebp = 0 --- Tracing command usb pid 15 tid 100042 td 0xc7036b80 sched_switch(c7036b80,0,104,191,f9426eb0,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7036b80,0,c0e97420,268,c7036b80,...) at sleepq_switch+0x162 sleepq_wait(c702fafc,0,e0bfccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c702fafc,c702fbfc,c0e632de,6b,c702fb04,...) at _cv_wait+0x243 usb_process(c702faf4,e0bfcd28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c702faf4,e0bfcd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0bfcd60, ebp = 0 --- Tracing command usb pid 15 tid 100041 td 0xc7037000 sched_switch(c7037000,0,104,191,f9425c80,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7037000,0,c0e97420,268,c7037000,...) at sleepq_switch+0x162 sleepq_wait(c702facc,0,e0bf9cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c702facc,c702fbfc,c0e632de,6b,c702fad4,...) at _cv_wait+0x243 usb_process(c702fac4,e0bf9d28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c702fac4,e0bf9d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0bf9d60, ebp = 0 --- Tracing command usb pid 15 tid 100039 td 0xc70252e0 sched_switch(c70252e0,0,104,191,f942453d,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70252e0,0,c0e97420,268,c70252e0,...) at sleepq_switch+0x162 sleepq_wait(c7028b5c,0,e0bf2cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7028b5c,c7028bfc,c0e632de,6b,c7028b64,...) at _cv_wait+0x243 usb_process(c7028b54,e0bf2d28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c7028b54,e0bf2d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0bf2d60, ebp = 0 --- Tracing command usb pid 15 tid 100038 td 0xc70255c0 sched_switch(c70255c0,0,104,191,a3c96650,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70255c0,0,c0e97420,268,c70255c0,...) at sleepq_switch+0x162 sleepq_wait(c7028b2c,0,e0befcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7028b2c,c7028bfc,c0e632de,6b,c7028b34,...) at _cv_wait+0x243 usb_process(c7028b24,e0befd28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c7028b24,e0befd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0befd60, ebp = 0 --- Tracing command usb pid 15 tid 100037 td 0xc70258a0 sched_switch(c70258a0,0,104,191,f8de0c1d,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70258a0,0,c0e97420,268,c70258a0,...) at sleepq_switch+0x162 sleepq_wait(c7028afc,0,e0beccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7028afc,c7028bfc,c0e632de,6b,c7028b04,...) at _cv_wait+0x243 usb_process(c7028af4,e0becd28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c7028af4,e0becd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0becd60, ebp = 0 --- Tracing command usb pid 15 tid 100036 td 0xc7025b80 sched_switch(c7025b80,0,104,191,f8ddfa71,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7025b80,0,c0e97420,268,c7025b80,...) at sleepq_switch+0x162 sleepq_wait(c7028acc,0,e0be9cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7028acc,c7028bfc,c0e632de,6b,c7028ad4,...) at _cv_wait+0x243 usb_process(c7028ac4,e0be9d28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c7028ac4,e0be9d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0be9d60, ebp = 0 --- Tracing command usb pid 15 tid 100034 td 0xc70272e0 sched_switch(c70272e0,0,104,191,f8dde5dd,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70272e0,0,c0e97420,268,c70272e0,...) at sleepq_switch+0x162 sleepq_wait(c701fb5c,0,e0be2cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c701fb5c,c701fbfc,c0e632de,6b,c701fb64,...) at _cv_wait+0x243 usb_process(c701fb54,e0be2d28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c701fb54,e0be2d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0be2d60, ebp = 0 --- Tracing command usb pid 15 tid 100033 td 0xc70275c0 sched_switch(c70275c0,0,104,191,a3c8da64,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70275c0,0,c0e97420,268,c70275c0,...) at sleepq_switch+0x162 sleepq_wait(c701fb2c,0,e0bdfcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c701fb2c,c701fbfc,c0e632de,6b,c701fb34,...) at _cv_wait+0x243 usb_process(c701fb24,e0bdfd28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c701fb24,e0bdfd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0bdfd60, ebp = 0 --- Tracing command usb pid 15 tid 100032 td 0xc70278a0 sched_switch(c70278a0,0,104,191,f879a5d4,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70278a0,0,c0e97420,268,c70278a0,...) at sleepq_switch+0x162 sleepq_wait(c701fafc,0,e0bdccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c701fafc,c701fbfc,c0e632de,6b,c701fb04,...) at _cv_wait+0x243 usb_process(c701faf4,e0bdcd28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c701faf4,e0bdcd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0bdcd60, ebp = 0 --- Tracing command usb pid 15 tid 100031 td 0xc7027b80 sched_switch(c7027b80,0,104,191,f87990a5,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7027b80,0,c0e97420,268,c7027b80,...) at sleepq_switch+0x162 sleepq_wait(c701facc,0,e0bd9cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c701facc,c701fbfc,c0e632de,6b,c701fad4,...) at _cv_wait+0x243 usb_process(c701fac4,e0bd9d28,c0e8c598,390,c7024578,...) at usb_process+0x193 fork_exit(c08543c0,c701fac4,e0bd9d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0bd9d60, ebp = 0 --- Tracing command yarrow pid 14 tid 100017 td 0xc6ecc5c0 sched_switch(c6ecc5c0,0,104,191,31d4215c,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c6ecc5c0,0,c0e97420,28b,2,...) at sleepq_switch+0x162 sleepq_timedwait(c101d884,0,c0e84d12,2,0,...) at sleepq_timedwait+0x6b _sleep(c101d884,0,0,c0e84d12,64,...) at _sleep+0x342 pause(c0e84d12,64,c0e51cae,111,0,...) at pause+0x47 random_kthread(0,c6b65d28,c0e8c598,390,c6dbc000,...) at random_kthread+0x1ef fork_exit(c07bf4a0,0,c6b65d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b65d60, ebp = 0 --- Tracing command geom pid 13 tid 100015 td 0xc6eccb80 sched_switch(c6eccb80,0,104,191,3bea2194,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c6eccb80,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c101b4a8,5c,c0e84d12,0,0,...) at sleepq_wait+0x63 _sleep(c101b4a8,c101b408,25c,c0e84d12,0,...) at _sleep+0x372 g_io_schedule_down(c6eccb80,0,c0e86865,6c,c6b5fd14,...) at g_io_schedule_down+0x5b g_down_procbody(0,c6b5fd28,c0e8c598,390,c6dbc2bc,...) at g_down_procbody+0x9d fork_exit(c0943060,0,c6b5fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b5fd60, ebp = 0 --- Tracing command geom pid 13 tid 100014 td 0xc6ece000 sched_switch(c6ece000,0,104,191,3c4f5f19,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c6ece000,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c101b4a4,5c,c0e84d12,0,0,...) at sleepq_wait+0x63 _sleep(c101b4a4,c101b428,25c,c0e84d12,0,...) at _sleep+0x372 g_io_schedule_up(c6ece000,0,c0e86865,5f,c6b5cd14,...) at g_io_schedule_up+0x134 g_up_procbody(0,c6b5cd28,c0e8c598,390,c6dbc2bc,...) at g_up_procbody+0x9d fork_exit(c0943100,0,c6b5cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b5cd60, ebp = 0 --- Tracing command geom pid 13 tid 100013 td 0xc6dbf2e0 sched_switch(c6dbf2e0,0,104,191,bcb12360,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c6dbf2e0,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c101b49c,5c,c0e84d12,0,0,...) at sleepq_wait+0x63 _sleep(c101b49c,c101b3cc,25c,c0e84d12,0,...) at _sleep+0x372 g_run_events(c102b700,0,c0e86865,79,c6b59d14,...) at g_run_events+0x526 g_event_procbody(0,c6b59d28,c0e8c598,390,c6dbc2bc,...) at g_event_procbody+0x90 fork_exit(c09431a0,0,c6b59d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b59d60, ebp = 0 --- Tracing command intr pid 12 tid 100065 td 0xc7176b80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100064 td 0xc7177000 sched_switch(c7177000,0,109,191,f875ceaa,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c7177000,...) at mi_switch+0x1eb ithread_loop(c6dbaa00,ecf85d28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbaa00,ecf85d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf85d60, ebp = 0 --- Tracing command intr pid 12 tid 100062 td 0xc70375c0 sched_switch(c70375c0,0,109,191,327c286e,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c70375c0,...) at mi_switch+0x1eb ithread_loop(c6dbaa10,ecf75d28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbaa10,ecf75d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf75d60, ebp = 0 --- Tracing command intr pid 12 tid 100059 td 0xc7043000 sched_switch(c7043000,0,109,191,f8750e6e,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c7043000,...) at mi_switch+0x1eb ithread_loop(c6dbaa40,ecf69d28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbaa40,ecf69d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf69d60, ebp = 0 --- Tracing command intr pid 12 tid 100058 td 0xc70432e0 sched_switch(c70432e0,0,109,191,13eb9753,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c70432e0,...) at mi_switch+0x1eb ithread_loop(c6dbaa70,ecf65d28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbaa70,ecf65d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf65d60, ebp = 0 --- Tracing command intr pid 12 tid 100053 td 0xc70452e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100040 td 0xc7025000 sched_switch(c7025000,0,109,191,1d8bb3c4,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c7025000,...) at mi_switch+0x1eb ithread_loop(c6dbaa90,e0bf6d28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbaa90,e0bf6d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0bf6d60, ebp = 0 --- Tracing command intr pid 12 tid 100035 td 0xc7027000 sched_switch(c7027000,0,109,191,1812267f,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c7027000,...) at mi_switch+0x1eb ithread_loop(c6dbaaa0,e0be6d28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbaaa0,e0be6d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0be6d60, ebp = 0 --- Tracing command intr pid 12 tid 100030 td 0xc6ece2e0 sched_switch(c6ece2e0,0,109,191,16249df9,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c6ece2e0,...) at mi_switch+0x1eb ithread_loop(c6dbaab0,e0bd6d28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbaab0,e0bd6d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0bd6d60, ebp = 0 --- Tracing command intr pid 12 tid 100029 td 0xc6ece5c0 sched_switch(c6ece5c0,0,109,191,3c4e00a2,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c6ece5c0,...) at mi_switch+0x1eb ithread_loop(c6dbab40,e0a80d28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbab40,e0a80d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0a80d60, ebp = 0 --- Tracing command intr pid 12 tid 100028 td 0xc6ece8a0 sched_switch(c6ece8a0,0,109,191,9e277f10,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c6ece8a0,...) at mi_switch+0x1eb ithread_loop(c6dbab50,e0a7cd28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbab50,e0a7cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0a7cd60, ebp = 0 --- Tracing command intr pid 12 tid 100027 td 0xc6eceb80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100026 td 0xc6fb3000 sched_switch(c6fb3000,0,109,191,3c4eb953,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c6fb3000,...) at mi_switch+0x1eb ithread_loop(c6dbab80,c6b80d28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbab80,c6b80d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b80d60, ebp = 0 --- Tracing command intr pid 12 tid 100021 td 0xc6e048a0 sched_switch(c6e048a0,0,109,191,ec2677c5,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c6e048a0,...) at mi_switch+0x1eb ithread_loop(c6dbabc0,c6b71d28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbabc0,c6b71d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b71d60, ebp = 0 --- Tracing command intr pid 12 tid 100019 td 0xc6ecc000 sched_switch(c6ecc000,0,109,191,36be39b1,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c6ecc000,...) at mi_switch+0x1eb ithread_loop(c6dbabd0,c6b6bd28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbabd0,c6b6bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b6bd60, ebp = 0 --- Tracing command intr pid 12 tid 100012 td 0xc6dbf5c0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100011 td 0xc6dbf8a0 sched_switch(c6dbf8a0,0,109,191,f1271e9,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c6dbf8a0,...) at mi_switch+0x1eb ithread_loop(c6dbabf0,c6b53d28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbabf0,c6b53d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b53d60, ebp = 0 --- Tracing command intr pid 12 tid 100010 td 0xc6dbfb80 sched_switch(c6dbfb80,0,109,191,5b9123f2,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c6dbfb80,...) at mi_switch+0x1eb ithread_loop(c6dbac00,c6b50d28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbac00,c6b50d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b50d60, ebp = 0 --- Tracing command intr pid 12 tid 100009 td 0xc6e04000 sched_switch(c6e04000,0,109,191,31d3e8f7,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c6e04000,...) at mi_switch+0x1eb ithread_loop(c6dbac10,c6b4dd28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbac10,c6b4dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b4dd60, ebp = 0 --- Tracing command intr pid 12 tid 100008 td 0xc6e042e0 sched_switch(c6e042e0,0,109,191,39c4a482,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c6e042e0,...) at mi_switch+0x1eb ithread_loop(c6dbac20,c6b4ad28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbac20,c6b4ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b4ad60, ebp = 0 --- Tracing command intr pid 12 tid 100007 td 0xc6e045c0 sched_switch(c6e045c0,0,109,191,3842109d,...) at sched_switch+0x3bc mi_switch(109,0,c0e8c8bd,554,c6e045c0,...) at mi_switch+0x1eb ithread_loop(c6dbac30,c6b47d28,c0e8c598,390,c6dbc578,...) at ithread_loop+0x1fb fork_exit(c0984580,c6dbac30,c6b47d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b47d60, ebp = 0 --- Tracing command idle pid 11 tid 100006 td 0xc6dbe000 cpustop_handler(8,c6b43c04,c0cdb166,c0ce2c2c,c11f2480,...) at cpustop_handler+0x49 ipi_nmi_handler(c0ce2c2c,c11f2480,0,c11c0524,c6dbc834,...) at ipi_nmi_handler+0x44 trap(c6b43c10) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0cc0075, esp = 0xc6b43c50, ebp = 0xc6b43c50 --- acpi_cpu_c1(c6b43c6c,c0fe3bd0,3,300,b89,...) at acpi_cpu_c1+0x5 acpi_cpu_idle(0,c6b43ca0,c0cce299,0,ffffffff,...) at acpi_cpu_idle+0x122 cpu_idle_acpi(0,ffffffff,c102ca80,2,c6b43ce8,...) at cpu_idle_acpi+0x2f cpu_idle(0,c6b43cc4,c0e93c4b,a05,c6dbe000,...) at cpu_idle+0x89 sched_idletd(0,c6b43d28,c0e8c598,390,c6dbc834,...) at sched_idletd+0x263 fork_exit(c09d3440,0,c6b43d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b43d60, ebp = 0 --- Tracing command idle pid 11 tid 100005 td 0xc6dbe2e0 sched_switch(c6dbe2e0,0,108,18c,5be30214,...) at sched_switch+0x3bc mi_switch(108,0,c0e93c4b,3b9,c6dbe2e0,...) at mi_switch+0x1eb sched_idletd(0,c6b40d28,c0e8c598,390,c6dbc834,...) at sched_idletd+0x19f fork_exit(c09d3440,0,c6b40d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b40d60, ebp = 0 --- Tracing command idle pid 11 tid 100004 td 0xc6dbe5c0 cpustop_handler(2,c6b3dc04,c0cdb166,f4,0,...) at cpustop_handler+0x49 ipi_nmi_handler(f4,0,1101774,c0cc0075,c6dbc834,...) at ipi_nmi_handler+0x44 trap(c6b3dc10) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0cc0075, esp = 0xc6b3dc50, ebp = 0xc6b3dc50 --- acpi_cpu_c1(c6b3dc6c,c0fe3bd0,1,300,b89,...) at acpi_cpu_c1+0x5 acpi_cpu_idle(0,c6b3dca0,c0cce299,0,ffffffff,...) at acpi_cpu_idle+0x122 cpu_idle_acpi(0,ffffffff,c102bd80,2,c6b3dce8,...) at cpu_idle_acpi+0x2f cpu_idle(0,c6b3dcc4,c0e93c4b,a05,c6dbe5c0,...) at cpu_idle+0x89 sched_idletd(0,c6b3dd28,c0e8c598,390,c6dbc834,...) at sched_idletd+0x263 fork_exit(c09d3440,0,c6b3dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b3dd60, ebp = 0 --- Tracing command idle pid 11 tid 100003 td 0xc6dbe8a0 kdb_enter(c0d188be,c0e5ba70,0,c716a080,0,...) at kdb_enter+0x3a uart_intr(c716a000,0,c11bfba0,c6b3abe8,0,...) at uart_intr+0x126 intr_event_handle(c6d88800,c6b3ac10,c6b3ac34,0,c6fe9200,...) at intr_event_handle+0x76 intr_execute_handlers(c6e020d0,c6b3ac10,0,c6b3ac50,c0cc4a75,...) at intr_execute_handlers+0x49 lapic_handle_intr(3b,c6b3ac10) at lapic_handle_intr+0x36 Xapic_isr1() at Xapic_isr1+0x35 --- interrupt, eip = 0xc0cc0075, esp = 0xc6b3ac50, ebp = 0xc6b3ac50 --- acpi_cpu_c1(c6b3ac6c,c0fe3bd0,0,300,b89,...) at acpi_cpu_c1+0x5 acpi_cpu_idle(0,c6b3aca0,c0cce299,0,ffffffff,...) at acpi_cpu_idle+0x122 cpu_idle_acpi(0,ffffffff,c102b700,2,c6b3ace8,...) at cpu_idle_acpi+0x2f cpu_idle(0,c6b3acc4,c0e93c4b,a05,c6dbe8a0,...) at cpu_idle+0x89 sched_idletd(0,c6b3ad28,c0e8c598,390,c6dbc834,...) at sched_idletd+0x263 fork_exit(c09d3440,0,c6b3ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b3ad60, ebp = 0 --- Tracing command init pid 1 tid 100002 td 0xc6dbeb80 sched_switch(c6dbeb80,0,104,191,94e054fe,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c6dbeb80,0,c0e97420,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0e97420,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6dbcaf0,6c,c0e9a1fe,100,0,...) at sleepq_wait_sig+0x17 _sleep(c6dbcaf0,c6dbcb78,16c,c0e9a1fe,0,...) at _sleep+0x35c kern_wait(c6dbeb80,ffffffff,c6b36c44,2,0,...) at kern_wait+0xbe6 wait4(c6dbeb80,c6b36cec,bfbfe848,1,0,...) at wait4+0x3b syscallenter(c6dbeb80,c6b36ce4,c0cdb8ad,c101d170,0,...) at syscallenter+0x263 syscall(c6b36d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x8060df7, esp = 0xbfbfe81c, ebp = 0xbfbfe838 --- Tracing command audit pid 10 tid 100001 td 0xc6dbf000 sched_switch(c6dbf000,0,104,191,f87768f1,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c6dbf000,0,c0e97420,268,c6dbf000,...) at sleepq_switch+0x162 sleepq_wait(c119aa00,0,c6b33c8c,1,0,...) at sleepq_wait+0x63 _cv_wait(c119aa00,c119a9e4,c0ec1d17,194,0,...) at _cv_wait+0x243 audit_worker(0,c6b33d28,c0e8c598,390,c6dbd000,...) at audit_worker+0x84 fork_exit(c0ba1200,0,c6b33d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b33d60, ebp = 0 --- Tracing command kernel pid 0 tid 100076 td 0xc70458a0 sched_switch(c70458a0,0,104,191,17256c97,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c70458a0,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c73729c0,0,c0e84d12,0,0,...) at sleepq_wait+0x63 _sleep(c73729c0,c73729d8,0,c0e84d12,0,...) at _sleep+0x372 taskqueue_thread_loop(c11f2520,ecfa9d28,c0e8c598,390,c101b580,...) at taskqueue_thread_loop+0xb5 fork_exit(c09f00b0,c11f2520,ecfa9d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecfa9d60, ebp = 0 --- Tracing command kernel pid 0 tid 100066 td 0xc71768a0 sched_switch(c71768a0,0,104,191,3038f59b,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c71768a0,0,c0e97420,28b,2,...) at sleepq_switch+0x162 sleepq_timedwait(c101d884,0,c0e84d12,2,0,...) at sleepq_timedwait+0x6b _sleep(c101d884,0,0,c0e84d12,bb8,...) at _sleep+0x342 pause(c0e84d12,bb8,124,122,c101b608,...) at pause+0x47 deadlkres(0,ecf8bd28,c0e8c598,390,c101b580,...) at deadlkres+0x324 fork_exit(c0964c50,0,ecf8bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf8bd60, ebp = 0 --- Tracing command kernel pid 0 tid 100060 td 0xc7037b80 sched_switch(c7037b80,0,104,191,31303467,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7037b80,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c7034480,0,c0e929a1,c0e84d12,0,...) at sleepq_wait+0x63 msleep_spin(c7034480,c7034498,c0e84d12,0,c0e90227,...) at msleep_spin+0x21d taskqueue_thread_loop(c706f49c,ecf6fd28,c0e8c598,390,c101b580,...) at taskqueue_thread_loop+0x8f fork_exit(c09f00b0,c706f49c,ecf6fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xecf6fd60, ebp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xc6fb32e0 sched_switch(c6fb32e0,0,104,191,12ec877e,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c6fb32e0,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c6f50540,0,c0e929a1,c0e84d12,0,...) at sleepq_wait+0x63 msleep_spin(c6f50540,c6f50558,c0e84d12,0,c0e90227,...) at msleep_spin+0x21d taskqueue_thread_loop(c0fe6838,c6b7dd28,c0e8c598,390,c101b580,...) at taskqueue_thread_loop+0x8f fork_exit(c09f00b0,c0fe6838,c6b7dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b7dd60, ebp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xc6fb35c0 sched_switch(c6fb35c0,0,104,191,12ec797a,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c6fb35c0,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c6f50540,0,c0e929a1,c0e84d12,0,...) at sleepq_wait+0x63 msleep_spin(c6f50540,c6f50558,c0e84d12,0,c0e90227,...) at msleep_spin+0x21d taskqueue_thread_loop(c0fe6838,c6b7ad28,c0e8c598,390,c101b580,...) at taskqueue_thread_loop+0x8f fork_exit(c09f00b0,c0fe6838,c6b7ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b7ad60, ebp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xc6fb38a0 sched_switch(c6fb38a0,0,104,191,12ec6789,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c6fb38a0,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c6f50540,0,c0e929a1,c0e84d12,0,...) at sleepq_wait+0x63 msleep_spin(c6f50540,c6f50558,c0e84d12,0,c0e90227,...) at msleep_spin+0x21d taskqueue_thread_loop(c0fe6838,c6b77d28,c0e8c598,390,c101b580,...) at taskqueue_thread_loop+0x8f fork_exit(c09f00b0,c0fe6838,c6b77d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b77d60, ebp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xc6fb3b80 sched_switch(c6fb3b80,0,104,191,12e60870,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c6fb3b80,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c6f50580,0,c0e84d12,0,0,...) at sleepq_wait+0x63 _sleep(c6f50580,c6f50598,0,c0e84d12,0,...) at _sleep+0x372 taskqueue_thread_loop(c101bf18,c6b74d28,c0e8c598,390,c101b580,...) at taskqueue_thread_loop+0xb5 fork_exit(c09f00b0,c101bf18,c6b74d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b74d60, ebp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xc6e04b80 sched_switch(c6e04b80,0,104,191,12e5f75b,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c6e04b80,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c6f50600,0,c0e84d12,0,0,...) at sleepq_wait+0x63 _sleep(c6f50600,c6f50618,0,c0e84d12,0,...) at _sleep+0x372 taskqueue_thread_loop(c119afe4,c6b6ed28,c0e8c598,390,c101b580,...) at taskqueue_thread_loop+0xb5 fork_exit(c09f00b0,c119afe4,c6b6ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b6ed60, ebp = 0 --- Tracing command kernel pid 0 tid 100018 td 0xc6ecc2e0 sched_switch(c6ecc2e0,0,104,191,15b106b6,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c6ecc2e0,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c6f50680,0,c0e84d12,0,0,...) at sleepq_wait+0x63 _sleep(c6f50680,c6f50698,0,c0e84d12,0,...) at _sleep+0x372 taskqueue_thread_loop(c103a488,c6b68d28,c0e8c598,390,c101b580,...) at taskqueue_thread_loop+0xb5 fork_exit(c09f00b0,c103a488,c6b68d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b68d60, ebp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xc6ecc8a0 sched_switch(c6ecc8a0,0,104,191,3b5e9b81,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c6ecc8a0,0,c0e97420,268,0,...) at sleepq_switch+0x162 sleepq_wait(c6d76b80,0,c0e84d12,0,0,...) at sleepq_wait+0x63 _sleep(c6d76b80,c6d76b98,0,c0e84d12,0,...) at _sleep+0x372 taskqueue_thread_loop(c1038ea0,c6b62d28,c0e8c598,390,c101b580,...) at taskqueue_thread_loop+0xb5 fork_exit(c09f00b0,c1038ea0,c6b62d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6b62d60, ebp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xc101b840 sched_switch(c101b840,0,104,191,420c7df9,...) at sched_switch+0x3bc mi_switch(104,0,c0e97420,1f3,54,...) at mi_switch+0x1eb sleepq_switch(c101b840,0,c0e97420,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c101b580,54,c0e9455d,0,0,...) at sleepq_timedwait+0x6b _sleep(c101b580,0,54,c0e9455d,2710,...) at _sleep+0x342 scheduler(0,141ec00,141ec00,141e000,1425000,...) at scheduler+0x240 mi_startup() at mi_startup+0xac begin() at begin+0x2c db:0:allt> call doadump Physical memory: 3439 MB Physical memory: 3439 MB Dumping 278 MB:Dumping 278 MB: 263 263 247 247 231 231 215 215 199 199 183 183 167 167 151 151 135 135 119 119 103 103 87 87 71 71 55 55 39 39 23 23 7 7 Dump complete Dump complete = 0 db:0:doadump> reset (kgdb) proc 2480 [Switching to thread 79 (Thread 100103)]#0 cpustop_handler () at ../../../i386/i386/mp_machdep.c:1516 1516 CPU_OR_ATOMIC(&stopped_cpus, &cpumask); (kgdb) bt #0 cpustop_handler () at ../../../i386/i386/mp_machdep.c:1516 #1 0xc0cd10d4 in ipi_nmi_handler () at ../../../i386/i386/mp_machdep.c:1494 #2 0xc0cdb166 in trap (frame=0xef20c348) at ../../../i386/i386/trap.c:219 #3 0xc0cc46ac in calltrap () at ../../../i386/i386/exception.s:168 #4 0xc09f59a0 in find_instance (list=Variable "list" is not available. ) at ../../../kern/subr_witness.c:2060 #5 0xc09f7b4c in witness_lock (lock=0xc736c048, flags=0x8, file=0xc0ec4aa9 "ufs/ffs/ffs_softdep.c", line=0x30cf) at ../../../kern/subr_witness.c:1368 #6 0xc099dae3 in _mtx_lock_flags (m=0xc736c048, opts=0x0, file=0xc0ec4aa0 "../../../ufs/ffs/ffs_softdep.c", line=0x30cf) at ../../../kern/kern_mutex.c:206 #7 0xc0bdeb6b in softdep_request_cleanup (fs=0xc7efa800, vp=0xc92ab2b8, cred=0xc6da0000, resource=0x4) at ../../../ufs/ffs/ffs_softdep.c:12495 #8 0xc0bbe0c0 in ffs_balloc_ufs2 (vp=0xc92ab2b8, startoffset=Variable "startoffset" is not available. ) at ../../../ufs/ffs/ffs_balloc.c:896 #9 0xc0bc6322 in ffs_copyonwrite (devvp=0xc82feae0, bp=0xe09a02a0) at ../../../ufs/ffs/ffs_snapshot.c:2332 #10 0xc0be2a2b in ffs_geom_strategy (bo=0xc82febd8, bp=0xe09a02a0) at ../../../ufs/ffs/ffs_vfsops.c:2006 #11 0xc0a2ebd9 in bufwrite (bp=0xe09a02a0) at buf.h:398 #12 0xc0be20c0 in ffs_bufwrite (bp=0xe09a02a0) at ../../../ufs/ffs/ffs_vfsops.c:1967 #13 0xc0bc09bc in ffs_update (vp=0xc9bce2b8, waitfor=0x1) at buf.h:386 #14 0xc0bde86f in softdep_request_cleanup (fs=0xc7efa800, vp=0xc9bce2b8, cred=0xc701c880, resource=0x4) at ../../../ufs/ffs/ffs_softdep.c:12432 #15 0xc0bba437 in ffs_realloccg (ip=0xc9b7a828, lbprev=0x0, bprev=0x60722, bpref=0x5be28, osize=0x1000, nsize=0x2000, flags=0x3010000, cred=0xc701c880, bpp=0xef20ca2c) at ../../../ufs/ffs/ffs_alloc.c:429 #16 0xc0bbd485 in ffs_balloc_ufs2 (vp=0xc9bce2b8, startoffset=Variable "startoffset" is not available. ) at ../../../ufs/ffs/ffs_balloc.c:718 #17 0xc0be741d in ffs_write (ap=0xef20cb94) at ../../../ufs/ffs/ffs_vnops.c:716 #18 0xc0cfe596 in VOP_WRITE_APV (vop=0xc0fc7500, a=0xef20cb94) at vnode_if.c:951 #19 0xc0a56bda in vn_write (fp=0xc744cab8, uio=0xef20cc28, active_cred=0xc701c880, flags=0x0, td=0xc74388a0) at vnode_if.h:413 #20 0xc09fa445 in dofilewrite (td=0xc74388a0, fd=0x3, fp=0xc744cab8, auio=0xef20cc28, offset=0xffffffffffffffff, flags=0x0) at file.h:239 #21 0xc09fbbe8 in kern_writev (td=0xc74388a0, fd=0x3, auio=0xef20cc28) at ../../../kern/sys_generic.c:447 #22 0xc09fbcff in write (td=0xc74388a0, uap=0xef20ccec) at ../../../kern/sys_generic.c:363 #23 0xc09f0473 in syscallenter (td=0xc74388a0, sa=0xef20cce4) at ../../../kern/subr_trap.c:344 #24 0xc0cdb04f in syscall (frame=0xef20cd28) at ../../../i386/i386/trap.c:1082 #25 0xc0cc4711 in Xint0x80_syscall () at ../../../i386/i386/exception.s:266 #26 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) f 7 #7 0xc0bdeb6b in softdep_request_cleanup (fs=0xc7efa800, vp=0xc92ab2b8, cred=0xc6da0000, resource=0x4) at ../../../ufs/ffs/ffs_softdep.c:12495 12495 UFS_LOCK(ump); (kgdb) l 12490 process_truncates(vp); 12491 if (ump->softdep_on_worklist > 0 && 12492 process_worklist_item(UFSTOVFS(ump), 1, LK_NOWAIT) != 0) { 12493 stat_worklist_push += 1; 12494 FREE_LOCK(&lk); 12495 UFS_LOCK(ump); 12496 continue; 12497 } 12498 FREE_LOCK(&lk); 12499 UFS_LOCK(ump); (kgdb) info loc ump = (struct ufsmount *) 0xc736c000 mp = (struct mount *) 0xc73f12d4 lvp = (struct vnode *) 0xc92ab2b8 mvp = (struct vnode *) 0x0 starttime = 0x4df45c48 needed = 0x20 error = Variable "error" is not available. (kgdb) p *ump $1 = {um_mountp = 0xc73f12d4, um_dev = 0xc7dad400, um_cp = 0xc7453bc0, um_bo = 0xc82febd8, um_devvp = 0xc82feae0, um_fstype = 0x2, um_fs = 0xc7efa800, um_extattr = { uepm_lock = {lock_object = {lo_name = 0xc0ec8a3b "ufs_extattr_sx", lo_flags = 0x2330000, lo_data = 0x0, lo_witness = 0xc6d61f28}, sx_lock = 0x1}, uepm_list = { lh_first = 0x0}, uepm_ucred = 0x0, uepm_flags = 0x1}, um_nindir = 0x800, um_bptrtodb = 0x2, um_seqinc = 0x8, um_lock = {lock_object = {lo_name = 0xc0ec7dd6 "FFS", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc6d61ec0}, mtx_lock = 0xc74388a0}, um_numindirdeps = 0x0, softdep_workitem_pending = {lh_first = 0xecfa6c70}, softdep_worklist_tail = 0xca783000, softdep_journal_pending = {lh_first = 0x0}, softdep_journal_tail = 0xc736c068, softdep_jblocks = 0xc72f7980, softdep_unlinked = { tqh_first = 0x0, tqh_last = 0xc736c074}, softdep_dirtycg = {lh_first = 0xcad56600}, softdep_on_journal = 0x0, softdep_on_worklist = 0x1b1, softdep_deps = 0x6a04, softdep_accdeps = 0x12fe7e, softdep_req = 0x0, um_quotas = {0x0, 0x0}, um_cred = {0x0, 0x0}, um_btime = {0x0, 0x0}, um_itime = {0x0, 0x0}, um_qflags = "\000", um_savedmaxfilesize = 0x0, um_candelete = 0x0, um_balloc = 0xc0bbc500 , um_blkatoff = 0xc0be16d0 , um_truncate = 0xc0bc0a70 , um_update = 0xc0bc0730 , um_valloc = 0xc0bb74a0 , um_vfree = 0xc0bb6190 , um_ifree = 0xc0be28c0 , um_rdonly = 0xc0bc0040 , um_snapgone = 0xc0bc25e0 } (kgdb) p *vp $2 = {v_type = VREG, v_tag = 0xc0e88914 "ufs", v_op = 0xc0fc7500, v_data = 0xc9282ae0, v_mount = 0xc73f12d4, v_nmntvnodes = {tqe_next = 0xc92ab000, tqe_prev = 0xca313dac}, v_un = {vu_mount = 0x0, vu_socket = 0x0, vu_cdev = 0x0, vu_fifoinfo = 0x0}, v_hashlist = {le_next = 0x0, le_prev = 0xc71a4b68}, v_hash = 0x7, v_cache_src = { lh_first = 0x0}, v_cache_dst = {tqh_first = 0xc8fbdc60, tqh_last = 0xc8fbdc70}, v_cache_dd = 0x0, v_cstart = 0x0, v_lasta = 0x0, v_lastw = 0x0, v_clen = 0x0, v_lock = { lock_object = {lo_name = 0xc0e88914 "ufs", lo_flags = 0x57b0000, lo_data = 0x0, lo_witness = 0xc6d61f90}, lk_lock = 0x1, lk_exslpfail = 0x0, lk_timo = 0x33, lk_pri = 0x60, lk_stack = {depth = 0xb, pcs = {0xc099587e, 0xc0be7e51, 0xc0cfda65, 0xc0a557f8, 0xc0bc7d59, 0xc0be5220, 0xc0a3eff7, 0xc0a3f664, 0xc09f0473, 0xc0cdb04f, 0xc0cc4711, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}}, v_interlock = {lock_object = {lo_name = 0xc0e99759 "vnode interlock", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc6d5b1e0}, mtx_lock = 0x4}, v_vnlock = 0xc78e6d1c, v_holdcnt = 0x185, v_usecount = 0x1, v_iflag = 0x0, v_vflag = 0x80, v_writecount = 0x0, v_freelist = { tqe_next = 0x0, tqe_prev = 0x0}, v_bufobj = {bo_mtx = {lock_object = {lo_name = 0xc0ea20b9 "bufobj interlock", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc6d60798}, mtx_lock = 0x4}, bo_clean = {bv_hd = {tqh_first = 0xe078c960, tqh_last = 0xe07845f8}, bv_root = 0xe08d1ca0, bv_cnt = 0x174}, bo_dirty = { bv_hd = {tqh_first = 0xe0885ec0, tqh_last = 0xe0999118}, bv_root = 0xe094a3e0, bv_cnt = 0xf}, bo_numoutput = 0x0, bo_flag = 0x1, bo_ops = 0xc0fb4680, bo_bsize = 0x4000, bo_object = 0xc8385c38, bo_synclist = {le_next = 0x0, le_prev = 0xca313ed8}, bo_private = 0xc92ab2b8, __bo_vnode = 0xc92ab2b8}, v_pollinfo = 0x0, v_label = 0x0, v_lockf = 0x0} (kgdb) p stat_worklist_push $3 = 0xa3b650af (kgdb) p/d stat_worklist_push $4 = -1548332881 (kgdb) $ svn diff -x -p /usr/src/sys Index: /usr/src/sys/ufs/ufs/ufs_vnops.c =================================================================== --- /usr/src/sys/ufs/ufs/ufs_vnops.c (revision 222975) +++ /usr/src/sys/ufs/ufs/ufs_vnops.c (working copy) @@ -1838,6 +1838,8 @@ ufs_mkdir(ap) #ifdef QUOTA if ((error = getinoquota(ip)) || (error = chkiq(ip, 1, ucp, 0))) { + if (DOINGSOFTDEP(tvp)) + softdep_revert_link(dp, ip); UFS_VFREE(tvp, ip->i_number, dmode); vput(tvp); return (error); @@ -1850,6 +1852,8 @@ ufs_mkdir(ap) #ifdef QUOTA if ((error = getinoquota(ip)) || (error = chkiq(ip, 1, cnp->cn_cred, 0))) { + if (DOINGSOFTDEP(tvp)) + softdep_revert_link(dp, ip); UFS_VFREE(tvp, ip->i_number, dmode); vput(tvp); return (error); @@ -2608,6 +2612,8 @@ ufs_makeinode(mode, dvp, vpp, cnp) #ifdef QUOTA if ((error = getinoquota(ip)) || (error = chkiq(ip, 1, ucp, 0))) { + if (DOINGSOFTDEP(tvp)) + softdep_revert_link(pdir, ip); UFS_VFREE(tvp, ip->i_number, mode); vput(tvp); return (error); @@ -2620,6 +2626,8 @@ ufs_makeinode(mode, dvp, vpp, cnp) #ifdef QUOTA if ((error = getinoquota(ip)) || (error = chkiq(ip, 1, cnp->cn_cred, 0))) { + if (DOINGSOFTDEP(tvp)) + softdep_revert_link(pdir, ip); UFS_VFREE(tvp, ip->i_number, mode); vput(tvp); return (error); Index: /usr/src/sys/ufs/ffs/ffs_softdep.c =================================================================== --- /usr/src/sys/ufs/ffs/ffs_softdep.c (revision 222975) +++ /usr/src/sys/ufs/ffs/ffs_softdep.c (working copy) @@ -584,6 +584,33 @@ softdep_get_depcounts(struct mount *mp, *softdepactiveaccp = 0; } +void +softdep_buf_append(bp, wkhd) + struct buf *bp; + struct workhead *wkhd; +{ + + panic("softdep_buf_appendwork called"); +} + +void +softdep_inode_append(ip, cred, wkhd) + struct inode *ip; + struct ucred *cred; + struct workhead *wkhd; +{ + + panic("softdep_inode_appendwork called"); +} + +void +softdep_freework(wkhd) + struct workhead *wkhd; +{ + + panic("softdep_freework called"); +} + #else FEATURE(softupdates, "FFS soft-updates support"); @@ -755,6 +782,7 @@ static int flush_newblk_dep(struct vnode *, struct static int flush_inodedep_deps(struct vnode *, struct mount *, ino_t); static int flush_deplist(struct allocdirectlst *, int, int *); static int sync_cgs(struct mount *, int); +static void sync_snap(struct mount *, int); static int handle_written_filepage(struct pagedep *, struct buf *); static int handle_written_sbdep(struct sbdep *, struct buf *); static void initiate_write_sbdep(struct sbdep *); @@ -867,7 +895,7 @@ static void freework_enqueue(struct freework *); static int handle_workitem_freeblocks(struct freeblks *, int); static int handle_complete_freeblocks(struct freeblks *, int); static void handle_workitem_indirblk(struct freework *); -static void handle_written_freework(struct freework *, int); +static void handle_written_freework(struct freework *); static void merge_inode_lists(struct allocdirectlst *,struct allocdirectlst *); static struct worklist *jnewblk_merge(struct worklist *, struct worklist *, struct workhead *); @@ -1632,6 +1660,7 @@ process_truncates(vp) if (cgwait) { FREE_LOCK(&lk); sync_cgs(mp, MNT_WAIT); + sync_snap(mp, MNT_WAIT); ACQUIRE_LOCK(&lk); continue; } @@ -5915,7 +5944,7 @@ complete_trunc_indir(freework) */ if (bp == NULL) { if (LIST_EMPTY(&indirdep->ir_freeblks->fb_jblkdephd)) - handle_written_freework(freework, 0); + handle_written_freework(freework); else WORKLIST_INSERT(&indirdep->ir_freeblks->fb_freeworkhd, &freework->fw_list); @@ -5967,7 +5996,7 @@ out: */ if (totblks > datablocks) return (0); - return (totblks - datablocks); + return (datablocks - totblks); } /* @@ -7221,6 +7250,7 @@ freework_freeblock(freework) cancel_jnewblk(jnewblk, &wkhd); needj = 0; } else if (needj) { + freework->fw_state |= DELAYEDFREE; freeblks->fb_cgwait++; WORKLIST_INSERT(&wkhd, &freework->fw_list); } @@ -7234,7 +7264,7 @@ freework_freeblock(freework) * made it to disk. We can immediately free the freeblk. */ if (needj == 0) - handle_written_freework(freework, 0); + handle_written_freework(freework); } /* @@ -7249,7 +7279,8 @@ freework_enqueue(freework) struct freeblks *freeblks; freeblks = freework->fw_freeblks; - WORKLIST_INSERT(&freeblks->fb_freeworkhd, &freework->fw_list); + if ((freework->fw_state & INPROGRESS) == 0) + WORKLIST_INSERT(&freeblks->fb_freeworkhd, &freework->fw_list); if ((freeblks->fb_state & (ONWORKLIST | INPROGRESS | ALLCOMPLETE)) == ALLCOMPLETE && LIST_EMPTY(&freeblks->fb_jblkdephd)) @@ -7275,13 +7306,14 @@ handle_workitem_indirblk(freework) ump = VFSTOUFS(freeblks->fb_list.wk_mp); fs = ump->um_fs; if (freework->fw_state & DEPCOMPLETE) { - handle_written_freework(freework, 0); + handle_written_freework(freework); return; } if (freework->fw_off == NINDIR(fs)) { freework_freeblock(freework); return; } + freework->fw_state |= INPROGRESS; FREE_LOCK(&lk); indir_trunc(freework, fsbtodb(fs, freework->fw_blkno), freework->fw_lbn); @@ -7294,16 +7326,16 @@ handle_workitem_indirblk(freework) * the freeblks is added back to the worklist if there is more work to do. */ static void -handle_written_freework(freework, cgwrite) +handle_written_freework(freework) struct freework *freework; - int cgwrite; { struct freeblks *freeblks; struct freework *parent; freeblks = freework->fw_freeblks; parent = freework->fw_parent; - freeblks->fb_cgwait -= cgwrite; + if (freework->fw_state & DELAYEDFREE) + freeblks->fb_cgwait--; freework->fw_state |= COMPLETE; if ((freework->fw_state & ALLCOMPLETE) == ALLCOMPLETE) WORKITEM_FREE(freework, D_FREEWORK); @@ -7545,6 +7577,8 @@ indir_trunc(freework, dbn, lbn) return; } ACQUIRE_LOCK(&lk); + /* Protects against a race with complete_trunc_indir(). */ + freework->fw_state &= ~INPROGRESS; /* * If we have an indirdep we need to enforce the truncation order * and discard it when it is complete. @@ -7668,7 +7702,7 @@ indir_trunc(freework, dbn, lbn) if (freework->fw_blkno == dbn) { freework->fw_state |= ALLCOMPLETE; ACQUIRE_LOCK(&lk); - handle_written_freework(freework, 0); + handle_written_freework(freework); FREE_LOCK(&lk); } return; @@ -10361,8 +10395,7 @@ softdep_disk_write_complete(bp) continue; case D_FREEWORK: - /* Freework on an indirect block, not bmsafemap. */ - handle_written_freework(WK_FREEWORK(wk), 0); + handle_written_freework(WK_FREEWORK(wk)); break; case D_JSEGDEP: @@ -10533,7 +10566,7 @@ handle_jwork(wkhd) free_freedep(WK_FREEDEP(wk)); continue; case D_FREEWORK: - handle_written_freework(WK_FREEWORK(wk), 1); + handle_written_freework(WK_FREEWORK(wk)); continue; default: panic("handle_jwork: Unknown type %s\n", @@ -11766,6 +11799,42 @@ sync_cgs(mp, waitfor) } /* + * sync snapshots to force freework records waiting on snapshots to claim + * blocks to free. + */ +void +sync_snap(mp, waitfor) + struct mount *mp; + int waitfor; +{ + struct snapdata *sn; + struct vnode *devvp; + struct vnode *vp; + struct inode *ip; + + devvp = VFSTOUFS(mp)->um_devvp; + if ((devvp->v_vflag & VV_COPYONWRITE) == 0) + return; + for (;;) { + VI_LOCK(devvp); + sn = devvp->v_rdev->si_snapdata; + if (sn == NULL) { + VI_UNLOCK(devvp); + return; + } + if (lockmgr(&sn->sn_lock, + LK_INTERLOCK | LK_EXCLUSIVE | LK_SLEEPFAIL, + VI_MTX(devvp)) == 0) + break; + } + TAILQ_FOREACH(ip, &sn->sn_head, i_nextsnap) { + vp = ITOV(ip); + ffs_syncvnode(vp, waitfor); + } + lockmgr(&sn->sn_lock, LK_RELEASE, NULL); +} + +/* * This routine is called when we are trying to synchronously flush a * file. This routine must eliminate any filesystem metadata dependencies * so that the syncing routine can succeed. @@ -12731,6 +12800,53 @@ clear_inodedeps(td) } } +void +softdep_buf_append(bp, wkhd) + struct buf *bp; + struct workhead *wkhd; +{ + struct worklist *wk; + + ACQUIRE_LOCK(&lk); + while ((wk = LIST_FIRST(wkhd)) != NULL) { + WORKLIST_REMOVE(wk); + WORKLIST_INSERT(&bp->b_dep, wk); + } + FREE_LOCK(&lk); + +} + +void +softdep_inode_append(ip, cred, wkhd) + struct inode *ip; + struct ucred *cred; + struct workhead *wkhd; +{ + struct buf *bp; + struct fs *fs; + int error; + + fs = ip->i_fs; + error = bread(ip->i_devvp, fsbtodb(fs, ino_to_fsba(fs, ip->i_number)), + (int)fs->fs_bsize, cred, &bp); + if (error) { + softdep_freework(wkhd); + return; + } + softdep_buf_append(bp, wkhd); + bqrelse(bp); +} + +void +softdep_freework(wkhd) + struct workhead *wkhd; +{ + + ACQUIRE_LOCK(&lk); + handle_jwork(wkhd); + FREE_LOCK(&lk); +} + /* * Function to determine if the buffer has outstanding dependencies * that will cause a roll-back if the buffer is written. If wantcount Index: /usr/src/sys/ufs/ffs/ffs_alloc.c =================================================================== --- /usr/src/sys/ufs/ffs/ffs_alloc.c (revision 222975) +++ /usr/src/sys/ufs/ffs/ffs_alloc.c (working copy) @@ -2035,7 +2035,7 @@ ffs_blkfree(ump, fs, devvp, bno, size, inum, dephd */ if (devvp->v_type != VREG && (devvp->v_vflag & VV_COPYONWRITE) && - ffs_snapblkfree(fs, devvp, bno, size, inum)) { + ffs_snapblkfree(fs, devvp, bno, size, inum, dephd)) { return; } if (!ump->um_candelete) { Index: /usr/src/sys/ufs/ffs/ffs_extern.h =================================================================== --- /usr/src/sys/ufs/ffs/ffs_extern.h (revision 222975) +++ /usr/src/sys/ufs/ffs/ffs_extern.h (working copy) @@ -80,7 +80,8 @@ int ffs_realloccg(struct inode *, ufs2_daddr_t, uf ufs2_daddr_t, int, int, int, struct ucred *, struct buf **); int ffs_sbupdate(struct ufsmount *, int, int); void ffs_setblock(struct fs *, u_char *, ufs1_daddr_t); -int ffs_snapblkfree(struct fs *, struct vnode *, ufs2_daddr_t, long, ino_t); +int ffs_snapblkfree(struct fs *, struct vnode *, ufs2_daddr_t, long, ino_t, + struct workhead *); void ffs_snapremove(struct vnode *vp); int ffs_snapshot(struct mount *mp, char *snapfile); void ffs_snapshot_mount(struct mount *mp); @@ -149,6 +150,9 @@ int softdep_prealloc(struct vnode *, int); int softdep_journal_lookup(struct mount *, struct vnode **); void softdep_journal_freeblocks(struct inode *, struct ucred *, off_t, int); void softdep_journal_fsync(struct inode *); +void softdep_buf_append(struct buf *, struct workhead *); +void softdep_inode_append(struct inode *, struct ucred *, struct workhead *); +void softdep_freework(struct workhead *); /* @@ -161,4 +165,14 @@ void softdep_journal_fsync(struct inode *); int ffs_rdonly(struct inode *); +TAILQ_HEAD(snaphead, inode); + +struct snapdata { + LIST_ENTRY(snapdata) sn_link; + struct snaphead sn_head; + daddr_t sn_listsize; + daddr_t *sn_blklist; + struct lock sn_lock; +}; + #endif /* !_UFS_FFS_EXTERN_H */ Index: /usr/src/sys/ufs/ffs/ffs_snapshot.c =================================================================== --- /usr/src/sys/ufs/ffs/ffs_snapshot.c (revision 222975) +++ /usr/src/sys/ufs/ffs/ffs_snapshot.c (working copy) @@ -81,12 +81,13 @@ ffs_snapshot(mp, snapfile) } int -ffs_snapblkfree(fs, devvp, bno, size, inum) +ffs_snapblkfree(fs, devvp, bno, size, inum, wkhd) struct fs *fs; struct vnode *devvp; ufs2_daddr_t bno; long size; ino_t inum; + struct workhead *wkhd; { return (EINVAL); } @@ -126,16 +127,6 @@ ffs_copyonwrite(devvp, bp) #else FEATURE(ffs_snapshot, "FFS snapshot support"); -TAILQ_HEAD(snaphead, inode); - -struct snapdata { - LIST_ENTRY(snapdata) sn_link; - struct snaphead sn_head; - daddr_t sn_listsize; - daddr_t *sn_blklist; - struct lock sn_lock; -}; - LIST_HEAD(, snapdata) snapfree; static struct mtx snapfree_lock; MTX_SYSINIT(ffs_snapfree, &snapfree_lock, "snapdata free list", MTX_DEF); @@ -1635,7 +1626,7 @@ ffs_snapremove(vp) DIP_SET(ip, i_db[blkno], 0); else if ((dblk == blkstofrags(fs, blkno) && ffs_snapblkfree(fs, ip->i_devvp, dblk, fs->fs_bsize, - ip->i_number))) { + ip->i_number, NULL))) { DIP_SET(ip, i_blocks, DIP(ip, i_blocks) - btodb(fs->fs_bsize)); DIP_SET(ip, i_db[blkno], 0); @@ -1660,7 +1651,7 @@ ffs_snapremove(vp) ((ufs1_daddr_t *)(ibp->b_data))[loc]= 0; else if ((dblk == blkstofrags(fs, blkno) && ffs_snapblkfree(fs, ip->i_devvp, dblk, - fs->fs_bsize, ip->i_number))) { + fs->fs_bsize, ip->i_number, NULL))) { ip->i_din1->di_blocks -= btodb(fs->fs_bsize); ((ufs1_daddr_t *)(ibp->b_data))[loc]= 0; @@ -1674,7 +1665,7 @@ ffs_snapremove(vp) ((ufs2_daddr_t *)(ibp->b_data))[loc] = 0; else if ((dblk == blkstofrags(fs, blkno) && ffs_snapblkfree(fs, ip->i_devvp, dblk, - fs->fs_bsize, ip->i_number))) { + fs->fs_bsize, ip->i_number, NULL))) { ip->i_din2->di_blocks -= btodb(fs->fs_bsize); ((ufs2_daddr_t *)(ibp->b_data))[loc] = 0; } @@ -1722,12 +1713,13 @@ ffs_snapremove(vp) * must always have been allocated from a BLK_NOCOPY location. */ int -ffs_snapblkfree(fs, devvp, bno, size, inum) +ffs_snapblkfree(fs, devvp, bno, size, inum, wkhd) struct fs *fs; struct vnode *devvp; ufs2_daddr_t bno; long size; ino_t inum; + struct workhead *wkhd; { struct buf *ibp, *cbp, *savedcbp = 0; struct thread *td = curthread; @@ -1825,6 +1817,17 @@ retry: "Grabonremove: snapino", ip->i_number, (intmax_t)lbn, inum); #endif + /* + * If journaling is tracking this write we must add + * the work to the inode or indirect being written. + */ + if (wkhd != NULL) { + if (lbn < NDADDR) + softdep_inode_append(ip, + curthread->td_ucred, wkhd); + else + softdep_buf_append(ibp, wkhd); + } if (lbn < NDADDR) { DIP_SET(ip, i_db[lbn], bno); } else if (ip->i_ump->um_fstype == UFS1) { @@ -1902,6 +1905,8 @@ retry: * not be freed. Although space will be lost, the snapshot * will stay consistent. */ + if (error != 0 && wkhd != NULL) + softdep_freework(wkhd); lockmgr(vp->v_vnlock, LK_RELEASE, NULL); return (error); } -- Test scenario: suj20.sh