GENERIC HEAD from 2011-11-14 18:00:15 UTC, r227502M, vmcore.223 KDB: debugger backends: ddb KDB: current backend: ddb 524288K of memory above 4GB ignored Copyright (c) 1992-2011 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #0 r227502M: Mon Nov 14 19:19:08 CET 2011 pho@x4.osted.lan:/usr/src/sys/i386/compile/PHO i386 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. CPU: AMD Phenom(tm) 9150e Quad-Core Processor (1800.02-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x100f23 Family = 10 Model = 2 Stepping = 3 Features=0x178bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x7ff TSC: P-state invariant real memory = 4294967296 (4096 MB) avail memory = 3531710464 (3368 MB) : Trying to mount root from ufs:/dev/ad4s1a [rw]... WARNING: / was not properly dismounted Setting hostuuid: 00000000-0000-0000-0000-00218515337d. Setting hostid: 0x6b64ac17. Starting ddb. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: /dev/ad4s1a: 8839 files, 882877 used, 130138 free (978 frags, 16145 blocks, 0.1% fragmentation) /dev/ad4s1f: ZERO LENGTH DIR I=400 OWNER=root MODE=40755 /dev/ad4s1f: SIZE=0 MTIME=Nov 14 21:22 2011 (CLEARED) /dev/ad4s1f: LINK COUNT DIR I=189100 OWNER=pho MODE=40755 /dev/ad4s1f: SIZE=6656 MTIME=Nov 14 21:22 2011 COUNT 4 SHOULD BE 3 (ADJUSTED) /dev/ad4s1f: FREE BLK COUNT(S) WRONG IN SUPERBLK (SALVAGED) /dev/ad4s1f: SUMMARY INFORMATION BAD (SALVAGED) /dev/ad4s1f: BLK(S) MISSING IN BIT MAPS (SALVAGED) /dev/ad4s1f: 20541 files, 875000 used, 138015 free (4439 frags, 16697 blocks, 0.4% fragmentation) /dev/label/tmp: LINK COUNT DIR I=2 OWNER=root MODE=41777 /dev/label/tmp: SIZE=22528 MTIME=Nov 14 21:22 2011 COUNT 74 SHOULD BE 72 (ADJUSTED) /dev/label/tmp: UNREF DIR I=9467904 OWNER=root MODE=40777 /dev/label/tmp: SIZE=512 MTIME=Nov 14 21:22 2011 (CLEARED) /dev/label/tmp: ZERO LENGTH DIR I=28333056 OWNER=root MODE=40777 /dev/label/tmp: SIZE=0 MTIME=Nov 14 21:22 2011 (CLEARED) /dev/label/tmp: FREE BLK COUNT(S) WRONG IN SUPERBLK (SALVAGED) /dev/label/tmp: SUMMARY INFORMATION BAD (SALVAGED) /dev/label/tmp: BLK(S) MISSING IN BIT MAPS (SALVAGED) /dev/label/tmp: 25577 files, 3645057 used, 114668924 free (1212 frags, 14333464 blocks, 0.0% fragmentation) /dev/ad4s1d: 854383 files, 8756060 used, 1398139 free (287667 frags, 138809 blocks, 2.8% fragmentation) /dev/ad4s1e: 1953056 files, 18369403 used, 7019114 free (69450 frags, 868708 blocks, 0.3% fragmentation) Mounting local file systems:. Setting hostname: x4.osted.lan. re0: link state changed to DOWN Starting Network: lo0 re0 fwe0 fwip0. lo0: flags=8049 metric 0 mtu 16384 options=3 inet 127.0.0.1 netmask 0xff000000 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0xa nd6 options=21 re0: flags=8843 metric 0 mtu 1500 options=389b ether 00:21:85:15:33:7d inet 192.168.1.101 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::221:85ff:fe15:337d%re0 prefixlen 64 tentative scopeid 0x1 nd6 options=29 media: Ethernet autoselect (none) status: no carrier fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb nd6 options=29 ch 1 dma -1 fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 nd6 options=29 Starting devd. Starting Network: usbus0. Starting Network: usbus1. Starting Network: usbus2. Starting Network: usbus3. Starting Network: usbus4. Starting Network: usbus5. Starting Network: fwe0. fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb nd6 options=29 ch 1 dma -1 Starting Network: fwip0. fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 nd6 options=29 add net default: gateway 192.168.1.1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 re0: link state changed to UP ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/kde4/lib /usr/local/lib /usr/local/lib/compat /usr/local/lib/compat/pkg /usr/local/lib/nss /usr/local/lib/qt4 /usr/local/lib/virtualbox /usr/local/lib/wine a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout Creating and/or trimming log files. Starting syslogd. savecore: reboot after panic: null with unref'ed lowervp Nov 14 21:35:36 x4 savecore: reboot after panic: null with unref'ed lowervp lock order reversal: 1st 0xe0b1b5e0 bufwait (bufwait) @ kern/vfs_bio.c:2658 2nd 0xc7577200 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:284 KDB: stack backtrace: db_trace_self_wrapper(c0f234e8,2e687361,38323a63,a34,f1843754,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a5647b,c0f26eef,c11fed50,11c,f18437e4,...) at kdb_backtrace+0x2a _witness_debugger(c0f26eef,c7577200,c0f56d26,c71693a0,c0f569b8,...) at _witness_debugger+0x25 witness_checkorder(c7577200,9,c0f569af,11c,0,...) at witness_checkorder+0x86f _sx_xlock(c7577200,0,c0f569af,11c,c7e6cc3c,...) at _sx_xlock+0x85 ufsdirhash_acquire(e0b1b580,c7e6cc3c,f1843918,e1783700,f18438b4,...) at ufsdirhash_acquire+0x48 ufsdirhash_add(c7e6cc3c,f1843918,3700,f18438a0,f18438a4,...) at ufsdirhash_add+0x13 ufs_direnter(c7e65414,c7e81414,f1843918,f1843bc0,0,...) at ufs_direnter+0x789 ufs_makeinode(f1843bc0,c106aba0,f1843b04,f1843a60,c0d7aa65,...) at ufs_makeinode+0x635 ufs_create(f1843b04,c0f7680e,0,0,f1843b80,...) at ufs_create+0x30 VOP_CREATE_APV(c106a620,f1843b04,f1843bc0,f1843a9c,0,...) at VOP_CREATE_APV+0xc5 vn_open_cred(f1843b80,f1843c48,180,0,c71a4e80,...) at vn_open_cred+0x205 vn_open(f1843b80,f1843c48,180,c78b3af0,c19c1010,...) at vn_open+0x3b kern_openat(c77f8000,ffffff9c,28450000,0,602,...) at kern_openat+0x1ec kern_open(c77f8000,28450000,0,601,180,...) at kern_open+0x35 sys_open(c77f8000,f1843cec,c0f6f44a,f1843d80,282,...) at sys_open+0x30 syscall(f1843d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (5, FreeBSD ELF32, sys_open), eip = 0x2818b5c3, esp = 0xbfbfe2ac, ebp = 0xbfbfedb8 --- savecore: writing core to vmcore.222 Writing crash summary to /var/crash/core.txt.222. Additional ABI support: linux. Starting rpcbind. NFS access cache time=60 Clearing /tmp (X related). Starting mountd. NFSv4 is disabled Starting nfsd. Recovering vi editor sessions:. Updating motd:. Starting ntpd. Starting default moused. Configuring syscons: keymap blanktime. Starting sshd. Starting cron. Local package initialization: backuplock order reversal: 1st 0xc7bb9310 ufs (ufs) @ kern/vfs_subr.c:2167 2nd 0xe0be4f80 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:260 3rd 0xc80ba46c ufs (ufs) @ kern/vfs_subr.c:2167 KDB: stack backtrace: db_trace_self_wrapper(c0f234e8,3a632e73,a303632,0,c0f1da4b,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a5647b,c0f26f08,c11fe720,877,f182d8b0,...) at kdb_backtrace+0x2a _witness_debugger(c0f26f08,c80ba46c,c0f15c7c,c7169338,c0f2eca5,...) at _witness_debugger+0x25 witness_checkorder(c80ba46c,9,c0f2ec9c,877,0,...) at witness_checkorder+0x86f __lockmgr_args(c80ba46c,80100,c80ba4d8,0,0,...) at __lockmgr_args+0x834 ffs_lock(f182d9d4,df,c7bbba7c,80100,c80ba414,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c106a620,f182d9d4,c71621e0,c1079680,c80ba414,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c80ba414,80100,c0f2ec9c,877,4,...) at _vn_lock+0x78 vget(c80ba414,80100,c77f8b80,50,0,...) at vget+0xbb vfs_hash_get(c7bd85a8,6d4000,80000,c77f8b80,f182db18,...) at vfs_hash_get+0xed ffs_vgetf(c7bd85a8,6d4000,80000,f182db18,1,...) at ffs_vgetf+0x49 softdep_sync_buf(c7bb92b8,e0be4f20,1,106,0,...) at softdep_sync_buf+0x4a3 ffs_syncvnode(c7bb92b8,1,c77f8b80,5ce,0,...) at ffs_syncvnode+0x24c ffs_sync(c7bd85a8,1,f182dc30,4f2,c7025000,...) at ffs_sync+0x26f dounmount(c7bd85a8,8080000,c77f8b80,483,5b85c5ea,...) at dounmount+0x3f8 sys_unmount(c77f8b80,f182dcec,c,c,206,...) at sys_unmount+0x331 syscall(f182dd28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280debdb, esp = 0xbfbfe64c, ebp = 0xbfbfe718 --- lock order reversal: 1st 0xc7bb9b38 ufs (ufs) @ kern/vfs_mount.c:1209 2nd 0xc7bbb9dc devfs (devfs) @ ufs/ffs/ffs_softdep.c:1834 KDB: stack backtrace: db_trace_self_wrapper(c0f234e8,38313a63,a3433,c0a6878c,8,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a5647b,c0f26eef,c11fe700,72a,f182da10,...) at kdb_backtrace+0x2a _witness_debugger(c0f26eef,c7bbb9dc,c0f0dae7,c7169198,c0f52eb9,...) at _witness_debugger+0x25 witness_checkorder(c7bbb9dc,9,c0f52eb0,72a,c7bbba48,...) at witness_checkorder+0x86f __lockmgr_args(c7bbb9dc,80400,c7bbba48,0,0,...) at __lockmgr_args+0x834 vop_stdlock(f182db30,4,c0f1da4b,80400,c7bbb984,...) at vop_stdlock+0x65 VOP_LOCK1_APV(c1040680,f182db30,c1232700,c1079680,c7bbb984,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c7bbb984,80400,c0f52eb0,72a,c7bd85a8,...) at _vn_lock+0x78 softdep_flushworklist(c7bd85a8,f182dbdc,c77f8b80,5dc,0,...) at softdep_flushworklist+0x47 ffs_sync(c7bd85a8,1,f182dc30,4f2,c7025000,...) at ffs_sync+0x2fd dounmount(c7bd85a8,8080000,c77f8b80,483,5b85c5ea,...) at dounmount+0x3f8 sys_unmount(c77f8b80,f182dcec,c,c,206,...) at sys_unmount+0x331 syscall(f182dd28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280debdb, esp = 0xbfbfe64c, ebp = 0xbfbfe718 --- ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 25577 files, 3645057 used, 114668924 free (1212 frags, 14333464 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes Nov 14 21:39:04 x4 su: pho to root on /dev/pts/0 ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 25577 files, 3645057 used, 114668924 free (1212 frags, 14333464 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** fsck -y /tmp watchdogd. Starting inetd. Mon Nov 14 21:39:59 CET 2011 FreeBSD/i386 (x4.osted.lan) (console) login: lock order reversal: 1st 0xc7be4058 pseudofs (pseudofs) @ kern/vfs_syscalls.c:4172 2nd 0xc10b4790 allproc (allproc) @ fs/pseudofs/pseudofs_vnops.c:759 KDB: stack backtrace: db_trace_self_wrapper(c0f234e8,73666f64,6f6e765f,632e7370,3935373a,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a5647b,c0f26eef,c11fd680,2f7,f188db2c,...) at kdb_backtrace+0x2a _witness_debugger(c0f26eef,c10b4790,c0f17996,c71610d0,c0f1255b,...) at _witness_debugger+0x25 witness_checkorder(c10b4790,1,c0f12552,2f7,0,...) at witness_checkorder+0x86f _sx_slock(c10b4790,0,c0f12552,2f7,c7562180,...) at _sx_slock+0x85 pfs_readdir(f188dc0c,c0f7620a,0,0,c7be4000,...) at pfs_readdir+0x126 VOP_READDIR_APV(c10436a0,f188dc0c,c0f302a7,104c,f188dc34,...) at VOP_READDIR_APV+0xc5 kern_getdirentries(c9d93b80,4,28450000,1000,f188dc60,...) at kern_getdirentries+0x234 sys_getdirentries(c9d93b80,f188dcec,c0f6f44a,c0f27a57,246,...) at sys_getdirentries+0x31 syscall(f188dd28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (196, FreeBSD ELF32, sys_getdirentries), eip = 0x2811505f, esp = 0xbf9fee8c, ebp = 0xbf9feeb8 --- lock order reversal: 1st 0xc9d37a2c filedesc structure (filedesc structure) @ kern/sys_generic.c:1404 2nd 0xc7657df0 devfs (devfs) @ kern/vfs_vnops.c:924 KDB: stack backtrace: db_trace_self_wrapper(c0f234e8,2029000a,656b2040,732f6e72,675f7379,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a5647b,c0f26eef,c11fd520,39c,f1a6a97c,...) at kdb_backtrace+0x2a _witness_debugger(c0f26eef,c7657df0,c0f0dae7,c7169198,c0f30481,...) at _witness_debugger+0x25 witness_checkorder(c7657df0,9,c0f30478,39c,c7657e5c,...) at witness_checkorder+0x86f __lockmgr_args(c7657df0,80400,c7657e5c,0,0,...) at __lockmgr_args+0x834 vop_stdlock(f1a6aa9c,c0aa6df5,c9ad55c8,80400,c7657d98,...) at vop_stdlock+0x65 VOP_LOCK1_APV(c1040580,f1a6aa9c,0,c1079680,c7657d98,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c7657d98,80400,c0f30478,39c,0,...) at _vn_lock+0x78 vn_poll(c9dc8498,100,c78d3400,c77f9000,c8056800,...) at vn_poll+0x77 sys_poll(c77f9000,f1a6acec,c0f6f44a,c0f27bf2,282,...) at sys_poll+0x20c syscall(f1a6ad28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x280feb81, esp = 0xbf6fbf60, ebp = 0xbf6fbfa8 --- tvotohz: negative time difference -1475869520 sec -999475 usec tvotohz: negative time difference -20447 sec -863392 usec tvotohz: negative time difference -2012740447 sec -999475 usec tvotohz: negative time difference -647 sec -930515 usec tvotohz: negative time difference -1073741824 sec 0 usec tvotohz: negative time difference -1557171007 sec -865483 usec tvotohz: negative time difference -763031412 sec -903078 usec Nov 15 08:06:21 x4 su: pho to root on /dev/pts/1 Stopping inetd. Waiting for PIDS: 2300. Shutting down local packages:. Stopping cron. Waiting for PIDS: 2171. Stopping sshd. Waiting for PIDS: 2149. Stopping moused. Waiting for PIDS: 2089. Stopping ntpd. Waiting for PIDS: 2011. Stopping nfsd. Waiting for PIDS: 1906 1913. Stopping mountd. Waiting for PIDS: 1896. Stopping rpcbind. Waiting for PIDS: 1797. Stopping devd. Waiting for PIDS: 1508. Writing entropy file:. Terminated . Nov 15 08:07:35 x4 syslogd: exiting on signal 15 Nov 15 08:07:55 init: some processes would not die; ps axl advised Enter full pathname of shell or RETURN for /bin/sh: # ps auxww USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND root 11 300.0 0.0 0 32 ?? RL 9:29PM 1983:50.79 [idle] nobody 34583 100.0 0.0 14428 1348 0- T 11:55PM 491:53.82 /tmp/syscall2 root 0 0.0 0.0 0 96 ?? DLs 9:29PM 0:04.62 [kernel] root 1 0.0 0.0 8032 604 ?? SLs 9:29PM 0:00.06 /sbin/init -- root 2 0.0 0.0 0 8 ?? IL 9:29PM 0:00.00 [fw0_probe] root 3 0.0 0.0 0 8 ?? DL 9:29PM 0:00.16 [fdc0] root 4 0.0 0.0 0 8 ?? DL 9:29PM 0:00.00 [sctp_iterator] root 5 0.0 0.0 0 8 ?? DL 9:29PM 0:00.00 [xpt_thrd] root 6 0.0 0.0 0 8 ?? DL 9:29PM 0:00.04 [pagedaemon] root 7 0.0 0.0 0 8 ?? DL 9:29PM 0:00.00 [vmdaemon] root 8 0.0 0.0 0 8 ?? DL 9:29PM 0:00.00 [pagezero] root 9 0.0 0.0 0 8 ?? DL 9:29PM 0:00.18 [bufdaemon] root 10 0.0 0.0 0 8 ?? DL 9:29PM 0:00.00 [audit] root 12 0.0 0.0 0 176 ?? WL 9:29PM 4:07.17 [intr] root 13 0.0 0.0 0 24 ?? DL 9:29PM 2:00.39 [geom] root 14 0.0 0.0 0 8 ?? DL 9:29PM 0:04.14 [yarrow] root 15 0.0 0.0 0 192 ?? DL 9:29PM 0:01.06 [usb] root 16 0.0 0.0 0 8 ?? DL 9:29PM 0:14.35 [vnlru] root 17 0.0 0.0 0 8 ?? DL 9:29PM 1:16.00 [syncer] root 18 0.0 0.0 0 8 ?? DL 9:29PM 0:26.60 [softdepflush] root 2692 0.0 0.0 0 8 ?? DL 11:43PM 0:00.03 [md5] root 2704 0.0 0.0 0 8 ?? DLs 11:43PM 0:00.01 [aiod1] root 2705 0.0 0.0 0 8 ?? DLs 11:43PM 0:00.00 [aiod2] root 2706 0.0 0.0 0 8 ?? DLs 11:43PM 0:00.00 [aiod3] root 2707 0.0 0.0 0 8 ?? DLs 11:43PM 0:00.01 [aiod4] root 18528 0.0 0.1 9936 1820 u0 Ss 8:07AM 0:00.01 -sh (sh) root 18529 0.0 0.0 9704 1248 u0 R+ 8:08AM 0:00.00 ps auxww # kill -9 34583 # kill -9 34583 # kill -9 34583 # ~KDB: enter: Break to debugger [ thread pid 11 tid 100003 ] Stopped at kdb_enter+0x3a: movl $0,kdb_why db> show mount 0xc789cb50 /dev/ad4s1a on / (ufs) 0xc789d000 devfs on /dev (devfs) 0xc789c87c /dev/ad4s1f on /home (ufs) 0xc7bd82d4 /dev/ad4s1d on /usr (ufs) 0xc7bd8000 /dev/ad4s1e on /var (ufs) 0xc789db50 procfs on /proc (procfs) 0xc789d87c linprocfs on /usr/compat/linux/proc (linprocfs) 0xc789c5a8 /dev/label/tmp on /tmp (ufs) 0xc789d5a8 /dev/md5a on /mnt (ufs) More info: show mount db> show mount 0xc789d5a8 0xc789d5a8 /dev/md5a on /mnt (ufs) mnt_flag = SOFTDEP, EXPORTED, LOCAL mnt_kern_flag = SOFTDEP, EXTENDED_SHARED, MPSAFE, LOOKUP_SHARED mnt_opt = fstype, fspath, from, errmsg mnt_stat = { version=537068824 type=53 flags=0x0000000000201100 bsize=4096 iosize=32768 blocks=516005 bfree=515972 bavail=474692 files=131070 ffree=130996 syncwrites=0 asyncwrites=0 syncreads=0 asyncreads=0 namemax=255 owner=0 fsid=[1321310619, 303689810] } mnt_cred = { uid=0 ruid=0 } mnt_ref = 2 mnt_gen = 1 mnt_nvnodelistsize = 2 mnt_writeopcount = 0 mnt_noasync = 1 mnt_maxsymlinklen = 120 mnt_iosize_max = 131072 mnt_hashseed = 24241451 mnt_secondary_writes = 0 mnt_secondary_accwrites = 109 mnt_gjprovider = NULL vnode 0xc9ad5570: tag ufs, type VDIR usecount 2, writecount 0, refcount 4 mountedhere 0 flags (VV_ROOT) v_object 0xc9afa220 ref 0 pages 1 lock type ufs: UNLOCKED #0 0xc0a04fa4 at __lockmgr_args+0x5c4 #1 0xc0c63451 at ffs_lock+0xa1 #2 0xc0d79a95 at VOP_LOCK1_APV+0xb5 #3 0xc0ac81a8 at _vn_lock+0x78 #4 0xc0aba55b at vget+0xbb #5 0xc0aaad0d at vfs_hash_get+0xed #6 0xc0c5c939 at ffs_vgetf+0x49 #7 0xc0c5cebe at ffs_vget+0x2e #8 0xc0c719f8 at ufs_root+0x28 #9 0xc0aac492 at lookup+0xa12 #10 0xc0aad06f at namei+0x61f #11 0xc0abfbee at kern_statfs+0xbe #12 0xc0abfe2b at sys_statfs+0x3b #13 0xc0d56514 at syscall+0x2a4 #14 0xc0d3f941 at Xint0x80_syscall+0x21 ino 2, on dev md5a vnode 0xcafe715c: tag syncer, type VNON usecount 1, writecount 0, refcount 1 mountedhere 0 flags () lock type syncer: UNLOCKED #0 0xc0a055fe at __lockmgr_args+0xc1e #1 0xc0aa6785 at vop_stdlock+0x65 #2 0xc0d79a95 at VOP_LOCK1_APV+0xb5 #3 0xc0ac81a8 at _vn_lock+0x78 #4 0xc0ab9c62 at sync_vnode+0x142 #5 0xc0ab9fe2 at sched_sync+0x282 #6 0xc09f0528 at fork_exit+0xb8 #7 0xc0d3f954 at fork_trampoline+0x8 db> run pho db:0:pho> bt Tracing pid 11 tid 100003 td 0xc71c88a0 kdb_enter(c0d94b3e,c0f23499,c6f3fba4,c088c62b,c755c000,...) at kdb_enter+0x3a kdb_break(c755c000,1f04160,c0d3b2a5,c755c080,c755c058,...) at kdb_break+0x28 uart_intr(c755c000,0,c1261860,c6f3fbe8,0,...) at uart_intr+0x11b intr_event_handle(c71c4d00,c6f3fc10,c6f3fc34,0,c73ee200,...) at intr_event_handle+0x76 intr_execute_handlers(c720b0d0,c6f3fc10,0,c6f3fc50,c0d3fca5,...) at intr_execute_handlers+0x49 lapic_handle_intr(3c,c6f3fc10) at lapic_handle_intr+0x36 Xapic_isr1() at Xapic_isr1+0x35 --- interrupt, eip = 0xc0d3b2a5, esp = 0xc6f3fc50, ebp = 0xc6f3fc50 --- acpi_cpu_c1(c6f3fc6c,c107add0,0,301,95d7,...) at acpi_cpu_c1+0x5 acpi_cpu_idle(0,c6f3fca0,c0d49549,0,ffffffff,...) at acpi_cpu_idle+0x122 cpu_idle_acpi(0,ffffffff,c10c2b00,2,c6f3fce8,...) at cpu_idle_acpi+0x2f cpu_idle(0,c6f3fcc4,c0f21562,3c0,c71c88a0,...) at cpu_idle+0x89 sched_idletd(0,c6f3fd28,c0f19db0,3db,c71c684c,...) at sched_idletd+0x263 fork_exit(c0a43630,0,c6f3fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f3fd60, ebp = 0 --- db:0:bt> show allpcpu Current CPU: 0 cpuid = 0 dynamic pcpu = 0x7ab200 curthread = 0xc71c88a0: pid 11 "idle: cpu0" curpcb = 0xc6f3fd80 fpcurthread = none idlethread = 0xc71c88a0: tid 100003 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: cpuid = 1 dynamic pcpu = 0x5eb5200 curthread = 0xc71c85c0: pid 11 "idle: cpu1" curpcb = 0xc6f42d80 fpcurthread = none idlethread = 0xc71c85c0: tid 100004 "idle: cpu1" APIC ID = 1 currentldt = 0x50 spin locks held: cpuid = 2 dynamic pcpu = 0x5eb8200 curthread = 0xc9d9f000: pid 34583 "syscall2" curpcb = 0xf1966d80 fpcurthread = none idlethread = 0xc71c82e0: tid 100005 "idle: cpu2" APIC ID = 2 currentldt = 0x50 spin locks held: cpuid = 3 dynamic pcpu = 0x5ebb200 curthread = 0xc71c8000: pid 11 "idle: cpu3" curpcb = 0xc6f48d80 fpcurthread = none idlethread = 0xc71c8000: tid 100006 "idle: cpu3" APIC ID = 3 currentldt = 0x50 spin locks held: db:0:allpcpu> show alllocks db:0:alllocks> show lockedvnods Locked vnodes db:0:lockedvnods> show mount 0xc789cb50 /dev/ad4s1a on / (ufs) 0xc789d000 devfs on /dev (devfs) 0xc789c87c /dev/ad4s1f on /home (ufs) 0xc7bd82d4 /dev/ad4s1d on /usr (ufs) 0xc7bd8000 /dev/ad4s1e on /var (ufs) 0xc789db50 procfs on /proc (procfs) 0xc789d87c linprocfs on /usr/compat/linux/proc (linprocfs) 0xc789c5a8 /dev/label/tmp on /tmp (ufs) 0xc789d5a8 /dev/md5a on /mnt (ufs) More info: show mount db:0:mount> ps pid ppid pgrp uid state wmesg wchan cmd 18528 1 18528 0 Ss+ ttyin 0xc72e8470 sh 34583 1 2566 65534 T+ (threaded) syscall2 780389 s syscall2 779872 s syscall2 779755 s syscall2 779361 s syscall2 778829 s syscall2 778533 Run CPU 2 syscall2 2707 1 2707 0 DLs aiordy 0xc93ba2e0 [aiod4] 2706 1 2706 0 DLs aiordy 0xc7d8d8a0 [aiod3] 2705 1 2705 0 DLs aiordy 0xc77f95c0 [aiod2] 2704 1 2704 0 DLs aiordy 0xc797c2e0 [aiod1] 2692 0 0 0 DL mdwait 0xc9d77800 [md5] 18 0 0 0 DL sdflush 0xc1232740 [softdepflush] 17 0 0 0 DL syncer 0xc122c074 [syncer] 16 0 0 0 DL vlruwt 0xc77ef84c [vnlru] 9 0 0 0 DL psleep 0xc122bd08 [bufdaemon] 8 0 0 0 DL pgzero 0xc12345dc [pagezero] 7 0 0 0 DL psleep 0xc12341c8 [vmdaemon] 6 0 0 0 DL psleep 0xc1234190 [pagedaemon] 5 0 0 0 DL ccb_scan 0xc107ab54 [xpt_thrd] 4 0 0 0 DL waiting_ 0xc122db78 [sctp_iterator] 3 0 0 0 DL - 0xc73eea3c [fdc0] 2 0 0 0 SL - 0xc747c000 [fw0_probe] 15 0 0 0 DL (threaded) [usb] 100058 D - 0xc745cd34 [usbus5] 100057 D - 0xc745cd04 [usbus5] 100056 D - 0xc745ccd4 [usbus5] 100055 D - 0xc745cca4 [usbus5] 100053 D - 0xc7451b5c [usbus4] 100052 D - 0xc7451b2c [usbus4] 100051 D - 0xc7451afc [usbus4] 100050 D - 0xc7451acc [usbus4] 100049 D - 0xc744db5c [usbus3] 100048 D - 0xc744db2c [usbus3] 100047 D - 0xc744dafc [usbus3] 100046 D - 0xc744dacc [usbus3] 100045 D - 0xc7443b5c [usbus2] 100044 D - 0xc7443b2c [usbus2] 100043 D - 0xc7443afc [usbus2] 100042 D - 0xc7443acc [usbus2] 100040 D - 0xc743bb5c [usbus1] 100039 D - 0xc743bb2c [usbus1] 100038 D - 0xc743bafc [usbus1] 100037 D - 0xc743bacc [usbus1] 100035 D - 0xc7428b5c [usbus0] 100034 D - 0xc7428b2c [usbus0] 100033 D - 0xc7428afc [usbus0] 100032 D - 0xc7428acc [usbus0] 14 0 0 0 DL - 0xc10b4ca4 [yarrow] 13 0 0 0 DL (threaded) [geom] 100015 D - 0xc10b28a8 [g_down] 100014 D - 0xc10b28a4 [g_up] 100013 D - 0xc10b289c [g_event] 12 0 0 0 WL (threaded) [intr] 100066 I [irq12: psm0] 100065 I [irq1: atkbd0] 100063 I [swi0: uart] 100060 I [irq20: fwohci0] 100059 I [irq14: ata0] 100054 I [irq19: ehci0] 100041 I [irq18: ohci2 ohci4] 100036 I [irq17: ohci1 ohci3] 100031 I [irq16: hdac1 ohci0] 100030 I [irq22: ahci0] 100029 I [irq257: re0] 100028 I [irq256: hdac0] 100027 I [swi6: Giant taskq] 100025 I [swi5: +] 100024 I [swi2: cambio] 100018 I [swi6: task queue] 100012 I [swi3: vm] 100011 I [swi4: clock] 100010 I [swi4: clock] 100009 I [swi4: clock] 100008 I [swi4: clock] 100007 I [swi1: netisr 0] 11 0 0 0 RL (threaded) [idle] 100006 Run CPU 3 [idle: cpu3] 100005 CanRun [idle: cpu2] 100004 Run CPU 1 [idle: cpu1] 100003 Run CPU 0 [idle: cpu0] 1 0 1 0 SLs wait 0xc71c6b10 [init] 10 0 0 0 DL audit_wo 0xc1231e80 [audit] 0 0 0 0 DLs (threaded) [kernel] 100184 D - 0xc78c7000 [aiod_bio taskq] 100077 D - 0xc77cfb00 [mca taskq] 100067 D - 0xc10b4ca4 [deadlkres] 100061 D - 0xc747b140 [fw0_taskq] 100026 D - 0xc7354400 [thread taskq] 100023 D - 0xc7354480 [kqueue taskq] 100022 D - 0xc73544c0 [acpi_task_2] 100021 D - 0xc73544c0 [acpi_task_1] 100020 D - 0xc73544c0 [acpi_task_0] 100019 D - 0xc7354500 [ffs_trim taskq] 100016 D - 0xc7195c40 [firmware taskq] 100000 D sched 0xc10b2980 [swapper] db:0:ps> allt Tracing command sh pid 18528 tid 100081 td 0xc78e25c0 sched_switch(c78e25c0,0,104,191,ddec44aa,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c78e25c0,0,c0f24e20,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a0d57a,c72e8404,0,c0f1da4b,c78e25c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c72e8470,0,f1801af0,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c72e8470,c72e8404,c0f294c8,527,0,...) at _cv_wait_sig+0x243 tty_wait(c72e8400,c72e8470,f1801c44,1,0,...) at tty_wait+0x71 ttydisc_read(c72e8400,f1801c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c71ad700,f1801c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c9df5310,f1801c44,c71a4e80,0,c78e25c0,...) at devfs_read_f+0x7e dofileread(f1801c44,ffffffff,ffffffff,0,f1801c20,...) at dofileread+0x9e kern_readv(c78e25c0,0,f1801c44,f1801c64,1,...) at kern_readv+0x68 sys_read(c78e25c0,f1801cec,c0f6f44a,c0f40be0,206,...) at sys_read+0x4f syscall(f1801d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281fd563, esp = 0xbfbfecac, ebp = 0xbfbfecd8 --- Tracing command syscall2 pid 34583 tid 780389 td 0xca0a78a0 sched_switch(ca0a78a0,0,20a,191,f4635f49,...) at sched_switch+0x3bc mi_switch(20a,0,c0f2080b,330,c8643b98,...) at mi_switch+0x1eb thread_suspend_check(0,0,c0f257be,f3,ca0a78a0,...) at thread_suspend_check+0x2b9 ast(f1a25d28) at ast+0x431 doreti_ast() at doreti_ast+0x17 Tracing command syscall2 pid 34583 tid 779872 td 0xc9d85b80 sched_switch(c9d85b80,0,20a,191,f4634fec,...) at sched_switch+0x3bc mi_switch(20a,0,c0f2080b,330,c8643b98,...) at mi_switch+0x1eb thread_suspend_check(0,0,c0f19b31,91,c11fd578,...) at thread_suspend_check+0x2b9 exit1(c9d85b80,b,c0f1f562,b40,0,...) at exit1+0xac sigexit(c9d85b80,b,c0f1f562,acf,c11fd578,...) at sigexit+0xe33 postsig(b,64,c0f257be,e8,0,...) at postsig+0x20b ast(f196fd28) at ast+0x398 doreti_ast() at doreti_ast+0x17 Tracing command syscall2 pid 34583 tid 779755 td 0xc9d93b80 sched_switch(c9d93b80,0,20a,191,f46329c1,...) at sched_switch+0x3bc mi_switch(20a,0,c0f2080b,330,c8643b98,...) at mi_switch+0x1eb thread_suspend_check(0,0,c0f257be,f3,c9d93b80,...) at thread_suspend_check+0x2b9 ast(f1931d28) at ast+0x431 doreti_ast() at doreti_ast+0x17 Tracing command syscall2 pid 34583 tid 779361 td 0xc9d86b80 sched_switch(c9d86b80,0,20a,191,f4638210,...) at sched_switch+0x3bc mi_switch(20a,0,c0f2080b,330,c8643b98,...) at mi_switch+0x1eb thread_suspend_check(0,0,c0f19b31,91,c11fd100,...) at thread_suspend_check+0x2b9 exit1(c9d86b80,b,c0f1f562,b40,0,...) at exit1+0xac sigexit(c9d86b80,b,c0f1f562,acf,c11fd100,...) at sigexit+0xe33 postsig(b,64,c0f257be,e8,0,...) at postsig+0x20b ast(f18d1d28) at ast+0x398 doreti_ast() at doreti_ast+0x17 Tracing command syscall2 pid 34583 tid 778829 td 0xc77f8000 sched_switch(c77f8000,0,10a,191,f462f5a5,...) at sched_switch+0x3bc mi_switch(10a,0,c0f2080b,350,c8643b98,...) at mi_switch+0x1eb thread_suspend_switch(c77f8000,0,c0f2080b,2a5,c8643b20,...) at thread_suspend_switch+0x2a4 thread_single(0,4,c0f1f562,b22,c0f593de,...) at thread_single+0x2b4 sigexit(c77f8000,b,c0f1f562,acf,c11fd470,...) at sigexit+0x77 postsig(b,64,c0f257be,e8,0,...) at postsig+0x20b ast(f183fd28) at ast+0x398 doreti_ast() at doreti_ast+0x17 Tracing command syscall2 pid 34583 tid 778533 td 0xc9d9f000 cpustop_handler(2,13,f1966b14,c0d56916,c104f798,...) at cpustop_handler+0x28 ipi_nmi_handler(c104f798,c9d9f000,c8643b98,367,c8643b10,...) at ipi_nmi_handler+0x47 trap(f1966b20) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0a68e80, esp = 0xf1966b60, ebp = 0xf1966bb0 --- witness_checkorder(c10b7070,9,c0f20e13,11f,0,...) at witness_checkorder+0x960 _mtx_lock_flags(c10b7070,0,c0f20e13,11f,0,...) at _mtx_lock_flags+0xc4 do_rw_wrlock(c9d9f000,f1966c30,ffffffff,c9d9f000,f1966c6c,...) at do_rw_wrlock+0x379 __umtx_op_rw_wrlock(c9d9f000,f1966cec,f1966d1c,c0d56514,c9d9f000,...) at __umtx_op_rw_wrlock+0x22 sys__umtx_op(c9d9f000,f1966cec,c0f6f44a,c0f288ce,286,...) at sys__umtx_op+0x27 syscall(f1966d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (454, FreeBSD ELF32, sys__umtx_op), eip = 0x2809e7e3, esp = 0xbfbfe64c, ebp = 0xbfbfe668 --- Tracing command aiod4 pid 2707 tid 100170 td 0xc93ba2e0 sched_switch(c93ba2e0,0,104,191,47564718,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c93ba2e0,0,c0f24e20,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c93ba2e0,5c,c9d5179e,0,0,...) at sleepq_timedwait+0x6b _sleep(c93ba2e0,c9d52964,5c,c9d5179e,7530,...) at _sleep+0x342 aio_daemon(4,f1996d28,c0f19db0,3db,c86442c4,...) at aio_daemon+0x5f3 fork_exit(c9d4e170,4,f1996d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xf1996d60, ebp = 0 --- Tracing command aiod3 pid 2706 tid 100155 td 0xc7d8d8a0 sched_switch(c7d8d8a0,0,104,191,53c85df3,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c7d8d8a0,0,c0f24e20,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c7d8d8a0,5c,c9d5179e,0,0,...) at sleepq_timedwait+0x6b _sleep(c7d8d8a0,c9d52964,5c,c9d5179e,7530,...) at _sleep+0x342 aio_daemon(3,f1969d28,c0f19db0,3db,c83cd588,...) at aio_daemon+0x5f3 fork_exit(c9d4e170,3,f1969d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xf1969d60, ebp = 0 --- Tracing command aiod2 pid 2705 tid 100082 td 0xc77f95c0 sched_switch(c77f95c0,0,104,191,7c56c56d,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c77f95c0,0,c0f24e20,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c77f95c0,5c,c9d5179e,0,0,...) at sleepq_timedwait+0x6b _sleep(c77f95c0,c9d52964,5c,c9d5179e,7530,...) at _sleep+0x342 aio_daemon(2,f1805d28,c0f19db0,3db,c7434b10,...) at aio_daemon+0x5f3 fork_exit(c9d4e170,2,f1805d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xf1805d60, ebp = 0 --- Tracing command aiod1 pid 2704 tid 100120 td 0xc797c2e0 sched_switch(c797c2e0,0,104,191,e28df2a6,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c797c2e0,0,c0f24e20,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c797c2e0,5c,c9d5179e,0,0,...) at sleepq_timedwait+0x6b _sleep(c797c2e0,c9d52964,5c,c9d5179e,7530,...) at _sleep+0x342 aio_daemon(1,f18c9d28,c0f19db0,3db,c79762c4,...) at aio_daemon+0x5f3 fork_exit(c9d4e170,1,f18c9d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xf18c9d60, ebp = 0 --- Tracing command md5 pid 2692 tid 100133 td 0xc7d8e8a0 sched_switch(c7d8e8a0,0,104,191,a2fb7004,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c7d8e8a0,0,c0f24e20,268,0,...) at sleepq_switch+0x162 sleepq_wait(c9d77800,5c,c0ebd8a2,0,0,...) at sleepq_wait+0x63 _sleep(c9d77800,c9d77820,25c,c0ebd8a2,0,...) at _sleep+0x372 md_kthread(c9d77800,f18f2d28,c0f19db0,3db,c7c9a84c,...) at md_kthread+0x11a fork_exit(c0736a40,c9d77800,f18f2d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xf18f2d60, ebp = 0 --- Tracing command softdepflush pid 18 tid 100076 td 0xc74578a0 sched_switch(c74578a0,0,104,191,37d8daaa,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,54,...) at mi_switch+0x1eb sleepq_switch(c74578a0,0,c0f24e20,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c1232740,54,c0f553f0,0,0,...) at sleepq_timedwait+0x6b _sleep(c1232740,c1232700,54,c0f553f0,3e8,...) at _sleep+0x342 softdep_flush(0,ed416d28,c0f19db0,3db,c77ef2c4,...) at softdep_flush+0x266 fork_exit(c0c58b40,0,ed416d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed416d60, ebp = 0 --- Tracing command syncer pid 17 tid 100075 td 0xc7457b80 sched_switch(c7457b80,0,104,191,37d8d21c,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7457b80,0,c0f24e20,28b,c7457b80,...) at sleepq_switch+0x162 sleepq_timedwait(c122c074,0,ed413c78,1,0,...) at sleepq_timedwait+0x6b _cv_timedwait(c122c074,c122c060,3e8,78b,4e20,...) at _cv_timedwait+0x252 sched_sync(0,ed413d28,c0f19db0,3db,c77ef588,...) at sched_sync+0x595 fork_exit(c0ab9d60,0,ed413d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed413d60, ebp = 0 --- Tracing command vnlru pid 16 tid 100074 td 0xc7573000 sched_switch(c7573000,0,104,191,5eaf5f04,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,60,...) at mi_switch+0x1eb sleepq_switch(c7573000,0,c0f24e20,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c77ef84c,60,c0f2fc04,0,0,...) at sleepq_timedwait+0x6b _sleep(c77ef84c,c122c034,260,c0f2fc04,3e8,...) at _sleep+0x342 vnlru_proc(0,ed410d28,c0f19db0,3db,c77ef84c,...) at vnlru_proc+0xe7 fork_exit(c0abbdc0,0,ed410d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed410d60, ebp = 0 --- Tracing command bufdaemon pid 9 tid 100073 td 0xc75732e0 sched_switch(c75732e0,0,104,191,7e1f1f07,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,54,...) at mi_switch+0x1eb sleepq_switch(c75732e0,0,c0f24e20,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c122bd08,54,c0f2cb6f,0,0,...) at sleepq_timedwait+0x6b _sleep(c122bd08,c122bd10,54,c0f2cb6f,3e8,...) at _sleep+0x342 buf_daemon(0,ed40dd28,c0f19db0,3db,c77efb10,...) at buf_daemon+0x175 fork_exit(c0aa0270,0,ed40dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed40dd60, ebp = 0 --- Tracing command pagezero pid 8 tid 100072 td 0xc75735c0 sched_switch(c75735c0,0,104,191,386cbe,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c75735c0,0,c0f24e20,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c12345dc,0,c0f5bf46,0,0,...) at sleepq_timedwait+0x6b _sleep(c12345dc,c1233080,0,c0f5bf46,493e0,...) at _sleep+0x342 vm_pagezero(0,ed40ad28,c0f19db0,3db,c77f0000,...) at vm_pagezero+0xdc fork_exit(c0c9ea00,0,ed40ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed40ad60, ebp = 0 --- Tracing command vmdaemon pid 7 tid 100071 td 0xc75738a0 sched_switch(c75738a0,0,104,191,69fb2d20,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,74,...) at mi_switch+0x1eb sleepq_switch(c75738a0,0,c0f24e20,268,0,...) at sleepq_switch+0x162 sleepq_wait(c12341c8,74,c0f2cb6f,0,0,...) at sleepq_wait+0x63 _sleep(c12341c8,c12341cc,74,c0f2cb6f,0,...) at _sleep+0x372 vm_daemon(0,ed407d28,c0f19db0,3db,c71c72c4,...) at vm_daemon+0x59 fork_exit(c0c98860,0,ed407d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed407d60, ebp = 0 --- Tracing command pagedaemon pid 6 tid 100070 td 0xc7573b80 sched_switch(c7573b80,0,104,191,31177072,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,54,...) at mi_switch+0x1eb sleepq_switch(c7573b80,0,c0f24e20,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c1234190,54,c0f2cb6f,0,0,...) at sleepq_timedwait+0x6b _sleep(c1234190,c1233080,54,c0f2cb6f,1388,...) at _sleep+0x342 vm_pageout(0,ed404d28,c0f19db0,3db,c71c7588,...) at vm_pageout+0x2c7 fork_exit(c0c99940,0,ed404d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed404d60, ebp = 0 --- Tracing command xpt_thrd pid 5 tid 100069 td 0xc7575000 sched_switch(c7575000,0,104,191,61df2ee5,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c7575000,0,c0f24e20,268,0,...) at sleepq_switch+0x162 sleepq_wait(c107ab54,5c,c0d7ea95,0,0,...) at sleepq_wait+0x63 _sleep(c107ab54,c107ab98,5c,c0d7ea95,0,...) at _sleep+0x372 xpt_scanner_thread(0,ed3a1d28,c0f19db0,3db,c71c784c,...) at xpt_scanner_thread+0x47 fork_exit(c048e480,0,ed3a1d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed3a1d60, ebp = 0 --- Tracing command sctp_iterator pid 4 tid 100068 td 0xc75752e0 sched_switch(c75752e0,0,104,191,d977833b,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c75752e0,0,c0f24e20,268,0,...) at sleepq_switch+0x162 sleepq_wait(c122db78,0,c0f431ff,0,0,...) at sleepq_wait+0x63 _sleep(c122db78,c122db40,0,c0f431ff,0,...) at _sleep+0x372 sctp_iterator_thread(0,ed39ed28,c0f19db0,3db,c71c7b10,...) at sctp_iterator_thread+0x5c fork_exit(c0b4d6f0,0,ed39ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed39ed60, ebp = 0 --- Tracing command fdc0 pid 3 tid 100064 td 0xc7576000 sched_switch(c7576000,0,104,191,6744c3ba,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c7576000,0,c0f24e20,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c73eea3c,5c,c0f12072,0,0,...) at sleepq_timedwait+0x6b _sleep(c73eea3c,c73eeaf0,5c,c0f12072,3e8,...) at _sleep+0x342 fdc_thread(c73eea00,ed392d28,c0f19db0,3db,c7434000,...) at fdc_thread+0x27d fork_exit(c0d1dcd0,c73eea00,ed392d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed392d60, ebp = 0 --- Tracing command fw0_probe pid 2 tid 100062 td 0xc744b5c0 sched_switch(c744b5c0,0,104,191,61df6708,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c744b5c0,0,c0f24e20,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f24e20,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c747c000,6c,c0f12072,100,0,...) at sleepq_wait_sig+0x17 _sleep(c747c000,c7480488,16c,c0f12072,0,...) at _sleep+0x35c fw_bus_probe_thread(c747c000,ed382d28,c0f19db0,3db,c74342c4,...) at fw_bus_probe_thread+0xa08 fork_exit(c06bdfa0,c747c000,ed382d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed382d60, ebp = 0 --- Tracing command usb pid 15 tid 100058 td 0xc74552e0 sched_switch(c74552e0,0,104,191,4f76c77f,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c74552e0,0,c0f24e20,268,c74552e0,...) at sleepq_switch+0x162 sleepq_wait(c745cd34,0,ed36bcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c745cd34,c745cdd4,c0ef86b1,74,c745cd3c,...) at _cv_wait+0x243 usb_process(c745cd2c,ed36bd28,c0f19db0,3db,c7434588,...) at usb_process+0x1a3 fork_exit(c08c0fe0,c745cd2c,ed36bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed36bd60, ebp = 0 --- Tracing command usb pid 15 tid 100057 td 0xc74555c0 sched_switch(c74555c0,0,104,191,a89dac59,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c74555c0,0,c0f24e20,268,c74555c0,...) at sleepq_switch+0x162 sleepq_wait(c745cd04,0,ed368cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c745cd04,c745cdd4,c0ef86b1,74,c745cd0c,...) at _cv_wait+0x243 usb_process(c745ccfc,ed368d28,c0f19db0,3db,c7434588,...) at usb_process+0x1a3 fork_exit(c08c0fe0,c745ccfc,ed368d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed368d60, ebp = 0 --- Tracing command usb pid 15 tid 100056 td 0xc74558a0 sched_switch(c74558a0,0,104,191,4f101a0e,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c74558a0,0,c0f24e20,268,c74558a0,...) at sleepq_switch+0x162 sleepq_wait(c745ccd4,0,ed365cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c745ccd4,c745cdd4,c0ef86b1,74,c745ccdc,...) at _cv_wait+0x243 usb_process(c745cccc,ed365d28,c0f19db0,3db,c7434588,...) at usb_process+0x1a3 fork_exit(c08c0fe0,c745cccc,ed365d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed365d60, ebp = 0 --- Tracing command usb pid 15 tid 100055 td 0xc7455b80 sched_switch(c7455b80,0,104,191,4f10093e,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7455b80,0,c0f24e20,268,c7455b80,...) at sleepq_switch+0x162 sleepq_wait(c745cca4,0,ed362cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c745cca4,c745cdd4,c0ef86b1,74,c745ccac,...) at _cv_wait+0x243 usb_process(c745cc9c,ed362d28,c0f19db0,3db,c7434588,...) at usb_process+0x1a3 fork_exit(c08c0fe0,c745cc9c,ed362d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed362d60, ebp = 0 --- Tracing command usb pid 15 tid 100053 td 0xc74572e0 sched_switch(c74572e0,0,104,191,4f0ff3e2,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c74572e0,0,c0f24e20,268,c74572e0,...) at sleepq_switch+0x162 sleepq_wait(c7451b5c,0,ed359cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7451b5c,c7451bfc,c0ef86b1,74,c7451b64,...) at _cv_wait+0x243 usb_process(c7451b54,ed359d28,c0f19db0,3db,c7434588,...) at usb_process+0x1a3 fork_exit(c08c0fe0,c7451b54,ed359d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed359d60, ebp = 0 --- Tracing command usb pid 15 tid 100052 td 0xc74575c0 sched_switch(c74575c0,0,104,191,a2ed7714,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c74575c0,0,c0f24e20,268,c74575c0,...) at sleepq_switch+0x162 sleepq_wait(c7451b2c,0,ed356cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7451b2c,c7451bfc,c0ef86b1,74,c7451b34,...) at _cv_wait+0x243 usb_process(c7451b24,ed356d28,c0f19db0,3db,c7434588,...) at usb_process+0x1a3 fork_exit(c08c0fe0,c7451b24,ed356d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed356d60, ebp = 0 --- Tracing command usb pid 15 tid 100051 td 0xc7449000 sched_switch(c7449000,0,104,191,4eabaa24,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7449000,0,c0f24e20,268,c7449000,...) at sleepq_switch+0x162 sleepq_wait(c7451afc,0,ed353cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7451afc,c7451bfc,c0ef86b1,74,c7451b04,...) at _cv_wait+0x243 usb_process(c7451af4,ed353d28,c0f19db0,3db,c7434588,...) at usb_process+0x1a3 fork_exit(c08c0fe0,c7451af4,ed353d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed353d60, ebp = 0 --- Tracing command usb pid 15 tid 100050 td 0xc74492e0 sched_switch(c74492e0,0,104,191,4eab996f,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c74492e0,0,c0f24e20,268,c74492e0,...) at sleepq_switch+0x162 sleepq_wait(c7451acc,0,ed350cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7451acc,c7451bfc,c0ef86b1,74,c7451ad4,...) at _cv_wait+0x243 usb_process(c7451ac4,ed350d28,c0f19db0,3db,c7434588,...) at usb_process+0x1a3 fork_exit(c08c0fe0,c7451ac4,ed350d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed350d60, ebp = 0 --- Tracing command usb pid 15 tid 100049 td 0xc74495c0 sched_switch(c74495c0,0,104,191,4eab831e,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c74495c0,0,c0f24e20,268,c74495c0,...) at sleepq_switch+0x162 sleepq_wait(c744db5c,0,ed34ccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c744db5c,c744dbfc,c0ef86b1,74,c744db64,...) at _cv_wait+0x243 usb_process(c744db54,ed34cd28,c0f19db0,3db,c7434588,...) at usb_process+0x1a3 fork_exit(c08c0fe0,c744db54,ed34cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed34cd60, ebp = 0 --- Tracing command usb pid 15 tid 100048 td 0xc74498a0 sched_switch(c74498a0,0,104,191,9d75c585,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c74498a0,0,c0f24e20,268,c74498a0,...) at sleepq_switch+0x162 sleepq_wait(c744db2c,0,ed349cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c744db2c,c744dbfc,c0ef86b1,74,c744db34,...) at _cv_wait+0x243 usb_process(c744db24,ed349d28,c0f19db0,3db,c7434588,...) at usb_process+0x1a3 fork_exit(c08c0fe0,c744db24,ed349d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed349d60, ebp = 0 --- Tracing command usb pid 15 tid 100047 td 0xc7449b80 sched_switch(c7449b80,0,104,191,48d0a9ad,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7449b80,0,c0f24e20,268,c7449b80,...) at sleepq_switch+0x162 sleepq_wait(c744dafc,0,ed346cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c744dafc,c744dbfc,c0ef86b1,74,c744db04,...) at _cv_wait+0x243 usb_process(c744daf4,ed346d28,c0f19db0,3db,c7434588,...) at usb_process+0x1a3 fork_exit(c08c0fe0,c744daf4,ed346d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed346d60, ebp = 0 --- Tracing command usb pid 15 tid 100046 td 0xc744a000 sched_switch(c744a000,0,104,191,48d098d7,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c744a000,0,c0f24e20,268,c744a000,...) at sleepq_switch+0x162 sleepq_wait(c744dacc,0,ed343cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c744dacc,c744dbfc,c0ef86b1,74,c744dad4,...) at _cv_wait+0x243 usb_process(c744dac4,ed343d28,c0f19db0,3db,c7434588,...) at usb_process+0x1a3 fork_exit(c08c0fe0,c744dac4,ed343d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed343d60, ebp = 0 --- Tracing command usb pid 15 tid 100045 td 0xc744a2e0 sched_switch(c744a2e0,0,104,191,48d0867a,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c744a2e0,0,c0f24e20,268,c744a2e0,...) at sleepq_switch+0x162 sleepq_wait(c7443b5c,0,ed33fcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7443b5c,c7443bfc,c0ef86b1,74,c7443b64,...) at _cv_wait+0x243 usb_process(c7443b54,ed33fd28,c0f19db0,3db,c7434588,...) at usb_process+0x1a3 fork_exit(c08c0fe0,c7443b54,ed33fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed33fd60, ebp = 0 --- Tracing command usb pid 15 tid 100044 td 0xc744a5c0 sched_switch(c744a5c0,0,104,191,9d7541da,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c744a5c0,0,c0f24e20,268,c744a5c0,...) at sleepq_switch+0x162 sleepq_wait(c7443b2c,0,ed33ccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7443b2c,c7443bfc,c0ef86b1,74,c7443b34,...) at _cv_wait+0x243 usb_process(c7443b24,ed33cd28,c0f19db0,3db,c7434588,...) at usb_process+0x1a3 fork_exit(c08c0fe0,c7443b24,ed33cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed33cd60, ebp = 0 --- Tracing command usb pid 15 tid 100043 td 0xc744a8a0 sched_switch(c744a8a0,0,104,191,486c5bdc,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c744a8a0,0,c0f24e20,268,c744a8a0,...) at sleepq_switch+0x162 sleepq_wait(c7443afc,0,ed339cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7443afc,c7443bfc,c0ef86b1,74,c7443b04,...) at _cv_wait+0x243 usb_process(c7443af4,ed339d28,c0f19db0,3db,c7434588,...) at usb_process+0x1a3 fork_exit(c08c0fe0,c7443af4,ed339d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed339d60, ebp = 0 --- Tracing command usb pid 15 tid 100042 td 0xc744ab80 sched_switch(c744ab80,0,104,191,486c4966,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c744ab80,0,c0f24e20,268,c744ab80,...) at sleepq_switch+0x162 sleepq_wait(c7443acc,0,ed336cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7443acc,c7443bfc,c0ef86b1,74,c7443ad4,...) at _cv_wait+0x243 usb_process(c7443ac4,ed336d28,c0f19db0,3db,c7434588,...) at usb_process+0x1a3 fork_exit(c08c0fe0,c7443ac4,ed336d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed336d60, ebp = 0 --- Tracing command usb pid 15 tid 100040 td 0xc742f000 sched_switch(c742f000,0,104,191,486c3497,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c742f000,0,c0f24e20,268,c742f000,...) at sleepq_switch+0x162 sleepq_wait(c743bb5c,0,e0ffecac,1,0,...) at sleepq_wait+0x63 _cv_wait(c743bb5c,c743bbfc,c0ef86b1,74,c743bb64,...) at _cv_wait+0x243 usb_process(c743bb54,e0ffed28,c0f19db0,3db,c7434588,...) at usb_process+0x1a3 fork_exit(c08c0fe0,c743bb54,e0ffed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0ffed60, ebp = 0 --- Tracing command usb pid 15 tid 100039 td 0xc742f2e0 sched_switch(c742f2e0,0,104,191,9d74d083,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c742f2e0,0,c0f24e20,268,c742f2e0,...) at sleepq_switch+0x162 sleepq_wait(c743bb2c,0,e0ffbcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c743bb2c,c743bbfc,c0ef86b1,74,c743bb34,...) at _cv_wait+0x243 usb_process(c743bb24,e0ffbd28,c0f19db0,3db,c7434588,...) at usb_process+0x1a3 fork_exit(c08c0fe0,c743bb24,e0ffbd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0ffbd60, ebp = 0 --- Tracing command usb pid 15 tid 100038 td 0xc742f5c0 sched_switch(c742f5c0,0,104,191,4808528e,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c742f5c0,0,c0f24e20,268,c742f5c0,...) at sleepq_switch+0x162 sleepq_wait(c743bafc,0,e0ff8cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c743bafc,c743bbfc,c0ef86b1,74,c743bb04,...) at _cv_wait+0x243 usb_process(c743baf4,e0ff8d28,c0f19db0,3db,c7434588,...) at usb_process+0x1a3 fork_exit(c08c0fe0,c743baf4,e0ff8d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0ff8d60, ebp = 0 --- Tracing command usb pid 15 tid 100037 td 0xc742f8a0 sched_switch(c742f8a0,0,104,191,480840f5,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c742f8a0,0,c0f24e20,268,c742f8a0,...) at sleepq_switch+0x162 sleepq_wait(c743bacc,0,e0ff5cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c743bacc,c743bbfc,c0ef86b1,74,c743bad4,...) at _cv_wait+0x243 usb_process(c743bac4,e0ff5d28,c0f19db0,3db,c7434588,...) at usb_process+0x1a3 fork_exit(c08c0fe0,c743bac4,e0ff5d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0ff5d60, ebp = 0 --- Tracing command usb pid 15 tid 100035 td 0xc7431000 sched_switch(c7431000,0,104,191,48082ba4,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c7431000,0,c0f24e20,268,c7431000,...) at sleepq_switch+0x162 sleepq_wait(c7428b5c,0,e0feecac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7428b5c,c7428bfc,c0ef86b1,74,c7428b64,...) at _cv_wait+0x243 usb_process(c7428b54,e0feed28,c0f19db0,3db,c7434588,...) at usb_process+0x1a3 fork_exit(c08c0fe0,c7428b54,e0feed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0feed60, ebp = 0 --- Tracing command usb pid 15 tid 100034 td 0xc74312e0 sched_switch(c74312e0,0,104,191,8fefe255,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c74312e0,0,c0f24e20,268,c74312e0,...) at sleepq_switch+0x162 sleepq_wait(c7428b2c,0,e0febcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7428b2c,c7428bfc,c0ef86b1,74,c7428b34,...) at _cv_wait+0x243 usb_process(c7428b24,e0febd28,c0f19db0,3db,c7434588,...) at usb_process+0x1a3 fork_exit(c08c0fe0,c7428b24,e0febd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0febd60, ebp = 0 --- Tracing command usb pid 15 tid 100033 td 0xc74315c0 sched_switch(c74315c0,0,104,191,47a452aa,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c74315c0,0,c0f24e20,268,c74315c0,...) at sleepq_switch+0x162 sleepq_wait(c7428afc,0,e0fe8cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7428afc,c7428bfc,c0ef86b1,74,c7428b04,...) at _cv_wait+0x243 usb_process(c7428af4,e0fe8d28,c0f19db0,3db,c7434588,...) at usb_process+0x1a3 fork_exit(c08c0fe0,c7428af4,e0fe8d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0fe8d60, ebp = 0 --- Tracing command usb pid 15 tid 100032 td 0xc74318a0 sched_switch(c74318a0,0,104,191,47a43f99,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c74318a0,0,c0f24e20,268,c74318a0,...) at sleepq_switch+0x162 sleepq_wait(c7428acc,0,e0fe5cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c7428acc,c7428bfc,c0ef86b1,74,c7428ad4,...) at _cv_wait+0x243 usb_process(c7428ac4,e0fe5d28,c0f19db0,3db,c7434588,...) at usb_process+0x1a3 fork_exit(c08c0fe0,c7428ac4,e0fe5d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0fe5d60, ebp = 0 --- Tracing command yarrow pid 14 tid 100017 td 0xc72d25c0 sched_switch(c72d25c0,0,104,191,984cf303,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c72d25c0,0,c0f24e20,28b,2,...) at sleepq_switch+0x162 sleepq_timedwait(c10b4ca4,0,c0f12072,2,0,...) at sleepq_timedwait+0x6b _sleep(c10b4ca4,0,0,c0f12072,64,...) at _sleep+0x342 pause(c0f12072,64,c0ed5bfe,111,0,...) at pause+0x47 random_kthread(0,c6f6ad28,c0f19db0,3db,c71c6000,...) at random_kthread+0x1ef fork_exit(c07d2af0,0,c6f6ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f6ad60, ebp = 0 --- Tracing command geom pid 13 tid 100015 td 0xc72d2b80 sched_switch(c72d2b80,0,104,191,7244328a,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c72d2b80,0,c0f24e20,268,0,...) at sleepq_switch+0x162 sleepq_wait(c10b28a8,5c,c0f12072,0,0,...) at sleepq_wait+0x63 _sleep(c10b28a8,c10b2808,25c,c0f12072,0,...) at _sleep+0x372 g_io_schedule_down(c72d2b80,0,c0f13bd1,6c,c6f64d14,...) at g_io_schedule_down+0x5b g_down_procbody(0,c6f64d28,c0f19db0,3db,c71c62c4,...) at g_down_procbody+0x9d fork_exit(c09b2110,0,c6f64d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f64d60, ebp = 0 --- Tracing command geom pid 13 tid 100014 td 0xc72d3000 sched_switch(c72d3000,0,104,191,7250d582,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c72d3000,0,c0f24e20,268,0,...) at sleepq_switch+0x162 sleepq_wait(c10b28a4,5c,c0f12072,0,0,...) at sleepq_wait+0x63 _sleep(c10b28a4,c10b2828,25c,c0f12072,0,...) at _sleep+0x372 g_io_schedule_up(c72d3000,0,c0f13bd1,5f,c6f61d14,...) at g_io_schedule_up+0x134 g_up_procbody(0,c6f61d28,c0f19db0,3db,c71c62c4,...) at g_up_procbody+0x9d fork_exit(c09b21b0,0,c6f61d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f61d60, ebp = 0 --- Tracing command geom pid 13 tid 100013 td 0xc71c92e0 sched_switch(c71c92e0,0,104,191,65454187,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c71c92e0,0,c0f24e20,268,0,...) at sleepq_switch+0x162 sleepq_wait(c10b289c,5c,c0f12072,0,0,...) at sleepq_wait+0x63 _sleep(c10b289c,c10b27cc,25c,c0f12072,0,...) at _sleep+0x372 g_run_events(c10c2b00,0,c0f13bd1,79,c6f5ed14,...) at g_run_events+0x54b g_event_procbody(0,c6f5ed28,c0f19db0,3db,c71c62c4,...) at g_event_procbody+0x90 fork_exit(c09b2250,0,c6f5ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f5ed60, ebp = 0 --- Tracing command intr pid 12 tid 100066 td 0xc75758a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100065 td 0xc7575b80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100063 td 0xc744b2e0 sched_switch(c744b2e0,0,109,191,ddebc1b6,...) at sched_switch+0x3bc mi_switch(109,0,c0f1a0d3,554,c744b2e0,...) at mi_switch+0x1eb ithread_loop(c720c6c0,ed385d28,c0f19db0,3db,c71c6588,...) at ithread_loop+0x1fb fork_exit(c09f4040,c720c6c0,ed385d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed385d60, ebp = 0 --- Tracing command intr pid 12 tid 100060 td 0xc744bb80 sched_switch(c744bb80,0,109,191,3d645466,...) at sched_switch+0x3bc mi_switch(109,0,c0f1a0d3,554,c744bb80,...) at mi_switch+0x1eb ithread_loop(c720c6d0,ed379d28,c0f19db0,3db,c71c6588,...) at ithread_loop+0x1fb fork_exit(c09f4040,c720c6d0,ed379d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed379d60, ebp = 0 --- Tracing command intr pid 12 tid 100059 td 0xc7455000 sched_switch(c7455000,0,109,191,75409e5c,...) at sched_switch+0x3bc mi_switch(109,0,c0f1a0d3,554,c7455000,...) at mi_switch+0x1eb ithread_loop(c720c6e0,ed371d28,c0f19db0,3db,c71c6588,...) at ithread_loop+0x1fb fork_exit(c09f4040,c720c6e0,ed371d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed371d60, ebp = 0 --- Tracing command intr pid 12 tid 100054 td 0xc7457000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100041 td 0xc744b000 sched_switch(c744b000,0,109,191,72a1aaea,...) at sched_switch+0x3bc mi_switch(109,0,c0f1a0d3,554,c744b000,...) at mi_switch+0x1eb ithread_loop(c720c700,ed333d28,c0f19db0,3db,c71c6588,...) at ithread_loop+0x1fb fork_exit(c09f4040,c720c700,ed333d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed333d60, ebp = 0 --- Tracing command intr pid 12 tid 100036 td 0xc742fb80 sched_switch(c742fb80,0,109,191,6ccccf94,...) at sched_switch+0x3bc mi_switch(109,0,c0f1a0d3,554,c742fb80,...) at mi_switch+0x1eb ithread_loop(c720c710,e0ff2d28,c0f19db0,3db,c71c6588,...) at ithread_loop+0x1fb fork_exit(c09f4040,c720c710,e0ff2d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0ff2d60, ebp = 0 --- Tracing command intr pid 12 tid 100031 td 0xc7431b80 sched_switch(c7431b80,0,109,191,7d5cf80f,...) at sched_switch+0x3bc mi_switch(109,0,c0f1a0d3,554,c7431b80,...) at mi_switch+0x1eb ithread_loop(c720c720,e0fe2d28,c0f19db0,3db,c71c6588,...) at ithread_loop+0x1fb fork_exit(c09f4040,c720c720,e0fe2d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0fe2d60, ebp = 0 --- Tracing command intr pid 12 tid 100030 td 0xc72d32e0 sched_switch(c72d32e0,0,109,191,724f4ac7,...) at sched_switch+0x3bc mi_switch(109,0,c0f1a0d3,554,c72d32e0,...) at mi_switch+0x1eb ithread_loop(c720c730,e0e8cd28,c0f19db0,3db,c71c6588,...) at ithread_loop+0x1fb fork_exit(c09f4040,c720c730,e0e8cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0e8cd60, ebp = 0 --- Tracing command intr pid 12 tid 100029 td 0xc72d35c0 sched_switch(c72d35c0,0,109,191,f338701f,...) at sched_switch+0x3bc mi_switch(109,0,c0f1a0d3,554,c72d35c0,...) at mi_switch+0x1eb ithread_loop(c720c740,e0e88d28,c0f19db0,3db,c71c6588,...) at ithread_loop+0x1fb fork_exit(c09f4040,c720c740,e0e88d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe0e88d60, ebp = 0 --- Tracing command intr pid 12 tid 100028 td 0xc72d38a0 sched_switch(c72d38a0,0,109,191,3e076db7,...) at sched_switch+0x3bc mi_switch(109,0,c0f1a0d3,554,c72d38a0,...) at mi_switch+0x1eb ithread_loop(c720c750,c6fb9d28,c0f19db0,3db,c71c6588,...) at ithread_loop+0x1fb fork_exit(c09f4040,c720c750,c6fb9d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6fb9d60, ebp = 0 --- Tracing command intr pid 12 tid 100027 td 0xc72d3b80 sched_switch(c72d3b80,0,109,191,8eb4f883,...) at sched_switch+0x3bc mi_switch(109,0,c0f1a0d3,554,c72d3b80,...) at mi_switch+0x1eb ithread_loop(c720c760,c6f88d28,c0f19db0,3db,c71c6588,...) at ithread_loop+0x1fb fork_exit(c09f4040,c720c760,c6f88d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f88d60, ebp = 0 --- Tracing command intr pid 12 tid 100025 td 0xc73bf2e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100024 td 0xc73bf5c0 sched_switch(c73bf5c0,0,109,191,725020e3,...) at sched_switch+0x3bc mi_switch(109,0,c0f1a0d3,554,c73bf5c0,...) at mi_switch+0x1eb ithread_loop(c720c780,c6f7fd28,c0f19db0,3db,c71c6588,...) at ithread_loop+0x1fb fork_exit(c09f4040,c720c780,c6f7fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f7fd60, ebp = 0 --- Tracing command intr pid 12 tid 100018 td 0xc72d22e0 sched_switch(c72d22e0,0,109,191,b00c7e4c,...) at sched_switch+0x3bc mi_switch(109,0,c0f1a0d3,554,c72d22e0,...) at mi_switch+0x1eb ithread_loop(c720c790,c6f6dd28,c0f19db0,3db,c71c6588,...) at ithread_loop+0x1fb fork_exit(c09f4040,c720c790,c6f6dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f6dd60, ebp = 0 --- Tracing command intr pid 12 tid 100012 td 0xc71c95c0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100011 td 0xc71c98a0 sched_switch(c71c98a0,0,109,191,37d8a4b3,...) at sched_switch+0x3bc mi_switch(109,0,c0f1a0d3,554,c71c98a0,...) at mi_switch+0x1eb ithread_loop(c720c7b0,c6f58d28,c0f19db0,3db,c71c6588,...) at ithread_loop+0x1fb fork_exit(c09f4040,c720c7b0,c6f58d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f58d60, ebp = 0 --- Tracing command intr pid 12 tid 100010 td 0xc71c9b80 sched_switch(c71c9b80,0,109,191,44014c37,...) at sched_switch+0x3bc mi_switch(109,0,c0f1a0d3,554,c71c9b80,...) at mi_switch+0x1eb ithread_loop(c720c7c0,c6f55d28,c0f19db0,3db,c71c6588,...) at ithread_loop+0x1fb fork_exit(c09f4040,c720c7c0,c6f55d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f55d60, ebp = 0 --- Tracing command intr pid 12 tid 100009 td 0xc720d000 sched_switch(c720d000,0,109,191,984cb277,...) at sched_switch+0x3bc mi_switch(109,0,c0f1a0d3,554,c720d000,...) at mi_switch+0x1eb ithread_loop(c720c7d0,c6f52d28,c0f19db0,3db,c71c6588,...) at ithread_loop+0x1fb fork_exit(c09f4040,c720c7d0,c6f52d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f52d60, ebp = 0 --- Tracing command intr pid 12 tid 100008 td 0xc720d2e0 sched_switch(c720d2e0,0,109,191,9d220f5a,...) at sched_switch+0x3bc mi_switch(109,0,c0f1a0d3,554,c720d2e0,...) at mi_switch+0x1eb ithread_loop(c720c7e0,c6f4fd28,c0f19db0,3db,c71c6588,...) at ithread_loop+0x1fb fork_exit(c09f4040,c720c7e0,c6f4fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f4fd60, ebp = 0 --- Tracing command intr pid 12 tid 100007 td 0xc720d5c0 sched_switch(c720d5c0,0,109,191,95a32e08,...) at sched_switch+0x3bc mi_switch(109,0,c0f1a0d3,554,c720d5c0,...) at mi_switch+0x1eb ithread_loop(c720c7f0,c6f4cd28,c0f19db0,3db,c71c6588,...) at ithread_loop+0x1fb fork_exit(c09f4040,c720c7f0,c6f4cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f4cd60, ebp = 0 --- Tracing command idle pid 11 tid 100006 td 0xc71c8000 cpustop_handler(3,13,c6f48c04,c0d56916,f5,...) at cpustop_handler+0x28 ipi_nmi_handler(f5,0,0,c12621e4,c71c684c,...) at ipi_nmi_handler+0x47 trap(c6f48c10) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0d3b2a5, esp = 0xc6f48c50, ebp = 0xc6f48c50 --- acpi_cpu_c1(c6f48c6c,c107add0,3,301,95d8,...) at acpi_cpu_c1+0x5 acpi_cpu_idle(0,c6f48ca0,c0d49549,0,ffffffff,...) at acpi_cpu_idle+0x122 cpu_idle_acpi(0,ffffffff,c10c3e80,2,c6f48ce8,...) at cpu_idle_acpi+0x2f cpu_idle(0,c6f48cc4,c0f21562,a0a,c71c8000,...) at cpu_idle+0x89 sched_idletd(0,c6f48d28,c0f19db0,3db,c71c684c,...) at sched_idletd+0x263 fork_exit(c0a43630,0,c6f48d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f48d60, ebp = 0 --- Tracing command idle pid 11 tid 100005 td 0xc71c82e0 sched_switch(c71c82e0,0,108,18c,8d095765,...) at sched_switch+0x3bc mi_switch(108,0,c0f21562,3be,c71c82e0,...) at mi_switch+0x1eb sched_idletd(0,c6f45d28,c0f19db0,3db,c71c684c,...) at sched_idletd+0x19f fork_exit(c0a43630,0,c6f45d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f45d60, ebp = 0 --- Tracing command idle pid 11 tid 100004 td 0xc71c85c0 cpustop_handler(1,13,c6f42c04,c0d56916,f5,...) at cpustop_handler+0x28 ipi_nmi_handler(f5,0,1f04160,c0d3b2a5,c71c684c,...) at ipi_nmi_handler+0x47 trap(c6f42c10) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0d3b2a5, esp = 0xc6f42c50, ebp = 0xc6f42c50 --- acpi_cpu_c1(c6f42c6c,c107add0,1,301,95d7,...) at acpi_cpu_c1+0x5 acpi_cpu_idle(0,c6f42ca0,c0d49549,0,ffffffff,...) at acpi_cpu_idle+0x122 cpu_idle_acpi(0,ffffffff,c10c3180,2,c6f42ce8,...) at cpu_idle_acpi+0x2f cpu_idle(0,c6f42cc4,c0f21562,3c0,c71c85c0,...) at cpu_idle+0x89 sched_idletd(0,c6f42d28,c0f19db0,3db,c71c684c,...) at sched_idletd+0x263 fork_exit(c0a43630,0,c6f42d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f42d60, ebp = 0 --- Tracing command idle pid 11 tid 100003 td 0xc71c88a0 kdb_enter(c0d94b3e,c0f23499,c6f3fba4,c088c62b,c755c000,...) at kdb_enter+0x3a kdb_break(c755c000,1f04160,c0d3b2a5,c755c080,c755c058,...) at kdb_break+0x28 uart_intr(c755c000,0,c1261860,c6f3fbe8,0,...) at uart_intr+0x11b intr_event_handle(c71c4d00,c6f3fc10,c6f3fc34,0,c73ee200,...) at intr_event_handle+0x76 intr_execute_handlers(c720b0d0,c6f3fc10,0,c6f3fc50,c0d3fca5,...) at intr_execute_handlers+0x49 lapic_handle_intr(3c,c6f3fc10) at lapic_handle_intr+0x36 Xapic_isr1() at Xapic_isr1+0x35 --- interrupt, eip = 0xc0d3b2a5, esp = 0xc6f3fc50, ebp = 0xc6f3fc50 --- acpi_cpu_c1(c6f3fc6c,c107add0,0,301,95d7,...) at acpi_cpu_c1+0x5 acpi_cpu_idle(0,c6f3fca0,c0d49549,0,ffffffff,...) at acpi_cpu_idle+0x122 cpu_idle_acpi(0,ffffffff,c10c2b00,2,c6f3fce8,...) at cpu_idle_acpi+0x2f cpu_idle(0,c6f3fcc4,c0f21562,3c0,c71c88a0,...) at cpu_idle+0x89 sched_idletd(0,c6f3fd28,c0f19db0,3db,c71c684c,...) at sched_idletd+0x263 fork_exit(c0a43630,0,c6f3fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f3fd60, ebp = 0 --- Tracing command init pid 1 tid 100002 td 0xc71c8b80 sched_switch(c71c8b80,0,104,191,f0de88ef,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c71c8b80,0,c0f24e20,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f24e20,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c71c6b10,6c,c0f27c03,100,0,...) at sleepq_wait_sig+0x17 _sleep(c71c6b10,c71c6b98,16c,c0f27c03,0,...) at _sleep+0x35c kern_wait(c71c8b80,ffffffff,c6f3bc60,2,0,...) at kern_wait+0x626 sys_wait4(c71c8b80,c6f3bcec,c,c,282,...) at sys_wait4+0x3b syscall(c6f3bd28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x8060df7, esp = 0xbfbfe81c, ebp = 0xbfbfe838 --- Tracing command audit pid 10 tid 100001 td 0xc71c9000 sched_switch(c71c9000,0,104,191,47a1ecb3,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c71c9000,0,c0f24e20,268,c71c9000,...) at sleepq_switch+0x162 sleepq_wait(c1231e80,0,c6f38c8c,1,0,...) at sleepq_wait+0x63 _cv_wait(c1231e80,c1231e64,c0f4fe37,194,0,...) at _cv_wait+0x243 audit_worker(0,c6f38d28,c0f19db0,3db,c71c7000,...) at audit_worker+0x84 fork_exit(c0c19b70,0,c6f38d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f38d60, ebp = 0 --- Tracing command kernel pid 0 tid 100184 td 0xc83e9b80 sched_switch(c83e9b80,0,104,191,589a79e3,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c83e9b80,0,c0f24e20,268,0,...) at sleepq_switch+0x162 sleepq_wait(c78c7000,0,c0f12072,0,0,...) at sleepq_wait+0x63 _sleep(c78c7000,c78c7018,0,c0f12072,0,...) at _sleep+0x372 taskqueue_thread_loop(c9d52900,f19ebd28,c0f19db0,3db,c10b2980,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a60310,c9d52900,f19ebd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xf19ebd60, ebp = 0 --- Tracing command kernel pid 0 tid 100077 td 0xc77f9b80 sched_switch(c77f9b80,0,104,191,27ba5d8a,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c77f9b80,0,c0f24e20,268,0,...) at sleepq_switch+0x162 sleepq_wait(c77cfb00,0,c0f12072,0,0,...) at sleepq_wait+0x63 _sleep(c77cfb00,c77cfb18,0,c0f12072,0,...) at _sleep+0x372 taskqueue_thread_loop(c12941e0,f17f1d28,c0f19db0,3db,c10b2980,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a60310,c12941e0,f17f1d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xf17f1d60, ebp = 0 --- Tracing command kernel pid 0 tid 100067 td 0xc75755c0 sched_switch(c75755c0,0,104,191,536c281e,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c75755c0,0,c0f24e20,28b,2,...) at sleepq_switch+0x162 sleepq_timedwait(c10b4ca4,0,c0f12072,2,0,...) at sleepq_timedwait+0x6b _sleep(c10b4ca4,0,0,c0f12072,bb8,...) at _sleep+0x342 pause(c0f12072,bb8,124,122,c10b2a08,...) at pause+0x47 deadlkres(0,ed39bd28,c0f19db0,3db,c10b2980,...) at deadlkres+0x324 fork_exit(c09d4260,0,ed39bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed39bd60, ebp = 0 --- Tracing command kernel pid 0 tid 100061 td 0xc744b8a0 sched_switch(c744b8a0,0,104,191,9d21eef4,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c744b8a0,0,c0f24e20,268,0,...) at sleepq_switch+0x162 sleepq_wait(c747b140,0,c0f20221,c0f12072,0,...) at sleepq_wait+0x63 msleep_spin(c747b140,c747b158,c0f12072,0,c0f1da4b,...) at msleep_spin+0x21d taskqueue_thread_loop(c748049c,ed37fd28,c0f19db0,3db,c10b2980,...) at taskqueue_thread_loop+0x8f fork_exit(c0a60310,c748049c,ed37fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xed37fd60, ebp = 0 --- Tracing command kernel pid 0 tid 100026 td 0xc73bf000 sched_switch(c73bf000,0,104,191,6a01a9ed,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c73bf000,0,c0f24e20,268,0,...) at sleepq_switch+0x162 sleepq_wait(c7354400,0,c0f12072,0,0,...) at sleepq_wait+0x63 _sleep(c7354400,c7354418,0,c0f12072,0,...) at _sleep+0x372 taskqueue_thread_loop(c10d186c,c6f85d28,c0f19db0,3db,c10b2980,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a60310,c10d186c,c6f85d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f85d60, ebp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xc73bf8a0 sched_switch(c73bf8a0,0,104,191,61e34564,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c73bf8a0,0,c0f24e20,268,0,...) at sleepq_switch+0x162 sleepq_wait(c7354480,0,c0f12072,0,0,...) at sleepq_wait+0x63 _sleep(c7354480,c7354498,0,c0f12072,0,...) at _sleep+0x372 taskqueue_thread_loop(c10b3318,c6f7cd28,c0f19db0,3db,c10b2980,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a60310,c10b3318,c6f7cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f7cd60, ebp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xc73bfb80 sched_switch(c73bfb80,0,104,191,61e332f5,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c73bfb80,0,c0f24e20,268,0,...) at sleepq_switch+0x162 sleepq_wait(c73544c0,0,c0f20221,c0f12072,0,...) at sleepq_wait+0x63 msleep_spin(c73544c0,c73544d8,c0f12072,0,c0f1da4b,...) at msleep_spin+0x21d taskqueue_thread_loop(c107da38,c6f79d28,c0f19db0,3db,c10b2980,...) at taskqueue_thread_loop+0x8f fork_exit(c0a60310,c107da38,c6f79d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f79d60, ebp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xc720d8a0 sched_switch(c720d8a0,0,104,191,61e32462,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c720d8a0,0,c0f24e20,268,0,...) at sleepq_switch+0x162 sleepq_wait(c73544c0,0,c0f20221,c0f12072,0,...) at sleepq_wait+0x63 msleep_spin(c73544c0,c73544d8,c0f12072,0,c0f1da4b,...) at msleep_spin+0x21d taskqueue_thread_loop(c107da38,c6f76d28,c0f19db0,3db,c10b2980,...) at taskqueue_thread_loop+0x8f fork_exit(c0a60310,c107da38,c6f76d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f76d60, ebp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xc720db80 sched_switch(c720db80,0,104,191,61e311a1,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c720db80,0,c0f24e20,268,0,...) at sleepq_switch+0x162 sleepq_wait(c73544c0,0,c0f20221,c0f12072,0,...) at sleepq_wait+0x63 msleep_spin(c73544c0,c73544d8,c0f12072,0,c0f1da4b,...) at msleep_spin+0x21d taskqueue_thread_loop(c107da38,c6f73d28,c0f19db0,3db,c10b2980,...) at taskqueue_thread_loop+0x8f fork_exit(c0a60310,c107da38,c6f73d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f73d60, ebp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xc72d2000 sched_switch(c72d2000,0,104,191,61df9009,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c72d2000,0,c0f24e20,268,0,...) at sleepq_switch+0x162 sleepq_wait(c7354500,0,c0f12072,0,0,...) at sleepq_wait+0x63 _sleep(c7354500,c7354518,0,c0f12072,0,...) at _sleep+0x372 taskqueue_thread_loop(c1232464,c6f70d28,c0f19db0,3db,c10b2980,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a60310,c1232464,c6f70d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f70d60, ebp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xc72d28a0 sched_switch(c72d28a0,0,104,191,2bd5067a,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c72d28a0,0,c0f24e20,268,0,...) at sleepq_switch+0x162 sleepq_wait(c7195c40,0,c0f12072,0,0,...) at sleepq_wait+0x63 _sleep(c7195c40,c7195c58,0,c0f12072,0,...) at _sleep+0x372 taskqueue_thread_loop(c10d0280,c6f67d28,c0f19db0,3db,c10b2980,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a60310,c10d0280,c6f67d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc6f67d60, ebp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xc10b2c50 sched_switch(c10b2c50,0,104,191,113c91dd,...) at sched_switch+0x3bc mi_switch(104,0,c0f24e20,1f3,54,...) at mi_switch+0x1eb sleepq_switch(c10b2c50,0,c0f24e20,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c10b2980,54,c0f21e74,0,0,...) at sleepq_timedwait+0x6b _sleep(c10b2980,0,54,c0f21e74,2710,...) at _sleep+0x342 scheduler(0,181ec00,181ec00,181e000,1825000,...) at scheduler+0x240 mi_startup() at mi_startup+0xac begin() at begin+0x2c db:0:allt> call doadump Physical memory: 3435 MB Dumping 268 MB: 253 237 221 205 189 173 157 141 125 109 93 77 61 45 29 13 Dump complete = 0 db:0:doadump> reset (kgdb) thread 85 [Switching to thread 85 (Thread 778533)]#0 cpustop_handler () at ../../../i386/i386/mp_machdep.c:1496 1496 CPU_SET_ATOMIC(cpu, &stopped_cpus); (kgdb) bt #0 cpustop_handler () at ../../../i386/i386/mp_machdep.c:1496 #1 0xc0d4bce7 in ipi_nmi_handler () at ../../../i386/i386/mp_machdep.c:1478 #2 0xc0d56916 in trap (frame=0xf1966b20) at ../../../i386/i386/trap.c:223 #3 0xc0d3f8dc in calltrap () at ../../../i386/i386/exception.s:168 #4 0xc0a68e80 in witness_checkorder (lock=0xc10b7070, flags=0x9, file=0xc0f20e13 "../../../kern/kern_umtx.c", line=0x11f, interlock=0x0) at ../../../kern/subr_witness.c:1353 #5 0xc0a0d8d4 in _mtx_lock_flags (m=0xc10b7070, opts=0x0, file=0xc0f20e13 "../../../kern/kern_umtx.c", line=0x11f) at ../../../kern/kern_mutex.c:200 #6 0xc0a36669 in do_rw_wrlock (td=Variable "td" is not available. ) at ../../../kern/kern_umtx.c:287 #7 0xc0a37142 in __umtx_op_rw_wrlock (td=0xc9d9f000, uap=0xf1966cec) at ../../../kern/kern_umtx.c:3163 #8 0xc0a31ff7 in sys__umtx_op (td=0xc9d9f000, uap=0xf1966cec) at ../../../kern/kern_umtx.c:3245 #9 0xc0d56514 in syscall (frame=0xf1966d28) at subr_syscall.c:131 #10 0xc0d3f941 in Xint0x80_syscall () at ../../../i386/i386/exception.s:266 #11 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) f 7 #7 0xc0a37142 in __umtx_op_rw_wrlock (td=0xc9d9f000, uap=0xf1966cec) at ../../../kern/kern_umtx.c:3163 3163 error = do_rw_wrlock(td, uap->obj, 0); (kgdb) p *uap $1 = {obj_l_ = 0xf1966cec "\200¿\n(\r", obj = 0x280aed80, obj_r_ = 0xf1966cf0 "\r", op_l_ = 0xf1966cf0 "\r", op = 0xd, op_r_ = 0xf1966cf4 "", val_l_ = 0xf1966cf4 "", val = 0x0, val_r_ = 0xf1966cf8 "", uaddr1_l_ = 0xf1966cf8 "", uaddr1 = 0x0, uaddr1_r_ = 0xf1966cfc "", uaddr2_l_ = 0xf1966cfc "", uaddr2 = 0x0, uaddr2_r_ = 0xf1966d00 ""} (kgdb) f 6 #6 0xc0a36669 in do_rw_wrlock (td=Variable "td" is not available. ) at ../../../kern/kern_umtx.c:287 287 mtx_lock(&uc->uc_lock); (kgdb) l 282 umtxq_lock(struct umtx_key *key) 283 { 284 struct umtxq_chain *uc; 285 286 uc = umtxq_getchain(key); 287 mtx_lock(&uc->uc_lock); 288 } 289 290 /* 291 * Unlock a chain. (kgdb) info loc uq = (struct umtx_q *) 0xc9d9e500 state = 0xffffffff oldstate = 0x0 blocked_writers = Variable "blocked_writers" is not available. (kgdb) p *uq $2 = {uq_link = {tqe_next = 0x0, tqe_prev = 0xc83c4d00}, uq_key = {hash = 0x27, type = 0x7, shared = 0x0, info = {shared = {object = 0xc7980e80, offset = 0x280aed80}, private = { vs = 0xc7980e80, addr = 0x280aed80}, both = {a = 0xc7980e80, b = 0x280aed80}}}, uq_flags = 0x0, uq_thread = 0xc9d9f000, uq_pi_blocked = 0x0, uq_lockq = {tqe_next = 0x0, tqe_prev = 0x0}, uq_pi_contested = {tqh_first = 0x0, tqh_last = 0xc9d9e530}, uq_inherited_pri = 0xff, uq_spare_queue = 0xc83c4d00, uq_cur_queue = 0x0} (kgdb) $ svn diff -x -p /usr/src/sys Index: /usr/src/sys/kern/vfs_default.c =================================================================== --- /usr/src/sys/kern/vfs_default.c (revision 227502) +++ /usr/src/sys/kern/vfs_default.c (working copy) @@ -844,7 +844,7 @@ out: free(dirbuf, M_TEMP); if (!error) { *buflen = i; - vhold(*dvp); + vref(*dvp); } if (covered) { vput(*dvp); Index: /usr/src/sys/kern/vfs_cache.c =================================================================== --- /usr/src/sys/kern/vfs_cache.c (revision 227502) +++ /usr/src/sys/kern/vfs_cache.c (working copy) @@ -1068,16 +1068,8 @@ vn_vptocnp(struct vnode **vp, struct ucred *cred, CACHE_RLOCK(); error = vn_vptocnp_locked(vp, cred, buf, buflen); - if (error == 0) { - /* - * vn_vptocnp_locked() dropped hold acquired by - * VOP_VPTOCNP immediately after locking the - * cache. Since we are going to drop the cache rlock, - * re-hold the result. - */ - vhold(*vp); + if (error == 0) CACHE_RUNLOCK(); - } return (error); } @@ -1096,6 +1088,9 @@ vn_vptocnp_locked(struct vnode **vp, struct ucred if (ncp != NULL) { if (*buflen < ncp->nc_nlen) { CACHE_RUNLOCK(); + vfslocked = VFS_LOCK_GIANT((*vp)->v_mount); + vrele(*vp); + VFS_UNLOCK_GIANT(vfslocked); numfullpathfail4++; error = ENOMEM; SDT_PROBE(vfs, namecache, fullpath, return, error, @@ -1106,18 +1101,23 @@ vn_vptocnp_locked(struct vnode **vp, struct ucred memcpy(buf + *buflen, ncp->nc_name, ncp->nc_nlen); SDT_PROBE(vfs, namecache, fullpath, hit, ncp->nc_dvp, ncp->nc_name, vp, 0, 0); + dvp = *vp; *vp = ncp->nc_dvp; + vref(*vp); + CACHE_RUNLOCK(); + vfslocked = VFS_LOCK_GIANT(dvp->v_mount); + vrele(dvp); + VFS_UNLOCK_GIANT(vfslocked); + CACHE_RLOCK(); return (0); } SDT_PROBE(vfs, namecache, fullpath, miss, vp, 0, 0, 0, 0); - vhold(*vp); CACHE_RUNLOCK(); vfslocked = VFS_LOCK_GIANT((*vp)->v_mount); vn_lock(*vp, LK_SHARED | LK_RETRY); error = VOP_VPTOCNP(*vp, &dvp, cred, buf, buflen); - VOP_UNLOCK(*vp, 0); - vdrop(*vp); + vput(*vp); VFS_UNLOCK_GIANT(vfslocked); if (error) { numfullpathfail2++; @@ -1128,16 +1128,20 @@ vn_vptocnp_locked(struct vnode **vp, struct ucred *vp = dvp; CACHE_RLOCK(); - if ((*vp)->v_iflag & VI_DOOMED) { + if (dvp->v_iflag & VI_DOOMED) { /* forced unmount */ CACHE_RUNLOCK(); - vdrop(*vp); + vfslocked = VFS_LOCK_GIANT(dvp->v_mount); + vrele(dvp); + VFS_UNLOCK_GIANT(vfslocked); error = ENOENT; SDT_PROBE(vfs, namecache, fullpath, return, error, vp, NULL, 0, 0); return (error); } - vdrop(*vp); + /* + * *vp has its use count incremented still. + */ return (0); } @@ -1149,10 +1153,11 @@ static int vn_fullpath1(struct thread *td, struct vnode *vp, struct vnode *rdir, char *buf, char **retbuf, u_int buflen) { - int error, slash_prefixed; + int error, slash_prefixed, vfslocked; #ifdef KDTRACE_HOOKS struct vnode *startvp = vp; #endif + struct vnode *vp1; buflen--; buf[buflen] = '\0'; @@ -1161,6 +1166,7 @@ vn_fullpath1(struct thread *td, struct vnode *vp, SDT_PROBE(vfs, namecache, fullpath, entry, vp, 0, 0, 0, 0); numfullpathcalls++; + vref(vp); CACHE_RLOCK(); if (vp->v_type != VDIR) { error = vn_vptocnp_locked(&vp, td->td_ucred, buf, &buflen); @@ -1168,6 +1174,9 @@ vn_fullpath1(struct thread *td, struct vnode *vp, return (error); if (buflen == 0) { CACHE_RUNLOCK(); + vfslocked = VFS_LOCK_GIANT(vp->v_mount); + vrele(vp); + VFS_UNLOCK_GIANT(vfslocked); return (ENOMEM); } buf[--buflen] = '/'; @@ -1177,16 +1186,29 @@ vn_fullpath1(struct thread *td, struct vnode *vp, if (vp->v_vflag & VV_ROOT) { if (vp->v_iflag & VI_DOOMED) { /* forced unmount */ CACHE_RUNLOCK(); + vfslocked = VFS_LOCK_GIANT(vp->v_mount); + vrele(vp); + VFS_UNLOCK_GIANT(vfslocked); error = ENOENT; SDT_PROBE(vfs, namecache, fullpath, return, error, vp, NULL, 0, 0); break; } - vp = vp->v_mount->mnt_vnodecovered; + vp1 = vp->v_mount->mnt_vnodecovered; + vref(vp1); + CACHE_RUNLOCK(); + vfslocked = VFS_LOCK_GIANT(vp->v_mount); + vrele(vp); + VFS_UNLOCK_GIANT(vfslocked); + vp = vp1; + CACHE_RLOCK(); continue; } if (vp->v_type != VDIR) { CACHE_RUNLOCK(); + vfslocked = VFS_LOCK_GIANT(vp->v_mount); + vrele(vp); + VFS_UNLOCK_GIANT(vfslocked); numfullpathfail1++; error = ENOTDIR; SDT_PROBE(vfs, namecache, fullpath, return, @@ -1198,6 +1220,9 @@ vn_fullpath1(struct thread *td, struct vnode *vp, break; if (buflen == 0) { CACHE_RUNLOCK(); + vfslocked = VFS_LOCK_GIANT(vp->v_mount); + vrele(vp); + VFS_UNLOCK_GIANT(vfslocked); error = ENOMEM; SDT_PROBE(vfs, namecache, fullpath, return, error, startvp, NULL, 0, 0); @@ -1211,6 +1236,9 @@ vn_fullpath1(struct thread *td, struct vnode *vp, if (!slash_prefixed) { if (buflen == 0) { CACHE_RUNLOCK(); + vfslocked = VFS_LOCK_GIANT(vp->v_mount); + vrele(vp); + VFS_UNLOCK_GIANT(vfslocked); numfullpathfail4++; SDT_PROBE(vfs, namecache, fullpath, return, ENOMEM, startvp, NULL, 0, 0); @@ -1220,6 +1248,9 @@ vn_fullpath1(struct thread *td, struct vnode *vp, } numfullpathfound++; CACHE_RUNLOCK(); + vfslocked = VFS_LOCK_GIANT(vp->v_mount); + vrele(vp); + VFS_UNLOCK_GIANT(vfslocked); SDT_PROBE(vfs, namecache, fullpath, return, 0, startvp, buf + buflen, 0, 0); Index: /usr/src/sys/fs/pseudofs/pseudofs_vnops.c =================================================================== --- /usr/src/sys/fs/pseudofs/pseudofs_vnops.c (revision 227502) +++ /usr/src/sys/fs/pseudofs/pseudofs_vnops.c (working copy) @@ -410,8 +410,7 @@ pfs_vptocnp(struct vop_vptocnp_args *ap) } *buflen = i; - vhold(*dvp); - vput(*dvp); + VOP_UNLOCK(*dvp, 0); vn_lock(vp, locked | LK_RETRY); vfs_unbusy(mp); @@ -632,8 +631,6 @@ pfs_read(struct vop_read_args *va) (offset = uio->uio_offset) != uio->uio_offset || (resid = uio->uio_resid) != uio->uio_resid || (buflen = offset + resid + 1) < offset || buflen > INT_MAX) { - if (proc != NULL) - PRELE(proc); error = EINVAL; goto ret; } @@ -713,6 +710,13 @@ pfs_iterate(struct thread *td, struct proc *proc, return (0); } +/* Directory entry list */ +struct pfsentry { + STAILQ_ENTRY(pfsentry) link; + struct dirent entry; +}; +STAILQ_HEAD(pfsdirentlist, pfsentry); + /* * Return directory entries. */ @@ -725,12 +729,14 @@ pfs_readdir(struct vop_readdir_args *va) pid_t pid = pvd->pvd_pid; struct proc *p, *proc; struct pfs_node *pn; - struct dirent *entry; struct uio *uio; + struct pfsentry *pfsent, *pfsent2; + struct pfsdirentlist lst; off_t offset; int error, i, resid; - char *buf, *ent; + STAILQ_INIT(&lst); + error = 0; KASSERT(pd->pn_info == vn->v_mount->mnt_data, ("%s(): pn_info does not match mountpoint", __func__)); PFS_TRACE(("%s pid %lu", pd->pn_name, (unsigned long)pid)); @@ -750,8 +756,6 @@ pfs_readdir(struct vop_readdir_args *va) if (resid == 0) PFS_RETURN (0); - /* can't do this while holding the proc lock... */ - buf = malloc(resid, M_IOV, M_WAITOK | M_ZERO); sx_slock(&allproc_lock); pfs_lock(pd); @@ -759,7 +763,6 @@ pfs_readdir(struct vop_readdir_args *va) if (!pfs_visible(curthread, pd, pid, &proc)) { sx_sunlock(&allproc_lock); pfs_unlock(pd); - free(buf, M_IOV); PFS_RETURN (ENOENT); } KASSERT(pid == NO_PID || proc != NULL, @@ -773,57 +776,64 @@ pfs_readdir(struct vop_readdir_args *va) PROC_UNLOCK(proc); pfs_unlock(pd); sx_sunlock(&allproc_lock); - free(buf, M_IOV); PFS_RETURN (0); } } /* fill in entries */ - ent = buf; while (pfs_iterate(curthread, proc, pd, &pn, &p) != -1 && resid >= PFS_DELEN) { - entry = (struct dirent *)ent; - entry->d_reclen = PFS_DELEN; - entry->d_fileno = pn_fileno(pn, pid); + if ((pfsent = malloc(sizeof(struct pfsentry), M_IOV, + M_NOWAIT | M_ZERO)) == NULL) { + error = ENOMEM; + break; + } + pfsent->entry.d_reclen = PFS_DELEN; + pfsent->entry.d_fileno = pn_fileno(pn, pid); /* PFS_DELEN was picked to fit PFS_NAMLEN */ for (i = 0; i < PFS_NAMELEN - 1 && pn->pn_name[i] != '\0'; ++i) - entry->d_name[i] = pn->pn_name[i]; - entry->d_name[i] = 0; - entry->d_namlen = i; + pfsent->entry.d_name[i] = pn->pn_name[i]; + pfsent->entry.d_name[i] = 0; + pfsent->entry.d_namlen = i; switch (pn->pn_type) { case pfstype_procdir: KASSERT(p != NULL, ("reached procdir node with p == NULL")); - entry->d_namlen = snprintf(entry->d_name, + pfsent->entry.d_namlen = snprintf(pfsent->entry.d_name, PFS_NAMELEN, "%d", p->p_pid); /* fall through */ case pfstype_root: case pfstype_dir: case pfstype_this: case pfstype_parent: - entry->d_type = DT_DIR; + pfsent->entry.d_type = DT_DIR; break; case pfstype_file: - entry->d_type = DT_REG; + pfsent->entry.d_type = DT_REG; break; case pfstype_symlink: - entry->d_type = DT_LNK; + pfsent->entry.d_type = DT_LNK; break; default: panic("%s has unexpected node type: %d", pn->pn_name, pn->pn_type); } - PFS_TRACE(("%s", entry->d_name)); + PFS_TRACE(("%s", pfsent->entry.d_name)); + STAILQ_INSERT_TAIL(&lst, pfsent, link); offset += PFS_DELEN; resid -= PFS_DELEN; - ent += PFS_DELEN; } if (proc != NULL) PROC_UNLOCK(proc); pfs_unlock(pd); sx_sunlock(&allproc_lock); - PFS_TRACE(("%zd bytes", ent - buf)); - error = uiomove(buf, ent - buf, uio); - free(buf, M_IOV); + i = 0; + STAILQ_FOREACH_SAFE(pfsent, &lst, link, pfsent2) { + if (error == 0) + error = uiomove(&pfsent->entry, PFS_DELEN, uio); + free(pfsent, M_IOV); + i++; + } + PFS_TRACE(("%zd bytes", i * PFS_DELEN)); PFS_RETURN (error); } Index: /usr/src/sys/fs/nullfs/null_vnops.c =================================================================== --- /usr/src/sys/fs/nullfs/null_vnops.c (revision 227502) +++ /usr/src/sys/fs/nullfs/null_vnops.c (working copy) @@ -808,8 +808,7 @@ null_vptocnp(struct vop_vptocnp_args *ap) #ifdef DIAGNOSTIC NULLVPTOLOWERVP(*dvp); #endif - vhold(*dvp); - vput(*dvp); + VOP_UNLOCK(*dvp, 0); /* keep reference on *dvp */ } else vput(ldvp); Index: /usr/src/sys/fs/devfs/devfs_vnops.c =================================================================== --- /usr/src/sys/fs/devfs/devfs_vnops.c (revision 227502) +++ /usr/src/sys/fs/devfs/devfs_vnops.c (working copy) @@ -261,7 +261,7 @@ devfs_vptocnp(struct vop_vptocnp_args *ap) } else if (vp->v_type == VDIR) { if (dd == dmp->dm_rootdir) { *dvp = vp; - vhold(*dvp); + vref(*dvp); goto finished; } i -= dd->de_dirent->d_namlen; @@ -289,6 +289,8 @@ devfs_vptocnp(struct vop_vptocnp_args *ap) mtx_unlock(&devfs_de_interlock); vholdl(*dvp); VI_UNLOCK(*dvp); + vref(*dvp); + vdrop(*dvp); } else { mtx_unlock(&devfs_de_interlock); error = ENOENT; Index: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ctldir.c =================================================================== --- /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ctldir.c (revision 227502) +++ /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ctldir.c (working copy) @@ -1594,7 +1594,7 @@ zfsctl_snapshot_vptocnp(struct vop_vptocnp_args *a *ap->a_buflen -= len; bcopy(sep->se_name, ap->a_buf + *ap->a_buflen, len); mutex_exit(&sdp->sd_lock); - vhold(dvp); + vref(dvp); *ap->a_vpp = dvp; } VN_RELE(dvp); -- Test scenario: syscall2.sh