GENERIC HEAD from 2012-02-21 01:05:12 UTC, r231973M, vmcore.291 KDB: debugger backends: ddb KDB: current backend: ddb 524288K of memory above 4GB ignored Copyright (c) 1992-2012 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #2 r231973M: Wed Feb 22 12:57:19 CET 2012 pho@x4.osted.lan:/usr/src/sys/i386/compile/PHO i386 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. CPU: AMD Phenom(tm) 9150e Quad-Core Processor (1800.02-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x100f23 Family = 10 Model = 2 Stepping = 3 Features=0x178bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x7ff TSC: P-state invariant real memory = 4294967296 (4096 MB) avail memory = 3531640832 (3368 MB) : Trying to mount root from ufs:/dev/ad4s1a [rw]... Setting hostuuid: 00000000-0000-0000-0000-00218515337d. Setting hostid: 0x6b64ac17. Starting ddb. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: /dev/ad4s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1a: clean, 116861 free (1309 frags, 14444 blocks, 0.1% fragmentation) /dev/ad4s1f: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1f: clean, 143406 free (4766 frags, 17330 blocks, 0.5% fragmentation) /dev/label/tmp: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/label/tmp: clean, 116765187 free (2211 frags, 14595372 blocks, 0.0% fragmentation) /dev/ad4s1d: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1d: clean, 3946145 free (205905 frags, 467530 blocks, 2.0% fragmentation) /dev/ad4s1e: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1e: clean, 11247402 free (46186 frags, 1400152 blocks, 0.2% fragmentation) Mounting local file systems:. Setting hostname: x4.osted.lan. re0: link state changed to DOWN Starting Network: lo0 re0 fwe0 fwip0. lo0: flags=8049 metric 0 mtu 16384 options=3 nd6 options=21 re0: flags=8843 metric 0 mtu 1500 options=209b ether 00:21:85:15:33:7d nd6 options=29 media: Ethernet autoselect (none) status: no carrier fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb nd6 options=29 ch 1 dma -1 fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 nd6 options=29 Starting devd. Starting Network: usbus0. Starting Network: usbus1. Starting Network: usbus2. Starting Network: usbus3. Starting Network: usbus4. Starting Network: usbus5. Starting Network: fwe0. fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb nd6 options=29 ch 1 dma -1 Starting Network: fwip0. fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 nd6 options=29 add net default: gateway 192.168.1.1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 re0: link state changed to UP ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/kde4/lib /usr/local/lib/compat /usr/local/lib/gcc46 /usr/local/lib/qt4 /usr/local/lib/virtualbox a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout Creating and/or trimming log files. Starting syslogd. No core dumps found. Additional ABI support: linux. Starting rpcbind. NFS access cache time=60 rpc.umntall: localhost: MOUNTPROG: RPC: Program not registered lock order reversal: 1st 0xe191d4c0 bufwait (bufwait) @ kern/vfs_bio.c:2658 2nd 0xca790a00 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:284 KDB: stack backtrace: db_trace_self_wrapper(c0f92553,2e687361,38323a63,a34,f27d47e8,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a9642b,c0f96020,c1274fd0,11c,f27d4878,...) at kdb_backtrace+0x2a _witness_debugger(c0f96020,ca790a00,c0fc61ba,c7d79678,c0fc5e4c,...) at _witness_debugger+0x25 witness_checkorder(ca790a00,9,c0fc5e43,11c,0,...) at witness_checkorder+0x86f _sx_xlock(ca790a00,0,c0fc5e43,11c,ca7f089c,...) at _sx_xlock+0xa0 ufsdirhash_acquire(e191d460,ca7f089c,f27d49f4,e23ce4dc,f27d4948,...) at ufsdirhash_acquire+0x48 ufsdirhash_add(ca7f089c,f27d49f4,24dc,f27d4934,f27d4938,...) at ufsdirhash_add+0x13 ufs_direnter(caaa1ae0,cac73d98,f27d49f4,f27d4b84,e191dae0,...) at ufs_direnter+0x7a9 ufs_mkdir(f27d4c14,c0fe7425,0,0,f27d4bac,...) at ufs_mkdir+0xa6f VOP_MKDIR_APV(c10de2c0,f27d4c14,f27d4b84,f27d4bac,0,...) at VOP_MKDIR_APV+0xc5 kern_mkdirat(caaae5c0,ffffff9c,28404020,0,1c0,...) at kern_mkdirat+0x291 kern_mkdir(caaae5c0,28404020,0,1c0,f27d4d1c,...) at kern_mkdir+0x2e sys_mkdir(caaae5c0,f27d4cec,c0fe02aa,c0f96d2e,202,...) at sys_mkdir+0x29 syscall(f27d4d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x2817f743, esp = 0xbfbfe8cc, ebp = 0xbfbfed78 --- Clearing /tmp (X related). Starting mountd. Feb 22 13:32:21 x4 mountd[1831]: can't delete exports for /dev: Invalid argument mount option is unknown NFSv4 is disabled Starting nfsd. Recovering vi editor sessions:lock order reversal: 1st 0xcadac310 ufs (ufs) @ kern/vfs_lookup.c:542 2nd 0xe1924d00 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:260 3rd 0xcadac058 ufs (ufs) @ kern/vfs_subr.c:2167 KDB: stack backtrace: db_trace_self_wrapper(c0f92553,3a632e73,a303632,8058300,80583ba,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a9642b,c0f96039,c1274c98,877,f283e33c,...) at kdb_backtrace+0x2a _witness_debugger(c0f96039,cadac058,c0f84b17,c7d79610,c0f9dff5,...) at _witness_debugger+0x25 witness_checkorder(cadac058,9,c0f9dfec,877,0,...) at witness_checkorder+0x86f __lockmgr_args(cadac058,80100,cadac0c4,0,0,...) at __lockmgr_args+0x834 ffs_lock(f283e460,60,0,80100,cadac000,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c10de2c0,f283e460,c7d721e0,c10ef2e0,cadac000,...) at VOP_LOCK1_APV+0xb5 _vn_lock(cadac000,80100,c0f9dfec,877,4,...) at _vn_lock+0x78 vget(cadac000,80100,cad91000,50,0,...) at vget+0xbb vfs_hash_get(ca7652d4,15ee3d,80000,cad91000,f283e5a4,...) at vfs_hash_get+0xed ffs_vgetf(ca7652d4,15ee3d,80000,f283e5a4,1,...) at ffs_vgetf+0x49 softdep_sync_buf(cadac2b8,e1924ca0,1,106,0,...) at softdep_sync_buf+0xac9 ffs_syncvnode(cadac2b8,1,c1274c88,cadac3b0,c7d75370,...) at ffs_syncvnode+0x24c ffs_truncate(cadac2b8,200,0,880,c7dc5680,...) at ffs_truncate+0x827 ufs_direnter(cadac2b8,cadac000,f283e918,f283ebc0,0,...) at ufs_direnter+0x9a1 ufs_makeinode(f283ebc0,c10de840,f283eb04,f283ea60,c0de5425,...) at ufs_makeinode+0x65d ufs_create(f283eb04,c0fe75eb,0,0,f283eb80,...) at ufs_create+0x30 VOP_CREATE_APV(c10de2c0,f283eb04,f283ebc0,f283ea9c,0,...) at VOP_CREATE_APV+0xc5 vn_open_cred(f283eb80,f283ec48,1b0,0,c7dc5680,...) at vn_open_cred+0x205 vn_open(f283eb80,f283ec48,1b0,ca7ab310,4f44e00d,...) at vn_open+0x3b kern_openat(cad91000,ffffff9c,28839f60,0,a03,...) at kern_openat+0x1ec kern_open(cad91000,28839f60,0,a02,1b0,...) at kern_open+0x35 sys_open(cad91000,f283ecec,c0fe02aa,c0f96d2e,206,...) at sys_open+0x30 syscall(f283ed28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (5, FreeBSD ELF32, sys_open), eip = 0x283eada3, esp = 0xbfbfc50c, ebp = 0xbfbfc598 --- . Updating motd:. Starting ntpd. Configuring syscons: keymap blanktime. Starting sshd. Starting cron. Local package initialization: backuplock order reversal: 1st 0xca802df0 ufs (ufs) @ kern/vfs_mount.c:1249 2nd 0xca801058 devfs (devfs) @ ufs/ffs/ffs_softdep.c:1835 KDB: stack backtrace: db_trace_self_wrapper(c0f92553,38313a63,a3533,c0aa8a1c,8,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a9642b,c0f96020,c12748c0,72b,f27179a0,...) at kdb_backtrace+0x2a _witness_debugger(c0f96020,ca801058,c0f7ca47,c7d79470,c0fc22c6,...) at _witness_debugger+0x25 witness_checkorder(ca801058,9,c0fc22bd,72b,ca8010c4,...) at witness_checkorder+0x86f __lockmgr_args(ca801058,80400,ca8010c4,0,0,...) at __lockmgr_args+0x834 vop_stdlock(f2717ac0,4,c0f8c92f,80400,ca801000,...) at vop_stdlock+0x65 VOP_LOCK1_APV(c10b3e20,f2717ac0,c12a8500,c10ef2e0,ca801000,...) at VOP_LOCK1_APV+0xb5 _vn_lock(ca801000,80400,c0fc22bd,72b,ca76587c,...) at _vn_lock+0x78 softdep_flushworklist(ca76587c,f2717b6c,ca7a08a0,5e9,0,...) at softdep_flushworklist+0x47 ffs_sync(ca76587c,1,f2717bc0,51a,c7c31000,...) at ffs_sync+0x32d dounmount(ca76587c,8080000,ca7a08a0,49c,0,...) at dounmount+0x3f8 sys_unmount(ca7a08a0,f2717cec,c,c,206,...) at sys_unmount+0x421 syscall(f2717d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280e7e0b, esp = 0xbfbfe64c, ebp = 0xbfbfe718 --- ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes lock order reversal: 1st 0xcaaa1724 syncer (syncer) @ kern/vfs_subr.c:1767 2nd 0xca7fac94 devfs (devfs) @ ufs/ffs/ffs_vfsops.c:1536 KDB: stack backtrace: db_trace_self_wrapper(c0f92553,3335313a,a36,0,0,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a9642b,c0f96020,c12753c0,600,ee31aa5c,...) at kdb_backtrace+0x2a _witness_debugger(c0f96020,ca7fac94,c0f7ca47,c7d79470,c0fc5199,...) at _witness_debugger+0x25 witness_checkorder(ca7fac94,9,c0fc5190,600,ca7fad00,...) at witness_checkorder+0x86f __lockmgr_args(ca7fac94,80400,ca7fad00,0,0,...) at __lockmgr_args+0x834 vop_stdlock(ee31ab7c,c0aa65ea,c12753d0,80400,ca7fac3c,...) at vop_stdlock+0x65 VOP_LOCK1_APV(c10b3e20,ee31ab7c,ee31ab9c,c10ef2e0,ca7fac3c,...) at VOP_LOCK1_APV+0xb5 _vn_lock(ca7fac3c,80400,c0fc5190,600,c0f8c92f,...) at _vn_lock+0x78 ffs_sync(ca7655a8,3,2,dd9,ca7655a8,...) at ffs_sync+0x3f7 sync_fsync(ee31ac6c,c0fe6f83,caaa1790,ee31ac6c,caaa16cc,...) at sync_fsync+0x156 VOP_FSYNC_APV(c10ccc20,ee31ac6c,c0f9dfec,6e7,c805eb80,...) at VOP_FSYNC_APV+0xc5 sync_vnode(c12a2234,c12a2220,3e8,78b,4e20,...) at sync_vnode+0x16b sched_sync(0,ee31ad28,c0f88c10,3db,ca6c0588,...) at sched_sync+0x282 fork_exit(c0afcd00,0,ee31ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee31ad60, ebp = 0 --- ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 75777 files, 1548794 used, 116765187 free (2211 frags, 14595372 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 75777 files, 1548794 used, 116765187 free (2211 frags, 14595372 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** fsck -y /tmpFeb 22 13:35:35 x4 mountd[1832]: can't delete exports for /dev: Invalid argument mount option is unknown watchdogd. Starting default moused. Starting inetd. Wed Feb 22 13:35:35 CET 2012 FreeBSD/i386 (x4.osted.lan) (console) login: Feb 22 13:36:31 x4 su: pho to root on /dev/pts/0 Feb 22 13:37:01 x4 su: pho to root on /dev/pts/1 Feb 22 13:37:28 x4 mountd[1832]: can't delete exports for /dev: Invalid argument mount option is unknown Feb 22 13:37:28 x4 mountd[1832]: bad exports list line /mnt Feb 22 13:37:28 x4 mountd[1832]: can't delete exports for /dev: Invalid argument mount option is unknown Feb 22 13:37:28 x4 mountd[183KDB: stack backtrace: 2]: bad exports dlist line /mnt b_trace_self_wrapper(c0f92553,113,c7d77c10,f2897988,c0aa7242,...) at db_trace_self_wrapper+0x26 kdb_backtrace(cb5ad310,f28979c4,c0de09e5,f28979d4,cb5ad414,...) at kdb_backtrace+0x2a vfs_badlock(cb616680,f28979d4,c10eef20,cb5ad414) at vfs_badlock+0x23 assert_vop_elocked(cb5ad414,c0f9ec8f,cb5ad414,0,0,...) at assert_vop_elocked+0x55 insmntque1(cb5ad414,cb1cdb50,cb613b00,ca00a7d0,ca00a7d0,...) at insmntque1+0x1a0 null_nodeget(cb1cdb50,cb5ad2b8,f2897a70,8000,cb5c5828,...) at null_nodeget+0x19d null_lookup(f2897afc,c0f9ecf4,c10eef20,200000,f2897bac,...) at null_lookup+0xc3 VOP_LOOKUP_APV(cb616680,f2897afc,f2897bec,343,f2897ad4,...) at VOP_LOOKUP_APV+0xe5 lookup(f2897bac,c0f9d3fc,10e,dc,3,...) at lookup+0x6eb namei(f2897bac,f2897c04,cb1ed000,f2897bcc,bfbfeab2,...) at namei+0x6bf kern_utimesat(cb3c9000,ffffff9c,bfbfeab2,0,bfbfe884,...) at kern_utimesat+0x92 kern_utimes(cb3c9000,bfbfeab2,0,bfbfe884,0,...) at kern_utimes+0x35 sys_utimes(cb3c9000,f2897cec,c0fe02aa,c0f96d2e,246,...) at sys_utimes+0x31 syscall(f2897d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (138, FreeBSD ELF32, sys_utimes), eip = 0x280e69ab, esp = 0xbfbfe7cc, ebp = 0xbfbfe8a8 --- insmntque: mp-safe fs and non-locked vp: 0xcb5ad414 is not exclusive locked but should be KDB: enter: lock violation [ thread pid 2289 tid 100168 ] Stopped at kdb_enter+0x3a: movl $0,kdb_why db> run pho db:0:pho> bt Tracing pid 2289 tid 100168 td 0xcb3c9000 kdb_enter(c0f9e1c6,c0f9f22f,cb5ad414,c0f9ebab,cb5ad414,...) at kdb_enter+0x3a vfs_badlock(cb616680,f28979d4,c10eef20,cb5ad414) at vfs_badlock+0x61 assert_vop_elocked(cb5ad414,c0f9ec8f,cb5ad414,0,0,...) at assert_vop_elocked+0x55 insmntque1(cb5ad414,cb1cdb50,cb613b00,ca00a7d0,ca00a7d0,...) at insmntque1+0x1a0 null_nodeget(cb1cdb50,cb5ad2b8,f2897a70,8000,cb5c5828,...) at null_nodeget+0x19d null_lookup(f2897afc,c0f9ecf4,c10eef20,200000,f2897bac,...) at null_lookup+0xc3 VOP_LOOKUP_APV(cb616680,f2897afc,f2897bec,343,f2897ad4,...) at VOP_LOOKUP_APV+0xe5 lookup(f2897bac,c0f9d3fc,10e,dc,3,...) at lookup+0x6eb namei(f2897bac,f2897c04,cb1ed000,f2897bcc,bfbfeab2,...) at namei+0x6bf kern_utimesat(cb3c9000,ffffff9c,bfbfeab2,0,bfbfe884,...) at kern_utimesat+0x92 kern_utimes(cb3c9000,bfbfeab2,0,bfbfe884,0,...) at kern_utimes+0x35 sys_utimes(cb3c9000,f2897cec,c0fe02aa,c0f96d2e,246,...) at sys_utimes+0x31 syscall(f2897d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (138, FreeBSD ELF32, sys_utimes), eip = 0x280e69ab, esp = 0xbfbfe7cc, ebp = 0xbfbfe8a8 --- db:0:bt> show allpcpu Current CPU: 0 cpuid = 0 dynamic pcpu = 0x735500 curthread = 0xcb3c9000: pid 2289 "touch" curpcb = 0xf2897d80 fpcurthread = none idlethread = 0xc7dc98a0: tid 100003 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: cpuid = 1 dynamic pcpu = 0x6a26500 curthread = 0xc7dc95c0: pid 11 "idle: cpu1" curpcb = 0xc7b29d80 fpcurthread = none idlethread = 0xc7dc95c0: tid 100004 "idle: cpu1" APIC ID = 1 currentldt = 0x50 spin locks held: cpuid = 2 dynamic pcpu = 0x6a29500 curthread = 0xc7dc92e0: pid 11 "idle: cpu2" curpcb = 0xc7b2cd80 fpcurthread = none idlethread = 0xc7dc92e0: tid 100005 "idle: cpu2" APIC ID = 2 currentldt = 0x50 spin locks held: cpuid = 3 dynamic pcpu = 0x6a2c500 curthread = 0xc7dc9000: pid 11 "idle: cpu3" curpcb = 0xc7b2fd80 fpcurthread = none idlethread = 0xc7dc9000: tid 100006 "idle: cpu3" APIC ID = 3 currentldt = 0x50 spin locks held: db:0:allpcpu> show alllocks Process 2289 (touch) thread 0xcb3c9000 (100168) shared lockmgr ufs (ufs) r = 0 (0xcb5ad310) locked @ kern/vfs_subr.c:2167 shared lockmgr ufs (ufs) r = 0 (0xca475310) locked @ /usr/src/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:595 db:0:alllocks> show lockedvnods Locked vnodes 0xca4752b8: tag ufs, type VDIR usecount 1, writecount 0, refcount 4 mountedhere 0 flags () v_object 0xc1b86990 ref 0 pages 1 lock type ufs: SHARED (count 1) #0 0xc0a43664 at __lockmgr_args+0x5c4 #1 0xc0ca6461 at ffs_lock+0xa1 #2 0xc0de4455 at VOP_LOCK1_APV+0xb5 #3 0xcb614e6f at null_lock+0xcf #4 0xc0de4455 at VOP_LOCK1_APV+0xb5 #5 0xc0b09f08 at _vn_lock+0x78 #6 0xcb613e65 at nullfs_root+0x55 #7 0xc0aedb61 at lookup+0xa21 #8 0xc0aee84f at namei+0x6bf #9 0xc0b04582 at kern_utimesat+0x92 #10 0xc0b04655 at kern_utimes+0x35 #11 0xc0b04691 at sys_utimes+0x31 #12 0xc0dc10f7 at syscall+0x2c7 #13 0xc0daa421 at Xint0x80_syscall+0x21 ino 70656, on dev ada0s1a 0xcb5ad2b8: tag ufs, type VREG usecount 1, writecount 0, refcount 1 mountedhere 0 flags () lock type ufs: SHARED (count 1) #0 0xc0a43664 at __lockmgr_args+0x5c4 #1 0xc0ca6461 at ffs_lock+0xa1 #2 0xc0de4455 at VOP_LOCK1_APV+0xb5 #3 0xc0b09f08 at _vn_lock+0x78 #4 0xc0afc9bb at vget+0xbb #5 0xc0ae5665 at cache_lookup+0x6c5 #6 0xc0ae584d at vfs_cache_lookup+0xbd #7 0xc0de55d5 at VOP_LOOKUP_APV+0xe5 #8 0xcb614d04 at null_lookup+0x94 #9 0xc0de55d5 at VOP_LOOKUP_APV+0xe5 #10 0xc0aed82b at lookup+0x6eb #11 0xc0aee84f at namei+0x6bf #12 0xc0b04582 at kern_utimesat+0x92 #13 0xc0b04655 at kern_utimes+0x35 #14 0xc0b04691 at sys_utimes+0x31 #15 0xc0dc10f7 at syscall+0x2c7 #16 0xc0daa421 at Xint0x80_syscall+0x21 ino 70751, on dev ada0s1a 0xcb5c5828: tag null, type VDIR usecount 2, writecount 0, refcount 2 mountedhere 0 flags (VV_ROOT) lock type ufs: SHARED (count 1) #0 0xc0a43664 at __lockmgr_args+0x5c4 #1 0xc0ca6461 at ffs_lock+0xa1 #2 0xc0de4455 at VOP_LOCK1_APV+0xb5 #3 0xcb614e6f at null_lock+0xcf #4 0xc0de4455 at VOP_LOCK1_APV+0xb5 #5 0xc0b09f08 at _vn_lock+0x78 #6 0xcb613e65 at nullfs_root+0x55 #7 0xc0aedb61 at lookup+0xa21 #8 0xc0aee84f at namei+0x6bf #9 0xc0b04582 at kern_utimesat+0x92 #10 0xc0b04655 at kern_utimes+0x35 #11 0xc0b04691 at sys_utimes+0x31 #12 0xc0dc10f7 at syscall+0x2c7 #13 0xc0daa421 at Xint0x80_syscall+0x21 vp=0xcb5c5828, lowervp=0xca4752b8 db:0:lockedvnods> show mount 0xca765b50 /dev/ad4s1a on / (ufs) 0xca766000 devfs on /dev (devfs) 0xcaaa35a8 /dev/ad4s1f on /home (ufs) 0xca7655a8 /dev/ad4s1d on /usr (ufs) 0xca7652d4 /dev/ad4s1e on /var (ufs) 0xca765000 procfs on /proc (procfs) 0xcb1ce000 /dev/label/tmp on /tmp (ufs) 0xcb60cb50 127.0.0.1:/tmp on /mnt (nfs) 0xcb1cdb50 /bin on /mnt2 (nullfs) More info: show mount db:0:mount> ps pid ppid pgrp uid state wmesg wchan cmd 2289 2274 2274 0 R+ CPU 0 touch 2274 2250 2274 0 S+ wait 0xcb3dc000 sh 2268 2264 2268 0 S+ ttyin 0xcb33d670 bash 2264 2263 2264 0 S+ pause 0xcb1eb5e0 csh 2263 2259 2263 1001 S+ wait 0xcaadbb10 su 2259 2258 2259 1001 Ss+ wait 0xcb1eb2c4 bash 2258 2256 2256 1001 S select 0xca764924 sshd 2256 2045 2256 0 Ss select 0xcaf5e8a4 sshd 2250 2246 2250 0 S+ wait 0xca79ab10 bash 2246 2245 2246 0 S+ pause 0xcad8c8a4 csh 2245 2121 2245 1001 S+ wait 0xcabcb588 su 2234 1 2234 0 Ss+ ttyin 0xc7d97470 getty 2233 1 2233 0 Ss+ ttyin 0xc7d97670 getty 2232 1 2232 0 Ss+ ttyin 0xc7d97870 getty 2231 1 2231 0 Ss+ ttyin 0xc7d97a70 getty 2230 1 2230 0 Ss+ ttyin 0xc7d97c70 getty 2229 1 2229 0 Ss+ ttyin 0xc7d97e70 getty 2228 1 2228 0 Ss+ ttyin 0xc8181070 getty 2227 1 2227 0 Ss+ ttyin 0xc8181270 getty 2226 1 2226 0 Ss+ ttyin 0xc8181470 getty 2187 1 2187 0 Ss select 0xcaf5ede4 inetd 2154 1 2154 0 Ss select 0xcaf5e1e4 moused 2133 1 2133 0 Ss nanslp 0xc112c064 watchdogd 2121 2120 2121 1001 Ss+ wait 0xca6c184c bash 2120 2118 2118 1001 S select 0xcaf5e724 sshd 2118 2045 2118 0 Ss select 0xca7648e4 sshd 2065 1 2065 0 Ss nanslp 0xc112c064 cron 2057 1 2057 25 Ss pause 0xcad8cb68 sendmail 2053 1 2053 0 Ss select 0xca763864 sendmail 2045 1 2045 0 Ss select 0xca6b4524 sshd 1947 1 1947 0 Ss select 0xcac4cd24 ntpd 1848 1842 1842 0 S (threaded) nfsd 100140 S rpcsvc 0xcaa9ba90 nfsd: service 100139 S rpcsvc 0xcaa9bad0 nfsd: service 100138 S rpcsvc 0xcaa9bb10 nfsd: service 100096 S rpcsvc 0xca7552d0 nfsd: master 1842 1 1842 0 Ss select 0xca7644e4 nfsd 1832 1 1832 0 Ss select 0xca6b4564 mountd 1732 1 1732 0 Ss select 0xca763d64 rpcbind 1704 1 1704 0 Ss select 0xca7642a4 syslogd 1497 1 1497 0 Ss select 0xca690ea4 devd 19 0 0 0 DL sdflush 0xc12a8540 [softdepflush] 18 0 0 0 DL vlruwt 0xca6c02c4 [vnlru] 17 0 0 0 DL syncer 0xc12a2234 [syncer] 16 0 0 0 DL psleep 0xc12a1ec8 [bufdaemon] 9 0 0 0 DL pgzero 0xc12aa3dc [pagezero] 8 0 0 0 DL psleep 0xc12a9fc8 [vmdaemon] 7 0 0 0 DL psleep 0xc12a9f90 [pagedaemon] 6 0 0 0 DL ccb_scan 0xc10f0854 [xpt_thrd] 5 0 0 0 DL waiting_ 0xc12a3d58 [sctp_iterator] 4 0 0 0 DL ctl_work 0xc819c000 [ctl_thrd] 3 0 0 0 DL - 0xc7ffa03c [fdc0] 2 0 0 0 SL - 0xc808f000 [fw0_probe] 15 0 0 0 DL (threaded) [usb] 100058 D - 0xc8063d34 [usbus5] 100057 D - 0xc8063d04 [usbus5] 100056 D - 0xc8063cd4 [usbus5] 100055 D - 0xc8063ca4 [usbus5] 100053 D - 0xc8057b5c [usbus4] 100052 D - 0xc8057b2c [usbus4] 100051 D - 0xc8057afc [usbus4] 100050 D - 0xc8057acc [usbus4] 100049 D - 0xc8052b5c [usbus3] 100048 D - 0xc8052b2c [usbus3] 100047 D - 0xc8052afc [usbus3] 100046 D - 0xc8052acc [usbus3] 100045 D - 0xc804ab5c [usbus2] 100044 D - 0xc804ab2c [usbus2] 100043 D - 0xc804aafc [usbus2] 100042 D - 0xc804aacc [usbus2] 100040 D - 0xc8044b5c [usbus1] 100039 D - 0xc8044b2c [usbus1] 100038 D - 0xc8044afc [usbus1] 100037 D - 0xc8044acc [usbus1] 100035 D - 0xc8039b5c [usbus0] 100034 D - 0xc8039b2c [usbus0] 100033 D - 0xc8039afc [usbus0] 100032 D - 0xc8039acc [usbus0] 14 0 0 0 DL - 0xc112ae64 [yarrow] 13 0 0 0 DL (threaded) [geom] 100015 D - 0xc1128a68 [g_down] 100014 D - 0xc1128a64 [g_up] 100013 D - 0xc1128a5c [g_event] 12 0 0 0 RL (threaded) [intr] 100066 I [irq12: psm0] 100065 I [irq1: atkbd0] 100063 I [swi0: uart] 100060 I [irq20: fwohci0] 100059 I [irq14: ata0] 100054 I [irq19: ehci0] 100041 I [irq18: ohci2 ohci4] 100036 I [irq17: ohci1 ohci3] 100031 I [irq16: hdac1 ohci0] 100030 I [irq22: ahci0] 100029 I [irq257: re0] 100028 I [irq256: hdac0] 100027 I [swi6: task queue] 100026 I [swi6: Giant taskq] 100024 I [swi5: +] 100023 I [swi2: cambio] 100012 I [swi4: clock] 100011 I [swi4: clock] 100010 I [swi4: clock] 100009 RunQ [swi4: clock] 100008 I [swi3: vm] 100007 I [swi1: netisr 0] 11 0 0 0 RL (threaded) [idle] 100006 Run CPU 3 [idle: cpu3] 100005 Run CPU 2 [idle: cpu2] 100004 Run CPU 1 [idle: cpu1] 100003 CanRun [idle: cpu0] 1 0 1 0 SLs wait 0xc7dc7b10 [init] 10 0 0 0 DL audit_wo 0xc12a7c80 [audit] 0 0 0 0 DLs (threaded) [kernel] 100078 D - 0xc8036380 [mca taskq] 100068 D - 0xc112ae64 [deadlkres] 100061 D - 0xc80366c0 [fw0_taskq] 100025 D - 0xc7f9ee80 [thread taskq] 100022 D - 0xc7f9f040 [acpi_task_2] 100021 D - 0xc7f9f040 [acpi_task_1] 100020 D - 0xc7f9f040 [acpi_task_0] 100019 D - 0xc7f9f080 [ffs_trim taskq] 100018 D - 0xc7f9f0c0 [kqueue taskq] 100016 D - 0xc7dade80 [firmware taskq] 100000 D sched 0xc1128b40 [swapper] db:0:ps> allt Tracing command touch pid 2289 tid 100168 td 0xcb3c9000 kdb_enter(c0f9e1c6,c0f9f22f,cb5ad414,c0f9ebab,cb5ad414,...) at kdb_enter+0x3a vfs_badlock(cb616680,f28979d4,c10eef20,cb5ad414) at vfs_badlock+0x61 assert_vop_elocked(cb5ad414,c0f9ec8f,cb5ad414,0,0,...) at assert_vop_elocked+0x55 insmntque1(cb5ad414,cb1cdb50,cb613b00,ca00a7d0,ca00a7d0,...) at insmntque1+0x1a0 null_nodeget(cb1cdb50,cb5ad2b8,f2897a70,8000,cb5c5828,...) at null_nodeget+0x19d null_lookup(f2897afc,c0f9ecf4,c10eef20,200000,f2897bac,...) at null_lookup+0xc3 VOP_LOOKUP_APV(cb616680,f2897afc,f2897bec,343,f2897ad4,...) at VOP_LOOKUP_APV+0xe5 lookup(f2897bac,c0f9d3fc,10e,dc,3,...) at lookup+0x6eb namei(f2897bac,f2897c04,cb1ed000,f2897bcc,bfbfeab2,...) at namei+0x6bf kern_utimesat(cb3c9000,ffffff9c,bfbfeab2,0,bfbfe884,...) at kern_utimesat+0x92 kern_utimes(cb3c9000,bfbfeab2,0,bfbfe884,0,...) at kern_utimes+0x35 sys_utimes(cb3c9000,f2897cec,c0fe02aa,c0f96d2e,246,...) at sys_utimes+0x31 syscall(f2897d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (138, FreeBSD ELF32, sys_utimes), eip = 0x280e69ab, esp = 0xbfbfe7cc, ebp = 0xbfbfe8a8 --- Tracing command sh pid 2274 tid 100173 td 0xcb3f5000 sched_switch(cb3f5000,0,104,1a3,4337cc41,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(cb3f5000,0,c0f93f40,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93f40,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cb3dc000,6c,c0f96e87,100,0,...) at sleepq_wait_sig+0x17 _sleep(cb3dc000,cb3dc088,16c,c0f96e87,0,...) at _sleep+0x385 kern_wait(cb3f5000,ffffffff,f28a6c60,2,0,...) at kern_wait+0x626 sys_wait4(cb3f5000,f28a6cec,c0fe02aa,c0f96e66,216,...) at sys_wait4+0x3b syscall(f28a6d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2817645b, esp = 0xbfbfe82c, ebp = 0xbfbfe848 --- Tracing command bash pid 2268 tid 100115 td 0xcaaaf5c0 sched_switch(caaaf5c0,0,104,1a3,a554bd45,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(caaaf5c0,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4bc5a,cb33d604,0,c0f8c92f,caaaf5c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cb33d670,0,f27c5af0,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(cb33d670,cb33d604,c0f98780,527,0,...) at _cv_wait_sig+0x260 tty_wait(cb33d600,cb33d670,f27c5c44,1,0,...) at tty_wait+0x71 ttydisc_read(cb33d600,f27c5c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(cb340100,f27c5c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(cac41e00,f27c5c44,ca815a80,0,caaaf5c0,...) at devfs_read_f+0x7e dofileread(f27c5c44,ffffffff,ffffffff,0,f27c5c20,...) at dofileread+0x9e kern_readv(caaaf5c0,0,f27c5c44,f27c5c64,1,...) at kern_readv+0x68 sys_read(caaaf5c0,f27c5cec,c0fe02aa,c0f96d76,246,...) at sys_read+0x4f syscall(f27c5d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x28368333, esp = 0xbfbfddec, ebp = 0xbfbfde18 --- Tracing command csh pid 2264 tid 100181 td 0xcb3c75c0 sched_switch(cb3c75c0,0,104,1a3,37e5789,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,74,...) at mi_switch+0x1ff sleepq_switch(cb3c75c0,0,c0f93f40,1a7,74,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93f40,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cb1eb5e0,74,c0e1556e,100,0,...) at sleepq_wait_sig+0x17 _sleep(cb1eb5e0,cb1eb610,174,c0e1556e,0,...) at _sleep+0x385 kern_sigsuspend(cb3c75c0,0,0,0,0,...) at kern_sigsuspend+0xba sys_sigsuspend(cb3c75c0,f28becec,c0fe02aa,c0f96d76,202,...) at sys_sigsuspend+0x4d syscall(f28bed28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28195ccb, esp = 0xbfbfe5bc, ebp = 0xbfbfe628 --- Tracing command su pid 2263 tid 100106 td 0xcaae42e0 sched_switch(caae42e0,0,104,1a3,b2bbeff0,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(caae42e0,0,c0f93f40,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93f40,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(caadbb10,6c,c0f96e87,100,0,...) at sleepq_wait_sig+0x17 _sleep(caadbb10,caadbb98,16c,c0f96e87,0,...) at _sleep+0x385 kern_wait(caae42e0,8d8,f27a5c60,2,0,...) at kern_wait+0x626 sys_wait4(caae42e0,f27a5cec,c,c,216,...) at sys_wait4+0x3b syscall(f27a5d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2813245b, esp = 0xbfbfe5ec, ebp = 0xbfbfe608 --- Tracing command bash pid 2259 tid 100182 td 0xcb3c72e0 sched_switch(cb3c72e0,0,104,1a3,8fd404d7,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(cb3c72e0,0,c0f93f40,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93f40,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cb1eb2c4,6c,c0f96e87,100,0,...) at sleepq_wait_sig+0x17 _sleep(cb1eb2c4,cb1eb34c,16c,c0f96e87,0,...) at _sleep+0x385 kern_wait(cb3c72e0,ffffffff,f28c1c60,6,0,...) at kern_wait+0x626 sys_wait4(cb3c72e0,f28c1cec,c0fe02aa,c0f96d76,216,...) at sys_wait4+0x3b syscall(f28c1d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x282d545b, esp = 0xbfbfe99c, ebp = 0xbfbfe9b8 --- Tracing command sshd pid 2258 tid 100160 td 0xc81858a0 sched_switch(c81858a0,0,104,1a3,a5565662,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c81858a0,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4bc5a,ca764910,0,c0f8c92f,c81858a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca764924,0,f287fa58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca764924,ca764910,c0f96690,6b2,cb3f2038,...) at _cv_wait_sig+0x260 seltdwait(cb3f2038,58,cad7c980,c81858a0,f287facc,...) at seltdwait+0xa2 kern_select(c81858a0,a,288060b8,288060c0,0,0,20,cb3de2c4,0) at kern_select+0x655 sys_select(c81858a0,f287fcec,c0fe02aa,c0f96fdb,206,...) at sys_select+0x66 syscall(f287fd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284d72b3, esp = 0xbfbfde2c, ebp = 0xbfbfde78 --- Tracing command sshd pid 2256 tid 100180 td 0xcb3c78a0 sched_switch(cb3c78a0,0,104,1a3,1907071e,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cb3c78a0,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4bc5a,caf5e890,0,c0f8c92f,cb3c78a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(caf5e8a4,0,f28bbabc,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(caf5e8a4,caf5e890,c0f96690,6b2,f28bbb44,...) at _cv_wait_sig+0x260 seltdwait(caaf2e2c,c0f96690,5af,cb3c78a0,8,...) at seltdwait+0xa2 sys_poll(cb3c78a0,f28bbcec,c0fe02aa,c0f7efb9,202,...) at sys_poll+0x321 syscall(f28bbd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x2847014b, esp = 0xbfbfde8c, ebp = 0xbfbfdec8 --- Tracing command bash pid 2250 tid 100079 td 0xca7a1b80 sched_switch(ca7a1b80,0,104,1a3,393da543,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(ca7a1b80,0,c0f93f40,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93f40,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca79ab10,6c,c0f96e87,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca79ab10,ca79ab98,16c,c0f96e87,0,...) at _sleep+0x385 kern_wait(ca7a1b80,ffffffff,f26ffc60,6,0,...) at kern_wait+0x626 sys_wait4(ca7a1b80,f26ffcec,c0fe02aa,c0f96d76,216,...) at sys_wait4+0x3b syscall(f26ffd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x282d545b, esp = 0xbfbfe79c, ebp = 0xbfbfe7b8 --- Tracing command csh pid 2246 tid 100145 td 0xcad91000 sched_switch(cad91000,0,104,1a3,931cd332,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,74,...) at mi_switch+0x1ff sleepq_switch(cad91000,0,c0f93f40,1a7,74,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93f40,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cad8c8a4,74,c0e1556e,100,0,...) at sleepq_wait_sig+0x17 _sleep(cad8c8a4,cad8c8d4,174,c0e1556e,0,...) at _sleep+0x385 kern_sigsuspend(cad91000,0,0,0,0,...) at kern_sigsuspend+0xba sys_sigsuspend(cad91000,f283ecec,c0fe02aa,c0f96d76,202,...) at sys_sigsuspend+0x4d syscall(f283ed28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28195ccb, esp = 0xbfbfe59c, ebp = 0xbfbfe608 --- Tracing command su pid 2245 tid 100116 td 0xcaaaf2e0 sched_switch(caaaf2e0,0,104,1a3,441afc95,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(caaaf2e0,0,c0f93f40,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93f40,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cabcb588,6c,c0f96e87,100,0,...) at sleepq_wait_sig+0x17 _sleep(cabcb588,cabcb610,16c,c0f96e87,0,...) at _sleep+0x385 kern_wait(caaaf2e0,8c6,f27c8c60,2,0,...) at kern_wait+0x626 sys_wait4(caaaf2e0,f27c8cec,c,c,216,...) at sys_wait4+0x3b syscall(f27c8d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2813245b, esp = 0xbfbfe5cc, ebp = 0xbfbfe5e8 --- Tracing command getty pid 2234 tid 100162 td 0xc81852e0 sched_switch(c81852e0,0,104,1a3,f952db60,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c81852e0,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93f40,160,0,c81852e0,c81852e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7d97470,0,c0f98da8,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7d97470,c112a710,c0f98780,527,0,...) at _cv_wait_sig+0x260 tty_wait(c7d97400,c7d97470,f2885c44,1,0,...) at tty_wait+0x71 ttydisc_read(c7d97400,f2885c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c8193400,f2885c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(ca7acab8,f2885c44,c7dc4e80,0,c81852e0,...) at devfs_read_f+0x7e dofileread(f2885c44,ffffffff,ffffffff,0,f2885c20,...) at dofileread+0x9e kern_readv(c81852e0,0,f2885c44,f2885c64,1,...) at kern_readv+0x68 sys_read(c81852e0,f2885cec,c0fe02aa,c0df39b6,246,...) at sys_read+0x4f syscall(f2885d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a6333, esp = 0xbfbfed7c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2233 tid 100163 td 0xcb3f68a0 sched_switch(cb3f68a0,0,104,1a3,f94ddf9f,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cb3f68a0,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93f40,160,0,cb3f68a0,cb3f68a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7d97670,0,c0f98da8,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7d97670,c112a710,c0f98780,527,0,...) at _cv_wait_sig+0x260 tty_wait(c7d97600,c7d97670,f2888c44,1,0,...) at tty_wait+0x71 ttydisc_read(c7d97600,f2888c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c8193500,f2888c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(ca7fdd58,f2888c44,c7dc4e80,0,cb3f68a0,...) at devfs_read_f+0x7e dofileread(f2888c44,ffffffff,ffffffff,0,f2888c20,...) at dofileread+0x9e kern_readv(cb3f68a0,0,f2888c44,f2888c64,1,...) at kern_readv+0x68 sys_read(cb3f68a0,f2888cec,c0fe02aa,c0df39b6,246,...) at sys_read+0x4f syscall(f2888d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a6333, esp = 0xbfbfed7c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2232 tid 100164 td 0xcb3f65c0 sched_switch(cb3f65c0,0,104,1a3,f9588abc,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cb3f65c0,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93f40,160,0,cb3f65c0,cb3f65c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7d97870,0,c0f98da8,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7d97870,c112a710,c0f98780,527,0,...) at _cv_wait_sig+0x260 tty_wait(c7d97800,c7d97870,f288bc44,1,0,...) at tty_wait+0x71 ttydisc_read(c7d97800,f288bc44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c8193600,f288bc44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(ca7d4700,f288bc44,c7dc4e80,0,cb3f65c0,...) at devfs_read_f+0x7e dofileread(f288bc44,ffffffff,ffffffff,0,f288bc20,...) at dofileread+0x9e kern_readv(cb3f65c0,0,f288bc44,f288bc64,1,...) at kern_readv+0x68 sys_read(cb3f65c0,f288bcec,c0fe02aa,c0df39b6,246,...) at sys_read+0x4f syscall(f288bd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a6333, esp = 0xbfbfed7c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2231 tid 100165 td 0xcb3f62e0 sched_switch(cb3f62e0,0,104,1a3,f967a915,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cb3f62e0,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93f40,160,0,cb3f62e0,cb3f62e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7d97a70,0,c0f98da8,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7d97a70,c112a710,c0f98780,527,0,...) at _cv_wait_sig+0x260 tty_wait(c7d97a00,c7d97a70,f288ec44,1,0,...) at tty_wait+0x71 ttydisc_read(c7d97a00,f288ec44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c8075500,f288ec44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(ca7ab2d8,f288ec44,c7dc4e80,0,cb3f62e0,...) at devfs_read_f+0x7e dofileread(f288ec44,ffffffff,ffffffff,0,f288ec20,...) at dofileread+0x9e kern_readv(cb3f62e0,0,f288ec44,f288ec64,1,...) at kern_readv+0x68 sys_read(cb3f62e0,f288ecec,c0fe02aa,c0df39b6,246,...) at sys_read+0x4f syscall(f288ed28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a6333, esp = 0xbfbfed7c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2230 tid 100166 td 0xcb3f6000 sched_switch(cb3f6000,0,104,1a3,f96b2e1c,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cb3f6000,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93f40,160,0,cb3f6000,cb3f6000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7d97c70,0,c0f98da8,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7d97c70,c112a710,c0f98780,527,0,...) at _cv_wait_sig+0x260 tty_wait(c7d97c00,c7d97c70,f2891c44,1,0,...) at tty_wait+0x71 ttydisc_read(c7d97c00,f2891c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c8075600,f2891c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(cabcfc40,f2891c44,c7dc4e80,0,cb3f6000,...) at devfs_read_f+0x7e dofileread(f2891c44,ffffffff,ffffffff,0,f2891c20,...) at dofileread+0x9e kern_readv(cb3f6000,0,f2891c44,f2891c64,1,...) at kern_readv+0x68 sys_read(cb3f6000,f2891cec,c0fe02aa,c0df39b6,246,...) at sys_read+0x4f syscall(f2891d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a6333, esp = 0xbfbfed7c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2229 tid 100169 td 0xcb3f5b80 sched_switch(cb3f5b80,0,104,1a3,f96d1430,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cb3f5b80,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93f40,160,0,cb3f5b80,cb3f5b80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7d97e70,0,c0f98da8,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7d97e70,c112a710,c0f98780,527,0,...) at _cv_wait_sig+0x260 tty_wait(c7d97e00,c7d97e70,f289ac44,1,0,...) at tty_wait+0x71 ttydisc_read(c7d97e00,f289ac44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c8075700,f289ac44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(cabcf5b0,f289ac44,c7dc4e80,0,cb3f5b80,...) at devfs_read_f+0x7e dofileread(f289ac44,ffffffff,ffffffff,0,f289ac20,...) at dofileread+0x9e kern_readv(cb3f5b80,0,f289ac44,f289ac64,1,...) at kern_readv+0x68 sys_read(cb3f5b80,f289acec,c0fe02aa,c0df39b6,246,...) at sys_read+0x4f syscall(f289ad28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a6333, esp = 0xbfbfed7c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2228 tid 100170 td 0xcb3f58a0 sched_switch(cb3f58a0,0,104,1a3,f9749316,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cb3f58a0,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93f40,160,0,cb3f58a0,cb3f58a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8181070,0,c0f98da8,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c8181070,c112a710,c0f98780,527,0,...) at _cv_wait_sig+0x260 tty_wait(c8181000,c8181070,f289dc44,1,0,...) at tty_wait+0x71 ttydisc_read(c8181000,f289dc44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c8075800,f289dc44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(ca7d4690,f289dc44,c7dc4e80,0,cb3f58a0,...) at devfs_read_f+0x7e dofileread(f289dc44,ffffffff,ffffffff,0,f289dc20,...) at dofileread+0x9e kern_readv(cb3f58a0,0,f289dc44,f289dc64,1,...) at kern_readv+0x68 sys_read(cb3f58a0,f289dcec,c0fe02aa,c0df39b6,246,...) at sys_read+0x4f syscall(f289dd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a6333, esp = 0xbfbfed7c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2227 tid 100171 td 0xcb3f55c0 sched_switch(cb3f55c0,0,104,1a3,f95e65f2,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cb3f55c0,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93f40,160,0,cb3f55c0,cb3f55c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8181270,0,c0f98da8,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c8181270,c112a710,c0f98780,527,0,...) at _cv_wait_sig+0x260 tty_wait(c8181200,c8181270,f28a0c44,1,0,...) at tty_wait+0x71 ttydisc_read(c8181200,f28a0c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c8075900,f28a0c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(ca7d4968,f28a0c44,c7dc4e80,0,cb3f55c0,...) at devfs_read_f+0x7e dofileread(f28a0c44,ffffffff,ffffffff,0,f28a0c20,...) at dofileread+0x9e kern_readv(cb3f55c0,0,f28a0c44,f28a0c64,1,...) at kern_readv+0x68 sys_read(cb3f55c0,f28a0cec,c0fe02aa,c0df39b6,246,...) at sys_read+0x4f syscall(f28a0d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a6333, esp = 0xbfbfed7c, ebp = 0xbfbfedb8 --- Tracing command getty pid 2226 tid 100172 td 0xcb3f52e0 sched_switch(cb3f52e0,0,104,1a3,467773c2,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cb3f52e0,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4bc5a,c8181404,0,c0f8c92f,cb3f52e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8181470,0,f28a3af0,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c8181470,c8181404,c0f98780,527,0,...) at _cv_wait_sig+0x260 tty_wait(c8181400,c8181470,f28a3c44,1,0,...) at tty_wait+0x71 ttydisc_read(c8181400,f28a3c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c7ed4e00,f28a3c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(cac413f0,f28a3c44,c7dc4e80,0,cb3f52e0,...) at devfs_read_f+0x7e dofileread(f28a3c44,ffffffff,ffffffff,0,f28a3c20,...) at dofileread+0x9e kern_readv(cb3f52e0,0,f28a3c44,f28a3c64,1,...) at kern_readv+0x68 sys_read(cb3f52e0,f28a3cec,c0fe02aa,c0df39b6,246,...) at sys_read+0x4f syscall(f28a3d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a6333, esp = 0xbfbfed7c, ebp = 0xbfbfedb8 --- Tracing command inetd pid 2187 tid 100155 td 0xcb3c95c0 sched_switch(cb3c95c0,0,104,1a3,df52738a,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cb3c95c0,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4bc5a,caf5edd0,0,c0f8c92f,cb3c95c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(caf5ede4,0,f2870a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(caf5ede4,caf5edd0,c0f96690,6b2,ca7fd0a8,...) at _cv_wait_sig+0x260 seltdwait(ca7fd0a8,58,c7dc4e80,cb3c95c0,cb3c9670,...) at seltdwait+0xa2 kern_select(cb3c95c0,6,bfbfdfa0,0,0,0,20,cb1ed588,4) at kern_select+0x655 sys_select(cb3c95c0,f2870cec,c0fe02aa,c0f96ed3,206,...) at sys_select+0x66 syscall(f2870d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281b82b3, esp = 0xbfbfdf5c, ebp = 0xbfbfede8 --- Tracing command moused pid 2154 tid 100154 td 0xcac5d2e0 sched_switch(cac5d2e0,0,104,1a3,bd548675,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cac5d2e0,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4bc5a,caf5e1d0,0,c0f8c92f,cac5d2e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(caf5e1e4,0,f286da58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(caf5e1e4,caf5e1d0,c0f96690,6b2,ca7fda10,...) at _cv_wait_sig+0x260 seltdwait(ca7fda10,58,c7dc4e80,cac5d2e0,f6,...) at seltdwait+0xa2 kern_select(cac5d2e0,400,bfbfea04,0,0,0,20,cb1ed84c,1) at kern_select+0x655 sys_select(cac5d2e0,f286dcec,c0fe02aa,c0df39b6,246,...) at sys_select+0x66 syscall(f286dd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281c52b3, esp = 0xbfbfe99c, ebp = 0xbfbfeb28 --- Tracing command watchdogd pid 2133 tid 100120 td 0xcaaae5c0 sched_switch(caaae5c0,0,104,1a3,35a54322,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(caaae5c0,0,c0f93f40,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(ea61,c0a9cc60,caaae5c0,0,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c112c064,6c,c0f8fa57,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c112c064,0,16c,c0f8fa57,ea61,...) at _sleep+0x34f kern_nanosleep(caaae5c0,f27d4c50,f27d4c58,3c,0,...) at kern_nanosleep+0xc1 sys_nanosleep(caaae5c0,f27d4cec,c0fe02aa,c0df39b6,206,...) at sys_nanosleep+0x6f syscall(f27d4d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x2819de27, esp = 0xbfbfecbc, ebp = 0xbfbfece8 --- Tracing command bash pid 2121 tid 100085 td 0xca7a08a0 sched_switch(ca7a08a0,0,104,1a3,50dabd5d,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(ca7a08a0,0,c0f93f40,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93f40,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca6c184c,6c,c0f96e87,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca6c184c,ca6c18d4,16c,c0f96e87,0,...) at _sleep+0x385 kern_wait(ca7a08a0,ffffffff,f2717c60,6,0,...) at kern_wait+0x626 sys_wait4(ca7a08a0,f2717cec,c0fe02aa,c0f96d76,216,...) at sys_wait4+0x3b syscall(f2717d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x282d545b, esp = 0xbfbfe99c, ebp = 0xbfbfe9b8 --- Tracing command sshd pid 2120 tid 100089 td 0xca6cd2e0 sched_switch(ca6cd2e0,0,104,1a3,432f0245,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca6cd2e0,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4bc5a,caf5e710,0,c0f8c92f,ca6cd2e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(caf5e724,0,f2729a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(caf5e724,caf5e710,c0f96690,6b2,ca7d4cb0,...) at _cv_wait_sig+0x260 seltdwait(ca7d4cb0,58,cad7c280,ca6cd2e0,f2729acc,...) at seltdwait+0xa2 kern_select(ca6cd2e0,a,288060b8,288060c0,0,0,20,c804284c,0) at kern_select+0x655 sys_select(ca6cd2e0,f2729cec,c0fe02aa,c0f96fdb,206,...) at sys_select+0x66 syscall(f2729d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284d72b3, esp = 0xbfbfde2c, ebp = 0xbfbfde78 --- Tracing command sshd pid 2118 tid 100130 td 0xcac5b8a0 sched_switch(cac5b8a0,0,104,1a3,84554598,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cac5b8a0,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4bc5a,ca7648d0,0,c0f8c92f,cac5b8a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca7648e4,0,f2811abc,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca7648e4,ca7648d0,c0f96690,6b2,f2811b44,...) at _cv_wait_sig+0x260 seltdwait(caace22c,c0f96690,5af,cac5b8a0,8,...) at seltdwait+0xa2 sys_poll(cac5b8a0,f2811cec,c0fe02aa,c0f7efb9,202,...) at sys_poll+0x321 syscall(f2811d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x2847014b, esp = 0xbfbfde8c, ebp = 0xbfbfdec8 --- Tracing command cron pid 2065 tid 100141 td 0xcad91b80 sched_switch(cad91b80,0,104,1a3,49639264,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(cad91b80,0,c0f93f40,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(ea61,c0a9cc60,cad91b80,0,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c112c064,6c,c0f8fa57,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c112c064,0,16c,c0f8fa57,ea61,...) at _sleep+0x34f kern_nanosleep(cad91b80,f2832c50,f2832c58,3c,0,...) at kern_nanosleep+0xc1 sys_nanosleep(cad91b80,f2832cec,c0fe02aa,c0f97334,206,...) at sys_nanosleep+0x6f syscall(f2832d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x2818fe27, esp = 0xbfbfec4c, ebp = 0xbfbfec78 --- Tracing command sendmail pid 2057 tid 100144 td 0xcad912e0 sched_switch(cad912e0,0,104,1a3,88eb6215,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,74,...) at mi_switch+0x1ff sleepq_switch(cad912e0,0,c0f93f40,1a7,74,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93f40,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cad8cb68,74,c0e1556e,100,0,...) at sleepq_wait_sig+0x17 _sleep(cad8cb68,cad8cb98,174,c0e1556e,0,...) at _sleep+0x385 kern_sigsuspend(cad912e0,0,0,0,0,...) at kern_sigsuspend+0xba sys_sigsuspend(cad912e0,f283bcec,c0fe02aa,c0f96d76,202,...) at sys_sigsuspend+0x4d syscall(f283bd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28348ccb, esp = 0xbfbfcf9c, ebp = 0xbfbfcfc8 --- Tracing command sendmail pid 2053 tid 100107 td 0xcaae4000 sched_switch(caae4000,0,104,1a3,4aa2d71b,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(caae4000,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f27a9a20,c0a4bc5a,ca763850,0,caae4000,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(ca763864,0,f27a9a58,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(ca763864,ca763850,1389,6b2,ca7ab888,...) at _cv_timedwait_sig+0x270 seltdwait(f27a9c10,f27a9c18,cad7c880,caae4000,c0a4849d,...) at seltdwait+0x8a kern_select(caae4000,5,bfbfc510,0,0,f27a9c5c,20,5,0) at kern_select+0x655 sys_select(caae4000,f27a9cec,c0fe02aa,c0f97293,206,...) at sys_select+0x66 syscall(f27a9d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x283f32b3, esp = 0xbfbfc47c, ebp = 0xbfbfcfa8 --- Tracing command sshd pid 2045 tid 100148 td 0xcad8e5c0 sched_switch(cad8e5c0,0,104,1a3,17fb82bd,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(cad8e5c0,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4bc5a,ca6b4510,0,c0f8c92f,cad8e5c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca6b4524,0,f2847a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca6b4524,ca6b4510,c0f96690,6b2,cabcf690,...) at _cv_wait_sig+0x260 seltdwait(cabcf690,58,cad7cb00,cad8e5c0,caad4e80,...) at seltdwait+0xa2 kern_select(cad8e5c0,7,2880c0b8,0,0,0,20,cad8c000,a) at kern_select+0x655 sys_select(cad8e5c0,f2847cec,c,c,202,...) at sys_select+0x66 syscall(f2847d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284d72b3, esp = 0xbfbfdeec, ebp = 0xbfbfedf8 --- Tracing command ntpd pid 1947 tid 100119 td 0xcaaae8a0 sched_switch(caaae8a0,0,104,1a3,ff4e458e,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(caaae8a0,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4bc5a,cac4cd10,0,c0f8c92f,caaae8a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(cac4cd24,0,f27d1a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(cac4cd24,cac4cd10,c0f96690,6b2,ca7ac620,...) at _cv_wait_sig+0x260 seltdwait(ca7ac620,58,c7dc4e80,caaae8a0,0,...) at seltdwait+0xa2 kern_select(caaae8a0,17,bfbfece8,0,0,0,20,caadeb10,0) at kern_select+0x655 sys_select(caaae8a0,f27d1cec,c0fe02aa,f27d1d80,206,...) at sys_select+0x66 syscall(f27d1d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x283722b3, esp = 0xbfbfecbc, ebp = 0xbfbfed88 --- Tracing command nfsd pid 1848 tid 100140 td 0xca6cc8a0 sched_switch(ca6cc8a0,0,104,1a3,4342c1cd,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca6cc8a0,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f282fbe0,c0a4bc5a,c800f900,0,ca6cc8a0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(caa9ba90,0,f282fc18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(caa9ba90,c800f900,1388,48a,0,...) at _cv_timedwait_sig+0x270 svc_run_internal(f282fd14,c0a2e798,c800f900,f282fd28,c0f88c10,...) at svc_run_internal+0x356 svc_thread_start(c800f900,f282fd28,c0f88c10,3db,caa9284c,...) at svc_thread_start+0x10 fork_exit(c0c4c520,c800f900,f282fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x2e, eip = 0xc, esp = 0x33, ebp = 0x1 --- Tracing command nfsd pid 1848 tid 100139 td 0xca6ccb80 sched_switch(ca6ccb80,0,104,1a3,7f17f331,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca6ccb80,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f282cbe0,c0a4bc5a,c800f900,0,ca6ccb80,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(caa9bad0,0,f282cc18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(caa9bad0,c800f900,1388,3b1,0,...) at _cv_timedwait_sig+0x270 svc_run_internal(f282cd14,c0a2e798,c800f900,f282cd28,c0f88c10,...) at svc_run_internal+0x356 svc_thread_start(c800f900,f282cd28,c0f88c10,3db,caa9284c,...) at svc_thread_start+0x10 fork_exit(c0c4c520,c800f900,f282cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x2e, eip = 0xc, esp = 0x33, ebp = 0x1 --- Tracing command nfsd pid 1848 tid 100138 td 0xca6cd000 sched_switch(ca6cd000,0,104,1a3,7f17cd55,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca6cd000,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f2829be0,c0a4bc5a,c800f900,0,ca6cd000,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(caa9bb10,0,f2829c18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(caa9bb10,c800f900,1388,3b1,0,...) at _cv_timedwait_sig+0x270 svc_run_internal(f2829d14,c0a2e798,c800f900,f2829d28,c0f88c10,...) at svc_run_internal+0x356 svc_thread_start(c800f900,f2829d28,c0f88c10,3db,caa9284c,...) at svc_thread_start+0x10 fork_exit(c0c4c520,c800f900,f2829d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x2e, eip = 0xc, esp = 0x33, ebp = 0x1 --- Tracing command nfsd pid 1848 tid 100096 td 0xcaab1000 sched_switch(caab1000,0,104,1a3,43435ac7,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(caab1000,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f2756498,c0a4bc5a,c800f900,0,caab1000,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(ca7552d0,0,f27564d0,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(ca7552d0,c800f900,1388,494,f2756518,...) at _cv_timedwait_sig+0x270 svc_run_internal(caab11b0,14,c0fbd79a,c0f8049b,f2756810,...) at svc_run_internal+0x356 svc_run(c800f900,0,c0f80476,19f,4,...) at svc_run+0x7b nfsrvd_nfsd(caab1000,f2756810,c,c10c3038,caab1000,...) at nfsrvd_nfsd+0x1c3 nfssvc_nfsd(caab1000,f2756cec,bfbfe870,caab1000,f2756d1c,...) at nfssvc_nfsd+0x177 sys_nfssvc(caab1000,f2756cec,c0fe02aa,c0f97293,206,...) at sys_nfssvc+0xeb syscall(f2756d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (155, FreeBSD ELF32, sys_nfssvc), eip = 0x280e890b, esp = 0xbfbfe82c, ebp = 0xbfbfea98 --- Tracing command nfsd pid 1842 tid 100091 td 0xca79cb80 sched_switch(ca79cb80,0,104,1a3,4045bc82,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca79cb80,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4bc5a,ca7644d0,0,c0f8c92f,ca79cb80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca7644e4,0,f2747a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca7644e4,ca7644d0,c0f96690,6b2,ca7ab038,...) at _cv_wait_sig+0x260 seltdwait(ca7ab038,58,c7dc4e80,ca79cb80,58,...) at seltdwait+0xa2 kern_select(ca79cb80,7,bfbfecb4,0,0,0,20,caa9384c,7) at kern_select+0x655 sys_select(ca79cb80,f2747cec,c0fe02aa,c0f7efb9,202,...) at sys_select+0x66 syscall(f2747d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281922b3, esp = 0xbfbfea9c, ebp = 0xbfbfede8 --- Tracing command mountd pid 1832 tid 100086 td 0xca7a05c0 sched_switch(ca7a05c0,0,104,1a3,435fa3e8,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca7a05c0,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4bc5a,ca6b4550,0,c0f8c92f,ca7a05c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca6b4564,0,f271ba58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca6b4564,ca6b4550,c0f96690,6b2,ca7fdb28,...) at _cv_wait_sig+0x260 seltdwait(ca7fdb28,58,c7dc4e80,ca7a05c0,478,...) at seltdwait+0xa2 kern_select(ca7a05c0,a,bfbfed04,0,0,0,20,ca6c1588,bfbfed04) at kern_select+0x655 sys_select(ca7a05c0,f271bcec,c0fe02aa,c0f7e257,206,...) at sys_select+0x66 syscall(f271bd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281aa2b3, esp = 0xbfbfec9c, ebp = 0xbfbfede8 --- Tracing command rpcbind pid 1732 tid 100093 td 0xca79c5c0 sched_switch(ca79c5c0,0,104,1a3,401b3959,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca79c5c0,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(f274da84,c0a4bc5a,ca763d50,0,ca79c5c0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(ca763d64,0,f274dabc,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(ca763d64,ca763d50,7531,6b2,f274db74,...) at _cv_timedwait_sig+0x270 seltdwait(f274dc44,f274dc4c,5af,ca79c5c0,0,...) at seltdwait+0x8a sys_poll(ca79c5c0,f274dcec,c0fe02aa,c0f96fb1,212,...) at sys_poll+0x321 syscall(f274dd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x2814a14b, esp = 0xbfbfcbac, ebp = 0xbfbfed78 --- Tracing command syslogd pid 1704 tid 100098 td 0xcaab08a0 sched_switch(caab08a0,0,104,1a3,5a33627c,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(caab08a0,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4bc5a,ca764290,0,c0f8c92f,caab08a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca7642a4,0,f275ca58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca7642a4,ca764290,c0f96690,6b2,ca7acb98,...) at _cv_wait_sig+0x260 seltdwait(ca7acb98,58,c7dc4e80,caab08a0,f275caf4,...) at seltdwait+0xa2 kern_select(caab08a0,9,2847f0ac,0,0,0,20,caa922c4,0) at kern_select+0x655 sys_select(caab08a0,f275ccec,c0fe02aa,c0fc2f8c,246,...) at sys_select+0x66 syscall(f275cd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281a92b3, esp = 0xbfbfe29c, ebp = 0xbfbfede8 --- Tracing command devd pid 1497 tid 100092 td 0xca79c8a0 sched_switch(ca79c8a0,0,104,1a3,18de7625,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca79c8a0,0,c0f93f40,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4bc5a,ca690e90,0,c0f8c92f,ca79c8a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(ca690ea4,0,f274aa58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca690ea4,ca690e90,c0f96690,6b2,ca7ab658,...) at _cv_wait_sig+0x260 seltdwait(ca7ab658,58,c7dc4e80,ca79c8a0,c1274f58,...) at seltdwait+0xa2 kern_select(ca79c8a0,5,bfbfe960,0,0,0,20,caa93588,34) at kern_select+0x655 sys_select(ca79c8a0,f274acec,c0fe02aa,c0deec83,206,...) at sys_select+0x66 syscall(f274ad28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x809152b, esp = 0xbfbfe92c, ebp = 0xbfbfee08 --- Tracing command softdepflush pid 19 tid 100077 td 0xca6cdb80 sched_switch(ca6cdb80,0,104,1a3,6220a91c,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,54,...) at mi_switch+0x1ff sleepq_switch(ca6cdb80,0,c0f93f40,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c12a8540,54,c0fc47fd,0,0,...) at sleepq_timedwait+0x6b _sleep(c12a8540,c12a8500,54,c0fc47fd,3e8,...) at _sleep+0x36a softdep_flush(0,ee320d28,c0f88c10,3db,ca6c0000,...) at softdep_flush+0x266 fork_exit(c0c9b9c0,0,ee320d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee320d60, ebp = 0 --- Tracing command vnlru pid 18 tid 100076 td 0xc805e8a0 sched_switch(c805e8a0,0,104,1a3,4dbc1ebe,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,60,...) at mi_switch+0x1ff sleepq_switch(c805e8a0,0,c0f93f40,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(ca6c02c4,60,c0f9ef5d,0,0,...) at sleepq_timedwait+0x6b _sleep(ca6c02c4,c12a21f4,260,c0f9ef5d,3e8,...) at _sleep+0x36a vnlru_proc(0,ee31dd28,c0f88c10,3db,ca6c02c4,...) at vnlru_proc+0xe7 fork_exit(c0afd990,0,ee31dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee31dd60, ebp = 0 --- Tracing command syncer pid 17 tid 100075 td 0xc805eb80 sched_switch(c805eb80,0,104,1a3,62042618,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c805eb80,0,c0f93f40,28b,c805eb80,...) at sleepq_switch+0x162 sleepq_timedwait(c12a2234,0,ee31ac78,1,0,...) at sleepq_timedwait+0x6b _cv_timedwait(c12a2234,c12a2220,3e8,78b,4e20,...) at _cv_timedwait+0x270 sched_sync(0,ee31ad28,c0f88c10,3db,ca6c0588,...) at sched_sync+0x595 fork_exit(c0afcd00,0,ee31ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee31ad60, ebp = 0 --- Tracing command bufdaemon pid 16 tid 100074 td 0xc8183000 sched_switch(c8183000,0,104,1a3,4dccb9dc,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,54,...) at mi_switch+0x1ff sleepq_switch(c8183000,0,c0f93f40,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c12a1ec8,54,c0f9be91,0,0,...) at sleepq_timedwait+0x6b _sleep(c12a1ec8,c12a1ed0,54,c0f9be91,3e8,...) at _sleep+0x36a buf_daemon(0,ee317d28,c0f88c10,3db,ca6c084c,...) at buf_daemon+0x175 fork_exit(c0ae14d0,0,ee317d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee317d60, ebp = 0 --- Tracing command pagezero pid 9 tid 100073 td 0xc81832e0 sched_switch(c81832e0,0,104,1a3,35bb3bf9,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c81832e0,0,c0f93f40,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c12aa3dc,0,c0fcb81e,0,0,...) at sleepq_timedwait+0x6b _sleep(c12aa3dc,c12a8e80,0,c0fcb81e,493e0,...) at _sleep+0x36a vm_pagezero(0,ee314d28,c0f88c10,3db,ca6c0b10,...) at vm_pagezero+0xdc fork_exit(c0ce2b60,0,ee314d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee314d60, ebp = 0 --- Tracing command vmdaemon pid 8 tid 100072 td 0xc81835c0 sched_switch(c81835c0,0,104,1a3,72a167c8,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,74,...) at mi_switch+0x1ff sleepq_switch(c81835c0,0,c0f93f40,268,0,...) at sleepq_switch+0x162 sleepq_wait(c12a9fc8,74,c0f9be91,0,0,...) at sleepq_wait+0x63 _sleep(c12a9fc8,c12a9fcc,74,c0f9be91,0,...) at _sleep+0x39c vm_daemon(0,ee311d28,c0f88c10,3db,ca6c1000,...) at vm_daemon+0x59 fork_exit(c0cdc470,0,ee311d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee311d60, ebp = 0 --- Tracing command pagedaemon pid 7 tid 100071 td 0xc81838a0 sched_switch(c81838a0,0,104,1a3,77427d3a,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,54,...) at mi_switch+0x1ff sleepq_switch(c81838a0,0,c0f93f40,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c12a9f90,54,c0f9be91,0,0,...) at sleepq_timedwait+0x6b _sleep(c12a9f90,c12a8e80,54,c0f9be91,1388,...) at _sleep+0x36a vm_pageout(0,ee30ed28,c0f88c10,3db,c7dc82c4,...) at vm_pageout+0x2c7 fork_exit(c0cdd530,0,ee30ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee30ed60, ebp = 0 --- Tracing command xpt_thrd pid 6 tid 100070 td 0xc8183b80 sched_switch(c8183b80,0,104,1a3,e8de7f2f,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c8183b80,0,c0f93f40,268,0,...) at sleepq_switch+0x162 sleepq_wait(c10f0854,5c,c0de9487,0,0,...) at sleepq_wait+0x63 _sleep(c10f0854,c10f0898,5c,c0de9487,0,...) at _sleep+0x39c xpt_scanner_thread(0,ee18bd28,c0f88c10,3db,c7dc8588,...) at xpt_scanner_thread+0x47 fork_exit(c049d5e0,0,ee18bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee18bd60, ebp = 0 --- Tracing command sctp_iterator pid 5 tid 100069 td 0xc8184000 sched_switch(c8184000,0,104,1a3,fb82c9b,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8184000,0,c0f93f40,268,0,...) at sleepq_switch+0x162 sleepq_wait(c12a3d58,0,c0fb25ff,0,0,...) at sleepq_wait+0x63 _sleep(c12a3d58,c12a3d20,0,c0fb25ff,0,...) at _sleep+0x39c sctp_iterator_thread(0,ee188d28,c0f88c10,3db,c7dc884c,...) at sctp_iterator_thread+0x5c fork_exit(c0b8f1c0,0,ee188d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee188d60, ebp = 0 --- Tracing command ctl_thrd pid 4 tid 100067 td 0xc81845c0 sched_switch(c81845c0,0,104,1a3,d6911281,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c81845c0,0,c0f93f40,268,0,...) at sleepq_switch+0x162 sleepq_wait(c819c000,5c,c0df3e92,0,0,...) at sleepq_wait+0x63 _sleep(c819c000,c819c000,5c,c0df3e92,0,...) at _sleep+0x39c ctl_work_thread(c819c000,ee182d28,c0f88c10,3db,c7dc8b10,...) at ctl_work_thread+0x22f6 fork_exit(c04c6a00,c819c000,ee182d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee182d60, ebp = 0 --- Tracing command fdc0 pid 3 tid 100064 td 0xc8185000 sched_switch(c8185000,0,104,1a3,5ecb337a,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c8185000,0,c0f93f40,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c7ffa03c,5c,c0f80f12,0,0,...) at sleepq_timedwait+0x6b _sleep(c7ffa03c,c7ffa0f0,5c,c0f80f12,3e8,...) at _sleep+0x36a fdc_thread(c7ffa000,ee179d28,c0f88c10,3db,c8042000,...) at fdc_thread+0x27d fork_exit(c0d61da0,c7ffa000,ee179d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee179d60, ebp = 0 --- Tracing command fw0_probe pid 2 tid 100062 td 0xc80515c0 sched_switch(c80515c0,0,104,1a3,e8deb902,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(c80515c0,0,c0f93f40,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93f40,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c808f000,6c,c0f80f12,100,0,...) at sleepq_wait_sig+0x17 _sleep(c808f000,c8093488,16c,c0f80f12,0,...) at _sleep+0x385 fw_bus_probe_thread(c808f000,ee169d28,c0f88c10,3db,c80422c4,...) at fw_bus_probe_thread+0xa08 fork_exit(c06f10c0,c808f000,ee169d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee169d60, ebp = 0 --- Tracing command usb pid 15 tid 100058 td 0xc805c2e0 sched_switch(c805c2e0,0,104,1a3,d68c09b9,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c805c2e0,0,c0f93f40,268,c805c2e0,...) at sleepq_switch+0x162 sleepq_wait(c8063d34,0,ee152cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8063d34,c8063e4c,c0f675d1,74,c8063d3c,...) at _cv_wait+0x260 usb_process(c8063d2c,ee152d28,c0f88c10,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fe090,c8063d2c,ee152d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee152d60, ebp = 0 --- Tracing command usb pid 15 tid 100057 td 0xc805c5c0 sched_switch(c805c5c0,0,104,1a3,192a566c,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c805c5c0,0,c0f93f40,268,c805c5c0,...) at sleepq_switch+0x162 sleepq_wait(c8063d04,0,ee14fcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8063d04,c8063e4c,c0f675d1,74,c8063d0c,...) at _cv_wait+0x260 usb_process(c8063cfc,ee14fd28,c0f88c10,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fe090,c8063cfc,ee14fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee14fd60, ebp = 0 --- Tracing command usb pid 15 tid 100056 td 0xc805c8a0 sched_switch(c805c8a0,0,104,1a3,d62556d7,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c805c8a0,0,c0f93f40,268,c805c8a0,...) at sleepq_switch+0x162 sleepq_wait(c8063cd4,0,ee14ccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8063cd4,c8063e4c,c0f675d1,74,c8063cdc,...) at _cv_wait+0x260 usb_process(c8063ccc,ee14cd28,c0f88c10,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fe090,c8063ccc,ee14cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee14cd60, ebp = 0 --- Tracing command usb pid 15 tid 100055 td 0xc805cb80 sched_switch(c805cb80,0,104,1a3,d625438d,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c805cb80,0,c0f93f40,268,c805cb80,...) at sleepq_switch+0x162 sleepq_wait(c8063ca4,0,ee149cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8063ca4,c8063e4c,c0f675d1,74,c8063cac,...) at _cv_wait+0x260 usb_process(c8063c9c,ee149d28,c0f88c10,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fe090,c8063c9c,ee149d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee149d60, ebp = 0 --- Tracing command usb pid 15 tid 100053 td 0xc805e2e0 sched_switch(c805e2e0,0,104,1a3,d6252e6a,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c805e2e0,0,c0f93f40,268,c805e2e0,...) at sleepq_switch+0x162 sleepq_wait(c8057b5c,0,ee140cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8057b5c,c8057c74,c0f675d1,74,c8057b64,...) at _cv_wait+0x260 usb_process(c8057b54,ee140d28,c0f88c10,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fe090,c8057b54,ee140d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee140d60, ebp = 0 --- Tracing command usb pid 15 tid 100052 td 0xc805e5c0 sched_switch(c805e5c0,0,104,1a3,105d3c95,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c805e5c0,0,c0f93f40,268,c805e5c0,...) at sleepq_switch+0x162 sleepq_wait(c8057b2c,0,ee13dcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8057b2c,c8057c74,c0f675d1,74,c8057b34,...) at _cv_wait+0x260 usb_process(c8057b24,ee13dd28,c0f88c10,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fe090,c8057b24,ee13dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee13dd60, ebp = 0 --- Tracing command usb pid 15 tid 100051 td 0xc804e000 sched_switch(c804e000,0,104,1a3,d5c1a1c2,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804e000,0,c0f93f40,268,c804e000,...) at sleepq_switch+0x162 sleepq_wait(c8057afc,0,ee13acac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8057afc,c8057c74,c0f675d1,74,c8057b04,...) at _cv_wait+0x260 usb_process(c8057af4,ee13ad28,c0f88c10,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fe090,c8057af4,ee13ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee13ad60, ebp = 0 --- Tracing command usb pid 15 tid 100050 td 0xc804e2e0 sched_switch(c804e2e0,0,104,1a3,d5c18dc5,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804e2e0,0,c0f93f40,268,c804e2e0,...) at sleepq_switch+0x162 sleepq_wait(c8057acc,0,ee137cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8057acc,c8057c74,c0f675d1,74,c8057ad4,...) at _cv_wait+0x260 usb_process(c8057ac4,ee137d28,c0f88c10,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fe090,c8057ac4,ee137d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee137d60, ebp = 0 --- Tracing command usb pid 15 tid 100049 td 0xc804e5c0 sched_switch(c804e5c0,0,104,1a3,d5c1756b,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804e5c0,0,c0f93f40,268,c804e5c0,...) at sleepq_switch+0x162 sleepq_wait(c8052b5c,0,ee133cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8052b5c,c8052c74,c0f675d1,74,c8052b64,...) at _cv_wait+0x260 usb_process(c8052b54,ee133d28,c0f88c10,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fe090,c8052b54,ee133d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee133d60, ebp = 0 --- Tracing command usb pid 15 tid 100048 td 0xc804e8a0 sched_switch(c804e8a0,0,104,1a3,dae89ae,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804e8a0,0,c0f93f40,268,c804e8a0,...) at sleepq_switch+0x162 sleepq_wait(c8052b2c,0,ee130cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8052b2c,c8052c74,c0f675d1,74,c8052b34,...) at _cv_wait+0x260 usb_process(c8052b24,ee130d28,c0f88c10,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fe090,c8052b24,ee130d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee130d60, ebp = 0 --- Tracing command usb pid 15 tid 100047 td 0xc804eb80 sched_switch(c804eb80,0,104,1a3,cffd05f3,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804eb80,0,c0f93f40,268,c804eb80,...) at sleepq_switch+0x162 sleepq_wait(c8052afc,0,ee12dcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8052afc,c8052c74,c0f675d1,74,c8052b04,...) at _cv_wait+0x260 usb_process(c8052af4,ee12dd28,c0f88c10,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fe090,c8052af4,ee12dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee12dd60, ebp = 0 --- Tracing command usb pid 15 tid 100046 td 0xc804f000 sched_switch(c804f000,0,104,1a3,cffcf324,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804f000,0,c0f93f40,268,c804f000,...) at sleepq_switch+0x162 sleepq_wait(c8052acc,0,ee12acac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8052acc,c8052c74,c0f675d1,74,c8052ad4,...) at _cv_wait+0x260 usb_process(c8052ac4,ee12ad28,c0f88c10,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fe090,c8052ac4,ee12ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee12ad60, ebp = 0 --- Tracing command usb pid 15 tid 100045 td 0xc804f2e0 sched_switch(c804f2e0,0,104,1a3,cffce037,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804f2e0,0,c0f93f40,268,c804f2e0,...) at sleepq_switch+0x162 sleepq_wait(c804ab5c,0,ee126cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c804ab5c,c804ac74,c0f675d1,74,c804ab64,...) at _cv_wait+0x260 usb_process(c804ab54,ee126d28,c0f88c10,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fe090,c804ab54,ee126d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee126d60, ebp = 0 --- Tracing command usb pid 15 tid 100044 td 0xc804f5c0 sched_switch(c804f5c0,0,104,1a3,c806b6b,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804f5c0,0,c0f93f40,268,c804f5c0,...) at sleepq_switch+0x162 sleepq_wait(c804ab2c,0,ee123cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c804ab2c,c804ac74,c0f675d1,74,c804ab34,...) at _cv_wait+0x260 usb_process(c804ab24,ee123d28,c0f88c10,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fe090,c804ab24,ee123d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee123d60, ebp = 0 --- Tracing command usb pid 15 tid 100043 td 0xc804f8a0 sched_switch(c804f8a0,0,104,1a3,cf98fb91,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804f8a0,0,c0f93f40,268,c804f8a0,...) at sleepq_switch+0x162 sleepq_wait(c804aafc,0,ee120cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c804aafc,c804ac74,c0f675d1,74,c804ab04,...) at _cv_wait+0x260 usb_process(c804aaf4,ee120d28,c0f88c10,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fe090,c804aaf4,ee120d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee120d60, ebp = 0 --- Tracing command usb pid 15 tid 100042 td 0xc804fb80 sched_switch(c804fb80,0,104,1a3,cf98e413,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c804fb80,0,c0f93f40,268,c804fb80,...) at sleepq_switch+0x162 sleepq_wait(c804aacc,0,ee11dcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c804aacc,c804ac74,c0f675d1,74,c804aad4,...) at _cv_wait+0x260 usb_process(c804aac4,ee11dd28,c0f88c10,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fe090,c804aac4,ee11dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee11dd60, ebp = 0 --- Tracing command usb pid 15 tid 100040 td 0xc803e000 sched_switch(c803e000,0,104,1a3,cf98ce5c,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c803e000,0,c0f93f40,268,c803e000,...) at sleepq_switch+0x162 sleepq_wait(c8044b5c,0,ee116cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8044b5c,c8044c74,c0f675d1,74,c8044b64,...) at _cv_wait+0x260 usb_process(c8044b54,ee116d28,c0f88c10,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fe090,c8044b54,ee116d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee116d60, ebp = 0 --- Tracing command usb pid 15 tid 100039 td 0xc803e2e0 sched_switch(c803e2e0,0,104,1a3,b36e3dc,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c803e2e0,0,c0f93f40,268,c803e2e0,...) at sleepq_switch+0x162 sleepq_wait(c8044b2c,0,ee113cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8044b2c,c8044c74,c0f675d1,74,c8044b34,...) at _cv_wait+0x260 usb_process(c8044b24,ee113d28,c0f88c10,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fe090,c8044b24,ee113d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee113d60, ebp = 0 --- Tracing command usb pid 15 tid 100038 td 0xc803e5c0 sched_switch(c803e5c0,0,104,1a3,cf349f3d,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c803e5c0,0,c0f93f40,268,c803e5c0,...) at sleepq_switch+0x162 sleepq_wait(c8044afc,0,ee110cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8044afc,c8044c74,c0f675d1,74,c8044b04,...) at _cv_wait+0x260 usb_process(c8044af4,ee110d28,c0f88c10,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fe090,c8044af4,ee110d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee110d60, ebp = 0 --- Tracing command usb pid 15 tid 100037 td 0xc803e8a0 sched_switch(c803e8a0,0,104,1a3,cf348c55,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c803e8a0,0,c0f93f40,268,c803e8a0,...) at sleepq_switch+0x162 sleepq_wait(c8044acc,0,ee10dcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8044acc,c8044c74,c0f675d1,74,c8044ad4,...) at _cv_wait+0x260 usb_process(c8044ac4,ee10dd28,c0f88c10,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fe090,c8044ac4,ee10dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee10dd60, ebp = 0 --- Tracing command usb pid 15 tid 100035 td 0xc8040000 sched_switch(c8040000,0,104,1a3,cf347540,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8040000,0,c0f93f40,268,c8040000,...) at sleepq_switch+0x162 sleepq_wait(c8039b5c,0,ee106cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8039b5c,c8039c74,c0f675d1,74,c8039b64,...) at _cv_wait+0x260 usb_process(c8039b54,ee106d28,c0f88c10,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fe090,c8039b54,ee106d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee106d60, ebp = 0 --- Tracing command usb pid 15 tid 100034 td 0xc80402e0 sched_switch(c80402e0,0,104,1a3,9640e50,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c80402e0,0,c0f93f40,268,c80402e0,...) at sleepq_switch+0x162 sleepq_wait(c8039b2c,0,ee103cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8039b2c,c8039c74,c0f675d1,74,c8039b34,...) at _cv_wait+0x260 usb_process(c8039b24,ee103d28,c0f88c10,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fe090,c8039b24,ee103d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee103d60, ebp = 0 --- Tracing command usb pid 15 tid 100033 td 0xc80405c0 sched_switch(c80405c0,0,104,1a3,cecffcc7,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c80405c0,0,c0f93f40,268,c80405c0,...) at sleepq_switch+0x162 sleepq_wait(c8039afc,0,ee100cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8039afc,c8039c74,c0f675d1,74,c8039b04,...) at _cv_wait+0x260 usb_process(c8039af4,ee100d28,c0f88c10,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fe090,c8039af4,ee100d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee100d60, ebp = 0 --- Tracing command usb pid 15 tid 100032 td 0xc80408a0 sched_switch(c80408a0,0,104,1a3,cecfea1c,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c80408a0,0,c0f93f40,268,c80408a0,...) at sleepq_switch+0x162 sleepq_wait(c8039acc,0,ee0fdcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c8039acc,c8039c74,c0f675d1,74,c8039ad4,...) at _cv_wait+0x260 usb_process(c8039ac4,ee0fdd28,c0f88c10,3db,c8042588,...) at usb_process+0x1a3 fork_exit(c08fe090,c8039ac4,ee0fdd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee0fdd60, ebp = 0 --- Tracing command yarrow pid 14 tid 100017 td 0xc7eda5c0 sched_switch(c7eda5c0,0,104,1a3,66694308,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7eda5c0,0,c0f93f40,28b,2,...) at sleepq_switch+0x162 sleepq_timedwait(c112ae64,0,c0f80f12,2,0,...) at sleepq_timedwait+0x6b _sleep(c112ae64,0,0,c0f80f12,64) at _sleep+0x36a pause(c0f80f12,64,c0f43390,111,0,...) at pause+0xa0 random_kthread(0,c7b51d28,c0f88c10,3db,c7dc7000,...) at random_kthread+0x1ef fork_exit(c0806320,0,c7b51d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b51d60, ebp = 0 --- Tracing command geom pid 13 tid 100015 td 0xc7edab80 sched_switch(c7edab80,0,104,1a3,5a2acb0f,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c7edab80,0,c0f93f40,268,0,...) at sleepq_switch+0x162 sleepq_wait(c1128a68,5c,c0f80f12,0,0,...) at sleepq_wait+0x63 _sleep(c1128a68,c11289c8,25c,c0f80f12,0,...) at _sleep+0x39c g_io_schedule_down(c7edab80,0,c0f82a61,6c,c7b4bd14,...) at g_io_schedule_down+0x5b g_down_procbody(0,c7b4bd28,c0f88c10,3db,c7dc72c4,...) at g_down_procbody+0x9d fork_exit(c09ef770,0,c7b4bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b4bd60, ebp = 0 --- Tracing command geom pid 13 tid 100014 td 0xc7edb000 sched_switch(c7edb000,0,104,1a3,5a328444,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c7edb000,0,c0f93f40,268,0,...) at sleepq_switch+0x162 sleepq_wait(c1128a64,5c,c0f80f12,0,0,...) at sleepq_wait+0x63 _sleep(c1128a64,c11289e8,25c,c0f80f12,0,...) at _sleep+0x39c g_io_schedule_up(c7edb000,0,c0f82a61,5f,c7b48d14,...) at g_io_schedule_up+0x134 g_up_procbody(0,c7b48d28,c0f88c10,3db,c7dc72c4,...) at g_up_procbody+0x9d fork_exit(c09ef810,0,c7b48d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b48d60, ebp = 0 --- Tracing command geom pid 13 tid 100013 td 0xc7dca2e0 sched_switch(c7dca2e0,0,104,1a3,e382c8f2,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c7dca2e0,0,c0f93f40,268,0,...) at sleepq_switch+0x162 sleepq_wait(c1128a5c,5c,c0f80f12,0,0,...) at sleepq_wait+0x63 _sleep(c1128a5c,c112898c,25c,c0f80f12,0,...) at _sleep+0x39c g_run_events(c1138cc0,0,c0f82a61,79,c7b45d14,...) at g_run_events+0x54b g_event_procbody(0,c7b45d28,c0f88c10,3db,c7dc72c4,...) at g_event_procbody+0x90 fork_exit(c09ef8b0,0,c7b45d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b45d60, ebp = 0 --- Tracing command intr pid 12 tid 100066 td 0xc81848a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100065 td 0xc8184b80 sched_switch(c8184b80,0,109,1a3,cecb84f5,...) at sched_switch+0x3ec mi_switch(109,0,c0f88f33,554,c8184b80,...) at mi_switch+0x1ff ithread_loop(c7e12680,ee17cd28,c0f88c10,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a322b0,c7e12680,ee17cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee17cd60, ebp = 0 --- Tracing command intr pid 12 tid 100063 td 0xc80512e0 sched_switch(c80512e0,0,109,1a3,467766cf,...) at sched_switch+0x3ec mi_switch(109,0,c0f88f33,554,c80512e0,...) at mi_switch+0x1ff ithread_loop(c7e12690,ee16cd28,c0f88c10,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a322b0,c7e12690,ee16cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee16cd60, ebp = 0 --- Tracing command intr pid 12 tid 100060 td 0xc8051b80 sched_switch(c8051b80,0,109,1a3,bf2c4564,...) at sched_switch+0x3ec mi_switch(109,0,c0f88f33,554,c8051b80,...) at mi_switch+0x1ff ithread_loop(c7e126a0,ee160d28,c0f88c10,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a322b0,c7e126a0,ee160d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee160d60, ebp = 0 --- Tracing command intr pid 12 tid 100059 td 0xc805c000 sched_switch(c805c000,0,109,1a3,70abff0c,...) at sched_switch+0x3ec mi_switch(109,0,c0f88f33,554,c805c000,...) at mi_switch+0x1ff ithread_loop(c7e126b0,ee158d28,c0f88c10,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a322b0,c7e126b0,ee158d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee158d60, ebp = 0 --- Tracing command intr pid 12 tid 100054 td 0xc805e000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100041 td 0xc8051000 sched_switch(c8051000,0,109,1a3,f37112b1,...) at sched_switch+0x3ec mi_switch(109,0,c0f88f33,554,c8051000,...) at mi_switch+0x1ff ithread_loop(c7e126d0,ee11ad28,c0f88c10,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a322b0,c7e126d0,ee11ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee11ad60, ebp = 0 --- Tracing command intr pid 12 tid 100036 td 0xc803eb80 sched_switch(c803eb80,0,109,1a3,ee4a88df,...) at sched_switch+0x3ec mi_switch(109,0,c0f88f33,554,c803eb80,...) at mi_switch+0x1ff ithread_loop(c7e126e0,ee10ad28,c0f88c10,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a322b0,c7e126e0,ee10ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee10ad60, ebp = 0 --- Tracing command intr pid 12 tid 100031 td 0xc8040b80 sched_switch(c8040b80,0,109,1a3,cacb1996,...) at sched_switch+0x3ec mi_switch(109,0,c0f88f33,554,c8040b80,...) at mi_switch+0x1ff ithread_loop(c7e126f0,ee0fad28,c0f88c10,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a322b0,c7e126f0,ee0fad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee0fad60, ebp = 0 --- Tracing command intr pid 12 tid 100030 td 0xc7edb2e0 sched_switch(c7edb2e0,0,109,1a3,5a30bc8f,...) at sched_switch+0x3ec mi_switch(109,0,c0f88f33,554,c7edb2e0,...) at mi_switch+0x1ff ithread_loop(c7e12700,edfa4d28,c0f88c10,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a322b0,c7e12700,edfa4d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xedfa4d60, ebp = 0 --- Tracing command intr pid 12 tid 100029 td 0xc7edb5c0 sched_switch(c7edb5c0,0,109,1a3,4de2ccbf,...) at sched_switch+0x3ec mi_switch(109,0,c0f88f33,554,c7edb5c0,...) at mi_switch+0x1ff ithread_loop(c7e12710,edfa0d28,c0f88c10,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a322b0,c7e12710,edfa0d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xedfa0d60, ebp = 0 --- Tracing command intr pid 12 tid 100028 td 0xc7edb8a0 sched_switch(c7edb8a0,0,109,1a3,c1054ad3,...) at sched_switch+0x3ec mi_switch(109,0,c0f88f33,554,c7edb8a0,...) at mi_switch+0x1ff ithread_loop(c7e12720,c7ba0d28,c0f88c10,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a322b0,c7e12720,c7ba0d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7ba0d60, ebp = 0 --- Tracing command intr pid 12 tid 100027 td 0xc7edbb80 sched_switch(c7edbb80,0,109,1a3,e654d4a1,...) at sched_switch+0x3ec mi_switch(109,0,c0f88f33,554,c7edbb80,...) at mi_switch+0x1ff ithread_loop(c7e12730,c7b6fd28,c0f88c10,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a322b0,c7e12730,c7b6fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b6fd60, ebp = 0 --- Tracing command intr pid 12 tid 100026 td 0xc7fc4000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100024 td 0xc7fc45c0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100023 td 0xc7fc48a0 sched_switch(c7fc48a0,0,109,1a3,5a315d3e,...) at sched_switch+0x3ec mi_switch(109,0,c0f88f33,554,c7fc48a0,...) at mi_switch+0x1ff ithread_loop(c7e12760,c7b63d28,c0f88c10,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a322b0,c7e12760,c7b63d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b63d60, ebp = 0 --- Tracing command intr pid 12 tid 100012 td 0xc7dca5c0 sched_switch(c7dca5c0,0,109,1a3,e2715391,...) at sched_switch+0x3ec mi_switch(109,0,c0f88f33,554,c7dca5c0,...) at mi_switch+0x1ff ithread_loop(c7e127a0,c7b42d28,c0f88c10,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a322b0,c7e127a0,c7b42d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b42d60, ebp = 0 --- Tracing command intr pid 12 tid 100011 td 0xc7dca8a0 sched_switch(c7dca8a0,0,109,1a3,5bae4949,...) at sched_switch+0x3ec mi_switch(109,0,c0f88f33,554,c7dca8a0,...) at mi_switch+0x1ff ithread_loop(c7e127b0,c7b3fd28,c0f88c10,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a322b0,c7e127b0,c7b3fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b3fd60, ebp = 0 --- Tracing command intr pid 12 tid 100010 td 0xc7dcab80 sched_switch(c7dcab80,0,109,1a3,66691430,...) at sched_switch+0x3ec mi_switch(109,0,c0f88f33,554,c7dcab80,...) at mi_switch+0x1ff ithread_loop(c7e127c0,c7b3cd28,c0f88c10,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a322b0,c7e127c0,c7b3cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b3cd60, ebp = 0 --- Tracing command intr pid 12 tid 100009 td 0xc7e13000 sched_switch(c7e13000,0,109,1a3,6979a641,...) at sched_switch+0x3ec mi_switch(109,0,c0f88f33,554,c7e13000,...) at mi_switch+0x1ff ithread_loop(c7e127d0,c7b39d28,c0f88c10,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a322b0,c7e127d0,c7b39d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b39d60, ebp = 0 --- Tracing command intr pid 12 tid 100008 td 0xc7e132e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100007 td 0xc7e135c0 sched_switch(c7e135c0,0,109,1a3,4df38fef,...) at sched_switch+0x3ec mi_switch(109,0,c0f88f33,554,c7e135c0,...) at mi_switch+0x1ff ithread_loop(c7e127f0,c7b33d28,c0f88c10,3db,c7dc7588,...) at ithread_loop+0x1fb fork_exit(c0a322b0,c7e127f0,c7b33d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b33d60, ebp = 0 --- Tracing command idle pid 11 tid 100006 td 0xc7dc9000 cpustop_handler(3,13,c7b2fc04,c0dc14f6,c0dc7b0c,...) at cpustop_handler+0x28 ipi_nmi_handler(c0dc7b0c,c1309440,0,c12d74e4,c7dc784c,...) at ipi_nmi_handler+0x47 trap(c7b2fc10) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0da5d85, esp = 0xc7b2fc50, ebp = 0xc7b2fc50 --- acpi_cpu_c1(c7b2fc6c,c10f0e30,3,301,142,...) at acpi_cpu_c1+0x5 acpi_cpu_idle(0,c7b2fca0,c0db4019,0,ffffffff,...) at acpi_cpu_idle+0x122 cpu_idle_acpi(0,ffffffff,c113a040,2,c7b2fce8,...) at cpu_idle_acpi+0x2f cpu_idle(0,c7b2fcc4,c0f905c2,a0b,c7dc9000,...) at cpu_idle+0x89 sched_idletd(0,c7b2fd28,c0f88c10,3db,c7dc784c,...) at sched_idletd+0x263 fork_exit(c0a830d0,0,c7b2fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b2fd60, ebp = 0 --- Tracing command idle pid 11 tid 100005 td 0xc7dc92e0 cpustop_handler(2,13,c7b2cc04,c0dc14f6,f5,...) at cpustop_handler+0x28 ipi_nmi_handler(f5,0,0,c12d71b8,c7dc784c,...) at ipi_nmi_handler+0x47 trap(c7b2cc10) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0da5d85, esp = 0xc7b2cc50, ebp = 0xc7b2cc50 --- acpi_cpu_c1(c7b2cc6c,c10f0e30,2,301,142,...) at acpi_cpu_c1+0x5 acpi_cpu_idle(0,c7b2cca0,c0db4019,0,ffffffff,...) at acpi_cpu_idle+0x122 cpu_idle_acpi(0,ffffffff,c11399c0,2,c7b2cce8,...) at cpu_idle_acpi+0x2f cpu_idle(0,c7b2ccc4,c0f905c2,a0b,c7dc92e0,...) at cpu_idle+0x89 sched_idletd(0,c7b2cd28,c0f88c10,3db,c7dc784c,...) at sched_idletd+0x263 fork_exit(c0a830d0,0,c7b2cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b2cd60, ebp = 0 --- Tracing command idle pid 11 tid 100004 td 0xc7dc95c0 cpustop_handler(1,13,c7b29bc0,c0dc14f6,c7dc95c0,...) at cpustop_handler+0x28 ipi_nmi_handler(c7dc95c0,c7dc95c0,c7b29b48,46,c7dc784c,...) at ipi_nmi_handler+0x47 trap(c7b29bcc) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0a121c6, esp = 0xc7b29c0c, ebp = 0xc7b29c20 --- statclock(0,0,c7b29c68,c0a68fdd,1,...) at statclock+0x106 handleevents(c7b29c74,142,8dfd49c9,572a4b66,6a26500,...) at handleevents+0x121 cpu_activeclock(0,ffffffff,c1139340,2,c7b29ce8,...) at cpu_activeclock+0x70 cpu_idle(0,c7b29cc4,c0f905c2,3c0,c7dc95c0,...) at cpu_idle+0x92 sched_idletd(0,c7b29d28,c0f88c10,3db,c7dc784c,...) at sched_idletd+0x263 fork_exit(c0a830d0,0,c7b29d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b29d60, ebp = 0 --- Tracing command idle pid 11 tid 100003 td 0xc7dc98a0 sched_switch(c7dc98a0,0,608,19e,445c1c52,...) at sched_switch+0x3ec mi_switch(608,0,c0f8ed3f,cd,0,...) at mi_switch+0x1ff critical_exit(0,2710,c1138cc0,2,c7b26ce8,...) at critical_exit+0xb5 cpu_idle(0,c7b26cc4,c0f905c2,a0b,c7dc98a0,...) at cpu_idle+0x97 sched_idletd(0,c7b26d28,c0f88c10,3db,c7dc784c,...) at sched_idletd+0x263 fork_exit(c0a830d0,0,c7b26d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b26d60, ebp = 0 --- Tracing command init pid 1 tid 100002 td 0xc7dc9b80 sched_switch(c7dc9b80,0,104,1a3,d423a357,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(c7dc9b80,0,c0f93f40,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f93f40,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c7dc7b10,6c,c0f96e87,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7dc7b10,c7dc7b98,16c,c0f96e87,0,...) at _sleep+0x385 kern_wait(c7dc9b80,ffffffff,c7b22c60,0,0,...) at kern_wait+0x626 sys_wait4(c7dc9b80,c7b22cec,c0fe02aa,c0f96cbe,286,...) at sys_wait4+0x3b syscall(c7b22d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x8060f97, esp = 0xbfbfe8bc, ebp = 0xbfbfe8d8 --- Tracing command audit pid 10 tid 100001 td 0xc7dca000 sched_switch(c7dca000,0,104,1a3,cece124c,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7dca000,0,c0f93f40,268,c7dca000,...) at sleepq_switch+0x162 sleepq_wait(c12a7c80,0,c7b1fc8c,1,0,...) at sleepq_wait+0x63 _cv_wait(c12a7c80,c12a7c64,c0fbf217,194,0,...) at _cv_wait+0x260 audit_worker(0,c7b1fd28,c0f88c10,3db,c7dc8000,...) at audit_worker+0x84 fork_exit(c0c5c870,0,c7b1fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b1fd60, ebp = 0 --- Tracing command kernel pid 0 tid 100078 td 0xca6cd8a0 sched_switch(ca6cd8a0,0,104,1a3,72a156fd,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(ca6cd8a0,0,c0f93f40,268,0,...) at sleepq_switch+0x162 sleepq_wait(c8036380,0,c0f80f12,0,0,...) at sleepq_wait+0x63 _sleep(c8036380,c8036398,0,c0f80f12,0,...) at _sleep+0x39c taskqueue_thread_loop(c13094e0,ee323d28,c0f88c10,3db,c1128b40,...) at taskqueue_thread_loop+0xb5 fork_exit(c0aa0360,c13094e0,ee323d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee323d60, ebp = 0 --- Tracing command kernel pid 0 tid 100068 td 0xc81842e0 sched_switch(c81842e0,0,104,1a3,5eccfb68,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c81842e0,0,c0f93f40,28b,2,...) at sleepq_switch+0x162 sleepq_timedwait(c112ae64,0,c0f80f12,2,0,...) at sleepq_timedwait+0x6b _sleep(c112ae64,0,0,c0f80f12,bb8) at _sleep+0x36a pause(c0f80f12,bb8,124,122,c1128bc8,...) at pause+0xa0 deadlkres(0,ee185d28,c0f88c10,3db,c1128b40,...) at deadlkres+0x324 fork_exit(c0a11a80,0,ee185d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee185d60, ebp = 0 --- Tracing command kernel pid 0 tid 100061 td 0xc80518a0 sched_switch(c80518a0,0,104,1a3,697a498e,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c80518a0,0,c0f93f40,268,c80518a0,...) at sleepq_switch+0x162 sleepq_wait(c80366c0,0,c0f8f261,c0f80f12,0,...) at sleepq_wait+0x63 msleep_spin(c80366c0,c80366d8,c0f80f12,0,c0f8c92f,...) at msleep_spin+0x259 taskqueue_thread_loop(c809349c,ee166d28,c0f88c10,3db,c1128b40,...) at taskqueue_thread_loop+0x8f fork_exit(c0aa0360,c809349c,ee166d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee166d60, ebp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xc7fc42e0 sched_switch(c7fc42e0,0,104,1a3,7127f63e,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7fc42e0,0,c0f93f40,268,0,...) at sleepq_switch+0x162 sleepq_wait(c7f9ee80,0,c0f80f12,0,0,...) at sleepq_wait+0x63 _sleep(c7f9ee80,c7f9ee98,0,c0f80f12,0,...) at _sleep+0x39c taskqueue_thread_loop(c1147a2c,c7b69d28,c0f88c10,3db,c1128b40,...) at taskqueue_thread_loop+0xb5 fork_exit(c0aa0360,c1147a2c,c7b69d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b69d60, ebp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xc7fc4b80 sched_switch(c7fc4b80,0,104,1a3,e8e2e4b5,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7fc4b80,0,c0f93f40,268,c7fc4b80,...) at sleepq_switch+0x162 sleepq_wait(c7f9f040,0,c0f8f261,c0f80f12,0,...) at sleepq_wait+0x63 msleep_spin(c7f9f040,c7f9f058,c0f80f12,0,c0f8c92f,...) at msleep_spin+0x259 taskqueue_thread_loop(c10f3ab8,c7b60d28,c0f88c10,3db,c1128b40,...) at taskqueue_thread_loop+0x8f fork_exit(c0aa0360,c10f3ab8,c7b60d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b60d60, ebp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xc7e138a0 sched_switch(c7e138a0,0,104,1a3,e8e2d6d2,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7e138a0,0,c0f93f40,268,c7e138a0,...) at sleepq_switch+0x162 sleepq_wait(c7f9f040,0,c0f8f261,c0f80f12,0,...) at sleepq_wait+0x63 msleep_spin(c7f9f040,c7f9f058,c0f80f12,0,c0f8c92f,...) at msleep_spin+0x259 taskqueue_thread_loop(c10f3ab8,c7b5dd28,c0f88c10,3db,c1128b40,...) at taskqueue_thread_loop+0x8f fork_exit(c0aa0360,c10f3ab8,c7b5dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b5dd60, ebp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xc7e13b80 sched_switch(c7e13b80,0,104,1a3,e8e2c5be,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7e13b80,0,c0f93f40,268,c7e13b80,...) at sleepq_switch+0x162 sleepq_wait(c7f9f040,0,c0f8f261,c0f80f12,0,...) at sleepq_wait+0x63 msleep_spin(c7f9f040,c7f9f058,c0f80f12,0,c0f8c92f,...) at msleep_spin+0x259 taskqueue_thread_loop(c10f3ab8,c7b5ad28,c0f88c10,3db,c1128b40,...) at taskqueue_thread_loop+0x8f fork_exit(c0aa0360,c10f3ab8,c7b5ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b5ad60, ebp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xc7eda000 sched_switch(c7eda000,0,104,1a3,e8deef12,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7eda000,0,c0f93f40,268,0,...) at sleepq_switch+0x162 sleepq_wait(c7f9f080,0,c0f80f12,0,0,...) at sleepq_wait+0x63 _sleep(c7f9f080,c7f9f098,0,c0f80f12,0,...) at _sleep+0x39c taskqueue_thread_loop(c12a8264,c7b57d28,c0f88c10,3db,c1128b40,...) at taskqueue_thread_loop+0xb5 fork_exit(c0aa0360,c12a8264,c7b57d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b57d60, ebp = 0 --- Tracing command kernel pid 0 tid 100018 td 0xc7eda2e0 sched_switch(c7eda2e0,0,104,1a3,e8dedde1,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7eda2e0,0,c0f93f40,268,0,...) at sleepq_switch+0x162 sleepq_wait(c7f9f0c0,0,c0f80f12,0,0,...) at sleepq_wait+0x63 _sleep(c7f9f0c0,c7f9f0d8,0,c0f80f12,0,...) at _sleep+0x39c taskqueue_thread_loop(c11294b8,c7b54d28,c0f88c10,3db,c1128b40,...) at taskqueue_thread_loop+0xb5 fork_exit(c0aa0360,c11294b8,c7b54d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b54d60, ebp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xc7eda8a0 sched_switch(c7eda8a0,0,104,1a3,acee6afb,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c7eda8a0,0,c0f93f40,268,0,...) at sleepq_switch+0x162 sleepq_wait(c7dade80,0,c0f80f12,0,0,...) at sleepq_wait+0x63 _sleep(c7dade80,c7dade98,0,c0f80f12,0,...) at _sleep+0x39c taskqueue_thread_loop(c1146440,c7b4ed28,c0f88c10,3db,c1128b40,...) at taskqueue_thread_loop+0xb5 fork_exit(c0aa0360,c1146440,c7b4ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b4ed60, ebp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xc1128e10 sched_switch(c1128e10,0,104,1a3,610f9cb1,...) at sched_switch+0x3ec mi_switch(104,0,c0f93f40,1f3,54,...) at mi_switch+0x1ff sleepq_switch(c1128e10,0,c0f93f40,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c1128b40,54,c0f90ed4,0,0,...) at sleepq_timedwait+0x6b _sleep(c1128b40,0,54,c0f90ed4,2710,...) at _sleep+0x36a scheduler(0,181ec00,181ec00,181e000,1825000,...) at scheduler+0x240 mi_startup() at mi_startup+0xac begin() at begin+0x2c db:0:allt> call doadump Physical memory: 3435 MB Dumping 191 MB: 176 160 144 128 112 96 80 64 48 32 16 Dump complete = 0 db:0:doadump> reset (kgdb) bt #0 doadump (textdump=0xf28975d8) at pcpu.h:244 #1 0xc0510ec9 in db_fncall (dummy1=0xc0a9166a, dummy2=0x0, dummy3=0xffffffff, dummy4=0xf289766c "\200v\211ò") at ../../../ddb/db_command.c:573 #2 0xc05112ff in db_command (last_cmdp=0xc10f1c7c, cmd_table=0x0, dopager=0x0) at ../../../ddb/db_command.c:449 #3 0xc05113b4 in db_command_script (command=0xc10f2b88 "call doadump") at ../../../ddb/db_command.c:520 #4 0xc05157a0 in db_script_exec (scriptname=0xc10f24e0 "doadump", warnifnotfound=Variable "warnifnotfound" is not available. ) at ../../../ddb/db_script.c:302 #5 0xc0515831 in db_run_cmd (addr=0x1, have_addr=0x0, count=0xc12d4de0, modif=0xf28977a4 "") at ../../../ddb/db_script.c:375 #6 0xc05112c1 in db_command (last_cmdp=0xc10f1c7c, cmd_table=0x0, dopager=0x1) at ../../../ddb/db_command.c:449 #7 0xc051141a in db_command_loop () at ../../../ddb/db_command.c:502 #8 0xc051340d in db_trap (type=0x3, code=0x0) at ../../../ddb/db_main.c:229 #9 0xc0a91b26 in kdb_trap (type=0x3, code=0x0, tf=0xf2897958) at ../../../kern/subr_kdb.c:629 #10 0xc0dc1b2b in trap (frame=0xf2897958) at ../../../i386/i386/trap.c:722 #11 0xc0daa3bc in calltrap () at ../../../i386/i386/exception.s:168 #12 0xc0a9166a in kdb_enter (why=0xc0f9e1c6 "vfslock", msg=0xc0f9f22f "lock violation") at cpufunc.h:71 #13 0xc0af7731 in vfs_badlock (msg=0xc0f9ebab "is not exclusive locked but should be", str=0xc0f9ec8f "insmntque: mp-safe fs and non-locked vp", vp=0xcb5ad414) at ../../../kern/vfs_subr.c:3814 #14 0xc0afb745 in assert_vop_elocked (vp=0xcb5ad414, str=0xc0f9ec8f "insmntque: mp-safe fs and non-locked vp") at ../../../kern/vfs_subr.c:3854 #15 0xc0afba00 in insmntque1 (vp=0xcb5ad414, mp=0xcb1cdb50, dtr=0xcb613b00 , dtr_arg=0xca00a7d0) at ../../../kern/vfs_subr.c:1124 #16 0xcb61397d in null_nodeget (mp=0xcb1cdb50, lowervp=0xcb5ad2b8, vpp=0xf2897a70) at /usr/src/sys/modules/nullfs/../../fs/nullfs/null_subr.c:252 #17 0xcb614d33 in null_lookup (ap=0xf2897afc) at /usr/src/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:367 #18 0xc0de55d5 in VOP_LOOKUP_APV (vop=0xcb616680, a=0xf2897afc) at vnode_if.c:123 #19 0xc0aed82b in lookup (ndp=0xf2897bac) at vnode_if.h:54 #20 0xc0aee84f in namei (ndp=0xf2897bac) at ../../../kern/vfs_lookup.c:307 #21 0xc0b04582 in kern_utimesat (td=0xcb3c9000, fd=0xffffff9c, path=0xbfbfeab2
, pathseg=UIO_USERSPACE, tptr=0xbfbfe884, tptrseg=UIO_USERSPACE) at ../../../kern/vfs_syscalls.c:3330 #22 0xc0b04655 in kern_utimes (td=0xcb3c9000, path=0xbfbfeab2
, pathseg=UIO_USERSPACE, tptr=0xbfbfe884, tptrseg=UIO_USERSPACE) at ../../../kern/vfs_syscalls.c:3314 #23 0xc0b04691 in sys_utimes (td=0xcb3c9000, uap=0xf2897cec) at ../../../kern/vfs_syscalls.c:3290 #24 0xc0dc10f7 in syscall (frame=0xf2897d28) at subr_syscall.c:135 #25 0xc0daa421 in Xint0x80_syscall () at ../../../i386/i386/exception.s:266 #26 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) f 15 #15 0xc0afba00 in insmntque1 (vp=0xcb5ad414, mp=0xcb1cdb50, dtr=0xcb613b00 , dtr_arg=0xca00a7d0) at ../../../kern/vfs_subr.c:1124 1124 ASSERT_VOP_ELOCKED(vp, (kgdb) l 1119 KASSERT(vp->v_mount == NULL, 1120 ("insmntque: vnode already on per mount vnode list")); 1121 VNASSERT(mp != NULL, vp, ("Don't call insmntque(foo, NULL)")); 1122 #ifdef DEBUG_VFS_LOCKS 1123 if (!VFS_NEEDSGIANT(mp)) 1124 ASSERT_VOP_ELOCKED(vp, 1125 "insmntque: mp-safe fs and non-locked vp"); 1126 #endif 1127 MNT_ILOCK(mp); 1128 if ((mp->mnt_kern_flag & MNTK_NOINSMNTQ) != 0 && (kgdb) p *vp $1 = {v_type = VREG, v_tag = 0xcb615266 "null", v_op = 0xcb616680, v_data = 0xca00a7d0, v_mount = 0x0, v_nmntvnodes = {tqe_next = 0x0, tqe_prev = 0x0}, v_un = { vu_mount = 0x0, vu_socket = 0x0, vu_cdev = 0x0, vu_fifoinfo = 0x0}, v_hashlist = {le_next = 0x0, le_prev = 0x0}, v_hash = 0x0, v_cache_src = {lh_first = 0x0}, v_cache_dst = {tqh_first = 0x0, tqh_last = 0xcb5ad444}, v_cache_dd = 0x0, v_cstart = 0x0, v_lasta = 0x0, v_lastw = 0x0, v_clen = 0x0, v_lock = {lock_object = { lo_name = 0xcb615266 "null", lo_flags = 0x5730008, lo_data = 0x0, lo_witness = 0xc7d7a038}, lk_lock = 0x1, lk_exslpfail = 0x0, lk_timo = 0x33, lk_pri = 0x60, lk_stack = {depth = 0x0, pcs = {0x0 }}}, v_interlock = {lock_object = {lo_name = 0xc0f961f8 "vnode interlock", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc7d721e0}, mtx_lock = 0x4}, v_vnlock = 0xcb5ad310, v_holdcnt = 0x1, v_usecount = 0x1, v_iflag = 0x0, v_vflag = 0x0, v_writecount = 0x0, v_freelist = {tqe_next = 0x0, tqe_prev = 0x0}, v_bufobj = {bo_mtx = {lock_object = {lo_name = 0xc0f9ef32 "bufobj interlock", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc7d77c10}, mtx_lock = 0x4}, bo_clean = {bv_hd = {tqh_first = 0x0, tqh_last = 0xcb5ad520}, bv_root = 0x0, bv_cnt = 0x0}, bo_dirty = {bv_hd = {tqh_first = 0x0, tqh_last = 0xcb5ad530}, bv_root = 0x0, bv_cnt = 0x0}, bo_numoutput = 0x0, bo_flag = 0x0, bo_ops = 0xc10cb4e0, bo_bsize = 0x4000, bo_object = 0x0, bo_synclist = {le_next = 0x0, le_prev = 0x0}, bo_private = 0xcb5ad414, __bo_vnode = 0xcb5ad414}, v_pollinfo = 0x0, v_label = 0x0, v_lockf = 0x0} (kgdb) p *mp $2 = {mnt_mtx = {lock_object = {lo_name = 0xc0f806d1 "struct mount mtx", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc7d72450}, mtx_lock = 0x4}, mnt_gen = 0x1, mnt_list = {tqe_next = 0x0, tqe_prev = 0xcb60cb68}, mnt_op = 0xcb616640, mnt_vfc = 0xcb616600, mnt_vnodecovered = 0xcb1e12b8, mnt_syncer = 0xcb5c56cc, mnt_ref = 0x2, mnt_nvnodelist = {tqh_first = 0xcb5c5828, tqh_last = 0xcb5c56e0}, mnt_nvnodelistsize = 0x2, mnt_writeopcount = 0x0, mnt_kern_flag = 0x60000040, mnt_flag = 0x1000, mnt_noasync = 0x0, mnt_opt = 0xca7464e0, mnt_optnew = 0x0, mnt_maxsymlinklen = 0x0, mnt_stat = { f_version = 0x20030518, f_type = 0x35, f_flags = 0x5000, f_bsize = 0x800, f_iosize = 0x4000, f_blocks = 0xf7517, f_bfree = 0x1c87d, f_bavail = 0x8bec, f_files = 0x44ffe, f_ffree = 0x42c05, f_syncwrites = 0x0, f_asyncwrites = 0x0, f_syncreads = 0x0, f_asyncreads = 0x0, f_spare = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, f_namemax = 0xff, f_owner = 0x0, f_fsid = {val = {0x2900ff03, 0x29}}, f_charspare = '\0' , f_fstypename = "nullfs\000\000\000\000\000\000\000\000\000", f_mntfromname = "/bin", '\0' , f_mntonname = "/mnt2", '\0' }, mnt_cred = 0xcac4bd00, mnt_data = 0xc7ee1810, mnt_time = 0x0, mnt_iosize_max = 0x10000, mnt_export = 0x0, mnt_label = 0x0, mnt_hashseed = 0x96ac4960, mnt_lockref = 0x0, mnt_secondary_writes = 0x0, mnt_secondary_accwrites = 0x0, mnt_susp_owner = 0x0, mnt_gjprovider = 0x0, mnt_explock = {lock_object = { lo_name = 0xc0f806e2 "explock", lo_flags = 0x5730000, lo_data = 0x0, lo_witness = 0xc7d77ee8}, lk_lock = 0x1, lk_exslpfail = 0x0, lk_timo = 0x0, lk_pri = 0x60, lk_stack = {depth = 0x6, pcs = {0xc0a43cbe, 0xc0ae97a7, 0xc0af203e, 0xc0af269f, 0xc0dc10f7, 0xc0daa421, 0x0 }}}} (kgdb) $ svn diff -x -p /usr/src/sys Index: /usr/src/sys/fs/nullfs/null_vnops.c =================================================================== --- /usr/src/sys/fs/nullfs/null_vnops.c (revision 231973) +++ /usr/src/sys/fs/nullfs/null_vnops.c (working copy) @@ -697,12 +697,18 @@ null_inactive(struct vop_inactive_args *ap) static int null_reclaim(struct vop_reclaim_args *ap) { - struct vnode *vp = ap->a_vp; - struct null_node *xp = VTONULL(vp); - struct vnode *lowervp = xp->null_lowervp; + struct vnode *vp; + struct null_node *xp; + struct vnode *lowervp; - if (lowervp) - null_hashrem(xp); + vp = ap->a_vp; + xp = VTONULL(vp); + lowervp = xp->null_lowervp; + + KASSERT(lowervp != NULL && vp->v_vnlock != &vp->v_lock, + ("Reclaiming inclomplete null vnode %p", vp)); + + null_hashrem(xp); /* * Use the interlock to protect the clearing of v_data to * prevent faults in null_lock(). @@ -713,10 +719,7 @@ null_reclaim(struct vop_reclaim_args *ap) vp->v_object = NULL; vp->v_vnlock = &vp->v_lock; VI_UNLOCK(vp); - if (lowervp) - vput(lowervp); - else - panic("null_reclaim: reclaiming a node with no lowervp"); + vput(lowervp); free(xp, M_NULLFSNODE); return (0); Index: /usr/src/sys/fs/nullfs/null_subr.c =================================================================== --- /usr/src/sys/fs/nullfs/null_subr.c (revision 231973) +++ /usr/src/sys/fs/nullfs/null_subr.c (working copy) @@ -169,19 +169,28 @@ null_hashins(mp, xp) } static void -null_insmntque_dtr(struct vnode *vp, void *xp) +null_destroy_proto(struct vnode *vp, void *xp) { - vput(((struct null_node *)xp)->null_lowervp); + VI_LOCK(vp); vp->v_data = NULL; vp->v_vnlock = &vp->v_lock; - free(xp, M_NULLFSNODE); vp->v_op = &dead_vnodeops; + VI_UNLOCK(vp); (void) vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); vgone(vp); vput(vp); + free(xp, M_NULLFSNODE); } +static void +null_insmntque_dtr(struct vnode *vp, void *xp) +{ + + vput(((struct null_node *)xp)->null_lowervp); + null_destroy_proto(vp, xp); +} + /* * Make a new or get existing nullfs node. * Vp is the alias vnode, lowervp is the lower vnode. @@ -250,9 +259,7 @@ null_nodeget(mp, lowervp, vpp) *vpp = null_hashins(mp, xp); if (*vpp != NULL) { vrele(lowervp); - vp->v_vnlock = &vp->v_lock; - xp->null_lowervp = NULL; - vrele(vp); + null_destroy_proto(vp, xp); return (0); } *vpp = vp; Index: /usr/src/sys/fs/nullfs/null_vfsops.c =================================================================== --- /usr/src/sys/fs/nullfs/null_vfsops.c (revision 231973) +++ /usr/src/sys/fs/nullfs/null_vfsops.c (working copy) @@ -180,7 +180,9 @@ nullfs_mount(struct mount *mp) MNT_IUNLOCK(mp); } MNT_ILOCK(mp); - mp->mnt_kern_flag |= lowerrootvp->v_mount->mnt_kern_flag & MNTK_MPSAFE; + mp->mnt_kern_flag |= lowerrootvp->v_mount->mnt_kern_flag & + (MNTK_MPSAFE | MNTK_EXTENDED_SHARED | MNTK_SHARED_WRITES | + MNTK_LOOKUP_SHARED); MNT_IUNLOCK(mp); mp->mnt_data = xmp; vfs_getnewfsid(mp); @@ -307,6 +309,12 @@ nullfs_vget(mp, ino, flags, vpp) struct vnode **vpp; { int error; + + KASSERT((flags & LK_TYPE_MASK) != 0, + ("nullfs_vget: no lock requested")); + flags &= ~LK_TYPE_MASK; + flags |= LK_EXCLUSIVE; + error = VFS_VGET(MOUNTTONULLMOUNT(mp)->nullm_vfs, ino, flags, vpp); if (error) return (error); -- Test scenario: + mount -t nullfs /bin /mnt2 + touch /mnt2/xxx