GENERIC HEAD from 2013-05-08 12:53:21 UTC, r250363M, vmcore.623 GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb 4718592K of memory above 4GB ignored Copyright (c) 1992-2013 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #0 r250363M: Wed May 8 15:23:31 CEST 2013 pho@x4.osted.lan:/usr/src/sys/i386/compile/PHO i386 FreeBSD clang version 3.3 (trunk 178860) 20130405 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. CPU: AMD Phenom(tm) 9150e Quad-Core Processor (1800.02-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x100f23 Family = 0x10 Model = 0x2 Stepping = 3 Features=0x178bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x7ff TSC: P-state invariant real memory = 8589934592 (8192 MB) avail memory = 3498315776 (3336 MB) : Trying to mount root from ufs:/dev/ad4s1a [rw]... Setting hostuuid: 00000000-0000-0000-0000-00218515337d. Setting hostid: 0x6b64ac17. Starting ddb. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: /dev/ad4s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1a: clean, 617594 free (1554 frags, 77005 blocks, 0.2% fragmentation) /dev/ad4s1f: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1f: clean, 184387 free (4395 frags, 22499 blocks, 0.4% fragmentation) /dev/label/tmp: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/label/tmp: clean, 48585772 free (3252 frags, 6072815 blocks, 0.0% fragmentation) /dev/ad4s1d: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1d: clean, 3621003 free (257779 frags, 420403 blocks, 2.5% fragmentation) /dev/ad4s1e: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1e: clean, 9367325 free (165469 frags, 1150232 blocks, 0.7% fragmentation) Mounting local file systems:. Setting hostname: x4.osted.lan. re0: link state changed to DOWN re0: link state changed to UP Starting Network: lo0 re0. lo0: flags=8049 metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 nd6 options=21 re0: flags=8843 metric 0 mtu 1500 options=8209b ether 00:21:85:15:33:7d inet 192.168.1.101 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::221:85ff:fe15:337d%re0 prefixlen 64 scopeid 0x1 nd6 options=29 media: Ethernet autoselect (100baseTX ) status: active Starting devd. add net default: gateway 192.168.1.1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/kde4/lib /usr/local/lib/compat /usr/local/lib/compat/pkg /usr/local/lib/gcc46 /usr/local/lib/mysql /usr/local/lib/nss /usr/local/lib/pth /usr/local/lib/qt4 /usr/local/lib/virtualbox a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout Creating and/or trimming log files. Starting syslogd. savecore: couldn't find media and/or sector size of /var/crash: Inappropriate ioctl for device May 8 16:15:23 x4 savecore: couldn't find media and/or sector size of /var/crash: Inappropriate ioctl for device No core dumps found. Additional ABI support: linux. Starting rpcbind. NFS access cache time=60 lock order reversal: 1st 0xe381fbf8 bufwait (bufwait) @ kern/vfs_bio.c:3070 2nd 0xc7d63a00 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:284 KDB: stack backtrace: db_trace_self_wrapper(c10d0970,c10da0ac,c1c,c0b38ce0,c80b5670,...) at db_trace_self_wrapper+0x2d/frame 0xf5c7a798 kdb_backtrace(c10d3b9f,c7d63a00,c1105d1d,c7586268,c1105962,...) at kdb_backtrace+0x30/frame 0xf5c7a800 witness_checkorder(c7d63a00,9,c1105959,11c,0,...) at witness_checkorder+0xbaa/frame 0xf5c7a850 _sx_xlock(c7d63a00,0,c1105959,11c,c80a4000,...) at _sx_xlock+0x77/frame 0xf5c7a880 ufsdirhash_add(c7d355e4,f5c7a978,56f0,f5c7a900,f5c7a904,...) at ufsdirhash_add+0x4a/frame 0xf5c7a8b0 ufs_direnter(c80b55a0,c82f8b40,f5c7a978,f5c7ab64,e38236a8,...) at ufs_direnter+0x63b/frame 0xf5c7a930 ufs_mkdir(f5c7ac00,c112b3d3,c10ff626,c80b55a0,f5c7abfc,...) at ufs_mkdir+0x884/frame 0xf5c7aad0 VOP_MKDIR_APV(c122e9ec,f5c7ac00,f5c7ab64,f5c7ab90,28803020,...) at VOP_MKDIR_APV+0x124/frame 0xf5c7ab00 kern_mkdirat(c7d75300,ffffff9c,28803020,0,1c0) at kern_mkdirat+0x1fc/frame 0xf5c7ac24 sys_mkdir(c7d75300,f5c7acc8,14,c10ceb5d,7b3,...) at sys_mkdir+0x31/frame 0xf5c7ac40 syscall(f5c7ad08) at syscall+0x2ed/frame 0xf5c7acfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5c7acfc --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28174f57, esp = 0xbfbfd92c, ebp = 0xbfbfddd0 --- Clearing /tmp (X related). Starting nfsuserd. Starting mountd. Starting nfsd. Recovering vi editor sessions:. Updating motd:. Starting ntpd. Removing stale Samba tdb files: ....... done Starting nmbd. Starting smbd. Starting winbindd. Starting smartd. Starting default moused. Configuring syscons: keymap blanktime. Starting sshd. Starting cron. Local package initialization: backuplock order reversal: 1st 0xc866ea0c ufs (ufs) @ kern/vfs_lookup.c:517 2nd 0xe3839ff8 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:262 3rd 0xc847b73c ufs (ufs) @ kern/vfs_subr.c:2168 KDB: stack backtrace: db_trace_self_wrapper(c10d0970,0,0,0,0,...) at db_trace_self_wrapper+0x2d/frame 0lock order reversal: x 1st 0xc809346c ufs (ufs) @ kern/vfs_mount.c:1236 f 2nd 0xc8093b74 devfs (devfs) @ ufs/ffs/ffs_softdep.c:1866 5KDB: stack backtrace: cb02b0d b_trace_self_wrakpdpbe_rb(acc1k0tdr0a9c7e0(,c01,03d33,bcb180,bcb8c497ab,7637cd,,c.1.0.b)b ca9ta ,c758d6b2_0t0r,acc1e0_dscefldf0_,w.r.a.p)p eart+ 0x2d/frakmdeb _0bxafc5kctcr9a8c3e0+ 0x30/frame 0xf5ckbd0b3_1b8a cktrace(c10d3b9fw,ict8n0e9s3sb_7c4h,ecc1k0ocr7d9ebr5(,cc874578b67036c0,,9c,1c11002d0c4f7c,7.,.8.7)8 ,act8 47b7a8,.k.d.b)_ baatc ktrace+0wxi3t0n/efsrsa_mceh e0cxkfo5rcdce9r8+908x baa/frame 0xf5cbw0i3t6n8e ss_checkorder(c_8_0l9o3cbk7m4g,r9_,acr1g1s0(2c0834e7,b7743ac,,c88009130b0e,0c,8.4.7.b)7 aa8t, 0,0,...)w iattn ess_chec_k_olrocdkemrg+r0x_baarag/sf+r0axm8e1 60/xffr5acmce9 80ex8f 5cb0448 __lockmgr_args(cf8f0s9_3lbo7c4k,(8f054c0b00,4cd800,9c37b5e802,307,00,,c07,5c71f110e200,3ce7,57844a7)9 8at, c757f1e0_,_.l.o.c)k magtr _args+0xf8f1s6_/lforcakm+e0 x0axdf/5fcrca9m9ec 80 xf5cb0488 vop_stdlock(f5ccV9OaP4_0L,O6C3K21,_4A,PfV5(ccc192a23ec9,ecc0,af95dcabb074,d.0.,.c)1 0adta 0ac,180,vco1p2_4s3t2d3l0o,c.k.+.0)x 53a/tf rame 0xfV5OcPc_9L9OfC8K 1_APV+0x112/framVeO P0_xLfO5CcKb10_4AbP8V (c11f41c0,f5cc9a_4v0n,_4l,ofc5kc(cc9a86487,bc710284,38203100,0.,.c.1)0 dactf c7,8V7O8P,_cL1O0CdKc12_3AdP,V.+.0.x)1 1a2t/ frame 0x_fv5nc_cl9oac2k8+ 0xca/frame 0xf5c_bv0n4_fl8o ck(c8093b40,8040v0g,ect1(1c0824073be7,0784,a8,0f150c0c,9ca7ad8e,1.9.0.0), 5a7t, lock order reversal: _0 1st 0xc80b519c syncer (syncer) @ kern/vfs_subr.c:1787 v, 2nd 0xc84c9304 ufs (ufs) @ kern/vfs_subr.c:2168 .nKDB: stack backtrace: ._.l)o dcbak_t+t 0rxaccae/_vfsgreealtmf+e_0 wx0r7x7af/p5fpcrecar9m(aec6 1800 xdf059c7b00,583,0sc o1f1t2d3e0pc_2f,lv2uf4ss6h_,wh0oa,rs.kh.l_.ig)se tta((tcc 870da2d28b2b_ebtc0r,,af1c55ece_cds9eeablf,fc8_,0wc0r70ad0pe,p0ce67r0d+0e0,1x69220dd0/,,f0fr,5a.cm.be.0) 6 01ax0tf, 0.7.9.6)9s 3oa0ft t dep_flusvhfwsok_rdhkbals_ihbs_atgc+ek0ttx+r60aexc/feff(r/caf1mr0ead m30ebx 9f0f5x,cfcc589c4abca09853 50c4 ,fcf1s0_bsbycn9ca(f,cfc8s70_5av82g62e2et0cf0,(,1cc,71fd052dc8ccbf9bdb006,,81.,5.5e.0d)fe ,acbt1, b8a0e0c01k04d,,bf._5.bc.ab)c0 k6at1tr0 a,c1e,+.0f.xf.3s)0_ /safytrn acm+e0 x0f2xfefs50_/7vf9gr6ea9tm9fe8+ 00xx4f75/cfcr9abmw3ei0 t 0nxefs5sc_bc0h5ebdc8ok uonrmdoeurn(tc(8cs48oc0f9at32d02e4ep,c_9,s,8yc0n18c00_d0bc0uf0fc,(7c,c88767d68ee,09c6d80840,c,e9433987330,9,cf.09.a8.f,)91 9,a61t50 ,8.,.0.,)w. i.at.tn) e sast_ dcohuencmskooourfndtted+re0+px0_4xs3bycan/acf/_rfbarumafem+ e00 xx09fx75f7c0/c7f99rb6a99m0ee 8 0sxyfs5_cubn0m6o_2u_0nl to(cck7mdger0_6a0fr0fg,ssf_(5sccy8cn49ccc9vc3n80o,4dc,e08(e0cd58c06e065,e0c9,8cd418c8,92153,c7090,0,c,070,5,08.,2..4..4).0 ),a cta1 t3 c8f5cs,y._s.__.lu)on cmaoktum ngtr+_0axfrf3gscs_d+s/0yfxnr8cav1mn6eo/ df0erx+af0m5xec2 ca093xc/f4f00r7 a9m6ea c08x fs5ycsbc0a6l7l8( fff5sc_cl9odc0k8(f)ff 0sa7_t9t 6rbu5n0c,sa0yt,secc(7ac5l88l64+650efx982d,e82d,4/26f0,r0ca,7m05e,8 84085x0ff,85c,c7.c6.9c.c0)f7 c8a 0t, ...)X ianfttf 0sx_8l0o_fcsfkys+s_0ctxaraludln/(cf)ar taaemt+e 0 x06xcfeX0/i7fn9rt6a0bmx0e88 0 0_xsfy5sccba0l8l3V+0O0 xP2_1L/OfCrKa1m_euA fP0sVx_(fdc5i1cr2ce29nect9feecrc (,-cf-80-67 69se6y9bsd5c80a,,lc2l84 46(7,2bc271038f,5,f3 55Fccrb,e0ce81Bf2S04D,3 f2E53Lc0Fb,30.2b.,c. c)s, y0as,t_. u.n.m)o uVanOttP )_,L OeCiuKpf1 _s=A_ Pd0Vix+r20e8xn01tc1e12ra/+00fbx,r7 aeemeds /p0f xr=af m00e7x 9b06fxbbf3f58dc 6ba048,b 0e b_pv n=_ l0oxcbkf(bucff8ds47_c7m90a2 kd-e0-i,n-8o d0e5(0f05,ccb100bdbc8f,cf75,c8b708,bcc1c3)f 5a3t6 c,...) autf s_makein_ovdne_+l0oxc5k3+80/xfcraa/mfer a0mxef 50cxbf00a72946 b78 ufs_create(f5cb0vbg1e0t,(cc18142ca9d2cd10,,c81005f0f06,2c67,9c478d32080b,c102,b2e,,.c.8.0)a 2a0t1 0,...) autf s_createv+g0ext2+f0/xf7r7a/mfer a0mxef 50cxbf00a73986 bb0 VOP_CREATE_APV(cv1f2s2_em9seycn,cf(5cc8b00ab21000,0f,52c,b20,bec7cd,,fc58c0ba02a0a000,,f.5.c.b)0 aadt0 ,...) atv fs_msync+V0OxPa_0C/RfErAaTmEe_ A0PxVf+007x9162b4f/8f rame 0xf5cb0a68 sync_fsync(f0796vcna_8o,pce1n1_2cbr2eade(,f850c4b000b,8c01,0fd5ccfbc07b,f6cf,b1,8.0.,.0), ca7t6 c0780,c7sdy8n0cc_bf0s)y nact+ 0x12e/frvanm_eo p0exnf_0c7r9e6dc+200x 2cf/frame 0xf5cbV0ObP3_8F SYNC_APV(c120ed1v4n,_fo0p7e9n6(cfa58c,bc01b800d,cff5cc7b,06bffbc,,01,8.0.,.c)7 da8t0 cb0,c7deV1O9P0_0F,S.Y.N.C)_ AaPtV +0x124/frvanm_eo p0exnf+007x936bc/5f0r ame 0xf5cb0b58 sched_sync(0,f0796d08,c1k0ce6r0ne_0o,p3edn7a,t0(,c.7.d.e)1 9a0t0 ,ffffff9scc,h2e8dc_8s0y3n6c0+,00x,3a80f2/,f1r8a0m)e a0tx f0796ccck ern_openat+0x1efco/rfkr_aemxei t0(xcf05bc5bc08c5200, 0,f0796d08) at sys_opefno(rck7_deex1i9t0+00,xf75fc/bf0rcacm8e, c00xefd0c7e9560c,fc47 1fdc90,0f,o.r.k._)t raatm poline()s yast_ open+0x3f8o/rfkr_atmrea m0pxofl5icnbe0+c04x08 /frame 0xf0796cfs4y s-c-a-l lt(rfa5pc b00,d 0e8i)p a=t 0, esp =s y0sxcfa0l7l9+60dx420e,d /efbrpa m=e 00 x-f-5-c b0cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5cb0cfc --- syscall (5, FreeBSD ELF32, sys_open), eip = 0x284171e7, esp = 0xbfbfa09c, ebp = 0xbfbfa128 --- ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes May 8 16:15:59 x4 su: pho to root on /dev/pts/0 ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 49712 files, 69728209 used, 48585772 free (3252 frags, 6072815 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 49712 files, 69728209 used, 48585772 free (3252 frags, 6072815 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** fsck -y /tmp watchdogd. Starting inetd. Wed May 8 16:19:39 CEST 2013 FreeBSD/i386 (x4.osted.lan) (console) login: 20130508 16:19:40 all (1/17): nullfs.sh 20130508 16:23:28 all (2/17): nullfs10.sh 20130508 16:23:29 all (3/17): nullfs11-cache.log 20130508 16:23:30 all (4/17): nullfs11-nocache.log 20130508 16:23:30 all (5/17): nullfs11.sh 20130508 16:28:34 all (6/17): nullfs12.sh 20130508 16:28:36 all (7/17): nullfs13.sh 20130508 16:28:37 all (8/17): nullfs2.sh 20130508 16:40:21 all (9/17): nullfs3.sh 20130508 16:40:26 all (10/17): nullfs4.sh 20130508 16:40:27 all (11/17): nullfs5.sh Expensive timeout(9) function: 0xc0946a50(0xc78ebaa0) 0.213117417 s 20130508 16:52:39 all (12/17): nullfs6.sh 20130508 16:52:41 all (13/17): nullfs7.sh 20130508 16:52:43 all (14/17): nullfs8.sh lock order reversal: 1st 0xcdca7034 newnfs (newnfs) @ kern/vfs_syscalls.c:4064 2nd 0xe38e5718 bufwait (bufwait) @ kern/vfs_bio.c:2294 3rd 0xc8302034 newnfs (newnfs) @ kern/vfs_subr.c:2168 KDB: stack backtrace: db_trace_self_wrapper(c10d0970,c757fba0,c757f110,0,c129d1fc,...) at db_trace_self_wrapper+0x2d/frame 0xf5cff2c0 kdb_backtrace(c10d3bb8,c8302034,c10b664e,c7586b58,c10dcfd0,...) at kdb_backtrace+0x30/frame 0xf5cff328 witness_checkorder(c8302034,9,c10dcfc7,878,c83020a0,...) at witness_checkorder+0xbaa/frame 0xf5cff378 __lockmgr_args(c8302034,80100,c83020a0,0,0,0,c10dcfc7,878) at __lockmgr_args+0x816/frame 0xf5cff458 nfs_lock1(f5cff4d0,c13c5208,c757f1e0,c7584798,c757f1e0,...) at nfs_lock1+0x9c/frame 0xf5cff488 VOP_LOCK1_APV(c11f55e8,f5cff4d0,8,c110758c,c1243230,...) at VOP_LOCK1_APV+0x112/frame 0xf5cff4b8 _vn_lock(c8302000,80100,c10dcfc7,878,c10dc23d,...) at _vn_lock+0xca/frame 0xf5cff4f8 vget(c8302000,80100,c7df9900,57,0,...) at vget+0x77/frame 0xf5cff530 vfs_hash_get(c9313000,38824534,80000,c7df9900,f5cff598,...) at vfs_hash_get+0xff/frame 0xf5cff55c nfscl_nget(c9313000,cdca7000,c9e8f040,f5cff764,c7df9900,...) at nfscl_nget+0x92/frame 0xf5cff5a8 nfsrpc_readdirplus(cdca7000,f5cff958,f5cff878,c7ca2300,c7df9900,...) at nfsrpc_readdirplus+0xb14/frame 0xf5cff840 ncl_readdirplusrpc(cdca7000,f5cff958,c7ca2300,c7df9900,8,...) at ncl_readdirplusrpc+0x10f/frame 0xf5cff920 ncl_doio(cdca7000,e38e56b8,c7ca2300,c7df9900,0,...) at ncl_doio+0x105/frame 0xf5cff998 ncl_bioread(cdca7000,f5cffbb8,0,c7ca2300,0,...) at ncl_bioread+0x681/frame 0xf5cffa98 nfs_readdir(f5cffbe0,c112b4c2,c10ff790,c10de6f0,fe0,...) at nfs_readdir+0x19a/frame 0xf5cffb48 VOP_READDIR_APV(c11f55e8,f5cffbe0,c10de6f0,fe0,f5cffbdc,...) at VOP_READDIR_APV+0x124/frame 0xf5cffb78 kern_getdirentries(c7df9900,5,2888b000,1000,f5cffc34,...) at kern_getdirentries+0x1a2/frame 0xf5cffc10 sys_getdirentries(c7df9900,f5cffcc8,c0edce50,c71fdc90,0,...) at sys_getdirentries+0x41/frame 0xf5cffc40 syscall(f5cffd08) at syscall+0x2ed/frame 0xf5cffcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5cffcfc --- syscall (196, FreeBSD ELF32, sys_getdirentries), eip = 0x280e0dcb, esp = 0xbfbfd774, ebp = 0xbfbfd7a4 --- 20130508 16:56:35 all (15/17): nullfs9.sh 20130508 16:56:36 all (16/17): pread.sh 20130508 16:56:44 all (17/17): sendfile5.sh 20130508 16:57:57 all (1/17): nullfs.sh 20130508 17:02:09 all (2/17): nullfs10.sh 20130508 17:02:10 all (3/17): nullfs11-cache.log 20130508 17:02:11 all (4/17): nullfs11-nocache.log 20130508 17:02:11 all (5/17): nullfs11.sh 20130508 17:07:22 all (6/17): nullfs12.sh 20130508 17:07:23 all (7/17): nullfs13.sh 20130508 17:07:24 all (8/17): nullfs2.sh 20130508 17:19:03 all (9/17): nullfs3.sh 20130508 17:19:07 all (10/17): nullfs4.sh 20130508 17:19:07 all (11/17): nullfs5.sh 20130508 17:33:10 all (12/17): nullfs6.sh 20130508 17:33:12 all (13/17): nullfs7.sh 20130508 17:33:13 all (14/17): nullfs8.sh 20130508 17:37:07 all (15/17): nullfs9.sh 20130508 17:37:08 all (16/17): pread.sh 20130508 17:37:16 all (17/17): sendfile5.sh 20130508 17:38:27 all (1/17): nullfs.sh 20130508 17:42:34 all (2/17): nullfs10.sh 20130508 17:42:35 all (3/17): nullfs11-cache.log 20130508 17:42:36 all (4/17): nullfs11-nocache.log 20130508 17:42:36 all (5/17): nullfs11.sh 20130508 17:47:46 all (6/17): nullfs12.sh 20130508 17:47:48 all (7/17): nullfs13.sh 20130508 17:47:48 all (8/17): nullfs2.sh 20130508 17:59:56 all (9/17): nullfs3.sh 20130508 18:00:07 all (10/17): nullfs4.sh 20130508 18:00:08 all (11/17): nullfs5.sh 20130508 18:14:18 all (12/17): nullfs6.sh 20130508 18:14:20 all (13/17): nullfs7.sh 20130508 18:14:22 all (14/17): nullfs8.sh 20130508 18:18:20 all (15/17): nullfs9.sh 20130508 18:18:20 all (16/17): pread.sh 20130508 18:18:29 all (17/17): sendfile5.sh 20130508 18:19:40 all (1/17): nullfs.sh 20130508 18:23:52 all (2/17): nullfs10.sh 20130508 18:23:53 all (3/17): nullfs11-cache.log 20130508 18:23:54 all (4/17): nullfs11-nocache.log 20130508 18:23:54 all (5/17): nullfs11.sh 20130508 18:29:04 all (6/17): nullfs12.sh 20130508 18:29:06 all (7/17): nullfs13.sh 20130508 18:29:06 all (8/17): nullfs2.sh 20130508 18:40:07 all (9/17): nullfs3.sh 20130508 18:40:13 all (10/17): nullfs4.sh 20130508 18:40:14 all (11/17): nullfs5.sh 20130508 18:50:54 all (12/17): nullfs6.sh 20130508 18:50:56 all (13/17): nullfs7.sh 20130508 18:50:58 all (14/17): nullfs8.sh 20130508 18:54:48 all (15/17): nullfs9.sh 20130508 18:54:48 all (16/17): pread.sh 20130508 18:54:57 all (17/17): sendfile5.sh 20130508 18:56:09 all (1/17): nullfs.sh 20130508 19:00:20 all (2/17): nullfs10.sh 20130508 19:00:21 all (3/17): nullfs11-cache.log 20130508 19:00:21 all (4/17): nullfs11-nocache.log 20130508 19:00:22 all (5/17): nullfs11.sh 20130508 19:05:33 all (6/17): nullfs12.sh 20130508 19:05:34 all (7/17): nullfs13.sh 20130508 19:05:35 all (8/17): nullfs2.sh 20130508 19:18:39 all (9/17): nullfs3.sh 20130508 19:18:42 all (10/17): nullfs4.sh 20130508 19:18:43 all (11/17): nullfs5.sh Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x0 fault code = supervisor read, page not present instruction pointer = 0x20:0xc099d095 stack pointer = 0x28:0xf6268914 frame pointer = 0x28:0xf6268934 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 2500 (rm) [ thread pid 2500 tid 100444 ] Stopped at fifo_close+0x15: movl 0(%ebx),%edi db> run pho db:0:pho> bt Tracing pid 2500 tid 100444 td 0xd1a22300 fifo_close(f6268a78,c112aef6,d1a22300,cf8887a0,ccebe438,...) at fifo_close+0x15/frame 0xf6268934 VOP_CLOSE_APV(c122eb0c,f6268a78) at VOP_CLOSE_APV+0x124/frame 0xf6268964 VOP_CLOSE_AP(f6268a78,d07d2cf9,107,f62689d8,c0affe80,...) at VOP_CLOSE_AP+0x1b/frame 0xf6268974 null_bypass(f6268a78,c112aef6,1,ccebe508,ccebe524,...) at null_bypass+0xfd/frame 0xf6268a20 VOP_CLOSE_APV(d07d3200,f6268a78,0,0,1,...) at VOP_CLOSE_APV+0x12c/frame 0xf6268a50 vgonel(ccebe4e8,0,c10dcfc7,a89,ccebe438,...) at vgonel+0x1e7/frame 0xf6268a9c vgone(ccebe438,ccc25ca8,c844c5d8,f6268b08,c0b58078,...) at vgone+0x38/frame 0xf6268abc nullfs_unlink_lowervp(c844c5d8,ccc25ca8,c10dcfc7,ab3,c12437f0,...) at nullfs_unlink_lowervp+0x37/frame 0xf6268ad0 vfs_notify_upper(ccc25ca8,2,ccc25ca8,f6268bd4,2,...) at vfs_notify_upper+0x1e8/frame 0xf6268b08 kern_unlinkat(d1a22300,ffffff9c,28820e18,0,0) at kern_unlinkat+0x23d/frame 0xf6268c24 sys_unlink(d1a22300,f6268cc8,c0edce50,c1825c90,0,...) at sys_unlink+0x32/frame 0xf6268c40 syscall(f6268d08) at syscall+0x2ed/frame 0xf6268cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf6268cfc --- syscall (10, FreeBSD ELF32, sys_unlink), eip = 0x281744d7, esp = 0xbfbfd7dc, ebp = 0xbfbfd874 --- db:0:bt> show allpcpu Current CPU: 1 cpuid = 0 dynamic pcpu = 0x5dc800 curthread = 0xc76c5900: pid 11 "idle: cpu0" curpcb = 0xc720ad60 fpcurthread = none idlethread = 0xc76c5900: tid 100003 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: cpuid = 1 dynamic pcpu = 0x5fb1800 curthread = 0xd1a22300: pid 2500 "rm" curpcb = 0xf6268d60 fpcurthread = none idlethread = 0xc76c5600: tid 100004 "idle: cpu1" APIC ID = 1 currentldt = 0x50 spin locks held: cpuid = 2 dynamic pcpu = 0x5fb4800 curthread = 0xc93ae000: pid 2518 "sed" curpcb = 0xf6010d60 fpcurthread = none idlethread = 0xc76c5300: tid 100005 "idle: cpu2" APIC ID = 2 currentldt = 0x50 spin locks held: cpuid = 3 dynamic pcpu = 0x5fb7800 curthread = 0xc7de1900: pid 2517 "vmstat" curpcb = 0xf5cb0d60 fpcurthread = none idlethread = 0xc76c5000: tid 100006 "idle: cpu3" APIC ID = 3 currentldt = 0x50 spin locks held: db:0:allpcpu> show alllocks Process 2518 (sed) thread 0xc93ae000 (100293) shared sx vm map (user) (vm map (user)) r = 0 (0xc7adeec8) locked @ vm/vm_map.c:3764 Process 2517 (vmstat) thread 0xc7de1900 (100092) exclusive sleep mutex pmap (pmap) r = 0 (0xcb9ca8d8) locked @ i386/i386/pmap.c:4393 exclusive rw pmap pv global (pmap pv global) r = 0 (0xc140c600) locked @ i386/i386/pmap.c:4392 Process 2500 (rm) thread 0xd1a22300 (100444) exclusive lockmgr ufs (ufs) r = 0 (0xccc25cdc) locked @ kern/vfs_subr.c:2168 exclusive lockmgr ufs (ufs) r = 0 (0xd7624034) locked @ kern/vfs_lookup.c:517 db:0:alllocks> show lockedvnods Locked vnodes 0xd7624000: tag ufs, type VDIR usecount 4, writecount 0, refcount 5 mountedhere 0 flags (VI_ACTIVE) v_object 0xce047784 ref 0 pages 0 lock type ufs: EXCL by thread 0xd1a22300 (pid 2500, rm, tid 100444) #0 0xc0a95173 at __lockmgr_args+0xf63 #1 0xc0d1938d at ffs_lock+0xad #2 0xc0efc4e2 at VOP_LOCK1_APV+0x112 #3 0xc0b67dca at _vn_lock+0xca #4 0xc0b4a549 at lookup+0xf9 #5 0xc0b4a06d at namei+0x47d #6 0xc0b6138c at kern_unlinkat+0xbc #7 0xc0b60fe2 at sys_unlink+0x32 #8 0xc0ed656d at syscall+0x2ed #9 0xc0ebeae1 at Xint0x80_syscall+0x21 ino 3061760, on dev label/tmp 0xccc25ca8: tag ufs, type VFIFO usecount 2, writecount 0, refcount 2 mountedhere 0 flags (VI_ACTIVE) lock type ufs: EXCL by thread 0xd1a22300 (pid 2500, rm, tid 100444) #0 0xc0a952bf at __lockmgr_args+0x10af #1 0xc0d1938d at ffs_lock+0xad #2 0xc0efc4e2 at VOP_LOCK1_APV+0x112 #3 0xc0b67dca at _vn_lock+0xca #4 0xc0b55937 at vget+0x77 #5 0xc0b4910f at vfs_hash_get+0xff #6 0xc0d13bf7 at ffs_vgetf+0x47 #7 0xc0d13b9e at ffs_vget+0x2e #8 0xc0d22bfe at ufs_lookup_ino+0xeee #9 0xc0d21d0a at ufs_lookup+0x2a #10 0xc0ef90f4 at VOP_CACHEDLOOKUP_APV+0x124 #11 0xc0b41b70 at vfs_cache_lookup+0xf0 #12 0xc0ef8f04 at VOP_LOOKUP_APV+0x124 #13 0xc0b4a978 at lookup+0x528 #14 0xc0b4a06d at namei+0x47d #15 0xc0b6138c at kern_unlinkat+0xbc #16 0xc0b60fe2 at sys_unlink+0x32 #17 0xc0ed656d at syscall+0x2ed ino 3063745, on dev label/tmp, NULL v_fifoinfo 0xcd2d8438: tag null, type VDIR usecount 0, writecount 0, refcount 0 mountedhere 0 flags (VI_FREE) v_object 0xce047784 ref 0 pages 0 lock type ufs: EXCL by thread 0xd1a22300 (pid 2500, rm, tid 100444) #0 0xc0a95173 at __lockmgr_args+0xf63 #1 0xc0d1938d at ffs_lock+0xad #2 0xc0efc4e2 at VOP_LOCK1_APV+0x112 #3 0xc0b67dca at _vn_lock+0xca #4 0xc0b4a549 at lookup+0xf9 #5 0xc0b4a06d at namei+0x47d #6 0xc0b6138c at kern_unlinkat+0xbc #7 0xc0b60fe2 at sys_unlink+0x32 #8 0xc0ed656d at syscall+0x2ed #9 0xc0ebeae1 at Xint0x80_syscall+0x21 vp=0xcd2d8438, lowervp=0xd7624000 0xccebe438: tag null, type VFIFO usecount 1, writecount 0, refcount 1 mountedhere 0 flags (VI_DOOMED|VI_ACTIVE) lock type ufs: EXCL by thread 0xd1a22300 (pid 2500, rm, tid 100444) #0 0xc0a952bf at __lockmgr_args+0x10af #1 0xc0d1938d at ffs_lock+0xad #2 0xc0efc4e2 at VOP_LOCK1_APV+0x112 #3 0xc0b67dca at _vn_lock+0xca #4 0xc0b55937 at vget+0x77 #5 0xc0b4910f at vfs_hash_get+0xff #6 0xc0d13bf7 at ffs_vgetf+0x47 #7 0xc0d13b9e at ffs_vget+0x2e #8 0xc0d22bfe at ufs_lookup_ino+0xeee #9 0xc0d21d0a at ufs_lookup+0x2a #10 0xc0ef90f4 at VOP_CACHEDLOOKUP_APV+0x124 #11 0xc0b41b70 at vfs_cache_lookup+0xf0 #12 0xc0ef8f04 at VOP_LOOKUP_APV+0x124 #13 0xc0b4a978 at lookup+0x528 #14 0xc0b4a06d at namei+0x47d #15 0xc0b6138c at kern_unlinkat+0xbc #16 0xc0b60fe2 at sys_unlink+0x32 #17 0xc0ed656d at syscall+0x2ed vp=0xccebe438, lowervp=0xccc25ca8 db:0:lockedvnods> show mount 0xc7d27bb0 /dev/ad4s1a on / (ufs) 0xc7d28000 devfs on /dev (devfs) 0xc80a25d8 /dev/ad4s1f on /home (ufs) 0xc80a2000 /dev/ad4s1d on /usr (ufs) 0xc7d28bb0 /dev/ad4s1e on /var (ufs) 0xc7d288c4 procfs on /proc (procfs) 0xc8f595d8 /dev/label/tmp on /tmp (ufs) 0xc844c5d8 /tmp on /home/pho/stress2/misc/mp1 (nullfs) More info: show mount db:0:mount> ps pid ppid pgrp uid state wmesg wchan cmd 2519 1082 1081 1001 S piperd 0xc94adb28 awk 2518 1082 1081 1001 RL CPU 2 sed 2517 1082 1081 1001 RE CPU 3 vmstat 2500 2488 2488 0 R+ CPU 1 rm 2488 796 2488 0 S+ wait 0xc93a52e8 sh 32115 32033 1061 0 T+ sh 32033 1061 1061 0 T+ sh 21764 21763 21764 1001 Ss+ ttyin 0xc9348270 bash 21763 21572 21572 1001 S select 0xd0f6d7a4 sshd 21572 726 21572 0 Ss select 0xd1cfaae4 sshd 1239 1 1239 0 Ss+ ttyin 0xc75a4670 getty 1238 1 1238 0 Ss+ ttyin 0xc75a4870 getty 1237 1 1237 0 Ss+ ttyin 0xc75a4a70 getty 1236 1 1236 0 Ss+ ttyin 0xc75a4c70 getty 1235 1 1235 0 Ss+ ttyin 0xc75a4e70 getty 1234 1 1234 0 Ss+ ttyin 0xc792e070 getty 1233 1 1233 0 Ss+ ttyin 0xc792e270 getty 1232 1 1232 0 Ss+ ttyin 0xc792e470 getty 1231 1 1231 0 Ss+ ttyin 0xc75a3070 getty 1214 1 1214 0 Ss select 0xc7d8e1a4 inetd 1190 1 1190 0 Ss nanslp 0xc127e458 watchdogd 1132 1108 1132 1001 S+ nanslp 0xc127e459 gnu-watch 1108 1107 1108 1001 SWs+ wait 0xc8ab98b8 bash 1107 1105 1105 1001 S select 0xc8abfba4 sshd 1105 726 1105 0 Ss select 0xc7d9f4e4 sshd 1089 1088 1089 1001 Ds swread 0xc2501a00 tail 1088 1076 1076 1001 S select 0xc7d618a4 sshd 1083 1081 1081 1001 S piperd 0xc7d82660 awk 1082 1081 1081 1001 S wait 0xc8ab9ba0 sh 1081 1080 1081 1001 SWs wait 0xc8f535d0 sh 1080 1073 1073 1001 S select 0xc8abfb24 sshd 1079 1078 1079 1001 Ss+ select 0xc757b624 top 1078 1072 1072 1001 S select 0xc757b4a4 sshd 1076 726 1076 0 Ss select 0xc7d9f6e4 sshd 1073 726 1073 0 Ss select 0xc8abf5a4 sshd 1072 726 1072 0 Ss select 0xc757bd24 sshd 1061 796 1061 0 T+ sh 796 793 796 0 S+ wait 0xc7ddaba0 bash 793 792 793 0 SW+ pause 0xc7df791c csh 792 787 792 1001 SW+ wait 0xc845c8b8 su 787 786 787 1001 SWs+ wait 0xc7dd85d0 bash 786 770 770 1001 S select 0xc7d8ea64 sshd 770 726 770 0 Ss select 0xc757be24 sshd 737 1 737 0 Ss nanslp 0xc127e459 cron 732 1 732 25 SWs pause 0xc7dd891c sendmail 729 1 729 0 Ss select 0xc7d61824 sendmail 726 1 726 0 Ss select 0xc7d619e4 sshd 694 1 694 0 Ss select 0xc7d8eca4 moused 686 1 685 0 SW nanslp 0xc127e459 smartd 684 672 672 0 S select 0xc757b4e4 winbindd 683 670 670 0 S select 0xc757a064 smbd 672 1 672 0 Ss select 0xc757a124 winbindd 670 1 670 0 Ss select 0xc7d8e724 smbd 668 1 668 0 Ss select 0xc7d61024 nmbd 644 1 644 0 Ss select 0xc7d61924 ntpd 587 583 583 0 S (threaded) nfsd 100114 S rpcsvc 0xc7c02390 nfsd: service 100113 S rpcsvc 0xc7c02350 nfsd: service 100112 S rpcsvc 0xc7c02310 nfsd: service 100087 S rpcsvc 0xc7d02390 nfsd: master 583 1 583 0 Ss select 0xc7d8e024 nfsd 577 1 577 0 Ss select 0xc757bbe4 mountd 570 566 566 0 S select 0xc757bb64 nfsuserd 569 566 566 0 S select 0xc7d612a4 nfsuserd 568 566 566 0 S select 0xc7d9f064 nfsuserd 567 566 566 0 S select 0xc7d612e4 nfsuserd 566 1 566 0 SWs pause 0xc7d6d91c nfsuserd 538 1 538 0 Ss select 0xc7d613a4 rpcbind 523 1 523 0 Ss select 0xc7d8e5a4 syslogd 420 1 420 0 Ss select 0xc757b1a4 devd 17 0 0 0 DL sdflush 0xc13f95e4 [softdepflush] 16 0 0 0 DL syncer 0xc13f53a4 [syncer] 9 0 0 0 DL vlruwt 0xc7c2aba0 [vnlru] 8 0 0 0 DL psleep 0xc13f511c [bufdaemon] 7 0 0 0 DL pgzero 0xc13f9ee4 [pagezero] 6 0 0 0 DL psleep 0xc13f9aac [vmdaemon] 5 0 0 0 DL psleep 0xc144d984 [pagedaemon] 4 0 0 0 DL ccb_scan 0xc1249524 [xpt_thrd] 3 0 0 0 DL waiting_ 0xc144968c [sctp_iterator] 2 0 0 0 DL - 0xc789303c [fdc0] 15 0 0 0 DL (threaded) [usb] 100058 D - 0xc78f7d34 [usbus5] 100057 D - 0xc78f7d04 [usbus5] 100056 D - 0xc78f7cd4 [usbus5] 100055 D - 0xc78f7ca4 [usbus5] 100053 D - 0xc78ebb5c [usbus4] 100052 D - 0xc78ebb2c [usbus4] 100051 D - 0xc78ebafc [usbus4] 100050 D - 0xc78ebacc [usbus4] 100049 D - 0xc78e6b5c [usbus3] 100048 D - 0xc78e6b2c [usbus3] 100047 D - 0xc78e6afc [usbus3] 100046 D - 0xc78e6acc [usbus3] 100045 D - 0xc78deb5c [usbus2] 100044 D - 0xc78deb2c [usbus2] 100043 D - 0xc78deafc [usbus2] 100042 D - 0xc78deacc [usbus2] 100040 D - 0xc78d9b5c [usbus1] 100039 D - 0xc78d9b2c [usbus1] 100038 D - 0xc78d9afc [usbus1] 100037 D - 0xc78d9acc [usbus1] 100035 D - 0xc78cdb5c [usbus0] 100034 D - 0xc78cdb2c [usbus0] 100033 D - 0xc78cdafc [usbus0] 100032 D - 0xc78cdacc [usbus0] 14 0 0 0 DL - 0xc12640dc [yarrow] 13 0 0 0 DL (threaded) [geom] 100016 D - 0xc14456e8 [g_down] 100015 D - 0xc14456e4 [g_up] 100014 D - 0xc14456e0 [g_event] 12 0 0 0 WL (threaded) [intr] 100063 I [irq12: psm0] 100062 I [irq1: atkbd0] 100060 I [swi0: uart] 100059 I [irq14: ata0] 100054 I [irq19: ehci0] 100041 I [irq18: ohci2 ohci4] 100036 I [irq17: ohci1 ohci3] 100031 I [irq16: hdac1 ohci0] 100030 I [irq22: ahci0] 100029 I [irq257: re0] 100028 I [irq256: hdac0] 100027 I [swi6: task queue] 100022 I [swi2: cambio] 100020 I [swi5: fast taskq] 100018 I [swi6: Giant taskq] 100012 I [swi3: vm] 100011 I [swi1: netisr 0] 100010 I [swi4: clock] 100009 I [swi4: clock] 100008 I [swi4: clock] 100007 I [swi4: clock] 11 0 0 0 RL (threaded) [idle] 100006 CanRun [idle: cpu3] 100005 CanRun [idle: cpu2] 100004 CanRun [idle: cpu1] 100003 Run CPU 0 [idle: cpu0] 1 0 1 0 SLs wait 0xc76c3ba0 [init] 10 0 0 0 DL audit_wo 0xc144c438 [audit] 0 0 0 0 DLs (threaded) [kernel] 100065 D - 0xc127d350 [deadlkres] 100064 D - 0xc7867d80 [mca taskq] 100026 D - 0xc75a8600 [ffs_trim taskq] 100025 D - 0xc75a8680 [acpi_task_2] 100024 D - 0xc75a8680 [acpi_task_1] 100023 D - 0xc75a8680 [acpi_task_0] 100021 D - 0xc75a8a00 [kqueue taskq] 100019 D - 0xc75a8b00 [thread taskq] 100013 D - 0xc75a7100 [firmware taskq] 100000 D sched 0xc1445764 [swapper] 32163 32115 1061 0 Z+ run 32054 32033 1061 0 Z+ kinfo 32055 32033 1061 0 Z+ kinfo 32051 32033 1061 0 Z+ kinfo 32052 32033 1061 0 Z+ kinfo db:0:ps> allt Tracing command awk pid 2519 tid 100370 td 0xd15ce600 sched_switch(d15ce600,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf60f7a1c mi_switch(104,0,c10d22d1,1f5,c10d22d1,...) at mi_switch+0x1c9/frame 0xf60f7a50 sleepq_switch(d15ce600,0,c10d22d1,1ab,f60f7ac8,...) at sleepq_switch+0x1b9/frame 0xf60f7a84 sleepq_catch_signals(0,c10d22d1,162,16c8,d15ce600,...) at sleepq_catch_signals+0x361/frame 0xf60f7acc sleepq_wait_sig(c94adb28,5c,c10d525a,100,0,...) at sleepq_wait_sig+0x14/frame 0xf60f7af0 _sleep(c94adb28,c94adca8,15c,c10d525a,0,...) at _sleep+0x35b/frame 0xf60f7b4c pipe_read(c7da0540,f60f7c10,c7cdb900,0,d15ce600,...) at pipe_read+0x2dc/frame 0xf60f7b90 dofileread(c7da0540,f60f7c10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf60f7bc4 kern_readv(d15ce600,0,f60f7c10,0,28855000,...) at kern_readv+0x66/frame 0xf60f7bf0 sys_read(d15ce600,f60f7cc8,c0edce50,c1825c90,0,...) at sys_read+0x5c/frame 0xf60f7c40 syscall(f60f7d08) at syscall+0x2ed/frame 0xf60f7cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf60f7cfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281c2077, esp = 0xbfbfdb70, ebp = 0xbfbfdb84 --- Tracing command sed pid 2518 tid 100293 td 0xc93ae000 cpustop_handler(2,13,f601087c,c0ed515b,2bfd48,...) at cpustop_handler+0x2f/frame 0xf6010730 ipi_nmi_handler(2bfd48,0,0,f6010784,c0a7515b,...) at ipi_nmi_handler+0x37/frame 0xf6010740 trap(f6010888) at trap+0x3b/frame 0xf601087c calltrap() at calltrap+0x6/frame 0xf601087c --- trap 0x13, eip = 0xc0aafd68, esp = 0xf60108c8, ebp = 0xf6010940 --- __rw_wlock_hard(c140c610,c93ae000,c11230c2,d66,0,...) at __rw_wlock_hard+0x238/frame 0xf6010940 _rw_wlock_cookie(c140c610,c11230c2,d66,c93ae000,cdb2e260,...) at _rw_wlock_cookie+0xdd/frame 0xf6010970 pmap_enter(c7adef30,2881c000,1,c3774ec0,3,...) at pmap_enter+0x7b/frame 0xf60109c0 vm_fault_hold(c7adee80,2881c000,1,0,0,...) at vm_fault_hold+0x1a67/frame 0xf6010b20 vm_fault(c7adee80,2881c000,1,0,c7adee80,...) at vm_fault+0x82/frame 0xf6010b48 trap_pfault(2881c000,0,0,f6010c04,c0a7515b,...) at trap_pfault+0x1ba/frame 0xf6010bc0 trap(f6010d08) at trap+0x5da/frame 0xf6010cfc calltrap() at calltrap+0x6/frame 0xf6010cfc --- trap 0xc, eip = 0x280f8078, esp = 0xbfbfd49c, ebp = 0xbfbfd4e8 --- Tracing command vmstat pid 2517 tid 100092 td 0xc7de1900 cpustop_handler(3,13,f5cb0a7c,c0ed515b,c7de1bdc,...) at cpustop_handler+0x2f/frame 0xf5cb0930 ipi_nmi_handler(c7de1bdc,3,297b30,0,0,...) at ipi_nmi_handler+0x37/frame 0xf5cb0940 trap(f5cb0a88) at trap+0x3b/frame 0xf5cb0a7c calltrap() at calltrap+0x6/frame 0xf5cb0a7c --- trap 0x13, eip = 0xc0affae3, esp = 0xf5cb0ac8, ebp = 0xf5cb0ac8 --- witness_checkorder(c144d900,9,c1109ac8,782,0,...) at witness_checkorder+0x3/frame 0xf5cb0ac8 __mtx_lock_flags(c144d910,0,c1109ac8,782,c503b8c0,...) at __mtx_lock_flags+0x98/frame 0xf5cb0afc vm_page_free_toq(c503b8c0,0,cb9ca8d8,c42f3f40,cb9ca8d8,...) at vm_page_free_toq+0x14b/frame 0xf5cb0b18 free_pv_chunk(8657e000,0,c11230c2,1129,7dc,...) at free_pv_chunk+0xfe/frame 0xf5cb0b34 pmap_remove_pages(cb9ca8d8,140,4,cb9ca8d8,c7dda5d0,...) at pmap_remove_pages+0x5b5/frame 0xf5cb0ba8 vmspace_exit(c7de1900,0,c10c5e5c,140,c7de1900,...) at vmspace_exit+0xb1/frame 0xf5cb0bd0 exit1(c7de1900,0,f5cb0cfc,c0ed656d,c7de1900,...) at exit1+0x644/frame 0xf5cb0c30 sys_sys_exit(c7de1900,f5cb0cc8,c0edce50,c7200c90,0,...) at sys_sys_exit+0x1d/frame 0xf5cb0c40 syscall(f5cb0d08) at syscall+0x2ed/frame 0xf5cb0cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5cb0cfc --- syscall (1, FreeBSD ELF32, sys_sys_exit), eip = 0x2814d8a3, esp = 0xbfbfd308, ebp = 0xbfbfd318 --- Tracing command rm pid 2500 tid 100444 td 0xd1a22300 fifo_close(f6268a78,c112aef6,d1a22300,cf8887a0,ccebe438,...) at fifo_close+0x15/frame 0xf6268934 VOP_CLOSE_APV(c122eb0c,f6268a78) at VOP_CLOSE_APV+0x124/frame 0xf6268964 VOP_CLOSE_AP(f6268a78,d07d2cf9,107,f62689d8,c0affe80,...) at VOP_CLOSE_AP+0x1b/frame 0xf6268974 null_bypass(f6268a78,c112aef6,1,ccebe508,ccebe524,...) at null_bypass+0xfd/frame 0xf6268a20 VOP_CLOSE_APV(d07d3200,f6268a78,0,0,1,...) at VOP_CLOSE_APV+0x12c/frame 0xf6268a50 vgonel(ccebe4e8,0,c10dcfc7,a89,ccebe438,...) at vgonel+0x1e7/frame 0xf6268a9c vgone(ccebe438,ccc25ca8,c844c5d8,f6268b08,c0b58078,...) at vgone+0x38/frame 0xf6268abc nullfs_unlink_lowervp(c844c5d8,ccc25ca8,c10dcfc7,ab3,c12437f0,...) at nullfs_unlink_lowervp+0x37/frame 0xf6268ad0 vfs_notify_upper(ccc25ca8,2,ccc25ca8,f6268bd4,2,...) at vfs_notify_upper+0x1e8/frame 0xf6268b08 kern_unlinkat(d1a22300,ffffff9c,28820e18,0,0) at kern_unlinkat+0x23d/frame 0xf6268c24 sys_unlink(d1a22300,f6268cc8,c0edce50,c1825c90,0,...) at sys_unlink+0x32/frame 0xf6268c40 syscall(f6268d08) at syscall+0x2ed/frame 0xf6268cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf6268cfc --- syscall (10, FreeBSD ELF32, sys_unlink), eip = 0x281744d7, esp = 0xbfbfd7dc, ebp = 0xbfbfd874 --- Tracing command sh pid 2488 tid 100284 td 0xc93afc00 sched_switch(c93afc00,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5ff598c mi_switch(104,0,c10d22d1,1f5,c10d22d1,...) at mi_switch+0x1c9/frame 0xf5ff59c0 sleepq_switch(c93afc00,0,c10d22d1,1ab,f5ff5a38,...) at sleepq_switch+0x1b9/frame 0xf5ff59f4 sleepq_catch_signals(0,c10d22d1,162,1170,c93afc00,...) at sleepq_catch_signals+0x361/frame 0xf5ff5a3c sleepq_wait_sig(c93a52e8,6c,c10d5749,100,0,...) at sleepq_wait_sig+0x14/frame 0xf5ff5a60 _sleep(c93a52e8,c93a537c,16c,c10d5749,0,...) at _sleep+0x35b/frame 0xf5ff5abc kern_wait6(c93afc00,7,0,0,f5ff5b4c,...) at kern_wait6+0x45c/frame 0xf5ff5b20 sys_wait4(c93afc00,f5ff5cc8,c11245ef,e9,c93afc00,...) at sys_wait4+0x94/frame 0xf5ff5c40 syscall(f5ff5d08) at syscall+0x2ed/frame 0xf5ff5cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5ff5cfc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x28152563, esp = 0xbfbfd7d8, ebp = 0xbfbfd7f0 --- Tracing command sh pid 32115 tid 100253 td 0xc9344300 sched_switch(c9344300,0,20a,1b5,c10cd987,...) at sched_switch+0x4b9/frame 0xf5f7bc6c mi_switch(20a,0,c10cd987,34e,c933f8d8,...) at mi_switch+0x1c9/frame 0xf5f7bca0 thread_suspend_check(0,0,c10d2e7f,118,c140f1a8,...) at thread_suspend_check+0x2a5/frame 0xf5f7bccc ast(f5f7bd08) at ast+0x457/frame 0xf5f7bcfc doreti_ast() at doreti_ast+0x1b/frame 0xbfbfd5e0 Tracing command sh pid 32033 tid 100461 td 0xd0d43600 sched_switch(d0d43600,0,20a,1b5,c10cd987,...) at sched_switch+0x4b9/frame 0xf629bc6c mi_switch(20a,0,c10cd987,34e,ca087020,...) at mi_switch+0x1c9/frame 0xf629bca0 thread_suspend_check(0,0,c10d2e7f,118,c140f1a8,...) at thread_suspend_check+0x2a5/frame 0xf629bccc ast(f629bd08) at ast+0x457/frame 0xf629bcfc doreti_ast() at doreti_ast+0x1b/frame 0xbfbfd8c0 Tracing command bash pid 21764 tid 100278 td 0xc7df9000 sched_switch(c7df9000,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5fe39bc mi_switch(104,0,c10d22d1,1f5,46,...) at mi_switch+0x1c9/frame 0xf5fe39f0 sleepq_switch(c7df9000,0,c10d22d1,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf5fe3a24 sleepq_catch_signals(f5fe3a90,c0a9d439,c9348214,0,0,...) at sleepq_catch_signals+0x361/frame 0xf5fe3a6c sleepq_wait_sig(c9348270,0,f5fe3ac0,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5fe3a90 _cv_wait_sig(c9348270,c9348204,c10d7041,531,f5fe3c10,...) at _cv_wait_sig+0x200/frame 0xf5fe3ad4 tty_wait(c9348200,c9348270,f5fe3c10,1,0,...) at tty_wait+0x4b/frame 0xf5fe3af8 ttydisc_read(c9348200,f5fe3c10,0,a1,f5fe3c10,...) at ttydisc_read+0x377/frame 0xf5fe3b3c ttydev_read(c85b7d00,f5fe3c10,0,1,0,...) at ttydev_read+0x81/frame 0xf5fe3b5c devfs_read_f(c93863f0,f5fe3c10,c9455600,0,c7df9000,...) at devfs_read_f+0x87/frame 0xf5fe3b90 dofileread(c93863f0,f5fe3c10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf5fe3bc4 kern_readv(c7df9000,0,f5fe3c10,0,bfbfd00f,...) at kern_readv+0x66/frame 0xf5fe3bf0 sys_read(c7df9000,f5fe3cc8,14,c10ceb5d,7b3,...) at sys_read+0x5c/frame 0xf5fe3c40 syscall(f5fe3d08) at syscall+0x2ed/frame 0xf5fe3cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5fe3cfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x28360077, esp = 0xbfbfcfec, ebp = 0xbfbfd018 --- Tracing command sshd pid 21763 tid 100204 td 0xc9277000 sched_switch(c9277000,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5e9790c mi_switch(104,0,c10d22d1,1f5,200046,...) at mi_switch+0x1c9/frame 0xf5e97940 sleepq_switch(c9277000,0,c10d22d1,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf5e97974 sleepq_catch_signals(f5e979e0,c0a9d439,d0f6d7a0,0,0,...) at sleepq_catch_signals+0x361/frame 0xf5e979bc sleepq_wait_sig(d0f6d7a4,0,f5e97a10,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5e979e0 _cv_wait_sig(d0f6d7a4,d0f6d790,c10d5015,68e,1b2,...) at _cv_wait_sig+0x200/frame 0xf5e97a24 seltdwait(ffffffff,0,0,c9277000,0,...) at seltdwait+0x90/frame 0xf5e97a58 kern_select(c9277000,c,28c0c1c8,28c0c1d0,0,...) at kern_select+0x88c/frame 0xf5e97bf8 sys_select(c9277000,f5e97cc8,8,c10ccc3c,db,...) at sys_select+0x69/frame 0xf5e97c40 syscall(f5e97d08) at syscall+0x2ed/frame 0xf5e97cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5e97cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28531037, esp = 0xbfbfd374, ebp = 0xbfbfd3c0 --- Tracing command sshd pid 21572 tid 100244 td 0xc9304300 sched_switch(c9304300,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5f54984 mi_switch(104,0,c10d22d1,1f5,200046,...) at mi_switch+0x1c9/frame 0xf5f549b8 sleepq_switch(c9304300,0,c10d22d1,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf5f549ec sleepq_catch_signals(f5f54a58,c0a9d439,d1cfaae0,0,0,...) at sleepq_catch_signals+0x361/frame 0xf5f54a34 sleepq_wait_sig(d1cfaae4,0,f5f54a88,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5f54a58 _cv_wait_sig(d1cfaae4,d1cfaad0,c10d5015,68e,5c3,...) at _cv_wait_sig+0x200/frame 0xf5f54a9c seltdwait(ffffffff,0,0,c9304300,c1206540,...) at seltdwait+0x90/frame 0xf5f54ad0 sys_poll(c9304300,f5f54cc8,c0edce50,c71fac90,0,...) at sys_poll+0x44c/frame 0xf5f54c40 syscall(f5f54d08) at syscall+0x2ed/frame 0xf5f54cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5f54cfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x284c738f, esp = 0xbfbfd3bc, ebp = 0xbfbfd408 --- Tracing command getty pid 1239 tid 100165 td 0xc90cc600 sched_switch(c90cc600,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5e229bc mi_switch(104,0,c10d22d1,1f5,c10d22d1,...) at mi_switch+0x1c9/frame 0xf5e229f0 sleepq_switch(c90cc600,0,c10d22d1,1ab,f5e22a68,...) at sleepq_switch+0x1b9/frame 0xf5e22a24 sleepq_catch_signals(0,c10d22d1,162,510,1,...) at sleepq_catch_signals+0x361/frame 0xf5e22a6c sleepq_wait_sig(c75a4670,0,c10d7055,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5e22a90 _cv_wait_sig(c75a4670,c1446098,c10d7041,531,f5e22c10,...) at _cv_wait_sig+0x200/frame 0xf5e22ad4 tty_wait(c75a4600,c75a4670,f5e22c10,1,0,...) at tty_wait+0x4b/frame 0xf5e22af8 ttydisc_read(c75a4600,f5e22c10,0,a1,f5e22c10,...) at ttydisc_read+0x377/frame 0xf5e22b3c ttydev_read(c7909500,f5e22c10,0,1,0,...) at ttydev_read+0x81/frame 0xf5e22b5c devfs_read_f(c7d41d90,f5e22c10,c76bfe80,0,c90cc600,...) at devfs_read_f+0x87/frame 0xf5e22b90 dofileread(c7d41d90,f5e22c10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf5e22bc4 kern_readv(c90cc600,0,f5e22c10,0,bfbfddfb,...) at kern_readv+0x66/frame 0xf5e22bf0 sys_read(c90cc600,f5e22cc8,c11245ef,d6,0,...) at sys_read+0x5c/frame 0xf5e22c40 syscall(f5e22d08) at syscall+0x2ed/frame 0xf5e22cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5e22cfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2819d077, esp = 0xbfbfdde0, ebp = 0xbfbfde08 --- Tracing command getty pid 1238 tid 100086 td 0xc7de1c00 sched_switch(c7de1c00,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5c969bc mi_switch(104,0,c10d22d1,1f5,c10d22d1,...) at mi_switch+0x1c9/frame 0xf5c969f0 sleepq_switch(c7de1c00,0,c10d22d1,1ab,f5c96a68,...) at sleepq_switch+0x1b9/frame 0xf5c96a24 sleepq_catch_signals(0,c10d22d1,162,540,1,...) at sleepq_catch_signals+0x361/frame 0xf5c96a6c sleepq_wait_sig(c75a4870,0,c10d7055,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5c96a90 _cv_wait_sig(c75a4870,c1446098,c10d7041,531,f5c96c10,...) at _cv_wait_sig+0x200/frame 0xf5c96ad4 tty_wait(c75a4800,c75a4870,f5c96c10,1,0,...) at tty_wait+0x4b/frame 0xf5c96af8 ttydisc_read(c75a4800,f5c96c10,0,a1,f5c96c10,...) at ttydisc_read+0x377/frame 0xf5c96b3c ttydev_read(c7909600,f5c96c10,0,1,0,...) at ttydev_read+0x81/frame 0xf5c96b5c devfs_read_f(c7da0bd0,f5c96c10,c76bfe80,0,c7de1c00,...) at devfs_read_f+0x87/frame 0xf5c96b90 dofileread(c7da0bd0,f5c96c10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf5c96bc4 kern_readv(c7de1c00,0,f5c96c10,0,bfbfddfb,...) at kern_readv+0x66/frame 0xf5c96bf0 sys_read(c7de1c00,f5c96cc8,c11245ef,d6,0,...) at sys_read+0x5c/frame 0xf5c96c40 syscall(f5c96d08) at syscall+0x2ed/frame 0xf5c96cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5c96cfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2819d077, esp = 0xbfbfdde0, ebp = 0xbfbfde08 --- Tracing command getty pid 1237 tid 100162 td 0xc90cd000 sched_switch(c90cd000,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5e199bc mi_switch(104,0,c10d22d1,1f5,c10d22d1,...) at mi_switch+0x1c9/frame 0xf5e199f0 sleepq_switch(c90cd000,0,c10d22d1,1ab,f5e19a68,...) at sleepq_switch+0x1b9/frame 0xf5e19a24 sleepq_catch_signals(0,c10d22d1,162,570,1,...) at sleepq_catch_signals+0x361/frame 0xf5e19a6c sleepq_wait_sig(c75a4a70,0,c10d7055,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5e19a90 _cv_wait_sig(c75a4a70,c1446098,c10d7041,531,f5e19c10,...) at _cv_wait_sig+0x200/frame 0xf5e19ad4 tty_wait(c75a4a00,c75a4a70,f5e19c10,1,0,...) at tty_wait+0x4b/frame 0xf5e19af8 ttydisc_read(c75a4a00,f5e19c10,0,a1,f5e19c10,...) at ttydisc_read+0x377/frame 0xf5e19b3c ttydev_read(c7909700,f5e19c10,0,1,0,...) at ttydev_read+0x81/frame 0xf5e19b5c devfs_read_f(c8676cb0,f5e19c10,c76bfe80,0,c90cd000,...) at devfs_read_f+0x87/frame 0xf5e19b90 dofileread(c8676cb0,f5e19c10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf5e19bc4 kern_readv(c90cd000,0,f5e19c10,0,bfbfddfb,...) at kern_readv+0x66/frame 0xf5e19bf0 sys_read(c90cd000,f5e19cc8,c0edce50,c71fac90,0,...) at sys_read+0x5c/frame 0xf5e19c40 syscall(f5e19d08) at syscall+0x2ed/frame 0xf5e19cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5e19cfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2819d077, esp = 0xbfbfdde0, ebp = 0xbfbfde08 --- Tracing command getty pid 1236 tid 100132 td 0xc7dd1c00 sched_switch(c7dd1c00,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5dbf9bc mi_switch(104,0,c10d22d1,1f5,c10d22d1,...) at mi_switch+0x1c9/frame 0xf5dbf9f0 sleepq_switch(c7dd1c00,0,c10d22d1,1ab,f5dbfa68,...) at sleepq_switch+0x1b9/frame 0xf5dbfa24 sleepq_catch_signals(0,c10d22d1,162,5a0,1,...) at sleepq_catch_signals+0x361/frame 0xf5dbfa6c sleepq_wait_sig(c75a4c70,0,c10d7055,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5dbfa90 _cv_wait_sig(c75a4c70,c1446098,c10d7041,531,f5dbfc10,...) at _cv_wait_sig+0x200/frame 0xf5dbfad4 tty_wait(c75a4c00,c75a4c70,f5dbfc10,1,0,...) at tty_wait+0x4b/frame 0xf5dbfaf8 ttydisc_read(c75a4c00,f5dbfc10,0,a1,f5dbfc10,...) at ttydisc_read+0x377/frame 0xf5dbfb3c ttydev_read(c7909800,f5dbfc10,0,1,0,...) at ttydev_read+0x81/frame 0xf5dbfb5c devfs_read_f(c86767a8,f5dbfc10,c76bfe80,0,c7dd1c00,...) at devfs_read_f+0x87/frame 0xf5dbfb90 dofileread(c86767a8,f5dbfc10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf5dbfbc4 kern_readv(c7dd1c00,0,f5dbfc10,0,bfbfddfb,...) at kern_readv+0x66/frame 0xf5dbfbf0 sys_read(c7dd1c00,f5dbfcc8,14,c10ceb5d,7b3,...) at sys_read+0x5c/frame 0xf5dbfc40 syscall(f5dbfd08) at syscall+0x2ed/frame 0xf5dbfcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5dbfcfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2819d077, esp = 0xbfbfdde0, ebp = 0xbfbfde08 --- Tracing command getty pid 1235 tid 100189 td 0xc90f3c00 sched_switch(c90f3c00,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5e6a9bc mi_switch(104,0,c10d22d1,1f5,c10d22d1,...) at mi_switch+0x1c9/frame 0xf5e6a9f0 sleepq_switch(c90f3c00,0,c10d22d1,1ab,f5e6aa68,...) at sleepq_switch+0x1b9/frame 0xf5e6aa24 sleepq_catch_signals(0,c10d22d1,162,5d0,1,...) at sleepq_catch_signals+0x361/frame 0xf5e6aa6c sleepq_wait_sig(c75a4e70,0,c10d7055,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5e6aa90 _cv_wait_sig(c75a4e70,c1446098,c10d7041,531,f5e6ac10,...) at _cv_wait_sig+0x200/frame 0xf5e6aad4 tty_wait(c75a4e00,c75a4e70,f5e6ac10,1,0,...) at tty_wait+0x4b/frame 0xf5e6aaf8 ttydisc_read(c75a4e00,f5e6ac10,0,a1,f5e6ac10,...) at ttydisc_read+0x377/frame 0xf5e6ab3c ttydev_read(c7909900,f5e6ac10,0,1,0,...) at ttydev_read+0x81/frame 0xf5e6ab5c devfs_read_f(c8a6d230,f5e6ac10,c76bfe80,0,c90f3c00,...) at devfs_read_f+0x87/frame 0xf5e6ab90 dofileread(c8a6d230,f5e6ac10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf5e6abc4 kern_readv(c90f3c00,0,f5e6ac10,0,bfbfddfb,...) at kern_readv+0x66/frame 0xf5e6abf0 sys_read(c90f3c00,f5e6acc8,c11245ef,d6,0,...) at sys_read+0x5c/frame 0xf5e6ac40 syscall(f5e6ad08) at syscall+0x2ed/frame 0xf5e6acfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5e6acfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2819d077, esp = 0xbfbfdde0, ebp = 0xbfbfde08 --- Tracing command getty pid 1234 tid 100154 td 0xc8f5a900 sched_switch(c8f5a900,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5e019bc mi_switch(104,0,c10d22d1,1f5,c10d22d1,...) at mi_switch+0x1c9/frame 0xf5e019f0 sleepq_switch(c8f5a900,0,c10d22d1,1ab,f5e01a68,...) at sleepq_switch+0x1b9/frame 0xf5e01a24 sleepq_catch_signals(0,c10d22d1,162,d80,1,...) at sleepq_catch_signals+0x361/frame 0xf5e01a6c sleepq_wait_sig(c792e070,0,c10d7055,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5e01a90 _cv_wait_sig(c792e070,c1446098,c10d7041,531,f5e01c10,...) at _cv_wait_sig+0x200/frame 0xf5e01ad4 tty_wait(c792e000,c792e070,f5e01c10,1,0,...) at tty_wait+0x4b/frame 0xf5e01af8 ttydisc_read(c792e000,f5e01c10,0,a1,f5e01c10,...) at ttydisc_read+0x377/frame 0xf5e01b3c ttydev_read(c7909a00,f5e01c10,0,1,0,...) at ttydev_read+0x81/frame 0xf5e01b5c devfs_read_f(c7d414d0,f5e01c10,c76bfe80,0,c8f5a900,...) at devfs_read_f+0x87/frame 0xf5e01b90 dofileread(c7d414d0,f5e01c10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf5e01bc4 kern_readv(c8f5a900,0,f5e01c10,0,bfbfddfb,...) at kern_readv+0x66/frame 0xf5e01bf0 sys_read(c8f5a900,f5e01cc8,f5e01c9c,c0edd4f7,0,...) at sys_read+0x5c/frame 0xf5e01c40 syscall(f5e01d08) at syscall+0x2ed/frame 0xf5e01cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5e01cfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2819d077, esp = 0xbfbfdde0, ebp = 0xbfbfde08 --- Tracing command getty pid 1233 tid 100142 td 0xc7dff900 sched_switch(c7dff900,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5ddd9bc mi_switch(104,0,c10d22d1,1f5,c10d22d1,...) at mi_switch+0x1c9/frame 0xf5ddd9f0 sleepq_switch(c7dff900,0,c10d22d1,1ab,f5ddda68,...) at sleepq_switch+0x1b9/frame 0xf5ddda24 sleepq_catch_signals(0,c10d22d1,162,db0,1,...) at sleepq_catch_signals+0x361/frame 0xf5ddda6c sleepq_wait_sig(c792e270,0,c10d7055,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5ddda90 _cv_wait_sig(c792e270,c1446098,c10d7041,531,f5dddc10,...) at _cv_wait_sig+0x200/frame 0xf5dddad4 tty_wait(c792e200,c792e270,f5dddc10,1,0,...) at tty_wait+0x4b/frame 0xf5dddaf8 ttydisc_read(c792e200,f5dddc10,0,a1,f5dddc10,...) at ttydisc_read+0x377/frame 0xf5dddb3c ttydev_read(c7909b00,f5dddc10,0,1,0,...) at ttydev_read+0x81/frame 0xf5dddb5c devfs_read_f(c8676738,f5dddc10,c76bfe80,0,c7dff900,...) at devfs_read_f+0x87/frame 0xf5dddb90 dofileread(c8676738,f5dddc10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf5dddbc4 kern_readv(c7dff900,0,f5dddc10,0,bfbfddfb,...) at kern_readv+0x66/frame 0xf5dddbf0 sys_read(c7dff900,f5dddcc8,c0edce50,c71fdc90,0,...) at sys_read+0x5c/frame 0xf5dddc40 syscall(f5dddd08) at syscall+0x2ed/frame 0xf5dddcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5dddcfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2819d077, esp = 0xbfbfdde0, ebp = 0xbfbfde08 --- Tracing command getty pid 1232 tid 100137 td 0xc846f900 sched_switch(c846f900,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5dce9bc mi_switch(104,0,c10d22d1,1f5,c10d22d1,...) at mi_switch+0x1c9/frame 0xf5dce9f0 sleepq_switch(c846f900,0,c10d22d1,1ab,f5dcea68,...) at sleepq_switch+0x1b9/frame 0xf5dcea24 sleepq_catch_signals(0,c10d22d1,162,de0,1,...) at sleepq_catch_signals+0x361/frame 0xf5dcea6c sleepq_wait_sig(c792e470,0,c10d7055,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5dcea90 _cv_wait_sig(c792e470,c1446098,c10d7041,531,f5dcec10,...) at _cv_wait_sig+0x200/frame 0xf5dcead4 tty_wait(c792e400,c792e470,f5dcec10,1,0,...) at tty_wait+0x4b/frame 0xf5dceaf8 ttydisc_read(c792e400,f5dcec10,0,a1,f5dcec10,...) at ttydisc_read+0x377/frame 0xf5dceb3c ttydev_read(c7909c00,f5dcec10,0,1,0,...) at ttydev_read+0x81/frame 0xf5dceb5c devfs_read_f(c8676508,f5dcec10,c76bfe80,0,c846f900,...) at devfs_read_f+0x87/frame 0xf5dceb90 dofileread(c8676508,f5dcec10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf5dcebc4 kern_readv(c846f900,0,f5dcec10,0,bfbfddfb,...) at kern_readv+0x66/frame 0xf5dcebf0 sys_read(c846f900,f5dcecc8,c0edce50,c1825c90,0,...) at sys_read+0x5c/frame 0xf5dcec40 syscall(f5dced08) at syscall+0x2ed/frame 0xf5dcecfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5dcecfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2819d077, esp = 0xbfbfdde0, ebp = 0xbfbfde08 --- Tracing command getty pid 1231 tid 100075 td 0xc78f2c00 sched_switch(c78f2c00,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5c6a9bc mi_switch(104,0,c10d22d1,1f5,46,...) at mi_switch+0x1c9/frame 0xf5c6a9f0 sleepq_switch(c78f2c00,0,c10d22d1,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf5c6aa24 sleepq_catch_signals(f5c6aa90,c0a9d439,c75a3014,0,0,...) at sleepq_catch_signals+0x361/frame 0xf5c6aa6c sleepq_wait_sig(c75a3070,0,f5c6aac0,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5c6aa90 _cv_wait_sig(c75a3070,c75a3004,c10d7041,531,f5c6ac10,...) at _cv_wait_sig+0x200/frame 0xf5c6aad4 tty_wait(c75a3000,c75a3070,f5c6ac10,1,0,...) at tty_wait+0x4b/frame 0xf5c6aaf8 ttydisc_read(c75a3000,f5c6ac10,0,a1,f5c6ac10,...) at ttydisc_read+0x377/frame 0xf5c6ab3c ttydev_read(c7771d00,f5c6ac10,0,1,0,...) at ttydev_read+0x81/frame 0xf5c6ab5c devfs_read_f(c7da06c8,f5c6ac10,c76bfe80,0,c78f2c00,...) at devfs_read_f+0x87/frame 0xf5c6ab90 dofileread(c7da06c8,f5c6ac10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf5c6abc4 kern_readv(c78f2c00,0,f5c6ac10,0,bfbfddfb,...) at kern_readv+0x66/frame 0xf5c6abf0 sys_read(c78f2c00,f5c6acc8,c11245ef,d6,0,...) at sys_read+0x5c/frame 0xf5c6ac40 syscall(f5c6ad08) at syscall+0x2ed/frame 0xf5c6acfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5c6acfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2819d077, esp = 0xbfbfdde0, ebp = 0xbfbfde08 --- Tracing command inetd pid 1214 tid 100117 td 0xc8471000 sched_switch(c8471000,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5d6b90c mi_switch(104,0,c10d22d1,1f5,46,...) at mi_switch+0x1c9/frame 0xf5d6b940 sleepq_switch(c8471000,0,c10d22d1,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf5d6b974 sleepq_catch_signals(f5d6b9e0,c0a9d439,c7d8e1a0,0,0,...) at sleepq_catch_signals+0x361/frame 0xf5d6b9bc sleepq_wait_sig(c7d8e1a4,0,f5d6ba10,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5d6b9e0 _cv_wait_sig(c7d8e1a4,c7d8e190,c10d5015,68e,c0b0c79d,...) at _cv_wait_sig+0x200/frame 0xf5d6ba24 seltdwait(ffffffff,0,0,c8471000,0,...) at seltdwait+0x90/frame 0xf5d6ba58 kern_select(c8471000,8,bfbfd070,0,0,...) at kern_select+0x88c/frame 0xf5d6bbf8 sys_select(c8471000,f5d6bcc8,c0edce50,c71fdc90,0,...) at sys_select+0x69/frame 0xf5d6bc40 syscall(f5d6bd08) at syscall+0x2ed/frame 0xf5d6bcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5d6bcfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281af037, esp = 0xbfbfd014, ebp = 0xbfbfde34 --- Tracing command watchdogd pid 1190 tid 100140 td 0xc846f000 sched_switch(c846f000,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5dd7a64 mi_switch(104,0,c10d22d1,1f5,c0af4e40,...) at mi_switch+0x1c9/frame 0xf5dd7a98 sleepq_switch(c846f000,0,c10d22d1,1ab,8,...) at sleepq_switch+0x1b9/frame 0xf5dd7acc sleepq_catch_signals(c0af4e40,c846f000,0,201,0,...) at sleepq_catch_signals+0x361/frame 0xf5dd7b14 sleepq_timedwait_sig(c127e458,6c,2e1a,c0000000,3,...) at sleepq_timedwait_sig+0x14/frame 0xf5dd7b3c _sleep(c127e458,0,16c,c10cdc4a,a6d6bdc3,...) at _sleep+0x331/frame 0xf5dd7b98 kern_nanosleep(bfbfdcb0,f5dd7c18,f5dd7c20,c770d6f0,3c,...) at kern_nanosleep+0x158/frame 0xf5dd7c00 sys_nanosleep(c846f000,f5dd7cc8,f5dd7c9c,c0edd4f7,0,...) at sys_nanosleep+0x6b/frame 0xf5dd7c40 syscall(f5dd7d08) at syscall+0x2ed/frame 0xf5dd7cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5dd7cfc --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x28178223, esp = 0xbfbfdca4, ebp = 0xbfbfdce8 --- Tracing command gnu-watch pid 1132 tid 100163 td 0xc90ccc00 sched_switch(c90ccc00,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5e1ca64 mi_switch(104,0,c10d22d1,1f5,c0af4e40,...) at mi_switch+0x1c9/frame 0xf5e1ca98 sleepq_switch(c90ccc00,0,c10d22d1,1ab,8,...) at sleepq_switch+0x1b9/frame 0xf5e1cacc sleepq_catch_signals(c0af4e40,c90ccc00,1,201,1,...) at sleepq_catch_signals+0x361/frame 0xf5e1cb14 sleepq_timedwait_sig(c127e459,6c,2df1,20000000,0,...) at sleepq_timedwait_sig+0x14/frame 0xf5e1cb3c _sleep(c127e459,0,16c,c10cdc4a,dbd46033,...) at _sleep+0x331/frame 0xf5e1cb98 kern_nanosleep(bfbfda50,f5e1cc18,f5e1cc20,2,2,...) at kern_nanosleep+0x158/frame 0xf5e1cc00 sys_nanosleep(c90ccc00,f5e1ccc8,c11245ef,e9,7b3,...) at sys_nanosleep+0x6b/frame 0xf5e1cc40 syscall(f5e1cd08) at syscall+0x2ed/frame 0xf5e1ccfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5e1ccfc --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x281a1223, esp = 0xbfbfda44, ebp = 0xbfbfda68 --- Tracing command bash pid 1108 tid 100131 td 0xc7dd2000 Tracing command sshd pid 1107 tid 100158 td 0xc90bec00 sched_switch(c90bec00,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5e0d90c mi_switch(104,0,c10d22d1,1f5,200046,...) at mi_switch+0x1c9/frame 0xf5e0d940 sleepq_switch(c90bec00,0,c10d22d1,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf5e0d974 sleepq_catch_signals(f5e0d9e0,c0a9d439,c8abfba0,0,0,...) at sleepq_catch_signals+0x361/frame 0xf5e0d9bc sleepq_wait_sig(c8abfba4,0,f5e0da10,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5e0d9e0 _cv_wait_sig(c8abfba4,c8abfb90,c10d5015,68e,1b2,...) at _cv_wait_sig+0x200/frame 0xf5e0da24 seltdwait(ffffffff,0,0,c90bec00,0,...) at seltdwait+0x90/frame 0xf5e0da58 kern_select(c90bec00,c,28c0c1c8,28c0c1d0,0,...) at kern_select+0x88c/frame 0xf5e0dbf8 sys_select(c90bec00,f5e0dcc8,c0edce50,c71fdc90,0,...) at sys_select+0x69/frame 0xf5e0dc40 syscall(f5e0dd08) at syscall+0x2ed/frame 0xf5e0dcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5e0dcfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28531037, esp = 0xbfbfd374, ebp = 0xbfbfd3c0 --- Tracing command sshd pid 1105 tid 100147 td 0xc8f5e000 sched_switch(c8f5e000,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5dec984 mi_switch(104,0,c10d22d1,1f5,200046,...) at mi_switch+0x1c9/frame 0xf5dec9b8 sleepq_switch(c8f5e000,0,c10d22d1,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf5dec9ec sleepq_catch_signals(f5deca58,c0a9d439,c7d9f4e0,0,0,...) at sleepq_catch_signals+0x361/frame 0xf5deca34 sleepq_wait_sig(c7d9f4e4,0,f5deca88,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5deca58 _cv_wait_sig(c7d9f4e4,c7d9f4d0,c10d5015,68e,5c3,...) at _cv_wait_sig+0x200/frame 0xf5deca9c seltdwait(ffffffff,0,0,c8f5e000,c1206540,...) at seltdwait+0x90/frame 0xf5decad0 sys_poll(c8f5e000,f5deccc8,c0edce50,c1825c90,0,...) at sys_poll+0x44c/frame 0xf5decc40 syscall(f5decd08) at syscall+0x2ed/frame 0xf5deccfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5deccfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x284c738f, esp = 0xbfbfd3bc, ebp = 0xbfbfd408 --- Tracing command tail pid 1089 tid 100106 td 0xc7df9c00 sched_switch(c7df9c00,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5cfc5f0 mi_switch(104,0,c10d22d1,1f5,101,...) at mi_switch+0x1c9/frame 0xf5cfc624 sleepq_switch(c7df9c00,0,c10d22d1,28e,c7df9c00,...) at sleepq_switch+0x1b9/frame 0xf5cfc658 sleepq_timedwait(c2501a00,50,13,0,0,...) at sleepq_timedwait+0x48/frame 0xf5cfc67c _sleep(c2501a00,c8f4f250,50,c11072e1,ffffe8e0,...) at _sleep+0x345/frame 0xf5cfc6d8 swap_pager_getpages(c8f4f250,f5cfc7a4,1,0,f5cfc87c,...) at swap_pager_getpages+0x4a3/frame 0xf5cfc738 vm_fault_hold(c1bb5000,ed13c000,2,0,0,...) at vm_fault_hold+0xa2f/frame 0xf5cfc898 vm_fault(c1bb5000,ed13c000,2,0,c83075d4,...) at vm_fault+0x82/frame 0xf5cfc8c0 trap_pfault(ed13c000,60,0,0,c7df9c00,...) at trap_pfault+0x223/frame 0xf5cfc938 trap(f5cfca84) at trap+0x53d/frame 0xf5cfca78 calltrap() at calltrap+0x6/frame 0xf5cfca78 --- trap 0xc, eip = 0xc0ed36ff, esp = 0xf5cfcac4, ebp = 0xf5cfcaf8 --- copyin(f5cfcc10,0) at copyin+0x33/frame 0xf5cfcaf8 uiomove(ed13c000,2da,f5cfcc10,4bd,c7df5664,...) at uiomove+0x1f/frame 0xf5cfcb08 pipe_write(c8a6d150,f5cfcc10,c7ca2080,0,c7df9c00,...) at pipe_write+0x642/frame 0xf5cfcb8c dofilewrite(c8a6d150,f5cfcc10,ffffffff,ffffffff,0,...) at dofilewrite+0x86/frame 0xf5cfcbc4 kern_writev(c7df9c00,1,f5cfcc10,0,2880b000,...) at kern_writev+0x66/frame 0xf5cfcbf0 sys_write(c7df9c00,f5cfccc8,f5cfcc8c,c0a82af0,c128b490,...) at sys_write+0x5c/frame 0xf5cfcc40 syscall(f5cfcd08) at syscall+0x2ed/frame 0xf5cfccfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5cfccfc --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x2818a067, esp = 0xbfbfdb1c, ebp = 0xbfbfdb30 --- Tracing command sshd pid 1088 tid 100101 td 0xc7dfec00 sched_switch(c7dfec00,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5cd790c mi_switch(104,0,c10d22d1,1f5,200046,...) at mi_switch+0x1c9/frame 0xf5cd7940 sleepq_switch(c7dfec00,0,c10d22d1,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf5cd7974 sleepq_catch_signals(f5cd79e0,c0a9d439,c7d618a0,0,0,...) at sleepq_catch_signals+0x361/frame 0xf5cd79bc sleepq_wait_sig(c7d618a4,0,f5cd7a10,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5cd79e0 _cv_wait_sig(c7d618a4,c7d61890,c10d5015,68e,5d0,...) at _cv_wait_sig+0x200/frame 0xf5cd7a24 seltdwait(ffffffff,0,0,c7dfec00,0,...) at seltdwait+0x90/frame 0xf5cd7a58 kern_select(c7dfec00,c,28c0c1c8,28c0c1d0,0,...) at kern_select+0x88c/frame 0xf5cd7bf8 sys_select(c7dfec00,f5cd7cc8,c0edce50,c71fdc90,0,...) at sys_select+0x69/frame 0xf5cd7c40 syscall(f5cd7d08) at syscall+0x2ed/frame 0xf5cd7cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5cd7cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28531037, esp = 0xbfbfd374, ebp = 0xbfbfd3c0 --- Tracing command awk pid 1083 tid 100082 td 0xc7d6f900 sched_switch(c7d6f900,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5c86a1c mi_switch(104,0,c10d22d1,1f5,c10d22d1,...) at mi_switch+0x1c9/frame 0xf5c86a50 sleepq_switch(c7d6f900,0,c10d22d1,1ab,f5c86ac8,...) at sleepq_switch+0x1b9/frame 0xf5c86a84 sleepq_catch_signals(0,c10d22d1,162,690,c7d6f900,...) at sleepq_catch_signals+0x361/frame 0xf5c86acc sleepq_wait_sig(c7d82660,5c,c10d525a,100,0,...) at sleepq_wait_sig+0x14/frame 0xf5c86af0 _sleep(c7d82660,c7d827e0,15c,c10d525a,0,...) at _sleep+0x35b/frame 0xf5c86b4c pipe_read(c7d85b98,f5c86c10,c7cdb900,0,c7d6f900,...) at pipe_read+0x2dc/frame 0xf5c86b90 dofileread(c7d85b98,f5c86c10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf5c86bc4 kern_readv(c7d6f900,0,f5c86c10,f5c86c4c,28855000,...) at kern_readv+0x66/frame 0xf5c86bf0 sys_read(c7d6f900,f5c86cc8,c0edce50,c71fac90,0,...) at sys_read+0x5c/frame 0xf5c86c40 syscall(f5c86d08) at syscall+0x2ed/frame 0xf5c86cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5c86cfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281c2077, esp = 0xbfbfd9e0, ebp = 0xbfbfd9f4 --- Tracing command sh pid 1082 tid 100130 td 0xc7dd2300 sched_switch(c7dd2300,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5db998c mi_switch(104,0,c10d22d1,1f5,c10d22d1,...) at mi_switch+0x1c9/frame 0xf5db99c0 sleepq_switch(c7dd2300,0,c10d22d1,1ab,f5db9a38,...) at sleepq_switch+0x1b9/frame 0xf5db99f4 sleepq_catch_signals(0,c10d22d1,162,588,c7dd2300,...) at sleepq_catch_signals+0x361/frame 0xf5db9a3c sleepq_wait_sig(c8ab9ba0,6c,c10d5749,100,0,...) at sleepq_wait_sig+0x14/frame 0xf5db9a60 _sleep(c8ab9ba0,c8ab9c34,16c,c10d5749,0,...) at _sleep+0x35b/frame 0xf5db9abc kern_wait6(c7dd2300,7,0,0,f5db9b4c,...) at kern_wait6+0x45c/frame 0xf5db9b20 sys_wait4(c7dd2300,f5db9cc8,c11245ef,e9,db,...) at sys_wait4+0x94/frame 0xf5db9c40 syscall(f5db9d08) at syscall+0x2ed/frame 0xf5db9cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5db9cfc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x28152563, esp = 0xbfbfd9b8, ebp = 0xbfbfd9d0 --- Tracing command sh pid 1081 tid 100144 td 0xc8f5e900 Tracing command sshd pid 1080 tid 100120 td 0xc8470600 sched_switch(c8470600,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5d7490c mi_switch(104,0,c10d22d1,1f5,200046,...) at mi_switch+0x1c9/frame 0xf5d74940 sleepq_switch(c8470600,0,c10d22d1,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf5d74974 sleepq_catch_signals(f5d749e0,c0a9d439,c8abfb20,0,0,...) at sleepq_catch_signals+0x361/frame 0xf5d749bc sleepq_wait_sig(c8abfb24,0,f5d74a10,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5d749e0 _cv_wait_sig(c8abfb24,c8abfb10,c10d5015,68e,5d0,...) at _cv_wait_sig+0x200/frame 0xf5d74a24 seltdwait(ffffffff,0,0,c8470600,0,...) at seltdwait+0x90/frame 0xf5d74a58 kern_select(c8470600,c,28c0c1c8,28c0c1d0,0,...) at kern_select+0x88c/frame 0xf5d74bf8 sys_select(c8470600,f5d74cc8,c0edce50,c7200c90,0,...) at sys_select+0x69/frame 0xf5d74c40 syscall(f5d74d08) at syscall+0x2ed/frame 0xf5d74cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5d74cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28531037, esp = 0xbfbfd374, ebp = 0xbfbfd3c0 --- Tracing command top pid 1079 tid 100094 td 0xc7de1300 sched_switch(c7de1300,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5cb9900 mi_switch(104,0,c10d22d1,1f5,400,...) at mi_switch+0x1c9/frame 0xf5cb9934 sleepq_switch(c7de1300,0,c10d22d1,1ab,c757b610,...) at sleepq_switch+0x1b9/frame 0xf5cb9968 sleepq_catch_signals(c12041a8,f5cb99d8,c0a9d439,c757b620,0,...) at sleepq_catch_signals+0x361/frame 0xf5cb99b0 sleepq_timedwait_sig(c757b624,0,f5cb9a10,10000000,0,...) at sleepq_timedwait_sig+0x14/frame 0xf5cb99d8 _cv_timedwait_sig_sbt(c757b624,c757b610,ce01a063,2df0,10000000,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf5cb9a24 seltdwait(2df0,10000000,0,c7de1300,0,...) at seltdwait+0xbc/frame 0xf5cb9a58 kern_select(c7de1300,2,bfbfdbe8,0,0,...) at kern_select+0x88c/frame 0xf5cb9bf8 sys_select(c7de1300,f5cb9cc8,c0edce50,c71fdc90,0,...) at sys_select+0x69/frame 0xf5cb9c40 syscall(f5cb9d08) at syscall+0x2ed/frame 0xf5cb9cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5cb9cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281f4037, esp = 0xbfbfdb8c, ebp = 0xbfbfdcf4 --- Tracing command sshd pid 1078 tid 100124 td 0xc7dd3600 sched_switch(c7dd3600,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5da790c mi_switch(104,0,c10d22d1,1f5,200046,...) at mi_switch+0x1c9/frame 0xf5da7940 sleepq_switch(c7dd3600,0,c10d22d1,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf5da7974 sleepq_catch_signals(f5da79e0,c0a9d439,c757b4a0,0,0,...) at sleepq_catch_signals+0x361/frame 0xf5da79bc sleepq_wait_sig(c757b4a4,0,f5da7a10,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5da79e0 _cv_wait_sig(c757b4a4,c757b490,c10d5015,68e,1b2,...) at _cv_wait_sig+0x200/frame 0xf5da7a24 seltdwait(ffffffff,0,0,c7dd3600,0,...) at seltdwait+0x90/frame 0xf5da7a58 kern_select(c7dd3600,a,28c0c1c8,28c0c1d0,0,...) at kern_select+0x88c/frame 0xf5da7bf8 sys_select(c7dd3600,f5da7cc8,c0edce50,c71fac90,0,...) at sys_select+0x69/frame 0xf5da7c40 syscall(f5da7d08) at syscall+0x2ed/frame 0xf5da7cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5da7cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28531037, esp = 0xbfbfd374, ebp = 0xbfbfd3c0 --- Tracing command sshd pid 1076 tid 100143 td 0xc8f5ec00 sched_switch(c8f5ec00,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5de0984 mi_switch(104,0,c10d22d1,1f5,200046,...) at mi_switch+0x1c9/frame 0xf5de09b8 sleepq_switch(c8f5ec00,0,c10d22d1,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf5de09ec sleepq_catch_signals(f5de0a58,c0a9d439,c7d9f6e0,0,0,...) at sleepq_catch_signals+0x361/frame 0xf5de0a34 sleepq_wait_sig(c7d9f6e4,0,f5de0a88,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5de0a58 _cv_wait_sig(c7d9f6e4,c7d9f6d0,c10d5015,68e,5c3,...) at _cv_wait_sig+0x200/frame 0xf5de0a9c seltdwait(ffffffff,0,0,c8f5ec00,0,...) at seltdwait+0x90/frame 0xf5de0ad0 sys_poll(c8f5ec00,f5de0cc8,c0edce50,c7200c90,0,...) at sys_poll+0x44c/frame 0xf5de0c40 syscall(f5de0d08) at syscall+0x2ed/frame 0xf5de0cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5de0cfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x284c738f, esp = 0xbfbfd3bc, ebp = 0xbfbfd408 --- Tracing command sshd pid 1073 tid 100136 td 0xc7dd1000 sched_switch(c7dd1000,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5dcb984 mi_switch(104,0,c10d22d1,1f5,200046,...) at mi_switch+0x1c9/frame 0xf5dcb9b8 sleepq_switch(c7dd1000,0,c10d22d1,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf5dcb9ec sleepq_catch_signals(f5dcba58,c0a9d439,c8abf5a0,0,0,...) at sleepq_catch_signals+0x361/frame 0xf5dcba34 sleepq_wait_sig(c8abf5a4,0,f5dcba88,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5dcba58 _cv_wait_sig(c8abf5a4,c8abf590,c10d5015,68e,5c3,...) at _cv_wait_sig+0x200/frame 0xf5dcba9c seltdwait(ffffffff,0,0,c7dd1000,0,...) at seltdwait+0x90/frame 0xf5dcbad0 sys_poll(c7dd1000,f5dcbcc8,8,c10ccc3c,db,...) at sys_poll+0x44c/frame 0xf5dcbc40 syscall(f5dcbd08) at syscall+0x2ed/frame 0xf5dcbcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5dcbcfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x284c738f, esp = 0xbfbfd3bc, ebp = 0xbfbfd408 --- Tracing command sshd pid 1072 tid 100127 td 0xc7dd2c00 sched_switch(c7dd2c00,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5db0984 mi_switch(104,0,c10d22d1,1f5,200046,...) at mi_switch+0x1c9/frame 0xf5db09b8 sleepq_switch(c7dd2c00,0,c10d22d1,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf5db09ec sleepq_catch_signals(f5db0a58,c0a9d439,c757bd20,0,0,...) at sleepq_catch_signals+0x361/frame 0xf5db0a34 sleepq_wait_sig(c757bd24,0,f5db0a88,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5db0a58 _cv_wait_sig(c757bd24,c757bd10,c10d5015,68e,5c3,...) at _cv_wait_sig+0x200/frame 0xf5db0a9c seltdwait(ffffffff,0,0,c7dd2c00,0,...) at seltdwait+0x90/frame 0xf5db0ad0 sys_poll(c7dd2c00,f5db0cc8,c0edce50,c1825c90,0,...) at sys_poll+0x44c/frame 0xf5db0c40 syscall(f5db0d08) at syscall+0x2ed/frame 0xf5db0cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5db0cfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x284c738f, esp = 0xbfbfd3bc, ebp = 0xbfbfd408 --- Tracing command sh pid 1061 tid 100133 td 0xc7dd1900 sched_switch(c7dd1900,0,20a,1b5,c10cd987,...) at sched_switch+0x4b9/frame 0xf5dc2c6c mi_switch(20a,0,c10cd987,34e,c8ab9308,...) at mi_switch+0x1c9/frame 0xf5dc2ca0 thread_suspend_check(0,0,c10d2e7f,118,c140f1a8,...) at thread_suspend_check+0x2a5/frame 0xf5dc2ccc ast(f5dc2d08) at ast+0x457/frame 0xf5dc2cfc doreti_ast() at doreti_ast+0x1b/frame 0xbfbfd568 Tracing command bash pid 796 tid 100085 td 0xc7de2000 sched_switch(c7de2000,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5c9298c mi_switch(104,0,c10d22d1,1f5,c10d22d1,...) at mi_switch+0x1c9/frame 0xf5c929c0 sleepq_switch(c7de2000,0,c10d22d1,1ab,f5c92a38,...) at sleepq_switch+0x1b9/frame 0xf5c929f4 sleepq_catch_signals(0,c10d22d1,162,108,c7de2000,...) at sleepq_catch_signals+0x361/frame 0xf5c92a3c sleepq_wait_sig(c7ddaba0,6c,c10d5749,100,0,...) at sleepq_wait_sig+0x14/frame 0xf5c92a60 _sleep(c7ddaba0,c7ddac34,16c,c10d5749,0,...) at _sleep+0x35b/frame 0xf5c92abc kern_wait6(c7de2000,7,0,0,f5c92b4c,...) at kern_wait6+0x45c/frame 0xf5c92b20 sys_wait4(c7de2000,f5c92cc8,f5c92c8c,c0a82af0,c128bb90,...) at sys_wait4+0x94/frame 0xf5c92c40 syscall(f5c92d08) at syscall+0x2ed/frame 0xf5c92cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5c92cfc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x282b0563, esp = 0xbfbfd7a0, ebp = 0xbfbfd7b8 --- Tracing command csh pid 793 tid 100100 td 0xc7dff000 Tracing command su pid 792 tid 100116 td 0xc8471300 Tracing command bash pid 787 tid 100097 td 0xc7de0900 Tracing command sshd pid 786 tid 100118 td 0xc8470c00 sched_switch(c8470c00,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5d6e90c mi_switch(104,0,c10d22d1,1f5,200046,...) at mi_switch+0x1c9/frame 0xf5d6e940 sleepq_switch(c8470c00,0,c10d22d1,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf5d6e974 sleepq_catch_signals(f5d6e9e0,c0a9d439,c7d8ea60,0,0,...) at sleepq_catch_signals+0x361/frame 0xf5d6e9bc sleepq_wait_sig(c7d8ea64,0,f5d6ea10,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5d6e9e0 _cv_wait_sig(c7d8ea64,c7d8ea50,c10d5015,68e,1b2,...) at _cv_wait_sig+0x200/frame 0xf5d6ea24 seltdwait(ffffffff,0,0,c8470c00,0,...) at seltdwait+0x90/frame 0xf5d6ea58 kern_select(c8470c00,c,28c0c1c8,28c0c1d0,0,...) at kern_select+0x88c/frame 0xf5d6ebf8 sys_select(c8470c00,f5d6ecc8,c0edce50,c1825c90,0,...) at sys_select+0x69/frame 0xf5d6ec40 syscall(f5d6ed08) at syscall+0x2ed/frame 0xf5d6ecfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5d6ecfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28531037, esp = 0xbfbfd374, ebp = 0xbfbfd3c0 --- Tracing command sshd pid 770 tid 100119 td 0xc8470900 sched_switch(c8470900,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5d71984 mi_switch(104,0,c10d22d1,1f5,200046,...) at mi_switch+0x1c9/frame 0xf5d719b8 sleepq_switch(c8470900,0,c10d22d1,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf5d719ec sleepq_catch_signals(f5d71a58,c0a9d439,c757be20,0,0,...) at sleepq_catch_signals+0x361/frame 0xf5d71a34 sleepq_wait_sig(c757be24,0,f5d71a88,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5d71a58 _cv_wait_sig(c757be24,c757be10,c10d5015,68e,5c3,...) at _cv_wait_sig+0x200/frame 0xf5d71a9c seltdwait(ffffffff,0,0,c8470900,c1206540,...) at seltdwait+0x90/frame 0xf5d71ad0 sys_poll(c8470900,f5d71cc8,c0edce50,c7200c90,0,...) at sys_poll+0x44c/frame 0xf5d71c40 syscall(f5d71d08) at syscall+0x2ed/frame 0xf5d71cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5d71cfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x284c738f, esp = 0xbfbfd3bc, ebp = 0xbfbfd408 --- Tracing command cron pid 737 tid 100090 td 0xc7dff600 sched_switch(c7dff600,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5ca7a64 mi_switch(104,0,c10d22d1,1f5,c0af4e40,...) at mi_switch+0x1c9/frame 0xf5ca7a98 sleepq_switch(c7dff600,0,c10d22d1,1ab,8,...) at sleepq_switch+0x1b9/frame 0xf5ca7acc sleepq_catch_signals(c0af4e40,c7dff600,1,201,1,...) at sleepq_catch_signals+0x361/frame 0xf5ca7b14 sleepq_timedwait_sig(c127e459,6c,2dfe,1cf2e709,2,...) at sleepq_timedwait_sig+0x14/frame 0xf5ca7b3c _sleep(c127e459,0,16c,c10cdc4a,3688238d,...) at _sleep+0x331/frame 0xf5ca7b98 kern_nanosleep(bfbfde28,f5ca7c18,f5ca7c20,0,21,...) at kern_nanosleep+0x158/frame 0xf5ca7c00 sys_nanosleep(c7dff600,f5ca7cc8,0,0,c7dff600,...) at sys_nanosleep+0x6b/frame 0xf5ca7c40 syscall(f5ca7d08) at syscall+0x2ed/frame 0xf5ca7cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5ca7cfc --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x28188223, esp = 0xbfbfdcb4, ebp = 0xbfbfde44 --- Tracing command sendmail pid 732 tid 100096 td 0xc7de0c00 Tracing command sendmail pid 729 tid 100111 td 0xc794a900 sched_switch(c794a900,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5d59900 mi_switch(104,0,c10d22d1,1f5,400,...) at mi_switch+0x1c9/frame 0xf5d59934 sleepq_switch(c794a900,0,c10d22d1,1ab,c7d61810,...) at sleepq_switch+0x1b9/frame 0xf5d59968 sleepq_catch_signals(c12041a8,f5d599d8,c0a9d439,c7d61820,0,...) at sleepq_catch_signals+0x361/frame 0xf5d599b0 sleepq_timedwait_sig(c7d61824,0,f5d59a10,50000000,0,...) at sleepq_timedwait_sig+0x14/frame 0xf5d599d8 _cv_timedwait_sig_sbt(c7d61824,c7d61810,3d9f3302,2df5,50000000,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf5d59a24 seltdwait(2df5,50000000,0,c794a900,0,...) at seltdwait+0xbc/frame 0xf5d59a58 kern_select(c794a900,5,bfbfbfc8,0,0,...) at kern_select+0x88c/frame 0xf5d59bf8 sys_select(c794a900,f5d59cc8,f5d59c8c,c0a82af0,c128b490,...) at sys_select+0x69/frame 0xf5d59c40 syscall(f5d59d08) at syscall+0x2ed/frame 0xf5d59cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5d59cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28420037, esp = 0xbfbfbf2c, ebp = 0xbfbfc9e8 --- Tracing command sshd pid 726 tid 100110 td 0xc794ac00 sched_switch(c794ac00,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5d5690c mi_switch(104,0,c10d22d1,1f5,200046,...) at mi_switch+0x1c9/frame 0xf5d56940 sleepq_switch(c794ac00,0,c10d22d1,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf5d56974 sleepq_catch_signals(f5d569e0,c0a9d439,c7d619e0,0,0,...) at sleepq_catch_signals+0x361/frame 0xf5d569bc sleepq_wait_sig(c7d619e4,0,f5d56a10,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5d569e0 _cv_wait_sig(c7d619e4,c7d619d0,c10d5015,68e,c0b0c79d,...) at _cv_wait_sig+0x200/frame 0xf5d56a24 seltdwait(ffffffff,0,0,c794ac00,0,...) at seltdwait+0x90/frame 0xf5d56a58 kern_select(c794ac00,9,28c0c140,0,0,...) at kern_select+0x88c/frame 0xf5d56bf8 sys_select(c794ac00,f5d56cc8,c0edce50,c7200c90,0,...) at sys_select+0x69/frame 0xf5d56c40 syscall(f5d56d08) at syscall+0x2ed/frame 0xf5d56cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5d56cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28531037, esp = 0xbfbfd424, ebp = 0xbfbfde44 --- Tracing command moused pid 694 tid 100091 td 0xc7dff300 sched_switch(c7dff300,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5cab90c mi_switch(104,0,c10d22d1,1f5,46,...) at mi_switch+0x1c9/frame 0xf5cab940 sleepq_switch(c7dff300,0,c10d22d1,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf5cab974 sleepq_catch_signals(f5cab9e0,c0a9d439,c7d8eca0,0,0,...) at sleepq_catch_signals+0x361/frame 0xf5cab9bc sleepq_wait_sig(c7d8eca4,0,f5caba10,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5cab9e0 _cv_wait_sig(c7d8eca4,c7d8ec90,c10d5015,68e,c7dff300,...) at _cv_wait_sig+0x200/frame 0xf5caba24 seltdwait(ffffffff,0,0,c7dff300,0,...) at seltdwait+0x90/frame 0xf5caba58 kern_select(c7dff300,400,bfbfdcb0,0,0,...) at kern_select+0x88c/frame 0xf5cabbf8 sys_select(c7dff300,f5cabcc8,c0edce50,c71fdc90,0,...) at sys_select+0x69/frame 0xf5cabc40 syscall(f5cabd08) at syscall+0x2ed/frame 0xf5cabcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5cabcfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281bd037, esp = 0xbfbfd894, ebp = 0xbfbfde24 --- Tracing command smartd pid 686 tid 100115 td 0xc8471600 Tracing command winbindd pid 684 tid 100093 td 0xc7de1600 sched_switch(c7de1600,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5cb4978 mi_switch(104,0,c10d22d1,1f5,400,...) at mi_switch+0x1c9/frame 0xf5cb49ac sleepq_switch(c7de1600,0,c10d22d1,1ab,c757b4d0,...) at sleepq_switch+0x1b9/frame 0xf5cb49e0 sleepq_catch_signals(c12041a8,f5cb4a50,c0a9d439,c757b4e0,0,...) at sleepq_catch_signals+0x361/frame 0xf5cb4a28 sleepq_timedwait_sig(c757b4e4,0,f5cb4a88,b7fbe76c,20c49,...) at sleepq_timedwait_sig+0x14/frame 0xf5cb4a50 _cv_timedwait_sig_sbt(c757b4e4,c757b4d0,4694ec31,20f278,b7fbe76c,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf5cb4a9c seltdwait(20f278,b7fbe76c,20c49,c7de1600,0,...) at seltdwait+0xbc/frame 0xf5cb4ad0 sys_poll(c7de1600,f5cb4cc8,c0edce50,c7200c90,0,...) at sys_poll+0x44c/frame 0xf5cb4c40 syscall(f5cb4d08) at syscall+0x2ed/frame 0xf5cb4cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5cb4cfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x2193b38f, esp = 0xbfbfc47c, ebp = 0xbfbfc4b8 --- Tracing command smbd pid 683 tid 100105 td 0xc7dfe000 sched_switch(c7dfe000,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5cf6978 mi_switch(104,0,c10d22d1,1f5,400,...) at mi_switch+0x1c9/frame 0xf5cf69ac sleepq_switch(c7dfe000,0,c10d22d1,1ab,c757a050,...) at sleepq_switch+0x1b9/frame 0xf5cf69e0 sleepq_catch_signals(c12041a8,f5cf6a50,c0a9d439,c757a060,0,...) at sleepq_catch_signals+0x361/frame 0xf5cf6a28 sleepq_timedwait_sig(c757a064,0,f5cf6a88,b7fbe76c,20c49,...) at sleepq_timedwait_sig+0x14/frame 0xf5cf6a50 _cv_timedwait_sig_sbt(c757a064,c757a050,710781df,20c4af,b7fbe76c,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf5cf6a9c seltdwait(20c4af,b7fbe76c,20c49,c7dfe000,0,...) at seltdwait+0xbc/frame 0xf5cf6ad0 sys_poll(c7dfe000,f5cf6cc8,c0edce50,c7200c90,0,...) at sys_poll+0x44c/frame 0xf5cf6c40 syscall(f5cf6d08) at syscall+0x2ed/frame 0xf5cf6cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5cf6cfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x21b6438f, esp = 0xbfbfd9ec, ebp = 0xbfbfda28 --- Tracing command winbindd pid 672 tid 100121 td 0xc8470300 sched_switch(c8470300,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5d77978 mi_switch(104,0,c10d22d1,1f5,400,...) at mi_switch+0x1c9/frame 0xf5d779ac sleepq_switch(c8470300,0,c10d22d1,1ab,c757a110,...) at sleepq_switch+0x1b9/frame 0xf5d779e0 sleepq_catch_signals(c12041a8,f5d77a50,c0a9d439,c757a120,0,...) at sleepq_catch_signals+0x361/frame 0xf5d77a28 sleepq_timedwait_sig(c757a124,0,f5d77a88,a0c89eaf,12,...) at sleepq_timedwait_sig+0x14/frame 0xf5d77a50 _cv_timedwait_sig_sbt(c757a124,c757a110,eb075c75,2f06,a0c89eaf,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf5d77a9c seltdwait(2f06,a0c89eaf,12,c8470300,0,...) at seltdwait+0xbc/frame 0xf5d77ad0 sys_poll(c8470300,f5d77cc8,14,c10ceb5d,7b3,...) at sys_poll+0x44c/frame 0xf5d77c40 syscall(f5d77d08) at syscall+0x2ed/frame 0xf5d77cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5d77cfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x2193b38f, esp = 0xbfbfdbcc, ebp = 0xbfbfdc08 --- Tracing command smbd pid 670 tid 100122 td 0xc8470000 sched_switch(c8470000,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5d7a978 mi_switch(104,0,c10d22d1,1f5,400,...) at mi_switch+0x1c9/frame 0xf5d7a9ac sleepq_switch(c8470000,0,c10d22d1,1ab,c7d8e710,...) at sleepq_switch+0x1b9/frame 0xf5d7a9e0 sleepq_catch_signals(c12041a8,f5d7aa50,c0a9d439,c7d8e720,0,...) at sleepq_catch_signals+0x361/frame 0xf5d7aa28 sleepq_timedwait_sig(c7d8e724,0,f5d7aa88,498d4c12,3,...) at sleepq_timedwait_sig+0x14/frame 0xf5d7aa50 _cv_timedwait_sig_sbt(c7d8e724,c7d8e710,f8a16a7,2e13,498d4c12,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf5d7aa9c seltdwait(2e13,498d4c12,3,c8470000,c0a7796c,...) at seltdwait+0xbc/frame 0xf5d7aad0 sys_poll(c8470000,f5d7acc8,0,0,c8470000,...) at sys_poll+0x44c/frame 0xf5d7ac40 syscall(f5d7ad08) at syscall+0x2ed/frame 0xf5d7acfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5d7acfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x21b6438f, esp = 0xbfbfda9c, ebp = 0xbfbfdad8 --- Tracing command nmbd pid 668 tid 100123 td 0xc846fc00 sched_switch(c846fc00,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5d7d978 mi_switch(104,0,c10d22d1,1f5,400,...) at mi_switch+0x1c9/frame 0xf5d7d9ac sleepq_switch(c846fc00,0,c10d22d1,1ab,c7d61010,...) at sleepq_switch+0x1b9/frame 0xf5d7d9e0 sleepq_catch_signals(c12041a8,f5d7da50,c0a9d439,c7d61020,0,...) at sleepq_catch_signals+0x361/frame 0xf5d7da28 sleepq_timedwait_sig(c7d61024,0,f5d7da88,9fffff47,0,...) at sleepq_timedwait_sig+0x14/frame 0xf5d7da50 _cv_timedwait_sig_sbt(c7d61024,c7d61010,de2239ab,2df5,9fffff47,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf5d7da9c seltdwait(2df5,9fffff47,0,c846fc00,c13f9750,...) at seltdwait+0xbc/frame 0xf5d7dad0 sys_poll(c846fc00,f5d7dcc8,c0edce50,c1825c90,0,...) at sys_poll+0x44c/frame 0xf5d7dc40 syscall(f5d7dd08) at syscall+0x2ed/frame 0xf5d7dcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5d7dcfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x2156338f, esp = 0xbfbfdb1c, ebp = 0xbfbfdb58 --- Tracing command ntpd pid 644 tid 100079 td 0xc7d75300 sched_switch(c7d75300,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5c7a90c mi_switch(104,0,c10d22d1,1f5,200046,...) at mi_switch+0x1c9/frame 0xf5c7a940 sleepq_switch(c7d75300,0,c10d22d1,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf5c7a974 sleepq_catch_signals(f5c7a9e0,c0a9d439,c7d61920,0,0,...) at sleepq_catch_signals+0x361/frame 0xf5c7a9bc sleepq_wait_sig(c7d61924,0,f5c7aa10,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5c7a9e0 _cv_wait_sig(c7d61924,c7d61910,c10d5015,68e,c0b0c79d,...) at _cv_wait_sig+0x200/frame 0xf5c7aa24 seltdwait(ffffffff,0,0,c7d75300,0,...) at seltdwait+0x90/frame 0xf5c7aa58 kern_select(c7d75300,1c,bfbfdd48,0,0,...) at kern_select+0x88c/frame 0xf5c7abf8 sys_select(c7d75300,f5c7acc8,0,0,c7d75300,...) at sys_select+0x69/frame 0xf5c7ac40 syscall(f5c7ad08) at syscall+0x2ed/frame 0xf5c7acfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5c7acfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28399037, esp = 0xbfbfdd24, ebp = 0xbfbfdde4 --- Tracing command nfsd pid 587 tid 100114 td 0xc8471900 sched_switch(c8471900,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5d62a84 mi_switch(104,0,c10d22d1,1f5,c0af4e40,...) at mi_switch+0x1c9/frame 0xf5d62ab8 sleepq_switch(c8471900,0,c10d22d1,1ab,c7931000,...) at sleepq_switch+0x1b9/frame 0xf5d62aec sleepq_catch_signals(c12041a8,f5d62b5c,c0a9d439,c7931010,0,...) at sleepq_catch_signals+0x361/frame 0xf5d62b34 sleepq_timedwait_sig(c7c02390,0,f5d62b94,0,0,...) at sleepq_timedwait_sig+0x14/frame 0xf5d62b5c _cv_timedwait_sig_sbt(c7c02390,c7931000,fffffa38,4,0,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf5d62ba8 svc_run_internal(f5d62cf4,c0a8063f,c7931000,f5d62d08,c10c60e0,...) at svc_run_internal+0xa5d/frame 0xf5d62cc4 svc_thread_start(c7931000,f5d62d08,c10c60e0,3d7,0,...) at svc_thread_start+0xd/frame 0xf5d62ccc fork_exit(c0cb9650,c7931000,f5d62d08) at fork_exit+0x7f/frame 0xf5d62cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf5d62cf4 --- trap 0x504b942e, eip = 0xc, esp = 0x33, ebp = 0xbfbfd937 --- Tracing command nfsd pid 587 tid 100113 td 0xc794a300 sched_switch(c794a300,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5d5fa84 mi_switch(104,0,c10d22d1,1f5,c0af4e40,...) at mi_switch+0x1c9/frame 0xf5d5fab8 sleepq_switch(c794a300,0,c10d22d1,1ab,c7931000,...) at sleepq_switch+0x1b9/frame 0xf5d5faec sleepq_catch_signals(c12041a8,f5d5fb5c,c0a9d439,c7931010,0,...) at sleepq_catch_signals+0x361/frame 0xf5d5fb34 sleepq_timedwait_sig(c7c02350,0,f5d5fb94,0,0,...) at sleepq_timedwait_sig+0x14/frame 0xf5d5fb5c _cv_timedwait_sig_sbt(c7c02350,c7931000,fffffa38,4,0,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf5d5fba8 svc_run_internal(f5d5fcf4,c0a8063f,c7931000,f5d5fd08,c10c60e0,...) at svc_run_internal+0xa5d/frame 0xf5d5fcc4 svc_thread_start(c7931000,f5d5fd08,c10c60e0,3d7,0,...) at svc_thread_start+0xd/frame 0xf5d5fccc fork_exit(c0cb9650,c7931000,f5d5fd08) at fork_exit+0x7f/frame 0xf5d5fcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf5d5fcf4 --- trap 0x504b942e, eip = 0xc, esp = 0x33, ebp = 0xbfbfd937 --- Tracing command nfsd pid 587 tid 100112 td 0xc794a600 sched_switch(c794a600,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5d5ca84 mi_switch(104,0,c10d22d1,1f5,c0af4e40,...) at mi_switch+0x1c9/frame 0xf5d5cab8 sleepq_switch(c794a600,0,c10d22d1,1ab,c7931000,...) at sleepq_switch+0x1b9/frame 0xf5d5caec sleepq_catch_signals(c12041a8,f5d5cb5c,c0a9d439,c7931010,0,...) at sleepq_catch_signals+0x361/frame 0xf5d5cb34 sleepq_timedwait_sig(c7c02310,0,f5d5cb94,0,0,...) at sleepq_timedwait_sig+0x14/frame 0xf5d5cb5c _cv_timedwait_sig_sbt(c7c02310,c7931000,fffffa38,4,0,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf5d5cba8 svc_run_internal(f5d5ccf4,c0a8063f,c7931000,f5d5cd08,c10c60e0,...) at svc_run_internal+0xa5d/frame 0xf5d5ccc4 svc_thread_start(c7931000,f5d5cd08,c10c60e0,3d7,0,...) at svc_thread_start+0xd/frame 0xf5d5cccc fork_exit(c0cb9650,c7931000,f5d5cd08) at fork_exit+0x7f/frame 0xf5d5ccf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf5d5ccf4 --- trap 0x504b942e, eip = 0xc, esp = 0x33, ebp = 0xbfbfd937 --- Tracing command nfsd pid 587 tid 100087 td 0xc7dd3900 sched_switch(c7dd3900,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5c9a2fc mi_switch(104,0,c10d22d1,1f5,c0af4e40,...) at mi_switch+0x1c9/frame 0xf5c9a330 sleepq_switch(c7dd3900,0,c10d22d1,1ab,c7931000,...) at sleepq_switch+0x1b9/frame 0xf5c9a364 sleepq_catch_signals(c12041a8,f5c9a3d4,c0a9d439,c7931010,0,...) at sleepq_catch_signals+0x361/frame 0xf5c9a3ac sleepq_timedwait_sig(c7d02390,0,f5c9a40c,0,0,...) at sleepq_timedwait_sig+0x14/frame 0xf5c9a3d4 _cv_timedwait_sig_sbt(c7d02390,c7931000,fffffa38,4,0,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf5c9a420 svc_run_internal(c0cb9650,c7931000,c78d68b8,f5c9a568,0,...) at svc_run_internal+0xa5d/frame 0xf5c9a53c svc_run(c7931000,0,c10b79ed,1b5,0,...) at svc_run+0xc3/frame 0xf5c9a578 nfsrvd_nfsd(c7dd3900,f5c9a6d0,c,c794e380,c7dd3900,...) at nfsrvd_nfsd+0x1a2/frame 0xf5c9a6b0 nfssvc_nfsd(c7dd3900,f5c9acc8,0,c7dd3900,f5c9acfc,...) at nfssvc_nfsd+0xf4/frame 0xf5c9ac28 sys_nfssvc(c7dd3900,f5c9acc8,c0edce50,c71fac90,0,...) at sys_nfssvc+0xbf/frame 0xf5c9ac40 syscall(f5c9ad08) at syscall+0x2ed/frame 0xf5c9acfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5c9acfc --- syscall (155, FreeBSD ELF32, sys_nfssvc), eip = 0x280c278b, esp = 0xbfbfd7e4, ebp = 0xbfbfda58 --- Tracing command nfsd pid 583 tid 100099 td 0xc7de0300 sched_switch(c7de0300,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5ccd90c mi_switch(104,0,c10d22d1,1f5,46,...) at mi_switch+0x1c9/frame 0xf5ccd940 sleepq_switch(c7de0300,0,c10d22d1,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf5ccd974 sleepq_catch_signals(f5ccd9e0,c0a9d439,c7d8e020,0,0,...) at sleepq_catch_signals+0x361/frame 0xf5ccd9bc sleepq_wait_sig(c7d8e024,0,f5ccda10,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5ccd9e0 _cv_wait_sig(c7d8e024,c7d8e010,c10d5015,68e,c0b0c79d,...) at _cv_wait_sig+0x200/frame 0xf5ccda24 seltdwait(ffffffff,0,0,c7de0300,0,...) at seltdwait+0x90/frame 0xf5ccda58 kern_select(c7de0300,7,bfbfdc30,0,0,...) at kern_select+0x88c/frame 0xf5ccdbf8 sys_select(c7de0300,f5ccdcc8,c0edce50,c71fac90,0,...) at sys_select+0x69/frame 0xf5ccdc40 syscall(f5ccdd08) at syscall+0x2ed/frame 0xf5ccdcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5ccdcfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x2818a037, esp = 0xbfbfda5c, ebp = 0xbfbfde34 --- Tracing command mountd pid 577 tid 100080 td 0xc7d75000 sched_switch(c7d75000,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5c7e90c mi_switch(104,0,c10d22d1,1f5,46,...) at mi_switch+0x1c9/frame 0xf5c7e940 sleepq_switch(c7d75000,0,c10d22d1,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf5c7e974 sleepq_catch_signals(f5c7e9e0,c0a9d439,c757bbe0,0,0,...) at sleepq_catch_signals+0x361/frame 0xf5c7e9bc sleepq_wait_sig(c757bbe4,0,f5c7ea10,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5c7e9e0 _cv_wait_sig(c757bbe4,c757bbd0,c10d5015,68e,c0b0c79d,...) at _cv_wait_sig+0x200/frame 0xf5c7ea24 seltdwait(ffffffff,0,0,c7d75000,0,...) at seltdwait+0x90/frame 0xf5c7ea58 kern_select(c7d75000,9,bfbfdd50,0,0,...) at kern_select+0x88c/frame 0xf5c7ebf8 sys_select(c7d75000,f5c7ecc8,0,0,c7d75000,...) at sys_select+0x69/frame 0xf5c7ec40 syscall(f5c7ed08) at syscall+0x2ed/frame 0xf5c7ecfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5c7ecfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281a1037, esp = 0xbfbfdd0c, ebp = 0xbfbfde34 --- Tracing command nfsuserd pid 570 tid 100095 td 0xc7de1000 sched_switch(c7de1000,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5cbd900 mi_switch(104,0,c10d22d1,1f5,400,...) at mi_switch+0x1c9/frame 0xf5cbd934 sleepq_switch(c7de1000,0,c10d22d1,1ab,c757bb50,...) at sleepq_switch+0x1b9/frame 0xf5cbd968 sleepq_catch_signals(c12041a8,f5cbd9d8,c0a9d439,c757bb60,0,...) at sleepq_catch_signals+0x361/frame 0xf5cbd9b0 sleepq_timedwait_sig(c757bb64,0,f5cbda10,e0000000,1,...) at sleepq_timedwait_sig+0x14/frame 0xf5cbd9d8 _cv_timedwait_sig_sbt(c757bb64,c757bb50,51d8b38e,2dfa,e0000000,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf5cbda24 seltdwait(2dfa,e0000000,1,c7de1000,0,...) at seltdwait+0xbc/frame 0xf5cbda58 kern_select(c7de1000,4,bfbfdc10,0,0,...) at kern_select+0x88c/frame 0xf5cbdbf8 sys_select(c7de1000,f5cbdcc8,8,c10ccc3c,db,...) at sys_select+0x69/frame 0xf5cbdc40 syscall(f5cbdd08) at syscall+0x2ed/frame 0xf5cbdcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5cbdcfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28189037, esp = 0xbfbfdb6c, ebp = 0xbfbfdca8 --- Tracing command nfsuserd pid 569 tid 100081 td 0xc7d6fc00 sched_switch(c7d6fc00,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5c82900 mi_switch(104,0,c10d22d1,1f5,400,...) at mi_switch+0x1c9/frame 0xf5c82934 sleepq_switch(c7d6fc00,0,c10d22d1,1ab,c7d61290,...) at sleepq_switch+0x1b9/frame 0xf5c82968 sleepq_catch_signals(c12041a8,f5c829d8,c0a9d439,c7d612a0,0,...) at sleepq_catch_signals+0x361/frame 0xf5c829b0 sleepq_timedwait_sig(c7d612a4,0,f5c82a10,e0000000,1,...) at sleepq_timedwait_sig+0x14/frame 0xf5c829d8 _cv_timedwait_sig_sbt(c7d612a4,c7d61290,3361e395,2e07,e0000000,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf5c82a24 seltdwait(2e07,e0000000,1,c7d6fc00,0,...) at seltdwait+0xbc/frame 0xf5c82a58 kern_select(c7d6fc00,4,bfbfdc10,0,0,...) at kern_select+0x88c/frame 0xf5c82bf8 sys_select(c7d6fc00,f5c82cc8,c11245ef,d6,0,...) at sys_select+0x69/frame 0xf5c82c40 syscall(f5c82d08) at syscall+0x2ed/frame 0xf5c82cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5c82cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28189037, esp = 0xbfbfdb6c, ebp = 0xbfbfdca8 --- Tracing command nfsuserd pid 568 tid 100076 td 0xc7d75c00 sched_switch(c7d75c00,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5c6e900 mi_switch(104,0,c10d22d1,1f5,400,...) at mi_switch+0x1c9/frame 0xf5c6e934 sleepq_switch(c7d75c00,0,c10d22d1,1ab,c7d9f050,...) at sleepq_switch+0x1b9/frame 0xf5c6e968 sleepq_catch_signals(c12041a8,f5c6e9d8,c0a9d439,c7d9f060,0,...) at sleepq_catch_signals+0x361/frame 0xf5c6e9b0 sleepq_timedwait_sig(c7d9f064,0,f5c6ea10,e0000000,1,...) at sleepq_timedwait_sig+0x14/frame 0xf5c6e9d8 _cv_timedwait_sig_sbt(c7d9f064,c7d9f050,304f749d,2e07,e0000000,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf5c6ea24 seltdwait(2e07,e0000000,1,c7d75c00,0,...) at seltdwait+0xbc/frame 0xf5c6ea58 kern_select(c7d75c00,4,bfbfdc10,0,0,...) at kern_select+0x88c/frame 0xf5c6ebf8 sys_select(c7d75c00,f5c6ecc8,c0edce50,c71fdc90,0,...) at sys_select+0x69/frame 0xf5c6ec40 syscall(f5c6ed08) at syscall+0x2ed/frame 0xf5c6ecfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5c6ecfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28189037, esp = 0xbfbfdb6c, ebp = 0xbfbfdca8 --- Tracing command nfsuserd pid 567 tid 100089 td 0xc7d6f300 sched_switch(c7d6f300,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5ca3900 mi_switch(104,0,c10d22d1,1f5,400,...) at mi_switch+0x1c9/frame 0xf5ca3934 sleepq_switch(c7d6f300,0,c10d22d1,1ab,c7d612d0,...) at sleepq_switch+0x1b9/frame 0xf5ca3968 sleepq_catch_signals(c12041a8,f5ca39d8,c0a9d439,c7d612e0,0,...) at sleepq_catch_signals+0x361/frame 0xf5ca39b0 sleepq_timedwait_sig(c7d612e4,0,f5ca3a10,e0000000,1,...) at sleepq_timedwait_sig+0x14/frame 0xf5ca39d8 _cv_timedwait_sig_sbt(c7d612e4,c7d612d0,a4473a3e,2e07,e0000000,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf5ca3a24 seltdwait(2e07,e0000000,1,c7d6f300,0,...) at seltdwait+0xbc/frame 0xf5ca3a58 kern_select(c7d6f300,4,bfbfdc10,0,0,...) at kern_select+0x88c/frame 0xf5ca3bf8 sys_select(c7d6f300,f5ca3cc8,c0edce50,c71fac90,0,...) at sys_select+0x69/frame 0xf5ca3c40 syscall(f5ca3d08) at syscall+0x2ed/frame 0xf5ca3cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5ca3cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28189037, esp = 0xbfbfdb6c, ebp = 0xbfbfdca8 --- Tracing command nfsuserd pid 566 tid 100077 td 0xc7d75900 Tracing command rpcbind pid 538 tid 100104 td 0xc7dfe300 sched_switch(c7dfe300,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5cef978 mi_switch(104,0,c10d22d1,1f5,400,...) at mi_switch+0x1c9/frame 0xf5cef9ac sleepq_switch(c7dfe300,0,c10d22d1,1ab,c7d61390,...) at sleepq_switch+0x1b9/frame 0xf5cef9e0 sleepq_catch_signals(c12041a8,f5cefa50,c0a9d439,c7d613a0,0,...) at sleepq_catch_signals+0x361/frame 0xf5cefa28 sleepq_timedwait_sig(c7d613a4,0,f5cefa88,dffffdd5,1,...) at sleepq_timedwait_sig+0x14/frame 0xf5cefa50 _cv_timedwait_sig_sbt(c7d613a4,c7d61390,777a3125,2dfb,dffffdd5,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf5cefa9c seltdwait(2dfb,dffffdd5,1,c7dfe300,0,...) at seltdwait+0xbc/frame 0xf5cefad0 sys_poll(c7dfe300,f5cefcc8,c0edce50,c1825c90,0,...) at sys_poll+0x44c/frame 0xf5cefc40 syscall(f5cefd08) at syscall+0x2ed/frame 0xf5cefcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5cefcfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x2813e38f, esp = 0xbfbfbbfc, ebp = 0xbfbfddd8 --- Tracing command syslogd pid 523 tid 100103 td 0xc7dfe600 sched_switch(c7dfe600,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5cea90c mi_switch(104,0,c10d22d1,1f5,46,...) at mi_switch+0x1c9/frame 0xf5cea940 sleepq_switch(c7dfe600,0,c10d22d1,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf5cea974 sleepq_catch_signals(f5cea9e0,c0a9d439,c7d8e5a0,0,0,...) at sleepq_catch_signals+0x361/frame 0xf5cea9bc sleepq_wait_sig(c7d8e5a4,0,f5ceaa10,101,0,...) at sleepq_wait_sig+0x14/frame 0xf5cea9e0 _cv_wait_sig(c7d8e5a4,c7d8e590,c10d5015,68e,c7dfe600,...) at _cv_wait_sig+0x200/frame 0xf5ceaa24 seltdwait(ffffffff,0,0,c7dfe600,0,...) at seltdwait+0x90/frame 0xf5ceaa58 kern_select(c7dfe600,9,28849100,0,0,...) at kern_select+0x88c/frame 0xf5ceabf8 sys_select(c7dfe600,f5ceacc8,0,0,c7dfe600,...) at sys_select+0x69/frame 0xf5ceac40 syscall(f5cead08) at syscall+0x2ed/frame 0xf5ceacfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5ceacfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281a0037, esp = 0xbfbfceac, ebp = 0xbfbfde34 --- Tracing command devd pid 420 tid 100084 td 0xc7dd3c00 sched_switch(c7dd3c00,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf5c8e900 mi_switch(104,0,c10d22d1,1f5,400,...) at mi_switch+0x1c9/frame 0xf5c8e934 sleepq_switch(c7dd3c00,0,c10d22d1,1ab,c757b190,...) at sleepq_switch+0x1b9/frame 0xf5c8e968 sleepq_catch_signals(c12041a8,f5c8e9d8,c0a9d439,c757b1a0,0,...) at sleepq_catch_signals+0x361/frame 0xf5c8e9b0 sleepq_timedwait_sig(c757b1a4,0,f5c8ea10,c0000000,3,...) at sleepq_timedwait_sig+0x14/frame 0xf5c8e9d8 _cv_timedwait_sig_sbt(c757b1a4,c757b190,60f3a89f,2e13,c0000000,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf5c8ea24 seltdwait(2e13,c0000000,3,c7dd3c00,0,...) at seltdwait+0xbc/frame 0xf5c8ea58 kern_select(c7dd3c00,5,bfbfbda0,0,0,...) at kern_select+0x88c/frame 0xf5c8ebf8 sys_select(c7dd3c00,f5c8ecc8,14,c10ceb5d,7b3,...) at sys_select+0x69/frame 0xf5c8ec40 syscall(f5c8ed08) at syscall+0x2ed/frame 0xf5c8ecfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5c8ecfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x8095dc3, esp = 0xbfbfbd44, ebp = 0xbfbfde54 --- Tracing command softdepflush pid 17 tid 100074 td 0xc7948000 sched_switch(c7948000,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf0799ba4 mi_switch(104,0,c10d22d1,1f5,101,...) at mi_switch+0x1c9/frame 0xf0799bd8 sleepq_switch(c7948000,0,c10d22d1,28e,c7948000,...) at sleepq_switch+0x1b9/frame 0xf0799c0c sleepq_timedwait(c13f95e4,54,0,0,0,...) at sleepq_timedwait+0x48/frame 0xf0799c30 _sleep(c13f95e4,c13f9564,54,c11048e0,fffffed8,...) at _sleep+0x345/frame 0xf0799c8c softdep_flush(0,f0799d08,c10c60e0,3d7,0,...) at softdep_flush+0x27c/frame 0xf0799ccc fork_exit(c0d124e0,0,f0799d08) at fork_exit+0x7f/frame 0xf0799cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf0799cf4 --- trap 0, eip = 0, esp = 0xf0799d40, ebp = 0 --- Tracing command syncer pid 16 tid 100073 td 0xc7948300 sched_switch(c7948300,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf0796b78 mi_switch(104,0,c10d22d1,1f5,8,...) at mi_switch+0x1c9/frame 0xf0796bac sleepq_switch(c7948300,0,c10d22d1,28e,0,...) at sleepq_switch+0x1b9/frame 0xf0796be0 sleepq_timedwait(c13f53a4,0,f0796c3c,0,0,...) at sleepq_timedwait+0x48/frame 0xf0796c04 _cv_timedwait_sbt(c13f53a4,c13f5384,fffffed8,0,0,...) at _cv_timedwait_sbt+0x228/frame 0xf0796c50 sched_sync(0,f0796d08,c10c60e0,3d7,0,...) at sched_sync+0x682/frame 0xf0796ccc fork_exit(c0b5c850,0,f0796d08) at fork_exit+0x7f/frame 0xf0796cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf0796cf4 --- trap 0, eip = 0, esp = 0xf0796d40, ebp = 0 --- Tracing command vnlru pid 9 tid 100072 td 0xc7948600 sched_switch(c7948600,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf0793b78 mi_switch(104,0,c10d22d1,1f5,101,...) at mi_switch+0x1c9/frame 0xf0793bac sleepq_switch(c7948600,0,c10d22d1,28e,c7948600,...) at sleepq_switch+0x1b9/frame 0xf0793be0 sleepq_timedwait(c7c2aba0,60,0,0,0,...) at sleepq_timedwait+0x48/frame 0xf0793c04 _sleep(c7c2aba0,c13f535c,260,c10de36f,fffffed8,...) at _sleep+0x345/frame 0xf0793c60 vnlru_proc(0,f0793d08,c10c60e0,3d7,0,...) at vnlru_proc+0xcf/frame 0xf0793ccc fork_exit(c0b5d090,0,f0793d08) at fork_exit+0x7f/frame 0xf0793cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf0793cf4 --- trap 0, eip = 0, esp = 0xf0793d40, ebp = 0 --- Tracing command bufdaemon pid 8 tid 100071 td 0xc7948900 sched_switch(c7948900,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xf0790bb0 mi_switch(104,0,c10d22d1,1f5,101,...) at mi_switch+0x1c9/frame 0xf0790be4 sleepq_switch(c7948900,0,c10d22d1,28e,c7948900,...) at sleepq_switch+0x1b9/frame 0xf0790c18 sleepq_timedwait(c13f511c,54,0,0,0,...) at sleepq_timedwait+0x48/frame 0xf0790c3c _sleep(c13f511c,c13f5104,54,c10dadfc,fffffed8,...) at _sleep+0x345/frame 0xf0790c98 buf_daemon(0,f0790d08,c10c60e0,3d7,0,...) at buf_daemon+0x8c/frame 0xf0790ccc fork_exit(c0b40210,0,f0790d08) at fork_exit+0x7f/frame 0xf0790cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf0790cf4 --- trap 0, eip = 0, esp = 0xf0790d40, ebp = 0 --- Tracing command pagezero pid 7 tid 100070 td 0xc7948c00 sched_switch(c7948c00,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3bfebac mi_switch(104,0,c10d22d1,1f5,101,...) at mi_switch+0x1c9/frame 0xe3bfebe0 sleepq_switch(c7948c00,0,c10d22d1,28e,c7948c00,...) at sleepq_switch+0x1b9/frame 0xe3bfec14 sleepq_timedwait(c13f9ee4,0,12b,0,0,...) at sleepq_timedwait+0x48/frame 0xe3bfec38 _sleep(c13f9ee4,c144d900,0,c110bcb3,fffea520,...) at _sleep+0x345/frame 0xe3bfec94 vm_pagezero(0,e3bfed08,c10c60e0,3d7,0,...) at vm_pagezero+0xc2/frame 0xe3bfeccc fork_exit(c0d59610,0,e3bfed08) at fork_exit+0x7f/frame 0xe3bfecf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3bfecf4 --- trap 0, eip = 0, esp = 0xe3bfed40, ebp = 0 --- Tracing command vmdaemon pid 6 tid 100069 td 0xc7949000 sched_switch(c7949000,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3bfbb78 mi_switch(104,0,c10d22d1,1f5,162,...) at mi_switch+0x1c9/frame 0xe3bfbbac sleepq_switch(c7949000,0,c10d22d1,26b,c7949000,...) at sleepq_switch+0x1b9/frame 0xe3bfbbe0 sleepq_wait(c13f9aac,74,c10dadfc,0,0,...) at sleepq_wait+0x48/frame 0xe3bfbc04 _sleep(c13f9aac,c13f9a94,74,c10dadfc,0,...) at _sleep+0x37c/frame 0xe3bfbc60 vm_daemon(0,e3bfbd08,c10c60e0,3d7,0,...) at vm_daemon+0xac/frame 0xe3bfbccc fork_exit(c0d51660,0,e3bfbd08) at fork_exit+0x7f/frame 0xe3bfbcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3bfbcf4 --- trap 0, eip = 0, esp = 0xe3bfbd40, ebp = 0 --- Tracing command pagedaemon pid 5 tid 100068 td 0xc7949300 sched_switch(c7949300,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3bf8b30 mi_switch(104,0,c10d22d1,1f5,101,...) at mi_switch+0x1c9/frame 0xe3bf8b64 sleepq_switch(c7949300,0,c10d22d1,28e,c7949300,...) at sleepq_switch+0x1b9/frame 0xe3bf8b98 sleepq_timedwait(c144d984,54,4,0,0,...) at sleepq_timedwait+0x48/frame 0xe3bf8bbc _sleep(c144d984,c144d900,54,c10dadfc,fffffa38,...) at _sleep+0x345/frame 0xe3bf8c18 vm_pageout(0,e3bf8d08,c10c60e0,3d7,0,...) at vm_pageout+0x290/frame 0xe3bf8ccc fork_exit(c0d51e80,0,e3bf8d08) at fork_exit+0x7f/frame 0xe3bf8cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3bf8cf4 --- trap 0, eip = 0, esp = 0xe3bf8d40, ebp = 0 --- Tracing command xpt_thrd pid 4 tid 100067 td 0xc7949600 sched_switch(c7949600,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3bb5bb0 mi_switch(104,0,c10d22d1,1f5,162,...) at mi_switch+0x1c9/frame 0xe3bb5be4 sleepq_switch(c7949600,0,c10d22d1,26b,c7949600,...) at sleepq_switch+0x1b9/frame 0xe3bb5c18 sleepq_wait(c1249524,5c,c0f04ba0,0,0,...) at sleepq_wait+0x48/frame 0xe3bb5c3c _sleep(c1249524,c1249574,5c,c0f04ba0,0,...) at _sleep+0x37c/frame 0xe3bb5c98 xpt_scanner_thread(0,e3bb5d08,c10c60e0,3d7,0,...) at xpt_scanner_thread+0x11c/frame 0xe3bb5ccc fork_exit(c04b8380,0,e3bb5d08) at fork_exit+0x7f/frame 0xe3bb5cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3bb5cf4 --- trap 0, eip = 0, esp = 0xe3bb5d40, ebp = 0 --- Tracing command sctp_iterator pid 3 tid 100066 td 0xc7949900 sched_switch(c7949900,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3bb2bb8 mi_switch(104,0,c10d22d1,1f5,162,...) at mi_switch+0x1c9/frame 0xe3bb2bec sleepq_switch(c7949900,0,c10d22d1,26b,c7949900,...) at sleepq_switch+0x1b9/frame 0xe3bb2c20 sleepq_wait(c144968c,0,c10f18a5,0,0,...) at sleepq_wait+0x48/frame 0xe3bb2c44 _sleep(c144968c,c1449654,0,c10f18a5,0,...) at _sleep+0x37c/frame 0xe3bb2ca0 sctp_iterator_thread(0,e3bb2d08,c10c60e0,3d7,0,...) at sctp_iterator_thread+0x7c/frame 0xe3bb2ccc fork_exit(c0bfc110,0,e3bb2d08) at fork_exit+0x7f/frame 0xe3bb2cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3bb2cf4 --- trap 0, eip = 0, esp = 0xe3bb2d40, ebp = 0 --- Tracing command fdc0 pid 2 tid 100061 td 0xc78e5900 sched_switch(c78e5900,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3ba2b48 mi_switch(104,0,c10d22d1,1f5,101,...) at mi_switch+0x1c9/frame 0xe3ba2b7c sleepq_switch(c78e5900,0,c10d22d1,28e,c78e5900,...) at sleepq_switch+0x1b9/frame 0xe3ba2bb0 sleepq_timedwait(c789303c,5c,0,0,0,...) at sleepq_timedwait+0x48/frame 0xe3ba2bd4 _sleep(c789303c,c78930f0,5c,c10b83ae,fffffed8,...) at _sleep+0x345/frame 0xe3ba2c30 fdc_thread(c7893000,e3ba2d08,c10c60e0,3d7,0,...) at fdc_thread+0x877/frame 0xe3ba2ccc fork_exit(c0e63e10,c7893000,e3ba2d08) at fork_exit+0x7f/frame 0xe3ba2cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3ba2cf4 --- trap 0, eip = 0, esp = 0xe3ba2d40, ebp = 0 --- Tracing command usb pid 15 tid 100058 td 0xc78ef300 sched_switch(c78ef300,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3b7ebd4 mi_switch(104,0,c10d22d1,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b7ec08 sleepq_switch(c78ef300,0,c10d22d1,26b,c78ef300,...) at sleepq_switch+0x1b9/frame 0xe3b7ec3c sleepq_wait(c78f7d34,0,e3b7ec90,1,0,...) at sleepq_wait+0x48/frame 0xe3b7ec60 _cv_wait(c78f7d34,c78f7e4c,c10a9b43,77,c78f7d3c,...) at _cv_wait+0x1f9/frame 0xe3b7eca4 usb_process(c78f7d2c,e3b7ed08,c10c60e0,3d7,0,...) at usb_process+0xe6/frame 0xe3b7eccc fork_exit(c0961120,c78f7d2c,e3b7ed08) at fork_exit+0x7f/frame 0xe3b7ecf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b7ecf4 --- trap 0, eip = 0, esp = 0xe3b7ed40, ebp = 0 --- Tracing command usb pid 15 tid 100057 td 0xc78ef600 sched_switch(c78ef600,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3b7bbd4 mi_switch(104,0,c10d22d1,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b7bc08 sleepq_switch(c78ef600,0,c10d22d1,26b,c78ef600,...) at sleepq_switch+0x1b9/frame 0xe3b7bc3c sleepq_wait(c78f7d04,0,e3b7bc90,1,0,...) at sleepq_wait+0x48/frame 0xe3b7bc60 _cv_wait(c78f7d04,c78f7e4c,c10a9b43,77,c78f7d0c,...) at _cv_wait+0x1f9/frame 0xe3b7bca4 usb_process(c78f7cfc,e3b7bd08,c10c60e0,3d7,0,...) at usb_process+0xe6/frame 0xe3b7bccc fork_exit(c0961120,c78f7cfc,e3b7bd08) at fork_exit+0x7f/frame 0xe3b7bcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b7bcf4 --- trap 0, eip = 0, esp = 0xe3b7bd40, ebp = 0 --- Tracing command usb pid 15 tid 100056 td 0xc78ef900 sched_switch(c78ef900,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3b78bd4 mi_switch(104,0,c10d22d1,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b78c08 sleepq_switch(c78ef900,0,c10d22d1,26b,c78ef900,...) at sleepq_switch+0x1b9/frame 0xe3b78c3c sleepq_wait(c78f7cd4,0,e3b78c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b78c60 _cv_wait(c78f7cd4,c78f7e4c,c10a9b43,77,c78f7cdc,...) at _cv_wait+0x1f9/frame 0xe3b78ca4 usb_process(c78f7ccc,e3b78d08,c10c60e0,3d7,0,...) at usb_process+0xe6/frame 0xe3b78ccc fork_exit(c0961120,c78f7ccc,e3b78d08) at fork_exit+0x7f/frame 0xe3b78cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b78cf4 --- trap 0, eip = 0, esp = 0xe3b78d40, ebp = 0 --- Tracing command usb pid 15 tid 100055 td 0xc78efc00 sched_switch(c78efc00,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3b75bd4 mi_switch(104,0,c10d22d1,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b75c08 sleepq_switch(c78efc00,0,c10d22d1,26b,c78efc00,...) at sleepq_switch+0x1b9/frame 0xe3b75c3c sleepq_wait(c78f7ca4,0,e3b75c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b75c60 _cv_wait(c78f7ca4,c78f7e4c,c10a9b43,77,c78f7cac,...) at _cv_wait+0x1f9/frame 0xe3b75ca4 usb_process(c78f7c9c,e3b75d08,c10c60e0,3d7,0,...) at usb_process+0xe6/frame 0xe3b75ccc fork_exit(c0961120,c78f7c9c,e3b75d08) at fork_exit+0x7f/frame 0xe3b75cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b75cf4 --- trap 0, eip = 0, esp = 0xe3b75d40, ebp = 0 --- Tracing command usb pid 15 tid 100053 td 0xc78f2300 sched_switch(c78f2300,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3b6cbd4 mi_switch(104,0,c10d22d1,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b6cc08 sleepq_switch(c78f2300,0,c10d22d1,26b,c78f2300,...) at sleepq_switch+0x1b9/frame 0xe3b6cc3c sleepq_wait(c78ebb5c,0,e3b6cc90,1,0,...) at sleepq_wait+0x48/frame 0xe3b6cc60 _cv_wait(c78ebb5c,c78ebc74,c10a9b43,77,c78ebb64,...) at _cv_wait+0x1f9/frame 0xe3b6cca4 usb_process(c78ebb54,e3b6cd08,c10c60e0,3d7,0,...) at usb_process+0xe6/frame 0xe3b6cccc fork_exit(c0961120,c78ebb54,e3b6cd08) at fork_exit+0x7f/frame 0xe3b6ccf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b6ccf4 --- trap 0, eip = 0, esp = 0xe3b6cd40, ebp = 0 --- Tracing command usb pid 15 tid 100052 td 0xc78f2600 sched_switch(c78f2600,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3b69bd4 mi_switch(104,0,c10d22d1,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b69c08 sleepq_switch(c78f2600,0,c10d22d1,26b,c78f2600,...) at sleepq_switch+0x1b9/frame 0xe3b69c3c sleepq_wait(c78ebb2c,0,e3b69c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b69c60 _cv_wait(c78ebb2c,c78ebc74,c10a9b43,77,c78ebb34,...) at _cv_wait+0x1f9/frame 0xe3b69ca4 usb_process(c78ebb24,e3b69d08,c10c60e0,3d7,0,...) at usb_process+0xe6/frame 0xe3b69ccc fork_exit(c0961120,c78ebb24,e3b69d08) at fork_exit+0x7f/frame 0xe3b69cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b69cf4 --- trap 0, eip = 0, esp = 0xe3b69d40, ebp = 0 --- Tracing command usb pid 15 tid 100051 td 0xc78e2000 sched_switch(c78e2000,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3b66bd4 mi_switch(104,0,c10d22d1,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b66c08 sleepq_switch(c78e2000,0,c10d22d1,26b,c78e2000,...) at sleepq_switch+0x1b9/frame 0xe3b66c3c sleepq_wait(c78ebafc,0,e3b66c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b66c60 _cv_wait(c78ebafc,c78ebc74,c10a9b43,77,c78ebb04,...) at _cv_wait+0x1f9/frame 0xe3b66ca4 usb_process(c78ebaf4,e3b66d08,c10c60e0,3d7,0,...) at usb_process+0xe6/frame 0xe3b66ccc fork_exit(c0961120,c78ebaf4,e3b66d08) at fork_exit+0x7f/frame 0xe3b66cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b66cf4 --- trap 0, eip = 0, esp = 0xe3b66d40, ebp = 0 --- Tracing command usb pid 15 tid 100050 td 0xc78e2300 sched_switch(c78e2300,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3b63bd4 mi_switch(104,0,c10d22d1,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b63c08 sleepq_switch(c78e2300,0,c10d22d1,26b,c78e2300,...) at sleepq_switch+0x1b9/frame 0xe3b63c3c sleepq_wait(c78ebacc,0,e3b63c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b63c60 _cv_wait(c78ebacc,c78ebc74,c10a9b43,77,c78ebad4,...) at _cv_wait+0x1f9/frame 0xe3b63ca4 usb_process(c78ebac4,e3b63d08,c10c60e0,3d7,0,...) at usb_process+0xe6/frame 0xe3b63ccc fork_exit(c0961120,c78ebac4,e3b63d08) at fork_exit+0x7f/frame 0xe3b63cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b63cf4 --- trap 0, eip = 0, esp = 0xe3b63d40, ebp = 0 --- Tracing command usb pid 15 tid 100049 td 0xc78e2600 sched_switch(c78e2600,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3b5fbd4 mi_switch(104,0,c10d22d1,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b5fc08 sleepq_switch(c78e2600,0,c10d22d1,26b,c78e2600,...) at sleepq_switch+0x1b9/frame 0xe3b5fc3c sleepq_wait(c78e6b5c,0,e3b5fc90,1,0,...) at sleepq_wait+0x48/frame 0xe3b5fc60 _cv_wait(c78e6b5c,c78e6c74,c10a9b43,77,c78e6b64,...) at _cv_wait+0x1f9/frame 0xe3b5fca4 usb_process(c78e6b54,e3b5fd08,c10c60e0,3d7,0,...) at usb_process+0xe6/frame 0xe3b5fccc fork_exit(c0961120,c78e6b54,e3b5fd08) at fork_exit+0x7f/frame 0xe3b5fcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b5fcf4 --- trap 0, eip = 0, esp = 0xe3b5fd40, ebp = 0 --- Tracing command usb pid 15 tid 100048 td 0xc78e2900 sched_switch(c78e2900,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3b5cbd4 mi_switch(104,0,c10d22d1,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b5cc08 sleepq_switch(c78e2900,0,c10d22d1,26b,c78e2900,...) at sleepq_switch+0x1b9/frame 0xe3b5cc3c sleepq_wait(c78e6b2c,0,e3b5cc90,1,0,...) at sleepq_wait+0x48/frame 0xe3b5cc60 _cv_wait(c78e6b2c,c78e6c74,c10a9b43,77,c78e6b34,...) at _cv_wait+0x1f9/frame 0xe3b5cca4 usb_process(c78e6b24,e3b5cd08,c10c60e0,3d7,0,...) at usb_process+0xe6/frame 0xe3b5cccc fork_exit(c0961120,c78e6b24,e3b5cd08) at fork_exit+0x7f/frame 0xe3b5ccf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b5ccf4 --- trap 0, eip = 0, esp = 0xe3b5cd40, ebp = 0 --- Tracing command usb pid 15 tid 100047 td 0xc78e2c00 sched_switch(c78e2c00,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3b59bd4 mi_switch(104,0,c10d22d1,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b59c08 sleepq_switch(c78e2c00,0,c10d22d1,26b,c78e2c00,...) at sleepq_switch+0x1b9/frame 0xe3b59c3c sleepq_wait(c78e6afc,0,e3b59c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b59c60 _cv_wait(c78e6afc,c78e6c74,c10a9b43,77,c78e6b04,...) at _cv_wait+0x1f9/frame 0xe3b59ca4 usb_process(c78e6af4,e3b59d08,c10c60e0,3d7,0,...) at usb_process+0xe6/frame 0xe3b59ccc fork_exit(c0961120,c78e6af4,e3b59d08) at fork_exit+0x7f/frame 0xe3b59cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b59cf4 --- trap 0, eip = 0, esp = 0xe3b59d40, ebp = 0 --- Tracing command usb pid 15 tid 100046 td 0xc78e3000 sched_switch(c78e3000,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3b56bd4 mi_switch(104,0,c10d22d1,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b56c08 sleepq_switch(c78e3000,0,c10d22d1,26b,c78e3000,...) at sleepq_switch+0x1b9/frame 0xe3b56c3c sleepq_wait(c78e6acc,0,e3b56c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b56c60 _cv_wait(c78e6acc,c78e6c74,c10a9b43,77,c78e6ad4,...) at _cv_wait+0x1f9/frame 0xe3b56ca4 usb_process(c78e6ac4,e3b56d08,c10c60e0,3d7,0,...) at usb_process+0xe6/frame 0xe3b56ccc fork_exit(c0961120,c78e6ac4,e3b56d08) at fork_exit+0x7f/frame 0xe3b56cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b56cf4 --- trap 0, eip = 0, esp = 0xe3b56d40, ebp = 0 --- Tracing command usb pid 15 tid 100045 td 0xc78e3300 sched_switch(c78e3300,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3b52bd4 mi_switch(104,0,c10d22d1,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b52c08 sleepq_switch(c78e3300,0,c10d22d1,26b,c78e3300,...) at sleepq_switch+0x1b9/frame 0xe3b52c3c sleepq_wait(c78deb5c,0,e3b52c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b52c60 _cv_wait(c78deb5c,c78dec74,c10a9b43,77,c78deb64,...) at _cv_wait+0x1f9/frame 0xe3b52ca4 usb_process(c78deb54,e3b52d08,c10c60e0,3d7,0,...) at usb_process+0xe6/frame 0xe3b52ccc fork_exit(c0961120,c78deb54,e3b52d08) at fork_exit+0x7f/frame 0xe3b52cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b52cf4 --- trap 0, eip = 0, esp = 0xe3b52d40, ebp = 0 --- Tracing command usb pid 15 tid 100044 td 0xc78e3600 sched_switch(c78e3600,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3b4fbd4 mi_switch(104,0,c10d22d1,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b4fc08 sleepq_switch(c78e3600,0,c10d22d1,26b,c78e3600,...) at sleepq_switch+0x1b9/frame 0xe3b4fc3c sleepq_wait(c78deb2c,0,e3b4fc90,1,0,...) at sleepq_wait+0x48/frame 0xe3b4fc60 _cv_wait(c78deb2c,c78dec74,c10a9b43,77,c78deb34,...) at _cv_wait+0x1f9/frame 0xe3b4fca4 usb_process(c78deb24,e3b4fd08,c10c60e0,3d7,0,...) at usb_process+0xe6/frame 0xe3b4fccc fork_exit(c0961120,c78deb24,e3b4fd08) at fork_exit+0x7f/frame 0xe3b4fcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b4fcf4 --- trap 0, eip = 0, esp = 0xe3b4fd40, ebp = 0 --- Tracing command usb pid 15 tid 100043 td 0xc78e3900 sched_switch(c78e3900,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3b4cbd4 mi_switch(104,0,c10d22d1,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b4cc08 sleepq_switch(c78e3900,0,c10d22d1,26b,c78e3900,...) at sleepq_switch+0x1b9/frame 0xe3b4cc3c sleepq_wait(c78deafc,0,e3b4cc90,1,0,...) at sleepq_wait+0x48/frame 0xe3b4cc60 _cv_wait(c78deafc,c78dec74,c10a9b43,77,c78deb04,...) at _cv_wait+0x1f9/frame 0xe3b4cca4 usb_process(c78deaf4,e3b4cd08,c10c60e0,3d7,0,...) at usb_process+0xe6/frame 0xe3b4cccc fork_exit(c0961120,c78deaf4,e3b4cd08) at fork_exit+0x7f/frame 0xe3b4ccf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b4ccf4 --- trap 0, eip = 0, esp = 0xe3b4cd40, ebp = 0 --- Tracing command usb pid 15 tid 100042 td 0xc78e3c00 sched_switch(c78e3c00,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3b49bd4 mi_switch(104,0,c10d22d1,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b49c08 sleepq_switch(c78e3c00,0,c10d22d1,26b,c78e3c00,...) at sleepq_switch+0x1b9/frame 0xe3b49c3c sleepq_wait(c78deacc,0,e3b49c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b49c60 _cv_wait(c78deacc,c78dec74,c10a9b43,77,c78dead4,...) at _cv_wait+0x1f9/frame 0xe3b49ca4 usb_process(c78deac4,e3b49d08,c10c60e0,3d7,0,...) at usb_process+0xe6/frame 0xe3b49ccc fork_exit(c0961120,c78deac4,e3b49d08) at fork_exit+0x7f/frame 0xe3b49cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b49cf4 --- trap 0, eip = 0, esp = 0xe3b49d40, ebp = 0 --- Tracing command usb pid 15 tid 100040 td 0xc78d2000 sched_switch(c78d2000,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3b42bd4 mi_switch(104,0,c10d22d1,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b42c08 sleepq_switch(c78d2000,0,c10d22d1,26b,c78d2000,...) at sleepq_switch+0x1b9/frame 0xe3b42c3c sleepq_wait(c78d9b5c,0,e3b42c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b42c60 _cv_wait(c78d9b5c,c78d9c74,c10a9b43,77,c78d9b64,...) at _cv_wait+0x1f9/frame 0xe3b42ca4 usb_process(c78d9b54,e3b42d08,c10c60e0,3d7,0,...) at usb_process+0xe6/frame 0xe3b42ccc fork_exit(c0961120,c78d9b54,e3b42d08) at fork_exit+0x7f/frame 0xe3b42cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b42cf4 --- trap 0, eip = 0, esp = 0xe3b42d40, ebp = 0 --- Tracing command usb pid 15 tid 100039 td 0xc78d2300 sched_switch(c78d2300,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3b3fbd4 mi_switch(104,0,c10d22d1,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b3fc08 sleepq_switch(c78d2300,0,c10d22d1,26b,c78d2300,...) at sleepq_switch+0x1b9/frame 0xe3b3fc3c sleepq_wait(c78d9b2c,0,e3b3fc90,1,0,...) at sleepq_wait+0x48/frame 0xe3b3fc60 _cv_wait(c78d9b2c,c78d9c74,c10a9b43,77,c78d9b34,...) at _cv_wait+0x1f9/frame 0xe3b3fca4 usb_process(c78d9b24,e3b3fd08,c10c60e0,3d7,0,...) at usb_process+0xe6/frame 0xe3b3fccc fork_exit(c0961120,c78d9b24,e3b3fd08) at fork_exit+0x7f/frame 0xe3b3fcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b3fcf4 --- trap 0, eip = 0, esp = 0xe3b3fd40, ebp = 0 --- Tracing command usb pid 15 tid 100038 td 0xc78d2600 sched_switch(c78d2600,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3b3cbd4 mi_switch(104,0,c10d22d1,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b3cc08 sleepq_switch(c78d2600,0,c10d22d1,26b,c78d2600,...) at sleepq_switch+0x1b9/frame 0xe3b3cc3c sleepq_wait(c78d9afc,0,e3b3cc90,1,0,...) at sleepq_wait+0x48/frame 0xe3b3cc60 _cv_wait(c78d9afc,c78d9c74,c10a9b43,77,c78d9b04,...) at _cv_wait+0x1f9/frame 0xe3b3cca4 usb_process(c78d9af4,e3b3cd08,c10c60e0,3d7,0,...) at usb_process+0xe6/frame 0xe3b3cccc fork_exit(c0961120,c78d9af4,e3b3cd08) at fork_exit+0x7f/frame 0xe3b3ccf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b3ccf4 --- trap 0, eip = 0, esp = 0xe3b3cd40, ebp = 0 --- Tracing command usb pid 15 tid 100037 td 0xc78d2900 sched_switch(c78d2900,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3b39bd4 mi_switch(104,0,c10d22d1,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b39c08 sleepq_switch(c78d2900,0,c10d22d1,26b,c78d2900,...) at sleepq_switch+0x1b9/frame 0xe3b39c3c sleepq_wait(c78d9acc,0,e3b39c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b39c60 _cv_wait(c78d9acc,c78d9c74,c10a9b43,77,c78d9ad4,...) at _cv_wait+0x1f9/frame 0xe3b39ca4 usb_process(c78d9ac4,e3b39d08,c10c60e0,3d7,0,...) at usb_process+0xe6/frame 0xe3b39ccc fork_exit(c0961120,c78d9ac4,e3b39d08) at fork_exit+0x7f/frame 0xe3b39cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b39cf4 --- trap 0, eip = 0, esp = 0xe3b39d40, ebp = 0 --- Tracing command usb pid 15 tid 100035 td 0xc78d4000 sched_switch(c78d4000,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3b32bd4 mi_switch(104,0,c10d22d1,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b32c08 sleepq_switch(c78d4000,0,c10d22d1,26b,c78d4000,...) at sleepq_switch+0x1b9/frame 0xe3b32c3c sleepq_wait(c78cdb5c,0,e3b32c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b32c60 _cv_wait(c78cdb5c,c78cdc74,c10a9b43,77,c78cdb64,...) at _cv_wait+0x1f9/frame 0xe3b32ca4 usb_process(c78cdb54,e3b32d08,c10c60e0,3d7,0,...) at usb_process+0xe6/frame 0xe3b32ccc fork_exit(c0961120,c78cdb54,e3b32d08) at fork_exit+0x7f/frame 0xe3b32cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b32cf4 --- trap 0, eip = 0, esp = 0xe3b32d40, ebp = 0 --- Tracing command usb pid 15 tid 100034 td 0xc78d4300 sched_switch(c78d4300,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3b2fbd4 mi_switch(104,0,c10d22d1,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b2fc08 sleepq_switch(c78d4300,0,c10d22d1,26b,c78d4300,...) at sleepq_switch+0x1b9/frame 0xe3b2fc3c sleepq_wait(c78cdb2c,0,e3b2fc90,1,0,...) at sleepq_wait+0x48/frame 0xe3b2fc60 _cv_wait(c78cdb2c,c78cdc74,c10a9b43,77,c78cdb34,...) at _cv_wait+0x1f9/frame 0xe3b2fca4 usb_process(c78cdb24,e3b2fd08,c10c60e0,3d7,0,...) at usb_process+0xe6/frame 0xe3b2fccc fork_exit(c0961120,c78cdb24,e3b2fd08) at fork_exit+0x7f/frame 0xe3b2fcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b2fcf4 --- trap 0, eip = 0, esp = 0xe3b2fd40, ebp = 0 --- Tracing command usb pid 15 tid 100033 td 0xc78d4600 sched_switch(c78d4600,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3b2cbd4 mi_switch(104,0,c10d22d1,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b2cc08 sleepq_switch(c78d4600,0,c10d22d1,26b,c78d4600,...) at sleepq_switch+0x1b9/frame 0xe3b2cc3c sleepq_wait(c78cdafc,0,e3b2cc90,1,0,...) at sleepq_wait+0x48/frame 0xe3b2cc60 _cv_wait(c78cdafc,c78cdc74,c10a9b43,77,c78cdb04,...) at _cv_wait+0x1f9/frame 0xe3b2cca4 usb_process(c78cdaf4,e3b2cd08,c10c60e0,3d7,0,...) at usb_process+0xe6/frame 0xe3b2cccc fork_exit(c0961120,c78cdaf4,e3b2cd08) at fork_exit+0x7f/frame 0xe3b2ccf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b2ccf4 --- trap 0, eip = 0, esp = 0xe3b2cd40, ebp = 0 --- Tracing command usb pid 15 tid 100032 td 0xc78d4900 sched_switch(c78d4900,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3b29bd4 mi_switch(104,0,c10d22d1,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b29c08 sleepq_switch(c78d4900,0,c10d22d1,26b,c78d4900,...) at sleepq_switch+0x1b9/frame 0xe3b29c3c sleepq_wait(c78cdacc,0,e3b29c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b29c60 _cv_wait(c78cdacc,c78cdc74,c10a9b43,77,c78cdad4,...) at _cv_wait+0x1f9/frame 0xe3b29ca4 usb_process(c78cdac4,e3b29d08,c10c60e0,3d7,0,...) at usb_process+0xe6/frame 0xe3b29ccc fork_exit(c0961120,c78cdac4,e3b29d08) at fork_exit+0x7f/frame 0xe3b29cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b29cf4 --- trap 0, eip = 0, esp = 0xe3b29d40, ebp = 0 --- Tracing command yarrow pid 14 tid 100017 td 0xc7773600 sched_switch(c7773600,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xc7236bb4 mi_switch(104,0,c10d22d1,1f5,c129a198,...) at mi_switch+0x1c9/frame 0xc7236be8 sleepq_switch(c7773600,0,c10d22d1,28e,c12640dc,...) at sleepq_switch+0x1b9/frame 0xc7236c1c sleepq_timedwait(c12640dc,0,c10cd1dd,c10b83ae,0,...) at sleepq_timedwait+0x48/frame 0xc7236c40 msleep_spin_sbt(c12640dc,c1444410,c10b83ae,19999999,0,...) at msleep_spin_sbt+0x225/frame 0xc7236c88 random_kthread(0,c7236d08,c10c60e0,3d7,0,...) at random_kthread+0x2ea/frame 0xc7236ccc fork_exit(c084f6f0,0,c7236d08) at fork_exit+0x7f/frame 0xc7236cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7236cf4 --- trap 0, eip = 0, esp = 0xc7236d40, ebp = 0 --- Tracing command geom pid 13 tid 100016 td 0xc7773900 sched_switch(c7773900,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xc7232b70 mi_switch(104,0,c10d22d1,1f5,162,...) at mi_switch+0x1c9/frame 0xc7232ba4 sleepq_switch(c7773900,0,c10d22d1,26b,c7773900,...) at sleepq_switch+0x1b9/frame 0xc7232bd8 sleepq_wait(c14456e8,5c,c10b83ae,0,0,...) at sleepq_wait+0x48/frame 0xc7232bfc _sleep(c14456e8,c127b67c,25c,c10b83ae,0,...) at _sleep+0x37c/frame 0xc7232c58 g_io_schedule_down(c7773900,0,c10ba0d5,6c,c7232cf4,...) at g_io_schedule_down+0x5c/frame 0xc7232cb4 g_down_procbody(0,c7232d08,c10c60e0,3d7,0,...) at g_down_procbody+0x9d/frame 0xc7232ccc fork_exit(c0a10240,0,c7232d08) at fork_exit+0x7f/frame 0xc7232cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7232cf4 --- trap 0, eip = 0, esp = 0xc7232d40, ebp = 0 --- Tracing command geom pid 13 tid 100015 td 0xc7773c00 sched_switch(c7773c00,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xc722fb98 mi_switch(104,0,c10d22d1,1f5,162,...) at mi_switch+0x1c9/frame 0xc722fbcc sleepq_switch(c7773c00,0,c10d22d1,26b,c7773c00,...) at sleepq_switch+0x1b9/frame 0xc722fc00 sleepq_wait(c14456e4,5c,c10b83ae,0,0,...) at sleepq_wait+0x48/frame 0xc722fc24 _sleep(c14456e4,c127b69c,25c,c10b83ae,0,...) at _sleep+0x37c/frame 0xc722fc80 g_io_schedule_up(c7773c00,0,c10ba0d5,5f,c722fcf4,...) at g_io_schedule_up+0xe6/frame 0xc722fcb4 g_up_procbody(0,c722fd08,c10c60e0,3d7,0,...) at g_up_procbody+0x9d/frame 0xc722fccc fork_exit(c0a101a0,0,c722fd08) at fork_exit+0x7f/frame 0xc722fcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc722fcf4 --- trap 0, eip = 0, esp = 0xc722fd40, ebp = 0 --- Tracing command geom pid 13 tid 100014 td 0xc7775000 sched_switch(c7775000,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xc722cba8 mi_switch(104,0,c10d22d1,1f5,162,...) at mi_switch+0x1c9/frame 0xc722cbdc sleepq_switch(c7775000,0,c10d22d1,26b,c7775000,...) at sleepq_switch+0x1b9/frame 0xc722cc10 sleepq_wait(c14456e0,5c,c10b83ae,0,0,...) at sleepq_wait+0x48/frame 0xc722cc34 _sleep(c14456e0,c127b654,25c,c10b83ae,0,...) at _sleep+0x37c/frame 0xc722cc90 g_run_events(0,c722cd08,c10c60e0,3d7,0,...) at g_run_events+0x566/frame 0xc722cccc fork_exit(c0a10100,0,c722cd08) at fork_exit+0x7f/frame 0xc722ccf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc722ccf4 --- trap 0, eip = 0, esp = 0xc722cd40, ebp = 0 --- Tracing command intr pid 12 tid 100063 td 0xc78e5300 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100062 td 0xc78e5600 sched_switch(c78e5600,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xe3ba5c60 mi_switch(109,0,c10c637d,55a,8,...) at mi_switch+0x1c9/frame 0xe3ba5c94 ithread_loop(c770d1f0,e3ba5d08,c10c60e0,3d7,0,...) at ithread_loop+0x100/frame 0xe3ba5ccc fork_exit(c0a833c0,c770d1f0,e3ba5d08) at fork_exit+0x7f/frame 0xe3ba5cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3ba5cf4 --- trap 0, eip = 0, esp = 0xe3ba5d40, ebp = 0 --- Tracing command intr pid 12 tid 100060 td 0xc78e5c00 sched_switch(c78e5c00,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xe3b95c60 mi_switch(109,0,c10c637d,55a,8,...) at mi_switch+0x1c9/frame 0xe3b95c94 ithread_loop(c770d5a0,e3b95d08,c10c60e0,3d7,0,...) at ithread_loop+0x100/frame 0xe3b95ccc fork_exit(c0a833c0,c770d5a0,e3b95d08) at fork_exit+0x7f/frame 0xe3b95cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b95cf4 --- trap 0, eip = 0, esp = 0xe3b95d40, ebp = 0 --- Tracing command intr pid 12 tid 100059 td 0xc78ef000 sched_switch(c78ef000,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xe3b84c60 mi_switch(109,0,c10c637d,55a,8,...) at mi_switch+0x1c9/frame 0xe3b84c94 ithread_loop(c770d0f0,e3b84d08,c10c60e0,3d7,0,...) at ithread_loop+0x100/frame 0xe3b84ccc fork_exit(c0a833c0,c770d0f0,e3b84d08) at fork_exit+0x7f/frame 0xe3b84cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b84cf4 --- trap 0, eip = 0, esp = 0xe3b84d40, ebp = 0 --- Tracing command intr pid 12 tid 100054 td 0xc78f2000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100041 td 0xc78e5000 sched_switch(c78e5000,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xe3b46c60 mi_switch(109,0,c10c637d,55a,8,...) at mi_switch+0x1c9/frame 0xe3b46c94 ithread_loop(c770d1e0,e3b46d08,c10c60e0,3d7,0,...) at ithread_loop+0x100/frame 0xe3b46ccc fork_exit(c0a833c0,c770d1e0,e3b46d08) at fork_exit+0x7f/frame 0xe3b46cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b46cf4 --- trap 0, eip = 0, esp = 0xe3b46d40, ebp = 0 --- Tracing command intr pid 12 tid 100036 td 0xc78d2c00 sched_switch(c78d2c00,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xe3b36c60 mi_switch(109,0,c10c637d,55a,8,...) at mi_switch+0x1c9/frame 0xe3b36c94 ithread_loop(c770d0c0,e3b36d08,c10c60e0,3d7,0,...) at ithread_loop+0x100/frame 0xe3b36ccc fork_exit(c0a833c0,c770d0c0,e3b36d08) at fork_exit+0x7f/frame 0xe3b36cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b36cf4 --- trap 0, eip = 0, esp = 0xe3b36d40, ebp = 0 --- Tracing command intr pid 12 tid 100031 td 0xc78d4c00 sched_switch(c78d4c00,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xe3b26c60 mi_switch(109,0,c10c637d,55a,8,...) at mi_switch+0x1c9/frame 0xe3b26c94 ithread_loop(c770d6a0,e3b26d08,c10c60e0,3d7,0,...) at ithread_loop+0x100/frame 0xe3b26ccc fork_exit(c0a833c0,c770d6a0,e3b26d08) at fork_exit+0x7f/frame 0xe3b26cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b26cf4 --- trap 0, eip = 0, esp = 0xe3b26d40, ebp = 0 --- Tracing command intr pid 12 tid 100030 td 0xc7775300 sched_switch(c7775300,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xc7393c60 mi_switch(109,0,c10c637d,55a,8,...) at mi_switch+0x1c9/frame 0xc7393c94 ithread_loop(c770d280,c7393d08,c10c60e0,3d7,0,...) at ithread_loop+0x100/frame 0xc7393ccc fork_exit(c0a833c0,c770d280,c7393d08) at fork_exit+0x7f/frame 0xc7393cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7393cf4 --- trap 0, eip = 0, esp = 0xc7393d40, ebp = 0 --- Tracing command intr pid 12 tid 100029 td 0xc7775600 sched_switch(c7775600,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xc738fc60 mi_switch(109,0,c10c637d,55a,8,...) at mi_switch+0x1c9/frame 0xc738fc94 ithread_loop(c770d6c0,c738fd08,c10c60e0,3d7,0,...) at ithread_loop+0x100/frame 0xc738fccc fork_exit(c0a833c0,c770d6c0,c738fd08) at fork_exit+0x7f/frame 0xc738fcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc738fcf4 --- trap 0, eip = 0, esp = 0xc738fd40, ebp = 0 --- Tracing command intr pid 12 tid 100028 td 0xc7775900 sched_switch(c7775900,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xc7286c60 mi_switch(109,0,c10c637d,55a,8,...) at mi_switch+0x1c9/frame 0xc7286c94 ithread_loop(c770d3b0,c7286d08,c10c60e0,3d7,0,...) at ithread_loop+0x100/frame 0xc7286ccc fork_exit(c0a833c0,c770d3b0,c7286d08) at fork_exit+0x7f/frame 0xc7286cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7286cf4 --- trap 0, eip = 0, esp = 0xc7286d40, ebp = 0 --- Tracing command intr pid 12 tid 100027 td 0xc7775c00 sched_switch(c7775c00,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xc7254c60 mi_switch(109,0,c10c637d,55a,8,...) at mi_switch+0x1c9/frame 0xc7254c94 ithread_loop(c770d760,c7254d08,c10c60e0,3d7,0,...) at ithread_loop+0x100/frame 0xc7254ccc fork_exit(c0a833c0,c770d760,c7254d08) at fork_exit+0x7f/frame 0xc7254cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7254cf4 --- trap 0, eip = 0, esp = 0xc7254d40, ebp = 0 --- Tracing command intr pid 12 tid 100022 td 0xc7860c00 sched_switch(c7860c00,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xc7245c60 mi_switch(109,0,c10c637d,55a,8,...) at mi_switch+0x1c9/frame 0xc7245c94 ithread_loop(c770d770,c7245d08,c10c60e0,3d7,0,...) at ithread_loop+0x100/frame 0xc7245ccc fork_exit(c0a833c0,c770d770,c7245d08) at fork_exit+0x7f/frame 0xc7245cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7245cf4 --- trap 0, eip = 0, esp = 0xc7245d40, ebp = 0 --- Tracing command intr pid 12 tid 100020 td 0xc770ec00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100018 td 0xc7773300 sched_switch(c7773300,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xc7239c60 mi_switch(109,0,c10c637d,55a,8,...) at mi_switch+0x1c9/frame 0xc7239c94 ithread_loop(c770d790,c7239d08,c10c60e0,3d7,0,...) at ithread_loop+0x100/frame 0xc7239ccc fork_exit(c0a833c0,c770d790,c7239d08) at fork_exit+0x7f/frame 0xc7239cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7239cf4 --- trap 0, eip = 0, esp = 0xc7239d40, ebp = 0 --- Tracing command intr pid 12 tid 100012 td 0xc76c7600 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100011 td 0xc76c7900 sched_switch(c76c7900,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xc7223c60 mi_switch(109,0,c10c637d,55a,8,...) at mi_switch+0x1c9/frame 0xc7223c94 ithread_loop(c770d7b0,c7223d08,c10c60e0,3d7,0,...) at ithread_loop+0x100/frame 0xc7223ccc fork_exit(c0a833c0,c770d7b0,c7223d08) at fork_exit+0x7f/frame 0xc7223cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7223cf4 --- trap 0, eip = 0, esp = 0xc7223d40, ebp = 0 --- Tracing command intr pid 12 tid 100010 td 0xc76c7c00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100009 td 0xc770e000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100008 td 0xc770e300 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100007 td 0xc770e600 sched_switch(c770e600,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xc7217c60 mi_switch(109,0,c10c637d,55a,8,...) at mi_switch+0x1c9/frame 0xc7217c94 ithread_loop(c770d7f0,c7217d08,c10c60e0,3d7,0,...) at ithread_loop+0x100/frame 0xc7217ccc fork_exit(c0a833c0,c770d7f0,c7217d08) at fork_exit+0x7f/frame 0xc7217cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7217cf4 --- trap 0, eip = 0, esp = 0xc7217d40, ebp = 0 --- Tracing command idle pid 11 tid 100006 td 0xc76c5000 sched_switch(c76c5000,0,108,1b0,c7213c50,...) at sched_switch+0x4b9/frame 0xc7213c1c mi_switch(108,0,c10ceb5d,a44,c128bc08,...) at mi_switch+0x1c9/frame 0xc7213c50 sched_idletd(0,c7213d08,c10c60e0,3d7,0,...) at sched_idletd+0xf1/frame 0xc7213ccc fork_exit(c0ad9bf0,0,c7213d08) at fork_exit+0x7f/frame 0xc7213cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7213cf4 --- trap 0, eip = 0, esp = 0xc7213d40, ebp = 0 --- Tracing command idle pid 11 tid 100005 td 0xc76c5300 sched_switch(c76c5300,0,608,1b0,2df0,...) at sched_switch+0x4b9/frame 0xc7210c00 mi_switch(608,0,c10ccc3c,d3,9,...) at mi_switch+0x1c9/frame 0xc7210c34 critical_exit(0,2,c10ceb5d,409,c128b508,...) at critical_exit+0xa3/frame 0xc7210c50 sched_idletd(0,c7210d08,c10c60e0,3d7,0,...) at sched_idletd+0x1d6/frame 0xc7210ccc fork_exit(c0ad9bf0,0,c7210d08) at fork_exit+0x7f/frame 0xc7210cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7210cf4 --- trap 0, eip = 0, esp = 0xc7210d40, ebp = 0 --- Tracing command idle pid 11 tid 100004 td 0xc76c5600 sched_switch(c76c5600,0,108,1b0,c10ceb5d,...) at sched_switch+0x4b9/frame 0xc720dc1c mi_switch(108,0,c10ceb5d,407,c128ae08,...) at mi_switch+0x1c9/frame 0xc720dc50 sched_idletd(0,c720dd08,c10c60e0,3d7,0,...) at sched_idletd+0x43b/frame 0xc720dccc fork_exit(c0ad9bf0,0,c720dd08) at fork_exit+0x7f/frame 0xc720dcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc720dcf4 --- trap 0, eip = 0, esp = 0xc720dd40, ebp = 0 --- Tracing command idle pid 11 tid 100003 td 0xc76c5900 cpustop_handler(0,13,c720ab90,c0ed515b,413bc8,...) at cpustop_handler+0x2f/frame 0xc720aa40 ipi_nmi_handler(413bc8,0,0,c720aa94,c0a7515b,...) at ipi_nmi_handler+0x37/frame 0xc720aa50 trap(c720ab9c) at trap+0x3b/frame 0xc720ab90 calltrap() at calltrap+0x6/frame 0xc720ab90 --- trap 0x13, eip = 0xc0ebab55, esp = 0xc720abdc, ebp = 0xc720abdc --- acpi_cpu_c1(c0eddf0b,c1825c90,0,c1125af7,303,...) at acpi_cpu_c1+0x5/frame 0xc720abdc acpi_cpu_idle(607c2dc,0,607c2dc,c720ac50,c0ec4caa,...) at acpi_cpu_idle+0x146/frame 0xc720ac20 cpu_idle_acpi(607c2dc,0,9,c128a708,c128a716,...) at cpu_idle_acpi+0x3f/frame 0xc720ac34 cpu_idle(0,2,c10ceb5d,409,c128a708,...) at cpu_idle+0x9a/frame 0xc720ac50 sched_idletd(0,c720ad08,c10c60e0,3d7,0,...) at sched_idletd+0x1d6/frame 0xc720accc fork_exit(c0ad9bf0,0,c720ad08) at fork_exit+0x7f/frame 0xc720acf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc720acf4 --- trap 0, eip = 0, esp = 0xc720ad40, ebp = 0 --- Tracing command init pid 1 tid 100002 td 0xc76c5c00 sched_switch(c76c5c00,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xc720698c mi_switch(104,0,c10d22d1,1f5,c10d22d1,...) at mi_switch+0x1c9/frame 0xc72069c0 sleepq_switch(c76c5c00,0,c10d22d1,1ab,c7206a38,...) at sleepq_switch+0x1b9/frame 0xc72069f4 sleepq_catch_signals(0,c10d22d1,162,e88,c76c5c00,...) at sleepq_catch_signals+0x361/frame 0xc7206a3c sleepq_wait_sig(c76c3ba0,6c,c10d5749,100,0,...) at sleepq_wait_sig+0x14/frame 0xc7206a60 _sleep(c76c3ba0,c76c3c34,16c,c10d5749,0,...) at _sleep+0x35b/frame 0xc7206abc kern_wait6(c76c5c00,7,0,0,c7206b4c,...) at kern_wait6+0x45c/frame 0xc7206b20 sys_wait4(c76c5c00,c7206cc8,c7206c8c,c0a82af0,c128a690,...) at sys_wait4+0x94/frame 0xc7206c40 syscall(c7206d08) at syscall+0x2ed/frame 0xc7206cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xc7206cfc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x805e3b7, esp = 0xbfbfd8f4, ebp = 0xbfbfd908 --- Tracing command audit pid 10 tid 100001 td 0xc76c7000 sched_switch(c76c7000,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xc7203ba8 mi_switch(104,0,c10d22d1,1f5,8,...) at mi_switch+0x1c9/frame 0xc7203bdc sleepq_switch(c76c7000,0,c10d22d1,26b,c76c7000,...) at sleepq_switch+0x1b9/frame 0xc7203c10 sleepq_wait(c144c438,0,c7203c64,1,0,...) at sleepq_wait+0x48/frame 0xc7203c34 _cv_wait(c144c438,c144c3c4,c10fddf2,18d,0,...) at _cv_wait+0x1f9/frame 0xc7203c78 audit_worker(0,c7203d08,c10c60e0,3d7,0,...) at audit_worker+0x94/frame 0xc7203ccc fork_exit(c0cc7720,0,c7203d08) at fork_exit+0x7f/frame 0xc7203cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7203cf4 --- trap 0, eip = 0, esp = 0xc7203d40, ebp = 0 --- Tracing command kernel pid 0 tid 100065 td 0xc7949c00 sched_switch(c7949c00,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3baeb68 mi_switch(104,0,c10d22d1,1f5,101,...) at mi_switch+0x1c9/frame 0xe3baeb9c sleepq_switch(c7949c00,0,c10d22d1,28e,c7949c00,...) at sleepq_switch+0x1b9/frame 0xe3baebd0 sleepq_timedwait(c127d350,0,2,0,0,...) at sleepq_timedwait+0x48/frame 0xe3baebf4 _sleep(c127d350,0,0,c10b83ae,fffffc88,...) at _sleep+0x345/frame 0xe3baec50 pause_sbt(c10b83ae,fffffc88,2,0,0,...) at pause_sbt+0x86/frame 0xe3baec8c deadlkres(0,e3baed08,c10c60e0,3d7,0,...) at deadlkres+0x82/frame 0xe3baeccc fork_exit(c0a61df0,0,e3baed08) at fork_exit+0x7f/frame 0xe3baecf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3baecf4 --- trap 0, eip = 0, esp = 0xe3baed40, ebp = 0 --- Tracing command kernel pid 0 tid 100064 td 0xc794a000 sched_switch(c794a000,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xe3babbc0 mi_switch(104,0,c10d22d1,1f5,c129aab0,...) at mi_switch+0x1c9/frame 0xe3babbf4 sleepq_switch(c794a000,0,c10d22d1,26b,c7867d80,...) at sleepq_switch+0x1b9/frame 0xe3babc28 sleepq_wait(c7867d80,0,c10cd1dd,c10b83ae,0,...) at sleepq_wait+0x48/frame 0xe3babc4c msleep_spin_sbt(c7867d80,c7867d98,c10b83ae,0,0,...) at msleep_spin_sbt+0x239/frame 0xe3babc94 taskqueue_thread_loop(c1441f00,e3babd08,c10c60e0,3d7,0,...) at taskqueue_thread_loop+0xcc/frame 0xe3babccc fork_exit(c0af90d0,c1441f00,e3babd08) at fork_exit+0x7f/frame 0xe3babcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3babcf4 --- trap 0, eip = 0, esp = 0xe3babd40, ebp = 0 --- Tracing command kernel pid 0 tid 100026 td 0xc7860000 sched_switch(c7860000,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xc7251bac mi_switch(104,0,c10d22d1,1f5,162,...) at mi_switch+0x1c9/frame 0xc7251be0 sleepq_switch(c7860000,0,c10d22d1,26b,c7860000,...) at sleepq_switch+0x1b9/frame 0xc7251c14 sleepq_wait(c75a8600,0,c10b83ae,0,0,...) at sleepq_wait+0x48/frame 0xc7251c38 _sleep(c75a8600,c75a8618,0,c10b83ae,0,...) at _sleep+0x37c/frame 0xc7251c94 taskqueue_thread_loop(c144c470,c7251d08,c10c60e0,3d7,0,...) at taskqueue_thread_loop+0x129/frame 0xc7251ccc fork_exit(c0af90d0,c144c470,c7251d08) at fork_exit+0x7f/frame 0xc7251cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7251cf4 --- trap 0, eip = 0, esp = 0xc7251d40, ebp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xc7860300 sched_switch(c7860300,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xc724ebc0 mi_switch(104,0,c10d22d1,1f5,c1299388,...) at mi_switch+0x1c9/frame 0xc724ebf4 sleepq_switch(c7860300,0,c10d22d1,26b,c75a8680,...) at sleepq_switch+0x1b9/frame 0xc724ec28 sleepq_wait(c75a8680,0,c10cd1dd,c10b83ae,0,...) at sleepq_wait+0x48/frame 0xc724ec4c msleep_spin_sbt(c75a8680,c75a8698,c10b83ae,0,0,...) at msleep_spin_sbt+0x239/frame 0xc724ec94 taskqueue_thread_loop(c1442a20,c724ed08,c10c60e0,3d7,0,...) at taskqueue_thread_loop+0xcc/frame 0xc724eccc fork_exit(c0af90d0,c1442a20,c724ed08) at fork_exit+0x7f/frame 0xc724ecf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc724ecf4 --- trap 0, eip = 0, esp = 0xc724ed40, ebp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xc7860600 sched_switch(c7860600,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xc724bbc0 mi_switch(104,0,c10d22d1,1f5,c1299388,...) at mi_switch+0x1c9/frame 0xc724bbf4 sleepq_switch(c7860600,0,c10d22d1,26b,c75a8680,...) at sleepq_switch+0x1b9/frame 0xc724bc28 sleepq_wait(c75a8680,0,c10cd1dd,c10b83ae,0,...) at sleepq_wait+0x48/frame 0xc724bc4c msleep_spin_sbt(c75a8680,c75a8698,c10b83ae,0,0,...) at msleep_spin_sbt+0x239/frame 0xc724bc94 taskqueue_thread_loop(c1442a20,c724bd08,c10c60e0,3d7,0,...) at taskqueue_thread_loop+0xcc/frame 0xc724bccc fork_exit(c0af90d0,c1442a20,c724bd08) at fork_exit+0x7f/frame 0xc724bcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc724bcf4 --- trap 0, eip = 0, esp = 0xc724bd40, ebp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xc7860900 sched_switch(c7860900,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xc7248bc0 mi_switch(104,0,c10d22d1,1f5,c1299388,...) at mi_switch+0x1c9/frame 0xc7248bf4 sleepq_switch(c7860900,0,c10d22d1,26b,c75a8680,...) at sleepq_switch+0x1b9/frame 0xc7248c28 sleepq_wait(c75a8680,0,c10cd1dd,c10b83ae,0,...) at sleepq_wait+0x48/frame 0xc7248c4c msleep_spin_sbt(c75a8680,c75a8698,c10b83ae,0,0,...) at msleep_spin_sbt+0x239/frame 0xc7248c94 taskqueue_thread_loop(c1442a20,c7248d08,c10c60e0,3d7,0,...) at taskqueue_thread_loop+0xcc/frame 0xc7248ccc fork_exit(c0af90d0,c1442a20,c7248d08) at fork_exit+0x7f/frame 0xc7248cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7248cf4 --- trap 0, eip = 0, esp = 0xc7248d40, ebp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xc770e900 sched_switch(c770e900,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xc7242bac mi_switch(104,0,c10d22d1,1f5,162,...) at mi_switch+0x1c9/frame 0xc7242be0 sleepq_switch(c770e900,0,c10d22d1,26b,c770e900,...) at sleepq_switch+0x1b9/frame 0xc7242c14 sleepq_wait(c75a8a00,0,c10b83ae,0,0,...) at sleepq_wait+0x48/frame 0xc7242c38 _sleep(c75a8a00,c75a8a18,0,c10b83ae,0,...) at _sleep+0x37c/frame 0xc7242c94 taskqueue_thread_loop(c1445edc,c7242d08,c10c60e0,3d7,0,...) at taskqueue_thread_loop+0x129/frame 0xc7242ccc fork_exit(c0af90d0,c1445edc,c7242d08) at fork_exit+0x7f/frame 0xc7242cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7242cf4 --- trap 0, eip = 0, esp = 0xc7242d40, ebp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xc7773000 sched_switch(c7773000,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xc723cbac mi_switch(104,0,c10d22d1,1f5,162,...) at mi_switch+0x1c9/frame 0xc723cbe0 sleepq_switch(c7773000,0,c10d22d1,26b,c7773000,...) at sleepq_switch+0x1b9/frame 0xc723cc14 sleepq_wait(c75a8b00,0,c10b83ae,0,0,...) at sleepq_wait+0x48/frame 0xc723cc38 _sleep(c75a8b00,c75a8b18,0,c10b83ae,0,...) at _sleep+0x37c/frame 0xc723cc94 taskqueue_thread_loop(c1448e40,c723cd08,c10c60e0,3d7,0,...) at taskqueue_thread_loop+0x129/frame 0xc723cccc fork_exit(c0af90d0,c1448e40,c723cd08) at fork_exit+0x7f/frame 0xc723ccf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc723ccf4 --- trap 0, eip = 0, esp = 0xc723cd40, ebp = 0 --- Tracing command kernel pid 0 tid 100013 td 0xc76c7300 sched_switch(c76c7300,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xc7229bac mi_switch(104,0,c10d22d1,1f5,162,...) at mi_switch+0x1c9/frame 0xc7229be0 sleepq_switch(c76c7300,0,c10d22d1,26b,c76c7300,...) at sleepq_switch+0x1b9/frame 0xc7229c14 sleepq_wait(c75a7100,0,c10b83ae,0,0,...) at sleepq_wait+0x48/frame 0xc7229c38 _sleep(c75a7100,c75a7118,0,c10b83ae,0,...) at _sleep+0x37c/frame 0xc7229c94 taskqueue_thread_loop(c12987c4,c7229d08,c10c60e0,3d7,0,...) at taskqueue_thread_loop+0x129/frame 0xc7229ccc fork_exit(c0af90d0,c12987c4,c7229d08) at fork_exit+0x7f/frame 0xc7229cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7229cf4 --- trap 0, eip = 0, esp = 0xc7229d40, ebp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xc1445a50 sched_switch(c1445a50,0,104,1b5,c0a9ea1e,...) at sched_switch+0x4b9/frame 0xc1820c0c mi_switch(104,0,c10d22d1,1f5,101,...) at mi_switch+0x1c9/frame 0xc1820c40 sleepq_switch(c1445a50,0,c10d22d1,28e,c1445a50,...) at sleepq_switch+0x1b9/frame 0xc1820c74 sleepq_timedwait(c1445764,54,9,0,0,...) at sleepq_timedwait+0x48/frame 0xc1820c98 _sleep(c1445764,0,54,c10fefb5,fffff470,...) at _sleep+0x345/frame 0xc1820cf4 scheduler(0,0,c1131d98,181e000,1825000,...) at scheduler+0x2b3/frame 0xc1820d38 mi_startup() at mi_startup+0xf7/frame 0xc1820d58 begin() at begin+0x2c db:0:allt> call doadump Physical memory: 3435 MB Dumping 303 MB: 288 272 256 240 224 208 192 176 160 144 128 112 96 80 64 48 32 16 Dump complete = 0 db:0:doadump> reset cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 1 (kgdb) bt #0 doadump (textdump=0xa) at pcpu.h:253 #1 0xc051275d in db_fncall (dummy1=0x19, dummy2=0x0, dummy3=0xc124b06c, dummy4=0xf6268470 "") at ../../../ddb/db_command.c:578 #2 0xc0512457 in db_command (cmd_table=) at ../../../ddb/db_command.c:449 #3 0xc0516334 in db_script_exec (scriptname=0xc1442784 "doadump", warnifnotfound=) at ../../../ddb/db_script.c:302 #4 0xc0512457 in db_command (cmd_table=) at ../../../ddb/db_command.c:449 #5 0xc0512170 in db_command_loop () at ../../../ddb/db_command.c:502 #6 0xc05149b0 in db_trap (type=, code=0xf6268664) at ../../../ddb/db_main.c:231 #7 0xc0ae9e48 in kdb_trap (type=, code=, tf=) at ../../../kern/subr_kdb.c:654 #8 0xc0ed5d3a in trap_fatal (frame=0xf62688d4, eva=0x0) at ../../../i386/i386/trap.c:1037 #9 0xc0ed5fff in trap_pfault (frame=0x0, usermode=0x0, eva=0x0) at ../../../i386/i386/trap.c:858 #10 0xc0ed565d in trap (frame=0xf62688d4) at ../../../i386/i386/trap.c:556 #11 0xc0ebea4c in calltrap () at ../../../i386/i386/exception.s:169 #12 0xc099d095 in fifo_close (ap=0xf6268a78) at ../../../fs/fifofs/fifo_vnops.c:266 #13 0xc0ef9a64 in VOP_CLOSE_APV (vop=, a=0xf6268a78) at vnode_if.c:535 #14 0xc0ef993b in VOP_CLOSE_AP (a=0xf6268a78) at vnode_if.c:513 #15 0xd07d1c3d in null_bypass (ap=0xf62689d8) at /usr/src/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:280 #16 0xc0ef9a6c in VOP_CLOSE_APV (vop=, a=) at vnode_if.c:537 #17 0xc0b57827 in vgonel (vp=0xccebe438) at vnode_if.h:225 #18 0xc0b57dd8 in vgone (vp=0xccebe438) at ../../../kern/vfs_subr.c:2698 #19 0xd07d1b27 in nullfs_unlink_lowervp (mp=0xc844c5d8, lowervp=0xccc25ca8) at /usr/src/sys/modules/nullfs/../../fs/nullfs/null_vfsops.c:407 #20 0xc0b58078 in vfs_notify_upper (vp=, event=0xc12437f0) at ../../../kern/vfs_subr.c:2745 #21 0xc0b6150d in kern_unlinkat (td=) at ../../../kern/vfs_syscalls.c:1849 #22 0xc0b60fe2 in sys_unlink (td=0xd1a22300, uap=0xf6268cc8) at ../../../kern/vfs_syscalls.c:1795 #23 0xc0ed656d in syscall (frame=) at subr_syscall.c:134 #24 0xc0ebeae1 in Xint0x80_syscall () at ../../../i386/i386/exception.s:267 #25 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) Current language: auto; currently minimal (kgdb) f 12 #12 0xc099d095 in fifo_close (ap=0xf6268a78) at ../../../fs/fifofs/fifo_vnops.c:266 266 fip = vp->v_fifoinfo; (kgdb) l 261 struct vnode *vp; 262 struct fifoinfo *fip; 263 struct pipe *cpipe; 264 265 vp = ap->a_vp; 266 fip = vp->v_fifoinfo; 267 cpipe = fip->fi_pipe; 268 ASSERT_VOP_ELOCKED(vp, "fifo_close"); 269 if (ap->a_fflag & FREAD) { 270 fip->fi_readers--; (kgdb) p *ap $1 = {a_gen = {a_desc = 0xc1241108}, a_vp = 0xccc25ca8, a_fflag = 0x4, a_cred = 0x0, a_td = 0xd1a22300} (kgdb) info loc vp = (struct vnode *) 0xccc25ca8 fip = (struct fifoinfo *) 0x0 cpipe = (kgdb) p *vp $2 = {v_tag = 0xc10bbc9a "ufs", v_op = 0xc122eb0c, v_data = 0xca7e03a0, v_mount = 0xc8f595d8, v_nmntvnodes = {tqe_next = 0xc91d1000, tqe_prev = 0xce9eecb8}, v_un = {vu_mount = 0x0, vu_socket = 0x0, vu_cdev = 0x0, vu_fifoinfo = 0x0}, v_hashlist = {le_next = 0x0, le_prev = 0xc7974cec}, v_cache_src = {lh_first = 0x0}, v_cache_dst = {tqh_first = 0x0, tqh_last = 0xccc25cd0}, v_cache_dd = 0x0, v_lock = {lock_object = { lo_name = 0xc10bbc9a "ufs", lo_flags = 0x57b0008, lo_data = 0x0, lo_witness = 0xc7586200}, lk_lock = 0xd1a22300, lk_exslpfail = 0x0, lk_timo = 0x33, lk_pri = 0x60, lk_stack = {depth = 0x12, pcs = {0xc0a952bf, 0xc0d1938d, 0xc0efc4e2, 0xc0b67dca, 0xc0b55937, 0xc0b4910f, 0xc0d13bf7, 0xc0d13b9e, 0xc0d22bfe, 0xc0d21d0a, 0xc0ef90f4, 0xc0b41b70, 0xc0ef8f04, 0xc0b4a978, 0xc0b4a06d, 0xc0b6138c, 0xc0b60fe2, 0xc0ed656d}}}, v_interlock = {lock_object = {lo_name = 0xc10d4881 "vnode interlock", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc757f1e0}, mtx_lock = 0x4}, v_vnlock = 0xccc25cdc, v_actfreelist = {tqe_next = 0xce9eeca8, tqe_prev = 0xc91d10b8}, v_bufobj = {bo_mtx = {lock_object = {lo_name = 0xc10dd031 "bufobj interlock", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc7584730}, mtx_lock = 0x4}, bo_ops = 0xc120ca70, bo_object = 0x0, bo_synclist = {le_next = 0x0, le_prev = 0x0}, bo_private = 0xccc25ca8, __bo_vnode = 0xccc25ca8, bo_clean = {bv_hd = {tqh_first = 0x0, tqh_last = 0xccc25d94}, bv_root = 0x0, bv_cnt = 0x0}, bo_dirty = {bv_hd = {tqh_first = 0x0, tqh_last = 0xccc25da4}, bv_root = 0x0, bv_cnt = 0x0}, bo_numoutput = 0x0, bo_flag = 0x0, bo_bsize = 0x4000}, v_pollinfo = 0x0, v_label = 0x0, v_lockf = 0x0, v_rl = { rl_waiters = {tqh_first = 0x0, tqh_last = 0xccc25dcc}, rl_currdep = 0x0}, v_cstart = 0x0, v_lasta = 0x0, v_lastw = 0x0, v_clen = 0x0, v_holdcnt = 0x2, v_usecount = 0x2, v_iflag = 0x200, v_vflag = 0x0, v_writecount = 0x0, v_hash = 0x2ebfc1, v_type = VFIFO} (kgdb) l *0xc099d095 0xc099d095 is in fifo_close (../../../fs/fifofs/fifo_vnops.c:267). 262 struct fifoinfo *fip; 263 struct pipe *cpipe; 264 265 vp = ap->a_vp; 266 fip = vp->v_fifoinfo; 267 cpipe = fip->fi_pipe; 268 ASSERT_VOP_ELOCKED(vp, "fifo_close"); 269 if (ap->a_fflag & FREAD) { 270 fip->fi_readers--; 271 if (fip->fi_readers == 0) { (kgdb) (kgdb) p *(struct inode *)(vp->v_data) $3 = {i_nextsnap = {tqe_next = 0x0, tqe_prev = 0x0}, i_vnode = 0xccc25ca8, i_ump = 0xc7c01a00, i_flag = 0x0, i_dev = 0xc7cff300, i_number = 0x2ebfc1, i_effnlink = 0x1, i_fs = 0xc90e3000, i_dquot = {0x0, 0x0}, i_count = 0x0, i_endoff = 0x0, i_diroff = 0x0, i_offset = 0x0, i_un = {dirhash = 0x0, snapblklist = 0x0}, i_ea_area = 0x0, i_ea_len = 0x0, i_ea_error = 0x0, i_ea_refs = 0x0, i_mode = 0x1180, i_nlink = 0x1, i_size = 0x0, i_flags = 0x0, i_gen = 0x139d01da, i_uid = 0x0, i_gid = 0x0, dinode_u = {din1 = 0xd00bce00, din2 = 0xd00bce00}} (kgdb) p *vp->v_mount $4 = {mnt_mtx = {lock_object = {lo_name = 0xc10b7d0a "struct mount mtx", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc757f5f0}, mtx_lock = 0x4}, mnt_gen = 0x1, mnt_list = {tqe_next = 0xc844c5d8, tqe_prev = 0xc7d288dc}, mnt_op = 0xc122e678, mnt_vfc = 0xc122e6f4, mnt_vnodecovered = 0xc8093438, mnt_syncer = 0xc8f60ca8, mnt_ref = 0x61b8, mnt_nvnodelist = {tqh_first = 0xc8f60e10, tqh_last = 0xd27e1880}, mnt_nvnodelistsize = 0x61b7, mnt_activevnodelist = {tqh_first = 0xc9a61168, tqh_last = 0xc8f60d60}, mnt_activevnodelistsize = 0x3b43, mnt_writeopcount = 0x1, mnt_kern_flag = 0x40802344, mnt_flag = 0x201100, mnt_opt = 0xc938ca20, mnt_optnew = 0x0, mnt_maxsymlinklen = 0x78, mnt_stat = {f_version = 0x20030518, f_type = 0x35, f_flags = 0x201100, f_bsize = 0x800, f_iosize = 0x4000, f_blocks = 0x70d53fd, f_bfree = 0x2ea89f2, f_bavail = 0x25a1cd4, f_files = 0x1d2d3fe, f_ffree = 0x1d1985b, f_syncwrites = 0x28c46d, f_asyncwrites = 0x2681f1, f_syncreads = 0x2c40a, f_asyncreads = 0x16a21, f_spare = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, f_namemax = 0xff, f_owner = 0x0, f_fsid = {val = {0x4c972dc2, 0x5b85c5ea}}, f_charspare = '\0' , f_fstypename = "ufs", '\0' , f_mntfromname = "/dev/label/tmp", '\0' , f_mntonname = "/tmp", '\0' }, mnt_cred = 0xc7ca2b00, mnt_data = 0xc7c01a00, mnt_time = 0x0, mnt_iosize_max = 0x20000, mnt_export = 0xc82d7400, mnt_label = 0x0, mnt_hashseed = 0x85d77a, mnt_lockref = 0x0, mnt_secondary_writes = 0x0, mnt_secondary_accwrites = 0x305b683, mnt_susp_owner = 0x0, mnt_gjprovider = 0x0, mnt_explock = { lock_object = {lo_name = 0xc10b7d1b "explock", lo_flags = 0x5730000, lo_data = 0x0, lo_witness = 0xc7584ad8}, lk_lock = 0x1, lk_exslpfail = 0x0, lk_timo = 0x0, lk_pri = 0x60, lk_stack = {depth = 0x6, pcs = { 0xc0a952bf, 0xc0b46d47, 0xc0b4d2f0, 0xc0b4c0d8, 0xc0ed656d, 0xc0ebeae1, 0x0 }}}, mnt_upper_link = {tqe_next = 0x0, tqe_prev = 0x0}, mnt_uppers = {tqh_first = 0xc844c5d8, tqh_last = 0xcfbefadc}} (kgdb) $ svn diff -x -p /usr/src/sys Index: /usr/src/sys/fs/nullfs/null_vnops.c =================================================================== --- /usr/src/sys/fs/nullfs/null_vnops.c (revision 250363) +++ /usr/src/sys/fs/nullfs/null_vnops.c (working copy) @@ -692,18 +692,21 @@ null_unlock(struct vop_unlock_args *ap) static int null_inactive(struct vop_inactive_args *ap __unused) { - struct vnode *vp; + struct vnode *vp, *lvp; struct mount *mp; struct null_mount *xmp; vp = ap->a_vp; + lvp = NULLVPTOLOWERVP(vp); mp = vp->v_mount; xmp = MOUNTTONULLMOUNT(mp); - if ((xmp->nullm_flags & NULLM_CACHE) == 0) { + if ((xmp->nullm_flags & NULLM_CACHE) == 0 || + (lvp->v_vflag & VV_NOSYNC) != 0) { /* * If this is the last reference and caching of the - * nullfs vnodes is not enabled, then free up the - * vnode so as not to tie up the lower vnodes. + * nullfs vnodes is not enabled, or the lower vnode is + * deleted, then free up the vnode so as not to tie up + * the lower vnodes. */ vp->v_object = NULL; vrecycle(vp); @@ -748,7 +751,10 @@ null_reclaim(struct vop_reclaim_args *ap) */ if (vp->v_writecount > 0) VOP_ADD_WRITECOUNT(lowervp, -1); - vput(lowervp); + if ((xp->null_flags & NULLV_NOUNLOCK) != 0) + vunref(lowervp); + else + vput(lowervp); free(xp, M_NULLFSNODE); return (0); Index: /usr/src/sys/fs/nullfs/null.h =================================================================== --- /usr/src/sys/fs/nullfs/null.h (revision 250363) +++ /usr/src/sys/fs/nullfs/null.h (working copy) @@ -53,8 +53,11 @@ struct null_node { LIST_ENTRY(null_node) null_hash; /* Hash list */ struct vnode *null_lowervp; /* VREFed once */ struct vnode *null_vnode; /* Back pointer */ + u_int null_flags; }; +#define NULLV_NOUNLOCK 0x0001 + #define MOUNTTONULLMOUNT(mp) ((struct null_mount *)((mp)->mnt_data)) #define VTONULL(vp) ((struct null_node *)(vp)->v_data) #define NULLTOV(xp) ((xp)->null_vnode) Index: /usr/src/sys/fs/nullfs/null_vfsops.c =================================================================== --- /usr/src/sys/fs/nullfs/null_vfsops.c (revision 250363) +++ /usr/src/sys/fs/nullfs/null_vfsops.c (working copy) @@ -65,7 +65,6 @@ static vfs_statfs_t nullfs_statfs; static vfs_unmount_t nullfs_unmount; static vfs_vget_t nullfs_vget; static vfs_extattrctl_t nullfs_extattrctl; -static vfs_reclaim_lowervp_t nullfs_reclaim_lowervp; /* * Mount null layer @@ -391,10 +390,24 @@ nullfs_reclaim_lowervp(struct mount *mp, struct vn vp = null_hashget(mp, lowervp); if (vp == NULL) return; + VTONULL(vp)->null_flags |= NULLV_NOUNLOCK; vgone(vp); - vn_lock(lowervp, LK_EXCLUSIVE | LK_RETRY); + vput(vp); } +static void +nullfs_unlink_lowervp(struct mount *mp, struct vnode *lowervp) +{ + struct vnode *vp; + + vp = null_hashget(mp, lowervp); + if (vp == NULL || vp->v_usecount > 1) + return; + VTONULL(vp)->null_flags |= NULLV_NOUNLOCK; + vgone(vp); + vput(vp); +} + static struct vfsops null_vfsops = { .vfs_extattrctl = nullfs_extattrctl, .vfs_fhtovp = nullfs_fhtovp, @@ -408,6 +421,7 @@ static struct vfsops null_vfsops = { .vfs_unmount = nullfs_unmount, .vfs_vget = nullfs_vget, .vfs_reclaim_lowervp = nullfs_reclaim_lowervp, + .vfs_unlink_lowervp = nullfs_unlink_lowervp, }; VFS_SET(null_vfsops, nullfs, VFCF_LOOPBACK | VFCF_JAIL); Index: /usr/src/sys/sys/mount.h =================================================================== --- /usr/src/sys/sys/mount.h (revision 250363) +++ /usr/src/sys/sys/mount.h (working copy) @@ -608,7 +608,7 @@ typedef int vfs_mount_t(struct mount *mp); typedef int vfs_sysctl_t(struct mount *mp, fsctlop_t op, struct sysctl_req *req); typedef void vfs_susp_clean_t(struct mount *mp); -typedef void vfs_reclaim_lowervp_t(struct mount *mp, struct vnode *lowervp); +typedef void vfs_notify_lowervp_t(struct mount *mp, struct vnode *lowervp); struct vfsops { vfs_mount_t *vfs_mount; @@ -626,7 +626,8 @@ struct vfsops { vfs_extattrctl_t *vfs_extattrctl; vfs_sysctl_t *vfs_sysctl; vfs_susp_clean_t *vfs_susp_clean; - vfs_reclaim_lowervp_t *vfs_reclaim_lowervp; + vfs_notify_lowervp_t *vfs_reclaim_lowervp; + vfs_notify_lowervp_t *vfs_unlink_lowervp; }; vfs_statfs_t __vfs_statfs; @@ -747,6 +748,14 @@ vfs_statfs_t __vfs_statfs; } \ } while (0) +#define VFS_UNLINK_LOWERVP(MP, VP) do { \ + if (*(MP)->mnt_op->vfs_unlink_lowervp != NULL) { \ + VFS_PROLOGUE(MP); \ + (*(MP)->mnt_op->vfs_unlink_lowervp)((MP), (VP)); \ + VFS_EPILOGUE(MP); \ + } \ +} while (0) + #define VFS_KNOTE_LOCKED(vp, hint) do \ { \ if (((vp)->v_vflag & VV_NOKNOTE) == 0) \ @@ -759,6 +768,9 @@ vfs_statfs_t __vfs_statfs; VN_KNOTE((vp), (hint), 0); \ } while (0) +#define VFS_NOTIFY_UPPER_RECLAIM 1 +#define VFS_NOTIFY_UPPER_UNLINK 2 + #include /* @@ -840,6 +852,7 @@ int vfs_modevent(module_t, int, void *); void vfs_mount_error(struct mount *, const char *, ...); void vfs_mountroot(void); /* mount our root filesystem */ void vfs_mountedfrom(struct mount *, const char *from); +void vfs_notify_upper(struct vnode *, int); void vfs_oexport_conv(const struct oexport_args *oexp, struct export_args *exp); void vfs_ref(struct mount *); Index: /usr/src/sys/kern/vfs_syscalls.c =================================================================== --- /usr/src/sys/kern/vfs_syscalls.c (revision 250363) +++ /usr/src/sys/kern/vfs_syscalls.c (working copy) @@ -1846,6 +1846,7 @@ restart: if (error) goto out; #endif + vfs_notify_upper(vp, VFS_NOTIFY_UPPER_UNLINK); error = VOP_REMOVE(nd.ni_dvp, vp, &nd.ni_cnd); #ifdef MAC out: @@ -3825,6 +3826,7 @@ restart: return (error); goto restart; } + vfs_notify_upper(vp, VFS_NOTIFY_UPPER_UNLINK); error = VOP_RMDIR(nd.ni_dvp, nd.ni_vp, &nd.ni_cnd); vn_finished_write(mp); out: Index: /usr/src/sys/kern/vfs_subr.c =================================================================== --- /usr/src/sys/kern/vfs_subr.c (revision 250363) +++ /usr/src/sys/kern/vfs_subr.c (working copy) @@ -2700,19 +2700,20 @@ vgone(struct vnode *vp) } static void -vgonel_reclaim_lowervp_vfs(struct mount *mp __unused, +notify_lowervp_vfs_dummy(struct mount *mp __unused, struct vnode *lowervp __unused) { } /* - * Notify upper mounts about reclaimed vnode. + * Notify upper mounts about reclaimed or unlinked vnode. */ -static void -vgonel_reclaim_lowervp(struct vnode *vp) +void +vfs_notify_upper(struct vnode *vp, int event) { static struct vfsops vgonel_vfsops = { - .vfs_reclaim_lowervp = vgonel_reclaim_lowervp_vfs + .vfs_reclaim_lowervp = notify_lowervp_vfs_dummy, + .vfs_unlink_lowervp = notify_lowervp_vfs_dummy, }; struct mount *mp, *ump, *mmp; @@ -2736,7 +2737,17 @@ static void } TAILQ_INSERT_AFTER(&mp->mnt_uppers, ump, mmp, mnt_upper_link); MNT_IUNLOCK(mp); - VFS_RECLAIM_LOWERVP(ump, vp); + switch (event) { + case VFS_NOTIFY_UPPER_RECLAIM: + VFS_RECLAIM_LOWERVP(ump, vp); + break; + case VFS_NOTIFY_UPPER_UNLINK: + VFS_UNLINK_LOWERVP(ump, vp); + break; + default: + KASSERT(0, ("invalid event %d", event)); + break; + } MNT_ILOCK(mp); ump = TAILQ_NEXT(mmp, mnt_upper_link); TAILQ_REMOVE(&mp->mnt_uppers, mmp, mnt_upper_link); @@ -2783,7 +2794,7 @@ vgonel(struct vnode *vp) active = vp->v_usecount; oweinact = (vp->v_iflag & VI_OWEINACT); VI_UNLOCK(vp); - vgonel_reclaim_lowervp(vp); + vfs_notify_upper(vp, VFS_NOTIFY_UPPER_RECLAIM); /* * Clean out any buffers associated with the vnode. -- Test scenario: nullfs5.sh + ^C + cleanup.sh