GENERIC from Sun Sep 2 19:02:18 2012 +0300, r239898+softdep_insmntque 5115806, vmcore.438 KDB: debugger backends: ddb KDB: current backend: ddb 524288K of memory above 4GB ignored Copyright (c) 1992-2012 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #0 r239898+5115806: Sun Sep 2 18:43:51 CEST 2012 pho@x4.osted.lan:/var/tmp/deviant2/sys/i386/compile/PHO i386 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. CPU: AMD Phenom(tm) 9150e Quad-Core Processor (1800.02-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x100f23 Family = 10 Model = 2 Stepping = 3 Features=0x178bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x7ff TSC: P-state invariant real memory = 4294967296 (4096 MB) avail memory = 3531583488 (3367 MB) : Trying to mount root from ufs:/dev/ad4s1a [rw]... WARNING: / was not properly dismounted Setting hostuuid: 00000000-0000-0000-0000-00218515337d. Setting hostid: 0x6b64ac17. Starting ddb. Entropy harvesting: interrupts ethernet point_to_point kickstart. warning: total configured swap (4194304 pages) exceeds maximum recommended amount (3890432 pages). warning: increase kern.maxswzone or reduce amount of swap. Starting file system checks: /dev/ad4s1a: 7561 files, 408788 used, 604227 free (1411 frags, 75352 blocks, 0.1% fragmentation) /dev/ad4s1f: 23307 files, 870975 used, 142040 free (4992 frags, 17131 blocks, 0.5% fragmentation) /dev/label/tmp: LINK COUNT DIR I=23552 OWNER=root MODE=40770 /dev/label/tmp: SIZE=512 MTIME=Sep 2 22:12 2012 COUNT 0 SHOULD BE 1 /dev/label/tmp: LINK COUNT INCREASING /dev/label/tmp: UNEXPECTED SOFT UPDATE INCONSISTENCY; RUN fsck MANUALLY. /dev/ad4s1d: 613178 files, 6542472 used, 3611727 free (243255 frags, 421059 blocks, 2.4% fragmentation) /dev/ad4s1e: 285891 files, 8200299 used, 17188218 free (54826 frags, 2141674 blocks, 0.2% fragmentation) THE FOLLOWING FILE SYSTEM HAD AN UNEXPECTED INCONSISTENCY: ufs: /dev/label/tmp (/tmp) File system preen failed, trying fsck -y ** /dev/ad4s1a ** Last Mounted on / ** Root file system ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 7561 files, 408788 used, 604227 free (1411 frags, 75352 blocks, 0.1% fragmentation) ***** FILE SYSTEM IS CLEAN ***** ** /dev/ad4s1f ** Last Mounted on /home ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 23307 files, 870975 used, 142040 free (4992 frags, 17131 blocks, 0.5% fragmentation) ***** FILE SYSTEM IS CLEAN ***** ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity UNREF DIR I=30546944 OWNER=root MODE=40770 SIZE=512 MTIME=Sep 2 22:12 2012 RECONNECT? yes DIR I=30546944 CONNECTED. PARENT WAS I=30476288 UNREF DIR I=30523392 OWNER=root MODE=40770 SIZE=512 MTIME=Sep 2 22:12 2012 RECONNECT? yes DIR I=30523392 CONNECTED. PARENT WAS I=30452736 UNREF DIR I=30499840 OWNER=root MODE=40770 SIZE=512 MTIME=Sep 2 22:12 2012 RECONNECT? yes DIR I=30499840 CONNECTED. PARENT WAS I=30334976 UNREF DIR I=30476288 OWNER=root MODE=40770 SIZE=512 MTIME=Sep 2 22:12 2012 RECONNECT? yes DIR I=30476288 CONNECTED. PARENT WAS I=30429184 UNREF DIR I=30452736 OWNER=root MODE=40770 SIZE=512 MTIME=Sep 2 22:12 2012 RECONNECT? yes DIR I=30452736 CONNECTED. PARENT WAS I=30382080 UNREF DIR I=30429184 OWNER=root MODE=40770 SIZE=512 MTIME=Sep 2 22:12 2012 RECONNECT? yes : : LINK COUNT DIR I=30523392 OWNER=root MODE=40770 SIZE=512 MTIME=Sep 2 22:12 2012 COUNT 0 SHOULD BE 2 LINK COUNT INCREASING UNEXPECTED SOFT UPDATE INCONSISTENCY ADJUST? yes LINK COUNT DIR I=30546944 OWNER=root MODE=40770 SIZE=512 MTIME=Sep 2 22:12 2012 COUNT 0 SHOULD BE 2 LINK COUNT INCREASING UNEXPECTED SOFT UPDATE INCONSISTENCY ADJUST? yes ** Phase 5 - Check Cyl groups FREE BLK COUNT(S) WRONG IN SUPERBLK SALVAGE? yes SUMMARY INFORMATION BAD SALVAGE? yes BLK(S) MISSING IN BIT MAPS SALVAGE? yes 2961 files, 56578133 used, 61735848 free (7344 frags, 7716063 blocks, 0.0% fragmentation) ***** FILE SYSTEM MARKED CLEAN ***** ***** FILE SYSTEM WAS MODIFIED ***** ** /dev/ad4s1d ** Last Mounted on /usr ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 613178 files, 6542472 used, 3611727 free (243255 frags, 421059 blocks, 2.4% fragmentation) ***** FILE SYSTEM IS CLEAN ***** ** /dev/ad4s1e ** Last Mounted on /var ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 285891 files, 8200299 used, 17188218 free (54826 frags, 2141674 blocks, 0.2% fragmentation) ***** FILE SYSTEM IS CLEAN ***** Mounting local file systems:. Setting hostname: x4.osted.lan. re0: link state changed to DOWN re0: link state changed to UP Starting Network: lo0 re0 fwe0 fwip0. lo0: flags=8049 metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 inet 127.0.0.1 netmask 0xff000000 nd6 options=21 re0: flags=8843 metric 0 mtu 1500 options=8209b ether 00:21:85:15:33:7d inet 192.168.1.101 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::221:85ff:fe15:337d%re0 prefixlen 64 scopeid 0x1 nd6 options=29 media: Ethernet autoselect (100baseTX ) status: active fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb nd6 options=29 ch 1 dma -1 fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 nd6 options=29 Starting devd. Starting Network: fwe0. fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb nd6 options=29 ch 1 dma -1 Starting Network: fwip0. fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 nd6 options=29 add net default: gateway 192.168.1.1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/kde4/lib /usr/local/lib/compat /usr/local/lib/compat/pkg /usr/local/lib/gcc46 /usr/local/lib/mysql /usr/local/lib/nss /usr/local/lib/qt4 /usr/local/lib/virtualbox a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout Creating and/or trimming log files. Starting syslogd. savecore: reboot Sep 2 22:35:3lo0 x4 savecore: rck eboot order reversal: 1st 0xe19145c0 bufwait (bufwait) @ kern/vfs_bio.c:2652 2nd 0xca892e00 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:284 KDB: stack backtrace: db_trace_self_wrapper(c0ff6c17,2e687361,38323a63,c1000a34,ca9820b4,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0adae4b,c0ffa70c,c12faeb8,11c,f272d7e0,...) at kdb_backtrace+0x2a _witness_debugger(c0ffa70c,ca892e00,c102c41a,c7d86818,c102c0ac,...) at _witness_debugger+0x25 witness_checkorder(ca892e00,9,c102c0a3,11c,0,...) at witness_checkorder+0x86f _sx_xlock(ca892e00,0,c102c0a3,11c,caeb8488,...) at _sx_xlock+0x9a ufsdirhash_acquire(e1914560,caeb8488,7c,e226d388,f272d8b0,...) at ufsdirhash_acquire+0x48 ufsdirhash_add(caeb8488,f272d91c,1388,f272d89c,f272d8a0,...) at ufsdirhash_add+0x13 ufs_direnter(cac5f870,caeaf9d8,f272d91c,f272dbc0,0,...) at ufs_direnter+0x799 ufs_makeinode(f272dbc0,c115b180,f272db14,f272da70,c0e44eb0,...) at ufs_makeinode+0x65d ufs_create(f272db14,c104e5ae,c116cfa0,cac5f870,f272db28,...) at ufs_create+0x30 VOP_CREATE_APV(c115abc0,f272db14,f272dbc0,f272daac,0,...) at VOP_CREATE_APV+0x100 vn_open_cred(f272db80,f272dc20,180,0,c7dbde80,...) at vn_open_cred+0x20a vn_open(f272db80,f272dc20,180,ca8d2498,0,...) at vn_open+0x3b kern_openat(ca982000,ffffff9c,28807000,0,602,...) at kern_openat+0x1dd kern_open(ca982000,28807000,0,601,180,...) at kern_open+0x35 sys_open(ca982000,f272dccc,c1046c7c,c0fdccd1,f272dcc4,...) at sys_open+0x30 syscall(f272dd08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (5, FreeBSD ELF32, sys_open), eip = 0x28188443, esp = 0xbfbfd2cc, ebp = 0xbfbfdde8 --- savecore: writing core to vmcore.437 Writing crash summary to /var/crash/core.txt.437. Additional ABI support: linux. Starting rpcbind. NFS access cache time=60 Clearing /tmp (X related). Starting mountd. Starting nfsuserd. Starting nfsd. Recovering vi editor sessions:lock order reversal: 1st 0xcb2ae058 ufs (ufs) @ kern/vfs_lookup.c:542 2nd 0xe1a798e0 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:261 3rd 0xcb1b5d00 ufs (ufs) @ kern/vfs_subr.c:2159 KDB: stack backtrace: db_trace_self_wrapper(c0ff6c17,3a632e73,a313632,0,60,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0adae4b,c0ffa725,c12fa878,86f,f2724354,...) at kdb_backtrace+0x2a _witness_debugger(c0ffa725,cb1b5d00,c0fe28cb,c7d867b0,c1002881,...) at _witness_debugger+0x25 witness_checkorder(cb1b5d00,9,c1002878,86f,0,...) at witness_checkorder+0x86f __lockmgr_args(cb1b5d00,80100,cb1b5d6c,0,0,...) at __lockmgr_args+0x8d5 ffs_lock(f272445c,c0aed5ec,c0fffc27,5e2,c0ff09b7,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c115abc0,f272445c,ca982694,c116cd20,cb1b5ca8,...) at VOP_LOCK1_APV+0xf3 _vn_lock(cb1b5ca8,80100,c1002878,86f,eb,...) at _vn_lock+0x78 vget(cb1b5ca8,80100,ca9825e0,50,0,...) at vget+0xbb vfs_hash_get(cac48000,15edc5,80000,ca9825e0,f27245a8,...) at vfs_hash_get+0xed ffs_vgetf(cac48000,15edc5,80000,f27245a8,1,...) at ffs_vgetf+0x49 softdep_sync_buf(cb2ae000,e1a79880,1,107,0,...) at softdep_sync_buf+0xac9 ffs_syncvnode(cb2ae000,1,0,ca982694,c12fa868,...) at ffs_syncvnode+0x24c ffs_truncate(cb2ae000,200,0,880,cb157600,...) at ffs_truncate+0x8b5 ufs_direnter(cb2ae000,cb1b5ca8,f272491c,f2724bc0,0,...) at ufs_direnter+0x980 ufs_makeinode(f2724bc0,c115b180,f2724b14,f2724a70,c0e44eb0,...) at ufs_makeinode+0x65d ufs_create(f2724b14,c104e5ae,c116cfa0,cb2ae000,f2724b28,...) at ufs_create+0x30 VOP_CREATE_APV(c115abc0,f2724b14,f2724bc0,f2724aac,0,...) at VOP_CREATE_APV+0x100 vn_open_cred(f2724b80,f2724c20,1b0,0,cb157600,...) at vn_open_cred+0x20a vn_open(f2724b80,f2724c20,1b0,ca928508,0,...) at vn_open+0x3b kern_openat(ca9825e0,ffffff9c,28c72f60,0,a03,...) at kern_openat+0x1dd kern_open(ca9825e0,28c72f60,0,a02,1b0,...) at kern_open+0x35 sys_open(ca9825e0,f2724ccc,c1046c7c,c0ffb45b,f2724cc4,...) at sys_open+0x30 syscall(f2724d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (5, FreeBSD ELF32, sys_open), eip = 0x283e1443, esp = 0xbfbfb52c, ebp = 0xbfbfb5b8 --- . Updating motd:. Starting ntpd. Starting default moused. Expensive timeout(9) function: 0xc08e2ca0(0xc1351700) 0.012533436 s Configuring syscons: keymap blanktime. Starting sshd. Starting cron. Local package initialization: backuplock order reversal: 1st 0xca95a328 ufs (ufs) @ kern/vfs_mount.c:1256 2nd 0xca9ec058 devfs (devfs) @ ufs/ffs/ffs_softdep.c:1834 KDB: stack backtrace: db_trace_self_wrapper(c0ff6c17,38313a63,a3433,cb4b8b98,8,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0adae4b,c0ffa70c,c12fa4a0,72a,f2868988,...) at kdb_backtrace+0x2a _witness_debugger(c0ffa70c,ca9ec058,c0fed854,c7d86610,c1028515,...) at _witness_debugger+0x25 witness_checkorder(ca9ec058,9,c102850c,72a,ca9ec0c4,...) at witness_checkorder+0x86f __lockmgr_args(ca9ec058,80400,ca9ec0c4,0,0,...) at __lockmgr_args+0x8d5 vop_stdlock(f2868a8c,c132fc40,4,c0ff09b7,eb,...) at vop_stdlock+0x65 VOP_LOCK1_APV(c111a580,f2868a8c,c132fc40,c116cd20,ca9ec000,...) at VOP_LOCK1_APV+0xf3 _vn_lock(ca9ec000,80400,c102850c,72a,0,...) at _vn_lock+0x78 softdep_flushworklist(cac485d8,f2868b44,cb53a8d0,612,cac3d4fc,...) at softdep_flushworklist+0x47 ffs_sync(cac485d8,1,f2868b98,522,c1baf710,...) at ffs_sync+0x3bf dounmount(cac485d8,8080000,cb53a8d0,4a3,f2868c1c,...) at dounmount+0x400 sys_unmount(cb53a8d0,f2868ccc,c1046c7c,c0ffb41e,f2868cc4,...) at sys_unmount+0x421 syscall(f2868d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280c487b, esp = 0xbfbfd67c, ebp = 0xbfbfd748 --- ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes Sep 2 22:38:37 x4 su: pho to root on /dev/pts/0 ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 2312 files, 56577213 used, 61736768 free (4664 frags, 7716513 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes Sep 2 22:41:22 x4 su: pho to root on /dev/pts/1 ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 2312 files, 56577213 used, 61736768 free (4664 frags, 7716513 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** fsck -y /tmp watchdogd. Starting inetd. Sun Sep 2 22:41:57 CEST 2012 FreeBSD/i386 (x4.osted.lan) (console) login: 20120902 22:43:44 all: nullfs.sh lock order reversal: 1st 0xcd722328 ufs (ufs) @ /var/tmp/deviant2/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:595 2nd 0xc11b05d4 allproc (allproc) @ kern/kern_descrip.c:2716 KDB: stack backtrace: db_trace_self_wrapper(c0ff6c17,3a632e70,36313732,7366000a,2f2e2e2f,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0adae4b,c0ffa70c,c12f9020,a9c,f275db04,...) at kdb_backtrace+0x2a _witness_debugger(c0ffa70c,c11b05d4,c0fea4a6,c7d7e0d0,c0feb935,...) at _witness_debugger+0x25 witness_checkorder(c11b05d4,1,c0feb92c,a9c,0,...) at witness_checkorder+0x86f _sx_slock(c11b05d4,0,c0feb92c,a9c,cd84e69c,...) at _sx_slock+0x9a mountcheckdirs(cd769870,cb6bd9d8,f275db98,522,c1baf710,...) at mountcheckdirs+0x41 dounmount(cd85fbb0,8080000,ca8f75e0,4a3,f275dc1c,...) at dounmount+0x3be sys_unmount(ca8f75e0,f275dccc,c1046c7c,c0ffb41e,f275dcc4,...) at sys_unmount+0x421 syscall(f275dd08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280c487b, esp = 0xbfbfd0fc, ebp = 0xbfbfd1c8 --- lock order reversal: 1st 0xcd722328 ufs (ufs) @ /var/tmp/deviant2/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:595 2nd 0xcd8a332c filedesc structure (filedesc structure) @ kern/kern_descrip.c:2721 KDB: stack backtrace: db_trace_self_wrapper(c0ff6c17,20296572,656b2040,6b2f6e72,5f6e7265,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0adae4b,c0ffa70c,c12f8a38,aa1,f28b4b04,...) at kdb_backtrace+0x2a _witness_debugger(c0ffa70c,cd8a332c,c0febbaf,c7d827e8,c0feb935,...) at _witness_debugger+0x25 witness_checkorder(cd8a332c,9,c0feb92c,aa1,0,...) at witness_checkorder+0x86f _sx_xlock(cd8a332c,0,c0feb92c,aa1,0,...) at _sx_xlock+0x9a mountcheckdirs(cb7a0168,cb6bd168,f28b4b98,522,c1baf710,...) at mountcheckdirs+0x94 dounmount(cd96c2ec,8080000,cd7eb000,4a3,f28b4c1c,...) at dounmount+0x3be sys_unmount(cd7eb000,f28b4ccc,c1046c7c,c0ffb41e,f28b4cc4,...) at sys_unmount+0x421 syscall(f28b4d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280c487b, esp = 0xbfbfd0fc, ebp = 0xbfbfd1c8 --- Fatal trap 12: page fault while in kernel mode cpuid = 3; apic id = 03 fault virtual address = 0x10 fault code = supervisor read, page not present instruction pointer = 0x20:0xc0a8c320 stack pointer = 0x28:0xf29f17e8 frame pointer = 0x28:0xf29f180c code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 47824 (ls) [ thread pid 47824 tid 100251 ] Stopped at _mtx_lock_flags+0x50: movl 0x10(%esi),%eax db> run pho db:0:pho> bt Tracing pid 47824 tid 100251 td 0xcd8ee000 _mtx_lock_flags(0,0,c1002878,ab1,cb74d5a0,...) at _mtx_lock_flags+0x50 vgonel(cb74d664,0,c1002878,a7e,cb74d5a0,...) at vgonel+0xe0 vgone(cb74d5a0,0,cd84e555,b3,0,...) at vgone+0x3c null_destroy_proto(cb74d5a0,cd944900,c1002878,45f,cd85f2ec,...) at null_destroy_proto+0xb9 insmntque1(cb74d5a0,cd85f2ec,cd84ccd0,cd944900,f29f1924,...) at insmntque1+0x13f null_nodeget(cd85f2ec,cb6d39d8,f29f1954,8000,cb6d9000,...) at null_nodeget+0x158 null_lookup(f29f19ec,c10035cf,c0b424ef,cd84fa80,f29f1980,...) at null_lookup+0xc3 VOP_LOOKUP_APV(cd84fa80,f29f19ec,f29f1aec,21e,c0aec75f,...) at VOP_LOOKUP_APV+0x108 lookup(f29f1aac,c1001c48,10e,dc,cd92e664,...) at lookup+0x6eb namei(f29f1aac,f29f1b14,60,0,cd8ee000,...) at namei+0x6ff kern_statat_vnhook(cd8ee000,200,ffffff9c,2882ac98,0,...) at kern_statat_vnhook+0xbf kern_statat(cd8ee000,200,ffffff9c,2882ac98,0,...) at kern_statat+0x3c kern_lstat(cd8ee000,2882ac98,0,f29f1bdc,2912ff78,...) at kern_lstat+0x36 sys_lstat(cd8ee000,f29f1ccc,c1046c7c,c0ffb46f,f29f1cc4,...) at sys_lstat+0x2f syscall(f29f1d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (190, FreeBSD ELF32, sys_lstat), eip = 0x281c1c13, esp = 0xbfbfd59c, ebp = 0xbfbfd628 --- db:0:bt> show allpcpu Current CPU: 3 cpuid = 0 dynamic pcpu = 0x6b1700 curthread = 0xc7e255e0: pid 12 "swi4: clock" curpcb = 0xc7b32d60 fpcurthread = none idlethread = 0xc7de28d0: tid 100003 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: cpuid = 1 dynamic pcpu = 0x69a1700 curthread = 0xcd8228d0: pid 47811 "grep" curpcb = 0xf299ad60 fpcurthread = none idlethread = 0xc7de25e0: tid 100004 "idle: cpu1" APIC ID = 1 currentldt = 0x50 spin locks held: cpuid = 2 dynamic pcpu = 0x69a4700 curthread = 0xcd8592f0: pid 47774 "umount" curpcb = 0xf292ad60 fpcurthread = none idlethread = 0xc7de22f0: tid 100005 "idle: cpu2" APIC ID = 2 currentldt = 0x50 spin locks held: cpuid = 3 dynamic pcpu = 0x69a7700 curthread = 0xcd8ee000: pid 47824 "ls" curpcb = 0xf29f1d60 fpcurthread = none idlethread = 0xc7de2000: tid 100006 "idle: cpu3" APIC ID = 3 currentldt = 0x50 spin locks held: db:0:allpcpu> show alllocks Process 47834 (sh) thread 0xcd8f3000 (100236) shared sx vm map (user) (vm map (user)) r = 0 (0xcac97ec8) locked @ vm/vm_map.c:3726 Process 47833 (sh) thread 0xcd99b8d0 (100287) shared sx vm map (user) (vm map (user)) r = 0 (0xca896130) locked @ vm/vm_map.c:3726 Process 47832 (mount_nullfs) thread 0xcd99bbc0 (100286) shared sx vm map (user) (vm map (user)) r = 0 (0xcd840c10) locked @ vm/vm_map.c:3726 Process 47831 (sh) thread 0xca8f72f0 (100098) shared sx vm map (user) (vm map (user)) r = 0 (0xcd8c14d0) locked @ vm/vm_map.c:3726 Process 47830 (grep) thread 0xcd8725e0 (100198) shared sx vm map (user) (vm map (user)) r = 0 (0xcd819788) locked @ vm/vm_map.c:3726 Process 47829 (sh) thread 0xcd7ec2f0 (100160) exclusive sleep mutex vm object (standard object) r = 0 (0xcd96f128) locked @ vm/vm_fault.c:938 shared sx vm map (user) (vm map (user)) r = 0 (0xca9424d0) locked @ vm/vm_map.c:3726 Process 47827 (grep) thread 0xca6e9000 (100111) exclusive sx vm map (user) (vm map (user)) r = 0 (0xca9425b8) locked @ vm/vm_mmap.c:577 Process 47826 (grep) thread 0xcd85d5e0 (100185) shared sx vm map (user) (vm map (user)) r = 0 (0xcd8196a0) locked @ vm/vm_map.c:3726 Process 47825 (sh) thread 0xcd99c5e0 (100283) shared sx vm map (user) (vm map (user)) r = 0 (0xc7de5788) locked @ vm/vm_map.c:3726 Process 47824 (ls) thread 0xcd8ee000 (100251) exclusive lockmgr null (null) r = 0 (0xcb74d5f8) locked @ /var/tmp/deviant2/sys/modules/nullfs/../../fs/nullfs/null_subr.c:174 shared lockmgr ufs (ufs) r = 0 (0xcd722328) locked @ /var/tmp/deviant2/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:595 Process 47822 (mount_nullfs) thread 0xca8f38d0 (100139) shared sx vm map (user) (vm map (user)) r = 0 (0xca942048) locked @ vm/vm_map.c:3726 Process 47818 (mount_nullfs) thread 0xcd9915e0 (100271) exclusive sleep mutex vm object (standard object) r = 0 (0xcd8cd128) locked @ vm/vm_fault.c:938 shared sx vm map (user) (vm map (user)) r = 0 (0xcd819218) locked @ vm/vm_map.c:3726 Process 47816 (grep) thread 0xcd99a000 (100295) exclusive sleep mutex vm page (vm page) r = 0 (0xc1330800) locked @ vm/vm_object.c:1890 exclusive sleep mutex vm object (standard object) r = 0 (0xcd8738ac) locked @ vm/vm_map.c:2727 exclusive sx vm map (user) (vm map (user)) r = 0 (0xcd818788) locked @ vm/vm_map.c:2869 Process 47812 (umount) thread 0xcd990bc0 (100274) exclusive lockmgr ufs (ufs) r = 0 (0xcb6bea30) locked @ kern/vfs_mount.c:1256 Process 47811 (grep) thread 0xcd8228d0 (100231) shared sx vm map (user) (vm map (user)) r = 0 (0xcd97ac10) locked @ vm/vm_map.c:3726 Process 47810 (grep) thread 0xcd7ed5e0 (100183) exclusive sleep mutex vm object (standard object) r = 0 (0xcd9fc4a0) locked @ vm/vm_fault.c:938 shared sx vm map (user) (vm map (user)) r = 0 (0xca896cf8) locked @ vm/vm_map.c:3726 Process 47809 (mount) thread 0xcd8f2000 (100241) shared sx vm map (user) (vm map (user)) r = 0 (0xcd840b28) locked @ vm/vm_map.c:3726 Process 47786 (grep) thread 0xca8a38d0 (100101) exclusive sleep mutex vm object (standard object) r = 0 (0xcd96f4a0) locked @ vm/vm_map.c:2727 exclusive sx vm map (user) (vm map (user)) r = 0 (0xcac81ec8) locked @ vm/vm_map.c:2869 Process 47775 (mount_nullfs) thread 0xcd9918d0 (100270) exclusive sleep mutex vm page (vm page) r = 0 (0xc1330c80) locked @ vm/vm_object.c:1890 exclusive sleep mutex vm object (standard object) r = 0 (0xcd72a250) locked @ vm/vm_map.c:2727 exclusive sx vm map (user) (vm map (user)) r = 0 (0xcd8194d0) locked @ vm/vm_map.c:2869 Process 47774 (umount) thread 0xcd8592f0 (100194) exclusive sleep mutex pmap (pmap) r = 0 (0xcac97d60) locked @ i386/i386/pmap.c:3455 exclusive rw pmap pv global (pmap pv global) r = 0 (0xc1350a80) locked @ i386/i386/pmap.c:3454 shared sx vm map (user) (vm map (user)) r = 0 (0xcac97cf8) locked @ vm/vm_map.c:3726 Process 1602 (sh) thread 0xcd821000 (100175) shared sx vm map (user) (vm map (user)) r = 0 (0xc7de4cf8) locked @ vm/vm_map.c:3726 Process 1593 (sh) thread 0xcb3732f0 (100131) exclusive sleep mutex vm object (standard object) r = 0 (0xcd8e7094) locked @ vm/vm_fault.c:938 shared sx vm map (user) (vm map (user)) r = 0 (0xcad61c10) locked @ vm/vm_map.c:3726 db:0:alllocks> show lockedvnods Locked vnodes 0xcb6be9d8: tag ufs, type VDIR usecount 1, writecount 0, refcount 1 mountedhere 0xcd85f2ec flags (VI(0x200)) v_object 0xcd76565c ref 0 pages 0 lock type ufs: EXCL by thread 0xcd990bc0 (pid 47812, umount, tid 100274) #0 0xc0a8399e at __lockmgr_args+0xd7e #1 0xc0cfd261 at ffs_lock+0xa1 #2 0xc0e43a73 at VOP_LOCK1_APV+0xf3 #3 0xc0b50bb8 at _vn_lock+0x78 #4 0xc0b3827c at dounmount+0x8c #5 0xc0b38ce1 at sys_unmount+0x421 #6 0xc0e1c35e at syscall+0x2fe #7 0xc0e06141 at Xint0x80_syscall+0x21 ino 141314, on dev ada0s1a 0xcd7222d0: tag ufs, type VDIR usecount 8, writecount 0, refcount 46 mountedhere 0 flags (VV_ROOT|VI(0x200)) v_object 0xcd76540c ref 0 pages 140 lock type ufs: SHARED (count 1) with exclusive waiters pending #0 0xc0a83288 at __lockmgr_args+0x668 #1 0xc0cfd261 at ffs_lock+0xa1 #2 0xc0e43a73 at VOP_LOCK1_APV+0xf3 #3 0xcd84e22f at null_lock+0xcf #4 0xc0e43a73 at VOP_LOCK1_APV+0xf3 #5 0xc0b50bb8 at _vn_lock+0x78 #6 0xc0b329a6 at lookup+0x106 #7 0xc0b33fef at namei+0x6ff #8 0xc0b4716f at kern_statat_vnhook+0xbf #9 0xc0b4736c at kern_statat+0x3c #10 0xc0b473a6 at kern_lstat+0x36 #11 0xc0b4744f at sys_lstat+0x2f #12 0xc0e1c35e at syscall+0x2fe #13 0xc0e06141 at Xint0x80_syscall+0x21 ino 2, on dev label/tmp 0xcb6d9000: tag null, type VDIR usecount 5, writecount 0, refcount 5 mountedhere 0 flags (VV_ROOT|VI(0x200)) v_object 0xcd76540c ref 0 pages 140 lock type ufs: SHARED (count 1) with exclusive waiters pending #0 0xc0a83288 at __lockmgr_args+0x668 #1 0xc0cfd261 at ffs_lock+0xa1 #2 0xc0e43a73 at VOP_LOCK1_APV+0xf3 #3 0xcd84e22f at null_lock+0xcf #4 0xc0e43a73 at VOP_LOCK1_APV+0xf3 #5 0xc0b50bb8 at _vn_lock+0x78 #6 0xc0b329a6 at lookup+0x106 #7 0xc0b33fef at namei+0x6ff #8 0xc0b4716f at kern_statat_vnhook+0xbf #9 0xc0b4736c at kern_statat+0x3c #10 0xc0b473a6 at kern_lstat+0x36 #11 0xc0b4744f at sys_lstat+0x2f #12 0xc0e1c35e at syscall+0x2fe #13 0xc0e06141 at Xint0x80_syscall+0x21 vp=0xcb6d9000, lowervp=0xcd7222d0 0xcd7699d8: tag null, type VDIR usecount 1, writecount 0, refcount 1 mountedhere 0 flags (VV_ROOT|VI(0x200)) v_object 0xcd76540c ref 0 pages 140 lock type ufs: SHARED (count 1) with exclusive waiters pending #0 0xc0a83288 at __lockmgr_args+0x668 #1 0xc0cfd261 at ffs_lock+0xa1 #2 0xc0e43a73 at VOP_LOCK1_APV+0xf3 #3 0xcd84e22f at null_lock+0xcf #4 0xc0e43a73 at VOP_LOCK1_APV+0xf3 #5 0xc0b50bb8 at _vn_lock+0x78 #6 0xc0b329a6 at lookup+0x106 #7 0xc0b33fef at namei+0x6ff #8 0xc0b4716f at kern_statat_vnhook+0xbf #9 0xc0b4736c at kern_statat+0x3c #10 0xc0b473a6 at kern_lstat+0x36 #11 0xc0b4744f at sys_lstat+0x2f #12 0xc0e1c35e at syscall+0x2fe #13 0xc0e06141 at Xint0x80_syscall+0x21 vp=0xcd7699d8, lowervp=0xcd7222d0 0xcb8f9e10: tag null, type VDIR usecount 1, writecount 0, refcount 1 mountedhere 0 flags (VV_ROOT|VI(0x200)) v_object 0xcd76540c ref 0 pages 140 lock type ufs: SHARED (count 1) with exclusive waiters pending #0 0xc0a83288 at __lockmgr_args+0x668 #1 0xc0cfd261 at ffs_lock+0xa1 #2 0xc0e43a73 at VOP_LOCK1_APV+0xf3 #3 0xcd84e22f at null_lock+0xcf #4 0xc0e43a73 at VOP_LOCK1_APV+0xf3 #5 0xc0b50bb8 at _vn_lock+0x78 #6 0xc0b329a6 at lookup+0x106 #7 0xc0b33fef at namei+0x6ff #8 0xc0b4716f at kern_statat_vnhook+0xbf #9 0xc0b4736c at kern_statat+0x3c #10 0xc0b473a6 at kern_lstat+0x36 #11 0xc0b4744f at sys_lstat+0x2f #12 0xc0e1c35e at syscall+0x2fe #13 0xc0e06141 at Xint0x80_syscall+0x21 vp=0xcb8f9e10, lowervp=0xcd7222d0 0xcb77d000: tag null, type VDIR usecount 1, writecount 0, refcount 1 mountedhere 0 flags (VV_ROOT|VI(0x200)) v_object 0xcd76540c ref 0 pages 140 lock type ufs: SHARED (count 1) with exclusive waiters pending #0 0xc0a83288 at __lockmgr_args+0x668 #1 0xc0cfd261 at ffs_lock+0xa1 #2 0xc0e43a73 at VOP_LOCK1_APV+0xf3 #3 0xcd84e22f at null_lock+0xcf #4 0xc0e43a73 at VOP_LOCK1_APV+0xf3 #5 0xc0b50bb8 at _vn_lock+0x78 #6 0xc0b329a6 at lookup+0x106 #7 0xc0b33fef at namei+0x6ff #8 0xc0b4716f at kern_statat_vnhook+0xbf #9 0xc0b4736c at kern_statat+0x3c #10 0xc0b473a6 at kern_lstat+0x36 #11 0xc0b4744f at sys_lstat+0x2f #12 0xc0e1c35e at syscall+0x2fe #13 0xc0e06141 at Xint0x80_syscall+0x21 vp=0xcb77d000, lowervp=0xcd7222d0 0xcb94cca8: tag null, type VDIR usecount 1, writecount 0, refcount 1 mountedhere 0 flags (VV_ROOT|VI(0x200)) lock type ufs: SHARED (count 1) with exclusive waiters pending #0 0xc0a83288 at __lockmgr_args+0x668 #1 0xc0cfd261 at ffs_lock+0xa1 #2 0xc0e43a73 at VOP_LOCK1_APV+0xf3 #3 0xcd84e22f at null_lock+0xcf #4 0xc0e43a73 at VOP_LOCK1_APV+0xf3 #5 0xc0b50bb8 at _vn_lock+0x78 #6 0xc0b329a6 at lookup+0x106 #7 0xc0b33fef at namei+0x6ff #8 0xc0b4716f at kern_statat_vnhook+0xbf #9 0xc0b4736c at kern_statat+0x3c #10 0xc0b473a6 at kern_lstat+0x36 #11 0xc0b4744f at sys_lstat+0x2f #12 0xc0e1c35e at syscall+0x2fe #13 0xc0e06141 at Xint0x80_syscall+0x21 vp=0xcb94cca8, lowervp=0xcd7222d0 0xcb76c870: tag null, type VDIR usecount 1, writecount 0, refcount 1 mountedhere 0 flags (VV_ROOT|VI(0x200)) v_object 0xcd76540c ref 0 pages 140 lock type ufs: SHARED (count 1) with exclusive waiters pending #0 0xc0a83288 at __lockmgr_args+0x668 #1 0xc0cfd261 at ffs_lock+0xa1 #2 0xc0e43a73 at VOP_LOCK1_APV+0xf3 #3 0xcd84e22f at null_lock+0xcf #4 0xc0e43a73 at VOP_LOCK1_APV+0xf3 #5 0xc0b50bb8 at _vn_lock+0x78 #6 0xc0b329a6 at lookup+0x106 #7 0xc0b33fef at namei+0x6ff #8 0xc0b4716f at kern_statat_vnhook+0xbf #9 0xc0b4736c at kern_statat+0x3c #10 0xc0b473a6 at kern_lstat+0x36 #11 0xc0b4744f at sys_lstat+0x2f #12 0xc0e1c35e at syscall+0x2fe #13 0xc0e06141 at Xint0x80_syscall+0x21 vp=0xcb76c870, lowervp=0xcd7222d0 0xcb7ad9d8: tag null, type VDIR usecount 1, writecount 0, refcount 1 mountedhere 0 flags (VV_ROOT|VI(0x200)) lock type ufs: SHARED (count 1) with exclusive waiters pending #0 0xc0a83288 at __lockmgr_args+0x668 #1 0xc0cfd261 at ffs_lock+0xa1 #2 0xc0e43a73 at VOP_LOCK1_APV+0xf3 #3 0xcd84e22f at null_lock+0xcf #4 0xc0e43a73 at VOP_LOCK1_APV+0xf3 #5 0xc0b50bb8 at _vn_lock+0x78 #6 0xc0b329a6 at lookup+0x106 #7 0xc0b33fef at namei+0x6ff #8 0xc0b4716f at kern_statat_vnhook+0xbf #9 0xc0b4736c at kern_statat+0x3c #10 0xc0b473a6 at kern_lstat+0x36 #11 0xc0b4744f at sys_lstat+0x2f #12 0xc0e1c35e at syscall+0x2fe #13 0xc0e06141 at Xint0x80_syscall+0x21 vp=0xcb7ad9d8, lowervp=0xcd7222d0 0xcb758438: tag null, type VDIR usecount 1, writecount 0, refcount 1 mountedhere 0 flags (VV_ROOT|VI(0x200)) v_object 0xcd76540c ref 0 pages 140 lock type ufs: SHARED (count 1) with exclusive waiters pending #0 0xc0a83288 at __lockmgr_args+0x668 #1 0xc0cfd261 at ffs_lock+0xa1 #2 0xc0e43a73 at VOP_LOCK1_APV+0xf3 #3 0xcd84e22f at null_lock+0xcf #4 0xc0e43a73 at VOP_LOCK1_APV+0xf3 #5 0xc0b50bb8 at _vn_lock+0x78 #6 0xc0b329a6 at lookup+0x106 #7 0xc0b33fef at namei+0x6ff #8 0xc0b4716f at kern_statat_vnhook+0xbf #9 0xc0b4736c at kern_statat+0x3c #10 0xc0b473a6 at kern_lstat+0x36 #11 0xc0b4744f at sys_lstat+0x2f #12 0xc0e1c35e at syscall+0x2fe #13 0xc0e06141 at Xint0x80_syscall+0x21 vp=0xcb758438, lowervp=0xcd7222d0 db:0:lockedvnods> show mount 0xca829bb0 /dev/ad4s1a on / (ufs) 0xca82a000 devfs on /dev (devfs) 0xca8298c4 /dev/ad4s1f on /home (ufs) 0xcac482ec /dev/ad4s1d on /usr (ufs) 0xcac48000 /dev/ad4s1e on /var (ufs) 0xca82abb0 procfs on /proc (procfs) 0xca82a8c4 /dev/label/tmp on /tmp (ufs) 0xcd85f2ec /tmp on /mnt6 (nullfs) 0xcd96b000 /tmp on /mnt12 (nullfs) 0xcd85fbb0 /tmp on /mnt3 (nullfs) 0xcd96c2ec /tmp on /mnt14 (nullfs) 0xcd8528c4 /tmp on /mnt8 (nullfs) 0xcd981bb0 /tmp on /mnt5 (nullfs) 0xcd852000 /tmp on /mnt9 (nullfs) 0xcd9725d8 /tmp on /mnt4 (nullfs) More info: show mount db:0:mount> ps pid ppid pgrp uid state wmesg wchan cmd 47834 1592 1421 0 RL+ sh 47833 1601 1421 0 RL+ sh 47832 1599 1421 0 RL+ mount_nullfs 47831 1604 1421 0 RL+ sh 47830 1604 1421 0 RL+ grep 47829 1604 1421 0 LL+ *vm page 0xcd85ba80 sh 47828 1593 1421 0 RE+ sh 47827 1594 1421 0 R+ grep 47826 1594 1421 0 RL+ grep 47825 1594 1421 0 RL+ sh 47824 47823 1368 0 R CPU 3 ls 47823 1368 1368 0 S wait 0xca977b30 sh 47822 1600 1421 0 RL+ mount_nullfs 47818 1591 1421 0 LL+ *vm page 0xcd85ba80 mount_nullfs 47816 1597 1421 0 RE+ grep 47813 2456 2450 1001 S nanslp 0xc11b1d64 sleep 47812 1595 1421 0 D+ ufs 0xcd722328 umount 47811 1603 1421 0 RL+ CPU 1 grep 47810 1603 1421 0 LL+ *vm page 0xcd85ba80 grep 47809 1603 1421 0 RL+ mount 47786 1596 1421 0 LE+ *vm page 0xcd85ae00 grep 47775 1598 1421 0 RE+ mount_nullfs 47774 1590 1421 0 RL+ CPU 2 umount 2457 2450 2450 1001 S piperd 0xca92a990 awk 2456 2450 2450 1001 S wait 0xcd855000 sh 2450 2449 2450 1001 Ss wait 0xca977598 sh 2449 2408 2408 1001 S select 0xcd8b22e4 sshd 2448 2447 2448 1001 Ss+ select 0xcd85cae4 top 2447 2397 2397 1001 S select 0xcd8b22a4 sshd 2440 2439 2440 1001 Ss kqread 0xcd7ef680 tail 2439 2394 2394 1001 S select 0xcb538664 sshd 2408 1208 2408 0 Ss select 0xca7990a4 sshd 2397 1208 2397 0 Ss select 0xcd85cb24 sshd 2394 1208 2394 0 Ss select 0xcd8b2324 sshd 1604 1536 1421 0 S+ wait 0xcd81d2cc sh 1603 1536 1421 0 S+ wait 0xcd81d598 sh 1602 1536 1421 0 RL+ sh 1601 1536 1421 0 S+ wait 0xcd81db30 sh 1600 1536 1421 0 S+ wait 0xcd81e000 sh 1599 1536 1421 0 S+ wait 0xcd81e2cc sh 1598 1536 1421 0 S+ wait 0xcd81e598 sh 1597 1536 1421 0 S+ wait 0xcd81e864 sh 1596 1536 1421 0 S+ wait 0xcd81eb30 sh 1595 1536 1421 0 S+ wait 0xcd81f000 sh 1594 1536 1421 0 S+ wait 0xcd7e8864 sh 1593 1536 1421 0 LL+ *vm page 0xcd85ba80 sh 1592 1536 1421 0 S+ wait 0xca8ef598 sh 1591 1536 1421 0 S+ wait 0xcd7e7864 sh 1590 1536 1421 0 S+ wait 0xcb528864 sh 1536 1421 1421 0 S+ wait 0xcd7e7598 sh 1421 1324 1421 0 S+ wait 0xcb51b2cc sh 1418 1 1418 0 Ss+ ttyin 0xc7f00870 getty 1417 1 1417 0 Ss+ ttyin 0xc7f00a70 getty 1416 1 1416 0 Ss+ ttyin 0xc7f00c70 getty 1415 1 1415 0 Ss+ ttyin 0xc7f00e70 getty 1414 1 1414 0 Ss+ ttyin 0xc9626070 getty 1413 1 1413 0 Ss+ ttyin 0xc9626270 getty 1412 1 1412 0 Ss+ ttyin 0xc9626470 getty 1411 1 1411 0 Ss+ ttyin 0xc7eff070 getty 1410 1 1410 0 Ss+ ttyin 0xc7eff270 getty 1393 1 1393 0 Ss select 0xc94e7424 inetd 1368 1 1368 0 Ss wait 0xcb528b30 watchdogd 1344 1341 1344 0 S+ ttyin 0xca626e70 bash 1341 1340 1341 0 S+ pause 0xca97b8bc csh 1340 1335 1340 1001 S+ wait 0xca97cb30 su 1335 1334 1335 1001 Ss+ wait 0xca6d5598 bash 1334 1332 1332 1001 S select 0xcb538764 sshd 1332 1208 1332 0 Ss select 0xca7fd7a4 sshd 1324 1321 1324 0 S+ wait 0xcb519598 bash 1321 1320 1321 0 S+ pause 0xcb5298bc csh 1320 1310 1320 1001 S+ wait 0xca97b2cc su 1310 1309 1310 1001 Ss+ wait 0xcb519b30 bash 1309 1307 1307 1001 S select 0xcb5387e4 sshd 1307 1208 1307 0 Ss select 0xc94e73e4 sshd 1219 1 1219 0 Ss nanslp 0xc11b1d64 cron 1214 1 1214 25 Ss pause 0xca8f25f0 sendmail 1211 1 1211 0 Ss select 0xca7fd964 sendmail 1208 1 1208 0 Ss select 0xc94e74e4 sshd 1176 1 1176 0 Ss select 0xca7fd9a4 moused 1151 1 1151 0 Ss select 0xc94e78e4 ntpd 1095 1092 1092 0 S (threaded) nfsd 100121 S rpcsvc 0xca7903d0 nfsd: service 100120 S rpcsvc 0xca790410 nfsd: service 100119 S rpcsvc 0xca790110 nfsd: service 100118 S rpcsvc 0xca7e6950 nfsd: master 1092 1 1092 0 Ss select 0xca7991a4 nfsd 1090 1086 1086 0 S select 0xca7fe424 nfsuserd 1089 1086 1086 0 S select 0xca7fd9e4 nfsuserd 1088 1086 1086 0 S select 0xca7fe464 nfsuserd 1087 1086 1086 0 S select 0xca7fda24 nfsuserd 1086 1 1086 0 Ss pause 0xca8ef058 nfsuserd 1070 1 1070 0 Ss select 0xca7fda64 mountd 1037 1 1037 0 Ss select 0xc94e7aa4 rpcbind 967 1 967 0 Ss select 0xca799164 syslogd 844 1 844 0 Ss select 0xc961c6a4 devd 19 0 0 0 DL sdflush 0xc132fc80 [softdepflush] 18 0 0 0 DL vlruwt 0xca6d42cc [vnlru] 17 0 0 0 DL syncer 0xc1327f74 [syncer] 16 0 0 0 DL psleep 0xc1327c08 [bufdaemon] 9 0 0 0 DL pgzero 0xc1331c5c [pagezero] 8 0 0 0 DL psleep 0xc13317c8 [vmdaemon] 7 0 0 0 DL psleep 0xc1331790 [pagedaemon] 6 0 0 0 DL ccb_scan 0xc1174654 [xpt_thrd] 5 0 0 0 DL waiting_ 0xc132b498 [sctp_iterator] 4 0 0 0 DL - 0xc949c63c [fdc0] 3 0 0 0 SL - 0xc9522000 [fw0_probe] 15 0 0 0 DL (threaded) [usb] 100059 D - 0xc9506d34 [usbus5] 100058 D - 0xc9506d04 [usbus5] 100057 D - 0xc9506cd4 [usbus5] 100056 D - 0xc9506ca4 [usbus5] 100054 D - 0xc94f4b5c [usbus4] 100053 D - 0xc94f4b2c [usbus4] 100052 D - 0xc94f4afc [usbus4] 100051 D - 0xc94f4acc [usbus4] 100050 D - 0xc94efb5c [usbus3] 100049 D - 0xc94efb2c [usbus3] 100048 D - 0xc94efafc [usbus3] 100047 D - 0xc94efacc [usbus3] 100046 D - 0xc94ebb5c [usbus2] 100045 D - 0xc94ebb2c [usbus2] 100044 D - 0xc94ebafc [usbus2] 100043 D - 0xc94ebacc [usbus2] 100041 D - 0xc94e2b5c [usbus1] 100040 D - 0xc94e2b2c [usbus1] 100039 D - 0xc94e2afc [usbus1] 100038 D - 0xc94e2acc [usbus1] 100036 D - 0xc94dcb5c [usbus0] 100035 D - 0xc94dcb2c [usbus0] 100034 D - 0xc94dcafc [usbus0] 100033 D - 0xc94dcacc [usbus0] 2 0 0 0 DL ctl_work 0xc7fdd000 [ctl_thrd] 14 0 0 0 DL - 0xc11b0b64 [yarrow] 13 0 0 0 DL (threaded) [geom] 100015 D - 0xc11adc08 [g_down] 100014 D - 0xc11adc04 [g_up] 100013 D - 0xc11adbfc [g_event] 12 0 0 0 RL (threaded) [intr] 100067 I [irq12: psm0] 100066 I [irq1: atkbd0] 100064 I [swi0: uart] 100061 I [irq20: fwohci0] 100060 I [irq14: ata0] 100055 I [irq19: ehci0] 100042 I [irq18: ohci2 ohci4] 100037 I [irq17: ohci1 ohci3] 100032 I [irq16: hdac1 ohci0] 100031 I [irq22: ahci0] 100030 I [irq257: re0] 100029 I [irq256: hdac0] 100026 I [swi5: fast taskq] 100024 I [swi2: cambio] 100020 I [swi6: task queue] 100019 I [swi6: Giant taskq] 100012 I [swi1: netisr 0] 100011 I [swi3: vm] 100010 I [swi4: clock] 100009 I [swi4: clock] 100008 I [swi4: clock] 100007 Run CPU 0 [swi4: clock] 11 0 0 0 RL (threaded) [idle] 100006 CanRun [idle: cpu3] 100005 CanRun [idle: cpu2] 100004 CanRun [idle: cpu1] 100003 CanRun [idle: cpu0] 1 0 1 0 SLs wait 0xc7de0b30 [init] 10 0 0 0 DL audit_wo 0xc132f3c0 [audit] 0 0 0 0 DLs (threaded) [kernel] 100069 D - 0xc11b0b64 [deadlkres] 100068 D - 0xc7d9a500 [mca taskq] 100062 D - 0xc7d9a740 [fw0_taskq] 100027 D - 0xc7d9a880 [ffs_trim taskq] 100025 D - 0xc7d9a900 [kqueue taskq] 100023 D - 0xc7d9a940 [acpi_task_2] 100022 D - 0xc7d9a940 [acpi_task_1] 100021 D - 0xc7d9a940 [acpi_task_0] 100018 D - 0xc7d9aa00 [thread taskq] 100016 D - 0xc7d9ab40 [firmware taskq] 100000 D sched 0xc11addc0 [swapper] db:0:ps> allt Tracing command sh pid 47834 tid 100236 td 0xcd8f3000 sched_switch(cd8f3000,0,103,1ae,c11bf6c0,...) at sched_switch+0x458 mi_switch(103,0,c0ff8ff6,2e8,cd8f3000,...) at mi_switch+0x243 turnstile_wait(c94fdb00,cd99bbc0,0,c0aed5ec,c0aeb477,...) at turnstile_wait+0x504 _rw_wlock_hard(c1350a80,cd8f3000,c104581a,d7e,0,...) at _rw_wlock_hard+0x365 _rw_wlock(c1350a80,c104581a,d7e,c7d82bf8,cd8f30b4,...) at _rw_wlock+0xc4 pmap_enter(cac97f30,bfbfd000,2,c46ea698,7,...) at pmap_enter+0xac vm_fault_hold(cac97e80,bfbfd000,2,0,0,...) at vm_fault_hold+0x194c vm_fault(cac97e80,bfbfd000,2,0,bfbfd5c8,...) at vm_fault+0x8b trap_pfault(c11be9c0,0,c0ff319f,d5,cd8ed000,...) at trap_pfault+0x1c2 trap(f29c4d08) at trap+0x3b6 calltrap() at calltrap+0x6 --- trap 0xc, eip = 0x8054c3f, esp = 0xbfbfd5b0, ebp = 0xbfbfd5d8 --- Tracing command sh pid 47833 tid 100287 td 0xcd99b8d0 sched_switch(cd99b8d0,0,103,1ae,c11bf6c0,...) at sched_switch+0x458 mi_switch(103,0,c0ff8ff6,2e8,cd99b8d0,...) at mi_switch+0x243 turnstile_wait(c94fdb00,cd99bbc0,0,c0aed5ec,c0aeb477,...) at turnstile_wait+0x504 _rw_wlock_hard(c1350a80,cd99b8d0,c104581a,d7e,0,...) at _rw_wlock_hard+0x365 _rw_wlock(c1350a80,c104581a,d7e,c7d82bf8,cd99b984,...) at _rw_wlock+0xc4 pmap_enter(ca896198,bfbfd000,2,c46f4610,7,...) at pmap_enter+0xac vm_fault_hold(ca8960e8,bfbfd000,2,0,0,...) at vm_fault_hold+0x194c vm_fault(ca8960e8,bfbfd000,2,0,bfbfd5c8,...) at vm_fault+0x8b trap_pfault(c0e1b86f,0,cd99b8d0,cd998000,cd998000,...) at trap_pfault+0x1c2 trap(f2a79d08) at trap+0x3b6 calltrap() at calltrap+0x6 --- trap 0xc, eip = 0x8054c3f, esp = 0xbfbfd5b0, ebp = 0xbfbfd5d8 --- Tracing command mount_nullfs pid 47832 tid 100286 td 0xcd99bbc0 sched_switch(cd99bbc0,0,602,1ae,0,...) at sched_switch+0x458 mi_switch(602,0,c0ff319f,cd,cd99bbc0,...) at mi_switch+0x243 critical_exit(c1391460,0,2,c1350a80,c1350a80,...) at critical_exit+0xb8 lapic_handle_timer(f2a768a8) at lapic_handle_timer+0x95 Xtimerint() at Xtimerint+0x20 --- interrupt, eip = 0xc0a9c4b3, esp = 0xf2a768e8, ebp = 0xf2a76948 --- _rw_wlock_hard(c1350a80,cd99bbc0,c104581a,d7e,0,...) at _rw_wlock_hard+0x1e3 _rw_wlock(c1350a80,c104581a,d7e,f2a769b4,c0a8bbba,...) at _rw_wlock+0xc4 pmap_enter(cd840c78,2819a000,1,c5618ac0,3,...) at pmap_enter+0xac vm_fault_hold(cd840bc8,2819a000,1,0,0,...) at vm_fault_hold+0x194c vm_fault(cd840bc8,2819a000,1,0,2819ab40,...) at vm_fault+0x8b trap_pfault(cd9982cc,f2a76c44,c0ae4fd8,cd99bbc0,cd9982cc,...) at trap_pfault+0x1c2 trap(f2a76d08) at trap+0x3b6 calltrap() at calltrap+0x6 --- trap 0xc, eip = 0x2817b110, esp = 0xbfbfd170, ebp = 0xbfbfd188 --- Tracing command sh pid 47831 tid 100098 td 0xca8f72f0 sched_switch(ca8f72f0,0,103,1ae,c11bf6c0,...) at sched_switch+0x458 mi_switch(103,0,c0ff8ff6,2e8,ca8f72f0,...) at mi_switch+0x243 turnstile_wait(c94fdb00,cd99bbc0,0,c0aed5ec,c0aeb477,...) at turnstile_wait+0x504 _rw_wlock_hard(c1350a80,ca8f72f0,c104581a,d7e,0,...) at _rw_wlock_hard+0x365 _rw_wlock(c1350a80,c104581a,d7e,c7d82bf8,ca8f73a4,...) at _rw_wlock+0xc4 pmap_enter(cd8c1538,bfbfd000,2,c46ed860,7,...) at pmap_enter+0xac vm_fault_hold(cd8c1488,bfbfd000,2,0,0,...) at vm_fault_hold+0x194c vm_fault(cd8c1488,bfbfd000,2,0,bfbfd6c8,...) at vm_fault+0x8b trap_pfault(c0e1b86f,0,ca8f72f0,ca8f2000,ca8f2000,...) at trap_pfault+0x1c2 trap(f2761d08) at trap+0x3b6 calltrap() at calltrap+0x6 --- trap 0xc, eip = 0x8054c3f, esp = 0xbfbfd6b0, ebp = 0xbfbfd6d8 --- Tracing command grep pid 47830 tid 100198 td 0xcd8725e0 sched_switch(cd8725e0,0,103,1ae,c11bf6c0,...) at sched_switch+0x458 mi_switch(103,0,c0ff8ff6,2e8,cd8725e0,...) at mi_switch+0x243 turnstile_wait(c94fdb00,cd99bbc0,0,c0aed5ec,d1e57974,...) at turnstile_wait+0x504 _rw_wlock_hard(c1350a80,cd8725e0,c104581a,d7e,0,...) at _rw_wlock_hard+0x365 _rw_wlock(c1350a80,c104581a,d7e,c7d82bf8,cd872694,...) at _rw_wlock+0xc4 pmap_enter(cd8197f0,2809b000,2,c561f798,3,...) at pmap_enter+0xac vm_fault_hold(cd819740,2809b000,2,0,0,...) at vm_fault_hold+0x194c vm_fault(cd819740,2809b000,2,0,2809be40,...) at vm_fault+0x8b trap_pfault(cd864598,f2937c44,c0ae4fd8,cd8725e0,cd864598,...) at trap_pfault+0x1c2 trap(f2937d08) at trap+0x3b6 calltrap() at calltrap+0x6 --- trap 0xc, eip = 0x28068c83, esp = 0xbfbfcef0, ebp = 0xbfbfcfa8 --- Tracing command sh pid 47829 tid 100160 td 0xcd7ec2f0 sched_switch(cd7ec2f0,0,103,1ae,c94fdb00,...) at sched_switch+0x458 mi_switch(103,0,c0ff8ff6,2e8,c1330800,...) at mi_switch+0x243 turnstile_wait(cd85ba80,cd99a000,0,1c5,0,...) at turnstile_wait+0x504 _mtx_lock_sleep(c1330800,cd7ec2f0,0,c102e410,3ab,...) at _mtx_lock_sleep+0x2b7 _mtx_lock_flags(c1330800,0,c102e410,3ab,3,...) at _mtx_lock_flags+0x10f vm_fault_hold(ca942488,28805000,2,0,0,...) at vm_fault_hold+0x1d2d vm_fault(ca942488,28805000,2,0,288050a0,...) at vm_fault+0x8b trap_pfault(c11afa70,0,c1046bd2,30a,cd7e8598,...) at trap_pfault+0x1c2 trap(f2892d08) at trap+0x3b6 calltrap() at calltrap+0x6 --- trap 0xc, eip = 0x281f719f, esp = 0xbfbfd4e0, ebp = 0xbfbfd528 --- Tracing command sh pid 47828 tid 100161 td 0xcd7ec000 sched_switch(cd7ec000,0,104,1b3,f2895a4c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,352,...) at mi_switch+0x243 sleepq_switch(cd7ec000,0,c0ff8600,291,2,...) at sleepq_switch+0x1a7 sleepq_timedwait(c11b0b64,0,c102fe92,2,0,...) at sleepq_timedwait+0x6b _sleep(c11b0b64,0,0,c102fe92,1) at _sleep+0x369 pause(c102fe92,1,c102fa61,227,246,...) at pause+0xa0 vm_object_deallocate(cda004a0,cd8ca678,0,cd968cf0,cd8d56c0,...) at vm_object_deallocate+0x44f vm_map_entry_deallocate(117c,cac97910,0,0,cac97910,...) at vm_map_entry_deallocate+0x21 vm_map_process_deferred(cac97958,c102ebff,b38,cd7e82cc,f2895bbc,...) at vm_map_process_deferred+0x75 _vm_map_unlock(cac97910,c102ebff,b38,cac979c0,cac97910,...) at _vm_map_unlock+0x4a vm_map_remove(cac97910,1000,bfc00000,cd7ec000,cd7e82cc,...) at vm_map_remove+0x6b vmspace_exit(cd7ec000,0,c0fec70b,139,f2895bf8,...) at vmspace_exit+0xc3 exit1(cd7ec000,0,f2895cfc,c0e1c35e,cd7ec000,...) at exit1+0x617 sys_sys_exit(cd7ec000,f2895ccc,c1046c7c,c0fc5865,c0ac9607,...) at sys_sys_exit+0x1d syscall(f2895d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (1, FreeBSD ELF32, sys_sys_exit), eip = 0x28173ccf, esp = 0xbfbfd7bc, ebp = 0xbfbfd848 --- Tracing command grep pid 47827 tid 100111 td 0xca6e9000 sched_switch(ca6e9000,0,103,1ae,c11bf6c0,...) at sched_switch+0x458 mi_switch(103,0,c0ff8ff6,2e8,ca6e9000,...) at mi_switch+0x243 turnstile_wait(c94fdb00,cd99bbc0,0,c0aed5ec,c7b15c80,...) at turnstile_wait+0x504 _rw_wlock_hard(c1350a80,ca6e9000,c104581a,b74,0,...) at _rw_wlock_hard+0x365 _rw_wlock(c1350a80,c104581a,b74,c7b15c80,f27ffb9c,...) at _rw_wlock+0xc4 pmap_remove(ca942620,2807a000,2807b000,69a1700,ca6e9000,...) at pmap_remove+0x3c vm_map_delete(ca942570,2807a000,2807b000,4,ca942570,...) at vm_map_delete+0x1a5 sys_munmap(ca6e9000,f27ffccc,c11bf040,f27ffc98,c0ac9607,...) at sys_munmap+0xf6 syscall(f27ffd08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (73, FreeBSD ELF32, sys_munmap), eip = 0x28066f33, esp = 0xbfbfceec, ebp = 0xbfbfcf98 --- Tracing command grep pid 47826 tid 100185 td 0xcd85d5e0 sched_switch(cd85d5e0,0,103,1ae,c11bf6c0,...) at sched_switch+0x458 mi_switch(103,0,c0ff8ff6,2e8,cd85d5e0,...) at mi_switch+0x243 turnstile_wait(c94fdb00,cd99bbc0,0,c0aed5ec,d1e57974,...) at turnstile_wait+0x504 _rw_wlock_hard(c1350a80,cd85d5e0,c104581a,d7e,0,...) at _rw_wlock_hard+0x365 _rw_wlock(c1350a80,c104581a,d7e,f29089b4,c0a8bbba,...) at _rw_wlock+0xc4 pmap_enter(cd819708,28438000,1,c2c40f78,3,...) at pmap_enter+0xac vm_fault_hold(cd819658,28438000,1,0,0,...) at vm_fault_hold+0x194c vm_fault(cd819658,28438000,1,0,28438000,...) at vm_fault+0x8b trap_pfault(c11afa70,0,c1046bd2,30a,cd856000,...) at trap_pfault+0x1c2 trap(f2908d08) at trap+0x3b6 calltrap() at calltrap+0x6 --- trap 0xc, eip = 0x28132ea3, esp = 0xbfbfcc60, ebp = 0xbfbfccb8 --- Tracing command sh pid 47825 tid 100283 td 0xcd99c5e0 sched_switch(cd99c5e0,0,103,1ae,c11bf6c0,...) at sched_switch+0x458 mi_switch(103,0,c0ff8ff6,2e8,cd99c5e0,...) at mi_switch+0x243 turnstile_wait(c94fdb00,cd99bbc0,0,c0aed5ec,c0aeb477,...) at turnstile_wait+0x504 _rw_wlock_hard(c1350a80,cd99c5e0,c104581a,d7e,0,...) at _rw_wlock_hard+0x365 _rw_wlock(c1350a80,c104581a,d7e,c7d82bf8,cd99c694,...) at _rw_wlock+0xc4 pmap_enter(c7de57f0,28849000,2,c3b71440,3,...) at pmap_enter+0xac vm_fault_hold(c7de5740,28849000,2,0,0,...) at vm_fault_hold+0x194c vm_fault(c7de5740,28849000,2,0,28849c8e,...) at vm_fault+0x8b trap_pfault(cd9918d0,f2a6dc50,c0ac6a0d,c11bff78,cd998b30,...) at trap_pfault+0x1c2 trap(f2a6dd08) at trap+0x3b6 calltrap() at calltrap+0x6 --- trap 0xc, eip = 0x804ebf2, esp = 0xbfbfd4c0, ebp = 0xbfbfd568 --- Tracing command ls pid 47824 tid 100251 td 0xcd8ee000 _mtx_lock_flags(0,0,c1002878,ab1,cb74d5a0,...) at _mtx_lock_flags+0x50 vgonel(cb74d664,0,c1002878,a7e,cb74d5a0,...) at vgonel+0xe0 vgone(cb74d5a0,0,cd84e555,b3,0,...) at vgone+0x3c null_destroy_proto(cb74d5a0,cd944900,c1002878,45f,cd85f2ec,...) at null_destroy_proto+0xb9 insmntque1(cb74d5a0,cd85f2ec,cd84ccd0,cd944900,f29f1924,...) at insmntque1+0x13f null_nodeget(cd85f2ec,cb6d39d8,f29f1954,8000,cb6d9000,...) at null_nodeget+0x158 null_lookup(f29f19ec,c10035cf,c0b424ef,cd84fa80,f29f1980,...) at null_lookup+0xc3 VOP_LOOKUP_APV(cd84fa80,f29f19ec,f29f1aec,21e,c0aec75f,...) at VOP_LOOKUP_APV+0x108 lookup(f29f1aac,c1001c48,10e,dc,cd92e664,...) at lookup+0x6eb namei(f29f1aac,f29f1b14,60,0,cd8ee000,...) at namei+0x6ff kern_statat_vnhook(cd8ee000,200,ffffff9c,2882ac98,0,...) at kern_statat_vnhook+0xbf kern_statat(cd8ee000,200,ffffff9c,2882ac98,0,...) at kern_statat+0x3c kern_lstat(cd8ee000,2882ac98,0,f29f1bdc,2912ff78,...) at kern_lstat+0x36 sys_lstat(cd8ee000,f29f1ccc,c1046c7c,c0ffb46f,f29f1cc4,...) at sys_lstat+0x2f syscall(f29f1d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (190, FreeBSD ELF32, sys_lstat), eip = 0x281c1c13, esp = 0xbfbfd59c, ebp = 0xbfbfd628 --- Tracing command sh pid 47823 tid 100092 td 0xca9812f0 sched_switch(ca9812f0,0,104,1b3,f2747ab4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf6c0,...) at mi_switch+0x243 sleepq_switch(ca9812f0,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(ca977b30,6c,c0ffb5a4,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca977b30,ca977bb8,16c,c0ffb5a4,0,...) at _sleep+0x384 kern_wait(ca9812f0,ffffffff,f2747c38,2,0,...) at kern_wait+0x597 sys_wait4(ca9812f0,f2747ccc,c1046c7c,c0ffb583,f2747cc4,...) at sys_wait4+0x3b syscall(f2747d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2815714b, esp = 0xbfbfdb0c, ebp = 0xbfbfdb28 --- Tracing command mount_nullfs pid 47822 tid 100139 td 0xca8f38d0 sched_switch(ca8f38d0,0,602,1ae,0,...) at sched_switch+0x458 mi_switch(602,0,c0ff319f,cd,ca8f38d0,...) at mi_switch+0x243 critical_exit(c1391460,0,0,c1350a80,c1350a80,...) at critical_exit+0xb8 lapic_handle_timer(f28538a8) at lapic_handle_timer+0x95 Xtimerint() at Xtimerint+0x20 --- interrupt, eip = 0xc0a9c4a5, esp = 0xf28538e8, ebp = 0xf2853948 --- _rw_wlock_hard(c1350a80,ca8f38d0,c104581a,d7e,0,...) at _rw_wlock_hard+0x1d5 _rw_wlock(c1350a80,c104581a,d7e,f28539b4,c0a8bbba,...) at _rw_wlock+0xc4 pmap_enter(ca9420b0,287bf000,1,c2a10d70,3,...) at pmap_enter+0xac vm_fault_hold(ca942000,287bf000,1,0,0,...) at vm_fault_hold+0x194c vm_fault(ca942000,287bf000,1,0,287bf000,...) at vm_fault+0x8b trap_pfault(c11afa70,0,c1046bd2,30a,cb529598,...) at trap_pfault+0x1c2 trap(f2853d08) at trap+0x3b6 calltrap() at calltrap+0x6 --- trap 0xc, eip = 0x280ecea3, esp = 0xbfbfcab0, ebp = 0xbfbfcb08 --- Tracing command mount_nullfs pid 47818 tid 100271 td 0xcd9915e0 sched_switch(cd9915e0,0,103,1ae,c94fdb00,...) at sched_switch+0x458 mi_switch(103,0,c0ff8ff6,2e8,c1330800,...) at mi_switch+0x243 turnstile_wait(cd85ba80,cd99a000,0,1c5,0,...) at turnstile_wait+0x504 _mtx_lock_sleep(c1330800,cd9915e0,0,c102e410,3ab,...) at _mtx_lock_sleep+0x2b7 _mtx_lock_flags(c1330800,0,c102e410,3ab,3,...) at _mtx_lock_flags+0x10f vm_fault_hold(cd8191d0,28800000,1,0,0,...) at vm_fault_hold+0x1d2d vm_fault(cd8191d0,28800000,1,0,28800000,...) at vm_fault+0x8b trap_pfault(cd98ab30,f2a49c44,c0ae4fd8,cd9915e0,cd98ab30,...) at trap_pfault+0x1c2 trap(f2a49d08) at trap+0x3b6 calltrap() at calltrap+0x6 --- trap 0xc, eip = 0x280ece96, esp = 0xbfbfce20, ebp = 0xbfbfce78 --- Tracing command grep pid 47816 tid 100295 td 0xcd99a000 sched_switch(cd99a000,0,103,1ae,c11bf6c0,...) at sched_switch+0x458 mi_switch(103,0,c0ff8ff6,2e8,cd99a000,...) at mi_switch+0x243 turnstile_wait(c94fdb00,cd99bbc0,0,c0aed5ec,c11be9c0,...) at turnstile_wait+0x504 _rw_wlock_hard(c1350a80,cd99a000,c104581a,10f1,0,...) at _rw_wlock_hard+0x365 _rw_wlock(c1350a80,c104581a,10f1,c27648a8,0,...) at _rw_wlock+0xc4 pmap_page_is_mapped(c27648a8,4,c103005a,7fa,c27648a8,...) at pmap_page_is_mapped+0x36 vm_page_free_toq(c27648a8,f2a91b30,c0d2b668,c27648a8,0,...) at vm_page_free_toq+0x48 vm_page_free(c27648a8,0,c102fa61,762,679,...) at vm_page_free+0x15 vm_object_page_remove(cd8738ac,225,0,a25,0,...) at vm_object_page_remove+0x238 vm_map_delete(cd818740,1000,bfc00000,cd8187f0,cd818740,...) at vm_map_delete+0x2ee vm_map_remove(cd818740,1000,bfc00000,cd99a000,cd98c598,...) at vm_map_remove+0x51 vmspace_exit(cd99a000,0,c0fec70b,139,0,...) at vmspace_exit+0xc3 exit1(cd99a000,0,f2a91cfc,c0e1c35e,cd99a000,...) at exit1+0x617 sys_sys_exit(cd99a000,f2a91ccc,c1046c7c,c0ffb4a3,f2a91cc4,...) at sys_sys_exit+0x1d syscall(f2a91d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (1, FreeBSD ELF32, sys_sys_exit), eip = 0x2813dccf, esp = 0xbfbfd6cc, ebp = 0xbfbfd6d8 --- Tracing command sleep pid 47813 tid 100292 td 0xcd99a8d0 sched_switch(cd99a8d0,0,104,1b3,f2a88aec,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,f2a88b10,...) at mi_switch+0x243 sleepq_switch(cd99a8d0,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(2711,c0ae16e0,cd99a8d0,2,100,...) at sleepq_catch_signals+0xfc sleepq_timedwait_sig(c11b1d64,6c,c0ff3f1d,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c11b1d64,0,16c,c0ff3f1d,2711,...) at _sleep+0x34e kern_nanosleep(cd99a8d0,f2a88c28,f2a88c30,a,0,...) at kern_nanosleep+0xc1 sys_nanosleep(cd99a8d0,f2a88ccc,c1046c7c,c0ffbdca,f2a88cc4,...) at sys_nanosleep+0x6f syscall(f2a88d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x28157e63, esp = 0xbfbfdc8c, ebp = 0xbfbfdcd8 --- Tracing command umount pid 47812 tid 100274 td 0xcd990bc0 sched_switch(cd990bc0,0,104,1b3,f2a52960,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c94fb340,...) at mi_switch+0x243 sleepq_switch(cd990bc0,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(cd722328,60,c0fe28cb,4,0,...) at sleepq_wait+0x6b __lockmgr_args(cd722328,80500,cd722394,0,0,...) at __lockmgr_args+0xc96 ffs_lock(f2a52abc,c0ff09b7,eb,0,cd84e555,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c115abc0,f2a52abc,cd84e768,246,80500,...) at VOP_LOCK1_APV+0xf3 null_lock(f2a52b18,f2a52af4,c0b2d4da,cd722328,f2a52b1c,...) at null_lock+0xcf VOP_LOCK1_APV(cd84fa80,f2a52b18,cb6d9000,c116cd20,cb6d9000,...) at VOP_LOCK1_APV+0xf3 _vn_lock(cb6d9000,80400,cd84e69c,10c,cb6be9d8,...) at _vn_lock+0x78 nullfs_root(cd85f2ec,80000,f2a52b98,522,c1baf710,...) at nullfs_root+0x55 dounmount(cd85f2ec,8080000,cd990bc0,4a3,f2a52c1c,...) at dounmount+0x3a1 sys_unmount(cd990bc0,f2a52ccc,c1046c7c,c0ffb41e,c0ac9607,...) at sys_unmount+0x421 syscall(f2a52d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280c487b, esp = 0xbfbfd0fc, ebp = 0xbfbfd1c8 --- Tracing command grep pid 47811 tid 100231 td 0xcd8228d0 cpustop_handler(1,f299a89c,c0e1c8d3,c0ff09b7,119,...) at cpustop_handler+0x26 ipi_nmi_handler(c0ff09b7,119,f299a864,c7b15c80,cd81f2cc,...) at ipi_nmi_handler+0x35 trap(f299a8a8) at trap+0x33 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0a9c4a5, esp = 0xf299a8e8, ebp = 0xf299a948 --- _rw_wlock_hard(c1350a80,cd8228d0,c104581a,d7e,0,...) at _rw_wlock_hard+0x1d5 _rw_wlock(c1350a80,c104581a,d7e,f299a9b4,c0a8bbba,...) at _rw_wlock+0xc4 pmap_enter(cd97ac78,28b66000,1,c337b468,3,...) at pmap_enter+0xac vm_fault_hold(cd97abc8,28b66000,1,0,0,...) at vm_fault_hold+0x194c vm_fault(cd97abc8,28b66000,1,0,28b66000,...) at vm_fault+0x8b trap_pfault(c11afa70,0,c1046bd2,30a,cd81f2cc,...) at trap_pfault+0x1c2 trap(f299ad08) at trap+0x3b6 calltrap() at calltrap+0x6 --- trap 0xc, eip = 0x28132ea3, esp = 0xbfbfd010, ebp = 0xbfbfd068 --- Tracing command grep pid 47810 tid 100183 td 0xcd7ed5e0 sched_switch(cd7ed5e0,0,103,1ae,c94fdb00,...) at sched_switch+0x458 mi_switch(103,0,c0ff8ff6,2e8,c1330800,...) at mi_switch+0x243 turnstile_wait(cd85ba80,cd99a000,0,1c5,0,...) at turnstile_wait+0x504 _mtx_lock_sleep(c1330800,cd7ed5e0,0,c102e410,3ab,...) at _mtx_lock_sleep+0x2b7 _mtx_lock_flags(c1330800,0,c102e410,3ab,3,...) at _mtx_lock_flags+0x10f vm_fault_hold(ca896cb0,28439000,1,0,0,...) at vm_fault_hold+0x1d2d vm_fault(ca896cb0,28439000,1,0,28439000,...) at vm_fault+0x8b trap_pfault(cd7eb8d0,f28f0c50,c0ac6a0d,c11bebf8,cd81c000,...) at trap_pfault+0x1c2 trap(f28f0d08) at trap+0x3b6 calltrap() at calltrap+0x6 --- trap 0xc, eip = 0x28132ea3, esp = 0xbfbfcc60, ebp = 0xbfbfccb8 --- Tracing command mount pid 47809 tid 100241 td 0xcd8f2000 sched_switch(cd8f2000,0,103,1ae,c11bf6c0,...) at sched_switch+0x458 mi_switch(103,0,c0ff8ff6,2e8,cd8f2000,...) at mi_switch+0x243 turnstile_wait(c94fdb00,cd99bbc0,0,c0aed5ec,c0aeb477,...) at turnstile_wait+0x504 _rw_wlock_hard(c1350a80,cd8f2000,c104581a,d7e,0,...) at _rw_wlock_hard+0x365 _rw_wlock(c1350a80,c104581a,d7e,f29d39b4,c0a8bbba,...) at _rw_wlock+0xc4 pmap_enter(cd840b90,281aa000,1,c3f595f8,3,...) at pmap_enter+0xac vm_fault_hold(cd840ae0,281aa000,1,0,0,...) at vm_fault_hold+0x194c vm_fault(cd840ae0,281aa000,1,0,281aa2a0,...) at vm_fault+0x8b trap_pfault(cd8228d0,f29d3c50,c0ac6a0d,c11bf278,cd8ec000,...) at trap_pfault+0x1c2 trap(f29d3d08) at trap+0x3b6 calltrap() at calltrap+0x6 --- trap 0xc, eip = 0x28059c22, esp = 0xbfbfd204, ebp = 0xbfbfd268 --- Tracing command grep pid 47786 tid 100101 td 0xca8a38d0 sched_switch(ca8a38d0,0,103,1ae,c11bf6c0,...) at sched_switch+0x458 mi_switch(103,0,c0ff8ff6,2e8,c1330c80,...) at mi_switch+0x243 turnstile_wait(cd85ae00,cd9918d0,0,1c5,c12f9298,...) at turnstile_wait+0x504 _mtx_lock_sleep(c1330c80,ca8a38d0,0,c102fa61,762,...) at _mtx_lock_sleep+0x2b7 _mtx_lock_flags(c1330c80,0,c102fa61,762,679,...) at _mtx_lock_flags+0x10f vm_object_page_remove(cd96f4a0,0,0,2a,0,...) at vm_object_page_remove+0x107 vm_map_delete(cac81e80,1000,bfc00000,cac81f30,cac81e80,...) at vm_map_delete+0x2ee vm_map_remove(cac81e80,1000,bfc00000,ca8a38d0,ca6d5864,...) at vm_map_remove+0x51 vmspace_exit(ca8a38d0,0,c0fec70b,139,0,...) at vmspace_exit+0xc3 exit1(ca8a38d0,100,f276fcfc,c0e1c35e,ca8a38d0,...) at exit1+0x617 sys_sys_exit(ca8a38d0,f276fccc,c1046c7c,c0ffb4a3,f276fcc4,...) at sys_sys_exit+0x1d syscall(f276fd08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (1, FreeBSD ELF32, sys_sys_exit), eip = 0x2813dccf, esp = 0xbfbfd75c, ebp = 0xbfbfd768 --- Tracing command mount_nullfs pid 47775 tid 100270 td 0xcd9918d0 sched_switch(cd9918d0,0,103,1ae,c11bf6c0,...) at sched_switch+0x458 mi_switch(103,0,c0ff8ff6,2e8,cd9918d0,...) at mi_switch+0x243 turnstile_wait(c94fdb00,cd99bbc0,0,c0aed5ec,0,...) at turnstile_wait+0x504 _rw_wlock_hard(c1350a80,cd9918d0,c104581a,10f1,0,...) at _rw_wlock_hard+0x365 _rw_wlock(c1350a80,c104581a,10f1,c3c41e20,0,...) at _rw_wlock+0xc4 pmap_page_is_mapped(c3c41e20,4,c103005a,7fa,c3c41e20,...) at pmap_page_is_mapped+0x36 vm_page_free_toq(c3c41e20,f2a46b30,c0d2b668,c3c41e20,0,...) at vm_page_free_toq+0x48 vm_page_free(c3c41e20,0,c102fa61,762,679,...) at vm_page_free+0x15 vm_object_page_remove(cd72a250,26b,0,a6b,0,...) at vm_object_page_remove+0x238 vm_map_delete(cd819488,1000,bfc00000,cd819538,cd819488,...) at vm_map_delete+0x2ee vm_map_remove(cd819488,1000,bfc00000,cd9918d0,cd98b000,...) at vm_map_remove+0x51 vmspace_exit(cd9918d0,0,c0fec70b,139,0,...) at vmspace_exit+0xc3 exit1(cd9918d0,0,f2a46cfc,c0e1c35e,cd9918d0,...) at exit1+0x617 sys_sys_exit(cd9918d0,f2a46ccc,c1046c7c,c0ffb4a3,f2a46cc4,...) at sys_sys_exit+0x1d syscall(f2a46d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (1, FreeBSD ELF32, sys_sys_exit), eip = 0x280f7ccf, esp = 0xbfbfd01c, ebp = 0xbfbfd028 --- Tracing command umount pid 47774 tid 100194 td 0xcd8592f0 cpustop_handler(2,f292a8e8,c0e1c8d3,f292a82c,c0e37d15,...) at cpustop_handler+0x26 ipi_nmi_handler(f292a82c,c0e37d15,c1391460,0,cd854b30,...) at ipi_nmi_handler+0x35 trap(f292a8f4) at trap+0x33 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0e13360, esp = 0xf292a934, ebp = 0xf292a950 --- free_pv_entry(c1350a80,4,c104581a,9b7) at free_pv_entry+0x1b0 pmap_pvh_free(c1350a80,4,c104581a,9b7,425,...) at pmap_pvh_free+0xcc pmap_enter(cac97d60,28bff000,1,c34d3f70,3,...) at pmap_enter+0x8ea vm_fault_hold(cac97cb0,28bff000,1,0,0,...) at vm_fault_hold+0x194c vm_fault(cac97cb0,28bff000,1,0,28bff000,...) at vm_fault+0x8b trap_pfault(c11afa70,0,c1046bd2,30a,cd854b30,...) at trap_pfault+0x1c2 trap(f292ad08) at trap+0x3b6 calltrap() at calltrap+0x6 --- trap 0xc, eip = 0x280edea3, esp = 0xbfbfcfa0, ebp = 0xbfbfcff8 --- Tracing command awk pid 2457 tid 100117 td 0xc9617bc0 sched_switch(c9617bc0,0,104,1b3,f2811a8c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bfd40,...) at mi_switch+0x243 sleepq_switch(c9617bc0,0,c0ff8600,1ac,5c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(ca92a990,5c,c0ffb0ae,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca92a990,ca92ab10,15c,c0ffb0ae,0,...) at _sleep+0x384 pipe_read(cd8428c0,f2811c1c,cbb4b800,0,c9617bc0,...) at pipe_read+0x427 dofileread(f2811c1c,ffffffff,ffffffff,0,f2811bf8,...) at dofileread+0x9e kern_readv(c9617bc0,0,f2811c1c,f2811c3c,1,...) at kern_readv+0x68 sys_read(c9617bc0,f2811ccc,c1046c7c,c0e4f55f,f2811cc4,...) at sys_read+0x4f syscall(f2811d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281b3a43, esp = 0xbfbfd96c, ebp = 0xbfbfd988 --- Tracing command sh pid 2456 tid 100193 td 0xcd8595e0 sched_switch(cd8595e0,0,104,1b3,f2926ab4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bfd40,...) at mi_switch+0x243 sleepq_switch(cd8595e0,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(cd855000,6c,c0ffb5a4,100,0,...) at sleepq_wait_sig+0x17 _sleep(cd855000,cd855088,16c,c0ffb5a4,0,...) at _sleep+0x384 kern_wait(cd8595e0,ffffffff,f2926c38,2,0,...) at kern_wait+0x597 sys_wait4(cd8595e0,f2926ccc,c1046c7c,c0ffb583,f2926cc4,...) at sys_wait4+0x3b syscall(f2926d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2815714b, esp = 0xbfbfd96c, ebp = 0xbfbfd988 --- Tracing command sh pid 2450 tid 100104 td 0xca97dbc0 sched_switch(ca97dbc0,0,104,1b3,f2780ab4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf6c0,...) at mi_switch+0x243 sleepq_switch(ca97dbc0,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(ca977598,6c,c0ffb5a4,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca977598,ca977620,16c,c0ffb5a4,0,...) at _sleep+0x384 kern_wait(ca97dbc0,ffffffff,f2780c38,2,0,...) at kern_wait+0x597 sys_wait4(ca97dbc0,f2780ccc,c1046c7c,c0ffb583,f2780cc4,...) at sys_wait4+0x3b syscall(f2780d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2815714b, esp = 0xbfbfdb4c, ebp = 0xbfbfdb68 --- Tracing command sshd pid 2449 tid 100186 td 0xcd85d2f0 sched_switch(cd85d2f0,0,104,1b3,f290c96c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf040,...) at mi_switch+0x243 sleepq_switch(cd85d2f0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0a8bc8a,cd8b22d0,0,c0ff09b7,cd85d2f0,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(cd8b22e4,0,f290ca30,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(cd8b22e4,cd8b22d0,c0ffadc5,6ae,cad75700,...) at _cv_wait_sig+0x263 seltdwait(cad75700,58,cb545c00,cd85d2f0,c0aed5ec,...) at seltdwait+0xa2 kern_select(cd85d2f0,c,28c0c170,28c0c1b0,0,0,20,cd855b30,0) at kern_select+0x667 sys_select(cd85d2f0,f290cccc,c1046c7c,c0ffb6f1,f290ccc4,...) at sys_select+0x66 syscall(f290cd08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284ed9c3, esp = 0xbfbfce5c, ebp = 0xbfbfcea8 --- Tracing command top pid 2448 tid 100215 td 0xcd8702f0 sched_switch(cd8702f0,0,104,1b3,f296a968,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,46,...) at mi_switch+0x243 sleepq_switch(cd8702f0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(f296a9fc,c0a8bc8a,cd85cad0,0,cd8702f0,...) at sleepq_catch_signals+0xfc sleepq_timedwait_sig(cd85cae4,0,f296aa30,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(cd85cae4,cd85cad0,3e9,6ae,cd842d58,...) at _cv_timedwait_sig+0x272 seltdwait(f296abe8,f296abf0,cbbe0800,cd8702f0,3548333b,...) at seltdwait+0x8a kern_select(cd8702f0,2,bfbfdbd4,0,0,f296ac34,20,1,0) at kern_select+0x667 sys_select(cd8702f0,f296accc,c1046c7c,c0fc5865,f296acc4,...) at sys_select+0x66 syscall(f296ad08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281ea9c3, esp = 0xbfbfdb6c, ebp = 0xbfbfdcd8 --- Tracing command sshd pid 2447 tid 100184 td 0xcd7ed2f0 sched_switch(cd7ed2f0,0,104,1b3,f290496c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bfd40,...) at mi_switch+0x243 sleepq_switch(cd7ed2f0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0a8bc8a,cd8b2290,0,c0ff09b7,cd7ed2f0,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(cd8b22a4,0,f2904a30,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(cd8b22a4,cd8b2290,c0ffadc5,6ae,cd842d20,...) at _cv_wait_sig+0x263 seltdwait(cd842d20,58,caf06780,cd7ed2f0,cd7ed3a4,...) at seltdwait+0xa2 kern_select(cd7ed2f0,a,28c0c170,28c0c1b0,0,0,20,cd8562cc,0) at kern_select+0x667 sys_select(cd7ed2f0,f2904ccc,c1046c7c,c0ffb6f1,f2904cc4,...) at sys_select+0x66 syscall(f2904d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284ed9c3, esp = 0xbfbfce5c, ebp = 0xbfbfcea8 --- Tracing command tail pid 2440 tid 100100 td 0xca8a3bc0 sched_switch(ca8a3bc0,0,104,1b3,f276b9d0,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,f276b9f4,...) at mi_switch+0x243 sleepq_switch(ca8a3bc0,0,c0ff8600,1ac,68,...) at sleepq_switch+0x1a7 sleepq_catch_signals(3e9,c0ae16e0,ca8a3bc0,1,100,...) at sleepq_catch_signals+0xfc sleepq_timedwait_sig(cd7ef680,68,c0fec52c,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(cd7ef680,cd7ef680,168,c0fec52c,3e9,...) at _sleep+0x34e kern_kevent(ca8a3bc0,4,0,1,f276bc1c,...) at kern_kevent+0x374 sys_kevent(ca8a3bc0,f276bccc,c1046c7c,c0fc5865,c0ac9607,...) at sys_kevent+0x19e syscall(f276bd08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (363, FreeBSD ELF32, sys_kevent), eip = 0x2815bddb, esp = 0xbfbfdb4c, ebp = 0xbfbfdc08 --- Tracing command sshd pid 2439 tid 100083 td 0xca6e92f0 sched_switch(ca6e92f0,0,104,1b3,f271e96c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11be9c0,...) at mi_switch+0x243 sleepq_switch(ca6e92f0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0a8bc8a,cb538650,0,c0ff09b7,ca6e92f0,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(cb538664,0,f271ea30,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(cb538664,cb538650,c0ffadc5,6ae,cad75268,...) at _cv_wait_sig+0x263 seltdwait(cad75268,58,cbbd5800,ca6e92f0,ca6e93a4,...) at seltdwait+0xa2 kern_select(ca6e92f0,c,28c0c170,28c0c1b0,0,0,20,c9427864,0) at kern_select+0x667 sys_select(ca6e92f0,f271eccc,c1046c7c,c0ffb6f1,f271ecc4,...) at sys_select+0x66 syscall(f271ed08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284ed9c3, esp = 0xbfbfce5c, ebp = 0xbfbfcea8 --- Tracing command sshd pid 2408 tid 100095 td 0xca8a4000 sched_switch(ca8a4000,0,104,1b3,f27549d0,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf6c0,...) at mi_switch+0x243 sleepq_switch(ca8a4000,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0a8bc8a,ca799090,0,c0ff09b7,ca8a4000,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(ca7990a4,0,f2754a94,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca7990a4,ca799090,c0ffadc5,6ae,f2754b1c,...) at _cv_wait_sig+0x263 seltdwait(cad5c82c,c0ffadc5,5ab,ca8a4000,0,...) at seltdwait+0xa2 sys_poll(ca8a4000,f2754ccc,c1046c7c,c0e4f55f,c0ac9607,...) at sys_poll+0x321 syscall(f2754d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x28484dcb, esp = 0xbfbfcebc, ebp = 0xbfbfcef8 --- Tracing command sshd pid 2397 tid 100195 td 0xcd859000 sched_switch(cd859000,0,104,1b3,f292e9d0,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf040,...) at mi_switch+0x243 sleepq_switch(cd859000,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0a8bc8a,cd85cb10,0,c0ff09b7,cd859000,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(cd85cb24,0,f292ea94,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(cd85cb24,cd85cb10,c0ffadc5,6ae,f292eb1c,...) at _cv_wait_sig+0x263 seltdwait(c961982c,c0ffadc5,5ab,cd859000,0,...) at seltdwait+0xa2 sys_poll(cd859000,f292eccc,c1046c7c,c0e4f55f,f292ecc4,...) at sys_poll+0x321 syscall(f292ed08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x28484dcb, esp = 0xbfbfcebc, ebp = 0xbfbfcef8 --- Tracing command sshd pid 2394 tid 100218 td 0xcd85d8d0 sched_switch(cd85d8d0,0,104,1b3,f29739d0,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf6c0,...) at mi_switch+0x243 sleepq_switch(cd85d8d0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0a8bc8a,cd8b2310,0,c0ff09b7,cd85d8d0,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(cd8b2324,0,f2973a94,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(cd8b2324,cd8b2310,c0ffadc5,6ae,f2973b1c,...) at _cv_wait_sig+0x263 seltdwait(ca8a5b2c,c0ffadc5,5ab,cd85d8d0,0,...) at seltdwait+0xa2 sys_poll(cd85d8d0,f2973ccc,c1046c7c,c0e4f55f,c0ac9607,...) at sys_poll+0x321 syscall(f2973d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x28484dcb, esp = 0xbfbfcebc, ebp = 0xbfbfcef8 --- Tracing command sh pid 1604 tid 100177 td 0xcd8208d0 sched_switch(cd8208d0,0,104,1b3,f28deab4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bfd40,...) at mi_switch+0x243 sleepq_switch(cd8208d0,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(cd81d2cc,6c,c0ffb5a4,100,0,...) at sleepq_wait_sig+0x17 _sleep(cd81d2cc,cd81d354,16c,c0ffb5a4,0,...) at _sleep+0x384 kern_wait(cd8208d0,ffffffff,f28dec38,2,0,...) at kern_wait+0x597 sys_wait4(cd8208d0,f28deccc,c1046c7c,c0ffb583,f28decc4,...) at sys_wait4+0x3b syscall(f28ded08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2815714b, esp = 0xbfbfd64c, ebp = 0xbfbfd668 --- Tracing command sh pid 1603 tid 100176 td 0xcd820bc0 sched_switch(cd820bc0,0,104,1b3,f28dbab4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf6c0,...) at mi_switch+0x243 sleepq_switch(cd820bc0,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(cd81d598,6c,c0ffb5a4,100,0,...) at sleepq_wait_sig+0x17 _sleep(cd81d598,cd81d620,16c,c0ffb5a4,0,...) at _sleep+0x384 kern_wait(cd820bc0,ffffffff,f28dbc38,2,0,...) at kern_wait+0x597 sys_wait4(cd820bc0,f28dbccc,c1046c7c,c0ffb583,f28dbc74,...) at sys_wait4+0x3b syscall(f28dbd08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2815714b, esp = 0xbfbfd64c, ebp = 0xbfbfd668 --- Tracing command sh pid 1602 tid 100175 td 0xcd821000 sched_switch(cd821000,0,103,1ae,c11bf040,...) at sched_switch+0x458 mi_switch(103,0,c0ff8ff6,2e8,cd821000,...) at mi_switch+0x243 turnstile_wait(cd7f0980,cd8228d0,0,c0aed5ec,c0aeb477,...) at turnstile_wait+0x504 _rw_wlock_hard(c1350a80,cd821000,c104581a,d7e,0,...) at _rw_wlock_hard+0x365 _rw_wlock(c1350a80,c104581a,d7e,c1331700,78f,...) at _rw_wlock+0xc4 pmap_enter(c7de4d60,8067000,2,c46f0488,3,...) at pmap_enter+0xac vm_fault_hold(c7de4cb0,8067000,2,0,0,...) at vm_fault_hold+0x194c vm_fault(c7de4cb0,8067000,2,0,8067280,...) at vm_fault+0x8b trap_pfault(cd81d864,f28d8c44,c0ae4fd8,cd821000,cd81d864,...) at trap_pfault+0x1c2 trap(f28d8d08) at trap+0x3b6 calltrap() at calltrap+0x6 --- trap 0xc, eip = 0x805714f, esp = 0xbfbfd5d0, ebp = 0xbfbfd5d8 --- Tracing command sh pid 1601 tid 100174 td 0xcd8212f0 sched_switch(cd8212f0,0,104,1b3,f28d5ab4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf6c0,...) at mi_switch+0x243 sleepq_switch(cd8212f0,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(cd81db30,6c,c0ffb5a4,100,0,...) at sleepq_wait_sig+0x17 _sleep(cd81db30,cd81dbb8,16c,c0ffb5a4,0,...) at _sleep+0x384 kern_wait(cd8212f0,ffffffff,f28d5c38,2,0,...) at kern_wait+0x597 sys_wait4(cd8212f0,f28d5ccc,c1046c7c,c0ffb583,c0ac9607,...) at sys_wait4+0x3b syscall(f28d5d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2815714b, esp = 0xbfbfd54c, ebp = 0xbfbfd568 --- Tracing command sh pid 1600 tid 100173 td 0xcd8215e0 sched_switch(cd8215e0,0,104,1b3,f28d2ab4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf040,...) at mi_switch+0x243 sleepq_switch(cd8215e0,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(cd81e000,6c,c0ffb5a4,100,0,...) at sleepq_wait_sig+0x17 _sleep(cd81e000,cd81e088,16c,c0ffb5a4,0,...) at _sleep+0x384 kern_wait(cd8215e0,ffffffff,f28d2c38,2,0,...) at kern_wait+0x597 sys_wait4(cd8215e0,f28d2ccc,c1046c7c,c0ffb583,f28d2cc4,...) at sys_wait4+0x3b syscall(f28d2d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2815714b, esp = 0xbfbfd54c, ebp = 0xbfbfd568 --- Tracing command sh pid 1599 tid 100172 td 0xcd8218d0 sched_switch(cd8218d0,0,104,1b3,f28cfab4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bfd40,...) at mi_switch+0x243 sleepq_switch(cd8218d0,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(cd81e2cc,6c,c0ffb5a4,100,0,...) at sleepq_wait_sig+0x17 _sleep(cd81e2cc,cd81e354,16c,c0ffb5a4,0,...) at _sleep+0x384 kern_wait(cd8218d0,ffffffff,f28cfc38,2,0,...) at kern_wait+0x597 sys_wait4(cd8218d0,f28cfccc,c1046c7c,c0ffb583,c0ac9607,...) at sys_wait4+0x3b syscall(f28cfd08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2815714b, esp = 0xbfbfd54c, ebp = 0xbfbfd568 --- Tracing command sh pid 1598 tid 100171 td 0xcd821bc0 sched_switch(cd821bc0,0,104,1b3,f28ccab4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf6c0,...) at mi_switch+0x243 sleepq_switch(cd821bc0,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(cd81e598,6c,c0ffb5a4,100,0,...) at sleepq_wait_sig+0x17 _sleep(cd81e598,cd81e620,16c,c0ffb5a4,0,...) at _sleep+0x384 kern_wait(cd821bc0,ffffffff,f28ccc38,2,0,...) at kern_wait+0x597 sys_wait4(cd821bc0,f28ccccc,c1046c7c,c0ffb583,f28cccc4,...) at sys_wait4+0x3b syscall(f28ccd08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2815714b, esp = 0xbfbfd54c, ebp = 0xbfbfd568 --- Tracing command sh pid 1597 tid 100170 td 0xcd822000 sched_switch(cd822000,0,104,1b3,f28c9ab4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11be9c0,...) at mi_switch+0x243 sleepq_switch(cd822000,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(cd81e864,6c,c0ffb5a4,100,0,...) at sleepq_wait_sig+0x17 _sleep(cd81e864,cd81e8ec,16c,c0ffb5a4,0,...) at _sleep+0x384 kern_wait(cd822000,ffffffff,f28c9c38,2,0,...) at kern_wait+0x597 sys_wait4(cd822000,f28c9ccc,c1046c7c,c0ffb3f2,c1825cb4,...) at sys_wait4+0x3b syscall(f28c9d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2815714b, esp = 0xbfbfd64c, ebp = 0xbfbfd668 --- Tracing command sh pid 1596 tid 100169 td 0xcd8222f0 sched_switch(cd8222f0,0,104,1b3,f28c6ab4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf6c0,...) at mi_switch+0x243 sleepq_switch(cd8222f0,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(cd81eb30,6c,c0ffb5a4,100,0,...) at sleepq_wait_sig+0x17 _sleep(cd81eb30,cd81ebb8,16c,c0ffb5a4,0,...) at _sleep+0x384 kern_wait(cd8222f0,ffffffff,f28c6c38,2,0,...) at kern_wait+0x597 sys_wait4(cd8222f0,f28c6ccc,c1046c7c,c0ffb3f2,c0ac9607,...) at sys_wait4+0x3b syscall(f28c6d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2815714b, esp = 0xbfbfd64c, ebp = 0xbfbfd668 --- Tracing command sh pid 1595 tid 100168 td 0xca9828d0 sched_switch(ca9828d0,0,104,1b3,f28c2ab4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bfd40,...) at mi_switch+0x243 sleepq_switch(ca9828d0,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(cd81f000,6c,c0ffb5a4,100,0,...) at sleepq_wait_sig+0x17 _sleep(cd81f000,cd81f088,16c,c0ffb5a4,0,...) at _sleep+0x384 kern_wait(ca9828d0,ffffffff,f28c2c38,2,0,...) at kern_wait+0x597 sys_wait4(ca9828d0,f28c2ccc,c1046c7c,c0ffb583,f28c2cc4,...) at sys_wait4+0x3b syscall(f28c2d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2815714b, esp = 0xbfbfd54c, ebp = 0xbfbfd568 --- Tracing command sh pid 1594 tid 100159 td 0xcd7ec5e0 sched_switch(cd7ec5e0,0,104,1b3,f288fab4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bfd40,...) at mi_switch+0x243 sleepq_switch(cd7ec5e0,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(cd7e8864,6c,c0ffb5a4,100,0,...) at sleepq_wait_sig+0x17 _sleep(cd7e8864,cd7e88ec,16c,c0ffb5a4,0,...) at _sleep+0x384 kern_wait(cd7ec5e0,ffffffff,f288fc38,2,0,...) at kern_wait+0x597 sys_wait4(cd7ec5e0,f288fccc,c1046c7c,c0ffb583,c0ff4bec,...) at sys_wait4+0x3b syscall(f288fd08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2815714b, esp = 0xbfbfd64c, ebp = 0xbfbfd668 --- Tracing command sh pid 1593 tid 100131 td 0xcb3732f0 sched_switch(cb3732f0,0,103,1ae,c94fdb00,...) at sched_switch+0x458 mi_switch(103,0,c0ff8ff6,2e8,c1330800,...) at mi_switch+0x243 turnstile_wait(cd85ba80,cd99a000,0,1c5,0,...) at turnstile_wait+0x504 _mtx_lock_sleep(c1330800,cb3732f0,0,c102e410,3ab,...) at _mtx_lock_sleep+0x2b7 _mtx_lock_flags(c1330800,0,c102e410,3ab,3,...) at _mtx_lock_flags+0x10f vm_fault_hold(cad61bc8,28876000,2,0,0,...) at vm_fault_hold+0x1d2d vm_fault(cad61bc8,28876000,2,0,288760d0,...) at vm_fault+0x8b trap_pfault(c11afa70,0,c1046bd2,30a,cb51a000,...) at trap_pfault+0x1c2 trap(f283bd08) at trap+0x3b6 calltrap() at calltrap+0x6 --- trap 0xc, eip = 0x8054cbc, esp = 0xbfbfd3c0, ebp = 0xbfbfd3e8 --- Tracing command sh pid 1592 tid 100106 td 0xca8f68d0 sched_switch(ca8f68d0,0,104,1b3,f278cab4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bfd40,...) at mi_switch+0x243 sleepq_switch(ca8f68d0,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(ca8ef598,6c,c0ffb5a4,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca8ef598,ca8ef620,16c,c0ffb5a4,0,...) at _sleep+0x384 kern_wait(ca8f68d0,ffffffff,f278cc38,2,0,...) at kern_wait+0x597 sys_wait4(ca8f68d0,f278cccc,c1046c7c,c0ffb583,c0ac9607,...) at sys_wait4+0x3b syscall(f278cd08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2815714b, esp = 0xbfbfd54c, ebp = 0xbfbfd568 --- Tracing command sh pid 1591 tid 100164 td 0xcd7eb5e0 sched_switch(cd7eb5e0,0,104,1b3,f289eab4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11be9c0,...) at mi_switch+0x243 sleepq_switch(cd7eb5e0,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(cd7e7864,6c,c0ffb5a4,100,0,...) at sleepq_wait_sig+0x17 _sleep(cd7e7864,cd7e78ec,16c,c0ffb5a4,0,...) at _sleep+0x384 kern_wait(cd7eb5e0,ffffffff,f289ec38,2,0,...) at kern_wait+0x597 sys_wait4(cd7eb5e0,f289eccc,c1046c7c,c0ffb583,c0ff4bec,...) at sys_wait4+0x3b syscall(f289ed08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2815714b, esp = 0xbfbfd54c, ebp = 0xbfbfd568 --- Tracing command sh pid 1590 tid 100144 td 0xcb53abc0 sched_switch(cb53abc0,0,104,1b3,f2862ab4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf040,...) at mi_switch+0x243 sleepq_switch(cb53abc0,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(cb528864,6c,c0ffb5a4,100,0,...) at sleepq_wait_sig+0x17 _sleep(cb528864,cb5288ec,16c,c0ffb5a4,0,...) at _sleep+0x384 kern_wait(cb53abc0,ffffffff,f2862c38,2,0,...) at kern_wait+0x597 sys_wait4(cb53abc0,f2862ccc,c1046c7c,c0ffb583,f2862cc4,...) at sys_wait4+0x3b syscall(f2862d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2815714b, esp = 0xbfbfd54c, ebp = 0xbfbfd568 --- Tracing command sh pid 1536 tid 100165 td 0xcd7eb2f0 sched_switch(cd7eb2f0,0,104,1b3,f28a1ab4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11be9c0,...) at mi_switch+0x243 sleepq_switch(cd7eb2f0,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(cd7e7598,6c,c0ffb5a4,100,0,...) at sleepq_wait_sig+0x17 _sleep(cd7e7598,cd7e7620,16c,c0ffb5a4,0,...) at _sleep+0x384 kern_wait(cd7eb2f0,ffffffff,f28a1c38,2,0,...) at kern_wait+0x597 sys_wait4(cd7eb2f0,f28a1ccc,270ba7ae,0,f28a1cc4,...) at sys_wait4+0x3b syscall(f28a1d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2815714b, esp = 0xbfbfd74c, ebp = 0xbfbfd768 --- Tracing command sh pid 1421 tid 100152 td 0xcb5395e0 sched_switch(cb5395e0,0,104,1b3,f287aab4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11be9c0,...) at mi_switch+0x243 sleepq_switch(cb5395e0,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(cb51b2cc,6c,c0ffb5a4,100,0,...) at sleepq_wait_sig+0x17 _sleep(cb51b2cc,cb51b354,16c,c0ffb5a4,0,...) at _sleep+0x384 kern_wait(cb5395e0,ffffffff,f287ac38,2,0,...) at kern_wait+0x597 sys_wait4(cb5395e0,f287accc,c1046c7c,c0ffb583,f287acc4,...) at sys_wait4+0x3b syscall(f287ad08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2815714b, esp = 0xbfbfd6cc, ebp = 0xbfbfd6e8 --- Tracing command getty pid 1418 tid 100153 td 0xca89f5e0 sched_switch(ca89f5e0,0,104,1b3,f287da08,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf040,...) at mi_switch+0x243 sleepq_switch(ca89f5e0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,ca89f5e0,ca89f5e0,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(c7f00870,0,c0ffd443,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7f00870,c11afa70,c0ffce14,538,0,...) at _cv_wait_sig+0x263 tty_wait(c7f00800,c7f00870,f287dc1c,1,0,...) at tty_wait+0x71 ttydisc_read(c7f00800,f287dc1c,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c9498a00,f287dc1c,0,1,0,...) at ttydev_read+0xaa devfs_read_f(ca928b28,f287dc1c,c7dbde80,0,ca89f5e0,...) at devfs_read_f+0x7d dofileread(f287dc1c,ffffffff,ffffffff,0,f287dbf8,...) at dofileread+0x9e kern_readv(ca89f5e0,0,f287dc1c,f287dc3c,1,...) at kern_readv+0x68 sys_read(ca89f5e0,f287dccc,c1046c7c,c0e5447e,f287dcc4,...) at sys_read+0x4f syscall(f287dd08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2818fa43, esp = 0xbfbfddac, ebp = 0xbfbfdde8 --- Tracing command getty pid 1417 tid 100087 td 0xca981bc0 sched_switch(ca981bc0,0,104,1b3,f2731a08,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11be9c0,...) at mi_switch+0x243 sleepq_switch(ca981bc0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,ca981bc0,ca981bc0,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(c7f00a70,0,c0ffd443,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7f00a70,c11afa70,c0ffce14,538,0,...) at _cv_wait_sig+0x263 tty_wait(c7f00a00,c7f00a70,f2731c1c,1,0,...) at tty_wait+0x71 ttydisc_read(c7f00a00,f2731c1c,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c9498b00,f2731c1c,0,1,0,...) at ttydev_read+0xaa devfs_read_f(ca8d2c40,f2731c1c,c7dbde80,0,ca981bc0,...) at devfs_read_f+0x7d dofileread(f2731c1c,ffffffff,ffffffff,0,f2731bf8,...) at dofileread+0x9e kern_readv(ca981bc0,0,f2731c1c,f2731c3c,1,...) at kern_readv+0x68 sys_read(ca981bc0,f2731ccc,c1046c7c,c0e5447e,f2731cc4,...) at sys_read+0x4f syscall(f2731d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2818fa43, esp = 0xbfbfddac, ebp = 0xbfbfdde8 --- Tracing command getty pid 1416 tid 100156 td 0xcd7ed000 sched_switch(cd7ed000,0,104,1b3,f2886a08,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf6c0,...) at mi_switch+0x243 sleepq_switch(cd7ed000,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,cd7ed000,cd7ed000,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(c7f00c70,0,c0ffd443,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7f00c70,c11afa70,c0ffce14,538,0,...) at _cv_wait_sig+0x263 tty_wait(c7f00c00,c7f00c70,f2886c1c,1,0,...) at tty_wait+0x71 ttydisc_read(c7f00c00,f2886c1c,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c9498c00,f2886c1c,0,1,0,...) at ttydev_read+0xaa devfs_read_f(ca928a10,f2886c1c,c7dbde80,0,cd7ed000,...) at devfs_read_f+0x7d dofileread(f2886c1c,ffffffff,ffffffff,0,f2886bf8,...) at dofileread+0x9e kern_readv(cd7ed000,0,f2886c1c,f2886c3c,1,...) at kern_readv+0x68 sys_read(cd7ed000,f2886ccc,c1046c7c,c0e5447e,f2886cc4,...) at sys_read+0x4f syscall(f2886d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2818fa43, esp = 0xbfbfddac, ebp = 0xbfbfdde8 --- Tracing command getty pid 1415 tid 100146 td 0xcb53a8d0 sched_switch(cb53a8d0,0,104,1b3,f2868a08,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf6c0,...) at mi_switch+0x243 sleepq_switch(cb53a8d0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,cb53a8d0,cb53a8d0,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(c7f00e70,0,c0ffd443,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7f00e70,c11afa70,c0ffce14,538,0,...) at _cv_wait_sig+0x263 tty_wait(c7f00e00,c7f00e70,f2868c1c,1,0,...) at tty_wait+0x71 ttydisc_read(c7f00e00,f2868c1c,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c9498d00,f2868c1c,0,1,0,...) at ttydev_read+0xaa devfs_read_f(ca8d2498,f2868c1c,c7dbde80,0,cb53a8d0,...) at devfs_read_f+0x7d dofileread(f2868c1c,ffffffff,ffffffff,0,f2868bf8,...) at dofileread+0x9e kern_readv(cb53a8d0,0,f2868c1c,f2868c3c,1,...) at kern_readv+0x68 sys_read(cb53a8d0,f2868ccc,c1046c7c,c0e5447e,f2868cc4,...) at sys_read+0x4f syscall(f2868d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2818fa43, esp = 0xbfbfddac, ebp = 0xbfbfdde8 --- Tracing command getty pid 1414 tid 100134 td 0xca8f6000 sched_switch(ca8f6000,0,104,1b3,f2844a08,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11be9c0,...) at mi_switch+0x243 sleepq_switch(ca8f6000,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,ca8f6000,ca8f6000,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(c9626070,0,c0ffd443,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c9626070,c11afa70,c0ffce14,538,0,...) at _cv_wait_sig+0x263 tty_wait(c9626000,c9626070,f2844c1c,1,0,...) at tty_wait+0x71 ttydisc_read(c9626000,f2844c1c,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c9498e00,f2844c1c,0,1,0,...) at ttydev_read+0xaa devfs_read_f(ca8d2b98,f2844c1c,c7dbde80,0,ca8f6000,...) at devfs_read_f+0x7d dofileread(f2844c1c,ffffffff,ffffffff,0,f2844bf8,...) at dofileread+0x9e kern_readv(ca8f6000,0,f2844c1c,f2844c3c,1,...) at kern_readv+0x68 sys_read(ca8f6000,f2844ccc,c1046c7c,c0e5447e,f2844cc4,...) at sys_read+0x4f syscall(f2844d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2818fa43, esp = 0xbfbfddac, ebp = 0xbfbfdde8 --- Tracing command getty pid 1413 tid 100079 td 0xca8a4bc0 sched_switch(ca8a4bc0,0,104,1b3,f270ea08,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bfd40,...) at mi_switch+0x243 sleepq_switch(ca8a4bc0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,ca8a4bc0,ca8a4bc0,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(c9626270,0,c0ffd443,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c9626270,c11afa70,c0ffce14,538,0,...) at _cv_wait_sig+0x263 tty_wait(c9626200,c9626270,f270ec1c,1,0,...) at tty_wait+0x71 ttydisc_read(c9626200,f270ec1c,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c961f000,f270ec1c,0,1,0,...) at ttydev_read+0xaa devfs_read_f(ca8d26c8,f270ec1c,c7dbde80,0,ca8a4bc0,...) at devfs_read_f+0x7d dofileread(f270ec1c,ffffffff,ffffffff,0,f270ebf8,...) at dofileread+0x9e kern_readv(ca8a4bc0,0,f270ec1c,f270ec3c,1,...) at kern_readv+0x68 sys_read(ca8a4bc0,f270eccc,c1046c7c,c0e5447e,f270ecc4,...) at sys_read+0x4f syscall(f270ed08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2818fa43, esp = 0xbfbfddac, ebp = 0xbfbfdde8 --- Tracing command getty pid 1412 tid 100129 td 0xcb3738d0 sched_switch(cb3738d0,0,104,1b3,f2835a08,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf6c0,...) at mi_switch+0x243 sleepq_switch(cb3738d0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,cb3738d0,cb3738d0,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(c9626470,0,c0ffd443,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c9626470,c11afa70,c0ffce14,538,0,...) at _cv_wait_sig+0x263 tty_wait(c9626400,c9626470,f2835c1c,1,0,...) at tty_wait+0x71 ttydisc_read(c9626400,f2835c1c,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c961f100,f2835c1c,0,1,0,...) at ttydev_read+0xaa devfs_read_f(ca8d15e8,f2835c1c,c7dbde80,0,cb3738d0,...) at devfs_read_f+0x7d dofileread(f2835c1c,ffffffff,ffffffff,0,f2835bf8,...) at dofileread+0x9e kern_readv(cb3738d0,0,f2835c1c,f2835c3c,1,...) at kern_readv+0x68 sys_read(cb3738d0,f2835ccc,c1046c7c,c0e5447e,f2835cc4,...) at sys_read+0x4f syscall(f2835d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2818fa43, esp = 0xbfbfddac, ebp = 0xbfbfdde8 --- Tracing command getty pid 1411 tid 100142 td 0xca8f32f0 sched_switch(ca8f32f0,0,104,1b3,f285ca08,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11be9c0,...) at mi_switch+0x243 sleepq_switch(ca8f32f0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,ca8f32f0,ca8f32f0,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(c7eff070,0,c0ffd443,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7eff070,c11afa70,c0ffce14,538,0,...) at _cv_wait_sig+0x263 tty_wait(c7eff000,c7eff070,f285cc1c,1,0,...) at tty_wait+0x71 ttydisc_read(c7eff000,f285cc1c,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c961f200,f285cc1c,0,1,0,...) at ttydev_read+0xaa devfs_read_f(ca8d1d58,f285cc1c,c7dbde80,0,ca8f32f0,...) at devfs_read_f+0x7d dofileread(f285cc1c,ffffffff,ffffffff,0,f285cbf8,...) at dofileread+0x9e kern_readv(ca8f32f0,0,f285cc1c,f285cc3c,1,...) at kern_readv+0x68 sys_read(ca8f32f0,f285cccc,c1046c7c,c0e5447e,f285ccc4,...) at sys_read+0x4f syscall(f285cd08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2818fa43, esp = 0xbfbfddac, ebp = 0xbfbfdde8 --- Tracing command getty pid 1410 tid 100084 td 0xca9825e0 sched_switch(ca9825e0,0,104,1b3,f2724a08,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf040,...) at mi_switch+0x243 sleepq_switch(ca9825e0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0a8bc8a,c7eff204,0,c0ff09b7,ca9825e0,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(c7eff270,0,f2724acc,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7eff270,c7eff204,c0ffce14,538,0,...) at _cv_wait_sig+0x263 tty_wait(c7eff200,c7eff270,f2724c1c,1,0,...) at tty_wait+0x71 ttydisc_read(c7eff200,f2724c1c,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c7ee7c00,f2724c1c,0,1,0,...) at ttydev_read+0xaa devfs_read_f(ca974428,f2724c1c,c7dbde80,0,ca9825e0,...) at devfs_read_f+0x7d dofileread(f2724c1c,ffffffff,ffffffff,0,f2724bf8,...) at dofileread+0x9e kern_readv(ca9825e0,0,f2724c1c,f2724c3c,1,...) at kern_readv+0x68 sys_read(ca9825e0,f2724ccc,c1046c7c,c0e5447e,f2724cc4,...) at sys_read+0x4f syscall(f2724d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2818fa43, esp = 0xbfbfddac, ebp = 0xbfbfdde8 --- Tracing command inetd pid 1393 tid 100124 td 0xcb3748d0 sched_switch(cb3748d0,0,104,1b3,f282696c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11be9c0,...) at mi_switch+0x243 sleepq_switch(cb3748d0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0a8bc8a,c94e7410,0,c0ff09b7,cb3748d0,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(c94e7424,0,f2826a30,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c94e7424,c94e7410,c0ffadc5,6ae,ca8d14d0,...) at _cv_wait_sig+0x263 seltdwait(ca8d14d0,58,c7dbde80,cb3748d0,202,...) at seltdwait+0xa2 kern_select(cb3748d0,8,bfbfcfd0,0,0,0,20,ca97c598,4) at kern_select+0x667 sys_select(cb3748d0,f2826ccc,c1046c7c,c0ffb5f0,f2826cc4,...) at sys_select+0x66 syscall(f2826d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281a29c3, esp = 0xbfbfcf8c, ebp = 0xbfbfde18 --- Tracing command watchdogd pid 1368 tid 100143 td 0xca8f3000 sched_switch(ca8f3000,0,104,1b3,f285fab4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf6c0,...) at mi_switch+0x243 sleepq_switch(ca8f3000,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(cb528b30,6c,c0ffb5a4,100,0,...) at sleepq_wait_sig+0x17 _sleep(cb528b30,cb528bb8,16c,c0ffb5a4,0,...) at _sleep+0x384 kern_wait(ca8f3000,bacf,f285fc38,0,0,...) at kern_wait+0x597 sys_wait4(ca8f3000,f285fccc,c1046c7c,c0feca1c,d,...) at sys_wait4+0x3b syscall(f285fd08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2810a14b, esp = 0xbfbfdc6c, ebp = 0xbfbfdd18 --- Tracing command bash pid 1344 tid 100141 td 0xca8f35e0 sched_switch(ca8f35e0,0,104,1b3,f2859a08,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bfd40,...) at mi_switch+0x243 sleepq_switch(ca8f35e0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0a8bc8a,ca626e04,0,c0ff09b7,ca8f35e0,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(ca626e70,0,f2859acc,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca626e70,ca626e04,c0ffce14,538,0,...) at _cv_wait_sig+0x263 tty_wait(ca626e00,ca626e70,f2859c1c,1,0,...) at tty_wait+0x71 ttydisc_read(ca626e00,f2859c1c,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(cb947500,f2859c1c,0,1,0,...) at ttydev_read+0xaa devfs_read_f(ca8d1ce8,f2859c1c,cb93c980,0,ca8f35e0,...) at devfs_read_f+0x7d dofileread(f2859c1c,ffffffff,ffffffff,0,f2859bf8,...) at dofileread+0x9e kern_readv(ca8f35e0,0,f2859c1c,f2859c3c,1,...) at kern_readv+0x68 sys_read(ca8f35e0,f2859ccc,c1046c7c,c0ffb4a3,f2859c80,...) at sys_read+0x4f syscall(f2859d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x28357a43, esp = 0xbfbfcdfc, ebp = 0xbfbfce28 --- Tracing command csh pid 1341 tid 100086 td 0xca982000 sched_switch(ca982000,0,104,1b3,f272db00,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bfd40,...) at mi_switch+0x243 sleepq_switch(ca982000,0,c0ff8600,1ac,74,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(ca97b8bc,74,c0e7704e,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca97b8bc,ca97b8ec,174,c0e7704e,0,...) at _sleep+0x384 kern_sigsuspend(ca982000,0,0,0,0,...) at kern_sigsuspend+0xba sys_sigsuspend(ca982000,f272dccc,c1046c7c,c0ffb4a3,f272dcc4,...) at sys_sigsuspend+0x4d syscall(f272dd08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x2817573b, esp = 0xbfbfd0bc, ebp = 0xbfbfd128 --- Tracing command su pid 1340 tid 100122 td 0xc96175e0 sched_switch(c96175e0,0,104,1b3,f2820ab4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf040,...) at mi_switch+0x243 sleepq_switch(c96175e0,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(ca97cb30,6c,c0ffb5a4,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca97cb30,ca97cbb8,16c,c0ffb5a4,0,...) at _sleep+0x384 kern_wait(c96175e0,53d,f2820c38,2,0,...) at kern_wait+0x597 sys_wait4(c96175e0,f2820ccc,c1046c7c,c0ffbdca,f2820cc4,...) at sys_wait4+0x3b syscall(f2820d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2811114b, esp = 0xbfbfd60c, ebp = 0xbfbfd628 --- Tracing command bash pid 1335 tid 100102 td 0xca8a35e0 sched_switch(ca8a35e0,0,104,1b3,f2773ab4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf040,...) at mi_switch+0x243 sleepq_switch(ca8a35e0,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(ca6d5598,6c,c0ffb5a4,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca6d5598,ca6d5620,16c,c0ffb5a4,0,...) at _sleep+0x384 kern_wait(ca8a35e0,ffffffff,f2773c38,6,0,...) at kern_wait+0x597 sys_wait4(ca8a35e0,f2773ccc,c1046c7c,c0ffb4a3,f2773cc4,...) at sys_wait4+0x3b syscall(f2773d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x282b814b, esp = 0xbfbfd9ac, ebp = 0xbfbfd9c8 --- Tracing command sshd pid 1334 tid 100089 td 0xca8a45e0 sched_switch(ca8a45e0,0,104,1b3,f273996c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf6c0,...) at mi_switch+0x243 sleepq_switch(ca8a45e0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0a8bc8a,cb538750,0,c0ff09b7,ca8a45e0,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(cb538764,0,f2739a30,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(cb538764,cb538750,c0ffadc5,6ae,cad75118,...) at _cv_wait_sig+0x263 seltdwait(cad75118,58,cb14ea00,ca8a45e0,ca8a4694,...) at seltdwait+0xa2 kern_select(ca8a45e0,c,28c0c170,28c0c1b0,0,0,20,ca89b598,0) at kern_select+0x667 sys_select(ca8a45e0,f2739ccc,c1046c7c,c0ffb6f1,f2739c80,...) at sys_select+0x66 syscall(f2739d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284ed9c3, esp = 0xbfbfce5c, ebp = 0xbfbfcea8 --- Tracing command sshd pid 1332 tid 100091 td 0xca9815e0 sched_switch(ca9815e0,0,104,1b3,f27419d0,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf040,...) at mi_switch+0x243 sleepq_switch(ca9815e0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0a8bc8a,ca7fd790,0,c0ff09b7,ca9815e0,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(ca7fd7a4,0,f2741a94,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca7fd7a4,ca7fd790,c0ffadc5,6ae,f2741b1c,...) at _cv_wait_sig+0x263 seltdwait(cac6fa2c,c0ffadc5,5ab,ca9815e0,c0a5c672,...) at seltdwait+0xa2 sys_poll(ca9815e0,f2741ccc,c1046c7c,c0fdccd1,f2741cc4,...) at sys_poll+0x321 syscall(f2741d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x28484dcb, esp = 0xbfbfcebc, ebp = 0xbfbfcef8 --- Tracing command bash pid 1324 tid 100136 td 0xcb3718d0 sched_switch(cb3718d0,0,104,1b3,f284aab4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bfd40,...) at mi_switch+0x243 sleepq_switch(cb3718d0,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(cb519598,6c,c0ffb5a4,100,0,...) at sleepq_wait_sig+0x17 _sleep(cb519598,cb519620,16c,c0ffb5a4,0,...) at _sleep+0x384 kern_wait(cb3718d0,ffffffff,f284ac38,6,0,...) at kern_wait+0x597 sys_wait4(cb3718d0,f284accc,c1046c7c,c0ffb4a3,f284acc4,...) at sys_wait4+0x3b syscall(f284ad08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x282b814b, esp = 0xbfbfd79c, ebp = 0xbfbfd7b8 --- Tracing command csh pid 1321 tid 100135 td 0xca8f3bc0 sched_switch(ca8f3bc0,0,104,1b3,f2847b00,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bfd40,...) at mi_switch+0x243 sleepq_switch(ca8f3bc0,0,c0ff8600,1ac,74,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(cb5298bc,74,c0e7704e,100,0,...) at sleepq_wait_sig+0x17 _sleep(cb5298bc,cb5298ec,174,c0e7704e,0,...) at _sleep+0x384 kern_sigsuspend(ca8f3bc0,0,0,0,0,...) at kern_sigsuspend+0xba sys_sigsuspend(ca8f3bc0,f2847ccc,c1046c7c,c0ffb4a3,f2847cc4,...) at sys_sigsuspend+0x4d syscall(f2847d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x2817573b, esp = 0xbfbfd09c, ebp = 0xbfbfd108 --- Tracing command su pid 1320 tid 100090 td 0xca9818d0 sched_switch(ca9818d0,0,104,1b3,f273dab4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11be9c0,...) at mi_switch+0x243 sleepq_switch(ca9818d0,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(ca97b2cc,6c,c0ffb5a4,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca97b2cc,ca97b354,16c,c0ffb5a4,0,...) at _sleep+0x384 kern_wait(ca9818d0,529,f273dc38,2,0,...) at kern_wait+0x597 sys_wait4(ca9818d0,f273dccc,c1046c7c,c0ffbdca,f273dcc4,...) at sys_wait4+0x3b syscall(f273dd08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2811114b, esp = 0xbfbfd5ec, ebp = 0xbfbfd608 --- Tracing command bash pid 1310 tid 100132 td 0xcb373000 sched_switch(cb373000,0,104,1b3,f283eab4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf040,...) at mi_switch+0x243 sleepq_switch(cb373000,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(cb519b30,6c,c0ffb5a4,100,0,...) at sleepq_wait_sig+0x17 _sleep(cb519b30,cb519bb8,16c,c0ffb5a4,0,...) at _sleep+0x384 kern_wait(cb373000,ffffffff,f283ec38,6,0,...) at kern_wait+0x597 sys_wait4(cb373000,f283eccc,c1046c7c,c0ffb4a3,f283ecc4,...) at sys_wait4+0x3b syscall(f283ed08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x282b814b, esp = 0xbfbfd9ac, ebp = 0xbfbfd9c8 --- Tracing command sshd pid 1309 tid 100123 td 0xc96172f0 sched_switch(c96172f0,0,104,1b3,f282396c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf040,...) at mi_switch+0x243 sleepq_switch(c96172f0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0a8bc8a,cb5387d0,0,c0ff09b7,c96172f0,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(cb5387e4,0,f2823a30,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(cb5387e4,cb5387d0,c0ffadc5,6ae,ca8d1b60,...) at _cv_wait_sig+0x263 seltdwait(ca8d1b60,58,cbbef880,c96172f0,c96173a4,...) at seltdwait+0xa2 kern_select(c96172f0,c,28c0c170,28c0c1b0,0,0,20,ca97c864,0) at kern_select+0x667 sys_select(c96172f0,f2823ccc,c1046c7c,c0ffb6f1,f2823cc4,...) at sys_select+0x66 syscall(f2823d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284ed9c3, esp = 0xbfbfce5c, ebp = 0xbfbfcea8 --- Tracing command sshd pid 1307 tid 100145 td 0xca982bc0 sched_switch(ca982bc0,0,104,1b3,f28659d0,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf040,...) at mi_switch+0x243 sleepq_switch(ca982bc0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0a8bc8a,c94e73d0,0,c0ff09b7,ca982bc0,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(c94e73e4,0,f2865a94,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c94e73e4,c94e73d0,c0ffadc5,6ae,f2865b1c,...) at _cv_wait_sig+0x263 seltdwait(cb53062c,c0ffadc5,5ab,ca982bc0,c0a5c672,...) at seltdwait+0xa2 sys_poll(ca982bc0,f2865ccc,c1046c7c,c0fdccd1,f2865cc4,...) at sys_poll+0x321 syscall(f2865d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x28484dcb, esp = 0xbfbfcebc, ebp = 0xbfbfcef8 --- Tracing command cron pid 1219 tid 100127 td 0xcb374000 sched_switch(cb374000,0,104,1b3,f282faec,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,f282fb10,...) at mi_switch+0x243 sleepq_switch(cb374000,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(ea61,c0ae16e0,cb374000,3,100,...) at sleepq_catch_signals+0xfc sleepq_timedwait_sig(c11b1d64,6c,c0ff3f1d,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c11b1d64,0,16c,c0ff3f1d,ea61,...) at _sleep+0x34e kern_nanosleep(cb374000,f282fc28,f282fc30,3c,0,...) at kern_nanosleep+0xc1 sys_nanosleep(cb374000,f282fccc,c1046c7c,c0ffb9fa,14,...) at sys_nanosleep+0x6f syscall(f282fd08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x28179e63, esp = 0xbfbfdc7c, ebp = 0xbfbfdca8 --- Tracing command sendmail pid 1214 tid 100096 td 0xca8f78d0 sched_switch(ca8f78d0,0,104,1b3,f2759b00,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bfd40,...) at mi_switch+0x243 sleepq_switch(ca8f78d0,0,c0ff8600,1ac,74,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(ca8f25f0,74,c0e7704e,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca8f25f0,ca8f2620,174,c0e7704e,0,...) at _sleep+0x384 kern_sigsuspend(ca8f78d0,0,0,0,0,...) at kern_sigsuspend+0xba sys_sigsuspend(ca8f78d0,f2759ccc,c1046c7c,c0ffb4a3,c0ac9607,...) at sys_sigsuspend+0x4d syscall(f2759d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x2833173b, esp = 0xbfbfbfcc, ebp = 0xbfbfbff8 --- Tracing command sendmail pid 1211 tid 100105 td 0xca8f6bc0 sched_switch(ca8f6bc0,0,104,1b3,f2785968,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,46,...) at mi_switch+0x243 sleepq_switch(ca8f6bc0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(f27859fc,c0a8bc8a,ca7fd950,0,ca8f6bc0,...) at sleepq_catch_signals+0xfc sleepq_timedwait_sig(ca7fd964,0,f2785a30,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(ca7fd964,ca7fd950,1389,6ae,ca9305b0,...) at _cv_timedwait_sig+0x272 seltdwait(f2785be8,f2785bf0,cb159900,ca8f6bc0,40,...) at seltdwait+0x8a kern_select(ca8f6bc0,5,bfbfb540,0,0,f2785c34,20,5,0) at kern_select+0x667 sys_select(ca8f6bc0,f2785ccc,c1046c7c,c0ffb980,f2785c80,...) at sys_select+0x66 syscall(f2785d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x283e99c3, esp = 0xbfbfb4ac, ebp = 0xbfbfbfd8 --- Tracing command sshd pid 1208 tid 100085 td 0xca9822f0 sched_switch(ca9822f0,0,104,1b3,f272896c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf6c0,...) at mi_switch+0x243 sleepq_switch(ca9822f0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0a8bc8a,c94e74d0,0,c0ff09b7,ca9822f0,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(c94e74e4,0,f2728a30,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c94e74e4,c94e74d0,c0ffadc5,6ae,ca974c40,...) at _cv_wait_sig+0x263 seltdwait(ca974c40,58,cb158400,ca9822f0,c12fa760,...) at seltdwait+0xa2 kern_select(ca9822f0,9,28c0c140,0,0,0,20,ca97bb30,a) at kern_select+0x667 sys_select(ca9822f0,f2728ccc,c1046c7c,c0fdccd1,f2728cc4,...) at sys_select+0x66 syscall(f2728d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284ed9c3, esp = 0xbfbfcf1c, ebp = 0xbfbfde28 --- Tracing command moused pid 1176 tid 100094 td 0xca8a42f0 sched_switch(ca8a42f0,0,104,1b3,f275096c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf6c0,...) at mi_switch+0x243 sleepq_switch(ca8a42f0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0a8bc8a,ca7fd990,0,c0ff09b7,ca8a42f0,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(ca7fd9a4,0,f2750a30,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca7fd9a4,ca7fd990,c0ffadc5,6ae,ca8d2460,...) at _cv_wait_sig+0x263 seltdwait(ca8d2460,58,c7dbde80,ca8a42f0,f6,...) at seltdwait+0xa2 kern_select(ca8a42f0,400,bfbfda34,0,0,0,20,ca89b2cc,1) at kern_select+0x667 sys_select(ca8a42f0,f2750ccc,c1046c7c,c0e5447e,ab,...) at sys_select+0x66 syscall(f2750d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281af9c3, esp = 0xbfbfd9cc, ebp = 0xbfbfdb58 --- Tracing command ntpd pid 1151 tid 100093 td 0xca981000 sched_switch(ca981000,0,104,1b3,f274b96c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bfd40,...) at mi_switch+0x243 sleepq_switch(ca981000,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0a8bc8a,c94e78d0,0,c0ff09b7,ca981000,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(c94e78e4,0,f274ba30,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c94e78e4,c94e78d0,c0ffadc5,6ae,ca9307a8,...) at _cv_wait_sig+0x263 seltdwait(ca9307a8,58,c7dbde80,ca981000,0,...) at seltdwait+0xa2 kern_select(ca981000,1c,bfbfdd18,0,0,0,20,ca977864,0) at kern_select+0x667 sys_select(ca981000,f274bccc,c1046c7c,c0ffbddc,e,...) at sys_select+0x66 syscall(f274bd08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x283669c3, esp = 0xbfbfdcec, ebp = 0xbfbfddb8 --- Tracing command nfsd pid 1095 tid 100121 td 0xca89f8d0 sched_switch(ca89f8d0,0,104,1b3,f281db30,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,46,...) at mi_switch+0x243 sleepq_switch(ca89f8d0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(f281dbc4,c0a8bc8a,c951e880,0,ca89f8d0,...) at sleepq_catch_signals+0xfc sleepq_timedwait_sig(ca7903d0,0,f281dbf8,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(ca7903d0,c951e880,1388,3b1,c62b9128,...) at _cv_timedwait_sig+0x272 svc_run_internal(f281dcf4,c0a6e1c8,c951e880,f281dd08,c0fec990,...) at svc_run_internal+0x36a svc_thread_start(c951e880,f281dd08,c0fec990,3db,cb2cd000,...) at svc_thread_start+0x10 fork_exit(c0c9f610,c951e880,f281dd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x2e, eip = 0xc, esp = 0x33, ebp = 0x1 --- Tracing command nfsd pid 1095 tid 100120 td 0xca89fbc0 sched_switch(ca89fbc0,0,104,1b3,f281ab30,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,46,...) at mi_switch+0x243 sleepq_switch(ca89fbc0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(f281abc4,c0a8bc8a,c951e880,0,ca89fbc0,...) at sleepq_catch_signals+0xfc sleepq_timedwait_sig(ca790410,0,f281abf8,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(ca790410,c951e880,1388,3b1,c62bf098,...) at _cv_timedwait_sig+0x272 svc_run_internal(f281acf4,c0a6e1c8,c951e880,f281ad08,c0fec990,...) at svc_run_internal+0x36a svc_thread_start(c951e880,f281ad08,c0fec990,3db,cb2cd000,...) at svc_thread_start+0x10 fork_exit(c0c9f610,c951e880,f281ad08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x2e, eip = 0xc, esp = 0x33, ebp = 0x1 --- Tracing command nfsd pid 1095 tid 100119 td 0xca8a3000 sched_switch(ca8a3000,0,104,1b3,f2817b30,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,46,...) at mi_switch+0x243 sleepq_switch(ca8a3000,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(f2817bc4,c0a8bc8a,c951e880,0,ca8a3000,...) at sleepq_catch_signals+0xfc sleepq_timedwait_sig(ca790110,0,f2817bf8,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(ca790110,c951e880,1388,3b1,0,...) at _cv_timedwait_sig+0x272 svc_run_internal(f2817cf4,c0a6e1c8,c951e880,f2817d08,c0fec990,...) at svc_run_internal+0x36a svc_thread_start(c951e880,f2817d08,c0fec990,3db,cb2cd000,...) at svc_thread_start+0x10 fork_exit(c0c9f610,c951e880,f2817d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x2e, eip = 0xc, esp = 0x33, ebp = 0x1 --- Tracing command nfsd pid 1095 tid 100118 td 0xc96178d0 sched_switch(c96178d0,0,104,1b3,f28143e0,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,46,...) at mi_switch+0x243 sleepq_switch(c96178d0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(f2814474,c0a8bc8a,c951e880,0,c96178d0,...) at sleepq_catch_signals+0xfc sleepq_timedwait_sig(ca7e6950,0,f28144a8,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(ca7e6950,c951e880,1388,3b1,3,...) at _cv_timedwait_sig+0x272 svc_run_internal(c9617a78,14,c102315a,c0fde1db,f28147e8,...) at svc_run_internal+0x36a svc_run(c951e880,0,c0fde1b6,19f,0,...) at svc_run+0x7b nfsrvd_nfsd(c96178d0,f28147e8,c,0,0,...) at nfsrvd_nfsd+0x1c3 nfssvc_nfsd(c96178d0,f2814ccc,0,c96178d0,f2814cfc,...) at nfssvc_nfsd+0x177 sys_nfssvc(c96178d0,f2814ccc,c1046c7c,c0ffb980,0,...) at sys_nfssvc+0xeb syscall(f2814d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (155, FreeBSD ELF32, sys_nfssvc), eip = 0x280c537b, esp = 0xbfbfd85c, ebp = 0xbfbfdac8 --- Tracing command nfsd pid 1092 tid 100108 td 0xca8f65e0 sched_switch(ca8f65e0,0,104,1b3,f279596c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11be9c0,...) at mi_switch+0x243 sleepq_switch(ca8f65e0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0a8bc8a,ca799190,0,c0ff09b7,ca8f65e0,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(ca7991a4,0,f2795a30,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca7991a4,ca799190,c0ffadc5,6ae,ca928188,...) at _cv_wait_sig+0x263 seltdwait(ca928188,58,c7dbde80,ca8f65e0,cac976a0,...) at seltdwait+0xa2 kern_select(ca8f65e0,7,bfbfdce4,0,0,0,20,ca8ef2cc,1) at kern_select+0x667 sys_select(ca8f65e0,f2795ccc,c1046c7c,c0ffb980,f2795cc4,...) at sys_select+0x66 syscall(f2795d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x2817c9c3, esp = 0xbfbfdacc, ebp = 0xbfbfde18 --- Tracing command nfsuserd pid 1090 tid 100116 td 0xca6e8000 sched_switch(ca6e8000,0,104,1b3,f280e968,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,46,...) at mi_switch+0x243 sleepq_switch(ca6e8000,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(f280e9fc,c0a8bc8a,ca7fe410,0,ca6e8000,...) at sleepq_catch_signals+0xfc sleepq_timedwait_sig(ca7fe424,0,f280ea30,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(ca7fe424,ca7fe410,7531,6ae,ca974620,...) at _cv_timedwait_sig+0x272 seltdwait(f280ebe8,f280ebf0,c7dbde80,ca6e8000,ca942788,...) at seltdwait+0x8a kern_select(ca6e8000,4,bfbfdbf4,0,0,f280ec34,20,1e,0) at kern_select+0x667 sys_select(ca6e8000,f280eccc,c1046c7c,c0ffb6da,c0ac9607,...) at sys_select+0x66 syscall(f280ed08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x2817b9c3, esp = 0xbfbfdb4c, ebp = 0xbfbfdc88 --- Tracing command nfsuserd pid 1089 tid 100115 td 0xca6e82f0 sched_switch(ca6e82f0,0,104,1b3,f280b968,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,46,...) at mi_switch+0x243 sleepq_switch(ca6e82f0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(f280b9fc,c0a8bc8a,ca7fd9d0,0,ca6e82f0,...) at sleepq_catch_signals+0xfc sleepq_timedwait_sig(ca7fd9e4,0,f280ba30,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(ca7fd9e4,ca7fd9d0,7531,6ae,ca974620,...) at _cv_timedwait_sig+0x272 seltdwait(f280bbe8,f280bbf0,c7dbde80,ca6e82f0,c7de5b28,...) at seltdwait+0x8a kern_select(ca6e82f0,4,bfbfdbf4,0,0,f280bc34,20,1e,0) at kern_select+0x667 sys_select(ca6e82f0,f280bccc,c1046c7c,c0ffb6da,0,...) at sys_select+0x66 syscall(f280bd08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x2817b9c3, esp = 0xbfbfdb4c, ebp = 0xbfbfdc88 --- Tracing command nfsuserd pid 1088 tid 100114 td 0xca6e85e0 sched_switch(ca6e85e0,0,104,1b3,f2808968,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,46,...) at mi_switch+0x243 sleepq_switch(ca6e85e0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(f28089fc,c0a8bc8a,ca7fe450,0,ca6e85e0,...) at sleepq_catch_signals+0xfc sleepq_timedwait_sig(ca7fe464,0,f2808a30,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(ca7fe464,ca7fe450,7531,6ae,ca974620,...) at _cv_timedwait_sig+0x272 seltdwait(f2808be8,f2808bf0,c7dbde80,ca6e85e0,c7de55b8,...) at seltdwait+0x8a kern_select(ca6e85e0,4,bfbfdbf4,0,0,f2808c34,20,1e,0) at kern_select+0x667 sys_select(ca6e85e0,f2808ccc,c1046c7c,c0ffb6da,0,...) at sys_select+0x66 syscall(f2808d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x2817b9c3, esp = 0xbfbfdb4c, ebp = 0xbfbfdc88 --- Tracing command nfsuserd pid 1087 tid 100113 td 0xca6e88d0 sched_switch(ca6e88d0,0,104,1b3,f2805968,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,46,...) at mi_switch+0x243 sleepq_switch(ca6e88d0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(f28059fc,c0a8bc8a,ca7fda10,0,ca6e88d0,...) at sleepq_catch_signals+0xfc sleepq_timedwait_sig(ca7fda24,0,f2805a30,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(ca7fda24,ca7fda10,7531,6ae,ca974620,...) at _cv_timedwait_sig+0x272 seltdwait(f2805be8,f2805bf0,c7dbde80,ca6e88d0,ca896870,...) at seltdwait+0x8a kern_select(ca6e88d0,4,bfbfdbf4,0,0,f2805c34,20,1e,0) at kern_select+0x667 sys_select(ca6e88d0,f2805ccc,c1046c7c,c0ffb6da,c62f3c58,...) at sys_select+0x66 syscall(f2805d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x2817b9c3, esp = 0xbfbfdb4c, ebp = 0xbfbfdc88 --- Tracing command nfsuserd pid 1086 tid 100109 td 0xca8f62f0 sched_switch(ca8f62f0,0,104,1b3,f279eb00,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11be9c0,...) at mi_switch+0x243 sleepq_switch(ca8f62f0,0,c0ff8600,1ac,74,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(ca8ef058,74,c0e7704e,100,0,...) at sleepq_wait_sig+0x17 _sleep(ca8ef058,ca8ef088,174,c0e7704e,0,...) at _sleep+0x384 kern_sigsuspend(ca8f62f0,0,0,0,0,...) at kern_sigsuspend+0xba sys_sigsuspend(ca8f62f0,f279eccc,c1046c7c,c0ffb980,c0ac9607,...) at sys_sigsuspend+0x4d syscall(f279ed08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x280c373b, esp = 0xbfbfdc8c, ebp = 0xbfbfde28 --- Tracing command mountd pid 1070 tid 100103 td 0xca8a32f0 sched_switch(ca8a32f0,0,104,1b3,f277796c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf6c0,...) at mi_switch+0x243 sleepq_switch(ca8a32f0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0a8bc8a,ca7fda50,0,c0ff09b7,ca8a32f0,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(ca7fda64,0,f2777a30,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca7fda64,ca7fda50,c0ffadc5,6ae,ca974380,...) at _cv_wait_sig+0x263 seltdwait(ca974380,58,c7dbde80,ca8a32f0,4,...) at seltdwait+0xa2 kern_select(ca8a32f0,9,bfbfdd34,0,0,0,20,ca6d52cc,bfbfdd34) at kern_select+0x667 sys_select(ca8a32f0,f2777ccc,c1046c7c,c0fdbf37,ab,...) at sys_select+0x66 syscall(f2777d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281939c3, esp = 0xbfbfdccc, ebp = 0xbfbfde18 --- Tracing command rpcbind pid 1037 tid 100081 td 0xca8f7bc0 sched_switch(ca8f7bc0,0,104,1b3,f27169cc,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,46,...) at mi_switch+0x243 sleepq_switch(ca8f7bc0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(f2716a60,c0a8bc8a,c94e7a90,0,ca8f7bc0,...) at sleepq_catch_signals+0xfc sleepq_timedwait_sig(c94e7aa4,0,f2716a94,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c94e7aa4,c94e7a90,7531,6ae,f2716b4c,...) at _cv_timedwait_sig+0x272 seltdwait(f2716c1c,f2716c24,5ab,ca8f7bc0,a22,...) at seltdwait+0x8a sys_poll(ca8f7bc0,f2716ccc,c1046c7c,c0ffb6da,f2716cc4,...) at sys_poll+0x321 syscall(f2716d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x28132dcb, esp = 0xbfbfbbdc, ebp = 0xbfbfdda8 --- Tracing command syslogd pid 967 tid 100099 td 0xca8f7000 sched_switch(ca8f7000,0,104,1b3,f276596c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11be9c0,...) at mi_switch+0x243 sleepq_switch(ca8f7000,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0a8bc8a,ca799150,0,c0ff09b7,ca8f7000,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(ca799164,0,f2765a30,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(ca799164,ca799150,c0ffadc5,6ae,ca8d2188,...) at _cv_wait_sig+0x263 seltdwait(ca8d2188,58,c7dbde80,ca8f7000,0,...) at seltdwait+0xa2 kern_select(ca8f7000,9,28849100,0,0,0,20,ca8efb30,0) at kern_select+0x667 sys_select(ca8f7000,f2765ccc,c1046c7c,c10291ec,e,...) at sys_select+0x66 syscall(f2765d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281939c3, esp = 0xbfbfd2cc, ebp = 0xbfbfde18 --- Tracing command devd pid 844 tid 100082 td 0xca6e95e0 sched_switch(ca6e95e0,0,104,1b3,f271a96c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf6c0,...) at mi_switch+0x243 sleepq_switch(ca6e95e0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0a8bc8a,c961c690,0,c0ff09b7,ca6e95e0,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(c961c6a4,0,f271aa30,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c961c6a4,c961c690,c0ffadc5,6ae,ca8d1620,...) at _cv_wait_sig+0x263 seltdwait(ca8d1620,58,c7dbde80,ca6e95e0,ca6e9694,...) at seltdwait+0xa2 kern_select(ca6e95e0,5,bfbfd990,0,0,0,20,c9427b30,0) at kern_select+0x667 sys_select(ca6e95e0,f271accc,c1046c7c,c0e4f55f,f271acc4,...) at sys_select+0x66 syscall(f271ad08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x808935b, esp = 0xbfbfd95c, ebp = 0xbfbfde38 --- Tracing command softdepflush pid 19 tid 100078 td 0xca6e98d0 sched_switch(ca6e98d0,0,104,1b3,ee332bf8,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,352,...) at mi_switch+0x243 sleepq_switch(ca6e98d0,0,c0ff8600,291,0,...) at sleepq_switch+0x1a7 sleepq_timedwait(c132fc80,54,c102aa51,0,0,...) at sleepq_timedwait+0x6b _sleep(c132fc80,c132fc40,54,c102aa51,3e8,...) at _sleep+0x369 softdep_flush(0,ee332d08,c0fec990,3db,ca6d4000,...) at softdep_flush+0x259 fork_exit(c0cf26b0,0,ee332d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee332d40, ebp = 0 --- Tracing command vnlru pid 18 tid 100077 td 0xca6e9bc0 sched_switch(ca6e9bc0,0,104,1b3,ee32fbcc,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,352,...) at mi_switch+0x243 sleepq_switch(ca6e9bc0,0,c0ff8600,291,0,...) at sleepq_switch+0x1a7 sleepq_timedwait(ca6d42cc,60,c100389c,0,0,...) at sleepq_timedwait+0x6b _sleep(ca6d42cc,c1327f34,260,c100389c,3e8,...) at _sleep+0x369 vnlru_proc(0,ee32fd08,c0fec990,3db,ca6d42cc,...) at vnlru_proc+0xe7 fork_exit(c0b447e0,0,ee32fd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee32fd40, ebp = 0 --- Tracing command syncer pid 17 tid 100076 td 0xc94fc8d0 sched_switch(c94fc8d0,0,104,1b3,ee32cbd4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c0ff09b7,...) at mi_switch+0x243 sleepq_switch(c94fc8d0,0,c0ff8600,291,c94fc8d0,...) at sleepq_switch+0x1a7 sleepq_timedwait(c1327f74,0,ee32cc58,1,0,...) at sleepq_timedwait+0x6b _cv_timedwait(c1327f74,c1327f60,3e8,783,4e20,...) at _cv_timedwait+0x272 sched_sync(0,ee32cd08,c0fec990,3db,ca6d4598,...) at sched_sync+0x595 fork_exit(c0b43b00,0,ee32cd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee32cd40, ebp = 0 --- Tracing command bufdaemon pid 16 tid 100075 td 0xc94fcbc0 sched_switch(c94fcbc0,0,104,1b3,ee329c04,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,352,...) at mi_switch+0x243 sleepq_switch(c94fcbc0,0,c0ff8600,291,0,...) at sleepq_switch+0x1a7 sleepq_timedwait(c1327c08,54,c10005f1,0,0,...) at sleepq_timedwait+0x6b _sleep(c1327c08,c1327c10,54,c10005f1,3e8,...) at _sleep+0x369 buf_daemon(0,ee329d08,c0fec990,3db,ca6d4864,...) at buf_daemon+0x175 fork_exit(c0b26450,0,ee329d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee329d40, ebp = 0 --- Tracing command pagezero pid 9 tid 100074 td 0xc9615000 sched_switch(c9615000,0,104,1b3,ee326c08,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,352,...) at mi_switch+0x243 sleepq_switch(c9615000,0,c0ff8600,291,0,...) at sleepq_switch+0x1a7 sleepq_timedwait(c1331c5c,0,c1031cc9,0,0,...) at sleepq_timedwait+0x6b _sleep(c1331c5c,c1330600,0,c1031cc9,493e0,...) at _sleep+0x369 vm_pagezero(0,ee326d08,c0fec990,3db,ca6d4b30,...) at vm_pagezero+0xdc fork_exit(c0d39910,0,ee326d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee326d40, ebp = 0 --- Tracing command vmdaemon pid 8 tid 100073 td 0xc96152f0 sched_switch(c96152f0,0,104,1b3,ee323bcc,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c7dbad80,...) at mi_switch+0x243 sleepq_switch(c96152f0,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(c13317c8,74,c10005f1,0,0,...) at sleepq_wait+0x6b _sleep(c13317c8,c13317cc,74,c10005f1,0,...) at _sleep+0x39b vm_daemon(0,ee323d08,c0fec990,3db,ca6d5000,...) at vm_daemon+0x59 fork_exit(c0d33a00,0,ee323d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee323d40, ebp = 0 --- Tracing command pagedaemon pid 7 tid 100072 td 0xc96155e0 sched_switch(c96155e0,0,104,1b3,ee320b60,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,352,...) at mi_switch+0x243 sleepq_switch(c96155e0,0,c0ff8600,291,0,...) at sleepq_switch+0x1a7 sleepq_timedwait(c1331790,54,c10005f1,0,0,...) at sleepq_timedwait+0x6b _sleep(c1331790,c1330600,54,c10005f1,1388,...) at _sleep+0x369 vm_pageout(0,ee320d08,c0fec990,3db,c7de12cc,...) at vm_pageout+0x2c7 fork_exit(c0d33f00,0,ee320d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee320d40, ebp = 0 --- Tracing command xpt_thrd pid 6 tid 100071 td 0xc96158d0 sched_switch(c96158d0,0,104,1b3,ee19dc04,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c7dbad00,...) at mi_switch+0x243 sleepq_switch(c96158d0,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(c1174654,5c,c0e48e88,0,0,...) at sleepq_wait+0x6b _sleep(c1174654,c1174698,5c,c0e48e88,0,...) at _sleep+0x39b xpt_scanner_thread(0,ee19dd08,c0fec990,3db,c7de1598,...) at xpt_scanner_thread+0x47 fork_exit(c04a8eb0,0,ee19dd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee19dd40, ebp = 0 --- Tracing command sctp_iterator pid 5 tid 100070 td 0xc9615bc0 sched_switch(c9615bc0,0,104,1b3,ee19ac0c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c7dbacc0,...) at mi_switch+0x243 sleepq_switch(c9615bc0,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(c132b498,0,c1017ec7,0,0,...) at sleepq_wait+0x6b _sleep(c132b498,c132b460,0,c1017ec7,0,...) at _sleep+0x39b sctp_iterator_thread(0,ee19ad08,c0fec990,3db,c7de1864,...) at sctp_iterator_thread+0x5c fork_exit(c0be0660,0,ee19ad08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee19ad40, ebp = 0 --- Tracing command fdc0 pid 4 tid 100065 td 0xc9616bc0 sched_switch(c9616bc0,0,104,1b3,ee18ab54,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,352,...) at mi_switch+0x243 sleepq_switch(c9616bc0,0,c0ff8600,291,0,...) at sleepq_switch+0x1a7 sleepq_timedwait(c949c63c,5c,c0fdec52,0,0,...) at sleepq_timedwait+0x6b _sleep(c949c63c,c949c6f0,5c,c0fdec52,3e8,...) at _sleep+0x369 fdc_thread(c949c600,ee18ad08,c0fec990,3db,c7de1b30,...) at fdc_thread+0x27d fork_exit(c0dbda00,c949c600,ee18ad08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee18ad40, ebp = 0 --- Tracing command fw0_probe pid 3 tid 100063 td 0xc94ea2f0 sched_switch(c94ea2f0,0,104,1b3,ee17a6f8,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11be9c0,...) at mi_switch+0x243 sleepq_switch(c94ea2f0,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(c9522000,6c,c0fdec52,100,0,...) at sleepq_wait_sig+0x17 _sleep(c9522000,c9526488,16c,c0fdec52,0,...) at _sleep+0x384 fw_bus_probe_thread(c9522000,ee17ad08,c0fec990,3db,c9427000,...) at fw_bus_probe_thread+0xa08 fork_exit(c0712b80,c9522000,ee17ad08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee17ad40, ebp = 0 --- Tracing command usb pid 15 tid 100059 td 0xc94fa000 sched_switch(c94fa000,0,104,1b3,ee159c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94fa000,0,c0ff8600,26e,c94fa000,...) at sleepq_switch+0x1a7 sleepq_wait(c9506d34,0,ee159c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c9506d34,c9506e4c,c0fd02d1,74,c9506d3c,...) at _cv_wait+0x263 usb_process(c9506d2c,ee159d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c9506d2c,ee159d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee159d40, ebp = 0 --- Tracing command usb pid 15 tid 100058 td 0xc94fa2f0 sched_switch(c94fa2f0,0,104,1b3,ee156c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94fa2f0,0,c0ff8600,26e,c94fa2f0,...) at sleepq_switch+0x1a7 sleepq_wait(c9506d04,0,ee156c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c9506d04,c9506e4c,c0fd02d1,74,c9506d0c,...) at _cv_wait+0x263 usb_process(c9506cfc,ee156d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c9506cfc,ee156d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee156d40, ebp = 0 --- Tracing command usb pid 15 tid 100057 td 0xc94fa5e0 sched_switch(c94fa5e0,0,104,1b3,ee153c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94fa5e0,0,c0ff8600,26e,c94fa5e0,...) at sleepq_switch+0x1a7 sleepq_wait(c9506cd4,0,ee153c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c9506cd4,c9506e4c,c0fd02d1,74,c9506cdc,...) at _cv_wait+0x263 usb_process(c9506ccc,ee153d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c9506ccc,ee153d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee153d40, ebp = 0 --- Tracing command usb pid 15 tid 100056 td 0xc94fa8d0 sched_switch(c94fa8d0,0,104,1b3,ee150c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94fa8d0,0,c0ff8600,26e,c94fa8d0,...) at sleepq_switch+0x1a7 sleepq_wait(c9506ca4,0,ee150c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c9506ca4,c9506e4c,c0fd02d1,74,c9506cac,...) at _cv_wait+0x263 usb_process(c9506c9c,ee150d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c9506c9c,ee150d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee150d40, ebp = 0 --- Tracing command usb pid 15 tid 100054 td 0xc94fc000 sched_switch(c94fc000,0,104,1b3,ee147c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94fc000,0,c0ff8600,26e,c94fc000,...) at sleepq_switch+0x1a7 sleepq_wait(c94f4b5c,0,ee147c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94f4b5c,c94f4c74,c0fd02d1,74,c94f4b64,...) at _cv_wait+0x263 usb_process(c94f4b54,ee147d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94f4b54,ee147d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee147d40, ebp = 0 --- Tracing command usb pid 15 tid 100053 td 0xc94fc2f0 sched_switch(c94fc2f0,0,104,1b3,ee144c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94fc2f0,0,c0ff8600,26e,c94fc2f0,...) at sleepq_switch+0x1a7 sleepq_wait(c94f4b2c,0,ee144c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94f4b2c,c94f4c74,c0fd02d1,74,c94f4b34,...) at _cv_wait+0x263 usb_process(c94f4b24,ee144d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94f4b24,ee144d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee144d40, ebp = 0 --- Tracing command usb pid 15 tid 100052 td 0xc94fc5e0 sched_switch(c94fc5e0,0,104,1b3,ee141c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94fc5e0,0,c0ff8600,26e,c94fc5e0,...) at sleepq_switch+0x1a7 sleepq_wait(c94f4afc,0,ee141c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94f4afc,c94f4c74,c0fd02d1,74,c94f4b04,...) at _cv_wait+0x263 usb_process(c94f4af4,ee141d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94f4af4,ee141d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee141d40, ebp = 0 --- Tracing command usb pid 15 tid 100051 td 0xc94e8000 sched_switch(c94e8000,0,104,1b3,ee13ec10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94e8000,0,c0ff8600,26e,c94e8000,...) at sleepq_switch+0x1a7 sleepq_wait(c94f4acc,0,ee13ec8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94f4acc,c94f4c74,c0fd02d1,74,c94f4ad4,...) at _cv_wait+0x263 usb_process(c94f4ac4,ee13ed08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94f4ac4,ee13ed08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee13ed40, ebp = 0 --- Tracing command usb pid 15 tid 100050 td 0xc94e82f0 sched_switch(c94e82f0,0,104,1b3,ee13ac10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94e82f0,0,c0ff8600,26e,c94e82f0,...) at sleepq_switch+0x1a7 sleepq_wait(c94efb5c,0,ee13ac8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94efb5c,c94efc74,c0fd02d1,74,c94efb64,...) at _cv_wait+0x263 usb_process(c94efb54,ee13ad08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94efb54,ee13ad08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee13ad40, ebp = 0 --- Tracing command usb pid 15 tid 100049 td 0xc94e85e0 sched_switch(c94e85e0,0,104,1b3,ee137c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94e85e0,0,c0ff8600,26e,c94e85e0,...) at sleepq_switch+0x1a7 sleepq_wait(c94efb2c,0,ee137c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94efb2c,c94efc74,c0fd02d1,74,c94efb34,...) at _cv_wait+0x263 usb_process(c94efb24,ee137d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94efb24,ee137d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee137d40, ebp = 0 --- Tracing command usb pid 15 tid 100048 td 0xc94e88d0 sched_switch(c94e88d0,0,104,1b3,ee134c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94e88d0,0,c0ff8600,26e,c94e88d0,...) at sleepq_switch+0x1a7 sleepq_wait(c94efafc,0,ee134c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94efafc,c94efc74,c0fd02d1,74,c94efb04,...) at _cv_wait+0x263 usb_process(c94efaf4,ee134d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94efaf4,ee134d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee134d40, ebp = 0 --- Tracing command usb pid 15 tid 100047 td 0xc94e8bc0 sched_switch(c94e8bc0,0,104,1b3,ee131c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94e8bc0,0,c0ff8600,26e,c94e8bc0,...) at sleepq_switch+0x1a7 sleepq_wait(c94efacc,0,ee131c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94efacc,c94efc74,c0fd02d1,74,c94efad4,...) at _cv_wait+0x263 usb_process(c94efac4,ee131d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94efac4,ee131d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee131d40, ebp = 0 --- Tracing command usb pid 15 tid 100046 td 0xc94e9000 sched_switch(c94e9000,0,104,1b3,ee12dc10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94e9000,0,c0ff8600,26e,c94e9000,...) at sleepq_switch+0x1a7 sleepq_wait(c94ebb5c,0,ee12dc8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94ebb5c,c94ebc74,c0fd02d1,74,c94ebb64,...) at _cv_wait+0x263 usb_process(c94ebb54,ee12dd08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94ebb54,ee12dd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee12dd40, ebp = 0 --- Tracing command usb pid 15 tid 100045 td 0xc94e92f0 sched_switch(c94e92f0,0,104,1b3,ee12ac10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94e92f0,0,c0ff8600,26e,c94e92f0,...) at sleepq_switch+0x1a7 sleepq_wait(c94ebb2c,0,ee12ac8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94ebb2c,c94ebc74,c0fd02d1,74,c94ebb34,...) at _cv_wait+0x263 usb_process(c94ebb24,ee12ad08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94ebb24,ee12ad08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee12ad40, ebp = 0 --- Tracing command usb pid 15 tid 100044 td 0xc94e95e0 sched_switch(c94e95e0,0,104,1b3,ee127c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94e95e0,0,c0ff8600,26e,c94e95e0,...) at sleepq_switch+0x1a7 sleepq_wait(c94ebafc,0,ee127c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94ebafc,c94ebc74,c0fd02d1,74,c94ebb04,...) at _cv_wait+0x263 usb_process(c94ebaf4,ee127d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94ebaf4,ee127d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee127d40, ebp = 0 --- Tracing command usb pid 15 tid 100043 td 0xc94e98d0 sched_switch(c94e98d0,0,104,1b3,ee124c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94e98d0,0,c0ff8600,26e,c94e98d0,...) at sleepq_switch+0x1a7 sleepq_wait(c94ebacc,0,ee124c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94ebacc,c94ebc74,c0fd02d1,74,c94ebad4,...) at _cv_wait+0x263 usb_process(c94ebac4,ee124d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94ebac4,ee124d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee124d40, ebp = 0 --- Tracing command usb pid 15 tid 100041 td 0xc94ea000 sched_switch(c94ea000,0,104,1b3,ee11dc10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94ea000,0,c0ff8600,26e,c94ea000,...) at sleepq_switch+0x1a7 sleepq_wait(c94e2b5c,0,ee11dc8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94e2b5c,c94e2c74,c0fd02d1,74,c94e2b64,...) at _cv_wait+0x263 usb_process(c94e2b54,ee11dd08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94e2b54,ee11dd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee11dd40, ebp = 0 --- Tracing command usb pid 15 tid 100040 td 0xc94c5000 sched_switch(c94c5000,0,104,1b3,ee11ac10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94c5000,0,c0ff8600,26e,c94c5000,...) at sleepq_switch+0x1a7 sleepq_wait(c94e2b2c,0,ee11ac8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94e2b2c,c94e2c74,c0fd02d1,74,c94e2b34,...) at _cv_wait+0x263 usb_process(c94e2b24,ee11ad08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94e2b24,ee11ad08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee11ad40, ebp = 0 --- Tracing command usb pid 15 tid 100039 td 0xc94c52f0 sched_switch(c94c52f0,0,104,1b3,ee117c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94c52f0,0,c0ff8600,26e,c94c52f0,...) at sleepq_switch+0x1a7 sleepq_wait(c94e2afc,0,ee117c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94e2afc,c94e2c74,c0fd02d1,74,c94e2b04,...) at _cv_wait+0x263 usb_process(c94e2af4,ee117d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94e2af4,ee117d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee117d40, ebp = 0 --- Tracing command usb pid 15 tid 100038 td 0xc94c55e0 sched_switch(c94c55e0,0,104,1b3,ee114c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94c55e0,0,c0ff8600,26e,c94c55e0,...) at sleepq_switch+0x1a7 sleepq_wait(c94e2acc,0,ee114c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94e2acc,c94e2c74,c0fd02d1,74,c94e2ad4,...) at _cv_wait+0x263 usb_process(c94e2ac4,ee114d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94e2ac4,ee114d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee114d40, ebp = 0 --- Tracing command usb pid 15 tid 100036 td 0xc94c5bc0 sched_switch(c94c5bc0,0,104,1b3,ee10dc10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94c5bc0,0,c0ff8600,26e,c94c5bc0,...) at sleepq_switch+0x1a7 sleepq_wait(c94dcb5c,0,ee10dc8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94dcb5c,c94dcc74,c0fd02d1,74,c94dcb64,...) at _cv_wait+0x263 usb_process(c94dcb54,ee10dd08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94dcb54,ee10dd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee10dd40, ebp = 0 --- Tracing command usb pid 15 tid 100035 td 0xc94c8000 sched_switch(c94c8000,0,104,1b3,ee10ac10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94c8000,0,c0ff8600,26e,c94c8000,...) at sleepq_switch+0x1a7 sleepq_wait(c94dcb2c,0,ee10ac8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94dcb2c,c94dcc74,c0fd02d1,74,c94dcb34,...) at _cv_wait+0x263 usb_process(c94dcb24,ee10ad08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94dcb24,ee10ad08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee10ad40, ebp = 0 --- Tracing command usb pid 15 tid 100034 td 0xc94c82f0 sched_switch(c94c82f0,0,104,1b3,ee107c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94c82f0,0,c0ff8600,26e,c94c82f0,...) at sleepq_switch+0x1a7 sleepq_wait(c94dcafc,0,ee107c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94dcafc,c94dcc74,c0fd02d1,74,c94dcb04,...) at _cv_wait+0x263 usb_process(c94dcaf4,ee107d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94dcaf4,ee107d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee107d40, ebp = 0 --- Tracing command usb pid 15 tid 100033 td 0xc94c85e0 sched_switch(c94c85e0,0,104,1b3,ee104c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94c85e0,0,c0ff8600,26e,c94c85e0,...) at sleepq_switch+0x1a7 sleepq_wait(c94dcacc,0,ee104c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94dcacc,c94dcc74,c0fd02d1,74,c94dcad4,...) at _cv_wait+0x263 usb_process(c94dcac4,ee104d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94dcac4,ee104d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee104d40, ebp = 0 --- Tracing command ctl_thrd pid 2 tid 100028 td 0xc7eea8d0 sched_switch(c7eea8d0,0,104,1b3,c7b7265c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c7db98c0,...) at mi_switch+0x243 sleepq_switch(c7eea8d0,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(c7fdd000,5c,c0e5491c,0,0,...) at sleepq_wait+0x6b _sleep(c7fdd000,c7fdd000,5c,c0e5491c,0,...) at _sleep+0x39b ctl_work_thread(c7fdd000,c7b72d08,c0fec990,3db,c9427598,...) at ctl_work_thread+0x22f6 fork_exit(c04d6960,c7fdd000,c7b72d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b72d40, ebp = 0 --- Tracing command yarrow pid 14 tid 100017 td 0xc7ee95e0 sched_switch(c7ee95e0,0,104,1b3,c7b50bd8,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,352,...) at mi_switch+0x243 sleepq_switch(c7ee95e0,0,c0ff8600,291,2,...) at sleepq_switch+0x1a7 sleepq_timedwait(c11b0b64,0,c0fdec52,2,0,...) at sleepq_timedwait+0x6b _sleep(c11b0b64,0,0,c0fdec52,64) at _sleep+0x369 pause(c0fdec52,64,c0fac230,111,0,...) at pause+0xa0 random_kthread(0,c7b50d08,c0fec990,3db,c7de0000,...) at random_kthread+0x1ef fork_exit(c0841f70,0,c7b50d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b50d40, ebp = 0 --- Tracing command geom pid 13 tid 100015 td 0xc7ee9bc0 sched_switch(c7ee9bc0,0,104,1b3,c7b4abc4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c7db99c0,...) at mi_switch+0x243 sleepq_switch(c7ee9bc0,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(c11adc08,5c,c0fdec52,0,0,...) at sleepq_wait+0x6b _sleep(c11adc08,c11adb68,25c,c0fdec52,0,...) at _sleep+0x39b g_io_schedule_down(c7ee9bc0,0,c0fe07e5,6c,c7b4acf4,...) at g_io_schedule_down+0x56 g_down_procbody(0,c7b4ad08,c0fec990,3db,c7de02cc,...) at g_down_procbody+0x9d fork_exit(c09fd6e0,0,c7b4ad08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b4ad40, ebp = 0 --- Tracing command geom pid 13 tid 100014 td 0xc7eea000 sched_switch(c7eea000,0,104,1b3,c7b47bf0,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c7db9980,...) at mi_switch+0x243 sleepq_switch(c7eea000,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(c11adc04,5c,c0fdec52,0,0,...) at sleepq_wait+0x6b _sleep(c11adc04,c11adb88,25c,c0fdec52,0,...) at _sleep+0x39b g_io_schedule_up(c7eea000,0,c0fe07e5,5f,c7b47cf4,...) at g_io_schedule_up+0x144 g_up_procbody(0,c7b47d08,c0fec990,3db,c7de02cc,...) at g_up_procbody+0x9d fork_exit(c09fd780,0,c7b47d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b47d40, ebp = 0 --- Tracing command geom pid 13 tid 100013 td 0xc7de32f0 sched_switch(c7de32f0,0,104,1b3,c7b44be8,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c7db9d00,...) at mi_switch+0x243 sleepq_switch(c7de32f0,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(c11adbfc,5c,c0fdec52,0,0,...) at sleepq_wait+0x6b _sleep(c11adbfc,c11adb2c,25c,c0fdec52,0,...) at _sleep+0x39b g_run_events(c11be9c0,0,c0fe07e5,79,c7b44cf4,...) at g_run_events+0x54b g_event_procbody(0,c7b44d08,c0fec990,3db,c7de02cc,...) at g_event_procbody+0x90 fork_exit(c09fd820,0,c7b44d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b44d40, ebp = 0 --- Tracing command intr pid 12 tid 100067 td 0xc96165e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100066 td 0xc96168d0 sched_switch(c96168d0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7d7c570,...) at mi_switch+0x243 ithread_loop(c7d77780,ee18dd08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d77780,ee18dd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee18dd40, ebp = 0 --- Tracing command intr pid 12 tid 100064 td 0xc9617000 sched_switch(c9617000,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c94fe2f0,...) at mi_switch+0x243 ithread_loop(c7d77520,ee17dd08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d77520,ee17dd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee17dd40, ebp = 0 --- Tracing command intr pid 12 tid 100061 td 0xc94ea8d0 sched_switch(c94ea8d0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7d7dc70,...) at mi_switch+0x243 ithread_loop(c7d771d0,ee171d08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d771d0,ee171d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee171d40, ebp = 0 --- Tracing command intr pid 12 tid 100060 td 0xc94eabc0 sched_switch(c94eabc0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7e24070,...) at mi_switch+0x243 ithread_loop(c7d77340,ee15fd08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d77340,ee15fd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee15fd40, ebp = 0 --- Tracing command intr pid 12 tid 100055 td 0xc94fabc0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100042 td 0xc94e9bc0 sched_switch(c94e9bc0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7d7dd70,...) at mi_switch+0x243 ithread_loop(c7d77490,ee121d08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d77490,ee121d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee121d40, ebp = 0 --- Tracing command intr pid 12 tid 100037 td 0xc94c58d0 sched_switch(c94c58d0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7d7ddf0,...) at mi_switch+0x243 ithread_loop(c7d774c0,ee111d08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d774c0,ee111d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee111d40, ebp = 0 --- Tracing command intr pid 12 tid 100032 td 0xc94c88d0 sched_switch(c94c88d0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7d7de70,...) at mi_switch+0x243 ithread_loop(c7d774f0,ee101d08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d774f0,ee101d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee101d40, ebp = 0 --- Tracing command intr pid 12 tid 100031 td 0xc94c8bc0 sched_switch(c94c8bc0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7d7db70,...) at mi_switch+0x243 ithread_loop(c7d77530,edfabd08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d77530,edfabd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xedfabd40, ebp = 0 --- Tracing command intr pid 12 tid 100030 td 0xc7eea2f0 sched_switch(c7eea2f0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7e241f0,...) at mi_switch+0x243 ithread_loop(c7d77590,edfa7d08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d77590,edfa7d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xedfa7d40, ebp = 0 --- Tracing command intr pid 12 tid 100029 td 0xc7eea5e0 sched_switch(c7eea5e0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7e24570,...) at mi_switch+0x243 ithread_loop(c7d776c0,c7ba4d08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d776c0,c7ba4d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7ba4d40, ebp = 0 --- Tracing command intr pid 12 tid 100026 td 0xc7fd4000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100024 td 0xc7fd45e0 sched_switch(c7fd45e0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7e24670,...) at mi_switch+0x243 ithread_loop(c7d776e0,c7b66d08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d776e0,c7b66d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b66d40, ebp = 0 --- Tracing command intr pid 12 tid 100020 td 0xc7e25bc0 sched_switch(c7e25bc0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7e249f0,...) at mi_switch+0x243 ithread_loop(c7d776f0,c7b5ad08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d776f0,c7b5ad08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b5ad40, ebp = 0 --- Tracing command intr pid 12 tid 100019 td 0xc7ee9000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100012 td 0xc7de35e0 sched_switch(c7de35e0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7d7d570,...) at mi_switch+0x243 ithread_loop(c7d77710,c7b41d08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d77710,c7b41d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b41d40, ebp = 0 --- Tracing command intr pid 12 tid 100011 td 0xc7de38d0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100010 td 0xc7de3bc0 sched_switch(c7de3bc0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7d7d670,...) at mi_switch+0x243 ithread_loop(c7d77730,c7b3bd08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d77730,c7b3bd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b3bd40, ebp = 0 --- Tracing command intr pid 12 tid 100009 td 0xc7e25000 sched_switch(c7e25000,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7d7d6f0,...) at mi_switch+0x243 ithread_loop(c7d77740,c7b38d08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d77740,c7b38d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b38d40, ebp = 0 --- Tracing command intr pid 12 tid 100008 td 0xc7e252f0 sched_switch(c7e252f0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7d7d770,...) at mi_switch+0x243 ithread_loop(c7d77750,c7b35d08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d77750,c7b35d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b35d40, ebp = 0 --- Tracing command intr pid 12 tid 100007 td 0xc7e255e0 cpustop_handler(0,c7b32b3c,c0e1c8d3,6b1700,c7e255e0,...) at cpustop_handler+0x26 ipi_nmi_handler(6b1700,c7e255e0,c1825cb4,c1825c80,c7de0598,...) at ipi_nmi_handler+0x35 trap(c7b32b48) at trap+0x33 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc054d6ea, esp = 0xc7b32b88, ebp = 0xc7b32b98 --- acpi_timer_get_timecount_safe(c1068a60,246,c11b1db0,c7b32bd0,c0a8b7f2,...) at acpi_timer_get_timecount_safe+0x1a binuptime(c7b32c58,0,c0ff40ce,1e7,c11be9c0,...) at binuptime+0x3d softclock(c11b1da0,c11be9c0,4,c0ff09b7,c7d7da38,...) at softclock+0x221 intr_event_execute_handlers(c7de0598,c7d7da00,c0feccb3,563,c7d7da70,...) at intr_event_execute_handlers+0x125 ithread_loop(c7d77760,c7b32d08,c0fec990,3db,c7de0598,...) at ithread_loop+0xac fork_exit(c0a71d20,c7d77760,c7b32d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b32d40, ebp = 0 --- Tracing command idle pid 11 tid 100006 td 0xc7de2000 sched_switch(c7de2000,0,108,1ae,c0a8b7f2,...) at sched_switch+0x458 mi_switch(108,0,c0ff4bec,3e6,c7de2000,...) at mi_switch+0x243 sched_idletd(0,c7b2ed08,c0fec990,3db,c7de0864,...) at sched_idletd+0x19c fork_exit(c0ac7db0,0,c7b2ed08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b2ed40, ebp = 0 --- Tracing command idle pid 11 tid 100005 td 0xc7de22f0 sched_switch(c7de22f0,0,108,1ae,c0a8b7f2,...) at sched_switch+0x458 mi_switch(108,0,c0ff4bec,3e6,c7de22f0,...) at mi_switch+0x243 sched_idletd(0,c7b2bd08,c0fec990,3db,c7de0864,...) at sched_idletd+0x19c fork_exit(c0ac7db0,0,c7b2bd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b2bd40, ebp = 0 --- Tracing command idle pid 11 tid 100004 td 0xc7de25e0 sched_switch(c7de25e0,0,608,1ae,0,...) at sched_switch+0x458 mi_switch(608,0,c0ff319f,cd,0,...) at mi_switch+0x243 critical_exit(0,2710,c11bf040,2,c7b28cc8,...) at critical_exit+0xb8 cpu_idle(0,c7b28ca0,c0ff4bec,3e8,c7de25e0,...) at cpu_idle+0x97 sched_idletd(0,c7b28d08,c0fec990,3db,c7de0864,...) at sched_idletd+0x263 fork_exit(c0ac7db0,0,c7b28d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b28d40, ebp = 0 --- Tracing command idle pid 11 tid 100003 td 0xc7de28d0 sched_switch(c7de28d0,0,608,1ae,0,...) at sched_switch+0x458 mi_switch(608,0,c0ff319f,cd,0,...) at mi_switch+0x243 critical_exit(0,2710,c11be9c0,2,c7b25cc8,...) at critical_exit+0xb8 cpu_idle(0,c7b25ca0,c0ff4bec,a39,c7de28d0,...) at cpu_idle+0x97 sched_idletd(0,c7b25d08,c0fec990,3db,c7de0864,...) at sched_idletd+0x263 fork_exit(c0ac7db0,0,c7b25d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b25d40, ebp = 0 --- Tracing command init pid 1 tid 100002 td 0xc7de2bc0 sched_switch(c7de2bc0,0,104,1b3,c7b21ab4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11be9c0,...) at mi_switch+0x243 sleepq_switch(c7de2bc0,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(c7de0b30,6c,c0ffb5a4,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7de0b30,c7de0bb8,16c,c0ffb5a4,0,...) at _sleep+0x384 kern_wait(c7de2bc0,ffffffff,c7b21c38,0,0,...) at kern_wait+0x597 sys_wait4(c7de2bc0,c7b21ccc,c1046c7c,c0ffb9fa,c7b21cc4,...) at sys_wait4+0x3b syscall(c7b21d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x805ff2f, esp = 0xbfbfd8ec, ebp = 0xbfbfd908 --- Tracing command audit pid 10 tid 100001 td 0xc7de3000 sched_switch(c7de3000,0,104,1b3,c7b1ebf0,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c7de3000,0,c0ff8600,26e,c7de3000,...) at sleepq_switch+0x1a7 sleepq_wait(c132f3c0,0,c7b1ec6c,1,0,...) at sleepq_wait+0x6b _cv_wait(c132f3c0,c132f3a4,c1024bd7,194,c7b1eca4,...) at _cv_wait+0x263 audit_worker(0,c7b1ed08,c0fec990,3db,c7de1000,...) at audit_worker+0x84 fork_exit(c0caf950,0,c7b1ed08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b1ed40, ebp = 0 --- Tracing command kernel pid 0 tid 100069 td 0xc9616000 sched_switch(c9616000,0,104,1b3,ee196bd0,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,352,...) at mi_switch+0x243 sleepq_switch(c9616000,0,c0ff8600,291,2,...) at sleepq_switch+0x1a7 sleepq_timedwait(c11b0b64,0,c0fdec52,2,0,...) at sleepq_timedwait+0x6b _sleep(c11b0b64,0,0,c0fdec52,bb8) at _sleep+0x369 pause(c0fdec52,bb8,12b,129,c11ade48,...) at pause+0xa0 deadlkres(0,ee196d08,c0fec990,3db,c11addc0,...) at deadlkres+0x324 fork_exit(c0a50920,0,ee196d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee196d40, ebp = 0 --- Tracing command kernel pid 0 tid 100068 td 0xc96162f0 sched_switch(c96162f0,0,104,1b3,ee193c1c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c0ff8600,...) at mi_switch+0x243 sleepq_switch(c96162f0,0,c0ff8600,26e,c96162f0,...) at sleepq_switch+0x1a7 sleepq_wait(c7d9a500,0,c0ff36b6,c0fdec52,0,...) at sleepq_wait+0x6b msleep_spin(c7d9a500,c7d9a518,c0fdec52,0,a6f,...) at msleep_spin+0x252 taskqueue_thread_loop(c1391520,ee193d08,c0fec990,3db,c11addc0,...) at taskqueue_thread_loop+0x8f fork_exit(c0ae4e00,c1391520,ee193d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee193d40, ebp = 0 --- Tracing command kernel pid 0 tid 100062 td 0xc94ea5e0 sched_switch(c94ea5e0,0,104,1b3,ee177c1c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c0ff8600,...) at mi_switch+0x243 sleepq_switch(c94ea5e0,0,c0ff8600,26e,c94ea5e0,...) at sleepq_switch+0x1a7 sleepq_wait(c7d9a740,0,c0ff36b6,c0fdec52,0,...) at sleepq_wait+0x6b msleep_spin(c7d9a740,c7d9a758,c0fdec52,0,a6f,...) at msleep_spin+0x252 taskqueue_thread_loop(c952649c,ee177d08,c0fec990,3db,c11addc0,...) at taskqueue_thread_loop+0x8f fork_exit(c0ae4e00,c952649c,ee177d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee177d40, ebp = 0 --- Tracing command kernel pid 0 tid 100027 td 0xc7eeabc0 sched_switch(c7eeabc0,0,104,1b3,c7b6fc04,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c7db9880,...) at mi_switch+0x243 sleepq_switch(c7eeabc0,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(c7d9a880,0,c0fdec52,0,0,...) at sleepq_wait+0x6b _sleep(c7d9a880,c7d9a898,0,c0fdec52,0,...) at _sleep+0x39b taskqueue_thread_loop(c132f9a4,c7b6fd08,c0fec990,3db,c11addc0,...) at taskqueue_thread_loop+0xb5 fork_exit(c0ae4e00,c132f9a4,c7b6fd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b6fd40, ebp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xc7fd42f0 sched_switch(c7fd42f0,0,104,1b3,c7b69c04,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c7db9800,...) at mi_switch+0x243 sleepq_switch(c7fd42f0,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(c7d9a900,0,c0fdec52,0,0,...) at sleepq_wait+0x6b _sleep(c7d9a900,c7d9a918,0,c0fdec52,0,...) at _sleep+0x39b taskqueue_thread_loop(c11ae758,c7b69d08,c0fec990,3db,c11addc0,...) at taskqueue_thread_loop+0xb5 fork_exit(c0ae4e00,c11ae758,c7b69d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b69d40, ebp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xc7fd48d0 sched_switch(c7fd48d0,0,104,1b3,c7b63c1c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c0ff8600,...) at mi_switch+0x243 sleepq_switch(c7fd48d0,0,c0ff8600,26e,c7fd48d0,...) at sleepq_switch+0x1a7 sleepq_wait(c7d9a940,0,c0ff36b6,c0fdec52,0,...) at sleepq_wait+0x6b msleep_spin(c7d9a940,c7d9a958,c0fdec52,0,a6f,...) at msleep_spin+0x252 taskqueue_thread_loop(c1177898,c7b63d08,c0fec990,3db,c11addc0,...) at taskqueue_thread_loop+0x8f fork_exit(c0ae4e00,c1177898,c7b63d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b63d40, ebp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xc7fd4bc0 sched_switch(c7fd4bc0,0,104,1b3,c7b60c1c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c0ff8600,...) at mi_switch+0x243 sleepq_switch(c7fd4bc0,0,c0ff8600,26e,c7fd4bc0,...) at sleepq_switch+0x1a7 sleepq_wait(c7d9a940,0,c0ff36b6,c0fdec52,0,...) at sleepq_wait+0x6b msleep_spin(c7d9a940,c7d9a958,c0fdec52,0,a6f,...) at msleep_spin+0x252 taskqueue_thread_loop(c1177898,c7b60d08,c0fec990,3db,c11addc0,...) at taskqueue_thread_loop+0x8f fork_exit(c0ae4e00,c1177898,c7b60d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b60d40, ebp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xc7e258d0 sched_switch(c7e258d0,0,104,1b3,c7b5dc1c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c0ff8600,...) at mi_switch+0x243 sleepq_switch(c7e258d0,0,c0ff8600,26e,c7e258d0,...) at sleepq_switch+0x1a7 sleepq_wait(c7d9a940,0,c0ff36b6,c0fdec52,0,...) at sleepq_wait+0x6b msleep_spin(c7d9a940,c7d9a958,c0fdec52,0,a6f,...) at msleep_spin+0x252 taskqueue_thread_loop(c1177898,c7b5dd08,c0fec990,3db,c11addc0,...) at taskqueue_thread_loop+0x8f fork_exit(c0ae4e00,c1177898,c7b5dd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b5dd40, ebp = 0 --- Tracing command kernel pid 0 tid 100018 td 0xc7ee92f0 sched_switch(c7ee92f0,0,104,1b3,c7b54c04,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c7db9a80,...) at mi_switch+0x243 sleepq_switch(c7ee92f0,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(c7d9aa00,0,c0fdec52,0,0,...) at sleepq_wait+0x6b _sleep(c7d9aa00,c7d9aa18,0,c0fdec52,0,...) at _sleep+0x39b taskqueue_thread_loop(c11cd76c,c7b54d08,c0fec990,3db,c11addc0,...) at taskqueue_thread_loop+0xb5 fork_exit(c0ae4e00,c11cd76c,c7b54d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b54d40, ebp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xc7ee98d0 sched_switch(c7ee98d0,0,104,1b3,c7b4dc04,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c7db9a00,...) at mi_switch+0x243 sleepq_switch(c7ee98d0,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(c7d9ab40,0,c0fdec52,0,0,...) at sleepq_wait+0x6b _sleep(c7d9ab40,c7d9ab58,0,c0fdec52,0,...) at _sleep+0x39b taskqueue_thread_loop(c11cc160,c7b4dd08,c0fec990,3db,c11addc0,...) at taskqueue_thread_loop+0xb5 fork_exit(c0ae4e00,c11cc160,c7b4dd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b4dd40, ebp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xc11ae090 sched_switch(c11ae090,0,104,1b3,c1820c64,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,352,...) at mi_switch+0x243 sleepq_switch(c11ae090,0,c0ff8600,291,0,...) at sleepq_switch+0x1a7 sleepq_timedwait(c11addc0,54,c10255b8,0,0,...) at sleepq_timedwait+0x6b _sleep(c11addc0,0,54,c10255b8,2710,...) at _sleep+0x369 scheduler(0,181ec00,181ec00,181e000,1825000,...) at scheduler+0x243 mi_startup() at mi_startup+0xac begin() at begin+0x2c db:0:allt> call doadump Physical memory: 3435 MB Dumping 312 MB: 297 281 265 249 233 217 201 185 169 153 137 121 105 89 73 57 41 25 9 Dump complete = 0 db:0:doadump> reset cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 3 (kgdb) bt #0 doadump (textdump=0xf29f1340) at pcpu.h:249 #1 0xc0525a79 in db_fncall (dummy1=0xc0a8c320, dummy2=0x0, dummy3=0xffffffff, dummy4=0xf29f13d4 "è\023\237ò") at ../../../ddb/db_command.c:573 #2 0xc0525eaf in db_command (last_cmdp=0xc1175a5c, cmd_table=0x0, dopager=0x0) at ../../../ddb/db_command.c:449 #3 0xc0525f64 in db_command_script (command=0xc1176968 "call doadump") at ../../../ddb/db_command.c:520 #4 0xc052a3b0 in db_script_exec (scriptname=0xc11762c0 "doadump", warnifnotfound=Variable "warnifnotfound" is not available. ) at ../../../ddb/db_script.c:302 #5 0xc052a441 in db_run_cmd (addr=0x1, have_addr=0x0, count=0xc1351700, modif=0xf29f150c "") at ../../../ddb/db_script.c:375 #6 0xc0525e71 in db_command (last_cmdp=0xc1175a5c, cmd_table=0x0, dopager=0x1) at ../../../ddb/db_command.c:449 #7 0xc0525fca in db_command_loop () at ../../../ddb/db_command.c:502 #8 0xc0527fbd in db_trap (type=0xc, code=0x0) at ../../../ddb/db_main.c:231 #9 0xc0ad6926 in kdb_trap (type=0xc, code=0x0, tf=0xf29f17a8) at ../../../kern/subr_kdb.c:654 #10 0xc0e1bc3f in trap_fatal (frame=0xf29f17a8, eva=0x10) at ../../../i386/i386/trap.c:1019 #11 0xc0e1bdb6 in trap_pfault (frame=0xf29f17a8, usermode=0x0, eva=0x10) at ../../../i386/i386/trap.c:881 #12 0xc0e1ceab in trap (frame=0xf29f17a8) at ../../../i386/i386/trap.c:552 #13 0xc0e060ac in calltrap () at ../../../i386/i386/exception.s:169 #14 0xc0a8c320 in _mtx_lock_flags (m=0x0, opts=0x0, file=0xc1002878 "../../../kern/vfs_subr.c", line=0xab1) at ../../../kern/kern_mutex.c:203 #15 0xc0b42e90 in vgonel (vp=0xcb74d5a0) at ../../../kern/vfs_subr.c:2737 #16 0xc0b4357c in vgone (vp=0xcb74d5a0) at ../../../kern/vfs_subr.c:2687 #17 0xcd84c9a9 in null_destroy_proto (vp=0xcb74d5a0, xp=0xcd944900) at /var/tmp/deviant2/sys/modules/nullfs/../../fs/nullfs/null_subr.c:180 #18 0xc0b4224f in insmntque1 (vp=0xcb74d5a0, mp=0xcd85f2ec, dtr=0xcd84ccd0 , dtr_arg=0xcd944900) at ../../../kern/vfs_subr.c:1121 #19 0xcd84cb38 in null_nodeget (mp=0xcd85f2ec, lowervp=0xcb6d39d8, vpp=0xf29f1954) at /var/tmp/deviant2/sys/modules/nullfs/../../fs/nullfs/null_subr.c:260 #20 0xcd84e0f3 in null_lookup (ap=0xf29f19ec) at /var/tmp/deviant2/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:367 #21 0xc0e450b8 in VOP_LOOKUP_APV (vop=0xcd84fa80, a=0xf29f19ec) at vnode_if.c:123 #22 0xc0b32f8b in lookup (ndp=0xf29f1aac) at vnode_if.h:54 #23 0xc0b33fef in namei (ndp=0xf29f1aac) at ../../../kern/vfs_lookup.c:307 #24 0xc0b4716f in kern_statat_vnhook (td=0xcd8ee000, flag=0x200, fd=0xffffff9c, path=0x2882ac98
, pathseg=UIO_USERSPACE, sbp=0xf29f1bdc, hook=0) at ../../../kern/vfs_syscalls.c:2406 #25 0xc0b4736c in kern_statat (td=0xcd8ee000, flag=0x200, fd=0xffffff9c, path=0x2882ac98
, pathseg=UIO_USERSPACE, sbp=0xf29f1bdc) at ../../../kern/vfs_syscalls.c:2387 #26 0xc0b473a6 in kern_lstat (td=0xcd8ee000, path=0x2882ac98
, pathseg=UIO_USERSPACE, sbp=0xf29f1bdc) at ../../../kern/vfs_syscalls.c:2460 #27 0xc0b4744f in sys_lstat (td=0xcd8ee000, uap=0xf29f1ccc) at ../../../kern/vfs_syscalls.c:2450 #28 0xc0e1c35e in syscall (frame=0xf29f1d08) at subr_syscall.c:135 #29 0xc0e06141 in Xint0x80_syscall () at ../../../i386/i386/exception.s:267 #30 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) f 17 #17 0xcd84c9a9 in null_destroy_proto (vp=0xcb74d5a0, xp=0xcd944900) at /var/tmp/deviant2/sys/modules/nullfs/../../fs/nullfs/null_subr.c:180 180 vgone(vp); (kgdb) l 175 VI_LOCK(vp); 176 vp->v_data = NULL; 177 vp->v_vnlock = &vp->v_lock; 178 vp->v_op = &dead_vnodeops; 179 VI_UNLOCK(vp); 180 vgone(vp); 181 vput(vp); 182 free(xp, M_NULLFSNODE); 183 } 184 (kgdb) p *vp $1 = {v_type = VSOCK, v_tag = 0xcd84e62e "null", v_op = 0xc1119f00, v_data = 0x0, v_mount = 0x0, v_nmntvnodes = {tqe_next = 0x0, tqe_prev = 0x0}, v_un = {vu_mount = 0x0, vu_socket = 0x0, vu_cdev = 0x0, vu_fifoinfo = 0x0}, v_hashlist = {le_next = 0x0, le_prev = 0x0}, v_hash = 0x0, v_cache_src = {lh_first = 0x0}, v_cache_dst = {tqh_first = 0x0, tqh_last = 0xcb74d5d0}, v_cache_dd = 0x0, v_cstart = 0x0, v_lasta = 0x0, v_lastw = 0x0, v_clen = 0x0, v_lock = {lock_object = {lo_name = 0xcd84e62e "null", lo_flags = 0x5730008, lo_data = 0x0, lo_witness = 0xc7d86f00}, lk_lock = 0xcd8ee000, lk_exslpfail = 0x0, lk_timo = 0x33, lk_pri = 0x60, lk_stack = {depth = 0xe, pcs = {0xc0a8399e, 0xcd84c94d, 0xc0b4224f, 0xcd84cb38, 0xcd84e0f3, 0xc0e450b8, 0xc0b32f8b, 0xc0b33fef, 0xc0b4716f, 0xc0b4736c, 0xc0b473a6, 0xc0b4744f, 0xc0e1c35e, 0xc0e06141, 0x0, 0x0, 0x0, 0x0}}}, v_interlock = {lock_object = {lo_name = 0xc0ffa8ee "vnode interlock", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc7d7f1e0}, mtx_lock = 0x4}, v_vnlock = 0xcb74d5f8, v_holdcnt = 0x1, v_usecount = 0x1, v_iflag = 0x80, v_vflag = 0x0, v_writecount = 0x0, v_actfreelist = {tqe_next = 0x0, tqe_prev = 0x0}, v_bufobj = {bo_mtx = {lock_object = { lo_name = 0xc1003871 "bufobj interlock", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc7d84d48}, mtx_lock = 0x4}, bo_clean = {bv_hd = {tqh_first = 0x0, tqh_last = 0xcb74d6ac}, bv_root = 0x0, bv_cnt = 0x0}, bo_dirty = {bv_hd = {tqh_first = 0x0, tqh_last = 0xcb74d6bc}, bv_root = 0x0, bv_cnt = 0x0}, bo_numoutput = 0x0, bo_flag = 0x0, bo_ops = 0xc1135120, bo_bsize = 0x4000, bo_object = 0x0, bo_synclist = {le_next = 0x0, le_prev = 0x0}, bo_private = 0xcb74d5a0, __bo_vnode = 0xcb74d5a0}, v_pollinfo = 0x0, v_label = 0x0, v_lockf = 0x0, v_rl = {rl_waiters = {tqh_first = 0x0, tqh_last = 0xcb74d6fc}, rl_currdep = 0x0}} (kgdb) p vp->v_mount $3 = (struct mount *) 0x0 (kgdb) (kgdb) f 19 #19 0xcd84cb38 in null_nodeget (mp=0xcd85f2ec, lowervp=0xcb6d39d8, vpp=0xf29f1954) at /var/tmp/deviant2/sys/modules/nullfs/../../fs/nullfs/null_subr.c:260 260 error = insmntque1(vp, mp, null_insmntque_dtr, xp); (kgdb) l 255 vp->v_type = lowervp->v_type; 256 vp->v_data = xp; 257 vp->v_vnlock = lowervp->v_vnlock; 258 if (vp->v_vnlock == NULL) 259 panic("null_nodeget: Passed a NULL vnlock.\n"); 260 error = insmntque1(vp, mp, null_insmntque_dtr, xp); 261 if (error != 0) 262 return (error); 263 /* 264 * Atomically insert our new node into the hash or vget existing (kgdb) p mp $4 = (struct mount *) 0xcd85f2ec (kgdb) p vp $5 = (struct vnode *) 0xcb74d5a0 (kgdb) git diff master diff --git a/sys/fs/nullfs/null.h b/sys/fs/nullfs/null.h index c8fc5de..0878e55 100644 --- a/sys/fs/nullfs/null.h +++ b/sys/fs/nullfs/null.h @@ -56,6 +56,7 @@ struct null_node { int nullfs_init(struct vfsconf *vfsp); int nullfs_uninit(struct vfsconf *vfsp); int null_nodeget(struct mount *mp, struct vnode *target, struct vnode **vpp); +struct vnode *null_hashget(struct mount *mp, struct vnode *lowervp); void null_hashrem(struct null_node *xp); int null_bypass(struct vop_generic_args *ap); diff --git a/sys/fs/nullfs/null_subr.c b/sys/fs/nullfs/null_subr.c index 7e3571d..5f926a6 100644 --- a/sys/fs/nullfs/null_subr.c +++ b/sys/fs/nullfs/null_subr.c @@ -67,7 +67,6 @@ struct mtx null_hashmtx; static MALLOC_DEFINE(M_NULLFSHASH, "nullfs_hash", "NULLFS hash table"); MALLOC_DEFINE(M_NULLFSNODE, "nullfs_node", "NULLFS vnode private part"); -static struct vnode * null_hashget(struct mount *, struct vnode *); static struct vnode * null_hashins(struct mount *, struct null_node *); /* @@ -98,7 +97,7 @@ nullfs_uninit(vfsp) * Return a VREF'ed alias for lower vnode if already exists, else 0. * Lower vnode should be locked on entry and will be left locked on exit. */ -static struct vnode * +struct vnode * null_hashget(mp, lowervp) struct mount *mp; struct vnode *lowervp; @@ -209,14 +208,10 @@ null_nodeget(mp, lowervp, vpp) struct vnode *vp; int error; - /* - * The insmntque1() call below requires the exclusive lock on - * the nullfs vnode. - */ - ASSERT_VOP_ELOCKED(lowervp, "lowervp"); - KASSERT(lowervp->v_usecount >= 1, ("Unreferenced vnode %p\n", lowervp)); + ASSERT_VOP_LOCKED(lowervp, "lowervp"); + KASSERT(lowervp->v_usecount >= 1, ("Unreferenced vnode %p", lowervp)); - /* Lookup the hash firstly */ + /* Lookup the hash firstly. */ *vpp = null_hashget(mp, lowervp); if (*vpp != NULL) { vrele(lowervp); @@ -224,6 +219,19 @@ null_nodeget(mp, lowervp, vpp) } /* + * The insmntque1() call below requires the exclusive lock on + * the nullfs vnode. Upgrade the lock now if hash failed to + * provide ready to use vnode. + */ + if (VOP_ISLOCKED(lowervp) != LK_EXCLUSIVE) { + vn_lock(lowervp, LK_UPGRADE | LK_RETRY); + if ((lowervp->v_iflag & VI_DOOMED) != 0) { + vput(lowervp); + return (ENOENT); + } + } + + /* * We do not serialize vnode creation, instead we will check for * duplicates later, when adding new vnode to hash. * Note that duplicate can only appear in hash if the lowervp is @@ -233,8 +241,7 @@ null_nodeget(mp, lowervp, vpp) * might cause a bogus v_data pointer to get dereferenced * elsewhere if MALLOC should block. */ - xp = malloc(sizeof(struct null_node), - M_NULLFSNODE, M_WAITOK); + xp = malloc(sizeof(struct null_node), M_NULLFSNODE, M_WAITOK); error = getnewvnode("null", mp, &null_vnodeops, &vp); if (error) { diff --git a/sys/fs/nullfs/null_vfsops.c b/sys/fs/nullfs/null_vfsops.c index 08f722a..84e5db7 100644 --- a/sys/fs/nullfs/null_vfsops.c +++ b/sys/fs/nullfs/null_vfsops.c @@ -65,6 +65,7 @@ static vfs_statfs_t nullfs_statfs; static vfs_unmount_t nullfs_unmount; static vfs_vget_t nullfs_vget; static vfs_extattrctl_t nullfs_extattrctl; +static vfs_reclaim_lowervp_t nullfs_reclaim_lowervp; /* * Mount null layer @@ -121,8 +122,10 @@ nullfs_mount(struct mount *mp) */ NDINIT(ndp, LOOKUP, FOLLOW|LOCKLEAF, UIO_SYSSPACE, target, curthread); error = namei(ndp); + /* * Re-lock vnode. + * XXXKIB This is deadlock-prone as well. */ if (isvnunlocked) vn_lock(mp->mnt_vnodecovered, LK_EXCLUSIVE | LK_RETRY); @@ -146,7 +149,7 @@ nullfs_mount(struct mount *mp) } xmp = (struct null_mount *) malloc(sizeof(struct null_mount), - M_NULLFSMNT, M_WAITOK); /* XXX */ + M_NULLFSMNT, M_WAITOK); /* * Save reference to underlying FS @@ -186,10 +189,14 @@ nullfs_mount(struct mount *mp) } MNT_ILOCK(mp); mp->mnt_kern_flag |= lowerrootvp->v_mount->mnt_kern_flag & - (MNTK_MPSAFE | MNTK_SHARED_WRITES); + (MNTK_MPSAFE | MNTK_SHARED_WRITES | MNTK_LOOKUP_SHARED | + MNTK_EXTENDED_SHARED); MNT_IUNLOCK(mp); mp->mnt_data = xmp; vfs_getnewfsid(mp); + MNT_ILOCK(xmp->nullm_vfs); + TAILQ_INSERT_TAIL(&xmp->nullm_vfs->mnt_uppers, mp, mnt_upper_link); + MNT_IUNLOCK(xmp->nullm_vfs); vfs_mountedfrom(mp, target); @@ -206,14 +213,16 @@ nullfs_unmount(mp, mntflags) struct mount *mp; int mntflags; { - void *mntdata; - int error; - int flags = 0; + struct null_mount *mntdata; + struct mount *ump; + int error, flags; NULLFSDEBUG("nullfs_unmount: mp = %p\n", (void *)mp); if (mntflags & MNT_FORCE) - flags |= FORCECLOSE; + flags = FORCECLOSE; + else + flags = 0; /* There is 1 extra root vnode reference (nullm_rootvp). */ error = vflush(mp, 1, flags, curthread); @@ -224,9 +233,17 @@ nullfs_unmount(mp, mntflags) * Finally, throw away the null_mount structure */ mntdata = mp->mnt_data; + ump = mntdata->nullm_vfs; + MNT_ILOCK(ump); + while ((ump->mnt_kern_flag & MNTK_VGONE_UPPER) != 0) { + ump->mnt_kern_flag |= MNTK_VGONE_WAITER; + msleep(&ump->mnt_uppers, &ump->mnt_mtx, 0, "vgnupw", 0); + } + TAILQ_REMOVE(&ump->mnt_uppers, mp, mnt_upper_link); + MNT_IUNLOCK(ump); mp->mnt_data = NULL; free(mntdata, M_NULLFSMNT); - return 0; + return (0); } static int @@ -316,13 +333,10 @@ nullfs_vget(mp, ino, flags, vpp) KASSERT((flags & LK_TYPE_MASK) != 0, ("nullfs_vget: no lock requested")); - flags &= ~LK_TYPE_MASK; - flags |= LK_EXCLUSIVE; error = VFS_VGET(MOUNTTONULLMOUNT(mp)->nullm_vfs, ino, flags, vpp); - if (error) + if (error != 0) return (error); - return (null_nodeget(mp, *vpp, vpp)); } @@ -334,11 +348,11 @@ nullfs_fhtovp(mp, fidp, flags, vpp) struct vnode **vpp; { int error; - error = VFS_FHTOVP(MOUNTTONULLMOUNT(mp)->nullm_vfs, fidp, LK_EXCLUSIVE, + + error = VFS_FHTOVP(MOUNTTONULLMOUNT(mp)->nullm_vfs, fidp, flags, vpp); - if (error) + if (error != 0) return (error); - return (null_nodeget(mp, *vpp, vpp)); } @@ -350,10 +364,22 @@ nullfs_extattrctl(mp, cmd, filename_vp, namespace, attrname) int namespace; const char *attrname; { - return VFS_EXTATTRCTL(MOUNTTONULLMOUNT(mp)->nullm_vfs, cmd, filename_vp, - namespace, attrname); + + return (VFS_EXTATTRCTL(MOUNTTONULLMOUNT(mp)->nullm_vfs, cmd, + filename_vp, namespace, attrname)); } +static void +nullfs_reclaim_lowervp(struct mount *mp, struct vnode *lowervp) +{ + struct vnode *vp; + + vp = null_hashget(mp, lowervp); + if (vp == NULL) + return; + vgone(vp); + vn_lock(lowervp, LK_EXCLUSIVE | LK_RETRY); +} static struct vfsops null_vfsops = { .vfs_extattrctl = nullfs_extattrctl, @@ -367,6 +393,7 @@ static struct vfsops null_vfsops = { .vfs_uninit = nullfs_uninit, .vfs_unmount = nullfs_unmount, .vfs_vget = nullfs_vget, + .vfs_reclaim_lowervp = nullfs_reclaim_lowervp, }; VFS_SET(null_vfsops, nullfs, VFCF_LOOPBACK | VFCF_JAIL); diff --git a/sys/fs/nullfs/null_vnops.c b/sys/fs/nullfs/null_vnops.c index 6f40233..5fb020b 100644 --- a/sys/fs/nullfs/null_vnops.c +++ b/sys/fs/nullfs/null_vnops.c @@ -665,33 +665,18 @@ null_unlock(struct vop_unlock_args *ap) } /* - * There is no way to tell that someone issued remove/rmdir operation - * on the underlying filesystem. For now we just have to release lowervp - * as soon as possible. - * - * Note, we can't release any resources nor remove vnode from hash before - * appropriate VXLOCK stuff is done because other process can find this - * vnode in hash during inactivation and may be sitting in vget() and waiting - * for null_inactive to unlock vnode. Thus we will do all those in VOP_RECLAIM. + * XXXKIB */ static int -null_inactive(struct vop_inactive_args *ap) +null_inactive(struct vop_inactive_args *ap __unused) { - struct vnode *vp = ap->a_vp; - - vp->v_object = NULL; - - /* - * If this is the last reference, then free up the vnode - * so as not to tie up the lower vnodes. - */ - vrecycle(vp); return (0); } /* - * Now, the VXLOCK is in force and we're free to destroy the null vnode. + * Now, the nullfs vnode and, due to the sharing lock, the lower + * vnode, are exclusively locked, and we shall destroy the null vnode. */ static int null_reclaim(struct vop_reclaim_args *ap) diff --git a/sys/kern/vfs_mount.c b/sys/kern/vfs_mount.c index e9d3abe..f397842 100644 --- a/sys/kern/vfs_mount.c +++ b/sys/kern/vfs_mount.c @@ -481,6 +481,7 @@ vfs_mount_alloc(struct vnode *vp, struct vfsconf *vfsp, const char *fspath, mac_mount_create(cred, mp); #endif arc4rand(&mp->mnt_hashseed, sizeof mp->mnt_hashseed, 0); + TAILQ_INIT(&mp->mnt_uppers); return (mp); } @@ -514,6 +515,7 @@ vfs_mount_destroy(struct mount *mp) vprint("", vp); panic("unmount: dangling vnode"); } + KASSERT(TAILQ_EMPTY(&mp->mnt_uppers), ("mnt_uppers")); if (mp->mnt_nvnodelistsize != 0) panic("vfs_mount_destroy: nonzero nvnodelistsize"); if (mp->mnt_activevnodelistsize != 0) @@ -1275,7 +1277,8 @@ dounmount(mp, flags, td) } MNT_ILOCK(mp); - if (mp->mnt_kern_flag & MNTK_UNMOUNT) { + if ((mp->mnt_kern_flag & MNTK_UNMOUNT) != 0 || + !TAILQ_EMPTY(&mp->mnt_uppers)) { MNT_IUNLOCK(mp); if (coveredvp) VOP_UNLOCK(coveredvp, 0); diff --git a/sys/kern/vfs_subr.c b/sys/kern/vfs_subr.c index 631d3f2..d212ddf 100644 --- a/sys/kern/vfs_subr.c +++ b/sys/kern/vfs_subr.c @@ -2688,6 +2688,12 @@ vgone(struct vnode *vp) VI_UNLOCK(vp); } +static void +vgonel_reclaim_lowervp(struct mount *mp __unused, + struct vnode *lowervp __unused) +{ +} + /* * vgone, with the vp interlock held. */ @@ -2697,7 +2703,10 @@ vgonel(struct vnode *vp) struct thread *td; int oweinact; int active; - struct mount *mp; + struct mount *mp, *ump, *mmp; + static struct vfsops vgonel_vfsops = { + .vfs_reclaim_lowervp = vgonel_reclaim_lowervp + }; ASSERT_VOP_ELOCKED(vp, "vgonel"); ASSERT_VI_LOCKED(vp, "vgonel"); @@ -2712,6 +2721,7 @@ vgonel(struct vnode *vp) if (vp->v_iflag & VI_DOOMED) return; vp->v_iflag |= VI_DOOMED; + /* * Check to see if the vnode is in use. If so, we have to call * VOP_CLOSE() and VOP_INACTIVE(). @@ -2719,6 +2729,37 @@ vgonel(struct vnode *vp) active = vp->v_usecount; oweinact = (vp->v_iflag & VI_OWEINACT); VI_UNLOCK(vp); + + /* + * Notify upper mounts about reclaimed vnode. + */ + mp = vp->v_mount; + MNT_ILOCK(mp); + if (!TAILQ_EMPTY(&mp->mnt_uppers)) { + MNT_IUNLOCK(mp); + mmp = malloc(sizeof(struct mount), M_TEMP, M_WAITOK | + M_ZERO); + mmp->mnt_op = &vgonel_vfsops; + MNT_ILOCK(mp); + mp->mnt_kern_flag |= MNTK_VGONE_UPPER; + for (ump = TAILQ_FIRST(&mp->mnt_uppers); ump != NULL;) { + TAILQ_INSERT_AFTER(&mp->mnt_uppers, ump, mmp, + mnt_upper_link); + MNT_IUNLOCK(mp); + VFS_RECLAIM_LOWERVP(ump, vp); + MNT_ILOCK(mp); + ump = TAILQ_NEXT(mmp, mnt_upper_link); + TAILQ_REMOVE(&mp->mnt_uppers, mmp, mnt_upper_link); + } + free(mmp, M_TEMP); + mp->mnt_kern_flag &= ~MNTK_VGONE_UPPER; + if ((mp->mnt_kern_flag & MNTK_VGONE_WAITER) != 0) { + mp->mnt_kern_flag &= ~MNTK_VGONE_WAITER; + wakeup(&mp->mnt_uppers); + } + } + MNT_IUNLOCK(mp); + /* * Clean out any buffers associated with the vnode. * If the flush fails, just toss the buffers. diff --git a/sys/sys/mount.h b/sys/sys/mount.h index 4b83ba4..8e65e1c 100644 --- a/sys/sys/mount.h +++ b/sys/sys/mount.h @@ -188,6 +188,8 @@ struct mount { #define mnt_endzero mnt_gjprovider char *mnt_gjprovider; /* gjournal provider name */ struct lock mnt_explock; /* vfs_export walkers lock */ + TAILQ_ENTRY(mount) mnt_upper_link; /* (m) we in the all uppers */ + TAILQ_HEAD(, mount) mnt_uppers; /* (m) upper mounts over us*/ }; /* @@ -373,6 +375,8 @@ void __mnt_vnode_markerfree(struct vnode **mvp, struct mount *mp); #define MNTK_NO_IOPF 0x00000100 /* Disallow page faults during reads and writes. Filesystem shall properly handle i/o state on EFAULT. */ +#define MNTK_VGONE_UPPER 0x00000200 +#define MNTK_VGONE_WAITER 0x00000200 #define MNTK_NOASYNC 0x00800000 /* disable async */ #define MNTK_UNMOUNT 0x01000000 /* unmount in progress */ #define MNTK_MWAIT 0x02000000 /* waiting for unmount to finish */ @@ -628,6 +632,7 @@ typedef int vfs_mount_t(struct mount *mp); typedef int vfs_sysctl_t(struct mount *mp, fsctlop_t op, struct sysctl_req *req); typedef void vfs_susp_clean_t(struct mount *mp); +typedef void vfs_reclaim_lowervp_t(struct mount *mp, struct vnode *lowervp); struct vfsops { vfs_mount_t *vfs_mount; @@ -645,6 +650,7 @@ struct vfsops { vfs_extattrctl_t *vfs_extattrctl; vfs_sysctl_t *vfs_sysctl; vfs_susp_clean_t *vfs_susp_clean; + vfs_reclaim_lowervp_t *vfs_reclaim_lowervp; }; vfs_statfs_t __vfs_statfs; @@ -670,6 +676,9 @@ vfs_statfs_t __vfs_statfs; #define VFS_SUSP_CLEAN(MP) \ ({if (*(MP)->mnt_op->vfs_susp_clean != NULL) \ (*(MP)->mnt_op->vfs_susp_clean)(MP); }) +#define VFS_RECLAIM_LOWERVP(MP, VP) \ + ({if (*(MP)->mnt_op->vfs_reclaim_lowervp != NULL) \ + (*(MP)->mnt_op->vfs_reclaim_lowervp)((MP), (VP)); }) #define VFS_NEEDSGIANT_(MP) \ ((MP) != NULL && ((MP)->mnt_kern_flag & MNTK_MPSAFE) == 0) -- Test scenario: nullfs.sh