GENERIC from Fri Mar 1 15:24:08 2013 +0200, r247443+vm1 b31f59d, vmcore.548 GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb 4718592K of memory above 4GB ignored Copyright (c) 1992-2013 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #0 r247443+b31f59d: Fri Mar 1 15:43:35 CET 2013 pho@x4.osted.lan:/var/tmp/deviant2/sys/i386/compile/PHO i386 FreeBSD clang version 3.2 (tags/RELEASE_32/final 170710) 20121221 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. CPU: AMD Phenom(tm) 9150e Quad-Core Processor (1800.02-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x100f23 Family = 0x10 Model = 0x2 Stepping = 3 Features=0x178bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x7ff TSC: P-state invariant real memory = 8589934592 (8192 MB) avail memory = 3531464704 (3367 MB) : Trying to mount root from ufs:/dev/ad4s1a [rw]... Setting hostuuid: 00000000-0000-0000-0000-00218515337d. Setting hostid: 0x6b64ac17. Starting ddb. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: /dev/ad4s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1a: clean, 627152 free (1200 frags, 78244 blocks, 0.1% fragmentation) /dev/ad4s1f: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1f: clean, 113501 free (5245 frags, 13532 blocks, 0.5% fragmentation) /dev/label/tmp: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/label/tmp: clean, 42350857 free (1353 frags, 5293688 blocks, 0.0% fragmentation) /dev/ad4s1d: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1d: clean, 784195 free (121251 frags, 82868 blocks, 1.2% fragmentation) /dev/ad4s1e: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1e: clean, 9364583 free (64463 frags, 1162515 blocks, 0.3% fragmentation) Mounting local file systems:. Setting hostname: x4.osted.lan. re0: link state changed to DOWN re0: link state changed to UP Starting Network: lo0 re0. lo0: flags=8049 metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 nd6 options=21 re0: flags=8843 metric 0 mtu 1500 options=8209b ether 00:21:85:15:33:7d inet 192.168.1.101 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::221:85ff:fe15:337d%re0 prefixlen 64 scopeid 0x1 nd6 options=29 media: Ethernet autoselect (100baseTX ) status: active Starting devd. add net default: gateway 192.168.1.1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/kde4/lib /usr/local/lib/compat /usr/local/lib/compat/pkg /usr/local/lib/gcc46 /usr/local/lib/mysql /usr/local/lib/nss /usr/local/lib/pth /usr/local/lib/qt4 /usr/local/lib/virtualbox a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout Creating and/or trimming log files. Starting syslogd. savecore: couldn't find media and/or sector size of /var/crash: Inappropriate ioctl for device Mar 1 20:25:27 x4 savecore: couldn't find media and/or sector size of /var/crash: Inappropriate ioctl for device No core dumps found. Additional ABI support: linux. Starting rpcbind. NFS access cache time=60 lock order reversal: 1st 0xe191e08c bufwait (bufwait) @ kern/vfs_bio.c:3029 2nd 0xca746c00 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:284 KDB: stack backtrace: db_trace_self_wrapper(c10896a4,c1092ce2,bf3,c0b185e0,cab453a0,...) at db_trace_self_wrapper+0x2d/frame 0xf96a77a8 kdb_backtrace(c108c7c2,ca746c00,c10be55d,c7d86540,c10be1a2,...) at kdb_backtrace+0x30/frame 0xf96a7810 witness_checkorder(ca746c00,9,c10be199,11c,0,...) at witness_checkorder+0xbad/frame 0xf96a7860 _sx_xlock(ca746c00,0,c10be199,11c,ca6dd000,...) at _sx_xlock+0x75/frame 0xf96a7890 ufsdirhash_add(ca73e3a0,f96a7988,1168,f96a7910,f96a7914,...) at ufsdirhash_add+0x4a/frame 0xf96a78c0 ufs_direnter(cab452d0,cad9db40,f96a7988,f96a7b68,e1921ce0,...) at ufs_direnter+0x63b/frame 0xf96a7940 ufs_mkdir(f96a7c00,c10e3579,c10b7e76,cab452d0,f96a7bfc,...) at ufs_mkdir+0x884/frame 0xf96a7ae0 VOP_MKDIR_APV(c11e2de8,f96a7c00,f96a7b68,f96a7b90,28803020,...) at VOP_MKDIR_APV+0x124/frame 0xf96a7b10 kern_mkdirat(ca7f82f0,ffffff9c,28803020,0,1c0) at kern_mkdirat+0x1ef/frame 0xf96a7c24 sys_mkdir(ca7f82f0,f96a7cc8,f96a7d08,1,1,...) at sys_mkdir+0x31/frame 0xf96a7c40 syscall(f96a7d08) at syscall+0x2ed/frame 0xf96a7cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf96a7cfc --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28173f03, esp = 0xbfbfd92c, ebp = 0xbfbfddd0 --- Clearing /tmp (X related). Starting mountd. Starting nfsuserd. Starting nfsd. Recovering vi editor sessions:lock order reversal: 1st 0xcaf31034 ufs (ufs) @ kern/vfs_lookup.c:516 2nd 0xe1926714 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:261 3rd 0xcaf30cdc ufs (ufs) @ kern/vfs_subr.c:2176 KDB: stack backtrace: db_trace_self_wrapper(c10896a4,0,0,0,0,...) at db_trace_self_wrapper+0x2d/frame 0xf96ab2b8 kdb_backtrace(c108c7db,caf30cdc,c1074cec,c7d864d8,c1095b80,...) at kdb_backtrace+0x30/frame 0xf96ab320 witness_checkorder(caf30cdc,9,c1095b77,880,caf30d48,...) at witness_checkorder+0xbad/frame 0xf96ab370 __lockmgr_args(caf30cdc,80100,caf30d48,0,0,...) at __lockmgr_args+0x87c/frame 0xf96ab450 ffs_lock(f96ab4d8,c137c148,c7d7f1e0,c7d84a70,c7d7f1e0,...) at ffs_lock+0xad/frame 0xf96ab490 VOP_LOCK1_APV(c11e2de8,f96ab4d8,200246,c13a7f00,c11f75c8,...) at VOP_LOCK1_APV+0x112/frame 0xf96ab4c0 _vn_lock(caf30ca8,80100,c1095b77,880,c1094ded,...) at _vn_lock+0xca/frame 0xf96ab500 vget(caf30ca8,80100,ca7f8000,57,0,...) at vget+0x77/frame 0xf96ab538 vfs_hash_get(ca706bb0,15eeb0,80000,ca7f8000,f96ab618,...) at vfs_hash_get+0xff/frame 0xf96ab564 ffs_vgetf(ca706bb0,15eeb0,80000,f96ab618,1,...) at ffs_vgetf+0x47/frame 0xf96ab5c0 softdep_sync_buf(caf31000,e19266b4,1,107,0,...) at softdep_sync_buf+0x977/frame 0xf96ab628 ffs_syncvnode(caf31000,1,0,f96ab6e4,c0ae0c3f,...) at ffs_syncvnode+0x2a3/frame 0xf96ab680 ffs_truncate(caf31000,200,0,880,ca4b4a00,...) at ffs_truncate+0x6c8/frame 0xf96ab838 ufs_direnter(caf31000,caf30ca8,f96ab8f8,f96abbc8,0,...) at ufs_direnter+0x7ed/frame 0xf96ab8b8 ufs_makeinode(f96abbb4,f96abbc8) at ufs_makeinode+0x537/frame 0xf96aba2c ufs_create(f96abb18,c10e2f67,c10b7e76,ca706bc0,2,...) at ufs_create+0x2f/frame 0xf96aba40 VOP_CREATE_APV(c11e2de8,f96abb18,f96abbc8,f96abaa8,c11f7738,...) at VOP_CREATE_APV+0x124/frame 0xf96aba70 vn_open_cred(f96abb88,f96abbfc,1b0,0,ca4b4a00,ca7ad5b0) at vn_open_cred+0x2cf/frame 0xf96abb40 vn_open(f96abb88,f96abbfc,1b0,ca7ad5b0,0,...) at vn_open+0x3b/frame 0xf96abb60 kern_openat(ca7f8000,ffffff9c,28c57f60,0,a02,1b0) at kern_openat+0x1e4/frame 0xf96abc20 sys_open(ca7f8000,f96abcc8,f96abd08,1,1,...) at sys_open+0x38/frame 0xf96abc40 syscall(f96abd08) at syscall+0x2ed/frame 0xf96abcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf96abcfc --- syscall (5, FreeBSD ELF32, sys_open), eip = 0x28414197, esp = 0xbfbfb50c, ebp = 0xbfbfb598 --- . Updating motd:. Starting ntpd. Removing stale Samba tdb files: ....... done Starting nmbd. Starting smbd. Starting winbindd. Starting default moused. Configuring syscons: keymap blanktime. Starting sshd. Starting cron. Local package initialization: backuplock order reversal: 1st 0xca764cdc ufs (ufs) @ kern/vfs_mount.c:1236 2nd 0xca7c7e44 devfs (devfs) @ ufs/ffs/ffs_softdep.c:1866 KDB: stack backtrace: db_trace_self_wrapper(c10896a4,0,33,c1074cec,677,...) at db_trace_self_wrapper+0x2d/frame 0xf96af848 kdb_backtrace(c108c7c2,ca7c7e44,c108080d,c7d86338,c10ba887,...) at kdb_backtrace+0x30/frame 0xf96af8b0 witness_checkorder(ca7c7e44,9,c10ba87e,74a,ca7c7eb0,...) at witness_checkorder+0xbad/frame 0xf96af900 __lockmgr_args(ca7c7e44,80400,ca7c7eb0,0,0,0,c10ba87e,74a) at __lockmgr_args+0x87c/frame 0xf96af9e0 vop_stdlock(f96afa58,632,4,f96afa54,c0a809f7,...) at vop_stdlock+0x53/frame 0xf96afa10 VOP_LOCK1_APV(c11a8a30,f96afa58,4,f96afa80,c11f75c8,...) at VOP_LOCK1_APV+0x112/frame 0xf96afa40 _vn_lock(ca7c7e10,80400,c10ba87e,74a,f96afac0,...) at _vn_lock+0xca/frame 0xf96afa80 softdep_flushworklist(cab422ec,f96afb14,ca7935e0,62d,0,...) at softdep_flushworklist+0x6e/frame 0xf96afac0 ffs_sync(cab422ec,1,f96afb70,50e,c7daa080,...) at ffs_sync+0x2e5/frame 0xf96afb48 dounmount(cab422ec,8080000,ca7935e0,493,0,...) at dounmount+0x3d2/frame 0xf96afb98 sys_unmount(ca7935e0,f96afcc8,c10dc7ab,d6,1,...) at sys_unmount+0x3dd/frame 0xf96afc40 syscall(f96afd08) at syscall+0x2ed/frame 0xf96afcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf96afcfc --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280c0d67, esp = 0xbfbfd6a4, ebp = 0xbfbfd770 --- ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 12359 files, 75963124 used, 42350857 free (1353 frags, 5293688 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes Mar 1 20:28:28 x4 su: pho to root on /dev/pts/0 ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 12359 files, 75963124 used, 42350857 free (1353 frags, 5293688 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** fsck -y /tmp watchdogd. Starting inetd. Fri Mar 1 20:29:59 CET 2013 FreeBSD/i386 (x4.osted.lan) (console) login: 20130301 20:30:04 all: graid3.sh GEOM_RAID3: Device raid3/data launched (3/3). lock order reversal: 1st 0xcab44e44 syncer (syncer) @ kern/vfs_subr.c:1795 2nd 0xcafb273c ufs (ufs) @ kern/vfs_subr.c:2176 KDB: stack backtrace: db_trace_self_wrapper(c10896a4,246,46,f,e1994088,...) at db_trace_self_wrapper+0x2d/frame 0xf419a938 kdb_backtrace(c108c7c2,cafb273c,c1074cec,c7d864d8,c1095b80,...) at kdb_backtrace+0x30/frame 0xf419a9a0 witness_checkorder(cafb273c,9,c1095b77,880,cafb27a8,...) at witness_checkorder+0xbad/frame 0xf419a9f0 __lockmgr_args(cafb273c,80500,cafb27a8,0,0,...) at __lockmgr_args+0x87c/frame 0xf419aad0 ffs_lock(f419ab58,c0a80772,c7d848d0,246,c7d848d0,...) at ffs_lock+0xad/frame 0xf419ab10 VOP_LOCK1_APV(c11e2de8,f419ab58,246,c13a8128,c11f75c8,...) at VOP_LOCK1_APV+0x112/frame 0xf419ab40 _vn_lock(cafb2708,80500,c1095b77,880,c13a8138,...) at _vn_lock+0xca/frame 0xf419ab80 vget(cafb2708,80500,c950d000,12bb,cab42010,...) at vget+0x77/frame 0xf419abb8 vfs_msync(cab42000,2,2,e7a,0,...) at vfs_msync+0xa0/frame 0xf419abfc sync_fsync(f419aca8,c10e3454,80400,c1095b77,703,...) at sync_fsync+0x130/frame 0xf419ac28 VOP_FSYNC_APV(c11c3198,f419aca8,c1095b77,703,4e20,...) at VOP_FSYNC_APV+0x124/frame 0xf419ac58 sched_sync(0,f419ad08,c107ef40,3d7,0,...) at sched_sync+0x38f/frame 0xf419accc fork_exit(c0b3bc40,0,f419ad08) at fork_exit+0x7f/frame 0xf419acf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf419acf4 --- trap 0, eip = 0, esp = 0xf419ad40, ebp = 0 --- GEOM_RAID3: Device data: provider raid3/data destroyed. GEOM_RAID3: Device data destroyed. 20130301 20:50:11 all: linger3.sh 20130301 20:50:24 all: mountro2.sh lock order reversal: 1st 0xcdeb60c0 bufobj interlock (bufobj interlock) @ ufs/ffs/ffs_vfsops.c:1611 2nd 0xca7052ec struct mount mtx (struct mount mtx) @ ufs/ffs/ffs_softdep.c:13455 KDB: stack backtrace: db_trace_self_wrapper(c10896a4,65647466,3a632e70,35343331,c7000a35,...) at db_trace_self_wrapper+0x2d/frame 0xf99da740 kdb_backtrace(c108c7c2,ca7052ec,c1071002,c7d7f5f0,c10ba887,...) at kdb_backtrace+0x30/frame 0xf99da7a8 witness_checkorder(ca7052ec,9,c10ba87e,348f,0,...) at witness_checkorder+0xbad/frame 0xf99da7f8 __mtx_lock_flags(ca7052fc,0,c10ba87e,348f,cdeb60c0,...) at __mtx_lock_flags+0x98/frame 0xf99da82c softdep_check_suspend(ca7052ec,cdeb6000,0,1faf,0,...) at softdep_check_suspend+0x130/frame 0xf99da858 ffs_sync(ca7052ec,4,c1097510,692,0,...) at ffs_sync+0x457/frame 0xf99da8dc vfs_write_suspend(ca7052ec,f99daa04,1,0,0,...) at vfs_write_suspend+0x12f/frame 0xf99da908 ffs_mount(ca7052ec,ca718d00,ff,3ab,0,...) at ffs_mount+0x6fb/frame 0xf99daa50 vfs_donmount(cc08ebc0,1,0,ca713000,ca713000,...) at vfs_donmount+0xeac/frame 0xf99dac18 sys_nmount(cc08ebc0,f99dacc8,f99dad08,1,1,...) at sys_nmount+0x78/frame 0xf99dac40 syscall(f99dad08) at syscall+0x2ed/frame 0xf99dacfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf99dacfc --- syscall (378, FreeBSD ELF32, sys_nmount), eip = 0x280d3367, esp = 0xbfbfc8e8, ebp = 0xbfbfce40 --- 20130301 20:50:26 all: nullfs.sh 20130301 20:53:35 all: suj10.sh 20130301 20:55:01 all: symlink2.sh 20130301 20:55:08 all: sem.sh 20130301 20:55:29 all: jail4.sh 20130301 20:55:29 all: quota4.sh lock order reversal: 1st 0xca724b48 FFS (FFS Lock) @ ufs/ufs/ufs_quota.c:714 2nd 0xcedb82ec struct mount mtx (struct mount mtx) @ ufs/ufs/ufs_quota.c:720 KDB: stack backtrace: db_trace_self_wrapper(c10896a4,a30,c1095b77,cedb82ec,cb28c8d0,...) at db_trace_self_wrapper+0x2d/frame 0xf9bf3948 kdb_backtrace(c108c7c2,cedb82ec,c1071002,c7d7f5f0,c10beca7,...) at kdb_backtrace+0x30/frame 0xf9bf39ac witness_checkorder(cedb82ec,9,c10bec9e,2d0,0,...) at witness_checkorder+0xbad/frame 0xf9bf39fc __mtx_lock_flags(cedb82fc,0,c10bec9e,2d0,ca724b58,...) at __mtx_lock_flags+0x98/frame 0xf9bf3a30 quotaoff_inchange(0,0,c10bec9e,2ec) at quotaoff_inchange+0x8c/frame 0xf9bf3a5c quotaoff(cb28c8d0,cedb82ec,0,cb28c8d0,ca724b00,...) at quotaoff+0xc0/frame 0xf9bf3a80 ffs_flushfiles(cedb82ec,a,cb28c8d0,ca724b1c,f9bf3b00,...) at ffs_flushfiles+0x7d/frame 0xf9bf3ac4 softdep_flushfiles(cedb82ec,2,cb28c8d0,0,0,...) at softdep_flushfiles+0x18f/frame 0xf9bf3b00 ffs_unmount(cedb82ec,8080000,f9bf3b70,50e,0,...) at ffs_unmount+0x1d5/frame 0xf9bf3b48 dounmount(cedb82ec,8080000,cb28c8d0,493,0,...) at dounmount+0x413/frame 0xf9bf3b98 sys_unmount(cb28c8d0,f9bf3cc8,f9bf3d08,1,1,...) at sys_unmount+0x3dd/frame 0xf9bf3c40 syscall(f9bf3d08) at syscall+0x2ed/frame 0xf9bf3cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9bf3cfc --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280c0d67, esp = 0xbfbfd124, ebp = 0xbfbfd1f0 --- 20130301 20:57:24 all: suj21.sh 20130301 20:58:49 all: mountu.sh 20130301 20:59:05 all: fpclone.sh 20130301 21:01:09 all: kevent3.sh 20130301 21:01:10 all: suj6.sh 20130301 21:02:16 all: tmpfs5.sh 20130301 21:02:37 all: zfs5.sh ZFS NOTICE: Prefetch is disabled by default on i386 -- to enable, add "vfs.zfs.prefetch_disable=0" to /boot/loader.conf. ZFS WARNING: Recommended minimum kmem_size is 512MB; expect unstable behavior. Consider tuning vm.kmem_size and vm.kmem_size_max in /boot/loader.conf. ZFS filesystem version: 5 ZFS storage pool version: features support (5000) 20130301 21:03:05 all: alternativeFlushPath.sh 20130301 21:33:10 all: syscall4.sh lock order reversal: 1st 0xcb1b9c2c filedesc structure (filedesc structure) @ kern/kern_descrip.c:840 2nd 0xcac52a0c ufs (ufs) @ kern/vfs_subr.c:4401 KDB: stack backtrace: db_trace_self_wrapper(c10896a4,63736564,2e706972,34383a63,c7000a30,...) at db_trace_self_wrapper+0x2d/frame 0xf97c08e8 kdb_backtrace(c108c7c2,cac52a0c,c1074cec,c7d864d8,c1095b80,...) at kdb_backtrace+0x30/frame 0xf97c0950 witness_checkorder(cac52a0c,9,c1095b77,1131,cac52a78,...) at witness_checkorder+0xbad/frame 0xf97c09a0 __lockmgr_args(cac52a0c,80400,cac52a78,0,0,...) at __lockmgr_args+0x87c/frame 0xf97c0a80 ffs_lock(f97c0b08,cb19cc74,c0b24618,cac52a0c,f97c0b08,...) at ffs_lock+0xad/frame 0xf97c0ac0 VOP_LOCK1_APV(c11e2de8,f97c0b08,cbc5001c,0,c11f75c8,...) at VOP_LOCK1_APV+0x112/frame 0xf97c0af0 _vn_lock(cac529d8,80400,c1095b77,1131) at _vn_lock+0xca/frame 0xf97c0b2c vfs_knllock(cac529d8,0,c107e850,72f,caf35a20,...) at vfs_knllock+0x29/frame 0xf97c0b44 knlist_remove_kq(0,0) at knlist_remove_kq+0x87/frame 0xf97c0b68 knlist_remove(cbc5001c,caf35a20,0) at knlist_remove+0x1f/frame 0xf97c0b78 filt_vfsdetach(caf35a20,0,c107e850,839,c11baa14,...) at filt_vfsdetach+0x2f/frame 0xf97c0b94 knote_fdclose(cb19cbc0,3,c107e0f8,47b,0,...) at knote_fdclose+0xde/frame 0xf97c0bc0 closefp(ca7653b8,cb19cbc0,1,91,cb1b9c2c,...) at closefp+0x6d/frame 0xf97c0bf0 do_dup(5,3,cb19ce14) at do_dup+0x29b/frame 0xf97c0c28 sys_dup2(cb19cbc0,f97c0cc8,14,c1087885,7b3,...) at sys_dup2+0x2d/frame 0xf97c0c40 syscall(f97c0d08) at syscall+0x2ed/frame 0xf97c0cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf97c0cfc --- syscall (90, FreeBSD ELF32, sys_dup2), eip = 0x2810b8a7, esp = 0xbfbfdb60, ebp = 0xbfbfdb98 --- 20130301 22:03:15 all: mountro3.sh 20130301 22:03:16 all: kevent2.sh 20130301 22:03:24 all: nfsrename.sh 20130301 22:06:14 all: rename3.sh 20130301 22:09:01 all: md2.sh lock order reversal: 1st 0xca60761c ufs_extattr_sx (ufs_extattr_sx) @ ufs/ufs/ufs_extattr.c:114 2nd 0xd3ad5a0c ufs (ufs) @ ufs/ffs/ffs_vfsops.c:1709 KDB: stack backtrace: db_trace_self_wrapper(c10896a4,3a632e72,a343131,c123fa00,0,...) at db_trace_self_wrapper+0x2d/frame 0xf9e39698 kdb_backtrace(c108c7c2,d3ad5a0c,c1074cec,c7d864d8,c10bd4b2,...) at kdb_backtrace+0x30/frame 0xf9e39700 witness_checkorder(d3ad5a0c,9,c10bd4a9,6ad,0,...) at witness_checkorder+0xbad/frame 0xf9e39750 __lockmgr_args(d3ad5a0c,80000,0,0,0,...) at __lockmgr_args+0x87c/frame 0xf9e3982c ffs_vgetf(cedc6000,2,80000,f9e398bc,0) at ffs_vgetf+0x165/frame 0xf9e39888 ffs_vget(cedc6000,2,80000,f9e398bc,0,...) at ffs_vget+0x2e/frame 0xf9e398a4 ufs_root(cedc6000,80000,f9e398f8,72,ca60761c,...) at ufs_root+0x43/frame 0xf9e398c8 ufs_extattr_autostart(cedc6000,d173f000,c10bd4a9,438,0,...) at ufs_extattr_autostart+0x7c/frame 0xf9e39908 ffs_mount(cedc6000,c11e2af0,ca71be80,ca6ef580,f9e39ab0,...) at ffs_mount+0x23b9/frame 0xf9e39a50 vfs_donmount(d173f000,0,0,d192c180,d192c180,...) at vfs_donmount+0xd33/frame 0xf9e39c18 sys_nmount(d173f000,f9e39cc8,f9e39d08,1,1,...) at sys_nmount+0x78/frame 0xf9e39c40 syscall(f9e39d08) at syscall+0x2ed/frame 0xf9e39cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9e39cfc --- syscall (378, FreeBSD ELF32, sys_nmount), eip = 0x280d3367, esp = 0xbfbfc8f8, ebp = 0xbfbfce50 --- lock order reversal: 1st 0xcac4fae0 so_snd_sx (so_snd_sx) @ kern/uipc_sockbuf.c:148 2nd 0xe18e25b4 bufwait (bufwait) @ vm/vm_pager.c:311 KDB: stack backtrace: db_trace_self_wrapper(c10896a4,c7d822a0,c7d84a08,c7d822a0,c137a7d8,...) at db_trace_self_wrapper+0x2d/frame 0xf9a9c4b0 kdb_backtrace(c108c7c2,e18e25b4,c1092cda,c7d82440,c10c34f0,...) at kdb_backtrace+0x30/frame 0xf9a9c518 witness_checkorder(e18e25b4,9,c10c34e7,137,0,...) at witness_checkorder+0xbad/frame 0xf9a9c568 __lockmgr_args(e18e25b4,80000,0,0,0,0,c10c34e7,137) at __lockmgr_args+0x87c/frame 0xf9a9c648 initpbuf(c14006a8,0,c10c34e7,162,0,...) at initpbuf+0xcc/frame 0xf9a9c674 getpbuf(c13ac49c,0,c10bf630,488,365,...) at getpbuf+0x115/frame 0xf9a9c698 swap_pager_getpages(cdb0b940,f9a9c754,6,1,f9a9c82c,...) at swap_pager_getpages+0x2b6/frame 0xf9a9c6e8 vm_fault_hold(d176aae0,28852000,1,0,0,...) at vm_fault_hold+0xa21/frame 0xf9a9c848 vm_fault(d176aae0,28852000,1,0,d176aae0,...) at vm_fault+0x82/frame 0xf9a9c870 trap_pfault(28852e20,f9a9c918,c0abac1a,c123f61c,d13de2f0,...) at trap_pfault+0x1ba/frame 0xf9a9c8e8 trap(f9a9ca34) at trap+0x53d/frame 0xf9a9ca28 calltrap() at calltrap+0x6/frame 0xf9a9ca28 --- trap 0xc, eip = 0xc0ea62bf, esp = 0xf9a9ca74, ebp = 0xf9a9caa8 --- copyin(f9a9cc10,0) at copyin+0x33/frame 0xf9a9caa8 uiomove(cbbee000,1e0,f9a9cc10,1,0,...) at uiomove+0x1f/frame 0xf9a9cab8 m_uiotombuf(f9a9cc10,2,bf64,0,0,...) at m_uiotombuf+0xe0/frame 0xf9a9caec sosend_generic(cac4f9c0,0,f9a9cc10,0,0,...) at sosend_generic+0x2d2/frame 0xf9a9cb5c soo_write(d646b498,f9a9cc10,ca6ef580,0,cba138d0,...) at soo_write+0x5d/frame 0xf9a9cb8c dofilewrite(d646b498,f9a9cc10,ffffffff,ffffffff,0,...) at dofilewrite+0x86/frame 0xf9a9cbc4 kern_writev(cba138d0,3,f9a9cc10,0,28852e20,...) at kern_writev+0x66/frame 0xf9a9cbf0 sys_write(cba138d0,f9a9ccc8,f9a9cd08,1,1,...) at sys_write+0x5c/frame 0xf9a9cc40 syscall(f9a9cd08) at syscall+0x2ed/frame 0xf9a9ccfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9a9ccfc --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28199fa7, esp = 0xbfbfd7b4, ebp = 0xbfbfd810 --- 20130301 22:30:37 all: recursiveflushes.sh lock order reversal: 1st 0xe1b869a0 bufwait (bufwait) @ kern/vfs_bio.c:3029 2nd 0xca8e6a1c snaplk (snaplk) @ ufs/ffs/ffs_snapshot.c:2298 KDB: stack backtrace: db_trace_self_wrapper(c10896a4,0,e1a24688,e194e4e8,0,...) at db_trace_self_wrapper+0x2d/frame 0xf419a8b8 kdb_backtrace(c108c7c2,ca8e6a1c,c10ba3e5,c7d87448,c10ba007,...) at kdb_backtrace+0x30/frame 0xf419a920 witness_checkorder(ca8e6a1c,9,c10b9ffe,8fa,ca8137a8,...) at witness_checkorder+0xbad/frame 0xf419a970 __lockmgr_args(ca8e6a1c,80900,ca8137a8,0,0,...) at __lockmgr_args+0x87c/frame 0xf419aa50 ffs_copyonwrite(ca813708,e194e4e8,c950d000,e194e4e8,4,...) at ffs_copyonwrite+0x1d1/frame 0xf419aac0 ffs_geom_strategy(ca8137c8,e194e4e8,423,110d,60,...) at ffs_geom_strategy+0xbf/frame 0xf419aadc bufwrite(e194e4e8,0,c10bd4a9,826,2ee0) at bufwrite+0x16c/frame 0xf419ab0c ffs_bufwrite(e194e4e8,e22e4000,1800,1800,0,...) at ffs_bufwrite+0x13b/frame 0xf419ab30 ffs_sbupdate(ca82e300,3,0,5bc,f419abb8,...) at ffs_sbupdate+0x1e8/frame 0xf419ab78 ffs_sync(ca706bb0,3,2,e7a,0,...) at ffs_sync+0x58d/frame 0xf419abfc sync_fsync(f419aca8,c10e3454,80400,c1095b77,703,...) at sync_fsync+0x162/frame 0xf419ac28 VOP_FSYNC_APV(c11c3198,f419aca8,c1095b77,703,4e20,...) at VOP_FSYNC_APV+0x124/frame 0xf419ac58 sched_sync(0,f419ad08,c107ef40,3d7,0,...) at sched_sync+0x38f/frame 0xf419accc fork_exit(c0b3bc40,0,f419ad08) at fork_exit+0x7f/frame 0xf419acf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf419acf4 --- trap 0, eip = 0, esp = 0xf419ad40, ebp = 0 --- lock order reversal: 1st 0xca8e6a1c snaplk (snaplk) @ ufs/ufs/ufs_vnops.c:968 2nd 0xd9158b74 ufs (ufs) @ ufs/ffs/ffs_snapshot.c:1627 KDB: stack backtrace: db_trace_self_wrapper(c10896a4,246,d6b068d0,c123ee80,d6b068d0,...) at db_trace_self_wrapper+0x2d/frame 0xf9c63670 kdb_backtrace(c108c7c2,d9158b74,c1074cec,c7d864d8,c10ba007,...) at kdb_backtrace+0x30/frame 0xf9c636d8 witness_checkorder(d9158b74,9,c10b9ffe,65b,0,...) at witness_checkorder+0xbad/frame 0xf9c63728 __lockmgr_args(d9158b74,80000,0,0,0,...) at __lockmgr_args+0x87c/frame 0xf9c63808 ffs_snapremove(d9158b40,c10b9ecf,c13614a0,c11b867c,246,...) at ffs_snapremove+0x118/frame 0xf9c63868 ffs_truncate(d9158b40,0,0,c00,0,...) at ffs_truncate+0x293/frame 0xf9c63a20 ufs_inactive(f9c63aa8,c10e3715,c1095b77,9d0,c11f4a90,...) at ufs_inactive+0x1fc/frame 0xf9c63a50 VOP_INACTIVE_APV(c11e2de8,f9c63aa8,c1095b77,9d0,f9c63ac8,...) at VOP_INACTIVE_APV+0x124/frame 0xf9c63a80 vinactive(d9158b40,d6b068d0,c1095b77,8ce,d9158bf0,...) at vinactive+0x191/frame 0xf9c63ac8 vputx(d9158b40,ffffffdf,d9158b40,f9c63bd0,0,...) at vputx+0x38a/frame 0xf9c63b10 kern_unlinkat(d6b068d0,ffffff9c,bfbfdc1a,0,0) at kern_unlinkat+0x29f/frame 0xf9c63c24 sys_unlink(d6b068d0,f9c63cc8,c10dc7ab,d6,1,...) at sys_unlink+0x32/frame 0xf9c63c40 syscall(f9c63d08) at syscall+0x2ed/frame 0xf9c63cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9c63cfc --- syscall (10, FreeBSD ELF32, sys_unlink), eip = 0x28173493, esp = 0xbfbfd99c, ebp = 0xbfbfda24 --- 20130301 22:34:09 all: suj16.sh 20130301 23:04:13 all: nullfs5.sh 20130301 23:17:54 all: suj9.sh 20130302 00:16:02 all: kevent4.sh 20130302 00:18:06 all: pts.sh 20130302 00:22:26 all: devfs.sh Mar 2 00:22:27 x4 mountd[574]: can't delete exports for /mnt8: Invalid argument Mar 2 00:22:27 x4 mountd[574]: can't delete exports for /mnt5: Invalid argument Mar 2 00:22:27 x4 mountd[574]: can't delete exports for /mnt10: Invalid argument Mar 2 00:22:27 x4 mountd[574]: can't delete exports for /mnt1: Invalid argument Mar 2 00:22:28 x4 mountd[574]: can't delete exports for /mnt3: Invalid argument Mar 2 00:22:28 x4 mountd[574]: can't delete exports for /mnt7: Invalid argument Mar 2 00:22:29 x4 mountd[574]: can't delete exports for /mnt4: Invalid argument Mar 2 00:22:29 x4 mountd[574]: can't delete exports for /mnt3: Invalid argument Mar 2 00:22:29 x4 mountd[574]: can't delete exports for /mnt4: Invalid argument Mar 2 00:22:30 x4 mountd[574]: can't delete exports for /mnt8: Invalid argument Mar 2 00:22:30 x4 mountd[574]: can't delete exports for /mnt5: Invalid argument Mar 2 00:22:30 x4 mountd[574]: can't delete exports for /mnt1: Invalid argument Mar 2 00:22:30 x4 mountd[574]: can't delete exports for /mnt10: Invalid argument Mar 2 00:22:30 x4 mountd[574]: can't delete exports for /mnt9: Invalid argument Mar 2 00:22:30 x4 mountd[574]: can't delete exports for /mnt7: Invalid argument Mar 2 00:22:30 x4 mountd[574]: can't delete exports for /mnt2: Invalid argument Mar 2 00:22:30 x4 mountd[574]: can't delete exports for /mnt5: Invalid argument Mar 2 00:22:30 x4 mountd[574]: can't delete exports for /mnt10: Invalid argument Mar 2 00:22:31 x4 mountd[574]: can't delete exports for /mnt6: Invalid argument Mar 2 00:22:31 x4 mountd[574]: can't delete exports for /mnt5: Invalid argument Mar 2 00:22:31 x4 mountd[574]: can't delete exports for /mnt7: Invalid argument Mar 2 00:22:31 x4 mountd[574]: can't delete exports for /mnt2: Device busy Mar 2 00:22:31 x4 mountd[574]: can't delete exports for /mnt1: Invalid argument Mar 2 00:22:31 x4 mountd[574]: can't delete exports for /mnt6: Invalid argument Mar 2 00:22:31 x4 mountd[574]: can't delete exports for /mnt7: Invalid argument Mar 2 00:22:32 x4 mountd[574]: can't delete exports for /mnt4: Invalid argument Mar 2 00:22:32 x4 mountd[574]: can't delete exports for /mnt9: Invalid argument Mar 2 00:22:32 x4 mountd[574]: can't delete exports for /mnt3: Invalid argument Mar 2 00:22:32 x4 mountd[574]: can't delete exports for /mnt7: Invalid argument Mar 2 00:22:32 x4 mountd[574]: can't delete exports for /mnt1: Invalid argument Mar 2 00:22:32 x4 mountd[574]: can't delete exports for /mnt9: Invalid argument Mar 2 00:22:33 x4 mountd[574]: can't delete exports for /mnt10: Invalid argument Mar 2 00:22:33 x4 mountd[574]: can't delete exports for /mnt5: Invalid argument Mar 2 00:22:33 x4 mountd[574]: can't delete exports for /mnt9: Invalid argument Mar 2 00:22:33 x4 mountd[574]: can't delete exports for /mnt3: Invalid argument Mar 2 00:22:33 x4 mountd[574]: can't delete exports for /mnt6: Invalid argument Mar 2 00:22:34 x4 mountd[574]: can't delete exports for /mnt2: Invalid argument Mar 2 00:22:34 x4 mountd[574]: can't delete exports for /mnt5: Invalid argument Mar 2 00:22:34 x4 mountd[574]: can't delete exports for /mnt10: Invalid argument Mar 2 00:22:34 x4 mountd[574]: can't delete exports for /mnt6: Device busy Mar 2 00:22:34 x4 mountd[574]: can't delete exports for /mnt10: Invalid argument Mar 2 00:22:34 x4 mountd[574]: can't delete exports for /mnt2: Invalid argument Mar 2 00:22:34 x4 mountd[574]: can't delete exports for /mnt9: Invalid argument Mar 2 00:22:34 x4 mountd[574]: can't delete exports for /mnt9: Invalid argument Mar 2 00:22:35 x4 mountd[574]: can't delete exports for /mnt5: Invalid argument Mar 2 00:22:35 x4 mountd[574]: can't delete exports for /mnt8: Invalid argument Mar 2 00:22:36 x4 mountd[574]: can't delete exports for /mnt7: Device busy Mar 2 00:22:36 x4 mountd[574]: can't delete exports for /mnt3: Invalid argument Mar 2 00:22:36 x4 mountd[574]: can't delete exports for /mnt8: Invalid argument Mar 2 00:22:36 x4 mountd[574]: can't delete exports for /mnt7: Invalid argument Mar 2 00:22:36 x4 mountd[574]: can't delete exports for /mnt8: Invalid argument Mar 2 00:22:36 x4 mountd[574]: can't delete exports for /mnt9: Invalid argument Mar 2 00:22:36 x4 mountd[574]: can't delete exports for /mnt4: Invalid argument Mar 2 00:22:37 x4 mountd[574]: can't delete exports for /mnt7: Invalid argument Mar 2 00:22:37 x4 mountd[574]: can't delete exports for /mnt9: Invalid argument Mar 2 00:22:37 x4 mountd[574]: can't delete exports for /mnt9: Invalid argument Mar 2 00:22:37 x4 mountd[574]: can't delete exports for /mnt5: Invalid argument Mar 2 00:22:37 x4 mountd[574]: can't delete exports for /mnt9: Invalid argument Mar 2 00:22:37 x4 mountd[574]: can't delete exports for /mnt9: Invalid argument Mar 2 00:22:37 x4 mountd[574]: can't delete exports for /mnt1: Invalid argument Mar 2 00:22:37 x4 mountd[574]: can't delete exports for /mnt5: Invalid argument Mar 2 00:22:37 x4 mountd[574]: can't delete exports for /mnt2: Invalid argument Mar 2 00:22:37 x4 mountd[574]: can't delete exports for /mnt2: Invalid argument Mar 2 00:22:38 x4 mountd[574]: can't delete exports for /mnt6: Invalid argument Mar 2 00:22:38 x4 mountd[574]: can't delete exports for /mnt4: Invalid argument Mar 2 00:22:38 x4 mountd[574]: can't delete exports for /mnt9: Invalid argument Mar 2 00:22:38 x4 mountd[574]: can't delete exports for /mnt1: Invalid argument Mar 2 00:22:38 x4 mountd[574]: can't delete exports for /mnt6: Invalid argument Mar 2 00:22:39 x4 mountd[574]: can't delete exports for /mnt8: Invalid argument Mar 2 00:22:39 x4 mountd[574]: can't delete exports for /mnt2: Invalid argument 20130302 00:22:40 all: kinfo.sh 20130302 00:52:41 all: tmpfs.sh 20130302 01:06:03 all: suj24.sh 20130302 01:06:04 all: umountf5.sh 20130302 01:06:57 all: namecache.sh 20130302 01:20:36 all: suj12.sh GEOM_NOP: Device md5.nop created. GEOM_NOP: Device md5.nop removed. 20130302 01:42:00 all: fuse.sh fuse-freebsd: version 0.4.4, FUSE ABI 7.8 20130302 02:05:01 all: zfs3.sh 20130302 02:05:30 all: fragments.sh Mar 2 02:05:42 x4 kernel: pid 60326 (fragments), uid 1004 inumber 119629 on /mnt: filesystem full Mar 2 02:05:42 x4 kernel: pid 60326 (fragments), uid 1004 inumber 98325 on /mnt: filesystem full Mar 2 02:05:43 x4 kernel: pid 60350 (fragments), uid 1004 inumber 5147 on /mnt: filesystem full : : Mar 2 02:47:40 x4 kernel: pid 64841 (fragments), uid 1004 inumber 5102 on /mnt: filesystem full Mar 2 02:47:41 x4 kernel: pid 64841 (fragments), uid 1004 inumber 5169 on /mnt: filesystem full Mar 2 02:47:42 x4 kernel: pid 64838 (fragments), uid 1004 inumber 5135 on /mnt: filesystem full Mar 2 02:47:43 x4 kernel: pid 64838 (fragments), uid 1004 inumber 5157 on /mnt: filesystem full Mar 2 02:47:44 x4 kernel: pid 64843 (fragments), uid 1004 inumber 5179 on /mnt: filesystem full Mar 2 02:47:45 x4 kernel: pid 64837 (fragments), uid 1004 inumber 5075 on /mnt: filesystem full Mar 2 02:47:46 x4 kernel: pid 64837 (fragments), uid 1004 inumber 5265 on /mnt: filesystem full Mar 2 02:47:47 x4 kernel: pid 64837 (fragments), uid 1004 inumber 5275 on /mnt: filesystem full Mar 2 02:47:48 x4 kernel: pid 64842 (fragments), uid 1004 inumber 5100 on /mnt: filesystem full Mar 2 02:47:49 x4 kernel: pid 64837 (fragments), uid 1004 inumber 5141 on /mnt: filesystem full Mar 2 02:47:50 x4 kernel: pid 64837 (fragments), uid 1004 inumber 5141 on /mnt: filesystem full Mar 2 02:47:51 x4 kernel: pid 64842 (fragments), uid 1004 inumber 5112 on /mnt: filesystem full Mar 2 02:47:53 x4 kernel: pid 64837 (fragments), uid 1004 inumber 5176 on /mnt: filesystem full Mar 2 02:47:54 x4 kernel: pid 64837 (fragments), uid 1004 inumber 5176 on /mnt: filesystem full Mar 2 02:47:55 x4 kernel: pid 64837 (fragments), uid 1004 inumber 5107 on /mnt: filesystem full Mar 2 02:47:56 x4 kernel: pid 64842 (fragments), uid 1004 inumber 5142 on /mnt: filesystem full Mar 2 02:47:57 x4 kernel: pid 64837 (fragments), uid 1004 inumber 5143 on /mnt: filesystem full 20130302 02:48:00 all: sndstat.sh 20130302 02:48:00 all: mlockall2.sh 20130302 03:50:42 all: tmpfs7.sh lock order reversal: 1st 0xd3b03120 so_snd_sx (so_snd_sx) @ kern/uipc_sockbuf.c:145 2nd 0xcc2f719c tmpfs (tmpfs) @ kern/uipc_syscalls.c:2105 KDB: stack backtrace: db_trace_self_wrapper(c10896a4,f99bc934,c0ae0c3f,1,c108d041,...) at db_trace_self_wrapper+0x2d/frame 0xf99bc8d8 kdb_backtrace(c108c7c2,cc2f719c,d1b06fbf,c7d87310,c1092394,...) at kdb_backtrace+0x30/frame 0xf99bc940 witness_checkorder(cc2f719c,9,c109238b,839,cc2f7208,...) at witness_checkorder+0xbad/frame 0xf99bc990 __lockmgr_args(cc2f719c,200000,cc2f7208,0,0,0,c109238b,839) at __lockmgr_args+0x87c/frame 0xf99bca70 vop_stdlock(f99bcae8,0,d40f32e4,f99bcadc,c0d23bf4,...) at vop_stdlock+0x53/frame 0xf99bcaa0 VOP_LOCK1_APV(d1b080b4,f99bcae8,f99bcb10,c0a809f7,c11f75c8,...) at VOP_LOCK1_APV+0x112/frame 0xf99bcad0 _vn_lock(cc2f7168,200000,c109238b,839,f99bcbc0,...) at _vn_lock+0xca/frame 0xf99bcb10 kern_sendfile(da391000,f99bccc8,0,0,0,...) at kern_sendfile+0x8b3/frame 0xf99bcbe8 do_sendfile(0) at do_sendfile+0xb7/frame 0xf99bcc34 sys_sendfile(da391000,f99bccc8,c10dc7ab,d6,f99bccc4,...) at sys_sendfile+0x16/frame 0xf99bcc40 syscall(f99bcd08) at syscall+0x2ed/frame 0xf99bccfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf99bccfc --- syscall (393, FreeBSD ELF32, sys_sendfile), eip = 0x280be2c7, esp = 0xbfbfd860, ebp = 0xbfbfd888 --- 20130302 03:50:48 all: crossmp2.sh Expensive timeout(9) function: 0xc092e840(0xc8062aa0) 0.175411139 s 20130302 03:52:01 all: ldt.sh 20130302 03:52:02 all: nfs6.sh 20130302 03:53:30 all: msdos4.sh 20130302 04:13:42 all: quota5.sh 20130302 04:13:43 all: snap6.sh Mar 2 04:18:36 x4 kernel: pid 88087 (dd), uid 0 inumber 143 on /var: filesystem full 20130302 04:19:37 all: md3.sh 20130302 04:31:32 all: nullfs9.sh 20130302 04:31:35 all: truncate.sh panic: kmem_malloc(4096): kmem_map too small: 432013312 total allocated cpuid = 1 KDB: enter: panic [ thread pid 17 tid 100074 ] Stopped at kdb_enter+0x3d: movl $0,kdb_why db> run pho db:0:pho> bt Tracing pid 17 tid 100074 td 0xc950d000 kdb_enter(c10853c1,c10853c1,c10c0ab6,f419a9c8,0,...) at kdb_enter+0x3d/frame 0xf419a970 vpanic(f419a9c8,f419a9f0,c0d13eea,c10c0ab6,1000,...) at vpanic+0x142/frame 0xf419a9b0 panic(c10c0ab6,1000,19c00000,c10c0ab0,7d0,...) at panic+0x12/frame 0xf419a9bc kmem_malloc(c1bb508c,1000,102) at kmem_malloc+0x2ca/frame 0xf419a9f0 page_alloc(c1b9fd00,1000,f419aa2f,102,c1ba1698,...) at page_alloc+0x27/frame 0xf419aa04 keg_alloc_slab(102,4,c10bfd74,887,f419aa88,...) at keg_alloc_slab+0xc3/frame 0xf419aa3c keg_fetch_slab(102,80,102,ced2bc48,f419aad0,...) at keg_fetch_slab+0x132/frame 0xf419aa70 zone_fetch_slab(c1b9fd00,0,102,83d,202,...) at zone_fetch_slab+0x3f/frame 0xf419aa88 uma_zalloc_arg(c1b9fd00,0,102) at uma_zalloc_arg+0x435/frame 0xf419aad0 g_alloc_bio(c7ef79b0,0,c107414a,ba,ca6bb300,...) at g_alloc_bio+0x23/frame 0xf419aae4 g_vfs_strategy(ca8137c8,e19aa7d4,423,110d,c0b1a47b,...) at g_vfs_strategy+0xa0/frame 0xf419ab0c bufwrite(e19aa7d4,0,c10bd4a9,826,c7d86338) at bufwrite+0x16c/frame 0xf419ab3c ffs_bufwrite(e19aa7d4,c137c9e0,c7d84a08,246,c950d0b4,...) at ffs_bufwrite+0x13b/frame 0xf419ab60 vfs_bio_awrite(e19aa7d4,0,c1094c23,29f,0,...) at vfs_bio_awrite+0x205/frame 0xf419abc4 vop_stdfsync(f419aca8,f419ac18,0,c11a8a30,f419aca8,...) at vop_stdfsync+0x238/frame 0xf419ac08 devfs_fsync(f419aca8,c10e3454,80400,c1095b77,703,...) at devfs_fsync+0x2a/frame 0xf419ac28 VOP_FSYNC_APV(c11a8a30,f419aca8,c1095b77,703,4e20,...) at VOP_FSYNC_APV+0x124/frame 0xf419ac58 sched_sync(0,f419ad08,c107ef40,3d7,0,...) at sched_sync+0x38f/frame 0xf419accc fork_exit(c0b3bc40,0,f419ad08) at fork_exit+0x7f/frame 0xf419acf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf419acf4 --- trap 0, eip = 0, esp = 0xf419ad40, ebp = 0 --- db:0:bt> show allpcpu Current CPU: 1 cpuid = 0 dynamic pcpu = 0x628480 curthread = 0xd1744000: pid 90322 "truncate" curpcb = 0xf9d4dd60 fpcurthread = none idlethread = 0xc7dde8d0: tid 100003 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: cpuid = 1 dynamic pcpu = 0x6938480 curthread = 0xc950d000: pid 17 "syncer" curpcb = 0xf419ad60 fpcurthread = none idlethread = 0xc7dde5e0: tid 100004 "idle: cpu1" APIC ID = 1 currentldt = 0x50 spin locks held: cpuid = 2 dynamic pcpu = 0x693b480 curthread = 0xcb25a8d0: pid 90325 "truncate" curpcb = 0xf9b8ad60 fpcurthread = none idlethread = 0xc7dde2f0: tid 100005 "idle: cpu2" APIC ID = 2 currentldt = 0x50 spin locks held: cpuid = 3 dynamic pcpu = 0x693e480 curthread = 0xd16e95e0: pid 90318 "truncate" curpcb = 0xf9d36d60 fpcurthread = none idlethread = 0xc7dde000: tid 100006 "idle: cpu3" APIC ID = 3 currentldt = 0x50 spin locks held: db:0:allpcpu> show alllocks Process 90760 (makewhatis) thread 0xd54535e0 (100392) shared lockmgr ufs (ufs) r = 0 (0xe11dfcdc) locked @ kern/vfs_subr.c:2176 Process 90327 (truncate) thread 0xcb279bc0 (100373) exclusive lockmgr ufs (ufs) r = 0 (0xcd93acdc) locked @ kern/vfs_vnops.c:795 Process 90326 (truncate) thread 0xcb25a2f0 (100395) exclusive lockmgr ufs (ufs) r = 0 (0xcc86146c) locked @ kern/vfs_vnops.c:795 Process 90325 (truncate) thread 0xcb25a8d0 (100393) exclusive lockmgr ufs (ufs) r = 0 (0xcf6bb5d4) locked @ kern/vfs_vnops.c:795 Process 90324 (truncate) thread 0xcbbba2f0 (100208) exclusive lockmgr ufs (ufs) r = 0 (0xd90e8b74) locked @ kern/vfs_vnops.c:795 Process 90323 (truncate) thread 0xcb25a5e0 (100394) exclusive lockmgr ufs (ufs) r = 0 (0xd0bde5d4) locked @ kern/vfs_vnops.c:1186 Process 90322 (truncate) thread 0xd1744000 (100519) exclusive sleep mutex Softdep Lock (Softdep Lock) r = 0 (0xc13ac2d4) locked @ ufs/ffs/ffs_softdep.c:2261 exclusive lockmgr bufwait (bufwait) r = 0 (0xe1aaa734) locked @ kern/vfs_bio.c:3029 exclusive lockmgr ufs (ufs) r = 0 (0xd0db746c) locked @ kern/vfs_vnops.c:795 Process 90321 (truncate) thread 0xd5a595e0 (100307) exclusive lockmgr ufs (ufs) r = 0 (0xd43858a4) locked @ kern/vfs_vnops.c:795 Process 90320 (truncate) thread 0xcb2878d0 (100441) exclusive lockmgr ufs (ufs) r = 0 (0xcc6e58a4) locked @ kern/vfs_vnops.c:795 Process 90319 (truncate) thread 0xcaeb3bc0 (100257) exclusive lockmgr bufwait (bufwait) r = 0 (0xe18eea94) locked @ kern/vfs_bio.c:2266 exclusive lockmgr ufs (ufs) r = 0 (0xd2da6b74) locked @ kern/vfs_vnops.c:795 Process 90318 (truncate) thread 0xd16e95e0 (100513) exclusive lockmgr ufs (ufs) r = 0 (0xd12c8304) locked @ kern/vfs_vnops.c:1186 Process 90317 (truncate) thread 0xca7dd5e0 (100130) exclusive lockmgr ufs (ufs) r = 0 (0xd0b93034) locked @ kern/vfs_vnops.c:795 Process 90316 (truncate) thread 0xd56e28d0 (100862) exclusive lockmgr ufs (ufs) r = 0 (0xd211b5d4) locked @ kern/vfs_vnops.c:795 Process 90315 (truncate) thread 0xd17c38d0 (100569) exclusive lockmgr ufs (ufs) r = 0 (0xced0a19c) locked @ kern/vfs_vnops.c:795 Process 90314 (truncate) thread 0xd56f4bc0 (100783) exclusive lockmgr ufs (ufs) r = 0 (0xd10ed46c) locked @ kern/vfs_vnops.c:795 Process 90313 (truncate) thread 0xd16dd8d0 (100870) exclusive lockmgr ufs (ufs) r = 0 (0xd42cd46c) locked @ kern/vfs_vnops.c:795 Process 90312 (truncate) thread 0xd58372f0 (100940) exclusive lockmgr ufs (ufs) r = 0 (0xcd88c304) locked @ kern/vfs_vnops.c:795 Process 90311 (truncate) thread 0xcb1a58d0 (100138) exclusive lockmgr ufs (ufs) r = 0 (0xd108373c) locked @ kern/vfs_vnops.c:795 Process 90310 (truncate) thread 0xd16e02f0 (100506) exclusive lockmgr ufs (ufs) r = 0 (0xcb400a0c) locked @ kern/vfs_vnops.c:795 Process 90309 (truncate) thread 0xcaeafbc0 (100266) exclusive lockmgr ufs (ufs) r = 0 (0xcc0d6034) locked @ kern/vfs_vnops.c:1186 Process 90308 (truncate) thread 0xcb1a4bc0 (100166) exclusive lockmgr ufs (ufs) r = 0 (0xd0ddccdc) locked @ kern/vfs_vnops.c:1186 Process 87141 (find) thread 0xd16d72f0 (100492) shared lockmgr ufs (ufs) r = 0 (0xe0cfa19c) locked @ kern/vfs_lookup.c:516 Process 17 (syncer) thread 0xc950d000 (100074) exclusive lockmgr devfs (devfs) r = 0 (0xca81373c) locked @ kern/vfs_subr.c:1795 db:0:alllocks> show lockedvnods Locked vnodes 0xca813708: tag devfs, type VCHR usecount 1, writecount 0, refcount 818 mountedhere 0xca603b00 flags (VI_ACTIVE) v_object 0xca7afafc ref 0 pages 11491 lock type devfs: EXCL by thread 0xc950d000 (pid 17, syncer, tid 100074) #0 0xc0a783d5 at __lockmgr_args+0x10c5 #1 0xc0b24813 at vop_stdlock+0x53 #2 0xc0ecf1b2 at VOP_LOCK1_APV+0x112 #3 0xc0b46faa at _vn_lock+0xca #4 0xc0b3bfa0 at sched_sync+0x360 #5 0xc0a6381f at fork_exit+0x7f #6 0xc0e91644 at fork_trampoline+0x8 dev ada0s1e 0xe11dfca8: tag ufs, type VDIR usecount 1, writecount 0, refcount 3 mountedhere 0 flags (VI_ACTIVE) v_object 0xd6290000 ref 0 pages 1 lock type ufs: SHARED (count 1) #0 0xc0a78270 at __lockmgr_args+0xf60 #1 0xc0cef38d at ffs_lock+0xad #2 0xc0ecf1b2 at VOP_LOCK1_APV+0x112 #3 0xc0b46faa at _vn_lock+0xca #4 0xc0b34d97 at vget+0x77 #5 0xc0b20138 at cache_lookup+0x5a8 #6 0xc0b2126c at vfs_cache_lookup+0xbc #7 0xc0ecbbd4 at VOP_LOOKUP_APV+0x124 #8 0xc0b29ffe at lookup+0x52e #9 0xc0b296ed at namei+0x47d #10 0xc0b412cd at kern_statat_vnhook+0xbd #11 0xc0b410dd at sys_stat+0x4d #12 0xc0ea909d at syscall+0x2ed #13 0xc0e91631 at Xint0x80_syscall+0x21 ino 1366079, on dev ada0s1d 0xe0cfa168: tag ufs, type VDIR usecount 3, writecount 0, refcount 6 mountedhere 0 flags (VI_ACTIVE) v_object 0xd81d8e74 ref 0 pages 1 lock type ufs: SHARED (count 1) #0 0xc0a78270 at __lockmgr_args+0xf60 #1 0xc0cef38d at ffs_lock+0xad #2 0xc0ecf1b2 at VOP_LOCK1_APV+0x112 #3 0xc0b46faa at _vn_lock+0xca #4 0xc0b29bc7 at lookup+0xf7 #5 0xc0b296ed at namei+0x47d #6 0xc0b412cd at kern_statat_vnhook+0xbd #7 0xc0b4148d at sys_lstat+0x4d #8 0xc0ea909d at syscall+0x2ed #9 0xc0e91631 at Xint0x80_syscall+0x21 ino 1771042, on dev ada0s1e 0xd0ddcca8: tag ufs, type VREG usecount 1, writecount 1, refcount 3 mountedhere 0 flags (VV_NOSYNC|VI_ACTIVE) v_object 0xd53dd000 ref 0 pages 1 lock type ufs: EXCL by thread 0xcb1a4bc0 (pid 90308, truncate, tid 100166) #0 0xc0a783d5 at __lockmgr_args+0x10c5 #1 0xc0cef38d at ffs_lock+0xad #2 0xc0ecf1b2 at VOP_LOCK1_APV+0x112 #3 0xc0b46faa at _vn_lock+0xca #4 0xc0b45e04 at vn_truncate+0x134 #5 0xc0ae5a69 at kern_ftruncate+0xa9 #6 0xc0ae5b28 at sys_ftruncate+0x28 #7 0xc0ea909d at syscall+0x2ed #8 0xc0e91631 at Xint0x80_syscall+0x21 ino 18818057, on dev label/tmp 0xd10ed438: tag ufs, type VREG usecount 1, writecount 1, refcount 1 mountedhere 0 flags (VV_NOSYNC|VI_ACTIVE) v_object 0xd2cc15c8 ref 0 pages 0 lock type ufs: EXCL by thread 0xd56f4bc0 (pid 90314, truncate, tid 100783) #0 0xc0a783d5 at __lockmgr_args+0x10c5 #1 0xc0cef38d at ffs_lock+0xad #2 0xc0ecf1b2 at VOP_LOCK1_APV+0x112 #3 0xc0b46faa at _vn_lock+0xca #4 0xc0b49593 at vn_write+0x243 #5 0xc0b459eb at vn_io_fault+0x24b #6 0xc0ae5866 at dofilewrite+0x86 #7 0xc0ae5576 at kern_writev+0x66 #8 0xc0ae54fc at sys_write+0x5c #9 0xc0ea909d at syscall+0x2ed #10 0xc0e91631 at Xint0x80_syscall+0x21 ino 18818073, on dev label/tmp 0xd1083708: tag ufs, type VREG usecount 1, writecount 1, refcount 1 mountedhere 0 flags (VV_NOSYNC|VI_ACTIVE) v_object 0xd3f575c8 ref 0 pages 0 lock type ufs: EXCL by thread 0xcb1a58d0 (pid 90311, truncate, tid 100138) #0 0xc0a783d5 at __lockmgr_args+0x10c5 #1 0xc0cef38d at ffs_lock+0xad #2 0xc0ecf1b2 at VOP_LOCK1_APV+0x112 #3 0xc0b46faa at _vn_lock+0xca #4 0xc0b49593 at vn_write+0x243 #5 0xc0b459eb at vn_io_fault+0x24b #6 0xc0ae5866 at dofilewrite+0x86 #7 0xc0ae5576 at kern_writev+0x66 #8 0xc0ae54fc at sys_write+0x5c #9 0xc0ea909d at syscall+0x2ed #10 0xc0e91631 at Xint0x80_syscall+0x21 ino 18818091, on dev label/tmp 0xd12c82d0: tag ufs, type VREG usecount 1, writecount 1, refcount 3 mountedhere 0 flags (VV_NOSYNC|VI_ACTIVE) v_object 0xd3ed1c24 ref 0 pages 1 lock type ufs: EXCL by thread 0xd16e95e0 (pid 90318, truncate, tid 100513) #0 0xc0a783d5 at __lockmgr_args+0x10c5 #1 0xc0cef38d at ffs_lock+0xad #2 0xc0ecf1b2 at VOP_LOCK1_APV+0x112 #3 0xc0b46faa at _vn_lock+0xca #4 0xc0b45e04 at vn_truncate+0x134 #5 0xc0ae5a69 at kern_ftruncate+0xa9 #6 0xc0ae5b28 at sys_ftruncate+0x28 #7 0xc0ea909d at syscall+0x2ed #8 0xc0e91631 at Xint0x80_syscall+0x21 ino 18818096, on dev label/tmp 0xcc0d6000: tag ufs, type VREG usecount 1, writecount 1, refcount 3 mountedhere 0 flags (VV_NOSYNC|VI_ACTIVE) v_object 0xd2c926f0 ref 0 pages 1 lock type ufs: EXCL by thread 0xcaeafbc0 (pid 90309, truncate, tid 100266) #0 0xc0a783d5 at __lockmgr_args+0x10c5 #1 0xc0cef38d at ffs_lock+0xad #2 0xc0ecf1b2 at VOP_LOCK1_APV+0x112 #3 0xc0b46faa at _vn_lock+0xca #4 0xc0b45e04 at vn_truncate+0x134 #5 0xc0ae5a69 at kern_ftruncate+0xa9 #6 0xc0ae5b28 at sys_ftruncate+0x28 #7 0xc0ea909d at syscall+0x2ed #8 0xc0e91631 at Xint0x80_syscall+0x21 ino 18818103, on dev label/tmp 0xced0a168: tag ufs, type VREG usecount 1, writecount 1, refcount 1 mountedhere 0 flags (VV_NOSYNC|VI_ACTIVE) v_object 0xcd274940 ref 0 pages 0 lock type ufs: EXCL by thread 0xd17c38d0 (pid 90315, truncate, tid 100569) #0 0xc0a783d5 at __lockmgr_args+0x10c5 #1 0xc0cef38d at ffs_lock+0xad #2 0xc0ecf1b2 at VOP_LOCK1_APV+0x112 #3 0xc0b46faa at _vn_lock+0xca #4 0xc0b49593 at vn_write+0x243 #5 0xc0b459eb at vn_io_fault+0x24b #6 0xc0ae5866 at dofilewrite+0x86 #7 0xc0ae5576 at kern_writev+0x66 #8 0xc0ae54fc at sys_write+0x5c #9 0xc0ea909d at syscall+0x2ed #10 0xc0e91631 at Xint0x80_syscall+0x21 ino 18818111, on dev label/tmp 0xd0bde5a0: tag ufs, type VREG usecount 1, writecount 1, refcount 1 mountedhere 0 flags (VV_NOSYNC|VI_ACTIVE) v_object 0xd64ec784 ref 0 pages 0 lock type ufs: EXCL by thread 0xcb25a5e0 (pid 90323, truncate, tid 100394) #0 0xc0a783d5 at __lockmgr_args+0x10c5 #1 0xc0cef38d at ffs_lock+0xad #2 0xc0ecf1b2 at VOP_LOCK1_APV+0x112 #3 0xc0b46faa at _vn_lock+0xca #4 0xc0b45e04 at vn_truncate+0x134 #5 0xc0ae5a69 at kern_ftruncate+0xa9 #6 0xc0ae5b28 at sys_ftruncate+0x28 #7 0xc0ea909d at syscall+0x2ed #8 0xc0e91631 at Xint0x80_syscall+0x21 ino 18818144, on dev label/tmp 0xd0db7438: tag ufs, type VREG usecount 1, writecount 1, refcount 1 mountedhere 0 flags (VV_NOSYNC|VI_ACTIVE) v_object 0xd2c9ab90 ref 0 pages 0 lock type ufs: EXCL by thread 0xd1744000 (pid 90322, truncate, tid 100519) #0 0xc0a783d5 at __lockmgr_args+0x10c5 #1 0xc0cef38d at ffs_lock+0xad #2 0xc0ecf1b2 at VOP_LOCK1_APV+0x112 #3 0xc0b46faa at _vn_lock+0xca #4 0xc0b49593 at vn_write+0x243 #5 0xc0b459eb at vn_io_fault+0x24b #6 0xc0ae5866 at dofilewrite+0x86 #7 0xc0ae5576 at kern_writev+0x66 #8 0xc0ae54fc at sys_write+0x5c #9 0xc0ea909d at syscall+0x2ed #10 0xc0e91631 at Xint0x80_syscall+0x21 ino 18818151, on dev label/tmp 0xd4385870: tag ufs, type VREG usecount 1, writecount 1, refcount 1 mountedhere 0 flags (VV_NOSYNC|VI_ACTIVE) v_object 0xd6762000 ref 0 pages 0 lock type ufs: EXCL by thread 0xd5a595e0 (pid 90321, truncate, tid 100307) #0 0xc0a783d5 at __lockmgr_args+0x10c5 #1 0xc0cef38d at ffs_lock+0xad #2 0xc0ecf1b2 at VOP_LOCK1_APV+0x112 #3 0xc0b46faa at _vn_lock+0xca #4 0xc0b49593 at vn_write+0x243 #5 0xc0b459eb at vn_io_fault+0x24b #6 0xc0ae5866 at dofilewrite+0x86 #7 0xc0ae5576 at kern_writev+0x66 #8 0xc0ae54fc at sys_write+0x5c #9 0xc0ea909d at syscall+0x2ed #10 0xc0e91631 at Xint0x80_syscall+0x21 ino 18818157, on dev label/tmp 0xcd88c2d0: tag ufs, type VREG usecount 1, writecount 1, refcount 1 mountedhere 0 flags (VV_NOSYNC|VI_ACTIVE) v_object 0xd8955c24 ref 0 pages 0 lock type ufs: EXCL by thread 0xd58372f0 (pid 90312, truncate, tid 100940) #0 0xc0a783d5 at __lockmgr_args+0x10c5 #1 0xc0cef38d at ffs_lock+0xad #2 0xc0ecf1b2 at VOP_LOCK1_APV+0x112 #3 0xc0b46faa at _vn_lock+0xca #4 0xc0b49593 at vn_write+0x243 #5 0xc0b459eb at vn_io_fault+0x24b #6 0xc0ae5866 at dofilewrite+0x86 #7 0xc0ae5576 at kern_writev+0x66 #8 0xc0ae54fc at sys_write+0x5c #9 0xc0ea909d at syscall+0x2ed #10 0xc0e91631 at Xint0x80_syscall+0x21 ino 18818159, on dev label/tmp 0xcf6bb5a0: tag ufs, type VREG usecount 1, writecount 1, refcount 1 mountedhere 0 flags (VV_NOSYNC|VI_ACTIVE) v_object 0xd90912e4 ref 0 pages 0 lock type ufs: EXCL by thread 0xcb25a8d0 (pid 90325, truncate, tid 100393) #0 0xc0a783d5 at __lockmgr_args+0x10c5 #1 0xc0cef38d at ffs_lock+0xad #2 0xc0ecf1b2 at VOP_LOCK1_APV+0x112 #3 0xc0b46faa at _vn_lock+0xca #4 0xc0b49593 at vn_write+0x243 #5 0xc0b459eb at vn_io_fault+0x24b #6 0xc0ae5866 at dofilewrite+0x86 #7 0xc0ae5576 at kern_writev+0x66 #8 0xc0ae54fc at sys_write+0x5c #9 0xc0ea909d at syscall+0x2ed #10 0xc0e91631 at Xint0x80_syscall+0x21 ino 18818167, on dev label/tmp 0xd2da6b40: tag ufs, type VREG usecount 1, writecount 1, refcount 3 mountedhere 0 flags (VV_NOSYNC|VI_ACTIVE) v_object 0xd479340c ref 0 pages 1 lock type ufs: EXCL by thread 0xcaeb3bc0 (pid 90319, truncate, tid 100257) #0 0xc0a783d5 at __lockmgr_args+0x10c5 #1 0xc0cef38d at ffs_lock+0xad #2 0xc0ecf1b2 at VOP_LOCK1_APV+0x112 #3 0xc0b46faa at _vn_lock+0xca #4 0xc0b49593 at vn_write+0x243 #5 0xc0b459eb at vn_io_fault+0x24b #6 0xc0ae5866 at dofilewrite+0x86 #7 0xc0ae5576 at kern_writev+0x66 #8 0xc0ae54fc at sys_write+0x5c #9 0xc0ea909d at syscall+0x2ed #10 0xc0e91631 at Xint0x80_syscall+0x21 ino 18818172, on dev label/tmp 0xcc6e5870: tag ufs, type VREG usecount 1, writecount 1, refcount 1 mountedhere 0 flags (VV_NOSYNC|VI_ACTIVE) v_object 0xd4f888ac ref 0 pages 0 lock type ufs: EXCL by thread 0xcb2878d0 (pid 90320, truncate, tid 100441) #0 0xc0a783d5 at __lockmgr_args+0x10c5 #1 0xc0cef38d at ffs_lock+0xad #2 0xc0ecf1b2 at VOP_LOCK1_APV+0x112 #3 0xc0b46faa at _vn_lock+0xca #4 0xc0b49593 at vn_write+0x243 #5 0xc0b459eb at vn_io_fault+0x24b #6 0xc0ae5866 at dofilewrite+0x86 #7 0xc0ae5576 at kern_writev+0x66 #8 0xc0ae54fc at sys_write+0x5c #9 0xc0ea909d at syscall+0x2ed #10 0xc0e91631 at Xint0x80_syscall+0x21 ino 18818174, on dev label/tmp 0xd90e8b40: tag ufs, type VREG usecount 1, writecount 1, refcount 1 mountedhere 0 flags (VV_NOSYNC|VI_ACTIVE) v_object 0xd45d8094 ref 0 pages 0 lock type ufs: EXCL by thread 0xcbbba2f0 (pid 90324, truncate, tid 100208) #0 0xc0a783d5 at __lockmgr_args+0x10c5 #1 0xc0cef38d at ffs_lock+0xad #2 0xc0ecf1b2 at VOP_LOCK1_APV+0x112 #3 0xc0b46faa at _vn_lock+0xca #4 0xc0b49593 at vn_write+0x243 #5 0xc0b459eb at vn_io_fault+0x24b #6 0xc0ae5866 at dofilewrite+0x86 #7 0xc0ae5576 at kern_writev+0x66 #8 0xc0ae54fc at sys_write+0x5c #9 0xc0ea909d at syscall+0x2ed #10 0xc0e91631 at Xint0x80_syscall+0x21 ino 18818177, on dev label/tmp 0xd211b5a0: tag ufs, type VREG usecount 1, writecount 1, refcount 1 mountedhere 0 flags (VV_NOSYNC|VI_ACTIVE) v_object 0xd4632818 ref 0 pages 0 lock type ufs: EXCL by thread 0xd56e28d0 (pid 90316, truncate, tid 100862) #0 0xc0a783d5 at __lockmgr_args+0x10c5 #1 0xc0cef38d at ffs_lock+0xad #2 0xc0ecf1b2 at VOP_LOCK1_APV+0x112 #3 0xc0b46faa at _vn_lock+0xca #4 0xc0b49593 at vn_write+0x243 #5 0xc0b459eb at vn_io_fault+0x24b #6 0xc0ae5866 at dofilewrite+0x86 #7 0xc0ae5576 at kern_writev+0x66 #8 0xc0ae54fc at sys_write+0x5c #9 0xc0ea909d at syscall+0x2ed #10 0xc0e91631 at Xint0x80_syscall+0x21 ino 18818181, on dev label/tmp 0xcb4009d8: tag ufs, type VREG usecount 1, writecount 1, refcount 1 mountedhere 0 flags (VV_NOSYNC|VI_ACTIVE) v_object 0xd41e52e4 ref 0 pages 0 lock type ufs: EXCL by thread 0xd16e02f0 (pid 90310, truncate, tid 100506) #0 0xc0a783d5 at __lockmgr_args+0x10c5 #1 0xc0cef38d at ffs_lock+0xad #2 0xc0ecf1b2 at VOP_LOCK1_APV+0x112 #3 0xc0b46faa at _vn_lock+0xca #4 0xc0b49593 at vn_write+0x243 #5 0xc0b459eb at vn_io_fault+0x24b #6 0xc0ae5866 at dofilewrite+0x86 #7 0xc0ae5576 at kern_writev+0x66 #8 0xc0ae54fc at sys_write+0x5c #9 0xc0ea909d at syscall+0x2ed #10 0xc0e91631 at Xint0x80_syscall+0x21 ino 18818186, on dev label/tmp 0xcc861438: tag ufs, type VREG usecount 1, writecount 1, refcount 1 mountedhere 0 flags (VV_NOSYNC|VI_ACTIVE) v_object 0xcd11e2e4 ref 0 pages 0 lock type ufs: EXCL by thread 0xcb25a2f0 (pid 90326, truncate, tid 100395) #0 0xc0a783d5 at __lockmgr_args+0x10c5 #1 0xc0cef38d at ffs_lock+0xad #2 0xc0ecf1b2 at VOP_LOCK1_APV+0x112 #3 0xc0b46faa at _vn_lock+0xca #4 0xc0b49593 at vn_write+0x243 #5 0xc0b459eb at vn_io_fault+0x24b #6 0xc0ae5866 at dofilewrite+0x86 #7 0xc0ae5576 at kern_writev+0x66 #8 0xc0ae54fc at sys_write+0x5c #9 0xc0ea909d at syscall+0x2ed #10 0xc0e91631 at Xint0x80_syscall+0x21 ino 18818199, on dev label/tmp 0xd42cd438: tag ufs, type VREG usecount 1, writecount 1, refcount 1 mountedhere 0 flags (VV_NOSYNC|VI_ACTIVE) v_object 0xce922a68 ref 0 pages 0 lock type ufs: EXCL by thread 0xd16dd8d0 (pid 90313, truncate, tid 100870) #0 0xc0a783d5 at __lockmgr_args+0x10c5 #1 0xc0cef38d at ffs_lock+0xad #2 0xc0ecf1b2 at VOP_LOCK1_APV+0x112 #3 0xc0b46faa at _vn_lock+0xca #4 0xc0b49593 at vn_write+0x243 #5 0xc0b459eb at vn_io_fault+0x24b #6 0xc0ae5866 at dofilewrite+0x86 #7 0xc0ae5576 at kern_writev+0x66 #8 0xc0ae54fc at sys_write+0x5c #9 0xc0ea909d at syscall+0x2ed #10 0xc0e91631 at Xint0x80_syscall+0x21 ino 18818207, on dev label/tmp 0xd0b93000: tag ufs, type VREG usecount 1, writecount 1, refcount 1 mountedhere 0 flags (VV_NOSYNC|VI_ACTIVE) v_object 0xd64ec940 ref 0 pages 0 lock type ufs: EXCL by thread 0xca7dd5e0 (pid 90317, truncate, tid 100130) #0 0xc0a783d5 at __lockmgr_args+0x10c5 #1 0xc0cef38d at ffs_lock+0xad #2 0xc0ecf1b2 at VOP_LOCK1_APV+0x112 #3 0xc0b46faa at _vn_lock+0xca #4 0xc0b49593 at vn_write+0x243 #5 0xc0b459eb at vn_io_fault+0x24b #6 0xc0ae5866 at dofilewrite+0x86 #7 0xc0ae5576 at kern_writev+0x66 #8 0xc0ae54fc at sys_write+0x5c #9 0xc0ea909d at syscall+0x2ed #10 0xc0e91631 at Xint0x80_syscall+0x21 ino 18818219, on dev label/tmp 0xcd93aca8: tag ufs, type VREG usecount 1, writecount 1, refcount 1 mountedhere 0 flags (VV_NOSYNC|VI_ACTIVE) v_object 0xd675e000 ref 0 pages 0 lock type ufs: EXCL by thread 0xcb279bc0 (pid 90327, truncate, tid 100373) #0 0xc0a783d5 at __lockmgr_args+0x10c5 #1 0xc0cef38d at ffs_lock+0xad #2 0xc0ecf1b2 at VOP_LOCK1_APV+0x112 #3 0xc0b46faa at _vn_lock+0xca #4 0xc0b49593 at vn_write+0x243 #5 0xc0b459eb at vn_io_fault+0x24b #6 0xc0ae5866 at dofilewrite+0x86 #7 0xc0ae5576 at kern_writev+0x66 #8 0xc0ae54fc at sys_write+0x5c #9 0xc0ea909d at syscall+0x2ed #10 0xc0e91631 at Xint0x80_syscall+0x21 ino 18818243, on dev label/tmp db:0:lockedvnods> show mount 0xca705bb0 /dev/ad4s1a on / (ufs) 0xca706000 devfs on /dev (devfs) 0xcab425d8 /dev/ad4s1f on /home (ufs) 0xcab42000 /dev/ad4s1d on /usr (ufs) 0xca706bb0 /dev/ad4s1e on /var (ufs) 0xca7068c4 procfs on /proc (procfs) 0xca7065d8 /dev/label/tmp on /tmp (ufs) More info: show mount db:0:mount> ps pid ppid pgrp uid state wmesg wchan cmd 90914 835 834 1001 S nanslp 0xc1231e80 sleep 90899 2667 2667 1001 S+ nanslp 0xc1231e80 sleep 90760 90757 87815 0 D nokva 0xf97906d4 makewhatis 90757 87826 87815 0 S wait 0xd16d22d0 sh 90327 90307 814 0 R+ truncate 90326 90307 814 0 R+ truncate 90325 90307 814 0 R+ CPU 2 truncate 90324 90307 814 0 R+ truncate 90323 90307 814 0 R+ truncate 90322 90307 814 0 R+ CPU 0 truncate 90321 90307 814 0 R+ truncate 90320 90307 814 0 R+ truncate 90319 90307 814 0 R+ truncate 90318 90307 814 0 R+ CPU 3 truncate 90317 90307 814 0 R+ truncate 90316 90307 814 0 R+ truncate 90315 90307 814 0 R+ truncate 90314 90307 814 0 R+ truncate 90313 90307 814 0 R+ truncate 90312 90307 814 0 R+ truncate 90311 90307 814 0 R+ truncate 90310 90307 814 0 R+ truncate 90309 90307 814 0 R+ truncate 90308 90307 814 0 R+ truncate 90307 90279 814 0 S+ wait 0xd20e45a0 truncate 90279 814 814 0 S+ wait 0xcbf725a0 sh 87827 87819 87815 0 S piperd 0xcbdc3660 mail 87826 87819 87815 0 S wait 0xd16b3000 sh 87819 87818 87815 0 SW wait 0xd00d5b40 sh 87818 87815 87815 0 SW wait 0xcc130870 lockf 87815 87813 87815 0 SWs wait 0xcc130b40 sh 87813 744 744 0 S piperd 0xcacb7b28 cron 87143 87140 51849 0 S piperd 0xcbc4d7f8 wc 87142 87140 51849 0 S piperd 0xd5306660 tee 87141 87140 51849 0 D nokva 0xf9cdb6d4 find 87140 87136 51849 0 SW wait 0xcee42b40 sh 87136 4181 51849 0 S piperd 0xcbc4ccc0 sh 4234 76795 51849 0 S piperd 0xcbdc04c8 mail 4181 76795 51849 0 SW wait 0xcc0692d0 sh 76795 75987 51849 0 SW wait 0xca7ec2d0 sh 75987 74867 51849 0 SW wait 0xcb1a12d0 lockf 74867 73701 51849 0 SW wait 0xd1576000 sh 73701 90105 51849 0 SW wait 0xd0f14870 sh 90238 63408 51849 0 S piperd 0xca7ea000 mail 90105 63408 51849 0 SW wait 0xd176d5a0 sh 63408 62593 51849 0 SW wait 0xd00d5870 sh 62593 51849 51849 0 SW wait 0xd20e4870 lockf 51849 50776 51849 0 SWs wait 0xce7f55a0 sh 50776 744 744 0 S piperd 0xced4f660 cron 83682 0 0 0 DL (threaded) [zfskern] 100575 D l2arc_fe 0xd1ca9ddc [l2arc_feed_thread] 100561 D arc_recl 0xd1ca2d10 [arc_reclaim_thread] 2667 2424 2667 1001 S+ wait 0xcbc78870 sh 2424 2404 2424 1001 SWs+ wait 0xd0f142d0 bash 2404 2319 2319 1001 S select 0xca78d5a4 sshd 2319 733 2319 0 Ss select 0xcde79824 sshd 1079 1 1079 0 Ss+ ttyin 0xc80ba270 getty 1078 1 1078 0 Ss+ ttyin 0xc80ba470 getty 1077 1 1077 0 Ss+ ttyin 0xc7f07070 getty 1076 1 1076 0 Ss+ ttyin 0xc7f07270 getty 1075 1 1075 0 Ss+ ttyin 0xc7f07470 getty 1074 1 1074 0 Ss+ ttyin 0xc7f07670 getty 1073 1 1073 0 Ss+ ttyin 0xc7f07870 getty 1072 1 1072 0 Ss+ ttyin 0xc7f07a70 getty 1071 1 1071 0 Ss+ ttyin 0xc7f07c70 getty 1054 1 1054 0 Ss select 0xca78d4a4 inetd 1029 1 1029 0 Ss nanslp 0xc1231e80 watchdogd 842 841 842 1001 Ss kqread 0xcbbbd780 tail 841 829 829 1001 S select 0xca78d464 sshd 836 834 834 1001 S piperd 0xcbc3a660 awk 835 834 834 1001 S wait 0xcb1a3870 sh 834 833 834 1001 SWs wait 0xcb1a32d0 sh 833 826 826 1001 S select 0xca78d564 sshd 832 831 832 1001 Ss+ select 0xca78d324 top 831 825 825 1001 S select 0xca78d0e4 sshd 829 733 829 0 Ss select 0xca7f0da4 sshd 826 733 826 0 Ss select 0xca7f0d24 sshd 825 733 825 0 Ss select 0xc7de4b64 sshd 814 810 814 0 S+ wait 0xcb1a15a0 sh 810 807 810 0 SW+ wait 0xca7ecb40 bash 807 806 807 0 SW+ pause 0xca7efb98 csh 806 794 806 1001 SW+ wait 0xca78a5a0 su 794 793 794 1001 SWs+ wait 0xcb1a1870 bash 793 791 791 1001 S select 0xca623264 sshd 791 733 791 0 Ss select 0xca7f0ca4 sshd 744 1 744 0 Ss nanslp 0xc1231e80 cron 739 1 739 25 SWs pause 0xca74f328 sendmail 736 1 736 0 Ss select 0xca78e524 sendmail 733 1 733 0 Ss select 0xca78d424 sshd 708 683 683 0 S select 0xc7de4c64 winbindd 700 1 700 0 Ss select 0xca78e324 moused 699 681 681 0 S select 0xca751e24 smbd 683 1 683 0 Ss select 0xca78d124 winbindd 681 1 681 0 Ss select 0xc7de4c24 smbd 679 1 679 0 Ss select 0xca7f0aa4 nmbd 655 1 655 0 Ss select 0xca7f08e4 ntpd 602 596 596 0 S (threaded) nfsd 100111 S rpcsvc 0xca6c0090 nfsd: service 100110 S rpcsvc 0xca6c1810 nfsd: service 100109 S rpcsvc 0xca6c00d0 nfsd: service 100084 S rpcsvc 0xca60ea50 nfsd: master 596 1 596 0 Ss select 0xcaed93a4 nfsd 594 590 590 0 S select 0xca78d964 nfsuserd 593 590 590 0 S select 0xcaed94a4 nfsuserd 592 590 590 0 S select 0xca78ed24 nfsuserd 591 590 590 0 S select 0xcaed94e4 nfsuserd 590 1 590 0 SWs pause 0xca78a8c8 nfsuserd 574 1 574 0 Ss select 0xca623164 mountd 541 1 541 0 Ss select 0xca7519a4 rpcbind 526 1 526 0 Ss select 0xca751ca4 syslogd 423 1 423 0 Ss select 0xca78eda4 devd 18 0 0 0 RL [softdepflush] 17 0 0 0 RL CPU 1 [syncer] 16 0 0 0 DL vlruwt 0xca62c870 [vnlru] 9 0 0 0 DL psleep 0xc13a7ee8 [bufdaemon] 8 0 0 0 DL pgzero 0xc13acc3c [pagezero] 7 0 0 0 DL psleep 0xc13ac808 [vmdaemon] 6 0 0 0 DL psleep 0xc1400684 [pagedaemon] 5 0 0 0 DL ccb_scan 0xc11fd8a4 [xpt_thrd] 4 0 0 0 DL waiting_ 0xc13fc374 [sctp_iterator] 3 0 0 0 DL ctl_work 0xc80bc000 [ctl_thrd] 2 0 0 0 DL - 0xc801443c [fdc0] 15 0 0 0 DL (threaded) [usb] 100058 D - 0xc8081d34 [usbus5] 100057 D - 0xc8081d04 [usbus5] 100056 D - 0xc8081cd4 [usbus5] 100055 D - 0xc8081ca4 [usbus5] 100053 D - 0xc8074b5c [usbus4] 100052 D - 0xc8074b2c [usbus4] 100051 D - 0xc8074afc [usbus4] 100050 D - 0xc8074acc [usbus4] 100049 D - 0xc806eb5c [usbus3] 100048 D - 0xc806eb2c [usbus3] 100047 D - 0xc806eafc [usbus3] 100046 D - 0xc806eacc [usbus3] 100045 D - 0xc8067b5c [usbus2] 100044 D - 0xc8067b2c [usbus2] 100043 D - 0xc8067afc [usbus2] 100042 D - 0xc8067acc [usbus2] 100040 D - 0xc8062b5c [usbus1] 100039 D - 0xc8062b2c [usbus1] 100038 D - 0xc8062afc [usbus1] 100037 D - 0xc8062acc [usbus1] 100035 D - 0xc8056b5c [usbus0] 100034 D - 0xc8056b2c [usbus0] 100033 D - 0xc8056afc [usbus0] 100032 D - 0xc8056acc [usbus0] 14 0 0 0 DL - 0xc1230da0 [yarrow] 13 0 0 0 DL (threaded) [geom] 100016 D - 0xc13f84a0 [g_down] 100015 D - 0xc13f849c [g_up] 100014 D - 0xc13f8498 [g_event] 12 0 0 0 WL (threaded) [intr] 100063 I [irq12: psm0] 100062 I [irq1: atkbd0] 100060 I [swi0: uart] 100059 I [irq14: ata0] 100054 I [irq19: ehci0] 100041 I [irq18: ohci2 ohci4] 100036 I [irq17: ohci1 ohci3] 100031 I [irq16: hdac1 ohci0] 100030 I [irq22: ahci0] 100029 I [irq257: re0] 100028 I [irq256: hdac0] 100026 I [swi5: fast taskq] 100024 I [swi6: Giant taskq] 100023 I [swi6: task queue] 100018 I [swi2: cambio] 100012 I [swi1: netisr 0] 100011 I [swi3: vm] 100010 I [swi4: clock] 100009 I [swi4: clock] 100008 I [swi4: clock] 100007 I [swi4: clock] 11 0 0 0 RL (threaded) [idle] 100006 CanRun [idle: cpu3] 100005 CanRun [idle: cpu2] 100004 CanRun [idle: cpu1] 100003 CanRun [idle: cpu0] 1 0 1 0 SLs wait 0xc7ddcb40 [init] 10 0 0 0 DL audit_wo 0xc13ff13c [audit] 0 0 0 0 DLs (threaded) [kernel] 100574 D - 0xcdc42580 [system_taskq_3] 100573 D - 0xcdc42580 [system_taskq_2] 100572 D - 0xcdc42580 [system_taskq_1] 100571 D - 0xcdc42580 [system_taskq_0] 100066 D - 0xc1230da0 [deadlkres] 100065 D - 0xca33b580 [mca taskq] 100027 D - 0xc7fdf4c0 [kqueue taskq] 100025 D - 0xc7f622c0 [thread taskq] 100022 D - 0xc7f62380 [acpi_task_2] 100021 D - 0xc7f62380 [acpi_task_1] 100020 D - 0xc7f62380 [acpi_task_0] 100019 D - 0xc7f623c0 [ffs_trim taskq] 100013 D - 0xc7d7a7c0 [firmware taskq] 100000 D sched 0xc13f8514 [swapper] db:0:ps> allt Tracing command sleep pid 90914 tid 100881 td 0xd58378d0 sched_switch(d58378d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9c76a80 mi_switch(104,0,c108af51,1f7,c13f9900,...) at mi_switch+0x1c9/frame 0xf9c76ab4 sleepq_switch(d58378d0,0,c108af51,1ab,f9c76b2c,...) at sleepq_switch+0x1b9/frame 0xf9c76ae8 sleepq_catch_signals(2711,c0ad5e10,d58378d0,0,0,...) at sleepq_catch_signals+0x3b1/frame 0xf9c76b30 sleepq_timedwait_sig(c1231e80,6c,c1086a3e,100,0,...) at sleepq_timedwait_sig+0x14/frame 0xf9c76b58 _sleep(c1231e80,0,16c,c1086a3e,2711,...) at _sleep+0x303/frame 0xf9c76bb0 kern_nanosleep(bfbfdce0,f9c76c18,f9c76c20,2,a,...) at kern_nanosleep+0xd9/frame 0xf9c76c00 sys_nanosleep(d58378d0,f9c76cc8,f9c76d08,1,1,...) at sys_nanosleep+0x6b/frame 0xf9c76c40 syscall(f9c76d08) at syscall+0x2ed/frame 0xf9c76cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9c76cfc --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x2816337f, esp = 0xbfbfdcac, ebp = 0xbfbfdcf4 --- Tracing command sleep pid 90899 tid 100564 td 0xd16e35e0 sched_switch(d16e35e0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9e00a80 mi_switch(104,0,c108af51,1f7,c13f9c00,...) at mi_switch+0x1c9/frame 0xf9e00ab4 sleepq_switch(d16e35e0,0,c108af51,1ab,f9e00b2c,...) at sleepq_switch+0x1b9/frame 0xf9e00ae8 sleepq_catch_signals(7531,c0ad5e10,d16e35e0,3,3,...) at sleepq_catch_signals+0x3b1/frame 0xf9e00b30 sleepq_timedwait_sig(c1231e80,6c,c1086a3e,100,0,...) at sleepq_timedwait_sig+0x14/frame 0xf9e00b58 _sleep(c1231e80,0,16c,c1086a3e,7531,...) at _sleep+0x303/frame 0xf9e00bb0 kern_nanosleep(bfbfdb20,f9e00c18,f9e00c20,2,1e,...) at kern_nanosleep+0xd9/frame 0xf9e00c00 sys_nanosleep(d16e35e0,f9e00cc8,c10dc7ab,d6,7b3,...) at sys_nanosleep+0x6b/frame 0xf9e00c40 syscall(f9e00d08) at syscall+0x2ed/frame 0xf9e00cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9e00cfc --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x2816337f, esp = 0xbfbfdaec, ebp = 0xbfbfdb34 --- Tracing command makewhatis pid 90760 tid 100392 td 0xd54535e0 sched_switch(d54535e0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf97905d0 mi_switch(104,0,c108af51,1f7,1,...) at mi_switch+0x1c9/frame 0xf9790604 sleepq_switch(d54535e0,0,c108af51,290,d54535e0,...) at sleepq_switch+0x1b9/frame 0xf9790638 sleepq_timedwait(f97906d4,0,c10c0ab0,0,0,...) at sleepq_timedwait+0x48/frame 0xf979065c _sleep(f97906d4,0,0,c10c0ab0,4e2,...) at _sleep+0x317/frame 0xf97906b4 kmem_malloc(c1bb508c,1000,102) at kmem_malloc+0xe9/frame 0xf97906e8 page_alloc(ca54fe00,1000,f9790727,102,ca4b6e98,...) at page_alloc+0x27/frame 0xf97906fc keg_alloc_slab(102,4,c10bfd74,887,f9790780,...) at keg_alloc_slab+0xc3/frame 0xf9790734 keg_fetch_slab(102,80,102,cbd9ec48,f97907c8,...) at keg_fetch_slab+0x132/frame 0xf9790768 zone_fetch_slab(ca54fe00,0,102,983,202,...) at zone_fetch_slab+0x3f/frame 0xf9790780 uma_zalloc_arg(ca54fe00,0,102,d54535e0,f97908cc,...) at uma_zalloc_arg+0x435/frame 0xf97907c8 ffs_vgetf(cab42000,14ecca,200000,f97908cc,0) at ffs_vgetf+0xa9/frame 0xf9790824 ffs_vget(cab42000,14ecca,200000,f97908cc,f97908dc,...) at ffs_vget+0x2e/frame 0xf9790840 ufs_lookup_ino(e11dfca8,f9790b5c,f9790b70,0) at ufs_lookup_ino+0xe27/frame 0xf9790908 ufs_lookup(f9790970,c10e2f12,c0b24618,e11dfcdc,f9790968,...) at ufs_lookup+0x2a/frame 0xf9790920 VOP_CACHEDLOOKUP_APV(c11e2de8,f9790970,f9790b70,0,0,...) at VOP_CACHEDLOOKUP_APV+0x124/frame 0xf9790950 vfs_cache_lookup(f9790a00,c10e2eb1,c10b7f3d,c1095b77,c11f4a90,...) at vfs_cache_lookup+0xf0/frame 0xf9790998 VOP_LOOKUP_APV(c11e2de8,f9790a00,f9790b70,31d,ca3d3d48,...) at VOP_LOOKUP_APV+0x124/frame 0xf97909c8 lookup(f9790b30,c1094fbb,106,ce,0,...) at lookup+0x52e/frame 0xf9790a28 namei(f9790b30,ffffffdf,d1a0d580,0,d54535e0,...) at namei+0x47d/frame 0xf9790ab0 kern_statat_vnhook(d54535e0,0,ffffff9c,2882a680,0,...) at kern_statat_vnhook+0xbd/frame 0xf9790bb0 sys_stat(d54535e0,f9790cc8,f9790d08,1,1,...) at sys_stat+0x4d/frame 0xf9790c40 syscall(f9790d08) at syscall+0x2ed/frame 0xf9790cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9790cfc --- syscall (188, FreeBSD ELF32, sys_stat), eip = 0x2818aef3, esp = 0xbfbfbf2c, ebp = 0xbfbfd400 --- Tracing command sh pid 90757 tid 100576 td 0xca6808d0 sched_switch(ca6808d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9817998 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf98179cc sleepq_switch(ca6808d0,0,c108af51,1ab,f9817a44,...) at sleepq_switch+0x1b9/frame 0xf9817a00 sleepq_catch_signals(0,c108af51,164,330,ca6808d0,...) at sleepq_catch_signals+0x3b1/frame 0xf9817a48 sleepq_wait_sig(d16d22d0,6c,c108e435,100,0,...) at sleepq_wait_sig+0x14/frame 0xf9817a6c _sleep(d16d22d0,d16d2358,16c,c108e435,0,...) at _sleep+0x32d/frame 0xf9817ac4 kern_wait6(ca6808d0,7,0,0,f9817b4c,...) at kern_wait6+0x43c/frame 0xf9817b20 sys_wait4(ca6808d0,f9817cc8,c10dc7ab,e9,1,...) at sys_wait4+0x94/frame 0xf9817c40 syscall(f9817d08) at syscall+0x2ed/frame 0xf9817cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9817cfc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2815098f, esp = 0xbfbfda38, ebp = 0xbfbfda50 --- Tracing command truncate pid 90327 tid 100373 td 0xcb279bc0 sched_switch(cb279bc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9b11460 mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xf9b11494 sleepq_switch(cb279bc0,0,c108af51,26d,0,...) at sleepq_switch+0x1b9/frame 0xf9b114c8 sleepq_wait(e1aaa734,60,c107cce6,4,0,...) at sleepq_wait+0x48/frame 0xf9b114ec sleeplk(ca7c7ed0,c107cce6,60,0,0,...) at sleeplk+0x11e/frame 0xf9b11528 __lockmgr_args(e1aaa734,81900,ca7c7ed0,c107cce6,60,...) at __lockmgr_args+0xaa4/frame 0xf9b11608 getblk(ca7c7e10,11f224c0,0,4000,0,...) at getblk+0x1d8/frame 0xf9b11678 breadn_flags(ca7c7e10,11f224c0,0,4000,0,...) at breadn_flags+0x40/frame 0xf9b116a8 ffs_alloccg(cdbcfc3c,320,0,0,800,...) at ffs_alloccg+0x126/frame 0xf9b11714 ffs_hashalloc(47b1980,0,800,800,c0cb9870,...) at ffs_hashalloc+0xce/frame 0xf9b11748 ffs_alloc(cdbcfc3c,0,0,47b1980,0,...) at ffs_alloc+0x29d/frame 0xf9b117b0 ffs_balloc_ufs2(cd93aca8,0,0,2,ca6ef580,...) at ffs_balloc_ufs2+0x124b/frame 0xf9b118ec ffs_write(f9b11a80,c10e32d1,f9b119d0,c0cef38d,c11f4a90,...) at ffs_write+0x346/frame 0xf9b11980 VOP_WRITE_APV(c11e2de8,f9b11a80,cd93aca8,31b,cb279bc0,...) at VOP_WRITE_APV+0x19f/frame 0xf9b11a40 vn_write(ca7e2c78,f9b11c10,ca6ef580,1,cb279bc0,...) at vn_write+0x379/frame 0xf9b11ab0 vn_io_fault(ca7e2c78,f9b11c10,ca6ef580,0,cb279bc0,...) at vn_io_fault+0x24b/frame 0xf9b11b8c dofilewrite(ca7e2c78,f9b11c10,ffffffff,ffffffff,0,...) at dofilewrite+0x86/frame 0xf9b11bc4 kern_writev(cb279bc0,b,f9b11c10,0,8049b04,...) at kern_writev+0x66/frame 0xf9b11bf0 sys_write(cb279bc0,f9b11cc8,14,c1087885,7b3,...) at sys_write+0x5c/frame 0xf9b11c40 syscall(f9b11d08) at syscall+0x2ed/frame 0xf9b11cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9b11cfc --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28185fa7, esp = 0xbfbfd7e8, ebp = 0xbfbfd8b0 --- Tracing command truncate pid 90326 tid 100395 td 0xcb25a2f0 sched_switch(cb25a2f0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9b90460 mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xf9b90494 sleepq_switch(cb25a2f0,0,c108af51,26d,0,...) at sleepq_switch+0x1b9/frame 0xf9b904c8 sleepq_wait(e1aaa734,60,c107cce6,4,0,...) at sleepq_wait+0x48/frame 0xf9b904ec sleeplk(ca7c7ed0,c107cce6,60,0,0,...) at sleeplk+0x11e/frame 0xf9b90528 __lockmgr_args(e1aaa734,81900,ca7c7ed0,c107cce6,60,...) at __lockmgr_args+0xaa4/frame 0xf9b90608 getblk(ca7c7e10,11f224c0,0,4000,0,...) at getblk+0x1d8/frame 0xf9b90678 breadn_flags(ca7c7e10,11f224c0,0,4000,0,...) at breadn_flags+0x40/frame 0xf9b906a8 ffs_alloccg(d12a7000,320,0,0,800,...) at ffs_alloccg+0x126/frame 0xf9b90714 ffs_hashalloc(47b1980,0,800,800,c0cb9870,...) at ffs_hashalloc+0xce/frame 0xf9b90748 ffs_alloc(d12a7000,0,0,47b1980,0,...) at ffs_alloc+0x29d/frame 0xf9b907b0 ffs_balloc_ufs2(cc861438,0,0,2,ca6ef580,...) at ffs_balloc_ufs2+0x124b/frame 0xf9b908ec ffs_write(f9b90a80,c10e32d1,f9b909d0,c0cef38d,c11f4a90,...) at ffs_write+0x346/frame 0xf9b90980 VOP_WRITE_APV(c11e2de8,f9b90a80,cc861438,31b,cb25a2f0,...) at VOP_WRITE_APV+0x19f/frame 0xf9b90a40 vn_write(d2d1cdc8,f9b90c10,ca6ef580,1,cb25a2f0,...) at vn_write+0x379/frame 0xf9b90ab0 vn_io_fault(d2d1cdc8,f9b90c10,ca6ef580,0,cb25a2f0,...) at vn_io_fault+0x24b/frame 0xf9b90b8c dofilewrite(d2d1cdc8,f9b90c10,ffffffff,ffffffff,0,...) at dofilewrite+0x86/frame 0xf9b90bc4 kern_writev(cb25a2f0,8,f9b90c10,0,8049b04,...) at kern_writev+0x66/frame 0xf9b90bf0 sys_write(cb25a2f0,f9b90cc8,f9b90d08,1,1,...) at sys_write+0x5c/frame 0xf9b90c40 syscall(f9b90d08) at syscall+0x2ed/frame 0xf9b90cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9b90cfc --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28185fa7, esp = 0xbfbfd7e8, ebp = 0xbfbfd8b0 --- Tracing command truncate pid 90325 tid 100393 td 0xcb25a8d0 cpustop_handler(2,13,f9b8a834,c0ea7c8b,ca7c7ed0,...) at cpustop_handler+0x2f/frame 0xf9b8a6e8 ipi_nmi_handler(ca7c7ed0,c0eb02e6,f9b8a728,0,cb25a8d0,...) at ipi_nmi_handler+0x37/frame 0xf9b8a6f8 trap(f9b8a840) at trap+0x3b/frame 0xf9b8a834 calltrap() at calltrap+0x6/frame 0xf9b8a834 --- trap 0x13, eip = 0xc0a80400, esp = 0xf9b8a880, ebp = 0xf9b8a8ac --- __mtx_lock_flags(d90912f4,0,c10c414e,17e,cb25a8d0,...) at __mtx_lock_flags+0x10/frame 0xf9b8a8ac vnode_pager_setsize(cf6bb5a0,2,0,d4385304,c1094c23,...) at vnode_pager_setsize+0x3d/frame 0xf9b8a8ec ffs_write(f9b8aa80,c10e32d1,f9b8a9d0,c0cef38d,c11f4a90,...) at ffs_write+0x2da/frame 0xf9b8a980 VOP_WRITE_APV(c11e2de8,f9b8aa80,cf6bb5a0,31b,cb25a8d0,...) at VOP_WRITE_APV+0x19f/frame 0xf9b8aa40 vn_write(d3d80b28,f9b8ac10,ca6ef580,1,cb25a8d0,...) at vn_write+0x379/frame 0xf9b8aab0 vn_io_fault(d3d80b28,f9b8ac10,ca6ef580,0,cb25a8d0,...) at vn_io_fault+0x24b/frame 0xf9b8ab8c dofilewrite(d3d80b28,f9b8ac10,ffffffff,ffffffff,0,...) at dofilewrite+0x86/frame 0xf9b8abc4 kern_writev(cb25a8d0,7,f9b8ac10,0,8049b04,...) at kern_writev+0x66/frame 0xf9b8abf0 sys_write(cb25a8d0,f9b8acc8,14,c1087885,7b3,...) at sys_write+0x5c/frame 0xf9b8ac40 syscall(f9b8ad08) at syscall+0x2ed/frame 0xf9b8acfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9b8acfc --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28185fa7, esp = 0xbfbfd7e8, ebp = 0xbfbfd8b0 --- Tracing command truncate pid 90324 tid 100208 td 0xcbbba2f0 sched_switch(cbbba2f0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf98da460 mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xf98da494 sleepq_switch(cbbba2f0,0,c108af51,26d,0,...) at sleepq_switch+0x1b9/frame 0xf98da4c8 sleepq_wait(e1aaa734,60,c107cce6,4,0,...) at sleepq_wait+0x48/frame 0xf98da4ec sleeplk(ca7c7ed0,c107cce6,60,0,0,...) at sleeplk+0x11e/frame 0xf98da528 __lockmgr_args(e1aaa734,81900,ca7c7ed0,c107cce6,60,...) at __lockmgr_args+0xaa4/frame 0xf98da608 getblk(ca7c7e10,11f224c0,0,4000,0,...) at getblk+0x1d8/frame 0xf98da678 breadn_flags(ca7c7e10,11f224c0,0,4000,0,...) at breadn_flags+0x40/frame 0xf98da6a8 ffs_alloccg(d1db26cc,320,0,0,800,...) at ffs_alloccg+0x126/frame 0xf98da714 ffs_hashalloc(47b1980,0,800,800,c0cb9870,...) at ffs_hashalloc+0xce/frame 0xf98da748 ffs_alloc(d1db26cc,0,0,47b1980,0,...) at ffs_alloc+0x29d/frame 0xf98da7b0 ffs_balloc_ufs2(d90e8b40,0,0,2,ca6ef580,...) at ffs_balloc_ufs2+0x124b/frame 0xf98da8ec ffs_write(f98daa80,c10e32d1,f98da9d0,c0cef38d,c11f4a90,...) at ffs_write+0x346/frame 0xf98da980 VOP_WRITE_APV(c11e2de8,f98daa80,d90e8b40,31b,cbbba2f0,...) at VOP_WRITE_APV+0x19f/frame 0xf98daa40 vn_write(d61465b0,f98dac10,ca6ef580,1,cbbba2f0,...) at vn_write+0x379/frame 0xf98daab0 vn_io_fault(d61465b0,f98dac10,ca6ef580,0,cbbba2f0,...) at vn_io_fault+0x24b/frame 0xf98dab8c dofilewrite(d61465b0,f98dac10,ffffffff,ffffffff,0,...) at dofilewrite+0x86/frame 0xf98dabc4 kern_writev(cbbba2f0,7,f98dac10,0,8049b04,...) at kern_writev+0x66/frame 0xf98dabf0 sys_write(cbbba2f0,f98dacc8,14,c1087885,7b3,...) at sys_write+0x5c/frame 0xf98dac40 syscall(f98dad08) at syscall+0x2ed/frame 0xf98dacfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf98dacfc --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28185fa7, esp = 0xbfbfd7e8, ebp = 0xbfbfd8b0 --- Tracing command truncate pid 90323 tid 100394 td 0xcb25a5e0 sched_switch(cb25a5e0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9b8d620 mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xf9b8d654 sleepq_switch(cb25a5e0,0,c108af51,26d,0,...) at sleepq_switch+0x1b9/frame 0xf9b8d688 sleepq_wait(e1a2beac,60,c107cce6,4,0,...) at sleepq_wait+0x48/frame 0xf9b8d6ac sleeplk(ca7c7ed0,c107cce6,60,0,0,...) at sleeplk+0x11e/frame 0xf9b8d6e8 __lockmgr_args(e1a2beac,81900,ca7c7ed0,c107cce6,60,...) at __lockmgr_args+0xaa4/frame 0xf9b8d7c8 getblk(ca7c7e10,11ec66e0,0,4000,0,...) at getblk+0x1d8/frame 0xf9b8d838 breadn_flags(ca7c7e10,11ec66e0,0,4000,0,...) at breadn_flags+0x40/frame 0xf9b8d868 ffs_update(d0bde5a0,0,0,800,246,...) at ffs_update+0x191/frame 0xf9b8d8e0 ffs_truncate(d0bde5a0,0,0,800,ca6ef580,...) at ffs_truncate+0x49d/frame 0xf9b8da98 ufs_setattr(f9b8dbd8,c10e31e2,c10b8120,c1097510,c11f4a90,...) at ufs_setattr+0x730/frame 0xf9b8db10 VOP_SETATTR_APV(c11e2de8,f9b8dbd8,d0bde5a0,4a2,7fffffff,...) at VOP_SETATTR_APV+0x124/frame 0xf9b8db40 vn_truncate(d39937e0,0,0,ca6ef580,cb25a5e0,...) at vn_truncate+0x1fa/frame 0xf9b8dbfc kern_ftruncate(cb25a5e0,7,0,0) at kern_ftruncate+0xa9/frame 0xf9b8dc28 sys_ftruncate(cb25a5e0,f9b8dcc8,14,c1087885,7b3,...) at sys_ftruncate+0x28/frame 0xf9b8dc40 syscall(f9b8dd08) at syscall+0x2ed/frame 0xf9b8dcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9b8dcfc --- syscall (480, FreeBSD ELF32, sys_ftruncate), eip = 0x2816c303, esp = 0xbfbfd7e8, ebp = 0xbfbfd8b0 --- Tracing command truncate pid 90322 tid 100519 td 0xd1744000 cpustop_handler(0,13,f9d4d630,c0ea7c8b,ca7c7ee0,...) at cpustop_handler+0x2f/frame 0xf9d4d4e0 ipi_nmi_handler(ca7c7ee0,d1744000,d1744000,e1aaa754,d1744000,...) at ipi_nmi_handler+0x37/frame 0xf9d4d4f0 trap(f9d4d63c) at trap+0x3b/frame 0xf9d4d630 calltrap() at calltrap+0x6/frame 0xf9d4d630 --- trap 0x13, eip = 0xc0a80981, esp = 0xf9d4d67c, ebp = 0xf9d4d6a8 --- __mtx_unlock_flags(e1aaa6d4,ca7065d8,47d8761,0,1,...) at __mtx_unlock_flags+0x1/frame 0xf9d4d6a8 ffs_alloccg(d3f1715c,320,0,0,800,...) at ffs_alloccg+0x3fe/frame 0xf9d4d714 ffs_hashalloc(47b1980,0,800,800,c0cb9870,...) at ffs_hashalloc+0xce/frame 0xf9d4d748 ffs_alloc(d3f1715c,0,0,47b1980,0,...) at ffs_alloc+0x29d/frame 0xf9d4d7b0 ffs_balloc_ufs2(d0db7438,0,0,2,ca6ef580,...) at ffs_balloc_ufs2+0x124b/frame 0xf9d4d8ec ffs_write(f9d4da80,c10e32d1,f9d4d9d0,c0cef38d,c11f4a90,...) at ffs_write+0x346/frame 0xf9d4d980 VOP_WRITE_APV(c11e2de8,f9d4da80,d0db7438,31b,d1744000,...) at VOP_WRITE_APV+0x19f/frame 0xf9d4da40 vn_write(d3c8c2a0,f9d4dc10,ca6ef580,1,d1744000,...) at vn_write+0x379/frame 0xf9d4dab0 vn_io_fault(d3c8c2a0,f9d4dc10,ca6ef580,0,d1744000,...) at vn_io_fault+0x24b/frame 0xf9d4db8c dofilewrite(d3c8c2a0,f9d4dc10,ffffffff,ffffffff,0,...) at dofilewrite+0x86/frame 0xf9d4dbc4 kern_writev(d1744000,7,f9d4dc10,0,8049b04,...) at kern_writev+0x66/frame 0xf9d4dbf0 sys_write(d1744000,f9d4dcc8,f9d4dd08,1,1,...) at sys_write+0x5c/frame 0xf9d4dc40 syscall(f9d4dd08) at syscall+0x2ed/frame 0xf9d4dcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9d4dcfc --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28185fa7, esp = 0xbfbfd7e8, ebp = 0xbfbfd8b0 --- Tracing command truncate pid 90321 tid 100307 td 0xd5a595e0 sched_switch(d5a595e0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf97a8460 mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xf97a8494 sleepq_switch(d5a595e0,0,c108af51,26d,0,...) at sleepq_switch+0x1b9/frame 0xf97a84c8 sleepq_wait(e1aaa734,60,c107cce6,4,0,...) at sleepq_wait+0x48/frame 0xf97a84ec sleeplk(ca7c7ed0,c107cce6,60,0,0,...) at sleeplk+0x11e/frame 0xf97a8528 __lockmgr_args(e1aaa734,81900,ca7c7ed0,c107cce6,60,...) at __lockmgr_args+0xaa4/frame 0xf97a8608 getblk(ca7c7e10,11f224c0,0,4000,0,...) at getblk+0x1d8/frame 0xf97a8678 breadn_flags(ca7c7e10,11f224c0,0,4000,0,...) at breadn_flags+0x40/frame 0xf97a86a8 ffs_alloccg(d4a51c3c,320,0,0,800,...) at ffs_alloccg+0x126/frame 0xf97a8714 ffs_hashalloc(47b1980,0,800,800,c0cb9870,...) at ffs_hashalloc+0xce/frame 0xf97a8748 ffs_alloc(d4a51c3c,0,0,47b1980,0,...) at ffs_alloc+0x29d/frame 0xf97a87b0 ffs_balloc_ufs2(d4385870,0,0,2,ca6ef580,...) at ffs_balloc_ufs2+0x124b/frame 0xf97a88ec ffs_write(f97a8a80,c10e32d1,f97a89d0,c0cef38d,c11f4a90,...) at ffs_write+0x346/frame 0xf97a8980 VOP_WRITE_APV(c11e2de8,f97a8a80,d4385870,31b,d5a595e0,...) at VOP_WRITE_APV+0x19f/frame 0xf97a8a40 vn_write(cbc4a3f0,f97a8c10,ca6ef580,1,d5a595e0,...) at vn_write+0x379/frame 0xf97a8ab0 vn_io_fault(cbc4a3f0,f97a8c10,ca6ef580,0,d5a595e0,...) at vn_io_fault+0x24b/frame 0xf97a8b8c dofilewrite(cbc4a3f0,f97a8c10,ffffffff,ffffffff,0,...) at dofilewrite+0x86/frame 0xf97a8bc4 kern_writev(d5a595e0,9,f97a8c10,0,8049b04,...) at kern_writev+0x66/frame 0xf97a8bf0 sys_write(d5a595e0,f97a8cc8,14,c1087885,7b3,...) at sys_write+0x5c/frame 0xf97a8c40 syscall(f97a8d08) at syscall+0x2ed/frame 0xf97a8cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf97a8cfc --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28185fa7, esp = 0xbfbfd7e8, ebp = 0xbfbfd8b0 --- Tracing command truncate pid 90320 tid 100441 td 0xcb2878d0 sched_switch(cb2878d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9c21460 mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xf9c21494 sleepq_switch(cb2878d0,0,c108af51,26d,0,...) at sleepq_switch+0x1b9/frame 0xf9c214c8 sleepq_wait(e1aaa734,60,c107cce6,4,0,...) at sleepq_wait+0x48/frame 0xf9c214ec sleeplk(ca7c7ed0,c107cce6,60,0,0,...) at sleeplk+0x11e/frame 0xf9c21528 __lockmgr_args(e1aaa734,81900,ca7c7ed0,c107cce6,60,...) at __lockmgr_args+0xaa4/frame 0xf9c21608 getblk(ca7c7e10,11f224c0,0,4000,0,...) at getblk+0x1d8/frame 0xf9c21678 breadn_flags(ca7c7e10,11f224c0,0,4000,0,...) at breadn_flags+0x40/frame 0xf9c216a8 ffs_alloccg(d4a519f8,320,0,0,800,...) at ffs_alloccg+0x126/frame 0xf9c21714 ffs_hashalloc(47b1980,0,800,800,c0cb9870,...) at ffs_hashalloc+0xce/frame 0xf9c21748 ffs_alloc(d4a519f8,0,0,47b1980,0,...) at ffs_alloc+0x29d/frame 0xf9c217b0 ffs_balloc_ufs2(cc6e5870,0,0,2,ca6ef580,...) at ffs_balloc_ufs2+0x124b/frame 0xf9c218ec ffs_write(f9c21a80,c10e32d1,f9c219d0,c0cef38d,c11f4a90,...) at ffs_write+0x346/frame 0xf9c21980 VOP_WRITE_APV(c11e2de8,f9c21a80,cc6e5870,31b,cb2878d0,...) at VOP_WRITE_APV+0x19f/frame 0xf9c21a40 vn_write(d5966000,f9c21c10,ca6ef580,1,cb2878d0,...) at vn_write+0x379/frame 0xf9c21ab0 vn_io_fault(d5966000,f9c21c10,ca6ef580,0,cb2878d0,...) at vn_io_fault+0x24b/frame 0xf9c21b8c dofilewrite(d5966000,f9c21c10,ffffffff,ffffffff,0,...) at dofilewrite+0x86/frame 0xf9c21bc4 kern_writev(cb2878d0,9,f9c21c10,0,8049b04,...) at kern_writev+0x66/frame 0xf9c21bf0 sys_write(cb2878d0,f9c21cc8,14,c1087885,c7b3bc80,...) at sys_write+0x5c/frame 0xf9c21c40 syscall(f9c21d08) at syscall+0x2ed/frame 0xf9c21cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9c21cfc --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28185fa7, esp = 0xbfbfd7e8, ebp = 0xbfbfd8b0 --- Tracing command truncate pid 90319 tid 100257 td 0xcaeb3bc0 sched_switch(caeb3bc0,0,602,1ae,f996d63c,...) at sched_switch+0x4b4/frame 0xf996d618 mi_switch(602,0,c1085aac,d3,caeb3bc0,...) at mi_switch+0x1c9/frame 0xf996d64c critical_exit(caeb3bc0,0,1) at critical_exit+0xa3/frame 0xf996d668 ipi_bitmap_handler(8,28,28,0,0,...) at ipi_bitmap_handler+0x6b/frame 0xf996d688 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0x2f/frame 0xf996d688 --- interrupt, eip = 0xc0a80788, esp = 0xf996d6d0, ebp = 0xf996d738 --- __mtx_lock_sleep(c13ac2e4,caeb3bc0,c10ba87e,c10ba87e,13e4,...) at __mtx_lock_sleep+0x208/frame 0xf996d738 __mtx_lock_flags(c13ac2e4,0,c10ba87e,13e4,c1092ce2,...) at __mtx_lock_flags+0xfd/frame 0xf996d76c softdep_setup_allocdirect(ce2e6910,0,0,47d8742,0,...) at softdep_setup_allocdirect+0xa5/frame 0xf996d7b0 ffs_balloc_ufs2(d2da6b40,0,0,2,ca6ef580,...) at ffs_balloc_ufs2+0x135f/frame 0xf996d8ec ffs_write(f996da80,c10e32d1,f996d9d0,c0cef38d,c11f4a90,...) at ffs_write+0x346/frame 0xf996d980 VOP_WRITE_APV(c11e2de8,f996da80,d2da6b40,31b,caeb3bc0,...) at VOP_WRITE_APV+0x19f/frame 0xf996da40 vn_write(d2d1c0a8,f996dc10,ca6ef580,1,caeb3bc0,...) at vn_write+0x379/frame 0xf996dab0 vn_io_fault(d2d1c0a8,f996dc10,ca6ef580,0,caeb3bc0,...) at vn_io_fault+0x24b/frame 0xf996db8c dofilewrite(d2d1c0a8,f996dc10,ffffffff,ffffffff,0,...) at dofilewrite+0x86/frame 0xf996dbc4 kern_writev(caeb3bc0,a,f996dc10,0,8049b04,...) at kern_writev+0x66/frame 0xf996dbf0 sys_write(caeb3bc0,f996dcc8,14,c1087885,7b3,...) at sys_write+0x5c/frame 0xf996dc40 syscall(f996dd08) at syscall+0x2ed/frame 0xf996dcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf996dcfc --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28185fa7, esp = 0xbfbfd7e8, ebp = 0xbfbfd8b0 --- Tracing command truncate pid 90318 tid 100513 td 0xd16e95e0 cpustop_handler(3,13,f9d36a74,c0ea7c8b,f9d3696c,...) at cpustop_handler+0x2f/frame 0xf9d36928 ipi_nmi_handler(f9d3696c,60,0,0,d16e95e0,...) at ipi_nmi_handler+0x37/frame 0xf9d36938 trap(f9d36a80) at trap+0x3b/frame 0xf9d36a74 calltrap() at calltrap+0x6/frame 0xf9d36a74 --- trap 0x13, eip = 0xc0ecb990, esp = 0xf9d36ac0, ebp = 0xf9d36ae8 --- VOP_ISLOCKED_APV(c11e2de8,f9d36b00,80400,d12c82d0,c11f4a90,...) at VOP_ISLOCKED_APV+0x60/frame 0xf9d36ae8 assert_vop_locked(d12c82d0,c10b8120,c1097510,4a2,0,...) at assert_vop_locked+0x5f/frame 0xf9d36b14 mac_vnode_check_write(ca6ef580,ca6ef580,d12c82d0,4a2,7fffffff,...) at mac_vnode_check_write+0x1c/frame 0xf9d36b40 vn_truncate(d3993498,0,0,ca6ef580,d16e95e0,...) at vn_truncate+0x15c/frame 0xf9d36bfc kern_ftruncate(d16e95e0,5,0,0) at kern_ftruncate+0xa9/frame 0xf9d36c28 sys_ftruncate(d16e95e0,f9d36cc8,f9d36cc0,c0a81712,2710,...) at sys_ftruncate+0x28/frame 0xf9d36c40 syscall(f9d36d08) at syscall+0x2ed/frame 0xf9d36cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9d36cfc --- syscall (480, FreeBSD ELF32, sys_ftruncate), eip = 0x2816c303, esp = 0xbfbfd7e8, ebp = 0xbfbfd8b0 --- Tracing command truncate pid 90317 tid 100130 td 0xca7dd5e0 sched_switch(ca7dd5e0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf97a2460 mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xf97a2494 sleepq_switch(ca7dd5e0,0,c108af51,26d,0,...) at sleepq_switch+0x1b9/frame 0xf97a24c8 sleepq_wait(e1aaa734,60,c107cce6,4,0,...) at sleepq_wait+0x48/frame 0xf97a24ec sleeplk(ca7c7ed0,c107cce6,60,0,0,...) at sleeplk+0x11e/frame 0xf97a2528 __lockmgr_args(e1aaa734,81900,ca7c7ed0,c107cce6,60,...) at __lockmgr_args+0xaa4/frame 0xf97a2608 getblk(ca7c7e10,11f224c0,0,4000,0,...) at getblk+0x1d8/frame 0xf97a2678 breadn_flags(ca7c7e10,11f224c0,0,4000,0,...) at breadn_flags+0x40/frame 0xf97a26a8 ffs_alloccg(cfa1c074,320,0,0,800,...) at ffs_alloccg+0x126/frame 0xf97a2714 ffs_hashalloc(47b1980,0,800,800,c0cb9870,...) at ffs_hashalloc+0xce/frame 0xf97a2748 ffs_alloc(cfa1c074,0,0,47b1980,0,...) at ffs_alloc+0x29d/frame 0xf97a27b0 ffs_balloc_ufs2(d0b93000,0,0,2,ca6ef580,...) at ffs_balloc_ufs2+0x124b/frame 0xf97a28ec ffs_write(f97a2a80,c10e32d1,f97a29d0,c0cef38d,c11f4a90,...) at ffs_write+0x346/frame 0xf97a2980 VOP_WRITE_APV(c11e2de8,f97a2a80,d0b93000,31b,ca7dd5e0,...) at VOP_WRITE_APV+0x19f/frame 0xf97a2a40 vn_write(d3c8cc78,f97a2c10,ca6ef580,1,ca7dd5e0,...) at vn_write+0x379/frame 0xf97a2ab0 vn_io_fault(d3c8cc78,f97a2c10,ca6ef580,0,ca7dd5e0,...) at vn_io_fault+0x24b/frame 0xf97a2b8c dofilewrite(d3c8cc78,f97a2c10,ffffffff,ffffffff,0,...) at dofilewrite+0x86/frame 0xf97a2bc4 kern_writev(ca7dd5e0,b,f97a2c10,0,8049b04,...) at kern_writev+0x66/frame 0xf97a2bf0 sys_write(ca7dd5e0,f97a2cc8,14,c1087885,7b3,...) at sys_write+0x5c/frame 0xf97a2c40 syscall(f97a2d08) at syscall+0x2ed/frame 0xf97a2cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf97a2cfc --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28185fa7, esp = 0xbfbfd7e8, ebp = 0xbfbfd8b0 --- Tracing command truncate pid 90316 tid 100862 td 0xd56e28d0 sched_switch(d56e28d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9bba460 mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xf9bba494 sleepq_switch(d56e28d0,0,c108af51,26d,0,...) at sleepq_switch+0x1b9/frame 0xf9bba4c8 sleepq_wait(e1aaa734,60,c107cce6,4,0,...) at sleepq_wait+0x48/frame 0xf9bba4ec sleeplk(ca7c7ed0,c107cce6,60,0,0,...) at sleeplk+0x11e/frame 0xf9bba528 __lockmgr_args(e1aaa734,81900,ca7c7ed0,c107cce6,60,...) at __lockmgr_args+0xaa4/frame 0xf9bba608 getblk(ca7c7e10,11f224c0,0,4000,0,...) at getblk+0x1d8/frame 0xf9bba678 breadn_flags(ca7c7e10,11f224c0,0,4000,0,...) at breadn_flags+0x40/frame 0xf9bba6a8 ffs_alloccg(d22d5910,320,0,0,800,...) at ffs_alloccg+0x126/frame 0xf9bba714 ffs_hashalloc(47b1980,0,800,800,c0cb9870,...) at ffs_hashalloc+0xce/frame 0xf9bba748 ffs_alloc(d22d5910,0,0,47b1980,0,...) at ffs_alloc+0x29d/frame 0xf9bba7b0 ffs_balloc_ufs2(d211b5a0,0,0,2,ca6ef580,...) at ffs_balloc_ufs2+0x124b/frame 0xf9bba8ec ffs_write(f9bbaa80,c10e32d1,f9bba9d0,c0cef38d,c11f4a90,...) at ffs_write+0x346/frame 0xf9bba980 VOP_WRITE_APV(c11e2de8,f9bbaa80,d211b5a0,31b,d56e28d0,...) at VOP_WRITE_APV+0x19f/frame 0xf9bbaa40 vn_write(d3c8ca48,f9bbac10,ca6ef580,1,d56e28d0,...) at vn_write+0x379/frame 0xf9bbaab0 vn_io_fault(d3c8ca48,f9bbac10,ca6ef580,0,d56e28d0,...) at vn_io_fault+0x24b/frame 0xf9bbab8c dofilewrite(d3c8ca48,f9bbac10,ffffffff,ffffffff,0,...) at dofilewrite+0x86/frame 0xf9bbabc4 kern_writev(d56e28d0,8,f9bbac10,0,8049b04,...) at kern_writev+0x66/frame 0xf9bbabf0 sys_write(d56e28d0,f9bbacc8,14,c1087885,7b3,...) at sys_write+0x5c/frame 0xf9bbac40 syscall(f9bbad08) at syscall+0x2ed/frame 0xf9bbacfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9bbacfc --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28185fa7, esp = 0xbfbfd7e8, ebp = 0xbfbfd8b0 --- Tracing command truncate pid 90315 tid 100569 td 0xd17c38d0 sched_switch(d17c38d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9e0f460 mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xf9e0f494 sleepq_switch(d17c38d0,0,c108af51,26d,0,...) at sleepq_switch+0x1b9/frame 0xf9e0f4c8 sleepq_wait(e1aaa734,60,c107cce6,4,0,...) at sleepq_wait+0x48/frame 0xf9e0f4ec sleeplk(ca7c7ed0,c107cce6,60,0,0,...) at sleeplk+0x11e/frame 0xf9e0f528 __lockmgr_args(e1aaa734,81900,ca7c7ed0,c107cce6,60,...) at __lockmgr_args+0xaa4/frame 0xf9e0f608 getblk(ca7c7e10,11f224c0,0,4000,0,...) at getblk+0x1d8/frame 0xf9e0f678 breadn_flags(ca7c7e10,11f224c0,0,4000,0,...) at breadn_flags+0x40/frame 0xf9e0f6a8 ffs_alloccg(cea4e4fc,320,0,0,800,...) at ffs_alloccg+0x126/frame 0xf9e0f714 ffs_hashalloc(47b1980,0,800,800,c0cb9870,...) at ffs_hashalloc+0xce/frame 0xf9e0f748 ffs_alloc(cea4e4fc,0,0,47b1980,0,...) at ffs_alloc+0x29d/frame 0xf9e0f7b0 ffs_balloc_ufs2(ced0a168,0,0,2,ca6ef580,...) at ffs_balloc_ufs2+0x124b/frame 0xf9e0f8ec ffs_write(f9e0fa80,c10e32d1,f9e0f9d0,c0cef38d,c11f4a90,...) at ffs_write+0x346/frame 0xf9e0f980 VOP_WRITE_APV(c11e2de8,f9e0fa80,ced0a168,31b,d17c38d0,...) at VOP_WRITE_APV+0x19f/frame 0xf9e0fa40 vn_write(d3c8c9a0,f9e0fc10,ca6ef580,1,d17c38d0,...) at vn_write+0x379/frame 0xf9e0fab0 vn_io_fault(d3c8c9a0,f9e0fc10,ca6ef580,0,d17c38d0,...) at vn_io_fault+0x24b/frame 0xf9e0fb8c dofilewrite(d3c8c9a0,f9e0fc10,ffffffff,ffffffff,0,...) at dofilewrite+0x86/frame 0xf9e0fbc4 kern_writev(d17c38d0,6,f9e0fc10,0,8049b04,...) at kern_writev+0x66/frame 0xf9e0fbf0 sys_write(d17c38d0,f9e0fcc8,14,c1087885,7b3,...) at sys_write+0x5c/frame 0xf9e0fc40 syscall(f9e0fd08) at syscall+0x2ed/frame 0xf9e0fcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9e0fcfc --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28185fa7, esp = 0xbfbfd7e8, ebp = 0xbfbfd8b0 --- Tracing command truncate pid 90314 tid 100783 td 0xd56f4bc0 sched_switch(d56f4bc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9c17460 mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xf9c17494 sleepq_switch(d56f4bc0,0,c108af51,26d,0,...) at sleepq_switch+0x1b9/frame 0xf9c174c8 sleepq_wait(e1aaa734,60,c107cce6,4,0,...) at sleepq_wait+0x48/frame 0xf9c174ec sleeplk(ca7c7ed0,c107cce6,60,0,0,...) at sleeplk+0x11e/frame 0xf9c17528 __lockmgr_args(e1aaa734,81900,ca7c7ed0,c107cce6,60,...) at __lockmgr_args+0xaa4/frame 0xf9c17608 getblk(ca7c7e10,11f224c0,0,4000,0,...) at getblk+0x1d8/frame 0xf9c17678 breadn_flags(ca7c7e10,11f224c0,0,4000,0,...) at breadn_flags+0x40/frame 0xf9c176a8 ffs_alloccg(cea4ecb0,320,0,0,800,...) at ffs_alloccg+0x126/frame 0xf9c17714 ffs_hashalloc(47b1980,0,800,800,c0cb9870,...) at ffs_hashalloc+0xce/frame 0xf9c17748 ffs_alloc(cea4ecb0,0,0,47b1980,0,...) at ffs_alloc+0x29d/frame 0xf9c177b0 ffs_balloc_ufs2(d10ed438,0,0,2,ca6ef580,...) at ffs_balloc_ufs2+0x124b/frame 0xf9c178ec ffs_write(f9c17a80,c10e32d1,f9c179d0,c0cef38d,c11f4a90,...) at ffs_write+0x346/frame 0xf9c17980 VOP_WRITE_APV(c11e2de8,f9c17a80,d10ed438,31b,d56f4bc0,...) at VOP_WRITE_APV+0x19f/frame 0xf9c17a40 vn_write(d5966230,f9c17c10,ca6ef580,1,d56f4bc0,...) at vn_write+0x379/frame 0xf9c17ab0 vn_io_fault(d5966230,f9c17c10,ca6ef580,0,d56f4bc0,...) at vn_io_fault+0x24b/frame 0xf9c17b8c dofilewrite(d5966230,f9c17c10,ffffffff,ffffffff,0,...) at dofilewrite+0x86/frame 0xf9c17bc4 kern_writev(d56f4bc0,3,f9c17c10,0,8049b04,...) at kern_writev+0x66/frame 0xf9c17bf0 sys_write(d56f4bc0,f9c17cc8,14,c1087885,7b3,...) at sys_write+0x5c/frame 0xf9c17c40 syscall(f9c17d08) at syscall+0x2ed/frame 0xf9c17cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9c17cfc --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28185fa7, esp = 0xbfbfd7e8, ebp = 0xbfbfd8b0 --- Tracing command truncate pid 90313 tid 100870 td 0xd16dd8d0 sched_switch(d16dd8d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9b52460 mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xf9b52494 sleepq_switch(d16dd8d0,0,c108af51,26d,0,...) at sleepq_switch+0x1b9/frame 0xf9b524c8 sleepq_wait(e1aaa734,60,c107cce6,4,0,...) at sleepq_wait+0x48/frame 0xf9b524ec sleeplk(ca7c7ed0,c107cce6,60,0,0,...) at sleeplk+0x11e/frame 0xf9b52528 __lockmgr_args(e1aaa734,81900,ca7c7ed0,c107cce6,60,...) at __lockmgr_args+0xaa4/frame 0xf9b52608 getblk(ca7c7e10,11f224c0,0,4000,0,...) at getblk+0x1d8/frame 0xf9b52678 breadn_flags(ca7c7e10,11f224c0,0,4000,0,...) at breadn_flags+0x40/frame 0xf9b526a8 ffs_alloccg(cd728e0c,320,0,0,800,...) at ffs_alloccg+0x126/frame 0xf9b52714 ffs_hashalloc(47b1980,0,800,800,c0cb9870,...) at ffs_hashalloc+0xce/frame 0xf9b52748 ffs_alloc(cd728e0c,0,0,47b1980,0,...) at ffs_alloc+0x29d/frame 0xf9b527b0 ffs_balloc_ufs2(d42cd438,0,0,2,ca6ef580,...) at ffs_balloc_ufs2+0x124b/frame 0xf9b528ec ffs_write(f9b52a80,c10e32d1,f9b529d0,c0cef38d,c11f4a90,...) at ffs_write+0x346/frame 0xf9b52980 VOP_WRITE_APV(c11e2de8,f9b52a80,d42cd438,31b,d16dd8d0,...) at VOP_WRITE_APV+0x19f/frame 0xf9b52a40 vn_write(d6146460,f9b52c10,ca6ef580,1,d16dd8d0,...) at vn_write+0x379/frame 0xf9b52ab0 vn_io_fault(d6146460,f9b52c10,ca6ef580,0,d16dd8d0,...) at vn_io_fault+0x24b/frame 0xf9b52b8c dofilewrite(d6146460,f9b52c10,ffffffff,ffffffff,0,...) at dofilewrite+0x86/frame 0xf9b52bc4 kern_writev(d16dd8d0,b,f9b52c10,0,8049b04,...) at kern_writev+0x66/frame 0xf9b52bf0 sys_write(d16dd8d0,f9b52cc8,14,c1087885,7b3,...) at sys_write+0x5c/frame 0xf9b52c40 syscall(f9b52d08) at syscall+0x2ed/frame 0xf9b52cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9b52cfc --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28185fa7, esp = 0xbfbfd7e8, ebp = 0xbfbfd8b0 --- Tracing command truncate pid 90312 tid 100940 td 0xd58372f0 sched_switch(d58372f0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9e4e460 mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xf9e4e494 sleepq_switch(d58372f0,0,c108af51,26d,0,...) at sleepq_switch+0x1b9/frame 0xf9e4e4c8 sleepq_wait(e1aaa734,60,c107cce6,4,0,...) at sleepq_wait+0x48/frame 0xf9e4e4ec sleeplk(ca7c7ed0,c107cce6,60,0,0,...) at sleeplk+0x11e/frame 0xf9e4e528 __lockmgr_args(e1aaa734,81900,ca7c7ed0,c107cce6,60,...) at __lockmgr_args+0xaa4/frame 0xf9e4e608 getblk(ca7c7e10,11f224c0,0,4000,0,...) at getblk+0x1d8/frame 0xf9e4e678 breadn_flags(ca7c7e10,11f224c0,0,4000,0,...) at breadn_flags+0x40/frame 0xf9e4e6a8 ffs_alloccg(d4044c3c,320,0,0,800,...) at ffs_alloccg+0x126/frame 0xf9e4e714 ffs_hashalloc(47b1980,0,800,800,c0cb9870,...) at ffs_hashalloc+0xce/frame 0xf9e4e748 ffs_alloc(d4044c3c,0,0,47b1980,0,...) at ffs_alloc+0x29d/frame 0xf9e4e7b0 ffs_balloc_ufs2(cd88c2d0,0,0,2,ca6ef580,...) at ffs_balloc_ufs2+0x124b/frame 0xf9e4e8ec ffs_write(f9e4ea80,c10e32d1,f9e4e9d0,c0cef38d,c11f4a90,...) at ffs_write+0x346/frame 0xf9e4e980 VOP_WRITE_APV(c11e2de8,f9e4ea80,cd88c2d0,31b,d58372f0,...) at VOP_WRITE_APV+0x19f/frame 0xf9e4ea40 vn_write(d51956c8,f9e4ec10,ca6ef580,1,d58372f0,...) at vn_write+0x379/frame 0xf9e4eab0 vn_io_fault(d51956c8,f9e4ec10,ca6ef580,0,d58372f0,...) at vn_io_fault+0x24b/frame 0xf9e4eb8c dofilewrite(d51956c8,f9e4ec10,ffffffff,ffffffff,0,...) at dofilewrite+0x86/frame 0xf9e4ebc4 kern_writev(d58372f0,7,f9e4ec10,0,8049b04,...) at kern_writev+0x66/frame 0xf9e4ebf0 sys_write(d58372f0,f9e4ecc8,14,c1087885,7b3,...) at sys_write+0x5c/frame 0xf9e4ec40 syscall(f9e4ed08) at syscall+0x2ed/frame 0xf9e4ecfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9e4ecfc --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28185fa7, esp = 0xbfbfd7e8, ebp = 0xbfbfd8b0 --- Tracing command truncate pid 90311 tid 100138 td 0xcb1a58d0 sched_switch(cb1a58d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf97ba460 mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xf97ba494 sleepq_switch(cb1a58d0,0,c108af51,26d,0,...) at sleepq_switch+0x1b9/frame 0xf97ba4c8 sleepq_wait(e1aaa734,60,c107cce6,4,0,...) at sleepq_wait+0x48/frame 0xf97ba4ec sleeplk(ca7c7ed0,c107cce6,60,0,0,...) at sleeplk+0x11e/frame 0xf97ba528 __lockmgr_args(e1aaa734,81900,ca7c7ed0,c107cce6,60,...) at __lockmgr_args+0xaa4/frame 0xf97ba608 getblk(ca7c7e10,11f224c0,0,4000,0,...) at getblk+0x1d8/frame 0xf97ba678 breadn_flags(ca7c7e10,11f224c0,0,4000,0,...) at breadn_flags+0x40/frame 0xf97ba6a8 ffs_alloccg(cf366e0c,320,0,0,800,...) at ffs_alloccg+0x126/frame 0xf97ba714 ffs_hashalloc(47b1980,0,800,800,c0cb9870,...) at ffs_hashalloc+0xce/frame 0xf97ba748 ffs_alloc(cf366e0c,0,0,47b1980,0,...) at ffs_alloc+0x29d/frame 0xf97ba7b0 ffs_balloc_ufs2(d1083708,0,0,2,ca6ef580,...) at ffs_balloc_ufs2+0x124b/frame 0xf97ba8ec ffs_write(f97baa80,c10e32d1,f97ba9d0,c0cef38d,c11f4a90,...) at ffs_write+0x346/frame 0xf97ba980 VOP_WRITE_APV(c11e2de8,f97baa80,d1083708,31b,cb1a58d0,...) at VOP_WRITE_APV+0x19f/frame 0xf97baa40 vn_write(d39935b0,f97bac10,ca6ef580,1,cb1a58d0,...) at vn_write+0x379/frame 0xf97baab0 vn_io_fault(d39935b0,f97bac10,ca6ef580,0,cb1a58d0,...) at vn_io_fault+0x24b/frame 0xf97bab8c dofilewrite(d39935b0,f97bac10,ffffffff,ffffffff,0,...) at dofilewrite+0x86/frame 0xf97babc4 kern_writev(cb1a58d0,9,f97bac10,0,8049b04,...) at kern_writev+0x66/frame 0xf97babf0 sys_write(cb1a58d0,f97bacc8,14,c1087885,7b3,...) at sys_write+0x5c/frame 0xf97bac40 syscall(f97bad08) at syscall+0x2ed/frame 0xf97bacfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf97bacfc --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28185fa7, esp = 0xbfbfd7e8, ebp = 0xbfbfd8b0 --- Tracing command truncate pid 90310 tid 100506 td 0xd16e02f0 sched_switch(d16e02f0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9d14460 mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xf9d14494 sleepq_switch(d16e02f0,0,c108af51,26d,0,...) at sleepq_switch+0x1b9/frame 0xf9d144c8 sleepq_wait(e1aaa734,60,c107cce6,4,0,...) at sleepq_wait+0x48/frame 0xf9d144ec sleeplk(ca7c7ed0,c107cce6,60,0,0,...) at sleeplk+0x11e/frame 0xf9d14528 __lockmgr_args(e1aaa734,81900,ca7c7ed0,c107cce6,60,...) at __lockmgr_args+0xaa4/frame 0xf9d14608 getblk(ca7c7e10,11f224c0,0,4000,0,...) at getblk+0x1d8/frame 0xf9d14678 breadn_flags(ca7c7e10,11f224c0,0,4000,0,...) at breadn_flags+0x40/frame 0xf9d146a8 ffs_alloccg(cba6cae0,320,0,0,800,...) at ffs_alloccg+0x126/frame 0xf9d14714 ffs_hashalloc(47b1980,0,800,800,c0cb9870,...) at ffs_hashalloc+0xce/frame 0xf9d14748 ffs_alloc(cba6cae0,0,0,47b1980,0,...) at ffs_alloc+0x29d/frame 0xf9d147b0 ffs_balloc_ufs2(cb4009d8,0,0,2,ca6ef580,...) at ffs_balloc_ufs2+0x124b/frame 0xf9d148ec ffs_write(f9d14a80,c10e32d1,f9d149d0,c0cef38d,c11f4a90,...) at ffs_write+0x346/frame 0xf9d14980 VOP_WRITE_APV(c11e2de8,f9d14a80,cb4009d8,31b,d16e02f0,...) at VOP_WRITE_APV+0x19f/frame 0xf9d14a40 vn_write(d65489d8,f9d14c10,ca6ef580,1,d16e02f0,...) at vn_write+0x379/frame 0xf9d14ab0 vn_io_fault(d65489d8,f9d14c10,ca6ef580,0,d16e02f0,...) at vn_io_fault+0x24b/frame 0xf9d14b8c dofilewrite(d65489d8,f9d14c10,ffffffff,ffffffff,0,...) at dofilewrite+0x86/frame 0xf9d14bc4 kern_writev(d16e02f0,8,f9d14c10,0,8049b04,...) at kern_writev+0x66/frame 0xf9d14bf0 sys_write(d16e02f0,f9d14cc8,14,c1087885,7b3,...) at sys_write+0x5c/frame 0xf9d14c40 syscall(f9d14d08) at syscall+0x2ed/frame 0xf9d14cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9d14cfc --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28185fa7, esp = 0xbfbfd7e8, ebp = 0xbfbfd8b0 --- Tracing command truncate pid 90309 tid 100266 td 0xcaeafbc0 sched_switch(caeafbc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9988620 mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xf9988654 sleepq_switch(caeafbc0,0,c108af51,26d,0,...) at sleepq_switch+0x1b9/frame 0xf9988688 sleepq_wait(e19459d4,60,c107cce6,4,0,...) at sleepq_wait+0x48/frame 0xf99886ac sleeplk(ca7c7ed0,c107cce6,60,0,0,...) at sleeplk+0x11e/frame 0xf99886e8 __lockmgr_args(e19459d4,81900,ca7c7ed0,c107cce6,60,...) at __lockmgr_args+0xaa4/frame 0xf99887c8 getblk(ca7c7e10,11ec66c0,0,4000,0,...) at getblk+0x1d8/frame 0xf9988838 breadn_flags(ca7c7e10,11ec66c0,0,4000,0,...) at breadn_flags+0x40/frame 0xf9988868 softdep_setup_freeblocks(cea4ee80,0,0,800,246,...) at softdep_setup_freeblocks+0x8a2/frame 0xf99888e0 ffs_truncate(cc0d6000,0,0,800,ca6ef580,...) at ffs_truncate+0x476/frame 0xf9988a98 ufs_setattr(f9988bd8,c10e31e2,c10b8120,c1097510,c11f4a90,...) at ufs_setattr+0x730/frame 0xf9988b10 VOP_SETATTR_APV(c11e2de8,f9988bd8,cc0d6000,4a2,7fffffff,...) at VOP_SETATTR_APV+0x124/frame 0xf9988b40 vn_truncate(d3c8cce8,0,0,ca6ef580,caeafbc0,...) at vn_truncate+0x1fa/frame 0xf9988bfc kern_ftruncate(caeafbc0,6,0,0) at kern_ftruncate+0xa9/frame 0xf9988c28 sys_ftruncate(caeafbc0,f9988cc8,14,c1087885,7b3,...) at sys_ftruncate+0x28/frame 0xf9988c40 syscall(f9988d08) at syscall+0x2ed/frame 0xf9988cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9988cfc --- syscall (480, FreeBSD ELF32, sys_ftruncate), eip = 0x2816c303, esp = 0xbfbfd7e8, ebp = 0xbfbfd8b0 --- Tracing command truncate pid 90308 tid 100166 td 0xcb1a4bc0 sched_switch(cb1a4bc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf980e620 mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xf980e654 sleepq_switch(cb1a4bc0,0,c108af51,26d,0,...) at sleepq_switch+0x1b9/frame 0xf980e688 sleepq_wait(e19459d4,60,c107cce6,4,0,...) at sleepq_wait+0x48/frame 0xf980e6ac sleeplk(ca7c7ed0,c107cce6,60,0,0,...) at sleeplk+0x11e/frame 0xf980e6e8 __lockmgr_args(e19459d4,81900,ca7c7ed0,c107cce6,60,...) at __lockmgr_args+0xaa4/frame 0xf980e7c8 getblk(ca7c7e10,11ec66c0,0,4000,0,...) at getblk+0x1d8/frame 0xf980e838 breadn_flags(ca7c7e10,11ec66c0,0,4000,0,...) at breadn_flags+0x40/frame 0xf980e868 softdep_setup_freeblocks(d2e252b8,0,0,800,246,...) at softdep_setup_freeblocks+0x8a2/frame 0xf980e8e0 ffs_truncate(d0ddcca8,0,0,800,ca6ef580,...) at ffs_truncate+0x476/frame 0xf980ea98 ufs_setattr(f980ebd8,c10e31e2,c10b8120,c1097510,c11f4a90,...) at ufs_setattr+0x730/frame 0xf980eb10 VOP_SETATTR_APV(c11e2de8,f980ebd8,d0ddcca8,4a2,7fffffff,...) at VOP_SETATTR_APV+0x124/frame 0xf980eb40 vn_truncate(d2d1cab8,0,0,ca6ef580,cb1a4bc0,...) at vn_truncate+0x1fa/frame 0xf980ebfc kern_ftruncate(cb1a4bc0,7,0,0) at kern_ftruncate+0xa9/frame 0xf980ec28 sys_ftruncate(cb1a4bc0,f980ecc8,14,c1087885,7b3,...) at sys_ftruncate+0x28/frame 0xf980ec40 syscall(f980ed08) at syscall+0x2ed/frame 0xf980ecfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf980ecfc --- syscall (480, FreeBSD ELF32, sys_ftruncate), eip = 0x2816c303, esp = 0xbfbfd7e8, ebp = 0xbfbfd8b0 --- Tracing command truncate pid 90307 tid 100549 td 0xcc1178d0 sched_switch(cc1178d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf98f5998 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf98f59cc sleepq_switch(cc1178d0,0,c108af51,1ab,f98f5a44,...) at sleepq_switch+0x1b9/frame 0xf98f5a00 sleepq_catch_signals(0,c108af51,164,678,cc1178d0,...) at sleepq_catch_signals+0x3b1/frame 0xf98f5a48 sleepq_wait_sig(d20e45a0,6c,c108e435,100,0,...) at sleepq_wait_sig+0x14/frame 0xf98f5a6c _sleep(d20e45a0,d20e4628,16c,c108e435,0,...) at _sleep+0x32d/frame 0xf98f5ac4 kern_wait6(cc1178d0,7,0,0,f98f5b4c,...) at kern_wait6+0x43c/frame 0xf98f5b20 sys_wait4(cc1178d0,f98f5cc8,c10dc7ab,d6,1,...) at sys_wait4+0x94/frame 0xf98f5c40 syscall(f98f5d08) at syscall+0x2ed/frame 0xf98f5cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf98f5cfc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x280d698f, esp = 0xbfbfd898, ebp = 0xbfbfd8b0 --- Tracing command sh pid 90279 tid 100253 td 0xcc117bc0 sched_switch(cc117bc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9961998 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf99619cc sleepq_switch(cc117bc0,0,c108af51,1ab,f9961a44,...) at sleepq_switch+0x1b9/frame 0xf9961a00 sleepq_catch_signals(0,c108af51,164,378,cc117bc0,...) at sleepq_catch_signals+0x3b1/frame 0xf9961a48 sleepq_wait_sig(cbf725a0,6c,c108e435,100,0,...) at sleepq_wait_sig+0x14/frame 0xf9961a6c _sleep(cbf725a0,cbf72628,16c,c108e435,0,...) at _sleep+0x32d/frame 0xf9961ac4 kern_wait6(cc117bc0,7,0,0,f9961b4c,...) at kern_wait6+0x43c/frame 0xf9961b20 sys_wait4(cc117bc0,f9961cc8,c10dc7ab,e9,1,...) at sys_wait4+0x94/frame 0xf9961c40 syscall(f9961d08) at syscall+0x2ed/frame 0xf9961cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9961cfc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2815098f, esp = 0xbfbfd7f8, ebp = 0xbfbfd810 --- Tracing command mail pid 87827 tid 101251 td 0xd72b28d0 sched_switch(d72b28d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9e8da30 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf9e8da64 sleepq_switch(d72b28d0,0,c108af51,1ab,f9e8dadc,...) at sleepq_switch+0x1b9/frame 0xf9e8da98 sleepq_catch_signals(0,c108af51,164,510,d72b28d0,...) at sleepq_catch_signals+0x3b1/frame 0xf9e8dae0 sleepq_wait_sig(cbdc3660,5c,c108df45,100,0,...) at sleepq_wait_sig+0x14/frame 0xf9e8db04 _sleep(cbdc3660,cbdc37e0,15c,c108df45,0,...) at _sleep+0x32d/frame 0xf9e8db5c pipe_read(d6146578,f9e8dc10,d1a0d580,0,d72b28d0,...) at pipe_read+0x29c/frame 0xf9e8db90 dofileread(d6146578,f9e8dc10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf9e8dbc4 kern_readv(d72b28d0,0,f9e8dc10,0,28808000,...) at kern_readv+0x66/frame 0xf9e8dbf0 sys_read(d72b28d0,f9e8dcc8,f9e8dd08,1,1,...) at sys_read+0x5c/frame 0xf9e8dc40 syscall(f9e8dd08) at syscall+0x2ed/frame 0xf9e8dcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9e8dcfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x28197fb7, esp = 0xbfbfcfe8, ebp = 0xbfbfcffc --- Tracing command sh pid 87826 tid 100434 td 0xd162a2f0 sched_switch(d162a2f0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9c09998 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf9c099cc sleepq_switch(d162a2f0,0,c108af51,1ab,f9c09a44,...) at sleepq_switch+0x1b9/frame 0xf9c09a00 sleepq_catch_signals(0,c108af51,164,480,d162a2f0,...) at sleepq_catch_signals+0x3b1/frame 0xf9c09a48 sleepq_wait_sig(d16b3000,6c,c108e435,100,0,...) at sleepq_wait_sig+0x14/frame 0xf9c09a6c _sleep(d16b3000,d16b3088,16c,c108e435,0,...) at _sleep+0x32d/frame 0xf9c09ac4 kern_wait6(d162a2f0,7,0,0,f9c09b4c,...) at kern_wait6+0x43c/frame 0xf9c09b20 sys_wait4(d162a2f0,f9c09cc8,c10dc7ab,e9,1,...) at sys_wait4+0x94/frame 0xf9c09c40 syscall(f9c09d08) at syscall+0x2ed/frame 0xf9c09cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9c09cfc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2815098f, esp = 0xbfbfd8d0, ebp = 0xbfbfd8e8 --- Tracing command sh pid 87819 tid 100304 td 0xd0d2e000 Tracing command lockf pid 87818 tid 100857 td 0xd6af55e0 Tracing command sh pid 87815 tid 100694 td 0xd54518d0 Tracing command cron pid 87813 tid 100167 td 0xcb1a48d0 sched_switch(cb1a48d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9811a30 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf9811a64 sleepq_switch(cb1a48d0,0,c108af51,1ab,f9811adc,...) at sleepq_switch+0x1b9/frame 0xf9811a98 sleepq_catch_signals(0,c108af51,164,b88,cb1a48d0,...) at sleepq_catch_signals+0x3b1/frame 0xf9811ae0 sleepq_wait_sig(cacb7b28,5c,c108df45,100,0,...) at sleepq_wait_sig+0x14/frame 0xf9811b04 _sleep(cacb7b28,cacb7ca8,15c,c108df45,0,...) at _sleep+0x32d/frame 0xf9811b5c pipe_read(d3d808f8,f9811c10,ca4b4900,0,cb1a48d0,...) at pipe_read+0x29c/frame 0xf9811b90 dofileread(d3d808f8,f9811c10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf9811bc4 kern_readv(cb1a48d0,5,f9811c10,0,2881b000,...) at kern_readv+0x66/frame 0xf9811bf0 sys_read(cb1a48d0,f9811cc8,c10dc7ab,e9,1,...) at sys_read+0x5c/frame 0xf9811c40 syscall(f9811d08) at syscall+0x2ed/frame 0xf9811cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9811cfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a7fb7, esp = 0xbfbfd634, ebp = 0xbfbfd648 --- Tracing command wc pid 87143 tid 101050 td 0xd548c000 sched_switch(d548c000,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9d06a30 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf9d06a64 sleepq_switch(d548c000,0,c108af51,1ab,f9d06adc,...) at sleepq_switch+0x1b9/frame 0xf9d06a98 sleepq_catch_signals(0,c108af51,164,828,d548c000,...) at sleepq_catch_signals+0x3b1/frame 0xf9d06ae0 sleepq_wait_sig(cbc4d7f8,5c,c108df45,100,0,...) at sleepq_wait_sig+0x14/frame 0xf9d06b04 _sleep(cbc4d7f8,cbc4d978,15c,c108df45,0,...) at _sleep+0x32d/frame 0xf9d06b5c pipe_read(d45bc850,f9d06c10,ce1b6d00,0,d548c000,...) at pipe_read+0x29c/frame 0xf9d06b90 dofileread(d45bc850,f9d06c10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf9d06bc4 kern_readv(d548c000,0,f9d06c10,0,bfbedda0,...) at kern_readv+0x66/frame 0xf9d06bf0 sys_read(d548c000,f9d06cc8,c10dc7ab,d6,1,...) at sys_read+0x5c/frame 0xf9d06c40 syscall(f9d06d08) at syscall+0x2ed/frame 0xf9d06cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9d06cfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x28186fb7, esp = 0xbfbedc54, ebp = 0xbfbfddb8 --- Tracing command tee pid 87142 tid 100314 td 0xd0d2d000 sched_switch(d0d2d000,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9a3da30 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf9a3da64 sleepq_switch(d0d2d000,0,c108af51,1ab,f9a3dadc,...) at sleepq_switch+0x1b9/frame 0xf9a3da98 sleepq_catch_signals(0,c108af51,164,990,d0d2d000,...) at sleepq_catch_signals+0x3b1/frame 0xf9a3dae0 sleepq_wait_sig(d5306660,5c,c108df45,100,0,...) at sleepq_wait_sig+0x14/frame 0xf9a3db04 _sleep(d5306660,d53067e0,15c,c108df45,0,...) at _sleep+0x32d/frame 0xf9a3db5c pipe_read(d3c8ce70,f9a3dc10,ce1b6d00,0,d0d2d000,...) at pipe_read+0x29c/frame 0xf9a3db90 dofileread(d3c8ce70,f9a3dc10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf9a3dbc4 kern_readv(d0d2d000,0,f9a3dc10,0,28803000,...) at kern_readv+0x66/frame 0xf9a3dbf0 sys_read(d0d2d000,f9a3dcc8,f9a3dd08,1,1,...) at sys_read+0x5c/frame 0xf9a3dc40 syscall(f9a3dd08) at syscall+0x2ed/frame 0xf9a3dcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9a3dcfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x28185fb7, esp = 0xbfbfddb8, ebp = 0xbfbfdde4 --- Tracing command find pid 87141 tid 100492 td 0xd16d72f0 sched_switch(d16d72f0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9cdb5d0 mi_switch(104,0,c108af51,1f7,2,...) at mi_switch+0x1c9/frame 0xf9cdb604 sleepq_switch(d16d72f0,0,c108af51,290,d16d72f0,...) at sleepq_switch+0x1b9/frame 0xf9cdb638 sleepq_timedwait(f9cdb6d4,0,c10c0ab0,0,0,...) at sleepq_timedwait+0x48/frame 0xf9cdb65c _sleep(f9cdb6d4,0,0,c10c0ab0,4e2,...) at _sleep+0x317/frame 0xf9cdb6b4 kmem_malloc(c1bb508c,1000,102) at kmem_malloc+0xe9/frame 0xf9cdb6e8 page_alloc(ca54fe00,1000,f9cdb727,102,ca4b6e98,...) at page_alloc+0x27/frame 0xf9cdb6fc keg_alloc_slab(102,4,c10bfd74,887,f9cdb780,...) at keg_alloc_slab+0xc3/frame 0xf9cdb734 keg_fetch_slab(102,80,102,ceb57830,f9cdb7c8,...) at keg_fetch_slab+0x132/frame 0xf9cdb768 zone_fetch_slab(ca54fe00,0,102,83d,202,...) at zone_fetch_slab+0x3f/frame 0xf9cdb780 uma_zalloc_arg(ca54fe00,0,102,d16d72f0,f9cdb8cc,...) at uma_zalloc_arg+0x435/frame 0xf9cdb7c8 ffs_vgetf(ca706bb0,1b0625,200000,f9cdb8cc,0) at ffs_vgetf+0xa9/frame 0xf9cdb824 ffs_vget(ca706bb0,1b0625,200000,f9cdb8cc,f9cdb8dc,...) at ffs_vget+0x2e/frame 0xf9cdb840 ufs_lookup_ino(e0cfa168,f9cdbb5c,f9cdbb70,0) at ufs_lookup_ino+0xe27/frame 0xf9cdb908 ufs_lookup(f9cdb970,c10e2f12,c0b24618,e0cfa19c,f9cdb968,...) at ufs_lookup+0x2a/frame 0xf9cdb920 VOP_CACHEDLOOKUP_APV(c11e2de8,f9cdb970,f9cdbb70,0,0,...) at VOP_CACHEDLOOKUP_APV+0x124/frame 0xf9cdb950 vfs_cache_lookup(f9cdba00,c10e2eb1,c10b7f3d,c1094fbb,c11f4a90,...) at vfs_cache_lookup+0xf0/frame 0xf9cdb998 VOP_LOOKUP_APV(c11e2de8,f9cdba00,f9cdbb70,204,c13752f0,...) at VOP_LOOKUP_APV+0x124/frame 0xf9cdb9c8 lookup(f9cdbb30,c1094fbb,106,ce,200,...) at lookup+0x52e/frame 0xf9cdba28 namei(f9cdbb30,ffffffdf,ce1b6d00,0,d16d72f0,...) at namei+0x47d/frame 0xf9cdbab0 kern_statat_vnhook(d16d72f0,200,ffffff9c,28854058,0,...) at kern_statat_vnhook+0xbd/frame 0xf9cdbbb0 sys_lstat(d16d72f0,f9cdbcc8,f9cdbd08,1,1,...) at sys_lstat+0x4d/frame 0xf9cdbc40 syscall(f9cdbd08) at syscall+0x2ed/frame 0xf9cdbcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9cdbcfc --- syscall (190, FreeBSD ELF32, sys_lstat), eip = 0x2817bee3, esp = 0xbfbfdb24, ebp = 0xbfbfdbb0 --- Tracing command sh pid 87140 tid 100391 td 0xcb25e000 Tracing command sh pid 87136 tid 100344 td 0xcba01bc0 sched_switch(cba01bc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9aaba30 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf9aaba64 sleepq_switch(cba01bc0,0,c108af51,1ab,f9aabadc,...) at sleepq_switch+0x1b9/frame 0xf9aaba98 sleepq_catch_signals(0,c108af51,164,720,cba01bc0,...) at sleepq_catch_signals+0x3b1/frame 0xf9aabae0 sleepq_wait_sig(cbc4ccc0,5c,c108df45,100,0,...) at sleepq_wait_sig+0x14/frame 0xf9aabb04 _sleep(cbc4ccc0,cbc4ce40,15c,c108df45,0,...) at _sleep+0x32d/frame 0xf9aabb5c pipe_read(d3d80498,f9aabc10,ce1b6d00,0,cba01bc0,...) at pipe_read+0x29c/frame 0xf9aabb90 dofileread(d3d80498,f9aabc10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf9aabbc4 kern_readv(cba01bc0,3,f9aabc10,0,bfbfda88,...) at kern_readv+0x66/frame 0xf9aabbf0 sys_read(cba01bc0,f9aabcc8,14,c1087885,7b3,...) at sys_read+0x5c/frame 0xf9aabc40 syscall(f9aabd08) at syscall+0x2ed/frame 0xf9aabcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9aabcfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281fffb7, esp = 0xbfbfd9f4, ebp = 0xbfbfdb1c --- Tracing command mail pid 4234 tid 100081 td 0xca794000 sched_switch(ca794000,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9683a30 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf9683a64 sleepq_switch(ca794000,0,c108af51,1ab,f9683adc,...) at sleepq_switch+0x1b9/frame 0xf9683a98 sleepq_catch_signals(0,c108af51,164,60,ca794000,...) at sleepq_catch_signals+0x3b1/frame 0xf9683ae0 sleepq_wait_sig(cbdc04c8,5c,c108df45,100,0,...) at sleepq_wait_sig+0x14/frame 0xf9683b04 _sleep(cbdc04c8,cbdc0648,15c,c108df45,0,...) at _sleep+0x32d/frame 0xf9683b5c pipe_read(d3993380,f9683c10,ce1b6d00,0,ca794000,...) at pipe_read+0x29c/frame 0xf9683b90 dofileread(d3993380,f9683c10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf9683bc4 kern_readv(ca794000,0,f9683c10,0,28808000,...) at kern_readv+0x66/frame 0xf9683bf0 sys_read(ca794000,f9683cc8,7b3,0,f9683cc4,...) at sys_read+0x5c/frame 0xf9683c40 syscall(f9683d08) at syscall+0x2ed/frame 0xf9683cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9683cfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x28197fb7, esp = 0xbfbfcfc8, ebp = 0xbfbfcfdc --- Tracing command sh pid 4181 tid 100195 td 0xcbbbabc0 Tracing command sh pid 76795 tid 100103 td 0xca7f18d0 Tracing command lockf pid 75987 tid 100275 td 0xd5451bc0 Tracing command sh pid 74867 tid 100415 td 0xd13de2f0 Tracing command sh pid 73701 tid 100289 td 0xcc08f000 Tracing command mail pid 90238 tid 100429 td 0xcb28c5e0 sched_switch(cb28c5e0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9bf6a30 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf9bf6a64 sleepq_switch(cb28c5e0,0,c108af51,1ab,f9bf6adc,...) at sleepq_switch+0x1b9/frame 0xf9bf6a98 sleepq_catch_signals(0,c108af51,164,300,cb28c5e0,...) at sleepq_catch_signals+0x3b1/frame 0xf9bf6ae0 sleepq_wait_sig(ca7ea000,5c,c108df45,100,0,...) at sleepq_wait_sig+0x14/frame 0xf9bf6b04 _sleep(ca7ea000,ca7ea180,15c,c108df45,0,...) at _sleep+0x32d/frame 0xf9bf6b5c pipe_read(d45bc150,f9bf6c10,ce1b6d00,0,cb28c5e0,...) at pipe_read+0x29c/frame 0xf9bf6b90 dofileread(d45bc150,f9bf6c10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf9bf6bc4 kern_readv(cb28c5e0,0,f9bf6c10,2,28808000,...) at kern_readv+0x66/frame 0xf9bf6bf0 sys_read(cb28c5e0,f9bf6cc8,7b3,0,f9bf6cc4,...) at sys_read+0x5c/frame 0xf9bf6c40 syscall(f9bf6d08) at syscall+0x2ed/frame 0xf9bf6cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9bf6cfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x28197fb7, esp = 0xbfbfcfe8, ebp = 0xbfbfcffc --- Tracing command sh pid 90105 tid 101244 td 0xd56f15e0 Tracing command sh pid 63408 tid 101080 td 0xda3c1000 Tracing command lockf pid 62593 tid 100570 td 0xcb34d2f0 Tracing command sh pid 51849 tid 100261 td 0xcc116bc0 Tracing command cron pid 50776 tid 100456 td 0xd16c25e0 sched_switch(d16c25e0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf98e9a30 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf98e9a64 sleepq_switch(d16c25e0,0,c108af51,1ab,f98e9adc,...) at sleepq_switch+0x1b9/frame 0xf98e9a98 sleepq_catch_signals(0,c108af51,164,b10,d16c25e0,...) at sleepq_catch_signals+0x3b1/frame 0xf98e9ae0 sleepq_wait_sig(ced4f660,5c,c108df45,100,0,...) at sleepq_wait_sig+0x14/frame 0xf98e9b04 _sleep(ced4f660,ced4f7e0,15c,c108df45,0,...) at _sleep+0x32d/frame 0xf98e9b5c pipe_read(d3993c40,f98e9c10,ca4b4900,0,d16c25e0,...) at pipe_read+0x29c/frame 0xf98e9b90 dofileread(d3993c40,f98e9c10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf98e9bc4 kern_readv(d16c25e0,5,f98e9c10,0,2881b000,...) at kern_readv+0x66/frame 0xf98e9bf0 sys_read(d16c25e0,f98e9cc8,c10dc7ab,e9,f98e9cc4,...) at sys_read+0x5c/frame 0xf98e9c40 syscall(f98e9d08) at syscall+0x2ed/frame 0xf98e9cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf98e9cfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a7fb7, esp = 0xbfbfd634, ebp = 0xbfbfd648 --- Tracing command zfskern pid 83682 tid 100575 td 0xd17402f0 sched_switch(d17402f0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9e30b1c mi_switch(104,0,c108af51,1f7,c124cfa4,...) at mi_switch+0x1c9/frame 0xf9e30b50 sleepq_switch(d17402f0,0,c108af51,290,0,...) at sleepq_switch+0x1b9/frame 0xf9e30b84 sleepq_timedwait(d1ca9ddc,0,f9e30bdc,1,0,...) at sleepq_timedwait+0x48/frame 0xf9e30ba8 _cv_timedwait(d1ca9ddc,d1ca9dc8,3e8,0,0,...) at _cv_timedwait+0x20c/frame 0xf9e30bf0 l2arc_feed_thread(0,f9e30d08,c107ef40,3d7,0,...) at l2arc_feed_thread+0xcbe/frame 0xf9e30ccc fork_exit(d1b73e90,0,f9e30d08) at fork_exit+0x7f/frame 0xf9e30cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf9e30cf4 --- trap 0, eip = 0, esp = 0xf9e30d40, ebp = 0 --- Tracing command zfskern pid 83682 tid 100561 td 0xd16e4000 sched_switch(d16e4000,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9df7b8c mi_switch(104,0,c108af51,1f7,c124d124,...) at mi_switch+0x1c9/frame 0xf9df7bc0 sleepq_switch(d16e4000,0,c108af51,290,0,...) at sleepq_switch+0x1b9/frame 0xf9df7bf4 sleepq_timedwait(d1ca2d10,0,f9df7c4c,1,0,...) at sleepq_timedwait+0x48/frame 0xf9df7c18 _cv_timedwait(d1ca2d10,d1ca2d4c,3e8,0,8ad,...) at _cv_timedwait+0x20c/frame 0xf9df7c60 arc_reclaim_thread(0,f9df7d08,c107ef40,3d7,0,...) at arc_reclaim_thread+0x380/frame 0xf9df7ccc fork_exit(d1b72c80,0,f9df7d08) at fork_exit+0x7f/frame 0xf9df7cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf9df7cf4 --- trap 0, eip = 0, esp = 0xf9df7d40, ebp = 0 --- Tracing command sh pid 2667 tid 100230 td 0xcc0838d0 sched_switch(cc0838d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf991c998 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf991c9cc sleepq_switch(cc0838d0,0,c108af51,1ab,f991ca44,...) at sleepq_switch+0x1b9/frame 0xf991ca00 sleepq_catch_signals(0,c108af51,164,c0,cc0838d0,...) at sleepq_catch_signals+0x3b1/frame 0xf991ca48 sleepq_wait_sig(cbc78870,6c,c108e435,100,0,...) at sleepq_wait_sig+0x14/frame 0xf991ca6c _sleep(cbc78870,cbc788f8,16c,c108e435,0,...) at _sleep+0x32d/frame 0xf991cac4 kern_wait6(cc0838d0,7,0,0,f991cb4c,...) at kern_wait6+0x43c/frame 0xf991cb20 sys_wait4(cc0838d0,f991ccc8,c10dc7ab,e9,1,...) at sys_wait4+0x94/frame 0xf991cc40 syscall(f991cd08) at syscall+0x2ed/frame 0xf991ccfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf991ccfc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2815098f, esp = 0xbfbfd778, ebp = 0xbfbfd790 --- Tracing command bash pid 2424 tid 100296 td 0xcc08e8d0 Tracing command sshd pid 2404 tid 100146 td 0xca7dc2f0 sched_switch(ca7dc2f0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf97d2918 mi_switch(104,0,c108af51,1f7,200046,...) at mi_switch+0x1c9/frame 0xf97d294c sleepq_switch(ca7dc2f0,0,c108af51,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf97d2980 sleepq_catch_signals(f97d29ec,c0a80379,ca78d5a0,0,0,...) at sleepq_catch_signals+0x3b1/frame 0xf97d29c8 sleepq_wait_sig(ca78d5a4,0,f97d2a20,101,0,...) at sleepq_wait_sig+0x14/frame 0xf97d29ec _cv_wait_sig(ca78d5a4,ca78d590,c108dd00,677,c,...) at _cv_wait_sig+0x200/frame 0xf97d2a34 seltdwait(cbdc2150,58,ca7e6580,ca7dc2f0,c137a518,...) at seltdwait+0x8c/frame 0xf97d2a58 kern_select(ca7dc2f0,c,28c0c1c8,28c0c1d0,0,...) at kern_select+0x85f/frame 0xf97d2bf8 sys_select(ca7dc2f0,f97d2cc8,f97d2d08,1,1,...) at sys_select+0x69/frame 0xf97d2c40 syscall(f97d2d08) at syscall+0x2ed/frame 0xf97d2cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf97d2cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28522f77, esp = 0xbfbfd3cc, ebp = 0xbfbfd418 --- Tracing command sshd pid 2319 tid 100232 td 0xcc0832f0 sched_switch(cc0832f0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9922988 mi_switch(104,0,c108af51,1f7,200046,...) at mi_switch+0x1c9/frame 0xf99229bc sleepq_switch(cc0832f0,0,c108af51,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf99229f0 sleepq_catch_signals(f9922a5c,c0a80379,cde79820,0,0,...) at sleepq_catch_signals+0x3b1/frame 0xf9922a38 sleepq_wait_sig(cde79824,0,f9922a90,101,0,...) at sleepq_wait_sig+0x14/frame 0xf9922a5c _cv_wait_sig(cde79824,cde79810,c108dd00,677,cc0832f0,...) at _cv_wait_sig+0x200/frame 0xf9922aa4 seltdwait(cb50372c,c108dd00,5ac,cc0832f0,ffffff80,...) at seltdwait+0x8c/frame 0xf9922ac8 sys_poll(cc0832f0,f9922cc8,f9922d08,1,1,...) at sys_poll+0x3e5/frame 0xf9922c40 syscall(f9922d08) at syscall+0x2ed/frame 0xf9922cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9922cfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x284b9aaf, esp = 0xbfbfd414, ebp = 0xbfbfd460 --- Tracing command getty pid 1079 tid 100127 td 0xca7dd8d0 sched_switch(ca7dd8d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf97999b8 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf97999ec sleepq_switch(ca7dd8d0,0,c108af51,1ab,f9799a64,...) at sleepq_switch+0x1b9/frame 0xf9799a20 sleepq_catch_signals(0,c108af51,164,330,1,...) at sleepq_catch_signals+0x3b1/frame 0xf9799a68 sleepq_wait_sig(c80ba270,0,c108fcc5,101,0,...) at sleepq_wait_sig+0x14/frame 0xf9799a8c _cv_wait_sig(c80ba270,c13f8e24,c108fcb1,531,f9799c10,...) at _cv_wait_sig+0x200/frame 0xf9799ad4 tty_wait(c80ba200,c80ba270,f9799c10,1,0,...) at tty_wait+0x4b/frame 0xf9799af8 ttydisc_read(c80ba200,f9799c10,0,a1,f9799c10,...) at ttydisc_read+0x377/frame 0xf9799b3c ttydev_read(c8093500,f9799c10,0,1,0,...) at ttydev_read+0x81/frame 0xf9799b5c devfs_read_f(ca7e2888,f9799c10,c7d7ce00,0,ca7dd8d0,...) at devfs_read_f+0x87/frame 0xf9799b90 dofileread(ca7e2888,f9799c10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf9799bc4 kern_readv(ca7dd8d0,0,f9799c10,0,bfbfddfb,...) at kern_readv+0x66/frame 0xf9799bf0 sys_read(ca7dd8d0,f9799cc8,f9799d08,1,1,...) at sys_read+0x5c/frame 0xf9799c40 syscall(f9799d08) at syscall+0x2ed/frame 0xf9799cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9799cfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2819bfb7, esp = 0xbfbfdde0, ebp = 0xbfbfde08 --- Tracing command getty pid 1078 tid 100153 td 0xcb19bbc0 sched_switch(cb19bbc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf97e79b8 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf97e79ec sleepq_switch(cb19bbc0,0,c108af51,1ab,f97e7a64,...) at sleepq_switch+0x1b9/frame 0xf97e7a20 sleepq_catch_signals(0,c108af51,164,360,1,...) at sleepq_catch_signals+0x3b1/frame 0xf97e7a68 sleepq_wait_sig(c80ba470,0,c108fcc5,101,0,...) at sleepq_wait_sig+0x14/frame 0xf97e7a8c _cv_wait_sig(c80ba470,c13f8e24,c108fcb1,531,f97e7c10,...) at _cv_wait_sig+0x200/frame 0xf97e7ad4 tty_wait(c80ba400,c80ba470,f97e7c10,1,0,...) at tty_wait+0x4b/frame 0xf97e7af8 ttydisc_read(c80ba400,f97e7c10,0,a1,f97e7c10,...) at ttydisc_read+0x377/frame 0xf97e7b3c ttydev_read(c8093600,f97e7c10,0,1,0,...) at ttydev_read+0x81/frame 0xf97e7b5c devfs_read_f(cb323818,f97e7c10,c7d7ce00,0,cb19bbc0,...) at devfs_read_f+0x87/frame 0xf97e7b90 dofileread(cb323818,f97e7c10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf97e7bc4 kern_readv(cb19bbc0,0,f97e7c10,0,bfbfddfb,...) at kern_readv+0x66/frame 0xf97e7bf0 sys_read(cb19bbc0,f97e7cc8,c10dc7ab,d6,1,...) at sys_read+0x5c/frame 0xf97e7c40 syscall(f97e7d08) at syscall+0x2ed/frame 0xf97e7cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf97e7cfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2819bfb7, esp = 0xbfbfdde0, ebp = 0xbfbfde08 --- Tracing command getty pid 1077 tid 100143 td 0xca7dc8d0 sched_switch(ca7dc8d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf97c99b8 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf97c99ec sleepq_switch(ca7dc8d0,0,c108af51,1ab,f97c9a64,...) at sleepq_switch+0x1b9/frame 0xf97c9a20 sleepq_catch_signals(0,c108af51,164,a80,1,...) at sleepq_catch_signals+0x3b1/frame 0xf97c9a68 sleepq_wait_sig(c7f07070,0,c108fcc5,101,0,...) at sleepq_wait_sig+0x14/frame 0xf97c9a8c _cv_wait_sig(c7f07070,c13f8e24,c108fcb1,531,f97c9c10,...) at _cv_wait_sig+0x200/frame 0xf97c9ad4 tty_wait(c7f07000,c7f07070,f97c9c10,1,0,...) at tty_wait+0x4b/frame 0xf97c9af8 ttydisc_read(c7f07000,f97c9c10,0,a1,f97c9c10,...) at ttydisc_read+0x377/frame 0xf97c9b3c ttydev_read(c8093700,f97c9c10,0,1,0,...) at ttydev_read+0x81/frame 0xf97c9b5c devfs_read_f(ca7e2508,f97c9c10,c7d7ce00,0,ca7dc8d0,...) at devfs_read_f+0x87/frame 0xf97c9b90 dofileread(ca7e2508,f97c9c10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf97c9bc4 kern_readv(ca7dc8d0,0,f97c9c10,0,bfbfddfb,...) at kern_readv+0x66/frame 0xf97c9bf0 sys_read(ca7dc8d0,f97c9cc8,f97c9d08,1,1,...) at sys_read+0x5c/frame 0xf97c9c40 syscall(f97c9d08) at syscall+0x2ed/frame 0xf97c9cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf97c9cfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2819bfb7, esp = 0xbfbfdde0, ebp = 0xbfbfde08 --- Tracing command getty pid 1076 tid 100126 td 0xca7ddbc0 sched_switch(ca7ddbc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf97969b8 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf97969ec sleepq_switch(ca7ddbc0,0,c108af51,1ab,f9796a64,...) at sleepq_switch+0x1b9/frame 0xf9796a20 sleepq_catch_signals(0,c108af51,164,ab0,1,...) at sleepq_catch_signals+0x3b1/frame 0xf9796a68 sleepq_wait_sig(c7f07270,0,c108fcc5,101,0,...) at sleepq_wait_sig+0x14/frame 0xf9796a8c _cv_wait_sig(c7f07270,c13f8e24,c108fcb1,531,f9796c10,...) at _cv_wait_sig+0x200/frame 0xf9796ad4 tty_wait(c7f07200,c7f07270,f9796c10,1,0,...) at tty_wait+0x4b/frame 0xf9796af8 ttydisc_read(c7f07200,f9796c10,0,a1,f9796c10,...) at ttydisc_read+0x377/frame 0xf9796b3c ttydev_read(c8093800,f9796c10,0,1,0,...) at ttydev_read+0x81/frame 0xf9796b5c devfs_read_f(cb323620,f9796c10,c7d7ce00,0,ca7ddbc0,...) at devfs_read_f+0x87/frame 0xf9796b90 dofileread(cb323620,f9796c10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf9796bc4 kern_readv(ca7ddbc0,0,f9796c10,0,bfbfddfb,...) at kern_readv+0x66/frame 0xf9796bf0 sys_read(ca7ddbc0,f9796cc8,63,c11a3f64,c80a8000,...) at sys_read+0x5c/frame 0xf9796c40 syscall(f9796d08) at syscall+0x2ed/frame 0xf9796cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9796cfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2819bfb7, esp = 0xbfbfdde0, ebp = 0xbfbfde08 --- Tracing command getty pid 1075 tid 100076 td 0xca756bc0 sched_switch(ca756bc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf966f9b8 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf966f9ec sleepq_switch(ca756bc0,0,c108af51,1ab,f966fa64,...) at sleepq_switch+0x1b9/frame 0xf966fa20 sleepq_catch_signals(0,c108af51,164,ae0,1,...) at sleepq_catch_signals+0x3b1/frame 0xf966fa68 sleepq_wait_sig(c7f07470,0,c108fcc5,101,0,...) at sleepq_wait_sig+0x14/frame 0xf966fa8c _cv_wait_sig(c7f07470,c13f8e24,c108fcb1,531,f966fc10,...) at _cv_wait_sig+0x200/frame 0xf966fad4 tty_wait(c7f07400,c7f07470,f966fc10,1,0,...) at tty_wait+0x4b/frame 0xf966faf8 ttydisc_read(c7f07400,f966fc10,0,a1,f966fc10,...) at ttydisc_read+0x377/frame 0xf966fb3c ttydev_read(c8093900,f966fc10,0,1,0,...) at ttydev_read+0x81/frame 0xf966fb5c devfs_read_f(cb323380,f966fc10,c7d7ce00,0,ca756bc0,...) at devfs_read_f+0x87/frame 0xf966fb90 dofileread(cb323380,f966fc10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf966fbc4 kern_readv(ca756bc0,0,f966fc10,0,bfbfddfb,...) at kern_readv+0x66/frame 0xf966fbf0 sys_read(ca756bc0,f966fcc8,14,c1087885,7b3,...) at sys_read+0x5c/frame 0xf966fc40 syscall(f966fd08) at syscall+0x2ed/frame 0xf966fcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf966fcfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2819bfb7, esp = 0xbfbfdde0, ebp = 0xbfbfde08 --- Tracing command getty pid 1074 tid 100104 td 0xca7f15e0 sched_switch(ca7f15e0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf96e09b8 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf96e09ec sleepq_switch(ca7f15e0,0,c108af51,1ab,f96e0a64,...) at sleepq_switch+0x1b9/frame 0xf96e0a20 sleepq_catch_signals(0,c108af51,164,b10,1,...) at sleepq_catch_signals+0x3b1/frame 0xf96e0a68 sleepq_wait_sig(c7f07670,0,c108fcc5,101,0,...) at sleepq_wait_sig+0x14/frame 0xf96e0a8c _cv_wait_sig(c7f07670,c13f8e24,c108fcb1,531,f96e0c10,...) at _cv_wait_sig+0x200/frame 0xf96e0ad4 tty_wait(c7f07600,c7f07670,f96e0c10,1,0,...) at tty_wait+0x4b/frame 0xf96e0af8 ttydisc_read(c7f07600,f96e0c10,0,a1,f96e0c10,...) at ttydisc_read+0x377/frame 0xf96e0b3c ttydev_read(c8093a00,f96e0c10,0,1,0,...) at ttydev_read+0x81/frame 0xf96e0b5c devfs_read_f(ca7e27a8,f96e0c10,c7d7ce00,0,ca7f15e0,...) at devfs_read_f+0x87/frame 0xf96e0b90 dofileread(ca7e27a8,f96e0c10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf96e0bc4 kern_readv(ca7f15e0,0,f96e0c10,0,bfbfddfb,...) at kern_readv+0x66/frame 0xf96e0bf0 sys_read(ca7f15e0,f96e0cc8,f96e0d08,1,1,...) at sys_read+0x5c/frame 0xf96e0c40 syscall(f96e0d08) at syscall+0x2ed/frame 0xf96e0cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf96e0cfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2819bfb7, esp = 0xbfbfdde0, ebp = 0xbfbfde08 --- Tracing command getty pid 1073 tid 100115 td 0xc950f5e0 sched_switch(c950f5e0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf97759b8 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf97759ec sleepq_switch(c950f5e0,0,c108af51,1ab,f9775a64,...) at sleepq_switch+0x1b9/frame 0xf9775a20 sleepq_catch_signals(0,c108af51,164,b40,1,...) at sleepq_catch_signals+0x3b1/frame 0xf9775a68 sleepq_wait_sig(c7f07870,0,c108fcc5,101,0,...) at sleepq_wait_sig+0x14/frame 0xf9775a8c _cv_wait_sig(c7f07870,c13f8e24,c108fcb1,531,f9775c10,...) at _cv_wait_sig+0x200/frame 0xf9775ad4 tty_wait(c7f07800,c7f07870,f9775c10,1,0,...) at tty_wait+0x4b/frame 0xf9775af8 ttydisc_read(c7f07800,f9775c10,0,a1,f9775c10,...) at ttydisc_read+0x377/frame 0xf9775b3c ttydev_read(c8093b00,f9775c10,0,1,0,...) at ttydev_read+0x81/frame 0xf9775b5c devfs_read_f(ca7e2af0,f9775c10,c7d7ce00,0,c950f5e0,...) at devfs_read_f+0x87/frame 0xf9775b90 dofileread(ca7e2af0,f9775c10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf9775bc4 kern_readv(c950f5e0,0,f9775c10,0,bfbfddfb,...) at kern_readv+0x66/frame 0xf9775bf0 sys_read(c950f5e0,f9775cc8,f9775d08,1,1,...) at sys_read+0x5c/frame 0xf9775c40 syscall(f9775d08) at syscall+0x2ed/frame 0xf9775cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9775cfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2819bfb7, esp = 0xbfbfdde0, ebp = 0xbfbfde08 --- Tracing command getty pid 1072 tid 100142 td 0xcb19c5e0 sched_switch(cb19c5e0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf97c69b8 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf97c69ec sleepq_switch(cb19c5e0,0,c108af51,1ab,f97c6a64,...) at sleepq_switch+0x1b9/frame 0xf97c6a20 sleepq_catch_signals(0,c108af51,164,b70,1,...) at sleepq_catch_signals+0x3b1/frame 0xf97c6a68 sleepq_wait_sig(c7f07a70,0,c108fcc5,101,0,...) at sleepq_wait_sig+0x14/frame 0xf97c6a8c _cv_wait_sig(c7f07a70,c13f8e24,c108fcb1,531,f97c6c10,...) at _cv_wait_sig+0x200/frame 0xf97c6ad4 tty_wait(c7f07a00,c7f07a70,f97c6c10,1,0,...) at tty_wait+0x4b/frame 0xf97c6af8 ttydisc_read(c7f07a00,f97c6c10,0,a1,f97c6c10,...) at ttydisc_read+0x377/frame 0xf97c6b3c ttydev_read(c8093c00,f97c6c10,0,1,0,...) at ttydev_read+0x81/frame 0xf97c6b5c devfs_read_f(ca7adcb0,f97c6c10,c7d7ce00,0,cb19c5e0,...) at devfs_read_f+0x87/frame 0xf97c6b90 dofileread(ca7adcb0,f97c6c10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf97c6bc4 kern_readv(cb19c5e0,0,f97c6c10,0,bfbfddfb,...) at kern_readv+0x66/frame 0xf97c6bf0 sys_read(cb19c5e0,f97c6cc8,f97c6d08,1,1,...) at sys_read+0x5c/frame 0xf97c6c40 syscall(f97c6d08) at syscall+0x2ed/frame 0xf97c6cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf97c6cfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2819bfb7, esp = 0xbfbfdde0, ebp = 0xbfbfde08 --- Tracing command getty pid 1071 tid 100102 td 0xca7f1bc0 sched_switch(ca7f1bc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf96d89b8 mi_switch(104,0,c108af51,1f7,46,...) at mi_switch+0x1c9/frame 0xf96d89ec sleepq_switch(ca7f1bc0,0,c108af51,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf96d8a20 sleepq_catch_signals(f96d8a8c,c0a80379,c7f07c14,0,0,...) at sleepq_catch_signals+0x3b1/frame 0xf96d8a68 sleepq_wait_sig(c7f07c70,0,f96d8ac0,101,0,...) at sleepq_wait_sig+0x14/frame 0xf96d8a8c _cv_wait_sig(c7f07c70,c7f07c04,c108fcb1,531,f96d8c10,...) at _cv_wait_sig+0x200/frame 0xf96d8ad4 tty_wait(c7f07c00,c7f07c70,f96d8c10,1,0,...) at tty_wait+0x4b/frame 0xf96d8af8 ttydisc_read(c7f07c00,f96d8c10,0,a1,f96d8c10,...) at ttydisc_read+0x377/frame 0xf96d8b3c ttydev_read(c7eeed00,f96d8c10,0,1,0,...) at ttydev_read+0x81/frame 0xf96d8b5c devfs_read_f(cb323348,f96d8c10,c7d7ce00,0,ca7f1bc0,...) at devfs_read_f+0x87/frame 0xf96d8b90 dofileread(cb323348,f96d8c10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf96d8bc4 kern_readv(ca7f1bc0,0,f96d8c10,0,bfbfddfb,...) at kern_readv+0x66/frame 0xf96d8bf0 sys_read(ca7f1bc0,f96d8cc8,14,c1087885,7b3,...) at sys_read+0x5c/frame 0xf96d8c40 syscall(f96d8d08) at syscall+0x2ed/frame 0xf96d8cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf96d8cfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2819bfb7, esp = 0xbfbfdde0, ebp = 0xbfbfde08 --- Tracing command inetd pid 1054 tid 100175 td 0xcbc7abc0 sched_switch(cbc7abc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9829918 mi_switch(104,0,c108af51,1f7,46,...) at mi_switch+0x1c9/frame 0xf982994c sleepq_switch(cbc7abc0,0,c108af51,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf9829980 sleepq_catch_signals(f98299ec,c0a80379,ca78d4a0,0,0,...) at sleepq_catch_signals+0x3b1/frame 0xf98299c8 sleepq_wait_sig(ca78d4a4,0,f9829a20,101,0,...) at sleepq_wait_sig+0x14/frame 0xf98299ec _cv_wait_sig(ca78d4a4,ca78d490,c108dd00,677,8,...) at _cv_wait_sig+0x200/frame 0xf9829a34 seltdwait(cbc4a0a8,58,c7d7ce00,cbc7abc0,cbc75bc8,...) at seltdwait+0x8c/frame 0xf9829a58 kern_select(cbc7abc0,8,bfbfd008,0,0,...) at kern_select+0x85f/frame 0xf9829bf8 sys_select(cbc7abc0,f9829cc8,f9829d08,1,1,...) at sys_select+0x69/frame 0xf9829c40 syscall(f9829d08) at syscall+0x2ed/frame 0xf9829cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9829cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281aef77, esp = 0xbfbfcf84, ebp = 0xbfbfde34 --- Tracing command watchdogd pid 1029 tid 100177 td 0xcbc7a5e0 sched_switch(cbc7a5e0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf982fa80 mi_switch(104,0,c108af51,1f7,c13f9900,...) at mi_switch+0x1c9/frame 0xf982fab4 sleepq_switch(cbc7a5e0,0,c108af51,1ab,f982fb2c,...) at sleepq_switch+0x1b9/frame 0xf982fae8 sleepq_catch_signals(ea61,c0ad5e10,cbc7a5e0,0,0,...) at sleepq_catch_signals+0x3b1/frame 0xf982fb30 sleepq_timedwait_sig(c1231e80,6c,c1086a3e,100,0,...) at sleepq_timedwait_sig+0x14/frame 0xf982fb58 _sleep(c1231e80,0,16c,c1086a3e,ea61,...) at _sleep+0x303/frame 0xf982fbb0 kern_nanosleep(bfbfdd08,f982fc18,f982fc20,dcb61390,3c,...) at kern_nanosleep+0xd9/frame 0xf982fc00 sys_nanosleep(cbc7a5e0,f982fcc8,14,c1087885,7b3,...) at sys_nanosleep+0x6b/frame 0xf982fc40 syscall(f982fd08) at syscall+0x2ed/frame 0xf982fcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf982fcfc --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x2817637f, esp = 0xbfbfdcfc, ebp = 0xbfbfdd40 --- Tracing command tail pid 842 tid 100140 td 0xcb19cbc0 sched_switch(cb19cbc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf97c0948 mi_switch(104,0,c108af51,1f7,c13f9c00,...) at mi_switch+0x1c9/frame 0xf97c097c sleepq_switch(cb19cbc0,0,c108af51,1ab,f97c09f4,...) at sleepq_switch+0x1b9/frame 0xf97c09b0 sleepq_catch_signals(3e9,c0ad5e10,cb19cbc0,3,3,...) at sleepq_catch_signals+0x3b1/frame 0xf97c09f8 sleepq_timedwait_sig(cbbbd780,68,c107ea50,100,0,...) at sleepq_timedwait_sig+0x14/frame 0xf97c0a20 _sleep(cbbbd780,cbbbd780,168,c107ea50,3e9,...) at _sleep+0x303/frame 0xf97c0a78 kern_kevent(cb19cbc0,4,0,1,f97c0c10,...) at kern_kevent+0x444/frame 0xf97c0bb8 sys_kevent(cb19cbc0,f97c0cc8,f97c0d08,1,1,...) at sys_kevent+0x12b/frame 0xf97c0c40 syscall(f97c0d08) at syscall+0x2ed/frame 0xf97c0cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf97c0cfc --- syscall (363, FreeBSD ELF32, sys_kevent), eip = 0x281674bf, esp = 0xbfbfdb9c, ebp = 0xbfbfdc48 --- Tracing command sshd pid 841 tid 100098 td 0xca752bc0 sched_switch(ca752bc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf96c7918 mi_switch(104,0,c108af51,1f7,200046,...) at mi_switch+0x1c9/frame 0xf96c794c sleepq_switch(ca752bc0,0,c108af51,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf96c7980 sleepq_catch_signals(f96c79ec,c0a80379,ca78d460,0,0,...) at sleepq_catch_signals+0x3b1/frame 0xf96c79c8 sleepq_wait_sig(ca78d464,0,f96c7a20,101,0,...) at sleepq_wait_sig+0x14/frame 0xf96c79ec _cv_wait_sig(ca78d464,ca78d450,c108dd00,677,c,...) at _cv_wait_sig+0x200/frame 0xf96c7a34 seltdwait(cbc4a118,58,ca7e5b00,ca752bc0,c137b7a8,...) at seltdwait+0x8c/frame 0xf96c7a58 kern_select(ca752bc0,c,28c0c1c8,28c0c1d0,0,...) at kern_select+0x85f/frame 0xf96c7bf8 sys_select(ca752bc0,f96c7cc8,7b3,0,f96c7cc4,...) at sys_select+0x69/frame 0xf96c7c40 syscall(f96c7d08) at syscall+0x2ed/frame 0xf96c7cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf96c7cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28522f77, esp = 0xbfbfd3cc, ebp = 0xbfbfd418 --- Tracing command awk pid 836 tid 100134 td 0xcb19d2f0 sched_switch(cb19d2f0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf97aea30 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf97aea64 sleepq_switch(cb19d2f0,0,c108af51,1ab,f97aeadc,...) at sleepq_switch+0x1b9/frame 0xf97aea98 sleepq_catch_signals(0,c108af51,164,390,cb19d2f0,...) at sleepq_catch_signals+0x3b1/frame 0xf97aeae0 sleepq_wait_sig(cbc3a660,5c,c108df45,100,0,...) at sleepq_wait_sig+0x14/frame 0xf97aeb04 _sleep(cbc3a660,cbc3a7e0,15c,c108df45,0,...) at _sleep+0x32d/frame 0xf97aeb5c pipe_read(ca766a80,f97aec10,ca60da00,0,cb19d2f0,...) at pipe_read+0x29c/frame 0xf97aeb90 dofileread(ca766a80,f97aec10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf97aebc4 kern_readv(cb19d2f0,0,f97aec10,0,28855000,...) at kern_readv+0x66/frame 0xf97aebf0 sys_read(cb19d2f0,f97aecc8,f97aed08,1,1,...) at sys_read+0x5c/frame 0xf97aec40 syscall(f97aed08) at syscall+0x2ed/frame 0xf97aecfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf97aecfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281c1fb7, esp = 0xbfbfd9e4, ebp = 0xbfbfd9f8 --- Tracing command sh pid 835 tid 100148 td 0xcbbbc000 sched_switch(cbbbc000,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf97d8998 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf97d89cc sleepq_switch(cbbbc000,0,c108af51,1ab,f97d8a44,...) at sleepq_switch+0x1b9/frame 0xf97d8a00 sleepq_catch_signals(0,c108af51,164,540,cbbbc000,...) at sleepq_catch_signals+0x3b1/frame 0xf97d8a48 sleepq_wait_sig(cb1a3870,6c,c108e435,100,0,...) at sleepq_wait_sig+0x14/frame 0xf97d8a6c _sleep(cb1a3870,cb1a38f8,16c,c108e435,0,...) at _sleep+0x32d/frame 0xf97d8ac4 kern_wait6(cbbbc000,7,0,0,f97d8b4c,...) at kern_wait6+0x43c/frame 0xf97d8b20 sys_wait4(cbbbc000,f97d8cc8,c10dc7ab,e9,1,...) at sys_wait4+0x94/frame 0xf97d8c40 syscall(f97d8d08) at syscall+0x2ed/frame 0xf97d8cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf97d8cfc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2815098f, esp = 0xbfbfd938, ebp = 0xbfbfd950 --- Tracing command sh pid 834 tid 100150 td 0xcbbbb8d0 Tracing command sshd pid 833 tid 100106 td 0xca7525e0 sched_switch(ca7525e0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf96e8918 mi_switch(104,0,c108af51,1f7,200046,...) at mi_switch+0x1c9/frame 0xf96e894c sleepq_switch(ca7525e0,0,c108af51,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf96e8980 sleepq_catch_signals(f96e89ec,c0a80379,ca78d560,0,0,...) at sleepq_catch_signals+0x3b1/frame 0xf96e89c8 sleepq_wait_sig(ca78d564,0,f96e8a20,101,0,...) at sleepq_wait_sig+0x14/frame 0xf96e89ec _cv_wait_sig(ca78d564,ca78d550,c108dd00,677,c,...) at _cv_wait_sig+0x200/frame 0xf96e8a34 seltdwait(ca765b98,58,ca7e5480,ca7525e0,c137b858,...) at seltdwait+0x8c/frame 0xf96e8a58 kern_select(ca7525e0,c,28c0c1c8,28c0c1d0,0,...) at kern_select+0x85f/frame 0xf96e8bf8 sys_select(ca7525e0,f96e8cc8,7b3,0,f96e8cc4,...) at sys_select+0x69/frame 0xf96e8c40 syscall(f96e8d08) at syscall+0x2ed/frame 0xf96e8cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf96e8cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28522f77, esp = 0xbfbfd3cc, ebp = 0xbfbfd418 --- Tracing command top pid 832 tid 100128 td 0xcb1a68d0 sched_switch(cb1a68d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf979c914 mi_switch(104,0,c108af51,1f7,c13f9c00,...) at mi_switch+0x1c9/frame 0xf979c948 sleepq_switch(cb1a68d0,0,c108af51,1ab,ca78d310,...) at sleepq_switch+0x1b9/frame 0xf979c97c sleepq_catch_signals(c11b867c,f979c9ec,c0a80379,ca78d320,0,...) at sleepq_catch_signals+0x3b1/frame 0xf979c9c4 sleepq_timedwait_sig(ca78d324,0,f979ca20,101,0,...) at sleepq_timedwait_sig+0x14/frame 0xf979c9ec _cv_timedwait_sig(ca78d324,ca78d310,3e9,677,f979cac0,...) at _cv_timedwait_sig+0x20c/frame 0xf979ca34 seltdwait(f979cac0,f979cac8,ca6ef380,cb1a68d0,325b1b30,...) at seltdwait+0x7e/frame 0xf979ca58 kern_select(cb1a68d0,2,bfbfdbe8,0,0,...) at kern_select+0x85f/frame 0xf979cbf8 sys_select(cb1a68d0,f979ccc8,8,c1085aac,db,...) at sys_select+0x69/frame 0xf979cc40 syscall(f979cd08) at syscall+0x2ed/frame 0xf979ccfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf979ccfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281f3f77, esp = 0xbfbfdb7c, ebp = 0xbfbfdcf4 --- Tracing command sshd pid 831 tid 100122 td 0xca78f000 sched_switch(ca78f000,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf978a918 mi_switch(104,0,c108af51,1f7,200046,...) at mi_switch+0x1c9/frame 0xf978a94c sleepq_switch(ca78f000,0,c108af51,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf978a980 sleepq_catch_signals(f978a9ec,c0a80379,ca78d0e0,0,0,...) at sleepq_catch_signals+0x3b1/frame 0xf978a9c8 sleepq_wait_sig(ca78d0e4,0,f978aa20,101,0,...) at sleepq_wait_sig+0x14/frame 0xf978a9ec _cv_wait_sig(ca78d0e4,ca78d0d0,c108dd00,677,a,...) at _cv_wait_sig+0x200/frame 0xf978aa34 seltdwait(ca7adb28,58,ca4b4b80,ca78f000,c137b8b0,...) at seltdwait+0x8c/frame 0xf978aa58 kern_select(ca78f000,a,28c0c1c8,28c0c1d0,0,...) at kern_select+0x85f/frame 0xf978abf8 sys_select(ca78f000,f978acc8,f978ad08,1,1,...) at sys_select+0x69/frame 0xf978ac40 syscall(f978ad08) at syscall+0x2ed/frame 0xf978acfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf978acfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28522f77, esp = 0xbfbfd3cc, ebp = 0xbfbfd418 --- Tracing command sshd pid 829 tid 100118 td 0xca78fbc0 sched_switch(ca78fbc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf977e988 mi_switch(104,0,c108af51,1f7,200046,...) at mi_switch+0x1c9/frame 0xf977e9bc sleepq_switch(ca78fbc0,0,c108af51,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf977e9f0 sleepq_catch_signals(f977ea5c,c0a80379,ca7f0da0,0,0,...) at sleepq_catch_signals+0x3b1/frame 0xf977ea38 sleepq_wait_sig(ca7f0da4,0,f977ea90,101,0,...) at sleepq_wait_sig+0x14/frame 0xf977ea5c _cv_wait_sig(ca7f0da4,ca7f0d90,c108dd00,677,ca78fbc0,...) at _cv_wait_sig+0x200/frame 0xf977eaa4 seltdwait(ca78632c,c108dd00,5ac,ca78fbc0,ca78fbc0,...) at seltdwait+0x8c/frame 0xf977eac8 sys_poll(ca78fbc0,f977ecc8,f977ed08,1,1,...) at sys_poll+0x3e5/frame 0xf977ec40 syscall(f977ed08) at syscall+0x2ed/frame 0xf977ecfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf977ecfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x284b9aaf, esp = 0xbfbfd414, ebp = 0xbfbfd460 --- Tracing command sshd pid 826 tid 100119 td 0xca78f8d0 sched_switch(ca78f8d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9781988 mi_switch(104,0,c108af51,1f7,200046,...) at mi_switch+0x1c9/frame 0xf97819bc sleepq_switch(ca78f8d0,0,c108af51,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf97819f0 sleepq_catch_signals(f9781a5c,c0a80379,ca7f0d20,0,0,...) at sleepq_catch_signals+0x3b1/frame 0xf9781a38 sleepq_wait_sig(ca7f0d24,0,f9781a90,101,0,...) at sleepq_wait_sig+0x14/frame 0xf9781a5c _cv_wait_sig(ca7f0d24,ca7f0d10,c108dd00,677,ca78f8d0,...) at _cv_wait_sig+0x200/frame 0xf9781aa4 seltdwait(ca787c2c,c108dd00,5ac,ca78f8d0,ca78f8d0,...) at seltdwait+0x8c/frame 0xf9781ac8 sys_poll(ca78f8d0,f9781cc8,f9781d08,1,1,...) at sys_poll+0x3e5/frame 0xf9781c40 syscall(f9781d08) at syscall+0x2ed/frame 0xf9781cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9781cfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x284b9aaf, esp = 0xbfbfd414, ebp = 0xbfbfd460 --- Tracing command sshd pid 825 tid 100120 td 0xca78f5e0 sched_switch(ca78f5e0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9784988 mi_switch(104,0,c108af51,1f7,200046,...) at mi_switch+0x1c9/frame 0xf97849bc sleepq_switch(ca78f5e0,0,c108af51,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf97849f0 sleepq_catch_signals(f9784a5c,c0a80379,c7de4b60,0,0,...) at sleepq_catch_signals+0x3b1/frame 0xf9784a38 sleepq_wait_sig(c7de4b64,0,f9784a90,101,0,...) at sleepq_wait_sig+0x14/frame 0xf9784a5c _cv_wait_sig(c7de4b64,c7de4b50,c108dd00,677,ca78f5e0,...) at _cv_wait_sig+0x200/frame 0xf9784aa4 seltdwait(cb1b932c,c108dd00,5ac,ca78f5e0,cab8a570,...) at seltdwait+0x8c/frame 0xf9784ac8 sys_poll(ca78f5e0,f9784cc8,f9784d08,1,1,...) at sys_poll+0x3e5/frame 0xf9784c40 syscall(f9784d08) at syscall+0x2ed/frame 0xf9784cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9784cfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x284b9aaf, esp = 0xbfbfd414, ebp = 0xbfbfd460 --- Tracing command sh pid 814 tid 100131 td 0xcb1a62f0 sched_switch(cb1a62f0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf97a5998 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf97a59cc sleepq_switch(cb1a62f0,0,c108af51,1ab,f97a5a44,...) at sleepq_switch+0x1b9/frame 0xf97a5a00 sleepq_catch_signals(0,c108af51,164,1f8,cb1a62f0,...) at sleepq_catch_signals+0x3b1/frame 0xf97a5a48 sleepq_wait_sig(cb1a15a0,6c,c108e435,100,0,...) at sleepq_wait_sig+0x14/frame 0xf97a5a6c _sleep(cb1a15a0,cb1a1628,16c,c108e435,0,...) at _sleep+0x32d/frame 0xf97a5ac4 kern_wait6(cb1a62f0,7,0,0,f97a5b4c,...) at kern_wait6+0x43c/frame 0xf97a5b20 sys_wait4(cb1a62f0,f97a5cc8,c10dc7ab,e9,c123f590,...) at sys_wait4+0x94/frame 0xf97a5c40 syscall(f97a5d08) at syscall+0x2ed/frame 0xf97a5cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf97a5cfc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2815098f, esp = 0xbfbfd718, ebp = 0xbfbfd730 --- Tracing command bash pid 810 tid 100100 td 0xca7f72f0 Tracing command csh pid 807 tid 100137 td 0xcb19d000 Tracing command su pid 806 tid 100092 td 0xca7935e0 Tracing command bash pid 794 tid 100129 td 0xcb1a65e0 Tracing command sshd pid 793 tid 100114 td 0xc950f8d0 sched_switch(c950f8d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9772918 mi_switch(104,0,c108af51,1f7,200046,...) at mi_switch+0x1c9/frame 0xf977294c sleepq_switch(c950f8d0,0,c108af51,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf9772980 sleepq_catch_signals(f97729ec,c0a80379,ca623260,0,0,...) at sleepq_catch_signals+0x3b1/frame 0xf97729c8 sleepq_wait_sig(ca623264,0,f9772a20,101,0,...) at sleepq_wait_sig+0x14/frame 0xf97729ec _cv_wait_sig(ca623264,ca623250,c108dd00,677,c,...) at _cv_wait_sig+0x200/frame 0xf9772a34 seltdwait(ca7ad7e0,58,ca7e5780,c950f8d0,c137bb18,...) at seltdwait+0x8c/frame 0xf9772a58 kern_select(c950f8d0,c,28c0c1c8,28c0c1d0,0,...) at kern_select+0x85f/frame 0xf9772bf8 sys_select(c950f8d0,f9772cc8,f9772d08,1,1,...) at sys_select+0x69/frame 0xf9772c40 syscall(f9772d08) at syscall+0x2ed/frame 0xf9772cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9772cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28522f77, esp = 0xbfbfd3cc, ebp = 0xbfbfd418 --- Tracing command sshd pid 791 tid 100145 td 0xcb1a55e0 sched_switch(cb1a55e0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf97cf988 mi_switch(104,0,c108af51,1f7,200046,...) at mi_switch+0x1c9/frame 0xf97cf9bc sleepq_switch(cb1a55e0,0,c108af51,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf97cf9f0 sleepq_catch_signals(f97cfa5c,c0a80379,ca7f0ca0,0,0,...) at sleepq_catch_signals+0x3b1/frame 0xf97cfa38 sleepq_wait_sig(ca7f0ca4,0,f97cfa90,101,0,...) at sleepq_wait_sig+0x14/frame 0xf97cfa5c _cv_wait_sig(ca7f0ca4,ca7f0c90,c108dd00,677,cb1a55e0,...) at _cv_wait_sig+0x200/frame 0xf97cfaa4 seltdwait(cb1b992c,c108dd00,5ac,cb1a55e0,ffffff80,...) at seltdwait+0x8c/frame 0xf97cfac8 sys_poll(cb1a55e0,f97cfcc8,f97cfd08,1,1,...) at sys_poll+0x3e5/frame 0xf97cfc40 syscall(f97cfd08) at syscall+0x2ed/frame 0xf97cfcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf97cfcfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x284b9aaf, esp = 0xbfbfd414, ebp = 0xbfbfd460 --- Tracing command cron pid 744 tid 100108 td 0xca7debc0 sched_switch(ca7debc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf96f7a80 mi_switch(104,0,c108af51,1f7,c13f9a00,...) at mi_switch+0x1c9/frame 0xf96f7ab4 sleepq_switch(ca7debc0,0,c108af51,1ab,f96f7b2c,...) at sleepq_switch+0x1b9/frame 0xf96f7ae8 sleepq_catch_signals(96fc,c0ad5e10,ca7debc0,1,1,...) at sleepq_catch_signals+0x3b1/frame 0xf96f7b30 sleepq_timedwait_sig(c1231e80,6c,c1086a3e,100,0,...) at sleepq_timedwait_sig+0x14/frame 0xf96f7b58 _sleep(c1231e80,0,16c,c1086a3e,96fc,...) at _sleep+0x303/frame 0xf96f7bb0 kern_nanosleep(bfbfde28,f96f7c18,f96f7c20,f96f7cc8,26,...) at kern_nanosleep+0xd9/frame 0xf96f7c00 sys_nanosleep(ca7debc0,f96f7cc8,f96f7d08,1,1,...) at sys_nanosleep+0x6b/frame 0xf96f7c40 syscall(f96f7d08) at syscall+0x2ed/frame 0xf96f7cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf96f7cfc --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x2818537f, esp = 0xbfbfdcb4, ebp = 0xbfbfde44 --- Tracing command sendmail pid 739 tid 100089 td 0xca7562f0 Tracing command sendmail pid 736 tid 100101 td 0xca7f7000 sched_switch(ca7f7000,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf96d4914 mi_switch(104,0,c108af51,1f7,c13f9b00,...) at mi_switch+0x1c9/frame 0xf96d4948 sleepq_switch(ca7f7000,0,c108af51,1ab,ca78e510,...) at sleepq_switch+0x1b9/frame 0xf96d497c sleepq_catch_signals(c11b867c,f96d49ec,c0a80379,ca78e520,0,...) at sleepq_catch_signals+0x3b1/frame 0xf96d49c4 sleepq_timedwait_sig(ca78e524,0,f96d4a20,101,0,...) at sleepq_timedwait_sig+0x14/frame 0xf96d49ec _cv_timedwait_sig(ca78e524,ca78e510,1389,677,f96d4ac0,...) at _cv_timedwait_sig+0x20c/frame 0xf96d4a34 seltdwait(f96d4ac0,f96d4ac8,ca60d300,ca7f7000,f96d4aa0,...) at seltdwait+0x7e/frame 0xf96d4a58 kern_select(ca7f7000,5,bfbfb478,0,0,...) at kern_select+0x85f/frame 0xf96d4bf8 sys_select(ca7f7000,f96d4cc8,f96d4d08,1,1,...) at sys_select+0x69/frame 0xf96d4c40 syscall(f96d4d08) at syscall+0x2ed/frame 0xf96d4cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf96d4cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x2841cf77, esp = 0xbfbfb36c, ebp = 0xbfbfbf98 --- Tracing command sshd pid 733 tid 100113 td 0xc950fbc0 sched_switch(c950fbc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf976f918 mi_switch(104,0,c108af51,1f7,200046,...) at mi_switch+0x1c9/frame 0xf976f94c sleepq_switch(c950fbc0,0,c108af51,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf976f980 sleepq_catch_signals(f976f9ec,c0a80379,ca78d420,0,0,...) at sleepq_catch_signals+0x3b1/frame 0xf976f9c8 sleepq_wait_sig(ca78d424,0,f976fa20,101,0,...) at sleepq_wait_sig+0x14/frame 0xf976f9ec _cv_wait_sig(ca78d424,ca78d410,c108dd00,677,5,...) at _cv_wait_sig+0x200/frame 0xf976fa34 seltdwait(ca7e21f8,58,ca707980,c950fbc0,0,...) at seltdwait+0x8c/frame 0xf976fa58 kern_select(c950fbc0,9,28c0c140,0,0,...) at kern_select+0x85f/frame 0xf976fbf8 sys_select(c950fbc0,f976fcc8,0,0,c950fbc0,...) at sys_select+0x69/frame 0xf976fc40 syscall(f976fd08) at syscall+0x2ed/frame 0xf976fcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf976fcfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28522f77, esp = 0xbfbfd47c, ebp = 0xbfbfde44 --- Tracing command winbindd pid 708 tid 100112 td 0xca752000 sched_switch(ca752000,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf976c984 mi_switch(104,0,c108af51,1f7,c13f9b00,...) at mi_switch+0x1c9/frame 0xf976c9b8 sleepq_switch(ca752000,0,c108af51,1ab,c7de4c50,...) at sleepq_switch+0x1b9/frame 0xf976c9ec sleepq_catch_signals(c11b867c,f976ca5c,c0a80379,c7de4c60,0,...) at sleepq_catch_signals+0x3b1/frame 0xf976ca34 sleepq_timedwait_sig(c7de4c64,0,f976ca90,101,0,...) at sleepq_timedwait_sig+0x14/frame 0xf976ca5c _cv_timedwait_sig(c7de4c64,c7de4c50,5265c00,677,ca752000,...) at _cv_timedwait_sig+0x20c/frame 0xf976caa4 seltdwait(f976cb10,f976cb18,5ac,ca752000,cafbb120,...) at seltdwait+0x7e/frame 0xf976cac8 sys_poll(ca752000,f976ccc8,14,c1087885,7b3,...) at sys_poll+0x3e5/frame 0xf976cc40 syscall(f976cd08) at syscall+0x2ed/frame 0xf976ccfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf976ccfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x21937aaf, esp = 0xbfbfc47c, ebp = 0xbfbfc4b8 --- Tracing command moused pid 700 tid 100078 td 0xca7948d0 sched_switch(ca7948d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9677918 mi_switch(104,0,c108af51,1f7,46,...) at mi_switch+0x1c9/frame 0xf967794c sleepq_switch(ca7948d0,0,c108af51,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf9677980 sleepq_catch_signals(f96779ec,c0a80379,ca78e320,0,0,...) at sleepq_catch_signals+0x3b1/frame 0xf96779c8 sleepq_wait_sig(ca78e324,0,f9677a20,101,0,...) at sleepq_wait_sig+0x14/frame 0xf96779ec _cv_wait_sig(ca78e324,ca78e310,c108dd00,677,6,...) at _cv_wait_sig+0x200/frame 0xf9677a34 seltdwait(ca7adbd0,58,c7d7ce00,ca7948d0,0,...) at seltdwait+0x8c/frame 0xf9677a58 kern_select(ca7948d0,400,bfbfdcb0,0,0,...) at kern_select+0x85f/frame 0xf9677bf8 sys_select(ca7948d0,f9677cc8,c1825c80,c1825ccc,0,...) at sys_select+0x69/frame 0xf9677c40 syscall(f9677d08) at syscall+0x2ed/frame 0xf9677cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9677cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281bcf77, esp = 0xbfbfd88c, ebp = 0xbfbfde24 --- Tracing command smbd pid 699 tid 100097 td 0xca756000 sched_switch(ca756000,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf96c3984 mi_switch(104,0,c108af51,1f7,c13f9b00,...) at mi_switch+0x1c9/frame 0xf96c39b8 sleepq_switch(ca756000,0,c108af51,1ab,ca751e10,...) at sleepq_switch+0x1b9/frame 0xf96c39ec sleepq_catch_signals(c11b867c,f96c3a5c,c0a80379,ca751e20,0,...) at sleepq_catch_signals+0x3b1/frame 0xf96c3a34 sleepq_timedwait_sig(ca751e24,0,f96c3a90,101,0,...) at sleepq_timedwait_sig+0x14/frame 0xf96c3a5c _cv_timedwait_sig(ca751e24,ca751e10,5265c00,677,ca756000,...) at _cv_timedwait_sig+0x20c/frame 0xf96c3aa4 seltdwait(f96c3b10,f96c3b18,5ac,ca756000,cab8abc8,...) at seltdwait+0x7e/frame 0xf96c3ac8 sys_poll(ca756000,f96c3cc8,f96c3d08,1,1,...) at sys_poll+0x3e5/frame 0xf96c3c40 syscall(f96c3d08) at syscall+0x2ed/frame 0xf96c3cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf96c3cfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x21b60aaf, esp = 0xbfbfd9ec, ebp = 0xbfbfda28 --- Tracing command winbindd pid 683 tid 100079 td 0xca7945e0 sched_switch(ca7945e0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf967b984 mi_switch(104,0,c108af51,1f7,c13f9b00,...) at mi_switch+0x1c9/frame 0xf967b9b8 sleepq_switch(ca7945e0,0,c108af51,1ab,ca78d110,...) at sleepq_switch+0x1b9/frame 0xf967b9ec sleepq_catch_signals(c11b867c,f967ba5c,c0a80379,ca78d120,0,...) at sleepq_catch_signals+0x3b1/frame 0xf967ba34 sleepq_timedwait_sig(ca78d124,0,f967ba90,101,0,...) at sleepq_timedwait_sig+0x14/frame 0xf967ba5c _cv_timedwait_sig(ca78d124,ca78d110,49388,677,ca7945e0,...) at _cv_timedwait_sig+0x20c/frame 0xf967baa4 seltdwait(f967bb10,f967bb18,5ac,ca7945e0,c7d86268,...) at seltdwait+0x7e/frame 0xf967bac8 sys_poll(ca7945e0,f967bcc8,14,c1087885,7b3,...) at sys_poll+0x3e5/frame 0xf967bc40 syscall(f967bd08) at syscall+0x2ed/frame 0xf967bcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf967bcfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x21937aaf, esp = 0xbfbfdbcc, ebp = 0xbfbfdc08 --- Tracing command smbd pid 681 tid 100107 td 0xca7522f0 sched_switch(ca7522f0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf96ec984 mi_switch(104,0,c108af51,1f7,c13f9c00,...) at mi_switch+0x1c9/frame 0xf96ec9b8 sleepq_switch(ca7522f0,0,c108af51,1ab,c7de4c10,...) at sleepq_switch+0x1b9/frame 0xf96ec9ec sleepq_catch_signals(c11b867c,f96eca5c,c0a80379,c7de4c20,0,...) at sleepq_catch_signals+0x3b1/frame 0xf96eca34 sleepq_timedwait_sig(c7de4c24,0,f96eca90,101,0,...) at sleepq_timedwait_sig+0x14/frame 0xf96eca5c _cv_timedwait_sig(c7de4c24,c7de4c10,ea61,677,ca7522f0,...) at _cv_timedwait_sig+0x20c/frame 0xf96ecaa4 seltdwait(f96ecb10,f96ecb18,5ac,ca7522f0,ca74aae0,...) at seltdwait+0x7e/frame 0xf96ecac8 sys_poll(ca7522f0,f96eccc8,0,0,ca7522f0,...) at sys_poll+0x3e5/frame 0xf96ecc40 syscall(f96ecd08) at syscall+0x2ed/frame 0xf96eccfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf96eccfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x21b60aaf, esp = 0xbfbfda9c, ebp = 0xbfbfdad8 --- Tracing command nmbd pid 679 tid 100077 td 0xca794bc0 sched_switch(ca794bc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9673984 mi_switch(104,0,c108af51,1f7,c13f9a00,...) at mi_switch+0x1c9/frame 0xf96739b8 sleepq_switch(ca794bc0,0,c108af51,1ab,ca7f0a90,...) at sleepq_switch+0x1b9/frame 0xf96739ec sleepq_catch_signals(c11b867c,f9673a5c,c0a80379,ca7f0aa0,0,...) at sleepq_catch_signals+0x3b1/frame 0xf9673a34 sleepq_timedwait_sig(ca7f0aa4,0,f9673a90,101,0,...) at sleepq_timedwait_sig+0x14/frame 0xf9673a5c _cv_timedwait_sig(ca7f0aa4,ca7f0a90,2711,677,ca794bc0,...) at _cv_timedwait_sig+0x20c/frame 0xf9673aa4 seltdwait(f9673b10,f9673b18,5ac,ca794bc0,cab8a9f8,...) at seltdwait+0x7e/frame 0xf9673ac8 sys_poll(ca794bc0,f9673cc8,f9673d08,1,1,...) at sys_poll+0x3e5/frame 0xf9673c40 syscall(f9673d08) at syscall+0x2ed/frame 0xf9673cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9673cfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x21562aaf, esp = 0xbfbfdb1c, ebp = 0xbfbfdb58 --- Tracing command ntpd pid 655 tid 100082 td 0xca793bc0 sched_switch(ca793bc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9687918 mi_switch(104,0,c108af51,1f7,200046,...) at mi_switch+0x1c9/frame 0xf968794c sleepq_switch(ca793bc0,0,c108af51,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf9687980 sleepq_catch_signals(f96879ec,c0a80379,ca7f08e0,0,0,...) at sleepq_catch_signals+0x3b1/frame 0xf96879c8 sleepq_wait_sig(ca7f08e4,0,f9687a20,101,0,...) at sleepq_wait_sig+0x14/frame 0xf96879ec _cv_wait_sig(ca7f08e4,ca7f08d0,c108dd00,677,1c,...) at _cv_wait_sig+0x200/frame 0xf9687a34 seltdwait(ca79c348,58,c7d7ce00,ca793bc0,0,...) at seltdwait+0x8c/frame 0xf9687a58 kern_select(ca793bc0,1c,bfbfdd48,0,0,...) at kern_select+0x85f/frame 0xf9687bf8 sys_select(ca793bc0,f9687cc8,0,0,ca793bc0,...) at sys_select+0x69/frame 0xf9687c40 syscall(f9687d08) at syscall+0x2ed/frame 0xf9687cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf9687cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28396f77, esp = 0xbfbfdd24, ebp = 0xbfbfdde4 --- Tracing command nfsd pid 602 tid 100111 td 0xca7de2f0 sched_switch(ca7de2f0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9765a90 mi_switch(104,0,c108af51,1f7,c13f9900,...) at mi_switch+0x1c9/frame 0xf9765ac4 sleepq_switch(ca7de2f0,0,c108af51,1ab,c803c980,...) at sleepq_switch+0x1b9/frame 0xf9765af8 sleepq_catch_signals(c11b867c,f9765b68,c0a80379,c803c990,0,...) at sleepq_catch_signals+0x3b1/frame 0xf9765b40 sleepq_timedwait_sig(ca6c0090,0,f9765b9c,101,0,...) at sleepq_timedwait_sig+0x14/frame 0xf9765b68 _cv_timedwait_sig(ca6c0090,c803c980,1388,3b1,0,...) at _cv_timedwait_sig+0x20c/frame 0xf9765bb0 svc_run_internal(f9765cf4,c0a6381f,c803c980,f9765d08,c107ef40,...) at svc_run_internal+0xa50/frame 0xf9765cc4 svc_thread_start(c803c980,f9765d08,c107ef40,3d7,0,...) at svc_thread_start+0xd/frame 0xf9765ccc fork_exit(c0c900f0,c803c980,f9765d08) at fork_exit+0x7f/frame 0xf9765cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf9765cf4 --- trap 0x2608f32e, eip = 0xc, esp = 0x33, ebp = 0xbfbfd92f --- Tracing command nfsd pid 602 tid 100110 td 0xca7de5e0 sched_switch(ca7de5e0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9762a90 mi_switch(104,0,c108af51,1f7,c13f9a00,...) at mi_switch+0x1c9/frame 0xf9762ac4 sleepq_switch(ca7de5e0,0,c108af51,1ab,c803c980,...) at sleepq_switch+0x1b9/frame 0xf9762af8 sleepq_catch_signals(c11b867c,f9762b68,c0a80379,c803c990,0,...) at sleepq_catch_signals+0x3b1/frame 0xf9762b40 sleepq_timedwait_sig(ca6c1810,0,f9762b9c,101,0,...) at sleepq_timedwait_sig+0x14/frame 0xf9762b68 _cv_timedwait_sig(ca6c1810,c803c980,1388,3b1,0,...) at _cv_timedwait_sig+0x20c/frame 0xf9762bb0 svc_run_internal(f9762cf4,c0a6381f,c803c980,f9762d08,c107ef40,...) at svc_run_internal+0xa50/frame 0xf9762cc4 svc_thread_start(c803c980,f9762d08,c107ef40,3d7,0,...) at svc_thread_start+0xd/frame 0xf9762ccc fork_exit(c0c900f0,c803c980,f9762d08) at fork_exit+0x7f/frame 0xf9762cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf9762cf4 --- trap 0x2608f32e, eip = 0xc, esp = 0x33, ebp = 0xbfbfd92f --- Tracing command nfsd pid 602 tid 100109 td 0xca7de8d0 sched_switch(ca7de8d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf975fa90 mi_switch(104,0,c108af51,1f7,c13f9a00,...) at mi_switch+0x1c9/frame 0xf975fac4 sleepq_switch(ca7de8d0,0,c108af51,1ab,c803c980,...) at sleepq_switch+0x1b9/frame 0xf975faf8 sleepq_catch_signals(c11b867c,f975fb68,c0a80379,c803c990,0,...) at sleepq_catch_signals+0x3b1/frame 0xf975fb40 sleepq_timedwait_sig(ca6c00d0,0,f975fb9c,101,0,...) at sleepq_timedwait_sig+0x14/frame 0xf975fb68 _cv_timedwait_sig(ca6c00d0,c803c980,1388,48a,0,...) at _cv_timedwait_sig+0x20c/frame 0xf975fbb0 svc_run_internal(f975fcf4,c0a6381f,c803c980,f975fd08,c107ef40,...) at svc_run_internal+0xa50/frame 0xf975fcc4 svc_thread_start(c803c980,f975fd08,c107ef40,3d7,0,...) at svc_thread_start+0xd/frame 0xf975fccc fork_exit(c0c900f0,c803c980,f975fd08) at fork_exit+0x7f/frame 0xf975fcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf975fcf4 --- trap 0x2608f32e, eip = 0xc, esp = 0x33, ebp = 0xbfbfd92f --- Tracing command nfsd pid 602 tid 100084 td 0xca7e1000 sched_switch(ca7e1000,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf968f318 mi_switch(104,0,c108af51,1f7,c13f9900,...) at mi_switch+0x1c9/frame 0xf968f34c sleepq_switch(ca7e1000,0,c108af51,1ab,c803c980,...) at sleepq_switch+0x1b9/frame 0xf968f380 sleepq_catch_signals(c11b867c,f968f3f0,c0a80379,c803c990,0,...) at sleepq_catch_signals+0x3b1/frame 0xf968f3c8 sleepq_timedwait_sig(ca60ea50,0,f968f424,101,0,...) at sleepq_timedwait_sig+0x14/frame 0xf968f3f0 _cv_timedwait_sig(ca60ea50,c803c980,1388,3b1,c7d7e340,...) at _cv_timedwait_sig+0x20c/frame 0xf968f438 svc_run_internal(c0c900f0,c803c980,c805fb40,f968f578,0,...) at svc_run_internal+0xa50/frame 0xf968f550 svc_run(c803c980,0,c1070c77,1b2,f968f5d0,...) at svc_run+0xc3/frame 0xf968f588 nfsrvd_nfsd(ca7e1000,f968f6e0,c,b1,1,...) at nfsrvd_nfsd+0x1a2/frame 0xf968f6c0 nfssvc_nfsd(ca7e1000,f968fcc8,0,ca7e1000,f968fcfc,...) at nfssvc_nfsd+0xf4/frame 0xf968fc28 sys_nfssvc(ca7e1000,f968fcc8,c10dc7ab,e9,1,...) at sys_nfssvc+0xbf/frame 0xf968fc40 syscall(f968fd08) at syscall+0x2ed/frame 0xf968fcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf968fcfc --- syscall (155, FreeBSD ELF32, sys_nfssvc), eip = 0x280c1ae7, esp = 0xbfbfd7dc, ebp = 0xbfbfda50 --- Tracing command nfsd pid 596 tid 100083 td 0xc807b8d0 sched_switch(c807b8d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf968b918 mi_switch(104,0,c108af51,1f7,46,...) at mi_switch+0x1c9/frame 0xf968b94c sleepq_switch(c807b8d0,0,c108af51,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf968b980 sleepq_catch_signals(f968b9ec,c0a80379,caed93a0,0,0,...) at sleepq_catch_signals+0x3b1/frame 0xf968b9c8 sleepq_wait_sig(caed93a4,0,f968ba20,101,0,...) at sleepq_wait_sig+0x14/frame 0xf968b9ec _cv_wait_sig(caed93a4,caed9390,c108dd00,677,7,...) at _cv_wait_sig+0x200/frame 0xf968ba34 seltdwait(ca766e00,58,c7d7ce00,c807b8d0,f968bad8,...) at seltdwait+0x8c/frame 0xf968ba58 kern_select(c807b8d0,7,bfbfdc30,0,0,...) at kern_select+0x85f/frame 0xf968bbf8 sys_select(c807b8d0,f968bcc8,c1825cb4,5,f968bc9c,...) at sys_select+0x69/frame 0xf968bc40 syscall(f968bd08) at syscall+0x2ed/frame 0xf968bcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf968bcfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28188f77, esp = 0xbfbfda54, ebp = 0xbfbfde34 --- Tracing command nfsuserd pid 594 tid 100080 td 0xca7942f0 sched_switch(ca7942f0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf967f914 mi_switch(104,0,c108af51,1f7,c13f9a00,...) at mi_switch+0x1c9/frame 0xf967f948 sleepq_switch(ca7942f0,0,c108af51,1ab,ca78d950,...) at sleepq_switch+0x1b9/frame 0xf967f97c sleepq_catch_signals(c11b867c,f967f9ec,c0a80379,ca78d960,0,...) at sleepq_catch_signals+0x3b1/frame 0xf967f9c4 sleepq_timedwait_sig(ca78d964,0,f967fa20,101,0,...) at sleepq_timedwait_sig+0x14/frame 0xf967f9ec _cv_timedwait_sig(ca78d964,ca78d950,7531,677,f967fac0,...) at _cv_timedwait_sig+0x20c/frame 0xf967fa34 seltdwait(f967fac0,f967fac8,c7d7ce00,ca7942f0,c123f61c,...) at seltdwait+0x7e/frame 0xf967fa58 kern_select(ca7942f0,4,bfbfdc10,0,0,...) at kern_select+0x85f/frame 0xf967fbf8 sys_select(ca7942f0,f967fcc8,7b3,0,f967fcc4,...) at sys_select+0x69/frame 0xf967fc40 syscall(f967fd08) at syscall+0x2ed/frame 0xf967fcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf967fcfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28187f77, esp = 0xbfbfdb64, ebp = 0xbfbfdca8 --- Tracing command nfsuserd pid 593 tid 100093 td 0xca7932f0 sched_switch(ca7932f0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf96b3914 mi_switch(104,0,c108af51,1f7,c13f9900,...) at mi_switch+0x1c9/frame 0xf96b3948 sleepq_switch(ca7932f0,0,c108af51,1ab,caed9490,...) at sleepq_switch+0x1b9/frame 0xf96b397c sleepq_catch_signals(c11b867c,f96b39ec,c0a80379,caed94a0,0,...) at sleepq_catch_signals+0x3b1/frame 0xf96b39c4 sleepq_timedwait_sig(caed94a4,0,f96b3a20,101,0,...) at sleepq_timedwait_sig+0x14/frame 0xf96b39ec _cv_timedwait_sig(caed94a4,caed9490,7531,677,f96b3ac0,...) at _cv_timedwait_sig+0x20c/frame 0xf96b3a34 seltdwait(f96b3ac0,f96b3ac8,c7d7ce00,ca7932f0,f9,...) at seltdwait+0x7e/frame 0xf96b3a58 kern_select(ca7932f0,4,bfbfdc10,0,0,...) at kern_select+0x85f/frame 0xf96b3bf8 sys_select(ca7932f0,f96b3cc8,f96b3d08,1,1,...) at sys_select+0x69/frame 0xf96b3c40 syscall(f96b3d08) at syscall+0x2ed/frame 0xf96b3cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf96b3cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28187f77, esp = 0xbfbfdb64, ebp = 0xbfbfdca8 --- Tracing command nfsuserd pid 592 tid 100088 td 0xca7565e0 sched_switch(ca7565e0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf969f914 mi_switch(104,0,c108af51,1f7,c13f9b00,...) at mi_switch+0x1c9/frame 0xf969f948 sleepq_switch(ca7565e0,0,c108af51,1ab,ca78ed10,...) at sleepq_switch+0x1b9/frame 0xf969f97c sleepq_catch_signals(c11b867c,f969f9ec,c0a80379,ca78ed20,0,...) at sleepq_catch_signals+0x3b1/frame 0xf969f9c4 sleepq_timedwait_sig(ca78ed24,0,f969fa20,101,0,...) at sleepq_timedwait_sig+0x14/frame 0xf969f9ec _cv_timedwait_sig(ca78ed24,ca78ed10,7531,677,f969fac0,...) at _cv_timedwait_sig+0x20c/frame 0xf969fa34 seltdwait(f969fac0,f969fac8,c7d7ce00,ca7565e0,f969fa84,...) at seltdwait+0x7e/frame 0xf969fa58 kern_select(ca7565e0,4,bfbfdc10,0,0,...) at kern_select+0x85f/frame 0xf969fbf8 sys_select(ca7565e0,f969fcc8,f969fd08,1,1,...) at sys_select+0x69/frame 0xf969fc40 syscall(f969fd08) at syscall+0x2ed/frame 0xf969fcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf969fcfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28187f77, esp = 0xbfbfdb64, ebp = 0xbfbfdca8 --- Tracing command nfsuserd pid 591 tid 100096 td 0xca793000 sched_switch(ca793000,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf96bf914 mi_switch(104,0,c108af51,1f7,c13f9a00,...) at mi_switch+0x1c9/frame 0xf96bf948 sleepq_switch(ca793000,0,c108af51,1ab,caed94d0,...) at sleepq_switch+0x1b9/frame 0xf96bf97c sleepq_catch_signals(c11b867c,f96bf9ec,c0a80379,caed94e0,0,...) at sleepq_catch_signals+0x3b1/frame 0xf96bf9c4 sleepq_timedwait_sig(caed94e4,0,f96bfa20,101,0,...) at sleepq_timedwait_sig+0x14/frame 0xf96bf9ec _cv_timedwait_sig(caed94e4,caed94d0,7531,677,f96bfac0,...) at _cv_timedwait_sig+0x20c/frame 0xf96bfa34 seltdwait(f96bfac0,f96bfac8,c7d7ce00,ca793000,f96bfa84,...) at seltdwait+0x7e/frame 0xf96bfa58 kern_select(ca793000,4,bfbfdc10,0,0,...) at kern_select+0x85f/frame 0xf96bfbf8 sys_select(ca793000,f96bfcc8,7b3,0,f96bfcc4,...) at sys_select+0x69/frame 0xf96bfc40 syscall(f96bfd08) at syscall+0x2ed/frame 0xf96bfcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf96bfcfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28187f77, esp = 0xbfbfdb64, ebp = 0xbfbfdca8 --- Tracing command nfsuserd pid 590 tid 100085 td 0xca7938d0 Tracing command mountd pid 574 tid 100087 td 0xca7f85e0 sched_switch(ca7f85e0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf969b918 mi_switch(104,0,c108af51,1f7,46,...) at mi_switch+0x1c9/frame 0xf969b94c sleepq_switch(ca7f85e0,0,c108af51,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf969b980 sleepq_catch_signals(f969b9ec,c0a80379,ca623160,0,0,...) at sleepq_catch_signals+0x3b1/frame 0xf969b9c8 sleepq_wait_sig(ca623164,0,f969ba20,101,0,...) at sleepq_wait_sig+0x14/frame 0xf969b9ec _cv_wait_sig(ca623164,ca623150,c108dd00,677,9,...) at _cv_wait_sig+0x200/frame 0xf969ba34 seltdwait(ca7ade70,58,c7d7ce00,ca7f85e0,c1095b77,...) at seltdwait+0x8c/frame 0xf969ba58 kern_select(ca7f85e0,9,bfbfdd50,0,0,...) at kern_select+0x85f/frame 0xf969bbf8 sys_select(ca7f85e0,f969bcc8,f969bd08,1,1,...) at sys_select+0x69/frame 0xf969bc40 syscall(f969bd08) at syscall+0x2ed/frame 0xf969bcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf969bcfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x2819ff77, esp = 0xbfbfdd0c, ebp = 0xbfbfde34 --- Tracing command rpcbind pid 541 tid 100094 td 0xca7f7bc0 sched_switch(ca7f7bc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf96b7984 mi_switch(104,0,c108af51,1f7,c13f9a00,...) at mi_switch+0x1c9/frame 0xf96b79b8 sleepq_switch(ca7f7bc0,0,c108af51,1ab,ca751990,...) at sleepq_switch+0x1b9/frame 0xf96b79ec sleepq_catch_signals(c11b867c,f96b7a5c,c0a80379,ca7519a0,0,...) at sleepq_catch_signals+0x3b1/frame 0xf96b7a34 sleepq_timedwait_sig(ca7519a4,0,f96b7a90,101,0,...) at sleepq_timedwait_sig+0x14/frame 0xf96b7a5c _cv_timedwait_sig(ca7519a4,ca751990,7531,677,ca7f7bc0,...) at _cv_timedwait_sig+0x20c/frame 0xf96b7aa4 seltdwait(f96b7b10,f96b7b18,5ac,ca7f7bc0,c10ddcb3,...) at seltdwait+0x7e/frame 0xf96b7ac8 sys_poll(ca7f7bc0,f96b7cc8,f96b7d08,1,1,...) at sys_poll+0x3e5/frame 0xf96b7c40 syscall(f96b7d08) at syscall+0x2ed/frame 0xf96b7cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf96b7cfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x2813eaaf, esp = 0xbfbfbbfc, ebp = 0xbfbfddd8 --- Tracing command syslogd pid 526 tid 100105 td 0xca7528d0 sched_switch(ca7528d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf96e4918 mi_switch(104,0,c108af51,1f7,46,...) at mi_switch+0x1c9/frame 0xf96e494c sleepq_switch(ca7528d0,0,c108af51,1ab,b1,...) at sleepq_switch+0x1b9/frame 0xf96e4980 sleepq_catch_signals(f96e49ec,c0a80379,ca751ca0,0,0,...) at sleepq_catch_signals+0x3b1/frame 0xf96e49c8 sleepq_wait_sig(ca751ca4,0,f96e4a20,101,0,...) at sleepq_wait_sig+0x14/frame 0xf96e49ec _cv_wait_sig(ca751ca4,ca751c90,c108dd00,677,9,...) at _cv_wait_sig+0x200/frame 0xf96e4a34 seltdwait(ca7654d0,58,c7d7ce00,ca7528d0,0,...) at seltdwait+0x8c/frame 0xf96e4a58 kern_select(ca7528d0,9,28849100,0,0,...) at kern_select+0x85f/frame 0xf96e4bf8 sys_select(ca7528d0,f96e4cc8,0,0,ca7528d0,...) at sys_select+0x69/frame 0xf96e4c40 syscall(f96e4d08) at syscall+0x2ed/frame 0xf96e4cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf96e4cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x2819ef77, esp = 0xbfbfceac, ebp = 0xbfbfde34 --- Tracing command devd pid 423 tid 100095 td 0xca7f78d0 sched_switch(ca7f78d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf96bb914 mi_switch(104,0,c108af51,1f7,c13f9a00,...) at mi_switch+0x1c9/frame 0xf96bb948 sleepq_switch(ca7f78d0,0,c108af51,1ab,ca78ed90,...) at sleepq_switch+0x1b9/frame 0xf96bb97c sleepq_catch_signals(c11b867c,f96bb9ec,c0a80379,ca78eda0,0,...) at sleepq_catch_signals+0x3b1/frame 0xf96bb9c4 sleepq_timedwait_sig(ca78eda4,0,f96bba20,101,0,...) at sleepq_timedwait_sig+0x14/frame 0xf96bb9ec _cv_timedwait_sig(ca78eda4,ca78ed90,ea61,677,f96bbac0,...) at _cv_timedwait_sig+0x20c/frame 0xf96bba34 seltdwait(f96bbac0,f96bbac8,c7d7ce00,ca7f78d0,ca58ccb8,...) at seltdwait+0x7e/frame 0xf96bba58 kern_select(ca7f78d0,5,bfbfd998,0,0,...) at kern_select+0x85f/frame 0xf96bbbf8 sys_select(ca7f78d0,f96bbcc8,f96bbd08,1,1,...) at sys_select+0x69/frame 0xf96bbc40 syscall(f96bbd08) at syscall+0x2ed/frame 0xf96bbcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf96bbcfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x8095903, esp = 0xbfbfd93c, ebp = 0xbfbfde54 --- Tracing command softdepflush pid 18 tid 100075 td 0xc807bbc0 sched_switch(c807bbc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf419d868 mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xf419d89c sleepq_switch(c807bbc0,0,c108af51,26d,0,...) at sleepq_switch+0x1b9/frame 0xf419d8d0 sleepq_wait(e1aaa734,60,c107cce6,4,0,...) at sleepq_wait+0x48/frame 0xf419d8f4 sleeplk(ca7c7ed0,c107cce6,60,0,0,...) at sleeplk+0x11e/frame 0xf419d930 __lockmgr_args(e1aaa734,81900,ca7c7ed0,c107cce6,60,...) at __lockmgr_args+0xaa4/frame 0xf419da0c getblk(ca7c7e10,11f224c0,0,4000,0,...) at getblk+0x1d8/frame 0xf419da7c breadn_flags(ca7c7e10,11f224c0,0,4000,0,...) at breadn_flags+0x40/frame 0xf419daac ffs_blkfree_cg(ca7c7e10,47df590,0,800,11f2468,...) at ffs_blkfree_cg+0x20f/frame 0xf419db24 ffs_blkfree(ca724a00,ca6da800,ca7c7e10,47df590,0,...) at ffs_blkfree+0xa5/frame 0xf419db60 freework_freeblock(c13ac2e4,0,c10ba87e,1dd4,c1097510,...) at freework_freeblock+0x667/frame 0xf419dbc8 handle_workitem_freeblocks(0,f419dc58,2,6da,ca724a00,...) at handle_workitem_freeblocks+0x197/frame 0xf419dc18 process_worklist_item(0,0,c10ba87e,609,513173e7,...) at process_worklist_item+0x29d/frame 0xf419dc70 softdep_process_worklist(ca7065d8,0,c10ba87e,582,3e8,...) at softdep_process_worklist+0x7c/frame 0xf419dc9c softdep_flush(0,f419dd08,c107ef40,3d7,0,...) at softdep_flush+0x139/frame 0xf419dccc fork_exit(c0ce8540,0,f419dd08) at fork_exit+0x7f/frame 0xf419dcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf419dcf4 --- trap 0, eip = 0, esp = 0xf419dd40, ebp = 0 --- Tracing command syncer pid 17 tid 100074 td 0xc950d000 kdb_enter(c10853c1,c10853c1,c10c0ab6,f419a9c8,0,...) at kdb_enter+0x3d/frame 0xf419a970 vpanic(f419a9c8,f419a9f0,c0d13eea,c10c0ab6,1000,...) at vpanic+0x142/frame 0xf419a9b0 panic(c10c0ab6,1000,19c00000,c10c0ab0,7d0,...) at panic+0x12/frame 0xf419a9bc kmem_malloc(c1bb508c,1000,102) at kmem_malloc+0x2ca/frame 0xf419a9f0 page_alloc(c1b9fd00,1000,f419aa2f,102,c1ba1698,...) at page_alloc+0x27/frame 0xf419aa04 keg_alloc_slab(102,4,c10bfd74,887,f419aa88,...) at keg_alloc_slab+0xc3/frame 0xf419aa3c keg_fetch_slab(102,80,102,ced2bc48,f419aad0,...) at keg_fetch_slab+0x132/frame 0xf419aa70 zone_fetch_slab(c1b9fd00,0,102,83d,202,...) at zone_fetch_slab+0x3f/frame 0xf419aa88 uma_zalloc_arg(c1b9fd00,0,102) at uma_zalloc_arg+0x435/frame 0xf419aad0 g_alloc_bio(c7ef79b0,0,c107414a,ba,ca6bb300,...) at g_alloc_bio+0x23/frame 0xf419aae4 g_vfs_strategy(ca8137c8,e19aa7d4,423,110d,c0b1a47b,...) at g_vfs_strategy+0xa0/frame 0xf419ab0c bufwrite(e19aa7d4,0,c10bd4a9,826,c7d86338) at bufwrite+0x16c/frame 0xf419ab3c ffs_bufwrite(e19aa7d4,c137c9e0,c7d84a08,246,c950d0b4,...) at ffs_bufwrite+0x13b/frame 0xf419ab60 vfs_bio_awrite(e19aa7d4,0,c1094c23,29f,0,...) at vfs_bio_awrite+0x205/frame 0xf419abc4 vop_stdfsync(f419aca8,f419ac18,0,c11a8a30,f419aca8,...) at vop_stdfsync+0x238/frame 0xf419ac08 devfs_fsync(f419aca8,c10e3454,80400,c1095b77,703,...) at devfs_fsync+0x2a/frame 0xf419ac28 VOP_FSYNC_APV(c11a8a30,f419aca8,c1095b77,703,4e20,...) at VOP_FSYNC_APV+0x124/frame 0xf419ac58 sched_sync(0,f419ad08,c107ef40,3d7,0,...) at sched_sync+0x38f/frame 0xf419accc fork_exit(c0b3bc40,0,f419ad08) at fork_exit+0x7f/frame 0xf419acf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf419acf4 --- trap 0, eip = 0, esp = 0xf419ad40, ebp = 0 --- Tracing command vnlru pid 16 tid 100073 td 0xc950d2f0 sched_switch(c950d2f0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf4197b8c mi_switch(104,0,c108af51,1f7,3,...) at mi_switch+0x1c9/frame 0xf4197bc0 sleepq_switch(c950d2f0,0,c108af51,290,c950d2f0,...) at sleepq_switch+0x1b9/frame 0xf4197bf4 sleepq_timedwait(ca62c870,60,c1096f0e,0,0,...) at sleepq_timedwait+0x48/frame 0xf4197c18 _sleep(ca62c870,c13a8128,260,c1096f0e,3e8,...) at _sleep+0x317/frame 0xf4197c70 vnlru_proc(0,f4197d08,c107ef40,3d7,0,...) at vnlru_proc+0x95/frame 0xf4197ccc fork_exit(c0b3c410,0,f4197d08) at fork_exit+0x7f/frame 0xf4197cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4197cf4 --- trap 0, eip = 0, esp = 0xf4197d40, ebp = 0 --- Tracing command bufdaemon pid 9 tid 100072 td 0xc950d5e0 sched_switch(c950d5e0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf4194bc8 mi_switch(104,0,c108af51,1f7,3,...) at mi_switch+0x1c9/frame 0xf4194bfc sleepq_switch(c950d5e0,0,c108af51,290,c950d5e0,...) at sleepq_switch+0x1b9/frame 0xf4194c30 sleepq_timedwait(c13a7ee8,54,c1093a04,0,0,...) at sleepq_timedwait+0x48/frame 0xf4194c54 _sleep(c13a7ee8,c13a7ed0,54,c1093a04,3e8,...) at _sleep+0x317/frame 0xf4194cac buf_daemon(0,f4194d08,c107ef40,3d7,0,...) at buf_daemon+0x8c/frame 0xf4194ccc fork_exit(c0b1f9c0,0,f4194d08) at fork_exit+0x7f/frame 0xf4194cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4194cf4 --- trap 0, eip = 0, esp = 0xf4194d40, ebp = 0 --- Tracing command pagezero pid 8 tid 100071 td 0xc950d8d0 sched_switch(c950d8d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf4191bc0 mi_switch(104,0,c108af51,1f7,1,...) at mi_switch+0x1c9/frame 0xf4191bf4 sleepq_switch(c950d8d0,0,c108af51,290,c950d8d0,...) at sleepq_switch+0x1b9/frame 0xf4191c28 sleepq_timedwait(c13acc3c,0,c10c40fa,0,0,...) at sleepq_timedwait+0x48/frame 0xf4191c4c _sleep(c13acc3c,c1400600,0,c10c40fa,493e0,...) at _sleep+0x317/frame 0xf4191ca4 vm_pagezero(0,f4191d08,c107ef40,3d7,0,...) at vm_pagezero+0x89/frame 0xf4191ccc fork_exit(c0d2e7c0,0,f4191d08) at fork_exit+0x7f/frame 0xf4191cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4191cf4 --- trap 0, eip = 0, esp = 0xf4191d40, ebp = 0 --- Tracing command vmdaemon pid 7 tid 100070 td 0xc950dbc0 sched_switch(c950dbc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf418eb8c mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xf418ebc0 sleepq_switch(c950dbc0,0,c108af51,26d,c950dbc0,...) at sleepq_switch+0x1b9/frame 0xf418ebf4 sleepq_wait(c13ac808,74,c1093a04,0,0,...) at sleepq_wait+0x48/frame 0xf418ec18 _sleep(c13ac808,c13ac7f0,74,c1093a04,0,...) at _sleep+0x34e/frame 0xf418ec70 vm_daemon(0,f418ed08,c107ef40,3d7,0,...) at vm_daemon+0x8c/frame 0xf418eccc fork_exit(c0d27900,0,f418ed08) at fork_exit+0x7f/frame 0xf418ecf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf418ecf4 --- trap 0, eip = 0, esp = 0xf418ed40, ebp = 0 --- Tracing command pagedaemon pid 6 tid 100069 td 0xc950e000 sched_switch(c950e000,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf418bb3c mi_switch(104,0,c108af51,1f7,2,...) at mi_switch+0x1c9/frame 0xf418bb70 sleepq_switch(c950e000,0,c108af51,290,c950e000,...) at sleepq_switch+0x1b9/frame 0xf418bba4 sleepq_timedwait(c1400684,54,c1093a04,0,0,...) at sleepq_timedwait+0x48/frame 0xf418bbc8 _sleep(c1400684,c1400600,54,c1093a04,1388,...) at _sleep+0x317/frame 0xf418bc20 vm_pageout(0,f418bd08,c107ef40,3d7,0,...) at vm_pageout+0x256/frame 0xf418bccc fork_exit(c0d28160,0,f418bd08) at fork_exit+0x7f/frame 0xf418bcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf418bcf4 --- trap 0, eip = 0, esp = 0xf418bd40, ebp = 0 --- Tracing command xpt_thrd pid 5 tid 100068 td 0xc950e2f0 sched_switch(c950e2f0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf4008bc4 mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xf4008bf8 sleepq_switch(c950e2f0,0,c108af51,26d,c950e2f0,...) at sleepq_switch+0x1b9/frame 0xf4008c2c sleepq_wait(c11fd8a4,5c,c0ed77b8,0,0,...) at sleepq_wait+0x48/frame 0xf4008c50 _sleep(c11fd8a4,c11fd8e8,5c,c0ed77b8,0,...) at _sleep+0x34e/frame 0xf4008ca8 xpt_scanner_thread(0,f4008d08,c107ef40,3d7,0,...) at xpt_scanner_thread+0xfc/frame 0xf4008ccc fork_exit(c04b30e0,0,f4008d08) at fork_exit+0x7f/frame 0xf4008cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4008cf4 --- trap 0, eip = 0, esp = 0xf4008d40, ebp = 0 --- Tracing command sctp_iterator pid 4 tid 100067 td 0xc950e5e0 sched_switch(c950e5e0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf4005bcc mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xf4005c00 sleepq_switch(c950e5e0,0,c108af51,26d,c950e5e0,...) at sleepq_switch+0x1b9/frame 0xf4005c34 sleepq_wait(c13fc374,0,c10aa425,0,0,...) at sleepq_wait+0x48/frame 0xf4005c58 _sleep(c13fc374,c13fc33c,0,c10aa425,0,...) at _sleep+0x34e/frame 0xf4005cb0 sctp_iterator_thread(0,f4005d08,c107ef40,3d7,0,...) at sctp_iterator_thread+0x5c/frame 0xf4005ccc fork_exit(c0bd7890,0,f4005d08) at fork_exit+0x7f/frame 0xf4005cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4005cf4 --- trap 0, eip = 0, esp = 0xf4005d40, ebp = 0 --- Tracing command ctl_thrd pid 3 tid 100064 td 0xc950f000 sched_switch(c950f000,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf3ffb64c mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xf3ffb680 sleepq_switch(c950f000,0,c108af51,26d,c950f000,...) at sleepq_switch+0x1b9/frame 0xf3ffb6b4 sleepq_wait(c80bc000,5c,c0ee23b3,0,0,...) at sleepq_wait+0x48/frame 0xf3ffb6d8 _sleep(c80bc000,c80bc000,5c,c0ee23b3,0,...) at _sleep+0x34e/frame 0xf3ffb730 ctl_work_thread(c80bc000,f3ffbd08,c107ef40,3d7,0,...) at ctl_work_thread+0x110/frame 0xf3ffbccc fork_exit(c04d5e60,c80bc000,f3ffbd08) at fork_exit+0x7f/frame 0xf3ffbcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3ffbcf4 --- trap 0, eip = 0, esp = 0xf3ffbd40, ebp = 0 --- Tracing command fdc0 pid 2 tid 100061 td 0xc806d8d0 sched_switch(c806d8d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf3ff2b50 mi_switch(104,0,c108af51,1f7,2,...) at mi_switch+0x1c9/frame 0xf3ff2b84 sleepq_switch(c806d8d0,0,c108af51,290,c806d8d0,...) at sleepq_switch+0x1b9/frame 0xf3ff2bb8 sleepq_timedwait(c801443c,5c,c10716a6,0,0,...) at sleepq_timedwait+0x48/frame 0xf3ff2bdc _sleep(c801443c,c80144f0,5c,c10716a6,3e8,...) at _sleep+0x317/frame 0xf3ff2c34 fdc_thread(c8014400,f3ff2d08,c107ef40,3d7,0,...) at fdc_thread+0x74a/frame 0xf3ff2ccc fork_exit(c0e38520,c8014400,f3ff2d08) at fork_exit+0x7f/frame 0xf3ff2cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3ff2cf4 --- trap 0, eip = 0, esp = 0xf3ff2d40, ebp = 0 --- Tracing command usb pid 15 tid 100058 td 0xc80782f0 sched_switch(c80782f0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf3fcebd4 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf3fcec08 sleepq_switch(c80782f0,0,c108af51,26d,c80782f0,...) at sleepq_switch+0x1b9/frame 0xf3fcec3c sleepq_wait(c8081d34,0,f3fcec90,1,0,...) at sleepq_wait+0x48/frame 0xf3fcec60 _cv_wait(c8081d34,c8081e4c,c1062e63,77,c8081d3c,...) at _cv_wait+0x1f9/frame 0xf3fceca4 usb_process(c8081d2c,f3fced08,c107ef40,3d7,0,...) at usb_process+0xe6/frame 0xf3fceccc fork_exit(c0948b90,c8081d2c,f3fced08) at fork_exit+0x7f/frame 0xf3fcecf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3fcecf4 --- trap 0, eip = 0, esp = 0xf3fced40, ebp = 0 --- Tracing command usb pid 15 tid 100057 td 0xc80785e0 sched_switch(c80785e0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf3fcbbd4 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf3fcbc08 sleepq_switch(c80785e0,0,c108af51,26d,c80785e0,...) at sleepq_switch+0x1b9/frame 0xf3fcbc3c sleepq_wait(c8081d04,0,f3fcbc90,1,0,...) at sleepq_wait+0x48/frame 0xf3fcbc60 _cv_wait(c8081d04,c8081e4c,c1062e63,77,c8081d0c,...) at _cv_wait+0x1f9/frame 0xf3fcbca4 usb_process(c8081cfc,f3fcbd08,c107ef40,3d7,0,...) at usb_process+0xe6/frame 0xf3fcbccc fork_exit(c0948b90,c8081cfc,f3fcbd08) at fork_exit+0x7f/frame 0xf3fcbcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3fcbcf4 --- trap 0, eip = 0, esp = 0xf3fcbd40, ebp = 0 --- Tracing command usb pid 15 tid 100056 td 0xc80788d0 sched_switch(c80788d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf3fc8bd4 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf3fc8c08 sleepq_switch(c80788d0,0,c108af51,26d,c80788d0,...) at sleepq_switch+0x1b9/frame 0xf3fc8c3c sleepq_wait(c8081cd4,0,f3fc8c90,1,0,...) at sleepq_wait+0x48/frame 0xf3fc8c60 _cv_wait(c8081cd4,c8081e4c,c1062e63,77,c8081cdc,...) at _cv_wait+0x1f9/frame 0xf3fc8ca4 usb_process(c8081ccc,f3fc8d08,c107ef40,3d7,0,...) at usb_process+0xe6/frame 0xf3fc8ccc fork_exit(c0948b90,c8081ccc,f3fc8d08) at fork_exit+0x7f/frame 0xf3fc8cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3fc8cf4 --- trap 0, eip = 0, esp = 0xf3fc8d40, ebp = 0 --- Tracing command usb pid 15 tid 100055 td 0xc8078bc0 sched_switch(c8078bc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf3fc5bd4 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf3fc5c08 sleepq_switch(c8078bc0,0,c108af51,26d,c8078bc0,...) at sleepq_switch+0x1b9/frame 0xf3fc5c3c sleepq_wait(c8081ca4,0,f3fc5c90,1,0,...) at sleepq_wait+0x48/frame 0xf3fc5c60 _cv_wait(c8081ca4,c8081e4c,c1062e63,77,c8081cac,...) at _cv_wait+0x1f9/frame 0xf3fc5ca4 usb_process(c8081c9c,f3fc5d08,c107ef40,3d7,0,...) at usb_process+0xe6/frame 0xf3fc5ccc fork_exit(c0948b90,c8081c9c,f3fc5d08) at fork_exit+0x7f/frame 0xf3fc5cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3fc5cf4 --- trap 0, eip = 0, esp = 0xf3fc5d40, ebp = 0 --- Tracing command usb pid 15 tid 100053 td 0xc807b2f0 sched_switch(c807b2f0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf3fbcbd4 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf3fbcc08 sleepq_switch(c807b2f0,0,c108af51,26d,c807b2f0,...) at sleepq_switch+0x1b9/frame 0xf3fbcc3c sleepq_wait(c8074b5c,0,f3fbcc90,1,0,...) at sleepq_wait+0x48/frame 0xf3fbcc60 _cv_wait(c8074b5c,c8074c74,c1062e63,77,c8074b64,...) at _cv_wait+0x1f9/frame 0xf3fbcca4 usb_process(c8074b54,f3fbcd08,c107ef40,3d7,0,...) at usb_process+0xe6/frame 0xf3fbcccc fork_exit(c0948b90,c8074b54,f3fbcd08) at fork_exit+0x7f/frame 0xf3fbccf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3fbccf4 --- trap 0, eip = 0, esp = 0xf3fbcd40, ebp = 0 --- Tracing command usb pid 15 tid 100052 td 0xc807b5e0 sched_switch(c807b5e0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf3fb9bd4 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf3fb9c08 sleepq_switch(c807b5e0,0,c108af51,26d,c807b5e0,...) at sleepq_switch+0x1b9/frame 0xf3fb9c3c sleepq_wait(c8074b2c,0,f3fb9c90,1,0,...) at sleepq_wait+0x48/frame 0xf3fb9c60 _cv_wait(c8074b2c,c8074c74,c1062e63,77,c8074b34,...) at _cv_wait+0x1f9/frame 0xf3fb9ca4 usb_process(c8074b24,f3fb9d08,c107ef40,3d7,0,...) at usb_process+0xe6/frame 0xf3fb9ccc fork_exit(c0948b90,c8074b24,f3fb9d08) at fork_exit+0x7f/frame 0xf3fb9cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3fb9cf4 --- trap 0, eip = 0, esp = 0xf3fb9d40, ebp = 0 --- Tracing command usb pid 15 tid 100051 td 0xc806b000 sched_switch(c806b000,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf3fb6bd4 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf3fb6c08 sleepq_switch(c806b000,0,c108af51,26d,c806b000,...) at sleepq_switch+0x1b9/frame 0xf3fb6c3c sleepq_wait(c8074afc,0,f3fb6c90,1,0,...) at sleepq_wait+0x48/frame 0xf3fb6c60 _cv_wait(c8074afc,c8074c74,c1062e63,77,c8074b04,...) at _cv_wait+0x1f9/frame 0xf3fb6ca4 usb_process(c8074af4,f3fb6d08,c107ef40,3d7,0,...) at usb_process+0xe6/frame 0xf3fb6ccc fork_exit(c0948b90,c8074af4,f3fb6d08) at fork_exit+0x7f/frame 0xf3fb6cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3fb6cf4 --- trap 0, eip = 0, esp = 0xf3fb6d40, ebp = 0 --- Tracing command usb pid 15 tid 100050 td 0xc806b2f0 sched_switch(c806b2f0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf3fb3bd4 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf3fb3c08 sleepq_switch(c806b2f0,0,c108af51,26d,c806b2f0,...) at sleepq_switch+0x1b9/frame 0xf3fb3c3c sleepq_wait(c8074acc,0,f3fb3c90,1,0,...) at sleepq_wait+0x48/frame 0xf3fb3c60 _cv_wait(c8074acc,c8074c74,c1062e63,77,c8074ad4,...) at _cv_wait+0x1f9/frame 0xf3fb3ca4 usb_process(c8074ac4,f3fb3d08,c107ef40,3d7,0,...) at usb_process+0xe6/frame 0xf3fb3ccc fork_exit(c0948b90,c8074ac4,f3fb3d08) at fork_exit+0x7f/frame 0xf3fb3cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3fb3cf4 --- trap 0, eip = 0, esp = 0xf3fb3d40, ebp = 0 --- Tracing command usb pid 15 tid 100049 td 0xc806b5e0 sched_switch(c806b5e0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf3fafbd4 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf3fafc08 sleepq_switch(c806b5e0,0,c108af51,26d,c806b5e0,...) at sleepq_switch+0x1b9/frame 0xf3fafc3c sleepq_wait(c806eb5c,0,f3fafc90,1,0,...) at sleepq_wait+0x48/frame 0xf3fafc60 _cv_wait(c806eb5c,c806ec74,c1062e63,77,c806eb64,...) at _cv_wait+0x1f9/frame 0xf3fafca4 usb_process(c806eb54,f3fafd08,c107ef40,3d7,0,...) at usb_process+0xe6/frame 0xf3fafccc fork_exit(c0948b90,c806eb54,f3fafd08) at fork_exit+0x7f/frame 0xf3fafcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3fafcf4 --- trap 0, eip = 0, esp = 0xf3fafd40, ebp = 0 --- Tracing command usb pid 15 tid 100048 td 0xc806b8d0 sched_switch(c806b8d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf3facbd4 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf3facc08 sleepq_switch(c806b8d0,0,c108af51,26d,c806b8d0,...) at sleepq_switch+0x1b9/frame 0xf3facc3c sleepq_wait(c806eb2c,0,f3facc90,1,0,...) at sleepq_wait+0x48/frame 0xf3facc60 _cv_wait(c806eb2c,c806ec74,c1062e63,77,c806eb34,...) at _cv_wait+0x1f9/frame 0xf3facca4 usb_process(c806eb24,f3facd08,c107ef40,3d7,0,...) at usb_process+0xe6/frame 0xf3facccc fork_exit(c0948b90,c806eb24,f3facd08) at fork_exit+0x7f/frame 0xf3faccf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3faccf4 --- trap 0, eip = 0, esp = 0xf3facd40, ebp = 0 --- Tracing command usb pid 15 tid 100047 td 0xc806bbc0 sched_switch(c806bbc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf3fa9bd4 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf3fa9c08 sleepq_switch(c806bbc0,0,c108af51,26d,c806bbc0,...) at sleepq_switch+0x1b9/frame 0xf3fa9c3c sleepq_wait(c806eafc,0,f3fa9c90,1,0,...) at sleepq_wait+0x48/frame 0xf3fa9c60 _cv_wait(c806eafc,c806ec74,c1062e63,77,c806eb04,...) at _cv_wait+0x1f9/frame 0xf3fa9ca4 usb_process(c806eaf4,f3fa9d08,c107ef40,3d7,0,...) at usb_process+0xe6/frame 0xf3fa9ccc fork_exit(c0948b90,c806eaf4,f3fa9d08) at fork_exit+0x7f/frame 0xf3fa9cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3fa9cf4 --- trap 0, eip = 0, esp = 0xf3fa9d40, ebp = 0 --- Tracing command usb pid 15 tid 100046 td 0xc806c000 sched_switch(c806c000,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf3fa6bd4 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf3fa6c08 sleepq_switch(c806c000,0,c108af51,26d,c806c000,...) at sleepq_switch+0x1b9/frame 0xf3fa6c3c sleepq_wait(c806eacc,0,f3fa6c90,1,0,...) at sleepq_wait+0x48/frame 0xf3fa6c60 _cv_wait(c806eacc,c806ec74,c1062e63,77,c806ead4,...) at _cv_wait+0x1f9/frame 0xf3fa6ca4 usb_process(c806eac4,f3fa6d08,c107ef40,3d7,0,...) at usb_process+0xe6/frame 0xf3fa6ccc fork_exit(c0948b90,c806eac4,f3fa6d08) at fork_exit+0x7f/frame 0xf3fa6cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3fa6cf4 --- trap 0, eip = 0, esp = 0xf3fa6d40, ebp = 0 --- Tracing command usb pid 15 tid 100045 td 0xc806c2f0 sched_switch(c806c2f0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf3fa2bd4 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf3fa2c08 sleepq_switch(c806c2f0,0,c108af51,26d,c806c2f0,...) at sleepq_switch+0x1b9/frame 0xf3fa2c3c sleepq_wait(c8067b5c,0,f3fa2c90,1,0,...) at sleepq_wait+0x48/frame 0xf3fa2c60 _cv_wait(c8067b5c,c8067c74,c1062e63,77,c8067b64,...) at _cv_wait+0x1f9/frame 0xf3fa2ca4 usb_process(c8067b54,f3fa2d08,c107ef40,3d7,0,...) at usb_process+0xe6/frame 0xf3fa2ccc fork_exit(c0948b90,c8067b54,f3fa2d08) at fork_exit+0x7f/frame 0xf3fa2cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3fa2cf4 --- trap 0, eip = 0, esp = 0xf3fa2d40, ebp = 0 --- Tracing command usb pid 15 tid 100044 td 0xc806c5e0 sched_switch(c806c5e0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf3f9fbd4 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf3f9fc08 sleepq_switch(c806c5e0,0,c108af51,26d,c806c5e0,...) at sleepq_switch+0x1b9/frame 0xf3f9fc3c sleepq_wait(c8067b2c,0,f3f9fc90,1,0,...) at sleepq_wait+0x48/frame 0xf3f9fc60 _cv_wait(c8067b2c,c8067c74,c1062e63,77,c8067b34,...) at _cv_wait+0x1f9/frame 0xf3f9fca4 usb_process(c8067b24,f3f9fd08,c107ef40,3d7,0,...) at usb_process+0xe6/frame 0xf3f9fccc fork_exit(c0948b90,c8067b24,f3f9fd08) at fork_exit+0x7f/frame 0xf3f9fcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3f9fcf4 --- trap 0, eip = 0, esp = 0xf3f9fd40, ebp = 0 --- Tracing command usb pid 15 tid 100043 td 0xc806c8d0 sched_switch(c806c8d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf3f9cbd4 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf3f9cc08 sleepq_switch(c806c8d0,0,c108af51,26d,c806c8d0,...) at sleepq_switch+0x1b9/frame 0xf3f9cc3c sleepq_wait(c8067afc,0,f3f9cc90,1,0,...) at sleepq_wait+0x48/frame 0xf3f9cc60 _cv_wait(c8067afc,c8067c74,c1062e63,77,c8067b04,...) at _cv_wait+0x1f9/frame 0xf3f9cca4 usb_process(c8067af4,f3f9cd08,c107ef40,3d7,0,...) at usb_process+0xe6/frame 0xf3f9cccc fork_exit(c0948b90,c8067af4,f3f9cd08) at fork_exit+0x7f/frame 0xf3f9ccf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3f9ccf4 --- trap 0, eip = 0, esp = 0xf3f9cd40, ebp = 0 --- Tracing command usb pid 15 tid 100042 td 0xc806cbc0 sched_switch(c806cbc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf3f99bd4 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf3f99c08 sleepq_switch(c806cbc0,0,c108af51,26d,c806cbc0,...) at sleepq_switch+0x1b9/frame 0xf3f99c3c sleepq_wait(c8067acc,0,f3f99c90,1,0,...) at sleepq_wait+0x48/frame 0xf3f99c60 _cv_wait(c8067acc,c8067c74,c1062e63,77,c8067ad4,...) at _cv_wait+0x1f9/frame 0xf3f99ca4 usb_process(c8067ac4,f3f99d08,c107ef40,3d7,0,...) at usb_process+0xe6/frame 0xf3f99ccc fork_exit(c0948b90,c8067ac4,f3f99d08) at fork_exit+0x7f/frame 0xf3f99cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3f99cf4 --- trap 0, eip = 0, esp = 0xf3f99d40, ebp = 0 --- Tracing command usb pid 15 tid 100040 td 0xc805b000 sched_switch(c805b000,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf3f92bd4 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf3f92c08 sleepq_switch(c805b000,0,c108af51,26d,c805b000,...) at sleepq_switch+0x1b9/frame 0xf3f92c3c sleepq_wait(c8062b5c,0,f3f92c90,1,0,...) at sleepq_wait+0x48/frame 0xf3f92c60 _cv_wait(c8062b5c,c8062c74,c1062e63,77,c8062b64,...) at _cv_wait+0x1f9/frame 0xf3f92ca4 usb_process(c8062b54,f3f92d08,c107ef40,3d7,0,...) at usb_process+0xe6/frame 0xf3f92ccc fork_exit(c0948b90,c8062b54,f3f92d08) at fork_exit+0x7f/frame 0xf3f92cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3f92cf4 --- trap 0, eip = 0, esp = 0xf3f92d40, ebp = 0 --- Tracing command usb pid 15 tid 100039 td 0xc805b2f0 sched_switch(c805b2f0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf3f8fbd4 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf3f8fc08 sleepq_switch(c805b2f0,0,c108af51,26d,c805b2f0,...) at sleepq_switch+0x1b9/frame 0xf3f8fc3c sleepq_wait(c8062b2c,0,f3f8fc90,1,0,...) at sleepq_wait+0x48/frame 0xf3f8fc60 _cv_wait(c8062b2c,c8062c74,c1062e63,77,c8062b34,...) at _cv_wait+0x1f9/frame 0xf3f8fca4 usb_process(c8062b24,f3f8fd08,c107ef40,3d7,0,...) at usb_process+0xe6/frame 0xf3f8fccc fork_exit(c0948b90,c8062b24,f3f8fd08) at fork_exit+0x7f/frame 0xf3f8fcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3f8fcf4 --- trap 0, eip = 0, esp = 0xf3f8fd40, ebp = 0 --- Tracing command usb pid 15 tid 100038 td 0xc805b5e0 sched_switch(c805b5e0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf3f8cbd4 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf3f8cc08 sleepq_switch(c805b5e0,0,c108af51,26d,c805b5e0,...) at sleepq_switch+0x1b9/frame 0xf3f8cc3c sleepq_wait(c8062afc,0,f3f8cc90,1,0,...) at sleepq_wait+0x48/frame 0xf3f8cc60 _cv_wait(c8062afc,c8062c74,c1062e63,77,c8062b04,...) at _cv_wait+0x1f9/frame 0xf3f8cca4 usb_process(c8062af4,f3f8cd08,c107ef40,3d7,0,...) at usb_process+0xe6/frame 0xf3f8cccc fork_exit(c0948b90,c8062af4,f3f8cd08) at fork_exit+0x7f/frame 0xf3f8ccf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3f8ccf4 --- trap 0, eip = 0, esp = 0xf3f8cd40, ebp = 0 --- Tracing command usb pid 15 tid 100037 td 0xc805b8d0 sched_switch(c805b8d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf3f89bd4 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf3f89c08 sleepq_switch(c805b8d0,0,c108af51,26d,c805b8d0,...) at sleepq_switch+0x1b9/frame 0xf3f89c3c sleepq_wait(c8062acc,0,f3f89c90,1,0,...) at sleepq_wait+0x48/frame 0xf3f89c60 _cv_wait(c8062acc,c8062c74,c1062e63,77,c8062ad4,...) at _cv_wait+0x1f9/frame 0xf3f89ca4 usb_process(c8062ac4,f3f89d08,c107ef40,3d7,0,...) at usb_process+0xe6/frame 0xf3f89ccc fork_exit(c0948b90,c8062ac4,f3f89d08) at fork_exit+0x7f/frame 0xf3f89cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3f89cf4 --- trap 0, eip = 0, esp = 0xf3f89d40, ebp = 0 --- Tracing command usb pid 15 tid 100035 td 0xc805e000 sched_switch(c805e000,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf3f82bd4 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf3f82c08 sleepq_switch(c805e000,0,c108af51,26d,c805e000,...) at sleepq_switch+0x1b9/frame 0xf3f82c3c sleepq_wait(c8056b5c,0,f3f82c90,1,0,...) at sleepq_wait+0x48/frame 0xf3f82c60 _cv_wait(c8056b5c,c8056c74,c1062e63,77,c8056b64,...) at _cv_wait+0x1f9/frame 0xf3f82ca4 usb_process(c8056b54,f3f82d08,c107ef40,3d7,0,...) at usb_process+0xe6/frame 0xf3f82ccc fork_exit(c0948b90,c8056b54,f3f82d08) at fork_exit+0x7f/frame 0xf3f82cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3f82cf4 --- trap 0, eip = 0, esp = 0xf3f82d40, ebp = 0 --- Tracing command usb pid 15 tid 100034 td 0xc805e2f0 sched_switch(c805e2f0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf3f7fbd4 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf3f7fc08 sleepq_switch(c805e2f0,0,c108af51,26d,c805e2f0,...) at sleepq_switch+0x1b9/frame 0xf3f7fc3c sleepq_wait(c8056b2c,0,f3f7fc90,1,0,...) at sleepq_wait+0x48/frame 0xf3f7fc60 _cv_wait(c8056b2c,c8056c74,c1062e63,77,c8056b34,...) at _cv_wait+0x1f9/frame 0xf3f7fca4 usb_process(c8056b24,f3f7fd08,c107ef40,3d7,0,...) at usb_process+0xe6/frame 0xf3f7fccc fork_exit(c0948b90,c8056b24,f3f7fd08) at fork_exit+0x7f/frame 0xf3f7fcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3f7fcf4 --- trap 0, eip = 0, esp = 0xf3f7fd40, ebp = 0 --- Tracing command usb pid 15 tid 100033 td 0xc805e5e0 sched_switch(c805e5e0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf3f7cbd4 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf3f7cc08 sleepq_switch(c805e5e0,0,c108af51,26d,c805e5e0,...) at sleepq_switch+0x1b9/frame 0xf3f7cc3c sleepq_wait(c8056afc,0,f3f7cc90,1,0,...) at sleepq_wait+0x48/frame 0xf3f7cc60 _cv_wait(c8056afc,c8056c74,c1062e63,77,c8056b04,...) at _cv_wait+0x1f9/frame 0xf3f7cca4 usb_process(c8056af4,f3f7cd08,c107ef40,3d7,0,...) at usb_process+0xe6/frame 0xf3f7cccc fork_exit(c0948b90,c8056af4,f3f7cd08) at fork_exit+0x7f/frame 0xf3f7ccf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3f7ccf4 --- trap 0, eip = 0, esp = 0xf3f7cd40, ebp = 0 --- Tracing command usb pid 15 tid 100032 td 0xc805e8d0 sched_switch(c805e8d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf3f79bd4 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xf3f79c08 sleepq_switch(c805e8d0,0,c108af51,26d,c805e8d0,...) at sleepq_switch+0x1b9/frame 0xf3f79c3c sleepq_wait(c8056acc,0,f3f79c90,1,0,...) at sleepq_wait+0x48/frame 0xf3f79c60 _cv_wait(c8056acc,c8056c74,c1062e63,77,c8056ad4,...) at _cv_wait+0x1f9/frame 0xf3f79ca4 usb_process(c8056ac4,f3f79d08,c107ef40,3d7,0,...) at usb_process+0xe6/frame 0xf3f79ccc fork_exit(c0948b90,c8056ac4,f3f79d08) at fork_exit+0x7f/frame 0xf3f79cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3f79cf4 --- trap 0, eip = 0, esp = 0xf3f79d40, ebp = 0 --- Tracing command yarrow pid 14 tid 100017 td 0xc7ef05e0 sched_switch(c7ef05e0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xc7b71b90 mi_switch(104,0,c108af51,1f7,3,...) at mi_switch+0x1c9/frame 0xc7b71bc4 sleepq_switch(c7ef05e0,0,c108af51,290,c7ef05e0,...) at sleepq_switch+0x1b9/frame 0xc7b71bf8 sleepq_timedwait(c1230da0,0,c10716a6,2,0,...) at sleepq_timedwait+0x48/frame 0xc7b71c1c _sleep(c1230da0,0,0,c10716a6,64) at _sleep+0x317/frame 0xc7b71c74 pause(c10716a6,64,c103e81e,111,0,...) at pause+0x85/frame 0xc7b71c98 random_kthread(0,c7b71d08,c107ef40,3d7,0,...) at random_kthread+0x30f/frame 0xc7b71ccc fork_exit(c0838ce0,0,c7b71d08) at fork_exit+0x7f/frame 0xc7b71cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b71cf4 --- trap 0, eip = 0, esp = 0xc7b71d40, ebp = 0 --- Tracing command geom pid 13 tid 100016 td 0xc7ef08d0 sched_switch(c7ef08d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xc7b6db78 mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xc7b6dbac sleepq_switch(c7ef08d0,0,c108af51,26d,c7ef08d0,...) at sleepq_switch+0x1b9/frame 0xc7b6dbe0 sleepq_wait(c13f84a0,5c,c10716a6,0,0,...) at sleepq_wait+0x48/frame 0xc7b6dc04 _sleep(c13f84a0,c122f0e8,25c,c10716a6,0,...) at _sleep+0x34e/frame 0xc7b6dc5c g_io_schedule_down(c7ef08d0,0,c10731e5,6c,c7b6dcf4,...) at g_io_schedule_down+0x4c/frame 0xc7b6dcb4 g_down_procbody(0,c7b6dd08,c107ef40,3d7,0,...) at g_down_procbody+0x9d/frame 0xc7b6dccc fork_exit(c09f5930,0,c7b6dd08) at fork_exit+0x7f/frame 0xc7b6dcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b6dcf4 --- trap 0, eip = 0, esp = 0xc7b6dd40, ebp = 0 --- Tracing command geom pid 13 tid 100015 td 0xc7ef0bc0 sched_switch(c7ef0bc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xc7b6abac mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xc7b6abe0 sleepq_switch(c7ef0bc0,0,c108af51,26d,c7ef0bc0,...) at sleepq_switch+0x1b9/frame 0xc7b6ac14 sleepq_wait(c13f849c,5c,c10716a6,0,0,...) at sleepq_wait+0x48/frame 0xc7b6ac38 _sleep(c13f849c,c122f108,25c,c10716a6,0,...) at _sleep+0x34e/frame 0xc7b6ac90 g_io_schedule_up(c7ef0bc0,0,c10731e5,5f,c7b6acf4,...) at g_io_schedule_up+0x8f/frame 0xc7b6acb4 g_up_procbody(0,c7b6ad08,c107ef40,3d7,0,...) at g_up_procbody+0x9d/frame 0xc7b6accc fork_exit(c09f5890,0,c7b6ad08) at fork_exit+0x7f/frame 0xc7b6acf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b6acf4 --- trap 0, eip = 0, esp = 0xc7b6ad40, ebp = 0 --- Tracing command geom pid 13 tid 100014 td 0xc7ef1000 sched_switch(c7ef1000,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xc7b67bbc mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xc7b67bf0 sleepq_switch(c7ef1000,0,c108af51,26d,c7ef1000,...) at sleepq_switch+0x1b9/frame 0xc7b67c24 sleepq_wait(c13f8498,5c,c10716a6,0,0,...) at sleepq_wait+0x48/frame 0xc7b67c48 _sleep(c13f8498,c122f0c0,25c,c10716a6,0,...) at _sleep+0x34e/frame 0xc7b67ca0 g_run_events(0,c7b67d08,c107ef40,3d7,0,...) at g_run_events+0x4f3/frame 0xc7b67ccc fork_exit(c09f57f0,0,c7b67d08) at fork_exit+0x7f/frame 0xc7b67cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b67cf4 --- trap 0, eip = 0, esp = 0xc7b67d40, ebp = 0 --- Tracing command intr pid 12 tid 100063 td 0xc806d2f0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100062 td 0xc806d5e0 sched_switch(c806d5e0,0,109,1b3,0,...) at sched_switch+0x4b4/frame 0xf3ff5c64 mi_switch(109,0,c107f1dd,563,c806d5e0,...) at mi_switch+0x1c9/frame 0xf3ff5c98 ithread_loop(c7e2a670,f3ff5d08,c107ef40,3d7,0,...) at ithread_loop+0x100/frame 0xf3ff5ccc fork_exit(c0a66560,c7e2a670,f3ff5d08) at fork_exit+0x7f/frame 0xf3ff5cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3ff5cf4 --- trap 0, eip = 0, esp = 0xf3ff5d40, ebp = 0 --- Tracing command intr pid 12 tid 100060 td 0xc806dbc0 sched_switch(c806dbc0,0,109,1b3,0,...) at sched_switch+0x4b4/frame 0xf3fe5c64 mi_switch(109,0,c107f1dd,563,c806dbc0,...) at mi_switch+0x1c9/frame 0xf3fe5c98 ithread_loop(c7e2a680,f3fe5d08,c107ef40,3d7,0,...) at ithread_loop+0x100/frame 0xf3fe5ccc fork_exit(c0a66560,c7e2a680,f3fe5d08) at fork_exit+0x7f/frame 0xf3fe5cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3fe5cf4 --- trap 0, eip = 0, esp = 0xf3fe5d40, ebp = 0 --- Tracing command intr pid 12 tid 100059 td 0xc8078000 sched_switch(c8078000,0,109,1b3,0,...) at sched_switch+0x4b4/frame 0xf3fd4c64 mi_switch(109,0,c107f1dd,563,c8078000,...) at mi_switch+0x1c9/frame 0xf3fd4c98 ithread_loop(c7e2a690,f3fd4d08,c107ef40,3d7,0,...) at ithread_loop+0x100/frame 0xf3fd4ccc fork_exit(c0a66560,c7e2a690,f3fd4d08) at fork_exit+0x7f/frame 0xf3fd4cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3fd4cf4 --- trap 0, eip = 0, esp = 0xf3fd4d40, ebp = 0 --- Tracing command intr pid 12 tid 100054 td 0xc807b000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100041 td 0xc806d000 sched_switch(c806d000,0,109,1b3,0,...) at sched_switch+0x4b4/frame 0xf3f96c64 mi_switch(109,0,c107f1dd,563,c806d000,...) at mi_switch+0x1c9/frame 0xf3f96c98 ithread_loop(c7e2a6b0,f3f96d08,c107ef40,3d7,0,...) at ithread_loop+0x100/frame 0xf3f96ccc fork_exit(c0a66560,c7e2a6b0,f3f96d08) at fork_exit+0x7f/frame 0xf3f96cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3f96cf4 --- trap 0, eip = 0, esp = 0xf3f96d40, ebp = 0 --- Tracing command intr pid 12 tid 100036 td 0xc805bbc0 sched_switch(c805bbc0,0,109,1b3,0,...) at sched_switch+0x4b4/frame 0xf3f86c64 mi_switch(109,0,c107f1dd,563,c805bbc0,...) at mi_switch+0x1c9/frame 0xf3f86c98 ithread_loop(c7e2a6c0,f3f86d08,c107ef40,3d7,0,...) at ithread_loop+0x100/frame 0xf3f86ccc fork_exit(c0a66560,c7e2a6c0,f3f86d08) at fork_exit+0x7f/frame 0xf3f86cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3f86cf4 --- trap 0, eip = 0, esp = 0xf3f86d40, ebp = 0 --- Tracing command intr pid 12 tid 100031 td 0xc805ebc0 sched_switch(c805ebc0,0,109,1b3,0,...) at sched_switch+0x4b4/frame 0xf3f76c64 mi_switch(109,0,c107f1dd,563,c805ebc0,...) at mi_switch+0x1c9/frame 0xf3f76c98 ithread_loop(c7e2a6d0,f3f76d08,c107ef40,3d7,0,...) at ithread_loop+0x100/frame 0xf3f76ccc fork_exit(c0a66560,c7e2a6d0,f3f76d08) at fork_exit+0x7f/frame 0xf3f76cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3f76cf4 --- trap 0, eip = 0, esp = 0xf3f76d40, ebp = 0 --- Tracing command intr pid 12 tid 100030 td 0xc7ef12f0 sched_switch(c7ef12f0,0,109,1b3,0,...) at sched_switch+0x4b4/frame 0xf3e20c64 mi_switch(109,0,c107f1dd,563,c7ef12f0,...) at mi_switch+0x1c9/frame 0xf3e20c98 ithread_loop(c7e2a6e0,f3e20d08,c107ef40,3d7,0,...) at ithread_loop+0x100/frame 0xf3e20ccc fork_exit(c0a66560,c7e2a6e0,f3e20d08) at fork_exit+0x7f/frame 0xf3e20cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3e20cf4 --- trap 0, eip = 0, esp = 0xf3e20d40, ebp = 0 --- Tracing command intr pid 12 tid 100029 td 0xc7ef15e0 sched_switch(c7ef15e0,0,109,1b3,0,...) at sched_switch+0x4b4/frame 0xf3e1cc64 mi_switch(109,0,c107f1dd,563,c7ef15e0,...) at mi_switch+0x1c9/frame 0xf3e1cc98 ithread_loop(c7e2a6f0,f3e1cd08,c107ef40,3d7,0,...) at ithread_loop+0x100/frame 0xf3e1cccc fork_exit(c0a66560,c7e2a6f0,f3e1cd08) at fork_exit+0x7f/frame 0xf3e1ccf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3e1ccf4 --- trap 0, eip = 0, esp = 0xf3e1cd40, ebp = 0 --- Tracing command intr pid 12 tid 100028 td 0xc7ef18d0 sched_switch(c7ef18d0,0,109,1b3,0,...) at sched_switch+0x4b4/frame 0xc7bc1c64 mi_switch(109,0,c107f1dd,563,c7ef18d0,...) at mi_switch+0x1c9/frame 0xc7bc1c98 ithread_loop(c7e2a740,c7bc1d08,c107ef40,3d7,0,...) at ithread_loop+0x100/frame 0xc7bc1ccc fork_exit(c0a66560,c7e2a740,c7bc1d08) at fork_exit+0x7f/frame 0xc7bc1cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7bc1cf4 --- trap 0, eip = 0, esp = 0xc7bc1d40, ebp = 0 --- Tracing command intr pid 12 tid 100026 td 0xc7fdd000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100024 td 0xc7fdd5e0 sched_switch(c7fdd5e0,0,109,1b3,0,...) at sched_switch+0x4b4/frame 0xc7b86c64 mi_switch(109,0,c107f1dd,563,c7fdd5e0,...) at mi_switch+0x1c9/frame 0xc7b86c98 ithread_loop(c7e2a770,c7b86d08,c107ef40,3d7,0,...) at ithread_loop+0x100/frame 0xc7b86ccc fork_exit(c0a66560,c7e2a770,c7b86d08) at fork_exit+0x7f/frame 0xc7b86cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b86cf4 --- trap 0, eip = 0, esp = 0xc7b86d40, ebp = 0 --- Tracing command intr pid 12 tid 100023 td 0xc7fdd8d0 sched_switch(c7fdd8d0,0,109,1b3,0,...) at sched_switch+0x4b4/frame 0xc7b83c64 mi_switch(109,0,c107f1dd,563,c7fdd8d0,...) at mi_switch+0x1c9/frame 0xc7b83c98 ithread_loop(c7e2a780,c7b83d08,c107ef40,3d7,0,...) at ithread_loop+0x100/frame 0xc7b83ccc fork_exit(c0a66560,c7e2a780,c7b83d08) at fork_exit+0x7f/frame 0xc7b83cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b83cf4 --- trap 0, eip = 0, esp = 0xc7b83d40, ebp = 0 --- Tracing command intr pid 12 tid 100018 td 0xc7ef02f0 sched_switch(c7ef02f0,0,109,1b3,0,...) at sched_switch+0x4b4/frame 0xc7b74c64 mi_switch(109,0,c107f1dd,563,c7ef02f0,...) at mi_switch+0x1c9/frame 0xc7b74c98 ithread_loop(c7e2a790,c7b74d08,c107ef40,3d7,0,...) at ithread_loop+0x100/frame 0xc7b74ccc fork_exit(c0a66560,c7e2a790,c7b74d08) at fork_exit+0x7f/frame 0xc7b74cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b74cf4 --- trap 0, eip = 0, esp = 0xc7b74d40, ebp = 0 --- Tracing command intr pid 12 tid 100012 td 0xc7de35e0 sched_switch(c7de35e0,0,109,1b3,0,...) at sched_switch+0x4b4/frame 0xc7b61c64 mi_switch(109,0,c107f1dd,563,c7de35e0,...) at mi_switch+0x1c9/frame 0xc7b61c98 ithread_loop(c7e2a7a0,c7b61d08,c107ef40,3d7,0,...) at ithread_loop+0x100/frame 0xc7b61ccc fork_exit(c0a66560,c7e2a7a0,c7b61d08) at fork_exit+0x7f/frame 0xc7b61cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b61cf4 --- trap 0, eip = 0, esp = 0xc7b61d40, ebp = 0 --- Tracing command intr pid 12 tid 100011 td 0xc7de38d0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100010 td 0xc7de3bc0 sched_switch(c7de3bc0,0,109,1b3,c107f1dd,...) at sched_switch+0x4b4/frame 0xc7b5bc64 mi_switch(109,0,c107f1dd,563,c7de3bc0,...) at mi_switch+0x1c9/frame 0xc7b5bc98 ithread_loop(c7e2a7c0,c7b5bd08,c107ef40,3d7,0,...) at ithread_loop+0x100/frame 0xc7b5bccc fork_exit(c0a66560,c7e2a7c0,c7b5bd08) at fork_exit+0x7f/frame 0xc7b5bcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b5bcf4 --- trap 0, eip = 0, esp = 0xc7b5bd40, ebp = 0 --- Tracing command intr pid 12 tid 100009 td 0xc7e2b000 sched_switch(c7e2b000,0,109,1b3,c107f1dd,...) at sched_switch+0x4b4/frame 0xc7b58c64 mi_switch(109,0,c107f1dd,563,c7e2b000,...) at mi_switch+0x1c9/frame 0xc7b58c98 ithread_loop(c7e2a7d0,c7b58d08,c107ef40,3d7,0,...) at ithread_loop+0x100/frame 0xc7b58ccc fork_exit(c0a66560,c7e2a7d0,c7b58d08) at fork_exit+0x7f/frame 0xc7b58cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b58cf4 --- trap 0, eip = 0, esp = 0xc7b58d40, ebp = 0 --- Tracing command intr pid 12 tid 100008 td 0xc7e2b2f0 sched_switch(c7e2b2f0,0,109,1b3,c107f1dd,...) at sched_switch+0x4b4/frame 0xc7b55c64 mi_switch(109,0,c107f1dd,563,c7e2b2f0,...) at mi_switch+0x1c9/frame 0xc7b55c98 ithread_loop(c7e2a7e0,c7b55d08,c107ef40,3d7,0,...) at ithread_loop+0x100/frame 0xc7b55ccc fork_exit(c0a66560,c7e2a7e0,c7b55d08) at fork_exit+0x7f/frame 0xc7b55cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b55cf4 --- trap 0, eip = 0, esp = 0xc7b55d40, ebp = 0 --- Tracing command intr pid 12 tid 100007 td 0xc7e2b5e0 sched_switch(c7e2b5e0,0,109,1b3,0,...) at sched_switch+0x4b4/frame 0xc7b52c64 mi_switch(109,0,c107f1dd,563,c7e2b5e0,...) at mi_switch+0x1c9/frame 0xc7b52c98 ithread_loop(c7e2a7f0,c7b52d08,c107ef40,3d7,0,...) at ithread_loop+0x100/frame 0xc7b52ccc fork_exit(c0a66560,c7e2a7f0,c7b52d08) at fork_exit+0x7f/frame 0xc7b52cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b52cf4 --- trap 0, eip = 0, esp = 0xc7b52d40, ebp = 0 --- Tracing command idle pid 11 tid 100006 td 0xc7dde000 sched_switch(c7dde000,0,108,1ae,c1087885,...) at sched_switch+0x4b4/frame 0xc7b4ec1c mi_switch(108,0,c1087885,407,c123f608,...) at mi_switch+0x1c9/frame 0xc7b4ec50 sched_idletd(0,c7b4ed08,c107ef40,3d7,0,...) at sched_idletd+0x45b/frame 0xc7b4eccc fork_exit(c0abb3e0,0,c7b4ed08) at fork_exit+0x7f/frame 0xc7b4ecf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b4ecf4 --- trap 0, eip = 0, esp = 0xc7b4ed40, ebp = 0 --- Tracing command idle pid 11 tid 100005 td 0xc7dde2f0 sched_switch(c7dde2f0,0,108,1ae,c1087885,...) at sched_switch+0x4b4/frame 0xc7b4bc1c mi_switch(108,0,c1087885,407,c123ef08,...) at mi_switch+0x1c9/frame 0xc7b4bc50 sched_idletd(0,c7b4bd08,c107ef40,3d7,0,...) at sched_idletd+0x45b/frame 0xc7b4bccc fork_exit(c0abb3e0,0,c7b4bd08) at fork_exit+0x7f/frame 0xc7b4bcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b4bcf4 --- trap 0, eip = 0, esp = 0xc7b4bd40, ebp = 0 --- Tracing command idle pid 11 tid 100004 td 0xc7dde5e0 sched_switch(c7dde5e0,0,108,1ae,c1087885,...) at sched_switch+0x4b4/frame 0xc7b48c1c mi_switch(108,0,c1087885,407,c123e808,...) at mi_switch+0x1c9/frame 0xc7b48c50 sched_idletd(0,c7b48d08,c107ef40,3d7,0,...) at sched_idletd+0x45b/frame 0xc7b48ccc fork_exit(c0abb3e0,0,c7b48d08) at fork_exit+0x7f/frame 0xc7b48cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b48cf4 --- trap 0, eip = 0, esp = 0xc7b48d40, ebp = 0 --- Tracing command idle pid 11 tid 100003 td 0xc7dde8d0 sched_switch(c7dde8d0,0,108,1ae,c1087885,...) at sched_switch+0x4b4/frame 0xc7b45c1c mi_switch(108,0,c1087885,407,c123e108,...) at mi_switch+0x1c9/frame 0xc7b45c50 sched_idletd(0,c7b45d08,c107ef40,3d7,0,...) at sched_idletd+0x45b/frame 0xc7b45ccc fork_exit(c0abb3e0,0,c7b45d08) at fork_exit+0x7f/frame 0xc7b45cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b45cf4 --- trap 0, eip = 0, esp = 0xc7b45d40, ebp = 0 --- Tracing command init pid 1 tid 100002 td 0xc7ddebc0 sched_switch(c7ddebc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xc7b41998 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xc7b419cc sleepq_switch(c7ddebc0,0,c108af51,1ab,c7b41a44,...) at sleepq_switch+0x1b9/frame 0xc7b41a00 sleepq_catch_signals(0,c108af51,164,708,c7ddebc0,...) at sleepq_catch_signals+0x3b1/frame 0xc7b41a48 sleepq_wait_sig(c7ddcb40,6c,c108e435,100,0,...) at sleepq_wait_sig+0x14/frame 0xc7b41a6c _sleep(c7ddcb40,c7ddcbc8,16c,c108e435,0,...) at _sleep+0x32d/frame 0xc7b41ac4 kern_wait6(c7ddebc0,7,0,0,c7b41b4c,...) at kern_wait6+0x43c/frame 0xc7b41b20 sys_wait4(c7ddebc0,c7b41cc8,7b3,0,c7b41cc4,...) at sys_wait4+0x94/frame 0xc7b41c40 syscall(c7b41d08) at syscall+0x2ed/frame 0xc7b41cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xc7b41cfc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x805dcf7, esp = 0xbfbfd8f4, ebp = 0xbfbfd908 --- Tracing command audit pid 10 tid 100001 td 0xc7de3000 sched_switch(c7de3000,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xc7b3eba8 mi_switch(104,0,c108af51,1f7,8,...) at mi_switch+0x1c9/frame 0xc7b3ebdc sleepq_switch(c7de3000,0,c108af51,26d,c7de3000,...) at sleepq_switch+0x1b9/frame 0xc7b3ec10 sleepq_wait(c13ff13c,0,c7b3ec64,1,0,...) at sleepq_wait+0x48/frame 0xc7b3ec34 _cv_wait(c13ff13c,c13ff0c8,c10b6642,18d,0,...) at _cv_wait+0x1f9/frame 0xc7b3ec78 audit_worker(0,c7b3ed08,c107ef40,3d7,0,...) at audit_worker+0x94/frame 0xc7b3eccc fork_exit(c0c9de80,0,c7b3ed08) at fork_exit+0x7f/frame 0xc7b3ecf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b3ecf4 --- trap 0, eip = 0, esp = 0xc7b3ed40, ebp = 0 --- Tracing command kernel pid 0 tid 100574 td 0xd17bf8d0 sched_switch(d17bf8d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9e2dbc0 mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xf9e2dbf4 sleepq_switch(d17bf8d0,0,c108af51,26d,d17bf8d0,...) at sleepq_switch+0x1b9/frame 0xf9e2dc28 sleepq_wait(cdc42580,0,c10716a6,0,0,...) at sleepq_wait+0x48/frame 0xf9e2dc4c _sleep(cdc42580,cdc42598,0,c10716a6,0,...) at _sleep+0x34e/frame 0xf9e2dca4 taskqueue_thread_loop(cee3b080,f9e2dd08,c107ef40,3d7,0,...) at taskqueue_thread_loop+0xb9/frame 0xf9e2dccc fork_exit(c0ad9f90,cee3b080,f9e2dd08) at fork_exit+0x7f/frame 0xf9e2dcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf9e2dcf4 --- trap 0, eip = 0, esp = 0xf9e2dd40, ebp = 0 --- Tracing command kernel pid 0 tid 100573 td 0xd17bfbc0 sched_switch(d17bfbc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9e2abc0 mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xf9e2abf4 sleepq_switch(d17bfbc0,0,c108af51,26d,d17bfbc0,...) at sleepq_switch+0x1b9/frame 0xf9e2ac28 sleepq_wait(cdc42580,0,c10716a6,0,0,...) at sleepq_wait+0x48/frame 0xf9e2ac4c _sleep(cdc42580,cdc42598,0,c10716a6,0,...) at _sleep+0x34e/frame 0xf9e2aca4 taskqueue_thread_loop(cee3b080,f9e2ad08,c107ef40,3d7,0,...) at taskqueue_thread_loop+0xb9/frame 0xf9e2accc fork_exit(c0ad9f90,cee3b080,f9e2ad08) at fork_exit+0x7f/frame 0xf9e2acf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf9e2acf4 --- trap 0, eip = 0, esp = 0xf9e2ad40, ebp = 0 --- Tracing command kernel pid 0 tid 100572 td 0xd17c3000 sched_switch(d17c3000,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9e27bc0 mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xf9e27bf4 sleepq_switch(d17c3000,0,c108af51,26d,d17c3000,...) at sleepq_switch+0x1b9/frame 0xf9e27c28 sleepq_wait(cdc42580,0,c10716a6,0,0,...) at sleepq_wait+0x48/frame 0xf9e27c4c _sleep(cdc42580,cdc42598,0,c10716a6,0,...) at _sleep+0x34e/frame 0xf9e27ca4 taskqueue_thread_loop(cee3b080,f9e27d08,c107ef40,3d7,0,...) at taskqueue_thread_loop+0xb9/frame 0xf9e27ccc fork_exit(c0ad9f90,cee3b080,f9e27d08) at fork_exit+0x7f/frame 0xf9e27cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf9e27cf4 --- trap 0, eip = 0, esp = 0xf9e27d40, ebp = 0 --- Tracing command kernel pid 0 tid 100571 td 0xd17c32f0 sched_switch(d17c32f0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf9e24bc0 mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xf9e24bf4 sleepq_switch(d17c32f0,0,c108af51,26d,d17c32f0,...) at sleepq_switch+0x1b9/frame 0xf9e24c28 sleepq_wait(cdc42580,0,c10716a6,0,0,...) at sleepq_wait+0x48/frame 0xf9e24c4c _sleep(cdc42580,cdc42598,0,c10716a6,0,...) at _sleep+0x34e/frame 0xf9e24ca4 taskqueue_thread_loop(cee3b080,f9e24d08,c107ef40,3d7,0,...) at taskqueue_thread_loop+0xb9/frame 0xf9e24ccc fork_exit(c0ad9f90,cee3b080,f9e24d08) at fork_exit+0x7f/frame 0xf9e24cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf9e24cf4 --- trap 0, eip = 0, esp = 0xf9e24d40, ebp = 0 --- Tracing command kernel pid 0 tid 100066 td 0xc950e8d0 sched_switch(c950e8d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf4001b8c mi_switch(104,0,c108af51,1f7,2,...) at mi_switch+0x1c9/frame 0xf4001bc0 sleepq_switch(c950e8d0,0,c108af51,290,c950e8d0,...) at sleepq_switch+0x1b9/frame 0xf4001bf4 sleepq_timedwait(c1230da0,0,c10716a6,2,0,...) at sleepq_timedwait+0x48/frame 0xf4001c18 _sleep(c1230da0,0,0,c10716a6,bb8) at _sleep+0x317/frame 0xf4001c70 pause(c10716a6,bb8,12c,12a,c13f8514,...) at pause+0x85/frame 0xf4001c94 deadlkres(0,f4001d08,c107ef40,3d7,0,...) at deadlkres+0x48/frame 0xf4001ccc fork_exit(c0a459f0,0,f4001d08) at fork_exit+0x7f/frame 0xf4001cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf4001cf4 --- trap 0, eip = 0, esp = 0xf4001d40, ebp = 0 --- Tracing command kernel pid 0 tid 100065 td 0xc950ebc0 sched_switch(c950ebc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xf3ffebd8 mi_switch(104,0,c108af51,1f7,c124d1e4,...) at mi_switch+0x1c9/frame 0xf3ffec0c sleepq_switch(c950ebc0,0,c108af51,26d,ca33b580,...) at sleepq_switch+0x1b9/frame 0xf3ffec40 sleepq_wait(ca33b580,0,c108604d,c10716a6,0,...) at sleepq_wait+0x48/frame 0xf3ffec64 msleep_spin(ca33b580,ca33b598,c10716a6,0,c950ebc0,...) at msleep_spin+0x216/frame 0xf3ffeca4 taskqueue_thread_loop(c13f4bd0,f3ffed08,c107ef40,3d7,0,...) at taskqueue_thread_loop+0x7c/frame 0xf3ffeccc fork_exit(c0ad9f90,c13f4bd0,f3ffed08) at fork_exit+0x7f/frame 0xf3ffecf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf3ffecf4 --- trap 0, eip = 0, esp = 0xf3ffed40, ebp = 0 --- Tracing command kernel pid 0 tid 100027 td 0xc7ef1bc0 sched_switch(c7ef1bc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xc7b8fbc0 mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xc7b8fbf4 sleepq_switch(c7ef1bc0,0,c108af51,26d,c7ef1bc0,...) at sleepq_switch+0x1b9/frame 0xc7b8fc28 sleepq_wait(c7fdf4c0,0,c10716a6,0,0,...) at sleepq_wait+0x48/frame 0xc7b8fc4c _sleep(c7fdf4c0,c7fdf4d8,0,c10716a6,0,...) at _sleep+0x34e/frame 0xc7b8fca4 taskqueue_thread_loop(c13f8c68,c7b8fd08,c107ef40,3d7,0,...) at taskqueue_thread_loop+0xb9/frame 0xc7b8fccc fork_exit(c0ad9f90,c13f8c68,c7b8fd08) at fork_exit+0x7f/frame 0xc7b8fcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b8fcf4 --- trap 0, eip = 0, esp = 0xc7b8fd40, ebp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xc7fdd2f0 sched_switch(c7fdd2f0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xc7b89bc0 mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xc7b89bf4 sleepq_switch(c7fdd2f0,0,c108af51,26d,c7fdd2f0,...) at sleepq_switch+0x1b9/frame 0xc7b89c28 sleepq_wait(c7f622c0,0,c10716a6,0,0,...) at sleepq_wait+0x48/frame 0xc7b89c4c _sleep(c7f622c0,c7f622d8,0,c10716a6,0,...) at _sleep+0x34e/frame 0xc7b89ca4 taskqueue_thread_loop(c13fbb2c,c7b89d08,c107ef40,3d7,0,...) at taskqueue_thread_loop+0xb9/frame 0xc7b89ccc fork_exit(c0ad9f90,c13fbb2c,c7b89d08) at fork_exit+0x7f/frame 0xc7b89cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b89cf4 --- trap 0, eip = 0, esp = 0xc7b89d40, ebp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xc7fddbc0 sched_switch(c7fddbc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xc7b80bd8 mi_switch(104,0,c108af51,1f7,c124d034,...) at mi_switch+0x1c9/frame 0xc7b80c0c sleepq_switch(c7fddbc0,0,c108af51,26d,c7f62380,...) at sleepq_switch+0x1b9/frame 0xc7b80c40 sleepq_wait(c7f62380,0,c108604d,c10716a6,0,...) at sleepq_wait+0x48/frame 0xc7b80c64 msleep_spin(c7f62380,c7f62398,c10716a6,0,c7fddbc0,...) at msleep_spin+0x216/frame 0xc7b80ca4 taskqueue_thread_loop(c13f57e8,c7b80d08,c107ef40,3d7,0,...) at taskqueue_thread_loop+0x7c/frame 0xc7b80ccc fork_exit(c0ad9f90,c13f57e8,c7b80d08) at fork_exit+0x7f/frame 0xc7b80cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b80cf4 --- trap 0, eip = 0, esp = 0xc7b80d40, ebp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xc7e2b8d0 sched_switch(c7e2b8d0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xc7b7dbd8 mi_switch(104,0,c108af51,1f7,c124d034,...) at mi_switch+0x1c9/frame 0xc7b7dc0c sleepq_switch(c7e2b8d0,0,c108af51,26d,c7f62380,...) at sleepq_switch+0x1b9/frame 0xc7b7dc40 sleepq_wait(c7f62380,0,c108604d,c10716a6,0,...) at sleepq_wait+0x48/frame 0xc7b7dc64 msleep_spin(c7f62380,c7f62398,c10716a6,0,c7e2b8d0,...) at msleep_spin+0x216/frame 0xc7b7dca4 taskqueue_thread_loop(c13f57e8,c7b7dd08,c107ef40,3d7,0,...) at taskqueue_thread_loop+0x7c/frame 0xc7b7dccc fork_exit(c0ad9f90,c13f57e8,c7b7dd08) at fork_exit+0x7f/frame 0xc7b7dcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b7dcf4 --- trap 0, eip = 0, esp = 0xc7b7dd40, ebp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xc7e2bbc0 sched_switch(c7e2bbc0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xc7b7abd8 mi_switch(104,0,c108af51,1f7,c124d034,...) at mi_switch+0x1c9/frame 0xc7b7ac0c sleepq_switch(c7e2bbc0,0,c108af51,26d,c7f62380,...) at sleepq_switch+0x1b9/frame 0xc7b7ac40 sleepq_wait(c7f62380,0,c108604d,c10716a6,0,...) at sleepq_wait+0x48/frame 0xc7b7ac64 msleep_spin(c7f62380,c7f62398,c10716a6,0,c7e2bbc0,...) at msleep_spin+0x216/frame 0xc7b7aca4 taskqueue_thread_loop(c13f57e8,c7b7ad08,c107ef40,3d7,0,...) at taskqueue_thread_loop+0x7c/frame 0xc7b7accc fork_exit(c0ad9f90,c13f57e8,c7b7ad08) at fork_exit+0x7f/frame 0xc7b7acf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b7acf4 --- trap 0, eip = 0, esp = 0xc7b7ad40, ebp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xc7ef0000 sched_switch(c7ef0000,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xc7b77bc0 mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xc7b77bf4 sleepq_switch(c7ef0000,0,c108af51,26d,c7ef0000,...) at sleepq_switch+0x1b9/frame 0xc7b77c28 sleepq_wait(c7f623c0,0,c10716a6,0,0,...) at sleepq_wait+0x48/frame 0xc7b77c4c _sleep(c7f623c0,c7f623d8,0,c10716a6,0,...) at _sleep+0x34e/frame 0xc7b77ca4 taskqueue_thread_loop(c13ff178,c7b77d08,c107ef40,3d7,0,...) at taskqueue_thread_loop+0xb9/frame 0xc7b77ccc fork_exit(c0ad9f90,c13ff178,c7b77d08) at fork_exit+0x7f/frame 0xc7b77cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b77cf4 --- trap 0, eip = 0, esp = 0xc7b77d40, ebp = 0 --- Tracing command kernel pid 0 tid 100013 td 0xc7de32f0 sched_switch(c7de32f0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xc7b64bc0 mi_switch(104,0,c108af51,1f7,164,...) at mi_switch+0x1c9/frame 0xc7b64bf4 sleepq_switch(c7de32f0,0,c108af51,26d,c7de32f0,...) at sleepq_switch+0x1b9/frame 0xc7b64c28 sleepq_wait(c7d7a7c0,0,c10716a6,0,0,...) at sleepq_wait+0x48/frame 0xc7b64c4c _sleep(c7d7a7c0,c7d7a7d8,0,c10716a6,0,...) at _sleep+0x34e/frame 0xc7b64ca4 taskqueue_thread_loop(c124c1c0,c7b64d08,c107ef40,3d7,0,...) at taskqueue_thread_loop+0xb9/frame 0xc7b64ccc fork_exit(c0ad9f90,c124c1c0,c7b64d08) at fork_exit+0x7f/frame 0xc7b64cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b64cf4 --- trap 0, eip = 0, esp = 0xc7b64d40, ebp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xc13f87f0 sched_switch(c13f87f0,0,104,1b3,c0a8195e,...) at sched_switch+0x4b4/frame 0xc1820c20 mi_switch(104,0,c108af51,1f7,1,...) at mi_switch+0x1c9/frame 0xc1820c54 sleepq_switch(c13f87f0,0,c108af51,290,c13f87f0,...) at sleepq_switch+0x1b9/frame 0xc1820c88 sleepq_timedwait(c13f8514,54,c10b7805,0,0,...) at sleepq_timedwait+0x48/frame 0xc1820cac _sleep(c13f8514,0,54,c10b7805,2710,...) at _sleep+0x317/frame 0xc1820d04 scheduler(0,0,c10e9f80,181e000,1825000,...) at scheduler+0x274/frame 0xc1820d38 mi_startup() at mi_startup+0xf7/frame 0xc1820d58 begin() at begin+0x2c db:0:allt> call doadump Physical memory: 3435 MB Dumping 573 MB: 558 542 526 510 494 478 462 446 430 414 398 382 366 350 334 318 302 286 270 254 238 222 206 190 174 158 142 126 110 94 78 62 46 30 14 Dump complete = 0 db:0:doadump> reset cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 1 (kgdb) bt #0 doadump (textdump=0xa) at pcpu.h:249 #1 0xc05263fd in db_fncall (dummy1=0xf419a4e4, dummy2=0xf419a4e4, dummy3=0xf419a4e4, dummy4=0xf419a4e4 "\n") at ../../../ddb/db_command.c:578 #2 0xc05260f7 in db_command (cmd_table=) at ../../../ddb/db_command.c:449 #3 0xc0529fa4 in db_script_exec (scriptname=0xc13f5550 "doadump", warnifnotfound=) at ../../../ddb/db_script.c:302 #4 0xc05260f7 in db_command (cmd_table=) at ../../../ddb/db_command.c:449 #5 0xc0525e10 in db_command_loop () at ../../../ddb/db_command.c:502 #6 0xc0528650 in db_trap (type=, code=0xf419a8d0) at ../../../ddb/db_main.c:231 #7 0xc0acb256 in kdb_trap (type=, code=, tf=) at ../../../kern/subr_kdb.c:654 #8 0xc0ea83ce in trap (frame=) at ../../../i386/i386/trap.c:720 #9 0xc0e9159c in calltrap () at exception.s:169 #10 0xc0acaafd in kdb_enter (why=0xc10853c1 "panic", msg=) at cpufunc.h:71 #11 0xc0a95ca2 in vpanic (fmt=, ap=) at ../../../kern/kern_shutdown.c:746 #12 0xc0a95cd2 in panic (fmt=0xf419a9b8 "È©\031ôð©\031ôê>ÑÀ¶\n\fÁ") at ../../../kern/kern_shutdown.c:682 #13 0xc0d13eea in kmem_malloc (map=, size=, flags=) at ../../../vm/vm_kern.c:446 #14 0xc0d0cc47 in page_alloc (zone=0xf419a9f8, bytes=0xf419a9f8, pflag=0xf419a9f8 "\214P»Á", wait=0xf419a9f8) at ../../../vm/uma_core.c:1017 #15 0xc0d0c603 in keg_alloc_slab (keg=0xc1ba1680, zone=0x102, wait=) at ../../../vm/uma_core.c:867 #16 0xc0d0d4b2 in keg_fetch_slab (keg=0xc1ba1680, zone=0xc1b9fd00, flags=) at ../../../vm/uma_core.c:2228 #17 0xc0d0d85f in zone_fetch_slab (zone=0xc1b9fd00, keg=, flags=) at ../../../vm/uma_core.c:2288 #18 0xc0d0b3f5 in uma_zalloc_arg (zone=, udata=0xc0a809f7, flags=) at ../../../vm/uma_core.c:2460 #19 0xc09f3d83 in g_alloc_bio () at uma.h:309 #20 0xc09fc1e0 in g_vfs_strategy (bo=0xf419aaec, bp=0xe19aa7d4) at ../../../geom/geom_vfs.c:188 #21 0xc0b1799c in bufwrite (bp=0xe19aa7d4) at buf.h:409 #22 0xc0cedf5b in ffs_bufwrite (bp=) at ../../../ufs/ffs/ffs_vfsops.c:2158 #23 0xc0b1a735 in vfs_bio_awrite (bp=0xe19aa7d4) at buf.h:397 #24 0xc0b25e38 in vop_stdfsync (ap=) at ../../../kern/vfs_default.c:678 #25 0xc0981d8a in devfs_fsync (ap=0xf419aca8) at ../../../fs/devfs/devfs_vnops.c:646 #26 0xc0ecdd14 in VOP_FSYNC_APV (vop=, a=0xf419aca8) at vnode_if.c:1330 #27 0xc0b3bfcf in sched_sync () at vnode_if.h:549 #28 0xc0a6381f in fork_exit (callout=0xc0b3bc40 ) at ../../../kern/kern_fork.c:991 #29 0xc0e91644 in fork_trampoline () at exception.s:276 Current language: auto; currently minimal (kgdb) f 14 #14 0xc0d0cc47 in page_alloc (zone=0xf419a9f8, bytes=0xf419a9f8, pflag=0xf419a9f8 "\214P»Á", wait=0xf419a9f8) at ../../../vm/uma_core.c:1017 1017 p = (void *) kmem_malloc(kmem_map, bytes, wait); (kgdb) l 1012 page_alloc(uma_zone_t zone, int bytes, u_int8_t *pflag, int wait) 1013 { 1014 void *p; /* Returned page */ 1015 1016 *pflag = UMA_SLAB_KMEM; 1017 p = (void *) kmem_malloc(kmem_map, bytes, wait); 1018 1019 return (p); 1020 } 1021 (kgdb) info loc p = (kgdb) Script started on Sat Mar 2 06:58:03 2013 x4# vmstat -z -N kernel.debug -M /var/crash/vmcore.548 ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP UMA Kegs: 128, 0, 256, 14, 268, 0, 0 UMA Zones: 236, 0, 256, 14, 268, 0, 0 UMA Slabs: 284, 0, 6717, 437, 194051, 0, 0 UMA RCntSlabs: 544, 0, 1139, 1010, 14707, 0, 0 UMA Hash: 128, 0, 80, 10, 86, 0, 0 16 Bucket: 76, 0, 151, 149, 356, 0, 0 32 Bucket: 140, 0, 89, 79, 339, 0, 0 64 Bucket: 268, 0, 77, 147, 620, 28, 0 128 Bucket: 524, 0, 664, 736, 376335,402358, 0 VM OBJECT: 148, 0, 72722, 44512,142209438, 0, 0 MAP: 140, 0, 8, 48, 8, 0, 0 KMAP ENTRY: 72, 155078, 184, 15345,42427175, 0, 0 MAP ENTRY: 72, 0, 2402, 2951,150083070, 0, 0 fakepg: 72, 0, 0, 371, 4764, 0, 0 mt_zone: 2060, 0, 361, 94, 363, 0, 0 16: 16, 0, 48, 764, 210049, 0, 0 16: 16, 0, 254, 1370,55467233, 0, 0 16: 16, 0, 47, 2186, 20839, 0, 0 16: 16, 0, 2957, 1509, 3034716, 0, 0 16: 16, 0, 311, 704, 4132758, 0, 0 16: 16, 0, 42, 15995, 8637279, 0, 0 16: 16, 0, 207, 20905, 5767641, 0, 0 16: 16, 0, 46, 766, 17903, 0, 0 32: 32, 0, 914, 668, 84835, 0, 0 32: 32, 0, 22, 769, 2405234, 0, 0 32: 32, 0, 34, 983, 17651, 0, 0 32: 32, 0, 1243, 5085,132538801, 0, 0 32: 32, 0, 78, 600, 980672, 0, 0 32: 32, 0, 49, 855, 3886328, 0, 0 32: 32, 0, 839, 8314,40455482, 0, 0 32: 32, 0, 68, 2079, 1100825, 0, 0 64: 64, 0, 1646, 773, 97350, 0, 0 64: 64, 0, 49, 1485,38114401, 0, 0 64: 64, 0, 227, 17945,145413602, 0, 0 64: 64, 0, 1684, 9526,141999071, 0, 0 64: 64, 0, 1338, 727,12695794, 0, 0 64: 64, 0, 2371, 1700, 177508, 0, 0 64: 64, 0, 1613, 1927,144146842, 0, 0 64: 64, 0, 200, 685, 792052, 0, 0 128: 128, 0, 21, 249, 8084, 0, 0 128: 128, 0, 23, 1627, 1755645, 0, 0 128: 128, 0, 533562, 1608, 9371927, 0, 0 128: 128, 0, 311, 1549,58936819, 0, 0 128: 128, 0, 62, 148, 59317, 0, 0 128: 128, 0, 2392, 1718, 166718, 0, 0 128: 128, 0, 731, 5089, 2662583, 0, 0 128: 128, 0, 1067017, 4343,25802491, 0, 0 256: 256, 0, 6, 144, 3367, 0, 0 256: 256, 0, 28, 62, 2518, 0, 0 256: 256, 0, 6, 24, 6, 0, 0 256: 256, 0, 20, 175, 4143, 0, 0 256: 256, 0, 735, 3180,71500827, 0, 0 256: 256, 0, 557, 4078,53099381, 0, 0 256: 256, 0, 556, 1709, 1878471, 0, 0 256: 256, 0, 1492, 848, 39281, 0, 0 512: 512, 0, 1005, 355, 375635, 0, 0 512: 512, 0, 64, 1440, 1336132, 0, 0 512: 512, 0, 2, 694, 1153699, 0, 0 512: 512, 0, 13, 171, 1433251, 0, 0 512: 512, 0, 270, 594, 2164644, 0, 0 512: 512, 0, 1, 15, 7, 0, 0 512: 512, 0, 31, 41, 1304, 0, 0 512: 512, 0, 0, 88, 5444, 0, 0 1024: 1024, 0, 3, 85, 2033, 0, 0 1024: 1024, 0, 2, 26, 54, 0, 0 1024: 1024, 0, 1, 7, 1, 0, 0 1024: 1024, 0, 8, 128, 2954, 0, 0 1024: 1024, 0, 1, 107,18866125, 0, 0 1024: 1024, 0, 6, 82, 129485, 0, 0 1024: 1024, 0, 19, 37, 2238, 0, 0 1024: 1024, 0, 8, 28, 387, 0, 0 2048: 2048, 0, 2, 70, 409, 0, 0 2048: 2048, 0, 47, 5, 90, 0, 0 2048: 2048, 0, 6, 2, 6, 0, 0 2048: 2048, 0, 358, 334, 2348793, 0, 0 2048: 2048, 0, 5065, 551, 1691428, 0, 0 2048: 2048, 0, 0, 8, 5, 0, 0 2048: 2048, 0, 10, 32, 4616, 0, 0 2048: 2048, 0, 13, 625, 89082, 0, 0 4096: 4096, 0, 4, 22, 16, 0, 0 4096: 4096, 0, 0, 0, 0, 0, 0 4096: 4096, 0, 2, 133, 439, 0, 0 4096: 4096, 0, 11, 33, 1975, 0, 0 4096: 4096, 0, 200, 84, 141256, 0, 0 4096: 4096, 0, 138, 526,10272047, 0, 0 4096: 4096, 0, 46, 31, 189, 0, 0 4096: 4096, 0, 45, 45, 138, 0, 0 Files: 56, 0, 484, 98944,62524315, 0, 0 TURNSTILE: 72, 0, 2336, 1954, 27346, 0, 0 rl_entry: 28, 0, 925, 599, 1032, 0, 0 umtx pi: 52, 0, 0, 0, 0, 0, 0 MAC labels: 20, 0, 0, 0, 0, 0, 0 PROC: 720, 0, 123, 412,10272029, 0, 0 THREAD: 752, 0, 600, 1735,22420758, 0, 0 SLEEPQUEUE: 48, 0, 2336, 1735, 27346, 0, 0 VMSPACE: 232, 0, 105, 796,10271961, 0, 0 cpuset: 40, 0, 85, 559, 87768, 0, 0 audit_record: 1096, 0, 0, 0, 0, 0, 0 mbuf_packet: 256, 419850, 256, 2222,19575249, 0, 0 mbuf: 256, 419850, 4, 2354,117574700, 0, 0 mbuf_cluster: 2048, 65600, 702, 584, 1551183, 0, 0 mbuf_jumbo_page: 4096, 32800, 0, 496, 4420239, 0, 0 mbuf_jumbo_9k: 9216, 29154, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 21864, 0, 0, 0, 0, 0 mbuf_ext_refcnt: 4, 0, 0, 0, 31, 0, 0 g_bio: 164, 0, 0, 984,86512260, 4, 0 ttyinq: 152, 0, 1020, 1840,20675055, 0, 0 ttyoutq: 256, 0, 537, 993,11026681, 0, 0 ata_request: 208, 0, 0, 380, 9872, 0, 0 ata_composite: 180, 0, 0, 0, 0, 0, 0 vtnet_tx_hdr: 16, 0, 0, 0, 0, 0, 0 VNODE: 360, 0, 101346, 151,54608624, 0, 0 VNODEPOLL: 60, 0, 1, 188, 4, 0, 0 S VFS Cache: 72, 0, 108243, 301,16894993, 0, 0 STS VFS Cache: 92, 0, 0, 1722, 331432, 0, 0 L VFS Cache: 292, 0, 602, 4091, 834176, 0, 0 LTS VFS Cache: 312, 0, 0, 72, 549, 0, 0 NAMEI: 1024, 0, 2, 654,228804180, 0, 0 DIRHASH: 1024, 0, 0, 768, 1011173, 0, 0 NCLNODE: 356, 0, 0, 1782, 170220, 0, 0 Mountpoints: 748, 0, 7, 98, 16693, 0, 0 capability: 16, 0, 0, 406, 8, 0, 0 pipe: 408, 0, 35, 307, 725903, 0, 0 ksiginfo: 80, 0, 526, 626, 321601, 0, 0 itimer: 220, 0, 1, 71, 3, 0, 0 KNOTE: 72, 0, 3, 368, 520620, 0, 0 socket: 416, 109935, 73, 503, 24342, 0, 0 unpcb: 172, 109940, 23, 92, 336, 0, 0 ipq: 32, 2147, 0, 0, 0, 0, 0 udp_inpcb: 252, 109935, 26, 124, 19207, 0, 0 udpcb: 8, 110026, 26, 583, 19207, 0, 0 tcp_inpcb: 252, 109935, 23, 1792, 4793, 0, 0 tcpcb: 688, 109935, 23, 97, 4793, 0, 0 tcptw: 52, 22032, 0, 2088, 1930, 0, 0 syncache: 120, 15360, 0, 160, 1980, 0, 0 hostcache: 76, 15400, 1, 99, 2, 0, 0 tcpreass: 20, 4225, 0, 0, 0, 0, 0 sackhole: 20, 0, 0, 0, 0, 0, 0 sctp_ep: 936, 109936, 0, 0, 0, 0, 0 sctp_asoc: 1504, 40000, 0, 0, 0, 0, 0 sctp_laddr: 24, 80040, 0, 290, 4, 0, 0 sctp_raddr: 500, 80000, 0, 0, 0, 0, 0 sctp_chunk: 96, 400000, 0, 0, 0, 0, 0 sctp_readq: 76, 400000, 0, 0, 0, 0, 0 sctp_stream_msg_out: 68, 400008, 0, 0, 0, 0, 0 sctp_asconf: 24, 400055, 0, 0, 0, 0, 0 sctp_asconf_ack: 24, 400055, 0, 0, 0, 0, 0 ripcb: 252, 109935, 0, 0, 0, 0, 0 rtentry: 108, 0, 17, 163, 17, 0, 0 selfd: 28, 0, 773, 624, 6978828, 0, 0 SWAPMETA: 276, 303940, 1849, 33473, 851036, 0, 0 FFS inode: 116, 0, 101217, 159,46132094, 0, 0 FFS1 dinode: 128, 0, 0, 8700, 2349216, 0, 0 FFS2 dinode: 256, 0, 101217, 273,43690134, 0, 0 taskq_zone: 24, 0, 0, 0, 0, 0, 0 space_seg_cache: 48, 0, 0, 0, 0, 0, 0 zio_cache: 744, 0, 0, 0, 0, 0, 0 zio_link_cache: 24, 0, 0, 0, 0, 0, 0 zio_buf_512: 512, 0, 0, 0, 0, 0, 0 zio_data_buf_512: 512, 0, 0, 0, 0, 0, 0 zio_buf_1024: 1024, 0, 0, 0, 0, 0, 0 zio_data_buf_1024: 1024, 0, 0, 0, 0, 0, 0 zio_buf_1536: 1536, 0, 0, 0, 0, 0, 0 zio_data_buf_1536: 1536, 0, 0, 0, 0, 0, 0 zio_buf_2048: 2048, 0, 0, 0, 0, 0, 0 zio_data_buf_2048: 2048, 0, 0, 0, 0, 0, 0 zio_buf_2560: 2560, 0, 0, 0, 0, 0, 0 zio_data_buf_2560: 2560, 0, 0, 0, 0, 0, 0 zio_buf_3072: 3072, 0, 0, 0, 0, 0, 0 zio_data_buf_3072: 3072, 0, 0, 0, 0, 0, 0 zio_buf_3584: 3584, 0, 0, 0, 0, 0, 0 zio_data_buf_3584: 3584, 0, 0, 0, 0, 0, 0 zio_buf_4096: 4096, 0, 0, 0, 0, 0, 0 zio_data_buf_4096: 4096, 0, 0, 0, 0, 0, 0 zio_buf_5120: 5120, 0, 0, 0, 0, 0, 0 zio_data_buf_5120: 5120, 0, 0, 0, 0, 0, 0 zio_buf_6144: 6144, 0, 0, 0, 0, 0, 0 zio_data_buf_6144: 6144, 0, 0, 0, 0, 0, 0 zio_buf_7168: 7168, 0, 0, 0, 0, 0, 0 zio_data_buf_7168: 7168, 0, 0, 0, 0, 0, 0 zio_buf_8192: 8192, 0, 0, 0, 0, 0, 0 zio_data_buf_8192: 8192, 0, 0, 0, 0, 0, 0 zio_buf_10240: 10240, 0, 0, 0, 0, 0, 0 zio_data_buf_10240: 10240, 0, 0, 0, 0, 0, 0 zio_buf_12288: 12288, 0, 0, 0, 0, 0, 0 zio_data_buf_12288: 12288, 0, 0, 0, 0, 0, 0 zio_buf_14336: 14336, 0, 0, 0, 0, 0, 0 zio_data_buf_14336: 14336, 0, 0, 0, 0, 0, 0 zio_buf_16384: 16384, 0, 0, 0, 0, 0, 0 zio_data_buf_16384: 16384, 0, 0, 0, 0, 0, 0 zio_buf_20480: 20480, 0, 0, 0, 0, 0, 0 zio_data_buf_20480: 20480, 0, 0, 0, 0, 0, 0 zio_buf_24576: 24576, 0, 0, 0, 0, 0, 0 zio_data_buf_24576: 24576, 0, 0, 0, 0, 0, 0 zio_buf_28672: 28672, 0, 0, 0, 0, 0, 0 zio_data_buf_28672: 28672, 0, 0, 0, 0, 0, 0 zio_buf_32768: 32768, 0, 0, 0, 0, 0, 0 zio_data_buf_32768: 32768, 0, 0, 0, 0, 0, 0 zio_buf_36864: 36864, 0, 0, 0, 0, 0, 0 zio_data_buf_36864: 36864, 0, 0, 0, 0, 0, 0 zio_buf_40960: 40960, 0, 0, 0, 0, 0, 0 zio_data_buf_40960: 40960, 0, 0, 0, 0, 0, 0 zio_buf_45056: 45056, 0, 0, 0, 0, 0, 0 zio_data_buf_45056: 45056, 0, 0, 0, 0, 0, 0 zio_buf_49152: 49152, 0, 0, 0, 0, 0, 0 zio_data_buf_49152: 49152, 0, 0, 0, 0, 0, 0 zio_buf_53248: 53248, 0, 0, 0, 0, 0, 0 zio_data_buf_53248: 53248, 0, 0, 0, 0, 0, 0 zio_buf_57344: 57344, 0, 0, 0, 0, 0, 0 zio_data_buf_57344: 57344, 0, 0, 0, 0, 0, 0 zio_buf_61440: 61440, 0, 0, 0, 0, 0, 0 zio_data_buf_61440: 61440, 0, 0, 0, 0, 0, 0 zio_buf_65536: 65536, 0, 0, 0, 0, 0, 0 zio_data_buf_65536: 65536, 0, 0, 0, 0, 0, 0 zio_buf_69632: 69632, 0, 0, 0, 0, 0, 0 zio_data_buf_69632: 69632, 0, 0, 0, 0, 0, 0 zio_buf_73728: 73728, 0, 0, 0, 0, 0, 0 zio_data_buf_73728: 73728, 0, 0, 0, 0, 0, 0 zio_buf_77824: 77824, 0, 0, 0, 0, 0, 0 zio_data_buf_77824: 77824, 0, 0, 0, 0, 0, 0 zio_buf_81920: 81920, 0, 0, 0, 0, 0, 0 zio_data_buf_81920: 81920, 0, 0, 0, 0, 0, 0 zio_buf_86016: 86016, 0, 0, 0, 0, 0, 0 zio_data_buf_86016: 86016, 0, 0, 0, 0, 0, 0 zio_buf_90112: 90112, 0, 0, 0, 0, 0, 0 zio_data_buf_90112: 90112, 0, 0, 0, 0, 0, 0 zio_buf_94208: 94208, 0, 0, 0, 0, 0, 0 zio_data_buf_94208: 94208, 0, 0, 0, 0, 0, 0 zio_buf_98304: 98304, 0, 0, 0, 0, 0, 0 zio_data_buf_98304: 98304, 0, 0, 0, 0, 0, 0 zio_buf_102400: 102400, 0, 0, 0, 0, 0, 0 zio_data_buf_102400: 102400, 0, 0, 0, 0, 0, 0 zio_buf_106496: 106496, 0, 0, 0, 0, 0, 0 zio_data_buf_106496: 106496, 0, 0, 0, 0, 0, 0 zio_buf_110592: 110592, 0, 0, 0, 0, 0, 0 zio_data_buf_110592: 110592, 0, 0, 0, 0, 0, 0 zio_buf_114688: 114688, 0, 0, 0, 0, 0, 0 zio_data_buf_114688: 114688, 0, 0, 0, 0, 0, 0 zio_buf_118784: 118784, 0, 0, 0, 0, 0, 0 zio_data_buf_118784: 118784, 0, 0, 0, 0, 0, 0 zio_buf_122880: 122880, 0, 0, 0, 0, 0, 0 zio_data_buf_122880: 122880, 0, 0, 0, 0, 0, 0 zio_buf_126976: 126976, 0, 0, 0, 0, 0, 0 zio_data_buf_126976: 126976, 0, 0, 0, 0, 0, 0 zio_buf_131072: 131072, 0, 0, 0, 0, 0, 0 zio_data_buf_131072: 131072, 0, 0, 0, 0, 0, 0 sa_cache: 44, 0, 0, 0, 0, 0, 0 dnode_t: 532, 0, 0, 0, 0, 0, 0 dmu_buf_impl_t: 136, 0, 0, 0, 0, 0, 0 arc_buf_hdr_t: 148, 0, 0, 0, 0, 0, 0 arc_buf_t: 40, 0, 0, 0, 0, 0, 0 zil_lwb_cache: 168, 0, 0, 0, 0, 0, 0 zfs_znode_cache: 252, 0, 0, 0, 0, 0, 0 mqnode: 344, 0, 3, 19, 3, 0, 0 mqueue: 132, 0, 0, 0, 0, 0, 0 mvdata: 32, 0, 0, 0, 0, 0, 0 mqnotifier: 136, 0, 0, 0, 0, 0, 0 fuse_ticket: 132, 0, 0, 1102, 2178625, 0, 0 x4# vmstat -m -N kernel.debug -M /var/crash/vmcore.548 Type InUse MemUse HighUse Requests Size(s) CAM periph 6 1K - 28 16,32,64,128 acpitask 1 2K - 1 2048 isadev 7 1K - 7 64 acpisem 17 2K - 17 64,128 CAM queue 26 1K - 140 16,32,256 acpidev 31 1K - 31 32 CAM dev queue 7 1K - 7 128 cdev 8 1K - 8 128 CAM SIM 7 1K - 7 128 ctlmem 5062 10113K - 5062 64,2048 filedesc 147 46K - 10272420 16,32,64,128,256,512,1024,2048,4096 sigio 1 1K - 1 32 kdtrace 725 158K - 32692789 64,256 kenv 86 8K - 107 16,32,64,128,4096 kqueue 2 2K - 253269 128,256,1024 proc-args 77 5K - 984996 16,32,64,128,256 CAM XPT 33 2K - 152 16,32,64,1024,2048 hhook 2 1K - 2 128 ithread 86 7K - 86 16,64,128 KTRACE 100 13K - 100 128 USB 42 15K - 42 16,32,64,256,2048 USBdev 24 2K - 24 32,64,128 linker 214 2839K - 2028 16,32,64,128,256,512,1024,2048,4096 lockf 82 5K - 170438368 32,64,128,256 loginclass 3 1K - 211 64 devbuf 2601 23846K - 28328 16,32,64,128,256,512,1024,2048,4096 temp 43 234K - 26621487 16,32,64,128,256,512,1024,2048,4096 ip6ndp 4 1K - 5 64,128 ctlblk 200 800K - 200 4096 ramdisk 1 4096K - 1 module 513 33K - 516 64,128 mtx_pool 2 8K - 2 4096 ctlpool 532 138K - 532 16,512 pci_link 16 2K - 16 64,128 osd 3 1K - 6 16,32,64 pmchooks 1 1K - 1 64 scsi_cd 0 0K - 11 16 pgrp 44 3K - 15091 64 session 39 5K - 8626 128 proc 2 8K - 2 4096 subproc 661 631K - 10272569 256,4096 cred 182 18K - 40332347 64,128,256,512,1024,2048,4096 plimit 24 6K - 2514 256 uidinfo 4 2K - 49651 64,1024 entropy 1024 64K - 1024 64 CAM DEV 9 18K - 26 2048 CAM CCB 37 74K - 60 2048 sysctl 0 0K - 421886 16,32,64 sysctloid 4388 135K - 4509 16,32,64,128 sysctltmp 0 0K - 9062 16,32,64,128,256,4096 tidhash 1 8K - 1 callout 3 768K - 3 umtx 4670 438K - 54690 64,128 p1003.1b 1 1K - 1 16 SWAP 2 2189K - 2 64 bus 1238 54K - 5350009 16,32,64,128,256,1024 bus-sc 81 116K - 3225 16,32,64,128,256,512,1024,2048,4096 DEVFS2 172 11K - 1376 16,32,64 devstat 12 25K - 12 16,4096 eventhandler 86 4K - 87 32,64,128 DEVFS3 199 25K - 1397414 128 DEVFS1 173 44K - 1336018 256 kobj 352 704K - 2768 2048 Per-cpu 1 1K - 1 16 DEVFS 32 1K - 9605 16,64 DEVFSP 0 0K - 11 32 msdosfs_node 0 0K - 490499 128 rman 228 14K - 641 16,32,64 sbuf 0 0K - 354363 16,32,64,128,256,512,1024,2048,4096 msdosfs_mount 0 0K - 2 512 msdosfs_fat 0 0K - 2 2048 stack 0 0K - 4 128 taskqueue 17 1K - 17 16,64 Unitno 76 3K - 55463895 16,64 Witness 1 104K - 1 NFSD srvcache 3 1K - 1421260 64 ioctlops 0 0K - 2341464 16,32,64,128,256,512,1024,2048,4096 select 566 36K - 636 64 iov 16 1K - 125285273 16,32,64,128,256,512,1024,2048,4096 msg 4 25K - 4 1024,4096 sem 18446744073709551597 92K - 4 1024,4096 shm 1 12K - 7705 1024 tty 57 29K - 1335761 512,1024,2048 pts 39 5K - 1335739 128 mbuf_tag 0 0K - 85 32,64 shmfd 1 4K - 9 1024,4096 soname 8 1K - 1578675 16,32,64,128,256 pcb 34 331K - 20826 16,64,512,1024,4096 acl 0 0K - 100074 4096 biobuf 0 0K - 88309 2048 vfscache 1 512K - 1 cl_savebuf 0 0K - 2330750 32,64 export_host 4 1K - 1808 128 vfs_hash 1 256K - 1 vnodes 1 1K - 572 16,128 NFSD string 1 1K - 1 16 NFSD usrgroup 77 5K - 78 64 mount 133 14K - 7626828 16,32,64,128,256 vnodemarker 0 0K - 1153697 512 fadvise 0 0K - 12 64 BPF 2 1K - 2 64 ifnet 3 3K - 3 64,1024 ifaddr 39 9K - 39 16,32,64,128,256,512,1024,2048 ether_multi 40 2K - 46 16,32,64 clone 7 1K - 7 128 arpcom 1 1K - 1 16 lltable 10 3K - 27 128,256 NFS fh 0 0K - 1309743 32 routetbl 37 5K - 3132 16,32,64,128,256 igmp 2 1K - 2 128 in_multi 2 1K - 2 128 NFSCL diroffdiroff 0 0K - 314 512 sctp_a_it 0 0K - 3 16 sctp_vrf 1 1K - 1 64 sctp_ifa 5 1K - 5 128 sctp_ifn 2 1K - 2 128 sctp_iter 0 0K - 3 256 hostcache 1 16K - 1 syncache 1 72K - 1 in6_mfilter 1 1K - 1 512 in6_multi 22 3K - 22 16,256 ip6_moptions 2 1K - 2 32,128 mld 2 1K - 2 128 rpc 29 11K - 3771946 16,32,64,128,256,512,1024,2048,4096 audit_evclass 180 3K - 219 16 mactemp 0 0K - 2749 16 pagedep 3 65K - 1753780 128 inodedep 14 260K - 53094969 256 bmsafemap 15 6K - 37423896 128,4096 newblk 533466 66748K - 16940696 128 indirdep 1 1K - 73472 64 freefrag 0 0K - 2357415 64 freeblks 533455 66682K - 8162981 128 freefile 4 1K - 37274403 32 diradd 2 1K - 53852767 64 mkdir 0 0K - 1522126 64 dirrem 3 1K - 53775694 64 newdirblk 0 0K - 918747 32 freework 533456 66682K - 8847609 32,128 freedep 0 0K - 17617 32 jaddref 0 0K - 36542553 64 jremref 0 0K - 36532949 64 jmvref 0 0K - 11525576 64 jnewblk 0 0K - 5296615 64 jfreefrag 0 0K - 1180130 64 jseg 0 0K - 1208817 128 jsegdep 0 0K - 79552247 32 sbdep 0 0K - 54276 32 savedino 0 0K - 38683070 256 jblocks 0 0K - 22 128 ufs_dirhash 999 500K - 716579 16,32,64,128,256,512,1024,2048,4096 ufs_quota 2 257K - 2 128 ufs_mount 17 64K - 154 256,2048,4096 vm_pgdata 2 257K - 2 64 UMAHash 2 5K - 33 256,512,1024,2048,4096 newnfsmnt 0 0K - 642 512 pfs_nodes 21 3K - 21 128 pfs_vncache 0 0K - 2405161 32 GEOM 226 43K - 12849 16,32,64,128,256,512,1024,2048,4096 atkbddev 2 1K - 2 32 hdaa 9 44K - 9 256,512,1024,2048 hdac 2 1K - 2 64,512 hdacc 2 1K - 2 16 kbdmux 6 18K - 6 16,256,1024,2048 feeder 21 1K - 27 16,64 raid_data 0 0K - 2214 16,128 md_disk 0 0K - 465 16,2048 md_sectors 0 0K - 437 4096 ddb_capture 1 48K - 1 memdesc 1 4K - 1 4096 mixer 4 16K - 4 4096 acpiintr 1 1K - 1 32 md_nvidia_data 0 0K - 368 512 apmdev 1 1K - 1 64 madt_table 0 0K - 1 2048 md_sii_data 0 0K - 368 512 io_apic 1 1K - 1 1024 MCA 6 1K - 6 128 acpica 2564 134K - 48198 16,32,64,128,256,512,1024 msi 2 1K - 2 64 nexusdev 4 1K - 4 16 CAM path 11 1K - 58 16 UART 3 2K - 3 16,512,1024 linux 15 1K - 15 32,64 NFS FHA 1 1K - 1 1024 nullfs_hash 1 256K - 1 nullfs_node 0 0K - 734554 16 nullfs_mount 0 0K - 15358 16 tmpfs mount 0 0K - 4 128 tmpfs name 0 0K - 10288479 16,32 solaris 104 4359K - 106 16,32,64,1024 kstat_data 5 1K - 5 32 fuse_vnode 0 0K - 105010 256 fuse_msgbuf 2204 5814K - 40275 256,512,1024,2048,4096 x4# exit exit Script done on Sat Mar 2 06:59:10 2013 git diff -- Test scenario: truncate.sh