GENERIC HEAD from 2013-05-12 04:05:01 UTC, r250556M, vmcore.638 GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb 4718592K of memory above 4GB ignored Copyright (c) 1992-2013 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #3 r250556M: Thu May 16 07:40:29 CEST 2013 pho@x4.osted.lan:/usr/src/sys/i386/compile/PHO i386 FreeBSD clang version 3.3 (trunk 178860) 20130405 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. CPU: AMD Phenom(tm) 9150e Quad-Core Processor (1800.02-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x100f23 Family = 0x10 Model = 0x2 Stepping = 3 Features=0x178bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x7ff TSC: P-state invariant real memory = 8589934592 (8192 MB) avail memory = 3498360832 (3336 MB) : Trying to mount root from ufs:/dev/ad4s1a [rw]... Setting hostuuid: 00000000-0000-0000-0000-00218515337d. Setting hostid: 0x6b64ac17. Starting ddb. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: /dev/ad4s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1a: clean, 618473 free (1193 frags, 77160 blocks, 0.1% fragmentation) /dev/ad4s1f: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1f: clean, 184070 free (4366 frags, 22463 blocks, 0.4% fragmentation) /dev/label/tmp: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/label/tmp: clean, 48407715 free (3499 frags, 6050527 blocks, 0.0% fragmentation) /dev/ad4s1d: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1d: clean, 3936948 free (272924 frags, 458003 blocks, 2.7% fragmentation) /dev/ad4s1e: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1e: clean, 7785035 free (160203 frags, 953104 blocks, 0.6% fragmentation) Mounting local file systems:. Setting hostname: x4.osted.lan. re0: link state changed to DOWN re0: link state changed to UP Starting Network: lo0 re0. lo0: flags=8049 metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 nd6 options=21 re0: flags=8843 metric 0 mtu 1500 options=8209b ether 00:21:85:15:33:7d inet 192.168.1.101 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::221:85ff:fe15:337d%re0 prefixlen 64 scopeid 0x1 nd6 options=29 media: Ethernet autoselect (100baseTX ) status: active Starting devd. add net default: gateway 192.168.1.1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/kde4/lib /usr/local/lib/compat /usr/local/lib/compat/pkg /usr/local/lib/gcc46 /usr/local/lib/mysql /usr/local/lib/nss /usr/local/lib/pth /usr/local/lib/qt4 /usr/local/lib/virtualbox a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout Creating and/or trimming log files. Starting syslogd. savecore: couldn't find media and/or sector size of /var/crash: Inappropriate ioctl for device May 17 11:02:08 x4 savecore: couldn't find media and/or sector size of /var/crash: Inappropriate ioctl for device No core dumps found. Additional ABI support: linux. Starting rpcbind. NFS access cache time=60 lock order reversal: 1st 0xe381e778 bufwait (bufwait) @ kern/vfs_bio.c:3070 2nd 0xc7d8b800 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:284 KDB: stack backtrace: db_trace_self_wrapper(c10d8de0,c10e2818,c1c,c0b3a9d0,c8100c10,...) at db_trace_self_wrapper+0x2d/frame 0xf5c6b798 kdb_backtrace(c10dc30b,c7d8b800,c110e50d,c7586268,c110e152,...) at kdb_backtrace+0x30/frame 0xf5c6b800 witness_checkorder(c7d8b800,9,c110e149,11c,0,...) at witness_checkorder+0xbaa/frame 0xf5c6b850 _sx_xlock(c7d8b800,0,c110e149,11c,c80f7000,...) at _sx_xlock+0x77/frame 0xf5c6b880 ufsdirhash_add(c7d84b54,f5c6b978,5770,f5c6b900,f5c6b904,...) at ufsdirhash_add+0x4a/frame 0xf5c6b8b0 ufs_direnter(c8100b40,c8360000,f5c6b978,f5c6bb64,e3822110,...) at ufs_direnter+0x63b/frame 0xf5c6b930 ufs_mkdir(f5c6bc00,c1133b33,c1107e16,c8100b40,f5c6bbfc,...) at ufs_mkdir+0x884/frame 0xf5c6bad0 VOP_MKDIR_APV(c1237ed0,f5c6bc00,f5c6bb64,f5c6bb90,28803020,...) at VOP_MKDIR_APV+0x124/frame 0xf5c6bb00 kern_mkdirat(c7deac00,ffffff9c,28803020,0,1c0) at kern_mkdirat+0x1fc/frame 0xf5c6bc24 sys_mkdir(c7deac00,f5c6bcc8,14,c10d6fcd,7b3,...) at sys_mkdir+0x31/frame 0xf5c6bc40 syscall(f5c6bd08) at syscall+0x2ed/frame 0xf5c6bcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5c6bcfc --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28174f57, esp = 0xbfbfd92c, ebp = 0xbfbfddd0 --- Clearing /tmp (X related). Starting nfsuserd. Starting mountd. Starting nfsd. Recovering vi editor sessions:. Updating motd:. Starting ntpd. Removing stale Samba tdb files: ....... done Starting nmbd. Starting smbd. Starting winbindd. Starting smartd. Starting default moused. Expensive timeout(9) function: 0xc08f5870(0xc14166e8) 0.013216624 s Configuring syscons: keymap blanktime. Starting sshd. Starting cron. Local package initialization: backuplock order reversal: 1st 0xc853d5d4 ufs (ufs) @ kern/vfs_lookup.c:517 2nd 0xe3838378 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:262 3rd 0xc870d19c ufs (ufs) @ kern/vfs_subr.c:2109 KDB: stack backtrace: db_trace_self_wrapper(c10d8de0,0,0,0,0,...) at db_trace_self_wrapper+0x2d/frame 0xf5d6a2b0 kdb_backtrace(c10dc324,c870d19c,c1lock order reversal: 0 1st 0xc7d6473c ufs (ufs) @ kern/vfs_mount.c:1236 c 2nd 0xc7e2a034 devfs (devfs) @ ufs/ffs/ffs_softdep.c:1866 4KDB: stack backtrace: 10a,c7d5b8_6t2r0a0c,ec_1s0eel5f7_3wcr,a.p.p.e)r (act1 0d8de0,0k,d3b3_,bca1c0kct4r1a0cae,+607xd3,0./.f.r)a maet 0xf5d6dab3_1t8r ace_self_wrapperw+i0txn2eds/sf_rcahmecek o0rxdfe5rd(9c4887300d 19c,9,c10e5733,8k3ddb,_cb8a7c0kdt2r0a8c,e.(.c.1)0 dact3 0b,c7e2aw0i3t4n,ecs1s0_ccfhee2c5k,ocr7d5e8r6+006x0b,aca1/1f0raam8e3 70,x.f.5.d)6 aa3t6 8 kdb_backtrace+0_x_3l0o/cfkrmagmre_ a0rxgfs5d(9c4887908d 19c,80100,c870d2w08i,t0n,e0s,s._.c.h)e cakt order(c7_e_2lao0c3k4m,g9r,_ca1r1g0sa+802xe8,1764/af,rca7mee2 a00xaf05,d.6.a.4)4 8a t witnessf_fcsh_elcokcokr(dfe5rd+60ax4bda0a,/cf7r5a8m2e3 700x,fc57d5974f81ee80 ,c7584798,c757f_1_el0o,c.k.m.g)r _aatr gs(c7e2af0f3s4_,l8o0c4k0+00,xca7de/2far0aam0e, 00,x0f,50d,6ca141808a 82e,74a) at VOP_L_O_ClKo1c_kAmPgV(rc_1a2r3g7se+d00x,8f156d/6far4adm0e, c01x0fe52d891489,c188 0,c124c730,...) voapt_ stdlock(VfO5Pd_9L4OaC4K01,_6A3P2V,+40,xf151d29/4far3acm,ec 00ax9fe54d967a,4.b.8. ) at vop__vsnt_dlloocckk(+c08x7503d/1f6r8a,m8e0 100x0f,5c1d09e4597f383 ,83d,c10e49a9,...V)O Pa_tL OCK1_APV_(vcn1_1lfodc6k3+40,xfc5ad/9f4raa4m0e ,04x,ff55dd69a44af86 8,c124c730,...) vagte t(c870d16V8O,P8_0L1O0C0K,1c_7A9P4Va+c00x01,1527/,f0r,a.m.e. )0 xaft5 d94a28 vget+0x77/frame _0vxnf_5ldo6cak5(3c07 e2a000,80400,c11v0fas_8h2aes,h7_4gae,tf(5cd79d46a0ab8b,0.,.1.5)e daetb ,80000,c_7v9n4_alco0c0k,+f05xdc6aa/6f1r0a,m.e. .0)x fa5td 94a68 vfs_hash_get+0xffs/offrtadmeep _0fxlfu5sdh6wao5r5kcl ist(c80f32ec,f5df9f4sa_fvcg,ect8f7(1c973d0600,b6b20d,,105,e.d.e.b,)8 0a0t0 0,f5d6sao6f1t0d,e1p,_.f.l.u)s hawto rklist+0xf6fes/_fvrgaemtef +00xxf457d/9f4raaam8e 0xf5d6a5b8 ffs_sync(c80f32esco,f1t,dfe5pd_9s4ybn6c8_,b5u0ff(,cc815b3ade5ca104,,e.3.8.3)8 3a2t0 ,1,108,0f,f.s._.s)y nact+ 0x2e5/frsaomfet d0expf_5sdy9n4cb_3b0u f+0x977/frame 0xdfo5udn6mao6u2n0t (c80f32ec,808000f0f,sc_8s7y1n9c3v0n0od,e4(9c38,5c30da5fab04,315,,0.,.c.7)5 8a2t4 40,c13d2d4obu4n,m.o.u.n)t a+t0 x43c/frafmfes_ s0yxnfc5vdn9o4dbe9+00 x2a3/frame 0xf5d6syas6_7u8n mount(c8719300,ff5fds9_4tcrcu8n,cfa5tde9(4cc885c3,dc50aa08,3240c00,,0c,1828903,bc970d,2.b.a.0)0 ,a.t. .) at sys_unmfofusn_tt+r0uxn3ccadt/ef+r0axm6ec e0/xffr5adm9e4 c04x0f 5d6a830 syscall(f5d94d08u)f sa_td irenter(scy8s5c3adl5la+00,xc28e7d0/df1r6a8m,ef 50dx6fa58df904,cff5cd 6abcc,0,.X.i.n)t 0axt8 0_syscalulf(s)_ daitr enter+0xX7iendt/0fxr8a0m_es y0sxcfa5ldl6+a08xb201 /frame 0xf5d94cfucf s-_-m-a kseyisncoadlel( f(52d26abb8,f,5 dF6raebecBcS)D aEtL F32, sysu_fusn_mmoaukneti)n,o deei+p0 x=5 308x/2f8r0acm1ea 00bx,f 5eds6pa a=2 40 xbfbfd6a4, ebp =u f0sx_bcfrbeaftde7(7f05 d-6-a-b 10,c1133521,c1107e16,c7d60bc0,2,...) at ufs_create+0x2f/frame 0xf5d6aa38 VOP_CREATE_APV(c1237ed0,f5d6ab10,f5d6abcc,f5d6aaa0,f5d6aad0,...) at VOP_CREATE_APV+0x124/frame 0xf5d6aa68 vn_open_cred(f5d6ab80,f5d6abfc,180,0,c7d2ba00,c870e118) at vn_open_cred+0x2cf/frame 0xf5d6ab38 vn_open(f5d6ab80,f5d6abfc,180,c870e118,c794ac00,...) at vn_open+0x3b/frame 0xf5d6ab58 lock order reversal: k 1st 0xc810073c syncer (syncer) @ kern/vfs_subr.c:1728 e 2nd 0xc853f19c ufs (ufs) @ kern/vfs_subr.c:2109 rKDB: stack backtrace: n_opendab_tt(rca7c9e4_asce0lf0_,wfrfafpfpfefr9(cc,1208dc88d0e306,0c,101,2ae0224,71,8306)9 ,abtf 9ffbc7,c7k9e4r8n3_0o0p,e.n.a.t)+ 0axt1 ec/frdabm_et r0axcfe5_ds6ealcf2_0w rapper+0x2d/frame s0yxse_3obpfecn9(3c07 94ac00,f5d6acc8,kcd0be_eb5a0cck0t,rca7c1ef(acc1900d,c03,0.b.,.c)8 5a3tf 19c,c10c4s1y0sa_,ocp7e5n8+602x0308,/cf1r0aem5e7 30cx,f.5.d.6)a ca4t0 kdb_backtrace+0xs3y0s/cfarlalm(ef 50dx6ea3db0f8c)9 9a8t syscallw+i0txn2eesds/_fcrhaemcek o0rxdfe5rd(6ca8c5f3cf 19c,9,c10Xei5n7t303x,8803_ds,ycs8c5a3lfl2(0)8 ,a.t. .) at Xint0xw8i0t_nseysssc_aclhle+c0kxo2r1d/efrr+a0mxeb a0ax/ff5rda6maec f0cx e-3-b-f cs9yes8c all (5, _F_rleoecBkSmDg rE_LaFr3g2s,( cs8y5s3_fo1p9ecn,)8,0 5e0i0p, c=8 503xf2280481,701,e07,,. .e.s)p a=t 0xbfbfa0_9_cl,o cekbmpg r=_ a0rxgbsf+b0fxa811268/ f-r-a- me 0xe3bfcac8 ffs_lock(e3bfcb50,0,c75845f8,246,c75845f8,...) at ffs_lock+0xad/frame 0xe3bfcb08 VOP_LOCK1_APV(c1237ed0,e3bfcb50,246,c13fe85c,c124c730,...) at VOP_LOCK1_APV+0x112/frame 0xe3bfcb38 _vn_lock(c853f168,80500,c10e5733,83d,c13fe86c,...) at _vn_lock+0xca/frame 0xe3bfcb78 vget(c853f168,80500,c7948300,1285,c80f3010,...) at vget+0x77/frame 0xe3bfcbb0 vfs_msync(c80f3000,2,2,e44,c80f3000,...) at vfs_msync+0xa0/frame 0xe3bfcbf8 sync_fsync(e3bfcca8,c1133a0e,80400,c10e5733,6c0,...) at sync_fsync+0x12e/frame 0xe3bfcc20 VOP_FSYNC_APV(c12181c4,e3bfcca8,c10e5733,6c0,0,...) at VOP_FSYNC_APV+0x124/frame 0xe3bfcc50 sched_sync(0,e3bfcd08,c10ce550,3d7,0,...) at sched_sync+0x38f/frame 0xe3bfcccc fork_exit(c0b5e360,0,e3bfcd08) at fork_exit+0x7f/frame 0xe3bfccf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3bfccf4 --- trap 0, eip = 0, esp = 0xe3bfcd40, ebp = 0 --- ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 50067 files, 69906266 used, 48407715 free (3499 frags, 6050527 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** fsck -y /tmp watchdogd. Starting inetd. Fri May 17 11:04:19 CEST 2013 FreeBSD/i386 (x4.osted.lan) (console) login: May 17 11:05:21 x4 su: pho to root on /dev/pts/0 Stopping inetd. Waiting for PIDS: 810. Shutting down local packages:. Stopping cron. Waiting for PIDS: 735. Stopping sshd. Waiting for PIDS: 724. Stopping moused. Waiting for PIDS: 692, 692. Stopping smartd. Waiting for PIDS: 684. Stopping winbindd. Waiting for PIDS: 670. Stopping smbd. Waiting for PIDS: 668. Stopping nmbd. Waiting for PIDS: 666. Stopping ntpd. Waiting for PIDS: 642. Stopping nfsd. Waiting for PIDS: 583 587. Stopping mountd. Waiting for PIDS: 577. Stopping nfsuserd. Waiting for PIDS: 566. Stopping rpcbind. Waiting for PIDS: 538. Stopping watchdogd. Waiting for PIDS: 786. Stopping devd. Waiting for PIDS: 420. Writing entropy file:. . Terminated May 17 11:05:27 x4 syslogd: exiting on signal 15 Enter full pathname of shell or RETURN for /bin/sh: # cd ~pho/stress2/wip # ./wire_no_page.sh panic: vm_page_set_invalid: page 0xc3dfe8c0 is busy cpuid = 0 KDB: enter: panic [ thread pid 1509 tid 100206 ] Stopped at kdb_enter+0x3d: movl $0,kdb_why db> run pho db:0:pho> bt Tracing pid 1509 tid 100206 td 0xc90bb600 kdb_enter(c10d49d3,c10d49d3,c1112b69,f5e876c0,f5e87690,...) at kdb_enter+0x3d/frame 0xf5e87650 vpanic(c12865c2,100,c1112b69,f5e876c0,f5e876c0,...) at vpanic+0x143/frame 0xf5e87690 kassert_panic(c1112b69,c3dfe8c0,c11122b8,a1e,c3dfe8c0,...) at kassert_panic+0xea/frame 0xf5e876b4 vm_page_set_invalid(c3dfe8c0,0,1000,c7599984,e38e4b34,...) at vm_page_set_invalid+0xcd/frame 0xf5e876d8 brelse(e38e4b34,5c,c10e2cce,1140,c90bb600,...) at brelse+0x41d/frame 0xf5e87734 bufwrite(e38e4b34,1000,b30,f5e878f8,c76bfe80,...) at bufwrite+0x295/frame 0xf5e87774 ffs_write(f5e87918,c113388b,c0ad9c9b,c1293b90,4,...) at ffs_write+0x4e7/frame 0xf5e87808 VOP_WRITE_APV(c1237ed0,f5e87918,488,b30,b30,...) at VOP_WRITE_APV+0x19f/frame 0xf5e878c8 vnode_pager_generic_putpages(c8a5b870,f5e87a3c,1000,3,f5e879f8) at vnode_pager_generic_putpages+0x259/frame 0xf5e8793c vop_stdputpages(f5e879a8,c113417a,c1114498,426,c8a5b8c4,...) at vop_stdputpages+0x32/frame 0xf5e87960 VOP_PUTPAGES_APV(c1237ed0,f5e879a8,426,c10c410a,c124d870,...) at VOP_PUTPAGES_APV+0x12a/frame 0xf5e87990 vnode_pager_putpages(c903b2e4,f5e87a3c,1,3,f5e879f8) at vnode_pager_putpages+0x94/frame 0xf5e879dc vm_pageout_flush(f5e87a3c,1,3,0,f5e87a90,f5e87aec) at vm_pageout_flush+0x131/frame 0xf5e87a1c vm_object_page_collect_flush(3,3,f5e87af0,f5e87aec,c120eb38,...) at vm_object_page_collect_flush+0x1d6/frame 0xf5e87aa4 vm_object_page_clean(c903b2e4,0,0,2000,0,...) at vm_object_page_clean+0x189/frame 0xf5e87b00 vm_object_sync(c903b2e4,0,0,2000,1,...) at vm_object_sync+0x20a/frame 0xf5e87b68 vm_map_sync(c7e3c0e8,281ec000,281ee000,1,1,...) at vm_map_sync+0x570/frame 0xf5e87c1c sys_msync(c90bb600,f5e87cc8,c112cd3f,d6,28049ee9,...) at sys_msync+0x69/frame 0xf5e87c40 syscall(f5e87d08) at syscall+0x2ed/frame 0xf5e87cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5e87cfc --- syscall (65, FreeBSD ELF32, sys_msync), eip = 0x280e191b, esp = 0xbfbfdcac, ebp = 0xbfbfdcc8 --- db:0:bt> show allpcpu Current CPU: 0 cpuid = 0 dynamic pcpu = 0x5d3300 curthread = 0xc90bb600: pid 1509 "wire_no_page" curpcb = 0xf5e87d60 fpcurthread = none idlethread = 0xc76c5900: tid 100003 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: cpuid = 1 dynamic pcpu = 0x5fa8300 curthread = 0xc76c5600: pid 11 "idle: cpu1" curpcb = 0xc720dd60 fpcurthread = none idlethread = 0xc76c5600: tid 100004 "idle: cpu1" APIC ID = 1 currentldt = 0x50 spin locks held: cpuid = 2 dynamic pcpu = 0x5fab300 curthread = 0xc903fc00: pid 1462 "wire_no_page" curpcb = 0xf5e30d60 fpcurthread = none idlethread = 0xc76c5300: tid 100005 "idle: cpu2" APIC ID = 2 currentldt = 0x50 spin locks held: cpuid = 3 dynamic pcpu = 0x5fae300 curthread = 0xc9074900: pid 1484 "wire_no_page" curpcb = 0xf5e09d60 fpcurthread = none idlethread = 0xc76c5000: tid 100006 "idle: cpu3" APIC ID = 3 currentldt = 0x50 spin locks held: db:0:allpcpu> show alllocks Process 1509 (wire_no_page) thread 0xc90bb600 (100206) exclusive rw vm object (vm object) r = 0 (0xc903b2e4) locked @ kern/vfs_bio.c:1537 exclusive lockmgr bufwait (bufwait) r = 0 (0xe38e4b8c) locked @ kern/vfs_bio.c:2294 exclusive lockmgr ufs (ufs) r = 0 (0xc8a5b8a4) locked @ vm/vm_object.c:1019 Process 1496 (wire_no_page) thread 0xc871a600 (100154) shared lockmgr ufs (ufs) r = 0 (0xc8e945d4) locked @ kern/vfs_subr.c:2109 Process 1484 (wire_no_page) thread 0xc9074900 (100164) shared sx vm map (user) (vm map (user)) r = 0 (0xc76c9870) locked @ vm/vm_map.c:3764 Process 1476 (wire_no_page) thread 0xc903f000 (100181) shared lockmgr ufs (ufs) r = 0 (0xc8e945d4) locked @ kern/vfs_subr.c:2109 Process 1462 (wire_no_page) thread 0xc903fc00 (100177) exclusive rw vm object (vm object) r = 0 (0xc8574784) locked @ vm/vm_object.c:568 db:0:alllocks> show lockedvnods Locked vnodes 0xc8e945a0: tag ufs, type VDIR usecount 2, writecount 0, refcount 39 mountedhere 0 flags (VV_ROOT|VI_ACTIVE) v_object 0xc8eb940c ref 0 pages 140 lock type ufs: SHARED (count 2) #0 0xc0a95b53 at __lockmgr_args+0xf63 #1 0xc0d1b0ad at ffs_lock+0xad #2 0xc0f047a2 at VOP_LOCK1_APV+0x112 #3 0xc0b6998a at _vn_lock+0xca #4 0xc0b57447 at vget+0x77 #5 0xc0b4adff at vfs_hash_get+0xff #6 0xc0d15917 at ffs_vgetf+0x47 #7 0xc0d158be at ffs_vget+0x2e #8 0xc0d2a273 at ufs_root+0x43 #9 0xc0b4c8ac at lookup+0x76c #10 0xc0b4bd5d at namei+0x47d #11 0xc0b690bd at vn_open_cred+0x24d #12 0xc0b68e6b at vn_open+0x3b #13 0xc0b616ec at kern_openat+0x1ec #14 0xc0b61478 at sys_open+0x38 #15 0xc0ede7dd at syscall+0x2ed #16 0xc0ec6d51 at Xint0x80_syscall+0x21 ino 2, on dev label/tmp 0xc8a5b870: tag ufs, type VREG usecount 61, writecount 27, refcount 78 mountedhere 0 flags (VI_ACTIVE) v_object 0xc903b2e4 ref 35 pages 2 lock type ufs: EXCL by thread 0xc90bb600 (pid 1509, wire_no_page, tid 100206) with exclusive waiters pending #0 0xc0a95c9f at __lockmgr_args+0x10af #1 0xc0d1b0ad at ffs_lock+0xad #2 0xc0f047a2 at VOP_LOCK1_APV+0x112 #3 0xc0b6998a at _vn_lock+0xca #4 0xc0d51ccb at vm_object_sync+0x16b #5 0xc0d49d00 at vm_map_sync+0x570 #6 0xc0d4ef89 at sys_msync+0x69 #7 0xc0ede7dd at syscall+0x2ed #8 0xc0ec6d51 at Xint0x80_syscall+0x21 ino 994, on dev label/tmp db:0:lockedvnods> show mount 0xc7d5fbb0 /dev/ad4s1a on / (ufs) 0xc7d60000 devfs on /dev (devfs) 0xc80f35d8 /dev/ad4s1f on /home (ufs) 0xc80f3000 /dev/ad4s1d on /usr (ufs) 0xc7d60bb0 /dev/ad4s1e on /var (ufs) 0xc7d608c4 procfs on /proc (procfs) 0xc8ec8000 /dev/label/tmp on /tmp (ufs) More info: show mount db:0:mount> ps pid ppid pgrp uid state wmesg wchan cmd 1538 1462 1141 0 DE+ ufs 0xc8a5b8a4 wire_no_page 1537 1 1141 0 DE+ ufs 0xc8a5b8a4 wire_no_page 1535 1 1141 0 DE+ ufs 0xc8a5b8a4 wire_no_page 1534 1 1141 0 DE+ ufs 0xc8a5b8a4 wire_no_page 1533 1 1141 0 DE+ ufs 0xc8a5b8a4 wire_no_page 1532 1 1141 0 DE+ ufs 0xc8a5b8a4 wire_no_page 1531 1 1141 0 DE+ ufs 0xc8a5b8a4 wire_no_page 1530 1 1141 0 DE+ ufs 0xc8a5b8a4 wire_no_page 1529 1 1141 0 DE+ ufs 0xc8a5b8a4 wire_no_page 1510 1141 1141 0 D+ ufs 0xc8a5b8a4 wire_no_page 1509 1141 1141 0 R+ CPU 0 wire_no_page 1507 1141 1141 0 D+ ufs 0xc8a5b8a4 wire_no_page 1505 1141 1141 0 D+ ufs 0xc8a5b8a4 wire_no_page 1503 1141 1141 0 D+ ufs 0xc8a5b8a4 wire_no_page 1501 1141 1141 0 D+ ufs 0xc8a5b8a4 wire_no_page 1496 1141 1141 0 D+ ufs 0xc8a5b8a4 wire_no_page 1495 1141 1141 0 D+ ufs 0xc8a5b8a4 wire_no_page 1494 1141 1141 0 D+ ufs 0xc8a5b8a4 wire_no_page 1491 1141 1141 0 D+ ufs 0xc8a5b8a4 wire_no_page 1488 1141 1141 0 D+ ufs 0xc8a5b8a4 wire_no_page 1484 1141 1141 0 R+ CPU 3 wire_no_page 1479 1141 1141 0 D+ ufs 0xc8a5b8a4 wire_no_page 1476 1141 1141 0 D+ ufs 0xc8a5b8a4 wire_no_page 1475 1141 1141 0 D+ ufs 0xc8a5b8a4 wire_no_page 1466 1141 1141 0 D+ ufs 0xc8a5b8a4 wire_no_page 1464 1141 1141 0 D+ ufs 0xc8a5b8a4 wire_no_page 1462 1141 1141 0 RE+ CPU 2 wire_no_page 1459 1141 1141 0 D+ ufs 0xc8a5b8a4 wire_no_page 1458 1141 1141 0 D+ ufs 0xc8a5b8a4 wire_no_page 1141 1140 1141 0 S+ pause 0xc7e3191c sh 1140 1 1140 0 Ss+ wait 0xc87125d0 sh 17 0 0 0 DL sdflush 0xc1402af4 [softdepflush] 16 0 0 0 DL syncer 0xc13fe8a4 [syncer] 9 0 0 0 DL vlruwt 0xc7c6fba0 [vnlru] 8 0 0 0 DL psleep 0xc13fe61c [bufdaemon] 7 0 0 0 DL pgzero 0xc14033f4 [pagezero] 6 0 0 0 DL psleep 0xc1402fbc [vmdaemon] 5 0 0 0 DL psleep 0xc1456e84 [pagedaemon] 4 0 0 0 DL ccb_scan 0xc1252a24 [xpt_thrd] 3 0 0 0 DL waiting_ 0xc1452b8c [sctp_iterator] 2 0 0 0 DL - 0xc789303c [fdc0] 15 0 0 0 DL (threaded) [usb] 100058 D - 0xc78f7d34 [usbus5] 100057 D - 0xc78f7d04 [usbus5] 100056 D - 0xc78f7cd4 [usbus5] 100055 D - 0xc78f7ca4 [usbus5] 100053 D - 0xc78ebb5c [usbus4] 100052 D - 0xc78ebb2c [usbus4] 100051 D - 0xc78ebafc [usbus4] 100050 D - 0xc78ebacc [usbus4] 100049 D - 0xc78e6b5c [usbus3] 100048 D - 0xc78e6b2c [usbus3] 100047 D - 0xc78e6afc [usbus3] 100046 D - 0xc78e6acc [usbus3] 100045 D - 0xc78deb5c [usbus2] 100044 D - 0xc78deb2c [usbus2] 100043 D - 0xc78deafc [usbus2] 100042 D - 0xc78deacc [usbus2] 100040 D - 0xc78d9b5c [usbus1] 100039 D - 0xc78d9b2c [usbus1] 100038 D - 0xc78d9afc [usbus1] 100037 D - 0xc78d9acc [usbus1] 100035 D - 0xc78cdb5c [usbus0] 100034 D - 0xc78cdb2c [usbus0] 100033 D - 0xc78cdafc [usbus0] 100032 D - 0xc78cdacc [usbus0] 14 0 0 0 DL - 0xc126d5dc [yarrow] 13 0 0 0 DL (threaded) [geom] 100015 D - 0xc144ebe8 [g_down] 100014 D - 0xc144ebe4 [g_up] 100013 D - 0xc144ebe0 [g_event] 12 0 0 0 WL (threaded) [intr] 100063 I [irq12: psm0] 100062 I [irq1: atkbd0] 100060 I [swi0: uart] 100059 I [irq14: ata0] 100054 I [irq19: ehci0] 100041 I [irq18: ohci2 ohci4] 100036 I [irq17: ohci1 ohci3] 100031 I [irq16: hdac1 ohci0] 100030 I [irq22: ahci0] 100029 I [irq257: re0] 100028 I [irq256: hdac0] 100027 I [swi6: task queue] 100025 I [swi2: cambio] 100020 I [swi5: fast taskq] 100018 I [swi6: Giant taskq] 100012 I [swi4: clock] 100011 I [swi4: clock] 100010 I [swi4: clock] 100009 I [swi4: clock] 100008 I [swi1: netisr 0] 100007 I [swi3: vm] 11 0 0 0 RL (threaded) [idle] 100006 CanRun [idle: cpu3] 100005 CanRun [idle: cpu2] 100004 Run CPU 1 [idle: cpu1] 100003 CanRun [idle: cpu0] 1 0 1 0 SLs wait 0xc76c3ba0 [init] 10 0 0 0 DL audit_wo 0xc1455938 [audit] 0 0 0 0 DLs (threaded) [kernel] 100065 D - 0xc1286853 [deadlkres] 100064 D - 0xc7867d80 [mca taskq] 100026 D - 0xc75a8600 [ffs_trim taskq] 100024 D - 0xc75a8980 [acpi_task_2] 100023 D - 0xc75a8980 [acpi_task_1] 100022 D - 0xc75a8980 [acpi_task_0] 100021 D - 0xc75a8a00 [kqueue taskq] 100019 D - 0xc75a8b00 [thread taskq] 100016 D - 0xc75a8c80 [firmware taskq] 100000 D sched 0xc144ec64 [swapper] db:0:ps> allt Tracing command wire_no_page pid 1538 tid 100132 td 0xc8718600 sched_switch(c8718600,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5daf7a4 mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xf5daf7d8 sleepq_switch(c8718600,0,c10daa3d,26b,60,...) at sleepq_switch+0x1b9/frame 0xf5daf80c sleepq_wait(c8a5b8a4,60,c10c410a,4,0,...) at sleepq_wait+0x48/frame 0xf5daf830 sleeplk(c8a5b910,c10c410a,60,33,0,...) at sleeplk+0x126/frame 0xf5daf878 __lockmgr_args(c8a5b8a4,80400,c8a5b910,0,0,...) at __lockmgr_args+0xa44/frame 0xf5daf958 ffs_lock(f5daf9e0,5e8,c8ec8010,0,c8718600,...) at ffs_lock+0xad/frame 0xf5daf998 VOP_LOCK1_APV(c1237ed0,f5daf9e0,c0a9dfaa,c8ec8000,c124c730,...) at VOP_LOCK1_APV+0x112/frame 0xf5daf9c8 _vn_lock(c8a5b870,80400,c10e70fc,16d,80400,...) at _vn_lock+0xca/frame 0xf5dafa08 vn_close(c8a5b870,3,c76bfe80,c8718600,c8718600,...) at vn_close+0xeb/frame 0xf5dafa50 vn_closefile(c7da8c40,c8718600,0,0,c8718600,...) at vn_closefile+0x53/frame 0xf5dafab4 _fdrop(c7da8c40,c8718600,2dafaec,8049000,c7582780,...) at _fdrop+0x2c/frame 0xf5dafad0 closef(c7da8c40,c8718600,7d3,7d1,c120d620,...) at closef+0x203/frame 0xf5dafb38 fdescfree(c8718600,0,c10ce2cc,11a,f5dafc40,...) at fdescfree+0x439/frame 0xf5dafbd0 exit1(c8718600,0,f5dafcfc,c0ede7dd,c8718600,...) at exit1+0x5a8/frame 0xf5dafc30 sys_sys_exit(c8718600,f5dafcc8,c0ee50c0,c71fdc90,0,...) at sys_sys_exit+0x1d/frame 0xf5dafc40 syscall(f5dafd08) at syscall+0x2ed/frame 0xf5dafcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5dafcfc --- syscall (1, FreeBSD ELF32, sys_sys_exit), eip = 0x281448a3, esp = 0xbfbfdccc, ebp = 0xbfbfddc4 --- Tracing command wire_no_page pid 1537 tid 100180 td 0xc903f300 sched_switch(c903f300,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5e397a4 mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xf5e397d8 sleepq_switch(c903f300,0,c10daa3d,26b,60,...) at sleepq_switch+0x1b9/frame 0xf5e3980c sleepq_wait(c8a5b8a4,60,c10c410a,4,0,...) at sleepq_wait+0x48/frame 0xf5e39830 sleeplk(c8a5b910,c10c410a,60,33,0,...) at sleeplk+0x126/frame 0xf5e39878 __lockmgr_args(c8a5b8a4,80400,c8a5b910,0,0,...) at __lockmgr_args+0xa44/frame 0xf5e39958 ffs_lock(f5e399e0,5e8,c8ec8010,0,c903f300,...) at ffs_lock+0xad/frame 0xf5e39998 VOP_LOCK1_APV(c1237ed0,f5e399e0,c0a9dfaa,c8ec8000,c124c730,...) at VOP_LOCK1_APV+0x112/frame 0xf5e399c8 _vn_lock(c8a5b870,80400,c10e70fc,16d,80400,...) at _vn_lock+0xca/frame 0xf5e39a08 vn_close(c8a5b870,3,c76bfe80,c903f300,c903f300,...) at vn_close+0xeb/frame 0xf5e39a50 vn_closefile(c7e00b98,c903f300,0,0,c903f300,...) at vn_closefile+0x53/frame 0xf5e39ab4 _fdrop(c7e00b98,c903f300,2e39aec,8049000,c7582780,...) at _fdrop+0x2c/frame 0xf5e39ad0 closef(c7e00b98,c903f300,7d3,7d1,c120d620,...) at closef+0x203/frame 0xf5e39b38 fdescfree(c903f300,0,c10ce2cc,11a,f5e39c40,...) at fdescfree+0x439/frame 0xf5e39bd0 exit1(c903f300,0,f5e39cfc,c0ede7dd,c903f300,...) at exit1+0x5a8/frame 0xf5e39c30 sys_sys_exit(c903f300,f5e39cc8,14,c10d6fcd,7b3,...) at sys_sys_exit+0x1d/frame 0xf5e39c40 syscall(f5e39d08) at syscall+0x2ed/frame 0xf5e39cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5e39cfc --- syscall (1, FreeBSD ELF32, sys_sys_exit), eip = 0x281448a3, esp = 0xbfbfdccc, ebp = 0xbfbfddc4 --- Tracing command wire_no_page pid 1535 tid 100214 td 0xc90b9c00 sched_switch(c90b9c00,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5e9f7a4 mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xf5e9f7d8 sleepq_switch(c90b9c00,0,c10daa3d,26b,60,...) at sleepq_switch+0x1b9/frame 0xf5e9f80c sleepq_wait(c8a5b8a4,60,c10c410a,4,0,...) at sleepq_wait+0x48/frame 0xf5e9f830 sleeplk(c8a5b910,c10c410a,60,33,0,...) at sleeplk+0x126/frame 0xf5e9f878 __lockmgr_args(c8a5b8a4,80400,c8a5b910,0,0,...) at __lockmgr_args+0xa44/frame 0xf5e9f958 ffs_lock(f5e9f9e0,5e8,c8ec8010,0,c90b9c00,...) at ffs_lock+0xad/frame 0xf5e9f998 VOP_LOCK1_APV(c1237ed0,f5e9f9e0,c0a9dfaa,c8ec8000,c124c730,...) at VOP_LOCK1_APV+0x112/frame 0xf5e9f9c8 _vn_lock(c8a5b870,80400,c10e70fc,16d,80400,...) at _vn_lock+0xca/frame 0xf5e9fa08 vn_close(c8a5b870,3,c76bfe80,c90b9c00,c90b9c00,...) at vn_close+0xeb/frame 0xf5e9fa50 vn_closefile(c7dd2e38,c90b9c00,0,0,c90b9c00,...) at vn_closefile+0x53/frame 0xf5e9fab4 _fdrop(c7dd2e38,c90b9c00,2e9faec,8049000,c7582780,...) at _fdrop+0x2c/frame 0xf5e9fad0 closef(c7dd2e38,c90b9c00,7d3,7d1,c120d620,...) at closef+0x203/frame 0xf5e9fb38 fdescfree(c90b9c00,0,c10ce2cc,11a,f5e9fc40,...) at fdescfree+0x439/frame 0xf5e9fbd0 exit1(c90b9c00,0,f5e9fcfc,c0ede7dd,c90b9c00,...) at exit1+0x5a8/frame 0xf5e9fc30 sys_sys_exit(c90b9c00,f5e9fcc8,0,0,0,...) at sys_sys_exit+0x1d/frame 0xf5e9fc40 syscall(f5e9fd08) at syscall+0x2ed/frame 0xf5e9fcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5e9fcfc --- syscall (1, FreeBSD ELF32, sys_sys_exit), eip = 0x281448a3, esp = 0xbfbfdccc, ebp = 0xbfbfddc4 --- Tracing command wire_no_page pid 1534 tid 100172 td 0xc9073000 sched_switch(c9073000,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5e217a4 mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xf5e217d8 sleepq_switch(c9073000,0,c10daa3d,26b,60,...) at sleepq_switch+0x1b9/frame 0xf5e2180c sleepq_wait(c8a5b8a4,60,c10c410a,4,0,...) at sleepq_wait+0x48/frame 0xf5e21830 sleeplk(c8a5b910,c10c410a,60,33,0,...) at sleeplk+0x126/frame 0xf5e21878 __lockmgr_args(c8a5b8a4,80400,c8a5b910,0,0,...) at __lockmgr_args+0xa44/frame 0xf5e21958 ffs_lock(f5e219e0,5e8,c8ec8010,0,c9073000,...) at ffs_lock+0xad/frame 0xf5e21998 VOP_LOCK1_APV(c1237ed0,f5e219e0,c0a9dfaa,c8ec8000,c124c730,...) at VOP_LOCK1_APV+0x112/frame 0xf5e219c8 _vn_lock(c8a5b870,80400,c10e70fc,16d,80400,...) at _vn_lock+0xca/frame 0xf5e21a08 vn_close(c8a5b870,3,c76bfe80,c9073000,c9073000,...) at vn_close+0xeb/frame 0xf5e21a50 vn_closefile(c7dd2348,c9073000,0,0,c9073000,...) at vn_closefile+0x53/frame 0xf5e21ab4 _fdrop(c7dd2348,c9073000,2e21aec,8049000,c7582780,...) at _fdrop+0x2c/frame 0xf5e21ad0 closef(c7dd2348,c9073000,7d3,7d1,c120d620,...) at closef+0x203/frame 0xf5e21b38 fdescfree(c9073000,0,c10ce2cc,11a,f5e21c40,...) at fdescfree+0x439/frame 0xf5e21bd0 exit1(c9073000,0,f5e21cfc,c0ede7dd,c9073000,...) at exit1+0x5a8/frame 0xf5e21c30 sys_sys_exit(c9073000,f5e21cc8,0,0,0,...) at sys_sys_exit+0x1d/frame 0xf5e21c40 syscall(f5e21d08) at syscall+0x2ed/frame 0xf5e21cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5e21cfc --- syscall (1, FreeBSD ELF32, sys_sys_exit), eip = 0x281448a3, esp = 0xbfbfdccc, ebp = 0xbfbfddc4 --- Tracing command wire_no_page pid 1533 tid 100187 td 0xc871cc00 sched_switch(c871cc00,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5e4e7a4 mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xf5e4e7d8 sleepq_switch(c871cc00,0,c10daa3d,26b,60,...) at sleepq_switch+0x1b9/frame 0xf5e4e80c sleepq_wait(c8a5b8a4,60,c10c410a,4,0,...) at sleepq_wait+0x48/frame 0xf5e4e830 sleeplk(c8a5b910,c10c410a,60,33,0,...) at sleeplk+0x126/frame 0xf5e4e878 __lockmgr_args(c8a5b8a4,80400,c8a5b910,0,0,...) at __lockmgr_args+0xa44/frame 0xf5e4e958 ffs_lock(f5e4e9e0,5e8,c8ec8010,0,c871cc00,...) at ffs_lock+0xad/frame 0xf5e4e998 VOP_LOCK1_APV(c1237ed0,f5e4e9e0,c0a9dfaa,c8ec8000,c124c730,...) at VOP_LOCK1_APV+0x112/frame 0xf5e4e9c8 _vn_lock(c8a5b870,80400,c10e70fc,16d,80400,...) at _vn_lock+0xca/frame 0xf5e4ea08 vn_close(c8a5b870,3,c76bfe80,c871cc00,c871cc00,...) at vn_close+0xeb/frame 0xf5e4ea50 vn_closefile(c7dd2070,c871cc00,0,0,c871cc00,...) at vn_closefile+0x53/frame 0xf5e4eab4 _fdrop(c7dd2070,c871cc00,2e4eaec,8049000,c7582780,...) at _fdrop+0x2c/frame 0xf5e4ead0 closef(c7dd2070,c871cc00,7d3,7d1,c120d620,...) at closef+0x203/frame 0xf5e4eb38 fdescfree(c871cc00,0,c10ce2cc,11a,f5e4ec40,...) at fdescfree+0x439/frame 0xf5e4ebd0 exit1(c871cc00,0,f5e4ecfc,c0ede7dd,c871cc00,...) at exit1+0x5a8/frame 0xf5e4ec30 sys_sys_exit(c871cc00,f5e4ecc8,14,c10d6fcd,7b3,...) at sys_sys_exit+0x1d/frame 0xf5e4ec40 syscall(f5e4ed08) at syscall+0x2ed/frame 0xf5e4ecfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5e4ecfc --- syscall (1, FreeBSD ELF32, sys_sys_exit), eip = 0x281448a3, esp = 0xbfbfdccc, ebp = 0xbfbfddc4 --- Tracing command wire_no_page pid 1532 tid 100213 td 0xc90ba000 sched_switch(c90ba000,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5e9c7a4 mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xf5e9c7d8 sleepq_switch(c90ba000,0,c10daa3d,26b,60,...) at sleepq_switch+0x1b9/frame 0xf5e9c80c sleepq_wait(c8a5b8a4,60,c10c410a,4,0,...) at sleepq_wait+0x48/frame 0xf5e9c830 sleeplk(c8a5b910,c10c410a,60,33,0,...) at sleeplk+0x126/frame 0xf5e9c878 __lockmgr_args(c8a5b8a4,80400,c8a5b910,0,0,...) at __lockmgr_args+0xa44/frame 0xf5e9c958 ffs_lock(f5e9c9e0,5e8,c8ec8010,0,c90ba000,...) at ffs_lock+0xad/frame 0xf5e9c998 VOP_LOCK1_APV(c1237ed0,f5e9c9e0,c0a9dfaa,c8ec8000,c124c730,...) at VOP_LOCK1_APV+0x112/frame 0xf5e9c9c8 _vn_lock(c8a5b870,80400,c10e70fc,16d,80400,...) at _vn_lock+0xca/frame 0xf5e9ca08 vn_close(c8a5b870,3,c76bfe80,c90ba000,c90ba000,...) at vn_close+0xeb/frame 0xf5e9ca50 vn_closefile(c7da99d8,c90ba000,0,0,c90ba000,...) at vn_closefile+0x53/frame 0xf5e9cab4 _fdrop(c7da99d8,c90ba000,2e9caec,8049000,c7582780,...) at _fdrop+0x2c/frame 0xf5e9cad0 closef(c7da99d8,c90ba000,7d3,7d1,c120d620,...) at closef+0x203/frame 0xf5e9cb38 fdescfree(c90ba000,0,c10ce2cc,11a,f5e9cc40,...) at fdescfree+0x439/frame 0xf5e9cbd0 exit1(c90ba000,0,f5e9ccfc,c0ede7dd,c90ba000,...) at exit1+0x5a8/frame 0xf5e9cc30 sys_sys_exit(c90ba000,f5e9ccc8,0,0,0,...) at sys_sys_exit+0x1d/frame 0xf5e9cc40 syscall(f5e9cd08) at syscall+0x2ed/frame 0xf5e9ccfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5e9ccfc --- syscall (1, FreeBSD ELF32, sys_sys_exit), eip = 0x281448a3, esp = 0xbfbfdccc, ebp = 0xbfbfddc4 --- Tracing command wire_no_page pid 1531 tid 100091 td 0xc7dea300 sched_switch(c7dea300,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5c9f7a4 mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xf5c9f7d8 sleepq_switch(c7dea300,0,c10daa3d,26b,60,...) at sleepq_switch+0x1b9/frame 0xf5c9f80c sleepq_wait(c8a5b8a4,60,c10c410a,4,0,...) at sleepq_wait+0x48/frame 0xf5c9f830 sleeplk(c8a5b910,c10c410a,60,33,0,...) at sleeplk+0x126/frame 0xf5c9f878 __lockmgr_args(c8a5b8a4,80400,c8a5b910,0,0,...) at __lockmgr_args+0xa44/frame 0xf5c9f958 ffs_lock(f5c9f9e0,5e8,c8ec8010,0,c7dea300,...) at ffs_lock+0xad/frame 0xf5c9f998 VOP_LOCK1_APV(c1237ed0,f5c9f9e0,c0a9dfaa,c8ec8000,c124c730,...) at VOP_LOCK1_APV+0x112/frame 0xf5c9f9c8 _vn_lock(c8a5b870,80400,c10e70fc,16d,80400,...) at _vn_lock+0xca/frame 0xf5c9fa08 vn_close(c8a5b870,3,c76bfe80,c7dea300,c7dea300,...) at vn_close+0xeb/frame 0xf5c9fa50 vn_closefile(c7dbfc08,c7dea300,0,0,c7dea300,...) at vn_closefile+0x53/frame 0xf5c9fab4 _fdrop(c7dbfc08,c7dea300,2c9faec,8049000,c7582780,...) at _fdrop+0x2c/frame 0xf5c9fad0 closef(c7dbfc08,c7dea300,7d3,7d1,c120d620,...) at closef+0x203/frame 0xf5c9fb38 fdescfree(c7dea300,0,c10ce2cc,11a,f5c9fc40,...) at fdescfree+0x439/frame 0xf5c9fbd0 exit1(c7dea300,0,f5c9fcfc,c0ede7dd,c7dea300,...) at exit1+0x5a8/frame 0xf5c9fc30 sys_sys_exit(c7dea300,f5c9fcc8,c0ee50c0,c71fac90,0,...) at sys_sys_exit+0x1d/frame 0xf5c9fc40 syscall(f5c9fd08) at syscall+0x2ed/frame 0xf5c9fcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5c9fcfc --- syscall (1, FreeBSD ELF32, sys_sys_exit), eip = 0x281448a3, esp = 0xbfbfdccc, ebp = 0xbfbfddc4 --- Tracing command wire_no_page pid 1530 tid 100212 td 0xc90ba300 sched_switch(c90ba300,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5e997a4 mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xf5e997d8 sleepq_switch(c90ba300,0,c10daa3d,26b,60,...) at sleepq_switch+0x1b9/frame 0xf5e9980c sleepq_wait(c8a5b8a4,60,c10c410a,4,0,...) at sleepq_wait+0x48/frame 0xf5e99830 sleeplk(c8a5b910,c10c410a,60,33,0,...) at sleeplk+0x126/frame 0xf5e99878 __lockmgr_args(c8a5b8a4,80400,c8a5b910,0,0,...) at __lockmgr_args+0xa44/frame 0xf5e99958 ffs_lock(f5e999e0,5e8,c8ec8010,0,c90ba300,...) at ffs_lock+0xad/frame 0xf5e99998 VOP_LOCK1_APV(c1237ed0,f5e999e0,c0a9dfaa,c8ec8000,c124c730,...) at VOP_LOCK1_APV+0x112/frame 0xf5e999c8 _vn_lock(c8a5b870,80400,c10e70fc,16d,80400,...) at _vn_lock+0xca/frame 0xf5e99a08 vn_close(c8a5b870,3,c76bfe80,c90ba300,c90ba300,...) at vn_close+0xeb/frame 0xf5e99a50 vn_closefile(c7dbfc78,c90ba300,0,0,c90ba300,...) at vn_closefile+0x53/frame 0xf5e99ab4 _fdrop(c7dbfc78,c90ba300,2e99aec,8049000,c7582780,...) at _fdrop+0x2c/frame 0xf5e99ad0 closef(c7dbfc78,c90ba300,7d3,7d1,c120d620,...) at closef+0x203/frame 0xf5e99b38 fdescfree(c90ba300,0,c10ce2cc,11a,f5e99c40,...) at fdescfree+0x439/frame 0xf5e99bd0 exit1(c90ba300,0,f5e99cfc,c0ede7dd,c90ba300,...) at exit1+0x5a8/frame 0xf5e99c30 sys_sys_exit(c90ba300,f5e99cc8,280b02ca,280b02da,280b02ea,...) at sys_sys_exit+0x1d/frame 0xf5e99c40 syscall(f5e99d08) at syscall+0x2ed/frame 0xf5e99cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5e99cfc --- syscall (1, FreeBSD ELF32, sys_sys_exit), eip = 0x281448a3, esp = 0xbfbfdccc, ebp = 0xbfbfddc4 --- Tracing command wire_no_page pid 1529 tid 100075 td 0xc7d98c00 sched_switch(c7d98c00,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5c5f7a4 mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xf5c5f7d8 sleepq_switch(c7d98c00,0,c10daa3d,26b,60,...) at sleepq_switch+0x1b9/frame 0xf5c5f80c sleepq_wait(c8a5b8a4,60,c10c410a,4,0,...) at sleepq_wait+0x48/frame 0xf5c5f830 sleeplk(c8a5b910,c10c410a,60,33,0,...) at sleeplk+0x126/frame 0xf5c5f878 __lockmgr_args(c8a5b8a4,80400,c8a5b910,0,0,...) at __lockmgr_args+0xa44/frame 0xf5c5f958 ffs_lock(f5c5f9e0,5e8,c8ec8010,0,c7d98c00,...) at ffs_lock+0xad/frame 0xf5c5f998 VOP_LOCK1_APV(c1237ed0,f5c5f9e0,c0a9dfaa,c8ec8000,c124c730,...) at VOP_LOCK1_APV+0x112/frame 0xf5c5f9c8 _vn_lock(c8a5b870,80400,c10e70fc,16d,80400,...) at _vn_lock+0xca/frame 0xf5c5fa08 vn_close(c8a5b870,3,c76bfe80,c7d98c00,c7d98c00,...) at vn_close+0xeb/frame 0xf5c5fa50 vn_closefile(c7dbf498,c7d98c00,0,0,c7d98c00,...) at vn_closefile+0x53/frame 0xf5c5fab4 _fdrop(c7dbf498,c7d98c00,2c5fb0c,8049000,c7582780,...) at _fdrop+0x2c/frame 0xf5c5fad0 closef(c7dbf498,c7d98c00,7d3,7d1,c120d620,...) at closef+0x203/frame 0xf5c5fb38 fdescfree(c7d98c00,0,c10ce2cc,11a,f5c5fc40,...) at fdescfree+0x439/frame 0xf5c5fbd0 exit1(c7d98c00,0,f5c5fcfc,c0ede7dd,c7d98c00,...) at exit1+0x5a8/frame 0xf5c5fc30 sys_sys_exit(c7d98c00,f5c5fcc8,c0ee50c0,c71fac90,0,...) at sys_sys_exit+0x1d/frame 0xf5c5fc40 syscall(f5c5fd08) at syscall+0x2ed/frame 0xf5c5fcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5c5fcfc --- syscall (1, FreeBSD ELF32, sys_sys_exit), eip = 0x281448a3, esp = 0xbfbfdccc, ebp = 0xbfbfddc4 --- Tracing command wire_no_page pid 1510 tid 100207 td 0xc90bb300 sched_switch(c90bb300,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5e8a82c mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xf5e8a860 sleepq_switch(c90bb300,0,c10daa3d,26b,60,...) at sleepq_switch+0x1b9/frame 0xf5e8a894 sleepq_wait(c8a5b8a4,60,c10c410a,4,0,...) at sleepq_wait+0x48/frame 0xf5e8a8b8 sleeplk(c8a5b910,c10c410a,60,33,0,...) at sleeplk+0x126/frame 0xf5e8a900 __lockmgr_args(c8a5b8a4,80100,c8a5b910,0,0,...) at __lockmgr_args+0xa44/frame 0xf5e8a9e0 ffs_lock(f5e8aa68,c1110d3e,5c9,0,c367f000,...) at ffs_lock+0xad/frame 0xf5e8aa20 VOP_LOCK1_APV(c1237ed0,f5e8aa68,c0a9dfaa,c8a5b910,c124c730,...) at VOP_LOCK1_APV+0x112/frame 0xf5e8aa50 _vn_lock(c8a5b870,80100,c10e5733,83d,c1111b08,...) at _vn_lock+0xca/frame 0xf5e8aa90 vget(c8a5b870,80000,c90bb300,5ee,c7dce740,...) at vget+0x77/frame 0xf5e8aac8 vm_mmap(c7dce740,f5e8ac18,2000,3,7,...) at vm_mmap+0x395/frame 0xf5e8abc0 sys_mmap(c90bb300,f5e8acc8,c0ee50c0,c71fdc90,0,...) at sys_mmap+0x2f6/frame 0xf5e8ac40 syscall(f5e8ad08) at syscall+0x2ed/frame 0xf5e8acfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5e8acfc --- syscall (477, FreeBSD ELF32, sys_mmap), eip = 0x2810bf6f, esp = 0xbfbfdccc, ebp = 0xbfbfddc4 --- Tracing command wire_no_page pid 1509 tid 100206 td 0xc90bb600 kdb_enter(c10d49d3,c10d49d3,c1112b69,f5e876c0,f5e87690,...) at kdb_enter+0x3d/frame 0xf5e87650 vpanic(c12865c2,100,c1112b69,f5e876c0,f5e876c0,...) at vpanic+0x143/frame 0xf5e87690 kassert_panic(c1112b69,c3dfe8c0,c11122b8,a1e,c3dfe8c0,...) at kassert_panic+0xea/frame 0xf5e876b4 vm_page_set_invalid(c3dfe8c0,0,1000,c7599984,e38e4b34,...) at vm_page_set_invalid+0xcd/frame 0xf5e876d8 brelse(e38e4b34,5c,c10e2cce,1140,c90bb600,...) at brelse+0x41d/frame 0xf5e87734 bufwrite(e38e4b34,1000,b30,f5e878f8,c76bfe80,...) at bufwrite+0x295/frame 0xf5e87774 ffs_write(f5e87918,c113388b,c0ad9c9b,c1293b90,4,...) at ffs_write+0x4e7/frame 0xf5e87808 VOP_WRITE_APV(c1237ed0,f5e87918,488,b30,b30,...) at VOP_WRITE_APV+0x19f/frame 0xf5e878c8 vnode_pager_generic_putpages(c8a5b870,f5e87a3c,1000,3,f5e879f8) at vnode_pager_generic_putpages+0x259/frame 0xf5e8793c vop_stdputpages(f5e879a8,c113417a,c1114498,426,c8a5b8c4,...) at vop_stdputpages+0x32/frame 0xf5e87960 VOP_PUTPAGES_APV(c1237ed0,f5e879a8,426,c10c410a,c124d870,...) at VOP_PUTPAGES_APV+0x12a/frame 0xf5e87990 vnode_pager_putpages(c903b2e4,f5e87a3c,1,3,f5e879f8) at vnode_pager_putpages+0x94/frame 0xf5e879dc vm_pageout_flush(f5e87a3c,1,3,0,f5e87a90,f5e87aec) at vm_pageout_flush+0x131/frame 0xf5e87a1c vm_object_page_collect_flush(3,3,f5e87af0,f5e87aec,c120eb38,...) at vm_object_page_collect_flush+0x1d6/frame 0xf5e87aa4 vm_object_page_clean(c903b2e4,0,0,2000,0,...) at vm_object_page_clean+0x189/frame 0xf5e87b00 vm_object_sync(c903b2e4,0,0,2000,1,...) at vm_object_sync+0x20a/frame 0xf5e87b68 vm_map_sync(c7e3c0e8,281ec000,281ee000,1,1,...) at vm_map_sync+0x570/frame 0xf5e87c1c sys_msync(c90bb600,f5e87cc8,c112cd3f,d6,28049ee9,...) at sys_msync+0x69/frame 0xf5e87c40 syscall(f5e87d08) at syscall+0x2ed/frame 0xf5e87cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5e87cfc --- syscall (65, FreeBSD ELF32, sys_msync), eip = 0x280e191b, esp = 0xbfbfdcac, ebp = 0xbfbfdcc8 --- Tracing command wire_no_page pid 1507 tid 100205 td 0xc90bb900 sched_switch(c90bb900,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5e8482c mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xf5e84860 sleepq_switch(c90bb900,0,c10daa3d,26b,60,...) at sleepq_switch+0x1b9/frame 0xf5e84894 sleepq_wait(c8a5b8a4,60,c10c410a,4,0,...) at sleepq_wait+0x48/frame 0xf5e848b8 sleeplk(c8a5b910,c10c410a,60,33,0,...) at sleeplk+0x126/frame 0xf5e84900 __lockmgr_args(c8a5b8a4,80100,c8a5b910,0,0,...) at __lockmgr_args+0xa44/frame 0xf5e849e0 ffs_lock(f5e84a68,20b,c110f020,1d1,f5e84a64,...) at ffs_lock+0xad/frame 0xf5e84a20 VOP_LOCK1_APV(c1237ed0,f5e84a68,c0a9dfaa,c8a5b910,c124c730,...) at VOP_LOCK1_APV+0x112/frame 0xf5e84a50 _vn_lock(c8a5b870,80100,c10e5733,83d,c1111b08,...) at _vn_lock+0xca/frame 0xf5e84a90 vget(c8a5b870,80000,c90bb900,5ee,4000,...) at vget+0x77/frame 0xf5e84ac8 vm_mmap(c8125cb0,f5e84c18,2000,3,7,...) at vm_mmap+0x395/frame 0xf5e84bc0 sys_mmap(c90bb900,f5e84cc8,c112cd3f,d6,0,...) at sys_mmap+0x2f6/frame 0xf5e84c40 syscall(f5e84d08) at syscall+0x2ed/frame 0xf5e84cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5e84cfc --- syscall (477, FreeBSD ELF32, sys_mmap), eip = 0x2810bf6f, esp = 0xbfbfdccc, ebp = 0xbfbfddc4 --- Tracing command wire_no_page pid 1505 tid 100203 td 0xc906f000 sched_switch(c906f000,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5e7e82c mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xf5e7e860 sleepq_switch(c906f000,0,c10daa3d,26b,60,...) at sleepq_switch+0x1b9/frame 0xf5e7e894 sleepq_wait(c8a5b8a4,60,c10c410a,4,0,...) at sleepq_wait+0x48/frame 0xf5e7e8b8 sleeplk(c8a5b910,c10c410a,60,33,0,...) at sleeplk+0x126/frame 0xf5e7e900 __lockmgr_args(c8a5b8a4,80100,c8a5b910,0,0,...) at __lockmgr_args+0xa44/frame 0xf5e7e9e0 ffs_lock(f5e7ea68,0,200000,c8a5b870,c367f000,...) at ffs_lock+0xad/frame 0xf5e7ea20 VOP_LOCK1_APV(c1237ed0,f5e7ea68,c0a9dfaa,c8a5b910,c124c730,...) at VOP_LOCK1_APV+0x112/frame 0xf5e7ea50 _vn_lock(c8a5b870,80100,c10e5733,83d,c1111b08,...) at _vn_lock+0xca/frame 0xf5e7ea90 vget(c8a5b870,80000,c906f000,5ee,c7d8f2b8,...) at vget+0x77/frame 0xf5e7eac8 vm_mmap(c7d8f2b8,f5e7ec18,2000,3,7,...) at vm_mmap+0x395/frame 0xf5e7ebc0 sys_mmap(c906f000,f5e7ecc8,c112cd3f,d6,104f68d,...) at sys_mmap+0x2f6/frame 0xf5e7ec40 syscall(f5e7ed08) at syscall+0x2ed/frame 0xf5e7ecfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5e7ecfc --- syscall (477, FreeBSD ELF32, sys_mmap), eip = 0x2810bf6f, esp = 0xbfbfdccc, ebp = 0xbfbfddc4 --- Tracing command wire_no_page pid 1503 tid 100201 td 0xc906f600 sched_switch(c906f600,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5e7889c mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xf5e788d0 sleepq_switch(c906f600,0,c10daa3d,26b,60,...) at sleepq_switch+0x1b9/frame 0xf5e78904 sleepq_wait(c8a5b8a4,60,c10c410a,4,0,...) at sleepq_wait+0x48/frame 0xf5e78928 sleeplk(c8a5b910,c10c410a,60,33,0,...) at sleeplk+0x126/frame 0xf5e78970 __lockmgr_args(c8a5b8a4,80400,c8a5b910,0,0,...) at __lockmgr_args+0xa44/frame 0xf5e78a50 ffs_lock(f5e78ad8,c8ec8000,8,c10e70fc,60c,...) at ffs_lock+0xad/frame 0xf5e78a90 VOP_LOCK1_APV(c1237ed0,f5e78ad8,c8ec8010,0,c124c730,...) at VOP_LOCK1_APV+0x112/frame 0xf5e78ac0 _vn_lock(c8a5b870,80400,c1111bf5,3fb,c0d43c0b,...) at _vn_lock+0xca/frame 0xf5e78b00 vm_object_sync(c903b2e4,0,0,2000,1,...) at vm_object_sync+0x16b/frame 0xf5e78b68 vm_map_sync(c7d8f570,281ec000,281ee000,1,1,...) at vm_map_sync+0x570/frame 0xf5e78c1c sys_msync(c906f600,f5e78cc8,c0ee50c0,c71fdc90,0,...) at sys_msync+0x69/frame 0xf5e78c40 syscall(f5e78d08) at syscall+0x2ed/frame 0xf5e78cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5e78cfc --- syscall (65, FreeBSD ELF32, sys_msync), eip = 0x280e191b, esp = 0xbfbfdcac, ebp = 0xbfbfdcc8 --- Tracing command wire_no_page pid 1501 tid 100199 td 0xc906fc00 sched_switch(c906fc00,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5e7282c mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xf5e72860 sleepq_switch(c906fc00,0,c10daa3d,26b,60,...) at sleepq_switch+0x1b9/frame 0xf5e72894 sleepq_wait(c8a5b8a4,60,c10c410a,4,0,...) at sleepq_wait+0x48/frame 0xf5e728b8 sleeplk(c8a5b910,c10c410a,60,33,0,...) at sleeplk+0x126/frame 0xf5e72900 __lockmgr_args(c8a5b8a4,80100,c8a5b910,0,0,...) at __lockmgr_args+0xa44/frame 0xf5e729e0 ffs_lock(f5e72a68,0,200000,c8a5b870,c367f000,...) at ffs_lock+0xad/frame 0xf5e72a20 VOP_LOCK1_APV(c1237ed0,f5e72a68,c0a9dfaa,c8a5b910,c124c730,...) at VOP_LOCK1_APV+0x112/frame 0xf5e72a50 _vn_lock(c8a5b870,80100,c10e5733,83d,c1111b08,...) at _vn_lock+0xca/frame 0xf5e72a90 vget(c8a5b870,80000,c906fc00,5ee,c81259f8,...) at vget+0x77/frame 0xf5e72ac8 vm_mmap(c81259f8,f5e72c18,2000,3,7,...) at vm_mmap+0x395/frame 0xf5e72bc0 sys_mmap(c906fc00,f5e72cc8,c0ee50c0,c7200c90,0,...) at sys_mmap+0x2f6/frame 0xf5e72c40 syscall(f5e72d08) at syscall+0x2ed/frame 0xf5e72cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5e72cfc --- syscall (477, FreeBSD ELF32, sys_mmap), eip = 0x2810bf6f, esp = 0xbfbfdccc, ebp = 0xbfbfddc4 --- Tracing command wire_no_page pid 1496 tid 100154 td 0xc871a600 sched_switch(c871a600,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5deb604 mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xf5deb638 sleepq_switch(c871a600,0,c10daa3d,26b,60,...) at sleepq_switch+0x1b9/frame 0xf5deb66c sleepq_wait(c8a5b8a4,60,c10c410a,4,0,...) at sleepq_wait+0x48/frame 0xf5deb690 sleeplk(c8a5b910,c10c410a,60,33,0,...) at sleeplk+0x126/frame 0xf5deb6d8 __lockmgr_args(c8a5b8a4,80100,c8a5b910,0,0,...) at __lockmgr_args+0xa44/frame 0xf5deb7b8 ffs_lock(f5deb840,f5deb810,c757f1e0,c7580b10,c757f1e0,...) at ffs_lock+0xad/frame 0xf5deb7f8 VOP_LOCK1_APV(c1237ed0,f5deb840,0,f5deb8b4,c124c730,...) at VOP_LOCK1_APV+0x112/frame 0xf5deb828 _vn_lock(c8a5b870,80100,c10e5733,83d,2a1,...) at _vn_lock+0xca/frame 0xf5deb868 vget(c8a5b870,80100,c871a600,29d,c1249bf8,...) at vget+0x77/frame 0xf5deb8a0 cache_lookup(c8e945a0,f5debbb8,f5debbcc,0,0,...) at cache_lookup+0x5a8/frame 0xf5deb900 vfs_cache_lookup(f5deb9b8,c113346b,c1107edd,c10e4b77,c1249bf8,...) at vfs_cache_lookup+0xbc/frame 0xf5deb948 VOP_LOOKUP_APV(c1237ed0,f5deb9b8,f5debbcc,322,f5deb9c0,...) at VOP_LOOKUP_APV+0x124/frame 0xf5deb978 lookup(f5debb80,c10e4b77,107,ce,c0d43bf3,...) at lookup+0x528/frame 0xf5deb9e0 namei(f5debb80,0,0,c8546a68,b,...) at namei+0x47d/frame 0xf5deba68 vn_open_cred(f5debb80,f5debbfc,0,0,c76bfe80,c7dd2380) at vn_open_cred+0x24d/frame 0xf5debb38 vn_open(f5debb80,f5debbfc,0,c7dd2380,c871a600,...) at vn_open+0x3b/frame 0xf5debb58 kern_openat(c871a600,ffffff9c,bfbfdf06,0,2,0) at kern_openat+0x1ec/frame 0xf5debc20 sys_open(c871a600,f5debcc8,c0ee50c0,c1825c90,0,...) at sys_open+0x38/frame 0xf5debc40 syscall(f5debd08) at syscall+0x2ed/frame 0xf5debcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5debcfc --- syscall (5, FreeBSD ELF32, sys_open), eip = 0x281a01e7, esp = 0xbfbfdca8, ebp = 0xbfbfdcc8 --- Tracing command wire_no_page pid 1495 tid 100156 td 0xc871a000 sched_switch(c871a000,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5df182c mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xf5df1860 sleepq_switch(c871a000,0,c10daa3d,26b,60,...) at sleepq_switch+0x1b9/frame 0xf5df1894 sleepq_wait(c8a5b8a4,60,c10c410a,4,0,...) at sleepq_wait+0x48/frame 0xf5df18b8 sleeplk(c8a5b910,c10c410a,60,33,0,...) at sleeplk+0x126/frame 0xf5df1900 __lockmgr_args(c8a5b8a4,80100,c8a5b910,0,0,...) at __lockmgr_args+0xa44/frame 0xf5df19e0 ffs_lock(f5df1a68,0,200000,c8a5b870,c367f000,...) at ffs_lock+0xad/frame 0xf5df1a20 VOP_LOCK1_APV(c1237ed0,f5df1a68,c0a9dfaa,c8a5b910,c124c730,...) at VOP_LOCK1_APV+0x112/frame 0xf5df1a50 _vn_lock(c8a5b870,80100,c10e5733,83d,c1111b08,...) at _vn_lock+0xca/frame 0xf5df1a90 vget(c8a5b870,80000,c871a000,5ee,c8226658,...) at vget+0x77/frame 0xf5df1ac8 vm_mmap(c8226658,f5df1c18,2000,3,7,...) at vm_mmap+0x395/frame 0xf5df1bc0 sys_mmap(c871a000,f5df1cc8,c112cd3f,d6,0,...) at sys_mmap+0x2f6/frame 0xf5df1c40 syscall(f5df1d08) at syscall+0x2ed/frame 0xf5df1cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5df1cfc --- syscall (477, FreeBSD ELF32, sys_mmap), eip = 0x2810bf6f, esp = 0xbfbfdccc, ebp = 0xbfbfddc4 --- Tracing command wire_no_page pid 1494 tid 100195 td 0xc9096c00 sched_switch(c9096c00,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5e6689c mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xf5e668d0 sleepq_switch(c9096c00,0,c10daa3d,26b,60,...) at sleepq_switch+0x1b9/frame 0xf5e66904 sleepq_wait(c8a5b8a4,60,c10c410a,4,0,...) at sleepq_wait+0x48/frame 0xf5e66928 sleeplk(c8a5b910,c10c410a,60,33,0,...) at sleeplk+0x126/frame 0xf5e66970 __lockmgr_args(c8a5b8a4,80400,c8a5b910,0,0,...) at __lockmgr_args+0xa44/frame 0xf5e66a50 ffs_lock(f5e66ad8,c8ec8000,8,c10e70fc,60c,...) at ffs_lock+0xad/frame 0xf5e66a90 VOP_LOCK1_APV(c1237ed0,f5e66ad8,c8ec8010,0,c124c730,...) at VOP_LOCK1_APV+0x112/frame 0xf5e66ac0 _vn_lock(c8a5b870,80400,c1111bf5,3fb,c0d43c0b,...) at _vn_lock+0xca/frame 0xf5e66b00 vm_object_sync(c903b2e4,0,0,2000,1,...) at vm_object_sync+0x16b/frame 0xf5e66b68 vm_map_sync(c8226488,281ec000,281ee000,1,1,...) at vm_map_sync+0x570/frame 0xf5e66c1c sys_msync(c9096c00,f5e66cc8,c112cd3f,d6,0,...) at sys_msync+0x69/frame 0xf5e66c40 syscall(f5e66d08) at syscall+0x2ed/frame 0xf5e66cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5e66cfc --- syscall (65, FreeBSD ELF32, sys_msync), eip = 0x280e191b, esp = 0xbfbfdcac, ebp = 0xbfbfdcc8 --- Tracing command wire_no_page pid 1491 tid 100080 td 0xc7d98000 sched_switch(c7d98000,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5c7382c mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xf5c73860 sleepq_switch(c7d98000,0,c10daa3d,26b,60,...) at sleepq_switch+0x1b9/frame 0xf5c73894 sleepq_wait(c8a5b8a4,60,c10c410a,4,0,...) at sleepq_wait+0x48/frame 0xf5c738b8 sleeplk(c8a5b910,c10c410a,60,33,0,...) at sleeplk+0x126/frame 0xf5c73900 __lockmgr_args(c8a5b8a4,80100,c8a5b910,0,0,...) at __lockmgr_args+0xa44/frame 0xf5c739e0 ffs_lock(f5c73a68,20b,c110f020,1d1,f5c73a64,...) at ffs_lock+0xad/frame 0xf5c73a20 VOP_LOCK1_APV(c1237ed0,f5c73a68,c0a9dfaa,c8a5b910,c124c730,...) at VOP_LOCK1_APV+0x112/frame 0xf5c73a50 _vn_lock(c8a5b870,80100,c10e5733,83d,c1111b08,...) at _vn_lock+0xca/frame 0xf5c73a90 vget(c8a5b870,80000,c7d98000,5ee,4000,...) at vget+0x77/frame 0xf5c73ac8 vm_mmap(c76c9bc8,f5c73c18,2000,3,7,...) at vm_mmap+0x395/frame 0xf5c73bc0 sys_mmap(c7d98000,f5c73cc8,c112cd3f,d6,f5c73cc4,...) at sys_mmap+0x2f6/frame 0xf5c73c40 syscall(f5c73d08) at syscall+0x2ed/frame 0xf5c73cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5c73cfc --- syscall (477, FreeBSD ELF32, sys_mmap), eip = 0x2810bf6f, esp = 0xbfbfdccc, ebp = 0xbfbfddc4 --- Tracing command wire_no_page pid 1488 tid 100105 td 0xc7de9600 sched_switch(c7de9600,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5ce082c mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xf5ce0860 sleepq_switch(c7de9600,0,c10daa3d,26b,60,...) at sleepq_switch+0x1b9/frame 0xf5ce0894 sleepq_wait(c8a5b8a4,60,c10c410a,4,0,...) at sleepq_wait+0x48/frame 0xf5ce08b8 sleeplk(c8a5b910,c10c410a,60,33,0,...) at sleeplk+0x126/frame 0xf5ce0900 __lockmgr_args(c8a5b8a4,80100,c8a5b910,0,0,...) at __lockmgr_args+0xa44/frame 0xf5ce09e0 ffs_lock(f5ce0a68,20b,c110f020,1d1,f5ce0a64,...) at ffs_lock+0xad/frame 0xf5ce0a20 VOP_LOCK1_APV(c1237ed0,f5ce0a68,c0a9dfaa,c8a5b910,c124c730,...) at VOP_LOCK1_APV+0x112/frame 0xf5ce0a50 _vn_lock(c8a5b870,80100,c10e5733,83d,c1111b08,...) at _vn_lock+0xca/frame 0xf5ce0a90 vget(c8a5b870,80000,c7de9600,5ee,4000,...) at vget+0x77/frame 0xf5ce0ac8 vm_mmap(c82261d0,f5ce0c18,2000,3,7,...) at vm_mmap+0x395/frame 0xf5ce0bc0 sys_mmap(c7de9600,f5ce0cc8,c0ee50c0,c7200c90,0,...) at sys_mmap+0x2f6/frame 0xf5ce0c40 syscall(f5ce0d08) at syscall+0x2ed/frame 0xf5ce0cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5ce0cfc --- syscall (477, FreeBSD ELF32, sys_mmap), eip = 0x2810bf6f, esp = 0xbfbfdccc, ebp = 0xbfbfddc4 --- Tracing command wire_no_page pid 1484 tid 100164 td 0xc9074900 cpustop_handler(3,13,f5e096fc,c0edd3cb,4124b2,...) at cpustop_handler+0x2f/frame 0xf5e095b0 ipi_nmi_handler(4124b2,c71fdc90,2710,3672c,38f7b,...) at ipi_nmi_handler+0x37/frame 0xf5e095c0 trap(f5e09708) at trap+0x3b/frame 0xf5e096fc calltrap() at calltrap+0x6/frame 0xf5e096fc --- trap 0x13, eip = 0xc0ab074a, esp = 0xf5e09748, ebp = 0xf5e097bc --- __rw_wlock_hard(c903b2f4,c9074900,c111059f,395,0,...) at __rw_wlock_hard+0x23a/frame 0xf5e097bc _rw_wlock_cookie(c903b2f4,c111059f,395) at _rw_wlock_cookie+0xdd/frame 0xf5e097ec vm_fault_hold(c76c9828,28067000,0,1,0,...) at vm_fault_hold+0x2339/frame 0xf5e09ad8 vm_fault(c76c9828,28067000,0,1,1,...) at vm_fault+0xdd/frame 0xf5e09b1c vm_fault_wire(c76c9828,28067000,28069000,0,5dba884d,...) at vm_fault_wire+0x63/frame 0xf5e09b5c vm_map_wire(c76c9828,28067000,28069000,1,c76c9828,...) at vm_map_wire+0x4b6/frame 0xf5e09c0c sys_mlock(c9074900,f5e09cc8,c112cd3f,d6,7b3,...) at sys_mlock+0x155/frame 0xf5e09c40 syscall(f5e09d08) at syscall+0x2ed/frame 0xf5e09cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5e09cfc --- syscall (203, FreeBSD ELF32, sys_mlock), eip = 0x280e165b, esp = 0xbfbfdccc, ebp = 0xbfbfddc4 --- Tracing command wire_no_page pid 1479 tid 100149 td 0xc9041300 sched_switch(c9041300,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5de282c mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xf5de2860 sleepq_switch(c9041300,0,c10daa3d,26b,60,...) at sleepq_switch+0x1b9/frame 0xf5de2894 sleepq_wait(c8a5b8a4,60,c10c410a,4,0,...) at sleepq_wait+0x48/frame 0xf5de28b8 sleeplk(c8a5b910,c10c410a,60,33,0,...) at sleeplk+0x126/frame 0xf5de2900 __lockmgr_args(c8a5b8a4,80100,c8a5b910,0,0,...) at __lockmgr_args+0xa44/frame 0xf5de29e0 ffs_lock(f5de2a68,20b,c110f020,1d1,f5de2a64,...) at ffs_lock+0xad/frame 0xf5de2a20 VOP_LOCK1_APV(c1237ed0,f5de2a68,c0a9dfaa,c8a5b910,c124c730,...) at VOP_LOCK1_APV+0x112/frame 0xf5de2a50 _vn_lock(c8a5b870,80100,c10e5733,83d,c1111b08,...) at _vn_lock+0xca/frame 0xf5de2a90 vget(c8a5b870,80000,c9041300,5ee,4000,...) at vget+0x77/frame 0xf5de2ac8 vm_mmap(c8125740,f5de2c18,2000,3,7,...) at vm_mmap+0x395/frame 0xf5de2bc0 sys_mmap(c9041300,f5de2cc8,c112cd3f,d6,0,...) at sys_mmap+0x2f6/frame 0xf5de2c40 syscall(f5de2d08) at syscall+0x2ed/frame 0xf5de2cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5de2cfc --- syscall (477, FreeBSD ELF32, sys_mmap), eip = 0x2810bf6f, esp = 0xbfbfdccc, ebp = 0xbfbfddc4 --- Tracing command wire_no_page pid 1476 tid 100181 td 0xc903f000 sched_switch(c903f000,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5e3c604 mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xf5e3c638 sleepq_switch(c903f000,0,c10daa3d,26b,60,...) at sleepq_switch+0x1b9/frame 0xf5e3c66c sleepq_wait(c8a5b8a4,60,c10c410a,4,0,...) at sleepq_wait+0x48/frame 0xf5e3c690 sleeplk(c8a5b910,c10c410a,60,33,0,...) at sleeplk+0x126/frame 0xf5e3c6d8 __lockmgr_args(c8a5b8a4,80100,c8a5b910,0,0,...) at __lockmgr_args+0xa44/frame 0xf5e3c7b8 ffs_lock(f5e3c840,f5e3c810,c757f1e0,c7580b10,c757f1e0,...) at ffs_lock+0xad/frame 0xf5e3c7f8 VOP_LOCK1_APV(c1237ed0,f5e3c840,0,f5e3c8b4,c124c730,...) at VOP_LOCK1_APV+0x112/frame 0xf5e3c828 _vn_lock(c8a5b870,80100,c10e5733,83d,2a1,...) at _vn_lock+0xca/frame 0xf5e3c868 vget(c8a5b870,80100,c903f000,29d,c1249bf8,...) at vget+0x77/frame 0xf5e3c8a0 cache_lookup(c8e945a0,f5e3cbb8,f5e3cbcc,0,0,...) at cache_lookup+0x5a8/frame 0xf5e3c900 vfs_cache_lookup(f5e3c9b8,c113346b,c1107edd,c10e4b77,c1249bf8,...) at vfs_cache_lookup+0xbc/frame 0xf5e3c948 VOP_LOOKUP_APV(c1237ed0,f5e3c9b8,f5e3cbcc,322,f5e3c9c0,...) at VOP_LOOKUP_APV+0x124/frame 0xf5e3c978 lookup(f5e3cb80,c10e4b77,107,ce,c0d43bf3,...) at lookup+0x528/frame 0xf5e3c9e0 namei(f5e3cb80,0,0,c8a47128,b,...) at namei+0x47d/frame 0xf5e3ca68 vn_open_cred(f5e3cb80,f5e3cbfc,0,0,c76bfe80,c7dd2118) at vn_open_cred+0x24d/frame 0xf5e3cb38 vn_open(f5e3cb80,f5e3cbfc,0,c7dd2118,c903f000,...) at vn_open+0x3b/frame 0xf5e3cb58 kern_openat(c903f000,ffffff9c,bfbfdf06,0,2,0) at kern_openat+0x1ec/frame 0xf5e3cc20 sys_open(c903f000,f5e3ccc8,c112cd3f,d6,0,...) at sys_open+0x38/frame 0xf5e3cc40 syscall(f5e3cd08) at syscall+0x2ed/frame 0xf5e3ccfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5e3ccfc --- syscall (5, FreeBSD ELF32, sys_open), eip = 0x281a01e7, esp = 0xbfbfdca8, ebp = 0xbfbfdcc8 --- Tracing command wire_no_page pid 1475 tid 100129 td 0xc8718c00 sched_switch(c8718c00,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5da582c mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xf5da5860 sleepq_switch(c8718c00,0,c10daa3d,26b,60,...) at sleepq_switch+0x1b9/frame 0xf5da5894 sleepq_wait(c8a5b8a4,60,c10c410a,4,0,...) at sleepq_wait+0x48/frame 0xf5da58b8 sleeplk(c8a5b910,c10c410a,60,33,0,...) at sleeplk+0x126/frame 0xf5da5900 __lockmgr_args(c8a5b8a4,80100,c8a5b910,0,0,...) at __lockmgr_args+0xa44/frame 0xf5da59e0 ffs_lock(f5da5a68,20b,c110f020,1d1,f5da5a64,...) at ffs_lock+0xad/frame 0xf5da5a20 VOP_LOCK1_APV(c1237ed0,f5da5a68,c0a9dfaa,c8a5b910,c124c730,...) at VOP_LOCK1_APV+0x112/frame 0xf5da5a50 _vn_lock(c8a5b870,80100,c10e5733,83d,c1111b08,...) at _vn_lock+0xca/frame 0xf5da5a90 vget(c8a5b870,80000,c8718c00,5ee,4000,...) at vget+0x77/frame 0xf5da5ac8 vm_mmap(c8522000,f5da5c18,2000,3,7,...) at vm_mmap+0x395/frame 0xf5da5bc0 sys_mmap(c8718c00,f5da5cc8,14,c10d6fcd,7b3,...) at sys_mmap+0x2f6/frame 0xf5da5c40 syscall(f5da5d08) at syscall+0x2ed/frame 0xf5da5cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5da5cfc --- syscall (477, FreeBSD ELF32, sys_mmap), eip = 0x2810bf6f, esp = 0xbfbfdccc, ebp = 0xbfbfddc4 --- Tracing command wire_no_page pid 1466 tid 100087 td 0xc78f2c00 sched_switch(c78f2c00,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5c8f82c mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xf5c8f860 sleepq_switch(c78f2c00,0,c10daa3d,26b,60,...) at sleepq_switch+0x1b9/frame 0xf5c8f894 sleepq_wait(c8a5b8a4,60,c10c410a,4,0,...) at sleepq_wait+0x48/frame 0xf5c8f8b8 sleeplk(c8a5b910,c10c410a,60,33,0,...) at sleeplk+0x126/frame 0xf5c8f900 __lockmgr_args(c8a5b8a4,80100,c8a5b910,0,0,...) at __lockmgr_args+0xa44/frame 0xf5c8f9e0 ffs_lock(f5c8fa68,0,200000,c8a5b870,c367f000,...) at ffs_lock+0xad/frame 0xf5c8fa20 VOP_LOCK1_APV(c1237ed0,f5c8fa68,c0a9dfaa,c8a5b910,c124c730,...) at VOP_LOCK1_APV+0x112/frame 0xf5c8fa50 _vn_lock(c8a5b870,80100,c10e5733,83d,c1111b08,...) at _vn_lock+0xca/frame 0xf5c8fa90 vget(c8a5b870,80000,c78f2c00,5ee,c8226ae0,...) at vget+0x77/frame 0xf5c8fac8 vm_mmap(c8226ae0,f5c8fc18,2000,3,7,...) at vm_mmap+0x395/frame 0xf5c8fbc0 sys_mmap(c78f2c00,f5c8fcc8,c112cd3f,d6,7b3,...) at sys_mmap+0x2f6/frame 0xf5c8fc40 syscall(f5c8fd08) at syscall+0x2ed/frame 0xf5c8fcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5c8fcfc --- syscall (477, FreeBSD ELF32, sys_mmap), eip = 0x2810bf6f, esp = 0xbfbfdccc, ebp = 0xbfbfddc4 --- Tracing command wire_no_page pid 1464 tid 100099 td 0xc7e38c00 sched_switch(c7e38c00,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5cc482c mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xf5cc4860 sleepq_switch(c7e38c00,0,c10daa3d,26b,60,...) at sleepq_switch+0x1b9/frame 0xf5cc4894 sleepq_wait(c8a5b8a4,60,c10c410a,4,0,...) at sleepq_wait+0x48/frame 0xf5cc48b8 sleeplk(c8a5b910,c10c410a,60,33,0,...) at sleeplk+0x126/frame 0xf5cc4900 __lockmgr_args(c8a5b8a4,80100,c8a5b910,0,0,...) at __lockmgr_args+0xa44/frame 0xf5cc49e0 ffs_lock(f5cc4a68,20b,c110f020,1d1,f5cc4a64,...) at ffs_lock+0xad/frame 0xf5cc4a20 VOP_LOCK1_APV(c1237ed0,f5cc4a68,c0a9dfaa,c8a5b910,c124c730,...) at VOP_LOCK1_APV+0x112/frame 0xf5cc4a50 _vn_lock(c8a5b870,80100,c10e5733,83d,c1111b08,...) at _vn_lock+0xca/frame 0xf5cc4a90 vget(c8a5b870,80000,c7e38c00,5ee,4000,...) at vget+0x77/frame 0xf5cc4ac8 vm_mmap(c8227740,f5cc4c18,2000,3,7,...) at vm_mmap+0x395/frame 0xf5cc4bc0 sys_mmap(c7e38c00,f5cc4cc8,c0ee50c0,c7200c90,0,...) at sys_mmap+0x2f6/frame 0xf5cc4c40 syscall(f5cc4d08) at syscall+0x2ed/frame 0xf5cc4cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5cc4cfc --- syscall (477, FreeBSD ELF32, sys_mmap), eip = 0x2810bf6f, esp = 0xbfbfdccc, ebp = 0xbfbfddc4 --- Tracing command wire_no_page pid 1462 tid 100177 td 0xc903fc00 cpustop_handler(2,13,f5e309e4,c0edd3cb,c757f998,...) at cpustop_handler+0x2f/frame 0xf5e30898 ipi_nmi_handler(c757f998,c757f380,c757f998,c757f380,c13d1ca8,...) at ipi_nmi_handler+0x37/frame 0xf5e308a8 trap(f5e309f0) at trap+0x3b/frame 0xf5e309e4 calltrap() at calltrap+0x6/frame 0xf5e309e4 --- trap 0x13, eip = 0xc0a9e525, esp = 0xf5e30a30, ebp = 0xf5e30a38 --- __mtx_unlock_flags(c1455ca4,0,c1111bf5,29d,c9026e84,...) at __mtx_unlock_flags+0x105/frame 0xf5e30a38 vm_object_destroy(c9026e74,c1111bf5,705,68d,269,...) at vm_object_destroy+0x87/frame 0xf5e30a54 vm_object_collapse(c8574784,c1111bf5,269,1ca,c1bb6b10,...) at vm_object_collapse+0x2c4/frame 0xf5e30a88 vm_object_deallocate(c810b818,c90231b0,c1bb6b00,0,c8ac9ea0,...) at vm_object_deallocate+0x4f0/frame 0xf5e30adc vm_map_entry_deallocate(c8ac9ea0,0,28069000,0,c903b2e4,...) at vm_map_entry_deallocate+0x36/frame 0xf5e30af8 vm_map_process_deferred(c76c8788,c1110d3e,b32,c76c8740,bfc00000,b32,c1110d3e,c76c8740) at vm_map_process_deferred+0x105/frame 0xf5e30b20 _vm_map_unlock(c76c8740,c1110d3e,b32,b32,bfc00000,...) at _vm_map_unlock+0x83/frame 0xf5e30b4c vm_map_remove(c76c8740,1000,bfc00000,c903fc00,c76c8740,...) at vm_map_remove+0xe6/frame 0xf5e30b88 vmspace_dofree(c76c8740,1,0,1,c90932e8,...) at vmspace_dofree+0x36/frame 0xf5e30ba4 vmspace_exit(c903fc00,0,c10ce2cc,140,f5e30c40,...) at vmspace_exit+0x161/frame 0xf5e30bd0 exit1(c903fc00,0,f5e30cfc,c0ede7dd,c903fc00,...) at exit1+0x644/frame 0xf5e30c30 sys_sys_exit(c903fc00,f5e30cc8,14,c10d6fcd,7b3,...) at sys_sys_exit+0x1d/frame 0xf5e30c40 syscall(f5e30d08) at syscall+0x2ed/frame 0xf5e30cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5e30cfc --- syscall (1, FreeBSD ELF32, sys_sys_exit), eip = 0x281448a3, esp = 0xbfbfddb4, ebp = 0xbfbfddc4 --- Tracing command wire_no_page pid 1459 tid 100197 td 0xc9096600 sched_switch(c9096600,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5e6c82c mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xf5e6c860 sleepq_switch(c9096600,0,c10daa3d,26b,60,...) at sleepq_switch+0x1b9/frame 0xf5e6c894 sleepq_wait(c8a5b8a4,60,c10c410a,4,0,...) at sleepq_wait+0x48/frame 0xf5e6c8b8 sleeplk(c8a5b910,c10c410a,60,33,0,...) at sleeplk+0x126/frame 0xf5e6c900 __lockmgr_args(c8a5b8a4,80100,c8a5b910,0,0,...) at __lockmgr_args+0xa44/frame 0xf5e6c9e0 ffs_lock(f5e6ca68,20b,c110f020,1d1,f5e6ca64,...) at ffs_lock+0xad/frame 0xf5e6ca20 VOP_LOCK1_APV(c1237ed0,f5e6ca68,c0a9dfaa,c8a5b910,c124c730,...) at VOP_LOCK1_APV+0x112/frame 0xf5e6ca50 _vn_lock(c8a5b870,80100,c10e5733,83d,c1111b08,...) at _vn_lock+0xca/frame 0xf5e6ca90 vget(c8a5b870,80000,c9096600,5ee,4000,...) at vget+0x77/frame 0xf5e6cac8 vm_mmap(c76c8828,f5e6cc18,2000,3,7,...) at vm_mmap+0x395/frame 0xf5e6cbc0 sys_mmap(c9096600,f5e6ccc8,c112cd3f,d6,0,...) at sys_mmap+0x2f6/frame 0xf5e6cc40 syscall(f5e6cd08) at syscall+0x2ed/frame 0xf5e6ccfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5e6ccfc --- syscall (477, FreeBSD ELF32, sys_mmap), eip = 0x2810bf6f, esp = 0xbfbfdccc, ebp = 0xbfbfddc4 --- Tracing command wire_no_page pid 1458 tid 100125 td 0xc871c600 sched_switch(c871c600,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5d9782c mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xf5d97860 sleepq_switch(c871c600,0,c10daa3d,26b,60,...) at sleepq_switch+0x1b9/frame 0xf5d97894 sleepq_wait(c8a5b8a4,60,c10c410a,4,0,...) at sleepq_wait+0x48/frame 0xf5d978b8 sleeplk(c8a5b910,c10c410a,60,33,0,...) at sleeplk+0x126/frame 0xf5d97900 __lockmgr_args(c8a5b8a4,80100,c8a5b910,0,0,...) at __lockmgr_args+0xa44/frame 0xf5d979e0 ffs_lock(f5d97a68,0,200000,c8a5b870,c367f000,...) at ffs_lock+0xad/frame 0xf5d97a20 VOP_LOCK1_APV(c1237ed0,f5d97a68,c0a9dfaa,c8a5b910,c124c730,...) at VOP_LOCK1_APV+0x112/frame 0xf5d97a50 _vn_lock(c8a5b870,80100,c10e5733,83d,c1111b08,...) at _vn_lock+0xca/frame 0xf5d97a90 vget(c8a5b870,80000,c871c600,5ee,c8125d98,...) at vget+0x77/frame 0xf5d97ac8 vm_mmap(c8125d98,f5d97c18,2000,3,7,...) at vm_mmap+0x395/frame 0xf5d97bc0 sys_mmap(c871c600,f5d97cc8,c112cd3f,d6,0,...) at sys_mmap+0x2f6/frame 0xf5d97c40 syscall(f5d97d08) at syscall+0x2ed/frame 0xf5d97cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5d97cfc --- syscall (477, FreeBSD ELF32, sys_mmap), eip = 0x2810bf6f, esp = 0xbfbfdccc, ebp = 0xbfbfddc4 --- Tracing command sh pid 1141 tid 100130 td 0xc8718900 sched_switch(c8718900,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5da8aa0 mi_switch(104,0,c10daa3d,1f5,8,...) at mi_switch+0x1c9/frame 0xf5da8ad4 sleepq_switch(c8718900,0,c10daa3d,1ab,f5da8b4c,...) at sleepq_switch+0x1b9/frame 0xf5da8b08 sleepq_catch_signals(0,c10daa3d,162,78,c8718900,...) at sleepq_catch_signals+0x361/frame 0xf5da8b50 sleepq_wait_sig(c7e3191c,74,c0f37104,100,0,...) at sleepq_wait_sig+0x14/frame 0xf5da8b74 _sleep(c7e3191c,c7e3194c,174,c0f37104,0,...) at _sleep+0x35b/frame 0xf5da8bd0 kern_sigsuspend(c8718900,0,0,0,0,...) at kern_sigsuspend+0xfb/frame 0xf5da8c10 sys_sigsuspend(c8718900,f5da8cc8,1,0,c8718900,...) at sys_sigsuspend+0x58/frame 0xf5da8c40 syscall(f5da8d08) at syscall+0x2ed/frame 0xf5da8cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5da8cfc --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x2813a15b, esp = 0xbfbfda54, ebp = 0xbfbfdae4 --- Tracing command sh pid 1140 tid 100119 td 0xc7e33300 sched_switch(c7e33300,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xf5d8398c mi_switch(104,0,c10daa3d,1f5,c10daa3d,...) at mi_switch+0x1c9/frame 0xf5d839c0 sleepq_switch(c7e33300,0,c10daa3d,1ab,f5d83a38,...) at sleepq_switch+0x1b9/frame 0xf5d839f4 sleepq_catch_signals(0,c10daa3d,162,16f8,c7e33300,...) at sleepq_catch_signals+0x361/frame 0xf5d83a3c sleepq_wait_sig(c87125d0,6c,c10ddeb5,100,0,...) at sleepq_wait_sig+0x14/frame 0xf5d83a60 _sleep(c87125d0,c8712664,16c,c10ddeb5,0,...) at _sleep+0x35b/frame 0xf5d83abc kern_wait6(c7e33300,7,0,0,f5d83b4c,...) at kern_wait6+0x45c/frame 0xf5d83b20 sys_wait4(c7e33300,f5d83cc8,c0ee50c0,c71fac90,0,...) at sys_wait4+0x94/frame 0xf5d83c40 syscall(f5d83d08) at syscall+0x2ed/frame 0xf5d83cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf5d83cfc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x28152563, esp = 0xbfbfdbe8, ebp = 0xbfbfdc00 --- Tracing command softdepflush pid 17 tid 100074 td 0xc7948000 sched_switch(c7948000,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3bffba4 mi_switch(104,0,c10daa3d,1f5,101,...) at mi_switch+0x1c9/frame 0xe3bffbd8 sleepq_switch(c7948000,0,c10daa3d,28e,c7948000,...) at sleepq_switch+0x1b9/frame 0xe3bffc0c sleepq_timedwait(c1402af4,54,0,0,0,...) at sleepq_timedwait+0x48/frame 0xe3bffc30 _sleep(c1402af4,c1402a74,54,c110d0d0,fffffed8,...) at _sleep+0x345/frame 0xe3bffc8c softdep_flush(0,e3bffd08,c10ce550,3d7,0,...) at softdep_flush+0x27c/frame 0xe3bffccc fork_exit(c0d14200,0,e3bffd08) at fork_exit+0x7f/frame 0xe3bffcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3bffcf4 --- trap 0, eip = 0, esp = 0xe3bffd40, ebp = 0 --- Tracing command syncer pid 16 tid 100073 td 0xc7948300 sched_switch(c7948300,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3bfcb78 mi_switch(104,0,c10daa3d,1f5,8,...) at mi_switch+0x1c9/frame 0xe3bfcbac sleepq_switch(c7948300,0,c10daa3d,28e,0,...) at sleepq_switch+0x1b9/frame 0xe3bfcbe0 sleepq_timedwait(c13fe8a4,0,e3bfcc3c,0,0,...) at sleepq_timedwait+0x48/frame 0xe3bfcc04 _cv_timedwait_sbt(c13fe8a4,c13fe884,fffffed8,0,0,...) at _cv_timedwait_sbt+0x228/frame 0xe3bfcc50 sched_sync(0,e3bfcd08,c10ce550,3d7,0,...) at sched_sync+0x682/frame 0xe3bfcccc fork_exit(c0b5e360,0,e3bfcd08) at fork_exit+0x7f/frame 0xe3bfccf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3bfccf4 --- trap 0, eip = 0, esp = 0xe3bfcd40, ebp = 0 --- Tracing command vnlru pid 9 tid 100072 td 0xc7948600 sched_switch(c7948600,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3bf9b78 mi_switch(104,0,c10daa3d,1f5,101,...) at mi_switch+0x1c9/frame 0xe3bf9bac sleepq_switch(c7948600,0,c10daa3d,28e,c7948600,...) at sleepq_switch+0x1b9/frame 0xe3bf9be0 sleepq_timedwait(c7c6fba0,60,0,0,0,...) at sleepq_timedwait+0x48/frame 0xe3bf9c04 _sleep(c7c6fba0,c13fe85c,260,c10e6af4,fffffed8,...) at _sleep+0x345/frame 0xe3bf9c60 vnlru_proc(0,e3bf9d08,c10ce550,3d7,0,...) at vnlru_proc+0xcf/frame 0xe3bf9ccc fork_exit(c0b5ec00,0,e3bf9d08) at fork_exit+0x7f/frame 0xe3bf9cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3bf9cf4 --- trap 0, eip = 0, esp = 0xe3bf9d40, ebp = 0 --- Tracing command bufdaemon pid 8 tid 100071 td 0xc7948900 sched_switch(c7948900,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3bf6bb0 mi_switch(104,0,c10daa3d,1f5,101,...) at mi_switch+0x1c9/frame 0xe3bf6be4 sleepq_switch(c7948900,0,c10daa3d,28e,c7948900,...) at sleepq_switch+0x1b9/frame 0xe3bf6c18 sleepq_timedwait(c13fe61c,54,0,0,0,...) at sleepq_timedwait+0x48/frame 0xe3bf6c3c _sleep(c13fe61c,c13fe604,54,c10e3568,fffffed8,...) at _sleep+0x345/frame 0xe3bf6c98 buf_daemon(0,e3bf6d08,c10ce550,3d7,0,...) at buf_daemon+0x8c/frame 0xe3bf6ccc fork_exit(c0b41f00,0,e3bf6d08) at fork_exit+0x7f/frame 0xe3bf6cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3bf6cf4 --- trap 0, eip = 0, esp = 0xe3bf6d40, ebp = 0 --- Tracing command pagezero pid 7 tid 100070 td 0xc7948c00 sched_switch(c7948c00,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3bf3bac mi_switch(104,0,c10daa3d,1f5,101,...) at mi_switch+0x1c9/frame 0xe3bf3be0 sleepq_switch(c7948c00,0,c10daa3d,28e,c7948c00,...) at sleepq_switch+0x1b9/frame 0xe3bf3c14 sleepq_timedwait(c14033f4,0,12b,0,0,...) at sleepq_timedwait+0x48/frame 0xe3bf3c38 _sleep(c14033f4,c1456e00,0,c1114444,fffea520,...) at _sleep+0x345/frame 0xe3bf3c94 vm_pagezero(0,e3bf3d08,c10ce550,3d7,0,...) at vm_pagezero+0xc2/frame 0xe3bf3ccc fork_exit(c0d61880,0,e3bf3d08) at fork_exit+0x7f/frame 0xe3bf3cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3bf3cf4 --- trap 0, eip = 0, esp = 0xe3bf3d40, ebp = 0 --- Tracing command vmdaemon pid 6 tid 100069 td 0xc7949000 sched_switch(c7949000,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3bf0b78 mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xe3bf0bac sleepq_switch(c7949000,0,c10daa3d,26b,c7949000,...) at sleepq_switch+0x1b9/frame 0xe3bf0be0 sleepq_wait(c1402fbc,74,c10e3568,0,0,...) at sleepq_wait+0x48/frame 0xe3bf0c04 _sleep(c1402fbc,c1402fa4,74,c10e3568,0,...) at _sleep+0x37c/frame 0xe3bf0c60 vm_daemon(0,e3bf0d08,c10ce550,3d7,0,...) at vm_daemon+0xac/frame 0xe3bf0ccc fork_exit(c0d59940,0,e3bf0d08) at fork_exit+0x7f/frame 0xe3bf0cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3bf0cf4 --- trap 0, eip = 0, esp = 0xe3bf0d40, ebp = 0 --- Tracing command pagedaemon pid 5 tid 100068 td 0xc7949300 sched_switch(c7949300,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3bedb30 mi_switch(104,0,c10daa3d,1f5,101,...) at mi_switch+0x1c9/frame 0xe3bedb64 sleepq_switch(c7949300,0,c10daa3d,28e,c7949300,...) at sleepq_switch+0x1b9/frame 0xe3bedb98 sleepq_timedwait(c1456e84,54,4,0,0,...) at sleepq_timedwait+0x48/frame 0xe3bedbbc _sleep(c1456e84,c1456e00,54,c10e3568,fffffa38,...) at _sleep+0x345/frame 0xe3bedc18 vm_pageout(0,e3bedd08,c10ce550,3d7,0,...) at vm_pageout+0x290/frame 0xe3bedccc fork_exit(c0d5a160,0,e3bedd08) at fork_exit+0x7f/frame 0xe3bedcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3bedcf4 --- trap 0, eip = 0, esp = 0xe3bedd40, ebp = 0 --- Tracing command xpt_thrd pid 4 tid 100067 td 0xc7949600 sched_switch(c7949600,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3baabb0 mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xe3baabe4 sleepq_switch(c7949600,0,c10daa3d,26b,c7949600,...) at sleepq_switch+0x1b9/frame 0xe3baac18 sleepq_wait(c1252a24,5c,c0f0ce60,0,0,...) at sleepq_wait+0x48/frame 0xe3baac3c _sleep(c1252a24,c1252a74,5c,c0f0ce60,0,...) at _sleep+0x37c/frame 0xe3baac98 xpt_scanner_thread(0,e3baad08,c10ce550,3d7,0,...) at xpt_scanner_thread+0x11c/frame 0xe3baaccc fork_exit(c04b84a0,0,e3baad08) at fork_exit+0x7f/frame 0xe3baacf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3baacf4 --- trap 0, eip = 0, esp = 0xe3baad40, ebp = 0 --- Tracing command sctp_iterator pid 3 tid 100066 td 0xc7949900 sched_switch(c7949900,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3ba7bb8 mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xe3ba7bec sleepq_switch(c7949900,0,c10daa3d,26b,c7949900,...) at sleepq_switch+0x1b9/frame 0xe3ba7c20 sleepq_wait(c1452b8c,0,c10fa095,0,0,...) at sleepq_wait+0x48/frame 0xe3ba7c44 _sleep(c1452b8c,c1452b54,0,c10fa095,0,...) at _sleep+0x37c/frame 0xe3ba7ca0 sctp_iterator_thread(0,e3ba7d08,c10ce550,3d7,0,...) at sctp_iterator_thread+0x7c/frame 0xe3ba7ccc fork_exit(c0bfde70,0,e3ba7d08) at fork_exit+0x7f/frame 0xe3ba7cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3ba7cf4 --- trap 0, eip = 0, esp = 0xe3ba7d40, ebp = 0 --- Tracing command fdc0 pid 2 tid 100061 td 0xc78e5900 sched_switch(c78e5900,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3b97b48 mi_switch(104,0,c10daa3d,1f5,101,...) at mi_switch+0x1c9/frame 0xe3b97b7c sleepq_switch(c78e5900,0,c10daa3d,28e,c78e5900,...) at sleepq_switch+0x1b9/frame 0xe3b97bb0 sleepq_timedwait(c789303c,5c,0,0,0,...) at sleepq_timedwait+0x48/frame 0xe3b97bd4 _sleep(c789303c,c78930f0,5c,c10c081e,fffffed8,...) at _sleep+0x345/frame 0xe3b97c30 fdc_thread(c7893000,e3b97d08,c10ce550,3d7,0,...) at fdc_thread+0x877/frame 0xe3b97ccc fork_exit(c0e6c080,c7893000,e3b97d08) at fork_exit+0x7f/frame 0xe3b97cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b97cf4 --- trap 0, eip = 0, esp = 0xe3b97d40, ebp = 0 --- Tracing command usb pid 15 tid 100058 td 0xc78ef300 sched_switch(c78ef300,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3b73bd4 mi_switch(104,0,c10daa3d,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b73c08 sleepq_switch(c78ef300,0,c10daa3d,26b,c78ef300,...) at sleepq_switch+0x1b9/frame 0xe3b73c3c sleepq_wait(c78f7d34,0,e3b73c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b73c60 _cv_wait(c78f7d34,c78f7e4c,c10b1fb3,77,c78f7d3c,...) at _cv_wait+0x1f9/frame 0xe3b73ca4 usb_process(c78f7d2c,e3b73d08,c10ce550,3d7,0,...) at usb_process+0xe6/frame 0xe3b73ccc fork_exit(c0961af0,c78f7d2c,e3b73d08) at fork_exit+0x7f/frame 0xe3b73cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b73cf4 --- trap 0, eip = 0, esp = 0xe3b73d40, ebp = 0 --- Tracing command usb pid 15 tid 100057 td 0xc78ef600 sched_switch(c78ef600,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3b70bd4 mi_switch(104,0,c10daa3d,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b70c08 sleepq_switch(c78ef600,0,c10daa3d,26b,c78ef600,...) at sleepq_switch+0x1b9/frame 0xe3b70c3c sleepq_wait(c78f7d04,0,e3b70c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b70c60 _cv_wait(c78f7d04,c78f7e4c,c10b1fb3,77,c78f7d0c,...) at _cv_wait+0x1f9/frame 0xe3b70ca4 usb_process(c78f7cfc,e3b70d08,c10ce550,3d7,0,...) at usb_process+0xe6/frame 0xe3b70ccc fork_exit(c0961af0,c78f7cfc,e3b70d08) at fork_exit+0x7f/frame 0xe3b70cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b70cf4 --- trap 0, eip = 0, esp = 0xe3b70d40, ebp = 0 --- Tracing command usb pid 15 tid 100056 td 0xc78ef900 sched_switch(c78ef900,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3b6dbd4 mi_switch(104,0,c10daa3d,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b6dc08 sleepq_switch(c78ef900,0,c10daa3d,26b,c78ef900,...) at sleepq_switch+0x1b9/frame 0xe3b6dc3c sleepq_wait(c78f7cd4,0,e3b6dc90,1,0,...) at sleepq_wait+0x48/frame 0xe3b6dc60 _cv_wait(c78f7cd4,c78f7e4c,c10b1fb3,77,c78f7cdc,...) at _cv_wait+0x1f9/frame 0xe3b6dca4 usb_process(c78f7ccc,e3b6dd08,c10ce550,3d7,0,...) at usb_process+0xe6/frame 0xe3b6dccc fork_exit(c0961af0,c78f7ccc,e3b6dd08) at fork_exit+0x7f/frame 0xe3b6dcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b6dcf4 --- trap 0, eip = 0, esp = 0xe3b6dd40, ebp = 0 --- Tracing command usb pid 15 tid 100055 td 0xc78efc00 sched_switch(c78efc00,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3b6abd4 mi_switch(104,0,c10daa3d,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b6ac08 sleepq_switch(c78efc00,0,c10daa3d,26b,c78efc00,...) at sleepq_switch+0x1b9/frame 0xe3b6ac3c sleepq_wait(c78f7ca4,0,e3b6ac90,1,0,...) at sleepq_wait+0x48/frame 0xe3b6ac60 _cv_wait(c78f7ca4,c78f7e4c,c10b1fb3,77,c78f7cac,...) at _cv_wait+0x1f9/frame 0xe3b6aca4 usb_process(c78f7c9c,e3b6ad08,c10ce550,3d7,0,...) at usb_process+0xe6/frame 0xe3b6accc fork_exit(c0961af0,c78f7c9c,e3b6ad08) at fork_exit+0x7f/frame 0xe3b6acf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b6acf4 --- trap 0, eip = 0, esp = 0xe3b6ad40, ebp = 0 --- Tracing command usb pid 15 tid 100053 td 0xc78f2300 sched_switch(c78f2300,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3b61bd4 mi_switch(104,0,c10daa3d,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b61c08 sleepq_switch(c78f2300,0,c10daa3d,26b,c78f2300,...) at sleepq_switch+0x1b9/frame 0xe3b61c3c sleepq_wait(c78ebb5c,0,e3b61c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b61c60 _cv_wait(c78ebb5c,c78ebc74,c10b1fb3,77,c78ebb64,...) at _cv_wait+0x1f9/frame 0xe3b61ca4 usb_process(c78ebb54,e3b61d08,c10ce550,3d7,0,...) at usb_process+0xe6/frame 0xe3b61ccc fork_exit(c0961af0,c78ebb54,e3b61d08) at fork_exit+0x7f/frame 0xe3b61cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b61cf4 --- trap 0, eip = 0, esp = 0xe3b61d40, ebp = 0 --- Tracing command usb pid 15 tid 100052 td 0xc78f2600 sched_switch(c78f2600,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3b5ebd4 mi_switch(104,0,c10daa3d,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b5ec08 sleepq_switch(c78f2600,0,c10daa3d,26b,c78f2600,...) at sleepq_switch+0x1b9/frame 0xe3b5ec3c sleepq_wait(c78ebb2c,0,e3b5ec90,1,0,...) at sleepq_wait+0x48/frame 0xe3b5ec60 _cv_wait(c78ebb2c,c78ebc74,c10b1fb3,77,c78ebb34,...) at _cv_wait+0x1f9/frame 0xe3b5eca4 usb_process(c78ebb24,e3b5ed08,c10ce550,3d7,0,...) at usb_process+0xe6/frame 0xe3b5eccc fork_exit(c0961af0,c78ebb24,e3b5ed08) at fork_exit+0x7f/frame 0xe3b5ecf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b5ecf4 --- trap 0, eip = 0, esp = 0xe3b5ed40, ebp = 0 --- Tracing command usb pid 15 tid 100051 td 0xc78e2000 sched_switch(c78e2000,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3b5bbd4 mi_switch(104,0,c10daa3d,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b5bc08 sleepq_switch(c78e2000,0,c10daa3d,26b,c78e2000,...) at sleepq_switch+0x1b9/frame 0xe3b5bc3c sleepq_wait(c78ebafc,0,e3b5bc90,1,0,...) at sleepq_wait+0x48/frame 0xe3b5bc60 _cv_wait(c78ebafc,c78ebc74,c10b1fb3,77,c78ebb04,...) at _cv_wait+0x1f9/frame 0xe3b5bca4 usb_process(c78ebaf4,e3b5bd08,c10ce550,3d7,0,...) at usb_process+0xe6/frame 0xe3b5bccc fork_exit(c0961af0,c78ebaf4,e3b5bd08) at fork_exit+0x7f/frame 0xe3b5bcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b5bcf4 --- trap 0, eip = 0, esp = 0xe3b5bd40, ebp = 0 --- Tracing command usb pid 15 tid 100050 td 0xc78e2300 sched_switch(c78e2300,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3b58bd4 mi_switch(104,0,c10daa3d,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b58c08 sleepq_switch(c78e2300,0,c10daa3d,26b,c78e2300,...) at sleepq_switch+0x1b9/frame 0xe3b58c3c sleepq_wait(c78ebacc,0,e3b58c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b58c60 _cv_wait(c78ebacc,c78ebc74,c10b1fb3,77,c78ebad4,...) at _cv_wait+0x1f9/frame 0xe3b58ca4 usb_process(c78ebac4,e3b58d08,c10ce550,3d7,0,...) at usb_process+0xe6/frame 0xe3b58ccc fork_exit(c0961af0,c78ebac4,e3b58d08) at fork_exit+0x7f/frame 0xe3b58cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b58cf4 --- trap 0, eip = 0, esp = 0xe3b58d40, ebp = 0 --- Tracing command usb pid 15 tid 100049 td 0xc78e2600 sched_switch(c78e2600,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3b54bd4 mi_switch(104,0,c10daa3d,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b54c08 sleepq_switch(c78e2600,0,c10daa3d,26b,c78e2600,...) at sleepq_switch+0x1b9/frame 0xe3b54c3c sleepq_wait(c78e6b5c,0,e3b54c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b54c60 _cv_wait(c78e6b5c,c78e6c74,c10b1fb3,77,c78e6b64,...) at _cv_wait+0x1f9/frame 0xe3b54ca4 usb_process(c78e6b54,e3b54d08,c10ce550,3d7,0,...) at usb_process+0xe6/frame 0xe3b54ccc fork_exit(c0961af0,c78e6b54,e3b54d08) at fork_exit+0x7f/frame 0xe3b54cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b54cf4 --- trap 0, eip = 0, esp = 0xe3b54d40, ebp = 0 --- Tracing command usb pid 15 tid 100048 td 0xc78e2900 sched_switch(c78e2900,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3b51bd4 mi_switch(104,0,c10daa3d,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b51c08 sleepq_switch(c78e2900,0,c10daa3d,26b,c78e2900,...) at sleepq_switch+0x1b9/frame 0xe3b51c3c sleepq_wait(c78e6b2c,0,e3b51c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b51c60 _cv_wait(c78e6b2c,c78e6c74,c10b1fb3,77,c78e6b34,...) at _cv_wait+0x1f9/frame 0xe3b51ca4 usb_process(c78e6b24,e3b51d08,c10ce550,3d7,0,...) at usb_process+0xe6/frame 0xe3b51ccc fork_exit(c0961af0,c78e6b24,e3b51d08) at fork_exit+0x7f/frame 0xe3b51cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b51cf4 --- trap 0, eip = 0, esp = 0xe3b51d40, ebp = 0 --- Tracing command usb pid 15 tid 100047 td 0xc78e2c00 sched_switch(c78e2c00,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3b4ebd4 mi_switch(104,0,c10daa3d,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b4ec08 sleepq_switch(c78e2c00,0,c10daa3d,26b,c78e2c00,...) at sleepq_switch+0x1b9/frame 0xe3b4ec3c sleepq_wait(c78e6afc,0,e3b4ec90,1,0,...) at sleepq_wait+0x48/frame 0xe3b4ec60 _cv_wait(c78e6afc,c78e6c74,c10b1fb3,77,c78e6b04,...) at _cv_wait+0x1f9/frame 0xe3b4eca4 usb_process(c78e6af4,e3b4ed08,c10ce550,3d7,0,...) at usb_process+0xe6/frame 0xe3b4eccc fork_exit(c0961af0,c78e6af4,e3b4ed08) at fork_exit+0x7f/frame 0xe3b4ecf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b4ecf4 --- trap 0, eip = 0, esp = 0xe3b4ed40, ebp = 0 --- Tracing command usb pid 15 tid 100046 td 0xc78e3000 sched_switch(c78e3000,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3b4bbd4 mi_switch(104,0,c10daa3d,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b4bc08 sleepq_switch(c78e3000,0,c10daa3d,26b,c78e3000,...) at sleepq_switch+0x1b9/frame 0xe3b4bc3c sleepq_wait(c78e6acc,0,e3b4bc90,1,0,...) at sleepq_wait+0x48/frame 0xe3b4bc60 _cv_wait(c78e6acc,c78e6c74,c10b1fb3,77,c78e6ad4,...) at _cv_wait+0x1f9/frame 0xe3b4bca4 usb_process(c78e6ac4,e3b4bd08,c10ce550,3d7,0,...) at usb_process+0xe6/frame 0xe3b4bccc fork_exit(c0961af0,c78e6ac4,e3b4bd08) at fork_exit+0x7f/frame 0xe3b4bcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b4bcf4 --- trap 0, eip = 0, esp = 0xe3b4bd40, ebp = 0 --- Tracing command usb pid 15 tid 100045 td 0xc78e3300 sched_switch(c78e3300,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3b47bd4 mi_switch(104,0,c10daa3d,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b47c08 sleepq_switch(c78e3300,0,c10daa3d,26b,c78e3300,...) at sleepq_switch+0x1b9/frame 0xe3b47c3c sleepq_wait(c78deb5c,0,e3b47c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b47c60 _cv_wait(c78deb5c,c78dec74,c10b1fb3,77,c78deb64,...) at _cv_wait+0x1f9/frame 0xe3b47ca4 usb_process(c78deb54,e3b47d08,c10ce550,3d7,0,...) at usb_process+0xe6/frame 0xe3b47ccc fork_exit(c0961af0,c78deb54,e3b47d08) at fork_exit+0x7f/frame 0xe3b47cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b47cf4 --- trap 0, eip = 0, esp = 0xe3b47d40, ebp = 0 --- Tracing command usb pid 15 tid 100044 td 0xc78e3600 sched_switch(c78e3600,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3b44bd4 mi_switch(104,0,c10daa3d,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b44c08 sleepq_switch(c78e3600,0,c10daa3d,26b,c78e3600,...) at sleepq_switch+0x1b9/frame 0xe3b44c3c sleepq_wait(c78deb2c,0,e3b44c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b44c60 _cv_wait(c78deb2c,c78dec74,c10b1fb3,77,c78deb34,...) at _cv_wait+0x1f9/frame 0xe3b44ca4 usb_process(c78deb24,e3b44d08,c10ce550,3d7,0,...) at usb_process+0xe6/frame 0xe3b44ccc fork_exit(c0961af0,c78deb24,e3b44d08) at fork_exit+0x7f/frame 0xe3b44cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b44cf4 --- trap 0, eip = 0, esp = 0xe3b44d40, ebp = 0 --- Tracing command usb pid 15 tid 100043 td 0xc78e3900 sched_switch(c78e3900,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3b41bd4 mi_switch(104,0,c10daa3d,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b41c08 sleepq_switch(c78e3900,0,c10daa3d,26b,c78e3900,...) at sleepq_switch+0x1b9/frame 0xe3b41c3c sleepq_wait(c78deafc,0,e3b41c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b41c60 _cv_wait(c78deafc,c78dec74,c10b1fb3,77,c78deb04,...) at _cv_wait+0x1f9/frame 0xe3b41ca4 usb_process(c78deaf4,e3b41d08,c10ce550,3d7,0,...) at usb_process+0xe6/frame 0xe3b41ccc fork_exit(c0961af0,c78deaf4,e3b41d08) at fork_exit+0x7f/frame 0xe3b41cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b41cf4 --- trap 0, eip = 0, esp = 0xe3b41d40, ebp = 0 --- Tracing command usb pid 15 tid 100042 td 0xc78e3c00 sched_switch(c78e3c00,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3b3ebd4 mi_switch(104,0,c10daa3d,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b3ec08 sleepq_switch(c78e3c00,0,c10daa3d,26b,c78e3c00,...) at sleepq_switch+0x1b9/frame 0xe3b3ec3c sleepq_wait(c78deacc,0,e3b3ec90,1,0,...) at sleepq_wait+0x48/frame 0xe3b3ec60 _cv_wait(c78deacc,c78dec74,c10b1fb3,77,c78dead4,...) at _cv_wait+0x1f9/frame 0xe3b3eca4 usb_process(c78deac4,e3b3ed08,c10ce550,3d7,0,...) at usb_process+0xe6/frame 0xe3b3eccc fork_exit(c0961af0,c78deac4,e3b3ed08) at fork_exit+0x7f/frame 0xe3b3ecf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b3ecf4 --- trap 0, eip = 0, esp = 0xe3b3ed40, ebp = 0 --- Tracing command usb pid 15 tid 100040 td 0xc78d2000 sched_switch(c78d2000,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3b37bd4 mi_switch(104,0,c10daa3d,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b37c08 sleepq_switch(c78d2000,0,c10daa3d,26b,c78d2000,...) at sleepq_switch+0x1b9/frame 0xe3b37c3c sleepq_wait(c78d9b5c,0,e3b37c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b37c60 _cv_wait(c78d9b5c,c78d9c74,c10b1fb3,77,c78d9b64,...) at _cv_wait+0x1f9/frame 0xe3b37ca4 usb_process(c78d9b54,e3b37d08,c10ce550,3d7,0,...) at usb_process+0xe6/frame 0xe3b37ccc fork_exit(c0961af0,c78d9b54,e3b37d08) at fork_exit+0x7f/frame 0xe3b37cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b37cf4 --- trap 0, eip = 0, esp = 0xe3b37d40, ebp = 0 --- Tracing command usb pid 15 tid 100039 td 0xc78d2300 sched_switch(c78d2300,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3b34bd4 mi_switch(104,0,c10daa3d,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b34c08 sleepq_switch(c78d2300,0,c10daa3d,26b,c78d2300,...) at sleepq_switch+0x1b9/frame 0xe3b34c3c sleepq_wait(c78d9b2c,0,e3b34c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b34c60 _cv_wait(c78d9b2c,c78d9c74,c10b1fb3,77,c78d9b34,...) at _cv_wait+0x1f9/frame 0xe3b34ca4 usb_process(c78d9b24,e3b34d08,c10ce550,3d7,0,...) at usb_process+0xe6/frame 0xe3b34ccc fork_exit(c0961af0,c78d9b24,e3b34d08) at fork_exit+0x7f/frame 0xe3b34cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b34cf4 --- trap 0, eip = 0, esp = 0xe3b34d40, ebp = 0 --- Tracing command usb pid 15 tid 100038 td 0xc78d2600 sched_switch(c78d2600,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3b31bd4 mi_switch(104,0,c10daa3d,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b31c08 sleepq_switch(c78d2600,0,c10daa3d,26b,c78d2600,...) at sleepq_switch+0x1b9/frame 0xe3b31c3c sleepq_wait(c78d9afc,0,e3b31c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b31c60 _cv_wait(c78d9afc,c78d9c74,c10b1fb3,77,c78d9b04,...) at _cv_wait+0x1f9/frame 0xe3b31ca4 usb_process(c78d9af4,e3b31d08,c10ce550,3d7,0,...) at usb_process+0xe6/frame 0xe3b31ccc fork_exit(c0961af0,c78d9af4,e3b31d08) at fork_exit+0x7f/frame 0xe3b31cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b31cf4 --- trap 0, eip = 0, esp = 0xe3b31d40, ebp = 0 --- Tracing command usb pid 15 tid 100037 td 0xc78d2900 sched_switch(c78d2900,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3b2ebd4 mi_switch(104,0,c10daa3d,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b2ec08 sleepq_switch(c78d2900,0,c10daa3d,26b,c78d2900,...) at sleepq_switch+0x1b9/frame 0xe3b2ec3c sleepq_wait(c78d9acc,0,e3b2ec90,1,0,...) at sleepq_wait+0x48/frame 0xe3b2ec60 _cv_wait(c78d9acc,c78d9c74,c10b1fb3,77,c78d9ad4,...) at _cv_wait+0x1f9/frame 0xe3b2eca4 usb_process(c78d9ac4,e3b2ed08,c10ce550,3d7,0,...) at usb_process+0xe6/frame 0xe3b2eccc fork_exit(c0961af0,c78d9ac4,e3b2ed08) at fork_exit+0x7f/frame 0xe3b2ecf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b2ecf4 --- trap 0, eip = 0, esp = 0xe3b2ed40, ebp = 0 --- Tracing command usb pid 15 tid 100035 td 0xc78d4000 sched_switch(c78d4000,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3b27bd4 mi_switch(104,0,c10daa3d,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b27c08 sleepq_switch(c78d4000,0,c10daa3d,26b,c78d4000,...) at sleepq_switch+0x1b9/frame 0xe3b27c3c sleepq_wait(c78cdb5c,0,e3b27c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b27c60 _cv_wait(c78cdb5c,c78cdc74,c10b1fb3,77,c78cdb64,...) at _cv_wait+0x1f9/frame 0xe3b27ca4 usb_process(c78cdb54,e3b27d08,c10ce550,3d7,0,...) at usb_process+0xe6/frame 0xe3b27ccc fork_exit(c0961af0,c78cdb54,e3b27d08) at fork_exit+0x7f/frame 0xe3b27cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b27cf4 --- trap 0, eip = 0, esp = 0xe3b27d40, ebp = 0 --- Tracing command usb pid 15 tid 100034 td 0xc78d4300 sched_switch(c78d4300,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3b24bd4 mi_switch(104,0,c10daa3d,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b24c08 sleepq_switch(c78d4300,0,c10daa3d,26b,c78d4300,...) at sleepq_switch+0x1b9/frame 0xe3b24c3c sleepq_wait(c78cdb2c,0,e3b24c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b24c60 _cv_wait(c78cdb2c,c78cdc74,c10b1fb3,77,c78cdb34,...) at _cv_wait+0x1f9/frame 0xe3b24ca4 usb_process(c78cdb24,e3b24d08,c10ce550,3d7,0,...) at usb_process+0xe6/frame 0xe3b24ccc fork_exit(c0961af0,c78cdb24,e3b24d08) at fork_exit+0x7f/frame 0xe3b24cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b24cf4 --- trap 0, eip = 0, esp = 0xe3b24d40, ebp = 0 --- Tracing command usb pid 15 tid 100033 td 0xc78d4600 sched_switch(c78d4600,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3b21bd4 mi_switch(104,0,c10daa3d,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b21c08 sleepq_switch(c78d4600,0,c10daa3d,26b,c78d4600,...) at sleepq_switch+0x1b9/frame 0xe3b21c3c sleepq_wait(c78cdafc,0,e3b21c90,1,0,...) at sleepq_wait+0x48/frame 0xe3b21c60 _cv_wait(c78cdafc,c78cdc74,c10b1fb3,77,c78cdb04,...) at _cv_wait+0x1f9/frame 0xe3b21ca4 usb_process(c78cdaf4,e3b21d08,c10ce550,3d7,0,...) at usb_process+0xe6/frame 0xe3b21ccc fork_exit(c0961af0,c78cdaf4,e3b21d08) at fork_exit+0x7f/frame 0xe3b21cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b21cf4 --- trap 0, eip = 0, esp = 0xe3b21d40, ebp = 0 --- Tracing command usb pid 15 tid 100032 td 0xc78d4900 sched_switch(c78d4900,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3b1ebd4 mi_switch(104,0,c10daa3d,1f5,8,...) at mi_switch+0x1c9/frame 0xe3b1ec08 sleepq_switch(c78d4900,0,c10daa3d,26b,c78d4900,...) at sleepq_switch+0x1b9/frame 0xe3b1ec3c sleepq_wait(c78cdacc,0,e3b1ec90,1,0,...) at sleepq_wait+0x48/frame 0xe3b1ec60 _cv_wait(c78cdacc,c78cdc74,c10b1fb3,77,c78cdad4,...) at _cv_wait+0x1f9/frame 0xe3b1eca4 usb_process(c78cdac4,e3b1ed08,c10ce550,3d7,0,...) at usb_process+0xe6/frame 0xe3b1eccc fork_exit(c0961af0,c78cdac4,e3b1ed08) at fork_exit+0x7f/frame 0xe3b1ecf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b1ecf4 --- trap 0, eip = 0, esp = 0xe3b1ed40, ebp = 0 --- Tracing command yarrow pid 14 tid 100017 td 0xc7776600 sched_switch(c7776600,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xc7236bb4 mi_switch(104,0,c10daa3d,1f5,c12a28d0,...) at mi_switch+0x1c9/frame 0xc7236be8 sleepq_switch(c7776600,0,c10daa3d,28e,c126d5dc,...) at sleepq_switch+0x1b9/frame 0xc7236c1c sleepq_timedwait(c126d5dc,0,c10d564d,c10c081e,0,...) at sleepq_timedwait+0x48/frame 0xc7236c40 msleep_spin_sbt(c126d5dc,c144d910,c10c081e,19999999,0,...) at msleep_spin_sbt+0x225/frame 0xc7236c88 random_kthread(0,c7236d08,c10ce550,3d7,0,...) at random_kthread+0x2ea/frame 0xc7236ccc fork_exit(c08500c0,0,c7236d08) at fork_exit+0x7f/frame 0xc7236cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7236cf4 --- trap 0, eip = 0, esp = 0xc7236d40, ebp = 0 --- Tracing command geom pid 13 tid 100015 td 0xc7776c00 sched_switch(c7776c00,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xc722fb70 mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xc722fba4 sleepq_switch(c7776c00,0,c10daa3d,26b,c7776c00,...) at sleepq_switch+0x1b9/frame 0xc722fbd8 sleepq_wait(c144ebe8,5c,c10c081e,0,0,...) at sleepq_wait+0x48/frame 0xc722fbfc _sleep(c144ebe8,c1284b7c,25c,c10c081e,0,...) at _sleep+0x37c/frame 0xc722fc58 g_io_schedule_down(c7776c00,0,c10c2545,6c,c722fcf4,...) at g_io_schedule_down+0x5c/frame 0xc722fcb4 g_down_procbody(0,c722fd08,c10ce550,3d7,0,...) at g_down_procbody+0x9d/frame 0xc722fccc fork_exit(c0a10c10,0,c722fd08) at fork_exit+0x7f/frame 0xc722fcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc722fcf4 --- trap 0, eip = 0, esp = 0xc722fd40, ebp = 0 --- Tracing command geom pid 13 tid 100014 td 0xc7778000 sched_switch(c7778000,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xc722cb98 mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xc722cbcc sleepq_switch(c7778000,0,c10daa3d,26b,c7778000,...) at sleepq_switch+0x1b9/frame 0xc722cc00 sleepq_wait(c144ebe4,5c,c10c081e,0,0,...) at sleepq_wait+0x48/frame 0xc722cc24 _sleep(c144ebe4,c1284b9c,25c,c10c081e,0,...) at _sleep+0x37c/frame 0xc722cc80 g_io_schedule_up(c7778000,0,c10c2545,5f,c722ccf4,...) at g_io_schedule_up+0xe6/frame 0xc722ccb4 g_up_procbody(0,c722cd08,c10ce550,3d7,0,...) at g_up_procbody+0x9d/frame 0xc722cccc fork_exit(c0a10b70,0,c722cd08) at fork_exit+0x7f/frame 0xc722ccf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc722ccf4 --- trap 0, eip = 0, esp = 0xc722cd40, ebp = 0 --- Tracing command geom pid 13 tid 100013 td 0xc76c7300 sched_switch(c76c7300,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xc7229ba8 mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xc7229bdc sleepq_switch(c76c7300,0,c10daa3d,26b,c76c7300,...) at sleepq_switch+0x1b9/frame 0xc7229c10 sleepq_wait(c144ebe0,5c,c10c081e,0,0,...) at sleepq_wait+0x48/frame 0xc7229c34 _sleep(c144ebe0,c1284b54,25c,c10c081e,0,...) at _sleep+0x37c/frame 0xc7229c90 g_run_events(0,c7229d08,c10ce550,3d7,0,...) at g_run_events+0x566/frame 0xc7229ccc fork_exit(c0a10ad0,0,c7229d08) at fork_exit+0x7f/frame 0xc7229cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7229cf4 --- trap 0, eip = 0, esp = 0xc7229d40, ebp = 0 --- Tracing command intr pid 12 tid 100063 td 0xc78e5300 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100062 td 0xc78e5600 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100060 td 0xc78e5c00 sched_switch(c78e5c00,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xe3b8ac60 mi_switch(109,0,c10ce7ed,55a,8,...) at mi_switch+0x1c9/frame 0xe3b8ac94 ithread_loop(c770d5a0,e3b8ad08,c10ce550,3d7,0,...) at ithread_loop+0x100/frame 0xe3b8accc fork_exit(c0a83d90,c770d5a0,e3b8ad08) at fork_exit+0x7f/frame 0xe3b8acf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b8acf4 --- trap 0, eip = 0, esp = 0xe3b8ad40, ebp = 0 --- Tracing command intr pid 12 tid 100059 td 0xc78ef000 sched_switch(c78ef000,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xe3b79c60 mi_switch(109,0,c10ce7ed,55a,8,...) at mi_switch+0x1c9/frame 0xe3b79c94 ithread_loop(c770d0f0,e3b79d08,c10ce550,3d7,0,...) at ithread_loop+0x100/frame 0xe3b79ccc fork_exit(c0a83d90,c770d0f0,e3b79d08) at fork_exit+0x7f/frame 0xe3b79cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b79cf4 --- trap 0, eip = 0, esp = 0xe3b79d40, ebp = 0 --- Tracing command intr pid 12 tid 100054 td 0xc78f2000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100041 td 0xc78e5000 sched_switch(c78e5000,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xe3b3bc60 mi_switch(109,0,c10ce7ed,55a,8,...) at mi_switch+0x1c9/frame 0xe3b3bc94 ithread_loop(c770d1e0,e3b3bd08,c10ce550,3d7,0,...) at ithread_loop+0x100/frame 0xe3b3bccc fork_exit(c0a83d90,c770d1e0,e3b3bd08) at fork_exit+0x7f/frame 0xe3b3bcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b3bcf4 --- trap 0, eip = 0, esp = 0xe3b3bd40, ebp = 0 --- Tracing command intr pid 12 tid 100036 td 0xc78d2c00 sched_switch(c78d2c00,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xe3b2bc60 mi_switch(109,0,c10ce7ed,55a,8,...) at mi_switch+0x1c9/frame 0xe3b2bc94 ithread_loop(c770d0c0,e3b2bd08,c10ce550,3d7,0,...) at ithread_loop+0x100/frame 0xe3b2bccc fork_exit(c0a83d90,c770d0c0,e3b2bd08) at fork_exit+0x7f/frame 0xe3b2bcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b2bcf4 --- trap 0, eip = 0, esp = 0xe3b2bd40, ebp = 0 --- Tracing command intr pid 12 tid 100031 td 0xc78d4c00 sched_switch(c78d4c00,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xe3b1bc60 mi_switch(109,0,c10ce7ed,55a,8,...) at mi_switch+0x1c9/frame 0xe3b1bc94 ithread_loop(c770d6a0,e3b1bd08,c10ce550,3d7,0,...) at ithread_loop+0x100/frame 0xe3b1bccc fork_exit(c0a83d90,c770d6a0,e3b1bd08) at fork_exit+0x7f/frame 0xe3b1bcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3b1bcf4 --- trap 0, eip = 0, esp = 0xe3b1bd40, ebp = 0 --- Tracing command intr pid 12 tid 100030 td 0xc7778300 sched_switch(c7778300,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xc7393c60 mi_switch(109,0,c10ce7ed,55a,8,...) at mi_switch+0x1c9/frame 0xc7393c94 ithread_loop(c770d280,c7393d08,c10ce550,3d7,0,...) at ithread_loop+0x100/frame 0xc7393ccc fork_exit(c0a83d90,c770d280,c7393d08) at fork_exit+0x7f/frame 0xc7393cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7393cf4 --- trap 0, eip = 0, esp = 0xc7393d40, ebp = 0 --- Tracing command intr pid 12 tid 100029 td 0xc7778600 sched_switch(c7778600,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xc738fc60 mi_switch(109,0,c10ce7ed,55a,8,...) at mi_switch+0x1c9/frame 0xc738fc94 ithread_loop(c770d6c0,c738fd08,c10ce550,3d7,0,...) at ithread_loop+0x100/frame 0xc738fccc fork_exit(c0a83d90,c770d6c0,c738fd08) at fork_exit+0x7f/frame 0xc738fcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc738fcf4 --- trap 0, eip = 0, esp = 0xc738fd40, ebp = 0 --- Tracing command intr pid 12 tid 100028 td 0xc7778900 sched_switch(c7778900,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xc7286c60 mi_switch(109,0,c10ce7ed,55a,8,...) at mi_switch+0x1c9/frame 0xc7286c94 ithread_loop(c770d3b0,c7286d08,c10ce550,3d7,0,...) at ithread_loop+0x100/frame 0xc7286ccc fork_exit(c0a83d90,c770d3b0,c7286d08) at fork_exit+0x7f/frame 0xc7286cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7286cf4 --- trap 0, eip = 0, esp = 0xc7286d40, ebp = 0 --- Tracing command intr pid 12 tid 100027 td 0xc7778c00 sched_switch(c7778c00,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xc7254c60 mi_switch(109,0,c10ce7ed,55a,8,...) at mi_switch+0x1c9/frame 0xc7254c94 ithread_loop(c770d760,c7254d08,c10ce550,3d7,0,...) at ithread_loop+0x100/frame 0xc7254ccc fork_exit(c0a83d90,c770d760,c7254d08) at fork_exit+0x7f/frame 0xc7254cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7254cf4 --- trap 0, eip = 0, esp = 0xc7254d40, ebp = 0 --- Tracing command intr pid 12 tid 100025 td 0xc785b300 sched_switch(c785b300,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xc724ec60 mi_switch(109,0,c10ce7ed,55a,8,...) at mi_switch+0x1c9/frame 0xc724ec94 ithread_loop(c770d770,c724ed08,c10ce550,3d7,0,...) at ithread_loop+0x100/frame 0xc724eccc fork_exit(c0a83d90,c770d770,c724ed08) at fork_exit+0x7f/frame 0xc724ecf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc724ecf4 --- trap 0, eip = 0, esp = 0xc724ed40, ebp = 0 --- Tracing command intr pid 12 tid 100020 td 0xc770ec00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100018 td 0xc7776300 sched_switch(c7776300,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xc7239c60 mi_switch(109,0,c10ce7ed,55a,8,...) at mi_switch+0x1c9/frame 0xc7239c94 ithread_loop(c770d790,c7239d08,c10ce550,3d7,0,...) at ithread_loop+0x100/frame 0xc7239ccc fork_exit(c0a83d90,c770d790,c7239d08) at fork_exit+0x7f/frame 0xc7239cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7239cf4 --- trap 0, eip = 0, esp = 0xc7239d40, ebp = 0 --- Tracing command intr pid 12 tid 100012 td 0xc76c7600 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100011 td 0xc76c7900 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100010 td 0xc76c7c00 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100009 td 0xc770e000 sched_switch(c770e000,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xc721dc60 mi_switch(109,0,c10ce7ed,55a,8,...) at mi_switch+0x1c9/frame 0xc721dc94 ithread_loop(c770d7d0,c721dd08,c10ce550,3d7,0,...) at ithread_loop+0x100/frame 0xc721dccc fork_exit(c0a83d90,c770d7d0,c721dd08) at fork_exit+0x7f/frame 0xc721dcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc721dcf4 --- trap 0, eip = 0, esp = 0xc721dd40, ebp = 0 --- Tracing command intr pid 12 tid 100008 td 0xc770e300 sched_switch(c770e300,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xc721ac60 mi_switch(109,0,c10ce7ed,55a,8,...) at mi_switch+0x1c9/frame 0xc721ac94 ithread_loop(c770d7e0,c721ad08,c10ce550,3d7,0,...) at ithread_loop+0x100/frame 0xc721accc fork_exit(c0a83d90,c770d7e0,c721ad08) at fork_exit+0x7f/frame 0xc721acf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc721acf4 --- trap 0, eip = 0, esp = 0xc721ad40, ebp = 0 --- Tracing command intr pid 12 tid 100007 td 0xc770e600 fork_trampoline() at fork_trampoline Tracing command idle pid 11 tid 100006 td 0xc76c5000 sched_switch(c76c5000,0,108,1b0,20,...) at sched_switch+0x4b9/frame 0xc7213c1c mi_switch(108,0,c10d6fcd,a44,c1295108,...) at mi_switch+0x1c9/frame 0xc7213c50 sched_idletd(0,c7213d08,c10ce550,3d7,0,...) at sched_idletd+0xf1/frame 0xc7213ccc fork_exit(c0ada5d0,0,c7213d08) at fork_exit+0x7f/frame 0xc7213cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7213cf4 --- trap 0, eip = 0, esp = 0xc7213d40, ebp = 0 --- Tracing command idle pid 11 tid 100005 td 0xc76c5300 sched_switch(c76c5300,0,108,1b0,c7210c50,...) at sched_switch+0x4b9/frame 0xc7210c1c mi_switch(108,0,c10d6fcd,a44,c1294a08,...) at mi_switch+0x1c9/frame 0xc7210c50 sched_idletd(0,c7210d08,c10ce550,3d7,0,...) at sched_idletd+0xf1/frame 0xc7210ccc fork_exit(c0ada5d0,0,c7210d08) at fork_exit+0x7f/frame 0xc7210cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7210cf4 --- trap 0, eip = 0, esp = 0xc7210d40, ebp = 0 --- Tracing command idle pid 11 tid 100004 td 0xc76c5600 cpustop_handler(1,13,c720dc0c,c0edd3cb,40dd14,...) at cpustop_handler+0x2f/frame 0xc720dac0 ipi_nmi_handler(40dd14,0,0,c720db14,c0a75b2b,...) at ipi_nmi_handler+0x37/frame 0xc720dad0 trap(c720dc18) at trap+0x3b/frame 0xc720dc0c calltrap() at calltrap+0x6/frame 0xc720dc0c --- trap 0x13, eip = 0xc0ada740, esp = 0xc720dc58, ebp = 0xc720dccc --- sched_idletd(0,c720dd08,c10ce550,3d7,0,...) at sched_idletd+0x170/frame 0xc720dccc fork_exit(c0ada5d0,0,c720dd08) at fork_exit+0x7f/frame 0xc720dcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc720dcf4 --- trap 0, eip = 0, esp = 0xc720dd40, ebp = 0 --- Tracing command idle pid 11 tid 100003 td 0xc76c5900 sched_switch(c76c5900,0,108,1b0,c720ac50,...) at sched_switch+0x4b9/frame 0xc720ac1c mi_switch(108,0,c10d6fcd,a44,c1293c08,...) at mi_switch+0x1c9/frame 0xc720ac50 sched_idletd(0,c720ad08,c10ce550,3d7,0,...) at sched_idletd+0xf1/frame 0xc720accc fork_exit(c0ada5d0,0,c720ad08) at fork_exit+0x7f/frame 0xc720acf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc720acf4 --- trap 0, eip = 0, esp = 0xc720ad40, ebp = 0 --- Tracing command init pid 1 tid 100002 td 0xc76c5c00 sched_switch(c76c5c00,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xc720698c mi_switch(104,0,c10daa3d,1f5,c10daa3d,...) at mi_switch+0x1c9/frame 0xc72069c0 sleepq_switch(c76c5c00,0,c10daa3d,1ab,c7206a38,...) at sleepq_switch+0x1b9/frame 0xc72069f4 sleepq_catch_signals(0,c10daa3d,162,e88,c76c5c00,...) at sleepq_catch_signals+0x361/frame 0xc7206a3c sleepq_wait_sig(c76c3ba0,6c,c10ddeb5,100,0,...) at sleepq_wait_sig+0x14/frame 0xc7206a60 _sleep(c76c3ba0,c76c3c34,16c,c10ddeb5,0,...) at _sleep+0x35b/frame 0xc7206abc kern_wait6(c76c5c00,7,0,0,c7206b4c,...) at kern_wait6+0x45c/frame 0xc7206b20 sys_wait4(c76c5c00,c7206cc8,7b3,2,c7206cc4,...) at sys_wait4+0x94/frame 0xc7206c40 syscall(c7206d08) at syscall+0x2ed/frame 0xc7206cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xc7206cfc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x805e3b7, esp = 0xbfbfd874, ebp = 0xbfbfd888 --- Tracing command audit pid 10 tid 100001 td 0xc76c7000 sched_switch(c76c7000,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xc7203ba8 mi_switch(104,0,c10daa3d,1f5,8,...) at mi_switch+0x1c9/frame 0xc7203bdc sleepq_switch(c76c7000,0,c10daa3d,26b,c76c7000,...) at sleepq_switch+0x1b9/frame 0xc7203c10 sleepq_wait(c1455938,0,c7203c64,1,0,...) at sleepq_wait+0x48/frame 0xc7203c34 _cv_wait(c1455938,c14558c4,c11065e2,18d,0,...) at _cv_wait+0x1f9/frame 0xc7203c78 audit_worker(0,c7203d08,c10ce550,3d7,0,...) at audit_worker+0x94/frame 0xc7203ccc fork_exit(c0cc9440,0,c7203d08) at fork_exit+0x7f/frame 0xc7203cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7203cf4 --- trap 0, eip = 0, esp = 0xc7203d40, ebp = 0 --- Tracing command kernel pid 0 tid 100065 td 0xc7949c00 sched_switch(c7949c00,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3ba3b68 mi_switch(104,0,c10daa3d,1f5,101,...) at mi_switch+0x1c9/frame 0xe3ba3b9c sleepq_switch(c7949c00,0,c10daa3d,28e,c7949c00,...) at sleepq_switch+0x1b9/frame 0xe3ba3bd0 sleepq_timedwait(c1286853,0,2,0,0,...) at sleepq_timedwait+0x48/frame 0xe3ba3bf4 _sleep(c1286853,0,0,c10c081e,fffffc88,...) at _sleep+0x345/frame 0xe3ba3c50 pause_sbt(c10c081e,fffffc88,2,0,0,...) at pause_sbt+0x86/frame 0xe3ba3c8c deadlkres(0,e3ba3d08,c10ce550,3d7,0,...) at deadlkres+0x82/frame 0xe3ba3ccc fork_exit(c0a627c0,0,e3ba3d08) at fork_exit+0x7f/frame 0xe3ba3cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3ba3cf4 --- trap 0, eip = 0, esp = 0xe3ba3d40, ebp = 0 --- Tracing command kernel pid 0 tid 100064 td 0xc794a000 sched_switch(c794a000,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xe3ba0bc0 mi_switch(104,0,c10daa3d,1f5,c12a3fb0,...) at mi_switch+0x1c9/frame 0xe3ba0bf4 sleepq_switch(c794a000,0,c10daa3d,26b,c7867d80,...) at sleepq_switch+0x1b9/frame 0xe3ba0c28 sleepq_wait(c7867d80,0,c10d564d,c10c081e,0,...) at sleepq_wait+0x48/frame 0xe3ba0c4c msleep_spin_sbt(c7867d80,c7867d98,c10c081e,0,0,...) at msleep_spin_sbt+0x239/frame 0xe3ba0c94 taskqueue_thread_loop(c144b400,e3ba0d08,c10ce550,3d7,0,...) at taskqueue_thread_loop+0xcc/frame 0xe3ba0ccc fork_exit(c0afaba0,c144b400,e3ba0d08) at fork_exit+0x7f/frame 0xe3ba0cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe3ba0cf4 --- trap 0, eip = 0, esp = 0xe3ba0d40, ebp = 0 --- Tracing command kernel pid 0 tid 100026 td 0xc785b000 sched_switch(c785b000,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xc7251bac mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xc7251be0 sleepq_switch(c785b000,0,c10daa3d,26b,c785b000,...) at sleepq_switch+0x1b9/frame 0xc7251c14 sleepq_wait(c75a8600,0,c10c081e,0,0,...) at sleepq_wait+0x48/frame 0xc7251c38 _sleep(c75a8600,c75a8618,0,c10c081e,0,...) at _sleep+0x37c/frame 0xc7251c94 taskqueue_thread_loop(c1455970,c7251d08,c10ce550,3d7,0,...) at taskqueue_thread_loop+0x129/frame 0xc7251ccc fork_exit(c0afaba0,c1455970,c7251d08) at fork_exit+0x7f/frame 0xc7251cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7251cf4 --- trap 0, eip = 0, esp = 0xc7251d40, ebp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xc785b600 sched_switch(c785b600,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xc724bbc0 mi_switch(104,0,c10daa3d,1f5,c12a28d0,...) at mi_switch+0x1c9/frame 0xc724bbf4 sleepq_switch(c785b600,0,c10daa3d,26b,c75a8980,...) at sleepq_switch+0x1b9/frame 0xc724bc28 sleepq_wait(c75a8980,0,c10d564d,c10c081e,0,...) at sleepq_wait+0x48/frame 0xc724bc4c msleep_spin_sbt(c75a8980,c75a8998,c10c081e,0,0,...) at msleep_spin_sbt+0x239/frame 0xc724bc94 taskqueue_thread_loop(c144bf20,c724bd08,c10ce550,3d7,0,...) at taskqueue_thread_loop+0xcc/frame 0xc724bccc fork_exit(c0afaba0,c144bf20,c724bd08) at fork_exit+0x7f/frame 0xc724bcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc724bcf4 --- trap 0, eip = 0, esp = 0xc724bd40, ebp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xc785b900 sched_switch(c785b900,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xc7248bc0 mi_switch(104,0,c10daa3d,1f5,c12a28d0,...) at mi_switch+0x1c9/frame 0xc7248bf4 sleepq_switch(c785b900,0,c10daa3d,26b,c75a8980,...) at sleepq_switch+0x1b9/frame 0xc7248c28 sleepq_wait(c75a8980,0,c10d564d,c10c081e,0,...) at sleepq_wait+0x48/frame 0xc7248c4c msleep_spin_sbt(c75a8980,c75a8998,c10c081e,0,0,...) at msleep_spin_sbt+0x239/frame 0xc7248c94 taskqueue_thread_loop(c144bf20,c7248d08,c10ce550,3d7,0,...) at taskqueue_thread_loop+0xcc/frame 0xc7248ccc fork_exit(c0afaba0,c144bf20,c7248d08) at fork_exit+0x7f/frame 0xc7248cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7248cf4 --- trap 0, eip = 0, esp = 0xc7248d40, ebp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xc785bc00 sched_switch(c785bc00,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xc7245bc0 mi_switch(104,0,c10daa3d,1f5,c12a28d0,...) at mi_switch+0x1c9/frame 0xc7245bf4 sleepq_switch(c785bc00,0,c10daa3d,26b,c75a8980,...) at sleepq_switch+0x1b9/frame 0xc7245c28 sleepq_wait(c75a8980,0,c10d564d,c10c081e,0,...) at sleepq_wait+0x48/frame 0xc7245c4c msleep_spin_sbt(c75a8980,c75a8998,c10c081e,0,0,...) at msleep_spin_sbt+0x239/frame 0xc7245c94 taskqueue_thread_loop(c144bf20,c7245d08,c10ce550,3d7,0,...) at taskqueue_thread_loop+0xcc/frame 0xc7245ccc fork_exit(c0afaba0,c144bf20,c7245d08) at fork_exit+0x7f/frame 0xc7245cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7245cf4 --- trap 0, eip = 0, esp = 0xc7245d40, ebp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xc770e900 sched_switch(c770e900,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xc7242bac mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xc7242be0 sleepq_switch(c770e900,0,c10daa3d,26b,c770e900,...) at sleepq_switch+0x1b9/frame 0xc7242c14 sleepq_wait(c75a8a00,0,c10c081e,0,0,...) at sleepq_wait+0x48/frame 0xc7242c38 _sleep(c75a8a00,c75a8a18,0,c10c081e,0,...) at _sleep+0x37c/frame 0xc7242c94 taskqueue_thread_loop(c144f3dc,c7242d08,c10ce550,3d7,0,...) at taskqueue_thread_loop+0x129/frame 0xc7242ccc fork_exit(c0afaba0,c144f3dc,c7242d08) at fork_exit+0x7f/frame 0xc7242cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7242cf4 --- trap 0, eip = 0, esp = 0xc7242d40, ebp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xc7776000 sched_switch(c7776000,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xc723cbac mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xc723cbe0 sleepq_switch(c7776000,0,c10daa3d,26b,c7776000,...) at sleepq_switch+0x1b9/frame 0xc723cc14 sleepq_wait(c75a8b00,0,c10c081e,0,0,...) at sleepq_wait+0x48/frame 0xc723cc38 _sleep(c75a8b00,c75a8b18,0,c10c081e,0,...) at _sleep+0x37c/frame 0xc723cc94 taskqueue_thread_loop(c1452340,c723cd08,c10ce550,3d7,0,...) at taskqueue_thread_loop+0x129/frame 0xc723cccc fork_exit(c0afaba0,c1452340,c723cd08) at fork_exit+0x7f/frame 0xc723ccf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc723ccf4 --- trap 0, eip = 0, esp = 0xc723cd40, ebp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xc7776900 sched_switch(c7776900,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xc7232bac mi_switch(104,0,c10daa3d,1f5,162,...) at mi_switch+0x1c9/frame 0xc7232be0 sleepq_switch(c7776900,0,c10daa3d,26b,c7776900,...) at sleepq_switch+0x1b9/frame 0xc7232c14 sleepq_wait(c75a8c80,0,c10c081e,0,0,...) at sleepq_wait+0x48/frame 0xc7232c38 _sleep(c75a8c80,c75a8c98,0,c10c081e,0,...) at _sleep+0x37c/frame 0xc7232c94 taskqueue_thread_loop(c12a1cc4,c7232d08,c10ce550,3d7,0,...) at taskqueue_thread_loop+0x129/frame 0xc7232ccc fork_exit(c0afaba0,c12a1cc4,c7232d08) at fork_exit+0x7f/frame 0xc7232cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7232cf4 --- trap 0, eip = 0, esp = 0xc7232d40, ebp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xc144ef50 sched_switch(c144ef50,0,104,1b5,c0a9f3fe,...) at sched_switch+0x4b9/frame 0xc1820c0c mi_switch(104,0,c10daa3d,1f5,101,...) at mi_switch+0x1c9/frame 0xc1820c40 sleepq_switch(c144ef50,0,c10daa3d,28e,c144ef50,...) at sleepq_switch+0x1b9/frame 0xc1820c74 sleepq_timedwait(c144ec64,54,9,0,0,...) at sleepq_timedwait+0x48/frame 0xc1820c98 _sleep(c144ec64,0,54,c11077a5,fffff470,...) at _sleep+0x345/frame 0xc1820cf4 scheduler(0,0,c113a508,181e000,1825000,...) at scheduler+0x2b3/frame 0xc1820d38 mi_startup() at mi_startup+0xf7/frame 0xc1820d58 begin() at begin+0x2c db:0:allt> call doadump Physical memory: 3435 MB Dumping 175 MB: 160 144 128 112 96 80 64 48 32 16 Dump complete = 0 db:0:doadump> reset (kgdb) bt #0 doadump (textdump=0xa) at pcpu.h:253 #1 0xc0512a7d in db_fncall (dummy1=0x19, dummy2=0x0, dummy3=0xc125456c, dummy4=0xf5e87268 "") at ../../../ddb/db_command.c:578 #2 0xc0512777 in db_command (cmd_table=) at ../../../ddb/db_command.c:449 #3 0xc0516654 in db_script_exec (scriptname=0xc144bc84 "doadump", warnifnotfound=) at ../../../ddb/db_script.c:302 #4 0xc0512777 in db_command (cmd_table=) at ../../../ddb/db_command.c:449 #5 0xc0512490 in db_command_loop () at ../../../ddb/db_command.c:502 #6 0xc0514cd0 in db_trap (type=, code=0xf5e875b0) at ../../../ddb/db_main.c:231 #7 0xc0aea828 in kdb_trap (type=, code=, tf=) at ../../../kern/subr_kdb.c:654 #8 0xc0eddb0e in trap (frame=) at ../../../i386/i386/trap.c:720 #9 0xc0ec6cbc in calltrap () at ../../../i386/i386/exception.s:169 #10 0xc0aea0ad in kdb_enter (why=0xc10d49d3 "panic", msg=) at cpufunc.h:71 #11 0xc0ab3963 in vpanic (fmt=, ap=) at ../../../kern/kern_shutdown.c:747 #12 0xc0ab381a in kassert_panic (fmt=) at ../../../kern/kern_shutdown.c:642 #13 0xc0d57e0d in vm_page_set_invalid (m=0xc3dfe8c0, base=0x0, size=0x1000) at ../../../vm/vm_page.c:2591 #14 0xc0b3bfad in brelse (bp=) at ../../../kern/vfs_bio.c:1579 #15 0xc0b39e65 in bufwrite (bp=) at ../../../kern/vfs_bio.c:1085 #16 0xc0d1ac97 in ffs_write (ap=) at buf.h:395 #17 0xc0f029ef in VOP_WRITE_APV (vop=, a=0xf5e87918) at vnode_if.c:999 #18 0xc0d63fd9 in vnode_pager_generic_putpages (vp=0xc8a5b870, bytecount=0xb30, flags=, rtvals=) at vnode_if.h:413 #19 0xc0b47272 in vop_stdputpages (ap=0xf5e879a8) at ../../../kern/vfs_default.c:751 #20 0xc0f05c9a in VOP_PUTPAGES_APV (vop=, a=0xf5e879a8) at vnode_if.c:2861 #21 0xc0d61e84 in vnode_pager_putpages (object=0x426, m=0xc10c410a, count=0xc124d870, sync=0xc8a5b870, rtvals=0xf5e87a3c) at vnode_if.h:1189 #22 0xc0d58971 in vm_pageout_flush (mc=0xf5e87a3c, count=0x1, mreq=, prunlen=0x12, eio=0xf5e87aec) at vm_pager.h:145 #23 0xc0d51b36 in vm_object_page_collect_flush (object=, p=, flags=0x3, clearobjflags=) at ../../../vm/vm_object.c:961 #24 0xc0d51839 in vm_object_page_clean (object=0xc903b2e4, start=0x0, end=0x2000, flags=) at ../../../vm/vm_object.c:887 #25 0xc0d51d6a in vm_object_sync (object=, offset=0x2000, size=0x2000, syncio=, invalidate=0x8) at ../../../vm/vm_object.c:1036 #26 0xc0d49d00 in vm_map_sync (map=0xc7e3c0e8, start=0x281ec000, end=0x281ee000, syncio=0x1, invalidate=0x1) at ../../../vm/vm_map.c:2651 #27 0xc0d4ef89 in sys_msync (td=, uap=0xf5e87cc8) at ../../../vm/vm_mmap.c:528 #28 0xc0ede7dd in syscall (frame=) at subr_syscall.c:134 #29 0xc0ec6d51 in Xint0x80_syscall () at ../../../i386/i386/exception.s:267 #30 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) Current language: auto; currently minimal (kgdb) f 13 #13 0xc0d57e0d in vm_page_set_invalid (m=0xc3dfe8c0, base=0x0, size=0x1000) at ../../../vm/vm_page.c:2591 2591 KASSERT((m->oflags & VPO_BUSY) == 0, (kgdb) l 2586 vm_page_set_invalid(vm_page_t m, int base, int size) 2587 { 2588 vm_page_bits_t bits; 2589 2590 VM_OBJECT_ASSERT_WLOCKED(m->object); 2591 KASSERT((m->oflags & VPO_BUSY) == 0, 2592 ("vm_page_set_invalid: page %p is busy", m)); 2593 bits = vm_page_bits(base, size); 2594 if (m->valid == VM_PAGE_BITS_ALL && bits != 0) 2595 pmap_remove_all(m); (kgdb) p *m $1 = {pageq = {tqe_next = 0xc367f000, tqe_prev = 0xc1402fe4}, listq = {tqe_next = 0xc367f000, tqe_prev = 0xc903b30c}, object = 0xc903b2e4, pindex = 0x0, phys_addr = 0x72824000, md = {pv_list = {tqh_first = 0xc59b340c, tqh_last = 0xc59b3410}, pat_mode = 0x6}, queue = 0xff, segind = 0x2, hold_count = 0x0, order = 0xb, pool = 0x0, cow = 0x0, wire_count = 0xbf, aflags = 0x3, oflags = 0x1, flags = 0x0, act_count = 0x0, busy = 0x0, valid = 0xff, dirty = 0x0} (kgdb) f 18 #18 0xc0d63fd9 in vnode_pager_generic_putpages (vp=0xc8a5b870, bytecount=0xb30, flags=, rtvals=) at vnode_if.h:413 413 return (VOP_WRITE_APV(vp->v_op, &a)); (kgdb) p *vp $2 = {v_tag = 0xc10c410a "ufs", v_op = 0xc1237ed0, v_data = 0xc8ec5570, v_mount = 0xc8ec8000, v_nmntvnodes = {tqe_next = 0x0, tqe_prev = 0xc9054cb8}, v_un = {vu_mount = 0x0, vu_socket = 0x0, vu_cdev = 0x0, vu_fifoinfo = 0x0}, v_hashlist = {le_next = 0x0, le_prev = 0xc79d233c}, v_cache_src = {lh_first = 0x0}, v_cache_dst = {tqh_first = 0xc9044e58, tqh_last = 0xc9044e68}, v_cache_dd = 0x0, v_lock = {lock_object = {lo_name = 0xc10c410a "ufs", lo_flags = 0x5fb0000, lo_data = 0x0, lo_witness = 0xc7586200}, lk_lock = 0xc90bb604, lk_exslpfail = 0x0, lk_timo = 0x33, lk_pri = 0x60, lk_stack = {depth = 0x9, pcs = {0xc0a95c9f, 0xc0d1b0ad, 0xc0f047a2, 0xc0b6998a, 0xc0d51ccb, 0xc0d49d00, 0xc0d4ef89, 0xc0ede7dd, 0xc0ec6d51, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}}, v_interlock = {lock_object = {lo_name = 0xc10dcfed "vnode interlock", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc757f1e0}, mtx_lock = 0x4}, v_vnlock = 0xc8a5b8a4, v_actfreelist = {tqe_next = 0xc9054ca8, tqe_prev = 0xc8ec8040}, v_bufobj = {bo_mtx = {lock_object = {lo_name = 0xc10e579d "bufobj interlock", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc7584730}, mtx_lock = 0x4}, bo_ops = 0xc1215f20, bo_object = 0xc903b2e4, bo_synclist = {le_next = 0xc9054d68, le_prev = 0xc7991008}, bo_private = 0xc8a5b870, __bo_vnode = 0xc8a5b870, bo_clean = {bv_hd = {tqh_first = 0xe38e4b34, tqh_last = 0xe38e4b6c}, bv_root = {pt_root = 0xe38e4c0d}, bv_cnt = 0x1}, bo_dirty = {bv_hd = {tqh_first = 0x0, tqh_last = 0xc8a5b96c}, bv_root = { pt_root = 0x0}, bv_cnt = 0x0}, bo_numoutput = 0x0, bo_flag = 0x0, bo_bsize = 0x4000}, v_pollinfo = 0x0, v_label = 0x0, v_lockf = 0x0, v_rl = {rl_waiters = {tqh_first = 0x0, tqh_last = 0xc8a5b994}, rl_currdep = 0x0}, v_cstart = 0x0, v_lasta = 0x0, v_lastw = 0x0, v_clen = 0x0, v_holdcnt = 0x4e, v_usecount = 0x3d, v_iflag = 0x200, v_vflag = 0x0, v_writecount = 0x1b, v_hash = 0x3e2, v_type = VREG} (kgdb) $ svn diff -x -p /usr/src/sys Index: /usr/src/sys/vm/vm_fault.c =================================================================== --- /usr/src/sys/vm/vm_fault.c (revision 250556) +++ /usr/src/sys/vm/vm_fault.c (working copy) @@ -924,8 +924,6 @@ vnode_locked: if (fault_flags & VM_FAULT_CHANGE_WIRING) { if (wired) vm_page_wire(fs.m); - else - vm_page_unwire(fs.m, 1); } else vm_page_activate(fs.m); if (m_hold != NULL) { Index: /usr/src/sys/vm/vm_page.c =================================================================== --- /usr/src/sys/vm/vm_page.c (revision 250556) +++ /usr/src/sys/vm/vm_page.c (working copy) @@ -878,6 +878,8 @@ vm_page_remove(vm_page_t m) if ((object = m->object) == NULL) return; VM_OBJECT_ASSERT_WLOCKED(object); +if (object->type == OBJT_VNODE && m->wire_count > 0) +panic("removing wired page from vnode object m %p obj %p", m, object); if (m->oflags & VPO_BUSY) { m->oflags &= ~VPO_BUSY; vm_page_flash(m); -- Test scenario: wip/wire_no_page.sh