GENERIC HEAD from 2018-02-23 07:43:52 UTC, r326118, vmcore.5 GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2018 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 12.0-CURRENT #2 r329862M: Fri Feb 23 16:47:54 CET 2018 pho@t2.osted.lan:/var/tmp/numa/sys/i386/compile/NUMA i386 FreeBSD clang version 6.0.0 (branches/release_60 325330) (based on LLVM 6.0.0) WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. SRAT: Ignoring memory at addr 0x100000000 SRAT: Ignoring memory at addr 0x840000000 SRAT: No memory found for CPU 32 Entering uma_startup with 12 boot pages configured startup_alloc from "UMA Kegs", 11 boot pages left startup_alloc from "UMA Zones", 10 boot pages left startup_alloc from "UMA Zones", 9 boot pages left startup_alloc from "UMA Hash", 8 boot pages left startup_alloc from "UMA Zones", 7 boot pages left startup_alloc from "vmem btag", 6 boot pages left startup_alloc from "UMA Zones", 5 boot pages left startup_alloc from "MAP", 4 boot pages left startup_alloc from "UMA Kegs", 3 boot pages left startup_alloc from "KMAP ENTRY", 2 boot pages left startup_alloc from "vmem", 1 boot pages left Entering uma_startup2 with 0 boot pages left VT(vga): resolution 640x480 CPU: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz (1995.23-MHz 686-class CPU) Origin="GenuineIntel" Id=0x206d7 Family=0x6 Model=0x2d Stepping=7 Features=0xbfebfbff Features2=0x1fbee3ff AMD Features=0x2c100000 AMD Features2=0x1 XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID TSC: P-state invariant, performance statistics real memory = 68719476736 (65536 MB) avail memory = 3049824256 (2908 MB) : Trying to mount root from ufs:/dev/gpt/ssdrootfs [rw]... Expensive timeout(9) function: 0xc0ae20d0(0xc190afb4) 0.006981238 s uhub1: 4 ports with 4 removable, self powered Setting hostuuid: 2bde2bde-f4e2-e111-aab2-001e6756b69b. Setting hostid: 0x0035ff86. Starting ddb. warning: total configured swap (17563648 pages) exceeds maximum recommended amount (1493360 pages). warning: increase kern.maxswzone or reduce amount of swap. Starting file system checks: /dev/gpt/ssdrootfs: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/gpt/ssdrootfs: clean, 46848303 free (179535 frags, 5833596 blocks, 0.3% fragmentation) /dev/da0p3: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/da0p3: clean, 25365339 free (1155 frags, 3170523 blocks, 0.0% fragmentation) panic: queue 3 underflow cpuid = 2 time = 1519401041 KDB: stack backtrace: db_trace_self_wrapper(c16617ec,db2c93b0,c124bb30,d8c22800,db2c93cc,...) at db_trace_self_wrapper+0x2a/frame 0xdb2c9398 kdb_backtrace(c165b767,5a903851,0,db2c9468,b,...) at kdb_backtrace+0x2d/frame 0xdb2c9400 vpanic(c166e26f,db2c9468,c166e26f,db2c9468,db2c9468,...) at vpanic+0x133/frame 0xdb2c9438 kassert_panic(c166e26f,3,c166d1e2,6d2,6d2,...) at kassert_panic+0xd9/frame 0xdb2c945c bq_remove(c192c580,ca1994e4,c166d1e2,56f,246,...) at bq_remove+0x145/frame 0xdb2c9490 binsfree(ca1994e4,2,a67,dd2,0,...) at binsfree+0x1a9/frame 0xdb2c950c bqrelse(ca1994e4,c166d47b,d8e81a80,db2c958c,246,2cf,db2c95b0,c0982220,c1a2bcb0,ca1994e4,d936fef8,d936fe38,d8e81a80) at bqrelse+0x26b/frame 0xdb2c956c bdwrite(ca1994e4,4,1,9,0,...) at bdwrite+0x1eb/frame 0xdb2c95b0 ffs_update(d8c9accc,0,d8e98000,0,ffffffff,...) at ffs_update+0x72c/frame 0xdb2c96d0 ufs_makeinode(db2c9a28,db2c9a3c,c16ad4f4,d94b7000,db2c9888,...) at ufs_makeinode+0x31f/frame 0xdb2c9838 ufs_create(db2c98b8,c1872c37,d94b7000,db2c98ac,c16b0e0a,...) at ufs_create+0x38/frame 0xdb2c985c VOP_CREATE_APV(c1957570,db2c98b8,db2c9a3c,db2c98d0,d94de3b8,...) at VOP_CREATE_APV+0xf6/frame 0xdb2c9888 vn_open_cred(db2c99e0,db2c9a98,180,0,d8e98000,d948b738) at vn_open_cred+0x25c/frame 0xdb2c9960 vn_open(db2c99e0,db2c9a98,180,d948b738,28494409,...) at vn_open+0x3d/frame 0xdb2c9988 kern_openat(d8e81a80,4,28494409,0,100206,180) at kern_openat+0x336/frame 0xdb2c9a80 sys_openat(d8e81a80,d8e81d00,c165238b,4,c186734f,...) at sys_openat+0x53/frame 0xdb2c9aa8 syscall(db2c9ba8) at syscall+0x6ee/frame 0xdb2c9b9c Xint0x80_syscall() at Xint0x80_syscall+0x2b/frame 0xdb2c9b9c --- syscall (499, FreeBSD ELF32, sys_openat), eip = 0x281a4f73, esp = 0xbfbfea68, ebp = 0xbfbfea90 --- KDB: enter: panic [ thread pid 59 tid 100097 ] Stopped at kdb_enter+0x3a: movl $0,kdb_why db> run pho db:0:pho> bt Tracing pid 59 tid 100097 td 0xd8e81a80 kdb_enter(c165b8f7,c165b8f7,0,db2c9468,b,...) at kdb_enter+0x3a/frame 0xdb2c9400 vpanic(c166e26f,db2c9468,c166e26f,db2c9468,db2c9468,...) at vpanic+0x150/frame 0xdb2c9438 kassert_panic(c166e26f,3,c166d1e2,6d2,6d2,...) at kassert_panic+0xd9/frame 0xdb2c945c bq_remove(c192c580,ca1994e4,c166d1e2,56f,246,...) at bq_remove+0x145/frame 0xdb2c9490 binsfree(ca1994e4,2,a67,dd2,0,...) at binsfree+0x1a9/frame 0xdb2c950c bqrelse(ca1994e4,c166d47b,d8e81a80,db2c958c,246,2cf,db2c95b0,c0982220,c1a2bcb0,ca1994e4,d936fef8,d936fe38,d8e81a80) at bqrelse+0x26b/frame 0xdb2c956c bdwrite(ca1994e4,4,1,9,0,...) at bdwrite+0x1eb/frame 0xdb2c95b0 ffs_update(d8c9accc,0,d8e98000,0,ffffffff,...) at ffs_update+0x72c/frame 0xdb2c96d0 ufs_makeinode(db2c9a28,db2c9a3c,c16ad4f4,d94b7000,db2c9888,...) at ufs_makeinode+0x31f/frame 0xdb2c9838 ufs_create(db2c98b8,c1872c37,d94b7000,db2c98ac,c16b0e0a,...) at ufs_create+0x38/frame 0xdb2c985c VOP_CREATE_APV(c1957570,db2c98b8,db2c9a3c,db2c98d0,d94de3b8,...) at VOP_CREATE_APV+0xf6/frame 0xdb2c9888 vn_open_cred(db2c99e0,db2c9a98,180,0,d8e98000,d948b738) at vn_open_cred+0x25c/frame 0xdb2c9960 vn_open(db2c99e0,db2c9a98,180,d948b738,28494409,...) at vn_open+0x3d/frame 0xdb2c9988 kern_openat(d8e81a80,4,28494409,0,100206,180) at kern_openat+0x336/frame 0xdb2c9a80 sys_openat(d8e81a80,d8e81d00,c165238b,4,c186734f,...) at sys_openat+0x53/frame 0xdb2c9aa8 syscall(db2c9ba8) at syscall+0x6ee/frame 0xdb2c9b9c Xint0x80_syscall() at Xint0x80_syscall+0x2b/frame 0xdb2c9b9c --- syscall (499, FreeBSD ELF32, sys_openat), eip = 0x281a4f73, esp = 0xbfbfea68, ebp = 0xbfbfea90 --- db:0:bt> show allpcpu Current CPU: 2 cpuid = 0 dynamic pcpu = 0x6b4880 curthread = 0xd5944700: pid 11 tid 100003 "idle: cpu0" curpcb = 0xd34d0c00 fpcurthread = none idlethread = 0xd5944700: tid 100003 "idle: cpu0" APIC ID = 0 currentldt = 0x50 curvnet = 0 spin locks held: cpuid = 1 dynamic pcpu = 0x8a8a880 curthread = 0xd5944380: pid 11 tid 100004 "idle: cpu1" curpcb = 0xd34d5c00 fpcurthread = none idlethread = 0xd5944380: tid 100004 "idle: cpu1" APIC ID = 1 currentldt = 0x50 curvnet = 0 spin locks held: cpuid = 2 dynamic pcpu = 0x8a8b880 curthread = 0xd8e81a80: pid 59 tid 100097 "mount" curpcb = 0xdb2c9c00 fpcurthread = none idlethread = 0xd5944000: tid 100005 "idle: cpu2" APIC ID = 2 currentldt = 0x50 curvnet = 0 spin locks held: cpuid = 3 dynamic pcpu = 0x8a8c880 curthread = 0xd5943a80: pid 11 tid 100006 "idle: cpu3" curpcb = 0xd34dfc00 fpcurthread = none idlethread = 0xd5943a80: tid 100006 "idle: cpu3" APIC ID = 3 currentldt = 0x50 curvnet = 0 spin locks held: db:0:allpcpu> show alllocks Process 59 (mount) thread 0xd8e81a80 (100097) exclusive sleep mutex bufq clean subqueue lock (bufq clean subqueue lock) r = 0 (0xc192c580) locked @ kern/vfs_bio.c:1365 exclusive lockmgr bufwait (bufwait) r = 0 (0xca199540) locked @ kern/vfs_bio.c:3735 exclusive lockmgr ufs (ufs) r = 0 (0xd8c9ad00) locked @ ufs/ffs/ffs_vfsops.c:1680 exclusive lockmgr ufs (ufs) r = 0 (0xd94b7034) locked @ kern/vfs_subr.c:2602 Process 14 (usb) thread 0xd8361700 (100048) exclusive sx USB config SX lock (USB config SX lock) r = 0 (0xd83cd828) locked @ dev/usb/usb_device.c:2884 Process 14 (usb) thread 0xd8923a80 (100061) exclusive sx USB config SX lock (USB config SX lock) r = 0 (0xd83cc828) locked @ dev/usb/usb_device.c:2884 db:0:alllocks> show lockedvnods Locked vnodes vnode 0xd94b7000: tag ufs, type VDIR usecount 2, writecount 0, refcount 5 mountedhere 0 flags (VI_ACTIVE) v_object 0xd8de5320 ref 0 pages 1 cleanbuf 1 dirtybuf 0 lock type ufs: EXCL by thread 0xd8e81a80 (pid 59, mount, tid 100097) #0 0xc0c4c327 at __lockmgr_args+0x667 #1 0xc0f9ef47 at ffs_lock+0xa7 #2 0xc129398a at VOP_LOCK1_APV+0xda #3 0xc0d66f31 at _vn_lock+0x61 #4 0xc0d52e61 at vget+0xc1 #5 0xc0d372eb at cache_lookup+0x97b #6 0xc0d3bc25 at vfs_cache_lookup+0xb5 #7 0xc128ffa6 at VOP_LOOKUP_APV+0xf6 #8 0xc0d45fb3 at lookup+0x663 #9 0xc0d45541 at namei+0x5f1 #10 0xc0d66603 at vn_open_cred+0xa3 #11 0xc0d6654d at vn_open+0x3d #12 0xc0d5dcb6 at kern_openat+0x336 #13 0xc0d5df73 at sys_openat+0x53 #14 0xc125df6e at syscall+0x6ee #15 0xc12473eb at Xint0x80_syscall+0x2b ino 19100960, on dev gpt/ssdrootfs vnode 0xd8c9accc: tag ufs, type VREG usecount 1, writecount 0, refcount 1 flags (VI_ACTIVE) lock type ufs: EXCL by thread 0xd8e81a80 (pid 59, mount, tid 100097) #0 0xc0c4cbba at __lockmgr_args+0xefa #1 0xc0f99d25 at ffs_vgetf+0x155 #2 0xc0f99bc7 at ffs_vget+0x37 #3 0xc0f6695d at ffs_valloc+0x67d #4 0xc0fb2f85 at ufs_makeinode+0xb5 #5 0xc0faefa8 at ufs_create+0x38 #6 0xc12903c6 at VOP_CREATE_APV+0xf6 #7 0xc0d667bc at vn_open_cred+0x25c #8 0xc0d6654d at vn_open+0x3d #9 0xc0d5dcb6 at kern_openat+0x336 #10 0xc0d5df73 at sys_openat+0x53 #11 0xc125df6e at syscall+0x6ee #12 0xc12473eb at Xint0x80_syscall+0x2b ino 19101278, on dev gpt/ssdrootfs db:0:lockedvnods> show mount 0xd936c000 /dev/gpt/ssdrootfs on / (ufs) 0xd936d000 devfs on /dev (devfs) More info: show mount db:0:mount> show page vm_cnt.v_free_count: 733017 vm_cnt.v_inactive_count: 81 vm_cnt.v_active_count: 199 vm_cnt.v_laundry_count: 0 vm_cnt.v_wire_count: 12796 vm_cnt.v_free_reserved: 1038 vm_cnt.v_free_min: 4770 vm_cnt.v_free_target: 15966 vm_cnt.v_inactive_target: 23949 db:0:page> ps pid ppid pgrp uid state wmesg wchan cmd 59 58 25 0 R+ CPU 2 mount 58 25 25 0 S+ wait 0xd839b6b0 sh 25 1 25 0 Ss+ wait 0xd839ba08 sh 24 0 0 0 DL syncer 0xc1bd11dc [syncer] 23 0 0 0 DL vlruwt 0xd8d23a08 [vnlru] 22 0 0 0 DL (threaded) [bufdaemon] 100083 D psleep 0xc1bd0b6c [bufdaemon] 100088 D - 0xc192d4c0 [bufspacedaemon-0] 100099 D sdflush 0xdcd5f884 [/ worker] 21 0 0 0 DL psleep 0xc1bd6204 [vmdaemon] 20 0 0 0 DL (threaded) [pagedaemon] 100081 D psleep 0xc1c03471 [pagedaemon] 100086 D launds 0xc1c03478 [laundry: dom0] 100087 D umarcl 0xc0fbf0c0 [uma] 19 0 0 0 DL idle 0xd839c000 [enc_daemon0] 18 0 0 0 DL - 0xc1a2bd0c [rand_harvestq] 17 0 0 0 DL waiting 0xc1c00758 [sctp_iterator] 16 0 0 0 DL - 0xc1bd0798 [soaiod4] 15 0 0 0 DL - 0xc1bd0798 [soaiod3] 9 0 0 0 DL - 0xc1bd0798 [soaiod2] 8 0 0 0 DL - 0xc1bd0798 [soaiod1] 14 0 0 0 DL (threaded) [usb] 100045 D - 0xd8227ca4 [usbus0] 100046 D - 0xd8227cd4 [usbus0] 100047 D - 0xd8227d04 [usbus0] 100048 D USBWAIT 0xc1b529aa [usbus0] 100049 D - 0xd8227d64 [usbus0] 100052 D - 0xd891ec50 [usbus1] 100053 D - 0xd891ec80 [usbus1] 100054 D - 0xd891ecb0 [usbus1] 100055 D - 0xd891ece0 [usbus1] 100056 D - 0xd891ed10 [usbus1] 100058 D - 0xd83a6ca4 [usbus2] 100059 D - 0xd83a6cd4 [usbus2] 100060 D - 0xd83a6d04 [usbus2] 100061 D USBWAIT 0xc1b529a8 [usbus2] 100062 D - 0xd83a6d64 [usbus2] 7 0 0 0 DL (threaded) [cam] 100041 D - 0xc19790c0 [doneq0] 100079 D - 0xc1978fec [scanner] 6 0 0 0 DL crypto_ 0xd5af00c4 [crypto returns 3] 5 0 0 0 DL crypto_ 0xd5af008c [crypto returns 2] 4 0 0 0 DL crypto_ 0xd5af0054 [crypto returns 1] 3 0 0 0 DL crypto_ 0xd5af001c [crypto returns 0] 2 0 0 0 DL crypto_ 0xc1bd4fb4 [crypto] 13 0 0 0 DL (threaded) [geom] 100028 D - 0xc1bfcbe8 [g_event] 100029 D - 0xc1bfcbec [g_up] 100030 D - 0xc1bfcbf0 [g_down] 12 0 0 0 WL (threaded) [intr] 100009 I [swi6: task queue] 100010 I [swi6: Giant taskq] 100012 I [swi5: fast taskq] 100022 I [swi1: netisr 0] 100023 I [swi3: vm] 100024 I [swi4: clock (0)] 100025 I [swi4: clock (1)] 100026 I [swi4: clock (2)] 100027 I [swi4: clock (3)] 100042 I [irq264: isci0] 100043 I [irq265: isci0] 100044 I [irq22: ehci0] 100050 I [irq266: hdac0] 100051 I [irq273: xhci0] 100057 I [irq20: ehci1] 100063 I [irq274: ahci0] 100064 I [irq275: hdac1] 100065 I [swi0: uart] 100066 I [irq14: ata0] 100067 I [irq15: ata1] 11 0 0 0 RL (threaded) [idle] 100003 Run CPU 0 [idle: cpu0] 100004 Run CPU 1 [idle: cpu1] 100005 CanRun [idle: cpu2] 100006 Run CPU 3 [idle: cpu3] 1 0 1 0 SLs wait 0xd5940a08 [init] 10 0 0 0 DL audit_w 0xc1c00f10 [audit] 0 0 0 0 DLs (threaded) [kernel] 100000 D swapin 0xc1bfcc04 [swapper] 100007 D - 0xd593c980 [kqueue_ctx taskq] 100008 D - 0xd593c900 [aiod_kick taskq] 100011 D - 0xd593c780 [thread taskq] 100013 D - 0xd593c680 [if_io_tqg_0] 100014 D - 0xd593c600 [if_io_tqg_1] 100015 D - 0xd593c580 [if_io_tqg_2] 100016 D - 0xd593c500 [if_io_tqg_3] 100017 D - 0xd593c480 [if_config_tqg_0] 100018 D - 0xd593c400 [softirq_0] 100019 D - 0xd593c380 [softirq_1] 100020 D - 0xd593c300 [softirq_2] 100021 D - 0xd593c280 [softirq_3] 100031 D - 0xd593c100 [firmware taskq] 100032 D - 0xd593bd80 [crypto_0] 100033 D - 0xd593bd80 [crypto_1] 100034 D - 0xd593bd80 [crypto_2] 100035 D - 0xd593bd80 [crypto_3] 100068 D - 0xd8c0e980 [mca taskq] 100074 D - 0xd8dedd80 [acpi_task_0] 100075 D - 0xd8dedd80 [acpi_task_1] 100076 D - 0xd8dedd80 [acpi_task_2] 100078 D - 0xd593bb80 [CAM taskq] 100089 D - 0xd8c0d300 [/ trim] db:0:ps> allt Tracing command mount pid 59 tid 100097 td 0xd8e81a80 (CPU 2) kdb_enter(c165b8f7,c165b8f7,0,db2c9468,b,...) at kdb_enter+0x3a/frame 0xdb2c9400 vpanic(c166e26f,db2c9468,c166e26f,db2c9468,db2c9468,...) at vpanic+0x150/frame 0xdb2c9438 kassert_panic(c166e26f,3,c166d1e2,6d2,6d2,...) at kassert_panic+0xd9/frame 0xdb2c945c bq_remove(c192c580,ca1994e4,c166d1e2,56f,246,...) at bq_remove+0x145/frame 0xdb2c9490 binsfree(ca1994e4,2,a67,dd2,0,...) at binsfree+0x1a9/frame 0xdb2c950c bqrelse(ca1994e4,c166d47b,d8e81a80,db2c958c,246,2cf,db2c95b0,c0982220,c1a2bcb0,ca1994e4,d936fef8,d936fe38,d8e81a80) at bqrelse+0x26b/frame 0xdb2c956c bdwrite(ca1994e4,4,1,9,0,...) at bdwrite+0x1eb/frame 0xdb2c95b0 ffs_update(d8c9accc,0,d8e98000,0,ffffffff,...) at ffs_update+0x72c/frame 0xdb2c96d0 ufs_makeinode(db2c9a28,db2c9a3c,c16ad4f4,d94b7000,db2c9888,...) at ufs_makeinode+0x31f/frame 0xdb2c9838 ufs_create(db2c98b8,c1872c37,d94b7000,db2c98ac,c16b0e0a,...) at ufs_create+0x38/frame 0xdb2c985c VOP_CREATE_APV(c1957570,db2c98b8,db2c9a3c,db2c98d0,d94de3b8,...) at VOP_CREATE_APV+0xf6/frame 0xdb2c9888 vn_open_cred(db2c99e0,db2c9a98,180,0,d8e98000,d948b738) at vn_open_cred+0x25c/frame 0xdb2c9960 vn_open(db2c99e0,db2c9a98,180,d948b738,28494409,...) at vn_open+0x3d/frame 0xdb2c9988 kern_openat(d8e81a80,4,28494409,0,100206,180) at kern_openat+0x336/frame 0xdb2c9a80 sys_openat(d8e81a80,d8e81d00,c165238b,4,c186734f,...) at sys_openat+0x53/frame 0xdb2c9aa8 syscall(db2c9ba8) at syscall+0x6ee/frame 0xdb2c9b9c Xint0x80_syscall() at Xint0x80_syscall+0x2b/frame 0xdb2c9b9c --- syscall (499, FreeBSD ELF32, sys_openat), eip = 0x281a4f73, esp = 0xbfbfea68, ebp = 0xbfbfea90 --- Tracing command sh pid 58 tid 100092 td 0xd8c56700 sched_switch(d8c56700,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xdb2a9800 mi_switch(104,0,c16637ed,22b,d8c56700,...) at mi_switch+0x1f3/frame 0xdb2a9840 sleepq_switch(d8c56700,0,c16637ed,1ca,6c,...) at sleepq_switch+0x110/frame 0xdb2a9874 sleepq_catch_signals(0,c16637ed,178,c1b6f114,d8c56700,...) at sleepq_catch_signals+0x360/frame 0xdb2a98b8 sleepq_wait_sig(d839b6b0,6c,c1667b66,100,0,...) at sleepq_wait_sig+0x14/frame 0xdb2a98dc _sleep(d839b6b0,d839b758,36c,c1667b66,0,...) at _sleep+0x386/frame 0xdb2a9930 kern_wait6(d8c56700,7,0,0,db2a99b4,...) at kern_wait6+0x567/frame 0xdb2a9988 sys_wait4(d8c56700,d8c56980,c165238b,4,0,...) at sys_wait4+0x87/frame 0xdb2a9aa8 syscall(db2a9ba8) at syscall+0x6ee/frame 0xdb2a9b9c Xint0x80_syscall() at Xint0x80_syscall+0x2b/frame 0xdb2a9b9c --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x281d9ec3, esp = 0xbfbfdcb8, ebp = 0xbfbfdcd0 --- Tracing command sh pid 25 tid 100090 td 0xd8c56a80 sched_switch(d8c56a80,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xdb29d800 mi_switch(104,0,c16637ed,22b,d8c56a80,...) at mi_switch+0x1f3/frame 0xdb29d840 sleepq_switch(d8c56a80,0,c16637ed,1ca,6c,...) at sleepq_switch+0x110/frame 0xdb29d874 sleepq_catch_signals(0,c16637ed,178,c1b71c14,d8c56a80,...) at sleepq_catch_signals+0x360/frame 0xdb29d8b8 sleepq_wait_sig(d839ba08,6c,c1667b66,100,0,...) at sleepq_wait_sig+0x14/frame 0xdb29d8dc _sleep(d839ba08,d839bab0,36c,c1667b66,0,...) at _sleep+0x386/frame 0xdb29d930 kern_wait6(d8c56a80,7,0,0,db29d9b4,...) at kern_wait6+0x567/frame 0xdb29d988 sys_wait4(d8c56a80,d8c56d00,c165238b,4,c186734f,...) at sys_wait4+0x87/frame 0xdb29daa8 syscall(db29dba8) at syscall+0x6ee/frame 0xdb29db9c Xint0x80_syscall() at Xint0x80_syscall+0x2b/frame 0xdb29db9c --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x281d9ec3, esp = 0xbfbfea60, ebp = 0xbfbfea78 --- Tracing command syncer pid 24 tid 100085 td 0xd8d38a80 sched_switch(d8d38a80,0,104,188,c1b613c0,...) at sched_switch+0x503/frame 0xd35eea08 mi_switch(104,0,c16637ed,22b,d8d38a80,...) at mi_switch+0x1f3/frame 0xd35eea44 sleepq_switch(d8d38a80,0,c16637ed,2d6,d8d38a80,...) at sleepq_switch+0x110/frame 0xd35eea78 sleepq_timedwait(c1bd11dc,0,d35eeadc,0,0,...) at sleepq_timedwait+0x53/frame 0xd35eeab8 _cv_timedwait_sbt(c1bd11dc,c1bd11bc,fffffed8,0,0,...) at _cv_timedwait_sbt+0x24a/frame 0xd35eeb00 sched_sync(0,d35eeba8,c165294c,406,0,...) at sched_sync+0x6e0/frame 0xd35eeb6c fork_exit(c0d583d0,0,d35eeba8) at fork_exit+0x7e/frame 0xd35eeb94 fork_trampoline() at fork_trampoline+0x8/frame 0xd35eeb94 --- trap 0, eip = 0, esp = 0xd35eebe0, ebp = 0 --- Tracing command vnlru pid 23 tid 100084 td 0xd8d51000 sched_switch(d8d51000,0,104,188,c1b613c0,...) at sched_switch+0x503/frame 0xd35e99f0 mi_switch(104,0,c16637ed,22b,d8d51000,...) at mi_switch+0x1f3/frame 0xd35e9a2c sleepq_switch(d8d51000,0,c16637ed,2d6,3,...) at sleepq_switch+0x110/frame 0xd35e9a60 sleepq_timedwait(d8d23a08,60,0,0,0,...) at sleepq_timedwait+0x53/frame 0xd35e9a9c _sleep(d8d23a08,c1bd1188,260,c1671a51,fffffed8,...) at _sleep+0x332/frame 0xd35e9af0 vnlru_proc(0,d35e9ba8,c165294c,406,0,...) at vnlru_proc+0xbc/frame 0xd35e9b6c fork_exit(c0d57a50,0,d35e9ba8) at fork_exit+0x7e/frame 0xd35e9b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd35e9b94 --- trap 0, eip = 0, esp = 0xd35e9be0, ebp = 0 --- Tracing command bufdaemon pid 22 tid 100083 td 0xd8d51380 sched_switch(d8d51380,0,104,188,c1b613c0,...) at sched_switch+0x503/frame 0xd35e49b0 mi_switch(104,0,c16637ed,22b,d8d51380,...) at mi_switch+0x1f3/frame 0xd35e49ec sleepq_switch(d8d51380,0,c16637ed,2d6,3,...) at sleepq_switch+0x110/frame 0xd35e4a20 sleepq_timedwait(c1bd0b6c,54,0,0,0,...) at sleepq_timedwait+0x53/frame 0xd35e4a5c _sleep(c1bd0b6c,c192c200,54,c166e3ef,fffffed8,...) at _sleep+0x332/frame 0xd35e4ab0 buf_daemon(0,d35e4ba8,c165294c,406,0,...) at buf_daemon+0x327/frame 0xd35e4b6c fork_exit(c0d338f0,0,d35e4ba8) at fork_exit+0x7e/frame 0xd35e4b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd35e4b94 --- trap 0, eip = 0, esp = 0xd35e4be0, ebp = 0 --- Tracing command bufdaemon pid 22 tid 100088 td 0xd8d38000 sched_switch(d8d38000,0,104,188,c1b62140,...) at sched_switch+0x503/frame 0xdb292990 mi_switch(104,0,c16637ed,22b,d8d38000,...) at mi_switch+0x1f3/frame 0xdb2929cc sleepq_switch(d8d38000,0,c16637ed,2d6,3,...) at sleepq_switch+0x110/frame 0xdb292a00 sleepq_timedwait(c192d4c0,5c,0,0,0,...) at sleepq_timedwait+0x53/frame 0xdb292a3c _sleep(c192d4c0,c192d440,25c,c1642b5b,fffffed8,...) at _sleep+0x332/frame 0xdb292a90 bufspace_daemon_wait(c192c380,c192cf80,25c,c166da5b,1999997c,...) at bufspace_daemon_wait+0x115/frame 0xdb292af4 bufspace_daemon(c192c380,db292ba8,c165294c,406,0,...) at bufspace_daemon+0x1ec/frame 0xdb292b6c fork_exit(c0d33cf0,c192c380,db292ba8) at fork_exit+0x7e/frame 0xdb292b94 fork_trampoline() at fork_trampoline+0x8/frame 0xdb292b94 --- trap 0, eip = 0, esp = 0xdb292be0, ebp = 0 --- Tracing command bufdaemon pid 22 tid 100099 td 0xd8c55700 sched_switch(d8c55700,0,104,188,c1b613c0,...) at sched_switch+0x503/frame 0xdb2d8a30 mi_switch(104,0,c16637ed,22b,d8c55700,...) at mi_switch+0x1f3/frame 0xdb2d8a6c sleepq_switch(d8c55700,0,c16637ed,2d6,3,...) at sleepq_switch+0x110/frame 0xdb2d8aa0 sleepq_timedwait(dcd5f884,54,0,0,0,...) at sleepq_timedwait+0x53/frame 0xdb2d8adc _sleep(dcd5f884,dcd5f800,54,c16a98d1,7fffff6c,...) at _sleep+0x332/frame 0xdb2d8b30 softdep_flush(d936c000,db2d8ba8,c165294c,406,0,...) at softdep_flush+0x199/frame 0xdb2d8b6c fork_exit(c0f7f5a0,d936c000,db2d8ba8) at fork_exit+0x7e/frame 0xdb2d8b94 fork_trampoline() at fork_trampoline+0x8/frame 0xdb2d8b94 --- trap 0, eip = 0, esp = 0xdb2d8be0, ebp = 0 --- Tracing command vmdaemon pid 21 tid 100082 td 0xd8d51700 sched_switch(d8d51700,0,104,188,c1b613c0,...) at sched_switch+0x503/frame 0xd35dfa10 mi_switch(104,0,c16637ed,22b,d8d51700,...) at mi_switch+0x1f3/frame 0xd35dfa4c sleepq_switch(d8d51700,0,c16637ed,2b3,d8d51700,...) at sleepq_switch+0x110/frame 0xd35dfa80 sleepq_wait(c1bd6204,74,c166e3ef,0,0,...) at sleepq_wait+0x48/frame 0xd35dfaa4 _sleep(c1bd6204,c1bd61f0,74,c166e3ef,0,...) at _sleep+0x396/frame 0xd35dfaf8 vm_daemon(0,d35dfba8,c165294c,406,0,...) at vm_daemon+0xa6/frame 0xd35dfb6c fork_exit(c0fedf60,0,d35dfba8) at fork_exit+0x7e/frame 0xd35dfb94 fork_trampoline() at fork_trampoline+0x8/frame 0xd35dfb94 --- trap 0, eip = 0, esp = 0xd35dfbe0, ebp = 0 --- Tracing command pagedaemon pid 20 tid 100081 td 0xd8d51a80 sched_switch(d8d51a80,0,104,188,c1b62140,...) at sched_switch+0x503/frame 0xd35da970 mi_switch(104,0,c16637ed,22b,d8d51a80,...) at mi_switch+0x1f3/frame 0xd35da9ac sleepq_switch(d8d51a80,0,c16637ed,2d6,3,...) at sleepq_switch+0x110/frame 0xd35da9e0 sleepq_timedwait(c1c03471,54,0,0,0,...) at sleepq_timedwait+0x53/frame 0xd35daa1c _sleep(c1c03471,c1c03300,254,c166e3ef,1999997c,...) at _sleep+0x332/frame 0xd35daa70 vm_pageout_worker(0,0,d8d2d6b0,0,0,...) at vm_pageout_worker+0x321/frame 0xd35dab3c vm_pageout(0,d35daba8,c165294c,406,0,...) at vm_pageout+0x14a/frame 0xd35dab6c fork_exit(c0fe3680,0,d35daba8) at fork_exit+0x7e/frame 0xd35dab94 fork_trampoline() at fork_trampoline+0x8/frame 0xd35dab94 --- trap 0, eip = 0, esp = 0xd35dabe0, ebp = 0 --- Tracing command pagedaemon pid 20 tid 100086 td 0xd8d38700 sched_switch(d8d38700,0,104,188,c1b613c0,...) at sched_switch+0x503/frame 0xdb2889d8 mi_switch(104,0,c16637ed,22b,d8d38700,...) at mi_switch+0x1f3/frame 0xdb288a14 sleepq_switch(d8d38700,0,c16637ed,2b3,d8d38700,...) at sleepq_switch+0x110/frame 0xdb288a48 sleepq_wait(c1c03478,54,c16b272e,0,0,...) at sleepq_wait+0x48/frame 0xdb288a6c _sleep(c1c03478,c1c02280,54,c16b272e,0,...) at _sleep+0x396/frame 0xdb288ac0 vm_pageout_laundry_worker(0,db288ba8,c165294c,406,0,...) at vm_pageout_laundry_worker+0xd18/frame 0xdb288b6c fork_exit(c0fe3830,0,db288ba8) at fork_exit+0x7e/frame 0xdb288b94 fork_trampoline() at fork_trampoline+0x8/frame 0xdb288b94 --- trap 0, eip = 0, esp = 0xdb288be0, ebp = 0 --- Tracing command pagedaemon pid 20 tid 100087 td 0xd8d38380 sched_switch(d8d38380,0,104,188,c1b613c0,...) at sched_switch+0x503/frame 0xdb28da30 mi_switch(104,0,c16637ed,22b,d8d38380,...) at mi_switch+0x1f3/frame 0xdb28da6c sleepq_switch(d8d38380,0,c16637ed,2d6,c1b6fb84,...) at sleepq_switch+0x110/frame 0xdb28daa0 sleepq_timedwait(c0fbf0c0,54,db28db10,0,0,...) at sleepq_timedwait+0x53/frame 0xdb28dae0 _sleep(c0fbf0c0,c1bd5e1c,54,c16ae86b,fffffed8,...) at _sleep+0x332/frame 0xdb28db34 uma_reclaim_worker(0,db28dba8,c165294c,406,0,...) at uma_reclaim_worker+0x147/frame 0xdb28db6c fork_exit(c0fbf750,0,db28dba8) at fork_exit+0x7e/frame 0xdb28db94 fork_trampoline() at fork_trampoline+0x8/frame 0xdb28db94 --- trap 0, eip = 0, esp = 0xdb28dbe0, ebp = 0 --- Tracing command enc_daemon0 pid 19 tid 100080 td 0xd8c57380 sched_switch(d8c57380,0,104,188,c1b613c0,...) at sched_switch+0x503/frame 0xd35d5a28 mi_switch(104,0,c16637ed,22b,d8c57380,...) at mi_switch+0x1f3/frame 0xd35d5a64 sleepq_switch(d8c57380,0,c16637ed,2b3,d8c57380,...) at sleepq_switch+0x110/frame 0xd35d5a98 sleepq_wait(d839c000,78,c12d17b0,0,0,...) at sleepq_wait+0x48/frame 0xd35d5abc _sleep(d839c000,d8bcac5c,78,c12d17b0,0,...) at _sleep+0x396/frame 0xd35d5b10 enc_daemon(d8c32000,d35d5ba8,c165294c,406,0,...) at enc_daemon+0x306/frame 0xd35d5b6c fork_exit(c05247f0,d8c32000,d35d5ba8) at fork_exit+0x7e/frame 0xd35d5b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd35d5b94 --- trap 0, eip = 0, esp = 0xd35d5be0, ebp = 0 --- Tracing command rand_harvestq pid 18 tid 100077 td 0xd8e93000 sched_switch(d8e93000,0,104,188,c1b61a80,...) at sched_switch+0x503/frame 0xd35c6a08 mi_switch(104,0,c16637ed,22b,d8e93000,...) at mi_switch+0x1f3/frame 0xd35c6a44 sleepq_switch(d8e93000,0,c16637ed,2d6,3,...) at sleepq_switch+0x110/frame 0xd35c6a78 sleepq_timedwait(c1a2bd0c,0,0,0,0,...) at sleepq_timedwait+0x53/frame 0xd35c6ab4 _sleep(c1a2bd0c,0,0,c1642b5b,19999999,...) at _sleep+0x332/frame 0xd35c6b08 random_kthread(0,d35c6ba8,c165294c,406,0,...) at random_kthread+0x7c/frame 0xd35c6b6c fork_exit(c09823c0,0,d35c6ba8) at fork_exit+0x7e/frame 0xd35c6b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd35c6b94 --- trap 0, eip = 0, esp = 0xd35c6be0, ebp = 0 --- Tracing command sctp_iterator pid 17 tid 100073 td 0xd8e94000 sched_switch(d8e94000,0,104,188,c1b60d00,...) at sched_switch+0x503/frame 0xd35b2a58 mi_switch(104,0,c16637ed,22b,d8e94000,...) at mi_switch+0x1f3/frame 0xd35b2a94 sleepq_switch(d8e94000,0,c16637ed,2b3,d8e94000,...) at sleepq_switch+0x110/frame 0xd35b2ac8 sleepq_wait(c1c00758,0,c168c26f,0,0,...) at sleepq_wait+0x48/frame 0xd35b2aec _sleep(c1c00758,c1c00720,0,c168c26f,0,...) at _sleep+0x396/frame 0xd35b2b40 sctp_iterator_thread(0,d35b2ba8,c165294c,406,0,...) at sctp_iterator_thread+0x7c/frame 0xd35b2b6c fork_exit(c0e23650,0,d35b2ba8) at fork_exit+0x7e/frame 0xd35b2b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd35b2b94 --- trap 0, eip = 0, esp = 0xd35b2be0, ebp = 0 --- Tracing command soaiod4 pid 16 tid 100072 td 0xd8d52000 sched_switch(d8d52000,0,104,188,c1b61a80,...) at sched_switch+0x503/frame 0xd35aca30 mi_switch(104,0,c16637ed,22b,d8d52000,...) at mi_switch+0x1f3/frame 0xd35aca6c sleepq_switch(d8d52000,0,c16637ed,2d6,1d,...) at sleepq_switch+0x110/frame 0xd35acaa0 sleepq_timedwait(c1bd0798,0,1d,0,0,...) at sleepq_timedwait+0x53/frame 0xd35acadc _sleep(c1bd0798,c1bd0778,0,c1642b5b,ffffdd50,...) at _sleep+0x332/frame 0xd35acb30 soaio_kproc_loop(4,d35acba8,c165294c,406,0,...) at soaio_kproc_loop+0x1a7/frame 0xd35acb6c fork_exit(c0cf06f0,4,d35acba8) at fork_exit+0x7e/frame 0xd35acb94 fork_trampoline() at fork_trampoline+0x8/frame 0xd35acb94 --- trap 0, eip = 0, esp = 0xd35acbe0, ebp = 0 --- Tracing command soaiod3 pid 15 tid 100071 td 0xd8d52380 sched_switch(d8d52380,0,104,188,c1b61a80,...) at sched_switch+0x503/frame 0xd35a7a30 mi_switch(104,0,c16637ed,22b,d8d52380,...) at mi_switch+0x1f3/frame 0xd35a7a6c sleepq_switch(d8d52380,0,c16637ed,2d6,1d,...) at sleepq_switch+0x110/frame 0xd35a7aa0 sleepq_timedwait(c1bd0798,0,1d,0,0,...) at sleepq_timedwait+0x53/frame 0xd35a7adc _sleep(c1bd0798,c1bd0778,0,c1642b5b,ffffdd50,...) at _sleep+0x332/frame 0xd35a7b30 soaio_kproc_loop(3,d35a7ba8,c165294c,406,0,...) at soaio_kproc_loop+0x1a7/frame 0xd35a7b6c fork_exit(c0cf06f0,3,d35a7ba8) at fork_exit+0x7e/frame 0xd35a7b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd35a7b94 --- trap 0, eip = 0, esp = 0xd35a7be0, ebp = 0 --- Tracing command soaiod2 pid 9 tid 100070 td 0xd8d52700 sched_switch(d8d52700,0,104,188,c1b62140,...) at sched_switch+0x503/frame 0xd35a2a30 mi_switch(104,0,c16637ed,22b,d8d52700,...) at mi_switch+0x1f3/frame 0xd35a2a6c sleepq_switch(d8d52700,0,c16637ed,2d6,1d,...) at sleepq_switch+0x110/frame 0xd35a2aa0 sleepq_timedwait(c1bd0798,0,1d,0,0,...) at sleepq_timedwait+0x53/frame 0xd35a2adc _sleep(c1bd0798,c1bd0778,0,c1642b5b,ffffdd50,...) at _sleep+0x332/frame 0xd35a2b30 soaio_kproc_loop(2,d35a2ba8,c165294c,406,0,...) at soaio_kproc_loop+0x1a7/frame 0xd35a2b6c fork_exit(c0cf06f0,2,d35a2ba8) at fork_exit+0x7e/frame 0xd35a2b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd35a2b94 --- trap 0, eip = 0, esp = 0xd35a2be0, ebp = 0 --- Tracing command soaiod1 pid 8 tid 100069 td 0xd8d52a80 sched_switch(d8d52a80,0,104,188,c1b62140,...) at sched_switch+0x503/frame 0xd359da30 mi_switch(104,0,c16637ed,22b,d8d52a80,...) at mi_switch+0x1f3/frame 0xd359da6c sleepq_switch(d8d52a80,0,c16637ed,2d6,1d,...) at sleepq_switch+0x110/frame 0xd359daa0 sleepq_timedwait(c1bd0798,0,1d,0,0,...) at sleepq_timedwait+0x53/frame 0xd359dadc _sleep(c1bd0798,c1bd0778,0,c1642b5b,ffffdd50,...) at _sleep+0x332/frame 0xd359db30 soaio_kproc_loop(1,d359dba8,c165294c,406,0,...) at soaio_kproc_loop+0x1a7/frame 0xd359db6c fork_exit(c0cf06f0,1,d359dba8) at fork_exit+0x7e/frame 0xd359db94 fork_trampoline() at fork_trampoline+0x8/frame 0xd359db94 --- trap 0, eip = 0, esp = 0xd359dbe0, ebp = 0 --- Tracing command usb pid 14 tid 100045 td 0xd8362380 sched_switch(d8362380,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd77e0a68 mi_switch(104,0,c16637ed,22b,d8362380,...) at mi_switch+0x1f3/frame 0xd77e0aa8 sleepq_switch(d8362380,0,c16637ed,2b3,c19202b8,...) at sleepq_switch+0x110/frame 0xd77e0adc sleepq_wait(d8227ca4,0,d77e0b20,1,0,...) at sleepq_wait+0x48/frame 0xd77e0b00 _cv_wait(d8227ca4,d8227ed0,c1624666,78,d8227cac,...) at _cv_wait+0x21f/frame 0xd77e0b44 usb_process(d8227c9c,d77e0ba8,c165294c,406,0,...) at usb_process+0x140/frame 0xd77e0b6c fork_exit(c0ab2be0,d8227c9c,d77e0ba8) at fork_exit+0x7e/frame 0xd77e0b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd77e0b94 --- trap 0, eip = 0, esp = 0xd77e0be0, ebp = 0 --- Tracing command usb pid 14 tid 100046 td 0xd8362000 sched_switch(d8362000,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd77e5a68 mi_switch(104,0,c16637ed,22b,d8362000,...) at mi_switch+0x1f3/frame 0xd77e5aa8 sleepq_switch(d8362000,0,c16637ed,2b3,c19202b8,...) at sleepq_switch+0x110/frame 0xd77e5adc sleepq_wait(d8227cd4,0,d77e5b20,1,0,...) at sleepq_wait+0x48/frame 0xd77e5b00 _cv_wait(d8227cd4,d8227ed0,c1624666,78,d8227cdc,...) at _cv_wait+0x21f/frame 0xd77e5b44 usb_process(d8227ccc,d77e5ba8,c165294c,406,0,...) at usb_process+0x140/frame 0xd77e5b6c fork_exit(c0ab2be0,d8227ccc,d77e5ba8) at fork_exit+0x7e/frame 0xd77e5b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd77e5b94 --- trap 0, eip = 0, esp = 0xd77e5be0, ebp = 0 --- Tracing command usb pid 14 tid 100047 td 0xd8361a80 sched_switch(d8361a80,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd77eaa68 mi_switch(104,0,c16637ed,22b,d8361a80,...) at mi_switch+0x1f3/frame 0xd77eaaa8 sleepq_switch(d8361a80,0,c16637ed,2b3,c19202b8,...) at sleepq_switch+0x110/frame 0xd77eaadc sleepq_wait(d8227d04,0,d77eab20,1,0,...) at sleepq_wait+0x48/frame 0xd77eab00 _cv_wait(d8227d04,d8227ed0,c1624666,78,d8227d0c,...) at _cv_wait+0x21f/frame 0xd77eab44 usb_process(d8227cfc,d77eaba8,c165294c,406,0,...) at usb_process+0x140/frame 0xd77eab6c fork_exit(c0ab2be0,d8227cfc,d77eaba8) at fork_exit+0x7e/frame 0xd77eab94 fork_trampoline() at fork_trampoline+0x8/frame 0xd77eab94 --- trap 0, eip = 0, esp = 0xd77eabe0, ebp = 0 --- Tracing command usb pid 14 tid 100048 td 0xd8361700 sched_switch(d8361700,0,104,188,c1b61a80,...) at sched_switch+0x503/frame 0xd77ef7d8 mi_switch(104,0,c16637ed,22b,d8361700,...) at mi_switch+0x1f3/frame 0xd77ef814 sleepq_switch(d8361700,0,c16637ed,2d6,3,...) at sleepq_switch+0x110/frame 0xd77ef848 sleepq_timedwait(c1b529aa,0,0,0,0,...) at sleepq_timedwait+0x53/frame 0xd77ef888 _sleep(c1b529aa,0,0,c16255a8,6978d484,...) at _sleep+0x332/frame 0xd77ef8dc pause_sbt(c16255a8,6978d484,0,0,0,100) at pause_sbt+0x121/frame 0xd77ef918 usb_pause_mtx(0,19b,2,8,c00c0002,...) at usb_pause_mtx+0xec/frame 0xd77ef940 uhub_attach(d8c0dd80,d8c0dd80,d77efa58,c166066b,80000003,...) at uhub_attach+0x87c/frame 0xd77efa10 device_attach(d8c0dd80,4,c165feec,b31,c0caf347,...) at device_attach+0x4e4/frame 0xd77efa6c device_probe_and_attach(d8c0dd80,d77efac8,ffffffff,b4b,1,...) at device_probe_and_attach+0x6e/frame 0xd77efa98 usb_probe_and_attach(d83cd800,ff,0,0,0,...) at usb_probe_and_attach+0x329/frame 0xd77efb0c usb_bus_attach(d8227ddc,d8227ed0,c1624666,78,d8227d3c,...) at usb_bus_attach+0x1be/frame 0xd77efb44 usb_process(d8227d2c,d77efba8,c165294c,406,0,...) at usb_process+0xf4/frame 0xd77efb6c fork_exit(c0ab2be0,d8227d2c,d77efba8) at fork_exit+0x7e/frame 0xd77efb94 fork_trampoline() at fork_trampoline+0x8/frame 0xd77efb94 --- trap 0, eip = 0, esp = 0xd77efbe0, ebp = 0 --- Tracing command usb pid 14 tid 100049 td 0xd8361380 sched_switch(d8361380,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd77f4a68 mi_switch(104,0,c16637ed,22b,d8361380,...) at mi_switch+0x1f3/frame 0xd77f4aa8 sleepq_switch(d8361380,0,c16637ed,2b3,c19202b8,...) at sleepq_switch+0x110/frame 0xd77f4adc sleepq_wait(d8227d64,0,d77f4b20,1,0,...) at sleepq_wait+0x48/frame 0xd77f4b00 _cv_wait(d8227d64,d8227ed0,c1624666,78,d8227d6c,...) at _cv_wait+0x21f/frame 0xd77f4b44 usb_process(d8227d5c,d77f4ba8,c165294c,406,0,...) at usb_process+0x140/frame 0xd77f4b6c fork_exit(c0ab2be0,d8227d5c,d77f4ba8) at fork_exit+0x7e/frame 0xd77f4b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd77f4b94 --- trap 0, eip = 0, esp = 0xd77f4be0, ebp = 0 --- Tracing command usb pid 14 tid 100052 td 0xd8358700 sched_switch(d8358700,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd3541a68 mi_switch(104,0,c16637ed,22b,d8358700,...) at mi_switch+0x1f3/frame 0xd3541aa8 sleepq_switch(d8358700,0,c16637ed,2b3,c19202b8,...) at sleepq_switch+0x110/frame 0xd3541adc sleepq_wait(d891ec50,0,d3541b20,1,0,...) at sleepq_wait+0x48/frame 0xd3541b00 _cv_wait(d891ec50,d891ee7c,c1624666,78,d891ec58,...) at _cv_wait+0x21f/frame 0xd3541b44 usb_process(d891ec48,d3541ba8,c165294c,406,0,...) at usb_process+0x140/frame 0xd3541b6c fork_exit(c0ab2be0,d891ec48,d3541ba8) at fork_exit+0x7e/frame 0xd3541b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd3541b94 --- trap 0, eip = 0, esp = 0xd3541be0, ebp = 0 --- Tracing command usb pid 14 tid 100053 td 0xd8358380 sched_switch(d8358380,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd3546a68 mi_switch(104,0,c16637ed,22b,d8358380,...) at mi_switch+0x1f3/frame 0xd3546aa8 sleepq_switch(d8358380,0,c16637ed,2b3,c19202b8,...) at sleepq_switch+0x110/frame 0xd3546adc sleepq_wait(d891ec80,0,d3546b20,1,0,...) at sleepq_wait+0x48/frame 0xd3546b00 _cv_wait(d891ec80,d891ee7c,c1624666,78,d891ec88,...) at _cv_wait+0x21f/frame 0xd3546b44 usb_process(d891ec78,d3546ba8,c165294c,406,0,...) at usb_process+0x140/frame 0xd3546b6c fork_exit(c0ab2be0,d891ec78,d3546ba8) at fork_exit+0x7e/frame 0xd3546b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd3546b94 --- trap 0, eip = 0, esp = 0xd3546be0, ebp = 0 --- Tracing command usb pid 14 tid 100054 td 0xd8358000 sched_switch(d8358000,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd354ba68 mi_switch(104,0,c16637ed,22b,d8358000,...) at mi_switch+0x1f3/frame 0xd354baa8 sleepq_switch(d8358000,0,c16637ed,2b3,c19202b8,...) at sleepq_switch+0x110/frame 0xd354badc sleepq_wait(d891ecb0,0,d354bb20,1,0,...) at sleepq_wait+0x48/frame 0xd354bb00 _cv_wait(d891ecb0,d891ee7c,c1624666,78,d891ecb8,...) at _cv_wait+0x21f/frame 0xd354bb44 usb_process(d891eca8,d354bba8,c165294c,406,0,...) at usb_process+0x140/frame 0xd354bb6c fork_exit(c0ab2be0,d891eca8,d354bba8) at fork_exit+0x7e/frame 0xd354bb94 fork_trampoline() at fork_trampoline+0x8/frame 0xd354bb94 --- trap 0, eip = 0, esp = 0xd354bbe0, ebp = 0 --- Tracing command usb pid 14 tid 100055 td 0xd8925380 sched_switch(d8925380,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd3550a68 mi_switch(104,0,c16637ed,22b,d8925380,...) at mi_switch+0x1f3/frame 0xd3550aa8 sleepq_switch(d8925380,0,c16637ed,2b3,c19202b8,...) at sleepq_switch+0x110/frame 0xd3550adc sleepq_wait(d891ece0,0,d3550b20,1,0,...) at sleepq_wait+0x48/frame 0xd3550b00 _cv_wait(d891ece0,d891ee7c,c1624666,78,d891ece8,...) at _cv_wait+0x21f/frame 0xd3550b44 usb_process(d891ecd8,d3550ba8,c165294c,406,0,...) at usb_process+0x140/frame 0xd3550b6c fork_exit(c0ab2be0,d891ecd8,d3550ba8) at fork_exit+0x7e/frame 0xd3550b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd3550b94 --- trap 0, eip = 0, esp = 0xd3550be0, ebp = 0 --- Tracing command usb pid 14 tid 100056 td 0xd8925000 sched_switch(d8925000,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd3555a68 mi_switch(104,0,c16637ed,22b,d8925000,...) at mi_switch+0x1f3/frame 0xd3555aa8 sleepq_switch(d8925000,0,c16637ed,2b3,c19202b8,...) at sleepq_switch+0x110/frame 0xd3555adc sleepq_wait(d891ed10,0,d3555b20,1,0,...) at sleepq_wait+0x48/frame 0xd3555b00 _cv_wait(d891ed10,d891ee7c,c1624666,78,d891ed18,...) at _cv_wait+0x21f/frame 0xd3555b44 usb_process(d891ed08,d3555ba8,c165294c,406,0,...) at usb_process+0x140/frame 0xd3555b6c fork_exit(c0ab2be0,d891ed08,d3555ba8) at fork_exit+0x7e/frame 0xd3555b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd3555b94 --- trap 0, eip = 0, esp = 0xd3555be0, ebp = 0 --- Tracing command usb pid 14 tid 100058 td 0xd8924700 sched_switch(d8924700,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd3560a68 mi_switch(104,0,c16637ed,22b,d8924700,...) at mi_switch+0x1f3/frame 0xd3560aa8 sleepq_switch(d8924700,0,c16637ed,2b3,c19202b8,...) at sleepq_switch+0x110/frame 0xd3560adc sleepq_wait(d83a6ca4,0,d3560b20,1,0,...) at sleepq_wait+0x48/frame 0xd3560b00 _cv_wait(d83a6ca4,d83a6ed0,c1624666,78,d83a6cac,...) at _cv_wait+0x21f/frame 0xd3560b44 usb_process(d83a6c9c,d3560ba8,c165294c,406,0,...) at usb_process+0x140/frame 0xd3560b6c fork_exit(c0ab2be0,d83a6c9c,d3560ba8) at fork_exit+0x7e/frame 0xd3560b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd3560b94 --- trap 0, eip = 0, esp = 0xd3560be0, ebp = 0 --- Tracing command usb pid 14 tid 100059 td 0xd8924380 sched_switch(d8924380,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd3565a68 mi_switch(104,0,c16637ed,22b,d8924380,...) at mi_switch+0x1f3/frame 0xd3565aa8 sleepq_switch(d8924380,0,c16637ed,2b3,c19202b8,...) at sleepq_switch+0x110/frame 0xd3565adc sleepq_wait(d83a6cd4,0,d3565b20,1,0,...) at sleepq_wait+0x48/frame 0xd3565b00 _cv_wait(d83a6cd4,d83a6ed0,c1624666,78,d83a6cdc,...) at _cv_wait+0x21f/frame 0xd3565b44 usb_process(d83a6ccc,d3565ba8,c165294c,406,0,...) at usb_process+0x140/frame 0xd3565b6c fork_exit(c0ab2be0,d83a6ccc,d3565ba8) at fork_exit+0x7e/frame 0xd3565b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd3565b94 --- trap 0, eip = 0, esp = 0xd3565be0, ebp = 0 --- Tracing command usb pid 14 tid 100060 td 0xd8924000 sched_switch(d8924000,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd356aa68 mi_switch(104,0,c16637ed,22b,d8924000,...) at mi_switch+0x1f3/frame 0xd356aaa8 sleepq_switch(d8924000,0,c16637ed,2b3,c19202b8,...) at sleepq_switch+0x110/frame 0xd356aadc sleepq_wait(d83a6d04,0,d356ab20,1,0,...) at sleepq_wait+0x48/frame 0xd356ab00 _cv_wait(d83a6d04,d83a6ed0,c1624666,78,d83a6d0c,...) at _cv_wait+0x21f/frame 0xd356ab44 usb_process(d83a6cfc,d356aba8,c165294c,406,0,...) at usb_process+0x140/frame 0xd356ab6c fork_exit(c0ab2be0,d83a6cfc,d356aba8) at fork_exit+0x7e/frame 0xd356ab94 fork_trampoline() at fork_trampoline+0x8/frame 0xd356ab94 --- trap 0, eip = 0, esp = 0xd356abe0, ebp = 0 --- Tracing command usb pid 14 tid 100061 td 0xd8923a80 sched_switch(d8923a80,0,104,188,c1b60d00,...) at sched_switch+0x503/frame 0xd356f7d8 mi_switch(104,0,c16637ed,22b,d8923a80,...) at mi_switch+0x1f3/frame 0xd356f814 sleepq_switch(d8923a80,0,c16637ed,2d6,3,...) at sleepq_switch+0x110/frame 0xd356f848 sleepq_timedwait(c1b529a8,0,0,0,0,...) at sleepq_timedwait+0x53/frame 0xd356f888 _sleep(c1b529a8,0,0,c16255a8,6978d484,...) at _sleep+0x332/frame 0xd356f8dc pause_sbt(c16255a8,6978d484,0,0,0,100) at pause_sbt+0x121/frame 0xd356f918 usb_pause_mtx(0,19b,2,8,c00c0002,...) at usb_pause_mtx+0xec/frame 0xd356f940 uhub_attach(d8dedc00,d5b32064,c1914fcc,c166066b,80000003,...) at uhub_attach+0x87c/frame 0xd356fa10 device_attach(d8dedc00,4,c165feec,b31,c0caf347,...) at device_attach+0x4e4/frame 0xd356fa6c device_probe_and_attach(d8dedc00,d356fac8,ffffffff,b4b,1,...) at device_probe_and_attach+0x6e/frame 0xd356fa98 usb_probe_and_attach(d83cc800,ff,0,0,0,...) at usb_probe_and_attach+0x329/frame 0xd356fb0c usb_bus_attach(d83a6ddc,d83a6ed0,c1624666,78,d83a6d3c,...) at usb_bus_attach+0x1be/frame 0xd356fb44 usb_process(d83a6d2c,d356fba8,c165294c,406,0,...) at usb_process+0xf4/frame 0xd356fb6c fork_exit(c0ab2be0,d83a6d2c,d356fba8) at fork_exit+0x7e/frame 0xd356fb94 fork_trampoline() at fork_trampoline+0x8/frame 0xd356fb94 --- trap 0, eip = 0, esp = 0xd356fbe0, ebp = 0 --- Tracing command usb pid 14 tid 100062 td 0xd8923700 sched_switch(d8923700,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd3574a68 mi_switch(104,0,c16637ed,22b,d8923700,...) at mi_switch+0x1f3/frame 0xd3574aa8 sleepq_switch(d8923700,0,c16637ed,2b3,c19202b8,...) at sleepq_switch+0x110/frame 0xd3574adc sleepq_wait(d83a6d64,0,d3574b20,1,0,...) at sleepq_wait+0x48/frame 0xd3574b00 _cv_wait(d83a6d64,d83a6ed0,c1624666,78,d83a6d6c,...) at _cv_wait+0x21f/frame 0xd3574b44 usb_process(d83a6d5c,d3574ba8,c165294c,406,0,...) at usb_process+0x140/frame 0xd3574b6c fork_exit(c0ab2be0,d83a6d5c,d3574ba8) at fork_exit+0x7e/frame 0xd3574b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd3574b94 --- trap 0, eip = 0, esp = 0xd3574be0, ebp = 0 --- Tracing command cam pid 7 tid 100041 td 0xd5af6700 sched_switch(d5af6700,0,104,188,c1b60d00,...) at sched_switch+0x503/frame 0xd76aea38 mi_switch(104,0,c16637ed,22b,d5af6700,...) at mi_switch+0x1f3/frame 0xd76aea74 sleepq_switch(d5af6700,0,c16637ed,2b3,d5af6700,...) at sleepq_switch+0x110/frame 0xd76aeaa8 sleepq_wait(c19790c0,5c,c1642b5b,0,0,...) at sleepq_wait+0x48/frame 0xd76aeacc _sleep(c19790c0,c1979080,5c,c1642b5b,0,...) at _sleep+0x396/frame 0xd76aeb20 xpt_done_td(c1979080,d76aeba8,c165294c,406,0,...) at xpt_done_td+0x9b/frame 0xd76aeb6c fork_exit(c04e2830,c1979080,d76aeba8) at fork_exit+0x7e/frame 0xd76aeb94 fork_trampoline() at fork_trampoline+0x8/frame 0xd76aeb94 --- trap 0, eip = 0, esp = 0xd76aebe0, ebp = 0 --- Tracing command cam pid 7 tid 100079 td 0xd8e92700 sched_switch(d8e92700,0,104,188,c1b60d00,...) at sched_switch+0x503/frame 0xd35d0a30 mi_switch(104,0,c16637ed,22b,d8e92700,...) at mi_switch+0x1f3/frame 0xd35d0a6c sleepq_switch(d8e92700,0,c16637ed,2b3,d8e92700,...) at sleepq_switch+0x110/frame 0xd35d0aa0 sleepq_wait(c1978fec,5c,c1642b5b,0,0,...) at sleepq_wait+0x48/frame 0xd35d0ac4 _sleep(c1978fec,c1979040,5c,c1642b5b,0,...) at _sleep+0x396/frame 0xd35d0b18 xpt_scanner_thread(0,d35d0ba8,c165294c,406,0,...) at xpt_scanner_thread+0x97/frame 0xd35d0b6c fork_exit(c04e2bb0,0,d35d0ba8) at fork_exit+0x7e/frame 0xd35d0b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd35d0b94 --- trap 0, eip = 0, esp = 0xd35d0be0, ebp = 0 --- Tracing command crypto returns 3 pid 6 tid 100040 td 0xd5af6a80 sched_switch(d5af6a80,0,104,188,c1b62140,...) at sched_switch+0x503/frame 0xd76a9a30 mi_switch(104,0,c16637ed,22b,d5af6a80,...) at mi_switch+0x1f3/frame 0xd76a9a6c sleepq_switch(d5af6a80,0,c16637ed,2b3,d5af6a80,...) at sleepq_switch+0x110/frame 0xd76a9aa0 sleepq_wait(d5af00c4,6c,c16a0b59,0,0,...) at sleepq_wait+0x48/frame 0xd76a9ac4 _sleep(d5af00c4,d5af00a8,6c,c16a0b59,0,...) at _sleep+0x396/frame 0xd76a9b18 crypto_ret_proc(d5af00a8,d76a9ba8,c165294c,406,0,...) at crypto_ret_proc+0x241/frame 0xd76a9b6c fork_exit(c0f24ba0,d5af00a8,d76a9ba8) at fork_exit+0x7e/frame 0xd76a9b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd76a9b94 --- trap 0, eip = 0, esp = 0xd76a9be0, ebp = 0 --- Tracing command crypto returns 2 pid 5 tid 100039 td 0xd59a0700 sched_switch(d59a0700,0,104,188,c1b62140,...) at sched_switch+0x503/frame 0xd76a4a30 mi_switch(104,0,c16637ed,22b,d59a0700,...) at mi_switch+0x1f3/frame 0xd76a4a6c sleepq_switch(d59a0700,0,c16637ed,2b3,d59a0700,...) at sleepq_switch+0x110/frame 0xd76a4aa0 sleepq_wait(d5af008c,6c,c16a0b59,0,0,...) at sleepq_wait+0x48/frame 0xd76a4ac4 _sleep(d5af008c,d5af0070,6c,c16a0b59,0,...) at _sleep+0x396/frame 0xd76a4b18 crypto_ret_proc(d5af0070,d76a4ba8,c165294c,406,0,...) at crypto_ret_proc+0x241/frame 0xd76a4b6c fork_exit(c0f24ba0,d5af0070,d76a4ba8) at fork_exit+0x7e/frame 0xd76a4b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd76a4b94 --- trap 0, eip = 0, esp = 0xd76a4be0, ebp = 0 --- Tracing command crypto returns 1 pid 4 tid 100038 td 0xd59a0a80 sched_switch(d59a0a80,0,104,188,c1b62140,...) at sched_switch+0x503/frame 0xd769fa30 mi_switch(104,0,c16637ed,22b,d59a0a80,...) at mi_switch+0x1f3/frame 0xd769fa6c sleepq_switch(d59a0a80,0,c16637ed,2b3,d59a0a80,...) at sleepq_switch+0x110/frame 0xd769faa0 sleepq_wait(d5af0054,6c,c16a0b59,0,0,...) at sleepq_wait+0x48/frame 0xd769fac4 _sleep(d5af0054,d5af0038,6c,c16a0b59,0,...) at _sleep+0x396/frame 0xd769fb18 crypto_ret_proc(d5af0038,d769fba8,c165294c,406,0,...) at crypto_ret_proc+0x241/frame 0xd769fb6c fork_exit(c0f24ba0,d5af0038,d769fba8) at fork_exit+0x7e/frame 0xd769fb94 fork_trampoline() at fork_trampoline+0x8/frame 0xd769fb94 --- trap 0, eip = 0, esp = 0xd769fbe0, ebp = 0 --- Tracing command crypto returns 0 pid 3 tid 100037 td 0xd5a16000 sched_switch(d5a16000,0,104,188,c1b61a80,...) at sched_switch+0x503/frame 0xd769aa30 mi_switch(104,0,c16637ed,22b,d5a16000,...) at mi_switch+0x1f3/frame 0xd769aa6c sleepq_switch(d5a16000,0,c16637ed,2b3,d5a16000,...) at sleepq_switch+0x110/frame 0xd769aaa0 sleepq_wait(d5af001c,6c,c16a0b59,0,0,...) at sleepq_wait+0x48/frame 0xd769aac4 _sleep(d5af001c,d5af0000,6c,c16a0b59,0,...) at _sleep+0x396/frame 0xd769ab18 crypto_ret_proc(d5af0000,d769aba8,c165294c,406,0,...) at crypto_ret_proc+0x241/frame 0xd769ab6c fork_exit(c0f24ba0,d5af0000,d769aba8) at fork_exit+0x7e/frame 0xd769ab94 fork_trampoline() at fork_trampoline+0x8/frame 0xd769ab94 --- trap 0, eip = 0, esp = 0xd769abe0, ebp = 0 --- Tracing command crypto pid 2 tid 100036 td 0xd5a16380 sched_switch(d5a16380,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd7695a40 mi_switch(104,0,c16637ed,22b,d5a16380,...) at mi_switch+0x1f3/frame 0xd7695a80 sleepq_switch(d5a16380,0,c16637ed,2b3,d5a16380,...) at sleepq_switch+0x110/frame 0xd7695ab4 sleepq_wait(c1bd4fb4,6c,c16a0b4d,0,0,...) at sleepq_wait+0x48/frame 0xd7695ad8 _sleep(c1bd4fb4,c1bd4f9c,6c,c16a0b4d,0,...) at _sleep+0x396/frame 0xd7695b2c crypto_proc(0,d7695ba8,c165294c,406,0,...) at crypto_proc+0x2a7/frame 0xd7695b6c fork_exit(c0f247a0,0,d7695ba8) at fork_exit+0x7e/frame 0xd7695b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd7695b94 --- trap 0, eip = 0, esp = 0xd7695be0, ebp = 0 --- Tracing command geom pid 13 tid 100028 td 0xd5a1c380 sched_switch(d5a1c380,0,104,188,c1b60d00,...) at sched_switch+0x503/frame 0xd766da48 mi_switch(104,0,c16637ed,22b,d5a1c380,...) at mi_switch+0x1f3/frame 0xd766da84 sleepq_switch(d5a1c380,0,c16637ed,2b3,d5a1c380,...) at sleepq_switch+0x110/frame 0xd766dab8 sleepq_wait(c1bfcbe8,5c,c1642b5b,0,0,...) at sleepq_wait+0x48/frame 0xd766dadc _sleep(c1bfcbe8,c1b50798,25c,c1642b5b,0,...) at _sleep+0x396/frame 0xd766db30 g_run_events(0,d766dba8,c165294c,406,0,...) at g_run_events+0x67/frame 0xd766db6c fork_exit(c0bbb7e0,0,d766dba8) at fork_exit+0x7e/frame 0xd766db94 fork_trampoline() at fork_trampoline+0x8/frame 0xd766db94 --- trap 0, eip = 0, esp = 0xd766dbe0, ebp = 0 --- Tracing command geom pid 13 tid 100029 td 0xd5a1c000 sched_switch(d5a1c000,0,104,188,c1b613c0,...) at sched_switch+0x503/frame 0xd7672a38 mi_switch(104,0,c16637ed,22b,d5a1c000,...) at mi_switch+0x1f3/frame 0xd7672a74 sleepq_switch(d5a1c000,0,c16637ed,2b3,d5a1c000,...) at sleepq_switch+0x110/frame 0xd7672aa8 sleepq_wait(c1bfcbec,5c,c1642b5b,0,0,...) at sleepq_wait+0x48/frame 0xd7672acc _sleep(c1bfcbec,c1b507e0,25c,c1642b5b,0,...) at _sleep+0x396/frame 0xd7672b20 g_io_schedule_up(d5a1c000,0,c1644be2,61,d7672b94,...) at g_io_schedule_up+0xcf/frame 0xd7672b54 g_up_procbody(0,d7672ba8,c165294c,406,0,...) at g_up_procbody+0x7d/frame 0xd7672b6c fork_exit(c0bbb860,0,d7672ba8) at fork_exit+0x7e/frame 0xd7672b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd7672b94 --- trap 0, eip = 0, esp = 0xd7672be0, ebp = 0 --- Tracing command geom pid 13 tid 100030 td 0xd5a17a80 sched_switch(d5a17a80,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd7677a30 mi_switch(104,0,c16637ed,22b,d5a17a80,...) at mi_switch+0x1f3/frame 0xd7677a70 sleepq_switch(d5a17a80,0,c16637ed,2b3,d5a17a80,...) at sleepq_switch+0x110/frame 0xd7677aa4 sleepq_wait(c1bfcbf0,5c,c1642b5b,0,0,...) at sleepq_wait+0x48/frame 0xd7677ac8 _sleep(c1bfcbf0,c1b507c0,25c,c1642b5b,0,...) at _sleep+0x396/frame 0xd7677b1c g_io_schedule_down(d5a17a80,0,c1644be2,6d,d7677b94,...) at g_io_schedule_down+0xec/frame 0xd7677b54 g_down_procbody(0,d7677ba8,c165294c,406,0,...) at g_down_procbody+0x7d/frame 0xd7677b6c fork_exit(c0bbb8e0,0,d7677ba8) at fork_exit+0x7e/frame 0xd7677b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd7677b94 --- trap 0, eip = 0, esp = 0xd7677be0, ebp = 0 --- Tracing command intr pid 12 tid 100009 td 0xd5943000 sched_switch(d5943000,0,109,188,8,...) at sched_switch+0x503/frame 0xd760eaf0 mi_switch(109,0,c1652cf1,5a1,c165294c,...) at mi_switch+0x1f3/frame 0xd760eb30 ithread_loop(d5937160,d760eba8,c165294c,406,0,...) at ithread_loop+0x200/frame 0xd760eb6c fork_exit(c0c39e60,d5937160,d760eba8) at fork_exit+0x7e/frame 0xd760eb94 fork_trampoline() at fork_trampoline+0x8/frame 0xd760eb94 --- trap 0, eip = 0, esp = 0xd760ebe0, ebp = 0 --- Tracing command intr pid 12 tid 100010 td 0xd5942a80 sched_switch(d5942a80,0,109,188,8,...) at sched_switch+0x503/frame 0xd7613af0 mi_switch(109,0,c1652cf1,5a1,c165294c,...) at mi_switch+0x1f3/frame 0xd7613b30 ithread_loop(d5937150,d7613ba8,c165294c,406,0,...) at ithread_loop+0x200/frame 0xd7613b6c fork_exit(c0c39e60,d5937150,d7613ba8) at fork_exit+0x7e/frame 0xd7613b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd7613b94 --- trap 0, eip = 0, esp = 0xd7613be0, ebp = 0 --- Tracing command intr pid 12 tid 100012 td 0xd5942380 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100022 td 0xd599e380 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100023 td 0xd599e000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100024 td 0xd5945a80 sched_switch(d5945a80,0,109,188,8,...) at sched_switch+0x503/frame 0xd7659af0 mi_switch(109,0,c1652cf1,5a1,c165294c,...) at mi_switch+0x1f3/frame 0xd7659b30 ithread_loop(d5937110,d7659ba8,c165294c,406,0,...) at ithread_loop+0x200/frame 0xd7659b6c fork_exit(c0c39e60,d5937110,d7659ba8) at fork_exit+0x7e/frame 0xd7659b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd7659b94 --- trap 0, eip = 0, esp = 0xd7659be0, ebp = 0 --- Tracing command intr pid 12 tid 100025 td 0xd5945700 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100026 td 0xd5945380 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100027 td 0xd5a1c700 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100042 td 0xd8363000 sched_switch(d8363000,0,109,188,8,...) at sched_switch+0x503/frame 0xd77d0af0 mi_switch(109,0,c1652cf1,5a1,c165294c,...) at mi_switch+0x1f3/frame 0xd77d0b30 ithread_loop(d8357050,d77d0ba8,c165294c,406,0,...) at ithread_loop+0x200/frame 0xd77d0b6c fork_exit(c0c39e60,d8357050,d77d0ba8) at fork_exit+0x7e/frame 0xd77d0b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd77d0b94 --- trap 0, eip = 0, esp = 0xd77d0be0, ebp = 0 --- Tracing command intr pid 12 tid 100043 td 0xd8362a80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100044 td 0xd8362700 sched_switch(d8362700,0,109,188,8,...) at sched_switch+0x503/frame 0xd77dbaf0 mi_switch(109,0,c1652cf1,5a1,c165294c,...) at mi_switch+0x1f3/frame 0xd77dbb30 ithread_loop(d8357030,d77dbba8,c165294c,406,0,...) at ithread_loop+0x200/frame 0xd77dbb6c fork_exit(c0c39e60,d8357030,d77dbba8) at fork_exit+0x7e/frame 0xd77dbb94 fork_trampoline() at fork_trampoline+0x8/frame 0xd77dbb94 --- trap 0, eip = 0, esp = 0xd77dbbe0, ebp = 0 --- Tracing command intr pid 12 tid 100050 td 0xd8361000 sched_switch(d8361000,0,109,188,8,...) at sched_switch+0x503/frame 0xd77fdaf0 mi_switch(109,0,c1652cf1,5a1,406,...) at mi_switch+0x1f3/frame 0xd77fdb30 ithread_loop(d8357020,d77fdba8,c165294c,406,0,...) at ithread_loop+0x200/frame 0xd77fdb6c fork_exit(c0c39e60,d8357020,d77fdba8) at fork_exit+0x7e/frame 0xd77fdb94 fork_trampoline() at fork_trampoline+0x8/frame 0xd77fdb94 --- trap 0, eip = 0, esp = 0xd77fdbe0, ebp = 0 --- Tracing command intr pid 12 tid 100051 td 0xd8358a80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100057 td 0xd8924a80 sched_switch(d8924a80,0,109,188,8,...) at sched_switch+0x503/frame 0xd355baf0 mi_switch(109,0,c1652cf1,5a1,c165294c,...) at mi_switch+0x1f3/frame 0xd355bb30 ithread_loop(d5938f80,d355bba8,c165294c,406,0,...) at ithread_loop+0x200/frame 0xd355bb6c fork_exit(c0c39e60,d5938f80,d355bba8) at fork_exit+0x7e/frame 0xd355bb94 fork_trampoline() at fork_trampoline+0x8/frame 0xd355bb94 --- trap 0, eip = 0, esp = 0xd355bbe0, ebp = 0 --- Tracing command intr pid 12 tid 100063 td 0xd8923380 sched_switch(d8923380,0,109,188,8,...) at sched_switch+0x503/frame 0xd357aaf0 mi_switch(109,0,c1652cf1,5a1,c165294c,...) at mi_switch+0x1f3/frame 0xd357ab30 ithread_loop(d5938f70,d357aba8,c165294c,406,0,...) at ithread_loop+0x200/frame 0xd357ab6c fork_exit(c0c39e60,d5938f70,d357aba8) at fork_exit+0x7e/frame 0xd357ab94 fork_trampoline() at fork_trampoline+0x8/frame 0xd357ab94 --- trap 0, eip = 0, esp = 0xd357abe0, ebp = 0 --- Tracing command intr pid 12 tid 100064 td 0xd8923000 sched_switch(d8923000,0,109,188,8,...) at sched_switch+0x503/frame 0xd3583af0 mi_switch(109,0,c1652cf1,5a1,c165294c,...) at mi_switch+0x1f3/frame 0xd3583b30 ithread_loop(d5938f60,d3583ba8,c165294c,406,0,...) at ithread_loop+0x200/frame 0xd3583b6c fork_exit(c0c39e60,d5938f60,d3583ba8) at fork_exit+0x7e/frame 0xd3583b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd3583b94 --- trap 0, eip = 0, esp = 0xd3583be0, ebp = 0 --- Tracing command intr pid 12 tid 100065 td 0xd8363a80 sched_switch(d8363a80,0,109,188,8,...) at sched_switch+0x503/frame 0xd3589af0 mi_switch(109,0,c1652cf1,5a1,c165294c,...) at mi_switch+0x1f3/frame 0xd3589b30 ithread_loop(d5938f50,d3589ba8,c165294c,406,0,...) at ithread_loop+0x200/frame 0xd3589b6c fork_exit(c0c39e60,d5938f50,d3589ba8) at fork_exit+0x7e/frame 0xd3589b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd3589b94 --- trap 0, eip = 0, esp = 0xd3589be0, ebp = 0 --- Tracing command intr pid 12 tid 100066 td 0xd8363700 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100067 td 0xd8363380 fork_trampoline() at fork_trampoline Tracing command idle pid 11 tid 100003 td 0xd5944700 (CPU 0) cpustop_handler(0,d34d0ab4,d34d0aa8,c125c7f3,d34d0a00,...) at cpustop_handler+0x25/frame 0xd34d09a0 ipi_nmi_handler(d34d0a00,c1264ec3,c2027c50,8,c186734f,...) at ipi_nmi_handler+0x36/frame 0xd34d09b0 trap(d34d0ab4) at trap+0xa3/frame 0xd34d0aa8 calltrap() at calltrap+0x6/frame 0xd34d0aa8 --- trap 0x13, eip = 0xc1276475, esp = 0xd34d0af4, ebp = 0xd34d0b08 --- cpu_idle(1,d34d0b40,c165eea8,a74,2a0b2b00,...) at cpu_idle+0xd5/frame 0xd34d0b08 sched_idletd(0,d34d0ba8,c165294c,406,0,...) at sched_idletd+0x3a6/frame 0xd34d0b6c fork_exit(c0ca8c70,0,d34d0ba8) at fork_exit+0x7e/frame 0xd34d0b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd34d0b94 --- trap 0, eip = 0, esp = 0xd34d0be0, ebp = 0 --- Tracing command idle pid 11 tid 100004 td 0xd5944380 (CPU 1) cpustop_handler(1,d34d5ab4,d34d5aa8,c125c7f3,d34d5a00,...) at cpustop_handler+0x25/frame 0xd34d59a0 ipi_nmi_handler(d34d5a00,c1264ec3,ca3fdc50,8,c186734f,...) at ipi_nmi_handler+0x36/frame 0xd34d59b0 trap(d34d5ab4) at trap+0xa3/frame 0xd34d5aa8 calltrap() at calltrap+0x6/frame 0xd34d5aa8 --- trap 0x13, eip = 0xc1276475, esp = 0xd34d5af4, ebp = 0xd34d5b08 --- cpu_idle(1,d34d5b40,c165eea8,a74,0,...) at cpu_idle+0xd5/frame 0xd34d5b08 sched_idletd(0,d34d5ba8,c165294c,406,0,...) at sched_idletd+0x3a6/frame 0xd34d5b6c fork_exit(c0ca8c70,0,d34d5ba8) at fork_exit+0x7e/frame 0xd34d5b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd34d5b94 --- trap 0, eip = 0, esp = 0xd34d5be0, ebp = 0 --- Tracing command idle pid 11 tid 100005 td 0xd5944000 sched_switch(d5944000,0,108,184,8,...) at sched_switch+0x503/frame 0xd34daac8 mi_switch(108,0,c165eea8,a72,0,...) at mi_switch+0x1f3/frame 0xd34dab08 sched_idletd(0,d34daba8,c165294c,406,0,...) at sched_idletd+0xb0/frame 0xd34dab6c fork_exit(c0ca8c70,0,d34daba8) at fork_exit+0x7e/frame 0xd34dab94 fork_trampoline() at fork_trampoline+0x8/frame 0xd34dab94 --- trap 0, eip = 0, esp = 0xd34dabe0, ebp = 0 --- Tracing command idle pid 11 tid 100006 td 0xd5943a80 (CPU 3) cpustop_handler(3,d34dfab4,d34dfaa8,c125c7f3,d34dfa00,...) at cpustop_handler+0x25/frame 0xd34df9a0 ipi_nmi_handler(d34dfa00,c1264ec3,ca3ffc50,8,c186734f,...) at ipi_nmi_handler+0x36/frame 0xd34df9b0 trap(d34dfab4) at trap+0xa3/frame 0xd34dfaa8 calltrap() at calltrap+0x6/frame 0xd34dfaa8 --- trap 0x13, eip = 0xc1276475, esp = 0xd34dfaf4, ebp = 0xd34dfb08 --- cpu_idle(1,d34dfb40,c165eea8,a74,0,...) at cpu_idle+0xd5/frame 0xd34dfb08 sched_idletd(0,d34dfba8,c165294c,406,0,...) at sched_idletd+0x3a6/frame 0xd34dfb6c fork_exit(c0ca8c70,0,d34dfba8) at fork_exit+0x7e/frame 0xd34dfb94 fork_trampoline() at fork_trampoline+0x8/frame 0xd34dfb94 --- trap 0, eip = 0, esp = 0xd34dfbe0, ebp = 0 --- Tracing command init pid 1 tid 100002 td 0xd5944a80 sched_switch(d5944a80,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xc7ffa800 mi_switch(104,0,c16637ed,22b,d5944a80,...) at mi_switch+0x1f3/frame 0xc7ffa840 sleepq_switch(d5944a80,0,c16637ed,1ca,6c,...) at sleepq_switch+0x110/frame 0xc7ffa874 sleepq_catch_signals(0,c16637ed,178,c1b6f014,d5944a80,...) at sleepq_catch_signals+0x360/frame 0xc7ffa8b8 sleepq_wait_sig(d5940a08,6c,c1667b66,100,0,...) at sleepq_wait_sig+0x14/frame 0xc7ffa8dc _sleep(d5940a08,d5940ab0,36c,c1667b66,0,...) at _sleep+0x386/frame 0xc7ffa930 kern_wait6(d5944a80,7,0,0,c7ffa9b4,...) at kern_wait6+0x567/frame 0xc7ffa988 sys_wait4(d5944a80,d5944d00,c165238b,4,c19202b8,...) at sys_wait4+0x87/frame 0xc7ffaaa8 syscall(c7ffaba8) at syscall+0x6ee/frame 0xc7ffab9c Xint0x80_syscall() at Xint0x80_syscall+0x2b/frame 0xc7ffab9c --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x806a257, esp = 0xbfbfe864, ebp = 0xbfbfe878 --- Tracing command audit pid 10 tid 100001 td 0xd5945000 sched_switch(d5945000,0,104,188,c1b613c0,...) at sched_switch+0x503/frame 0xc7ff5a40 mi_switch(104,0,c16637ed,22b,d5945000,...) at mi_switch+0x1f3/frame 0xc7ff5a7c sleepq_switch(d5945000,0,c16637ed,2b3,c19202b8,...) at sleepq_switch+0x110/frame 0xc7ff5ab0 sleepq_wait(c1c00f10,0,c7ff5af4,1,0,...) at sleepq_wait+0x48/frame 0xc7ff5ad4 _cv_wait(c1c00f10,c1c00e98,c16a32be,1be,8,...) at _cv_wait+0x21f/frame 0xc7ff5b18 audit_worker(0,c7ff5ba8,c165294c,406,0,...) at audit_worker+0x94/frame 0xc7ff5b6c fork_exit(c0f46ec0,0,c7ff5ba8) at fork_exit+0x7e/frame 0xc7ff5b94 fork_trampoline() at fork_trampoline+0x8/frame 0xc7ff5b94 --- trap 0, eip = 0, esp = 0xc7ff5be0, ebp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xc1bfcf60 sched_switch(c1bfcf60,0,104,188,c1b613c0,...) at sched_switch+0x503/frame 0xc2022ab0 mi_switch(104,0,c16637ed,22b,c1bfcf60,...) at mi_switch+0x1f3/frame 0xc2022aec sleepq_switch(c1bfcf60,0,c16637ed,2d6,c,...) at sleepq_switch+0x110/frame 0xc2022b20 sleepq_timedwait(c1bfcc04,54,9,0,0,...) at sleepq_timedwait+0x53/frame 0xc2022b5c _sleep(c1bfcc04,0,54,c16b400c,fffff470,...) at _sleep+0x332/frame 0xc2022bb0 swapper() at swapper+0x2c8/frame 0xc2022bf8 begin() at begin+0x2f Tracing command kernel pid 0 tid 100007 td 0xd5943700 sched_switch(d5943700,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd7604a48 mi_switch(104,0,c16637ed,22b,d5943700,...) at mi_switch+0x1f3/frame 0xd7604a88 sleepq_switch(d5943700,0,c16637ed,2b3,d5943700,...) at sleepq_switch+0x110/frame 0xd7604abc sleepq_wait(d593c980,0,c1642b5b,0,0,...) at sleepq_wait+0x48/frame 0xd7604ae0 _sleep(d593c980,d593c99c,0,c1642b5b,0,...) at _sleep+0x396/frame 0xd7604b34 taskqueue_thread_loop(c1bfd4f8,d7604ba8,c165294c,406,0,...) at taskqueue_thread_loop+0x144/frame 0xd7604b6c fork_exit(c0cd47f0,c1bfd4f8,d7604ba8) at fork_exit+0x7e/frame 0xd7604b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd7604b94 --- trap 0, eip = 0, esp = 0xd7604be0, ebp = 0 --- Tracing command kernel pid 0 tid 100008 td 0xd5943380 sched_switch(d5943380,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd7609a48 mi_switch(104,0,c16637ed,22b,d5943380,...) at mi_switch+0x1f3/frame 0xd7609a88 sleepq_switch(d5943380,0,c16637ed,2b3,d5943380,...) at sleepq_switch+0x110/frame 0xd7609abc sleepq_wait(d593c900,0,c1642b5b,0,0,...) at sleepq_wait+0x48/frame 0xd7609ae0 _sleep(d593c900,d593c91c,0,c1642b5b,0,...) at _sleep+0x396/frame 0xd7609b34 taskqueue_thread_loop(c1c002f8,d7609ba8,c165294c,406,0,...) at taskqueue_thread_loop+0x144/frame 0xd7609b6c fork_exit(c0cd47f0,c1c002f8,d7609ba8) at fork_exit+0x7e/frame 0xd7609b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd7609b94 --- trap 0, eip = 0, esp = 0xd7609be0, ebp = 0 --- Tracing command kernel pid 0 tid 100011 td 0xd5942700 sched_switch(d5942700,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd7618a48 mi_switch(104,0,c16637ed,22b,d5942700,...) at mi_switch+0x1f3/frame 0xd7618a88 sleepq_switch(d5942700,0,c16637ed,2b3,d5942700,...) at sleepq_switch+0x110/frame 0xd7618abc sleepq_wait(d593c780,0,c1642b5b,0,0,...) at sleepq_wait+0x48/frame 0xd7618ae0 _sleep(d593c780,d593c79c,0,c1642b5b,0,...) at _sleep+0x396/frame 0xd7618b34 taskqueue_thread_loop(c1c0029c,d7618ba8,c165294c,406,0,...) at taskqueue_thread_loop+0x144/frame 0xd7618b6c fork_exit(c0cd47f0,c1c0029c,d7618ba8) at fork_exit+0x7e/frame 0xd7618b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd7618b94 --- trap 0, eip = 0, esp = 0xd7618be0, ebp = 0 --- Tracing command kernel pid 0 tid 100013 td 0xd5942000 sched_switch(d5942000,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd7622a58 mi_switch(104,0,c16637ed,22b,d5942000,...) at mi_switch+0x1f3/frame 0xd7622a98 sleepq_switch(d5942000,0,c16637ed,2b3,d593c680,...) at sleepq_switch+0x110/frame 0xd7622acc sleepq_wait(d593c680,0,c165ccb3,c1642b5b,0,...) at sleepq_wait+0x48/frame 0xd7622af0 msleep_spin_sbt(d593c680,d593c69c,c1642b5b,0,0,...) at msleep_spin_sbt+0x226/frame 0xd7622b34 gtaskqueue_thread_loop(d599c004,d7622ba8,c165294c,406,0,...) at gtaskqueue_thread_loop+0xf3/frame 0xd7622b6c fork_exit(c0cbe660,d599c004,d7622ba8) at fork_exit+0x7e/frame 0xd7622b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd7622b94 --- trap 0, eip = 0, esp = 0xd7622be0, ebp = 0 --- Tracing command kernel pid 0 tid 100014 td 0xd59a0380 sched_switch(d59a0380,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd7627a58 mi_switch(104,0,c16637ed,22b,d59a0380,...) at mi_switch+0x1f3/frame 0xd7627a98 sleepq_switch(d59a0380,0,c16637ed,2b3,d593c600,...) at sleepq_switch+0x110/frame 0xd7627acc sleepq_wait(d593c600,0,c165ccb3,c1642b5b,0,...) at sleepq_wait+0x48/frame 0xd7627af0 msleep_spin_sbt(d593c600,d593c61c,c1642b5b,0,0,...) at msleep_spin_sbt+0x226/frame 0xd7627b34 gtaskqueue_thread_loop(d599c014,d7627ba8,c165294c,406,0,...) at gtaskqueue_thread_loop+0xf3/frame 0xd7627b6c fork_exit(c0cbe660,d599c014,d7627ba8) at fork_exit+0x7e/frame 0xd7627b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd7627b94 --- trap 0, eip = 0, esp = 0xd7627be0, ebp = 0 --- Tracing command kernel pid 0 tid 100015 td 0xd59a0000 sched_switch(d59a0000,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd762ca58 mi_switch(104,0,c16637ed,22b,d59a0000,...) at mi_switch+0x1f3/frame 0xd762ca98 sleepq_switch(d59a0000,0,c16637ed,2b3,d593c580,...) at sleepq_switch+0x110/frame 0xd762cacc sleepq_wait(d593c580,0,c165ccb3,c1642b5b,0,...) at sleepq_wait+0x48/frame 0xd762caf0 msleep_spin_sbt(d593c580,d593c59c,c1642b5b,0,0,...) at msleep_spin_sbt+0x226/frame 0xd762cb34 gtaskqueue_thread_loop(d599c024,d762cba8,c165294c,406,0,...) at gtaskqueue_thread_loop+0xf3/frame 0xd762cb6c fork_exit(c0cbe660,d599c024,d762cba8) at fork_exit+0x7e/frame 0xd762cb94 fork_trampoline() at fork_trampoline+0x8/frame 0xd762cb94 --- trap 0, eip = 0, esp = 0xd762cbe0, ebp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xd599fa80 sched_switch(d599fa80,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd7631a58 mi_switch(104,0,c16637ed,22b,d599fa80,...) at mi_switch+0x1f3/frame 0xd7631a98 sleepq_switch(d599fa80,0,c16637ed,2b3,d593c500,...) at sleepq_switch+0x110/frame 0xd7631acc sleepq_wait(d593c500,0,c165ccb3,c1642b5b,0,...) at sleepq_wait+0x48/frame 0xd7631af0 msleep_spin_sbt(d593c500,d593c51c,c1642b5b,0,0,...) at msleep_spin_sbt+0x226/frame 0xd7631b34 gtaskqueue_thread_loop(d599c034,d7631ba8,c165294c,406,0,...) at gtaskqueue_thread_loop+0xf3/frame 0xd7631b6c fork_exit(c0cbe660,d599c034,d7631ba8) at fork_exit+0x7e/frame 0xd7631b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd7631b94 --- trap 0, eip = 0, esp = 0xd7631be0, ebp = 0 --- Tracing command kernel pid 0 tid 100017 td 0xd599f700 sched_switch(d599f700,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd7636a58 mi_switch(104,0,c16637ed,22b,d599f700,...) at mi_switch+0x1f3/frame 0xd7636a98 sleepq_switch(d599f700,0,c16637ed,2b3,d593c480,...) at sleepq_switch+0x110/frame 0xd7636acc sleepq_wait(d593c480,0,c165ccb3,c1642b5b,0,...) at sleepq_wait+0x48/frame 0xd7636af0 msleep_spin_sbt(d593c480,d593c49c,c1642b5b,0,0,...) at msleep_spin_sbt+0x226/frame 0xd7636b34 gtaskqueue_thread_loop(d599bc04,d7636ba8,c165294c,406,0,...) at gtaskqueue_thread_loop+0xf3/frame 0xd7636b6c fork_exit(c0cbe660,d599bc04,d7636ba8) at fork_exit+0x7e/frame 0xd7636b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd7636b94 --- trap 0, eip = 0, esp = 0xd7636be0, ebp = 0 --- Tracing command kernel pid 0 tid 100018 td 0xd599f380 sched_switch(d599f380,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd763ba58 mi_switch(104,0,c16637ed,22b,d599f380,...) at mi_switch+0x1f3/frame 0xd763ba98 sleepq_switch(d599f380,0,c16637ed,2b3,d593c400,...) at sleepq_switch+0x110/frame 0xd763bacc sleepq_wait(d593c400,0,c165ccb3,c1642b5b,0,...) at sleepq_wait+0x48/frame 0xd763baf0 msleep_spin_sbt(d593c400,d593c41c,c1642b5b,0,0,...) at msleep_spin_sbt+0x226/frame 0xd763bb34 gtaskqueue_thread_loop(d599b804,d763bba8,c165294c,406,0,...) at gtaskqueue_thread_loop+0xf3/frame 0xd763bb6c fork_exit(c0cbe660,d599b804,d763bba8) at fork_exit+0x7e/frame 0xd763bb94 fork_trampoline() at fork_trampoline+0x8/frame 0xd763bb94 --- trap 0, eip = 0, esp = 0xd763bbe0, ebp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xd599f000 sched_switch(d599f000,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd7640a58 mi_switch(104,0,c16637ed,22b,d599f000,...) at mi_switch+0x1f3/frame 0xd7640a98 sleepq_switch(d599f000,0,c16637ed,2b3,d593c380,...) at sleepq_switch+0x110/frame 0xd7640acc sleepq_wait(d593c380,0,c165ccb3,c1642b5b,0,...) at sleepq_wait+0x48/frame 0xd7640af0 msleep_spin_sbt(d593c380,d593c39c,c1642b5b,0,0,...) at msleep_spin_sbt+0x226/frame 0xd7640b34 gtaskqueue_thread_loop(d599b814,d7640ba8,c165294c,406,0,...) at gtaskqueue_thread_loop+0xf3/frame 0xd7640b6c fork_exit(c0cbe660,d599b814,d7640ba8) at fork_exit+0x7e/frame 0xd7640b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd7640b94 --- trap 0, eip = 0, esp = 0xd7640be0, ebp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xd599ea80 sched_switch(d599ea80,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd7645a58 mi_switch(104,0,c16637ed,22b,d599ea80,...) at mi_switch+0x1f3/frame 0xd7645a98 sleepq_switch(d599ea80,0,c16637ed,2b3,d593c300,...) at sleepq_switch+0x110/frame 0xd7645acc sleepq_wait(d593c300,0,c165ccb3,c1642b5b,0,...) at sleepq_wait+0x48/frame 0xd7645af0 msleep_spin_sbt(d593c300,d593c31c,c1642b5b,0,0,...) at msleep_spin_sbt+0x226/frame 0xd7645b34 gtaskqueue_thread_loop(d599b824,d7645ba8,c165294c,406,0,...) at gtaskqueue_thread_loop+0xf3/frame 0xd7645b6c fork_exit(c0cbe660,d599b824,d7645ba8) at fork_exit+0x7e/frame 0xd7645b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd7645b94 --- trap 0, eip = 0, esp = 0xd7645be0, ebp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xd599e700 sched_switch(d599e700,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd764aa58 mi_switch(104,0,c16637ed,22b,d599e700,...) at mi_switch+0x1f3/frame 0xd764aa98 sleepq_switch(d599e700,0,c16637ed,2b3,d593c280,...) at sleepq_switch+0x110/frame 0xd764aacc sleepq_wait(d593c280,0,c165ccb3,c1642b5b,0,...) at sleepq_wait+0x48/frame 0xd764aaf0 msleep_spin_sbt(d593c280,d593c29c,c1642b5b,0,0,...) at msleep_spin_sbt+0x226/frame 0xd764ab34 gtaskqueue_thread_loop(d599b834,d764aba8,c165294c,406,0,...) at gtaskqueue_thread_loop+0xf3/frame 0xd764ab6c fork_exit(c0cbe660,d599b834,d764aba8) at fork_exit+0x7e/frame 0xd764ab94 fork_trampoline() at fork_trampoline+0x8/frame 0xd764ab94 --- trap 0, eip = 0, esp = 0xd764abe0, ebp = 0 --- Tracing command kernel pid 0 tid 100031 td 0xd5a17700 sched_switch(d5a17700,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd767ca48 mi_switch(104,0,c16637ed,22b,d5a17700,...) at mi_switch+0x1f3/frame 0xd767ca88 sleepq_switch(d5a17700,0,c16637ed,2b3,d5a17700,...) at sleepq_switch+0x110/frame 0xd767cabc sleepq_wait(d593c100,0,c1642b5b,0,0,...) at sleepq_wait+0x48/frame 0xd767cae0 _sleep(d593c100,d593c11c,0,c1642b5b,0,...) at _sleep+0x396/frame 0xd767cb34 taskqueue_thread_loop(c1b6e624,d767cba8,c165294c,406,0,...) at taskqueue_thread_loop+0x144/frame 0xd767cb6c fork_exit(c0cd47f0,c1b6e624,d767cba8) at fork_exit+0x7e/frame 0xd767cb94 fork_trampoline() at fork_trampoline+0x8/frame 0xd767cb94 --- trap 0, eip = 0, esp = 0xd767cbe0, ebp = 0 --- Tracing command kernel pid 0 tid 100032 td 0xd5a17380 sched_switch(d5a17380,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd7681a48 mi_switch(104,0,c16637ed,22b,d5a17380,...) at mi_switch+0x1f3/frame 0xd7681a88 sleepq_switch(d5a17380,0,c16637ed,2b3,d5a17380,...) at sleepq_switch+0x110/frame 0xd7681abc sleepq_wait(d593bd80,0,c1642b5b,0,0,...) at sleepq_wait+0x48/frame 0xd7681ae0 _sleep(d593bd80,d593bd9c,0,c1642b5b,0,...) at _sleep+0x396/frame 0xd7681b34 taskqueue_thread_loop(c1bd5058,d7681ba8,c165294c,406,0,...) at taskqueue_thread_loop+0x144/frame 0xd7681b6c fork_exit(c0cd47f0,c1bd5058,d7681ba8) at fork_exit+0x7e/frame 0xd7681b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd7681b94 --- trap 0, eip = 0, esp = 0xd7681be0, ebp = 0 --- Tracing command kernel pid 0 tid 100033 td 0xd5a17000 sched_switch(d5a17000,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd7686a48 mi_switch(104,0,c16637ed,22b,d5a17000,...) at mi_switch+0x1f3/frame 0xd7686a88 sleepq_switch(d5a17000,0,c16637ed,2b3,d5a17000,...) at sleepq_switch+0x110/frame 0xd7686abc sleepq_wait(d593bd80,0,c1642b5b,0,0,...) at sleepq_wait+0x48/frame 0xd7686ae0 _sleep(d593bd80,d593bd9c,0,c1642b5b,0,...) at _sleep+0x396/frame 0xd7686b34 taskqueue_thread_loop(c1bd5058,d7686ba8,c165294c,406,0,...) at taskqueue_thread_loop+0x144/frame 0xd7686b6c fork_exit(c0cd47f0,c1bd5058,d7686ba8) at fork_exit+0x7e/frame 0xd7686b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd7686b94 --- trap 0, eip = 0, esp = 0xd7686be0, ebp = 0 --- Tracing command kernel pid 0 tid 100034 td 0xd5a16a80 sched_switch(d5a16a80,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd768ba48 mi_switch(104,0,c16637ed,22b,d5a16a80,...) at mi_switch+0x1f3/frame 0xd768ba88 sleepq_switch(d5a16a80,0,c16637ed,2b3,d5a16a80,...) at sleepq_switch+0x110/frame 0xd768babc sleepq_wait(d593bd80,0,c1642b5b,0,0,...) at sleepq_wait+0x48/frame 0xd768bae0 _sleep(d593bd80,d593bd9c,0,c1642b5b,0,...) at _sleep+0x396/frame 0xd768bb34 taskqueue_thread_loop(c1bd5058,d768bba8,c165294c,406,0,...) at taskqueue_thread_loop+0x144/frame 0xd768bb6c fork_exit(c0cd47f0,c1bd5058,d768bba8) at fork_exit+0x7e/frame 0xd768bb94 fork_trampoline() at fork_trampoline+0x8/frame 0xd768bb94 --- trap 0, eip = 0, esp = 0xd768bbe0, ebp = 0 --- Tracing command kernel pid 0 tid 100035 td 0xd5a16700 sched_switch(d5a16700,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd7690a48 mi_switch(104,0,c16637ed,22b,d5a16700,...) at mi_switch+0x1f3/frame 0xd7690a88 sleepq_switch(d5a16700,0,c16637ed,2b3,d5a16700,...) at sleepq_switch+0x110/frame 0xd7690abc sleepq_wait(d593bd80,0,c1642b5b,0,0,...) at sleepq_wait+0x48/frame 0xd7690ae0 _sleep(d593bd80,d593bd9c,0,c1642b5b,0,...) at _sleep+0x396/frame 0xd7690b34 taskqueue_thread_loop(c1bd5058,d7690ba8,c165294c,406,0,...) at taskqueue_thread_loop+0x144/frame 0xd7690b6c fork_exit(c0cd47f0,c1bd5058,d7690ba8) at fork_exit+0x7e/frame 0xd7690b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd7690b94 --- trap 0, eip = 0, esp = 0xd7690be0, ebp = 0 --- Tracing command kernel pid 0 tid 100068 td 0xd8c57700 sched_switch(d8c57700,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd3598a58 mi_switch(104,0,c16637ed,22b,d8c57700,...) at mi_switch+0x1f3/frame 0xd3598a98 sleepq_switch(d8c57700,0,c16637ed,2b3,d8c0e980,...) at sleepq_switch+0x110/frame 0xd3598acc sleepq_wait(d8c0e980,0,c165ccb3,c1642b5b,0,...) at sleepq_wait+0x48/frame 0xd3598af0 msleep_spin_sbt(d8c0e980,d8c0e99c,c1642b5b,0,0,...) at msleep_spin_sbt+0x226/frame 0xd3598b34 taskqueue_thread_loop(c1bf111c,d3598ba8,c165294c,406,0,...) at taskqueue_thread_loop+0xf3/frame 0xd3598b6c fork_exit(c0cd47f0,c1bf111c,d3598ba8) at fork_exit+0x7e/frame 0xd3598b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd3598b94 --- trap 0, eip = 0, esp = 0xd3598be0, ebp = 0 --- Tracing command kernel pid 0 tid 100074 td 0xd8e93a80 sched_switch(d8e93a80,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd35b7a58 mi_switch(104,0,c16637ed,22b,d8e93a80,...) at mi_switch+0x1f3/frame 0xd35b7a98 sleepq_switch(d8e93a80,0,c16637ed,2b3,d8dedd80,...) at sleepq_switch+0x110/frame 0xd35b7acc sleepq_wait(d8dedd80,0,c165ccb3,c1642b5b,0,...) at sleepq_wait+0x48/frame 0xd35b7af0 msleep_spin_sbt(d8dedd80,d8dedd9c,c1642b5b,0,0,...) at msleep_spin_sbt+0x226/frame 0xd35b7b34 taskqueue_thread_loop(c1bf2d0c,d35b7ba8,c165294c,406,0,...) at taskqueue_thread_loop+0xf3/frame 0xd35b7b6c fork_exit(c0cd47f0,c1bf2d0c,d35b7ba8) at fork_exit+0x7e/frame 0xd35b7b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd35b7b94 --- trap 0, eip = 0, esp = 0xd35b7be0, ebp = 0 --- Tracing command kernel pid 0 tid 100075 td 0xd8e93700 sched_switch(d8e93700,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd35bca58 mi_switch(104,0,c16637ed,22b,d8e93700,...) at mi_switch+0x1f3/frame 0xd35bca98 sleepq_switch(d8e93700,0,c16637ed,2b3,d8dedd80,...) at sleepq_switch+0x110/frame 0xd35bcacc sleepq_wait(d8dedd80,0,c165ccb3,c1642b5b,0,...) at sleepq_wait+0x48/frame 0xd35bcaf0 msleep_spin_sbt(d8dedd80,d8dedd9c,c1642b5b,0,0,...) at msleep_spin_sbt+0x226/frame 0xd35bcb34 taskqueue_thread_loop(c1bf2d0c,d35bcba8,c165294c,406,0,...) at taskqueue_thread_loop+0xf3/frame 0xd35bcb6c fork_exit(c0cd47f0,c1bf2d0c,d35bcba8) at fork_exit+0x7e/frame 0xd35bcb94 fork_trampoline() at fork_trampoline+0x8/frame 0xd35bcb94 --- trap 0, eip = 0, esp = 0xd35bcbe0, ebp = 0 --- Tracing command kernel pid 0 tid 100076 td 0xd8e93380 sched_switch(d8e93380,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd35c1a58 mi_switch(104,0,c16637ed,22b,d8e93380,...) at mi_switch+0x1f3/frame 0xd35c1a98 sleepq_switch(d8e93380,0,c16637ed,2b3,d8dedd80,...) at sleepq_switch+0x110/frame 0xd35c1acc sleepq_wait(d8dedd80,0,c165ccb3,c1642b5b,0,...) at sleepq_wait+0x48/frame 0xd35c1af0 msleep_spin_sbt(d8dedd80,d8dedd9c,c1642b5b,0,0,...) at msleep_spin_sbt+0x226/frame 0xd35c1b34 taskqueue_thread_loop(c1bf2d0c,d35c1ba8,c165294c,406,0,...) at taskqueue_thread_loop+0xf3/frame 0xd35c1b6c fork_exit(c0cd47f0,c1bf2d0c,d35c1ba8) at fork_exit+0x7e/frame 0xd35c1b94 fork_trampoline() at fork_trampoline+0x8/frame 0xd35c1b94 --- trap 0, eip = 0, esp = 0xd35c1be0, ebp = 0 --- Tracing command kernel pid 0 tid 100078 td 0xd8e92a80 sched_switch(d8e92a80,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xd35cba48 mi_switch(104,0,c16637ed,22b,d8e92a80,...) at mi_switch+0x1f3/frame 0xd35cba88 sleepq_switch(d8e92a80,0,c16637ed,2b3,d8e92a80,...) at sleepq_switch+0x110/frame 0xd35cbabc sleepq_wait(d593bb80,0,c1642b5b,0,0,...) at sleepq_wait+0x48/frame 0xd35cbae0 _sleep(d593bb80,d593bb9c,0,c1642b5b,0,...) at _sleep+0x396/frame 0xd35cbb34 taskqueue_thread_loop(c1979068,d35cbba8,c165294c,406,0,...) at taskqueue_thread_loop+0x144/frame 0xd35cbb6c fork_exit(c0cd47f0,c1979068,d35cbba8) at fork_exit+0x7e/frame 0xd35cbb94 fork_trampoline() at fork_trampoline+0x8/frame 0xd35cbb94 --- trap 0, eip = 0, esp = 0xd35cbbe0, ebp = 0 --- Tracing command kernel pid 0 tid 100089 td 0xd8c57000 sched_switch(d8c57000,0,104,188,c124bb30,...) at sched_switch+0x503/frame 0xdb297a48 mi_switch(104,0,c16637ed,22b,d8c57000,...) at mi_switch+0x1f3/frame 0xdb297a88 sleepq_switch(d8c57000,0,c16637ed,2b3,d8c57000,...) at sleepq_switch+0x110/frame 0xdb297abc sleepq_wait(d8c0d300,0,c1642b5b,0,0,...) at sleepq_wait+0x48/frame 0xdb297ae0 _sleep(d8c0d300,d8c0d31c,0,c1642b5b,0,...) at _sleep+0x396/frame 0xdb297b34 taskqueue_thread_loop(d946429c,db297ba8,c165294c,406,0,...) at taskqueue_thread_loop+0x144/frame 0xdb297b6c fork_exit(c0cd47f0,d946429c,db297ba8) at fork_exit+0x7e/frame 0xdb297b94 fork_trampoline() at fork_trampoline+0x8/frame 0xdb297b94 --- trap 0, eip = 0, esp = 0xdb297be0, ebp = 0 --- db:0:allt> dump Physical memory: 2967 MB Dumping 129 MB: 114 98 82 66 50 34 18 2 Dump complete db:0:dump> reset cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 2 (kgdb) bt #0 doadump (textdump=) at pcpu.h:216 #1 0xc05ad2ee in db_dump (dummy=0xc0cbfc2a, dummy2=0x0, dummy3=0xffffffff, dummy4=0xdb2c9068 "") at ../../../ddb/db_command.c:574 #2 0xc05ad0c7 in db_command (cmd_table=) at ../../../ddb/db_command.c:481 #3 0xc05b1c08 in db_script_exec (scriptname=0xc1bf2a74 "dump", warnifnotfound=) at ../../../ddb/db_script.c:304 #4 0xc05ad0c7 in db_command (cmd_table=) at ../../../ddb/db_command.c:481 #5 0xc05ace20 in db_command_loop () at ../../../ddb/db_command.c:534 #6 0xc05b005d in db_trap (type=0x3, code=0xffff0ff0) at ../../../ddb/db_main.c:250 #7 0xc0cc03de in kdb_trap (type=0x3, code=0xffff0ff0, tf=0xdb2c93b4) at ../../../kern/subr_kdb.c:697 #8 0xc125cf67 in trap (frame=0xdb2c93b4) at ../../../i386/i386/trap.c:677 #9 0xc1247337 in calltrap () at ../../../i386/i386/exception.s:172 #10 0x00000000 in ?? () Current language: auto; currently minimal (kgdb) $ svnlite diff -x -p /var/tmp/numa Index: /var/tmp/numa/sys/kern/vfs_bio.c =================================================================== --- /var/tmp/numa/sys/kern/vfs_bio.c (revision 329862) +++ /var/tmp/numa/sys/kern/vfs_bio.c (working copy) @@ -1717,7 +1717,7 @@ bd_init(struct bufdomain *bd) domain = bd - bdclean; bd->bd_cleanq = &bd->bd_subq[mp_ncpus]; - bq_init(bd->bd_cleanq, QUEUE_CLEAN, -1, "bufq clean lock"); + bq_init(bd->bd_cleanq, QUEUE_CLEAN, mp_ncpus, "bufq clean lock"); for (i = 0; i <= mp_maxid; i++) bq_init(&bd->bd_subq[i], QUEUE_CLEAN, i, "bufq clean subqueue lock"); Index: /var/tmp/numa/sys/vm/vm_phys.h =================================================================== --- /var/tmp/numa/sys/vm/vm_phys.h (revision 329862) +++ /var/tmp/numa/sys/vm/vm_phys.h (working copy) @@ -51,6 +51,7 @@ struct mem_affinity { #ifdef NUMA extern struct mem_affinity *mem_affinity; extern int *mem_locality; +extern int vm_ndomains; /* pho XXX */ #endif struct vm_freelist { $