HEAD r224866 Watchdogd fired after one hour. $ console leopard3 [`leopard3' -- console is connecting] [Enter `^Ec?' for help] GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2011 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-BETA1 #0: Sun Aug 14 18:16:12 CEST 2011 test@leopard3.netperf.freebsd.org:/mnt/pho/obj/mnt/pho/src/sys/PHO i386 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. CPU: Intel(R) Xeon(R) CPU 3050 @ 2.13GHz (2128.04-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x6f2 Family = 6 Model = f Stepping = 2 Features=0xbfebfbff Features2=0xe3bd AMD Features=0x20100000 AMD Features2=0x1 TSC: P-state invariant, performance statistics real memory = 2147483648 (2048 MB) avail memory = 2080460800 (1984 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs FreeBSD/SMP: 1 package(s) x 2 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard ioapic2 irqs 48-71 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 cpu0: on acpi0 cpu1: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: irq 16 at device 1.0 on pci0 pci1: on pcib1 pcib2: at device 0.0 on pci1 pci2: on pcib2 pcib3: at device 0.2 on pci1 pci3: on pcib3 pcib4: irq 16 at device 3.0 on pci0 pci10: on pcib4 pci10: at device 0.0 (no driver attached) pcib5: irq 17 at device 28.0 on pci0 pcib5: failed to allocate initial I/O port window: 0-0xfff pcib5: failed to allocate initial memory window: 0-0xfffff pcib5: failed to allocate initial prefetch window: 0-0xfffff pci14: on pcib5 pcib6: irq 17 at device 28.4 on pci0 pci18: on pcib6 em0: port 0x5000-0x501f mem 0xe0300000-0xe031ffff irq 16 at device 0.0 on pci18 em0: Using an MSI interrupt em0: Ethernet address: 00:30:48:8e:d0:68 pcib7: irq 16 at device 28.5 on pci0 pci19: on pcib7 em1: port 0x6000-0x601f mem 0xe0400000-0xe041ffff irq 17 at device 0.0 on pci19 em1: Using an MSI interrupt em1: Ethernet address: 00:30:48:8e:d0:69 uhci0: port 0x3000-0x301f irq 23 at device 29.0 on pci0 usbus0: on uhci0 uhci1: port 0x3020-0x303f irq 19 at device 29.1 on pci0 usbus1: on uhci1 uhci2: port 0x3040-0x305f irq 18 at device 29.2 on pci0 usbus2: on uhci2 uhci3: port 0x3060-0x307f irq 16 at device 29.3 on pci0 usbus3: on uhci3 ehci0: mem 0xe0000000-0xe00003ff irq 23 at device 29.7 on pci0 usbus4: EHCI version 1.0 usbus4: on ehci0 pcib8: at device 30.0 on pci0 pci20: on pcib8 vgapci0: port 0x7000-0x70ff mem 0xe8000000-0xefffffff,0xe0500000-0xe050ffff irq 16 at device 4.0 on pci20 isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x30a0-0x30af at device 31.1 on pci0 ata0: on atapci0 ahci0: port 0x30e8-0x30ef,0x30dc-0x30df,0x30e0-0x30e7,0x30d8-0x30db,0x30b0-0x30bf mem 0xe0000400-0xe00007ff irq 19 at device 31.2 on pci0 ahci0: AHCI v1.10 with 4 3Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 pci0: at device 31.3 (no driver attached) acpi_button0: on acpi0 atrtc0: port 0x70-0x71 irq 8 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: console (115200,n,8,1) uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fd0: <1440-KB 3.5" drive> on fdc0 drive 0 pmtimer0 on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x100> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] ppc0: parallel port not found. est0: on cpu0 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 828082806000828 device_attach: est0 attach returned 6 p4tcc0: on cpu0 est1: on cpu1 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 828082806000828 device_attach: est1 attach returned 6 p4tcc1: on cpu1 Timecounters tick every 1.000 msec usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 12Mbps Full Speed USB v1.0 usbus3: 12Mbps Full Speed USB v1.0 ugen0.1: at usbus0 uhub0: on usbus0 Expensive timeout(9) function: 0xc0a472d0(0xc5b1d5c0) 0.011031848 s ugen1.1: at usbus1 uhub1: on usbus1 ugen2.1: at usbus2 uhub2: on usbus2 ugen3.1: at usbus3 uhub3: on usbus3 usbus4: 480Mbps High Speed USB v2.0 ugen4.1: at usbus4 uhub4: on usbus4 Expensive timeout(9) function: 0xc0527ed0(0xc5b38500) 0.020067634 s ada0 at ahcich0 bus 0 scbus1 target 0 lun 0 ada0: ATA-7cd0 at ata0 bus 0 scbus0 target 0 lun 0 cd0: Removable CD-ROM SCSI-0 device cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present SATA 2.x device ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 238475MB (488397168 512 byte sectors: 16H 63S/T 16383C) ada0: Previously was known as ad4 SMP: AP CPU #1 Launched! Timecounter "TSC-low" frequency 16625343 Hz quality 1000 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. uhub1: 2 ports with 2 removable, self powered uhub0: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 uhub4: 8 ports with 8 removable, self powered Trying to mount root from nfs: []... NFS ROOT: 192.168.5.1:/zoo/leopard3 em0: link state changed to UP Interface em0 IP-Address 192.168.5.109 Broadcast 192.168.5.255 Setting hostuuid: 64a93476-27b1-3004-c299-0030488ed164. Setting hostid: 0x7acc54f1. Starting ddb. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: /dev/ada0d: 62426 files, 4319462 used, 111905664 free (1496 frags, 13988021 blocks, 0.0% fragmentation) mount_nfs: can't update /var/db/mounttab for 192.168.5.1:/zoo/leopard3 Mounting local file systems:. Setting hostname: leopard3.netperf.freebsd.org. Starting Network: lo0 em0 em1. lo0: flags=8049 metric 0 mtu 16384 options=3 inet 127.0.0.1 netmask 0xff000000 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x8 nd6 options=21 em0: flags=8843 metric 0 mtu 1500 options=219b ether 00:30:48:8e:d0:68 inet 192.168.5.109 netmask 0xffffff00 broadcast 192.168.5.255 inet6 fe80::230:48ff:fe8e:d068%em0 prefixlen 64 tentative scopeid 0x1 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active em1: flags=8802 metric 0 mtu 1500 options=219b ether 00:30:48:8e:d0:69 media: Ethernet autoselect status: no carrier Starting devd. Starting Network: em1. em1: flags=8802 metric 0 mtu 1500 options=219b ether 00:30:48:8e:d0:69 media: Ethernet autoselect status: no carrier Starting Network: usbus0. Starting Network: usbus1. Starting Network: usbus2. Starting Network: usbus3. Starting Network: usbus4. add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 Creating and/or trimming log files. Starting syslogd. savecore: reboot Aug 15 16:51:35 leopard3 savecore: reboot savecore: writing core to vmcore.28 Writing crash summary to /work/crash/core.txt.28. ELF ldconfig path: /lib /usr/lib /usr/lib/compat a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout Setting date via ntp. 15 Aug 16:56:29 ntpdate[1444]: step time server 192.168.5.1 offset -0.424919 sec Starting rpcbind. NFS access cache time=60 Clearing /tmp (X related). Starting statd. Updating motd:. Configuring syscons: blanktime. Starting sshd. Starting cron. Starting inetd. Mon Aug 15 16:56:31 UTC 2011 FreeBSD/i386 (leopard3.netperf.freebsd.org) (ttyu0) login: lock order reversal: 1st 0xc656d940 so_snd_sx (so_snd_sx) @ /mnt/pho/src/sys/kern/uipc_sockbuf.c:148 2nd 0xc642832c filedesc structure (filedesc structure) @ /mnt/pho/src/sys/kern/uipc_usrreq.c:1864 KDB: stack backtrace: db_trace_self_wrapper(c0f00d1e,20296572,6d2f2040,702f746e,732f6f68,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a40afb,c0f0475f,c594b060,c5947780,edd71a5c,...) at kdb_backtrace+0x2a _witness_debugger(c0f0475f,c642832c,c0ef685f,c5947780,c0f093a3,...) at _witness_debugger+0x25 witness_checkorder(c642832c,1,c0f093a3,748,0,...) at witness_checkorder+0x839 _sx_slock(c642832c,0,c0f093a3,748,0,...) at _sx_slock+0x85 uipc_send(c656d820,0,c6162e00,0,0,...) at uipc_send+0x1fa sosend_generic(c656d820,0,edd71ba4,0,c5f6ac00,...) at sosend_generic+0x50d sosend(c656d820,0,edd71ba4,0,c5f6ac00,...) at sosend+0x3f kern_sendit(c6489000,5,edd71c24,0,c5f6ac00,...) at kern_sendit+0x1d4 sendit(0,1,edd71c40,28,0,...) at sendit+0xb1 sendmsg(c6489000,edd71cec,edd71d28,c0f02f8a,0,...) at sendmsg+0x71 syscallenter(c6489000,edd71ce4,edd71ce4,0,0,...) at syscallenter+0x263 syscall(edd71d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (28, FreeBSD ELF32, sendmsg), eip = 0x2849626b, esp = 0xbfbfdc8c, ebp = 0xbfbfdd08 --- Aug 15 18:59:45 leopard3 su: test to root on /dev/pts/0 lock order reversal: 1st 0xc65809dc ufs (ufs) @ /mnt/pho/src/sys/ufs/ffs/ffs_snapshot.c:425 2nd 0xdf3bc8c0 bufwait (bufwait) @ /mnt/pho/src/sys/kern/vfs_bio.c:2658 3rd 0xc6581058 ufs (ufs) @ /mnt/pho/src/sys/ufs/ffs/ffs_snapshot.c:546 KDB: stack backtrace: db_trace_self_wrapper(c0f00d1e,7366662f,616e735f,6f687370,3a632e74,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a40afb,c0f04778,c5947370,c594b5a8,edda1408,...) at kdb_backtrace+0x2a _witness_debugger(c0f04778,c6581058,c0ef378c,c594b5a8,c0f2ffde,...) at _witness_debugger+0x25 witness_checkorder(c6581058,9,c0f2ffde,222,0,...) at witness_checkorder+0x839 __lockmgr_args(c6581058,80100,c65810c4,0,0,...) at __lockmgr_args+0x834 ffs_lock(edda152c,c11e18b8,c65aa390,80100,c6581000,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c104be20,edda152c,edda154c,c105c620,c6581000,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c6581000,80100,c0f2ffde,222,c5986000,...) at _vn_lock+0x78 ffs_snapshot(c5eb1b50,c5eb8c40,c0f33490,1a2,0,...) at ffs_snapshot+0x14f7 ffs_mount(c5eb1b50,c5e3d500,ff,394,c5944450,...) at ffs_mount+0x1c13 vfs_donmount(c65aa2e0,11000,c5e4c200,c5e4c200,c6597b00,...) at vfs_donmount+0x1254 nmount(c65aa2e0,edda1cec,edda1d28,c0f02f8a,0,...) at nmount+0x84 syscallenter(c65aa2e0,edda1ce4,edda1ce4,0,0,...) at syscallenter+0x263 syscall(edda1d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (378, FreeBSD ELF32, nmount), eip = 0x280da65b, esp = 0xbfbfe25c, ebp = 0xbfbfe9d8 --- lock order reversal: 1st 0xdf3bc8c0 bufwait (bufwait) @ /mnt/pho/src/sys/kern/vfs_bio.c:2658 2nd 0xc5e3d41c snaplk (snaplk) @ /mnt/pho/src/sys/ufs/ffs/ffs_snapshot.c:818 KDB: stack backtrace: db_trace_self_wrapper(c0f00d1e,662f7366,662f7366,735f7366,7370616e,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a40afb,c0f0475f,c5947370,c594b7b0,edda1408,...) at kdb_backtrace+0x2a _witness_debugger(c0f0475f,c5e3d41c,c0f30065,c594b7b0,c0f2ffde,...) at _witness_debugger+0x25 witness_checkorder(c5e3d41c,9,c0f2ffde,332,c6580a48,...) at witness_checkorder+0x839 __lockmgr_args(c5e3d41c,80400,c6580a48,0,0,...) at __lockmgr_args+0x834 ffs_lock(edda152c,df3af8c0,100000,80400,c6580984,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c104be20,edda152c,df3af968,c105c620,c6580984,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c6580984,80400,c0f2ffde,332,0,...) at _vn_lock+0x78 ffs_snapshot(c5eb1b50,c5eb8c40,c0f33490,1a2,0,...) at ffs_snapshot+0x29b1 ffs_mount(c5eb1b50,c5e3d500,ff,394,c5944450,...) at ffs_mount+0x1c13 lock order reversal: vfs_donmount(c65aa2e0,11000,c5e4c200,c5e4c200,c6597b00,...) at 1st 0xdf3afa60 bufwait (bufwait) @ /mnt/pho/src/sys/kern/vfs_bio.c:2658 vfs_donmount+0x1254 2nd 0xc65c0400 dirhash (dirhash) @ /mnt/pho/src/sys/ufs/ufs/ufs_dirhash.c:284 nmount(c65aa2e0,edda1cec,edda1d28,c0f02f8a,0,...) at KDB: stack backtrace: nmount+0x84 db_tracsyscallenter(c65aa2e0,edda1ce4,edda1ce4,0,0,...) at e_selsyscallenter+0x263 f_wrapper(syscall(edda1d28) at c0f00syscall+0x4f d1e,7Xint0x80_syscall() at 36675Xint0x80_syscall+0x21 --- syscall (3782f,7, FreeBSD ELF32, nmount), eip = 0x280da65b, esp = 0xbfbfe25c, ebp = 0xbfbfe9d8 --- 366752f,7366752f,7269645f,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a40afb,c0f0475f,c5947370,c594b748,df78d85c,...) at kdb_backtrace+0x2a _witness_debugger(c0f0475f,c65c0400,c0f3443e,c594b748,c0f340bf,...) at _witness_debugger+0x25 witness_checkorder(c65c0400,9,c0f340bf,11c,0,...) at witness_checkorder+0x839 _sx_xlock(c65c0400,0,c0f340bf,11c,c6391910,...) at _sx_xlock+0x85 ufsdirhash_acquire(df3afa00,c6391910,df78d9d8,e19ac028,df78d92c,...) at ufsdirhash_acquire+0x48 ufsdirhash_add(c6391910,df78d9d8,28,df78d918,df78d91c,...) at ufsdirhash_add+0x13 ufs_direnter(c6581000,c6580570,df78d9d8,df78db68,df3bec20,...) at ufs_direnter+0x789 ufs_mkdir(df78dbf8,c0f526d6,0,0,df78db90,...) at ufs_mkdir+0xa36 VOP_MKDIR_APV(c104be20,df78dbf8,df78db68,df78db90,0,...) at VOP_MKDIR_APV+0xc5 kern_mkdirat(c5e6b5c0,ffffff9c,bfbfedcf,0,1f8,...) at kern_mkdirat+0x291 kern_mkdir(c5e6b5c0,bfbfedcf,0,1f8,df78dc7c,...) at kern_mkdir+0x2e mkdir(c5e6b5c0,df78dcec,df78dd28,c0f02f8a,0,...) at mkdir+0x29 syscallenter(c5e6b5c0,df78dce4,df78dce4,2,0,...) at syscallenter+0x263 syscall(df78dd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, mkdir), eip = 0x28185b33, esp = 0xbfbfe93c, ebp = 0xbfbfe9c8 --- interrupt total irq4: uart0 4055 irq6: fdc0 7 irq14: ata0 19 cpu0:timer 2731115 irq256: em0 27568 irq258: ahci0 151904 cpu1:timer 4801084 541015871 538976288 538976288 538976288 2105376 829518441 1952522298 811885163 538976288 2105376 1634890867 1919492217 538980721 538976288 2105376 812741225 1952522298 1701669236 538980466 2105376 1634890867 1919492217 538980465 538976288 2105376 863072873 1635065914 540111986 538976288 2105376 1634890867 1919492217 538981233 538976288 2105376 879850089 1635065914 540046450 538976288 2105376 1634890867 1919492217 538981489 538976288 2105376 896627305 538976314 538976288 538976288 2105376 1634890867 1919492217 538981745 538976288 2105376 913404521 1684414522 538980451 538976288 2105376 1634890867 1919492217 538982001 538976288 2105376 930181737 538976314 538976288 538976288 2105376 1634890867 1919492217 538982257 538976288 2105376 946958953 1952522298 811824242 538976288 2105376 1634890867 1919492217 538982513 538976288 2105376 963736169 1667309626 540043632 538976288 2105376 1634890867 1919492217 538982769 538976288 2105376 829518441 538982960 538976288 538976288 2105376 1634890867 1919492217 540029297 538976288 2105376 829518441 538982961 538976288 538976288 2105376 1634890867 1919492217 540094833 538976288 2105376 829518441 538982962 538976288 538976288 2105376 1634890867 1919492217 540160369 538976288 2105376 829518441 538982963 538976288 538976288 2105376 1634890867 1919492217 540225905 538976288 2105376 829518441 1629502004 540041588 538976288 2105376 1634890867 1919492217 540291441 538976288 2105376 829518441 538982965 538976288 538976288 2105376 1634890867 1919492217 540356977 538976288 2105376 829518441 1965046326 862544744 538976288 2105376 1634890867 1919492217 540422513 538976288 2105376 829518441 538982967 538976288 538976288 2105376 1634890867 1919492217 540488049 538976288 2105376 829518441 1965046328 845767528 538976288 2105376 1634890867 1919492217 540553585 538976288 2105376 829518441 1965046329 828990312 538976288 2105376 1634890867 1919492217 540619121 538976288 2105376 846295657 538982960 538976288 538976288 2105376 1634890867 1919492217 540029553 538976288 2105376 846295657 538982961 538976288 538976288 2105376 1634890867 1919492217 540095089 538976288 2105376 846295657 538982962 538976288 538976288 2105376 1634890867 1919492217 540160625 538976288 2105376 846295657 1965046323 812213096 1667786016 2109545 1634890867 1919492217 540226161 538976288 2105376 846295657 538982964 538976288 538976288 2105376 1634890867 1919492217 540291697 538976288 2105376 846295657 538982965 538976288 538976288 2105376 1634890867 1919492217 540357233 538976288 2105376 846295657 538982966 538976288 538976288 2105376 1634890867 1919492217 540422769 538976288 2105376 846295657 538982967 538976288 538976288 2105376 1634890867 1919492217 540488305 538976288 2105376 846295657 538982968 538976288 538976288 2105376 1634890867 1919492217 540553841 538976288 2105376 846295657 538982969 538976288 538976288 2105376 1634890867 1919492217 540619377 538976288 2105376 863072873 538982960 538976288 538976288 2105376 1634890867 1919492217 540029809 538976288 2105376 863072873 538982961 538976288 538976288 2105376 1634890867 1919492217 540095345 538976288 2105376 863072873 538982962 538976288 538976288 2105376 1634890867 1919492217 540160881 538976288 2105376 863072873 538982963 538976288 538976288 2105376 1634890867 1919492217 540226417 538976288 2105376 863072873 538982964 538976288 538976288 2105376 1634890867 1919492217 540291953 538976288 2105376 863072873 538982965 538976288 538976288 2105376 1634890867 1919492217 540357489 538976288 2105376 863072873 538982966 538976288 538976288 2105376 1634890867 1919492217 540423025 538976288 2105376 863072873 538982967 538976288 538976288 2105376 1634890867 1919492217 540488561 538976288 2105376 863072873 538982968 538976288 538976288 2105376 1634890867 1919492217 540554097 538976288 2105376 863072873 538982969 538976288 538976288 2105376 1634890867 1919492217 540619633 538976288 2105376 879850089 538982960 538976288 538976288 2105376 1634890867 1919492217 540030065 538976288 2105376 879850089 538982961 538976288 538976288 2105376 1634890867 1919492217 540095601 538976288 2105376 879850089 538982962 538976288 538976288 2105376 1634890867 1919492217 540161137 538976288 2105376 879850089 538982963 538976288 538976288 2105376 1634890867 1919492217 540226673 538976288 2105376 879850089 538982964 538976288 538976288 2105376 1634890867 1919492217 540292209 538976288 2105376 879850089 538982965 538976288 538976288 2105376 1634890867 1919492217 540357745 538976288 2105376 879850089 538982966 538976288 538976288 2105376 1634890867 1919492217 540423281 538976288 2105376 879850089 538982967 538976288 538976288 2105376 1634890867 1919492217 540488817 538976288 2105376 879850089 538982968 538976288 538976288 2105376 1634890867 1919492217 540554353 538976288 2105376 879850089 538982969 538976288 538976288 2105376 1634890867 1919492217 540619889 538976288 2105376 896627305 538982960 538976288 538976288 2105376 1634890867 1919492217 540030321 538976288 2105376 896627305 538982961 538976288 538976288 2105376 1634890867 1919492217 540095857 538976288 2105376 896627305 538982962 538976288 538976288 2105376 1634890867 1919492217 540161393 538976288 2105376 896627305 538982963 538976288 538976288 2105376 1634890867 1919492217 540226929 538976288 2105376 896627305 538982964 538976288 538976288 2105376 1634890867 1919492217 540292465 538976288 2105376 896627305 538982965 538976288 538976288 2105376 1634890867 1919492217 540358001 538976288 2105376 896627305 538982966 538976288 538976288 2105376 1634890867 1919492217 540423537 538976288 2105376 896627305 538982967 538976288 538976288 2105376 1634890867 1919492217 540489073 538976288 2105376 896627305 538982968 538976288 538976288 2105376 1634890867 1919492217 540554609 538976288 2105376 896627305 538982969 538976288 538976288 2105376 1634890867 1919492217 540620145 538976288 2105376 913404521 538982960 538976288 538976288 2105376 1634890867 1919492217 540030577 538976288 2105376 913404521 538982961 538976288 538976288 2105376 1634890867 1919492217 540096113 538976288 2105376 913404521 538982962 538976288 538976288 2105376 1634890867 1919492217 540161649 538976288 2105376 913404521 538982963 538976288 538976288 2105376 1634890867 1919492217 540227185 538976288 2105376 913404521 538982964 538976288 538976288 2105376 1634890867 1919492217 540292721 538976288 2105376 913404521 538982965 538976288 538976288 2105376 1634890867 1919492217 540358257 538976288 2105376 913404521 538982966 538976288 538976288 2105376 1634890867 1919492217 540423793 538976288 2105376 913404521 538982967 538976288 538976288 2105376 1634890867 1919492217 540489329 538976288 2105376 913404521 538982968 538976288 538976288 2105376 1634890867 1919492217 540554865 538976288 2105376 913404521 538982969 538976288 538976288 2105376 1634890867 1919492217 540620401 538976288 2105376 930181737 538982960 538976288 538976288 2105376 1634890867 1919492217 540030833 538976288 2105376 930181737 538982961 538976288 538976288 2105376 1634890867 1919492217 540096369 538976288 2105376 813002851 1835627578 538997349 538976288 2105376 846295657 540685877 540044645 538976288 2105376 1634890867 1919492217 909455985 538976288 2105376 846295657 540686133 540110181 538976288 2105376 1634890867 1919492217 926233201 538976288 2105376 846295657 540686389 1768122465 538976304 2105376 1634890867 1919492217 943010417 538976288 2105376 829780067 1835627578 538997349 538976288 2105376 Total 559140544982 KDB: stack backtrace: db_trace_self_wrapper(c0f00d1e,4176ea,df73dad8,c0d4726e,0,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0ef5252,2f5fa1d6,82,2f5fa1d6,82,...) at kdb_backtrace+0x2a watchdog_fire(1,0,c0ef5117,216,c5c1e5c0,...) at watchdog_fire+0xa0 hardclock_anycpu(1,0,df73dba8,c0a13ffd,0,...) at hardclock_anycpu+0x26f handleevents(df73dbc8,4,0,c5b48200,c5b36080,100,12a7,e808f957,1d0eff8f,7c8280) at handleevents+0x108 timercb(c1277080,0,0,c1097470,0,...) at timercb+0x2ad lapic_handle_timer(df73dc0c) at lapic_handle_timer+0x92 Xtimerint() at Xtimerint+0x20 --- interrupt, eip = 0xc09f8077, esp = 0xdf73dc4c, ebp = 0xdf73dc68 --- _mtx_lock_sleep(c1097470,c5c1e5c0,0,c0ef78eb,4e8,...) at _mtx_lock_sleep+0x97 _mtx_lock_flags(c1097470,0,c0ef78eb,4e8,c59ede38,...) at _mtx_lock_flags+0xf7 intr_event_execute_handlers(c59a5580,c59ede00,c0ef78eb,55e,c5c1e5c0,...) at intr_event_execute_handlers+0x11a ithread_loop(c59ee710,df73dd28,c0ef75b0,3a5,c59a5580,...) at ithread_loop+0xa9 fork_exit(c09de980,c59ee710,df73dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf73dd60, ebp = 0 --- KDB: enter: watchdog timeout [ thread pid 12 tid 100052 ] Stopped at kdb_enter+0x3a: movl $0,kdb_why db> run pho db:0:pho> bt Tracing pid 12 tid 100052 td 0xc5c1e5c0 kdb_enter(c0d964dd,c0e891f5,82,2f5fa1d6,82) at kdb_enter+0x3a watchdog_fire(1,0,c0ef5117,216,c5c1e5c0,...) at watchdog_fire+0xb4 hardclock_anycpu(1,0,df73dba8,c0a13ffd,0,...) at hardclock_anycpu+0x26f handleevents(df73dbc8,4,0,c5b48200,c5b36080,100,12a7,e808f957,1d0eff8f,7c8280) at handleevents+0x108 timercb(c1277080,0,0,c1097470,0,...) at timercb+0x2ad lapic_handle_timer(df73dc0c) at lapic_handle_timer+0x92 Xtimerint() at Xtimerint+0x20 --- interrupt, eip = 0xc09f8077, esp = 0xdf73dc4c, ebp = 0xdf73dc68 --- _mtx_lock_sleep(c1097470,c5c1e5c0,0,c0ef78eb,4e8,...) at _mtx_lock_sleep+0x97 _mtx_lock_flags(c1097470,0,c0ef78eb,4e8,c59ede38,...) at _mtx_lock_flags+0xf7 intr_event_execute_handlers(c59a5580,c59ede00,c0ef78eb,55e,c5c1e5c0,...) at intr_event_execute_handlers+0x11a ithread_loop(c59ee710,df73dd28,c0ef75b0,3a5,c59a5580,...) at ithread_loop+0xa9 fork_exit(c09de980,c59ee710,df73dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf73dd60, ebp = 0 --- db:0:bt> show allpcpu Current CPU: 0 cpuid = 0 dynamic pcpu = 0x7c8280 curthread = 0xc5c1e5c0: pid 12 "swi0: uart uart" curpcb = 0xdf73dd80 fpcurthread = none idlethread = 0xc59a78a0: tid 100003 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: cpuid = 1 dynamic pcpu = 0x45aa280 curthread = 0xc65aa2e0: pid 1877 "mksnap_ffs" curpcb = 0xedda1d80 fpcurthread = none idlethread = 0xc59a75c0: tid 100004 "idle: cpu1" APIC ID = 1 currentldt = 0x50 spin locks held: db:0:allpcpu> show alllocks Process 1878 (run) thread 0xc5e6b5c0 (100073) exclusive lockmgr bufwait (bufwait) r = 0 (0xdf3afa60) locked @ /mnt/pho/src/sys/kern/vfs_bio.c:2658 exclusive lockmgr bufwait (bufwait) r = 0 (0xdf3bec80) locked @ /mnt/pho/src/sys/kern/vfs_bio.c:1911 exclusive lockmgr ufs (ufs) r = 0 (0xc65805c8) locked @ /mnt/pho/src/sys/ufs/ffs/ffs_vfsops.c:1625 exclusive lockmgr ufs (ufs) r = 0 (0xc6581058) locked @ /mnt/pho/src/sys/kern/vfs_subr.c:2134 Process 1877 (mksnap_ffs) thread 0xc65aa2e0 (100132) exclusive sleep mutex Softdep Lock (Softdep Lock) r = 0 (0xc12155f0) locked @ /mnt/pho/src/sys/ufs/ffs/ffs_softdep.c:12011 exclusive lockmgr bufwait (bufwait) r = 0 (0xdf3bd900) locked @ /mnt/pho/src/sys/ufs/ffs/ffs_vnops.c:260 exclusive sleep mutex Giant (Giant) r = 0 (0xc1097470) locked @ /mnt/pho/src/sys/kern/vfs_mount.c:1069 exclusive lockmgr snaplk (snaplk) r = 0 (0xc5e3d41c) locked @ /mnt/pho/src/sys/ufs/ffs/ffs_snapshot.c:818 Process 1817 (sshd) thread 0xc6489000 (100116) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xc656d8ac) locked @ /mnt/pho/src/sys/kern/uipc_sockbuf.c:148 db:0:alllocks> show lockedvnods Locked vnodes 0xc6581000: 0xc6581000: tag ufs, type VDIR tag ufs, type VDIR usecount 2, writecount 0, refcount 6 mountedhere 0 usecount 2, writecount 0, refcount 6 mountedhere 0 flags (VV_ROOT|VI_MOUNT) flags (VV_ROOT|VI_MOUNT) v_object 0xc62cfe58 ref 0 pages 4 v_object 0xc62cfe58 ref 0 pages 4 lock type ufs: EXCL by thread 0xc5e6b5c0 (pid 1878) lock type ufs: EXCL by thread 0xc5e6b5c0 (pid 1878) with shared waiters pending with shared waiters pending #0 0xc09ef9a4 at __lockmgr_args+0x5c4 #0 0xc09ef9a4 at __lockmgr_args+0x5c4 #1 0xc0c4b791 at ffs_lock+0xa1 #1 0xc0c4b791 at ffs_lock+0xa1 #2 0xc0d616f5 at VOP_LOCK1_APV+0xb5 #2 0xc0d616f5 at VOP_LOCK1_APV+0xb5 #3 0xc0ab1a18 at _vn_lock+0x78 #3 0xc0ab1a18 at _vn_lock+0x78 #4 0xc0aa424b at vget+0xbb #4 0xc0aa424b at vget+0xbb #5 0xc0a94b8d at vfs_hash_get+0xed #5 0xc0a94b8d at vfs_hash_get+0xed #6 0xc0c44c79 at ffs_vgetf+0x49 #6 0xc0c44c79 at ffs_vgetf+0x49 #7 0xc0c451fe at ffs_vget+0x2e #7 0xc0c451fe at ffs_vget+0x2e #8 0xc0c59d58 at ufs_root+0x28 #8 0xc0c59d58 at ufs_root+0x28 #9 0xc0a96212 at lookup+0x9c2 #9 0xc0a96212 at lookup+0x9c2 #10 0xc0a96d6f at namei+0x5af #10 0xc0a96d6f at namei+0x5af #11 0xc0aa7bbb at kern_mkdirat+0xbb #11 0xc0aa7bbb at kern_mkdirat+0xbb #12 0xc0aa7e6e at kern_mkdir+0x2e #12 0xc0aa7e6e at kern_mkdir+0x2e #13 0xc0aa7e99 at mkdir+0x29 #13 0xc0aa7e99 at mkdir+0x29 #14 0xc0a4ad73 at syscallenter+0x263 #14 0xc0a4ad73 at syscallenter+0x263 #15 0xc0d3e65f at syscall+0x4f #15 0xc0d3e65f at syscall+0x4f #16 0xc0d27cf1 at Xint0x80_syscall+0x21 #16 0xc0d27cf1 at Xint0x80_syscall+0x21 ino 2, on dev md5a ino 2, on dev md5a 0xc6580984: 0xc6580984: tag ufs, type VREG tag ufs, type VREG usecount 1, writecount 0, refcount 83 mountedhere 0 usecount 1, writecount 0, refcount 83 mountedhere 0 flags (VV_SYSTEM) flags (VV_SYSTEM) lock type snaplk: EXCL by thread 0xc65aa2e0 (pid 1877) lock type snaplk: EXCL by thread 0xc65aa2e0 (pid 1877) with exclusive waiters pending with exclusive waiters pending #0 0xc09efffe at __lockmgr_args+0xc1e #0 0xc09efffe at __lockmgr_args+0xc1e #1 0xc0c4b791 at ffs_lock+0xa1 #1 0xc0c4b791 at ffs_lock+0xa1 #2 0xc0d616f5 at VOP_LOCK1_APV+0xb5 #2 0xc0d616f5 at VOP_LOCK1_APV+0xb5 #3 0xc0ab1a18 at _vn_lock+0x78 #3 0xc0ab1a18 at _vn_lock+0x78 #4 0xc0c2bbf1 at ffs_snapshot+0x29b1 #4 0xc0c2bbf1 at ffs_snapshot+0x29b1 #5 0xc0c48113 at ffs_mount+0x1c13 #5 0xc0c48113 at ffs_mount+0x1c13 #6 0xc0a9a484 at vfs_donmount+0x1254 #6 0xc0a9a484 at vfs_donmount+0x1254 #7 0xc0a9ab54 at nmount+0x84 #7 0xc0a9ab54 at nmount+0x84 #8 0xc0a4ad73 at syscallenter+0x263 #8 0xc0a4ad73 at syscallenter+0x263 #9 0xc0d3e65f at syscall+0x4f #9 0xc0d3e65f at syscall+0x4f #10 0xc0d27cf1 at Xint0x80_syscall+0x21 #10 0xc0d27cf1 at Xint0x80_syscall+0x21 ino 5, on dev md5a ino 5, on dev md5a 0xc6580570: 0xc6580570: tag ufs, type VDIR tag ufs, type VDIR usecount 1, writecount 0, refcount 2 mountedhere 0 usecount 1, writecount 0, refcount 2 mountedhere 0 flags () flags () lock type ufs: EXCL by thread 0xc5e6b5c0 (pid 1878) lock type ufs: EXCL by thread 0xc5e6b5c0 (pid 1878) #0 0xc09efffe at __lockmgr_args+0xc1e #0 0xc09efffe at __lockmgr_args+0xc1e #1 0xc0c44db9 at ffs_vgetf+0x189 #1 0xc0c44db9 at ffs_vgetf+0x189 #2 0xc0c451fe at ffs_vget+0x2e #2 0xc0c451fe at ffs_vget+0x2e #3 0xc0c18b00 at ffs_valloc+0x500 #3 0xc0c18b00 at ffs_valloc+0x500 #4 0xc0c5e479 at ufs_mkdir+0xa9 #4 0xc0c5e479 at ufs_mkdir+0xa9 #5 0xc0d61b75 at VOP_MKDIR_APV+0xc5 #5 0xc0d61b75 at VOP_MKDIR_APV+0xc5 #6 0xc0aa7d91 at kern_mkdirat+0x291 #6 0xc0aa7d91 at kern_mkdirat+0x291 #7 0xc0aa7e6e at kern_mkdir+0x2e #7 0xc0aa7e6e at kern_mkdir+0x2e #8 0xc0aa7e99 at mkdir+0x29 #8 0xc0aa7e99 at mkdir+0x29 #9 0xc0a4ad73 at syscallenter+0x263 #9 0xc0a4ad73 at syscallenter+0x263 #10 0xc0d3e65f at syscall+0x4f #10 0xc0d3e65f at syscall+0x4f #11 0xc0d27cf1 at Xint0x80_syscall+0x21 #11 0xc0d27cf1 at Xint0x80_syscall+0x21 ino 211968, on dev md5a ino 211968, on dev md5a db:0:lockedvnods> show mount 0xc5eb0b50 192.168.5.1:/zoo/leopard3 on / (nfs) 0xc5eb1000 devfs on /dev (devfs) 0xc63885a8 /dev/ada0d on /work (ufs) 0xc63882d4 /dev/md0 on /var (ufs) 0xc6388000 /dev/md1 on /tmp (ufs) 0xc5eb1b50 /dev/md5a on /mnt (ufs) More info: show mount db:0:mount> ps pid ppid pgrp uid state wmesg wchan cmd 1880 1879 1845 0 D ufs 0xc6581058 ls 1879 1845 1845 0 S wait 0xc659a000 sh 1878 1868 1878 1001 D+ snaplk 0xc5e3d41c run 1877 1875 1852 0 R+ CPU 1 mksnap_ffs 1876 1852 1852 0 S+ piperd 0xc647c000 grep 1875 1852 1852 0 S+ wait 0xc6598000 sh 1868 1852 1852 0 S+ wait 0xc6598840 su 1863 0 0 0 DL mdwait 0xc5ea1000 [md5] 1852 1851 1852 0 S+ wait 0xc62ceb00 sh 1851 1825 1851 0 S+ wait 0xc626d840 sh 1845 1 1845 0 Ss wait 0xc6597840 watchdogd 1825 1824 1825 0 S+ pause 0xc62cd898 csh 1824 1821 1824 1001 S+ wait 0xc62cd000 su 1821 1820 1821 1001 Ss+ wait 0xc6484580 sh 1820 1817 1817 1001 S select 0xc5e5faa4 sshd 1817 1719 1817 0 Ss sbwait 0xc656d8d8 sshd 1806 1 1806 0 Ss+ ttyin 0xc5a4d270 getty 1805 1 1805 0 Ss+ ttyin 0xc5a4e670 getty 1804 1 1804 0 Ss+ ttyin 0xc5a4e870 getty 1803 1 1803 0 Ss+ ttyin 0xc5a4ea70 getty 1802 1 1802 0 Ss+ ttyin 0xc5a4ec70 getty 1801 1 1801 0 Ss+ ttyin 0xc5a4ee70 getty 1800 1 1800 0 Ss+ ttyin 0xc5c2d070 getty 1799 1 1799 0 Ss+ ttyin 0xc5c2d270 getty 1798 1 1798 0 Ss+ ttyin 0xc5c2d470 getty 1760 1 1760 0 Ss select 0xc5e6e1a4 inetd 1730 1 1730 0 Ss nanslp 0xc1098d84 cron 1719 1 1719 0 Ss select 0xc5e6e4a4 sshd 1580 1 1580 0 Ss select 0xc5e5fb24 rpc.statd 1507 0 0 0 DL mdwait 0xc5ea3000 [md1] 1452 1 1452 0 Ss select 0xc5e5fb64 rpcbind 1365 1 1365 0 Ss select 0xc5e6e324 syslogd 1186 1 1186 0 Ss select 0xc5e6e364 devd 99 0 0 0 DL mdwait 0xc5ea4000 [md0] 17 0 0 0 DL sdflush 0xc1215640 [softdepflush] 16 0 0 0 DL vlruwt 0xc5e61840 [vnlru] 9 0 0 0 DL syncer 0xc120ef74 [syncer] 8 0 0 0 DL psleep 0xc120ec08 [bufdaemon] 7 0 0 0 DL pgzero 0xc12174dc [pagezero] 6 0 0 0 DL psleep 0xc12170c8 [vmdaemon] 5 0 0 0 DL psleep 0xc1217090 [pagedaemon] 4 0 0 0 DL ccb_scan 0xc105dad4 [xpt_thrd] 3 0 0 0 DL waiting_ 0xc1210a98 [sctp_iterator] 2 0 0 0 DL - 0xc5b3de3c [fdc0] 15 0 0 0 DL (threaded) [usb] 100049 D - 0xc5be2d34 [usbus4] 100048 D - 0xc5be2d04 [usbus4] 100047 D - 0xc5be2cd4 [usbus4] 100046 D - 0xc5be2ca4 [usbus4] 100045 D - 0xc5bccdac [usbus3] 100044 D - 0xc5bccd7c [usbus3] 100043 D - 0xc5bccd4c [usbus3] 100042 D - 0xc5bccd1c [usbus3] 100040 D - 0xc5bb9dac [usbus2] 100039 D - 0xc5bb9d7c [usbus2] 100038 D - 0xc5bb9d4c [usbus2] 100037 D - 0xc5bb9d1c [usbus2] 100035 D - 0xc5ba2dac [usbus1] 100034 D - 0xc5ba2d7c [usbus1] 100033 D - 0xc5ba2d4c [usbus1] 100032 D - 0xc5ba2d1c [usbus1] 100030 D - 0xc5b88dac [usbus0] 100029 D - 0xc5b88d7c [usbus0] 100028 D - 0xc5b88d4c [usbus0] 100027 D - 0xc5b88d1c [usbus0] 14 0 0 0 DL - 0xc1097b84 [yarrow] 13 0 0 0 DL (threaded) [geom] 100011 D - 0xc10957a8 [g_down] 100010 D - 0xc10957a4 [g_up] 100009 D - 0xc109579c [g_event] 12 0 0 0 RL (threaded) [intr] 100054 I [irq1: atkbd0] 100052 Run CPU 0 [swi0: uart uart] 100051 I [irq258: ahci0] 100050 I [irq14: ata0] 100041 I [irq16: uhci3] 100036 I [irq18: uhci2] 100031 I [irq19: uhci1] 100026 I [irq23: uhci0 ehci0] 100023 I [swi5: +] 100022 I [swi2: cambio] 100017 I [swi6: task queue] 100015 I [swi6: Giant taskq] 100008 I [swi3: vm] 100007 I [swi4: clock] 100006 L *Giant 0xc5961e80 [swi4: clock] 100005 I [swi1: netisr 0] 11 0 0 0 RL (threaded) [idle] 100004 CanRun [idle: cpu1] 100003 CanRun [idle: cpu0] 1 0 1 0 SLs wait 0xc59a5b00 [init] 10 0 0 0 DL audit_wo 0xc1214d60 [audit] 0 0 0 0 DLs (threaded) [kernel] 100065 D - 0xc5c02180 [mca taskq] 100055 D - 0xc1097b84 [deadlkres] 100025 D - 0xc5b51740 [em1 taskq] 100024 D - 0xc5b51980 [em0 taskq] 100021 D - 0xc59a8100 [acpi_task_2] 100020 D - 0xc59a8100 [acpi_task_1] 100019 D - 0xc59a8100 [acpi_task_0] 100018 D - 0xc59a8140 [kqueue taskq] 100016 D - 0xc59a81c0 [ffs_trim taskq] 100014 D - 0xc59a84c0 [thread taskq] 100012 D - 0xc59a8500 [firmware taskq] 100000 D sched 0xc1095880 [swapper] db:0:ps> allt Tracing command ls pid 1880 tid 100130 td 0xc63612e0 sched_switch(c63612e0,0,104,191,ad43edd8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,60,...) at mi_switch+0x1eb sleepq_switch(c63612e0,0,c0f02600,268,c6581058,...) at sleepq_switch+0x162 sleepq_wait(c6581058,60,c0ef378c,4,1,...) at sleepq_wait+0x63 __lockmgr_args(c6581058,200100,c65810c4,0,0,...) at __lockmgr_args+0x4ed ffs_lock(edd9b894,c0a524ab,c0f0b852,200100,c6581000,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c104be20,edd9b894,c6361390,c105c620,c6581000,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c6581000,200100,c0f0c516,856,4,...) at _vn_lock+0x78 vget(c6581000,200100,c63612e0,50,0,...) at vget+0xbb vfs_hash_get(c5eb1b50,2,200000,c63612e0,edd9b9a8,...) at vfs_hash_get+0xed ffs_vgetf(c5eb1b50,2,200000,edd9b9a8,0,...) at ffs_vgetf+0x49 ffs_vget(c5eb1b50,2,200000,edd9b9a8,edd9bab8,...) at ffs_vget+0x2e ufs_root(c5eb1b50,200000,edd9ba14,212,edd9bae4,...) at ufs_root+0x28 lookup(edd9bab8,c0f0b949,109,d7,edd9ba58,...) at lookup+0x9c2 namei(edd9bab8,edd9bb20,60,0,c63612e0,...) at namei+0x5af kern_statat_vnhook(c63612e0,200,ffffff9c,2841d3b8,0,...) at kern_statat_vnhook+0xbf kern_statat(c63612e0,200,ffffff9c,2841d3b8,0,...) at kern_statat+0x3c kern_lstat(c63612e0,2841d3b8,0,edd9bbe8,200ff00,...) at kern_lstat+0x36 lstat(c63612e0,edd9bcec,edd9bd28,c0f02f8a,0,...) at lstat+0x2f syscallenter(c63612e0,edd9bce4,edd9bce4,0,0,...) at syscallenter+0x263 syscall(edd9bd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (190, FreeBSD ELF32, lstat), eip = 0x281c8af3, esp = 0xbfbfe3fc, ebp = 0xbfbfe488 --- Tracing command sh pid 1879 tid 100126 td 0xc6485000 sched_switch(c6485000,0,104,191,acdcc740,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c6485000,0,c0f02600,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c659a000,6c,c0f0545a,100,0,...) at sleepq_wait_sig+0x17 _sleep(c659a000,c659a088,16c,c0f0545a,0,...) at _sleep+0x35c kern_wait(c6485000,ffffffff,edd8fc44,2,0,...) at kern_wait+0xb96 wait4(c6485000,edd8fcec,28206b08,1,0,...) at wait4+0x3b syscallenter(c6485000,edd8fce4,c0d3eebd,c1097470,0,...) at syscallenter+0x263 syscall(edd8fd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x28161ffb, esp = 0xbfbfe90c, ebp = 0xbfbfe928 --- Tracing command run pid 1878 tid 100073 td 0xc5e6b5c0 sched_switch(c5e6b5c0,0,104,191,96d81a00,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,60,...) at mi_switch+0x1eb sleepq_switch(c5e6b5c0,0,c0f02600,268,60,...) at sleepq_switch+0x162 sleepq_wait(c5e3d41c,60,c0f30065,4,0,...) at sleepq_wait+0x63 __lockmgr_args(c5e3d41c,80900,c65b20c4,0,0,...) at __lockmgr_args+0xb3f ffs_copyonwrite(c65b2000,df3afa00,0,df3afa00,c6581000,...) at ffs_copyonwrite+0x2c2 ffs_geom_strategy(c65b20f8,df3afa00,df78d850,c5e6b5c0,c62cfe58,...) at ffs_geom_strategy+0xeb ufs_strategy(df78d88c,c0f52597,c62cfe58,df3afa00,0,...) at ufs_strategy+0xd3 VOP_STRATEGY_APV(c104be20,df78d88c,c62cfe58,c105c680,c6581000,...) at VOP_STRATEGY_APV+0xe5 bufstrategy(c65810f8,df3afa00,df3afa00,c6581000,0,...) at bufstrategy+0x61 bufwrite(df3afa00,e19ac000,0,df78d918,df78d91c,...) at bufwrite+0x159 ufs_direnter(c6581000,c6580570,df78d9d8,df78db68,df3bec20,...) at ufs_direnter+0x8ac ufs_mkdir(df78dbf8,c0f526d6,0,0,df78db90,...) at ufs_mkdir+0xa36 VOP_MKDIR_APV(c104be20,df78dbf8,df78db68,df78db90,0,...) at VOP_MKDIR_APV+0xc5 kern_mkdirat(c5e6b5c0,ffffff9c,bfbfedcf,0,1f8,...) at kern_mkdirat+0x291 kern_mkdir(c5e6b5c0,bfbfedcf,0,1f8,df78dc7c,...) at kern_mkdir+0x2e mkdir(c5e6b5c0,df78dcec,df78dd28,c0f02f8a,0,...) at mkdir+0x29 syscallenter(c5e6b5c0,df78dce4,df78dce4,2,0,...) at syscallenter+0x263 syscall(df78dd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, mkdir), eip = 0x28185b33, esp = 0xbfbfe93c, ebp = 0xbfbfe9c8 --- Tracing command mksnap_ffs pid 1877 tid 100132 td 0xc65aa2e0 cpustop_handler(1,13,edda1428,c0d3e776,c0d580a5,...) at cpustop_handler+0x28 ipi_nmi_handler(c0d580a5,c1277080,0,1,c6597b00,...) at ipi_nmi_handler+0x47 trap(edda1434) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0c3bfa5, esp = 0xedda1474, ebp = 0xedda147c --- jwait(c12155f0,0,c0f305c0,2eeb,edda14d0,...) at jwait+0x95 softdep_sync_buf(c6580984,df3bd8a0,1,106,0,...) at softdep_sync_buf+0xb8 ffs_syncvnode(c6580984,1,c0f2ffde,34c,0,...) at ffs_syncvnode+0x24c ffs_snapshot(c5eb1b50,c5eb8c40,c0f33490,1a2,0,...) at ffs_snapshot+0x2bf3 ffs_mount(c5eb1b50,c5e3d500,ff,394,c5944450,...) at ffs_mount+0x1c13 vfs_donmount(c65aa2e0,11000,c5e4c200,c5e4c200,c6597b00,...) at vfs_donmount+0x1254 nmount(c65aa2e0,edda1cec,edda1d28,c0f02f8a,0,...) at nmount+0x84 syscallenter(c65aa2e0,edda1ce4,edda1ce4,0,0,...) at syscallenter+0x263 syscall(edda1d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (378, FreeBSD ELF32, nmount), eip = 0x280da65b, esp = 0xbfbfe25c, ebp = 0xbfbfe9d8 --- Tracing command grep pid 1876 tid 100080 td 0xc5e6cb80 sched_switch(c5e6cb80,0,104,191,27b08d48,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c5e6cb80,0,c0f02600,1a7,5c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c647c000,5c,c0f04e7b,100,0,...) at sleepq_wait_sig+0x17 _sleep(c647c000,c647c170,15c,c0f04e7b,0,...) at _sleep+0x35c pipe_read(c6429118,df7aac28,c5985100,0,c5e6cb80,...) at pipe_read+0x417 dofileread(df7aac28,ffffffff,ffffffff,0,df7aac04,...) at dofileread+0x9e kern_readv(c5e6cb80,0,df7aac28,df7aac48,1,...) at kern_readv+0x68 read(c5e6cb80,df7aacec,df7aad28,c0f02f8a,0,...) at read+0x4f syscallenter(c5e6cb80,df7aace4,df7aace4,0,0,...) at syscallenter+0x263 syscall(df7aad28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x281c6dc3, esp = 0xbfbfe7bc, ebp = 0xbfbfe848 --- Tracing command sh pid 1875 tid 100131 td 0xc65aa5c0 sched_switch(c65aa5c0,0,104,191,27457df0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c65aa5c0,0,c0f02600,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6598000,6c,c0f0545a,100,0,...) at sleepq_wait_sig+0x17 _sleep(c6598000,c6598088,16c,c0f0545a,0,...) at _sleep+0x35c kern_wait(c65aa5c0,ffffffff,edd9ec44,2,0,...) at kern_wait+0xb96 wait4(c65aa5c0,edd9ecec,edd9ed28,c0f02f8a,0,...) at wait4+0x3b syscallenter(c65aa5c0,edd9ece4,edd9ece4,0,c10a5a00,...) at syscallenter+0x263 syscall(edd9ed28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x28161ffb, esp = 0xbfbfe73c, ebp = 0xbfbfe758 --- Tracing command su pid 1868 tid 100128 td 0xc63618a0 sched_switch(c63618a0,0,104,191,286c62f8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c63618a0,0,c0f02600,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6598840,6c,c0f0545a,100,0,...) at sleepq_wait_sig+0x17 _sleep(c6598840,c65988c8,16c,c0f0545a,0,...) at _sleep+0x35c kern_wait(c63618a0,756,edd95c44,2,0,...) at kern_wait+0xb96 wait4(c63618a0,edd95cec,281c8b08,1,0,...) at wait4+0x3b syscallenter(c63618a0,edd95ce4,c0d3eebd,c1097470,0,...) at syscallenter+0x263 syscall(edd95d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x28123ffb, esp = 0xbfbfe49c, ebp = 0xbfbfe4b8 --- Tracing command md5 pid 1863 tid 100088 td 0xc629b000 sched_switch(c629b000,0,104,191,58eae588,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c629b000,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c5ea1000,5c,c0ea2001,0,0,...) at sleepq_wait+0x63 _sleep(c5ea1000,c5ea1020,25c,c0ea2001,0,...) at _sleep+0x372 md_kthread(c5ea1000,df7cdd28,c0ef75b0,3a5,c626d000,...) at md_kthread+0x11a fork_exit(c0732bf0,c5ea1000,df7cdd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf7cdd60, ebp = 0 --- Tracing command sh pid 1852 tid 100108 td 0xc62d22e0 sched_switch(c62d22e0,0,104,191,27410478,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c62d22e0,0,c0f02600,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c62ceb00,6c,c0f0545a,100,0,...) at sleepq_wait_sig+0x17 _sleep(c62ceb00,c62ceb88,16c,c0f0545a,0,...) at _sleep+0x35c kern_wait(c62d22e0,ffffffff,edd31c44,2,0,...) at kern_wait+0xb96 wait4(c62d22e0,edd31cec,edd31d28,c0f02f8a,0,...) at wait4+0x3b syscallenter(c62d22e0,edd31ce4,edd31ce4,0,c10a5a00,...) at syscallenter+0x263 syscall(edd31d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x28161ffb, esp = 0xbfbfe9ac, ebp = 0xbfbfe9c8 --- Tracing command sh pid 1851 tid 100085 td 0xc629b8a0 sched_switch(c629b8a0,0,104,191,fb3d850,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c629b8a0,0,c0f02600,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c626d840,6c,c0f0545a,100,0,...) at sleepq_wait_sig+0x17 _sleep(c626d840,c626d8c8,16c,c0f0545a,0,...) at _sleep+0x35c kern_wait(c629b8a0,ffffffff,df7c0c44,2,0,...) at kern_wait+0xb96 wait4(c629b8a0,df7c0cec,28206b08,1,0,...) at wait4+0x3b syscallenter(c629b8a0,df7c0ce4,c0d3eebd,c1097470,0,...) at syscallenter+0x263 syscall(df7c0d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x28161ffb, esp = 0xbfbfe9ac, ebp = 0xbfbfe9c8 --- Tracing command watchdogd pid 1845 tid 100133 td 0xc65aa000 sched_switch(c65aa000,0,104,191,ac50d028,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c65aa000,0,c0f02600,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6597840,6c,c0f0545a,100,0,...) at sleepq_wait_sig+0x17 _sleep(c6597840,c65978c8,16c,c0f0545a,0,...) at _sleep+0x35c kern_wait(c65aa000,757,edda4c44,0,0,...) at kern_wait+0xb96 wait4(c65aa000,edda4cec,bfbfea3c,1,0,...) at wait4+0x3b syscallenter(c65aa000,edda4ce4,c0d3eebd,c1097470,0,...) at syscallenter+0x263 syscall(edda4d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x2811cffb, esp = 0xbfbfea2c, ebp = 0xbfbfead8 --- Tracing command csh pid 1825 tid 100091 td 0xc629a5c0 sched_switch(c629a5c0,0,104,191,e83834d0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,74,...) at mi_switch+0x1eb sleepq_switch(c629a5c0,0,c0f02600,1a7,74,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c62cd898,74,c0d8dd4e,100,0,...) at sleepq_wait_sig+0x17 _sleep(c62cd898,c62cd8c8,174,c0d8dd4e,0,...) at _sleep+0x35c kern_sigsuspend(c629a5c0,0,0,0,0,...) at kern_sigsuspend+0xba sigsuspend(c629a5c0,df7ddcec,df7ddd28,c0f02f8a,0,...) at sigsuspend+0x4d syscallenter(c629a5c0,df7ddce4,df7ddce4,0,246,...) at syscallenter+0x263 syscall(df7ddd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, write), eip = 0x2817d8fb, esp = 0xbfbfe74c, ebp = 0xbfbfe7b8 --- Tracing command su pid 1824 tid 100094 td 0xc6299b80 sched_switch(c6299b80,0,104,191,f8401418,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c6299b80,0,c0f02600,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c62cd000,6c,c0f0545a,100,0,...) at sleepq_wait_sig+0x17 _sleep(c62cd000,c62cd088,16c,c0f0545a,0,...) at _sleep+0x35c kern_wait(c6299b80,721,df7e6c44,2,0,...) at kern_wait+0xb96 wait4(c6299b80,df7e6cec,281d1b08,1,0,...) at wait4+0x3b syscallenter(c6299b80,df7e6ce4,c0d3eebd,c1097470,0,...) at syscallenter+0x263 syscall(df7e6d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x2812cffb, esp = 0xbfbfe77c, ebp = 0xbfbfe798 --- Tracing command sh pid 1821 tid 100113 td 0xc62d12e0 sched_switch(c62d12e0,0,104,191,f1a584e0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c62d12e0,0,c0f02600,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6484580,6c,c0f0545a,100,0,...) at sleepq_wait_sig+0x17 _sleep(c6484580,c6484608,16c,c0f0545a,0,...) at _sleep+0x35c kern_wait(c62d12e0,ffffffff,edd68c44,2,0,...) at kern_wait+0xb96 wait4(c62d12e0,edd68cec,edd68d28,c0f02f8a,0,...) at wait4+0x3b syscallenter(c62d12e0,edd68ce4,edd68ce4,0,0,...) at syscallenter+0x263 syscall(edd68d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x28161ffb, esp = 0xbfbfeaac, ebp = 0xbfbfeac8 --- Tracing command sshd pid 1820 tid 100082 td 0xc5e6c5c0 sched_switch(c5e6c5c0,0,104,191,13811428,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5e6c5c0,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c09f7eba,c5e5fa90,0,c0efb2e7,c5e6c5c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5e5faa4,0,df7b3a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5e5faa4,c5e5fa90,c0f04cae,64c,c6496150,...) at _cv_wait_sig+0x243 seltdwait(c6496150,58,c5986280,c5e6c5c0,c0f25b82,...) at seltdwait+0xa2 kern_select(c5e6c5c0,a,288060b8,288060cc,0,0,20,c626e2c0,c626e2c0) at kern_select+0x504 select(c5e6c5c0,df7b3cec,df7b3d28,c0f02f8a,0,...) at select+0x66 syscallenter(c5e6c5c0,df7b3ce4,df7b3ce4,0,0,...) at syscallenter+0x263 syscall(df7b3d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x284b0d43, esp = 0xbfbfe19c, ebp = 0xbfbfe1e8 --- Tracing command sshd pid 1817 tid 100116 td 0xc6489000 sched_switch(c6489000,0,104,191,463cb3f0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,68,...) at mi_switch+0x1eb sleepq_switch(c6489000,0,c0f02600,1a7,68,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c656d8d8,68,c0f08b71,100,0,...) at sleepq_wait_sig+0x17 _sleep(c656d8d8,c656d898,168,c0f08b71,0) at _sleep+0x35c sbwait(c656d874,4,c0f08c49,5ec,c656d898,...) at sbwait+0x76 soreceive_generic(c656d820,0,edd71c28,0,0,...) at soreceive_generic+0x320 soreceive(c656d820,0,edd71c28,0,0,0) at soreceive+0x38 soo_read(c62387e0,edd71c28,c5986e80,0,c6489000,...) at soo_read+0x4e dofileread(edd71c28,ffffffff,ffffffff,0,edd71c04,...) at dofileread+0x9e kern_readv(c6489000,5,edd71c28,edd71c48,1,...) at kern_readv+0x68 read(c6489000,edd71cec,edd71d28,c0f02f8a,0,...) at read+0x4f syscallenter(c6489000,edd71ce4,edd71ce4,0,c10a6080,...) at syscallenter+0x263 syscall(edd71d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x284b0dc3, esp = 0xbfbfdd6c, ebp = 0xbfbfdda8 --- Tracing command getty pid 1806 tid 100101 td 0xc63608a0 sched_switch(c63608a0,0,104,191,f9294700,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c63608a0,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c09f7eba,c5a4d204,0,c0efb2e7,c63608a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5a4d270,0,df7fead4,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5a4d270,c5a4d204,c0f06d24,524,0,...) at _cv_wait_sig+0x243 tty_wait(c5a4d200,c5a4d270,df7fec28,1,0,...) at tty_wait+0x71 ttydisc_read(c5a4d200,df7fec28,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c5c21300,df7fec28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c6238118,df7fec28,c5985e80,0,c63608a0,...) at devfs_read_f+0x7e dofileread(df7fec28,ffffffff,ffffffff,0,df7fec04,...) at dofileread+0x9e kern_readv(c63608a0,0,df7fec28,df7fec48,1,...) at kern_readv+0x68 read(c63608a0,df7fecec,df7fed28,c0f02f8a,0,...) at read+0x4f syscallenter(c63608a0,df7fece4,df7fece4,0,0,...) at syscallenter+0x263 syscall(df7fed28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1805 tid 100087 td 0xc629b2e0 sched_switch(c629b2e0,0,104,191,c2847af8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c629b2e0,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,c629b2e0,c629b2e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5a4e670,0,c0f07364,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5a4e670,c1097470,c0f06d24,524,0,...) at _cv_wait_sig+0x243 tty_wait(c5a4e600,c5a4e670,df7c9c28,1,0,...) at tty_wait+0x71 ttydisc_read(c5a4e600,df7c9c28,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c5c21e00,df7c9c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c6226380,df7c9c28,c5985e80,0,c629b2e0,...) at devfs_read_f+0x7e dofileread(df7c9c28,ffffffff,ffffffff,0,df7c9c04,...) at dofileread+0x9e kern_readv(c629b2e0,0,df7c9c28,df7c9c48,1,...) at kern_readv+0x68 read(c629b2e0,df7c9cec,df7c9d28,c0f02f8a,0,...) at read+0x4f syscallenter(c629b2e0,df7c9ce4,df7c9ce4,0,0,...) at syscallenter+0x263 syscall(df7c9d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1804 tid 100079 td 0xc5e6b000 sched_switch(c5e6b000,0,104,191,c2b98df0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5e6b000,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,c5e6b000,c5e6b000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5a4e870,0,c0f07364,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5a4e870,c1097470,c0f06d24,524,0,...) at _cv_wait_sig+0x243 tty_wait(c5a4e800,c5a4e870,df7a5c28,1,0,...) at tty_wait+0x71 ttydisc_read(c5a4e800,df7a5c28,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c5c4b000,df7a5c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c6238268,df7a5c28,c5985e80,0,c5e6b000,...) at devfs_read_f+0x7e dofileread(df7a5c28,ffffffff,ffffffff,0,df7a5c04,...) at dofileread+0x9e kern_readv(c5e6b000,0,df7a5c28,df7a5c48,1,...) at kern_readv+0x68 read(c5e6b000,df7a5cec,df7a5d28,c0f02f8a,0,...) at read+0x4f syscallenter(c5e6b000,df7a5ce4,df7a5ce4,0,0,...) at syscallenter+0x263 syscall(df7a5d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1803 tid 100097 td 0xc62992e0 sched_switch(c62992e0,0,104,191,c2ad8e60,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c62992e0,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,c62992e0,c62992e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5a4ea70,0,c0f07364,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5a4ea70,c1097470,c0f06d24,524,0,...) at _cv_wait_sig+0x243 tty_wait(c5a4ea00,c5a4ea70,df7f0c28,1,0,...) at tty_wait+0x71 ttydisc_read(c5a4ea00,df7f0c28,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c5c4b100,df7f0c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c6226b98,df7f0c28,c5985e80,0,c62992e0,...) at devfs_read_f+0x7e dofileread(df7f0c28,ffffffff,ffffffff,0,df7f0c04,...) at dofileread+0x9e kern_readv(c62992e0,0,df7f0c28,df7f0c48,1,...) at kern_readv+0x68 read(c62992e0,df7f0cec,df7f0d28,c0f02f8a,0,...) at read+0x4f syscallenter(c62992e0,df7f0ce4,df7f0ce4,0,0,...) at syscallenter+0x263 syscall(df7f0d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1802 tid 100096 td 0xc62995c0 sched_switch(c62995c0,0,104,191,c2678bc8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c62995c0,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,c62995c0,c62995c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5a4ec70,0,c0f07364,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5a4ec70,c1097470,c0f06d24,524,0,...) at _cv_wait_sig+0x243 tty_wait(c5a4ec00,c5a4ec70,df7edc28,1,0,...) at tty_wait+0x71 ttydisc_read(c5a4ec00,df7edc28,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c5c4b200,df7edc28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c6226d58,df7edc28,c5985e80,0,c62995c0,...) at devfs_read_f+0x7e dofileread(df7edc28,ffffffff,ffffffff,0,df7edc04,...) at dofileread+0x9e kern_readv(c62995c0,0,df7edc28,df7edc48,1,...) at kern_readv+0x68 read(c62995c0,df7edcec,df7edd28,c0f02f8a,0,...) at read+0x4f syscallenter(c62995c0,df7edce4,df7edce4,0,0,...) at syscallenter+0x263 syscall(df7edd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1801 tid 100076 td 0xc62195c0 sched_switch(c62195c0,0,104,191,c2a43840,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c62195c0,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,c62195c0,c62195c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5a4ee70,0,c0f07364,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5a4ee70,c1097470,c0f06d24,524,0,...) at _cv_wait_sig+0x243 tty_wait(c5a4ee00,c5a4ee70,df799c28,1,0,...) at tty_wait+0x71 ttydisc_read(c5a4ee00,df799c28,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c5c4b300,df799c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c6429930,df799c28,c5985e80,0,c62195c0,...) at devfs_read_f+0x7e dofileread(df799c28,ffffffff,ffffffff,0,df799c04,...) at dofileread+0x9e kern_readv(c62195c0,0,df799c28,df799c48,1,...) at kern_readv+0x68 read(c62195c0,df799cec,df799d28,c0f02f8a,0,...) at read+0x4f syscallenter(c62195c0,df799ce4,df799ce4,0,0,...) at syscallenter+0x263 syscall(df799d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1800 tid 100112 td 0xc62d15c0 sched_switch(c62d15c0,0,104,191,c23b77a0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c62d15c0,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,c62d15c0,c62d15c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5c2d070,0,c0f07364,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5c2d070,c1097470,c0f06d24,524,0,...) at _cv_wait_sig+0x243 tty_wait(c5c2d000,c5c2d070,edd65c28,1,0,...) at tty_wait+0x71 ttydisc_read(c5c2d000,edd65c28,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c5c4b400,edd65c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c6227b28,edd65c28,c5985e80,0,c62d15c0,...) at devfs_read_f+0x7e dofileread(edd65c28,ffffffff,ffffffff,0,edd65c04,...) at dofileread+0x9e kern_readv(c62d15c0,0,edd65c28,edd65c48,1,...) at kern_readv+0x68 read(c62d15c0,edd65cec,edd65d28,c0f02f8a,0,...) at read+0x4f syscallenter(c62d15c0,edd65ce4,edd65ce4,0,0,...) at syscallenter+0x263 syscall(edd65d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1799 tid 100077 td 0xc62192e0 sched_switch(c62192e0,0,104,191,c29bc330,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c62192e0,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,c62192e0,c62192e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5c2d270,0,c0f07364,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5c2d270,c1097470,c0f06d24,524,0,...) at _cv_wait_sig+0x243 tty_wait(c5c2d200,c5c2d270,df79dc28,1,0,...) at tty_wait+0x71 ttydisc_read(c5c2d200,df79dc28,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c5aca300,df79dc28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c6238818,df79dc28,c5985e80,0,c62192e0,...) at devfs_read_f+0x7e dofileread(df79dc28,ffffffff,ffffffff,0,df79dc04,...) at dofileread+0x9e kern_readv(c62192e0,0,df79dc28,df79dc48,1,...) at kern_readv+0x68 read(c62192e0,df79dcec,df79dd28,c0f02f8a,0,...) at read+0x4f syscallenter(c62192e0,df79dce4,df79dce4,0,0,...) at syscallenter+0x263 syscall(df79dd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1798 tid 100118 td 0xc64868a0 sched_switch(c64868a0,0,104,191,c2c4bc28,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c64868a0,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,c64868a0,c64868a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5c2d470,0,c0f07364,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5c2d470,c1097470,c0f06d24,524,0,...) at _cv_wait_sig+0x243 tty_wait(c5c2d400,c5c2d470,edd77c28,1,0,...) at tty_wait+0x71 ttydisc_read(c5c2d400,edd77c28,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c5aca400,edd77c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c6226ce8,edd77c28,c5985e80,0,c64868a0,...) at devfs_read_f+0x7e dofileread(edd77c28,ffffffff,ffffffff,0,edd77c04,...) at dofileread+0x9e kern_readv(c64868a0,0,edd77c28,edd77c48,1,...) at kern_readv+0x68 read(c64868a0,edd77cec,edd77d28,c0f02f8a,0,...) at read+0x4f syscallenter(c64868a0,edd77ce4,edd77ce4,0,0,...) at syscallenter+0x263 syscall(edd77d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command inetd pid 1760 tid 100086 td 0xc629b5c0 sched_switch(c629b5c0,0,104,191,ae5b8500,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c629b5c0,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c09f7eba,c5e6e190,0,c0efb2e7,c629b5c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5e6e1a4,0,df7c4a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5e6e1a4,c5e6e190,c0f04cae,64c,c6429e38,...) at _cv_wait_sig+0x243 seltdwait(c6429e38,58,c5985e80,c629b5c0,c0a5254c,...) at seltdwait+0xa2 kern_select(c629b5c0,8,bfbfdfa0,0,0,0,20,c626d580,c626d580) at kern_select+0x504 select(c629b5c0,df7c4cec,df7c4d28,c0f02f8a,0,...) at select+0x66 syscallenter(c629b5c0,df7c4ce4,df7c4ce4,0,0,...) at syscallenter+0x263 syscall(df7c4d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x281a8d43, esp = 0xbfbfdf5c, ebp = 0xbfbfede8 --- Tracing command cron pid 1730 tid 100107 td 0xc62d25c0 sched_switch(c62d25c0,0,104,191,a85fda48,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c62d25c0,0,c0f02600,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(ea61,c0a472d0,c62d25c0,0,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c1098d84,6c,c0efe2df,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c1098d84,0,16c,c0efe2df,ea61,...) at _sleep+0x328 kern_nanosleep(c62d25c0,edd2ec34,edd2ec3c,3c,0,...) at kern_nanosleep+0xc1 nanosleep(c62d25c0,edd2ecec,edd2ed28,c0f02f8a,0,...) at nanosleep+0x6f syscallenter(c62d25c0,edd2ece4,edd2ece4,0,0,...) at syscallenter+0x263 syscall(edd2ed28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x281850b7, esp = 0xbfbfec4c, ebp = 0xbfbfec78 --- Tracing command sshd pid 1719 tid 100084 td 0xc629bb80 sched_switch(c629bb80,0,104,191,2f899b88,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c629bb80,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c09f7eba,c5e6e490,0,c0efb2e7,c629bb80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5e6e4a4,0,df7bba4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5e6e4a4,c5e6e490,c0f04cae,64c,c6226af0,...) at _cv_wait_sig+0x243 seltdwait(c6226af0,58,c5986080,c629bb80,c59aa570,...) at seltdwait+0xa2 kern_select(c629bb80,5,2880c0b0,0,0,0,20,c626db00,c626db00) at kern_select+0x504 select(c629bb80,df7bbcec,2880d070,1,0,...) at select+0x66 syscallenter(c629bb80,df7bbce4,c0d3eebd,c1097470,0,...) at syscallenter+0x263 syscall(df7bbd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x284b0d43, esp = 0xbfbfe25c, ebp = 0xbfbfedf8 --- Tracing command rpc.statd pid 1580 tid 100109 td 0xc62d2000 sched_switch(c62d2000,0,104,191,e3049898,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c62d2000,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(edd34a14,c09f7eba,c5e5fb10,0,c62d2000,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c5e5fb24,0,edd34a4c,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c5e5fb24,c5e5fb10,7531,64c,c6227a48,...) at _cv_timedwait_sig+0x252 seltdwait(edd34bf8,edd34c00,c5985e80,c62d2000,28801000,...) at seltdwait+0x8a kern_select(c62d2000,8,bfbfed14,0,0,edd34c40,20,1e,0) at kern_select+0x504 select(c62d2000,edd34cec,edd34d28,c0f02f8a,0,...) at select+0x66 syscallenter(c62d2000,edd34ce4,edd34ce4,0,0,...) at syscallenter+0x263 syscall(edd34d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x2818cd43, esp = 0xbfbfec6c, ebp = 0xbfbfeda8 --- Tracing command md1 pid 1507 tid 100110 td 0xc62d1b80 sched_switch(c62d1b80,0,104,191,1e870200,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c62d1b80,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c5ea3000,5c,c0ea2001,0,0,...) at sleepq_wait+0x63 _sleep(c5ea3000,c5ea3020,25c,c0ea2001,0,...) at _sleep+0x372 md_kthread(c5ea3000,edd37d28,c0ef75b0,3a5,c62ce580,...) at md_kthread+0x11a fork_exit(c0732bf0,c5ea3000,edd37d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xedd37d60, ebp = 0 --- Tracing command rpcbind pid 1452 tid 100098 td 0xc6299000 sched_switch(c6299000,0,104,191,c021c500,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c6299000,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(df7f5a6c,c09f7eba,c5e5fb50,0,c6299000,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c5e5fb64,0,df7f5aa4,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c5e5fb64,c5e5fb50,7531,64c,df7f5b5c,...) at _cv_timedwait_sig+0x252 seltdwait(df7f5c2c,df7f5c34,55a,c6299000,df7f5b2c,...) at seltdwait+0x8a poll(c6299000,df7f5cec,df7f5d28,c0f02f8a,0,...) at poll+0x300 syscallenter(c6299000,df7f5ce4,df7f5ce4,0,c10a6080,...) at syscallenter+0x263 syscall(df7f5d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, poll), eip = 0x2813ff4b, esp = 0xbfbfcbac, ebp = 0xbfbfed78 --- Tracing command syslogd pid 1365 tid 100105 td 0xc62d2b80 sched_switch(c62d2b80,0,104,191,5c2571b8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c62d2b80,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c09f7eba,c5e6e310,0,c0efb2e7,c62d2b80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5e6e324,0,edd28a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5e6e324,c5e6e310,c0f04cae,64c,c6238738,...) at _cv_wait_sig+0x243 seltdwait(c6238738,58,c5985e80,c62d2b80,5e8,...) at seltdwait+0xa2 kern_select(c62d2b80,9,284760ac,0,0,0,20,c6380580,c6380580) at kern_select+0x504 select(c62d2b80,edd28cec,edd28d28,c0f02f8a,0,...) at select+0x66 syscallenter(c62d2b80,edd28ce4,edd28ce4,0,0,...) at syscallenter+0x263 syscall(edd28d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x28199d43, esp = 0xbfbfe29c, ebp = 0xbfbfede8 --- Tracing command devd pid 1186 tid 100081 td 0xc5e6c8a0 sched_switch(c5e6c8a0,0,104,191,23b80400,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5e6c8a0,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c09f7eba,c5e6e350,0,c0efb2e7,c5e6c8a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5e6e364,0,df7aea4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5e6e364,c5e6e350,c0f04cae,64c,c6227e00,...) at _cv_wait_sig+0x243 seltdwait(c6227e00,58,c5985e80,c5e6c8a0,c5e6c8a0,...) at seltdwait+0xa2 kern_select(c5e6c8a0,5,bfbfe960,0,0,0,20,c626e580,c626e580) at kern_select+0x504 select(c5e6c8a0,df7aecec,df7aed28,c0f02f8a,0,...) at select+0x66 syscallenter(c5e6c8a0,df7aece4,df7aece4,0,0,...) at syscallenter+0x263 syscall(df7aed28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x808ba43, esp = 0xbfbfe92c, ebp = 0xbfbfee08 --- Tracing command md0 pid 99 tid 100106 td 0xc62d28a0 sched_switch(c62d28a0,0,104,191,5c24f840,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c62d28a0,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c5ea4000,5c,c0ea2001,0,0,...) at sleepq_wait+0x63 _sleep(c5ea4000,c5ea4020,25c,c0ea2001,0,...) at _sleep+0x372 md_kthread(c5ea4000,edd2bd28,c0ef75b0,3a5,c63802c0,...) at md_kthread+0x11a fork_exit(c0732bf0,c5ea4000,edd2bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xedd2bd60, ebp = 0 --- Tracing command softdepflush pid 17 tid 100064 td 0xc5e6c000 sched_switch(c5e6c000,0,104,191,188b25c0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,54,...) at mi_switch+0x1eb sleepq_switch(c5e6c000,0,c0f02600,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c1215640,54,c0f32c1d,0,0,...) at sleepq_timedwait+0x6b _sleep(c1215640,c12155f0,54,c0f32c1d,3e8,...) at _sleep+0x342 softdep_flush(0,df76bd28,c0ef75b0,3a5,c5e61580,...) at softdep_flush+0x266 fork_exit(c0c42460,0,df76bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf76bd60, ebp = 0 --- Tracing command vnlru pid 16 tid 100063 td 0xc5bde2e0 sched_switch(c5bde2e0,0,104,191,884a53a0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,60,...) at mi_switch+0x1eb sleepq_switch(c5bde2e0,0,c0f02600,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c5e61840,60,c0f0d48f,0,0,...) at sleepq_timedwait+0x6b _sleep(c5e61840,c120ef34,260,c0f0d48f,3e8,...) at _sleep+0x342 vnlru_proc(0,df768d28,c0ef75b0,3a5,c5e61840,...) at vnlru_proc+0xe7 fork_exit(c0aa5ab0,0,df768d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf768d60, ebp = 0 --- Tracing command syncer pid 9 tid 100062 td 0xc5bde5c0 sched_switch(c5bde5c0,0,104,191,fbc002a0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bde5c0,0,c0f02600,28b,c5bde5c0,...) at sleepq_switch+0x162 sleepq_timedwait(c120ef74,0,df765c78,1,0,...) at sleepq_timedwait+0x6b _cv_timedwait(c120ef74,c120ef60,3e8,76a,4e20,...) at _cv_timedwait+0x252 sched_sync(0,df765d28,c0ef75b0,3a5,c5e61b00,...) at sched_sync+0x595 fork_exit(c0aa3a50,0,df765d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf765d60, ebp = 0 --- Tracing command bufdaemon pid 8 tid 100061 td 0xc5bde8a0 sched_switch(c5bde8a0,0,104,191,1a51d8a8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,54,...) at mi_switch+0x1eb sleepq_switch(c5bde8a0,0,c0f02600,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c120ec08,54,c0f0a467,0,0,...) at sleepq_timedwait+0x6b _sleep(c120ec08,c120ec10,54,c0f0a467,3e8,...) at _sleep+0x342 buf_daemon(0,df762d28,c0ef75b0,3a5,c5e62000,...) at buf_daemon+0x175 fork_exit(c0a8a880,0,df762d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf762d60, ebp = 0 --- Tracing command pagezero pid 7 tid 100060 td 0xc5bdeb80 sched_switch(c5bdeb80,0,104,191,7bc23d58,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bdeb80,0,c0f02600,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c12174dc,0,c0f39624,0,0,...) at sleepq_timedwait+0x6b _sleep(c12174dc,c1215f80,0,c0f39624,493e0,...) at _sleep+0x342 vm_pagezero(0,df75fd28,c0ef75b0,3a5,c59a62c0,...) at vm_pagezero+0xdc fork_exit(c0c86dc0,0,df75fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf75fd60, ebp = 0 --- Tracing command vmdaemon pid 6 tid 100059 td 0xc5c1c000 sched_switch(c5c1c000,0,104,191,741408a8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,74,...) at mi_switch+0x1eb sleepq_switch(c5c1c000,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c12170c8,74,c0f0a467,0,0,...) at sleepq_wait+0x63 _sleep(c12170c8,c12170cc,74,c0f0a467,0,...) at _sleep+0x372 vm_daemon(0,df75cd28,c0ef75b0,3a5,c59a6580,...) at vm_daemon+0x59 fork_exit(c0c80b90,0,df75cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf75cd60, ebp = 0 --- Tracing command pagedaemon pid 5 tid 100058 td 0xc5c1c2e0 sched_switch(c5c1c2e0,0,104,191,8ceccb68,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,54,...) at mi_switch+0x1eb sleepq_switch(c5c1c2e0,0,c0f02600,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c1217090,54,c0f0a467,0,0,...) at sleepq_timedwait+0x6b _sleep(c1217090,c1215f80,54,c0f0a467,1388,...) at _sleep+0x342 vm_pageout(0,df759d28,c0ef75b0,3a5,c59a6840,...) at vm_pageout+0x2c7 fork_exit(c0c81c70,0,df759d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf759d60, ebp = 0 --- Tracing command xpt_thrd pid 4 tid 100057 td 0xc5c1c5c0 sched_switch(c5c1c5c0,0,104,191,1ee95468,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c5c1c5c0,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c105dad4,5c,c0d666fd,0,0,...) at sleepq_wait+0x63 _sleep(c105dad4,c105db18,5c,c0d666fd,0,...) at _sleep+0x372 xpt_scanner_thread(0,df756d28,c0ef75b0,3a5,c59a6b00,...) at xpt_scanner_thread+0x47 fork_exit(c0491220,0,df756d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf756d60, ebp = 0 --- Tracing command sctp_iterator pid 3 tid 100056 td 0xc5c1c8a0 sched_switch(c5c1c8a0,0,104,191,e1246160,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5c1c8a0,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c1210a98,0,c0f2091f,0,0,...) at sleepq_wait+0x63 _sleep(c1210a98,c1210a60,0,c0f2091f,0,...) at _sleep+0x372 sctp_iterator_thread(0,df753d28,c0ef75b0,3a5,c5b98000,...) at sctp_iterator_thread+0x5c fork_exit(c0b36160,0,df753d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf753d60, ebp = 0 --- Tracing command fdc0 pid 2 tid 100053 td 0xc5c1e2e0 sched_switch(c5c1e2e0,0,104,191,4f3643b0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c5c1e2e0,0,c0f02600,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c5b3de3c,5c,c0eefada,0,0,...) at sleepq_timedwait+0x6b _sleep(c5b3de3c,c5b3def0,5c,c0eefada,3e8,...) at _sleep+0x342 fdc_thread(c5b3de00,df74ad28,c0ef75b0,3a5,c5b982c0,...) at fdc_thread+0x27d fork_exit(c0d06070,c5b3de00,df74ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf74ad60, ebp = 0 --- Tracing command usb pid 15 tid 100049 td 0xc5bdb5c0 sched_switch(c5bdb5c0,0,104,191,fe1cecc8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bdb5c0,0,c0f02600,268,c5bdb5c0,...) at sleepq_switch+0x162 sleepq_wait(c5be2d34,0,c5714cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5be2d34,c5be2dd4,c0ed699e,6b,c5be2d3c,...) at _cv_wait+0x243 usb_process(c5be2d2c,c5714d28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5be2d2c,c5714d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5714d60, ebp = 0 --- Tracing command usb pid 15 tid 100048 td 0xc5bdb8a0 sched_switch(c5bdb8a0,0,104,191,9a76f670,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bdb8a0,0,c0f02600,268,c5bdb8a0,...) at sleepq_switch+0x162 sleepq_wait(c5be2d04,0,c5711cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5be2d04,c5be2dd4,c0ed699e,6b,c5be2d0c,...) at _cv_wait+0x243 usb_process(c5be2cfc,c5711d28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5be2cfc,c5711d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5711d60, ebp = 0 --- Tracing command usb pid 15 tid 100047 td 0xc5bdbb80 sched_switch(c5bdbb80,0,104,191,fd9f0468,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bdbb80,0,c0f02600,268,c5bdbb80,...) at sleepq_switch+0x162 sleepq_wait(c5be2cd4,0,c570ecac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5be2cd4,c5be2dd4,c0ed699e,6b,c5be2cdc,...) at _cv_wait+0x243 usb_process(c5be2ccc,c570ed28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5be2ccc,c570ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc570ed60, ebp = 0 --- Tracing command usb pid 15 tid 100046 td 0xc5bdd000 sched_switch(c5bdd000,0,104,191,fd9ef118,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bdd000,0,c0f02600,268,c5bdd000,...) at sleepq_switch+0x162 sleepq_wait(c5be2ca4,0,c570bcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5be2ca4,c5be2dd4,c0ed699e,6b,c5be2cac,...) at _cv_wait+0x243 usb_process(c5be2c9c,c570bd28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5be2c9c,c570bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc570bd60, ebp = 0 --- Tracing command usb pid 15 tid 100045 td 0xc5bdd2e0 sched_switch(c5bdd2e0,0,104,191,fd9ed9c8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bdd2e0,0,c0f02600,268,c5bdd2e0,...) at sleepq_switch+0x162 sleepq_wait(c5bccdac,0,c5705cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5bccdac,c5bcce4c,c0ed699e,6b,c5bccdb4,...) at _cv_wait+0x243 usb_process(c5bccda4,c5705d28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5bccda4,c5705d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5705d60, ebp = 0 --- Tracing command usb pid 15 tid 100044 td 0xc5bdd5c0 sched_switch(c5bdd5c0,0,104,191,f9f992f8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bdd5c0,0,c0f02600,268,c5bdd5c0,...) at sleepq_switch+0x162 sleepq_wait(c5bccd7c,0,c5702cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5bccd7c,c5bcce4c,c0ed699e,6b,c5bccd84,...) at _cv_wait+0x243 usb_process(c5bccd74,c5702d28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5bccd74,c5702d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5702d60, ebp = 0 --- Tracing command usb pid 15 tid 100043 td 0xc5bdd8a0 sched_switch(c5bdd8a0,0,104,191,f68b5eb8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bdd8a0,0,c0f02600,268,c5bdd8a0,...) at sleepq_switch+0x162 sleepq_wait(c5bccd4c,0,c56ffcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5bccd4c,c5bcce4c,c0ed699e,6b,c5bccd54,...) at _cv_wait+0x243 usb_process(c5bccd44,c56ffd28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5bccd44,c56ffd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56ffd60, ebp = 0 --- Tracing command usb pid 15 tid 100042 td 0xc5bddb80 sched_switch(c5bddb80,0,104,191,f68b4bc0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bddb80,0,c0f02600,268,c5bddb80,...) at sleepq_switch+0x162 sleepq_wait(c5bccd1c,0,c56fccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5bccd1c,c5bcce4c,c0ed699e,6b,c5bccd24,...) at _cv_wait+0x243 usb_process(c5bccd14,c56fcd28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5bccd14,c56fcd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56fcd60, ebp = 0 --- Tracing command usb pid 15 tid 100040 td 0xc5bb1000 sched_switch(c5bb1000,0,104,191,f68b3340,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bb1000,0,c0f02600,268,c5bb1000,...) at sleepq_switch+0x162 sleepq_wait(c5bb9dac,0,c56f5cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5bb9dac,c5bb9e4c,c0ed699e,6b,c5bb9db4,...) at _cv_wait+0x243 usb_process(c5bb9da4,c56f5d28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5bb9da4,c56f5d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56f5d60, ebp = 0 --- Tracing command usb pid 15 tid 100039 td 0xc5bb12e0 sched_switch(c5bb12e0,0,104,191,f95737b0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bb12e0,0,c0f02600,268,c5bb12e0,...) at sleepq_switch+0x162 sleepq_wait(c5bb9d7c,0,c56f2cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5bb9d7c,c5bb9e4c,c0ed699e,6b,c5bb9d84,...) at _cv_wait+0x243 usb_process(c5bb9d74,c56f2d28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5bb9d74,c56f2d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56f2d60, ebp = 0 --- Tracing command usb pid 15 tid 100038 td 0xc5bb15c0 sched_switch(c5bb15c0,0,104,191,f610ed90,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bb15c0,0,c0f02600,268,c5bb15c0,...) at sleepq_switch+0x162 sleepq_wait(c5bb9d4c,0,c56efcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5bb9d4c,c5bb9e4c,c0ed699e,6b,c5bb9d54,...) at _cv_wait+0x243 usb_process(c5bb9d44,c56efd28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5bb9d44,c56efd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56efd60, ebp = 0 --- Tracing command usb pid 15 tid 100037 td 0xc5bb18a0 sched_switch(c5bb18a0,0,104,191,f610d8e8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bb18a0,0,c0f02600,268,c5bb18a0,...) at sleepq_switch+0x162 sleepq_wait(c5bb9d1c,0,c56eccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5bb9d1c,c5bb9e4c,c0ed699e,6b,c5bb9d24,...) at _cv_wait+0x243 usb_process(c5bb9d14,c56ecd28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5bb9d14,c56ecd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56ecd60, ebp = 0 --- Tracing command usb pid 15 tid 100035 td 0xc5bb3000 sched_switch(c5bb3000,0,104,191,f610c140,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bb3000,0,c0f02600,268,c5bb3000,...) at sleepq_switch+0x162 sleepq_wait(c5ba2dac,0,c56e5cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5ba2dac,c5ba2e4c,c0ed699e,6b,c5ba2db4,...) at _cv_wait+0x243 usb_process(c5ba2da4,c56e5d28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5ba2da4,c56e5d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56e5d60, ebp = 0 --- Tracing command usb pid 15 tid 100034 td 0xc5bb32e0 sched_switch(c5bb32e0,0,104,191,f8129860,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bb32e0,0,c0f02600,268,c5bb32e0,...) at sleepq_switch+0x162 sleepq_wait(c5ba2d7c,0,c56e2cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5ba2d7c,c5ba2e4c,c0ed699e,6b,c5ba2d84,...) at _cv_wait+0x243 usb_process(c5ba2d74,c56e2d28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5ba2d74,c56e2d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56e2d60, ebp = 0 --- Tracing command usb pid 15 tid 100033 td 0xc5bb35c0 sched_switch(c5bb35c0,0,104,191,f59694c0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bb35c0,0,c0f02600,268,c5bb35c0,...) at sleepq_switch+0x162 sleepq_wait(c5ba2d4c,0,c56dfcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5ba2d4c,c5ba2e4c,c0ed699e,6b,c5ba2d54,...) at _cv_wait+0x243 usb_process(c5ba2d44,c56dfd28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5ba2d44,c56dfd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56dfd60, ebp = 0 --- Tracing command usb pid 15 tid 100032 td 0xc5bb38a0 sched_switch(c5bb38a0,0,104,191,f5968208,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bb38a0,0,c0f02600,268,c5bb38a0,...) at sleepq_switch+0x162 sleepq_wait(c5ba2d1c,0,c56dccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5ba2d1c,c5ba2e4c,c0ed699e,6b,c5ba2d24,...) at _cv_wait+0x243 usb_process(c5ba2d14,c56dcd28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5ba2d14,c56dcd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56dcd60, ebp = 0 --- Tracing command usb pid 15 tid 100030 td 0xc5b1d2e0 sched_switch(c5b1d2e0,0,104,191,f59646c0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b1d2e0,0,c0f02600,268,c5b1d2e0,...) at sleepq_switch+0x162 sleepq_wait(c5b88dac,0,c56d5cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5b88dac,c5b88e4c,c0ed699e,6b,c5b88db4,...) at _cv_wait+0x243 usb_process(c5b88da4,c56d5d28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5b88da4,c56d5d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56d5d60, ebp = 0 --- Tracing command usb pid 15 tid 100029 td 0xc5b1d5c0 sched_switch(c5b1d5c0,0,104,191,f8b4e8a8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b1d5c0,0,c0f02600,268,c5b1d5c0,...) at sleepq_switch+0x162 sleepq_wait(c5b88d7c,0,c56d2cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5b88d7c,c5b88e4c,c0ed699e,6b,c5b88d84,...) at _cv_wait+0x243 usb_process(c5b88d74,c56d2d28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5b88d74,c56d2d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56d2d60, ebp = 0 --- Tracing command usb pid 15 tid 100028 td 0xc5b1d8a0 sched_switch(c5b1d8a0,0,104,191,f51b5780,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b1d8a0,0,c0f02600,268,c5b1d8a0,...) at sleepq_switch+0x162 sleepq_wait(c5b88d4c,0,c56cfcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5b88d4c,c5b88e4c,c0ed699e,6b,c5b88d54,...) at _cv_wait+0x243 usb_process(c5b88d44,c56cfd28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5b88d44,c56cfd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56cfd60, ebp = 0 --- Tracing command usb pid 15 tid 100027 td 0xc5b1db80 sched_switch(c5b1db80,0,104,191,f51b4420,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b1db80,0,c0f02600,268,c5b1db80,...) at sleepq_switch+0x162 sleepq_wait(c5b88d1c,0,c56cccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5b88d1c,c5b88e4c,c0ed699e,6b,c5b88d24,...) at _cv_wait+0x243 usb_process(c5b88d14,c56ccd28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5b88d14,c56ccd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56ccd60, ebp = 0 --- Tracing command yarrow pid 14 tid 100013 td 0xc59a92e0 sched_switch(c59a92e0,0,104,191,58e5a8d8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c59a92e0,0,c0f02600,28b,2,...) at sleepq_switch+0x162 sleepq_timedwait(c1097b84,0,c0eefada,2,0,...) at sleepq_timedwait+0x6b _sleep(c1097b84,0,0,c0eefada,64,...) at _sleep+0x342 pause(c0eefada,64,c0eb632e,111,0,...) at pause+0x47 random_kthread(0,c5636d28,c0ef75b0,3a5,c59a5000,...) at random_kthread+0x1ef fork_exit(c07c6b20,0,c5636d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5636d60, ebp = 0 --- Tracing command geom pid 13 tid 100011 td 0xc59a98a0 sched_switch(c59a98a0,0,104,191,5c24a030,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c59a98a0,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c10957a8,5c,c0eefada,0,0,...) at sleepq_wait+0x63 _sleep(c10957a8,c1095708,25c,c0eefada,0,...) at _sleep+0x372 g_io_schedule_down(c59a98a0,0,c0ef1679,6c,c562cd14,...) at g_io_schedule_down+0x5b g_down_procbody(0,c562cd28,c0ef75b0,3a5,c59a52c0,...) at g_down_procbody+0x9d fork_exit(c099d200,0,c562cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc562cd60, ebp = 0 --- Tracing command geom pid 13 tid 100010 td 0xc59a9b80 sched_switch(c59a9b80,0,104,191,5c251cf0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c59a9b80,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c10957a4,5c,c0eefada,0,0,...) at sleepq_wait+0x63 _sleep(c10957a4,c1095728,25c,c0eefada,0,...) at _sleep+0x372 g_io_schedule_up(c59a9b80,0,c0ef1679,5f,c5629d14,...) at g_io_schedule_up+0x134 g_up_procbody(0,c5629d28,c0ef75b0,3a5,c59a52c0,...) at g_up_procbody+0x9d fork_exit(c099d2a0,0,c5629d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5629d60, ebp = 0 --- Tracing command geom pid 13 tid 100009 td 0xc59ef000 sched_switch(c59ef000,0,104,191,26894cf8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c59ef000,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c109579c,5c,c0eefada,0,0,...) at sleepq_wait+0x63 _sleep(c109579c,c10956cc,25c,c0eefada,0,...) at _sleep+0x372 g_run_events(c10a5a00,0,c0ef1679,79,c5626d14,...) at g_run_events+0x526 g_event_procbody(0,c5626d28,c0ef75b0,3a5,c59a52c0,...) at g_event_procbody+0x90 fork_exit(c099d340,0,c5626d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5626d60, ebp = 0 --- Tracing command intr pid 12 tid 100054 td 0xc5c1e000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100052 td 0xc5c1e5c0 kdb_enter(c0d964dd,c0e891f5,82,2f5fa1d6,82) at kdb_enter+0x3a watchdog_fire(1,0,c0ef5117,216,c5c1e5c0,...) at watchdog_fire+0xb4 hardclock_anycpu(1,0,df73dba8,c0a13ffd,0,...) at hardclock_anycpu+0x26f handleevents(df73dbc8,4,0,c5b48200,c5b36080,100,12a7,e808f957,1d0eff8f,7c8280) at handleevents+0x108 timercb(c1277080,0,0,c1097470,0,...) at timercb+0x2ad lapic_handle_timer(df73dc0c) at lapic_handle_timer+0x92 Xtimerint() at Xtimerint+0x20 --- interrupt, eip = 0xc09f8077, esp = 0xdf73dc4c, ebp = 0xdf73dc68 --- _mtx_lock_sleep(c1097470,c5c1e5c0,0,c0ef78eb,4e8,...) at _mtx_lock_sleep+0x97 _mtx_lock_flags(c1097470,0,c0ef78eb,4e8,c59ede38,...) at _mtx_lock_flags+0xf7 intr_event_execute_handlers(c59a5580,c59ede00,c0ef78eb,55e,c5c1e5c0,...) at intr_event_execute_handlers+0x11a ithread_loop(c59ee710,df73dd28,c0ef75b0,3a5,c59a5580,...) at ithread_loop+0xa9 fork_exit(c09de980,c59ee710,df73dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf73dd60, ebp = 0 --- Tracing command intr pid 12 tid 100051 td 0xc5bdb000 sched_switch(c5bdb000,0,109,191,5c120608,...) at sched_switch+0x3bc mi_switch(109,0,c0ef78eb,554,c5bdb000,...) at mi_switch+0x1eb ithread_loop(c59ee720,c573ed28,c0ef75b0,3a5,c59a5580,...) at ithread_loop+0x1fb fork_exit(c09de980,c59ee720,c573ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc573ed60, ebp = 0 --- Tracing command intr pid 12 tid 100050 td 0xc5bdb2e0 sched_switch(c5bdb2e0,0,109,191,827423e8,...) at sched_switch+0x3bc mi_switch(109,0,c0ef78eb,554,c5bdb2e0,...) at mi_switch+0x1eb ithread_loop(c59ee730,c573ad28,c0ef75b0,3a5,c59a5580,...) at ithread_loop+0x1fb fork_exit(c09de980,c59ee730,c573ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc573ad60, ebp = 0 --- Tracing command intr pid 12 tid 100041 td 0xc5bde000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100036 td 0xc5bb1b80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100031 td 0xc5bb3b80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100026 td 0xc5b20000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100023 td 0xc5b208a0 sched_switch(c5b208a0,0,109,191,5b15be98,...) at sched_switch+0x3bc mi_switch(109,0,c0ef78eb,554,c5b208a0,...) at mi_switch+0x1eb ithread_loop(c59ee780,c5654d28,c0ef75b0,3a5,c59a5580,...) at ithread_loop+0x1fb fork_exit(c09de980,c59ee780,c5654d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5654d60, ebp = 0 --- Tracing command intr pid 12 tid 100022 td 0xc5b20b80 sched_switch(c5b20b80,0,109,191,5c123498,...) at sched_switch+0x3bc mi_switch(109,0,c0ef78eb,554,c5b20b80,...) at mi_switch+0x1eb ithread_loop(c59ee790,c5651d28,c0ef75b0,3a5,c59a5580,...) at ithread_loop+0x1fb fork_exit(c09de980,c59ee790,c5651d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5651d60, ebp = 0 --- Tracing command intr pid 12 tid 100017 td 0xc5b1c5c0 sched_switch(c5b1c5c0,0,109,191,5b7e3078,...) at sched_switch+0x3bc mi_switch(109,0,c0ef78eb,554,c5b1c5c0,...) at mi_switch+0x1eb ithread_loop(c59ee7a0,c5642d28,c0ef75b0,3a5,c59a5580,...) at ithread_loop+0x1fb fork_exit(c09de980,c59ee7a0,c5642d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5642d60, ebp = 0 --- Tracing command intr pid 12 tid 100015 td 0xc5b1cb80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100008 td 0xc59ef2e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100007 td 0xc59ef5c0 sched_switch(c59ef5c0,0,109,191,b9ca2a18,...) at sched_switch+0x3bc mi_switch(109,0,c0ef78eb,554,c59ef5c0,...) at mi_switch+0x1eb ithread_loop(c59ee7d0,c5620d28,c0ef75b0,3a5,c59a5580,...) at ithread_loop+0x1fb fork_exit(c09de980,c59ee7d0,c5620d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5620d60, ebp = 0 --- Tracing command intr pid 12 tid 100006 td 0xc59a7000 sched_switch(c59a7000,0,103,18c,884a2db0,...) at sched_switch+0x3bc mi_switch(103,0,c0f0305c,2e1,ae0,...) at mi_switch+0x1eb turnstile_wait(c5961e80,c65aa2e0,0,1ac,c1097470,...) at turnstile_wait+0x4a4 _mtx_lock_sleep(c1097470,c59a7000,0,c0efb2e7,93,...) at _mtx_lock_sleep+0x14d _mtx_lock_flags(c1097470,0,c0efb2e7,93,c561dc90,...) at _mtx_lock_flags+0xf7 lock_mtx(c1097470,1,c0efe486,20f,4,...) at lock_mtx+0x29 softclock(c1098dc0,c561dcb8,c09f7a94,c10a5a00,c59ecc38,...) at softclock+0x1ea intr_event_execute_handlers(c59a5580,c59ecc00,c0ef78eb,55e,c59a7000,...) at intr_event_execute_handlers+0x125 ithread_loop(c59ee7e0,c561dd28,c0ef75b0,3a5,c59a5580,...) at ithread_loop+0xa9 fork_exit(c09de980,c59ee7e0,c561dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc561dd60, ebp = 0 --- Tracing command intr pid 12 tid 100005 td 0xc59a72e0 sched_switch(c59a72e0,0,109,191,94a78818,...) at sched_switch+0x3bc mi_switch(109,0,c0ef78eb,554,c59a72e0,...) at mi_switch+0x1eb ithread_loop(c59ee7f0,c561ad28,c0ef75b0,3a5,c59a5580,...) at ithread_loop+0x1fb fork_exit(c09de980,c59ee7f0,c561ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc561ad60, ebp = 0 --- Tracing command idle pid 11 tid 100004 td 0xc59a75c0 sched_switch(c59a75c0,0,108,18c,58eb5360,...) at sched_switch+0x3bc mi_switch(108,0,c0efede3,3b9,c59a75c0,...) at mi_switch+0x1eb sched_idletd(0,c5614d28,c0ef75b0,3a5,c59a5840,...) at sched_idletd+0x19f fork_exit(c0a2dd60,0,c5614d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5614d60, ebp = 0 --- Tracing command idle pid 11 tid 100003 td 0xc59a78a0 sched_switch(c59a78a0,0,608,18c,6b702fd0,...) at sched_switch+0x3bc mi_switch(608,0,c0efd59f,cd,ffffffff,...) at mi_switch+0x1eb critical_exit(0,ffffffff,c10a5a00,1,c5611ce8,...) at critical_exit+0xa8 cpu_idle(0,c5611cc4,c0efede3,3bb,c59a78a0,...) at cpu_idle+0x97 sched_idletd(0,c5611d28,c0ef75b0,3a5,c59a5840,...) at sched_idletd+0x263 fork_exit(c0a2dd60,0,c5611d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5611d60, ebp = 0 --- Tracing command init pid 1 tid 100002 td 0xc59a7b80 sched_switch(c59a7b80,0,104,191,b9cbe4b8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c59a7b80,0,c0f02600,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c59a5b00,6c,c0f0545a,100,0,...) at sleepq_wait_sig+0x17 _sleep(c59a5b00,c59a5b88,16c,c0f0545a,0,...) at _sleep+0x35c kern_wait(c59a7b80,ffffffff,c560dc44,0,0,...) at kern_wait+0xb96 wait4(c59a7b80,c560dcec,c560dd28,c0f02f8a,0,...) at wait4+0x3b syscallenter(c59a7b80,c560dce4,c560dce4,0,0,...) at syscallenter+0x263 syscall(c560dd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x80551d3, esp = 0xbfbfe8bc, ebp = 0xbfbfe8d8 --- Tracing command audit pid 10 tid 100001 td 0xc59a9000 sched_switch(c59a9000,0,104,191,f517c6b0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c59a9000,0,c0f02600,268,c59a9000,...) at sleepq_switch+0x162 sleepq_wait(c1214d60,0,c560ac8c,1,0,...) at sleepq_wait+0x63 _cv_wait(c1214d60,c1214d44,c0f2d4ef,194,0,...) at _cv_wait+0x243 audit_worker(0,c560ad28,c0ef75b0,3a5,c59a6000,...) at audit_worker+0x84 fork_exit(c0c02210,0,c560ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc560ad60, ebp = 0 --- Tracing command kernel pid 0 tid 100065 td 0xc5e6bb80 sched_switch(c5e6bb80,0,104,191,32fc6378,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5e6bb80,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c5c02180,0,c0eefada,0,0,...) at sleepq_wait+0x63 _sleep(c5c02180,c5c02198,0,c0eefada,0,...) at _sleep+0x372 taskqueue_thread_loop(c1277120,df76ed28,c0ef75b0,3a5,c1095880,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a4a9b0,c1277120,df76ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf76ed60, ebp = 0 --- Tracing command kernel pid 0 tid 100055 td 0xc5c1cb80 sched_switch(c5c1cb80,0,104,191,db484c98,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5c1cb80,0,c0f02600,28b,2,...) at sleepq_switch+0x162 sleepq_timedwait(c1097b84,0,c0eefada,2,0,...) at sleepq_timedwait+0x6b _sleep(c1097b84,0,0,c0eefada,bb8,...) at _sleep+0x342 pause(c0eefada,bb8,124,122,c1095908,...) at pause+0x47 deadlkres(0,df750d28,c0ef75b0,3a5,c1095880,...) at deadlkres+0x324 fork_exit(c09bed70,0,df750d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf750d60, ebp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xc5b202e0 sched_switch(c5b202e0,0,104,191,f5151540,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b202e0,0,c0f02600,268,c1095b40,...) at sleepq_switch+0x162 sleepq_wait(c5b51740,0,c0efdad9,c0eefada,0,...) at sleepq_wait+0x63 msleep_spin(c5b51740,c5b51758,c0eefada,0,c0efb2e7,...) at msleep_spin+0x21d taskqueue_thread_loop(c5b74568,c56c5d28,c0ef75b0,3a5,c1095880,...) at taskqueue_thread_loop+0x8f fork_exit(c0a4a9b0,c5b74568,c56c5d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56c5d60, ebp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xc5b205c0 sched_switch(c5b205c0,0,104,191,d716f480,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b205c0,0,c0f02600,268,c65aa2e0,...) at sleepq_switch+0x162 sleepq_wait(c5b51980,0,c0efdad9,c0eefada,0,...) at sleepq_wait+0x63 msleep_spin(c5b51980,c5b51998,c0eefada,0,c0efb2e7,...) at msleep_spin+0x21d taskqueue_thread_loop(c5b5f568,c569ad28,c0ef75b0,3a5,c1095880,...) at taskqueue_thread_loop+0x8f fork_exit(c0a4a9b0,c5b5f568,c569ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc569ad60, ebp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xc59ef8a0 sched_switch(c59ef8a0,0,104,191,1eeb64d0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c59ef8a0,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c59a8100,0,c0efdad9,c0eefada,0,...) at sleepq_wait+0x63 msleep_spin(c59a8100,c59a8118,c0eefada,0,c0efb2e7,...) at msleep_spin+0x21d taskqueue_thread_loop(c10609b8,c564ed28,c0ef75b0,3a5,c1095880,...) at taskqueue_thread_loop+0x8f fork_exit(c0a4a9b0,c10609b8,c564ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc564ed60, ebp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xc59efb80 sched_switch(c59efb80,0,104,191,1eeb5750,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c59efb80,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c59a8100,0,c0efdad9,c0eefada,0,...) at sleepq_wait+0x63 msleep_spin(c59a8100,c59a8118,c0eefada,0,c0efb2e7,...) at msleep_spin+0x21d taskqueue_thread_loop(c10609b8,c564bd28,c0ef75b0,3a5,c1095880,...) at taskqueue_thread_loop+0x8f fork_exit(c0a4a9b0,c10609b8,c564bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc564bd60, ebp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xc5b1c000 sched_switch(c5b1c000,0,104,191,1eeb44a8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b1c000,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c59a8100,0,c0efdad9,c0eefada,0,...) at sleepq_wait+0x63 msleep_spin(c59a8100,c59a8118,c0eefada,0,c0efb2e7,...) at msleep_spin+0x21d taskqueue_thread_loop(c10609b8,c5648d28,c0ef75b0,3a5,c1095880,...) at taskqueue_thread_loop+0x8f fork_exit(c0a4a9b0,c10609b8,c5648d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5648d60, ebp = 0 --- Tracing command kernel pid 0 tid 100018 td 0xc5b1c2e0 sched_switch(c5b1c2e0,0,104,191,1ee9dd10,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b1c2e0,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c59a8140,0,c0eefada,0,0,...) at sleepq_wait+0x63 _sleep(c59a8140,c59a8158,0,c0eefada,0,...) at _sleep+0x372 taskqueue_thread_loop(c1096218,c5645d28,c0ef75b0,3a5,c1095880,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a4a9b0,c1096218,c5645d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5645d60, ebp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xc5b1c8a0 sched_switch(c5b1c8a0,0,104,191,1ee9cd10,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b1c8a0,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c59a81c0,0,c0eefada,0,0,...) at sleepq_wait+0x63 _sleep(c59a81c0,c59a81d8,0,c0eefada,0,...) at _sleep+0x372 taskqueue_thread_loop(c1215344,c563fd28,c0ef75b0,3a5,c1095880,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a4a9b0,c1215344,c563fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc563fd60, ebp = 0 --- Tracing command kernel pid 0 tid 100014 td 0xc5b1d000 sched_switch(c5b1d000,0,104,191,db5d6ee8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b1d000,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c59a84c0,0,c0eefada,0,0,...) at sleepq_wait+0x63 _sleep(c59a84c0,c59a84d8,0,c0eefada,0,...) at _sleep+0x372 taskqueue_thread_loop(c10b4788,c5639d28,c0ef75b0,3a5,c1095880,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a4a9b0,c10b4788,c5639d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5639d60, ebp = 0 --- Tracing command kernel pid 0 tid 100012 td 0xc59a95c0 sched_switch(c59a95c0,0,104,191,629897d8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c59a95c0,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c59a8500,0,c0eefada,0,0,...) at sleepq_wait+0x63 _sleep(c59a8500,c59a8518,0,c0eefada,0,...) at _sleep+0x372 taskqueue_thread_loop(c10b31a0,c562fd28,c0ef75b0,3a5,c1095880,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a4a9b0,c10b31a0,c562fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc562fd60, ebp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xc1095b40 sched_switch(c1095b40,0,104,191,1a5ab508,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,54,...) at mi_switch+0x1eb sleepq_switch(c1095b40,0,c0f02600,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c1095880,54,c0eff6fd,0,0,...) at sleepq_timedwait+0x6b _sleep(c1095880,0,54,c0eff6fd,2710,...) at _sleep+0x342 scheduler(0,181ec00,181ec00,181e000,1825000,...) at scheduler+0x240 mi_startup() at mi_startup+0xac begin() at begin+0x2c db:0:allt> call doadump Physical memory: 2026 MB Physical memory: 2026 MB Dumping 210 MB:Dumping 210 MB: 194 194 178 178 162 162 146 146 130 130 114 114 98 98 82 82 66 66 50 50 34 34 18 18 2 2 Dump complete Dump complete = 0 db:0:doadump> reset (kgdb) proc 1877 [Switching to thread 97 (Thread 100132)]#0 cpustop_handler () at /mnt/pho/src/sys/i386/i386/mp_machdep.c:1496 1496 CPU_SET_ATOMIC(cpu, &stopped_cpus); (kgdb) bt #0 cpustop_handler () at /mnt/pho/src/sys/i386/i386/mp_machdep.c:1496 #1 0xc0d34097 in ipi_nmi_handler () at /mnt/pho/src/sys/i386/i386/mp_machdep.c:1478 #2 0xc0d3e776 in trap (frame=0xedda1434) at /mnt/pho/src/sys/i386/i386/trap.c:219 #3 0xc0d27c8c in calltrap () at /mnt/pho/src/sys/i386/i386/exception.s:168 #4 0xc0c3bfa5 in jwait (wk=0xc5eaaac0, waitfor=1) at /mnt/pho/src/sys/ufs/ffs/ffs_softdep.c:4292 #5 0xc0c42be8 in softdep_sync_buf (vp=0xc6580984, bp=0xdf3bd8a0, waitfor=1) at /mnt/pho/src/sys/ufs/ffs/ffs_softdep.c:12029 #6 0xc0c4ba9c in ffs_syncvnode (vp=0xc6580984, waitfor=1) at /mnt/pho/src/sys/ufs/ffs/ffs_vnops.c:268 #7 0xc0c2be33 in ffs_snapshot (mp=0xc5eb1b50, snapfile=0xc5eb8c40 "/mnt/.snap/suj16") at /mnt/pho/src/sys/ufs/ffs/ffs_snapshot.c:847 #8 0xc0c48113 in ffs_mount (mp=0xc5eb1b50) at /mnt/pho/src/sys/ufs/ffs/ffs_vfsops.c:474 #9 0xc0a9a484 in vfs_donmount (td=0xc65aa2e0, fsflags=69632, fsoptions=0xc5e4c200) at /mnt/pho/src/sys/kern/vfs_mount.c:925 #10 0xc0a9ab54 in nmount (td=0xc65aa2e0, uap=0xedda1cec) at /mnt/pho/src/sys/kern/vfs_mount.c:410 #11 0xc0a4ad73 in syscallenter (td=0xc65aa2e0, sa=0xedda1ce4) at /mnt/pho/src/sys/kern/subr_trap.c:344 #12 0xc0d3e65f in syscall (frame=0xedda1d28) at /mnt/pho/src/sys/i386/i386/trap.c:1082 #13 0xc0d27cf1 in Xint0x80_syscall () at /mnt/pho/src/sys/i386/i386/exception.s:266 #14 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) f 4 #4 0xc0c3bfa5 in jwait (wk=0xc5eaaac0, waitfor=1) at /mnt/pho/src/sys/ufs/ffs/ffs_softdep.c:4292 4292 return (0); (kgdb) l 4287 return (0); 4288 } 4289 if (waitfor != MNT_WAIT) 4290 return (EBUSY); 4291 wait_worklist(wk, "jwait"); 4292 return (0); 4293 } 4294 4295 /* 4296 * Lookup an inodedep based on an inode pointer and set the nlinkdelta as (kgdb) p *wk $1 = {wk_list = {le_next = 0xc65a74c0, le_prev = 0xc5e3ee6c}, wk_mp = 0xc5eb1b50, wk_type = 19, wk_state = 32777} (kgdb) up #5 0xc0c42be8 in softdep_sync_buf (vp=0xc6580984, bp=0xdf3bd8a0, waitfor=1) at /mnt/pho/src/sys/ufs/ffs/ffs_softdep.c:12029 12029 jwait(&newblk->nb_jnewblk->jn_list, waitfor); (kgdb) l 12024 if (newblk->nb_jnewblk != NULL) { 12025 if (waitfor == MNT_NOWAIT) { 12026 error = EBUSY; 12027 goto out_unlock; 12028 } 12029 jwait(&newblk->nb_jnewblk->jn_list, waitfor); 12030 goto top; 12031 } 12032 if (newblk->nb_state & DEPCOMPLETE || 12033 waitfor == MNT_NOWAIT) (kgdb) -- Test scenario: suj16.sh