GENERIC HEAD r231181M KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2012 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #0: Wed Feb 8 07:38:30 CET 2012 test@leopard3.netperf.freebsd.org:/mnt/pho/obj/mnt/pho/src/sys/PHO i386 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. CPU: Intel(R) Xeon(R) CPU 3050 @ 2.13GHz (2128.05-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x6f2 Family = 6 Model = f Stepping = 2 Features=0xbfebfbff Features2=0xe3bd AMD Features=0x20100000 AMD Features2=0x1 TSC: P-state invariant, performance statistics real memory = 2147483648 (2048 MB) avail memory = 2080346112 (1983 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs FreeBSD/SMP: 1 package(s) x 2 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 random device not loaded; using insecure entropy ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard ioapic2 irqs 48-71 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: Power Button (fixed) cpu0: on acpi0 cpu1: on acpi0 atrtc0: port 0x70-0x71 irq 8 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: irq 16 at device 1.0 on pci0 pci1: on pcib1 pcib2: at device 0.0 on pci1 pci2: on pcib2 pcib3: at device 0.2 on pci1 pci3: on pcib3 pcib4: irq 16 at device 3.0 on pci0 pci10: on pcib4 pci10: at device 0.0 (no driver attached) pcib5: irq 17 at device 28.0 on pci0 pcib5: failed to allocate initial I/O port window: 0-0xfff pcib5: failed to allocate initial memory window: 0-0xfffff pcib5: failed to allocate initial prefetch window: 0-0xfffff pci14: on pcib5 pcib6: irq 17 at device 28.4 on pci0 pci18: on pcib6 em0: port 0x5000-0x501f mem 0xe0300000-0xe031ffff irq 16 at device 0.0 on pci18 em0: Using an MSI interrupt em0: Ethernet address: 00:30:48:8e:d0:68 pcib7: irq 16 at device 28.5 on pci0 pci19: on pcib7 em1: port 0x6000-0x601f mem 0xe0400000-0xe041ffff irq 17 at device 0.0 on pci19 em1: Using an MSI interrupt em1: Ethernet address: 00:30:48:8e:d0:69 uhci0: port 0x3000-0x301f irq 23 at device 29.0 on pci0 usbus0: on uhci0 uhci1: port 0x3020-0x303f irq 19 at device 29.1 on pci0 usbus1: on uhci1 uhci2: port 0x3040-0x305f irq 18 at device 29.2 on pci0 usbus2: on uhci2 uhci3: port 0x3060-0x307f irq 16 at device 29.3 on pci0 usbus3: on uhci3 ehci0: mem 0xe0000000-0xe00003ff irq 23 at device 29.7 on pci0 usbus4: EHCI version 1.0 usbus4: on ehci0 pcib8: at device 30.0 on pci0 pci20: on pcib8 vgapci0: port 0x7000-0x70ff mem 0xe8000000-0xefffffff,0xe0500000-0xe050ffff irq 16 at device 4.0 on pci20 isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x30a0-0x30af at device 31.1 on pci0 ata0: at channel 0 on atapci0 ahci0: port 0x30e8-0x30ef,0x30dc-0x30df,0x30e0-0x30e7,0x30d8-0x30db,0x30b0-0x30bf mem 0xe0000400-0xe00007ff irq 19 at device 31.2 on pci0 ahci0: AHCI v1.10 with 4 3Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 pci0: at device 31.3 (no driver attached) acpi_button0: on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: console (115200,n,8,1) uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fd0: <1440-KB 3.5" drive> on fdc0 drive 0 pmtimer0 on isa0 orm0: at iomem 0xc0000-0xcafff,0xcb000-0xcbfff pnpid ORM0000 on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x100> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] ppc0: parallel port not found. ctl: CAM Target Layer loaded est0: on cpu0 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 828082806000828 device_attach: est0 attach returned 6 p4tcc0: on cpu0 est1: on cpu1 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 828082806000828 device_attach: est1 attach returned 6 p4tcc1: on cpu1 Timecounters tick every 1.000 msec usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 12Mbps Full Speed USB v1.0 usbus3: 12Mbps Full Speed USB v1.0 ugen0.1: at usbus0 uhub0: on usbus0 Expensive timeout(9) function: 0xc0a9b270(0xc63acb80) 0.011088560 s ugen1.1: at usbus1 uhub1: on usbus1 ugen2.1: at usbus2 uhub2: on usbus2 ugen3.1: at usbus3 uhub3: on usbus3 usbus4: 480Mbps High Speed USB v2.0 ugen4.1: at usbus4 uhub4: on usbus4 Expensive timeout(9) function: 0xc0554a10(0xc6359d80) 0.020105627 s uhub0: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered uhub4: 8 ports with 8 removable, self powered ada0 at ahcich0 bus 0 scbus1 target 0 lun 0 ada0: ATA-7cd0 at ata0 bus 0 scbus0 target 0 lun 0 cd0: Removable CD-ROM SCSI-0 device cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present SATA 2.x device ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 238475MB (488397168 512 byte sectors: 16H 63S/T 16383C) ada0: Previously was known as ad4 SMP: AP CPU #1 Launched! Timecounter "TSC-low" frequency 16625362 Hz quality 1000 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. Trying to mount root from nfs: []... NFS ROOT: 192.168.5.1:/zoo/leopard3 em0: link state changed to UP Interface IP-Address Broadcast Setting hostuuid: 64a93476-27b1-3004-c299-0030488ed164. Setting hostid: 0x7acc54f1. Starting ddb. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: /dev/ada0d: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ada0d: clean, 109689879 free (1503 frags, 13711047 blocks, 0.0% fragmentation) mount_nfs: can't update /var/db/mounttab for 192.168.5.1:/zoo/leopard3 Mounting local file systems:. Setting hostname: leopard3.netperf.freebsd.org. Starting Network: lo0 em0 em1. lo0: flags=8049 metric 0 mtu 16384 options=3 nd6 options=21 em0: flags=8843 metric 0 mtu 1500 options=4219b ether 00:30:48:8e:d0:68 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active em1: flags=8c02 metric 0 mtu 1500 options=4219b ether 00:30:48:8e:d0:69 media: Ethernet autoselect status: no carrier Starting devd. Starting Network: em1. em1: flags=8c02 metric 0 mtu 1500 options=4219b ether 00:30:48:8e:d0:69 media: Ethernet autoselect status: no carrier Starting Network: usbus0. Starting Network: usbus1. Starting Network: usbus2. Starting Network: usbus3. Starting Network: usbus4. add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 Creating and/or trimming log files. Starting syslogd. No core dumps found. ELF ldconfig path: /lib /usr/lib /usr/lib/compat a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout Setting date via ntp. 8 Feb 07:15:43 ntpdate[1391]: step time server 192.168.5.1 offset -0.225374 sec Starting rpcbind. NFS access cache time=60 Clearing /tmp (X related). Starting mountd. Starting nfsd. Starting statd. Updating motd:. Configuring syscons: blanktime. Starting sshd. Starting cron. Local package initialization: watchdogd. Starting inetd. Wed Feb 8 07:15:44 UTC 2012 FreeBSD/i386 (leopard3.netperf.freebsd.org) (ttyu0) login: root Feb 8 07:16:08 leopard3 login: ROOT LOGIN (root) ON ttyu0 FreeBSD 10.0-CURRENT (PHO) #0: Wed Feb 8 07:38:30 CET 2012 Welcome to FreeBSD! Before seeking technical support, please use the following resources: o Security advisories and updated errata information for all releases are at http://www.FreeBSD.org/releases/ - always consult the ERRATA section for your release first as it's updated frequently. o The Handbook and FAQ documents are at http://www.FreeBSD.org/ and, along with the mailing lists, can be searched by going to http://www.FreeBSD.org/search/. If the doc distribution has been installed, they're also available formatted in /usr/share/doc. If you still have a question or problem, please take the output of `uname -a', along with any relevant error messages, and email it as a question to the questions@FreeBSD.org mailing list. If you are unfamiliar with FreeBSD's directory layout, please refer to the hier(7) manual page. If you are not familiar with manual pages, type `man man'. You may also use sysinstall(8) to re-enter the installation and configuration utility. Edit /etc/motd to change this login announcement. leopard3# ~test/keepalive.sh Wed Feb 8 08:16:13 CET 2012 8:16AM up 46 secs, 1 user, load averages: 1.60, 0.60, 0.23 Feb 8 08:16:31 leopard3 su: test to root on /dev/pts/0 lock order reversal: 1st 0xdfd18a00 bufwait (bufwait) @ /mnt/pho/src/sys/kern/vfs_bio.c:2658 2nd 0xc643fc00 dirhash (dirhash) @ /mnt/pho/src/sys/ufs/ufs/ufs_dirhash.c:284 KDB: stack backtrace: db_trace_self_wrapper(c0f9095b,7366752f,7366752f,7366752f,7269645f,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a94a3b,c0f94460,c12720d0,11c,ee8547dc,...) at kdb_backtrace+0x2a _witness_debugger(c0f94460,c643fc00,c0fc4a32,c6158a88,c0fc46b3,...) at _witness_debugger+0x25 witness_checkorder(c643fc00,9,c0fc46b3,11c,0,...) at witness_checkorder+0x86f _sx_xlock(c643fc00,0,c0fc46b3,11c,c90df414,...) at _sx_xlock+0xa0 ufsdirhash_acquire(dfd189a0,c90df414,ee854918,e0c70028,ee8548ac,...) at ufsdirhash_acquire+0x48 ufsdirhash_add(c90df414,ee854918,28,ee854898,ee85489c,...) at ufsdirhash_add+0x13 ufs_direnter(c90d5d98,c90d5414,ee854918,ee854bc0,0,...) at ufs_direnter+0x7a9 ufs_makeinode(ee854bc0,c10dc260,ee854b04,ee854a60,c0de2ff5,...) at ufs_makeinode+0x65d ufs_create(ee854b04,c0fe5fcb,0,0,ee854b80,...) at ufs_create+0x30 VOP_CREATE_APV(c10dbce0,ee854b04,ee854bc0,ee854a9c,0,...) at VOP_CREATE_APV+0xc5 vn_open_cred(ee854b80,ee854c48,0,0,c891f980,...) at vn_open_cred+0x205 vn_open(ee854b80,ee854c48,0,c8cc18f8,246,...) at vn_open+0x3b kern_openat(c90dc000,ffffff9c,2842ca15,0,603,...) at kern_openat+0x1ec kern_open(c90dc000,2842ca15,0,602,0,...) at kern_open+0x35 sys_open(c90dc000,ee854cec,c0fdec62,c0f7c2a5,246,...) at sys_open+0x30 syscall(ee854d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (5, FreeBSD ELF32, sys_open), eip = 0x28187b73, esp = 0xbfbfe18c, ebp = 0xbfbfe858 --- lock order reversal: 1st 0xc90d5df0 ufs (ufs) @ /mnt/pho/src/sys/kern/vfs_subr.c:2167 2nd 0xdfd18a00 bufwait (bufwait) @ /mnt/pho/src/sys/ufs/ffs/ffs_vnops.c:260 3rd 0xc90d546c ufs (ufs) @ /mnt/pho/src/sys/kern/vfs_subr.c:2167 KDB: stack backtrace: db_trace_self_wrapper(c0f9095b,75735f73,632e7262,3631323a,6f000a37,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a94a3b,c0f94479,c12720c0,877,ee85433c,...) at kdb_backtrace+0x2a _witness_debugger(c0f94479,c90d546c,c0f82d74,c61588e8,c0f9c534,...) at _witness_debugger+0x25 witness_checkorder(c90d546c,9,c0f9c534,877,0,...) at witness_checkorder+0x86f __lockmgr_args(c90d546c,80100,c90d54d8,0,0,...) at __lockmgr_args+0x834 ffs_lock(ee854460,0,c0f93685,80100,c90d5414,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c10dbce0,ee854460,c61511e0,c10ecd00,c90d5414,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c90d5414,80100,c0f9c534,877,4,...) at _vn_lock+0x78 vget(c90d5414,80100,c90dc000,50,0,...) at vget+0xbb vfs_hash_get(c90dd5a8,5,80000,c90dc000,ee8545a4,...) at vfs_hash_get+0xed ffs_vgetf(c90dd5a8,5,80000,ee8545a4,1,...) at ffs_vgetf+0x49 softdep_sync_buf(c90d5d98,dfd189a0,1,106,0,...) at softdep_sync_buf+0xac9 ffs_syncvnode(c90d5d98,1,bfbfe18c,bfbfe858,bfbfe858,...) at ffs_syncvnode+0x24c ffs_truncate(c90d5d98,400,0,880,c891f980,...) at ffs_truncate+0x827 ufs_direnter(c90d5d98,c90d5414,ee854918,ee854bc0,0,...) at ufs_direnter+0x9a1 ufs_makeinode(ee854bc0,c10dc260,ee854b04,ee854a60,c0de2ff5,...) at ufs_makeinode+0x65d ufs_create(ee854b04,c0fe5fcb,0,0,ee854b80,...) at ufs_create+0x30 VOP_CREATE_APV(c10dbce0,ee854b04,ee854bc0,ee854a9c,0,...) at VOP_CREATE_APV+0xc5 vn_open_cred(ee854b80,ee854c48,0,0,c891f980,...) at vn_open_cred+0x205 vn_open(ee854b80,ee854c48,0,c8cc18f8,246,...) at vn_open+0x3b kern_openat(c90dc000,ffffff9c,2842ca15,0,603,...) at kern_openat+0x1ec kern_open(c90dc000,2842ca15,0,602,0,...) at kern_open+0x35 sys_open(c90dc000,ee854cec,c0fdec62,c0f7c2a5,246,...) at sys_open+0x30 syscall(ee854d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (5, FreeBSD ELF32, sys_open), eip = 0x28187b73, esp = 0xbfbfe18c, ebp = 0xbfbfe858 --- lock order reversal: 1st 0xdfd21aa0 bufwait (bufwait) @ /mnt/pho/src/sys/kern/vfs_bio.c:2658 2nd 0xc61a061c snaplk (snaplk) @ /mnt/pho/src/sys/ufs/ffs/ffs_snapshot.c:1759 KDB: stack backtrace: db_trace_self_wrapper(c0f9095b,662f7366,662f7366,735f7366,7370616e,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a94a3b,c0f94460,c12714c8,6df,ee88f5f0,...) at kdb_backtrace+0x2a _witness_debugger(c0f94460,c61a061c,c0fc05c5,c6158c90,c0fc053e,...) at _witness_debugger+0x25 witness_checkorder(c61a061c,9,c0fc053e,6df,0,...) at witness_checkorder+0x86f __lockmgr_args(c61a061c,80900,c909f37c,0,0,...) at __lockmgr_args+0x834 ffs_snapblkfree(c90d2000,c909f2b8,17bd8,0,4000,...) at ffs_snapblkfree+0xea ffs_blkfree(c61a0400,c90d2000,c909f2b8,17bd8,0,...) at ffs_blkfree+0x59 handle_workitem_freefrag(c12a5ee0,0,c0fc0b20,1551,c,0,c86d3880,c90dd5a8,0,c629c200) at handle_workitem_freefrag+0xf0 softdep_setup_allocindir_page(c8e722b8,c,0,dfd21a40,0,...) at softdep_setup_allocindir_page+0x15c ffs_reallocblks(ee88fa20,c0fe54ab,0,c8fc42b8,c61746e4,...) at ffs_reallocblks+0x1451 VOP_REALLOCBLKS_APV(c10dbce0,ee88fa20,0,4000,0,...) at VOP_REALLOCBLKS_APV+0xd6 cluster_write(c8fc42b8,dfd21be0,34000,0,35,...) at cluster_write+0x3c2 ffs_write(ee88fbac,c0fe5f01,c8fc42b8,0,c8fc42b8,...) at ffs_write+0x52b VOP_WRITE_APV(c10dbce0,ee88fbac,c8fc42b8,280,0,...) at VOP_WRITE_APV+0x136 vn_write(c8cc0498,ee88fc44,c90cd780,0,c911d2e0,...) at vn_write+0x2bc dofilewrite(ee88fc44,ffffffff,ffffffff,0,ee88fc20,...) at dofilewrite+0x95 kern_writev(c911d2e0,4,ee88fc44,ee88fc64,1,...) at kern_writev+0x68 sys_write(c911d2e0,ee88fcec,c0fdec62,c0f5a9c7,206,...) at sys_write+0x4f syscall(ee88fd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28195da3, esp = 0xbfbfd84c, ebp = 0xbfbfe908 --- lock order reversal: 1st 0xc61a061c snaplk (snaplk) @ /mnt/pho/src/sys/ufs/ffs/ffs_snapshot.c:2680 2nd 0xc90d646c ufs (ufs) @ /mnt/pho/src/sys/ufs/ffs/ffs_snapshot.c:2539 KDB: stack backtrace: db_trace_self_wrapper(c0f9095b,7366662f,616e735f,6f687370,3a632e74,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a94a3b,c0f94460,c12710f0,9eb,ee8be3dc,...) at kdb_backtrace+0x2a _witness_debugger(c0f94460,c90d646c,c0f82d74,c61588e8,c0fc053e,...) at _witness_debugger+0x25 witness_checkorder(c90d646c,9,c0fc053e,9eb,0,...) at witness_checkorder+0x86f __lockmgr_args(c90d646c,80100,c90d64d8,0,0,...) at __lockmgr_args+0x834 ffs_lock(ee8be500,c0aa4bfa,c1271100,80100,c90d6414,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c10dbce0,ee8be500,ee8be520,c10ecd00,c90d6414,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c90d6414,80100,c0fc053e,9eb,c90dcb80,...) at _vn_lock+0x78 process_deferred_inactive(c90dd5a8,0,c0f9dd58,4c9,c9338c00,...) at process_deferred_inactive+0x130 vfs_write_resume(c90dd5a8,c0fc0a4f,c0fc053e,2a9,0,...) at vfs_write_resume+0xbd ffs_snapshot(c90dd5a8,c8918500,c0fc39f0,1a7,0,...) at ffs_snapshot+0x2cfc ffs_mount(c90dd5a8,c9130900,ff,3b4,c0ae655a,...) at ffs_mount+0x1ca7 vfs_donmount(c90dcb80,13100,0,c641aa80,c641aa80,...) at vfs_donmount+0x12eb sys_nmount(c90dcb80,ee8becec,c0fdec62,c0f95263,213,...) at sys_nmount+0x7f syscall(ee8bed28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (378, FreeBSD ELF32, sys_nmount), eip = 0x280da65b, esp = 0xbfbfe1cc, ebp = 0xbfbfe948 --- fsync: giving up on dirty 0xc909f2b8: tag devfs, type VCHR usecount 1, writecount 0, refcount 145 mountedhere 0xc88ce800 flags (VV_COPYONWRITE) v_object 0xc90af510 ref 0 pages 566 lock type devfs: EXCL by thread 0xc93598a0 (pid 1878) #0 0xc0a426ee at __lockmgr_args+0xc1e #1 0xc0ae6475 at vop_stdlock+0x65 #2 0xc0de2025 at VOP_LOCK1_APV+0xb5 #3 0xc0b08528 at _vn_lock+0x78 #4 0xc0c993a7 at softdep_flushworklist+0x47 #5 0xc0c9e36d at ffs_sync+0x32d #6 0xc0b08122 at vfs_write_suspend+0x122 #7 0xc0c8250d at ffs_snapshot+0xced #8 0xc0ca08a7 at ffs_mount+0x1ca7 #9 0xc0af062b at vfs_donmount+0x12eb #10 0xc0af0ccf at sys_nmount+0x7f #11 0xc0dbecc7 at syscall+0x2c7 #12 0xc0da7ff1 at Xint0x80_syscall+0x21 dev md5a fsync: giving up on dirty 0xc909f2b8: tag devfs, type VCHR usecount 1, writecount 0, refcount 858 mountedhere 0xc88ce800 flags (VV_COPYONWRITE) v_object 0xc90af510 ref 0 pages 3418 lock type devfs: EXCL by thread 0xc93588a0 (pid 1887) #0 0xc0a426ee at __lockmgr_args+0xc1e #1 0xc0ae6475 at vop_stdlock+0x65 #2 0xc0de2025 at VOP_LOCK1_APV+0xb5 #3 0xc0b08528 at _vn_lock+0x78 #4 0xc0c993a7 at softdep_flushworklist+0x47 #5 0xc0c9e36d at ffs_sync+0x32d #6 0xc0b08122 at vfs_write_suspend+0x122 #7 0xc0c8250d at ffs_snapshot+0xced #8 0xc0ca08a7 at ffs_mount+0x1ca7 #9 0xc0af062b at vfs_donmount+0x12eb #10 0xc0af0ccf at sys_nmount+0x7f #11 0xc0dbecc7 at syscall+0x2c7 #12 0xc0da7ff1 at Xint0x80_syscall+0x21 dev md5a fsync: giving up on dirty 0xc909f2b8: tag devfs, type VCHR usecount 1, writecount 0, refcount 860 mountedhere 0xc88ce800 flags (VV_COPYONWRITE) v_object 0xc90af510 ref 0 pages 3426 lock type devfs: EXCL by thread 0xc8f7b5c0 (pid 1891) #0 0xc0a426ee at __lockmgr_args+0xc1e #1 0xc0ae6475 at vop_stdlock+0x65 #2 0xc0de2025 at VOP_LOCK1_APV+0xb5 #3 0xc0b08528 at _vn_lock+0x78 #4 0xc0c993a7 at softdep_flushworklist+0x47 #5 0xc0c9e36d at ffs_sync+0x32d #6 0xc0b08122 at vfs_write_suspend+0x122 #7 0xc0c8250d at ffs_snapshot+0xced #8 0xc0ca08a7 at ffs_mount+0x1ca7 #9 0xc0af062b at vfs_donmount+0x12eb #10 0xc0af0ccf at sys_nmount+0x7f #11 0xc0dbecc7 at syscall+0x2c7 #12 0xc0da7ff1 at Xint0x80_syscall+0x21 dev md5a fsync: giving up on dirty 0xc909f2b8: tag devfs, type VCHR usecount 1, writecount 0, refcount 862 mountedhere 0xc88ce800 flags (VV_COPYONWRITE) v_object 0xc90af510 ref 0 pages 3434 lock type devfs: EXCL by thread 0xc8d8f000 (pid 1895) #0 0xc0a426ee at __lockmgr_args+0xc1e #1 0xc0ae6475 at vop_stdlock+0x65 #2 0xc0de2025 at VOP_LOCK1_APV+0xb5 #3 0xc0b08528 at _vn_lock+0x78 #4 0xc0c993a7 at softdep_flushworklist+0x47 #5 0xc0c9e36d at ffs_sync+0x32d #6 0xc0b08122 at vfs_write_suspend+0x122 #7 0xc0c8250d at ffs_snapshot+0xced #8 0xc0ca08a7 at ffs_mount+0x1ca7 #9 0xc0af062b at vfs_donmount+0x12eb #10 0xc0af0ccf at sys_nmount+0x7f #11 0xc0dbecc7 at syscall+0x2c7 #12 0xc0da7ff1 at Xint0x80_syscall+0x21 dev md5a fsync: giving up on dirty 0xc909f2b8: tag devfs, type VCHR usecount 1, writecount 0, refcount 864 mountedhere 0xc88ce800 flags (VV_COPYONWRITE) v_object 0xc90af510 ref 0 pages 3442 lock type devfs: EXCL by thread 0xc93598a0 (pid 1899) #0 0xc0a426ee at __lockmgr_args+0xc1e #1 0xc0ae6475 at vop_stdlock+0x65 #2 0xc0de2025 at VOP_LOCK1_APV+0xb5 #3 0xc0b08528 at _vn_lock+0x78 #4 0xc0c993a7 at softdep_flushworklist+0x47 #5 0xc0c9e36d at ffs_sync+0x32d #6 0xc0b08122 at vfs_write_suspend+0x122 #7 0xc0c8250d at ffs_snapshot+0xced #8 0xc0ca08a7 at ffs_mount+0x1ca7 #9 0xc0af062b at vfs_donmount+0x12eb #10 0xc0af0ccf at sys_nmount+0x7f #11 0xc0dbecc7 at syscall+0x2c7 #12 0xc0da7ff1 at Xint0x80_syscall+0x21 dev md5a fsync: giving up on dirty 0xc909f2b8: tag devfs, type VCHR usecount 1, writecount 0, refcount 863 mountedhere 0xc88ce800 flags (VV_COPYONWRITE) v_object 0xc90af510 ref 0 pages 3438 lock type devfs: EXCL by thread 0xc93598a0 (pid 1899) #0 0xc0a426ee at __lockmgr_args+0xc1e #1 0xc0ae6475 at vop_stdlock+0x65 #2 0xc0de2025 at VOP_LOCK1_APV+0xb5 #3 0xc0b08528 at _vn_lock+0x78 #4 0xc0c9e437 at ffs_sync+0x3f7 #5 0xc0b08122 at vfs_write_suspend+0x122 #6 0xc0c8250d at ffs_snapshot+0xced #7 0xc0ca08a7 at ffs_mount+0x1ca7 #8 0xc0af062b at vfs_donmount+0x12eb #9 0xc0af0ccf at sys_nmount+0x7f #10 0xc0dbecc7 at syscall+0x2c7 #11 0xc0da7ff1 at Xint0x80_syscall+0x21 dev md5a fsync: giving up on dirty 0xc909f2b8: tag devfs, type VCHR usecount 1, writecount 0, refcount 822 mountedhere 0xc88ce800 flags (VV_COPYONWRITE) v_object 0xc90af510 ref 0 pages 3434 lock type devfs: EXCL by thread 0xc88ea8a0 (pid 1903) #0 0xc0a426ee at __lockmgr_args+0xc1e #1 0xc0ae6475 at vop_stdlock+0x65 #2 0xc0de2025 at VOP_LOCK1_APV+0xb5 #3 0xc0b08528 at _vn_lock+0x78 #4 0xc0c993a7 at softdep_flushworklist+0x47 #5 0xc0c9e36d at ffs_sync+0x32d #6 0xc0b08122 at vfs_write_suspend+0x122 #7 0xc0c8250d at ffs_snapshot+0xced #8 0xc0ca08a7 at ffs_mount+0x1ca7 #9 0xc0af062b at vfs_donmount+0x12eb #10 0xc0af0ccf at sys_nmount+0x7f #11 0xc0dbecc7 at syscall+0x2c7 #12 0xc0da7ff1 at Xint0x80_syscall+0x21 dev md5a fsync: giving up on dirty 0xc909f2b8: tag devfs, type VCHR usecount 1, writecount 0, refcount 821 mountedhere 0xc88ce800 flags (VV_COPYONWRITE) v_object 0xc90af510 ref 0 pages 3430 lock type devfs: EXCL by thread 0xc88ea8a0 (pid 1903) #0 0xc0a426ee at __lockmgr_args+0xc1e #1 0xc0ae6475 at vop_stdlock+0x65 #2 0xc0de2025 at VOP_LOCK1_APV+0xb5 #3 0xc0b08528 at _vn_lock+0x78 #4 0xc0c9e437 at ffs_sync+0x3f7 #5 0xc0b08122 at vfs_write_suspend+0x122 #6 0xc0c8250d at ffs_snapshot+0xced #7 0xc0ca08a7 at ffs_mount+0x1ca7 #8 0xc0af062b at vfs_donmount+0x12eb #9 0xc0af0ccf at sys_nmount+0x7f #10 0xc0dbecc7 at syscall+0x2c7 #11 0xc0da7ff1 at Xint0x80_syscall+0x21 dev md5a Feb 8 07:18:54 leopard3 kernel: pid 1955 (rw), uid 1001 inumber 47117 on /mnt: filesystem full Feb 8 07:23:03 leopard3 kernel: pid 2387 (rw), uid 1001 inumber 23605 on /mnt: filesystem full Wed Feb 8 08:26:13 CET 2012 8:26AM up 11 mins, 2 users, load averages: 13.22, 11.20, 6.82 Feb 8 07:27:06 leopard3 kernel: pid 2843 (rw), uid 1001 inumber 23586 on /mnt: filesystem full Feb 8 07:33:08 leopard3 kernel: pid 3499 (rw), uid 1001 inumber 47105 on /mnt: filesystem full Feb 8 07:35:10 leopard3 kernel: pid 3714 (rw), uid 1001 inumber 47147 on /mnt: filesystem full Wed Feb 8 08:36:13 CET 2012 8:36AM up 21 mins, 2 users, load averages: 15.46, 16.37, 12.13 lock order reversal: 1st 0xc8ea0c94 newnfs (newnfs) @ /mnt/pho/src/sys/kern/vfs_mount.c:1249 2nd 0xc8fc5c94 syncer (syncer) @ /mnt/pho/src/sys/kern/vfs_subr.c:2279 KDB: stack backtrace: db_trace_self_wrapper(c0f9095b,2f6e7265,5f736676,72627573,323a632e,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a94a3b,c0f94460,c1272058,8e7,ee8159dc,...) at kdb_backtrace+0x2a _witness_debugger(c0f94460,c8fc5c94,c0f9d0f4,c6158748,c0f9c534,...) at _witness_debugger+0x25 witness_checkorder(c8fc5c94,9,c0f9c534,8e7,0,...) at witness_checkorder+0x86f __lockmgr_args(c8fc5c94,80100,c8fc5d00,0,0,...) at __lockmgr_args+0x834 vop_stdlock(ee815afc,c0aa4bfa,c0f94856,80100,c8fc5c3c,...) at vop_stdlock+0x65 VOP_LOCK1_APV(c10ca6a0,ee815afc,c0a4aa5c,c10ecd00,c8fc5c3c,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c8fc5c3c,80100,c0f9c534,8e7,c129fc20,...) at _vn_lock+0x78 vputx(ee815b80,c0afd19d,c8fc5c3c,0,c0f9c534,...) at vputx+0x266 vrele(c8fc5c3c,0,c0f9c534,db7,c8ea0c3c,...) at vrele+0x10 vfs_deallocate_syncvnode(c90dd5a8,0,c0f9bb87,51a,0,...) at vfs_deallocate_syncvnode+0x6d dounmount(c90dd5a8,8000000,c8dba5c0,49c,0,...) at dounmount+0x376 sys_unmount(c8dba5c0,ee815cec,c,c,206,...) at sys_unmount+0x421 syscall(ee815d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280dcb3f, esp = 0xbfbfe1bc, ebp = 0xbfbfe288 --- lock order reversal: 1st 0xc8ea0c94 newnfs (newnfs) @ /mnt/pho/src/sys/kern/vfs_mount.c:1249 2nd 0xc909f310 devfs (devfs) @ /mnt/pho/src/sys/ufs/ffs/ffs_softdep.c:1835 KDB: stack backtrace: db_trace_self_wrapper(c0f9095b,7366662f,7366662f,666f735f,70656474,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a94a3b,c0f94460,c1272058,72b,ee8159a0,...) at kdb_backtrace+0x2a _witness_debugger(c0f94460,c909f310,c0f7aacf,c61582d0,c0fc0b20,...) at _witness_debugger+0x25 witness_checkorder(c909f310,9,c0fc0b20,72b,c909f37c,...) at witness_checkorder+0x86f __lockmgr_args(c909f310,80400,c909f37c,0,0,...) at __lockmgr_args+0x834 vop_stdlock(ee815ac0,4,c0f8ac2b,80400,c909f2b8,...) at vop_stdlock+0x65 VOP_LOCK1_APV(c10b19e0,ee815ac0,c12a5ee0,c10ecd00,c909f2b8,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c909f2b8,80400,c0fc0b20,72b,c90dd5a8,...) at _vn_lock+0x78 softdep_flushworklist(c90dd5a8,ee815b6c,c8dba5c0,5e9,c0f9c534,...) at softdep_flushworklist+0x47 ffs_sync(c90dd5a8,1,c0f9bb87,51a,0,...) at ffs_sync+0x32d dounmount(c90dd5a8,8000000,c8dba5c0,49c,0,...) at dounmount+0x3f8 sys_unmount(c8dba5c0,ee815cec,c,c,206,...) at sys_unmount+0x421 syscall(ee815d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280dcb3f, esp = 0xbfbfe1bc, ebp = 0xbfbfe288 --- panic: dqflush: stray dquot cpuid = 0 KDB: enter: panic [ thread pid 3926 tid 100117 ] Stopped at kdb_enter+0x3a: movl $0,kdb_why db> run pho db:0:pho> bt Tracing pid 3926 tid 100117 td 0xc8dba5c0 kdb_enter(c0f8c850,c0f8c850,c0fc538f,ee815a30,0,...) at kdb_enter+0x3a panic(c0fc538f,0,c0fc5191,64f,0,...) at panic+0x1ac quotaoff1(ee815aa8,c0a4a601,c61a0448,4,c0f8ac2b,...) at quotaoff1+0x318 quotaoff_inchange(c61a0448,0,c0fc5191,2fe,0,...) at quotaoff_inchange+0x18 quotaoff(c8dba5c0,c90dd5a8,0,c8dba5c0,c61a0400,...) at quotaoff+0xc6 ffs_flushfiles(c90dd5a8,0,c8dba5c0,ee815b34,3,...) at ffs_flushfiles+0x6b softdep_flushfiles(c90dd5a8,0,c8dba5c0,0,1df32a6,...) at softdep_flushfiles+0x2e ffs_unmount(c90dd5a8,8000000,c0f9bb87,51a,0,...) at ffs_unmount+0x1bf dounmount(c90dd5a8,8000000,c8dba5c0,49c,0,...) at dounmount+0x419 sys_unmount(c8dba5c0,ee815cec,c,c,206,...) at sys_unmount+0x421 syscall(ee815d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280dcb3f, esp = 0xbfbfe1bc, ebp = 0xbfbfe288 --- db:0:bt> show allpcpu Current CPU: 0 cpuid = 0 dynamic pcpu = 0x737b00 curthread = 0xc8dba5c0: pid 3926 "umount" curpcb = 0xee815d80 fpcurthread = none idlethread = 0xc61b88a0: tid 100003 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: cpuid = 1 dynamic pcpu = 0x4dbab00 curthread = 0xc61b85c0: pid 11 "idle: cpu1" curpcb = 0xc5eb5d80 fpcurthread = none idlethread = 0xc61b85c0: tid 100004 "idle: cpu1" APIC ID = 1 currentldt = 0x50 spin locks held: db:0:allpcpu> show alllocks Process 3926 (umount) thread 0xc8dba5c0 (100117) exclusive sleep mutex dqhlock (dqhlock) r = 0 (0xc12a5fe8) locked @ /mnt/pho/src/sys/ufs/ufs/ufs_quota.c:1615 exclusive sleep mutex Giant (Giant) r = 0 (0xc1128110) locked @ /mnt/pho/src/sys/kern/vfs_mount.c:1164 exclusive lockmgr newnfs (newnfs) r = 0 (0xc8ea0c94) locked @ /mnt/pho/src/sys/kern/vfs_mount.c:1249 Process 1782 (sshd) thread 0xc8dbb5c0 (100109) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xc8f6f08c) locked @ /mnt/pho/src/sys/kern/uipc_sockbuf.c:148 db:0:alllocks> show lockedvnods Locked vnodes 0xc8ea0c3c: tag newnfs, type VDIR usecount 1, writecount 0, refcount 1 mountedhere 0xc90dd5a8 flags () v_object 0xc8f88000 ref 0 pages 0 lock type newnfs: EXCL by thread 0xc8dba5c0 (pid 3926) #0 0xc0a426ee at __lockmgr_args+0xc1e #1 0xc09ab39d at nfs_lock1+0xcd #2 0xc0de2025 at VOP_LOCK1_APV+0xb5 #3 0xc0b08528 at _vn_lock+0x78 #4 0xc0af0d7c at dounmount+0x8c #5 0xc0af17d1 at sys_unmount+0x421 #6 0xc0dbecc7 at syscall+0x2c7 #7 0xc0da7ff1 at Xint0x80_syscall+0x21 fileid 19383889 fsid 0x3a00ff01 db:0:lockedvnods> show mount 0xc891db50 192.168.5.1:/zoo/leopard3 on / (nfs) 0xc891e000 devfs on /dev (devfs) 0xc891d87c /dev/ada0d on /work (ufs) 0xc891d5a8 procfs on /proc (procfs) 0xc891d2d4 /dev/md0 on /var (ufs) 0xc891d000 /dev/md1 on /tmp (ufs) 0xc90dd5a8 /dev/md5a on /mnt (ufs) More info: show mount db:0:mount> ps pid ppid pgrp uid state wmesg wchan cmd 3926 1797 1797 0 R+ CPU 0 umount 3746 1778 1778 0 S+ nanslp 0xc1129a64 sleep 1810 0 0 0 DL mdwait 0xc90d2800 [md5] 1797 1793 1797 0 S+ wait 0xc8e7984c sh 1793 1789 1793 0 S+ wait 0xc8e7bb10 sh 1789 1788 1789 0 S+ pause 0xc88e98a4 csh 1788 1787 1788 1001 S+ wait 0xc8cb1588 su 1787 1786 1787 1001 Ss+ wait 0xc8db9588 sh 1786 1782 1782 1001 S select 0xc8fce1a4 sshd 1782 1678 1782 0 Ss sbwait 0xc8f6f0b8 sshd 1778 1774 1778 0 S+ wait 0xc8db6b10 sh 1774 1773 1774 0 S+ pause 0xc8db8058 csh 1773 1 1773 0 Ss+ wait 0xc8f75588 login 1772 1 1772 0 Ss+ ttyin 0xc6263670 getty 1771 1 1771 0 Ss+ ttyin 0xc6263870 getty 1770 1 1770 0 Ss+ ttyin 0xc6263a70 getty 1769 1 1769 0 Ss+ ttyin 0xc6263c70 getty 1768 1 1768 0 Ss+ ttyin 0xc6263e70 getty 1767 1 1767 0 Ss+ ttyin 0xc643f070 getty 1766 1 1766 0 Ss+ ttyin 0xc643f270 getty 1765 1 1765 0 Ss+ ttyin 0xc643f470 getty 1727 1 1727 0 Ss select 0xc88fa3a4 inetd 1699 1 1699 0 Ss nanslp 0xc1129a64 watchdogd 1689 1 1689 0 Ss nanslp 0xc1129a64 cron 1678 1 1678 0 Ss select 0xc884bba4 sshd 1537 1 1537 0 Ss select 0xc645c324 rpc.statd 1530 1529 1529 0 S (threaded) nfsd 100116 S rpcsvc 0xc62d2510 nfsd: service 100115 S rpcsvc 0xc6316e50 nfsd: service 100114 S rpcsvc 0xc88d0010 nfsd: service 100083 S rpcsvc 0xc88d0190 nfsd: master 1529 1 1529 0 Ss select 0xc645c3e4 nfsd 1520 1 1520 0 Ss select 0xc645c2e4 mountd 1454 0 0 0 DL mdwait 0xc8e7e000 [md1] 1399 1 1399 0 Ss select 0xc88fa164 rpcbind 1369 1 1369 0 Ss select 0xc884b9e4 syslogd 1190 1 1190 0 Ss select 0xc88fa364 devd 103 0 0 0 DL mdwait 0xc8e7f000 [md0] 18 0 0 0 DL sdflush 0xc12a5f20 [softdepflush] 17 0 0 0 DL vlruwt 0xc88e7588 [vnlru] 16 0 0 0 DL syncer 0xc129fc34 [syncer] 9 0 0 0 DL psleep 0xc129f8c8 [bufdaemon] 8 0 0 0 DL pgzero 0xc12a7ddc [pagezero] 7 0 0 0 DL psleep 0xc12a79c8 [vmdaemon] 6 0 0 0 DL psleep 0xc12a7990 [pagedaemon] 5 0 0 0 DL ccb_scan 0xc10ee254 [xpt_thrd] 4 0 0 0 DL waiting_ 0xc12a1758 [sctp_iterator] 3 0 0 0 DL ctl_work 0xc6462000 [ctl_thrd] 2 0 0 0 DL - 0xc634de3c [fdc0] 15 0 0 0 DL (threaded) [usb] 100051 D - 0xc63edd34 [usbus4] 100050 D - 0xc63edd04 [usbus4] 100049 D - 0xc63edcd4 [usbus4] 100048 D - 0xc63edca4 [usbus4] 100047 D - 0xc63dadac [usbus3] 100046 D - 0xc63dad7c [usbus3] 100045 D - 0xc63dad4c [usbus3] 100044 D - 0xc63dad1c [usbus3] 100042 D - 0xc63c5dac [usbus2] 100041 D - 0xc63c5d7c [usbus2] 100040 D - 0xc63c5d4c [usbus2] 100039 D - 0xc63c5d1c [usbus2] 100037 D - 0xc63b2dac [usbus1] 100036 D - 0xc63b2d7c [usbus1] 100035 D - 0xc63b2d4c [usbus1] 100034 D - 0xc63b2d1c [usbus1] 100032 D - 0xc6398dac [usbus0] 100031 D - 0xc6398d7c [usbus0] 100030 D - 0xc6398d4c [usbus0] 100029 D - 0xc6398d1c [usbus0] 14 0 0 0 DL - 0xc1128864 [yarrow] 13 0 0 0 DL (threaded) [geom] 100011 D - 0xc1126468 [g_down] 100010 D - 0xc1126464 [g_up] 100009 D - 0xc112645c [g_event] 12 0 0 0 WL (threaded) [intr] 100056 I [irq1: atkbd0] 100054 I [swi0: uart uart] 100053 I [irq258: ahci0] 100052 I [irq14: ata0] 100043 I [irq16: uhci3] 100038 I [irq18: uhci2] 100033 I [irq19: uhci1] 100028 I [irq23: uhci0 ehci0] 100023 I [swi6: task queue] 100022 I [swi6: Giant taskq] 100020 I [swi5: +] 100019 I [swi2: cambio] 100008 I [swi3: vm] 100007 I [swi4: clock] 100006 I [swi4: clock] 100005 I [swi1: netisr 0] 11 0 0 0 RL (threaded) [idle] 100004 Run CPU 1 [idle: cpu1] 100003 CanRun [idle: cpu0] 1 0 1 0 SLs wait 0xc61b6b10 [init] 10 0 0 0 DL audit_wo 0xc12a5660 [audit] 0 0 0 0 DLs (threaded) [kernel] 100068 D - 0xc6248280 [mca taskq] 100058 D - 0xc1128864 [deadlkres] 100027 D - 0xc62485c0 [em1 txq] 100026 D - 0xc6248600 [em1 que] 100025 D - 0xc6248640 [em0 txq] 100024 D - 0xc6248680 [em0 que] 100021 D - 0xc6248740 [thread taskq] 100018 D - 0xc62487c0 [kqueue taskq] 100017 D - 0xc6248800 [ffs_trim taskq] 100016 D - 0xc6248840 [acpi_task_2] 100015 D - 0xc6248840 [acpi_task_1] 100014 D - 0xc6248840 [acpi_task_0] 100012 D - 0xc6248880 [firmware taskq] 100000 D sched 0xc1126540 [swapper] db:0:ps> allt Tracing command umount pid 3926 tid 100117 td 0xc8dba5c0 kdb_enter(c0f8c850,c0f8c850,c0fc538f,ee815a30,0,...) at kdb_enter+0x3a panic(c0fc538f,0,c0fc5191,64f,0,...) at panic+0x1ac quotaoff1(ee815aa8,c0a4a601,c61a0448,4,c0f8ac2b,...) at quotaoff1+0x318 quotaoff_inchange(c61a0448,0,c0fc5191,2fe,0,...) at quotaoff_inchange+0x18 quotaoff(c8dba5c0,c90dd5a8,0,c8dba5c0,c61a0400,...) at quotaoff+0xc6 ffs_flushfiles(c90dd5a8,0,c8dba5c0,ee815b34,3,...) at ffs_flushfiles+0x6b softdep_flushfiles(c90dd5a8,0,c8dba5c0,0,1df32a6,...) at softdep_flushfiles+0x2e ffs_unmount(c90dd5a8,8000000,c0f9bb87,51a,0,...) at ffs_unmount+0x1bf dounmount(c90dd5a8,8000000,c8dba5c0,49c,0,...) at dounmount+0x419 sys_unmount(c8dba5c0,ee815cec,c,c,206,...) at sys_unmount+0x421 syscall(ee815d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280dcb3f, esp = 0xbfbfe1bc, ebp = 0xbfbfe288 --- Tracing command sleep pid 3746 tid 100168 td 0xc9359b80 sched_switch(c9359b80,0,104,1a3,9f808528,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(c9359b80,0,c0f92340,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(927c1,c0a9b270,c9359b80,0,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c1129a64,6c,c0f8de0b,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c1129a64,0,16c,c0f8de0b,927c1,...) at _sleep+0x34f kern_nanosleep(c9359b80,ee8c4c50,ee8c4c58,258,0,...) at kern_nanosleep+0xc1 sys_nanosleep(c9359b80,ee8c4cec,c0fdec62,c0f95b40,246,...) at sys_nanosleep+0x6f syscall(ee8c4d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x281660b7, esp = 0xbfbfec2c, ebp = 0xbfbfec78 --- Tracing command md5 pid 1810 tid 100133 td 0xc8dbf8a0 sched_switch(c8dbf8a0,0,104,1a3,7f64118,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c8dbf8a0,0,c0f92340,268,0,...) at sleepq_switch+0x162 sleepq_wait(c90d2800,5c,c0f28c8a,0,0,...) at sleepq_wait+0x63 _sleep(c90d2800,c90d2820,25c,c0f28c8a,0,...) at _sleep+0x39c md_kthread(c90d2800,ee845d28,c0f86ef0,3db,c90cc2c4,...) at md_kthread+0x11a fork_exit(c07692a0,c90d2800,ee845d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee845d60, ebp = 0 --- Tracing command sh pid 1797 tid 100106 td 0xc8dbe000 sched_switch(c8dbe000,0,104,1a3,ce19b870,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(c8dbe000,0,c0f92340,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f92340,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8e7984c,6c,c0f952bf,100,0,...) at sleepq_wait_sig+0x17 _sleep(c8e7984c,c8e798d4,16c,c0f952bf,0,...) at _sleep+0x385 kern_wait(c8dbe000,ffffffff,ee7cbc60,2,0,...) at kern_wait+0x626 sys_wait4(c8dbe000,ee7cbcec,c0fdec62,c0f9529e,212,...) at sys_wait4+0x3b syscall(ee7cbd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x28161ffb, esp = 0xbfbfe8bc, ebp = 0xbfbfe8d8 --- Tracing command sh pid 1793 tid 100128 td 0xc8f7b8a0 sched_switch(c8f7b8a0,0,104,1a3,90933d88,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(c8f7b8a0,0,c0f92340,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f92340,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8e7bb10,6c,c0f952bf,100,0,...) at sleepq_wait_sig+0x17 _sleep(c8e7bb10,c8e7bb98,16c,c0f952bf,0,...) at _sleep+0x385 kern_wait(c8f7b8a0,ffffffff,ee836c60,2,0,...) at kern_wait+0x626 sys_wait4(c8f7b8a0,ee836cec,c0fdec62,c0f9529e,212,...) at sys_wait4+0x3b syscall(ee836d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x28161ffb, esp = 0xbfbfe98c, ebp = 0xbfbfe9a8 --- Tracing command csh pid 1789 tid 100078 td 0xc8cb28a0 sched_switch(c8cb28a0,0,104,1a3,ff323d80,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,74,...) at mi_switch+0x1ff sleepq_switch(c8cb28a0,0,c0f92340,1a7,74,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f92340,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c88e98a4,74,c0e1316e,100,0,...) at sleepq_wait_sig+0x17 _sleep(c88e98a4,c88e98d4,174,c0e1316e,0,...) at _sleep+0x385 kern_sigsuspend(c8cb28a0,0,0,0,0,...) at kern_sigsuspend+0xba sys_sigsuspend(c8cb28a0,ee75ecec,c0fdec62,c0f951ae,202,...) at sys_sigsuspend+0x4d syscall(ee75ed28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x2817d8fb, esp = 0xbfbfe72c, ebp = 0xbfbfe798 --- Tracing command su pid 1788 tid 100072 td 0xc8cb55c0 sched_switch(c8cb55c0,0,104,1a3,992e3868,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(c8cb55c0,0,c0f92340,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f92340,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8cb1588,6c,c0f952bf,100,0,...) at sleepq_wait_sig+0x17 _sleep(c8cb1588,c8cb1610,16c,c0f952bf,0,...) at _sleep+0x385 kern_wait(c8cb55c0,6fd,ee744c60,2,0,...) at kern_wait+0x626 sys_wait4(c8cb55c0,ee744cec,c,c,212,...) at sys_wait4+0x3b syscall(ee744d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2812cffb, esp = 0xbfbfe76c, ebp = 0xbfbfe788 --- Tracing command sh pid 1787 tid 100112 td 0xc8dbab80 sched_switch(c8dbab80,0,104,1a3,9279c680,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(c8dbab80,0,c0f92340,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f92340,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8db9588,6c,c0f952bf,100,0,...) at sleepq_wait_sig+0x17 _sleep(c8db9588,c8db9610,16c,c0f952bf,0,...) at _sleep+0x385 kern_wait(c8dbab80,ffffffff,ee7ddc60,2,0,...) at kern_wait+0x626 sys_wait4(c8dbab80,ee7ddcec,c,c,212,...) at sys_wait4+0x3b syscall(ee7ddd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x28161ffb, esp = 0xbfbfeaac, ebp = 0xbfbfeac8 --- Tracing command sshd pid 1786 tid 100079 td 0xc8cb25c0 sched_switch(c8cb25c0,0,104,1a3,54ae5bb0,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8cb25c0,0,c0f92340,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4a68a,c8fce190,0,c0f8ac2b,c8cb25c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8fce1a4,0,ee762a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c8fce1a4,c8fce190,c0f94ad8,6a4,c8cc07e0,...) at _cv_wait_sig+0x260 seltdwait(c8cc07e0,58,c908c100,c8cb25c0,ee762acc,...) at seltdwait+0xa2 kern_select(c8cb25c0,a,288060b8,288060cc,0,0,20,c88e9588,0) at kern_select+0x655 sys_select(c8cb25c0,ee762cec,c0fdec62,c0f5a9c7,206,...) at sys_select+0x66 syscall(ee762d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284b0d43, esp = 0xbfbfe19c, ebp = 0xbfbfe1e8 --- Tracing command sshd pid 1782 tid 100109 td 0xc8dbb5c0 sched_switch(c8dbb5c0,0,104,1a3,18d75988,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,68,...) at mi_switch+0x1ff sleepq_switch(c8dbb5c0,0,c0f92340,1a7,68,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f92340,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8f6f0b8,68,c0f98a78,100,0,...) at sleepq_wait_sig+0x17 _sleep(c8f6f0b8,c8f6f078,168,c0f98a78,0) at _sleep+0x385 sbwait(c8f6f054,4,c0f98b70,5ee,c8f6f078,...) at sbwait+0x76 soreceive_generic(c8f6f000,0,ee7d4c44,0,0,...) at soreceive_generic+0x320 soreceive(c8f6f000,0,ee7d4c44,0,0,0) at soreceive+0x38 soo_read(c8cc10e0,ee7d4c44,c908c280,0,c8dbb5c0,...) at soo_read+0x4e dofileread(ee7d4c44,ffffffff,ffffffff,0,ee7d4c20,...) at dofileread+0x9e kern_readv(c8dbb5c0,5,ee7d4c44,ee7d4c64,1,...) at kern_readv+0x68 sys_read(c8dbb5c0,ee7d4cec,c0fdec62,c0f7d0a1,286,...) at sys_read+0x4f syscall(ee7d4d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x284b0dc3, esp = 0xbfbfdd6c, ebp = 0xbfbfdda8 --- Tracing command sh pid 1778 tid 100097 td 0xc8d8e8a0 sched_switch(c8d8e8a0,0,104,1a3,9ead25c0,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(c8d8e8a0,0,c0f92340,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f92340,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8db6b10,6c,c0f952bf,100,0,...) at sleepq_wait_sig+0x17 _sleep(c8db6b10,c8db6b98,16c,c0f952bf,0,...) at _sleep+0x385 kern_wait(c8d8e8a0,ffffffff,ee7afc60,2,0,...) at kern_wait+0x626 sys_wait4(c8d8e8a0,ee7afcec,c0fdec62,c0f9529e,212,...) at sys_wait4+0x3b syscall(ee7afd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x28161ffb, esp = 0xbfbfe9dc, ebp = 0xbfbfe9f8 --- Tracing command csh pid 1774 tid 100096 td 0xc8d8eb80 sched_switch(c8d8eb80,0,104,1a3,439bf30,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,74,...) at mi_switch+0x1ff sleepq_switch(c8d8eb80,0,c0f92340,1a7,74,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f92340,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8db8058,74,c0e1316e,100,0,...) at sleepq_wait_sig+0x17 _sleep(c8db8058,c8db8088,174,c0e1316e,0,...) at _sleep+0x385 kern_sigsuspend(c8d8eb80,0,0,0,0,...) at kern_sigsuspend+0xba sys_sigsuspend(c8d8eb80,ee7abcec,c0fdec62,c0f951ae,202,...) at sys_sigsuspend+0x4d syscall(ee7abd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x2817d8fb, esp = 0xbfbfe7ec, ebp = 0xbfbfe858 --- Tracing command login pid 1773 tid 100120 td 0xc8f7d2e0 sched_switch(c8f7d2e0,0,104,1a3,741f2b18,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(c8f7d2e0,0,c0f92340,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f92340,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c8f75588,6c,c0f952bf,100,0,...) at sleepq_wait_sig+0x17 _sleep(c8f75588,c8f75610,16c,c0f952bf,0,...) at _sleep+0x385 kern_wait(c8f7d2e0,6ee,ee81ec60,0,0,...) at kern_wait+0x626 sys_wait4(c8f7d2e0,ee81ecec,c,c,212,...) at sys_wait4+0x3b syscall(ee81ed28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x28125ffb, esp = 0xbfbfecec, ebp = 0xbfbfed08 --- Tracing command getty pid 1772 tid 100081 td 0xc8cb2000 sched_switch(c8cb2000,0,104,1a3,86a226a0,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8cb2000,0,c0f92340,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f92340,160,0,c8cb2000,c8cb2000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6263670,0,c0f97208,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6263670,c1128110,c0f96bc8,527,0,...) at _cv_wait_sig+0x260 tty_wait(c6263600,c6263670,ee76ac44,1,0,...) at tty_wait+0x71 ttydisc_read(c6263600,ee76ac44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c630c600,ee76ac44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c8cc15e8,ee76ac44,c6197e80,0,c8cb2000,...) at devfs_read_f+0x7e dofileread(ee76ac44,ffffffff,ffffffff,0,ee76ac20,...) at dofileread+0x9e kern_readv(c8cb2000,0,ee76ac44,ee76ac64,1,...) at kern_readv+0x68 sys_read(c8cb2000,ee76acec,c0fdec62,c0df15ae,246,...) at sys_read+0x4f syscall(ee76ad28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1771 tid 100123 td 0xc8f7c8a0 sched_switch(c8f7c8a0,0,104,1a3,867dc3a8,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8f7c8a0,0,c0f92340,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f92340,160,0,c8f7c8a0,c8f7c8a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6263870,0,c0f97208,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6263870,c1128110,c0f96bc8,527,0,...) at _cv_wait_sig+0x260 tty_wait(c6263800,c6263870,ee827c44,1,0,...) at tty_wait+0x71 ttydisc_read(c6263800,ee827c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c630c700,ee827c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c8cd2a10,ee827c44,c6197e80,0,c8f7c8a0,...) at devfs_read_f+0x7e dofileread(ee827c44,ffffffff,ffffffff,0,ee827c20,...) at dofileread+0x9e kern_readv(c8f7c8a0,0,ee827c44,ee827c64,1,...) at kern_readv+0x68 sys_read(c8f7c8a0,ee827cec,c0fdec62,c0df15ae,246,...) at sys_read+0x4f syscall(ee827d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1770 tid 100107 td 0xc8dbbb80 sched_switch(c8dbbb80,0,104,1a3,85bfc3b8,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8dbbb80,0,c0f92340,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f92340,160,0,c8dbbb80,c8dbbb80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6263a70,0,c0f97208,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6263a70,c1128110,c0f96bc8,527,0,...) at _cv_wait_sig+0x260 tty_wait(c6263a00,c6263a70,ee7cec44,1,0,...) at tty_wait+0x71 ttydisc_read(c6263a00,ee7cec44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c630c800,ee7cec44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c8ed7000,ee7cec44,c6197e80,0,c8dbbb80,...) at devfs_read_f+0x7e dofileread(ee7cec44,ffffffff,ffffffff,0,ee7cec20,...) at dofileread+0x9e kern_readv(c8dbbb80,0,ee7cec44,ee7cec64,1,...) at kern_readv+0x68 sys_read(c8dbbb80,ee7cecec,c0fdec62,c0df15ae,246,...) at sys_read+0x4f syscall(ee7ced28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1769 tid 100094 td 0xc8d8f2e0 sched_switch(c8d8f2e0,0,104,1a3,868e4688,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8d8f2e0,0,c0f92340,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f92340,160,0,c8d8f2e0,c8d8f2e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6263c70,0,c0f97208,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6263c70,c1128110,c0f96bc8,527,0,...) at _cv_wait_sig+0x260 tty_wait(c6263c00,c6263c70,ee7a5c44,1,0,...) at tty_wait+0x71 ttydisc_read(c6263c00,ee7a5c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c630c900,ee7a5c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c8cd2540,ee7a5c44,c6197e80,0,c8d8f2e0,...) at devfs_read_f+0x7e dofileread(ee7a5c44,ffffffff,ffffffff,0,ee7a5c20,...) at dofileread+0x9e kern_readv(c8d8f2e0,0,ee7a5c44,ee7a5c64,1,...) at kern_readv+0x68 sys_read(c8d8f2e0,ee7a5cec,c0fdec62,c0df15ae,246,...) at sys_read+0x4f syscall(ee7a5d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1768 tid 100080 td 0xc8cb22e0 sched_switch(c8cb22e0,0,104,1a3,866b7018,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8cb22e0,0,c0f92340,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f92340,160,0,c8cb22e0,c8cb22e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6263e70,0,c0f97208,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c6263e70,c1128110,c0f96bc8,527,0,...) at _cv_wait_sig+0x260 tty_wait(c6263e00,c6263e70,ee766c44,1,0,...) at tty_wait+0x71 ttydisc_read(c6263e00,ee766c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c630ca00,ee766c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c8cc0a10,ee766c44,c6197e80,0,c8cb22e0,...) at devfs_read_f+0x7e dofileread(ee766c44,ffffffff,ffffffff,0,ee766c20,...) at dofileread+0x9e kern_readv(c8cb22e0,0,ee766c44,ee766c64,1,...) at kern_readv+0x68 sys_read(c8cb22e0,ee766cec,c0fdec62,c0df15ae,246,...) at sys_read+0x4f syscall(ee766d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1767 tid 100089 td 0xc8d902e0 sched_switch(c8d902e0,0,104,1a3,86350000,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8d902e0,0,c0f92340,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f92340,160,0,c8d902e0,c8d902e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c643f070,0,c0f97208,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c643f070,c1128110,c0f96bc8,527,0,...) at _cv_wait_sig+0x260 tty_wait(c643f000,c643f070,ee78fc44,1,0,...) at tty_wait+0x71 ttydisc_read(c643f000,ee78fc44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c630cb00,ee78fc44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c8cd2818,ee78fc44,c6197e80,0,c8d902e0,...) at devfs_read_f+0x7e dofileread(ee78fc44,ffffffff,ffffffff,0,ee78fc20,...) at dofileread+0x9e kern_readv(c8d902e0,0,ee78fc44,ee78fc64,1,...) at kern_readv+0x68 sys_read(c8d902e0,ee78fcec,c0fdec62,c0df15ae,246,...) at sys_read+0x4f syscall(ee78fd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1766 tid 100126 td 0xc8f7c000 sched_switch(c8f7c000,0,104,1a3,8699b4f8,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8f7c000,0,c0f92340,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f92340,160,0,c8f7c000,c8f7c000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c643f270,0,c0f97208,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c643f270,c1128110,c0f96bc8,527,0,...) at _cv_wait_sig+0x260 tty_wait(c643f200,c643f270,ee830c44,1,0,...) at tty_wait+0x71 ttydisc_read(c643f200,ee830c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c630cc00,ee830c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c8cd2230,ee830c44,c6197e80,0,c8f7c000,...) at devfs_read_f+0x7e dofileread(ee830c44,ffffffff,ffffffff,0,ee830c20,...) at dofileread+0x9e kern_readv(c8f7c000,0,ee830c44,ee830c64,1,...) at kern_readv+0x68 sys_read(c8f7c000,ee830cec,c0fdec62,c0df15ae,246,...) at sys_read+0x4f syscall(ee830d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1765 tid 100125 td 0xc8f7c2e0 sched_switch(c8f7c2e0,0,104,1a3,86901468,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8f7c2e0,0,c0f92340,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f92340,160,0,c8f7c2e0,c8f7c2e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c643f470,0,c0f97208,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c643f470,c1128110,c0f96bc8,527,0,...) at _cv_wait_sig+0x260 tty_wait(c643f400,c643f470,ee82dc44,1,0,...) at tty_wait+0x71 ttydisc_read(c643f400,ee82dc44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c630cd00,ee82dc44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c8cc0b60,ee82dc44,c6197e80,0,c8f7c2e0,...) at devfs_read_f+0x7e dofileread(ee82dc44,ffffffff,ffffffff,0,ee82dc20,...) at dofileread+0x9e kern_readv(c8f7c2e0,0,ee82dc44,ee82dc64,1,...) at kern_readv+0x68 sys_read(c8f7c2e0,ee82dcec,c0fdec62,c0df15ae,246,...) at sys_read+0x4f syscall(ee82dd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command inetd pid 1727 tid 100103 td 0xc8dbe8a0 sched_switch(c8dbe8a0,0,104,1a3,74ce6a48,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8dbe8a0,0,c0f92340,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4a68a,c88fa390,0,c0f8ac2b,c8dbe8a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c88fa3a4,0,ee7c2a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c88fa3a4,c88fa390,c0f94ad8,6a4,c8cd2b28,...) at _cv_wait_sig+0x260 seltdwait(c8cd2b28,58,c6197e80,c8dbe8a0,c61546b0,...) at seltdwait+0xa2 kern_select(c8dbe8a0,8,bfbfdfa0,0,0,0,20,c8e7b2c4,4) at kern_select+0x655 sys_select(c8dbe8a0,ee7c2cec,c0fdec62,c0f9530b,202,...) at sys_select+0x66 syscall(ee7c2d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281a8d43, esp = 0xbfbfdf5c, ebp = 0xbfbfede8 --- Tracing command watchdogd pid 1699 tid 100073 td 0xc8cb52e0 sched_switch(c8cb52e0,0,104,1a3,92451ac8,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(c8cb52e0,0,c0f92340,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(ea61,c0a9b270,c8cb52e0,1,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c1129a64,6c,c0f8de0b,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c1129a64,0,16c,c0f8de0b,ea61,...) at _sleep+0x34f kern_nanosleep(c8cb52e0,ee748c50,ee748c58,3c,0,...) at kern_nanosleep+0xc1 sys_nanosleep(c8cb52e0,ee748cec,c0fdec62,c0df15ae,206,...) at sys_nanosleep+0x6f syscall(ee748d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x281920b7, esp = 0xbfbfecbc, ebp = 0xbfbfece8 --- Tracing command cron pid 1689 tid 100084 td 0xc88ec5c0 sched_switch(c88ec5c0,0,104,1a3,b3ab16e8,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(c88ec5c0,0,c0f92340,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(ea61,c0a9b270,c88ec5c0,1,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c1129a64,6c,c0f8de0b,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c1129a64,0,16c,c0f8de0b,ea61,...) at _sleep+0x34f kern_nanosleep(c88ec5c0,ee779c50,ee779c58,3c,0,...) at kern_nanosleep+0xc1 sys_nanosleep(c88ec5c0,ee779cec,c0fdec62,c0f9576c,206,...) at sys_nanosleep+0x6f syscall(ee779d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x281850b7, esp = 0xbfbfec4c, ebp = 0xbfbfec78 --- Tracing command sshd pid 1678 tid 100111 td 0xc8dbb000 sched_switch(c8dbb000,0,104,1a3,1769ea78,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8dbb000,0,c0f92340,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4a68a,c884bb90,0,c0f8ac2b,c8dbb000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c884bba4,0,ee7daa58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c884bba4,c884bb90,c0f94ad8,6a4,c8cd29a0,...) at _cv_wait_sig+0x260 seltdwait(c8cd29a0,58,c6202b80,c8dbb000,c8ebf570,...) at seltdwait+0xa2 kern_select(c8dbb000,7,2880c0b0,0,0,0,20,c8db984c,a) at kern_select+0x655 sys_select(c8dbb000,ee7dacec,c,c,202,...) at sys_select+0x66 syscall(ee7dad28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284b0d43, esp = 0xbfbfe25c, ebp = 0xbfbfedf8 --- Tracing command rpc.statd pid 1537 tid 100075 td 0xc8cb2b80 sched_switch(c8cb2b80,0,104,1a3,70758b68,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8cb2b80,0,c0f92340,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(ee750a20,c0a4a68a,c645c310,0,c8cb2b80,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c645c324,0,ee750a58,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c645c324,c645c310,7531,6a4,c8cd2c08,...) at _cv_timedwait_sig+0x270 seltdwait(ee750c10,ee750c18,c6197e80,c8cb2b80,c8ee6d98,...) at seltdwait+0x8a kern_select(c8cb2b80,8,bfbfed14,0,0,ee750c5c,20,1e,0) at kern_select+0x655 sys_select(c8cb2b80,ee750cec,c0fdec62,c0f953fc,202,...) at sys_select+0x66 syscall(ee750d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x2818cd43, esp = 0xbfbfec6c, ebp = 0xbfbfeda8 --- Tracing command nfsd pid 1530 tid 100116 td 0xc88ea000 sched_switch(c88ea000,0,104,1a3,f2f5f170,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c88ea000,0,c0f92340,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(ee812be0,c0a4a68a,c8929380,0,c88ea000,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c62d2510,0,ee812c18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c62d2510,c8929380,1388,3b1,0,...) at _cv_timedwait_sig+0x270 svc_run_internal(ee812d14,c0a2d3d8,c8929380,ee812d28,c0f86ef0,...) at svc_run_internal+0x356 svc_thread_start(c8929380,ee812d28,c0f86ef0,3db,c8d66588,...) at svc_thread_start+0x10 fork_exit(c0c4a140,c8929380,ee812d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x804c72e, eip = 0xc, esp = 0x33, ebp = 0 --- Tracing command nfsd pid 1530 tid 100115 td 0xc88ea2e0 sched_switch(c88ea2e0,0,104,1a3,6680bc48,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c88ea2e0,0,c0f92340,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(ee80fbe0,c0a4a68a,c8929380,0,c88ea2e0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c6316e50,0,ee80fc18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c6316e50,c8929380,1388,3b1,0,...) at _cv_timedwait_sig+0x270 svc_run_internal(ee80fd14,c0a2d3d8,c8929380,ee80fd28,c0f86ef0,...) at svc_run_internal+0x356 svc_thread_start(c8929380,ee80fd28,c0f86ef0,3db,c8d66588,...) at svc_thread_start+0x10 fork_exit(c0c4a140,c8929380,ee80fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x804c72e, eip = 0xc, esp = 0x33, ebp = 0 --- Tracing command nfsd pid 1530 tid 100114 td 0xc88ea5c0 sched_switch(c88ea5c0,0,104,1a3,19d17390,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c88ea5c0,0,c0f92340,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(ee80cbe0,c0a4a68a,c8929380,0,c88ea5c0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c88d0010,0,ee80cc18,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c88d0010,c8929380,1388,3b1,8051c40,...) at _cv_timedwait_sig+0x270 svc_run_internal(ee80cd14,c0a2d3d8,c8929380,ee80cd28,c0f86ef0,...) at svc_run_internal+0x356 svc_thread_start(c8929380,ee80cd28,c0f86ef0,3db,c8d66588,...) at svc_thread_start+0x10 fork_exit(c0c4a140,c8929380,ee80cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0x804c72e, eip = 0xc, esp = 0x33, ebp = 0 --- Tracing command nfsd pid 1530 tid 100083 td 0xc88ec8a0 sched_switch(c88ec8a0,0,104,1a3,1d4bba50,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c88ec8a0,0,c0f92340,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(ee774a40,c0a4a68a,c8929380,0,c88ec8a0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c88d0190,0,ee774a78,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c88d0190,c8929380,1388,3b1,ee774ac0,...) at _cv_timedwait_sig+0x270 svc_run_internal(c88eca50,14,c0fbbfa2,c8f0e9cf,ee774c2c,...) at svc_run_internal+0x356 svc_run(c8929380,0,c8f0ee17,1bf,c0a667b8,...) at svc_run+0x7b nfssvc_nfsd(bfbfe870,ee774c2c,c,c6197e80,ee774c40,...) at nfssvc_nfsd+0x175 nfssvc_nfsserver(c88ec8a0,ee774cec,bfbfe870,c88ec8a0,ee774d1c,...) at nfssvc_nfsserver+0x55 sys_nfssvc(c88ec8a0,ee774cec,c0fdec62,c0f956cb,202,...) at sys_nfssvc+0x89 syscall(ee774d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (155, FreeBSD ELF32, sys_nfssvc), eip = 0x280dd4fb, esp = 0xbfbfe82c, ebp = 0xbfbfea98 --- Tracing command nfsd pid 1529 tid 100091 td 0xc8d8fb80 sched_switch(c8d8fb80,0,104,1a3,12dee5e0,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8d8fb80,0,c0f92340,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4a68a,c645c3d0,0,c0f8ac2b,c8d8fb80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c645c3e4,0,ee79ca58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c645c3e4,c645c3d0,c0f94ad8,6a4,c8cc1c40,...) at _cv_wait_sig+0x260 seltdwait(c8cc1c40,58,c6197e80,c8d8fb80,c8c4fae0,...) at seltdwait+0xa2 kern_select(c8d8fb80,5,bfbfecb0,0,0,0,20,c8db9000,1) at kern_select+0x655 sys_select(c8d8fb80,ee79ccec,c,c,202,...) at sys_select+0x66 syscall(ee79cd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28184d43, esp = 0xbfbfea9c, ebp = 0xbfbfede8 --- Tracing command mountd pid 1520 tid 100093 td 0xc8d8f5c0 sched_switch(c8d8f5c0,0,104,1a3,a0f55290,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8d8f5c0,0,c0f92340,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4a68a,c645c2d0,0,c0f8ac2b,c8d8f5c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c645c2e4,0,ee7a2a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c645c2e4,c645c2d0,c0f94ad8,6a4,c8cd29d8,...) at _cv_wait_sig+0x260 seltdwait(c8cd29d8,58,c6197e80,c8d8f5c0,478,...) at seltdwait+0xa2 kern_select(c8d8f5c0,9,bfbfed4c,0,0,0,20,c8db884c,bfbfed4c) at kern_select+0x655 sys_select(c8d8f5c0,ee7a2cec,c0fdec62,c0f7d0a1,206,...) at sys_select+0x66 syscall(ee7a2d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x2819ad43, esp = 0xbfbfed1c, ebp = 0xbfbfede8 --- Tracing command md1 pid 1454 tid 100100 td 0xc8d8e000 sched_switch(c8d8e000,0,104,1a3,ce8e3d90,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c8d8e000,0,c0f92340,268,0,...) at sleepq_switch+0x162 sleepq_wait(c8e7e000,5c,c0f28c8a,0,0,...) at sleepq_wait+0x63 _sleep(c8e7e000,c8e7e020,25c,c0f28c8a,0,...) at _sleep+0x39c md_kthread(c8e7e000,ee7b9d28,c0f86ef0,3db,c8db62c4,...) at md_kthread+0x11a fork_exit(c07692a0,c8e7e000,ee7b9d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee7b9d60, ebp = 0 --- Tracing command rpcbind pid 1399 tid 100104 td 0xc8dbe5c0 sched_switch(c8dbe5c0,0,104,1a3,70166100,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8dbe5c0,0,c0f92340,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(ee7c5a84,c0a4a68a,c88fa150,0,c8dbe5c0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c88fa164,0,ee7c5abc,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c88fa164,c88fa150,7531,6a4,ee7c5b74,...) at _cv_timedwait_sig+0x270 seltdwait(ee7c5c44,ee7c5c4c,5a1,c8dbe5c0,c6154ac0,...) at seltdwait+0x8a sys_poll(c8dbe5c0,ee7c5cec,c0fdec62,c0f953fc,202,...) at sys_poll+0x321 syscall(ee7c5d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x2813ff4b, esp = 0xbfbfcbac, ebp = 0xbfbfed78 --- Tracing command syslogd pid 1369 tid 100101 td 0xc8dbf000 sched_switch(c8dbf000,0,104,1a3,f2f85048,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8dbf000,0,c0f92340,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4a68a,c884b9d0,0,c0f8ac2b,c8dbf000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c884b9e4,0,ee7bca58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c884b9e4,c884b9d0,c0f94ad8,6a4,c8cc0e70,...) at _cv_wait_sig+0x260 seltdwait(c8cc0e70,58,c6197e80,c8dbf000,ee7bcaf4,...) at seltdwait+0xa2 kern_select(c8dbf000,9,284760ac,0,0,0,20,c8db6000,0) at kern_select+0x655 sys_select(c8dbf000,ee7bccec,c0fdec62,c0fc17f7,246,...) at sys_select+0x66 syscall(ee7bcd28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28199d43, esp = 0xbfbfe29c, ebp = 0xbfbfede8 --- Tracing command devd pid 1190 tid 100113 td 0xc8dba8a0 sched_switch(c8dba8a0,0,104,1a3,a04ea408,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c8dba8a0,0,c0f92340,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a4a68a,c88fa350,0,c0f8ac2b,c8dba8a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c88fa364,0,ee7e0a58,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c88fa364,c88fa350,c0f94ad8,6a4,c8cc0dc8,...) at _cv_wait_sig+0x260 seltdwait(c8cc0dc8,58,c6197e80,c8dba8a0,c1272c70,...) at seltdwait+0xa2 kern_select(c8dba8a0,5,bfbfe960,0,0,0,20,c8db92c4,46) at kern_select+0x655 sys_select(c8dba8a0,ee7e0cec,c0fdec62,c0dec85f,206,...) at sys_select+0x66 syscall(ee7e0d28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x808ba43, esp = 0xbfbfe92c, ebp = 0xbfbfee08 --- Tracing command md0 pid 103 tid 100105 td 0xc8dbe2e0 sched_switch(c8dbe2e0,0,104,1a3,f2f7bc00,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c8dbe2e0,0,c0f92340,268,0,...) at sleepq_switch+0x162 sleepq_wait(c8e7f000,5c,c0f28c8a,0,0,...) at sleepq_wait+0x63 _sleep(c8e7f000,c8e7f020,25c,c0f28c8a,0,...) at _sleep+0x39c md_kthread(c8e7f000,ee7c8d28,c0f86ef0,3db,c8e79b10,...) at md_kthread+0x11a fork_exit(c07692a0,c8e7f000,ee7c8d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee7c8d60, ebp = 0 --- Tracing command softdepflush pid 18 tid 100067 td 0xc88eb5c0 sched_switch(c88eb5c0,0,104,1a3,7de6b50,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,54,...) at mi_switch+0x1ff sleepq_switch(c88eb5c0,0,c0f92340,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c12a5f20,54,c0fc3068,0,0,...) at sleepq_timedwait+0x6b _sleep(c12a5f20,c12a5ee0,54,c0fc3068,3e8,...) at _sleep+0x36a softdep_flush(0,ec546d28,c0f86ef0,3db,c88e72c4,...) at softdep_flush+0x266 fork_exit(c0c99570,0,ec546d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xec546d60, ebp = 0 --- Tracing command vnlru pid 17 tid 100066 td 0xc88eb8a0 sched_switch(c88eb8a0,0,104,1a3,a3220e98,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,60,...) at mi_switch+0x1ff sleepq_switch(c88eb8a0,0,c0f92340,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c88e7588,60,c0f9d4ad,0,0,...) at sleepq_timedwait+0x6b _sleep(c88e7588,c129fbf4,260,c0f9d4ad,3e8,...) at _sleep+0x36a vnlru_proc(0,ec543d28,c0f86ef0,3db,c88e7588,...) at vnlru_proc+0xe7 fork_exit(c0afbfc0,0,ec543d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xec543d60, ebp = 0 --- Tracing command syncer pid 16 tid 100065 td 0xc88ebb80 sched_switch(c88ebb80,0,104,1a3,f2e40b58,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c88ebb80,0,c0f92340,28b,c88ebb80,...) at sleepq_switch+0x162 sleepq_timedwait(c129fc34,0,ec540c78,1,0,...) at sleepq_timedwait+0x6b _cv_timedwait(c129fc34,c129fc20,3e8,78b,4e20,...) at _cv_timedwait+0x270 sched_sync(0,ec540d28,c0f86ef0,3db,c88e784c,...) at sched_sync+0x595 fork_exit(c0afb330,0,ec540d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xec540d60, ebp = 0 --- Tracing command bufdaemon pid 9 tid 100064 td 0xc88ec000 sched_switch(c88ec000,0,104,1a3,e6fe3c08,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,54,...) at mi_switch+0x1ff sleepq_switch(c88ec000,0,c0f92340,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c129f8c8,54,c0f9a389,0,0,...) at sleepq_timedwait+0x6b _sleep(c129f8c8,c129f8d0,54,c0f9a389,3e8,...) at _sleep+0x36a buf_daemon(0,ec53dd28,c0f86ef0,3db,c88e7b10,...) at buf_daemon+0x175 fork_exit(c0adfb20,0,ec53dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xec53dd60, ebp = 0 --- Tracing command pagezero pid 8 tid 100063 td 0xc63d62e0 sched_switch(c63d62e0,0,104,1a3,13688260,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c63d62e0,0,c0f92340,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c12a7ddc,0,c0fca186,0,0,...) at sleepq_timedwait+0x6b _sleep(c12a7ddc,c12a6880,0,c0fca186,493e0,...) at _sleep+0x36a vm_pagezero(0,ec53ad28,c0f86ef0,3db,c88e9000,...) at vm_pagezero+0xdc fork_exit(c0ce0720,0,ec53ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xec53ad60, ebp = 0 --- Tracing command vmdaemon pid 7 tid 100062 td 0xc63d65c0 sched_switch(c63d65c0,0,104,1a3,65d2b448,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,74,...) at mi_switch+0x1ff sleepq_switch(c63d65c0,0,c0f92340,268,0,...) at sleepq_switch+0x162 sleepq_wait(c12a79c8,74,c0f9a389,0,0,...) at sleepq_wait+0x63 _sleep(c12a79c8,c12a79cc,74,c0f9a389,0,...) at _sleep+0x39c vm_daemon(0,ec537d28,c0f86ef0,3db,c61b72c4,...) at vm_daemon+0x59 fork_exit(c0cda030,0,ec537d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xec537d60, ebp = 0 --- Tracing command pagedaemon pid 6 tid 100061 td 0xc63d68a0 sched_switch(c63d68a0,0,104,1a3,405ba6d0,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,54,...) at mi_switch+0x1ff sleepq_switch(c63d68a0,0,c0f92340,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c12a7990,54,c0f9a389,0,0,...) at sleepq_timedwait+0x6b _sleep(c12a7990,c12a6880,54,c0f9a389,1388,...) at _sleep+0x36a vm_pageout(0,ec534d28,c0f86ef0,3db,c61b7588,...) at vm_pageout+0x2c7 fork_exit(c0cdb0f0,0,ec534d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xec534d60, ebp = 0 --- Tracing command xpt_thrd pid 5 tid 100060 td 0xc63d6b80 sched_switch(c63d6b80,0,104,1a3,8c7a8ab0,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c63d6b80,0,c0f92340,268,0,...) at sleepq_switch+0x162 sleepq_wait(c10ee254,5c,c0de7057,0,0,...) at sleepq_wait+0x63 _sleep(c10ee254,c10ee298,5c,c0de7057,0,...) at _sleep+0x39c xpt_scanner_thread(0,ec531d28,c0f86ef0,3db,c61b784c,...) at xpt_scanner_thread+0x47 fork_exit(c049d300,0,ec531d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xec531d60, ebp = 0 --- Tracing command sctp_iterator pid 4 tid 100059 td 0xc640d000 sched_switch(c640d000,0,104,1a3,ce2da198,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c640d000,0,c0f92340,268,0,...) at sleepq_switch+0x162 sleepq_wait(c12a1758,0,c0fb0cff,0,0,...) at sleepq_wait+0x63 _sleep(c12a1758,c12a1720,0,c0fb0cff,0,...) at _sleep+0x39c sctp_iterator_thread(0,ec52ed28,c0f86ef0,3db,c61b7b10,...) at sctp_iterator_thread+0x5c fork_exit(c0b8d6a0,0,ec52ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xec52ed60, ebp = 0 --- Tracing command ctl_thrd pid 3 tid 100057 td 0xc640d5c0 sched_switch(c640d5c0,0,104,1a3,6bc06740,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c640d5c0,0,c0f92340,268,0,...) at sleepq_switch+0x162 sleepq_wait(c6462000,5c,c0df1a8a,0,0,...) at sleepq_wait+0x63 _sleep(c6462000,c6462000,5c,c0df1a8a,0,...) at _sleep+0x39c ctl_work_thread(c6462000,ec528d28,c0f86ef0,3db,c63a8000,...) at ctl_work_thread+0x22f6 fork_exit(c04c6700,c6462000,ec528d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xec528d60, ebp = 0 --- Tracing command fdc0 pid 2 tid 100055 td 0xc640db80 sched_switch(c640db80,0,104,1a3,b974cba8,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c640db80,0,c0f92340,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c634de3c,5c,c0f7f0ba,0,0,...) at sleepq_timedwait+0x6b _sleep(c634de3c,c634def0,5c,c0f7f0ba,3e8,...) at _sleep+0x36a fdc_thread(c634de00,ec522d28,c0f86ef0,3db,c63a82c4,...) at fdc_thread+0x27d fork_exit(c0d5f970,c634de00,ec522d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xec522d60, ebp = 0 --- Tracing command usb pid 15 tid 100051 td 0xc63d4000 sched_switch(c63d4000,0,104,1a3,6bbac9d8,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c63d4000,0,c0f92340,268,c63d4000,...) at sleepq_switch+0x162 sleepq_wait(c63edd34,0,c5fbbcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c63edd34,c63ede4c,c0f654b1,74,c63edd3c,...) at _cv_wait+0x260 usb_process(c63edd2c,c5fbbd28,c0f86ef0,3db,c63a8588,...) at usb_process+0x1a3 fork_exit(c08fd190,c63edd2c,c5fbbd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5fbbd60, ebp = 0 --- Tracing command usb pid 15 tid 100050 td 0xc63d42e0 sched_switch(c63d42e0,0,104,1a3,65e7fb50,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c63d42e0,0,c0f92340,268,c63d42e0,...) at sleepq_switch+0x162 sleepq_wait(c63edd04,0,c5fb8cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c63edd04,c63ede4c,c0f654b1,74,c63edd0c,...) at _cv_wait+0x260 usb_process(c63edcfc,c5fb8d28,c0f86ef0,3db,c63a8588,...) at usb_process+0x1a3 fork_exit(c08fd190,c63edcfc,c5fb8d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5fb8d60, ebp = 0 --- Tracing command usb pid 15 tid 100049 td 0xc63d45c0 sched_switch(c63d45c0,0,104,1a3,6b3c7010,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c63d45c0,0,c0f92340,268,c63d45c0,...) at sleepq_switch+0x162 sleepq_wait(c63edcd4,0,c5fb5cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c63edcd4,c63ede4c,c0f654b1,74,c63edcdc,...) at _cv_wait+0x260 usb_process(c63edccc,c5fb5d28,c0f86ef0,3db,c63a8588,...) at usb_process+0x1a3 fork_exit(c08fd190,c63edccc,c5fb5d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5fb5d60, ebp = 0 --- Tracing command usb pid 15 tid 100048 td 0xc63d48a0 sched_switch(c63d48a0,0,104,1a3,6b3c5a60,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c63d48a0,0,c0f92340,268,c63d48a0,...) at sleepq_switch+0x162 sleepq_wait(c63edca4,0,c5fb2cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c63edca4,c63ede4c,c0f654b1,74,c63edcac,...) at _cv_wait+0x260 usb_process(c63edc9c,c5fb2d28,c0f86ef0,3db,c63a8588,...) at usb_process+0x1a3 fork_exit(c08fd190,c63edc9c,c5fb2d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5fb2d60, ebp = 0 --- Tracing command usb pid 15 tid 100047 td 0xc63d4b80 sched_switch(c63d4b80,0,104,1a3,6b3c4418,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c63d4b80,0,c0f92340,268,c63d4b80,...) at sleepq_switch+0x162 sleepq_wait(c63dadac,0,c5faccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c63dadac,c63daec4,c0f654b1,74,c63dadb4,...) at _cv_wait+0x260 usb_process(c63dada4,c5facd28,c0f86ef0,3db,c63a8588,...) at usb_process+0x1a3 fork_exit(c08fd190,c63dada4,c5facd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5facd60, ebp = 0 --- Tracing command usb pid 15 tid 100046 td 0xc63d5000 sched_switch(c63d5000,0,104,1a3,c792b2b0,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c63d5000,0,c0f92340,268,c63d5000,...) at sleepq_switch+0x162 sleepq_wait(c63dad7c,0,c5fa9cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c63dad7c,c63daec4,c0f654b1,74,c63dad84,...) at _cv_wait+0x260 usb_process(c63dad74,c5fa9d28,c0f86ef0,3db,c63a8588,...) at usb_process+0x1a3 fork_exit(c08fd190,c63dad74,c5fa9d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5fa9d60, ebp = 0 --- Tracing command usb pid 15 tid 100045 td 0xc63d52e0 sched_switch(c63d52e0,0,104,1a3,64243f80,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c63d52e0,0,c0f92340,268,c63d52e0,...) at sleepq_switch+0x162 sleepq_wait(c63dad4c,0,c5fa6cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c63dad4c,c63daec4,c0f654b1,74,c63dad54,...) at _cv_wait+0x260 usb_process(c63dad44,c5fa6d28,c0f86ef0,3db,c63a8588,...) at usb_process+0x1a3 fork_exit(c08fd190,c63dad44,c5fa6d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5fa6d60, ebp = 0 --- Tracing command usb pid 15 tid 100044 td 0xc63d55c0 sched_switch(c63d55c0,0,104,1a3,64242b60,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c63d55c0,0,c0f92340,268,c63d55c0,...) at sleepq_switch+0x162 sleepq_wait(c63dad1c,0,c5fa3cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c63dad1c,c63daec4,c0f654b1,74,c63dad24,...) at _cv_wait+0x260 usb_process(c63dad14,c5fa3d28,c0f86ef0,3db,c63a8588,...) at usb_process+0x1a3 fork_exit(c08fd190,c63dad14,c5fa3d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5fa3d60, ebp = 0 --- Tracing command usb pid 15 tid 100042 td 0xc63d5b80 sched_switch(c63d5b80,0,104,1a3,642411e8,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c63d5b80,0,c0f92340,268,c63d5b80,...) at sleepq_switch+0x162 sleepq_wait(c63c5dac,0,c5f9ccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c63c5dac,c63c5ec4,c0f654b1,74,c63c5db4,...) at _cv_wait+0x260 usb_process(c63c5da4,c5f9cd28,c0f86ef0,3db,c63a8588,...) at usb_process+0x1a3 fork_exit(c08fd190,c63c5da4,c5f9cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5f9cd60, ebp = 0 --- Tracing command usb pid 15 tid 100041 td 0xc63d6000 sched_switch(c63d6000,0,104,1a3,c6f059c8,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c63d6000,0,c0f92340,268,c63d6000,...) at sleepq_switch+0x162 sleepq_wait(c63c5d7c,0,c5f99cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c63c5d7c,c63c5ec4,c0f654b1,74,c63c5d84,...) at _cv_wait+0x260 usb_process(c63c5d74,c5f99d28,c0f86ef0,3db,c63a8588,...) at usb_process+0x1a3 fork_exit(c08fd190,c63c5d74,c5f99d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5f99d60, ebp = 0 --- Tracing command usb pid 15 tid 100040 td 0xc63a9000 sched_switch(c63a9000,0,104,1a3,63a9db68,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c63a9000,0,c0f92340,268,c63a9000,...) at sleepq_switch+0x162 sleepq_wait(c63c5d4c,0,c5f96cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c63c5d4c,c63c5ec4,c0f654b1,74,c63c5d54,...) at _cv_wait+0x260 usb_process(c63c5d44,c5f96d28,c0f86ef0,3db,c63a8588,...) at usb_process+0x1a3 fork_exit(c08fd190,c63c5d44,c5f96d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5f96d60, ebp = 0 --- Tracing command usb pid 15 tid 100039 td 0xc63a92e0 sched_switch(c63a92e0,0,104,1a3,63a9c7d0,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c63a92e0,0,c0f92340,268,c63a92e0,...) at sleepq_switch+0x162 sleepq_wait(c63c5d1c,0,c5f93cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c63c5d1c,c63c5ec4,c0f654b1,74,c63c5d24,...) at _cv_wait+0x260 usb_process(c63c5d14,c5f93d28,c0f86ef0,3db,c63a8588,...) at usb_process+0x1a3 fork_exit(c08fd190,c63c5d14,c5f93d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5f93d60, ebp = 0 --- Tracing command usb pid 15 tid 100037 td 0xc63a98a0 sched_switch(c63a98a0,0,104,1a3,63a9b038,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c63a98a0,0,c0f92340,268,c63a98a0,...) at sleepq_switch+0x162 sleepq_wait(c63b2dac,0,c5f8ccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c63b2dac,c63b2ec4,c0f654b1,74,c63b2db4,...) at _cv_wait+0x260 usb_process(c63b2da4,c5f8cd28,c0f86ef0,3db,c63a8588,...) at usb_process+0x1a3 fork_exit(c08fd190,c63b2da4,c5f8cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5f8cd60, ebp = 0 --- Tracing command usb pid 15 tid 100036 td 0xc63a9b80 sched_switch(c63a9b80,0,104,1a3,c64df528,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c63a9b80,0,c0f92340,268,c63a9b80,...) at sleepq_switch+0x162 sleepq_wait(c63b2d7c,0,c5f89cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c63b2d7c,c63b2ec4,c0f654b1,74,c63b2d84,...) at _cv_wait+0x260 usb_process(c63b2d74,c5f89d28,c0f86ef0,3db,c63a8588,...) at usb_process+0x1a3 fork_exit(c08fd190,c63b2d74,c5f89d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5f89d60, ebp = 0 --- Tracing command usb pid 15 tid 100035 td 0xc63ac000 sched_switch(c63ac000,0,104,1a3,632f0418,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c63ac000,0,c0f92340,268,c63ac000,...) at sleepq_switch+0x162 sleepq_wait(c63b2d4c,0,c5f86cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c63b2d4c,c63b2ec4,c0f654b1,74,c63b2d54,...) at _cv_wait+0x260 usb_process(c63b2d44,c5f86d28,c0f86ef0,3db,c63a8588,...) at usb_process+0x1a3 fork_exit(c08fd190,c63b2d44,c5f86d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5f86d60, ebp = 0 --- Tracing command usb pid 15 tid 100034 td 0xc63ac2e0 sched_switch(c63ac2e0,0,104,1a3,632eedf0,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c63ac2e0,0,c0f92340,268,c63ac2e0,...) at sleepq_switch+0x162 sleepq_wait(c63b2d1c,0,c5f83cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c63b2d1c,c63b2ec4,c0f654b1,74,c63b2d24,...) at _cv_wait+0x260 usb_process(c63b2d14,c5f83d28,c0f86ef0,3db,c63a8588,...) at usb_process+0x1a3 fork_exit(c08fd190,c63b2d14,c5f83d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5f83d60, ebp = 0 --- Tracing command usb pid 15 tid 100032 td 0xc63ac8a0 sched_switch(c63ac8a0,0,104,1a3,632ed318,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c63ac8a0,0,c0f92340,268,c63ac8a0,...) at sleepq_switch+0x162 sleepq_wait(c6398dac,0,c5f7ccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c6398dac,c6398ec4,c0f654b1,74,c6398db4,...) at _cv_wait+0x260 usb_process(c6398da4,c5f7cd28,c0f86ef0,3db,c63a8588,...) at usb_process+0x1a3 fork_exit(c08fd190,c6398da4,c5f7cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5f7cd60, ebp = 0 --- Tracing command usb pid 15 tid 100031 td 0xc63acb80 sched_switch(c63acb80,0,104,1a3,c5abdf68,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c63acb80,0,c0f92340,268,c63acb80,...) at sleepq_switch+0x162 sleepq_wait(c6398d7c,0,c5f79cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c6398d7c,c6398ec4,c0f654b1,74,c6398d84,...) at _cv_wait+0x260 usb_process(c6398d74,c5f79d28,c0f86ef0,3db,c63a8588,...) at usb_process+0x1a3 fork_exit(c08fd190,c6398d74,c5f79d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5f79d60, ebp = 0 --- Tracing command usb pid 15 tid 100030 td 0xc63332e0 sched_switch(c63332e0,0,104,1a3,62b340f8,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c63332e0,0,c0f92340,268,c63332e0,...) at sleepq_switch+0x162 sleepq_wait(c6398d4c,0,c5f76cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c6398d4c,c6398ec4,c0f654b1,74,c6398d54,...) at _cv_wait+0x260 usb_process(c6398d44,c5f76d28,c0f86ef0,3db,c63a8588,...) at usb_process+0x1a3 fork_exit(c08fd190,c6398d44,c5f76d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5f76d60, ebp = 0 --- Tracing command usb pid 15 tid 100029 td 0xc63335c0 sched_switch(c63335c0,0,104,1a3,62b32d20,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c63335c0,0,c0f92340,268,c63335c0,...) at sleepq_switch+0x162 sleepq_wait(c6398d1c,0,c5f73cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c6398d1c,c6398ec4,c0f654b1,74,c6398d24,...) at _cv_wait+0x260 usb_process(c6398d14,c5f73d28,c0f86ef0,3db,c63a8588,...) at usb_process+0x1a3 fork_exit(c08fd190,c6398d14,c5f73d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5f73d60, ebp = 0 --- Tracing command yarrow pid 14 tid 100013 td 0xc61bd2e0 sched_switch(c61bd2e0,0,104,1a3,ff79eee0,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c61bd2e0,0,c0f92340,28b,2,...) at sleepq_switch+0x162 sleepq_timedwait(c1128864,0,c0f7f0ba,2,0,...) at sleepq_timedwait+0x6b _sleep(c1128864,0,0,c0f7f0ba,64) at _sleep+0x36a pause(c0f7f0ba,64,c0f41110,111,0,...) at pause+0xa0 random_kthread(0,c5ed7d28,c0f86ef0,3db,c61b6000,...) at random_kthread+0x1ef fork_exit(c0805670,0,c5ed7d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5ed7d60, ebp = 0 --- Tracing command geom pid 13 tid 100011 td 0xc61bd8a0 sched_switch(c61bd8a0,0,104,1a3,7f612c0,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c61bd8a0,0,c0f92340,268,0,...) at sleepq_switch+0x162 sleepq_wait(c1126468,5c,c0f7f0ba,0,0,...) at sleepq_wait+0x63 _sleep(c1126468,c11263c8,25c,c0f7f0ba,0,...) at _sleep+0x39c g_io_schedule_down(c61bd8a0,0,c0f80c61,6c,c5ecdd14,...) at g_io_schedule_down+0x5b g_down_procbody(0,c5ecdd28,c0f86ef0,3db,c61b62c4,...) at g_down_procbody+0x9d fork_exit(c09ee4b0,0,c5ecdd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5ecdd60, ebp = 0 --- Tracing command geom pid 13 tid 100010 td 0xc61bdb80 sched_switch(c61bdb80,0,104,1a3,7f67c30,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c61bdb80,0,c0f92340,268,0,...) at sleepq_switch+0x162 sleepq_wait(c1126464,5c,c0f7f0ba,0,0,...) at sleepq_wait+0x63 _sleep(c1126464,c11263e8,25c,c0f7f0ba,0,...) at _sleep+0x39c g_io_schedule_up(c61bdb80,0,c0f80c61,5f,c5ecad14,...) at g_io_schedule_up+0x134 g_up_procbody(0,c5ecad28,c0f86ef0,3db,c61b62c4,...) at g_up_procbody+0x9d fork_exit(c09ee550,0,c5ecad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5ecad60, ebp = 0 --- Tracing command geom pid 13 tid 100009 td 0xc6203000 sched_switch(c6203000,0,104,1a3,a04c6e78,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,5c,...) at mi_switch+0x1ff sleepq_switch(c6203000,0,c0f92340,268,0,...) at sleepq_switch+0x162 sleepq_wait(c112645c,5c,c0f7f0ba,0,0,...) at sleepq_wait+0x63 _sleep(c112645c,c112638c,25c,c0f7f0ba,0,...) at _sleep+0x39c g_run_events(c11366c0,0,c0f80c61,79,c5ec7d14,...) at g_run_events+0x54b g_event_procbody(0,c5ec7d28,c0f86ef0,3db,c61b62c4,...) at g_event_procbody+0x90 fork_exit(c09ee5f0,0,c5ec7d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5ec7d60, ebp = 0 --- Tracing command intr pid 12 tid 100056 td 0xc640d8a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100054 td 0xc6410000 sched_switch(c6410000,0,109,1a3,9fafc270,...) at sched_switch+0x3ec mi_switch(109,0,c0f8722b,554,c6410000,...) at mi_switch+0x1ff ithread_loop(c640a360,ec515d28,c0f86ef0,3db,c61b6588,...) at ithread_loop+0x1fb fork_exit(c0a30ef0,c640a360,ec515d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xec515d60, ebp = 0 --- Tracing command intr pid 12 tid 100053 td 0xc64102e0 sched_switch(c64102e0,0,109,1a3,ce8c3d78,...) at sched_switch+0x3ec mi_switch(109,0,c0f8722b,554,c64102e0,...) at mi_switch+0x1ff ithread_loop(c6405b60,c5fe5d28,c0f86ef0,3db,c61b6588,...) at ithread_loop+0x1fb fork_exit(c0a30ef0,c6405b60,c5fe5d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5fe5d60, ebp = 0 --- Tracing command intr pid 12 tid 100052 td 0xc64105c0 sched_switch(c64105c0,0,109,1a3,65e3fc78,...) at sched_switch+0x3ec mi_switch(109,0,c0f8722b,554,c64105c0,...) at mi_switch+0x1ff ithread_loop(c640a040,c5fe1d28,c0f86ef0,3db,c61b6588,...) at ithread_loop+0x1fb fork_exit(c0a30ef0,c640a040,c5fe1d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5fe1d60, ebp = 0 --- Tracing command intr pid 12 tid 100043 td 0xc63d58a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100038 td 0xc63a95c0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100033 td 0xc63ac5c0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100028 td 0xc63338a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100023 td 0xc63368a0 sched_switch(c63368a0,0,109,1a3,c90e4fd0,...) at sched_switch+0x3ec mi_switch(109,0,c0f8722b,554,c63368a0,...) at mi_switch+0x1ff ithread_loop(c618a4b0,c5ef5d28,c0f86ef0,3db,c61b6588,...) at ithread_loop+0x1fb fork_exit(c0a30ef0,c618a4b0,c5ef5d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5ef5d60, ebp = 0 --- Tracing command intr pid 12 tid 100022 td 0xc6336b80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100020 td 0xc6203b80 sched_switch(c6203b80,0,109,1a3,c8a5eb58,...) at sched_switch+0x3ec mi_switch(109,0,c0f8722b,554,c6203b80,...) at mi_switch+0x1ff ithread_loop(c618a4d0,c5eecd28,c0f86ef0,3db,c61b6588,...) at ithread_loop+0x1fb fork_exit(c0a30ef0,c618a4d0,c5eecd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5eecd60, ebp = 0 --- Tracing command intr pid 12 tid 100019 td 0xc6332000 sched_switch(c6332000,0,109,1a3,ce8c6c60,...) at sched_switch+0x3ec mi_switch(109,0,c0f8722b,554,c6332000,...) at mi_switch+0x1ff ithread_loop(c618a4e0,c5ee9d28,c0f86ef0,3db,c61b6588,...) at ithread_loop+0x1fb fork_exit(c0a30ef0,c618a4e0,c5ee9d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5ee9d60, ebp = 0 --- Tracing command intr pid 12 tid 100008 td 0xc62032e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100007 td 0xc62035c0 sched_switch(c62035c0,0,109,1a3,ff79c4c8,...) at sched_switch+0x3ec mi_switch(109,0,c0f8722b,554,c62035c0,...) at mi_switch+0x1ff ithread_loop(c618a720,c5ec1d28,c0f86ef0,3db,c61b6588,...) at ithread_loop+0x1fb fork_exit(c0a30ef0,c618a720,c5ec1d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5ec1d60, ebp = 0 --- Tracing command intr pid 12 tid 100006 td 0xc61b8000 sched_switch(c61b8000,0,109,1a3,7895a38,...) at sched_switch+0x3ec mi_switch(109,0,c0f8722b,554,c61b8000,...) at mi_switch+0x1ff ithread_loop(c618a730,c5ebed28,c0f86ef0,3db,c61b6588,...) at ithread_loop+0x1fb fork_exit(c0a30ef0,c618a730,c5ebed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5ebed60, ebp = 0 --- Tracing command intr pid 12 tid 100005 td 0xc61b82e0 sched_switch(c61b82e0,0,109,1a3,7e63cdc8,...) at sched_switch+0x3ec mi_switch(109,0,c0f8722b,554,c61b82e0,...) at mi_switch+0x1ff ithread_loop(c618a740,c5ebbd28,c0f86ef0,3db,c61b6588,...) at ithread_loop+0x1fb fork_exit(c0a30ef0,c618a740,c5ebbd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5ebbd60, ebp = 0 --- Tracing command idle pid 11 tid 100004 td 0xc61b85c0 cpustop_handler(1,13,c5eb5c5c,c0dbf0c6,4dbab00,...) at cpustop_handler+0x28 ipi_nmi_handler(4dbab00,c61b85c0,c5ea8cb4,c5ea8c80,c61b684c,...) at ipi_nmi_handler+0x47 trap(c5eb5c68) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0a81d23, esp = 0xc5eb5ca8, ebp = 0xc5eb5ce8 --- sched_idletd(0,c5eb5d28,c0f86ef0,3db,c61b684c,...) at sched_idletd+0x223 fork_exit(c0a81b00,0,c5eb5d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5eb5d60, ebp = 0 --- Tracing command idle pid 11 tid 100003 td 0xc61b88a0 sched_switch(c61b88a0,0,108,19e,7f67260,...) at sched_switch+0x3ec mi_switch(108,0,c0f8e9b2,a09,c61b88a0,...) at mi_switch+0x1ff sched_idletd(0,c5eb2d28,c0f86ef0,3db,c61b684c,...) at sched_idletd+0x2bf fork_exit(c0a81b00,0,c5eb2d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5eb2d60, ebp = 0 --- Tracing command init pid 1 tid 100002 td 0xc61b8b80 sched_switch(c61b8b80,0,104,1a3,69218e60,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,6c,...) at mi_switch+0x1ff sleepq_switch(c61b8b80,0,c0f92340,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f92340,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c61b6b10,6c,c0f952bf,100,0,...) at sleepq_wait_sig+0x17 _sleep(c61b6b10,c61b6b98,16c,c0f952bf,0,...) at _sleep+0x385 kern_wait(c61b8b80,ffffffff,c5eaec60,0,0,...) at kern_wait+0x626 sys_wait4(c61b8b80,c5eaecec,c0fdec62,c0f950f6,286,...) at sys_wait4+0x3b syscall(c5eaed28) at syscall+0x2c7 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x80551d3, esp = 0xbfbfe8bc, ebp = 0xbfbfe8d8 --- Tracing command audit pid 10 tid 100001 td 0xc61bd000 sched_switch(c61bd000,0,104,1a3,62aeedd8,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c61bd000,0,c0f92340,268,c61bd000,...) at sleepq_switch+0x162 sleepq_wait(c12a5660,0,c5eabc8c,1,0,...) at sleepq_wait+0x63 _cv_wait(c12a5660,c12a5644,c0fbda6f,194,0,...) at _cv_wait+0x260 audit_worker(0,c5eabd28,c0f86ef0,3db,c61b7000,...) at audit_worker+0x84 fork_exit(c0c5a490,0,c5eabd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5eabd60, ebp = 0 --- Tracing command kernel pid 0 tid 100068 td 0xc88eb2e0 sched_switch(c88eb2e0,0,104,1a3,65cbf788,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c88eb2e0,0,c0f92340,268,0,...) at sleepq_switch+0x162 sleepq_wait(c6248280,0,c0f7f0ba,0,0,...) at sleepq_wait+0x63 _sleep(c6248280,c6248298,0,c0f7f0ba,0,...) at _sleep+0x39c taskqueue_thread_loop(c1306ee0,ec549d28,c0f86ef0,3db,c1126540,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a9e970,c1306ee0,ec549d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xec549d60, ebp = 0 --- Tracing command kernel pid 0 tid 100058 td 0xc640d2e0 sched_switch(c640d2e0,0,104,1a3,f2e900a8,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c640d2e0,0,c0f92340,28b,2,...) at sleepq_switch+0x162 sleepq_timedwait(c1128864,0,c0f7f0ba,2,0,...) at sleepq_timedwait+0x6b _sleep(c1128864,0,0,c0f7f0ba,bb8) at _sleep+0x36a pause(c0f7f0ba,bb8,124,122,c11265c8,...) at pause+0xa0 deadlkres(0,ec52bd28,c0f86ef0,3db,c1126540,...) at deadlkres+0x324 fork_exit(c0a106b0,0,ec52bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xec52bd60, ebp = 0 --- Tracing command kernel pid 0 tid 100027 td 0xc6333b80 sched_switch(c6333b80,0,104,1a3,62abfe80,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c6333b80,0,c0f92340,268,c6333b80,...) at sleepq_switch+0x162 sleepq_wait(c62485c0,0,c0f8d5d9,c0f7f0ba,0,...) at sleepq_wait+0x63 msleep_spin(c62485c0,c62485d8,c0f7f0ba,0,c0f8ac2b,...) at msleep_spin+0x259 taskqueue_thread_loop(c635b66c,c5f6cd28,c0f86ef0,3db,c1126540,...) at taskqueue_thread_loop+0x8f fork_exit(c0a9e970,c635b66c,c5f6cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5f6cd60, ebp = 0 --- Tracing command kernel pid 0 tid 100026 td 0xc6336000 sched_switch(c6336000,0,104,1a3,62abed50,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c6336000,0,c0f92340,268,c6336000,...) at sleepq_switch+0x162 sleepq_wait(c6248600,0,c0f8d5d9,c0f7f0ba,0,...) at sleepq_wait+0x63 msleep_spin(c6248600,c6248618,c0f7f0ba,0,c0f8ac2b,...) at msleep_spin+0x259 taskqueue_thread_loop(c6385544,c5f69d28,c0f86ef0,3db,c1126540,...) at taskqueue_thread_loop+0x8f fork_exit(c0a9e970,c6385544,c5f69d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5f69d60, ebp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xc63362e0 sched_switch(c63362e0,0,104,1a3,62abdc68,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c63362e0,0,c0f92340,268,c63362e0,...) at sleepq_switch+0x162 sleepq_wait(c6248640,0,c0f8d5d9,c0f7f0ba,0,...) at sleepq_wait+0x63 msleep_spin(c6248640,c6248658,c0f7f0ba,0,c0f8ac2b,...) at msleep_spin+0x259 taskqueue_thread_loop(c635ba6c,c5f3ed28,c0f86ef0,3db,c1126540,...) at taskqueue_thread_loop+0x8f fork_exit(c0a9e970,c635ba6c,c5f3ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5f3ed60, ebp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xc63365c0 sched_switch(c63365c0,0,104,1a3,f2e86990,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c63365c0,0,c0f92340,268,c63365c0,...) at sleepq_switch+0x162 sleepq_wait(c6248680,0,c0f8d5d9,c0f7f0ba,0,...) at sleepq_wait+0x63 msleep_spin(c6248680,c6248698,c0f7f0ba,0,c0f8ac2b,...) at msleep_spin+0x259 taskqueue_thread_loop(c6371544,c5f3bd28,c0f86ef0,3db,c1126540,...) at taskqueue_thread_loop+0x8f fork_exit(c0a9e970,c6371544,c5f3bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5f3bd60, ebp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xc62038a0 sched_switch(c62038a0,0,104,1a3,f40fb208,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c62038a0,0,c0f92340,268,0,...) at sleepq_switch+0x162 sleepq_wait(c6248740,0,c0f7f0ba,0,0,...) at sleepq_wait+0x63 _sleep(c6248740,c6248758,0,c0f7f0ba,0,...) at _sleep+0x39c taskqueue_thread_loop(c114542c,c5eefd28,c0f86ef0,3db,c1126540,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a9e970,c114542c,c5eefd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5eefd60, ebp = 0 --- Tracing command kernel pid 0 tid 100018 td 0xc63322e0 sched_switch(c63322e0,0,104,1a3,8c7bed00,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c63322e0,0,c0f92340,268,0,...) at sleepq_switch+0x162 sleepq_wait(c62487c0,0,c0f7f0ba,0,0,...) at sleepq_wait+0x63 _sleep(c62487c0,c62487d8,0,c0f7f0ba,0,...) at _sleep+0x39c taskqueue_thread_loop(c1126eb8,c5ee6d28,c0f86ef0,3db,c1126540,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a9e970,c1126eb8,c5ee6d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5ee6d60, ebp = 0 --- Tracing command kernel pid 0 tid 100017 td 0xc63325c0 sched_switch(c63325c0,0,104,1a3,8c7bde88,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c63325c0,0,c0f92340,268,0,...) at sleepq_switch+0x162 sleepq_wait(c6248800,0,c0f7f0ba,0,0,...) at sleepq_wait+0x63 _sleep(c6248800,c6248818,0,c0f7f0ba,0,...) at _sleep+0x39c taskqueue_thread_loop(c12a5c44,c5ee3d28,c0f86ef0,3db,c1126540,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a9e970,c12a5c44,c5ee3d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5ee3d60, ebp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xc63328a0 sched_switch(c63328a0,0,104,1a3,8c7bcab8,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c63328a0,0,c0f92340,268,c63328a0,...) at sleepq_switch+0x162 sleepq_wait(c6248840,0,c0f8d5d9,c0f7f0ba,0,...) at sleepq_wait+0x63 msleep_spin(c6248840,c6248858,c0f7f0ba,0,c0f8ac2b,...) at msleep_spin+0x259 taskqueue_thread_loop(c10f14b8,c5ee0d28,c0f86ef0,3db,c1126540,...) at taskqueue_thread_loop+0x8f fork_exit(c0a9e970,c10f14b8,c5ee0d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5ee0d60, ebp = 0 --- Tracing command kernel pid 0 tid 100015 td 0xc6332b80 sched_switch(c6332b80,0,104,1a3,8c7bba80,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c6332b80,0,c0f92340,268,c6332b80,...) at sleepq_switch+0x162 sleepq_wait(c6248840,0,c0f8d5d9,c0f7f0ba,0,...) at sleepq_wait+0x63 msleep_spin(c6248840,c6248858,c0f7f0ba,0,c0f8ac2b,...) at msleep_spin+0x259 taskqueue_thread_loop(c10f14b8,c5eddd28,c0f86ef0,3db,c1126540,...) at taskqueue_thread_loop+0x8f fork_exit(c0a9e970,c10f14b8,c5eddd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5eddd60, ebp = 0 --- Tracing command kernel pid 0 tid 100014 td 0xc6333000 sched_switch(c6333000,0,104,1a3,8c7bac30,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c6333000,0,c0f92340,268,c6333000,...) at sleepq_switch+0x162 sleepq_wait(c6248840,0,c0f8d5d9,c0f7f0ba,0,...) at sleepq_wait+0x63 msleep_spin(c6248840,c6248858,c0f7f0ba,0,c0f8ac2b,...) at msleep_spin+0x259 taskqueue_thread_loop(c10f14b8,c5edad28,c0f86ef0,3db,c1126540,...) at taskqueue_thread_loop+0x8f fork_exit(c0a9e970,c10f14b8,c5edad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5edad60, ebp = 0 --- Tracing command kernel pid 0 tid 100012 td 0xc61bd5c0 sched_switch(c61bd5c0,0,104,1a3,f45fecd8,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,0,...) at mi_switch+0x1ff sleepq_switch(c61bd5c0,0,c0f92340,268,0,...) at sleepq_switch+0x162 sleepq_wait(c6248880,0,c0f7f0ba,0,0,...) at sleepq_wait+0x63 _sleep(c6248880,c6248898,0,c0f7f0ba,0,...) at _sleep+0x39c taskqueue_thread_loop(c1143e40,c5ed0d28,c0f86ef0,3db,c1126540,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a9e970,c1143e40,c5ed0d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5ed0d60, ebp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xc1126810 sched_switch(c1126810,0,104,1a3,aaa11830,...) at sched_switch+0x3ec mi_switch(104,0,c0f92340,1f3,54,...) at mi_switch+0x1ff sleepq_switch(c1126810,0,c0f92340,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c1126540,54,c0f8f2cc,0,0,...) at sleepq_timedwait+0x6b _sleep(c1126540,0,54,c0f8f2cc,2710,...) at _sleep+0x36a scheduler(0,181ec00,181ec00,181e000,1825000,...) at scheduler+0x240 mi_startup() at mi_startup+0xac begin() at begin+0x2c db:0:allt> call doadump Physical memory: 2026 MB Dumping 241 MB: 226 210 194 178 162 146 130 114 98 82 66 50 34 18 2 Dump complete = 0 db:0:doadump> reset (kgdb) bt #0 doadump (textdump=-293513684) at pcpu.h:244 #1 0xc0510a19 in db_fncall (dummy1=-1062666086, dummy2=0, dummy3=-1, dummy4=0xee8156c0 "ÔV\201î") at /mnt/pho/src/sys/ddb/db_command.c:573 #2 0xc0510e4f in db_command (last_cmdp=0xc10ef67c, cmd_table=0x0, dopager=0) at /mnt/pho/src/sys/ddb/db_command.c:449 #3 0xc0510f04 in db_command_script (command=0xc10f0588 "call doadump") at /mnt/pho/src/sys/ddb/db_command.c:520 #4 0xc05152f0 in db_script_exec (scriptname=0xc10efee0 "doadump", warnifnotfound=Variable "warnifnotfound" is not available. ) at /mnt/pho/src/sys/ddb/db_script.c:302 #5 0xc0515381 in db_run_cmd (addr=-1055940032, have_addr=0, count=-1066928135, modif=0xee8157f8 "") at /mnt/pho/src/sys/ddb/db_script.c:375 #6 0xc0510e11 in db_command (last_cmdp=0xc10ef67c, cmd_table=0x0, dopager=1) at /mnt/pho/src/sys/ddb/db_command.c:449 #7 0xc0510f6a in db_command_loop () at /mnt/pho/src/sys/ddb/db_command.c:502 #8 0xc0512f5d in db_trap (type=3, code=0) at /mnt/pho/src/sys/ddb/db_main.c:229 #9 0xc0a90556 in kdb_trap (type=3, code=0, tf=0xee8159ac) at /mnt/pho/src/sys/kern/subr_kdb.c:629 #10 0xc0dbf6fb in trap (frame=0xee8159ac) at /mnt/pho/src/sys/i386/i386/trap.c:722 #11 0xc0da7f8c in calltrap () at /mnt/pho/src/sys/i386/i386/exception.s:168 #12 0xc0a9009a in kdb_enter (why=0xc0f8c850 "panic", msg=0xc0f8c850 "panic") at cpufunc.h:71 #13 0xc0a5af6c in panic (fmt=0xc0fc538f "dqflush: stray dquot") at /mnt/pho/src/sys/kern/kern_shutdown.c:633 #14 0xc0cb14f8 in quotaoff1 (td=0xc8dba5c0, mp=0xc90dd5a8, type=0) at /mnt/pho/src/sys/ufs/ufs/ufs_quota.c:1622 #15 0xc0cb1758 in quotaoff_inchange (td=Variable "td" is not available. ) at /mnt/pho/src/sys/ufs/ufs/ufs_quota.c:729 #16 0xc0cb1906 in quotaoff (td=0xc8dba5c0, mp=0xc90dd5a8, type=0) at /mnt/pho/src/sys/ufs/ufs/ufs_quota.c:768 #17 0xc0c9db5b in ffs_flushfiles (mp=0xc90dd5a8, flags=0, td=0xc8dba5c0) at /mnt/pho/src/sys/ufs/ffs/ffs_vfsops.c:1349 #18 0xc0c9944e in softdep_flushfiles (oldmnt=0xc90dd5a8, flags=0, td=0xc8dba5c0) at /mnt/pho/src/sys/ufs/ffs/ffs_softdep.c:1898 #19 0xc0c9e79f in ffs_unmount (mp=0xc90dd5a8, mntflags=134217728) at /mnt/pho/src/sys/ufs/ffs/ffs_vfsops.c:1256 #20 0xc0af1109 in dounmount (mp=0xc90dd5a8, flags=134217728, td=0xc8dba5c0) at /mnt/pho/src/sys/kern/vfs_mount.c:1326 #21 0xc0af17d1 in sys_unmount (td=0xc8dba5c0, uap=0xee815cec) at /mnt/pho/src/sys/kern/vfs_mount.c:1224 #22 0xc0dbecc7 in syscall (frame=0xee815d28) at subr_syscall.c:131 #23 0xc0da7ff1 in Xint0x80_syscall () at /mnt/pho/src/sys/i386/i386/exception.s:266 #24 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) f 14 #14 0xc0cb14f8 in quotaoff1 (td=0xc8dba5c0, mp=0xc90dd5a8, type=0) at /mnt/pho/src/sys/ufs/ufs/ufs_quota.c:1622 1622 panic("dqflush: stray dquot"); (kgdb) l 1617 for (dq = LIST_FIRST(dqh); dq; dq = nextdq) { 1618 nextdq = LIST_NEXT(dq, dq_hash); 1619 if (dq->dq_ump->um_quotas[dq->dq_type] != vp) 1620 continue; 1621 if (dq->dq_cnt) 1622 panic("dqflush: stray dquot"); 1623 LIST_REMOVE(dq, dq_hash); 1624 dq->dq_ump = (struct ufsmount *)0; 1625 } 1626 } (kgdb) info loc vp = (struct vnode *) 0x0 qvp = (struct vnode *) 0xc90d5414 mvp = (struct vnode *) 0x0 ump = (struct ufsmount *) 0xc61a0400 dq = Variable "dq" is not available. (kgdb) p *mp $1 = {mnt_mtx = {lock_object = {lo_name = 0xc0f7e819 "struct mount mtx", lo_flags = 16973824, lo_data = 0, lo_witness = 0xc6151450}, mtx_lock = 4}, mnt_gen = 1, mnt_list = { tqe_next = 0x0, tqe_prev = 0xc891d018}, mnt_op = 0xc10dba80, mnt_vfc = 0xc10db8e0, mnt_vnodecovered = 0xc8ea0c3c, mnt_syncer = 0x0, mnt_ref = 5, mnt_nvnodelist = { tqh_first = 0xc90d5414, tqh_last = 0xca3902cc}, mnt_nvnodelistsize = 4, mnt_writeopcount = 1, mnt_kern_flag = 1627390028, mnt_flag = 4294979840, mnt_noasync = 1, mnt_opt = 0xc8913180, mnt_optnew = 0xc89130f0, mnt_maxsymlinklen = 120, mnt_stat = {f_version = 537068824, f_type = 53, f_flags = 4294979840, f_bsize = 2048, f_iosize = 16384, f_blocks = 506483, f_bfree = 382241, f_bavail = 341723, f_files = 141310, f_ffree = 140664, f_syncwrites = 0, f_asyncwrites = 0, f_syncreads = 0, f_asyncreads = 0, f_spare = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, f_namemax = 255, f_owner = 0, f_fsid = {val = {1328685407, -1441716566}}, f_charspare = '\0' , f_fstypename = "ufs", '\0' , f_mntfromname = "/dev/md5a", '\0' , f_mntonname = "/mnt", '\0' }, mnt_cred = 0xc90cd100, mnt_data = 0xc61a0400, mnt_time = 0, mnt_iosize_max = 131072, mnt_export = 0xc8f77000, mnt_label = 0x0, mnt_hashseed = 1033638646, mnt_lockref = 0, mnt_secondary_writes = 0, mnt_secondary_accwrites = 31404711, mnt_susp_owner = 0x0, mnt_gjprovider = 0x0, mnt_explock = {lock_object = {lo_name = 0xc0f7e82a "explock", lo_flags = 91422720, lo_data = 0, lo_witness = 0xc61571c0}, lk_lock = 1, lk_exslpfail = 0, lk_timo = 0, lk_pri = 96, lk_stack = {depth = 6, pcs = {3231983342, 3232660951, 3232695918, 3232697551, 3235638471, 3235545073, 0 }}}} (kgdb) (kgdb) p &dqhashtbl[dqhash] $4 = (struct dqhash *) 0xc884affc (kgdb) p dqhashtbl[dqhash] $5 = {lh_first = 0x0} (kgdb) x/8x 0xc884affc 0xc884affc: 0x00000000 0xc82c1c00 0xc645ce40 0x00000a49 0xc884b00c: 0x82000003 0xc88d4500 0x00000002 0xc82c1b90 $ svn diff -x -p sys Index: sys/ufs/ffs/ffs_snapshot.c =================================================================== --- sys/ufs/ffs/ffs_snapshot.c (revision 231181) +++ sys/ufs/ffs/ffs_snapshot.c (working copy) @@ -225,18 +225,10 @@ ffs_snapshot(mp, snapfile) ump = VFSTOUFS(mp); fs = ump->um_fs; sn = NULL; - /* - * At the moment, journaled soft updates cannot support - * taking snapshots. - */ - if (MOUNTEDSUJ(mp)) { - vfs_mount_error(mp, "%s: Snapshots are not yet supported when " - "running with journaled soft updates", fs->fs_fsmnt); - return (EOPNOTSUPP); - } MNT_ILOCK(mp); flag = mp->mnt_flag; MNT_IUNLOCK(mp); + /* * Need to serialize access to snapshot code per filesystem. */ -- Test scenario: suj19.sh