KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2011 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #0: Sat Nov 19 10:24:38 CET 2011 test@leopard3.netperf.freebsd.org:/mnt/pho/obj/mnt/pho/src/sys/PHO i386 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. CPU: Intel(R) Xeon(R) CPU 3050 @ 2.13GHz (2128.05-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x6f2 Family = 6 Model = f Stepping = 2 Features=0xbfebfbff Features2=0xe3bd AMD Features=0x20100000 AMD Features2=0x1 TSC: P-state invariant, performance statistics real memory = 2147483648 (2048 MB) avail memory = 2080423936 (1984 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs FreeBSD/SMP: 1 package(s) x 2 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard ioapic2 irqs 48-71 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 cpu0: on acpi0 cpu1: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: irq 16 at device 1.0 on pci0 pci1: on pcib1 pcib2: at device 0.0 on pci1 pci2: on pcib2 pcib3: at device 0.2 on pci1 pci3: on pcib3 pcib4: irq 16 at device 3.0 on pci0 pci10: on pcib4 pci10: at device 0.0 (no driver attached) pcib5: irq 17 at device 28.0 on pci0 pcib5: failed to allocate initial I/O port window: 0-0xfff pcib5: failed to allocate initial memory window: 0-0xfffff pcib5: failed to allocate initial prefetch window: 0-0xfffff pci14: on pcib5 pcib6: irq 17 at device 28.4 on pci0 pci18: on pcib6 em0: port 0x5000-0x501f mem 0xe0300000-0xe031ffff irq 16 at device 0.0 on pci18 em0: Using an MSI interrupt em0: Ethernet address: 00:30:48:8e:d0:68 pcib7: irq 16 at device 28.5 on pci0 pci19: on pcib7 em1: port 0x6000-0x601f mem 0xe0400000-0xe041ffff irq 17 at device 0.0 on pci19 em1: Using an MSI interrupt em1: Ethernet address: 00:30:48:8e:d0:69 uhci0: port 0x3000-0x301f irq 23 at device 29.0 on pci0 usbus0: on uhci0 uhci1: port 0x3020-0x303f irq 19 at device 29.1 on pci0 usbus1: on uhci1 uhci2: port 0x3040-0x305f irq 18 at device 29.2 on pci0 usbus2: on uhci2 uhci3: port 0x3060-0x307f irq 16 at device 29.3 on pci0 usbus3: on uhci3 ehci0: mem 0xe0000000-0xe00003ff irq 23 at device 29.7 on pci0 usbus4: EHCI version 1.0 usbus4: on ehci0 pcib8: at device 30.0 on pci0 pci20: on pcib8 vgapci0: port 0x7000-0x70ff mem 0xe8000000-0xefffffff,0xe0500000-0xe050ffff irq 16 at device 4.0 on pci20 isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x30a0-0x30af at device 31.1 on pci0 ata0: at channel 0 on atapci0 ahci0: port 0x30e8-0x30ef,0x30dc-0x30df,0x30e0-0x30e7,0x30d8-0x30db,0x30b0-0x30bf mem 0xe0000400-0xe00007ff irq 19 at device 31.2 on pci0 ahci0: AHCI v1.10 with 4 3Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 pci0: at device 31.3 (no driver attached) acpi_button0: on acpi0 atrtc0: port 0x70-0x71 irq 8 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: console (115200,n,8,1) uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fd0: <1440-KB 3.5" drive> on fdc0 drive 0 pmtimer0 on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x100> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] ppc0: parallel port not found. est0: on cpu0 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 828082806000828 device_attach: est0 attach returned 6 p4tcc0: on cpu0 est1: on cpu1 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 828082806000828 device_attach: est1 attach returned 6 p4tcc1: on cpu1 Timecounters tick every 1.000 msec usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 12Mbps Full Speed USB v1.0 usbus3: 12Mbps Full Speed USB v1.0 ugen0.1: at usbus0 uhub0: on usbus0 Expensive timeout(9) function: 0xc0a5e8e0(0xc5b235c0) 0.011070122 s ugen1.1: at usbus1 uhub1: on usbus1 ugen2.1: at usbus2 uhub2: on usbus2 ugen3.1: at usbus3 uhub3: on usbus3 usbus4: 480Mbps High Speed USB v2.0 ugen4.1: at usbus4 uhub4: on usbus4 Expensive timeout(9) function: 0xc0526f30(0xc5b3f500) 0.020004218 s ada0 at ahcich0 bus 0 scbus1 target 0 lun 0 ada0: ATA-7 SATA 2.x device ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 238475MB (488397168 512 byte sectors: 16H 63S/T 16383C) ada0: Previously was known as ad4 cd0 at ata0 bus 0 scbus0 target 0 lun 0 cd0: SMP: AP CPU #1 Launched! Removable CD-ROM SCSI-0 device cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present Timecounter "TSC-low" frequency 16625356 Hz quality 1000 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. uhub1: 2 ports with 2 removable, self powered uhub0: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 uhub4: 8 ports with 8 removable, self powered Trying to mount root from nfs: []... NFS ROOT: 192.168.5.1:/zoo/leopard3 em0: link state changed to UP Interface em0 IP-Address 192.168.5.109 Broadcast 192.168.5.255 Setting hostuuid: 64a93476-27b1-3004-c299-0030488ed164. Setting hostid: 0x7acc54f1. Starting ddb. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: /dev/ada0d: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ada0d: clean, 111893202 free (1474 frags, 13986466 blocks, 0.0% fragmentation) mount_nfs: can't update /var/db/mounttab for 192.168.5.1:/zoo/leopard3 Mounting local file systems:. Setting hostname: leopard3.netperf.freebsd.org. Starting Network: lo0 em0 em1. lo0: flags=8049 metric 0 mtu 16384 options=3 inet 127.0.0.1 netmask 0xff000000 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x8 nd6 options=21 em0: flags=8843 metric 0 mtu 1500 options=219b ether 00:30:48:8e:d0:68 inet 192.168.5.109 netmask 0xffffff00 broadcast 192.168.5.255 inet6 fe80::230:48ff:fe8e:d068%em0 prefixlen 64 tentative scopeid 0x1 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active em1: flags=8802 metric 0 mtu 1500 options=219b ether 00:30:48:8e:d0:69 media: Ethernet autoselect status: no carrier Starting devd. Starting Network: em1. em1: flags=8802 metric 0 mtu 1500 options=219b ether 00:30:48:8e:d0:69 media: Ethernet autoselect status: no carrier Starting Network: usbus0. Starting Network: usbus1. Starting Network: usbus2. Starting Network: usbus3. Starting Network: usbus4. add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 Creating and/or trimming log files. Starting syslogd. No core dumps found. ELF ldconfig path: /lib /usr/lib /usr/lib/compat a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout Setting date via ntp. 19 Nov 10:18:07 ntpdate[1388]: step time server 192.168.5.1 offset -1.035197 sec Starting rpcbind. NFS access cache time=60 Clearing /tmp (X related). Starting statd. Updating motd:. Configuring syscons: blanktime. Starting sshd. Starting cron. Local package initialization: watchdogd. Starting inetd. Sat Nov 19 10:18:08 UTC 2011 FreeBSD/i386 (leopard3.netperf.freebsd.org) (ttyu0) login: root Nov 19 10:18:10 leopard3 login: ROOT LOGIN (root) ON ttyu0 FreeBSD 10.0-CURRENT (PHO) #0: Sat Nov 19 10:24:38 CET 2011 Welcome to FreeBSD! Before seeking technical support, please use the following resources: o Security advisories and updated errata information for all releases are at http://www.FreeBSD.org/releases/ - always consult the ERRATA section for your release first as it's updated frequently. o The Handbook and FAQ documents are at http://www.FreeBSD.org/ and, along with the mailing lists, can be searched by going to http://www.FreeBSD.org/search/. If the doc distribution has been installed, they're also available formatted in /usr/share/doc. If you still have a question or problem, please take the output of `uname -a', along with any relevant error messages, and email it as a question to the questions@FreeBSD.org mailing list. If you are unfamiliar with FreeBSD's directory layout, please refer to the hier(7) manual page. If you are not familiar with manual pages, type `man man'. You may also use sysinstall(8) to re-enter the installation and configuration utility. Edit /etc/motd to change this login announcement. leopard3# ~test/keepalive.sh Sat Nov 19 11:18:16 CET 2011 11:18AM up 25 secs, 1 user, load averages: 2.20, 0.63, 0.23 Nov 19 11:18:35 leopard3 su: test to root on /dev/pts/0 lock order reversal: 1st 0xc6448c94 ufs (ufs) @ /mnt/pho/src/sys/kern/vfs_subr.c:2167 2nd 0xc65d51b4 newnfs (newnfs) @ /mnt/pho/src/sys/kern/vfs_syscalls.c:1658 KDB: stack backtrace: db_trace_self_wrapper(c0f26830,2f6e7265,5f736676,63737973,736c6c61,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a5814b,c0f2a30f,c1202328,67a,edda39dc,...) at kdb_backtrace+0x2a _witness_debugger(c0f2a30f,c65d51b4,c0f133c3,c594b2d0,c0f337e7,...) at _witness_debugger+0x25 witness_checkorder(c65d51b4,9,c0f337e7,67a,c65d5220,...) at witness_checkorder+0x86f __lockmgr_args(c65d51b4,80400,c65d5220,0,0,...) at __lockmgr_args+0x834 nfs_lock1(edda3afc,c5ebeb50,0,80400,c65d515c,...) at nfs_lock1+0xcd VOP_LOCK1_APV(c1045440,edda3afc,429,c107d300,c65d515c,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c65d515c,80400,c0f337e7,67a,0,...) at _vn_lock+0x78 kern_linkat(c65e95c0,ffffff9c,ffffff9c,2840e0c0,2843e280,...) at kern_linkat+0x239 kern_link(c65e95c0,2840e0c0,2843e280,0,edda3d1c,...) at kern_link+0x3e sys_link(c65e95c0,edda3cec,9,0,246,...) at sys_link+0x29 syscall(edda3d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (9, FreeBSD ELF32, sys_link), eip = 0x80d0ead, esp = 0x28295f9c, ebp = 0x28295ff4 --- lock order reversal: 1st 0xc6568b38 ufs (ufs) @ /mnt/pho/src/sys/kern/vfs_subr.c:2167 2nd 0xdf318040 bufwait (bufwait) @ /mnt/pho/src/sys/ufs/ffs/ffs_vnops.c:260 3rd 0xc6633058 ufs (ufs) @ /mnt/pho/src/sys/kern/vfs_subr.c:2167 KDB: stack backtrace: db_trace_self_wrapper(c0f26830,75735f73,632e7262,3631323a,6f000a37,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a5814b,c0f2a328,c1202338,877,edda93c8,...) at kdb_backtrace+0x2a _witness_debugger(c0f2a328,c6633058,c0f18dac,c594b610,c0f321e4,...) at _witness_debugger+0x25 witness_checkorder(c6633058,9,c0f321e4,877,0,...) at witness_checkorder+0x86f __lockmgr_args(c6633058,80100,c66330c4,0,0,...) at __lockmgr_args+0x834 ffs_lock(edda94ec,c0c8286f,c19cbd88,80100,c6633000,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c106e280,edda94ec,c59441e0,c107d300,c6633000,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c6633000,80100,c0f321e4,877,4,...) at _vn_lock+0x78 vget(c6633000,80100,c65e9000,50,0,...) at vget+0xbb vfs_hash_get(c5ebfb50,11401,80000,c65e9000,edda9630,...) at vfs_hash_get+0xed ffs_vgetf(c5ebfb50,11401,80000,edda9630,1,...) at ffs_vgetf+0x49 softdep_sync_buf(c6568ae0,df317fe0,1,106,0,...) at softdep_sync_buf+0x4a3 ffs_syncvnode(c6568ae0,1,c1202328,c6568bd8,c5947370,...) at ffs_syncvnode+0x24c ffs_truncate(c6568ae0,400,0,880,c5987880,...) at ffs_truncate+0x817 ufs_direnter(c6568ae0,c6632570,edda999c,edda9b80,0,...) at ufs_direnter+0x971 ufs_makeinode(edda9b80,c6568b38,edda9ad0,c0aa853a,c6568b38,...) at ufs_makeinode+0x635 ufs_symlink(edda9c10,c0f7a1e2,0,0,edda9ba8,...) at ufs_symlink+0x33 VOP_SYMLINK_APV(c106e280,edda9c10,edda9b80,edda9ba8,c62a9c00,...) at VOP_SYMLINK_APV+0xc5 kern_symlinkat(c65e9000,28499100,ffffff9c,284a0d00,0,...) at kern_symlinkat+0x2f2 kern_symlink(c65e9000,28499100,284a0d00,0,edda9d1c,...) at kern_symlink+0x2e sys_symlink(c65e9000,edda9cec,c,0,246,...) at sys_symlink+0x29 syscall(edda9d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (57, FreeBSD ELF32, sys_symlink), eip = 0x80d0ead, esp = 0x28295f9c, ebp = 0x28295ff4 --- lock order reversal: 1st 0xc6568b38 ufs (ufs) @ /mnt/pho/src/sys/kern/vfs_subr.c:2167 2nd 0xc646eb38 devfs (devfs) @ /mnt/pho/src/sys/kern/vfs_syscalls.c:1658 KDB: stack backtrace: db_trace_self_wrapper(c0f26830,66762f6e,79735f73,6c616373,632e736c,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a5814b,c0f2a30f,c1202328,67a,eddaf9dc,...) at kdb_backtrace+0x2a _witness_debugger(c0f2a30f,c646eb38,c0f109ef,c594aff8,c0f337e7,...) at _witness_debugger+0x25 witness_checkorder(c646eb38,9,c0f337e7,67a,c646eba4,...) at witness_checkorder+0x86f __lockmgr_args(c646eb38,80400,c646eba4,0,0,...) at __lockmgr_args+0x834 vop_stdlock(eddafafc,c5ebf000,0,80400,c646eae0,...) at vop_stdlock+0x65 VOP_LOCK1_APV(c10442e0,eddafafc,429,c107d300,c646eae0,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c646eae0,80400,c0f337e7,67a,0,...) at _vn_lock+0x78 kern_linkat(c65e68a0,ffffff9c,ffffff9c,284a21e0,284a3200,...) at kern_linkat+0x239 kern_link(c65e68a0,284a21e0,284a3200,0,eddafd1c,...) at kern_link+0x3e sys_link(c65e68a0,eddafcec,c0f73002,c0f2af7b,246,...) at sys_link+0x29 syscall(eddafd28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (9, FreeBSD ELF32, sys_link), eip = 0x80d0ead, esp = 0x28295f9c, ebp = 0x28295ff4 --- lock order reversal: 1st 0xdf318040 bufwait (bufwait) @ /mnt/pho/src/sys/kern/vfs_bio.c:2658 2nd 0xc5b77800 dirhash (dirhash) @ /mnt/pho/src/sys/ufs/ufs/ufs_dirhash.c:284 KDB: stack backtrace: db_trace_self_wrapper(c0f26830,7366752f,7366752f,7366752f,7269645f,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a5814b,c0f2a30f,c1202348,11c,df7c3ab4,...) at kdb_backtrace+0x2a _witness_debugger(c0f2a30f,c5b77800,c0f5a6de,c594b7b0,c0f5a35f,...) at _witness_debugger+0x25 witness_checkorder(c5b77800,9,c0f5a35f,11c,0,...) at witness_checkorder+0x86f _sx_xlock(c5b77800,0,c0f5a35f,11c,e0444774,...) at _sx_xlock+0x85 ufsdirhash_acquire(0,e,c63a4000,df317fe0,e0444774,...) at ufsdirhash_acquire+0x48 ufsdirhash_remove(c6397074,e0444774,774,df7c3b44,df7c3b40,...) at ufsdirhash_remove+0x14 ufs_dirremove(c6568ae0,c66886cc,500800c,1,c6568ae0,...) at ufs_dirremove+0x143 ufs_rmdir(df7c3c20,c0f7a227,0,0,c6682ae0,...) at ufs_rmdir+0xf1 VOP_RMDIR_APV(c106e280,df7c3c20,2,df7c3bf8,28443140,...) at VOP_RMDIR_APV+0xf6 kern_rmdirat(c62a68a0,ffffff9c,28443140,0,df7c3c6c,...) at kern_rmdirat+0x1db kern_rmdir(c62a68a0,28443140,0,df7c3d1c,c0d58824,...) at kern_rmdir+0x27 sys_rmdir(c62a68a0,df7c3cec,9,0,246,...) at sys_rmdir+0x22 syscall(df7c3d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (137, FreeBSD ELF32, sys_rmdir), eip = 0x80d0ead, esp = 0x28295f9c, ebp = 0x28295ff4 --- lock order reversal: 1st 0xc63d5310 newnfs (newnfs) @ /mnt/pho/src/sys/kern/vfs_mount.c:1209 2nd 0xc6569724 devfs (devfs) @ /mnt/pho/src/sys/ufs/ffs/ffs_softdep.c:1834 KDB: stack backtrace: db_trace_self_wrapper(c0f26830,7366662f,7366662f,666f735f,70656474,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a5814b,c0f2a30f,c1202430,72a,edd34a10,...) at kdb_backtrace+0x2a _witness_debugger(c0f2a30f,c6569724,c0f109ef,c594aff8,c0f56860,...) at _witness_debugger+0x25 witness_checkorder(c6569724,9,c0f56860,72a,c6569790,...) at witness_checkorder+0x86f __lockmgr_args(c6569724,80400,c6569790,0,0,...) at __lockmgr_args+0x834 vop_stdlock(edd34b30,4,c0f20cab,80400,c65696cc,...) at vop_stdlock+0x65 VOP_LOCK1_APV(c10442e0,edd34b30,c1236380,c107d300,c65696cc,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c65696cc,80400,c0f56860,72a,c5ebfb50,...) at _vn_lock+0x78 softdep_flushworklist(c5ebfb50,edd34bdc,c62df5c0,5dc,c0f321e4,...) at softdep_flushworklist+0x47 ffs_sync(c5ebfb50,1,c0f31837,4f2,c5825000,...) at ffs_sync+0x2fd dounmount(c5ebfb50,8000000,c62df5c0,483,a2189f44,...) at dounmount+0x3f8 sys_unmount(c62df5c0,edd34cec,c,c,206,...) at sys_unmount+0x331 syscall(edd34d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280dcb3f, esp = 0xbfbfe1fc, ebp = 0xbfbfe2c8 --- Sat Nov 19 11:28:16 CET 2011 11:28AM up 10 mins, 2 users, load averages: 0.02, 0.33, 0.29 Sat Nov 19 11:38:16 CET 2011 11:38AM up 20 mins, 2 users, load averages: 0.00, 0.03, 0.13 Sat Nov 19 11:48:16 CET 2011 11:48AM up 30 mins, 2 users, load averages: 0.00, 0.00, 0.04 Sat Nov 19 11:58:16 CET 2011 11:58AM up 40 mins, 2 users, load averages: 0.00, 0.00, 0.00 Sat Nov 19 12:08:16 CET 2011 12:08PM up 50 mins, 2 users, load averages: 0.00, 0.00, 0.00 Sat Nov 19 12:18:16 CET 2011 12:18PM up 1 hr, 2 users, load averages: 0.00, 0.00, 0.00 Sat Nov 19 12:28:16 CET 2011 12:28PM up 1:10, 2 users, load averages: 0.00, 0.00, 0.00 Sat Nov 19 12:38:16 CET 2011 12:38PM up 1:20, 2 users, load averages: 0.79, 0.40, 0.16 Sat Nov 19 12:48:17 CET 2011 12:48PM up 1:30, 2 users, load averages: 0.01, 0.27, 0.25 Sat Nov 19 12:58:17 CET 2011 12:58PM up 1:40, 2 users, load averages: 0.73, 0.72, 0.51 Sat Nov 19 13:08:17 CET 2011 1:08PM up 1:50, 2 users, load averages: 0.64, 0.72, 0.61 Sat Nov 19 13:18:17 CET 2011 1:18PM up 2 hrs, 2 users, load averages: 0.86, 0.82, 0.71 Sat Nov 19 13:28:17 CET 2011 1:28PM up 2:10, 2 users, load averages: 0.87, 0.83, 0.76 Sat Nov 19 13:38:17 CET 2011 1:38PM up 2:20, 2 users, load averages: 0.80, 0.81, 0.77 Sat Nov 19 13:48:17 CET 2011 1:48PM up 2:30, 2 users, load averages: 0.73, 0.81, 0.79 Nov 19 12:53:33 leopard3 kernel: pid 1366 (syslogd), uid 0 inumber 3284 on /var: filesystem full Nov 19 12:54:04 leopard3 last message repeated 31 times Nov 19 12:54:59 leopard3 last message repeated 55 times Nov 19 12:55:00 leopard3 kernel: pid 10062 (dd), uid 2 inumber 2197 on /var: filesystem full Nov 19 12:55:00 leopard3 kernel: pid 1366 (syslogd), uid 0 inumber 3284 on /var: filesystem full Nov 19 12:55:31 leopard3 last message repeated 31 times Nov 19 12:57:32 leopard3 last message repeated 121 times Sat Nov 19 13:58:17 CET 2011 1:58PM up 2:40, 2 users, load averages: 0.74, 0.81, 0.78 Nov 19 13:00:00 leopard3 last message repeated 147 times Nov 19 13:00:00 leopard3 kernel: pid 4008 (dd), uid 2 inumber 2190 on /var: filesystem full Nov 19 13:00:00 leopard3 sendmail[3924]: pAJD00Ym003924: low on space (SMTP-DAEMON needs 233 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 Nov 19 13:00:00 leopard3 sendmail[3924]: pAJD00Yn003924: low on space (SMTP-DAEMON needs 0 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 Nov 19 13:00:00 leopard3 sendmail[3924]: pAJD00Ym003924: SYSERR(root): returntosender: cannot select queue for root Nov 19 13:00:00 leopard3 sendmail[3924]: pAJD00Yo003924: low on space (SMTP-DAEMON needs 0 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 Nov 19 13:00:00 leopard3 sendmail[3924]: pAJD00Ym003924: SYSERR(root): returntosender: cannot select queue for postmaster Nov 19 13:00:00 leopard3 sendmail[3924]: pAJD00Ym003924: low on space (SMTP-DAEMON needs 286 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 Nov 19 13:00:00 leopard3 sendmail[3924]: pAJD00Ym003924: low on space (SMTP-DAEMON needs 286 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 Nov 19 13:00:00 leopard3 sendmail[3924]: pAJD00Ym003924: low on space (SMTP-DAEMON needs 286 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 Nov 19 13:00:00 leopard3 sendmail[3924]: pAJD00Ym003924: low on space (SMTP-DAEMON needs 286 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 Nov 19 13:00:00 leopard3 sendmail[3924]: pAJD00Ym003924: low on space (SMTP-DAEMON needs 286 bytes + 100 blocks in /var/spool/clientmqueue), max avail: 0 Nov 19 13:00:00 leopard3 sendmail[3924]: pAJD00Ym003924: SYSERR(root): putbody: write error: No space left on device Nov 19 13:00:00 leopard3 sendmail[3924]: pAJD00Ym003924: SYSERR(root): Error writing control file qfpAJD00Ym003924: No space left on device Nov 19 13:00:01 leopard3 kernel: pid 1366 (syslogd), uid 0 inumber 3284 on /var: filesystem full Nov 19 13:00:32 leopard3 last message repeated 31 times Nov 19 13:02:33 leopard3 last message repeated 121 times Sat Nov 19 14:08:17 CET 2011 2:08PM up 2:50, 2 users, load averages: 1.20, 0.99, 0.88 Nov 19 13:11:00 leopard3 last message repeated 507 times Nov 19 13:11:00 leopard3 kernel: pid 31144 (dd), uid 2 inumber 2191 on /var: filesystem full : : Nov 21 03:44:00 leopard3 kernel: pid 61178 (dd), uid 2 inumber 2190 on /var: filesystem full Nov 21 03:44:01 leopard3 kernel: pid 1366 (syslogd), uid 0 inumber 3302 on /var: filesystem full Mon Nov 21 04:44:28 CET 2011 4:44AM up 1 day, 17:27, 2 users, load averages: 1.01, 1.03, 1.02 Nov 21 03:44:32 leopard3 last message repeated 31 times Nov 21 03:46:33 leopard3 last message repeated 121 times panic: vn_lock 0xc65b5828: zero hold count cpuid = 0 KDB: enter: panic [ thread pid 96470 tid 100171 ] Stopped at kdb_enter+0x3a: movl $0,kdb_why db> db> run pho db:0:pho> bt Tracing pid 96470 tid 100171 td 0xc65e6000 kdb_enter(c0f22780,c0f22780,c0f33a7a,edda6c08,0,...) at kdb_enter+0x3a panic(c0f33a7a,c65b5828,c65e6000,0,edda6cec,...) at panic+0x134 _vn_lock(c65b5828,200400,c0f337e7,b27,edda6c5c,...) at _vn_lock+0x4f sys_fchflags(c65e6000,edda6cec,c0f73002,c0f2aebd,246,...) at sys_fchflags+0xe4 syscall(edda6d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (35, FreeBSD ELF32, sys_fchflags), eip = 0x80d0ead, esp = 0x28295f9c, ebp = 0x28295ff4 --- db:0:bt> show allpcpu Current CPU: 0 cpuid = 0 dynamic pcpu = 0x7a7580 curthread = 0xc65e6000: pid 96470 "iknowthis" curpcb = 0xedda6d80 fpcurthread = none idlethread = 0xc59aa8a0: tid 100003 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: cpuid = 1 dynamic pcpu = 0x4591580 curthread = 0xc59aa5c0: pid 11 "idle: cpu1" curpcb = 0xc561cd80 fpcurthread = none idlethread = 0xc59aa5c0: tid 100004 "idle: cpu1" APIC ID = 1 currentldt = 0x50 spin locks held: db:0:allpcpu> show alllocks Process 94133 (iknowthis) thread 0xc683f8a0 (101265) exclusive rw tidhash (tidhash) r = 0 (0xc10b8a28) locked @ /mnt/pho/src/sys/kern/kern_thread.c:1015 exclusive sx allproc (allproc) r = 0 (0xc10b8410) locked @ /mnt/pho/src/sys/kern/kern_fork.c:898 Process 1767 (sshd) thread 0xc62de000 (100113) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xc655370c) locked @ /mnt/pho/src/sys/kern/uipc_sockbuf.c:148 db:0:alllocks> show lockedvnods Locked vnodes db:0:lockedvnods> show mount 0xc5ebeb50 192.168.5.1:/zoo/leopard3 on / (nfs) 0xc5ebf000 devfs on /dev (devfs) 0xc638f5a8 /dev/ada0d on /work (ufs) 0xc638f2d4 /dev/md0 on /var (ufs) 0xc638f000 /dev/md1 on /tmp (ufs) 0xc68bf87c /dev/md5a on /mnt (ufs) More info: show mount db:0:mount> ps pid ppid pgrp uid state wmesg wchan cmd 96471 94133 86381 65534 R+ iknowthis 96470 94133 86381 65534 R+ CPU 0 iknowthis 94135 94133 86381 0 S+ (threaded) iknowthis 101737 S select 0xc62b0a24 iknowthis 102249 S pause 0xc654e5e0 iknowthis 94133 66358 86381 65534 S+ wait 0xc698284c iknowthis 16022 83287 83287 0 S+ nanslp 0xc10b9b24 sleep 66377 0 0 0 DL mdwait 0xc5ea4800 [md5] 66358 86381 86381 0 S+ wait 0xc62dc000 sh 83287 1759 83287 0 S+ wait 0xc65d284c sh 86381 1778 86381 0 S+ wait 0xc5b9eb10 sh 1778 1774 1778 0 S+ wait 0xc62dc84c sh 1774 1773 1774 0 S+ pause 0xc654eb68 csh 1773 1772 1773 1001 S+ wait 0xc654f000 su 1772 1771 1772 1001 Ss+ wait 0xc62282c4 sh 1771 1767 1767 1001 S select 0xc62b04a4 sshd 1767 1663 1767 0 Ss sbwait 0xc6553738 sshd 1759 1758 1759 0 S+ pause 0xc62dcb68 csh 1758 1 1758 0 Ss+ wait 0xc639f588 login 1757 1 1757 0 Ss+ ttyin 0xc5a4c670 getty 1756 1 1756 0 Ss+ ttyin 0xc5a4c870 getty 1755 1 1755 0 Ss+ ttyin 0xc5a4ca70 getty 1754 1 1754 0 Ss+ ttyin 0xc5a4cc70 getty 1753 1 1753 0 Ss+ ttyin 0xc5a4ce70 getty 1752 1 1752 0 Ss+ ttyin 0xc5c35070 getty 1751 1 1751 0 Ss+ ttyin 0xc5c35270 getty 1750 1 1750 0 Ss+ ttyin 0xc5c35470 getty 1712 1 1712 0 Ss select 0xc62b0564 inetd 1684 1 1684 0 Ss nanslp 0xc10b9b24 watchdogd 1674 1 1674 0 Ss nanslp 0xc10b9b24 cron 1663 1 1663 0 Ss select 0xc62b05a4 sshd 1524 1 1524 0 Ss select 0xc5afb0a4 rpc.statd 1451 0 0 0 DL mdwait 0xc5eac800 [md1] 1396 1 1396 0 Ss select 0xc62b0664 rpcbind 1366 1 1366 0 Ss select 0xc62b06e4 syslogd 1187 1 1187 0 Ss select 0xc62b0724 devd 100 0 0 0 DL mdwait 0xc5ead000 [md0] 17 0 0 0 DL sdflush 0xc12363c0 [softdepflush] 16 0 0 0 DL syncer 0xc122fcf4 [syncer] 9 0 0 0 DL vlruwt 0xc5e6eb10 [vnlru] 8 0 0 0 DL psleep 0xc122f988 [bufdaemon] 7 0 0 0 DL pgzero 0xc123825c [pagezero] 6 0 0 0 DL psleep 0xc1237e48 [vmdaemon] 5 0 0 0 DL psleep 0xc1237e10 [pagedaemon] 4 0 0 0 DL ccb_scan 0xc107e7d4 [xpt_thrd] 3 0 0 0 DL waiting_ 0xc12317f8 [sctp_iterator] 2 0 0 0 DL - 0xc5b7703c [fdc0] 15 0 0 0 DL (threaded) [usb] 100049 D - 0xc5bead34 [usbus4] 100048 D - 0xc5bead04 [usbus4] 100047 D - 0xc5beacd4 [usbus4] 100046 D - 0xc5beaca4 [usbus4] 100045 D - 0xc5bd3dac [usbus3] 100044 D - 0xc5bd3d7c [usbus3] 100043 D - 0xc5bd3d4c [usbus3] 100042 D - 0xc5bd3d1c [usbus3] 100040 D - 0xc5bc0dac [usbus2] 100039 D - 0xc5bc0d7c [usbus2] 100038 D - 0xc5bc0d4c [usbus2] 100037 D - 0xc5bc0d1c [usbus2] 100035 D - 0xc5ba8dac [usbus1] 100034 D - 0xc5ba8d7c [usbus1] 100033 D - 0xc5ba8d4c [usbus1] 100032 D - 0xc5ba8d1c [usbus1] 100030 D - 0xc5b8fdac [usbus0] 100029 D - 0xc5b8fd7c [usbus0] 100028 D - 0xc5b8fd4c [usbus0] 100027 D - 0xc5b8fd1c [usbus0] 14 0 0 0 DL - 0xc10b8924 [yarrow] 13 0 0 0 DL (threaded) [geom] 100011 D - 0xc10b6528 [g_down] 100010 D - 0xc10b6524 [g_up] 100009 D - 0xc10b651c [g_event] 12 0 0 0 RL (threaded) [intr] 100054 I [irq1: atkbd0] 100052 I [swi0: uart uart] 100051 I [irq258: ahci0] 100050 I [irq14: ata0] 100041 I [irq16: uhci3] 100036 I [irq18: uhci2] 100031 I [irq19: uhci1] 100026 I [irq23: uhci0 ehci0] 100023 I [swi6: Giant taskq] 100021 I [swi5: +] 100019 I [swi2: cambio] 100014 I [swi6: task queue] 100008 I [swi3: vm] 100007 I [swi4: clock] 100006 RunQ [swi4: clock] 100005 I [swi1: netisr 0] 11 0 0 0 RL (threaded) [idle] 100004 Run CPU 1 [idle: cpu1] 100003 CanRun [idle: cpu0] 1 0 1 0 SLs wait 0xc59a8b10 [init] 10 0 0 0 DL audit_wo 0xc1235b00 [audit] 0 0 0 0 DLs (threaded) [kernel] 100065 D - 0xc5e7a780 [mca taskq] 100055 D - 0xc10b8924 [deadlkres] 100025 D - 0xc5b758c0 [em1 taskq] 100024 D - 0xc5b5e980 [em0 taskq] 100022 D - 0xc5ab7400 [thread taskq] 100020 D - 0xc5ab7480 [kqueue taskq] 100018 D - 0xc5ab74c0 [ffs_trim taskq] 100017 D - 0xc5ab7500 [acpi_task_2] 100016 D - 0xc5ab7500 [acpi_task_1] 100015 D - 0xc5ab7500 [acpi_task_0] 100012 D - 0xc5977c40 [firmware taskq] 100000 D sched 0xc10b6600 [swapper] db:0:ps> allt Tracing command iknowthis pid 96471 tid 102216 td 0xc6558000 fork_trampoline() at fork_trampoline Tracing command iknowthis pid 96470 tid 100171 td 0xc65e6000 kdb_enter(c0f22780,c0f22780,c0f33a7a,edda6c08,0,...) at kdb_enter+0x3a panic(c0f33a7a,c65b5828,c65e6000,0,edda6cec,...) at panic+0x134 _vn_lock(c65b5828,200400,c0f337e7,b27,edda6c5c,...) at _vn_lock+0x4f sys_fchflags(c65e6000,edda6cec,c0f73002,c0f2aebd,246,...) at sys_fchflags+0xe4 syscall(edda6d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (35, FreeBSD ELF32, sys_fchflags), eip = 0x80d0ead, esp = 0x28295f9c, ebp = 0x28295ff4 --- Tracing command iknowthis pid 94135 tid 101737 td 0xc68c72e0 sched_switch(c68c72e0,0,104,191,bdb66288,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c68c72e0,0,c0f28200,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a0f2ea,c62b0a10,0,c0f20cab,c68c72e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c62b0a24,0,eddd7a60,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c62b0a24,c62b0a10,c0f2a85e,6a4,c68138c0,...) at _cv_wait_sig+0x243 seltdwait(c68138c0,58,c6703900,c68c72e0,0,...) at seltdwait+0xa2 kern_select(c68c72e0,5,bf9fef00,bf9fee80,bf9fee00,0,20,c654e588,1) at kern_select+0x587 sys_select(c68c72e0,eddd7cec,0,0,246,...) at sys_select+0x66 syscall(eddd7d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x811532b, esp = 0xbf9fed9c, ebp = 0xbf9fedc8 --- Tracing command iknowthis pid 94135 tid 102249 td 0xc86838a0 sched_switch(c86838a0,0,104,191,bdba8d48,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,74,...) at mi_switch+0x1eb sleepq_switch(c86838a0,0,c0f28200,1a7,74,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f28200,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c654e5e0,74,c0da910e,100,0,...) at sleepq_wait_sig+0x17 _sleep(c654e5e0,c654e610,174,c0da910e,0,...) at _sleep+0x35c kern_sigsuspend(c86838a0,0,0,0,0,...) at kern_sigsuspend+0xba sys_sigsuspend(c86838a0,edde6cec,c0f73002,c0f2af32,246,...) at sys_sigsuspend+0x4d syscall(edde6d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x80d0d27, esp = 0xbfbfe80c, ebp = 0xbfbfe838 --- Tracing command iknowthis pid 94133 tid 101265 td 0xc683f8a0 sched_switch(c683f8a0,0,104,191,6110f9d0,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c683f8a0,0,c0f28200,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f28200,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c698284c,6c,c0f2b043,100,0,...) at sleepq_wait_sig+0x17 _sleep(c698284c,c69828d4,16c,c0f2b043,0,...) at _sleep+0x35c kern_wait(c683f8a0,178d6,eddf8c60,0,0,...) at kern_wait+0x626 sys_wait4(c683f8a0,eddf8cec,c0f73002,c0f2b5a2,282,...) at sys_wait4+0x3b syscall(eddf8d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x80dafef, esp = 0xbfbfe71c, ebp = 0xbfbfe738 --- Tracing command sleep pid 16022 tid 102230 td 0xc683e2e0 sched_switch(c683e2e0,0,104,191,fc98720,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c683e2e0,0,c0f28200,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(927c1,c0a5e8e0,c683e2e0,1,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c10b9b24,6c,c0f23cff,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c10b9b24,0,16c,c0f23cff,927c1,...) at _sleep+0x328 kern_nanosleep(c683e2e0,df7b2c50,df7b2c58,258,0,...) at kern_nanosleep+0xc1 sys_nanosleep(c683e2e0,df7b2cec,c0f73002,c0f2b898,246,...) at sys_nanosleep+0x6f syscall(df7b2d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x281660b7, esp = 0xbfbfec2c, ebp = 0xbfbfec78 --- Tracing command md5 pid 66377 tid 101143 td 0xc5e798a0 sched_switch(c5e798a0,0,104,191,5ab7e510,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c5e798a0,0,c0f28200,268,0,...) at sleepq_switch+0x162 sleepq_wait(c5ea4800,5c,c0ec00ea,0,0,...) at sleepq_wait+0x63 _sleep(c5ea4800,c5ea4820,25c,c0ec00ea,0,...) at _sleep+0x372 md_kthread(c5ea4800,df7c3d28,c0f1cf70,3db,c62dd2c4,...) at md_kthread+0x11a fork_exit(c0737810,c5ea4800,df7c3d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf7c3d60, ebp = 0 --- Tracing command sh pid 66358 tid 102231 td 0xc8684000 sched_switch(c8684000,0,104,191,bbd365e0,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c8684000,0,c0f28200,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f28200,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c62dc000,6c,c0f2b043,100,0,...) at sleepq_wait_sig+0x17 _sleep(c62dc000,c62dc088,16c,c0f2b043,0,...) at _sleep+0x35c kern_wait(c8684000,ffffffff,df7ccc60,2,0,...) at kern_wait+0x626 sys_wait4(c8684000,df7cccec,c0f73002,c0f2b022,212,...) at sys_wait4+0x3b syscall(df7ccd28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x28161ffb, esp = 0xbfbfe97c, ebp = 0xbfbfe998 --- Tracing command sh pid 83287 tid 101288 td 0xc685f000 sched_switch(c685f000,0,104,191,f933d98,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c685f000,0,c0f28200,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f28200,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c65d284c,6c,c0f2b043,100,0,...) at sleepq_wait_sig+0x17 _sleep(c65d284c,c65d28d4,16c,c0f2b043,0,...) at _sleep+0x35c kern_wait(c685f000,ffffffff,df7a9c60,2,0,...) at kern_wait+0x626 sys_wait4(c685f000,df7a9cec,c0f73002,c0f2b022,212,...) at sys_wait4+0x3b syscall(df7a9d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x28161ffb, esp = 0xbfbfe9dc, ebp = 0xbfbfe9f8 --- Tracing command sh pid 86381 tid 100074 td 0xc5e785c0 sched_switch(c5e785c0,0,104,191,58b13148,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c5e785c0,0,c0f28200,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f28200,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5b9eb10,6c,c0f2b043,100,0,...) at sleepq_wait_sig+0x17 _sleep(c5b9eb10,c5b9eb98,16c,c0f2b043,0,...) at _sleep+0x35c kern_wait(c5e785c0,ffffffff,df799c60,2,0,...) at kern_wait+0x626 sys_wait4(c5e785c0,df799cec,c0f73002,c0f2b022,212,...) at sys_wait4+0x3b syscall(df799d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x28161ffb, esp = 0xbfbfe84c, ebp = 0xbfbfe868 --- Tracing command sh pid 1778 tid 100091 td 0xc62a52e0 sched_switch(c62a52e0,0,104,191,a1b87e0,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c62a52e0,0,c0f28200,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f28200,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c62dc84c,6c,c0f2b043,100,0,...) at sleepq_wait_sig+0x17 _sleep(c62dc84c,c62dc8d4,16c,c0f2b043,0,...) at _sleep+0x35c kern_wait(c62a52e0,ffffffff,df7e5c60,2,0,...) at kern_wait+0x626 sys_wait4(c62a52e0,df7e5cec,c0f73002,c0f2b022,212,...) at sys_wait4+0x3b syscall(df7e5d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x28161ffb, esp = 0xbfbfe98c, ebp = 0xbfbfe9a8 --- Tracing command csh pid 1774 tid 100118 td 0xc65585c0 sched_switch(c65585c0,0,104,191,a36263a0,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,74,...) at mi_switch+0x1eb sleepq_switch(c65585c0,0,c0f28200,1a7,74,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f28200,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c654eb68,74,c0da910e,100,0,...) at sleepq_wait_sig+0x17 _sleep(c654eb68,c654eb98,174,c0da910e,0,...) at _sleep+0x35c kern_sigsuspend(c65585c0,0,0,0,0,...) at kern_sigsuspend+0xba sys_sigsuspend(c65585c0,edd7fcec,c0f73002,c0f2af32,202,...) at sys_sigsuspend+0x4d syscall(edd7fd28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x2817d8fb, esp = 0xbfbfe72c, ebp = 0xbfbfe798 --- Tracing command su pid 1773 tid 100117 td 0xc65588a0 sched_switch(c65588a0,0,104,191,48f3b7f8,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c65588a0,0,c0f28200,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f28200,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c654f000,6c,c0f2b043,100,0,...) at sleepq_wait_sig+0x17 _sleep(c654f000,c654f088,16c,c0f2b043,0,...) at _sleep+0x35c kern_wait(c65588a0,6ee,edd7cc60,2,0,...) at kern_wait+0x626 sys_wait4(c65588a0,edd7ccec,c,c,212,...) at sys_wait4+0x3b syscall(edd7cd28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2812cffb, esp = 0xbfbfe76c, ebp = 0xbfbfe788 --- Tracing command sh pid 1772 tid 100070 td 0xc5e80000 sched_switch(c5e80000,0,104,191,42a49788,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c5e80000,0,c0f28200,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f28200,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c62282c4,6c,c0f2b043,100,0,...) at sleepq_wait_sig+0x17 _sleep(c62282c4,c622834c,16c,c0f2b043,0,...) at _sleep+0x35c kern_wait(c5e80000,ffffffff,df789c60,2,0,...) at kern_wait+0x626 sys_wait4(c5e80000,df789cec,c,c,212,...) at sys_wait4+0x3b syscall(df789d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x28161ffb, esp = 0xbfbfeaac, ebp = 0xbfbfeac8 --- Tracing command sshd pid 1771 tid 100116 td 0xc6558b80 sched_switch(c6558b80,0,104,191,badb7ee8,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c6558b80,0,c0f28200,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a0f2ea,c62b0490,0,c0f20cab,c6558b80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c62b04a4,0,edd79a60,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c62b04a4,c62b0490,c0f2a85e,6a4,c6236818,...) at _cv_wait_sig+0x243 seltdwait(c6236818,58,c5986000,c6558b80,c6558c30,...) at seltdwait+0xa2 kern_select(c6558b80,a,288060b8,288060cc,0,0,20,c654f2c4,0) at kern_select+0x587 sys_select(c6558b80,edd79cec,edd79cc4,c0d60068,206,...) at sys_select+0x66 syscall(edd79d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284b0d43, esp = 0xbfbfe19c, ebp = 0xbfbfe1e8 --- Tracing command sshd pid 1767 tid 100113 td 0xc62de000 sched_switch(c62de000,0,104,191,5751d718,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,68,...) at mi_switch+0x1eb sleepq_switch(c62de000,0,c0f28200,1a7,68,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f28200,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6553738,68,c0f2e771,100,0,...) at sleepq_wait_sig+0x17 _sleep(c6553738,c65536f8,168,c0f2e771,0) at _sleep+0x35c sbwait(c65536d4,4,c0f2e849,5ee,c65536f8,...) at sbwait+0x76 soreceive_generic(c6553680,0,edd70c44,0,0,...) at soreceive_generic+0x320 soreceive(c6553680,0,edd70c44,0,0,0) at soreceive+0x38 soo_read(c6446540,edd70c44,c5987900,0,c62de000,...) at soo_read+0x4e dofileread(edd70c44,ffffffff,ffffffff,0,edd70c20,...) at dofileread+0x9e kern_readv(c62de000,5,edd70c44,edd70c64,1,...) at kern_readv+0x68 sys_read(c62de000,edd70cec,c0f73002,c0f131c1,286,...) at sys_read+0x4f syscall(edd70d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x284b0dc3, esp = 0xbfbfdd6c, ebp = 0xbfbfdda8 --- Tracing command csh pid 1759 tid 100090 td 0xc62a55c0 sched_switch(c62a55c0,0,104,191,bb022ac0,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,74,...) at mi_switch+0x1eb sleepq_switch(c62a55c0,0,c0f28200,1a7,74,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f28200,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c62dcb68,74,c0da910e,100,0,...) at sleepq_wait_sig+0x17 _sleep(c62dcb68,c62dcb98,174,c0da910e,0,...) at _sleep+0x35c kern_sigsuspend(c62a55c0,0,0,0,0,...) at kern_sigsuspend+0xba sys_sigsuspend(c62a55c0,df7e2cec,c0f73002,c0f2af32,202,...) at sys_sigsuspend+0x4d syscall(df7e2d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x2817d8fb, esp = 0xbfbfe7ec, ebp = 0xbfbfe858 --- Tracing command login pid 1758 tid 100105 td 0xc62df8a0 sched_switch(c62df8a0,0,104,191,3543a740,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c62df8a0,0,c0f28200,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f28200,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c639f588,6c,c0f2b043,100,0,...) at sleepq_wait_sig+0x17 _sleep(c639f588,c639f610,16c,c0f2b043,0,...) at _sleep+0x35c kern_wait(c62df8a0,6df,edd31c60,0,0,...) at kern_wait+0x626 sys_wait4(c62df8a0,edd31cec,c,c,212,...) at sys_wait4+0x3b syscall(edd31d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x28125ffb, esp = 0xbfbfecec, ebp = 0xbfbfed08 --- Tracing command getty pid 1757 tid 100077 td 0xc5e7b000 sched_switch(c5e7b000,0,104,191,6c539060,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5e7b000,0,c0f28200,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f28200,160,0,c5e7b000,c5e7b000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5a4c670,0,c0f2cf60,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5a4c670,c10b81f0,c0f2c920,527,0,...) at _cv_wait_sig+0x243 tty_wait(c5a4c600,c5a4c670,df7a5c44,1,0,...) at tty_wait+0x71 ttydisc_read(c5a4c600,df7a5c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c5c2ae00,df7a5c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c62365e8,df7a5c44,c5986e80,0,c5e7b000,...) at devfs_read_f+0x7e dofileread(df7a5c44,ffffffff,ffffffff,0,df7a5c20,...) at dofileread+0x9e kern_readv(c5e7b000,0,df7a5c44,df7a5c64,1,...) at kern_readv+0x68 sys_read(c5e7b000,df7a5cec,c0f73002,c0f44288,246,...) at sys_read+0x4f syscall(df7a5d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1756 tid 100075 td 0xc5e7b5c0 sched_switch(c5e7b5c0,0,104,191,6bb16658,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5e7b5c0,0,c0f28200,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f28200,160,0,c5e7b5c0,c5e7b5c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5a4c870,0,c0f2cf60,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5a4c870,c10b81f0,c0f2c920,527,0,...) at _cv_wait_sig+0x243 tty_wait(c5a4c800,c5a4c870,df79dc44,1,0,...) at tty_wait+0x71 ttydisc_read(c5a4c800,df79dc44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c5c53000,df79dc44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c6236658,df79dc44,c5986e80,0,c5e7b5c0,...) at devfs_read_f+0x7e dofileread(df79dc44,ffffffff,ffffffff,0,df79dc20,...) at dofileread+0x9e kern_readv(c5e7b5c0,0,df79dc44,df79dc64,1,...) at kern_readv+0x68 sys_read(c5e7b5c0,df79dcec,c5e7b5c0,df79dd80,246,...) at sys_read+0x4f syscall(df79dd28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1755 tid 100099 td 0xc62e4b80 sched_switch(c62e4b80,0,104,191,6c376178,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c62e4b80,0,c0f28200,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f28200,160,0,c62e4b80,c62e4b80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5a4ca70,0,c0f2cf60,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5a4ca70,c10b81f0,c0f2c920,527,0,...) at _cv_wait_sig+0x243 tty_wait(c5a4ca00,c5a4ca70,edd1fc44,1,0,...) at tty_wait+0x71 ttydisc_read(c5a4ca00,edd1fc44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c5c53100,edd1fc44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c6446ce8,edd1fc44,c5986e80,0,c62e4b80,...) at devfs_read_f+0x7e dofileread(edd1fc44,ffffffff,ffffffff,0,edd1fc20,...) at dofileread+0x9e kern_readv(c62e4b80,0,edd1fc44,edd1fc64,1,...) at kern_readv+0x68 sys_read(c62e4b80,edd1fcec,c0f73002,c0f44288,246,...) at sys_read+0x4f syscall(edd1fd28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1754 tid 100092 td 0xc62a5000 sched_switch(c62a5000,0,104,191,6b3f9920,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c62a5000,0,c0f28200,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f28200,160,0,c62a5000,c62a5000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5a4cc70,0,c0f2cf60,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5a4cc70,c10b81f0,c0f2c920,527,0,...) at _cv_wait_sig+0x243 tty_wait(c5a4cc00,c5a4cc70,df7e8c44,1,0,...) at tty_wait+0x71 ttydisc_read(c5a4cc00,df7e8c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c5c53200,df7e8c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c6235690,df7e8c44,c5986e80,0,c62a5000,...) at devfs_read_f+0x7e dofileread(df7e8c44,ffffffff,ffffffff,0,df7e8c20,...) at dofileread+0x9e kern_readv(c62a5000,0,df7e8c44,df7e8c64,1,...) at kern_readv+0x68 sys_read(c62a5000,df7e8cec,c0f73002,df7e8d80,246,...) at sys_read+0x4f syscall(df7e8d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1753 tid 100108 td 0xc62df000 sched_switch(c62df000,0,104,191,6c368320,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c62df000,0,c0f28200,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f28200,160,0,c62df000,c62df000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5a4ce70,0,c0f2cf60,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5a4ce70,c10b81f0,c0f2c920,527,0,...) at _cv_wait_sig+0x243 tty_wait(c5a4ce00,c5a4ce70,edd3ac44,1,0,...) at tty_wait+0x71 ttydisc_read(c5a4ce00,edd3ac44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c5c53300,edd3ac44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c64461c0,edd3ac44,c5986e80,0,c62df000,...) at devfs_read_f+0x7e dofileread(edd3ac44,ffffffff,ffffffff,0,edd3ac20,...) at dofileread+0x9e kern_readv(c62df000,0,edd3ac44,edd3ac64,1,...) at kern_readv+0x68 sys_read(c62df000,edd3acec,c0f73002,c0f44288,246,...) at sys_read+0x4f syscall(edd3ad28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1752 tid 100104 td 0xc62dfb80 sched_switch(c62dfb80,0,104,191,6baeea40,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c62dfb80,0,c0f28200,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f28200,160,0,c62dfb80,c62dfb80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5c35070,0,c0f2cf60,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5c35070,c10b81f0,c0f2c920,527,0,...) at _cv_wait_sig+0x243 tty_wait(c5c35000,c5c35070,edd2ec44,1,0,...) at tty_wait+0x71 ttydisc_read(c5c35000,edd2ec44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c5c53400,edd2ec44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c64464d0,edd2ec44,c5986e80,0,c62dfb80,...) at devfs_read_f+0x7e dofileread(edd2ec44,ffffffff,ffffffff,0,edd2ec20,...) at dofileread+0x9e kern_readv(c62dfb80,0,edd2ec44,edd2ec64,1,...) at kern_readv+0x68 sys_read(c62dfb80,edd2ecec,c0f73002,c0f44288,246,...) at sys_read+0x4f syscall(edd2ed28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1751 tid 100072 td 0xc5e7b8a0 sched_switch(c5e7b8a0,0,104,191,6bfc8108,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5e7b8a0,0,c0f28200,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f28200,160,0,c5e7b8a0,c5e7b8a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5c35270,0,c0f2cf60,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5c35270,c10b81f0,c0f2c920,527,0,...) at _cv_wait_sig+0x243 tty_wait(c5c35200,c5c35270,df791c44,1,0,...) at tty_wait+0x71 ttydisc_read(c5c35200,df791c44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c5ac8300,df791c44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c62355e8,df791c44,c5986e80,0,c5e7b8a0,...) at devfs_read_f+0x7e dofileread(df791c44,ffffffff,ffffffff,0,df791c20,...) at dofileread+0x9e kern_readv(c5e7b8a0,0,df791c44,df791c64,1,...) at kern_readv+0x68 sys_read(c5e7b8a0,df791cec,c0f73002,c0f44288,246,...) at sys_read+0x4f syscall(df791d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1750 tid 100093 td 0xc62a4b80 sched_switch(c62a4b80,0,104,191,6c5201f8,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c62a4b80,0,c0f28200,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f28200,160,0,c62a4b80,c62a4b80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5c35470,0,c0f2cf60,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5c35470,c10b81f0,c0f2c920,527,0,...) at _cv_wait_sig+0x243 tty_wait(c5c35400,c5c35470,df7ebc44,1,0,...) at tty_wait+0x71 ttydisc_read(c5c35400,df7ebc44,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c5ac8400,df7ebc44,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c64462a0,df7ebc44,c5986e80,0,c62a4b80,...) at devfs_read_f+0x7e dofileread(df7ebc44,ffffffff,ffffffff,0,df7ebc20,...) at dofileread+0x9e kern_readv(c62a4b80,0,df7ebc44,df7ebc64,1,...) at kern_readv+0x68 sys_read(c62a4b80,df7ebcec,c0f73002,c0f44288,246,...) at sys_read+0x4f syscall(df7ebd28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command inetd pid 1712 tid 100095 td 0xc62a45c0 sched_switch(c62a45c0,0,104,191,e82af200,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c62a45c0,0,c0f28200,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a0f2ea,c62b0550,0,c0f20cab,c62a45c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c62b0564,0,df7f2a60,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c62b0564,c62b0550,c0f2a85e,6a4,c6236230,...) at _cv_wait_sig+0x243 seltdwait(c6236230,58,c5986e80,c62a45c0,df7f2ad0,...) at seltdwait+0xa2 kern_select(c62a45c0,8,bfbfdfa0,0,0,0,20,c62dab10,4) at kern_select+0x587 sys_select(c62a45c0,df7f2cec,c0f73002,c0f2b08f,202,...) at sys_select+0x66 syscall(df7f2d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281a8d43, esp = 0xbfbfdf5c, ebp = 0xbfbfede8 --- Tracing command watchdogd pid 1684 tid 100069 td 0xc5e802e0 sched_switch(c5e802e0,0,104,191,c6e77550,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c5e802e0,0,c0f28200,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(ea61,c0a5e8e0,c5e802e0,0,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c10b9b24,6c,c0f23cff,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c10b9b24,0,16c,c0f23cff,ea61,...) at _sleep+0x328 kern_nanosleep(c5e802e0,df785c50,df785c58,3c,0,...) at kern_nanosleep+0xc1 sys_nanosleep(c5e802e0,df785cec,c0f73002,c0f44288,206,...) at sys_nanosleep+0x6f syscall(df785d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x281920b7, esp = 0xbfbfecbc, ebp = 0xbfbfece8 --- Tracing command cron pid 1674 tid 100097 td 0xc62a4000 sched_switch(c62a4000,0,104,191,dc27c8f8,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c62a4000,0,c0f28200,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(ea61,c0a5e8e0,c62a4000,1,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c10b9b24,6c,c0f23cff,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c10b9b24,0,16c,c0f23cff,ea61,...) at _sleep+0x328 kern_nanosleep(c62a4000,df7f8c50,df7f8c58,3c,0,...) at kern_nanosleep+0xc1 sys_nanosleep(c62a4000,df7f8cec,c0f73002,c0f2b4f0,206,...) at sys_nanosleep+0x6f syscall(df7f8d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x281850b7, esp = 0xbfbfec4c, ebp = 0xbfbfec78 --- Tracing command sshd pid 1663 tid 100101 td 0xc62e45c0 sched_switch(c62e45c0,0,104,191,50a2f4c8,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c62e45c0,0,c0f28200,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a0f2ea,c62b0590,0,c0f20cab,c62e45c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c62b05a4,0,edd25a60,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c62b05a4,c62b0590,c0f2a85e,6a4,c6247dc8,...) at _cv_wait_sig+0x243 seltdwait(c6247dc8,58,c5987100,c62e45c0,c0f5c677,...) at seltdwait+0xa2 kern_select(c62e45c0,7,2880c0b0,0,0,0,20,c63a02c4,a) at kern_select+0x587 sys_select(c62e45c0,edd25cec,c,c,202,...) at sys_select+0x66 syscall(edd25d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x284b0d43, esp = 0xbfbfe25c, ebp = 0xbfbfedf8 --- Tracing command rpc.statd pid 1524 tid 100096 td 0xc62a42e0 sched_switch(c62a42e0,0,104,191,b58d17a8,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c62a42e0,0,c0f28200,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(df7f5a28,c0a0f2ea,c5afb090,0,c62a42e0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c5afb0a4,0,df7f5a60,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c5afb0a4,c5afb090,7531,6a4,c62364d0,...) at _cv_timedwait_sig+0x252 seltdwait(df7f5c14,df7f5c1c,c5986e80,c62a42e0,df7f5ad0,...) at seltdwait+0x8a kern_select(c62a42e0,8,bfbfed14,0,0,df7f5c5c,20,1e,0) at kern_select+0x587 sys_select(c62a42e0,df7f5cec,c0f73002,c0f2b180,202,...) at sys_select+0x66 syscall(df7f5d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x2818cd43, esp = 0xbfbfec6c, ebp = 0xbfbfeda8 --- Tracing command md1 pid 1451 tid 100073 td 0xc5e788a0 sched_switch(c5e788a0,0,104,191,5acc0ba8,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c5e788a0,0,c0f28200,268,0,...) at sleepq_switch+0x162 sleepq_wait(c5eac800,5c,c0ec00ea,0,0,...) at sleepq_wait+0x63 _sleep(c5eac800,c5eac820,25c,c0ec00ea,0,...) at _sleep+0x372 md_kthread(c5eac800,df795d28,c0f1cf70,3db,c5e6e000,...) at md_kthread+0x11a fork_exit(c0737810,c5eac800,df795d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf795d60, ebp = 0 --- Tracing command rpcbind pid 1396 tid 100087 td 0xc62a6000 sched_switch(c62a6000,0,104,191,b6910d10,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c62a6000,0,c0f28200,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(df7d1a88,c0a0f2ea,c62b0650,0,c62a6000,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c62b0664,0,df7d1ac0,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c62b0664,c62b0650,7531,6a4,df7d1b78,...) at _cv_timedwait_sig+0x252 seltdwait(df7d1c48,df7d1c50,5a1,c62a6000,df7d1b48,...) at seltdwait+0x8a sys_poll(c62a6000,df7d1cec,c0f73002,c0f2b180,202,...) at sys_poll+0x300 syscall(df7d1d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x2813ff4b, esp = 0xbfbfcbac, ebp = 0xbfbfed78 --- Tracing command syslogd pid 1366 tid 100082 td 0xc5e792e0 sched_switch(c5e792e0,0,104,191,5b77e050,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5e792e0,0,c0f28200,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a0f2ea,c62b06d0,0,c0f20cab,c5e792e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c62b06e4,0,df7bba60,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c62b06e4,c62b06d0,c0f2a85e,6a4,c6236b98,...) at _cv_wait_sig+0x243 seltdwait(c6236b98,58,c5986e80,c5e792e0,c5e79390,...) at seltdwait+0xa2 kern_select(c5e792e0,9,284760ac,0,0,0,20,c627b2c4,0) at kern_select+0x587 sys_select(c5e792e0,df7bbcec,c0f73002,c0d86593,246,...) at sys_select+0x66 syscall(df7bbd28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28199d43, esp = 0xbfbfe29c, ebp = 0xbfbfede8 --- Tracing command devd pid 1187 tid 100088 td 0xc62a5b80 sched_switch(c62a5b80,0,104,191,60947228,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c62a5b80,0,c0f28200,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0a0f2ea,c62b0710,0,c0f20cab,c62a5b80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c62b0724,0,df7d5a60,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c62b0724,c62b0710,c0f2a85e,6a4,c6247000,...) at _cv_wait_sig+0x243 seltdwait(c6247000,58,c5986e80,c62a5b80,c1202d20,...) at seltdwait+0xa2 kern_select(c62a5b80,5,bfbfe960,0,0,0,20,c627a000,46) at kern_select+0x587 sys_select(c62a5b80,df7d5cec,c0f73002,c0d86593,206,...) at sys_select+0x66 syscall(df7d5d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x808ba43, esp = 0xbfbfe92c, ebp = 0xbfbfee08 --- Tracing command md0 pid 100 tid 100102 td 0xc62e42e0 sched_switch(c62e42e0,0,104,191,5b725a18,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c62e42e0,0,c0f28200,268,0,...) at sleepq_switch+0x162 sleepq_wait(c5ead000,5c,c0ec00ea,0,0,...) at sleepq_wait+0x63 _sleep(c5ead000,c5ead020,25c,c0ec00ea,0,...) at _sleep+0x372 md_kthread(c5ead000,edd28d28,c0f1cf70,3db,c63a0000,...) at md_kthread+0x11a fork_exit(c0737810,c5ead000,edd28d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xedd28d60, ebp = 0 --- Tracing command softdepflush pid 17 tid 100064 td 0xc5e79000 sched_switch(c5e79000,0,104,191,5b75c8a0,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,54,...) at mi_switch+0x1eb sleepq_switch(c5e79000,0,c0f28200,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c12363c0,54,c0f58da8,0,0,...) at sleepq_timedwait+0x6b _sleep(c12363c0,c1236380,54,c0f58da8,3e8,...) at _sleep+0x342 softdep_flush(0,df773d28,c0f1cf70,3db,c5e6e588,...) at softdep_flush+0x266 fork_exit(c0c5a7d0,0,df773d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf773d60, ebp = 0 --- Tracing command syncer pid 16 tid 100063 td 0xc5be52e0 sched_switch(c5be52e0,0,104,191,5e141008,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5be52e0,0,c0f28200,28b,c5be52e0,...) at sleepq_switch+0x162 sleepq_timedwait(c122fcf4,0,df770c78,1,0,...) at sleepq_timedwait+0x6b _cv_timedwait(c122fcf4,c122fce0,3e8,78b,4e20,...) at _cv_timedwait+0x252 sched_sync(0,df770d28,c0f1cf70,3db,c5e6e84c,...) at sched_sync+0x595 fork_exit(c0abba30,0,df770d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf770d60, ebp = 0 --- Tracing command vnlru pid 9 tid 100062 td 0xc5be55c0 sched_switch(c5be55c0,0,104,191,e5fbc470,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,60,...) at mi_switch+0x1eb sleepq_switch(c5be55c0,0,c0f28200,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c5e6eb10,60,c0f33154,0,0,...) at sleepq_timedwait+0x6b _sleep(c5e6eb10,c122fcb4,260,c0f33154,3e8,...) at _sleep+0x342 vnlru_proc(0,df76dd28,c0f1cf70,3db,c5e6eb10,...) at vnlru_proc+0xe7 fork_exit(c0abda90,0,df76dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf76dd60, ebp = 0 --- Tracing command bufdaemon pid 8 tid 100061 td 0xc5be58a0 sched_switch(c5be58a0,0,104,191,5b72d0b8,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,54,...) at mi_switch+0x1eb sleepq_switch(c5be58a0,0,c0f28200,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c122f988,54,c0f30067,0,0,...) at sleepq_timedwait+0x6b _sleep(c122f988,c122f990,54,c0f30067,3e8,...) at _sleep+0x342 buf_daemon(0,df76ad28,c0f1cf70,3db,c5e6f000,...) at buf_daemon+0x175 fork_exit(c0aa1f40,0,df76ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf76ad60, ebp = 0 --- Tracing command pagezero pid 7 tid 100060 td 0xc5be5b80 sched_switch(c5be5b80,0,104,191,47829200,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5be5b80,0,c0f28200,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c123825c,0,c0f5fa92,0,0,...) at sleepq_timedwait+0x6b _sleep(c123825c,c1236d00,0,c0f5fa92,493e0,...) at _sleep+0x342 vm_pagezero(0,df767d28,c0f1cf70,3db,c59a92c4,...) at vm_pagezero+0xdc fork_exit(c0ca0d00,0,df767d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf767d60, ebp = 0 --- Tracing command vmdaemon pid 6 tid 100059 td 0xc5c25000 sched_switch(c5c25000,0,104,191,6e175c78,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,74,...) at mi_switch+0x1eb sleepq_switch(c5c25000,0,c0f28200,268,0,...) at sleepq_switch+0x162 sleepq_wait(c1237e48,74,c0f30067,0,0,...) at sleepq_wait+0x63 _sleep(c1237e48,c1237e4c,74,c0f30067,0,...) at _sleep+0x372 vm_daemon(0,df764d28,c0f1cf70,3db,c59a9588,...) at vm_daemon+0x59 fork_exit(c0c9ac40,0,df764d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf764d60, ebp = 0 --- Tracing command pagedaemon pid 5 tid 100058 td 0xc5c252e0 sched_switch(c5c252e0,0,104,191,59308a10,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,54,...) at mi_switch+0x1eb sleepq_switch(c5c252e0,0,c0f28200,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c1237e10,54,c0f30067,0,0,...) at sleepq_timedwait+0x6b _sleep(c1237e10,c1236d00,54,c0f30067,1388,...) at _sleep+0x342 vm_pageout(0,df761d28,c0f1cf70,3db,c59a984c,...) at vm_pageout+0x2c7 fork_exit(c0c9bd20,0,df761d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf761d60, ebp = 0 --- Tracing command xpt_thrd pid 4 tid 100057 td 0xc5c255c0 sched_switch(c5c255c0,0,104,191,581218b0,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c5c255c0,0,c0f28200,268,0,...) at sleepq_switch+0x162 sleepq_wait(c107e7d4,5c,c0d80dad,0,0,...) at sleepq_wait+0x63 _sleep(c107e7d4,c107e818,5c,c0d80dad,0,...) at _sleep+0x372 xpt_scanner_thread(0,df75ed28,c0f1cf70,3db,c59a9b10,...) at xpt_scanner_thread+0x47 fork_exit(c048e4e0,0,df75ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf75ed60, ebp = 0 --- Tracing command sctp_iterator pid 3 tid 100056 td 0xc5c258a0 sched_switch(c5c258a0,0,104,191,69161e40,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5c258a0,0,c0f28200,268,0,...) at sleepq_switch+0x162 sleepq_wait(c12317f8,0,c0f4691f,0,0,...) at sleepq_wait+0x63 _sleep(c12317f8,c12317c0,0,c0f4691f,0,...) at _sleep+0x372 sctp_iterator_thread(0,df75bd28,c0f1cf70,3db,c5b9e000,...) at sctp_iterator_thread+0x5c fork_exit(c0b4f3c0,0,df75bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf75bd60, ebp = 0 --- Tracing command fdc0 pid 2 tid 100053 td 0xc5c272e0 sched_switch(c5c272e0,0,104,191,85b7c70,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c5c272e0,0,c0f28200,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c5b7703c,5c,c0f150fa,0,0,...) at sleepq_timedwait+0x6b _sleep(c5b7703c,c5b770f0,5c,c0f150fa,3e8,...) at _sleep+0x342 fdc_thread(c5b77000,df752d28,c0f1cf70,3db,c5b9e2c4,...) at fdc_thread+0x27d fork_exit(c0d1ffd0,c5b77000,df752d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf752d60, ebp = 0 --- Tracing command usb pid 15 tid 100049 td 0xc5be25c0 sched_switch(c5be25c0,0,104,191,3753afd0,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5be25c0,0,c0f28200,268,c5be25c0,...) at sleepq_switch+0x162 sleepq_wait(c5bead34,0,c571ccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5bead34,c5beadd4,c0efb411,74,c5bead3c,...) at _cv_wait+0x243 usb_process(c5bead2c,c571cd28,c0f1cf70,3db,c5b9e588,...) at usb_process+0x1a3 fork_exit(c08c2e70,c5bead2c,c571cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc571cd60, ebp = 0 --- Tracing command usb pid 15 tid 100048 td 0xc5be28a0 sched_switch(c5be28a0,0,104,191,7f3edac0,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5be28a0,0,c0f28200,268,c5be28a0,...) at sleepq_switch+0x162 sleepq_wait(c5bead04,0,c5719cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5bead04,c5beadd4,c0efb411,74,c5bead0c,...) at _cv_wait+0x243 usb_process(c5beacfc,c5719d28,c0f1cf70,3db,c5b9e588,...) at usb_process+0x1a3 fork_exit(c08c2e70,c5beacfc,c5719d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5719d60, ebp = 0 --- Tracing command usb pid 15 tid 100047 td 0xc5be2b80 sched_switch(c5be2b80,0,104,191,36d52f40,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5be2b80,0,c0f28200,268,c5be2b80,...) at sleepq_switch+0x162 sleepq_wait(c5beacd4,0,c5716cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5beacd4,c5beadd4,c0efb411,74,c5beacdc,...) at _cv_wait+0x243 usb_process(c5beaccc,c5716d28,c0f1cf70,3db,c5b9e588,...) at usb_process+0x1a3 fork_exit(c08c2e70,c5beaccc,c5716d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5716d60, ebp = 0 --- Tracing command usb pid 15 tid 100046 td 0xc5be4000 sched_switch(c5be4000,0,104,191,36d518f8,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5be4000,0,c0f28200,268,c5be4000,...) at sleepq_switch+0x162 sleepq_wait(c5beaca4,0,c5713cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5beaca4,c5beadd4,c0efb411,74,c5beacac,...) at _cv_wait+0x243 usb_process(c5beac9c,c5713d28,c0f1cf70,3db,c5b9e588,...) at usb_process+0x1a3 fork_exit(c08c2e70,c5beac9c,c5713d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5713d60, ebp = 0 --- Tracing command usb pid 15 tid 100045 td 0xc5be42e0 sched_switch(c5be42e0,0,104,191,36d4ff48,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5be42e0,0,c0f28200,268,c5be42e0,...) at sleepq_switch+0x162 sleepq_wait(c5bd3dac,0,c570dcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5bd3dac,c5bd3e4c,c0efb411,74,c5bd3db4,...) at _cv_wait+0x243 usb_process(c5bd3da4,c570dd28,c0f1cf70,3db,c5b9e588,...) at usb_process+0x1a3 fork_exit(c08c2e70,c5bd3da4,c570dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc570dd60, ebp = 0 --- Tracing command usb pid 15 tid 100044 td 0xc5be45c0 sched_switch(c5be45c0,0,104,191,df4350a0,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5be45c0,0,c0f28200,268,c5be45c0,...) at sleepq_switch+0x162 sleepq_wait(c5bd3d7c,0,c570acac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5bd3d7c,c5bd3e4c,c0efb411,74,c5bd3d84,...) at _cv_wait+0x243 usb_process(c5bd3d74,c570ad28,c0f1cf70,3db,c5b9e588,...) at usb_process+0x1a3 fork_exit(c08c2e70,c5bd3d74,c570ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc570ad60, ebp = 0 --- Tracing command usb pid 15 tid 100043 td 0xc5be48a0 sched_switch(c5be48a0,0,104,191,2fbdb1e8,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5be48a0,0,c0f28200,268,c5be48a0,...) at sleepq_switch+0x162 sleepq_wait(c5bd3d4c,0,c5707cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5bd3d4c,c5bd3e4c,c0efb411,74,c5bd3d54,...) at _cv_wait+0x243 usb_process(c5bd3d44,c5707d28,c0f1cf70,3db,c5b9e588,...) at usb_process+0x1a3 fork_exit(c08c2e70,c5bd3d44,c5707d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5707d60, ebp = 0 --- Tracing command usb pid 15 tid 100042 td 0xc5be4b80 sched_switch(c5be4b80,0,104,191,2fbd9cf0,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5be4b80,0,c0f28200,268,c5be4b80,...) at sleepq_switch+0x162 sleepq_wait(c5bd3d1c,0,c5704cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5bd3d1c,c5bd3e4c,c0efb411,74,c5bd3d24,...) at _cv_wait+0x243 usb_process(c5bd3d14,c5704d28,c0f1cf70,3db,c5b9e588,...) at usb_process+0x1a3 fork_exit(c08c2e70,c5bd3d14,c5704d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5704d60, ebp = 0 --- Tracing command usb pid 15 tid 100040 td 0xc5bb8000 sched_switch(c5bb8000,0,104,191,2fbd83f8,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bb8000,0,c0f28200,268,c5bb8000,...) at sleepq_switch+0x162 sleepq_wait(c5bc0dac,0,c56fdcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5bc0dac,c5bc0e4c,c0efb411,74,c5bc0db4,...) at _cv_wait+0x243 usb_process(c5bc0da4,c56fdd28,c0f1cf70,3db,c5b9e588,...) at usb_process+0x1a3 fork_exit(c08c2e70,c5bc0da4,c56fdd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56fdd60, ebp = 0 --- Tracing command usb pid 15 tid 100039 td 0xc5bb82e0 sched_switch(c5bb82e0,0,104,191,dea0ab98,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bb82e0,0,c0f28200,268,c5bb82e0,...) at sleepq_switch+0x162 sleepq_wait(c5bc0d7c,0,c56facac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5bc0d7c,c5bc0e4c,c0efb411,74,c5bc0d84,...) at _cv_wait+0x243 usb_process(c5bc0d74,c56fad28,c0f1cf70,3db,c5b9e588,...) at usb_process+0x1a3 fork_exit(c08c2e70,c5bc0d74,c56fad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56fad60, ebp = 0 --- Tracing command usb pid 15 tid 100038 td 0xc5bb85c0 sched_switch(c5bb85c0,0,104,191,2f42c0d8,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bb85c0,0,c0f28200,268,c5bb85c0,...) at sleepq_switch+0x162 sleepq_wait(c5bc0d4c,0,c56f7cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5bc0d4c,c5bc0e4c,c0efb411,74,c5bc0d54,...) at _cv_wait+0x243 usb_process(c5bc0d44,c56f7d28,c0f1cf70,3db,c5b9e588,...) at usb_process+0x1a3 fork_exit(c08c2e70,c5bc0d44,c56f7d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56f7d60, ebp = 0 --- Tracing command usb pid 15 tid 100037 td 0xc5bb88a0 sched_switch(c5bb88a0,0,104,191,2f42abf8,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bb88a0,0,c0f28200,268,c5bb88a0,...) at sleepq_switch+0x162 sleepq_wait(c5bc0d1c,0,c56f4cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5bc0d1c,c5bc0e4c,c0efb411,74,c5bc0d24,...) at _cv_wait+0x243 usb_process(c5bc0d14,c56f4d28,c0f1cf70,3db,c5b9e588,...) at usb_process+0x1a3 fork_exit(c08c2e70,c5bc0d14,c56f4d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56f4d60, ebp = 0 --- Tracing command usb pid 15 tid 100035 td 0xc5bba000 sched_switch(c5bba000,0,104,191,2f429640,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bba000,0,c0f28200,268,c5bba000,...) at sleepq_switch+0x162 sleepq_wait(c5ba8dac,0,c56edcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5ba8dac,c5ba8e4c,c0efb411,74,c5ba8db4,...) at _cv_wait+0x243 usb_process(c5ba8da4,c56edd28,c0f1cf70,3db,c5b9e588,...) at usb_process+0x1a3 fork_exit(c08c2e70,c5ba8da4,c56edd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56edd60, ebp = 0 --- Tracing command usb pid 15 tid 100034 td 0xc5bba2e0 sched_switch(c5bba2e0,0,104,191,dd5c7540,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bba2e0,0,c0f28200,268,c5bba2e0,...) at sleepq_switch+0x162 sleepq_wait(c5ba8d7c,0,c56eacac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5ba8d7c,c5ba8e4c,c0efb411,74,c5ba8d84,...) at _cv_wait+0x243 usb_process(c5ba8d74,c56ead28,c0f1cf70,3db,c5b9e588,...) at usb_process+0x1a3 fork_exit(c08c2e70,c5ba8d74,c56ead28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56ead60, ebp = 0 --- Tracing command usb pid 15 tid 100033 td 0xc5bba5c0 sched_switch(c5bba5c0,0,104,191,2ec8a2c8,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bba5c0,0,c0f28200,268,c5bba5c0,...) at sleepq_switch+0x162 sleepq_wait(c5ba8d4c,0,c56e7cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5ba8d4c,c5ba8e4c,c0efb411,74,c5ba8d54,...) at _cv_wait+0x243 usb_process(c5ba8d44,c56e7d28,c0f1cf70,3db,c5b9e588,...) at usb_process+0x1a3 fork_exit(c08c2e70,c5ba8d44,c56e7d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56e7d60, ebp = 0 --- Tracing command usb pid 15 tid 100032 td 0xc5bba8a0 sched_switch(c5bba8a0,0,104,191,2ec88d68,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bba8a0,0,c0f28200,268,c5bba8a0,...) at sleepq_switch+0x162 sleepq_wait(c5ba8d1c,0,c56e4cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5ba8d1c,c5ba8e4c,c0efb411,74,c5ba8d24,...) at _cv_wait+0x243 usb_process(c5ba8d14,c56e4d28,c0f1cf70,3db,c5b9e588,...) at usb_process+0x1a3 fork_exit(c08c2e70,c5ba8d14,c56e4d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56e4d60, ebp = 0 --- Tracing command usb pid 15 tid 100030 td 0xc5b232e0 sched_switch(c5b232e0,0,104,191,2ec874c8,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b232e0,0,c0f28200,268,c5b232e0,...) at sleepq_switch+0x162 sleepq_wait(c5b8fdac,0,c56ddcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5b8fdac,c5b8fe4c,c0efb411,74,c5b8fdb4,...) at _cv_wait+0x243 usb_process(c5b8fda4,c56ddd28,c0f1cf70,3db,c5b9e588,...) at usb_process+0x1a3 fork_exit(c08c2e70,c5b8fda4,c56ddd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56ddd60, ebp = 0 --- Tracing command usb pid 15 tid 100029 td 0xc5b235c0 sched_switch(c5b235c0,0,104,191,ddfea2e0,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b235c0,0,c0f28200,268,c5b235c0,...) at sleepq_switch+0x162 sleepq_wait(c5b8fd7c,0,c56dacac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5b8fd7c,c5b8fe4c,c0efb411,74,c5b8fd84,...) at _cv_wait+0x243 usb_process(c5b8fd74,c56dad28,c0f1cf70,3db,c5b9e588,...) at usb_process+0x1a3 fork_exit(c08c2e70,c5b8fd74,c56dad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56dad60, ebp = 0 --- Tracing command usb pid 15 tid 100028 td 0xc5b238a0 sched_switch(c5b238a0,0,104,191,2e4cd238,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b238a0,0,c0f28200,268,c5b238a0,...) at sleepq_switch+0x162 sleepq_wait(c5b8fd4c,0,c56d7cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5b8fd4c,c5b8fe4c,c0efb411,74,c5b8fd54,...) at _cv_wait+0x243 usb_process(c5b8fd44,c56d7d28,c0f1cf70,3db,c5b9e588,...) at usb_process+0x1a3 fork_exit(c08c2e70,c5b8fd44,c56d7d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56d7d60, ebp = 0 --- Tracing command usb pid 15 tid 100027 td 0xc5b23b80 sched_switch(c5b23b80,0,104,191,2e4cbda0,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b23b80,0,c0f28200,268,c5b23b80,...) at sleepq_switch+0x162 sleepq_wait(c5b8fd1c,0,c56d4cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5b8fd1c,c5b8fe4c,c0efb411,74,c5b8fd24,...) at _cv_wait+0x243 usb_process(c5b8fd14,c56d4d28,c0f1cf70,3db,c5b9e588,...) at usb_process+0x1a3 fork_exit(c08c2e70,c5b8fd14,c56d4d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56d4d60, ebp = 0 --- Tracing command yarrow pid 14 tid 100013 td 0xc59ab2e0 sched_switch(c59ab2e0,0,104,191,54ba6fa8,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c59ab2e0,0,c0f28200,28b,2,...) at sleepq_switch+0x162 sleepq_timedwait(c10b8924,0,c0f150fa,2,0,...) at sleepq_timedwait+0x6b _sleep(c10b8924,0,0,c0f150fa,64,...) at _sleep+0x342 pause(c0f150fa,64,c0ed85fe,111,0,...) at pause+0x47 random_kthread(0,c563ed28,c0f1cf70,3db,c59a8000,...) at random_kthread+0x1ef fork_exit(c07d3ca0,0,c563ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc563ed60, ebp = 0 --- Tracing command geom pid 13 tid 100011 td 0xc59ab8a0 sched_switch(c59ab8a0,0,104,191,5c5d1aa0,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c59ab8a0,0,c0f28200,268,0,...) at sleepq_switch+0x162 sleepq_wait(c10b6528,5c,c0f150fa,0,0,...) at sleepq_wait+0x63 _sleep(c10b6528,c10b6488,25c,c0f150fa,0,...) at _sleep+0x372 g_io_schedule_down(c59ab8a0,0,c0f16c99,6c,c5634d14,...) at g_io_schedule_down+0x5b g_down_procbody(0,c5634d28,c0f1cf70,3db,c59a82c4,...) at g_down_procbody+0x9d fork_exit(c09b3ea0,0,c5634d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5634d60, ebp = 0 --- Tracing command geom pid 13 tid 100010 td 0xc59abb80 sched_switch(c59abb80,0,104,191,5c631d70,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c59abb80,0,c0f28200,268,0,...) at sleepq_switch+0x162 sleepq_wait(c10b6524,5c,c0f150fa,0,0,...) at sleepq_wait+0x63 _sleep(c10b6524,c10b64a8,25c,c0f150fa,0,...) at _sleep+0x372 g_io_schedule_up(c59abb80,0,c0f16c99,5f,c5631d14,...) at g_io_schedule_up+0x134 g_up_procbody(0,c5631d28,c0f1cf70,3db,c59a82c4,...) at g_up_procbody+0x9d fork_exit(c09b3f40,0,c5631d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5631d60, ebp = 0 --- Tracing command geom pid 13 tid 100009 td 0xc59f1000 sched_switch(c59f1000,0,104,191,60925aa0,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c59f1000,0,c0f28200,268,0,...) at sleepq_switch+0x162 sleepq_wait(c10b651c,5c,c0f150fa,0,0,...) at sleepq_wait+0x63 _sleep(c10b651c,c10b644c,25c,c0f150fa,0,...) at _sleep+0x372 g_run_events(c10c6780,0,c0f16c99,79,c562ed14,...) at g_run_events+0x54b g_event_procbody(0,c562ed28,c0f1cf70,3db,c59a82c4,...) at g_event_procbody+0x90 fork_exit(c09b3fe0,0,c562ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc562ed60, ebp = 0 --- Tracing command intr pid 12 tid 100054 td 0xc5c27000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100052 td 0xc5c275c0 sched_switch(c5c275c0,0,109,191,db8d4ce0,...) at sched_switch+0x3bc mi_switch(109,0,c0f1d2ab,554,c5c275c0,...) at mi_switch+0x1eb ithread_loop(c59f0710,df745d28,c0f1cf70,3db,c59a8588,...) at ithread_loop+0x1fb fork_exit(c09f5db0,c59f0710,df745d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf745d60, ebp = 0 --- Tracing command intr pid 12 tid 100051 td 0xc5be2000 sched_switch(c5be2000,0,109,191,5c62ad78,...) at sched_switch+0x3bc mi_switch(109,0,c0f1d2ab,554,c5be2000,...) at mi_switch+0x1eb ithread_loop(c59f0720,c5746d28,c0f1cf70,3db,c59a8588,...) at ithread_loop+0x1fb fork_exit(c09f5db0,c59f0720,c5746d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5746d60, ebp = 0 --- Tracing command intr pid 12 tid 100050 td 0xc5be22e0 sched_switch(c5be22e0,0,109,191,c3c85ca8,...) at sched_switch+0x3bc mi_switch(109,0,c0f1d2ab,554,c5be22e0,...) at mi_switch+0x1eb ithread_loop(c59f0730,c5742d28,c0f1cf70,3db,c59a8588,...) at ithread_loop+0x1fb fork_exit(c09f5db0,c59f0730,c5742d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5742d60, ebp = 0 --- Tracing command intr pid 12 tid 100041 td 0xc5be5000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100036 td 0xc5bb8b80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100031 td 0xc5bbab80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100026 td 0xc5b26000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100023 td 0xc5b268a0 sched_switch(c5b268a0,0,109,191,53a4dac0,...) at sched_switch+0x3bc mi_switch(109,0,c0f1d2ab,554,c5b268a0,...) at mi_switch+0x1eb ithread_loop(c59f0780,c565cd28,c0f1cf70,3db,c59a8588,...) at ithread_loop+0x1fb fork_exit(c09f5db0,c59f0780,c565cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc565cd60, ebp = 0 --- Tracing command intr pid 12 tid 100021 td 0xc59f18a0 sched_switch(c59f18a0,0,109,191,8c28b870,...) at sched_switch+0x3bc mi_switch(109,0,c0f1d2ab,554,c59f18a0,...) at mi_switch+0x1eb ithread_loop(c59f0790,c5656d28,c0f1cf70,3db,c59a8588,...) at ithread_loop+0x1fb fork_exit(c09f5db0,c59f0790,c5656d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5656d60, ebp = 0 --- Tracing command intr pid 12 tid 100019 td 0xc5b22000 sched_switch(c5b22000,0,109,191,5c62e160,...) at sched_switch+0x3bc mi_switch(109,0,c0f1d2ab,554,c5b22000,...) at mi_switch+0x1eb ithread_loop(c59f07a0,c5650d28,c0f1cf70,3db,c59a8588,...) at ithread_loop+0x1fb fork_exit(c09f5db0,c59f07a0,c5650d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5650d60, ebp = 0 --- Tracing command intr pid 12 tid 100014 td 0xc5b23000 sched_switch(c5b23000,0,109,191,8c916768,...) at sched_switch+0x3bc mi_switch(109,0,c0f1d2ab,554,c5b23000,...) at mi_switch+0x1eb ithread_loop(c59f07b0,c5641d28,c0f1cf70,3db,c59a8588,...) at ithread_loop+0x1fb fork_exit(c09f5db0,c59f07b0,c5641d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5641d60, ebp = 0 --- Tracing command intr pid 12 tid 100008 td 0xc59f12e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100007 td 0xc59f15c0 sched_switch(c59f15c0,0,109,191,e28ef830,...) at sched_switch+0x3bc mi_switch(109,0,c0f1d2ab,554,c59f15c0,...) at mi_switch+0x1eb ithread_loop(c59f07d0,c5628d28,c0f1cf70,3db,c59a8588,...) at ithread_loop+0x1fb fork_exit(c09f5db0,c59f07d0,c5628d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5628d60, ebp = 0 --- Tracing command intr pid 12 tid 100006 td 0xc59aa000 sched_switch(c59aa000,0,109,191,5e5b2e18,...) at sched_switch+0x3bc mi_switch(109,0,c0f1d2ab,554,c59aa000,...) at mi_switch+0x1eb ithread_loop(c59f07e0,c5625d28,c0f1cf70,3db,c59a8588,...) at ithread_loop+0x1fb fork_exit(c09f5db0,c59f07e0,c5625d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5625d60, ebp = 0 --- Tracing command intr pid 12 tid 100005 td 0xc59aa2e0 sched_switch(c59aa2e0,0,109,191,9e080588,...) at sched_switch+0x3bc mi_switch(109,0,c0f1d2ab,554,c59aa2e0,...) at mi_switch+0x1eb ithread_loop(c59f07f0,c5622d28,c0f1cf70,3db,c59a8588,...) at ithread_loop+0x1fb fork_exit(c09f5db0,c59f07f0,c5622d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5622d60, ebp = 0 --- Tracing command idle pid 11 tid 100004 td 0xc59aa5c0 cpustop_handler(1,13,c561cc08,c0d58c26,c0a0eec4,...) at cpustop_handler+0x28 ipi_nmi_handler(c0a0eec4,c560fc80,4,c0f20cab,c59a884c,...) at ipi_nmi_handler+0x47 trap(c561cc14) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0a26480, esp = 0xc561cc54, ebp = 0xc561cc58 --- critical_enter(c10c6780,9,c0f24892,330,0,...) at critical_enter _mtx_lock_spin_flags(c10c6780,0,c0f24892,330,c10c6780,...) at _mtx_lock_spin_flags+0x10a tdq_lock_pair(c10d5274,c561ccc4,c0f24892,3c0,c59aa5c0,...) at tdq_lock_pair+0x76 sched_idletd(0,c561cd28,c0f1cf70,3db,c59a884c,...) at sched_idletd+0x127 fork_exit(c0a45430,0,c561cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc561cd60, ebp = 0 --- Tracing command idle pid 11 tid 100003 td 0xc59aa8a0 sched_switch(c59aa8a0,0,108,18c,61108968,...) at sched_switch+0x3bc mi_switch(108,0,c0f24892,a08,c59aa8a0,...) at mi_switch+0x1eb sched_idletd(0,c5619d28,c0f1cf70,3db,c59a884c,...) at sched_idletd+0x2bf fork_exit(c0a45430,0,c5619d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5619d60, ebp = 0 --- Tracing command init pid 1 tid 100002 td 0xc59aab80 sched_switch(c59aab80,0,104,191,bbd35ca0,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c59aab80,0,c0f28200,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f28200,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c59a8b10,6c,c0f2b043,100,0,...) at sleepq_wait_sig+0x17 _sleep(c59a8b10,c59a8b98,16c,c0f2b043,0,...) at _sleep+0x35c kern_wait(c59aab80,ffffffff,c5615c60,0,0,...) at kern_wait+0x626 sys_wait4(c59aab80,c5615cec,c0f73002,c0f2ae7a,286,...) at sys_wait4+0x3b syscall(c5615d28) at syscall+0x2a4 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x80551d3, esp = 0xbfbfe8bc, ebp = 0xbfbfe8d8 --- Tracing command audit pid 10 tid 100001 td 0xc59ab000 sched_switch(c59ab000,0,104,191,2e48b450,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c59ab000,0,c0f28200,268,c59ab000,...) at sleepq_switch+0x162 sleepq_wait(c1235b00,0,c5612c8c,1,0,...) at sleepq_wait+0x63 _cv_wait(c1235b00,c1235ae4,c0f5378f,194,0,...) at _cv_wait+0x243 audit_worker(0,c5612d28,c0f1cf70,3db,c59a9000,...) at audit_worker+0x84 fork_exit(c0c1b800,0,c5612d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5612d60, ebp = 0 --- Tracing command kernel pid 0 tid 100065 td 0xc5e80b80 sched_switch(c5e80b80,0,104,191,684c3ef8,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5e80b80,0,c0f28200,268,0,...) at sleepq_switch+0x162 sleepq_wait(c5e7a780,0,c0f150fa,0,0,...) at sleepq_wait+0x63 _sleep(c5e7a780,c5e7a798,0,c0f150fa,0,...) at _sleep+0x372 taskqueue_thread_loop(c1297e60,df776d28,c0f1cf70,3db,c10b6600,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a61fe0,c1297e60,df776d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf776d60, ebp = 0 --- Tracing command kernel pid 0 tid 100055 td 0xc5c25b80 sched_switch(c5c25b80,0,104,191,b0f82ab0,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5c25b80,0,c0f28200,28b,2,...) at sleepq_switch+0x162 sleepq_timedwait(c10b8924,0,c0f150fa,2,0,...) at sleepq_timedwait+0x6b _sleep(c10b8924,0,0,c0f150fa,bb8,...) at _sleep+0x342 pause(c0f150fa,bb8,124,122,c10b6688,...) at pause+0x47 deadlkres(0,df758d28,c0f1cf70,3db,c10b6600,...) at deadlkres+0x324 fork_exit(c09d5fd0,0,df758d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf758d60, ebp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xc5b262e0 sched_switch(c5b262e0,0,104,191,2e46ae70,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b262e0,0,c0f28200,268,c10b68d0,...) at sleepq_switch+0x162 sleepq_wait(c5b758c0,0,c0f234f9,c0f150fa,0,...) at sleepq_wait+0x63 msleep_spin(c5b758c0,c5b758d8,c0f150fa,0,c0f20cab,...) at msleep_spin+0x21d taskqueue_thread_loop(c5b7c568,c56cdd28,c0f1cf70,3db,c10b6600,...) at taskqueue_thread_loop+0x8f fork_exit(c0a61fe0,c5b7c568,c56cdd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56cdd60, ebp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xc5b265c0 sched_switch(c5b265c0,0,104,191,60d5b900,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b265c0,0,c0f28200,268,0,...) at sleepq_switch+0x162 sleepq_wait(c5b5e980,0,c0f234f9,c0f150fa,0,...) at sleepq_wait+0x63 msleep_spin(c5b5e980,c5b5e998,c0f150fa,0,c0f20cab,...) at msleep_spin+0x21d taskqueue_thread_loop(c5b67568,c56a2d28,c0f1cf70,3db,c10b6600,...) at taskqueue_thread_loop+0x8f fork_exit(c0a61fe0,c5b67568,c56a2d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56a2d60, ebp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xc5b26b80 sched_switch(c5b26b80,0,104,191,52ddaca8,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b26b80,0,c0f28200,268,0,...) at sleepq_switch+0x162 sleepq_wait(c5ab7400,0,c0f150fa,0,0,...) at sleepq_wait+0x63 _sleep(c5ab7400,c5ab7418,0,c0f150fa,0,...) at _sleep+0x372 taskqueue_thread_loop(c10d54ec,c5659d28,c0f1cf70,3db,c10b6600,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a61fe0,c10d54ec,c5659d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5659d60, ebp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xc59f1b80 sched_switch(c59f1b80,0,104,191,5813f3f0,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c59f1b80,0,c0f28200,268,0,...) at sleepq_switch+0x162 sleepq_wait(c5ab7480,0,c0f150fa,0,0,...) at sleepq_wait+0x63 _sleep(c5ab7480,c5ab7498,0,c0f150fa,0,...) at _sleep+0x372 taskqueue_thread_loop(c10b6f98,c5653d28,c0f1cf70,3db,c10b6600,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a61fe0,c10b6f98,c5653d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5653d60, ebp = 0 --- Tracing command kernel pid 0 tid 100018 td 0xc5b222e0 sched_switch(c5b222e0,0,104,191,5813e290,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b222e0,0,c0f28200,268,0,...) at sleepq_switch+0x162 sleepq_wait(c5ab74c0,0,c0f150fa,0,0,...) at sleepq_wait+0x63 _sleep(c5ab74c0,c5ab74d8,0,c0f150fa,0,...) at _sleep+0x372 taskqueue_thread_loop(c12360e4,c564dd28,c0f1cf70,3db,c10b6600,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a61fe0,c12360e4,c564dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc564dd60, ebp = 0 --- Tracing command kernel pid 0 tid 100017 td 0xc5b225c0 sched_switch(c5b225c0,0,104,191,5813d298,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b225c0,0,c0f28200,268,0,...) at sleepq_switch+0x162 sleepq_wait(c5ab7500,0,c0f234f9,c0f150fa,0,...) at sleepq_wait+0x63 msleep_spin(c5ab7500,c5ab7518,c0f150fa,0,c0f20cab,...) at msleep_spin+0x21d taskqueue_thread_loop(c10816b8,c564ad28,c0f1cf70,3db,c10b6600,...) at taskqueue_thread_loop+0x8f fork_exit(c0a61fe0,c10816b8,c564ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc564ad60, ebp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xc5b228a0 sched_switch(c5b228a0,0,104,191,5813c5d0,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b228a0,0,c0f28200,268,0,...) at sleepq_switch+0x162 sleepq_wait(c5ab7500,0,c0f234f9,c0f150fa,0,...) at sleepq_wait+0x63 msleep_spin(c5ab7500,c5ab7518,c0f150fa,0,c0f20cab,...) at msleep_spin+0x21d taskqueue_thread_loop(c10816b8,c5647d28,c0f1cf70,3db,c10b6600,...) at taskqueue_thread_loop+0x8f fork_exit(c0a61fe0,c10816b8,c5647d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5647d60, ebp = 0 --- Tracing command kernel pid 0 tid 100015 td 0xc5b22b80 sched_switch(c5b22b80,0,104,191,5813b5d8,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b22b80,0,c0f28200,268,0,...) at sleepq_switch+0x162 sleepq_wait(c5ab7500,0,c0f234f9,c0f150fa,0,...) at sleepq_wait+0x63 msleep_spin(c5ab7500,c5ab7518,c0f150fa,0,c0f20cab,...) at msleep_spin+0x21d taskqueue_thread_loop(c10816b8,c5644d28,c0f1cf70,3db,c10b6600,...) at taskqueue_thread_loop+0x8f fork_exit(c0a61fe0,c10816b8,c5644d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5644d60, ebp = 0 --- Tracing command kernel pid 0 tid 100012 td 0xc59ab5c0 sched_switch(c59ab5c0,0,104,191,9b098de8,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c59ab5c0,0,c0f28200,268,0,...) at sleepq_switch+0x162 sleepq_wait(c5977c40,0,c0f150fa,0,0,...) at sleepq_wait+0x63 _sleep(c5977c40,c5977c58,0,c0f150fa,0,...) at _sleep+0x372 taskqueue_thread_loop(c10d3f00,c5637d28,c0f1cf70,3db,c10b6600,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a61fe0,c10d3f00,c5637d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5637d60, ebp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xc10b68d0 sched_switch(c10b68d0,0,104,191,defc2050,...) at sched_switch+0x3bc mi_switch(104,0,c0f28200,1f3,54,...) at mi_switch+0x1eb sleepq_switch(c10b68d0,0,c0f28200,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c10b6600,54,c0f251ac,0,0,...) at sleepq_timedwait+0x6b _sleep(c10b6600,0,54,c0f251ac,2710,...) at _sleep+0x342 scheduler(0,181ec00,181ec00,181e000,1825000,...) at scheduler+0x240 mi_startup() at mi_startup+0xac begin() at begin+0x2c db:0:allt> call doadump Physical memory: 2026 MB Dumping 181 MB: 166 150 134 118 102 86 70 54 38 22 6 Dump complete = 0 db:0:doadump> reset (kgdb) bt #0 doadump (textdump=-304453620) at pcpu.h:244 #1 0xc04e3009 in db_fncall (dummy1=-1062913750, dummy2=0, dummy3=-1, dummy4=0xedda68a0 "´hÚí") at /mnt/pho/src/sys/ddb/db_command.c:572 #2 0xc04e343f in db_command (last_cmdp=0xc107f87c, cmd_table=0x0, dopager=0) at /mnt/pho/src/sys/ddb/db_command.c:448 #3 0xc04e34f4 in db_command_script (command=0xc1080788 "call doadump") at /mnt/pho/src/sys/ddb/db_command.c:519 #4 0xc04e7830 in db_script_exec (scriptname=0xc10800e0 "doadump", warnifnotfound=Variable "warnifnotfound" is not available. ) at /mnt/pho/src/sys/ddb/db_script.c:302 #5 0xc04e78c1 in db_run_cmd (addr=-1056398272, have_addr=0, count=-1067116647, modif=0xedda69d8 "") at /mnt/pho/src/sys/ddb/db_script.c:375 #6 0xc04e3401 in db_command (last_cmdp=0xc107f87c, cmd_table=0x0, dopager=1) at /mnt/pho/src/sys/ddb/db_command.c:448 #7 0xc04e355a in db_command_loop () at /mnt/pho/src/sys/ddb/db_command.c:501 #8 0xc04e554d in db_trap (type=3, code=0) at /mnt/pho/src/sys/ddb/db_main.c:229 #9 0xc0a53b23 in kdb_trap (type=3, code=0, tf=0xedda6b88) at /mnt/pho/src/sys/kern/subr_kdb.c:625 #10 0xc0d5925b in trap (frame=0xedda6b88) at /mnt/pho/src/sys/i386/i386/trap.c:722 #11 0xc0d41bec in calltrap () at /mnt/pho/src/sys/i386/i386/exception.s:168 #12 0xc0a5392a in kdb_enter (why=0xc0f22780 "panic", msg=0xc0f22780 "panic") at cpufunc.h:71 #13 0xc0a1edf4 in panic (fmt=0xc0f33a7a "vn_lock %p: zero hold count") at /mnt/pho/src/sys/kern/kern_shutdown.c:600 #14 0xc0ac9e4f in _vn_lock (vp=0xc65b5828, flags=2098176, file=0xc0f337e7 "/mnt/pho/src/sys/kern/vfs_syscalls.c", line=2855) at /mnt/pho/src/sys/kern/vfs_vnops.c:948 #15 0xc0ac86b4 in sys_fchflags (td=0xc65e6000, uap=0xedda6cec) at /mnt/pho/src/sys/kern/vfs_syscalls.c:2855 #16 0xc0d58824 in syscall (frame=0xedda6d28) at subr_syscall.c:131 #17 0xc0d41c51 in Xint0x80_syscall () at /mnt/pho/src/sys/i386/i386/exception.s:266 #18 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) f 15 #15 0xc0ac86b4 in sys_fchflags (td=0xc65e6000, uap=0xedda6cec) at /mnt/pho/src/sys/kern/vfs_syscalls.c:2855 2855 vn_lock(fp->f_vnode, LK_SHARED | LK_RETRY); (kgdb) l 2850 if ((error = getvnode(td->td_proc->p_fd, uap->fd, CAP_FCHFLAGS, 2851 &fp)) != 0) 2852 return (error); 2853 vfslocked = VFS_LOCK_GIANT(fp->f_vnode->v_mount); 2854 #ifdef AUDIT 2855 vn_lock(fp->f_vnode, LK_SHARED | LK_RETRY); 2856 AUDIT_ARG_VNODE1(fp->f_vnode); 2857 VOP_UNLOCK(fp->f_vnode, 0); 2858 #endif 2859 error = setfflags(td, fp->f_vnode, uap->flags); (kgdb) p *uap $1 = {fd_l_ = 0xedda6cec "\002", fd = 2, fd_r_ = 0xedda6cf0 "\003\002XA", flags_l_ = 0xedda6cf0 "\003\002XA", flags = 1096286723, flags_r_ = 0xedda6cf4 ""} (kgdb) p *fp $2 = {f_data = 0xc5990400, f_ops = 0xc104ec80, f_cred = 0xc67d1d80, f_vnode = 0xc65b5828, f_type = 0, f_vnread_flags = 0, f_flag = 65, f_count = 2, f_seqcount = 0, f_nextoff = 0, f_vnun = {fvn_cdevpriv = 0x0, fvn_advice = 0x0}, f_offset = 0, f_label = 0x0} (kgdb) p *fp->f_vnode $3 = {v_type = VCHR, v_tag = 0xc0f109ef "devfs", v_op = 0xc10442e0, v_data = 0xc5dd3b00, v_mount = 0xc5ebf000, v_nmntvnodes = {tqe_next = 0xc65c96cc, tqe_prev = 0xc65b5998}, v_un = {vu_mount = 0xc5990400, vu_socket = 0xc5990400, vu_cdev = 0xc5990400, vu_fifoinfo = 0xc5990400}, v_hashlist = {le_next = 0x0, le_prev = 0x0}, v_hash = 0, v_cache_src = {lh_first = 0x0}, v_cache_dst = {tqh_first = 0x0, tqh_last = 0xc65b5858}, v_cache_dd = 0x0, v_cstart = 0, v_lasta = 0, v_lastw = 0, v_clen = 0, v_lock = {lock_object = {lo_name = 0xc0f109ef "devfs", lo_flags = 91422720, lo_data = 0, lo_witness = 0xc594aff8}, lk_lock = 1, lk_exslpfail = 0, lk_timo = 51, lk_pri = 96, lk_stack = {depth = 13, pcs = {3231740782, 3232400469, 3235364261, 3232538232, 3230885022, 3235362965, 3232537063, 3232537387, 3232523852, 3232524981, 3232525040, 3235219492, 3235126353, 0, 0, 0, 0, 0}}}, v_interlock = {lock_object = {lo_name = 0xc0f2a509 "vnode interlock", lo_flags = 16973824, lo_data = 0, lo_witness = 0xc59441e0}, mtx_lock = 4}, v_vnlock = 0xc65b5880, v_holdcnt = 0, v_usecount = 0, v_iflag = 256, v_vflag = 8, v_writecount = 0, v_freelist = {tqe_next = 0xc6b0e414, tqe_prev = 0xc65b5a74}, v_bufobj = {bo_mtx = {lock_object = {lo_name = 0xc0f33129 "bufobj interlock", lo_flags = 16973824, lo_data = 0, lo_witness = 0xc5949a70}, mtx_lock = 4}, bo_clean = {bv_hd = {tqh_first = 0x0, tqh_last = 0xc65b5934}, bv_root = 0x0, bv_cnt = 0}, bo_dirty = {bv_hd = {tqh_first = 0x0, tqh_last = 0xc65b5944}, bv_root = 0x0, bv_cnt = 0}, bo_numoutput = 0, bo_flag = 0, bo_ops = 0xc105b540, bo_bsize = 512, bo_object = 0x0, bo_synclist = {le_next = 0x0, le_prev = 0x0}, bo_private = 0xc65b5828, __bo_vnode = 0xc65b5828}, v_pollinfo = 0x0, v_label = 0x0, v_lockf = 0x0} (kgdb) p *(struct cdev *)fp->f_data $4 = {__si_reserved = 0x0, si_flags = 5, si_atime = {tv_sec = 1321697871, tv_nsec = 549780000}, si_ctime = {tv_sec = 1321697871, tv_nsec = 549780000}, si_mtime = {tv_sec = 1321697871, tv_nsec = 549780000}, si_uid = 0, si_gid = 0, si_mode = 438, si_cred = 0x0, si_drv0 = 2, si_refcount = 2, si_list = {le_next = 0xc5990600, le_prev = 0xc104ed78}, si_clone = {le_next = 0x0, le_prev = 0x0}, si_children = { lh_first = 0xc5990300}, si_siblings = {le_next = 0x0, le_prev = 0x0}, si_parent = 0x0, si_name = 0xc5990478 "fd/2", si_drv1 = 0x0, si_drv2 = 0x0, si_devsw = 0xc104ed20, si_iosize_max = 65536, si_usecount = 0, si_threadcount = 0, __si_u = {__sid_snapdata = 0x0}, __si_namebuf = "fd/2", '\0' } (kgdb) f 14 #14 0xc0ac9e4f in _vn_lock (vp=0xc65b5828, flags=0x200400, file=0xc0f337e7 "/mnt/pho/src/sys/kern/vfs_syscalls.c", line=0xb27) at /mnt/pho/src/sys/kern/vfs_vnops.c:948 948 KASSERT(vp->v_holdcnt != 0, (kgdb) l 943 944 VNASSERT((flags & LK_TYPE_MASK) != 0, vp, 945 ("vn_lock called with no locktype.")); 946 do { 947 #ifdef DEBUG_VFS_LOCKS 948 KASSERT(vp->v_holdcnt != 0, 949 ("vn_lock %p: zero hold count", vp)); 950 #endif 951 error = VOP_LOCK1(vp, flags, file, line); 952 flags &= ~LK_INTERLOCK; /* Interlock is always dropped. */ (kgdb) info loc error = 0xc0f22780 (kgdb) p flags $4 = 0x200400 (kgdb) -- Test scenario: iknowthis