HEAD r224866 $ console leopard3 [`leopard3' -- console is connecting] [Enter `^Ec?' for help] GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2011 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-BETA1 #0: Sun Aug 14 18:16:12 CEST 2011 test@leopard3.netperf.freebsd.org:/mnt/pho/obj/mnt/pho/src/sys/PHO i386 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. CPU: Intel(R) Xeon(R) CPU 3050 @ 2.13GHz (2128.05-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x6f2 Family = 6 Model = f Stepping = 2 Features=0xbfebfbff Features2=0xe3bd AMD Features=0x20100000 AMD Features2=0x1 TSC: P-state invariant, performance statistics real memory = 2147483648 (2048 MB) avail memory = 2080460800 (1984 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs FreeBSD/SMP: 1 package(s) x 2 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard ioapic2 irqs 48-71 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 cpu0: on acpi0 cpu1: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: irq 16 at device 1.0 on pci0 pci1: on pcib1 pcib2: at device 0.0 on pci1 pci2: on pcib2 pcib3: at device 0.2 on pci1 pci3: on pcib3 pcib4: irq 16 at device 3.0 on pci0 pci10: on pcib4 pci10: at device 0.0 (no driver attached) pcib5: irq 17 at device 28.0 on pci0 pcib5: failed to allocate initial I/O port window: 0-0xfff pcib5: failed to allocate initial memory window: 0-0xfffff pcib5: failed to allocate initial prefetch window: 0-0xfffff pci14: on pcib5 pcib6: irq 17 at device 28.4 on pci0 pci18: on pcib6 em0: port 0x5000-0x501f mem 0xe0300000-0xe031ffff irq 16 at device 0.0 on pci18 em0: Using an MSI interrupt em0: Ethernet address: 00:30:48:8e:d0:68 pcib7: irq 16 at device 28.5 on pci0 pci19: on pcib7 em1: port 0x6000-0x601f mem 0xe0400000-0xe041ffff irq 17 at device 0.0 on pci19 em1: Using an MSI interrupt em1: Ethernet address: 00:30:48:8e:d0:69 uhci0: port 0x3000-0x301f irq 23 at device 29.0 on pci0 usbus0: on uhci0 uhci1: port 0x3020-0x303f irq 19 at device 29.1 on pci0 usbus1: on uhci1 uhci2: port 0x3040-0x305f irq 18 at device 29.2 on pci0 usbus2: on uhci2 uhci3: port 0x3060-0x307f irq 16 at device 29.3 on pci0 usbus3: on uhci3 ehci0: mem 0xe0000000-0xe00003ff irq 23 at device 29.7 on pci0 usbus4: EHCI version 1.0 usbus4: on ehci0 pcib8: at device 30.0 on pci0 pci20: on pcib8 vgapci0: port 0x7000-0x70ff mem 0xe8000000-0xefffffff,0xe0500000-0xe050ffff irq 16 at device 4.0 on pci20 isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x30a0-0x30af at device 31.1 on pci0 ata0: on atapci0 ahci0: port 0x30e8-0x30ef,0x30dc-0x30df,0x30e0-0x30e7,0x30d8-0x30db,0x30b0-0x30bf mem 0xe0000400-0xe00007ff irq 19 at device 31.2 on pci0 ahci0: AHCI v1.10 with 4 3Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 pci0: at device 31.3 (no driver attached) acpi_button0: on acpi0 atrtc0: port 0x70-0x71 irq 8 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: console (115200,n,8,1) uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fd0: <1440-KB 3.5" drive> on fdc0 drive 0 pmtimer0 on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x100> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] ppc0: parallel port not found. est0: on cpu0 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 828082806000828 device_attach: est0 attach returned 6 p4tcc0: on cpu0 est1: on cpu1 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 828082806000828 device_attach: est1 attach returned 6 p4tcc1: on cpu1 Timecounters tick every 1.000 msec usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 12Mbps Full Speed USB v1.0 usbus3: 12Mbps Full Speed USB v1.0 ugen0.1: at usbus0 uhub0: on usbus0 Expensive timeout(9) function: 0xc0a472d0(0xc5b1d5c0) 0.011050566 s ugen1.1: at usbus1 uhub1: on usbus1 ugen2.1: at usbus2 uhub2: on usbus2 ugen3.1: at usbus3 uhub3: on usbus3 usbus4: 480Mbps High Speed USB v2.0 ugen4.1: at usbus4 uhub4: on usbus4 Expensive timeout(9) function: 0xc0527ed0(0xc5b38500) 0.020131050 s ada0 at ahcich0 bus 0 scbus1 target 0 lun 0 ada0: ATA-7cd0 at ata0 bus 0 scbus0 target 0 lun 0 cd0: Removable CD-ROM SCSI-0 device cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present SATA 2.x device ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 238475MB (488397168 512 byte sectors: 16H 63S/T 16383C) ada0: Previously was known as ad4 SMP: AP CPU #1 Launched! Timecounter "TSC-low" frequency 16625355 Hz quality 1000 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. uhub1: 2 ports with 2 removable, self powered uhub0: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 uhub4: 8 ports with 8 removable, self powered Trying to mount root from nfs: []... NFS ROOT: 192.168.5.1:/zoo/leopard3 em0: link state changed to UP Interface em0 IP-Address 192.168.5.109 Broadcast 192.168.5.255 Setting hostuuid: 64a93476-27b1-3004-c299-0030488ed164. Setting hostid: 0x7acc54f1. Starting ddb. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: /dev/ada0d: 65460 files, 4458673 used, 111766453 free (2933 frags, 13970440 blocks, 0.0% fragmentation) mount_nfs: can't update /var/db/mounttab for 192.168.5.1:/zoo/leopard3 Mounting local file systems:. Setting hostname: leopard3.netperf.freebsd.org. Starting Network: lo0 em0 em1. lo0: flags=8049 metric 0 mtu 16384 options=3 inet 127.0.0.1 netmask 0xff000000 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x8 nd6 options=21 em0: flags=8843 metric 0 mtu 1500 options=219b ether 00:30:48:8e:d0:68 inet 192.168.5.109 netmask 0xffffff00 broadcast 192.168.5.255 inet6 fe80::230:48ff:fe8e:d068%em0 prefixlen 64 tentative scopeid 0x1 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active em1: flags=8802 metric 0 mtu 1500 options=219b ether 00:30:48:8e:d0:69 media: Ethernet autoselect status: no carrier Starting devd. Starting Network: em1. em1: flags=8802 metric 0 mtu 1500 options=219b ether 00:30:48:8e:d0:69 media: Ethernet autoselect status: no carrier Starting Network: usbus0. Starting Network: usbus1. Starting Network: usbus2. Starting Network: usbus3. Starting Network: usbus4. add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 Creating and/or trimming log files. Starting syslogd. No core dumps found. ELF ldconfig path: /lib /usr/lib /usr/lib/compat a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout Setting date via ntp. 16 Aug 05:57:37 ntpdate[1389]: step time server 192.168.5.1 offset -0.825231 sec Starting rpcbind. NFS access cache time=60 Clearing /tmp (X related). Starting statd. Updating motd:. Configuring syscons: blanktime. Starting sshd. Starting cron. Local package initialization: watchdogd. Starting inetd. Tue Aug 16 05:57:39 UTC 2011 FreeBSD/i386 (leopard3.netperf.freebsd.org) (ttyu0) login: root Aug 16 18:01:25 leopard3 login: ROOT LOGIN (root) ON ttyu0 FreeBSD 9.0-BETA1 (PHO) #0: Sun Aug 14 18:16:12 CEST 2011 Welcome to FreeBSD! Before seeking technical support, please use the following resources: o Security advisories and updated errata information for all releases are at http://www.FreeBSD.org/releases/ - always consult the ERRATA section for your release first as it's updated frequently. o The Handbook and FAQ documents are at http://www.FreeBSD.org/ and, along with the mailing lists, can be searched by going to http://www.FreeBSD.org/search/. If the doc distribution has been installed, they're also available formatted in /usr/share/doc. If you still have a question or problem, please take the output of `uname -a', along with any relevant error messages, and email it as a question to the questions@FreeBSD.org mailing list. If you are unfamiliar with FreeBSD's directory layout, please refer to the hier(7) manual page. If you are not familiar with manual pages, type `man man'. You may also use sysinstall(8) to re-enter the installation and configuration utility. Edit /etc/motd to change this login announcement. leopard3# ~test/keepalive.sh Tue Aug 16 20:01:30 CEST 2011 8:01PM up 12:06, 2 users, load averages: 0.00, 0.00, 0.00 lock order reversal: 1st 0xdf317b60 bufwait (bufwait) @ /mnt/pho/src/sys/kern/vfs_bio.c:2658 2nd 0xc65cb400 dirhash (dirhash) @ /mnt/pho/src/sys/ufs/ufs/ufs_dirhash.c:284 KDB: stack backtrace: db_trace_self_wrapper(c0f00d1e,7366752f,7366752f,7366752f,7269645f,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a40afb,c0f0475f,c5947370,c594b748,edda2a98,...) at kdb_backtrace+0x2a _witness_debugger(c0f0475f,c65cb400,c0f3443e,c594b748,c0f340bf,...) at _witness_debugger+0x25 witness_checkorder(c65cb400,9,c0f340bf,11c,0,...) at witness_checkorder+0x839 _sx_xlock(c65cb400,0,c0f340bf,11c,e0418018,...) at _sx_xlock+0x85 ufsdirhash_acquire(0,e,c5b0f800,df317b00,e0418018,...) at ufsdirhash_acquire+0x48 ufsdirhash_remove(c640c89c,e0418018,18,edda2b28,edda2b24,...) at ufsdirhash_remove+0x14 ufs_dirremove(c65ca570,c640c828,500800c,1,c65ca570,...) at ufs_dirremove+0x143 ufs_rmdir(edda2c04,c0f52695,0,0,c65ca414,...) at ufs_rmdir+0xf1 VOP_RMDIR_APV(c104be20,edda2c04,2,edda2bdc,284661b8,...) at VOP_RMDIR_APV+0xf6 kern_rmdirat(c655c2e0,ffffff9c,284661b8,0,edda2c50,...) at kern_rmdirat+0x1db kern_rmdir(c655c2e0,284661b8,0,edda2c7c,c0a4ad73,...) at kern_rmdir+0x27 rmdir(c655c2e0,edda2cec,edda2d80,edda2c88,0,...) at rmdir+0x22 syscallenter(c655c2e0,edda2ce4,c0d3eb76,0,0,...) at syscallenter+0x263 syscall(edda2d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (137, FreeBSD ELF32, rmdir), eip = 0x280e25bb, esp = 0xbfbfea2c, ebp = 0xbfbfea48 --- Tue Aug 16 20:11:30 CEST 2011 8:11PM up 12:16, 2 users, load averages: 11.80, 13.45, 7.90 kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) Tue Aug 16 20:21:30 CEST 2011 8:21PM up 12:26, 2 users, load averages: 12.21, 13.30, 10.75 kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) tvotohz: negative time difference -1962400699 sec -923855 usec Tue Aug 16 20:31:30 CEST 2011 8:31PM up 12:36, 2 users, load averages: 17.37, 16.33, 13.51 Tue Aug 16 20:41:30 CEST 2011 8:41PM up 12:46, 2 users, load averages: 10.70, 14.24, 13.88 Tue Aug 16 20:51:30 CEST 2011 8:51PM up 12:56, 2 users, load averages: 8.67, 10.79, 12.16 Tue Aug 16 21:01:31 CEST 2011 9:01PM up 13:06, 2 users, load averages: 8.42, 12.86, 13.01 kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) Tue Aug 16 21:11:31 CEST 2011 9:11PM up 13:16, 2 users, load averages: 12.68, 12.21, 12.59 kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) Tue Aug 16 21:21:33 CEST 2011 9:21PM up 13:26, 2 users, load averages: 17.85, 16.76, 15.01 kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) Tue Aug 16 21:31:34 CEST 2011 9:31PM up 13:36, 2 users, load averages: 15.21, 15.39, 14.76 Tue Aug 16 21:41:35 CEST 2011 9:41PM up 13:46, 2 users, load averages: 10.71, 13.51, 13.99 kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) Tue Aug 16 21:51:36 CEST 2011 9:51PM up 13:56, 2 users, load averages: 14.93, 15.23, 14.89 Expensive timeout(9) function: 0xc0cf78b0(0xc1217aa0) 0.109818045 s Tue Aug 16 22:01:39 CEST 2011 10:01PM up 14:06, 2 users, load averages: 14.76, 15.21, 15.30 tvotohz: negative time difference -799700531 sec -923830 usec tvotohz: negative time difference -2071624704 sec -966446 usec Tue Aug 16 22:11:40 CEST 2011 10:11PM up 14:16, 2 users, load averages: 19.72, 16.81, 15.82 kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) Tue Aug 16 22:21:42 CEST 2011 10:21PM up 14:26, 2 users, load averages: 12.23, 14.32, 14.93 Tue Aug 16 22:31:43 CEST 2011 10:31PM up 14:36, 2 users, load averages: 11.16, 14.46, 15.01 kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) tvotohz: negative time difference -10451512 sec -999935 usec Tue Aug 16 22:41:43 CEST 2011 10:41PM up 14:46, 2 users, load averages: 14.69, 14.30, 14.65 kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) Tue Aug 16 22:51:43 CEST 2011 10:51PM up 14:56, 2 users, load averages: 18.90, 15.24, 14.69 tvotohz: negative time difference -1357643776 sec 0 usec tvotohz: negative time difference -2095821172 sec -987810 usec tvotohz: negative time difference -1794939 sec -745181 usec kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) Tue Aug 16 23:01:43 CEST 2011 11:01PM up 15:06, 2 users, load averages: 11.36, 13.38, 14.12 kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) Tue Aug 16 23:11:45 CEST 2011 11:11PM up 15:16, 2 users, load averages: 20.77, 14.85, 14.18 Tue Aug 16 23:21:45 CEST 2011 11:21PM up 15:26, 2 users, load averages: 14.75, 13.27, 13.35 kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.maxfiles limit exceeded by uid 1001, please see tuning(7). kern.maxfiles limit exceeded by uid 1001, please see tuning(7). kern.maxfiles limit exceeded by uid 1001, please see tuning(7). kern.maxfiles limit exceeded by uid 1001, please see tuning(7). kern.maxfiles limit exceeded by uid 1001, please see tuning(7). kern.maxfiles limit exceeded by uid 1001, please see tuning(7). kern.maxfiles limit exceeded by uid 1001, please see tuning(7). kern.maxfiles limit exceeded by uid 1001, please see tuning(7). kern.maxfiles limit exceeded by uid 1001, please see tuning(7). kern.maxfiles limit exceeded by uid 1001, please see tuning(7). kern.maxfiles limit exceeded by uid 1001, please see tuning(7). kern.maxfiles limit exceeded by uid 1001, please see tuning(7). kern.maxfiles limit exceeded by uid 1001, please see tuning(7). Tue Aug 16 23:31:45 CEST 2011 11:31PM up 15:36, 2 users, load averages: 13.92, 15.49, 14.80 kern.maxfiles limit exceeded by uid 1001, please see tuning(7). kern.maxfiles limit exceeded by uid 1001, please see tuning(7). kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) Tue Aug 16 23:41:46 CEST 2011 11:41PM up 15:46, 2 users, load averages: 19.40, 16.87, 16.04 Tue Aug 16 23:51:48 CEST 2011 11:51PM up 15:56, 2 users, load averages: 20.18, 15.70, 15.16 kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) Wed Aug 17 00:01:49 CEST 2011 12:01AM up 16:06, 2 users, load averages: 10.23, 13.67, 14.48 kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) kern.ipc.maxpipekva exceeded; see tuning(7) panic: attempt to create a too small sbuf cpuid = 1 KDB: enter: panic [ thread pid 69319 tid 120959 ] Stopped at kdb_enter+0x3a: movl $0,kdb_why db> run pho db:0:pho> bt Tracing pid 69319 tid 120959 td 0xc6c8a000 kdb_enter(c0efcd52,c0efcd52,c0f0206a,ee15fb98,1,...) at kdb_enter+0x3a panic(c0f0206a,1c,c19c9c80,ee15fbf4,0,...) at panic+0x134 sbuf_newbuf(0,c19c9c10,ee15fc00,c0c68cff,0,...) at sbuf_newbuf+0x47 sbuf_new(ee15fbf4,c5e43770,1,0,c09f4983,...) at sbuf_new+0x66 mac_cred_externalize_label(0,c5e43780,c5e43770,1,1,...) at mac_cred_externalize_label+0x2a __mac_get_proc(c6c8a000,ee15fcec,ee15fd28,c0f02f8a,0,...) at __mac_get_proc+0xe0 syscallenter(c6c8a000,ee15fce4,ee15fce4,e,0,...) at syscallenter+0x263 syscall(ee15fd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (384, FreeBSD ELF32, __mac_get_proc), eip = 0x280eeaf1, esp = 0xbfbfe940, ebp = 0xbfbfe988 --- db:0:bt> show allpcpu Current CPU: 1 cpuid = 0 dynamic pcpu = 0x7c8280 curthread = 0xc6033b80: pid 69318 "syscall" curpcb = 0xee11ad80 fpcurthread = none idlethread = 0xc59a78a0: tid 100003 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: cpuid = 1 dynamic pcpu = 0x45aa280 curthread = 0xc6c8a000: pid 69319 "syscall" curpcb = 0xee15fd80 fpcurthread = none idlethread = 0xc59a75c0: tid 100004 "idle: cpu1" APIC ID = 1 currentldt = 0x50 spin locks held: db:0:allpcpu> show alllocks Process 69316 (syscall) thread 0xc6aac5c0 (120928) exclusive lockmgr bufwait (bufwait) r = 0 (0xdf2e9ce0) locked @ /mnt/pho/src/sys/vm/vm_pager.c:309 shared lockmgr newnfs (newnfs) r = 0 (0xc5c651b4) locked @ /mnt/pho/src/sys/kern/vfs_subr.c:2134 Process 1839 (sshd) thread 0xc623d8a0 (100075) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xc656970c) locked @ /mnt/pho/src/sys/kern/uipc_sockbuf.c:148 db:0:alllocks> show lockedvnods Locked vnodes 0xc5c6515c: 0xc5c6515c: tag newnfs, type VREG tag newnfs, type VREG usecount 67, writecount 0, refcount 69 mountedhere 0 usecount 67, writecount 0, refcount 69 mountedhere 0 flags (VV_TEXT) flags (VV_TEXT) v_object 0xc1990c38 ref 66 pages 30 v_object 0xc1990c38 ref 66 pages 30 lock type newnfs: SHARED (count 1) lock type newnfs: SHARED (count 1) #0 0xc09ef9a4 at __lockmgr_args+0x5c4 #0 0xc09ef9a4 at __lockmgr_args+0x5c4 #1 0xc095a4bd at nfs_lock1+0xcd #1 0xc095a4bd at nfs_lock1+0xcd #2 0xc0d616f5 at VOP_LOCK1_APV+0xb5 #2 0xc0d616f5 at VOP_LOCK1_APV+0xb5 #3 0xc0ab1a18 at _vn_lock+0x78 #3 0xc0ab1a18 at _vn_lock+0x78 #4 0xc0aa424b at vget+0xbb #4 0xc0aa424b at vget+0xbb #5 0xc0c6c10b at vm_fault_hold+0xacb #5 0xc0c6c10b at vm_fault_hold+0xacb #6 0xc0c6d826 at vm_fault+0x46 #6 0xc0c6d826 at vm_fault+0x46 #7 0xc0d3e51d at trap_pfault+0x10d #7 0xc0d3e51d at trap_pfault+0x10d #8 0xc0d3ec05 at trap+0x4c5 #8 0xc0d3ec05 at trap+0x4c5 #9 0xc0d27c8c at calltrap+0x6 #9 0xc0d27c8c at calltrap+0x6 #10 0xc0aad983 at kern_fchmodat+0xa3 #10 0xc0aad983 at kern_fchmodat+0xa3 #11 0xc0aada77 at kern_chmod+0x37 #11 0xc0aada77 at kern_chmod+0x37 #12 0xc0aadaa9 at chmod+0x29 #12 0xc0aadaa9 at chmod+0x29 #13 0xc0a4ad73 at syscallenter+0x263 #13 0xc0a4ad73 at syscallenter+0x263 #14 0xc0d3e65f at syscall+0x4f #14 0xc0d3e65f at syscall+0x4f #15 0xc0d27cf1 at Xint0x80_syscall+0x21 #15 0xc0d27cf1 at Xint0x80_syscall+0x21 fileid 19394854 fsid 0x400ff01 fileid 19394854 fsid 0x400ff01 db:0:lockedvnods> show mount 0xc5eb1b50 192.168.5.1:/zoo/leopard3 on / (nfs) 0xc5eb2000 devfs on /dev (devfs) 0xc5eb187c /dev/ada0d on /work (ufs) 0xc5eb15a8 /dev/md0 on /var (ufs) 0xc5eb12d4 /dev/md1 on /tmp (ufs) More info: show mount db:0:mount> ps pid ppid pgrp uid state wmesg wchan cmd 69324 69320 4386 1001 S+ nanslp 0xc1098d84 swap 69323 69320 4386 1001 S+ nanslp 0xc1098d84 swap 69322 69320 4386 1001 S+ nanslp 0xc1098d84 swap 69321 69320 4386 1001 S+ nanslp 0xc1098d84 swap 69320 69311 4386 1001 S+ wait 0xc6c6a000 swap 69319 69313 4386 1001 R+ CPU 1 syscall 69318 69313 4386 1001 R+ CPU 0 syscall 69317 69313 4386 1001 R+ syscall 69316 69313 4386 1001 RL+ syscall 69315 69313 4386 1001 R+ syscall 69314 69313 4386 1001 R+ syscall 69313 69312 4386 1001 S+ wait 0xc6348000 syscall 69312 4394 4386 1001 S+ nanslp 0xc1098d84 syscall 69311 4394 4386 1001 S+ nanslp 0xc1098d84 swap 43576 4382 4382 0 SW+ nanslp 0xc1098d84 sleep 4394 4393 4386 1001 S+ wait 0xc634b580 run 4393 4392 4386 1001 SW+ wait 0xc654d580 run 4392 4386 4386 1001 S+ nanslp 0xc1098d84 run 4386 1845 4386 1001 SW+ wait 0xc634a840 sh 4382 4378 4382 0 SW+ wait 0xc659a000 sh 4378 1759 4378 0 SW+ pause 0xc62f05d8 csh 1845 1844 1845 1001 SWs+ wait 0xc62f0b00 sh 1844 1839 1839 1001 S select 0xc5e6e024 sshd 1839 1664 1839 0 SWs sbwait 0xc6569738 sshd 1759 1 1759 0 SWs+ wait 0xc654f000 login 1758 1 1758 0 Ss+ ttyin 0xc5a4e670 getty 1757 1 1757 0 Ss+ ttyin 0xc5a4e870 getty 1756 1 1756 0 Ss+ ttyin 0xc5a4ea70 getty 1755 1 1755 0 Ss+ ttyin 0xc5a4ec70 getty 1754 1 1754 0 Ss+ ttyin 0xc5a4ee70 getty 1753 1 1753 0 Ss+ ttyin 0xc5c2d070 getty 1752 1 1752 0 Ss+ ttyin 0xc5c2d270 getty 1751 1 1751 0 Ss+ ttyin 0xc5c2d470 getty 1713 1 1713 0 Ss select 0xc5e6e264 inetd 1685 1 1685 0 Ss nanslp 0xc1098d84 watchdogd 1675 1 1675 0 SWs nanslp 0xc1098d84 cron 1664 1 1664 0 Ss select 0xc5e6e364 sshd 1525 1 1525 0 Ss select 0xc5e5fc24 rpc.statd 1452 0 0 0 DL mdwait 0xc5ea3000 [md1] 1397 1 1397 0 Ss select 0xc5e6e324 rpcbind 1367 1 1367 0 Ss select 0xc5e6e3a4 syslogd 1188 1 1188 0 Ss select 0xc5e6e3e4 devd 101 0 0 0 DL mdwait 0xc5ea4000 [md0] 17 0 0 0 DL sdflush 0xc1215640 [softdepflush] 16 0 0 0 DL vlruwt 0xc5e61840 [vnlru] 9 0 0 0 DL syncer 0xc120ef74 [syncer] 8 0 0 0 DL psleep 0xc120ec08 [bufdaemon] 7 0 0 0 DL pgzero 0xc12174dc [pagezero] 6 0 0 0 DL psleep 0xc12170c8 [vmdaemon] 5 0 0 0 DL psleep 0xc1217090 [pagedaemon] 4 0 0 0 DL ccb_scan 0xc105dad4 [xpt_thrd] 3 0 0 0 DL waiting_ 0xc1210a98 [sctp_iterator] 2 0 0 0 DL - 0xc5b3de3c [fdc0] 15 0 0 0 DL (threaded) [usb] 100049 D - 0xc5be2d34 [usbus4] 100048 D - 0xc5be2d04 [usbus4] 100047 D - 0xc5be2cd4 [usbus4] 100046 D - 0xc5be2ca4 [usbus4] 100045 D - 0xc5bccdac [usbus3] 100044 D - 0xc5bccd7c [usbus3] 100043 D - 0xc5bccd4c [usbus3] 100042 D - 0xc5bccd1c [usbus3] 100040 D - 0xc5bb9dac [usbus2] 100039 D - 0xc5bb9d7c [usbus2] 100038 D - 0xc5bb9d4c [usbus2] 100037 D - 0xc5bb9d1c [usbus2] 100035 D - 0xc5ba2dac [usbus1] 100034 D - 0xc5ba2d7c [usbus1] 100033 D - 0xc5ba2d4c [usbus1] 100032 D - 0xc5ba2d1c [usbus1] 100030 D - 0xc5b88dac [usbus0] 100029 D - 0xc5b88d7c [usbus0] 100028 D - 0xc5b88d4c [usbus0] 100027 D - 0xc5b88d1c [usbus0] 14 0 0 0 DL - 0xc1097b84 [yarrow] 13 0 0 0 DL (threaded) [geom] 100011 D - 0xc10957a8 [g_down] 100010 D - 0xc10957a4 [g_up] 100009 D - 0xc109579c [g_event] 12 0 0 0 WL (threaded) [intr] 100054 I [irq1: atkbd0] 100052 I [swi0: uart uart] 100051 I [irq258: ahci0] 100050 I [irq14: ata0] 100041 I [irq16: uhci3] 100036 I [irq18: uhci2] 100031 I [irq19: uhci1] 100026 I [irq23: uhci0 ehci0] 100023 I [swi5: +] 100022 I [swi2: cambio] 100017 I [swi6: task queue] 100015 I [swi6: Giant taskq] 100008 I [swi3: vm] 100007 I [swi4: clock] 100006 I [swi4: clock] 100005 I [swi1: netisr 0] 11 0 0 0 RL (threaded) [idle] 100004 CanRun [idle: cpu1] 100003 CanRun [idle: cpu0] 1 0 1 0 SLs wait 0xc59a5b00 [init] 10 0 0 0 DL audit_wo 0xc1214d60 [audit] 0 0 0 0 DLs (threaded) [kernel] 100065 D - 0xc5c02180 [mca taskq] 100055 D - 0xc1097b84 [deadlkres] 100025 D - 0xc5b51740 [em1 taskq] 100024 D - 0xc5b51980 [em0 taskq] 100021 D - 0xc59a8100 [acpi_task_2] 100020 D - 0xc59a8100 [acpi_task_1] 100019 D - 0xc59a8100 [acpi_task_0] 100018 D - 0xc59a8140 [kqueue taskq] 100016 D - 0xc59a81c0 [ffs_trim taskq] 100014 D - 0xc59a84c0 [thread taskq] 100012 D - 0xc59a8500 [firmware taskq] 100000 D sched 0xc1095880 [swapper] db:0:ps> allt Tracing command swap pid 69324 tid 118289 td 0xc5f7a000 sched_switch(c5f7a000,0,104,191,df8bc0e0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c5f7a000,0,c0f02600,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(2711,c0a472d0,c5f7a000,1,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c1098d84,6c,c0efe2df,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c1098d84,0,16c,c0efe2df,2711,...) at _sleep+0x328 kern_nanosleep(c5f7a000,ee0b4c34,ee0b4c3c,a,0,...) at kern_nanosleep+0xc1 nanosleep(c5f7a000,ee0b4cec,281aa044,1,0,...) at nanosleep+0x6f syscallenter(c5f7a000,ee0b4ce4,c0d3eebd,c1097470,0,...) at syscallenter+0x263 syscall(ee0b4d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2817a0b7, esp = 0xbfbfe95c, ebp = 0xbfbfe988 --- Tracing command swap pid 69323 tid 100202 td 0xc5e6a2e0 sched_switch(c5e6a2e0,0,104,191,df81bc38,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c5e6a2e0,0,c0f02600,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(1b59,c0a472d0,c5e6a2e0,0,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c1098d84,6c,c0efe2df,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c1098d84,0,16c,c0efe2df,1b59,...) at _sleep+0x328 kern_nanosleep(c5e6a2e0,edea5c34,edea5c3c,7,0,...) at kern_nanosleep+0xc1 nanosleep(c5e6a2e0,edea5cec,281aa044,1,0,...) at nanosleep+0x6f syscallenter(c5e6a2e0,edea5ce4,c0d3eebd,c1097470,0,...) at syscallenter+0x263 syscall(edea5d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2817a0b7, esp = 0xbfbfe95c, ebp = 0xbfbfe988 --- Tracing command swap pid 69322 tid 118257 td 0xc623a8a0 sched_switch(c623a8a0,0,104,191,df75f670,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c623a8a0,0,c0f02600,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(1b59,c0a472d0,c623a8a0,0,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c1098d84,6c,c0efe2df,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c1098d84,0,16c,c0efe2df,1b59,...) at _sleep+0x328 kern_nanosleep(c623a8a0,ee045c34,ee045c3c,7,0,...) at kern_nanosleep+0xc1 nanosleep(c623a8a0,ee045cec,281aa044,1,0,...) at nanosleep+0x6f syscallenter(c623a8a0,ee045ce4,c0d3eebd,c1097470,0,...) at syscallenter+0x263 syscall(ee045d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2817a0b7, esp = 0xbfbfe95c, ebp = 0xbfbfe988 --- Tracing command swap pid 69321 tid 100118 td 0xc655d8a0 sched_switch(c655d8a0,0,104,191,df7fd3e8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c655d8a0,0,c0f02600,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(bb9,c0a472d0,c655d8a0,0,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c1098d84,6c,c0efe2df,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c1098d84,0,16c,c0efe2df,bb9,...) at _sleep+0x328 kern_nanosleep(c655d8a0,edd77c34,edd77c3c,3,0,...) at kern_nanosleep+0xc1 nanosleep(c655d8a0,edd77cec,edd77d80,edd77c88,0,...) at nanosleep+0x6f syscallenter(c655d8a0,edd77ce4,c0d3eb76,c0a2f648,c10a5a00,...) at syscallenter+0x263 syscall(edd77d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2817a0b7, esp = 0xbfbfe95c, ebp = 0xbfbfe988 --- Tracing command swap pid 69320 tid 120986 td 0xc6c935c0 sched_switch(c6c935c0,0,104,191,df846c28,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c6c935c0,0,c0f02600,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6c6a000,6c,c0f0545a,100,0,...) at sleepq_wait_sig+0x17 _sleep(c6c6a000,c6c6a088,16c,c0f0545a,0,...) at _sleep+0x35c kern_wait(c6c935c0,10ec9,ee1b0c44,0,0,...) at kern_wait+0xb96 wait4(c6c935c0,ee1b0cec,281a9b08,1,0,...) at wait4+0x3b syscallenter(c6c935c0,ee1b0ce4,c0d3eebd,c1097470,0,...) at syscallenter+0x263 syscall(ee1b0d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x28104ffb, esp = 0xbfbfe98c, ebp = 0xbfbfe9a8 --- Tracing command syscall pid 69319 tid 120959 td 0xc6c8a000 kdb_enter(c0efcd52,c0efcd52,c0f0206a,ee15fb98,1,...) at kdb_enter+0x3a panic(c0f0206a,1c,c19c9c80,ee15fbf4,0,...) at panic+0x134 sbuf_newbuf(0,c19c9c10,ee15fc00,c0c68cff,0,...) at sbuf_newbuf+0x47 sbuf_new(ee15fbf4,c5e43770,1,0,c09f4983,...) at sbuf_new+0x66 mac_cred_externalize_label(0,c5e43780,c5e43770,1,1,...) at mac_cred_externalize_label+0x2a __mac_get_proc(c6c8a000,ee15fcec,ee15fd28,c0f02f8a,0,...) at __mac_get_proc+0xe0 syscallenter(c6c8a000,ee15fce4,ee15fce4,e,0,...) at syscallenter+0x263 syscall(ee15fd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (384, FreeBSD ELF32, __mac_get_proc), eip = 0x280eeaf1, esp = 0xbfbfe940, ebp = 0xbfbfe988 --- Tracing command syscall pid 69318 tid 120936 td 0xc6033b80 cpustop_handler(0,13,ee11ad1c,c0d3e776,c0878a6b,...) at cpustop_handler+0x28 ipi_nmi_handler(c0878a6b,c5a7b200,1cca240,28183925,c6c78000,...) at ipi_nmi_handler+0x47 trap(ee11ad28) at trap+0x36 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0x28183925, esp = 0xbfbfe8d0, ebp = 0xbfbfe938 --- Tracing command syscall pid 69317 tid 111784 td 0xc62182e0 sched_switch(c62182e0,0,207,18c,e105f98,...) at sched_switch+0x3bc mi_switch(207,0,c0f02f41,d4,c1277080,...) at mi_switch+0x1eb ast(edff6d28) at ast+0x2b6 doreti_ast() at doreti_ast+0x17 Tracing command syscall pid 69316 tid 120928 td 0xc6aac5c0 sched_switch(c6aac5c0,0,104,191,13eeddf0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c6aac5c0,0,c0f02600,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c6eca800,0,c0eec61f,0,0,...) at sleepq_timedwait+0x6b _sleep(c6eca800,c6177100,0,c0eec61f,493e1,...) at _sleep+0x342 clnt_vc_call(c6176320,ee10252c,6,c5fdc800,ee102640,...) at clnt_vc_call+0x815 clnt_reconnect_call(c6176360,ee10252c,6,c5fdc800,ee102640,...) at clnt_reconnect_call+0x670 newnfs_request(ee102640,c6175a00,0,c6175b10,c5c6515c,c6aac5c0,c5986680,186a3,3,0,1,0) at newnfs_request+0xbf3 nfscl_request(ee102640,c5c6515c,c6aac5c0,c5986680,0,...) at nfscl_request+0x82 nfsrpc_read(c5c6515c,ee1027d4,c5986680,c6aac5c0,ee1026fc,...) at nfsrpc_read+0x4f9 ncl_readrpc(c5c6515c,ee1027d4,c5986680,93,78,...) at ncl_readrpc+0x4b ncl_getpages(ee102840,c0f51fd0,c5c6515c,c5c6515c,c1990c38,...) at ncl_getpages+0x389 VOP_GETPAGES_APV(c1022ac0,ee102840,c0f39678,297,c105c780,...) at VOP_GETPAGES_APV+0xd6 vnode_pager_getpages(c1990c38,ee102930,1,0,ee1029b4,...) at vnode_pager_getpages+0xb4 vm_fault_hold(c667b2b8,28070000,1,0,0,...) at vm_fault_hold+0x103f vm_fault(c667b2b8,28070000,1,0,2807010a,...) at vm_fault+0x46 trap_pfault(5,0,c0f4bfc6,c0f35af0,c6c79840,...) at trap_pfault+0x10d trap(ee102acc) at trap+0x4c5 calltrap() at calltrap+0x6 --- trap 0xc, eip = 0xc0d3c5f7, esp = 0xee102b0c, ebp = 0xee102b88 --- copyinstr(ee102ba4,8,3,4,41d50002,...) at copyinstr+0x37 kern_fchmodat(c6aac5c0,ffffff9c,2807010a,0,41d5,...) at kern_fchmodat+0xa3 kern_chmod(c6aac5c0,2807010a,0,855641d5,ee102c7c,...) at kern_chmod+0x37 chmod(c6aac5c0,ee102cec,ee102d28,c0f02f8a,0,...) at chmod+0x29 syscallenter(c6aac5c0,ee102ce4,ee102ce4,e,0,...) at syscallenter+0x263 syscall(ee102d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (15, FreeBSD ELF32, chmod), eip = 0x280eeaf1, esp = 0xbfbfe940, ebp = 0xbfbfe988 --- Tracing command syscall pid 69315 tid 100069 td 0xc62405c0 sched_switch(c62405c0,0,207,18c,19c45b90,...) at sched_switch+0x3bc mi_switch(207,0,c0f02f41,d4,c1277080,...) at mi_switch+0x1eb ast(df77dd28) at ast+0x2b6 doreti_ast() at doreti_ast+0x17 Tracing command syscall pid 69314 tid 121576 td 0xc6e53000 sched_switch(c6e53000,0,207,18c,8123d68,...) at sched_switch+0x3bc mi_switch(207,0,c0f02f41,d4,c1277080,...) at mi_switch+0x1eb ast(ee20ad28) at ast+0x2b6 doreti_ast() at doreti_ast+0x17 Tracing command syscall pid 69313 tid 100099 td 0xc635a000 sched_switch(c635a000,0,104,191,de2f1130,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c635a000,0,c0f02600,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c6348000,6c,c0f0545a,100,0,...) at sleepq_wait_sig+0x17 _sleep(c6348000,c6348088,16c,c0f0545a,0,...) at _sleep+0x35c kern_wait(c635a000,10ec2,df7f8c44,0,0,...) at kern_wait+0xb96 wait4(c635a000,df7f8cec,281a8b08,1,0,...) at wait4+0x3b syscallenter(c635a000,df7f8ce4,c0d3eebd,c1097470,0,...) at syscallenter+0x263 syscall(df7f8d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x28103ffb, esp = 0xbfbfe98c, ebp = 0xbfbfe9a8 --- Tracing command syscall pid 69312 tid 100141 td 0xc659d5c0 sched_switch(c659d5c0,0,104,191,e00dff40,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c659d5c0,0,c0f02600,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(3e9,c0a472d0,c659d5c0,0,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c1098d84,6c,c0efe2df,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c1098d84,0,16c,c0efe2df,3e9,...) at _sleep+0x328 kern_nanosleep(c659d5c0,eddd2c34,eddd2c3c,1,0,...) at kern_nanosleep+0xc1 nanosleep(c659d5c0,eddd2cec,eddd2d28,c0f02f8a,0,...) at nanosleep+0x6f syscallenter(c659d5c0,eddd2ce4,eddd2ce4,0,0,...) at syscallenter+0x263 syscall(eddd2d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x281790b7, esp = 0xbfbfe9ac, ebp = 0xbfbfe9d8 --- Tracing command swap pid 69311 tid 100172 td 0xc66795c0 sched_switch(c66795c0,0,104,191,dd43afc8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c66795c0,0,c0f02600,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(3e9,c0a472d0,c66795c0,1,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c1098d84,6c,c0efe2df,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c1098d84,0,16c,c0efe2df,3e9,...) at _sleep+0x328 kern_nanosleep(c66795c0,ede46c34,ede46c3c,1,0,...) at kern_nanosleep+0xc1 nanosleep(c66795c0,ede46cec,ede46d28,c0f02f8a,0,...) at nanosleep+0x6f syscallenter(c66795c0,ede46ce4,ede46ce4,0,0,...) at syscallenter+0x263 syscall(ede46d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2817a0b7, esp = 0xbfbfe9ac, ebp = 0xbfbfe9d8 --- Tracing command sleep pid 43576 tid 100123 td 0xc655c8a0 Tracing command run pid 4394 tid 100110 td 0xc6350b80 sched_switch(c6350b80,0,104,191,dd412f38,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c6350b80,0,c0f02600,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c634b580,6c,c0f0545a,100,0,...) at sleepq_wait_sig+0x17 _sleep(c634b580,c634b608,16c,c0f0545a,0,...) at _sleep+0x35c kern_wait(c6350b80,10ebf,edd37c44,0,0,...) at kern_wait+0xb96 wait4(c6350b80,edd37cec,bfbfe4e8,1,0,...) at wait4+0x3b syscallenter(c6350b80,edd37ce4,c0d3eebd,c1097470,0,...) at syscallenter+0x263 syscall(edd37d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x28103ffb, esp = 0xbfbfe49c, ebp = 0xbfbfe4b8 --- Tracing command run pid 4393 tid 100120 td 0xc655d2e0 Tracing command run pid 4392 tid 100072 td 0xc623db80 sched_switch(c623db80,0,104,191,fbf5d670,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c623db80,0,c0f02600,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(3e9,c0a472d0,c623db80,0,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c1098d84,6c,c0efe2df,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c1098d84,0,16c,c0efe2df,3e9,...) at _sleep+0x328 kern_nanosleep(c623db80,df789c34,df789c3c,1,0,...) at kern_nanosleep+0xc1 nanosleep(c623db80,df789cec,df789d28,c0f02f8a,0,...) at nanosleep+0x6f syscallenter(c623db80,df789ce4,df789ce4,0,0,...) at syscallenter+0x263 syscall(df789d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x281790b7, esp = 0xbfbfe96c, ebp = 0xbfbfe998 --- Tracing command sh pid 4386 tid 100091 td 0xc631b5c0 Tracing command sh pid 4382 tid 100129 td 0xc635a5c0 Tracing command csh pid 4378 tid 100081 td 0xc5e6c8a0 Tracing command sh pid 1845 tid 100078 td 0xc623d000 Tracing command sshd pid 1844 tid 100109 td 0xc6351000 sched_switch(c6351000,0,104,191,de0ad6a0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c6351000,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c09f7eba,c5e6e010,0,c0efb2e7,c6351000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5e6e024,0,edd34a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5e6e024,c5e6e010,c0f04cae,64c,c624b968,...) at _cv_wait_sig+0x243 seltdwait(c624b968,58,c5986400,c6351000,c0f25b82,...) at seltdwait+0xa2 kern_select(c6351000,a,288060b8,288060cc,0,0,20,c634b840,c634b840) at kern_select+0x504 select(c6351000,edd34cec,edd34d28,c0f02f8a,0,...) at select+0x66 syscallenter(c6351000,edd34ce4,edd34ce4,0,0,...) at syscallenter+0x263 syscall(edd34d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x284b0d43, esp = 0xbfbfe19c, ebp = 0xbfbfe1e8 --- Tracing command sshd pid 1839 tid 100075 td 0xc623d8a0 Tracing command login pid 1759 tid 100112 td 0xc63505c0 Tracing command getty pid 1758 tid 100073 td 0xc5e6b5c0 sched_switch(c5e6b5c0,0,104,191,3002e398,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5e6b5c0,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,c5e6b5c0,c5e6b5c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5a4e670,0,c0f07364,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5a4e670,c1097470,c0f06d24,524,0,...) at _cv_wait_sig+0x243 tty_wait(c5a4e600,c5a4e670,df78dc28,1,0,...) at tty_wait+0x71 ttydisc_read(c5a4e600,df78dc28,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c5c21e00,df78dc28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c624c620,df78dc28,c5985e80,0,c5e6b5c0,...) at devfs_read_f+0x7e dofileread(df78dc28,ffffffff,ffffffff,0,df78dc04,...) at dofileread+0x9e kern_readv(c5e6b5c0,0,df78dc28,df78dc48,1,...) at kern_readv+0x68 read(c5e6b5c0,df78dcec,df78dd28,c0f02f8a,0,...) at read+0x4f syscallenter(c5e6b5c0,df78dce4,df78dce4,0,0,...) at syscallenter+0x263 syscall(df78dd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1757 tid 100067 td 0xc5e6b8a0 sched_switch(c5e6b8a0,0,104,191,309d80d0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5e6b8a0,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,c5e6b8a0,c5e6b8a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5a4e870,0,c0f07364,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5a4e870,c1097470,c0f06d24,524,0,...) at _cv_wait_sig+0x243 tty_wait(c5a4e800,c5a4e870,df776c28,1,0,...) at tty_wait+0x71 ttydisc_read(c5a4e800,df776c28,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c5c4b000,df776c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c644f690,df776c28,c5985e80,0,c5e6b8a0,...) at devfs_read_f+0x7e dofileread(df776c28,ffffffff,ffffffff,0,df776c04,...) at dofileread+0x9e kern_readv(c5e6b8a0,0,df776c28,df776c48,1,...) at kern_readv+0x68 read(c5e6b8a0,df776cec,df776d28,c0f02f8a,0,...) at read+0x4f syscallenter(c5e6b8a0,df776ce4,df776ce4,0,0,...) at syscallenter+0x263 syscall(df776d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1756 tid 100106 td 0xc63518a0 sched_switch(c63518a0,0,104,191,30322858,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c63518a0,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,c63518a0,c63518a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5a4ea70,0,c0f07364,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5a4ea70,c1097470,c0f06d24,524,0,...) at _cv_wait_sig+0x243 tty_wait(c5a4ea00,c5a4ea70,edd2bc28,1,0,...) at tty_wait+0x71 ttydisc_read(c5a4ea00,edd2bc28,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c5c4b100,edd2bc28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c644f428,edd2bc28,c5985e80,0,c63518a0,...) at devfs_read_f+0x7e dofileread(edd2bc28,ffffffff,ffffffff,0,edd2bc04,...) at dofileread+0x9e kern_readv(c63518a0,0,edd2bc28,edd2bc48,1,...) at kern_readv+0x68 read(c63518a0,edd2bcec,edd2bd28,c0f02f8a,0,...) at read+0x4f syscallenter(c63518a0,edd2bce4,edd2bce4,0,0,...) at syscallenter+0x263 syscall(edd2bd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1755 tid 100103 td 0xc63542e0 sched_switch(c63542e0,0,104,191,30733860,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c63542e0,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,c63542e0,c63542e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5a4ec70,0,c0f07364,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5a4ec70,c1097470,c0f06d24,524,0,...) at _cv_wait_sig+0x243 tty_wait(c5a4ec00,c5a4ec70,edd22c28,1,0,...) at tty_wait+0x71 ttydisc_read(c5a4ec00,edd22c28,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c5c4b200,edd22c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c625d118,edd22c28,c5985e80,0,c63542e0,...) at devfs_read_f+0x7e dofileread(edd22c28,ffffffff,ffffffff,0,edd22c04,...) at dofileread+0x9e kern_readv(c63542e0,0,edd22c28,edd22c48,1,...) at kern_readv+0x68 read(c63542e0,edd22cec,edd22d28,c0f02f8a,0,...) at read+0x4f syscallenter(c63542e0,edd22ce4,edd22ce4,0,0,...) at syscallenter+0x263 syscall(edd22d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1754 tid 100098 td 0xc631a000 sched_switch(c631a000,0,104,191,3086a8f0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c631a000,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,c631a000,c631a000,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5a4ee70,0,c0f07364,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5a4ee70,c1097470,c0f06d24,524,0,...) at _cv_wait_sig+0x243 tty_wait(c5a4ee00,c5a4ee70,df7f3c28,1,0,...) at tty_wait+0x71 ttydisc_read(c5a4ee00,df7f3c28,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c5c4b300,df7f3c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c625d428,df7f3c28,c5985e80,0,c631a000,...) at devfs_read_f+0x7e dofileread(df7f3c28,ffffffff,ffffffff,0,df7f3c04,...) at dofileread+0x9e kern_readv(c631a000,0,df7f3c28,df7f3c48,1,...) at kern_readv+0x68 read(c631a000,df7f3cec,df7f3d28,c0f02f8a,0,...) at read+0x4f syscallenter(c631a000,df7f3ce4,df7f3ce4,0,0,...) at syscallenter+0x263 syscall(df7f3d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1753 tid 100089 td 0xc631bb80 sched_switch(c631bb80,0,104,191,30a41d48,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c631bb80,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,c631bb80,c631bb80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5c2d070,0,c0f07364,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5c2d070,c1097470,c0f06d24,524,0,...) at _cv_wait_sig+0x243 tty_wait(c5c2d000,c5c2d070,df7d7c28,1,0,...) at tty_wait+0x71 ttydisc_read(c5c2d000,df7d7c28,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c5c4b400,df7d7c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c644f038,df7d7c28,c5985e80,0,c631bb80,...) at devfs_read_f+0x7e dofileread(df7d7c28,ffffffff,ffffffff,0,df7d7c04,...) at dofileread+0x9e kern_readv(c631bb80,0,df7d7c28,df7d7c48,1,...) at kern_readv+0x68 read(c631bb80,df7d7cec,df7d7d28,c0f02f8a,0,...) at read+0x4f syscallenter(c631bb80,df7d7ce4,df7d7ce4,0,0,...) at syscallenter+0x263 syscall(df7d7d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1752 tid 100101 td 0xc63548a0 sched_switch(c63548a0,0,104,191,3071a830,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c63548a0,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,c63548a0,c63548a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5c2d270,0,c0f07364,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5c2d270,c1097470,c0f06d24,524,0,...) at _cv_wait_sig+0x243 tty_wait(c5c2d200,c5c2d270,df7fec28,1,0,...) at tty_wait+0x71 ttydisc_read(c5c2d200,df7fec28,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c5aca300,df7fec28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c624c7e0,df7fec28,c5985e80,0,c63548a0,...) at devfs_read_f+0x7e dofileread(df7fec28,ffffffff,ffffffff,0,df7fec04,...) at dofileread+0x9e kern_readv(c63548a0,0,df7fec28,df7fec48,1,...) at kern_readv+0x68 read(c63548a0,df7fecec,df7fed28,c0f02f8a,0,...) at read+0x4f syscallenter(c63548a0,df7fece4,df7fece4,0,0,...) at syscallenter+0x263 syscall(df7fed28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command getty pid 1751 tid 100074 td 0xc5e6b2e0 sched_switch(c5e6b2e0,0,104,191,30a4bbc8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5e6b2e0,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,c5e6b2e0,c5e6b2e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5c2d470,0,c0f07364,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5c2d470,c1097470,c0f06d24,524,0,...) at _cv_wait_sig+0x243 tty_wait(c5c2d400,c5c2d470,df791c28,1,0,...) at tty_wait+0x71 ttydisc_read(c5c2d400,df791c28,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c5aca400,df791c28,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c624c348,df791c28,c5985e80,0,c5e6b2e0,...) at devfs_read_f+0x7e dofileread(df791c28,ffffffff,ffffffff,0,df791c04,...) at dofileread+0x9e kern_readv(c5e6b2e0,0,df791c28,df791c48,1,...) at kern_readv+0x68 read(c5e6b2e0,df791cec,df791d28,c0f02f8a,0,...) at read+0x4f syscallenter(c5e6b2e0,df791ce4,df791ce4,0,0,...) at syscallenter+0x263 syscall(df791d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, read), eip = 0x28196dc3, esp = 0xbfbfed8c, ebp = 0xbfbfedb8 --- Tracing command inetd pid 1713 tid 100082 td 0xc5e6c5c0 sched_switch(c5e6c5c0,0,104,191,1e58c650,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5e6c5c0,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c09f7eba,c5e6e250,0,c0efb2e7,c5e6c5c0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5e6e264,0,df7b3a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5e6e264,c5e6e250,c0f04cae,64c,c625d818,...) at _cv_wait_sig+0x243 seltdwait(c625d818,58,c5985e80,c5e6c5c0,c0a5254c,...) at seltdwait+0xa2 kern_select(c5e6c5c0,8,bfbfdfa0,0,0,0,20,c62f02c0,c62f02c0) at kern_select+0x504 select(c5e6c5c0,df7b3cec,df7b3d28,c0f02f8a,0,...) at select+0x66 syscallenter(c5e6c5c0,df7b3ce4,df7b3ce4,0,0,...) at syscallenter+0x263 syscall(df7b3d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x281a8d43, esp = 0xbfbfdf5c, ebp = 0xbfbfede8 --- Tracing command watchdogd pid 1685 tid 100092 td 0xc631b2e0 sched_switch(c631b2e0,0,104,191,ae31a410,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c631b2e0,0,c0f02600,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(ea61,c0a472d0,c631b2e0,1,100,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c1098d84,6c,c0efe2df,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c1098d84,0,16c,c0efe2df,ea61,...) at _sleep+0x328 kern_nanosleep(c631b2e0,df7e0c34,df7e0c3c,3c,0,...) at kern_nanosleep+0xc1 nanosleep(c631b2e0,df7e0cec,281920b0,1,0,...) at nanosleep+0x6f syscallenter(c631b2e0,df7e0ce4,c0d3eebd,c1097470,0,...) at syscallenter+0x263 syscall(df7e0d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x281920b7, esp = 0xbfbfecbc, ebp = 0xbfbfece8 --- Tracing command cron pid 1675 tid 100094 td 0xc631ab80 Tracing command sshd pid 1664 tid 100068 td 0xc62408a0 sched_switch(c62408a0,0,104,191,73950bc0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c62408a0,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c09f7eba,c5e6e350,0,c0efb2e7,c62408a0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5e6e364,0,df779a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5e6e364,c5e6e350,c0f04cae,64c,c624c818,...) at _cv_wait_sig+0x243 seltdwait(c624c818,58,c5986b80,c62408a0,c61db000,...) at seltdwait+0xa2 kern_select(c62408a0,5,2880c0b0,0,0,0,20,c623c840,c623c840) at kern_select+0x504 select(c62408a0,df779cec,2880d070,1,0,...) at select+0x66 syscallenter(c62408a0,df779ce4,c0d3eebd,c1097470,0,...) at syscallenter+0x263 syscall(df779d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x284b0d43, esp = 0xbfbfe25c, ebp = 0xbfbfedf8 --- Tracing command rpc.statd pid 1525 tid 100097 td 0xc631a2e0 sched_switch(c631a2e0,0,104,191,20ce9f0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c631a2e0,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(df7f0a14,c09f7eba,c5e5fc10,0,c631a2e0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c5e5fc24,0,df7f0a4c,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c5e5fc24,c5e5fc10,7531,64c,c624c188,...) at _cv_timedwait_sig+0x252 seltdwait(df7f0bf8,df7f0c00,c5985e80,c631a2e0,c61dbbc8,...) at seltdwait+0x8a kern_select(c631a2e0,8,bfbfed14,0,0,df7f0c40,20,1e,0) at kern_select+0x504 select(c631a2e0,df7f0cec,28472048,1,0,...) at select+0x66 syscallenter(c631a2e0,df7f0ce4,c0d3eebd,c1097470,0,...) at syscallenter+0x263 syscall(df7f0d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x2818cd43, esp = 0xbfbfec6c, ebp = 0xbfbfeda8 --- Tracing command md1 pid 1452 tid 100093 td 0xc631b000 sched_switch(c631b000,0,104,191,df73c0a8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c631b000,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c5ea3000,5c,c0ea2001,0,0,...) at sleepq_wait+0x63 _sleep(c5ea3000,c5ea3020,25c,c0ea2001,0,...) at _sleep+0x372 md_kthread(c5ea3000,df7e3d28,c0ef75b0,3a5,c634a2c0,...) at md_kthread+0x11a fork_exit(c0732bf0,c5ea3000,df7e3d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf7e3d60, ebp = 0 --- Tracing command rpcbind pid 1397 tid 100086 td 0xc631c5c0 sched_switch(c631c5c0,0,104,191,43464630,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c631c5c0,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(df7c4a6c,c09f7eba,c5e6e310,0,c631c5c0,...) at sleepq_catch_signals+0xf9 sleepq_timedwait_sig(c5e6e324,0,df7c4aa4,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c5e6e324,c5e6e310,7531,64c,df7c4b5c,...) at _cv_timedwait_sig+0x252 seltdwait(df7c4c2c,df7c4c34,55a,c631c5c0,df7c4b2c,...) at seltdwait+0x8a poll(c631c5c0,df7c4cec,28417048,1,0,...) at poll+0x300 syscallenter(c631c5c0,df7c4ce4,c0d3eebd,c1097470,0,...) at syscallenter+0x263 syscall(df7c4d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (209, FreeBSD ELF32, poll), eip = 0x2813ff4b, esp = 0xbfbfcbac, ebp = 0xbfbfed78 --- Tracing command syslogd pid 1367 tid 100083 td 0xc5e6c2e0 sched_switch(c5e6c2e0,0,104,191,4411ec0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5e6c2e0,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c09f7eba,c5e6e390,0,c0efb2e7,c5e6c2e0,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5e6e3a4,0,df7b7a4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5e6e3a4,c5e6e390,c0f04cae,64c,c625db98,...) at _cv_wait_sig+0x243 seltdwait(c625db98,58,c5985e80,c5e6c2e0,c6442910,...) at seltdwait+0xa2 kern_select(c5e6c2e0,9,284760ac,0,0,0,20,c62f0000,c62f0000) at kern_select+0x504 select(c5e6c2e0,df7b7cec,284760ac,1,0,...) at select+0x66 syscallenter(c5e6c2e0,df7b7ce4,c0d3eebd,c1097470,0,...) at syscallenter+0x263 syscall(df7b7d28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x28199d43, esp = 0xbfbfe29c, ebp = 0xbfbfede8 --- Tracing command devd pid 1188 tid 100100 td 0xc6354b80 sched_switch(c6354b80,0,104,191,74fd46e8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c6354b80,0,c0f02600,1a7,0,...) at sleepq_switch+0x162 sleepq_catch_signals(c09f7eba,c5e6e3d0,0,c0efb2e7,c6354b80,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c5e6e3e4,0,df7fba4c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5e6e3e4,c5e6e3d0,c0f04cae,64c,c624b8f8,...) at _cv_wait_sig+0x243 seltdwait(c624b8f8,58,c5985e80,c6354b80,c6354b80,...) at seltdwait+0xa2 kern_select(c6354b80,5,bfbfe960,0,0,0,20,c63fc580,c63fc580) at kern_select+0x504 select(c6354b80,df7fbcec,df7fbd28,c0f02f8a,0,...) at select+0x66 syscallenter(c6354b80,df7fbce4,df7fbce4,0,0,...) at syscallenter+0x263 syscall(df7fbd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (93, FreeBSD ELF32, select), eip = 0x808ba43, esp = 0xbfbfe92c, ebp = 0xbfbfee08 --- Tracing command md0 pid 101 tid 100105 td 0xc6351b80 sched_switch(c6351b80,0,104,191,fee81340,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c6351b80,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c5ea4000,5c,c0ea2001,0,0,...) at sleepq_wait+0x63 _sleep(c5ea4000,c5ea4020,25c,c0ea2001,0,...) at _sleep+0x372 md_kthread(c5ea4000,edd28d28,c0ef75b0,3a5,c63fb580,...) at md_kthread+0x11a fork_exit(c0732bf0,c5ea4000,edd28d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xedd28d60, ebp = 0 --- Tracing command softdepflush pid 17 tid 100064 td 0xc5e6c000 sched_switch(c5e6c000,0,104,191,e152a5a8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,54,...) at mi_switch+0x1eb sleepq_switch(c5e6c000,0,c0f02600,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c1215640,54,c0f32c1d,0,0,...) at sleepq_timedwait+0x6b _sleep(c1215640,c12155f0,54,c0f32c1d,3e8,...) at _sleep+0x342 softdep_flush(0,df76bd28,c0ef75b0,3a5,c5e61580,...) at softdep_flush+0x266 fork_exit(c0c42460,0,df76bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf76bd60, ebp = 0 --- Tracing command vnlru pid 16 tid 100063 td 0xc5bde2e0 sched_switch(c5bde2e0,0,104,191,c58b6f80,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,60,...) at mi_switch+0x1eb sleepq_switch(c5bde2e0,0,c0f02600,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c5e61840,60,c0f0d48f,0,0,...) at sleepq_timedwait+0x6b _sleep(c5e61840,c120ef34,260,c0f0d48f,3e8,...) at _sleep+0x342 vnlru_proc(0,df768d28,c0ef75b0,3a5,c5e61840,...) at vnlru_proc+0xe7 fork_exit(c0aa5ab0,0,df768d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf768d60, ebp = 0 --- Tracing command syncer pid 9 tid 100062 td 0xc5bde5c0 sched_switch(c5bde5c0,0,104,191,fc97f740,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bde5c0,0,c0f02600,28b,c5bde5c0,...) at sleepq_switch+0x162 sleepq_timedwait(c120ef74,0,df765c78,1,0,...) at sleepq_timedwait+0x6b _cv_timedwait(c120ef74,c120ef60,3e8,76a,4e20,...) at _cv_timedwait+0x252 sched_sync(0,df765d28,c0ef75b0,3a5,c5e61b00,...) at sched_sync+0x595 fork_exit(c0aa3a50,0,df765d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf765d60, ebp = 0 --- Tracing command bufdaemon pid 8 tid 100061 td 0xc5bde8a0 sched_switch(c5bde8a0,0,104,191,a40f0680,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,54,...) at mi_switch+0x1eb sleepq_switch(c5bde8a0,0,c0f02600,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c120ec08,54,c0f0a467,0,0,...) at sleepq_timedwait+0x6b _sleep(c120ec08,c120ec10,54,c0f0a467,3e8,...) at _sleep+0x342 buf_daemon(0,df762d28,c0ef75b0,3a5,c5e62000,...) at buf_daemon+0x175 fork_exit(c0a8a880,0,df762d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf762d60, ebp = 0 --- Tracing command pagezero pid 7 tid 100060 td 0xc5bdeb80 sched_switch(c5bdeb80,0,104,191,5b814478,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bdeb80,0,c0f02600,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c12174dc,0,c0f39624,0,0,...) at sleepq_timedwait+0x6b _sleep(c12174dc,c1215f80,0,c0f39624,493e0,...) at _sleep+0x342 vm_pagezero(0,df75fd28,c0ef75b0,3a5,c59a62c0,...) at vm_pagezero+0xdc fork_exit(c0c86dc0,0,df75fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf75fd60, ebp = 0 --- Tracing command vmdaemon pid 6 tid 100059 td 0xc5c1c000 sched_switch(c5c1c000,0,104,191,880077e0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,74,...) at mi_switch+0x1eb sleepq_switch(c5c1c000,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c12170c8,74,c0f0a467,0,0,...) at sleepq_wait+0x63 _sleep(c12170c8,c12170cc,74,c0f0a467,0,...) at _sleep+0x372 vm_daemon(0,df75cd28,c0ef75b0,3a5,c59a6580,...) at vm_daemon+0x59 fork_exit(c0c80b90,0,df75cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf75cd60, ebp = 0 --- Tracing command pagedaemon pid 5 tid 100058 td 0xc5c1c2e0 sched_switch(c5c1c2e0,0,104,191,39830568,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,54,...) at mi_switch+0x1eb sleepq_switch(c5c1c2e0,0,c0f02600,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c1217090,54,c0f0a467,0,0,...) at sleepq_timedwait+0x6b _sleep(c1217090,c1215f80,54,c0f0a467,1388,...) at _sleep+0x342 vm_pageout(0,df759d28,c0ef75b0,3a5,c59a6840,...) at vm_pageout+0x2c7 fork_exit(c0c81c70,0,df759d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf759d60, ebp = 0 --- Tracing command xpt_thrd pid 4 tid 100057 td 0xc5c1c5c0 sched_switch(c5c1c5c0,0,104,191,ee3fa8f0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c5c1c5c0,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c105dad4,5c,c0d666fd,0,0,...) at sleepq_wait+0x63 _sleep(c105dad4,c105db18,5c,c0d666fd,0,...) at _sleep+0x372 xpt_scanner_thread(0,df756d28,c0ef75b0,3a5,c59a6b00,...) at xpt_scanner_thread+0x47 fork_exit(c0491220,0,df756d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf756d60, ebp = 0 --- Tracing command sctp_iterator pid 3 tid 100056 td 0xc5c1c8a0 sched_switch(c5c1c8a0,0,104,191,977738d0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5c1c8a0,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c1210a98,0,c0f2091f,0,0,...) at sleepq_wait+0x63 _sleep(c1210a98,c1210a60,0,c0f2091f,0,...) at _sleep+0x372 sctp_iterator_thread(0,df753d28,c0ef75b0,3a5,c5b98000,...) at sctp_iterator_thread+0x5c fork_exit(c0b36160,0,df753d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf753d60, ebp = 0 --- Tracing command fdc0 pid 2 tid 100053 td 0xc5c1e2e0 sched_switch(c5c1e2e0,0,104,191,173b0370,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c5c1e2e0,0,c0f02600,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c5b3de3c,5c,c0eefada,0,0,...) at sleepq_timedwait+0x6b _sleep(c5b3de3c,c5b3def0,5c,c0eefada,3e8,...) at _sleep+0x342 fdc_thread(c5b3de00,df74ad28,c0ef75b0,3a5,c5b982c0,...) at fdc_thread+0x27d fork_exit(c0d06070,c5b3de00,df74ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf74ad60, ebp = 0 --- Tracing command usb pid 15 tid 100049 td 0xc5bdb5c0 sched_switch(c5bdb5c0,0,104,191,cd7a7808,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bdb5c0,0,c0f02600,268,c5bdb5c0,...) at sleepq_switch+0x162 sleepq_wait(c5be2d34,0,c5714cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5be2d34,c5be2dd4,c0ed699e,6b,c5be2d3c,...) at _cv_wait+0x243 usb_process(c5be2d2c,c5714d28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5be2d2c,c5714d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5714d60, ebp = 0 --- Tracing command usb pid 15 tid 100048 td 0xc5bdb8a0 sched_switch(c5bdb8a0,0,104,191,ee4c4d88,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bdb8a0,0,c0f02600,268,c5bdb8a0,...) at sleepq_switch+0x162 sleepq_wait(c5be2d04,0,c5711cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5be2d04,c5be2dd4,c0ed699e,6b,c5be2d0c,...) at _cv_wait+0x243 usb_process(c5be2cfc,c5711d28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5be2cfc,c5711d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5711d60, ebp = 0 --- Tracing command usb pid 15 tid 100047 td 0xc5bdbb80 sched_switch(c5bdbb80,0,104,191,ccfc55e8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bdbb80,0,c0f02600,268,c5bdbb80,...) at sleepq_switch+0x162 sleepq_wait(c5be2cd4,0,c570ecac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5be2cd4,c5be2dd4,c0ed699e,6b,c5be2cdc,...) at _cv_wait+0x243 usb_process(c5be2ccc,c570ed28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5be2ccc,c570ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc570ed60, ebp = 0 --- Tracing command usb pid 15 tid 100046 td 0xc5bdd000 sched_switch(c5bdd000,0,104,191,ccfc41a0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bdd000,0,c0f02600,268,c5bdd000,...) at sleepq_switch+0x162 sleepq_wait(c5be2ca4,0,c570bcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5be2ca4,c5be2dd4,c0ed699e,6b,c5be2cac,...) at _cv_wait+0x243 usb_process(c5be2c9c,c570bd28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5be2c9c,c570bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc570bd60, ebp = 0 --- Tracing command usb pid 15 tid 100045 td 0xc5bdd2e0 sched_switch(c5bdd2e0,0,104,191,ccfc2a70,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bdd2e0,0,c0f02600,268,c5bdd2e0,...) at sleepq_switch+0x162 sleepq_wait(c5bccdac,0,c5705cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5bccdac,c5bcce4c,c0ed699e,6b,c5bccdb4,...) at _cv_wait+0x243 usb_process(c5bccda4,c5705d28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5bccda4,c5705d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5705d60, ebp = 0 --- Tracing command usb pid 15 tid 100044 td 0xc5bdd5c0 sched_switch(c5bdd5c0,0,104,191,4dced8d8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bdd5c0,0,c0f02600,268,c5bdd5c0,...) at sleepq_switch+0x162 sleepq_wait(c5bccd7c,0,c5702cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5bccd7c,c5bcce4c,c0ed699e,6b,c5bccd84,...) at _cv_wait+0x243 usb_process(c5bccd74,c5702d28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5bccd74,c5702d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5702d60, ebp = 0 --- Tracing command usb pid 15 tid 100043 td 0xc5bdd8a0 sched_switch(c5bdd8a0,0,104,191,c5e4fcb0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bdd8a0,0,c0f02600,268,c5bdd8a0,...) at sleepq_switch+0x162 sleepq_wait(c5bccd4c,0,c56ffcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5bccd4c,c5bcce4c,c0ed699e,6b,c5bccd54,...) at _cv_wait+0x243 usb_process(c5bccd44,c56ffd28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5bccd44,c56ffd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56ffd60, ebp = 0 --- Tracing command usb pid 15 tid 100042 td 0xc5bddb80 sched_switch(c5bddb80,0,104,191,c5e4e970,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bddb80,0,c0f02600,268,c5bddb80,...) at sleepq_switch+0x162 sleepq_wait(c5bccd1c,0,c56fccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5bccd1c,c5bcce4c,c0ed699e,6b,c5bccd24,...) at _cv_wait+0x243 usb_process(c5bccd14,c56fcd28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5bccd14,c56fcd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56fcd60, ebp = 0 --- Tracing command usb pid 15 tid 100040 td 0xc5bb1000 sched_switch(c5bb1000,0,104,191,c5e4d048,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bb1000,0,c0f02600,268,c5bb1000,...) at sleepq_switch+0x162 sleepq_wait(c5bb9dac,0,c56f5cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5bb9dac,c5bb9e4c,c0ed699e,6b,c5bb9db4,...) at _cv_wait+0x243 usb_process(c5bb9da4,c56f5d28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5bb9da4,c56f5d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56f5d60, ebp = 0 --- Tracing command usb pid 15 tid 100039 td 0xc5bb12e0 sched_switch(c5bb12e0,0,104,191,4d0bf4f0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bb12e0,0,c0f02600,268,c5bb12e0,...) at sleepq_switch+0x162 sleepq_wait(c5bb9d7c,0,c56f2cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5bb9d7c,c5bb9e4c,c0ed699e,6b,c5bb9d84,...) at _cv_wait+0x243 usb_process(c5bb9d74,c56f2d28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5bb9d74,c56f2d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56f2d60, ebp = 0 --- Tracing command usb pid 15 tid 100038 td 0xc5bb15c0 sched_switch(c5bb15c0,0,104,191,c56a4cc0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bb15c0,0,c0f02600,268,c5bb15c0,...) at sleepq_switch+0x162 sleepq_wait(c5bb9d4c,0,c56efcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5bb9d4c,c5bb9e4c,c0ed699e,6b,c5bb9d54,...) at _cv_wait+0x243 usb_process(c5bb9d44,c56efd28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5bb9d44,c56efd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56efd60, ebp = 0 --- Tracing command usb pid 15 tid 100037 td 0xc5bb18a0 sched_switch(c5bb18a0,0,104,191,c56a3a30,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bb18a0,0,c0f02600,268,c5bb18a0,...) at sleepq_switch+0x162 sleepq_wait(c5bb9d1c,0,c56eccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5bb9d1c,c5bb9e4c,c0ed699e,6b,c5bb9d24,...) at _cv_wait+0x243 usb_process(c5bb9d14,c56ecd28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5bb9d14,c56ecd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56ecd60, ebp = 0 --- Tracing command usb pid 15 tid 100035 td 0xc5bb3000 sched_switch(c5bb3000,0,104,191,c56a2338,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bb3000,0,c0f02600,268,c5bb3000,...) at sleepq_switch+0x162 sleepq_wait(c5ba2dac,0,c56e5cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5ba2dac,c5ba2e4c,c0ed699e,6b,c5ba2db4,...) at _cv_wait+0x243 usb_process(c5ba2da4,c56e5d28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5ba2da4,c56e5d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56e5d60, ebp = 0 --- Tracing command usb pid 15 tid 100034 td 0xc5bb32e0 sched_switch(c5bb32e0,0,104,191,4bc78ea8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bb32e0,0,c0f02600,268,c5bb32e0,...) at sleepq_switch+0x162 sleepq_wait(c5ba2d7c,0,c56e2cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5ba2d7c,c5ba2e4c,c0ed699e,6b,c5ba2d84,...) at _cv_wait+0x243 usb_process(c5ba2d74,c56e2d28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5ba2d74,c56e2d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56e2d60, ebp = 0 --- Tracing command usb pid 15 tid 100033 td 0xc5bb35c0 sched_switch(c5bb35c0,0,104,191,c4efb1a0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bb35c0,0,c0f02600,268,c5bb35c0,...) at sleepq_switch+0x162 sleepq_wait(c5ba2d4c,0,c56dfcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5ba2d4c,c5ba2e4c,c0ed699e,6b,c5ba2d54,...) at _cv_wait+0x243 usb_process(c5ba2d44,c56dfd28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5ba2d44,c56dfd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56dfd60, ebp = 0 --- Tracing command usb pid 15 tid 100032 td 0xc5bb38a0 sched_switch(c5bb38a0,0,104,191,c4ef9d00,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5bb38a0,0,c0f02600,268,c5bb38a0,...) at sleepq_switch+0x162 sleepq_wait(c5ba2d1c,0,c56dccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5ba2d1c,c5ba2e4c,c0ed699e,6b,c5ba2d24,...) at _cv_wait+0x243 usb_process(c5ba2d14,c56dcd28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5ba2d14,c56dcd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56dcd60, ebp = 0 --- Tracing command usb pid 15 tid 100030 td 0xc5b1d2e0 sched_switch(c5b1d2e0,0,104,191,c4ef8470,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b1d2e0,0,c0f02600,268,c5b1d2e0,...) at sleepq_switch+0x162 sleepq_wait(c5b88dac,0,c56d5cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5b88dac,c5b88e4c,c0ed699e,6b,c5b88db4,...) at _cv_wait+0x243 usb_process(c5b88da4,c56d5d28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5b88da4,c56d5d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56d5d60, ebp = 0 --- Tracing command usb pid 15 tid 100029 td 0xc5b1d5c0 sched_switch(c5b1d5c0,0,104,191,4c698bb0,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b1d5c0,0,c0f02600,268,c5b1d5c0,...) at sleepq_switch+0x162 sleepq_wait(c5b88d7c,0,c56d2cac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5b88d7c,c5b88e4c,c0ed699e,6b,c5b88d84,...) at _cv_wait+0x243 usb_process(c5b88d74,c56d2d28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5b88d74,c56d2d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56d2d60, ebp = 0 --- Tracing command usb pid 15 tid 100028 td 0xc5b1d8a0 sched_switch(c5b1d8a0,0,104,191,c47416b8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b1d8a0,0,c0f02600,268,c5b1d8a0,...) at sleepq_switch+0x162 sleepq_wait(c5b88d4c,0,c56cfcac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5b88d4c,c5b88e4c,c0ed699e,6b,c5b88d54,...) at _cv_wait+0x243 usb_process(c5b88d44,c56cfd28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5b88d44,c56cfd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56cfd60, ebp = 0 --- Tracing command usb pid 15 tid 100027 td 0xc5b1db80 sched_switch(c5b1db80,0,104,191,c4740340,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b1db80,0,c0f02600,268,c5b1db80,...) at sleepq_switch+0x162 sleepq_wait(c5b88d1c,0,c56cccac,1,0,...) at sleepq_wait+0x63 _cv_wait(c5b88d1c,c5b88e4c,c0ed699e,6b,c5b88d24,...) at _cv_wait+0x243 usb_process(c5b88d14,c56ccd28,c0ef75b0,3a5,c5b98580,...) at usb_process+0x193 fork_exit(c08acc30,c5b88d14,c56ccd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56ccd60, ebp = 0 --- Tracing command yarrow pid 14 tid 100013 td 0xc59a92e0 sched_switch(c59a92e0,0,104,191,11e87f70,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c59a92e0,0,c0f02600,28b,2,...) at sleepq_switch+0x162 sleepq_timedwait(c1097b84,0,c0eefada,2,0,...) at sleepq_timedwait+0x6b _sleep(c1097b84,0,0,c0eefada,64,...) at _sleep+0x342 pause(c0eefada,64,c0eb632e,111,0,...) at pause+0x47 random_kthread(0,c5636d28,c0ef75b0,3a5,c59a5000,...) at random_kthread+0x1ef fork_exit(c07c6b20,0,c5636d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5636d60, ebp = 0 --- Tracing command geom pid 13 tid 100011 td 0xc59a98a0 sched_switch(c59a98a0,0,104,191,30584f18,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c59a98a0,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c10957a8,5c,c0eefada,0,0,...) at sleepq_wait+0x63 _sleep(c10957a8,c1095708,25c,c0eefada,0,...) at _sleep+0x372 g_io_schedule_down(c59a98a0,0,c0ef1679,6c,c562cd14,...) at g_io_schedule_down+0x5b g_down_procbody(0,c562cd28,c0ef75b0,3a5,c59a52c0,...) at g_down_procbody+0x9d fork_exit(c099d200,0,c562cd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc562cd60, ebp = 0 --- Tracing command geom pid 13 tid 100010 td 0xc59a9b80 sched_switch(c59a9b80,0,104,191,305c0ca8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c59a9b80,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c10957a4,5c,c0eefada,0,0,...) at sleepq_wait+0x63 _sleep(c10957a4,c1095728,25c,c0eefada,0,...) at _sleep+0x372 g_io_schedule_up(c59a9b80,0,c0ef1679,5f,c5629d14,...) at g_io_schedule_up+0x134 g_up_procbody(0,c5629d28,c0ef75b0,3a5,c59a52c0,...) at g_up_procbody+0x9d fork_exit(c099d2a0,0,c5629d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5629d60, ebp = 0 --- Tracing command geom pid 13 tid 100009 td 0xc59ef000 sched_switch(c59ef000,0,104,191,8243ba98,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,5c,...) at mi_switch+0x1eb sleepq_switch(c59ef000,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c109579c,5c,c0eefada,0,0,...) at sleepq_wait+0x63 _sleep(c109579c,c10956cc,25c,c0eefada,0,...) at _sleep+0x372 g_run_events(c10a5a00,0,c0ef1679,79,c5626d14,...) at g_run_events+0x526 g_event_procbody(0,c5626d28,c0ef75b0,3a5,c59a52c0,...) at g_event_procbody+0x90 fork_exit(c099d340,0,c5626d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5626d60, ebp = 0 --- Tracing command intr pid 12 tid 100054 td 0xc5c1e000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100052 td 0xc5c1e5c0 sched_switch(c5c1e5c0,0,109,191,6563daf8,...) at sched_switch+0x3bc mi_switch(109,0,c0ef78eb,554,c5c1e5c0,...) at mi_switch+0x1eb ithread_loop(c59ee710,df73dd28,c0ef75b0,3a5,c59a5580,...) at ithread_loop+0x1fb fork_exit(c09de980,c59ee710,df73dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf73dd60, ebp = 0 --- Tracing command intr pid 12 tid 100051 td 0xc5bdb000 sched_switch(c5bdb000,0,109,191,305b82a0,...) at sched_switch+0x3bc mi_switch(109,0,c0ef78eb,554,c5bdb000,...) at mi_switch+0x1eb ithread_loop(c59ee720,c573ed28,c0ef75b0,3a5,c59a5580,...) at ithread_loop+0x1fb fork_exit(c09de980,c59ee720,c573ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc573ed60, ebp = 0 --- Tracing command intr pid 12 tid 100050 td 0xc5bdb2e0 sched_switch(c5bdb2e0,0,109,191,51d04f20,...) at sched_switch+0x3bc mi_switch(109,0,c0ef78eb,554,c5bdb2e0,...) at mi_switch+0x1eb ithread_loop(c59ee730,c573ad28,c0ef75b0,3a5,c59a5580,...) at ithread_loop+0x1fb fork_exit(c09de980,c59ee730,c573ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc573ad60, ebp = 0 --- Tracing command intr pid 12 tid 100041 td 0xc5bde000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100036 td 0xc5bb1b80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100031 td 0xc5bb3b80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100026 td 0xc5b20000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100023 td 0xc5b208a0 sched_switch(c5b208a0,0,109,191,7fc3ac0,...) at sched_switch+0x3bc mi_switch(109,0,c0ef78eb,554,c5b208a0,...) at mi_switch+0x1eb ithread_loop(c59ee780,c5654d28,c0ef75b0,3a5,c59a5580,...) at ithread_loop+0x1fb fork_exit(c09de980,c59ee780,c5654d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5654d60, ebp = 0 --- Tracing command intr pid 12 tid 100022 td 0xc5b20b80 sched_switch(c5b20b80,0,109,191,305bc1c0,...) at sched_switch+0x3bc mi_switch(109,0,c0ef78eb,554,c5b20b80,...) at mi_switch+0x1eb ithread_loop(c59ee790,c5651d28,c0ef75b0,3a5,c59a5580,...) at ithread_loop+0x1fb fork_exit(c09de980,c59ee790,c5651d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5651d60, ebp = 0 --- Tracing command intr pid 12 tid 100017 td 0xc5b1c5c0 sched_switch(c5b1c5c0,0,109,191,8645bf0,...) at sched_switch+0x3bc mi_switch(109,0,c0ef78eb,554,c5b1c5c0,...) at mi_switch+0x1eb ithread_loop(c59ee7a0,c5642d28,c0ef75b0,3a5,c59a5580,...) at ithread_loop+0x1fb fork_exit(c09de980,c59ee7a0,c5642d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5642d60, ebp = 0 --- Tracing command intr pid 12 tid 100015 td 0xc5b1cb80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100008 td 0xc59ef2e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100007 td 0xc59ef5c0 sched_switch(c59ef5c0,0,109,191,fc97ccf0,...) at sched_switch+0x3bc mi_switch(109,0,c0ef78eb,554,c59ef5c0,...) at mi_switch+0x1eb ithread_loop(c59ee7d0,c5620d28,c0ef75b0,3a5,c59a5580,...) at ithread_loop+0x1fb fork_exit(c09de980,c59ee7d0,c5620d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5620d60, ebp = 0 --- Tracing command intr pid 12 tid 100006 td 0xc59a7000 sched_switch(c59a7000,0,109,191,19c43a20,...) at sched_switch+0x3bc mi_switch(109,0,c0ef78eb,554,c59a7000,...) at mi_switch+0x1eb ithread_loop(c59ee7e0,c561dd28,c0ef75b0,3a5,c59a5580,...) at ithread_loop+0x1fb fork_exit(c09de980,c59ee7e0,c561dd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc561dd60, ebp = 0 --- Tracing command intr pid 12 tid 100005 td 0xc59a72e0 sched_switch(c59a72e0,0,109,191,4c9ee1e0,...) at sched_switch+0x3bc mi_switch(109,0,c0ef78eb,554,c59a72e0,...) at mi_switch+0x1eb ithread_loop(c59ee7f0,c561ad28,c0ef75b0,3a5,c59a5580,...) at ithread_loop+0x1fb fork_exit(c09de980,c59ee7f0,c561ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc561ad60, ebp = 0 --- Tracing command idle pid 11 tid 100004 td 0xc59a75c0 sched_switch(c59a75c0,0,108,18c,e57ba7c8,...) at sched_switch+0x3bc mi_switch(108,0,c0efede3,3b9,c59a75c0,...) at mi_switch+0x1eb sched_idletd(0,c5614d28,c0ef75b0,3a5,c59a5840,...) at sched_idletd+0x19f fork_exit(c0a2dd60,0,c5614d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5614d60, ebp = 0 --- Tracing command idle pid 11 tid 100003 td 0xc59a78a0 sched_switch(c59a78a0,0,108,18c,fe39d9b8,...) at sched_switch+0x3bc mi_switch(108,0,c0efede3,3b9,c59a78a0,...) at mi_switch+0x1eb sched_idletd(0,c5611d28,c0ef75b0,3a5,c59a5840,...) at sched_idletd+0x19f fork_exit(c0a2dd60,0,c5611d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5611d60, ebp = 0 --- Tracing command init pid 1 tid 100002 td 0xc59a7b80 sched_switch(c59a7b80,0,104,191,44aa6078,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,6c,...) at mi_switch+0x1eb sleepq_switch(c59a7b80,0,c0f02600,1a7,6c,...) at sleepq_switch+0x162 sleepq_catch_signals(c0f02600,160,0,100,100,...) at sleepq_catch_signals+0xf9 sleepq_wait_sig(c59a5b00,6c,c0f0545a,100,0,...) at sleepq_wait_sig+0x17 _sleep(c59a5b00,c59a5b88,16c,c0f0545a,0,...) at _sleep+0x35c kern_wait(c59a7b80,ffffffff,c560dc44,0,0,...) at kern_wait+0xb96 wait4(c59a7b80,c560dcec,835,1,0,...) at wait4+0x3b syscallenter(c59a7b80,c560dce4,c0d3eebd,c1097470,0,...) at syscallenter+0x263 syscall(c560dd28) at syscall+0x4f Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x80551d3, esp = 0xbfbfe8bc, ebp = 0xbfbfe8d8 --- Tracing command audit pid 10 tid 100001 td 0xc59a9000 sched_switch(c59a9000,0,104,191,c4707d38,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c59a9000,0,c0f02600,268,c59a9000,...) at sleepq_switch+0x162 sleepq_wait(c1214d60,0,c560ac8c,1,0,...) at sleepq_wait+0x63 _cv_wait(c1214d60,c1214d44,c0f2d4ef,194,0,...) at _cv_wait+0x243 audit_worker(0,c560ad28,c0ef75b0,3a5,c59a6000,...) at audit_worker+0x84 fork_exit(c0c02210,0,c560ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc560ad60, ebp = 0 --- Tracing command kernel pid 0 tid 100065 td 0xc5e6bb80 sched_switch(c5e6bb80,0,104,191,5288cf20,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5e6bb80,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c5c02180,0,c0eefada,0,0,...) at sleepq_wait+0x63 _sleep(c5c02180,c5c02198,0,c0eefada,0,...) at _sleep+0x372 taskqueue_thread_loop(c1277120,df76ed28,c0ef75b0,3a5,c1095880,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a4a9b0,c1277120,df76ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf76ed60, ebp = 0 --- Tracing command kernel pid 0 tid 100055 td 0xc5c1cb80 sched_switch(c5c1cb80,0,104,191,5f101368,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5c1cb80,0,c0f02600,28b,2,...) at sleepq_switch+0x162 sleepq_timedwait(c1097b84,0,c0eefada,2,0,...) at sleepq_timedwait+0x6b _sleep(c1097b84,0,0,c0eefada,bb8,...) at _sleep+0x342 pause(c0eefada,bb8,124,122,c1095908,...) at pause+0x47 deadlkres(0,df750d28,c0ef75b0,3a5,c1095880,...) at deadlkres+0x324 fork_exit(c09bed70,0,df750d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xdf750d60, ebp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xc5b202e0 sched_switch(c5b202e0,0,104,191,c46d5de8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b202e0,0,c0f02600,268,c1095b40,...) at sleepq_switch+0x162 sleepq_wait(c5b51740,0,c0efdad9,c0eefada,0,...) at sleepq_wait+0x63 msleep_spin(c5b51740,c5b51758,c0eefada,0,c0efb2e7,...) at msleep_spin+0x21d taskqueue_thread_loop(c5b74568,c56c5d28,c0ef75b0,3a5,c1095880,...) at taskqueue_thread_loop+0x8f fork_exit(c0a4a9b0,c5b74568,c56c5d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56c5d60, ebp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xc5b205c0 sched_switch(c5b205c0,0,104,191,13fbcad8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b205c0,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c5b51980,0,c0efdad9,c0eefada,0,...) at sleepq_wait+0x63 msleep_spin(c5b51980,c5b51998,c0eefada,0,c0efb2e7,...) at msleep_spin+0x21d taskqueue_thread_loop(c5b5f568,c569ad28,c0ef75b0,3a5,c1095880,...) at taskqueue_thread_loop+0x8f fork_exit(c0a4a9b0,c5b5f568,c569ad28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc569ad60, ebp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xc59ef8a0 sched_switch(c59ef8a0,0,104,191,ee41b0e8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c59ef8a0,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c59a8100,0,c0efdad9,c0eefada,0,...) at sleepq_wait+0x63 msleep_spin(c59a8100,c59a8118,c0eefada,0,c0efb2e7,...) at msleep_spin+0x21d taskqueue_thread_loop(c10609b8,c564ed28,c0ef75b0,3a5,c1095880,...) at taskqueue_thread_loop+0x8f fork_exit(c0a4a9b0,c10609b8,c564ed28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc564ed60, ebp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xc59efb80 sched_switch(c59efb80,0,104,191,ee41a170,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c59efb80,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c59a8100,0,c0efdad9,c0eefada,0,...) at sleepq_wait+0x63 msleep_spin(c59a8100,c59a8118,c0eefada,0,c0efb2e7,...) at msleep_spin+0x21d taskqueue_thread_loop(c10609b8,c564bd28,c0ef75b0,3a5,c1095880,...) at taskqueue_thread_loop+0x8f fork_exit(c0a4a9b0,c10609b8,c564bd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc564bd60, ebp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xc5b1c000 sched_switch(c5b1c000,0,104,191,ee419118,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b1c000,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c59a8100,0,c0efdad9,c0eefada,0,...) at sleepq_wait+0x63 msleep_spin(c59a8100,c59a8118,c0eefada,0,c0efb2e7,...) at msleep_spin+0x21d taskqueue_thread_loop(c10609b8,c5648d28,c0ef75b0,3a5,c1095880,...) at taskqueue_thread_loop+0x8f fork_exit(c0a4a9b0,c10609b8,c5648d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5648d60, ebp = 0 --- Tracing command kernel pid 0 tid 100018 td 0xc5b1c2e0 sched_switch(c5b1c2e0,0,104,191,ee402ef8,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b1c2e0,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c59a8140,0,c0eefada,0,0,...) at sleepq_wait+0x63 _sleep(c59a8140,c59a8158,0,c0eefada,0,...) at _sleep+0x372 taskqueue_thread_loop(c1096218,c5645d28,c0ef75b0,3a5,c1095880,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a4a9b0,c1096218,c5645d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5645d60, ebp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xc5b1c8a0 sched_switch(c5b1c8a0,0,104,191,ee401d00,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b1c8a0,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c59a81c0,0,c0eefada,0,0,...) at sleepq_wait+0x63 _sleep(c59a81c0,c59a81d8,0,c0eefada,0,...) at _sleep+0x372 taskqueue_thread_loop(c1215344,c563fd28,c0ef75b0,3a5,c1095880,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a4a9b0,c1215344,c563fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc563fd60, ebp = 0 --- Tracing command kernel pid 0 tid 100014 td 0xc5b1d000 sched_switch(c5b1d000,0,104,191,d5facf60,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c5b1d000,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c59a84c0,0,c0eefada,0,0,...) at sleepq_wait+0x63 _sleep(c59a84c0,c59a84d8,0,c0eefada,0,...) at _sleep+0x372 taskqueue_thread_loop(c10b4788,c5639d28,c0ef75b0,3a5,c1095880,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a4a9b0,c10b4788,c5639d28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5639d60, ebp = 0 --- Tracing command kernel pid 0 tid 100012 td 0xc59a95c0 sched_switch(c59a95c0,0,104,191,f7ad650,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,0,...) at mi_switch+0x1eb sleepq_switch(c59a95c0,0,c0f02600,268,0,...) at sleepq_switch+0x162 sleepq_wait(c59a8500,0,c0eefada,0,0,...) at sleepq_wait+0x63 _sleep(c59a8500,c59a8518,0,c0eefada,0,...) at _sleep+0x372 taskqueue_thread_loop(c10b31a0,c562fd28,c0ef75b0,3a5,c1095880,...) at taskqueue_thread_loop+0xb5 fork_exit(c0a4a9b0,c10b31a0,c562fd28) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc562fd60, ebp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xc1095b40 sched_switch(c1095b40,0,104,191,fa504f90,...) at sched_switch+0x3bc mi_switch(104,0,c0f02600,1f3,54,...) at mi_switch+0x1eb sleepq_switch(c1095b40,0,c0f02600,28b,0,...) at sleepq_switch+0x162 sleepq_timedwait(c1095880,54,c0eff6fd,0,0,...) at sleepq_timedwait+0x6b _sleep(c1095880,0,54,c0eff6fd,2710,...) at _sleep+0x342 scheduler(0,181ec00,181ec00,181e000,1825000,...) at scheduler+0x240 mi_startup() at mi_startup+0xac begin() at begin+0x2c db:0:allt> call doadump Physical memory: 2026 MB Physical memory: 2026 MB Dumping 117 MB:Dumping 117 MB: 102 102 86 86 70 70 54 54 38 38 22 22 6 6 Dump complete Dump complete = 0 db:0:doadump> reset cpu_reset: Restarting BSP cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 1 cpu_reset_proxy: Stopped CPU 1 (kgdb) bt #0 doadump (textdump=-300550244) at pcpu.h:244 #1 0xc04e41a9 in db_fncall (dummy1=-1063009526, dummy2=0, dummy3=-1, dummy4=0xee15f830 "Dø\025î") at /mnt/pho/src/sys/ddb/db_command.c:572 #2 0xc04e45df in db_command (last_cmdp=0xc105eb7c, cmd_table=0x0, dopager=0) at /mnt/pho/src/sys/ddb/db_command.c:448 #3 0xc04e4694 in db_command_script (command=0xc105fa88 "call doadump") at /mnt/pho/src/sys/ddb/db_command.c:519 #4 0xc04e89d0 in db_script_exec (scriptname=0xc105f3e0 "doadump", warnifnotfound=Variable "warnifnotfound" is not available. ) at /mnt/pho/src/sys/ddb/db_script.c:302 #5 0xc04e8a61 in db_run_cmd (addr=-1056532704, have_addr=0, count=-1067130183, modif=0xee15f968 "") at /mnt/pho/src/sys/ddb/db_script.c:375 #6 0xc04e45a1 in db_command (last_cmdp=0xc105eb7c, cmd_table=0x0, dopager=1) at /mnt/pho/src/sys/ddb/db_command.c:448 #7 0xc04e46fa in db_command_loop () at /mnt/pho/src/sys/ddb/db_command.c:501 #8 0xc04e66ed in db_trap (type=3, code=0) at /mnt/pho/src/sys/ddb/db_main.c:229 #9 0xc0a3c4d3 in kdb_trap (type=3, code=0, tf=0xee15fb18) at /mnt/pho/src/sys/kern/subr_kdb.c:539 #10 0xc0d3edab in trap (frame=0xee15fb18) at /mnt/pho/src/sys/i386/i386/trap.c:719 #11 0xc0d27c8c in calltrap () at /mnt/pho/src/sys/i386/i386/exception.s:168 #12 0xc0a3c30a in kdb_enter (why=0xc0efcd52 "panic", msg=0xc0efcd52 "panic") at cpufunc.h:71 #13 0xc0a07854 in panic (fmt=0xc0f0206a "attempt to create a too small sbuf") at /mnt/pho/src/sys/kern/kern_shutdown.c:587 #14 0xc0a44a97 in sbuf_newbuf (s=0xee15fbf4, buf=0xc5e43770 "", length=1, flags=0) at /mnt/pho/src/sys/kern/subr_sbuf.c:187 #15 0xc0a44b56 in sbuf_new (s=0xee15fbf4, buf=0xc5e43770 "", length=1, flags=0) at /mnt/pho/src/sys/kern/subr_sbuf.c:220 #16 0xc0c036fa in mac_cred_externalize_label (label=0x0, elements=0xc5e43780 "", outbuf=0xc5e43770 "", outbuflen=1) at /mnt/pho/src/sys/security/mac/mac_cred.c:161 #17 0xc0c0c5d0 in __mac_get_proc (td=0xc6c8a000, uap=0xee15fcec) at /mnt/pho/src/sys/security/mac/mac_syscalls.c:151 #18 0xc0a4ad73 in syscallenter (td=0xc6c8a000, sa=0xee15fce4) at /mnt/pho/src/sys/kern/subr_trap.c:344 #19 0xc0d3e65f in syscall (frame=0xee15fd28) at /mnt/pho/src/sys/i386/i386/trap.c:1082 #20 0xc0d27cf1 in Xint0x80_syscall () at /mnt/pho/src/sys/i386/i386/exception.s:266 #21 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) f 17 #17 0xc0c0c5d0 in __mac_get_proc (td=0xc6c8a000, uap=0xee15fcec) at /mnt/pho/src/sys/security/mac/mac_syscalls.c:151 151 error = mac_cred_externalize_label(td->td_ucred->cr_label, (kgdb) l 146 free(elements, M_MACTEMP); 147 return (error); 148 } 149 150 buffer = malloc(mac.m_buflen, M_MACTEMP, M_WAITOK | M_ZERO); 151 error = mac_cred_externalize_label(td->td_ucred->cr_label, 152 elements, buffer, mac.m_buflen); 153 if (error == 0) 154 error = copyout(buffer, mac.m_string, strlen(buffer)+1); 155 (kgdb) p mac $1 = {m_buflen = 1, m_string = 0x82444c7
} (kgdb) -- Test scenario: run.sh syscall.cfg