KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2010 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-CURRENT #0: Fri Apr 23 05:18:32 UTC 2010 root@worm2.netperf.freebsd.org:/usr/src/sys/amd64/compile/PHO amd64 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Quad-Core AD Opteron(tm) Processor 8354 (2211.34-Hz K8-class CPU) Origin = "AuthenticAD" Id = 0x100f23 Family = 10 odel = 2 Stepping = 3 Features=0x178bfbff Features2=0x802009 AD Features=0xee500800 AD Features2=0x7ff TSC: P-state invariant real memory = 18119393280 (17280 B) avail memory = 16531505152 (15765 B) ACPI APIC Table: FreeBSD/SP: ultiprocessor System Detected: 16 CPUs FreeBSD/SP: 4 package(s) x 4 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 cpu4 (AP): APIC ID: 4 cpu5 (AP): APIC ID: 5 cpu6 (AP): APIC ID: 6 cpu7 (AP): APIC ID: 7 cpu8 (AP): APIC ID: 8 cpu9 (AP): APIC ID: 9 cpu10 (AP): APIC ID: 10 cpu11 (AP): APIC ID: 11 cpu12 (AP): APIC ID: 12 cpu13 (AP): APIC ID: 13 cpu14 (AP): APIC ID: 14 cpu15 (AP): APIC ID: 15 ioapic0: Changing APIC ID to 16 ioapic1: Changing APIC ID to 17 ioapic2: Changing APIC ID to 18 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 40-46 on motherboard ioapic2 irqs 47-53 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) acpi0: reservation of fec00000, 1000 (3) failed acpi0: reservation of fee00000, 1000 (3) failed acpi0: reservation of e0000000, 10000000 (3) failed acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, c7f00000 (3) failed Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545Hz> port 0x2008-0x200b on acpi0 cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 cpu4: on acpi0 cpu5: on acpi0 cpu6: on acpi0 cpu7: on acpi0 cpu8: on acpi0 cpu9: on acpi0 cpu10: on acpi0 cpu11: on acpi0 cpu12: on acpi0 cpu13: on acpi0 cpu14: on acpi0 cpu15: on acpi0 acpi_hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 25000000 Hz quality 900 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pci0: at device 0.0 (no driver attached) isab0: at device 1.0 on pci0 isa0: on isab0 pci0: at device 1.1 (no driver attached) ohci0: mem 0xdeebf000-0xdeebffff irq 21 at device 2.0 on pci0 ohci0: [ITHREAD] usbus0: on ohci0 ehci0: mem 0xdeebec00-0xdeebecff irq 22 at device 2.1 on pci0 ehci0: [ITHREAD] usbus1: EHCI version 1.0 usbus1: on ehci0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xffa0-0xffaf at device 4.0 on pci0 ata0: on atapci0 ata0: [ITHREAD] ata1: on atapci0 ata1: [ITHREAD] atapci1: port 0xc480-0xc487,0xc400-0xc403,0xc080-0xc087,0xc000-0xc003,0xbc00-0xbc0f mem 0xdeebd000-0xdeebdfff irq 23 at device 5.0 on pci0 atapci1: [ITHREAD] ata2: on atapci1 ata2: [ITHREAD] ata3: on atapci1 ata3: [ITHREAD] atapci2: port 0xb880-0xb887,0xb800-0xb803,0xb480-0xb487,0xb400-0xb403,0xb080-0xb08f mem 0xdeebc000-0xdeebcfff irq 20 at device 5.1 on pci0 atapci2: [ITHREAD] ata4: on atapci2 ata4: [ITHREAD] ata5: on atapci2 ata5: [ITHREAD] atapci3: port 0xb000-0xb007,0xac00-0xac03,0xa880-0xa887,0xa800-0xa803,0xa480-0xa48f mem 0xdeebb000-0xdeebbfff irq 21 at device 5.2 on pci0 atapci3: [ITHREAD] ata6: on atapci3 ata6: [ITHREAD] ata7: on atapci3 ata7: [ITHREAD] pcib1: at device 6.0 on pci0 pci1: on pcib1 vgapci0: port 0xd000-0xd0ff mem 0xd0000000-0xd7ffffff,0xdeff0000-0xdeffffff irq 16 at device 1.0 on pci1 pcib2: at device 10.0 on pci0 pci2: on pcib2 cxgbc0: mem 0xdffff000-0xdfffffff,0xdf000000-0xdf7fffff,0xdfffe000-0xdfffefff irq 17 at device 0.0 on pci2 cxgbc0: using SI-X interrupts (9 vectors) cxgb0: on cxgbc0 cxgb0: Ethernet address: 00:07:43:05:ea:17 cxgb1: on cxgbc0 cxgb1: Ethernet address: 00:07:43:05:ea:18 cxgbc0: Firmware Version 7.8.0 pcib3: at device 11.0 on pci0 pci3: on pcib3 pcib4: at device 12.0 on pci0 pci4: on pcib4 pcib5: at device 13.0 on pci0 pci5: on pcib5 pcib6: at device 14.0 on pci0 pci6: on pcib6 pcib7: at device 15.0 on pci0 pci7: on pcib7 pcib8: on acpi0 pci8: on pcib8 pcib9: at device 1.0 on pci8 pci9: on pcib9 pcib10: at device 2.0 on pci8 pci10: on pcib10 em0: port 0xec00-0xec3f mem 0xcffe0000-0xcfffffff irq 47 at device 3.0 on pci10 em0: [FILTER] em0: Ethernet address: 00:30:48:bf:aa:aa em1: port 0xe880-0xe8bf mem 0xcffa0000-0xcffbffff irq 48 at device 3.1 on pci10 em1: [FILTER] em1: Ethernet address: 00:30:48:bf:aa:ab acpi_button0: on acpi0 atrtc0: port 0x70-0x71 irq 8 on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: [FILTER] uart0: console (115200,n,8,1) uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 uart1: [FILTER] fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: [FILTER] ppc0: port 0x378-0x37f irq 7 on acpi0 ppc0: Generic chipset (NIBBLE-only) in COPATIBLE mode ppc0: [ITHREAD] ppbus0: on ppc0 plip0: on ppbus0 plip0: [ITHREAD] lpt0: on ppbus0 lpt0: [ITHREAD] lpt0: Interrupt-driven port ppi0: on ppbus0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] orm0: at iomem 0xc0000-0xcafff,0xcb000-0xcbfff,0xcc000-0xccfff,0xcd000-0xcdfff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 hwpstate0: on cpu0 Timecounters tick every 1.000 msec usbus0: 12bps Full Speed USB v1.0 usbus1: 480bps High Speed USB v2.0 ugen0.1: at usbus0 uhub0: on usbus0 Expensive timeout(9) function: 0xffffffff805e68a0(0xffffff0002ff3720) 0.011847594 s ugen1.1: at usbus1 uhub1: on usbus1 uhub0: 10 ports with 10 removable, self powered acd0: DVDRO at ata0-slave UDA33 SP: AP CPU #2 Launched! SP: AP CPU #1 Launched! SP: AP CPU #3 Launched! SP: AP CPU #7 Launched! SP: AP CPU #5 Launched! SP: AP CPU #6 Launched! SP: AP CPU #8 Launched! SP: AP CPU #12 Launched! SP: AP CPU #10 Launched! SP: AP CPU #9 Launched! SP: AP CPU #15 Launched! SP: AP CPU #4 Launched! SP: AP CPU #11 Launched! SP: AP CPU #14 Launched! SP: AP CPU #13 Launched! WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. Root mount waiting for: usbus1 Root mount waiting for: usbus1 Root mount waiting for: usbus1 Root mount waiting for: usbus1 uhub1: 10 ports with 10 removable, self powered Root mount waiting for: usbus1 ugen1.2: at usbus1 ums0: on usbus1 ums0: 3 buttons and [Z] coordinates ID=0 ukbd0: on usbus1 kbd2 at ukbd0 Trying to mount root from nfs: NFS ROOT: 192.168.5.1:/zoo/worm2/ em0: link state changed to UP Interface em0 IP-Address 192.168.5.121 Broadcast 192.168.5.255 Setting hostuuid: 00020003-0004-0005-0006-000700080009. Setting hostid: 0x81f4ec68. No suitable dump device was found. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: mount_nfs: can't update /var/db/mounttab for 192.168.5.1:/zoo/worm2 ounting local file systems:. Setting hostname: worm2.netperf.freebsd.org. Starting Network: lo0 cxgb0 cxgb1 em0 em1 plip0. lo0: flags=8049 metric 0 mtu 16384 options=3 inet 127.0.0.1 netmask 0xff000000 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6 nd6 options=21 cxgb0: flags=8802 metric 0 mtu 9000 options=405bb ether 00:07:43:05:ea:17 media: Ethernet 10Gbase-SR status: no carrier cxgb1: flags=8802 metric 0 mtu 9000 options=405bb ether 00:07:43:05:ea:18 media: Ethernet 10Gbase-SR status: no carrier em0: flags=8843 metric 0 mtu 1500 options=389b ether 00:30:48:bf:aa:aa inet 192.168.5.121 netmask 0xffffff00 broadcast 192.168.5.255 inet6 fe80::230:48ff:febf:aaaa%em0 prefixlen 64 tentative scopeid 0x3 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active em1: flags=8802 metric 0 mtu 1500 options=389b ether 00:30:48:bf:aa:ab media: Ethernet autoselect status: no carrier plip0: flags=8810 metric 0 mtu 1500 Starting devd. Starting Network: cxgb0. cxgb0: flags=8802 metric 0 mtu 9000 options=405bb ether 00:07:43:05:ea:17 media: Ethernet 10Gbase-SR status: no carrier Starting Network: cxgb1. cxgb1: flags=8802 metric 0 mtu 9000 options=405bb ether 00:07:43:05:ea:18 media: Ethernet 10Gbase-SR status: no carrier Starting Network: em1. em1: flags=8802 metric 0 mtu 1500 options=389b ether 00:30:48:bf:aa:ab media: Ethernet autoselect status: no carrier Starting Network: plip0. plip0: flags=8810 metric 0 mtu 1500 Starting ums0 moused. DHCPDISCOVER on em0 to 255.255.255.255 port 67 interval 3 DHCPOFFER from 192.168.5.1 DHCPREQUEST on em0 to 255.255.255.255 port 67 DHCPACK from 192.168.5.1 ifaddr cache = 0xffffff000756bb00 is deleted bound to 192.168.5.121 -- renewal in 300 seconds. add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 Waiting 30s for the default route interface: ............................. Creating and/or trimming log files. Starting syslogd. /etc/rc: WARNING: Dump device does not exist. Savecore not run. ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib 32-bit compatibility ldconfig path: /usr/lib32 Starting rpcbind. NFS access cache time=60 Clearing /tmp (X related). Starting statd. Starting lockd. Updating motd:. Configuring syscons: blanktime. Starting sshd. Starting cron. Starting background file system checks in 60 seconds. Sun Apr 25 10:59:23 UTC 2010 FreeBSD/amd64 (worm2.netperf.freebsd.org) (ttyu0) login: Apr 25 11:00:31 worm2 su: test to root on /dev/pts/0 lock order reversal: 1st 0xffffff814aaac988 bufwait (bufwait) @ kern/vfs_bio.c:2559 2nd 0xffffff0007e34800 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:285 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 ufsdirhash_acquire() at ufsdirhash_acquire+0x44 ufsdirhash_add() at ufsdirhash_add+0x19 ufs_direnter() at ufs_direnter+0x889 ufs_makeinode() at ufs_makeinode+0x239 VOP_CREATE_APV() at VOP_CREATE_APV+0xb3 vn_open_cred() at vn_open_cred+0x46b kern_openat() at kern_openat+0x179 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (5, FreeBSD ELF64, open), rip = 0x800735e7c, rsp = 0x7fffffffe2b8, rbp = 0x7fffffffe770 --- Apr 25 11:04:41 worm2 kernel: pid 2061 (fstool), uid 0 inumber 1859697 on /mnt: out of inodes lock order reversal: 1st 0xffffff0007b12098 nfs (nfs) @ kern/vfs_mount.c:1201 2nd 0xffffff0007ee6098 devfs (devfs) @ ufs/ffs/ffs_vfsops.c:1232 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e __lockmgr_args() at __lockmgr_args+0xd32 vop_stdlock() at vop_stdlock+0x39 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x57 ffs_flushfiles() at ffs_flushfiles+0xc5 ffs_unmount() at ffs_unmount+0x67 dounmount() at dounmount+0x2e6 unmount() at unmount+0x27e syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (22, FreeBSD ELF64, unmount), rip = 0x8006a3a6c, rsp = 0x7fffffffe0f8, rbp = 0x800c08360 --- Apr 25 11:28:40 worm2 kernel: pid 2581 (cp), uid 0 inumber 284765 on /mnt: filesystem full Apr 25 11:28:40 worm2 kernel: pid 2579 (cp), uid 0 inumber 332246 on /mnt: filesystem full Apr 25 13:03:04 worm2 kernel: pid 49966 (fstool), uid 0 inumber 1859995 on /mnt: out of inodes Apr 25 13:26:54 worm2 kernel: pid 50468 (cp), uid 0 inumber 144046 on /mnt: filesystem full Apr 25 13:26:55 worm2 kernel: pid 50468 (cp), uid 0 inumber 96261 on /mnt: filesystem full Apr 25 15:00:50 worm2 kernel: pid 97721 (fstool), uid 0 inumber 1859356 on /mnt: out of inodes Apr 25 15:24:39 worm2 kernel: pid 98257 (cp), uid 0 inumber 636459 on /mnt: filesystem full Apr 25 15:24:39 worm2 kernel: pid 98256 (cp), uid 0 inumber 97517 on /mnt: filesystem full lock order reversal: 1st 0xffffff01a631ca78 ufs (ufs) @ kern/vfs_vnops.c:611 2nd 0xffffff814ad6f288 bufwait (bufwait) @ kern/vfs_bio.c:1835 3rd 0xffffff019f76f310 ufs (ufs) @ kern/vfs_subr.c:2091 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e __lockmgr_args() at __lockmgr_args+0xdae ffs_lock() at ffs_lock+0x9c VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x57 vget() at vget+0x7b vm_fault() at vm_fault+0x1606 trap_pfault() at trap_pfault+0x118 trap() at trap+0x37f calltrap() at calltrap+0x8 --- trap 0xc, rip = 0xffffffff80897c5d, rsp = 0xffffff81d01707c0, rbp = 0xffffff81d0170840 --- copyin() at copyin+0x3d ffs_write() at ffs_write+0x32d VOP_WRITE_APV() at VOP_WRITE_APV+0x129 vn_write() at vn_write+0x2a2 dofilewrite() at dofilewrite+0x85 kern_writev() at kern_writev+0x60 write() at write+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip = 0x800748fec, rsp = 0x7fffffffe6c8, rbp = 0x80053e000 --- Apr 25 16:58:29 worm2 kernel: pid 45661 (fstool), uid 0 inumber 1859387 on /mnt: out of inodes Apr 25 17:24:59 worm2 kernel: pid 46230 (cp), uid 0 inumber 168196 on /mnt: filesystem full Apr 25 17:25:00 worm2 kernel: pid 46230 (cp), uid 0 inumber 145361 on /mnt: filesystem full Apr 25 18:59:56 worm2 kernel: pid 93543 (fstool), uid 0 inumber 1859409 on /mnt: out of inodes Apr 25 19:23:44 worm2 kernel: pid 94097 (cp), uid 0 inumber 214349 on /mnt: filesystem full Apr 25 19:23:44 worm2 kernel: pid 94099 (cp), uid 0 inumber 544430 on /mnt: filesystem full Apr 25 20:58:21 worm2 kernel: pid 41620 (fstool), uid 0 inumber 1859427 on /mnt: out of inodes Apr 25 21:22:11 worm2 kernel: pid 42190 (cp), uid 0 inumber 331396 on /mnt: filesystem full Apr 25 21:22:11 worm2 kernel: pid 42188 (cp), uid 0 inumber 285351 on /mnt: filesystem full Apr 25 22:56:20 worm2 kernel: pid 89533 (fstool), uid 0 inumber 1860254 on /mnt: out of inodes Apr 25 23:23:20 worm2 kernel: pid 90057 (cp), uid 0 inumber 451025 on /mnt: filesystem full Apr 25 23:23:20 worm2 kernel: pid 90055 (cp), uid 0 inumber 544213 on /mnt: filesystem full Apr 26 00:57:54 worm2 kernel: pid 37521 (fstool), uid 0 inumber 1859678 on /mnt: out of inodes Apr 26 01:22:32 worm2 kernel: pid 38110 (cp), uid 0 inumber 120953 on /mnt: filesystem full Apr 26 01:22:32 worm2 kernel: pid 38110 (cp), uid 0 inumber 166720 on /mnt: filesystem full Apr 26 02:57:00 worm2 kernel: pid 85526 (fstool), uid 0 inumber 1860484 on /mnt: out of inodes Apr 26 03:23:07 worm2 kernel: pid 86398 (cp), uid 0 inumber 331776 on /mnt: filesystem full Apr 26 03:23:08 worm2 kernel: pid 86397 (cp), uid 0 inumber 615239 on /mnt: filesystem full Apr 26 04:57:10 worm2 kernel: pid 33952 (fstool), uid 0 inumber 1859969 on /mnt: out of inodes Apr 26 05:23:02 worm2 kernel: pid 34508 (cp), uid 0 inumber 566596 on /mnt: filesystem full Apr 26 05:23:02 worm2 kernel: pid 34509 (cp), uid 0 inumber 73260 on /mnt: filesystem full Apr 26 05:23:03 worm2 kernel: pid 34508 (cp), uid 0 inumber 567896 on /mnt: filesystem full Apr 26 05:23:04 worm2 kernel: pid 34508 (cp), uid 0 inumber 614798 on /mnt: filesystem full Apr 26 05:23:05 worm2 kernel: pid 34508 (cp), uid 0 inumber 565963 on /mnt: filesystem full Apr 26 06:21:38 worm2 kernel: pid 81247 (swap), uid 0, was killed: out of swap space Apr 26 06:21:41 worm2 kernel: pid 81235 (swap), uid 0, was killed: out of swap space Apr 26 06:21:45 worm2 kernel: pid 81267 (swap), uid 0, was killed: out of swap space Apr 26 06:21:50 worm2 kernel: pid 81249 (swap), uid 0, was killed: out of swap space Apr 26 06:21:54 worm2 kernel: pid 81282 (swap), uid 0, was killed: out of swap space Apr 26 06:21:58 worm2 kernel: pid 81276 (swap), uid 0, was killed: out of swap space Apr 26 06:22:02 worm2 kernel: pid 81280 (swap), uid 0, was killed: out of swap space Apr 26 06:22:08 worm2 kernel: pid 81283 (swap), uid 0, was killed: out of swap space Apr 26 06:22:18 worm2 kernel: pid 81279 (swap), uid 0, was killed: out of swap space Apr 26 06:23:47 worm2 kernel: pid 81766 (swap), uid 0, was killed: out of swap space Apr 26 06:23:50 worm2 kernel: pid 81763 (swap), uid 0, was killed: out of swap space Apr 26 06:23:54 worm2 kernel: pid 81706 (swap), uid 0, was killed: out of swap space Apr 26 06:23:58 worm2 kernel: pid 81761 (swap), uid 0, was killed: out of swap space Apr 26 06:24:01 worm2 kernel: pid 81757 (swap), uid 0, was killed: out of swap space Apr 26 06:24:06 worm2 kernel: pid 81756 (swap), uid 0, was killed: out of swap space Apr 26 06:24:11 worm2 kernel: pid 81683 (swap), uid 0, was killed: out of swap space Apr 26 06:24:15 worm2 kernel: pid 81769 (swap), uid 0, was killed: out of swap space Apr 26 06:24:20 worm2 kernel: pid 81768 (swap), uid 0, was killed: out of swap space Apr 26 06:24:23 worm2 kernel: pid 81767 (swap), uid 0, was killed: out of swap space Apr 26 06:24:25 worm2 kernel: pid 81765 (swap), uid 0, was killed: out of swap space Apr 26 06:28:10 worm2 kernel: pid 82969 (swap), uid 0, was killed: out of swap space Apr 26 06:28:16 worm2 kernel: pid 82964 (swap), uid 0, was killed: out of swap space Apr 26 06:29:40 worm2 kernel: pid 82956 (swap), uid 0, was killed: out of swap space Apr 26 06:30:41 worm2 kernel: pid 83820 (swap), uid 0, was killed: out of swap space Apr 26 06:30:45 worm2 kernel: pid 83863 (swap), uid 0, was killed: out of swap space Apr 26 06:30:49 worm2 kernel: pid 83864 (swap), uid 0, was killed: out of swap space Apr 26 06:30:55 worm2 kernel: pid 83862 (swap), uid 0, was killed: out of swap space Apr 26 06:32:56 worm2 kernel: pid 84255 (swap), uid 0, was killed: out of swap space Apr 26 06:33:01 worm2 kernel: pid 84299 (swap), uid 0, was killed: out of swap space Apr 26 06:33:06 worm2 kernel: pid 84298 (swap), uid 0, was killed: out of swap space Apr 26 06:33:13 worm2 kernel: pid 84300 (swap), uid 0, was killed: out of swap space Apr 26 06:33:19 worm2 kernel: pid 84297 (swap), uid 0, was killed: out of swap space Apr 26 06:36:39 worm2 kernel: pid 84406 (swap), uid 0, was killed: out of swap space Apr 26 06:37:46 worm2 kernel: pid 84535 (swap), uid 0, was killed: out of swap space Apr 26 06:37:52 worm2 kernel: pid 84578 (swap), uid 0, was killed: out of swap space Apr 26 06:37:55 worm2 kernel: pid 84579 (swap), uid 0, was killed: out of swap space Apr 26 06:37:58 worm2 kernel: pid 84576 (swap), uid 0, was killed: out of swap space Apr 26 07:05:21 worm2 kernel: pid 88343 (swap), uid 0, was killed: out of swap space Apr 26 07:05:27 worm2 kernel: pid 88340 (swap), uid 0, was killed: out of swap space Apr 26 07:05:32 worm2 kernel: pid 88339 (swap), uid 0, was killed: out of swap space Apr 26 07:09:34 worm2 kernel: pid 89796 (swap), uid 0, was killed: out of swap space Apr 26 07:11:49 worm2 kernel: pid 91274 (swap), uid 0, was killed: out of swap space Apr 26 07:11:51 worm2 kernel: pid 91309 (swap), uid 0, was killed: out of swap space Apr 26 07:11:58 worm2 kernel: pid 91279 (swap), uid 0, was killed: out of swap space Apr 26 07:12:00 worm2 kernel: pid 91313 (swap), uid 0, was killed: out of swap space Apr 26 07:14:13 worm2 kernel: pid 92040 (swap), uid 0, was killed: out of swap space Apr 26 07:14:17 worm2 kernel: pid 92082 (swap), uid 0, was killed: out of swap space Apr 26 07:14:20 worm2 kernel: pid 92080 (swap), uid 0, was killed: out of swap space Apr 26 07:14:25 worm2 kernel: pid 92079 (swap), uid 0, was killed: out of swap space Apr 26 07:16:12 worm2 kernel: pid 92236 (swap), uid 0, was killed: out of swap space Apr 26 07:16:15 worm2 kernel: pid 92264 (swap), uid 0, was killed: out of swap space Apr 26 07:16:19 worm2 kernel: pid 92268 (swap), uid 0, was killed: out of swap space Apr 26 07:16:22 worm2 kernel: pid 92267 (swap), uid 0, was killed: out of swap space Apr 26 07:16:24 worm2 kernel: pid 92265 (swap), uid 0, was killed: out of swap space Apr 26 07:20:16 worm2 kernel: pid 92805 (swap), uid 0, was killed: out of swap space Apr 26 07:20:52 worm2 kernel: pid 93098 (swap), uid 0, was killed: out of swap space Apr 26 07:21:00 worm2 kernel: pid 93118 (swap), uid 0, was killed: out of swap space Apr 26 07:23:15 worm2 kernel: pid 93235 (swap), uid 0, was killed: out of swap space Apr 26 07:23:18 worm2 kernel: pid 93319 (swap), uid 0, was killed: out of swap space Apr 26 07:23:22 worm2 kernel: pid 93318 (swap), uid 0, was killed: out of swap space Apr 26 07:23:26 worm2 kernel: pid 93313 (swap), uid 0, was killed: out of swap space Apr 26 07:23:30 worm2 kernel: pid 93314 (swap), uid 0, was killed: out of swap space Apr 26 07:23:37 worm2 kernel: pid 93315 (swap), uid 0, was killed: out of swap space Apr 26 07:23:40 worm2 kernel: pid 93317 (swap), uid 0, was killed: out of swap space Apr 26 07:23:43 worm2 kernel: pid 93316 (swap), uid 0, was killed: out of swap space Apr 26 07:55:29 worm2 kernel: pid 94551 (cp), uid 0 inumber 497354 on /mnt: filesystem full Apr 26 07:55:29 worm2 kernel: pid 94553 (cp), uid 0 inumber 427425 on /mnt: filesystem full Apr 26 09:31:14 worm2 kernel: pid 42020 (fstool), uid 0 inumber 1860072 on /mnt: out of inodes Apr 26 09:32:09 worm2 kernel: pid 42232 (swap), uid 0, was killed: out of swap space Apr 26 09:32:13 worm2 kernel: pid 42211 (swap), uid 0, was killed: out of swap space Apr 26 09:32:16 worm2 kernel: pid 42227 (swap), uid 0, was killed: out of swap space Apr 26 09:32:21 worm2 kernel: pid 42208 (swap), uid 0, was killed: out of swap space Apr 26 09:32:26 worm2 kernel: pid 42216 (swap), uid 0, was killed: out of swap space Apr 26 09:32:31 worm2 kernel: pid 42231 (swap), uid 0, was killed: out of swap space Apr 26 09:34:22 worm2 kernel: pid 43538 (swap), uid 0, was killed: out of swap space Apr 26 09:34:27 worm2 kernel: pid 43570 (swap), uid 0, was killed: out of swap space Apr 26 09:34:33 worm2 kernel: pid 43572 (swap), uid 0, was killed: out of swap space Apr 26 09:34:36 worm2 kernel: pid 43567 (swap), uid 0, was killed: out of swap space Apr 26 09:34:46 worm2 kernel: pid 43568 (swap), uid 0, was killed: out of swap space Apr 26 09:36:15 worm2 kernel: pid 44208 (swap), uid 0, was killed: out of swap space Apr 26 09:36:18 worm2 kernel: pid 44211 (swap), uid 0, was killed: out of swap space Apr 26 09:36:27 worm2 kernel: pid 44210 (swap), uid 0, was killed: out of swap space Apr 26 09:36:37 worm2 kernel: pid 44204 (swap), uid 0, was killed: out of swap space Apr 26 09:38:48 worm2 kernel: pid 44483 (swap), uid 0, was killed: out of swap space Apr 26 09:38:51 worm2 kernel: pid 44500 (swap), uid 0, was killed: out of swap space Apr 26 09:38:55 worm2 kernel: pid 44489 (swap), uid 0, was killed: out of swap space Apr 26 09:39:01 worm2 kernel: pid 44487 (swap), uid 0, was killed: out of swap space Apr 26 09:39:02 worm2 kernel: pid 44485 (swap), uid 0, was killed: out of swap space Apr 26 09:39:09 worm2 kernel: pid 44521 (swap), uid 0, was killed: out of swap space Apr 26 09:41:21 worm2 kernel: pid 45322 (swap), uid 0, was killed: out of swap space Apr 26 09:43:22 worm2 kernel: pid 45408 (swap), uid 0, was killed: out of swap space Apr 26 09:43:25 worm2 kernel: pid 45445 (swap), uid 0, was killed: out of swap space Apr 26 09:51:25 worm2 kernel: pid 49384 (swap), uid 0, was killed: out of swap space Apr 26 09:51:57 worm2 kernel: pid 49622 (dd), uid 0 inumber 2204 on /var: filesystem full Apr 26 09:55:01 worm2 kernel: pid 51102 (dd), uid 2 inumber 1113 on /var: filesystem full Apr 26 10:00:00 worm2 kernel: pid 51183 (dd), uid 2 inumber 1114 on /var: filesystem full Apr 26 10:11:00 worm2 kernel: pid 51351 (dd), uid 2 inumber 1115 on /var: filesystem full Apr 26 10:17:53 worm2 kernel: pid 51445 (cp), uid 0 inumber 402632 on /mnt: filesystem full Apr 26 10:17:54 worm2 kernel: pid 51447 (cp), uid 0 inumber 425064 on /mnt: filesystem full Apr 26 10:22:00 worm2 kernel: pid 74024 (dd), uid 2 inumber 1116 on /var: filesystem full Apr 26 10:33:01 worm2 kernel: pid 74389 (dd), uid 2 inumber 1117 on /var: filesystem full Apr 26 10:44:00 worm2 kernel: pid 95965 (dd), uid 2 inumber 1119 on /var: filesystem full Apr 26 10:55:01 worm2 kernel: pid 97403 (dd), uid 2 inumber 1106 on /var: filesystem full Apr 26 11:00:01 worm2 kernel: pid 97567 (dd), uid 2 inumber 1112 on /var: filesystem full Apr 26 11:11:01 worm2 kernel: pid 97987 (dd), uid 2 inumber 1113 on /var: filesystem full Apr 26 11:22:00 worm2 kernel: pid 98042 (dd), uid 2 inumber 1114 on /var: filesystem full Apr 26 11:33:00 worm2 kernel: pid 98291 (dd), uid 2 inumber 1115 on /var: filesystem full Apr 26 11:44:00 worm2 kernel: pid 98630 (dd), uid 2 inumber 1116 on /var: filesystem full Apr 26 11:52:03 worm2 kernel: pid 98835 (fstool), uid 0 inumber 1859336 on /mnt: out of inodes Apr 26 11:53:11 worm2 kernel: pid 99063 (swap), uid 0, was killed: out of swap space Apr 26 11:53:19 worm2 kernel: pid 99062 (swap), uid 0, was killed: out of swap space Apr 26 11:55:00 worm2 kernel: pid 99333 (dd), uid 2 inumber 1117 on /var: filesystem full Apr 26 11:57:07 worm2 kernel: pid 99518 (swap), uid 0, was killed: out of swap space Apr 26 11:57:18 worm2 kernel: pid 99533 (swap), uid 0, was killed: out of swap space Apr 26 11:59:07 worm2 kernel: pid 262 (swap), uid 0, was killed: out of swap space Apr 26 11:59:11 worm2 kernel: pid 324 (swap), uid 0, was killed: out of swap space Apr 26 11:59:14 worm2 kernel: pid 302 (swap), uid 0, was killed: out of swap space Apr 26 11:59:17 worm2 kernel: pid 321 (swap), uid 0, was killed: out of swap space Apr 26 11:59:20 worm2 kernel: pid 309 (swap), uid 0, was killed: out of swap space Apr 26 11:59:24 worm2 kernel: pid 315 (swap), uid 0, was killed: out of swap space Apr 26 11:59:27 worm2 kernel: pid 308 (swap), uid 0, was killed: out of swap space Apr 26 11:59:32 worm2 kernel: pid 311 (swap), uid 0, was killed: out of swap space Apr 26 11:59:36 worm2 kernel: pid 325 (swap), uid 0, was killed: out of swap space Apr 26 11:59:40 worm2 kernel: pid 320 (swap), uid 0, was killed: out of swap space Apr 26 11:59:44 worm2 kernel: pid 318 (swap), uid 0, was killed: out of swap space Apr 26 11:59:52 worm2 kernel: pid 305 (swap), uid 0, was killed: out of swap space Apr 26 12:00:05 worm2 kernel: pid 628 (dd), uid 2 inumber 1119 on /var: filesystem full Apr 26 12:01:36 worm2 kernel: pid 780 (swap), uid 0, was killed: out of swap space Apr 26 12:01:41 worm2 kernel: pid 827 (swap), uid 0, was killed: out of swap space Apr 26 12:01:45 worm2 kernel: pid 826 (swap), uid 0, was killed: out of swap space Apr 26 12:01:51 worm2 kernel: pid 825 (swap), uid 0, was killed: out of swap space Apr 26 12:01:57 worm2 kernel: pid 828 (swap), uid 0, was killed: out of swap space Apr 26 12:02:01 worm2 kernel: pid 824 (swap), uid 0, was killed: out of swap space Apr 26 12:06:04 worm2 kernel: pid 1693 (swap), uid 0, was killed: out of swap space Apr 26 12:06:08 worm2 kernel: pid 1700 (swap), uid 0, was killed: out of swap space Apr 26 12:06:11 worm2 kernel: pid 1703 (swap), uid 0, was killed: out of swap space Apr 26 12:06:18 worm2 kernel: pid 1729 (swap), uid 0, was killed: out of swap space Apr 26 12:07:22 worm2 kernel: pid 1725 (swap), uid 0, was killed: out of swap space Apr 26 12:08:32 worm2 kernel: pid 2387 (swap), uid 0, was killed: out of swap space Apr 26 12:09:29 worm2 kernel: pid 2392 (swap), uid 0, was killed: out of swap space Apr 26 12:10:47 worm2 kernel: pid 3996 (swap), uid 0, was killed: out of swap space Apr 26 12:10:53 worm2 kernel: pid 4051 (swap), uid 0, was killed: out of swap space Apr 26 12:10:57 worm2 kernel: pid 4053 (swap), uid 0, was killed: out of swap space Apr 26 12:11:00 worm2 kernel: pid 4050 (swap), uid 0, was killed: out of swap space Apr 26 12:11:06 worm2 kernel: pid 4121 (dd), uid 2 inumber 1106 on /var: filesystem full Apr 26 12:22:00 worm2 kernel: pid 4426 (dd), uid 2 inumber 1112 on /var: filesystem full Apr 26 12:33:01 worm2 kernel: pid 4671 (dd), uid 2 inumber 1113 on /var: filesystem full Apr 26 12:36:29 worm2 kernel: pid 4683 (cp), uid 0 inumber 544855 on /mnt: filesystem full Apr 26 12:36:29 worm2 kernel: pid 4683 (cp), uid 0 inumber 402403 on /mnt: filesystem full panic: bad pte cpuid = 0 KDB: enter: panic [ thread pid 16043 tid 100136 ] Stopped at kdb_enter+0x3d: movq $0,0x6bdde0(%rip) db> bt Tracing pid 16043 tid 100136 td 0xffffff0007a29390 kdb_enter() at kdb_enter+0x3d panic() at panic+0x17b pmap_remove_pages() at pmap_remove_pages+0x632 vmspace_exit() at vmspace_exit+0x9a exit1() at exit1+0x376 sys_exit() at sys_exit+0xe syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (1, FreeBSD ELF64, sys_exit), rip = 0x8006f6f3c, rsp = 0x7fffffffe6c8, rbp = 0 --- db> set $lines 0 db> show allpcpu Current CPU: 0 cpuid = 0 dynamic pcpu = 0x48be80 curthread = 0xffffff0007a29390: pid 16043 "cp" curpcb = 0xffffff81cfd10d40 fpcurthread = none idlethread = 0xffffff0002fd3ab0: pid 11 "idle: cpu0" curpmap = 0 tssp = 0xffffffff80e85200 commontssp = 0xffffffff80e85200 rsp0 = 0xffffff81cfd10d40 gs32p = 0xffffffff80e84038 ldt = 0xffffffff80e84078 tss = 0xffffffff80e84068 spin locks held: cpuid = 1 dynamic pcpu = 0xffffff807f405e80 curthread = 0xffffff0002fd4000: pid 11 "idle: cpu1" curpcb = 0xffffff80000ffd40 fpcurthread = none idlethread = 0xffffff0002fd4000: pid 11 "idle: cpu1" curpmap = 0 tssp = 0xffffffff80e85268 commontssp = 0xffffffff80e85268 rsp0 = 0xffffff80000ffd40 gs32p = 0xffffffff80e840a0 ldt = 0xffffffff80e840e0 tss = 0xffffffff80e840d0 spin locks held: cpuid = 2 dynamic pcpu = 0xffffff807f40ce80 curthread = 0xffffff0002fd4390: pid 11 "idle: cpu2" curpcb = 0xffffff80000fad40 fpcurthread = none idlethread = 0xffffff0002fd4390: pid 11 "idle: cpu2" curpmap = 0 tssp = 0xffffffff80e852d0 commontssp = 0xffffffff80e852d0 rsp0 = 0xffffff80000fad40 gs32p = 0xffffffff80e84108 ldt = 0xffffffff80e84148 tss = 0xffffffff80e84138 spin locks held: cpuid = 3 dynamic pcpu = 0xffffff807f413e80 curthread = 0xffffff0002fd4720: pid 11 "idle: cpu3" curpcb = 0xffffff80000f5d40 fpcurthread = none idlethread = 0xffffff0002fd4720: pid 11 "idle: cpu3" curpmap = 0 tssp = 0xffffffff80e85338 commontssp = 0xffffffff80e85338 rsp0 = 0xffffff80000f5d40 gs32p = 0xffffffff80e84170 ldt = 0xffffffff80e841b0 tss = 0xffffffff80e841a0 spin locks held: cpuid = 4 dynamic pcpu = 0xffffff807f41ae80 curthread = 0xffffff0002fd4ab0: pid 11 "idle: cpu4" curpcb = 0xffffff80000f0d40 fpcurthread = none idlethread = 0xffffff0002fd4ab0: pid 11 "idle: cpu4" curpmap = 0 tssp = 0xffffffff80e853a0 commontssp = 0xffffffff80e853a0 rsp0 = 0xffffff80000f0d40 gs32p = 0xffffffff80e841d8 ldt = 0xffffffff80e84218 tss = 0xffffffff80e84208 spin locks held: cpuid = 5 dynamic pcpu = 0xffffff807f421e80 curthread = 0xffffff0002fd5000: pid 11 "idle: cpu5" curpcb = 0xffffff80000ebd40 fpcurthread = none idlethread = 0xffffff0002fd5000: pid 11 "idle: cpu5" curpmap = 0 tssp = 0xffffffff80e85408 commontssp = 0xffffffff80e85408 rsp0 = 0xffffff80000ebd40 gs32p = 0xffffffff80e84240 ldt = 0xffffffff80e84280 tss = 0xffffffff80e84270 spin locks held: cpuid = 6 dynamic pcpu = 0xffffff807f428e80 curthread = 0xffffff0002fd5390: pid 11 "idle: cpu6" curpcb = 0xffffff80000e6d40 fpcurthread = none idlethread = 0xffffff0002fd5390: pid 11 "idle: cpu6" curpmap = 0 tssp = 0xffffffff80e85470 commontssp = 0xffffffff80e85470 rsp0 = 0xffffff80000e6d40 gs32p = 0xffffffff80e842a8 ldt = 0xffffffff80e842e8 tss = 0xffffffff80e842d8 spin locks held: cpuid = 7 dynamic pcpu = 0xffffff807f42fe80 curthread = 0xffffff0002fc9390: pid 11 "idle: cpu7" curpcb = 0xffffff80000e1d40 fpcurthread = none idlethread = 0xffffff0002fc9390: pid 11 "idle: cpu7" curpmap = 0 tssp = 0xffffffff80e854d8 commontssp = 0xffffffff80e854d8 rsp0 = 0xffffff80000e1d40 gs32p = 0xffffffff80e84310 ldt = 0xffffffff80e84350 tss = 0xffffffff80e84340 spin locks held: cpuid = 8 dynamic pcpu = 0xffffff807f436e80 curthread = 0xffffff0002fc9720: pid 11 "idle: cpu8" curpcb = 0xffffff80000dcd40 fpcurthread = none idlethread = 0xffffff0002fc9720: pid 11 "idle: cpu8" curpmap = 0 tssp = 0xffffffff80e85540 commontssp = 0xffffffff80e85540 rsp0 = 0xffffff80000dcd40 gs32p = 0xffffffff80e84378 ldt = 0xffffffff80e843b8 tss = 0xffffffff80e843a8 spin locks held: cpuid = 9 dynamic pcpu = 0xffffff807f43de80 curthread = 0xffffff0002fc9ab0: pid 11 "idle: cpu9" curpcb = 0xffffff80000d7d40 fpcurthread = none idlethread = 0xffffff0002fc9ab0: pid 11 "idle: cpu9" curpmap = 0 tssp = 0xffffffff80e855a8 commontssp = 0xffffffff80e855a8 rsp0 = 0xffffff80000d7d40 gs32p = 0xffffffff80e843e0 ldt = 0xffffffff80e84420 tss = 0xffffffff80e84410 spin locks held: cpuid = 10 dynamic pcpu = 0xffffff807f444e80 curthread = 0xffffff0002fd3000: pid 11 "idle: cpu10" curpcb = 0xffffff80000d2d40 fpcurthread = none idlethread = 0xffffff0002fd3000: pid 11 "idle: cpu10" curpmap = 0 tssp = 0xffffffff80e85610 commontssp = 0xffffffff80e85610 rsp0 = 0xffffff80000d2d40 gs32p = 0xffffffff80e84448 ldt = 0xffffffff80e84488 tss = 0xffffffff80e84478 spin locks held: cpuid = 11 dynamic pcpu = 0xffffff807f44be80 curthread = 0xffffff0002fd3390: pid 11 "idle: cpu11" curpcb = 0xffffff80000cdd40 fpcurthread = none idlethread = 0xffffff0002fd3390: pid 11 "idle: cpu11" curpmap = 0 tssp = 0xffffffff80e85678 commontssp = 0xffffffff80e85678 rsp0 = 0xffffff80000cdd40 gs32p = 0xffffffff80e844b0 ldt = 0xffffffff80e844f0 tss = 0xffffffff80e844e0 spin locks held: cpuid = 12 dynamic pcpu = 0xffffff807f452e80 curthread = 0xffffff0002fd3720: pid 11 "idle: cpu12" curpcb = 0xffffff80000c8d40 fpcurthread = none idlethread = 0xffffff0002fd3720: pid 11 "idle: cpu12" curpmap = 0 tssp = 0xffffffff80e856e0 commontssp = 0xffffffff80e856e0 rsp0 = 0xffffff80000c8d40 gs32p = 0xffffffff80e84518 ldt = 0xffffffff80e84558 tss = 0xffffffff80e84548 spin locks held: cpuid = 13 dynamic pcpu = 0xffffff807f459e80 curthread = 0xffffff0002fc7000: pid 11 "idle: cpu13" curpcb = 0xffffff80000c3d40 fpcurthread = none idlethread = 0xffffff0002fc7000: pid 11 "idle: cpu13" curpmap = 0 tssp = 0xffffffff80e85748 commontssp = 0xffffffff80e85748 rsp0 = 0xffffff80000c3d40 gs32p = 0xffffffff80e84580 ldt = 0xffffffff80e845c0 tss = 0xffffffff80e845b0 spin locks held: cpuid = 14 dynamic pcpu = 0xffffff807f460e80 curthread = 0xffffff0002fc7390: pid 11 "idle: cpu14" curpcb = 0xffffff80000bed40 fpcurthread = none idlethread = 0xffffff0002fc7390: pid 11 "idle: cpu14" curpmap = 0 tssp = 0xffffffff80e857b0 commontssp = 0xffffffff80e857b0 rsp0 = 0xffffff80000bed40 gs32p = 0xffffffff80e845e8 ldt = 0xffffffff80e84628 tss = 0xffffffff80e84618 spin locks held: cpuid = 15 dynamic pcpu = 0xffffff807f467e80 curthread = 0xffffff0002fc7720: pid 11 "idle: cpu15" curpcb = 0xffffff80000b9d40 fpcurthread = none idlethread = 0xffffff0002fc7720: pid 11 "idle: cpu15" curpmap = 0 tssp = 0xffffffff80e85818 commontssp = 0xffffffff80e85818 rsp0 = 0xffffff80000b9d40 gs32p = 0xffffffff80e84650 ldt = 0xffffffff80e84690 tss = 0xffffffff80e84680 spin locks held: db> show alllocks Process 16043 (cp) thread 0xffffff0007a29390 (100136) exclusive sleep mutex pmap (pmap) r = 0 (0xffffff014cb7f2b8) locked @ amd64/amd64/pmap.c:4002 exclusive sleep mutex vm page queue mutex (vm page queue mutex) r = 0 (0xffffffff80e4b980) locked @ amd64/amd64/pmap.c:4001 Process 2064 (sshd) thread 0xffffff01aa290720 (100184) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xffffff01aa31f3a0) locked @ kern/uipc_sockbuf.c:148 Process 1955 (sshd) thread 0xffffff0007b96000 (100153) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xffffff0007e07b98) locked @ kern/uipc_sockbuf.c:148 db> show lockedvnods Locked vnodes db> show mount 0xffffff0007661720 192.168.5.1:/zoo/worm2/ on / (nfs) 0xffffff0007655000 devfs on /dev (devfs) 0xffffff0007654ab0 /dev/md0 on /var (ufs) 0xffffff0007654720 /dev/md1 on /tmp (ufs) 0xffffff01fd296ab0 /dev/md5a on /mnt (ufs) ore info: show mount db> show threads 100136 (0xffffff0007a29390) kdb_enter() at kdb_enter+0x3d 100455 (0xffffff03382d5390) sched_switch() at sched_switch+0x154 101161 (0xffffff030059f390) sched_switch() at sched_switch+0x154 100741 (0xffffff03754b1720) sched_switch() at sched_switch+0x154 100446 (0xffffff01f0756000) sched_switch() at sched_switch+0x154 100843 (0xffffff02adcb3720) sched_switch() at sched_switch+0x154 101305 (0xffffff015686e000) sched_switch() at sched_switch+0x154 101140 (0xffffff0173cb0ab0) sched_switch() at sched_switch+0x154 101062 (0xffffff040f0ea720) sched_switch() at sched_switch+0x154 100827 (0xffffff02eb63d720) sched_switch() at sched_switch+0x154 100947 (0xffffff005d0e0720) sched_switch() at sched_switch+0x154 100105 (0xffffff000745f390) sched_switch() at sched_switch+0x154 100187 (0xffffff01aa35b390) sched_switch() at sched_switch+0x154 100173 (0xffffff0007e02000) sched_switch() at sched_switch+0x154 100179 (0xffffff000782aab0) sched_switch() at sched_switch+0x154 100178 (0xffffff000782b000) sched_switch() at sched_switch+0x154 100177 (0xffffff000782b390) sched_switch() at sched_switch+0x154 100176 (0xffffff000782b720) sched_switch() at sched_switch+0x154 100175 (0xffffff000782bab0) sched_switch() at sched_switch+0x154 100174 (0xffffff000782c000) sched_switch() at sched_switch+0x154 100157 (0xffffff0007d51000) sched_switch() at sched_switch+0x154 100144 (0xffffff0007900ab0) sched_switch() at sched_switch+0x154 100148 (0xffffff000782cab0) sched_switch() at sched_switch+0x154 100152 (0xffffff0007b96390) sched_switch() at sched_switch+0x154 100093 (0xffffff0007907ab0) sched_switch() at sched_switch+0x154 100100 (0xffffff000753c720) sched_switch() at sched_switch+0x154 100107 (0xffffff0007a37390) sched_switch() at sched_switch+0x154 100145 (0xffffff0007900720) sched_switch() at sched_switch+0x154 100155 (0xffffff0007d51720) sched_switch() at sched_switch+0x154 100094 (0xffffff0007907720) sched_switch() at sched_switch+0x154 100098 (0xffffff000753d000) sched_switch() at sched_switch+0x154 100092 (0xffffff0007908000) sched_switch() at sched_switch+0x154 100088 (0xffffff0007909000) sched_switch() at sched_switch+0x154 100104 (0xffffff000745f720) sched_switch() at sched_switch+0x154 100082 (0xffffff0007540720) sched_switch() at sched_switch+0x154 100081 (0xffffff000745c000) sched_switch() at sched_switch+0x154 100080 (0xffffff000745c390) sched_switch() at sched_switch+0x154 100079 (0xffffff000745c720) sched_switch() at sched_switch+0x154 100078 (0xffffff000745cab0) sched_switch() at sched_switch+0x154 100077 (0xffffff000745d000) sched_switch() at sched_switch+0x154 100076 (0xffffff000745d390) sched_switch() at sched_switch+0x154 100075 (0xffffff000745d720) sched_switch() at sched_switch+0x154 100074 (0xffffff000745dab0) sched_switch() at sched_switch+0x154 100070 (0xffffff000745eab0) sched_switch() at sched_switch+0x154 100061 (0xffffff0007301ab0) sched_switch() at sched_switch+0x154 100060 (0xffffff0007303000) sched_switch() at sched_switch+0x154 100059 (0xffffff0007303390) sched_switch() at sched_switch+0x154 100058 (0xffffff0007303720) sched_switch() at sched_switch+0x154 100056 (0xffffff0002ff3390) sched_switch() at sched_switch+0x154 100055 (0xffffff0002ff3720) sched_switch() at sched_switch+0x154 100054 (0xffffff0002ff3ab0) sched_switch() at sched_switch+0x154 100053 (0xffffff0007181000) sched_switch() at sched_switch+0x154 100041 (0xffffff0002feeab0) sched_switch() at sched_switch+0x154 100039 (0xffffff0002fef390) sched_switch() at sched_switch+0x154 100038 (0xffffff0002fef720) sched_switch() at sched_switch+0x154 100037 (0xffffff0002fefab0) sched_switch() at sched_switch+0x154 100072 (0xffffff000745e390) fork_trampoline() at fork_trampoline 100071 (0xffffff000745e720) fork_trampoline() at fork_trampoline 100069 (0xffffff000745f000) sched_switch() at sched_switch+0x154 100065 (0xffffff0007300ab0) fork_trampoline() at fork_trampoline 100064 (0xffffff0007301000) fork_trampoline() at fork_trampoline 100063 (0xffffff0007301390) fork_trampoline() at fork_trampoline 100062 (0xffffff0007301720) sched_switch() at sched_switch+0x154 100057 (0xffffff0007303ab0) sched_switch() at sched_switch+0x154 100052 (0xffffff0007181390) sched_switch() at sched_switch+0x154 100051 (0xffffff0007181720) fork_trampoline() at fork_trampoline 100050 (0xffffff0007181ab0) sched_switch() at sched_switch+0x154 100049 (0xffffff0007182000) fork_trampoline() at fork_trampoline 100047 (0xffffff0007182720) sched_switch() at sched_switch+0x154 100045 (0xffffff0002febab0) fork_trampoline() at fork_trampoline 100036 (0xffffff0002ff3000) sched_switch() at sched_switch+0x154 100035 (0xffffff0002fe5720) sched_switch() at sched_switch+0x154 100034 (0xffffff0002fe5ab0) sched_switch() at sched_switch+0x154 100033 (0xffffff0002fe8000) sched_switch() at sched_switch+0x154 100032 (0xffffff0002fe8390) sched_switch() at sched_switch+0x154 100031 (0xffffff0002fe8720) sched_switch() at sched_switch+0x154 100030 (0xffffff0002fe8ab0) sched_switch() at sched_switch+0x154 100029 (0xffffff0002feb000) sched_switch() at sched_switch+0x154 100028 (0xffffff0002feb390) sched_switch() at sched_switch+0x154 100027 (0xffffff0002feb720) sched_switch() at sched_switch+0x154 100026 (0xffffff0002fd5720) sched_switch() at sched_switch+0x154 100025 (0xffffff0002fd5ab0) sched_switch() at sched_switch+0x154 100024 (0xffffff0002fe3000) sched_switch() at sched_switch+0x154 100023 (0xffffff0002fe3390) sched_switch() at sched_switch+0x154 100022 (0xffffff0002fe3720) sched_switch() at sched_switch+0x154 100021 (0xffffff0002fe3ab0) sched_switch() at sched_switch+0x154 100020 (0xffffff0002fe5000) fork_trampoline() at fork_trampoline 100019 (0xffffff0002fe5390) sched_switch() at sched_switch+0x154 100018 (0xffffff0002fd3ab0) sched_switch() at sched_switch+0x154 100017 (0xffffff0002fd4000) cpustop_handler() at cpustop_handler+0x40 100016 (0xffffff0002fd4390) cpustop_handler() at cpustop_handler+0x40 100015 (0xffffff0002fd4720) cpustop_handler() at cpustop_handler+0x40 100014 (0xffffff0002fd4ab0) cpustop_handler() at cpustop_handler+0x40 100013 (0xffffff0002fd5000) cpustop_handler() at cpustop_handler+0x40 100012 (0xffffff0002fd5390) cpustop_handler() at cpustop_handler+0x40 100011 (0xffffff0002fc9390) cpustop_handler() at cpustop_handler+0x40 100010 (0xffffff0002fc9720) cpustop_handler() at cpustop_handler+0x40 100009 (0xffffff0002fc9ab0) cpustop_handler() at cpustop_handler+0x40 100008 (0xffffff0002fd3000) cpustop_handler() at cpustop_handler+0x40 100007 (0xffffff0002fd3390) cpustop_handler() at cpustop_handler+0x40 100006 (0xffffff0002fd3720) cpustop_handler() at cpustop_handler+0x40 100005 (0xffffff0002fc7000) cpustop_handler() at cpustop_handler+0x40 100004 (0xffffff0002fc7390) cpustop_handler() at cpustop_handler+0x40 100003 (0xffffff0002fc7720) cpustop_handler() at cpustop_handler+0x40 100002 (0xffffff0002fc7ab0) sched_switch() at sched_switch+0x154 100001 (0xffffff0002fc9000) sched_switch() at sched_switch+0x154 100073 (0xffffff000745e000) sched_switch() at sched_switch+0x154 100068 (0xffffff0007300000) sched_switch() at sched_switch+0x154 100067 (0xffffff0007300390) sched_switch() at sched_switch+0x154 100066 (0xffffff0007300720) sched_switch() at sched_switch+0x154 100048 (0xffffff0007182390) sched_switch() at sched_switch+0x154 100046 (0xffffff0007182ab0) sched_switch() at sched_switch+0x154 100044 (0xffffff0002fee000) sched_switch() at sched_switch+0x154 100043 (0xffffff0002fee390) sched_switch() at sched_switch+0x154 100042 (0xffffff0002fee720) sched_switch() at sched_switch+0x154 100040 (0xffffff0002fef000) sched_switch() at sched_switch+0x154 100000 (0xffffffff80c77180) sched_switch() at sched_switch+0x154 db> allt Tracing command cp pid 16043 tid 100136 td 0xffffff0007a29390 kdb_enter() at kdb_enter+0x3d panic() at panic+0x17b pmap_remove_pages() at pmap_remove_pages+0x632 vmspace_exit() at vmspace_exit+0x9a exit1() at exit1+0x376 sys_exit() at sys_exit+0xe syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (1, FreeBSD ELF64, sys_exit), rip = 0x8006f6f3c, rsp = 0x7fffffffe6c8, rbp = 0 --- Tracing command sh pid 4883 tid 100455 td 0xffffff03382d5390 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip = 0x80093c81c, rsp = 0x7fffffffe658, rbp = 0x7fffffffe828 --- Tracing command find pid 4882 tid 101161 td 0xffffff030059f390 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_write() at pipe_write+0x523 dofilewrite() at dofilewrite+0x85 kern_writev() at kern_writev+0x60 write() at write+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip = 0x80074efec, rsp = 0x7fffffffe738, rbp = 0x1000 --- Tracing command md5 pid 4878 tid 100741 td 0xffffff03754b1720 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 md_kthread() at md_kthread+0x15a fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81d08e1d30, rbp = 0 --- Tracing command sh pid 4869 tid 100446 td 0xffffff01f0756000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip = 0x80093c81c, rsp = 0x7fffffffe998, rbp = 0 --- Tracing command nfsiod 0 pid 4659 tid 100843 td 0xffffff02adcb3720 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f nfssvc_iod() at nfssvc_iod+0x2d6 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81d0adfd30, rbp = 0 --- Tracing command sh pid 50931 tid 101305 td 0xffffff015686e000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip = 0x80093c81c, rsp = 0x7fffffffe538, rbp = 0x7fffffffe708 --- Tracing command md9 pid 49678 tid 101140 td 0xffffff0173cb0ab0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 md_kthread() at md_kthread+0x15a fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81d10acd30, rbp = 0 --- Tracing command md8 pid 49665 tid 101062 td 0xffffff040f0ea720 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 md_kthread() at md_kthread+0x15a fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81d0f26d30, rbp = 0 --- Tracing command md7 pid 49652 tid 100827 td 0xffffff02eb63d720 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 md_kthread() at md_kthread+0x15a fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81d0a8fd30, rbp = 0 --- Tracing command md6 pid 49639 tid 100947 td 0xffffff005d0e0720 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 md_kthread() at md_kthread+0x15a fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81d0ce7d30, rbp = 0 --- Tracing command top pid 2069 tid 100105 td 0xffffff000745f390 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x640 select() at select+0x5d syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800ac6f8c, rsp = 0x7fffffffea48, rbp = 0x2 --- Tracing command sh pid 2068 tid 100188 td 0xffffff01aa290390 Tracing command sshd pid 2067 tid 100187 td 0xffffff01aa35b390 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x640 select() at select+0x5d syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8016eaf8c, rsp = 0x7fffffffe388, rbp = 0x7fffffffe410 --- Tracing command sshd pid 2064 tid 100184 td 0xffffff01aa290720 Tracing command sh pid 1975 tid 100167 td 0xffffff0007b92720 Tracing command csh pid 1971 tid 100146 td 0xffffff0007900390 Tracing command su pid 1970 tid 100158 td 0xffffff0007dcdab0 Tracing command sh pid 1959 tid 100163 td 0xffffff0007bbd720 Tracing command sshd pid 1958 tid 100173 td 0xffffff0007e02000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x640 select() at select+0x5d syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8016eaf8c, rsp = 0x7fffffffe388, rbp = 0x7fffffffe410 --- Tracing command sshd pid 1955 tid 100153 td 0xffffff0007b96000 Tracing command getty pid 1954 tid 100179 td 0xffffff000782aab0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydev_open() at ttydev_open+0x343 devfs_open() at devfs_open+0x186 VOP_OPEN_APV() at VOP_OPEN_APV+0xaf vn_open_cred() at vn_open_cred+0x316 kern_openat() at kern_openat+0x179 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (5, FreeBSD ELF64, open), rip = 0x800848e7c, rsp = 0x7fffffffecd8, rbp = 0x5086e0 --- Tracing command getty pid 1953 tid 100178 td 0xffffff000782b000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80085900c, rsp = 0x7fffffffecc8, rbp = 0 --- Tracing command getty pid 1952 tid 100177 td 0xffffff000782b390 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80085900c, rsp = 0x7fffffffecc8, rbp = 0 --- Tracing command getty pid 1951 tid 100176 td 0xffffff000782b720 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80085900c, rsp = 0x7fffffffecc8, rbp = 0 --- Tracing command getty pid 1950 tid 100175 td 0xffffff000782bab0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80085900c, rsp = 0x7fffffffecc8, rbp = 0 --- Tracing command getty pid 1949 tid 100174 td 0xffffff000782c000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80085900c, rsp = 0x7fffffffecc8, rbp = 0 --- Tracing command cron pid 1883 tid 100157 td 0xffffff0007d51000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x80094839c, rsp = 0x7fffffffeb38, rbp = 0x3c --- Tracing command sendmail pid 1876 tid 100144 td 0xffffff0007900ab0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_sigsuspend() at kern_sigsuspend+0x9a sigsuspend() at sigsuspend+0x34 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip = 0x800d4c7dc, rsp = 0x7fffffffd898, rbp = 0x1 --- Tracing command sendmail pid 1872 tid 100148 td 0xffffff000782cab0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x640 select() at select+0x5d syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800df1f8c, rsp = 0x7fffffffcd88, rbp = 0x7fffffffce20 --- Tracing command sshd pid 1864 tid 100152 td 0xffffff0007b96390 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x640 select() at select+0x5d syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8016eaf8c, rsp = 0x7fffffffe4a8, rbp = 0x2 --- Tracing command rpc.lockd pid 1734 tid 100093 td 0xffffff0007907ab0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c svc_run_internal() at svc_run_internal+0x81a svc_run() at svc_run+0x94 nlm_syscall() at nlm_syscall+0x875 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (154, FreeBSD ELF64, nlm_syscall), rip = 0x8008c43dc, rsp = 0x7fffffffec38, rbp = 0x7fffffffed60 --- Tracing command rpc.statd pid 1727 tid 100100 td 0xffffff000753c720 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x640 select() at select+0x5d syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800851f8c, rsp = 0x7fffffffeb58, rbp = 0 --- Tracing command md1 pid 1654 tid 100107 td 0xffffff0007a37390 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 md_kthread() at md_kthread+0x15a fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81cfc7fd30, rbp = 0 --- Tracing command rpcbind pid 1599 tid 100145 td 0xffffff0007900720 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80091024c, rsp = 0x7fffffffca88, rbp = 0x800c5b0c0 --- Tracing command syslogd pid 1568 tid 100155 td 0xffffff0007d51720 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x640 select() at select+0x5d syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x80085bf8c, rsp = 0x7fffffffe158, rbp = 0x7fffffffe2c0 --- Tracing command devd pid 1273 tid 100094 td 0xffffff0007907720 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x640 select() at select+0x5d syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x4474bc, rsp = 0x7fffffffe898, rbp = 0x7fffffffe8b0 --- Tracing command dhclient pid 1272 tid 100098 td 0xffffff000753d000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80070024c, rsp = 0x7fffffffecd8, rbp = 0x493e0 --- Tracing command dhclient pid 1256 tid 100092 td 0xffffff0007908000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac poll() at poll+0x2b6 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80070024c, rsp = 0x7fffffffece8, rbp = 0x4 --- Tracing command moused pid 1213 tid 100088 td 0xffffff0007909000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x640 select() at select+0x5d syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x80097bf8c, rsp = 0x7fffffffe878, rbp = 0x7fffffffed10 --- Tracing command md0 pid 104 tid 100104 td 0xffffff000745f720 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 md_kthread() at md_kthread+0x15a fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81cfc70d30, rbp = 0 --- Tracing command flowcleaner pid 19 tid 100082 td 0xffffff0007540720 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _cv_timedwait() at _cv_timedwait+0x18c flowtable_cleaner() at flowtable_cleaner+0xd8 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81cd88ad30, rbp = 0 --- Tracing command softdepflush pid 18 tid 100081 td 0xffffff000745c000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 softdep_flush() at softdep_flush+0x259 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81cd885d30, rbp = 0 --- Tracing command vnlru pid 17 tid 100080 td 0xffffff000745c390 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 vnlru_proc() at vnlru_proc+0x5cf fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81cd880d30, rbp = 0 --- Tracing command syncer pid 16 tid 100079 td 0xffffff000745c720 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _cv_timedwait() at _cv_timedwait+0x18c sched_sync() at sched_sync+0x4db fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81cd87bd30, rbp = 0 --- Tracing command bufdaemon pid 15 tid 100078 td 0xffffff000745cab0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 buf_daemon() at buf_daemon+0x14a fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81cd876d30, rbp = 0 --- Tracing command pagezero pid 9 tid 100077 td 0xffffff000745d000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 vm_pagezero() at vm_pagezero+0x73 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81cd871d30, rbp = 0 --- Tracing command vmdaemon pid 8 tid 100076 td 0xffffff000745d390 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 vm_daemon() at vm_daemon+0x4d fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81cd86cd30, rbp = 0 --- Tracing command pagedaemon pid 7 tid 100075 td 0xffffff000745d720 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 vm_pageout() at vm_pageout+0x855 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81cd867d30, rbp = 0 --- Tracing command xpt_thrd pid 6 tid 100074 td 0xffffff000745dab0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 xpt_scanner_thread() at xpt_scanner_thread+0xd5 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81cd862d30, rbp = 0 --- Tracing command fdc0 pid 5 tid 100070 td 0xffffff000745eab0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 fdc_thread() at fdc_thread+0x74a fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81cd84ad30, rbp = 0 --- Tracing command usb pid 14 tid 100061 td 0xffffff0007301ab0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x15f fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81cd6fbd30, rbp = 0 --- Tracing command usb pid 14 tid 100060 td 0xffffff0007303000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x15f fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81cd6f6d30, rbp = 0 --- Tracing command usb pid 14 tid 100059 td 0xffffff0007303390 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x15f fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81cd6f1d30, rbp = 0 --- Tracing command usb pid 14 tid 100058 td 0xffffff0007303720 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x15f fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81cd6ecd30, rbp = 0 --- Tracing command usb pid 14 tid 100056 td 0xffffff0002ff3390 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x15f fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff814f6b4d30, rbp = 0 --- Tracing command usb pid 14 tid 100055 td 0xffffff0002ff3720 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x15f fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff814f6afd30, rbp = 0 --- Tracing command usb pid 14 tid 100054 td 0xffffff0002ff3ab0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x15f fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80001fcd30, rbp = 0 --- Tracing command usb pid 14 tid 100053 td 0xffffff0007181000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x15f fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80001f7d30, rbp = 0 --- Tracing command yarrow pid 13 tid 100041 td 0xffffff0002feeab0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 random_kthread() at random_kthread+0x1ad fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000177d30, rbp = 0 --- Tracing command g_down pid 4 tid 100039 td 0xffffff0002fef390 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 g_io_schedule_down() at g_io_schedule_down+0x245 g_down_procbody() at g_down_procbody+0x6f fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff800016dd30, rbp = 0 --- Tracing command g_up pid 3 tid 100038 td 0xffffff0002fef720 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 g_io_schedule_up() at g_io_schedule_up+0x133 g_up_procbody() at g_up_procbody+0x6f fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000168d30, rbp = 0 --- Tracing command g_event pid 2 tid 100037 td 0xffffff0002fefab0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 g_event_procbody() at g_event_procbody+0xa1 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000163d30, rbp = 0 --- Tracing command intr pid 12 tid 100072 td 0xffffff000745e390 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100071 td 0xffffff000745e720 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100069 td 0xffffff000745f000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81cd83bd30, rbp = 0 --- Tracing command intr pid 12 tid 100065 td 0xffffff0007300ab0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100064 td 0xffffff0007301000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100063 td 0xffffff0007301390 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100062 td 0xffffff0007301720 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81cd723d30, rbp = 0 --- Tracing command intr pid 12 tid 100057 td 0xffffff0007303ab0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81cd6e7d30, rbp = 0 --- Tracing command intr pid 12 tid 100052 td 0xffffff0007181390 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80001f2d30, rbp = 0 --- Tracing command intr pid 12 tid 100051 td 0xffffff0007181720 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100050 td 0xffffff0007181ab0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80001a4d30, rbp = 0 --- Tracing command intr pid 12 tid 100049 td 0xffffff0007182000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100047 td 0xffffff0007182720 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000195d30, rbp = 0 --- Tracing command intr pid 12 tid 100045 td 0xffffff0002febab0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100036 td 0xffffff0002ff3000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff800015ed30, rbp = 0 --- Tracing command intr pid 12 tid 100035 td 0xffffff0002fe5720 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000159d30, rbp = 0 --- Tracing command intr pid 12 tid 100034 td 0xffffff0002fe5ab0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000154d30, rbp = 0 --- Tracing command intr pid 12 tid 100033 td 0xffffff0002fe8000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff800014fd30, rbp = 0 --- Tracing command intr pid 12 tid 100032 td 0xffffff0002fe8390 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff800014ad30, rbp = 0 --- Tracing command intr pid 12 tid 100031 td 0xffffff0002fe8720 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000145d30, rbp = 0 --- Tracing command intr pid 12 tid 100030 td 0xffffff0002fe8ab0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000140d30, rbp = 0 --- Tracing command intr pid 12 tid 100029 td 0xffffff0002feb000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff800013bd30, rbp = 0 --- Tracing command intr pid 12 tid 100028 td 0xffffff0002feb390 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000136d30, rbp = 0 --- Tracing command intr pid 12 tid 100027 td 0xffffff0002feb720 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000131d30, rbp = 0 --- Tracing command intr pid 12 tid 100026 td 0xffffff0002fd5720 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff800012cd30, rbp = 0 --- Tracing command intr pid 12 tid 100025 td 0xffffff0002fd5ab0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000127d30, rbp = 0 --- Tracing command intr pid 12 tid 100024 td 0xffffff0002fe3000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000122d30, rbp = 0 --- Tracing command intr pid 12 tid 100023 td 0xffffff0002fe3390 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff800011dd30, rbp = 0 --- Tracing command intr pid 12 tid 100022 td 0xffffff0002fe3720 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000118d30, rbp = 0 --- Tracing command intr pid 12 tid 100021 td 0xffffff0002fe3ab0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000113d30, rbp = 0 --- Tracing command intr pid 12 tid 100020 td 0xffffff0002fe5000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100019 td 0xffffff0002fe5390 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000109d30, rbp = 0 --- Tracing command idle pid 11 tid 100018 td 0xffffff0002fd3ab0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sched_preempt() at sched_preempt+0xbc ipi_bitmap_handler() at ipi_bitmap_handler+0x70 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0x9b --- interrupt, rip = 0xffffffff808761f6, rsp = 0xffffff8000104b50, rbp = 0xffffff8000104b60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1f6 sched_idletd() at sched_idletd+0x123 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000104d30, rbp = 0 --- Tracing command idle pid 11 tid 100017 td 0xffffff0002fd4000 cpustop_handler() at cpustop_handler+0x40 ipi_nmi_handler() at ipi_nmi_handler+0x30 trap() at trap+0x19a nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff808761f6, rsp = 0xffffff8000037fe0, rbp = 0xffffff80000ffb60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1f6 sched_idletd() at sched_idletd+0x123 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000ffd30, rbp = 0 --- Tracing command idle pid 11 tid 100016 td 0xffffff0002fd4390 cpustop_handler() at cpustop_handler+0x40 ipi_nmi_handler() at ipi_nmi_handler+0x30 trap() at trap+0x19a nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff808761f6, rsp = 0xffffff800003efe0, rbp = 0xffffff80000fab60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1f6 sched_idletd() at sched_idletd+0x123 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000fad30, rbp = 0 --- Tracing command idle pid 11 tid 100015 td 0xffffff0002fd4720 cpustop_handler() at cpustop_handler+0x40 ipi_nmi_handler() at ipi_nmi_handler+0x30 trap() at trap+0x19a nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff808761f6, rsp = 0xffffff8000045fe0, rbp = 0xffffff80000f5b60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1f6 sched_idletd() at sched_idletd+0x123 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000f5d30, rbp = 0 --- Tracing command idle pid 11 tid 100014 td 0xffffff0002fd4ab0 cpustop_handler() at cpustop_handler+0x40 ipi_nmi_handler() at ipi_nmi_handler+0x30 trap() at trap+0x19a nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff808761f6, rsp = 0xffffff800004cfe0, rbp = 0xffffff80000f0b60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1f6 sched_idletd() at sched_idletd+0x123 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000f0d30, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xffffff0002fd5000 cpustop_handler() at cpustop_handler+0x40 ipi_nmi_handler() at ipi_nmi_handler+0x30 trap() at trap+0x19a nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff808761f6, rsp = 0xffffff8000053fe0, rbp = 0xffffff80000ebb60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1f6 sched_idletd() at sched_idletd+0x123 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000ebd30, rbp = 0 --- Tracing command idle pid 11 tid 100012 td 0xffffff0002fd5390 cpustop_handler() at cpustop_handler+0x40 ipi_nmi_handler() at ipi_nmi_handler+0x30 trap() at trap+0x19a nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff808761f6, rsp = 0xffffff800005afe0, rbp = 0xffffff80000e6b60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1f6 sched_idletd() at sched_idletd+0x123 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000e6d30, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xffffff0002fc9390 cpustop_handler() at cpustop_handler+0x40 ipi_nmi_handler() at ipi_nmi_handler+0x30 trap() at trap+0x19a nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff808761f6, rsp = 0xffffff8000061fe0, rbp = 0xffffff80000e1b60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1f6 sched_idletd() at sched_idletd+0x123 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000e1d30, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xffffff0002fc9720 cpustop_handler() at cpustop_handler+0x40 ipi_nmi_handler() at ipi_nmi_handler+0x30 trap() at trap+0x19a nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff808761f6, rsp = 0xffffff8000068fe0, rbp = 0xffffff80000dcb60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1f6 sched_idletd() at sched_idletd+0x123 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000dcd30, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xffffff0002fc9ab0 cpustop_handler() at cpustop_handler+0x40 ipi_nmi_handler() at ipi_nmi_handler+0x30 trap() at trap+0x19a nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff808761f6, rsp = 0xffffff800006ffe0, rbp = 0xffffff80000d7b60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1f6 sched_idletd() at sched_idletd+0x123 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000d7d30, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xffffff0002fd3000 cpustop_handler() at cpustop_handler+0x40 ipi_nmi_handler() at ipi_nmi_handler+0x30 trap() at trap+0x19a nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff808761f6, rsp = 0xffffff8000076fe0, rbp = 0xffffff80000d2b60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1f6 sched_idletd() at sched_idletd+0x123 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000d2d30, rbp = 0 --- Tracing command idle pid 11 tid 100007 td 0xffffff0002fd3390 cpustop_handler() at cpustop_handler+0x40 ipi_nmi_handler() at ipi_nmi_handler+0x30 trap() at trap+0x19a nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff808761f6, rsp = 0xffffff800007dfe0, rbp = 0xffffff80000cdb60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1f6 sched_idletd() at sched_idletd+0x123 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000cdd30, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xffffff0002fd3720 cpustop_handler() at cpustop_handler+0x40 ipi_nmi_handler() at ipi_nmi_handler+0x30 trap() at trap+0x19a nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff808761f6, rsp = 0xffffff8000084fe0, rbp = 0xffffff80000c8b60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1f6 sched_idletd() at sched_idletd+0x123 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000c8d30, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xffffff0002fc7000 cpustop_handler() at cpustop_handler+0x40 ipi_nmi_handler() at ipi_nmi_handler+0x30 trap() at trap+0x19a nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff808761f6, rsp = 0xffffff800008bfe0, rbp = 0xffffff80000c3b60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1f6 sched_idletd() at sched_idletd+0x123 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000c3d30, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xffffff0002fc7390 cpustop_handler() at cpustop_handler+0x40 ipi_nmi_handler() at ipi_nmi_handler+0x30 trap() at trap+0x19a nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff808761f6, rsp = 0xffffff8000092fe0, rbp = 0xffffff80000beb60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1f6 sched_idletd() at sched_idletd+0x123 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000bed30, rbp = 0 --- Tracing command idle pid 11 tid 100003 td 0xffffff0002fc7720 cpustop_handler() at cpustop_handler+0x40 ipi_nmi_handler() at ipi_nmi_handler+0x30 trap() at trap+0x19a nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff808761f6, rsp = 0xffffff8000099fe0, rbp = 0xffffff80000b9b60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1f6 sched_idletd() at sched_idletd+0x123 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000b9d30, rbp = 0 --- Tracing command init pid 1 tid 100002 td 0xffffff0002fc7ab0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip = 0x40cafc, rsp = 0x7fffffffe838, rbp = 0x401d80 --- Tracing command audit pid 10 tid 100001 td 0xffffff0002fc9000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a audit_worker() at audit_worker+0x77 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000afd30, rbp = 0 --- Tracing command kernel pid 0 tid 100073 td 0xffffff000745e000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 deadlkres() at deadlkres+0x1ec fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81cd85dd30, rbp = 0 --- Tracing command kernel pid 0 tid 100068 td 0xffffff0007300000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d msleep_spin() at msleep_spin+0x209 taskqueue_thread_loop() at taskqueue_thread_loop+0x62 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81cd836d30, rbp = 0 --- Tracing command kernel pid 0 tid 100067 td 0xffffff0007300390 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d msleep_spin() at msleep_spin+0x209 taskqueue_thread_loop() at taskqueue_thread_loop+0x62 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81cd831d30, rbp = 0 --- Tracing command kernel pid 0 tid 100066 td 0xffffff0007300720 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81cd82cd30, rbp = 0 --- Tracing command kernel pid 0 tid 100048 td 0xffffff0007182390 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff800019ad30, rbp = 0 --- Tracing command kernel pid 0 tid 100046 td 0xffffff0007182ab0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000190d30, rbp = 0 --- Tracing command kernel pid 0 tid 100044 td 0xffffff0002fee000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d msleep_spin() at msleep_spin+0x209 taskqueue_thread_loop() at taskqueue_thread_loop+0x62 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000186d30, rbp = 0 --- Tracing command kernel pid 0 tid 100043 td 0xffffff0002fee390 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d msleep_spin() at msleep_spin+0x209 taskqueue_thread_loop() at taskqueue_thread_loop+0x62 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000181d30, rbp = 0 --- Tracing command kernel pid 0 tid 100042 td 0xffffff0002fee720 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d msleep_spin() at msleep_spin+0x209 taskqueue_thread_loop() at taskqueue_thread_loop+0x62 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff800017cd30, rbp = 0 --- Tracing command kernel pid 0 tid 100040 td 0xffffff0002fef000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000172d30, rbp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xffffffff80c77180 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 scheduler() at scheduler+0x29c mi_startup() at mi_startup+0x59 btext() at btext+0x2c db> ps pid ppid pgrp uid state wmesg wchan cmd 16043 4883 50931 0 RE+ CPU 0 cp 4883 4869 50931 0 S+ wait 0xffffff03004ed8c0 sh 4882 4869 50931 0 S+ pipewr 0xffffff01f2b865b0 find 4878 0 0 0 DL mdwait 0xffffff01a6c39000 [md5] 4869 50931 50931 0 S+ wait 0xffffff01fb4fa000 sh 4659 0 0 0 SL - 0xffffffff80e42b60 [nfsiod 0] 50931 1975 50931 0 S+ wait 0xffffff02e49358c0 sh 49678 0 0 0 DL mdwait 0xffffff022bf1f000 [md9] 49665 0 0 0 DL mdwait 0xffffff02b151d000 [md8] 49652 0 0 0 DL mdwait 0xffffff019f2e9800 [md7] 49639 0 0 0 DL mdwait 0xffffff01838d9800 [md6] 2069 2068 2069 1001 S+ select 0xffffff000756f040 top 2068 2067 2068 1001 SWs+ wait 0xffffff01aa289000 sh 2067 2064 2064 1001 S select 0xffffff018305b9c0 sshd 2064 1864 2064 0 SWs sbwait 0xffffff01aa31f3ec sshd 1975 1971 1975 0 SW+ wait 0xffffff0007db1000 sh 1971 1970 1971 0 SW+ pause 0xffffff0007d3a960 csh 1970 1959 1970 1001 SW+ wait 0xffffff0007bca000 su 1959 1958 1959 1001 SWs+ wait 0xffffff0007b8a460 sh 1958 1955 1955 1001 S select 0xffffff018305aa40 sshd 1955 1864 1955 0 SWs sbwait 0xffffff0007e07be4 sshd 1954 1 1 0 S ttydcd 0xffffff00074660e8 getty 1953 1 1953 0 Ss+ ttyin 0xffffff0007453ca8 getty 1952 1 1952 0 Ss+ ttyin 0xffffff000747c0a8 getty 1951 1 1951 0 Ss+ ttyin 0xffffff000747c4a8 getty 1950 1 1950 0 Ss+ ttyin 0xffffff000747c8a8 getty 1949 1 1949 0 Ss+ ttyin 0xffffff000747cca8 getty 1883 1 1883 0 Ss nanslp 0xffffffff80c7a748 cron 1876 1 1876 25 Ss pause 0xffffff0007d3b500 sendmail 1872 1 1872 0 Ss select 0xffffff0007dcc240 sendmail 1864 1 1864 0 Ss select 0xffffff0007dcc040 sshd 1734 1 1734 0 Ss rpcsvc 0xffffff0007a502a0 NL: master 1727 1 1727 0 Ss select 0xffffff0007c2fb40 rpc.statd 1654 0 0 0 DL mdwait 0xffffff000751f000 [md1] 1599 1 1599 0 Ss select 0xffffff00075bd540 rpcbind 1568 1 1568 0 Ss select 0xffffff0007c7f7c0 syslogd 1273 1 1273 0 Ss select 0xffffff0007c7f640 devd 1272 1 1272 65 Ss select 0xffffff0007c7f840 dhclient 1256 1 1256 0 Ss select 0xffffff0007c7fbc0 dhclient 1213 1 1213 0 Ss select 0xffffff000753f4c0 moused 104 0 0 0 DL mdwait 0xffffff0007520000 [md0] 19 0 0 0 DL flowclea 0xffffffff80e3b9d0 [flowcleaner] 18 0 0 0 DL sdflush 0xffffffff80e4a958 [softdepflush] 17 0 0 0 DL vlruwt 0xffffff000753a8c0 [vnlru] 16 0 0 0 DL syncer 0xffffffff80e3b720 [syncer] 15 0 0 0 DL psleep 0xffffffff80e3b248 [bufdaemon] 9 0 0 0 DL pgzero 0xffffffff80e4c6cc [pagezero] 8 0 0 0 DL psleep 0xffffffff80e4ba68 [vmdaemon] 7 0 0 0 DL psleep 0xffffffff80e4ba2c [pagedaemon] 6 0 0 0 DL ccb_scan 0xffffffff80c42be0 [xpt_thrd] 5 0 0 0 DL - 0xffffff0007460048 [fdc0] 14 0 0 0 DL (threaded) [usb] 100061 D - 0xffffff80009dcdd0 [usbus1] 100060 D - 0xffffff80009dcd78 [usbus1] 100059 D - 0xffffff80009dcd20 [usbus1] 100058 D - 0xffffff80009dccc8 [usbus1] 100056 D - 0xffffff80009d4460 [usbus0] 100055 D - 0xffffff80009d4408 [usbus0] 100054 D - 0xffffff80009d43b0 [usbus0] 100053 D - 0xffffff80009d4358 [usbus0] 13 0 0 0 DL - 0xffffffff80c7a424 [yarrow] 4 0 0 0 DL - 0xffffffff80c76be8 [g_down] 3 0 0 0 DL - 0xffffffff80c76be0 [g_up] 2 0 0 0 DL - 0xffffffff80c76bd0 [g_event] 12 0 0 0 WL (threaded) [intr] 100072 I [irq1: atkbd0] 100071 I [irq7: ppc0] 100069 I [swi0: uart uart] 100065 I [irq20: atapci2] 100064 I [irq23: atapci1] 100063 I [irq15: ata1] 100062 I [irq14: ata0] 100057 I [irq22: ehci0] 100052 I [irq21: ohci0+] 100051 I [irq9: acpi0] 100050 I [swi6: task queue] 100049 I [swi6: Giant taskq] 100047 I [swi5: +] 100045 I [swi2: cambio] 100036 I [swi4: clock] 100035 I [swi4: clock] 100034 I [swi4: clock] 100033 I [swi4: clock] 100032 I [swi4: clock] 100031 I [swi4: clock] 100030 I [swi4: clock] 100029 I [swi4: clock] 100028 I [swi4: clock] 100027 I [swi4: clock] 100026 I [swi4: clock] 100025 I [swi4: clock] 100024 I [swi4: clock] 100023 I [swi4: clock] 100022 I [swi4: clock] 100021 I [swi4: clock] 100020 I [swi3: vm] 100019 I [swi1: netisr 0] 11 0 0 0 RL (threaded) [idle] 100018 CanRun [idle: cpu0] 100017 Run CPU 1 [idle: cpu1] 100016 Run CPU 2 [idle: cpu2] 100015 Run CPU 3 [idle: cpu3] 100014 Run CPU 4 [idle: cpu4] 100013 Run CPU 5 [idle: cpu5] 100012 Run CPU 6 [idle: cpu6] 100011 Run CPU 7 [idle: cpu7] 100010 Run CPU 8 [idle: cpu8] 100009 Run CPU 9 [idle: cpu9] 100008 Run CPU 10 [idle: cpu10] 100007 Run CPU 11 [idle: cpu11] 100006 Run CPU 12 [idle: cpu12] 100005 Run CPU 13 [idle: cpu13] 100004 Run CPU 14 [idle: cpu14] 100003 Run CPU 15 [idle: cpu15] 1 0 1 0 SLs wait 0xffffff0002fc48c0 [init] 10 0 0 0 DL audit_wo 0xffffffff80e49c90 [audit] 0 0 0 0 DLs (threaded) [kernel] 100073 D deadlkre 0xffffffff80c7a424 [deadlkres] 100068 D - 0xffffff0007431e00 [em1 taskq] 100067 D - 0xffffff0007433d00 [em0 taskq] 100066 D - 0xffffff00073c9080 [cxgbc0 taskq] 100048 D - 0xffffff0007183d80 [thread taskq] 100046 D - 0xffffff0007168380 [kqueue taskq] 100044 D - 0xffffff0007168680 [acpi_task_2] 100043 D - 0xffffff0007168680 [acpi_task_1] 100042 D - 0xffffff0007168680 [acpi_task_0] 100040 D - 0xffffff0002ff0580 [firmware taskq] 100000 D sched 0xffffffff80c76d20 [swapper] db> p/u time_uptime 18446744071574526512 db> reset -- Test scenario: t4.sh