GENERIC from Fri Dec 9 12:52:37 2011 +0200, r228344+amd64_ctx 7ad0f7c, vmcore.9 KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2011 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #2: Thu Dec 15 18:27:53 CET 2011 pho@x4.osted.lan:/var/tmp/deviant2/sys/amd64/compile/PHO amd64 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. CPU: AMD Phenom(tm) 9150e Quad-Core Processor (1800.02-MHz K8-class CPU) Origin = "AuthenticAMD" Id = 0x100f23 Family = 10 Model = 2 Stepping = 3 Features=0x178bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x7ff TSC: P-state invariant real memory = 4294967296 (4096 MB) avail memory = 3963416576 (3779 MB) : Trying to mount root from ufs:/dev/ada0s3a [rw]... WARNING: / was not properly dismounted Setting hostuuid: 00000000-0000-0000-0000-00218515337d. Setting hostid: 0x6b64ac17. Starting ddb. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: /dev/ada0s3a: 3315 files, 327447 used, 685568 free (960 frags, 85576 blocks, 0.1% fragmentation) /dev/ada0s1f: 20772 files, 882103 used, 130912 free (4392 frags, 15815 blocks, 0.4% fragmentation) /dev/ada0s1g: 25583 files, 3606120 used, 114707861 free (1245 frags, 14338327 blocks, 0.0% fragmentation) /dev/ada0s3d: 737247 files, 4297676 used, 5856523 free (162539 frags, 711748 blocks, 1.6% fragmentation) /dev/ada0s3e: 518740 files, 7212049 used, 17548144 free (15136 frags, 2191626 blocks, 0.1% fragmentation) Mounting local file systems:. Setting hostname: x4.osted.lan. re0: link state changed to DOWN Starting Network: lo0 re0 fwe0 fwip0. lo0: flags=8049 metric 0 mtu 16384 options=3 inet 127.0.0.1 netmask 0xff000000 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0xa nd6 options=21 re0: flags=8843 metric 0 mtu 1500 options=209b ether 00:21:85:15:33:7d inet 192.168.1.101 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::221:85ff:fe15:337d%re0 prefixlen 64 tentative scopeid 0x1 nd6 options=29 media: Ethernet autoselect (none) status: no carrier fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb nd6 options=29 ch 1 dma -1 fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 nd6 options=29 Starting devd. Starting Network: usbus0. Starting Network: usbus1. Starting Network: usbus2. Starting Network: usbus3. Starting Network: usbus4. Starting Network: usbus5. Starting Network: fwe0. fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb nd6 options=29 ch 1 dma -1 Starting Network: fwip0. fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 nd6 options=29 add net default: gateway 192.168.1.1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 re0: link state changed to UP ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/kde4/lib /usr/local/lib/qt4 32-bit compatibility ldconfig path: /usr/lib32 Creating and/or trimming log files. Starting syslogd. savecore: reboot Dec 15 21:18:58 x4 savecore: reboot savecore: writing core to vmcore.8 Writing crash summary to /var/crash/core.txt.8. lock order reversal: 1st 0xffffff80ef713258 bufwait (bufwait) @ kern/vfs_bio.c:2658 2nd 0xfffffe00061a0400 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:284 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at kdb_backtrace+0x37 _witness_debugger() at _witness_debugger+0x2c witness_checkorder() at witness_checkorder+0x854 _sx_xlock() at _sx_xlock+0x55 ufsdirhash_acquire() at ufsdirhash_acquire+0x44 ufsdirhash_add() at ufsdirhash_add+0x19 ufs_direnter() at ufs_direnter+0x6a0 ufs_makeinode() at ufs_makeinode+0x32e VOP_CREATE_APV() at VOP_CREATE_APV+0xb3 vn_open_cred() at vn_open_cred+0x46b kern_openat() at kern_openat+0x1f9 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (5, FreeBSD ELF64, sys_open), rip = 0x800947f6c, rsp = 0x7fffffffd798, rbp = 0x800c07050 --- Additional ABI support: linux. Starting rpcbind. NFS access cache time=60 Clearing /tmp (X related). Starting mountd. Starting nfsd. Recovering vi editor sessions:. Updating motd:. Starting ntpd. Configuring syscons: keymap blanktime. Starting sshd. Starting cron. Starting default moused. Starting inetd. Thu Dec 15 21:19:29 CET 2011 FreeBSD/amd64 (x4.osted.lan) (console) login: Dec 15 21:30:17 x4 su: pho to root on /dev/pts/0 lock order reversal: 1st 0xfffffe003b1f3098 ufs (ufs) @ kern/vfs_mount.c:1209 2nd 0xfffffe0040460cf0 devfs (devfs) @ ufs/ffs/ffs_vfsops.c:1371 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at kdb_backtrace+0x37 _witness_debugger() at _witness_debugger+0x2c witness_checkorder() at witness_checkorder+0x854 __lockmgr_args() at __lockmgr_args+0xddd vop_stdlock() at vop_stdlock+0x39 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x57 ffs_flushfiles() at ffs_flushfiles+0x109 softdep_flushfiles() at softdep_flushfiles+0x63 ffs_unmount() at ffs_unmount+0x1d3 dounmount() at dounmount+0x297 sys_unmount() at sys_unmount+0x280 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (22, FreeBSD ELF64, sys_unmount), rip = 0x8008afbac, rsp = 0x7fffffffcec8, rbp = 0x800c08010 --- kernel trap 12 with interrupts disabled Fatal trap 12: page fault while in kernel mode cpuid = 2; apic id = 02 fault virtual address = 0xffffff81153a7d78 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff80b122e6 stack pointer = 0x28:0xffffff81153bba80 frame pointer = 0x28:0xffffff81153bbaf0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = resume, IOPL = 0 current process = 66098 (syscall4) [ thread pid 66098 tid 8849028 ] Stopped at done_store_dr+0x33: movq 0x68(%r8),%rcx db> findstack 0xffffff81153a7d78 Thread 0xfffffe004d7308c0 db> show thread 0xfffffe004d7308c0 Thread 8849024 at 0xfffffe004d7308c0: proc (pid 66098): 0xfffffe0006091000 name: syscall4 stack: 0xffffff81153a4000-0xffffff81153a7fff flags: 0x9884 pflags: 0 state: RUNNING (CPU 255) priority: 152 container lock: sched lock 2 (0xffffffff81165c00) db> bt 8849024 Tracing pid 66098 tid 8849024 td 0xfffffe004d7308c0 db> run pho db:0:pho> bt Tracing pid 66098 tid 8849028 td 0xfffffe004d3a98c0 done_store_dr() at done_store_dr+0x33 mi_switch() at mi_switch+0x1ea turnstile_wait() at turnstile_wait+0x274 _mtx_lock_sleep() at _mtx_lock_sleep+0xd6 _mtx_lock_flags() at _mtx_lock_flags+0xd4 ast() at ast+0x407 doreti_ast() at doreti_ast+0x1f db:0:bt> show allpcpu Current CPU: 2 cpuid = 0 dynamic pcpu = 0x4ce680 curthread = 0xfffffe004daab000: pid 66098 "syscall4" curpcb = 0xffffff81153b1d10 fpcurthread = none idlethread = 0xfffffe00028fd460: tid 100003 "idle: cpu0" curpmap = 0xfffffe004d644be8 tssp = 0xffffffff813668e0 commontssp = 0xffffffff813668e0 rsp0 = 0xffffff81153b1d10 gs32p = 0xffffffff81364a38 ldt = 0xffffffff81364a78 tss = 0xffffffff81364a68 spin locks held: cpuid = 1 dynamic pcpu = 0xffffff807f108680 curthread = 0xfffffe004d9aa8c0: pid 66098 "syscall4" curpcb = 0xffffff81153edd10 fpcurthread = none idlethread = 0xfffffe00028fd000: tid 100004 "idle: cpu1" curpmap = 0xfffffe004d644be8 tssp = 0xffffffff81366948 commontssp = 0xffffffff81366948 rsp0 = 0xffffff81153edd10 gs32p = 0xffffffff81364aa0 ldt = 0xffffffff81364ae0 tss = 0xffffffff81364ad0 spin locks held: cpuid = 2 dynamic pcpu = 0xffffff807f10f680 curthread = 0xfffffe004d3a98c0: pid 66098 "syscall4" curpcb = 0xffffff81153bbd10 fpcurthread = none idlethread = 0xfffffe00029088c0: tid 100005 "idle: cpu2" curpmap = 0xfffffe004d644be8 tssp = 0xffffffff813669b0 commontssp = 0xffffffff813669b0 rsp0 = 0xffffff81153bbd10 gs32p = 0xffffffff81364b08 ldt = 0xffffffff81364b48 tss = 0xffffffff81364b38 spin locks held: cpuid = 3 dynamic pcpu = 0xffffff807f116680 curthread = 0xfffffe004d3a2000: pid 66098 "syscall4" curpcb = 0xffffff81153b6d10 fpcurthread = none idlethread = 0xfffffe0002908460: tid 100006 "idle: cpu3" curpmap = 0xfffffe004d644be8 tssp = 0xffffffff81366a18 commontssp = 0xffffffff81366a18 rsp0 = 0xffffff81153b6d10 gs32p = 0xffffffff81364b70 ldt = 0xffffffff81364bb0 tss = 0xffffffff81364ba0 spin locks held: db:0:allpcpu> show alllocks db:0:alllocks> show lockedvnods Locked vnodes db:0:lockedvnods> show mount 0xfffffe000617bac8 /dev/ada0s3a on / (ufs) 0xfffffe000617c000 devfs on /dev (devfs) 0xfffffe000623c730 /dev/ada0s1f on /home (ufs) 0xfffffe000623c398 /dev/ada0s1g on /tmp (ufs) 0xfffffe000623c000 /dev/ada0s3d on /usr (ufs) 0xfffffe000617cac8 /dev/ada0s3e on /var (ufs) 0xfffffe000617c730 /dev/md5a on /mnt (ufs) More info: show mount db:0:mount> ps pid ppid pgrp uid state wmesg wchan cmd 66098 50992 2276 65534 T+ (threaded) syscall4 8849043 L *process 0xfffffe00064b99c0 syscall4 8849040 L *process 0xfffffe00064b99c0 syscall4 8849039 L *process 0xfffffe00064b99c0 syscall4 8849038 Run CPU 1 syscall4 8849037 RunQ syscall4 8849036 L *process 0xfffffe00064b99c0 syscall4 8849035 L *process 0xfffffe00064b99c0 syscall4 8849034 L *process 0xfffffe00064b99c0 syscall4 8849033 RunQ syscall4 8849032 L *process 0xfffffe00064b99c0 syscall4 8849031 L *process 0xfffffe00064b99c0 syscall4 8849030 L *process 0xfffffe00064b99c0 syscall4 8849029 RunQ syscall4 8849028 L *process 0xfffffe00064b99c0 syscall4 8849027 Run CPU 3 syscall4 8849025 RunQ syscall4 8849024 Run CPU 255 syscall4 8849022 L *process 0xfffffe00064b99c0 syscall4 8849020 RunQ syscall4 8849019 RunQ syscall4 8848503 s syscall4 8848494 L *process 0xfffffe00064b99c0 syscall4 8848491 L *process 0xfffffe00064b99c0 syscall4 8848484 CanRun syscall4 8848483 RunQ syscall4 8847956 RunQ syscall4 8847952 RunQ syscall4 8847948 L *process 0xfffffe00064b99c0 syscall4 8847728 RunQ syscall4 66057 2679 66057 1001 S+ nanslp 0xffffffff8114e028 sleep 65762 5005 4995 1001 S nanslp 0xffffffff8114e028 sleep 62079 50992 2276 65534 TX+ (threaded) syscall4 8751690 s syscall4 8751689 s syscall4 8751688 s umtxn 0xfffffe000628b880 syscall4 8751687 s syscall4 8751686 s umtxn 0xfffffe0047ff4300 syscall4 8751685 s umtxn 0xfffffe004d8ce580 syscall4 8751684 s syscall4 8751683 s syscall4 8751682 s syscall4 8751681 s umtxn 0xfffffe00064d6a00 syscall4 8751680 s syscall4 8751679 s umtxn 0xfffffe004da4d080 syscall4 8751678 s umtxn 0xfffffe0006cbca00 syscall4 8751677 s umtxn 0xfffffe004d9a4d00 syscall4 8751676 s syscall4 8751675 s syscall4 8751674 s syscall4 8751673 s syscall4 8751672 s umtxn 0xfffffe0006e41a80 syscall4 8751671 s syscall4 8751670 s syscall4 8751669 s syscall4 8751668 s syscall4 8751667 s syscall4 8751666 s syscall4 8751154 s syscall4 8751152 s syscall4 8751150 s umtxn 0xfffffe004d79b680 syscall4 8751149 s syscall4 8751145 s syscall4 8751142 s syscall4 8751141 s umtxn 0xfffffe0006e1bb00 syscall4 8751138 s syscall4 8751137 s syscall4 8751131 s syscall4 8751130 s syscall4 8750618 s syscall4 8750614 s syscall4 8750610 s syscall4 8750596 s syscall4 8750594 s syscall4 8750588 s syscall4 8750077 s syscall4 8750076 s syscall4 8750073 s syscall4 8750071 s syscall4 8750059 s syscall4 8749540 s syscall4 8749534 s syscall4 8749012 s syscall4 8746797 s syscall4 8749494 Ss uwait 0xfffffe0006e42e00 syscall4 50993 69772 2276 0 S+ nanslp 0xffffffff8114e028 sleep 50992 69772 2276 65534 S+ wait 0xfffffe004d8c5910 syscall4 69803 0 0 0 DL mdwait 0xfffffe000290f800 [md5] 69772 2276 2276 0 S+ wait 0xfffffe004d8c6000 sh 5006 4995 4995 1001 S piperd 0xfffffe0006a0e2d8 awk 5005 4995 4995 1001 S wait 0xfffffe0006b2a488 sh 5002 5001 5002 1001 Ss kqread 0xfffffe0006117800 tail 5001 4958 4958 1001 S select 0xfffffe000631f640 sshd 4999 4998 4999 1001 Ss+ select 0xfffffe0006958840 top 4998 4960 4960 1001 S select 0xfffffe0006958bc0 sshd 4995 4994 4995 1001 Ss wait 0xfffffe004055c000 sh 4994 4955 4955 1001 S select 0xfffffe00069585c0 sshd 4960 2072 4960 0 Ss select 0xfffffe003b1321c0 sshd 4958 2072 4958 0 Ss select 0xfffffe0006996c40 sshd 4955 2072 4955 0 Ss select 0xfffffe00069583c0 sshd 2679 2677 2679 1001 Ss+ wait 0xfffffe0006beb000 bash 2677 2633 2633 1001 S select 0xfffffe003b132240 sshd 2633 2072 2633 0 Ss select 0xfffffe0006996440 sshd 2276 2270 2276 0 S+ wait 0xfffffe0006beb488 sh 2270 2266 2270 0 S+ wait 0xfffffe0006aa8000 bash 2266 2265 2266 0 S+ pause 0xfffffe00062d60a0 csh 2265 2261 2265 1001 S+ wait 0xfffffe0006291910 su 2261 2260 2261 1001 Ss+ wait 0xfffffe00069f6488 bash 2260 2258 2258 1001 S select 0xfffffe00028ed440 sshd 2258 2072 2258 0 Ss select 0xfffffe0006996040 sshd 2197 1 2197 0 Ss+ ttyin 0xfffffe0002daf8a8 getty 2196 1 2196 0 Ss+ ttyin 0xfffffe0002dafca8 getty 2195 1 2195 0 Ss+ ttyin 0xfffffe0002db00a8 getty 2194 1 2194 0 Ss+ ttyin 0xfffffe0002db04a8 getty 2193 1 2193 0 Ss+ ttyin 0xfffffe000293d8a8 getty 2192 1 2192 0 Ss+ ttyin 0xfffffe0002b178a8 getty 2191 1 2191 0 Ss+ ttyin 0xfffffe000293d4a8 getty 2190 1 2190 0 Ss+ ttyin 0xfffffe000293c4a8 getty 2189 1 2189 0 Ss+ ttyin 0xfffffe0002b170a8 getty 2150 1 2150 0 Ss select 0xfffffe00069580c0 inetd 2117 1 2117 0 Ss select 0xfffffe0006996140 moused 2093 1 2093 0 Ss nanslp 0xffffffff8114e028 cron 2086 1 2086 25 Ss pause 0xfffffe0006955528 sendmail 2080 1 2080 0 Ss select 0xfffffe0006ab56c0 sendmail 2072 1 2072 0 Ss select 0xfffffe00028ed7c0 sshd 2026 1 2026 0 Ss select 0xfffffe0006ab5640 ntpd 1898 1892 1892 0 S (threaded) nfsd 100129 S rpcsvc 0xfffffe00060ae820 nfsd: service 100128 S rpcsvc 0xfffffe00060ae8a0 nfsd: service 100127 S rpcsvc 0xfffffe00060ae920 nfsd: service 100113 S rpcsvc 0xfffffe00064df020 nfsd: master 1892 1 1892 0 Ss select 0xfffffe00069962c0 nfsd 1883 1 1883 0 Ss select 0xfffffe00028ed9c0 mountd 1785 1 1785 0 Ss select 0xfffffe00028ec040 rpcbind 1704 1 1704 0 Ss select 0xfffffe00028ec2c0 syslogd 1496 1 1496 0 Ss select 0xfffffe00028ede40 devd 18 0 0 0 DL sdflush 0xffffffff81342978 [softdepflush] 17 0 0 0 DL syncer 0xffffffff81339c20 [syncer] 16 0 0 0 DL vlruwt 0xfffffe0006092000 [vnlru] 9 0 0 0 DL psleep 0xffffffff813396a8 [bufdaemon] 8 0 0 0 DL pgzero 0xffffffff813547bc [pagezero] 7 0 0 0 DL psleep 0xffffffff81353a68 [vmdaemon] 6 0 0 0 DL psleep 0xffffffff81353a2c [pagedaemon] 5 0 0 0 DL ccb_scan 0xffffffff8110d860 [xpt_thrd] 4 0 0 0 DL waiting_ 0xffffffff8133cc80 [sctp_iterator] 3 0 0 0 DL - 0xfffffe0002d60048 [fdc0] 2 0 0 0 SL - 0xffffff8000866000 [fw0_probe] 15 0 0 0 DL (threaded) [usb] 100058 D - 0xffffff8000863e18 [usbus5] 100057 D - 0xffffff8000863dc0 [usbus5] 100056 D - 0xffffff8000863d68 [usbus5] 100055 D - 0xffffff8000863d10 [usbus5] 100053 D - 0xffffff800085b460 [usbus4] 100052 D - 0xffffff800085b408 [usbus4] 100051 D - 0xffffff800085b3b0 [usbus4] 100050 D - 0xffffff800085b358 [usbus4] 100049 D - 0xffffff8000858460 [usbus3] 100048 D - 0xffffff8000858408 [usbus3] 100047 D - 0xffffff80008583b0 [usbus3] 100046 D - 0xffffff8000858358 [usbus3] 100045 D - 0xffffff8000855460 [usbus2] 100044 D - 0xffffff8000855408 [usbus2] 100043 D - 0xffffff80008553b0 [usbus2] 100042 D - 0xffffff8000855358 [usbus2] 100040 D - 0xffffff8000852460 [usbus1] 100039 D - 0xffffff8000852408 [usbus1] 100038 D - 0xffffff80008523b0 [usbus1] 100037 D - 0xffffff8000852358 [usbus1] 100035 D - 0xffffff800084f460 [usbus0] 100034 D - 0xffffff800084f408 [usbus0] 100033 D - 0xffffff800084f3b0 [usbus0] 100032 D - 0xffffff800084f358 [usbus0] 14 0 0 0 DL - 0xffffffff8114ccc4 [yarrow] 13 0 0 0 DL (threaded) [geom] 100015 D - 0xffffffff81149130 [g_down] 100014 D - 0xffffffff81149128 [g_up] 100013 D - 0xffffffff81149118 [g_event] 12 0 0 0 WL (threaded) [intr] 100066 I [irq12: psm0] 100065 I [irq1: atkbd0] 100063 I [swi0: uart] 100060 I [irq20: fwohci0] 100059 I [irq14: ata0] 100054 I [irq19: ehci0] 100041 I [irq18: ohci2 ohci4] 100036 I [irq17: ohci1 ohci3] 100031 I [irq16: hdac1 ohci0] 100030 I [irq22: ahci0] 100029 I [irq257: re0] 100028 I [irq256: hdac0] 100026 I [swi5: +] 100020 I [swi2: cambio] 100019 I [swi6: task queue] 100018 I [swi6: Giant taskq] 100012 I [swi3: vm] 100011 I [swi4: clock] 100010 I [swi4: clock] 100009 I [swi4: clock] 100008 I [swi4: clock] 100007 I [swi1: netisr 0] 11 0 0 0 RL (threaded) [idle] 100006 CanRun [idle: cpu3] 100005 CanRun [idle: cpu2] 100004 CanRun [idle: cpu1] 100003 CanRun [idle: cpu0] 1 0 1 0 SLs wait 0xfffffe00028fb910 [init] 10 0 0 0 DL audit_wo 0xffffffff813418f0 [audit] 0 0 0 0 DLs (threaded) [kernel] 100077 D - 0xfffffe000606e200 [mca taskq] 100067 D - 0xffffffff8114ccc4 [deadlkres] 100061 D - 0xfffffe0002d4db80 [fw0_taskq] 100027 D - 0xfffffe0002a68b00 [thread taskq] 100025 D - 0xfffffe0002a68c00 [acpi_task_2] 100024 D - 0xfffffe0002a68c00 [acpi_task_1] 100023 D - 0xfffffe0002a68c00 [acpi_task_0] 100022 D - 0xfffffe0002a68c80 [kqueue taskq] 100021 D - 0xfffffe0002a68d00 [ffs_trim taskq] 100016 D - 0xfffffe00028cea00 [firmware taskq] 100000 D sched 0xffffffff81149260 [swapper] db:0:ps> allt Tracing command syscall4 pid 66098 tid 8849043 td 0xfffffe0006dfe460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea turnstile_wait() at turnstile_wait+0x274 _mtx_lock_sleep() at _mtx_lock_sleep+0xd6 _mtx_lock_flags() at _mtx_lock_flags+0xd4 ast() at ast+0x407 doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 66098 tid 8849040 td 0xfffffe004d9a68c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea turnstile_wait() at turnstile_wait+0x274 _mtx_lock_sleep() at _mtx_lock_sleep+0xd6 _mtx_lock_flags() at _mtx_lock_flags+0xd4 ast() at ast+0x407 doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 66098 tid 8849039 td 0xfffffe0006cdf8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea turnstile_wait() at turnstile_wait+0x274 _mtx_lock_sleep() at _mtx_lock_sleep+0xd6 _mtx_lock_flags() at _mtx_lock_flags+0xd4 ast() at ast+0x407 doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 66098 tid 8849038 td 0xfffffe004d9aa8c0 cpustop_handler() at cpustop_handler+0x2b ipi_nmi_handler() at ipi_nmi_handler+0x50 trap() at trap+0x1a8 nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff80b7fe57, rsp = 0xffffff8000213fe0, rbp = 0xffffff81153edab0 --- DELAY() at DELAY+0xa7 _mtx_lock_spin() at _mtx_lock_spin+0x6a _mtx_lock_spin_flags() at _mtx_lock_spin_flags+0x115 sched_add() at sched_add+0xcf turnstile_unpend() at turnstile_unpend+0x1d1 _mtx_unlock_sleep() at _mtx_unlock_sleep+0x68 thread_suspend_check() at thread_suspend_check+0x331 ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 66098 tid 8849037 td 0xfffffe004dac18c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x24d kern_nanosleep() at kern_nanosleep+0x118 sys_nanosleep() at sys_nanosleep+0x6e amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x800b535dc, rsp = 0x7ffffa3d1f28, rbp = 0x801012808 --- Tracing command syscall4 pid 66098 tid 8849036 td 0xfffffe00061fb460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea turnstile_wait() at turnstile_wait+0x274 _mtx_lock_sleep() at _mtx_lock_sleep+0xd6 _mtx_lock_flags() at _mtx_lock_flags+0xd4 ast() at ast+0x407 doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 66098 tid 8849035 td 0xfffffe004d9a3460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea turnstile_wait() at turnstile_wait+0x274 _mtx_lock_sleep() at _mtx_lock_sleep+0xd6 _mtx_lock_flags() at _mtx_lock_flags+0xd4 ast() at ast+0x407 doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 66098 tid 8849034 td 0xfffffe00064af000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea turnstile_wait() at turnstile_wait+0x274 _mtx_lock_sleep() at _mtx_lock_sleep+0xd6 _mtx_lock_flags() at _mtx_lock_flags+0xd4 ast() at ast+0x407 doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 66098 tid 8849033 td 0xfffffe004d6bc460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x24d kern_nanosleep() at kern_nanosleep+0x118 sys_nanosleep() at sys_nanosleep+0x6e amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x800b535dc, rsp = 0x7ffffabd5f28, rbp = 0x801011808 --- Tracing command syscall4 pid 66098 tid 8849032 td 0xfffffe0006cf8460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea turnstile_wait() at turnstile_wait+0x274 _mtx_lock_sleep() at _mtx_lock_sleep+0xd6 _mtx_lock_flags() at _mtx_lock_flags+0xd4 ast() at ast+0x407 doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 66098 tid 8849031 td 0xfffffe0006de28c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea turnstile_wait() at turnstile_wait+0x274 _mtx_lock_sleep() at _mtx_lock_sleep+0xd6 _mtx_lock_flags() at _mtx_lock_flags+0xd4 ast() at ast+0x407 doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 66098 tid 8849030 td 0xfffffe00061e6460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea turnstile_wait() at turnstile_wait+0x274 _mtx_lock_sleep() at _mtx_lock_sleep+0xd6 _mtx_lock_flags() at _mtx_lock_flags+0xd4 ast() at ast+0x407 doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 66098 tid 8849029 td 0xfffffe0006e168c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x24d kern_nanosleep() at kern_nanosleep+0x118 sys_nanosleep() at sys_nanosleep+0x6e amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x800b535dc, rsp = 0x7ffffb3d9f28, rbp = 0x801010808 --- Tracing command syscall4 pid 66098 tid 8849028 td 0xfffffe004d3a98c0 done_store_dr() at done_store_dr+0x33 mi_switch() at mi_switch+0x1ea turnstile_wait() at turnstile_wait+0x274 _mtx_lock_sleep() at _mtx_lock_sleep+0xd6 _mtx_lock_flags() at _mtx_lock_flags+0xd4 ast() at ast+0x407 doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 66098 tid 8849027 td 0xfffffe004d3a2000 cpustop_handler() at cpustop_handler+0x2b ipi_nmi_handler() at ipi_nmi_handler+0x50 trap() at trap+0x1a8 nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff8082b902, rsp = 0xffffff8000221fe0, rbp = 0xffffff81153b6b80 --- _mtx_lock_sleep() at _mtx_lock_sleep+0x102 _mtx_lock_flags() at _mtx_lock_flags+0xd4 thread_suspend_check() at thread_suspend_check+0x34f ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 66098 tid 8849025 td 0xfffffe00064e78c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x24d kern_nanosleep() at kern_nanosleep+0x118 sys_nanosleep() at sys_nanosleep+0x6e amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x800b535dc, rsp = 0x7ffffbbddf28, rbp = 0x80100f808 --- Tracing command syscall4 pid 66098 tid 8849024 td 0xfffffe004d7308c0 Tracing command sleep pid 66057 tid 8840233 td 0xfffffe0006ceb8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x24d kern_nanosleep() at kern_nanosleep+0x118 sys_nanosleep() at sys_nanosleep+0x6e amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x80092f5dc, rsp = 0x7fffffffd998, rbp = 0x7fffffffda10 --- Tracing command sleep pid 65762 tid 8834241 td 0xfffffe0006cec000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x24d kern_nanosleep() at kern_nanosleep+0x118 sys_nanosleep() at sys_nanosleep+0x6e amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x80092f5dc, rsp = 0x7fffffffdb58, rbp = 0x7fffffffdbd0 --- Tracing command syscall4 pid 62079 tid 8751690 td 0xfffffe0006ca38c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_switch() at thread_suspend_switch+0x103 ptracestop() at ptracestop+0x98 cursig() at cursig+0x4dd ast() at ast+0x1b6 Xfast_syscall() at Xfast_syscall+0x15c --- syscall (78, FreeBSD ELF64, sys_mincore), rip = 0x800ad0ccc, rsp = 0x7ffff97cbf68, rbp = 0x8d --- Tracing command syscall4 pid 62079 tid 8751689 td 0xfffffe00064aa8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8751688 td 0xfffffe0006ca3000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x272 sleepq_catch_signals() at sleepq_catch_signals+0x2c4 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a _do_lock_normal() at _do_lock_normal+0x66f do_lock_umutex() at do_lock_umutex+0x17c __umtx_op_wait_umutex() at __umtx_op_wait_umutex+0x63 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x80085f9bc, rsp = 0x7ffff9bcde98, rbp = 0x858a48 --- Tracing command syscall4 pid 62079 tid 8751687 td 0xfffffe004d6bb460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8751686 td 0xfffffe003b30c8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x272 sleepq_catch_signals() at sleepq_catch_signals+0x2c4 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a _do_lock_normal() at _do_lock_normal+0x66f do_lock_umutex() at do_lock_umutex+0x17c __umtx_op_wait_umutex() at __umtx_op_wait_umutex+0x63 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x80085f9bc, rsp = 0x7ffff9fcfe98, rbp = 0x858a46 --- Tracing command syscall4 pid 62079 tid 8751685 td 0xfffffe004d8cf8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x272 sleepq_catch_signals() at sleepq_catch_signals+0x2c4 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a _do_lock_normal() at _do_lock_normal+0x66f do_lock_umutex() at do_lock_umutex+0x17c __umtx_op_wait_umutex() at __umtx_op_wait_umutex+0x63 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x80085f9bc, rsp = 0x7ffffa1d0e98, rbp = 0x858a45 --- Tracing command syscall4 pid 62079 tid 8751684 td 0xfffffe004d87b000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8751683 td 0xfffffe004d98a8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e Xfast_syscall() at Xfast_syscall+0x15c --- syscall (0, FreeBSD ELF64, nosys), rip = 0x80085f9bc, rsp = 0x7ffffa5d2ef8, rbp = 0x801012400 --- Tracing command syscall4 pid 62079 tid 8751682 td 0xfffffe004d884460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8751681 td 0xfffffe0006e028c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x272 sleepq_catch_signals() at sleepq_catch_signals+0x2c4 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a _do_lock_normal() at _do_lock_normal+0x66f do_lock_umutex() at do_lock_umutex+0x17c __umtx_op_wait_umutex() at __umtx_op_wait_umutex+0x63 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x80085f9bc, rsp = 0x7ffffa9d4e98, rbp = 0x858a41 --- Tracing command syscall4 pid 62079 tid 8751680 td 0xfffffe004d317460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8751679 td 0xfffffe004d730460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x272 sleepq_catch_signals() at sleepq_catch_signals+0x2c4 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a _do_lock_normal() at _do_lock_normal+0x66f do_lock_umutex() at do_lock_umutex+0x17c __umtx_op_wait_umutex() at __umtx_op_wait_umutex+0x63 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x80085f9bc, rsp = 0x7ffffadd6e98, rbp = 0x858a3f --- Tracing command syscall4 pid 62079 tid 8751678 td 0xfffffe0006e158c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x272 sleepq_catch_signals() at sleepq_catch_signals+0x2c4 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a _do_lock_normal() at _do_lock_normal+0x66f do_lock_umutex() at do_lock_umutex+0x17c __umtx_op_wait_umutex() at __umtx_op_wait_umutex+0x63 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x80085f9bc, rsp = 0x7ffffafd7e98, rbp = 0x858a3e --- Tracing command syscall4 pid 62079 tid 8751677 td 0xfffffe004dab18c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x272 sleepq_catch_signals() at sleepq_catch_signals+0x2c4 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a _do_lock_normal() at _do_lock_normal+0x66f do_lock_umutex() at do_lock_umutex+0x17c __umtx_op_wait_umutex() at __umtx_op_wait_umutex+0x63 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x80085f9bc, rsp = 0x7ffffb1d8e98, rbp = 0x858a3d --- Tracing command syscall4 pid 62079 tid 8751676 td 0xfffffe004d6b7000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8751675 td 0xfffffe004d9a7000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8751674 td 0xfffffe004d8cf000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8751673 td 0xfffffe0006dcb460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x272 sleepq_catch_signals() at sleepq_catch_signals+0x2c4 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a _do_lock_normal() at _do_lock_normal+0x66f do_lock_umutex() at do_lock_umutex+0x17c __umtx_op_wait_umutex() at __umtx_op_wait_umutex+0x63 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x80085f9bc, rsp = 0x7ffffb9dce98, rbp = 0x858a39 --- Tracing command syscall4 pid 62079 tid 8751672 td 0xfffffe004d38f000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x272 sleepq_catch_signals() at sleepq_catch_signals+0x2c4 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a _do_lock_normal() at _do_lock_normal+0x66f do_lock_umutex() at do_lock_umutex+0x17c __umtx_op_wait_umutex() at __umtx_op_wait_umutex+0x63 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x80085f9bc, rsp = 0x7ffffbbdde98, rbp = 0x858a38 --- Tracing command syscall4 pid 62079 tid 8751671 td 0xfffffe004d3aa8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8751670 td 0xfffffe004da848c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8751669 td 0xfffffe004d1bf8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e Xfast_syscall() at Xfast_syscall+0x15c --- syscall (62079), rip = 0x800b718ec, rsp = 0x7ffffc1e0f18, rbp = 0x1b1 --- Tracing command syscall4 pid 62079 tid 8751668 td 0xfffffe00474028c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8751667 td 0xfffffe004d87c8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8751666 td 0xfffffe0006497000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x272 sleepq_catch_signals() at sleepq_catch_signals+0x2c4 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a _do_lock_normal() at _do_lock_normal+0x66f do_lock_umutex() at do_lock_umutex+0x17c __umtx_op_wait_umutex() at __umtx_op_wait_umutex+0x63 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x80085f9bc, rsp = 0x7ffffc7e3e98, rbp = 0x858a32 --- Tracing command syscall4 pid 62079 tid 8751154 td 0xfffffe004d973000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8751152 td 0xfffffe003b0a88c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8751150 td 0xfffffe004d8b2000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x272 sleepq_catch_signals() at sleepq_catch_signals+0x2c4 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a _do_lock_normal() at _do_lock_normal+0x66f do_lock_umutex() at do_lock_umutex+0x17c __umtx_op_wait_umutex() at __umtx_op_wait_umutex+0x63 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x80085f9bc, rsp = 0x7ffffcde6e98, rbp = 0x85882e --- Tracing command syscall4 pid 62079 tid 8751149 td 0xfffffe004d6bc8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8751145 td 0xfffffe004d9728c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8751142 td 0xfffffe004d8eb8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x272 sleepq_catch_signals() at sleepq_catch_signals+0x2c4 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a _do_lock_normal() at _do_lock_normal+0x66f do_lock_umutex() at do_lock_umutex+0x17c __umtx_op_wait_umutex() at __umtx_op_wait_umutex+0x63 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x80085f9bc, rsp = 0x7ffffd3e9e98, rbp = 0x858826 --- Tracing command syscall4 pid 62079 tid 8751141 td 0xfffffe003b778000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x272 sleepq_catch_signals() at sleepq_catch_signals+0x2c4 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a _do_lock_normal() at _do_lock_normal+0x66f do_lock_umutex() at do_lock_umutex+0x17c __umtx_op_wait_umutex() at __umtx_op_wait_umutex+0x63 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x80085f9bc, rsp = 0x7ffffd5eae98, rbp = 0x858825 --- Tracing command syscall4 pid 62079 tid 8751138 td 0xfffffe0006275460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a _do_lock_normal() at _do_lock_normal+0x66f do_lock_umutex() at do_lock_umutex+0x17c __umtx_op_wait_umutex() at __umtx_op_wait_umutex+0x63 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x80085f9bc, rsp = 0x7ffffd7ebe98, rbp = 0x858822 --- Tracing command syscall4 pid 62079 tid 8751137 td 0xfffffe004d3a1000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8751131 td 0xfffffe004d6ad8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8751130 td 0xfffffe004d98b460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8750618 td 0xfffffe0006d03000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x272 sleepq_catch_signals() at sleepq_catch_signals+0x2c4 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a _do_lock_normal() at _do_lock_normal+0x66f do_lock_umutex() at do_lock_umutex+0x17c __umtx_op_wait_umutex() at __umtx_op_wait_umutex+0x63 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x80085f9bc, rsp = 0x7ffffdfefea8, rbp = 0x85861a --- Tracing command syscall4 pid 62079 tid 8750614 td 0xfffffe004d8b8460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8750610 td 0xfffffe004d38e000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8750596 td 0xfffffe0006e578c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8750594 td 0xfffffe004d3268c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8750588 td 0xfffffe004d3a5000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8750077 td 0xfffffe0006de2460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8750076 td 0xfffffe004d327000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8750073 td 0xfffffe0006db1000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8750071 td 0xfffffe0006dca460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8750059 td 0xfffffe0006496000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8749540 td 0xfffffe0006275000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8749534 td 0xfffffe004d9e1000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8749012 td 0xfffffe004d3a8000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8746797 td 0xfffffe00064aa460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea thread_suspend_check() at thread_suspend_check+0x16b ast() at ast+0x40e doreti_ast() at doreti_ast+0x1f Tracing command syscall4 pid 62079 tid 8749494 td 0xfffffe004d6ba000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a do_wait() at do_wait+0x3fd __umtx_op_wait() at __umtx_op_wait+0x68 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x80085f9bc, rsp = 0x7fffffffd498, rbp = 0x801007400 --- Tracing command sleep pid 50993 tid 7827649 td 0xfffffe004d972460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x24d kern_nanosleep() at kern_nanosleep+0x118 sys_nanosleep() at sys_nanosleep+0x6e amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x80092f5dc, rsp = 0x7fffffffd688, rbp = 0x7fffffffd700 --- Tracing command syscall4 pid 50992 tid 8476303 td 0xfffffe004d793000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a kern_wait() at kern_wait+0x3eb sys_wait4() at sys_wait4+0x35 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x800ae693c, rsp = 0x7fffffffd518, rbp = 0x7fffffffd740 --- Tracing command md5 pid 69803 tid 6526927 td 0xfffffe0047f538c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x369 md_kthread() at md_kthread+0x16a fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8115118d00, rbp = 0 --- Tracing command sh pid 69772 tid 6526721 td 0xfffffe0006db3000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a kern_wait() at kern_wait+0x3eb sys_wait4() at sys_wait4+0x35 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x800d5793c, rsp = 0x7fffffffd428, rbp = 0x7fffffffd5f8 --- Tracing command awk pid 5006 tid 139348 td 0xfffffe0006e02460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a pipe_read() at pipe_read+0x4a3 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x6c sys_read() at sys_read+0x55 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800b919dc, rsp = 0x7fffffffd858, rbp = 0x800dc7ba0 --- Tracing command sh pid 5005 tid 140875 td 0xfffffe0006e02000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a kern_wait() at kern_wait+0x3eb sys_wait4() at sys_wait4+0x35 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x800d5793c, rsp = 0x7fffffffd6c8, rbp = 0x7fffffffd898 --- Tracing command tail pid 5002 tid 163781 td 0xfffffe004d39e000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x24d kern_kevent() at kern_kevent+0x331 sys_kevent() at sys_kevent+0x90 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (363, FreeBSD ELF64, sys_kevent), rip = 0x80093450c, rsp = 0x7fffffffd9f8, rbp = 0x800c07148 --- Tracing command sshd pid 5001 tid 142912 td 0xfffffe0006cbd460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x6f0 sys_select() at sys_select+0x5d amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80250795c, rsp = 0x7fffffffcc68, rbp = 0x7fffffffccf0 --- Tracing command top pid 4999 tid 143329 td 0xfffffe0006259000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x6f0 sys_select() at sys_select+0x5d amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800fd295c, rsp = 0x7fffffffd9b8, rbp = 0xe --- Tracing command sshd pid 4998 tid 163134 td 0xfffffe0006dcb000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x6f0 sys_select() at sys_select+0x5d amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80250795c, rsp = 0x7fffffffcc68, rbp = 0x7fffffffccf0 --- Tracing command sh pid 4995 tid 142837 td 0xfffffe004d39b000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a kern_wait() at kern_wait+0x3eb sys_wait4() at sys_wait4+0x35 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x800d5793c, rsp = 0x7fffffffd9b8, rbp = 0 --- Tracing command sshd pid 4994 tid 142859 td 0xfffffe00064968c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x6f0 sys_select() at sys_select+0x5d amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80250795c, rsp = 0x7fffffffcc68, rbp = 0x7fffffffccf0 --- Tracing command sshd pid 4960 tid 147712 td 0xfffffe0006cea460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac sys_poll() at sys_poll+0x2b6 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8024a42cc, rsp = 0x7fffffffcd08, rbp = 0x8030171c0 --- Tracing command sshd pid 4958 tid 162103 td 0xfffffe0006cf8000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac sys_poll() at sys_poll+0x2b6 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8024a42cc, rsp = 0x7fffffffcd08, rbp = 0x8030171c0 --- Tracing command sshd pid 4955 tid 149158 td 0xfffffe0006d1b460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac sys_poll() at sys_poll+0x2b6 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8024a42cc, rsp = 0x7fffffffcd08, rbp = 0x8030171c0 --- Tracing command bash pid 2679 tid 106225 td 0xfffffe0006e148c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a kern_wait() at kern_wait+0x3eb sys_wait4() at sys_wait4+0x35 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x800cab93c, rsp = 0x7fffffffd558, rbp = 0x801125a20 --- Tracing command sshd pid 2677 tid 104923 td 0xfffffe0040832000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x6f0 sys_select() at sys_select+0x5d amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80250795c, rsp = 0x7fffffffcc68, rbp = 0x7fffffffccf0 --- Tracing command sshd pid 2633 tid 105718 td 0xfffffe0006d02460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac sys_poll() at sys_poll+0x2b6 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8024a42cc, rsp = 0x7fffffffcd08, rbp = 0x8030171c0 --- Tracing command sh pid 2276 tid 100153 td 0xfffffe00069b98c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a kern_wait() at kern_wait+0x3eb sys_wait4() at sys_wait4+0x35 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x800d5793c, rsp = 0x7fffffffd368, rbp = 0x7fffffffd538 --- Tracing command bash pid 2270 tid 100130 td 0xfffffe00064998c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a kern_wait() at kern_wait+0x3eb sys_wait4() at sys_wait4+0x35 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x800cab93c, rsp = 0x7fffffffd668, rbp = 0x801156b80 --- Tracing command csh pid 2266 tid 100105 td 0xfffffe00062be000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a kern_sigsuspend() at kern_sigsuspend+0x9b sys_sigsuspend() at sys_sigsuspend+0x34 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x800d6da6c, rsp = 0x7fffffffd348, rbp = 0x801413600 --- Tracing command su pid 2265 tid 100106 td 0xfffffe00062598c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a kern_wait() at kern_wait+0x3eb sys_wait4() at sys_wait4+0x35 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x800ef993c, rsp = 0x7fffffffd488, rbp = 0x8da --- Tracing command bash pid 2261 tid 100150 td 0xfffffe00069fe8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a kern_wait() at kern_wait+0x3eb sys_wait4() at sys_wait4+0x35 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x800cab93c, rsp = 0x7fffffffd8c8, rbp = 0x801125860 --- Tracing command sshd pid 2260 tid 100133 td 0xfffffe000646c8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x6f0 sys_select() at sys_select+0x5d amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80250795c, rsp = 0x7fffffffcc68, rbp = 0x7fffffffccf0 --- Tracing command sshd pid 2258 tid 100138 td 0xfffffe00064aa000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac sys_poll() at sys_poll+0x2b6 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8024a42cc, rsp = 0x7fffffffcd08, rbp = 0x8030171c0 --- Tracing command getty pid 2197 tid 100094 td 0xfffffe00061fb000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x88 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x6c sys_read() at sys_read+0x55 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800b649dc, rsp = 0x7fffffffdca8, rbp = 0x3 --- Tracing command getty pid 2196 tid 100079 td 0xfffffe000608a000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x88 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x6c sys_read() at sys_read+0x55 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800b649dc, rsp = 0x7fffffffdca8, rbp = 0x3 --- Tracing command getty pid 2195 tid 100103 td 0xfffffe00062be8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x88 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x6c sys_read() at sys_read+0x55 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800b649dc, rsp = 0x7fffffffdca8, rbp = 0x3 --- Tracing command getty pid 2194 tid 100102 td 0xfffffe00062bf000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x88 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x6c sys_read() at sys_read+0x55 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800b649dc, rsp = 0x7fffffffdca8, rbp = 0x3 --- Tracing command getty pid 2193 tid 100110 td 0xfffffe0006498460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x88 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x6c sys_read() at sys_read+0x55 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800b649dc, rsp = 0x7fffffffdca8, rbp = 0x3 --- Tracing command getty pid 2192 tid 100100 td 0xfffffe00062bf8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x88 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x6c sys_read() at sys_read+0x55 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800b649dc, rsp = 0x7fffffffdca8, rbp = 0x3 --- Tracing command getty pid 2191 tid 100088 td 0xfffffe00061e0000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x88 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x6c sys_read() at sys_read+0x55 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800b649dc, rsp = 0x7fffffffdca8, rbp = 0x3 --- Tracing command getty pid 2190 tid 100083 td 0xfffffe00060888c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x88 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x6c sys_read() at sys_read+0x55 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800b649dc, rsp = 0x7fffffffdca8, rbp = 0x3 --- Tracing command getty pid 2189 tid 100091 td 0xfffffe00061fb8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x88 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x6c sys_read() at sys_read+0x55 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800b649dc, rsp = 0x7fffffffdca8, rbp = 0x3 --- Tracing command inetd pid 2150 tid 100145 td 0xfffffe0006237000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x6f0 sys_select() at sys_select+0x5d amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800f7a95c, rsp = 0x7fffffffcd78, rbp = 0x1 --- Tracing command moused pid 2117 tid 100107 td 0xfffffe0006259460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x6f0 sys_select() at sys_select+0x5d amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800d8a95c, rsp = 0x7fffffffd858, rbp = 0x7fffffffdf31 --- Tracing command cron pid 2093 tid 100139 td 0xfffffe00069858c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x24d kern_nanosleep() at kern_nanosleep+0x118 sys_nanosleep() at sys_nanosleep+0x6e amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x800d515dc, rsp = 0x7fffffffdae8, rbp = 0x3c --- Tracing command sendmail pid 2086 tid 100122 td 0xfffffe000625e000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a kern_sigsuspend() at kern_sigsuspend+0x9b sys_sigsuspend() at sys_sigsuspend+0x34 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x801360a6c, rsp = 0x7fffffffbdb8, rbp = 0x1 --- Tracing command sendmail pid 2080 tid 100117 td 0xfffffe00062718c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x6f0 sys_select() at sys_select+0x5d amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80140395c, rsp = 0x7fffffffb228, rbp = 0x7fffffffb2c0 --- Tracing command sshd pid 2072 tid 100142 td 0xfffffe00063268c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x6f0 sys_select() at sys_select+0x5d amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80250795c, rsp = 0x7fffffffcd88, rbp = 0x2 --- Tracing command ntpd pid 2026 tid 100123 td 0xfffffe00062f2000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x6f0 sys_select() at sys_select+0x5d amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80137b95c, rsp = 0x7fffffffdbd8, rbp = 0x7fffffffdd08 --- Tracing command nfsd pid 1898 tid 100129 td 0xfffffe000623e000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c svc_run_internal() at svc_run_internal+0x81a svc_thread_start() at svc_thread_start+0xb fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0xc, rip = 0x8008b06ac, rsp = 0x7fffffffd698, rbp = 0x5 --- Tracing command nfsd pid 1898 tid 100128 td 0xfffffe000623e460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c svc_run_internal() at svc_run_internal+0x81a svc_thread_start() at svc_thread_start+0xb fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0xc, rip = 0x8008b06ac, rsp = 0x7fffffffd698, rbp = 0x5 --- Tracing command nfsd pid 1898 tid 100127 td 0xfffffe000623e8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c svc_run_internal() at svc_run_internal+0x81a svc_thread_start() at svc_thread_start+0xb fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0xc, rip = 0x8008b06ac, rsp = 0x7fffffffd698, rbp = 0x5 --- Tracing command nfsd pid 1898 tid 100113 td 0xfffffe0006274000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c svc_run_internal() at svc_run_internal+0x81a svc_run() at svc_run+0x8f nfsrvd_nfsd() at nfsrvd_nfsd+0x1c7 nfssvc_nfsd() at nfssvc_nfsd+0x9b sys_nfssvc() at sys_nfssvc+0xab amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (155, FreeBSD ELF64, sys_nfssvc), rip = 0x8008b06ac, rsp = 0x7fffffffd698, rbp = 0x5 --- Tracing command nfsd pid 1892 tid 100095 td 0xfffffe00062748c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x6f0 sys_select() at sys_select+0x5d amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80095295c, rsp = 0x7fffffffd938, rbp = 0x7fffffffdc10 --- Tracing command mountd pid 1883 tid 100114 td 0xfffffe00062738c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x6f0 sys_select() at sys_select+0x5d amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800b6995c, rsp = 0x7fffffffdb88, rbp = 0x801086060 --- Tracing command rpcbind pid 1785 tid 100121 td 0xfffffe000625e460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 sys_poll() at sys_poll+0x2b6 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x800d0f2cc, rsp = 0x7fffffffba58, rbp = 0x8010640c0 --- Tracing command syslogd pid 1704 tid 100085 td 0xfffffe00061e0460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x6f0 sys_select() at sys_select+0x5d amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800b6795c, rsp = 0x7fffffffd128, rbp = 0x80108c0d0 --- Tracing command devd pid 1496 tid 100089 td 0xfffffe00062758c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x6f0 sys_select() at sys_select+0x5d amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x449a1c, rsp = 0x7fffffffd838, rbp = 0x7fffffffd850 --- Tracing command softdepflush pid 18 tid 100076 td 0xfffffe000608b000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x352 softdep_flush() at softdep_flush+0x264 fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8112c21d00, rbp = 0 --- Tracing command syncer pid 17 tid 100075 td 0xfffffe000608b460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _cv_timedwait() at _cv_timedwait+0x18c sched_sync() at sched_sync+0x541 fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8112c1cd00, rbp = 0 --- Tracing command vnlru pid 16 tid 100074 td 0xfffffe000608b8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x352 vnlru_proc() at vnlru_proc+0x586 fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8112c17d00, rbp = 0 --- Tracing command bufdaemon pid 9 tid 100073 td 0xfffffe000608d000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x352 buf_daemon() at buf_daemon+0x192 fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8112c12d00, rbp = 0 --- Tracing command pagezero pid 8 tid 100072 td 0xfffffe0002d7c000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x352 vm_pagezero() at vm_pagezero+0x73 fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8112c0dd00, rbp = 0 --- Tracing command vmdaemon pid 7 tid 100071 td 0xfffffe0002d7c460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x369 vm_daemon() at vm_daemon+0x4d fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8112c08d00, rbp = 0 --- Tracing command pagedaemon pid 6 tid 100070 td 0xfffffe0002d7c8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x352 vm_pageout() at vm_pageout+0x8cb fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8112c03d00, rbp = 0 --- Tracing command xpt_thrd pid 5 tid 100069 td 0xfffffe0002d7e000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x369 xpt_scanner_thread() at xpt_scanner_thread+0xd5 fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8112b9ed00, rbp = 0 --- Tracing command sctp_iterator pid 4 tid 100068 td 0xfffffe0002d7e460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x369 sctp_iterator_thread() at sctp_iterator_thread+0x3f fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8112b99d00, rbp = 0 --- Tracing command fdc0 pid 3 tid 100064 td 0xfffffe0002d808c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x352 fdc_thread() at fdc_thread+0x74a fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8112b81d00, rbp = 0 --- Tracing command fw0_probe pid 2 tid 100062 td 0xfffffe0002d81460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a fw_bus_probe_thread() at fw_bus_probe_thread+0xbc fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8112b69d00, rbp = 0 --- Tracing command usb pid 15 tid 100058 td 0xfffffe0002d46000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x16b fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8112808d00, rbp = 0 --- Tracing command usb pid 15 tid 100057 td 0xfffffe0002d46460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x16b fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8112803d00, rbp = 0 --- Tracing command usb pid 15 tid 100056 td 0xfffffe0002d468c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x16b fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81127fed00, rbp = 0 --- Tracing command usb pid 15 tid 100055 td 0xfffffe0002d47000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x16b fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81127f9d00, rbp = 0 --- Tracing command usb pid 15 tid 100053 td 0xfffffe0002d478c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x16b fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff811266cd00, rbp = 0 --- Tracing command usb pid 15 tid 100052 td 0xfffffe0002d48000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x16b fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8112667d00, rbp = 0 --- Tracing command usb pid 15 tid 100051 td 0xfffffe0002d48460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x16b fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8112662d00, rbp = 0 --- Tracing command usb pid 15 tid 100050 td 0xfffffe0002d488c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x16b fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff811265dd00, rbp = 0 --- Tracing command usb pid 15 tid 100049 td 0xfffffe0002d36000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x16b fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8112614d00, rbp = 0 --- Tracing command usb pid 15 tid 100048 td 0xfffffe0002d36460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x16b fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff811260fd00, rbp = 0 --- Tracing command usb pid 15 tid 100047 td 0xfffffe0002d368c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x16b fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff811260ad00, rbp = 0 --- Tracing command usb pid 15 tid 100046 td 0xfffffe0002d37000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x16b fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8112605d00, rbp = 0 --- Tracing command usb pid 15 tid 100045 td 0xfffffe0002d37460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x16b fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81125bcd00, rbp = 0 --- Tracing command usb pid 15 tid 100044 td 0xfffffe0002d378c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x16b fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81125b7d00, rbp = 0 --- Tracing command usb pid 15 tid 100043 td 0xfffffe0002d39000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x16b fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81125b2d00, rbp = 0 --- Tracing command usb pid 15 tid 100042 td 0xfffffe0002d39460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x16b fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81125add00, rbp = 0 --- Tracing command usb pid 15 tid 100040 td 0xfffffe0002d3a000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x16b fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff811255fd00, rbp = 0 --- Tracing command usb pid 15 tid 100039 td 0xfffffe0002d2f000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x16b fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff811255ad00, rbp = 0 --- Tracing command usb pid 15 tid 100038 td 0xfffffe0002d2f460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x16b fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8112555d00, rbp = 0 --- Tracing command usb pid 15 tid 100037 td 0xfffffe0002d2f8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x16b fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8112550d00, rbp = 0 --- Tracing command usb pid 15 tid 100035 td 0xfffffe0002d30460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x16b fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8112502d00, rbp = 0 --- Tracing command usb pid 15 tid 100034 td 0xfffffe0002d308c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x16b fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81124fdd00, rbp = 0 --- Tracing command usb pid 15 tid 100033 td 0xfffffe0002d32000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x16b fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81124f8d00, rbp = 0 --- Tracing command usb pid 15 tid 100032 td 0xfffffe0002d32460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x16b fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81124f3d00, rbp = 0 --- Tracing command yarrow pid 14 tid 100017 td 0xfffffe000292f8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x352 random_kthread() at random_kthread+0x1ad fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000278d00, rbp = 0 --- Tracing command geom pid 13 tid 100015 td 0xfffffe0002915000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x369 g_io_schedule_down() at g_io_schedule_down+0x228 g_down_procbody() at g_down_procbody+0x72 fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff800026ed00, rbp = 0 --- Tracing command geom pid 13 tid 100014 td 0xfffffe0002915460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x369 g_io_schedule_up() at g_io_schedule_up+0x10a g_up_procbody() at g_up_procbody+0x72 fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000269d00, rbp = 0 --- Tracing command geom pid 13 tid 100013 td 0xfffffe00029158c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x369 g_run_events() at g_run_events+0x4a8 fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000264d00, rbp = 0 --- Tracing command intr pid 12 tid 100066 td 0xfffffe0002d80000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100065 td 0xfffffe0002d80460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea ithread_loop() at ithread_loop+0x25d fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8112b86d00, rbp = 0 --- Tracing command intr pid 12 tid 100063 td 0xfffffe0002d81000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea ithread_loop() at ithread_loop+0x25d fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8112b72d00, rbp = 0 --- Tracing command intr pid 12 tid 100060 td 0xfffffe0002d3a460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea ithread_loop() at ithread_loop+0x25d fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8112835d00, rbp = 0 --- Tracing command intr pid 12 tid 100059 td 0xfffffe0002d3a8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea ithread_loop() at ithread_loop+0x25d fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8112830d00, rbp = 0 --- Tracing command intr pid 12 tid 100054 td 0xfffffe0002d47460 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100041 td 0xfffffe0002d398c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea ithread_loop() at ithread_loop+0x25d fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81125a8d00, rbp = 0 --- Tracing command intr pid 12 tid 100036 td 0xfffffe0002d30000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea ithread_loop() at ithread_loop+0x25d fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff811254bd00, rbp = 0 --- Tracing command intr pid 12 tid 100031 td 0xfffffe0002d328c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea ithread_loop() at ithread_loop+0x25d fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff81124eed00, rbp = 0 --- Tracing command intr pid 12 tid 100030 td 0xfffffe0002931460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea ithread_loop() at ithread_loop+0x25d fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80003bdd00, rbp = 0 --- Tracing command intr pid 12 tid 100029 td 0xfffffe00029318c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea ithread_loop() at ithread_loop+0x25d fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80003b8d00, rbp = 0 --- Tracing command intr pid 12 tid 100028 td 0xfffffe0002ae7000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea ithread_loop() at ithread_loop+0x25d fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80002afd00, rbp = 0 --- Tracing command intr pid 12 tid 100026 td 0xfffffe0002ae78c0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100020 td 0xfffffe000292e8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea ithread_loop() at ithread_loop+0x25d fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000287d00, rbp = 0 --- Tracing command intr pid 12 tid 100019 td 0xfffffe000292f000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea ithread_loop() at ithread_loop+0x25d fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000282d00, rbp = 0 --- Tracing command intr pid 12 tid 100018 td 0xfffffe000292f460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea ithread_loop() at ithread_loop+0x25d fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff800027dd00, rbp = 0 --- Tracing command intr pid 12 tid 100012 td 0xfffffe0002917000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100011 td 0xfffffe0002917460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea ithread_loop() at ithread_loop+0x25d fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff800025ad00, rbp = 0 --- Tracing command intr pid 12 tid 100010 td 0xfffffe00029178c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea ithread_loop() at ithread_loop+0x25d fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000255d00, rbp = 0 --- Tracing command intr pid 12 tid 100009 td 0xfffffe00028fe460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea ithread_loop() at ithread_loop+0x25d fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000250d00, rbp = 0 --- Tracing command intr pid 12 tid 100008 td 0xfffffe00028fe8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea ithread_loop() at ithread_loop+0x25d fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff800024bd00, rbp = 0 --- Tracing command intr pid 12 tid 100007 td 0xfffffe0002908000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea ithread_loop() at ithread_loop+0x25d fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000246d00, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xfffffe0002908460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sched_idletd() at sched_idletd+0x11b fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000241d00, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffffe00029088c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sched_idletd() at sched_idletd+0x2b0 fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff800023cd00, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xfffffe00028fd000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea critical_exit() at critical_exit+0xb0 lapic_handle_timer() at lapic_handle_timer+0x96 Xtimerint() at Xtimerint+0x88 --- interrupt, rip = 0xffffffff80b19f5a, rsp = 0xffffff8000237b50, rbp = 0xffffff8000237b70 --- cpu_idle_mwait() at cpu_idle_mwait+0x5a sched_idletd() at sched_idletd+0x27e fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000237d00, rbp = 0 --- Tracing command idle pid 11 tid 100003 td 0xfffffe00028fd460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sched_idletd() at sched_idletd+0x11b fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000232d00, rbp = 0 --- Tracing command init pid 1 tid 100002 td 0xfffffe00028fd8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2a2 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x31a kern_wait() at kern_wait+0x3eb sys_wait4() at sys_wait4+0x35 amd64_syscall() at amd64_syscall+0x27b Xfast_syscall() at Xfast_syscall+0xf7 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x412f9c, rsp = 0x7fffffffd7a8, rbp = 0x7d --- Tracing command audit pid 10 tid 100001 td 0xfffffe00028fe000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a audit_worker() at audit_worker+0x77 fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000228d00, rbp = 0 --- Tracing command kernel pid 0 tid 100077 td 0xfffffe000608a8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x369 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8112c26d00, rbp = 0 --- Tracing command kernel pid 0 tid 100067 td 0xfffffe0002d7e8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x352 deadlkres() at deadlkres+0x2ac fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8112b94d00, rbp = 0 --- Tracing command kernel pid 0 tid 100061 td 0xfffffe0002d818c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d msleep_spin() at msleep_spin+0x209 taskqueue_thread_loop() at taskqueue_thread_loop+0x5f fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8112aa4d00, rbp = 0 --- Tracing command kernel pid 0 tid 100027 td 0xfffffe0002ae7460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x369 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80002aad00, rbp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xfffffe0002aea000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d msleep_spin() at msleep_spin+0x209 taskqueue_thread_loop() at taskqueue_thread_loop+0x5f fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80002a0d00, rbp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xfffffe0002aea460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d msleep_spin() at msleep_spin+0x209 taskqueue_thread_loop() at taskqueue_thread_loop+0x5f fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff800029bd00, rbp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xfffffe0002aea8c0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d msleep_spin() at msleep_spin+0x209 taskqueue_thread_loop() at taskqueue_thread_loop+0x5f fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000296d00, rbp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xfffffe000292e000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x369 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000291d00, rbp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xfffffe000292e460 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x369 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff800028cd00, rbp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xfffffe0002931000 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x369 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x135 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000273d00, rbp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xffffffff811496f0 sched_switch() at sched_switch+0x157 mi_switch() at mi_switch+0x1ea sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x352 scheduler() at scheduler+0x2a1 mi_startup() at mi_startup+0x77 btext() at btext+0x2c db:0:allt> call doadump Dumping 645 out of 3947 MB:..3%..13%..23%..33%..43%..53%..62%..72%..82%..92% Dump complete = 0 db:0:doadump> reset cpu_reset: Restarting BSP timeout stopping cpus cpu_reset_proxy: Stopped CPU 2 (kgdb) p *(struct thread *)0xfffffe004d7308c0 $1 = {td_lock = 0xffffffff81165c00, td_proc = 0xfffffe0006091000, td_plist = {tqe_next = 0xfffffe0006dc7460, tqe_prev = 0xfffffe00064e78d0}, td_runq = {tqe_next = 0x0, tqe_prev = 0xffffffff81166310}, td_slpq = {tqe_next = 0x0, tqe_prev = 0xfffffe0006ceb8f0}, td_lockq = {tqe_next = 0xfffffe000626d000, tqe_prev = 0xffffff81153edb70}, td_hash = { le_next = 0x0, le_prev = 0xffffff80006b1400}, td_cpuset = 0xfffffe00028e8dc8, td_sel = 0x0, td_sleepqueue = 0xfffffe004d399280, td_turnstile = 0xfffffe0006413b40, td_umtxq = 0xfffffe004da4d180, td_tid = 0x870680, td_sigqueue = {sq_signals = {__bits = {0x0, 0x0, 0x0, 0x0}}, sq_kill = {__bits = {0x0, 0x0, 0x0, 0x0}}, sq_list = {tqh_first = 0x0, tqh_last = 0xfffffe004d730970}, sq_proc = 0xfffffe0006091000, sq_flags = 0x1}, td_lend_user_pri = 0xff, td_flags = 0x9884, td_inhibitors = 0x0, td_pflags = 0x0, td_dupfd = 0x0, td_sqqueue = 0x0, td_wchan = 0x0, td_wmesg = 0x0, td_lastcpu = 0x1, td_oncpu = 0xff, td_owepreempt = 0x0, td_tsqueue = 0x0, td_locks = 0x0, td_rw_rlocks = 0x0, td_lk_slocks = 0x0, td_blocked = 0x0, td_lockname = 0x0, td_contested = {lh_first = 0x0}, td_sleeplocks = 0xffffffff812dbb80, td_intr_nesting_level = 0x0, td_pinned = 0x0, td_ucred = 0xfffffe004d6c1400, td_estcpu = 0x0, td_slptick = 0x0, td_blktick = 0x0, td_swvoltick = 0x34a724, td_ru = {ru_utime = {tv_sec = 0x0, tv_usec = 0x0}, ru_stime = {tv_sec = 0x0, tv_usec = 0x0}, ru_maxrss = 0x0, ru_ixrss = 0x0, ru_idrss = 0x0, ru_isrss = 0x0, ru_minflt = 0x1, ru_majflt = 0x0, ru_nswap = 0x0, ru_inblock = 0x0, ru_oublock = 0x0, ru_msgsnd = 0x0, ru_msgrcv = 0x0, ru_nsignals = 0x0, ru_nvcsw = 0x1, ru_nivcsw = 0x0}, td_rux = {rux_runtime = 0x0, rux_uticks = 0x0, rux_sticks = 0x0, rux_iticks = 0x0, rux_uu = 0x0, rux_su = 0x0, rux_tu = 0x0}, td_incruntime = 0x95e2, td_runtime = 0x95e2, td_pticks = 0x0, td_sticks = 0x0, td_iticks = 0x0, td_uticks = 0x0, td_intrval = 0x4, td_oldsigmask = {__bits = {0x0, 0x0, 0x0, 0x0}}, td_sigmask = {__bits = {0x0, 0x0, 0x0, 0x0}}, td_generation = 0x1, td_sigstk = {ss_sp = 0x0, ss_size = 0x0, ss_flags = 0x0}, td_xsig = 0x0, td_profil_addr = 0x0, td_profil_ticks = 0x0, td_name = "syscall4", '\0' , td_fpop = 0x0, td_dbgflags = 0x0, td_dbgksi = {ksi_link = {tqe_next = 0x0, tqe_prev = 0x0}, ksi_info = {si_signo = 0x0, si_errno = 0x0, si_code = 0x0, si_pid = 0x0, si_uid = 0x0, si_status = 0x0, si_addr = 0x0, si_value = {sival_int = 0x0, sival_ptr = 0x0, sigval_int = 0x0, sigval_ptr = 0x0}, _reason = {_fault = { _trapno = 0x0}, _timer = {_timerid = 0x0, _overrun = 0x0}, _mesgq = {_mqd = 0x0}, _poll = {_band = 0x0}, __spare__ = {__spare1__ = 0x0, __spare2__ = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}}}, ksi_flags = 0x0, ksi_sigq = 0x0}, td_ng_outbound = 0x0, td_osd = {osd_nslots = 0x0, osd_slots = 0x0, osd_next = {le_next = 0x0, le_prev = 0x0}}, td_map_def_user = 0x0, td_dbg_forked = 0x0, td_rqindex = 0x2c, td_base_pri = 0x98, td_priority = 0x98, td_pri_class = 0x3, td_user_pri = 0xb0, td_base_user_pri = 0xb0, td_pcb = 0xffffff81153a7d10, td_state = TDS_RUNNING, td_retval = {0x0, 0x0}, td_slpcallout = {c_links = {sle = {sle_next = 0xfffffe00064e7c38}, tqe = {tqe_next = 0xfffffe00064e7c38, tqe_prev = 0xffffff80006e5260}}, c_time = 0x34a726, c_arg = 0xfffffe004d7308c0, c_func = 0xffffffff8087c6c0 , c_lock = 0x0, c_flags = 0x12, c_cpu = 0x1}, td_frame = 0xffffff81153a7c50, td_kstack_obj = 0xfffffe004d625b60, td_kstack = 0xffffff81153a4000, td_kstack_pages = 0x4, td_critnest = 0x1, td_md = {md_spinlock_count = 0x1, md_saved_flags = 0x246}, td_sched = 0xfffffe004d730ce8, td_ar = 0x0, td_lprof = {{lh_first = 0x0}, {lh_first = 0x0}}, td_dtrace = 0x0, td_errno = 0x4, td_vnet = 0x0, td_vnet_lpush = 0x0, td_intr_frame = 0x0} (kgdb) git diff master diff --git a/sys/amd64/amd64/pmap.c b/sys/amd64/amd64/pmap.c index f7c0d2d..9cbe61f 100644 --- a/sys/amd64/amd64/pmap.c +++ b/sys/amd64/amd64/pmap.c @@ -1396,13 +1396,15 @@ pmap_qenter(vm_offset_t sva, vm_page_t *ma, int count) PAGE_SIZE); } +void vm_check_stack_unmap(vm_offset_t sva, int count); +void pmap_qremove_stack(vm_offset_t sva, int count); /* * This routine tears out page mappings from the * kernel -- it is meant only for temporary mappings. * Note: SMP coherent. Uses a ranged shootdown IPI. */ void -pmap_qremove(vm_offset_t sva, int count) +pmap_qremove_stack(vm_offset_t sva, int count) { vm_offset_t va; @@ -1414,6 +1416,14 @@ pmap_qremove(vm_offset_t sva, int count) pmap_invalidate_range(kernel_pmap, sva, va); } +void +pmap_qremove(vm_offset_t sva, int count) +{ + + vm_check_stack_unmap(sva, count); + pmap_qremove_stack(sva, count); +} + /*************************************************** * Page table page management routines..... ***************************************************/ diff --git a/sys/ddb/db_command.c b/sys/ddb/db_command.c index f2e2c42..e0f14b9 100644 --- a/sys/ddb/db_command.c +++ b/sys/ddb/db_command.c @@ -140,6 +140,7 @@ static struct command db_cmds[] = { { "unscript", db_unscript_cmd, CS_OWN, 0 }, { "capture", db_capture_cmd, CS_OWN, 0 }, { "textdump", db_textdump_cmd, CS_OWN, 0 }, + { "findstack", db_findstack_cmd, 0, 0 }, }; struct command_table db_cmd_table = LIST_HEAD_INITIALIZER(db_cmd_table); diff --git a/sys/ddb/db_ps.c b/sys/ddb/db_ps.c index 0c45183..fcd1bd0 100644 --- a/sys/ddb/db_ps.c +++ b/sys/ddb/db_ps.c @@ -37,6 +37,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -429,3 +430,39 @@ DB_SHOW_COMMAND(proc, db_show_proc) break; } } + +void +db_findstack_cmd(db_expr_t addr, boolean_t have_addr, + db_expr_t dummy3 __unused, char *dummy4 __unused) +{ + struct proc *p; + struct thread *td; + struct kstack_cache_entry *ks_ce; + vm_offset_t saddr; + + if (have_addr) + saddr = addr; + else { + db_printf("Usage: findstack
\n"); + return; + } + + for (p = LIST_FIRST(&allproc); p != NULL; p = LIST_NEXT(p, p_list)) { + FOREACH_THREAD_IN_PROC(p, td) { + if (td->td_kstack <= saddr && saddr < td->td_kstack + + PAGE_SIZE * td->td_kstack_pages) { + db_printf("Thread %p\n", td); + return; + } + } + } + + for (ks_ce = kstack_cache; ks_ce != NULL; + ks_ce = ks_ce->next_ks_entry) { + if ((vm_offset_t)ks_ce <= saddr && saddr < (vm_offset_t)ks_ce + + PAGE_SIZE * KSTACK_PAGES) { + db_printf("Cached stack %p\n", ks_ce); + return; + } + } +} diff --git a/sys/ddb/db_thread.c b/sys/ddb/db_thread.c index 8ddb240..18d5ffe 100644 --- a/sys/ddb/db_thread.c +++ b/sys/ddb/db_thread.c @@ -94,7 +94,7 @@ db_show_threads(db_expr_t addr, boolean_t hasaddr, db_expr_t cnt, char *mod) thr = kdb_thr_first(); while (!db_pager_quit && thr != NULL) { - db_printf(" %6ld (%p) ", (long)thr->td_tid, thr); + db_printf(" %6ld (%p) stack %p ", (long)thr->td_tid, (void *)thr->td_kstack, thr); prev_jb = kdb_jmpbuf(jb); if (setjmp(jb) == 0) { if (db_trace_thread(thr, 1) != 0) @@ -109,7 +109,7 @@ db_show_threads(db_expr_t addr, boolean_t hasaddr, db_expr_t cnt, char *mod) * Lookup a thread based on a db expression address. We assume that the * address was parsed in hexadecimal. We reparse the address in decimal * first and try to treat it as a thread ID to find an associated thread. - * If that fails and check_pid is true, we terat the decimal value as a + * If that fails and check_pid is true, we treat the decimal value as a * PID. If that matches a process, we return the first thread in that * process. Otherwise, we treat the addr as a pointer to a thread. */ diff --git a/sys/ddb/ddb.h b/sys/ddb/ddb.h index 9b240b6..93bf713 100644 --- a/sys/ddb/ddb.h +++ b/sys/ddb/ddb.h @@ -226,6 +226,7 @@ db_cmdfcn_t db_delete_cmd; db_cmdfcn_t db_deletehwatch_cmd; db_cmdfcn_t db_deletewatch_cmd; db_cmdfcn_t db_examine_cmd; +db_cmdfcn_t db_findstack_cmd; db_cmdfcn_t db_hwatchpoint_cmd; db_cmdfcn_t db_listbreak_cmd; db_cmdfcn_t db_scripts_cmd; diff --git a/sys/sys/_kstack_cache.h b/sys/sys/_kstack_cache.h new file mode 100644 index 0000000..5b33734 --- /dev/null +++ b/sys/sys/_kstack_cache.h @@ -0,0 +1,43 @@ +/*- + * Copyright (c) 2009 Konstantin Belousov + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 4. Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#ifndef _SYS__KSTACK_CACHE_H +#define _SYS__KSTACK_CACHE_H + +struct kstack_cache_entry { + struct vm_object *ksobj; + struct kstack_cache_entry *next_ks_entry; +}; + +extern struct kstack_cache_entry *kstack_cache; + +#endif + + diff --git a/sys/vm/vm_glue.c b/sys/vm/vm_glue.c index c552cb7..9f516b4 100644 --- a/sys/vm/vm_glue.c +++ b/sys/vm/vm_glue.c @@ -77,7 +77,7 @@ __FBSDID("$FreeBSD$"); #include #include #include - +#include #include #include #include @@ -331,12 +331,7 @@ vm_sync_icache(vm_map_t map, vm_offset_t va, vm_offset_t sz) pmap_sync_icache(map->pmap, va, sz); } -struct kstack_cache_entry { - vm_object_t ksobj; - struct kstack_cache_entry *next_ks_entry; -}; - -static struct kstack_cache_entry *kstack_cache; +struct kstack_cache_entry *kstack_cache; static int kstack_cache_size = 128; static int kstacks; static struct mtx kstack_cache_mtx; @@ -349,6 +344,32 @@ SYSCTL_INT(_vm, OID_AUTO, kstacks, CTLFLAG_RD, &kstacks, 0, #define KSTACK_MAX_PAGES 32 #endif +static struct stack_va { + vm_offset_t va; + int count; +} stack_vas[2048]; + +void vm_check_stack_unmap(vm_offset_t sva, int count); +void pmap_qremove_stack(vm_offset_t sva, int count); + +void +vm_check_stack_unmap(vm_offset_t sva, int count) +{ + struct stack_va *v; + int i; + + for (i = 0; i < sizeof(stack_vas) / sizeof(struct stack_va); i++) { + v = &stack_vas[i]; + if (v->va == 0) + continue; + if ((sva >= v->va && sva < v->va + v->count * PAGE_SIZE) || + (sva <= v->va && sva + count * PAGE_SIZE > v->va)) { + panic("pmap_qremove stack sva %jx count %d", + (uintmax_t)sva, count); + } + } +} + /* * Create the kernel stack (including pcb for i386) for a new thread. * This routine directly affects the fork perf for a process and @@ -411,7 +432,7 @@ vm_thread_new(struct thread *td, int pages) atomic_add_int(&kstacks, 1); if (KSTACK_GUARD_PAGES != 0) { - pmap_qremove(ks, KSTACK_GUARD_PAGES); + pmap_qremove_stack(ks, KSTACK_GUARD_PAGES); ks += KSTACK_GUARD_PAGES * PAGE_SIZE; } td->td_kstack_obj = ksobj; @@ -436,6 +457,21 @@ vm_thread_new(struct thread *td, int pages) m->valid = VM_PAGE_BITS_ALL; } VM_OBJECT_UNLOCK(ksobj); + mtx_lock(&kstack_cache_mtx); + for (i = 0; i < sizeof(stack_vas) / sizeof(struct stack_va); i++) { + struct stack_va *v; + + v = &stack_vas[i]; + if (v->va == 0) { + v->va = ks; + v->count = pages; + break; + } + } + mtx_unlock(&kstack_cache_mtx); + if (i == sizeof(stack_vas) / sizeof(struct stack_va)) + panic("stack_vas too small"); + pmap_qenter(ks, ma, pages); return (1); } @@ -447,7 +483,21 @@ vm_thread_stack_dispose(vm_object_t ksobj, vm_offset_t ks, int pages) int i; atomic_add_int(&kstacks, -1); - pmap_qremove(ks, pages); + pmap_qremove_stack(ks, pages); + mtx_lock(&kstack_cache_mtx); + for (i = 0; i < sizeof(stack_vas) / sizeof(struct stack_va); i++) { + struct stack_va *v; + + v = &stack_vas[i]; + if (v->va == ks) { + v->va = 0; + break; + } + } + mtx_unlock(&kstack_cache_mtx); + if (i == sizeof(stack_vas) / sizeof(struct stack_va)) + panic("stack prot not found"); + VM_OBJECT_LOCK(ksobj); for (i = 0; i < pages; i++) { m = vm_page_lookup(ksobj, i); @@ -536,7 +586,7 @@ vm_thread_swapout(struct thread *td) cpu_thread_swapout(td); pages = td->td_kstack_pages; ksobj = td->td_kstack_obj; - pmap_qremove(td->td_kstack, pages); + pmap_qremove_stack(td->td_kstack, pages); VM_OBJECT_LOCK(ksobj); for (i = 0; i < pages; i++) { m = vm_page_lookup(ksobj, i); -- Test scenario: syscall4.sh