GENERIC HEAD from 2014-03-14 06:38:22 UTC, r263154, vmcore.104 GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2014 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.0-CURRENT #1 r263154: Fri Mar 14 20:46:06 CET 2014 pho@t1.osted.lan:/usr/src/sys/amd64/compile/PHO amd64 FreeBSD clang version 3.4 (tags/RELEASE_34/final 197956) 20140216 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. CPU: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz (1995.23-MHz K8-class CPU) Origin="GenuineIntel" Id=0x206d7 Family=0x6 Model=0x2d Stepping=7 Features=0xbfebfbff Features2=0x1fbee3ff AMD Features=0x2c100800 AMD Features2=0x1 TSC: P-state invariant, performance statistics real memory = 68719476736 (65536 MB) avail memory = 3006582784 (2867 MB) : Trying to mount root from ufs:/dev/da0p2 [rw]... Setting hostuuid: 3141f787-07e3-e111-b968-001e6756c168. Setting hostid: 0x564ac5b6. Starting ddb. Entropy harvesting: interrupts ethernet point_to_point swi. warning: total configured swap (17039360 pages) exceeds maximum recommended amount (5922592 pages). warning: increase kern.maxswzone or reduce amount of swap. Starting file system checks: /dev/da0p2: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/da0p2: clean, 68190258 free (291282 frags, 8487372 blocks, 0.3% fragmentation) /dev/da0p3: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/da0p3: clean, 20774612 free (1316 frags, 2596662 blocks, 0.0% fragmentation) Mounting local file systems:. lock order reversal: 1st 0xfffff800082cf548 ufs (ufs) @ kern/vfs_subr.c:2101 2nd 0xfffffe00b5033708 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:262 3rd 0xfffff80008729c98 ufs (ufs) @ kern/vfs_subr.c:2101 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00d8922c70 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe00d8922d20 witness_checkorder() at witness_checkorder+0xdc2/frame 0xfffffe00d8922db0 __lockmgr_args() at __lockmgr_args+0x9d2/frame 0xfffffe00d8922ee0 ffs_lock() at ffs_lock+0x92/frame 0xfffffe00d8922f30 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xfc/frame 0xfffffe00d8922f60 _vn_lock() at _vn_lock+0xd2/frame 0xfffffe00d8922fd0 vget() at vget+0x67/frame 0xfffffe00d8923010 vfs_hash_get() at vfs_hash_get+0xe1/frame 0xfffffe00d8923060 ffs_vgetf() at ffs_vgetf+0x40/frame 0xfffffe00d89230f0 softdep_sync_buf() at softdep_sync_buf+0xafc/frame 0xfffffe00d89231d0 ffs_syncvnode() at ffs_syncvnode+0x286/frame 0xfffffe00d8923250 ffs_truncate() at ffs_truncate+0x705/frame 0xfffffe00d8923430 ufs_direnter() at ufs_direnter+0x81a/frame 0xfffffe00d89234f0 ufs_makeinode() at ufs_makeinode+0x560/frame 0xfffffe00d89236a0 VOP_CREATE_APV() at VOP_CREATE_APV+0x10f/frame 0xfffffe00d89236d0 vn_open_cred() at vn_open_cred+0x2db/frame 0xfffffe00d8923820 kern_openat() at kern_openat+0x26f/frame 0xfffffe00d89239a0 amd64_syscall() at amd64_syscall+0x278/frame 0xfffffe00d8923ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe00d8923ab0 --- syscall (5, FreeBSD ELF64, sys_open), rip = 0x800dde74a, rsp = 0x7fffffffc608, rbp = 0x7fffffffc6f0 --- Writing entropy file:. Setting hostname: t1.osted.lan. Starting Network: lo0 igb0 igb1. lo0: flags=8049 metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 vhid 165 nd6 options=21 igb0: flags=8843 metric 0 mtu 1500 options=403bb ether 00:1e:67:56:c1:68 inet 192.168.1.108 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::21e:67ff:fe56:c168%igb0 prefixlen 64 scopeid 0x1 vhid 4 nd6 options=29 media: Ethernet autoselect status: no carrier igb1: flags=8c02 metric 0 mtu 1500 options=403bb ether 00:1e:67:56:c1:69 nd6 options=29 media: Ethernet autoselect status: no carrier Starting devd. Starting Network: igb1. igb1: flags=8c02 metric 0 mtu 1500 options=403bb ether 00:1e:67:56:c1:69 nd6 options=29 media: Ethernet autoselect status: no carrier igb0: link state changed to UP Configuring keyboard: keymap. ums0: on usbus2 ums0: 3 buttons and [Z] coordinates ID=0 Starting ums0 moused. Expensive timeout(9) function: 0xffffffff80712960(0xfffff80006179000) 0.010943474 s Starting pflogd: add net default: gateway 192.168.1.1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/nss /usr/local/lib/qt4 32-bit compatibility ldconfig path: /usr/lib32 Creating and/or trimming log files. Starting syslogd. Starting rpcbind. NFS access cache time=60 No core dumps found. Starting casperd. lock order reversal: 1st 0xfffffe00b506e308 bufwait (bufwait) @ kern/vfs_bio.c:3081 2nd 0xfffff80002621a00 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:284 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00d89692d0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe00d8969380 witness_checkorder() at witness_checkorder+0xdc2/frame 0xfffffe00d8969410 _sx_xlock() at _sx_xlock+0x75/frame 0xfffffe00d8969450 ufsdirhash_add() at ufsdirhash_add+0x4a/frame 0xfffffe00d8969490 ufs_direnter() at ufs_direnter+0x6a0/frame 0xfffffe00d8969550 ufs_mkdir() at ufs_mkdir+0x89c/frame 0xfffffe00d8969740 VOP_MKDIR_APV() at VOP_MKDIR_APV+0x115/frame 0xfffffe00d8969770 kern_mkdirat() at kern_mkdirat+0x1fd/frame 0xfffffe00d89699a0 amd64_syscall() at amd64_syscall+0x278/frame 0xfffffe00d8969ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe00d8969ab0 --- syscall (136, FreeBSD ELF64, sys_mkdir), rip = 0x80093c20a, rsp = 0x7fffffffd788, rbp = 0x7fffffffdc70 --- Clearing /tmp (X related). Starting nfsuserd. Starting mountd. Starting nfsd. Recovering vi editor sessions:. Updating motd:. Mounting late file systems:. Starting ntpd. Starting powerd. Configuring syscons: keymap blanktime. Performing sanity check on sshd configuration. Starting sshd. Starting sendmail_submit. Starting sendmail_msp_queue. Starting cron. Local package initialization: backup. Starting default mousedmoused: unable to open /dev/psm0: No such file or directory . Starting inetd. Fri Mar 14 20:50:18 CET 2014 FreeBSD/amd64 (t1.osted.lan) (console) login: Mar 14 20:51:10 t1 su: pho to root on /dev/pts/0 20140314 20:51:20 all (1/1): stealer.sh Stopping inetd. Waiting for PIDS: 1162. Shutting down local packages:. Stopping cron. Waiting for PIDS: 1101, 1101. Stopping sshd. Waiting for PIDS: 1091. Stopping powerd. Waiting for PIDS: 1047. Stopping ntpd. Waiting for PIDS: 1044. Stopping nfsd. Waiting for PIDS: 1004 1007, 1004. Stopping mountd. Waiting for PIDS: 998. Stopping nfsuserd. Waiting for PIDS: 987 988 989 990 991. Stopping casperd. Waiting for PIDS: 966. Stopping rpcbind. Waiting for PIDS: 940. Stopping devd. Waiting for PIDS: 805. Writing entropy file:. . Terminated Mar 14 20:55:30 t1 syslogd: exiting on signal 15 Mar 14 20:55:50 init: some processes would not die; ps axl advised Enter full pathname of shell or RETURN for /bin/sh: # ps auxwwl USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND UID PPID CPU PRI NI MWCHAN root 11 400.0 0.0 0 64 - RL 8:49PM 8:56.97 [idle] 0 0 0 155 0 - root 0 0.0 0.0 0 304 - DLs 8:49PM 0:00.06 [kernel] 0 0 0 -8 0 - root 1 0.0 0.0 9332 964 - SLs 8:49PM 0:00.07 /sbin/init -- 0 0 0 20 0 wait root 2 0.0 0.0 0 32 - DL 8:49PM 0:00.48 [cam] 0 0 0 -16 0 ccb_scan root 3 0.0 0.0 0 16 - DL 8:49PM 0:00.00 [sctp_iterator] 0 0 0 -16 0 waiting_ root 4 0.0 0.0 0 16 - DL 8:49PM 0:00.00 [enc_daemon0] 0 0 0 -16 0 idle root 5 0.0 0.0 0 16 - DL 8:49PM 0:05.83 [pagedaemon] 0 0 0 -16 0 psleep root 6 0.0 0.0 0 16 - DL 8:49PM 0:00.14 [vmdaemon] 0 0 0 -16 0 psleep root 7 0.0 0.0 0 16 - DL 8:49PM 0:00.00 [pagezero] 0 0 0 155 0 pgzero root 8 0.0 0.0 0 16 - DL 8:49PM 0:00.00 [bufdaemon] 0 0 0 -16 0 psleep root 9 0.0 0.0 0 16 - DL 8:49PM 0:00.05 [syncer] 0 0 0 16 0 syncer root 10 0.0 0.0 0 16 - DL 8:49PM 0:00.00 [audit] 0 0 0 -16 0 audit_wo root 12 0.0 0.0 0 448 - WL 8:49PM 0:01.01 [intr] 0 0 0 -76 0 - root 13 0.0 0.0 0 48 - DL 8:49PM 0:00.34 [geom] 0 0 0 -8 0 - root 14 0.0 0.0 0 16 - DL 8:49PM 0:00.07 [rand_harvestq] 0 0 0 -16 0 - root 15 0.0 0.0 0 192 - DL 8:49PM 0:00.18 [usb] 0 0 0 -68 0 - root 16 0.0 0.0 0 16 - DL 8:49PM 0:00.00 [vnlru] 0 0 0 -16 0 vlruwt root 17 0.0 0.0 0 16 - DL 8:49PM 0:00.03 [softdepflush] 0 0 0 20 0 sdflush root 1342 0.0 0.0 0 16 - DL 8:51PM 0:01.45 [md5] 0 0 0 -8 0 mdwait root 2051 0.0 0.1 16932 2944 u1 Ss 8:55PM 0:00.01 -sh (sh) 0 1 0 20 0 wait root 2052 0.0 0.1 16532 1884 u1 R+ 8:55PM 0:00.00 ps auxwwl 0 2051 0 20 0 - stress 1419 0.0 0.0 0 16 0- DE 8:51PM 0:03.39 /tmp/stealer 555 1002 1 0 20 0 objtrm # kill -9 1419 # kill -9 1419 # kill -9 1419 # umount -t ufa -a # # KDB: enter: Break to debugger [ thread pid 11 tid 100004 ] Stopped at kdb_alt_break_internal+0x17f: movq $0,kdb_why db> run pho db:0:pho> bt Tracing pid 11 tid 100004 td 0xfffff800027ac920 kdb_alt_break_internal() at kdb_alt_break_internal+0x17f/frame 0xfffffe00d338b730 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00d338b740 uart_intr_rxready() at uart_intr_rxready+0x99/frame 0xfffffe00d338b770 uart_intr() at uart_intr+0x111/frame 0xfffffe00d338b7b0 intr_event_handle() at intr_event_handle+0x8c/frame 0xfffffe00d338b800 intr_execute_handlers() at intr_execute_handlers+0x48/frame 0xfffffe00d338b830 lapic_handle_intr() at lapic_handle_intr+0x73/frame 0xfffffe00d338b860 Xapic_isr1() at Xapic_isr1+0xa4/frame 0xfffffe00d338b860 --- interrupt, rip = 0xffffffff80cd8916, rsp = 0xfffffe00d338b920, rbp = 0xfffffe00d338b930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00d338b930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe00d338b980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe00d338b9a0 cpu_idle() at cpu_idle+0x90/frame 0xfffffe00d338b9c0 sched_idletd() at sched_idletd+0x426/frame 0xfffffe00d338ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d338bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d338bab0 --- trap 0, rip = 0, rsp = 0xfffffe00d338bb70, rbp = 0 --- db:0:bt> show allpcpu Current CPU: 1 cpuid = 0 dynamic pcpu = 0x716a80 curthread = 0xfffff800027ad000: pid 11 "idle: cpu0" curpcb = 0xfffffe00d3386b80 fpcurthread = none idlethread = 0xfffff800027ad000: tid 100003 "idle: cpu0" curpmap = 0xffffffff818a3698 tssp = 0xffffffff818bab90 commontssp = 0xffffffff818bab90 rsp0 = 0xfffffe00d3386b80 gs32p = 0xffffffff818bc5e8 ldt = 0xffffffff818bc628 tss = 0xffffffff818bc618 spin locks held: cpuid = 1 dynamic pcpu = 0xfffffe0133b3ba80 curthread = 0xfffff800027ac920: pid 11 "idle: cpu1" curpcb = 0xfffffe00d338bb80 fpcurthread = none idlethread = 0xfffff800027ac920: tid 100004 "idle: cpu1" curpmap = 0xffffffff818a3698 tssp = 0xffffffff818babf8 commontssp = 0xffffffff818babf8 rsp0 = 0xfffffe00d338bb80 gs32p = 0xffffffff818bc650 ldt = 0xffffffff818bc690 tss = 0xffffffff818bc680 spin locks held: cpuid = 2 dynamic pcpu = 0xfffffe0133b43a80 curthread = 0xfffff800027ac490: pid 11 "idle: cpu2" curpcb = 0xfffffe00d3390b80 fpcurthread = none idlethread = 0xfffff800027ac490: tid 100005 "idle: cpu2" curpmap = 0xffffffff818a3698 tssp = 0xffffffff818bac60 commontssp = 0xffffffff818bac60 rsp0 = 0xfffffe00d3390b80 gs32p = 0xffffffff818bc6b8 ldt = 0xffffffff818bc6f8 tss = 0xffffffff818bc6e8 spin locks held: cpuid = 3 dynamic pcpu = 0xfffffe0133b4ba80 curthread = 0xfffff800027ac000: pid 11 "idle: cpu3" curpcb = 0xfffffe00d3395b80 fpcurthread = none idlethread = 0xfffff800027ac000: tid 100006 "idle: cpu3" curpmap = 0xffffffff818a3698 tssp = 0xffffffff818bacc8 commontssp = 0xffffffff818bacc8 rsp0 = 0xfffffe00d3395b80 gs32p = 0xffffffff818bc720 ldt = 0xffffffff818bc760 tss = 0xffffffff818bc750 spin locks held: db:0:allpcpu> show alllocks db:0:alllocks> show lockedvnods Locked vnodes db:0:lockedvnods> show mount 0xfffff80008563b58 /dev/da0p2 on / (ufs) 0xfffff80008564000 devfs on /dev (devfs) 0xfffff8000875f3c8 /dev/da0p3 on /tmp (ufs) 0xfffff80008563790 procfs on /proc (procfs) More info: show mount db:0:mount> ps pid ppid pgrp uid state wmesg wchan cmd 2051 1 2051 0 Ss+ ttyin 0xfffff800028a38a8 sh 1419 1 1353 1002 DE+ objtrm 0xfffff80032741b00 stealer 1342 0 0 0 DL mdwait 0xfffff80008385000 [md5] 17 0 0 0 DL sdflush 0xffffffff8182e978 [softdepflush] 16 0 0 0 DL vlruwt 0xfffff80008418970 [vnlru] 9 0 0 0 DL syncer 0xffffffff81827cb0 [syncer] 8 0 0 0 DL psleep 0xffffffff81827684 [bufdaemon] 7 0 0 0 DL pgzero 0xffffffff8182ff8c [pagezero] 6 0 0 0 DL psleep 0xffffffff8182f6c0 [vmdaemon] 5 0 0 0 DL psleep 0xffffffff818ba404 [pagedaemon] 4 0 0 0 DL idle 0xfffff8000841a970 [enc_daemon0] 3 0 0 0 DL waiting_ 0xffffffff818ae258 [sctp_iterator] 15 0 0 0 DL (threaded) [usb] 100063 D - 0xfffffe0000b87e18 [usbus2] 100062 D - 0xfffffe0000b87dc0 [usbus2] 100061 D - 0xfffffe0000b87d68 [usbus2] 100060 D - 0xfffffe0000b87d10 [usbus2] 100058 D - 0xfffffe0000b801b0 [usbus1] 100057 D - 0xfffffe0000b80158 [usbus1] 100056 D - 0xfffffe0000b80100 [usbus1] 100055 D - 0xfffffe0000b800a8 [usbus1] 100034 D - 0xfffffe0000985e18 [usbus0] 100033 D - 0xfffffe0000985dc0 [usbus0] 100032 D - 0xfffffe0000985d68 [usbus0] 100031 D - 0xfffffe0000985d10 [usbus0] 2 0 0 0 DL (threaded) [cam] 100071 D ccb_scan 0xffffffff814af740 [scanner] 100021 D - 0xffffffff814af900 [doneq0] 14 0 0 0 DL - 0xffffffff814cc9a0 [rand_harvestq] 13 0 0 0 DL (threaded) [geom] 100015 D - 0xffffffff818a2b30 [g_down] 100014 D - 0xffffffff818a2b28 [g_up] 100013 D - 0xffffffff818a2b20 [g_event] 12 0 0 0 WL (threaded) [intr] 100066 I [swi0: uart] 100065 I [irq279: hdac1] 100064 I [irq278: ahci0] 100059 I [irq20: ehci1] 100054 I [irq277: xhci0] 100053 I [irq276: igb1:link] 100051 I [irq275: igb1:que 3] 100049 I [irq274: igb1:que 2] 100047 I [irq273: igb1:que 1] 100045 I [irq272: igb1:que 0] 100044 I [irq271: igb0:link] 100042 I [irq270: igb0:que 3] 100040 I [irq269: igb0:que 2] 100038 I [irq268: igb0:que 1] 100036 I [irq267: igb0:que 0] 100035 I [irq266: hdac0] 100030 I [irq22: ehci0] 100029 I [irq265: isci0] 100028 I [irq264: isci0] 100027 I [swi6: task queue] 100020 I [swi5: fast taskq] 100018 I [swi6: Giant taskq] 100012 I [swi4: clock (3)] 100011 I [swi4: clock (2)] 100010 I [swi4: clock (1)] 100009 I [swi4: clock (0)] 100008 I [swi1: netisr 0] 100007 I [swi3: vm] 11 0 0 0 RL (threaded) [idle] 100006 Run CPU 3 [idle: cpu3] 100005 Run CPU 2 [idle: cpu2] 100004 Run CPU 1 [idle: cpu1] 100003 Run CPU 0 [idle: cpu0] 1 0 1 0 SLs wait 0xfffff800027a64b8 [init] 10 0 0 0 DL audit_wo 0xffffffff818b19c8 [audit] 0 0 0 0 DLs (threaded) [kernel] 100070 D - 0xfffff8000276e200 [CAM taskq] 100068 D - 0xffffffff814f6f40 [deadlkres] 100067 D - 0xfffff800061d3c00 [mca taskq] 100052 D - 0xfffff8000276d100 [igb1 que] 100050 D - 0xfffff8000276d200 [igb1 que] 100048 D - 0xfffff8000276d300 [igb1 que] 100046 D - 0xfffff8000276d400 [igb1 que] 100043 D - 0xfffff8000276d500 [igb0 que] 100041 D - 0xfffff8000276d600 [igb0 que] 100039 D - 0xfffff8000276d700 [igb0 que] 100037 D - 0xfffff8000276d800 [igb0 que] 100026 D - 0xfffff8000276de00 [acpi_task_2] 100025 D - 0xfffff8000276de00 [acpi_task_1] 100024 D - 0xfffff8000276de00 [acpi_task_0] 100023 D - 0xfffff8000276e000 [kqueue taskq] 100022 D - 0xfffff8000276e100 [ffs_trim taskq] 100019 D - 0xfffff8000276e400 [thread taskq] 100016 D - 0xfffff8000276ed00 [firmware taskq] 100000 D swapin 0xffffffff818a2c28 [swapper] db:0:ps> allt Tracing command sh pid 2051 tid 100366 td 0xfffff80055485000 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d8eb85f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d8eb8630 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d8eb8670 sleepq_catch_signals() at sleepq_catch_signals+0x2dc/frame 0xfffffe00d8eb86e0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00d8eb8710 _cv_wait_sig() at _cv_wait_sig+0x1dd/frame 0xfffffe00d8eb8780 tty_wait() at tty_wait+0x42/frame 0xfffffe00d8eb87b0 ttydisc_read() at ttydisc_read+0x34f/frame 0xfffffe00d8eb8810 ttydev_read() at ttydev_read+0x6d/frame 0xfffffe00d8eb8840 devfs_read_f() at devfs_read_f+0xbb/frame 0xfffffe00d8eb88b0 dofileread() at dofileread+0x95/frame 0xfffffe00d8eb8900 kern_readv() at kern_readv+0x68/frame 0xfffffe00d8eb8950 sys_read() at sys_read+0x63/frame 0xfffffe00d8eb89a0 amd64_syscall() at amd64_syscall+0x278/frame 0xfffffe00d8eb8ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe00d8eb8ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800de6b6a, rsp = 0x7fffffffdba8, rbp = 0x7fffffffdbd0 --- Tracing command stealer pid 1419 tid 100232 td 0xfffff8007a8c6000 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d8c185e0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d8c18620 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d8c18660 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d8c18690 _sleep() at _sleep+0x366/frame 0xfffffe00d8c18730 vm_object_pip_wait() at vm_object_pip_wait+0x92/frame 0xfffffe00d8c18760 vm_object_terminate() at vm_object_terminate+0x4b/frame 0xfffffe00d8c187a0 vm_object_deallocate() at vm_object_deallocate+0x822/frame 0xfffffe00d8c187f0 vm_map_entry_deallocate() at vm_map_entry_deallocate+0x35/frame 0xfffffe00d8c18820 vm_map_process_deferred() at vm_map_process_deferred+0x115/frame 0xfffffe00d8c18850 _vm_map_unlock() at _vm_map_unlock+0x68/frame 0xfffffe00d8c18880 vm_map_remove() at vm_map_remove+0xbe/frame 0xfffffe00d8c188c0 vmspace_dofree() at vmspace_dofree+0x22/frame 0xfffffe00d8c188e0 vmspace_exit() at vmspace_exit+0x15e/frame 0xfffffe00d8c18920 exit1() at exit1+0x541/frame 0xfffffe00d8c18990 sys_sys_exit() at sys_sys_exit+0xe/frame 0xfffffe00d8c189a0 amd64_syscall() at amd64_syscall+0x278/frame 0xfffffe00d8c18ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe00d8c18ab0 --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8008e4cfa, rsp = 0x7fffffffd108, rbp = 0x7fffffffd120 --- Tracing command md5 pid 1342 tid 100096 td 0xfffff800085c3920 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d896e8d0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d896e910 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d896e950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d896e980 _sleep() at _sleep+0x366/frame 0xfffffe00d896ea20 md_kthread() at md_kthread+0xe5/frame 0xfffffe00d896ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d896eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d896eab0 --- trap 0, rip = 0, rsp = 0xfffffe00d896eb70, rbp = 0 --- Tracing command softdepflush pid 17 tid 100079 td 0xfffff80008425920 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d89188e0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d8918920 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d8918960 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe00d8918990 _sleep() at _sleep+0x33a/frame 0xfffffe00d8918a30 softdep_flush() at softdep_flush+0x189/frame 0xfffffe00d8918a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d8918ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d8918ab0 --- trap 0, rip = 0, rsp = 0xfffffe00d8918b70, rbp = 0 --- Tracing command vnlru pid 16 tid 100078 td 0xfffff80008426000 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d89138a0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d89138e0 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d8913920 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe00d8913950 _sleep() at _sleep+0x33a/frame 0xfffffe00d89139f0 vnlru_proc() at vnlru_proc+0x48/frame 0xfffffe00d8913a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d8913ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d8913ab0 --- trap 0, rip = 0, rsp = 0xfffffe00d8913b70, rbp = 0 --- Tracing command syncer pid 9 tid 100077 td 0xfffff80008426490 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d890e8a0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d890e8e0 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d890e920 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe00d890e950 _cv_timedwait_sbt() at _cv_timedwait_sbt+0x200/frame 0xfffffe00d890e9d0 sched_sync() at sched_sync+0x5ee/frame 0xfffffe00d890ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d890eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d890eab0 --- trap 0, rip = 0, rsp = 0xfffffe00d890eb70, rbp = 0 --- Tracing command bufdaemon pid 8 tid 100076 td 0xfffff80008426920 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d89098f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d8909930 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d8909970 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe00d89099a0 _sleep() at _sleep+0x33a/frame 0xfffffe00d8909a40 buf_daemon() at buf_daemon+0x68/frame 0xfffffe00d8909a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d8909ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d8909ab0 --- trap 0, rip = 0, rsp = 0xfffffe00d8909b70, rbp = 0 --- Tracing command pagezero pid 7 tid 100075 td 0xfffff8000842b000 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d89048f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d8904930 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d8904970 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe00d89049a0 _sleep() at _sleep+0x33a/frame 0xfffffe00d8904a40 vm_pagezero() at vm_pagezero+0x88/frame 0xfffffe00d8904a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d8904ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d8904ab0 --- trap 0, rip = 0, rsp = 0xfffffe00d8904b70, rbp = 0 --- Tracing command vmdaemon pid 6 tid 100074 td 0xfffff8000842b490 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d87fe8b0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d87fe8f0 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d87fe930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d87fe960 _sleep() at _sleep+0x366/frame 0xfffffe00d87fea00 vm_daemon() at vm_daemon+0x7e/frame 0xfffffe00d87fea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d87feab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d87feab0 --- trap 0, rip = 0, rsp = 0xfffffe00d87feb70, rbp = 0 --- Tracing command pagedaemon pid 5 tid 100073 td 0xfffff8000842b920 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d87f9890 mi_switch() at mi_switch+0x179/frame 0xfffffe00d87f98d0 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d87f9910 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe00d87f9940 _sleep() at _sleep+0x33a/frame 0xfffffe00d87f99e0 vm_pageout() at vm_pageout+0x239/frame 0xfffffe00d87f9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d87f9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d87f9ab0 --- trap 0, rip = 0, rsp = 0xfffffe00d87f9b70, rbp = 0 --- Tracing command enc_daemon0 pid 4 tid 100072 td 0xfffff80006039000 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d87f48b0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d87f48f0 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d87f4930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d87f4960 _sleep() at _sleep+0x366/frame 0xfffffe00d87f4a00 enc_daemon() at enc_daemon+0x110/frame 0xfffffe00d87f4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d87f4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d87f4ab0 --- trap 0, rip = 0, rsp = 0xfffffe00d87f4b70, rbp = 0 --- Tracing command sctp_iterator pid 3 tid 100069 td 0xfffff8000603e000 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d8725900 mi_switch() at mi_switch+0x179/frame 0xfffffe00d8725940 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d8725980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d87259b0 _sleep() at _sleep+0x366/frame 0xfffffe00d8725a50 sctp_iterator_thread() at sctp_iterator_thread+0x59/frame 0xfffffe00d8725a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d8725ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d8725ab0 --- trap 0, rip = 0, rsp = 0xfffffe00d8725b70, rbp = 0 --- Tracing command usb pid 15 tid 100063 td 0xfffff80006005000 sched_switch() at sched_switch+0x48b/frame 0xfffffe0000282920 mi_switch() at mi_switch+0x179/frame 0xfffffe0000282960 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00002829a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00002829d0 _cv_wait() at _cv_wait+0x1da/frame 0xfffffe0000282a40 usb_process() at usb_process+0xd4/frame 0xfffffe0000282a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000282ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000282ab0 --- trap 0, rip = 0, rsp = 0xfffffe0000282b70, rbp = 0 --- Tracing command usb pid 15 tid 100062 td 0xfffff80006005490 sched_switch() at sched_switch+0x48b/frame 0xfffffe000027d920 mi_switch() at mi_switch+0x179/frame 0xfffffe000027d960 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe000027d9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000027d9d0 _cv_wait() at _cv_wait+0x1da/frame 0xfffffe000027da40 usb_process() at usb_process+0xd4/frame 0xfffffe000027da70 fork_exit() at fork_exit+0x84/frame 0xfffffe000027dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000027dab0 --- trap 0, rip = 0, rsp = 0xfffffe000027db70, rbp = 0 --- Tracing command usb pid 15 tid 100061 td 0xfffff80006005920 sched_switch() at sched_switch+0x48b/frame 0xfffffe0000278920 mi_switch() at mi_switch+0x179/frame 0xfffffe0000278960 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00002789a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00002789d0 _cv_wait() at _cv_wait+0x1da/frame 0xfffffe0000278a40 usb_process() at usb_process+0xd4/frame 0xfffffe0000278a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000278ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000278ab0 --- trap 0, rip = 0, rsp = 0xfffffe0000278b70, rbp = 0 --- Tracing command usb pid 15 tid 100060 td 0xfffff80006006000 sched_switch() at sched_switch+0x48b/frame 0xfffffe0000273920 mi_switch() at mi_switch+0x179/frame 0xfffffe0000273960 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00002739a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00002739d0 _cv_wait() at _cv_wait+0x1da/frame 0xfffffe0000273a40 usb_process() at usb_process+0xd4/frame 0xfffffe0000273a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000273ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000273ab0 --- trap 0, rip = 0, rsp = 0xfffffe0000273b70, rbp = 0 --- Tracing command usb pid 15 tid 100058 td 0xfffff80006006920 sched_switch() at sched_switch+0x48b/frame 0xfffffe0000267920 mi_switch() at mi_switch+0x179/frame 0xfffffe0000267960 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00002679a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00002679d0 _cv_wait() at _cv_wait+0x1da/frame 0xfffffe0000267a40 usb_process() at usb_process+0xd4/frame 0xfffffe0000267a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000267ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000267ab0 --- trap 0, rip = 0, rsp = 0xfffffe0000267b70, rbp = 0 --- Tracing command usb pid 15 tid 100057 td 0xfffff80006007000 sched_switch() at sched_switch+0x48b/frame 0xfffffe0000262920 mi_switch() at mi_switch+0x179/frame 0xfffffe0000262960 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00002629a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00002629d0 _cv_wait() at _cv_wait+0x1da/frame 0xfffffe0000262a40 usb_process() at usb_process+0xd4/frame 0xfffffe0000262a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000262ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000262ab0 --- trap 0, rip = 0, rsp = 0xfffffe0000262b70, rbp = 0 --- Tracing command usb pid 15 tid 100056 td 0xfffff80006007490 sched_switch() at sched_switch+0x48b/frame 0xfffffe000025d920 mi_switch() at mi_switch+0x179/frame 0xfffffe000025d960 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe000025d9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000025d9d0 _cv_wait() at _cv_wait+0x1da/frame 0xfffffe000025da40 usb_process() at usb_process+0xd4/frame 0xfffffe000025da70 fork_exit() at fork_exit+0x84/frame 0xfffffe000025dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000025dab0 --- trap 0, rip = 0, rsp = 0xfffffe000025db70, rbp = 0 --- Tracing command usb pid 15 tid 100055 td 0xfffff80006007920 sched_switch() at sched_switch+0x48b/frame 0xfffffe0000258920 mi_switch() at mi_switch+0x179/frame 0xfffffe0000258960 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00002589a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00002589d0 _cv_wait() at _cv_wait+0x1da/frame 0xfffffe0000258a40 usb_process() at usb_process+0xd4/frame 0xfffffe0000258a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000258ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000258ab0 --- trap 0, rip = 0, rsp = 0xfffffe0000258b70, rbp = 0 --- Tracing command usb pid 15 tid 100034 td 0xfffff80005db0920 sched_switch() at sched_switch+0x48b/frame 0xfffffe00b5f7a920 mi_switch() at mi_switch+0x179/frame 0xfffffe00b5f7a960 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00b5f7a9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b5f7a9d0 _cv_wait() at _cv_wait+0x1da/frame 0xfffffe00b5f7aa40 usb_process() at usb_process+0xd4/frame 0xfffffe00b5f7aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b5f7aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b5f7aab0 --- trap 0, rip = 0, rsp = 0xfffffe00b5f7ab70, rbp = 0 --- Tracing command usb pid 15 tid 100033 td 0xfffff80005db1000 sched_switch() at sched_switch+0x48b/frame 0xfffffe00b5f75920 mi_switch() at mi_switch+0x179/frame 0xfffffe00b5f75960 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00b5f759a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b5f759d0 _cv_wait() at _cv_wait+0x1da/frame 0xfffffe00b5f75a40 usb_process() at usb_process+0xd4/frame 0xfffffe00b5f75a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b5f75ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b5f75ab0 --- trap 0, rip = 0, rsp = 0xfffffe00b5f75b70, rbp = 0 --- Tracing command usb pid 15 tid 100032 td 0xfffff80005db1490 sched_switch() at sched_switch+0x48b/frame 0xfffffe00b5f70920 mi_switch() at mi_switch+0x179/frame 0xfffffe00b5f70960 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00b5f709a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b5f709d0 _cv_wait() at _cv_wait+0x1da/frame 0xfffffe00b5f70a40 usb_process() at usb_process+0xd4/frame 0xfffffe00b5f70a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b5f70ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b5f70ab0 --- trap 0, rip = 0, rsp = 0xfffffe00b5f70b70, rbp = 0 --- Tracing command usb pid 15 tid 100031 td 0xfffff80005db1920 sched_switch() at sched_switch+0x48b/frame 0xfffffe00b5f6b920 mi_switch() at mi_switch+0x179/frame 0xfffffe00b5f6b960 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00b5f6b9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b5f6b9d0 _cv_wait() at _cv_wait+0x1da/frame 0xfffffe00b5f6ba40 usb_process() at usb_process+0xd4/frame 0xfffffe00b5f6ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b5f6bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b5f6bab0 --- trap 0, rip = 0, rsp = 0xfffffe00b5f6bb70, rbp = 0 --- Tracing command cam pid 2 tid 100071 td 0xfffff80006039490 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d872f8c0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d872f900 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d872f940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d872f970 _sleep() at _sleep+0x366/frame 0xfffffe00d872fa10 xpt_scanner_thread() at xpt_scanner_thread+0x5c/frame 0xfffffe00d872fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d872fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d872fab0 --- trap 0, rip = 0, rsp = 0xfffffe00d872fb70, rbp = 0 --- Tracing command cam pid 2 tid 100021 td 0xfffff80002996000 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d33e08d0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d33e0910 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d33e0950 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d33e0980 _sleep() at _sleep+0x366/frame 0xfffffe00d33e0a20 xpt_done_td() at xpt_done_td+0xa5/frame 0xfffffe00d33e0a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d33e0ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d33e0ab0 --- trap 0, rip = 0, rsp = 0xfffffe00d33e0b70, rbp = 0 --- Tracing command rand_harvestq pid 14 tid 100017 td 0xfffff800027e3490 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d33cc8f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d33cc930 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d33cc970 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe00d33cc9a0 msleep_spin_sbt() at msleep_spin_sbt+0x1ff/frame 0xfffffe00d33cca30 random_kthread() at random_kthread+0x1d0/frame 0xfffffe00d33cca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d33ccab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d33ccab0 --- trap 0, rip = 0, rsp = 0xfffffe00d33ccb70, rbp = 0 --- Tracing command geom pid 13 tid 100015 td 0xfffff800027e4000 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d33c28e0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d33c2920 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d33c2960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d33c2990 _sleep() at _sleep+0x366/frame 0xfffffe00d33c2a30 g_io_schedule_down() at g_io_schedule_down+0x3c/frame 0xfffffe00d33c2a60 g_down_procbody() at g_down_procbody+0x7d/frame 0xfffffe00d33c2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d33c2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d33c2ab0 --- trap 0, rip = 0, rsp = 0xfffffe00d33c2b70, rbp = 0 --- Tracing command geom pid 13 tid 100014 td 0xfffff800027e4490 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d33bd8e0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d33bd920 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d33bd960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d33bd990 _sleep() at _sleep+0x366/frame 0xfffffe00d33bda30 g_io_schedule_up() at g_io_schedule_up+0xbc/frame 0xfffffe00d33bda60 g_up_procbody() at g_up_procbody+0x7d/frame 0xfffffe00d33bda70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d33bdab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d33bdab0 --- trap 0, rip = 0, rsp = 0xfffffe00d33bdb70, rbp = 0 --- Tracing command geom pid 13 tid 100013 td 0xfffff800027e4920 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d33b88e0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d33b8920 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d33b8960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d33b8990 _sleep() at _sleep+0x366/frame 0xfffffe00d33b8a30 g_run_events() at g_run_events+0x4d/frame 0xfffffe00d33b8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d33b8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d33b8ab0 --- trap 0, rip = 0, rsp = 0xfffffe00d33b8b70, rbp = 0 --- Tracing command intr pid 12 tid 100066 td 0xfffff8000603f000 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d87119f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d8711a30 ithread_loop() at ithread_loop+0x1cf/frame 0xfffffe00d8711a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d8711ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d8711ab0 --- trap 0, rip = 0, rsp = 0xfffffe00d8711b70, rbp = 0 --- Tracing command intr pid 12 tid 100065 td 0xfffff8000603f490 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d87099f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d8709a30 ithread_loop() at ithread_loop+0x1cf/frame 0xfffffe00d8709a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d8709ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d8709ab0 --- trap 0, rip = 0, rsp = 0xfffffe00d8709b70, rbp = 0 --- Tracing command intr pid 12 tid 100064 td 0xfffff8000603f920 sched_switch() at sched_switch+0x48b/frame 0xfffffe00002879f0 mi_switch() at mi_switch+0x179/frame 0xfffffe0000287a30 ithread_loop() at ithread_loop+0x1cf/frame 0xfffffe0000287a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000287ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000287ab0 --- trap 0, rip = 0, rsp = 0xfffffe0000287b70, rbp = 0 --- Tracing command intr pid 12 tid 100059 td 0xfffff80006006490 sched_switch() at sched_switch+0x48b/frame 0xfffffe000026e9f0 mi_switch() at mi_switch+0x179/frame 0xfffffe000026ea30 ithread_loop() at ithread_loop+0x1cf/frame 0xfffffe000026ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe000026eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000026eab0 --- trap 0, rip = 0, rsp = 0xfffffe000026eb70, rbp = 0 --- Tracing command intr pid 12 tid 100054 td 0xfffff80005dfb000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100053 td 0xfffff80005dfb490 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100051 td 0xfffff80005dfc000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100049 td 0xfffff80005dfc920 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100047 td 0xfffff80005dfd490 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100045 td 0xfffff80005deb000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100044 td 0xfffff80005deb490 sched_switch() at sched_switch+0x48b/frame 0xfffffe00b5fd69f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00b5fd6a30 ithread_loop() at ithread_loop+0x1cf/frame 0xfffffe00b5fd6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b5fd6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b5fd6ab0 --- trap 0, rip = 0, rsp = 0xfffffe00b5fd6b70, rbp = 0 --- Tracing command intr pid 12 tid 100042 td 0xfffff80005dec000 sched_switch() at sched_switch+0x48b/frame 0xfffffe00b5fcc9f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00b5fcca30 ithread_loop() at ithread_loop+0x1cf/frame 0xfffffe00b5fcca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b5fccab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b5fccab0 --- trap 0, rip = 0, rsp = 0xfffffe00b5fccb70, rbp = 0 --- Tracing command intr pid 12 tid 100040 td 0xfffff80005dec920 sched_switch() at sched_switch+0x48b/frame 0xfffffe00b5fc29f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00b5fc2a30 ithread_loop() at ithread_loop+0x1cf/frame 0xfffffe00b5fc2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b5fc2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b5fc2ab0 --- trap 0, rip = 0, rsp = 0xfffffe00b5fc2b70, rbp = 0 --- Tracing command intr pid 12 tid 100038 td 0xfffff80005ded490 sched_switch() at sched_switch+0x48b/frame 0xfffffe00b5fb89f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00b5fb8a30 ithread_loop() at ithread_loop+0x1cf/frame 0xfffffe00b5fb8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b5fb8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b5fb8ab0 --- trap 0, rip = 0, rsp = 0xfffffe00b5fb8b70, rbp = 0 --- Tracing command intr pid 12 tid 100036 td 0xfffff80005db0000 sched_switch() at sched_switch+0x48b/frame 0xfffffe00b5fae9f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00b5faea30 ithread_loop() at ithread_loop+0x1cf/frame 0xfffffe00b5faea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b5faeab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b5faeab0 --- trap 0, rip = 0, rsp = 0xfffffe00b5faeb70, rbp = 0 --- Tracing command intr pid 12 tid 100035 td 0xfffff80005db0490 sched_switch() at sched_switch+0x48b/frame 0xfffffe00b5f7f9f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00b5f7fa30 ithread_loop() at ithread_loop+0x1cf/frame 0xfffffe00b5f7fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b5f7fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b5f7fab0 --- trap 0, rip = 0, rsp = 0xfffffe00b5f7fb70, rbp = 0 --- Tracing command intr pid 12 tid 100030 td 0xfffff80005db2000 sched_switch() at sched_switch+0x48b/frame 0xfffffe00b5f669f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00b5f66a30 ithread_loop() at ithread_loop+0x1cf/frame 0xfffffe00b5f66a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b5f66ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b5f66ab0 --- trap 0, rip = 0, rsp = 0xfffffe00b5f66b70, rbp = 0 --- Tracing command intr pid 12 tid 100029 td 0xfffff80005db2490 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100028 td 0xfffff80005db2920 sched_switch() at sched_switch+0x48b/frame 0xfffffe00b5f5c9f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00b5f5ca30 ithread_loop() at ithread_loop+0x1cf/frame 0xfffffe00b5f5ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b5f5cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b5f5cab0 --- trap 0, rip = 0, rsp = 0xfffffe00b5f5cb70, rbp = 0 --- Tracing command intr pid 12 tid 100027 td 0xfffff80002994000 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d33fe9f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d33fea30 ithread_loop() at ithread_loop+0x1cf/frame 0xfffffe00d33fea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d33feab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d33feab0 --- trap 0, rip = 0, rsp = 0xfffffe00d33feb70, rbp = 0 --- Tracing command intr pid 12 tid 100020 td 0xfffff80002996490 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100018 td 0xfffff800027e3000 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d33d19f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d33d1a30 ithread_loop() at ithread_loop+0x1cf/frame 0xfffffe00d33d1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d33d1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d33d1ab0 --- trap 0, rip = 0, rsp = 0xfffffe00d33d1b70, rbp = 0 --- Tracing command intr pid 12 tid 100012 td 0xfffff800027e5000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100011 td 0xfffff800027e5490 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100010 td 0xfffff800027e5920 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100009 td 0xfffff800027a7000 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d33a49f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d33a4a30 ithread_loop() at ithread_loop+0x1cf/frame 0xfffffe00d33a4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d33a4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d33a4ab0 --- trap 0, rip = 0, rsp = 0xfffffe00d33a4b70, rbp = 0 --- Tracing command intr pid 12 tid 100008 td 0xfffff800027a7490 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d339f9f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d339fa30 ithread_loop() at ithread_loop+0x1cf/frame 0xfffffe00d339fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d339fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d339fab0 --- trap 0, rip = 0, rsp = 0xfffffe00d339fb70, rbp = 0 --- Tracing command intr pid 12 tid 100007 td 0xfffff800027a7920 fork_trampoline() at fork_trampoline Tracing command idle pid 11 tid 100006 td 0xfffff800027ac000 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00b4ff8cf0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe00b4ff8d10 trap() at trap+0x3a/frame 0xfffffe00b4ff8f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00b4ff8f20 --- trap 0x13, rip = 0xffffffff80cd8916, rsp = 0xfffffe00b4ff8fe0, rbp = 0xfffffe00d3395930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00d3395930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe00d3395980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe00d33959a0 cpu_idle() at cpu_idle+0x90/frame 0xfffffe00d33959c0 sched_idletd() at sched_idletd+0x426/frame 0xfffffe00d3395a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d3395ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d3395ab0 --- trap 0, rip = 0, rsp = 0xfffffe00d3395b70, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffff800027ac490 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00b4ff0cf0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe00b4ff0d10 trap() at trap+0x3a/frame 0xfffffe00b4ff0f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00b4ff0f20 --- trap 0x13, rip = 0xffffffff80cd8916, rsp = 0xfffffe00b4ff0fe0, rbp = 0xfffffe00d3390930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00d3390930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe00d3390980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe00d33909a0 cpu_idle() at cpu_idle+0x90/frame 0xfffffe00d33909c0 sched_idletd() at sched_idletd+0x426/frame 0xfffffe00d3390a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d3390ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d3390ab0 --- trap 0, rip = 0, rsp = 0xfffffe00d3390b70, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xfffff800027ac920 kdb_alt_break_internal() at kdb_alt_break_internal+0x17f/frame 0xfffffe00d338b730 kdb_alt_break() at kdb_alt_break+0xb/frame 0xfffffe00d338b740 uart_intr_rxready() at uart_intr_rxready+0x99/frame 0xfffffe00d338b770 uart_intr() at uart_intr+0x111/frame 0xfffffe00d338b7b0 intr_event_handle() at intr_event_handle+0x8c/frame 0xfffffe00d338b800 intr_execute_handlers() at intr_execute_handlers+0x48/frame 0xfffffe00d338b830 lapic_handle_intr() at lapic_handle_intr+0x73/frame 0xfffffe00d338b860 Xapic_isr1() at Xapic_isr1+0xa4/frame 0xfffffe00d338b860 --- interrupt, rip = 0xffffffff80cd8916, rsp = 0xfffffe00d338b920, rbp = 0xfffffe00d338b930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00d338b930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe00d338b980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe00d338b9a0 cpu_idle() at cpu_idle+0x90/frame 0xfffffe00d338b9c0 sched_idletd() at sched_idletd+0x426/frame 0xfffffe00d338ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d338bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d338bab0 --- trap 0, rip = 0, rsp = 0xfffffe00d338bb70, rbp = 0 --- Tracing command idle pid 11 tid 100003 td 0xfffff800027ad000 cpustop_handler() at cpustop_handler+0x28/frame 0xffffffff81834430 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xffffffff81834450 trap() at trap+0x3a/frame 0xffffffff81834660 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff81834660 --- trap 0x13, rip = 0xffffffff80cd8916, rsp = 0xffffffff81834720, rbp = 0xfffffe00d3386930 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xfffffe00d3386930 acpi_cpu_idle() at acpi_cpu_idle+0x13b/frame 0xfffffe00d3386980 cpu_idle_acpi() at cpu_idle_acpi+0x3f/frame 0xfffffe00d33869a0 cpu_idle() at cpu_idle+0x90/frame 0xfffffe00d33869c0 sched_idletd() at sched_idletd+0x426/frame 0xfffffe00d3386a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d3386ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d3386ab0 --- trap 0, rip = 0, rsp = 0xfffffe00d3386b70, rbp = 0 --- Tracing command init pid 1 tid 100002 td 0xfffff800027ad490 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d3381540 mi_switch() at mi_switch+0x179/frame 0xfffffe00d3381580 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d33815c0 sleepq_catch_signals() at sleepq_catch_signals+0x2dc/frame 0xfffffe00d3381630 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00d3381660 _sleep() at _sleep+0x349/frame 0xfffffe00d3381700 kern_wait6() at kern_wait6+0x404/frame 0xfffffe00d33817b0 sys_wait4() at sys_wait4+0x72/frame 0xfffffe00d33819a0 amd64_syscall() at amd64_syscall+0x278/frame 0xfffffe00d3381ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe00d3381ab0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x4137fa, rsp = 0x7fffffffd6d8, rbp = 0x7fffffffd7d0 --- Tracing command audit pid 10 tid 100001 td 0xfffff800027ad920 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d337c8f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d337c930 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d337c970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d337c9a0 _cv_wait() at _cv_wait+0x1da/frame 0xfffffe00d337ca10 audit_worker() at audit_worker+0xa3/frame 0xfffffe00d337ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d337cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d337cab0 --- trap 0, rip = 0, rsp = 0xfffffe00d337cb70, rbp = 0 --- Tracing command kernel pid 0 tid 100070 td 0xfffff80006039920 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d872a8f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d872a930 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d872a970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d872a9a0 _sleep() at _sleep+0x366/frame 0xfffffe00d872aa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe00d872aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d872aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d872aab0 --- trap 0, rip = 0, rsp = 0xfffffe00d872ab70, rbp = 0 --- Tracing command kernel pid 0 tid 100068 td 0xfffff8000603e490 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d871f880 mi_switch() at mi_switch+0x179/frame 0xfffffe00d871f8c0 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d871f900 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe00d871f930 _sleep() at _sleep+0x33a/frame 0xfffffe00d871f9d0 pause_sbt() at pause_sbt+0xe4/frame 0xfffffe00d871fa20 deadlkres() at deadlkres+0x2c/frame 0xfffffe00d871fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d871fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d871fab0 --- trap 0, rip = 0, rsp = 0xfffffe00d871fb70, rbp = 0 --- Tracing command kernel pid 0 tid 100067 td 0xfffff8000603e920 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d871a900 mi_switch() at mi_switch+0x179/frame 0xfffffe00d871a940 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d871a980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d871a9b0 msleep_spin_sbt() at msleep_spin_sbt+0x209/frame 0xfffffe00d871aa40 taskqueue_thread_loop() at taskqueue_thread_loop+0x8d/frame 0xfffffe00d871aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d871aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d871aab0 --- trap 0, rip = 0, rsp = 0xfffffe00d871ab70, rbp = 0 --- Tracing command kernel pid 0 tid 100052 td 0xfffff80005dfb920 sched_switch() at sched_switch+0x48b/frame 0xfffffe000022b8f0 mi_switch() at mi_switch+0x179/frame 0xfffffe000022b930 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe000022b970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000022b9a0 _sleep() at _sleep+0x366/frame 0xfffffe000022ba40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe000022ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe000022bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000022bab0 --- trap 0, rip = 0, rsp = 0xfffffe000022bb70, rbp = 0 --- Tracing command kernel pid 0 tid 100050 td 0xfffff80005dfc490 sched_switch() at sched_switch+0x48b/frame 0xfffffe00002218f0 mi_switch() at mi_switch+0x179/frame 0xfffffe0000221930 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe0000221970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00002219a0 _sleep() at _sleep+0x366/frame 0xfffffe0000221a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe0000221a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000221ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000221ab0 --- trap 0, rip = 0, rsp = 0xfffffe0000221b70, rbp = 0 --- Tracing command kernel pid 0 tid 100048 td 0xfffff80005dfd000 sched_switch() at sched_switch+0x48b/frame 0xfffffe00002178f0 mi_switch() at mi_switch+0x179/frame 0xfffffe0000217930 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe0000217970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00002179a0 _sleep() at _sleep+0x366/frame 0xfffffe0000217a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe0000217a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000217ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000217ab0 --- trap 0, rip = 0, rsp = 0xfffffe0000217b70, rbp = 0 --- Tracing command kernel pid 0 tid 100046 td 0xfffff80005dfd920 sched_switch() at sched_switch+0x48b/frame 0xfffffe000020d8f0 mi_switch() at mi_switch+0x179/frame 0xfffffe000020d930 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe000020d970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000020d9a0 _sleep() at _sleep+0x366/frame 0xfffffe000020da40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe000020da70 fork_exit() at fork_exit+0x84/frame 0xfffffe000020dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000020dab0 --- trap 0, rip = 0, rsp = 0xfffffe000020db70, rbp = 0 --- Tracing command kernel pid 0 tid 100043 td 0xfffff80005deb920 sched_switch() at sched_switch+0x48b/frame 0xfffffe00b5fd18f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00b5fd1930 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00b5fd1970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b5fd19a0 _sleep() at _sleep+0x366/frame 0xfffffe00b5fd1a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe00b5fd1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b5fd1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b5fd1ab0 --- trap 0, rip = 0, rsp = 0xfffffe00b5fd1b70, rbp = 0 --- Tracing command kernel pid 0 tid 100041 td 0xfffff80005dec490 sched_switch() at sched_switch+0x48b/frame 0xfffffe00b5fc78f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00b5fc7930 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00b5fc7970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b5fc79a0 _sleep() at _sleep+0x366/frame 0xfffffe00b5fc7a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe00b5fc7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b5fc7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b5fc7ab0 --- trap 0, rip = 0, rsp = 0xfffffe00b5fc7b70, rbp = 0 --- Tracing command kernel pid 0 tid 100039 td 0xfffff80005ded000 sched_switch() at sched_switch+0x48b/frame 0xfffffe00b5fbd8f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00b5fbd930 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00b5fbd970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b5fbd9a0 _sleep() at _sleep+0x366/frame 0xfffffe00b5fbda40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe00b5fbda70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b5fbdab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b5fbdab0 --- trap 0, rip = 0, rsp = 0xfffffe00b5fbdb70, rbp = 0 --- Tracing command kernel pid 0 tid 100037 td 0xfffff80005ded920 sched_switch() at sched_switch+0x48b/frame 0xfffffe00b5fb38f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00b5fb3930 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00b5fb3970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00b5fb39a0 _sleep() at _sleep+0x366/frame 0xfffffe00b5fb3a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe00b5fb3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00b5fb3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b5fb3ab0 --- trap 0, rip = 0, rsp = 0xfffffe00b5fb3b70, rbp = 0 --- Tracing command kernel pid 0 tid 100026 td 0xfffff80002994490 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d33f9900 mi_switch() at mi_switch+0x179/frame 0xfffffe00d33f9940 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d33f9980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d33f99b0 msleep_spin_sbt() at msleep_spin_sbt+0x209/frame 0xfffffe00d33f9a40 taskqueue_thread_loop() at taskqueue_thread_loop+0x8d/frame 0xfffffe00d33f9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d33f9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d33f9ab0 --- trap 0, rip = 0, rsp = 0xfffffe00d33f9b70, rbp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xfffff80002994920 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d33f4900 mi_switch() at mi_switch+0x179/frame 0xfffffe00d33f4940 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d33f4980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d33f49b0 msleep_spin_sbt() at msleep_spin_sbt+0x209/frame 0xfffffe00d33f4a40 taskqueue_thread_loop() at taskqueue_thread_loop+0x8d/frame 0xfffffe00d33f4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d33f4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d33f4ab0 --- trap 0, rip = 0, rsp = 0xfffffe00d33f4b70, rbp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xfffff80002995000 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d33ef900 mi_switch() at mi_switch+0x179/frame 0xfffffe00d33ef940 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d33ef980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d33ef9b0 msleep_spin_sbt() at msleep_spin_sbt+0x209/frame 0xfffffe00d33efa40 taskqueue_thread_loop() at taskqueue_thread_loop+0x8d/frame 0xfffffe00d33efa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d33efab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d33efab0 --- trap 0, rip = 0, rsp = 0xfffffe00d33efb70, rbp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xfffff80002995490 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d33ea8f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d33ea930 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d33ea970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d33ea9a0 _sleep() at _sleep+0x366/frame 0xfffffe00d33eaa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe00d33eaa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d33eaab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d33eaab0 --- trap 0, rip = 0, rsp = 0xfffffe00d33eab70, rbp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xfffff80002995920 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d33e58f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d33e5930 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d33e5970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d33e59a0 _sleep() at _sleep+0x366/frame 0xfffffe00d33e5a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe00d33e5a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d33e5ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d33e5ab0 --- trap 0, rip = 0, rsp = 0xfffffe00d33e5b70, rbp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xfffff80002996920 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d33d68f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d33d6930 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d33d6970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d33d69a0 _sleep() at _sleep+0x366/frame 0xfffffe00d33d6a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe00d33d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d33d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d33d6ab0 --- trap 0, rip = 0, rsp = 0xfffffe00d33d6b70, rbp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xfffff800027e3920 sched_switch() at sched_switch+0x48b/frame 0xfffffe00d33c78f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00d33c7930 sleepq_switch() at sleepq_switch+0x152/frame 0xfffffe00d33c7970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00d33c79a0 _sleep() at _sleep+0x366/frame 0xfffffe00d33c7a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe00d33c7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00d33c7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d33c7ab0 --- trap 0, rip = 0, rsp = 0xfffffe00d33c7b70, rbp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xffffffff818a30f0 sched_switch() at sched_switch+0x48b/frame 0xffffffff81bc39d0 mi_switch() at mi_switch+0x179/frame 0xffffffff81bc3a10 sleepq_switch() at sleepq_switch+0x152/frame 0xffffffff81bc3a50 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xffffffff81bc3a80 _sleep() at _sleep+0x33a/frame 0xffffffff81bc3b20 swapper() at swapper+0x208/frame 0xffffffff81bc3b70 btext() at btext+0x2c db:0:allt> call doadump Dumping 281 out of 2969 MB:..6%..12%..23%..35%..46%..52%..63%..74%..86%..91% Dump complete = 0 db:0:doadump> reset cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 1 (kgdb) proc 1419 [Switching to thread 82 (Thread 100232)]#0 sched_switch (td=0xfffff8007a8c6000, newtd=, flags=) at ../../../kern/sched_ule.c:1928 1928 cpuid = PCPU_GET(cpuid); Current language: auto; currently minimal (kgdb) bt #0 sched_switch (td=0xfffff8007a8c6000, newtd=, flags=) at ../../../kern/sched_ule.c:1928 #1 0xffffffff808ed6d9 in mi_switch (flags=0x104, newtd=0x0) at ../../../kern/kern_synch.c:493 #2 0xffffffff8092a182 in sleepq_switch (wchan=, pri=) at ../../../kern/subr_sleepqueue.c:537 #3 0xffffffff80929fe3 in sleepq_wait (wchan=0xfffff80032741b00, pri=0x54) at ../../../kern/subr_sleepqueue.c:616 #4 0xffffffff808ecfe6 in _sleep (ident=, lock=, priority=, wmesg=, sbt=, pr=) at ../../../kern/kern_synch.c:255 #5 0xffffffff80b79852 in vm_object_pip_wait (object=0xfffff80032741b00, waitid=0xffffffff80fb81b8 "objtrm") at ../../../vm/vm_object.c:396 #6 0xffffffff80b7ae2b in vm_object_terminate (object=0xfffff80032741b00) at ../../../vm/vm_object.c:715 #7 0xffffffff80b7a1e2 in vm_object_deallocate (object=0xfffff80032741b00) at ../../../vm/vm_object.c:652 #8 0xffffffff80b768c5 in vm_map_entry_deallocate (entry=0xfffff80008602380, system_map=0x0) at ../../../vm/vm_map.c:2763 #9 0xffffffff80b6da45 in vm_map_process_deferred () at ../../../vm/vm_map.c:475 #10 0xffffffff80b6d928 in _vm_map_unlock (map=0xfffff8007a7d9700, file=0xffffffff80fb70fb "../../../vm/vm_map.c", line=0xb80) at ../../../vm/vm_map.c:488 #11 0xffffffff80b739fe in vm_map_remove (map=0xfffff8007a7d9700, start=0x1000, end=0x800000000000) at ../../../vm/vm_map.c:2944 #12 0xffffffff80b6d472 in vmspace_dofree (vm=0xfffff8007a7d9700) at ../../../vm/vm_map.c:327 #13 0xffffffff80b6d69e in vmspace_exit (td=0xfffff8007a8c6000) at ../../../vm/vm_map.c:405 #14 0xffffffff808adcb1 in exit1 (td=0xfffff8007a8c6000, rv=) at ../../../kern/kern_exit.c:319 #15 0xffffffff808ad76e in sys_sys_exit (td=0x0, uap=) at ../../../kern/kern_exit.c:119 #16 0xffffffff80cfb968 in amd64_syscall (td=0xfffff8007a8c6000, traced=0x0) at subr_syscall.c:133 #17 0xffffffff80cddb8b in Xfast_syscall () at ../../../amd64/amd64/exception.S:390 #18 0x00000008008e4cfa in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) f 5 #5 0xffffffff80b79852 in vm_object_pip_wait (object=0xfffff80032741b00, waitid=0xffffffff80fb81b8 "objtrm") at ../../../vm/vm_object.c:396 396 VM_OBJECT_SLEEP(object, object, PVM, waitid, 0); (kgdb) l 391 { 392 393 VM_OBJECT_ASSERT_WLOCKED(object); 394 while (object->paging_in_progress) { 395 object->flags |= OBJ_PIPWNT; 396 VM_OBJECT_SLEEP(object, object, PVM, waitid, 0); 397 } 398 } 399 400 /* (kgdb) p *object $1 = {lock = {lock_object = {lo_name = 0xffffffff80fb8076 "vm object", lo_flags = 0x5630000, lo_data = 0x0, lo_witness = 0xfffffe00006bd680}, rw_lock = 0x1}, object_list = {tqe_next = 0xfffff8003261a100, tqe_prev = 0xfffff80032741820}, shadow_head = {lh_first = 0x0}, shadow_list = {le_next = 0x0, le_prev = 0x0}, memq = {tqh_first = 0xfffff800b7574980, tqh_last = 0xfffff800b773acf8}, rtree = { rt_root = 0xfffff8007ad46630, rt_flags = 0x0}, size = 0xc00, generation = 0x1, ref_count = 0x0, shadow_count = 0x0, memattr = 0x6, type = 0x1, flags = 0x3048, pg_color = 0x1000, paging_in_progress = 0x0, resident_page_count = 0xc00, backing_object = 0x0, backing_object_offset = 0x0, pager_object_list = {tqe_next = 0x0, tqe_prev = 0x0}, rvq = {lh_first = 0xfffff800b6370800}, cache = {rt_root = 0x0, rt_flags = 0x0}, handle = 0x0, un_pager = {vnp = {vnp_size = 0x200, writemappings = 0x0}, devp = {devp_pglist = {tqh_first = 0x200, tqh_last = 0x0}, ops = 0x0, dev = 0x0}, sgp = {sgp_pglist = { tqh_first = 0x200, tqh_last = 0x0}}, swp = {swp_tmpfs = 0x200, swp_bcount = 0x0}}, cred = 0xfffff800086f4700, charge = 0xc00000} (kgdb) p object->flags $2 = 0x3048 (kgdb) f 8 #8 0xffffffff80b768c5 in vm_map_entry_deallocate (entry=0xfffff80008602380, system_map=0x0) at ../../../vm/vm_map.c:2763 2763 vm_object_deallocate(entry->object.vm_object); (kgdb) l 2758 static void 2759 vm_map_entry_deallocate(vm_map_entry_t entry, boolean_t system_map) 2760 { 2761 2762 if ((entry->eflags & MAP_ENTRY_IS_SUB_MAP) == 0) 2763 vm_object_deallocate(entry->object.vm_object); 2764 uma_zfree(system_map ? kmapentzone : mapentzone, entry); 2765 } 2766 2767 /* (kgdb) p *entry $3 = {prev = 0xfffff8007a7d9700, next = 0xfffff8007a8b2100, left = 0x0, right = 0xfffff8007a8b2080, start = 0x801000000, end = 0x801c00000, avail_ssize = 0x0, adj_free = 0x7ff7fe3df000, max_free = 0x7ff7fe3df000, object = {vm_object = 0xfffff80032741b00, sub_map = 0xfffff80032741b00}, offset = 0x0, eflags = 0x0, protection = 0x3, max_protection = 0x7, inheritance = 0x1, read_ahead = 0xf, wired_count = 0x0, next_read = 0x0, cred = 0x0, wiring_thread = 0x0} (kgdb) f 11 #11 0xffffffff80b739fe in vm_map_remove (map=0xfffff8007a7d9700, start=0x1000, end=0x800000000000) at ../../../vm/vm_map.c:2944 2944 vm_map_unlock(map); (kgdb) p *map $4 = {header = {prev = 0xfffff8007a7d9700, next = 0xfffff8007a7d9700, left = 0x0, right = 0x0, start = 0x1000, end = 0x800000000000, avail_ssize = 0x0, adj_free = 0x0, max_free = 0x0, object = { vm_object = 0x0, sub_map = 0x0}, offset = 0x0, eflags = 0x0, protection = 0x0, max_protection = 0x0, inheritance = 0x0, read_ahead = 0x0, wired_count = 0x0, next_read = 0x0, cred = 0x0, wiring_thread = 0x0}, lock = {lock_object = {lo_name = 0xffffffff80f7ee42 "vm map (user)", lo_flags = 0x2330000, lo_data = 0x0, lo_witness = 0xfffffe00006bd600}, sx_lock = 0x1}, system_mtx = { lock_object = {lo_name = 0xffffffff80f7ee14 "vm map (system)", lo_flags = 0x1430000, lo_data = 0x0, lo_witness = 0xfffffe00006bd400}, mtx_lock = 0x4}, nentries = 0x0, size = 0x0, timestamp = 0x85, needs_wakeup = 0x0, system_map = 0x0, flags = 0x0, root = 0x0, pmap = 0xfffff8007a7d9838, busy = 0x0} (kgdb) $ svn diff -x -p /usr/src/sys -- Test scenario: steeler.sh