GENERIC HEAD from 2013-10-10 19:56:31 UTC, r256297, vmcore.49 GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2013 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.0-CURRENT #1 r256297: Fri Oct 11 06:59:31 CEST 2013 pho@t2.osted.lan:/usr/src/sys/amd64/compile/PHO amd64 FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. CPU: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz (1995.23-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x206d7 Family = 0x6 Model = 0x2d Stepping = 7 Features=0xbfebfbff Features2=0x1fbee3ff AMD Features=0x2c100800 AMD Features2=0x1 TSC: P-state invariant, performance statistics real memory = 68719476736 (65536 MB) avail memory = 32153292800 (30663 MB) : Trying to mount root from ufs:/dev/da0p2 [rw]... Setting hostuuid: 2bde2bde-f4e2-e111-aab2-001e6756b69b. Setting hostid: 0x0035ff86. Starting ddb. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: /dev/da0p2: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/da0p2: clean, 75172029 free (156813 frags, 9376902 blocks, 0.2% fragmentation) /dev/da0p3: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/da0p3: clean, 23758591 free (2151 frags, 2969555 blocks, 0.0% fragmentation) Mounting local file systems:. lock order reversal: 1st 0xfffff80073924c98 ufs (ufs) @ kern/vfs_subr.c:2101 2nd 0xfffffe0785e31f88 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:262 3rd 0xfffff800830667b8 ufs (ufs) @ kern/vfs_subr.c:2101 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe081deeac90 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe081deead40 witness_checkorder() at witness_checkorder+0xd23/frame 0xfffffe081deeadd0 __lockmgr_args() at __lockmgr_args+0x86c/frame 0xfffffe081deeaf00 ffs_lock() at ffs_lock+0x92/frame 0xfffffe081deeaf50 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xf5/frame 0xfffffe081deeaf80 _vn_lock() at _vn_lock+0xc6/frame 0xfffffe081deeaff0 vget() at vget+0x70/frame 0xfffffe081deeb040 vfs_hash_get() at vfs_hash_get+0xf5/frame 0xfffffe081deeb090 ffs_vgetf() at ffs_vgetf+0x41/frame 0xfffffe081deeb120 softdep_sync_buf() at softdep_sync_buf+0x8fa/frame 0xfffffe081deeb1d0 ffs_syncvnode() at ffs_syncvnode+0x258/frame 0xfffffe081deeb250 ffs_truncate() at ffs_truncate+0x5f3/frame 0xfffffe081deeb430 ufs_direnter() at ufs_direnter+0x891/frame 0xfffffe081deeb4f0 ufs_makeinode() at ufs_makeinode+0x573/frame 0xfffffe081deeb6b0 VOP_CREATE_APV() at VOP_CREATE_APV+0x108/frame 0xfffffe081deeb6e0 vn_open_cred() at vn_open_cred+0x2f0/frame 0xfffffe081deeb830 kern_openat() at kern_openat+0x261/frame 0xfffffe081deeb9a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081deebab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081deebab0 --- syscall (5, FreeBSD ELF64, sys_open), rip = 0x800dd926a, rsp = 0x7fffffffc4f8, rbp = 0x7fffffffc5e0 --- Writing entropy file:. Setting hostname: t2.osted.lan. igb0: link state changed to UP Starting Network: lo0 igb0 igb1. lo0: flags=8049 metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 nd6 options=21 igb0: flags=8843 metric 0 mtu 1500 options=403bb ether 00:1e:67:56:b6:9b inet 192.168.1.109 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::21e:67ff:fe56:b69b%igb0 prefixlen 64 scopeid 0x1 nd6 options=29 media: Ethernet autoselect (100baseTX ) status: active igb1: flags=8c02 metric 0 mtu 1500 options=403bb ether 00:1e:67:56:b6:9c nd6 options=29 media: Ethernet autoselect status: no carrier Starting devd. Limiting icmp unreach response from 25521 to 200 packets/sec Starting Network: igb1. igb1: flags=8c02 metric 0 mtu 1500 options=403bb ether 00:1e:67:56:b6:9c nd6 options=29 media: Ethernet autoselect status: no carrier Limiting icmp unreach response from 26127 to 200 packets/sec Configuring keyboard: keymap. ums0: on usbus2 ums0: 3 buttons and [Z] coordinates ID=0 Starting ums0 moused. Expensive timeout(9) function: 0xffffffff806f7e40(0xffffffff816a95b0) 0.010745476 s add net default: gateway 192.168.1.1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Limiting icmp unreach response from 25729 to 200 packets/sec ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/nss 32-bit compatibility ldconfig path: /usr/lib32 Creating and/or trimming log files. Starting syslogd. Starting watchdogd. No core dumps found. Starting rpcbind. NFS access cache time=60 rpc.umntall: 127.0.0.1: MOUNTPROG: RPC: Program not registered Starting amd. lock order reversal: 1st 0xfffffe0785ee40e8 bufwait (bufwait) @ kern/vfs_bio.c:3050 2nd 0xfffff8018a56a800 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:284 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe081e02c2d0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe081e02c380 witness_checkorder() at witness_checkorder+0xd23/frame 0xfffffe081e02c410 _sx_xlock() at _sx_xlock+0x75/frame 0xfffffe081e02c450 ufsdirhash_add() at ufsdirhash_add+0x4c/frame 0xfffffe081e02c490 ufs_direnter() at ufs_direnter+0x688/frame 0xfffffe081e02c550 ufs_mkdir() at ufs_mkdir+0x863/frame 0xfffffe081e02c750 VOP_MKDIR_APV() at VOP_MKDIR_APV+0x10e/frame 0xfffffe081e02c780 kern_mkdirat() at kern_mkdirat+0x1f3/frame 0xfffffe081e02c9a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e02cab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e02cab0 --- syscall (136, FreeBSD ELF64, sys_mkdir), rip = 0x80093951a, rsp = 0x7fffffffd798, rbp = 0x7fffffffdc70 --- Clearing /tmp (X related). Starting nfsuserd. Starting mountd. Starting nfsd. Updating motd:. Mounting late file systems:. Starting ntpd. Starting powerd. Configuring syscons: keymap blanktime. Performing sanity check on sshd configuration. Starting sshd. Starting cron. Local package initialization:Oct 11 07:04:06 t2 su: pho to root on /dev/pts/0 backup. Starting default mousedmoused: unable to open /dev/psm0: No such file or directory . Fri Oct 11 07:04:11 CEST 2013 FreeBSD/amd64 (t2.osted.lan) (console) login: 20131011 07:04:14 all (1/1): udp.sh 20131011 07:04:16 all (1/1): udp.sh Expensive timeout(9) function: 0xffffffff8092df60(0) 0.018822117 s Expensive timeout(9) function: 0xffffffff80a68bb0(0xfffff801c5e21000) 0.076005082 s Expensive timeout(9) function: 0xffffffff8092df60(0) 2.209555534 s 20131011 07:15:57 all (1/1): udp.sh 20131011 07:16:00 all (1/1): udp.sh 20131011 07:28:33 all (1/1): udp.sh interrupt total irq3: uart1 480 irq9: acpi0 2 irq20: ehci1 6729 irq22: ehci0 3908 cpu0:timer 2234779 irq264: isci0 376868 irq266: hdac0 101 irq267: igb0:que 0 6858726 irq268: igb0:que 1 6712513 irq269: igb0:que 2 6734946 irq270: igb0:que 3 6859602 irq271: igb0:que 4 7013521 irq272: igb0:que 5 6766285 irq273: igb0:que 6 6580873 irq274: igb0:que 7 6666352 irq275: igb0:link 2 irq286: ahci0 219 irq287: hdac1 5 cpu1:timer 2176390 cpu9:timer 2042149 cpu12:timer 2029707 cpu8:timer 2089226 cpu17:timer 2062107 cpu11:timer 2067032 cpu20:timer 2094341 cpu10:timer 2052425 cpu21:timer 2064753 cpu22:timer 2113127 cpu4:timer 2194595 cpu18:timer 2128010 cpu19:timer 2037757 cpu2:timer 2169171 cpu13:timer 2128566 cpu5:timer 2181747 cpu14:timer 2089682 cpu15:timer 2079308 cpu23:timer 2066067 cpu7:timer 2178778 cpu6:timer 2168018 cpu16:timer 2092893 cpu3:timer 2167816 Total 105290122 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe081b81d5e0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe081b81d690 watchdog_fire() at watchdog_fire+0x8d/frame 0xfffffe081b81d6c0 hardclock_cnt() at hardclock_cnt+0x297/frame 0xfffffe081b81d730 handleevents() at handleevents+0xbe/frame 0xfffffe081b81d780 timercb() at timercb+0x208/frame 0xfffffe081b81d7e0 lapic_handle_timer() at lapic_handle_timer+0x9c/frame 0xfffffe081b81d810 Xtimerint() at Xtimerint+0x8c/frame 0xfffffe081b81d810 --- interrupt, rip = 0xffffffff808a73f2, rsp = 0xfffffe081b81d8d0, rbp = 0xfffffe081b81d950 --- __mtx_lock_sleep() at __mtx_lock_sleep+0x222/frame 0xfffffe081b81d950 __mtx_lock_flags() at __mtx_lock_flags+0xf5/frame 0xfffffe081b81d9a0 igb_msix_que() at igb_msix_que+0x8d/frame 0xfffffe081b81d9f0 intr_event_execute_handlers() at intr_event_execute_handlers+0x93/frame 0xfffffe081b81da30 ithread_loop() at ithread_loop+0xa6/frame 0xfffffe081b81da70 fork_exit() at fork_exit+0x84/frame 0xfffffe081b81dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081b81dab0 --- trap 0, rip = 0, rsp = 0xfffffe081b81db70, rbp = 0 --- KDB: enter: watchdog timeout [ thread pid 12 tid 100086 ] Stopped at kdb_enter+0x3e: movq $0,kdb_why db> run pho db:0:pho> bt Tracing pid 12 tid 100086 td 0xfffff8000c874920 kdb_enter() at kdb_enter+0x3e/frame 0xfffffe081b81d6c0 hardclock_cnt() at hardclock_cnt+0x297/frame 0xfffffe081b81d730 handleevents() at handleevents+0xbe/frame 0xfffffe081b81d780 timercb() at timercb+0x208/frame 0xfffffe081b81d7e0 lapic_handle_timer() at lapic_handle_timer+0x9c/frame 0xfffffe081b81d810 Xtimerint() at Xtimerint+0x8c/frame 0xfffffe081b81d810 --- interrupt, rip = 0xffffffff808a73f2, rsp = 0xfffffe081b81d8d0, rbp = 0xfffffe081b81d950 --- __mtx_lock_sleep() at __mtx_lock_sleep+0x222/frame 0xfffffe081b81d950 __mtx_lock_flags() at __mtx_lock_flags+0xf5/frame 0xfffffe081b81d9a0 igb_msix_que() at igb_msix_que+0x8d/frame 0xfffffe081b81d9f0 intr_event_execute_handlers() at intr_event_execute_handlers+0x93/frame 0xfffffe081b81da30 ithread_loop() at ithread_loop+0xa6/frame 0xfffffe081b81da70 fork_exit() at fork_exit+0x84/frame 0xfffffe081b81dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081b81dab0 --- trap 0, rip = 0, rsp = 0xfffffe081b81db70, rbp = 0 --- db:0:bt> show allpcpu Current CPU: 5 cpuid = 0 dynamic pcpu = 0x5bf100 curthread = 0xfffff8059550a000: pid 2276 "udp" curpcb = 0xfffffe081e204b80 fpcurthread = none idlethread = 0xfffff8000b212920: tid 100003 "idle: cpu0" curpmap = 0xfffff80795727bb8 tssp = 0xffffffff81713e10 commontssp = 0xffffffff81713e10 rsp0 = 0xfffffe081e204b80 gs32p = 0xffffffff81715868 ldt = 0xffffffff817158a8 tss = 0xffffffff81715898 spin locks held: cpuid = 1 dynamic pcpu = 0xfffffe087aef9100 curthread = 0xfffff8000c86f920: pid 12 "irq268: igb0:que 1" curpcb = 0xfffffe00003f4b80 fpcurthread = none idlethread = 0xfffff8000b212490: tid 100004 "idle: cpu1" curpmap = 0xffffffff816fc888 tssp = 0xffffffff81713e78 commontssp = 0xffffffff81713e78 rsp0 = 0xfffffe00003f4b80 gs32p = 0xffffffff817158d0 ldt = 0xffffffff81715910 tss = 0xffffffff81715900 spin locks held: cpuid = 2 dynamic pcpu = 0xfffffe087af01100 curthread = 0xfffff8000c86f000: pid 12 "irq269: igb0:que 2" curpcb = 0xfffffe00003feb80 fpcurthread = none idlethread = 0xfffff8000b212000: tid 100005 "idle: cpu2" curpmap = 0xffffffff816fc888 tssp = 0xffffffff81713ee0 commontssp = 0xffffffff81713ee0 rsp0 = 0xfffffe00003feb80 gs32p = 0xffffffff81715938 ldt = 0xffffffff81715978 tss = 0xffffffff81715968 spin locks held: cpuid = 3 dynamic pcpu = 0xfffffe087af09100 curthread = 0xfffff8000c871920: pid 12 "irq270: igb0:que 3" curpcb = 0xfffffe081b809b80 fpcurthread = none idlethread = 0xfffff8000b229000: tid 100006 "idle: cpu3" curpmap = 0xffffffff816fc888 tssp = 0xffffffff81713f48 commontssp = 0xffffffff81713f48 rsp0 = 0xfffffe081b809b80 gs32p = 0xffffffff817159a0 ldt = 0xffffffff817159e0 tss = 0xffffffff817159d0 spin locks held: cpuid = 4 dynamic pcpu = 0xfffffe087af11100 curthread = 0xfffff8000c871000: pid 12 "irq271: igb0:que 4" curpcb = 0xfffffe081b813b80 fpcurthread = none idlethread = 0xfffff8000b228920: tid 100007 "idle: cpu4" curpmap = 0xffffffff816fc888 tssp = 0xffffffff81713fb0 commontssp = 0xffffffff81713fb0 rsp0 = 0xfffffe081b813b80 gs32p = 0xffffffff81715a08 ldt = 0xffffffff81715a48 tss = 0xffffffff81715a38 spin locks held: cpuid = 5 dynamic pcpu = 0xfffffe087af19100 curthread = 0xfffff8000c874920: pid 12 "irq272: igb0:que 5" curpcb = 0xfffffe081b81db80 fpcurthread = none idlethread = 0xfffff8000b228490: tid 100008 "idle: cpu5" curpmap = 0xffffffff816fc888 tssp = 0xffffffff81714018 commontssp = 0xffffffff81714018 rsp0 = 0xfffffe081b81db80 gs32p = 0xffffffff81715a70 ldt = 0xffffffff81715ab0 tss = 0xffffffff81715aa0 spin locks held: cpuid = 6 dynamic pcpu = 0xfffffe087af21100 curthread = 0xfffff8007309f490: pid 5 "pagedaemon" curpcb = 0xfffffe081dbd8b80 fpcurthread = none idlethread = 0xfffff8000b228000: tid 100009 "idle: cpu6" curpmap = 0xffffffff816fc888 tssp = 0xffffffff81714080 commontssp = 0xffffffff81714080 rsp0 = 0xfffffe081dbd8b80 gs32p = 0xffffffff81715ad8 ldt = 0xffffffff81715b18 tss = 0xffffffff81715b08 spin locks held: cpuid = 7 dynamic pcpu = 0xfffffe087af29100 curthread = 0xfffff8000c872490: pid 12 "irq274: igb0:que 7" curpcb = 0xfffffe081b831b80 fpcurthread = none idlethread = 0xfffff8000b218920: tid 100010 "idle: cpu7" curpmap = 0xffffffff816fc888 tssp = 0xffffffff817140e8 commontssp = 0xffffffff817140e8 rsp0 = 0xfffffe081b831b80 gs32p = 0xffffffff81715b40 ldt = 0xffffffff81715b80 tss = 0xffffffff81715b70 spin locks held: cpuid = 8 dynamic pcpu = 0xfffffe087af31100 curthread = 0xfffff80073962920: pid 2260 "udp" curpcb = 0xfffffe081df95b80 fpcurthread = none idlethread = 0xfffff8000b22a920: tid 100011 "idle: cpu8" curpmap = 0xfffff80599ce42f8 tssp = 0xffffffff81714150 commontssp = 0xffffffff81714150 rsp0 = 0xfffffe081df95b80 gs32p = 0xffffffff81715ba8 ldt = 0xffffffff81715be8 tss = 0xffffffff81715bd8 spin locks held: cpuid = 9 dynamic pcpu = 0xfffffe087af39100 curthread = 0xfffff8059512c000: pid 2266 "udp" curpcb = 0xfffffe081e295b80 fpcurthread = 0xfffff8059512c000: pid 2266 "udp" idlethread = 0xfffff8000b22a490: tid 100012 "idle: cpu9" curpmap = 0xfffff80073c1fd78 tssp = 0xffffffff817141b8 commontssp = 0xffffffff817141b8 rsp0 = 0xfffffe081e295b80 gs32p = 0xffffffff81715c10 ldt = 0xffffffff81715c50 tss = 0xffffffff81715c40 spin locks held: cpuid = 10 dynamic pcpu = 0xfffffe087af41100 curthread = 0xfffff800832fd920: pid 1374 "top" curpcb = 0xfffffe081dff7b80 fpcurthread = none idlethread = 0xfffff8000b22a000: tid 100013 "idle: cpu10" curpmap = 0xfffff8000b220bb8 tssp = 0xffffffff81714220 commontssp = 0xffffffff81714220 rsp0 = 0xfffffe081dff7b80 gs32p = 0xffffffff81715c78 ldt = 0xffffffff81715cb8 tss = 0xffffffff81715ca8 spin locks held: cpuid = 11 dynamic pcpu = 0xfffffe087af49100 curthread = 0xfffff8018a132920: pid 2261 "swap" curpcb = 0xfffffe081e167b80 fpcurthread = none idlethread = 0xfffff8000b229920: tid 100014 "idle: cpu11" curpmap = 0xfffff80073bbf138 tssp = 0xffffffff81714288 commontssp = 0xffffffff81714288 rsp0 = 0xfffffe081e167b80 gs32p = 0xffffffff81715ce0 ldt = 0xffffffff81715d20 tss = 0xffffffff81715d10 spin locks held: cpuid = 12 dynamic pcpu = 0xfffffe087af51100 curthread = 0xfffff80073962000: pid 2275 "udp" curpcb = 0xfffffe081dedcb80 fpcurthread = none idlethread = 0xfffff8000b229490: tid 100015 "idle: cpu12" curpmap = 0xfffff8008326f678 tssp = 0xffffffff817142f0 commontssp = 0xffffffff817142f0 rsp0 = 0xfffffe081dedcb80 gs32p = 0xffffffff81715d48 ldt = 0xffffffff81715d88 tss = 0xffffffff81715d78 spin locks held: cpuid = 13 dynamic pcpu = 0xfffffe087af59100 curthread = 0xfffff800738ad000: pid 2274 "swap" curpcb = 0xfffffe081df90b80 fpcurthread = none idlethread = 0xfffff8000b22c490: tid 100016 "idle: cpu13" curpmap = 0xfffff80703a059f8 tssp = 0xffffffff81714358 commontssp = 0xffffffff81714358 rsp0 = 0xfffffe081df90b80 gs32p = 0xffffffff81715db0 ldt = 0xffffffff81715df0 tss = 0xffffffff81715de0 spin locks held: cpuid = 14 dynamic pcpu = 0xfffffe087af61100 curthread = 0xfffff8018a220490: pid 2262 "udp" curpcb = 0xfffffe081e29fb80 fpcurthread = 0xfffff8018a220490: pid 2262 "udp" idlethread = 0xfffff8000b22c000: tid 100017 "idle: cpu14" curpmap = 0xfffff8008326fbb8 tssp = 0xffffffff817143c0 commontssp = 0xffffffff817143c0 rsp0 = 0xfffffe081e29fb80 gs32p = 0xffffffff81715e18 ldt = 0xffffffff81715e58 tss = 0xffffffff81715e48 spin locks held: cpuid = 15 dynamic pcpu = 0xfffffe087af69100 curthread = 0xfffff801c5d7e920: pid 2269 "swap" curpcb = 0xfffffe081e126b80 fpcurthread = none idlethread = 0xfffff8000b22b920: tid 100018 "idle: cpu15" curpmap = 0xfffff807382914b8 tssp = 0xffffffff81714428 commontssp = 0xffffffff81714428 rsp0 = 0xfffffe081e126b80 gs32p = 0xffffffff81715e80 ldt = 0xffffffff81715ec0 tss = 0xffffffff81715eb0 spin locks held: cpuid = 16 dynamic pcpu = 0xfffffe087af71100 curthread = 0xfffff8011cc4a000: pid 2258 "udp" curpcb = 0xfffffe081dfa9b80 fpcurthread = none idlethread = 0xfffff8000b22b490: tid 100019 "idle: cpu16" curpmap = 0xfffff801c5dbf138 tssp = 0xffffffff81714490 commontssp = 0xffffffff81714490 rsp0 = 0xfffffe081dfa9b80 gs32p = 0xffffffff81715ee8 ldt = 0xffffffff81715f28 tss = 0xffffffff81715f18 spin locks held: cpuid = 17 dynamic pcpu = 0xfffffe087af79100 curthread = 0xfffff801c5f76000: pid 2267 "swap" curpcb = 0xfffffe081e1cbb80 fpcurthread = none idlethread = 0xfffff8000b22b000: tid 100020 "idle: cpu17" curpmap = 0xfffff80073aa7838 tssp = 0xffffffff817144f8 commontssp = 0xffffffff817144f8 rsp0 = 0xfffffe081e1cbb80 gs32p = 0xffffffff81715f50 ldt = 0xffffffff81715f90 tss = 0xffffffff81715f80 spin locks held: cpuid = 18 dynamic pcpu = 0xfffffe087af81100 curthread = 0xfffff8018a80d000: pid 2251 "swap" curpcb = 0xfffffe081e077b80 fpcurthread = none idlethread = 0xfffff8000b230000: tid 100021 "idle: cpu18" curpmap = 0xfffff802ec412d78 tssp = 0xffffffff81714560 commontssp = 0xffffffff81714560 rsp0 = 0xfffffe081e077b80 gs32p = 0xffffffff81715fb8 ldt = 0xffffffff81715ff8 tss = 0xffffffff81715fe8 spin locks held: cpuid = 19 dynamic pcpu = 0xfffffe087af89100 curthread = 0xfffff8000c87a490: pid 0 "igb0 que" curpcb = 0xfffffe081b836b80 fpcurthread = none idlethread = 0xfffff8000b22f920: tid 100022 "idle: cpu19" curpmap = 0xffffffff816fc888 tssp = 0xffffffff817145c8 commontssp = 0xffffffff817145c8 rsp0 = 0xfffffe081b836b80 gs32p = 0xffffffff81716020 ldt = 0xffffffff81716060 tss = 0xffffffff81716050 spin locks held: cpuid = 20 dynamic pcpu = 0xfffffe087af91100 curthread = 0xfffff80073b15000: pid 2244 "udp" curpcb = 0xfffffe081deebb80 fpcurthread = none idlethread = 0xfffff8000b22f490: tid 100023 "idle: cpu20" curpmap = 0xfffff80073969bb8 tssp = 0xffffffff81714630 commontssp = 0xffffffff81714630 rsp0 = 0xfffffe081deebb80 gs32p = 0xffffffff81716088 ldt = 0xffffffff817160c8 tss = 0xffffffff817160b8 spin locks held: cpuid = 21 dynamic pcpu = 0xfffffe087af99100 curthread = 0xfffff8018a132490: pid 2252 "udp" curpcb = 0xfffffe081e317b80 fpcurthread = none idlethread = 0xfffff8000b22f000: tid 100024 "idle: cpu21" curpmap = 0xfffff801e0b80138 tssp = 0xffffffff81714698 commontssp = 0xffffffff81714698 rsp0 = 0xfffffe081e317b80 gs32p = 0xffffffff817160f0 ldt = 0xffffffff81716130 tss = 0xffffffff81716120 spin locks held: cpuid = 22 dynamic pcpu = 0xfffffe087afa1100 curthread = 0xfffff801c5d4f000: pid 2254 "udp" curpcb = 0xfffffe081e2ccb80 fpcurthread = none idlethread = 0xfffff8000b22c920: tid 100025 "idle: cpu22" curpmap = 0xfffff80674529678 tssp = 0xffffffff81714700 commontssp = 0xffffffff81714700 rsp0 = 0xfffffe081e2ccb80 gs32p = 0xffffffff81716158 ldt = 0xffffffff81716198 tss = 0xffffffff81716188 spin locks held: cpuid = 23 dynamic pcpu = 0xfffffe08048b0100 curthread = 0xfffff8018a0e9490: pid 2247 "swap" curpcb = 0xfffffe081dfedb80 fpcurthread = none idlethread = 0xfffff8000b231920: tid 100026 "idle: cpu23" curpmap = 0xfffff800832c72f8 tssp = 0xffffffff81714768 commontssp = 0xffffffff81714768 rsp0 = 0xfffffe081dfedb80 gs32p = 0xffffffff817161c0 ldt = 0xffffffff81716200 tss = 0xffffffff817161f0 spin locks held: db:0:allpcpu> show alllocks Process 2279 (udp) thread 0xfffff800831d7000 (100179) shared rw udpinp (udpinp) r = 0 (0xfffff80083345da8) locked @ netinet/udp_usrreq.c:981 Process 2275 (udp) thread 0xfffff80073962000 (100136) exclusive sleep mutex pmap (pmap) r = 0 (0xfffff8008326f678) locked @ amd64/amd64/pmap.c:5818 shared sx vm map (user) (vm map (user)) r = 0 (0xfffff8008326f5c0) locked @ vm/vm_map.c:2039 Process 2273 (udp) thread 0xfffff8011ccdc920 (100181) shared rw udpinp (udpinp) r = 0 (0xfffff8011c63cc20) locked @ netinet/udp_usrreq.c:981 Process 2268 (udp) thread 0xfffff8018a69e000 (100246) exclusive sleep mutex pmap (pmap) r = 0 (0xfffff80073b469f8) locked @ amd64/amd64/pmap.c:5818 shared sx vm map (user) (vm map (user)) r = 0 (0xfffff80073b46940) locked @ vm/vm_map.c:2039 Process 2266 (udp) thread 0xfffff8059512c000 (100323) exclusive sleep mutex pmap (pmap) r = 0 (0xfffff80073c1fd78) locked @ amd64/amd64/pmap.c:5818 shared sx vm map (user) (vm map (user)) r = 0 (0xfffff80073c1fcc0) locked @ vm/vm_map.c:2039 Process 2264 (udp) thread 0xfffff80073f0d920 (100348) exclusive sleep mutex pmap (pmap) r = 0 (0xfffff80073fc1d78) locked @ amd64/amd64/pmap.c:5818 shared sx vm map (user) (vm map (user)) r = 0 (0xfffff80073fc1cc0) locked @ vm/vm_map.c:2039 Process 2262 (udp) thread 0xfffff8018a220490 (100325) shared rw udpinp (udpinp) r = 0 (0xfffff8008327bda8) locked @ netinet/udp_usrreq.c:981 Process 2260 (udp) thread 0xfffff80073962920 (100173) exclusive sleep mutex igb0:tx(0) (igb0:tx(0)) r = 0 (0xfffff8000c841008) locked @ dev/e1000/if_igb.c:990 shared rw udpinp (udpinp) r = 0 (0xfffff8011c63c478) locked @ netinet/udp_usrreq.c:981 Process 2258 (udp) thread 0xfffff8011cc4a000 (100177) shared rw udpinp (udpinp) r = 0 (0xfffff8011ce36f30) locked @ netinet/udp_usrreq.c:981 Process 2256 (udp) thread 0xfffff80073f49000 (100150) shared rw udpinp (udpinp) r = 0 (0xfffff80083314a98) locked @ netinet/udp_usrreq.c:981 Process 2254 (udp) thread 0xfffff801c5d4f000 (100334) exclusive sleep mutex igb0:tx(6) (igb0:tx(6)) r = 0 (0xfffff8000c841608) locked @ dev/e1000/if_igb.c:990 shared rw udpinp (udpinp) r = 0 (0xfffff8018a709c20) locked @ netinet/udp_usrreq.c:981 Process 2252 (udp) thread 0xfffff8018a132490 (100349) exclusive sleep mutex igb0:tx(5) (igb0:tx(5)) r = 0 (0xfffff8000c841508) locked @ dev/e1000/if_igb.c:990 shared rw udpinp (udpinp) r = 0 (0xfffff805ea9a0a98) locked @ netinet/udp_usrreq.c:981 Process 2248 (udp) thread 0xfffff8018ad55920 (100292) exclusive sleep mutex pmap (pmap) r = 0 (0xfffff80674529138) locked @ amd64/amd64/pmap.c:5818 shared sx vm map (user) (vm map (user)) r = 0 (0xfffff80674529080) locked @ vm/vm_map.c:2039 Process 1374 (top) thread 0xfffff800832fd920 (100190) shared sx vm map (user) (vm map (user)) r = 0 (0xfffff8000b220b00) locked @ vm/vm_map.c:2039 Process 823 (syslogd) thread 0xfffff80083132920 (100191) exclusive lockmgr ufs (ufs) r = 0 (0xfffff8008321e7b8) locked @ kern/vfs_vnops.c:807 Process 5 (pagedaemon) thread 0xfffff8007309f490 (100128) exclusive sleep mutex pmap (pmap) r = 0 (0xfffff8000b220bb8) locked @ amd64/amd64/pmap.c:5722 exclusive rw pmap pv list (pmap pv list) r = 0 (0xffffffff81696f20) locked @ amd64/amd64/pmap.c:5616 shared rw pmap pv global (pmap pv global) r = 0 (0xffffffff81696600) locked @ amd64/amd64/pmap.c:5615 exclusive rw vm object (vm object) r = 0 (0xfffff801b7829500) locked @ vm/vm_pageout.c:997 exclusive sleep mutex vm page (vm page) r = 0 (0xffffffff8170d380) locked @ vm/vm_pageout.c:305 Process 12 (intr) thread 0xfffff8000c872490 (100090) exclusive sleep mutex igb0:tx(7) (igb0:tx(7)) r = 0 (0xfffff8000c841708) locked @ dev/e1000/if_igb.c:1608 Process 12 (intr) thread 0xfffff8000c871000 (100084) exclusive sleep mutex igb0:rx(7) (igb0:rx(7)) r = 0 (0xfffff8000c85c4c0) locked @ dev/e1000/if_igb.c:4799 Process 12 (intr) thread 0xfffff8000c871920 (100082) exclusive sleep mutex igb0:rx(7) (igb0:rx(7)) r = 0 (0xfffff8000c85c3b0) locked @ dev/e1000/if_igb.c:4799 Process 12 (intr) thread 0xfffff8000c86f000 (100080) exclusive sleep mutex igb0:tx(2) (igb0:tx(2)) r = 0 (0xfffff8000c841208) locked @ dev/e1000/if_igb.c:1608 Process 12 (intr) thread 0xfffff8000c86f920 (100078) exclusive sleep mutex igb0:tx(1) (igb0:tx(1)) r = 0 (0xfffff8000c841108) locked @ dev/e1000/if_igb.c:1608 db:0:alllocks> show lockedvnods Locked vnodes 0xfffff8008321e750: tag ufs, type VREG usecount 1, writecount 1, refcount 8 mountedhere 0 flags (VI_ACTIVE) v_object 0xfffff80073987b00 ref 0 pages 44 cleanbuf 0 dirtybuf 6 lock type ufs: EXCL by thread 0xfffff80083132920 (pid 823, syslogd, tid 100191) #0 0xffffffff8089ee57 at __lockmgr_args+0xb57 #1 0xffffffff80b10442 at ffs_lock+0x92 #2 0xffffffff80dcd5b5 at VOP_LOCK1_APV+0xf5 #3 0xffffffff80973bd6 at _vn_lock+0xc6 #4 0xffffffff80975eea at vn_write+0x1da #5 0xffffffff8097244a at vn_io_fault+0x22a #6 0xffffffff80912a15 at dofilewrite+0x85 #7 0xffffffff80912725 at kern_writev+0x65 #8 0xffffffff80912966 at sys_writev+0x36 #9 0xffffffff80cc1402 at amd64_syscall+0x282 #10 0xffffffff80ca139b at Xfast_syscall+0xfb ino 43980382, on dev da0p2 db:0:lockedvnods> show mount 0xfffff800735f4b58 /dev/da0p2 on / (ufs) 0xfffff800735f5000 devfs on /dev (devfs) 0xfffff800735f4790 /dev/da0p3 on /tmp (ufs) 0xfffff800830c43c8 procfs on /proc (procfs) 0xfffff8018a2a1b58 pid849@t2:/net on /net (nfs) 0xfffff8018a4ef790 pid849@t2:/host on /host (nfs) More info: show mount db:0:mount> ps pid ppid pgrp uid state wmesg wchan cmd 2279 2243 2120 1004 R+ udp 2278 2243 2120 1004 R+ udp 2277 2243 2120 1004 R+ udp 2276 2243 2120 1004 R+ CPU 0 udp 2275 2243 2120 1004 R+ CPU 12 udp 2274 2245 2120 1004 R+ CPU 13 swap 2273 2243 2120 1004 R+ udp 2272 2245 2120 1004 DL+ pfault 0xffffffff8170b118 swap 2271 2243 2120 1004 R+ udp 2270 2245 2120 1004 R+ swap 2269 2245 2120 1004 R+ CPU 15 swap 2268 2243 2120 1004 R+ udp 2267 2245 2120 1004 R+ CPU 17 swap 2266 2243 2120 1004 R+ CPU 9 udp 2265 2245 2120 1004 R+ swap 2264 2243 2120 1004 R+ udp 2263 2245 2120 1004 DL+ pfault 0xffffffff8170b118 swap 2262 2243 2120 1004 R+ CPU 14 udp 2261 2245 2120 1004 R+ CPU 11 swap 2260 2243 2120 1004 R+ CPU 8 udp 2259 2245 2120 1004 DL+ pfault 0xffffffff8170b118 swap 2258 2243 2120 1004 R+ CPU 16 udp 2257 2245 2120 1004 DL+ pfault 0xffffffff8170b118 swap 2256 2243 2120 1004 R+ udp 2255 2245 2120 1004 DL+ pfault 0xffffffff8170b118 swap 2254 2243 2120 1004 R+ CPU 22 udp 2253 2245 2120 1004 DL+ pfault 0xffffffff8170b118 swap 2252 2243 2120 1004 R+ CPU 21 udp 2251 2245 2120 1004 R+ CPU 18 swap 2250 2243 2120 1004 R+ udp 2249 2245 2120 1004 DL+ pfault 0xffffffff8170b118 swap 2248 2243 2120 1004 R+ udp 2247 2245 2120 1004 R+ CPU 23 swap 2246 2243 2120 1004 R+ udp 2245 2241 2120 1004 SW+ wait 0xfffff80083272970 swap 2244 2243 2120 1004 R+ CPU 20 udp 2243 2242 2120 1004 SW+ wait 0xfffff801c5d384b8 udp 2242 2122 2120 1004 S+ nanslp 0xffffffff81498ed3 udp 2241 2122 2120 1004 S+ nanslp 0xffffffff81498eca swap 2122 2121 2120 1004 SW+ wait 0xfffff8018a6ae4b8 run 2121 2120 2120 1004 SW+ wait 0xfffff8018a80c970 run 2120 2119 2120 1004 S+ nanslp 0xffffffff81498ecb run 2119 2113 1073 0 SW+ wait 0xfffff801c5da5000 su 2113 1073 1073 0 SW+ wait 0xfffff801c80f8000 sh 1374 1318 1374 1001 R+ CPU 10 top 1318 1317 1318 1001 SWs+ wait 0xfffff801c5d4e4b8 bash 1317 1313 1313 1001 S select 0xfffff8011cc9b940 sshd 1313 961 1313 0 Ss select 0xfffff8011c94d8c0 sshd 1073 1000 1073 0 SW+ wait 0xfffff80073f3a000 sh 1072 1 1072 0 Ss+ ttyin 0xfffff8000b323ca8 getty 1071 1 1071 0 Ss+ ttyin 0xfffff8000b3240a8 getty 1070 1 1070 0 Ss+ ttyin 0xfffff8000b3244a8 getty 1069 1 1069 0 Ss+ ttyin 0xfffff8000b3248a8 getty 1068 1 1068 0 Ss+ ttyin 0xfffff8000b324ca8 getty 1067 1 1067 0 Ss+ ttyin 0xfffff8000b3250a8 getty 1066 1 1066 0 Ss+ ttyin 0xfffff8000b3254a8 getty 1065 1 1065 0 Ss+ ttyin 0xfffff8000b3258a8 getty 1064 1 1064 0 Ss+ ttyin 0xfffff8000b325ca8 getty 1000 998 1000 0 SW+ wait 0xfffff800831d5000 bash 998 997 998 0 SW+ pause 0xfffff8007389b560 csh 997 992 997 1001 SW+ wait 0xfffff80073c84000 su 992 991 992 1001 SWs+ wait 0xfffff80073dbf970 bash 991 989 989 1001 S select 0xfffff800830036c0 sshd 989 961 989 0 Ss select 0xfffff8011cc9b9c0 sshd 971 1 971 0 ?Ws cron 967 1 967 25 SWs pause 0xfffff800830870a8 sendmail 964 1 964 0 Ss select 0xfffff80005ce4440 sendmail 961 1 961 0 Ss select 0xfffff8018a2fbac0 sshd 932 1 932 0 Ss select 0xfffff80073491340 powerd 929 1 929 0 DLs pfault 0xffffffff8170b118 ntpd 892 889 889 0 S (threaded) nfsd 100208 S rpcsvc 0xfffff8007312e4a0 nfsd: service 100207 S rpcsvc 0xfffff8007312e520 nfsd: service 100206 S rpcsvc 0xfffff8007312e5a0 nfsd: service 100183 S rpcsvc 0xfffff8018a7792a0 nfsd: master 889 1 889 0 Ss select 0xfffff80005ce45c0 nfsd 883 1 883 0 Ss select 0xfffff80073cb15c0 mountd 876 872 872 0 S select 0xfffff80073d0d440 nfsuserd 875 872 872 0 S select 0xfffff8011c94d940 nfsuserd 874 872 872 0 S select 0xfffff8008324f740 nfsuserd 873 872 872 0 S select 0xfffff80073e7c640 nfsuserd 872 1 872 0 SWs pause 0xfffff80083087a18 nfsuserd 849 1 849 0 Ss select 0xfffff80005ce4640 amd 838 1 838 0 Ss select 0xfffff80083003740 rpcbind 823 1 823 0 Ds getblk 0xfffffe0786213368 syslogd 691 1 691 0 Ss select 0xfffff80073cf64c0 devd 674 1 674 0 Ss select 0xfffff8011cf8eb40 moused 17 0 0 0 DL sdflush 0xffffffff8168ef8c [softdepflush] 16 0 0 0 DL syncer 0xffffffff816882b0 [syncer] 9 0 0 0 DL vlruwt 0xfffff8000b2d84b8 [vnlru] 8 0 0 0 DL psleep 0xffffffff81687c84 [bufdaemon] 7 0 0 0 DL pgzero 0xffffffff8169058c [pagezero] 6 0 0 0 DL psleep 0xffffffff8168fcc0 [vmdaemon] 5 0 0 0 RL CPU 6 [pagedaemon] 4 0 0 0 DL idle 0xfffff8000b211970 [enc_daemon0] 3 0 0 0 DL ccb_scan 0xffffffff8145b030 [xpt_thrd] 2 0 0 0 DL waiting_ 0xffffffff81707478 [sctp_iterator] 15 0 0 0 DL (threaded) [usb] 100119 D - 0xfffffe0001177e18 [usbus2] 100118 D - 0xfffffe0001177dc0 [usbus2] 100117 D - 0xfffffe0001177d68 [usbus2] 100116 D - 0xfffffe0001177d10 [usbus2] 100114 D - 0xfffffe00011701b0 [usbus1] 100113 D - 0xfffffe0001170158 [usbus1] 100112 D - 0xfffffe0001170100 [usbus1] 100111 D - 0xfffffe00011700a8 [usbus1] 100074 D - 0xfffffe0000ebde18 [usbus0] 100073 D - 0xfffffe0000ebddc0 [usbus0] 100072 D - 0xfffffe0000ebdd68 [usbus0] 100071 D - 0xfffffe0000ebdd10 [usbus0] 14 0 0 0 DL - 0xffffffff814741a0 [rand_harvestq] 13 0 0 0 DL (threaded) [geom] 100055 D - 0xffffffff816fbd20 [g_down] 100054 D - 0xffffffff816fbd18 [g_up] 100053 D - 0xffffffff816fbd10 [g_event] 12 0 0 0 RL (threaded) [intr] 100122 I [swi0: uart] 100121 I [irq287: hdac1] 100120 I [irq286: ahci0] 100115 I [irq20: ehci1] 100110 I [irq285: xhci0] 100109 I [irq284: igb1:link] 100107 I [irq283: igb1:que 7] 100105 I [irq282: igb1:que 6] 100103 I [irq281: igb1:que 5] 100101 I [irq280: igb1:que 4] 100099 I [irq279: igb1:que 3] 100097 I [irq278: igb1:que 2] 100095 I [irq277: igb1:que 1] 100093 I [irq276: igb1:que 0] 100092 I [irq275: igb0:link] 100090 Run CPU 7 [irq274: igb0:que 7] 100088 RunQ [irq273: igb0:que 6] 100086 Run CPU 5 [irq272: igb0:que 5] 100084 Run CPU 4 [irq271: igb0:que 4] 100082 Run CPU 3 [irq270: igb0:que 3] 100080 Run CPU 2 [irq269: igb0:que 2] 100078 Run CPU 1 [irq268: igb0:que 1] 100076 I [irq267: igb0:que 0] 100075 I [irq266: hdac0] 100070 I [irq22: ehci0] 100069 I [irq265: isci0] 100068 I [irq264: isci0] 100066 I [swi6: Giant taskq] 100065 I [swi6: task queue] 100060 I [swi2: cambio] 100059 I [swi5: fast taskq] 100052 I [swi4: clock] 100051 I [swi4: clock] 100050 I [swi4: clock] 100049 I [swi4: clock] 100048 I [swi4: clock] 100047 I [swi4: clock] 100046 I [swi4: clock] 100045 I [swi4: clock] 100044 I [swi4: clock] 100043 I [swi4: clock] 100042 I [swi4: clock] 100041 I [swi4: clock] 100040 I [swi4: clock] 100039 I [swi4: clock] 100038 I [swi4: clock] 100037 I [swi4: clock] 100036 I [swi4: clock] 100035 I [swi4: clock] 100034 I [swi4: clock] 100033 I [swi4: clock] 100032 I [swi4: clock] 100031 I [swi4: clock] 100030 I [swi4: clock] 100029 I [swi4: clock] 100028 I [swi3: vm] 100027 I [swi1: netisr 0] 11 0 0 0 RL (threaded) [idle] 100026 CanRun [idle: cpu23] 100025 CanRun [idle: cpu22] 100024 CanRun [idle: cpu21] 100023 CanRun [idle: cpu20] 100022 CanRun [idle: cpu19] 100021 CanRun [idle: cpu18] 100020 CanRun [idle: cpu17] 100019 CanRun [idle: cpu16] 100018 CanRun [idle: cpu15] 100017 CanRun [idle: cpu14] 100016 CanRun [idle: cpu13] 100015 CanRun [idle: cpu12] 100014 CanRun [idle: cpu11] 100013 CanRun [idle: cpu10] 100012 CanRun [idle: cpu9] 100011 CanRun [idle: cpu8] 100010 CanRun [idle: cpu7] 100009 CanRun [idle: cpu6] 100008 CanRun [idle: cpu5] 100007 CanRun [idle: cpu4] 100006 CanRun [idle: cpu3] 100005 CanRun [idle: cpu2] 100004 CanRun [idle: cpu1] 100003 CanRun [idle: cpu0] 1 0 1 0 SLs wait 0xfffff8000b211000 [init] 10 0 0 0 DL audit_wo 0xffffffff8170abe8 [audit] 0 0 0 0 RLs (threaded) [kernel] 100124 D - 0xffffffff81497d2a [deadlkres] 100123 D - 0xfffff80010091300 [mca taskq] 100108 D - 0xfffff8000c899200 [igb1 que] 100106 D - 0xfffff8000c899300 [igb1 que] 100104 D - 0xfffff8000c899400 [igb1 que] 100102 D - 0xfffff8000c7d8300 [igb1 que] 100100 D - 0xfffff8000c7d8400 [igb1 que] 100098 D - 0xfffff8000c7d8500 [igb1 que] 100096 D - 0xfffff8000c7d8600 [igb1 que] 100094 D - 0xfffff8000c7d8700 [igb1 que] 100091 Run CPU 19 [igb0 que] 100089 L *igb0:tx( 0xfffff8008312d780 [igb0 que] 100087 L *igb0:tx( 0xfffff8001000f540 [igb0 que] 100085 D - 0xfffff8000c7d8b00 [igb0 que] 100083 D - 0xfffff8000c7d8c00 [igb0 que] 100081 D - 0xfffff8000c7d8d00 [igb0 que] 100079 D - 0xfffff8000c7d8e00 [igb0 que] 100077 L *igb0:tx( 0xfffff8018a771cc0 [igb0 que] 100067 D - 0xfffff8000b1f1d00 [thread taskq] 100064 D - 0xfffff8000b1f2100 [acpi_task_2] 100063 D - 0xfffff8000b1f2100 [acpi_task_1] 100062 D - 0xfffff8000b1f2100 [acpi_task_0] 100061 D - 0xfffff8000b1f2200 [kqueue taskq] 100058 D - 0xfffff8000b1f2500 [ffs_trim taskq] 100056 D - 0xfffff8000b1f2a00 [firmware taskq] 100000 D vmwait 0xffffffff8170b118 [swapper] db:0:ps> allt Tracing command udp pid 2279 tid 100179 td 0xfffff800831d7000 sched_switch() at sched_switch+0x482/frame 0xfffffe081dfb3480 mi_switch() at mi_switch+0x179/frame 0xfffffe081dfb34c0 critical_exit() at critical_exit+0x8b/frame 0xfffffe081dfb34e0 spinlock_exit() at spinlock_exit+0x14/frame 0xfffffe081dfb3500 __mtx_unlock_flags() at __mtx_unlock_flags+0xf0/frame 0xfffffe081dfb3540 taskqueue_enqueue() at taskqueue_enqueue+0x84/frame 0xfffffe081dfb3570 igb_mq_start() at igb_mq_start+0x19e/frame 0xfffffe081dfb35d0 ether_output() at ether_output+0x5c8/frame 0xfffffe081dfb3640 ip_output() at ip_output+0xf1b/frame 0xfffffe081dfb3730 udp_send() at udp_send+0x866/frame 0xfffffe081dfb37f0 sosend_dgram() at sosend_dgram+0x36b/frame 0xfffffe081dfb3870 soo_write() at soo_write+0x49/frame 0xfffffe081dfb38b0 dofilewrite() at dofilewrite+0x85/frame 0xfffffe081dfb3900 kern_writev() at kern_writev+0x65/frame 0xfffffe081dfb3950 sys_write() at sys_write+0x63/frame 0xfffffe081dfb39a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081dfb3ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081dfb3ab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x800b5da0a, rsp = 0x7fffffffd658, rbp = 0x7fffffffd6a0 --- Tracing command udp pid 2278 tid 100324 td 0xfffff8059512a920 sched_switch() at sched_switch+0x482/frame 0xfffffe081e29a490 mi_switch() at mi_switch+0x179/frame 0xfffffe081e29a4d0 turnstile_wait() at turnstile_wait+0x3af/frame 0xfffffe081e29a520 __mtx_lock_sleep() at __mtx_lock_sleep+0x2a9/frame 0xfffffe081e29a5a0 __mtx_lock_flags() at __mtx_lock_flags+0xf5/frame 0xfffffe081e29a5f0 zone_fetch_slab() at zone_fetch_slab+0x44/frame 0xfffffe081e29a620 zone_import() at zone_import+0x42/frame 0xfffffe081e29a690 zone_alloc_item() at zone_alloc_item+0x38/frame 0xfffffe081e29a6d0 uma_zalloc_arg() at uma_zalloc_arg+0x5d7/frame 0xfffffe081e29a740 m_getm2() at m_getm2+0x10f/frame 0xfffffe081e29a7a0 m_uiotombuf() at m_uiotombuf+0x60/frame 0xfffffe081e29a7f0 sosend_dgram() at sosend_dgram+0x23c/frame 0xfffffe081e29a870 soo_write() at soo_write+0x49/frame 0xfffffe081e29a8b0 dofilewrite() at dofilewrite+0x85/frame 0xfffffe081e29a900 kern_writev() at kern_writev+0x65/frame 0xfffffe081e29a950 sys_write() at sys_write+0x63/frame 0xfffffe081e29a9a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e29aab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e29aab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x800b5da0a, rsp = 0x7fffffffd658, rbp = 0x7fffffffd6a0 --- Tracing command udp pid 2277 tid 100335 td 0xfffff8018fb17920 sched_switch() at sched_switch+0x482/frame 0xfffffe081e2d1570 mi_switch() at mi_switch+0x179/frame 0xfffffe081e2d15b0 critical_exit() at critical_exit+0x8b/frame 0xfffffe081e2d15d0 spinlock_exit() at spinlock_exit+0x14/frame 0xfffffe081e2d15f0 __mtx_unlock_flags() at __mtx_unlock_flags+0xf0/frame 0xfffffe081e2d1630 uma_dbg_getslab() at uma_dbg_getslab+0xd4/frame 0xfffffe081e2d1660 uma_dbg_alloc() at uma_dbg_alloc+0x36/frame 0xfffffe081e2d1690 zone_alloc_item() at zone_alloc_item+0xf4/frame 0xfffffe081e2d16d0 uma_zalloc_arg() at uma_zalloc_arg+0x5d7/frame 0xfffffe081e2d1740 m_getm2() at m_getm2+0x10f/frame 0xfffffe081e2d17a0 m_uiotombuf() at m_uiotombuf+0x60/frame 0xfffffe081e2d17f0 sosend_dgram() at sosend_dgram+0x23c/frame 0xfffffe081e2d1870 soo_write() at soo_write+0x49/frame 0xfffffe081e2d18b0 dofilewrite() at dofilewrite+0x85/frame 0xfffffe081e2d1900 kern_writev() at kern_writev+0x65/frame 0xfffffe081e2d1950 sys_write() at sys_write+0x63/frame 0xfffffe081e2d19a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e2d1ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e2d1ab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x800b5da0a, rsp = 0x7fffffffd658, rbp = 0x7fffffffd6a0 --- Tracing command udp pid 2276 tid 100294 td 0xfffff8059550a000 cpustop_handler() at cpustop_handler+0x28/frame 0xffffffff81694b00 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xffffffff81694b20 trap() at trap+0x42/frame 0xffffffff81694d40 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff81694d40 --- trap 0x13, rip = 0xffffffff808a741f, rsp = 0xffffffff81694e00, rbp = 0xfffffe081e2045e0 --- __mtx_lock_sleep() at __mtx_lock_sleep+0x24f/frame 0xfffffe081e2045e0 __mtx_lock_flags() at __mtx_lock_flags+0xf5/frame 0xfffffe081e204630 uma_dbg_getslab() at uma_dbg_getslab+0x5b/frame 0xfffffe081e204660 uma_dbg_alloc() at uma_dbg_alloc+0x36/frame 0xfffffe081e204690 zone_alloc_item() at zone_alloc_item+0xf4/frame 0xfffffe081e2046d0 uma_zalloc_arg() at uma_zalloc_arg+0x5d7/frame 0xfffffe081e204740 m_getm2() at m_getm2+0x10f/frame 0xfffffe081e2047a0 m_uiotombuf() at m_uiotombuf+0x60/frame 0xfffffe081e2047f0 sosend_dgram() at sosend_dgram+0x23c/frame 0xfffffe081e204870 soo_write() at soo_write+0x49/frame 0xfffffe081e2048b0 dofilewrite() at dofilewrite+0x85/frame 0xfffffe081e204900 kern_writev() at kern_writev+0x65/frame 0xfffffe081e204950 sys_write() at sys_write+0x63/frame 0xfffffe081e2049a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e204ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e204ab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x800b5da0a, rsp = 0x7fffffffd658, rbp = 0x7fffffffd6a0 --- Tracing command udp pid 2275 tid 100136 td 0xfffff80073962000 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc3a9ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc3a9d00 trap() at trap+0x42/frame 0xfffffe07fc3a9f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc3a9f20 --- trap 0x13, rip = 0xffffffff80db9702, rsp = 0xfffffe07fc3a9fe0, rbp = 0xfffffe081dedc660 --- DELAY() at DELAY+0x172/frame 0xfffffe081dedc660 _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0x1ca/frame 0xfffffe081dedc6d0 __mtx_lock_spin_flags() at __mtx_lock_spin_flags+0xe8/frame 0xfffffe081dedc710 smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0xbe/frame 0xfffffe081dedc770 pmap_invalidate_all() at pmap_invalidate_all+0x494/frame 0xfffffe081dedc810 pmap_advise() at pmap_advise+0x574/frame 0xfffffe081dedc8c0 vm_map_madvise() at vm_map_madvise+0x466/frame 0xfffffe081dedc980 sys_madvise() at sys_madvise+0x82/frame 0xfffffe081dedc9a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081dedcab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081dedcab0 --- syscall (75, FreeBSD ELF64, sys_madvise), rip = 0x800ac846a, rsp = 0x7fffffffd428, rbp = 0x7fffffffd430 --- Tracing command swap pid 2274 tid 100172 td 0xfffff800738ad000 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc3b1ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc3b1d00 trap() at trap+0x42/frame 0xfffffe07fc3b1f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc3b1f20 --- trap 0x13, rip = 0x401b7e, rsp = 0x7fffffffd680, rbp = 0x7fffffffd6a0 --- Tracing command udp pid 2273 tid 100181 td 0xfffff8011ccdc920 sched_switch() at sched_switch+0x482/frame 0xfffffe081dfbd480 mi_switch() at mi_switch+0x179/frame 0xfffffe081dfbd4c0 critical_exit() at critical_exit+0x8b/frame 0xfffffe081dfbd4e0 spinlock_exit() at spinlock_exit+0x14/frame 0xfffffe081dfbd500 __mtx_unlock_flags() at __mtx_unlock_flags+0xf0/frame 0xfffffe081dfbd540 taskqueue_enqueue() at taskqueue_enqueue+0x84/frame 0xfffffe081dfbd570 igb_mq_start() at igb_mq_start+0x19e/frame 0xfffffe081dfbd5d0 ether_output() at ether_output+0x5c8/frame 0xfffffe081dfbd640 ip_output() at ip_output+0xf1b/frame 0xfffffe081dfbd730 udp_send() at udp_send+0x866/frame 0xfffffe081dfbd7f0 sosend_dgram() at sosend_dgram+0x36b/frame 0xfffffe081dfbd870 soo_write() at soo_write+0x49/frame 0xfffffe081dfbd8b0 dofilewrite() at dofilewrite+0x85/frame 0xfffffe081dfbd900 kern_writev() at kern_writev+0x65/frame 0xfffffe081dfbd950 sys_write() at sys_write+0x63/frame 0xfffffe081dfbd9a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081dfbdab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081dfbdab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x800b5da0a, rsp = 0x7fffffffd658, rbp = 0x7fffffffd6a0 --- Tracing command swap pid 2272 tid 100253 td 0xfffff801c5de6490 sched_switch() at sched_switch+0x482/frame 0xfffffe081e1353f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081e135430 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e135470 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081e1354a0 _sleep() at _sleep+0x351/frame 0xfffffe081e135540 vm_waitpfault() at vm_waitpfault+0x72/frame 0xfffffe081e135560 vm_fault_hold() at vm_fault_hold+0xedb/frame 0xfffffe081e1357b0 vm_fault() at vm_fault+0x77/frame 0xfffffe081e1357f0 trap_pfault() at trap_pfault+0x1c2/frame 0xfffffe081e135890 trap() at trap+0x52b/frame 0xfffffe081e135ab0 calltrap() at calltrap+0x8/frame 0xfffffe081e135ab0 --- trap 0xc, rip = 0x800b5f0d4, rsp = 0x7fffffffd178, rbp = 0x7fffffffd1b0 --- Tracing command udp pid 2271 tid 100213 td 0xfffff8018a80d490 sched_switch() at sched_switch+0x482/frame 0xfffffe081e06d400 mi_switch() at mi_switch+0x179/frame 0xfffffe081e06d440 critical_exit() at critical_exit+0x8b/frame 0xfffffe081e06d460 ipi_bitmap_handler() at ipi_bitmap_handler+0x7a/frame 0xfffffe081e06d4a0 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0x9b/frame 0xfffffe081e06d4a0 --- interrupt, rip = 0xffffffff808a73f9, rsp = 0xfffffe081e06d560, rbp = 0xfffffe081e06d5e0 --- __mtx_lock_sleep() at __mtx_lock_sleep+0x229/frame 0xfffffe081e06d5e0 __mtx_lock_flags() at __mtx_lock_flags+0xf5/frame 0xfffffe081e06d630 uma_dbg_getslab() at uma_dbg_getslab+0x5b/frame 0xfffffe081e06d660 uma_dbg_alloc() at uma_dbg_alloc+0x36/frame 0xfffffe081e06d690 zone_alloc_item() at zone_alloc_item+0xf4/frame 0xfffffe081e06d6d0 uma_zalloc_arg() at uma_zalloc_arg+0x5d7/frame 0xfffffe081e06d740 m_getm2() at m_getm2+0x10f/frame 0xfffffe081e06d7a0 m_uiotombuf() at m_uiotombuf+0x60/frame 0xfffffe081e06d7f0 sosend_dgram() at sosend_dgram+0x23c/frame 0xfffffe081e06d870 soo_write() at soo_write+0x49/frame 0xfffffe081e06d8b0 dofilewrite() at dofilewrite+0x85/frame 0xfffffe081e06d900 kern_writev() at kern_writev+0x65/frame 0xfffffe081e06d950 sys_write() at sys_write+0x63/frame 0xfffffe081e06d9a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e06dab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e06dab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x800b5da0a, rsp = 0x7fffffffd658, rbp = 0x7fffffffd6a0 --- Tracing command swap pid 2270 tid 100299 td 0xfffff801c5d50490 sched_switch() at sched_switch+0x482/frame 0xfffffe081e21d9a0 mi_switch() at mi_switch+0x179/frame 0xfffffe081e21d9e0 critical_exit() at critical_exit+0x8b/frame 0xfffffe081e21da00 intr_event_handle() at intr_event_handle+0x13d/frame 0xfffffe081e21da50 intr_execute_handlers() at intr_execute_handlers+0x48/frame 0xfffffe081e21da80 lapic_handle_intr() at lapic_handle_intr+0x73/frame 0xfffffe081e21dab0 Xapic_isr1() at Xapic_isr1+0xa4/frame 0xfffffe081e21dab0 --- interrupt, rip = 0x401b7e, rsp = 0x7fffffffd680, rbp = 0x7fffffffd6a0 --- Tracing command swap pid 2269 tid 100250 td 0xfffff801c5d7e920 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc3c1ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc3c1d00 trap() at trap+0x42/frame 0xfffffe07fc3c1f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc3c1f20 --- trap 0x13, rip = 0x401b7e, rsp = 0x7fffffffd680, rbp = 0x7fffffffd6a0 --- Tracing command udp pid 2268 tid 100246 td 0xfffff8018a69e000 sched_switch() at sched_switch+0x482/frame 0xfffffe081e1124b0 mi_switch() at mi_switch+0x179/frame 0xfffffe081e1124f0 critical_exit() at critical_exit+0x8b/frame 0xfffffe081e112510 ipi_bitmap_handler() at ipi_bitmap_handler+0x7a/frame 0xfffffe081e112550 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0x9b/frame 0xfffffe081e112550 --- interrupt, rip = 0xffffffff80db96c6, rsp = 0xfffffe081e112610, rbp = 0xfffffe081e112660 --- DELAY() at DELAY+0x136/frame 0xfffffe081e112660 _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0x1ca/frame 0xfffffe081e1126d0 __mtx_lock_spin_flags() at __mtx_lock_spin_flags+0xe8/frame 0xfffffe081e112710 smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0xbe/frame 0xfffffe081e112770 pmap_invalidate_all() at pmap_invalidate_all+0x494/frame 0xfffffe081e112810 pmap_advise() at pmap_advise+0x574/frame 0xfffffe081e1128c0 vm_map_madvise() at vm_map_madvise+0x466/frame 0xfffffe081e112980 sys_madvise() at sys_madvise+0x82/frame 0xfffffe081e1129a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e112ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e112ab0 --- syscall (75, FreeBSD ELF64, sys_madvise), rip = 0x800ac846a, rsp = 0x7fffffffd428, rbp = 0x7fffffffd430 --- Tracing command swap pid 2267 tid 100283 td 0xfffff801c5f76000 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc3d1ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc3d1d00 trap() at trap+0x42/frame 0xfffffe07fc3d1f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc3d1f20 --- trap 0x13, rip = 0x401b7e, rsp = 0x7fffffffd680, rbp = 0x7fffffffd6a0 --- Tracing command udp pid 2266 tid 100323 td 0xfffff8059512c000 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc391ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc391d00 trap() at trap+0x42/frame 0xfffffe07fc391f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc391f20 --- trap 0x13, rip = 0xffffffff80cab192, rsp = 0xfffffe07fc391fe0, rbp = 0xfffffe081e295770 --- smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0x282/frame 0xfffffe081e295770 pmap_invalidate_all() at pmap_invalidate_all+0x494/frame 0xfffffe081e295810 pmap_advise() at pmap_advise+0x574/frame 0xfffffe081e2958c0 vm_map_madvise() at vm_map_madvise+0x466/frame 0xfffffe081e295980 sys_madvise() at sys_madvise+0x82/frame 0xfffffe081e2959a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e295ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e295ab0 --- syscall (75, FreeBSD ELF64, sys_madvise), rip = 0x800ac846a, rsp = 0x7fffffffd428, rbp = 0x7fffffffd430 --- Tracing command swap pid 2265 tid 100291 td 0xfffff8059580d490 sched_switch() at sched_switch+0x482/frame 0xfffffe081e1f39a0 mi_switch() at mi_switch+0x179/frame 0xfffffe081e1f39e0 critical_exit() at critical_exit+0x8b/frame 0xfffffe081e1f3a00 intr_event_handle() at intr_event_handle+0x13d/frame 0xfffffe081e1f3a50 intr_execute_handlers() at intr_execute_handlers+0x48/frame 0xfffffe081e1f3a80 lapic_handle_intr() at lapic_handle_intr+0x73/frame 0xfffffe081e1f3ab0 Xapic_isr1() at Xapic_isr1+0xa4/frame 0xfffffe081e1f3ab0 --- interrupt, rip = 0x401b79, rsp = 0x7fffffffd680, rbp = 0x7fffffffd6a0 --- Tracing command udp pid 2264 tid 100348 td 0xfffff80073f0d920 sched_switch() at sched_switch+0x482/frame 0xfffffe081e312440 mi_switch() at mi_switch+0x179/frame 0xfffffe081e312480 critical_exit() at critical_exit+0x8b/frame 0xfffffe081e3124a0 intr_event_handle() at intr_event_handle+0x13d/frame 0xfffffe081e3124f0 intr_execute_handlers() at intr_execute_handlers+0x48/frame 0xfffffe081e312520 lapic_handle_intr() at lapic_handle_intr+0x73/frame 0xfffffe081e312550 Xapic_isr1() at Xapic_isr1+0xa4/frame 0xfffffe081e312550 --- interrupt, rip = 0xffffffff80db9722, rsp = 0xfffffe081e312610, rbp = 0xfffffe081e312660 --- DELAY() at DELAY+0x192/frame 0xfffffe081e312660 _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0x1ca/frame 0xfffffe081e3126d0 __mtx_lock_spin_flags() at __mtx_lock_spin_flags+0xe8/frame 0xfffffe081e312710 smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0xbe/frame 0xfffffe081e312770 pmap_invalidate_all() at pmap_invalidate_all+0x494/frame 0xfffffe081e312810 pmap_advise() at pmap_advise+0x574/frame 0xfffffe081e3128c0 vm_map_madvise() at vm_map_madvise+0x466/frame 0xfffffe081e312980 sys_madvise() at sys_madvise+0x82/frame 0xfffffe081e3129a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e312ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e312ab0 --- syscall (75, FreeBSD ELF64, sys_madvise), rip = 0x800ac846a, rsp = 0x7fffffffd428, rbp = 0x7fffffffd430 --- Tracing command swap pid 2263 tid 100174 td 0xfffff80073f3d920 sched_switch() at sched_switch+0x482/frame 0xfffffe081df9a3f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081df9a430 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081df9a470 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081df9a4a0 _sleep() at _sleep+0x351/frame 0xfffffe081df9a540 vm_waitpfault() at vm_waitpfault+0x72/frame 0xfffffe081df9a560 vm_fault_hold() at vm_fault_hold+0xedb/frame 0xfffffe081df9a7b0 vm_fault() at vm_fault+0x77/frame 0xfffffe081df9a7f0 trap_pfault() at trap_pfault+0x1c2/frame 0xfffffe081df9a890 trap() at trap+0x52b/frame 0xfffffe081df9aab0 calltrap() at calltrap+0x8/frame 0xfffffe081df9aab0 --- trap 0xc, rip = 0x800b5f0d4, rsp = 0x7fffffffd178, rbp = 0x7fffffffd1b0 --- Tracing command udp pid 2262 tid 100325 td 0xfffff8018a220490 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc3b9ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc3b9d00 trap() at trap+0x42/frame 0xfffffe07fc3b9f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc3b9f20 --- trap 0x13, rip = 0xffffffff804deef0, rsp = 0xfffffe07fc3b9fe0, rbp = 0xfffffe081e29f5d0 --- igb_mq_start() at igb_mq_start+0x70/frame 0xfffffe081e29f5d0 ether_output() at ether_output+0x5c8/frame 0xfffffe081e29f640 ip_output() at ip_output+0xf1b/frame 0xfffffe081e29f730 udp_send() at udp_send+0x866/frame 0xfffffe081e29f7f0 sosend_dgram() at sosend_dgram+0x36b/frame 0xfffffe081e29f870 soo_write() at soo_write+0x49/frame 0xfffffe081e29f8b0 dofilewrite() at dofilewrite+0x85/frame 0xfffffe081e29f900 kern_writev() at kern_writev+0x65/frame 0xfffffe081e29f950 sys_write() at sys_write+0x63/frame 0xfffffe081e29f9a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e29fab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e29fab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x800b5da0a, rsp = 0x7fffffffd658, rbp = 0x7fffffffd6a0 --- Tracing command swap pid 2261 tid 100263 td 0xfffff8018a132920 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc3a1ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc3a1d00 trap() at trap+0x42/frame 0xfffffe07fc3a1f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc3a1f20 --- trap 0x13, rip = 0x401b7e, rsp = 0x7fffffffd680, rbp = 0x7fffffffd6a0 --- Tracing command udp pid 2260 tid 100173 td 0xfffff80073962920 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc389ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc389d00 trap() at trap+0x42/frame 0xfffffe07fc389f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc389f20 --- trap 0x13, rip = 0xffffffff808a741f, rsp = 0xfffffe07fc389fe0, rbp = 0xfffffe081df94f60 --- __mtx_lock_sleep() at __mtx_lock_sleep+0x24f/frame 0xfffffe081df94f60 __mtx_lock_flags() at __mtx_lock_flags+0xf5/frame 0xfffffe081df94fb0 uma_dbg_getslab() at uma_dbg_getslab+0x5b/frame 0xfffffe081df94fe0 uma_dbg_free() at uma_dbg_free+0x33/frame 0xfffffe081df95000 uma_zfree_arg() at uma_zfree_arg+0x42/frame 0xfffffe081df95060 m_freem() at m_freem+0x18/frame 0xfffffe081df95080 igb_txeof() at igb_txeof+0x1b7/frame 0xfffffe081df950d0 igb_mq_start_locked() at igb_mq_start_locked+0x833/frame 0xfffffe081df95570 igb_mq_start() at igb_mq_start+0x151/frame 0xfffffe081df955d0 ether_output() at ether_output+0x5c8/frame 0xfffffe081df95640 ip_output() at ip_output+0xf1b/frame 0xfffffe081df95730 udp_send() at udp_send+0x866/frame 0xfffffe081df957f0 sosend_dgram() at sosend_dgram+0x36b/frame 0xfffffe081df95870 soo_write() at soo_write+0x49/frame 0xfffffe081df958b0 dofilewrite() at dofilewrite+0x85/frame 0xfffffe081df95900 kern_writev() at kern_writev+0x65/frame 0xfffffe081df95950 sys_write() at sys_write+0x63/frame 0xfffffe081df959a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081df95ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081df95ab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x800b5da0a, rsp = 0x7fffffffd658, rbp = 0x7fffffffd6a0 --- Tracing command swap pid 2259 tid 100298 td 0xfffff801c5d50920 sched_switch() at sched_switch+0x482/frame 0xfffffe081e2183f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081e218430 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e218470 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081e2184a0 _sleep() at _sleep+0x351/frame 0xfffffe081e218540 vm_waitpfault() at vm_waitpfault+0x72/frame 0xfffffe081e218560 vm_fault_hold() at vm_fault_hold+0xedb/frame 0xfffffe081e2187b0 vm_fault() at vm_fault+0x77/frame 0xfffffe081e2187f0 trap_pfault() at trap_pfault+0x1c2/frame 0xfffffe081e218890 trap() at trap+0x52b/frame 0xfffffe081e218ab0 calltrap() at calltrap+0x8/frame 0xfffffe081e218ab0 --- trap 0xc, rip = 0x800b5f0d4, rsp = 0x7fffffffd178, rbp = 0x7fffffffd1b0 --- Tracing command udp pid 2258 tid 100177 td 0xfffff8011cc4a000 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc3c9ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc3c9d00 trap() at trap+0x42/frame 0xfffffe07fc3c9f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc3c9f20 --- trap 0x13, rip = 0xffffffff804deef0, rsp = 0xfffffe07fc3c9fe0, rbp = 0xfffffe081dfa95d0 --- igb_mq_start() at igb_mq_start+0x70/frame 0xfffffe081dfa95d0 ether_output() at ether_output+0x5c8/frame 0xfffffe081dfa9640 ip_output() at ip_output+0xf1b/frame 0xfffffe081dfa9730 udp_send() at udp_send+0x866/frame 0xfffffe081dfa97f0 sosend_dgram() at sosend_dgram+0x36b/frame 0xfffffe081dfa9870 soo_write() at soo_write+0x49/frame 0xfffffe081dfa98b0 dofilewrite() at dofilewrite+0x85/frame 0xfffffe081dfa9900 kern_writev() at kern_writev+0x65/frame 0xfffffe081dfa9950 sys_write() at sys_write+0x63/frame 0xfffffe081dfa99a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081dfa9ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081dfa9ab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x800b5da0a, rsp = 0x7fffffffd658, rbp = 0x7fffffffd6a0 --- Tracing command swap pid 2257 tid 100182 td 0xfffff800830a5920 sched_switch() at sched_switch+0x482/frame 0xfffffe081dfc23f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081dfc2430 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081dfc2470 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081dfc24a0 _sleep() at _sleep+0x351/frame 0xfffffe081dfc2540 vm_waitpfault() at vm_waitpfault+0x72/frame 0xfffffe081dfc2560 vm_fault_hold() at vm_fault_hold+0xedb/frame 0xfffffe081dfc27b0 vm_fault() at vm_fault+0x77/frame 0xfffffe081dfc27f0 trap_pfault() at trap_pfault+0x1c2/frame 0xfffffe081dfc2890 trap() at trap+0x52b/frame 0xfffffe081dfc2ab0 calltrap() at calltrap+0x8/frame 0xfffffe081dfc2ab0 --- trap 0xc, rip = 0x800b5f0d4, rsp = 0x7fffffffd178, rbp = 0x7fffffffd1b0 --- Tracing command udp pid 2256 tid 100150 td 0xfffff80073f49000 sched_switch() at sched_switch+0x482/frame 0xfffffe081df22220 mi_switch() at mi_switch+0x179/frame 0xfffffe081df22260 critical_exit() at critical_exit+0x8b/frame 0xfffffe081df22280 ipi_bitmap_handler() at ipi_bitmap_handler+0x7a/frame 0xfffffe081df222c0 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0x9b/frame 0xfffffe081df222c0 --- interrupt, rip = 0xffffffff808a73f9, rsp = 0xfffffe081df22380, rbp = 0xfffffe081df22400 --- __mtx_lock_sleep() at __mtx_lock_sleep+0x229/frame 0xfffffe081df22400 __mtx_lock_flags() at __mtx_lock_flags+0xf5/frame 0xfffffe081df22450 zone_fetch_slab() at zone_fetch_slab+0x44/frame 0xfffffe081df22480 zone_import() at zone_import+0x42/frame 0xfffffe081df224f0 zone_alloc_item() at zone_alloc_item+0x38/frame 0xfffffe081df22530 uma_zalloc_arg() at uma_zalloc_arg+0x5d7/frame 0xfffffe081df225a0 ip_fragment() at ip_fragment+0x18e/frame 0xfffffe081df22640 ip_output() at ip_output+0xe70/frame 0xfffffe081df22730 udp_send() at udp_send+0x866/frame 0xfffffe081df227f0 sosend_dgram() at sosend_dgram+0x36b/frame 0xfffffe081df22870 soo_write() at soo_write+0x49/frame 0xfffffe081df228b0 dofilewrite() at dofilewrite+0x85/frame 0xfffffe081df22900 kern_writev() at kern_writev+0x65/frame 0xfffffe081df22950 sys_write() at sys_write+0x63/frame 0xfffffe081df229a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081df22ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081df22ab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x800b5da0a, rsp = 0x7fffffffd658, rbp = 0x7fffffffd6a0 --- Tracing command swap pid 2255 tid 100277 td 0xfffff801c8092000 sched_switch() at sched_switch+0x482/frame 0xfffffe081e1ad3f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081e1ad430 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e1ad470 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081e1ad4a0 _sleep() at _sleep+0x351/frame 0xfffffe081e1ad540 vm_waitpfault() at vm_waitpfault+0x72/frame 0xfffffe081e1ad560 vm_fault_hold() at vm_fault_hold+0xedb/frame 0xfffffe081e1ad7b0 vm_fault() at vm_fault+0x77/frame 0xfffffe081e1ad7f0 trap_pfault() at trap_pfault+0x1c2/frame 0xfffffe081e1ad890 trap() at trap+0x52b/frame 0xfffffe081e1adab0 calltrap() at calltrap+0x8/frame 0xfffffe081e1adab0 --- trap 0xc, rip = 0x800b5f0d4, rsp = 0x7fffffffd178, rbp = 0x7fffffffd1b0 --- Tracing command udp pid 2254 tid 100334 td 0xfffff801c5d4f000 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc3f9ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc3f9d00 trap() at trap+0x42/frame 0xfffffe07fc3f9f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc3f9f20 --- trap 0x13, rip = 0xffffffff80b2dec5, rsp = 0xfffffe07fc3f9fe0, rbp = 0xfffffe081e2cc000 --- uma_dbg_free() at uma_dbg_free+0x85/frame 0xfffffe081e2cc000 uma_zfree_arg() at uma_zfree_arg+0x42/frame 0xfffffe081e2cc060 m_freem() at m_freem+0x63/frame 0xfffffe081e2cc080 igb_txeof() at igb_txeof+0x1b7/frame 0xfffffe081e2cc0d0 igb_mq_start_locked() at igb_mq_start_locked+0x833/frame 0xfffffe081e2cc570 igb_mq_start() at igb_mq_start+0x151/frame 0xfffffe081e2cc5d0 ether_output() at ether_output+0x5c8/frame 0xfffffe081e2cc640 ip_output() at ip_output+0x1030/frame 0xfffffe081e2cc730 udp_send() at udp_send+0x866/frame 0xfffffe081e2cc7f0 sosend_dgram() at sosend_dgram+0x36b/frame 0xfffffe081e2cc870 soo_write() at soo_write+0x49/frame 0xfffffe081e2cc8b0 dofilewrite() at dofilewrite+0x85/frame 0xfffffe081e2cc900 kern_writev() at kern_writev+0x65/frame 0xfffffe081e2cc950 sys_write() at sys_write+0x63/frame 0xfffffe081e2cc9a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e2ccab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e2ccab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x800b5da0a, rsp = 0x7fffffffd658, rbp = 0x7fffffffd6a0 --- Tracing command swap pid 2253 tid 100290 td 0xfffff801c8092490 sched_switch() at sched_switch+0x482/frame 0xfffffe081e1ee3f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081e1ee430 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e1ee470 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081e1ee4a0 _sleep() at _sleep+0x351/frame 0xfffffe081e1ee540 vm_waitpfault() at vm_waitpfault+0x72/frame 0xfffffe081e1ee560 vm_fault_hold() at vm_fault_hold+0xedb/frame 0xfffffe081e1ee7b0 vm_fault() at vm_fault+0x77/frame 0xfffffe081e1ee7f0 trap_pfault() at trap_pfault+0x1c2/frame 0xfffffe081e1ee890 trap() at trap+0x52b/frame 0xfffffe081e1eeab0 calltrap() at calltrap+0x8/frame 0xfffffe081e1eeab0 --- trap 0xc, rip = 0x800b5f0d4, rsp = 0x7fffffffd178, rbp = 0x7fffffffd1b0 --- Tracing command udp pid 2252 tid 100349 td 0xfffff8018a132490 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc3f1ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc3f1d00 trap() at trap+0x42/frame 0xfffffe07fc3f1f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc3f1f20 --- trap 0x13, rip = 0xffffffff80b2aad0, rsp = 0xfffffe07fc3f1fe0, rbp = 0xfffffe081e317060 --- uma_zfree_arg() at uma_zfree_arg+0x320/frame 0xfffffe081e317060 m_freem() at m_freem+0x63/frame 0xfffffe081e317080 igb_txeof() at igb_txeof+0x1b7/frame 0xfffffe081e3170d0 igb_mq_start_locked() at igb_mq_start_locked+0x833/frame 0xfffffe081e317570 igb_mq_start() at igb_mq_start+0x151/frame 0xfffffe081e3175d0 ether_output() at ether_output+0x5c8/frame 0xfffffe081e317640 ip_output() at ip_output+0x1030/frame 0xfffffe081e317730 udp_send() at udp_send+0x866/frame 0xfffffe081e3177f0 sosend_dgram() at sosend_dgram+0x36b/frame 0xfffffe081e317870 soo_write() at soo_write+0x49/frame 0xfffffe081e3178b0 dofilewrite() at dofilewrite+0x85/frame 0xfffffe081e317900 kern_writev() at kern_writev+0x65/frame 0xfffffe081e317950 sys_write() at sys_write+0x63/frame 0xfffffe081e3179a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e317ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e317ab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x800b5da0a, rsp = 0x7fffffffd658, rbp = 0x7fffffffd6a0 --- Tracing command swap pid 2251 tid 100215 td 0xfffff8018a80d000 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc3d9ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc3d9d00 trap() at trap+0x42/frame 0xfffffe07fc3d9f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc3d9f20 --- trap 0x13, rip = 0x401b7e, rsp = 0x7fffffffd680, rbp = 0x7fffffffd6a0 --- Tracing command udp pid 2250 tid 100141 td 0xfffff80073956490 sched_switch() at sched_switch+0x482/frame 0xfffffe081def5390 mi_switch() at mi_switch+0x179/frame 0xfffffe081def53d0 critical_exit() at critical_exit+0x8b/frame 0xfffffe081def53f0 intr_event_handle() at intr_event_handle+0x13d/frame 0xfffffe081def5440 intr_execute_handlers() at intr_execute_handlers+0x48/frame 0xfffffe081def5470 lapic_handle_intr() at lapic_handle_intr+0x73/frame 0xfffffe081def54a0 Xapic_isr1() at Xapic_isr1+0xa4/frame 0xfffffe081def54a0 --- interrupt, rip = 0xffffffff808a73f9, rsp = 0xfffffe081def5560, rbp = 0xfffffe081def55e0 --- __mtx_lock_sleep() at __mtx_lock_sleep+0x229/frame 0xfffffe081def55e0 __mtx_lock_flags() at __mtx_lock_flags+0xf5/frame 0xfffffe081def5630 uma_dbg_getslab() at uma_dbg_getslab+0x5b/frame 0xfffffe081def5660 uma_dbg_alloc() at uma_dbg_alloc+0x36/frame 0xfffffe081def5690 zone_alloc_item() at zone_alloc_item+0xf4/frame 0xfffffe081def56d0 uma_zalloc_arg() at uma_zalloc_arg+0x5d7/frame 0xfffffe081def5740 m_getm2() at m_getm2+0x10f/frame 0xfffffe081def57a0 m_uiotombuf() at m_uiotombuf+0x60/frame 0xfffffe081def57f0 sosend_dgram() at sosend_dgram+0x23c/frame 0xfffffe081def5870 soo_write() at soo_write+0x49/frame 0xfffffe081def58b0 dofilewrite() at dofilewrite+0x85/frame 0xfffffe081def5900 kern_writev() at kern_writev+0x65/frame 0xfffffe081def5950 sys_write() at sys_write+0x63/frame 0xfffffe081def59a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081def5ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081def5ab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x800b5da0a, rsp = 0x7fffffffd658, rbp = 0x7fffffffd6a0 --- Tracing command swap pid 2249 tid 100245 td 0xfffff8018a69e490 sched_switch() at sched_switch+0x482/frame 0xfffffe081e10d3f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081e10d430 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e10d470 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081e10d4a0 _sleep() at _sleep+0x351/frame 0xfffffe081e10d540 vm_waitpfault() at vm_waitpfault+0x72/frame 0xfffffe081e10d560 vm_fault_hold() at vm_fault_hold+0xedb/frame 0xfffffe081e10d7b0 vm_fault() at vm_fault+0x77/frame 0xfffffe081e10d7f0 trap_pfault() at trap_pfault+0x1c2/frame 0xfffffe081e10d890 trap() at trap+0x52b/frame 0xfffffe081e10dab0 calltrap() at calltrap+0x8/frame 0xfffffe081e10dab0 --- trap 0xc, rip = 0x800b5f0d4, rsp = 0x7fffffffd178, rbp = 0x7fffffffd1b0 --- Tracing command udp pid 2248 tid 100292 td 0xfffff8018ad55920 sched_switch() at sched_switch+0x482/frame 0xfffffe081e1f84b0 mi_switch() at mi_switch+0x179/frame 0xfffffe081e1f84f0 critical_exit() at critical_exit+0x8b/frame 0xfffffe081e1f8510 ipi_bitmap_handler() at ipi_bitmap_handler+0x7a/frame 0xfffffe081e1f8550 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0x9b/frame 0xfffffe081e1f8550 --- interrupt, rip = 0xffffffff80db95c1, rsp = 0xfffffe081e1f8610, rbp = 0xfffffe081e1f8660 --- DELAY() at DELAY+0x31/frame 0xfffffe081e1f8660 _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0x1ca/frame 0xfffffe081e1f86d0 __mtx_lock_spin_flags() at __mtx_lock_spin_flags+0xe8/frame 0xfffffe081e1f8710 smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0xbe/frame 0xfffffe081e1f8770 pmap_invalidate_all() at pmap_invalidate_all+0x494/frame 0xfffffe081e1f8810 pmap_advise() at pmap_advise+0x574/frame 0xfffffe081e1f88c0 vm_map_madvise() at vm_map_madvise+0x466/frame 0xfffffe081e1f8980 sys_madvise() at sys_madvise+0x82/frame 0xfffffe081e1f89a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e1f8ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e1f8ab0 --- syscall (75, FreeBSD ELF64, sys_madvise), rip = 0x800ac846a, rsp = 0x7fffffffd428, rbp = 0x7fffffffd430 --- Tracing command swap pid 2247 tid 100188 td 0xfffff8018a0e9490 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe0785d08ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe0785d08d00 trap() at trap+0x42/frame 0xfffffe0785d08f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0785d08f20 --- trap 0x13, rip = 0x401b7e, rsp = 0x7fffffffd680, rbp = 0x7fffffffd6a0 --- Tracing command udp pid 2246 tid 100262 td 0xfffff801c5d7e490 sched_switch() at sched_switch+0x482/frame 0xfffffe081e162400 mi_switch() at mi_switch+0x179/frame 0xfffffe081e162440 critical_exit() at critical_exit+0x8b/frame 0xfffffe081e162460 ipi_bitmap_handler() at ipi_bitmap_handler+0x7a/frame 0xfffffe081e1624a0 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0x9b/frame 0xfffffe081e1624a0 --- interrupt, rip = 0xffffffff808a7490, rsp = 0xfffffe081e162560, rbp = 0xfffffe081e1625e0 --- __mtx_lock_sleep() at __mtx_lock_sleep+0x2c0/frame 0xfffffe081e1625e0 __mtx_lock_flags() at __mtx_lock_flags+0xf5/frame 0xfffffe081e162630 uma_dbg_getslab() at uma_dbg_getslab+0x5b/frame 0xfffffe081e162660 uma_dbg_alloc() at uma_dbg_alloc+0x36/frame 0xfffffe081e162690 zone_alloc_item() at zone_alloc_item+0xf4/frame 0xfffffe081e1626d0 uma_zalloc_arg() at uma_zalloc_arg+0x5d7/frame 0xfffffe081e162740 m_getm2() at m_getm2+0x10f/frame 0xfffffe081e1627a0 m_uiotombuf() at m_uiotombuf+0x60/frame 0xfffffe081e1627f0 sosend_dgram() at sosend_dgram+0x23c/frame 0xfffffe081e162870 soo_write() at soo_write+0x49/frame 0xfffffe081e1628b0 dofilewrite() at dofilewrite+0x85/frame 0xfffffe081e162900 kern_writev() at kern_writev+0x65/frame 0xfffffe081e162950 sys_write() at sys_write+0x63/frame 0xfffffe081e1629a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e162ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e162ab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x800b5da0a, rsp = 0x7fffffffd658, rbp = 0x7fffffffd6a0 --- Tracing command swap pid 2245 tid 100233 td 0xfffff8011cc46000 Tracing command udp pid 2244 tid 100139 td 0xfffff80073b15000 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc3e9ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc3e9d00 trap() at trap+0x42/frame 0xfffffe07fc3e9f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc3e9f20 --- trap 0x13, rip = 0xffffffff80cc2193, rsp = 0xfffffe07fc3e9fe0, rbp = 0xfffffe081deeb9a0 --- cpu_set_syscall_retval() at cpu_set_syscall_retval+0x83/frame 0xfffffe081deeb9a0 amd64_syscall() at amd64_syscall+0x348/frame 0xfffffe081deebab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081deebab0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x800b5da0a, rsp = 0x7fffffffd658, rbp = 0x7fffffffd6a0 --- Tracing command udp pid 2243 tid 100249 td 0xfffff801c5d50000 Tracing command udp pid 2242 tid 100140 td 0xfffff80073956920 sched_switch() at sched_switch+0x482/frame 0xfffffe081def0740 mi_switch() at mi_switch+0x179/frame 0xfffffe081def0780 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081def07c0 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081def0830 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe081def0860 _sleep() at _sleep+0x314/frame 0xfffffe081def0900 kern_nanosleep() at kern_nanosleep+0x10e/frame 0xfffffe081def0960 sys_nanosleep() at sys_nanosleep+0x53/frame 0xfffffe081def09a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081def0ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081def0ab0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x800b3dada, rsp = 0x7fffffffd648, rbp = 0x7fffffffd6a0 --- Tracing command swap pid 2241 tid 100314 td 0xfffff8073297b490 sched_switch() at sched_switch+0x482/frame 0xfffffe081e268740 mi_switch() at mi_switch+0x179/frame 0xfffffe081e268780 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e2687c0 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081e268830 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe081e268860 _sleep() at _sleep+0x314/frame 0xfffffe081e268900 kern_nanosleep() at kern_nanosleep+0x10e/frame 0xfffffe081e268960 sys_nanosleep() at sys_nanosleep+0x53/frame 0xfffffe081e2689a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e268ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e268ab0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x800b3eada, rsp = 0x7fffffffd648, rbp = 0x7fffffffd6a0 --- Tracing command run pid 2122 tid 100230 td 0xfffff801c5c67000 Tracing command run pid 2121 tid 100255 td 0xfffff8018ad54490 Tracing command run pid 2120 tid 100189 td 0xfffff8018a132000 sched_switch() at sched_switch+0x482/frame 0xfffffe081dff2740 mi_switch() at mi_switch+0x179/frame 0xfffffe081dff2780 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081dff27c0 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081dff2830 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe081dff2860 _sleep() at _sleep+0x314/frame 0xfffffe081dff2900 kern_nanosleep() at kern_nanosleep+0x10e/frame 0xfffffe081dff2960 sys_nanosleep() at sys_nanosleep+0x53/frame 0xfffffe081dff29a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081dff2ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081dff2ab0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x800b3dada, rsp = 0x7fffffffd608, rbp = 0x7fffffffd660 --- Tracing command su pid 2119 tid 100338 td 0xfffff8018acae920 Tracing command sh pid 2113 tid 100282 td 0xfffff801c80f9920 Tracing command top pid 1374 tid 100190 td 0xfffff800832fd920 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc399ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc399d00 trap() at trap+0x42/frame 0xfffffe07fc399f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc399f20 --- trap 0x13, rip = 0xffffffff808a73f2, rsp = 0xfffffe07fc399fe0, rbp = 0xfffffe081dff77c0 --- __mtx_lock_sleep() at __mtx_lock_sleep+0x222/frame 0xfffffe081dff77c0 __mtx_lock_flags() at __mtx_lock_flags+0xf5/frame 0xfffffe081dff7810 pmap_advise() at pmap_advise+0xc7/frame 0xfffffe081dff78c0 vm_map_madvise() at vm_map_madvise+0x466/frame 0xfffffe081dff7980 sys_madvise() at sys_madvise+0x82/frame 0xfffffe081dff79a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081dff7ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081dff7ab0 --- syscall (75, FreeBSD ELF64, sys_madvise), rip = 0x800f3a46a, rsp = 0x7fffffffd538, rbp = 0x7fffffffd540 --- Tracing command bash pid 1318 tid 100261 td 0xfffff801c5de6000 Tracing command sshd pid 1317 tid 100163 td 0xfffff80073ce4000 sched_switch() at sched_switch+0x482/frame 0xfffffe081df63590 mi_switch() at mi_switch+0x179/frame 0xfffffe081df635d0 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081df63610 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081df63680 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe081df636b0 _cv_wait_sig() at _cv_wait_sig+0x1e1/frame 0xfffffe081df63720 seltdwait() at seltdwait+0x78/frame 0xfffffe081df63760 kern_select() at kern_select+0x8a2/frame 0xfffffe081df63960 sys_select() at sys_select+0x54/frame 0xfffffe081df639a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081df63ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081df63ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8032099aa, rsp = 0x7fffffffd168, rbp = 0x7fffffffd1d0 --- Tracing command sshd pid 1313 tid 100154 td 0xfffff80083132000 sched_switch() at sched_switch+0x482/frame 0xfffffe081df36630 mi_switch() at mi_switch+0x179/frame 0xfffffe081df36670 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081df366b0 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081df36720 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe081df36750 _cv_wait_sig() at _cv_wait_sig+0x1e1/frame 0xfffffe081df367c0 seltdwait() at seltdwait+0x78/frame 0xfffffe081df36800 sys_poll() at sys_poll+0x3c9/frame 0xfffffe081df369a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081df36ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081df36ab0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8031a1e7a, rsp = 0x7fffffffd1f8, rbp = 0x7fffffffd250 --- Tracing command sh pid 1073 tid 100175 td 0xfffff80073f3d490 Tracing command getty pid 1072 tid 100243 td 0xfffff8018acad490 sched_switch() at sched_switch+0x482/frame 0xfffffe081e103610 mi_switch() at mi_switch+0x179/frame 0xfffffe081e103650 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e103690 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081e103700 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe081e103730 _cv_wait_sig() at _cv_wait_sig+0x1e1/frame 0xfffffe081e1037a0 tty_wait() at tty_wait+0x42/frame 0xfffffe081e1037d0 ttydisc_read() at ttydisc_read+0x353/frame 0xfffffe081e103830 ttydev_read() at ttydev_read+0x6d/frame 0xfffffe081e103860 devfs_read_f() at devfs_read_f+0x7f/frame 0xfffffe081e1038c0 dofileread() at dofileread+0x7b/frame 0xfffffe081e103900 kern_readv() at kern_readv+0x65/frame 0xfffffe081e103950 sys_read() at sys_read+0x63/frame 0xfffffe081e1039a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e103ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e103ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800b5fa2a, rsp = 0x7fffffffdc98, rbp = 0x7fffffffdcd0 --- Tracing command getty pid 1071 tid 100242 td 0xfffff8018acad920 sched_switch() at sched_switch+0x482/frame 0xfffffe081e0fe610 mi_switch() at mi_switch+0x179/frame 0xfffffe081e0fe650 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e0fe690 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081e0fe700 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe081e0fe730 _cv_wait_sig() at _cv_wait_sig+0x1e1/frame 0xfffffe081e0fe7a0 tty_wait() at tty_wait+0x42/frame 0xfffffe081e0fe7d0 ttydisc_read() at ttydisc_read+0x353/frame 0xfffffe081e0fe830 ttydev_read() at ttydev_read+0x6d/frame 0xfffffe081e0fe860 devfs_read_f() at devfs_read_f+0x7f/frame 0xfffffe081e0fe8c0 dofileread() at dofileread+0x7b/frame 0xfffffe081e0fe900 kern_readv() at kern_readv+0x65/frame 0xfffffe081e0fe950 sys_read() at sys_read+0x63/frame 0xfffffe081e0fe9a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e0feab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e0feab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800b5fa2a, rsp = 0x7fffffffdc98, rbp = 0x7fffffffdcd0 --- Tracing command getty pid 1070 tid 100241 td 0xfffff8018acae000 sched_switch() at sched_switch+0x482/frame 0xfffffe081e0f9610 mi_switch() at mi_switch+0x179/frame 0xfffffe081e0f9650 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e0f9690 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081e0f9700 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe081e0f9730 _cv_wait_sig() at _cv_wait_sig+0x1e1/frame 0xfffffe081e0f97a0 tty_wait() at tty_wait+0x42/frame 0xfffffe081e0f97d0 ttydisc_read() at ttydisc_read+0x353/frame 0xfffffe081e0f9830 ttydev_read() at ttydev_read+0x6d/frame 0xfffffe081e0f9860 devfs_read_f() at devfs_read_f+0x7f/frame 0xfffffe081e0f98c0 dofileread() at dofileread+0x7b/frame 0xfffffe081e0f9900 kern_readv() at kern_readv+0x65/frame 0xfffffe081e0f9950 sys_read() at sys_read+0x63/frame 0xfffffe081e0f99a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e0f9ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e0f9ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800b5fa2a, rsp = 0x7fffffffdc98, rbp = 0x7fffffffdcd0 --- Tracing command getty pid 1069 tid 100240 td 0xfffff8018acae490 sched_switch() at sched_switch+0x482/frame 0xfffffe081e0f4610 mi_switch() at mi_switch+0x179/frame 0xfffffe081e0f4650 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e0f4690 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081e0f4700 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe081e0f4730 _cv_wait_sig() at _cv_wait_sig+0x1e1/frame 0xfffffe081e0f47a0 tty_wait() at tty_wait+0x42/frame 0xfffffe081e0f47d0 ttydisc_read() at ttydisc_read+0x353/frame 0xfffffe081e0f4830 ttydev_read() at ttydev_read+0x6d/frame 0xfffffe081e0f4860 devfs_read_f() at devfs_read_f+0x7f/frame 0xfffffe081e0f48c0 dofileread() at dofileread+0x7b/frame 0xfffffe081e0f4900 kern_readv() at kern_readv+0x65/frame 0xfffffe081e0f4950 sys_read() at sys_read+0x63/frame 0xfffffe081e0f49a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e0f4ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e0f4ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800b5fa2a, rsp = 0x7fffffffdc98, rbp = 0x7fffffffdcd0 --- Tracing command getty pid 1068 tid 100155 td 0xfffff800831d7920 sched_switch() at sched_switch+0x482/frame 0xfffffe081df3b610 mi_switch() at mi_switch+0x179/frame 0xfffffe081df3b650 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081df3b690 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081df3b700 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe081df3b730 _cv_wait_sig() at _cv_wait_sig+0x1e1/frame 0xfffffe081df3b7a0 tty_wait() at tty_wait+0x42/frame 0xfffffe081df3b7d0 ttydisc_read() at ttydisc_read+0x353/frame 0xfffffe081df3b830 ttydev_read() at ttydev_read+0x6d/frame 0xfffffe081df3b860 devfs_read_f() at devfs_read_f+0x7f/frame 0xfffffe081df3b8c0 dofileread() at dofileread+0x7b/frame 0xfffffe081df3b900 kern_readv() at kern_readv+0x65/frame 0xfffffe081df3b950 sys_read() at sys_read+0x63/frame 0xfffffe081df3b9a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081df3bab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081df3bab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800b5fa2a, rsp = 0x7fffffffdc98, rbp = 0x7fffffffdcd0 --- Tracing command getty pid 1067 tid 100239 td 0xfffff80073a66000 sched_switch() at sched_switch+0x482/frame 0xfffffe081e0ef610 mi_switch() at mi_switch+0x179/frame 0xfffffe081e0ef650 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e0ef690 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081e0ef700 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe081e0ef730 _cv_wait_sig() at _cv_wait_sig+0x1e1/frame 0xfffffe081e0ef7a0 tty_wait() at tty_wait+0x42/frame 0xfffffe081e0ef7d0 ttydisc_read() at ttydisc_read+0x353/frame 0xfffffe081e0ef830 ttydev_read() at ttydev_read+0x6d/frame 0xfffffe081e0ef860 devfs_read_f() at devfs_read_f+0x7f/frame 0xfffffe081e0ef8c0 dofileread() at dofileread+0x7b/frame 0xfffffe081e0ef900 kern_readv() at kern_readv+0x65/frame 0xfffffe081e0ef950 sys_read() at sys_read+0x63/frame 0xfffffe081e0ef9a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e0efab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e0efab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800b5fa2a, rsp = 0x7fffffffdc98, rbp = 0x7fffffffdcd0 --- Tracing command getty pid 1066 tid 100228 td 0xfffff80073a66490 sched_switch() at sched_switch+0x482/frame 0xfffffe081e0b8610 mi_switch() at mi_switch+0x179/frame 0xfffffe081e0b8650 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e0b8690 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081e0b8700 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe081e0b8730 _cv_wait_sig() at _cv_wait_sig+0x1e1/frame 0xfffffe081e0b87a0 tty_wait() at tty_wait+0x42/frame 0xfffffe081e0b87d0 ttydisc_read() at ttydisc_read+0x353/frame 0xfffffe081e0b8830 ttydev_read() at ttydev_read+0x6d/frame 0xfffffe081e0b8860 devfs_read_f() at devfs_read_f+0x7f/frame 0xfffffe081e0b88c0 dofileread() at dofileread+0x7b/frame 0xfffffe081e0b8900 kern_readv() at kern_readv+0x65/frame 0xfffffe081e0b8950 sys_read() at sys_read+0x63/frame 0xfffffe081e0b89a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e0b8ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e0b8ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800b5fa2a, rsp = 0x7fffffffdc98, rbp = 0x7fffffffdcd0 --- Tracing command getty pid 1065 tid 100232 td 0xfffff8018a0d0490 sched_switch() at sched_switch+0x482/frame 0xfffffe081e0cc610 mi_switch() at mi_switch+0x179/frame 0xfffffe081e0cc650 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e0cc690 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081e0cc700 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe081e0cc730 _cv_wait_sig() at _cv_wait_sig+0x1e1/frame 0xfffffe081e0cc7a0 tty_wait() at tty_wait+0x42/frame 0xfffffe081e0cc7d0 ttydisc_read() at ttydisc_read+0x353/frame 0xfffffe081e0cc830 ttydev_read() at ttydev_read+0x6d/frame 0xfffffe081e0cc860 devfs_read_f() at devfs_read_f+0x7f/frame 0xfffffe081e0cc8c0 dofileread() at dofileread+0x7b/frame 0xfffffe081e0cc900 kern_readv() at kern_readv+0x65/frame 0xfffffe081e0cc950 sys_read() at sys_read+0x63/frame 0xfffffe081e0cc9a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e0ccab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e0ccab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800b5fa2a, rsp = 0x7fffffffdc98, rbp = 0x7fffffffdcd0 --- Tracing command getty pid 1064 tid 100135 td 0xfffff800738ad490 sched_switch() at sched_switch+0x482/frame 0xfffffe081ded7610 mi_switch() at mi_switch+0x179/frame 0xfffffe081ded7650 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081ded7690 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081ded7700 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe081ded7730 _cv_wait_sig() at _cv_wait_sig+0x1e1/frame 0xfffffe081ded77a0 tty_wait() at tty_wait+0x42/frame 0xfffffe081ded77d0 ttydisc_read() at ttydisc_read+0x353/frame 0xfffffe081ded7830 ttydev_read() at ttydev_read+0x6d/frame 0xfffffe081ded7860 devfs_read_f() at devfs_read_f+0x7f/frame 0xfffffe081ded78c0 dofileread() at dofileread+0x7b/frame 0xfffffe081ded7900 kern_readv() at kern_readv+0x65/frame 0xfffffe081ded7950 sys_read() at sys_read+0x63/frame 0xfffffe081ded79a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081ded7ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081ded7ab0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800b5fa2a, rsp = 0x7fffffffdc98, rbp = 0x7fffffffdcd0 --- Tracing command bash pid 1000 tid 100198 td 0xfffff80083132490 Tracing command csh pid 998 tid 100224 td 0xfffff800738ad920 Tracing command su pid 997 tid 100167 td 0xfffff80073c87490 Tracing command bash pid 992 tid 100180 td 0xfffff80073f05000 Tracing command sshd pid 991 tid 100222 td 0xfffff80073a66920 sched_switch() at sched_switch+0x482/frame 0xfffffe081e09a590 mi_switch() at mi_switch+0x179/frame 0xfffffe081e09a5d0 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e09a610 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081e09a680 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe081e09a6b0 _cv_wait_sig() at _cv_wait_sig+0x1e1/frame 0xfffffe081e09a720 seltdwait() at seltdwait+0x78/frame 0xfffffe081e09a760 kern_select() at kern_select+0x8a2/frame 0xfffffe081e09a960 sys_select() at sys_select+0x54/frame 0xfffffe081e09a9a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e09aab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e09aab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8032099aa, rsp = 0x7fffffffd168, rbp = 0x7fffffffd1d0 --- Tracing command sshd pid 989 tid 100221 td 0xfffff8018ad55000 sched_switch() at sched_switch+0x482/frame 0xfffffe081e095630 mi_switch() at mi_switch+0x179/frame 0xfffffe081e095670 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e0956b0 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081e095720 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe081e095750 _cv_wait_sig() at _cv_wait_sig+0x1e1/frame 0xfffffe081e0957c0 seltdwait() at seltdwait+0x78/frame 0xfffffe081e095800 sys_poll() at sys_poll+0x3c9/frame 0xfffffe081e0959a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e095ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e095ab0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8031a1e7a, rsp = 0x7fffffffd1f8, rbp = 0x7fffffffd250 --- Tracing command cron pid 971 tid 100217 td 0xfffff8018a773920 Tracing command sendmail pid 967 tid 100216 td 0xfffff80083099490 Tracing command sendmail pid 964 tid 100214 td 0xfffff8011cc46490 sched_switch() at sched_switch+0x482/frame 0xfffffe081e072580 mi_switch() at mi_switch+0x179/frame 0xfffffe081e0725c0 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e072600 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081e072670 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe081e0726a0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x1fd/frame 0xfffffe081e072720 seltdwait() at seltdwait+0x8b/frame 0xfffffe081e072760 kern_select() at kern_select+0x8a2/frame 0xfffffe081e072960 sys_select() at sys_select+0x54/frame 0xfffffe081e0729a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e072ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e072ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80145a9aa, rsp = 0x7fffffffbd18, rbp = 0x7fffffffc810 --- Tracing command sshd pid 961 tid 100212 td 0xfffff80083273000 sched_switch() at sched_switch+0x482/frame 0xfffffe081e068590 mi_switch() at mi_switch+0x179/frame 0xfffffe081e0685d0 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e068610 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081e068680 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe081e0686b0 _cv_wait_sig() at _cv_wait_sig+0x1e1/frame 0xfffffe081e068720 seltdwait() at seltdwait+0x78/frame 0xfffffe081e068760 kern_select() at kern_select+0x8a2/frame 0xfffffe081e068960 sys_select() at sys_select+0x54/frame 0xfffffe081e0689a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e068ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e068ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8032099aa, rsp = 0x7fffffffd288, rbp = 0x7fffffffdd10 --- Tracing command powerd pid 932 tid 100149 td 0xfffff80073dc0490 sched_switch() at sched_switch+0x482/frame 0xfffffe081df1d580 mi_switch() at mi_switch+0x179/frame 0xfffffe081df1d5c0 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081df1d600 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081df1d670 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe081df1d6a0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x1fd/frame 0xfffffe081df1d720 seltdwait() at seltdwait+0x8b/frame 0xfffffe081df1d760 kern_select() at kern_select+0x8a2/frame 0xfffffe081df1d960 sys_select() at sys_select+0x54/frame 0xfffffe081df1d9a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081df1dab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081df1dab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800b5d9aa, rsp = 0x7fffffffdb88, rbp = 0x7fffffffdd00 --- Tracing command ntpd pid 929 tid 100209 td 0xfffff80073b09000 sched_switch() at sched_switch+0x482/frame 0xfffffe081e0593f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081e059430 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e059470 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081e0594a0 _sleep() at _sleep+0x351/frame 0xfffffe081e059540 vm_waitpfault() at vm_waitpfault+0x72/frame 0xfffffe081e059560 vm_fault_hold() at vm_fault_hold+0xedb/frame 0xfffffe081e0597b0 vm_fault() at vm_fault+0x77/frame 0xfffffe081e0597f0 trap_pfault() at trap_pfault+0x1c2/frame 0xfffffe081e059890 trap() at trap+0x52b/frame 0xfffffe081e059ab0 calltrap() at calltrap+0x8/frame 0xfffffe081e059ab0 --- trap 0xc, rip = 0x8013c8b10, rsp = 0x7fffffffc220, rbp = 0x7fffffffc240 --- Tracing command nfsd pid 892 tid 100208 td 0xfffff8018a74f490 sched_switch() at sched_switch+0x482/frame 0xfffffe081e054730 mi_switch() at mi_switch+0x179/frame 0xfffffe081e054770 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e0547b0 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081e054820 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe081e054850 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x1fd/frame 0xfffffe081e0548d0 svc_run_internal() at svc_run_internal+0x9c7/frame 0xfffffe081e054a60 svc_thread_start() at svc_thread_start+0xb/frame 0xfffffe081e054a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081e054ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081e054ab0 --- trap 0xc, rip = 0x80088bc7a, rsp = 0x7fffffffd5d8, rbp = 0x7fffffffd880 --- Tracing command nfsd pid 892 tid 100207 td 0xfffff8018a74f920 sched_switch() at sched_switch+0x482/frame 0xfffffe081e04f730 mi_switch() at mi_switch+0x179/frame 0xfffffe081e04f770 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e04f7b0 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081e04f820 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe081e04f850 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x1fd/frame 0xfffffe081e04f8d0 svc_run_internal() at svc_run_internal+0x9c7/frame 0xfffffe081e04fa60 svc_thread_start() at svc_thread_start+0xb/frame 0xfffffe081e04fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe081e04fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081e04fab0 --- trap 0xc, rip = 0x80088bc7a, rsp = 0x7fffffffd5d8, rbp = 0x7fffffffd880 --- Tracing command nfsd pid 892 tid 100206 td 0xfffff8018a773000 sched_switch() at sched_switch+0x482/frame 0xfffffe081e04a730 mi_switch() at mi_switch+0x179/frame 0xfffffe081e04a770 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e04a7b0 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081e04a820 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe081e04a850 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x1fd/frame 0xfffffe081e04a8d0 svc_run_internal() at svc_run_internal+0x9c7/frame 0xfffffe081e04aa60 svc_thread_start() at svc_thread_start+0xb/frame 0xfffffe081e04aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe081e04aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081e04aab0 --- trap 0xc, rip = 0x80088bc7a, rsp = 0x7fffffffd5d8, rbp = 0x7fffffffd880 --- Tracing command nfsd pid 892 tid 100183 td 0xfffff80073f74000 sched_switch() at sched_switch+0x482/frame 0xfffffe081dfc6e80 mi_switch() at mi_switch+0x179/frame 0xfffffe081dfc6ec0 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081dfc6f00 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081dfc6f70 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe081dfc6fa0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x1fd/frame 0xfffffe081dfc7020 svc_run_internal() at svc_run_internal+0x9c7/frame 0xfffffe081dfc71b0 svc_run() at svc_run+0xd4/frame 0xfffffe081dfc71f0 nfsrvd_nfsd() at nfsrvd_nfsd+0x194/frame 0xfffffe081dfc7350 nfssvc_nfsd() at nfssvc_nfsd+0x11a/frame 0xfffffe081dfc7970 sys_nfssvc() at sys_nfssvc+0xd2/frame 0xfffffe081dfc79a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081dfc7ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081dfc7ab0 --- syscall (155, FreeBSD ELF64, sys_nfssvc), rip = 0x80088bc7a, rsp = 0x7fffffffd5d8, rbp = 0x7fffffffd880 --- Tracing command nfsd pid 889 tid 100205 td 0xfffff80083273490 sched_switch() at sched_switch+0x482/frame 0xfffffe081e045590 mi_switch() at mi_switch+0x179/frame 0xfffffe081e0455d0 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e045610 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081e045680 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe081e0456b0 _cv_wait_sig() at _cv_wait_sig+0x1e1/frame 0xfffffe081e045720 seltdwait() at seltdwait+0x78/frame 0xfffffe081e045760 kern_select() at kern_select+0x8a2/frame 0xfffffe081e045960 sys_select() at sys_select+0x54/frame 0xfffffe081e0459a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e045ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e045ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80094c9aa, rsp = 0x7fffffffd888, rbp = 0x7fffffffdce0 --- Tracing command mountd pid 883 tid 100142 td 0xfffff80073956000 sched_switch() at sched_switch+0x482/frame 0xfffffe081defa590 mi_switch() at mi_switch+0x179/frame 0xfffffe081defa5d0 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081defa610 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081defa680 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe081defa6b0 _cv_wait_sig() at _cv_wait_sig+0x1e1/frame 0xfffffe081defa720 seltdwait() at seltdwait+0x78/frame 0xfffffe081defa760 kern_select() at kern_select+0x8a2/frame 0xfffffe081defa960 sys_select() at sys_select+0x54/frame 0xfffffe081defa9a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081defaab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081defaab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800b649aa, rsp = 0x7fffffffdb88, rbp = 0x7fffffffdd00 --- Tracing command nfsuserd pid 876 tid 100162 td 0xfffff800739bc000 sched_switch() at sched_switch+0x482/frame 0xfffffe081df5e580 mi_switch() at mi_switch+0x179/frame 0xfffffe081df5e5c0 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081df5e600 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081df5e670 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe081df5e6a0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x1fd/frame 0xfffffe081df5e720 seltdwait() at seltdwait+0x8b/frame 0xfffffe081df5e760 kern_select() at kern_select+0x8a2/frame 0xfffffe081df5e960 sys_select() at sys_select+0x54/frame 0xfffffe081df5e9a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081df5eab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081df5eab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80094b9aa, rsp = 0x7fffffffd9f8, rbp = 0x7fffffffdb30 --- Tracing command nfsuserd pid 875 tid 100203 td 0xfffff80073cd8000 sched_switch() at sched_switch+0x482/frame 0xfffffe081e03b580 mi_switch() at mi_switch+0x179/frame 0xfffffe081e03b5c0 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e03b600 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081e03b670 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe081e03b6a0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x1fd/frame 0xfffffe081e03b720 seltdwait() at seltdwait+0x8b/frame 0xfffffe081e03b760 kern_select() at kern_select+0x8a2/frame 0xfffffe081e03b960 sys_select() at sys_select+0x54/frame 0xfffffe081e03b9a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e03bab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e03bab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80094b9aa, rsp = 0x7fffffffd9f8, rbp = 0x7fffffffdb30 --- Tracing command nfsuserd pid 874 tid 100202 td 0xfffff80073cd8490 sched_switch() at sched_switch+0x482/frame 0xfffffe081e036580 mi_switch() at mi_switch+0x179/frame 0xfffffe081e0365c0 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e036600 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081e036670 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe081e0366a0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x1fd/frame 0xfffffe081e036720 seltdwait() at seltdwait+0x8b/frame 0xfffffe081e036760 kern_select() at kern_select+0x8a2/frame 0xfffffe081e036960 sys_select() at sys_select+0x54/frame 0xfffffe081e0369a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e036ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e036ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80094b9aa, rsp = 0x7fffffffd9f8, rbp = 0x7fffffffdb30 --- Tracing command nfsuserd pid 873 tid 100201 td 0xfffff80073cd8920 sched_switch() at sched_switch+0x482/frame 0xfffffe081e031580 mi_switch() at mi_switch+0x179/frame 0xfffffe081e0315c0 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081e031600 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081e031670 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe081e0316a0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x1fd/frame 0xfffffe081e031720 seltdwait() at seltdwait+0x8b/frame 0xfffffe081e031760 kern_select() at kern_select+0x8a2/frame 0xfffffe081e031960 sys_select() at sys_select+0x54/frame 0xfffffe081e0319a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081e031ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e031ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80094b9aa, rsp = 0x7fffffffd9f8, rbp = 0x7fffffffdb30 --- Tracing command nfsuserd pid 872 tid 100184 td 0xfffff800830a5000 Tracing command amd pid 849 tid 100170 td 0xfffff80083289000 sched_switch() at sched_switch+0x482/frame 0xfffffe081df86580 mi_switch() at mi_switch+0x179/frame 0xfffffe081df865c0 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081df86600 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081df86670 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe081df866a0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x1fd/frame 0xfffffe081df86720 seltdwait() at seltdwait+0x8b/frame 0xfffffe081df86760 kern_select() at kern_select+0x8a2/frame 0xfffffe081df86960 sys_select() at sys_select+0x54/frame 0xfffffe081df869a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081df86ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081df86ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800b7f9aa, rsp = 0x7fffffffd938, rbp = 0x7fffffffd960 --- Tracing command rpcbind pid 838 tid 100138 td 0xfffff80073a72490 sched_switch() at sched_switch+0x482/frame 0xfffffe081dee6620 mi_switch() at mi_switch+0x179/frame 0xfffffe081dee6660 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081dee66a0 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081dee6710 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe081dee6740 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x1fd/frame 0xfffffe081dee67c0 seltdwait() at seltdwait+0x8b/frame 0xfffffe081dee6800 sys_poll() at sys_poll+0x3c9/frame 0xfffffe081dee69a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081dee6ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081dee6ab0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x800d04e7a, rsp = 0x7fffffffba68, rbp = 0x7fffffffdc80 --- Tracing command syslogd pid 823 tid 100191 td 0xfffff80083132920 sched_switch() at sched_switch+0x482/frame 0xfffffe081dffc020 mi_switch() at mi_switch+0x179/frame 0xfffffe081dffc060 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081dffc0a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081dffc0d0 sleeplk() at sleeplk+0x11a/frame 0xfffffe081dffc120 __lockmgr_args() at __lockmgr_args+0xb0a/frame 0xfffffe081dffc250 getblk() at getblk+0x231/frame 0xfffffe081dffc300 breadn_flags() at breadn_flags+0x28/frame 0xfffffe081dffc340 ffs_balloc_ufs2() at ffs_balloc_ufs2+0xd90/frame 0xfffffe081dffc500 ffs_write() at ffs_write+0x310/frame 0xfffffe081dffc5b0 VOP_WRITE_APV() at VOP_WRITE_APV+0x183/frame 0xfffffe081dffc6c0 vn_write() at vn_write+0x2eb/frame 0xfffffe081dffc750 vn_io_fault() at vn_io_fault+0x22a/frame 0xfffffe081dffc8d0 dofilewrite() at dofilewrite+0x85/frame 0xfffffe081dffc920 kern_writev() at kern_writev+0x65/frame 0xfffffe081dffc970 sys_writev() at sys_writev+0x36/frame 0xfffffe081dffc9a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081dffcab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081dffcab0 --- syscall (121, FreeBSD ELF64, sys_writev), rip = 0x800b4a74a, rsp = 0x7fffffffbfc8, rbp = 0x7fffffffc700 --- Tracing command devd pid 691 tid 100165 td 0xfffff80073f0d000 sched_switch() at sched_switch+0x482/frame 0xfffffe081df6d580 mi_switch() at mi_switch+0x179/frame 0xfffffe081df6d5c0 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081df6d600 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081df6d670 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe081df6d6a0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x1fd/frame 0xfffffe081df6d720 seltdwait() at seltdwait+0x8b/frame 0xfffffe081df6d760 kern_select() at kern_select+0x8a2/frame 0xfffffe081df6d960 sys_select() at sys_select+0x54/frame 0xfffffe081df6d9a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081df6dab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081df6dab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x49c2aa, rsp = 0x7fffffffbbb8, rbp = 0x7fffffffdd20 --- Tracing command moused pid 674 tid 100186 td 0xfffff80073962490 sched_switch() at sched_switch+0x482/frame 0xfffffe081dfe3590 mi_switch() at mi_switch+0x179/frame 0xfffffe081dfe35d0 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081dfe3610 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe081dfe3680 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe081dfe36b0 _cv_wait_sig() at _cv_wait_sig+0x1e1/frame 0xfffffe081dfe3720 seltdwait() at seltdwait+0x78/frame 0xfffffe081dfe3760 kern_select() at kern_select+0x8a2/frame 0xfffffe081dfe3960 sys_select() at sys_select+0x54/frame 0xfffffe081dfe39a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe081dfe3ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081dfe3ab0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800d889aa, rsp = 0x7fffffffd6f8, rbp = 0x7fffffffdca0 --- Tracing command softdepflush pid 17 tid 100134 td 0xfffff800730a0920 sched_switch() at sched_switch+0x482/frame 0xfffffe081dbf68e0 mi_switch() at mi_switch+0x179/frame 0xfffffe081dbf6920 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081dbf6960 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe081dbf6990 _sleep() at _sleep+0x326/frame 0xfffffe081dbf6a30 softdep_flush() at softdep_flush+0x20c/frame 0xfffffe081dbf6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081dbf6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081dbf6ab0 --- trap 0, rip = 0, rsp = 0xfffffe081dbf6b70, rbp = 0 --- Tracing command syncer pid 16 tid 100133 td 0xfffff8007314f000 sched_switch() at sched_switch+0x482/frame 0xfffffe081dbf18a0 mi_switch() at mi_switch+0x179/frame 0xfffffe081dbf18e0 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081dbf1920 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe081dbf1950 _cv_timedwait_sbt() at _cv_timedwait_sbt+0x1fd/frame 0xfffffe081dbf19d0 sched_sync() at sched_sync+0x5cb/frame 0xfffffe081dbf1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081dbf1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081dbf1ab0 --- trap 0, rip = 0, rsp = 0xfffffe081dbf1b70, rbp = 0 --- Tracing command vnlru pid 9 tid 100132 td 0xfffff8007314f490 sched_switch() at sched_switch+0x482/frame 0xfffffe081dbec8a0 mi_switch() at mi_switch+0x179/frame 0xfffffe081dbec8e0 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081dbec920 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe081dbec950 _sleep() at _sleep+0x326/frame 0xfffffe081dbec9f0 vnlru_proc() at vnlru_proc+0x48/frame 0xfffffe081dbeca70 fork_exit() at fork_exit+0x84/frame 0xfffffe081dbecab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081dbecab0 --- trap 0, rip = 0, rsp = 0xfffffe081dbecb70, rbp = 0 --- Tracing command bufdaemon pid 8 tid 100131 td 0xfffff8007314f920 sched_switch() at sched_switch+0x482/frame 0xfffffe081dbe78f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081dbe7930 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081dbe7970 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe081dbe79a0 _sleep() at _sleep+0x326/frame 0xfffffe081dbe7a40 buf_daemon() at buf_daemon+0x68/frame 0xfffffe081dbe7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081dbe7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081dbe7ab0 --- trap 0, rip = 0, rsp = 0xfffffe081dbe7b70, rbp = 0 --- Tracing command pagezero pid 7 tid 100130 td 0xfffff80010011920 sched_switch() at sched_switch+0x482/frame 0xfffffe081dbe28f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081dbe2930 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081dbe2970 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe081dbe29a0 _sleep() at _sleep+0x326/frame 0xfffffe081dbe2a40 vm_pagezero() at vm_pagezero+0x88/frame 0xfffffe081dbe2a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081dbe2ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081dbe2ab0 --- trap 0, rip = 0, rsp = 0xfffffe081dbe2b70, rbp = 0 --- Tracing command vmdaemon pid 6 tid 100129 td 0xfffff8007309f000 sched_switch() at sched_switch+0x482/frame 0xfffffe081dbdd8b0 mi_switch() at mi_switch+0x179/frame 0xfffffe081dbdd8f0 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081dbdd930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081dbdd960 _sleep() at _sleep+0x351/frame 0xfffffe081dbdda00 vm_daemon() at vm_daemon+0x7e/frame 0xfffffe081dbdda70 fork_exit() at fork_exit+0x84/frame 0xfffffe081dbddab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081dbddab0 --- trap 0, rip = 0, rsp = 0xfffffe081dbddb70, rbp = 0 --- Tracing command pagedaemon pid 5 tid 100128 td 0xfffff8007309f490 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc379ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc379d00 trap() at trap+0x42/frame 0xfffffe07fc379f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc379f20 --- trap 0x13, rip = 0xffffffff80db9706, rsp = 0xfffffe07fc379fe0, rbp = 0xfffffe081dbd87c0 --- DELAY() at DELAY+0x176/frame 0xfffffe081dbd87c0 _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0x1ca/frame 0xfffffe081dbd8830 __mtx_lock_spin_flags() at __mtx_lock_spin_flags+0xe8/frame 0xfffffe081dbd8870 smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0xbe/frame 0xfffffe081dbd88d0 pmap_invalidate_page() at pmap_invalidate_page+0x395/frame 0xfffffe081dbd8950 pmap_ts_referenced() at pmap_ts_referenced+0x80b/frame 0xfffffe081dbd89f0 vm_pageout() at vm_pageout+0x7a4/frame 0xfffffe081dbd8a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081dbd8ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081dbd8ab0 --- trap 0, rip = 0, rsp = 0xfffffe081dbd8b70, rbp = 0 --- Tracing command enc_daemon0 pid 4 tid 100127 td 0xfffff8007309f920 sched_switch() at sched_switch+0x482/frame 0xfffffe081dbd38b0 mi_switch() at mi_switch+0x179/frame 0xfffffe081dbd38f0 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081dbd3930 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081dbd3960 _sleep() at _sleep+0x351/frame 0xfffffe081dbd3a00 enc_daemon() at enc_daemon+0x109/frame 0xfffffe081dbd3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081dbd3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081dbd3ab0 --- trap 0, rip = 0, rsp = 0xfffffe081dbd3b70, rbp = 0 --- Tracing command xpt_thrd pid 3 tid 100126 td 0xfffff800730a0000 sched_switch() at sched_switch+0x482/frame 0xfffffe081db4e8f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081db4e930 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081db4e970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081db4e9a0 _sleep() at _sleep+0x351/frame 0xfffffe081db4ea40 xpt_scanner_thread() at xpt_scanner_thread+0xdc/frame 0xfffffe081db4ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe081db4eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081db4eab0 --- trap 0, rip = 0, rsp = 0xfffffe081db4eb70, rbp = 0 --- Tracing command sctp_iterator pid 2 tid 100125 td 0xfffff8001000d000 sched_switch() at sched_switch+0x482/frame 0xfffffe081db49900 mi_switch() at mi_switch+0x179/frame 0xfffffe081db49940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081db49980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081db499b0 _sleep() at _sleep+0x351/frame 0xfffffe081db49a50 sctp_iterator_thread() at sctp_iterator_thread+0x59/frame 0xfffffe081db49a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081db49ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081db49ab0 --- trap 0, rip = 0, rsp = 0xfffffe081db49b70, rbp = 0 --- Tracing command usb pid 15 tid 100119 td 0xfffff8000c8a4920 sched_switch() at sched_switch+0x482/frame 0xfffffe081baa6920 mi_switch() at mi_switch+0x179/frame 0xfffffe081baa6960 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081baa69a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081baa69d0 _cv_wait() at _cv_wait+0x1dc/frame 0xfffffe081baa6a40 usb_process() at usb_process+0xd4/frame 0xfffffe081baa6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081baa6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081baa6ab0 --- trap 0, rip = 0, rsp = 0xfffffe081baa6b70, rbp = 0 --- Tracing command usb pid 15 tid 100118 td 0xfffff8000c8b6000 sched_switch() at sched_switch+0x482/frame 0xfffffe081baa1920 mi_switch() at mi_switch+0x179/frame 0xfffffe081baa1960 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081baa19a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081baa19d0 _cv_wait() at _cv_wait+0x1dc/frame 0xfffffe081baa1a40 usb_process() at usb_process+0xd4/frame 0xfffffe081baa1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081baa1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081baa1ab0 --- trap 0, rip = 0, rsp = 0xfffffe081baa1b70, rbp = 0 --- Tracing command usb pid 15 tid 100117 td 0xfffff8000c8b6490 sched_switch() at sched_switch+0x482/frame 0xfffffe081ba9c920 mi_switch() at mi_switch+0x179/frame 0xfffffe081ba9c960 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081ba9c9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081ba9c9d0 _cv_wait() at _cv_wait+0x1dc/frame 0xfffffe081ba9ca40 usb_process() at usb_process+0xd4/frame 0xfffffe081ba9ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe081ba9cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081ba9cab0 --- trap 0, rip = 0, rsp = 0xfffffe081ba9cb70, rbp = 0 --- Tracing command usb pid 15 tid 100116 td 0xfffff8000c8b6920 sched_switch() at sched_switch+0x482/frame 0xfffffe081ba97920 mi_switch() at mi_switch+0x179/frame 0xfffffe081ba97960 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081ba979a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081ba979d0 _cv_wait() at _cv_wait+0x1dc/frame 0xfffffe081ba97a40 usb_process() at usb_process+0xd4/frame 0xfffffe081ba97a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081ba97ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081ba97ab0 --- trap 0, rip = 0, rsp = 0xfffffe081ba97b70, rbp = 0 --- Tracing command usb pid 15 tid 100114 td 0xfffff8000c8a3000 sched_switch() at sched_switch+0x482/frame 0xfffffe081b90a920 mi_switch() at mi_switch+0x179/frame 0xfffffe081b90a960 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081b90a9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081b90a9d0 _cv_wait() at _cv_wait+0x1dc/frame 0xfffffe081b90aa40 usb_process() at usb_process+0xd4/frame 0xfffffe081b90aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe081b90aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081b90aab0 --- trap 0, rip = 0, rsp = 0xfffffe081b90ab70, rbp = 0 --- Tracing command usb pid 15 tid 100113 td 0xfffff8000c8a3490 sched_switch() at sched_switch+0x482/frame 0xfffffe081b905920 mi_switch() at mi_switch+0x179/frame 0xfffffe081b905960 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081b9059a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081b9059d0 _cv_wait() at _cv_wait+0x1dc/frame 0xfffffe081b905a40 usb_process() at usb_process+0xd4/frame 0xfffffe081b905a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081b905ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081b905ab0 --- trap 0, rip = 0, rsp = 0xfffffe081b905b70, rbp = 0 --- Tracing command usb pid 15 tid 100112 td 0xfffff8000c8a3920 sched_switch() at sched_switch+0x482/frame 0xfffffe081b900920 mi_switch() at mi_switch+0x179/frame 0xfffffe081b900960 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081b9009a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081b9009d0 _cv_wait() at _cv_wait+0x1dc/frame 0xfffffe081b900a40 usb_process() at usb_process+0xd4/frame 0xfffffe081b900a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081b900ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081b900ab0 --- trap 0, rip = 0, rsp = 0xfffffe081b900b70, rbp = 0 --- Tracing command usb pid 15 tid 100111 td 0xfffff8000c8a4000 sched_switch() at sched_switch+0x482/frame 0xfffffe081b8fb920 mi_switch() at mi_switch+0x179/frame 0xfffffe081b8fb960 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081b8fb9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081b8fb9d0 _cv_wait() at _cv_wait+0x1dc/frame 0xfffffe081b8fba40 usb_process() at usb_process+0xd4/frame 0xfffffe081b8fba70 fork_exit() at fork_exit+0x84/frame 0xfffffe081b8fbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081b8fbab0 --- trap 0, rip = 0, rsp = 0xfffffe081b8fbb70, rbp = 0 --- Tracing command usb pid 15 tid 100074 td 0xfffff8000b422920 sched_switch() at sched_switch+0x482/frame 0xfffffe0000396920 mi_switch() at mi_switch+0x179/frame 0xfffffe0000396960 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe00003969a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00003969d0 _cv_wait() at _cv_wait+0x1dc/frame 0xfffffe0000396a40 usb_process() at usb_process+0xd4/frame 0xfffffe0000396a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000396ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000396ab0 --- trap 0, rip = 0, rsp = 0xfffffe0000396b70, rbp = 0 --- Tracing command usb pid 15 tid 100073 td 0xfffff8000c852000 sched_switch() at sched_switch+0x482/frame 0xfffffe0000391920 mi_switch() at mi_switch+0x179/frame 0xfffffe0000391960 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe00003919a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00003919d0 _cv_wait() at _cv_wait+0x1dc/frame 0xfffffe0000391a40 usb_process() at usb_process+0xd4/frame 0xfffffe0000391a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000391ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000391ab0 --- trap 0, rip = 0, rsp = 0xfffffe0000391b70, rbp = 0 --- Tracing command usb pid 15 tid 100072 td 0xfffff8000c852490 sched_switch() at sched_switch+0x482/frame 0xfffffe000038c920 mi_switch() at mi_switch+0x179/frame 0xfffffe000038c960 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe000038c9a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe000038c9d0 _cv_wait() at _cv_wait+0x1dc/frame 0xfffffe000038ca40 usb_process() at usb_process+0xd4/frame 0xfffffe000038ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe000038cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000038cab0 --- trap 0, rip = 0, rsp = 0xfffffe000038cb70, rbp = 0 --- Tracing command usb pid 15 tid 100071 td 0xfffff8000c852920 sched_switch() at sched_switch+0x482/frame 0xfffffe0000387920 mi_switch() at mi_switch+0x179/frame 0xfffffe0000387960 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe00003879a0 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00003879d0 _cv_wait() at _cv_wait+0x1dc/frame 0xfffffe0000387a40 usb_process() at usb_process+0xd4/frame 0xfffffe0000387a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000387ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000387ab0 --- trap 0, rip = 0, rsp = 0xfffffe0000387b70, rbp = 0 --- Tracing command rand_harvestq pid 14 tid 100057 td 0xfffff8000b27f490 sched_switch() at sched_switch+0x482/frame 0xfffffe078aa548e0 mi_switch() at mi_switch+0x179/frame 0xfffffe078aa54920 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe078aa54960 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe078aa54990 msleep_spin_sbt() at msleep_spin_sbt+0x1f0/frame 0xfffffe078aa54a20 random_kthread() at random_kthread+0x1b2/frame 0xfffffe078aa54a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078aa54ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078aa54ab0 --- trap 0, rip = 0, rsp = 0xfffffe078aa54b70, rbp = 0 --- Tracing command geom pid 13 tid 100055 td 0xfffff8000b26b920 sched_switch() at sched_switch+0x482/frame 0xfffffe078aa4a8c0 mi_switch() at mi_switch+0x179/frame 0xfffffe078aa4a900 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe078aa4a940 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe078aa4a970 _sleep() at _sleep+0x351/frame 0xfffffe078aa4aa10 g_io_schedule_down() at g_io_schedule_down+0x4c/frame 0xfffffe078aa4aa60 g_down_procbody() at g_down_procbody+0x7d/frame 0xfffffe078aa4aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe078aa4aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078aa4aab0 --- trap 0, rip = 0, rsp = 0xfffffe078aa4ab70, rbp = 0 --- Tracing command geom pid 13 tid 100054 td 0xfffff8000b26c000 sched_switch() at sched_switch+0x482/frame 0xfffffe078aa458e0 mi_switch() at mi_switch+0x179/frame 0xfffffe078aa45920 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe078aa45960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe078aa45990 _sleep() at _sleep+0x351/frame 0xfffffe078aa45a30 g_io_schedule_up() at g_io_schedule_up+0xbc/frame 0xfffffe078aa45a60 g_up_procbody() at g_up_procbody+0x7d/frame 0xfffffe078aa45a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078aa45ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078aa45ab0 --- trap 0, rip = 0, rsp = 0xfffffe078aa45b70, rbp = 0 --- Tracing command geom pid 13 tid 100053 td 0xfffff8000b26c490 sched_switch() at sched_switch+0x482/frame 0xfffffe078aa408e0 mi_switch() at mi_switch+0x179/frame 0xfffffe078aa40920 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe078aa40960 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe078aa40990 _sleep() at _sleep+0x351/frame 0xfffffe078aa40a30 g_run_events() at g_run_events+0x4f4/frame 0xfffffe078aa40a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078aa40ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078aa40ab0 --- trap 0, rip = 0, rsp = 0xfffffe078aa40b70, rbp = 0 --- Tracing command intr pid 12 tid 100122 td 0xfffff80010011000 sched_switch() at sched_switch+0x482/frame 0xfffffe081db359f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081db35a30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe081db35a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081db35ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081db35ab0 --- trap 0, rip = 0, rsp = 0xfffffe081db35b70, rbp = 0 --- Tracing command intr pid 12 tid 100121 td 0xfffff80010011490 sched_switch() at sched_switch+0x482/frame 0xfffffe081db2d9f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081db2da30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe081db2da70 fork_exit() at fork_exit+0x84/frame 0xfffffe081db2dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081db2dab0 --- trap 0, rip = 0, rsp = 0xfffffe081db2db70, rbp = 0 --- Tracing command intr pid 12 tid 100120 td 0xfffff8000c8a4490 sched_switch() at sched_switch+0x482/frame 0xfffffe081baab9f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081baaba30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe081baaba70 fork_exit() at fork_exit+0x84/frame 0xfffffe081baabab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081baabab0 --- trap 0, rip = 0, rsp = 0xfffffe081baabb70, rbp = 0 --- Tracing command intr pid 12 tid 100115 td 0xfffff8000c89b920 sched_switch() at sched_switch+0x482/frame 0xfffffe081ba929f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081ba92a30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe081ba92a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081ba92ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081ba92ab0 --- trap 0, rip = 0, rsp = 0xfffffe081ba92b70, rbp = 0 --- Tracing command intr pid 12 tid 100110 td 0xfffff8000c89a000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100109 td 0xfffff8000c89a490 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100107 td 0xfffff8000c89b000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100105 td 0xfffff8000c894490 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100103 td 0xfffff8000c895000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100101 td 0xfffff8000c895920 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100099 td 0xfffff8000c893000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100097 td 0xfffff8000c893920 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100095 td 0xfffff8000c875000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100093 td 0xfffff8000c875920 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100092 td 0xfffff8000c87a000 sched_switch() at sched_switch+0x482/frame 0xfffffe081b83b9f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081b83ba30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe081b83ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe081b83bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081b83bab0 --- trap 0, rip = 0, rsp = 0xfffffe081b83bb70, rbp = 0 --- Tracing command intr pid 12 tid 100090 td 0xfffff8000c872490 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc381ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc381d00 trap() at trap+0x42/frame 0xfffffe07fc381f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc381f20 --- trap 0x13, rip = 0xffffffff808a741f, rsp = 0xfffffe07fc381fe0, rbp = 0xfffffe081b831830 --- __mtx_lock_sleep() at __mtx_lock_sleep+0x24f/frame 0xfffffe081b831830 __mtx_lock_flags() at __mtx_lock_flags+0xf5/frame 0xfffffe081b831880 zone_release() at zone_release+0x50/frame 0xfffffe081b8318d0 uma_zfree_arg() at uma_zfree_arg+0x337/frame 0xfffffe081b831930 m_freem() at m_freem+0x63/frame 0xfffffe081b831950 igb_txeof() at igb_txeof+0x1b7/frame 0xfffffe081b8319a0 igb_msix_que() at igb_msix_que+0x95/frame 0xfffffe081b8319f0 intr_event_execute_handlers() at intr_event_execute_handlers+0x93/frame 0xfffffe081b831a30 ithread_loop() at ithread_loop+0xa6/frame 0xfffffe081b831a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081b831ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081b831ab0 --- trap 0, rip = 0, rsp = 0xfffffe081b831b70, rbp = 0 --- Tracing command intr pid 12 tid 100088 td 0xfffff8000c874000 sched_switch() at sched_switch+0x482/frame 0xfffffe081b8279f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081b827a30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe081b827a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081b827ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081b827ab0 --- trap 0, rip = 0, rsp = 0xfffffe081b827b70, rbp = 0 --- Tracing command intr pid 12 tid 100086 td 0xfffff8000c874920 kdb_enter() at kdb_enter+0x3e/frame 0xfffffe081b81d6c0 hardclock_cnt() at hardclock_cnt+0x297/frame 0xfffffe081b81d730 handleevents() at handleevents+0xbe/frame 0xfffffe081b81d780 timercb() at timercb+0x208/frame 0xfffffe081b81d7e0 lapic_handle_timer() at lapic_handle_timer+0x9c/frame 0xfffffe081b81d810 Xtimerint() at Xtimerint+0x8c/frame 0xfffffe081b81d810 --- interrupt, rip = 0xffffffff808a73f2, rsp = 0xfffffe081b81d8d0, rbp = 0xfffffe081b81d950 --- __mtx_lock_sleep() at __mtx_lock_sleep+0x222/frame 0xfffffe081b81d950 __mtx_lock_flags() at __mtx_lock_flags+0xf5/frame 0xfffffe081b81d9a0 igb_msix_que() at igb_msix_que+0x8d/frame 0xfffffe081b81d9f0 intr_event_execute_handlers() at intr_event_execute_handlers+0x93/frame 0xfffffe081b81da30 ithread_loop() at ithread_loop+0xa6/frame 0xfffffe081b81da70 fork_exit() at fork_exit+0x84/frame 0xfffffe081b81dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081b81dab0 --- trap 0, rip = 0, rsp = 0xfffffe081b81db70, rbp = 0 --- Tracing command intr pid 12 tid 100084 td 0xfffff8000c871000 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc369ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc369d00 trap() at trap+0x42/frame 0xfffffe07fc369f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc369f20 --- trap 0x13, rip = 0xffffffff808a73f9, rsp = 0xfffffe07fc369fe0, rbp = 0xfffffe081b8136f0 --- __mtx_lock_sleep() at __mtx_lock_sleep+0x229/frame 0xfffffe081b8136f0 __mtx_lock_flags() at __mtx_lock_flags+0xf5/frame 0xfffffe081b813740 uma_dbg_getslab() at uma_dbg_getslab+0x5b/frame 0xfffffe081b813770 uma_dbg_alloc() at uma_dbg_alloc+0x36/frame 0xfffffe081b8137a0 zone_alloc_item() at zone_alloc_item+0xf4/frame 0xfffffe081b8137e0 uma_zalloc_arg() at uma_zalloc_arg+0x5d7/frame 0xfffffe081b813850 m_getjcl() at m_getjcl+0x35/frame 0xfffffe081b813890 igb_refresh_mbufs() at igb_refresh_mbufs+0x158/frame 0xfffffe081b813920 igb_rxeof() at igb_rxeof+0x624/frame 0xfffffe081b8139a0 igb_msix_que() at igb_msix_que+0xe1/frame 0xfffffe081b8139f0 intr_event_execute_handlers() at intr_event_execute_handlers+0x93/frame 0xfffffe081b813a30 ithread_loop() at ithread_loop+0xa6/frame 0xfffffe081b813a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081b813ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081b813ab0 --- trap 0, rip = 0, rsp = 0xfffffe081b813b70, rbp = 0 --- Tracing command intr pid 12 tid 100082 td 0xfffff8000c871920 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc361ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc361d00 trap() at trap+0x42/frame 0xfffffe07fc361f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc361f20 --- trap 0x13, rip = 0xffffffff808a741f, rsp = 0xfffffe07fc361fe0, rbp = 0xfffffe081b8096f0 --- __mtx_lock_sleep() at __mtx_lock_sleep+0x24f/frame 0xfffffe081b8096f0 __mtx_lock_flags() at __mtx_lock_flags+0xf5/frame 0xfffffe081b809740 uma_dbg_getslab() at uma_dbg_getslab+0x5b/frame 0xfffffe081b809770 uma_dbg_alloc() at uma_dbg_alloc+0x36/frame 0xfffffe081b8097a0 zone_alloc_item() at zone_alloc_item+0xf4/frame 0xfffffe081b8097e0 uma_zalloc_arg() at uma_zalloc_arg+0x5d7/frame 0xfffffe081b809850 m_getjcl() at m_getjcl+0x35/frame 0xfffffe081b809890 igb_refresh_mbufs() at igb_refresh_mbufs+0x158/frame 0xfffffe081b809920 igb_rxeof() at igb_rxeof+0x624/frame 0xfffffe081b8099a0 igb_msix_que() at igb_msix_que+0xe1/frame 0xfffffe081b8099f0 intr_event_execute_handlers() at intr_event_execute_handlers+0x93/frame 0xfffffe081b809a30 ithread_loop() at ithread_loop+0xa6/frame 0xfffffe081b809a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081b809ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081b809ab0 --- trap 0, rip = 0, rsp = 0xfffffe081b809b70, rbp = 0 --- Tracing command intr pid 12 tid 100080 td 0xfffff8000c86f000 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc359ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc359d00 trap() at trap+0x42/frame 0xfffffe07fc359f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc359f20 --- trap 0x13, rip = 0xffffffff808a741f, rsp = 0xfffffe07fc359fe0, rbp = 0xfffffe00003fe830 --- __mtx_lock_sleep() at __mtx_lock_sleep+0x24f/frame 0xfffffe00003fe830 __mtx_lock_flags() at __mtx_lock_flags+0xf5/frame 0xfffffe00003fe880 zone_release() at zone_release+0x50/frame 0xfffffe00003fe8d0 uma_zfree_arg() at uma_zfree_arg+0x337/frame 0xfffffe00003fe930 m_freem() at m_freem+0x63/frame 0xfffffe00003fe950 igb_txeof() at igb_txeof+0x1b7/frame 0xfffffe00003fe9a0 igb_msix_que() at igb_msix_que+0x95/frame 0xfffffe00003fe9f0 intr_event_execute_handlers() at intr_event_execute_handlers+0x93/frame 0xfffffe00003fea30 ithread_loop() at ithread_loop+0xa6/frame 0xfffffe00003fea70 fork_exit() at fork_exit+0x84/frame 0xfffffe00003feab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00003feab0 --- trap 0, rip = 0, rsp = 0xfffffe00003feb70, rbp = 0 --- Tracing command intr pid 12 tid 100078 td 0xfffff8000c86f920 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc351ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc351d00 trap() at trap+0x42/frame 0xfffffe07fc351f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc351f20 --- trap 0x13, rip = 0xffffffff808a7105, rsp = 0xfffffe07fc351fe0, rbp = 0xfffffe00003f4880 --- __mtx_lock_flags() at __mtx_lock_flags+0xb5/frame 0xfffffe00003f4880 uma_dbg_getslab() at uma_dbg_getslab+0x5b/frame 0xfffffe00003f48b0 uma_dbg_free() at uma_dbg_free+0x33/frame 0xfffffe00003f48d0 uma_zfree_arg() at uma_zfree_arg+0x42/frame 0xfffffe00003f4930 m_freem() at m_freem+0x63/frame 0xfffffe00003f4950 igb_txeof() at igb_txeof+0x1b7/frame 0xfffffe00003f49a0 igb_msix_que() at igb_msix_que+0x95/frame 0xfffffe00003f49f0 intr_event_execute_handlers() at intr_event_execute_handlers+0x93/frame 0xfffffe00003f4a30 ithread_loop() at ithread_loop+0xa6/frame 0xfffffe00003f4a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00003f4ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00003f4ab0 --- trap 0, rip = 0, rsp = 0xfffffe00003f4b70, rbp = 0 --- Tracing command intr pid 12 tid 100076 td 0xfffff8000c870490 sched_switch() at sched_switch+0x482/frame 0xfffffe00003ea9f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00003eaa30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe00003eaa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00003eaab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00003eaab0 --- trap 0, rip = 0, rsp = 0xfffffe00003eab70, rbp = 0 --- Tracing command intr pid 12 tid 100075 td 0xfffff8000b422490 sched_switch() at sched_switch+0x482/frame 0xfffffe000039b9f0 mi_switch() at mi_switch+0x179/frame 0xfffffe000039ba30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe000039ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe000039bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000039bab0 --- trap 0, rip = 0, rsp = 0xfffffe000039bb70, rbp = 0 --- Tracing command intr pid 12 tid 100070 td 0xfffff8000b41c920 sched_switch() at sched_switch+0x482/frame 0xfffffe00003829f0 mi_switch() at mi_switch+0x179/frame 0xfffffe0000382a30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe0000382a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000382ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000382ab0 --- trap 0, rip = 0, rsp = 0xfffffe0000382b70, rbp = 0 --- Tracing command intr pid 12 tid 100069 td 0xfffff8000b421000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100068 td 0xfffff8000b421490 sched_switch() at sched_switch+0x482/frame 0xfffffe078ab989f0 mi_switch() at mi_switch+0x179/frame 0xfffffe078ab98a30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe078ab98a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078ab98ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078ab98ab0 --- trap 0, rip = 0, rsp = 0xfffffe078ab98b70, rbp = 0 --- Tracing command intr pid 12 tid 100066 td 0xfffff8000b422000 sched_switch() at sched_switch+0x482/frame 0xfffffe078aa819f0 mi_switch() at mi_switch+0x179/frame 0xfffffe078aa81a30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe078aa81a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078aa81ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078aa81ab0 --- trap 0, rip = 0, rsp = 0xfffffe078aa81b70, rbp = 0 --- Tracing command intr pid 12 tid 100065 td 0xfffff8000b414000 sched_switch() at sched_switch+0x482/frame 0xfffffe078aa7c9f0 mi_switch() at mi_switch+0x179/frame 0xfffffe078aa7ca30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe078aa7ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe078aa7cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078aa7cab0 --- trap 0, rip = 0, rsp = 0xfffffe078aa7cb70, rbp = 0 --- Tracing command intr pid 12 tid 100060 td 0xfffff8000b26d490 sched_switch() at sched_switch+0x482/frame 0xfffffe078aa639f0 mi_switch() at mi_switch+0x179/frame 0xfffffe078aa63a30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe078aa63a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078aa63ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078aa63ab0 --- trap 0, rip = 0, rsp = 0xfffffe078aa63b70, rbp = 0 --- Tracing command intr pid 12 tid 100059 td 0xfffff8000b26d920 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100052 td 0xfffff8000b26c920 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100051 td 0xfffff8000b26d000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100050 td 0xfffff8000b26a000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100049 td 0xfffff8000b26a490 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100048 td 0xfffff8000b26a920 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100047 td 0xfffff8000b26b000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100046 td 0xfffff8000b26b490 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100045 td 0xfffff8000b262490 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100044 td 0xfffff8000b262920 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100043 td 0xfffff8000b263000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100042 td 0xfffff8000b263490 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100041 td 0xfffff8000b263920 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100040 td 0xfffff8000b25e920 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100039 td 0xfffff8000b261000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100038 td 0xfffff8000b261490 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100037 td 0xfffff8000b261920 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100036 td 0xfffff8000b262000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100035 td 0xfffff8000b25d000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100034 td 0xfffff8000b25d490 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100033 td 0xfffff8000b25d920 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100032 td 0xfffff8000b25e000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100031 td 0xfffff8000b25e490 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100030 td 0xfffff8000b230490 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100029 td 0xfffff8000b230920 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d9c9f0 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d9ca30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe0785d9ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe0785d9cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0785d9cab0 --- trap 0, rip = 0, rsp = 0xfffffe0785d9cb70, rbp = 0 --- Tracing command intr pid 12 tid 100028 td 0xfffff8000b231000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100027 td 0xfffff8000b231490 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d929f0 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d92a30 ithread_loop() at ithread_loop+0x1d2/frame 0xfffffe0785d92a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0785d92ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0785d92ab0 --- trap 0, rip = 0, rsp = 0xfffffe0785d92b70, rbp = 0 --- Tracing command idle pid 11 tid 100026 td 0xfffff8000b231920 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d8d980 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d8d9c0 sched_idletd() at sched_idletd+0x3dd/frame 0xfffffe0785d8da70 fork_exit() at fork_exit+0x84/frame 0xfffffe0785d8dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0785d8dab0 --- trap 0, rip = 0, rsp = 0xfffffe0785d8db70, rbp = 0 --- Tracing command idle pid 11 tid 100025 td 0xfffff8000b22c920 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d88980 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d889c0 sched_idletd() at sched_idletd+0x3dd/frame 0xfffffe0785d88a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0785d88ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0785d88ab0 --- trap 0, rip = 0, rsp = 0xfffffe0785d88b70, rbp = 0 --- Tracing command idle pid 11 tid 100024 td 0xfffff8000b22f000 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d83980 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d839c0 sched_idletd() at sched_idletd+0x3dd/frame 0xfffffe0785d83a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0785d83ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0785d83ab0 --- trap 0, rip = 0, rsp = 0xfffffe0785d83b70, rbp = 0 --- Tracing command idle pid 11 tid 100023 td 0xfffff8000b22f490 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d7e980 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d7e9c0 sched_idletd() at sched_idletd+0x3dd/frame 0xfffffe0785d7ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe0785d7eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0785d7eab0 --- trap 0, rip = 0, rsp = 0xfffffe0785d7eb70, rbp = 0 --- Tracing command idle pid 11 tid 100022 td 0xfffff8000b22f920 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d79980 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d799c0 sched_idletd() at sched_idletd+0x3dd/frame 0xfffffe0785d79a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0785d79ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0785d79ab0 --- trap 0, rip = 0, rsp = 0xfffffe0785d79b70, rbp = 0 --- Tracing command idle pid 11 tid 100021 td 0xfffff8000b230000 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d74980 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d749c0 sched_idletd() at sched_idletd+0x3dd/frame 0xfffffe0785d74a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0785d74ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0785d74ab0 --- trap 0, rip = 0, rsp = 0xfffffe0785d74b70, rbp = 0 --- Tracing command idle pid 11 tid 100020 td 0xfffff8000b22b000 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d6f980 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d6f9c0 sched_idletd() at sched_idletd+0x3dd/frame 0xfffffe0785d6fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0785d6fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0785d6fab0 --- trap 0, rip = 0, rsp = 0xfffffe0785d6fb70, rbp = 0 --- Tracing command idle pid 11 tid 100019 td 0xfffff8000b22b490 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d6a980 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d6a9c0 sched_idletd() at sched_idletd+0x3dd/frame 0xfffffe0785d6aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0785d6aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0785d6aab0 --- trap 0, rip = 0, rsp = 0xfffffe0785d6ab70, rbp = 0 --- Tracing command idle pid 11 tid 100018 td 0xfffff8000b22b920 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d65980 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d659c0 sched_idletd() at sched_idletd+0x3dd/frame 0xfffffe0785d65a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0785d65ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0785d65ab0 --- trap 0, rip = 0, rsp = 0xfffffe0785d65b70, rbp = 0 --- Tracing command idle pid 11 tid 100017 td 0xfffff8000b22c000 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d60980 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d609c0 sched_idletd() at sched_idletd+0x3dd/frame 0xfffffe0785d60a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0785d60ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0785d60ab0 --- trap 0, rip = 0, rsp = 0xfffffe0785d60b70, rbp = 0 --- Tracing command idle pid 11 tid 100016 td 0xfffff8000b22c490 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d5b980 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d5b9c0 sched_idletd() at sched_idletd+0x3dd/frame 0xfffffe0785d5ba70 fork_exit() at fork_exit+0x84/frame 0xfffffe0785d5bab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0785d5bab0 --- trap 0, rip = 0, rsp = 0xfffffe0785d5bb70, rbp = 0 --- Tracing command idle pid 11 tid 100015 td 0xfffff8000b229490 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d56980 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d569c0 sched_idletd() at sched_idletd+0x3dd/frame 0xfffffe0785d56a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0785d56ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0785d56ab0 --- trap 0, rip = 0, rsp = 0xfffffe0785d56b70, rbp = 0 --- Tracing command idle pid 11 tid 100014 td 0xfffff8000b229920 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d51980 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d519c0 sched_idletd() at sched_idletd+0x3dd/frame 0xfffffe0785d51a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0785d51ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0785d51ab0 --- trap 0, rip = 0, rsp = 0xfffffe0785d51b70, rbp = 0 --- Tracing command idle pid 11 tid 100013 td 0xfffff8000b22a000 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d4c980 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d4c9c0 sched_idletd() at sched_idletd+0x3dd/frame 0xfffffe0785d4ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe0785d4cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0785d4cab0 --- trap 0, rip = 0, rsp = 0xfffffe0785d4cb70, rbp = 0 --- Tracing command idle pid 11 tid 100012 td 0xfffff8000b22a490 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d47980 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d479c0 sched_idletd() at sched_idletd+0x3dd/frame 0xfffffe0785d47a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0785d47ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0785d47ab0 --- trap 0, rip = 0, rsp = 0xfffffe0785d47b70, rbp = 0 --- Tracing command idle pid 11 tid 100011 td 0xfffff8000b22a920 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d42980 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d429c0 sched_idletd() at sched_idletd+0x3dd/frame 0xfffffe0785d42a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0785d42ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0785d42ab0 --- trap 0, rip = 0, rsp = 0xfffffe0785d42b70, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xfffff8000b218920 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d3d980 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d3d9c0 sched_idletd() at sched_idletd+0x3dd/frame 0xfffffe0785d3da70 fork_exit() at fork_exit+0x84/frame 0xfffffe0785d3dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0785d3dab0 --- trap 0, rip = 0, rsp = 0xfffffe0785d3db70, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xfffff8000b228000 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d38980 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d389c0 sched_idletd() at sched_idletd+0x3dd/frame 0xfffffe0785d38a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0785d38ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0785d38ab0 --- trap 0, rip = 0, rsp = 0xfffffe0785d38b70, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xfffff8000b228490 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d33980 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d339c0 sched_idletd() at sched_idletd+0x3dd/frame 0xfffffe0785d33a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0785d33ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0785d33ab0 --- trap 0, rip = 0, rsp = 0xfffffe0785d33b70, rbp = 0 --- Tracing command idle pid 11 tid 100007 td 0xfffff8000b228920 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d2e980 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d2e9c0 sched_idletd() at sched_idletd+0x3dd/frame 0xfffffe0785d2ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe0785d2eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0785d2eab0 --- trap 0, rip = 0, rsp = 0xfffffe0785d2eb70, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xfffff8000b229000 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d29980 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d299c0 sched_idletd() at sched_idletd+0x3dd/frame 0xfffffe0785d29a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0785d29ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0785d29ab0 --- trap 0, rip = 0, rsp = 0xfffffe0785d29b70, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xfffff8000b212000 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d24980 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d249c0 sched_idletd() at sched_idletd+0x3dd/frame 0xfffffe0785d24a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0785d24ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0785d24ab0 --- trap 0, rip = 0, rsp = 0xfffffe0785d24b70, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xfffff8000b212490 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d1f980 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d1f9c0 sched_idletd() at sched_idletd+0x3dd/frame 0xfffffe0785d1fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0785d1fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0785d1fab0 --- trap 0, rip = 0, rsp = 0xfffffe0785d1fb70, rbp = 0 --- Tracing command idle pid 11 tid 100003 td 0xfffff8000b212920 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d1a980 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d1a9c0 sched_idletd() at sched_idletd+0x3dd/frame 0xfffffe0785d1aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe0785d1aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0785d1aab0 --- trap 0, rip = 0, rsp = 0xfffffe0785d1ab70, rbp = 0 --- Tracing command init pid 1 tid 100002 td 0xfffff8000b218000 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d15550 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d15590 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe0785d155d0 sleepq_catch_signals() at sleepq_catch_signals+0x2f4/frame 0xfffffe0785d15640 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0785d15670 _sleep() at _sleep+0x334/frame 0xfffffe0785d15710 kern_wait6() at kern_wait6+0x3e7/frame 0xfffffe0785d157b0 sys_wait4() at sys_wait4+0x76/frame 0xfffffe0785d159a0 amd64_syscall() at amd64_syscall+0x282/frame 0xfffffe0785d15ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0785d15ab0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x41323a, rsp = 0x7fffffffd758, rbp = 0x7fffffffd7c0 --- Tracing command audit pid 10 tid 100001 td 0xfffff8000b218490 sched_switch() at sched_switch+0x482/frame 0xfffffe0785d108f0 mi_switch() at mi_switch+0x179/frame 0xfffffe0785d10930 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe0785d10970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe0785d109a0 _cv_wait() at _cv_wait+0x1dc/frame 0xfffffe0785d10a10 audit_worker() at audit_worker+0xa3/frame 0xfffffe0785d10a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0785d10ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0785d10ab0 --- trap 0, rip = 0, rsp = 0xfffffe0785d10b70, rbp = 0 --- Tracing command kernel pid 0 tid 100124 td 0xfffff8001000d490 sched_switch() at sched_switch+0x482/frame 0xfffffe081db43880 mi_switch() at mi_switch+0x179/frame 0xfffffe081db438c0 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081db43900 sleepq_timedwait() at sleepq_timedwait+0x43/frame 0xfffffe081db43930 _sleep() at _sleep+0x326/frame 0xfffffe081db439d0 pause_sbt() at pause_sbt+0xe4/frame 0xfffffe081db43a20 deadlkres() at deadlkres+0x66/frame 0xfffffe081db43a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081db43ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081db43ab0 --- trap 0, rip = 0, rsp = 0xfffffe081db43b70, rbp = 0 --- Tracing command kernel pid 0 tid 100123 td 0xfffff8001000d920 sched_switch() at sched_switch+0x482/frame 0xfffffe081db3e900 mi_switch() at mi_switch+0x179/frame 0xfffffe081db3e940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081db3e980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081db3e9b0 msleep_spin_sbt() at msleep_spin_sbt+0x200/frame 0xfffffe081db3ea40 taskqueue_thread_loop() at taskqueue_thread_loop+0x8d/frame 0xfffffe081db3ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe081db3eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081db3eab0 --- trap 0, rip = 0, rsp = 0xfffffe081db3eb70, rbp = 0 --- Tracing command kernel pid 0 tid 100108 td 0xfffff8000c89a920 sched_switch() at sched_switch+0x482/frame 0xfffffe081b8cb8f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081b8cb930 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081b8cb970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081b8cb9a0 _sleep() at _sleep+0x351/frame 0xfffffe081b8cba40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe081b8cba70 fork_exit() at fork_exit+0x84/frame 0xfffffe081b8cbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081b8cbab0 --- trap 0, rip = 0, rsp = 0xfffffe081b8cbb70, rbp = 0 --- Tracing command kernel pid 0 tid 100106 td 0xfffff8000c89b490 sched_switch() at sched_switch+0x482/frame 0xfffffe081b8c18f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081b8c1930 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081b8c1970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081b8c19a0 _sleep() at _sleep+0x351/frame 0xfffffe081b8c1a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe081b8c1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081b8c1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081b8c1ab0 --- trap 0, rip = 0, rsp = 0xfffffe081b8c1b70, rbp = 0 --- Tracing command kernel pid 0 tid 100104 td 0xfffff8000c894920 sched_switch() at sched_switch+0x482/frame 0xfffffe081b8b78f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081b8b7930 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081b8b7970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081b8b79a0 _sleep() at _sleep+0x351/frame 0xfffffe081b8b7a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe081b8b7a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081b8b7ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081b8b7ab0 --- trap 0, rip = 0, rsp = 0xfffffe081b8b7b70, rbp = 0 --- Tracing command kernel pid 0 tid 100102 td 0xfffff8000c895490 sched_switch() at sched_switch+0x482/frame 0xfffffe081b8ad8f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081b8ad930 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081b8ad970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081b8ad9a0 _sleep() at _sleep+0x351/frame 0xfffffe081b8ada40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe081b8ada70 fork_exit() at fork_exit+0x84/frame 0xfffffe081b8adab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081b8adab0 --- trap 0, rip = 0, rsp = 0xfffffe081b8adb70, rbp = 0 --- Tracing command kernel pid 0 tid 100100 td 0xfffff8000c87a920 sched_switch() at sched_switch+0x482/frame 0xfffffe081b8a38f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081b8a3930 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081b8a3970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081b8a39a0 _sleep() at _sleep+0x351/frame 0xfffffe081b8a3a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe081b8a3a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081b8a3ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081b8a3ab0 --- trap 0, rip = 0, rsp = 0xfffffe081b8a3b70, rbp = 0 --- Tracing command kernel pid 0 tid 100098 td 0xfffff8000c893490 sched_switch() at sched_switch+0x482/frame 0xfffffe081b8998f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081b899930 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081b899970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081b8999a0 _sleep() at _sleep+0x351/frame 0xfffffe081b899a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe081b899a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081b899ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081b899ab0 --- trap 0, rip = 0, rsp = 0xfffffe081b899b70, rbp = 0 --- Tracing command kernel pid 0 tid 100096 td 0xfffff8000c894000 sched_switch() at sched_switch+0x482/frame 0xfffffe081b88f8f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081b88f930 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081b88f970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081b88f9a0 _sleep() at _sleep+0x351/frame 0xfffffe081b88fa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe081b88fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe081b88fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081b88fab0 --- trap 0, rip = 0, rsp = 0xfffffe081b88fb70, rbp = 0 --- Tracing command kernel pid 0 tid 100094 td 0xfffff8000c875490 sched_switch() at sched_switch+0x482/frame 0xfffffe081b8858f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081b885930 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081b885970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081b8859a0 _sleep() at _sleep+0x351/frame 0xfffffe081b885a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe081b885a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081b885ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081b885ab0 --- trap 0, rip = 0, rsp = 0xfffffe081b885b70, rbp = 0 --- Tracing command kernel pid 0 tid 100091 td 0xfffff8000c87a490 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe07fc3e1ce0 ipi_nmi_handler() at ipi_nmi_handler+0x3f/frame 0xfffffe07fc3e1d00 trap() at trap+0x42/frame 0xfffffe07fc3e1f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe07fc3e1f20 --- trap 0x13, rip = 0xffffffff808a73f0, rsp = 0xfffffe07fc3e1fe0, rbp = 0xfffffe081b836970 --- __mtx_lock_sleep() at __mtx_lock_sleep+0x220/frame 0xfffffe081b836970 __mtx_lock_flags() at __mtx_lock_flags+0xf5/frame 0xfffffe081b8369c0 igb_deferred_mq_start() at igb_deferred_mq_start+0x2d/frame 0xfffffe081b8369f0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe081b836a40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe081b836a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081b836ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081b836ab0 --- trap 0, rip = 0, rsp = 0xfffffe081b836b70, rbp = 0 --- Tracing command kernel pid 0 tid 100089 td 0xfffff8000c872920 sched_switch() at sched_switch+0x482/frame 0xfffffe081b82c860 mi_switch() at mi_switch+0x179/frame 0xfffffe081b82c8a0 turnstile_wait() at turnstile_wait+0x3af/frame 0xfffffe081b82c8f0 __mtx_lock_sleep() at __mtx_lock_sleep+0x2a9/frame 0xfffffe081b82c970 __mtx_lock_flags() at __mtx_lock_flags+0xf5/frame 0xfffffe081b82c9c0 igb_deferred_mq_start() at igb_deferred_mq_start+0x2d/frame 0xfffffe081b82c9f0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe081b82ca40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe081b82ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe081b82cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081b82cab0 --- trap 0, rip = 0, rsp = 0xfffffe081b82cb70, rbp = 0 --- Tracing command kernel pid 0 tid 100087 td 0xfffff8000c874490 sched_switch() at sched_switch+0x482/frame 0xfffffe081b822860 mi_switch() at mi_switch+0x179/frame 0xfffffe081b8228a0 turnstile_wait() at turnstile_wait+0x3af/frame 0xfffffe081b8228f0 __mtx_lock_sleep() at __mtx_lock_sleep+0x2a9/frame 0xfffffe081b822970 __mtx_lock_flags() at __mtx_lock_flags+0xf5/frame 0xfffffe081b8229c0 igb_deferred_mq_start() at igb_deferred_mq_start+0x2d/frame 0xfffffe081b8229f0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe081b822a40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe081b822a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081b822ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081b822ab0 --- trap 0, rip = 0, rsp = 0xfffffe081b822b70, rbp = 0 --- Tracing command kernel pid 0 tid 100085 td 0xfffff8000c870920 sched_switch() at sched_switch+0x482/frame 0xfffffe081b8188f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081b818930 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081b818970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081b8189a0 _sleep() at _sleep+0x351/frame 0xfffffe081b818a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe081b818a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081b818ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081b818ab0 --- trap 0, rip = 0, rsp = 0xfffffe081b818b70, rbp = 0 --- Tracing command kernel pid 0 tid 100083 td 0xfffff8000c871490 sched_switch() at sched_switch+0x482/frame 0xfffffe081b80e8f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081b80e930 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081b80e970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081b80e9a0 _sleep() at _sleep+0x351/frame 0xfffffe081b80ea40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe081b80ea70 fork_exit() at fork_exit+0x84/frame 0xfffffe081b80eab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081b80eab0 --- trap 0, rip = 0, rsp = 0xfffffe081b80eb70, rbp = 0 --- Tracing command kernel pid 0 tid 100081 td 0xfffff8000c872000 sched_switch() at sched_switch+0x482/frame 0xfffffe081b8048f0 mi_switch() at mi_switch+0x179/frame 0xfffffe081b804930 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe081b804970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe081b8049a0 _sleep() at _sleep+0x351/frame 0xfffffe081b804a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe081b804a70 fork_exit() at fork_exit+0x84/frame 0xfffffe081b804ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe081b804ab0 --- trap 0, rip = 0, rsp = 0xfffffe081b804b70, rbp = 0 --- Tracing command kernel pid 0 tid 100079 td 0xfffff8000c86f490 sched_switch() at sched_switch+0x482/frame 0xfffffe00003f98f0 mi_switch() at mi_switch+0x179/frame 0xfffffe00003f9930 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe00003f9970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe00003f99a0 _sleep() at _sleep+0x351/frame 0xfffffe00003f9a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe00003f9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00003f9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00003f9ab0 --- trap 0, rip = 0, rsp = 0xfffffe00003f9b70, rbp = 0 --- Tracing command kernel pid 0 tid 100077 td 0xfffff8000c870000 sched_switch() at sched_switch+0x482/frame 0xfffffe00003ef850 mi_switch() at mi_switch+0x179/frame 0xfffffe00003ef890 turnstile_wait() at turnstile_wait+0x3af/frame 0xfffffe00003ef8e0 __mtx_lock_sleep() at __mtx_lock_sleep+0x2a9/frame 0xfffffe00003ef960 __mtx_lock_flags() at __mtx_lock_flags+0xf5/frame 0xfffffe00003ef9b0 igb_handle_que() at igb_handle_que+0x57/frame 0xfffffe00003ef9f0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe00003efa40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe00003efa70 fork_exit() at fork_exit+0x84/frame 0xfffffe00003efab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00003efab0 --- trap 0, rip = 0, rsp = 0xfffffe00003efb70, rbp = 0 --- Tracing command kernel pid 0 tid 100067 td 0xfffff8000b421920 sched_switch() at sched_switch+0x482/frame 0xfffffe078aa868f0 mi_switch() at mi_switch+0x179/frame 0xfffffe078aa86930 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe078aa86970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe078aa869a0 _sleep() at _sleep+0x351/frame 0xfffffe078aa86a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe078aa86a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078aa86ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078aa86ab0 --- trap 0, rip = 0, rsp = 0xfffffe078aa86b70, rbp = 0 --- Tracing command kernel pid 0 tid 100064 td 0xfffff8000b414490 sched_switch() at sched_switch+0x482/frame 0xfffffe078aa77900 mi_switch() at mi_switch+0x179/frame 0xfffffe078aa77940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe078aa77980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe078aa779b0 msleep_spin_sbt() at msleep_spin_sbt+0x200/frame 0xfffffe078aa77a40 taskqueue_thread_loop() at taskqueue_thread_loop+0x8d/frame 0xfffffe078aa77a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078aa77ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078aa77ab0 --- trap 0, rip = 0, rsp = 0xfffffe078aa77b70, rbp = 0 --- Tracing command kernel pid 0 tid 100063 td 0xfffff8000b414920 sched_switch() at sched_switch+0x482/frame 0xfffffe078aa72900 mi_switch() at mi_switch+0x179/frame 0xfffffe078aa72940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe078aa72980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe078aa729b0 msleep_spin_sbt() at msleep_spin_sbt+0x200/frame 0xfffffe078aa72a40 taskqueue_thread_loop() at taskqueue_thread_loop+0x8d/frame 0xfffffe078aa72a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078aa72ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078aa72ab0 --- trap 0, rip = 0, rsp = 0xfffffe078aa72b70, rbp = 0 --- Tracing command kernel pid 0 tid 100062 td 0xfffff8000b41c000 sched_switch() at sched_switch+0x482/frame 0xfffffe078aa6d900 mi_switch() at mi_switch+0x179/frame 0xfffffe078aa6d940 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe078aa6d980 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe078aa6d9b0 msleep_spin_sbt() at msleep_spin_sbt+0x200/frame 0xfffffe078aa6da40 taskqueue_thread_loop() at taskqueue_thread_loop+0x8d/frame 0xfffffe078aa6da70 fork_exit() at fork_exit+0x84/frame 0xfffffe078aa6dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078aa6dab0 --- trap 0, rip = 0, rsp = 0xfffffe078aa6db70, rbp = 0 --- Tracing command kernel pid 0 tid 100061 td 0xfffff8000b41c490 sched_switch() at sched_switch+0x482/frame 0xfffffe078aa688f0 mi_switch() at mi_switch+0x179/frame 0xfffffe078aa68930 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe078aa68970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe078aa689a0 _sleep() at _sleep+0x351/frame 0xfffffe078aa68a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe078aa68a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078aa68ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078aa68ab0 --- trap 0, rip = 0, rsp = 0xfffffe078aa68b70, rbp = 0 --- Tracing command kernel pid 0 tid 100058 td 0xfffff8000b27f000 sched_switch() at sched_switch+0x482/frame 0xfffffe078aa598f0 mi_switch() at mi_switch+0x179/frame 0xfffffe078aa59930 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe078aa59970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe078aa599a0 _sleep() at _sleep+0x351/frame 0xfffffe078aa59a40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe078aa59a70 fork_exit() at fork_exit+0x84/frame 0xfffffe078aa59ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078aa59ab0 --- trap 0, rip = 0, rsp = 0xfffffe078aa59b70, rbp = 0 --- Tracing command kernel pid 0 tid 100056 td 0xfffff8000b27f920 sched_switch() at sched_switch+0x482/frame 0xfffffe078aa4f8f0 mi_switch() at mi_switch+0x179/frame 0xfffffe078aa4f930 sleepq_switch() at sleepq_switch+0x185/frame 0xfffffe078aa4f970 sleepq_wait() at sleepq_wait+0x43/frame 0xfffffe078aa4f9a0 _sleep() at _sleep+0x351/frame 0xfffffe078aa4fa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8/frame 0xfffffe078aa4fa70 fork_exit() at fork_exit+0x84/frame 0xfffffe078aa4fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe078aa4fab0 --- trap 0, rip = 0, rsp = 0xfffffe078aa4fb70, rbp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xffffffff816fc2e0 sched_switch() at sched_switch+0x482/frame 0xffffffff81a179b0 mi_switch() at mi_switch+0x179/frame 0xffffffff81a179f0 sleepq_switch() at sleepq_switch+0x185/frame 0xffffffff81a17a30 sleepq_wait() at sleepq_wait+0x43/frame 0xffffffff81a17a60 _sleep() at _sleep+0x351/frame 0xffffffff81a17b00 vm_wait() at vm_wait+0xb7/frame 0xffffffff81a17b20 swapper() at swapper+0x25/frame 0xffffffff81a17b70 btext() at btext+0x2c db:0:allt> call doadump Dumping 1397 out of 31643 MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% Dump complete = 0 db:0:doadump> reset cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 5 (kgdb) bt #0 doadump (textdump=0xd) at pcpu.h:219 #1 0xffffffff80341725 in db_fncall (dummy1=, dummy2=, dummy3=, dummy4=) at ../../../ddb/db_command.c:578 #2 0xffffffff8034140d in db_command (cmd_table=) at ../../../ddb/db_command.c:449 #3 0xffffffff8034566f in db_script_exec (scriptname=0xffffffff816f09a0 "doadump", warnifnotfound=) at ../../../ddb/db_script.c:302 #4 0xffffffff8034140d in db_command (cmd_table=) at ../../../ddb/db_command.c:449 #5 0xffffffff80341184 in db_command_loop () at ../../../ddb/db_command.c:502 #6 0xffffffff80343b30 in db_trap (type=, code=0x0) at ../../../ddb/db_main.c:231 #7 0xffffffff808f3853 in kdb_trap (type=0x3, code=0x0, tf=) at ../../../kern/subr_kdb.c:654 #8 0xffffffff80cc077b in trap (frame=0xfffffe081b81d5f0) at ../../../amd64/amd64/trap.c:579 #9 0xffffffff80ca10b2 in calltrap () at ../../../amd64/amd64/exception.S:232 #10 0xffffffff808f302e in kdb_enter (why=0xffffffff80e13ea5 "watchdog", msg=) at cpufunc.h:63 #11 0xffffffff8086ce67 in hardclock_cnt (cnt=, usermode=) at ../../../kern/kern_clock.c:563 #12 0xffffffff80d96eae in handleevents (now=0x81779701b4d, fake=0x0) at ../../../kern/kern_clocksource.c:195 #13 0xffffffff80d97668 in timercb (et=, arg=) at ../../../kern/kern_clocksource.c:365 #14 0xffffffff80dc240c in lapic_handle_timer (frame=0xfffffe081b81d820) at ../../../x86/x86/local_apic.c:816 #15 0xffffffff80ca1c0c in Xtimerint () at apic_vector.S:109 #16 0xffffffff808a73f2 in __mtx_lock_sleep (c=0xfffff8000c841520, tid=0xfffff8000c874920, opts=, file=, line=) at cpufunc.h:309 #17 0xffffffff808a7145 in __mtx_lock_flags (c=, opts=, file=0xffffffff80e997dd "../../../dev/e1000/if_igb.c", line=0x648) at ../../../kern/kern_mutex.c:224 #18 0xffffffff804de47d in igb_msix_que (arg=0xfffff8000b37aa08) at ../../../dev/e1000/if_igb.c:1608 #19 0xffffffff8088d023 in intr_event_execute_handlers (p=, ie=0xfffff8000b255400) at ../../../kern/kern_intr.c:1263 #20 0xffffffff8088d9f6 in ithread_loop (arg=0xfffff8000b25a940) at ../../../kern/kern_intr.c:1276 #21 0xffffffff8088aab4 in fork_exit (callout=0xffffffff8088d950 , arg=0xfffff8000b25a940, frame=0xfffffe081b81dac0) at ../../../kern/kern_fork.c:995 #22 0xffffffff80ca15ee in fork_trampoline () at ../../../amd64/amd64/exception.S:606 #23 0x0000000000000000 in ?? () Current language: auto; currently minimal (kgdb) f 18 #18 0xffffffff804de47d in igb_msix_que (arg=0xfffff8000b37aa08) at ../../../dev/e1000/if_igb.c:1608 1608 IGB_TX_LOCK(txr); (kgdb) l 1603 return; 1604 1605 E1000_WRITE_REG(&adapter->hw, E1000_EIMC, que->eims); 1606 ++que->irqs; 1607 1608 IGB_TX_LOCK(txr); 1609 igb_txeof(txr); 1610 #ifndef IGB_LEGACY_TX 1611 /* Process the stack queue only if not depleted */ 1612 if (((txr->queue_status & IGB_QUEUE_DEPLETED) == 0) && (kgdb) info loc No locals. (kgdb) $ svn diff -x -p /usr/src/sys -- Test scenario: udp.sh