GENERIC HEAD from 2013-07-01 05:01:27 UTC, r252443M, vmcore.670 GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb 4718592K of memory above 4GB ignored Copyright (c) 1992-2013 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #0 r252443M: Mon Jul 1 19:00:36 CEST 2013 pho@x4.osted.lan:/usr/src/sys/i386/compile/PHO i386 FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. CPU: AMD Phenom(tm) 9150e Quad-Core Processor (1800.02-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x100f23 Family = 0x10 Model = 0x2 Stepping = 3 Features=0x178bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x7ff TSC: P-state invariant real memory = 8589934592 (8192 MB) avail memory = 3540086784 (3376 MB) : Trying to mount root from ufs:/dev/ad4s1a [rw]... Setting hostuuid: 00000000-0000-0000-0000-00218515337d. Setting hostid: 0x6b64ac17. Starting ddb. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: /dev/ad4s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1a: clean, 615107 free (1275 frags, 76729 blocks, 0.1% fragmentation) /dev/ad4s1f: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1f: clean, 793202 free (8378 frags, 98103 blocks, 0.8% fragmentation) /dev/label/tmp: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/label/tmp: clean, 44053725 free (7317 frags, 5505801 blocks, 0.0% fragmentation) /dev/ad4s1d: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1d: clean, 4152609 free (298185 frags, 481803 blocks, 2.9% fragmentation) /dev/ad4s1e: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ad4s1e: clean, 10137712 free (185928 frags, 1243973 blocks, 0.7% fragmentation) Mounting local file systems:. Setting hostname: x4.osted.lan. re0: link state changed to DOWN re0: link state changed to UP Starting Network: lo0 re0. lo0: flags=8049 metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 nd6 options=21 re0: flags=8843 metric 0 mtu 1500 options=8209b ether 00:21:85:15:33:7d inet 192.168.1.101 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::221:85ff:fe15:337d%re0 prefixlen 64 scopeid 0x1 nd6 options=29 media: Ethernet autoselect (100baseTX ) status: active Starting devd. add net default: gateway 192.168.1.1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/kde4/lib /usr/local/lib/compat /usr/local/lib/compat/pkg /usr/local/lib/gcc46 /usr/local/lib/mysql /usr/local/lib/nss /usr/local/lib/pth /usr/local/lib/qt4 /usr/local/lib/virtualbox a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout Creating and/or trimming log files. Starting syslogd. savecore: couldn't find media and/or sector size of /var/crash: Inappropriate ioctl for device Jul 1 19:16:19 x4 savecore: couldn't find media and/or sector size of /var/crash: Inappropriate ioctl for device No core dumps found. Additional ABI support: linux. Starting rpcbind. NFS access cache time=60 rpc.umntall: 127.0.0.1: MOUNTPROG: RPC: Program not registered lock order reversal: 1st 0xe350f120 bufwait (bufwait) @ kern/vfs_bio.c:3058 2nd 0xc7db3a00 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:284 KDB: stack backtrace: db_trace_self_wrapper(c111181c,e350f0c8,f7d117c4,c0b43c6a,e350f120,...) at db_trace_self_wrapper+0x2d/frame 0xf7d11798 kdb_backtrace(c11153f1,c7db3a00,c11476cd,c75866e0,c1147312,...) at kdb_backtrace+0x30/frame 0xf7d11800 witness_checkorder(c7db3a00,9,c1147309,11c,0,...) at witness_checkorder+0xccc/frame 0xf7d11850 _sx_xlock(c7db3a00,0,c1147309,11c,c7f0e000,...) at _sx_xlock+0x77/frame 0xf7d11880 ufsdirhash_add(c7eb0000,f7d11978,7a84,f7d11900,f7d11904,...) at ufsdirhash_add+0x4a/frame 0xf7d118b0 ufs_direnter(c8167708,c844b000,f7d11978,f7d11b64,e3512ab8,...) at ufs_direnter+0x63b/frame 0xf7d11930 ufs_mkdir(f7d11c00,c116cf23,c1140fa6,c8167708,f7d11bfc,...) at ufs_mkdir+0x884/frame 0xf7d11ad0 VOP_MKDIR_APV(c123e980,f7d11c00,f7d11b64,f7d11b90,28803020,...) at VOP_MKDIR_APV+0x124/frame 0xf7d11b00 kern_mkdirat(c7e0bc40,ffffff9c,28803020,0,1c0) at kern_mkdirat+0x1fc/frame 0xf7d11c24 sys_mkdir(c7e0bc40,f7d11cc8,14,c110f9e5,7b3,...) at sys_mkdir+0x31/frame 0xf7d11c40 syscall(f7d11d08) at syscall+0x2ed/frame 0xf7d11cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7d11cfc --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28175837, esp = 0xbfbfd92c, ebp = 0xbfbfddd0 --- Clearing /tmp (X related). Starting nfsuserd. Starting mountd. Starting nfsd. Recovering vi editor sessions:. Updating motd:. Starting ntpd. Removing stale Samba tdb files: ....... done Starting nmbd. Starting smbd. Starting winbindd. Starting smartd. Starting default moused. Configuring syscons: keymap blanktime. Starting sshd. Starting cron. Local package initialization: backuplock order reversal: 1st 0xc8167cdc ufs (ufs) @ kern/vfs_mount.c:1236 2nd 0xc81675d4 syncer (syncer) @ kern/vfs_subr.c:2210 KDB: stack backtrace: db_trace_self_wrapper(c111181c,1c,f7dd58e0,c0ac6de3,f7dd5900,...) at db_trace_self_wrapper+0x2d/frame 0xf7dd58c0 kdb_backtrace(c11153f1,c81675d4,c111eece,c7586748,c111e804,...) at kdb_backtrace+0x30/frame 0xf7dd5928 witness_checkorder(c81675d4,9,c111e7fb,8a2,c8167640,...) at witness_checkorder+0xccc/frame 0xf7dd5978 __lockmgr_args(c81675d4,80100,c8167640,0,0,0,c111e7fb,8a2) at __lockmgr_args+0x817/frame 0xf7dd5a4c vop_stdlock(f7dd5ac0,246,c13d675c,c13d6774,c13d6760,...) at vop_stdlock+0x53/frame 0xf7dd5a7c VOP_LOCK1_APV(c121ec48,f7dd5ac0,c0aa263a,c8167640,c1253290,...) at VOP_LOCK1_APV+0x104/frame 0xf7dd5aa8 _vn_lock(c81675a0,80100,c111e7fb,8a2,c8167650,...) at _vn_lock+0xca/frame 0xf7dd5ae8 vputx(c7d95bb0,0,c111de0b,50f,c1214040,...) at vputx+0x330/frame 0xf7dd5b30 dounmount(c7d95bb0,8080000,c82b5930,493,0,...) at dounmount+0x371/frame 0xf7dd5b90 sys_unmount(c82b5930,f7dd5cc8,c0ee61d0,c7200c90,0,...) at sys_unmount+0x3c1/frame 0xf7dd5c40 syscall(f7dd5d08) at syscall+0x2ed/frame 0xf7dd5cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7dd5cfc --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280c1ecb, esp = 0xbfbfd6a4, ebp = 0xbfbfd770 --- lock order reversal: 1st 0xc816773c ufs (ufs) @ kern/vfs_subr.c:2099 2nd 0xe350ef84 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:262 3rd 0xc82c319c ufs (ufs) @ kern/vfs_subr.c:2099 KDB: stack backtrace: db_trace_self_wrapper(c111181c,c1167633,ef,c604a853,15,...) at db_trace_self_wrapper+0x2d/frame 0xf7dd56f0 kdb_backtrace(c111540a,c82c319c,c10fc926,c7586678,c111e804,...) at kdb_backtrace+0x30/frame 0xf7dd5758 witness_checkorder(c82c319c,9,c111e7fb,833,c82c3208,...) at witness_checkorder+0xccc/frame 0xf7dd57a8 __lockmgr_args(c82c319c,80100,c82c3208,0,0,...) at __lockmgr_args+0x817/frame 0xf7dd587c ffs_lock(f7dd5900,c757f1e0,c7584c10,c757f1e0,c7584c10,...) at ffs_lock+0xad/frame 0xf7dd58bc VOP_LOCK1_APV(c123e980,f7dd5900,f7dd593c,c0b430f5,c1253290,...) at VOP_LOCK1_APV+0x104/frame 0xf7dd58e8 _vn_lock(c82c3168,80100,c111e7fb,833,c111da71,...) at _vn_lock+0xca/frame 0xf7dd5928 vget(c82c3168,80100,c82b5930,57,0,...) at vget+0x77/frame 0xf7dd5960 vfs_hash_get(c7d95bb0,4b7800,80000,c82b5930,f7dd5a40,...) at vfs_hash_get+0xff/frame 0xf7dd598c ffs_vgetf(c7d95bb0,4b7800,80000,f7dd5a40,1,...) at ffs_vgetf+0x47/frame 0xf7dd59e8 softdep_sync_buf(c8167708,e350ef2c,1,0,0,...) at softdep_sync_buf+0x2d2/frame 0xf7dd5a50 ffs_syncvnode(c8167708,1,0,61a,0,...) at ffs_syncvnode+0x287/frame 0xf7dd5aa8 ffs_sync(c7d95bb0,1,f7dd5b68,50f,c1214040,...) at ffs_sync+0x23e/frame 0xf7dd5b30 dounmount(c7d95bb0,8080000,c82b5930,493,0,...) at dounmount+0x43c/frame 0xf7dd5b90 sys_unmount(c82b5930,f7dd5cc8,c0ee61d0,c7200c90,0,...) at sys_unmount+0x3c1/frame 0xf7dd5c40 syscall(f7dd5d08) at syscall+0x2ed/frame 0xf7dd5cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7dd5cfc --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280c1ecb, esp = 0xbfbfd6a4, ebp = 0xbfbfd770 --- lock order reversal: 1st 0xc8167cdc ufs (ufs) @ kern/vfs_mount.c:1236 2nd 0xc7e7773c devfs (devfs) @ ufs/ffs/ffs_softdep.c:1870 KDB: stack backtrace: db_trace_self_wrapper(c111181c,c1214040,c82b59e4,246,c11496e2,...) at db_trace_self_wrapper+0x2d/frame 0xf7dd5840 kdb_backtrace(c11153f1,c7e7773c,c110864d,c75864d8,c11439cb,...) at kdb_backtrace+0x30/frame 0xf7dd58a8 witness_checkorder(c7e7773c,9,c11439c2,74e,0,...) at witness_checkorder+0xccc/frame 0xf7dd58f8 __lockmgr_args(c7e7773c,80400,c7e777a8,0,0,0,c11439c2,74e) at __lockmgr_args+0x817/frame 0xf7dd59cc vop_stdlock(f7dd5a40,c14073bc,c14073ac,f7dd5a40,c0ab579f,...) at vop_stdlock+0x53/frame 0xf7dd59fc VOP_LOCK1_APV(c1204028,f7dd5a40,4,4,c1253290,...) at VOP_LOCK1_APV+0x104/frame 0xf7dd5a28 _vn_lock(c7e77708,80400,c11439c2,74e,f7dd5aa8,...) at _vn_lock+0xca/frame 0xf7dd5a68 softdep_flushworklist(c7d95bb0,f7dd5afc,c82b5930,62e,0,...) at softdep_flushworklist+0x6e/frame 0xf7dd5aa8 ffs_sync(c7d95bb0,1,f7dd5b68,50f,c1214040,...) at ffs_sync+0x2e5/frame 0xf7dd5b30 dounmount(c7d95bb0,8080000,c82b5930,493,0,...) at dounmount+0x43c/frame 0xf7dd5b90 sys_unmount(c82b5930,f7dd5cc8,c0ee61d0,c7200c90,0,...) at sys_unmount+0x3c1/frame 0xf7dd5c40 syscall(f7dd5d08) at syscall+0x2ed/frame 0xf7dd5cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7dd5cfc --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280c1ecb, esp = 0xbfbfd6a4, ebp = 0xbfbfd770 --- ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 43566 files, 74260256 used, 44053725 free (7317 frags, 5505801 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** fsck -y /tmp watchdogd. Starting inetd. Mon Jul 1 19:18:33 CEST 2013 FreeBSD/i386 (x4.osted.lan) (console) login: Jul 1 20:12:27 x4 su: pho to root on /dev/pts/1 panic: vm_page_insert_after: mpred doesn't precede pindex cpuid = 3 KDB: enter: panic [ thread pid 2310 tid 100122 ] Stopped at kdb_enter+0x3d: movl $0,kdb_why db> run pho db:0:pho> bt Tracing pid 2310 tid 100122 td 0xc82ad930 kdb_enter(c110d3eb,c110d3eb,c114c16f,f7df78fc,c114c16f,...) at kdb_enter+0x3d/frame 0xf7df7890 vpanic(c128a2d2,100,c114c16f,f7df78fc,f7df78fc,...) at vpanic+0x143/frame 0xf7df78cc kassert_panic(c114c16f,4,c114b49b,347,160,...) at kassert_panic+0xea/frame 0xf7df78f0 vm_page_insert_after(0,0,c4f8e5c0,330) at vm_page_insert_after+0x117/frame 0xf7df791c vm_page_insert(c913a000,c910a534,0,0,1,...) at vm_page_insert+0x5d/frame 0xf7df7940 old_dev_pager_fault(c910a534,0,0,1,f7df7a24,...) at old_dev_pager_fault+0x1ff/frame 0xf7df7984 dev_pager_getpages(c910a534,f7df7a24,1,0,f7df7afc,...) at dev_pager_getpages+0x79/frame 0xf7df79b8 vm_fault_hold(c7e83d98,28224000,1,0,0,...) at vm_fault_hold+0xa1f/frame 0xf7df7b18 vm_fault(c7e83d98,28224000,1,0,c7e83d98,...) at vm_fault+0x82/frame 0xf7df7b40 trap_pfault(28224000,c0d4f55a,c7e83d98,f7df7c18,9000,...) at trap_pfault+0x1ba/frame 0xf7df7bb8 trap(f7df7d08) at trap+0x614/frame 0xf7df7cfc calltrap() at calltrap+0x6/frame 0xf7df7cfc --- trap 0xc, eip = 0x281c602e, esp = 0xbfbfd3dc, ebp = 0xbfbfd400 --- db:0:bt> show allpcpu Current CPU: 3 cpuid = 0 dynamic pcpu = 0x5cc780 curthread = 0xc76e4000: pid 11 "idle: cpu0" curpcb = 0xc720ad60 fpcurthread = none idlethread = 0xc76e4000: tid 100003 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: cpuid = 1 dynamic pcpu = 0x5fa1780 curthread = 0xc76e3c40: pid 11 "idle: cpu1" curpcb = 0xc720dd60 fpcurthread = none idlethread = 0xc76e3c40: tid 100004 "idle: cpu1" APIC ID = 1 currentldt = 0x50 spin locks held: cpuid = 2 dynamic pcpu = 0x5fa4780 curthread = 0xc76e3930: pid 11 "idle: cpu2" curpcb = 0xc7210d60 fpcurthread = none idlethread = 0xc76e3930: tid 100005 "idle: cpu2" APIC ID = 2 currentldt = 0x50 spin locks held: cpuid = 3 dynamic pcpu = 0x5fa7780 curthread = 0xc82ad930: pid 2310 "mdconfig" curpcb = 0xf7df7d60 fpcurthread = 0xc82ad930: pid 2310 "mdconfig" idlethread = 0xc76e3620: tid 100006 "idle: cpu3" APIC ID = 3 currentldt = 0x50 spin locks held: db:0:allpcpu> show alllocks Process 2310 (mdconfig) thread 0xc82ad930 (100122) exclusive rw vm object (vm object) r = 0 (0xc910a534) locked @ vm/device_pager.c:352 db:0:alllocks> show lockedvnods Locked vnodes db:0:lockedvnods> show mount 0xc7d962ec /dev/ad4s1a on / (ufs) 0xc7d965d8 devfs on /dev (devfs) 0xc7d96000 /dev/ad4s1f on /home (ufs) 0xc7d958c4 /dev/ad4s1d on /usr (ufs) 0xc7d955d8 /dev/ad4s1e on /var (ufs) 0xc8177000 procfs on /proc (procfs) 0xc7d952ec /dev/label/tmp on /tmp (ufs) More info: show mount db:0:mount> ps pid ppid pgrp uid state wmesg wchan cmd 2311 2242 1101 0 S+ piperd 0xc910e000 grep 2310 2242 1101 0 RL+ CPU 3 mdconfig 2278 1053 1050 1001 S nanslp 0xc128b668 sleep 2242 1689 1101 0 S+ wait 0xc9018000 sh 1689 1101 1101 0 S+ wait 0xc88722e8 sh 1101 1097 1101 0 S+ wait 0xc88715d0 sh 1097 1094 1097 0 S+ wait 0xc8871000 bash 1094 1093 1094 0 S+ pause 0xc9018c04 csh 1093 1073 1093 1001 S+ wait 0xc90188b8 su 1073 1072 1073 1001 Ss+ wait 0xc8871ba0 bash 1072 1070 1070 1001 S select 0xc8151e24 sshd 1070 727 1070 0 Ss select 0xc7d14d64 sshd 1054 1050 1050 1001 S piperd 0xc84dd198 awk 1053 1050 1050 1001 S wait 0xc82ab8b8 sh 1052 1049 1052 1001 Ss+ select 0xc8151da4 top 1051 1048 1051 1001 Ss kqread 0xc7d80480 tail 1050 1047 1050 1001 Ss wait 0xc84518b8 sh 1049 1043 1043 1001 S select 0xc8151e64 sshd 1048 1042 1042 1001 S select 0xc7d14de4 sshd 1047 1041 1041 1001 S select 0xc78e6724 sshd 1043 727 1043 0 Ss select 0xc78e6ca4 sshd 1042 727 1042 0 Ss select 0xc78e67a4 sshd 1041 727 1041 0 Ss select 0xc7d14da4 sshd 838 1 838 0 Ss+ ttyin 0xc77bb270 getty 837 1 837 0 Ss+ ttyin 0xc77bb470 getty 836 1 836 0 Ss+ ttyin 0xc77bb670 getty 835 1 835 0 Ss+ ttyin 0xc77bb870 getty 834 1 834 0 Ss+ ttyin 0xc77bba70 getty 833 1 833 0 Ss+ ttyin 0xc77bbc70 getty 832 1 832 0 Ss+ ttyin 0xc77bbe70 getty 831 1 831 0 Ss+ ttyin 0xc77bc070 getty 830 1 830 0 Ss+ ttyin 0xc77bc270 getty 813 1 813 0 Ss select 0xc78e6824 inetd 789 1 789 0 Ss nanslp 0xc128b66b watchdogd 738 1 738 0 Ss nanslp 0xc128b668 cron 733 1 733 25 Ss pause 0xc7e53c04 sendmail 730 1 730 0 Ss select 0xc7d14e24 sendmail 727 1 727 0 Ss select 0xc7d15264 sshd 695 1 695 0 Ss select 0xc7d152a4 moused 687 1 686 0 S nanslp 0xc128b66a smartd 685 673 673 0 S select 0xc8151f24 winbindd 684 671 671 0 S select 0xc7d14e64 smbd 673 1 673 0 Ss select 0xc8151ea4 winbindd 671 1 671 0 Ss select 0xc8151ee4 smbd 669 1 669 0 Ss select 0xc8152124 nmbd 645 1 645 0 Ss select 0xc81520e4 ntpd 589 584 584 0 S (threaded) nfsd 100098 S rpcsvc 0xc78df9d0 nfsd: service 100097 S rpcsvc 0xc78dfa10 nfsd: service 100096 S rpcsvc 0xc78dfa50 nfsd: service 100086 S rpcsvc 0xc78dead0 nfsd: master 584 1 584 0 Ss select 0xc7d15f24 nfsd 578 1 578 0 Ss select 0xc7d14ee4 mountd 571 567 567 0 S select 0xc78e6664 nfsuserd 570 567 567 0 S select 0xc7d8e064 nfsuserd 569 567 567 0 S select 0xc7d8e164 nfsuserd 568 567 567 0 S select 0xc8152164 nfsuserd 567 1 567 0 Ss pause 0xc7e54064 nfsuserd 538 1 538 0 Ss select 0xc78e6be4 rpcbind 523 1 523 0 Ss select 0xc7d15064 syslogd 420 1 420 0 Ss select 0xc7d150a4 devd 17 0 0 0 DL sdflush 0xc140742c [softdepflush] 16 0 0 0 DL syncer 0xc14031e0 [syncer] 9 0 0 0 DL vlruwt 0xc7c94ba0 [vnlru] 8 0 0 0 DL psleep 0xc1402d04 [bufdaemon] 7 0 0 0 DL pgzero 0xc1408188 [pagezero] 6 0 0 0 DL psleep 0xc1407d58 [vmdaemon] 5 0 0 0 DL psleep 0xc145bc04 [pagedaemon] 4 0 0 0 DL ccb_scan 0xc12595a4 [xpt_thrd] 3 0 0 0 DL waiting_ 0xc145790c [sctp_iterator] 2 0 0 0 DL - 0xc78bba3c [fdc0] 15 0 0 0 DL (threaded) [usb] 100058 D - 0xc791bd34 [usbus5] 100057 D - 0xc791bd04 [usbus5] 100056 D - 0xc791bcd4 [usbus5] 100055 D - 0xc791bca4 [usbus5] 100053 D - 0xc7913b5c [usbus4] 100052 D - 0xc7913b2c [usbus4] 100051 D - 0xc7913afc [usbus4] 100050 D - 0xc7913acc [usbus4] 100049 D - 0xc790db5c [usbus3] 100048 D - 0xc790db2c [usbus3] 100047 D - 0xc790dafc [usbus3] 100046 D - 0xc790dacc [usbus3] 100045 D - 0xc7903b5c [usbus2] 100044 D - 0xc7903b2c [usbus2] 100043 D - 0xc7903afc [usbus2] 100042 D - 0xc7903acc [usbus2] 100040 D - 0xc78fdb5c [usbus1] 100039 D - 0xc78fdb2c [usbus1] 100038 D - 0xc78fdafc [usbus1] 100037 D - 0xc78fdacc [usbus1] 100035 D - 0xc78f6b5c [usbus0] 100034 D - 0xc78f6b2c [usbus0] 100033 D - 0xc78f6afc [usbus0] 100032 D - 0xc78f6acc [usbus0] 14 0 0 0 DL - 0xc12712d0 [yarrow] 13 0 0 0 DL (threaded) [geom] 100015 D - 0xc145396c [g_down] 100014 D - 0xc1453968 [g_up] 100013 D - 0xc1453964 [g_event] 12 0 0 0 WL (threaded) [intr] 100063 I [irq12: psm0] 100062 I [irq1: atkbd0] 100060 I [swi0: uart] 100059 I [irq14: ata0] 100054 I [irq19: ehci0] 100041 I [irq18: ohci2 ohci4] 100036 I [irq17: ohci1 ohci3] 100031 I [irq16: hdac1 ohci0] 100030 I [irq22: ahci0] 100029 I [irq257: re0] 100028 I [irq256: hdac0] 100027 I [swi5: fast taskq] 100025 I [swi6: Giant taskq] 100023 I [swi6: task queue] 100018 I [swi2: cambio] 100012 I [swi1: netisr 0] 100011 I [swi4: clock] 100010 I [swi4: clock] 100009 I [swi4: clock] 100008 I [swi4: clock] 100007 I [swi3: vm] 11 0 0 0 RL (threaded) [idle] 100006 CanRun [idle: cpu3] 100005 Run CPU 2 [idle: cpu2] 100004 Run CPU 1 [idle: cpu1] 100003 Run CPU 0 [idle: cpu0] 1 0 1 0 SLs wait 0xc76dc2e8 [init] 10 0 0 0 DL audit_wo 0xc145a6b8 [audit] 0 0 0 0 DLs (threaded) [kernel] 100065 D - 0xc128a563 [deadlkres] 100064 D - 0xc76b3800 [mca taskq] 100026 D - 0xc76b3b00 [thread taskq] 100024 D - 0xc76b3c80 [ffs_trim taskq] 100022 D - 0xc76b3e00 [kqueue taskq] 100021 D - 0xc76b3e80 [acpi_task_2] 100020 D - 0xc76b3e80 [acpi_task_1] 100019 D - 0xc76b3e80 [acpi_task_0] 100016 D - 0xc76b4000 [firmware taskq] 100000 D sched 0xc14539e4 [swapper] db:0:ps> allt Tracing command grep pid 2311 tid 100143 td 0xc8874930 sched_switch(c8874930,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7e38a1c mi_switch(104,0,c1113475,1f5,c1113475,...) at mi_switch+0x1c9/frame 0xf7e38a50 sleepq_switch(c8874930,0,c1113475,1ab,f7e38ac8,...) at sleepq_switch+0x1c5/frame 0xf7e38a84 sleepq_catch_signals(0,c1113475,162,1500,c910e000,...) at sleepq_catch_signals+0x36c/frame 0xf7e38acc sleepq_wait_sig(c910e000,5c,c1116a8e,100,0,...) at sleepq_wait_sig+0x14/frame 0xf7e38af0 _sleep(c910e000,c910e180,15c,c1116a8e,0,...) at _sleep+0x364/frame 0xf7e38b4c pipe_read(c9124268,f7e38c10,c7dfa780,0,c8874930,...) at pipe_read+0x2dc/frame 0xf7e38b90 dofileread(c9124268,f7e38c10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf7e38bc4 kern_readv(c8874930,0,f7e38c10,0,28876000,...) at kern_readv+0x66/frame 0xf7e38bf0 sys_read(c8874930,f7e38cc8,c0ee61d0,c7200c90,0,...) at sys_read+0x5c/frame 0xf7e38c40 syscall(f7e38d08) at syscall+0x2ed/frame 0xf7e38cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7e38cfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281d19c7, esp = 0xbfbfd6dc, ebp = 0xbfbfd724 --- Tracing command mdconfig pid 2310 tid 100122 td 0xc82ad930 kdb_enter(c110d3eb,c110d3eb,c114c16f,f7df78fc,c114c16f,...) at kdb_enter+0x3d/frame 0xf7df7890 vpanic(c128a2d2,100,c114c16f,f7df78fc,f7df78fc,...) at vpanic+0x143/frame 0xf7df78cc kassert_panic(c114c16f,4,c114b49b,347,160,...) at kassert_panic+0xea/frame 0xf7df78f0 vm_page_insert_after(0,0,c4f8e5c0,330) at vm_page_insert_after+0x117/frame 0xf7df791c vm_page_insert(c913a000,c910a534,0,0,1,...) at vm_page_insert+0x5d/frame 0xf7df7940 old_dev_pager_fault(c910a534,0,0,1,f7df7a24,...) at old_dev_pager_fault+0x1ff/frame 0xf7df7984 dev_pager_getpages(c910a534,f7df7a24,1,0,f7df7afc,...) at dev_pager_getpages+0x79/frame 0xf7df79b8 vm_fault_hold(c7e83d98,28224000,1,0,0,...) at vm_fault_hold+0xa1f/frame 0xf7df7b18 vm_fault(c7e83d98,28224000,1,0,c7e83d98,...) at vm_fault+0x82/frame 0xf7df7b40 trap_pfault(28224000,c0d4f55a,c7e83d98,f7df7c18,9000,...) at trap_pfault+0x1ba/frame 0xf7df7bb8 trap(f7df7d08) at trap+0x614/frame 0xf7df7cfc calltrap() at calltrap+0x6/frame 0xf7df7cfc --- trap 0xc, eip = 0x281c602e, esp = 0xbfbfd3dc, ebp = 0xbfbfd400 --- Tracing command sleep pid 2278 tid 100147 td 0xc8876c40 sched_switch(c8876c40,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7f22a64 mi_switch(104,0,c1113475,1f5,c0afc930,...) at mi_switch+0x1c9/frame 0xf7f22a98 sleepq_switch(c8876c40,0,c1113475,1ab,8,...) at sleepq_switch+0x1c5/frame 0xf7f22acc sleepq_catch_signals(c0afc930,c8876c40,0,201,0,...) at sleepq_catch_signals+0x36c/frame 0xf7f22b14 sleepq_timedwait_sig(c128b668,6c,d4a,a0000000,0,...) at sleepq_timedwait_sig+0x14/frame 0xf7f22b3c _sleep(c128b668,0,16c,c110ead2,396a9013,...) at _sleep+0x33c/frame 0xf7f22b98 kern_nanosleep(bfbfdce0,f7f22c18,f7f22c20,2,a,...) at kern_nanosleep+0x158/frame 0xf7f22c00 sys_nanosleep(c8876c40,f7f22cc8,c116612a,d6,7b3,...) at sys_nanosleep+0x6b/frame 0xf7f22c40 syscall(f7f22d08) at syscall+0x2ed/frame 0xf7f22cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7f22cfc --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x28164d03, esp = 0xbfbfdcac, ebp = 0xbfbfdcf4 --- Tracing command sh pid 2242 tid 100149 td 0xc9129930 sched_switch(c9129930,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7f2898c mi_switch(104,0,c1113475,1f5,c1113475,...) at mi_switch+0x1c9/frame 0xf7f289c0 sleepq_switch(c9129930,0,c1113475,1ab,f7f28a38,...) at sleepq_switch+0x1c5/frame 0xf7f289f4 sleepq_catch_signals(0,c1113475,162,c00,c9018000,...) at sleepq_catch_signals+0x36c/frame 0xf7f28a3c sleepq_wait_sig(c9018000,6c,c1116f7d,100,0,...) at sleepq_wait_sig+0x14/frame 0xf7f28a60 _sleep(c9018000,c9018094,16c,c1116f7d,0,...) at _sleep+0x364/frame 0xf7f28abc kern_wait6(c9129930,7,0,0,f7f28b4c,...) at kern_wait6+0x45c/frame 0xf7f28b20 sys_wait4(c9129930,f7f28cc8,c116612a,e9,0,...) at sys_wait4+0x94/frame 0xf7f28c40 syscall(f7f28d08) at syscall+0x2ed/frame 0xf7f28cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7f28cfc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x28151e73, esp = 0xbfbfd780, ebp = 0xbfbfd798 --- Tracing command sh pid 1689 tid 100138 td 0xc8875930 sched_switch(c8875930,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7e2998c mi_switch(104,0,c1113475,1f5,c1113475,...) at mi_switch+0x1c9/frame 0xf7e299c0 sleepq_switch(c8875930,0,c1113475,1ab,f7e29a38,...) at sleepq_switch+0x1c5/frame 0xf7e299f4 sleepq_catch_signals(0,c1113475,162,12f0,c88722e8,...) at sleepq_catch_signals+0x36c/frame 0xf7e29a3c sleepq_wait_sig(c88722e8,6c,c1116f7d,100,0,...) at sleepq_wait_sig+0x14/frame 0xf7e29a60 _sleep(c88722e8,c887237c,16c,c1116f7d,0,...) at _sleep+0x364/frame 0xf7e29abc kern_wait6(c8875930,7,0,0,f7e29b4c,...) at kern_wait6+0x45c/frame 0xf7e29b20 sys_wait4(c8875930,f7e29cc8,c116612a,e9,0,...) at sys_wait4+0x94/frame 0xf7e29c40 syscall(f7e29d08) at syscall+0x2ed/frame 0xf7e29cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7e29cfc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x28151e73, esp = 0xbfbfc700, ebp = 0xbfbfc718 --- Tracing command sh pid 1101 tid 100142 td 0xc8874c40 sched_switch(c8874c40,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7e3598c mi_switch(104,0,c1113475,1f5,c1113475,...) at mi_switch+0x1c9/frame 0xf7e359c0 sleepq_switch(c8874c40,0,c1113475,1ab,f7e35a38,...) at sleepq_switch+0x1c5/frame 0xf7e359f4 sleepq_catch_signals(0,c1113475,162,1278,c88715d0,...) at sleepq_catch_signals+0x36c/frame 0xf7e35a3c sleepq_wait_sig(c88715d0,6c,c1116f7d,100,0,...) at sleepq_wait_sig+0x14/frame 0xf7e35a60 _sleep(c88715d0,c8871664,16c,c1116f7d,0,...) at _sleep+0x364/frame 0xf7e35abc kern_wait6(c8874c40,7,0,0,f7e35b4c,...) at kern_wait6+0x45c/frame 0xf7e35b20 sys_wait4(c8874c40,f7e35cc8,c116612a,e9,0,...) at sys_wait4+0x94/frame 0xf7e35c40 syscall(f7e35d08) at syscall+0x2ed/frame 0xf7e35cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7e35cfc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x28151e73, esp = 0xbfbfd810, ebp = 0xbfbfd828 --- Tracing command bash pid 1097 tid 100144 td 0xc8874620 sched_switch(c8874620,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7e3c98c mi_switch(104,0,c1113475,1f5,c1113475,...) at mi_switch+0x1c9/frame 0xf7e3c9c0 sleepq_switch(c8874620,0,c1113475,1ab,f7e3ca38,...) at sleepq_switch+0x1c5/frame 0xf7e3c9f4 sleepq_catch_signals(0,c1113475,162,180,c8871000,...) at sleepq_catch_signals+0x36c/frame 0xf7e3ca3c sleepq_wait_sig(c8871000,6c,c1116f7d,100,0,...) at sleepq_wait_sig+0x14/frame 0xf7e3ca60 _sleep(c8871000,c8871094,16c,c1116f7d,0,...) at _sleep+0x364/frame 0xf7e3cabc kern_wait6(c8874620,7,0,0,f7e3cb4c,...) at kern_wait6+0x45c/frame 0xf7e3cb20 sys_wait4(c8874620,f7e3ccc8,c0ee61d0,c71fdc90,0,...) at sys_wait4+0x94/frame 0xf7e3cc40 syscall(f7e3cd08) at syscall+0x2ed/frame 0xf7e3ccfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7e3ccfc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x282b0e73, esp = 0xbfbfd7c0, ebp = 0xbfbfd7d8 --- Tracing command csh pid 1094 tid 100135 td 0xc8aaa310 sched_switch(c8aaa310,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7e20aa0 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xf7e20ad4 sleepq_switch(c8aaa310,0,c1113475,1ab,f7e20b4c,...) at sleepq_switch+0x1c5/frame 0xf7e20b08 sleepq_catch_signals(0,c1113475,162,cc0,c9018c04,...) at sleepq_catch_signals+0x36c/frame 0xf7e20b50 sleepq_wait_sig(c9018c04,74,c0f390d4,100,0,...) at sleepq_wait_sig+0x14/frame 0xf7e20b74 _sleep(c9018c04,c9018c34,174,c0f390d4,0,...) at _sleep+0x364/frame 0xf7e20bd0 kern_sigsuspend(c8aaa310,0,0,0,0,...) at kern_sigsuspend+0xfb/frame 0xf7e20c10 sys_sigsuspend(c8aaa310,f7e20cc8,c116612a,e9,0,...) at sys_sigsuspend+0x58/frame 0xf7e20c40 syscall(f7e20d08) at syscall+0x2ed/frame 0xf7e20cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7e20cfc --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x2816c61b, esp = 0xbfbfd134, ebp = 0xbfbfd19c --- Tracing command su pid 1093 tid 100145 td 0xc8aaa000 sched_switch(c8aaa000,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7e4098c mi_switch(104,0,c1113475,1f5,c1113475,...) at mi_switch+0x1c9/frame 0xf7e409c0 sleepq_switch(c8aaa000,0,c1113475,1ab,f7e40a38,...) at sleepq_switch+0x1c5/frame 0xf7e409f4 sleepq_catch_signals(0,c1113475,162,480,c90188b8,...) at sleepq_catch_signals+0x36c/frame 0xf7e40a3c sleepq_wait_sig(c90188b8,6c,c1116f7d,100,0,...) at sleepq_wait_sig+0x14/frame 0xf7e40a60 _sleep(c90188b8,c901894c,16c,c1116f7d,0,...) at _sleep+0x364/frame 0xf7e40abc kern_wait6(c8aaa000,0,446,0,f7e40b4c,...) at kern_wait6+0x45c/frame 0xf7e40b20 sys_wait4(c8aaa000,f7e40cc8,c0ee61d0,c71fdc90,0,...) at sys_wait4+0x94/frame 0xf7e40c40 syscall(f7e40d08) at syscall+0x2ed/frame 0xf7e40cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7e40cfc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2810fe73, esp = 0xbfbfd650, ebp = 0xbfbfd668 --- Tracing command bash pid 1073 tid 100140 td 0xc8875310 sched_switch(c8875310,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7e2f98c mi_switch(104,0,c1113475,1f5,c1113475,...) at mi_switch+0x1c9/frame 0xf7e2f9c0 sleepq_switch(c8875310,0,c1113475,1ab,f7e2fa38,...) at sleepq_switch+0x1c5/frame 0xf7e2f9f4 sleepq_catch_signals(0,c1113475,162,1188,c8871ba0,...) at sleepq_catch_signals+0x36c/frame 0xf7e2fa3c sleepq_wait_sig(c8871ba0,6c,c1116f7d,100,0,...) at sleepq_wait_sig+0x14/frame 0xf7e2fa60 _sleep(c8871ba0,c8871c34,16c,c1116f7d,0,...) at _sleep+0x364/frame 0xf7e2fabc kern_wait6(c8875310,7,0,0,f7e2fb4c,...) at kern_wait6+0x45c/frame 0xf7e2fb20 sys_wait4(c8875310,f7e2fcc8,c0ee61d0,c1825c90,0,...) at sys_wait4+0x94/frame 0xf7e2fc40 syscall(f7e2fd08) at syscall+0x2ed/frame 0xf7e2fcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7e2fcfc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x282b0e73, esp = 0xbfbfd9b0, ebp = 0xbfbfd9c8 --- Tracing command sshd pid 1072 tid 100114 td 0xc7e57930 sched_switch(c7e57930,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7dde90c mi_switch(104,0,c1113475,1f5,200046,...) at mi_switch+0x1c9/frame 0xf7dde940 sleepq_switch(c7e57930,0,c1113475,1ab,c8151e20,...) at sleepq_switch+0x1c5/frame 0xf7dde974 sleepq_catch_signals(f7dde9e0,c0aa24a9,c8151e20,0,0,...) at sleepq_catch_signals+0x36c/frame 0xf7dde9bc sleepq_wait_sig(c8151e24,0,f7ddea10,101,0,...) at sleepq_wait_sig+0x14/frame 0xf7dde9e0 _cv_wait_sig(c8151e24,c8151e10,c1116849,6c6,1b2,...) at _cv_wait_sig+0x200/frame 0xf7ddea24 seltdwait(ffffffff,0,0,c7e57930,0,...) at seltdwait+0x90/frame 0xf7ddea58 kern_select(c7e57930,c,28c0c1c8,28c0c1d0,0,...) at kern_select+0x863/frame 0xf7ddebf8 sys_select(c7e57930,f7ddecc8,ef,a7c03d21,d34,...) at sys_select+0x69/frame 0xf7ddec40 syscall(f7dded08) at syscall+0x2ed/frame 0xf7ddecfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7ddecfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28532987, esp = 0xbfbfd374, ebp = 0xbfbfd3c0 --- Tracing command sshd pid 1070 tid 100137 td 0xc8875c40 sched_switch(c8875c40,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7e26984 mi_switch(104,0,c1113475,1f5,200046,...) at mi_switch+0x1c9/frame 0xf7e269b8 sleepq_switch(c8875c40,0,c1113475,1ab,c7d14d60,...) at sleepq_switch+0x1c5/frame 0xf7e269ec sleepq_catch_signals(f7e26a58,c0aa24a9,c7d14d60,0,0,...) at sleepq_catch_signals+0x36c/frame 0xf7e26a34 sleepq_wait_sig(c7d14d64,0,f7e26a88,101,0,...) at sleepq_wait_sig+0x14/frame 0xf7e26a58 _cv_wait_sig(c7d14d64,c7d14d50,c1116849,6c6,5c3,...) at _cv_wait_sig+0x200/frame 0xf7e26a9c seltdwait(ffffffff,0,0,c8875c40,c12163f4,...) at seltdwait+0x90/frame 0xf7e26ad0 sys_poll(c8875c40,f7e26cc8,c0ee61d0,c71fdc90,0,...) at sys_poll+0x44c/frame 0xf7e26c40 syscall(f7e26d08) at syscall+0x2ed/frame 0xf7e26cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7e26cfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x284c8adf, esp = 0xbfbfd3bc, ebp = 0xbfbfd408 --- Tracing command awk pid 1054 tid 100136 td 0xc82ad310 sched_switch(c82ad310,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7e23a1c mi_switch(104,0,c1113475,1f5,c1113475,...) at mi_switch+0x1c9/frame 0xf7e23a50 sleepq_switch(c82ad310,0,c1113475,1ab,f7e23ac8,...) at sleepq_switch+0x1c5/frame 0xf7e23a84 sleepq_catch_signals(0,c1113475,162,6d8,c84dd198,...) at sleepq_catch_signals+0x36c/frame 0xf7e23acc sleepq_wait_sig(c84dd198,5c,c1116a8e,100,0,...) at sleepq_wait_sig+0x14/frame 0xf7e23af0 _sleep(c84dd198,c84dd318,15c,c1116a8e,0,...) at _sleep+0x364/frame 0xf7e23b4c pipe_read(c7e4d968,f7e23c10,c76b2600,0,c82ad310,...) at pipe_read+0x2dc/frame 0xf7e23b90 dofileread(c7e4d968,f7e23c10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf7e23bc4 kern_readv(c82ad310,0,f7e23c10,0,28855000,...) at kern_readv+0x66/frame 0xf7e23bf0 sys_read(c82ad310,f7e23cc8,c0ee61d0,c7200c90,0,...) at sys_read+0x5c/frame 0xf7e23c40 syscall(f7e23d08) at syscall+0x2ed/frame 0xf7e23cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7e23cfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281c79c7, esp = 0xbfbfd9e0, ebp = 0xbfbfd9f4 --- Tracing command sh pid 1053 tid 100123 td 0xc82ad620 sched_switch(c82ad620,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7dfa98c mi_switch(104,0,c1113475,1f5,c1113475,...) at mi_switch+0x1c9/frame 0xf7dfa9c0 sleepq_switch(c82ad620,0,c1113475,1ab,f7dfaa38,...) at sleepq_switch+0x1c5/frame 0xf7dfa9f4 sleepq_catch_signals(0,c1113475,162,0,c82ab8b8,...) at sleepq_catch_signals+0x36c/frame 0xf7dfaa3c sleepq_wait_sig(c82ab8b8,6c,c1116f7d,100,0,...) at sleepq_wait_sig+0x14/frame 0xf7dfaa60 _sleep(c82ab8b8,c82ab94c,16c,c1116f7d,0,...) at _sleep+0x364/frame 0xf7dfaabc kern_wait6(c82ad620,7,0,0,f7dfab4c,...) at kern_wait6+0x45c/frame 0xf7dfab20 sys_wait4(c82ad620,f7dfacc8,c116612a,e9,0,...) at sys_wait4+0x94/frame 0xf7dfac40 syscall(f7dfad08) at syscall+0x2ed/frame 0xf7dfacfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7dfacfc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x28151e73, esp = 0xbfbfd940, ebp = 0xbfbfd958 --- Tracing command top pid 1052 tid 100125 td 0xc8aae000 sched_switch(c8aae000,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7e00900 mi_switch(104,0,c1113475,1f5,400,...) at mi_switch+0x1c9/frame 0xf7e00934 sleepq_switch(c8aae000,0,c1113475,1ab,c110af0a,...) at sleepq_switch+0x1c5/frame 0xf7e00968 sleepq_catch_signals(c1214040,f7e009d8,c0aa24a9,c8151da0,0,...) at sleepq_catch_signals+0x36c/frame 0xf7e009b0 sleepq_timedwait_sig(c8151da4,0,f7e00a10,10000000,0,...) at sleepq_timedwait_sig+0x14/frame 0xf7e009d8 _cv_timedwait_sig_sbt(c8151da4,c8151d90,91fe40c0,d40,10000000,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf7e00a24 seltdwait(d40,10000000,0,c8aae000,0,...) at seltdwait+0xbc/frame 0xf7e00a58 kern_select(c8aae000,2,bfbfdbe8,0,0,...) at kern_select+0x863/frame 0xf7e00bf8 sys_select(c8aae000,f7e00cc8,c0ee61d0,c7200c90,0,...) at sys_select+0x69/frame 0xf7e00c40 syscall(f7e00d08) at syscall+0x2ed/frame 0xf7e00cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7e00cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281f9987, esp = 0xbfbfdb8c, ebp = 0xbfbfdcf4 --- Tracing command tail pid 1051 tid 100115 td 0xc7e57620 sched_switch(c7e57620,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7de193c mi_switch(104,0,c1113475,1f5,c0afc930,...) at mi_switch+0x1c9/frame 0xf7de1970 sleepq_switch(c7e57620,0,c1113475,1ab,8,...) at sleepq_switch+0x1c5/frame 0xf7de19a4 sleepq_catch_signals(c0afc930,c7e57620,1,201,1,...) at sleepq_catch_signals+0x36c/frame 0xf7de19ec sleepq_timedwait_sig(c7d80480,68,d40,10000000,0,...) at sleepq_timedwait_sig+0x14/frame 0xf7de1a14 _sleep(c7d80480,c7d80480,168,c1106879,f44bc63d,...) at _sleep+0x33c/frame 0xf7de1a70 kern_kevent(c7e57620,4,0,1,f7de1c10,...) at kern_kevent+0x44c/frame 0xf7de1bb8 sys_kevent(c7e57620,f7de1cc8,c0ee61d0,c1825c90,0,...) at sys_kevent+0x12b/frame 0xf7de1c40 syscall(f7de1d08) at syscall+0x2ed/frame 0xf7de1cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7de1cfc --- syscall (363, FreeBSD ELF32, sys_kevent), eip = 0x28168e1f, esp = 0xbfbfdb9c, ebp = 0xbfbfdc48 --- Tracing command sh pid 1050 tid 100109 td 0xc8454000 sched_switch(c8454000,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7dcf98c mi_switch(104,0,c1113475,1f5,c1113475,...) at mi_switch+0x1c9/frame 0xf7dcf9c0 sleepq_switch(c8454000,0,c1113475,1ab,f7dcfa38,...) at sleepq_switch+0x1c5/frame 0xf7dcf9f4 sleepq_catch_signals(0,c1113475,162,f00,c84518b8,...) at sleepq_catch_signals+0x36c/frame 0xf7dcfa3c sleepq_wait_sig(c84518b8,6c,c1116f7d,100,0,...) at sleepq_wait_sig+0x14/frame 0xf7dcfa60 _sleep(c84518b8,c845194c,16c,c1116f7d,0,...) at _sleep+0x364/frame 0xf7dcfabc kern_wait6(c8454000,7,0,0,f7dcfb4c,...) at kern_wait6+0x45c/frame 0xf7dcfb20 sys_wait4(c8454000,f7dcfcc8,c0ee61d0,c71fdc90,0,...) at sys_wait4+0x94/frame 0xf7dcfc40 syscall(f7dcfd08) at syscall+0x2ed/frame 0xf7dcfcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7dcfcfc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x28151e73, esp = 0xbfbfdb28, ebp = 0xbfbfdb40 --- Tracing command sshd pid 1049 tid 100118 td 0xc82b5620 sched_switch(c82b5620,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7dea90c mi_switch(104,0,c1113475,1f5,200046,...) at mi_switch+0x1c9/frame 0xf7dea940 sleepq_switch(c82b5620,0,c1113475,1ab,c8151e60,...) at sleepq_switch+0x1c5/frame 0xf7dea974 sleepq_catch_signals(f7dea9e0,c0aa24a9,c8151e60,0,0,...) at sleepq_catch_signals+0x36c/frame 0xf7dea9bc sleepq_wait_sig(c8151e64,0,f7deaa10,101,0,...) at sleepq_wait_sig+0x14/frame 0xf7dea9e0 _cv_wait_sig(c8151e64,c8151e50,c1116849,6c6,1b2,...) at _cv_wait_sig+0x200/frame 0xf7deaa24 seltdwait(ffffffff,0,0,c82b5620,0,...) at seltdwait+0x90/frame 0xf7deaa58 kern_select(c82b5620,a,28c0c1c8,28c0c1d0,0,...) at kern_select+0x863/frame 0xf7deabf8 sys_select(c82b5620,f7deacc8,c0ee61d0,c71fac90,0,...) at sys_select+0x69/frame 0xf7deac40 syscall(f7dead08) at syscall+0x2ed/frame 0xf7deacfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7deacfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28532987, esp = 0xbfbfd374, ebp = 0xbfbfd3c0 --- Tracing command sshd pid 1048 tid 100110 td 0xc82b5c40 sched_switch(c82b5c40,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7dd290c mi_switch(104,0,c1113475,1f5,200046,...) at mi_switch+0x1c9/frame 0xf7dd2940 sleepq_switch(c82b5c40,0,c1113475,1ab,c7d14de0,...) at sleepq_switch+0x1c5/frame 0xf7dd2974 sleepq_catch_signals(f7dd29e0,c0aa24a9,c7d14de0,0,0,...) at sleepq_catch_signals+0x36c/frame 0xf7dd29bc sleepq_wait_sig(c7d14de4,0,f7dd2a10,101,0,...) at sleepq_wait_sig+0x14/frame 0xf7dd29e0 _cv_wait_sig(c7d14de4,c7d14dd0,c1116849,6c6,5d0,...) at _cv_wait_sig+0x200/frame 0xf7dd2a24 seltdwait(ffffffff,0,0,c82b5c40,0,...) at seltdwait+0x90/frame 0xf7dd2a58 kern_select(c82b5c40,c,28c0c1c8,28c0c1d0,0,...) at kern_select+0x863/frame 0xf7dd2bf8 sys_select(c82b5c40,f7dd2cc8,c0ee61d0,c1825c90,0,...) at sys_select+0x69/frame 0xf7dd2c40 syscall(f7dd2d08) at syscall+0x2ed/frame 0xf7dd2cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7dd2cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28532987, esp = 0xbfbfd374, ebp = 0xbfbfd3c0 --- Tracing command sshd pid 1047 tid 100111 td 0xc82b5930 sched_switch(c82b5930,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7dd590c mi_switch(104,0,c1113475,1f5,200046,...) at mi_switch+0x1c9/frame 0xf7dd5940 sleepq_switch(c82b5930,0,c1113475,1ab,c78e6720,...) at sleepq_switch+0x1c5/frame 0xf7dd5974 sleepq_catch_signals(f7dd59e0,c0aa24a9,c78e6720,0,0,...) at sleepq_catch_signals+0x36c/frame 0xf7dd59bc sleepq_wait_sig(c78e6724,0,f7dd5a10,101,0,...) at sleepq_wait_sig+0x14/frame 0xf7dd59e0 _cv_wait_sig(c78e6724,c78e6710,c1116849,6c6,5d0,...) at _cv_wait_sig+0x200/frame 0xf7dd5a24 seltdwait(ffffffff,0,0,c82b5930,0,...) at seltdwait+0x90/frame 0xf7dd5a58 kern_select(c82b5930,c,28c0c1c8,28c0c1d0,0,...) at kern_select+0x863/frame 0xf7dd5bf8 sys_select(c82b5930,f7dd5cc8,8,c110dac4,db,...) at sys_select+0x69/frame 0xf7dd5c40 syscall(f7dd5d08) at syscall+0x2ed/frame 0xf7dd5cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7dd5cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28532987, esp = 0xbfbfd374, ebp = 0xbfbfd3c0 --- Tracing command sshd pid 1043 tid 100126 td 0xc7e57310 sched_switch(c7e57310,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7e03984 mi_switch(104,0,c1113475,1f5,200046,...) at mi_switch+0x1c9/frame 0xf7e039b8 sleepq_switch(c7e57310,0,c1113475,1ab,c78e6ca0,...) at sleepq_switch+0x1c5/frame 0xf7e039ec sleepq_catch_signals(f7e03a58,c0aa24a9,c78e6ca0,0,0,...) at sleepq_catch_signals+0x36c/frame 0xf7e03a34 sleepq_wait_sig(c78e6ca4,0,f7e03a88,101,0,...) at sleepq_wait_sig+0x14/frame 0xf7e03a58 _cv_wait_sig(c78e6ca4,c78e6c90,c1116849,6c6,5c3,...) at _cv_wait_sig+0x200/frame 0xf7e03a9c seltdwait(ffffffff,0,0,c7e57310,0,...) at seltdwait+0x90/frame 0xf7e03ad0 sys_poll(c7e57310,f7e03cc8,14,c110f9e5,7b3,...) at sys_poll+0x44c/frame 0xf7e03c40 syscall(f7e03d08) at syscall+0x2ed/frame 0xf7e03cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7e03cfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x284c8adf, esp = 0xbfbfd3bc, ebp = 0xbfbfd408 --- Tracing command sshd pid 1042 tid 100102 td 0xc8455620 sched_switch(c8455620,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7d97984 mi_switch(104,0,c1113475,1f5,200046,...) at mi_switch+0x1c9/frame 0xf7d979b8 sleepq_switch(c8455620,0,c1113475,1ab,c78e67a0,...) at sleepq_switch+0x1c5/frame 0xf7d979ec sleepq_catch_signals(f7d97a58,c0aa24a9,c78e67a0,0,0,...) at sleepq_catch_signals+0x36c/frame 0xf7d97a34 sleepq_wait_sig(c78e67a4,0,f7d97a88,101,0,...) at sleepq_wait_sig+0x14/frame 0xf7d97a58 _cv_wait_sig(c78e67a4,c78e6790,c1116849,6c6,5c3,...) at _cv_wait_sig+0x200/frame 0xf7d97a9c seltdwait(ffffffff,0,0,c8455620,0,...) at seltdwait+0x90/frame 0xf7d97ad0 sys_poll(c8455620,f7d97cc8,c0ee61d0,c71fdc90,0,...) at sys_poll+0x44c/frame 0xf7d97c40 syscall(f7d97d08) at syscall+0x2ed/frame 0xf7d97cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7d97cfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x284c8adf, esp = 0xbfbfd3bc, ebp = 0xbfbfd408 --- Tracing command sshd pid 1041 tid 100075 td 0xc7bcc930 sched_switch(c7bcc930,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7cd6984 mi_switch(104,0,c1113475,1f5,200046,...) at mi_switch+0x1c9/frame 0xf7cd69b8 sleepq_switch(c7bcc930,0,c1113475,1ab,c7d14da0,...) at sleepq_switch+0x1c5/frame 0xf7cd69ec sleepq_catch_signals(f7cd6a58,c0aa24a9,c7d14da0,0,0,...) at sleepq_catch_signals+0x36c/frame 0xf7cd6a34 sleepq_wait_sig(c7d14da4,0,f7cd6a88,101,0,...) at sleepq_wait_sig+0x14/frame 0xf7cd6a58 _cv_wait_sig(c7d14da4,c7d14d90,c1116849,6c6,5c3,...) at _cv_wait_sig+0x200/frame 0xf7cd6a9c seltdwait(ffffffff,0,0,c7bcc930,0,...) at seltdwait+0x90/frame 0xf7cd6ad0 sys_poll(c7bcc930,f7cd6cc8,14,c110f9e5,7b3,...) at sys_poll+0x44c/frame 0xf7cd6c40 syscall(f7cd6d08) at syscall+0x2ed/frame 0xf7cd6cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7cd6cfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x284c8adf, esp = 0xbfbfd3bc, ebp = 0xbfbfd408 --- Tracing command getty pid 838 tid 100133 td 0xc8aaa930 sched_switch(c8aaa930,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7e1a9bc mi_switch(104,0,c1113475,1f5,c1113475,...) at mi_switch+0x1c9/frame 0xf7e1a9f0 sleepq_switch(c8aaa930,0,c1113475,1ab,f7e1aa68,...) at sleepq_switch+0x1c5/frame 0xf7e1aa24 sleepq_catch_signals(0,c1113475,162,1230,1,...) at sleepq_catch_signals+0x36c/frame 0xf7e1aa6c sleepq_wait_sig(c77bb270,0,c1118891,101,0,...) at sleepq_wait_sig+0x14/frame 0xf7e1aa90 _cv_wait_sig(c77bb270,c1454320,c111887d,531,f7e1ac10,...) at _cv_wait_sig+0x200/frame 0xf7e1aad4 tty_wait(c77bb200,c77bb270,f7e1ac10,1,0,...) at tty_wait+0x4b/frame 0xf7e1aaf8 ttydisc_read(c77bb200,f7e1ac10,0,a1,0,...) at ttydisc_read+0x377/frame 0xf7e1ab3c ttydev_read(c7792b00,f7e1ac10,0,1,0,...) at ttydev_read+0x81/frame 0xf7e1ab5c devfs_read_f(c7e4dd90,f7e1ac10,c76b4f00,0,c8aaa930,...) at devfs_read_f+0x84/frame 0xf7e1ab90 dofileread(c7e4dd90,f7e1ac10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf7e1abc4 kern_readv(c8aaa930,0,f7e1ac10,0,bfbfddfb,...) at kern_readv+0x66/frame 0xf7e1abf0 sys_read(c8aaa930,f7e1acc8,c0ee61d0,c71fdc90,0,...) at sys_read+0x5c/frame 0xf7e1ac40 syscall(f7e1ad08) at syscall+0x2ed/frame 0xf7e1acfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7e1acfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2819d9c7, esp = 0xbfbfdde0, ebp = 0xbfbfde08 --- Tracing command getty pid 837 tid 100132 td 0xc8aaac40 sched_switch(c8aaac40,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7e179bc mi_switch(104,0,c1113475,1f5,c1113475,...) at mi_switch+0x1c9/frame 0xf7e179f0 sleepq_switch(c8aaac40,0,c1113475,1ab,f7e17a68,...) at sleepq_switch+0x1c5/frame 0xf7e17a24 sleepq_catch_signals(0,c1113475,162,1260,1,...) at sleepq_catch_signals+0x36c/frame 0xf7e17a6c sleepq_wait_sig(c77bb470,0,c1118891,101,0,...) at sleepq_wait_sig+0x14/frame 0xf7e17a90 _cv_wait_sig(c77bb470,c1454320,c111887d,531,f7e17c10,...) at _cv_wait_sig+0x200/frame 0xf7e17ad4 tty_wait(c77bb400,c77bb470,f7e17c10,1,0,...) at tty_wait+0x4b/frame 0xf7e17af8 ttydisc_read(c77bb400,f7e17c10,0,a1,0,...) at ttydisc_read+0x377/frame 0xf7e17b3c ttydev_read(c7792c00,f7e17c10,0,1,0,...) at ttydev_read+0x81/frame 0xf7e17b5c devfs_read_f(c7e4de38,f7e17c10,c76b4f00,0,c8aaac40,...) at devfs_read_f+0x84/frame 0xf7e17b90 dofileread(c7e4de38,f7e17c10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf7e17bc4 kern_readv(c8aaac40,0,f7e17c10,0,bfbfddfb,...) at kern_readv+0x66/frame 0xf7e17bf0 sys_read(c8aaac40,f7e17cc8,c0ee61d0,c1825c90,0,...) at sys_read+0x5c/frame 0xf7e17c40 syscall(f7e17d08) at syscall+0x2ed/frame 0xf7e17cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7e17cfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2819d9c7, esp = 0xbfbfdde0, ebp = 0xbfbfde08 --- Tracing command getty pid 836 tid 100131 td 0xc8aab000 sched_switch(c8aab000,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7e149bc mi_switch(104,0,c1113475,1f5,c1113475,...) at mi_switch+0x1c9/frame 0xf7e149f0 sleepq_switch(c8aab000,0,c1113475,1ab,f7e14a68,...) at sleepq_switch+0x1c5/frame 0xf7e14a24 sleepq_catch_signals(0,c1113475,162,1290,1,...) at sleepq_catch_signals+0x36c/frame 0xf7e14a6c sleepq_wait_sig(c77bb670,0,c1118891,101,0,...) at sleepq_wait_sig+0x14/frame 0xf7e14a90 _cv_wait_sig(c77bb670,c1454320,c111887d,531,f7e14c10,...) at _cv_wait_sig+0x200/frame 0xf7e14ad4 tty_wait(c77bb600,c77bb670,f7e14c10,1,0,...) at tty_wait+0x4b/frame 0xf7e14af8 ttydisc_read(c77bb600,f7e14c10,0,a1,0,...) at ttydisc_read+0x377/frame 0xf7e14b3c ttydev_read(c7792d00,f7e14c10,0,1,0,...) at ttydev_read+0x81/frame 0xf7e14b5c devfs_read_f(c7e8f230,f7e14c10,c76b4f00,0,c8aab000,...) at devfs_read_f+0x84/frame 0xf7e14b90 dofileread(c7e8f230,f7e14c10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf7e14bc4 kern_readv(c8aab000,0,f7e14c10,0,bfbfddfb,...) at kern_readv+0x66/frame 0xf7e14bf0 sys_read(c8aab000,f7e14cc8,14,c110f9e5,7b3,...) at sys_read+0x5c/frame 0xf7e14c40 syscall(f7e14d08) at syscall+0x2ed/frame 0xf7e14cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7e14cfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2819d9c7, esp = 0xbfbfdde0, ebp = 0xbfbfde08 --- Tracing command getty pid 835 tid 100130 td 0xc8aab310 sched_switch(c8aab310,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7e119bc mi_switch(104,0,c1113475,1f5,c1113475,...) at mi_switch+0x1c9/frame 0xf7e119f0 sleepq_switch(c8aab310,0,c1113475,1ab,f7e11a68,...) at sleepq_switch+0x1c5/frame 0xf7e11a24 sleepq_catch_signals(0,c1113475,162,12c0,1,...) at sleepq_catch_signals+0x36c/frame 0xf7e11a6c sleepq_wait_sig(c77bb870,0,c1118891,101,0,...) at sleepq_wait_sig+0x14/frame 0xf7e11a90 _cv_wait_sig(c77bb870,c1454320,c111887d,531,f7e11c10,...) at _cv_wait_sig+0x200/frame 0xf7e11ad4 tty_wait(c77bb800,c77bb870,f7e11c10,1,0,...) at tty_wait+0x4b/frame 0xf7e11af8 ttydisc_read(c77bb800,f7e11c10,0,a1,0,...) at ttydisc_read+0x377/frame 0xf7e11b3c ttydev_read(c7792e00,f7e11c10,0,1,0,...) at ttydev_read+0x81/frame 0xf7e11b5c devfs_read_f(c7e4dd58,f7e11c10,c76b4f00,0,c8aab310,...) at devfs_read_f+0x84/frame 0xf7e11b90 dofileread(c7e4dd58,f7e11c10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf7e11bc4 kern_readv(c8aab310,0,f7e11c10,0,bfbfddfb,...) at kern_readv+0x66/frame 0xf7e11bf0 sys_read(c8aab310,f7e11cc8,c0ee61d0,c71fac90,0,...) at sys_read+0x5c/frame 0xf7e11c40 syscall(f7e11d08) at syscall+0x2ed/frame 0xf7e11cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7e11cfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2819d9c7, esp = 0xbfbfdde0, ebp = 0xbfbfde08 --- Tracing command getty pid 834 tid 100129 td 0xc8aab620 sched_switch(c8aab620,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7e0e9bc mi_switch(104,0,c1113475,1f5,c1113475,...) at mi_switch+0x1c9/frame 0xf7e0e9f0 sleepq_switch(c8aab620,0,c1113475,1ab,f7e0ea68,...) at sleepq_switch+0x1c5/frame 0xf7e0ea24 sleepq_catch_signals(0,c1113475,162,12f0,1,...) at sleepq_catch_signals+0x36c/frame 0xf7e0ea6c sleepq_wait_sig(c77bba70,0,c1118891,101,0,...) at sleepq_wait_sig+0x14/frame 0xf7e0ea90 _cv_wait_sig(c77bba70,c1454320,c111887d,531,f7e0ec10,...) at _cv_wait_sig+0x200/frame 0xf7e0ead4 tty_wait(c77bba00,c77bba70,f7e0ec10,1,0,...) at tty_wait+0x4b/frame 0xf7e0eaf8 ttydisc_read(c77bba00,f7e0ec10,0,a1,0,...) at ttydisc_read+0x377/frame 0xf7e0eb3c ttydev_read(c7793000,f7e0ec10,0,1,0,...) at ttydev_read+0x81/frame 0xf7e0eb5c devfs_read_f(c7e25e70,f7e0ec10,c76b4f00,0,c8aab620,...) at devfs_read_f+0x84/frame 0xf7e0eb90 dofileread(c7e25e70,f7e0ec10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf7e0ebc4 kern_readv(c8aab620,0,f7e0ec10,0,bfbfddfb,...) at kern_readv+0x66/frame 0xf7e0ebf0 sys_read(c8aab620,f7e0ecc8,c0ee61d0,c7200c90,0,...) at sys_read+0x5c/frame 0xf7e0ec40 syscall(f7e0ed08) at syscall+0x2ed/frame 0xf7e0ecfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7e0ecfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2819d9c7, esp = 0xbfbfdde0, ebp = 0xbfbfde08 --- Tracing command getty pid 833 tid 100128 td 0xc8aab930 sched_switch(c8aab930,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7e0b9bc mi_switch(104,0,c1113475,1f5,c1113475,...) at mi_switch+0x1c9/frame 0xf7e0b9f0 sleepq_switch(c8aab930,0,c1113475,1ab,f7e0ba68,...) at sleepq_switch+0x1c5/frame 0xf7e0ba24 sleepq_catch_signals(0,c1113475,162,1320,1,...) at sleepq_catch_signals+0x36c/frame 0xf7e0ba6c sleepq_wait_sig(c77bbc70,0,c1118891,101,0,...) at sleepq_wait_sig+0x14/frame 0xf7e0ba90 _cv_wait_sig(c77bbc70,c1454320,c111887d,531,f7e0bc10,...) at _cv_wait_sig+0x200/frame 0xf7e0bad4 tty_wait(c77bbc00,c77bbc70,f7e0bc10,1,0,...) at tty_wait+0x4b/frame 0xf7e0baf8 ttydisc_read(c77bbc00,f7e0bc10,0,a1,0,...) at ttydisc_read+0x377/frame 0xf7e0bb3c ttydev_read(c7793100,f7e0bc10,0,1,0,...) at ttydev_read+0x81/frame 0xf7e0bb5c devfs_read_f(c7e4cea8,f7e0bc10,c76b4f00,0,c8aab930,...) at devfs_read_f+0x84/frame 0xf7e0bb90 dofileread(c7e4cea8,f7e0bc10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf7e0bbc4 kern_readv(c8aab930,0,f7e0bc10,0,bfbfddfb,...) at kern_readv+0x66/frame 0xf7e0bbf0 sys_read(c8aab930,f7e0bcc8,c0ee61d0,c1825c90,0,...) at sys_read+0x5c/frame 0xf7e0bc40 syscall(f7e0bd08) at syscall+0x2ed/frame 0xf7e0bcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7e0bcfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2819d9c7, esp = 0xbfbfdde0, ebp = 0xbfbfde08 --- Tracing command getty pid 832 tid 100127 td 0xc8aabc40 sched_switch(c8aabc40,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7e089bc mi_switch(104,0,c1113475,1f5,c1113475,...) at mi_switch+0x1c9/frame 0xf7e089f0 sleepq_switch(c8aabc40,0,c1113475,1ab,f7e08a68,...) at sleepq_switch+0x1c5/frame 0xf7e08a24 sleepq_catch_signals(0,c1113475,162,1350,1,...) at sleepq_catch_signals+0x36c/frame 0xf7e08a6c sleepq_wait_sig(c77bbe70,0,c1118891,101,0,...) at sleepq_wait_sig+0x14/frame 0xf7e08a90 _cv_wait_sig(c77bbe70,c1454320,c111887d,531,f7e08c10,...) at _cv_wait_sig+0x200/frame 0xf7e08ad4 tty_wait(c77bbe00,c77bbe70,f7e08c10,1,0,...) at tty_wait+0x4b/frame 0xf7e08af8 ttydisc_read(c77bbe00,f7e08c10,0,a1,0,...) at ttydisc_read+0x377/frame 0xf7e08b3c ttydev_read(c7793200,f7e08c10,0,1,0,...) at ttydev_read+0x81/frame 0xf7e08b5c devfs_read_f(c7dc7b28,f7e08c10,c76b4f00,0,c8aabc40,...) at devfs_read_f+0x84/frame 0xf7e08b90 dofileread(c7dc7b28,f7e08c10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf7e08bc4 kern_readv(c8aabc40,0,f7e08c10,0,bfbfddfb,...) at kern_readv+0x66/frame 0xf7e08bf0 sys_read(c8aabc40,f7e08cc8,c0ee61d0,c7200c90,0,...) at sys_read+0x5c/frame 0xf7e08c40 syscall(f7e08d08) at syscall+0x2ed/frame 0xf7e08cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7e08cfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2819d9c7, esp = 0xbfbfdde0, ebp = 0xbfbfde08 --- Tracing command getty pid 831 tid 100112 td 0xc82b4000 sched_switch(c82b4000,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7dd89bc mi_switch(104,0,c1113475,1f5,c1113475,...) at mi_switch+0x1c9/frame 0xf7dd89f0 sleepq_switch(c82b4000,0,c1113475,1ab,f7dd8a68,...) at sleepq_switch+0x1c5/frame 0xf7dd8a24 sleepq_catch_signals(0,c1113475,162,1080,1,...) at sleepq_catch_signals+0x36c/frame 0xf7dd8a6c sleepq_wait_sig(c77bc070,0,c1118891,101,0,...) at sleepq_wait_sig+0x14/frame 0xf7dd8a90 _cv_wait_sig(c77bc070,c1454320,c111887d,531,f7dd8c10,...) at _cv_wait_sig+0x200/frame 0xf7dd8ad4 tty_wait(c77bc000,c77bc070,f7dd8c10,1,0,...) at tty_wait+0x4b/frame 0xf7dd8af8 ttydisc_read(c77bc000,f7dd8c10,0,a1,0,...) at ttydisc_read+0x377/frame 0xf7dd8b3c ttydev_read(c7793300,f7dd8c10,0,1,0,...) at ttydev_read+0x81/frame 0xf7dd8b5c devfs_read_f(c7dc8b60,f7dd8c10,c76b4f00,0,c82b4000,...) at devfs_read_f+0x84/frame 0xf7dd8b90 dofileread(c7dc8b60,f7dd8c10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf7dd8bc4 kern_readv(c82b4000,0,f7dd8c10,0,bfbfddfb,...) at kern_readv+0x66/frame 0xf7dd8bf0 sys_read(c82b4000,f7dd8cc8,c0ee61d0,c71fdc90,0,...) at sys_read+0x5c/frame 0xf7dd8c40 syscall(f7dd8d08) at syscall+0x2ed/frame 0xf7dd8cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7dd8cfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2819d9c7, esp = 0xbfbfdde0, ebp = 0xbfbfde08 --- Tracing command getty pid 830 tid 100113 td 0xc82adc40 sched_switch(c82adc40,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7ddb9bc mi_switch(104,0,c1113475,1f5,46,...) at mi_switch+0x1c9/frame 0xf7ddb9f0 sleepq_switch(c82adc40,0,c1113475,1ab,c77bc214,...) at sleepq_switch+0x1c5/frame 0xf7ddba24 sleepq_catch_signals(f7ddba90,c0aa24a9,c77bc214,0,0,...) at sleepq_catch_signals+0x36c/frame 0xf7ddba6c sleepq_wait_sig(c77bc270,0,f7ddbac0,101,0,...) at sleepq_wait_sig+0x14/frame 0xf7ddba90 _cv_wait_sig(c77bc270,c77bc204,c111887d,531,f7ddbc10,...) at _cv_wait_sig+0x200/frame 0xf7ddbad4 tty_wait(c77bc200,c77bc270,f7ddbc10,1,0,...) at tty_wait+0x4b/frame 0xf7ddbaf8 ttydisc_read(c77bc200,f7ddbc10,0,a1,0,...) at ttydisc_read+0x377/frame 0xf7ddbb3c ttydev_read(c7795d00,f7ddbc10,0,1,0,...) at ttydev_read+0x81/frame 0xf7ddbb5c devfs_read_f(c7e4cf50,f7ddbc10,c76b4f00,0,c82adc40,...) at devfs_read_f+0x84/frame 0xf7ddbb90 dofileread(c7e4cf50,f7ddbc10,ffffffff,ffffffff,0,...) at dofileread+0x99/frame 0xf7ddbbc4 kern_readv(c82adc40,0,f7ddbc10,0,bfbfddfb,...) at kern_readv+0x66/frame 0xf7ddbbf0 sys_read(c82adc40,f7ddbcc8,c0ee61d0,c71fac90,0,...) at sys_read+0x5c/frame 0xf7ddbc40 syscall(f7ddbd08) at syscall+0x2ed/frame 0xf7ddbcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7ddbcfc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x2819d9c7, esp = 0xbfbfdde0, ebp = 0xbfbfde08 --- Tracing command inetd pid 813 tid 100120 td 0xc8876310 sched_switch(c8876310,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7df090c mi_switch(104,0,c1113475,1f5,46,...) at mi_switch+0x1c9/frame 0xf7df0940 sleepq_switch(c8876310,0,c1113475,1ab,c78e6820,...) at sleepq_switch+0x1c5/frame 0xf7df0974 sleepq_catch_signals(f7df09e0,c0aa24a9,c78e6820,0,0,...) at sleepq_catch_signals+0x36c/frame 0xf7df09bc sleepq_wait_sig(c78e6824,0,f7df0a10,101,0,...) at sleepq_wait_sig+0x14/frame 0xf7df09e0 _cv_wait_sig(c78e6824,c78e6810,c1116849,6c6,c0b1747d,...) at _cv_wait_sig+0x200/frame 0xf7df0a24 seltdwait(ffffffff,0,0,c8876310,0,...) at seltdwait+0x90/frame 0xf7df0a58 kern_select(c8876310,8,bfbfd070,0,0,...) at kern_select+0x863/frame 0xf7df0bf8 sys_select(c8876310,f7df0cc8,ef,c169014d,16,...) at sys_select+0x69/frame 0xf7df0c40 syscall(f7df0d08) at syscall+0x2ed/frame 0xf7df0cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7df0cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281b0987, esp = 0xbfbfd014, ebp = 0xbfbfde34 --- Tracing command watchdogd pid 789 tid 100116 td 0xc7bcc620 sched_switch(c7bcc620,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7de4a64 mi_switch(104,0,c1113475,1f5,c0afc930,...) at mi_switch+0x1c9/frame 0xf7de4a98 sleepq_switch(c7bcc620,0,c1113475,1ab,8,...) at sleepq_switch+0x1c5/frame 0xf7de4acc sleepq_catch_signals(c0afc930,c7bcc620,3,201,3,...) at sleepq_catch_signals+0x36c/frame 0xf7de4b14 sleepq_timedwait_sig(c128b66b,6c,d46,c0000000,3,...) at sleepq_timedwait_sig+0x14/frame 0xf7de4b3c _sleep(c128b66b,0,16c,c110ead2,3e898d39,...) at _sleep+0x33c/frame 0xf7de4b98 kern_nanosleep(bfbfdcb0,f7de4c18,f7de4c20,c7ec4760,3c,...) at kern_nanosleep+0x158/frame 0xf7de4c00 sys_nanosleep(c7bcc620,f7de4cc8,c0ee61d0,c71fdc90,0,...) at sys_nanosleep+0x6b/frame 0xf7de4c40 syscall(f7de4d08) at syscall+0x2ed/frame 0xf7de4cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7de4cfc --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x28178d03, esp = 0xbfbfdca4, ebp = 0xbfbfdce8 --- Tracing command cron pid 738 tid 100078 td 0xc7e19c40 sched_switch(c7e19c40,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7ce5a64 mi_switch(104,0,c1113475,1f5,c0afc930,...) at mi_switch+0x1c9/frame 0xf7ce5a98 sleepq_switch(c7e19c40,0,c1113475,1ab,8,...) at sleepq_switch+0x1c5/frame 0xf7ce5acc sleepq_catch_signals(c0afc930,c7e19c40,0,201,0,...) at sleepq_catch_signals+0x36c/frame 0xf7ce5b14 sleepq_timedwait_sig(c128b668,6c,d50,2547f1f8,2,...) at sleepq_timedwait_sig+0x14/frame 0xf7ce5b3c _sleep(c128b668,0,16c,c110ead2,7721beb3,...) at _sleep+0x33c/frame 0xf7ce5b98 kern_nanosleep(bfbfde28,f7ce5c18,f7ce5c20,f7ce5cc8,22,...) at kern_nanosleep+0x158/frame 0xf7ce5c00 sys_nanosleep(c7e19c40,f7ce5cc8,c0ee61d0,c1825c90,0,...) at sys_nanosleep+0x6b/frame 0xf7ce5c40 syscall(f7ce5d08) at syscall+0x2ed/frame 0xf7ce5cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7ce5cfc --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x28187d03, esp = 0xbfbfdcb4, ebp = 0xbfbfde44 --- Tracing command sendmail pid 733 tid 100080 td 0xc7e64310 sched_switch(c7e64310,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7cecaa0 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xf7cecad4 sleepq_switch(c7e64310,0,c1113475,1ab,f7cecb4c,...) at sleepq_switch+0x1c5/frame 0xf7cecb08 sleepq_catch_signals(0,c1113475,162,540,c7e53c04,...) at sleepq_catch_signals+0x36c/frame 0xf7cecb50 sleepq_wait_sig(c7e53c04,74,c0f390d4,100,0,...) at sleepq_wait_sig+0x14/frame 0xf7cecb74 _sleep(c7e53c04,c7e53c34,174,c0f390d4,0,...) at _sleep+0x364/frame 0xf7cecbd0 kern_sigsuspend(c7e64310,0,0,0,0,...) at kern_sigsuspend+0xfb/frame 0xf7cecc10 sys_sigsuspend(c7e64310,f7ceccc8,1,0,c7e64310,...) at sys_sigsuspend+0x58/frame 0xf7cecc40 syscall(f7cecd08) at syscall+0x2ed/frame 0xf7ceccfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7ceccfc --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x2835a61b, esp = 0xbfbfc9dc, ebp = 0xbfbfca08 --- Tracing command sendmail pid 730 tid 100083 td 0xc7e19310 sched_switch(c7e19310,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7cfc900 mi_switch(104,0,c1113475,1f5,400,...) at mi_switch+0x1c9/frame 0xf7cfc934 sleepq_switch(c7e19310,0,c1113475,1ab,c110af0a,...) at sleepq_switch+0x1c5/frame 0xf7cfc968 sleepq_catch_signals(c1214040,f7cfc9d8,c0aa24a9,c7d14e20,0,...) at sleepq_catch_signals+0x36c/frame 0xf7cfc9b0 sleepq_timedwait_sig(c7d14e24,0,f7cfca10,50000000,0,...) at sleepq_timedwait_sig+0x14/frame 0xf7cfc9d8 _cv_timedwait_sig_sbt(c7d14e24,c7d14e10,5d00314f,d42,50000000,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf7cfca24 seltdwait(d42,50000000,0,c7e19310,0,...) at seltdwait+0xbc/frame 0xf7cfca58 kern_select(c7e19310,5,bfbfbfc8,0,0,...) at kern_select+0x863/frame 0xf7cfcbf8 sys_select(c7e19310,f7cfccc8,7b3,2,f7cfccc4,...) at sys_select+0x69/frame 0xf7cfcc40 syscall(f7cfcd08) at syscall+0x2ed/frame 0xf7cfccfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7cfccfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28422987, esp = 0xbfbfbf2c, ebp = 0xbfbfc9e8 --- Tracing command sshd pid 727 tid 100108 td 0xc8454310 sched_switch(c8454310,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7dcc90c mi_switch(104,0,c1113475,1f5,200046,...) at mi_switch+0x1c9/frame 0xf7dcc940 sleepq_switch(c8454310,0,c1113475,1ab,c7d15260,...) at sleepq_switch+0x1c5/frame 0xf7dcc974 sleepq_catch_signals(f7dcc9e0,c0aa24a9,c7d15260,0,0,...) at sleepq_catch_signals+0x36c/frame 0xf7dcc9bc sleepq_wait_sig(c7d15264,0,f7dcca10,101,0,...) at sleepq_wait_sig+0x14/frame 0xf7dcc9e0 _cv_wait_sig(c7d15264,c7d15250,c1116849,6c6,c0b1747d,...) at _cv_wait_sig+0x200/frame 0xf7dcca24 seltdwait(ffffffff,0,0,c8454310,0,...) at seltdwait+0x90/frame 0xf7dcca58 kern_select(c8454310,9,28c0c148,0,0,...) at kern_select+0x863/frame 0xf7dccbf8 sys_select(c8454310,f7dcccc8,0,0,0,...) at sys_select+0x69/frame 0xf7dccc40 syscall(f7dccd08) at syscall+0x2ed/frame 0xf7dcccfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7dcccfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28532987, esp = 0xbfbfd424, ebp = 0xbfbfde44 --- Tracing command moused pid 695 tid 100107 td 0xc8454620 sched_switch(c8454620,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7db190c mi_switch(104,0,c1113475,1f5,46,...) at mi_switch+0x1c9/frame 0xf7db1940 sleepq_switch(c8454620,0,c1113475,1ab,c7d152a0,...) at sleepq_switch+0x1c5/frame 0xf7db1974 sleepq_catch_signals(f7db19e0,c0aa24a9,c7d152a0,0,0,...) at sleepq_catch_signals+0x36c/frame 0xf7db19bc sleepq_wait_sig(c7d152a4,0,f7db1a10,101,0,...) at sleepq_wait_sig+0x14/frame 0xf7db19e0 _cv_wait_sig(c7d152a4,c7d15290,c1116849,6c6,c8454620,...) at _cv_wait_sig+0x200/frame 0xf7db1a24 seltdwait(ffffffff,0,0,c8454620,0,...) at seltdwait+0x90/frame 0xf7db1a58 kern_select(c8454620,400,bfbfdcb0,0,0,...) at kern_select+0x863/frame 0xf7db1bf8 sys_select(c8454620,f7db1cc8,c0ee61d0,c71fac90,0,...) at sys_select+0x69/frame 0xf7db1c40 syscall(f7db1d08) at syscall+0x2ed/frame 0xf7db1cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7db1cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281c2987, esp = 0xbfbfd894, ebp = 0xbfbfde24 --- Tracing command smartd pid 687 tid 100106 td 0xc8454930 sched_switch(c8454930,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7dada64 mi_switch(104,0,c1113475,1f5,c0afc930,...) at mi_switch+0x1c9/frame 0xf7dada98 sleepq_switch(c8454930,0,c1113475,1ab,8,...) at sleepq_switch+0x1c5/frame 0xf7dadacc sleepq_catch_signals(c0afc930,c8454930,2,201,2,...) at sleepq_catch_signals+0x36c/frame 0xf7dadb14 sleepq_timedwait_sig(c128b66a,6c,e25,80000000,70,...) at sleepq_timedwait_sig+0x14/frame 0xf7dadb3c _sleep(c128b66a,0,16c,c110ead2,962092b8,...) at _sleep+0x33c/frame 0xf7dadb98 kern_nanosleep(bfbf53c8,f7dadc18,f7dadc20,f7dadcc8,708,...) at kern_nanosleep+0x158/frame 0xf7dadc00 sys_nanosleep(c8454930,f7dadcc8,ef,6a5ec7b9,13,...) at sys_nanosleep+0x6b/frame 0xf7dadc40 syscall(f7dadd08) at syscall+0x2ed/frame 0xf7dadcfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7dadcfc --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x282f0d03, esp = 0xbfbf53bc, ebp = 0xbfbf5400 --- Tracing command winbindd pid 685 tid 100104 td 0xc8455000 sched_switch(c8455000,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7da5978 mi_switch(104,0,c1113475,1f5,400,...) at mi_switch+0x1c9/frame 0xf7da59ac sleepq_switch(c8455000,0,c1113475,1ab,c110af0a,...) at sleepq_switch+0x1c5/frame 0xf7da59e0 sleepq_catch_signals(c1214040,f7da5a50,c0aa24a9,c8151f20,0,...) at sleepq_catch_signals+0x36c/frame 0xf7da5a28 sleepq_timedwait_sig(c8151f24,0,f7da5a88,b7fbe76c,20c49,...) at sleepq_timedwait_sig+0x14/frame 0xf7da5a50 _cv_timedwait_sig_sbt(c8151f24,c8151f10,b3051596,20d194,b7fbe76c,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf7da5a9c seltdwait(20d194,b7fbe76c,20c49,c8455000,c111a598,...) at seltdwait+0xbc/frame 0xf7da5ad0 sys_poll(c8455000,f7da5cc8,c0ee61d0,c7200c90,0,...) at sys_poll+0x44c/frame 0xf7da5c40 syscall(f7da5d08) at syscall+0x2ed/frame 0xf7da5cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7da5cfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x2193badf, esp = 0xbfbfc47c, ebp = 0xbfbfc4b8 --- Tracing command smbd pid 684 tid 100105 td 0xc8454c40 sched_switch(c8454c40,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7da8978 mi_switch(104,0,c1113475,1f5,400,...) at mi_switch+0x1c9/frame 0xf7da89ac sleepq_switch(c8454c40,0,c1113475,1ab,c110af0a,...) at sleepq_switch+0x1c5/frame 0xf7da89e0 sleepq_catch_signals(c1214040,f7da8a50,c0aa24a9,c7d14e60,0,...) at sleepq_catch_signals+0x36c/frame 0xf7da8a28 sleepq_timedwait_sig(c7d14e64,0,f7da8a88,b7fbe76c,20c49,...) at sleepq_timedwait_sig+0x14/frame 0xf7da8a50 _cv_timedwait_sig_sbt(c7d14e64,c7d14e50,48d5e080,20c4ae,b7fbe76c,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf7da8a9c seltdwait(20c4ae,b7fbe76c,20c49,c8454c40,0,...) at seltdwait+0xbc/frame 0xf7da8ad0 sys_poll(c8454c40,f7da8cc8,0,0,0,...) at sys_poll+0x44c/frame 0xf7da8c40 syscall(f7da8d08) at syscall+0x2ed/frame 0xf7da8cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7da8cfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x21b64adf, esp = 0xbfbfd9ec, ebp = 0xbfbfda28 --- Tracing command winbindd pid 673 tid 100085 td 0xc7e0bc40 sched_switch(c7e0bc40,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7d11978 mi_switch(104,0,c1113475,1f5,400,...) at mi_switch+0x1c9/frame 0xf7d119ac sleepq_switch(c7e0bc40,0,c1113475,1ab,c110af0a,...) at sleepq_switch+0x1c5/frame 0xf7d119e0 sleepq_catch_signals(c1214040,f7d11a50,c0aa24a9,c8151ea0,0,...) at sleepq_catch_signals+0x36c/frame 0xf7d11a28 sleepq_timedwait_sig(c8151ea4,0,f7d11a88,bfffea52,12,...) at sleepq_timedwait_sig+0x14/frame 0xf7d11a50 _cv_timedwait_sig_sbt(c8151ea4,c8151e90,334543ed,e25,bfffea52,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf7d11a9c seltdwait(e25,bfffea52,12,c7e0bc40,c0f01ffd,...) at seltdwait+0xbc/frame 0xf7d11ad0 sys_poll(c7e0bc40,f7d11cc8,c0ee61d0,c71fdc90,0,...) at sys_poll+0x44c/frame 0xf7d11c40 syscall(f7d11d08) at syscall+0x2ed/frame 0xf7d11cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7d11cfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x2193badf, esp = 0xbfbfdbcc, ebp = 0xbfbfdc08 --- Tracing command smbd pid 671 tid 100099 td 0xc82b4310 sched_switch(c82b4310,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7d89978 mi_switch(104,0,c1113475,1f5,400,...) at mi_switch+0x1c9/frame 0xf7d899ac sleepq_switch(c82b4310,0,c1113475,1ab,c110af0a,...) at sleepq_switch+0x1c5/frame 0xf7d899e0 sleepq_catch_signals(c1214040,f7d89a50,c0aa24a9,c8151ee0,0,...) at sleepq_catch_signals+0x36c/frame 0xf7d89a28 sleepq_timedwait_sig(c8151ee4,0,f7d89a88,bffffbaa,3,...) at sleepq_timedwait_sig+0x14/frame 0xf7d89a50 _cv_timedwait_sig_sbt(c8151ee4,c8151ed0,1c7cf50e,d76,bffffbaa,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf7d89a9c seltdwait(d76,bffffbaa,3,c82b4310,c0a7cb0c,...) at seltdwait+0xbc/frame 0xf7d89ad0 sys_poll(c82b4310,f7d89cc8,0,0,c82b4310,...) at sys_poll+0x44c/frame 0xf7d89c40 syscall(f7d89d08) at syscall+0x2ed/frame 0xf7d89cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7d89cfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x21b64adf, esp = 0xbfbfda9c, ebp = 0xbfbfdad8 --- Tracing command nmbd pid 669 tid 100101 td 0xc7e57c40 sched_switch(c7e57c40,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7d94978 mi_switch(104,0,c1113475,1f5,400,...) at mi_switch+0x1c9/frame 0xf7d949ac sleepq_switch(c7e57c40,0,c1113475,1ab,c110af0a,...) at sleepq_switch+0x1c5/frame 0xf7d949e0 sleepq_catch_signals(c1214040,f7d94a50,c0aa24a9,c8152120,0,...) at sleepq_catch_signals+0x36c/frame 0xf7d94a28 sleepq_timedwait_sig(c8152124,0,f7d94a88,9fffff47,0,...) at sleepq_timedwait_sig+0x14/frame 0xf7d94a50 _cv_timedwait_sig_sbt(c8152124,c8152110,bbf9d254,d47,9fffff47,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf7d94a9c seltdwait(d47,9fffff47,0,c7e57c40,c1b83db0,...) at seltdwait+0xbc/frame 0xf7d94ad0 sys_poll(c7e57c40,f7d94cc8,c0ee61d0,c1825c90,0,...) at sys_poll+0x44c/frame 0xf7d94c40 syscall(f7d94d08) at syscall+0x2ed/frame 0xf7d94cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7d94cfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x21564adf, esp = 0xbfbfdb1c, ebp = 0xbfbfdb58 --- Tracing command ntpd pid 645 tid 100103 td 0xc8455310 sched_switch(c8455310,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7d9a90c mi_switch(104,0,c1113475,1f5,200046,...) at mi_switch+0x1c9/frame 0xf7d9a940 sleepq_switch(c8455310,0,c1113475,1ab,c81520e0,...) at sleepq_switch+0x1c5/frame 0xf7d9a974 sleepq_catch_signals(f7d9a9e0,c0aa24a9,c81520e0,0,0,...) at sleepq_catch_signals+0x36c/frame 0xf7d9a9bc sleepq_wait_sig(c81520e4,0,f7d9aa10,101,0,...) at sleepq_wait_sig+0x14/frame 0xf7d9a9e0 _cv_wait_sig(c81520e4,c81520d0,c1116849,6c6,c0b1747d,...) at _cv_wait_sig+0x200/frame 0xf7d9aa24 seltdwait(ffffffff,0,0,c8455310,0,...) at seltdwait+0x90/frame 0xf7d9aa58 kern_select(c8455310,1c,bfbfdd48,0,0,...) at kern_select+0x863/frame 0xf7d9abf8 sys_select(c8455310,f7d9acc8,0,0,c8455310,...) at sys_select+0x69/frame 0xf7d9ac40 syscall(f7d9ad08) at syscall+0x2ed/frame 0xf7d9acfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7d9acfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x283a0987, esp = 0xbfbfdd24, ebp = 0xbfbfdde4 --- Tracing command nfsd pid 589 tid 100098 td 0xc82b4620 sched_switch(c82b4620,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7d85a8c mi_switch(104,0,c1113475,1f5,c0afc930,...) at mi_switch+0x1c9/frame 0xf7d85ac0 sleepq_switch(c82b4620,0,c1113475,1ab,c110af0a,...) at sleepq_switch+0x1c5/frame 0xf7d85af4 sleepq_catch_signals(c1214040,f7d85b64,c0aa24a9,c7912490,0,...) at sleepq_catch_signals+0x36c/frame 0xf7d85b3c sleepq_timedwait_sig(c78df9d0,0,f7d85b9c,0,0,...) at sleepq_timedwait_sig+0x14/frame 0xf7d85b64 _cv_timedwait_sig_sbt(c78df9d0,c7912480,fffffa38,4,0,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf7d85bb0 svc_run_internal(f7d85cf4,c0a857df,c7912480,f7d85d08,c1106d94,...) at svc_run_internal+0xa86/frame 0xf7d85cc4 svc_thread_start(c7912480,f7d85d08,c1106d94,3d7,0,...) at svc_thread_start+0xd/frame 0xf7d85ccc fork_exit(c0cc2f40,c7912480,f7d85d08) at fork_exit+0x7f/frame 0xf7d85cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf7d85cf4 --- trap 0xcf26092e, eip = 0xc, esp = 0x33, ebp = 0xbfbfd937 --- Tracing command nfsd pid 589 tid 100097 td 0xc82b4930 sched_switch(c82b4930,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7d82a8c mi_switch(104,0,c1113475,1f5,c0afc930,...) at mi_switch+0x1c9/frame 0xf7d82ac0 sleepq_switch(c82b4930,0,c1113475,1ab,c110af0a,...) at sleepq_switch+0x1c5/frame 0xf7d82af4 sleepq_catch_signals(c1214040,f7d82b64,c0aa24a9,c7912490,0,...) at sleepq_catch_signals+0x36c/frame 0xf7d82b3c sleepq_timedwait_sig(c78dfa10,0,f7d82b9c,0,0,...) at sleepq_timedwait_sig+0x14/frame 0xf7d82b64 _cv_timedwait_sig_sbt(c78dfa10,c7912480,fffffa38,4,0,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf7d82bb0 svc_run_internal(f7d82cf4,c0a857df,c7912480,f7d82d08,c1106d94,...) at svc_run_internal+0xa86/frame 0xf7d82cc4 svc_thread_start(c7912480,f7d82d08,c1106d94,3d7,0,...) at svc_thread_start+0xd/frame 0xf7d82ccc fork_exit(c0cc2f40,c7912480,f7d82d08) at fork_exit+0x7f/frame 0xf7d82cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf7d82cf4 --- trap 0xcf26092e, eip = 0xc, esp = 0x33, ebp = 0xbfbfd937 --- Tracing command nfsd pid 589 tid 100096 td 0xc82b4c40 sched_switch(c82b4c40,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7d7fa8c mi_switch(104,0,c1113475,1f5,c0afc930,...) at mi_switch+0x1c9/frame 0xf7d7fac0 sleepq_switch(c82b4c40,0,c1113475,1ab,c110af0a,...) at sleepq_switch+0x1c5/frame 0xf7d7faf4 sleepq_catch_signals(c1214040,f7d7fb64,c0aa24a9,c7912490,0,...) at sleepq_catch_signals+0x36c/frame 0xf7d7fb3c sleepq_timedwait_sig(c78dfa50,0,f7d7fb9c,0,0,...) at sleepq_timedwait_sig+0x14/frame 0xf7d7fb64 _cv_timedwait_sig_sbt(c78dfa50,c7912480,fffffa38,4,0,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf7d7fbb0 svc_run_internal(f7d7fcf4,c0a857df,c7912480,f7d7fd08,c1106d94,...) at svc_run_internal+0xa86/frame 0xf7d7fcc4 svc_thread_start(c7912480,f7d7fd08,c1106d94,3d7,0,...) at svc_thread_start+0xd/frame 0xf7d7fccc fork_exit(c0cc2f40,c7912480,f7d7fd08) at fork_exit+0x7f/frame 0xf7d7fcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf7d7fcf4 --- trap 0xcf26092e, eip = 0xc, esp = 0x33, ebp = 0xbfbfd937 --- Tracing command nfsd pid 589 tid 100086 td 0xc7e0b930 sched_switch(c7e0b930,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7d16304 mi_switch(104,0,c1113475,1f5,c0afc930,...) at mi_switch+0x1c9/frame 0xf7d16338 sleepq_switch(c7e0b930,0,c1113475,1ab,c110af0a,...) at sleepq_switch+0x1c5/frame 0xf7d1636c sleepq_catch_signals(c1214040,f7d163dc,c0aa24a9,c7912490,0,...) at sleepq_catch_signals+0x36c/frame 0xf7d163b4 sleepq_timedwait_sig(c78dead0,0,f7d16414,0,0,...) at sleepq_timedwait_sig+0x14/frame 0xf7d163dc _cv_timedwait_sig_sbt(c78dead0,c7912480,fffffa38,4,0,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf7d16428 svc_run_internal(c0cc2f40,c7912480,c7e092e8,f7d16568,0,...) at svc_run_internal+0xa86/frame 0xf7d1653c svc_run(c7912480,0,c10f8685,1b5,c13d6f48,...) at svc_run+0xc3/frame 0xf7d16578 nfsrvd_nfsd(c7e0b930,f7d166d0,c,c818be64,1,...) at nfsrvd_nfsd+0x1a2/frame 0xf7d166b0 nfssvc_nfsd(c7e0b930,f7d16cc8,0,c7e0b930,f7d16cfc,...) at nfssvc_nfsd+0xf4/frame 0xf7d16c28 sys_nfssvc(c7e0b930,f7d16cc8,c116612a,d6,0,...) at sys_nfssvc+0xbf/frame 0xf7d16c40 syscall(f7d16d08) at syscall+0x2ed/frame 0xf7d16cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7d16cfc --- syscall (155, FreeBSD ELF32, sys_nfssvc), eip = 0x280c2c4b, esp = 0xbfbfd7e4, ebp = 0xbfbfda58 --- Tracing command nfsd pid 584 tid 100089 td 0xc7e0b000 sched_switch(c7e0b000,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7d5290c mi_switch(104,0,c1113475,1f5,46,...) at mi_switch+0x1c9/frame 0xf7d52940 sleepq_switch(c7e0b000,0,c1113475,1ab,c7d15f20,...) at sleepq_switch+0x1c5/frame 0xf7d52974 sleepq_catch_signals(f7d529e0,c0aa24a9,c7d15f20,0,0,...) at sleepq_catch_signals+0x36c/frame 0xf7d529bc sleepq_wait_sig(c7d15f24,0,f7d52a10,101,0,...) at sleepq_wait_sig+0x14/frame 0xf7d529e0 _cv_wait_sig(c7d15f24,c7d15f10,c1116849,6c6,c0b1747d,...) at _cv_wait_sig+0x200/frame 0xf7d52a24 seltdwait(ffffffff,0,0,c7e0b000,0,...) at seltdwait+0x90/frame 0xf7d52a58 kern_select(c7e0b000,7,bfbfdc30,0,0,...) at kern_select+0x863/frame 0xf7d52bf8 sys_select(c7e0b000,f7d52cc8,c0ee61d0,c71fac90,0,...) at sys_select+0x69/frame 0xf7d52c40 syscall(f7d52d08) at syscall+0x2ed/frame 0xf7d52cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7d52cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x2818a987, esp = 0xbfbfda5c, ebp = 0xbfbfde34 --- Tracing command mountd pid 578 tid 100095 td 0xc8455930 sched_switch(c8455930,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7d7890c mi_switch(104,0,c1113475,1f5,46,...) at mi_switch+0x1c9/frame 0xf7d78940 sleepq_switch(c8455930,0,c1113475,1ab,c7d14ee0,...) at sleepq_switch+0x1c5/frame 0xf7d78974 sleepq_catch_signals(f7d789e0,c0aa24a9,c7d14ee0,0,0,...) at sleepq_catch_signals+0x36c/frame 0xf7d789bc sleepq_wait_sig(c7d14ee4,0,f7d78a10,101,0,...) at sleepq_wait_sig+0x14/frame 0xf7d789e0 _cv_wait_sig(c7d14ee4,c7d14ed0,c1116849,6c6,c0b1747d,...) at _cv_wait_sig+0x200/frame 0xf7d78a24 seltdwait(ffffffff,0,0,c8455930,0,...) at seltdwait+0x90/frame 0xf7d78a58 kern_select(c8455930,9,bfbfdd50,0,0,...) at kern_select+0x863/frame 0xf7d78bf8 sys_select(c8455930,f7d78cc8,c0ee61d0,c71fdc90,0,...) at sys_select+0x69/frame 0xf7d78c40 syscall(f7d78d08) at syscall+0x2ed/frame 0xf7d78cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7d78cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281a1987, esp = 0xbfbfdd0c, ebp = 0xbfbfde34 --- Tracing command nfsuserd pid 571 tid 100094 td 0xc8455c40 sched_switch(c8455c40,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7d73900 mi_switch(104,0,c1113475,1f5,400,...) at mi_switch+0x1c9/frame 0xf7d73934 sleepq_switch(c8455c40,0,c1113475,1ab,c110af0a,...) at sleepq_switch+0x1c5/frame 0xf7d73968 sleepq_catch_signals(c1214040,f7d739d8,c0aa24a9,c78e6660,0,...) at sleepq_catch_signals+0x36c/frame 0xf7d739b0 sleepq_timedwait_sig(c78e6664,0,f7d73a10,e0000000,1,...) at sleepq_timedwait_sig+0x14/frame 0xf7d739d8 _cv_timedwait_sig_sbt(c78e6664,c78e6650,ba49497d,d55,e0000000,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf7d73a24 seltdwait(d55,e0000000,1,c8455c40,0,...) at seltdwait+0xbc/frame 0xf7d73a58 kern_select(c8455c40,4,bfbfdc10,0,0,...) at kern_select+0x863/frame 0xf7d73bf8 sys_select(c8455c40,f7d73cc8,0,0,0,...) at sys_select+0x69/frame 0xf7d73c40 syscall(f7d73d08) at syscall+0x2ed/frame 0xf7d73cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7d73cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28189987, esp = 0xbfbfdb6c, ebp = 0xbfbfdca8 --- Tracing command nfsuserd pid 570 tid 100093 td 0xc7bce930 sched_switch(c7bce930,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7d70900 mi_switch(104,0,c1113475,1f5,400,...) at mi_switch+0x1c9/frame 0xf7d70934 sleepq_switch(c7bce930,0,c1113475,1ab,c110af0a,...) at sleepq_switch+0x1c5/frame 0xf7d70968 sleepq_catch_signals(c1214040,f7d709d8,c0aa24a9,c7d8e060,0,...) at sleepq_catch_signals+0x36c/frame 0xf7d709b0 sleepq_timedwait_sig(c7d8e064,0,f7d70a10,e0000000,1,...) at sleepq_timedwait_sig+0x14/frame 0xf7d709d8 _cv_timedwait_sig_sbt(c7d8e064,c7d8e050,72c28ba4,d56,e0000000,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf7d70a24 seltdwait(d56,e0000000,1,c7bce930,0,...) at seltdwait+0xbc/frame 0xf7d70a58 kern_select(c7bce930,4,bfbfdc10,0,0,...) at kern_select+0x863/frame 0xf7d70bf8 sys_select(c7bce930,f7d70cc8,7b3,2,f7d70cc4,...) at sys_select+0x69/frame 0xf7d70c40 syscall(f7d70d08) at syscall+0x2ed/frame 0xf7d70cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7d70cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28189987, esp = 0xbfbfdb6c, ebp = 0xbfbfdca8 --- Tracing command nfsuserd pid 569 tid 100090 td 0xc7bcec40 sched_switch(c7bcec40,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7d5e900 mi_switch(104,0,c1113475,1f5,400,...) at mi_switch+0x1c9/frame 0xf7d5e934 sleepq_switch(c7bcec40,0,c1113475,1ab,c110af0a,...) at sleepq_switch+0x1c5/frame 0xf7d5e968 sleepq_catch_signals(c1214040,f7d5e9d8,c0aa24a9,c7d8e160,0,...) at sleepq_catch_signals+0x36c/frame 0xf7d5e9b0 sleepq_timedwait_sig(c7d8e164,0,f7d5ea10,e0000000,1,...) at sleepq_timedwait_sig+0x14/frame 0xf7d5e9d8 _cv_timedwait_sig_sbt(c7d8e164,c7d8e150,f3c8adca,d55,e0000000,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf7d5ea24 seltdwait(d55,e0000000,1,c7bcec40,0,...) at seltdwait+0xbc/frame 0xf7d5ea58 kern_select(c7bcec40,4,bfbfdc10,0,0,...) at kern_select+0x863/frame 0xf7d5ebf8 sys_select(c7bcec40,f7d5ecc8,c0ee61d0,c71fac90,0,...) at sys_select+0x69/frame 0xf7d5ec40 syscall(f7d5ed08) at syscall+0x2ed/frame 0xf7d5ecfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7d5ecfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28189987, esp = 0xbfbfdb6c, ebp = 0xbfbfdca8 --- Tracing command nfsuserd pid 568 tid 100076 td 0xc7e1a000 sched_switch(c7e1a000,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7cd9900 mi_switch(104,0,c1113475,1f5,400,...) at mi_switch+0x1c9/frame 0xf7cd9934 sleepq_switch(c7e1a000,0,c1113475,1ab,c110af0a,...) at sleepq_switch+0x1c5/frame 0xf7cd9968 sleepq_catch_signals(c1214040,f7cd99d8,c0aa24a9,c8152160,0,...) at sleepq_catch_signals+0x36c/frame 0xf7cd99b0 sleepq_timedwait_sig(c8152164,0,f7cd9a10,e0000000,1,...) at sleepq_timedwait_sig+0x14/frame 0xf7cd99d8 _cv_timedwait_sig_sbt(c8152164,c8152150,8ae3e5c1,d57,e0000000,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf7cd9a24 seltdwait(d57,e0000000,1,c7e1a000,0,...) at seltdwait+0xbc/frame 0xf7cd9a58 kern_select(c7e1a000,4,bfbfdc10,0,0,...) at kern_select+0x863/frame 0xf7cd9bf8 sys_select(c7e1a000,f7cd9cc8,7b3,2,f7cd9cc4,...) at sys_select+0x69/frame 0xf7cd9c40 syscall(f7cd9d08) at syscall+0x2ed/frame 0xf7cd9cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7cd9cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28189987, esp = 0xbfbfdb6c, ebp = 0xbfbfdca8 --- Tracing command nfsuserd pid 567 tid 100079 td 0xc7e64620 sched_switch(c7e64620,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7ce9aa0 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xf7ce9ad4 sleepq_switch(c7e64620,0,c1113475,1ab,f7ce9b4c,...) at sleepq_switch+0x1c5/frame 0xf7ce9b08 sleepq_catch_signals(0,c1113475,162,360,c7e54064,...) at sleepq_catch_signals+0x36c/frame 0xf7ce9b50 sleepq_wait_sig(c7e54064,74,c0f390d4,100,0,...) at sleepq_wait_sig+0x14/frame 0xf7ce9b74 _sleep(c7e54064,c7e54094,174,c0f390d4,0,...) at _sleep+0x364/frame 0xf7ce9bd0 kern_sigsuspend(c7e64620,0,0,0,0,...) at kern_sigsuspend+0xfb/frame 0xf7ce9c10 sys_sigsuspend(c7e64620,f7ce9cc8,0,0,0,...) at sys_sigsuspend+0x58/frame 0xf7ce9c40 syscall(f7ce9d08) at syscall+0x2ed/frame 0xf7ce9cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7ce9cfc --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x280c161b, esp = 0xbfbfdcac, ebp = 0xbfbfde44 --- Tracing command rpcbind pid 538 tid 100091 td 0xc82b5310 sched_switch(c82b5310,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7d65978 mi_switch(104,0,c1113475,1f5,400,...) at mi_switch+0x1c9/frame 0xf7d659ac sleepq_switch(c82b5310,0,c1113475,1ab,c110af0a,...) at sleepq_switch+0x1c5/frame 0xf7d659e0 sleepq_catch_signals(c1214040,f7d65a50,c0aa24a9,c78e6be0,0,...) at sleepq_catch_signals+0x36c/frame 0xf7d65a28 sleepq_timedwait_sig(c78e6be4,0,f7d65a88,dffffdd5,1,...) at sleepq_timedwait_sig+0x14/frame 0xf7d65a50 _cv_timedwait_sig_sbt(c78e6be4,c78e6bd0,cabeb0bd,d56,dffffdd5,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf7d65a9c seltdwait(d56,dffffdd5,1,c82b5310,f7d65b18,...) at seltdwait+0xbc/frame 0xf7d65ad0 sys_poll(c82b5310,f7d65cc8,7b3,2,f7d65cc4,...) at sys_poll+0x44c/frame 0xf7d65c40 syscall(f7d65d08) at syscall+0x2ed/frame 0xf7d65cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7d65cfc --- syscall (209, FreeBSD ELF32, sys_poll), eip = 0x2813eadf, esp = 0xbfbfbbfc, ebp = 0xbfbfddd8 --- Tracing command syslogd pid 523 tid 100092 td 0xc82b5000 sched_switch(c82b5000,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7d6890c mi_switch(104,0,c1113475,1f5,46,...) at mi_switch+0x1c9/frame 0xf7d68940 sleepq_switch(c82b5000,0,c1113475,1ab,c7d15060,...) at sleepq_switch+0x1c5/frame 0xf7d68974 sleepq_catch_signals(f7d689e0,c0aa24a9,c7d15060,0,0,...) at sleepq_catch_signals+0x36c/frame 0xf7d689bc sleepq_wait_sig(c7d15064,0,f7d68a10,101,0,...) at sleepq_wait_sig+0x14/frame 0xf7d689e0 _cv_wait_sig(c7d15064,c7d15050,c1116849,6c6,c82b5000,...) at _cv_wait_sig+0x200/frame 0xf7d68a24 seltdwait(ffffffff,0,0,c82b5000,0,...) at seltdwait+0x90/frame 0xf7d68a58 kern_select(c82b5000,9,28844108,0,0,...) at kern_select+0x863/frame 0xf7d68bf8 sys_select(c82b5000,f7d68cc8,0,0,c82b5000,...) at sys_select+0x69/frame 0xf7d68c40 syscall(f7d68d08) at syscall+0x2ed/frame 0xf7d68cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7d68cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281a0987, esp = 0xbfbfceac, ebp = 0xbfbfde34 --- Tracing command devd pid 420 tid 100088 td 0xc7e0b310 sched_switch(c7e0b310,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf7d31900 mi_switch(104,0,c1113475,1f5,400,...) at mi_switch+0x1c9/frame 0xf7d31934 sleepq_switch(c7e0b310,0,c1113475,1ab,c110af0a,...) at sleepq_switch+0x1c5/frame 0xf7d31968 sleepq_catch_signals(c1214040,f7d319d8,c0aa24a9,c7d150a0,0,...) at sleepq_catch_signals+0x36c/frame 0xf7d319b0 sleepq_timedwait_sig(c7d150a4,0,f7d31a10,c0000000,3,...) at sleepq_timedwait_sig+0x14/frame 0xf7d319d8 _cv_timedwait_sig_sbt(c7d150a4,c7d15090,a7c082e1,d70,c0000000,...) at _cv_timedwait_sig_sbt+0x228/frame 0xf7d31a24 seltdwait(d70,c0000000,3,c7e0b310,0,...) at seltdwait+0xbc/frame 0xf7d31a58 kern_select(c7e0b310,5,bfbfbda0,0,0,...) at kern_select+0x863/frame 0xf7d31bf8 sys_select(c7e0b310,f7d31cc8,c0ee61d0,c71fac90,0,...) at sys_select+0x69/frame 0xf7d31c40 syscall(f7d31d08) at syscall+0x2ed/frame 0xf7d31cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf7d31cfc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x8095e63, esp = 0xbfbfbd44, ebp = 0xbfbfde54 --- Tracing command softdepflush pid 17 tid 100074 td 0xc7bccc40 sched_switch(c7bccc40,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf0413ba4 mi_switch(104,0,c1113475,1f5,101,...) at mi_switch+0x1c9/frame 0xf0413bd8 sleepq_switch(c7bccc40,0,c1113475,28e,c140742c,...) at sleepq_switch+0x1c5/frame 0xf0413c0c sleepq_timedwait(c140742c,54,0,0,0,...) at sleepq_timedwait+0x48/frame 0xf0413c30 _sleep(c140742c,c14073ac,54,c114628e,fffffed8,...) at _sleep+0x350/frame 0xf0413c8c softdep_flush(0,f0413d08,c1106d94,3d7,0,...) at softdep_flush+0x264/frame 0xf0413ccc fork_exit(c0d1b310,0,f0413d08) at fork_exit+0x7f/frame 0xf0413cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf0413cf4 --- trap 0, eip = 0, esp = 0xf0413d40, ebp = 0 --- Tracing command syncer pid 16 tid 100073 td 0xc7bcd000 sched_switch(c7bcd000,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf0410b78 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xf0410bac sleepq_switch(c7bcd000,0,c1113475,28e,0,...) at sleepq_switch+0x1c5/frame 0xf0410be0 sleepq_timedwait(c14031e0,0,f0410c3c,0,0,...) at sleepq_timedwait+0x48/frame 0xf0410c04 _cv_timedwait_sbt(c14031e0,c14031c0,fffffed8,0,0,...) at _cv_timedwait_sbt+0x228/frame 0xf0410c50 sched_sync(0,f0410d08,c1106d94,3d7,0,...) at sched_sync+0x682/frame 0xf0410ccc fork_exit(c0b66b80,0,f0410d08) at fork_exit+0x7f/frame 0xf0410cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf0410cf4 --- trap 0, eip = 0, esp = 0xf0410d40, ebp = 0 --- Tracing command vnlru pid 9 tid 100072 td 0xc7bcd310 sched_switch(c7bcd310,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf040db78 mi_switch(104,0,c1113475,1f5,101,...) at mi_switch+0x1c9/frame 0xf040dbac sleepq_switch(c7bcd310,0,c1113475,28e,c7c94ba0,...) at sleepq_switch+0x1c5/frame 0xf040dbe0 sleepq_timedwait(c7c94ba0,60,0,0,0,...) at sleepq_timedwait+0x48/frame 0xf040dc04 _sleep(c7c94ba0,c1403198,260,c111fbc3,fffffed8,...) at _sleep+0x350/frame 0xf040dc60 vnlru_proc(0,f040dd08,c1106d94,3d7,0,...) at vnlru_proc+0xcf/frame 0xf040dccc fork_exit(c0b67420,0,f040dd08) at fork_exit+0x7f/frame 0xf040dcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf040dcf4 --- trap 0, eip = 0, esp = 0xf040dd40, ebp = 0 --- Tracing command bufdaemon pid 8 tid 100071 td 0xc7bcd620 sched_switch(c7bcd620,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf040abb0 mi_switch(104,0,c1113475,1f5,101,...) at mi_switch+0x1c9/frame 0xf040abe4 sleepq_switch(c7bcd620,0,c1113475,28e,c1402d04,...) at sleepq_switch+0x1c5/frame 0xf040ac18 sleepq_timedwait(c1402d04,54,0,0,0,...) at sleepq_timedwait+0x48/frame 0xf040ac3c _sleep(c1402d04,c1402c80,54,c111c5c2,fffffed8,...) at _sleep+0x350/frame 0xf040ac98 buf_daemon(0,f040ad08,c1106d94,3d7,0,...) at buf_daemon+0x8c/frame 0xf040accc fork_exit(c0b4a960,0,f040ad08) at fork_exit+0x7f/frame 0xf040acf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf040acf4 --- trap 0, eip = 0, esp = 0xf040ad40, ebp = 0 --- Tracing command pagezero pid 7 tid 100070 td 0xc7bcd930 sched_switch(c7bcd930,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf0407bac mi_switch(104,0,c1113475,1f5,101,...) at mi_switch+0x1c9/frame 0xf0407be0 sleepq_switch(c7bcd930,0,c1113475,28e,c1408188,...) at sleepq_switch+0x1c5/frame 0xf0407c14 sleepq_timedwait(c1408188,0,12b,0,0,...) at sleepq_timedwait+0x48/frame 0xf0407c38 _sleep(c1408188,c145bb80,0,c114d81d,fffea520,...) at _sleep+0x350/frame 0xf0407c94 vm_pagezero(0,f0407d08,c1106d94,3d7,0,...) at vm_pagezero+0xc2/frame 0xf0407ccc fork_exit(c0d63250,0,f0407d08) at fork_exit+0x7f/frame 0xf0407cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf0407cf4 --- trap 0, eip = 0, esp = 0xf0407d40, ebp = 0 --- Tracing command vmdaemon pid 6 tid 100069 td 0xc7bcdc40 sched_switch(c7bcdc40,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf0404b78 mi_switch(104,0,c1113475,1f5,162,...) at mi_switch+0x1c9/frame 0xf0404bac sleepq_switch(c7bcdc40,0,c1113475,26b,c1407d58,...) at sleepq_switch+0x1c5/frame 0xf0404be0 sleepq_wait(c1407d58,74,c111c5c2,0,0,...) at sleepq_wait+0x48/frame 0xf0404c04 _sleep(c1407d58,c1407d40,74,c111c5c2,0,...) at _sleep+0x385/frame 0xf0404c60 vm_daemon(0,f0404d08,c1106d94,3d7,0,...) at vm_daemon+0xac/frame 0xf0404ccc fork_exit(c0d5b4b0,0,f0404d08) at fork_exit+0x7f/frame 0xf0404cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf0404cf4 --- trap 0, eip = 0, esp = 0xf0404d40, ebp = 0 --- Tracing command pagedaemon pid 5 tid 100068 td 0xc7bce000 sched_switch(c7bce000,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf0401b30 mi_switch(104,0,c1113475,1f5,101,...) at mi_switch+0x1c9/frame 0xf0401b64 sleepq_switch(c7bce000,0,c1113475,28e,c145bc04,...) at sleepq_switch+0x1c5/frame 0xf0401b98 sleepq_timedwait(c145bc04,54,4,0,0,...) at sleepq_timedwait+0x48/frame 0xf0401bbc _sleep(c145bc04,c145bb80,54,c111c5c2,fffffa38,...) at _sleep+0x350/frame 0xf0401c18 vm_pageout(0,f0401d08,c1106d94,3d7,0,...) at vm_pageout+0x290/frame 0xf0401ccc fork_exit(c0d5bce0,0,f0401d08) at fork_exit+0x7f/frame 0xf0401cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf0401cf4 --- trap 0, eip = 0, esp = 0xf0401d40, ebp = 0 --- Tracing command xpt_thrd pid 4 tid 100067 td 0xc7bce310 sched_switch(c7bce310,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf027ebb0 mi_switch(104,0,c1113475,1f5,162,...) at mi_switch+0x1c9/frame 0xf027ebe4 sleepq_switch(c7bce310,0,c1113475,26b,c12595a4,...) at sleepq_switch+0x1c5/frame 0xf027ec18 sleepq_wait(c12595a4,5c,c0f0de07,0,0,...) at sleepq_wait+0x48/frame 0xf027ec3c _sleep(c12595a4,c12595f4,5c,c0f0de07,0,...) at _sleep+0x385/frame 0xf027ec98 xpt_scanner_thread(0,f027ed08,c1106d94,3d7,0,...) at xpt_scanner_thread+0x11c/frame 0xf027eccc fork_exit(c04b9080,0,f027ed08) at fork_exit+0x7f/frame 0xf027ecf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf027ecf4 --- trap 0, eip = 0, esp = 0xf027ed40, ebp = 0 --- Tracing command sctp_iterator pid 3 tid 100066 td 0xc7bce620 sched_switch(c7bce620,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf027bbb8 mi_switch(104,0,c1113475,1f5,162,...) at mi_switch+0x1c9/frame 0xf027bbec sleepq_switch(c7bce620,0,c1113475,26b,c145790c,...) at sleepq_switch+0x1c5/frame 0xf027bc20 sleepq_wait(c145790c,0,c1133225,0,0,...) at sleepq_wait+0x48/frame 0xf027bc44 _sleep(c145790c,c14578d4,0,c1133225,0,...) at _sleep+0x385/frame 0xf027bca0 sctp_iterator_thread(0,f027bd08,c1106d94,3d7,0,...) at sctp_iterator_thread+0x7c/frame 0xf027bccc fork_exit(c0c05d30,0,f027bd08) at fork_exit+0x7f/frame 0xf027bcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf027bcf4 --- trap 0, eip = 0, esp = 0xf027bd40, ebp = 0 --- Tracing command fdc0 pid 2 tid 100061 td 0xc7916310 sched_switch(c7916310,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf026bb48 mi_switch(104,0,c1113475,1f5,101,...) at mi_switch+0x1c9/frame 0xf026bb7c sleepq_switch(c7916310,0,c1113475,28e,c78bba3c,...) at sleepq_switch+0x1c5/frame 0xf026bbb0 sleepq_timedwait(c78bba3c,5c,0,0,0,...) at sleepq_timedwait+0x48/frame 0xf026bbd4 _sleep(c78bba3c,c78bbaf0,5c,c10f9046,fffffed8,...) at _sleep+0x350/frame 0xf026bc30 fdc_thread(c78bba00,f026bd08,c1106d94,3d7,0,...) at fdc_thread+0x877/frame 0xf026bccc fork_exit(c0e6d960,c78bba00,f026bd08) at fork_exit+0x7f/frame 0xf026bcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf026bcf4 --- trap 0, eip = 0, esp = 0xf026bd40, ebp = 0 --- Tracing command usb pid 15 tid 100058 td 0xc7916c40 sched_switch(c7916c40,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf0247bd4 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xf0247c08 sleepq_switch(c7916c40,0,c1113475,26b,c7916c40,...) at sleepq_switch+0x1c5/frame 0xf0247c3c sleepq_wait(c791bd34,0,f0247c90,1,0,...) at sleepq_wait+0x48/frame 0xf0247c60 _cv_wait(c791bd34,c791be4c,c10ea7c7,77,c791bd3c,...) at _cv_wait+0x1f9/frame 0xf0247ca4 usb_process(c791bd2c,f0247d08,c1106d94,3d7,0,...) at usb_process+0xe6/frame 0xf0247ccc fork_exit(c0968280,c791bd2c,f0247d08) at fork_exit+0x7f/frame 0xf0247cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf0247cf4 --- trap 0, eip = 0, esp = 0xf0247d40, ebp = 0 --- Tracing command usb pid 15 tid 100057 td 0xc7917000 sched_switch(c7917000,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf0244bd4 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xf0244c08 sleepq_switch(c7917000,0,c1113475,26b,c7917000,...) at sleepq_switch+0x1c5/frame 0xf0244c3c sleepq_wait(c791bd04,0,f0244c90,1,0,...) at sleepq_wait+0x48/frame 0xf0244c60 _cv_wait(c791bd04,c791be4c,c10ea7c7,77,c791bd0c,...) at _cv_wait+0x1f9/frame 0xf0244ca4 usb_process(c791bcfc,f0244d08,c1106d94,3d7,0,...) at usb_process+0xe6/frame 0xf0244ccc fork_exit(c0968280,c791bcfc,f0244d08) at fork_exit+0x7f/frame 0xf0244cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf0244cf4 --- trap 0, eip = 0, esp = 0xf0244d40, ebp = 0 --- Tracing command usb pid 15 tid 100056 td 0xc7917310 sched_switch(c7917310,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf0241bd4 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xf0241c08 sleepq_switch(c7917310,0,c1113475,26b,c7917310,...) at sleepq_switch+0x1c5/frame 0xf0241c3c sleepq_wait(c791bcd4,0,f0241c90,1,0,...) at sleepq_wait+0x48/frame 0xf0241c60 _cv_wait(c791bcd4,c791be4c,c10ea7c7,77,c791bcdc,...) at _cv_wait+0x1f9/frame 0xf0241ca4 usb_process(c791bccc,f0241d08,c1106d94,3d7,0,...) at usb_process+0xe6/frame 0xf0241ccc fork_exit(c0968280,c791bccc,f0241d08) at fork_exit+0x7f/frame 0xf0241cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf0241cf4 --- trap 0, eip = 0, esp = 0xf0241d40, ebp = 0 --- Tracing command usb pid 15 tid 100055 td 0xc7917620 sched_switch(c7917620,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf023ebd4 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xf023ec08 sleepq_switch(c7917620,0,c1113475,26b,c7917620,...) at sleepq_switch+0x1c5/frame 0xf023ec3c sleepq_wait(c791bca4,0,f023ec90,1,0,...) at sleepq_wait+0x48/frame 0xf023ec60 _cv_wait(c791bca4,c791be4c,c10ea7c7,77,c791bcac,...) at _cv_wait+0x1f9/frame 0xf023eca4 usb_process(c791bc9c,f023ed08,c1106d94,3d7,0,...) at usb_process+0xe6/frame 0xf023eccc fork_exit(c0968280,c791bc9c,f023ed08) at fork_exit+0x7f/frame 0xf023ecf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf023ecf4 --- trap 0, eip = 0, esp = 0xf023ed40, ebp = 0 --- Tracing command usb pid 15 tid 100053 td 0xc7917c40 sched_switch(c7917c40,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf0235bd4 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xf0235c08 sleepq_switch(c7917c40,0,c1113475,26b,c7917c40,...) at sleepq_switch+0x1c5/frame 0xf0235c3c sleepq_wait(c7913b5c,0,f0235c90,1,0,...) at sleepq_wait+0x48/frame 0xf0235c60 _cv_wait(c7913b5c,c7913c74,c10ea7c7,77,c7913b64,...) at _cv_wait+0x1f9/frame 0xf0235ca4 usb_process(c7913b54,f0235d08,c1106d94,3d7,0,...) at usb_process+0xe6/frame 0xf0235ccc fork_exit(c0968280,c7913b54,f0235d08) at fork_exit+0x7f/frame 0xf0235cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf0235cf4 --- trap 0, eip = 0, esp = 0xf0235d40, ebp = 0 --- Tracing command usb pid 15 tid 100052 td 0xc7893c40 sched_switch(c7893c40,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf0232bd4 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xf0232c08 sleepq_switch(c7893c40,0,c1113475,26b,c7893c40,...) at sleepq_switch+0x1c5/frame 0xf0232c3c sleepq_wait(c7913b2c,0,f0232c90,1,0,...) at sleepq_wait+0x48/frame 0xf0232c60 _cv_wait(c7913b2c,c7913c74,c10ea7c7,77,c7913b34,...) at _cv_wait+0x1f9/frame 0xf0232ca4 usb_process(c7913b24,f0232d08,c1106d94,3d7,0,...) at usb_process+0xe6/frame 0xf0232ccc fork_exit(c0968280,c7913b24,f0232d08) at fork_exit+0x7f/frame 0xf0232cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf0232cf4 --- trap 0, eip = 0, esp = 0xf0232d40, ebp = 0 --- Tracing command usb pid 15 tid 100051 td 0xc7900000 sched_switch(c7900000,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf022fbd4 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xf022fc08 sleepq_switch(c7900000,0,c1113475,26b,c7900000,...) at sleepq_switch+0x1c5/frame 0xf022fc3c sleepq_wait(c7913afc,0,f022fc90,1,0,...) at sleepq_wait+0x48/frame 0xf022fc60 _cv_wait(c7913afc,c7913c74,c10ea7c7,77,c7913b04,...) at _cv_wait+0x1f9/frame 0xf022fca4 usb_process(c7913af4,f022fd08,c1106d94,3d7,0,...) at usb_process+0xe6/frame 0xf022fccc fork_exit(c0968280,c7913af4,f022fd08) at fork_exit+0x7f/frame 0xf022fcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf022fcf4 --- trap 0, eip = 0, esp = 0xf022fd40, ebp = 0 --- Tracing command usb pid 15 tid 100050 td 0xc7900310 sched_switch(c7900310,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf022cbd4 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xf022cc08 sleepq_switch(c7900310,0,c1113475,26b,c7900310,...) at sleepq_switch+0x1c5/frame 0xf022cc3c sleepq_wait(c7913acc,0,f022cc90,1,0,...) at sleepq_wait+0x48/frame 0xf022cc60 _cv_wait(c7913acc,c7913c74,c10ea7c7,77,c7913ad4,...) at _cv_wait+0x1f9/frame 0xf022cca4 usb_process(c7913ac4,f022cd08,c1106d94,3d7,0,...) at usb_process+0xe6/frame 0xf022cccc fork_exit(c0968280,c7913ac4,f022cd08) at fork_exit+0x7f/frame 0xf022ccf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf022ccf4 --- trap 0, eip = 0, esp = 0xf022cd40, ebp = 0 --- Tracing command usb pid 15 tid 100049 td 0xc7900620 sched_switch(c7900620,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf0228bd4 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xf0228c08 sleepq_switch(c7900620,0,c1113475,26b,c7900620,...) at sleepq_switch+0x1c5/frame 0xf0228c3c sleepq_wait(c790db5c,0,f0228c90,1,0,...) at sleepq_wait+0x48/frame 0xf0228c60 _cv_wait(c790db5c,c790dc74,c10ea7c7,77,c790db64,...) at _cv_wait+0x1f9/frame 0xf0228ca4 usb_process(c790db54,f0228d08,c1106d94,3d7,0,...) at usb_process+0xe6/frame 0xf0228ccc fork_exit(c0968280,c790db54,f0228d08) at fork_exit+0x7f/frame 0xf0228cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf0228cf4 --- trap 0, eip = 0, esp = 0xf0228d40, ebp = 0 --- Tracing command usb pid 15 tid 100048 td 0xc7900930 sched_switch(c7900930,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf0225bd4 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xf0225c08 sleepq_switch(c7900930,0,c1113475,26b,c7900930,...) at sleepq_switch+0x1c5/frame 0xf0225c3c sleepq_wait(c790db2c,0,f0225c90,1,0,...) at sleepq_wait+0x48/frame 0xf0225c60 _cv_wait(c790db2c,c790dc74,c10ea7c7,77,c790db34,...) at _cv_wait+0x1f9/frame 0xf0225ca4 usb_process(c790db24,f0225d08,c1106d94,3d7,0,...) at usb_process+0xe6/frame 0xf0225ccc fork_exit(c0968280,c790db24,f0225d08) at fork_exit+0x7f/frame 0xf0225cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf0225cf4 --- trap 0, eip = 0, esp = 0xf0225d40, ebp = 0 --- Tracing command usb pid 15 tid 100047 td 0xc7900c40 sched_switch(c7900c40,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf0222bd4 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xf0222c08 sleepq_switch(c7900c40,0,c1113475,26b,c7900c40,...) at sleepq_switch+0x1c5/frame 0xf0222c3c sleepq_wait(c790dafc,0,f0222c90,1,0,...) at sleepq_wait+0x48/frame 0xf0222c60 _cv_wait(c790dafc,c790dc74,c10ea7c7,77,c790db04,...) at _cv_wait+0x1f9/frame 0xf0222ca4 usb_process(c790daf4,f0222d08,c1106d94,3d7,0,...) at usb_process+0xe6/frame 0xf0222ccc fork_exit(c0968280,c790daf4,f0222d08) at fork_exit+0x7f/frame 0xf0222cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf0222cf4 --- trap 0, eip = 0, esp = 0xf0222d40, ebp = 0 --- Tracing command usb pid 15 tid 100046 td 0xc7901000 sched_switch(c7901000,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf021fbd4 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xf021fc08 sleepq_switch(c7901000,0,c1113475,26b,c7901000,...) at sleepq_switch+0x1c5/frame 0xf021fc3c sleepq_wait(c790dacc,0,f021fc90,1,0,...) at sleepq_wait+0x48/frame 0xf021fc60 _cv_wait(c790dacc,c790dc74,c10ea7c7,77,c790dad4,...) at _cv_wait+0x1f9/frame 0xf021fca4 usb_process(c790dac4,f021fd08,c1106d94,3d7,0,...) at usb_process+0xe6/frame 0xf021fccc fork_exit(c0968280,c790dac4,f021fd08) at fork_exit+0x7f/frame 0xf021fcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf021fcf4 --- trap 0, eip = 0, esp = 0xf021fd40, ebp = 0 --- Tracing command usb pid 15 tid 100045 td 0xc7901310 sched_switch(c7901310,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf021bbd4 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xf021bc08 sleepq_switch(c7901310,0,c1113475,26b,c7901310,...) at sleepq_switch+0x1c5/frame 0xf021bc3c sleepq_wait(c7903b5c,0,f021bc90,1,0,...) at sleepq_wait+0x48/frame 0xf021bc60 _cv_wait(c7903b5c,c7903c74,c10ea7c7,77,c7903b64,...) at _cv_wait+0x1f9/frame 0xf021bca4 usb_process(c7903b54,f021bd08,c1106d94,3d7,0,...) at usb_process+0xe6/frame 0xf021bccc fork_exit(c0968280,c7903b54,f021bd08) at fork_exit+0x7f/frame 0xf021bcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf021bcf4 --- trap 0, eip = 0, esp = 0xf021bd40, ebp = 0 --- Tracing command usb pid 15 tid 100044 td 0xc7901620 sched_switch(c7901620,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf0218bd4 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xf0218c08 sleepq_switch(c7901620,0,c1113475,26b,c7901620,...) at sleepq_switch+0x1c5/frame 0xf0218c3c sleepq_wait(c7903b2c,0,f0218c90,1,0,...) at sleepq_wait+0x48/frame 0xf0218c60 _cv_wait(c7903b2c,c7903c74,c10ea7c7,77,c7903b34,...) at _cv_wait+0x1f9/frame 0xf0218ca4 usb_process(c7903b24,f0218d08,c1106d94,3d7,0,...) at usb_process+0xe6/frame 0xf0218ccc fork_exit(c0968280,c7903b24,f0218d08) at fork_exit+0x7f/frame 0xf0218cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf0218cf4 --- trap 0, eip = 0, esp = 0xf0218d40, ebp = 0 --- Tracing command usb pid 15 tid 100043 td 0xc7901930 sched_switch(c7901930,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf0215bd4 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xf0215c08 sleepq_switch(c7901930,0,c1113475,26b,c7901930,...) at sleepq_switch+0x1c5/frame 0xf0215c3c sleepq_wait(c7903afc,0,f0215c90,1,0,...) at sleepq_wait+0x48/frame 0xf0215c60 _cv_wait(c7903afc,c7903c74,c10ea7c7,77,c7903b04,...) at _cv_wait+0x1f9/frame 0xf0215ca4 usb_process(c7903af4,f0215d08,c1106d94,3d7,0,...) at usb_process+0xe6/frame 0xf0215ccc fork_exit(c0968280,c7903af4,f0215d08) at fork_exit+0x7f/frame 0xf0215cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf0215cf4 --- trap 0, eip = 0, esp = 0xf0215d40, ebp = 0 --- Tracing command usb pid 15 tid 100042 td 0xc7901c40 sched_switch(c7901c40,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf0212bd4 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xf0212c08 sleepq_switch(c7901c40,0,c1113475,26b,c7901c40,...) at sleepq_switch+0x1c5/frame 0xf0212c3c sleepq_wait(c7903acc,0,f0212c90,1,0,...) at sleepq_wait+0x48/frame 0xf0212c60 _cv_wait(c7903acc,c7903c74,c10ea7c7,77,c7903ad4,...) at _cv_wait+0x1f9/frame 0xf0212ca4 usb_process(c7903ac4,f0212d08,c1106d94,3d7,0,...) at usb_process+0xe6/frame 0xf0212ccc fork_exit(c0968280,c7903ac4,f0212d08) at fork_exit+0x7f/frame 0xf0212cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf0212cf4 --- trap 0, eip = 0, esp = 0xf0212d40, ebp = 0 --- Tracing command usb pid 15 tid 100040 td 0xc7902310 sched_switch(c7902310,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf020bbd4 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xf020bc08 sleepq_switch(c7902310,0,c1113475,26b,c7902310,...) at sleepq_switch+0x1c5/frame 0xf020bc3c sleepq_wait(c78fdb5c,0,f020bc90,1,0,...) at sleepq_wait+0x48/frame 0xf020bc60 _cv_wait(c78fdb5c,c78fdc74,c10ea7c7,77,c78fdb64,...) at _cv_wait+0x1f9/frame 0xf020bca4 usb_process(c78fdb54,f020bd08,c1106d94,3d7,0,...) at usb_process+0xe6/frame 0xf020bccc fork_exit(c0968280,c78fdb54,f020bd08) at fork_exit+0x7f/frame 0xf020bcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf020bcf4 --- trap 0, eip = 0, esp = 0xf020bd40, ebp = 0 --- Tracing command usb pid 15 tid 100039 td 0xc779a310 sched_switch(c779a310,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf0208bd4 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xf0208c08 sleepq_switch(c779a310,0,c1113475,26b,c779a310,...) at sleepq_switch+0x1c5/frame 0xf0208c3c sleepq_wait(c78fdb2c,0,f0208c90,1,0,...) at sleepq_wait+0x48/frame 0xf0208c60 _cv_wait(c78fdb2c,c78fdc74,c10ea7c7,77,c78fdb34,...) at _cv_wait+0x1f9/frame 0xf0208ca4 usb_process(c78fdb24,f0208d08,c1106d94,3d7,0,...) at usb_process+0xe6/frame 0xf0208ccc fork_exit(c0968280,c78fdb24,f0208d08) at fork_exit+0x7f/frame 0xf0208cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf0208cf4 --- trap 0, eip = 0, esp = 0xf0208d40, ebp = 0 --- Tracing command usb pid 15 tid 100038 td 0xc779a620 sched_switch(c779a620,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf0205bd4 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xf0205c08 sleepq_switch(c779a620,0,c1113475,26b,c779a620,...) at sleepq_switch+0x1c5/frame 0xf0205c3c sleepq_wait(c78fdafc,0,f0205c90,1,0,...) at sleepq_wait+0x48/frame 0xf0205c60 _cv_wait(c78fdafc,c78fdc74,c10ea7c7,77,c78fdb04,...) at _cv_wait+0x1f9/frame 0xf0205ca4 usb_process(c78fdaf4,f0205d08,c1106d94,3d7,0,...) at usb_process+0xe6/frame 0xf0205ccc fork_exit(c0968280,c78fdaf4,f0205d08) at fork_exit+0x7f/frame 0xf0205cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf0205cf4 --- trap 0, eip = 0, esp = 0xf0205d40, ebp = 0 --- Tracing command usb pid 15 tid 100037 td 0xc779a930 sched_switch(c779a930,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf0202bd4 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xf0202c08 sleepq_switch(c779a930,0,c1113475,26b,c779a930,...) at sleepq_switch+0x1c5/frame 0xf0202c3c sleepq_wait(c78fdacc,0,f0202c90,1,0,...) at sleepq_wait+0x48/frame 0xf0202c60 _cv_wait(c78fdacc,c78fdc74,c10ea7c7,77,c78fdad4,...) at _cv_wait+0x1f9/frame 0xf0202ca4 usb_process(c78fdac4,f0202d08,c1106d94,3d7,0,...) at usb_process+0xe6/frame 0xf0202ccc fork_exit(c0968280,c78fdac4,f0202d08) at fork_exit+0x7f/frame 0xf0202cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf0202cf4 --- trap 0, eip = 0, esp = 0xf0202d40, ebp = 0 --- Tracing command usb pid 15 tid 100035 td 0xc7892000 sched_switch(c7892000,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf01fcbd4 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xf01fcc08 sleepq_switch(c7892000,0,c1113475,26b,c7892000,...) at sleepq_switch+0x1c5/frame 0xf01fcc3c sleepq_wait(c78f6b5c,0,f01fcc90,1,0,...) at sleepq_wait+0x48/frame 0xf01fcc60 _cv_wait(c78f6b5c,c78f6c74,c10ea7c7,77,c78f6b64,...) at _cv_wait+0x1f9/frame 0xf01fcca4 usb_process(c78f6b54,f01fcd08,c1106d94,3d7,0,...) at usb_process+0xe6/frame 0xf01fcccc fork_exit(c0968280,c78f6b54,f01fcd08) at fork_exit+0x7f/frame 0xf01fccf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf01fccf4 --- trap 0, eip = 0, esp = 0xf01fcd40, ebp = 0 --- Tracing command usb pid 15 tid 100034 td 0xc7892310 sched_switch(c7892310,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf01f9bd4 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xf01f9c08 sleepq_switch(c7892310,0,c1113475,26b,c7892310,...) at sleepq_switch+0x1c5/frame 0xf01f9c3c sleepq_wait(c78f6b2c,0,f01f9c90,1,0,...) at sleepq_wait+0x48/frame 0xf01f9c60 _cv_wait(c78f6b2c,c78f6c74,c10ea7c7,77,c78f6b34,...) at _cv_wait+0x1f9/frame 0xf01f9ca4 usb_process(c78f6b24,f01f9d08,c1106d94,3d7,0,...) at usb_process+0xe6/frame 0xf01f9ccc fork_exit(c0968280,c78f6b24,f01f9d08) at fork_exit+0x7f/frame 0xf01f9cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf01f9cf4 --- trap 0, eip = 0, esp = 0xf01f9d40, ebp = 0 --- Tracing command usb pid 15 tid 100033 td 0xc7892620 sched_switch(c7892620,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xe37febd4 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xe37fec08 sleepq_switch(c7892620,0,c1113475,26b,c7892620,...) at sleepq_switch+0x1c5/frame 0xe37fec3c sleepq_wait(c78f6afc,0,e37fec90,1,0,...) at sleepq_wait+0x48/frame 0xe37fec60 _cv_wait(c78f6afc,c78f6c74,c10ea7c7,77,c78f6b04,...) at _cv_wait+0x1f9/frame 0xe37feca4 usb_process(c78f6af4,e37fed08,c1106d94,3d7,0,...) at usb_process+0xe6/frame 0xe37feccc fork_exit(c0968280,c78f6af4,e37fed08) at fork_exit+0x7f/frame 0xe37fecf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe37fecf4 --- trap 0, eip = 0, esp = 0xe37fed40, ebp = 0 --- Tracing command usb pid 15 tid 100032 td 0xc7892930 sched_switch(c7892930,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xe37fbbd4 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xe37fbc08 sleepq_switch(c7892930,0,c1113475,26b,c7892930,...) at sleepq_switch+0x1c5/frame 0xe37fbc3c sleepq_wait(c78f6acc,0,e37fbc90,1,0,...) at sleepq_wait+0x48/frame 0xe37fbc60 _cv_wait(c78f6acc,c78f6c74,c10ea7c7,77,c78f6ad4,...) at _cv_wait+0x1f9/frame 0xe37fbca4 usb_process(c78f6ac4,e37fbd08,c1106d94,3d7,0,...) at usb_process+0xe6/frame 0xe37fbccc fork_exit(c0968280,c78f6ac4,e37fbd08) at fork_exit+0x7f/frame 0xe37fbcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe37fbcf4 --- trap 0, eip = 0, esp = 0xe37fbd40, ebp = 0 --- Tracing command yarrow pid 14 tid 100017 td 0xc7799620 sched_switch(c7799620,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xc7236bb4 mi_switch(104,0,c1113475,1f5,c12a7728,...) at mi_switch+0x1c9/frame 0xc7236be8 sleepq_switch(c7799620,0,c1113475,28e,c12712d0,...) at sleepq_switch+0x1c5/frame 0xc7236c1c sleepq_timedwait(c12712d0,0,c110e065,c10f9046,0,...) at sleepq_timedwait+0x48/frame 0xc7236c40 msleep_spin_sbt(c12712d0,c1452694,c10f9046,19999999,0,...) at msleep_spin_sbt+0x225/frame 0xc7236c88 random_kthread(0,c7236d08,c1106d94,3d7,0,...) at random_kthread+0x2ea/frame 0xc7236ccc fork_exit(c0856cb0,0,c7236d08) at fork_exit+0x7f/frame 0xc7236cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7236cf4 --- trap 0, eip = 0, esp = 0xc7236d40, ebp = 0 --- Tracing command geom pid 13 tid 100015 td 0xc7799c40 sched_switch(c7799c40,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xc722fb70 mi_switch(104,0,c1113475,1f5,162,...) at mi_switch+0x1c9/frame 0xc722fba4 sleepq_switch(c7799c40,0,c1113475,26b,c145396c,...) at sleepq_switch+0x1c5/frame 0xc722fbd8 sleepq_wait(c145396c,5c,c10f9046,0,0,...) at sleepq_wait+0x48/frame 0xc722fbfc _sleep(c145396c,c128887c,25c,c10f9046,0,...) at _sleep+0x385/frame 0xc722fc58 g_io_schedule_down(c7799c40,0,c10fad52,6c,c722fcf4,...) at g_io_schedule_down+0x5c/frame 0xc722fcb4 g_down_procbody(0,c722fd08,c1106d94,3d7,0,...) at g_down_procbody+0x9d/frame 0xc722fccc fork_exit(c0a16d30,0,c722fd08) at fork_exit+0x7f/frame 0xc722fcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc722fcf4 --- trap 0, eip = 0, esp = 0xc722fd40, ebp = 0 --- Tracing command geom pid 13 tid 100014 td 0xc779a000 sched_switch(c779a000,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xc722cb98 mi_switch(104,0,c1113475,1f5,162,...) at mi_switch+0x1c9/frame 0xc722cbcc sleepq_switch(c779a000,0,c1113475,26b,c1453968,...) at sleepq_switch+0x1c5/frame 0xc722cc00 sleepq_wait(c1453968,5c,c10f9046,0,0,...) at sleepq_wait+0x48/frame 0xc722cc24 _sleep(c1453968,c128889c,25c,c10f9046,0,...) at _sleep+0x385/frame 0xc722cc80 g_io_schedule_up(c779a000,0,c10fad52,5f,c722ccf4,...) at g_io_schedule_up+0xe6/frame 0xc722ccb4 g_up_procbody(0,c722cd08,c1106d94,3d7,0,...) at g_up_procbody+0x9d/frame 0xc722cccc fork_exit(c0a16c90,0,c722cd08) at fork_exit+0x7f/frame 0xc722ccf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc722ccf4 --- trap 0, eip = 0, esp = 0xc722cd40, ebp = 0 --- Tracing command geom pid 13 tid 100013 td 0xc76dd000 sched_switch(c76dd000,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xc7229ba8 mi_switch(104,0,c1113475,1f5,162,...) at mi_switch+0x1c9/frame 0xc7229bdc sleepq_switch(c76dd000,0,c1113475,26b,c1453964,...) at sleepq_switch+0x1c5/frame 0xc7229c10 sleepq_wait(c1453964,5c,c10f9046,0,0,...) at sleepq_wait+0x48/frame 0xc7229c34 _sleep(c1453964,c1288854,25c,c10f9046,0,...) at _sleep+0x385/frame 0xc7229c90 g_run_events(0,c7229d08,c1106d94,3d7,0,...) at g_run_events+0x586/frame 0xc7229ccc fork_exit(c0a16bf0,0,c7229d08) at fork_exit+0x7f/frame 0xc7229cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7229cf4 --- trap 0, eip = 0, esp = 0xc7229d40, ebp = 0 --- Tracing command intr pid 12 tid 100063 td 0xc7902c40 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100062 td 0xc7916000 sched_switch(c7916000,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xf026ec60 mi_switch(109,0,c1107005,55a,8,...) at mi_switch+0x1c9/frame 0xf026ec94 ithread_loop(c77305c0,f026ed08,c1106d94,3d7,0,...) at ithread_loop+0x100/frame 0xf026eccc fork_exit(c0a88780,c77305c0,f026ed08) at fork_exit+0x7f/frame 0xf026ecf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf026ecf4 --- trap 0, eip = 0, esp = 0xf026ed40, ebp = 0 --- Tracing command intr pid 12 tid 100060 td 0xc7916620 sched_switch(c7916620,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xf025ec60 mi_switch(109,0,c1107005,55a,8,...) at mi_switch+0x1c9/frame 0xf025ec94 ithread_loop(c77305d0,f025ed08,c1106d94,3d7,0,...) at ithread_loop+0x100/frame 0xf025eccc fork_exit(c0a88780,c77305d0,f025ed08) at fork_exit+0x7f/frame 0xf025ecf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf025ecf4 --- trap 0, eip = 0, esp = 0xf025ed40, ebp = 0 --- Tracing command intr pid 12 tid 100059 td 0xc7916930 sched_switch(c7916930,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xf024dc60 mi_switch(109,0,c1107005,55a,8,...) at mi_switch+0x1c9/frame 0xf024dc94 ithread_loop(c77305f0,f024dd08,c1106d94,3d7,0,...) at ithread_loop+0x100/frame 0xf024dccc fork_exit(c0a88780,c77305f0,f024dd08) at fork_exit+0x7f/frame 0xf024dcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf024dcf4 --- trap 0, eip = 0, esp = 0xf024dd40, ebp = 0 --- Tracing command intr pid 12 tid 100054 td 0xc7917930 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100041 td 0xc7902000 sched_switch(c7902000,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xf020fc60 mi_switch(109,0,c1107005,55a,8,...) at mi_switch+0x1c9/frame 0xf020fc94 ithread_loop(c7730610,f020fd08,c1106d94,3d7,0,...) at ithread_loop+0x100/frame 0xf020fccc fork_exit(c0a88780,c7730610,f020fd08) at fork_exit+0x7f/frame 0xf020fcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf020fcf4 --- trap 0, eip = 0, esp = 0xf020fd40, ebp = 0 --- Tracing command intr pid 12 tid 100036 td 0xc779ac40 sched_switch(c779ac40,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xf01ffc60 mi_switch(109,0,c1107005,55a,8,...) at mi_switch+0x1c9/frame 0xf01ffc94 ithread_loop(c7730620,f01ffd08,c1106d94,3d7,0,...) at ithread_loop+0x100/frame 0xf01ffccc fork_exit(c0a88780,c7730620,f01ffd08) at fork_exit+0x7f/frame 0xf01ffcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf01ffcf4 --- trap 0, eip = 0, esp = 0xf01ffd40, ebp = 0 --- Tracing command intr pid 12 tid 100031 td 0xc7892c40 sched_switch(c7892c40,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xe37f8c60 mi_switch(109,0,c1107005,55a,8,...) at mi_switch+0x1c9/frame 0xe37f8c94 ithread_loop(c7730630,e37f8d08,c1106d94,3d7,0,...) at ithread_loop+0x100/frame 0xe37f8ccc fork_exit(c0a88780,c7730630,e37f8d08) at fork_exit+0x7f/frame 0xe37f8cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xe37f8cf4 --- trap 0, eip = 0, esp = 0xe37f8d40, ebp = 0 --- Tracing command intr pid 12 tid 100030 td 0xc7893000 sched_switch(c7893000,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xc7393c60 mi_switch(109,0,c1107005,55a,8,...) at mi_switch+0x1c9/frame 0xc7393c94 ithread_loop(c7730640,c7393d08,c1106d94,3d7,0,...) at ithread_loop+0x100/frame 0xc7393ccc fork_exit(c0a88780,c7730640,c7393d08) at fork_exit+0x7f/frame 0xc7393cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7393cf4 --- trap 0, eip = 0, esp = 0xc7393d40, ebp = 0 --- Tracing command intr pid 12 tid 100029 td 0xc7893310 sched_switch(c7893310,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xc738fc60 mi_switch(109,0,c1107005,55a,8,...) at mi_switch+0x1c9/frame 0xc738fc94 ithread_loop(c7730650,c738fd08,c1106d94,3d7,0,...) at ithread_loop+0x100/frame 0xc738fccc fork_exit(c0a88780,c7730650,c738fd08) at fork_exit+0x7f/frame 0xc738fcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc738fcf4 --- trap 0, eip = 0, esp = 0xc738fd40, ebp = 0 --- Tracing command intr pid 12 tid 100028 td 0xc7893620 sched_switch(c7893620,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xc7286c60 mi_switch(109,0,c1107005,55a,8,...) at mi_switch+0x1c9/frame 0xc7286c94 ithread_loop(c7730660,c7286d08,c1106d94,3d7,0,...) at ithread_loop+0x100/frame 0xc7286ccc fork_exit(c0a88780,c7730660,c7286d08) at fork_exit+0x7f/frame 0xc7286cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7286cf4 --- trap 0, eip = 0, esp = 0xc7286d40, ebp = 0 --- Tracing command intr pid 12 tid 100027 td 0xc7893930 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100025 td 0xc76e4c40 sched_switch(c76e4c40,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xc724ec60 mi_switch(109,0,c1107005,55a,8,...) at mi_switch+0x1c9/frame 0xc724ec94 ithread_loop(c7730690,c724ed08,c1106d94,3d7,0,...) at ithread_loop+0x100/frame 0xc724eccc fork_exit(c0a88780,c7730690,c724ed08) at fork_exit+0x7f/frame 0xc724ecf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc724ecf4 --- trap 0, eip = 0, esp = 0xc724ed40, ebp = 0 --- Tracing command intr pid 12 tid 100023 td 0xc7798310 sched_switch(c7798310,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xc7248c60 mi_switch(109,0,c1107005,55a,8,...) at mi_switch+0x1c9/frame 0xc7248c94 ithread_loop(c77306b0,c7248d08,c1106d94,3d7,0,...) at ithread_loop+0x100/frame 0xc7248ccc fork_exit(c0a88780,c77306b0,c7248d08) at fork_exit+0x7f/frame 0xc7248cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7248cf4 --- trap 0, eip = 0, esp = 0xc7248d40, ebp = 0 --- Tracing command intr pid 12 tid 100018 td 0xc7799310 sched_switch(c7799310,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xc7239c60 mi_switch(109,0,c1107005,55a,8,...) at mi_switch+0x1c9/frame 0xc7239c94 ithread_loop(c77306e0,c7239d08,c1106d94,3d7,0,...) at ithread_loop+0x100/frame 0xc7239ccc fork_exit(c0a88780,c77306e0,c7239d08) at fork_exit+0x7f/frame 0xc7239cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7239cf4 --- trap 0, eip = 0, esp = 0xc7239d40, ebp = 0 --- Tracing command intr pid 12 tid 100012 td 0xc76dd310 sched_switch(c76dd310,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xc7226c60 mi_switch(109,0,c1107005,55a,8,...) at mi_switch+0x1c9/frame 0xc7226c94 ithread_loop(c7730770,c7226d08,c1106d94,3d7,0,...) at ithread_loop+0x100/frame 0xc7226ccc fork_exit(c0a88780,c7730770,c7226d08) at fork_exit+0x7f/frame 0xc7226cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7226cf4 --- trap 0, eip = 0, esp = 0xc7226d40, ebp = 0 --- Tracing command intr pid 12 tid 100011 td 0xc76dd620 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100010 td 0xc76dd930 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100009 td 0xc76ddc40 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100008 td 0xc76e3000 sched_switch(c76e3000,0,109,1b5,0,...) at sched_switch+0x4b9/frame 0xc721ac60 mi_switch(109,0,c1107005,55a,8,...) at mi_switch+0x1c9/frame 0xc721ac94 ithread_loop(c77307b0,c721ad08,c1106d94,3d7,0,...) at ithread_loop+0x100/frame 0xc721accc fork_exit(c0a88780,c77307b0,c721ad08) at fork_exit+0x7f/frame 0xc721acf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc721acf4 --- trap 0, eip = 0, esp = 0xc721ad40, ebp = 0 --- Tracing command intr pid 12 tid 100007 td 0xc76e3310 fork_trampoline() at fork_trampoline Tracing command idle pid 11 tid 100006 td 0xc76e3620 sched_switch(c76e3620,0,608,1b0,d40,...) at sched_switch+0x4b9/frame 0xc7213c00 mi_switch(608,0,c110dac4,d3,23,...) at mi_switch+0x1c9/frame 0xc7213c34 critical_exit(0,2,c110f9e5,409,c1298e08,...) at critical_exit+0xa3/frame 0xc7213c50 sched_idletd(0,c7213d08,c1106d94,3d7,0,...) at sched_idletd+0x1d6/frame 0xc7213ccc fork_exit(c0ae0420,0,c7213d08) at fork_exit+0x7f/frame 0xc7213cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7213cf4 --- trap 0, eip = 0, esp = 0xc7213d40, ebp = 0 --- Tracing command idle pid 11 tid 100005 td 0xc76e3930 cpustop_handler(2,13,c7210b90,c0ede3ce,52c2c127,...) at cpustop_handler+0x2f/frame 0xc7210a38 ipi_nmi_handler(52c2c127,51d1b952,6b3eaa08,f1ed3d5d,0,...) at ipi_nmi_handler+0x37/frame 0xc7210a48 trap(c7210b9c) at trap+0x3e/frame 0xc7210b90 calltrap() at calltrap+0x6/frame 0xc7210b90 --- trap 0x13, eip = 0xc0ec3f95, esp = 0xc7210bdc, ebp = 0xc7210bdc --- acpi_cpu_c1(c0ee728b,c71fdc90,0,c1167633,303,...) at acpi_cpu_c1+0x5/frame 0xc7210bdc acpi_cpu_idle(3726e144,0,3726e144,c7210c50,c0ece06a,...) at acpi_cpu_idle+0x149/frame 0xc7210c20 cpu_idle_acpi(3726e144,0,25,c1298708,c1298716,...) at cpu_idle_acpi+0x3f/frame 0xc7210c34 cpu_idle(0,2,c110f9e5,409,c1298708,...) at cpu_idle+0x9a/frame 0xc7210c50 sched_idletd(0,c7210d08,c1106d94,3d7,0,...) at sched_idletd+0x1d6/frame 0xc7210ccc fork_exit(c0ae0420,0,c7210d08) at fork_exit+0x7f/frame 0xc7210cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7210cf4 --- trap 0, eip = 0, esp = 0xc7210d40, ebp = 0 --- Tracing command idle pid 11 tid 100004 td 0xc76e3c40 cpustop_handler(1,13,c720db90,c0ede3ce,42a1c268,...) at cpustop_handler+0x2f/frame 0xc720da38 ipi_nmi_handler(42a1c268,51d1b952,6b3eaa08,f1ed3d5d,0,...) at ipi_nmi_handler+0x37/frame 0xc720da48 trap(c720db9c) at trap+0x3e/frame 0xc720db90 calltrap() at calltrap+0x6/frame 0xc720db90 --- trap 0x13, eip = 0xc0ec3f95, esp = 0xc720dbdc, ebp = 0xc720dbdc --- acpi_cpu_c1(c0ee728b,c71fac90,0,c1167633,303,...) at acpi_cpu_c1+0x5/frame 0xc720dbdc acpi_cpu_idle(24a0794f,0,24a0794f,c720dc50,c0ece06a,...) at acpi_cpu_idle+0x149/frame 0xc720dc20 cpu_idle_acpi(24a0794f,0,10,c1298008,c1298016,...) at cpu_idle_acpi+0x3f/frame 0xc720dc34 cpu_idle(0,2,c110f9e5,a46,c1298008,...) at cpu_idle+0x9a/frame 0xc720dc50 sched_idletd(0,c720dd08,c1106d94,3d7,0,...) at sched_idletd+0x1d6/frame 0xc720dccc fork_exit(c0ae0420,0,c720dd08) at fork_exit+0x7f/frame 0xc720dcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc720dcf4 --- trap 0, eip = 0, esp = 0xc720dd40, ebp = 0 --- Tracing command idle pid 11 tid 100003 td 0xc76e4000 cpustop_handler(0,13,c720ab90,c0ede3ce,c1298690,...) at cpustop_handler+0x2f/frame 0xc720aa38 ipi_nmi_handler(c1298690,c76e3000,c76e3000,c129791c,c720aa84,...) at ipi_nmi_handler+0x37/frame 0xc720aa48 trap(c720ab9c) at trap+0x3e/frame 0xc720ab90 calltrap() at calltrap+0x6/frame 0xc720ab90 --- trap 0x13, eip = 0xc0ec3f95, esp = 0xc720abdc, ebp = 0xc720abdc --- acpi_cpu_c1(c0ee728b,c1825c90,0,c1167633,303,...) at acpi_cpu_c1+0x5/frame 0xc720abdc acpi_cpu_idle(c0d1751,0,c0d1751,c720ac50,c0ece06a,...) at acpi_cpu_idle+0x149/frame 0xc720ac20 cpu_idle_acpi(c0d1751,0,27,c1297908,c1297916,...) at cpu_idle_acpi+0x3f/frame 0xc720ac34 cpu_idle(0,2,c110f9e5,409,c1297908,...) at cpu_idle+0x9a/frame 0xc720ac50 sched_idletd(0,c720ad08,c1106d94,3d7,0,...) at sched_idletd+0x1d6/frame 0xc720accc fork_exit(c0ae0420,0,c720ad08) at fork_exit+0x7f/frame 0xc720acf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc720acf4 --- trap 0, eip = 0, esp = 0xc720ad40, ebp = 0 --- Tracing command init pid 1 tid 100002 td 0xc76e4310 sched_switch(c76e4310,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xc720698c mi_switch(104,0,c1113475,1f5,c1113475,...) at mi_switch+0x1c9/frame 0xc72069c0 sleepq_switch(c76e4310,0,c1113475,1ab,c7206a38,...) at sleepq_switch+0x1c5/frame 0xc72069f4 sleepq_catch_signals(0,c1113475,162,3f0,c76dc2e8,...) at sleepq_catch_signals+0x36c/frame 0xc7206a3c sleepq_wait_sig(c76dc2e8,6c,c1116f7d,100,0,...) at sleepq_wait_sig+0x14/frame 0xc7206a60 _sleep(c76dc2e8,c76dc37c,16c,c1116f7d,0,...) at _sleep+0x364/frame 0xc7206abc kern_wait6(c76e4310,7,0,0,c7206b4c,...) at kern_wait6+0x45c/frame 0xc7206b20 sys_wait4(c76e4310,c7206cc8,c116612a,d6,bfbfefe4,...) at sys_wait4+0x94/frame 0xc7206c40 syscall(c7206d08) at syscall+0x2ed/frame 0xc7206cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xc7206cfc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x805eb07, esp = 0xbfbfd8f4, ebp = 0xbfbfd908 --- Tracing command audit pid 10 tid 100001 td 0xc76e4620 sched_switch(c76e4620,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xc7203ba8 mi_switch(104,0,c1113475,1f5,8,...) at mi_switch+0x1c9/frame 0xc7203bdc sleepq_switch(c76e4620,0,c1113475,26b,c76e4620,...) at sleepq_switch+0x1c5/frame 0xc7203c10 sleepq_wait(c145a6b8,0,c7203c64,1,0,...) at sleepq_wait+0x48/frame 0xc7203c34 _cv_wait(c145a6b8,c145a644,c113f772,18d,0,...) at _cv_wait+0x1f9/frame 0xc7203c78 audit_worker(0,c7203d08,c1106d94,3d7,0,...) at audit_worker+0x94/frame 0xc7203ccc fork_exit(c0cd1040,0,c7203d08) at fork_exit+0x7f/frame 0xc7203cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7203cf4 --- trap 0, eip = 0, esp = 0xc7203d40, ebp = 0 --- Tracing command kernel pid 0 tid 100065 td 0xc7902620 sched_switch(c7902620,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf0277b68 mi_switch(104,0,c1113475,1f5,101,...) at mi_switch+0x1c9/frame 0xf0277b9c sleepq_switch(c7902620,0,c1113475,28e,c128a563,...) at sleepq_switch+0x1c5/frame 0xf0277bd0 sleepq_timedwait(c128a563,0,2,0,0,...) at sleepq_timedwait+0x48/frame 0xf0277bf4 _sleep(c128a563,0,0,c10f9046,fffffc88,...) at _sleep+0x350/frame 0xf0277c50 pause_sbt(c10f9046,fffffc88,2,0,0,...) at pause_sbt+0x86/frame 0xf0277c8c deadlkres(0,f0277d08,c1106d94,3d7,0,...) at deadlkres+0x82/frame 0xf0277ccc fork_exit(c0a67040,0,f0277d08) at fork_exit+0x7f/frame 0xf0277cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf0277cf4 --- trap 0, eip = 0, esp = 0xf0277d40, ebp = 0 --- Tracing command kernel pid 0 tid 100064 td 0xc7902930 sched_switch(c7902930,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xf0274bc0 mi_switch(104,0,c1113475,1f5,c12a6a38,...) at mi_switch+0x1c9/frame 0xf0274bf4 sleepq_switch(c7902930,0,c1113475,26b,c76b3800,...) at sleepq_switch+0x1c5/frame 0xf0274c28 sleepq_wait(c76b3800,0,c110e065,c10f9046,0,...) at sleepq_wait+0x48/frame 0xf0274c4c msleep_spin_sbt(c76b3800,c76b3818,c10f9046,0,0,...) at msleep_spin_sbt+0x239/frame 0xf0274c94 taskqueue_thread_loop(c1450180,f0274d08,c1106d94,3d7,0,...) at taskqueue_thread_loop+0xcc/frame 0xf0274ccc fork_exit(c0b00bf0,c1450180,f0274d08) at fork_exit+0x7f/frame 0xf0274cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xf0274cf4 --- trap 0, eip = 0, esp = 0xf0274d40, ebp = 0 --- Tracing command kernel pid 0 tid 100026 td 0xc76e4930 sched_switch(c76e4930,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xc7251bac mi_switch(104,0,c1113475,1f5,162,...) at mi_switch+0x1c9/frame 0xc7251be0 sleepq_switch(c76e4930,0,c1113475,26b,c76b3b00,...) at sleepq_switch+0x1c5/frame 0xc7251c14 sleepq_wait(c76b3b00,0,c10f9046,0,0,...) at sleepq_wait+0x48/frame 0xc7251c38 _sleep(c76b3b00,c76b3b18,0,c10f9046,0,...) at _sleep+0x385/frame 0xc7251c94 taskqueue_thread_loop(c14570c0,c7251d08,c1106d94,3d7,0,...) at taskqueue_thread_loop+0x129/frame 0xc7251ccc fork_exit(c0b00bf0,c14570c0,c7251d08) at fork_exit+0x7f/frame 0xc7251cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7251cf4 --- trap 0, eip = 0, esp = 0xc7251d40, ebp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xc7798000 sched_switch(c7798000,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xc724bbac mi_switch(104,0,c1113475,1f5,162,...) at mi_switch+0x1c9/frame 0xc724bbe0 sleepq_switch(c7798000,0,c1113475,26b,c76b3c80,...) at sleepq_switch+0x1c5/frame 0xc724bc14 sleepq_wait(c76b3c80,0,c10f9046,0,0,...) at sleepq_wait+0x48/frame 0xc724bc38 _sleep(c76b3c80,c76b3c98,0,c10f9046,0,...) at _sleep+0x385/frame 0xc724bc94 taskqueue_thread_loop(c145a6f0,c724bd08,c1106d94,3d7,0,...) at taskqueue_thread_loop+0x129/frame 0xc724bccc fork_exit(c0b00bf0,c145a6f0,c724bd08) at fork_exit+0x7f/frame 0xc724bcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc724bcf4 --- trap 0, eip = 0, esp = 0xc724bd40, ebp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xc7798620 sched_switch(c7798620,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xc7245bac mi_switch(104,0,c1113475,1f5,162,...) at mi_switch+0x1c9/frame 0xc7245be0 sleepq_switch(c7798620,0,c1113475,26b,c76b3e00,...) at sleepq_switch+0x1c5/frame 0xc7245c14 sleepq_wait(c76b3e00,0,c10f9046,0,0,...) at sleepq_wait+0x48/frame 0xc7245c38 _sleep(c76b3e00,c76b3e18,0,c10f9046,0,...) at _sleep+0x385/frame 0xc7245c94 taskqueue_thread_loop(c1454160,c7245d08,c1106d94,3d7,0,...) at taskqueue_thread_loop+0x129/frame 0xc7245ccc fork_exit(c0b00bf0,c1454160,c7245d08) at fork_exit+0x7f/frame 0xc7245cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7245cf4 --- trap 0, eip = 0, esp = 0xc7245d40, ebp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xc7798930 sched_switch(c7798930,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xc7242bc0 mi_switch(104,0,c1113475,1f5,c12a76c8,...) at mi_switch+0x1c9/frame 0xc7242bf4 sleepq_switch(c7798930,0,c1113475,26b,c76b3e80,...) at sleepq_switch+0x1c5/frame 0xc7242c28 sleepq_wait(c76b3e80,0,c110e065,c10f9046,0,...) at sleepq_wait+0x48/frame 0xc7242c4c msleep_spin_sbt(c76b3e80,c76b3e98,c10f9046,0,0,...) at msleep_spin_sbt+0x239/frame 0xc7242c94 taskqueue_thread_loop(c1450ca4,c7242d08,c1106d94,3d7,0,...) at taskqueue_thread_loop+0xcc/frame 0xc7242ccc fork_exit(c0b00bf0,c1450ca4,c7242d08) at fork_exit+0x7f/frame 0xc7242cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7242cf4 --- trap 0, eip = 0, esp = 0xc7242d40, ebp = 0 --- Tracing command kernel pid 0 tid 100020 td 0xc7798c40 sched_switch(c7798c40,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xc723fbc0 mi_switch(104,0,c1113475,1f5,c12a76c8,...) at mi_switch+0x1c9/frame 0xc723fbf4 sleepq_switch(c7798c40,0,c1113475,26b,c76b3e80,...) at sleepq_switch+0x1c5/frame 0xc723fc28 sleepq_wait(c76b3e80,0,c110e065,c10f9046,0,...) at sleepq_wait+0x48/frame 0xc723fc4c msleep_spin_sbt(c76b3e80,c76b3e98,c10f9046,0,0,...) at msleep_spin_sbt+0x239/frame 0xc723fc94 taskqueue_thread_loop(c1450ca4,c723fd08,c1106d94,3d7,0,...) at taskqueue_thread_loop+0xcc/frame 0xc723fccc fork_exit(c0b00bf0,c1450ca4,c723fd08) at fork_exit+0x7f/frame 0xc723fcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc723fcf4 --- trap 0, eip = 0, esp = 0xc723fd40, ebp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xc7799000 sched_switch(c7799000,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xc723cbc0 mi_switch(104,0,c1113475,1f5,c12a76c8,...) at mi_switch+0x1c9/frame 0xc723cbf4 sleepq_switch(c7799000,0,c1113475,26b,c76b3e80,...) at sleepq_switch+0x1c5/frame 0xc723cc28 sleepq_wait(c76b3e80,0,c110e065,c10f9046,0,...) at sleepq_wait+0x48/frame 0xc723cc4c msleep_spin_sbt(c76b3e80,c76b3e98,c10f9046,0,0,...) at msleep_spin_sbt+0x239/frame 0xc723cc94 taskqueue_thread_loop(c1450ca4,c723cd08,c1106d94,3d7,0,...) at taskqueue_thread_loop+0xcc/frame 0xc723cccc fork_exit(c0b00bf0,c1450ca4,c723cd08) at fork_exit+0x7f/frame 0xc723ccf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc723ccf4 --- trap 0, eip = 0, esp = 0xc723cd40, ebp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xc7799930 sched_switch(c7799930,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xc7232bac mi_switch(104,0,c1113475,1f5,162,...) at mi_switch+0x1c9/frame 0xc7232be0 sleepq_switch(c7799930,0,c1113475,26b,c76b4000,...) at sleepq_switch+0x1c5/frame 0xc7232c14 sleepq_wait(c76b4000,0,c10f9046,0,0,...) at sleepq_wait+0x48/frame 0xc7232c38 _sleep(c76b4000,c76b4018,0,c10f9046,0,...) at _sleep+0x385/frame 0xc7232c94 taskqueue_thread_loop(c12a59c4,c7232d08,c1106d94,3d7,0,...) at taskqueue_thread_loop+0x129/frame 0xc7232ccc fork_exit(c0b00bf0,c12a59c4,c7232d08) at fork_exit+0x7f/frame 0xc7232cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7232cf4 --- trap 0, eip = 0, esp = 0xc7232d40, ebp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xc1453cd0 sched_switch(c1453cd0,0,104,1b5,c0aa3a8e,...) at sched_switch+0x4b9/frame 0xc1820c0c mi_switch(104,0,c1113475,1f5,101,...) at mi_switch+0x1c9/frame 0xc1820c40 sleepq_switch(c1453cd0,0,c1113475,28e,c14539e4,...) at sleepq_switch+0x1c5/frame 0xc1820c74 sleepq_timedwait(c14539e4,54,9,0,0,...) at sleepq_timedwait+0x48/frame 0xc1820c98 _sleep(c14539e4,0,54,c1140935,fffff470,...) at _sleep+0x350/frame 0xc1820cf4 scheduler(0,0,c1173924,181e000,1825000,...) at scheduler+0x2b3/frame 0xc1820d38 mi_startup() at mi_startup+0xf7/frame 0xc1820d58 begin() at begin+0x2c db:0:allt> call doadump Physical memory: 3435 MB Dumping 135 MB: 120 104 88 72 56 40 24 8 Dump complete = 0 db:0:doadump> reset cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 3 (kgdb) bt #0 doadump (textdump=0xa) at pcpu.h:253 #1 0xc051339d in db_fncall (dummy1=0x19, dummy2=0x0, dummy3=0xc125b0ec, dummy4=0xf7df74a0 "") at ../../../ddb/db_command.c:578 #2 0xc0513087 in db_command (cmd_table=) at ../../../ddb/db_command.c:449 #3 0xc0516fc4 in db_script_exec (scriptname=0xc1450a04 "doadump", warnifnotfound=) at ../../../ddb/db_script.c:302 #4 0xc0513087 in db_command (cmd_table=) at ../../../ddb/db_command.c:449 #5 0xc0512da0 in db_command_loop () at ../../../ddb/db_command.c:502 #6 0xc0515630 in db_trap (type=, code=0xf7df77f0) at ../../../ddb/db_main.c:231 #7 0xc0af06e8 in kdb_trap (type=, code=, tf=) at ../../../kern/subr_kdb.c:654 #8 0xc0edeb5f in trap (frame=) at ../../../i386/i386/trap.c:720 #9 0xc0ec7ebc in calltrap () at ../../../i386/i386/exception.s:170 #10 0xc0aeff8d in kdb_enter (why=0xc110d3eb "panic", msg=) at cpufunc.h:71 #11 0xc0ab85a3 in vpanic (fmt=, ap=) at ../../../kern/kern_shutdown.c:747 #12 0xc0ab845a in kassert_panic (fmt=) at ../../../kern/kern_shutdown.c:642 #13 0xc0d56517 in vm_page_insert_after (m=0xc913a000, object=0xc910a534, pindex=0x0, mpred=0xc4f8e5c0) at ../../../vm/vm_page.c:846 #14 0xc0d563ed in vm_page_insert (m=0xc913a000, object=0xc910a534, pindex=0x0) at ../../../vm/vm_page.c:818 #15 0xc0d369ef in old_dev_pager_fault (object=, offset=0x0, prot=, mres=) at ../../../vm/device_pager.c:353 #16 0xc0d36179 in dev_pager_getpages (object=, ma=0xf7df7a24, count=0xf7df7a24, reqpage=0x0) at ../../../vm/device_pager.c:264 #17 0xc0d4299f in vm_fault_hold (vaddr=0x1, fault_type=0x1, fault_flags=, m_hold=) at vm_pager.h:127 #18 0xc0d41f42 in vm_fault (map=0xc7e83d98, vaddr=, fault_type=0x90, fault_flags=) at ../../../vm/vm_fault.c:224 #19 0xc0edf21a in trap_pfault (frame=0xf7df7d08, usermode=0x1, eva=) at ../../../i386/i386/trap.c:928 #20 0xc0ede9a4 in trap (frame=0x2) at ../../../i386/i386/trap.c:417 #21 0xc0ec7ebc in calltrap () at ../../../i386/i386/exception.s:170 #22 0x281c602e in ?? () Previous frame inner to this frame (corrupt stack?) Current language: auto; currently minimal (kgdb) f 13 #13 0xc0d56517 in vm_page_insert_after (m=0xc913a000, object=0xc910a534, pindex=0x0, mpred=0xc4f8e5c0) at ../../../vm/vm_page.c:846 846 KASSERT(mpred->pindex < pindex, (kgdb) l 841 ("vm_page_insert_after: page already inserted")); 842 if (mpred != NULL) { 843 KASSERT(mpred->object == object || 844 (mpred->flags & PG_SLAB) != 0, 845 ("vm_page_insert_after: object doesn't contain mpred")); 846 KASSERT(mpred->pindex < pindex, 847 ("vm_page_insert_after: mpred doesn't precede pindex")); 848 msucc = TAILQ_NEXT(mpred, listq); 849 } else 850 msucc = TAILQ_FIRST(&object->memq); (kgdb) info loc sobj = (kgdb) p *mpred $1 = {pageq = {tqe_next = 0xc2fcb9c0, tqe_prev = 0xc1407da0}, listq = {tqe_next = 0x0, tqe_prev = 0xc910a55c}, object = 0xc910a534, pindex = 0x0, phys_addr = 0xb8c18000, md = {pv_list = {tqh_first = 0x0, tqh_last = 0xc4f8e5e0}, pat_mode = 0x6}, queue = 0xff, segind = 0x2, hold_count = 0x0, order = 0xb, pool = 0x0, cow = 0x0, wire_count = 0x0, aflags = 0x0, oflags = 0x5, flags = 0x0, act_count = 0x0, busy = 0x0, valid = 0x0, dirty = 0x0} (kgdb) p *object $2 = {lock = {lock_object = {lo_name = 0xc114adb2 "vm object", lo_flags = 0x5630000, lo_data = 0x0, lo_witness = 0xc757f318}, rw_lock = 0xc82ad930}, object_list = {tqe_next = 0x0, tqe_prev = 0xc9113d60}, shadow_head = {lh_first = 0x0}, shadow_list = {le_next = 0x0, le_prev = 0xc9112e00}, memq = {tqh_first = 0xc4f8e5c0, tqh_last = 0xc4f8e5c8}, rtree = {rt_root = 0xc4f8e5c1}, size = 0x8, generation = 0x1, ref_count = 0x9, shadow_count = 0x0, memattr = 0x6, type = 0x3, flags = 0x1003, pg_color = 0x347e, paging_in_progress = 0x1, resident_page_count = 0x1, backing_object = 0x0, backing_object_offset = 0x0, pager_object_list = {tqe_next = 0x0, tqe_prev = 0xc140751c}, rvq = {lh_first = 0x0}, cache = { rt_root = 0x0}, handle = 0xc7bd5700, un_pager = {vnp = {vnp_size = 0xc910a5ac00000000, writemappings = 0xc7bd5700c123f25c}, devp = {devp_pglist = {tqh_first = 0x0, tqh_last = 0xc910a5ac}, ops = 0xc123f25c, dev = 0xc7bd5700}, sgp = { sgp_pglist = {tqh_first = 0x0, tqh_last = 0xc910a5ac}}, swp = {swp_tmpfs = 0x0, swp_bcount = 0xc910a5ac}}, cred = 0x0, charge = 0x0} (kgdb) p *m $3 = {pageq = {tqe_next = 0x0, tqe_prev = 0x0}, listq = {tqe_next = 0x0, tqe_prev = 0x0}, object = 0x0, pindex = 0x0, phys_addr = 0x347e000, md = {pv_list = {tqh_first = 0x0, tqh_last = 0x0}, pat_mode = 0x6}, queue = 0xff, segind = 0x0, hold_count = 0x0, order = 0x0, pool = 0x0, cow = 0x0, wire_count = 0x1, aflags = 0x0, oflags = 0x5, flags = 0x4, act_count = 0x0, busy = 0x0, valid = 0x0, dirty = 0x0} (kgdb) $ svn diff -x -p /usr/src/sys Index: /usr/src/sys/vm/vm_radix.c =================================================================== --- /usr/src/sys/vm/vm_radix.c (revision 252443) +++ /usr/src/sys/vm/vm_radix.c (working copy) @@ -103,8 +103,7 @@ struct vm_radix_node { static uma_zone_t vm_radix_node_zone; /* - * Allocate a radix node. Pre-allocation should ensure that the request - * will always be satisfied. + * Allocate a radix node. */ static __inline struct vm_radix_node * vm_radix_node_get(vm_pindex_t owner, uint16_t count, uint16_t clevel) @@ -112,21 +111,8 @@ vm_radix_node_get(vm_pindex_t owner, uint16_t coun struct vm_radix_node *rnode; rnode = uma_zalloc(vm_radix_node_zone, M_NOWAIT); - - /* - * The required number of nodes should already be pre-allocated - * by vm_radix_prealloc(). However, UMA can hold a few nodes - * in per-CPU buckets, which will not be accessible by the - * current CPU. Thus, the allocation could return NULL when - * the pre-allocated pool is close to exhaustion. Anyway, - * in practice this should never occur because a new node - * is not always required for insert. Thus, the pre-allocated - * pool should have some extra pages that prevent this from - * becoming a problem. - */ if (rnode == NULL) - panic("%s: uma_zalloc() returned NULL for a new node", - __func__); + return (NULL); rnode->rn_owner = owner; rnode->rn_count = count; rnode->rn_clev = clevel; @@ -309,30 +295,29 @@ vm_radix_node_zone_init(void *mem, int size __unus } /* - * Pre-allocate intermediate nodes from the UMA slab zone. + * Reserve the KVA necessary to satisfy the node allocation. + * This is particular useful in architectures not supporting direct + * mapping as they will need otherwise to carve into the kernel maps, + * resulting into deadlocks for several consumers. */ static void -vm_radix_prealloc(void *arg __unused) +vm_radix_reserve_kva(void *arg __unused) { - int nodes; /* * Calculate the number of reserved nodes, discounting the pages that * are needed to store them. */ - nodes = ((vm_paddr_t)cnt.v_page_count * PAGE_SIZE) / (PAGE_SIZE + - sizeof(struct vm_radix_node)); - if (!uma_zone_reserve_kva(vm_radix_node_zone, nodes)) + if (!uma_zone_reserve_kva(vm_radix_node_zone, + ((vm_paddr_t)cnt.v_page_count * PAGE_SIZE) / (PAGE_SIZE + + sizeof(struct vm_radix_node)))) panic("%s: unable to create new zone", __func__); - uma_prealloc(vm_radix_node_zone, nodes); } -SYSINIT(vm_radix_prealloc, SI_SUB_KMEM, SI_ORDER_SECOND, vm_radix_prealloc, - NULL); +SYSINIT(vm_radix_reserve_kva, SI_SUB_KMEM, SI_ORDER_SECOND, + vm_radix_reserve_kva, NULL); /* * Initialize the UMA slab zone. - * Until vm_radix_prealloc() is called, the zone will be served by the - * UMA boot-time pre-allocated pool of pages. */ void vm_radix_init(void) @@ -345,15 +330,14 @@ vm_radix_init(void) #else NULL, #endif - vm_radix_node_zone_init, NULL, VM_RADIX_PAD, UMA_ZONE_VM | - UMA_ZONE_NOFREE); + vm_radix_node_zone_init, NULL, VM_RADIX_PAD, UMA_ZONE_VM); } /* * Inserts the key-value pair into the trie. * Panics if the key already exists. */ -void +int vm_radix_insert(struct vm_radix *rtree, vm_page_t page) { vm_pindex_t index, newind; @@ -372,7 +356,7 @@ vm_radix_insert(struct vm_radix *rtree, vm_page_t rnode = vm_radix_getroot(rtree); if (rnode == NULL) { rtree->rt_root = (uintptr_t)page | VM_RADIX_ISLEAF; - return; + return (0); } parentp = (void **)&rtree->rt_root; for (;;) { @@ -384,17 +368,19 @@ vm_radix_insert(struct vm_radix *rtree, vm_page_t clev = vm_radix_keydiff(m->pindex, index); tmp = vm_radix_node_get(vm_radix_trimkey(index, clev + 1), 2, clev); + if (tmp == NULL) + return (ENOMEM); *parentp = tmp; vm_radix_addpage(tmp, index, clev, page); vm_radix_addpage(tmp, m->pindex, clev, m); - return; + return (0); } else if (vm_radix_keybarr(rnode, index)) break; slot = vm_radix_slot(index, rnode->rn_clev); if (rnode->rn_child[slot] == NULL) { rnode->rn_count++; vm_radix_addpage(rnode, index, rnode->rn_clev, page); - return; + return (0); } parentp = &rnode->rn_child[slot]; rnode = rnode->rn_child[slot]; @@ -409,10 +395,13 @@ vm_radix_insert(struct vm_radix *rtree, vm_page_t clev = vm_radix_keydiff(newind, index); tmp = vm_radix_node_get(vm_radix_trimkey(index, clev + 1), 2, clev); + if (tmp == NULL) + return (ENOMEM); *parentp = tmp; vm_radix_addpage(tmp, index, clev, page); slot = vm_radix_slot(newind, clev); tmp->rn_child[slot] = rnode; + return (0); } /* Index: /usr/src/sys/vm/vm_radix.h =================================================================== --- /usr/src/sys/vm/vm_radix.h (revision 252443) +++ /usr/src/sys/vm/vm_radix.h (working copy) @@ -36,7 +36,7 @@ #ifdef _KERNEL void vm_radix_init(void); -void vm_radix_insert(struct vm_radix *rtree, vm_page_t page); +int vm_radix_insert(struct vm_radix *rtree, vm_page_t page); vm_page_t vm_radix_lookup(struct vm_radix *rtree, vm_pindex_t index); vm_page_t vm_radix_lookup_ge(struct vm_radix *rtree, vm_pindex_t index); vm_page_t vm_radix_lookup_le(struct vm_radix *rtree, vm_pindex_t index); Index: /usr/src/sys/vm/vm_page.c =================================================================== --- /usr/src/sys/vm/vm_page.c (revision 252443) +++ /usr/src/sys/vm/vm_page.c (working copy) @@ -159,11 +159,15 @@ SYSCTL_INT(_vm, OID_AUTO, tryrelock_restart, CTLFL static uma_zone_t fakepg_zone; static struct vnode *vm_page_alloc_init(vm_page_t m); +static void vm_page_alloc_turn_free(vm_page_t m); +static void vm_page_cache_turn_free(vm_page_t m); static void vm_page_clear_dirty_mask(vm_page_t m, vm_page_bits_t pagebits); static void vm_page_enqueue(int queue, vm_page_t m); static void vm_page_init_fakepg(void *dummy); -static void vm_page_insert_after(vm_page_t m, vm_object_t object, +static int vm_page_insert_after(vm_page_t m, vm_object_t object, vm_pindex_t pindex, vm_page_t mpred); +static void vm_page_insert_radixdone(vm_page_t m, vm_object_t object, + vm_page_t mpred); SYSINIT(vm_page, SI_SUB_VM, SI_ORDER_SECOND, vm_page_init_fakepg, NULL); @@ -804,14 +808,14 @@ vm_page_dirty_KBI(vm_page_t m) * * The object must be locked. */ -void +int vm_page_insert(vm_page_t m, vm_object_t object, vm_pindex_t pindex) { vm_page_t mpred; VM_OBJECT_ASSERT_WLOCKED(object); mpred = vm_radix_lookup_le(&object->rtree, pindex); - vm_page_insert_after(m, object, pindex, mpred); + return (vm_page_insert_after(m, object, pindex, mpred)); } /* @@ -824,10 +828,12 @@ vm_page_insert(vm_page_t m, vm_object_t object, vm * * The object must be locked. */ -static void +static int vm_page_insert_after(vm_page_t m, vm_object_t object, vm_pindex_t pindex, vm_page_t mpred) { + vm_pindex_t sidx; + vm_object_t sobj; vm_page_t msucc; VM_OBJECT_ASSERT_WLOCKED(object); @@ -844,22 +850,58 @@ vm_page_insert_after(vm_page_t m, vm_object_t obje msucc = TAILQ_FIRST(&object->memq); if (msucc != NULL) KASSERT(msucc->pindex > pindex, - ("vm_page_insert_after: msucc doesn't succeed pindex")); + ("vm_page_insert_after: msucc %lx doesn't succeed pindex %lx")); /* * Record the object/offset pair in this page */ + sobj = m->object; + sidx = m->pindex; m->object = object; m->pindex = pindex; /* * Now link into the object's ordered list of backed pages. */ + if (vm_radix_insert(&object->rtree, m)) { + m->object = sobj; + m->pindex = sidx; + return (1); + } + vm_page_insert_radixdone(m, object, mpred); + return (0); +} + +/* + * vm_page_insert_radixdone: + * + * Complete page "m" insertion into the specified object after the + * radix trie hooking. + * + * The page "mpred" must precede the offset "m->pindex" within the + * specified object. + * + * The object must be locked. + */ +static void +vm_page_insert_radixdone(vm_page_t m, vm_object_t object, vm_page_t mpred) +{ + + VM_OBJECT_ASSERT_WLOCKED(object); + KASSERT(object != NULL && m->object == object, + ("vm_page_insert_radixdone: page %p has inconsistent object", m)); + if (mpred != NULL) { + KASSERT(mpred->object == object || + (mpred->flags & PG_SLAB) != 0, + ("vm_page_insert_after: object doesn't contain mpred")); + KASSERT(mpred->pindex < m->pindex, + ("vm_page_insert_after: mpred doesn't precede pindex")); + } + if (mpred != NULL) TAILQ_INSERT_AFTER(&object->memq, mpred, m, listq); else TAILQ_INSERT_HEAD(&object->memq, m, listq); - vm_radix_insert(&object->rtree, m); /* * Show that the object has one more resident page. @@ -1013,18 +1055,97 @@ vm_page_prev(vm_page_t m) * or vm_page_dirty() will panic. Dirty pages are not allowed * on the cache. * - * The objects must be locked. The page must be locked if it is managed. + * The objects must be locked. */ -void +int vm_page_rename(vm_page_t m, vm_object_t new_object, vm_pindex_t new_pindex) { + vm_page_t mpred; + vm_pindex_t opidx; + VM_OBJECT_ASSERT_WLOCKED(new_object); + + mpred = vm_radix_lookup_le(&new_object->rtree, new_pindex); + KASSERT(mpred == NULL || mpred->pindex != new_pindex, + ("vm_page_rename: pindex already renamed")); + + /* + * Create a custom version of vm_page_insert() which does not depend + * by m_prev and can cheat on the implementation aspects of the + * function. + */ + opidx = m->pindex; + m->pindex = new_pindex; + if (vm_radix_insert(&new_object->rtree, m)) { + m->pindex = opidx; + return (1); + } + + /* + * The operation cannot fail anymore. The removal must happen before + * the listq iterator is tainted. + */ + m->pindex = opidx; + vm_page_lock(m); vm_page_remove(m); - vm_page_insert(m, new_object, new_pindex); + vm_page_unlock(m); + + /* Return back to the new pindex to complete vm_page_insert(). */ + m->pindex = new_pindex; + m->object = new_object; + vm_page_insert_radixdone(m, new_object, mpred); vm_page_dirty(m); + return (0); } /* + * Turn a cached page into a free page, by changing its attributes. + * Keep the statistics up-to-date. + * + * The free page queue must be locked. + */ +static void +vm_page_cache_turn_free(vm_page_t m) +{ + + mtx_assert(&vm_page_queue_free_mtx, MA_OWNED); + + m->object = NULL; + m->valid = 0; + /* Clear PG_CACHED and set PG_FREE. */ + m->flags ^= PG_CACHED | PG_FREE; + KASSERT((m->flags & (PG_CACHED | PG_FREE)) == PG_FREE, + ("vm_page_cache_free: page %p has inconsistent flags", m)); + cnt.v_cache_count--; + cnt.v_free_count++; +} + +/* + * Turn a partially allocated page into a free page, by changing its + * attributes. + * Keep the statistics up-to-date. + * + * The free page queue must be locked. + */ +static void +vm_page_alloc_turn_free(vm_page_t m) +{ + + mtx_assert(&vm_page_queue_free_mtx, MA_OWNED); + + m->object = NULL; + m->valid = 0; + m->oflags = 0; + if (m->wire_count != 0) { + m->wire_count = 0; + atomic_subtract_int(&cnt.v_wire_count, 1); + } + m->flags &= ~PG_ZERO; + m->flags |= PG_FREE; + cnt.v_free_count++; +} + +/* * Convert all of the given object's cached pages that have a * pindex within the given range into free pages. If the value * zero is given for "end", then the range's upper bound is @@ -1047,14 +1168,7 @@ vm_page_cache_free(vm_object_t object, vm_pindex_t if (end != 0 && m->pindex >= end) break; vm_radix_remove(&object->cache, m->pindex); - m->object = NULL; - m->valid = 0; - /* Clear PG_CACHED and set PG_FREE. */ - m->flags ^= PG_CACHED | PG_FREE; - KASSERT((m->flags & (PG_CACHED | PG_FREE)) == PG_FREE, - ("vm_page_cache_free: page %p has inconsistent flags", m)); - cnt.v_cache_count--; - cnt.v_free_count++; + vm_page_cache_turn_free(m); } empty = vm_radix_is_empty(&object->cache); mtx_unlock(&vm_page_queue_free_mtx); @@ -1134,7 +1248,8 @@ vm_page_cache_transfer(vm_object_t orig_object, vm /* Update the page's object and offset. */ m->object = new_object; m->pindex -= offidxstart; - vm_radix_insert(&new_object->cache, m); + if (vm_radix_insert(&new_object->cache, m)) + vm_page_cache_turn_free(m); } mtx_unlock(&vm_page_queue_free_mtx); } @@ -1201,7 +1316,7 @@ vm_page_alloc(vm_object_t object, vm_pindex_t pind struct vnode *vp = NULL; vm_object_t m_object; vm_page_t m, mpred; - int flags, req_class; + int flags, freelock_owned, req_class; mpred = 0; /* XXX: pacify gcc */ KASSERT((object != NULL) == ((req & VM_ALLOC_NOOBJ) == 0), @@ -1222,7 +1337,15 @@ vm_page_alloc(vm_object_t object, vm_pindex_t pind KASSERT(mpred == NULL || mpred->pindex != pindex, ("vm_page_alloc: pindex already allocated")); } - mtx_lock(&vm_page_queue_free_mtx); + + /* + * The page allocation request can came from consumers which already + * hold the free page queue mutex, like vm_page_insert() in + * vm_page_cache(). + */ + freelock_owned = mtx_owned(&vm_page_queue_free_mtx); + if (!freelock_owned) + mtx_lock(&vm_page_queue_free_mtx); if (cnt.v_free_count + cnt.v_cache_count > cnt.v_free_reserved || (req_class == VM_ALLOC_SYSTEM && cnt.v_free_count + cnt.v_cache_count > cnt.v_interrupt_free_min) || @@ -1235,7 +1358,8 @@ vm_page_alloc(vm_object_t object, vm_pindex_t pind if (object != NULL && (m = vm_page_cache_lookup(object, pindex)) != NULL) { if ((req & VM_ALLOC_IFNOTCACHED) != 0) { - mtx_unlock(&vm_page_queue_free_mtx); + if (!freelock_owned) + mtx_unlock(&vm_page_queue_free_mtx); return (NULL); } if (vm_phys_unfree_page(m)) @@ -1248,7 +1372,8 @@ vm_page_alloc(vm_object_t object, vm_pindex_t pind panic("vm_page_alloc: cache page %p is missing" " from the free queue", m); } else if ((req & VM_ALLOC_IFCACHED) != 0) { - mtx_unlock(&vm_page_queue_free_mtx); + if (!freelock_owned) + mtx_unlock(&vm_page_queue_free_mtx); return (NULL); #if VM_NRESERVLEVEL > 0 } else if (object == NULL || (object->flags & (OBJ_COLORED | @@ -1271,7 +1396,8 @@ vm_page_alloc(vm_object_t object, vm_pindex_t pind /* * Not allocatable, give up. */ - mtx_unlock(&vm_page_queue_free_mtx); + if (!freelock_owned) + mtx_unlock(&vm_page_queue_free_mtx); atomic_add_int(&vm_pageout_deficit, max((u_int)req >> VM_ALLOC_COUNT_SHIFT, 1)); pagedaemon_wakeup(); @@ -1326,7 +1452,8 @@ vm_page_alloc(vm_object_t object, vm_pindex_t pind if (req & VM_ALLOC_NODUMP) flags |= PG_NODUMP; m->flags = flags; - mtx_unlock(&vm_page_queue_free_mtx); + if (!freelock_owned) + mtx_unlock(&vm_page_queue_free_mtx); m->aflags = 0; m->oflags = object == NULL || (object->flags & OBJ_UNMANAGED) != 0 ? VPO_UNMANAGED : 0; @@ -1343,11 +1470,23 @@ vm_page_alloc(vm_object_t object, vm_pindex_t pind m->act_count = 0; if (object != NULL) { + if (vm_page_insert_after(m, object, pindex, mpred)) { + /* See the comment below about hold count. */ + if (vp != NULL) + vdrop(vp); + if (!freelock_owned) + mtx_lock(&vm_page_queue_free_mtx); + vm_page_alloc_turn_free(m); + if (!freelock_owned) + mtx_unlock(&vm_page_queue_free_mtx); + pagedaemon_wakeup(); + return (NULL); + } + /* Ignore device objects; the pager sets "memattr" for them. */ if (object->memattr != VM_MEMATTR_DEFAULT && (object->flags & OBJ_FICTITIOUS) == 0) pmap_page_set_memattr(m, object->memattr); - vm_page_insert_after(m, object, pindex, mpred); } else m->pindex = pindex; @@ -1413,7 +1552,7 @@ vm_page_alloc_contig(vm_object_t object, vm_pindex vm_paddr_t boundary, vm_memattr_t memattr) { struct vnode *drop; - vm_page_t deferred_vdrop_list, m, m_ret; + vm_page_t deferred_vdrop_list, m, m_tmp, m_ret; u_int flags, oflags; int req_class; @@ -1507,12 +1646,30 @@ retry: m->wire_count = 1; /* Unmanaged pages don't use "act_count". */ m->oflags = oflags; + if (object != NULL) { + if (vm_page_insert(m, object, pindex)) { + for (m_tmp = m_ret; m_tmp < m; m_tmp++) { + vm_page_lock(m_tmp); + vm_page_remove(m_tmp); + vm_page_unlock(m_tmp); + } + while (deferred_vdrop_list != NULL) { + vdrop((struct vnode *)deferred_vdrop_list->pageq.tqe_prev); + deferred_vdrop_list = + deferred_vdrop_list->pageq.tqe_next; + } + mtx_lock(&vm_page_queue_free_mtx); + for (m = m_ret; m < &m_ret[npages]; m++) + vm_page_alloc_turn_free(m); + mtx_unlock(&vm_page_queue_free_mtx); + if (vm_paging_needed()) + pagedaemon_wakeup(); + return (NULL); + } + } else + m->pindex = pindex; if (memattr != VM_MEMATTR_DEFAULT) pmap_page_set_memattr(m, memattr); - if (object != NULL) - vm_page_insert(m, object, pindex); - else - m->pindex = pindex; pindex++; } while (deferred_vdrop_list != NULL) { @@ -2183,7 +2340,6 @@ vm_page_cache(vm_page_t m) } KASSERT((m->flags & PG_CACHED) == 0, ("vm_page_cache: page %p is already cached", m)); - PCPU_INC(cnt.v_tcached); /* * Remove the page from the paging queues. @@ -2210,10 +2366,17 @@ vm_page_cache(vm_page_t m) */ m->flags &= ~PG_ZERO; mtx_lock(&vm_page_queue_free_mtx); + cache_was_empty = vm_radix_is_empty(&object->cache); + if (vm_radix_insert(&object->cache, m)) { + mtx_unlock(&vm_page_queue_free_mtx); + vm_page_free(m); + if (object->resident_page_count == 0) + vdrop(object->handle); + return; + } m->flags |= PG_CACHED; cnt.v_cache_count++; - cache_was_empty = vm_radix_is_empty(&object->cache); - vm_radix_insert(&object->cache, m); + PCPU_INC(cnt.v_tcached); #if VM_NRESERVLEVEL > 0 if (!vm_reserv_free_page(m)) { #else @@ -2807,7 +2970,13 @@ vm_page_cowfault(vm_page_t m) vm_page_lock(mnew); vm_page_free(mnew); vm_page_unlock(mnew); - vm_page_insert(m, object, pindex); + for (;;) { + if (!vm_page_insert(m, object, pindex)) + break; + VM_OBJECT_WUNLOCK(object); + VM_WAIT; + VM_OBJECT_WLOCK(object); + } } else { /* clear COW & copy page */ if (!so_zerocp_fullpage) pmap_copy_page(m, mnew); Index: /usr/src/sys/vm/vm_page.h =================================================================== --- /usr/src/sys/vm/vm_page.h (revision 252443) +++ /usr/src/sys/vm/vm_page.h (working copy) @@ -388,7 +388,7 @@ void vm_page_dequeue_locked(vm_page_t m); vm_page_t vm_page_find_least(vm_object_t, vm_pindex_t); vm_page_t vm_page_getfake(vm_paddr_t paddr, vm_memattr_t memattr); void vm_page_initfake(vm_page_t m, vm_paddr_t paddr, vm_memattr_t memattr); -void vm_page_insert (vm_page_t, vm_object_t, vm_pindex_t); +int vm_page_insert (vm_page_t, vm_object_t, vm_pindex_t); boolean_t vm_page_is_cached(vm_object_t object, vm_pindex_t pindex); vm_page_t vm_page_lookup (vm_object_t, vm_pindex_t); vm_page_t vm_page_next(vm_page_t m); @@ -398,7 +398,7 @@ void vm_page_putfake(vm_page_t m); void vm_page_readahead_finish(vm_page_t m); void vm_page_reference(vm_page_t m); void vm_page_remove (vm_page_t); -void vm_page_rename (vm_page_t, vm_object_t, vm_pindex_t); +int vm_page_rename (vm_page_t, vm_object_t, vm_pindex_t); void vm_page_requeue(vm_page_t m); void vm_page_requeue_locked(vm_page_t m); void vm_page_set_valid_range(vm_page_t m, int base, int size); Index: /usr/src/sys/vm/sg_pager.c =================================================================== --- /usr/src/sys/vm/sg_pager.c (revision 252443) +++ /usr/src/sys/vm/sg_pager.c (working copy) @@ -42,6 +42,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -181,16 +182,21 @@ sg_pager_getpages(vm_object_t object, vm_page_t *m /* Construct a new fake page. */ page = vm_page_getfake(paddr, memattr); - VM_OBJECT_WLOCK(object); + for (;;) { + VM_OBJECT_WLOCK(object); + if (!vm_page_insert(page, object, offset)) + break; + VM_OBJECT_WUNLOCK(object); + VM_WAIT; + } TAILQ_INSERT_TAIL(&object->un_pager.sgp.sgp_pglist, page, pageq); - /* Free the original pages and insert this fake page into the object. */ + /* Free the original pages. */ for (i = 0; i < count; i++) { vm_page_lock(m[i]); vm_page_free(m[i]); vm_page_unlock(m[i]); } - vm_page_insert(page, object, offset); m[reqpage] = page; page->valid = VM_PAGE_BITS_ALL; Index: /usr/src/sys/vm/vm_object.c =================================================================== --- /usr/src/sys/vm/vm_object.c (revision 252443) +++ /usr/src/sys/vm/vm_object.c (working copy) @@ -1351,6 +1351,16 @@ retry: VM_OBJECT_WLOCK(new_object); goto retry; } + + /* vm_page_rename() will handle dirty and cache. */ + if (vm_page_rename(m, new_object, idx)) { + VM_OBJECT_WUNLOCK(new_object); + VM_OBJECT_WUNLOCK(orig_object); + VM_WAIT; + VM_OBJECT_WLOCK(orig_object); + VM_OBJECT_WLOCK(new_object); + goto retry; + } #if VM_NRESERVLEVEL > 0 /* * If some of the reservation's allocated pages remain with @@ -1366,10 +1376,6 @@ retry: */ vm_reserv_rename(m, new_object, orig_object, offidxstart); #endif - vm_page_lock(m); - vm_page_rename(m, new_object, idx); - vm_page_unlock(m); - /* page automatically made dirty by rename and cache handled */ if (orig_object->type == OBJT_SWAP) vm_page_busy(m); } @@ -1527,21 +1533,14 @@ vm_object_backing_scan(vm_object_t object, int op) ("vm_object_backing_scan: object mismatch") ); - /* - * Destroy any associated swap - */ - if (backing_object->type == OBJT_SWAP) { - swap_pager_freespace( - backing_object, - p->pindex, - 1 - ); - } - if ( p->pindex < backing_offset_index || new_pindex >= object->size ) { + if (backing_object->type == OBJT_SWAP) + swap_pager_freespace(backing_object, + p->pindex, 1); + /* * Page is out of the parent object's range, we * can simply destroy it. @@ -1563,6 +1562,10 @@ vm_object_backing_scan(vm_object_t object, int op) (op & OBSC_COLLAPSE_NOWAIT) != 0 && (pp != NULL && pp->valid == 0) ) { + if (backing_object->type == OBJT_SWAP) + swap_pager_freespace(backing_object, + p->pindex, 1); + /* * The page in the parent is not (yet) valid. * We don't know anything about the state of @@ -1581,6 +1584,10 @@ vm_object_backing_scan(vm_object_t object, int op) pp != NULL || vm_pager_has_page(object, new_pindex, NULL, NULL) ) { + if (backing_object->type == OBJT_SWAP) + swap_pager_freespace(backing_object, + p->pindex, 1); + /* * page already exists in parent OR swap exists * for this location in the parent. Destroy @@ -1600,6 +1607,31 @@ vm_object_backing_scan(vm_object_t object, int op) continue; } + /* + * Page does not exist in parent, rename the + * page from the backing object to the main object. + * + * If the page was mapped to a process, it can remain + * mapped through the rename. + * vm_page_rename() will handle dirty and cache. + */ + if (vm_page_rename(p, object, new_pindex)) { + if (op & OBSC_COLLAPSE_NOWAIT) { + p = next; + continue; + } + VM_OBJECT_WLOCK(backing_object); + VM_OBJECT_WUNLOCK(object); + VM_WAIT; + VM_OBJECT_WLOCK(object); + VM_OBJECT_WLOCK(backing_object); + p = TAILQ_FIRST(&backing_object->memq); + continue; + } + if (backing_object->type == OBJT_SWAP) + swap_pager_freespace(backing_object, p->pindex, + 1); + #if VM_NRESERVLEVEL > 0 /* * Rename the reservation. @@ -1607,18 +1639,6 @@ vm_object_backing_scan(vm_object_t object, int op) vm_reserv_rename(p, object, backing_object, backing_offset_index); #endif - - /* - * Page does not exist in parent, rename the - * page from the backing object to the main object. - * - * If the page was mapped to a process, it can remain - * mapped through the rename. - */ - vm_page_lock(p); - vm_page_rename(p, object, new_pindex); - vm_page_unlock(p); - /* page automatically made dirty by rename */ } p = next; } Index: /usr/src/sys/vm/vm_fault.c =================================================================== --- /usr/src/sys/vm/vm_fault.c (revision 252443) +++ /usr/src/sys/vm/vm_fault.c (working copy) @@ -741,9 +741,11 @@ vnode_locked: * process'es object. The page is * automatically made dirty. */ - vm_page_lock(fs.m); - vm_page_rename(fs.m, fs.first_object, fs.first_pindex); - vm_page_unlock(fs.m); + if (vm_page_rename(fs.m, fs.first_object, + fs.first_pindex)) { + unlock_and_deallocate(&fs); + goto RetryFault; + } vm_page_busy(fs.m); fs.first_m = fs.m; fs.m = NULL; Index: /usr/src/sys/vm/device_pager.c =================================================================== --- /usr/src/sys/vm/device_pager.c (revision 252443) +++ /usr/src/sys/vm/device_pager.c (working copy) @@ -51,6 +51,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -347,12 +348,17 @@ old_dev_pager_fault(vm_object_t object, vm_ooffset * free up the all of the original pages. */ page = vm_page_getfake(paddr, memattr); - VM_OBJECT_WLOCK(object); + for (;;) { + VM_OBJECT_WLOCK(object); + if (!vm_page_insert(page, object, pidx)) + break; + VM_OBJECT_WUNLOCK(object); + VM_WAIT; + } vm_page_lock(*mres); vm_page_free(*mres); vm_page_unlock(*mres); *mres = page; - vm_page_insert(page, object, pidx); } page->valid = VM_PAGE_BITS_ALL; return (VM_PAGER_OK); Index: /usr/src/sys/i386/i386/pmap.c =================================================================== --- /usr/src/sys/i386/i386/pmap.c (revision 252443) +++ /usr/src/sys/i386/i386/pmap.c (working copy) @@ -304,7 +304,7 @@ static boolean_t pmap_enter_pde(pmap_t pmap, vm_of static vm_page_t pmap_enter_quick_locked(pmap_t pmap, vm_offset_t va, vm_page_t m, vm_prot_t prot, vm_page_t mpte); static void pmap_flush_page(vm_page_t m); -static void pmap_insert_pt_page(pmap_t pmap, vm_page_t mpte); +static int pmap_insert_pt_page(pmap_t pmap, vm_page_t mpte); static void pmap_fill_ptp(pt_entry_t *firstpte, pt_entry_t newpte); static boolean_t pmap_is_modified_pvh(struct md_page *pvh); static boolean_t pmap_is_referenced_pvh(struct md_page *pvh); @@ -1604,12 +1604,12 @@ pmap_add_delayed_free_list(vm_page_t m, vm_page_t * for mapping a distinct range of virtual addresses. The pmap's collection is * ordered by this virtual address range. */ -static __inline void +static __inline int pmap_insert_pt_page(pmap_t pmap, vm_page_t mpte) { PMAP_LOCK_ASSERT(pmap, MA_OWNED); - vm_radix_insert(&pmap->pm_root, mpte); + return (vm_radix_insert(&pmap->pm_root, mpte)); } /* @@ -3364,7 +3364,13 @@ setpte: ("pmap_promote_pde: page table page is out of range")); KASSERT(mpte->pindex == va >> PDRSHIFT, ("pmap_promote_pde: page table page's pindex is wrong")); - pmap_insert_pt_page(pmap, mpte); + if (pmap_insert_pt_page(pmap, mpte)) { + pmap_pde_p_failures++; + CTR2(KTR_PMAP, + "pmap_promote_pde: failure for va %#x in pmap %p", va, + pmap); + return; + } /* * Promote the pv entries. Index: /usr/src/sys/amd64/amd64/pmap.c =================================================================== --- /usr/src/sys/amd64/amd64/pmap.c (revision 252443) +++ /usr/src/sys/amd64/amd64/pmap.c (working copy) @@ -283,7 +283,7 @@ static boolean_t pmap_enter_pde(pmap_t pmap, vm_of static vm_page_t pmap_enter_quick_locked(pmap_t pmap, vm_offset_t va, vm_page_t m, vm_prot_t prot, vm_page_t mpte, struct rwlock **lockp); static void pmap_fill_ptp(pt_entry_t *firstpte, pt_entry_t newpte); -static void pmap_insert_pt_page(pmap_t pmap, vm_page_t mpte); +static int pmap_insert_pt_page(pmap_t pmap, vm_page_t mpte); static boolean_t pmap_is_modified_pvh(struct md_page *pvh); static boolean_t pmap_is_referenced_pvh(struct md_page *pvh); static void pmap_kenter_attr(vm_offset_t va, vm_paddr_t pa, int mode); @@ -1525,12 +1525,12 @@ pmap_add_delayed_free_list(vm_page_t m, vm_page_t * for mapping a distinct range of virtual addresses. The pmap's collection is * ordered by this virtual address range. */ -static __inline void +static __inline int pmap_insert_pt_page(pmap_t pmap, vm_page_t mpte) { PMAP_LOCK_ASSERT(pmap, MA_OWNED); - vm_radix_insert(&pmap->pm_root, mpte); + return (vm_radix_insert(&pmap->pm_root, mpte)); } /* @@ -3392,7 +3392,13 @@ setpte: ("pmap_promote_pde: page table page is out of range")); KASSERT(mpte->pindex == pmap_pde_pindex(va), ("pmap_promote_pde: page table page's pindex is wrong")); - pmap_insert_pt_page(pmap, mpte); + if (pmap_insert_pt_page(pmap, mpte)) { + atomic_add_long(&pmap_pde_p_failures, 1); + CTR2(KTR_PMAP, + "pmap_promote_pde: failure for va %#lx in pmap %p", va, + pmap); + return; + } /* * Promote the pv entries. Index: /usr/src/sys/kern/subr_uio.c =================================================================== --- /usr/src/sys/kern/subr_uio.c (revision 252443) +++ /usr/src/sys/kern/subr_uio.c (working copy) @@ -56,6 +56,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #ifdef SOCKET_SEND_COW #include @@ -122,7 +123,12 @@ retry: if (uobject->backing_object != NULL) pmap_remove(map->pmap, uaddr, uaddr + PAGE_SIZE); } - vm_page_insert(kern_pg, uobject, upindex); + if (vm_page_insert(kern_pg, uobject, upindex)) { + VM_OBJECT_WUNLOCK(uobject); + VM_WAIT; + VM_OBJECT_WLOCK(uobject); + goto retry; + } vm_page_dirty(kern_pg); VM_OBJECT_WUNLOCK(uobject); vm_map_lookup_done(map, entry); Index: /usr/src/sys/dev/drm2/i915/i915_gem.c =================================================================== --- /usr/src/sys/dev/drm2/i915/i915_gem.c (revision 252443) +++ /usr/src/sys/dev/drm2/i915/i915_gem.c (working copy) @@ -64,6 +64,9 @@ __FBSDID("$FreeBSD$"); #include #include +#include +#include + static void i915_gem_object_flush_cpu_write_domain( struct drm_i915_gem_object *obj); static uint32_t i915_gem_get_gtt_size(struct drm_device *dev, uint32_t size, @@ -1440,8 +1443,14 @@ unlocked_vmobj: vm_page_sleep(m, "915pbs"); goto retry; } + if (vm_page_insert(m, vm_obj, OFF_TO_IDX(offset))) { + DRM_UNLOCK(dev); + VM_OBJECT_WUNLOCK(vm_obj); + VM_WAIT; + VM_OBJECT_WLOCK(vm_obj); + goto retry; + } m->valid = VM_PAGE_BITS_ALL; - vm_page_insert(m, vm_obj, OFF_TO_IDX(offset)); have_page: *mres = m; vm_page_busy(m); Index: /usr/src/sys/dev/drm2/ttm/ttm_bo_vm.c =================================================================== --- /usr/src/sys/dev/drm2/ttm/ttm_bo_vm.c (revision 252443) +++ /usr/src/sys/dev/drm2/ttm/ttm_bo_vm.c (working copy) @@ -47,6 +47,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #define TTM_BO_VM_NUM_PREFAULT 16 @@ -218,11 +219,16 @@ reserve: ttm_bo_unreserve(bo); goto retry; } + if (vm_page_insert(m, vm_obj, OFF_TO_IDX(offset))) { + VM_OBJECT_WUNLOCK(vm_obj); + VM_WAIT; + VM_OBJECT_WLOCK(vm_obj); + ttm_mem_io_unlock(man); + ttm_bo_unreserve(bo); + goto retry; + } m->valid = VM_PAGE_BITS_ALL; *mres = m; - vm_page_lock(m); - vm_page_insert(m, vm_obj, OFF_TO_IDX(offset)); - vm_page_unlock(m); vm_page_busy(m); if (oldm != NULL) {