GENERIC from Sun Sep 2 19:02:18 2012 +0300, r239898+softdep_insmntque 5115806, vmcore.437 KDB: debugger backends: ddb KDB: current backend: ddb 524288K of memory above 4GB ignored Copyright (c) 1992-2012 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #0 r239898+5115806: Sun Sep 2 18:43:51 CEST 2012 pho@x4.osted.lan:/var/tmp/deviant2/sys/i386/compile/PHO i386 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. CPU: AMD Phenom(tm) 9150e Quad-Core Processor (1800.02-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x100f23 Family = 10 Model = 2 Stepping = 3 Features=0x178bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x7ff TSC: P-state invariant real memory = 4294967296 (4096 MB) avail memory = 3531583488 (3367 MB) : Trying to mount root from ufs:/dev/ad4s1a [rw]... WARNING: / was not properly dismounted Setting hostuuid: 00000000-0000-0000-0000-00218515337d. Setting hostid: 0x6b64ac17. Starting ddb. Entropy harvesting: interrupts ethernet point_to_point kickstart. warning: total configured swap (4194304 pages) exceeds maximum recommended amount (3890432 pages). warning: increase kern.maxswzone or reduce amount of swap. Starting file system checks: /dev/ad4s1a: 7561 files, 408788 used, 604227 free (1411 frags, 75352 blocks, 0.1% fragmentation) /dev/ad4s1f: 23307 files, 870975 used, 142040 free (4992 frags, 17131 blocks, 0.5% fragmentation) /dev/label/tmp: LINK COUNT DIR I=5511174 OWNER=root MODE=40770 /dev/label/tmp: SIZE=28672 MTIME=Sep 2 19:56 2012 COUNT 0 SHOULD BE 1 /dev/label/tmp: LINK COUNT INCREASING /dev/label/tmp: UNEXPECTED SOFT UPDATE INCONSISTENCY; RUN fsck MANUALLY. /dev/ad4s1d: 613178 files, 6542472 used, 3611727 free (243255 frags, 421059 blocks, 2.4% fragmentation) /dev/ad4s1e: FREE BLK COUNT(S) WRONG IN SUPERBLK (SALVAGED) /dev/ad4s1e: SUMMARY INFORMATION BAD (SALVAGED) /dev/ad4s1e: BLK(S) MISSING IN BIT MAPS (SALVAGED) /dev/ad4s1e: 285891 files, 8200293 used, 17188224 free (54832 frags, 2141674 blocks, 0.2% fragmentation) THE FOLLOWING FILE SYSTEM HAD AN UNEXPECTED INCONSISTENCY: ufs: /dev/label/tmp (/tmp) File system preen failed, trying fsck -y ** /dev/ad4s1a ** Last Mounted on / ** Root file system ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 7561 files, 408788 used, 604227 free (1411 frags, 75352 blocks, 0.1% fragmentation) ***** FILE SYSTEM IS CLEAN ***** ** /dev/ad4s1f ** Last Mounted on /home ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 23307 files, 870975 used, 142040 free (4992 frags, 17131 blocks, 0.5% fragmentation) ***** FILE SYSTEM IS CLEAN ***** ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity UNREF DIR I=27532315 OWNER=root MODE=40770 SIZE=512 MTIME=Sep 2 19:57 2012 RECONNECT? yes DIR I=27532315 CONNECTED. PARENT WAS I=27532314 UNREF DIR I=27532314 OWNER=root MODE=40770 SIZE=512 MTIME=Sep 2 19:57 2012 RECONNECT? yes DIR I=27532314 CONNECTED. PARENT WAS I=27532313 UNREF DIR I=27532313 OWNER=root MODE=40770 SIZE=512 MTIME=Sep 2 19:57 2012 RECONNECT? yes DIR I=27532313 CONNECTED. PARENT WAS I=27439138 UNREF DIR I=27532309 OWNER=root MODE=40770 SIZE=512 MTIME=Sep 2 19:57 2012 RECONNECT? yes DIR I=27532309 CONNECTED. PARENT WAS I=27439124 UNREF DIR I=27532308 OWNER=root MODE=40770 SIZE=512 MTIME=Sep 2 19:57 2012 RECONNECT? yes DIR I=27532308 CONNECTED. PARENT WAS I=27532307 : : ADJUST? yes LINK COUNT DIR I=27532313 OWNER=root MODE=40770 SIZE=512 MTIME=Sep 2 19:57 2012 COUNT 0 SHOULD BE 2 LINK COUNT INCREASING UNEXPECTED SOFT UPDATE INCONSISTENCY ADJUST? yes LINK COUNT DIR I=27532314 OWNER=root MODE=40770 SIZE=512 MTIME=Sep 2 19:57 2012 COUNT 0 SHOULD BE 2 LINK COUNT INCREASING UNEXPECTED SOFT UPDATE INCONSISTENCY ADJUST? yes LINK COUNT DIR I=27532315 OWNER=root MODE=40770 SIZE=512 MTIME=Sep 2 19:57 2012 COUNT 0 SHOULD BE 2 LINK COUNT INCREASING UNEXPECTED SOFT UPDATE INCONSISTENCY ADJUST? yes ** Phase 5 - Check Cyl groups FREE BLK COUNT(S) WRONG IN SUPERBLK SALVAGE? yes SUMMARY INFORMATION BAD SALVAGE? yes BLK(S) MISSING IN BIT MAPS SALVAGE? yes 3546 files, 56581619 used, 61732362 free (5530 frags, 7715854 blocks, 0.0% fragmentation) ***** FILE SYSTEM MARKED CLEAN ***** ***** FILE SYSTEM WAS MODIFIED ***** ** /dev/ad4s1d ** Last Mounted on /usr ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 613178 files, 6542472 used, 3611727 free (243255 frags, 421059 blocks, 2.4% fragmentation) ***** FILE SYSTEM IS CLEAN ***** ** /dev/ad4s1e ** Last Mounted on /var ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 285891 files, 8200293 used, 17188224 free (54832 frags, 2141674 blocks, 0.2% fragmentation) ***** FILE SYSTEM IS CLEAN ***** Mounting local file systems:. Setting hostname: x4.osted.lan. re0: link state changed to DOWN re0: link state changed to UP Starting Network: lo0 re0 fwe0 fwip0. lo0: flags=8049 metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 inet 127.0.0.1 netmask 0xff000000 nd6 options=21 re0: flags=8843 metric 0 mtu 1500 options=8209b ether 00:21:85:15:33:7d inet 192.168.1.101 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::221:85ff:fe15:337d%re0 prefixlen 64 scopeid 0x1 nd6 options=29 media: Ethernet autoselect (100baseTX ) status: active fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb nd6 options=29 ch 1 dma -1 fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 nd6 options=29 Starting devd. Starting Network: fwe0. fwe0: flags=8802 metric 0 mtu 1500 options=8 ether 02:dc:10:62:ad:eb nd6 options=29 ch 1 dma -1 Starting Network: fwip0. fwip0: flags=8802 metric 0 mtu 1500 lladdr 0.dc.10.0.1.62.ad.eb.a.2.ff.fe.0.0.0.0 nd6 options=29 add net default: gateway 192.168.1.1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/kde4/lib /usr/local/lib/compat /usr/local/lib/compat/pkg /usr/local/lib/gcc46 /usr/local/lib/mysql /usr/local/lib/nss /usr/local/lib/qt4 /usr/local/lib/virtualbox a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout Creating and/or trimming log files. Starting syslogd. No core dumps found. Additional ABI support: linux. Starting rpcbind. NFS access cache time=60 lock order reversal: 1st 0xe191bac0 bufwait (bufwait) @ kern/vfs_bio.c:2652 2nd 0xcac3f000 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:284 KDB: stack backtrace: db_trace_self_wrapper(c0ff6c17,2e687361,38323a63,c1000a34,ca6e70b4,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0adae4b,c0ffa70c,c12fac60,11c,f27ae844,...) at kdb_backtrace+0x2a _witness_debugger(c0ffa70c,cac3f000,c102c41a,c7d86818,c102c0ac,...) at _witness_debugger+0x25 witness_checkorder(cac3f000,9,c102c0a3,11c,0,...) at witness_checkorder+0x86f _sx_xlock(cac3f000,0,c102c0a3,11c,ca874570,...) at _sx_xlock+0x9a ufsdirhash_acquire(e191ba60,ca874570,3c,e238cec4,f27ae914,...) at ufsdirhash_acquire+0x48 ufsdirhash_add(ca874570,f27ae9c0,ec4,f27ae900,f27ae904,...) at ufsdirhash_add+0x13 ufs_direnter(cac38e10,cae502d0,f27ae9c0,f27aeb5c,e191f680,...) at ufs_direnter+0x799 ufs_mkdir(f27aebec,c104e3e7,c116cfa0,cac38e10,f27aec00,...) at ufs_mkdir+0xa6f VOP_MKDIR_APV(c115abc0,f27aebec,f27aeb5c,f27aeb84,0,...) at VOP_MKDIR_APV+0x100 kern_mkdirat(ca6e7000,ffffff9c,28803020,0,1c0,...) at kern_mkdirat+0x291 kern_mkdir(ca6e7000,28803020,0,1c0,f27aecfc,...) at kern_mkdir+0x2e sys_mkdir(ca6e7000,f27aeccc,c1046c7c,c0ffb45b,f27aecc4,...) at sys_mkdir+0x29 syscall(f27aed08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (136, FreeBSD ELF32, sys_mkdir), eip = 0x28169c53, esp = 0xbfbfd8fc, ebp = 0xbfbfdda8 --- Clearing /tmp (X related). Starting mountd. Starting nfsuserd. Starting nfsd. Recovering vi editor sessions:lock order reversal: 1st 0xcafcd1c0 ufs (ufs) @ kern/vfs_lookup.c:542 2nd 0xe19241a0 bufwait (bufwait) @ ufs/ffs/ffs_vnops.c:261 3rd 0xcafcce68 ufs (ufs) @ kern/vfs_subr.c:2159 KDB: stack backtrace: db_trace_self_wrapper(c0ff6c17,3a632e73,a313632,0,c62bd200,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0adae4b,c0ffa725,c12fa878,86f,f2732354,...) at kdb_backtrace+0x2a _witness_debugger(c0ffa725,cafcce68,c0fe28cb,c7d867b0,c1002881,...) at _witness_debugger+0x25 witness_checkorder(cafcce68,9,c1002878,86f,0,...) at witness_checkorder+0x86f __lockmgr_args(cafcce68,80100,cafcced4,0,0,...) at __lockmgr_args+0x8d5 ffs_lock(f273245c,c0aed5ec,c0fffc27,5e2,c0ff09b7,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c115abc0,f273245c,ca88fc74,c116cd20,cafcce10,...) at VOP_LOCK1_APV+0xf3 _vn_lock(cafcce10,80100,c1002878,86f,eb,...) at _vn_lock+0x78 vget(cafcce10,80100,ca88fbc0,50,0,...) at vget+0xbb vfs_hash_get(ca7b7bb0,15edc5,80000,ca88fbc0,f27325a8,...) at vfs_hash_get+0xed ffs_vgetf(ca7b7bb0,15edc5,80000,f27325a8,1,...) at ffs_vgetf+0x49 softdep_sync_buf(cafcd168,e1924140,1,107,0,...) at softdep_sync_buf+0xac9 ffs_syncvnode(cafcd168,1,0,3,0,...) at ffs_syncvnode+0x24c ffs_truncate(cafcd168,200,0,880,c952a500,...) at ffs_truncate+0x8b5 ufs_direnter(cafcd168,cafcce10,f273291c,f2732bc0,0,...) at ufs_direnter+0x980 ufs_makeinode(f2732bc0,c115b180,f2732b14,f2732a70,c0e44eb0,...) at ufs_makeinode+0x65d ufs_create(f2732b14,c104e5ae,c116cfa0,cafcd168,f2732b28,...) at ufs_create+0x30 VOP_CREATE_APV(c115abc0,f2732b14,f2732bc0,f2732aac,0,...) at VOP_CREATE_APV+0x100 vn_open_cred(f2732b80,f2732c20,1b0,0,c952a500,...) at vn_open_cred+0x20a vn_open(f2732b80,f2732c20,1b0,ca8bda10,0,...) at vn_open+0x3b kern_openat(ca88fbc0,ffffff9c,28c72f60,0,a03,...) at kern_openat+0x1dd kern_open(ca88fbc0,28c72f60,0,a02,1b0,...) at kern_open+0x35 sys_open(ca88fbc0,f2732ccc,c1046c7c,c0ffb45b,f2732cc4,...) at sys_open+0x30 syscall(f2732d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (5, FreeBSD ELF32, sys_open), eip = 0x283e1443, esp = 0xbfbfb52c, ebp = 0xbfbfb5b8 --- . Updating motd:. Starting ntpd. Starting default moused. Configuring syscons: keymap blanktime. Starting sshd. Starting cron. Local package initialization: backuplock order reversal: 1st 0xca8c7490 ufs (ufs) @ kern/vfs_mount.c:1256 2nd 0xca8c7a30 devfs (devfs) @ ufs/ffs/ffs_softdep.c:1834 KDB: stack backtrace: db_trace_self_wrapper(c0ff6c17,38313a63,a3433,cb092058,8,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0adae4b,c0ffa70c,c12fa4a0,72a,f2736988,...) at kdb_backtrace+0x2a _witness_debugger(c0ffa70c,ca8c7a30,c0fed854,c7d86610,c1028515,...) at _witness_debugger+0x25 witness_checkorder(ca8c7a30,9,c102850c,72a,ca8c7a9c,...) at witness_checkorder+0x86f __lockmgr_args(ca8c7a30,80400,ca8c7a9c,0,0,...) at __lockmgr_args+0x8d5 vop_stdlock(f2736a8c,c132fc40,4,c0ff09b7,eb,...) at vop_stdlock+0x65 VOP_LOCK1_APV(c111a580,f2736a8c,c132fc40,c116cd20,ca8c79d8,...) at VOP_LOCK1_APV+0xf3 _vn_lock(ca8c79d8,80400,c102850c,72a,0,...) at _vn_lock+0x78 softdep_flushworklist(cac2e2ec,f2736b44,ca6e75e0,612,cac38ed4,...) at softdep_flushworklist+0x47 ffs_sync(cac2e2ec,1,f2736b98,522,c1baf710,...) at ffs_sync+0x3bf dounmount(cac2e2ec,8080000,ca6e75e0,4a3,f2736c1c,...) at dounmount+0x400 sys_unmount(ca6e75e0,f2736ccc,c1046c7c,c0ffb41e,f2736cc4,...) at sys_unmount+0x421 syscall(f2736d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (22, FreeBSD ELF32, sys_unmount), eip = 0x280c487b, esp = 0xbfbfd67c, ebp = 0xbfbfd748 --- ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 2312 files, 56577213 used, 61736768 free (4680 frags, 7716511 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** ** /dev/label/tmp ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 2312 files, 56577213 used, 61736768 free (4680 frags, 7716511 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** fsck -y /tmp watchdogd. Starting inetd. Sun Sep 2 22:05:34 CEST 2012 FreeBSD/i386 (x4.osted.lan) (console) login: Sep 2 22:09:01 x4 su: pho to root on /dev/pts/0 Stopping inetd. Waiting for PIDS: 1243. Shutting down local packages:. Stopping cron. Stopping sshd. Waiting for PIDS: 1153. Stopping moused. Waiting for PIDS: 1121. Stopping ntpd. Waiting for PIDS: 1096. Stopping nfsd. Waiting for PIDS: 1037 1040. Stopping mountd. Waiting for PIDS: 1015. Stopping nfsuserd. Waiting for PIDS: 1031 1032 1033 1034 1035. Stopping rpcbind. Waiting for PIDS: 982. Stopping watchdogd. Waiting for PIDS: 1218. Stopping devd. Waiting for PIDS: 844. Writing entropy file:. Terminated . Sep 2 22:18:40 x4 syslogd: exiting on signal 15 Sep 2 22:19:00 init: some processes would not die; ps axl advised Enter full pathname of shell or RETURN for /bin/sh: # ps -L %cpu %mem acflag acflg args blocked caught class comm command cow cpu cputime egid egroup emul etime etimes euid f flags gid group ignored inblk inblock jid jobc ktrace label lim lockname login logname lstart lwp majflt minflt msgrcv msgsnd mwchan ni nice nivcsw nlwp nsignals nsigs nswap nvcsw nwchan oublk oublock paddr pagein pcpu pending pgid pid pmem ppid pri re rgid rgroup rss rtprio ruid ruser sid sig sigcatch sigignore sigmask sl start stat state svgid svuid systime tdaddr tdev tdnam time tpgid tsid tsiz tt tty ucomm uid upr uprocp user usertime usrpri vsize vsz wchan xstat # ps -l UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND 0 47175 1 0 20 0 10328 9912 wait Ss u0 0:00.03 -sh (sh) 0 47177 47175 0 21 0 10004 9524 - R+ u0 0:00.02 ps -l 0 1307 1 0 20 0 9904 1356 ufs D 0- 0:02.70 /tmp/kinfo 0 1308 1 0 20 0 9904 1336 ufs D 0- 0:05.52 /tmp/kinfo 0 1309 1 0 20 0 9904 1376 ufs D 0- 0:04.13 /tmp/kinfo 0 1310 1307 0 75 0 0 0 - Z 0- 5:28.17 0 1311 1 0 20 0 9904 1364 ufs D 0- 0:05.48 /tmp/kinfo 0 1312 1308 0 76 0 0 0 - Z 0- 5:29.61 0 1313 1 0 20 0 9904 1372 ufs D 0- 0:05.13 /tmp/kinfo 0 1318 1311 0 75 0 0 0 - Z 0- 5:24.79 0 1319 1309 0 75 0 0 0 - Z 0- 5:35.11 0 1320 1313 0 75 0 0 0 - Z 0- 5:25.55 0 2083 1 0 20 0 10040 1024 ufs D 0- 0:00.48 mkdir # ~KDB: enter: Break to debugger [ thread pid 11 tid 100005 ] Stopped at kdb_enter+0x3a: movl $0,kdb_why db> run pho db:0:pho> bt Tracing pid 11 tid 100005 td 0xc7de22f0 kdb_enter(c0e64b49,c0ff6bbe,c7b2bb64,c091102d,c9485400,...) at kdb_enter+0x3a kdb_break(c9485400,c7b2bb6c,c0e37c70,c0feccb3,39e,...) at kdb_break+0x28 uart_intr(c9485400,c7b2bb7c,246,c11b1e50,0,...) at uart_intr+0x12d intr_event_handle(c7d7c380,c7b2bbd0,1e69eb,2e84d682,35a,...) at intr_event_handle+0x76 intr_execute_handlers(c7e230d0,c7b2bbd0,2,c7b2bc10,c0e064a5,...) at intr_execute_handlers+0x49 lapic_handle_intr(30,c7b2bbd0) at lapic_handle_intr+0x36 Xapic_isr1() at Xapic_isr1+0x35 --- interrupt, eip = 0xc0e021b5, esp = 0xc7b2bc10, ebp = 0xc7b2bc10 --- acpi_cpu_c1(c0ff09b7,119,c7b18c80,0,c7b18c80,...) at acpi_cpu_c1+0x5 acpi_cpu_idle(0,c7b2bc7c,c0e0ecc9,0,2,...) at acpi_cpu_idle+0x11e cpu_idle_acpi(0,2,c11bf6c0,2,c7b2bcc8,...) at cpu_idle_acpi+0x2f cpu_idle(0,c7b2bca0,c0ff4bec,a39,c7de22f0,...) at cpu_idle+0x89 sched_idletd(0,c7b2bd08,c0fec990,3db,c7de0864,...) at sched_idletd+0x263 fork_exit(c0ac7db0,0,c7b2bd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b2bd40, ebp = 0 --- db:0:bt> show allpcpu Current CPU: 2 cpuid = 0 dynamic pcpu = 0x6b1700 curthread = 0xc7de28d0: pid 11 "idle: cpu0" curpcb = 0xc7b25d60 fpcurthread = none idlethread = 0xc7de28d0: tid 100003 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: cpuid = 1 dynamic pcpu = 0x69a1700 curthread = 0xc7de25e0: pid 11 "idle: cpu1" curpcb = 0xc7b28d60 fpcurthread = none idlethread = 0xc7de25e0: tid 100004 "idle: cpu1" APIC ID = 1 currentldt = 0x50 spin locks held: cpuid = 2 dynamic pcpu = 0x69a4700 curthread = 0xc7de22f0: pid 11 "idle: cpu2" curpcb = 0xc7b2bd60 fpcurthread = none idlethread = 0xc7de22f0: tid 100005 "idle: cpu2" APIC ID = 2 currentldt = 0x50 spin locks held: cpuid = 3 dynamic pcpu = 0x69a7700 curthread = 0xc7de2000: pid 11 "idle: cpu3" curpcb = 0xc7b2ed60 fpcurthread = none idlethread = 0xc7de2000: tid 100006 "idle: cpu3" APIC ID = 3 currentldt = 0x50 spin locks held: db:0:allpcpu> show alllocks Process 2083 (mkdir) thread 0xcbcd8000 (100210) exclusive lockmgr ufs (ufs) r = 0 (0xce922328) locked @ /var/tmp/deviant2/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:595 Process 1307 (kinfo) thread 0xca8905e0 (100083) shared lockmgr ufs (ufs) r = 0 (0xcbbd15f8) locked @ kern/vfs_cache.c:676 db:0:alllocks> show lockedvnods Locked vnodes 0xce9222d0: tag ufs, type VDIR usecount 2, writecount 0, refcount 8 mountedhere 0 flags (VI(0x200)) v_object 0xce95c784 ref 0 pages 1 lock type ufs: EXCL by thread 0xcbcd8000 (pid 2083, mkdir, tid 100210) with exclusive waiters pending with shared waiters pending #0 0xc0a83288 at __lockmgr_args+0x668 #1 0xc0cfd261 at ffs_lock+0xa1 #2 0xc0e43a73 at VOP_LOCK1_APV+0xf3 #3 0xcbc6822f at null_lock+0xcf #4 0xc0e43a73 at VOP_LOCK1_APV+0xf3 #5 0xc0b50bb8 at _vn_lock+0x78 #6 0xc0b329a6 at lookup+0x106 #7 0xc0b33fef at namei+0x6ff #8 0xc0b46520 at kern_rmdirat+0x80 #9 0xc0b46727 at kern_rmdir+0x27 #10 0xc0b46752 at sys_rmdir+0x22 #11 0xc0e1c35e at syscall+0x2fe #12 0xc0e06141 at Xint0x80_syscall+0x21 ino 20089856, on dev label/tmp 0xcbbd15a0: tag ufs, type VDIR usecount 1, writecount 0, refcount 4 mountedhere 0 flags (VI(0x200)) v_object 0xce9445c8 ref 0 pages 1 lock type ufs: SHARED (count 1) with exclusive waiters pending #0 0xc0a83288 at __lockmgr_args+0x668 #1 0xc0cfd261 at ffs_lock+0xa1 #2 0xc0e43a73 at VOP_LOCK1_APV+0xf3 #3 0xc0b50bb8 at _vn_lock+0x78 #4 0xc0b437bb at vget+0xbb #5 0xc0b31b2d at vfs_hash_get+0xed #6 0xc0cf6459 at ffs_vgetf+0x49 #7 0xc0cf69fe at ffs_vget+0x2e #8 0xc0b512e4 at vn_vget_ino+0x134 #9 0xc0d05597 at ufs_lookup_ino+0xa87 #10 0xc0d0578a at ufs_lookup+0x2a #11 0xc0e41200 at VOP_CACHEDLOOKUP_APV+0x100 #12 0xc0b2af16 at vfs_cache_lookup+0xe6 #13 0xc0e450b8 at VOP_LOOKUP_APV+0x108 #14 0xcbc680c4 at null_lookup+0x94 #15 0xc0e450b8 at VOP_LOOKUP_APV+0x108 #16 0xc0b32f8b at lookup+0x6eb #17 0xc0b33fef at namei+0x6ff ino 20136960, on dev label/tmp 0xce094b40: tag null, type VDIR usecount 2, writecount 0, refcount 2 mountedhere 0 flags (VI(0x200)) v_object 0xce9445c8 ref 0 pages 1 lock type ufs: SHARED (count 1) with exclusive waiters pending #0 0xc0a83288 at __lockmgr_args+0x668 #1 0xc0cfd261 at ffs_lock+0xa1 #2 0xc0e43a73 at VOP_LOCK1_APV+0xf3 #3 0xc0b50bb8 at _vn_lock+0x78 #4 0xc0b437bb at vget+0xbb #5 0xc0b31b2d at vfs_hash_get+0xed #6 0xc0cf6459 at ffs_vgetf+0x49 #7 0xc0cf69fe at ffs_vget+0x2e #8 0xc0b512e4 at vn_vget_ino+0x134 #9 0xc0d05597 at ufs_lookup_ino+0xa87 #10 0xc0d0578a at ufs_lookup+0x2a #11 0xc0e41200 at VOP_CACHEDLOOKUP_APV+0x100 #12 0xc0b2af16 at vfs_cache_lookup+0xe6 #13 0xc0e450b8 at VOP_LOOKUP_APV+0x108 #14 0xcbc680c4 at null_lookup+0x94 #15 0xc0e450b8 at VOP_LOOKUP_APV+0x108 #16 0xc0b32f8b at lookup+0x6eb #17 0xc0b33fef at namei+0x6ff vp=0xce094b40, lowervp=0xcbbd15a0 0xcd385870: tag null, type VDIR usecount 10, writecount 0, refcount 10 mountedhere 0 flags (VI(0x200)) lock type ufs: EXCL by thread 0xcbcd8000 (pid 2083, mkdir, tid 100210) with exclusive waiters pending with shared waiters pending #0 0xc0a83288 at __lockmgr_args+0x668 #1 0xc0cfd261 at ffs_lock+0xa1 #2 0xc0e43a73 at VOP_LOCK1_APV+0xf3 #3 0xcbc6822f at null_lock+0xcf #4 0xc0e43a73 at VOP_LOCK1_APV+0xf3 #5 0xc0b50bb8 at _vn_lock+0x78 #6 0xc0b329a6 at lookup+0x106 #7 0xc0b33fef at namei+0x6ff #8 0xc0b46520 at kern_rmdirat+0x80 #9 0xc0b46727 at kern_rmdir+0x27 #10 0xc0b46752 at sys_rmdir+0x22 #11 0xc0e1c35e at syscall+0x2fe #12 0xc0e06141 at Xint0x80_syscall+0x21 vp=0xcd385870, lowervp=0xce9222d0 db:0:lockedvnods> show mount 0xca7b6bb0 /dev/ad4s1a on / (ufs) 0xca7b7000 devfs on /dev (devfs) 0xcac2e5d8 /dev/ad4s1f on /home (ufs) 0xcac2e000 /dev/ad4s1d on /usr (ufs) 0xca7b7bb0 /dev/ad4s1e on /var (ufs) 0xca7b78c4 procfs on /proc (procfs) 0xca7b68c4 /dev/label/tmp on /tmp (ufs) 0xcbc6c000 /tmp on /home/pho/stress2/misc/mp1 (nullfs) More info: show mount db:0:mount> ps pid ppid pgrp uid state wmesg wchan cmd 47175 1 47175 0 Ss+ ttyin 0xc7eff270 sh 2083 1 1291 0 D+ ufs 0xcbbd15f8 mkdir 1313 1 1291 0 D+ ufs 0xce922328 kinfo 1311 1 1291 0 D+ ufs 0xce922328 kinfo 1309 1 1291 0 D+ ufs 0xce922328 kinfo 1308 1 1291 0 D+ ufs 0xce922328 kinfo 1307 1 1291 0 D+ ufs 0xce922328 kinfo 19 0 0 0 DL sdflush 0xc132fc80 [softdepflush] 18 0 0 0 DL vlruwt 0xca6d22cc [vnlru] 17 0 0 0 DL syncer 0xc1327f74 [syncer] 16 0 0 0 DL psleep 0xc1327c08 [bufdaemon] 9 0 0 0 DL pgzero 0xc1331c5c [pagezero] 8 0 0 0 DL psleep 0xc13317c8 [vmdaemon] 7 0 0 0 DL psleep 0xc1331790 [pagedaemon] 6 0 0 0 DL ccb_scan 0xc1174654 [xpt_thrd] 5 0 0 0 DL waiting_ 0xc132b498 [sctp_iterator] 4 0 0 0 DL - 0xc949c63c [fdc0] 3 0 0 0 SL - 0xc9522000 [fw0_probe] 15 0 0 0 DL (threaded) [usb] 100059 D - 0xc9506d34 [usbus5] 100058 D - 0xc9506d04 [usbus5] 100057 D - 0xc9506cd4 [usbus5] 100056 D - 0xc9506ca4 [usbus5] 100054 D - 0xc94f4b5c [usbus4] 100053 D - 0xc94f4b2c [usbus4] 100052 D - 0xc94f4afc [usbus4] 100051 D - 0xc94f4acc [usbus4] 100050 D - 0xc94efb5c [usbus3] 100049 D - 0xc94efb2c [usbus3] 100048 D - 0xc94efafc [usbus3] 100047 D - 0xc94efacc [usbus3] 100046 D - 0xc94ebb5c [usbus2] 100045 D - 0xc94ebb2c [usbus2] 100044 D - 0xc94ebafc [usbus2] 100043 D - 0xc94ebacc [usbus2] 100041 D - 0xc94e2b5c [usbus1] 100040 D - 0xc94e2b2c [usbus1] 100039 D - 0xc94e2afc [usbus1] 100038 D - 0xc94e2acc [usbus1] 100036 D - 0xc94dcb5c [usbus0] 100035 D - 0xc94dcb2c [usbus0] 100034 D - 0xc94dcafc [usbus0] 100033 D - 0xc94dcacc [usbus0] 2 0 0 0 DL ctl_work 0xc7fdd000 [ctl_thrd] 14 0 0 0 DL - 0xc11b0b64 [yarrow] 13 0 0 0 DL (threaded) [geom] 100015 D - 0xc11adc08 [g_down] 100014 D - 0xc11adc04 [g_up] 100013 D - 0xc11adbfc [g_event] 12 0 0 0 WL (threaded) [intr] 100067 I [irq12: psm0] 100066 I [irq1: atkbd0] 100064 I [swi0: uart] 100061 I [irq20: fwohci0] 100060 I [irq14: ata0] 100055 I [irq19: ehci0] 100042 I [irq18: ohci2 ohci4] 100037 I [irq17: ohci1 ohci3] 100032 I [irq16: hdac1 ohci0] 100031 I [irq22: ahci0] 100030 I [irq257: re0] 100029 I [irq256: hdac0] 100026 I [swi5: fast taskq] 100024 I [swi2: cambio] 100020 I [swi6: task queue] 100019 I [swi6: Giant taskq] 100012 I [swi1: netisr 0] 100011 I [swi3: vm] 100010 I [swi4: clock] 100009 I [swi4: clock] 100008 I [swi4: clock] 100007 I [swi4: clock] 11 0 0 0 RL (threaded) [idle] 100006 Run CPU 3 [idle: cpu3] 100005 Run CPU 2 [idle: cpu2] 100004 Run CPU 1 [idle: cpu1] 100003 Run CPU 0 [idle: cpu0] 1 0 1 0 SLs wait 0xc7de0b30 [init] 10 0 0 0 DL audit_wo 0xc132f3c0 [audit] 0 0 0 0 DLs (threaded) [kernel] 100069 D - 0xc11b0b64 [deadlkres] 100068 D - 0xc7d9a500 [mca taskq] 100062 D - 0xc7d9a740 [fw0_taskq] 100027 D - 0xc7d9a880 [ffs_trim taskq] 100025 D - 0xc7d9a900 [kqueue taskq] 100023 D - 0xc7d9a940 [acpi_task_2] 100022 D - 0xc7d9a940 [acpi_task_1] 100021 D - 0xc7d9a940 [acpi_task_0] 100018 D - 0xc7d9aa00 [thread taskq] 100016 D - 0xc7d9ab40 [firmware taskq] 100000 D sched 0xc11addc0 [swapper] 1310 1307 1291 0 Z+ kinfo 1318 1311 1291 0 Z+ kinfo 1319 1309 1291 0 Z+ kinfo 1312 1308 1291 0 Z+ kinfo 1320 1313 1291 0 Z+ kinfo db:0:ps> allt Tracing command sh pid 47175 tid 100258 td 0xcbc275e0 sched_switch(cbc275e0,0,104,1b3,f2a1ba08,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bfd40,...) at mi_switch+0x243 sleepq_switch(cbc275e0,0,c0ff8600,1ac,0,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0a8bc8a,c7eff204,0,c0ff09b7,cbc275e0,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(c7eff270,0,f2a1bacc,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c7eff270,c7eff204,c0ffce14,538,0,...) at _cv_wait_sig+0x263 tty_wait(c7eff200,c7eff270,f2a1bc1c,1,0,...) at tty_wait+0x71 ttydisc_read(c7eff200,f2a1bc1c,0,a1,0,...) at ttydisc_read+0x22c ttydev_read(c7ee7c00,f2a1bc1c,0,1,0,...) at ttydev_read+0xaa devfs_read_f(ce424620,f2a1bc1c,c7dbde80,0,cbc275e0,...) at devfs_read_f+0x7d dofileread(f2a1bc1c,ffffffff,ffffffff,0,f2a1bbf8,...) at dofileread+0x9e kern_readv(cbc275e0,0,f2a1bc1c,f2a1bc3c,1,...) at kern_readv+0x68 sys_read(cbc275e0,f2a1bccc,c1046c7c,c0e5447e,c94c8bc0,...) at sys_read+0x4f syscall(f2a1bd08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281f6a43, esp = 0xbfbfdccc, ebp = 0xbfbfdcf8 --- Tracing command mkdir pid 2083 tid 100210 td 0xcbcd8000 sched_switch(cbcd8000,0,104,1b3,f295b67c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,ca82f500,...) at mi_switch+0x243 sleepq_switch(cbcd8000,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(cbbd15f8,60,c0fe28cb,4,0,...) at sleepq_wait+0x6b __lockmgr_args(cbbd15f8,80100,cbbd1664,0,0,...) at __lockmgr_args+0xc96 ffs_lock(f295b7d0,c0aed5ec,cbcd8000,f295b7c0,c0a8c569,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c115abc0,f295b7d0,cbcd80b4,c116cd20,cbbd15a0,...) at VOP_LOCK1_APV+0xf3 _vn_lock(cbbd15a0,80100,c1002878,86f,eb,...) at _vn_lock+0x78 vget(cbbd15a0,80100,cbcd8000,50,0,...) at vget+0xbb vfs_hash_get(ca7b68c4,1334400,80000,cbcd8000,f295b948,...) at vfs_hash_get+0xed ffs_vgetf(ca7b68c4,1334400,80000,f295b948,0,...) at ffs_vgetf+0x49 ffs_vget(ca7b68c4,1334400,80000,f295b948,f295b958,...) at ffs_vget+0x2e ufs_lookup_ino(ce9222d0,f295ba44,f295bbd0,0,f295b9a8,...) at ufs_lookup_ino+0x8bb ufs_lookup(f295b9d0,c104dcd3,c0b2d4da,ce922328,f295b9c0,...) at ufs_lookup+0x2a VOP_CACHEDLOOKUP_APV(c115abc0,f295b9d0,f295bbd0,0,0,...) at VOP_CACHEDLOOKUP_APV+0x100 vfs_cache_lookup(f295ba34,c10035cf,ce922394,0,cbc68555,...) at vfs_cache_lookup+0xe6 VOP_LOOKUP_APV(c115abc0,f295ba34,161,8000,cd385870,...) at VOP_LOOKUP_APV+0x108 null_lookup(f295bae0,c10035cf,400500,c1001c48,2da,...) at null_lookup+0x94 VOP_LOOKUP_APV(cbc69a80,f295bae0,c1001c48,2da,c0aec75f,...) at VOP_LOOKUP_APV+0x108 lookup(f295bb90,c1001c48,10e,dc,ce094c04,...) at lookup+0x6eb namei(f295bb90,0,ce094b40,f295bb94,bfbf4f87,...) at namei+0x6ff kern_rmdirat(cbcd8000,ffffff9c,bfbf4f87,0,f295bc44,...) at kern_rmdirat+0x80 kern_rmdir(cbcd8000,bfbf4f87,0,f295bcfc,c0e1c35e,...) at kern_rmdir+0x27 sys_rmdir(cbcd8000,f295bccc,c1046c7c,c1025de7,1,...) at sys_rmdir+0x22 syscall(f295bd08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (137, FreeBSD ELF32, sys_rmdir), eip = 0x280d743b, esp = 0xbfbf4b1c, ebp = 0xbfbf4f58 --- Tracing command kinfo pid 1313 tid 100112 td 0xca6e6bc0 sched_switch(ca6e6bc0,0,104,1b3,f27b1694,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,ca82f700,...) at mi_switch+0x243 sleepq_switch(ca6e6bc0,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(ce922328,60,c0fe28cb,4,1,...) at sleepq_wait+0x6b __lockmgr_args(ce922328,200500,ce922394,0,0,...) at __lockmgr_args+0x595 ffs_lock(f27b17f0,c0ff09b7,eb,0,cbc68555,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c115abc0,f27b17f0,cbc68768,246,200500,...) at VOP_LOCK1_APV+0xf3 null_lock(f27b184c,c1327cbc,c12f9ed8,ca6e6c74,c12f9ec8,...) at null_lock+0xcf VOP_LOCK1_APV(cbc69a80,f27b184c,0,c116cd20,cd385870,...) at VOP_LOCK1_APV+0xf3 _vn_lock(cd385870,200400,c1000f7a,4df,c0ff09b7,...) at _vn_lock+0x78 vn_vptocnp_locked(f27b1910,c1000f7a,513,f27b1900,c0aa7bce,...) at vn_vptocnp_locked+0x31b vn_fullpath1(cde61800,f27b1a90,3ff,47f,cd385870,...) at vn_fullpath1+0x439 vn_fullpath(ca6e6bc0,cd385870,f27b1a90,f27b1a8c,ffffffff,...) at vn_fullpath+0xb6 export_fd_for_sysctl(1,ffffffff,ffffffff,ffffffff,0,...) at export_fd_for_sysctl+0xc6 sysctl_kern_proc_filedesc(c11265e0,f27b1be0,1,f27b1b68,f27b1b68,...) at sysctl_kern_proc_filedesc+0x2e0 sysctl_root(f27b1b68,0,c0ff380c,656,ca6e6bc0,...) at sysctl_root+0x20e userland_sysctl(ca6e6bc0,f27b1bd4,4,0,bfbfd808,...) at userland_sysctl+0x17a sys___sysctl(ca6e6bc0,f27b1ccc,c1046c7c,c0ffb980,c0ac9607,...) at sys___sysctl+0x94 syscall(f27b1d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (202, FreeBSD ELF32, sys___sysctl), eip = 0x2818c167, esp = 0xbfbfd78c, ebp = 0xbfbfd7b8 --- Tracing command kinfo pid 1311 tid 100107 td 0xca831bc0 sched_switch(ca831bc0,0,104,1b3,f2795694,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,ca82f700,...) at mi_switch+0x243 sleepq_switch(ca831bc0,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(ce922328,60,c0fe28cb,4,1,...) at sleepq_wait+0x6b __lockmgr_args(ce922328,200500,ce922394,0,0,...) at __lockmgr_args+0x595 ffs_lock(f27957f0,c0ff09b7,eb,0,cbc68555,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c115abc0,f27957f0,cbc68768,246,200500,...) at VOP_LOCK1_APV+0xf3 null_lock(f279584c,c1327cbc,c12f9fe0,ca831c74,c12f9fd0,...) at null_lock+0xcf VOP_LOCK1_APV(cbc69a80,f279584c,0,c116cd20,cd385870,...) at VOP_LOCK1_APV+0xf3 _vn_lock(cd385870,200400,c1000f7a,4df,c0ff09b7,...) at _vn_lock+0x78 vn_vptocnp_locked(f2795910,c1000f7a,513,f2795900,c0aa7bce,...) at vn_vptocnp_locked+0x31b vn_fullpath1(cea6a000,f2795a90,3ff,47f,cd385870,...) at vn_fullpath1+0x439 vn_fullpath(ca831bc0,cd385870,f2795a90,f2795a8c,ffffffff,...) at vn_fullpath+0xb6 export_fd_for_sysctl(1,ffffffff,ffffffff,ffffffff,0,...) at export_fd_for_sysctl+0xc6 sysctl_kern_proc_filedesc(c11265e0,f2795be0,1,f2795b68,f2795b68,...) at sysctl_kern_proc_filedesc+0x2e0 sysctl_root(f2795b68,0,c0ff380c,656,ca831bc0,...) at sysctl_root+0x20e userland_sysctl(ca831bc0,f2795bd4,4,0,bfbfd808,...) at userland_sysctl+0x17a sys___sysctl(ca831bc0,f2795ccc,c1046c7c,c0ffb980,f2795cc4,...) at sys___sysctl+0x94 syscall(f2795d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (202, FreeBSD ELF32, sys___sysctl), eip = 0x2818c167, esp = 0xbfbfd78c, ebp = 0xbfbfd7b8 --- Tracing command kinfo pid 1309 tid 100144 td 0xcbc265e0 sched_switch(cbc265e0,0,104,1b3,f2874694,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,ca82f700,...) at mi_switch+0x243 sleepq_switch(cbc265e0,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(ce922328,60,c0fe28cb,4,1,...) at sleepq_wait+0x6b __lockmgr_args(ce922328,200500,ce922394,0,0,...) at __lockmgr_args+0x595 ffs_lock(f28747f0,c0ff09b7,eb,0,cbc68555,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c115abc0,f28747f0,cbc68768,246,200500,...) at VOP_LOCK1_APV+0xf3 null_lock(f287484c,c1327cbc,c12fa038,cbc26694,c12fa028,...) at null_lock+0xcf VOP_LOCK1_APV(cbc69a80,f287484c,0,c116cd20,cd385870,...) at VOP_LOCK1_APV+0xf3 _vn_lock(cd385870,200400,c1000f7a,4df,c0ff09b7,...) at _vn_lock+0x78 vn_vptocnp_locked(f2874910,c1000f7a,513,f2874900,c0aa7bce,...) at vn_vptocnp_locked+0x31b vn_fullpath1(cde39c00,f2874a90,3ff,47f,cd385870,...) at vn_fullpath1+0x439 vn_fullpath(cbc265e0,cd385870,f2874a90,f2874a8c,ffffffff,...) at vn_fullpath+0xb6 export_fd_for_sysctl(1,ffffffff,ffffffff,ffffffff,0,...) at export_fd_for_sysctl+0xc6 sysctl_kern_proc_filedesc(c11265e0,f2874be0,1,f2874b68,f2874b68,...) at sysctl_kern_proc_filedesc+0x2e0 sysctl_root(f2874b68,0,c0ff380c,656,cbc265e0,...) at sysctl_root+0x20e userland_sysctl(cbc265e0,f2874bd4,4,0,bfbfd808,...) at userland_sysctl+0x17a sys___sysctl(cbc265e0,f2874ccc,c1046c7c,c0ffb980,f2874cc4,...) at sys___sysctl+0x94 syscall(f2874d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (202, FreeBSD ELF32, sys___sysctl), eip = 0x2818c167, esp = 0xbfbfd78c, ebp = 0xbfbfd7b8 --- Tracing command kinfo pid 1308 tid 100093 td 0xca90e5e0 sched_switch(ca90e5e0,0,104,1b3,f2746694,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,ca82f700,...) at mi_switch+0x243 sleepq_switch(ca90e5e0,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(ce922328,60,c0fe28cb,4,1,...) at sleepq_wait+0x6b __lockmgr_args(ce922328,200500,ce922394,0,0,...) at __lockmgr_args+0x595 ffs_lock(f27467f0,c0ff09b7,eb,0,cbc68555,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c115abc0,f27467f0,cbc68768,246,200500,...) at VOP_LOCK1_APV+0xf3 null_lock(f274684c,c1327cbc,c12fa090,ca90e694,c12fa080,...) at null_lock+0xcf VOP_LOCK1_APV(cbc69a80,f274684c,0,c116cd20,cd385870,...) at VOP_LOCK1_APV+0xf3 _vn_lock(cd385870,200400,c1000f7a,4df,c0ff09b7,...) at _vn_lock+0x78 vn_vptocnp_locked(f2746910,c1000f7a,513,f2746900,c0aa7bce,...) at vn_vptocnp_locked+0x31b vn_fullpath1(ce2e2400,f2746a90,3ff,47f,cd385870,...) at vn_fullpath1+0x439 vn_fullpath(ca90e5e0,cd385870,f2746a90,f2746a8c,ffffffff,...) at vn_fullpath+0xb6 export_fd_for_sysctl(1,ffffffff,ffffffff,ffffffff,0,...) at export_fd_for_sysctl+0xc6 sysctl_kern_proc_filedesc(c11265e0,f2746be0,1,f2746b68,f2746b68,...) at sysctl_kern_proc_filedesc+0x2e0 sysctl_root(f2746b68,0,c0ff380c,656,ca90e5e0,...) at sysctl_root+0x20e userland_sysctl(ca90e5e0,f2746bd4,4,0,bfbfd808,...) at userland_sysctl+0x17a sys___sysctl(ca90e5e0,f2746ccc,c1046c7c,c0ffb980,c0ac9607,...) at sys___sysctl+0x94 syscall(f2746d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (202, FreeBSD ELF32, sys___sysctl), eip = 0x2818c167, esp = 0xbfbfd78c, ebp = 0xbfbfd7b8 --- Tracing command kinfo pid 1307 tid 100083 td 0xca8905e0 sched_switch(ca8905e0,0,104,1b3,f271e24c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,ca82f700,...) at mi_switch+0x243 sleepq_switch(ca8905e0,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(ce922328,60,c0fe28cb,4,0,...) at sleepq_wait+0x6b __lockmgr_args(ce922328,400400,ce922394,0,0,...) at __lockmgr_args+0xc96 ffs_lock(f271e3a0,f271e39c,c0a8bbba,cbc69bd0,4,...) at ffs_lock+0xa1 VOP_LOCK1_APV(c115abc0,f271e3a0,c0e3e53a,c116cd20,ce9222d0,...) at VOP_LOCK1_APV+0xf3 _vn_lock(ce9222d0,400400,cbc68555,e3,f271e400,...) at _vn_lock+0x78 null_nodeget(cbc6c000,ce9222d0,f271e430,8000,ce094b40,...) at null_nodeget+0xa4 null_lookup(f271e4c8,c10035cf,c0b424ef,cbc69a80,f271e45c,...) at null_lookup+0xc3 VOP_LOOKUP_APV(cbc69a80,f271e4c8,f271e724,21e,c0aec75f,...) at VOP_LOOKUP_APV+0x108 lookup(f271e6e4,c1001c48,fa,dc,1,...) at lookup+0x6eb namei(f271e6e4,c0ff09b7,eb,cbbd1664,cbbd1664,...) at namei+0x6ff vn_open_cred(f271e6e4,f271e7ec,0,1,cb9d7500,...) at vn_open_cred+0x324 vop_stdvptocnp(f271e898,f271e838,c0e43ac5,f271e84c,f271e8b0,...) at vop_stdvptocnp+0x19c null_vptocnp(f271e898,c104e053,200500,c1000f7a,4df,...) at null_vptocnp+0xfe VOP_VPTOCNP_APV(cbc69a80,f271e898,c1000f7a,4df,c0ff09b7,...) at VOP_VPTOCNP_APV+0x111 vn_vptocnp_locked(f271e910,c1000f7a,513,f271e900,c0aa7bce,...) at vn_vptocnp_locked+0x34e vn_fullpath1(cbc70c00,f271ea90,3e7,47f,cda92168,...) at vn_fullpath1+0x439 vn_fullpath(ca8905e0,cda92168,f271ea90,f271ea8c,ffffffff,...) at vn_fullpath+0xb6 export_fd_for_sysctl(1,ffffffff,ffffffff,ffffffff,0,...) at export_fd_for_sysctl+0xc6 sysctl_kern_proc_filedesc(c11265e0,f271ebe0,1,f271eb68,f271eb68,...) at sysctl_kern_proc_filedesc+0x2e0 sysctl_root(f271eb68,0,c0ff380c,656,ca8905e0,...) at sysctl_root+0x20e userland_sysctl(ca8905e0,f271ebd4,4,0,bfbfd808,...) at userland_sysctl+0x17a sys___sysctl(ca8905e0,f271eccc,c1046c7c,c0ffb980,c0ac9607,...) at sys___sysctl+0x94 syscall(f271ed08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (202, FreeBSD ELF32, sys___sysctl), eip = 0x2818c167, esp = 0xbfbfd78c, ebp = 0xbfbfd7b8 --- Tracing command softdepflush pid 19 tid 100078 td 0xca6e78d0 sched_switch(ca6e78d0,0,104,1b3,ee332bf8,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,352,...) at mi_switch+0x243 sleepq_switch(ca6e78d0,0,c0ff8600,291,0,...) at sleepq_switch+0x1a7 sleepq_timedwait(c132fc80,54,c102aa51,0,0,...) at sleepq_timedwait+0x6b _sleep(c132fc80,c132fc40,54,c102aa51,3e8,...) at _sleep+0x369 softdep_flush(0,ee332d08,c0fec990,3db,ca6d2000,...) at softdep_flush+0x259 fork_exit(c0cf26b0,0,ee332d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee332d40, ebp = 0 --- Tracing command vnlru pid 18 tid 100077 td 0xca6e7bc0 sched_switch(ca6e7bc0,0,104,1b3,ee32fbcc,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,352,...) at mi_switch+0x243 sleepq_switch(ca6e7bc0,0,c0ff8600,291,0,...) at sleepq_switch+0x1a7 sleepq_timedwait(ca6d22cc,60,c100389c,0,0,...) at sleepq_timedwait+0x6b _sleep(ca6d22cc,c1327f34,260,c100389c,3e8,...) at _sleep+0x369 vnlru_proc(0,ee32fd08,c0fec990,3db,ca6d22cc,...) at vnlru_proc+0xe7 fork_exit(c0b447e0,0,ee32fd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee32fd40, ebp = 0 --- Tracing command syncer pid 17 tid 100076 td 0xc94fc8d0 sched_switch(c94fc8d0,0,104,1b3,ee32cbd4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c0ff09b7,...) at mi_switch+0x243 sleepq_switch(c94fc8d0,0,c0ff8600,291,c94fc8d0,...) at sleepq_switch+0x1a7 sleepq_timedwait(c1327f74,0,ee32cc58,1,0,...) at sleepq_timedwait+0x6b _cv_timedwait(c1327f74,c1327f60,3e8,783,4e20,...) at _cv_timedwait+0x272 sched_sync(0,ee32cd08,c0fec990,3db,ca6d2598,...) at sched_sync+0x595 fork_exit(c0b43b00,0,ee32cd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee32cd40, ebp = 0 --- Tracing command bufdaemon pid 16 tid 100075 td 0xc94fcbc0 sched_switch(c94fcbc0,0,104,1b3,ee329c04,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,352,...) at mi_switch+0x243 sleepq_switch(c94fcbc0,0,c0ff8600,291,0,...) at sleepq_switch+0x1a7 sleepq_timedwait(c1327c08,54,c10005f1,0,0,...) at sleepq_timedwait+0x6b _sleep(c1327c08,c1327c10,54,c10005f1,3e8,...) at _sleep+0x369 buf_daemon(0,ee329d08,c0fec990,3db,ca6d2864,...) at buf_daemon+0x175 fork_exit(c0b26450,0,ee329d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee329d40, ebp = 0 --- Tracing command pagezero pid 9 tid 100074 td 0xc9615000 sched_switch(c9615000,0,104,1b3,ee326c08,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,352,...) at mi_switch+0x243 sleepq_switch(c9615000,0,c0ff8600,291,0,...) at sleepq_switch+0x1a7 sleepq_timedwait(c1331c5c,0,c1031cc9,0,0,...) at sleepq_timedwait+0x6b _sleep(c1331c5c,c1330600,0,c1031cc9,493e0,...) at _sleep+0x369 vm_pagezero(0,ee326d08,c0fec990,3db,ca6d2b30,...) at vm_pagezero+0xdc fork_exit(c0d39910,0,ee326d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee326d40, ebp = 0 --- Tracing command vmdaemon pid 8 tid 100073 td 0xc96152f0 sched_switch(c96152f0,0,104,1b3,ee323bcc,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c7dbad80,...) at mi_switch+0x243 sleepq_switch(c96152f0,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(c13317c8,74,c10005f1,0,0,...) at sleepq_wait+0x6b _sleep(c13317c8,c13317cc,74,c10005f1,0,...) at _sleep+0x39b vm_daemon(0,ee323d08,c0fec990,3db,ca6d3000,...) at vm_daemon+0x59 fork_exit(c0d33a00,0,ee323d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee323d40, ebp = 0 --- Tracing command pagedaemon pid 7 tid 100072 td 0xc96155e0 sched_switch(c96155e0,0,104,1b3,ee320b60,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,352,...) at mi_switch+0x243 sleepq_switch(c96155e0,0,c0ff8600,291,0,...) at sleepq_switch+0x1a7 sleepq_timedwait(c1331790,54,c10005f1,0,0,...) at sleepq_timedwait+0x6b _sleep(c1331790,c1330600,54,c10005f1,1388,...) at _sleep+0x369 vm_pageout(0,ee320d08,c0fec990,3db,c7de12cc,...) at vm_pageout+0x2c7 fork_exit(c0d33f00,0,ee320d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee320d40, ebp = 0 --- Tracing command xpt_thrd pid 6 tid 100071 td 0xc96158d0 sched_switch(c96158d0,0,104,1b3,ee19dc04,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c7dbad00,...) at mi_switch+0x243 sleepq_switch(c96158d0,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(c1174654,5c,c0e48e88,0,0,...) at sleepq_wait+0x6b _sleep(c1174654,c1174698,5c,c0e48e88,0,...) at _sleep+0x39b xpt_scanner_thread(0,ee19dd08,c0fec990,3db,c7de1598,...) at xpt_scanner_thread+0x47 fork_exit(c04a8eb0,0,ee19dd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee19dd40, ebp = 0 --- Tracing command sctp_iterator pid 5 tid 100070 td 0xc9615bc0 sched_switch(c9615bc0,0,104,1b3,ee19ac0c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c7dbacc0,...) at mi_switch+0x243 sleepq_switch(c9615bc0,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(c132b498,0,c1017ec7,0,0,...) at sleepq_wait+0x6b _sleep(c132b498,c132b460,0,c1017ec7,0,...) at _sleep+0x39b sctp_iterator_thread(0,ee19ad08,c0fec990,3db,c7de1864,...) at sctp_iterator_thread+0x5c fork_exit(c0be0660,0,ee19ad08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee19ad40, ebp = 0 --- Tracing command fdc0 pid 4 tid 100065 td 0xc9616bc0 sched_switch(c9616bc0,0,104,1b3,ee18ab54,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,352,...) at mi_switch+0x243 sleepq_switch(c9616bc0,0,c0ff8600,291,0,...) at sleepq_switch+0x1a7 sleepq_timedwait(c949c63c,5c,c0fdec52,0,0,...) at sleepq_timedwait+0x6b _sleep(c949c63c,c949c6f0,5c,c0fdec52,3e8,...) at _sleep+0x369 fdc_thread(c949c600,ee18ad08,c0fec990,3db,c7de1b30,...) at fdc_thread+0x27d fork_exit(c0dbda00,c949c600,ee18ad08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee18ad40, ebp = 0 --- Tracing command fw0_probe pid 3 tid 100063 td 0xc94ea2f0 sched_switch(c94ea2f0,0,104,1b3,ee17a6f8,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11be9c0,...) at mi_switch+0x243 sleepq_switch(c94ea2f0,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(c9522000,6c,c0fdec52,100,0,...) at sleepq_wait_sig+0x17 _sleep(c9522000,c9526488,16c,c0fdec52,0,...) at _sleep+0x384 fw_bus_probe_thread(c9522000,ee17ad08,c0fec990,3db,c9427000,...) at fw_bus_probe_thread+0xa08 fork_exit(c0712b80,c9522000,ee17ad08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee17ad40, ebp = 0 --- Tracing command usb pid 15 tid 100059 td 0xc94fa000 sched_switch(c94fa000,0,104,1b3,ee159c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94fa000,0,c0ff8600,26e,c94fa000,...) at sleepq_switch+0x1a7 sleepq_wait(c9506d34,0,ee159c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c9506d34,c9506e4c,c0fd02d1,74,c9506d3c,...) at _cv_wait+0x263 usb_process(c9506d2c,ee159d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c9506d2c,ee159d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee159d40, ebp = 0 --- Tracing command usb pid 15 tid 100058 td 0xc94fa2f0 sched_switch(c94fa2f0,0,104,1b3,ee156c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94fa2f0,0,c0ff8600,26e,c94fa2f0,...) at sleepq_switch+0x1a7 sleepq_wait(c9506d04,0,ee156c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c9506d04,c9506e4c,c0fd02d1,74,c9506d0c,...) at _cv_wait+0x263 usb_process(c9506cfc,ee156d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c9506cfc,ee156d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee156d40, ebp = 0 --- Tracing command usb pid 15 tid 100057 td 0xc94fa5e0 sched_switch(c94fa5e0,0,104,1b3,ee153c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94fa5e0,0,c0ff8600,26e,c94fa5e0,...) at sleepq_switch+0x1a7 sleepq_wait(c9506cd4,0,ee153c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c9506cd4,c9506e4c,c0fd02d1,74,c9506cdc,...) at _cv_wait+0x263 usb_process(c9506ccc,ee153d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c9506ccc,ee153d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee153d40, ebp = 0 --- Tracing command usb pid 15 tid 100056 td 0xc94fa8d0 sched_switch(c94fa8d0,0,104,1b3,ee150c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94fa8d0,0,c0ff8600,26e,c94fa8d0,...) at sleepq_switch+0x1a7 sleepq_wait(c9506ca4,0,ee150c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c9506ca4,c9506e4c,c0fd02d1,74,c9506cac,...) at _cv_wait+0x263 usb_process(c9506c9c,ee150d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c9506c9c,ee150d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee150d40, ebp = 0 --- Tracing command usb pid 15 tid 100054 td 0xc94fc000 sched_switch(c94fc000,0,104,1b3,ee147c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94fc000,0,c0ff8600,26e,c94fc000,...) at sleepq_switch+0x1a7 sleepq_wait(c94f4b5c,0,ee147c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94f4b5c,c94f4c74,c0fd02d1,74,c94f4b64,...) at _cv_wait+0x263 usb_process(c94f4b54,ee147d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94f4b54,ee147d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee147d40, ebp = 0 --- Tracing command usb pid 15 tid 100053 td 0xc94fc2f0 sched_switch(c94fc2f0,0,104,1b3,ee144c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94fc2f0,0,c0ff8600,26e,c94fc2f0,...) at sleepq_switch+0x1a7 sleepq_wait(c94f4b2c,0,ee144c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94f4b2c,c94f4c74,c0fd02d1,74,c94f4b34,...) at _cv_wait+0x263 usb_process(c94f4b24,ee144d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94f4b24,ee144d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee144d40, ebp = 0 --- Tracing command usb pid 15 tid 100052 td 0xc94fc5e0 sched_switch(c94fc5e0,0,104,1b3,ee141c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94fc5e0,0,c0ff8600,26e,c94fc5e0,...) at sleepq_switch+0x1a7 sleepq_wait(c94f4afc,0,ee141c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94f4afc,c94f4c74,c0fd02d1,74,c94f4b04,...) at _cv_wait+0x263 usb_process(c94f4af4,ee141d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94f4af4,ee141d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee141d40, ebp = 0 --- Tracing command usb pid 15 tid 100051 td 0xc94e8000 sched_switch(c94e8000,0,104,1b3,ee13ec10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94e8000,0,c0ff8600,26e,c94e8000,...) at sleepq_switch+0x1a7 sleepq_wait(c94f4acc,0,ee13ec8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94f4acc,c94f4c74,c0fd02d1,74,c94f4ad4,...) at _cv_wait+0x263 usb_process(c94f4ac4,ee13ed08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94f4ac4,ee13ed08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee13ed40, ebp = 0 --- Tracing command usb pid 15 tid 100050 td 0xc94e82f0 sched_switch(c94e82f0,0,104,1b3,ee13ac10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94e82f0,0,c0ff8600,26e,c94e82f0,...) at sleepq_switch+0x1a7 sleepq_wait(c94efb5c,0,ee13ac8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94efb5c,c94efc74,c0fd02d1,74,c94efb64,...) at _cv_wait+0x263 usb_process(c94efb54,ee13ad08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94efb54,ee13ad08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee13ad40, ebp = 0 --- Tracing command usb pid 15 tid 100049 td 0xc94e85e0 sched_switch(c94e85e0,0,104,1b3,ee137c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94e85e0,0,c0ff8600,26e,c94e85e0,...) at sleepq_switch+0x1a7 sleepq_wait(c94efb2c,0,ee137c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94efb2c,c94efc74,c0fd02d1,74,c94efb34,...) at _cv_wait+0x263 usb_process(c94efb24,ee137d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94efb24,ee137d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee137d40, ebp = 0 --- Tracing command usb pid 15 tid 100048 td 0xc94e88d0 sched_switch(c94e88d0,0,104,1b3,ee134c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94e88d0,0,c0ff8600,26e,c94e88d0,...) at sleepq_switch+0x1a7 sleepq_wait(c94efafc,0,ee134c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94efafc,c94efc74,c0fd02d1,74,c94efb04,...) at _cv_wait+0x263 usb_process(c94efaf4,ee134d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94efaf4,ee134d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee134d40, ebp = 0 --- Tracing command usb pid 15 tid 100047 td 0xc94e8bc0 sched_switch(c94e8bc0,0,104,1b3,ee131c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94e8bc0,0,c0ff8600,26e,c94e8bc0,...) at sleepq_switch+0x1a7 sleepq_wait(c94efacc,0,ee131c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94efacc,c94efc74,c0fd02d1,74,c94efad4,...) at _cv_wait+0x263 usb_process(c94efac4,ee131d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94efac4,ee131d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee131d40, ebp = 0 --- Tracing command usb pid 15 tid 100046 td 0xc94e9000 sched_switch(c94e9000,0,104,1b3,ee12dc10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94e9000,0,c0ff8600,26e,c94e9000,...) at sleepq_switch+0x1a7 sleepq_wait(c94ebb5c,0,ee12dc8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94ebb5c,c94ebc74,c0fd02d1,74,c94ebb64,...) at _cv_wait+0x263 usb_process(c94ebb54,ee12dd08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94ebb54,ee12dd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee12dd40, ebp = 0 --- Tracing command usb pid 15 tid 100045 td 0xc94e92f0 sched_switch(c94e92f0,0,104,1b3,ee12ac10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94e92f0,0,c0ff8600,26e,c94e92f0,...) at sleepq_switch+0x1a7 sleepq_wait(c94ebb2c,0,ee12ac8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94ebb2c,c94ebc74,c0fd02d1,74,c94ebb34,...) at _cv_wait+0x263 usb_process(c94ebb24,ee12ad08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94ebb24,ee12ad08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee12ad40, ebp = 0 --- Tracing command usb pid 15 tid 100044 td 0xc94e95e0 sched_switch(c94e95e0,0,104,1b3,ee127c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94e95e0,0,c0ff8600,26e,c94e95e0,...) at sleepq_switch+0x1a7 sleepq_wait(c94ebafc,0,ee127c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94ebafc,c94ebc74,c0fd02d1,74,c94ebb04,...) at _cv_wait+0x263 usb_process(c94ebaf4,ee127d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94ebaf4,ee127d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee127d40, ebp = 0 --- Tracing command usb pid 15 tid 100043 td 0xc94e98d0 sched_switch(c94e98d0,0,104,1b3,ee124c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94e98d0,0,c0ff8600,26e,c94e98d0,...) at sleepq_switch+0x1a7 sleepq_wait(c94ebacc,0,ee124c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94ebacc,c94ebc74,c0fd02d1,74,c94ebad4,...) at _cv_wait+0x263 usb_process(c94ebac4,ee124d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94ebac4,ee124d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee124d40, ebp = 0 --- Tracing command usb pid 15 tid 100041 td 0xc94ea000 sched_switch(c94ea000,0,104,1b3,ee11dc10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94ea000,0,c0ff8600,26e,c94ea000,...) at sleepq_switch+0x1a7 sleepq_wait(c94e2b5c,0,ee11dc8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94e2b5c,c94e2c74,c0fd02d1,74,c94e2b64,...) at _cv_wait+0x263 usb_process(c94e2b54,ee11dd08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94e2b54,ee11dd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee11dd40, ebp = 0 --- Tracing command usb pid 15 tid 100040 td 0xc94c5000 sched_switch(c94c5000,0,104,1b3,ee11ac10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94c5000,0,c0ff8600,26e,c94c5000,...) at sleepq_switch+0x1a7 sleepq_wait(c94e2b2c,0,ee11ac8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94e2b2c,c94e2c74,c0fd02d1,74,c94e2b34,...) at _cv_wait+0x263 usb_process(c94e2b24,ee11ad08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94e2b24,ee11ad08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee11ad40, ebp = 0 --- Tracing command usb pid 15 tid 100039 td 0xc94c52f0 sched_switch(c94c52f0,0,104,1b3,ee117c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94c52f0,0,c0ff8600,26e,c94c52f0,...) at sleepq_switch+0x1a7 sleepq_wait(c94e2afc,0,ee117c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94e2afc,c94e2c74,c0fd02d1,74,c94e2b04,...) at _cv_wait+0x263 usb_process(c94e2af4,ee117d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94e2af4,ee117d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee117d40, ebp = 0 --- Tracing command usb pid 15 tid 100038 td 0xc94c55e0 sched_switch(c94c55e0,0,104,1b3,ee114c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94c55e0,0,c0ff8600,26e,c94c55e0,...) at sleepq_switch+0x1a7 sleepq_wait(c94e2acc,0,ee114c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94e2acc,c94e2c74,c0fd02d1,74,c94e2ad4,...) at _cv_wait+0x263 usb_process(c94e2ac4,ee114d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94e2ac4,ee114d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee114d40, ebp = 0 --- Tracing command usb pid 15 tid 100036 td 0xc94c5bc0 sched_switch(c94c5bc0,0,104,1b3,ee10dc10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94c5bc0,0,c0ff8600,26e,c94c5bc0,...) at sleepq_switch+0x1a7 sleepq_wait(c94dcb5c,0,ee10dc8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94dcb5c,c94dcc74,c0fd02d1,74,c94dcb64,...) at _cv_wait+0x263 usb_process(c94dcb54,ee10dd08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94dcb54,ee10dd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee10dd40, ebp = 0 --- Tracing command usb pid 15 tid 100035 td 0xc94c8000 sched_switch(c94c8000,0,104,1b3,ee10ac10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94c8000,0,c0ff8600,26e,c94c8000,...) at sleepq_switch+0x1a7 sleepq_wait(c94dcb2c,0,ee10ac8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94dcb2c,c94dcc74,c0fd02d1,74,c94dcb34,...) at _cv_wait+0x263 usb_process(c94dcb24,ee10ad08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94dcb24,ee10ad08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee10ad40, ebp = 0 --- Tracing command usb pid 15 tid 100034 td 0xc94c82f0 sched_switch(c94c82f0,0,104,1b3,ee107c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94c82f0,0,c0ff8600,26e,c94c82f0,...) at sleepq_switch+0x1a7 sleepq_wait(c94dcafc,0,ee107c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94dcafc,c94dcc74,c0fd02d1,74,c94dcb04,...) at _cv_wait+0x263 usb_process(c94dcaf4,ee107d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94dcaf4,ee107d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee107d40, ebp = 0 --- Tracing command usb pid 15 tid 100033 td 0xc94c85e0 sched_switch(c94c85e0,0,104,1b3,ee104c10,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c94c85e0,0,c0ff8600,26e,c94c85e0,...) at sleepq_switch+0x1a7 sleepq_wait(c94dcacc,0,ee104c8c,1,0,...) at sleepq_wait+0x6b _cv_wait(c94dcacc,c94dcc74,c0fd02d1,74,c94dcad4,...) at _cv_wait+0x263 usb_process(c94dcac4,ee104d08,c0fec990,3db,c94272cc,...) at usb_process+0x1a3 fork_exit(c0945fe0,c94dcac4,ee104d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee104d40, ebp = 0 --- Tracing command ctl_thrd pid 2 tid 100028 td 0xc7eea8d0 sched_switch(c7eea8d0,0,104,1b3,c7b7265c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c7db98c0,...) at mi_switch+0x243 sleepq_switch(c7eea8d0,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(c7fdd000,5c,c0e5491c,0,0,...) at sleepq_wait+0x6b _sleep(c7fdd000,c7fdd000,5c,c0e5491c,0,...) at _sleep+0x39b ctl_work_thread(c7fdd000,c7b72d08,c0fec990,3db,c9427598,...) at ctl_work_thread+0x22f6 fork_exit(c04d6960,c7fdd000,c7b72d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b72d40, ebp = 0 --- Tracing command yarrow pid 14 tid 100017 td 0xc7ee95e0 sched_switch(c7ee95e0,0,104,1b3,c7b50bd8,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,352,...) at mi_switch+0x243 sleepq_switch(c7ee95e0,0,c0ff8600,291,2,...) at sleepq_switch+0x1a7 sleepq_timedwait(c11b0b64,0,c0fdec52,2,0,...) at sleepq_timedwait+0x6b _sleep(c11b0b64,0,0,c0fdec52,64) at _sleep+0x369 pause(c0fdec52,64,c0fac230,111,0,...) at pause+0xa0 random_kthread(0,c7b50d08,c0fec990,3db,c7de0000,...) at random_kthread+0x1ef fork_exit(c0841f70,0,c7b50d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b50d40, ebp = 0 --- Tracing command geom pid 13 tid 100015 td 0xc7ee9bc0 sched_switch(c7ee9bc0,0,104,1b3,c7b4abc4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c7db99c0,...) at mi_switch+0x243 sleepq_switch(c7ee9bc0,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(c11adc08,5c,c0fdec52,0,0,...) at sleepq_wait+0x6b _sleep(c11adc08,c11adb68,25c,c0fdec52,0,...) at _sleep+0x39b g_io_schedule_down(c7ee9bc0,0,c0fe07e5,6c,c7b4acf4,...) at g_io_schedule_down+0x56 g_down_procbody(0,c7b4ad08,c0fec990,3db,c7de02cc,...) at g_down_procbody+0x9d fork_exit(c09fd6e0,0,c7b4ad08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b4ad40, ebp = 0 --- Tracing command geom pid 13 tid 100014 td 0xc7eea000 sched_switch(c7eea000,0,104,1b3,c7b47bf0,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c7db9980,...) at mi_switch+0x243 sleepq_switch(c7eea000,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(c11adc04,5c,c0fdec52,0,0,...) at sleepq_wait+0x6b _sleep(c11adc04,c11adb88,25c,c0fdec52,0,...) at _sleep+0x39b g_io_schedule_up(c7eea000,0,c0fe07e5,5f,c7b47cf4,...) at g_io_schedule_up+0x144 g_up_procbody(0,c7b47d08,c0fec990,3db,c7de02cc,...) at g_up_procbody+0x9d fork_exit(c09fd780,0,c7b47d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b47d40, ebp = 0 --- Tracing command geom pid 13 tid 100013 td 0xc7de32f0 sched_switch(c7de32f0,0,104,1b3,c7b44be8,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c7db9d00,...) at mi_switch+0x243 sleepq_switch(c7de32f0,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(c11adbfc,5c,c0fdec52,0,0,...) at sleepq_wait+0x6b _sleep(c11adbfc,c11adb2c,25c,c0fdec52,0,...) at _sleep+0x39b g_run_events(c11be9c0,0,c0fe07e5,79,c7b44cf4,...) at g_run_events+0x54b g_event_procbody(0,c7b44d08,c0fec990,3db,c7de02cc,...) at g_event_procbody+0x90 fork_exit(c09fd820,0,c7b44d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b44d40, ebp = 0 --- Tracing command intr pid 12 tid 100067 td 0xc96165e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100066 td 0xc96168d0 sched_switch(c96168d0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7d7c570,...) at mi_switch+0x243 ithread_loop(c7d77780,ee18dd08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d77780,ee18dd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee18dd40, ebp = 0 --- Tracing command intr pid 12 tid 100064 td 0xc9617000 sched_switch(c9617000,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c94fe2f0,...) at mi_switch+0x243 ithread_loop(c7d77520,ee17dd08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d77520,ee17dd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee17dd40, ebp = 0 --- Tracing command intr pid 12 tid 100061 td 0xc94ea8d0 sched_switch(c94ea8d0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7d7dc70,...) at mi_switch+0x243 ithread_loop(c7d771d0,ee171d08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d771d0,ee171d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee171d40, ebp = 0 --- Tracing command intr pid 12 tid 100060 td 0xc94eabc0 sched_switch(c94eabc0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7e24070,...) at mi_switch+0x243 ithread_loop(c7d77340,ee15fd08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d77340,ee15fd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee15fd40, ebp = 0 --- Tracing command intr pid 12 tid 100055 td 0xc94fabc0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100042 td 0xc94e9bc0 sched_switch(c94e9bc0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7d7dd70,...) at mi_switch+0x243 ithread_loop(c7d77490,ee121d08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d77490,ee121d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee121d40, ebp = 0 --- Tracing command intr pid 12 tid 100037 td 0xc94c58d0 sched_switch(c94c58d0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7d7ddf0,...) at mi_switch+0x243 ithread_loop(c7d774c0,ee111d08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d774c0,ee111d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee111d40, ebp = 0 --- Tracing command intr pid 12 tid 100032 td 0xc94c88d0 sched_switch(c94c88d0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7d7de70,...) at mi_switch+0x243 ithread_loop(c7d774f0,ee101d08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d774f0,ee101d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee101d40, ebp = 0 --- Tracing command intr pid 12 tid 100031 td 0xc94c8bc0 sched_switch(c94c8bc0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7d7db70,...) at mi_switch+0x243 ithread_loop(c7d77530,edfabd08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d77530,edfabd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xedfabd40, ebp = 0 --- Tracing command intr pid 12 tid 100030 td 0xc7eea2f0 sched_switch(c7eea2f0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7e241f0,...) at mi_switch+0x243 ithread_loop(c7d77590,edfa7d08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d77590,edfa7d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xedfa7d40, ebp = 0 --- Tracing command intr pid 12 tid 100029 td 0xc7eea5e0 sched_switch(c7eea5e0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7e24570,...) at mi_switch+0x243 ithread_loop(c7d776c0,c7ba4d08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d776c0,c7ba4d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7ba4d40, ebp = 0 --- Tracing command intr pid 12 tid 100026 td 0xc7fd4000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100024 td 0xc7fd45e0 sched_switch(c7fd45e0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7e24670,...) at mi_switch+0x243 ithread_loop(c7d776e0,c7b66d08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d776e0,c7b66d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b66d40, ebp = 0 --- Tracing command intr pid 12 tid 100020 td 0xc7e25bc0 sched_switch(c7e25bc0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7e249f0,...) at mi_switch+0x243 ithread_loop(c7d776f0,c7b5ad08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d776f0,c7b5ad08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b5ad40, ebp = 0 --- Tracing command intr pid 12 tid 100019 td 0xc7ee9000 sched_switch(c7ee9000,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7e24a70,...) at mi_switch+0x243 ithread_loop(c7d77700,c7b57d08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d77700,c7b57d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b57d40, ebp = 0 --- Tracing command intr pid 12 tid 100012 td 0xc7de35e0 sched_switch(c7de35e0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7d7d570,...) at mi_switch+0x243 ithread_loop(c7d77710,c7b41d08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d77710,c7b41d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b41d40, ebp = 0 --- Tracing command intr pid 12 tid 100011 td 0xc7de38d0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100010 td 0xc7de3bc0 sched_switch(c7de3bc0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7d7d670,...) at mi_switch+0x243 ithread_loop(c7d77730,c7b3bd08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d77730,c7b3bd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b3bd40, ebp = 0 --- Tracing command intr pid 12 tid 100009 td 0xc7e25000 sched_switch(c7e25000,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7d7d6f0,...) at mi_switch+0x243 ithread_loop(c7d77740,c7b38d08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d77740,c7b38d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b38d40, ebp = 0 --- Tracing command intr pid 12 tid 100008 td 0xc7e252f0 sched_switch(c7e252f0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7d7d770,...) at mi_switch+0x243 ithread_loop(c7d77750,c7b35d08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d77750,c7b35d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b35d40, ebp = 0 --- Tracing command intr pid 12 tid 100007 td 0xc7e255e0 sched_switch(c7e255e0,0,109,1b3,0,...) at sched_switch+0x458 mi_switch(109,0,c0feccb3,559,c7d7da70,...) at mi_switch+0x243 ithread_loop(c7d77760,c7b32d08,c0fec990,3db,c7de0598,...) at ithread_loop+0x1fb fork_exit(c0a71d20,c7d77760,c7b32d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b32d40, ebp = 0 --- Tracing command idle pid 11 tid 100006 td 0xc7de2000 cpustop_handler(3,c7b2ebc4,c0e1c8d3,c0ff09b7,119,...) at cpustop_handler+0x26 ipi_nmi_handler(c0ff09b7,119,c7b2eb8c,c7b1bc80,c7de0864,...) at ipi_nmi_handler+0x35 trap(c7b2ebd0) at trap+0x33 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0e021b5, esp = 0xc7b2ec10, ebp = 0xc7b2ec10 --- acpi_cpu_c1(c0ff09b7,119,c7b1bc80,0,c7b1bc80,...) at acpi_cpu_c1+0x5 acpi_cpu_idle(0,c7b2ec7c,c0e0ecc9,0,3,...) at acpi_cpu_idle+0x11e cpu_idle_acpi(0,3,c11bfd40,2,c7b2ecc8,...) at cpu_idle_acpi+0x2f cpu_idle(0,c7b2eca0,c0ff4bec,a39,c7de2000,...) at cpu_idle+0x89 sched_idletd(0,c7b2ed08,c0fec990,3db,c7de0864,...) at sched_idletd+0x263 fork_exit(c0ac7db0,0,c7b2ed08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b2ed40, ebp = 0 --- Tracing command idle pid 11 tid 100005 td 0xc7de22f0 kdb_enter(c0e64b49,c0ff6bbe,c7b2bb64,c091102d,c9485400,...) at kdb_enter+0x3a kdb_break(c9485400,c7b2bb6c,c0e37c70,c0feccb3,39e,...) at kdb_break+0x28 uart_intr(c9485400,c7b2bb7c,246,c11b1e50,0,...) at uart_intr+0x12d intr_event_handle(c7d7c380,c7b2bbd0,1e69eb,2e84d682,35a,...) at intr_event_handle+0x76 intr_execute_handlers(c7e230d0,c7b2bbd0,2,c7b2bc10,c0e064a5,...) at intr_execute_handlers+0x49 lapic_handle_intr(30,c7b2bbd0) at lapic_handle_intr+0x36 Xapic_isr1() at Xapic_isr1+0x35 --- interrupt, eip = 0xc0e021b5, esp = 0xc7b2bc10, ebp = 0xc7b2bc10 --- acpi_cpu_c1(c0ff09b7,119,c7b18c80,0,c7b18c80,...) at acpi_cpu_c1+0x5 acpi_cpu_idle(0,c7b2bc7c,c0e0ecc9,0,2,...) at acpi_cpu_idle+0x11e cpu_idle_acpi(0,2,c11bf6c0,2,c7b2bcc8,...) at cpu_idle_acpi+0x2f cpu_idle(0,c7b2bca0,c0ff4bec,a39,c7de22f0,...) at cpu_idle+0x89 sched_idletd(0,c7b2bd08,c0fec990,3db,c7de0864,...) at sched_idletd+0x263 fork_exit(c0ac7db0,0,c7b2bd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b2bd40, ebp = 0 --- Tracing command idle pid 11 tid 100004 td 0xc7de25e0 cpustop_handler(1,c7b28bc4,c0e1c8d3,c0ff09b7,119,...) at cpustop_handler+0x26 ipi_nmi_handler(c0ff09b7,119,c7b28b8c,c7b15c80,c7de0864,...) at ipi_nmi_handler+0x35 trap(c7b28bd0) at trap+0x33 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0e021b5, esp = 0xc7b28c10, ebp = 0xc7b28c10 --- acpi_cpu_c1(c0ff09b7,119,c7b15c80,0,c7b15c80,...) at acpi_cpu_c1+0x5 acpi_cpu_idle(0,c7b28c7c,c0e0ecc9,0,1,...) at acpi_cpu_idle+0x11e cpu_idle_acpi(0,1,c11bf040,2,c7b28cc8,...) at cpu_idle_acpi+0x2f cpu_idle(0,c7b28ca0,c0ff4bec,a39,c7de25e0,...) at cpu_idle+0x89 sched_idletd(0,c7b28d08,c0fec990,3db,c7de0864,...) at sched_idletd+0x263 fork_exit(c0ac7db0,0,c7b28d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b28d40, ebp = 0 --- Tracing command idle pid 11 tid 100003 td 0xc7de28d0 cpustop_handler(0,c7b25bc4,c0e1c8d3,c0ff09b7,119,...) at cpustop_handler+0x26 ipi_nmi_handler(c0ff09b7,119,c7b25b8c,c1825c80,c7de0864,...) at ipi_nmi_handler+0x35 trap(c7b25bd0) at trap+0x33 calltrap() at calltrap+0x6 --- trap 0x13, eip = 0xc0e021b5, esp = 0xc7b25c10, ebp = 0xc7b25c10 --- acpi_cpu_c1(c0ff09b7,119,c1825c80,0,c1825c80,...) at acpi_cpu_c1+0x5 acpi_cpu_idle(0,c7b25c7c,c0e0ecc9,0,0,...) at acpi_cpu_idle+0x11e cpu_idle_acpi(0,0,c11be9c0,2,c7b25cc8,...) at cpu_idle_acpi+0x2f cpu_idle(0,c7b25ca0,c0ff4bec,a39,c7de28d0,...) at cpu_idle+0x89 sched_idletd(0,c7b25d08,c0fec990,3db,c7de0864,...) at sched_idletd+0x263 fork_exit(c0ac7db0,0,c7b25d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b25d40, ebp = 0 --- Tracing command init pid 1 tid 100002 td 0xc7de2bc0 sched_switch(c7de2bc0,0,104,1b3,c7b21ab4,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c11bf040,...) at mi_switch+0x243 sleepq_switch(c7de2bc0,0,c0ff8600,1ac,6c,...) at sleepq_switch+0x1a7 sleepq_catch_signals(c0ff8600,165,0,100,100,...) at sleepq_catch_signals+0xfc sleepq_wait_sig(c7de0b30,6c,c0ffb5a4,100,0,...) at sleepq_wait_sig+0x17 _sleep(c7de0b30,c7de0bb8,16c,c0ffb5a4,0,...) at _sleep+0x384 kern_wait(c7de2bc0,ffffffff,c7b21c38,2,0,...) at kern_wait+0x597 sys_wait4(c7de2bc0,c7b21ccc,c1046c7c,c0feca1c,c7b21cc4,...) at sys_wait4+0x3b syscall(c7b21d08) at syscall+0x2fe Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x805ff2f, esp = 0xbfbfd84c, ebp = 0xbfbfd868 --- Tracing command audit pid 10 tid 100001 td 0xc7de3000 sched_switch(c7de3000,0,104,1b3,c7b1ebf0,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,eb,...) at mi_switch+0x243 sleepq_switch(c7de3000,0,c0ff8600,26e,c7de3000,...) at sleepq_switch+0x1a7 sleepq_wait(c132f3c0,0,c7b1ec6c,1,0,...) at sleepq_wait+0x6b _cv_wait(c132f3c0,c132f3a4,c1024bd7,194,c7b1eca4,...) at _cv_wait+0x263 audit_worker(0,c7b1ed08,c0fec990,3db,c7de1000,...) at audit_worker+0x84 fork_exit(c0caf950,0,c7b1ed08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b1ed40, ebp = 0 --- Tracing command kernel pid 0 tid 100069 td 0xc9616000 sched_switch(c9616000,0,104,1b3,ee196bd0,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,352,...) at mi_switch+0x243 sleepq_switch(c9616000,0,c0ff8600,291,2,...) at sleepq_switch+0x1a7 sleepq_timedwait(c11b0b64,0,c0fdec52,2,0,...) at sleepq_timedwait+0x6b _sleep(c11b0b64,0,0,c0fdec52,bb8) at _sleep+0x369 pause(c0fdec52,bb8,12b,129,c11ade48,...) at pause+0xa0 deadlkres(0,ee196d08,c0fec990,3db,c11addc0,...) at deadlkres+0x324 fork_exit(c0a50920,0,ee196d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee196d40, ebp = 0 --- Tracing command kernel pid 0 tid 100068 td 0xc96162f0 sched_switch(c96162f0,0,104,1b3,ee193c1c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c0ff8600,...) at mi_switch+0x243 sleepq_switch(c96162f0,0,c0ff8600,26e,c96162f0,...) at sleepq_switch+0x1a7 sleepq_wait(c7d9a500,0,c0ff36b6,c0fdec52,0,...) at sleepq_wait+0x6b msleep_spin(c7d9a500,c7d9a518,c0fdec52,0,a6f,...) at msleep_spin+0x252 taskqueue_thread_loop(c1391520,ee193d08,c0fec990,3db,c11addc0,...) at taskqueue_thread_loop+0x8f fork_exit(c0ae4e00,c1391520,ee193d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee193d40, ebp = 0 --- Tracing command kernel pid 0 tid 100062 td 0xc94ea5e0 sched_switch(c94ea5e0,0,104,1b3,ee177c1c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c0ff8600,...) at mi_switch+0x243 sleepq_switch(c94ea5e0,0,c0ff8600,26e,c94ea5e0,...) at sleepq_switch+0x1a7 sleepq_wait(c7d9a740,0,c0ff36b6,c0fdec52,0,...) at sleepq_wait+0x6b msleep_spin(c7d9a740,c7d9a758,c0fdec52,0,a6f,...) at msleep_spin+0x252 taskqueue_thread_loop(c952649c,ee177d08,c0fec990,3db,c11addc0,...) at taskqueue_thread_loop+0x8f fork_exit(c0ae4e00,c952649c,ee177d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xee177d40, ebp = 0 --- Tracing command kernel pid 0 tid 100027 td 0xc7eeabc0 sched_switch(c7eeabc0,0,104,1b3,c7b6fc04,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c7db9880,...) at mi_switch+0x243 sleepq_switch(c7eeabc0,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(c7d9a880,0,c0fdec52,0,0,...) at sleepq_wait+0x6b _sleep(c7d9a880,c7d9a898,0,c0fdec52,0,...) at _sleep+0x39b taskqueue_thread_loop(c132f9a4,c7b6fd08,c0fec990,3db,c11addc0,...) at taskqueue_thread_loop+0xb5 fork_exit(c0ae4e00,c132f9a4,c7b6fd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b6fd40, ebp = 0 --- Tracing command kernel pid 0 tid 100025 td 0xc7fd42f0 sched_switch(c7fd42f0,0,104,1b3,c7b69c04,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c7db9800,...) at mi_switch+0x243 sleepq_switch(c7fd42f0,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(c7d9a900,0,c0fdec52,0,0,...) at sleepq_wait+0x6b _sleep(c7d9a900,c7d9a918,0,c0fdec52,0,...) at _sleep+0x39b taskqueue_thread_loop(c11ae758,c7b69d08,c0fec990,3db,c11addc0,...) at taskqueue_thread_loop+0xb5 fork_exit(c0ae4e00,c11ae758,c7b69d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b69d40, ebp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xc7fd48d0 sched_switch(c7fd48d0,0,104,1b3,c7b63c1c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c0ff8600,...) at mi_switch+0x243 sleepq_switch(c7fd48d0,0,c0ff8600,26e,c7fd48d0,...) at sleepq_switch+0x1a7 sleepq_wait(c7d9a940,0,c0ff36b6,c0fdec52,0,...) at sleepq_wait+0x6b msleep_spin(c7d9a940,c7d9a958,c0fdec52,0,a6f,...) at msleep_spin+0x252 taskqueue_thread_loop(c1177898,c7b63d08,c0fec990,3db,c11addc0,...) at taskqueue_thread_loop+0x8f fork_exit(c0ae4e00,c1177898,c7b63d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b63d40, ebp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xc7fd4bc0 sched_switch(c7fd4bc0,0,104,1b3,c7b60c1c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c0ff8600,...) at mi_switch+0x243 sleepq_switch(c7fd4bc0,0,c0ff8600,26e,c7fd4bc0,...) at sleepq_switch+0x1a7 sleepq_wait(c7d9a940,0,c0ff36b6,c0fdec52,0,...) at sleepq_wait+0x6b msleep_spin(c7d9a940,c7d9a958,c0fdec52,0,a6f,...) at msleep_spin+0x252 taskqueue_thread_loop(c1177898,c7b60d08,c0fec990,3db,c11addc0,...) at taskqueue_thread_loop+0x8f fork_exit(c0ae4e00,c1177898,c7b60d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b60d40, ebp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xc7e258d0 sched_switch(c7e258d0,0,104,1b3,c7b5dc1c,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c0ff8600,...) at mi_switch+0x243 sleepq_switch(c7e258d0,0,c0ff8600,26e,c7e258d0,...) at sleepq_switch+0x1a7 sleepq_wait(c7d9a940,0,c0ff36b6,c0fdec52,0,...) at sleepq_wait+0x6b msleep_spin(c7d9a940,c7d9a958,c0fdec52,0,a6f,...) at msleep_spin+0x252 taskqueue_thread_loop(c1177898,c7b5dd08,c0fec990,3db,c11addc0,...) at taskqueue_thread_loop+0x8f fork_exit(c0ae4e00,c1177898,c7b5dd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b5dd40, ebp = 0 --- Tracing command kernel pid 0 tid 100018 td 0xc7ee92f0 sched_switch(c7ee92f0,0,104,1b3,c7b54c04,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c7db9a80,...) at mi_switch+0x243 sleepq_switch(c7ee92f0,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(c7d9aa00,0,c0fdec52,0,0,...) at sleepq_wait+0x6b _sleep(c7d9aa00,c7d9aa18,0,c0fdec52,0,...) at _sleep+0x39b taskqueue_thread_loop(c11cd76c,c7b54d08,c0fec990,3db,c11addc0,...) at taskqueue_thread_loop+0xb5 fork_exit(c0ae4e00,c11cd76c,c7b54d08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b54d40, ebp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xc7ee98d0 sched_switch(c7ee98d0,0,104,1b3,c7b4dc04,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,c7db9a00,...) at mi_switch+0x243 sleepq_switch(c7ee98d0,0,c0ff8600,26e,0,...) at sleepq_switch+0x1a7 sleepq_wait(c7d9ab40,0,c0fdec52,0,0,...) at sleepq_wait+0x6b _sleep(c7d9ab40,c7d9ab58,0,c0fdec52,0,...) at _sleep+0x39b taskqueue_thread_loop(c11cc160,c7b4dd08,c0fec990,3db,c11addc0,...) at taskqueue_thread_loop+0xb5 fork_exit(c0ae4e00,c11cc160,c7b4dd08) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc7b4dd40, ebp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xc11ae090 sched_switch(c11ae090,0,104,1b3,c1820c64,...) at sched_switch+0x458 mi_switch(104,0,c0ff8600,1f8,352,...) at mi_switch+0x243 sleepq_switch(c11ae090,0,c0ff8600,291,0,...) at sleepq_switch+0x1a7 sleepq_timedwait(c11addc0,54,c10255b8,0,0,...) at sleepq_timedwait+0x6b _sleep(c11addc0,0,54,c10255b8,2710,...) at _sleep+0x369 scheduler(0,181ec00,181ec00,181e000,1825000,...) at scheduler+0x243 mi_startup() at mi_startup+0xac begin() at begin+0x2c db:0:allt> call doadump Physical memory: 3435 MB Dumping 328 MB: 313 297 281 265 249 233 217 201 185 169 153 137 121 105 89 73 57 41 25 9 Dump complete = 0 db:0:doadump> reset cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 2 (kgdb) proc 2083 [Switching to thread 90 (Thread 100210)]#0 sched_switch (td=0xcbcd8000, newtd=0xca891bc0, flags=0x104) at ../../../kern/sched_ule.c:1919 1919 cpuid = PCPU_GET(cpuid); (kgdb) bt #0 sched_switch (td=0xcbcd8000, newtd=0xca891bc0, flags=0x104) at ../../../kern/sched_ule.c:1919 #1 0xc0aa8923 in mi_switch (flags=0x104, newtd=0x0) at ../../../kern/kern_synch.c:485 #2 0xc0ae1c97 in sleepq_switch (wchan=0xcbbd15f8, pri=0x60) at ../../../kern/subr_sleepqueue.c:544 #3 0xc0ae29db in sleepq_wait (wchan=0xcbbd15f8, pri=0x60) at ../../../kern/subr_sleepqueue.c:623 #4 0xc0a838b6 in __lockmgr_args (lk=0xcbbd15f8, flags=0x80100, ilk=0xcbbd1664, wmesg=0x0, pri=0x0, timo=0x0, file=0xc1002878 "../../../kern/vfs_subr.c", line=0x86f) at ../../../kern/kern_lock.c:225 #5 0xc0cfd261 in ffs_lock (ap=0xf295b7d0) at lockmgr.h:97 #6 0xc0e43a73 in VOP_LOCK1_APV (vop=0xc115abc0, a=0xf295b7d0) at vnode_if.c:1988 #7 0xc0b50bb8 in _vn_lock (vp=0xcbbd15a0, flags=0x80100, file=0xc1002878 "../../../kern/vfs_subr.c", line=0x86f) at vnode_if.h:859 #8 0xc0b437bb in vget (vp=0xcbbd15a0, flags=0x80100, td=0xcbcd8000) at ../../../kern/vfs_subr.c:2159 #9 0xc0b31b2d in vfs_hash_get (mp=0xca7b68c4, hash=0x1334400, flags=Variable "flags" is not available. ) at ../../../kern/vfs_hash.c:81 #10 0xc0cf6459 in ffs_vgetf (mp=0xca7b68c4, ino=0x1334400, flags=0x80000, vpp=0xf295b948, ffs_flags=0x0) at ../../../ufs/ffs/ffs_vfsops.c:1644 #11 0xc0cf69fe in ffs_vget (mp=0xca7b68c4, ino=0x1334400, flags=0x80000, vpp=0xf295b948) at ../../../ufs/ffs/ffs_vfsops.c:1625 #12 0xc0d053cb in ufs_lookup_ino (vdp=0xce9222d0, vpp=0xf295ba44, cnp=0xf295bbd0, dd_ino=0x0) at ../../../ufs/ufs/ufs_lookup.c:607 #13 0xc0d0578a in ufs_lookup (ap=0xf295b9d0) at ../../../ufs/ufs/ufs_lookup.c:214 #14 0xc0e41200 in VOP_CACHEDLOOKUP_APV (vop=0xc115b180, a=0xf295b9d0) at vnode_if.c:187 #15 0xc0b2af16 in vfs_cache_lookup (ap=0xf295ba34) at vnode_if.h:80 #16 0xc0e450b8 in VOP_LOOKUP_APV (vop=0xc115b180, a=0xf295ba34) at vnode_if.c:123 #17 0xcbc680c4 in null_lookup (ap=0xf295bae0) at vnode_if.h:54 #18 0xc0e450b8 in VOP_LOOKUP_APV (vop=0xcbc69a80, a=0xf295bae0) at vnode_if.c:123 #19 0xc0b32f8b in lookup (ndp=0xf295bb90) at vnode_if.h:54 #20 0xc0b33fef in namei (ndp=0xf295bb90) at ../../../kern/vfs_lookup.c:307 #21 0xc0b46520 in kern_rmdirat (td=0xcbcd8000, fd=0xffffff9c, path=0xbfbf4f87
, pathseg=UIO_USERSPACE) at ../../../kern/vfs_syscalls.c:3881 #22 0xc0b46727 in kern_rmdir (td=0xcbcd8000, path=0xbfbf4f87
, pathseg=UIO_USERSPACE) at ../../../kern/vfs_syscalls.c:3865 #23 0xc0b46752 in sys_rmdir (td=0xcbcd8000, uap=0xf295bccc) at ../../../kern/vfs_syscalls.c:3858 #24 0xc0e1c35e in syscall (frame=0xf295bd08) at subr_syscall.c:135 #25 0xc0e06141 in Xint0x80_syscall () at ../../../i386/i386/exception.s:267 #26 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) f 8 #8 0xc0b437bb in vget (vp=0xcbbd15a0, flags=0x80100, td=0xcbcd8000) at ../../../kern/vfs_subr.c:2159 2159 if ((error = vn_lock(vp, flags | LK_INTERLOCK)) != 0) { (kgdb) p *vp $1 = {v_type = VDIR, v_tag = 0xc0fe28cb "ufs", v_op = 0xc115abc0, v_data = 0xcbaf2c3c, v_mount = 0xca7b68c4, v_nmntvnodes = {tqe_next = 0xce975870, tqe_prev = 0xce975b54}, v_un = { vu_mount = 0x0, vu_socket = 0x0, vu_cdev = 0x0, vu_fifoinfo = 0x0}, v_hashlist = {le_next = 0xcdfb6e10, le_prev = 0xccf58cc8}, v_hash = 0x1334400, v_cache_src = {lh_first = 0x0}, v_cache_dst = {tqh_first = 0x0, tqh_last = 0xcbbd15d0}, v_cache_dd = 0xcbac5900, v_cstart = 0x0, v_lasta = 0x0, v_lastw = 0x0, v_clen = 0x0, v_lock = {lock_object = { lo_name = 0xc0fe28cb "ufs", lo_flags = 0x57b0000, lo_data = 0x0, lo_witness = 0xc7d867b0}, lk_lock = 0x15, lk_exslpfail = 0x0, lk_timo = 0x33, lk_pri = 0x60, lk_stack = { depth = 0x12, pcs = {0xc0a83288, 0xc0cfd261, 0xc0e43a73, 0xc0b50bb8, 0xc0b437bb, 0xc0b31b2d, 0xc0cf6459, 0xc0cf69fe, 0xc0b512e4, 0xc0d05597, 0xc0d0578a, 0xc0e41200, 0xc0b2af16, 0xc0e450b8, 0xcbc680c4, 0xc0e450b8, 0xc0b32f8b, 0xc0b33fef}}}, v_interlock = {lock_object = {lo_name = 0xc0ffa8ee "vnode interlock", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc7d7f1e0}, mtx_lock = 0x4}, v_vnlock = 0xcbbd15f8, v_holdcnt = 0x4, v_usecount = 0x1, v_iflag = 0x200, v_vflag = 0x0, v_writecount = 0x0, v_actfreelist = { tqe_next = 0xce975b40, tqe_prev = 0xce975960}, v_bufobj = {bo_mtx = {lock_object = {lo_name = 0xc1003871 "bufobj interlock", lo_flags = 0x1030000, lo_data = 0x0, lo_witness = 0xc7d84d48}, mtx_lock = 0x4}, bo_clean = {bv_hd = {tqh_first = 0x0, tqh_last = 0xcbbd16ac}, bv_root = 0x0, bv_cnt = 0x0}, bo_dirty = {bv_hd = { tqh_first = 0xe1b788e0, tqh_last = 0xe1b78918}, bv_root = 0xe1b788e0, bv_cnt = 0x1}, bo_numoutput = 0x0, bo_flag = 0x1, bo_ops = 0xc1135120, bo_bsize = 0x4000, bo_object = 0xce9445c8, bo_synclist = {le_next = 0x0, le_prev = 0xc7dde980}, bo_private = 0xcbbd15a0, __bo_vnode = 0xcbbd15a0}, v_pollinfo = 0x0, v_label = 0x0, v_lockf = 0x0, v_rl = {rl_waiters = {tqh_first = 0x0, tqh_last = 0xcbbd16fc}, rl_currdep = 0x0}} (kgdb) git diff master diff --git a/sys/fs/nullfs/null.h b/sys/fs/nullfs/null.h index c8fc5de..0878e55 100644 --- a/sys/fs/nullfs/null.h +++ b/sys/fs/nullfs/null.h @@ -56,6 +56,7 @@ struct null_node { int nullfs_init(struct vfsconf *vfsp); int nullfs_uninit(struct vfsconf *vfsp); int null_nodeget(struct mount *mp, struct vnode *target, struct vnode **vpp); +struct vnode *null_hashget(struct mount *mp, struct vnode *lowervp); void null_hashrem(struct null_node *xp); int null_bypass(struct vop_generic_args *ap); diff --git a/sys/fs/nullfs/null_subr.c b/sys/fs/nullfs/null_subr.c index 7e3571d..5f926a6 100644 --- a/sys/fs/nullfs/null_subr.c +++ b/sys/fs/nullfs/null_subr.c @@ -67,7 +67,6 @@ struct mtx null_hashmtx; static MALLOC_DEFINE(M_NULLFSHASH, "nullfs_hash", "NULLFS hash table"); MALLOC_DEFINE(M_NULLFSNODE, "nullfs_node", "NULLFS vnode private part"); -static struct vnode * null_hashget(struct mount *, struct vnode *); static struct vnode * null_hashins(struct mount *, struct null_node *); /* @@ -98,7 +97,7 @@ nullfs_uninit(vfsp) * Return a VREF'ed alias for lower vnode if already exists, else 0. * Lower vnode should be locked on entry and will be left locked on exit. */ -static struct vnode * +struct vnode * null_hashget(mp, lowervp) struct mount *mp; struct vnode *lowervp; @@ -209,14 +208,10 @@ null_nodeget(mp, lowervp, vpp) struct vnode *vp; int error; - /* - * The insmntque1() call below requires the exclusive lock on - * the nullfs vnode. - */ - ASSERT_VOP_ELOCKED(lowervp, "lowervp"); - KASSERT(lowervp->v_usecount >= 1, ("Unreferenced vnode %p\n", lowervp)); + ASSERT_VOP_LOCKED(lowervp, "lowervp"); + KASSERT(lowervp->v_usecount >= 1, ("Unreferenced vnode %p", lowervp)); - /* Lookup the hash firstly */ + /* Lookup the hash firstly. */ *vpp = null_hashget(mp, lowervp); if (*vpp != NULL) { vrele(lowervp); @@ -224,6 +219,19 @@ null_nodeget(mp, lowervp, vpp) } /* + * The insmntque1() call below requires the exclusive lock on + * the nullfs vnode. Upgrade the lock now if hash failed to + * provide ready to use vnode. + */ + if (VOP_ISLOCKED(lowervp) != LK_EXCLUSIVE) { + vn_lock(lowervp, LK_UPGRADE | LK_RETRY); + if ((lowervp->v_iflag & VI_DOOMED) != 0) { + vput(lowervp); + return (ENOENT); + } + } + + /* * We do not serialize vnode creation, instead we will check for * duplicates later, when adding new vnode to hash. * Note that duplicate can only appear in hash if the lowervp is @@ -233,8 +241,7 @@ null_nodeget(mp, lowervp, vpp) * might cause a bogus v_data pointer to get dereferenced * elsewhere if MALLOC should block. */ - xp = malloc(sizeof(struct null_node), - M_NULLFSNODE, M_WAITOK); + xp = malloc(sizeof(struct null_node), M_NULLFSNODE, M_WAITOK); error = getnewvnode("null", mp, &null_vnodeops, &vp); if (error) { diff --git a/sys/fs/nullfs/null_vfsops.c b/sys/fs/nullfs/null_vfsops.c index 08f722a..84e5db7 100644 --- a/sys/fs/nullfs/null_vfsops.c +++ b/sys/fs/nullfs/null_vfsops.c @@ -65,6 +65,7 @@ static vfs_statfs_t nullfs_statfs; static vfs_unmount_t nullfs_unmount; static vfs_vget_t nullfs_vget; static vfs_extattrctl_t nullfs_extattrctl; +static vfs_reclaim_lowervp_t nullfs_reclaim_lowervp; /* * Mount null layer @@ -121,8 +122,10 @@ nullfs_mount(struct mount *mp) */ NDINIT(ndp, LOOKUP, FOLLOW|LOCKLEAF, UIO_SYSSPACE, target, curthread); error = namei(ndp); + /* * Re-lock vnode. + * XXXKIB This is deadlock-prone as well. */ if (isvnunlocked) vn_lock(mp->mnt_vnodecovered, LK_EXCLUSIVE | LK_RETRY); @@ -146,7 +149,7 @@ nullfs_mount(struct mount *mp) } xmp = (struct null_mount *) malloc(sizeof(struct null_mount), - M_NULLFSMNT, M_WAITOK); /* XXX */ + M_NULLFSMNT, M_WAITOK); /* * Save reference to underlying FS @@ -186,10 +189,14 @@ nullfs_mount(struct mount *mp) } MNT_ILOCK(mp); mp->mnt_kern_flag |= lowerrootvp->v_mount->mnt_kern_flag & - (MNTK_MPSAFE | MNTK_SHARED_WRITES); + (MNTK_MPSAFE | MNTK_SHARED_WRITES | MNTK_LOOKUP_SHARED | + MNTK_EXTENDED_SHARED); MNT_IUNLOCK(mp); mp->mnt_data = xmp; vfs_getnewfsid(mp); + MNT_ILOCK(xmp->nullm_vfs); + TAILQ_INSERT_TAIL(&xmp->nullm_vfs->mnt_uppers, mp, mnt_upper_link); + MNT_IUNLOCK(xmp->nullm_vfs); vfs_mountedfrom(mp, target); @@ -206,14 +213,16 @@ nullfs_unmount(mp, mntflags) struct mount *mp; int mntflags; { - void *mntdata; - int error; - int flags = 0; + struct null_mount *mntdata; + struct mount *ump; + int error, flags; NULLFSDEBUG("nullfs_unmount: mp = %p\n", (void *)mp); if (mntflags & MNT_FORCE) - flags |= FORCECLOSE; + flags = FORCECLOSE; + else + flags = 0; /* There is 1 extra root vnode reference (nullm_rootvp). */ error = vflush(mp, 1, flags, curthread); @@ -224,9 +233,17 @@ nullfs_unmount(mp, mntflags) * Finally, throw away the null_mount structure */ mntdata = mp->mnt_data; + ump = mntdata->nullm_vfs; + MNT_ILOCK(ump); + while ((ump->mnt_kern_flag & MNTK_VGONE_UPPER) != 0) { + ump->mnt_kern_flag |= MNTK_VGONE_WAITER; + msleep(&ump->mnt_uppers, &ump->mnt_mtx, 0, "vgnupw", 0); + } + TAILQ_REMOVE(&ump->mnt_uppers, mp, mnt_upper_link); + MNT_IUNLOCK(ump); mp->mnt_data = NULL; free(mntdata, M_NULLFSMNT); - return 0; + return (0); } static int @@ -316,13 +333,10 @@ nullfs_vget(mp, ino, flags, vpp) KASSERT((flags & LK_TYPE_MASK) != 0, ("nullfs_vget: no lock requested")); - flags &= ~LK_TYPE_MASK; - flags |= LK_EXCLUSIVE; error = VFS_VGET(MOUNTTONULLMOUNT(mp)->nullm_vfs, ino, flags, vpp); - if (error) + if (error != 0) return (error); - return (null_nodeget(mp, *vpp, vpp)); } @@ -334,11 +348,11 @@ nullfs_fhtovp(mp, fidp, flags, vpp) struct vnode **vpp; { int error; - error = VFS_FHTOVP(MOUNTTONULLMOUNT(mp)->nullm_vfs, fidp, LK_EXCLUSIVE, + + error = VFS_FHTOVP(MOUNTTONULLMOUNT(mp)->nullm_vfs, fidp, flags, vpp); - if (error) + if (error != 0) return (error); - return (null_nodeget(mp, *vpp, vpp)); } @@ -350,10 +364,22 @@ nullfs_extattrctl(mp, cmd, filename_vp, namespace, attrname) int namespace; const char *attrname; { - return VFS_EXTATTRCTL(MOUNTTONULLMOUNT(mp)->nullm_vfs, cmd, filename_vp, - namespace, attrname); + + return (VFS_EXTATTRCTL(MOUNTTONULLMOUNT(mp)->nullm_vfs, cmd, + filename_vp, namespace, attrname)); } +static void +nullfs_reclaim_lowervp(struct mount *mp, struct vnode *lowervp) +{ + struct vnode *vp; + + vp = null_hashget(mp, lowervp); + if (vp == NULL) + return; + vgone(vp); + vn_lock(lowervp, LK_EXCLUSIVE | LK_RETRY); +} static struct vfsops null_vfsops = { .vfs_extattrctl = nullfs_extattrctl, @@ -367,6 +393,7 @@ static struct vfsops null_vfsops = { .vfs_uninit = nullfs_uninit, .vfs_unmount = nullfs_unmount, .vfs_vget = nullfs_vget, + .vfs_reclaim_lowervp = nullfs_reclaim_lowervp, }; VFS_SET(null_vfsops, nullfs, VFCF_LOOPBACK | VFCF_JAIL); diff --git a/sys/fs/nullfs/null_vnops.c b/sys/fs/nullfs/null_vnops.c index 6f40233..5fb020b 100644 --- a/sys/fs/nullfs/null_vnops.c +++ b/sys/fs/nullfs/null_vnops.c @@ -665,33 +665,18 @@ null_unlock(struct vop_unlock_args *ap) } /* - * There is no way to tell that someone issued remove/rmdir operation - * on the underlying filesystem. For now we just have to release lowervp - * as soon as possible. - * - * Note, we can't release any resources nor remove vnode from hash before - * appropriate VXLOCK stuff is done because other process can find this - * vnode in hash during inactivation and may be sitting in vget() and waiting - * for null_inactive to unlock vnode. Thus we will do all those in VOP_RECLAIM. + * XXXKIB */ static int -null_inactive(struct vop_inactive_args *ap) +null_inactive(struct vop_inactive_args *ap __unused) { - struct vnode *vp = ap->a_vp; - - vp->v_object = NULL; - - /* - * If this is the last reference, then free up the vnode - * so as not to tie up the lower vnodes. - */ - vrecycle(vp); return (0); } /* - * Now, the VXLOCK is in force and we're free to destroy the null vnode. + * Now, the nullfs vnode and, due to the sharing lock, the lower + * vnode, are exclusively locked, and we shall destroy the null vnode. */ static int null_reclaim(struct vop_reclaim_args *ap) diff --git a/sys/kern/vfs_mount.c b/sys/kern/vfs_mount.c index e9d3abe..f397842 100644 --- a/sys/kern/vfs_mount.c +++ b/sys/kern/vfs_mount.c @@ -481,6 +481,7 @@ vfs_mount_alloc(struct vnode *vp, struct vfsconf *vfsp, const char *fspath, mac_mount_create(cred, mp); #endif arc4rand(&mp->mnt_hashseed, sizeof mp->mnt_hashseed, 0); + TAILQ_INIT(&mp->mnt_uppers); return (mp); } @@ -514,6 +515,7 @@ vfs_mount_destroy(struct mount *mp) vprint("", vp); panic("unmount: dangling vnode"); } + KASSERT(TAILQ_EMPTY(&mp->mnt_uppers), ("mnt_uppers")); if (mp->mnt_nvnodelistsize != 0) panic("vfs_mount_destroy: nonzero nvnodelistsize"); if (mp->mnt_activevnodelistsize != 0) @@ -1275,7 +1277,8 @@ dounmount(mp, flags, td) } MNT_ILOCK(mp); - if (mp->mnt_kern_flag & MNTK_UNMOUNT) { + if ((mp->mnt_kern_flag & MNTK_UNMOUNT) != 0 || + !TAILQ_EMPTY(&mp->mnt_uppers)) { MNT_IUNLOCK(mp); if (coveredvp) VOP_UNLOCK(coveredvp, 0); diff --git a/sys/kern/vfs_subr.c b/sys/kern/vfs_subr.c index 631d3f2..d212ddf 100644 --- a/sys/kern/vfs_subr.c +++ b/sys/kern/vfs_subr.c @@ -2688,6 +2688,12 @@ vgone(struct vnode *vp) VI_UNLOCK(vp); } +static void +vgonel_reclaim_lowervp(struct mount *mp __unused, + struct vnode *lowervp __unused) +{ +} + /* * vgone, with the vp interlock held. */ @@ -2697,7 +2703,10 @@ vgonel(struct vnode *vp) struct thread *td; int oweinact; int active; - struct mount *mp; + struct mount *mp, *ump, *mmp; + static struct vfsops vgonel_vfsops = { + .vfs_reclaim_lowervp = vgonel_reclaim_lowervp + }; ASSERT_VOP_ELOCKED(vp, "vgonel"); ASSERT_VI_LOCKED(vp, "vgonel"); @@ -2712,6 +2721,7 @@ vgonel(struct vnode *vp) if (vp->v_iflag & VI_DOOMED) return; vp->v_iflag |= VI_DOOMED; + /* * Check to see if the vnode is in use. If so, we have to call * VOP_CLOSE() and VOP_INACTIVE(). @@ -2719,6 +2729,37 @@ vgonel(struct vnode *vp) active = vp->v_usecount; oweinact = (vp->v_iflag & VI_OWEINACT); VI_UNLOCK(vp); + + /* + * Notify upper mounts about reclaimed vnode. + */ + mp = vp->v_mount; + MNT_ILOCK(mp); + if (!TAILQ_EMPTY(&mp->mnt_uppers)) { + MNT_IUNLOCK(mp); + mmp = malloc(sizeof(struct mount), M_TEMP, M_WAITOK | + M_ZERO); + mmp->mnt_op = &vgonel_vfsops; + MNT_ILOCK(mp); + mp->mnt_kern_flag |= MNTK_VGONE_UPPER; + for (ump = TAILQ_FIRST(&mp->mnt_uppers); ump != NULL;) { + TAILQ_INSERT_AFTER(&mp->mnt_uppers, ump, mmp, + mnt_upper_link); + MNT_IUNLOCK(mp); + VFS_RECLAIM_LOWERVP(ump, vp); + MNT_ILOCK(mp); + ump = TAILQ_NEXT(mmp, mnt_upper_link); + TAILQ_REMOVE(&mp->mnt_uppers, mmp, mnt_upper_link); + } + free(mmp, M_TEMP); + mp->mnt_kern_flag &= ~MNTK_VGONE_UPPER; + if ((mp->mnt_kern_flag & MNTK_VGONE_WAITER) != 0) { + mp->mnt_kern_flag &= ~MNTK_VGONE_WAITER; + wakeup(&mp->mnt_uppers); + } + } + MNT_IUNLOCK(mp); + /* * Clean out any buffers associated with the vnode. * If the flush fails, just toss the buffers. diff --git a/sys/sys/mount.h b/sys/sys/mount.h index 4b83ba4..8e65e1c 100644 --- a/sys/sys/mount.h +++ b/sys/sys/mount.h @@ -188,6 +188,8 @@ struct mount { #define mnt_endzero mnt_gjprovider char *mnt_gjprovider; /* gjournal provider name */ struct lock mnt_explock; /* vfs_export walkers lock */ + TAILQ_ENTRY(mount) mnt_upper_link; /* (m) we in the all uppers */ + TAILQ_HEAD(, mount) mnt_uppers; /* (m) upper mounts over us*/ }; /* @@ -373,6 +375,8 @@ void __mnt_vnode_markerfree(struct vnode **mvp, struct mount *mp); #define MNTK_NO_IOPF 0x00000100 /* Disallow page faults during reads and writes. Filesystem shall properly handle i/o state on EFAULT. */ +#define MNTK_VGONE_UPPER 0x00000200 +#define MNTK_VGONE_WAITER 0x00000200 #define MNTK_NOASYNC 0x00800000 /* disable async */ #define MNTK_UNMOUNT 0x01000000 /* unmount in progress */ #define MNTK_MWAIT 0x02000000 /* waiting for unmount to finish */ @@ -628,6 +632,7 @@ typedef int vfs_mount_t(struct mount *mp); typedef int vfs_sysctl_t(struct mount *mp, fsctlop_t op, struct sysctl_req *req); typedef void vfs_susp_clean_t(struct mount *mp); +typedef void vfs_reclaim_lowervp_t(struct mount *mp, struct vnode *lowervp); struct vfsops { vfs_mount_t *vfs_mount; @@ -645,6 +650,7 @@ struct vfsops { vfs_extattrctl_t *vfs_extattrctl; vfs_sysctl_t *vfs_sysctl; vfs_susp_clean_t *vfs_susp_clean; + vfs_reclaim_lowervp_t *vfs_reclaim_lowervp; }; vfs_statfs_t __vfs_statfs; @@ -670,6 +676,9 @@ vfs_statfs_t __vfs_statfs; #define VFS_SUSP_CLEAN(MP) \ ({if (*(MP)->mnt_op->vfs_susp_clean != NULL) \ (*(MP)->mnt_op->vfs_susp_clean)(MP); }) +#define VFS_RECLAIM_LOWERVP(MP, VP) \ + ({if (*(MP)->mnt_op->vfs_reclaim_lowervp != NULL) \ + (*(MP)->mnt_op->vfs_reclaim_lowervp)((MP), (VP)); }) #define VFS_NEEDSGIANT_(MP) \ ((MP) != NULL && ((MP)->mnt_kern_flag & MNTK_MPSAFE) == 0) -- Test scenario: nullfs5.sh