GENERIC HEAD from 2012-12-19 12:14:08 UTC, r244255M, vmcore.523 GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb 524288K of memory above 4GB ignored Copyright (c) 1992-2012 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #0 r244437M: Wed Dec 19 17:51:31 CET 2012 pho@x4.osted.lan:/usr/src/sys/i386/compile/PHO i386 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. CPU: AMD Phenom(tm) 9150e Quad-Core Processor (1800.02-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x100f23 Family = 0x10 Model = 0x2 Stepping = 3 Features=0x178bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x7ff TSC: P-state invariant real memory = 4294967296 (4096 MB) avail memory = 3527544832 (3364 MB) : Trying to mount root from ufs:/dev/ad4s1a [rw]... Enter full pathname of shell or RETURN for /bin/sh: Cannot read termcap database; using dumb terminal settings. # swapon -a warning: total configured swap (4194304 pages) exceeds maximum recommended amount (3890432 pages). warning: increase kern.maxswzone or reduce amount of swap. swapon: adding /dev/ad4s1b as swap device # dumpon /dev/ad4s1b # /etc/rc.d/ddb start Starting ddb. # exit panic: userret: Returning with with pinned thread cpuid = 1 KDB: enter: panic [ thread pid 31 tid 100079 ] Stopped at kdb_enter+0x3d: movl $0,kdb_why db> run pho db:0:pho> bt Tracing pid 31 tid 100079 td 0xca96f5e0 kdb_enter(c1083d85,c1083d85,c108a2da,f4221c30,f4221c00,...) at kdb_enter+0x3d/frame 0xf4221bc0 vpanic(c1231e62,100,c108a2da,f4221c30,f4221c30,...) at vpanic+0x142/frame 0xf4221c00 kassert_panic(c108a2da,0,c108a225,0,ca96f5e0,...) at kassert_panic+0xea/frame 0xf4221c24 userret(ca96f5e0,f4221d08,f4221d08,1,1,...) at userret+0x13a/frame 0xf4221c40 syscall(f4221d08) at syscall+0x470/frame 0xf4221cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf4221cfc --- syscall (4096), eip = 0x80904e7, esp = 0xbfbfce14, ebp = 0xbfbfce44 --- db:0:bt> show allpcpu Current CPU: 1 cpuid = 0 dynamic pcpu = 0x62b200 curthread = 0xc7dfb8d0: pid 11 "idle: cpu0" curpcb = 0xc7b45d60 fpcurthread = none idlethread = 0xc7dfb8d0: tid 100003 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: cpuid = 1 dynamic pcpu = 0x693b200 curthread = 0xca96f5e0: pid 31 "init" curpcb = 0xf4221d60 fpcurthread = none idlethread = 0xc7dfb5e0: tid 100004 "idle: cpu1" APIC ID = 1 currentldt = 0x50 spin locks held: cpuid = 2 dynamic pcpu = 0x693e200 curthread = 0xc7dfb2f0: pid 11 "idle: cpu2" curpcb = 0xc7b4bd60 fpcurthread = none idlethread = 0xc7dfb2f0: tid 100005 "idle: cpu2" APIC ID = 2 currentldt = 0x50 spin locks held: cpuid = 3 dynamic pcpu = 0x6941200 curthread = 0xc7dfb000: pid 11 "idle: cpu3" curpcb = 0xc7b4ed60 fpcurthread = none idlethread = 0xc7dfb000: tid 100006 "idle: cpu3" APIC ID = 3 currentldt = 0x50 spin locks held: db:0:allpcpu> show alllocks db:0:alllocks> show lockedvnods Locked vnodes db:0:lockedvnods> show mount 0xcaab5bb0 /dev/ad4s1a on / (ufs) 0xcaab6000 devfs on /dev (devfs) More info: show mount db:0:mount> ps pid ppid pgrp uid state wmesg wchan cmd 31 1 31 0 Rs+ CPU 1 init 19 0 0 0 DL sdflush 0xc13ad6b0 [softdepflush] 18 0 0 0 DL syncer 0xc13a94d4 [syncer] 17 0 0 0 DL vlruwt 0xca96a5a0 [vnlru] 16 0 0 0 DL psleep 0xc13a9268 [bufdaemon] 9 0 0 0 DL pgzero 0xc13ae0c4 [pagezero] 8 0 0 0 DL psleep 0xc13adc90 [vmdaemon] 7 0 0 0 DL psleep 0xc1401b04 [pagedaemon] 6 0 0 0 DL ccb_scan 0xc11fab24 [xpt_thrd] 5 0 0 0 DL waiting_ 0xc13fd800 [sctp_iterator] 4 0 0 0 DL - 0xc96f083c [fdc0] 3 0 0 0 SL - 0xc9785000 [fw0_probe] 15 0 0 0 DL (threaded) [usb] 100059 D - 0xc975dd34 [usbus5] 100058 D - 0xc975dd04 [usbus5] 100057 D - 0xc975dcd4 [usbus5] 100056 D - 0xc975dca4 [usbus5] 100054 D - 0xc9750b5c [usbus4] 100053 D - 0xc9750b2c [usbus4] 100052 D - 0xc9750afc [usbus4] 100051 D - 0xc9750acc [usbus4] 100050 D - 0xc974cb5c [usbus3] 100049 D - 0xc974cb2c [usbus3] 100048 D - 0xc974cafc [usbus3] 100047 D - 0xc974cacc [usbus3] 100046 D - 0xc9747b5c [usbus2] 100045 D - 0xc9747b2c [usbus2] 100044 D - 0xc9747afc [usbus2] 100043 D - 0xc9747acc [usbus2] 100041 D - 0xc973eb5c [usbus1] 100040 D - 0xc973eb2c [usbus1] 100039 D - 0xc973eafc [usbus1] 100038 D - 0xc973eacc [usbus1] 100036 D - 0xc9735b5c [usbus0] 100035 D - 0xc9735b2c [usbus0] 100034 D - 0xc9735afc [usbus0] 100033 D - 0xc9735acc [usbus0] 2 0 0 0 DL ctl_work 0xc8236000 [ctl_thrd] 14 0 0 0 DL - 0xc12320f0 [yarrow] 13 0 0 0 DL (threaded) [geom] 100015 D - 0xc13f9918 [g_down] 100014 D - 0xc13f9914 [g_up] 100013 D - 0xc13f9910 [g_event] 12 0 0 0 WL (threaded) [intr] 100067 I [irq12: psm0] 100066 I [irq1: atkbd0] 100064 I [swi0: uart] 100061 I [irq20: fwohci0] 100060 I [irq14: ata0] 100055 I [irq19: ehci0] 100042 I [irq18: ohci2 ohci4] 100037 I [irq17: ohci1 ohci3] 100032 I [irq16: hdac1 ohci0] 100031 I [irq22: ahci0] 100030 I [irq257: re0] 100029 I [irq256: hdac0] 100026 I [swi6: Giant taskq] 100025 I [swi6: task queue] 100020 I [swi2: cambio] 100018 I [swi5: fast taskq] 100012 I [swi3: vm] 100011 I [swi1: netisr 0] 100010 I [swi4: clock] 100009 I [swi4: clock] 100008 I [swi4: clock] 100007 I [swi4: clock] 11 0 0 0 RL (threaded) [idle] 100006 Run CPU 3 [idle: cpu3] 100005 Run CPU 2 [idle: cpu2] 100004 CanRun [idle: cpu1] 100003 Run CPU 0 [idle: cpu0] 1 0 1 0 SLs wait 0xc7df9b40 [init] 10 0 0 0 DL audit_wo 0xc14005c8 [audit] 0 0 0 0 DLs (threaded) [kernel] 100069 D - 0xc12320f0 [deadlkres] 100068 D - 0xc9744140 [mca taskq] 100062 D - 0xc9744700 [fw0_taskq] 100027 D - 0xc7dc3a00 [thread taskq] 100024 D - 0xc7dc3ac0 [ffs_trim taskq] 100023 D - 0xc7dc3c40 [acpi_task_2] 100022 D - 0xc7dc3c40 [acpi_task_1] 100021 D - 0xc7dc3c40 [acpi_task_0] 100019 D - 0xc7dc3c80 [kqueue taskq] 100016 D - 0xc7dc3e40 [firmware taskq] 100000 D sched 0xc13f9994 [swapper] db:0:ps> allt Tracing command init pid 31 tid 100079 td 0xca96f5e0 kdb_enter(c1083d85,c1083d85,c108a2da,f4221c30,f4221c00,...) at kdb_enter+0x3d/frame 0xf4221bc0 vpanic(c1231e62,100,c108a2da,f4221c30,f4221c30,...) at vpanic+0x142/frame 0xf4221c00 kassert_panic(c108a2da,0,c108a225,0,ca96f5e0,...) at kassert_panic+0xea/frame 0xf4221c24 userret(ca96f5e0,f4221d08,f4221d08,1,1,...) at userret+0x13a/frame 0xf4221c40 syscall(f4221d08) at syscall+0x470/frame 0xf4221cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf4221cfc --- syscall (4096), eip = 0x80904e7, esp = 0xbfbfce14, ebp = 0xbfbfce44 --- Tracing command softdepflush pid 19 tid 100078 td 0xca96f8d0 sched_switch(ca96f8d0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefe45bb8 mi_switch(104,0,c1089834,1f9,3,...) at mi_switch+0x1c9/frame 0xefe45bec sleepq_switch(ca96f8d0,0,c1089834,292,ca96f8d0,...) at sleepq_switch+0x1b9/frame 0xefe45c20 sleepq_timedwait(c13ad6b0,54,c10bb520,0,0,...) at sleepq_timedwait+0x48/frame 0xefe45c44 _sleep(c13ad6b0,c13ad63c,54,c10bb520,3e8,...) at _sleep+0x317/frame 0xefe45c9c softdep_flush(0,efe45d08,c107d928,3d7,0,...) at softdep_flush+0x23e/frame 0xefe45ccc fork_exit(c0cf33a0,0,efe45d08) at fork_exit+0x7f/frame 0xefe45cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefe45cf4 --- trap 0, eip = 0, esp = 0xefe45d40, ebp = 0 --- Tracing command syncer pid 18 tid 100077 td 0xca96fbc0 sched_switch(ca96fbc0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefe42b84 mi_switch(104,0,c1089834,1f9,8,...) at mi_switch+0x1c9/frame 0xefe42bb8 sleepq_switch(ca96fbc0,0,c1089834,292,0,...) at sleepq_switch+0x1b9/frame 0xefe42bec sleepq_timedwait(c13a94d4,0,efe42c44,1,0,...) at sleepq_timedwait+0x48/frame 0xefe42c10 _cv_timedwait(c13a94d4,c13a94b4,3e8,787,4e20,...) at _cv_timedwait+0x20c/frame 0xefe42c58 sched_sync(0,efe42d08,c107d928,3d7,0,...) at sched_sync+0x616/frame 0xefe42ccc fork_exit(c0b48220,0,efe42d08) at fork_exit+0x7f/frame 0xefe42cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefe42cf4 --- trap 0, eip = 0, esp = 0xefe42d40, ebp = 0 --- Tracing command vnlru pid 17 tid 100076 td 0xc97578d0 sched_switch(c97578d0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefe3fb8c mi_switch(104,0,c1089834,1f9,0,...) at mi_switch+0x1c9/frame 0xefe3fbc0 sleepq_switch(c97578d0,0,c1089834,292,c97578d0,...) at sleepq_switch+0x1b9/frame 0xefe3fbf4 sleepq_timedwait(ca96a5a0,60,c10956ed,0,0,...) at sleepq_timedwait+0x48/frame 0xefe3fc18 _sleep(ca96a5a0,c13a9490,260,c10956ed,3e8,...) at _sleep+0x317/frame 0xefe3fc70 vnlru_proc(0,efe3fd08,c107d928,3d7,0,...) at vnlru_proc+0x95/frame 0xefe3fccc fork_exit(c0b48a10,0,efe3fd08) at fork_exit+0x7f/frame 0xefe3fcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefe3fcf4 --- trap 0, eip = 0, esp = 0xefe3fd40, ebp = 0 --- Tracing command bufdaemon pid 16 tid 100075 td 0xc9757bc0 sched_switch(c9757bc0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefe3cbc8 mi_switch(104,0,c1089834,1f9,1,...) at mi_switch+0x1c9/frame 0xefe3cbfc sleepq_switch(c9757bc0,0,c1089834,292,c9757bc0,...) at sleepq_switch+0x1b9/frame 0xefe3cc30 sleepq_timedwait(c13a9268,54,c109221d,0,0,...) at sleepq_timedwait+0x48/frame 0xefe3cc54 _sleep(c13a9268,c13a9250,54,c109221d,3e8,...) at _sleep+0x317/frame 0xefe3ccac buf_daemon(0,efe3cd08,c107d928,3d7,0,...) at buf_daemon+0x8c/frame 0xefe3cccc fork_exit(c0b2bfa0,0,efe3cd08) at fork_exit+0x7f/frame 0xefe3ccf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefe3ccf4 --- trap 0, eip = 0, esp = 0xefe3cd40, ebp = 0 --- Tracing command pagezero pid 9 tid 100074 td 0xc9886000 sched_switch(c9886000,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefe39bc0 mi_switch(104,0,c1089834,1f9,2,...) at mi_switch+0x1c9/frame 0xefe39bf4 sleepq_switch(c9886000,0,c1089834,292,c9886000,...) at sleepq_switch+0x1b9/frame 0xefe39c28 sleepq_timedwait(c13ae0c4,0,c10c2460,0,0,...) at sleepq_timedwait+0x48/frame 0xefe39c4c _sleep(c13ae0c4,c1401a80,0,c10c2460,493e0,...) at _sleep+0x317/frame 0xefe39ca4 vm_pagezero(0,efe39d08,c107d928,3d7,0,...) at vm_pagezero+0x89/frame 0xefe39ccc fork_exit(c0d392e0,0,efe39d08) at fork_exit+0x7f/frame 0xefe39cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefe39cf4 --- trap 0, eip = 0, esp = 0xefe39d40, ebp = 0 --- Tracing command vmdaemon pid 8 tid 100073 td 0xc98862f0 sched_switch(c98862f0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefe36b8c mi_switch(104,0,c1089834,1f9,166,...) at mi_switch+0x1c9/frame 0xefe36bc0 sleepq_switch(c98862f0,0,c1089834,26f,c98862f0,...) at sleepq_switch+0x1b9/frame 0xefe36bf4 sleepq_wait(c13adc90,74,c109221d,0,0,...) at sleepq_wait+0x48/frame 0xefe36c18 _sleep(c13adc90,c13adc78,74,c109221d,0,...) at _sleep+0x34e/frame 0xefe36c70 vm_daemon(0,efe36d08,c107d928,3d7,0,...) at vm_daemon+0x8c/frame 0xefe36ccc fork_exit(c0d32430,0,efe36d08) at fork_exit+0x7f/frame 0xefe36cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefe36cf4 --- trap 0, eip = 0, esp = 0xefe36d40, ebp = 0 --- Tracing command pagedaemon pid 7 tid 100072 td 0xc98865e0 sched_switch(c98865e0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefe33b3c mi_switch(104,0,c1089834,1f9,0,...) at mi_switch+0x1c9/frame 0xefe33b70 sleepq_switch(c98865e0,0,c1089834,292,c98865e0,...) at sleepq_switch+0x1b9/frame 0xefe33ba4 sleepq_timedwait(c1401b04,54,c109221d,0,0,...) at sleepq_timedwait+0x48/frame 0xefe33bc8 _sleep(c1401b04,c1401a80,54,c109221d,1388,...) at _sleep+0x317/frame 0xefe33c20 vm_pageout(0,efe33d08,c107d928,3d7,0,...) at vm_pageout+0x256/frame 0xefe33ccc fork_exit(c0d32c90,0,efe33d08) at fork_exit+0x7f/frame 0xefe33cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefe33cf4 --- trap 0, eip = 0, esp = 0xefe33d40, ebp = 0 --- Tracing command xpt_thrd pid 6 tid 100071 td 0xc98868d0 sched_switch(c98868d0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefcb0bc4 mi_switch(104,0,c1089834,1f9,166,...) at mi_switch+0x1c9/frame 0xefcb0bf8 sleepq_switch(c98868d0,0,c1089834,26f,c98868d0,...) at sleepq_switch+0x1b9/frame 0xefcb0c2c sleepq_wait(c11fab24,5c,c0ed4ce0,0,0,...) at sleepq_wait+0x48/frame 0xefcb0c50 _sleep(c11fab24,c11fab68,5c,c0ed4ce0,0,...) at _sleep+0x34e/frame 0xefcb0ca8 xpt_scanner_thread(0,efcb0d08,c107d928,3d7,0,...) at xpt_scanner_thread+0xfc/frame 0xefcb0ccc fork_exit(c04b2b00,0,efcb0d08) at fork_exit+0x7f/frame 0xefcb0cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefcb0cf4 --- trap 0, eip = 0, esp = 0xefcb0d40, ebp = 0 --- Tracing command sctp_iterator pid 5 tid 100070 td 0xc9886bc0 sched_switch(c9886bc0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefcadbcc mi_switch(104,0,c1089834,1f9,166,...) at mi_switch+0x1c9/frame 0xefcadc00 sleepq_switch(c9886bc0,0,c1089834,26f,c9886bc0,...) at sleepq_switch+0x1b9/frame 0xefcadc34 sleepq_wait(c13fd800,0,c10a8885,0,0,...) at sleepq_wait+0x48/frame 0xefcadc58 _sleep(c13fd800,c13fd7c8,0,c10a8885,0,...) at _sleep+0x34e/frame 0xefcadcb0 sctp_iterator_thread(0,efcadd08,c107d928,3d7,0,...) at sctp_iterator_thread+0x5c/frame 0xefcadccc fork_exit(c0be34a0,0,efcadd08) at fork_exit+0x7f/frame 0xefcadcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefcadcf4 --- trap 0, eip = 0, esp = 0xefcadd40, ebp = 0 --- Tracing command fdc0 pid 4 tid 100065 td 0xc9887bc0 sched_switch(c9887bc0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefc9db50 mi_switch(104,0,c1089834,1f9,3,...) at mi_switch+0x1c9/frame 0xefc9db84 sleepq_switch(c9887bc0,0,c1089834,292,c9887bc0,...) at sleepq_switch+0x1b9/frame 0xefc9dbb8 sleepq_timedwait(c96f083c,5c,c10703ea,0,0,...) at sleepq_timedwait+0x48/frame 0xefc9dbdc _sleep(c96f083c,c96f08f0,5c,c10703ea,3e8,...) at _sleep+0x317/frame 0xefc9dc34 fdc_thread(c96f0800,efc9dd08,c107d928,3d7,0,...) at fdc_thread+0x74a/frame 0xefc9dccc fork_exit(c0e42f70,c96f0800,efc9dd08) at fork_exit+0x7f/frame 0xefc9dcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc9dcf4 --- trap 0, eip = 0, esp = 0xefc9dd40, ebp = 0 --- Tracing command fw0_probe pid 3 tid 100063 td 0xc97462f0 sched_switch(c97462f0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefc8d6b4 mi_switch(104,0,c1089834,1f9,c1089834,...) at mi_switch+0x1c9/frame 0xefc8d6e8 sleepq_switch(c97462f0,0,c1089834,1ad,efc8d760,...) at sleepq_switch+0x1b9/frame 0xefc8d71c sleepq_catch_signals(0,c1089834,166,780,c97462f0,...) at sleepq_catch_signals+0x3b1/frame 0xefc8d764 sleepq_wait_sig(c9785000,6c,c10703ea,100,0,...) at sleepq_wait_sig+0x14/frame 0xefc8d788 _sleep(c9785000,c9789488,16c,c10703ea,0,...) at _sleep+0x32d/frame 0xefc8d7e0 fw_bus_probe_thread(c9785000,efc8dd08,c107d928,3d7,0,...) at fw_bus_probe_thread+0x88/frame 0xefc8dccc fork_exit(c0707340,c9785000,efc8dd08) at fork_exit+0x7f/frame 0xefc8dcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc8dcf4 --- trap 0, eip = 0, esp = 0xefc8dd40, ebp = 0 --- Tracing command usb pid 15 tid 100059 td 0xc9754000 sched_switch(c9754000,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefc6cbd4 mi_switch(104,0,c1089834,1f9,8,...) at mi_switch+0x1c9/frame 0xefc6cc08 sleepq_switch(c9754000,0,c1089834,26f,c9754000,...) at sleepq_switch+0x1b9/frame 0xefc6cc3c sleepq_wait(c975dd34,0,efc6cc90,1,0,...) at sleepq_wait+0x48/frame 0xefc6cc60 _cv_wait(c975dd34,c975de4c,c1061c77,73,c975dd3c,...) at _cv_wait+0x1f9/frame 0xefc6cca4 usb_process(c975dd2c,efc6cd08,c107d928,3d7,0,...) at usb_process+0xe6/frame 0xefc6cccc fork_exit(c0957fb0,c975dd2c,efc6cd08) at fork_exit+0x7f/frame 0xefc6ccf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc6ccf4 --- trap 0, eip = 0, esp = 0xefc6cd40, ebp = 0 --- Tracing command usb pid 15 tid 100058 td 0xc97542f0 sched_switch(c97542f0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefc69bd4 mi_switch(104,0,c1089834,1f9,8,...) at mi_switch+0x1c9/frame 0xefc69c08 sleepq_switch(c97542f0,0,c1089834,26f,c97542f0,...) at sleepq_switch+0x1b9/frame 0xefc69c3c sleepq_wait(c975dd04,0,efc69c90,1,0,...) at sleepq_wait+0x48/frame 0xefc69c60 _cv_wait(c975dd04,c975de4c,c1061c77,73,c975dd0c,...) at _cv_wait+0x1f9/frame 0xefc69ca4 usb_process(c975dcfc,efc69d08,c107d928,3d7,0,...) at usb_process+0xe6/frame 0xefc69ccc fork_exit(c0957fb0,c975dcfc,efc69d08) at fork_exit+0x7f/frame 0xefc69cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc69cf4 --- trap 0, eip = 0, esp = 0xefc69d40, ebp = 0 --- Tracing command usb pid 15 tid 100057 td 0xc97545e0 sched_switch(c97545e0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefc66bd4 mi_switch(104,0,c1089834,1f9,8,...) at mi_switch+0x1c9/frame 0xefc66c08 sleepq_switch(c97545e0,0,c1089834,26f,c97545e0,...) at sleepq_switch+0x1b9/frame 0xefc66c3c sleepq_wait(c975dcd4,0,efc66c90,1,0,...) at sleepq_wait+0x48/frame 0xefc66c60 _cv_wait(c975dcd4,c975de4c,c1061c77,73,c975dcdc,...) at _cv_wait+0x1f9/frame 0xefc66ca4 usb_process(c975dccc,efc66d08,c107d928,3d7,0,...) at usb_process+0xe6/frame 0xefc66ccc fork_exit(c0957fb0,c975dccc,efc66d08) at fork_exit+0x7f/frame 0xefc66cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc66cf4 --- trap 0, eip = 0, esp = 0xefc66d40, ebp = 0 --- Tracing command usb pid 15 tid 100056 td 0xc97548d0 sched_switch(c97548d0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefc63bd4 mi_switch(104,0,c1089834,1f9,8,...) at mi_switch+0x1c9/frame 0xefc63c08 sleepq_switch(c97548d0,0,c1089834,26f,c97548d0,...) at sleepq_switch+0x1b9/frame 0xefc63c3c sleepq_wait(c975dca4,0,efc63c90,1,0,...) at sleepq_wait+0x48/frame 0xefc63c60 _cv_wait(c975dca4,c975de4c,c1061c77,73,c975dcac,...) at _cv_wait+0x1f9/frame 0xefc63ca4 usb_process(c975dc9c,efc63d08,c107d928,3d7,0,...) at usb_process+0xe6/frame 0xefc63ccc fork_exit(c0957fb0,c975dc9c,efc63d08) at fork_exit+0x7f/frame 0xefc63cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc63cf4 --- trap 0, eip = 0, esp = 0xefc63d40, ebp = 0 --- Tracing command usb pid 15 tid 100054 td 0xc9757000 sched_switch(c9757000,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefc5abd4 mi_switch(104,0,c1089834,1f9,8,...) at mi_switch+0x1c9/frame 0xefc5ac08 sleepq_switch(c9757000,0,c1089834,26f,c9757000,...) at sleepq_switch+0x1b9/frame 0xefc5ac3c sleepq_wait(c9750b5c,0,efc5ac90,1,0,...) at sleepq_wait+0x48/frame 0xefc5ac60 _cv_wait(c9750b5c,c9750c74,c1061c77,73,c9750b64,...) at _cv_wait+0x1f9/frame 0xefc5aca4 usb_process(c9750b54,efc5ad08,c107d928,3d7,0,...) at usb_process+0xe6/frame 0xefc5accc fork_exit(c0957fb0,c9750b54,efc5ad08) at fork_exit+0x7f/frame 0xefc5acf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc5acf4 --- trap 0, eip = 0, esp = 0xefc5ad40, ebp = 0 --- Tracing command usb pid 15 tid 100053 td 0xc97572f0 sched_switch(c97572f0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefc57bd4 mi_switch(104,0,c1089834,1f9,8,...) at mi_switch+0x1c9/frame 0xefc57c08 sleepq_switch(c97572f0,0,c1089834,26f,c97572f0,...) at sleepq_switch+0x1b9/frame 0xefc57c3c sleepq_wait(c9750b2c,0,efc57c90,1,0,...) at sleepq_wait+0x48/frame 0xefc57c60 _cv_wait(c9750b2c,c9750c74,c1061c77,73,c9750b34,...) at _cv_wait+0x1f9/frame 0xefc57ca4 usb_process(c9750b24,efc57d08,c107d928,3d7,0,...) at usb_process+0xe6/frame 0xefc57ccc fork_exit(c0957fb0,c9750b24,efc57d08) at fork_exit+0x7f/frame 0xefc57cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc57cf4 --- trap 0, eip = 0, esp = 0xefc57d40, ebp = 0 --- Tracing command usb pid 15 tid 100052 td 0xc97575e0 sched_switch(c97575e0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefc54bd4 mi_switch(104,0,c1089834,1f9,8,...) at mi_switch+0x1c9/frame 0xefc54c08 sleepq_switch(c97575e0,0,c1089834,26f,c97575e0,...) at sleepq_switch+0x1b9/frame 0xefc54c3c sleepq_wait(c9750afc,0,efc54c90,1,0,...) at sleepq_wait+0x48/frame 0xefc54c60 _cv_wait(c9750afc,c9750c74,c1061c77,73,c9750b04,...) at _cv_wait+0x1f9/frame 0xefc54ca4 usb_process(c9750af4,efc54d08,c107d928,3d7,0,...) at usb_process+0xe6/frame 0xefc54ccc fork_exit(c0957fb0,c9750af4,efc54d08) at fork_exit+0x7f/frame 0xefc54cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc54cf4 --- trap 0, eip = 0, esp = 0xefc54d40, ebp = 0 --- Tracing command usb pid 15 tid 100051 td 0xc9742000 sched_switch(c9742000,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefc51bd4 mi_switch(104,0,c1089834,1f9,8,...) at mi_switch+0x1c9/frame 0xefc51c08 sleepq_switch(c9742000,0,c1089834,26f,c9742000,...) at sleepq_switch+0x1b9/frame 0xefc51c3c sleepq_wait(c9750acc,0,efc51c90,1,0,...) at sleepq_wait+0x48/frame 0xefc51c60 _cv_wait(c9750acc,c9750c74,c1061c77,73,c9750ad4,...) at _cv_wait+0x1f9/frame 0xefc51ca4 usb_process(c9750ac4,efc51d08,c107d928,3d7,0,...) at usb_process+0xe6/frame 0xefc51ccc fork_exit(c0957fb0,c9750ac4,efc51d08) at fork_exit+0x7f/frame 0xefc51cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc51cf4 --- trap 0, eip = 0, esp = 0xefc51d40, ebp = 0 --- Tracing command usb pid 15 tid 100050 td 0xc97422f0 sched_switch(c97422f0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefc4dbd4 mi_switch(104,0,c1089834,1f9,8,...) at mi_switch+0x1c9/frame 0xefc4dc08 sleepq_switch(c97422f0,0,c1089834,26f,c97422f0,...) at sleepq_switch+0x1b9/frame 0xefc4dc3c sleepq_wait(c974cb5c,0,efc4dc90,1,0,...) at sleepq_wait+0x48/frame 0xefc4dc60 _cv_wait(c974cb5c,c974cc74,c1061c77,73,c974cb64,...) at _cv_wait+0x1f9/frame 0xefc4dca4 usb_process(c974cb54,efc4dd08,c107d928,3d7,0,...) at usb_process+0xe6/frame 0xefc4dccc fork_exit(c0957fb0,c974cb54,efc4dd08) at fork_exit+0x7f/frame 0xefc4dcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc4dcf4 --- trap 0, eip = 0, esp = 0xefc4dd40, ebp = 0 --- Tracing command usb pid 15 tid 100049 td 0xc97425e0 sched_switch(c97425e0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefc4abd4 mi_switch(104,0,c1089834,1f9,8,...) at mi_switch+0x1c9/frame 0xefc4ac08 sleepq_switch(c97425e0,0,c1089834,26f,c97425e0,...) at sleepq_switch+0x1b9/frame 0xefc4ac3c sleepq_wait(c974cb2c,0,efc4ac90,1,0,...) at sleepq_wait+0x48/frame 0xefc4ac60 _cv_wait(c974cb2c,c974cc74,c1061c77,73,c974cb34,...) at _cv_wait+0x1f9/frame 0xefc4aca4 usb_process(c974cb24,efc4ad08,c107d928,3d7,0,...) at usb_process+0xe6/frame 0xefc4accc fork_exit(c0957fb0,c974cb24,efc4ad08) at fork_exit+0x7f/frame 0xefc4acf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc4acf4 --- trap 0, eip = 0, esp = 0xefc4ad40, ebp = 0 --- Tracing command usb pid 15 tid 100048 td 0xc97428d0 sched_switch(c97428d0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefc47bd4 mi_switch(104,0,c1089834,1f9,8,...) at mi_switch+0x1c9/frame 0xefc47c08 sleepq_switch(c97428d0,0,c1089834,26f,c97428d0,...) at sleepq_switch+0x1b9/frame 0xefc47c3c sleepq_wait(c974cafc,0,efc47c90,1,0,...) at sleepq_wait+0x48/frame 0xefc47c60 _cv_wait(c974cafc,c974cc74,c1061c77,73,c974cb04,...) at _cv_wait+0x1f9/frame 0xefc47ca4 usb_process(c974caf4,efc47d08,c107d928,3d7,0,...) at usb_process+0xe6/frame 0xefc47ccc fork_exit(c0957fb0,c974caf4,efc47d08) at fork_exit+0x7f/frame 0xefc47cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc47cf4 --- trap 0, eip = 0, esp = 0xefc47d40, ebp = 0 --- Tracing command usb pid 15 tid 100047 td 0xc9742bc0 sched_switch(c9742bc0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefc44bd4 mi_switch(104,0,c1089834,1f9,8,...) at mi_switch+0x1c9/frame 0xefc44c08 sleepq_switch(c9742bc0,0,c1089834,26f,c9742bc0,...) at sleepq_switch+0x1b9/frame 0xefc44c3c sleepq_wait(c974cacc,0,efc44c90,1,0,...) at sleepq_wait+0x48/frame 0xefc44c60 _cv_wait(c974cacc,c974cc74,c1061c77,73,c974cad4,...) at _cv_wait+0x1f9/frame 0xefc44ca4 usb_process(c974cac4,efc44d08,c107d928,3d7,0,...) at usb_process+0xe6/frame 0xefc44ccc fork_exit(c0957fb0,c974cac4,efc44d08) at fork_exit+0x7f/frame 0xefc44cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc44cf4 --- trap 0, eip = 0, esp = 0xefc44d40, ebp = 0 --- Tracing command usb pid 15 tid 100046 td 0xc9743000 sched_switch(c9743000,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefc40bd4 mi_switch(104,0,c1089834,1f9,8,...) at mi_switch+0x1c9/frame 0xefc40c08 sleepq_switch(c9743000,0,c1089834,26f,c9743000,...) at sleepq_switch+0x1b9/frame 0xefc40c3c sleepq_wait(c9747b5c,0,efc40c90,1,0,...) at sleepq_wait+0x48/frame 0xefc40c60 _cv_wait(c9747b5c,c9747c74,c1061c77,73,c9747b64,...) at _cv_wait+0x1f9/frame 0xefc40ca4 usb_process(c9747b54,efc40d08,c107d928,3d7,0,...) at usb_process+0xe6/frame 0xefc40ccc fork_exit(c0957fb0,c9747b54,efc40d08) at fork_exit+0x7f/frame 0xefc40cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc40cf4 --- trap 0, eip = 0, esp = 0xefc40d40, ebp = 0 --- Tracing command usb pid 15 tid 100045 td 0xc97432f0 sched_switch(c97432f0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefc3dbd4 mi_switch(104,0,c1089834,1f9,8,...) at mi_switch+0x1c9/frame 0xefc3dc08 sleepq_switch(c97432f0,0,c1089834,26f,c97432f0,...) at sleepq_switch+0x1b9/frame 0xefc3dc3c sleepq_wait(c9747b2c,0,efc3dc90,1,0,...) at sleepq_wait+0x48/frame 0xefc3dc60 _cv_wait(c9747b2c,c9747c74,c1061c77,73,c9747b34,...) at _cv_wait+0x1f9/frame 0xefc3dca4 usb_process(c9747b24,efc3dd08,c107d928,3d7,0,...) at usb_process+0xe6/frame 0xefc3dccc fork_exit(c0957fb0,c9747b24,efc3dd08) at fork_exit+0x7f/frame 0xefc3dcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc3dcf4 --- trap 0, eip = 0, esp = 0xefc3dd40, ebp = 0 --- Tracing command usb pid 15 tid 100044 td 0xc97435e0 sched_switch(c97435e0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefc3abd4 mi_switch(104,0,c1089834,1f9,8,...) at mi_switch+0x1c9/frame 0xefc3ac08 sleepq_switch(c97435e0,0,c1089834,26f,c97435e0,...) at sleepq_switch+0x1b9/frame 0xefc3ac3c sleepq_wait(c9747afc,0,efc3ac90,1,0,...) at sleepq_wait+0x48/frame 0xefc3ac60 _cv_wait(c9747afc,c9747c74,c1061c77,73,c9747b04,...) at _cv_wait+0x1f9/frame 0xefc3aca4 usb_process(c9747af4,efc3ad08,c107d928,3d7,0,...) at usb_process+0xe6/frame 0xefc3accc fork_exit(c0957fb0,c9747af4,efc3ad08) at fork_exit+0x7f/frame 0xefc3acf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc3acf4 --- trap 0, eip = 0, esp = 0xefc3ad40, ebp = 0 --- Tracing command usb pid 15 tid 100043 td 0xc97438d0 sched_switch(c97438d0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefc37bd4 mi_switch(104,0,c1089834,1f9,8,...) at mi_switch+0x1c9/frame 0xefc37c08 sleepq_switch(c97438d0,0,c1089834,26f,c97438d0,...) at sleepq_switch+0x1b9/frame 0xefc37c3c sleepq_wait(c9747acc,0,efc37c90,1,0,...) at sleepq_wait+0x48/frame 0xefc37c60 _cv_wait(c9747acc,c9747c74,c1061c77,73,c9747ad4,...) at _cv_wait+0x1f9/frame 0xefc37ca4 usb_process(c9747ac4,efc37d08,c107d928,3d7,0,...) at usb_process+0xe6/frame 0xefc37ccc fork_exit(c0957fb0,c9747ac4,efc37d08) at fork_exit+0x7f/frame 0xefc37cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc37cf4 --- trap 0, eip = 0, esp = 0xefc37d40, ebp = 0 --- Tracing command usb pid 15 tid 100041 td 0xc9746000 sched_switch(c9746000,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefc30bd4 mi_switch(104,0,c1089834,1f9,8,...) at mi_switch+0x1c9/frame 0xefc30c08 sleepq_switch(c9746000,0,c1089834,26f,c9746000,...) at sleepq_switch+0x1b9/frame 0xefc30c3c sleepq_wait(c973eb5c,0,efc30c90,1,0,...) at sleepq_wait+0x48/frame 0xefc30c60 _cv_wait(c973eb5c,c973ec74,c1061c77,73,c973eb64,...) at _cv_wait+0x1f9/frame 0xefc30ca4 usb_process(c973eb54,efc30d08,c107d928,3d7,0,...) at usb_process+0xe6/frame 0xefc30ccc fork_exit(c0957fb0,c973eb54,efc30d08) at fork_exit+0x7f/frame 0xefc30cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc30cf4 --- trap 0, eip = 0, esp = 0xefc30d40, ebp = 0 --- Tracing command usb pid 15 tid 100040 td 0xc9719000 sched_switch(c9719000,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefc2dbd4 mi_switch(104,0,c1089834,1f9,8,...) at mi_switch+0x1c9/frame 0xefc2dc08 sleepq_switch(c9719000,0,c1089834,26f,c9719000,...) at sleepq_switch+0x1b9/frame 0xefc2dc3c sleepq_wait(c973eb2c,0,efc2dc90,1,0,...) at sleepq_wait+0x48/frame 0xefc2dc60 _cv_wait(c973eb2c,c973ec74,c1061c77,73,c973eb34,...) at _cv_wait+0x1f9/frame 0xefc2dca4 usb_process(c973eb24,efc2dd08,c107d928,3d7,0,...) at usb_process+0xe6/frame 0xefc2dccc fork_exit(c0957fb0,c973eb24,efc2dd08) at fork_exit+0x7f/frame 0xefc2dcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc2dcf4 --- trap 0, eip = 0, esp = 0xefc2dd40, ebp = 0 --- Tracing command usb pid 15 tid 100039 td 0xc97192f0 sched_switch(c97192f0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefc2abd4 mi_switch(104,0,c1089834,1f9,8,...) at mi_switch+0x1c9/frame 0xefc2ac08 sleepq_switch(c97192f0,0,c1089834,26f,c97192f0,...) at sleepq_switch+0x1b9/frame 0xefc2ac3c sleepq_wait(c973eafc,0,efc2ac90,1,0,...) at sleepq_wait+0x48/frame 0xefc2ac60 _cv_wait(c973eafc,c973ec74,c1061c77,73,c973eb04,...) at _cv_wait+0x1f9/frame 0xefc2aca4 usb_process(c973eaf4,efc2ad08,c107d928,3d7,0,...) at usb_process+0xe6/frame 0xefc2accc fork_exit(c0957fb0,c973eaf4,efc2ad08) at fork_exit+0x7f/frame 0xefc2acf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc2acf4 --- trap 0, eip = 0, esp = 0xefc2ad40, ebp = 0 --- Tracing command usb pid 15 tid 100038 td 0xc97195e0 sched_switch(c97195e0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefc27bd4 mi_switch(104,0,c1089834,1f9,8,...) at mi_switch+0x1c9/frame 0xefc27c08 sleepq_switch(c97195e0,0,c1089834,26f,c97195e0,...) at sleepq_switch+0x1b9/frame 0xefc27c3c sleepq_wait(c973eacc,0,efc27c90,1,0,...) at sleepq_wait+0x48/frame 0xefc27c60 _cv_wait(c973eacc,c973ec74,c1061c77,73,c973ead4,...) at _cv_wait+0x1f9/frame 0xefc27ca4 usb_process(c973eac4,efc27d08,c107d928,3d7,0,...) at usb_process+0xe6/frame 0xefc27ccc fork_exit(c0957fb0,c973eac4,efc27d08) at fork_exit+0x7f/frame 0xefc27cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc27cf4 --- trap 0, eip = 0, esp = 0xefc27d40, ebp = 0 --- Tracing command usb pid 15 tid 100036 td 0xc9719bc0 sched_switch(c9719bc0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefc20bd4 mi_switch(104,0,c1089834,1f9,8,...) at mi_switch+0x1c9/frame 0xefc20c08 sleepq_switch(c9719bc0,0,c1089834,26f,c9719bc0,...) at sleepq_switch+0x1b9/frame 0xefc20c3c sleepq_wait(c9735b5c,0,efc20c90,1,0,...) at sleepq_wait+0x48/frame 0xefc20c60 _cv_wait(c9735b5c,c9735c74,c1061c77,73,c9735b64,...) at _cv_wait+0x1f9/frame 0xefc20ca4 usb_process(c9735b54,efc20d08,c107d928,3d7,0,...) at usb_process+0xe6/frame 0xefc20ccc fork_exit(c0957fb0,c9735b54,efc20d08) at fork_exit+0x7f/frame 0xefc20cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc20cf4 --- trap 0, eip = 0, esp = 0xefc20d40, ebp = 0 --- Tracing command usb pid 15 tid 100035 td 0xc971b000 sched_switch(c971b000,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefc1dbd4 mi_switch(104,0,c1089834,1f9,8,...) at mi_switch+0x1c9/frame 0xefc1dc08 sleepq_switch(c971b000,0,c1089834,26f,c971b000,...) at sleepq_switch+0x1b9/frame 0xefc1dc3c sleepq_wait(c9735b2c,0,efc1dc90,1,0,...) at sleepq_wait+0x48/frame 0xefc1dc60 _cv_wait(c9735b2c,c9735c74,c1061c77,73,c9735b34,...) at _cv_wait+0x1f9/frame 0xefc1dca4 usb_process(c9735b24,efc1dd08,c107d928,3d7,0,...) at usb_process+0xe6/frame 0xefc1dccc fork_exit(c0957fb0,c9735b24,efc1dd08) at fork_exit+0x7f/frame 0xefc1dcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc1dcf4 --- trap 0, eip = 0, esp = 0xefc1dd40, ebp = 0 --- Tracing command usb pid 15 tid 100034 td 0xc971b2f0 sched_switch(c971b2f0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefc1abd4 mi_switch(104,0,c1089834,1f9,8,...) at mi_switch+0x1c9/frame 0xefc1ac08 sleepq_switch(c971b2f0,0,c1089834,26f,c971b2f0,...) at sleepq_switch+0x1b9/frame 0xefc1ac3c sleepq_wait(c9735afc,0,efc1ac90,1,0,...) at sleepq_wait+0x48/frame 0xefc1ac60 _cv_wait(c9735afc,c9735c74,c1061c77,73,c9735b04,...) at _cv_wait+0x1f9/frame 0xefc1aca4 usb_process(c9735af4,efc1ad08,c107d928,3d7,0,...) at usb_process+0xe6/frame 0xefc1accc fork_exit(c0957fb0,c9735af4,efc1ad08) at fork_exit+0x7f/frame 0xefc1acf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc1acf4 --- trap 0, eip = 0, esp = 0xefc1ad40, ebp = 0 --- Tracing command usb pid 15 tid 100033 td 0xc971b5e0 sched_switch(c971b5e0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefc17bd4 mi_switch(104,0,c1089834,1f9,8,...) at mi_switch+0x1c9/frame 0xefc17c08 sleepq_switch(c971b5e0,0,c1089834,26f,c971b5e0,...) at sleepq_switch+0x1b9/frame 0xefc17c3c sleepq_wait(c9735acc,0,efc17c90,1,0,...) at sleepq_wait+0x48/frame 0xefc17c60 _cv_wait(c9735acc,c9735c74,c1061c77,73,c9735ad4,...) at _cv_wait+0x1f9/frame 0xefc17ca4 usb_process(c9735ac4,efc17d08,c107d928,3d7,0,...) at usb_process+0xe6/frame 0xefc17ccc fork_exit(c0957fb0,c9735ac4,efc17d08) at fork_exit+0x7f/frame 0xefc17cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc17cf4 --- trap 0, eip = 0, esp = 0xefc17d40, ebp = 0 --- Tracing command ctl_thrd pid 2 tid 100028 td 0xc81498d0 sched_switch(c81498d0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xc7b9264c mi_switch(104,0,c1089834,1f9,166,...) at mi_switch+0x1c9/frame 0xc7b92680 sleepq_switch(c81498d0,0,c1089834,26f,c81498d0,...) at sleepq_switch+0x1b9/frame 0xc7b926b4 sleepq_wait(c8236000,5c,c0edf8bf,0,0,...) at sleepq_wait+0x48/frame 0xc7b926d8 _sleep(c8236000,c8236000,5c,c0edf8bf,0,...) at _sleep+0x34e/frame 0xc7b92730 ctl_work_thread(c8236000,c7b92d08,c107d928,3d7,0,...) at ctl_work_thread+0x110/frame 0xc7b92ccc fork_exit(c04d5970,c8236000,c7b92d08) at fork_exit+0x7f/frame 0xc7b92cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b92cf4 --- trap 0, eip = 0, esp = 0xc7b92d40, ebp = 0 --- Tracing command yarrow pid 14 tid 100017 td 0xc81485e0 sched_switch(c81485e0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xc7b71b90 mi_switch(104,0,c1089834,1f9,2,...) at mi_switch+0x1c9/frame 0xc7b71bc4 sleepq_switch(c81485e0,0,c1089834,292,c81485e0,...) at sleepq_switch+0x1b9/frame 0xc7b71bf8 sleepq_timedwait(c12320f0,0,c10703ea,2,0,...) at sleepq_timedwait+0x48/frame 0xc7b71c1c _sleep(c12320f0,0,0,c10703ea,64) at _sleep+0x317/frame 0xc7b71c74 pause(c10703ea,64,c103da7e,111,0,...) at pause+0x85/frame 0xc7b71c98 random_kthread(0,c7b71d08,c107d928,3d7,0,...) at random_kthread+0x30f/frame 0xc7b71ccc fork_exit(c0848500,0,c7b71d08) at fork_exit+0x7f/frame 0xc7b71cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b71cf4 --- trap 0, eip = 0, esp = 0xc7b71d40, ebp = 0 --- Tracing command geom pid 13 tid 100015 td 0xc8148bc0 sched_switch(c8148bc0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xc7b6ab78 mi_switch(104,0,c1089834,1f9,166,...) at mi_switch+0x1c9/frame 0xc7b6abac sleepq_switch(c8148bc0,0,c1089834,26f,c8148bc0,...) at sleepq_switch+0x1b9/frame 0xc7b6abe0 sleepq_wait(c13f9918,5c,c10703ea,0,0,...) at sleepq_wait+0x48/frame 0xc7b6ac04 _sleep(c13f9918,c1230440,25c,c10703ea,0,...) at _sleep+0x34e/frame 0xc7b6ac5c g_io_schedule_down(c8148bc0,0,c1071ea3,6c,c7b6acf4,...) at g_io_schedule_down+0x3c/frame 0xc7b6acb4 g_down_procbody(0,c7b6ad08,c107d928,3d7,0,...) at g_down_procbody+0x9d/frame 0xc7b6accc fork_exit(c0a043f0,0,c7b6ad08) at fork_exit+0x7f/frame 0xc7b6acf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b6acf4 --- trap 0, eip = 0, esp = 0xc7b6ad40, ebp = 0 --- Tracing command geom pid 13 tid 100014 td 0xc8149000 sched_switch(c8149000,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xc7b67bac mi_switch(104,0,c1089834,1f9,166,...) at mi_switch+0x1c9/frame 0xc7b67be0 sleepq_switch(c8149000,0,c1089834,26f,c8149000,...) at sleepq_switch+0x1b9/frame 0xc7b67c14 sleepq_wait(c13f9914,5c,c10703ea,0,0,...) at sleepq_wait+0x48/frame 0xc7b67c38 _sleep(c13f9914,c1230460,25c,c10703ea,0,...) at _sleep+0x34e/frame 0xc7b67c90 g_io_schedule_up(c8149000,0,c1071ea3,5f,c7b67cf4,...) at g_io_schedule_up+0x8f/frame 0xc7b67cb4 g_up_procbody(0,c7b67d08,c107d928,3d7,0,...) at g_up_procbody+0x9d/frame 0xc7b67ccc fork_exit(c0a04350,0,c7b67d08) at fork_exit+0x7f/frame 0xc7b67cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b67cf4 --- trap 0, eip = 0, esp = 0xc7b67d40, ebp = 0 --- Tracing command geom pid 13 tid 100013 td 0xc7dfc2f0 sched_switch(c7dfc2f0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xc7b64bbc mi_switch(104,0,c1089834,1f9,166,...) at mi_switch+0x1c9/frame 0xc7b64bf0 sleepq_switch(c7dfc2f0,0,c1089834,26f,c7dfc2f0,...) at sleepq_switch+0x1b9/frame 0xc7b64c24 sleepq_wait(c13f9910,5c,c10703ea,0,0,...) at sleepq_wait+0x48/frame 0xc7b64c48 _sleep(c13f9910,c1230418,25c,c10703ea,0,...) at _sleep+0x34e/frame 0xc7b64ca0 g_run_events(0,c7b64d08,c107d928,3d7,0,...) at g_run_events+0x4f3/frame 0xc7b64ccc fork_exit(c0a042b0,0,c7b64d08) at fork_exit+0x7f/frame 0xc7b64cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b64cf4 --- trap 0, eip = 0, esp = 0xc7b64d40, ebp = 0 --- Tracing command intr pid 12 tid 100067 td 0xc98875e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100066 td 0xc98878d0 sched_switch(c98878d0,0,109,1b3,0,...) at sched_switch+0x4ba/frame 0xefca0c64 mi_switch(109,0,c107dbc5,563,c98878d0,...) at mi_switch+0x1c9/frame 0xefca0c98 ithread_loop(c7df5970,efca0d08,c107d928,3d7,0,...) at ithread_loop+0x100/frame 0xefca0ccc fork_exit(c0a74480,c7df5970,efca0d08) at fork_exit+0x7f/frame 0xefca0cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefca0cf4 --- trap 0, eip = 0, esp = 0xefca0d40, ebp = 0 --- Tracing command intr pid 12 tid 100064 td 0xc9888000 sched_switch(c9888000,0,109,1b3,0,...) at sched_switch+0x4ba/frame 0xefc90c64 mi_switch(109,0,c107dbc5,563,c9888000,...) at mi_switch+0x1c9/frame 0xefc90c98 ithread_loop(c7df6130,efc90d08,c107d928,3d7,0,...) at ithread_loop+0x100/frame 0xefc90ccc fork_exit(c0a74480,c7df6130,efc90d08) at fork_exit+0x7f/frame 0xefc90cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc90cf4 --- trap 0, eip = 0, esp = 0xefc90d40, ebp = 0 --- Tracing command intr pid 12 tid 100061 td 0xc97468d0 sched_switch(c97468d0,0,109,1b3,0,...) at sched_switch+0x4ba/frame 0xefc84c64 mi_switch(109,0,c107dbc5,563,c97468d0,...) at mi_switch+0x1c9/frame 0xefc84c98 ithread_loop(c7df5980,efc84d08,c107d928,3d7,0,...) at ithread_loop+0x100/frame 0xefc84ccc fork_exit(c0a74480,c7df5980,efc84d08) at fork_exit+0x7f/frame 0xefc84cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc84cf4 --- trap 0, eip = 0, esp = 0xefc84d40, ebp = 0 --- Tracing command intr pid 12 tid 100060 td 0xc9746bc0 sched_switch(c9746bc0,0,109,1b3,0,...) at sched_switch+0x4ba/frame 0xefc72c64 mi_switch(109,0,c107dbc5,563,c9746bc0,...) at mi_switch+0x1c9/frame 0xefc72c98 ithread_loop(c7df5910,efc72d08,c107d928,3d7,0,...) at ithread_loop+0x100/frame 0xefc72ccc fork_exit(c0a74480,c7df5910,efc72d08) at fork_exit+0x7f/frame 0xefc72cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc72cf4 --- trap 0, eip = 0, esp = 0xefc72d40, ebp = 0 --- Tracing command intr pid 12 tid 100055 td 0xc9754bc0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100042 td 0xc9743bc0 sched_switch(c9743bc0,0,109,1b3,0,...) at sched_switch+0x4ba/frame 0xefc34c64 mi_switch(109,0,c107dbc5,563,c9743bc0,...) at mi_switch+0x1c9/frame 0xefc34c98 ithread_loop(c7df6150,efc34d08,c107d928,3d7,0,...) at ithread_loop+0x100/frame 0xefc34ccc fork_exit(c0a74480,c7df6150,efc34d08) at fork_exit+0x7f/frame 0xefc34cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc34cf4 --- trap 0, eip = 0, esp = 0xefc34d40, ebp = 0 --- Tracing command intr pid 12 tid 100037 td 0xc97198d0 sched_switch(c97198d0,0,109,1b3,0,...) at sched_switch+0x4ba/frame 0xefc24c64 mi_switch(109,0,c107dbc5,563,c97198d0,...) at mi_switch+0x1c9/frame 0xefc24c98 ithread_loop(c7df59d0,efc24d08,c107d928,3d7,0,...) at ithread_loop+0x100/frame 0xefc24ccc fork_exit(c0a74480,c7df59d0,efc24d08) at fork_exit+0x7f/frame 0xefc24cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc24cf4 --- trap 0, eip = 0, esp = 0xefc24d40, ebp = 0 --- Tracing command intr pid 12 tid 100032 td 0xc971b8d0 sched_switch(c971b8d0,0,109,1b3,0,...) at sched_switch+0x4ba/frame 0xefc14c64 mi_switch(109,0,c107dbc5,563,c971b8d0,...) at mi_switch+0x1c9/frame 0xefc14c98 ithread_loop(c7df5900,efc14d08,c107d928,3d7,0,...) at ithread_loop+0x100/frame 0xefc14ccc fork_exit(c0a74480,c7df5900,efc14d08) at fork_exit+0x7f/frame 0xefc14cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc14cf4 --- trap 0, eip = 0, esp = 0xefc14d40, ebp = 0 --- Tracing command intr pid 12 tid 100031 td 0xc971bbc0 sched_switch(c971bbc0,0,109,1b3,0,...) at sched_switch+0x4ba/frame 0xefabec64 mi_switch(109,0,c107dbc5,563,c971bbc0,...) at mi_switch+0x1c9/frame 0xefabec98 ithread_loop(c7df5800,efabed08,c107d928,3d7,0,...) at ithread_loop+0x100/frame 0xefabeccc fork_exit(c0a74480,c7df5800,efabed08) at fork_exit+0x7f/frame 0xefabecf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefabecf4 --- trap 0, eip = 0, esp = 0xefabed40, ebp = 0 --- Tracing command intr pid 12 tid 100030 td 0xc81492f0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100029 td 0xc81495e0 sched_switch(c81495e0,0,109,1b3,0,...) at sched_switch+0x4ba/frame 0xc7bc4c64 mi_switch(109,0,c107dbc5,563,c81495e0,...) at mi_switch+0x1c9/frame 0xc7bc4c98 ithread_loop(c7df5a90,c7bc4d08,c107d928,3d7,0,...) at ithread_loop+0x100/frame 0xc7bc4ccc fork_exit(c0a74480,c7df5a90,c7bc4d08) at fork_exit+0x7f/frame 0xc7bc4cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7bc4cf4 --- trap 0, eip = 0, esp = 0xc7bc4d40, ebp = 0 --- Tracing command intr pid 12 tid 100026 td 0xc8235000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100025 td 0xc82352f0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100020 td 0xc7e40bc0 sched_switch(c7e40bc0,0,109,1b3,0,...) at sched_switch+0x4ba/frame 0xc7b7ac64 mi_switch(109,0,c107dbc5,563,c7e40bc0,...) at mi_switch+0x1c9/frame 0xc7b7ac98 ithread_loop(c7df5b30,c7b7ad08,c107d928,3d7,0,...) at ithread_loop+0x100/frame 0xc7b7accc fork_exit(c0a74480,c7df5b30,c7b7ad08) at fork_exit+0x7f/frame 0xc7b7acf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b7acf4 --- trap 0, eip = 0, esp = 0xc7b7ad40, ebp = 0 --- Tracing command intr pid 12 tid 100018 td 0xc81482f0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100012 td 0xc7dfc5e0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100011 td 0xc7dfc8d0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100010 td 0xc7dfcbc0 sched_switch(c7dfcbc0,0,109,1b3,c107dbc5,...) at sched_switch+0x4ba/frame 0xc7b5bc64 mi_switch(109,0,c107dbc5,563,c7dfcbc0,...) at mi_switch+0x1c9/frame 0xc7b5bc98 ithread_loop(c7df5ba0,c7b5bd08,c107d928,3d7,0,...) at ithread_loop+0x100/frame 0xc7b5bccc fork_exit(c0a74480,c7df5ba0,c7b5bd08) at fork_exit+0x7f/frame 0xc7b5bcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b5bcf4 --- trap 0, eip = 0, esp = 0xc7b5bd40, ebp = 0 --- Tracing command intr pid 12 tid 100009 td 0xc7e40000 sched_switch(c7e40000,0,109,1b3,c107dbc5,...) at sched_switch+0x4ba/frame 0xc7b58c64 mi_switch(109,0,c107dbc5,563,c7e40000,...) at mi_switch+0x1c9/frame 0xc7b58c98 ithread_loop(c7df5bb0,c7b58d08,c107d928,3d7,0,...) at ithread_loop+0x100/frame 0xc7b58ccc fork_exit(c0a74480,c7df5bb0,c7b58d08) at fork_exit+0x7f/frame 0xc7b58cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b58cf4 --- trap 0, eip = 0, esp = 0xc7b58d40, ebp = 0 --- Tracing command intr pid 12 tid 100008 td 0xc7e402f0 sched_switch(c7e402f0,0,109,1b3,c107dbc5,...) at sched_switch+0x4ba/frame 0xc7b55c64 mi_switch(109,0,c107dbc5,563,c7e402f0,...) at mi_switch+0x1c9/frame 0xc7b55c98 ithread_loop(c7df5bc0,c7b55d08,c107d928,3d7,0,...) at ithread_loop+0x100/frame 0xc7b55ccc fork_exit(c0a74480,c7df5bc0,c7b55d08) at fork_exit+0x7f/frame 0xc7b55cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b55cf4 --- trap 0, eip = 0, esp = 0xc7b55d40, ebp = 0 --- Tracing command intr pid 12 tid 100007 td 0xc7e405e0 sched_switch(c7e405e0,0,109,1b3,0,...) at sched_switch+0x4ba/frame 0xc7b52c64 mi_switch(109,0,c107dbc5,563,c7e405e0,...) at mi_switch+0x1c9/frame 0xc7b52c98 ithread_loop(c7df5bd0,c7b52d08,c107d928,3d7,0,...) at ithread_loop+0x100/frame 0xc7b52ccc fork_exit(c0a74480,c7df5bd0,c7b52d08) at fork_exit+0x7f/frame 0xc7b52cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b52cf4 --- trap 0, eip = 0, esp = 0xc7b52d40, ebp = 0 --- Tracing command idle pid 11 tid 100006 td 0xc7dfb000 cpustop_handler(3,13,c7b4eba8,c0ea61cb,c7b4eaa8,...) at cpustop_handler+0x2f/frame 0xc7b4ea58 ipi_nmi_handler(c7b4eaa8,404b44,c7b4eb68,5f590101,c7b4ead8,...) at ipi_nmi_handler+0x37/frame 0xc7b4ea68 trap(c7b4ebb4) at trap+0x3b/frame 0xc7b4eba8 calltrap() at calltrap+0x6/frame 0xc7b4eba8 --- trap 0x13, eip = 0xc0e8baf2, esp = 0xc7b4ebf4, ebp = 0xc7b4ec34 --- acpi_cpu_c1(0,c7b4ec50,c0e95d3d,0,9,...) at acpi_cpu_c1+0x2/frame 0xc7b4ec34 cpu_idle_acpi(0,9,c7b4eccc,c0ac926a,0,...) at cpu_idle_acpi+0x2f/frame 0xc7b4ec40 cpu_idle(0,2,c1086275,a46,c1240988,...) at cpu_idle+0x7d/frame 0xc7b4ec50 sched_idletd(0,c7b4ed08,c107d928,3d7,0,...) at sched_idletd+0x1fa/frame 0xc7b4eccc fork_exit(c0ac9070,0,c7b4ed08) at fork_exit+0x7f/frame 0xc7b4ecf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b4ecf4 --- trap 0, eip = 0, esp = 0xc7b4ed40, ebp = 0 --- Tracing command idle pid 11 tid 100005 td 0xc7dfb2f0 cpustop_handler(2,13,c7b4bba8,c0ea61cb,8,...) at cpustop_handler+0x2f/frame 0xc7b4ba58 ipi_nmi_handler(8,c107dbc5,39e,2,c7dfb2f0,...) at ipi_nmi_handler+0x37/frame 0xc7b4ba68 trap(c7b4bbb4) at trap+0x3b/frame 0xc7b4bba8 calltrap() at calltrap+0x6/frame 0xc7b4bba8 --- trap 0x13, eip = 0xc0e8baf2, esp = 0xc7b4bbf4, ebp = 0xc7b4bc34 --- acpi_cpu_c1(0,c7b4bc50,c0e95d3d,0,e,...) at acpi_cpu_c1+0x2/frame 0xc7b4bc34 cpu_idle_acpi(0,e,c7b4bccc,c0ac926a,0,...) at cpu_idle_acpi+0x2f/frame 0xc7b4bc40 cpu_idle(0,2,c1086275,a46,c1240288,...) at cpu_idle+0x7d/frame 0xc7b4bc50 sched_idletd(0,c7b4bd08,c107d928,3d7,0,...) at sched_idletd+0x1fa/frame 0xc7b4bccc fork_exit(c0ac9070,0,c7b4bd08) at fork_exit+0x7f/frame 0xc7b4bcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b4bcf4 --- trap 0, eip = 0, esp = 0xc7b4bd40, ebp = 0 --- Tracing command idle pid 11 tid 100004 td 0xc7dfb5e0 sched_switch(c7dfb5e0,0,608,1ae,c7b48c40,...) at sched_switch+0x4ba/frame 0xc7b48c00 mi_switch(608,0,c10844a4,d3,6,...) at mi_switch+0x1c9/frame 0xc7b48c34 critical_exit(0,2,c1086275,a46,c123fb88,...) at critical_exit+0xa3/frame 0xc7b48c50 sched_idletd(0,c7b48d08,c107d928,3d7,0,...) at sched_idletd+0x1fa/frame 0xc7b48ccc fork_exit(c0ac9070,0,c7b48d08) at fork_exit+0x7f/frame 0xc7b48cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b48cf4 --- trap 0, eip = 0, esp = 0xc7b48d40, ebp = 0 --- Tracing command idle pid 11 tid 100003 td 0xc7dfb8d0 cpustop_handler(0,13,c7b45ba8,c0ea61cb,c7b45a80,...) at cpustop_handler+0x2f/frame 0xc7b45a58 ipi_nmi_handler(c7b45a80,c0ab1f03,c7b45aa0,c10f9928,c7b45b08,...) at ipi_nmi_handler+0x37/frame 0xc7b45a68 trap(c7b45bb4) at trap+0x3b/frame 0xc7b45ba8 calltrap() at calltrap+0x6/frame 0xc7b45ba8 --- trap 0x13, eip = 0xc0e8baf2, esp = 0xc7b45bf4, ebp = 0xc7b45c34 --- acpi_cpu_c1(0,c7b45c50,c0e95d3d,0,e,...) at acpi_cpu_c1+0x2/frame 0xc7b45c34 cpu_idle_acpi(0,e,c7b45ccc,c0ac926a,0,...) at cpu_idle_acpi+0x2f/frame 0xc7b45c40 cpu_idle(0,2,c1086275,409,c123f488,...) at cpu_idle+0x7d/frame 0xc7b45c50 sched_idletd(0,c7b45d08,c107d928,3d7,0,...) at sched_idletd+0x1fa/frame 0xc7b45ccc fork_exit(c0ac9070,0,c7b45d08) at fork_exit+0x7f/frame 0xc7b45cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b45cf4 --- trap 0, eip = 0, esp = 0xc7b45d40, ebp = 0 --- Tracing command init pid 1 tid 100002 td 0xc7dfbbc0 sched_switch(c7dfbbc0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xc7b41978 mi_switch(104,0,c1089834,1f9,8,...) at mi_switch+0x1c9/frame 0xc7b419ac sleepq_switch(c7dfbbc0,0,c1089834,1ad,c7b41a24,...) at sleepq_switch+0x1b9/frame 0xc7b419e0 sleepq_catch_signals(0,c1089834,166,288,c7dfbbc0,...) at sleepq_catch_signals+0x3b1/frame 0xc7b41a28 sleepq_wait_sig(c7df9b40,6c,c108cce9,100,0,...) at sleepq_wait_sig+0x14/frame 0xc7b41a4c _sleep(c7df9b40,c7df9bc8,16c,c108cce9,0,...) at _sleep+0x32d/frame 0xc7b41aa4 kern_wait6(c7dfbbc0,7,0,0,c7b41bec,...) at kern_wait6+0x3ec/frame 0xc7b41b00 kern_wait(c7dfbbc0,ffffffff,c7b41bec,2,0,...) at kern_wait+0xd8/frame 0xc7b41bd0 sys_wait4(c7dfbbc0,c7b41cc8,c7b41d08,1,1,...) at sys_wait4+0x41/frame 0xc7b41c40 syscall(c7b41d08) at syscall+0x2ed/frame 0xc7b41cfc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xc7b41cfc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x805dce7, esp = 0xbfbfd8bc, ebp = 0xbfbfd8d0 --- Tracing command audit pid 10 tid 100001 td 0xc7dfc000 sched_switch(c7dfc000,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xc7b3eba8 mi_switch(104,0,c1089834,1f9,8,...) at mi_switch+0x1c9/frame 0xc7b3ebdc sleepq_switch(c7dfc000,0,c1089834,26f,c7dfc000,...) at sleepq_switch+0x1b9/frame 0xc7b3ec10 sleepq_wait(c14005c8,0,c7b3ec64,1,0,...) at sleepq_wait+0x48/frame 0xc7b3ec34 _cv_wait(c14005c8,c1400554,c10b4a42,18d,0,...) at _cv_wait+0x1f9/frame 0xc7b3ec78 audit_worker(0,c7b3ed08,c107d928,3d7,0,...) at audit_worker+0x94/frame 0xc7b3eccc fork_exit(c0ca92a0,0,c7b3ed08) at fork_exit+0x7f/frame 0xc7b3ecf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b3ecf4 --- trap 0, eip = 0, esp = 0xc7b3ed40, ebp = 0 --- Tracing command kernel pid 0 tid 100069 td 0xc9887000 sched_switch(c9887000,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefca9b8c mi_switch(104,0,c1089834,1f9,1,...) at mi_switch+0x1c9/frame 0xefca9bc0 sleepq_switch(c9887000,0,c1089834,292,c9887000,...) at sleepq_switch+0x1b9/frame 0xefca9bf4 sleepq_timedwait(c12320f0,0,c10703ea,2,0,...) at sleepq_timedwait+0x48/frame 0xefca9c18 _sleep(c12320f0,0,0,c10703ea,bb8) at _sleep+0x317/frame 0xefca9c70 pause(c10703ea,bb8,12b,129,c13f9994,...) at pause+0x85/frame 0xefca9c94 deadlkres(0,efca9d08,c107d928,3d7,0,...) at deadlkres+0x48/frame 0xefca9ccc fork_exit(c0a539f0,0,efca9d08) at fork_exit+0x7f/frame 0xefca9cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefca9cf4 --- trap 0, eip = 0, esp = 0xefca9d40, ebp = 0 --- Tracing command kernel pid 0 tid 100068 td 0xc98872f0 sched_switch(c98872f0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefca6bd8 mi_switch(104,0,c1089834,1f9,c124e684,...) at mi_switch+0x1c9/frame 0xefca6c0c sleepq_switch(c98872f0,0,c1089834,26f,c9744140,...) at sleepq_switch+0x1b9/frame 0xefca6c40 sleepq_wait(c9744140,0,c1084a45,c10703ea,0,...) at sleepq_wait+0x48/frame 0xefca6c64 msleep_spin(c9744140,c9744158,c10703ea,0,c98872f0,...) at msleep_spin+0x216/frame 0xefca6ca4 taskqueue_thread_loop(c13f6048,efca6d08,c107d928,3d7,0,...) at taskqueue_thread_loop+0x7c/frame 0xefca6ccc fork_exit(c0ae7070,c13f6048,efca6d08) at fork_exit+0x7f/frame 0xefca6cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefca6cf4 --- trap 0, eip = 0, esp = 0xefca6d40, ebp = 0 --- Tracing command kernel pid 0 tid 100062 td 0xc97465e0 sched_switch(c97465e0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xefc8abd8 mi_switch(104,0,c1089834,1f9,c124e714,...) at mi_switch+0x1c9/frame 0xefc8ac0c sleepq_switch(c97465e0,0,c1089834,26f,c9744700,...) at sleepq_switch+0x1b9/frame 0xefc8ac40 sleepq_wait(c9744700,0,c1084a45,c10703ea,0,...) at sleepq_wait+0x48/frame 0xefc8ac64 msleep_spin(c9744700,c9744718,c10703ea,0,c97465e0,...) at msleep_spin+0x216/frame 0xefc8aca4 taskqueue_thread_loop(c978949c,efc8ad08,c107d928,3d7,0,...) at taskqueue_thread_loop+0x7c/frame 0xefc8accc fork_exit(c0ae7070,c978949c,efc8ad08) at fork_exit+0x7f/frame 0xefc8acf4 fork_trampoline() at fork_trampoline+0x8/frame 0xefc8acf4 --- trap 0, eip = 0, esp = 0xefc8ad40, ebp = 0 --- Tracing command kernel pid 0 tid 100027 td 0xc8149bc0 sched_switch(c8149bc0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xc7b8fbc0 mi_switch(104,0,c1089834,1f9,166,...) at mi_switch+0x1c9/frame 0xc7b8fbf4 sleepq_switch(c8149bc0,0,c1089834,26f,c8149bc0,...) at sleepq_switch+0x1b9/frame 0xc7b8fc28 sleepq_wait(c7dc3a00,0,c10703ea,0,0,...) at sleepq_wait+0x48/frame 0xc7b8fc4c _sleep(c7dc3a00,c7dc3a18,0,c10703ea,0,...) at _sleep+0x34e/frame 0xc7b8fca4 taskqueue_thread_loop(c13fcfb8,c7b8fd08,c107d928,3d7,0,...) at taskqueue_thread_loop+0xb9/frame 0xc7b8fccc fork_exit(c0ae7070,c13fcfb8,c7b8fd08) at fork_exit+0x7f/frame 0xc7b8fcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b8fcf4 --- trap 0, eip = 0, esp = 0xc7b8fd40, ebp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xc82355e0 sched_switch(c82355e0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xc7b86bc0 mi_switch(104,0,c1089834,1f9,166,...) at mi_switch+0x1c9/frame 0xc7b86bf4 sleepq_switch(c82355e0,0,c1089834,26f,c82355e0,...) at sleepq_switch+0x1b9/frame 0xc7b86c28 sleepq_wait(c7dc3ac0,0,c10703ea,0,0,...) at sleepq_wait+0x48/frame 0xc7b86c4c _sleep(c7dc3ac0,c7dc3ad8,0,c10703ea,0,...) at _sleep+0x34e/frame 0xc7b86ca4 taskqueue_thread_loop(c1400600,c7b86d08,c107d928,3d7,0,...) at taskqueue_thread_loop+0xb9/frame 0xc7b86ccc fork_exit(c0ae7070,c1400600,c7b86d08) at fork_exit+0x7f/frame 0xc7b86cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b86cf4 --- trap 0, eip = 0, esp = 0xc7b86d40, ebp = 0 --- Tracing command kernel pid 0 tid 100023 td 0xc82358d0 sched_switch(c82358d0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xc7b83bd8 mi_switch(104,0,c1089834,1f9,c124e60c,...) at mi_switch+0x1c9/frame 0xc7b83c0c sleepq_switch(c82358d0,0,c1089834,26f,c7dc3c40,...) at sleepq_switch+0x1b9/frame 0xc7b83c40 sleepq_wait(c7dc3c40,0,c1084a45,c10703ea,0,...) at sleepq_wait+0x48/frame 0xc7b83c64 msleep_spin(c7dc3c40,c7dc3c58,c10703ea,0,c82358d0,...) at msleep_spin+0x216/frame 0xc7b83ca4 taskqueue_thread_loop(c13f6c60,c7b83d08,c107d928,3d7,0,...) at taskqueue_thread_loop+0x7c/frame 0xc7b83ccc fork_exit(c0ae7070,c13f6c60,c7b83d08) at fork_exit+0x7f/frame 0xc7b83cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b83cf4 --- trap 0, eip = 0, esp = 0xc7b83d40, ebp = 0 --- Tracing command kernel pid 0 tid 100022 td 0xc8235bc0 sched_switch(c8235bc0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xc7b80bd8 mi_switch(104,0,c1089834,1f9,c124e60c,...) at mi_switch+0x1c9/frame 0xc7b80c0c sleepq_switch(c8235bc0,0,c1089834,26f,c7dc3c40,...) at sleepq_switch+0x1b9/frame 0xc7b80c40 sleepq_wait(c7dc3c40,0,c1084a45,c10703ea,0,...) at sleepq_wait+0x48/frame 0xc7b80c64 msleep_spin(c7dc3c40,c7dc3c58,c10703ea,0,c8235bc0,...) at msleep_spin+0x216/frame 0xc7b80ca4 taskqueue_thread_loop(c13f6c60,c7b80d08,c107d928,3d7,0,...) at taskqueue_thread_loop+0x7c/frame 0xc7b80ccc fork_exit(c0ae7070,c13f6c60,c7b80d08) at fork_exit+0x7f/frame 0xc7b80cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b80cf4 --- trap 0, eip = 0, esp = 0xc7b80d40, ebp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xc7e408d0 sched_switch(c7e408d0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xc7b7dbd8 mi_switch(104,0,c1089834,1f9,c124e60c,...) at mi_switch+0x1c9/frame 0xc7b7dc0c sleepq_switch(c7e408d0,0,c1089834,26f,c7dc3c40,...) at sleepq_switch+0x1b9/frame 0xc7b7dc40 sleepq_wait(c7dc3c40,0,c1084a45,c10703ea,0,...) at sleepq_wait+0x48/frame 0xc7b7dc64 msleep_spin(c7dc3c40,c7dc3c58,c10703ea,0,c7e408d0,...) at msleep_spin+0x216/frame 0xc7b7dca4 taskqueue_thread_loop(c13f6c60,c7b7dd08,c107d928,3d7,0,...) at taskqueue_thread_loop+0x7c/frame 0xc7b7dccc fork_exit(c0ae7070,c13f6c60,c7b7dd08) at fork_exit+0x7f/frame 0xc7b7dcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b7dcf4 --- trap 0, eip = 0, esp = 0xc7b7dd40, ebp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xc8148000 sched_switch(c8148000,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xc7b77bc0 mi_switch(104,0,c1089834,1f9,166,...) at mi_switch+0x1c9/frame 0xc7b77bf4 sleepq_switch(c8148000,0,c1089834,26f,c8148000,...) at sleepq_switch+0x1b9/frame 0xc7b77c28 sleepq_wait(c7dc3c80,0,c10703ea,0,0,...) at sleepq_wait+0x48/frame 0xc7b77c4c _sleep(c7dc3c80,c7dc3c98,0,c10703ea,0,...) at _sleep+0x34e/frame 0xc7b77ca4 taskqueue_thread_loop(c13fa0e8,c7b77d08,c107d928,3d7,0,...) at taskqueue_thread_loop+0xb9/frame 0xc7b77ccc fork_exit(c0ae7070,c13fa0e8,c7b77d08) at fork_exit+0x7f/frame 0xc7b77cf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b77cf4 --- trap 0, eip = 0, esp = 0xc7b77d40, ebp = 0 --- Tracing command kernel pid 0 tid 100016 td 0xc81488d0 sched_switch(c81488d0,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xc7b6dbc0 mi_switch(104,0,c1089834,1f9,166,...) at mi_switch+0x1c9/frame 0xc7b6dbf4 sleepq_switch(c81488d0,0,c1089834,26f,c81488d0,...) at sleepq_switch+0x1b9/frame 0xc7b6dc28 sleepq_wait(c7dc3e40,0,c10703ea,0,0,...) at sleepq_wait+0x48/frame 0xc7b6dc4c _sleep(c7dc3e40,c7dc3e58,0,c10703ea,0,...) at _sleep+0x34e/frame 0xc7b6dca4 taskqueue_thread_loop(c124d540,c7b6dd08,c107d928,3d7,0,...) at taskqueue_thread_loop+0xb9/frame 0xc7b6dccc fork_exit(c0ae7070,c124d540,c7b6dd08) at fork_exit+0x7f/frame 0xc7b6dcf4 fork_trampoline() at fork_trampoline+0x8/frame 0xc7b6dcf4 --- trap 0, eip = 0, esp = 0xc7b6dd40, ebp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xc13f9c70 sched_switch(c13f9c70,0,104,1b3,c0a8f79e,...) at sched_switch+0x4ba/frame 0xc1820c20 mi_switch(104,0,c1089834,1f9,0,...) at mi_switch+0x1c9/frame 0xc1820c54 sleepq_switch(c13f9c70,0,c1089834,292,c13f9c70,...) at sleepq_switch+0x1b9/frame 0xc1820c88 sleepq_timedwait(c13f9994,54,c10b5c05,0,0,...) at sleepq_timedwait+0x48/frame 0xc1820cac _sleep(c13f9994,0,54,c10b5c05,2710,...) at _sleep+0x317/frame 0xc1820d04 scheduler(0,0,c10e5f2c,181e000,1825000,...) at scheduler+0x274/frame 0xc1820d38 mi_startup() at mi_startup+0xf7/frame 0xc1820d58 begin() at begin+0x2c db:0:allt> call doadump Physical memory: 3435 MB Dumping 149 MB: 134 118 102 86 70 54 38 22 6 Dump complete = 0 db:0:doadump> reset cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 1 (kgdb) bt #0 doadump (textdump=0xa) at pcpu.h:249 #1 0xc05255dd in db_fncall (dummy1=0xf4221734, dummy2=0xf4221734, dummy3=0xf4221734, dummy4=0xf4221734 "\n") at ../../../ddb/db_command.c:578 #2 0xc05252d7 in db_command (cmd_table=) at ../../../ddb/db_command.c:449 #3 0xc0529184 in db_script_exec (scriptname=0xc13f69c8 "doadump", warnifnotfound=) at ../../../ddb/db_script.c:302 #4 0xc05252d7 in db_command (cmd_table=) at ../../../ddb/db_command.c:449 #5 0xc0524ff0 in db_command_loop () at ../../../ddb/db_command.c:502 #6 0xc0527830 in db_trap (type=, code=0xf4221b20) at ../../../ddb/db_main.c:231 #7 0xc0ad82a6 in kdb_trap (type=, code=, tf=) at ../../../kern/subr_kdb.c:654 #8 0xc0ea690e in trap (frame=) at ../../../i386/i386/trap.c:720 #9 0xc0e8fb1c in calltrap () at exception.s:169 #10 0xc0ad7b4d in kdb_enter (why=0xc1083d85 "panic", msg=) at cpufunc.h:71 #11 0xc0aa3a92 in vpanic (fmt=, ap=) at ../../../kern/kern_shutdown.c:746 #12 0xc0aa394a in kassert_panic (fmt=) at ../../../kern/kern_shutdown.c:641 #13 0xc0ae788a in userret (td=0xca96f5e0, frame=) at ../../../kern/subr_trap.c:163 #14 0xc0ea7760 in syscall (frame=) at subr_syscall.c:177 #15 0xc0e8fbb1 in Xint0x80_syscall () at exception.s:267 #16 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) Current language: auto; currently minimal (kgdb) f 13 #13 0xc0ae788a in userret (td=0xca96f5e0, frame=) at ../../../kern/subr_trap.c:163 163 KASSERT(td->td_pinned == 0 || (td->td_pflags & TDP_CALLCHAIN) != 0, (kgdb) p *td $1 = {td_lock = 0xc123fb00, td_proc = 0xc9681b40, td_plist = {tqe_next = 0x0, tqe_prev = 0xc9681b48}, td_runq = {tqe_next = 0x0, tqe_prev = 0xc123fc94}, td_slpq = {tqe_next = 0x0, tqe_prev = 0xc7dd1840}, td_lockq = {tqe_next = 0x0, tqe_prev = 0x0}, td_hash = {le_next = 0x0, le_prev = 0xc7dcfbbc}, td_cpuset = 0xc7dd2e38, td_sel = 0x0, td_sleepqueue = 0xc7dd1840, td_turnstile = 0xc971a480, td_rlqe = 0xcaaea000, td_umtxq = 0xc7df4e80, td_tid = 0x186ef, td_sigqueue = {sq_signals = {__bits = {0x0, 0x0, 0x0, 0x0}}, sq_kill = {__bits = {0x0, 0x0, 0x0, 0x0}}, sq_list = {tqh_first = 0x0, tqh_last = 0xca96f64c}, sq_proc = 0xc9681b40, sq_flags = 0x1}, td_lend_user_pri = 0xff, td_flags = 0x4, td_inhibitors = 0x0, td_pflags = 0x0, td_dupfd = 0x0, td_sqqueue = 0x0, td_wchan = 0x0, td_wmesg = 0x0, td_lastcpu = 0x1, td_oncpu = 0x1, td_owepreempt = 0x0, td_tsqueue = 0x0, td_locks = 0x0, td_rw_rlocks = 0x0, td_lk_slocks = 0x0, td_stopsched = 0x1, td_blocked = 0x0, td_lockname = 0x0, td_contested = {lh_first = 0x0}, td_sleeplocks = 0xc137daa0, td_intr_nesting_level = 0x0, td_pinned = 0x2, td_ucred = 0xc7dd4e80, td_estcpu = 0x0, td_slptick = 0x0, td_blktick = 0x0, td_swvoltick = 0xcd3b, td_cow = 0xe, td_ru = {ru_utime = {tv_sec = 0x0, tv_usec = 0x0}, ru_stime = {tv_sec = 0x0, tv_usec = 0x0}, ru_maxrss = 0x0, ru_ixrss = 0x0, ru_idrss = 0x0, ru_isrss = 0x0, ru_minflt = 0x11, ru_majflt = 0x0, ru_nswap = 0x0, ru_inblock = 0x1, ru_oublock = 0x0, ru_msgsnd = 0x0, ru_msgrcv = 0x0, ru_nsignals = 0x0, ru_nvcsw = 0x1, ru_nivcsw = 0x0}, td_rux = {rux_runtime = 0x0, rux_uticks = 0x0, rux_sticks = 0x0, rux_iticks = 0x0, rux_uu = 0x0, rux_su = 0x0, rux_tu = 0x0}, td_incruntime = 0xf0ce8, td_runtime = 0xf0ce8, td_pticks = 0x0, td_sticks = 0x0, td_iticks = 0x0, td_uticks = 0x0, td_intrval = 0x0, td_oldsigmask = {__bits = {0x0, 0x0, 0x0, 0x0}}, td_generation = 0x1, td_sigstk = {ss_sp = 0x0, ss_size = 0x0, ss_flags = 0x4}, td_xsig = 0x0, td_profil_addr = 0x0, td_profil_ticks = 0x0, td_name = "init", '\0' , td_fpop = 0x0, td_dbgflags = 0x0, td_dbgksi = {ksi_link = { tqe_next = 0x0, tqe_prev = 0x0}, ksi_info = {si_signo = 0x0, si_errno = 0x0, si_code = 0x0, si_pid = 0x0, si_uid = 0x0, si_status = 0x0, si_addr = 0x0, si_value = {sival_int = 0x0, sival_ptr = 0x0, sigval_int = 0x0, sigval_ptr = 0x0}, _reason = {_fault = {_trapno = 0x0}, _timer = {_timerid = 0x0, _overrun = 0x0}, _mesgq = {_mqd = 0x0}, _poll = {_band = 0x0}, __spare__ = {__spare1__ = 0x0, __spare2__ = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}}}, ksi_flags = 0x0, ksi_sigq = 0x0}, td_ng_outbound = 0x0, td_osd = {osd_nslots = 0x0, osd_slots = 0x0, osd_next = {le_next = 0x0, le_prev = 0x0}}, td_map_def_user = 0x0, td_dbg_forked = 0x0, td_vp_reserv = 0x0, td_sigmask = {__bits = {0x0, 0x0, 0x0, 0x0}}, td_rqindex = 0x1e, td_base_pri = 0x7b, td_priority = 0x7b, td_pri_class = 0x3, td_user_pri = 0x7b, td_base_user_pri = 0x7b, td_pcb = 0xf4221d60, td_state = TDS_RUNNING, td_retval = {0x1000, 0x0}, td_slpcallout = {c_links = {sle = {sle_next = 0x0}, tqe = {tqe_next = 0x0, tqe_prev = 0x0}}, c_time = 0x0, c_arg = 0x0, c_func = 0, c_lock = 0x0, c_flags = 0x10, c_cpu = 0x0}, td_frame = 0xf4221d08, td_kstack_obj = 0xcaacc1bc, td_kstack = 0xf4220000, td_kstack_pages = 0x2, td_critnest = 0x1, td_md = {md_spinlock_count = 0x1, md_saved_flags = 0x246, md_spurflt_addr = 0x28820000}, td_sched = 0xca96f8ac, td_ar = 0x0, td_lprof = {{lh_first = 0x0}, {lh_first = 0x0}}, td_dtrace = 0xca96c100, td_errno = 0x0, td_vnet = 0x0, td_vnet_lpush = 0x0, td_intr_frame = 0x0, td_rfppwait_p = 0x0, td_ma = 0x0, td_ma_cnt = 0x0} (kgdb) $ svn diff -x -p /usr/src/sys Index: /usr/src/sys/cam/cam_ccb.h =================================================================== --- /usr/src/sys/cam/cam_ccb.h (revision 244437) +++ /usr/src/sys/cam/cam_ccb.h (working copy) @@ -42,6 +42,7 @@ #include #include +struct vm_page; /* General allocation length definitions for CCB structures */ #define IOCDBLEN CAM_MAX_CDBLEN /* Space for CDB bytes/pointer */ @@ -713,7 +714,9 @@ struct ccb_ataio { struct ata_cmd cmd; /* ATA command register set */ struct ata_res res; /* ATA result register set */ u_int8_t *data_ptr; /* Ptr to the data buf/SG list */ + struct vm_page **ma; u_int32_t dxfer_len; /* Data transfer length */ + int ma_offset; u_int32_t resid; /* Transfer residual length: 2's comp */ u_int8_t tag_action; /* What to do for tag queueing */ /* @@ -1201,6 +1204,13 @@ cam_fill_ataio(struct ccb_ataio *ataio, u_int32_t u_int32_t timeout); static __inline void +cam_fill_ataio_U(struct ccb_ataio *ataio, u_int32_t retries, + void (*cbfcnp)(struct cam_periph *, union ccb *), + u_int32_t flags, u_int tag_action, + u_int8_t *data_ptr, struct vm_page **ma, u_int32_t dxfer_len, + int ma_offset, u_int32_t timeout); + +static __inline void cam_fill_smpio(struct ccb_smpio *smpio, uint32_t retries, void (*cbfcnp)(struct cam_periph *, union ccb *), uint32_t flags, uint8_t *smp_request, int smp_request_len, @@ -1248,6 +1258,25 @@ cam_fill_ctio(struct ccb_scsiio *csio, u_int32_t r } static __inline void +cam_fill_ataio_U(struct ccb_ataio *ataio, u_int32_t retries, + void (*cbfcnp)(struct cam_periph *, union ccb *), + u_int32_t flags, u_int tag_action, + u_int8_t *data_ptr, struct vm_page **ma, u_int32_t dxfer_len, + int ma_offset, u_int32_t timeout) +{ + ataio->ccb_h.func_code = XPT_ATA_IO; + ataio->ccb_h.flags = flags; + ataio->ccb_h.retry_count = retries; + ataio->ccb_h.cbfcnp = cbfcnp; + ataio->ccb_h.timeout = timeout; + ataio->data_ptr = data_ptr; + ataio->ma = ma; + ataio->ma_offset = ma_offset; + ataio->dxfer_len = dxfer_len; + ataio->tag_action = tag_action; +} + +static __inline void cam_fill_ataio(struct ccb_ataio *ataio, u_int32_t retries, void (*cbfcnp)(struct cam_periph *, union ccb *), u_int32_t flags, u_int tag_action, @@ -1260,6 +1289,7 @@ cam_fill_ataio(struct ccb_ataio *ataio, u_int32_t ataio->ccb_h.cbfcnp = cbfcnp; ataio->ccb_h.timeout = timeout; ataio->data_ptr = data_ptr; + ataio->ma = NULL; ataio->dxfer_len = dxfer_len; ataio->tag_action = tag_action; } Index: /usr/src/sys/cam/ata/ata_da.c =================================================================== --- /usr/src/sys/cam/ata/ata_da.c (revision 244437) +++ /usr/src/sys/cam/ata/ata_da.c (working copy) @@ -1167,6 +1167,7 @@ adaregister(struct cam_periph *periph, void *arg) ((softc->flags & ADA_FLAG_CAN_CFA) && !(softc->flags & ADA_FLAG_CAN_48BIT))) softc->disk->d_flags |= DISKFLAG_CANDELETE; + softc->disk->d_flags |= DISKFLAG_NOTMAPPED_BIO; strlcpy(softc->disk->d_descr, cgd->ident_data.model, MIN(sizeof(softc->disk->d_descr), sizeof(cgd->ident_data.model))); strlcpy(softc->disk->d_ident, cgd->ident_data.serial, @@ -1431,14 +1432,16 @@ adastart(struct cam_periph *periph, union ccb *sta return; } #endif - cam_fill_ataio(ataio, + cam_fill_ataio_U(ataio, ada_retry_count, adadone, bp->bio_cmd == BIO_READ ? CAM_DIR_IN : CAM_DIR_OUT, tag_code, bp->bio_data, + (bp->bio_flags & BIO_NOTMAPPED) ? bp->bio_ma : NULL, bp->bio_bcount, + bp->bio_ma_offset, ada_default_timeout*1000); if ((softc->flags & ADA_FLAG_CAN_NCQ) && tag_code) { Index: /usr/src/sys/fs/cd9660/cd9660_vnops.c =================================================================== --- /usr/src/sys/fs/cd9660/cd9660_vnops.c (revision 244437) +++ /usr/src/sys/fs/cd9660/cd9660_vnops.c (working copy) @@ -329,7 +329,7 @@ cd9660_read(ap) if (lblktosize(imp, rablock) < ip->i_size) error = cluster_read(vp, (off_t)ip->i_size, lbn, size, NOCRED, uio->uio_resid, - (ap->a_ioflag >> 16), &bp); + (ap->a_ioflag >> 16), 0, &bp); else error = bread(vp, lbn, size, NOCRED, &bp); } else { Index: /usr/src/sys/fs/ext2fs/ext2_balloc.c =================================================================== --- /usr/src/sys/fs/ext2fs/ext2_balloc.c (revision 244437) +++ /usr/src/sys/fs/ext2fs/ext2_balloc.c (working copy) @@ -281,7 +281,7 @@ ext2_balloc(ip, lbn, size, cred, bpp, flags) if (seqcount && (vp->v_mount->mnt_flag & MNT_NOCLUSTERR) == 0) { error = cluster_read(vp, ip->i_size, lbn, (int)fs->e2fs_bsize, NOCRED, - MAXBSIZE, seqcount, &nbp); + MAXBSIZE, seqcount, 0, &nbp); } else { error = bread(vp, lbn, (int)fs->e2fs_bsize, NOCRED, &nbp); } Index: /usr/src/sys/fs/ext2fs/ext2_vnops.c =================================================================== --- /usr/src/sys/fs/ext2fs/ext2_vnops.c (revision 244437) +++ /usr/src/sys/fs/ext2fs/ext2_vnops.c (working copy) @@ -1748,10 +1748,11 @@ ext2_read(ap) if (lblktosize(fs, nextlbn) >= ip->i_size) error = bread(vp, lbn, size, NOCRED, &bp); - else if ((vp->v_mount->mnt_flag & MNT_NOCLUSTERR) == 0) + else if ((vp->v_mount->mnt_flag & MNT_NOCLUSTERR) == 0) { error = cluster_read(vp, ip->i_size, lbn, size, - NOCRED, blkoffset + uio->uio_resid, seqcount, &bp); - else if (seqcount > 1) { + NOCRED, blkoffset + uio->uio_resid, seqcount, + 0, &bp); + } else if (seqcount > 1) { int nextsize = blksize(fs, ip, nextlbn); error = breadn(vp, lbn, size, &nextlbn, &nextsize, 1, NOCRED, &bp); @@ -1967,7 +1968,7 @@ ext2_write(ap) } else if (xfersize + blkoffset == fs->e2fs_fsize) { if ((vp->v_mount->mnt_flag & MNT_NOCLUSTERW) == 0) { bp->b_flags |= B_CLUSTEROK; - cluster_write(vp, bp, ip->i_size, seqcount); + cluster_write(vp, bp, ip->i_size, seqcount, 0); } else { bawrite(bp); } Index: /usr/src/sys/fs/udf/udf_vnops.c =================================================================== --- /usr/src/sys/fs/udf/udf_vnops.c (revision 244437) +++ /usr/src/sys/fs/udf/udf_vnops.c (working copy) @@ -478,8 +478,9 @@ udf_read(struct vop_read_args *ap) rablock = lbn + 1; if ((vp->v_mount->mnt_flag & MNT_NOCLUSTERR) == 0) { if (lblktosize(udfmp, rablock) < fsize) { - error = cluster_read(vp, fsize, lbn, size, NOCRED, - uio->uio_resid, (ap->a_ioflag >> 16), &bp); + error = cluster_read(vp, fsize, lbn, size, + NOCRED, uio->uio_resid, + (ap->a_ioflag >> 16), 0, &bp); } else { error = bread(vp, lbn, size, NOCRED, &bp); } Index: /usr/src/sys/fs/msdosfs/msdosfs_vnops.c =================================================================== --- /usr/src/sys/fs/msdosfs/msdosfs_vnops.c (revision 244437) +++ /usr/src/sys/fs/msdosfs/msdosfs_vnops.c (working copy) @@ -600,7 +600,7 @@ msdosfs_read(ap) error = bread(vp, lbn, blsize, NOCRED, &bp); } else if ((vp->v_mount->mnt_flag & MNT_NOCLUSTERR) == 0) { error = cluster_read(vp, dep->de_FileSize, lbn, blsize, - NOCRED, on + uio->uio_resid, seqcount, &bp); + NOCRED, on + uio->uio_resid, seqcount, 0, &bp); } else if (seqcount > 1) { rasize = blsize; error = breadn(vp, lbn, @@ -820,7 +820,7 @@ msdosfs_write(ap) else if (n + croffset == pmp->pm_bpcluster) { if ((vp->v_mount->mnt_flag & MNT_NOCLUSTERW) == 0) cluster_write(vp, bp, dep->de_FileSize, - seqcount); + seqcount, 0); else bawrite(bp); } else Index: /usr/src/sys/vm/vm_kern.c =================================================================== --- /usr/src/sys/vm/vm_kern.c (revision 244437) +++ /usr/src/sys/vm/vm_kern.c (working copy) @@ -85,11 +85,12 @@ __FBSDID("$FreeBSD$"); #include #include -vm_map_t kernel_map=0; -vm_map_t kmem_map=0; -vm_map_t exec_map=0; +vm_map_t kernel_map; +vm_map_t kmem_map; +vm_map_t exec_map; vm_map_t pipe_map; -vm_map_t buffer_map=0; +vm_map_t buffer_map; +vm_map_t bio_transient_map; const void *zero_region; CTASSERT((ZERO_REGION_SIZE & PAGE_MASK) == 0); Index: /usr/src/sys/vm/swap_pager.c =================================================================== --- /usr/src/sys/vm/swap_pager.c (revision 244437) +++ /usr/src/sys/vm/swap_pager.c (working copy) @@ -758,6 +758,15 @@ swp_pager_strategy(struct buf *bp) TAILQ_FOREACH(sp, &swtailq, sw_list) { if (bp->b_blkno >= sp->sw_first && bp->b_blkno < sp->sw_end) { mtx_unlock(&sw_dev_mtx); + if ((sp->sw_flags & SW_NOTMAPPED) != 0) { + bp->b_kvaalloc = bp->b_data; + bp->b_data = unmapped_buf; + bp->b_kvabase = unmapped_buf; + bp->b_flags |= B_NOTMAPPED; + } else { + pmap_qenter((vm_offset_t)bp->b_data, + &bp->b_pages[0], bp->b_bcount / PAGE_SIZE); + } sp->sw_strategy(bp, sp); return; } @@ -1155,11 +1164,6 @@ swap_pager_getpages(vm_object_t object, vm_page_t bp = getpbuf(&nsw_rcount); bp->b_flags |= B_PAGING; - /* - * map our page(s) into kva for input - */ - pmap_qenter((vm_offset_t)bp->b_data, m + i, j - i); - bp->b_iocmd = BIO_READ; bp->b_iodone = swp_pager_async_iodone; bp->b_rcred = crhold(thread0.td_ucred); @@ -1371,8 +1375,6 @@ swap_pager_putpages(vm_object_t object, vm_page_t bp->b_flags |= B_PAGING; bp->b_iocmd = BIO_WRITE; - pmap_qenter((vm_offset_t)bp->b_data, &m[i], n); - bp->b_rcred = crhold(thread0.td_ucred); bp->b_wcred = crhold(thread0.td_ucred); bp->b_bcount = PAGE_SIZE * n; @@ -1484,7 +1486,12 @@ swp_pager_async_iodone(struct buf *bp) /* * remove the mapping for kernel virtual */ - pmap_qremove((vm_offset_t)bp->b_data, bp->b_npages); + if ((bp->b_flags & B_NOTMAPPED) != 0) { + bp->b_data = bp->b_kvaalloc; + bp->b_kvabase = bp->b_kvaalloc; + bp->b_flags &= ~B_NOTMAPPED; + } else + pmap_qremove((vm_offset_t)bp->b_data, bp->b_npages); if (bp->b_npages) { object = bp->b_pages[0]->object; @@ -2144,7 +2151,8 @@ swapon_check_swzone(unsigned long npages) } static void -swaponsomething(struct vnode *vp, void *id, u_long nblks, sw_strategy_t *strategy, sw_close_t *close, dev_t dev) +swaponsomething(struct vnode *vp, void *id, u_long nblks, + sw_strategy_t *strategy, sw_close_t *close, dev_t dev, int flags) { struct swdevt *sp, *tsp; swblk_t dvbase; @@ -2180,6 +2188,7 @@ static void sp->sw_used = 0; sp->sw_strategy = strategy; sp->sw_close = close; + sp->sw_flags = flags; sp->sw_blist = blist_create(nblks, M_WAITOK); /* @@ -2537,10 +2546,18 @@ swapgeom_strategy(struct buf *bp, struct swdevt *s bio->bio_caller2 = bp; bio->bio_cmd = bp->b_iocmd; - bio->bio_data = bp->b_data; bio->bio_offset = (bp->b_blkno - sp->sw_first) * PAGE_SIZE; bio->bio_length = bp->b_bcount; bio->bio_done = swapgeom_done; + if ((bp->b_flags & B_NOTMAPPED) != 0) { + bio->bio_ma = bp->b_pages; + bio->bio_data = unmapped_buf; + bio->bio_ma_offset = (vm_offset_t)bp->b_offset & PAGE_MASK; + bio->bio_flags |= BIO_NOTMAPPED; + } else { + bio->bio_data = bp->b_data; + bio->bio_ma = NULL; + } g_io_request(bio, cp); return; } @@ -2630,9 +2647,9 @@ swapongeom_ev(void *arg, int flags) } nblks = pp->mediasize / DEV_BSIZE; swaponsomething(swh->vp, cp, nblks, swapgeom_strategy, - swapgeom_close, dev2udev(swh->dev)); + swapgeom_close, dev2udev(swh->dev), + (pp->flags & G_PF_ACCEPT_UNMAPPED) != 0 ? SW_NOTMAPPED : 0); swh->error = 0; - return; } static int @@ -2721,6 +2738,6 @@ swaponvp(struct thread *td, struct vnode *vp, u_lo return (error); swaponsomething(vp, vp, nblks, swapdev_strategy, swapdev_close, - NODEV); + NODEV, 0); return (0); } Index: /usr/src/sys/vm/swap_pager.h =================================================================== --- /usr/src/sys/vm/swap_pager.h (revision 244437) +++ /usr/src/sys/vm/swap_pager.h (working copy) @@ -68,6 +68,7 @@ struct swdevt { sw_close_t *sw_close; }; +#define SW_NOTMAPPED 0x01 #define SW_CLOSING 0x04 #ifdef _KERNEL Index: /usr/src/sys/vm/vm.h =================================================================== --- /usr/src/sys/vm/vm.h (revision 244437) +++ /usr/src/sys/vm/vm.h (working copy) @@ -136,6 +136,8 @@ struct kva_md_info { vm_offset_t clean_eva; vm_offset_t pager_sva; vm_offset_t pager_eva; + vm_offset_t bio_transient_sva; + vm_offset_t bio_transient_eva; }; extern struct kva_md_info kmi; Index: /usr/src/sys/vm/pmap.h =================================================================== --- /usr/src/sys/vm/pmap.h (revision 244437) +++ /usr/src/sys/vm/pmap.h (working copy) @@ -108,6 +108,8 @@ void pmap_clear_modify(vm_page_t m); void pmap_clear_reference(vm_page_t m); void pmap_copy(pmap_t, pmap_t, vm_offset_t, vm_size_t, vm_offset_t); void pmap_copy_page(vm_page_t, vm_page_t); +void pmap_copy_pages(vm_page_t ma[], vm_offset_t a_offset, + vm_page_t mb[], vm_offset_t b_offset, int xfersize); void pmap_enter(pmap_t, vm_offset_t, vm_prot_t, vm_page_t, vm_prot_t, boolean_t); void pmap_enter_object(pmap_t pmap, vm_offset_t start, Index: /usr/src/sys/vm/vm_init.c =================================================================== --- /usr/src/sys/vm/vm_init.c (revision 244437) +++ /usr/src/sys/vm/vm_init.c (working copy) @@ -186,10 +186,14 @@ again: panic("startup: table size inconsistency"); clean_map = kmem_suballoc(kernel_map, &kmi->clean_sva, &kmi->clean_eva, - (long)nbuf * BKVASIZE + (long)nswbuf * MAXPHYS, TRUE); + (long)nbuf * BKVASIZE + (long)nswbuf * MAXPHYS + + (long)/*XXXKIB*/1024 * BKVASIZE, TRUE); buffer_map = kmem_suballoc(clean_map, &kmi->buffer_sva, &kmi->buffer_eva, (long)nbuf * BKVASIZE, FALSE); buffer_map->system_map = 1; + bio_transient_map = kmem_suballoc(clean_map, &kmi->bio_transient_sva, + &kmi->bio_transient_eva, /*XXXKIB*/1024 * BKVASIZE, FALSE); + bio_transient_map->system_map = 1; pager_map = kmem_suballoc(clean_map, &kmi->pager_sva, &kmi->pager_eva, (long)nswbuf * MAXPHYS, FALSE); pager_map->system_map = 1; Index: /usr/src/sys/vm/vnode_pager.c =================================================================== --- /usr/src/sys/vm/vnode_pager.c (revision 244437) +++ /usr/src/sys/vm/vnode_pager.c (working copy) @@ -697,6 +697,7 @@ vnode_pager_generic_getpages(vp, m, bytecount, req int runpg; int runend; struct buf *bp; + struct mount *mp; int count; int error; @@ -899,12 +900,21 @@ vnode_pager_generic_getpages(vp, m, bytecount, req } bp = getpbuf(&vnode_pbuf_freecnt); - kva = (vm_offset_t) bp->b_data; + kva = (vm_offset_t)bp->b_data; /* - * and map the pages to be read into the kva + * and map the pages to be read into the kva, if the filesystem + * requires mapped buffers. */ - pmap_qenter(kva, m, count); + mp = vp->v_mount; + if (mp != NULL && (mp->mnt_kern_flag & MNTK_UNMAPPED_BUFS) != 0) { + bp->b_data = unmapped_buf; + bp->b_kvabase = unmapped_buf; + bp->b_flags |= B_NOTMAPPED; + for (i = 0; i < count; i++) + bp->b_pages[i] = m[i]; + } else + pmap_qenter(kva, m, count); /* build a minimal buffer header */ bp->b_iocmd = BIO_READ; @@ -933,11 +943,17 @@ vnode_pager_generic_getpages(vp, m, bytecount, req if ((bp->b_ioflags & BIO_ERROR) != 0) error = EIO; - if (!error) { - if (size != count * PAGE_SIZE) - bzero((caddr_t) kva + size, PAGE_SIZE * count - size); + if (mp != NULL && (mp->mnt_kern_flag & MNTK_UNMAPPED_BUFS) != 0) { + bp->b_data = (caddr_t)kva; + bp->b_kvabase = (caddr_t)kva; + bp->b_flags &= ~B_NOTMAPPED; + for (i = 0; i < count; i++) + bp->b_pages[i] = NULL; + } else { + if (error != 0 && size != count * PAGE_SIZE) + bzero((caddr_t)kva + size, PAGE_SIZE * count - size); + pmap_qremove(kva, count); } - pmap_qremove(kva, count); /* * free the buffer header back to the swap buffer pool Index: /usr/src/sys/i386/i386/pmap.c =================================================================== --- /usr/src/sys/i386/i386/pmap.c (revision 244437) +++ /usr/src/sys/i386/i386/pmap.c (working copy) @@ -4224,6 +4224,48 @@ pmap_copy_page(vm_page_t src, vm_page_t dst) mtx_unlock(&sysmaps->lock); } +void +pmap_copy_pages(vm_page_t ma[], vm_offset_t a_offset, vm_page_t mb[], + vm_offset_t b_offset, int xfersize) +{ + struct sysmaps *sysmaps; + vm_page_t a_pg, b_pg; + char *a_cp, *b_cp; + vm_offset_t a_pg_offset, b_pg_offset; + int cnt; + + sysmaps = &sysmaps_pcpu[PCPU_GET(cpuid)]; + mtx_lock(&sysmaps->lock); + if (*sysmaps->CMAP1) + panic("pmap_copy_page: CMAP1 busy"); + if (*sysmaps->CMAP2) + panic("pmap_copy_page: CMAP2 busy"); + sched_pin(); + while (xfersize > 0) { + invlpg((u_int)sysmaps->CADDR1); + invlpg((u_int)sysmaps->CADDR2); + a_pg = ma[a_offset >> PAGE_SHIFT]; + a_pg_offset = a_offset & PAGE_MASK; + cnt = min(xfersize, PAGE_SIZE - a_pg_offset); + b_pg = mb[b_offset >> PAGE_SHIFT]; + b_pg_offset = b_offset & PAGE_MASK; + cnt = min(cnt, PAGE_SIZE - b_pg_offset); + *sysmaps->CMAP1 = PG_V | VM_PAGE_TO_PHYS(a_pg) | PG_A | + pmap_cache_bits(b_pg->md.pat_mode, 0); + *sysmaps->CMAP2 = PG_V | PG_RW | VM_PAGE_TO_PHYS(b_pg) | PG_A | + PG_M | pmap_cache_bits(b_pg->md.pat_mode, 0); + a_cp = sysmaps->CADDR1 + a_pg_offset; + b_cp = sysmaps->CADDR2 + b_pg_offset; + bcopy(a_cp, b_cp, cnt); + *sysmaps->CMAP1 = 0; + *sysmaps->CMAP2 = 0; + a_offset += cnt; + b_offset += cnt; + xfersize -= cnt; + } + mtx_unlock(&sysmaps->lock); +} + /* * Returns true if the pmap's pv is one of the first * 16 pvs linked to from this page. This count may Index: /usr/src/sys/amd64/amd64/pmap.c =================================================================== --- /usr/src/sys/amd64/amd64/pmap.c (revision 244437) +++ /usr/src/sys/amd64/amd64/pmap.c (working copy) @@ -4216,6 +4216,30 @@ pmap_copy_page(vm_page_t msrc, vm_page_t mdst) pagecopy((void *)src, (void *)dst); } +void +pmap_copy_pages(vm_page_t ma[], vm_offset_t a_offset, vm_page_t mb[], + vm_offset_t b_offset, int xfersize) +{ + void *a_cp, *b_cp; + vm_offset_t a_pg_offset, b_pg_offset; + int cnt; + + while (xfersize > 0) { + a_pg_offset = a_offset & PAGE_MASK; + cnt = min(xfersize, PAGE_SIZE - a_pg_offset); + a_cp = (char *)PHYS_TO_DMAP(ma[a_offset >> PAGE_SHIFT]-> + phys_addr) + a_pg_offset; + b_pg_offset = b_offset & PAGE_MASK; + cnt = min(cnt, PAGE_SIZE - b_pg_offset); + b_cp = (char *)PHYS_TO_DMAP(mb[b_offset >> PAGE_SHIFT]-> + phys_addr) + b_pg_offset; + bcopy(a_cp, b_cp, cnt); + a_offset += cnt; + b_offset += cnt; + xfersize -= cnt; + } +} + /* * Returns true if the pmap's pv is one of the first * 16 pvs linked to from this page. This count may Index: /usr/src/sys/sys/mount.h =================================================================== --- /usr/src/sys/sys/mount.h (revision 244437) +++ /usr/src/sys/sys/mount.h (working copy) @@ -374,6 +374,7 @@ void __mnt_vnode_markerfree(struct vnode #define MNTK_VGONE_WAITER 0x00000400 #define MNTK_LOOKUP_EXCL_DOTDOT 0x00000800 #define MNTK_MARKER 0x00001000 +#define MNTK_UNMAPPED_BUFS 0x00002000 #define MNTK_NOASYNC 0x00800000 /* disable async */ #define MNTK_UNMOUNT 0x01000000 /* unmount in progress */ #define MNTK_MWAIT 0x02000000 /* waiting for unmount to finish */ Index: /usr/src/sys/sys/buf.h =================================================================== --- /usr/src/sys/sys/buf.h (revision 244437) +++ /usr/src/sys/sys/buf.h (working copy) @@ -117,6 +117,7 @@ struct buf { long b_bufsize; /* Allocated buffer size. */ long b_runningbufspace; /* when I/O is running, pipelining */ caddr_t b_kvabase; /* base kva for buffer */ + caddr_t b_kvaalloc; /* allocated kva for B_KVAALLOC */ int b_kvasize; /* size of kva for buffer */ daddr_t b_lblkno; /* Logical block number. */ struct vnode *b_vp; /* Device vnode. */ @@ -202,8 +203,8 @@ struct buf { #define B_PERSISTENT 0x00000100 /* Perm. ref'ed while EXT2FS mounted. */ #define B_DONE 0x00000200 /* I/O completed. */ #define B_EINTR 0x00000400 /* I/O was interrupted */ -#define B_00000800 0x00000800 /* Available flag. */ -#define B_00001000 0x00001000 /* Available flag. */ +#define B_NOTMAPPED 0x00000800 /* KVA is not mapped. */ +#define B_KVAALLOC 0x00001000 /* But allocated. */ #define B_INVAL 0x00002000 /* Does not contain valid info. */ #define B_00004000 0x00004000 /* Available flag. */ #define B_NOCACHE 0x00008000 /* Do not cache block after use. */ @@ -453,7 +454,9 @@ buf_countdeps(struct buf *bp, int i) */ #define GB_LOCK_NOWAIT 0x0001 /* Fail if we block on a buf lock. */ #define GB_NOCREAT 0x0002 /* Don't create a buf if not found. */ -#define GB_NOWAIT_BD 0x0004 /* Do not wait for bufdaemon */ +#define GB_NOWAIT_BD 0x0004 /* Do not wait for bufdaemon. */ +#define GB_NOTMAPPED 0x0008 /* Do not mmap buffer pages. */ +#define GB_KVAALLOC 0x0010 /* But allocate KVA. */ #ifdef _KERNEL extern int nbuf; /* The number of buffer headers */ @@ -470,6 +473,7 @@ extern struct buf *swbuf; /* Swap I/O buffer head extern int nswbuf; /* Number of swap I/O buffer headers. */ extern int cluster_pbuf_freecnt; /* Number of pbufs for clusters */ extern int vnode_pbuf_freecnt; /* Number of pbufs for vnode pager */ +extern caddr_t unmapped_buf; void runningbufwakeup(struct buf *); void waitrunningbufspace(void); @@ -480,7 +484,10 @@ int buf_dirty_count_severe(void); void bremfree(struct buf *); void bremfreef(struct buf *); /* XXX Force bremfree, only for nfs. */ #define bread(vp, blkno, size, cred, bpp) \ - breadn_flags(vp, blkno, size, 0, 0, 0, cred, 0, bpp) + breadn_flags(vp, blkno, size, NULL, NULL, 0, cred, 0, bpp) +#define bread_gb(vp, blkno, size, cred, gbflags, bpp) \ + breadn_flags(vp, blkno, size, NULL, NULL, 0, cred, \ + gbflags, bpp) #define breadn(vp, blkno, size, rablkno, rabsize, cnt, cred, bpp) \ breadn_flags(vp, blkno, size, rablkno, rabsize, cnt, cred, 0, bpp) int breadn_flags(struct vnode *, daddr_t, int, daddr_t *, int *, int, @@ -506,9 +513,10 @@ void bufdone_finish(struct buf *); void bd_speedup(void); int cluster_read(struct vnode *, u_quad_t, daddr_t, long, - struct ucred *, long, int, struct buf **); -int cluster_wbuild(struct vnode *, long, daddr_t, int); -void cluster_write(struct vnode *, struct buf *, u_quad_t, int); + struct ucred *, long, int, int, struct buf **); +int cluster_wbuild(struct vnode *, long, daddr_t, int, int); +void cluster_write(struct vnode *, struct buf *, u_quad_t, int, int); +void vfs_bio_bzero_buf(struct buf *bp, int base, int size); void vfs_bio_set_valid(struct buf *, int base, int size); void vfs_bio_clrbuf(struct buf *); void vfs_busy_pages(struct buf *, int clear_modify); Index: /usr/src/sys/sys/bio.h =================================================================== --- /usr/src/sys/sys/bio.h (revision 244437) +++ /usr/src/sys/sys/bio.h (working copy) @@ -55,10 +55,13 @@ #define BIO_DONE 0x02 #define BIO_ONQUEUE 0x04 #define BIO_ORDERED 0x08 +#define BIO_NOTMAPPED 0x10 +#define BIO_TRANSIENT_MAPPING 0x20 #ifdef _KERNEL struct disk; struct bio; +struct vm_map; /* Empty classifier tag, to prevent further classification. */ #define BIO_NOTCLASSIFIED (void *)(~0UL) @@ -78,6 +81,8 @@ struct bio { off_t bio_offset; /* Offset into file. */ long bio_bcount; /* Valid bytes in buffer. */ caddr_t bio_data; /* Memory, superblocks, indirect etc. */ + struct vm_page **bio_ma; /* Or unmapped. */ + int bio_ma_offset; /* Offset in the first page of bio_ma. */ int bio_error; /* Errno for BIO_ERROR. */ long bio_resid; /* Remaining I/O in bytes. */ void (*bio_done)(struct bio *); @@ -121,6 +126,8 @@ struct bio_queue_head { struct bio *insert_point; }; +extern struct vm_map *bio_transient_map; + void biodone(struct bio *bp); void biofinish(struct bio *bp, struct devstat *stat, int error); int biowait(struct bio *bp, const char *wchan); Index: /usr/src/sys/sys/vnode.h =================================================================== --- /usr/src/sys/sys/vnode.h (revision 244437) +++ /usr/src/sys/sys/vnode.h (working copy) @@ -687,6 +687,8 @@ int vn_vget_ino(struct vnode *vp, ino_t ino, int l struct vnode **rvp); int vn_io_fault_uiomove(char *data, int xfersize, struct uio *uio); +int vn_io_fault_pgmove(vm_page_t ma[], vm_offset_t offset, int xfersize, + struct uio *uio); #define vn_rangelock_unlock(vp, cookie) \ rangelock_unlock(&(vp)->v_rl, (cookie), VI_MTX(vp)) Index: /usr/src/sys/ufs/ufs/ufs_extern.h =================================================================== --- /usr/src/sys/ufs/ufs/ufs_extern.h (revision 244437) +++ /usr/src/sys/ufs/ufs/ufs_extern.h (working copy) @@ -121,6 +121,7 @@ void softdep_revert_rmdir(struct inode *, struct i */ #define BA_CLRBUF 0x00010000 /* Clear invalid areas of buffer. */ #define BA_METAONLY 0x00020000 /* Return indirect block buffer. */ +#define BA_NOTMAPPED 0x00040000 /* Do not mmap resulted buffer. */ #define BA_SEQMASK 0x7F000000 /* Bits holding seq heuristic. */ #define BA_SEQSHIFT 24 #define BA_SEQMAX 0x7F Index: /usr/src/sys/ufs/ffs/ffs_vnops.c =================================================================== --- /usr/src/sys/ufs/ffs/ffs_vnops.c (revision 244437) +++ /usr/src/sys/ufs/ffs/ffs_vnops.c (working copy) @@ -508,7 +508,8 @@ ffs_read(ap) /* * Don't do readahead if this is the end of the file. */ - error = bread(vp, lbn, size, NOCRED, &bp); + error = bread_gb(vp, lbn, size, NOCRED, + GB_NOTMAPPED, &bp); } else if ((vp->v_mount->mnt_flag & MNT_NOCLUSTERR) == 0) { /* * Otherwise if we are allowed to cluster, @@ -518,7 +519,8 @@ ffs_read(ap) * doing sequential access. */ error = cluster_read(vp, ip->i_size, lbn, - size, NOCRED, blkoffset + uio->uio_resid, seqcount, &bp); + size, NOCRED, blkoffset + uio->uio_resid, + seqcount, GB_NOTMAPPED, &bp); } else if (seqcount > 1) { /* * If we are NOT allowed to cluster, then @@ -529,15 +531,16 @@ ffs_read(ap) * the 6th argument. */ int nextsize = blksize(fs, ip, nextlbn); - error = breadn(vp, lbn, - size, &nextlbn, &nextsize, 1, NOCRED, &bp); + error = breadn_flags(vp, lbn, size, &nextlbn, + &nextsize, 1, NOCRED, GB_NOTMAPPED, &bp); } else { /* * Failing all of the above, just read what the * user asked for. Interestingly, the same as * the first option above. */ - error = bread(vp, lbn, size, NOCRED, &bp); + error = bread_gb(vp, lbn, size, NOCRED, + GB_NOTMAPPED, &bp); } if (error) { brelse(bp); @@ -568,8 +571,13 @@ ffs_read(ap) xfersize = size; } - error = vn_io_fault_uiomove((char *)bp->b_data + blkoffset, - (int)xfersize, uio); + if ((bp->b_flags & B_NOTMAPPED) == 0) { + error = vn_io_fault_uiomove((char *)bp->b_data + + blkoffset, (int)xfersize, uio); + } else { + error = vn_io_fault_pgmove(bp->b_pages, blkoffset, + (int)xfersize, uio); + } if (error) break; @@ -700,6 +708,7 @@ ffs_write(ap) flags = seqcount << BA_SEQSHIFT; if ((ioflag & IO_SYNC) && !DOINGASYNC(vp)) flags |= IO_SYNC; + flags |= BA_NOTMAPPED; for (error = 0; uio->uio_resid > 0;) { lbn = lblkno(fs, uio->uio_offset); @@ -739,8 +748,13 @@ ffs_write(ap) if (size < xfersize) xfersize = size; - error = vn_io_fault_uiomove((char *)bp->b_data + blkoffset, - (int)xfersize, uio); + if ((bp->b_flags & B_NOTMAPPED) == 0) { + error = vn_io_fault_uiomove((char *)bp->b_data + + blkoffset, (int)xfersize, uio); + } else { + error = vn_io_fault_pgmove(bp->b_pages, blkoffset, + (int)xfersize, uio); + } /* * If the buffer is not already filled and we encounter an * error while trying to fill it, we have to clear out any @@ -783,7 +797,8 @@ ffs_write(ap) } else if (xfersize + blkoffset == fs->fs_bsize) { if ((vp->v_mount->mnt_flag & MNT_NOCLUSTERW) == 0) { bp->b_flags |= B_CLUSTEROK; - cluster_write(vp, bp, ip->i_size, seqcount); + cluster_write(vp, bp, ip->i_size, seqcount, + GB_NOTMAPPED); } else { bawrite(bp); } Index: /usr/src/sys/ufs/ffs/ffs_vfsops.c =================================================================== --- /usr/src/sys/ufs/ffs/ffs_vfsops.c (revision 244437) +++ /usr/src/sys/ufs/ffs/ffs_vfsops.c (working copy) @@ -1076,7 +1076,7 @@ ffs_mountfs(devvp, mp, td) */ MNT_ILOCK(mp); mp->mnt_kern_flag |= MNTK_LOOKUP_SHARED | MNTK_EXTENDED_SHARED | - MNTK_NO_IOPF; + MNTK_NO_IOPF | MNTK_UNMAPPED_BUFS; MNT_IUNLOCK(mp); #ifdef UFS_EXTATTR #ifdef UFS_EXTATTR_AUTOSTART @@ -2095,6 +2095,7 @@ ffs_bufwrite(struct buf *bp) * set b_lblkno and BKGRDMARKER before calling bgetvp() * to avoid confusing the splay tree and gbincore(). */ + KASSERT((bp->b_flags & B_NOTMAPPED) == 0, ("Unmapped cg")); memcpy(newbp->b_data, bp->b_data, bp->b_bufsize); newbp->b_lblkno = bp->b_lblkno; newbp->b_xflags |= BX_BKGRDMARKER; Index: /usr/src/sys/ufs/ffs/ffs_balloc.c =================================================================== --- /usr/src/sys/ufs/ffs/ffs_balloc.c (revision 244437) +++ /usr/src/sys/ufs/ffs/ffs_balloc.c (working copy) @@ -107,7 +107,7 @@ ffs_balloc_ufs1(struct vnode *vp, off_t startoffse int saved_inbdflush; static struct timeval lastfail; static int curfail; - int reclaimed; + int gbflags, reclaimed; ip = VTOI(vp); dp = ip->i_din1; @@ -123,6 +123,7 @@ ffs_balloc_ufs1(struct vnode *vp, off_t startoffse return (EOPNOTSUPP); if (lbn < 0) return (EFBIG); + gbflags = (flags & BA_NOTMAPPED) != 0 ? GB_NOTMAPPED : 0; if (DOINGSOFTDEP(vp)) softdep_prealloc(vp, MNT_WAIT); @@ -211,7 +212,7 @@ ffs_balloc_ufs1(struct vnode *vp, off_t startoffse nsize, flags, cred, &newb); if (error) return (error); - bp = getblk(vp, lbn, nsize, 0, 0, 0); + bp = getblk(vp, lbn, nsize, 0, 0, gbflags); bp->b_blkno = fsbtodb(fs, newb); if (flags & BA_CLRBUF) vfs_bio_clrbuf(bp); @@ -255,7 +256,7 @@ ffs_balloc_ufs1(struct vnode *vp, off_t startoffse nb = newb; *allocblk++ = nb; *lbns_remfree++ = indirs[1].in_lbn; - bp = getblk(vp, indirs[1].in_lbn, fs->fs_bsize, 0, 0, 0); + bp = getblk(vp, indirs[1].in_lbn, fs->fs_bsize, 0, 0, gbflags); bp->b_blkno = fsbtodb(fs, nb); vfs_bio_clrbuf(bp); if (DOINGSOFTDEP(vp)) { @@ -389,7 +390,7 @@ retry: nb = newb; *allocblk++ = nb; *lbns_remfree++ = lbn; - nbp = getblk(vp, lbn, fs->fs_bsize, 0, 0, 0); + nbp = getblk(vp, lbn, fs->fs_bsize, 0, 0, gbflags); nbp->b_blkno = fsbtodb(fs, nb); if (flags & BA_CLRBUF) vfs_bio_clrbuf(nbp); @@ -418,16 +419,17 @@ retry: if (seqcount && (vp->v_mount->mnt_flag & MNT_NOCLUSTERR) == 0) { error = cluster_read(vp, ip->i_size, lbn, (int)fs->fs_bsize, NOCRED, - MAXBSIZE, seqcount, &nbp); + MAXBSIZE, seqcount, gbflags, &nbp); } else { - error = bread(vp, lbn, (int)fs->fs_bsize, NOCRED, &nbp); + error = bread_gb(vp, lbn, (int)fs->fs_bsize, NOCRED, + gbflags, &nbp); } if (error) { brelse(nbp); goto fail; } } else { - nbp = getblk(vp, lbn, fs->fs_bsize, 0, 0, 0); + nbp = getblk(vp, lbn, fs->fs_bsize, 0, 0, gbflags); nbp->b_blkno = fsbtodb(fs, nb); } curthread_pflags_restore(saved_inbdflush); @@ -539,7 +541,7 @@ ffs_balloc_ufs2(struct vnode *vp, off_t startoffse int saved_inbdflush; static struct timeval lastfail; static int curfail; - int reclaimed; + int gbflags, reclaimed; ip = VTOI(vp); dp = ip->i_din2; @@ -553,6 +555,7 @@ ffs_balloc_ufs2(struct vnode *vp, off_t startoffse *bpp = NULL; if (lbn < 0) return (EFBIG); + gbflags = (flags & BA_NOTMAPPED) != 0 ? GB_NOTMAPPED : 0; if (DOINGSOFTDEP(vp)) softdep_prealloc(vp, MNT_WAIT); @@ -603,7 +606,8 @@ ffs_balloc_ufs2(struct vnode *vp, off_t startoffse panic("ffs_balloc_ufs2: BA_METAONLY for ext block"); nb = dp->di_extb[lbn]; if (nb != 0 && dp->di_extsize >= smalllblktosize(fs, lbn + 1)) { - error = bread(vp, -1 - lbn, fs->fs_bsize, NOCRED, &bp); + error = bread_gb(vp, -1 - lbn, fs->fs_bsize, NOCRED, + gbflags, &bp); if (error) { brelse(bp); return (error); @@ -620,7 +624,8 @@ ffs_balloc_ufs2(struct vnode *vp, off_t startoffse osize = fragroundup(fs, blkoff(fs, dp->di_extsize)); nsize = fragroundup(fs, size); if (nsize <= osize) { - error = bread(vp, -1 - lbn, osize, NOCRED, &bp); + error = bread_gb(vp, -1 - lbn, osize, NOCRED, + gbflags, &bp); if (error) { brelse(bp); return (error); @@ -653,7 +658,7 @@ ffs_balloc_ufs2(struct vnode *vp, off_t startoffse nsize, flags, cred, &newb); if (error) return (error); - bp = getblk(vp, -1 - lbn, nsize, 0, 0, 0); + bp = getblk(vp, -1 - lbn, nsize, 0, 0, gbflags); bp->b_blkno = fsbtodb(fs, newb); bp->b_xflags |= BX_ALTDATA; if (flags & BA_CLRBUF) @@ -679,9 +684,9 @@ ffs_balloc_ufs2(struct vnode *vp, off_t startoffse if (osize < fs->fs_bsize && osize > 0) { UFS_LOCK(ump); error = ffs_realloccg(ip, nb, dp->di_db[nb], - ffs_blkpref_ufs2(ip, lastlbn, (int)nb, - &dp->di_db[0]), osize, (int)fs->fs_bsize, - flags, cred, &bp); + ffs_blkpref_ufs2(ip, lastlbn, (int)nb, + &dp->di_db[0]), osize, (int)fs->fs_bsize, + flags, cred, &bp); if (error) return (error); if (DOINGSOFTDEP(vp)) @@ -707,7 +712,8 @@ ffs_balloc_ufs2(struct vnode *vp, off_t startoffse panic("ffs_balloc_ufs2: BA_METAONLY for direct block"); nb = dp->di_db[lbn]; if (nb != 0 && ip->i_size >= smalllblktosize(fs, lbn + 1)) { - error = bread(vp, lbn, fs->fs_bsize, NOCRED, &bp); + error = bread_gb(vp, lbn, fs->fs_bsize, NOCRED, + gbflags, &bp); if (error) { brelse(bp); return (error); @@ -723,7 +729,8 @@ ffs_balloc_ufs2(struct vnode *vp, off_t startoffse osize = fragroundup(fs, blkoff(fs, ip->i_size)); nsize = fragroundup(fs, size); if (nsize <= osize) { - error = bread(vp, lbn, osize, NOCRED, &bp); + error = bread_gb(vp, lbn, osize, NOCRED, + gbflags, &bp); if (error) { brelse(bp); return (error); @@ -733,7 +740,7 @@ ffs_balloc_ufs2(struct vnode *vp, off_t startoffse UFS_LOCK(ump); error = ffs_realloccg(ip, lbn, dp->di_db[lbn], ffs_blkpref_ufs2(ip, lbn, (int)lbn, - &dp->di_db[0]), osize, nsize, flags, + &dp->di_db[0]), osize, nsize, flags, cred, &bp); if (error) return (error); @@ -753,7 +760,7 @@ ffs_balloc_ufs2(struct vnode *vp, off_t startoffse &dp->di_db[0]), nsize, flags, cred, &newb); if (error) return (error); - bp = getblk(vp, lbn, nsize, 0, 0, 0); + bp = getblk(vp, lbn, nsize, 0, 0, gbflags); bp->b_blkno = fsbtodb(fs, newb); if (flags & BA_CLRBUF) vfs_bio_clrbuf(bp); @@ -797,7 +804,8 @@ ffs_balloc_ufs2(struct vnode *vp, off_t startoffse nb = newb; *allocblk++ = nb; *lbns_remfree++ = indirs[1].in_lbn; - bp = getblk(vp, indirs[1].in_lbn, fs->fs_bsize, 0, 0, 0); + bp = getblk(vp, indirs[1].in_lbn, fs->fs_bsize, 0, 0, + GB_NOTMAPPED); bp->b_blkno = fsbtodb(fs, nb); vfs_bio_clrbuf(bp); if (DOINGSOFTDEP(vp)) { @@ -862,7 +870,8 @@ retry: nb = newb; *allocblk++ = nb; *lbns_remfree++ = indirs[i].in_lbn; - nbp = getblk(vp, indirs[i].in_lbn, fs->fs_bsize, 0, 0, 0); + nbp = getblk(vp, indirs[i].in_lbn, fs->fs_bsize, 0, 0, + GB_NOTMAPPED); nbp->b_blkno = fsbtodb(fs, nb); vfs_bio_clrbuf(nbp); if (DOINGSOFTDEP(vp)) { @@ -931,7 +940,7 @@ retry: nb = newb; *allocblk++ = nb; *lbns_remfree++ = lbn; - nbp = getblk(vp, lbn, fs->fs_bsize, 0, 0, 0); + nbp = getblk(vp, lbn, fs->fs_bsize, 0, 0, gbflags); nbp->b_blkno = fsbtodb(fs, nb); if (flags & BA_CLRBUF) vfs_bio_clrbuf(nbp); @@ -966,16 +975,17 @@ retry: if (seqcount && (vp->v_mount->mnt_flag & MNT_NOCLUSTERR) == 0) { error = cluster_read(vp, ip->i_size, lbn, (int)fs->fs_bsize, NOCRED, - MAXBSIZE, seqcount, &nbp); + MAXBSIZE, seqcount, gbflags, &nbp); } else { - error = bread(vp, lbn, (int)fs->fs_bsize, NOCRED, &nbp); + error = bread_gb(vp, lbn, (int)fs->fs_bsize, + NOCRED, gbflags, &nbp); } if (error) { brelse(nbp); goto fail; } } else { - nbp = getblk(vp, lbn, fs->fs_bsize, 0, 0, 0); + nbp = getblk(vp, lbn, fs->fs_bsize, 0, 0, gbflags); nbp->b_blkno = fsbtodb(fs, nb); } curthread_pflags_restore(saved_inbdflush); Index: /usr/src/sys/ufs/ffs/ffs_alloc.c =================================================================== --- /usr/src/sys/ufs/ffs/ffs_alloc.c (revision 244437) +++ /usr/src/sys/ufs/ffs/ffs_alloc.c (working copy) @@ -254,7 +254,7 @@ ffs_realloccg(ip, lbprev, bprev, bpref, osize, nsi struct buf *bp; struct ufsmount *ump; u_int cg, request, reclaimed; - int error; + int error, gbflags; ufs2_daddr_t bno; static struct timeval lastfail; static int curfail; @@ -265,6 +265,8 @@ ffs_realloccg(ip, lbprev, bprev, bpref, osize, nsi fs = ip->i_fs; bp = NULL; ump = ip->i_ump; + gbflags = (flags & BA_NOTMAPPED) != 0 ? GB_NOTMAPPED : 0; + mtx_assert(UFS_MTX(ump), MA_OWNED); #ifdef INVARIANTS if (vp->v_mount->mnt_kern_flag & MNTK_SUSPENDED) @@ -296,7 +298,7 @@ retry: /* * Allocate the extra space in the buffer. */ - error = bread(vp, lbprev, osize, NOCRED, &bp); + error = bread_gb(vp, lbprev, osize, NOCRED, gbflags, &bp); if (error) { brelse(bp); return (error); @@ -332,7 +334,7 @@ retry: ip->i_flag |= IN_CHANGE | IN_UPDATE; allocbuf(bp, nsize); bp->b_flags |= B_DONE; - bzero(bp->b_data + osize, nsize - osize); + vfs_bio_bzero_buf(bp, osize, nsize - osize); if ((bp->b_flags & (B_MALLOC | B_VMIO)) == B_VMIO) vfs_bio_set_valid(bp, osize, nsize - osize); *bpp = bp; @@ -400,7 +402,7 @@ retry: ip->i_flag |= IN_CHANGE | IN_UPDATE; allocbuf(bp, nsize); bp->b_flags |= B_DONE; - bzero(bp->b_data + osize, nsize - osize); + vfs_bio_bzero_buf(bp, osize, nsize - osize); if ((bp->b_flags & (B_MALLOC | B_VMIO)) == B_VMIO) vfs_bio_set_valid(bp, osize, nsize - osize); *bpp = bp; Index: /usr/src/sys/kern/vfs_vnops.c =================================================================== --- /usr/src/sys/kern/vfs_vnops.c (revision 244437) +++ /usr/src/sys/kern/vfs_vnops.c (working copy) @@ -1121,6 +1121,45 @@ vn_io_fault_uiomove(char *data, int xfersize, stru return (error); } +int +vn_io_fault_pgmove(vm_page_t ma[], vm_offset_t offset, int xfersize, + struct uio *uio) +{ + struct thread *td; + vm_offset_t iov_base; + int cnt, pgadv; + + td = curthread; + if ((td->td_pflags & TDP_UIOHELD) == 0 || + uio->uio_segflg != UIO_USERSPACE) + return (uiomove_fromphys(ma, offset, xfersize, uio)); + + KASSERT(uio->uio_iovcnt == 1, ("uio_iovcnt %d", uio->uio_iovcnt)); + cnt = xfersize > uio->uio_resid ? uio->uio_resid : xfersize; + iov_base = (vm_offset_t)uio->uio_iov->iov_base; + switch (uio->uio_rw) { + case UIO_WRITE: + pmap_copy_pages(td->td_ma, iov_base & PAGE_MASK, ma, + offset, cnt); + break; + case UIO_READ: + pmap_copy_pages(ma, offset, td->td_ma, iov_base & PAGE_MASK, + cnt); + break; + } + pgadv = ((iov_base + cnt) >> PAGE_SHIFT) - (iov_base >> PAGE_SHIFT); + td->td_ma += pgadv; + KASSERT(td->td_ma_cnt >= pgadv, ("consumed pages %d %d", td->td_ma_cnt, + pgadv)); + td->td_ma_cnt -= pgadv; + uio->uio_iov->iov_base = (char *)(iov_base + cnt); + uio->uio_iov->iov_len -= cnt; + uio->uio_resid -= cnt; + uio->uio_offset += cnt; + return (0); +} + + /* * File table truncate routine. */ Index: /usr/src/sys/kern/vfs_bio.c =================================================================== --- /usr/src/sys/kern/vfs_bio.c (revision 244437) +++ /usr/src/sys/kern/vfs_bio.c (working copy) @@ -91,6 +91,7 @@ struct buf_ops buf_ops_bio = { * carnal knowledge of buffers. This knowledge should be moved to vfs_bio.c. */ struct buf *buf; /* buffer header pool */ +caddr_t unmapped_buf; static struct proc *bufdaemonproc; @@ -131,6 +132,10 @@ SYSCTL_PROC(_vfs, OID_AUTO, bufspace, CTLTYPE_LONG SYSCTL_LONG(_vfs, OID_AUTO, bufspace, CTLFLAG_RD, &bufspace, 0, "Virtual memory used for buffers"); #endif +static long unmapped_bufspace; +SYSCTL_LONG(_vfs, OID_AUTO, unmapped_bufspace, CTLFLAG_RD, + &unmapped_bufspace, 0, + "Amount of unmapped buffers, inclusive in the bufspace"); static long maxbufspace; SYSCTL_LONG(_vfs, OID_AUTO, maxbufspace, CTLFLAG_RD, &maxbufspace, 0, "Maximum allowed value of bufspace (including buf_daemon)"); @@ -200,6 +205,10 @@ SYSCTL_INT(_vfs, OID_AUTO, getnewbufcalls, CTLFLAG static int getnewbufrestarts; SYSCTL_INT(_vfs, OID_AUTO, getnewbufrestarts, CTLFLAG_RW, &getnewbufrestarts, 0, "Number of times getnewbuf has had to restart a buffer aquisition"); +static int mappingrestarts; +SYSCTL_INT(_vfs, OID_AUTO, mappingrestarts, CTLFLAG_RW, &mappingrestarts, 0, + "Number of times getblk has had to restart a buffer mapping for " + "unmapped buffer"); static int flushbufqtarget = 100; SYSCTL_INT(_vfs, OID_AUTO, flushbufqtarget, CTLFLAG_RW, &flushbufqtarget, 0, "Amount of work to do in flushbufqueues when helping bufdaemon"); @@ -671,8 +680,57 @@ bufinit(void) bogus_page = vm_page_alloc(NULL, 0, VM_ALLOC_NOOBJ | VM_ALLOC_NORMAL | VM_ALLOC_WIRED); + unmapped_buf = (caddr_t)kmem_alloc_nofault(kernel_map, MAXPHYS); } +#ifdef INVARIANTS +static inline void +vfs_buf_check_mapped(struct buf *bp) +{ + + KASSERT((bp->b_flags & B_NOTMAPPED) == 0, + ("mapped buf %p %x", bp, bp->b_flags)); + KASSERT(bp->b_kvabase != unmapped_buf, + ("mapped buf: b_kvabase was not updated %p", bp)); + KASSERT(bp->b_data != unmapped_buf, + ("mapped buf: b_data was not updated %p", bp)); +} + +static inline void +vfs_buf_check_unmapped(struct buf *bp) +{ + + KASSERT((bp->b_flags & B_NOTMAPPED) == B_NOTMAPPED, + ("unmapped buf %p %x", bp, bp->b_flags)); + KASSERT(bp->b_kvabase == unmapped_buf, + ("unmapped buf: corrupted b_kvabase %p", bp)); + KASSERT(bp->b_data == unmapped_buf, + ("unmapped buf: corrupted b_data %p", bp)); +} + +#define BUF_CHECK_MAPPED(bp) vfs_buf_check_mapped(bp) +#define BUF_CHECK_UNMAPPED(bp) vfs_buf_check_unmapped(bp) +#else +#define BUF_CHECK_MAPPED(bp) do {} while (0) +#define BUF_CHECK_UNMAPPED(bp) do {} while (0) +#endif + +static void +bpmap_qenter(struct buf *bp) +{ + + BUF_CHECK_MAPPED(bp); + + /* + * bp->b_data is relative to bp->b_offset, but + * bp->b_offset may be offset into the first page. + */ + bp->b_data = (caddr_t)trunc_page((vm_offset_t)bp->b_data); + pmap_qenter((vm_offset_t)bp->b_data, bp->b_pages, bp->b_npages); + bp->b_data = (caddr_t)((vm_offset_t)bp->b_data | + (vm_offset_t)(bp->b_offset & PAGE_MASK)); +} + /* * bfreekva() - free the kva allocation for a buffer. * @@ -682,14 +740,26 @@ static void bfreekva(struct buf *bp) { - if (bp->b_kvasize) { - atomic_add_int(&buffreekvacnt, 1); - atomic_subtract_long(&bufspace, bp->b_kvasize); - vm_map_remove(buffer_map, (vm_offset_t) bp->b_kvabase, - (vm_offset_t) bp->b_kvabase + bp->b_kvasize); - bp->b_kvasize = 0; - bufspacewakeup(); + if (bp->b_kvasize == 0) + return; + + atomic_add_int(&buffreekvacnt, 1); + atomic_subtract_long(&bufspace, bp->b_kvasize); + if ((bp->b_flags & B_NOTMAPPED) == 0) { + BUF_CHECK_MAPPED(bp); + vm_map_remove(buffer_map, (vm_offset_t)bp->b_kvabase, + (vm_offset_t)bp->b_kvabase + bp->b_kvasize); + } else { + BUF_CHECK_UNMAPPED(bp); + if ((bp->b_flags & B_KVAALLOC) != 0) { + vm_map_remove(buffer_map, (vm_offset_t)bp->b_kvaalloc, + (vm_offset_t)bp->b_kvaalloc + bp->b_kvasize); + } + atomic_subtract_long(&unmapped_bufspace, bp->b_kvasize); + bp->b_flags &= ~(B_NOTMAPPED | B_KVAALLOC); } + bp->b_kvasize = 0; + bufspacewakeup(); } /* @@ -826,9 +896,8 @@ breada(struct vnode * vp, daddr_t * rablkno, int * * getblk(). Also starts asynchronous I/O on read-ahead blocks. */ int -breadn_flags(struct vnode * vp, daddr_t blkno, int size, - daddr_t * rablkno, int *rabsize, int cnt, - struct ucred * cred, int flags, struct buf **bpp) +breadn_flags(struct vnode *vp, daddr_t blkno, int size, daddr_t *rablkno, + int *rabsize, int cnt, struct ucred *cred, int flags, struct buf **bpp) { struct buf *bp; int rv = 0, readwait = 0; @@ -1363,7 +1432,8 @@ brelse(struct buf *bp) } } - if ((bp->b_flags & B_INVAL) == 0) { + if ((bp->b_flags & (B_INVAL | B_NOTMAPPED)) == 0) { + BUF_CHECK_MAPPED(bp); pmap_qenter( trunc_page((vm_offset_t)bp->b_data), bp->b_pages, bp->b_npages); @@ -1612,7 +1682,11 @@ vfs_vmio_release(struct buf *bp) int i; vm_page_t m; - pmap_qremove(trunc_page((vm_offset_t)bp->b_data), bp->b_npages); + if ((bp->b_flags & B_NOTMAPPED) == 0) { + BUF_CHECK_MAPPED(bp); + pmap_qremove(trunc_page((vm_offset_t)bp->b_data), bp->b_npages); + } else + BUF_CHECK_UNMAPPED(bp); VM_OBJECT_LOCK(bp->b_bufobj->bo_object); for (i = 0; i < bp->b_npages; i++) { m = bp->b_pages[i]; @@ -1716,8 +1790,10 @@ vfs_bio_awrite(struct buf *bp) int nwritten; int size; int maxcl; + int gbflags; bo = &vp->v_bufobj; + gbflags = (bp->b_flags & B_NOTMAPPED) != 0 ? GB_NOTMAPPED : 0; /* * right now we support clustered writing only to regular files. If * we find a clusterable block we could be in the middle of a cluster @@ -1748,8 +1824,9 @@ vfs_bio_awrite(struct buf *bp) */ if (ncl != 1) { BUF_UNLOCK(bp); - nwritten = cluster_wbuild(vp, size, lblkno - j, ncl); - return nwritten; + nwritten = cluster_wbuild(vp, size, lblkno - j, ncl, + gbflags); + return (nwritten); } } bremfree(bp); @@ -1796,6 +1873,9 @@ getnewbuf(struct vnode *vp, int slpflag, int slpti int nqindex; static int flushingbufs; + KASSERT((gbflags & (GB_NOTMAPPED | GB_KVAALLOC)) != GB_KVAALLOC, + ("GB_KVAALLOC only makes sense with GB_NOTMAPPED")); + td = curthread; /* * We can't afford to block since we might be holding a vnode lock, @@ -1969,7 +2049,7 @@ restart: if (bp->b_bufsize) allocbuf(bp, 0); - bp->b_flags = 0; + bp->b_flags &= B_NOTMAPPED; bp->b_ioflags = 0; bp->b_xflags = 0; KASSERT((bp->b_vflags & BV_INFREECNT) == 0, @@ -2096,6 +2176,14 @@ restart: } } mtx_unlock(&nblock); + } else if ((gbflags & (GB_NOTMAPPED | GB_KVAALLOC)) == GB_NOTMAPPED) { + bfreekva(bp); + bp->b_flags |= B_NOTMAPPED; + bp->b_kvabase = bp->b_data = unmapped_buf; + bp->b_kvasize = maxsize; + atomic_add_long(&bufspace, bp->b_kvasize); + atomic_add_long(&unmapped_bufspace, bp->b_kvasize); + atomic_add_int(&bufreusecnt, 1); } else { /* * We finally have a valid bp. We aren't quite out of the @@ -2105,7 +2193,8 @@ restart: */ maxsize = (maxsize + BKVAMASK) & ~BKVAMASK; - if (maxsize != bp->b_kvasize) { + if (maxsize != bp->b_kvasize || + (bp->b_flags & (B_NOTMAPPED | B_KVAALLOC)) == B_NOTMAPPED) { vm_offset_t addr = 0; int rv; @@ -2126,18 +2215,37 @@ restart: goto restart; } rv = vm_map_insert(buffer_map, NULL, 0, addr, - addr + maxsize, VM_PROT_ALL, VM_PROT_ALL, + addr + maxsize, VM_PROT_RW, VM_PROT_RW, MAP_NOFAULT); KASSERT(rv == KERN_SUCCESS, ("vm_map_insert(buffer_map) rv %d", rv)); vm_map_unlock(buffer_map); - bp->b_kvabase = (caddr_t)addr; + if ((gbflags & GB_NOTMAPPED) == 0) { + bp->b_kvabase = (caddr_t)addr; + } else if ((gbflags & GB_KVAALLOC) != 0) { + KASSERT((gbflags & GB_NOTMAPPED) != 0, + ("GB_KVAALLOC without GB_NOTMAPPED")); + bp->b_kvaalloc = (caddr_t)addr; + bp->b_flags |= B_KVAALLOC; + atomic_add_long(&unmapped_bufspace, + bp->b_kvasize); + } bp->b_kvasize = maxsize; atomic_add_long(&bufspace, bp->b_kvasize); atomic_add_int(&bufreusecnt, 1); + } else if ((bp->b_flags & B_KVAALLOC) != 0 && + (gbflags & (GB_NOTMAPPED | GB_KVAALLOC)) == 0) { + bp->b_kvabase = bp->b_kvaalloc; + bp->b_flags &= ~B_KVAALLOC; + atomic_subtract_long(&unmapped_bufspace, + bp->b_kvasize); } - bp->b_saveaddr = bp->b_kvabase; - bp->b_data = bp->b_saveaddr; + if ((gbflags & GB_NOTMAPPED) == 0) { + bp->b_saveaddr = bp->b_kvabase; + bp->b_data = bp->b_saveaddr; + bp->b_flags &= ~B_NOTMAPPED; + BUF_CHECK_MAPPED(bp); + } } return (bp); } @@ -2584,14 +2692,18 @@ vfs_setdirty_locked_object(struct buf *bp) * prior to issuing the READ. biodone() will *not* clear B_INVAL. */ struct buf * -getblk(struct vnode * vp, daddr_t blkno, int size, int slpflag, int slptimeo, +getblk(struct vnode *vp, daddr_t blkno, int size, int slpflag, int slptimeo, int flags) { - struct buf *bp; + struct buf *bp, *scratch_bp; struct bufobj *bo; - int error; + vm_offset_t addr; + int bsize, error, maxsize, need_mapping, need_kva, vmio; + off_t offset; CTR3(KTR_BUF, "getblk(%p, %ld, %d)", vp, (long)blkno, size); + KASSERT((flags & (GB_NOTMAPPED | GB_KVAALLOC)) != GB_KVAALLOC, + ("GB_KVAALLOC only makes sense with GB_NOTMAPPED")); ASSERT_VOP_LOCKED(vp, "getblk"); if (size > MAXBSIZE) panic("getblk: size(%d) > MAXBSIZE(%d)\n", size, MAXBSIZE); @@ -2659,9 +2771,8 @@ loop: } /* - * check for size inconsistancies for non-VMIO case. + * check for size inconsistencies for non-VMIO case. */ - if (bp->b_bcount != size) { if ((bp->b_flags & B_VMIO) == 0 || (size > bp->b_kvasize)) { @@ -2694,13 +2805,85 @@ loop: } } + need_mapping = (bp->b_flags & B_NOTMAPPED) != 0 && + (flags & GB_NOTMAPPED) == 0; + need_kva = (bp->b_flags & (B_KVAALLOC | B_NOTMAPPED)) == + B_NOTMAPPED && (flags & GB_KVAALLOC) != 0; + if (need_mapping || need_kva) { + BUF_CHECK_UNMAPPED(bp); + + if (need_mapping && (bp->b_flags & B_KVAALLOC) != 0) { + bp->b_flags &= ~B_KVAALLOC; + bp->b_kvabase = bp->b_kvaalloc; + atomic_subtract_long(&unmapped_bufspace, + bp->b_kvasize); + goto has_addr; + } + + bsize = vn_isdisk(vp, NULL) ? DEV_BSIZE : bo->bo_bsize; + offset = blkno * bsize; + maxsize = size + (offset & PAGE_MASK); + maxsize = imax(maxsize, bsize); +mapping_loop: + vm_map_lock(buffer_map); + if (vm_map_findspace(buffer_map, + vm_map_min(buffer_map), maxsize, &addr)) { + vm_map_unlock(buffer_map); + scratch_bp = getnewbuf(vp, 0, 0, size, + maxsize, flags | (GB_NOTMAPPED | + GB_KVAALLOC)); + if (scratch_bp == NULL) { + if ((flags & GB_NOWAIT_BD) != 0) { + /* + * XXXKIB: not sure + * what else to do. + */ + panic("GB_NOWAIT_BD and " + "B_NOTMAPPED %p", bp); + } + atomic_add_int(&mappingrestarts, 1); + goto mapping_loop; + } + atomic_subtract_long(&bufspace, bp->b_kvasize); + bufspacewakeup(); + addr = (vm_offset_t)scratch_bp->b_kvaalloc; + maxsize = scratch_bp->b_kvasize; + scratch_bp->b_kvasize = 0; + scratch_bp->b_flags |= B_INVAL | B_NOTMAPPED; + brelse(scratch_bp); + } else { + KASSERT(addr != 0, ("addr == 0")); + vm_map_insert(buffer_map, NULL, 0, addr, + addr + maxsize, VM_PROT_ALL, VM_PROT_ALL, + MAP_NOFAULT); + vm_map_unlock(buffer_map); + } + if (need_mapping) { + bp->b_kvabase = (caddr_t)addr; + bp->b_flags &= ~B_KVAALLOC; + atomic_subtract_long(&unmapped_bufspace, + bp->b_kvasize); + } else /* if (need_kva) */ { + bp->b_kvaalloc = (caddr_t)addr; + bp->b_flags |= B_KVAALLOC; + } + bp->b_kvasize = maxsize; + if (need_mapping) { +has_addr: + bp->b_saveaddr = bp->b_kvabase; + bp->b_data = bp->b_saveaddr; + bp->b_flags &= ~B_NOTMAPPED; + BUF_CHECK_MAPPED(bp); + bpmap_qenter(bp); + } + } + /* * If the size is inconsistant in the VMIO case, we can resize * the buffer. This might lead to B_CACHE getting set or * cleared. If the size has not changed, B_CACHE remains * unchanged from its previous state. */ - if (bp->b_bcount != size) allocbuf(bp, size); @@ -2741,9 +2924,6 @@ loop: } bp->b_flags &= ~B_DONE; } else { - int bsize, maxsize, vmio; - off_t offset; - /* * Buffer is not in-core, create new buffer. The buffer * returned by getnewbuf() is locked. Note that the returned @@ -2989,10 +3169,14 @@ allocbuf(struct buf *bp, int size) if (desiredpages < bp->b_npages) { vm_page_t m; - pmap_qremove((vm_offset_t)trunc_page( - (vm_offset_t)bp->b_data) + - (desiredpages << PAGE_SHIFT), - (bp->b_npages - desiredpages)); + if ((bp->b_flags & B_NOTMAPPED) == 0) { + BUF_CHECK_MAPPED(bp); + pmap_qremove((vm_offset_t)trunc_page( + (vm_offset_t)bp->b_data) + + (desiredpages << PAGE_SHIFT), + (bp->b_npages - desiredpages)); + } else + BUF_CHECK_UNMAPPED(bp); VM_OBJECT_LOCK(bp->b_bufobj->bo_object); for (i = desiredpages; i < bp->b_npages; i++) { /* @@ -3098,21 +3282,12 @@ allocbuf(struct buf *bp, int size) VM_OBJECT_UNLOCK(obj); /* - * Step 3, fixup the KVM pmap. Remember that - * bp->b_data is relative to bp->b_offset, but - * bp->b_offset may be offset into the first page. + * Step 3, fixup the KVM pmap. */ - - bp->b_data = (caddr_t) - trunc_page((vm_offset_t)bp->b_data); - pmap_qenter( - (vm_offset_t)bp->b_data, - bp->b_pages, - bp->b_npages - ); - - bp->b_data = (caddr_t)((vm_offset_t)bp->b_data | - (vm_offset_t)(bp->b_offset & PAGE_MASK)); + if ((bp->b_flags & B_NOTMAPPED) == 0) + bpmap_qenter(bp); + else + BUF_CHECK_UNMAPPED(bp); } } if (newbsize < bp->b_bufsize) @@ -3127,16 +3302,28 @@ biodone(struct bio *bp) { struct mtx *mtxp; void (*done)(struct bio *); + vm_offset_t start, end; + int transient; mtxp = mtx_pool_find(mtxpool_sleep, bp); mtx_lock(mtxp); bp->bio_flags |= BIO_DONE; + if ((bp->bio_flags & BIO_TRANSIENT_MAPPING) != 0) { + start = trunc_page((vm_offset_t)bp->bio_data); + end = round_page((vm_offset_t)bp->bio_data + bp->bio_length); + transient = 1; + } else + transient = 0; done = bp->bio_done; if (done == NULL) wakeup(bp); mtx_unlock(mtxp); if (done != NULL) done(bp); + if (transient) { + pmap_qremove(start, OFF_TO_IDX(end - start)); + vm_map_remove(bio_transient_map, start, end); + } } /* @@ -3393,9 +3580,11 @@ bufdone_finish(struct buf *bp) } vm_object_pip_wakeupn(obj, 0); VM_OBJECT_UNLOCK(obj); - if (bogus) + if (bogus && (bp->b_flags & B_NOTMAPPED) == 0) { + BUF_CHECK_MAPPED(bp); pmap_qenter(trunc_page((vm_offset_t)bp->b_data), bp->b_pages, bp->b_npages); + } } /* @@ -3438,8 +3627,12 @@ vfs_unbusy_pages(struct buf *bp) if (!m) panic("vfs_unbusy_pages: page missing\n"); bp->b_pages[i] = m; - pmap_qenter(trunc_page((vm_offset_t)bp->b_data), - bp->b_pages, bp->b_npages); + if ((bp->b_flags & B_NOTMAPPED) == 0) { + BUF_CHECK_MAPPED(bp); + pmap_qenter(trunc_page((vm_offset_t)bp->b_data), + bp->b_pages, bp->b_npages); + } else + BUF_CHECK_UNMAPPED(bp); } vm_object_pip_subtract(obj, 1); vm_page_io_finish(m); @@ -3604,9 +3797,11 @@ vfs_busy_pages(struct buf *bp, int clear_modify) foff = (foff + PAGE_SIZE) & ~(off_t)PAGE_MASK; } VM_OBJECT_UNLOCK(obj); - if (bogus) + if (bogus && (bp->b_flags & B_NOTMAPPED) == 0) { + BUF_CHECK_MAPPED(bp); pmap_qenter(trunc_page((vm_offset_t)bp->b_data), bp->b_pages, bp->b_npages); + } } /* @@ -3662,8 +3857,7 @@ vfs_bio_set_valid(struct buf *bp, int base, int si void vfs_bio_clrbuf(struct buf *bp) { - int i, j, mask; - caddr_t sa, ea; + int i, j, mask, sa, ea, slide; if ((bp->b_flags & (B_VMIO | B_MALLOC)) != B_VMIO) { clrbuf(bp); @@ -3681,39 +3875,69 @@ vfs_bio_clrbuf(struct buf *bp) if ((bp->b_pages[0]->valid & mask) == mask) goto unlock; if ((bp->b_pages[0]->valid & mask) == 0) { - bzero(bp->b_data, bp->b_bufsize); + pmap_zero_page_area(bp->b_pages[0], 0, bp->b_bufsize); bp->b_pages[0]->valid |= mask; goto unlock; } } - ea = sa = bp->b_data; - for(i = 0; i < bp->b_npages; i++, sa = ea) { - ea = (caddr_t)trunc_page((vm_offset_t)sa + PAGE_SIZE); - ea = (caddr_t)(vm_offset_t)ulmin( - (u_long)(vm_offset_t)ea, - (u_long)(vm_offset_t)bp->b_data + bp->b_bufsize); + sa = bp->b_offset & PAGE_MASK; + slide = 0; + for (i = 0; i < bp->b_npages; i++) { + slide = imin(slide + PAGE_SIZE, bp->b_bufsize + sa); + ea = slide & PAGE_MASK; + if (ea == 0) + ea = PAGE_SIZE; if (bp->b_pages[i] == bogus_page) continue; - j = ((vm_offset_t)sa & PAGE_MASK) / DEV_BSIZE; + j = sa / DEV_BSIZE; mask = ((1 << ((ea - sa) / DEV_BSIZE)) - 1) << j; VM_OBJECT_LOCK_ASSERT(bp->b_pages[i]->object, MA_OWNED); if ((bp->b_pages[i]->valid & mask) == mask) continue; if ((bp->b_pages[i]->valid & mask) == 0) - bzero(sa, ea - sa); + pmap_zero_page_area(bp->b_pages[i], sa, ea - sa); else { for (; sa < ea; sa += DEV_BSIZE, j++) { - if ((bp->b_pages[i]->valid & (1 << j)) == 0) - bzero(sa, DEV_BSIZE); + if ((bp->b_pages[i]->valid & (1 << j)) == 0) { + pmap_zero_page_area(bp->b_pages[i], + sa, DEV_BSIZE); + } } } bp->b_pages[i]->valid |= mask; + sa = 0; } unlock: VM_OBJECT_UNLOCK(bp->b_bufobj->bo_object); bp->b_resid = 0; } +void +vfs_bio_bzero_buf(struct buf *bp, int base, int size) +{ + vm_page_t m; + int i, n; + + if ((bp->b_flags & B_NOTMAPPED) == 0) { + BUF_CHECK_MAPPED(bp); + bzero(bp->b_data + base, size); + } else { + BUF_CHECK_UNMAPPED(bp); + n = PAGE_SIZE - (base & PAGE_MASK); + VM_OBJECT_LOCK(bp->b_bufobj->bo_object); + for (i = base / PAGE_SIZE; size > 0 && i < bp->b_npages; ++i) { + m = bp->b_pages[i]; + if (n > size) + n = size; + pmap_zero_page_area(m, base & PAGE_MASK, n); + base += n; + size -= n; + n = PAGE_SIZE; + } + VM_OBJECT_UNLOCK(bp->b_bufobj->bo_object); + } +} + /* * vm_hold_load_pages and vm_hold_free_pages get pages into * a buffers address space. The pages are anonymous and are @@ -3726,6 +3950,8 @@ vm_hold_load_pages(struct buf *bp, vm_offset_t fro vm_page_t p; int index; + BUF_CHECK_MAPPED(bp); + to = round_page(to); from = round_page(from); index = (from - trunc_page((vm_offset_t)bp->b_data)) >> PAGE_SHIFT; @@ -3757,6 +3983,8 @@ vm_hold_free_pages(struct buf *bp, int newbsize) vm_page_t p; int index, newnpages; + BUF_CHECK_MAPPED(bp); + from = round_page((vm_offset_t)bp->b_data + newbsize); newnpages = (from - trunc_page((vm_offset_t)bp->b_data)) >> PAGE_SHIFT; if (bp->b_npages > newnpages) Index: /usr/src/sys/kern/vfs_cluster.c =================================================================== --- /usr/src/sys/kern/vfs_cluster.c (revision 244437) +++ /usr/src/sys/kern/vfs_cluster.c (working copy) @@ -60,11 +60,11 @@ SYSCTL_INT(_debug, OID_AUTO, rcluster, CTLFLAG_RW, static MALLOC_DEFINE(M_SEGMENT, "cl_savebuf", "cluster_save buffer"); -static struct cluster_save * - cluster_collectbufs(struct vnode *vp, struct buf *last_bp); -static struct buf * - cluster_rbuild(struct vnode *vp, u_quad_t filesize, daddr_t lbn, - daddr_t blkno, long size, int run, struct buf *fbp); +static struct cluster_save *cluster_collectbufs(struct vnode *vp, + struct buf *last_bp, int gbflags); +static struct buf *cluster_rbuild(struct vnode *vp, u_quad_t filesize, + daddr_t lbn, daddr_t blkno, long size, int run, int gbflags, + struct buf *fbp); static void cluster_callback(struct buf *); static int write_behind = 1; @@ -83,15 +83,9 @@ extern vm_page_t bogus_page; * cluster_read replaces bread. */ int -cluster_read(vp, filesize, lblkno, size, cred, totread, seqcount, bpp) - struct vnode *vp; - u_quad_t filesize; - daddr_t lblkno; - long size; - struct ucred *cred; - long totread; - int seqcount; - struct buf **bpp; +cluster_read(struct vnode *vp, u_quad_t filesize, daddr_t lblkno, long size, + struct ucred *cred, long totread, int seqcount, int gbflags, + struct buf **bpp) { struct buf *bp, *rbp, *reqbp; struct bufobj *bo; @@ -117,7 +111,7 @@ int /* * get the requested block */ - *bpp = reqbp = bp = getblk(vp, lblkno, size, 0, 0, 0); + *bpp = reqbp = bp = getblk(vp, lblkno, size, 0, 0, gbflags); origblkno = lblkno; /* @@ -208,7 +202,7 @@ int if (ncontig < nblks) nblks = ncontig; bp = cluster_rbuild(vp, filesize, lblkno, - blkno, size, nblks, bp); + blkno, size, nblks, gbflags, bp); lblkno += (bp->b_bufsize / size); } else { bp->b_flags |= B_RAM; @@ -252,14 +246,14 @@ int if (ncontig) { ncontig = min(ncontig + 1, racluster); rbp = cluster_rbuild(vp, filesize, lblkno, blkno, - size, ncontig, NULL); + size, ncontig, gbflags, NULL); lblkno += (rbp->b_bufsize / size); if (rbp->b_flags & B_DELWRI) { bqrelse(rbp); continue; } } else { - rbp = getblk(vp, lblkno, size, 0, 0, 0); + rbp = getblk(vp, lblkno, size, 0, 0, gbflags); lblkno += 1; if (rbp->b_flags & B_DELWRI) { bqrelse(rbp); @@ -298,14 +292,8 @@ int * and then parcel them up into logical blocks in the buffer hash table. */ static struct buf * -cluster_rbuild(vp, filesize, lbn, blkno, size, run, fbp) - struct vnode *vp; - u_quad_t filesize; - daddr_t lbn; - daddr_t blkno; - long size; - int run; - struct buf *fbp; +cluster_rbuild(struct vnode *vp, u_quad_t filesize, daddr_t lbn, + daddr_t blkno, long size, int run, int gbflags, struct buf *fbp) { struct bufobj *bo; struct buf *bp, *tbp; @@ -329,7 +317,7 @@ static struct buf * tbp = fbp; tbp->b_iocmd = BIO_READ; } else { - tbp = getblk(vp, lbn, size, 0, 0, 0); + tbp = getblk(vp, lbn, size, 0, 0, gbflags); if (tbp->b_flags & B_CACHE) return tbp; tbp->b_flags |= B_ASYNC | B_RAM; @@ -350,9 +338,14 @@ static struct buf * * address may not be either. Inherit the b_data offset * from the original buffer. */ - bp->b_data = (char *)((vm_offset_t)bp->b_data | - ((vm_offset_t)tbp->b_data & PAGE_MASK)); bp->b_flags = B_ASYNC | B_CLUSTER | B_VMIO; + if ((gbflags & GB_NOTMAPPED) != 0) { + bp->b_flags |= B_NOTMAPPED; + bp->b_data = unmapped_buf; + } else { + bp->b_data = (char *)((vm_offset_t)bp->b_data | + ((vm_offset_t)tbp->b_data & PAGE_MASK)); + } bp->b_iocmd = BIO_READ; bp->b_iodone = cluster_callback; bp->b_blkno = blkno; @@ -499,8 +492,10 @@ static struct buf * bp->b_bufsize, bp->b_kvasize); bp->b_kvasize = bp->b_bufsize; - pmap_qenter(trunc_page((vm_offset_t) bp->b_data), - (vm_page_t *)bp->b_pages, bp->b_npages); + if ((bp->b_flags & B_NOTMAPPED) == 0) { + pmap_qenter(trunc_page((vm_offset_t) bp->b_data), + (vm_page_t *)bp->b_pages, bp->b_npages); + } return (bp); } @@ -523,7 +518,10 @@ cluster_callback(bp) if (bp->b_ioflags & BIO_ERROR) error = bp->b_error; - pmap_qremove(trunc_page((vm_offset_t) bp->b_data), bp->b_npages); + if ((bp->b_flags & B_NOTMAPPED) == 0) { + pmap_qremove(trunc_page((vm_offset_t) bp->b_data), + bp->b_npages); + } /* * Move memory from the large cluster buffer into the component * buffers and mark IO as done on these. @@ -565,18 +563,19 @@ cluster_callback(bp) */ static __inline int -cluster_wbuild_wb(struct vnode *vp, long size, daddr_t start_lbn, int len) +cluster_wbuild_wb(struct vnode *vp, long size, daddr_t start_lbn, int len, + int gbflags) { int r = 0; - switch(write_behind) { + switch (write_behind) { case 2: if (start_lbn < len) break; start_lbn -= len; /* FALLTHROUGH */ case 1: - r = cluster_wbuild(vp, size, start_lbn, len); + r = cluster_wbuild(vp, size, start_lbn, len, gbflags); /* FALLTHROUGH */ default: /* FALLTHROUGH */ @@ -596,7 +595,8 @@ static __inline int * 4. end of a cluster - asynchronously write cluster */ void -cluster_write(struct vnode *vp, struct buf *bp, u_quad_t filesize, int seqcount) +cluster_write(struct vnode *vp, struct buf *bp, u_quad_t filesize, int seqcount, + int gbflags) { daddr_t lbn; int maxclen, cursize; @@ -642,13 +642,13 @@ void lbn != vp->v_lastw + 1 || vp->v_clen <= cursize) { if (!async && seqcount > 0) { cluster_wbuild_wb(vp, lblocksize, - vp->v_cstart, cursize); + vp->v_cstart, cursize, gbflags); } } else { struct buf **bpp, **endbp; struct cluster_save *buflist; - buflist = cluster_collectbufs(vp, bp); + buflist = cluster_collectbufs(vp, bp, gbflags); endbp = &buflist->bs_children [buflist->bs_nchildren - 1]; if (VOP_REALLOCBLKS(vp, buflist)) { @@ -667,7 +667,7 @@ void if (seqcount > 1) { cluster_wbuild_wb(vp, lblocksize, vp->v_cstart, - cursize); + cursize, gbflags); } } else { /* @@ -715,8 +715,10 @@ void * update daemon handle it. */ bdwrite(bp); - if (seqcount > 1) - cluster_wbuild_wb(vp, lblocksize, vp->v_cstart, vp->v_clen + 1); + if (seqcount > 1) { + cluster_wbuild_wb(vp, lblocksize, vp->v_cstart, + vp->v_clen + 1, gbflags); + } vp->v_clen = 0; vp->v_cstart = lbn + 1; } else if (vm_page_count_severe()) { @@ -742,11 +744,8 @@ void * the current block (if last_bp == NULL). */ int -cluster_wbuild(vp, size, start_lbn, len) - struct vnode *vp; - long size; - daddr_t start_lbn; - int len; +cluster_wbuild(struct vnode *vp, long size, daddr_t start_lbn, int len, + int gbflags) { struct buf *bp, *tbp; struct bufobj *bo; @@ -832,10 +831,16 @@ int * address may not be either. Inherit the b_data offset * from the original buffer. */ - bp->b_data = (char *)((vm_offset_t)bp->b_data | - ((vm_offset_t)tbp->b_data & PAGE_MASK)); - bp->b_flags |= B_CLUSTER | - (tbp->b_flags & (B_VMIO | B_NEEDCOMMIT)); + if ((gbflags & GB_NOTMAPPED) == 0 || + (tbp->b_flags & B_VMIO) == 0) { + bp->b_data = (char *)((vm_offset_t)bp->b_data | + ((vm_offset_t)tbp->b_data & PAGE_MASK)); + } else { + bp->b_flags |= B_NOTMAPPED; + bp->b_data = unmapped_buf; + } + bp->b_flags |= B_CLUSTER | (tbp->b_flags & (B_VMIO | + B_NEEDCOMMIT)); bp->b_iodone = cluster_callback; pbgetvp(vp, bp); /* @@ -956,8 +961,10 @@ int tbp, b_cluster.cluster_entry); } finishcluster: - pmap_qenter(trunc_page((vm_offset_t) bp->b_data), - (vm_page_t *) bp->b_pages, bp->b_npages); + if ((bp->b_flags & B_NOTMAPPED) == 0) { + pmap_qenter(trunc_page((vm_offset_t) bp->b_data), + (vm_page_t *)bp->b_pages, bp->b_npages); + } if (bp->b_bufsize > bp->b_kvasize) panic( "cluster_wbuild: b_bufsize(%ld) > b_kvasize(%d)\n", @@ -978,9 +985,7 @@ int * Plus add one additional buffer. */ static struct cluster_save * -cluster_collectbufs(vp, last_bp) - struct vnode *vp; - struct buf *last_bp; +cluster_collectbufs(struct vnode *vp, struct buf *last_bp, int gbflags) { struct cluster_save *buflist; struct buf *bp; @@ -993,7 +998,8 @@ static struct cluster_save * buflist->bs_nchildren = 0; buflist->bs_children = (struct buf **) (buflist + 1); for (lbn = vp->v_cstart, i = 0; i < len; lbn++, i++) { - (void) bread(vp, lbn, last_bp->b_bcount, NOCRED, &bp); + (void)bread_gb(vp, lbn, last_bp->b_bcount, NOCRED, + gbflags, &bp); buflist->bs_children[i] = bp; if (bp->b_blkno == bp->b_lblkno) VOP_BMAP(vp, bp->b_lblkno, NULL, &bp->b_blkno, Index: /usr/src/sys/dev/ahci/ahci.c =================================================================== --- /usr/src/sys/dev/ahci/ahci.c (revision 244437) +++ /usr/src/sys/dev/ahci/ahci.c (working copy) @@ -37,6 +37,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -51,6 +52,14 @@ __FBSDID("$FreeBSD$"); #include #include +#include +#include + +static int ahci_unmapped; +SYSCTL_INT(_debug, OID_AUTO, ahci_unmapped, CTLFLAG_RD, + &ahci_unmapped, 0, + ""); + /* local prototypes */ static int ahci_setup_interrupt(device_t dev); static void ahci_intr(void *data); @@ -95,6 +104,10 @@ static void ahci_process_request_sense(device_t de static void ahciaction(struct cam_sim *sim, union ccb *ccb); static void ahcipoll(struct cam_sim *sim); +struct vm_page; +static void ahci_unmappedprd(struct ahci_slot *slot, struct vm_page **ma, + int ma_offset, bus_size_t size); + static MALLOC_DEFINE(M_AHCI, "AHCI driver", "AHCI driver data buffers"); static struct { @@ -1675,12 +1688,19 @@ ahci_begin_transaction(device_t dev, union ccb *cc /* If request moves data, setup and load SG list */ if ((ccb->ccb_h.flags & CAM_DIR_MASK) != CAM_DIR_NONE) { void *buf; + struct vm_page **ma; bus_size_t size; slot->state = AHCI_SLOT_LOADING; if (ccb->ccb_h.func_code == XPT_ATA_IO) { + ma = ccb->ataio.ma; + size = ccb->ataio.dxfer_len; + if (ma != NULL) { + ahci_unmappedprd(slot, ma, ccb->ataio.ma_offset, + size); + return; + } buf = ccb->ataio.data_ptr; - size = ccb->ataio.dxfer_len; } else { buf = ccb->csio.data_ptr; size = ccb->csio.dxfer_len; @@ -1691,6 +1711,37 @@ ahci_begin_transaction(device_t dev, union ccb *cc ahci_execute_transaction(slot); } +static void +ahci_unmappedprd(struct ahci_slot *slot, struct vm_page **ma, int ma_offset, + bus_size_t size) +{ + struct ahci_channel *ch = device_get_softc(slot->dev); + struct ahci_cmd_tab *ctp; + struct ahci_dma_prd *prd; + bus_size_t left, c; + int i, npages; + + npages = (size + PAGE_SIZE - 1) / PAGE_SIZE; + KASSERT(npages <= AHCI_SG_ENTRIES, ("too many DMA segment entries")); + ctp = (struct ahci_cmd_tab *) + (ch->dma.work + AHCI_CT_OFFSET + (AHCI_CT_SIZE * slot->slot)); + prd = &ctp->prd_tab[0]; + for (i = 0, left = size; i < npages; i++, left -= PAGE_SIZE) { + prd[i].dba = htole64(VM_PAGE_TO_PHYS(ma[i]) + ma_offset); + ma_offset = 0; + c = (left > PAGE_SIZE) ? PAGE_SIZE : left; + prd[i].dbc = htole32((c - 1) & AHCI_PRD_MASK); + } + slot->dma.nsegs = npages; +#if 0 + bus_dmamap_sync(ch->dma.data_tag, slot->dma.data_map, + ((slot->ccb->ccb_h.flags & CAM_DIR_IN) ? + BUS_DMASYNC_PREREAD : BUS_DMASYNC_PREWRITE)); +#endif + atomic_add_int(&ahci_unmapped, 1); + ahci_execute_transaction(slot); +} + /* Locked by busdma engine. */ static void ahci_dmasetprd(void *arg, bus_dma_segment_t *segs, int nsegs, int error) Index: /usr/src/sys/dev/md/md.c =================================================================== --- /usr/src/sys/dev/md/md.c (revision 244437) +++ /usr/src/sys/dev/md/md.c (working copy) @@ -110,6 +110,19 @@ static int md_malloc_wait; SYSCTL_INT(_vm, OID_AUTO, md_malloc_wait, CTLFLAG_RW, &md_malloc_wait, 0, "Allow malloc to wait for memory allocations"); +static int md_unmapped_swap; +SYSCTL_INT(_debug, OID_AUTO, md_unmapped_swap, CTLFLAG_RD, + &md_unmapped_swap, 0, + ""); +static int md_unmapped_vnode; +SYSCTL_INT(_debug, OID_AUTO, md_unmapped_vnode, CTLFLAG_RD, + &md_unmapped_vnode, 0, + ""); +static int md_unmapped_malloc; +SYSCTL_INT(_debug, OID_AUTO, md_unmapped_malloc, CTLFLAG_RD, + &md_unmapped_malloc, 0, + ""); + #if defined(MD_ROOT) && !defined(MD_ROOT_FSTYPE) #define MD_ROOT_FSTYPE "ufs" #endif @@ -414,13 +427,102 @@ g_md_start(struct bio *bp) wakeup(sc); } +#define MD_MALLOC_MOVE_ZERO 1 +#define MD_MALLOC_MOVE_FILL 2 +#define MD_MALLOC_MOVE_READ 3 +#define MD_MALLOC_MOVE_WRITE 4 +#define MD_MALLOC_MOVE_CMP 5 + static int +md_malloc_move(vm_page_t **mp, vm_offset_t *ma_offs, unsigned sectorsize, + void *ptr, u_char fill, int op) +{ + struct sf_buf *sf; + vm_page_t m, *mp1; + char *p, first; + vm_offset_t ma_offs1; + off_t *uc; + unsigned n; + int error, i, sz, first_read; + + m = NULL; + error = 0; + sf = NULL; + if (op == MD_MALLOC_MOVE_CMP) { + first_read = 0; + uc = ptr; + mp1 = *mp; + ma_offs1 = *ma_offs; + } + sched_pin(); + for (n = sectorsize; n != 0; n -= sz) { + sz = imin(PAGE_SIZE - *ma_offs, n); + if (m != **mp) { + if (sf != NULL) + sf_buf_free(sf); + m = **mp; + sf = sf_buf_alloc(m, SFB_CPUPRIVATE | + (md_malloc_wait ? 0 : SFB_NOWAIT)); + if (sf == NULL) { + error = ENOMEM; + break; + } + } + p = (char *)sf_buf_kva(sf) + *ma_offs; + switch (op) { + case MD_MALLOC_MOVE_ZERO: + bzero(p, sz); + break; + case MD_MALLOC_MOVE_FILL: + memset(p, fill, sz); + break; + case MD_MALLOC_MOVE_READ: + bcopy(ptr, p, sz); + cpu_flush_dcache(p, sz); + break; + case MD_MALLOC_MOVE_WRITE: + bcopy(p, ptr, sz); + break; + case MD_MALLOC_MOVE_CMP: + for (i = 0; i < sz; i++, p++) { + if (!first_read) { + *uc = *p; + first = *p; + first_read = 1; + } else if (*p != first) { + error = EDOOFUS; + break; + } + } + break; + } + if (error != 0) + break; + *ma_offs += sz; + *ma_offs %= PAGE_SIZE; + if (*ma_offs == 0) + (*mp)++; + } + + if (sf != NULL) + sf_buf_free(sf); + sched_unpin(); + if (op == MD_MALLOC_MOVE_CMP) { + *mp = mp1; + *ma_offs = ma_offs1; + } + return (error); +} + +static int mdstart_malloc(struct md_s *sc, struct bio *bp) { - int i, error; u_char *dst; + vm_page_t *m; + int i, error, error1, notmapped; off_t secno, nsec, uc; uintptr_t sp, osp; + vm_offset_t ma_offs; switch (bp->bio_cmd) { case BIO_READ: @@ -431,9 +533,16 @@ mdstart_malloc(struct md_s *sc, struct bio *bp) return (EOPNOTSUPP); } + notmapped = (bp->bio_flags & BIO_NOTMAPPED) != 0; + if (notmapped) { + m = bp->bio_ma; + ma_offs = bp->bio_ma_offset; + } else { + dst = bp->bio_data; + } + nsec = bp->bio_length / sc->sectorsize; secno = bp->bio_offset / sc->sectorsize; - dst = bp->bio_data; error = 0; while (nsec--) { osp = s_read(sc->indir, secno); @@ -441,21 +550,45 @@ mdstart_malloc(struct md_s *sc, struct bio *bp) if (osp != 0) error = s_write(sc->indir, secno, 0); } else if (bp->bio_cmd == BIO_READ) { - if (osp == 0) - bzero(dst, sc->sectorsize); - else if (osp <= 255) - memset(dst, osp, sc->sectorsize); - else { - bcopy((void *)osp, dst, sc->sectorsize); - cpu_flush_dcache(dst, sc->sectorsize); + if (osp == 0) { + if (notmapped) { + error = md_malloc_move(&m, &ma_offs, + sc->sectorsize, NULL, 0, + MD_MALLOC_MOVE_ZERO); + } else + bzero(dst, sc->sectorsize); + } else if (osp <= 255) { + if (notmapped) { + error = md_malloc_move(&m, &ma_offs, + sc->sectorsize, NULL, osp, + MD_MALLOC_MOVE_FILL); + } else + memset(dst, osp, sc->sectorsize); + } else { + if (notmapped) { + error = md_malloc_move(&m, &ma_offs, + sc->sectorsize, (void *)osp, 0, + MD_MALLOC_MOVE_READ); + } else { + bcopy((void *)osp, dst, sc->sectorsize); + cpu_flush_dcache(dst, sc->sectorsize); + } } osp = 0; } else if (bp->bio_cmd == BIO_WRITE) { if (sc->flags & MD_COMPRESS) { - uc = dst[0]; - for (i = 1; i < sc->sectorsize; i++) - if (dst[i] != uc) - break; + if (notmapped) { + error1 = md_malloc_move(&m, &ma_offs, + sc->sectorsize, &uc, 0, + MD_MALLOC_MOVE_CMP); + i = error1 == 0 ? sc->sectorsize : 0; + } else { + uc = dst[0]; + for (i = 1; i < sc->sectorsize; i++) { + if (dst[i] != uc) + break; + } + } } else { i = 0; uc = 0; @@ -472,10 +605,26 @@ mdstart_malloc(struct md_s *sc, struct bio *bp) error = ENOSPC; break; } - bcopy(dst, (void *)sp, sc->sectorsize); + if (notmapped) { + error = md_malloc_move(&m, + &ma_offs, sc->sectorsize, + (void *)sp, 0, + MD_MALLOC_MOVE_WRITE); + } else { + bcopy(dst, (void *)sp, + sc->sectorsize); + } error = s_write(sc->indir, secno, sp); } else { - bcopy(dst, (void *)osp, sc->sectorsize); + if (notmapped) { + error = md_malloc_move(&m, + &ma_offs, sc->sectorsize, + (void *)osp, 0, + MD_MALLOC_MOVE_WRITE); + } else { + bcopy(dst, (void *)osp, + sc->sectorsize); + } osp = 0; } } @@ -487,7 +636,8 @@ mdstart_malloc(struct md_s *sc, struct bio *bp) if (error != 0) break; secno++; - dst += sc->sectorsize; + if (!notmapped) + dst += sc->sectorsize; } bp->bio_resid = 0; return (error); @@ -628,11 +778,12 @@ mdstart_vnode(struct md_s *sc, struct bio *bp) static int mdstart_swap(struct md_s *sc, struct bio *bp) { - struct sf_buf *sf; - int rv, offs, len, lastend; - vm_pindex_t i, lastp; vm_page_t m; u_char *p; + struct uio uio; + struct iovec iov[1]; + vm_pindex_t i, lastp; + int rv, ma_offs, offs, len, lastend, j; switch (bp->bio_cmd) { case BIO_READ: @@ -644,6 +795,16 @@ mdstart_swap(struct md_s *sc, struct bio *bp) } p = bp->bio_data; + if ((bp->bio_flags & BIO_NOTMAPPED) == 0) { + uio.uio_segflg = UIO_SYSSPACE; + uio.uio_rw = bp->bio_cmd == BIO_READ ? UIO_READ : UIO_WRITE; + uio.uio_td = curthread; + uio.uio_iov = iov; + uio.uio_iovcnt = 1; + } else { + atomic_add_int(&md_unmapped_swap, 1); + ma_offs = bp->bio_ma_offset; + } /* * offs is the offset at which to start operating on the @@ -659,21 +820,14 @@ mdstart_swap(struct md_s *sc, struct bio *bp) rv = VM_PAGER_OK; VM_OBJECT_LOCK(sc->object); vm_object_pip_add(sc->object, 1); - for (i = bp->bio_offset / PAGE_SIZE; i <= lastp; i++) { + for (i = bp->bio_offset / PAGE_SIZE, j = 0; i <= lastp; i++, j++) { len = ((i == lastp) ? lastend : PAGE_SIZE) - offs; - - m = vm_page_grab(sc->object, i, - VM_ALLOC_NORMAL|VM_ALLOC_RETRY); - VM_OBJECT_UNLOCK(sc->object); - sched_pin(); - sf = sf_buf_alloc(m, SFB_CPUPRIVATE); - VM_OBJECT_LOCK(sc->object); + m = vm_page_grab(sc->object, i, VM_ALLOC_NORMAL | + VM_ALLOC_RETRY); if (bp->bio_cmd == BIO_READ) { if (m->valid != VM_PAGE_BITS_ALL) rv = vm_pager_get_pages(sc->object, &m, 1, 0); if (rv == VM_PAGER_ERROR) { - sf_buf_free(sf); - sched_unpin(); vm_page_wakeup(m); break; } else if (rv == VM_PAGER_FAIL) { @@ -683,40 +837,52 @@ mdstart_swap(struct md_s *sc, struct bio *bp) * valid. Do not set dirty, the page * can be recreated if thrown out. */ - bzero((void *)sf_buf_kva(sf), PAGE_SIZE); + pmap_zero_page(m); m->valid = VM_PAGE_BITS_ALL; } - bcopy((void *)(sf_buf_kva(sf) + offs), p, len); - cpu_flush_dcache(p, len); + if ((bp->bio_flags & BIO_NOTMAPPED) != 0) { + pmap_copy_pages(&m, offs, &bp->bio_ma[j], + ma_offs, len); + } else { + uio.uio_resid = len; + uio.uio_offset = offs; + iov[0].iov_base = p; + iov[0].iov_len = len; + uiomove_fromphys(&m, offs, len, &uio); + cpu_flush_dcache(p, len); + } } else if (bp->bio_cmd == BIO_WRITE) { if (len != PAGE_SIZE && m->valid != VM_PAGE_BITS_ALL) rv = vm_pager_get_pages(sc->object, &m, 1, 0); if (rv == VM_PAGER_ERROR) { - sf_buf_free(sf); - sched_unpin(); vm_page_wakeup(m); break; } - bcopy(p, (void *)(sf_buf_kva(sf) + offs), len); + if ((bp->bio_flags & BIO_NOTMAPPED) != 0) { + pmap_copy_pages(&bp->bio_ma[j], ma_offs, &m, + offs, len); + } else { + uio.uio_resid = len; + uio.uio_offset = offs; + iov[0].iov_base = p; + iov[0].iov_len = len; + uiomove_fromphys(&m, offs, len, &uio); + } m->valid = VM_PAGE_BITS_ALL; } else if (bp->bio_cmd == BIO_DELETE) { if (len != PAGE_SIZE && m->valid != VM_PAGE_BITS_ALL) rv = vm_pager_get_pages(sc->object, &m, 1, 0); if (rv == VM_PAGER_ERROR) { - sf_buf_free(sf); - sched_unpin(); vm_page_wakeup(m); break; } if (len != PAGE_SIZE) { - bzero((void *)(sf_buf_kva(sf) + offs), len); + pmap_zero_page_area(m, offs, len); vm_page_clear_dirty(m, offs, len); m->valid = VM_PAGE_BITS_ALL; } else vm_pager_page_unswapped(m); } - sf_buf_free(sf); - sched_unpin(); vm_page_wakeup(m); vm_page_lock(m); if (bp->bio_cmd == BIO_DELETE && len == PAGE_SIZE) @@ -730,6 +896,7 @@ mdstart_swap(struct md_s *sc, struct bio *bp) /* Actions on further pages start at offset 0 */ p += PAGE_SIZE - offs; offs = 0; + ma_offs = 0; } vm_object_pip_subtract(sc->object, 1); VM_OBJECT_UNLOCK(sc->object); @@ -845,6 +1012,14 @@ mdinit(struct md_s *sc) pp = g_new_providerf(gp, "md%d", sc->unit); pp->mediasize = sc->mediasize; pp->sectorsize = sc->sectorsize; + switch (sc->type) { + case MD_SWAP: + case MD_MALLOC: + pp->flags |= G_PF_ACCEPT_UNMAPPED; + break; + default: + break; + } sc->gp = gp; sc->pp = pp; g_error_provider(pp, 0); Index: /usr/src/sys/geom/geom_io.c =================================================================== --- /usr/src/sys/geom/geom_io.c (revision 244437) +++ /usr/src/sys/geom/geom_io.c (working copy) @@ -51,6 +51,13 @@ __FBSDID("$FreeBSD$"); #include #include +#include +#include +#include +#include +#include +#include +#include static struct g_bioq g_bio_run_down; static struct g_bioq g_bio_run_up; @@ -182,10 +189,12 @@ g_clone_bio(struct bio *bp) * ordering restrictions, so this flag needs to be cloned. * Other bio flags are not suitable for cloning. */ - bp2->bio_flags = bp->bio_flags & BIO_ORDERED; + bp2->bio_flags = bp->bio_flags & (BIO_ORDERED | BIO_NOTMAPPED); bp2->bio_length = bp->bio_length; bp2->bio_offset = bp->bio_offset; bp2->bio_data = bp->bio_data; + bp2->bio_ma = bp->bio_ma; + bp2->bio_ma_offset = bp->bio_ma_offset; bp2->bio_attribute = bp->bio_attribute; /* Inherit classification info from the parent */ bp2->bio_classifier1 = bp->bio_classifier1; @@ -210,11 +219,14 @@ g_duplicate_bio(struct bio *bp) struct bio *bp2; bp2 = uma_zalloc(biozone, M_WAITOK | M_ZERO); + bp2->bio_flags = bp->bio_flags & BIO_NOTMAPPED; bp2->bio_parent = bp; bp2->bio_cmd = bp->bio_cmd; bp2->bio_length = bp->bio_length; bp2->bio_offset = bp->bio_offset; bp2->bio_data = bp->bio_data; + bp2->bio_ma = bp->bio_ma; + bp2->bio_ma_offset = bp->bio_ma_offset; bp2->bio_attribute = bp->bio_attribute; bp->bio_children++; #ifdef KTR @@ -577,7 +589,9 @@ g_io_schedule_down(struct thread *tp __unused) { struct bio *bp; off_t excess; - int error; + vm_offset_t addr; + long size; + int error, rv; for(;;) { g_bioq_lock(&g_bio_run_down); @@ -633,6 +647,32 @@ g_io_schedule_down(struct thread *tp __unused) default: break; } + if ((bp->bio_flags & BIO_NOTMAPPED) != 0 && + (bp->bio_to->flags & G_PF_ACCEPT_UNMAPPED) == 0 && + (bp->bio_cmd == BIO_READ || bp->bio_cmd == BIO_WRITE)) { + size = round_page(bp->bio_ma_offset + bp->bio_length); + addr = 0; + vm_map_lock(bio_transient_map); + if (vm_map_findspace(bio_transient_map, + vm_map_min(bio_transient_map), size, &addr)) { + vm_map_unlock(bio_transient_map); + g_io_deliver(bp, EDEADLK/* XXXKIB */); + CTR2(KTR_GEOM, "g_down cannot map " + "bp %p provider %s", bp, bp->bio_to->name); + continue; + } + rv = vm_map_insert(bio_transient_map, NULL, 0, addr, + addr + size, VM_PROT_RW, VM_PROT_RW, MAP_NOFAULT); + KASSERT(rv == KERN_SUCCESS, + ("vm_map_insert(bio_transient_map) rv %d %jx %lx", + rv, (uintmax_t)addr, size)); + vm_map_unlock(bio_transient_map); + pmap_qenter((vm_offset_t)addr, bp->bio_ma, + OFF_TO_IDX(size)); + bp->bio_data = (caddr_t)addr + bp->bio_ma_offset; + bp->bio_flags |= BIO_TRANSIENT_MAPPING; + bp->bio_flags &= ~BIO_NOTMAPPED; + } THREAD_NO_SLEEPING(); CTR4(KTR_GEOM, "g_down starting bp %p provider %s off %ld " "len %ld", bp, bp->bio_to->name, bp->bio_offset, Index: /usr/src/sys/geom/part/g_part.c =================================================================== --- /usr/src/sys/geom/part/g_part.c (revision 244437) +++ /usr/src/sys/geom/part/g_part.c (working copy) @@ -427,6 +427,7 @@ g_part_new_provider(struct g_geom *gp, struct g_pa entry->gpe_pp->stripeoffset = pp->stripeoffset + entry->gpe_offset; if (pp->stripesize > 0) entry->gpe_pp->stripeoffset %= pp->stripesize; + entry->gpe_pp->flags |= pp->flags & G_PF_ACCEPT_UNMAPPED; g_error_provider(entry->gpe_pp, 0); } Index: /usr/src/sys/geom/geom.h =================================================================== --- /usr/src/sys/geom/geom.h (revision 244437) +++ /usr/src/sys/geom/geom.h (working copy) @@ -205,6 +205,7 @@ struct g_provider { u_int flags; #define G_PF_WITHER 0x2 #define G_PF_ORPHAN 0x4 +#define G_PF_ACCEPT_UNMAPPED 0x8 /* Two fields for the implementing class to use */ void *private; Index: /usr/src/sys/geom/geom_vfs.c =================================================================== --- /usr/src/sys/geom/geom_vfs.c (revision 244437) +++ /usr/src/sys/geom/geom_vfs.c (working copy) @@ -188,7 +188,15 @@ g_vfs_strategy(struct bufobj *bo, struct buf *bp) bip = g_alloc_bio(); bip->bio_cmd = bp->b_iocmd; bip->bio_offset = bp->b_iooffset; - bip->bio_data = bp->b_data; + if ((bp->b_flags & B_NOTMAPPED) != 0) { + bip->bio_ma = bp->b_pages; + bip->bio_data = unmapped_buf; + bip->bio_ma_offset = (vm_offset_t)bp->b_offset & PAGE_MASK; + bip->bio_flags |= BIO_NOTMAPPED; + } else { + bip->bio_data = bp->b_data; + bip->bio_ma = NULL; + } bip->bio_done = g_vfs_done; bip->bio_caller2 = bp; bip->bio_length = bp->b_bcount; Index: /usr/src/sys/geom/geom_disk.c =================================================================== --- /usr/src/sys/geom/geom_disk.c (revision 244437) +++ /usr/src/sys/geom/geom_disk.c (working copy) @@ -488,6 +488,8 @@ g_disk_create(void *arg, int flag) pp->sectorsize = dp->d_sectorsize; pp->stripeoffset = dp->d_stripeoffset; pp->stripesize = dp->d_stripesize; + if ((dp->d_flags & DISKFLAG_NOTMAPPED_BIO) != 0) + pp->flags |= G_PF_ACCEPT_UNMAPPED; if (bootverbose) printf("GEOM: new disk %s\n", gp->name); sysctl_ctx_init(&sc->sysctl_ctx); Index: /usr/src/sys/geom/geom_disk.h =================================================================== --- /usr/src/sys/geom/geom_disk.h (revision 244437) +++ /usr/src/sys/geom/geom_disk.h (working copy) @@ -103,6 +103,7 @@ struct disk { #define DISKFLAG_OPEN 0x2 #define DISKFLAG_CANDELETE 0x4 #define DISKFLAG_CANFLUSHCACHE 0x8 +#define DISKFLAG_NOTMAPPED_BIO 0x10 struct disk *disk_alloc(void); void disk_create(struct disk *disk, int version);