This file compares a new hot-patching SDT implementation with FreeBSD's current implementation, and with a FreeBSD kernel that has KDTRACE_HOOKS omitted from its configuration. redis-benchmark is used with its default options to time various Redis requests. These are transmitted over TCP loopback, providing an opportunity to measure the performance loss caused by SDT probes in the network and locking code. The benchmarks are performed using a default CPU set of {0}; this means that all benchmark threads and the netisr threads are pinned to a single CPU. This helps isolate kernel latency that might otherwise be obscured by parallelism in the Redis client and server. SMT is disabled. The CPU is a Xeon E5-2630v3 (Haswell-EP). Each pair of graphs correspond to a Redis request type. In each graph, data points from the new SDT implementation are marked as "+". For each request type, the first graph compares new SDT with a KDTRACE_HOOKS-disabled kernel, and the second compares new SDT with the current SDT. Data points represent the number of requests per second executed while executing 100000 requests of the same type, so larger numbers are better. During trials, I see up to 350000pps on the loopback and several million lock acquisitions per second. --------------------------------------------- GET --------------------------------------------- x GET.nosdt.tmp + GET.newsdt.tmp +--------------------------------------------------------------------------+ | + x | | + x | | + x | | + x | | + x | | ++ x | | ++ x | | ++ x | | ++ xx | | ++ xx | | ++ xx | | ++ xx | | + x x x ++ xx | |+ x+ ++x x+ x x + ++ xx x | | |____|______________A___A_____M____M____|__|| +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 30 33134.53 48614.49 48053.82 45221.502 5071.7795 + 30 31055.9 47125.36 46981.445 44315.204 5304.2081 No difference proven at 95.0% confidence x GET.oldsdt.tmp + GET.newsdt.tmp +--------------------------------------------------------------------------+ | + | | + | | + | | + | | + | | + | | + | | x + | | x + | | x + | | xx + | | xx + | | xx + | | xx ++ | | xx ++ | | xxx++ | | x + x xxx++ | |+ x* *+ x + x + xxxx++ | | ||_________________A_A______M___M_____|___|| +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 30 33036.01 46554.93 46125.46 43807.462 4718.27 + 30 31055.9 47125.36 46981.445 44315.204 5304.2081 No difference proven at 95.0% confidence --------------------------------------------- INCR --------------------------------------------- x INCR.nosdt.tmp + INCR.newsdt.tmp +--------------------------------------------------------------------------+ | + | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + xx | | + xx | | + + xx | | + + x x x ++ xx | |+ + + x+ x + *+ xxx + x +++ xx | | |______________|_______A________A______M___M__|__|| +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 30 35149.38 48332.53 48030.74 44957.816 4646.4582 + 30 29797.38 47036.69 46904.32 42552.523 6201.7935 No difference proven at 95.0% confidence x INCR.oldsdt.tmp + INCR.newsdt.tmp +--------------------------------------------------------------------------+ | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x ++ | | + xx ++ | | x + x xx ++ | |x + ++ xx + x + + xx*+ x * xx*++ | | |_|___________________AA___________M__M___|__|| +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 30 28571.43 46425.25 45966.445 42398.716 5433.2123 + 30 29797.38 47036.69 46904.32 42552.523 6201.7935 No difference proven at 95.0% confidence --------------------------------------------- LPOP --------------------------------------------- x LPOP.nosdt.tmp + LPOP.newsdt.tmp +--------------------------------------------------------------------------+ | +x*x | | + x x +x*x*+ | |++ x+ x +x +xxxx + x + ++ +**x*+++ ++ + x x ++ ++ x| | ||________________A__A____M________|_____| | +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 30 34566.2 56818.18 47721.315 45339.256 5517.735 + 30 33433.64 56274.62 47881.39 46228.923 6550.3577 No difference proven at 95.0% confidence x LPOP.oldsdt.tmp + LPOP.newsdt.tmp +--------------------------------------------------------------------------+ | x+ + | | x+x + | | + x x*x++ + + | |x x x x++x+ x+x x+ x+ x* x*+x****x*x+x + *+ ++| | |______|_________A_____A_M_M______|____| | +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 30 26602.82 54141.85 46893.325 43603.52 6946.2194 + 30 33433.64 56274.62 47881.39 46228.923 6550.3577 No difference proven at 95.0% confidence --------------------------------------------- LPUSH --------------------------------------------- x LPUSH.nosdt.tmp + LPUSH.newsdt.tmp +--------------------------------------------------------------------------+ | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x x+ | | + x x+ x x | | x + x x x ++*+x ++*+ xx+ *xx x | |+xx+x x +**+x + ++*xx+ *+++ x ++*** ****x**++x*x*+ xx + +| | ||_____________AM_M___________|_| | +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 60 28760.43 57471.27 47709.93 46844.043 7320.4096 + 60 28105.68 61881.19 46772.695 46293.15 6902.931 No difference proven at 95.0% confidence x LPUSH.oldsdt.tmp + LPUSH.newsdt.tmp +--------------------------------------------------------------------------+ | + | | + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | xx + + x++ | | x * + x + xx +x* *+*+x++x + x | |x * + xxxx*+ *+x+xx*+x +xx +*+ xx*****x*+*+x****+ * x + +| | |____|________A__MAM________|__| | +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 60 23872.04 56850.48 45756.14 44200.094 7449.639 + 60 28105.68 61881.19 46772.695 46293.15 6902.931 No difference proven at 95.0% confidence --------------------------------------------- LRANGE_100 --------------------------------------------- x LRANGE_100.nosdt.tmp + LRANGE_100.newsdt.tmp +--------------------------------------------------------------------------+ | + | | ++ | | ++ x | | ++ x x | | +*+x x | | + x x +**xxxx | |x ++ x + + +x +xx x + x x + + +******xx | | |__|_________________A_A________M_M_______||| +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 30 11954.57 20529.67 19966.06 18571.901 2488.2088 + 30 12759.98 20296.33 19741.385 18330.314 2575.5694 No difference proven at 95.0% confidence x LRANGE_100.oldsdt.tmp + LRANGE_100.newsdt.tmp +--------------------------------------------------------------------------+ | x + | | xx ++ | | xx ++ | | xx*+++ | | + + x xx*+*+ | |x +x * +x x*x + x x + x +*xx***++++ | | |___|________________A____A________M__M____|____|| +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 30 12303.15 19868.87 19353.59 17739.434 2547.1378 + 30 12759.98 20296.33 19741.385 18330.314 2575.5694 No difference proven at 95.0% confidence --------------------------------------------- LRANGE_300 --------------------------------------------- x LRANGE_300.nosdt.tmp + LRANGE_300.newsdt.tmp +--------------------------------------------------------------------------+ | +x | | +x | | +x | | +x | | +x | | +x | | +x | | +x | | +x | | +x | | +x | | +x | | +x | | +x | | +x | | ++x | | ++x | | ++x | |+ + x x + * *xx + x ++*xx | | |_|_________________AA_______MM________|| +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 30 6432.94 8344.46 8318.78 7977.3947 656.96081 + 30 6009.98 8298.07 8284.66 7941.003 700.80496 No difference proven at 95.0% confidence x LRANGE_300.oldsdt.tmp + LRANGE_300.newsdt.tmp +--------------------------------------------------------------------------+ | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x++ | | x++ | | xx++ | |x + x x* x + *+ + x x**+ | | |____|______________A__A______M_M_______||| +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 30 5787.04 8233.84 8213.55 7814.8047 782.09508 + 30 6009.98 8298.07 8284.66 7941.003 700.80496 No difference proven at 95.0% confidence --------------------------------------------- LRANGE_500 --------------------------------------------- x LRANGE_500.nosdt.tmp + LRANGE_500.newsdt.tmp +--------------------------------------------------------------------------+ | +x | | +x | | +x | | +x | | +x | | +x | | +x | | +x | | +x | | +x | | +x | | +x | | +x | | +x | | +x | | +x | | +x | | + x ++x | |+++ ++x xxxx + + x x + **x | | |______|_________________A___A__________MM________||| +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 30 4536.59 5706.79 5702.555 5456.1627 451.2192 + 30 4422.63 5690.87 5684.08 5384.217 510.87281 No difference proven at 95.0% confidence x LRANGE_500.oldsdt.tmp + LRANGE_500.newsdt.tmp +--------------------------------------------------------------------------+ | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x++ | | x++ | | x++ | | x++ | | x++ | | x + x x++ | |xxxxx+++ ++ + + x +x++ | | ||______________________A___________MM_________||| +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 30 4305.33 5656.11 5650.835 5385.6873 530.39542 + 30 4422.63 5690.87 5684.08 5384.217 510.87281 No difference proven at 95.0% confidence --------------------------------------------- LRANGE_600 --------------------------------------------- x LRANGE_600.nosdt.tmp + LRANGE_600.newsdt.tmp +--------------------------------------------------------------------------+ | +x | | +x | | +x | | +x | | +* | | ++* | | + +** | | + + xx + +** | |+++++ xxxxxx + x x + + x+** | | |_____|_____________________A___A_______________M_M_____|_|| +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 30 3715.81 4463.29 4458.215 4227.569 331.69785 + 30 3630.16 4452.76 4435.87 4174.5357 355.52366 No difference proven at 95.0% confidence x LRANGE_600.oldsdt.tmp + LRANGE_600.newsdt.tmp +--------------------------------------------------------------------------+ | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x++ | | x + x++ | |xx xx +++ xx++ | |xxxxx ++++ + + ++ x x*++ | | |_____|______________________A__A_________________MM______||| +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 30 3551.51 4431.05 4428.11 4143.0057 406.20603 + 30 3630.16 4452.76 4435.87 4174.5357 355.52366 No difference proven at 95.0% confidence --------------------------------------------- MSET --------------------------------------------- x MSET.nosdt.tmp + MSET.newsdt.tmp +--------------------------------------------------------------------------+ | + | | + | | + | | + | | + x | | + x | | + x | | + x | | + x | | + x | | + xx | | + xx | | ++ xx | | ++ xx | | ++ xx | | ++ xx | | ++ xx | | ++ xx | | ++ xx | |x + x + x x ++ xx | | |_|_________A_MA___M__|______|| +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 30 21272.07 36968.58 36805.3 35670.772 3519.7955 + 30 25000 35752.59 35587.19 34952.433 2490.19 No difference proven at 95.0% confidence x MSET.oldsdt.tmp + MSET.newsdt.tmp +--------------------------------------------------------------------------+ | + | | + | | + | | + | | + | | + | | + | | + | | + | | + | | x + | | x + | | x + | | x + | | x + | | x + | | xx + | | xx + | | xx + | | xx + | | xx + | | xx + | | xx + | | xx + | | xx + | | xx + | |x x + + xx++ | | |_________|___A___A_M_____|_|| +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 30 14634.86 34952.81 34843.21 33646.664 4583.5533 + 30 25000 35752.59 35587.19 34952.433 2490.19 No difference proven at 95.0% confidence --------------------------------------------- PING_BULK --------------------------------------------- x PING_BULK.nosdt.tmp + PING_BULK.newsdt.tmp +--------------------------------------------------------------------------+ | + | | + | | + | | + | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | x + x | |+ x +x * * x+ x+ xx x | | |_______________A_A___M___M_____|__|| +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 30 33692.72 49333.99 48804.29 46418.341 5089.1712 + 30 30075.19 47824 47675.81 45837.756 4652.3939 No difference proven at 95.0% confidence x PING_BULK.oldsdt.tmp + PING_BULK.newsdt.tmp +--------------------------------------------------------------------------+ | + | | + | | + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | xx + | | xx ++ | |x* x x + + + x + xxxx ++ | | |______|___________A____A_M___M______|__|| +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 30 29770.77 47080.98 46544.1 44422.541 5272.2143 + 30 30075.19 47824 47675.81 45837.756 4652.3939 No difference proven at 95.0% confidence --------------------------------------------- PING_INLINE --------------------------------------------- x PING_INLINE.nosdt.tmp + PING_INLINE.newsdt.tmp +--------------------------------------------------------------------------+ | + | | + | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | x + x | |+x + + x*x + x +xx | | ||____________AA___M_M______||| +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 30 8927.77 48262.55 48100.05 44656.796 8598.4798 + 30 8054.12 47236.65 47058.82 43979.29 8522.1425 No difference proven at 95.0% confidence x PING_INLINE.oldsdt.tmp + PING_INLINE.newsdt.tmp +--------------------------------------------------------------------------+ | + | | + | | + | | + | | + | | + | | + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | xx+ | | xx+ | | xx+ | | xx+ | | xx+ | |x+ + + x+x x x +x xx+ | | ||___________AA__M_M______|_|| +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 30 7333.53 46146.75 45913.68 43410.126 7673.3601 + 30 8054.12 47236.65 47058.82 43979.29 8522.1425 No difference proven at 95.0% confidence --------------------------------------------- SADD --------------------------------------------- x SADD.nosdt.tmp + SADD.newsdt.tmp +--------------------------------------------------------------------------+ | x x + | | + + + * x+ x x +* * x | |+ ++ x + + xxxx x + * ++* xxx+ x x**** + ** x + + x +| | |_________|__________A__AM_M__________|__| | +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 30 36205.65 59311.98 50697.165 49717.229 5288.2591 + 30 32372.94 60386.47 50188.205 48343.24 7822.397 No difference proven at 95.0% confidence x SADD.oldsdt.tmp + SADD.newsdt.tmp +--------------------------------------------------------------------------+ | x + | | + + xx + + x+ xx ++ + | |+xx ++ x x + + x xxx +xx+x**+ x x+xxxx++++x * +* xx + + +| | |_|_______________A__A___M________|______| | +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 30 32938.08 56689.34 48374.385 47401.047 6220.6714 + 30 32372.94 60386.47 50188.205 48343.24 7822.397 No difference proven at 95.0% confidence --------------------------------------------- SET --------------------------------------------- x SET.nosdt.tmp + SET.newsdt.tmp +--------------------------------------------------------------------------+ | + | | + | | + | | + | | + | | + | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + xx | | + xx | | + xx | | x + xx | | + x + xx | |+ + x* +x x + x + ++ xxx | | |_____|_____________A____A____M___M_____|__|| +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 30 32829.94 47687.18 47180.94 44571.185 5007.9484 + 30 29815.14 46253.47 46061.72 43384.476 5342.9601 No difference proven at 95.0% confidence x SET.oldsdt.tmp + SET.newsdt.tmp +--------------------------------------------------------------------------+ | + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | xx ++ | |+ x *x +++ x x x + x + xxx ++ | | ||________________A__A______M___M____|____|| +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 30 30478.51 45475.22 45075.5 42633.145 4880.1759 + 30 29815.14 46253.47 46061.72 43384.476 5342.9601 No difference proven at 95.0% confidence --------------------------------------------- SPOP --------------------------------------------- x SPOP.nosdt.tmp + SPOP.newsdt.tmp +--------------------------------------------------------------------------+ | + x x | | + + x* x ++ xxx | |+x+ +x + x+ x + x * +++++x**xx*+*+xx** x + +++x x x| | |______|__________A_M__AM__________|__| | +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 30 36791.76 61462.82 52246.615 51768.257 5370.6725 + 30 36589.83 58411.21 50879.09 50056.374 5813.0851 No difference proven at 95.0% confidence x SPOP.oldsdt.tmp + SPOP.newsdt.tmp +--------------------------------------------------------------------------+ | + x++ + x ++ x+ | |x * + xx x+ * x +x + x +xx*++** *+ x*++*xx+xx x+x*++x x| | |____|______________A_A_M_____________|_| | +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 30 34094.78 60168.47 50829.075 49360.126 7199.0906 + 30 36589.83 58411.21 50879.09 50056.374 5813.0851 No difference proven at 95.0% confidence