people.freebsd.org / ~nprice / diagrams
Design and review drawings for FreeBSD networking and storage work. The design proposals below are prototyped and measured on a two-socket Xeon E5-2470 v2 (Ivy Bridge, QPI) and a Ryzen 9 5950X (two CCDs); the numbers appear on each page.
SO_REUSEPORT_LB receive-CPU affinity
End-to-End Receive Localityoverview
One flow from wire to worker, the measured cost ladder, and how the three commits connect RSS steering to the accepting worker. Start here.
Receive-Queue Spreadcommit 1
iflib optionally spreads a NIC's queue CPUs across cache groups, so the deal has a local worker to hand out on each.
Receive-CPU Dealingcommit 2
Each receiving CPU deals its share of connections to the members nearest it; every member is owed an equal share, so no worker starves.
The Ring Modelcommit 2 detail
Where "nearest" comes from — the scheduler's own group chain — and a deal traced ring by ring.
per-NIC RSS
PCI passthrough
how existing mechanisms work
iflib & RSS: From Packet to Processorprimer
How iflib and RSS steer inbound traffic by default, and how the socket-layer affinity work connects that steering to the application.
Magicsock Path States
Tailscale's path selection between direct and relayed connections.
The Flow-Steering Landscape
Where RSS and hardware flow steering can and cannot be programmed across the FreeBSD stack - the read-only, dead, and vendor-private pieces, and the hole in the middle.
Memory Ordering: x86-64 & arm64
How x86-64 and arm64 order memory, how Linux and FreeBSD ask for order on each (with the per-call SPL mapping), and where the ZIO batch race fits. Companion to the batch page.
umtx Chain-Hash Distribution
How umtxq_hash spreads wait words across chains, why a sparse multiplier collapses at power-of-two strides, and the fair-constant swap that fixes it.
a defect, and how it was fixed
ZFS Error Log Routing
How ZFS files, keeps, and reads a permanent error, and the routing bug that made scrub entries permanent.
ZIO Batch Arrival Race
An arm64 txg wedge: a ZIO batch publish/release with no barrier pairing on relaxed SPL atomics.
superseded designs, kept for the record