found 2026-10-17: smbd connects timing out on case (DIAGNOSTIC + KASAN)trigger: fork() of a process with many map entriesfix: a linear depth-first check; validated in a bhyve guest, 2026-10-17O(n × depth) → O(n)
With options DIAGNOSTIC, _vm_map_assert_consistent() re-checks a process's whole map once the map has been changed more times than it has entries. The old check found every entry by searching from the root, so one check cost the sum of all entry depths.
fork() triggers this specifically, and makes that sum quadratic. The child's map is built by inserting the entries in address order, which leaves a single path, and building it uses up the change budget, so the child's first page fault runs the check on the worst tree it will ever have. Samba hard-codes a minimum of 70,566 mutex hash chains by default, one map entry each, so every forked smbd made over a billion node visits before it could serve its client.
Where smbd's 70,566 map entries come from
The count comes from Samba's defaults, not from traffic. Each database's mutex area is created and initialized in full when the file is created, so an idle server has the same 70,588 mappings as a busy one. On Linux the mutex lives inside the tdb file and costs no mapping; FreeBSD's libthr gives every process-shared mutex its own page.
fork() copies the map in address order
The copy is the worst shape a search tree can have. Linking an entry splays the tree at its address; an address above every existing entry leaves the whole tree on the new entry's left. After n links the child's tree is a path n deep, and its update counter is exactly n.
The first fault: one splay, then the whole check
The trigger is a count, not an event. The full check runs once the map has had more updates than it has entries. The fork's own links bring the count to exactly n, so the child's first lookup tips it over and the check runs on the freshly copied tree with its depth only halved.
Old check: one search from the root per entry
Cost is the sum of all depths. On a balanced tree that is O(n log n). On the post-fork tree the depths grow linearly along the path, so their sum grows with n², which is why doubling the entries made the stall about four times longer.
New check: one depth-first pass with bounds
Bounds replace the root searches. A depth-first walk knows each subtree's lower and upper bound from its ancestors, which is what the old per-entry search recomputed for every entry. Each link is checked as a thread (equal to its bound) or a child (inside the bounds), and the header's first and last links against the leftmost and rightmost entries, so no separate walk over the list is needed.
Measured: fork() with N map entries
Quadratic against nearly flat. The stock check curves upward with the square of the entries: 3.8 s at 70k, 15.4 s at 140k. The fixed check stays near the axis, 0.13 s at 140k, and part of that is the fork and exit themselves. Example from our test system: an 8-vCPU bhyve guest running GENERIC + options DIAGNOSTIC, both kernels built from the same tree except for the fix; the parent holds one mapping with alternating page protections, and the child writes one page and exits. Both kernels were measured at the same 15 sizes.