Thread transitions and scheduler locking ======================================== Switching away from a thread ---------------------------- :c:func:`mi_switch` in ``kern_synch.c`` performs the machine-independent parts of a context switch. Its entry contract requires the thread lock; the normal return path no longer holds that lock. It records voluntary or involuntary switching and runtime accounting, then calls the selected ``sched_switch()`` implementation. The implementation decides what to do with the outgoing thread, chooses the next thread, and enters the machine-dependent switch machinery. The reason and flags passed to ``mi_switch()`` distinguish operations such as voluntary blocking from preemption. A caller must establish a valid thread state before entering that path. Waking and becoming runnable ---------------------------- Sleep queues and turnstiles manage different kinds of waiting. Their wakeup paths eventually coordinate with scheduler operations to make a thread eligible for execution. Wakeup does not guarantee that the awakened thread runs immediately or that it will resume on its previous CPU. :c:func:`setrunnable` checks the thread's state and calls ``sched_wakeup()`` when it is ready to transition from ``TDS_CAN_RUN``. Its normal behavior releases the thread lock, with ``SRQ_HOLD`` and ``SRQ_HOLDTD`` selecting different lock-retention behavior. A thread still inhibited by another condition is not made runnable merely by clearing one reason to wait. The implementation then applies placement, priority, and preemption policy. For a remote CPU, notification and a later scheduling point can separate enqueueing from actual execution. The thread lock can change -------------------------- The lock protecting scheduler-related thread state is reached through the thread's lock association. Scheduler operations and migration can transfer that association between locks. Use ``thread_lock()`` and the established scheduler helpers rather than assuming a thread permanently belongs to one particular run-queue lock. ULE's queue locks protect much of each ``tdq``. 4BSD has its own scheduler locking protocol. Their ``td_sched`` structures also have different field layouts and protection rules. Code shared between implementations must respect the interface instead of inspecting an implementation's private extension as though it were universal thread state. Preemption, pinning, and affinity --------------------------------- A critical section and a thread's pin count serve different purposes. Preventing an immediate scheduling action is distinct from restricting CPU migration. Likewise, an allowed cpuset is a set of eligible CPUs, while binding selects a particular CPU through the scheduler's binding interface. Use the interfaces for the actual invariant needed by the caller. Do not substitute a priority boost for a lock or treat affinity as protection for data that another CPU can still access. Reviewing a change ------------------ For each transition, identify the incoming state, held lock, outgoing state, and outgoing lock ownership. Check the wakeup, timeout, signal, and error paths as well as the successful wait path. Priority lending through a turnstile adds another state transition that must be reconciled when a lock owner stops needing the borrowed priority. The source assertions and WITNESS/INVARIANTS configurations help detect violations. Useful tracing distinguishes enqueue, dequeue, priority change, off-CPU, and on-CPU events; observing only a switch can hide the earlier event that caused the scheduling decision.