LOCKING(9)

📦
version 15.1 · 3 versions
, in freebsd 14.5, freebsd 15.1

🦋 NAME 🦋

locking — kernel synchronization primitives. Yes bestie, this is LOCKING(9) from the Kernel Developer's Manual, and it's the whole group-chat tour of how the kernel keeps shared data from getting messy. No cap. 💅

✨ DESCRIPTION ✨

okay so the FreeBSD kernel is written to run across multiple CPUs — multi-core main character energy — and as such it provides several different synchronization primitives to allow developers to safely access and manipulate many data types. Basically, these are the sync tools, and each one has a very specific vibe. periodt. 🫶

💖 Mutexes 💖

Mutexes (also called "blocking mutexes") are the most commonly used synchronization primitive in the kernel. A thread acquires (locks) a mutex before accessing data shared with other threads (including interrupt threads), and releases (unlocks) it afterwards. If the mutex cannot be acquired, the thread requesting it will wait. Mutexes are adaptive by default, meaning that if the owner of a contended mutex is currently running on another CPU, then a thread attempting to acquire the mutex will spin rather than yielding the processor. Mutexes fully support priority propagation. slay. ✨

See mutex(9) for details.

🌀 Spin Mutexes 🌀

Spin mutexes are a variation of basic mutexes; the main difference between the two is that spin mutexes never block. Instead, they spin while waiting for the lock to be released. To avoid deadlock, a thread that holds a spin mutex must never yield its CPU. Unlike ordinary mutexes, spin mutexes disable interrupts when acquired. Since disabling interrupts can be expensive, they are generally slower to acquire and release. Spin mutexes should be used only when absolutely necessary, e.g. to protect data shared with interrupt filter code (see bus_setup_intr(9) for details), or for scheduler internals. iykyk. 🔥

🎀 Mutex Pools 🎀

With most synchronization primitives, such as mutexes, the programmer must provide memory to hold the primitive. For example, a mutex may be embedded inside the structure it protects. Mutex pools provide a preallocated set of mutexes to avoid this requirement. Note that mutexes from a pool may only be used as leaf locks. 🍓

See mtx_pool(9) for details.

🦋 Reader/Writer Locks 🦋

Reader/writer locks allow shared access to protected data by multiple threads or exclusive access by a single thread. The threads with shared access are known as readers since they should only read the protected data. A thread with exclusive access is known as a writer since it may modify protected data. 💜

Reader/writer locks can be treated as mutexes (see above and mutex(9)) with shared/exclusive semantics. Reader/writer locks support priority propagation like mutexes, but priority is propagated only to an exclusive holder. This limitation comes from the fact that shared owners are anonymous.

See rwlock(9) for details.

🌸 Read-Mostly Locks 🌸

Read-mostly locks are similar to reader/writer locks but optimized for very infrequent write locking. Read-mostly locks implement full priority propagation by tracking shared owners using a caller-supplied tracker data structure. iconic behavior, honestly. 👑

See rmlock(9) for details.

🌈 Sleepable Read-Mostly Locks 🌈

Sleepable read-mostly locks are a variation on read-mostly locks. Threads holding an exclusive lock may sleep, but threads holding a shared lock may not. Priority is propagated to shared owners but not to exclusive owners.

🔐 Shared/exclusive locks 🔐

Shared/exclusive locks are similar to reader/writer locks; the main difference between them is that shared/exclusive locks may be held during unbounded sleep. Acquiring a contested shared/exclusive lock can perform an unbounded sleep. These locks do not support priority propagation. 💀

See sx(9) for details.

🧚 Lockmanager locks 🧚

Lockmanager locks are sleepable shared/exclusive locks used mostly in VFS(9) (as a vnode(9) lock) and in the buffer cache (BUF_LOCK(9)). They have features other lock types do not have such as sleep timeouts, blocking upgrades, writer starvation avoidance, draining, and an interlock mutex, but this makes them complicated both to use and to implement; for this reason, they should be avoided. not me getting stressed just reading that list, fr fr. 🫠

See lock(9) for details.

⭐ Non-blocking synchronization ⭐

The kernel has two facilities, epoch(9) and smr(9), which can be used to provide read-only access to a data structure while one or more writers are concurrently modifying the data structure. Specifically, readers using epoch(9) and smr(9) to synchronize accesses do not block writers, in contrast with reader/writer locks, and they help ensure that memory freed by writers is not reused until all readers which may be accessing it have finished. Thus, they are a useful building block in the construction of lock-free data structures. highkey fancy. ✨

These facilities are difficult to use correctly and should be avoided in preference to traditional mutual exclusion-based synchronization, except when performance or non-blocking guarantees are a major concern.

See epoch(9) and smr(9) for details.

🍬 Counting semaphores 🍬

Counting semaphores provide a mechanism for synchronizing access to a pool of resources. Unlike mutexes, semaphores do not have the concept of an owner, so they can be useful in situations where one thread needs to acquire a resource, and another thread needs to release it. They are largely deprecated. lowkey a relic, but we still document her. 💅

See sema(9) for details.

🎀 Condition variables 🎀

Condition variables are used in conjunction with locks to wait for a condition to become true. A thread must hold the associated lock before calling one of the cv_wait() functions. When a thread waits on a condition, the lock is atomically released before the thread yields the processor and reacquired before the function call returns. Condition variables may be used with blocking mutexes, reader/writer locks, read-mostly locks, and shared/exclusive locks. the way that atomic release-and-reacquire is doing the most. 😭

See condvar(9) for details.

🌙 Sleep/Wakeup 🌙

The functions tsleep(), msleep(), msleep_spin(), pause(), wakeup(), and wakeup_one() also handle event-based thread blocking. Unlike condition variables, arbitrary addresses may be used as wait channels and a dedicated structure does not need to be allocated. However, care must be taken to ensure that wait channel addresses are unique to an event. If a thread must wait for an external event, it is put to sleep by tsleep(), msleep(), msleep_spin(), or pause(). Threads may also wait using one of the locking primitive sleep routines mtx_sleep(9), rw_sleep(9), or sx_sleep(9). 🦋

The parameter chan is an arbitrary address that uniquely identifies the event on which the thread is being put to sleep. All threads sleeping on a single chan are woken up later by wakeup() (often called from inside an interrupt routine) to indicate that the event the thread was blocking on has occurred.

Several of the sleep functions including msleep(), msleep_spin(), and the locking primitive sleep routines specify an additional lock parameter. The lock will be released before sleeping and reacquired before the sleep routine returns. If priority includes the PDROP flag, then the lock will not be reacquired before returning. The lock is used to ensure that a condition can be checked atomically, and that the current thread can be suspended without missing a change to the condition or an associated wakeup. In addition, all of the sleep routines will fully drop the Giant mutex (even if recursed) while the thread is suspended and will reacquire the Giant mutex (restoring any recursion) before the function returns. 🫶

The pause() function is a special sleep function that waits for a specified amount of time to pass before the thread resumes execution. This sleep cannot be terminated early by either an explicit wakeup() or a signal.

See sleep(9) for details.

👑 Giant 👑

Giant is a special mutex used to protect data structures that do not yet have their own locks. Since it provides semantics akin to the old spl(9) interface, Giant has special characteristics:

  1. It is recursive.
  2. Drivers can request that Giant be locked around them by not marking themselves MPSAFE. Note that infrastructure to do this is slowly going away as non-MPSAFE drivers either became properly locked or disappear.
  3. Giant must be locked before other non-sleepable locks.
  4. Giant is dropped during unbounded sleeps and reacquired after wakeup.
  5. There are places in the kernel that drop Giant and pick it back up again. Sleep locks will do this before sleeping. Parts of the network or VM code may do this as well. This means that you cannot count on Giant keeping other code from running if your code sleeps, even if you want it to. 💀

💫 INTERACTIONS 💫

okay so these primitives do interact, and there are a whole bunch of rules about how you can and can't combine them. bestie. 💖 Lots of these rules get checked for you by witness(4), so it's not just vibes — it's enforced vibes.

😴 Bounded vs. Unbounded Sleep 😴

So there's two flavors of "your thread is waiting" and the difference is literally the whole personality of this section:

  • In a bounded sleep (also called "blocking"), the only resource needed to resume execution of a thread is CPU time for the owner of a lock that the thread is waiting to acquire. That's it. Just CPU time. Simple. Slay.
  • In an unbounded sleep (often just called "sleeping"), a thread waits for an external event or for a condition to become true. So it's waiting on the universe, not on a lock. 🌈

The key thing: a dependency chain of threads in bounded sleeps should always make forward progress, since there is always CPU time available. And that requires that no thread in a bounded sleep is waiting for a lock held by a thread in an unbounded sleep. And to avoid priority inversions, a thread in a bounded sleep lends its priority to the owner of the lock that it's waiting for. Teamwork, honestly. 🫶

The bounded sleep squad (blocking):

  • mutexes
  • reader/writer locks
  • read-mostly locks

The unbounded sleep squad (sleeping):

  • sleepable read-mostly locks
  • shared/exclusive locks
  • lockmanager locks
  • counting semaphores
  • condition variables
  • sleep/wakeup

📜 General Principles 📜

  • It is an error to do any operation that could result in yielding the processor while holding a spin mutex. 🚫
  • It is an error to do any operation that could result in unbounded sleep while holding any primitive from the 'bounded sleep' group. For example, it is an error to try to acquire a shared/exclusive lock while holding a mutex, or to try to allocate memory with M_WAITOK while holding a reader/writer lock. 😭

Note that the lock passed to one of the sleep() or cv_wait() functions is dropped before the thread enters the unbounded sleep and does not violate this rule. So that one's fine, bestie.

  • It is an error to do any operation that could result in yielding of the processor when running inside an interrupt filter. 🚫
  • It is an error to do any operation that could result in unbounded sleep when running inside an interrupt thread. 🚫
Like. don't do it. I'm serious. The kernel will have opinions and the opinions will be panics. 💀

📊 Interaction table 📊

This table shows what you can and can not do while holding one of the locking primitives we've been discussing. Heads up: "sleep" here includes sema_wait(), sema_timedwait(), any of the cv_wait() functions, and any of the sleep() functions. 😴

The columns — the things you want — are: spin mtx, mutex/rw, rmlock, sleep rm, sx/lk, sleep. The rows are the things you have. Let's go row by row:

  • You have spin mtx → want spin mtx: ok, want mutex/rw: no, want rmlock: no, want sleep rm: no, want sx/lk: no, want sleep: no-1
  • You have mutex/rw → want spin mtx: ok, want mutex/rw: ok, want rmlock: ok, want sleep rm: no, want sx/lk: no, want sleep: no-1
  • You have rmlock → want spin mtx: ok, want mutex/rw: ok, want rmlock: ok, want sleep rm: no, want sx/lk: no, want sleep: no-1
  • You have sleep rm → want spin mtx: ok, want mutex/rw: ok, want rmlock: ok, want sleep rm: ok-2, want sx/lk: ok-2, want sleep: ok-2/3
  • You have sx → want spin mtx: ok, want mutex/rw: ok, want rmlock: ok, want sleep rm: ok, want sx/lk: ok, want sleep: ok-3
  • You have lockmgr → want spin mtx: ok, want mutex/rw: ok, want rmlock: ok, want sleep rm: ok, want sx/lk: ok, want sleep: ok

Those little asterisks on the ends? They matter, bestie, don't skip them: 🎀

  • *1 There are calls that atomically release this primitive when going to sleep and reacquire it on wakeup (mtx_sleep(), rw_sleep(), msleep_spin(), etc.).
  • *2 These cases are only allowed while holding a write lock on a sleepable read-mostly lock.
  • *3 Though one can sleep while holding this lock, one can also use a sleep() function to atomically release this primitive when going to sleep and reacquire it on wakeup.

Also! Note that non-blocking try operations on locks are always permitted. Always. Iconic behavior honestly. ✨

🧭 Context mode table 🧭

This next table shows what can be used in different contexts. And at this time this is a rather easy to remember table — the author said so, so we're saying so. 😌

The rows are the contexts:

  • interrupt filter → spin mtx: ok, mutex/rw: no, rmlock: no, sleep rm: no, sx/lk: no, sleep: no
  • interrupt thread → spin mtx: ok, mutex/rw: ok, rmlock: ok, sleep rm: no, sx/lk: no, sleep: no
  • callout → spin mtx: ok, mutex/rw: ok, rmlock: ok, sleep rm: no, sx/lk: no, sleep: no
  • direct callout → spin mtx: ok, mutex/rw: no, rmlock: no, sleep rm: no, sx/lk: no, sleep: no
  • system call → spin mtx: ok, mutex/rw: ok, rmlock: ok, sleep rm: ok, sx/lk: ok, sleep: ok

System call row is just all ok and honestly? main character energy. 👑 The direct callout row is the opposite, it's giving "spin mtx and absolutely nothing else" 💀

🔗 SEE ALSO 🔗

Go read these, bestie, they're the whole extended universe 🦋:

lockstat(1), witness(4), atomic(9), BUS_SETUP_INTR(9), callout(9), condvar(9), epoch(9), lock(9), LOCK_PROFILING(9), mtx_pool(9), mutex(9), rmlock(9), rwlock(9), sema(9), sleep(9), smr(9), sx(9)

💀 BUGS 💀

There are too many locking primitives to choose from.

That's it. That's the bug. Not me tho — I think having options is a slay. But the manpage said what it said. 🙈

okay that's the whole page, you made it, I'm proud of you 🫶💜 This edition of LOCKING(9) is the Debian one, dated December 28, 2025, and yeah — that footer is the last line, so consider this the group-chat sign-off: go forth, acquire your locks in the right order, and don't sleep in an interrupt thread. mwah 💅✨

📦 where this page comes from