FENCE SIGNALLING ANNOTATION(9)

💅 fence signalling annotation(9) 💅

okay so bestie, welcome to the Kernel API Manual page for fence signalling annotation(9) — this is the one about proving correctness of all the kernel code around dma_fence through code review and testing, and ngl it's tricky for a few reasons:

• It's a cross-driver contract 🫶 — and therefore all drivers must follow the same rules for lock nesting order, calling contexts for various functions, and anything else significant for in-kernel interfaces. But it is also impossible to test all drivers in a single machine, hence brute-force N vs. N testing of all combinations is impossible. Even just limiting to the possible combinations is infeasible.
• There is an enormous amount of driver code involved 😭. For render drivers there's the tail of command submission, after fences are published, scheduler code, interrupt and workers to process job completion, and timeout, gpu reset and gpu hang recovery code. Plus for integration with core mm with have mmu_notifier, respectively mmu_interval_notifier, and shrinker. For modesetting drivers there's the commit tail functions between when fences for an atomic modeset are published, and when the corresponding vblank completes, including any interrupt processing and related workers. Auditing all that code, across all drivers, is not feasible.
• The wiggle-room is extremely thin 💀. Due to how many other subsystems are involved and the locking hierarchies this pulls in there is extremely thin wiggle-room for driver-specific differences. dma_fence interacts with almost all of the core memory handling through page fault handlers via dma_resv, dma_resv_lock and dma_resv_unlock. On the other side it also interacts through all allocation sites through mmu_notifier and shrinker.

Furthermore, lockdep does not handle cross-release dependencies, which means any deadlocks between dma_fence_wait and dma_fence_signal can't be caught at runtime with some quick testing. The simplest example is one thread waiting on a dma_fence while holding a lock:

lock(A);
dma_fence_wait(B);
unlock(A);

while the other thread is stuck trying to acquire the same lock, which prevents it from signalling the fence the previous thread is stuck waiting on:

lock(A);
unlock(A);
dma_fence_signal(B);

By manually annotating all code relevant to signalling a dma_fence we can teach lockdep about these dependencies, which also helps with the validation headache since now lockdep can check all the rules for us:

cookie = dma_fence_begin_signalling;
lock(A);
unlock(A);
dma_fence_signal(B);
dma_fence_end_signalling(cookie);

For using dma_fence_begin_signalling and dma_fence_end_signalling to annotate critical sections the following rules need to be observed:

• All code necessary to complete a dma_fence must be annotated ✨ — from the point where a fence is accessible to other threads, to the point where dma_fence_signal is called. Un-annotated code can contain deadlock issues, and due to the very strict rules and many corner cases it is infeasible to catch these just with review or normal stress testing.
• struct dma_resv deserves a special note 👑 — since the readers are only protected by rcu. This means the signalling critical section starts as soon as the new fences are installed, even before dma_resv_unlock is called.
• The only exception are fast paths and opportunistic signalling code 🏃‍♀️ — which calls dma_fence_signal purely as an optimization, but is not required to guarantee completion of a dma_fence. The usual example is a wait IOCTL which calls dma_fence_signal, while the mandatory completion path goes through a hardware interrupt and possible job completion worker.
• To aid composability of code, the annotations can be freely nested 🧚 — as long as the overall locking hierarchy is consistent. The annotations also work both in interrupt and process context. Due to implementation details this requires that callers pass an opaque cookie from dma_fence_begin_signalling to dma_fence_end_signalling.
• Validation against the cross driver contract is implemented by priming lockdep with the relevant hierarchy at boot-up 🌈 — this means even just testing with a single device is enough to validate a driver, at least as far as deadlocks with dma_fence_wait against dma_fence_signal are concerned.

👀 SEE ALSO 👀

Kernel file ./drivers/dma-buf/dma-fence.c — then the whole reference squad: DMA fences overview(9), fence cross-driver contract(9), dma_fence_get_stub(9), dma_fence_allocate_private_stub(9), dma_fence_context_alloc(9), dma_fence_begin_signalling(9), dma_fence_end_signalling(9), dma_fence_signal_timestamp_locked(9), dma_fence_signal_timestamp(9), dma_fence_signal_locked(9), dma_fence_check_and_signal_locked(9), dma_fence_check_and_signal(9), dma_fence_signal(9), dma_fence_wait_timeout(9), dma_fence_release(9), dma_fence_free(9), dma_fence_enable_signaling(9), dma_fence_add_callback(9), dma_fence_get_status(9), dma_fence_remove_callback(9), dma_fence_default_wait(9), dma_fence_wait_any_timeout(9), deadline hints(9), dma_fence_set_deadline(9), dma_fence_describe(9), dma_fence_init(9), dma_fence_init64(9), dma_fence_driver_name(9), dma_fence_timeline_name(9).

mwah 💋 — source/version path is ./drivers/dma-buf, date stamp is October 2026, and this page is literally fence signalling annotation(9). periodt. 🎀