criticalstat(8)

💅 criticalstat(8) — the manual page, but make it make sense

okay bestie, sit down, hydrate, because we're about to talk about criticalstat, a tracer from the System Manager's Manual that finds long atomic critical sections in the kernel and reports them. yes it's giving detective energy. yes it's iconic. let's go 🕵️♀️✨

---

✨ NAME ✨

criticalstat — a tracer that finds and reports long atomic critical sections in the kernel. no cap, that's the whole job. 🎀

---

🌸 SYNOPSIS 🌸

criticalstat [-h] [-p] [-i] [-d DURATION]

---

🦋 DESCRIPTION 🦋

so like, criticalstat traces and reports occurrences of atomic critical sections in the kernel, and it hands you useful stacktraces showing where they came from. origin story, but for latency. 🔥

these critical sections pop up all the time: spinlocks, drivers explicitly disabling interrupts or preemption, whatever. IRQ routines in Linux also run with interrupts disabled, so there's a whole buffet of reasons. and bestie, such critical sections are a source of long latency / responsiveness issues for real-time systems. that's the villain of this story, fr fr. 💀

it works by probing the preempt/irq and cpuidle tracepoints in the kernel. and because it uses BPF, only the root user can use this tool. also your kernel has to be built with certain CONFIG options enabled — see below, i got you. 👑

---

⭐ REQUIREMENTS ⭐

enable the following kernel configurations, depending on which kernel version you're on:

• CONFIG_DEBUG_PREEMPT
• CONFIG_PREEMPT_TRACER

for kernel 4.19 and later:

• CONFIG_PREEMPTIRQ_TRACEPOINTS
• CONFIG_TRACE_IRQFLAGS
• CONFIG_TRACE_PREEMPT_TOGGLE

for kernel 4.15 to 4.18:

• CONFIG_PREEMPTIRQ_EVENTS
• CONFIG_PROVE_LOCKING
• CONFIG_DEBUG_PREEMPT

additionally, the following option should be turned off on older kernels:

• CONFIG_LOCKDEP

💅 vibe check: no config, no trace. the kernel has to be built for this. periodt.

---

💖 OPTIONS 💖

• -h — print usage message. the classic "help me bestie" flag. 🙈
• -p — find long sections where preemption was disabled on the local CPU.
• -i — find long sections where interrupt was disabled on the local CPU.
• -d DURATION — only identify sections that are longer than DURATION, measured in microseconds.

---

🌈 EXAMPLES 🌈

run with default options: irq disabled for more than 100 uS

# criticalstat

find sections with preemption disabled for more than 100 uS

# criticalstat -p

find sections with IRQ disabled for more than 500 uS

# criticalstat -d 500

find sections with preemption disabled for more than 500 uS

# criticalstat -p -d 500

---

🍓 OVERHEAD 🍓

ngl, this tool can cause overhead if your application is spending a lot of time in kernel mode. the overhead is variable, but it can be 2-4% of performance degradation. 😭 if the overhead is looking like too much, please pass a higher DURATION to the -d option so it filters more aggressively. main character energy, but make it efficient. 💅

---

🧚 SOURCE 🧚

this is from bcc.

https://github.com/iovisor/bcc

also look in the bcc distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool. it's the director's cut, basically. 🎬

---

💜 OS 💜

Linux. obviously. 🐧

---

🔥 STABILITY 🔥

Unstable - in development. she's in her experimental era, we respect it. 🫶

---

👑 AUTHOR 👑

Joel Fernandes

---

🌟 SEE ALSO 🌟

Linux kernel's preemptoff and irqoff tracers.

---

and that's the whole page, bestie!! this little number is from the USER COMMANDS corner of the manual, dated 2018-06-07, and it goes by criticalstat(8). go forth and find those long critical sections. slay. 💖✨🎀🦋