KTR(4)

✨ NAME ✨

ktr — the kernel tracing facility, and yeah bestie, this whole page is living in the Kernel Interfaces Manual 🎀

---

✨ SYNOPSIS ✨

options KTR
options ALQ
options KTR_ALQ
options KTR_COMPILE=(KTR_LOCK|KTR_INTR|KTR_PROC)
options KTR_CPUMASK=0x3
options KTR_ENTRIES=8192
options KTR_MASK=(KTR_INTR|KTR_PROC)
options KTR_VERBOSE

---

🌸 DESCRIPTION 🌸

okay so picture this: the ktr facility lets kernel events get logged while the kernel is out here doing its thing, so you can go back and look at them later when you're debugging. that's the whole vibe 💅. the only mandatory option to turn ktr on is options KTR — like, that's the bare minimum, the main character requirement, no cap.

KTR_ENTRIES sets the size of the buffer of events. you can check the size of the buffer in the kernel that's currently running via the sysctl debug.ktr.entries. by default the buffer holds 1024 entries 🫶.

🦋 Event Masking 🦋

event levels can be turned on or off to trim down excessive and overly verbose logging — because girl, some of these events are a lot 😭. first up, you specify a mask of events at compile time via the KTR_COMPILE option, which limits which events actually get compiled into the kernel. the default for this option? all events enabled ✨.

secondly, the actual events logged while the kernel runs can be masked even further via the run time event mask. KTR_MASK sets the default value of the run time event mask. the runtime event mask can also be set by loader(8) through the debug.ktr.mask environment variable. it can also be examined and set after booting via the debug.ktr.mask sysctl. by default the run time mask is set to block any tracing 💀. the definitions of the event mask bits live in <sys/ktr_class.h>.

and there's more — there's also a CPU event mask, whose default value you can change via the KTR_CPUMASK option. when you use two or more parameters to KTR_CPUMASK, it is important they are not separated by whitespace (she means it 🙈). a CPU has to have the bit corresponding to its logical id set in this bitmask for events happening on it to get logged. this mask can be set by loader(8) via the debug.ktr.cpumask environment variable. it can also be examined and set after booting via the debug.ktr.cpumask sysctl. by default, only CPUs specified in KTR_CPUMASK will log events. see sys/conf/NOTES for more info, iykyk 🌈.

⭐ Verbose Mode ⭐

by default, events are only logged to the internal buffer so you can examine them later. but if the verbose flag is set, they get dumped to the kernel console too — extra, but in a useful way 🫶. this flag can also be set from the loader via the debug.ktr.verbose environment variable, or it can be examined and set after booting via the debug.ktr.verbose sysctl. here's the tea:

• if the flag is set to zero, which is the default, verbose output is disabled.
• if the flag is set to one, the contents of the log message and the CPU number get printed to the kernel console.
• if the flag is greater than one, the filename and line number of the event get output to the console in addition to the log message and the CPU number 👑

and KTR_VERBOSE sets the flag to one.

💜 Examining the Events 💜

you can examine the KTR buffer from inside ddb(4) via the show ktr [/vV] command. this command displays the contents of the trace buffer one page at a time. at the --more-- prompt:

• Enter displays one more entry and then prompts again
• the spacebar displays another page of entries
• any other key quits

by default the timestamp, filename, and line number are not displayed with each log entry. if you specify the /v modifier, then they're displayed in addition to the normal output. if you specify the /V modifier, then just the timestamp is displayed in addition to the normal output. and heads up 🫶 — the events are displayed in reverse chronological order, meaning the most recent events show up first. iconic behavior honestly.

🔥 Logging ktr to Disk 🔥

the KTR_ALQ option can be used to log ktr entries to disk for post analysis using the ktrdump(8) utility. this option depends on the ALQ option. because of the potentially high volume of trace messages, the trace mask should be selected carefully 👀. this feature is configured through a group of sysctls:

• debug.ktr.alq_file — displays or sets the file that ktr will log to. by default its value is /tmp/ktr.out. if the file name is changed while ktr is enabled, it will not take effect until the next invocation.
• debug.ktr.alq_enable — enables logging of ktr entries to disk if it's set to one. setting this to 0 will terminate logging to disk and revert to logging to the normal ktr ring buffer. data is not sent to the ring buffer while logging to disk.
• debug.ktr.alq_max — the maximum number of entries that will be recorded to disk, or 0 for infinite. this is helpful for limiting the number of particularly high frequency entries that get recorded.
• debug.ktr.alq_depth — determines the number of entries in the write buffer. this is the buffer that holds entries before they're written to disk, and it defaults to the value of the KTR_ENTRIES option.
• debug.ktr.alq_failed — records the number of times we failed to write an entry due to overflowing the write buffer 😭. this may happen if the frequency of the logged ktr messages outpaces the depth of the queue.
• debug.ktr.alq_cnt — records the number of entries that have currently been written to disk 💖.

---

🌈 SEE ALSO 🌈

ktrdump(8), alq(9), ktr(9)

---

🎀 HISTORY 🎀

the KTR kernel tracing facility first appeared in BSD/OS 3.0 and was imported into FreeBSD 5.0. a true legend 👑.

---

that's the whole page bestie, go trace some kernels 💅✨

this iconic little doc came from the Debian manual set, dated March 26, 2021 — mwah, kiss kiss 🫶💖