HWT(4)

okay bestie, welcome to HWT(4) — the Kernel Interfaces Manual page for the Hardware Trace Framework, and it’s giving main-character energy 💅✨

---

🌸 NAME 🌸

hwt — Hardware Trace Framework 💖

---

✨ SYNOPSIS ✨

okay so here’s the build/load recipe, bestie — copy it exactly, no improvising 🫶

options HWT_HOOKS
device hwt

At least one of:
device intel_pt
(amd64)
device coresight
(arm64)
device spe
(arm64)

In rc.conf(5):
kld_list="hwt"

---

🦋 DESCRIPTION 🦋

The hwt framework is the infrastructure for hardware-assisted tracing. It collects super-detailed info about software execution and stores it as events in a highly compressed format into DRAM. Those events cover info about control flow changes of a program, whether branches are taken or not, exceptions taken, timing information, cycles elapsed and more. The info collected lets you reconstruct the entire program flow of a given application without a noticeable performance impact. Slay. 💅

---

🎀 HARDWARE 🎀

The framework supports several tracing technologies found on arm64 and amd64 systems:

• ARM Coresight
• ARM Statistical Profiling Extension (SPE)
• Intel Processor Trace (PT)

The hwt framework supports two modes of operation:

• CPU mode — capture CPU activity in kernel mode.
• Thread mode — capture activity of each of a process’s threads in user mode.

---

👑 MANAGEMENT 👑

When loaded into kernel, the hwt framework provides the /dev/hwt character device. The only ioctl(2) request it accepts is HWT_IOC_ALLOC. This request allocates kernel tracing context (CTX) based on requested mode of operation, set of CPUs and/or pid.

bestie, that’s the only ioctl request /dev/hwt accepts — no extras, fr fr.

Upon successful CTX allocation, the ioctl returns a CTX identification number (ident).

Each CTX is then managed using its own dedicated character device found at /dev/hwt_${ident}_${d}, where ident is a unique identification number of tracing context, d is either cpu_id (in HWT CPU mode) or process pid (in HWT Thread mode).

---

🪝 HOOKS 🪝

During tracing of a target process, HWT records runtime events such as threads creation, exec and mmap system calls. These events are logged as "records" within a particular CTX associated with traced process.

Additionally, HWT can suspend the target thread upon exec or mmap system calls if requested by the user. This pause allows user-space tools to retrieve the records and adjust tracing settings before execution continues. This feature is especially useful when address range filtering is enabled, allowing tracing of specific functions within the target executable or a dynamic library. Obsessed with the control this gives you, ngl 💖

---

⚙️ KERNEL OPTIONS ⚙️

The following options in the kernel configuration file are mandatory and related to hwt operation:

• HWT_HOOKS — enable kernel hooks.

---

🔥 IOCTL INTERFACE 🔥

Once a CTX is allocated, its management character device accepts several ioctl(2) requests:

• HWT_IOC_START — start tracing. In HWT CPU mode the tracing does actually start with this ioctl(2) request. In the Thread mode, the tracing "running" flag set, but tracing begins after scheduler switches the target thread onto CPU and return to user mode.
• HWT_IOC_STOP — stop tracing of the particular CTX.
• HWT_IOC_RECORD_GET — copy all or part of records collected during hook invocation and associated with this CTX to userspace.
• HWT_IOC_BUFPTR_GET — get current pointer in buffer that is filled by tracing units in real-time.
• HWT_IOC_SET_CONFIG — set architecture-specific config (optional).
• HWT_IOC_WAKEUP — wake up a thread that has been put to sleep by HWT framework hooks.
• HWT_IOC_SVC_BUF — for SPE-only, the kernel is waiting for userspace to notify that it has copied out a buffer to avoid data loss/overwriting buffers.

---

🌈 SEE ALSO 🌈

• tracing(7)
• hwt(8)

---

⭐ HISTORY ⭐

The hwt framework first appeared in FreeBSD 15.0.

---

💜 AUTHORS 💜

• Ruslan Bukin <br@FreeBSD.org>
• Bojan Novković <bnovkov@freebsd.org>
• Zachary Leaf <zachary.leaf@arm.com>

---

made with 💖 on Debian • July 12, 2025 • HWT(4) — byeee bestie 🫶