btrfsslower(8)

💖 btrfsslower(8) 💖

okay bestie, welcome to the System Manager's Manual page for btrfsslower(8)! 🎀 it's the cutest little tracer for finding slow btrfs file ops ✨

✨ NAME ✨

btrfsslower - Trace slow btrfs file operations, with per-event details. 💅

✨ SYNOPSIS ✨

btrfsslower [-h] [-j] [-p PID] [min_ms] [-d DURATION]

✨ DESCRIPTION ✨

okay so this tool traces the common btrfs file operations: reads, writes, opens, and syncs 🧚 It measures how long each one takes and prints the details for any that went over a threshold. main character energy, literally 👑

WARNING: See the OVERHEAD section. 💀

By default the minimum threshold is 10 milliseconds. If you use a threshold of 0, every event gets printed (warning: verbose 🙈).

Since this works by tracing the btrfs_file_operations interface functions, it's going to need updating to match any changes to those functions. no cap 🫶

Since this uses BPF, only the root user can use this tool. so be root, bestie 🔒

✨ REQUIREMENTS ✨

CONFIG_BPF and bcc. 🌸

✨ OPTIONS ✨

• -p PID — trace this PID only 🎯
• min_ms — the minimum I/O latency (duration) to trace, in milliseconds. the default is 10 ms ⏱️
• -d DURATION — the total duration of the trace, in seconds ⏳

✨ EXAMPLES ✨

Trace synchronous file reads and writes slower than 10 ms:

# btrfsslower

Trace slower than 1 ms:

# btrfsslower 1

Trace slower than 1 ms, and output just the fields in parsable format (csv):

# btrfsslower -j 1

Trace all file reads and writes (warning: the output will be verbose 🙈):

# btrfsslower 0

Trace slower than 1 ms, for PID 181 only:

# btrfsslower -p 181 1

Trace for 10 seconds only:

# btrfsslower -d 10

✨ FIELDS ✨

okay the output columns, let's go 💖

• TIME(s) — time of I/O completion since the first I/O seen, in seconds ⏰
• COMM — process name 🧑‍💻
• PID — process ID 🆔
• T — type of operation. R == read, W == write, O == open, S == fsync 🔤
• OFF_KB — file offset for the I/O, in Kbytes 📦
• BYTES — size of the I/O, in bytes 🧮
• LAT(ms) — latency (duration) of the I/O, measured from when VFS issued it to the filesystem, until it completed. this time is inclusive of block device I/O, file system CPU cycles, file system locks, run queue latency, etc. it's a way more accurate measure of the latency apps actually feel doing file system I/O than measuring down at the block device interface. ✨
• FILENAME — a cached kernel file name (comes from dentry->d_name.name) 📝
• ENDTIME_us — completion timestamp, in microseconds (-j only) ⌚
• OFFSET_b — file offset, in bytes (-j only) 📍
• LATENCY_us — latency (duration) of the I/O, in microseconds (-j only) 🚀

✨ OVERHEAD ✨

this adds low-overhead instrumentation to btrfs writes and fsyncs, as well as to all system reads and opens (because of how the current btrfs_file_operations interface is implemented). the tricky part: all reads and writes from the file system cache will take extra overhead while tracing. those can be super frequent (depending on the workload, eg, 1M/sec), and at that point the tool's overhead can become noticeable. 😭

so measure and quantify before you use it 📊 if it keeps being a problem, consider switching to a tool that only prints in-kernel summaries, like btrfsdist(8) 🦋

also, the overhead of this tool should be lower than fileslower(8), since this one only targets btrfs functions and not every file read/write path (which can include socket I/O) 🌈

✨ SOURCE ✨

this is from bcc 🫧

https://github.com/iovisor/bcc

also check out the bcc distribution for a companion _examples.txt file with example usage, output, and commentary for this tool 📚

✨ OS ✨

Linux 🐧

✨ STABILITY ✨

Unstable - in development. 🚧

✨ AUTHOR ✨

Brendan Gregg 👑

✨ SEE ALSO ✨

btrfsdist(8), biosnoop(8), funccount(8), fileslower(8) 🔗

---

bestie, that's the whole page! 🌈💜 this is from the USER COMMANDS section, dated 2016-02-15, and I'm obsessed that you made it this far 🫶✨