ext4slower(8)

🎀 ext4slower(8) 🎀

System Manager's Manual 💜

✨ NAME ✨

okay so ext4slower is the tool that traces slow ext4 file operations and gives you per-event details 🔍 bestie, it's giving detective energy.

💅 SYNOPSIS 💅

ext4slower [-h] [-j] [-p PID] [min_ms]

📖 DESCRIPTION 📖

this tool keeps an eye on the common ext4 file operations: reads, writes, opens, and syncs 👀 it times each one and prints the details for any that go over a threshold. main character energy ✨

🚨 WARNING: see the OVERHEAD section. no cap, read it.

by default the minimum threshold is 10 milliseconds 💖 if you set it to 0, every single event gets printed (warning: verbose, we are not kidding 🙈).

lowkey important: this works by tracing the ext4_file_operations interface functions, so it'll need updating whenever those functions change 🔧

also, since this uses BPF, only the root user can run it 👑

🧰 REQUIREMENTS 🧰

you'll need CONFIG_BPF and bcc 🌸

🎀 OPTIONS 🎀

• -p PID — trace this PID only 🦋
• min_ms — the minimum I/O latency (duration) to trace, in milliseconds ⏱️ the default is 10 ms.

🍓 EXAMPLES 🍓

trace synchronous file reads and writes slower than 10 ms:

# ext4slower

trace slower than 1 ms:

# ext4slower 1

trace slower than 1 ms, and output just the fields in parsable format (csv) 🧚:

# ext4slower -j 1

trace all file reads and writes (warning: the output will be verbose 🙈):

# ext4slower 0

trace slower than 1 ms, for PID 181 only 💜:

# ext4slower -p 181 1

🔥 FIELDS 🔥

• TIME(s) — the time of I/O completion since the first I/O seen, in seconds ⭐
• COMM — the process name 🫶
• PID — the process ID 🆔
• T — the type of operation. R == read, W == write, O == open, S == fsync 💅
• OFF_KB — the file offset for the I/O, in Kbytes 📍
• BYTES — the size of the I/O, in bytes 📦
• LAT(ms) — the latency (duration) of the I/O, measured from when VFS issued it to the filesystem, until it completed. this time is inclusive of block device I/O, file system CPU cycles, file system locks, run queue latency, etc. it's a more accurate measure of the latency applications actually feel doing file system I/O, than measuring down at the block device interface 🥺✨
• FILENAME — a cached kernel file name (comes from dentry->d_name.name) 📄
• ENDTIME_us — the completion timestamp, in microseconds (-j only) ⏰
• OFFSET_b — the file offset, in bytes (-j only) 🧮
• LATENCY_us — the latency (duration) of the I/O, in microseconds (-j only) 🌈

🐢 OVERHEAD 🐢

⚠️ this adds low-overhead instrumentation to these ext4 operations, including reads and writes from the file system cache. those reads and writes can be super frequent (depending on the workload; eg, 1M/sec), and at that point the overhead of this tool can start to get significant, even if it prints no "slower" events 😭 measure and quantify before you use it. if it keeps being a problem, consider switching to a tool that only prints in-kernel summaries.

the overhead of this tool should be less than fileslower(8), though, because this one only targets ext4 functions and not every file read/write path (which can include socket I/O) 🫶

🌸 SOURCE 🌸

this is from bcc 💖

https://github.com/iovisor/bcc

also look in the bcc distribution for a companion _examples.txt file with example usage, output, and commentary for this tool 📚

💻 OS 💻

Linux 🐧

🧪 STABILITY 🧪

Unstable - in development 🚧

👩‍💻 AUTHOR 👩‍💻

Brendan Gregg 👑

🔗 SEE ALSO 🔗

biosnoop(8), funccount(8), fileslower(8) 🦋

---

bestie, that's the whole tea ☕💖 this page is from the USER COMMANDS section, dated 2016-02-11, and sourced from bcc, so go forth and trace those slow ext4 ops 🧚✨💅