funcslower(8)

🌸 NAME 🌸

okay bestie, welcome to funcslower(8) from the System Manager's Manual — the icon of tracing slow kernel or user function calls. The name is funcslower, and the job is literally: trace slow kernel or user function calls. That's it. That's the mission. 💅

✨ SYNOPSIS ✨

funcslower [-hf] [-p PID] [-U | -K] [-m MIN_MS] [-u MIN_US] [-a
ARGUMENTS] [-T] [-t] [-v] function [function ...]

💅 DESCRIPTION 💅

This script traces a kernel or user function's entry and return points, and prints a message when the function's latency exceeded the specified threshold. Multiple functions are supported, and you can mix kernel functions with user functions in different libraries. versatile queen behavior ✨

WARNING: See the OVERHEAD section. Do not skip it, bestie, I'm serious.

By default, a minimum millisecond threshold of 1 is used. Recursive functions are not supported: only the inner-most recursive invocation will be traced. iykyk 🌀

Since this uses BPF, only the root user can use this tool. No cap.

🧚 REQUIREMENTS 🧚

CONFIG_BPF and bcc. That's the whole requirements list. Short, sweet, slay. 🎀

🎀 OPTIONS 🎀

• -p PID — trace this PID only. Just this one. main character PID energy.
• -m MIN_NS — minimum duration to trace, in milliseconds. Default is 1 ms. (Yes, the option line literally says MIN_NS while the synopsis says MIN_MS, and the description says milliseconds — I am keeping every bit exact, bestie. No fixing, only preserving. 💖)
• -u MIN_US — minimum duration to trace, in microseconds.
• -a ARGUMENTS — print the function's arguments, up to 6. Six little argument besties, max.
• -T — print a HH:MM:SS timestamp with each entry.
• -t — print a seconds timestamp with each entry, at microsecond resolution.
• -f — print output in folded stack format.
• -U — show stacks from user space only (no kernel space stacks).
• -K — show stacks from kernel space only (no user space stacks).
• -v — print the resulting BPF program, for debugging purposes. debug girlies rise 🛠️
• -h — it appears in the synopsis as part of [-hf], but the original OPTIONS section does not give it its own paragraph, so I am not inventing behavior for it. You still see it here, bestie. 💀
• function — the function to trace -- multiple functions are supported. If a plain function name is provided, the function is assumed to be a kernel function. For user functions, provide the library name and the function name, e.g. bash:readline or c:malloc.

🔥 EXAMPLES 🔥

Trace vfs_write calls slower than 1ms:

# funcslower vfs_write

Trace open() calls in libc slower than 10us:

# funcslower -u 10 c:open

Trace both malloc() and free() slower than 10us, in pid 135 only:

# funcslower -p 135 -u 10 c:malloc c:free

Trace the write syscall and print its first 4 arguments:

# funcslower -a 4 SyS_write

Trace opens from libc and print the user and kernel stack frames:

# funcslower -UK c:open

every example preserved, every command exact, no crumbs left behind 🌈

🦋 FIELDS 🦋

• TIME — time of the event as a human-readable HH:MM:SS format, or a timestamp in seconds at microsecond-accuracy from the first event seen.
• COMM — process name.
• PID — process ID.
• LAT — latency of the operation in either microseconds (us) or milliseconds (ms).
• RVAL — the return value from the function. Often useful for diagnosing a relationship between slow and failed function calls.
• FUNC — the function name, followed by its arguments if requested.

💀 OVERHEAD 💀

Depending on the function(s) being traced, overhead can become severe. For example, tracing a common function like malloc() can slow down a C/C++ program by a factor of 2 or more. On the other hand, tracing a low-frequency event like the SyS_setreuid() function will probably not be as prohibitive, and in fact negligible for functions that are called up to 100-1000 times per second. choose your traced functions wisely, bestie 🫠

You should first use the funclatency and argdist tools for investigation, because they summarize data in-kernel and have a much lower overhead than this tool. To get a general idea of the number of times a particular function is called (and estimate the overhead), use the funccount tool, e.g.:

# funccount c:open

Fr fr: this tool can get heavy. Start with the low-overhead summarizers first, then come back. 💀

🌈 SOURCE 🌈

This is from bcc.

https://github.com/iovisor/bcc

Also look in the bcc distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool. homework but make it cute 🎀

🐧 OS 🐧

Linux. Obviously. 🐧💚

⭐ STABILITY ⭐

Unstable - in development. she's growing, she's evolving, she's in development 🦋

👑 AUTHOR 👑

Sasha Goldshtein. Icon behavior. 👑

💜 SEE ALSO 💜

funccount(8), funclatency(8), argdist(8), trace(8). Go read the whole friend group. 💜

---

okay that's the whole page, bestie — cute little sign-off with USER COMMANDS, dated 2017-03-30, officially funcslower(8). mwah 💋✨