wakeuptime(8)

✨ wakeuptime(8) ✨

System Manager's Manual, so yes, this one's official 💅

🎀 NAME 🎀

wakeuptime is your go-to for summarizing sleep-to-wakeup time by waker kernel stack! It uses Linux eBPF/bcc 🧚

📜 SYNOPSIS 📜

wakeuptime [-h] [-u] [-p PID] [-v] [-f] [--stack-storage-size STACK_STORAGE_SIZE] [-m MIN_BLOCK_TIME] [-M MAX_BLOCK_TIME] [duration]

💖 DESCRIPTION 💖

okay so this program shows the kernel stack traces for threads that woke up other blocked threads 😭 It also gives you the process names of the waker and the target, plus a sum of how long the target was blocked, which is called the "blocked time". It works by tracing when threads block and when they get woken up, then measuring the time delta between those two moments. This time measurement is going to be very similar to off-CPU time, but off-CPU time might include a little extra time spent waiting on a run queue to be scheduled. The stack traces, process names, and blocked time all get summarized in the kernel using an eBPF map, which is how it stays efficient 🔥

The output summary helps you figure out why threads were blocking by showing who woke them up and how long they were stuck. It spans all types of blocking activity: disk I/O, network I/O, locks, page faults, involuntary context switches, etc. literally everything that makes a thread wait 🙈

This can be used together with offcputime, which shows the stack trace of the blocked thread. wakeuptime shows the stack trace of the waker thread, so you get both sides of the story ✨

Check out http://www.brendangregg.com/FlameGraphs/offcpuflamegraphs.html for more on this.

⚠️ This tool only works on Linux 4.6+. It uses the new BPF_STACK_TRACE table APIs to generate the in-kernel stack traces, so make sure your kernel is up to date bestie 💀

---

🔮 REQUIREMENTS 🔮

You need CONFIG_BPF and bcc. No cap, that's the whole list 🫶

🌸 OPTIONS 🌸

• -h — Print the usage message 🙋‍♀️
• -f — Print the output in folded stack format 🦋
• -u — Only trace user threads, so kernel threads are left out 🌈
• -v — Show raw addresses (for the non-folded format) 🧐
• -p PID — Trace this process ID only (filtered in-kernel) 🎯
• --stack-storage-size STACK_STORAGE_SIZE — Change the number of unique stack traces that can be stored and displayed 📚
• duration — How long to trace, in seconds ⏱️
• -m MIN_BLOCK_TIME — The amount of time in microseconds over which we store traces (default 1) 🍓
• -M MAX_BLOCK_TIME — The amount of time in microseconds under which we store traces (default U64_MAX) 🍑

✨ EXAMPLES ✨

Trace all thread blocking events, and summarize (in-kernel) by kernel stack trace and total blocked time. Main character energy 👑

# wakeuptime

Trace user-mode target threads only 🫧

# wakeuptime -u

Trace for 5 seconds only ⏳

# wakeuptime 5

Trace for 5 seconds and emit output in folded stack format, which is perfect for flame graphs 🔥

# wakeuptime -f 5

Trace PID 185 only 💜

# wakeuptime -p 185

🐢 OVERHEAD 🐢

This summarizes unique stack traces in-kernel, which is super efficient and lets it trace a higher rate of events than methods that post-process in user space. The stack trace and time data only gets copied to user space once, when the output is printed. Those techniques lower overhead a ton, but scheduler events are still high frequency, since they can exceed 1 million events per second, so be careful 😭 Test before production use.

If the overhead is still a problem, take a look at the min block option. If you're chasing down longer blocking events, you can increase it to filter out shorter blocking events, which lowers overhead even more 🫶

🌍 SOURCE 🌍

This is from bcc. Slay 💅

https://github.com/iovisor/bcc

Also look in the bcc distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool. It's giving helpful 📖

🐧 OS 🐧

Linux 🐧

🧪 STABILITY 🧪

Unstable, since it's still in development 🙈

👩‍💻 AUTHOR 👩‍💻

Brendan Gregg 👑

🔗 SEE ALSO 🔗

offcputime(8), stackcount(8) 🦋

---

Bestie, that's the whole page! 💖 Section USER COMMANDS, dated 2016-01-27, wakeuptime(8) 🌈 ttyl 🧚✨