softirqs(8)

๐Ÿ“ฆ
version 0.35.0 ยท 2 versions
  • 0.35.0 ยท debian bookworm, debian trixie, ubuntu noble, ubuntu questing, ubuntu resolute, ubuntu stonking (latest)
  • 0.18.0 ยท ubuntu jammy
, in debian bookworm, debian trixie, ubuntu noble, ubuntu questing, ubuntu resolute, ubuntu stonking

๐Ÿฆ‹ NAME ๐Ÿฆ‹

okay so softirqs is the tool that measures soft IRQ (soft interrupt) event time ๐Ÿงš It uses Linux eBPF/bcc, so it's giving low-level, and I'm obsessed ๐Ÿ’…

๐Ÿ’– SYNOPSIS ๐Ÿ’–

softirqs [-h] [-T] [-N] [-C] [-d] [-c CPU] [interval] [count]

โœจ DESCRIPTION โœจ

bestie, this one summarizes the time spent servicing soft IRQs (soft interrupts) ๐Ÿซถ You can see that time as plain totals or as histogram distributions, your choice ๐Ÿ“Š A system-wide version of this time lives in the %soft column of mpstat(1), and soft IRQ event counts (but not times) are chilling in /proc/softirqs ๐ŸŒธ

This tool uses the irq:softirq_enter and irq:softirq_exit kernel tracepoints, which is a stable tracing mechanism, so no notes ๐Ÿ’œ BPF programs can attach to tracepoints from Linux 4.7 only, so keep that in mind ๐Ÿ™ˆ An older version is in tools/old, and it's kprobes-based instead of tracepoint-based. Very retro ๐Ÿ“ผ

Since this uses BPF, only the root user can use this tool ๐Ÿ”’ Sorry not sorry, that's the rule.

๐ŸŽ€ REQUIREMENTS ๐ŸŽ€

You need CONFIG_BPF and bcc. No way around it ๐Ÿ’€

๐Ÿ‘‘ OPTIONS ๐Ÿ‘‘

  • -h โ€” pulls up the usage message ๐Ÿ“–
  • -T โ€” slaps timestamps onto the output โฐ
  • -N โ€” outputs in nanoseconds ๐Ÿ”ฌ
  • -C โ€” shows the number of soft irq events ๐Ÿ”ข
  • -d โ€” shows the IRQ time distribution as histograms ๐Ÿ“Š
  • -c CPU โ€” traces on this CPU only, the one you pick ๐ŸŽฏ

๐ŸŒˆ EXAMPLES ๐ŸŒˆ

Sum the soft IRQ event time until you hit Ctrl-C:

# softirqs

Show the number of soft irq events:

# softirqs -C

Show soft IRQ event time as histograms:

# softirqs -d

Print 1 second summaries, 10 times:

# softirqs 1 10

1 second summaries, printed in nanoseconds, with timestamps:

# softirqs -NT 1

Sum soft IRQ event time on CPU 1 until Ctrl-C:

# softirqs -c 1

๐Ÿ“ FIELDS ๐Ÿ“

  • SOFTIRQ โ€” the kernel function name doing the soft IRQ action ๐Ÿ› ๏ธ
  • TOTAL_usecs โ€” total time spent in this soft IRQ function, in microseconds โฑ๏ธ
  • TOTAL_nsecs โ€” total time spent in this soft IRQ function, in nanoseconds โฑ๏ธ
  • usecs โ€” the microsecond range for this bucket ๐Ÿ“
  • nsecs โ€” the nanosecond range for this bucket ๐Ÿ“
  • count โ€” how many soft IRQs landed in this time range ๐Ÿงฎ
  • distribution โ€” an ASCII picture of the distribution (the count column) ๐Ÿ“ˆ

๐Ÿ”ฅ OVERHEAD ๐Ÿ”ฅ

okay real talk, this traces kernel functions and keeps in-kernel counts, which get asynchronously copied to user-space. If the interrupt rate is very high (>1M/sec), it's still a relatively efficient way to trace these events, so overhead should be small for normal workloads ๐Ÿฅน It could get noticeable for heavy workloads though. Measure in a test environment before use ๐Ÿงช

๐ŸŒธ SOURCE ๐ŸŒธ

This is from bcc, obviously ๐Ÿ’œ

https://github.com/iovisor/bcc

Also look in the bcc distribution for a companion _examples.txt file with example usage, output, and commentary for this tool ๐Ÿ“š

๐Ÿง OS ๐Ÿง

Linux ๐Ÿง

๐Ÿงช STABILITY ๐Ÿงช

Unstable, still in development ๐Ÿšง

๐Ÿซถ AUTHORS ๐Ÿซถ

Brendan Gregg, Sasha Goldshtein, Rocky Xing ๐Ÿ‘‘

๐Ÿฆ‹ SEE ALSO ๐Ÿฆ‹

hardirqs(8) ๐Ÿ’…


bye bestie ๐Ÿ’– this was softirqs(8) from the USER COMMANDS section, dated 2015-10-20 ๐ŸŽ€ slay forever โœจ