TIPTOP(1)

πŸ“¦
version 2.3.2 Β· 2 versions
, in fedora 43, fedora 44, ubuntu resolute, ubuntu stonking

πŸ’– TIPTOP(1) β€” the tiptop manual, but make it make sense πŸ’–

okay bestie, sit down, hydrate, because this one is a whole journey ✨ This is the manual page for tiptop, living in section 1 of the manual. It's from Inria, it's all about staring at hardware performance counters for Linux tasks, and yes it is exactly as iconic as it sounds. πŸŽ€


✨ NAME ✨

tiptop β€” display hardware performance counters for Linux tasks πŸ¦‹


🌸 SYNOPSIS 🌸

tiptop [OPTION]

tiptop [OPTION] -- command  (EXPERIMENTAL)

ptiptop PATTERN [OPTIONS]

πŸ’… DESCRIPTION πŸ’…

So. The tiptop program gives you a dynamic, real-time view of the tasks running on your system. It's very similar to top (1) β€” like, spiritual siblings fr fr β€” except the info it shows you comes from hardware counters. That's the whole vibe. πŸ‘‘

tiptop has two running modes, and you need to know both: live-mode and batch-mode. In both modes, the system gets periodically queried for the values of hardware counters, and then various ratios get printed for each task. ✨

  • In live-mode, the display is regularly updated with new values at constant time intervals. It's giving main character energy on your terminal. πŸ’–
  • In batch-mode, that same information is emitted to stdout. Batch-mode is the move when you want to save stuff to a file or do further processing on it. And heads up: no interaction is possible in batch-mode. None. Zero. Don't even try. πŸ™ˆ

Also important: unless tiptop is run by root, or the executable is setuid-root, a user can only monitor the tasks it owns. That's just the rules, bestie. πŸ’€

The results produced by tiptop are organized in screens. A screen consists in rows representing tasks, and columns reporting various values and ratios collected from hardware counters. Many screens can be defined. Only one screen is displayed at a time. The default screen (number 0) reports target independent values as defined in the file /usr/include/linux/event_counter.h. Other screens may rely on target-dependent counters. 🌈

Now, the lil edge cases you must know about:

  • When an expression would result in a division by zero, a '-' sign is printed.
  • When a counter involved in an expression could not be read, a '?' sign is printed.

No cap, those two little characters will save you so much confusion. 🫢

If -- appears in the command line, tiptop treats the rest of the line as a command. A new process is forked, and hardware counters are attached just before execvp is called. This makes it possible to trace an application from the first instruction. Only the child is traced, and idle-mode is enabled (in live mode, this can be overridden by hitting keys 'p' and 'i'). This is commonly used in combination with sticky mode to track a command from start to finish. This is experimental! ⚠️

ptiptop is simply a shortcut for tiptop -p. That's it. That's the whole thing. πŸ’…

And finally: tiptop requires Linux 2.6.31+. Old kernels need not apply. πŸŽ€


πŸŽ€ COMMAND-LINE OPTIONS πŸŽ€

Quick rules before we dive in, because they matter: command line options with a parameter override values specified in the configuration file. Toggles set the value or invert the value read in the configuration file (if any). Got it? Good. Let's go. ✨

  • -b β€” Start tiptop in batch-mode. Output is sent to stdout, and no interactive command is accepted. tiptop will run forever, or until the number of iterations specified by -n. ♾️
  • -c β€” display the command line of the task instead of its name. (toggle)
  • --cpu-min VALUE β€” %CPU activity threshold. Below this value, a task is considered idle and is not reported (unless idle-mode is on, see flag -i). πŸ¦‹
  • -d VALUE β€” Specify the delay between refreshes. VALUE can be fractional. It must be larger than 0.01. ⏱️
  • -E FILENAME β€” Specify file where errors are logged. By default errors are logged to stderr in batch-mode, and a temporary file in live-mode. πŸ“
  • --epoch β€” Print the Epoch at each refresh. In batch-mode, it is printed at the beginning of each row. In live-mode, it is at the bottom of the display. (toggle)
  • -h, --help β€” Print a brief help message and exit. πŸ‘‹
  • -H β€” Show threads. (toggle)
  • --hypervisor β€” Include hypervisor activity in the reported values. ☁️
  • -i β€” Show idle tasks. (toggle)
  • -K, --kernel β€” Include kernel activity in the reported values. This is only possible is the user is root, the tiptop executable is setuid root, or the paranoia level is low enough. (toggle) See file /proc/sys/kernel/perf_event_paranoid (perf_counter_paranoid on Linux 2.6.31).
  • --list-screens β€” List available screens and exit. πŸ“‹
  • -n VALUE β€” Automatically exit after VALUE iterations.
  • --no-collect β€” By default, when the maximum number of open files is reached, and new processes appear, tiptop searches for idle processes and closes the files in order to make room for the new ones. This flag disables this behavior: in case of file shortage, new processes will not get more chances than older ones. 😀
  • -o FILENAME β€” Specify the filename for the output of batch mode. πŸ’Ύ
  • --only-conf β€” Only screens defined in configuration file displayed (no default).
  • -p, --pid VALUE β€” Filters processes according to VALUE. VALUE can be either the numeric value PID, or a string. In case of a string, all tasks whose names or command lines (depending on the display, see -c) contain VALUE are reported. πŸ”
  • -S VALUE β€” Start tiptop with screen number VALUE if VALUE is an integer. Otherwise looks for the first screen whose name contains VALUE. 🌟
  • --sticky β€” Start in sticky mode: tasks stay in the list after they die. In live-mode, they appear in a different color (when supported). In batch-mode, the word DEAD is appended. (toggle) πŸ’€
  • --timestamp β€” Print a timestamp at the beginning of each row. The timestamp is the number of refreshes so far. In batch-mode, it is printed at the beginning of each row. In live-mode, it is at the bottom of the display. (toggle)
  • -u USER β€” Only show tasks owned by USER. USER can be either a login name, or the numeric value UID.
  • -U β€” Show the owner of each task. (toggle)
  • -v β€” Display build information and exit. πŸ”§
  • --version β€” Display version information and disclaimer and exit. πŸ“œ
  • -w VALUE β€” Watch the task specified by VALUE. VALUE can be either the numeric value PID, or a string. In case of a string, all tasks whose names or command lines (depending on the display, see -c) contain VALUE are reported. In live-mode, watched tasks are shown in a different color (when supported). In batch-mode, an ASCII arrow points to the watched tasks. πŸ‘€
  • -W PATH β€” Directory where the configuration file is located. πŸ“

🌈 INTERACTIVE COMMANDS 🌈

In live-mode, tiptop accepts single-key commands. Little keyboard queen. πŸ‘‘

  • LEFT, RIGHT β€” Rotate through available screens. πŸ”„
  • <, > β€” Change the reference column for sorting to the left or to the right.
  • c β€” Toggle between showing task names and command lines.
  • d β€” Change the refresh interval. The new value is queried. Fractional values larger than 0.01 are accepted. ⏱️
  • e β€” Display the errors encoutered so far. Scroll with UP, DOWN, PAGE_UP, PAGE_DOWN, HOME and END. 😭
  • h β€” Display a brief description of the screen and each column. πŸ’‘
  • H β€” Toggle between showing individual threads and accumulating values per process.
  • i β€” Toggle between showing only active tasks and showing also idle tasks.
  • K β€” Toggle between showing kernel activity and only user activity. Kernel mode is only available to root. Switching to and from kernel mode resets all counters. ⚠️
  • k β€” Kill a process. The user is asked for the PID, and the signal to send. ☠️
  • p β€” Filter tasks by name or PID. The user is asked for a PID or string. In case a string is entered, only the tasks whose name or command line contain the string are displayed. Changing the filter resets all counters. πŸ”
  • q β€” Quit. πŸ‘‹
  • R β€” Change sorting order: ascending or descending. ↕️
  • S β€” Toggle sticky mode.
  • s β€” Same as d. (twins fr) πŸ‘―
  • u β€” Filter tasks by user. The user name or PID is queried. Note that, unless tiptop is run by root or setuid root, tasks owned by somebody else cannot be monitored. Changing the filter resets all counters.
  • U β€” Toggle displaying each task's owner.
  • w β€” Used to track a particular task. The user is asked for a PID or string. In case a string is entered, all tasks whose name or command line contain the string are highlighted. ✨
  • W β€” Writes a configuration file for the current state in the current directory. πŸ’Ύ

and bestie… we're not done yet, there's more coming πŸ‘€

✨ FILES ✨

Okay bestie, when tiptop boots up it's gonna go looking for a config file, and that file has exactly one name it answers to: .tiptoprc. πŸ’…

Where does it look? It goes on a little quest, in this order:

  1. the current directory first 🫢
  2. then the directory that the environment variable TIPTOP points to, if that variable even exists
  3. and finally, home sweet home: the user's home directory 🏠

πŸŽ€ Syntax

The file is XML. Yes, XML. We're doing XML. Here's how the whole thing is structured:

Root of tree 🌳

The root of the xml tree is tiptop. Like:

<tiptop> ... </tiptop>

Options βš™οΈ

Options get specified inside an <options> block, bestie:

<options>
  <option name="option1" value="value_option1"/>
  <option name="option2" value="value_option2"/>
       ...  </options>

Here's every recognized option, with the command-line option it's lowkey twinning with:

  • cpu_threshold (--cpu-min)
  • delay (-d)
  • idle (-i)
  • max_iter (-n)
  • show_cmdline (-c)
  • show_epoch (--epoch)
  • show_hypervisor (--hypervisor)
  • show_kernel (-K)
  • show_timestamp (--timestamp)
  • show_threads (-H)
  • show_user (-U)
  • watch_name (-w)
  • sticky (--sticky)
  • watch_uid (-w)

Screens πŸ–₯️

Screens live inside a <screen> block. A screen is made of counters and columns, and it has a name plus an optional description. So:

<screen name="my_screen" desc="what this screen is about">
       ....  </screen>

Counters must provide an alias (that's what gets used for further reference) and a config. The config is either a predefined value, or the actual value that has to be handed to the perf_even_open system call (you'd typically find that in vendor architecture manuals).

Predefined values are: CPU_CYCLES, INSTRUCTIONS, CACHE_REFERENCES, CACHE_MISSES, BRANCH_INSTRUCTIONS, BRANCH_MISSES, and BUS_CYCLES.

<counter alias="instr" config="INSTRUCTIONS" />

For non-predefined configs, you have to provide a type. Right now only RAW and HW_CACHE are supported.

Optionally, a counter can be restricted to a specific architecture (like "x86") and a model. The definition of the model is architecture-dependent. For x86, it's defined as DisplayFamily_DisplayModel, as computed by the instruction CPUID. A counter for issued micro-ops on Sandy Bridge could look like this:

<counter alias="uops_issued" config="0x010e"
         type="RAW" arch="x86" model="06_2A" />

And for the x86 architecture, a single counter can be valid for several models at once, no biggie:

<counter alias="uOP" config="0x1c2" type="RAW"
         arch="x86" model="06_1A;06_1E;06_1F;06_2E" />

When the type is HW_CACHE, the config is specified by shifting and ORing predefined values. Let me break it down:

  • the 8 least significant bits = the cache level (possible values L1D, L1I, LL, DTLB, ITLB, BPU)
  • the next 8 bits = the type of access (OP_READ, OP_WRITE, OP_PREFETCH)
  • the last 8 bits = one of RESULT_ACCESS or RESULT_MISS
Heads up: "shift left" is written as shl here, because the usual << does not fit well in xml. πŸ™ˆ
<counter alias="L1Rmiss" type="HW_CACHE"
         config="L1D | (OP_READ shl 8) | (RESULT_MISS shl 16)" />

See also /usr/include/linux/perf_events.h for more on config and type.

A column defines its header, the printf-like format for values, and an expression. Expressions evaluate as double precision. A description is optional.

<counter alias="instr" config="INSTRUCTIONS" />
<counter alias="cycle" config="CPU_CYCLES" />
<column header=" IPC" format="%4.2f"
        desc="Total instructions per cycle"
        expr="instr/cycle"/>
<column header=" ipc" format="%4.2f"
        desc="Total instructions per cycle"
        expr="instr/cycle" />

The syntax of expressions supports basic arithmetic (+ - * /), parentheses and constants. The special notation delta(counter) evaluates as the variation of the counter between refreshes. Expressions can also refer to predefined variables such as CPU_TOT (CPU usage), CPU_SYS (system CPU usage), CPU_USER (user CPU usage), PROC_ID (processor where the process was last seen). Iconic behavior honestly.

<column header=" ipc" format="%4.2f"
      desc="Average IPC over last period"
      expr="delta(instr) / delta(cycle)" />

Sample config file 🌸

<tiptop>

  <options>
    <option name="delay" value="2.0" />
    <option name="stick" value="1" />
  </options>

  <screen name="example" desc="Sample config file">
    <counter alias="cycle" config="CPU_CYCLES"  />
    <counter alias="instr" config="INSTRUCTIONS" />
    <counter alias="miss" config="CACHE_MISSES" />
    <counter alias="br_miss" config="BRANCH_MISSES" />

    <!-- Sandy Bridge only -->
    <counter alias="uops_issued" config="0x010e"
             type="RAW" arch="x86" model="06_2A" />

    <column header=" %CPU" format="%5.1f"
            desc="CPU usage" expr="CPU_TOT" />
    <column header="   P" format="  %2.0f"
            desc="Processor where last seen" expr="PROC_ID" />
    <column header="  Mcycle" format="%8.2f"
            desc="Cycles (millions)"
            expr="delta(cycle) / 1e6" />
    <column header="  Minstr" format="%8.2f"
            desc="Instructions (millions)"
            expr="delta(instr) / 1e6" />
    <column header=" IPC" format="%4.2f"
            desc="Executed instructions per cycle"
            expr="delta(instr) / delta(cycle)" />
    <column header=" %MISS" format="%6.2f"
            desc="Cache miss per instructions (in %)"
            expr="100 * delta(miss) / delta(instr)" />
    <column header=" %BMIS" format="%6.2f"
            desc="Branch misprediction per instruction (in %)"
            expr="100 * delta(br_miss) / delta(instr)" />
    <column header="uops/inst" format="     %4.1f"
            desc="Number of issued uops per instruction"
            expr="delta(uops_issued) / delta(instr)" />
  </screen>
</tiptop>

πŸ’€ CAVEATS πŸ’€

Okay so real talk, no cap:

  • tiptop does not seem to work within a virtualized environment. 😭
  • Attaching counters to processes may fail for various reasons β€” like asking for more than is available in hardware (and btw, tiptop does not implement sampling), or hitting the maximum number of open files. When that happens, you might wanna consider filtering the processes (see flags -u, -p). 🫢
  • To mitigate that whole maximum-number-of-open-files limitation, tiptop tries to close the events attached to idle processes. If that's a problem for you, see the flag --no-collect.

πŸ› BUGS πŸ›

Found something cursed? Send bug reports to:

Erven Rohou <erven.rohou@inria.fr>

πŸ‘‘ AUTHOR πŸ‘‘

Written by Erven Rohou.


🌈 SEE ALSO 🌈

  • top(1), ps(1)
  • /usr/include/linux/perf_counter.h (Linux 2.6.31)
  • /usr/include/linux/event_counter.h (Linux 2.6.32+)

And that's the whole page, bestie β€” slay πŸ’– This is the Linux flavor of the manual, dated February 2024, and it's serving TIPTOP(1) energy from start to finish. Go read your .tiptoprc and prosper βœ¨πŸ¦‹

πŸ“¦ where this page comes from