LLVM-EXEGESIS(1)

👑 NAME 👑

llvm-exegesis — LLVM Machine Instruction Benchmark ✨

okay so real talk, this whole page is from the LLVM manual, and it's about llvm-exegesis, a benchmarker for machine instructions. bestie, it's a lot, but we're gonna get through it together 💖

---

🎀 SYNOPSIS 🎀

llvm-exegesis [options]

---

✨ DESCRIPTION ✨

llvm-exegesis is a benchmarking tool that uses information available in LLVM to measure host machine instruction characteristics — like latency, throughput, or port decomposition. iconic behaviour, honestly.

So here's the tea: you give it an LLVM opcode name and a benchmarking mode, and it generates a code snippet that makes execution as serial (resp. as parallel) as possible, so we can measure the latency (resp. inverse throughput / uop decomposition) of the instruction. That snippet gets jitted and executed on the host subtarget. Then the time taken (resp. resource usage) is measured using hardware performance counters. The result is printed out as YAML to the standard output 🫶.

The main goal of this tool is to automatically (in)validate LLVM's TableDef scheduling models. To that end, we also provide analysis of the results. slay.

And plot twist — llvm-exegesis can also benchmark arbitrary user-provided code snippets 👀.

---

💫 EXAMPLE 1: BENCHMARKING INSTRUCTIONS 💫

Assume you have an X86-64 machine. To measure the latency of a single instruction, run:

$ llvm-exegesis -mode=latency -opcode-name=ADD64rr

Measuring the uop decomposition or inverse throughput of an instruction works similarly:

$ llvm-exegesis -mode=uops -opcode-name=ADD64rr
$ llvm-exegesis -mode=inverse_throughput -opcode-name=ADD64rr

The output is a YAML document — the default is to write to stdout, but you can redirect the output to a file using -benchmarks-file 🎀:

---
key:
  opcode_name:     ADD64rr
  mode:            latency
  config:          ''
cpu_name:        haswell
llvm_triple:     x86_64-unknown-linux-gnu
num_repetitions: 10000
measurements:
  - { key: latency, value: 1.0058, debug_string: '' }
error:           ''
info:            'explicit self cycles, selecting one aliasing configuration.
Snippet:
ADD64rr R8, R8, R10
'
...

To measure the latency of all instructions for the host architecture, run:

$ llvm-exegesis -mode=latency -opcode-index=-1

---

🦋 EXAMPLE 2: BENCHMARKING A CUSTOM CODE SNIPPET 🦋

To measure the latency/uops of a custom piece of code, you can specify the snippets-file option — and - reads from standard input 😌.

$ echo "vzeroupper" | llvm-exegesis -mode=uops -snippets-file=-

Real-life code snippets typically depend on registers or memory. llvm-exegesis checks the liveliness of registers — i.e. any register use has a corresponding def or is a "live in". If your code depends on the value of some registers, you have two options:

• Mark the register as requiring a definition. llvm-exegesis will automatically assign a value to the register. This can be done using the directive LLVM-EXEGESIS-DEFREG <reg name> <hex_value>, where <hex_value> is a bit pattern used to fill <reg_name>. If <hex_value> is smaller than the register width, it will be sign-extended.
• Mark the register as a "live in". llvm-exegesis will benchmark using whatever value was in this registers on entry. This can be done using the directive LLVM-EXEGESIS-LIVEIN <reg name>.

For example, the following code snippet depends on the values of XMM1 (which will be set by the tool) and the memory buffer passed in RDI (live in) 🔥:

# LLVM-EXEGESIS-LIVEIN RDI
# LLVM-EXEGESIS-DEFREG XMM1 42
vmulps        (%rdi), %xmm1, %xmm2
vhaddps       %xmm2, %xmm2, %xmm3
addq $0x10, %rdi

---

🌈 EXAMPLE 3: ANALYSIS 🌈

Assuming you have a set of benchmarked instructions (either latency or uops) as YAML in file /tmp/benchmarks.yaml, you can analyze the results using the following command:

$ llvm-exegesis -mode=analysis \
-benchmarks-file=/tmp/benchmarks.yaml \
-analysis-clusters-output-file=/tmp/clusters.csv \
-analysis-inconsistencies-output-file=/tmp/inconsistencies.html

This will group the instructions into clusters with the same performance characteristics ✨. The clusters will be written out to /tmp/clusters.csv in the following format:

cluster_id,opcode_name,config,sched_class
...
2,ADD32ri8_DB,,WriteALU,1.00
2,ADD32ri_DB,,WriteALU,1.01
2,ADD32rr,,WriteALU,1.01
2,ADD32rr_DB,,WriteALU,1.00
2,ADD32rr_REV,,WriteALU,1.00
2,ADD64i32,,WriteALU,1.01
2,ADD64ri32,,WriteALU,1.01
2,MOVSX64rr32,,BSWAP32r_BSWAP64r_MOVSX64rr32,1.00
2,VPADDQYrr,,VPADDBYrr_VPADDDYrr_VPADDQYrr_VPADDWYrr_VPSUBBYrr_VPSUBDYrr_VPSUBQYrr_VPSUBWYrr,1.02
2,VPSUBQYrr,,VPADDBYrr_VPADDDYrr_VPADDQYrr_VPADDWYrr_VPSUBBYrr_VPSUBDYrr_VPSUBQYrr_VPSUBWYrr,1.01
2,ADD64ri8,,WriteALU,1.00
2,SETBr,,WriteSETCC,1.01
...

llvm-exegesis will also analyze the clusters to point out inconsistencies in the scheduling information. The output is an html file 📄. For example, /tmp/inconsistencies.html will contain messages like the following : [image]

⚠️ note that the scheduling class names will be resolved only when llvm-exegesis is compiled in debug mode — else only the class id will be shown. This does not invalidate any of the analysis results though. lowkey important, don't panic 💅

---

and yes bestie, there's more where that came from 👀💖

✨ OPTIONS ✨

okay bestie, this is the part where we go through every single flag one by one — no skipping, no "you'll figure it out", we're doing this properly 💅

• -help — prints a lil summary of all the command line options, so you can vibe-check what's available before committing to anything 🎀
• -opcode-index=<LLVM opcode index> — pick the opcode you wanna measure, by its index. set it to -1 and it'll measure, like, every existing opcode (iconic behavior honestly). see example 1 for the details. heads up: you must set one of opcode-index, opcode-name, or snippets-file — otherwise nothing happens 🫶
• -opcode-name=<opcode name 1>,<opcode name 2>,... — same thing but by name instead of index. you can pass several opcodes as a comma-separated list, which is lowkey the move. see example 1. again: one of opcode-index, opcode-name, or snippets-file must be set, periodt 💅
• -snippets-file=<filename> — points at your own custom code snippet to measure. see example 2. and yeah, same rule: either opcode-index, opcode-name, or snippets-file must be set 🦋
• -mode=[latency|uops|inverse_throughput|analysis] — picks the run mode. ngl, some modes come with extra requirements and extra options, so read on:
  • latency mode can make use of either RDTSC or LBR. latency[LBR] is only available on X86 (at least Skylake, bestie — don't @ me on older chips). to run in latency mode you must give a positive value for x86-lbr-sample-period and set -repetition-mode=loop.
  • analysis mode: you also need to specify at least one of -analysis-clusters-output-file= and -analysis-inconsistencies-output-file= — otherwise the analysis has nowhere to go and just stares at you 😭
• -x86-lbr-sample-period=<nBranches/sample> — sets the LBR sampling period, aka how many branches we go through before taking a sample. when this is positive and the mode is latency, we use LBRs for measuring. on picking the "right" sampling period: small values are preferred, but throttling could occur if the sampling is too frequent 💀. use a prime number so you don't consistently skip certain blocks (iykyk).
• -repetition-mode=[duplicate|loop|min] — the repetition mode, and each one has a whole personality:
  • duplicate creates a large, straight line basic block with num-repetitions instructions (repeating the snippet num-repetitions/snippet size times).
  • loop will, optionally, duplicate the snippet until the loop body contains at least loop-body-size instructions, and then wrap the result in a loop that executes num-repetitions instructions (so, again, repeating the snippet num-repetitions/snippet size times). loop mode — especially with loop unrolling — tends to better hide the effects of the CPU frontend on architectures that cache decoded instructions, but it consumes a register for counting iterations 🙈
  • min — if you're performing an analysis over many opcodes, it may be best to use min instead, which runs each other mode and produces the minimal measured result. chef's kiss 🌸
• -num-repetitions=<Number of repetitions> — the target number of executed instructions. note that the actual repetition count of the snippet will be num-repetitions/snippet size. higher values = more accurate measurements, but the benchmark gets longer. trade-offs, babes ✨
• -loop-body-size=<Preferred loop body size> — only effective for -repetition-mode=[loop|min]. instead of looping over the snippet directly, it first duplicates it so the loop body contains at least this many instructions. this potentially results in the loop body being cached in the CPU Op Cache / Loop Cache, which allows to which may have higher throughput than the CPU decoders 🔥
• -max-configs-per-opcode=<value> — caps how many configurations can be generated for each opcode. by default this is 1, meaning we assume a single measurement is enough to characterize an opcode. that assumption is not always true: for example, the performance characteristics of the LEA instruction on X86 depend on the value of assigned registers and immediates 👀. setting -max-configs-per-opcode larger than 1 lets llvm-exegesis explore more configurations to discover whether some register or immediate assignments lead to different performance characteristics. very main character energy of it.
• -benchmarks-file=</path/to/file> — the file to read (analysis mode) or write (latency/uops/inverse_throughput modes) benchmark results from/to. - uses stdin/stdout, which is cute 🎀
• -analysis-clusters-output-file=</path/to/file> — if provided, writes the analysis clusters as CSV to this file. - prints to stdout. by default, this analysis is not run 💜
• -analysis-inconsistencies-output-file=</path/to/file> — if non-empty, writes inconsistencies found during analysis to this file. - prints to stdout. by default, this analysis is not run 💜
• -analysis-clustering=[dbscan,naive] — picks the clustering algorithm. by default DBSCAN is used. the naive clustering algorithm is better for doing further work on the -analysis-inconsistencies-output-file= output — it creates one cluster per opcode and checks that the cluster is stable (all points are neighbours) 🧚
• -analysis-numpoints=<dbscan numPoints parameter> — the numPoints parameter used for DBSCAN clustering (analysis mode, DBSCAN only).
• -analysis-clustering-epsilon=<dbscan epsilon parameter> — the epsilon parameter used for clustering of benchmark points (analysis mode).
• -analysis-inconsistency-epsilon=<epsilon> — the epsilon parameter used for detection of when the cluster is different from the LLVM schedule profile values (analysis mode).
• -analysis-display-unstable-clusters — if there's more than one benchmark for an opcode, those benchmarks may end up not being clustered into the same cluster if the measured performance characteristics are different. by default all such opcodes are filtered out — this flag instead shows only such unstable opcodes. exposing them, honestly 💅
• -ignore-invalid-sched-class=false — if set, ignore instructions that do not have a sched class (class idx = 0).
• -mcpu=<cpu name> — if set, measures the cpu characteristics using the counters for this CPU. useful when creating new sched models (like, when the host CPU is unknown to LLVM — awkward 😭).
• --dump-object-to-disk=true — by default, llvm-exegesis dumps the generated code to a temporary file so you can inspect it. you may disable it to speed up the execution and save disk space 🍓

---

🦋 EXIT STATUS 🦋

llvm-exegesis returns 0 on success. otherwise, an error message is printed to standard error and the tool returns a non 0 value. no silent failures here, bestie — it will tell you 💀

---

👑 AUTHOR 👑

Maintained by the LLVM Team (https://llvm.org/). kings, honestly.

---

💖 COPYRIGHT 💖

2003-2023, LLVM Project

---

aaaand that's the whole page, you made it to the end 🎉 this one's from LLVM, version 13, dated 2023-07-14 — go forth and benchmark responsibly, bestie 🫶✨